Xymon Mailing List Archive search

Checking disk space on a "backup" system

5 messages in this thread

list Michael Beatty · Wed, 10 Apr 2013 10:27:43 -0400 ·
All of the hosts that I will be monitoring have a backup.  This backup is, for most all practical purposes, idle.  It does nothing but sits and wait for someone to move it into production.  The primary system copies files and database exports over to this backup system.  I had no intention on running a Xymon client on these backup systems (though one would be installed and waiting to be started), the only monitoring is a ping test.

With that said, one exposure that exists on these backup systems is that the disk space in the file system has the potential to fill up.  Is there any functionality built into xymon that would let me check the disk space on this remote system and report it back from the primary system?

My initial thought was to write an extension script that does an 'rsh backup du /home/user/' and parsing the results and test against a threshold, but I would prefer to utilize what is built into xymon before recreating the wheel.

Thanks

-- 
Michael Beatty
list Steve Anderson · Wed, 10 Apr 2013 16:09:22 +0100 ·
You could always grab a copy of xymonclient.sh and xymonclient-linux.sh and adjust the information gathering parts to run everything over ssh (never rsh. Authentication and encryption for ftw)

Just hardwire the xymonclient.sh to send the details as the server you're monitoring.


Steve Anderson
quoted from Michael Beatty

-----Original Message-----
From: xymon-bounces at xymon.com [mailto:xymon-bounces at xymon.com] On Behalf Of Michael Beatty
Sent: 10 April 2013 15:28
To: xymon at xymon.com
Subject: [Xymon] Checking disk space on a "backup" system

All of the hosts that I will be monitoring have a backup.  This backup
is, for most all practical purposes, idle.  It does nothing but sits and
wait for someone to move it into production.  The primary system copies
files and database exports over to this backup system.  I had no
intention on running a Xymon client on these backup systems (though one
would be installed and waiting to be started), the only monitoring is a
ping test.

With that said, one exposure that exists on these backup systems is that
the disk space in the file system has the potential to fill up.  Is
there any functionality built into xymon that would let me check the
disk space on this remote system and report it back from the primary system?

My initial thought was to write an extension script that does an 'rsh
backup du /home/user/' and parsing the results and test against a
threshold, but I would prefer to utilize what is built into xymon before
recreating the wheel.

Thanks

--
Michael Beatty


BiP Solutions Limited is a company registered in Scotland with Company
Number SC086146 and VAT number 383030966 and having its registered
office at Medius, 60 Pacific Quay, Glasgow, G51 1DZ.

In order to improve the quality of the service we offer, calls may be recorded
for quality management and training purposes.

****************************************************************************
This e-mail (and any attachment) is intended only for the attention of
the addressee(s). Its unauthorised use, disclosure, storage or copying
is not permitted. If you are not the intended recipient, please destroy
all copies and inform the sender by return e-mail.
This e-mail (whether you are the sender or the recipient) may be
monitored, recorded and retained by BiP Solutions Ltd.
E-mail monitoring/ blocking software may be used, and e-mail content may
be read at any time.You have a responsibility to ensure laws are not
broken when composing or forwarding e-mails and their contents.
****************************************************************************
list Adam Goryachev · Thu, 11 Apr 2013 01:26:47 +1000 ·
quoted from Michael Beatty
On 11/04/13 00:27, Michael Beatty wrote:
All of the hosts that I will be monitoring have a backup.  This backup
is, for most all practical purposes, idle.  It does nothing but sits
and wait for someone to move it into production.  The primary system
copies files and database exports over to this backup system.  I had
no intention on running a Xymon client on these backup systems (though
one would be installed and waiting to be started), the only monitoring
is a ping test.

With that said, one exposure that exists on these backup systems is
that the disk space in the file system has the potential to fill up. 
Is there any functionality built into xymon that would let me check
the disk space on this remote system and report it back from the
primary system?

My initial thought was to write an extension script that does an 'rsh
backup du /home/user/' and parsing the results and test against a
threshold, but I would prefer to utilize what is built into xymon
before recreating the wheel.
Just mount the backup FS onto the live machine? Use sshfs, NFS, or
whatever else is convenient. Make sure xymon is configured to report
this type of remote mounted filesystems (I think the default is to not
check remote mounts).

Personally, I'd prefer to install an actual xymon client, and monitor
the backup machine fully to ensure that it will be fully operational
when you need it. Nothing worse than having a problem on the live
server, and then find the backup machine died last week but is still
pingable, or whatever. Just consider Murphy's Law, it will fail in the
worst possible way, at the worst possible time. (Or whatever he actually
said).

Oh, going back to your original question...
Adjust the xymonclient to call a local script instead of /usr/bin/df or
whatever. Make that script call the local /usr/bin/df, as well as
collect the data from the backup machine. Prepend "backup" to all device
names and mountpoints on the backup server, and then combine the local
and remote data together. This allows you to differentiate the backup
system within xymon (ie, 75% full for production and 90% full for backup
systems, etc).

Hope one of the above helps....

Regards,
Adam

-- 
Adam Goryachev
Website Managers
www.websitemanagers.com.au
list Jeremy Laidman · Thu, 11 Apr 2013 10:49:53 +1000 ·
quoted from Michael Beatty
On 11 April 2013 00:27, Michael Beatty <user-4aea7c115850@xymon.invalid> wrote:
My initial thought was to write an extension script that does an 'rsh
backup du /home/user/' and parsing the results and test against a
threshold, but I would prefer to utilize what is built into xymon before
recreating the wheel.
There's nothing within Xymon that supports this, without a client
installation.  As others have said, this might not be a bad idea anyway.
 Just have the client run and report its details.  But if you don't want to
do this...

If you have snmp enabled on the server, you can get disk space using
devmon, which is not part of Xymon, but a well-used add-on.

You have identified one risk: that the disk fills up.  But what about the
risk that the backup copy process fails for another reason (account
expires, rsh/ssh daemon stopped, out of inodes, etc).  So you probably want
to build some tests into your copy script and have it report success to
Xymon.  While you're at it, just get the "du" output and report that also.
 If you use a client message, you can report as little or as much info as
you like.  A status message would be required for the backup result:

SERVERNAME=name.of.server
REPORT=`backup_to_server $SERVERNAME`  # calls backup function, which
returns empty report if OK
# report status
if [ "$REPORT" ]; then
  printf "status $SERVERNAME.backup red `date` Backup to $SERVERNAME
failed\n$REPORT\n"
else
  printf "status $SERVERNAME.backup green `date` Backup to $SERVERNAME OK\n"
fi  | xymoncmd sh -c 'xymon $XYMSRV @'
# report client data
CLIENTDATA=`ssh $SERVERNAME 'echo "[uptime]"; uptime; echo "[df]"; df -Pl
/home/usr; echo "[inode]"; df -Pil /home/usr'`
printf "client $SERVERNAME.linux\n$CLIENTDATA\n" | xymoncmd sh -c 'xymon
$XYMSRV @'


J
list Ulric Eriksson · Thu, 11 Apr 2013 16:39:24 +0200 ·
quoted from Michael Beatty
On 04/10/2013 04:27 PM, Michael Beatty wrote:
I had no
intention on running a Xymon client on these backup systems (though one
would be installed and waiting to be started), the only monitoring is a
ping test.
Why not? It must be by far the easiest and most efficient solution, 
especially since the client is already installed.

Ulric