Graphing issue with data that only changes once every 24 hours 🔗 link

Hi All,

I've been working on a VS backup solution which I have implemented
using proftpd with mySQL, ftpmail (perl script) and quotas on Debian.
Essentially whenever a client uploads data, ftpmail reads the transfer
log from a fifo, and I've edited the perl script to send status
messages through to Xymon.  The solution is provided as only being
used for backups and the client software sends in its own logs to our
Xymon server.  This is a differential backup and the client software
handles the deletion of aged backup files.

All is working robustly on this front and in Xymon I collect the
following data for graphing:

 usage : 83.09
 allowance : 100.00

These figures are currently sent in MB (I need to write a percentile
calculation), but only once every 24 hours.  This all falls over when
it comes to trying to graph the data in Xymon as it is not a constant
feed of data every 300 seconds like other tests.  From the rrdtool
documentation it seems like the data I am presenting is unsuitable for
graphing in the normal method.  I've tried just about every data type
and I only get a line drawn when data is received (else the graph
drops out on every step without data).

I realise I could run a client test on the VS that runs du on each
chrooted FTP folder every 5 minutes, but this seems like an
unnecessary load when the data changes with the 24 hour frequency and
it scales horribly.  Also throwing out ~100 status messages for ~100
different clients every few minutes to one server (say if I was to
only run the test every 24 hours and repeat the data sent every 5
minutes) may be a little unreliable.  Possibly changing the xymon
client on the VS to a server and running everything locally would be
the way to go; but still load wise this isn't great when compared to
the methodology of only sending data on upload - though functional I
guess.

Is there a way of providing a custom step in rrdtool for a single test
from Xymon?  Has anyone got any other ideas of how I might graph this?
 Am I missing something obvious?

Cheers,
Phil

In <AANLkTi=user-816ccbaf59ec@xymon.invalid> Phil Meech <user-472323a743c7@xymon.invalid> writes:

When the RRD file gets created, one of the parameters is how often 
updates should be expected (it's called the "step" value). Xymon
uses 300 seconds for this value. Another parameter is the "heartbeat"
setting, which is the maximum time allowed between updates - if 
updates do not arrive within "heartbeat" seconds, then RRD won't 
show a graph because it believes the data is too sparse to make sense.
Xymon uses a value of 600 seconds for the heartbeat (allowing us to
miss one update and still keep the graph).

All of these settings are fixed when the RRD file is created. So what 
you need to do is re-create the RRD file with settings appropriate
for your data. Something like this would probably work:

rrdtool create mydata.rrd \
	--step 86400 \
	DS:mbytes:GAUGE:192800:0:U \
	RRA:AVERAGE:0.5:1:730

This create an RRD file where updates are expected once a day
(86400 seconds = 24 hours). There is one set of data in it,
called "mbytes" - this is a GAUGE, and it is minimum 0 and
has no upper limit ("U" = undefined). There is only one RRA archive
which holds 730 data-points, so you have room for two years of
data.

You can use "rrdtool info" on your current rrd file to see what
it looks like, in case you are not sure of some of these settings.

Once the RRD file exists, Xymon shouldn't have any problems 
with performing the updates or building graphs.


Regards,
Henrik