Xymon Mailing List Archive search

RRD Graphing configuration

8 messages in this thread

list Bill Richardson · Mon, 11 Aug 2008 09:53:56 -0400 ·
I have looked high and low for a way to change how custom rrd's are
created in hobbit.
 
I have created a custom script to graph and have 2 problems that I would
like to figure out.
 
1.) I would like to change the step from 300 secs to 60 secs in the
rrd's and run the script ever 60 secs instead of 300 secs. "The default
rrd behavior is set to 300 secs"
2.) I would like to get the true values not average. The problem is I
have data that I want to graph that can range from 0 to 5  "thats 0, 1,
2, 3, 4  or 5" and the graph shows average so my graphs come out with
numbers like 6m "6 milli" for a true value of 2 and so on. The problem
is the data will sit a 0 for an hour or more and then go to 1 or 2 for
just one sample. Average is not the right choice and I cant get the true
figures I 'm looking for.
 
 Seems like the way to do this is in the hobbitserver.cfg. I added the
following to the hobbitserver.cfg
 
NCV_testgraph="test1total:GAUGE,test2total:GAUGE,test3total:GAUGE"
 
What other values can you pass to rrd other than GAUGE? Can I set a step
to say 60 instead or 300 and can I change the default behavior of
AVERAGE to something else that may work for my above example? The rrd
docs are not clear on this.

 
Early on I was thinking that maybe I could fix the AVERAGE problem with
a change to the hobbitgraph.cfg but this is what I found:
The hobbitgraph.cfg does not have any effect on how rrd file is created.
I deleted the testgraph.rrd made the changes to hobbitgraph.cfg and no
go. You cant see the graphs if you use anything but AVERAGE after
changing the hobbitgraph.cfg. It looks like the *.rrd's are created by
default "AVERAGE" and you just edit the hobbitgraph.cfg to view whats
already built. I need to figure out how to change the way the rrd's are
built and its not thru the hobbitgraph.cfg.
 
BTW here is the output from rrdtool dump see attached. 
 
You will see two areas that need to be changed "the step 300 and
AVERAGE"
1:
<!-- Round Robin Database Dump --><rrd> <version> 0003 </version>

<step> 300 </step> <!-- Seconds -->

2:

<!-- Round Robin Archives --> <rra>

<cf> AVERAGE </cf>

 
Thanks for any clues... : )
 
Bill Richardson
list Galen Johnson · Mon, 11 Aug 2008 12:37:39 -0400 ·
Sigurður Guðbrandsson<mailto:user-f6f2e0690a1f@xymon.invalid> wrote a pretty detailed tutorial on what he through setting up a custom graph that I put up on The Shire's tutorial page.

=G=
quoted from Bill Richardson

From: Bill Richardson [mailto:user-47ee0335f310@xymon.invalid]
Sent: Monday, August 11, 2008 9:54 AM
To: user-ae9b8668bcde@xymon.invalid
Subject: [hobbit] RRD Graphing configuration

I have looked high and low for a way to change how custom rrd's are created in hobbit.

I have created a custom script to graph and have 2 problems that I would like to figure out.

1.) I would like to change the step from 300 secs to 60 secs in the rrd's and run the script ever 60 secs instead of 300 secs. "The default rrd behavior is set to 300 secs"
2.) I would like to get the true values not average. The problem is I have data that I want to graph that can range from 0 to 5  "thats 0, 1, 2, 3, 4  or 5" and the graph shows average so my graphs come out with numbers like 6m "6 milli" for a true value of 2 and so on. The problem is the data will sit a 0 for an hour or more and then go to 1 or 2 for just one sample. Average is not the right choice and I cant get the true figures I 'm looking for.

 Seems like the way to do this is in the hobbitserver.cfg. I added the following to the hobbitserver.cfg

NCV_testgraph="test1total:GAUGE,test2total:GAUGE,test3total:GAUGE"

What other values can you pass to rrd other than GAUGE? Can I set a step to say 60 instead or 300 and can I change the default behavior of AVERAGE to something else that may work for my above example? The rrd docs are not clear on this.

Early on I was thinking that maybe I could fix the AVERAGE problem with a change to the hobbitgraph.cfg but this is what I found:
The hobbitgraph.cfg does not have any effect on how rrd file is created. I deleted the testgraph.rrd made the changes to hobbitgraph.cfg and no go. You cant see the graphs if you use anything but AVERAGE after changing the hobbitgraph.cfg. It looks like the *.rrd's are created by default "AVERAGE" and you just edit the hobbitgraph.cfg to view whats already built. I need to figure out how to change the way the rrd's are built and its not thru the hobbitgraph.cfg.

BTW here is the output from rrdtool dump see attached.

You will see two areas that need to be changed "the step 300 and AVERAGE"
1:
<!-- Round Robin Database Dump --><rrd> <version> 0003 </version>

<step> 300 </step> <!-- Seconds -->

2:

<!-- Round Robin Archives --> <rra>

<cf> AVERAGE </cf>

Thanks for any clues... : )

Bill Richardson
list Ralph Mitchell · Mon, 11 Aug 2008 12:00:53 -0500 ·
quoted from Bill Richardson
On Mon, Aug 11, 2008 at 8:53 AM, Bill Richardson <user-47ee0335f310@xymon.invalid>wrote:
 I have looked high and low for a way to change how custom rrd's are
created in hobbit.

I have created a custom script to graph and have 2 problems that I would
like to figure out.

1.) I would like to change the step from 300 secs to 60 secs in the rrd's
and run the script ever 60 secs instead of 300 secs. "The default rrd
behavior is set to 300 secs"
2.) I would like to get the true values not average. The problem is I
have data that I want to graph that can range from 0 to 5  "thats 0, 1, 2,
3, 4  or 5" and the graph shows average so my graphs come out with numbers
like 6m "6 milli" for a true value of 2 and so on. The problem is the
data will sit a 0 for an hour or more and then go to 1 or 2 for just one
sample. Average is not the right choice and I cant get the true figures I 'm
looking for.

 Seems like the way to do this is in the hobbitserver.cfg. I added the
following to the hobbitserver.cfg

NCV_testgraph="test1total:GAUGE,test2total:GAUGE,test3total:GAUGE"

 What other values can you pass to rrd other than GAUGE? Can I set a step
to say 60 instead or 300 and can I change the default behavior of AVERAGE to
something else that may work for my above example? The rrd docs are not
clear on this.

Early on I was thinking that maybe I could fix the AVERAGE problem with a
change to the hobbitgraph.cfg but this is what I found:
The hobbitgraph.cfg does not have any effect on how rrd file is created. I
deleted the testgraph.rrd made the changes to hobbitgraph.cfg and no go.
You cant see the graphs if you use anything but AVERAGE after changing the
hobbitgraph.cfg. It looks like the *.rrd's are created by default "AVERAGE"
and you just edit the hobbitgraph.cfg to view whats already built. I need to
figure out how to change the way the rrd's are built and its not thru
the hobbitgraph.cfg.
If you're comfortable about fooling with the source, there are a couple of
places where you can make changes to achieve what you want:

hobbit-4.2.0/hobbitd/do_rrd.c contains these definitions:

     static char rra1[] = "RRA:AVERAGE:0.5:1:576";
     static char rra2[] = "RRA:AVERAGE:0.5:6:576";
     static char rra3[] = "RRA:AVERAGE:0.5:24:576";
     static char rra4[] = "RRA:AVERAGE:0.5:288:576";

BUT - if you change those, I think *every* new rrd created by Hobbit would
pick up the same changes.

do_rrd.c also includes a bunch of test-specific files, one of which is
hobbitd/rrd/do_ncv.c.  That contains the code that examines NCV_xxx and sets
up DS entries to match.:

         sprintf(dsdef, "DS:%s:%s:600:0:U", dsname, dstype);
or
         sprintf(dsdef, "DS:%s:DERIVE:600:0:U", dsname);

A bit further down it copies in the above rra1 - rra4 definitions.  So, you
could change the hardwired DS and RRA values in do_ncv.c, but ONLY if you
don't have any other NCV that would  need to be different...

Another way to get towards what you want would be to hand-create the rrd.
Hobbit won't overwrite an existing rrd.  I think the rrdcreate would look
something like this:

     rrdtool create testgraph \
        DS:test1total:GAUGE:60:0:U \
        DS:test2total:GAUGE:60:0:U \
        DS:test3total:GAUGE:60:0:U \
        RRA:AVERAGE:0.5:1:576 \
        RRA:AVERAGE:0.5:6:576 \
        RRA:AVERAGE:0.5:24:576 \
        RRA:AVERAGE:0.5:288:576

I haven't tried this myself, but I think Hobbit would be able to do updates
correctly.

Ralph Mitchell
list Darren Cotton · Mon, 11 Aug 2008 19:13:24 +0200 ·
You need to retune the rrd file - using rrdtune (rrdtool tune....)

http://oss.oetiker.ch/rrdtool/doc/rrdtune.en.html

rrdtool tune /var/lib/hobbit/rrd/$host$/testgraph.rrd --heartbeat
test1total:60 --heartbeat test2total:60 --heartbeat test3total:60

Normally I would expect the Average to be OK - maybe your graphs are
spurious due to the heartbeat being incorrect.

Darren Cotton


             "Bill Richardson"                                             
             <wrichardson at llbe                                             
             an.com>                                                    To 
                                       <user-ae9b8668bcde@xymon.invalid>                    
             11/08/2008 18:04                                           cc 
                                                                           
                                                                   Subject 
             Please respond to         [hobbit] RRD Graphing configuration 
              user-ae9b8668bcde@xymon.invalid                                               
quoted from Ralph Mitchell
                                                                           
                                                                           
I have looked high and low for a way to change how custom rrd's are created
in hobbit.

I have created a custom script to graph and have 2 problems that I would
like to figure out.

1.) I would like to change the step from 300 secs to 60 secs in the rrd's
and run the script ever 60 secs instead of 300 secs. "The default rrd
behavior is set to 300 secs"
2.) I would like to get the true values not average. The problem is I have
data that I want to graph that can range from 0 to 5  "thats 0, 1, 2, 3, 4
or 5" and the graph shows average so my graphs come out with numbers like
6m "6 milli" for a true value of 2 and so on. The problem is the data will
sit a 0 for an hour or more and then go to 1 or 2 for just one sample.
Average is not the right choice and I cant get the true figures I 'm
looking for.

 Seems like the way to do this is in the hobbitserver.cfg. I added the
following to the hobbitserver.cfg

NCV_testgraph="test1total:GAUGE,test2total:GAUGE,test3total:GAUGE"

What other values can you pass to rrd other than GAUGE? Can I set a step to
say 60 instead or 300 and can I change the default behavior of AVERAGE to
something else that may work for my above example? The rrd docs are not
clear on this.

Early on I was thinking that maybe I could fix the AVERAGE problem with a
change to the hobbitgraph.cfg but this is what I found:
The hobbitgraph.cfg does not have any effect on how rrd file is created. I
deleted the testgraph.rrd made the changes to hobbitgraph.cfg and no go.
You cant see the graphs if you use anything but AVERAGE after changing the
hobbitgraph.cfg. It looks like the *.rrd's are created by default "AVERAGE"
and you just edit the hobbitgraph.cfg to view whats already built. I need
to figure out how to change the way the rrd's are built and its not thru
the hobbitgraph.cfg.

BTW here is the output from rrdtool dump see attached.

You will see two areas that need to be changed "the step 300 and AVERAGE"
1:
<!-- Round Robin Database Dump --><rrd> <version> 0003 </version>


<step> 300 </step> <!-- Seconds -->


2:


<!-- Round Robin Archives --> <rra>


<cf> AVERAGE </cf>


Thanks for any clues... : )

Bill Richardson


DISCLAIMER: 
This e-mail contains proprietary information some or all of which may be legally privileged. It is for the intended recipient only. If an addressing or transmission error has misdirected this e-mail, please notify the author by replying to this e-mail. If you are not the intended recipient you must not use, disclose, distribute, copy, print, or rely on this e-mail.
list Bill Richardson · Mon, 11 Aug 2008 13:29:49 -0400 ·
I see at http://www.trantor.org/theshire/doku.php/tutorials a link for "Develop your own graphing monitor using NCV" but it looks to be a dead link. Looks like that may have my answer? Anyone have the details?
 
Thanks
 
Bill Richardson
quoted from Darren Cotton
 
Sigurður Guðbrandsson<mailto:user-f6f2e0690a1f@xymon.invalid> wrote a pretty detailed tutorial on what he through setting up a custom graph that I put up on The Shire's tutorial page.

=G=

From: Bill Richardson [mailto:user-47ee0335f310@xymon.invalid]
Sent: Monday, August 11, 2008 9:54 AM
To: user-ae9b8668bcde@xymon.invalid
Subject: [hobbit] RRD Graphing configuration

I have looked high and low for a way to change how custom rrd's are created in hobbit.

I have created a custom script to graph and have 2 problems that I would like to figure out.

1.) I would like to change the step from 300 secs to 60 secs in the rrd's and run the script ever 60 secs instead of 300 secs. "The default rrd behavior is set to 300 secs"
2.) I would like to get the true values not average. The problem is I have data that I want to graph that can range from 0 to 5  "thats 0, 1, 2, 3, 4  or 5" and the graph shows average so my graphs come out with numbers like 6m "6 milli" for a true value of 2 and so on. The problem is the data will sit a 0 for an hour or more and then go to 1 or 2 for just one sample. Average is not the right choice and I cant get the true figures I 'm looking for.

Seems like the way to do this is in the hobbitserver.cfg. I added the following to the hobbitserver.cfg

NCV_testgraph="test1total:GAUGE,test2total:GAUGE,test3total:GAUGE"

What other values can you pass to rrd other than GAUGE? Can I set a step to say 60 instead or 300 and can I change the default behavior of AVERAGE to something else that may work for my above example? The rrd docs are not clear on this.

Early on I was thinking that maybe I could fix the AVERAGE problem with a change to the hobbitgraph.cfg but this is what I found:
The hobbitgraph.cfg does not have any effect on how rrd file is created. I deleted the testgraph.rrd made the changes to hobbitgraph.cfg and no go. You cant see the graphs if you use anything but AVERAGE after changing the hobbitgraph.cfg. It looks like the *.rrd's are created by default "AVERAGE" and you just edit the hobbitgraph.cfg to view whats already built. I need to figure out how to change the way the rrd's are built and its not thru the hobbitgraph.cfg.

BTW here is the output from rrdtool dump see attached.

You will see two areas that need to be changed "the step 300 and AVERAGE"
1:
<!-- Round Robin Database Dump --><rrd> <version> 0003 </version>

<step> 300 </step> <!-- Seconds -->

2:

<!-- Round Robin Archives --> <rra>

<cf> AVERAGE </cf>

Thanks for any clues... : )

Bill Richardson
list Bill Richardson · Mon, 11 Aug 2008 14:57:28 -0400 ·
Thanks for the help.. I think i'll try the rrdcreate and let you know.
quoted from Bill Richardson
 
On Mon, Aug 11, 2008 at 8:53 AM, Bill Richardson <user-47ee0335f310@xymon.invalid>wrote:
 I have looked high and low for a way to change how custom rrd's are
created in hobbit.

I have created a custom script to graph and have 2 problems that I
would
like to figure out.

1.) I would like to change the step from 300 secs to 60 secs in the
rrd's
and run the script ever 60 secs instead of 300 secs. "The default rrd
behavior is set to 300 secs"
2.) I would like to get the true values not average. The problem is I
have data that I want to graph that can range from 0 to 5  "thats 0,
1, 2,
3, 4  or 5" and the graph shows average so my graphs come out with
numbers
like 6m "6 milli" for a true value of 2 and so on. The problem is the
data will sit a 0 for an hour or more and then go to 1 or 2 for just
one
sample. Average is not the right choice and I cant get the true
figures I 'm
looking for.

 Seems like the way to do this is in the hobbitserver.cfg. I added the
following to the hobbitserver.cfg

NCV_testgraph="test1total:GAUGE,test2total:GAUGE,test3total:GAUGE"

 What other values can you pass to rrd other than GAUGE? Can I set a
step
to say 60 instead or 300 and can I change the default behavior of
AVERAGE to
something else that may work for my above example? The rrd docs are
not
clear on this.

Early on I was thinking that maybe I could fix the AVERAGE problem
with a
change to the hobbitgraph.cfg but this is what I found:
The hobbitgraph.cfg does not have any effect on how rrd file is
created. I
deleted the testgraph.rrd made the changes to hobbitgraph.cfg and no
go.
You cant see the graphs if you use anything but AVERAGE after changing
the
hobbitgraph.cfg. It looks like the *.rrd's are created by default
"AVERAGE"
and you just edit the hobbitgraph.cfg to view whats already built. I
need to
figure out how to change the way the rrd's are built and its not thru
the hobbitgraph.cfg.
If you're comfortable about fooling with the source, there are a couple
of
places where you can make changes to achieve what you want:

hobbit-4.2.0/hobbitd/do_rrd.c contains these definitions:

     static char rra1[] = "RRA:AVERAGE:0.5:1:576";
     static char rra2[] = "RRA:AVERAGE:0.5:6:576";
     static char rra3[] = "RRA:AVERAGE:0.5:24:576";
     static char rra4[] = "RRA:AVERAGE:0.5:288:576";

BUT - if you change those, I think *every* new rrd created by Hobbit
would
pick up the same changes.

do_rrd.c also includes a bunch of test-specific files, one of which is
hobbitd/rrd/do_ncv.c.  That contains the code that examines NCV_xxx and
sets
up DS entries to match.:

         sprintf(dsdef, "DS:%s:%s:600:0:U", dsname, dstype);
or
         sprintf(dsdef, "DS:%s:DERIVE:600:0:U", dsname);

A bit further down it copies in the above rra1 - rra4 definitions.  So,
you
could change the hardwired DS and RRA values in do_ncv.c, but ONLY if
you
don't have any other NCV that would  need to be different...

Another way to get towards what you want would be to hand-create the
rrd.
Hobbit won't overwrite an existing rrd.  I think the rrdcreate would
look
something like this:

     rrdtool create testgraph \
        DS:test1total:GAUGE:60:0:U \
        DS:test2total:GAUGE:60:0:U \
        DS:test3total:GAUGE:60:0:U \
        RRA:AVERAGE:0.5:1:576 \
        RRA:AVERAGE:0.5:6:576 \
        RRA:AVERAGE:0.5:24:576 \
        RRA:AVERAGE:0.5:288:576

I haven't tried this myself, but I think Hobbit would be able to do
updates
correctly.

Ralph Mitchell
list Galen Johnson · Mon, 11 Aug 2008 16:29:29 -0400 ·
It's a "FIXME" which means it's one I've asked for that noone's given...Sigur's tutorial is the one below it.
quoted from Bill Richardson

From: Bill Richardson [mailto:user-47ee0335f310@xymon.invalid]
Sent: Monday, August 11, 2008 1:30 PM
To: user-ae9b8668bcde@xymon.invalid
Subject: [hobbit] RE: RRD Graphing configuration

I see at http://www.trantor.org/theshire/doku.php/tutorials a link for "Develop your own graphing monitor using NCV" but it looks to be a dead link. Looks like that may have my answer? Anyone have the details?

Thanks

Bill Richardson

Sigurður Guðbrandsson<mailto:user-f6f2e0690a1f@xymon.invalid> wrote a pretty detailed tutorial on what he through setting up a custom graph that I put up on The Shire's tutorial page.

=G=

From: Bill Richardson [mailto:user-47ee0335f310@xymon.invalid]
Sent: Monday, August 11, 2008 9:54 AM
To: user-ae9b8668bcde@xymon.invalid
Subject: [hobbit] RRD Graphing configuration

I have looked high and low for a way to change how custom rrd's are created in hobbit.

I have created a custom script to graph and have 2 problems that I would like to figure out.

1.) I would like to change the step from 300 secs to 60 secs in the rrd's and run the script ever 60 secs instead of 300 secs. "The default rrd behavior is set to 300 secs"
2.) I would like to get the true values not average. The problem is I have data that I want to graph that can range from 0 to 5  "thats 0, 1, 2, 3, 4  or 5" and the graph shows average so my graphs come out with numbers like 6m "6 milli" for a true value of 2 and so on. The problem is the data will sit a 0 for an hour or more and then go to 1 or 2 for just one sample. Average is not the right choice and I cant get the true figures I 'm looking for.

Seems like the way to do this is in the hobbitserver.cfg. I added the following to the hobbitserver.cfg

NCV_testgraph="test1total:GAUGE,test2total:GAUGE,test3total:GAUGE"

What other values can you pass to rrd other than GAUGE? Can I set a step to say 60 instead or 300 and can I change the default behavior of AVERAGE to something else that may work for my above example? The rrd docs are not clear on this.

Early on I was thinking that maybe I could fix the AVERAGE problem with a change to the hobbitgraph.cfg but this is what I found:
The hobbitgraph.cfg does not have any effect on how rrd file is created. I deleted the testgraph.rrd made the changes to hobbitgraph.cfg and no go. You cant see the graphs if you use anything but AVERAGE after changing the hobbitgraph.cfg. It looks like the *.rrd's are created by default "AVERAGE" and you just edit the hobbitgraph.cfg to view whats already built. I need to figure out how to change the way the rrd's are built and its not thru the hobbitgraph.cfg.

BTW here is the output from rrdtool dump see attached.

You will see two areas that need to be changed "the step 300 and AVERAGE"
1:
<!-- Round Robin Database Dump --><rrd> <version> 0003 </version>

<step> 300 </step> <!-- Seconds -->

2:

<!-- Round Robin Archives --> <rra>

<cf> AVERAGE </cf>

Thanks for any clues... : )

Bill Richardson
list Bill Richardson · Tue, 12 Aug 2008 09:09:10 -0400 ·
I now have the graph step at 60 secs and heartbeat is at 120 secs and
the graph is working. This one problem remaining...
 
I deleted the rrd and recreated it using the following:
 
rrdtool create testgraph.rrd --step 60 /
 DS:test1total:GAUGE:120:0:U /
 DS:test2total:GAUGE:120:0:U / 
 DS:test3total:GAUGE:120:0:U /
 RRA:AVERAGE:0.5:1:576 /
 RRA:AVERAGE:0.5:6:576 /
 RRA:AVERAGE:0.5:24:576 /
 RRA:AVERAGE:0.5:288:576
 
 
The problem I still have is the averaging issue. I would like to get the
quoted from Galen Johnson
true values not average. The problem is I
have data that I want to graph that can range from 0 to 5  "that's 0, 1,
2, 3, 4  or 5" and the graph shows average

so my graphs come out with numbers like 200m while the true value is
really 2 and so on. The problem is the
data will sit a 0 for maybe an hour and then go to 1 or 2 for a sample
or two.
 
Any thoughts on how I could recreate the graphs to show true values?
 
thanks
 
Bill