Graphs gone crazy

10 messages in this thread

list Vernon Everett · Wed, 14 Jul 2010 11:50:57 +0800 ·

Hi all

Hoping somebody has seen this one before.
All of a sudden, all my graphs have gone completely crazy.

The rrd files look good, and contain sane looking data.
The curr/min/max summary looks fine, but the graphs themselves are
either showing a wash of colour, corresponding to the first defined
line, or not showing at all. Just get the link and the magnifying
glass. (See attached.)

There appear to be no errors in the web server logs, and none in any
of the xymon logs either. :-(

Any assistance appreciated.

Regards
     Vernon

list Henrik Størner · Wed, 14 Jul 2010 05:22:23 +0000 (UTC) ·

▸ quoted from Vernon Everett

On Wed, 14 Jul 2010 11:50:57 +0800 Vernon Everett wrote:

The rrd files look good, and contain sane looking data. The curr/min/max
summary looks fine, but the graphs themselves are either showing a wash
of colour, corresponding to the first defined line, or not showing at
all. Just get the link and the magnifying glass. (See attached.)

My guess would be a problem with your hobbitgraphs.cfg file.


Regards
Henrik

list Vernon Everett · Wed, 14 Jul 2010 14:47:34 +0800 ·

Hi Henrik

Wish it was that simple.
I restored a known working copy of my hobbitgraph.cfg and I get the
same results :-(

Regards
      Vernon

▸ quoted from Henrik Størner



On Wed, Jul 14, 2010 at 1:22 PM, Henrik Størner <user-ce4a2c883f75@xymon.invalid> wrote:

On Wed, 14 Jul 2010 11:50:57 +0800 Vernon Everett wrote:

The rrd files look good, and contain sane looking data. The curr/min/max
summary looks fine, but the graphs themselves are either showing a wash
of colour, corresponding to the first defined line, or not showing at
all. Just get the link and the magnifying glass. (See attached.)

My guess would be a problem with your hobbitgraphs.cfg file.


Regards
Henrik

list Josh Luthman · Wed, 14 Jul 2010 10:37:24 -0400 ·

This isn't the issue with Firefox, is it?  Have you used IE/Chrome/Opera to
see the graphs?

Josh Luthman
Office: XXX-XXX-XXXX
Direct: XXX-XXX-XXXX
XXXX Wayne St
Suite XXXX
Troy, OH XXXXX

▸ quoted from Vernon Everett



On Wed, Jul 14, 2010 at 2:47 AM, Vernon Everett <user-b3f8dacb72c8@xymon.invalid>wrote:

Hi Henrik

Wish it was that simple.
I restored a known working copy of my hobbitgraph.cfg and I get the
same results :-(

Regards
      Vernon


On Wed, Jul 14, 2010 at 1:22 PM, Henrik Størner <user-ce4a2c883f75@xymon.invalid> wrote:

On Wed, 14 Jul 2010 11:50:57 +0800 Vernon Everett wrote:

The rrd files look good, and contain sane looking data. The curr/min/max
summary looks fine, but the graphs themselves are either showing a wash
of colour, corresponding to the first defined line, or not showing at
all. Just get the link and the magnifying glass. (See attached.)

My guess would be a problem with your hobbitgraphs.cfg file.


Regards
Henrik

list Tim McCloskey · Wed, 14 Jul 2010 11:29:16 -0700 ·

Hi,

You say ALL of your graphs are crazy, you mean for ALL hosts and ALL tests?  I'm sure you've probably already investigated, but what has recently changed with the O/S patches, env, or changes to hobbit?

You could try to create a manual graph from the rrd file to isolate a bit, short example below.

rdtool graph splat.png \
         -W tm at .com \
        "DEF:NA1CPU=./NA1CPU.rrd:NA1CPU:AVERAGE" \
        "AREA:NA1CPU#00CC00:NA1CPU" \
        "GPRINT:NA1CPU:LAST: \: %5.0lf cur" \
        "GPRINT:NA1CPU:MAX: \: %5.0lf max" \
        "GPRINT:NA1CPU:MIN: \: %5.0lf min" \
        "GPRINT:NA1CPU:AVERAGE: \: %5.0lf avg\n"


Regards,

Tim

▸ quoted from Vernon Everett

From: Vernon Everett [user-b3f8dacb72c8@xymon.invalid]
Sent: Tuesday, July 13, 2010 8:50 PM
To: xymon at xymon.com
Subject: [xymon] Graphs gone crazy

Hi all

Hoping somebody has seen this one before.
All of a sudden, all my graphs have gone completely crazy.

The rrd files look good, and contain sane looking data.
The curr/min/max summary looks fine, but the graphs themselves are
either showing a wash of colour, corresponding to the first defined
line, or not showing at all. Just get the link and the magnifying
glass. (See attached.)

There appear to be no errors in the web server logs, and none in any
of the xymon logs either. :-(

Any assistance appreciated.

Regards
     Vernon

list Vernon Everett · Thu, 15 Jul 2010 10:43:14 +0800 ·

Hi Tim

Thanks for those pointers. I have moved the problem along, in so much
as I realise now what's borked.
Still not sure how to fix it, but I have a theory about why it's broken.

Appears rrd is borked.
To replicate what's happening, I modified your command, and tried to
recreate a cpu graph (using live data)
 rrdtool graph splat.png \
       "DEF:avg=./la.rrd:la:AVERAGE" \
       "CDEF:la=avg,100,/" \
       "AREA:la#00CC00:la" \
       "GPRINT:la:LAST: \: %5.0lf xxx" \
       "GPRINT:la:MAX: \: %5.0lf max" \
       "GPRINT:la:MIN: \: %5.0lf min" \
       "GPRINT:la:AVERAGE: \: %5.0lf avg\n"
And got the attached output.
This is pretty much what I get in the trends column.

So I tried again with one of the graphs that don't show up at all. The
hobbitd incoming messages graph.
And got this error message.
ERROR: LINE width '2:in#00CCCC:Incoming messages' is out of range in
line 'LINE2:in#00CCCC:Incoming messages'

Odd! Tried a bit of reformatting - still no graph.
So I took the one above, which is known to at least produce output,
and replaced AREA with LINE2.
ERROR: LINE width '2:la#00CC00:la' is out of range in line 'LINE2:la#00CC00:la'
Odd that this message isn't showing up in any of the logs though.

So I went back to my graphs, and the penny dropped.
The graphs that appear with the wash of colour are all AREA graphs, or
contain an AREA component.
The missing ones are all LINE or LINE2 graphs.

Sound familiar to anybody?

My theory is that somebody upgraded the libraries. (We are using the
CSW libraries) and the upgrade may have caused an issue.

The investigation continues.
Any ideas clues or pointers from the list appreciated.

Regards
     Vernon

▸ quoted from Tim McCloskey



On Thu, Jul 15, 2010 at 12:48 AM, Tim McCloskey <user-440820cc07d6@xymon.invalid> wrote:

Hi,

You say ALL of your graphs are crazy, you mean for ALL hosts and ALL tests?  I'm sure you've probably already investigated, but what has recently changed with the O/S patches, env, or changes to hobbit?

You could try to create a manual graph from the rrd file to isolate a bit, short example below.

rdtool graph splat.png \
        -W tm at .com \
       "DEF:NA1CPU=./NA1CPU.rrd:NA1CPU:AVERAGE" \
       "AREA:NA1CPU#00CC00:NA1CPU" \
       "GPRINT:NA1CPU:LAST: \: %5.0lf cur" \
       "GPRINT:NA1CPU:MAX: \: %5.0lf max" \
       "GPRINT:NA1CPU:MIN: \: %5.0lf min" \
       "GPRINT:NA1CPU:AVERAGE: \: %5.0lf avg\n"


Regards,

Tim


From: Vernon Everett [user-b3f8dacb72c8@xymon.invalid]
Sent: Tuesday, July 13, 2010 8:50 PM
To: xymon at xymon.com
Subject: [xymon] Graphs gone crazy

Hi all

Hoping somebody has seen this one before.
All of a sudden, all my graphs have gone completely crazy.

The rrd files look good, and contain sane looking data.
The curr/min/max summary looks fine, but the graphs themselves are
either showing a wash of colour, corresponding to the first defined
line, or not showing at all. Just get the link and the magnifying
glass. (See attached.)

There appear to be no errors in the web server logs, and none in any
of the xymon logs either. :-(

Any assistance appreciated.

Regards
    Vernon

list Tim McCloskey · Wed, 14 Jul 2010 20:05:59 -0700 ·

Hi Vernon, 

I seem to recall a 'all-in-one' rrdtool/lib/deps set of install instructions from Tobias but it's been a long, long time.  Anyway, my point is that you might be able to isolate even further by building the latest rrdtool, with all libs/deps, from source, and placing the whole thing in a unique dir, then testing rrdtool from that install.  Or send the .rrd to some other system with a clean rrdtool install and test...

Or, you could take a look here to see if it's relevant to your system.  It's a bug from about three weeks ago, I think.
http://oss.oetiker.ch/rrdtool-trac/ticket/271

▸ quoted from Vernon Everett

Regards, 

Tim

From: Vernon Everett [user-b3f8dacb72c8@xymon.invalid]
Sent: Wednesday, July 14, 2010 7:43 PM
To: xymon at xymon.com
Subject: Re: [xymon] Graphs gone crazy

Hi Tim

Thanks for those pointers. I have moved the problem along, in so much
as I realise now what's borked.
Still not sure how to fix it, but I have a theory about why it's broken.

Appears rrd is borked.
To replicate what's happening, I modified your command, and tried to
recreate a cpu graph (using live data)
 rrdtool graph splat.png \
       "DEF:avg=./la.rrd:la:AVERAGE" \
       "CDEF:la=avg,100,/" \
       "AREA:la#00CC00:la" \
       "GPRINT:la:LAST: \: %5.0lf xxx" \
       "GPRINT:la:MAX: \: %5.0lf max" \
       "GPRINT:la:MIN: \: %5.0lf min" \
       "GPRINT:la:AVERAGE: \: %5.0lf avg\n"
And got the attached output.
This is pretty much what I get in the trends column.

So I tried again with one of the graphs that don't show up at all. The
hobbitd incoming messages graph.
And got this error message.
ERROR: LINE width '2:in#00CCCC:Incoming messages' is out of range in
line 'LINE2:in#00CCCC:Incoming messages'

Odd! Tried a bit of reformatting - still no graph.
So I took the one above, which is known to at least produce output,
and replaced AREA with LINE2.
ERROR: LINE width '2:la#00CC00:la' is out of range in line 'LINE2:la#00CC00:la'
Odd that this message isn't showing up in any of the logs though.

So I went back to my graphs, and the penny dropped.
The graphs that appear with the wash of colour are all AREA graphs, or
contain an AREA component.
The missing ones are all LINE or LINE2 graphs.

Sound familiar to anybody?

My theory is that somebody upgraded the libraries. (We are using the
CSW libraries) and the upgrade may have caused an issue.

The investigation continues.
Any ideas clues or pointers from the list appreciated.

Regards
     Vernon

On Thu, Jul 15, 2010 at 12:48 AM, Tim McCloskey <user-440820cc07d6@xymon.invalid> wrote:

Hi,

You say ALL of your graphs are crazy, you mean for ALL hosts and ALL tests?  I'm sure you've probably already investigated, but what has recently changed with the O/S patches, env, or changes to hobbit?

You could try to create a manual graph from the rrd file to isolate a bit, short example below.

rdtool graph splat.png \
        -W tm at .com \
       "DEF:NA1CPU=./NA1CPU.rrd:NA1CPU:AVERAGE" \
       "AREA:NA1CPU#00CC00:NA1CPU" \
       "GPRINT:NA1CPU:LAST: \: %5.0lf cur" \
       "GPRINT:NA1CPU:MAX: \: %5.0lf max" \
       "GPRINT:NA1CPU:MIN: \: %5.0lf min" \
       "GPRINT:NA1CPU:AVERAGE: \: %5.0lf avg\n"


Regards,

Tim


From: Vernon Everett [user-b3f8dacb72c8@xymon.invalid]
Sent: Tuesday, July 13, 2010 8:50 PM
To: xymon at xymon.com
Subject: [xymon] Graphs gone crazy

Hi all

Hoping somebody has seen this one before.
All of a sudden, all my graphs have gone completely crazy.

The rrd files look good, and contain sane looking data.
The curr/min/max summary looks fine, but the graphs themselves are
either showing a wash of colour, corresponding to the first defined
line, or not showing at all. Just get the link and the magnifying
glass. (See attached.)

There appear to be no errors in the web server logs, and none in any
of the xymon logs either. :-(

Any assistance appreciated.

Regards
    Vernon

list Vernon Everett · Thu, 15 Jul 2010 11:21:26 +0800 ·

Hi Tim

I just did something similar.
I downloaded rrdtool and all required libraries from sunfreeware.com
and installed them into default location.
All good.
I then tried your command again, using the newly installed rrdtool,
and it works.
I get line graphs and correct area graphs.

Now I need to convince Xymon to use the new version.

I changed the path in hobbitserver.cfg and restarted. No good.
I even linked the new rrdtool to the location of the old bad one. No change.

Any idea where in the configs we define which rrdtool binary to use
for graphing?

Cheers

▸ quoted from Tim McCloskey

      Vernon


On Thu, Jul 15, 2010 at 11:05 AM, Tim McCloskey <user-440820cc07d6@xymon.invalid> wrote:

Hi Vernon,

I seem to recall a 'all-in-one' rrdtool/lib/deps set of install instructions from Tobias but it's been a long, long time.  Anyway, my point is that you might be able to isolate even further by building the latest rrdtool, with all libs/deps, from source, and placing the whole thing in a unique dir, then testing rrdtool from that install.  Or send the .rrd to some other system with a clean rrdtool install and test...

Or, you could take a look here to see if it's relevant to your system.  It's a bug from about three weeks ago, I think.
http://oss.oetiker.ch/rrdtool-trac/ticket/271

Regards,

Tim


From: Vernon Everett [user-b3f8dacb72c8@xymon.invalid]
Sent: Wednesday, July 14, 2010 7:43 PM
To: xymon at xymon.com
Subject: Re: [xymon] Graphs gone crazy

Hi Tim

Thanks for those pointers. I have moved the problem along, in so much
as I realise now what's borked.
Still not sure how to fix it, but I have a theory about why it's broken.

Appears rrd is borked.
To replicate what's happening, I modified your command, and tried to
recreate a cpu graph (using live data)
 rrdtool graph splat.png \
      "DEF:avg=./la.rrd:la:AVERAGE" \
      "CDEF:la=avg,100,/" \
      "AREA:la#00CC00:la" \
      "GPRINT:la:LAST: \: %5.0lf xxx" \
      "GPRINT:la:MAX: \: %5.0lf max" \
      "GPRINT:la:MIN: \: %5.0lf min" \
      "GPRINT:la:AVERAGE: \: %5.0lf avg\n"
And got the attached output.
This is pretty much what I get in the trends column.

So I tried again with one of the graphs that don't show up at all. The
hobbitd incoming messages graph.
And got this error message.
ERROR: LINE width '2:in#00CCCC:Incoming messages' is out of range in
line 'LINE2:in#00CCCC:Incoming messages'

Odd! Tried a bit of reformatting - still no graph.
So I took the one above, which is known to at least produce output,
and replaced AREA with LINE2.
ERROR: LINE width '2:la#00CC00:la' is out of range in line 'LINE2:la#00CC00:la'
Odd that this message isn't showing up in any of the logs though.

So I went back to my graphs, and the penny dropped.
The graphs that appear with the wash of colour are all AREA graphs, or
contain an AREA component.
The missing ones are all LINE or LINE2 graphs.

Sound familiar to anybody?

My theory is that somebody upgraded the libraries. (We are using the
CSW libraries) and the upgrade may have caused an issue.

The investigation continues.
Any ideas clues or pointers from the list appreciated.

Regards
    Vernon


On Thu, Jul 15, 2010 at 12:48 AM, Tim McCloskey <user-440820cc07d6@xymon.invalid> wrote:

Hi,

You say ALL of your graphs are crazy, you mean for ALL hosts and ALL tests?  I'm sure you've probably already investigated, but what has recently changed with the O/S patches, env, or changes to hobbit?

You could try to create a manual graph from the rrd file to isolate a bit, short example below.

rdtool graph splat.png \
        -W tm at .com \
       "DEF:NA1CPU=./NA1CPU.rrd:NA1CPU:AVERAGE" \
       "AREA:NA1CPU#00CC00:NA1CPU" \
       "GPRINT:NA1CPU:LAST: \: %5.0lf cur" \
       "GPRINT:NA1CPU:MAX: \: %5.0lf max" \
       "GPRINT:NA1CPU:MIN: \: %5.0lf min" \
       "GPRINT:NA1CPU:AVERAGE: \: %5.0lf avg\n"


Regards,

Tim


From: Vernon Everett [user-b3f8dacb72c8@xymon.invalid]
Sent: Tuesday, July 13, 2010 8:50 PM
To: xymon at xymon.com
Subject: [xymon] Graphs gone crazy

Hi all

Hoping somebody has seen this one before.
All of a sudden, all my graphs have gone completely crazy.

The rrd files look good, and contain sane looking data.
The curr/min/max summary looks fine, but the graphs themselves are
either showing a wash of colour, corresponding to the first defined
line, or not showing at all. Just get the link and the magnifying
glass. (See attached.)

There appear to be no errors in the web server logs, and none in any
of the xymon logs either. :-(

Any assistance appreciated.

Regards
    Vernon

list Tim McCloskey · Wed, 14 Jul 2010 20:45:30 -0700 ·

Hi Vernon, 

I don't use csw but in the thread from https://www.opencsw.org/mantis/view.php?id=4380 it is mentioned that this if fixed
and the new csw packages are at: http://mirror.opencsw.org/experimental.html#rrdtool.  

As for where you define rrdtool in hobbit, I think it is determined at build time, not sure.  I'm sure the group can provide better details on this aspect.


$ strings hobbitgraph.cgi  | grep -i rrdtool
/snip/rrdtool/lib:/snip

▸ quoted from Vernon Everett

Regards, 

Tim

From: Vernon Everett [user-b3f8dacb72c8@xymon.invalid]
Sent: Wednesday, July 14, 2010 8:21 PM
To: xymon at xymon.com
Subject: Re: [xymon] Graphs gone crazy

Hi Tim

I just did something similar.
I downloaded rrdtool and all required libraries from sunfreeware.com
and installed them into default location.
All good.
I then tried your command again, using the newly installed rrdtool,
and it works.
I get line graphs and correct area graphs.

Now I need to convince Xymon to use the new version.

I changed the path in hobbitserver.cfg and restarted. No good.
I even linked the new rrdtool to the location of the old bad one. No change.

Any idea where in the configs we define which rrdtool binary to use
for graphing?

Cheers
      Vernon

On Thu, Jul 15, 2010 at 11:05 AM, Tim McCloskey <user-440820cc07d6@xymon.invalid> wrote:

Hi Vernon,

I seem to recall a 'all-in-one' rrdtool/lib/deps set of install instructions from Tobias but it's been a long, long time.  Anyway, my point is that you might be able to isolate even further by building the latest rrdtool, with all libs/deps, from source, and placing the whole thing in a unique dir, then testing rrdtool from that install.  Or send the .rrd to some other system with a clean rrdtool install and test...

Or, you could take a look here to see if it's relevant to your system.  It's a bug from about three weeks ago, I think.
http://oss.oetiker.ch/rrdtool-trac/ticket/271

Regards,

Tim


From: Vernon Everett [user-b3f8dacb72c8@xymon.invalid]
Sent: Wednesday, July 14, 2010 7:43 PM
To: xymon at xymon.com
Subject: Re: [xymon] Graphs gone crazy

Hi Tim

Thanks for those pointers. I have moved the problem along, in so much
as I realise now what's borked.
Still not sure how to fix it, but I have a theory about why it's broken.

Appears rrd is borked.
To replicate what's happening, I modified your command, and tried to
recreate a cpu graph (using live data)
 rrdtool graph splat.png \
      "DEF:avg=./la.rrd:la:AVERAGE" \
      "CDEF:la=avg,100,/" \
      "AREA:la#00CC00:la" \
      "GPRINT:la:LAST: \: %5.0lf xxx" \
      "GPRINT:la:MAX: \: %5.0lf max" \
      "GPRINT:la:MIN: \: %5.0lf min" \
      "GPRINT:la:AVERAGE: \: %5.0lf avg\n"
And got the attached output.
This is pretty much what I get in the trends column.

So I tried again with one of the graphs that don't show up at all. The
hobbitd incoming messages graph.
And got this error message.
ERROR: LINE width '2:in#00CCCC:Incoming messages' is out of range in
line 'LINE2:in#00CCCC:Incoming messages'

Odd! Tried a bit of reformatting - still no graph.
So I took the one above, which is known to at least produce output,
and replaced AREA with LINE2.
ERROR: LINE width '2:la#00CC00:la' is out of range in line 'LINE2:la#00CC00:la'
Odd that this message isn't showing up in any of the logs though.

So I went back to my graphs, and the penny dropped.
The graphs that appear with the wash of colour are all AREA graphs, or
contain an AREA component.
The missing ones are all LINE or LINE2 graphs.

Sound familiar to anybody?

My theory is that somebody upgraded the libraries. (We are using the
CSW libraries) and the upgrade may have caused an issue.

The investigation continues.
Any ideas clues or pointers from the list appreciated.

Regards
    Vernon


On Thu, Jul 15, 2010 at 12:48 AM, Tim McCloskey <user-440820cc07d6@xymon.invalid> wrote:

Hi,

You say ALL of your graphs are crazy, you mean for ALL hosts and ALL tests?  I'm sure you've probably already investigated, but what has recently changed with the O/S patches, env, or changes to hobbit?

You could try to create a manual graph from the rrd file to isolate a bit, short example below.

rdtool graph splat.png \
        -W tm at .com \
       "DEF:NA1CPU=./NA1CPU.rrd:NA1CPU:AVERAGE" \
       "AREA:NA1CPU#00CC00:NA1CPU" \
       "GPRINT:NA1CPU:LAST: \: %5.0lf cur" \
       "GPRINT:NA1CPU:MAX: \: %5.0lf max" \
       "GPRINT:NA1CPU:MIN: \: %5.0lf min" \
       "GPRINT:NA1CPU:AVERAGE: \: %5.0lf avg\n"


Regards,

Tim


From: Vernon Everett [user-b3f8dacb72c8@xymon.invalid]
Sent: Tuesday, July 13, 2010 8:50 PM
To: xymon at xymon.com
Subject: [xymon] Graphs gone crazy

Hi all

Hoping somebody has seen this one before.
All of a sudden, all my graphs have gone completely crazy.

The rrd files look good, and contain sane looking data.
The curr/min/max summary looks fine, but the graphs themselves are
either showing a wash of colour, corresponding to the first defined
line, or not showing at all. Just get the link and the magnifying
glass. (See attached.)

There appear to be no errors in the web server logs, and none in any
of the xymon logs either. :-(

Any assistance appreciated.

Regards
    Vernon

list Vernon Everett · Thu, 15 Jul 2010 12:05:53 +0800 ·

Looks useful.
Researching it now.

▸ quoted from Tim McCloskey



On Thu, Jul 15, 2010 at 11:45 AM, Tim McCloskey <user-440820cc07d6@xymon.invalid> wrote:

Hi Vernon,

I don't use csw but in the thread from https://www.opencsw.org/mantis/view.php?id=4380 it is mentioned that this if fixed
and the new csw packages are at: http://mirror.opencsw.org/experimental.html#rrdtool.

As for where you define rrdtool in hobbit, I think it is determined at build time, not sure.  I'm sure the group can provide better details on this aspect.


$ strings hobbitgraph.cgi  | grep -i rrdtool
/snip/rrdtool/lib:/snip


Regards,

Tim

From: Vernon Everett [user-b3f8dacb72c8@xymon.invalid]
Sent: Wednesday, July 14, 2010 8:21 PM
To: xymon at xymon.com
Subject: Re: [xymon] Graphs gone crazy

Hi Tim

I just did something similar.
I downloaded rrdtool and all required libraries from sunfreeware.com
and installed them into default location.
All good.
I then tried your command again, using the newly installed rrdtool,
and it works.
I get line graphs and correct area graphs.

Now I need to convince Xymon to use the new version.

I changed the path in hobbitserver.cfg and restarted. No good.
I even linked the new rrdtool to the location of the old bad one. No change.

Any idea where in the configs we define which rrdtool binary to use
for graphing?

Cheers
     Vernon


On Thu, Jul 15, 2010 at 11:05 AM, Tim McCloskey <user-440820cc07d6@xymon.invalid> wrote:

Hi Vernon,

I seem to recall a 'all-in-one' rrdtool/lib/deps set of install instructions from Tobias but it's been a long, long time.  Anyway, my point is that you might be able to isolate even further by building the latest rrdtool, with all libs/deps, from source, and placing the whole thing in a unique dir, then testing rrdtool from that install.  Or send the .rrd to some other system with a clean rrdtool install and test...

Or, you could take a look here to see if it's relevant to your system.  It's a bug from about three weeks ago, I think.
http://oss.oetiker.ch/rrdtool-trac/ticket/271

Regards,

Tim


From: Vernon Everett [user-b3f8dacb72c8@xymon.invalid]
Sent: Wednesday, July 14, 2010 7:43 PM
To: xymon at xymon.com
Subject: Re: [xymon] Graphs gone crazy

Hi Tim

Thanks for those pointers. I have moved the problem along, in so much
as I realise now what's borked.
Still not sure how to fix it, but I have a theory about why it's broken.

Appears rrd is borked.
To replicate what's happening, I modified your command, and tried to
recreate a cpu graph (using live data)
 rrdtool graph splat.png \
      "DEF:avg=./la.rrd:la:AVERAGE" \
      "CDEF:la=avg,100,/" \
      "AREA:la#00CC00:la" \
      "GPRINT:la:LAST: \: %5.0lf xxx" \
      "GPRINT:la:MAX: \: %5.0lf max" \
      "GPRINT:la:MIN: \: %5.0lf min" \
      "GPRINT:la:AVERAGE: \: %5.0lf avg\n"
And got the attached output.
This is pretty much what I get in the trends column.

So I tried again with one of the graphs that don't show up at all. The
hobbitd incoming messages graph.
And got this error message.
ERROR: LINE width '2:in#00CCCC:Incoming messages' is out of range in
line 'LINE2:in#00CCCC:Incoming messages'

Odd! Tried a bit of reformatting - still no graph.
So I took the one above, which is known to at least produce output,
and replaced AREA with LINE2.
ERROR: LINE width '2:la#00CC00:la' is out of range in line 'LINE2:la#00CC00:la'
Odd that this message isn't showing up in any of the logs though.

So I went back to my graphs, and the penny dropped.
The graphs that appear with the wash of colour are all AREA graphs, or
contain an AREA component.
The missing ones are all LINE or LINE2 graphs.

Sound familiar to anybody?

My theory is that somebody upgraded the libraries. (We are using the
CSW libraries) and the upgrade may have caused an issue.

The investigation continues.
Any ideas clues or pointers from the list appreciated.

Regards
    Vernon


On Thu, Jul 15, 2010 at 12:48 AM, Tim McCloskey <user-440820cc07d6@xymon.invalid> wrote:

Hi,

You say ALL of your graphs are crazy, you mean for ALL hosts and ALL tests?  I'm sure you've probably already investigated, but what has recently changed with the O/S patches, env, or changes to hobbit?

You could try to create a manual graph from the rrd file to isolate a bit, short example below.

rdtool graph splat.png \
        -W tm at .com \
       "DEF:NA1CPU=./NA1CPU.rrd:NA1CPU:AVERAGE" \
       "AREA:NA1CPU#00CC00:NA1CPU" \
       "GPRINT:NA1CPU:LAST: \: %5.0lf cur" \
       "GPRINT:NA1CPU:MAX: \: %5.0lf max" \
       "GPRINT:NA1CPU:MIN: \: %5.0lf min" \
       "GPRINT:NA1CPU:AVERAGE: \: %5.0lf avg\n"


Regards,

Tim


From: Vernon Everett [user-b3f8dacb72c8@xymon.invalid]
Sent: Tuesday, July 13, 2010 8:50 PM
To: xymon at xymon.com
Subject: [xymon] Graphs gone crazy

Hi all

Hoping somebody has seen this one before.
All of a sudden, all my graphs have gone completely crazy.

The rrd files look good, and contain sane looking data.
The curr/min/max summary looks fine, but the graphs themselves are
either showing a wash of colour, corresponding to the first defined
line, or not showing at all. Just get the link and the magnifying
glass. (See attached.)

There appear to be no errors in the web server logs, and none in any
of the xymon logs either. :-(

Any assistance appreciated.

Regards
    Vernon

Graphs gone crazy 🔗 link

Graphs gone crazy