Graphs not working on individual views, but working on trends view
list Mark Stoltzfus
Hi, we just did a forklift upgrade from our old Hobbit servers to two brand new boxes running Xymon 4.2.3, and for the most part, it's been a seemless transition. We noticed some strange behavior, though, with the graphs. The graphs under disk, cpu, and memory don't show up, but under trends, they show up just fine. I took a look at the different shortcuts, and they look very similar to me, with the differences highlighted in red: Link copied from "disk" section: https://monitor.us.corp.pmigroup.com/xymon-cgi/hobbitgraph.sh?host=wal-d dbs-01.us.corp.pmigroup.com&service=disk&graph_width=576&graph_heigh t=120&first=1&count=3&disp=wal%2dddbs%2d01%2eus%2ecorp%2epmigroup%2ecom& graph=hourly&action=selzoom Link copied from "trends" section: https://monitor.us.corp.pmigroup.com/xymon-cgi/hobbitgraph.sh?host=wal-d dbs-01.us.corp.pmigroup.com&service=disk&graph_width=576&graph_heigh t=120&first=1&count=3&disp=wal%2dddbs%2d01%2eus%2ecorp%2epmigroup%2ecom& nostale&color=green&graph=hourly&action=selzoom Any ideas as to what's going on? I did find some old posts from Henrik about troubleshooting graph generation, but the case he was dealing with there was where the graphs wouldn't generate at all. In this case, the rrds are all there and the graphs are being generated, just not in all views. Thoughts? Mark
list Patrick Nixon
If it helps any, my graphs are being generated with the same links as yours (nostale vs no nostales) without issue. There's probably some other underlying issue. On Wed, Mar 10, 2010 at 2:31 PM, Mark Stoltzfus
▸
<user-5b4ba45c8394@xymon.invalid> wrote:Hi, we just did a forklift upgrade from our old Hobbit servers to two brand new boxes running Xymon 4.2.3, and for the most part, it’s been a seemless transition. We noticed some strange behavior, though, with the graphs. The graphs under disk, cpu, and memory don’t show up, but under trends, they show up just fine. I took a look at the different shortcuts, and they look very similar to me, with the differences highlighted in red: Link copied from “disk” section:
https://monitor.us.corp.pmigroup.com/xymon-cgi/hobbitgraph.sh?host=wal-ddbs-01.us.corp.pmigroup.com&service=disk&graph_width=576&graph_heigh t=120&first=1&count=3&disp=wal%2dddbs%2d01%2eus%2ecorp%2epmigroup%2ecom&graph=hourly&action=selzoom Link copied from “trends” section: https://monitor.us.corp.pmigroup.com/xymon-cgi/hobbitgraph.sh?host=wal-ddbs-01.us.corp.pmigroup.com&service=disk&graph_width=576&graph_heigh t=120&first=1&count=3&disp=wal%2dddbs%2d01%2eus%2ecorp%2epmigroup%2ecom&nostale&color=green&graph=hourly&action=selzoom
▸
Any ideas as to what’s going on? I did find some old posts from Henrik
about troubleshooting graph generation, but the case he was dealing with
there was where the graphs wouldn’t generate at all. In this case, the rrds
are all there and the graphs are being generated, just not in all views.
Thoughts?
Marklist Mark Stoltzfus
I've done some more digging, and come up with another puzzling piece of information. Hobbitgraph.sh is returning images, but they are very small. Here are the apache logs from the two distinct page views: Disk view (note the size of the file returned, 145 bytes): 10.3.144.37 - - [10/Mar/2010:13:26:52 -0800] "GET /xymon-cgi/hobbitgraph.sh?host=wal-ddbs-01.us.corp.pmigroup.com&service=disk&graph_width=576&graph_height=120&first=1&count=3&disp=wal%2dddbs%2d01%2eus%2ecorp%2epmigroup%2ecom&nostale&color=green&graph=hourly&action=view HTTP/1.1" 200 145 Trends view (a more reasonable file size, 20297 bytes): 10.3.144.37 - - [10/Mar/2010:13:28:26 -0800] "GET /xymon-cgi/hobbitgraph.sh?host=wal-ddbs-01.us.corp.pmigroup.com&service=disk&graph_width=576&graph_height=120&first=1&count=3&disp=wal%2dddbs%2d01%2eus%2ecorp%2epmigroup%2ecom&graph=hourly&action=view HTTP/1.1" 200 20297 Anyone have any ideas? Mark
▸
-----Original Message-----
From: Patrick Nixon [mailto:user-1f2406f832af@xymon.invalid]
Sent: Wednesday, March 10, 2010 12:47 PM
To: user-ae9b8668bcde@xymon.invalid
Subject: Re: [hobbit] Graphs not working on individual views, but working on trends view
If it helps any, my graphs are being generated with the same links as
yours (nostale vs no nostales) without issue.
There's probably some other underlying issue.
On Wed, Mar 10, 2010 at 2:31 PM, Mark Stoltzfus
<user-5b4ba45c8394@xymon.invalid> wrote:Hi, we just did a forklift upgrade from our old Hobbit servers to two brand new boxes running Xymon 4.2.3, and for the most part, it's been a seemless transition. We noticed some strange behavior, though, with the graphs. The graphs under disk, cpu, and memory don't show up, but under trends, they show up just fine. I took a look at the different shortcuts, and they look very similar to me, with the differences highlighted in red: Link copied from "disk" section: https://monitor.us.corp.pmigroup.com/xymon-cgi/hobbitgraph.sh?host=wal-ddbs-01.us.corp.pmigroup.com&service=disk&graph_width=576&graph_heigh t=120&first=1&count=3&disp=wal%2dddbs%2d01%2eus%2ecorp%2epmigroup%2ecom&graph=hourly&action=selzoom Link copied from "trends" section: https://monitor.us.corp.pmigroup.com/xymon-cgi/hobbitgraph.sh?host=wal-ddbs-01.us.corp.pmigroup.com&service=disk&graph_width=576&graph_heigh t=120&first=1&count=3&disp=wal%2dddbs%2d01%2eus%2ecorp%2epmigroup%2ecom&nostale&color=green&graph=hourly&action=selzoom Any ideas as to what's going on? I did find some old posts from Henrik about troubleshooting graph generation, but the case he was dealing with there was where the graphs wouldn't generate at all. In this case, the rrds are all there and the graphs are being generated, just not in all views. Thoughts? Mark
list dOCtoR MADneSs
On Wed, 10 Mar 2010 11:31:31 -0800, "Mark Stoltzfus"
▸
<user-5b4ba45c8394@xymon.invalid> wrote:Hi, we just did a forklift upgrade from our old Hobbit servers to two brand new boxes running Xymon 4.2.3, and for the most part, it's been a seemless transition. We noticed some strange behavior, though, with the graphs. The graphs under disk, cpu, and memory don't show up, but under trends, they show up just fine. I took a look at the different shortcuts, and they look very similar to me, with the differences highlighted in red: Link copied from "disk" section: https://monitor.us.corp.pmigroup.com/xymon-cgi/hobbitgraph.sh?host=wal-d dbs-01.us.corp.pmigroup.com&service=disk&graph_width=576&graph_heigh t=120&first=1&count=3&disp=wal%2dddbs%2d01%2eus%2ecorp%2epmigroup%2ecom& graph=hourly&action=selzoom Link copied from "trends" section: https://monitor.us.corp.pmigroup.com/xymon-cgi/hobbitgraph.sh?host=wal-d dbs-01.us.corp.pmigroup.com&service=disk&graph_width=576&graph_heigh t=120&first=1&count=3&disp=wal%2dddbs%2d01%2eus%2ecorp%2epmigroup%2ecom& nostale&color=green&graph=hourly&action=selzoom Any ideas as to what's going on? I did find some old posts from Henrik about troubleshooting graph generation, but the case he was dealing with there was where the graphs wouldn't generate at all. In this case, the rrds are all there and the graphs are being generated, just not in all views. Thoughts? Mark
Hi, I got different issue with graphs, but I think my problem should be related to your comment :
▸
" I did find some old posts from Henrik about troubleshooting graph
generation, but the case he was dealing with there was where the graphs
wouldn't generate at all."
Could you give me informations about those posts (date, subject, or any
criteria that could help me to find them) ?
Regards,
Damien
list David Baldwin
▸
user-d54077869176@xymon.invalid wrote:
On Wed, 10 Mar 2010 11:31:31 -0800, "Mark Stoltzfus" <user-5b4ba45c8394@xymon.invalid> wrote:Hi, we just did a forklift upgrade from our old Hobbit servers to two brand new boxes running Xymon 4.2.3, and for the most part, it's been a seemless transition. We noticed some strange behavior, though, with the graphs. The graphs under disk, cpu, and memory don't show up, but under trends, they show up just fine. I took a look at the different shortcuts, and they look very similar to me, with the differences highlighted in red: Link copied from "disk" section: https://monitor.us.corp.pmigroup.com/xymon-cgi/hobbitgraph.sh?host=wal-d dbs-01.us.corp.pmigroup.com&service=disk&graph_width=576&graph_heigh t=120&first=1&count=3&disp=wal%2dddbs%2d01%2eus%2ecorp%2epmigroup%2ecom& graph=hourly&action=selzoom Link copied from "trends" section: https://monitor.us.corp.pmigroup.com/xymon-cgi/hobbitgraph.sh?host=wal-d dbs-01.us.corp.pmigroup.com&service=disk&graph_width=576&graph_heigh t=120&first=1&count=3&disp=wal%2dddbs%2d01%2eus%2ecorp%2epmigroup%2ecom& nostale&color=green&graph=hourly&action=selzoom Any ideas as to what's going on? I did find some old posts from Henrik about troubleshooting graph generation, but the case he was dealing with there was where the graphs wouldn't generate at all. In this case, the rrds are all there and the graphs are being generated, just not in all views. Thoughts? MarkHi, I got different issue with graphs, but I think my problem should be related to your comment : " I did find some old posts from Henrik about troubleshooting graph generation, but the case he was dealing with there was where the graphs wouldn't generate at all."
It's worth checking hobbitgraph.cfg if you have added a number of your
own custom graphs.
The FNPATTERN operator is doing a regexp match, but in some cases the
default pattern definition is very loose.
For example, I'd added a bunch of devmon based system health check
reporting stuff which generated file names like
/var/lib/hobbit/rrd/HOSTNAME/temp.memory11.rrd
These were causing the memory graph to die because it was attempting to
query non-conformant RRDs for the particular graph. Solved by tightening
the regexp definition as below (adding ^ in front, which means the
filename has to start with memory, not just have it as a substring).
[memory]
FNPATTERN ^memory.(.+).rrd
TITLE Memory Utilization
Note that the FNPATTERN is not used when deciding if there are any RRD
files to use to display a graph at all on the test status page. It just
seems to glob TEST*.rrd to count the number of files to decide how many
lines to display on the graph for TEST. This is with 4.2.3.
David.
--
David Baldwin - IT Unit
Australian Sports Commission www.ausport.gov.au
Tel 02 62147830 Fax 02 62141830 PO Box 176 Belconnen ACT 2616
user-cbbf693f2c89@xymon.invalid Leverrier Street Bruce ACT 2617
Keep up to date with what's happening in Australian sport visit http://www.ausport.gov.au
This message is intended for the addressee named and may contain confidential and privileged information. If you are not the intended recipient please note that any form of distribution, copying or use of this communication or the information in it is strictly prohibited and may be unlawful. If you receive this message in error, please delete it and notify the sender.
list dOCtoR MADneSs
On Thu, 11 Mar 2010 19:46:57 +1100, David Baldwin
▸
<user-cbbf693f2c89@xymon.invalid> wrote:user-d54077869176@xymon.invalid wrote:On Wed, 10 Mar 2010 11:31:31 -0800, "Mark Stoltzfus" <user-5b4ba45c8394@xymon.invalid> wrote:Hi, we just did a forklift upgrade from our old Hobbit servers to two brand new boxes running Xymon 4.2.3, and for the most part, it's been a seemless transition. We noticed some strange behavior, though, with the graphs. The graphs under disk, cpu, and memory don't show up, but under trends, they show up just fine. I took a look at the different shortcuts, and they look very similar to me, with the differences highlighted in red: Link copied from "disk" section:
https://monitor.us.corp.pmigroup.com/xymon-cgi/hobbitgraph.sh?host=wal-d
dbs-01.us.corp.pmigroup.com&service=disk&graph_width=576&graph_heigh
t=120&first=1&count=3&disp=wal%2dddbs%2d01%2eus%2ecorp%2epmigroup%2ecom&
graph=hourly&action=selzoom Link copied from "trends" section:
https://monitor.us.corp.pmigroup.com/xymon-cgi/hobbitgraph.sh?host=wal-d
dbs-01.us.corp.pmigroup.com&service=disk&graph_width=576&graph_heigh
t=120&first=1&count=3&disp=wal%2dddbs%2d01%2eus%2ecorp%2epmigroup%2ecom&
nostale&color=green&graph=hourly&action=selzoom Any ideas as to what's going on? I did find some old posts from
Henrik
about troubleshooting graph generation, but the case he was dealing with there was where the graphs wouldn't generate at all. In this case, the rrds are all there and the graphs are being generated, just not in all views. Thoughts? MarkHi, I got different issue with graphs, but I think my problem should be related to your comment : " I did find some old posts from Henrik about troubleshooting graph generation, but the case he was dealing with there was where the graphs wouldn't generate at all."It's worth checking hobbitgraph.cfg if you have added a number of your own custom graphs. The FNPATTERN operator is doing a regexp match, but in some cases the default pattern definition is very loose. For example, I'd added a bunch of devmon based system health check reporting stuff which generated file names like /var/lib/hobbit/rrd/HOSTNAME/temp.memory11.rrd These were causing the memory graph to die because it was attempting to query non-conformant RRDs for the particular graph. Solved by tightening the regexp definition as below (adding ^ in front, which means the filename has to start with memory, not just have it as a substring). [memory] FNPATTERN ^memory.(.+).rrd TITLE Memory Utilization Note that the FNPATTERN is not used when deciding if there are any RRD files to use to display a graph at all on the test status page. It just seems to glob TEST*.rrd to count the number of files to decide how many lines to display on the graph for TEST. This is with 4.2.3. David.
Thank you for your advices. I migrated my xymon server (and some clients) to 4.3.0-0.beta2, without migrating (is it needed anyway ?) my rrd files. My issue is very simple (and strange), I've 7 partitions monitored by xymon, all are reported correctly. I've 2 graphs for 3 partitions, but there should be a 3rd one for the 7th partition. Instead, I've a broken link. I've the same issue on custom tests/graphs. Many datas are graphed, but some graphs are missing. I think there should have a way to deal with it, in modifying how many values are on the same graph, or in correcting the root cause. Actually I don't have any skill in C nor php, so I can't solve it myself. Do you think this behaviour is related to your advice (changing FNPATTERN) ? Regards, Damien
list Mark Stoltzfus
If you go through the whole thread, there's a lot of good information regarding troubleshooting graph generation in there: http://www.hswn.dk/hobbiton/2005/12/msg00201.html Hope that helps, Mark
▸
-----Original Message-----
From: user-d54077869176@xymon.invalid [mailto:user-d54077869176@xymon.invalid]
Sent: Thursday, March 11, 2010 12:58 AM
To: user-ae9b8668bcde@xymon.invalid
Subject: Re: [hobbit] Graphs not working on individual views, but
working on trends view
On Thu, 11 Mar 2010 19:46:57 +1100, David Baldwin
<user-cbbf693f2c89@xymon.invalid> wrote:user-d54077869176@xymon.invalid wrote:On Wed, 10 Mar 2010 11:31:31 -0800, "Mark Stoltzfus" <user-5b4ba45c8394@xymon.invalid> wrote:Hi, we just did a forklift upgrade from our old Hobbit servers to two brand new boxes running Xymon 4.2.3, and for the most part, it's been a seemless transition. We noticed some strange behavior, though, with the graphs. The graphs under disk, cpu, and memory don't show up, but under trends, they show up just fine. I took a look at the different shortcuts, and they look very similar to me, with the differences highlighted in red: Link copied from "disk" section:
https://monitor.us.corp.pmigroup.com/xymon-cgi/hobbitgraph.sh?host=wal-d
dbs-01.us.corp.pmigroup.com&service=disk&graph_width=576&graph_heigh
t=120&first=1&count=3&disp=wal%2dddbs%2d01%2eus%2ecorp%2epmigroup%2ecom&
graph=hourly&action=selzoom Link copied from "trends" section:
https://monitor.us.corp.pmigroup.com/xymon-cgi/hobbitgraph.sh?host=wal-d
dbs-01.us.corp.pmigroup.com&service=disk&graph_width=576&graph_heigh
t=120&first=1&count=3&disp=wal%2dddbs%2d01%2eus%2ecorp%2epmigroup%2ecom&
nostale&color=green&graph=hourly&action=selzoom Any ideas as to what's going on? I did find some old posts from
Henrik
about troubleshooting graph generation, but the case he was dealing with there was where the graphs wouldn't generate at all. In this case, the rrds are all there and the graphs are being generated, just not in all views. Thoughts? MarkHi, I got different issue with graphs, but I think my problem should be related to your comment : " I did find some old posts from Henrik about troubleshooting graph generation, but the case he was dealing with there was where the graphs wouldn't generate at all."It's worth checking hobbitgraph.cfg if you have added a number of your own custom graphs. The FNPATTERN operator is doing a regexp match, but in some cases the default pattern definition is very loose. For example, I'd added a bunch of devmon based system health check reporting stuff which generated file names like /var/lib/hobbit/rrd/HOSTNAME/temp.memory11.rrd These were causing the memory graph to die because it was attempting to query non-conformant RRDs for the particular graph. Solved by tightening the regexp definition as below (adding ^ in front, which means the filename has to start with memory, not just have it as a substring). [memory] FNPATTERN ^memory.(.+).rrd TITLE Memory Utilization Note that the FNPATTERN is not used when deciding if there are any RRD files to use to display a graph at all on the test status page. It just seems to glob TEST*.rrd to count the number of files to decide how many lines to display on the graph for TEST. This is with 4.2.3. David.
Thank you for your advices. I migrated my xymon server (and some clients) to 4.3.0-0.beta2, without migrating (is it needed anyway ?) my rrd files. My issue is very simple (and strange), I've 7 partitions monitored by xymon, all are reported correctly. I've 2 graphs for 3 partitions, but there should be a 3rd one for the 7th partition. Instead, I've a broken link. I've the same issue on custom tests/graphs. Many datas are graphed, but some graphs are missing. I think there should have a way to deal with it, in modifying how many values are on the same graph, or in correcting the root cause. Actually I don't have any skill in C nor php, so I can't solve it myself. Do you think this behaviour is related to your advice (changing FNPATTERN) ? Regards, Damien