Xymon Mailing List Archive search

Graphing in Xymon

list Japheth Cleaver
Mon, 9 Sep 2019 10:45:02 -0700
Message-Id: <user-f403ebfcd6d3@xymon.invalid>

You should see four different RRA's (round-robin archives) when you run 
'rrdtool info' on the actual RRD file, each being populated as "roll-up" 
averages of the various data point. For example: ]# rrdtool info clock.rrd
filename = "clock.rrd"
rrd_version = "0003"
step = 300
last_update = 1568049660
header_size = 1208
ds[la].index = 0
ds[la].type = "GAUGE"
ds[la].minimal_heartbeat = 600
ds[la].min = NaN
ds[la].max = NaN
ds[la].last_ds = "0"
ds[la].value = 0.0000000000e+00
ds[la].unknown_sec = 0
rra[0].cf = "AVERAGE"
rra[0].rows = 576
rra[0].cur_row = 253
rra[0].pdp_per_row = 1
rra[0].xff = 5.0000000000e-01
rra[0].cdp_prep[0].value = NaN
rra[0].cdp_prep[0].unknown_datapoints = 0
rra[1].cf = "AVERAGE"
rra[1].rows = 576
rra[1].cur_row = 419
rra[1].pdp_per_row = 6
rra[1].xff = 5.0000000000e-01
rra[1].cdp_prep[0].value = 0.0000000000e+00
rra[1].cdp_prep[0].unknown_datapoints = 0
rra[2].cf = "AVERAGE"
rra[2].rows = 576
rra[2].cur_row = 520
rra[2].pdp_per_row = 24
rra[2].xff = 5.0000000000e-01
rra[2].cdp_prep[0].value = 0.0000000000e+00
rra[2].cdp_prep[0].unknown_datapoints = 0
rra[3].cf = "AVERAGE"
rra[3].rows = 576
rra[3].cur_row = 392
rra[3].pdp_per_row = 288
rra[3].xff = 5.0000000000e-01
rra[3].cdp_prep[0].value = 3.4666666667e-01
rra[3].cdp_prep[0].unknown_datapoints = 0

I suspect the issue here is the lack of data in the originating source. 
It seems like a once-a-day datapoint should at least be made visible in 
the 2d RRA, but I've not actually tried to configure one with low 
frequency before. Would you be able to artificially submit that point 
several times within a 5m period and see if it populates? Keeping 
--no-cache back on may be beneficial here too.

Regards,
-jc

On 9/6/2019 11:01 AM, Root, Paul T wrote:
I don?t think the rrd file is working right.

It is there, and the latest numbers are in, but I don?t see any 
archive numbers populating.

Where can I check for that?

*From:*Ralph Mitchell <user-00a5e44c48c0@xymon.invalid>
*Sent:* Friday, September 06, 2019 10:07 AM
*To:* Root, Paul T <user-76fdb6883669@xymon.invalid>
*Cc:* Japheth Cleaver <user-87556346d4af@xymon.invalid>; Stef Coene 
<user-dbffe946c0f4@xymon.invalid>; xymon at xymon.com
*Subject:* Re: [Xymon] Graphing in Xymon

Could be wrong, but I think the graphs need two or three data points 
in order to show anything.

Ralph Mitchell

On Fri, Sep 6, 2019 at 9:47 AM Root, Paul T <user-76fdb6883669@xymon.invalid 
<mailto:user-76fdb6883669@xymon.invalid>> wrote:

    Looking in the rrd file, it doesn't seem like there are the
    historical data in the file.

    The data is only going to change once a day, so I was only calling
    the script once a day.

    I'm going to run the xymon script every hour to see if that gets
    the pump primed, and give me more data.

    Does the graphing thing need more frequent data?


    -----Original Message-----
    From: Xymon <xymon-bounces at xymon.com
    <mailto:xymon-bounces at xymon.com>> On Behalf Of Root, Paul T
    Sent: Friday, September 06, 2019 8:28 AM
    To: 'Japheth Cleaver' <user-87556346d4af@xymon.invalid
    <mailto:user-87556346d4af@xymon.invalid>>; 'Stef Coene'
    <user-dbffe946c0f4@xymon.invalid <mailto:user-dbffe946c0f4@xymon.invalid>>;
    xymon at xymon.com <mailto:xymon at xymon.com>
    Subject: Re: [Xymon] Graphing in Xymon

    No difference.

    I didn't restart xymon for that, just removed the --no-cache from
    the lines in tasks.cfg , and killed the rrd processes.


    -----Original Message-----
    From: Japheth Cleaver <user-87556346d4af@xymon.invalid
    Sent: Thursday, September 05, 2019 5:19 PM
    To: Root, Paul T <user-76fdb6883669@xymon.invalid
    <mailto:user-76fdb6883669@xymon.invalid>>; 'Stef Coene'
    <user-dbffe946c0f4@xymon.invalid <mailto:user-dbffe946c0f4@xymon.invalid>>;
    xymon at xymon.com <mailto:xymon at xymon.com>
    Subject: Re: [Xymon] Graphing in Xymon

    Hmm.

    That's definitely odd, but if at least some of the graphs are showing
    then it's not an overall systemic issue. Have you tried removing the
    --no-cache from the RRD calls by any chance?

    -jc

    On 9/5/2019 1:48 PM, Root, Paul T wrote:
Those files look ok

$ fuser *
rrdctl.2339:? ? ? ? ? 2339
rrdctl.2437:? ? ? ? ? 2437
$ ps -ef |grep 2339
xymon? ? ?2339? 2323? 0 15:09 ?? ? ? ? 00:00:09 xymond_rrd
    --rrddir=/var/lib/xymon/rrd --no-cache
root? ? ? 5083? 4674? 0 15:43 pts/0? ? 00:00:00 grep 2339
$ ps -ef |grep 2437
xymon? ? ?2437? 2324? 0 15:09 ?? ? ? ? 00:00:00 xymond_rrd
    --rrddir=/var/lib/xymon/rrd --no-cache
root? ? ? 5086? 4674? 0 15:43 pts/0? ? 00:00:00 grep 2437
$ ls -la
total 19964
drwxr-xr-x.? 2 xymon xymon? ? ?4096 Sep? 5 15:39 .
drwxr-xr-x. 12 xymon xymon? ? ?4096 Mar? 1? 2016 ..
-rw-rw-r--.? 1 xymon xymon? ? 52136 Sep? 5 15:43 alert.chk
-rw-rw-r--.? 1 xymon xymon? ? ? ?96 Sep? 5 15:43 alert.chk.sub
-rw-r--r--.? 1 xymon xymon? ? ? ?24 Sep? 5 15:39 ping..status
srw-rw-rw-.? 1 xymon xymon? ? ? ? 0 Sep? 5 15:09 rrdctl.2339
srw-rw-rw-.? 1 xymon xymon? ? ? ? 0 Sep? 5 15:09 rrdctl.2437
-rw-r--r--.? 1 xymon xymon? ? ? ?22 Oct 19? 2018 ssh_8022..status
-rw-r--r--.? 1 xymon xymon 20366660 Sep? 5 15:39 xymond.chk

Selinux is set to permissive.

Most graphs are working for other tests. Some complicated ones
    with variable data points. I didn't set those up, but they are
    working well.

-----Original Message-----
From: Japheth Cleaver <user-87556346d4af@xymon.invalid
Sent: Thursday, September 05, 2019 3:33 PM
To: Root, Paul T <user-76fdb6883669@xymon.invalid
    <mailto:user-76fdb6883669@xymon.invalid>>; 'Stef Coene'
    <user-dbffe946c0f4@xymon.invalid <mailto:user-dbffe946c0f4@xymon.invalid>>;
    xymon at xymon.com <mailto:xymon at xymon.com>
Subject: Re: [Xymon] Graphing in Xymon

This is probably a sign of invalid rrdctl socket files, as
    that's what
showgraph is trying to use to communicate with xymond_rrd.

These should be in the TMP/RUN dir on a source compile, named
"rrdctl.<pid>", one for each of the currently running xymond_rrd
processes (usually two,m one for status, one for data). They'll
    need to
be writable by your apache user at least, and SELinux may require
additional permissions for the sendto if you have it enabled and
enforcing. If you see lots of stale ones, be sure to remove them
    -- as
showgraph.sh doesn't know what's current, I believe it tries to
    cycle
through all of them, which could end up in timeouts.

Regardless, premature header ending isn't a desired result no matter
what the situation. Trying to find all cases of this is a longer
    term
goal for all the CGIs.

HTH,
-jc

On 9/5/2019 1:16 PM, Root, Paul T wrote:
Ah,
? ? ? ? ? ?I wasn't thinking about apache logs, I was looking in
    xymon logs. Yes, errors:


[Thu Sep 05 15:12:22 2019] [error] [client 10.139.63.193]
    2019-09-05 15:12:22.832251 Sendto failed: Connection refused,
    referer:
    https://nsmdenvp117.corp.intranet/xymon-cgi/svcstatus.sh?HOST=nsmdenvp185&SERVICE=ConfigPulls
[Thu Sep 05 15:12:22 2019] [error] [client 10.139.63.193]
    2019-09-05 15:12:22.832306 Sendto failed: Connection refused,
    referer:
    https://nsmdenvp117.corp.intranet/xymon-cgi/svcstatus.sh?HOST=nsmdenvp185&SERVICE=ConfigPulls
[Thu Sep 05 15:12:23 2019] [error] [client 10.139.63.193]
    Premature end of script headers: showgraph.sh, referer:
    https://nsmdenvp117.corp.intranet/xymon-cgi/svcstatus.sh?HOST=nsmdenvp185&SERVICE=ConfigPulls
? ? ? ? ? ?And yes, the rrd file exists in the right directory.
    It only has 1 data point at this time, but again. Running rrdtool
    graph on that file produces a graph

? ? ? ? ? ?Oh, I never mentioned before. ?This is CentOS 6 with
    my own build for 4.3.21.? I'm starting to move to the Terabithia
    repo, but haven't got there yet on the server.

Paul.


-----Original Message-----
From: Japheth Cleaver <user-87556346d4af@xymon.invalid
Sent: Thursday, September 05, 2019 3:00 PM
To: Root, Paul T <user-76fdb6883669@xymon.invalid
    <mailto:user-76fdb6883669@xymon.invalid>>; 'Stef Coene'
    <user-dbffe946c0f4@xymon.invalid <mailto:user-dbffe946c0f4@xymon.invalid>>;
    xymon at xymon.com <mailto:xymon at xymon.com>
Subject: Re: [Xymon] Graphing in Xymon

A 500 ISE isn't a good sign. Is there anything in the web error
    logs?

And just to confirm, the file does actually existing in the RRD
directory as expected?

Regards,
-jc

On 9/5/2019 12:56 PM, Root, Paul T wrote:
Not much help here:

$ wget
    "http://nsmdenvp117.corp.intranet/xymon-cgi/showgraph.sh?host=nsmdenvp185&service=ncv:ConfigPulls&graph_width=576&graph_height=120&disp=nsmdenvp185&nostale&color=green&graph_start=1567539940&graph_end=1567712740&graph=hourly&action=view";
--2019-09-05 14:46:58--
    http://nsmdenvp117.corp.intranet/xymon-cgi/showgraph.sh?host=nsmdenvp185&service=ncv:ConfigPulls&graph_width=576&graph_height=120&disp=nsmdenvp185&nostale&color=green&graph_start=1567539940&graph_end=1567712740&graph=hourly&action=view
Resolving nsmdenvp117.corp.intranet... 151.119.9.117
Connecting to nsmdenvp117.corp.intranet|151.119.9.117|:80...
    connected.
HTTP request sent, awaiting response... 500 Internal Server Error
2019-09-05 14:46:59 ERROR 500: Internal Server Error.


Running the rrdtool graph by hand, does work now, and I
    transferred the lines to graphs.cfg. But that hasn't translated
    into a graph.

rrdtool graph ConfigPulls.jpg \
? ? ? ? ? ? --title "NCM Config Pulls" \
"DEF:NetAtoD=ConfigPulls.rrd:NetAtoD:AVERAGE" \
"DEF:NetEtoN=ConfigPulls.rrd:NetEtoN:AVERAGE" \
"DEF:NetOtoZ1to9=ConfigPulls.rrd:NetOtoZ1to9:AVERAGE" \
"DEF:Other=ConfigPulls.rrd:Other:AVERAGE" \
"DEF:InfraGen=ConfigPulls.rrd:InfraGen:AVERAGE" \
"DEF:InfraSec=ConfigPulls.rrd:InfraSec:AVERAGE" \
? ? ? ? ? ? "LINE2:NetAtoD#00FF00:Net A to D" \
? ? ? ? ? ? "LINE2:NetEtoN#00CCCC:Net E to N" \
? ? ? ? ? ? "LINE2:NetOtoZ1to9#FFCC00:Net O to Z & 1 to 9" \
? ? ? ? ? ? "LINE2:Other#FF0000:Other" \
"LINE2:InfraGen#FFFF00:Infrastructure General" \
"LINE2:InfraSec#FFFFCC:Infrastructure Secure" \
? ? ? ? ? ? "COMMENT:\n"


-----Original Message-----
From: Xymon <xymon-bounces at xymon.com
    <mailto:xymon-bounces at xymon.com>> On Behalf Of Stef Coene
Sent: Thursday, September 05, 2019 2:18 PM
To: xymon at xymon.com <mailto:xymon at xymon.com>
Subject: Re: [Xymon] Graphing in Xymon

On 4/09/19 19:59, Root, Paul T wrote:
The graph doesn't show up, just a broken link in both the
    test and
trends pages.

Any help?
Then there is a problem generating the RRD, the error is
    returned as
text. But you browser expects a graph, not some text so it
    displays a
broken image symbol.

When yoy right click on one of the broken image, you can say
    'open image
in new tab'.
This will give you the direct URL of the graph.
If you do a wget of this URL (don't forget quotes!) on your xymon
server, you will download the real error.
Just do a vi of the downloaded file to see this error.


Stef

    This communication is the property of CenturyLink and may contain
    confidential or privileged information. Unauthorized use of this
    communication is strictly prohibited and may be unlawful. If you
    have received this communication in error, please immediately
    notify the sender by reply e-mail and destroy all copies of the
    communication and any attachments.
    

    This communication is the property of CenturyLink and may contain
    confidential or privileged information. Unauthorized use of this
    communication is strictly prohibited and may be unlawful. If you
    have received this communication in error, please immediately
    notify the sender by reply e-mail and destroy all copies of the
    communication and any attachments.


This communication is the property of CenturyLink and may contain 
confidential or privileged information. Unauthorized use of this 
communication is strictly prohibited and may be unlawful. If you have 
received this communication in error, please immediately notify the 
sender by reply e-mail and destroy all copies of the communication and 
any attachments.