Preventing Purple - Periodical Checks
list Tres Finocchiaro
I have a custom service check that is only done during certain times of the day and only on certain days of the week. We use crontab and a shell script to generate the output and pipe it into the front-end using the xymon executable. xymon 127.0.0.1 "status $server.$service $green|$red|$yellow $timestamp,$description" This check works terrific when it runs, but in between checks it goes purple. Without creating a fake service check in between each scheduled cron entry, what's the quickest way to configure that this check goes purple less often? (or if that's not possible, never go purple?) How are others handling occasional or daily service checks? Are you hard-coding the times into the script itself? -Tres - user-88678e65ced1@xymon.invalid
list Steve Coile
I think the "right" way would be to put the scheduling logic in the script, to have it always report green off-hours, rather than try to tell Xymon to ignore missing reports at certain times. Xymon is specifically designed (AFAIK) to receive regular data feeds, not intermittent ones. -- *Steve Coile*Senior Network and Systems Engineer, McClatchy Interactive <http://www.mcclatchyinteractive.com/>; Office: XXX-XXX-XXXX | Mobile: XXX-XXX-XXXX | Fax: XXX-XXX-XXXX On Mon, Sep 22, 2014 at 3:55 PM, Tres Finocchiaro <
▸
user-88678e65ced1@xymon.invalid> wrote:
I have a custom service check that is only done during certain times of the day and only on certain days of the week. We use crontab and a shell script to generate the output and pipe it into the front-end using the xymon executable. xymon 127.0.0.1 "status $server.$service $green|$red|$yellow $timestamp,$description" This check works terrific when it runs, but in between checks it goes purple. Without creating a fake service check in between each scheduled cron entry, what's the quickest way to configure that this check goes purple less often? (or if that's not possible, never go purple?) How are others handling occasional or daily service checks? Are you hard-coding the times into the script itself? -Tres - user-88678e65ced1@xymon.invalid
list Paul Root
Set the time the status is good for Xymon 127.0.0.1 “status+1d blah blah blah” Day, hour, minute, week, I believe are all valid. Check the man page of xymon.
▸
From: Xymon [mailto:xymon-bounces at xymon.com] On Behalf Of Tres Finocchiaro
Sent: Monday, September 22, 2014 2:55 PM
To: xymon at xymon.com
Subject: [Xymon] Preventing Purple - Periodical Checks
I have a custom service check that is only done during certain times of the day and only on certain days of the week.
We use crontab and a shell script to generate the output and pipe it into the front-end using the xymon executable.
xymon 127.0.0.1 "status $server.$service $green|$red|$yellow $timestamp,$description"
This check works terrific when it runs, but in between checks it goes purple.
Without creating a fake service check in between each scheduled cron entry, what's the quickest way to configure that this check goes purple less often? (or if that's not possible, never go purple?) How are others handling occasional or daily service checks? Are you hard-coding the times into the script itself?
-Tres
- user-88678e65ced1@xymon.invalid<mailto:user-88678e65ced1@xymon.invalid>
list Michael Short
Under BB, I would put in a "status+$delay" where the $delay was just a bit longer than needed to get me to the next check time. For example, if a check was to run once a day, I'd put in a delay of 25 hrs. this way, I would at least get a Purple light if it didn't run as often as I expected.
▸
From: Xymon [mailto:xymon-bounces at xymon.com] On Behalf Of Steve Coile
Sent: Monday, September 22, 2014 1:02 PM
To: Tres Finocchiaro
Cc: xymon at xymon.com
Subject: Re: [Xymon] Preventing Purple - Periodical Checks
I think the "right" way would be to put the scheduling logic in the script, to have it always report green off-hours, rather than try to tell Xymon to ignore missing reports at certain times.
Xymon is specifically designed (AFAIK) to receive regular data feeds, not intermittent ones.
--Steve Coile Senior Network and Systems Engineer, McClatchy Interactive<http://www.mcclatchyinteractive.com/>;
▸
Office: XXX-XXX-XXXX | Mobile: XXX-XXX-XXXX | Fax: XXX-XXX-XXXX
On Mon, Sep 22, 2014 at 3:55 PM, Tres Finocchiaro <user-88678e65ced1@xymon.invalid<mailto:user-88678e65ced1@xymon.invalid>> wrote:
I have a custom service check that is only done during certain times of the day and only on certain days of the week.
We use crontab and a shell script to generate the output and pipe it into the front-end using the xymon executable.
xymon 127.0.0.1 "status $server.$service $green|$red|$yellow $timestamp,$description"
This check works terrific when it runs, but in between checks it goes purple.
Without creating a fake service check in between each scheduled cron entry, what's the quickest way to configure that this check goes purple less often? (or if that's not possible, never go purple?) How are others handling occasional or daily service checks? Are you hard-coding the times into the script itself?
-Tres
- user-88678e65ced1@xymon.invalid<mailto:user-88678e65ced1@xymon.invalid>
****************************************************************************************** This message may contain confidential or proprietary information intended only for the use of the addressee(s) named above or may contain information that is legally privileged. If you are not the intended addressee, or the person responsible for delivering it to the intended addressee, you are hereby notified that reading, disseminating, distributing or copying this message is strictly prohibited. If you have received this message by mistake, please immediately notify us by replying to the message and delete the original message and any copies immediately thereafter.
Thank you. ****************************************************************************************** CLLD
list Tres Finocchiaro
@Paul/Mike, Top notch. I decided to put the entire delay value in the string to prevent a syntax error if it's blank. In our case, we check a few times from noon until 6PM, so I chose 20 hours. # Delay for Xymon to prevent purple
DELAY="+20h";
# Friday, add two more days
if [[ $(date +%u) == 5 ]] ; then
DELAY="+68h";
fi
# [...]
xymon 127.0.0.1 "status${DELAY} $server.$service $color $msg"P.S. I don't know how we ever survived on a Windows platform. This product was meant for Linux and it's tremendously useful scripting capabilities. :) Thanks again. -Tres
list Roland Soderstrom
I got a slightly different approach that seems to work fairly well for several same type tests.
I got several things I run the same way, periodically so xymon would go purple.
Usually it is cronjobs that runs doing whatever, but we added some xymon functionality to them.
They output xymon data to a couple of files that xymon then checks through a standard xymon script.
DATAFILE that shows whatever data from the cronjob.
COLOR that just shows the color.
HEADER just a one liner test heading in HTML.
I then have xymon checking the age of the color file and just report on that color.
For one test that is a binary we just run a wrapper cronscript that makes the xymon data and runs the binary.
The application team has all control over what is put in the xymon files without the need to become root or the xymon user.
TEST=MYrpt
MYRPTDIR=/var/tmp
MYRPTBASE=MY-reports-xym
MYRPTDATAFILE=${MYRPTDIR}/${MYRPTBASE}.data
MYRPTCOLOR=${MYRPTDIR}/${MYRPTBASE}.color
MYRPTHEADER=${MYRPTDIR}/${MYRPTBASE}.header
ERRORTEXT="<H3>ERROR: the ${MYRPTCOLOR} file is older than one day.</H3><BR>
That probably means that the cronjob for checking the MY reports hasn't run.<BR>
Please check this in `hostname`<BR>
`ls -l ${MYRPTCOLOR}`"
# Check that the color file is less than 1 day old, if not the cronjob haven't run.
AGE=`find ${MYRPTDIR} -name ${MYRPTBASE}.color -a -mtime +1`
if [ -n "$AGE" ]
then
HEADER=`cat ${MYRPTHEADER}`
COLOR=red
DATA="$ERRORTEXT"
else
HEADER=`cat ${MYRPTHEADER}`
COLOR=`cat ${MYRPTCOLOR}`
DATA=`cat ${MYRPTDATAFILE}`
fi
LINE="status $MACHINE.$TEST $COLOR `date`
${HEADER}
${DATA}
"
# Send to Xymon Server
$XYMON $XYMSRV "$LINE"
- Roland
▸
From: Xymon [mailto:xymon-bounces at xymon.com] On Behalf Of Tres Finocchiaro
Sent: Tuesday, 23 September 2014 6:28 AM
To: Michael Short
Cc: xymon at xymon.com
Subject: Re: [Xymon] Preventing Purple - Periodical Checks
@Paul/Mike,
Top notch. I decided to put the entire delay value in the string to prevent a syntax error if it's blank. In our case, we check a few times from noon until 6PM, so I chose 20 hours.
# Delay for Xymon to prevent purple
DELAY="+20h";
# Friday, add two more days
if [[ $(date +%u) == 5 ]] ; then
DELAY="+68h";
fi
# [...]
xymon 127.0.0.1 "status${DELAY} $server.$service $color $msg"
P.S. I don't know how we ever survived on a Windows platform. This product was meant for Linux and it's tremendously useful scripting capabilities. :)
Thanks again.
-Tres
list Paul Root
Actually, you’d want it to report what the test is already set as.
▸
From: Xymon [mailto:xymon-bounces at xymon.com] On Behalf Of Steve Coile Sent: Monday, September 22, 2014 3:01 PM To: Tres Finocchiaro Cc: xymon at xymon.com Subject: Re: [Xymon] Preventing Purple - Periodical Checks I think the "right" way would be to put the scheduling logic in the script, to have it always report green off-hours, rather than try to tell Xymon to ignore missing reports at certain times. Xymon is specifically designed (AFAIK) to receive regular data feeds, not intermittent ones. -- Steve Coile Senior Network and Systems Engineer, McClatchy Interactive<http://www.mcclatchyinteractive.com/>; Office: XXX-XXX-XXXX | Mobile: XXX-XXX-XXXX | Fax: XXX-XXX-XXXX On Mon, Sep 22, 2014 at 3:55 PM, Tres Finocchiaro <user-88678e65ced1@xymon.invalid<mailto:user-88678e65ced1@xymon.invalid>> wrote: I have a custom service check that is only done during certain times of the day and only on certain days of the week. We use crontab and a shell script to generate the output and pipe it into the front-end using the xymon executable. xymon 127.0.0.1 "status $server.$service $green|$red|$yellow $timestamp,$description" This check works terrific when it runs, but in between checks it goes purple. Without creating a fake service check in between each scheduled cron entry, what's the quickest way to configure that this check goes purple less often? (or if that's not possible, never go purple?) How are others handling occasional or daily service checks? Are you hard-coding the times into the script itself? -Tres - user-88678e65ced1@xymon.invalid<mailto:user-88678e65ced1@xymon.invalid>
list John Rothlisberger
What we do, and it has proven to be very helpful, at least for purple indications of Windows clients. We send email alerts when “disk” has been purple for over 30 minutes. Why, you may ask. There are specific instances when tests go purple and there is no need to alert on all tests. If disk is purple, most likely all the rest are purple except for connection. * The number one cause of purple alerts is that the BBWin service is not running. Please check this service FIRST and start/restart if necessary. * The client server is hung, sometimes it will respond to a ping and not otherwise indicate that it is down. * The C drive is full. 99% of the time the purple is not indicative of a server issue, but when it is, sometimes this is the only way to catch it early. Thanks, John Upcoming PTO: (none) John Rothlisberger IT Strategy, Infrastructure & Security - Technology Growth Platform TGP for Business Process Outsourcing Accenture XXX.XXX.XXXX office
▸
From: Xymon [mailto:xymon-bounces at xymon.com] On Behalf Of Tres Finocchiaro
Sent: Monday, September 22, 2014 2:55 PM
To: xymon at xymon.com
Subject: [Xymon] Preventing Purple - Periodical Checks
I have a custom service check that is only done during certain times of the day and only on certain days of the week.
We use crontab and a shell script to generate the output and pipe it into the front-end using the xymon executable.
xymon 127.0.0.1 "status $server.$service $green|$red|$yellow $timestamp,$description"
This check works terrific when it runs, but in between checks it goes purple.
Without creating a fake service check in between each scheduled cron entry, what's the quickest way to configure that this check goes purple less often? (or if that's not possible, never go purple?) How are others handling occasional or daily service checks? Are you hard-coding the times into the script itself?
-Tres
- user-88678e65ced1@xymon.invalid<mailto:user-88678e65ced1@xymon.invalid>
This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited. Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy.
www.accenture.com
list Tracy di Marco White
We have a bunch of feeds that happen on odd schedules, and they go red or yellow on various problems, but are usually green. On days or times that there is no feed to process, the test goes clear. We usually set the lifetime to at least two iterations of the test, so there are no purples unless there is a different problem. -Tracy
▸
On Sep 23, 2014, at 8:48 AM, <user-7adce57665bb@xymon.invalid> <user-7adce57665bb@xymon.invalid> wrote: What we do, and it has proven to be very helpful, at least for purple indications of Windows clients. We send email alerts when “disk” has been purple for over 30 minutes. Why, you may ask. There are specific instances when tests go purple and there is no need to alert on all tests. If disk is purple, most likely all the rest are purple except for connection. The number one cause of purple alerts is that the BBWin service is not running. Please check this service FIRST and start/restart if necessary. The client server is hung, sometimes it will respond to a ping and not otherwise indicate that it is down. The C drive is full. 99% of the time the purple is not indicative of a server issue, but when it is, sometimes this is the only way to catch it early. Thanks, John Upcoming PTO: (none) John Rothlisberger IT Strategy, Infrastructure & Security - Technology Growth Platform TGP for Business Process Outsourcing Accenture XXX.XXX.XXXX office From: Xymon [mailto:xymon-bounces at xymon.com] On Behalf Of Tres Finocchiaro Sent: Monday, September 22, 2014 2:55 PM To: xymon at xymon.com Subject: [Xymon] Preventing Purple - Periodical Checks I have a custom service check that is only done during certain times of the day and only on certain days of the week. We use crontab and a shell script to generate the output and pipe it into the front-end using the xymon executable. xymon 127.0.0.1 "status $server.$service $green|$red|$yellow $timestamp,$description" This check works terrific when it runs, but in between checks it goes purple. Without creating a fake service check in between each scheduled cron entry, what's the quickest way to configure that this check goes purple less often? (or if that's not possible, never go purple?) How are others handling occasional or daily service checks? Are you hard-coding the times into the script itself? -Tres - user-88678e65ced1@xymon.invalid This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited. Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy. ______________________________________________________________________________________ www.accenture.com