Xymon Mailing List Archive search

Handling Downtime across timezones?

6 messages in this thread

list Thomas R. Brand · Wed, 8 Jul 2009 15:33:41 -0400 ·
I have client systems spread over several time zones from AST (Atlantic
Standard Time) to HST (Hawaii Standard Time) and everything in between.

 
My hobbit server is in EST time zone.

 
The client systems run an 'End of Day' [EOD] process every night.  

This process starts sometime between 12:30 am and 3:30am client LOCAL
time and runs about 15 minutes.

During the EOD run, several background processes are intentionally
stopped. 

 
I don't want to have alerts generated if these processes are
down/stopped during EOD  but do want to be alerted if the processes are
down outside of the EOD window.

 
E.g.:

 * client systems in Puerto Rico run EOD at 12:30am AST, which is 11:30
pm EST (previous day)

 * client systems in Hawaii run EOD at 3:30 am HST, which is 9:30 am EST

 
I have not figured out a way to do this - at least not with the DOWNTIME
flag.

 
It appears that the DOWNTIME flag on the hobbit server uses the hobbit
servers' current time to determine if an event is occurring during the
DOWNTIME interval; that's a problem for me since from the hobbit servers
perspective,  the EOD can happen over a 10 hour period.

 
Any suggestions on how to configure things so that I don't get alerts
for processes being 'down' from 12:30 - 3:30 am client local time?

 
N.B - I'm going to be dealing with over 7000 clients

 
Thanks,

 
Tom Brand
list Josh Luthman · Wed, 8 Jul 2009 15:42:23 -0400 ·
You specify the DOWNTIME tag for those hosts in the different timezones.
Were you looking for a less labor intensive option?

Josh Luthman
Office: XXX-XXX-XXXX
Direct: XXX-XXX-XXXX
XXXX Wayne St
Suite XXXX
Troy, OH XXXXX

"When you have eliminated the impossible, that which remains, however
improbable, must be the truth."
--- Sir Arthur Conan Doyle
quoted from Thomas R. Brand


On Wed, Jul 8, 2009 at 3:33 PM, Brand, Thomas R. <user-10a840458972@xymon.invalid> wrote:
 I have client systems spread over several time zones from AST (Atlantic
Standard Time) to HST (Hawaii Standard Time) and everything in between.


My hobbit server is in EST time zone.


The client systems run an ‘End of Day’ [EOD] process every night.

This process starts sometime between 12:30 am and 3:30am client LOCAL time
and runs about 15 minutes.

During the EOD run, several background processes are intentionally stopped.


I don’t want to have alerts generated if these processes are down/stopped
during EOD  but do want to be alerted if the processes are down outside of
the EOD window.


E.g.:

 * client systems in Puerto Rico run EOD at 12:30am AST, which is 11:30 pm
EST (previous day)

 * client systems in Hawaii run EOD at 3:30 am HST, which is 9:30 am EST


I have not figured out a way to do this – at least not with the DOWNTIME
flag.


It appears that the DOWNTIME flag on the hobbit server uses the hobbit
servers’ current time to determine if an event is occurring during the
DOWNTIME interval; that’s a problem for me since from the hobbit servers
perspective,  the EOD can happen over a 10 hour period.


Any suggestions on how to configure things so that I don’t get alerts for
processes being ‘down’ from 12:30 – 3:30 am client local time?


N.B – I’m going to be dealing with over 7000 clients


Thanks,


*Tom Brand*
list Thomas R. Brand · Wed, 8 Jul 2009 15:51:04 -0400 ·
Yes I was; this is just one example where the clients time zone is
affecting the desired result.

 
The hobbit server has no knowledge of the clients timezone; but the
events from the client come up with the timezone in the timestamp.

E.g.:

 
The 'CPU' page shows the servers time in the top right (which is EST),
but the client status shows PDT:

 
Wed Jul 08 15:46:33 2009


Wed Jul 8 12:46:26 PDT 2009 up: 11:58, 0 users, 113 procs, load=0.02


I guess I was hoping for someway to tell the event/alert handler to look
at the timestamp of the message sent by the client.
quoted from Josh Luthman

 
From: Josh Luthman [mailto:user-4c45a83f15cb@xymon.invalid] 
Sent: Wednesday, July 08, 2009 3:42 PM
To: user-ae9b8668bcde@xymon.invalid
Subject: Re: [hobbit] Handling Downtime across timezones?

 
You specify the DOWNTIME tag for those hosts in the different timezones.
Were you looking for a less labor intensive option?

Josh Luthman
Office: XXX-XXX-XXXX
Direct: XXX-XXX-XXXX
XXXX Wayne St
Suite XXXX
Troy, OH XXXXX

"When you have eliminated the impossible, that which remains, however
improbable, must be the truth."
--- Sir Arthur Conan Doyle


On Wed, Jul 8, 2009 at 3:33 PM, Brand, Thomas R. <user-10a840458972@xymon.invalid>
wrote:

I have client systems spread over several time zones from AST (Atlantic
Standard Time) to HST (Hawaii Standard Time) and everything in between.

 
My hobbit server is in EST time zone.

 
The client systems run an 'End of Day' [EOD] process every night.  

This process starts sometime between 12:30 am and 3:30am client LOCAL
time and runs about 15 minutes.

During the EOD run, several background processes are intentionally
stopped. 

 
I don't want to have alerts generated if these processes are
down/stopped during EOD  but do want to be alerted if the processes are
down outside of the EOD window.

 
E.g.:

 * client systems in Puerto Rico run EOD at 12:30am AST, which is 11:30
pm EST (previous day)

 * client systems in Hawaii run EOD at 3:30 am HST, which is 9:30 am EST

 
I have not figured out a way to do this - at least not with the DOWNTIME
flag.

 
It appears that the DOWNTIME flag on the hobbit server uses the hobbit
servers' current time to determine if an event is occurring during the
DOWNTIME interval; that's a problem for me since from the hobbit servers
perspective,  the EOD can happen over a 10 hour period.

 
Any suggestions on how to configure things so that I don't get alerts
for processes being 'down' from 12:30 - 3:30 am client local time?

 
N.B - I'm going to be dealing with over 7000 clients

 
Thanks,

 
Tom Brand
list Josh Luthman · Wed, 8 Jul 2009 15:58:10 -0400 ·
Well the DOWNTIME processing is done on the server, so I'm willing to bet
that the time it calculates is based on server time.  The CPU output is
simply a text dump and transmitted, I believe.

Maybe someone has come across this before.  If so, please help us out =)
quoted from Thomas R. Brand

Josh Luthman
Office: XXX-XXX-XXXX
Direct: XXX-XXX-XXXX
XXXX Wayne St
Suite XXXX
Troy, OH XXXXX

"When you have eliminated the impossible, that which remains, however
improbable, must be the truth."
--- Sir Arthur Conan Doyle


On Wed, Jul 8, 2009 at 3:51 PM, Brand, Thomas R. <user-10a840458972@xymon.invalid> wrote:
 Yes I was; this is just one example where the clients time zone is
affecting the desired result.


The hobbit server has no knowledge of the clients timezone; but the events
from the client come up with the timezone in the timestamp.

E.g.:


The ‘CPU’ page shows the servers time in the top right (which is EST), but
the client status shows PDT:


*Wed Jul 08 15:46:33 2009*
*Wed Jul 8 12:46:26 PDT 2009 up: 11:58, 0 users, 113 procs, load=0.02*


I guess I was hoping for someway to tell the event/alert handler to look at
the timestamp of the message sent by the client.


*From:* Josh Luthman [mailto:user-4c45a83f15cb@xymon.invalid]
*Sent:* Wednesday, July 08, 2009 3:42 PM
*To:* user-ae9b8668bcde@xymon.invalid
*Subject:* Re: [hobbit] Handling Downtime across timezones?


You specify the DOWNTIME tag for those hosts in the different timezones.
Were you looking for a less labor intensive option?

Josh Luthman
Office: XXX-XXX-XXXX
Direct: XXX-XXX-XXXX
XXXX Wayne St
Suite XXXX
Troy, OH XXXXX

"When you have eliminated the impossible, that which remains, however
improbable, must be the truth."
--- Sir Arthur Conan Doyle

 On Wed, Jul 8, 2009 at 3:33 PM, Brand, Thomas R. <user-10a840458972@xymon.invalid> wrote:

I have client systems spread over several time zones from AST (Atlantic
Standard Time) to HST (Hawaii Standard Time) and everything in between.


My hobbit server is in EST time zone.


The client systems run an ‘End of Day’ [EOD] process every night.

This process starts sometime between 12:30 am and 3:30am client LOCAL time
and runs about 15 minutes.

During the EOD run, several background processes are intentionally stopped.


I don’t want to have alerts generated if these processes are down/stopped
during EOD  but do want to be alerted if the processes are down outside of
the EOD window.


E.g.:

 * client systems in Puerto Rico run EOD at 12:30am AST, which is 11:30 pm
EST (previous day)

 * client systems in Hawaii run EOD at 3:30 am HST, which is 9:30 am EST


I have not figured out a way to do this – at least not with the DOWNTIME
flag.


It appears that the DOWNTIME flag on the hobbit server uses the hobbit
servers’ current time to determine if an event is occurring during the
DOWNTIME interval; that’s a problem for me since from the hobbit servers
perspective,  the EOD can happen over a 10 hour period.


Any suggestions on how to configure things so that I don’t get alerts for
processes being ‘down’ from 12:30 – 3:30 am client local time?


N.B – I’m going to be dealing with over 7000 clients


Thanks,


*Tom Brand*

list Scot Kreienkamp · Wed, 8 Jul 2009 16:06:03 -0400 ·
Why not use the client to send a disable message for the duration of
your down window as part of your EOD process?  That way it would be in
your local client time zones.  Send the length of your down window in
the disable message and then the server will resume monitoring at the
expiration of the down window, or it can send an enable message when the
EOD process is complete.

 
Thanks,

 
Scot Kreienkamp
quoted from Josh Luthman


From: Brand, Thomas R. [mailto:user-10a840458972@xymon.invalid] 
Sent: Wednesday, July 08, 2009 3:34 PM
To: user-ae9b8668bcde@xymon.invalid
Subject: [hobbit] Handling Downtime across timezones?

 
I have client systems spread over several time zones from AST (Atlantic
Standard Time) to HST (Hawaii Standard Time) and everything in between.

 
My hobbit server is in EST time zone.

 
The client systems run an 'End of Day' [EOD] process every night.  

This process starts sometime between 12:30 am and 3:30am client LOCAL
time and runs about 15 minutes.

During the EOD run, several background processes are intentionally
stopped. 

 
I don't want to have alerts generated if these processes are
down/stopped during EOD  but do want to be alerted if the processes are
down outside of the EOD window.

 
E.g.:

 * client systems in Puerto Rico run EOD at 12:30am AST, which is 11:30
pm EST (previous day)

 * client systems in Hawaii run EOD at 3:30 am HST, which is 9:30 am EST

 
I have not figured out a way to do this - at least not with the DOWNTIME
flag.

 
It appears that the DOWNTIME flag on the hobbit server uses the hobbit
servers' current time to determine if an event is occurring during the
DOWNTIME interval; that's a problem for me since from the hobbit servers
perspective,  the EOD can happen over a 10 hour period.

 
Any suggestions on how to configure things so that I don't get alerts
for processes being 'down' from 12:30 - 3:30 am client local time?

 
N.B - I'm going to be dealing with over 7000 clients

 
Thanks,

 
Tom Brand
list Josh Luthman · Wed, 8 Jul 2009 16:12:07 -0400 ·
See what I mean? =)

That would accomplish the same thing on the server.  I think that would work
for what you're asking, however, it may be a bit labor intensive.
quoted from Scot Kreienkamp

Josh Luthman
Office: XXX-XXX-XXXX
Direct: XXX-XXX-XXXX
XXXX Wayne St
Suite XXXX
Troy, OH XXXXX

"When you have eliminated the impossible, that which remains, however
improbable, must be the truth."
--- Sir Arthur Conan Doyle


On Wed, Jul 8, 2009 at 4:06 PM, Scot Kreienkamp <user-462cf0b6d846@xymon.invalid>wrote:
 Why not use the client to send a disable message for the duration of your
down window as part of your EOD process?  That way it would be in your local
client time zones.  Send the length of your down window in the disable
message and then the server will resume monitoring at the expiration of the
down window, or it can send an enable message when the EOD process is
complete.


Thanks,


Scot Kreienkamp

*From:* Brand, Thomas R. [mailto:user-10a840458972@xymon.invalid]
*Sent:* Wednesday, July 08, 2009 3:34 PM
*To:* user-ae9b8668bcde@xymon.invalid
*Subject:* [hobbit] Handling Downtime across timezones?


I have client systems spread over several time zones from AST (Atlantic
Standard Time) to HST (Hawaii Standard Time) and everything in between.


My hobbit server is in EST time zone.


The client systems run an ‘End of Day’ [EOD] process every night.

This process starts sometime between 12:30 am and 3:30am client LOCAL time
and runs about 15 minutes.

During the EOD run, several background processes are intentionally stopped.


I don’t want to have alerts generated if these processes are down/stopped
during EOD  but do want to be alerted if the processes are down outside of
the EOD window.


E.g.:

 * client systems in Puerto Rico run EOD at 12:30am AST, which is 11:30 pm
EST (previous day)

 * client systems in Hawaii run EOD at 3:30 am HST, which is 9:30 am EST


I have not figured out a way to do this – at least not with the DOWNTIME
flag.


It appears that the DOWNTIME flag on the hobbit server uses the hobbit
servers’ current time to determine if an event is occurring during the
DOWNTIME interval; that’s a problem for me since from the hobbit servers
perspective,  the EOD can happen over a 10 hour period.


Any suggestions on how to configure things so that I don’t get alerts for
processes being ‘down’ from 12:30 – 3:30 am client local time?


N.B – I’m going to be dealing with over 7000 clients


Thanks,


*Tom Brand*