Intelligent alerting across multiple servers?

Date: Thu, 30 Jul 2009 14:57:06 +0100
From: user-c15424b7e83a@xymon.invalid
To: user-ae9b8668bcde@xymon.invalid
CC: user-1f2406f832af@xymon.invalid
Subject: Re: [hobbit] Intelligent alerting across multiple servers?

TJ, we have 3 data centres and each data centre contains a xymon server. All clients in a data centre only report to their local xymon server. However the xymon servers can communicate with each other using BBDISPLAYS (its a little more complicated than that as we utilise a bbproxy in each DC to take the messages and spray them to all 3 xymons).

 
This is new architecture that I was not aware of thus the inclusion your notes here.

http://en.wikibooks.org/wiki/System_Monitoring_with_Xymon/Administration_Guide#HA-WAN_2_approach

Please explain more in details on the wiki page if you like to.


Thanks for the information

T.J. Yang

---- Patrick Nixon  wrote:

Wouldn't it be better on the secondary server(s) to put a conditional
statement in the alert rules that if the primary server is online not
to alert?

I'm pretty sure the rules can handle that.

On Thu, Jul 30, 2009 at 8:32 AM, T.J. Yang wrote:

Date: Thu, 30 Jul 2009 11:15:14 +0100
From: user-c15424b7e83a@xymon.invalid
To: user-ae9b8668bcde@xymon.invalid
Subject: [hobbit] Intelligent alerting across multiple servers?

Hello everyone,

We have 3 Xymon servers communicating together and I have noticed they are
clever enough to communicate enable/disables across the board. For example
if I was to disable a check on the gui from one xymon server it will
communicate this with the other 2 and blue them out automatically.

I have a two servers setting for redunancy.
This is a new feature to me as a hobbit 4.2.0 user, which version are you
using ?

for blue(maintenance mode) records cloning from one server to the other, we
need to dump the blue records on primary and rsync the blue records to
secondary and finally reimport blue records into secondary.

I was wondering if there is something similar for the alerts? We have 3
xymons for redundancy however as far as I can tell if I set up the alerts
config on all 3 then we are going to get 3x identical alerts. Is there any
way they can talk to each other and only send out one?

To avoid alert duplication from secondary servers with hobbit 4.2.0 ,  we
need to write a perl script to check primary from secondary. if primary has
outage of network connection or hobbit server is down then we enable alert
module of secondary hobbbit server. at any given time only one hobbit server
is allowed to send out alerts.

So are you configuring all your hobbit clients to send message to all three
servers ?
This not a concern for a LAN setting but it may become a problem if the
deployment is at the scale of thousands of clients and sending message
across WAN the has smaller wan pipe.


tj

Many thanks
James

Bing™ brings you maps, menus, and reviews organized in one place. Try it
now.

NEW mobile Hotmail. Optimized for YOUR phone.  Click here.
http://windowslive.com/Mobile?ocid=TXT_TAGLM_WL_CS_MB_new_hotmail_072009