How do I clone maintenance mode record ?

11 messages in this thread

list T.J. Yang · Thu, 9 Jul 2009 12:04:52 -0500 ·


I am rsyncing configuration files from hobbit1( primary) to hobbit2 (standby) server.

But without maintenance mode records of hobbit1 copy over, hobbit2 will generate alerts for hosts that had been put into maintenance mode already.

Currently I tried to rsync "data/disabled" directory and restart hobbit2 server but the maintenace web page still show less hosts on hobbit2.

Anyone know where is the maintenance records store at beside "data/disabled" ?


T.J. Yang


Insert movie times and more without leaving Hotmail®. 
http://windowslive.com/Tutorial/Hotmail/QuickAdd?ocid=TXT_TAGLM_WL_HM_Tutorial_QuickAdd_062009

list Alan Sparks · Thu, 09 Jul 2009 11:10:23 -0600 ·

▸ quoted from T.J. Yang

T.J. Yang wrote:

I am rsyncing configuration files from hobbit1( primary) to hobbit2 (standby) server.

But without maintenance mode records of hobbit1 copy over, hobbit2 will generate alerts for hosts that had been put into maintenance mode already.

Currently I tried to rsync "data/disabled" directory and restart hobbit2 server but the maintenace web page still show less hosts on hobbit2.

Anyone know where is the maintenance records store at beside "data/disabled" ?

I recall, from a code browse some time ago, that the alerts module used
the acknowledge.log in deciding whether alerts had been sent -- you may
need to copy that over.  I thought it odd, having that in the logs/
directory but used as a data file - but you might look at that.
-Alan

list Paul Root · Thu, 9 Jul 2009 13:22:13 -0500 ·

I have a separate hobbit-alerts.cfg file for the secondary. It simply monitors the primary, and if the primary goes down, runs a script to copy the primaries -alerts file into place.
Then it has a recovery script to turn itself back off. 

The primary rdists the pertinent files over nightly.
 

Paul Root         IM/MNS Infrastructure

▸ quoted from Alan Sparks

-----Original Message-----
From: Alan Sparks [mailto:user-8f2174fd8b66@xymon.invalid] 
Sent: Thursday, July 09, 2009 12:10 PM
To: user-ae9b8668bcde@xymon.invalid
Subject: Re: [hobbit] How do I clone maintenance mode record ?

T.J. Yang wrote:

I am rsyncing configuration files from hobbit1( primary) to hobbit2 (standby) server.

But without maintenance mode records of hobbit1 copy over, hobbit2 will generate alerts for hosts that had been put into maintenance mode already.

Currently I tried to rsync "data/disabled" directory and restart hobbit2 server but the maintenace web page still show less hosts on hobbit2.

Anyone know where is the maintenance records store at beside "data/disabled" ?

I recall, from a code browse some time ago, that the alerts module used
the acknowledge.log in deciding whether alerts had been sent -- you may
need to copy that over.  I thought it odd, having that in the logs/
directory but used as a data file - but you might look at that.
-Alan

list T.J. Yang · Thu, 9 Jul 2009 13:27:13 -0500 ·

▸ quoted from Paul Root

Date: Thu, 9 Jul 2009 11:10:23 -0600
From: user-8f2174fd8b66@xymon.invalid
To: user-ae9b8668bcde@xymon.invalid
Subject: Re: [hobbit] How do I clone maintenance mode record ?

T.J. Yang wrote:

I am rsyncing configuration files from hobbit1( primary) to hobbit2 (standby) server.

But without maintenance mode records of hobbit1 copy over, hobbit2 will generate alerts for hosts that had been put into maintenance mode already.

Currently I tried to rsync "data/disabled" directory and restart hobbit2 server but the maintenace web page still show less hosts on hobbit2.

Anyone know where is the maintenance records store at beside "data/disabled" ?

I recall, from a code browse some time ago, that the alerts module used
the acknowledge.log in deciding whether alerts had been sent -- you may
need to copy that over. I thought it odd, having that in the logs/
directory but used as a data file - but you might look at that.

Code tracing,

F1: hobbit-enadis.sh
  -> F2:  hobbit-enadis.c

Turns out hobbit-enadis.c is just an front end to issue "bb disable/enabled ..." command.

so the question is now turn into how hobbit daemon perform disabling and writing logs. 

F1: http://hobbitmon.svn.sourceforge.net/viewvc/hobbitmon/branches/4.2.0/web/hobbit-enadis.sh.DIST?revision=5935&view=markup
F2: http://hobbitmon.svn.sourceforge.net/viewvc/hobbitmon/branches/4.2.0/web/hobbit-enadis.c?revision=5944&view=markup

tj

-Alan

Hotmail® has ever-growing storage! Don’t worry about storage limits. 
http://windowslive.com/Tutorial/Hotmail/Storage?ocid=TXT_TAGLM_WL_HM_Tutorial_Storage_062009

list T.J. Yang · Thu, 9 Jul 2009 13:34:13 -0500 ·

▸ quoted from Paul Root

From: user-c80045f511e8@xymon.invalid
To: user-ae9b8668bcde@xymon.invalid
Date: Thu, 9 Jul 2009 13:22:13 -0500
Subject: RE: [hobbit] How do I clone maintenance mode record ?

I have a separate hobbit-alerts.cfg file for the secondary. It simply monitors the primary, and if the primary goes down, runs a script to copy the primaries -alerts file into place.
Then it has a recovery script to turn itself back off.

we use cron job to pull selected bb-host*,hobbit-alert*,hobbit-data and others from hobbit1 to hobbit2 using rsync.
and with hobbit server side module to check health of hobbit1, if hobbit1 is bad hobbit2 will enable its bbpage worker module to assume the failed hobbit1 alert service.

Two issues remains.

1. maintenance records are not failed over yet.thus hobbit2 send out false alerts.
2. rrd files on hobbit1 need to be patched with rrd data from hobbit2 for hobbit1's down period.


tj

▸ quoted from T.J. Yang

The primary rdists the pertinent files over nightly.


Paul Root IM/MNS Infrastructure
-----Original Message-----
From: Alan Sparks [mailto:user-8f2174fd8b66@xymon.invalid]
Sent: Thursday, July 09, 2009 12:10 PM
To: user-ae9b8668bcde@xymon.invalid
Subject: Re: [hobbit] How do I clone maintenance mode record ?

T.J. Yang wrote:

I am rsyncing configuration files from hobbit1( primary) to hobbit2 (standby) server.

But without maintenance mode records of hobbit1 copy over, hobbit2 will generate alerts for hosts that had been put into maintenance mode already.

Currently I tried to rsync "data/disabled" directory and restart hobbit2 server but the maintenace web page still show less hosts on hobbit2.

Anyone know where is the maintenance records store at beside "data/disabled" ?

I recall, from a code browse some time ago, that the alerts module used
the acknowledge.log in deciding whether alerts had been sent -- you may
need to copy that over. I thought it odd, having that in the logs/
directory but used as a data file - but you might look at that.
-Alan

Lauren found her dream laptop. Find the PC that’s right for you.
http://www.microsoft.com/windows/choosepc/?ocid=ftp_val_wl_290

list Ralph Mitchell · Fri, 10 Jul 2009 01:00:38 -0500 ·

I think maybe the server keeps the disabled status in memory, so simply
copying files just isn't going to do it.  This should do the trick:
===== server/ext/getblues.sh =====
      #!/bin/bash

      . /home/xymon/server/etc/hobbitserver.cfg

      $BB localhost "hobbitdboard color=blue
fields=hostname,testname,disabletime,dismsg" | $SED -e 's/\\/\\\\/g'
===== end of file =====

===== server/ext/putblues.sh =====
       #!/bin/bash

       . /home/xymon/server/etc/hobbitserver.cfg

       cat $1 | while read line
       do
          OFS="$IFS"
          IFS="|"
          set $line
          IFS="$OFS"
          x=`echo "$4" | $SED -e 's/\\\n/\n/g'`
          $BB localhost "disable $1.$2 $3 $x"
       done
===== end of file =====

1) on the primary, whenever the other files are copied, run this and copy
the file:

      server/ext/getblues.sh > server/tmp/blues.out

2) whenever the secondary needs to take over, have it run the other script:

     server/etc/putblues.sh server/tmp/blues.out

That should load up the disabled status messages into the secondary.

NB: I've only tested this on a single server, making a slight change to the
messages in the blues.out file before reloading it.  The change loads up
just fine, so it would *probably* work on a different server.  YMMV  :)

Ralph Mitchell

▸ quoted from T.J. Yang



On Thu, Jul 9, 2009 at 1:34 PM, T.J. Yang <user-8e841282cda5@xymon.invalid> wrote:

From: user-c80045f511e8@xymon.invalid
To: user-ae9b8668bcde@xymon.invalid
Date: Thu, 9 Jul 2009 13:22:13 -0500
Subject: RE: [hobbit] How do I clone maintenance mode record ?

I have a separate hobbit-alerts.cfg file for the secondary. It simply
monitors the primary, and if the primary goes down, runs a script to copy
the primaries -alerts file into place.
Then it has a recovery script to turn itself back off.

we use cron job to pull selected bb-host*,hobbit-alert*,hobbit-data and
others from hobbit1 to hobbit2 using rsync.
and with hobbit server side module to check health of hobbit1, if hobbit1
is bad hobbit2 will enable its bbpage worker module to assume the failed
hobbit1 alert service.

Two issues remains.

1. maintenance records are not failed over yet.thus hobbit2 send out false
alerts.
2. rrd files on hobbit1 need to be patched with rrd data from hobbit2 for
hobbit1's down period.


tj

The primary rdists the pertinent files over nightly.


Paul Root IM/MNS Infrastructure
-----Original Message-----
From: Alan Sparks [mailto:user-8f2174fd8b66@xymon.invalid]
Sent: Thursday, July 09, 2009 12:10 PM
To: user-ae9b8668bcde@xymon.invalid
Subject: Re: [hobbit] How do I clone maintenance mode record ?

T.J. Yang wrote:

I am rsyncing configuration files from hobbit1( primary) to hobbit2

(standby) server.

But without maintenance mode records of hobbit1 copy over, hobbit2 will
generate alerts for hosts that had been put into maintenance mode already.

Currently I tried to rsync "data/disabled" directory and restart hobbit2
server but the maintenace web page still show less hosts on hobbit2.

Anyone know where is the maintenance records store at beside

"data/disabled" ?

I recall, from a code browse some time ago, that the alerts module used
the acknowledge.log in deciding whether alerts had been sent -- you may
need to copy that over. I thought it odd, having that in the logs/
directory but used as a data file - but you might look at that.

list Dominique Frise · Fri, 10 Jul 2009 08:29:55 +0200 ·

Xymon does not use "data/disabled" dir.

 From hobbitlaunch.cfg:

# "bbenadis" updates the files used to indicate that a host or test has been enabled or disabled.
# Unless you are running server side scripts that look at these files, you do not need this.

All Xymon status are available by querying the hobbitdboard. See the man page of bb(1)


Dominique

▸ quoted from T.J. Yang


T.J. Yang wrote:


I am rsyncing configuration files from hobbit1( primary) to hobbit2 (standby) server.

But without maintenance mode records of hobbit1 copy over, hobbit2 will generate alerts for hosts that had been put into maintenance mode already.

Currently I tried to rsync "data/disabled" directory and restart hobbit2 server but the maintenace web page still show less hosts on hobbit2.

Anyone know where is the maintenance records store at beside "data/disabled" ?


T.J. Yang


Insert movie times and more without leaving Hotmail®. http://windowslive.com/Tutorial/Hotmail/QuickAdd?ocid=TXT_TAGLM_WL_HM_Tutorial_QuickAdd_062009

list David Peters · Fri, 10 Jul 2009 18:58:37 +1000 ·

I have a script that sync enabled and disabled between a primary and DR
hobbit server if you are interested.

▸ quoted from Dominique Frise

-----Original Message-----
From: Dominique Frise [mailto:user-78ab6673b600@xymon.invalid] Sent: Friday, 10 July 2009 4:30 PM
To: user-ae9b8668bcde@xymon.invalid
Subject: Re: [hobbit] How do I clone maintenance mode record ?

Xymon does not use "data/disabled" dir.

 From hobbitlaunch.cfg:

# "bbenadis" updates the files used to indicate that a host or test has been enabled or disabled.
# Unless you are running server side scripts that look at these files, you do not need this.

All Xymon status are available by querying the hobbitdboard. See the man page of bb(1)

Dominique

T.J. Yang wrote:


I am rsyncing configuration files from hobbit1( primary) to hobbit2

(standby) server.

But without maintenance mode records of hobbit1 copy over, hobbit2 will
generate alerts for hosts that had been put into maintenance mode already.

Currently I tried to rsync "data/disabled" directory and restart hobbit2
server but the maintenace web page still show less hosts on hobbit2.

Anyone know where is the maintenance records store at beside

"data/disabled" ?


T.J. Yang


Insert movie times and more without leaving HotmailR.

http://windowslive.com/Tutorial/Hotmail/QuickAdd?ocid=TXT_TAGLM_WL_HM_Tutori
al_QuickAdd_062009

list T.J. Yang · Fri, 10 Jul 2009 08:19:22 -0500 ·

▸ quoted from David Peters

From: user-66917457e380@xymon.invalid
To: user-ae9b8668bcde@xymon.invalid
Date: Fri, 10 Jul 2009 18:58:37 +1000
Subject: RE: [hobbit] How do I clone maintenance mode record ?

I have a script that sync enabled and disabled between a primary and DR
hobbit server if you are interested.


In Ralph Mitchell (user-00a5e44c48c0@xymon.invalid)
                reply email,
There are getblue.sh and putblue.sh to do the collecting on primary and populating on DR/standby hobbit server.

I tired it but I found the putblue.sh is not working. I still can't see the same list of maintenance hosts between the twos, even the get and put blue shell ran sucessfully.

This may caused by lacking of running bbpage alert module on DR/Secondary module. Is your DR hobbit server has alert module disabled and your script still works ? 

Would you mind to send to the list or send it to my email account ? I like to learn how you did it.


tj

▸ quoted from Dominique Frise

-----Original Message-----
From: Dominique Frise [mailto:user-78ab6673b600@xymon.invalid]
Sent: Friday, 10 July 2009 4:30 PM
To: user-ae9b8668bcde@xymon.invalid
Subject: Re: [hobbit] How do I clone maintenance mode record ?

Xymon does not use "data/disabled" dir.

From hobbitlaunch.cfg:

# "bbenadis" updates the files used to indicate that a host or test has
been enabled or disabled.
# Unless you are running server side scripts that look at these files,
you do not need this.

All Xymon status are available by querying the hobbitdboard. See the man
page of bb(1)

Dominique

T.J. Yang wrote:


I am rsyncing configuration files from hobbit1( primary) to hobbit2

(standby) server.

But without maintenance mode records of hobbit1 copy over, hobbit2 will
generate alerts for hosts that had been put into maintenance mode already.

Currently I tried to rsync "data/disabled" directory and restart hobbit2
server but the maintenace web page still show less hosts on hobbit2.

Anyone know where is the maintenance records store at beside

"data/disabled" ?


T.J. Yang


Insert movie times and more without leaving HotmailR.

http://windowslive.com/Tutorial/Hotmail/QuickAdd?ocid=TXT_TAGLM_WL_HM_Tutori
al_QuickAdd_062009

Insert movie times and more without leaving Hotmail®. 
http://windowslive.com/Tutorial/Hotmail/QuickAdd?ocid=TXT_TAGLM_WL_HM_Tutorial_QuickAdd_062009

list Ralph Mitchell · Sat, 11 Jul 2009 00:52:29 -0500 ·

▸ quoted from T.J. Yang

On Fri, Jul 10, 2009 at 8:19 AM, T.J. Yang <user-8e841282cda5@xymon.invalid> wrote:

In Ralph Mitchell (user-00a5e44c48c0@xymon.invalid)
               reply email,
There are getblue.sh and putblue.sh to do the collecting on primary and
populating on DR/standby hobbit server.

I tired it but I found the putblue.sh is not working. I still can't see the
same list of maintenance hosts between the twos, even the get and put blue
shell ran sucessfully.

This may caused by lacking of running bbpage alert module on DR/Secondary
module. Is your DR hobbit server has alert module disabled and your script
still works ?

Would you mind to send to the list or send it to my email account ? I like
to learn how you did it.


As I said, I only tried it in Ubuntu, so I would expect it to work in any
Linux distribution.  I'm still setting up other servers in VMware for
fooling around with this kind of thing.  What OS do you have??

Ralph Mitchell

list Ralph Mitchell · Sun, 12 Jul 2009 01:32:39 -0500 ·

I realised earlier that the disabletime value coming from hobbitdboard is
actually a unix timestamp, but the disable command wants a duration in the
form of seconds, minutes or hours.  Again, this works with Ubuntu, and
probably any other Linux, but maybe not in a different Unix.  It relies on
the GNU date command, for a start...  :)

========== putblues.sh ==========
     #!/bin/sh

▸ quoted from Ralph Mitchell


     . /home/xymon/server/etc/hobbitserver.cfg

     cat $1 | while read line
     do
       OFS="$IFS"
       IFS="|"
       set $line
       IFS="$OFS"


       if [ "$3" -eq "-1" ]; then
         # found "disable until OK"
         lifetime="-1"
       else
         now=`date +%s`
         lifetime=`$EXPR $3 - $now`
         if [ "$lifetime" -le "0" ]; then
           # this one expired, ignore it
           continue
         else
           lifetime="$lifetime"s
         fi
       fi
       msg=`echo "$4" | $SED -e 's/\\\n/\n/g'`
       $BB localhost "disable $1.$2 $lifetime $msg"
     done
========== cut here ==========

Ralph Mitchell

▸ quoted from Ralph Mitchell



On Sat, Jul 11, 2009 at 12:52 AM, Ralph Mitchell <user-00a5e44c48c0@xymon.invalid>wrote:

On Fri, Jul 10, 2009 at 8:19 AM, T.J. Yang <user-8e841282cda5@xymon.invalid> wrote:

In Ralph Mitchell (user-00a5e44c48c0@xymon.invalid)
               reply email,
There are getblue.sh and putblue.sh to do the collecting on primary and
populating on DR/standby hobbit server.

I tired it but I found the putblue.sh is not working. I still can't see
the same list of maintenance hosts between the twos, even the get and put
blue shell ran sucessfully.

This may caused by lacking of running bbpage alert module on DR/Secondary
module. Is your DR hobbit server has alert module disabled and your script
still works ?

Would you mind to send to the list or send it to my email account ? I like
to learn how you did it.


As I said, I only tried it in Ubuntu, so I would expect it to work in any
Linux distribution.  I'm still setting up other servers in VMware for
fooling around with this kind of thing.  What OS do you have??

Ralph Mitchell

How do I clone maintenance mode record ? 🔗 link

How do I clone maintenance mode record ?