hobbitping oddities

7 messages in this thread

list Eric E *hs Schwimmer · Thu, 18 May 2006 10:58:47 -0400 ·

Had a some unexpected behaviour from hobbitping that I thought
I'd let you know about.  When a host goes down, hobbitping
will report it red, but the next poll cycle, it will report
green again.  The next poll cycle it will go red once more
(this time with a message in the conn page that the host
has been down for 3 consecutive polling cyclings), then it
goes green again, etc etc.

This wouldn't be quite so troublesome if it didn't page
us once for every red conn message :)

This is on hobbit 4.1.2p1, and all of our hosts are using
the badconn:1:1:2 flag.

Regards.
-Eric

-----Original Message-----
From: Henrik Stoerner [mailto:user-ce4a2c883f75@xymon.invalid] Sent: Tuesday, May 02, 2006 8:35 AM
To: user-ae9b8668bcde@xymon.invalid
Subject: Re: [hobbit] fping tuning

On Tue, May 02, 2006 at 10:00:25AM +0200, Fr?d?ric Mangeant wrote:

many thanks for writing hobbitping ! It works fine on my test system > (Gentoo Linux x86, glibc 2.3.6, GCC 3.4.6).

Is it / will it be possible to run it on top of a 4.1.2p1 installation ?

Shouldn't be a problem, since it's a very simple stand-alone tool.
Just build it - you've already done that - and copy over the hobbitping
binary to your 4.1.2 installation. Then change the FPING setting in
hobbitserver.cfg to run hobbitping instead of fping.

Make sure you have hobbitping installed suid-root.


Regards,
Henrik

list Henrik Størner · Thu, 18 May 2006 17:06:33 +0200 ·

▸ quoted from Eric E *hs Schwimmer

On Thu, May 18, 2006 at 10:58:47AM -0400, Schwimmer, Eric E *HS wrote:

Had a some unexpected behaviour from hobbitping that I thought
I'd let you know about.  When a host goes down, hobbitping
will report it red, but the next poll cycle, it will report
green again.  The next poll cycle it will go red once more
(this time with a message in the conn page that the host
has been down for 3 consecutive polling cyclings), then it
goes green again, etc etc.

Sounds like it is the re-test thing that is broken. Do you have the
[bbretest] section enabled in hobbitlaunch.cfg ?

▸ quoted from Eric E *hs Schwimmer

This wouldn't be quite so troublesome if it didn't page
us once for every red conn message :)

Ouch. 


Regards,
Henrik

list Eric E *hs Schwimmer · Thu, 18 May 2006 11:36:17 -0400 ·

▸ quoted from Henrik Størner

Sounds like it is the re-test thing that is broken. Do you have the
[bbretest] section enabled in hobbitlaunch.cfg ?

Yep:
[bbretest]
        ENVFILE /usr/local/hobbit/server/etc/hobbitserver.cfg
        NEEDS hobbitd
        CMD $BBHOME/ext/bbretest-net.sh
        LOGFILE $BBSERVERLOGS/bb-retest.log
        INTERVAL 1m

list Eric E *hs Schwimmer · Mon, 22 May 2006 16:57:04 -0400 ·

More info: this doesn't look like its exclusive to hobbitping.
I swapped hobbitping out for fping, and I'm still getting a lot of the red->green->red flapping when a host is unreachable.

-Eric Schwimmer
Network Engineer
UVA HSCS Network Engineering

▸ quoted from Eric E *hs Schwimmer

-----Original Message-----
From: Schwimmer, Eric E *HS [mailto:user-1e1008b069d5@xymon.invalid] Sent: Thursday, May 18, 2006 11:36 AM
To: user-ae9b8668bcde@xymon.invalid
Subject: RE: [hobbit] hobbitping oddities

Sounds like it is the re-test thing that is broken. Do you have the
[bbretest] section enabled in hobbitlaunch.cfg ?

Yep:
[bbretest]
        ENVFILE /usr/local/hobbit/server/etc/hobbitserver.cfg
        NEEDS hobbitd
        CMD $BBHOME/ext/bbretest-net.sh
        LOGFILE $BBSERVERLOGS/bb-retest.log
        INTERVAL 1m

list Henrik Størner · Mon, 22 May 2006 23:10:49 +0200 ·

▸ quoted from Eric E *hs Schwimmer

On Mon, May 22, 2006 at 04:57:04PM -0400, Schwimmer, Eric E *HS wrote:

More info: this doesn't look like its exclusive to hobbitping.
I swapped hobbitping out for fping, and I'm still getting a lot of the red->green->red flapping when a host is unreachable.

Which could point again at a problem with the re-tests.

What interval do you use for the normal tests (i.e. the interval you
run the [bbnet] task?) Is it the default 5 minutes, or have you lowered
it ? 
I'd be interested to know if disabling the [bbretest] task makes this flapping go away.


Regards,
Henrik

▸ quoted from Eric E *hs Schwimmer

-Eric Schwimmer
Network Engineer
UVA HSCS Network Engineering

-----Original Message-----
From: Schwimmer, Eric E *HS [mailto:user-1e1008b069d5@xymon.invalid] > Sent: Thursday, May 18, 2006 11:36 AM
To: user-ae9b8668bcde@xymon.invalid
Subject: RE: [hobbit] hobbitping oddities

Sounds like it is the re-test thing that is broken. Do you have the

[bbretest] section enabled in hobbitlaunch.cfg ?
Yep:

[bbretest]
        ENVFILE /usr/local/hobbit/server/etc/hobbitserver.cfg
        NEEDS hobbitd
        CMD $BBHOME/ext/bbretest-net.sh
        LOGFILE $BBSERVERLOGS/bb-retest.log
        INTERVAL 1m

--


Henrik Storner

list David Gore · Mon, 22 May 2006 21:14:05 +0000 ·

Not that this is your problem, but I had a similar problem when I had two instances of the hobbit server running on two different boxes.  I had copied the original server without modifying all of the configs on the new server.  In effect the new instance was sending it's data to the original server causing the ping checks to go red every 5 minutes (just briefly).  Perhaps that will give you some ideas?

▸ quoted from Henrik Størner


Schwimmer, Eric E *HS wrote:

More info: this doesn't look like its exclusive to hobbitping.
I swapped hobbitping out for fping, and I'm still getting a lot of the red->green->red flapping when a host is unreachable.

-Eric Schwimmer
Network Engineer
UVA HSCS Network Engineering

-----Original Message-----
From: Schwimmer, Eric E *HS [mailto:user-1e1008b069d5@xymon.invalid] Sent: Thursday, May 18, 2006 11:36 AM
To: user-ae9b8668bcde@xymon.invalid
Subject: RE: [hobbit] hobbitping oddities

Sounds like it is the re-test thing that is broken. Do you have the
[bbretest] section enabled in hobbitlaunch.cfg ?

Yep:
[bbretest]
        ENVFILE /usr/local/hobbit/server/etc/hobbitserver.cfg
        NEEDS hobbitd
        CMD $BBHOME/ext/bbretest-net.sh
        LOGFILE $BBSERVERLOGS/bb-retest.log
        INTERVAL 1m

list Eric E *hs Schwimmer · Tue, 23 May 2006 09:03:34 -0400 ·

We did indeed lower the test interval to 1 minute.  I'll try disabling
the bbretest stanza and let you know how that goes.

▸ quoted from David Gore


-Eric Schwimmer
Network Engineer
UVA HSCS Network Engineering

-----Original Message-----
From: Henrik Stoerner [mailto:user-ce4a2c883f75@xymon.invalid] Sent: Monday, May 22, 2006 5:11 PM
To: user-ae9b8668bcde@xymon.invalid
Subject: Re: [hobbit] hobbitping oddities

On Mon, May 22, 2006 at 04:57:04PM -0400, Schwimmer, Eric E *HS wrote:

More info: this doesn't look like its exclusive to hobbitping.

I swapped hobbitping out for fping, and I'm still getting a lot of > the red->green->red flapping when a host is unreachable.

Which could point again at a problem with the re-tests.

What interval do you use for the normal tests (i.e. the interval you
run the [bbnet] task?) Is it the default 5 minutes, or have you lowered
it ? 
I'd be interested to know if disabling the [bbretest] task makes this flapping go away.


Regards,
Henrik

-Eric Schwimmer

Network Engineer
UVA HSCS Network Engineering  > > > -----Original Message-----

From: Schwimmer, Eric E *HS [mailto:user-1e1008b069d5@xymon.invalid] > > Sent: Thursday, May 18, 2006 11:36 AM
To: user-ae9b8668bcde@xymon.invalid
Subject: RE: [hobbit] hobbitping oddities

Sounds like it is the re-test thing that is broken. Do you have the

[bbretest] section enabled in hobbitlaunch.cfg ?

Yep:

[bbretest]
        ENVFILE /usr/local/hobbit/server/etc/hobbitserver.cfg
        NEEDS hobbitd
        CMD $BBHOME/ext/bbretest-net.sh
        LOGFILE $BBSERVERLOGS/bb-retest.log
        INTERVAL 1m

-- 
Henrik Storner

hobbitping oddities 🔗 link

hobbitping oddities