Differences between bb and hobbit -
list Joe Sloan
Hello list, I've gotten a pair of hobbit servers set up, one in each data center, in a specific lan. I'm noticing that, due to the many transient network glitches in wan connectivity, that the hobbit history shows a lot of red/green/red/green. This appears to be different behavior from bb, which gives us some knobs to turn, mitigating the effects of the transient glitches and preventing the alternating red/green connectivity status. (We've got people watching bbdisplay pages, who panic when they see red.) If a bb-net test fails, bb waits "n" seconds and tries again, and if it still fails, it waits "m" seconds and tries a third time, and if that fails as well, the connectivity goes red. The delays between try 1 and try 2, and the delay between try 2 and try 3, are configurable in bb, but I don't see any such option in hobbit, nor is it obvious whether there is more than 1 attempt made to perform a network test. Can anyone confirm or deny that this is the case, and if not, point me to the correct configuration option? Thanks & Regards, Joe
list Larry Barber
Look up "badhttp" in the bb-hosts man pages. Thanks, Larry Barber
▸
On 11/5/07, Sloan <user-b1d2c84d244b@xymon.invalid> wrote:Hello list, I've gotten a pair of hobbit servers set up, one in each data center, in a specific lan. I'm noticing that, due to the many transient network glitches in wan connectivity, that the hobbit history shows a lot of red/green/red/green. This appears to be different behavior from bb, which gives us some knobs to turn, mitigating the effects of the transient glitches and preventing the alternating red/green connectivity status. (We've got people watching bbdisplay pages, who panic when they see red.) If a bb-net test fails, bb waits "n" seconds and tries again, and if it still fails, it waits "m" seconds and tries a third time, and if that fails as well, the connectivity goes red. The delays between try 1 and try 2, and the delay between try 2 and try 3, are configurable in bb, but I don't see any such option in hobbit, nor is it obvious whether there is more than 1 attempt made to perform a network test. Can anyone confirm or deny that this is the case, and if not, point me to the correct configuration option? Thanks & Regards, Joe
list Henrik Størner
▸
On Mon, Nov 05, 2007 at 04:26:17PM -0800, Sloan wrote:
I'm noticing that, due to the many transient network glitches in wan connectivity, that the hobbit history shows a lot of red/green/red/green. This appears to be different behavior from bb, which gives us some knobs to turn, mitigating the effects of the transient glitches and preventing the alternating red/green connectivity status. (We've got people watching bbdisplay pages, who panic when they see red.)
Hobbit uses either "hobbitping" or "fping" to do the network tests. Both
have commandline options that you can use to make them more tolerant of
brief network outages, see their man-pages.
If that is not sufficient (ie the transient glitches last more than 30
seconds), then you can use the "badconn:A:B:C" setting in the bb-hosts
file to delay when the status goes red. From the bb-hosts man-page:
Normally when a network test fails, the status changes to
red immediately. With a "badTEST:x:y:z" tag this behaviour
changes:
* While "z" or more successive tests fail, the column goes RED.
* While "y" or more successive tests fail, but fewer than "z",
the column goes YELLOW.
* While "x" or more successive tests fail, but fewer than "y",
the column goes CLEAR.
If you're monitoring hosts behind these unstable WAN links, you may also
want to look at the "depends" tag so you won't generate alerts on the
hosts when the WAN link to them is down.
Regards,
Henrik
list Joe Sloan
▸
Henrik Stoerner wrote:
Hobbit uses either "hobbitping" or "fping" to do the network tests. Both
have commandline options that you can use to make them more tolerant of
brief network outages, see their man-pages.
If that is not sufficient (ie the transient glitches last more than 30
seconds), then you can use the "badconn:A:B:C" setting in the bb-hosts
file to delay when the status goes red. From the bb-hosts man-page:
Normally when a network test fails, the status changes to
red immediately. With a "badTEST:x:y:z" tag this behaviour
changes:
* While "z" or more successive tests fail, the column goes RED.
* While "y" or more successive tests fail, but fewer than "z",
the column goes YELLOW.
* While "x" or more successive tests fail, but fewer than "y",
the column goes CLEAR.
If you're monitoring hosts behind these unstable WAN links, you may also
want to look at the "depends" tag so you won't generate alerts on the
hosts when the WAN link to them is down.
Excellent, this is exactly what I needed, thanks for helping a hobbit noob find the way. Joe
list Joe Sloan
Excellent, that will do it. Joe
▸
Larry Barber wrote:Look up "badhttp" in the bb-hosts man pages.
Thanks,
Larry Barber
On 11/5/07, *Sloan* <user-b1d2c84d244b@xymon.invalid <mailto:user-b1d2c84d244b@xymon.invalid>> wrote:
Hello list,
I've gotten a pair of hobbit servers set up, one in each data
center, in
a specific lan.
I'm noticing that, due to the many transient network glitches in wan
connectivity, that the hobbit history shows a lot of
red/green/red/green. This appears to be different behavior from bb,
which gives us some knobs to turn, mitigating the effects of the
transient glitches and preventing the alternating red/green
connectivity
status. (We've got people watching bbdisplay pages, who panic when
they
see red.)
If a bb-net test fails, bb waits "n" seconds and tries again, and
if it
still fails, it waits "m" seconds and tries a third time, and if that
fails as well, the connectivity goes red. The delays between try 1
and
try 2, and the delay between try 2 and try 3, are configurable in bb,
but I don't see any such option in hobbit, nor is it obvious whether
there is more than 1 attempt made to perform a network test.
Can anyone confirm or deny that this is the case, and if not, point me
to the correct configuration option?
Thanks & Regards,
Joe