Problems with hobbitping when the Network is unreachable (BUG)
list Sebastian Auriol
Hi Henrik, I rebooted my hobbit server and it lost its default gateway, unbeknown to me. The only symptoms of something wrong was a purple in a network test is was supposed to be doing and a purple in the server's bbtest column. Now, I reckon one or other or both of these should have both gone red and not purple, and the reason that they did not is due to hobbitping going into a spin: what looks like a fairly tight loop. Hobbitping was using quite a lot of CPU (I saw it at well over 30%) and server/tmp/ping-stdout.1407 was empty but server/tmp/ping-stderr.5499 was getting many (about 60) "Failed to send ICMP packet: Network is unreachable" written to it every second. I reckon this message should have been in the report of the connection test it was trying to do (although you could argue that the colour should be clear and not red, since one doesn't want dozens of alarms for a problem that is actually on the network-test server). It took me some time to figure out what was wrong because hobbit didn't report this problem in any of its logs (presumably because hobbitping never returned and so bbtest-net never returned its status) (and hobbitd_client stopped working for some other reason (core dump) yesterday and, although restarted, remains not reporting with nothing in the logs). Kind regards, Sebastian
list Sebastian Auriol
Forgot to mention that I am using the latest version of hobbitping (snapshot from 18-Mar-08). From: Sebastian [mailto:user-7b2156f36779@xymon.invalid] Sent: 22 May 2008 19:08 To: user-ae9b8668bcde@xymon.invalid Subject: [hobbit] Problems with hobbitping when the Network is unreachable (BUG)
▸
Hi Henrik,
I rebooted my hobbit server and it lost its default gateway, unbeknown to
me. The only symptoms of something wrong was a purple in a network test is
was supposed to be doing and a purple in the server's bbtest column. Now, I
reckon one or other or both of these should have both gone red and not
purple, and the reason that they did not is due to hobbitping going into a
spin: what looks like a fairly tight loop. Hobbitping was using quite a
lot of CPU (I saw it at well over 30%) and server/tmp/ping-stdout.1407 was
empty but server/tmp/ping-stderr.5499 was getting many (about 60) "Failed to
send ICMP packet: Network is unreachable" written to it every second. I
reckon this message should have been in the report of the connection test it
was trying to do (although you could argue that the colour should be clear
and not red, since one doesn't want dozens of alarms for a problem that is
actually on the network-test server). It took me some time to figure out
what was wrong because hobbit didn't report this problem in any of its logs
(presumably because hobbitping never returned and so bbtest-net never
returned its status) (and hobbitd_client stopped working for some other
reason (core dump) yesterday and, although restarted, remains not reporting
with nothing in the logs).
Kind regards,
Sebastian
list Greg L Hubbard
Try fping. I think Henrik has not had time to fix problems with hobbitping. It may be one of those "it seemed a good idea at the time" things... GLH
▸
From: Sebastian [mailto:user-7b2156f36779@xymon.invalid]
Sent: Thursday, May 22, 2008 1:08 PM
To: user-ae9b8668bcde@xymon.invalid
Subject: [hobbit] Problems with hobbitping when the Network is
unreachable (BUG)
Hi Henrik,
I rebooted my hobbit server and it lost its default gateway,
unbeknown to me. The only symptoms of something wrong was a purple in a
network test is was supposed to be doing and a purple in the server's
bbtest column. Now, I reckon one or other or both of these should have
both gone red and not purple, and the reason that they did not is due to
hobbitping going into a spin: what looks like a fairly tight loop.
Hobbitping was using quite a lot of CPU (I saw it at well over 30%) and
server/tmp/ping-stdout.1407 was empty but server/tmp/ping-stderr.5499
was getting many (about 60) "Failed to send ICMP packet: Network is
unreachable" written to it every second. I reckon this message should
have been in the report of the connection test it was trying to do
(although you could argue that the colour should be clear and not red,
since one doesn't want dozens of alarms for a problem that is actually
on the network-test server). It took me some time to figure out what
was wrong because hobbit didn't report this problem in any of its logs
(presumably because hobbitping never returned and so bbtest-net never
returned its status) (and hobbitd_client stopped working for some other
reason (core dump) yesterday and, although restarted, remains not
reporting with nothing in the logs).
Kind regards,
Sebastian
list Henrik Størner
▸
On Thu, May 22, 2008 at 07:07:42PM +0100, Sebastian wrote:
I rebooted my hobbit server and it lost its default gateway, unbeknown to me. The only symptoms of something wrong was a purple in a network test is was supposed to be doing and a purple in the server's bbtest column. Now, I reckon one or other or both of these should have both gone red and not purple, and the reason that they did not is due to hobbitping going into a spin: what looks like a fairly tight loop. Hobbitping was using quite a lot of CPU (I saw it at well over 30%) and server/tmp/ping-stdout.1407 was empty but server/tmp/ping-stderr.5499 was getting many (about 60) "Failed to send ICMP packet: Network is unreachable" written to it every second.
Thanks, the attached patch should solve this problem. Regards, Henrik
Attachments (1)
list Sebastian Auriol
Tested and it does solve the problem. (Network test goes red with: Service conn on XXXGW1 is not OK : Host responds to ping on 2 of 7 IP's ... Traceroute results: socket: Network is unreachable ) Thanks for the quick patch. Sebastian
▸
-----Original Message----- From: Henrik Stoerner [mailto:user-ce4a2c883f75@xymon.invalid] Sent: 23 May 2008 09:49 To: user-ae9b8668bcde@xymon.invalid Subject: Re: [hobbit] Problems with hobbitping when the Network isunreachable (BUG) On Thu, May 22, 2008 at 07:07:42PM +0100, Sebastian wrote:I rebooted my hobbit server and it lost its default gateway, unbeknown to me. The only symptoms of something wrong was a purple in a network test is was supposed to be doing and a purple in the server's bbtest column. Now, I reckon one or other or both of these should have both gone red and not purple, and the reason that they did not is due to hobbitping going into a spin: what looks like a fairly tight loop. Hobbitping was using quite a lot of CPU (I saw it at well over 30%) and server/tmp/ping-stdout.1407 was empty but server/tmp/ping-stderr.5499 was getting many (about 60) "Failed to send ICMP packet: Network is unreachable" written to it every second.Thanks, the attached patch should solve this problem. Regards, Henrik