Strange pinging issue
list Shawn Maschino
Hi all, we are seeing a strange fping issue with Hobbit and I was
wondering if anyone may have seen the same.
We have changed the IP addresses on a large number of systems we are
monitoring due to moving them to a new location. We updated the
bb-hosts file to have the new IPs, and DNS was also updated. However
Hobbit occasionally falls back to testing the old IP for no apparent
reason.
This happens quite regularly as well, as seen here:
Fri Nov 30 10:56:08 2007
<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVIC
E=conn&TIMEBUF=Fri_Nov_30_10:56:08_2007> 0:04:19
Fri Nov 30 10:48:31 2007
<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVIC
E=conn&TIMEBUF=Fri_Nov_30_10:48:31_2007> 0:07:37
Fri Nov 30 10:48:08 2007
<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVIC
E=conn&TIMEBUF=Fri_Nov_30_10:48:08_2007> 0:00:23
Fri Nov 30 10:42:27 2007
<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVIC
E=conn&TIMEBUF=Fri_Nov_30_10:42:27_2007> 0:05:41
Fri Nov 30 10:39:53 2007
<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVIC
E=conn&TIMEBUF=Fri_Nov_30_10:39:53_2007> 0:02:34
Fri Nov 30 10:36:49 2007
<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVIC
E=conn&TIMEBUF=Fri_Nov_30_10:36:49_2007> 0:03:04
Fri Nov 30 10:31:57 2007
<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVIC
E=conn&TIMEBUF=Fri_Nov_30_10:31:57_2007> 0:04:52
Fri Nov 30 10:25:27 2007
<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVIC
E=conn&TIMEBUF=Fri_Nov_30_10:25:27_2007> 0:06:30
Fri Nov 30 10:23:58 2007
<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVIC
E=conn&TIMEBUF=Fri_Nov_30_10:23:58_2007> 0:01:29
Fri Nov 30 10:19:19 2007
<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVIC
E=conn&TIMEBUF=Fri_Nov_30_10:19:19_2007> 0:04:39
Fri Nov 30 10:16:26 2007
<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVIC
E=conn&TIMEBUF=Fri_Nov_30_10:16:26_2007> 0:02:53
The conn status page when green shows the correct IP address, and
when red shows the wrong (old) IP address. Nothing in the bb-hosts or
DNS change during this time.
I ran a "fping -AeC 1000 -i 5000 <host>" from the command line to
see if fping was switching IPs, and it does not seem to be, even when
Hobbit shows red, the fping from the command line still his the
new/current IP.
I've tried forcing a "testip" in bb-hosts to see if DNS was
involved, and with that forced it still changes in Hobbit.
Any suggestions would be much appreciated.
Thanks,
Shawn
list Josh Luthman
I get a very similar situation, of which I was just about to report! Mine will be green for hours or days, then show up red for much less time (5-45 seconds). I've mine set as default.. FPING="/usr/sbin/fping" I have testip on almost every single host. On 11/30/07, Maschino, Shawn (SABIC Innovative Plastics) <
▸
user-6359c5a3b499@xymon.invalid> wrote:
Hi all, we are seeing a strange fping issue with Hobbit and I was
wondering if anyone may have seen the same.
We have changed the IP addresses on a large number of systems we are
monitoring due to moving them to a new location. We updated the bb-hosts
file to have the new IPs, and DNS was also updated. However Hobbit
occasionally falls back to testing the old IP for no apparent reason.
This happens quite regularly as well, as seen here:
Fri Nov 30 10:56:08 2007 [image: red]<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=conn&TIMEBUF=Fri_Nov_30_10:56:08_2007>; 0:04:19 Fri Nov 30 10:48:31 2007 [image: green]<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=conn&TIMEBUF=Fri_Nov_30_10:48:31_2007>; 0:07:37 Fri Nov 30 10:48:08 2007 [image: red]<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=conn&TIMEBUF=Fri_Nov_30_10:48:08_2007>; 0:00:23 Fri Nov 30 10:42:27 2007 [image: green]<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=conn&TIMEBUF=Fri_Nov_30_10:42:27_2007>; 0:05:41 Fri Nov 30 10:39:53 2007 [image: red]<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=conn&TIMEBUF=Fri_Nov_30_10:39:53_2007>; 0:02:34 Fri Nov 30 10:36:49 2007 [image: green]<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=conn&TIMEBUF=Fri_Nov_30_10:36:49_2007>; 0:03:04 Fri Nov 30 10:31:57 2007 [image: red]<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=conn&TIMEBUF=Fri_Nov_30_10:31:57_2007>; 0:04:52 Fri Nov 30 10:25:27 2007 [image: green]<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=conn&TIMEBUF=Fri_Nov_30_10:25:27_2007>; 0:06:30 Fri Nov 30 10:23:58 2007 [image: red]<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=conn&TIMEBUF=Fri_Nov_30_10:23:58_2007>; 0:01:29 Fri Nov 30 10:19:19 2007 [image: green]<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=conn&TIMEBUF=Fri_Nov_30_10:19:19_2007>; 0:04:39 Fri Nov 30 10:16:26 2007 [image: red]<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=conn&TIMEBUF=Fri_Nov_30_10:16:26_2007>;
▸
0:02:53
The conn status page when green shows the correct IP address, and when
red shows the wrong (old) IP address. Nothing in the bb-hosts or DNS change
during this time.
I ran a "fping -AeC 1000 -i 5000 <host>" from the command line to see
if fping was switching IPs, and it does not seem to be, even when Hobbit
shows red, the fping from the command line still his the new/current IP.
I've tried forcing a "testip" in bb-hosts to see if DNS was involved,
and with that forced it still changes in Hobbit.
Any suggestions would be much appreciated.
Thanks,
Shawn
--
Josh Luthman
Office: XXX-XXX-XXXX
Direct: XXX-XXX-XXXX
XXXX Wayne St
Suite XXXX
Troy, OH XXXXX
Those who don't understand UNIX are condemned to reinvent it, poorly.
--- Henry Spencer
list Sebastian Auriol
Based mostly on my BB experience, but it looks equally applicable to Hobbit: Shawn's problem looks it might be caused by two BBNETs, one of which can see the other system and one of which cannot, both reporting to the same BBDISPLAY - hence the flapping. Josh's issue just looks like glitches (occasional packet loss, probably) in the network path from time to time. Sebastian
▸
From: Josh Luthman [mailto:user-4c45a83f15cb@xymon.invalid]
Sent: 30 November 2007 21:48
To: user-ae9b8668bcde@xymon.invalid
Subject: Re: [hobbit] Strange pinging issue
I get a very similar situation, of which I was just about to report! Mine
will be green for hours or days, then show up red for much less time (5-45
seconds). I've mine set as default..
FPING="/usr/sbin/fping"
I have testip on almost every single host.
On 11/30/07, Maschino, Shawn (SABIC Innovative Plastics) <
user-6359c5a3b499@xymon.invalid <mailto:user-6359c5a3b499@xymon.invalid> > wrote:
Hi all, we are seeing a strange fping issue with Hobbit and I was
wondering if anyone may have seen the same.
We have changed the IP addresses on a large number of systems we are
monitoring due to moving them to a new location. We updated the bb-hosts
file to have the new IPs, and DNS was also updated. However Hobbit
occasionally falls back to testing the old IP for no apparent reason.
This happens quite regularly as well, as seen here:
Fri Nov 30 10:56:08 2007<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=co nn&TIMEBUF=Fri_Nov_30_10:56:08_2007> red 0:04:19 Fri Nov 30 10:48:31 2007 <http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=co nn&TIMEBUF=Fri_Nov_30_10:48:31_2007> green 0:07:37 Fri Nov 30 10:48:08 2007 <http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=co nn&TIMEBUF=Fri_Nov_30_10:48:08_2007> red 0:00:23 Fri Nov 30 10:42:27 2007 <http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=co nn&TIMEBUF=Fri_Nov_30_10:42:27_2007> green 0:05:41 Fri Nov 30 10:39:53 2007 <http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=co nn&TIMEBUF=Fri_Nov_30_10:39:53_2007> red 0:02:34 Fri Nov 30 10:36:49 2007 <http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=co nn&TIMEBUF=Fri_Nov_30_10:36:49_2007> green 0:03:04 Fri Nov 30 10:31:57 2007 <http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=co nn&TIMEBUF=Fri_Nov_30_10:31:57_2007> red 0:04:52 Fri Nov 30 10:25:27 2007 <http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=co nn&TIMEBUF=Fri_Nov_30_10:25:27_2007> green 0:06:30 Fri Nov 30 10:23:58 2007 <http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=co nn&TIMEBUF=Fri_Nov_30_10:23:58_2007> red 0:01:29 Fri Nov 30 10:19:19 2007 <http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=co nn&TIMEBUF=Fri_Nov_30_10:19:19_2007> green 0:04:39 Fri Nov 30 10:16:26 2007 <http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=co nn&TIMEBUF=Fri_Nov_30_10:16:26_2007> red 0:02:53
▸
The conn status page when green shows the correct IP address, and when
red shows the wrong (old) IP address. Nothing in the bb-hosts or DNS change
during this time.
I ran a "fping -AeC 1000 -i 5000 <host>" from the command line to see if
fping was switching IPs, and it does not seem to be, even when Hobbit shows
red, the fping from the command line still his the new/current IP.
I've tried forcing a "testip" in bb-hosts to see if DNS was involved,
and with that forced it still changes in Hobbit.
Any suggestions would be much appreciated.
Thanks,
Shawn
--
Josh Luthman
Office: XXX-XXX-XXXX
Direct: XXX-XXX-XXXX
XXXX Wayne St
Suite XXXX
Troy, OH XXXXX
Those who don't understand UNIX are condemned to reinvent it, poorly.
--- Henry Spencer
list Josh Luthman
I really don't think my problem is packet loss or anything like that - I have a hard time believing that 127.0.0.1 or the public IP is unreachable of itself. It should noted that the duration is never more then a minute, sometimes 7 seconds up to 45 seconds - definitely not getting two polls in (according to what I've read it polls every 60 or 120 seconds, never quite that 7-45 second duration). If I ping the host from a shell I never get any packet loss or any issues, yet Hobbit still thinks it was down for a few seconds. This is every host I have monitored, by the way. Josh
▸
On 12/3/07, Sebastian <user-7b2156f36779@xymon.invalid> wrote:Based mostly on my BB experience, but it looks equally applicable to Hobbit: Shawn's problem looks it might be caused by two BBNETs, one of which can see the other system and one of which cannot, both reporting to the same BBDISPLAY - hence the flapping. Josh's issue just looks like glitches (occasional packet loss, probably) in the network path from time to time. Sebastian *From:* Josh Luthman [mailto:user-4c45a83f15cb@xymon.invalid] *Sent:* 30 November 2007 21:48 *To:* user-ae9b8668bcde@xymon.invalid *Subject:* Re: [hobbit] Strange pinging issue I get a very similar situation, of which I was just about to report! Mine will be green for hours or days, then show up red for much less time (5-45 seconds). I've mine set as default.. FPING="/usr/sbin/fping" I have testip on almost every single host. On 11/30/07, Maschino, Shawn (SABIC Innovative Plastics) <user-6359c5a3b499@xymon.invalid> wrote:Hi all, we are seeing a strange fping issue with Hobbit and I was wondering if anyone may have seen the same. We have changed the IP addresses on a large number of systems we are monitoring due to moving them to a new location. We updated the bb-hosts file to have the new IPs, and DNS was also updated. However Hobbit occasionally falls back to testing the old IP for no apparent reason. This happens quite regularly as well, as seen here: Fri Nov 30 10:56:08 2007 [image: red]<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=conn&TIMEBUF=Fri_Nov_30_10:56:08_2007>; 0:04:19 Fri Nov 30 10:48:31 2007 [image: green]<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=conn&TIMEBUF=Fri_Nov_30_10:48:31_2007>; 0:07:37 Fri Nov 30 10:48:08 2007 [image: red]<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=conn&TIMEBUF=Fri_Nov_30_10:48:08_2007>; 0:00:23 Fri Nov 30 10:42:27 2007 [image: green]<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=conn&TIMEBUF=Fri_Nov_30_10:42:27_2007>; 0:05:41 Fri Nov 30 10:39:53 2007 [image: red]<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=conn&TIMEBUF=Fri_Nov_30_10:39:53_2007>; 0:02:34 Fri Nov 30 10:36:49 2007 [image: green]<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=conn&TIMEBUF=Fri_Nov_30_10:36:49_2007>; 0:03:04 Fri Nov 30 10:31:57 2007 [image: red]<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=conn&TIMEBUF=Fri_Nov_30_10:31:57_2007>; 0:04:52 Fri Nov 30 10:25:27 2007 [image: green]<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=conn&TIMEBUF=Fri_Nov_30_10:25:27_2007>; 0:06:30 Fri Nov 30 10:23:58 2007 [image: red]<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=conn&TIMEBUF=Fri_Nov_30_10:23:58_2007>; 0:01:29 Fri Nov 30 10:19:19 2007 [image: green]<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=conn&TIMEBUF=Fri_Nov_30_10:19:19_2007>; 0:04:39 Fri Nov 30 10:16:26 2007 [image: red]<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=conn&TIMEBUF=Fri_Nov_30_10:16:26_2007>; 0:02:53 The conn status page when green shows the correct IP address, and when red shows the wrong (old) IP address. Nothing in the bb-hosts or DNS change during this time. I ran a "fping -AeC 1000 -i 5000 <host>" from the command line to see if fping was switching IPs, and it does not seem to be, even when Hobbit shows red, the fping from the command line still his the new/current IP. I've tried forcing a "testip" in bb-hosts to see if DNS was involved, and with that forced it still changes in Hobbit. Any suggestions would be much appreciated. Thanks, Shawn-- Josh Luthman Office: XXX-XXX-XXXX Direct: XXX-XXX-XXXX XXXX Wayne St Suite XXXX Troy, OH XXXXX Those who don't understand UNIX are condemned to reinvent it, poorly. --- Henry Spencer
-- Josh Luthman Office: XXX-XXX-XXXX Direct: XXX-XXX-XXXX XXXX Wayne St Suite XXXX Troy, OH XXXXX Those who don't understand UNIX are condemned to reinvent it, poorly. --- Henry Spencer
list Sebastian Auriol
Ah, does look like a non-network problem then. Have you tried hobbitping instead of fping? Do you have iptables or similar running?
▸
Sebastian From: Josh Luthman [mailto:user-4c45a83f15cb@xymon.invalid] Sent: 03 December 2007 16:36 To: user-ae9b8668bcde@xymon.invalid Subject: Re: [hobbit] Strange pinging issue I really don't think my problem is packet loss or anything like that - I have a hard time believing that 127.0.0.1 or the public IP is unreachable of itself. It should noted that the duration is never more then a minute, sometimes 7 seconds up to 45 seconds - definitely not getting two polls in (according to what I've read it polls every 60 or 120 seconds, never quite that 7-45 second duration). If I ping the host from a shell I never get any packet loss or any issues, yet Hobbit still thinks it was down for a few seconds. This is every host I have monitored, by the way. Josh On 12/3/07, Sebastian <user-7b2156f36779@xymon.invalid> wrote: Based mostly on my BB experience, but it looks equally applicable to Hobbit: Shawn's problem looks it might be caused by two BBNETs, one of which can see the other system and one of which cannot, both reporting to the same BBDISPLAY - hence the flapping. Josh's issue just looks like glitches (occasional packet loss, probably) in the network path from time to time. Sebastian From: Josh Luthman [mailto:user-4c45a83f15cb@xymon.invalid] Sent: 30 November 2007 21:48 To: user-ae9b8668bcde@xymon.invalid Subject: Re: [hobbit] Strange pinging issue I get a very similar situation, of which I was just about to report! Mine will be green for hours or days, then show up red for much less time (5-45 seconds). I've mine set as default.. FPING="/usr/sbin/fping" I have testip on almost every single host. On 11/30/07, Maschino, Shawn (SABIC Innovative Plastics) < <mailto:user-6359c5a3b499@xymon.invalid> user-6359c5a3b499@xymon.invalid> wrote: Hi all, we are seeing a strange fping issue with Hobbit and I was wondering if anyone may have seen the same. We have changed the IP addresses on a large number of systems we are monitoring due to moving them to a new location. We updated the bb-hosts file to have the new IPs, and DNS was also updated. However Hobbit occasionally falls back to testing the old IP for no apparent reason. This happens quite regularly as well, as seen here: Fri Nov 30 10:56:08 2007 <http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=co nn&TIMEBUF=Fri_Nov_30_10:56:08_2007> red 0:04:19 Fri Nov 30 10:48:31 2007 <http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=co nn&TIMEBUF=Fri_Nov_30_10:48:31_2007> green 0:07:37 Fri Nov 30 10:48:08 2007 <http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=co nn&TIMEBUF=Fri_Nov_30_10:48:08_2007> red 0:00:23 Fri Nov 30 10:42:27 2007 <http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=co nn&TIMEBUF=Fri_Nov_30_10:42:27_2007> green 0:05:41 Fri Nov 30 10:39:53 2007 <http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=co nn&TIMEBUF=Fri_Nov_30_10:39:53_2007> red 0:02:34 Fri Nov 30 10:36:49 2007 <http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=co nn&TIMEBUF=Fri_Nov_30_10:36:49_2007> green 0:03:04 Fri Nov 30 10:31:57 2007 <http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=co nn&TIMEBUF=Fri_Nov_30_10:31:57_2007> red 0:04:52 Fri Nov 30 10:25:27 2007 <http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=co nn&TIMEBUF=Fri_Nov_30_10:25:27_2007> green 0:06:30 Fri Nov 30 10:23:58 2007 <http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=co nn&TIMEBUF=Fri_Nov_30_10:23:58_2007> red 0:01:29 Fri Nov 30 10:19:19 2007 <http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=co nn&TIMEBUF=Fri_Nov_30_10:19:19_2007> green 0:04:39 Fri Nov 30 10:16:26 2007 <http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=co nn&TIMEBUF=Fri_Nov_30_10:16:26_2007> red 0:02:53 The conn status page when green shows the correct IP address, and when red shows the wrong (old) IP address. Nothing in the bb-hosts or DNS change during this time. I ran a "fping -AeC 1000 -i 5000 <host>" from the command line to see if fping was switching IPs, and it does not seem to be, even when Hobbit shows red, the fping from the command line still his the new/current IP. I've tried forcing a "testip" in bb-hosts to see if DNS was involved, and with that forced it still changes in Hobbit. Any suggestions would be much appreciated. Thanks, Shawn -- Josh Luthman Office: XXX-XXX-XXXX Direct: XXX-XXX-XXXX XXXX Wayne St Suite XXXX Troy, OH XXXXX Those who don't understand UNIX are condemned to reinvent it, poorly. --- Henry Spencer -- Josh Luthman Office: XXX-XXX-XXXX Direct: XXX-XXX-XXXX XXXX Wayne St Suite XXXX Troy, OH XXXXX Those who don't understand UNIX are condemned to reinvent it, poorly. --- Henry Spencer
list Josh Luthman
I started with hobbitping and switched to fping - same results. IPtables is installed, but not running nor ever set to run.
▸
Josh
On 12/3/07, Sebastian <user-7b2156f36779@xymon.invalid> wrote:Ah, does look like a non-network problem then. Have you tried hobbitping instead of fping? Do you have iptables or similar running? Sebastian *From:* Josh Luthman [mailto:user-4c45a83f15cb@xymon.invalid] *Sent:* 03 December 2007 16:36 *To:* user-ae9b8668bcde@xymon.invalid *Subject:* Re: [hobbit] Strange pinging issue I really don't think my problem is packet loss or anything like that - I have a hard time believing that 127.0.0.1 or the public IP is unreachable of itself. It should noted that the duration is never more then a minute, sometimes 7 seconds up to 45 seconds - definitely not getting two polls in (according to what I've read it polls every 60 or 120 seconds, never quite that 7-45 second duration). If I ping the host from a shell I never get any packet loss or any issues, yet Hobbit still thinks it was down for a few seconds. This is every host I have monitored, by the way. Josh On 12/3/07, Sebastian <user-7b2156f36779@xymon.invalid> wrote:Based mostly on my BB experience, but it looks equally applicable to Hobbit: Shawn's problem looks it might be caused by two BBNETs, one of which can see the other system and one of which cannot, both reporting to the same BBDISPLAY - hence the flapping. Josh's issue just looks like glitches (occasional packet loss, probably) in the network path from time to time. Sebastian *From:* Josh Luthman [mailto:user-4c45a83f15cb@xymon.invalid] *Sent:* 30 November 2007 21:48 *To:* user-ae9b8668bcde@xymon.invalid *Subject:* Re: [hobbit] Strange pinging issue I get a very similar situation, of which I was just about to report! Mine will be green for hours or days, then show up red for much less time (5-45 seconds). I've mine set as default.. FPING="/usr/sbin/fping" I have testip on almost every single host. On 11/30/07, Maschino, Shawn (SABIC Innovative Plastics) <user-6359c5a3b499@xymon.invalid> wrote:Hi all, we are seeing a strange fping issue with Hobbit and I was wondering if anyone may have seen the same. We have changed the IP addresses on a large number of systems we are monitoring due to moving them to a new location. We updated the bb-hosts file to have the new IPs, and DNS was also updated. However Hobbit occasionally falls back to testing the old IP for no apparent reason. This happens quite regularly as well, as seen here: Fri Nov 30 10:56:08 2007 [image: red]<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=conn&TIMEBUF=Fri_Nov_30_10:56:08_2007>; 0:04:19 Fri Nov 30 10:48:31 2007 [image: green]<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=conn&TIMEBUF=Fri_Nov_30_10:48:31_2007>; 0:07:37 Fri Nov 30 10:48:08 2007 [image: red]<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=conn&TIMEBUF=Fri_Nov_30_10:48:08_2007>; 0:00:23 Fri Nov 30 10:42:27 2007 [image: green]<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=conn&TIMEBUF=Fri_Nov_30_10:42:27_2007>; 0:05:41 Fri Nov 30 10:39:53 2007 [image: red]<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=conn&TIMEBUF=Fri_Nov_30_10:39:53_2007>; 0:02:34 Fri Nov 30 10:36:49 2007 [image: green]<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=conn&TIMEBUF=Fri_Nov_30_10:36:49_2007>; 0:03:04 Fri Nov 30 10:31:57 2007 [image: red]<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=conn&TIMEBUF=Fri_Nov_30_10:31:57_2007>; 0:04:52 Fri Nov 30 10:25:27 2007 [image: green]<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=conn&TIMEBUF=Fri_Nov_30_10:25:27_2007>; 0:06:30 Fri Nov 30 10:23:58 2007 [image: red]<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=conn&TIMEBUF=Fri_Nov_30_10:23:58_2007>; 0:01:29 Fri Nov 30 10:19:19 2007 [image: green]<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=conn&TIMEBUF=Fri_Nov_30_10:19:19_2007>; 0:04:39 Fri Nov 30 10:16:26 2007 [image: red]<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=conn&TIMEBUF=Fri_Nov_30_10:16:26_2007>; 0:02:53 The conn status page when green shows the correct IP address, and when red shows the wrong (old) IP address. Nothing in the bb-hosts or DNS change during this time. I ran a "fping -AeC 1000 -i 5000 <host>" from the command line to see if fping was switching IPs, and it does not seem to be, even when Hobbit shows red, the fping from the command line still his the new/current IP. I've tried forcing a "testip" in bb-hosts to see if DNS was involved, and with that forced it still changes in Hobbit. Any suggestions would be much appreciated. Thanks, Shawn-- Josh Luthman Office: XXX-XXX-XXXX Direct: XXX-XXX-XXXX XXXX Wayne St Suite XXXX Troy, OH XXXXX Those who don't understand UNIX are condemned to reinvent it, poorly. --- Henry Spencer-- Josh Luthman Office: XXX-XXX-XXXX Direct: XXX-XXX-XXXX XXXX Wayne St Suite XXXX Troy, OH XXXXX Those who don't understand UNIX are condemned to reinvent it, poorly. --- Henry Spencer
-- Josh Luthman Office: XXX-XXX-XXXX Direct: XXX-XXX-XXXX XXXX Wayne St Suite XXXX Troy, OH XXXXX Those who don't understand UNIX are condemned to reinvent it, poorly. --- Henry Spencer
list Shawn Maschino
Good call on my issue, that indeed was the case. We cloned our Hobbit server to have another for redundancy, and in the setup had some config problems with them integrating with each other. It's all set now. Thanks!
▸
From: Sebastian [mailto:user-7b2156f36779@xymon.invalid]
Sent: Monday, December 03, 2007 10:56 AM
To: user-ae9b8668bcde@xymon.invalid
Subject: RE: [hobbit] Strange pinging issue
Based mostly on my BB experience, but it looks equally applicable to
Hobbit:
Shawn's problem looks it might be caused by two BBNETs, one of which can
see the other system and one of which cannot, both reporting to the same
BBDISPLAY - hence the flapping.
Josh's issue just looks like glitches (occasional packet loss, probably)
in the network path from time to time.
Sebastian
From: Josh Luthman [mailto:user-4c45a83f15cb@xymon.invalid]
Sent: 30 November 2007 21:48
To: user-ae9b8668bcde@xymon.invalid
Subject: Re: [hobbit] Strange pinging issue
I get a very similar situation, of which I was just about to
report! Mine will be green for hours or days, then show up red for much
less time (5-45 seconds). I've mine set as default..
FPING="/usr/sbin/fping"
I have testip on almost every single host.
On 11/30/07, Maschino, Shawn (SABIC Innovative Plastics) <
user-6359c5a3b499@xymon.invalid <mailto:user-6359c5a3b499@xymon.invalid> >
wrote:
Hi all, we are seeing a strange fping issue with
Hobbit and I was wondering if anyone may have seen the same.
We have changed the IP addresses on a large number
of systems we are monitoring due to moving them to a new location. We
updated the bb-hosts file to have the new IPs, and DNS was also updated.
However Hobbit occasionally falls back to testing the old IP for no
apparent reason.
This happens quite regularly as well, as seen here:
Fri Nov 30 10:56:08 2007 red <http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVIC E=conn&TIMEBUF=Fri_Nov_30_10:56:08_2007> 0:04:19 Fri Nov 30 10:48:31 2007 green <http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVIC E=conn&TIMEBUF=Fri_Nov_30_10:48:31_2007> 0:07:37 Fri Nov 30 10:48:08 2007 red <http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVIC E=conn&TIMEBUF=Fri_Nov_30_10:48:08_2007> 0:00:23 Fri Nov 30 10:42:27 2007 green <http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVIC E=conn&TIMEBUF=Fri_Nov_30_10:42:27_2007> 0:05:41 Fri Nov 30 10:39:53 2007 red <http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVIC E=conn&TIMEBUF=Fri_Nov_30_10:39:53_2007> 0:02:34 Fri Nov 30 10:36:49 2007 green <http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVIC E=conn&TIMEBUF=Fri_Nov_30_10:36:49_2007> 0:03:04 Fri Nov 30 10:31:57 2007 red <http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVIC E=conn&TIMEBUF=Fri_Nov_30_10:31:57_2007> 0:04:52 Fri Nov 30 10:25:27 2007 green <http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVIC E=conn&TIMEBUF=Fri_Nov_30_10:25:27_2007> 0:06:30 Fri Nov 30 10:23:58 2007 red <http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVIC E=conn&TIMEBUF=Fri_Nov_30_10:23:58_2007> 0:01:29 Fri Nov 30 10:19:19 2007 green <http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVIC E=conn&TIMEBUF=Fri_Nov_30_10:19:19_2007> 0:04:39 Fri Nov 30 10:16:26 2007 red
▸
<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVIC E=conn&TIMEBUF=Fri_Nov_30_10:16:26_2007> 0:02:53 The conn status page when green shows the correct IP address, and when red shows the wrong (old) IP address. Nothing in the bb-hosts or DNS change during this time. I ran a "fping -AeC 1000 -i 5000 <host>" from the command line to see if fping was switching IPs, and it does not seem to be, even when Hobbit shows red, the fping from the command line still his the new/current IP. I've tried forcing a "testip" in bb-hosts to see if DNS was involved, and with that forced it still changes in Hobbit. Any suggestions would be much appreciated. Thanks, Shawn -- Josh Luthman Office: XXX-XXX-XXXX Direct: XXX-XXX-XXXX XXXX Wayne St Suite XXXX Troy, OH XXXXX Those who don't understand UNIX are condemned to reinvent it, poorly. --- Henry Spencer
list Josh Luthman
Unfortunately my issue is still nagging at me! I've found that hosts on the same switch and hosts on one LAN have the issue, while some hosts going through a router are good, others are bad. Everyone of the red pages say it is "System unreachable for 1 poll periods (0 seconds)" and never lasts more then a minute. Anyone have a clue where to look? Josh On 12/5/07, Maschino, Shawn (SABIC Innovative Plastics) <
▸
user-6359c5a3b499@xymon.invalid> wrote:Good call on my issue, that indeed was the case. We cloned our Hobbit server to have another for redundancy, and in the setup had some config problems with them integrating with each other. It's all set now. Thanks! *From:* Sebastian [mailto:user-7b2156f36779@xymon.invalid] *Sent:* Monday, December 03, 2007 10:56 AM *To:* user-ae9b8668bcde@xymon.invalid *Subject:* RE: [hobbit] Strange pinging issue Based mostly on my BB experience, but it looks equally applicable to Hobbit: Shawn's problem looks it might be caused by two BBNETs, one of which can see the other system and one of which cannot, both reporting to the same BBDISPLAY - hence the flapping. Josh's issue just looks like glitches (occasional packet loss, probably) in the network path from time to time. Sebastian *From:* Josh Luthman [mailto:user-4c45a83f15cb@xymon.invalid] *Sent:* 30 November 2007 21:48 *To:* user-ae9b8668bcde@xymon.invalid *Subject:* Re: [hobbit] Strange pinging issue I get a very similar situation, of which I was just about to report! Mine will be green for hours or days, then show up red for much less time (5-45 seconds). I've mine set as default.. FPING="/usr/sbin/fping" I have testip on almost every single host. On 11/30/07, Maschino, Shawn (SABIC Innovative Plastics) <user-6359c5a3b499@xymon.invalid> wrote:Hi all, we are seeing a strange fping issue with Hobbit and I was wondering if anyone may have seen the same. We have changed the IP addresses on a large number of systems we are monitoring due to moving them to a new location. We updated the bb-hosts file to have the new IPs, and DNS was also updated. However Hobbit occasionally falls back to testing the old IP for no apparent reason. This happens quite regularly as well, as seen here: Fri Nov 30 10:56:08 2007 [image: red]<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=conn&TIMEBUF=Fri_Nov_30_10:56:08_2007>; 0:04:19 Fri Nov 30 10:48:31 2007 [image: green]<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=conn&TIMEBUF=Fri_Nov_30_10:48:31_2007>; 0:07:37 Fri Nov 30 10:48:08 2007 [image: red]<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=conn&TIMEBUF=Fri_Nov_30_10:48:08_2007>; 0:00:23 Fri Nov 30 10:42:27 2007 [image: green]<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=conn&TIMEBUF=Fri_Nov_30_10:42:27_2007>; 0:05:41 Fri Nov 30 10:39:53 2007 [image: red]<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=conn&TIMEBUF=Fri_Nov_30_10:39:53_2007>; 0:02:34 Fri Nov 30 10:36:49 2007 [image: green]<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=conn&TIMEBUF=Fri_Nov_30_10:36:49_2007>; 0:03:04 Fri Nov 30 10:31:57 2007 [image: red]<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=conn&TIMEBUF=Fri_Nov_30_10:31:57_2007>; 0:04:52 Fri Nov 30 10:25:27 2007 [image: green]<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=conn&TIMEBUF=Fri_Nov_30_10:25:27_2007>; 0:06:30 Fri Nov 30 10:23:58 2007 [image: red]<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=conn&TIMEBUF=Fri_Nov_30_10:23:58_2007>; 0:01:29 Fri Nov 30 10:19:19 2007 [image: green]<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=conn&TIMEBUF=Fri_Nov_30_10:19:19_2007>; 0:04:39 Fri Nov 30 10:16:26 2007 [image: red]<http://lacin002v.gep.ge.com/cgi-bin/bb-histlog.sh?HOST=SACIN053V&SERVICE=conn&TIMEBUF=Fri_Nov_30_10:16:26_2007>; 0:02:53 The conn status page when green shows the correct IP address, and when red shows the wrong (old) IP address. Nothing in the bb-hosts or DNS change during this time. I ran a "fping -AeC 1000 -i 5000 <host>" from the command line to see if fping was switching IPs, and it does not seem to be, even when Hobbit shows red, the fping from the command line still his the new/current IP. I've tried forcing a "testip" in bb-hosts to see if DNS was involved, and with that forced it still changes in Hobbit. Any suggestions would be much appreciated. Thanks, Shawn-- Josh Luthman Office: XXX-XXX-XXXX Direct: XXX-XXX-XXXX XXXX Wayne St Suite XXXX Troy, OH XXXXX Those who don't understand UNIX are condemned to reinvent it, poorly. --- Henry Spencer
-- Josh Luthman Office: XXX-XXX-XXXX Direct: XXX-XXX-XXXX XXXX Wayne St Suite XXXX Troy, OH XXXXX Those who don't understand UNIX are condemned to reinvent it, poorly. --- Henry Spencer
list Oliver Grube
Campbell’s Germany GmbH Registergericht Luebeck • Reg. Nr. HRB 4082 Geschaeftsfuehrer: Joseph B. Folds III Geschaeftssitz: Geniner Strasse 88 - 100 23560 Luebeck Deutschland ********************************************************************** This e-mail and any files transmitted with it may contain confidential information and is intended solely for use by the individual to whom it is addressed. If you received this e-mail in error, please notify the sender, do not disclose its contents to others and delete it from your system. **********************************************************************