Xymon Mailing List Archive search

4.1.1 crashing

4 messages in this thread

list Craig Cook · Fri, 09 Sep 2005 01:28:04 -0500 ·
I'm not sure how I have managed to crash bbnet but I have ;)

2005-09-09 14:13:56 Execution of '/usr/sbin/fping -Ae' failed with error-code 99
2005-09-09 14:13:56 fping invocation failed: Permission denied
*** glibc detected *** corrupted double-linked list: 0x09bee5d8 ***
*** glibc detected *** malloc(): memory corruption: 0x09768388 ***
*** glibc detected *** malloc(): memory corruption: 0x096675d8 ***
*** glibc detected *** malloc(): memory corruption: 0x097745d8 ***
*** glibc detected *** malloc(): memory corruption: 0x08f45388 ***

(I fixed the fping problem)

2005-09-09 15:36:05 Task bbnet terminated by signal 6
2005-09-09 15:51:07 Task bbnet terminated by signal 6
2005-09-09 15:56:11 Task bbnet terminated by signal 6
2005-09-09 16:01:15 Task bbnet terminated by signal 6
2005-09-09 16:10:49 Task bbnet terminated by signal 6
2005-09-09 16:15:53 Task bbnet terminated by signal 6
2005-09-09 16:17:30 hobbitlaunch starting
2005-09-09 16:17:30 Loading tasklist configuration from /home/hobbit/server/etc/hobbitlaunch.cfg
2005-09-09 16:17:30 Loading hostnames
2005-09-09 16:17:30 Loading saved state
2005-09-09 16:17:30 Setting up network listener on 0.0.0.0:1984
2005-09-09 16:17:30 Setting up signal handlers
2005-09-09 16:17:30 Setting up hobbitd channels
2005-09-09 16:17:30 Setting up logfiles
2005-09-09 16:17:35 Task bbnet terminated by signal 6

bb-hosts had 2 devices (server and one client).  I added a 3rd device to do dns testing and I crashed the network testing.

CentOS 4 box:
Linux <linux_box> 2.6.9-5.0.3.ELsmp #1 SMP Sat Feb 19 19:38:02 CST 2005 i686 i686 i386 GNU/Linux

[hobbit at linux server]$ pwd
/home/hobbit/server
[hobbit at linux server]$ ls -l core*
-rw-------  1 hobbit hobbit  712704 Sep  9 15:36 core.25462
-rw-------  1 hobbit hobbit  598016 Sep  9 15:51 core.26756
-rw-------  1 hobbit hobbit 2301952 Sep  9 15:56 core.27198
-rw-------  1 hobbit hobbit 2465792 Sep  9 16:01 core.27638

I compiled hobbit on this box today.

Craig Cook
--
Systems Monitoring Consulting and Support Services
http://www.cookitservices.com
list Craig Cook · Sun, 11 Sep 2005 19:45:10 -0500 ·
Did some more reading... hope this helps:
CentOS 4 Server

Linux linux.domain.com 2.6.9-5.0.3.ELsmp #1 SMP Sat Feb 19 19:38:02 CST 2005 i686 i686 i386 GNU/Linux

Trying to do dns testing
bb-hosts:
[snip]
0.0.0.0  dns.domain.com # dns


[hobbit at linux server]$ gdb bin/bbtest-net tmp/core.24725
GNU gdb Red Hat Linux (6.3.0.0-0.31rh)
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-redhat-linux-gnu"...Using host libthread_db library "/lib/tls/libthread_db.so.1".

Core was generated by `bbtest-net --report --ping --checkresponse'.
Program terminated with signal 6, Aborted.
Reading symbols from /usr/lib/libldap-2.2.so.7...done.
Loaded symbols for /usr/lib/libldap-2.2.so.7
Reading symbols from /usr/lib/liblber-2.2.so.7...done.
Loaded symbols for /usr/lib/liblber-2.2.so.7
Reading symbols from /lib/libssl.so.4...done.
Loaded symbols for /lib/libssl.so.4
Reading symbols from /lib/libcrypto.so.4...done.
Loaded symbols for /lib/libcrypto.so.4
Reading symbols from /lib/tls/libc.so.6...done.
Loaded symbols for /lib/tls/libc.so.6
Reading symbols from /lib/libresolv.so.2...done.
Loaded symbols for /lib/libresolv.so.2
Reading symbols from /usr/lib/libsasl2.so.2...done.
Loaded symbols for /usr/lib/libsasl2.so.2
Reading symbols from /usr/lib/libgssapi_krb5.so.2...done.
Loaded symbols for /usr/lib/libgssapi_krb5.so.2
Reading symbols from /usr/lib/libkrb5.so.3...done.
Loaded symbols for /usr/lib/libkrb5.so.3
Reading symbols from /lib/libcom_err.so.2...done.
Loaded symbols for /lib/libcom_err.so.2
Reading symbols from /usr/lib/libk5crypto.so.3...done.
Loaded symbols for /usr/lib/libk5crypto.so.3
Reading symbols from /lib/libdl.so.2...done.
Loaded symbols for /lib/libdl.so.2
Reading symbols from /usr/lib/libz.so.1...done.
Loaded symbols for /usr/lib/libz.so.1
Reading symbols from /lib/ld-linux.so.2...done.
Loaded symbols for /lib/ld-linux.so.2
Reading symbols from /lib/libcrypt.so.1...done.
Loaded symbols for /lib/libcrypt.so.1
Reading symbols from /lib/libnss_files.so.2...done.
Loaded symbols for /lib/libnss_files.so.2
#0  0x0073a7a2 in ?? () from /lib/ld-linux.so.2
(gdb) bt
#0  0x0073a7a2 in ?? () from /lib/ld-linux.so.2
#1  0x0077a7d5 in raise () from /lib/tls/libc.so.6
#2  0x0077c149 in abort () from /lib/tls/libc.so.6
#3  0x0805dc16 in sigsegv_handler (signum=11) at sig.c:57
#4  0x0077a8c8 in killpg () from /lib/tls/libc.so.6
#5  0x08060677 in next_server (channel=0x884c318, query=0x100e00, now=1126485130) at ares_process.c:404
#6  0x08060d5a in ares_process (channel=0x884c318, read_fds=0xbff38850, write_fds=0xbff387d0) at ares_process.c:288
#7  0x08055b36 in dns_queue_run (channel=0x884c318) at dns.c:202
#8  0x08055de6 in flush_dnsqueue () at dns.c:231
#9  0x0804f898 in main (argc=4, argv=0xbff39c94) at bbtest-net.c:2194
(gdb)


Craig Cook
--
Systems Monitoring Consulting and Support Services
http://www.cookitservices.com
list Henrik Størner · Mon, 12 Sep 2005 05:58:54 +0200 ·
quoted from Craig Cook
On Sun, Sep 11, 2005 at 07:45:10PM -0500, Craig Cook wrote:
Did some more reading... hope this helps:
#3  0x0805dc16 in sigsegv_handler (signum=11) at sig.c:57
#4  0x0077a8c8 in killpg () from /lib/tls/libc.so.6
#5  0x08060677 in next_server (channel=0x884c318, query=0x100e00, now=1126485130) at ares_process.c:404
#6  0x08060d5a in ares_process (channel=0x884c318, read_fds=0xbff38850, write_fds=0xbff387d0) at ares_process.c:288
#7  0x08055b36 in dns_queue_run (channel=0x884c318) at dns.c:202
#8  0x08055de6 in flush_dnsqueue () at dns.c:231
#9  0x0804f898 in main (argc=4, argv=0xbff39c94) at bbtest-net.c:2194

Looks like a problem with the C-ARES library that Hobbit uses for
DNS lookups - the trace is identical to one I've seen before.

If you run bbtest-net with the "--no-ares" option, I'll bet that
it works OK.


This is hopefully fixed in the next C-ARES version. If you want to try
that, you can pick up the latest Hobbit snapshot - build it and you can
just copy the bbnet/bbtest-net binary on top of your current
installation.


Henrik
list Craig Cook · Sun, 11 Sep 2005 23:18:20 -0500 ·
Yes, the --no-ares did the trick.

Thanks

Craig Cook
--
Systems Monitoring Consulting and Support Services
http://www.cookitservices.com