Frequent core dumps 🔗 link

On Thu, Mar 5, 2015 at 9:18 AM, Colin Coe <user-5b250cd7a540@xymon.invalid> wrote:

Hi all

Over the last day or so,been getting a lot of xymonnet core dumps.

Here's the debugging info as per
http://xymon.sourceforge.net/xymon/help/known-issues.html

---
[root at xymon_srv 09:06 /var/lib/xymon/tmp]# file core.15685
core.15685: ELF 64-bit LSB core file x86-64, version 1 (SYSV),
SVR4-style, from '/usr/libexec/xymon/xymonnet --ping --checkresponse
portal01 portal02 TAB002'
[root at xymon_srv 09:06 /var/lib/xymon/tmp]# gdb
/usr/libexec/xymon/xymonnet core.15685
GNU gdb (GDB) Red Hat Enterprise Linux (7.2-75.el6)
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>;
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>;...
Reading symbols from /usr/libexec/xymon/xymonnet...Reading symbols
from /usr/lib/debug/usr/libexec/xymon/xymonnet.debug...done.
done.
[New Thread 15685]
Missing separate debuginfo for
Try: yum --enablerepo='*-debug*' install
/usr/lib/debug/.build-id/f1/5837b82aa5f929c8237a97e4a8b924b457ad7e
Reading symbols from /usr/lib64/libcares.so.2...(no debugging symbols
found)...done.
Loaded symbols for /usr/lib64/libcares.so.2
Reading symbols from /lib64/libldap-2.4.so.2...(no debugging symbols
found)...done.
Loaded symbols for /lib64/libldap-2.4.so.2
Reading symbols from /lib64/librt.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib64/librt.so.1
Reading symbols from /lib64/libz.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib64/libz.so.1
Reading symbols from /usr/lib64/libssl.so.10...(no debugging symbols
found)...done.
Loaded symbols for /usr/lib64/libssl.so.10
Reading symbols from /usr/lib64/libcrypto.so.10...(no debugging
symbols found)...done.
Loaded symbols for /usr/lib64/libcrypto.so.10
Reading symbols from /lib64/libpcre.so.0...(no debugging symbols found)...done.
Loaded symbols for /lib64/libpcre.so.0
Reading symbols from /lib64/libc.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib64/libc.so.6
Reading symbols from /lib64/liblber-2.4.so.2...(no debugging symbols
found)...done.
Loaded symbols for /lib64/liblber-2.4.so.2
Reading symbols from /lib64/libresolv.so.2...(no debugging symbols
found)...done.
Loaded symbols for /lib64/libresolv.so.2
Reading symbols from /usr/lib64/libsasl2.so.2...(no debugging symbols
found)...done.
Loaded symbols for /usr/lib64/libsasl2.so.2
Reading symbols from /usr/lib64/libssl3.so...(no debugging symbols
found)...done.
Loaded symbols for /usr/lib64/libssl3.so
Reading symbols from /usr/lib64/libsmime3.so...(no debugging symbols
found)...done.
Loaded symbols for /usr/lib64/libsmime3.so
Reading symbols from /usr/lib64/libnss3.so...(no debugging symbols
found)...done.
Loaded symbols for /usr/lib64/libnss3.so
Reading symbols from /usr/lib64/libnssutil3.so...(no debugging symbols
found)...done.
Loaded symbols for /usr/lib64/libnssutil3.so
Reading symbols from /lib64/libplds4.so...(no debugging symbols found)...done.
Loaded symbols for /lib64/libplds4.so
Reading symbols from /lib64/libplc4.so...(no debugging symbols found)...done.
Loaded symbols for /lib64/libplc4.so
Reading symbols from /lib64/libnspr4.so...(no debugging symbols found)...done.
Loaded symbols for /lib64/libnspr4.so
Reading symbols from /lib64/libpthread.so.0...(no debugging symbols
found)...done.
[Thread debugging using libthread_db enabled]
Loaded symbols for /lib64/libpthread.so.0
Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging
symbols found)...done.
Loaded symbols for /lib64/ld-linux-x86-64.so.2
Reading symbols from /lib64/libgssapi_krb5.so.2...(no debugging
symbols found)...done.
Loaded symbols for /lib64/libgssapi_krb5.so.2
Reading symbols from /lib64/libkrb5.so.3...(no debugging symbols found)...done.
Loaded symbols for /lib64/libkrb5.so.3
Reading symbols from /lib64/libcom_err.so.2...(no debugging symbols
found)...done.
Loaded symbols for /lib64/libcom_err.so.2
Reading symbols from /lib64/libk5crypto.so.3...(no debugging symbols
found)...done.
Loaded symbols for /lib64/libk5crypto.so.3
Reading symbols from /lib64/libdl.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/libdl.so.2
Reading symbols from /lib64/libcrypt.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib64/libcrypt.so.1
Reading symbols from /lib64/libkrb5support.so.0...(no debugging
symbols found)...done.
Loaded symbols for /lib64/libkrb5support.so.0
Reading symbols from /lib64/libkeyutils.so.1...(no debugging symbols
found)...done.
Loaded symbols for /lib64/libkeyutils.so.1
Reading symbols from /lib64/libfreebl3.so...(no debugging symbols found)...done.
Loaded symbols for /lib64/libfreebl3.so
Reading symbols from /lib64/libselinux.so.1...(no debugging symbols
found)...done.
Loaded symbols for /lib64/libselinux.so.1
Reading symbols from /lib64/libnss_files.so.2...(no debugging symbols
found)...done.
Loaded symbols for /lib64/libnss_files.so.2
Core was generated by `/usr/libexec/xymon/xymonnet --ping
--checkresponse portal01 portal02 TAB002'.
Program terminated with signal 6, Aborted.
#0  0x0000003fcda32625 in raise () from /lib64/libc.so.6
Missing separate debuginfos, use: debuginfo-install
c-ares-1.10.0-3.el6.x86_64 cyrus-sasl-lib-2.1.23-15.el6_6.1.x86_64
glibc-2.12-1.149.el6_6.5.x86_64 keyutils-libs-1.4-5.el6.x86_64
krb5-libs-1.10.3-33.el6.x86_64 libcom_err-1.41.12-21.el6.x86_64
libselinux-2.0.94-5.8.el6.x86_64 nspr-4.10.6-1.el6_5.x86_64
nss-3.16.2.3-3.el6_6.x86_64 nss-softokn-freebl-3.14.3-22.el6_6.x86_64
nss-util-3.16.2.3-2.el6_6.x86_64 openldap-2.4.39-8.el6.x86_64
openssl-1.0.1e-30.el6_6.5.x86_64 pcre-7.8-6.el6.x86_64
zlib-1.2.3-29.el6.x86_64
(gdb) bt
#0  0x0000003fcda32625 in raise () from /lib64/libc.so.6
#1  0x0000003fcda33e05 in abort () from /lib64/libc.so.6
#2  0x0000000000420ab3 in sigsegv_handler (signum=<value optimized
out>) at sig.c:57
#3  <signal handler called>
#4  0x0000003fcda81171 in __strlen_sse2 () from /lib64/libc.so.6
#5  0x0000003fcda80e86 in strdup () from /lib64/libc.so.6
#6  0x000000000040d2a4 in setup_ssl (item=0xa136c0) at contest.c:653
#7  0x000000000040edef in do_tcp_tests (timeout=10, concurrency=256)
at contest.c:1150
#8  0x000000000040aaf3 in main (argc=6, argv=0x7fffe1c7a008) at xymonnet.c:2461
(gdb)
[root at xymon_srv 09:08 /var/lib/xymon/tmp]# cat ping-stderr.15685.00
ICMP Network Unreachable from 172.22.31.1 for ICMP Echo sent to 172.31.64.27
ICMP Network Unreachable from 172.22.31.1 for ICMP Echo sent to 172.31.64.27
ICMP Network Unreachable from 172.22.31.1 for ICMP Echo sent to 172.31.64.27
ICMP Network Unreachable from 172.22.31.1 for ICMP Echo sent to 172.31.64.27
[root at xymon_srv 09:08 /var/lib/xymon/tmp]#
---

Notes:
- TAB002 is 172.31.64.27
- TAB002 hosts.cfg entry is "172.31.64.27 TAB002 # conn nonongreen"
- server is RHEL6 x86_64 with RPM 4.3.18-1.el6.x86_64 from Terabithia

Anyone else seeing this?

Thanks

CC

On 3/4/2015 4:31 PM, Colin Coe wrote:

Found the problem.  There was a missing closing round bracket ')' on
the hosts.cfg line for portal02.

On Wed, March 4, 2015 5:39 pm, John Thurston wrote:

On 3/4/2015 4:31 PM, Colin Coe wrote:

Found the problem.  There was a missing closing round bracket ')' on
the hosts.cfg line for portal02.

yowser. That's a pretty brutal failure mode for a syntax error in
hosts.cfg   If you feed that file to xymongrep or xymoncfg, do either of
them complain of the syntax error?

(I'm just wondering if I should ad a post-edit syntax reality check to
my workflow.)
--

On Fri, Mar 6, 2015 at 2:43 AM, J.C. Cleaver <user-87556346d4af@xymon.invalid> wrote:

On Wed, March 4, 2015 5:39 pm, John Thurston wrote:

On 3/4/2015 4:31 PM, Colin Coe wrote:

Found the problem.  There was a missing closing round bracket ')' on
the hosts.cfg line for portal02.

yowser. That's a pretty brutal failure mode for a syntax error in
hosts.cfg   If you feed that file to xymongrep or xymoncfg, do either of
them complain of the syntax error?

(I'm just wondering if I should ad a post-edit syntax reality check to
my workflow.)
--

Colin, can you forward me a copy of the relevant snippet?

If we died here, it was probably a result of an un-sanity-checked URL
trying to get analyzed for SSL. We can probably add a little more
protection around this.


Regards,

-jc