hobbitd_rrd crash 🔗 link

I have notiched that my graphs are not updated and thus I found a lot of core files. Is this a filesystem size error ?

-bash-2.05b$ file core.9965
core.9965: ELF 32-bit LSB core file Intel 80386, version 1 (SYSV), SVR4-style, from 'hobbitd_rrd'
-bash-2.05b$ gdb bin/hobbitd_rrd core.9965
GNU gdb Red Hat Linux (6.1post-1.20040607.52rh)
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-redhat-linux-gnu"...Using host libthread_db library "/lib/tls/libthread_db.so.1".

Core was generated by `hobbitd_rrd --rrddir=/hobbit/hobbit/data/rrd'.
Program terminated with signal 25, File size limit exceeded.
Reading symbols from /usr/lib/librrd.so.0...done.
Loaded symbols for /usr/lib/librrd.so.0
Reading symbols from /usr/lib/libpng12.so.0...done.
Loaded symbols for /usr/lib/libpng12.so.0
Reading symbols from /lib/libpcre.so.0...done.
Loaded symbols for /lib/libpcre.so.0
Reading symbols from /lib/tls/libc.so.6...done.
Loaded symbols for /lib/tls/libc.so.6
Reading symbols from /usr/lib/libz.so.1...done.
Loaded symbols for /usr/lib/libz.so.1
Reading symbols from /lib/tls/libm.so.6...done.
Loaded symbols for /lib/tls/libm.so.6
Reading symbols from /lib/ld-linux.so.2...done.
Loaded symbols for /lib/ld-linux.so.2
#0  0x0029bf8e in __write_nocancel () from /lib/tls/libc.so.6
(gdb) bt
#0  0x0029bf8e in __write_nocancel () from /lib/tls/libc.so.6
#1  0x00238b3f in _IO_new_file_write () from /lib/tls/libc.so.6
#2  0x00237ade in new_do_write () from /lib/tls/libc.so.6
#3  0x00238ccf in _IO_new_file_xsputn () from /lib/tls/libc.so.6
#4  0x002156fe in buffered_vfprintf () from /lib/tls/libc.so.6
#5  0x002113d9 in vfprintf () from /lib/tls/libc.so.6
#6  0x00219f7f in fprintf () from /lib/tls/libc.so.6
#7  0x080512a7 in errprintf (fmt=0x8059554 "RRD error updating %s from %s: %s\n") at errormsg.c:47
#8  0x0804a824 in create_and_update_rrd (hostname=0xb7563038 "comclor04.lu.mach.com",
    fn=0xffffffff <Address 0xffffffff out of bounds>, creparams=0x805e200,
    template=0x92794b0 "udpInDatagrams:udpOutDatagrams:udpInErrors:tcpActiveOpens:tcpPassiveOpens:tcpAttemptFails:tcpEstabResets:tcpCurrEstab:tcpOutDataBytes:tcpInInorderBytes:tcpInUnorderBytes:tcpRetransBytes:tcpOutDataPack"...) at do_rrd.c:176
#9  0x08050679 in update_rrd (hostname=0xb7563038 "comclor04.lu.mach.com", testname=0xb756304e "netstat",
    msg=0xb756307f "status comclor04,lu,mach,com.netstat green Wed Jan 25 22:22:57 CET 2006  <netstat>\nsolaris\n\nRAWIP\n\trawipInDatagrams    =5006345\trawipInErrors       =     0\n\trawipInCksumErrs    =     0\trawipOutDatagra"..., tstamp=1138224178,
    sender=0xffffffe5 <Address 0xffffffe5 out of bounds>, ldef=0xffffffe5) at do_rrd.c:293
#10 0x08049cbc in main (argc=-27, argv=0xbfff93e4) at hobbitd_rrd.c:199

Sorry found my own previous post that solved this problem. I really need to make logrotate work ;-)

Thomas wrote:

Hello,

I would like to submit you this dump I generated after one of the multiples hobbitd_rrd crashes, if someone could help me to solve this problem :

I use Xymon 4.2.3 on solaris 10 Sparc with Hobbit perl client 1.21.

root at t10admin # gdb /export/xymon/server/bin/hobbitd_rrd /var/cores/core.30102.hobbitd_rrd.1264406521
GNU gdb 6.2.1

Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.

This GDB was configured as "sparc-sun-solaris2.10"...
Core was generated by `hobbitd_rrd --rrddir=/export/xymon/data/rrd'.
Program terminated with signal 6, Aborted.
Reading symbols from /opt/local/lib/librrd.so.2...done.
Loaded symbols for /usr/local/lib/librrd.so.2
Reading symbols from /usr/lib/libpng.so.3...done.
Loaded symbols for /usr/lib/libpng.so.3
Reading symbols from /opt/sfw/lib/libpcre.so.0...done.
Loaded symbols for /opt/sfw/lib/libpcre.so.0
Reading symbols from /usr/lib/libresolv.so.2...done.
Loaded symbols for /usr/lib/libresolv.so.2
Reading symbols from /usr/lib/libsocket.so.1...done.
Loaded symbols for /usr/lib/libsocket.so.1
Reading symbols from /usr/lib/libnsl.so.1...done.
Loaded symbols for /usr/lib/libnsl.so.1
Reading symbols from /usr/lib/librt.so.1...done.
Loaded symbols for /usr/lib/librt.so.1
Reading symbols from /usr/lib/libc.so.1...done.
Loaded symbols for /usr/lib/libc.so.1
Reading symbols from /usr/lib/libpng12.so.0...done.
Loaded symbols for /usr/lib/libpng12.so.0
Reading symbols from /usr/lib/libart_lgpl_2.so.2...done.
Loaded symbols for /usr/lib/libart_lgpl_2.so.2
Reading symbols from /usr/lib/libm.so.2...done.
Loaded symbols for /usr/lib/libm.so.2
Reading symbols from /opt/local/lib/libfreetype.so.6...done.
Loaded symbols for /usr/local/lib/libfreetype.so.6
Reading symbols from /usr/lib/libz.so...done.
Loaded symbols for /usr/lib/libz.so
Reading symbols from /usr/sfw/lib/libgcc_s.so.1...done.
Loaded symbols for /usr/sfw/lib/libgcc_s.so.1
Reading symbols from /usr/lib/libaio.so.1...done.
Loaded symbols for /usr/lib/libaio.so.1
Reading symbols from /usr/lib/libmd.so.1...done.
Loaded symbols for /usr/lib/libmd.so.1
Reading symbols from /platform/SUNW,Sun-Blade-1000/lib/libc_psr.so.1...done.
Loaded symbols for /platform/SUNW,Sun-Blade-1000/lib/libc_psr.so.1
#0  0xff0c599c in _lwp_kill () from /usr/lib/libc.so.1
(gdb) bt
#0  0xff0c599c in _lwp_kill () from /usr/lib/libc.so.1
#1  0xff0649d0 in raise () from /usr/lib/libc.so.1
#2  0xff0410c0 in abort () from /usr/lib/libc.so.1
#3  0x0002447c in sigsegv_handler (signum=11) at sig.c:57
#4  0xff0c48dc in __sighndlr () from /usr/lib/libc.so.1
#5  0xff0b98b4 in call_user_handler () from /usr/lib/libc.so.1
#6  0x000164b0 in do_dbcheck_tablespace_rrd (hostname=0x10dc34 "ERICSE", testname=0x10dc3b "TblSpace",
    classname=0x10dc6f "status+60 ERICSE.TblSpace green lun. janv. 25 09:02:00 2010 \n\n\nTablespace check (def. warning  Use% >= 90%, def. alert  Use% >= 95%)\nTableSpace/DBSpace", ' ' <repeats 11 times>, "Size    Used    Free Use%%   Alloc Act"...,
    pagepaths=0x4b5d4ff8 <Address 0x4b5d4ff8 out of bounds>, msg=0x2cc2a "", tstamp=183328) at do_dbcheck.c:264
#7  0x00017048 in do_disk_rrd (hostname=0x10dc34 "ERICSE", testname=0x10dc3b "TblSpace",
    msg=0x10dc6f "status+60 ERICSE.TblSpace green lun. janv. 25 09:02:00 2010 \n\n\nTablespace check (def. warning  Use% >= 90%, def. alert  Use% >= 95%)\nTableSpace/DBSpace", ' ' <repeats 11 times>, "Size    Used    Free Use%%   Alloc Act"..., tstamp=1264406520) at do_disk.c:30
#8  0x0001ecfc in update_rrd (hostname=0x10dc34 "ERICSE", testname=0x10dc3b "TblSpace",
    msg=0x10dc6f "status+60 ERICSE.TblSpace green lun. janv. 25 09:02:00 2010 \n\n\nTablespace check (def. warning  Use% >= 90%, def. alert  Use% >= 95%)\nTableSpace/DBSpace", ' ' <repeats 11 times>, "Size    Used    Free Use%%   Alloc Act"..., tstamp=1264406520,
    sender=0x10dc29 "10.92.1.4", ldef=0x45800) at do_rrd.c:407
#9  0x00013b64 in main (argc=1104937, argv=0xffbfe81c) at hobbitd_rrd.c:161
(gdb)


Regards,

Cordialement,
 
Frédéric BOURDON