Some service graphs not showing up on host pages 🔗 link

I am converting over from BB to HOBBIT and have encountered the following problem:
(I am pretty new to the whole BB/HOBBIT world)

server1 (RH ES3.0) - Brought up  HOBBIT 4.0.2 & 4.0.3b.  Everything works fine.

server2 (RH ES3.0) - Was running BB and converted to HOBBIT 4.0.2 & 4.0.3b. Everything works fine

server3 (RH ES3.0) - Was running BB and was working fine.  Converted to HOBBIT 4.0.2 & 4.0.3b and the following tests don't end up with a graph on the host pages: conn, disk, memory, ssh, iostat
other test such as cpu, netstat vmstat work fine.

I am running rrdtool-1.2.6 on all machines.  Any idea what is wrong with server3 or how I troubleshoot it?  No errors show up in either /var/log/httpd/error_log or /var/log/hobbit/*.log

Other that this, everything seems to be working well.
Thanks for any help. Winn

SPECIAL NOTICE

All information transmitted hereby is intended only for the use of the
addressee(s) named above and may contain confidential and privileged
information. Any unauthorized review, use, disclosure or distribution
of confidential and privileged information is prohibited. If the reader
of this message is not the intended recipient(s) or the employee or agent
responsible for delivering the message to the intended recipient, you are
hereby notified that you must not read this transmission and that disclosure,
copying, printing, distribution or use of any of the information contained
in or attached to this transmission is STRICTLY PROHIBITED.

Anyone who receives confidential and privileged information in error should
notify us immediately by telephone and mail the original message to us at
the above address and destroy all copies.  To the extent any portion of this
communication contains public information, no such restrictions apply to that
information. (gate01)

On Thu, May 12, 2005 at 02:53:27PM -0600, Winn Beutler wrote:

Just so that I understand - all of these servers run the Hobbit *server*
programs - they are not just clients reporting to a single Hobbit
server?

I am running rrdtool-1.2.6 on all machines.  Any idea what is wrong with server3 or how I troubleshoot it?  No errors show up in either /var/log/httpd/error_log or /var/log/hobbit/*.log

First thing to check is if the server3 RRD-files are generated at all,
since this is the basis of having graphs. Check what is in the
~hobbit/data/rrd/server3/ directory.


Regards,
Henrik

Henrik,
Thanks for the quick reply,
1. Yes all three servers are independent of each other and are running the HOBBIT server SW.

2. The files for the various hosts appear to getting updated regularly.  (Dont know how valid the contents are since they are not appearing.)   Here is a sample of the contents of the 'rrd' directories for three of my hosts:
cd /opt/hobbit/data/rrd/
ls -l kakapo.mcdata.com   orafinjr.mcdata.com  yale.mcdata.com

kakapo.mcdata.com:
total 2080
-rw-r--r--    1 hobbit   hobbit      38532 May 13 07:22 disk,opt.rrd
-rw-r--r--    1 hobbit   hobbit      38532 May 13 07:22 disk,root.rrd
-rw-r--r--    1 hobbit   hobbit      38532 May 13 07:22 disk,tmp.rrd
-rw-r--r--    1 hobbit   hobbit      38532 May 13 07:22 disk,usr.rrd
-rw-r--r--    1 hobbit   hobbit      38532 May 13 07:22 disk,var.rrd
-rw-r--r--    1 hobbit   hobbit      38532 May 13 07:22 disk,var,run.rrd
-rw-r--r--    1 hobbit   hobbit     266340 May 13 07:18 iostat.,opt.rrd
-rw-r--r--    1 hobbit   hobbit     266340 May 13 07:18 iostat.,root.rrd
-rw-r--r--    1 hobbit   hobbit     266340 May 13 07:18 iostat.,usr.rrd
-rw-r--r--    1 hobbit   hobbit     266340 May 13 07:18 iostat.,var.rrd
-rw-r--r--    1 hobbit   hobbit      19548 May 13 07:22 la.rrd
-rw-r--r--    1 hobbit   hobbit      19548 May 13 07:22 memory.actual.rrd
-rw-r--r--    1 hobbit   hobbit      19548 May 13 07:22 memory.real.rrd
-rw-r--r--    1 hobbit   hobbit      19548 May 13 07:22 memory.swap.rrd
-rw-r--r--    1 hobbit   hobbit     304308 May 13 07:22 netstat.rrd
-rw-r--r--    1 hobbit   hobbit      19548 May 13 07:22 procs.rrd
-rw-r--r--    1 hobbit   hobbit      19548 May 13 07:22 tcp.conn.rrd
-rw-r--r--    1 hobbit   hobbit      19548 May 13 07:22 tcp.ssh.rrd
-rw-r--r--    1 hobbit   hobbit      19548 May 13 07:22 users.rrd
-rw-r--r--    1 hobbit   hobbit     304308 May 13 07:18 vmstat.rrd

orafinjr.mcdata.com:
total 5164
-rw-r--r--    1 hobbit   hobbit      38532 May 13 07:21 disk,devdg.rrd
-rw-r--r--    1 hobbit   hobbit      38532 May 12 04:46 disk,mnt,standby,opt.rrd
-rw-r--r--    1 hobbit   hobbit      38532 May 12 04:46 disk,mnt,standby.rrd
-rw-r--r--    1 hobbit   hobbit      38532 May 12 04:46 disk,mnt,standby,usr.rrd
-rw-r--r--    1 hobbit   hobbit      38532 May 12 04:46 disk,mnt,standby,var.rrd
-rw-r--r--    1 hobbit   hobbit      38532 May 13 07:21 disk,newdisk2.rrd
-rw-r--r--    1 hobbit   hobbit      38532 May 13 07:21 disk,newdisk.rrd
-rw-r--r--    1 hobbit   hobbit      38532 May 13 07:21 disk,opt.rrd
-rw-r--r--    1 hobbit   hobbit      38532 Apr 21 14:36 disk,red.rrd
-rw-r--r--    1 hobbit   hobbit      38532 May 13 07:21 disk,root.rrd
-rw-r--r--    1 hobbit   hobbit      38532 May 13 07:21 disk,testdg_app.rrd
-rw-r--r--    1 hobbit   hobbit      38532 May 13 07:21 disk,testdg_db.rrd
-rw-r--r--    1 hobbit   hobbit      38532 May 13 07:21 disk,tmp.rrd
-rw-r--r--    1 hobbit   hobbit      38532 May 13 07:21 disk,usr.rrd
-rw-r--r--    1 hobbit   hobbit      38532 May 13 07:21 disk,var.rrd
-rw-r--r--    1 hobbit   hobbit      38532 May 13 07:21 disk,var,run.rrd
-rw-r--r--    1 hobbit   hobbit     266340 May 13 07:18 iostat.,devdg.rrd
-rw-r--r--    1 hobbit   hobbit     266340 May 12 04:48 iostat.,mnt,standby,opt.rrd
-rw-r--r--    1 hobbit   hobbit     266340 May 12 04:48 iostat.,mnt,standby.rrd
-rw-r--r--    1 hobbit   hobbit     266340 May 12 04:48 iostat.,mnt,standby,usr.rrd
-rw-r--r--    1 hobbit   hobbit     266340 May 12 04:48 iostat.,mnt,standby,var.rrd
d-wxr----t    2 hobbit   hobbit       4096 May 10 14:52 iostat.,ne
-rw-r--r--    1 hobbit   hobbit     266340 May 13 07:18 iostat.,newdisk2.rrd
-rw-r--r--    1 hobbit   hobbit     266340 May 13 07:18 iostat.,newdisk.rrd
-rw-r--r--    1 hobbit   hobbit     266340 May 13 07:18 iostat.,opt.rrd
-rw-r--r--    1 hobbit   hobbit     266340 Apr 22 10:01 iostat..rrd
-rw-r--r--    1 hobbit   hobbit     266340 May 13 07:18 iostat.,testdg_app.rrd
-rw-r--r--    1 hobbit   hobbit     266340 May 13 07:18 iostat.,testdg_db.rrd
-rw-r--r--    1 hobbit   hobbit     266340 May 13 07:18 iostat.,usr.rrd
-rw-r--r--    1 hobbit   hobbit     266340 May 13 07:18 iostat.,var.rrd
-rw-r--r--    1 hobbit   hobbit      19548 May 13 07:21 la.rrd
-rw-r--r--    1 hobbit   hobbit      19548 May 13 07:21 memory.actual.rrd
-rw-r--r--    1 hobbit   hobbit      19548 May 13 07:21 memory.real.rrd
-rw-r--r--    1 hobbit   hobbit      19548 May 13 07:21 memory.swap.rrd
-rw-r--r--    1 hobbit   hobbit     304308 May 13 07:21 netstat.rrd
-rw-r--r--    1 hobbit   hobbit      19548 May 13 07:21 procs.rrd
-rw-r--r--    1 hobbit   hobbit      19548 May 13 07:22 tcp.conn.rrd
-rw-r--r--    1 hobbit   hobbit      19548 May 13 07:22 tcp.ssh.rrd
-rw-r--r--    1 hobbit   hobbit      19548 May 13 07:21 users.rrd
-rw-r--r--    1 hobbit   hobbit     304308 May 13 07:18 vmstat.rrdbit

yale.mcdata.com:
total 944
-rw-r--r--    1 hobbit   hobbit      38532 May 13 07:19 disk,boot.rrd
-rw-r--r--    1 hobbit   hobbit      38532 May 13 07:19 disk,root.rrd
-rw-r--r--    1 hobbit   hobbit      38532 Mar 16 18:22 disk,tmp,a.rrd
-rw-r--r--    1 hobbit   hobbit      38532 May 13 07:19 disk,u01.rrd
-rw-r--r--    1 hobbit   hobbit      19548 May 13 07:19 la.rrd
-rw-r--r--    1 hobbit   hobbit      19548 May 13 07:19 memory.actual.rrd
-rw-r--r--    1 hobbit   hobbit      19548 May 13 07:19 memory.real.rrd
-rw-r--r--    1 hobbit   hobbit      19548 May 13 07:19 memory.swap.rrd
-rw-r--r--    1 hobbit   hobbit     304308 May 13 07:19 netstat.rrd
-rw-r--r--    1 hobbit   hobbit      19548 May 13 07:19 procs.rrd
-rw-r--r--    1 hobbit   hobbit      19548 May 13 07:22 tcp.conn.rrd
-rw-r--r--    1 hobbit   hobbit      19548 May 13 07:22 tcp.ssh.rrd
-rw-r--r--    1 hobbit   hobbit      19548 May 13 07:19 users.rrd
-rw-r--r--    1 hobbit   hobbit     323292 May 13 07:21 vmstat.rrd

-----Original Message-----
From: Henrik Stoerner [mailto:user-ce4a2c883f75@xymon.invalid]
Sent: Thursday, May 12, 2005 11:05 PM
To: user-ae9b8668bcde@xymon.invalid
Subject: Re: [hobbit] Some service graphs not showing up on host pages


On Thu, May 12, 2005 at 02:53:27PM -0600, Winn Beutler wrote:

Just so that I understand - all of these servers run the Hobbit *server*
programs - they are not just clients reporting to a single Hobbit
server?

First thing to check is if the server3 RRD-files are generated at all,
since this is the basis of having graphs. Check what is in the
~hobbit/data/rrd/server3/ directory.


Regards,
Henrik


SPECIAL NOTICE

All information transmitted hereby is intended only for the use of the
addressee(s) named above and may contain confidential and privileged
information. Any unauthorized review, use, disclosure or distribution
of confidential and privileged information is prohibited. If the reader
of this message is not the intended recipient(s) or the employee or agent
responsible for delivering the message to the intended recipient, you are
hereby notified that you must not read this transmission and that disclosure,
copying, printing, distribution or use of any of the information contained
in or attached to this transmission is STRICTLY PROHIBITED.

Anyone who receives confidential and privileged information in error should
notify us immediately by telephone and mail the original message to us at
the above address and destroy all copies.  To the extent any portion of this
communication contains public information, no such restrictions apply to that
information. (gate01)

On Fri, May 13, 2005 at 07:30:18AM -0600, Winn Beutler wrote:

Odd, really. On the status-pages where the graphs should have been, do
you get a "broken image" link ? Or if you do a "view source" on eg the
cpu status page, is there a section like this:

<IMG BORDER=0 SRC="/hobbit-cgi/hobbitgraph.sh?host=somehostname&amp;service=la ....


If there is, it would be interesting to know if the web server logs have
logged any errors from the hobbitgraph.sh CGI script.


Regards,
Henrik

Henrik,
Since it was working well on 2 other machines, I have migrated off the problem server onto another and it too works properly.  I am giving up on trying to figure out what is wrong with the bad server.  I think it is a library problem with rrdtool.
Thanks for trying to help - Winn

-----Original Message-----
From: Henrik Stoerner [mailto:user-ce4a2c883f75@xymon.invalid]
Sent: Monday, May 16, 2005 2:41 PM
To: user-ae9b8668bcde@xymon.invalid
Subject: Re: [hobbit] Some service graphs not showing up on host pages


On Fri, May 13, 2005 at 07:30:18AM -0600, Winn Beutler wrote:

Odd, really. On the status-pages where the graphs should have been, do
you get a "broken image" link ? Or if you do a "view source" on eg the
cpu status page, is there a section like this:

<IMG BORDER=0 SRC="/hobbit-cgi/hobbitgraph.sh?host=somehostname&amp;service=la ....


If there is, it would be interesting to know if the web server logs have
logged any errors from the hobbitgraph.sh CGI script.


Regards,
Henrik


SPECIAL NOTICE

All information transmitted hereby is intended only for the use of the
addressee(s) named above and may contain confidential and privileged
information. Any unauthorized review, use, disclosure or distribution
of confidential and privileged information is prohibited. If the reader
of this message is not the intended recipient(s) or the employee or agent
responsible for delivering the message to the intended recipient, you are
hereby notified that you must not read this transmission and that disclosure,
copying, printing, distribution or use of any of the information contained
in or attached to this transmission is STRICTLY PROHIBITED.

Anyone who receives confidential and privileged information in error should
notify us immediately by telephone and mail the original message to us at
the above address and destroy all copies.  To the extent any portion of this
communication contains public information, no such restrictions apply to that
information. (gate01)

Unfortunately after upgrading to solaris 10 we have had some 
difficulties, just running hobbit causes the server to become 
unresponsive including from the console.  So I thought I should do a 
recompile which ends like so...

gcc -g -O2 -Wall -Wno-unused -D_REENTRANT -DHAVE_RPCENT -DSunOS 
-DMAXMSG=32768 -DBBDPORTNUMBER=1984 -I. 
-I/var/tmp/hobbit-4.0.3rc2/include -I. -I../include    -c -o misc.o misc.c
In file included from /usr/include/sys/wait.h:24,
                  from misc.c:20:
/usr/include/sys/siginfo.h:259: error: parse error before "ctid_t"
/usr/include/sys/siginfo.h:292: error: parse error before '}' token
/usr/include/sys/siginfo.h:294: error: parse error before '}' token
/usr/include/sys/siginfo.h:390: error: parse error before "ctid_t"
/usr/include/sys/siginfo.h:392: error: conflicting types for `__proc'
/usr/include/sys/siginfo.h:261: error: previous declaration of `__proc'
/usr/include/sys/siginfo.h:398: error: conflicting types for `__fault'
/usr/include/sys/siginfo.h:267: error: previous declaration of `__fault'
/usr/include/sys/siginfo.h:404: error: conflicting types for `__file'
/usr/include/sys/siginfo.h:273: error: previous declaration of `__file'
/usr/include/sys/siginfo.h:420: error: conflicting types for `__prof'
/usr/include/sys/siginfo.h:287: error: previous declaration of `__prof'
/usr/include/sys/siginfo.h:424: error: conflicting types for `__rctl'
/usr/include/sys/siginfo.h:291: error: previous declaration of `__rctl'
/usr/include/sys/siginfo.h:426: error: parse error before '}' token
/usr/include/sys/siginfo.h:428: error: parse error before '}' token
/usr/include/sys/siginfo.h:432: error: parse error before "k_siginfo_t"
/usr/include/sys/siginfo.h:437: error: parse error before '}' token
In file included from /usr/include/sys/procset.h:24,
                  from /usr/include/sys/wait.h:25,
                  from misc.c:20:
/usr/include/sys/signal.h:85: error: parse error before "siginfo_t"
In file included from misc.c:20:
/usr/include/sys/wait.h:86: error: parse error before "siginfo_t"
make[1]: *** [misc.o] Error 1
make[1]: Leaving directory `/var/tmp/hobbit-4.0.3rc2/lib'
make: *** [lib-build] Error 2

Any ideas?

David

David Gore (v965-3670)
Enhanced Technology Support (ETS)
Network Management Systems (NMS)
IMPACT Transport Team Lead - SCSA, SCNA
Page: 1-800-PAG-eMCI pin 1406090
Vnet: 965-3676

David Gore wrote:

Solution:

root at hobbit:/usr/local/lib/gcc-lib/sparc-sun-solaris2.10/3.3.2/install-tools> 
./mkheaders
fixproto: populating 
`/usr/local/lib/gcc-lib/sparc-sun-solaris2.10/3.3.2/include'
(No *.h files in 
/usr/local/lib/gcc-lib/sparc-sun-solaris2.10/3.3.2/include/root)
(No *.h files in 
/usr/local/lib/gcc-lib/sparc-sun-solaris2.10/3.3.2/include/root/usr)
(No *.h files in 
/usr/local/lib/gcc-lib/sparc-sun-solaris2.10/3.3.2/include/root/usr/openwin)
(No *.h files in 
/usr/local/lib/gcc-lib/sparc-sun-solaris2.10/3.3.2/include/root/usr/openwin/include)
(No *.h files in 
/usr/local/lib/gcc-lib/sparc-sun-solaris2.10/3.3.2/include/root/usr/openwin/include/X11)
(No *.h files in 
/usr/local/lib/gcc-lib/sparc-sun-solaris2.10/3.3.2/include/X11)

Compiles fine now.