trends columns all turned purple 🔗 link

Deal, Richard wrote:

Things looked for a little bit then all the trends columns turned
purple.
Any one else see this or better yet solve it?
Thanks

-----Original Message-----
From: David Gore [mailto:user-0ef10ea25696@xymon.invalid] Sent: Thursday, March 31, 2005 5:53 PM
To: user-ae9b8668bcde@xymon.invalid
Subject: Re: [hobbit] trends columns all turned purple

Deal, Richard wrote:

Things looked for a little bit then all the trends columns turned
purple.
Any one else see this or better yet solve it?
Thanks

I am seeing the same thing.  I do not know how to fix it.  Has only occurred since installing 4.0.1.  Running on solaris 9.

~David

I am running 4.0 on solaris 9.
I did a hobbit.sh stop 
Then a rm data/hist/*.trends 
And restarted hobbit

It has cleared up the problem for now but it has only been 10 mins or
so.

Has anyone else notices maint.pl lists now hosts, only "ALL"?


-----Original Message-----
From: David Gore [mailto:user-0ef10ea25696@xymon.invalid] 
Sent: Thursday, March 31, 2005 5:53 PM
To: user-ae9b8668bcde@xymon.invalid
Subject: Re: [hobbit] trends columns all turned purple

Deal, Richard wrote:

Things looked for a little bit then all the trends columns turned
purple.
Any one else see this or better yet solve it?
Thanks

I am seeing the same thing. I do not know how to fix it. Has only 
occurred since installing 4.0.1. Running on solaris 9.

~David

-----Original Message-----
From: Henrik Stoerner [mailto:user-ce4a2c883f75@xymon.invalid] 
Sent: Friday, April 01, 2005 1:34 AM
To: user-ae9b8668bcde@xymon.invalid
Subject: Re: [hobbit] trends columns all turned purple

On Thu, Mar 31, 2005 at 06:04:14PM -0500, Deal, Richard wrote:

[trends column goes purple]

Has anyone else notices maint.pl lists now hosts, only "ALL"?

Neither of these show up on any of the hosts I've tested. sounds
like there's some problem with the hobbit daemon on your box,
could you try stopping Hobbit, then do a "ps" to make sure all of the
tasks have stopped, then restarting it ?

Regards,
Henrik

Deal, Richard wrote:

I tried but I still don't get an actual list of hosts just the option
for "ALL".
The trends column has remained green though.  It may have been from
starting and stopping a few times when I was testing at the beginning
that the history logs were created.  
-----Original Message-----
From: Henrik Stoerner [mailto:user-ce4a2c883f75@xymon.invalid] Sent: Friday, April 01, 2005 1:34 AM
To: user-ae9b8668bcde@xymon.invalid
Subject: Re: [hobbit] trends columns all turned purple

On Thu, Mar 31, 2005 at 06:04:14PM -0500, Deal, Richard wrote:

[trends column goes purple]

Has anyone else notices maint.pl lists now hosts, only "ALL"?

Neither of these show up on any of the hosts I've tested. sounds
like there's some problem with the hobbit daemon on your box,
could you try stopping Hobbit, then do a "ps" to make sure all of the
tasks have stopped, then restarting it ?

Regards,
Henrik

The maint.pl lists and 'ALL' may still be a problem, it does occur in
both Firefox and IE.  It can be alleviated by changing views.
Typically we monitor bb2.html, but if we switch to the Main view and
try enable/disable, it typically works, this only occurred since RC6,
I think.

-----Original Message-----
From: David Gore [mailto:user-0ef10ea25696@xymon.invalid] Sent: Friday, April 01, 2005 12:08 PM
To: user-ae9b8668bcde@xymon.invalid
Subject: Re: [hobbit] trends columns all turned purple

I re-installed RC6 to get rid of the purple trends.  The maint.pl lists and 'ALL' may still be a problem, it does occur in both Firefox and IE.   It can be alleviated by changing views.  Typically we monitor bb2.html, but if we switch to the Main view and try enable/disable, it typically works, this only occurred since RC6, I think.

Richard, you are running Solaris 9 also?

~David


Deal, Richard wrote:

I tried but I still don't get an actual list of hosts just the option
for "ALL".
The trends column has remained green though.  It may have been from
starting and stopping a few times when I was testing at the beginning
that the history logs were created.  
-----Original Message-----
From: Henrik Stoerner [mailto:user-ce4a2c883f75@xymon.invalid] Sent: Friday, April 01, 2005 1:34 AM
To: user-ae9b8668bcde@xymon.invalid
Subject: Re: [hobbit] trends columns all turned purple

On Thu, Mar 31, 2005 at 06:04:14PM -0500, Deal, Richard wrote:

[trends column goes purple]

Has anyone else notices maint.pl lists now hosts, only "ALL"?

Neither of these show up on any of the hosts I've tested. sounds
like there's some problem with the hobbit daemon on your box,
could you try stopping Hobbit, then do a "ps" to make sure all of the
tasks have stopped, then restarting it ?

Regards,
Henrik

 

-----Original Message-----
From: Henrik Storner [mailto:user-ce4a2c883f75@xymon.invalid] 
Sent: Friday, April 01, 2005 12:19 PM
To: user-ae9b8668bcde@xymon.invalid
Subject: Re: [hobbit] trends columns all turned purple

OK, I think there are two completely un-related issues here.


In <user-87577d67ef7d@xymon.invalid> David Gore <user-0ef10ea25696@xymon.invalid> writes:

I re-installed RC6 to get rid of the purple trends.  

The "trends" don't go purple by themselves, they go purple because the
bb-larrdcolumn tool doesn't update them.

I'd like you to check for unusual messages in the
/var/log/hobbit/bb-display.log file, and also for any core-files left
behind from bb-larrdcolumn. They should be in the ~/server/tmp/
directory, but please check in the ~/data/rrd/ directory also.

The maint.pl lists and 'ALL' may still be a problem, it does occur in
both Firefox and IE.  It can be alleviated by changing views.
Typically we monitor bb2.html, but if we switch to the Main view and
try enable/disable, it typically works, this only occurred since RC6,
I think.

The missing hosts on the maint.pl display sounds like you're still
running the version of maint.pl that uses cookies to try and display
only the hosts from the page you were on. This was removed in the 4.0
release precisely because it was causing problems. Could you check
your maint.pl script and see if lines 432-433 look like this:

    432   # open (HOBBITDLIST, "bb ".$BBENV{'BBDISP'}." \"hobbitdboard
".$filter."\" |");
    433   open (HOBBITDLIST, "bb ".$BBENV{'BBDISP'}." hobbitdboard |");


Thanks,
Henrik

-----Original Message-----
From: Henrik Storner [mailto:user-ce4a2c883f75@xymon.invalid] 
Sent: Friday, April 01, 2005 12:19 PM
To: user-ae9b8668bcde@xymon.invalid
Subject: Re: [hobbit] trends columns all turned purple

OK, I think there are two completely un-related issues here.


In <user-87577d67ef7d@xymon.invalid> David Gore <user-0ef10ea25696@xymon.invalid> writes:

I re-installed RC6 to get rid of the purple trends.  

The "trends" don't go purple by themselves, they go purple because the
bb-larrdcolumn tool doesn't update them.

I'd like you to check for unusual messages in the
/var/log/hobbit/bb-display.log file, and also for any core-files left
behind from bb-larrdcolumn. They should be in the ~/server/tmp/
directory, but please check in the ~/data/rrd/ directory also.

The maint.pl lists and 'ALL' may still be a problem, it does occur in
both Firefox and IE.  It can be alleviated by changing views.
Typically we monitor bb2.html, but if we switch to the Main view and
try enable/disable, it typically works, this only occurred since RC6,
I think.

The missing hosts on the maint.pl display sounds like you're still
running the version of maint.pl that uses cookies to try and display
only the hosts from the page you were on. This was removed in the 4.0
release precisely because it was causing problems. Could you check
your maint.pl script and see if lines 432-433 look like this:

    432   # open (HOBBITDLIST, "bb ".$BBENV{'BBDISP'}." \"hobbitdboard
".$filter."\" |");
    433   open (HOBBITDLIST, "bb ".$BBENV{'BBDISP'}." hobbitdboard |");


Thanks,
Henrik

Henrik Storner wrote:

OK, I think there are two completely un-related issues here.


In <user-87577d67ef7d@xymon.invalid> David Gore <user-0ef10ea25696@xymon.invalid> writes:

I re-installed RC6 to get rid of the purple trends.  

The "trends" don't go purple by themselves, they go purple because the
bb-larrdcolumn tool doesn't update them.

I'd like you to check for unusual messages in the
/var/log/hobbit/bb-display.log file, and also for any core-files left
behind from bb-larrdcolumn. They should be in the ~/server/tmp/
directory, but please check in the ~/data/rrd/ directory also.

The maint.pl lists and 'ALL' may still be a problem, it does occur in
both Firefox and IE.  It can be alleviated by changing views.
Typically we monitor bb2.html, but if we switch to the Main view and
try enable/disable, it typically works, this only occurred since RC6,
I think.

The missing hosts on the maint.pl display sounds like you're still
running the version of maint.pl that uses cookies to try and display
only the hosts from the page you were on. This was removed in the 4.0
release precisely because it was causing problems. Could you check
your maint.pl script and see if lines 432-433 look like this:

    432   # open (HOBBITDLIST, "bb ".$BBENV{'BBDISP'}." \"hobbitdboard ".$filter."\" |");
    433   open (HOBBITDLIST, "bb ".$BBENV{'BBDISP'}." hobbitdboard |");


Thanks,
Henrik

David Gore wrote:

Yes one core file:

hobbit at hobbit ~/server> find . -name core
./tmp/core
hobbit at hobbit ~/server> file tmp/core
tmp/core:       ELF 32-bit MSB core file SPARC Version 1, from 'hobbitd'
hobbit at hobbit ~/server> ls -al tmp/core
-rw-------    1 hobbit   other     8322084 Apr  1 03:27 tmp/core

I checked to make sure the lines were correct in maint.pl.  I am 
reinstalling 4.0.1, again and will let you know how it goes.

~David

Henrik Storner wrote:

OK, I think there are two completely un-related issues here.


In <user-87577d67ef7d@xymon.invalid> David Gore <user-0ef10ea25696@xymon.invalid> writes:

I re-installed RC6 to get rid of the purple trends.  

The "trends" don't go purple by themselves, they go purple because the
bb-larrdcolumn tool doesn't update them.

I'd like you to check for unusual messages in the
/var/log/hobbit/bb-display.log file, and also for any core-files left
behind from bb-larrdcolumn. They should be in the ~/server/tmp/
directory, but please check in the ~/data/rrd/ directory also.

The maint.pl lists and 'ALL' may still be a problem, it does occur in
both Firefox and IE.  It can be alleviated by changing views.
Typically we monitor bb2.html, but if we switch to the Main view and
try enable/disable, it typically works, this only occurred since RC6,
I think.

The missing hosts on the maint.pl display sounds like you're still
running the version of maint.pl that uses cookies to try and display
only the hosts from the page you were on. This was removed in the 4.0
release precisely because it was causing problems. Could you check
your maint.pl script and see if lines 432-433 look like this:

    432   # open (HOBBITDLIST, "bb ".$BBENV{'BBDISP'}." \"hobbitdboard 
".$filter."\" |");
    433   open (HOBBITDLIST, "bb ".$BBENV{'BBDISP'}." hobbitdboard |");


Thanks,
Henrik

./tmp/core
hobbit at hobbit ~/server> file ./tmp/core
./tmp/core:     ELF 32-bit MSB core file SPARC Version 1, from 'hobbitd'

David Gore wrote:

David Gore wrote:

Yes one core file:

hobbit at hobbit ~/server> find . -name core
./tmp/core
hobbit at hobbit ~/server> file tmp/core
tmp/core:       ELF 32-bit MSB core file SPARC Version 1, from 'hobbitd'
hobbit at hobbit ~/server> ls -al tmp/core
-rw-------    1 hobbit   other     8322084 Apr  1 03:27 tmp/core

I checked to make sure the lines were correct in maint.pl.  I am 
reinstalling 4.0.1, again and will let you know how it goes.

~David

Henrik Storner wrote:

OK, I think there are two completely un-related issues here.


In <user-87577d67ef7d@xymon.invalid> David Gore <user-0ef10ea25696@xymon.invalid> writes:

I re-installed RC6 to get rid of the purple trends.  

The "trends" don't go purple by themselves, they go purple because the
bb-larrdcolumn tool doesn't update them.

I'd like you to check for unusual messages in the
/var/log/hobbit/bb-display.log file, and also for any core-files left
behind from bb-larrdcolumn. They should be in the ~/server/tmp/
directory, but please check in the ~/data/rrd/ directory also.

The maint.pl lists and 'ALL' may still be a problem, it does occur in
both Firefox and IE.  It can be alleviated by changing views.
Typically we monitor bb2.html, but if we switch to the Main view and
try enable/disable, it typically works, this only occurred since RC6,
I think.

The missing hosts on the maint.pl display sounds like you're still
running the version of maint.pl that uses cookies to try and display
only the hosts from the page you were on. This was removed in the 4.0
release precisely because it was causing problems. Could you check
your maint.pl script and see if lines 432-433 look like this:

    432   # open (HOBBITDLIST, "bb ".$BBENV{'BBDISP'}." 
\"hobbitdboard ".$filter."\" |");
    433   open (HOBBITDLIST, "bb ".$BBENV{'BBDISP'}." hobbitdboard |");


Thanks,
Henrik

New core and the purples are back:

find -namehobbit at hobbit ~/server> find -name core -ls
141454 8024 -rw-------   1 hobbit   other     8207396 Apr  1 22:08 
./tmp/core
hobbit at hobbit ~/server> file ./tmp/core
./tmp/core:     ELF 32-bit MSB core file SPARC Version 1, from 'hobbitd'

David Gore wrote:

David Gore wrote:

Yes one core file:

hobbit at hobbit ~/server> find . -name core
./tmp/core
hobbit at hobbit ~/server> file tmp/core
tmp/core:       ELF 32-bit MSB core file SPARC Version 1, from 'hobbitd'
hobbit at hobbit ~/server> ls -al tmp/core
-rw-------    1 hobbit   other     8322084 Apr  1 03:27 tmp/core

I checked to make sure the lines were correct in maint.pl.  I am 
reinstalling 4.0.1, again and will let you know how it goes.

~David

Henrik Storner wrote:

OK, I think there are two completely un-related issues here.


In <user-87577d67ef7d@xymon.invalid> David Gore <user-0ef10ea25696@xymon.invalid> writes:

I re-installed RC6 to get rid of the purple trends.  

The "trends" don't go purple by themselves, they go purple because the
bb-larrdcolumn tool doesn't update them.

I'd like you to check for unusual messages in the
/var/log/hobbit/bb-display.log file, and also for any core-files left
behind from bb-larrdcolumn. They should be in the ~/server/tmp/
directory, but please check in the ~/data/rrd/ directory also.

The maint.pl lists and 'ALL' may still be a problem, it does occur in
both Firefox and IE.  It can be alleviated by changing views.
Typically we monitor bb2.html, but if we switch to the Main view and
try enable/disable, it typically works, this only occurred since RC6,
I think.

The missing hosts on the maint.pl display sounds like you're still
running the version of maint.pl that uses cookies to try and display
only the hosts from the page you were on. This was removed in the 4.0
release precisely because it was causing problems. Could you check
your maint.pl script and see if lines 432-433 look like this:

    432   # open (HOBBITDLIST, "bb ".$BBENV{'BBDISP'}." 
\"hobbitdboard ".$filter."\" |");
    433   open (HOBBITDLIST, "bb ".$BBENV{'BBDISP'}." hobbitdboard |");


Thanks,
Henrik

New core and the purples are back:

find -namehobbit at hobbit ~/server> find -name core -ls
141454 8024 -rw-------   1 hobbit   other     8207396 Apr  1 22:08 
./tmp/core
hobbit at hobbit ~/server> file ./tmp/core
./tmp/core:     ELF 32-bit MSB core file SPARC Version 1, from 'hobbitd'

On Fri, Apr 01, 2005 at 09:51:45PM +0000, David Gore wrote:

Yes one core file:

hobbit at hobbit ~/server> find . -name core
./tmp/core
hobbit at hobbit ~/server> file tmp/core
tmp/core:       ELF 32-bit MSB core file SPARC Version 1, from 'hobbitd'
hobbit at hobbit ~/server> ls -al tmp/core
-rw-------    1 hobbit   other     8322084 Apr  1 03:27 tmp/core

I checked to make sure the lines were correct in maint.pl.  I am 
reinstalling 4.0.1, again and will let you know how it goes.

Henrik Stoerner wrote:

On Fri, Apr 01, 2005 at 09:51:45PM +0000, David Gore wrote:
 

Yes one core file:

hobbit at hobbit ~/server> find . -name core
./tmp/core
hobbit at hobbit ~/server> file tmp/core
tmp/core:       ELF 32-bit MSB core file SPARC Version 1, from 'hobbitd'
hobbit at hobbit ~/server> ls -al tmp/core
-rw-------    1 hobbit   other     8322084 Apr  1 03:27 tmp/core

I checked to make sure the lines were correct in maint.pl.  I am 
reinstalling 4.0.1, again and will let you know how it goes.
   

OK, so it does dump core.

To get some more info about this, you need the core file *and* the
hobbitd binary that generated it. Then run

  $ gdb bin/hobbitd tmp/core
  [messages from gdb]
  gdb> bt

to load the core file and the hobbitd binary into gdb (the GNU
debugger), and the the "bt" command will provide a call trace of what
happened when the program crashed, that is the first piece of
information that is needed to find the bug.


Regards,
Henrik

David Gore wrote:

Here is the first core trace after the re-install and start 4.0.1:

hobbit at hobbit ~/server> gdb bin/hobbitd tmp/core
GNU gdb 6.0
Copyright 2003 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "sparc-sun-solaris2.9"...
Core was generated by `hobbitd --restart=/export/home/hobbit/server/tmp/hobbitd.chk --checkpoint-file='.
Program terminated with signal 6, Aborted.
Reading symbols from /usr/lib/libresolv.so.2...done.
Loaded symbols for /usr/lib/libresolv.so.2
Reading symbols from /usr/lib/libsocket.so.1...done.
Loaded symbols for /usr/lib/libsocket.so.1
Reading symbols from /usr/lib/libnsl.so.1...done.
Loaded symbols for /usr/lib/libnsl.so.1
Reading symbols from /usr/lib/libc.so.1...done.
Loaded symbols for /usr/lib/libc.so.1
Reading symbols from /usr/lib/libdl.so.1...done.
Loaded symbols for /usr/lib/libdl.so.1
Reading symbols from /usr/lib/libmp.so.2...done.
Loaded symbols for /usr/lib/libmp.so.2
Reading symbols from /usr/platform/SUNW,Ultra-4/lib/libc_psr.so.1...done.
Loaded symbols for /usr/platform/SUNW,Ultra-4/lib/libc_psr.so.1
#0  0xff19d3d4 in _libc_kill () from /usr/lib/libc.so.1
(gdb) bt
#0  0xff19d3d4 in _libc_kill () from /usr/lib/libc.so.1
#1  0xff135698 in abort () from /usr/lib/libc.so.1
#2  0x0001db68 in sigsegv_handler (signum=10) at sig.c:57
#3  <signal handler called>


Henrik Stoerner wrote:

On Fri, Apr 01, 2005 at 09:51:45PM +0000, David Gore wrote:
 

Yes one core file:

hobbit at hobbit ~/server> find . -name core
./tmp/core
hobbit at hobbit ~/server> file tmp/core
tmp/core:       ELF 32-bit MSB core file SPARC Version 1, from 'hobbitd'
hobbit at hobbit ~/server> ls -al tmp/core
-rw-------    1 hobbit   other     8322084 Apr  1 03:27 tmp/core

I checked to make sure the lines were correct in maint.pl.  I am reinstalling 4.0.1, again and will let you know how it goes.
  

OK, so it does dump core.

To get some more info about this, you need the core file *and* the
hobbitd binary that generated it. Then run

  $ gdb bin/hobbitd tmp/core
  [messages from gdb]
  gdb> bt

to load the core file and the hobbitd binary into gdb (the GNU
debugger), and the the "bt" command will provide a call trace of what
happened when the program crashed, that is the first piece of
information that is needed to find the bug.


Regards,
Henrik

David Gore wrote:

David Gore wrote:

Here is the first core trace after the re-install and start 4.0.1:

hobbit at hobbit ~/server> gdb bin/hobbitd tmp/core
GNU gdb 6.0
Copyright 2003 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "sparc-sun-solaris2.9"...
Core was generated by `hobbitd --restart=/export/home/hobbit/server/tmp/hobbitd.chk --checkpoint-file='.
Program terminated with signal 6, Aborted.
Reading symbols from /usr/lib/libresolv.so.2...done.
Loaded symbols for /usr/lib/libresolv.so.2
Reading symbols from /usr/lib/libsocket.so.1...done.
Loaded symbols for /usr/lib/libsocket.so.1
Reading symbols from /usr/lib/libnsl.so.1...done.
Loaded symbols for /usr/lib/libnsl.so.1
Reading symbols from /usr/lib/libc.so.1...done.
Loaded symbols for /usr/lib/libc.so.1
Reading symbols from /usr/lib/libdl.so.1...done.
Loaded symbols for /usr/lib/libdl.so.1
Reading symbols from /usr/lib/libmp.so.2...done.
Loaded symbols for /usr/lib/libmp.so.2
Reading symbols from /usr/platform/SUNW,Ultra-4/lib/libc_psr.so.1...done.
Loaded symbols for /usr/platform/SUNW,Ultra-4/lib/libc_psr.so.1
#0  0xff19d3d4 in _libc_kill () from /usr/lib/libc.so.1
(gdb) bt
#0  0xff19d3d4 in _libc_kill () from /usr/lib/libc.so.1
#1  0xff135698 in abort () from /usr/lib/libc.so.1
#2  0x0001db68 in sigsegv_handler (signum=10) at sig.c:57
#3  <signal handler called>


Henrik Stoerner wrote:

On Fri, Apr 01, 2005 at 09:51:45PM +0000, David Gore wrote:
 

Yes one core file:

hobbit at hobbit ~/server> find . -name core
./tmp/core
hobbit at hobbit ~/server> file tmp/core
tmp/core:       ELF 32-bit MSB core file SPARC Version 1, from 'hobbitd'
hobbit at hobbit ~/server> ls -al tmp/core
-rw-------    1 hobbit   other     8322084 Apr  1 03:27 tmp/core

I checked to make sure the lines were correct in maint.pl.  I am reinstalling 4.0.1, again and will let you know how it goes.
  

OK, so it does dump core.

To get some more info about this, you need the core file *and* the
hobbitd binary that generated it. Then run

  $ gdb bin/hobbitd tmp/core
  [messages from gdb]
  gdb> bt

to load the core file and the hobbitd binary into gdb (the GNU
debugger), and the the "bt" command will provide a call trace of what
happened when the program crashed, that is the first piece of
information that is needed to find the bug.


Regards,
Henrik

Based on the core trace, I decided to remove all files from ~/server/tmp/.  I do not think we had removed any files from running RC1,2,4-6 from '~/server/tmp/' so perhaps it was choking on something (hobbitd.chk)?  It has been running clean for more than an hour.  It typically dumps core before an hour has passed and the purple trends occur before an hour of running too.  I did have to re-disable my disabled hosts of course.  So far it looks good!  Thanks for your help Henrik.

~David