hobbitclient stopped reporting
list Longina Przybyszewska
After server reboot hobbitclient stopped reported and turned PURPLE. Hobbitclientd is running properly and is gathering data in tmp. It is not name related problem - in bb-hosts there is full qualified name and CLIENT:nodename. Before rebootstraping there was YELLOW status on disk monitoring. I really can't see in logs any help to explain the problem. It is rather important to get it up again - it is very important server. regards Longina -- Longina Przybyszewska, system programmer Dept. of Math. & Comp. Sci. - IMADA University of Southern Denmark, Odense Campusvej 55,DK-5230 Odense M, Denmark tel: +XX XXXX XXXX - http://www.imada.sdu.dk email: user-db0a93c0e294@xymon.invalid --
list Henrik Størner
▸
On Sat, Apr 22, 2006 at 11:40:30AM +0200, Longina Przybyszewska wrote:
After server reboot hobbitclient stopped reported and turned PURPLE. Hobbitclientd is running properly and is gathering data in tmp. It is not name related problem - in bb-hosts there is full qualified name and CLIENT:nodename. Before rebootstraping there was YELLOW status on disk monitoring. I really can't see in logs any help to explain the problem. It is rather important to get it up again - it is very important server.
Check that you have "BBHOSTS=2" set in hobbitserver.cfg in the server side. If not, change it and restart Hobbit. Wait 5 minutes, then check the "hobbitd" status to see if your client is listed there - if it is, then the hostname used by the client doesn't match what is in bb-hosts. On the client side, see if you can telnet to the Hobbit server on port 1984. If not, something is blocking data from going to your Hobbit server. Finally, on the client you can try this. On the Hobbit SERVER, login as the "hobbit" user and run bbcmd hobbitd_channel --channel=client grep "@@" Then on the CLIENT, setup the Hobbit client environment settings with: $ HOBBITCLIENTHOME=/usr/lib/hobbit/client bbcmd --env=/usr/lib/hobbit/client/etc/hobbitclient.cfg Then try running the command that sends a status to the server: $ $BB $BBDISP "@" <$BBTMP/msg.txt On the Hobbit SERVER, you should then see the header of the client message, like this: @@client#506|1145699641.027178|127.0.0.1|localhost|linux Does that give you any ideas ? Henrik
list Longina Przybyszewska
▸
-- On Sat, 22 Apr 2006, Henrik Stoerner wrote:
On Sat, Apr 22, 2006 at 11:40:30AM +0200, Longina Przybyszewska wrote:After server reboot hobbitclient stopped reported and turned PURPLE. Hobbitclientd is running properly and is gathering data in tmp. It is not name related problem - in bb-hosts there is full qualified name and CLIENT:nodename. Before rebootstraping there was YELLOW status on disk monitoring. I really can't see in logs any help to explain the problem. It is rather important to get it up again - it is very important server.Check that you have "BBHOSTS=2" set in hobbitserver.cfg in the server side. If not, change it and restart Hobbit. Wait 5 minutes, then check the "hobbitd" status to see if your client is listed there - if it is, then the hostname used by the client doesn't match what is in bb-hosts.
you mean BBGHOSTS ? it is set , and client er ikke listed there.
▸
On the client side, see if you can telnet to the Hobbit server on port 1984. If not, something is blocking data from going to your Hobbit server.
I can telnet host 1984: --- Trying 130.225.128.14... Connected to 130.225.128.14. Escape character is '^]'. Connection closed by foreign host.
▸
----
Finally, on the client you can try this. On the Hobbit SERVER, login as the "hobbit" user and run bbcmd hobbitd_channel --channel=client grep "@@" Then on the CLIENT, setup the Hobbit client environment settings with: $ HOBBITCLIENTHOME=/usr/lib/hobbit/client bbcmd --env=/usr/lib/hobbit/client/etc/hobbitclient.cfg Then try running the command that sends a status to the server: $ $BB $BBDISP "@" <$BBTMP/msg.txt On the Hobbit SERVER, you should then see the header of the client message, like this: @@client#506|1145699641.027178|127.0.0.1|localhost|linux
2006-04-22 15:37:39 Recipient listed as '130.225.128.14' 2006-04-22 15:37:39 Standard BB protocol on port 1984 2006-04-22 15:37:39 Will connect to address 130.225.128.14 port 1984 2006-04-22 15:37:39 Connect status is 0 2006-04-22 15:37:39 Sent 11971 bytes 2006-04-22 15:37:39 Closing connection 2006-04-22 15:37:39 15 status messages merged into 7 transmissions 2006-04-22 15:37:39 Recipient listed as '130.225.128.14' 2006-04-22 15:37:39 Standard BB protocol on port 1984 2006-04-22 15:37:39 Will connect to address 130.225.128.14 port 1984 2006-04-22 15:37:39 Connect status is 0 2006-04-22 15:37:39 Sent 1038 bytes 2006-04-22 15:37:39 Closing connection 2006-04-22 15:37:39 Recipient listed as '130.225.128.14' 2006-04-22 15:37:39 Standard BB protocol on port 1984 2006-04-22 15:37:39 Will connect to address 130.225.128.14 port 1984 2006-04-22 15:37:39 Connect status is 0 2006-04-22 15:37:39 Sent 116 bytes 2006-04-22 15:37:39 Closing connection 2006-04-22 15:37:47 hobbitd_client: Got message 165 @@client#165|1145713067.445669|130.225.128.9|bach.imada.sdu.dk|sunos 2006-04-22 15:37:47 2006-04-22 15:37:47 Worker process died with exit code 134, terminating Sometimes I can see that data come through, but sometimes I can see "worker process died". Server is Sparc/solaris9 - not enough resources? Longina
list Henrik Størner
▸
On Sat, Apr 22, 2006 at 03:49:09PM +0200, Longina Przybyszewska wrote:
2006-04-22 15:37:39 Closing connection 2006-04-22 15:37:47 hobbitd_client: Got message 165 @@client#165|1145713067.445669|130.225.128.9|bach.imada.sdu.dk|sunos 2006-04-22 15:37:47 2006-04-22 15:37:47 Worker process died with exit code 134, terminating Sometimes I can see that data come through, but sometimes I can see "worker process died".
Ouch - the hobbitd_client task crashed. There ought to be a "core" file in the ~hobbit/server/tmp/ directory; if you could run that through gdb as described in http://www.hswn.dk/hobbit/help/known-issues.html#bugreport it would help a lot. You can also send me a copy of the ~hobbit/client/tmp/msg.txt file from the client. Henrik