multiple hosts 🔗 link

Rolf Schrittenlocher wrote:

Hi,

as there was no answer to my first mail, I try again with other words:

Our machines may have one or more aliases (virtual hosts) which all have DNS-entries. Normally we refer to the machines not using the hostname but by the alias, each of them combined to a bundle of functions and tasks. So a machine "frodo.xxx.de" might host "webserver.xxx.de", "dnsserver.xxx.de" and "tolkserver.xxx.de".  These aliases and of course the tasks related to them are transferred to another machine in case of hardware failure, maintenance, etc.
In bb-hosts we have an entry for each machine (to get data from this machine in case there is no alias set on it) and of course for all relevant aliases. What we need now is that the general client data (cpu, disk, ssh, ...) for each machine appears both with the hostname and all aliases set on it.
I added a script to runclient.sh which creates ~/client/etc/clientlaunch.cfg dynamically according to the aliases set on the machine before starting the client. This assures that the correct data is polled on the client. I may modify $MACHINE or Í may use the CLIENT:hostname tag in bb-hosts to make the client send messages with a different name. But that would require several instances of the client running, somethin I dislike. I hope there is a way to send the client data once and to make the server look at some configuration file to see where he should  present these data. Maybe that possibility exists but I haven't found it yet. So I hope that someone could indicate me where to search for the places which I have to "hack" hoping that it is possible to do that with shell or perl as my C-knowledge isn't so good. But even if C is necessary: Any hints are appreciated.

Thanks Rolf

-----Original Message-----
From: David Gore [mailto:user-0ef10ea25696@xymon.invalid] Sent: Wednesday, January 11, 2006 9:52 AM
To: user-ae9b8668bcde@xymon.invalid
Subject: Re: [hobbit] multiple hosts

Rolf,

I asked a similar question many months ago for our Digital UNIX (4.x) boxes running in multiple clusters.  I did not get any answer, so I had to create a hack that does run multiple clients on each host.  One client is for the physical host and does, cpu, disk, etc, and one client for the logical/virtual host that does the tasks specific to the logical/virtual host.  The runclient.sh script runs every ten minutes via crontab, to make sure both clients are running and starts either client if they are not.  I also have custom versions of some of the default Hobbit client scripts to support multiple clients on a single host.  So in short as far as I know, you are going to have to hack/script your own solution.  I don't mind sending you what my hobbit scripts have turned into, but it may not be what you are looking for.

David Gore 
*e-mail via SUSE Linux 9.3 and other open source tools. 


Rolf Schrittenlocher wrote:

Hi,

as there was no answer to my first mail, I try again with other words:

Our machines may have one or more aliases (virtual hosts) which all have DNS-entries. Normally we refer to the machines not using the hostname but by the alias, each of them combined to a bundle of functions and tasks. So a machine "frodo.xxx.de" might host "webserver.xxx.de", "dnsserver.xxx.de" and "tolkserver.xxx.de".  These aliases and of course the tasks related to them are transferred to another machine in case of hardware failure, maintenance, etc.
In bb-hosts we have an entry for each machine (to get data from this machine in case there is no alias set on it) and of course for all relevant aliases. What we need now is that the general client data (cpu, disk, ssh, ...) for each machine appears both with the hostname and all aliases set on it.
I added a script to runclient.sh which creates ~/client/etc/clientlaunch.cfg dynamically according to the aliases set on the machine before starting the client. This assures that the correct data is polled on the client. I may modify $MACHINE or Í may use the CLIENT:hostname tag in bb-hosts to make the client send messages with a different name. But that would require several instances of the client running, somethin I dislike. I hope there is a way to send the client data once and to make the server look at some configuration file to see where he should  present these data. Maybe that possibility exists but I haven't found it yet. So I hope that someone could indicate me where to search for the places which I have to "hack" hoping that it is possible to do that with shell or perl as my C-knowledge isn't so good. But even if C is necessary: Any hints are appreciated.

Thanks Rolf

On 1/11/06, Rob Munsch <user-f39e4aae1456@xymon.invalid> wrote:

So I'm staring at

- Program crashed
Fatal signal caught!

from the purple hobbitd_client status for the server, which is very odd,
considering the server (box itself, not just Hobbit binary...) has been
completely restarted about an hour ago.

There were some horrible, horrible disk problems (many backups and a
fsck.reiserfs later... don't ask) due to someone screwing building power
whilst i was away and zapping this box with something like 5 restarts in
half an hour.  ugh.

However, having fixed everything else, and up and running from a clean
boot, and ONLY running hobbit.sh from the /hobbit/server dir...  i'm
confused.  Hobbitd col shows green.  There's only one hobbitd_client
running on the box, and it's the one that appeared when the server
itself was started.

Why does this happen, and how can i stop it?

thanks...

Larry Barber wrote:

The hobbitd_client only issues a status message when it dies. After a restart it will go purple in a half-hour. You just need to do a "drop" of the hobbitd service.

Thanks,
Larry Barber

On 1/11/06, *Rob Munsch* <user-f39e4aae1456@xymon.invalid <mailto:user-f39e4aae1456@xymon.invalid>> wrote:

    So I'm staring at

    - Program crashed
    Fatal signal caught!

    from the purple hobbitd_client status for the server, which is
    very odd,
    considering the server (box itself, not just Hobbit binary...) has
    been
    completely restarted about an hour ago.

    There were some horrible, horrible disk problems (many backups and a
    fsck.reiserfs later... don't ask) due to someone screwing building
    power
    whilst i was away and zapping this box with something like 5
    restarts in
    half an hour.  ugh.

    However, having fixed everything else, and up and running from a clean
    boot, and ONLY running hobbit.sh from the /hobbit/server dir...  i'm
    confused.  Hobbitd col shows green.  There's only one hobbitd_client
    running on the box, and it's the one that appeared when the server
    itself was started.

    Why does this happen, and how can i stop it?

    thanks...

On Wed, Jan 11, 2006 at 02:52:58PM -0500, Rob Munsch wrote:

ooookay (and i assume you mean the hobbitd_client service), that does indeed make it Go Away - but, if it ever crashes again for any reason, it will reappear red then go perma-purple again, requiring another manual Drop, yes?

Surely there's a better way to handle this..?

Henrik Stoerner wrote:

On Wed, Jan 11, 2006 at 02:52:58PM -0500, Rob Munsch wrote:
 

ooookay (and i assume you mean the hobbitd_client service), that does indeed make it Go Away - but, if it ever crashes again for any reason, it will reappear red then go perma-purple again, requiring another manual Drop, yes?

Surely there's a better way to handle this..?
   

Yes there is - fix the bug that makes hobbitd_client crash, and you
won't see any hobbitd_client statuses. Ever.

For starters, check that you are running the latest version (4.1.2p1 -
the older versions do have bugs in the hobbitd_client code). And check the "Reporting Bugs" part of
http://www.hswn.dk/hobbit/help/known-issues.html


Regards,
Henrik

Rob Munsch wrote:

gdb> bt
#0  0xb7e5b941 in kill () from /lib/libc.so.6
#1  0xb7e5b6e5 in raise () from /lib/libc.so.6
#2  0xb7e5ca66 in abort () from /lib/libc.so.6
#3  0x08056651 in sigsegv_handler (signum=0x0) at sig.c:57
#4  <signal handler called>
#5  0xb7e9930c in mempcpy () from /lib/libc.so.6
#6  0xb7e905a1 in _IO_default_xsputn () from /lib/libc.so.6
#7  0xb7e6ef60 in vfprintf () from /lib/libc.so.6
#8  0xb7e876ec in vsprintf () from /lib/libc.so.6
#9  0xb7e7561d in sprintf () from /lib/libc.so.6
#10 0x08049ed0 in unix_cpu_report (hostname=0x20202020 <Address 0x20202020 out of bounds>, hinfo=0x20c7, fromline=0x303a3030 <Address 0x303a3030 out of bounds>, timestr=0x34313a31 <Address 0x34313a31 out of bounds>, uptimestr=0x736f7020 <Address 0x736f7020 out of bounds>, whostr=0x65726774 <Address 0x65726774 out of bounds>, psstr=0x73203a73 <Address 0x73203a73 out of bounds>, topstr=0x6e616863 <Address 0x6e616863 out of bounds>) at hobbitd_client.c:242
#11 0x3f203430 in ?? ()
#12 0x20202020 in ?? ()
#13 0x20202020 in ?? ()

... at this point, please not that lines #14 - #2387 (!!) all look the same as the previous three - the address varies, but it's 2300+ lines of "<address> in ??."  If that whole list is somehow useful, let me know.. otherwise i'll just cut to the end:

#2384 0x20203937 in ?? ()
#2385 0x614a2030 in ?? ()
#2386 0x2034306e in ?? ()
#2387 0x2020203f in ?? ()
Cannot access memory at address 0xbfea0000
gdb>


Henrik Stoerner wrote:

On Wed, Jan 11, 2006 at 02:52:58PM -0500, Rob Munsch wrote:
 

ooookay (and i assume you mean the hobbitd_client service), that does indeed make it Go Away - but, if it ever crashes again for any reason, it will reappear red then go perma-purple again, requiring another manual Drop, yes?

Surely there's a better way to handle this..?
  

Yes there is - fix the bug that makes hobbitd_client crash, and you
won't see any hobbitd_client statuses. Ever.

For starters, check that you are running the latest version (4.1.2p1 -
the older versions do have bugs in the hobbitd_client code). And check the "Reporting Bugs" part of
http://www.hswn.dk/hobbit/help/known-issues.html


Regards,
Henrik

Rob Munsch wrote:

I've been away for a week and no longer sure if this was updated. grr.  i'll have to check if this is -.2 or -.2p1 now.  sigh...

...

For starters, check that you are running the latest version (4.1.2p1 -

On Wed, Jan 11, 2006 at 04:54:33PM -0500, Rob Munsch wrote:

What is the sound of one forehead slapping?

So yeah.  Apparently this is an install of .2 that was done shortly before .2p1 was released, and i missed noticing the transition.  So since my mind has apparently turned to mush in a mere week of offtime, before i b0rk this up further, are there any caveats or tricks to upgrading here, or do i just need to replace wholesale all the various components entirely..?

Henrik Stoerner wrote:

On Wed, Jan 11, 2006 at 04:54:33PM -0500, Rob Munsch wrote:
 

What is the sound of one forehead slapping?

So yeah.  Apparently this is an install of .2 that was done shortly before .2p1 was released, and i missed noticing the transition.  So since my mind has apparently turned to mush in a mere week of offtime, before i b0rk this up further, are there any caveats or tricks to upgrading here, or do i just need to replace wholesale all the various components entirely..?
   

Doing a "make; make install" should be safe.


Regards,
Henrik

Scheblein, Adam wrote:

I know for us here (running trucluster 5) we ran into the same problem, here is my modified runclient.sh script (the changes are basically I added the `hostname` part to the lock files:

#!/bin/sh
#----------------------------------------------------------------------------#
# Hobbit client bootup script.                                               #
#                                                                            #
# This invokes hobbitlaunch, which in turn runs the Hobbit client and any    #
# extensions configured.                                                     #
#                                                                            #
# Copyright (C) 2005 Henrik Storner <user-ce4a2c883f75@xymon.invalid>                         #
#                                                                            #
# This program is released under the GNU General Public License (GPL),       #
# version 2. See the file "COPYING" for details.                             #
#                                                                            #
#----------------------------------------------------------------------------#
#
# $Id: runclient.sh,v 1.6 2005/09/28 21:21:17 henrik Exp $

# Default settings for this client
MACHINEDOTS="`uname -n`"                        # This systems hostname
BBOSTYPE="`uname -s | tr '[A-Z]' '[a-z]'`"      # This systems operating system in lowercase
BBOSSCRIPT="hobbitclient-$BBOSTYPE.sh"

# Commandline mods for the defaults
while test "$1" != ""
do
       case "$1" in
         --hostname=*)
               MACHINEDOTS="`echo $1 | sed -e's/--hostname=//'`"
               ;;
         --os=*)
               BBOSTYPE="`echo $1 | sed -e's/--os=//' | tr '[A-Z]' '[a-z]'`"
               ;;
         --help)
               echo "Usage: $0 [--hostname=CLIENTNAME] [--os=rhel3|linux22] start|stop"
               exit 0
               ;;
         start)
               CMD=$1
               ;;
         stop)
               CMD=$1
               ;;
         restart)
               CMD=$1
               ;;
       esac

       shift
done

OLDDIR="`pwd`"
cd "`dirname $0`"
HOBBITCLIENTHOME="`pwd`"
cd "$OLDDIR"

MACHINE="`echo $MACHINEDOTS | sed -e's/\./,/g'`"

export MACHINE MACHINEDOTS BBOSTYPE BBOSSCRIPT HOBBITCLIENTHOME

case "$CMD" in
 "start")
       if test ! -w $HOBBITCLIENTHOME/logs; then
               echo "Cannot write to the $HOBBITCLIENTHOME/logs directory"
               exit 1
       fi
       if test ! -w $HOBBITCLIENTHOME/tmp; then
               echo "Cannot write to the $HOBBITCLIENTHOME/tmp directory"
               exit 1
       fi

       if test -f $HOBBITCLIENTHOME/logs/clientlaunch.`hostname`.pid; then
               echo "Hobbit client already running, re-starting it"
               $0 stop
               rm -f $HOBBITCLIENTHOME/logs/clientlaunch.`hostname`.pid
       fi

       $HOBBITCLIENTHOME/bin/hobbitlaunch --config=$HOBBITCLIENTHOME/etc/clientlaunch.cfg --log=$HOBBITCLIENTHOME/logs/clientlaunch.log --pidfile=$HOBBITCLIENTHOME/logs/clientlaunch.`hostname`.pid
       if test $? -eq 0; then
               echo "Hobbit client for $BBOSTYPE started on $MACHINEDOTS"
       else
               echo "Hobbit client startup failed"
       fi
       ;;

 "stop")
       if test -f $HOBBITCLIENTHOME/logs/clientlaunch.`hostname`.pid; then
               kill `cat $HOBBITCLIENTHOME/logs/clientlaunch.$MACHINEDOTS.pid`
               echo "Hobbit client stopped"
       else
               echo "Hobbit client not running"
       fi
       ;;

 "restart")
       if test -f $HOBBITCLIENTHOME/logs/clientlaunch.`hostname`.pid; then
               $0 stop
       else
               echo "Hobbit client not running, continuing to start it"
       fi

       $0 start
       ;;
esac

exit 0


-----Original Message-----
From: David Gore [mailto:user-0ef10ea25696@xymon.invalid] Sent: Wednesday, January 11, 2006 9:52 AM
To: user-ae9b8668bcde@xymon.invalid
Subject: Re: [hobbit] multiple hosts

Rolf,

I asked a similar question many months ago for our Digital UNIX (4.x) boxes running in multiple clusters.  I did not get any answer, so I had to create a hack that does run multiple clients on each host.  One client is for the physical host and does, cpu, disk, etc, and one client for the logical/virtual host that does the tasks specific to the logical/virtual host.  The runclient.sh script runs every ten minutes via crontab, to make sure both clients are running and starts either client if they are not.  I also have custom versions of some of the default Hobbit client scripts to support multiple clients on a single host.  So in short as far as I know, you are going to have to hack/script your own solution.  I don't mind sending you what my hobbit scripts have turned into, but it may not be what you are looking for.

David Gore 
*e-mail via SUSE Linux 9.3 and other open source tools. 


Rolf Schrittenlocher wrote:
 

Hi,

as there was no answer to my first mail, I try again with other words:

Our machines may have one or more aliases (virtual hosts) which all have DNS-entries. Normally we refer to the machines not using the hostname but by the alias, each of them combined to a bundle of functions and tasks. So a machine "frodo.xxx.de" might host "webserver.xxx.de", "dnsserver.xxx.de" and "tolkserver.xxx.de".  These aliases and of course the tasks related to them are transferred to another machine in case of hardware failure, maintenance, etc.
In bb-hosts we have an entry for each machine (to get data from this machine in case there is no alias set on it) and of course for all relevant aliases. What we need now is that the general client data (cpu, disk, ssh, ...) for each machine appears both with the hostname and all aliases set on it.
I added a script to runclient.sh which creates ~/client/etc/clientlaunch.cfg dynamically according to the aliases set on the machine before starting the client. This assures that the correct data is polled on the client. I may modify $MACHINE or Í may use the CLIENT:hostname tag in bb-hosts to make the client send messages with a different name. But that would require several instances of the client running, somethin I dislike. I hope there is a way to send the client data once and to make the server look at some configuration file to see where he should  present these data. Maybe that possibility exists but I haven't found it yet. So I hope that someone could indicate me where to search for the places which I have to "hack" hoping that it is possible to do that with shell or perl as my C-knowledge isn't so good. But even if C is necessary: Any hints are appreciated.

Thanks Rolf

   

Hi,

as there was no answer to my first mail, I try again with other words:

Our machines may have one or more aliases (virtual hosts) which all have DNS-entries. Normally we refer to the machines not using the hostname but by the alias, each of them combined to a bundle of functions and tasks. So a machine "frodo.xxx.de" might host "webserver.xxx.de", "dnsserver.xxx.de" and "tolkserver.xxx.de".  These aliases and of course the tasks related to them are transferred to another machine in case of hardware failure, maintenance, etc.
In bb-hosts we have an entry for each machine (to get data from this machine in case there is no alias set on it) and of course for all relevant aliases. What we need now is that the general client data (cpu, disk, ssh, ...) for each machine appears both with the hostname and all aliases set on it.
I added a script to runclient.sh which creates ~/client/etc/clientlaunch.cfg dynamically according to the aliases set on the machine before starting the client. This assures that the correct data is polled on the client. I may modify $MACHINE or Í may use the CLIENT:hostname tag in bb-hosts to make the client send messages with a different name. But that would require several instances of the client running, somethin I dislike. I hope there is a way to send the client data once and to make the server look at some configuration file to see where he should  present these data. Maybe that possibility exists but I haven't found it yet. So I hope that someone could indicate me where to search for the places which I have to "hack" hoping that it is possible to do that with shell or perl as my C-knowledge isn't so good. But even if C is necessary: Any hints are appreciated.

Thanks Rolf

-- 
Mit freundlichen Gruessen
Rolf Schrittenlocher

HRZ/BDV, Senckenberganlage 31, 60054 Frankfurt Tel: (XX) XX - XXX XXXXX   Fax: (XX) XX XXX XXXX
LBS: user-1e39a1813094@xymon.invalid
Persoenlich: user-6ea8e907e200@xymon.invalid

-----Original Message-----
From: Henrik Stoerner [mailto:user-ce4a2c883f75@xymon.invalid]
Sent: 11 January 2006 20:47
To: user-ae9b8668bcde@xymon.invalid
Subject: Re: [hobbit] eternally purple hobbitd_client on the server box


On Wed, Jan 11, 2006 at 02:52:58PM -0500, Rob Munsch wrote:

ooookay (and i assume you mean the hobbitd_client service), that does
indeed make it Go Away - but, if it ever crashes again for any reason,
it will reappear red then go perma-purple again, requiring another
manual Drop, yes?

Surely there's a better way to handle this..?

Yes there is - fix the bug that makes hobbitd_client crash, and you
won't see any hobbitd_client statuses. Ever.

For starters, check that you are running the latest version (4.1.2p1 -
the older versions do have bugs in the hobbitd_client code). And
check the "Reporting Bugs" part of
http://www.hswn.dk/hobbit/help/known-issues.html


Regards,
Henrik