Hobbit migration to a new box 🔗 link

Pardon me if this question has been asked before.

 
I am installing a new server since the previous server we had has been
crashing do to hardware issues.  Is there a best practices for migrating the
Hobbit data from the old box to the new?

 
Thanks in advance.

 
David Gilmore

Consultant

sc_logo

X Traverse Street

Providence, RI  XXXXX

XXX-XXX-XXXX   Voice

XXX-XXX-XXXX   Fax

 
Cert_Partner_rgb

if the new hardware would take over the IP and name of the crashing one, I
don't think there's much to change. If you do the cut-over quick enough, you
won't have a purple either :)
as for the data, 'rpm -ql hobbit' should give you a full list of files &
diretories to backup & restore, if Linux. what OS you are running on?

On 1/23/07, David Gilmore <user-70507ff7198d@xymon.invalid> wrote:

[image: sc_logo]

X Traverse Street

Providence, RI  XXXXX

XXX-XXX-XXXX   Voice

XXX-XXX-XXXX   Fax


[image: Cert_Partner_rgb]

Besides re-installing the hobbit application itself...

I would think all you need to do is tar up your /home/hobbit directory and move that over...
that will contain all your custom configurations and settings.....

Michael A. Price
Performance Network Engineering
NASA/GSFC Code 440.8/LMB
Greenbelt, Maryland 20770
            Phone:  XXX-XXX-XXXX
            Cell:   XXX-XXX-XXXX
            e-mail: user-2a9e3c790857@xymon.invalid

David Gilmore wrote:

Old box was FC4 with an install from the source code.  New box will be
running FC5 and was planning on installing from RPM to cut down on the
configuration time, but I do like the control provided by installing from
source.

 
The old box is still up, and gets backed up nightly, so I can just copy the
historical data to the new box once the install is done.

 
David

 
From: hobbit-return-11735-david=user-9e293dd11111@xymon.invalid
[mailto:hobbit-return-11735-david=user-9e293dd11111@xymon.invalid] On Behalf
Of Jerry Yu
Sent: Tuesday, January 23, 2007 3:14 PM
To: user-ae9b8668bcde@xymon.invalid
Subject: Re: [hobbit] Hobbit migration to a new box

 
if the new hardware would take over the IP and name of the crashing one, I
don't think there's much to change. If you do the cut-over quick enough, you
won't have a purple either :)
as for the data, 'rpm -ql hobbit' should give you a full list of files &
diretories to backup & restore, if Linux. what OS you are running on? 


On 1/23/07, David Gilmore <user-70507ff7198d@xymon.invalid> wrote:

Pardon me if this question has been asked before.

 
I am installing a new server since the previous server we had has been
crashing do to hardware issues.  Is there a best practices for migrating the
Hobbit data from the old box to the new?

 
Thanks in advance.

 
David Gilmore

Consultant

Error! Filename not specified.

X Traverse Street

Providence, RI  XXXXX

XXX-XXX-XXXX   Voice

XXX-XXX-XXXX   Fax

 
Error! Filename not specified.

On Tue, Jan 23, 2007 at 02:41:33PM -0500, David Gilmore wrote:

Since you installed from source, the simplest way is to just copy
everything across - provided you use the same directory layouts,
hostname, IP-address for the server etc. If not, you'll have to tweak
these settings in hobbitserver.cfg - if changing IP, then also the
client side configurations will need changing (so don't do that).

If there are some incompatible libraries, then re-compiling Hobbit and
running a "make install" on top of the existing (copied over) setup will
take care of that.

Essentially, Hobbit only stores data in the ~hobbit/data/ directory,
and configuration files in ~hobbit/server/etc/ . If you have some custom
extension-scripts in the ~hobbit/server/ext/ directory, then you'll have
to handle them yourself.

The only real problem one may run into with migrating a Hobbit server is
if you are going from one hardware platform to another - that involves
dump'ing all of the RRD files to XML, and the import'ing all of them
from the XML files onto the new server. It's scriptable, but may take
some time depending on the number of files (had to do this with some
25.000 files recently when moving all of our Hobbit RRD's from a Sun
SPARC server to an Intel box, it took 3 hours - because the Sun server
was busy).


Regards,
Henrik

-----Original Message-----
From: Henrik Stoerner [mailto:user-ce4a2c883f75@xymon.invalid]
Sent: Tue 1/23/2007 10:07 PM
To: user-ae9b8668bcde@xymon.invalid
Subject: Re: [hobbit] Hobbit migration to a new box
 

The only real problem one may run into with migrating a Hobbit server is
if you are going from one hardware platform to another - that involves
dump'ing all of the RRD files to XML, and the import'ing all of them
from the XML files onto the new server. It's scriptable, but may take
some time depending on the number of files 

Do you have a handy script for this kind of task and are you willing to share it with the community?

Regards
Johann

On Tue, Jan 23, 2007 at 10:10:58PM +0100, Johann Eggers wrote:

From: Henrik Stoerner [mailto:user-ce4a2c883f75@xymon.invalid]

Nothing fancy, but see below. It runs on the old Hobbit server, and uses
ssh (with ssh-keys to avoid a password prompt for each file) to send the
XML file across to the new server and import it there.

It also has a list of the per-host RRD directories in /tmp/rrddirs.txt
(e.g. created by the commented-out "ls -ld ..." command). The trick was
that I could interrupt the script at any time, and just remove those
lines from the rrddirs.txt file that had already been processed, then
restart the script and it would continue.

The reason for first doing an "rm" of the file on the new server was
that the new server was already being fed updates, so it was creating
and updating RRD files while this script was running. "rrdtool restore"
aborts if the file exists, so I had to delete any existing rrd file on
the new server.


Henrik


#!/bin/sh

MYNEWSERVER=rrdserver.foo.com

cd /var/lib/hobbit/rrd
#ls -1d * >/tmp/rrddirs.txt

cat /tmp/rrddirs.txt | while read H
do
   NUM=`find $H -type f -a -name "*.rrd" -a -mtime -30 | wc -l`
   if [ $NUM -gt 0 ]
   then
      echo "Processing $H ($NUM files)"
      find $H -type f -a -name "*.rrd" -a -mtime -30 | while read f
      do
         FULLFN="/var/lib/hobbit/rrd/$f"
         rrdtool dump "$f" | ssh $MYNEWSERVER "rm \"$FULLFN\"; rrdtool restore - \"$FULLFN\""
      done
   else
      echo "Skipped $H - no up-to-date files"
   fi
done

exit 0

unless u r asking for a wrapper,  the dumping to XML then back is part of
RRDTOOL
rrdtool dump filename.rrd > filename.xml
rrdtool restore filename.xml filename.rrd

On 1/23/07, Johann Eggers <user-769b09132207@xymon.invalid> wrote:

Henrik,

Thank you, as always.  I am staying on the Intel platform, so luckily I will
not face the RRD issue you describe.  But since the boxes are running on the
same subnet right now I had to configure the new box with a different host
name and IP.  The old install is pretty much the default, as I am not that
talented at scripting or working with Linux, but I am learning.

I guess I will recompile from the source.  Obviously I can copy the BBHOSTS
to the new machine.  Can I still copy the data files over and keep the
history or am I going to lose that?

Changing IP will only hurt a little bit, as I only have two servers on the
local subnet to monitor.  The other clients are on other networks and are
passing traffic through our firewall to the hobbit server.

David


-----Original Message-----
From: hobbit-return-11738-david=user-9e293dd11111@xymon.invalid
[mailto:hobbit-return-11738-david=user-9e293dd11111@xymon.invalid] On Behalf
Of Henrik Stoerner
Sent: Tuesday, January 23, 2007 4:07 PM

To: user-ae9b8668bcde@xymon.invalid
Subject: Re: [hobbit] Hobbit migration to a new box

On Tue, Jan 23, 2007 at 02:41:33PM -0500, David Gilmore wrote:

Since you installed from source, the simplest way is to just copy
everything across - provided you use the same directory layouts,
hostname, IP-address for the server etc. If not, you'll have to tweak
these settings in hobbitserver.cfg - if changing IP, then also the
client side configurations will need changing (so don't do that).

If there are some incompatible libraries, then re-compiling Hobbit and
running a "make install" on top of the existing (copied over) setup will
take care of that.

Essentially, Hobbit only stores data in the ~hobbit/data/ directory,
and configuration files in ~hobbit/server/etc/ . If you have some custom
extension-scripts in the ~hobbit/server/ext/ directory, then you'll have
to handle them yourself.

The only real problem one may run into with migrating a Hobbit server is
if you are going from one hardware platform to another - that involves
dump'ing all of the RRD files to XML, and the import'ing all of them
from the XML files onto the new server. It's scriptable, but may take
some time depending on the number of files (had to do this with some
25.000 files recently when moving all of our Hobbit RRD's from a Sun
SPARC server to an Intel box, it took 3 hours - because the Sun server
was busy).


Regards,
Henrik

-----Original Message-----
From: Henrik Stoerner [mailto:user-ce4a2c883f75@xymon.invalid]
Sent: Tue 1/23/2007 10:26 PM
To: user-ae9b8668bcde@xymon.invalid
Subject: Re: [hobbit] Hobbit migration to a new box
 
On Tue, Jan 23, 2007 at 10:10:58PM +0100, Johann Eggers wrote:

From: Henrik Stoerner [mailto:user-ce4a2c883f75@xymon.invalid]

The only real problem one may run into with migrating a Hobbit server is
if you are going from one hardware platform to another - that involves
dump'ing all of the RRD files to XML, and the import'ing all of them
from the XML files onto the new server. It's scriptable, but may take
some time depending on the number of files 

Do you have a handy script for this kind of task and are you willing to share it with the community?

Nothing fancy, but see below. It runs on the old Hobbit server, and uses
ssh (with ssh-keys to avoid a password prompt for each file) to send the
XML file across to the new server and import it there.

It also has a list of the per-host RRD directories in /tmp/rrddirs.txt
(e.g. created by the commented-out "ls -ld ..." command). The trick was
that I could interrupt the script at any time, and just remove those
lines from the rrddirs.txt file that had already been processed, then
restart the script and it would continue.

The reason for first doing an "rm" of the file on the new server was
that the new server was already being fed updates, so it was creating
and updating RRD files while this script was running. "rrdtool restore"
aborts if the file exists, so I had to delete any existing rrd file on
the new server.


Henrik


#!/bin/sh

MYNEWSERVER=rrdserver.foo.com

cd /var/lib/hobbit/rrd
#ls -1d * >/tmp/rrddirs.txt

cat /tmp/rrddirs.txt | while read H
do
   NUM=`find $H -type f -a -name "*.rrd" -a -mtime -30 | wc -l`
   if [ $NUM -gt 0 ]
   then
      echo "Processing $H ($NUM files)"
      find $H -type f -a -name "*.rrd" -a -mtime -30 | while read f
      do
         FULLFN="/var/lib/hobbit/rrd/$f"
         rrdtool dump "$f" | ssh $MYNEWSERVER "rm \"$FULLFN\"; rrdtool restore - \"$FULLFN\""
      done
   else
      echo "Skipped $H - no up-to-date files"
   fi
done

exit 0

Thx!!

On Tue, Jan 23, 2007 at 04:31:28PM -0500, David Gilmore wrote:

No problem with copying the data/ directory over - that will give you
all of the old history and graphs on the new server.

As far as I recall, changing IP and hostname of the server only requires
changing the hobbitserver.cfg file. If in doubt, run a "grep
OLDSERVERNAME ~hobbit/server/etc/*" and see what shows up.


Regards,
Henrik

I'll try that.  Is that from the terminal on the old box? Did I mention I
know very little about Linux?  :)

David


-----Original Message-----
From: hobbit-return-11744-david=user-9e293dd11111@xymon.invalid
[mailto:hobbit-return-11744-david=user-9e293dd11111@xymon.invalid] On Behalf
Of Henrik Stoerner
Sent: Tuesday, January 23, 2007 4:37 PM

To: user-ae9b8668bcde@xymon.invalid
Subject: Re: [hobbit] Hobbit migration to a new box

On Tue, Jan 23, 2007 at 04:31:28PM -0500, David Gilmore wrote:

BBHOSTS

No problem with copying the data/ directory over - that will give you
all of the old history and graphs on the new server.

As far as I recall, changing IP and hostname of the server only requires
changing the hobbitserver.cfg file. If in doubt, run a "grep
OLDSERVERNAME ~hobbit/server/etc/*" and see what shows up.


Regards,
Henrik