Monitor for client TSM-Backup

7 messages in this thread

list Martin Flemming · Fri, 4 Jan 2008 15:13:30 +0100 (CET) ·

Hi !

I want to monitor the tsm-backup on my clients ...

... and i myself be not the Scripting-master :-(

Any help or scripts are much appreciated

cheers,

       Martin 


Martin Flemming
DESY / IT          office : Building 2b / 008a
Notkestr. 85       phone  : XXX - XXXX - XXXX
22603 Hamburg      mail   : user-f286aaa49a76@xymon.invalid

list Thomas Kern · Fri, 4 Jan 2008 10:24:46 -0500 ·

This is what I use for backing up a server with TSM and reporting the
outcome to an email list and to hobbit.
I am not very experienced with linux and shell scripting (being an
old-time mainframer), so the hobbit reporting script was stolen from a
bigbrother example at deadcat and mercilessly mangled into this.

linux:~ # cat /etc/cron.d/tsm

SHELL=/bin/sh
PATH=/usr/bin:/usr/sbin:/sbin:/bin:/usr/lib/news/bin
MAILTO=user-f1ebafb19faf@xymon.invalid
#
# Run the TSM backup client and the Hobbit script for reporting
# every morning except Sunday when the TSM server is down.

00 3  * * 1-6   root  /root/bin/run_dsmc
00 6  * * 1-6   root  /usr/lib/hobbit/client/bin/hobbit-dsmc.sh

linux:~ # cat /root/bin/run_dsmc

#! /bin/sh

# /etc/init.d/oracle10 stop             # stop whatever needs to be
stopped

/opt/tivoli/tsm/client/ba/bin/dsmc incremental / 2>&1 \
    | tee -a /var/log/tsm/run_dsm.log  \
    | tee -a /var/log/tsm/run_dsm.`date '+%Y%m%d'`  \
    | grep -f /etc/tsm/tsmrpt.pattern \
    | mail -s "TSM Backup Statistics" user-a0ecc2496b16@xymon.invalid

# /etc/init.d/oracle10 start            # restart whatever

linux:~ # cat /etc/tsm/tsmrpt.pattern

Node Name:
Session established with server ADSM:
  Server Version
  Server date/time:
Successful incremental backup of
Total number of
Data transfer time:
Network data transfer rate:
Aggregate data transfer rate:
Objects compressed by:
Elapsed processing time:
ANS4037E
ANS1228E
ANS1802E

linux:~ # cat /usr/lib/hobbit/client/bin/hobbit-dsmc.sh

#!/bin/sh
#
# Leif Gustafsson
# Giga Consulting AB
# 2002-07-04 Initial
# 2002-07-05 Status Started for backup is NOT a fault
#            and some nicer code. I will do some more
#            during the vacation
# 2007-01-24 T.L.Kern - began modification for Hobbit client
#

HOME=/usr/lib/hobbit/client
BBCMD="/usr/lib/hobbit/client/bin/bbcmd -ENV=/etc/default/hobbit"
BB=$HOME/bin/bb
BBDISP="10.23.15.29"
# . $BBHOME/etc/bbdef.sh

TODAY=$1

if [ x${1}x == xx ];
then
    TODAY=`date '+%Y%m%d'`
fi

LOG_HOME="/var/log/hobbit"
HOSTNAME=`/bin/hostname`
TSMLOG="/var/log/tsm/run_dsm.$TODAY"
LOGFILE="$LOG_HOME/tsmevent"
PATTERNFILE="/etc/tsm/tsmrpt.pattern"

grep -f $PATTERNFILE $TSMLOG > $LOGFILE
if [ $? = 0 ];
then
        MSG="status+1440 $HOSTNAME.backup green `date` - TSM Backup was
successful `cat $LOGFILE`"
        $BB $BBDISP "$MSG"
        exit 0
fi

grep -f $PATTERNFILE $TSMLOG > $LOGFILE
if [ $? = 1 ]
then
        MSG="status+1440 $HOSTNAME.backup yellow `date` - Problems with
TSM Backup.<br>Please check /var/log/tsm/run_dsmc.$TODAY `cat $LOGFILE`"
        $BB $BBDISP "$MSG"
        exit 0
fi

grep -f $PATTERNFILE $TSMLOG > $LOGFILE
if [ $? = 2 ]
then
        MSG="status+1440 $HOSTNAME.backup red `date` - Problems with TSM
Backup.<br>File /var/log/tsm/run_dsmc.$TODAY does not exist."
        $BB $BBDISP "$MSG"
        exit 0
fi


/Thomas Kern
/XXX-XXX-XXXX (O)
/XXX-XXX-XXXX (M)

▸ quoted from Martin Flemming

-----Original Message-----
From: Martin Flemming [mailto:user-f286aaa49a76@xymon.invalid] 
Sent: Friday, January 04, 2008 09:14
To: user-ae9b8668bcde@xymon.invalid
Subject: [hobbit] Monitor for client TSM-Backup

I want to monitor the tsm-backup on my clients ...

... and i myself be not the Scripting-master :-(

Any help or scripts are much appreciated

cheers,

       Martin

list Holly Lund · Fri, 4 Jan 2008 11:49:46 -0500 ·

 
Martin

this is what I use for my monitoring

cron ran script on same day backup is completed 

#/bin/sh

#  Get data from tivoli dsmsched.log for use by Hobbit client for backup
mon
itoring

grep  `date '+%m/%d/%y'` /opt/tivoli/tsm/client/ba/bin/dsmsched.log  |
grep -v S
ent|grep -v Changed > /opt/hobbit/client/tmp/tsmoutput

chown hobbit:nobody /opt/hobbit/client/tmp/tsmoutput


script ran by hobbit client

▸ quoted from Thomas Kern


#/bin/sh
#
# Leif Gustafsson
# Giga Consulting AB
# 2002-07-04 Initial
# 2002-07-05 Status Started for backup is NOT a fault
#            and some nicer code. I will do some more
#            during the vacation
#


COLUMN=backup   # Name of the column
COLOR=green          # By default, everything is OK
MSG="TSM backups complete"

BBHOME=/opt/hobbit/client


#. $BBHOME/etc/bbdef.sh

LOG_HOME=/opt/hobbit/client/tmp
HOSTNAME=`/bin/hostname`

LOGFILE="$LOG_HOME/tsmevent_$HOSTNAME"
OUTFILE="$LOG_HOME/tsmoutput"
TMPFILE="$LOG_HOME/tsmtmp_$HOSTNAME"
NBFILE="$LOG_HOME/tsmnb_$HOSTNAME"
TFFILE="$LOG_HOME/tsmtransfer_$HOSTNAME"
PROCFILE="$LOG_HOME/tsmproctime_$HOSTNAME"

cp /dev/null $NBFILE
cp /dev/null $TFFILE
cp /dev/null $PROCFILE

echo "\n" > $NBFILE
grep  -i "Network data transfer rate" $OUTFILE >> $NBFILE

echo "\n" > $TFFILE
grep -i "Data transfer time" $OUTFILE >> $TFFILE

echo "\n" > $PROCTIMEFILE
grep -i "Elapsed Processing Time"  $OUTFILE >> $PROCFILE

echo "\n" > $LOGFILE
grep -i "Successful incremental backup" $OUTFILE >> $LOGFILE
grep -i fail $OUTFILE >> $LOGFILE
grep -i "Completed successfully" $OUTFILE >> $LOGFILE
if [ $? = 1 ]
then
        COLOR=yellow
        $BB $BBDISP "status $MACHINE.$COLUMN $COLOR `date`
        MSG=status $MACHINE.backup yellow Problems with backup. Please
check /op
t/tivoli/tsm/client/ba/bin/dsmsched.log` cat $LOGFILE` `cat $NBFILE`
`cat $TFFIL
E` `cat $PROCFILE`"
else
       $BB $BBDISP "status $MACHINE.$COLUMN $COLOR `date`
        MSG=status $MACHINE.backup green Backup was successful `cat
$LOGFILE` `c
at $NBFILE` `cat $TFFILE` `cat $PROCFILE`"
fi
echo "message is" $MSG
exit 0


give this ouptut

 Backup was successful

01/04/08   02:13:38 Successful incremental backup of '/boot'
01/04/08   02:14:48 Successful incremental backup of '/home'
01/04/08   02:16:14 Successful incremental backup of '/var'
01/04/08   02:17:02 Successful incremental backup of '/opt'
01/04/08   02:24:07 Successful incremental backup of '/usr'
01/04/08   02:24:33 Successful incremental backup of '/'
01/04/08   02:24:34 Total number of objects failed:           0
01/04/08   02:24:34 Scheduled event 'EARLYAM-WD' completed successfully.

01/04/08   02:24:34 Network data transfer rate:        120,398.69 KB/sec

01/04/08   02:24:34 Data transfer time:                    0.07 sec 
01/04/08   02:24:34 Elapsed processing time:           00:11:11


Holly Lund
XXX-XXX-XXXX
XXX-XXX-XXXX

▸ quoted from Martin Flemming


-----Original Message-----
From: Martin Flemming [mailto:user-f286aaa49a76@xymon.invalid] 
Sent: Friday, January 04, 2008 9:14 AM
To: user-ae9b8668bcde@xymon.invalid
Subject: [hobbit] Monitor for client TSM-Backup


Hi !

I want to monitor the tsm-backup on my clients ...

... and i myself be not the Scripting-master :-(

Any help or scripts are much appreciated

cheers,

       Martin 


Martin Flemming
DESY / IT          office : Building 2b / 008a
Notkestr. 85       phone  : XXX - XXXX - XXXX
22603 Hamburg      mail   : user-f286aaa49a76@xymon.invalid

list Martin Flemming · Fri, 4 Jan 2008 17:59:20 +0100 (CET) ·

Great, Thanks !!

i will check it ... the first thing what i
saw, which could be problematical for me is,
that my tsm-log-file is not proberly sorted with date
but only in one file .. but i figure it out ..

many thanks
 		
	martin

▸ quoted from Thomas Kern



On Fri, 4 Jan 2008, Kern, Thomas wrote:

This is what I use for backing up a server with TSM and reporting the
outcome to an email list and to hobbit.
I am not very experienced with linux and shell scripting (being an
old-time mainframer), so the hobbit reporting script was stolen from a
bigbrother example at deadcat and mercilessly mangled into this.

linux:~ # cat /etc/cron.d/tsm

SHELL=/bin/sh
PATH=/usr/bin:/usr/sbin:/sbin:/bin:/usr/lib/news/bin
MAILTO=user-f1ebafb19faf@xymon.invalid
#
# Run the TSM backup client and the Hobbit script for reporting
# every morning except Sunday when the TSM server is down.

00 3  * * 1-6   root  /root/bin/run_dsmc
00 6  * * 1-6   root  /usr/lib/hobbit/client/bin/hobbit-dsmc.sh

linux:~ # cat /root/bin/run_dsmc

#! /bin/sh

# /etc/init.d/oracle10 stop             # stop whatever needs to be
stopped

/opt/tivoli/tsm/client/ba/bin/dsmc incremental / 2>&1 \
    | tee -a /var/log/tsm/run_dsm.log  \
    | tee -a /var/log/tsm/run_dsm.`date '+%Y%m%d'`  \
    | grep -f /etc/tsm/tsmrpt.pattern \
    | mail -s "TSM Backup Statistics" user-a0ecc2496b16@xymon.invalid

# /etc/init.d/oracle10 start            # restart whatever

linux:~ # cat /etc/tsm/tsmrpt.pattern

Node Name:
Session established with server ADSM:
  Server Version
  Server date/time:
Successful incremental backup of
Total number of
Data transfer time:
Network data transfer rate:
Aggregate data transfer rate:
Objects compressed by:
Elapsed processing time:
ANS4037E
ANS1228E
ANS1802E

linux:~ # cat /usr/lib/hobbit/client/bin/hobbit-dsmc.sh

#!/bin/sh
#
# Leif Gustafsson
# Giga Consulting AB
# 2002-07-04 Initial
# 2002-07-05 Status Started for backup is NOT a fault
#            and some nicer code. I will do some more
#            during the vacation
# 2007-01-24 T.L.Kern - began modification for Hobbit client
#

HOME=/usr/lib/hobbit/client
BBCMD="/usr/lib/hobbit/client/bin/bbcmd -ENV=/etc/default/hobbit"
BB=$HOME/bin/bb
BBDISP="10.23.15.29"
# . $BBHOME/etc/bbdef.sh

TODAY=$1

if [ x${1}x == xx ];
then
    TODAY=`date '+%Y%m%d'`
fi

LOG_HOME="/var/log/hobbit"
HOSTNAME=`/bin/hostname`
TSMLOG="/var/log/tsm/run_dsm.$TODAY"
LOGFILE="$LOG_HOME/tsmevent"
PATTERNFILE="/etc/tsm/tsmrpt.pattern"

grep -f $PATTERNFILE $TSMLOG > $LOGFILE
if [ $? = 0 ];
then
        MSG="status+1440 $HOSTNAME.backup green `date` - TSM Backup was
successful `cat $LOGFILE`"
        $BB $BBDISP "$MSG"
        exit 0
fi

grep -f $PATTERNFILE $TSMLOG > $LOGFILE
if [ $? = 1 ]
then
        MSG="status+1440 $HOSTNAME.backup yellow `date` - Problems with
TSM Backup.<br>Please check /var/log/tsm/run_dsmc.$TODAY `cat $LOGFILE`"
        $BB $BBDISP "$MSG"
        exit 0
fi

grep -f $PATTERNFILE $TSMLOG > $LOGFILE
if [ $? = 2 ]
then
        MSG="status+1440 $HOSTNAME.backup red `date` - Problems with TSM
Backup.<br>File /var/log/tsm/run_dsmc.$TODAY does not exist."
        $BB $BBDISP "$MSG"
        exit 0
fi


/Thomas Kern
/XXX-XXX-XXXX (O)
/XXX-XXX-XXXX (M)

-----Original Message-----
From: Martin Flemming [mailto:user-f286aaa49a76@xymon.invalid] 
Sent: Friday, January 04, 2008 09:14
To: user-ae9b8668bcde@xymon.invalid
Subject: [hobbit] Monitor for client TSM-Backup

I want to monitor the tsm-backup on my clients ...

... and i myself be not the Scripting-master :-(

Any help or scripts are much appreciated

cheers,

       Martin

Gruss

       Martin Flemming

▸ quoted from Holly Lund



Martin Flemming
DESY / IT          office : Building 2b / 008a
Notkestr. 85       phone  : XXX - XXXX - XXXX
22603 Hamburg      mail   : user-f286aaa49a76@xymon.invalid

list Martin Flemming · Fri, 4 Jan 2008 19:10:36 +0100 (CET) ·

Thanks, Holly !

That's what i'm looking for ..

.. i will test it !

thanks & cheers

▸ quoted from Holly Lund

	
	martin

On Fri, 4 Jan 2008, Lund, Holly wrote:

 Martin

this is what I use for my monitoring

cron ran script on same day backup is completed 
#/bin/sh

#  Get data from tivoli dsmsched.log for use by Hobbit client for backup
mon
itoring

grep  `date '+%m/%d/%y'` /opt/tivoli/tsm/client/ba/bin/dsmsched.log  |
grep -v S
ent|grep -v Changed > /opt/hobbit/client/tmp/tsmoutput

chown hobbit:nobody /opt/hobbit/client/tmp/tsmoutput


script ran by hobbit client

#/bin/sh
#
# Leif Gustafsson
# Giga Consulting AB
# 2002-07-04 Initial
# 2002-07-05 Status Started for backup is NOT a fault
#            and some nicer code. I will do some more
#            during the vacation
#
COLUMN=backup   # Name of the column
COLOR=green          # By default, everything is OK
MSG="TSM backups complete"

BBHOME=/opt/hobbit/client


#. $BBHOME/etc/bbdef.sh

LOG_HOME=/opt/hobbit/client/tmp
HOSTNAME=`/bin/hostname`

LOGFILE="$LOG_HOME/tsmevent_$HOSTNAME"
OUTFILE="$LOG_HOME/tsmoutput"
TMPFILE="$LOG_HOME/tsmtmp_$HOSTNAME"
NBFILE="$LOG_HOME/tsmnb_$HOSTNAME"
TFFILE="$LOG_HOME/tsmtransfer_$HOSTNAME"
PROCFILE="$LOG_HOME/tsmproctime_$HOSTNAME"

cp /dev/null $NBFILE
cp /dev/null $TFFILE
cp /dev/null $PROCFILE

echo "\n" > $NBFILE
grep  -i "Network data transfer rate" $OUTFILE >> $NBFILE

echo "\n" > $TFFILE
grep -i "Data transfer time" $OUTFILE >> $TFFILE

echo "\n" > $PROCTIMEFILE
grep -i "Elapsed Processing Time"  $OUTFILE >> $PROCFILE

echo "\n" > $LOGFILE
grep -i "Successful incremental backup" $OUTFILE >> $LOGFILE
grep -i fail $OUTFILE >> $LOGFILE
grep -i "Completed successfully" $OUTFILE >> $LOGFILE
if [ $? = 1 ]
then
        COLOR=yellow
        $BB $BBDISP "status $MACHINE.$COLUMN $COLOR `date`
        MSG=status $MACHINE.backup yellow Problems with backup. Please
check /op
t/tivoli/tsm/client/ba/bin/dsmsched.log` cat $LOGFILE` `cat $NBFILE`
`cat $TFFIL
E` `cat $PROCFILE`"
else
       $BB $BBDISP "status $MACHINE.$COLUMN $COLOR `date`
        MSG=status $MACHINE.backup green Backup was successful `cat
$LOGFILE` `c
at $NBFILE` `cat $TFFILE` `cat $PROCFILE`"
fi
echo "message is" $MSG
exit 0


give this ouptut

 Backup was successful

01/04/08   02:13:38 Successful incremental backup of '/boot'
01/04/08   02:14:48 Successful incremental backup of '/home'
01/04/08   02:16:14 Successful incremental backup of '/var'
01/04/08   02:17:02 Successful incremental backup of '/opt'
01/04/08   02:24:07 Successful incremental backup of '/usr'
01/04/08   02:24:33 Successful incremental backup of '/'
01/04/08   02:24:34 Total number of objects failed:           0
01/04/08   02:24:34 Scheduled event 'EARLYAM-WD' completed successfully.

01/04/08   02:24:34 Network data transfer rate:        120,398.69 KB/sec

01/04/08   02:24:34 Data transfer time:                    0.07 sec 01/04/08   02:24:34 Elapsed processing time:           00:11:11


Holly Lund
XXX-XXX-XXXX
XXX-XXX-XXXX

-----Original Message-----
From: Martin Flemming [mailto:user-f286aaa49a76@xymon.invalid] Sent: Friday, January 04, 2008 9:14 AM
To: user-ae9b8668bcde@xymon.invalid
Subject: [hobbit] Monitor for client TSM-Backup


Hi !

I want to monitor the tsm-backup on my clients ...

... and i myself be not the Scripting-master :-(

Any help or scripts are much appreciated

cheers,

       Martin 

Martin Flemming
DESY / IT          office : Building 2b / 008a
Notkestr. 85       phone  : XXX - XXXX - XXXX
22603 Hamburg      mail   : user-f286aaa49a76@xymon.invalid

Gruss

       Martin Flemming


Martin Flemming
DESY / IT          office : Building 2b / 008a
Notkestr. 85       phone  : XXX - XXXX - XXXX
22603 Hamburg      mail   : user-f286aaa49a76@xymon.invalid

list Thomas Kern · Fri, 4 Jan 2008 13:23:34 -0500 ·

When using the DSMCAD for automatic launching, the logfile is monolithic
and the availability of a timestamp is (I think) a function of which
TSM/client version/release you are using. My client is at the Version 5,
Release 4, Level 1.0 level. Because of previous problems with DSMCAD and
memory leaks (and a DBA who likes backups to run at set times), I have
switched to using CRON to initiate the backups and then I can run each
day's output into its own file as well as a monolithic log for our TSM
admin. 

When I was using the DSMCAD scheduler and had timestamps, I used this
kind of a subsetting to get a block of status messages for the email
list and separately for reporting to BigBrother/Hobbit. But it you don't
have date/time stamps on each record, you might have to grab the last
nnnn records after making sure that the dsmsched.log was indeed updated
on the day you expect it to be updated on (today/yesterday, etc).

linux:~/bin # cat tsmrpt
#! /bin/sh

grep  `date '+%m/%d/%y' --date='yesterday'` /var/log/tsm/dsmsched.log \
  | grep --before-context=30 'SCHEDULEREC STATUS END'                 \ 
  | grep --after-context=30 'SCHEDULEREC STATUS BEGIN'                \

▸ signature

  | mail -s "TSM Backup Statistics" user-a0ecc2496b16@xymon.invalid


/Thomas Kern
/XXX-XXX-XXXX (O)
/XXX-XXX-XXXX (M)

-----Original Message-----

▸ quoted from Martin Flemming

From: Martin Flemming [mailto:user-f286aaa49a76@xymon.invalid] 
Sent: Friday, January 04, 2008 11:59
To: user-ae9b8668bcde@xymon.invalid
Subject: RE: [hobbit] Monitor for client TSM-Backup

Great, Thanks !!

i will check it ... the first thing what i
saw, which could be problematical for me is,
that my tsm-log-file is not proberly sorted with date
but only in one file .. but i figure it out ..

many thanks

	martin

list Martin Flemming · Fri, 4 Jan 2008 20:52:26 +0100 (CET) ·


Great !

Thanks Thomas & Holly ..

Now i've got two working solutions .. ;-)
.. great hobbit-community ...

▸ quoted from Thomas Kern


thanks & cheers

		martin

On Fri, 4 Jan 2008, Kern, Thomas wrote:

When using the DSMCAD for automatic launching, the logfile is monolithic
and the availability of a timestamp is (I think) a function of which
TSM/client version/release you are using. My client is at the Version 5,
Release 4, Level 1.0 level. Because of previous problems with DSMCAD and
memory leaks (and a DBA who likes backups to run at set times), I have
switched to using CRON to initiate the backups and then I can run each
day's output into its own file as well as a monolithic log for our TSM
admin. 

When I was using the DSMCAD scheduler and had timestamps, I used this
kind of a subsetting to get a block of status messages for the email
list and separately for reporting to BigBrother/Hobbit. But it you don't
have date/time stamps on each record, you might have to grab the last
nnnn records after making sure that the dsmsched.log was indeed updated
on the day you expect it to be updated on (today/yesterday, etc).

linux:~/bin # cat tsmrpt
#! /bin/sh

grep  `date '+%m/%d/%y' --date='yesterday'` /var/log/tsm/dsmsched.log \
  | grep --before-context=30 'SCHEDULEREC STATUS END'                 \ 
  | grep --after-context=30 'SCHEDULEREC STATUS BEGIN'                \
  | mail -s "TSM Backup Statistics" user-a0ecc2496b16@xymon.invalid


/Thomas Kern
/XXX-XXX-XXXX (O)
/XXX-XXX-XXXX (M)

-----Original Message-----
From: Martin Flemming [mailto:user-f286aaa49a76@xymon.invalid] 
Sent: Friday, January 04, 2008 11:59
To: user-ae9b8668bcde@xymon.invalid
Subject: RE: [hobbit] Monitor for client TSM-Backup

Great, Thanks !!

i will check it ... the first thing what i
saw, which could be problematical for me is,
that my tsm-log-file is not proberly sorted with date
but only in one file .. but i figure it out ..

many thanks

	martin

Gruss

       Martin Flemming


Martin Flemming
DESY / IT          office : Building 2b / 008a
Notkestr. 85       phone  : XXX - XXXX - XXXX
22603 Hamburg      mail   : user-f286aaa49a76@xymon.invalid

Monitor for client TSM-Backup 🔗 link

Monitor for client TSM-Backup