On Sat, Dec 17, 2005 at 08:43:31AM, Henrik Stoerner wrote:
On Fri, Dec 16, 2005 at 10:11:18PM -0500, Asif Iqbal wrote:
Hi All
I am running solaris 9. My hobbit is running off of /export/home/hobbit
dir. The `df -i` shows
/dev/md/dsk/d30 1005696 1005005 691 100% /export/home
[snip]
Either I think of using these resources, like may be creating some RAM
disks (Henrik had a tip on that, cant find it) or get rid of `hist'
folder which has about 4000 files in it. I really like to keep the
`hist' files for report and going back to check for old events.
Keep the "hist" folder - there's only one file per status you have in
Hobbit, and there won't be more.
What you should have a look at is the "histlogs" folder. This holds a
copy of each status log whenever a status change occurred - and it gets
*really* large over time. These historical status logs are only used
when you look at the history of a status, and click on the red button
off the "Jun 16 2003 14:21:58" event to see the detailed status log.
What I do on my production systems is to archive the history logs once
a month, and then deleting those history logs that are older than 3
months. So you can get the full, detailed status of any event that
happened within the past 3 months; if you need to go further back than
that, it is still visible in the history view, but you'll need to
extract the detailed status log from the tar file to see all of the
details.
Feel free to use this snippet from my monthly cron job - it requires
the GNU date utility. Right now, you will of course need to clean up
your histlogs directory by hand, the snippet below only processes
the histlog-files one month at a time.
cd /var/lib/hobbit/data
# Archive histlogs generated in the past month
FNPATTERN=`date --date="today -1 month" +"???_%b_*_%Y"`
ARCFN=`date --date="today -1 month" +"histlogs_%Y_%m.cpio.gz"`
find histlogs -name "$FNPATTERN" | cpio -o --format=newc | gzip >archive/$ARCFN
# Delete histlogs from 4 months ago (so we always have the last 3 months online)
FNPATTERN=`date --date="today -4 months" +"???_%b_*_%Y"`
find histlogs -name "$FNPATTERN" | xargs rm
Excellent. I am slowly moving old logs. I wonder if that will skew my
monthly and yearly reports.
snippet of the yearly report that gets called by bbcmd.
YEAR=$1
DATE=/usr/local/bin/date
MKDIR=/usr/local/bin/mkdir
YEAR=${YEAR:=`$DATE +%Y`}
START=`$DATE +%s --date="01 Jan $YEAR 00:00:00"`
END=`$DATE +%s --date="31 Dec $YEAR 23:59:59"`
$MKDIR -p /usr/local/apache/htdocs/reports/yearly/$YEAR
BBWEB=/reports/yearly/$YEAR \
$BBHOME/bin/bbgen --reportopts=$START:$END:1:crit \
--ignorecolumns=ssh,bbgen,bbtest,hobbitd,info,trends,msgs,scanmsg,http
--subpagecolumns=2 \
/usr/local/apache/htdocs/reports/yearly/$YEAR
I noticed I have about 70000 files under histlogs. Wow!!. I guess would
be nice to use database instead to store histlogs and make report still
work. I know there are suggestion out there regarding mysql use to store
logs.
(hobbit)@bigbrother:~/data$ find histlogs -name "???_Nov_*_2005" | wc -l
67912
Do you have any plan to have a DB plugin for that in the future? Or may
be a suggestion, on storing log in database, that you think best work
with hobbit?
Thanks
Regards,
Henrik
--
Asif Iqbal
PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu
"..there are two kinds of people: those who work and those who take the credit...try
to be in the first group;...less competition there." - Indira Gandhi