Hobbit 4.0.3 released 🔗 link

Version 4.0.3 of the Hobbit Monitor was uploaded to sourceforge.net
a few minutes ago.

This release adds a new maintenance script, native to Hobbit, so
Hobbit no longer relies on any external tools for the core
functionality.

Beginning with this release, Hobbit can no longer be used as
an add-on to Big Brother. Hobbit retains compatibility with 
BB clients and extension scripts, but you can no longer run
eg. the Hobbit network test tool and have it report to a 
Big Brother server.

All log files have been moved to the Hobbit log-directory,
usually /var/log/hobbit/. If you have scripts that access 
these, you should change them or setup links for the old
locations of the BBVAR/acks/* logfiles.

Commandline options for all of the CGI tools have been 
collected into a "hobbitcgi.cfg" file. If you have customized
your CGI scripts with additional commandline options, you should
put them into this file instead of modifying the CGI wrappers
directly, to ease upgrading.

And of course, a number of bugs have been fixed. I am grateful
to the people on the Hobbit mailing list for their great effort
with reporting bugs, suggesting improvements and testing the
early versions.

The changelog for this release is included below.


Regards,
Henrik


Changes from 4.0.2 -> 4.0.3

Bugfixes (general):
* The bb-datepage and bbmessage CGI tools were reading the 
  POST data in an incorrect way; it worked on most systems, 
  but did not adhere to the CGI specification.

Bugfixes (alerting):
* Acknowledgments were broken in 4.0.2.
* Using PAGE=... in hobbit-alerts.cfg to pick out hosts
  on the front-page was not possible. The front page is
  now recognized with the name "/", so PAGE=/ will find
  them.
* The hobbitd_alert --test option would not pick up the 
  correct page-path settings from the bb-hosts file.
* The BBALPHAMSG text passed to scripts as a default alert
  message now includes the URL link to the statuslog.

Bugfixes (graphs and trends):
* hobbitgraph.cgi could show "@RRDPARAM@" on graphs if
  it was matched by a NULL string (would happen for mailq
  graphs).
* RRD files were not being updated while a status was 
  blue (disabled), even though status messages were received.
  Changed so that blue logs are passed off to the RRD
* The "disk1" graph definition failed to take into account
  that the numbers logged were already in KB of data. So
  the axis-label was wrong.
  parser - we might as well track data when it's there.
* The "mailq" RRD handler now finds the queue-length 
  regardless of whether it is before or after the "requests"
  keyword in the "mailq" or "nmailq" status message.
* If the "apache" data collection was specified for a host,
  but the host had no "http" tests, a bogus http status report
  was being generated.
* Network statistics used a conversion that would overflow
  on 32-bit systems.

Bugfixes (web pages):
* Some header/footer files for the snapshot report
  were missing.
* The "info_header" and "info_footer" files were not being
  used for the "info" column pages.

Bugfixes (installation):
* Fix file-descriptor leak in the "setup-newfiles" tool
  used during installation. This could cause "make install"
  to abort while installing new files, on systems with a 
  low setting for the max. number of simultaneous open files.
* Some systems keep libraries in a /lib64 directory - this was
  not being searched by the configure script.
* If Hobbit was built without SSL support and an SSL network
  test is configured, it will now always fail with a meaningful
  error message.
* If the configure script found SSL- or LDAP-libraries, but
  these could not link, the build would not disable SSL-
  and LDAP-support and therefore it failed. If the libraries
  do not work, disable the support.
* The SSL test would link without required network support
  libraries on some platforms.
* The options for overriding SSL- and LDAP-include files
  did not match the documented name.
* AIX systems built with gcc were missing the OSDEF compile-
  flags.


Improvements:
* All remnants of Big Brother compatibility have been 
  removed. If you want to stick with the old Big Brother
  tool, use bbgen. This allowed for some much needed 
  cleaning up of the bbgen code that loaded the status
  data, especially for the handling of purple status logs.
  Also, the sending of messages has dropped all support 
  for the Big Brother method of sending "page" messages
  when a status-message is sent with an alert-color, so
  the BBPAGE and BBPAGERS settings are no longer used.
* The maint.pl script has been removed. A new tool, 
  hobbit-enadis.cgi, replaces this with a native Hobbit
  tool. If you are upgrading, you should change the
  ~/server/www/menu/menu_items.js file to point link to
  "hobbit-enadis.sh" instead of "maint.pl".
* The "info" column page now includes a form to disable
  and enable tests for a single host. If you prefer not
  to have this on the info page, add the option "--no-disable"
  to the hobbit-cgi/bb-hostsvc.sh wrapper.
* Some logfiles have been moved to the normal log-file
  directory - /var/log/hobbit:
  - nkstatus.log (from ~hobbit/server/)
  - notifications.log (from ~hobbit/data/acks/)
  - acklog (from ~hobbit/data/acks), also renamed to 
    acknowledge.log
* The hobbit-mailack CGI will now find a "delay=DURATION"
  line in the mail message, and use that as the duration of
  the acknowledgment. Reports show that there are some
  types of mail/SMS systems where you cannot modify the 
  message subject.
* Paul D. Backer contributed "favicon" images generated
  from the Hobbit "recent" GIF files. These have been added
  and are now loaded from the Hobbit web/*_header files,
  so that browsers supporting this (Mozilla, Firefox) will
  display a favicon-image in the titlebar or on the
  page-tab holding the Hobbit webpage.
* Two new settings in hobbitserver.cfg can be used to
  restrict which filesystems are being tracked by Hobbit.
  The NORRDDISKS / RRDDISKS settings are regular-expressions
  that the filesystem names are matched against. See the
  hobbitserver.cfg(5) man-page.
* Hobbit now works with RRDtool 1.2.x. Due to a bug in
  the first 1.2.x releases of RRDtool, you must use at least 
  version 1.2.2 with Hobbit.
* A new report-output option lets you save a CSV (comma-
  separated values) file with the availability data for
  each host+test in Hobbit. This allows for further 
  processing of the availability data eg. importing it
  into a spreadsheet. This can be selected from the report
  request webpage.
* It is now possible to define different values for an environment
  setting in hobbitserver.cfg, depending on a new "--area=..."
  command-line option for all tools. See hobbitserver.cfg.
* Apache 1.x performance data are now graphed correctly,
  by mapping the "BusyServers/IdleServers" data into the
  BusyWorkers/IdleWorkers datasets used by Apache 2.0 RRD
  graphs.
* sendmail RRD's for sendmail 8.13+ now tracks the number
  of "quarantined" messages also. You must delete any existing
  sendmail.rrd files for this new data item to be tracked.
* All RRD-updates now use the RRDtool "--template" option to
  map values to RRD datasets. This should keep us from 
  updating datasets with wrong values.
* Graphs can now get their title from a script, instead of
  using a static string in hobbitgraph.cfg. This lets you 
  e.g. pick up the graphtitle from the mrtg.cfg.
* All CGI's now pick up commandline options from a new
  configuration file, hobbitcgi.cfg. This is to ease
  packaging and make sure the CGI's can be updated without
  losing local configurations.
* The debian- and rpm-specific packaging files are now 
  included in the distribution tarfile.
* A historical status-log now includes a "Full History"
  button to go to the history overview.
* "irix" is now recognized as operating system. A preliminary
  vmstat RRD will be generated, but this may change in a
  future version. "netstat" reports are not yet handled. 

-- 
Henrik Storner

I'm upp running 4.0.3

Regards
Lars
Hobbithobbyist

Http://leopg9.no-ip.org

----- Original Message ----- 
From: "Henrik Stoerner" <user-ce4a2c883f75@xymon.invalid>
To: <user-ae9b8668bcde@xymon.invalid>
Sent: Sunday, May 22, 2005 12:47 PM
Subject: [hobbit] Hobbit 4.0.3 released

Henrik,

In an attempt to install this on SLES9, I've run into this during 'make 
install':

BBHOME="/home/hobbit/server" MANROOT="/usr/local/man" ARESVER="1.2.1" 
INSTALLROOT="" INSTALLBINDIR="/home/hobbit/server/bin" 
INSTALLETCDIR="/home/hobbit/server/etc" 
INSTALLEXTDIR="/home/hobbit/server/ext" 
INSTALLTMPDIR="/home/hobbit/server/tmp" 
INSTALLWEBDIR="/home/hobbit/server/web" 
INSTALLWWWDIR="/home/hobbit/server/www" make -C bbnet install
make[1]: Entering directory `/home/hobbit/hobbit-4.0.3/bbnet'
ranlib c-ares/.libs/libcares.a || echo "ranlib failure - ignored"
cp c-ares/.libs/libcares.a .
cc  -I./c-ares -c -o dns.o dns.c
dns.c:24:22: libbbgen.h: No such file or directory
dns.c: In function `add_url_to_dns_queue':
dns.c:163: error: `bburl_t' undeclared (first use in this function)
dns.c:163: error: (Each undeclared identifier is reported only once
dns.c:163: error: for each function it appears in.)
dns.c:163: error: parse error before "bburl"
dns.c:165: error: `bburl' undeclared (first use in this function)
dns.c: In function `dns_test_server':
dns.c:303: warning: assignment makes pointer from integer without a cast
make[1]: *** [dns.o] Error 1
make[1]: Leaving directory `/home/hobbit/hobbit-4.0.3/bbnet'
make: *** [install-bbnet] Error 2

Any thoughts on how to get past this?

Thanks.

Henrik Stoerner wrote:

 ... big snip.

-- 
Rich Smrcina
VM Assist, Inc.
Main: (262)392-2026
Cell: (XXX)XXX-XXXX
Ans Service:  (360)715-2467
user-61add9955ef9@xymon.invalid

Catch the WAVV!  http://www.wavv.org
WAVV 2005 - Colorado Springs - May 20-24, 2005

I am getting some dead.letter issues as it seems hobbit is mis-reading my
alert config... either that or I have laid out my hobbit-alerts.cfg file
badly!

It's still a bit of a shambles from an organisation point of view, with a
lot of repitition rather than macros, so I'll use a small example:


EXTRACT FROM hobbit-alerts.cfg :

  HOST=zd1sv1,zt1sv1,zp1sv1,zp1sv2
        MAIL    andy            REPEAT=60m RECOVERED FORMAT=TEXT
        MAIL    ernstd          REPEAT=60m RECOVERED FORMAT=TEXT
SERVICE=disk COLOR=red DURATION>60m
        SCRIPT  /export/home/hobbit/bin/qpage   support-pg      REPEAT=60m
RECOVERED FORMAT=SMS COLOR=red
        SCRIPT  /export/home/hobbit/bin/etxt    andy-ph         RECOVERED
FORMAT=SMS COLOR=red


RESULTS OF ~/server/bin/bbcmd hobbitd_alert --test zp1sv1 disk :

  00013427 2005-05-23 15:28:04 send_alert zp1sv1:disk state Paging
  00013427 2005-05-23 15:28:04 Matching host:service:page
'zp1sv1:disk:sap/sap-r3' against rule line 109
  00013427 2005-05-23 15:28:04 *** Match with
'HOST=zd1sv1,zt1sv1,zp1sv1,zp1sv2' ***
  00013427 2005-05-23 15:28:04 Matching host:service:page
'zp1sv1:disk:sap/sap-r3' against rule line 110
  00013427 2005-05-23 15:28:04 *** Match with 'MAIL
andy            REPEAT=60m RECOVERED FORMAT=TEXT' ***
  00013427 2005-05-23 15:28:04 Mail alert with command '/opt/csw/bin/nail
-s "Hobbit [12345] zp1sv1:disk CRITICAL (RED)" andy'
  00013427 2005-05-23 15:28:04 Matching host:service:page
'zp1sv1:disk:sap/sap-r3' against rule line 110
  00013427 2005-05-23 15:28:04 *** Match with 'MAIL
andy            REPEAT=60m RECOVERED FORMAT=TEXT' ***
  00013427 2005-05-23 15:28:04 Mail alert with command '/opt/csw/bin/nail
-s "Hobbit [12345] zp1sv1:disk CRITICAL (RED)"  REPEAT=60m'
  00013427 2005-05-23 15:28:04 Matching host:service:page
'zp1sv1:disk:sap/sap-r3' against rule line 111
  00013427 2005-05-23 15:28:04 Failed 'MAIL
ernstd          REPEAT=60m RECOVERED FORMAT=TEXT SERVICE=disk COLOR=red
DURATION>60m' (min. duration 0<3600)
  00013427 2005-05-23 15:28:04 Matching host:service:page
'zp1sv1:disk:sap/sap-r3' against rule line 111
  00013427 2005-05-23 15:28:04 Failed 'MAIL
ernstd          REPEAT=60m RECOVERED FORMAT=TEXT SERVICE=disk COLOR=red
DURATION>60m' (min. duration 0<3600)
  00013427 2005-05-23 15:28:04 Matching host:service:page
'zp1sv1:disk:sap/sap-r3' against rule line 112
  00013427 2005-05-23 15:28:04 *** Match with 'SCRIPT
/export/home/hobbit/bin/qpage   support-pg      REPEAT=60m RECOVERED
FORMAT=SMS COLOR=red' ***
  00013427 2005-05-23 15:28:04 Script alert with command
'/export/home/hobbit/bin/qpage' and recipient support-pg
  00013427 2005-05-23 15:28:04 Matching host:service:page
'zp1sv1:disk:sap/sap-r3' against rule line 113
  00013427 2005-05-23 15:28:04 *** Match with 'SCRIPT
/export/home/hobbit/bin/etxt    andy-ph         RECOVERED FORMAT=SMS
COLOR=red' ***
  00013427 2005-05-23 15:28:04 Script alert with command
'/export/home/hobbit/bin/etxt' and recipient andy-ph
  00013427 2005-05-23 15:28:04 Matching host:service:page
'zp1sv1:disk:sap/sap-r3' against rule line 113
  00013427 2005-05-23 15:28:04 *** Match with 'SCRIPT
/export/home/hobbit/bin/etxt    andy-ph         RECOVERED FORMAT=SMS
COLOR=red' ***
  00013427 2005-05-23 15:28:04 Script alert with command
'/export/home/hobbit/bin/etxt' and recipient     RECOVERED


As you can see, I get some funny duplications and errors:

- Why does the first MAIL line pick up the correct recipient "andy", and
then send to user "REPEAT=60m" as well?
- Why does ernstd not get matched when SERVICE=disk is on his config line?
- Why does the etxt script get called for both "andy-ph" and "RECOVERED"
while the qpage script is OK?
- Although it's not shown here, I am getting repeat emails to "andy" every
30 minutes even though the config says 60.

I'm starting to suspect it is to do with the order of the arguments, or
maybe whitespace issues.  Or, God forbid, oddball control characters which
have somehow ended up in the file.

If anyone can help out in getting this working I would appreciate it.

Currently running Hobbit 4.03. on Solaris 9 x86 (SunOS zpnz-mtm-bb01 5.9
Generic_117172-05 i86pc i386 i86pc).  Short to long mail name handling is
done via entries in /etc/mail/aliases.

Cheers,
Andy.

#####################################################################################

This email is intended for the person to whom it is addressed
only. If you are not the intended recipient, do not read, copy
or use the contents in any way. The opinions expressed may not
necessarily reflect those of ZESPRI Group of Companies ('ZESPRI').

While every effort has been made to verify the information
contained herein, ZESPRI does not make any representations 
as to the accuracy of the information or to the performance
of any data, information or the products mentioned herein.
ZESPRI will not accept liability for any losses, damage or
consequence, however, resulting directly or indirectly from
the use of this e-mail/attachments.
#####################################################################################

On Sun, May 22, 2005 at 07:01:33PM -0500, Rich Smrcina wrote:

It's trying to build some stuff during the "make install" - that should not happen.

Check the clock on this box, thrn try doing "make clean; make; make install".
If that fails, then do it again and log all of the output from make,
then mail me that log.


Regards,
Henrik

On Mon, May 23, 2005 at 03:49:41PM +1200, Andy France wrote:

[snip]

I think this last explanation is one you should check. The piece of your
hobbit-alerts.cfg file that you included in the message had some hex
0xA0 characters in them - I don't know if they were added by your e-mail
client, but if they are in the file I can understand why it would behave
the way it does.

Try running "od -cx hobbit-alerts.cfg" and see if you have some of those 
0xA0 characters in the file.


Regards,
Henrik

On Mon, May 23, 2005 at 03:49:41PM +1200, Andy France wrote:

Try this patch - it will complain loudly in the page.log file if it
finds high-ascii (> 0x7f) characters in the file, and then drop them
from the input.


Henrik


-------------- next part --------------
--- lib/misc.c	2005/05/20 20:54:08	1.32
+++ lib/misc.c	2005/05/23 06:19:20
@@ -209,14 +209,15 @@
 	return result;
 }
 
-void grok_input(char *l)
+void grok_input(unsigned char *l)
 {
 	/*
 	 * This routine sanitizes an input line, stripping off whitespace,
 	 * removing comments and un-escaping \-escapes and quotes.
 	 */
-	char *p, *outp;
+	unsigned char *p, *outp;
 	int inquote, inhyphen;
+	int didwarn = 0;
 
 	p = strchr(l, '\n'); if (p) *p = '\0';
 
@@ -238,6 +239,13 @@
 		else if ((*p == '#') && !inquote && !inhyphen) {
 			*p = '\0';
 		}
+		else if (*p > 0x7f) {
+			if (!didwarn) {
+				errprintf("Hi-ASCII character %02x dropped from input line '%s'\n", (int)*p, l);
+				didwarn = 1;
+			}
+			p++;
+		}
 		else {
 			if (outp != p) *outp = *p;
 			outp++; p++;
--- lib/misc.h	2005/05/20 20:54:08	1.18
+++ lib/misc.h	2005/05/23 06:06:27
@@ -24,7 +24,7 @@
 extern void addtobuffer(char **buf, int *bufsz, char *newtext);
 extern char *msg_data(char *msg);
 extern char *gettok(char *s, char *delims);
-extern void grok_input(char *s);
+extern void grok_input(unsigned char *s);
 extern unsigned int IPtou32(int ip1, int ip2, int ip3, int ip4);
 extern char *u32toIP(unsigned int ip32);
 extern const char *textornull(const char *text);

user-ce4a2c883f75@xymon.invalid (Henrik Stoerner) wrote on 23/05/2005 17:52:51:

God only knows what Lotus Notes gets up to with it's formatting :-)

"od -cx hobbit-alerts.cfg | grep -i 0xA0" doesn't turn up anything.

I ran through the following procedure...

  gmake distclean
  gpatch -p0 < cfg-hiascii.patch
  gmake
  ~/server/hobbit.sh stop
  gmake install
  ~/server/hobbit.sh start

However  "~/server/bin/bbcmd hobbitd_alert --test zp1sv1 disk" still shows
the same erroneous recipients and there are no complaints in page.log.

Regards,
Henrik

Thanks,

Andy.

#####################################################################################

This email is intended for the person to whom it is addressed
only. If you are not the intended recipient, do not read, copy
or use the contents in any way. The opinions expressed may not
necessarily reflect those of ZESPRI Group of Companies ('ZESPRI').

While every effort has been made to verify the information
contained herein, ZESPRI does not make any representations 
as to the accuracy of the information or to the performance
of any data, information or the products mentioned herein.
ZESPRI will not accept liability for any losses, damage or
consequence, however, resulting directly or indirectly from
the use of this e-mail/attachments.
#####################################################################################

Henrik Stoerner schrieb:

Great. It works. And the netstat error is gone.
Thank alot :-)

Dirk

On Mon, May 23, 2005 at 08:46:55PM +1200, Andy France wrote:

Agreed :-)

Leave out the "0x" - 'od' output looks like this:

0000000           H   O   S   T   =   z   d   1   s   v   1   ,   z   t
        20a0 4f48 5453 7a3d 3164 7673 2c31 747a
0000020   1   s   v   1   ,   z   p   1   s   v   1   ,   z   p 1   s
        7331 3176 7a2c 3170 7673 2c31 707a 7331
0000040   v   2  \n                             M   A   I   L
        3276 a00a a020 a020 a020 4d20 4941 204c
0000060               a   n   d   y                          
        20a0 61a0 646e 2079 20a0 20a0 20a0 20a0

Could you pack up the bb-hosts, hobbitserver.cfg and hobbit-alerts.cfg
files in a tar-file, and mail it directly to me ?


Thanks,
Henrik

On Mon, May 23, 2005 at 03:49:41PM +1200, Andy France wrote:

Argh ... so easy to find once you get the right test-case.

The code reading the alert config only handles TAB's at the beginning of
a line, not in the middle of a line.

Patch attached.


Henrik

-------------- next part --------------
--- hobbitd/do_alert.c	2005/05/22 07:31:22	1.65
+++ hobbitd/do_alert.c	2005/05/23 09:59:17
@@ -13,7 +13,7 @@
 /*                                                                            */
 /*----------------------------------------------------------------------------*/
 
-static char rcsid[] = "$Id: do_alert.c,v 1.65 2005/05/22 07:31:22 henrik Exp $";
+static char rcsid[] = "$Id: do_alert.c,v 1.66 2005/05/23 09:59:07 henrik Exp $";
 
 /*
  * The alert API defines three functions that must be implemented:
@@ -679,7 +679,7 @@
 				errprintf("Ignored unknown/unexpected token '%s' at line %d\n", p, cfid);
 			}
 
-			if (p) p = strtok(NULL, " ");
+			if (p) p = strtok(NULL, " \t");
 		}
 
 		if (curlinerecips && currcp && (curlinerecips != currcp)) {

user-ce4a2c883f75@xymon.invalid (Henrik Stoerner) wrote on 23/05/2005 22:00:36:

He shoots he scores!

That did the trick,  You know, I was going to try converting all the tabs
to spaces at one point but I guess I was just too lazy ;-)

Henrik

As always, thank you very much Henrik!

Andy.

#####################################################################################

This email is intended for the person to whom it is addressed
only. If you are not the intended recipient, do not read, copy
or use the contents in any way. The opinions expressed may not
necessarily reflect those of ZESPRI Group of Companies ('ZESPRI').

While every effort has been made to verify the information
contained herein, ZESPRI does not make any representations 
as to the accuracy of the information or to the performance
of any data, information or the products mentioned herein.
ZESPRI will not accept liability for any losses, damage or
consequence, however, resulting directly or indirectly from
the use of this e-mail/attachments.
#####################################################################################

Henrik,

The clock is advancing like it is supposed to.  I re-ran the build process and it failed.  I did make clean and ran make again, redirecting stdout to a file for you.  When I ran make install, it worked!

I'm not sure what to say.  I'll try the build again and see if I can make it fail.

Thank you.

Henrik Stoerner wrote:

-- 
Rich Smrcina
VM Assist, Inc.
Main: (262)392-2026
Cell: (XXX)XXX-XXXX
Ans Service:  (360)715-2467
user-61add9955ef9@xymon.invalid

Catch the WAVV!  http://www.wavv.org
WAVV 2005 - Colorado Springs - May 20-24, 2005