Alternate log monitor?

Larry Bonham wrote:

-----Original Message-----
From: Andy Smith [mailto:user-982f5f6d4d28@xymon.invalid]
Sent: Wednesday, June 03, 2015 1:48 PM
To: Larry Bonham
Cc: xymon at xymon.com
Subject: Re: [Xymon] Alternate log monitor?

Larry Bonham wrote:

Has anyone successfully integrated a different log monitor into xymon?
The internal msgs monitor works fine for /var/log/messages or any other
system log where a similar notification is used for each system (in our
case the systems group).


But I have a need to monitor multiple Apache logs on various systems
where a different user group would be notified if any yellow or red
alerts were created.  I don't see a way to do that and have
/var/log/messages alerts going somewhere else for the same system.


Running 4.3.18 on RHEL 6.6


swatch maybe?  Everything I'm finding is either too simplistic or way
overkill for what I need.  The lighter the footprint the better.


Thanks.

=========================================================

Larry D. Bonham

Hi, like you, we found the internal msgs tool is perfectly adequate in
terms of functionality, but does not lend itself well to notifying
different groups unless you are relying on email notification, in which
case you can define a GROUP rule in alert.cfg/analysis.cfg.  In our
place, we do not use email, instead, we have a 24x7 operations team
viewing the critical page, managing the incidents in real time, so we
needed different column names to be able to identify the requirements in
critical.cfg.  I wrote this simple script to direct a given log analysis
to a given column name, it may work for you :-

https://wiki.xymonton.org/doku.php/monitors:msgs

When we have need for more sophisticated log monitoring and we use SEC
(http://simple-evcorr.sourceforge.net/) which is extremely powerful, we
employ just a fraction of its potential.  We have a perl module that
goes with SEC so that SEC delivers status messages to Xymon in exactly
the same way as the internal msgs tool.
--
Andy


Andy,

I appreciate the response.  I should have mentioned I already tried your msgs.sh add on.  Only problem is the original output stays in MSGS as well as getting copied to the alternate test name.  We don't want the normal alerts going out to MSGS owners.  Just to the owners of the alternate test name.

Unless I missed something in setting it up, I wasn't able to do that.  I could do special notification on the alternate test but could not differentiate those in the normal test.

Please let me know if I'm misunderstanding something there (e.g. possible GROUP usage).  Wouldn't be the first time.

I have used SEC in the past with snmp trap translation.  That is one of those overkill solutions I mentioned.  I'm sure it would work but I wanted to keep the solution as simple as possible for support purposes down the road.  I was really hoping to accomplish this all within xymon itself if possible.

Thanks for your suggestions.

Larry

For instance, on our apache web servers we suppress the msgs column 
completely (group-except msgs blah blah blah in hosts.cfg) and direct 
messages-ng to a new column called syslog which ultimately gets the the 
UNIX SA called out and the httpd error_log and access_log get directed 
to a column called weblog which gets our middleware team called out.

I would disagree that SEC is overkill.  Some of our apache access logs 
are over 4GB per day, the best way we found to watch traffic in these 
volatile logs is with SEC.  Admittedly, it is nothing more sophisticated 
than regex patterns we look for but there is no way we can afford to 
ship this amount of data to the central server for analysis.  Yes, we 
generate some impact at the client side by analyzing locally instead of 
centrally, but extend this to tomcat logs and weblogic logs and get the 
power of rules like singlewithsuppress and contexts and we can easily 
find the needle in the haystack for our developers.
-- 
Andy