Feature request for log test
list Johan Sjöberg
Hi. We have a syslog server which receives logs from a number of servers and network devices. Currently we use the log test in Xymon to check for errors in these logs, and it works fine. But it is a little blunt since all log test end up in the same msgs test. It could also be a problem if we get an error in one log file, and need to ack or disable the test. In this case we would not get any alert if there were errors in one of the other logs. It would have been nice if you could add a client definition to a logfile test in analysis.conf, and report each log file as the device which is originates from. Or maybe as a separate syslog test to distinguish it from the msgs test. This way we could also set up individual alerts for the different logged devices. I could, of course, write a client- or serverside script for this, but I always find it difficult to do good log monitoring scripts and it would be nice to be able to use the logic already in Xymon What do you think? Would anyone else be interested in this feature? I also have no idea how much work it would be. /Johan
list David Baldwin
Johan,
▸
Hi. We have a syslog server which receives logs from a number of servers and network devices. Currently we use the log test in Xymon to check for errors in these logs, and it works fine. But it is a little blunt since all log test end up in the same msgs test. It could also be a problem if we get an error in one log file, and need to ack or disable the test. In this case we would not get any alert if there were errors in one of the other logs. It would have been nice if you could add a client definition to a logfile test in analysis.conf, and report each log file as the device which is originates from. Or maybe as a separate syslog test to distinguish it from the msgs test. This way we could also set up individual alerts for the different logged devices. I could, of course, write a client- or serverside script for this, but I always find it difficult to do good log monitoring scripts and it would be nice to be able to use the logic already in Xymon What do you think? Would anyone else be interested in this feature? I also have no idea how much work it would be.
I have also been looking at this same problem just recently. My log structure is keyed by IP address with daily files - e.g. /var/log/rsyslog/IP/messages-YYYMMDD - how the files are set up would need to be accommodated. I've thought of 2 approaches: * writing a utility from scratch to examine the log files - however this then requires all the message rules to be reimplemented rather than using analysis.cfg * writing a utility that uses 'logfetch' (xymon client utility) to grab relevant section of logfile and then send a client message (still need to work out what class and other details to include in the header) on behalf of the device which contains [msgs:/var/log/messages] section for the log file. With this approach, if the client has no other client message reporting we're OK, but if not, I'm not sure if it will cause problems also. Mostly I'd be looking at this for switches/firewalls/etc so no direct client report in place. David. -- David Baldwin - IT Unit Australian Sports Commission www.ausport.gov.au Tel 02 62147830 Fax 02 62141830 PO Box 176 Belconnen ACT 2616 user-cbbf693f2c89@xymon.invalid Leverrier Street Bruce ACT 2617 Keep up to date with what's happening in Australian sport visit http://www.ausport.gov.au This message is intended for the addressee named and may contain confidential and privileged information. If you are not the intended recipient please note that any form of distribution, copying or use of this communication or the information in it is strictly prohibited and may be unlawful. If you receive this message in error, please delete it and notify the sender.
list Henrik Størner
▸
On 09-05-2011 11:31, Johan Sjöberg wrote:
It would have been nice if you could add a client definition to a logfile test in analysis.conf, and report each log file as the device which is originates from. Or maybe as a separate syslog test to distinguish it from the msgs test. This way we could also set up individual alerts for the different logged devices.
I agree wholeheartedly. There are a handful of statuses - msgs, procs, svcs - that bundle all kinds of services into one status. I'd like to make this configurable so you can configure what status column the results go into. No code yet, but definitely something that should be done. Regards, Henrik
list Raymond Storer
So, on Windows, for example, are you suggesting some sort of mapping where if the event source = "Service Control Manager" then that is a general service related thing where you'd send a generic services status message to Xymon. However, if the event source = "MySql" then you could send a status message specifically concerning MySql. Is that the gist of this? I haven't seen many programs take advantage of it, but application programmers can also create their own event logs, separate from the standard three in Windows. Ray
▸
-----Original Message-----
From: xymon-bounces at xymon.com [mailto:xymon-bounces at xymon.com] On Behalf Of Henrik Størner
Sent: Tuesday, May 10, 2011 1:22 AM
To: xymon at xymon.com
Subject: Re: [Xymon] Feature request for log test
On 09-05-2011 11:31, Johan Sjöberg wrote:It would have been nice if you could add a client definition to a logfile test in analysis.conf, and report each log file as the device which is originates from. Or maybe as a separate syslog test to distinguish it from the msgs test. This way we could also set up individual alerts for the different logged devices.
I agree wholeheartedly. There are a handful of statuses - msgs, procs, svcs - that bundle all kinds of services into one status. I'd like to make this configurable so you can configure what status column the results go into. No code yet, but definitely something that should be done. Regards, Henrik
CONFIDENTIALITY NOTICE: This email and any attachments are for the
exclusive and confidential use of the intended recipient. If you are not
the intended recipient, please do not read, distribute or take action in
reliance upon this message. If you have received this in error, please
notify us immediately by return email and promptly delete this message
and its attachments from your computer system. We do not waive
attorney-client or work product privilege by the transmission of this
message.
list Johan Sjöberg
Hi David. I thought I would try to take a grip on this now after vacations and stuff. I think your second approach looks like a good way. Have you done any progress regarding this? I thought I should start by figuring out how logfetch works and how to send the messages to the server. I am also primarily looking to use this for switches and other network equipment that do not have any clients of their own. /Johan
▸
From: David Baldwin [mailto:user-cbbf693f2c89@xymon.invalid]
Sent: den 10 maj 2011 01:48
To: Johan Sjöberg
Cc: xymon at xymon.com
Subject: Re: [Xymon] Feature request for log test
Johan,
Hi.
We have a syslog server which receives logs from a number of servers and network devices.
Currently we use the log test in Xymon to check for errors in these logs, and it works fine. But it is a little blunt since all log test end up in the same msgs test. It could also be a problem if we get an error in one log file, and need to ack or disable the test. In this case we would not get any alert if there were errors in one of the other logs.
It would have been nice if you could add a client definition to a logfile test in analysis.conf, and report each log file as the device which is originates from. Or maybe as a separate syslog test to distinguish it from the msgs test.
This way we could also set up individual alerts for the different logged devices.
I could, of course, write a client- or serverside script for this, but I always find it difficult to do good log monitoring scripts and it would be nice to be able to use the logic already in Xymon
What do you think? Would anyone else be interested in this feature? I also have no idea how much work it would be.
I have also been looking at this same problem just recently.
My log structure is keyed by IP address with daily files - e.g. /var/log/rsyslog/IP/messages-YYYMMDD - how the files are set up would need to be accommodated.
I've thought of 2 approaches:
* writing a utility from scratch to examine the log files - however this then requires all the message rules to be reimplemented rather than using analysis.cfg
* writing a utility that uses 'logfetch' (xymon client utility) to grab relevant section of logfile and then send a client message (still need to work out what class and other details to include in the header) on behalf of the device which contains [msgs:/var/log/messages] section for the log file. With this approach, if the client has no other client message reporting we're OK, but if not, I'm not sure if it will cause problems also. Mostly I'd be looking at this for switches/firewalls/etc so no direct client report in place.
David.
--
David Baldwin - IT Unit
Australian Sports Commission www.ausport.gov.au<http://www.ausport.gov.au>; Tel 02 62147830 Fax 02 62141830 PO Box 176 Belconnen ACT 2616 user-cbbf693f2c89@xymon.invalid<mailto:user-cbbf693f2c89@xymon.invalid> Leverrier Street Bruce ACT 2617 Keep up to date with what's happening in Australian sport visit www.ausport.gov.au<http://www.ausport.gov.au>;
▸
This message is intended for the addressee named and may contain confidential and privileged information. If you are not the intended recipient please note that any form of distribution, copying or use of this communication or the information in it is strictly prohibited and may be unlawful. If you receive this message in error, please delete it and notify the sender.
list David Baldwin
Johan,
▸
Hi David. I thought I would try to take a grip on this now after vacations and stuff. I think your second approach looks like a good way. Have you done any progress regarding this? I thought I should start by figuring out how logfetch works and how to send the messages to the server. I am also primarily looking to use this for switches and other network equipment that do not have any clients of their own.
Sorry for no reply - been on a long vacation. I haven't made any more progress on this yet. The client message with just [msgs:*] section could be a problem with viewing the client data at any given time if there is another client message coming in for the host - it would flip-flop between the regular message with all the other sections ([cpu] [disk] etc), and the one with the [msgs:*] section. I guess I could pull the existing client message and inject the additional [msgs:*] sections... Anyhow, not likely to be a high priority to implement, but if that changes I'll let you know. David.
/Johan *From:*David Baldwin [mailto:user-cbbf693f2c89@xymon.invalid]
▸
*Sent:* den 10 maj 2011 01:48
*To:* Johan Sjöberg
*Cc:* xymon at xymon.com
*Subject:* Re: [Xymon] Feature request for log test
Johan,
Hi.
We have a syslog server which receives logs from a number of servers
and network devices.
Currently we use the log test in Xymon to check for errors in these
logs, and it works fine. But it is a little blunt since all log test
end up in the same msgs test. It could also be a problem if we get an
error in one log file, and need to ack or disable the test. In this
case we would not get any alert if there were errors in one of the
other logs.
It would have been nice if you could add a client definition to a
logfile test in analysis.conf, and report each log file as the device
which is originates from. Or maybe as a separate syslog test to
distinguish it from the msgs test.
This way we could also set up individual alerts for the different
logged devices.
I could, of course, write a client- or serverside script for this, but
I always find it difficult to do good log monitoring scripts and it
would be nice to be able to use the logic already in Xymon
What do you think? Would anyone else be interested in this feature? I
also have no idea how much work it would be.
I have also been looking at this same problem just recently.
My log structure is keyed by IP address with daily files - e.g.
/var/log/rsyslog/IP/messages-YYYMMDD - how the files are set up would
need to be accommodated.
I've thought of 2 approaches:
* writing a utility from scratch to examine the log files - however
this then requires all the message rules to be reimplemented rather
than using analysis.cfg
* writing a utility that uses 'logfetch' (xymon client utility) to
grab relevant section of logfile and then send a client message (still
need to work out what class and other details to include in the
header) on behalf of the device which contains
[msgs:/var/log/messages] section for the log file. With this approach,
if the client has no other client message reporting we're OK, but if
not, I'm not sure if it will cause problems also. Mostly I'd be
looking at this for switches/firewalls/etc so no direct client report
in place.
David.
--
David Baldwin - Assistant Director, Infrastructure (acting) Information and Communication Technology Services Australian Sports Commission http://ausport.gov.au
▸
Tel 02 62147830 Fax 02 62141830 PO Box 176 Belconnen ACT 2616 user-cbbf693f2c89@xymon.invalid Leverrier Street Bruce ACT 2617 Keep up to date with what's happening in Australian sport visit http://www.ausport.gov.au This message is intended for the addressee named and may contain confidential and privileged information. If you are not the intended recipient please note that any form of distribution, copying or use of this communication or the information in it is strictly prohibited and may be unlawful. If you receive this message in error, please delete it and notify the sender.