Log/file monitoring based on occurrence?
list Mike Burger
I've got a situation at work where we're being tasked with monitoring failed logins. We've got logging and audit file creation in place...what I'm looking for is whether or not there's a way to have Xymon monitor those logs/files to look for X entries/occurrences within Y amount of time. The requirement is to try to do as much as this programatically as possible without having to resort to scripting or other measures that may be easily countered by a system admin. Thanks. -- Mike Burger http://www.bubbanfriends.org "It's always suicide-mission this, save-the-planet that. No one ever just stops by to say 'hi' anymore." --Colonel Jack O'Neill, SG1
list Galen Johnson
yes...you should be able to do this natively. Lok at the analysis.cfg man page for xymon. =G=
▸
From: xymon-bounces at xymon.com [xymon-bounces at xymon.com] on behalf of Mike Burger [user-cc5c6e80f4c5@xymon.invalid] Sent: Wednesday, January 09, 2013 3:50 PM To: xymon at xymon.com Subject: [Xymon] Log/file monitoring based on occurrence? I've got a situation at work where we're being tasked with monitoring failed logins. We've got logging and audit file creation in place...what I'm looking for is whether or not there's a way to have Xymon monitor those logs/files to look for X entries/occurrences within Y amount of time. The requirement is to try to do as much as this programatically as possible without having to resort to scripting or other measures that may be easily countered by a system admin. Thanks. -- Mike Burger http://www.bubbanfriends.org "It's always suicide-mission this, save-the-planet that. No one ever just stops by to say 'hi' anymore." --Colonel Jack O'Neill, SG1
list Michael Beatty
You will need tell client-local.cfg which log to gather data from. While you can't specify in the last Y amount of time, you can specify Y amount of data. So if you know your log file writes about 1000 bytes per hour... and you want to look back 4 hours, you would specify in client-local.cfg "log:/var/log/mylogfile:4000" This will return the most recent 4000 bytes worth of data to Xymon. Your next step is to configure analysis.cfg to tell xymon how to report on that data. If you want xymon to alert if it sees the word "ERROR", your analysis.cfg would have a line in it "LOG /var/log/mylogfile %ERROR" Look at the analysis.cfg and client-local.cfg man pages for information on the LOG check. Michael Beatty Sherwin-Williams IT Analyst/Developer user-4aea7c115850@xymon.invalid XXX-XXX-XXXX
▸
On 01/09/2013 03:50 PM, Mike Burger wrote:I've got a situation at work where we're being tasked with monitoring failed logins. We've got logging and audit file creation in place...what I'm looking for is whether or not there's a way to have Xymon monitor those logs/files to look for X entries/occurrences within Y amount of time. The requirement is to try to do as much as this programatically as possible without having to resort to scripting or other measures that may be easily countered by a system admin. Thanks.
list Mike Burger
That's what I figured, after having looked at the analysis.cfg man page multiple times. If I want to do this, then, I'm going to have to script something to analyze X amount of time and do something if it sees occurrences>=Y and then feed that to Xymon somehow.
▸
Thanks. -- Mike Burger http://www.bubbanfriends.org "It's always suicide-mission this, save-the-planet that. No one ever just stops by to say 'hi' anymore." --Colonel Jack O'Neill, SG1
You will need tell client-local.cfg which log to gather data from. While you can't specify in the last Y amount of time, you can specify Y amount of data. So if you know your log file writes about 1000 bytes per hour... and you want to look back 4 hours, you would specify in client-local.cfg "log:/var/log/mylogfile:4000" This will return the most recent 4000 bytes worth of data to Xymon. Your next step is to configure analysis.cfg to tell xymon how to report on that data. If you want xymon to alert if it sees the word "ERROR", your analysis.cfg would have a line in it "LOG /var/log/mylogfile %ERROR" Look at the analysis.cfg and client-local.cfg man pages for information on the LOG check. Michael Beatty Sherwin-Williams IT Analyst/Developer user-4aea7c115850@xymon.invalid XXX-XXX-XXXX On 01/09/2013 03:50 PM, Mike Burger wrote:I've got a situation at work where we're being tasked with monitoring failed logins. We've got logging and audit file creation in place...what I'm looking for is whether or not there's a way to have Xymon monitor those logs/files to look for X entries/occurrences within Y amount of time. The requirement is to try to do as much as this programatically as possible without having to resort to scripting or other measures that may be easily countered by a system admin. Thanks.
list Adam Goryachev
▸
On 11/01/13 00:19, Mike Burger wrote:
That's what I figured, after having looked at the analysis.cfg man page multiple times. If I want to do this, then, I'm going to have to script something to analyze X amount of time and do something if it sees occurrences>=Y and then feed that to Xymon somehow. Thanks.
You might be able to use something like fail2ban, and configure it to simply add some text to a logfile instead of adding a iptables entry.... Then let xymon monitor this fail2ban logfile.... Possibly overkill, but just thought I'd mention it... better to re-use something that already exists... Regards, Adam -- Adam Goryachev Website Managers www.websitemanagers.com.au
list Mike Burger
▸
On 11/01/13 00:19, Mike Burger wrote:That's what I figured, after having looked at the analysis.cfg man page multiple times. If I want to do this, then, I'm going to have to script something to analyze X amount of time and do something if it sees occurrences>=Y and then feed that to Xymon somehow. Thanks.You might be able to use something like fail2ban, and configure it to simply add some text to a logfile instead of adding a iptables entry.... Then let xymon monitor this fail2ban logfile.... Possibly overkill, but just thought I'd mention it... better to re-use something that already exists...
At home, I use DenyHosts to do something similar on my publicly connected systems. At work, I've got two issues preventing this: A) No iptables in use on the internally networked Linux systems. B) The system where I'm looking to implement this approach is an AIX system, so there's no iptables or any other onboard firewall. The real reason we're looking at this, at all, is for security auditing purposes. We can't keep an active eye on failed logins, all day, so we're looking for something that can be used to alert us if an arbitrary number of failed logins occurs within an arbitrary amount of time, based on the audit logger's stream.
▸
-- Mike Burger http://www.bubbanfriends.org "It's always suicide-mission this, save-the-planet that. No one ever just stops by to say 'hi' anymore." --Colonel Jack O'Neill, SG1
list Adam Goryachev
▸
On 11/01/13 01:25, Mike Burger wrote:
On 11/01/13 00:19, Mike Burger wrote:That's what I figured, after having looked at the analysis.cfg man page multiple times. If I want to do this, then, I'm going to have to script something to analyze X amount of time and do something if it sees occurrences>=Y and then feed that to Xymon somehow. Thanks.You might be able to use something like fail2ban, and configure it to simply add some text to a logfile instead of adding a iptables entry.... Then let xymon monitor this fail2ban logfile.... Possibly overkill, but just thought I'd mention it... better to re-use something that already exists...At home, I use DenyHosts to do something similar on my publicly connected systems. At work, I've got two issues preventing this: A) No iptables in use on the internally networked Linux systems. B) The system where I'm looking to implement this approach is an AIX system, so there's no iptables or any other onboard firewall. The real reason we're looking at this, at all, is for security auditing purposes. We can't keep an active eye on failed logins, all day, so we're looking for something that can be used to alert us if an arbitrary number of failed logins occurs within an arbitrary amount of time, based on the audit logger's stream.
Right, and fail2ban (which uses python and I'm assuming is portable to AIX) can be configured to do anything you ask it, by default, it adds a firewall rule to iptables. There is nothing stopping you from disabling the iptables calls, and simply using the fail2ban log itself, or changing the iptables command to instead add some log entry somewhere which is then fed into xymon. Regards, Adam -- Adam Goryachev Website Managers www.websitemanagers.com.au
list Jeremy Laidman
You can do this with a "log" configuration entry (in client-local.cfg)
using backticks that counts the lines, calculates a delta, and returns a
suitable message. Something like this:
[name-of-server]
log:`exec 2>/dev/null; Z=$(printf "\x3a"); F=/var/log/messages; Y=1; R=6;
T=/tmp/badlogins; C=$(egrep -c "PAM${Z} (Authentication failure|User not
known)" $F); L=$(cat $T.last); D=$(expr $C - $L); ( [ "$D" = "" ] || { [ $D
-gt $R ] && echo "red${Z} $D login failures exceeded red threshold of $R"
|| { [ $D -gt $Y ] && echo "yellow${Z} $D login failures exceeded yellow
threshold of $Y" || echo "green${Z} $D login failures is OK"; }; } ) >
$T.status; echo "$C" > $T.last; echo $T.status`:1024
Adjust filename and thresholds ($F, $Y and $R) to your liking. The sample
window size is whatever frequency Xymon client is run, typically 5 minutes,
so Y and R thresholds would be a count per 5 minutes. Note that colons and
backticks are not permitted in the command string, so I'm fudging them in
other ways.
Then alert on the message with a couple of LOG entries in analysis.cfg like
so:
HOST=name-of-server
LOG /tmp/badlogins.status %red COLOR=red
LOG /tmp/badlogins.status %yellow COLOR=yellow
Cheers
Jeremy
▸
On 11 January 2013 00:19, Mike Burger <user-cc5c6e80f4c5@xymon.invalid> wrote:
That's what I figured, after having looked at the analysis.cfg man page multiple times. If I want to do this, then, I'm going to have to script something to analyze X amount of time and do something if it sees occurrences>=Y and then feed that to Xymon somehow. Thanks. -- Mike Burger http://www.bubbanfriends.org "It's always suicide-mission this, save-the-planet that. No one ever just stops by to say 'hi' anymore." --Colonel Jack O'Neill, SG1You will need tell client-local.cfg which log to gather data from. While you can't specify in the last Y amount of time, you can specify Y amount of data. So if you know your log file writes about 1000 bytes per hour... and you want to look back 4 hours, you would specify in client-local.cfg "log:/var/log/mylogfile:4000" This will return the most recent 4000 bytes worth of data to Xymon. Your next step is to configure analysis.cfg to tell xymon how to report on that data. If you want xymon to alert if it sees the word "ERROR", your analysis.cfg would have a line in it "LOG /var/log/mylogfile %ERROR" Look at the analysis.cfg and client-local.cfg man pages for information on the LOG check. Michael Beatty Sherwin-Williams IT Analyst/Developer user-4aea7c115850@xymon.invalid XXX-XXX-XXXX On 01/09/2013 03:50 PM, Mike Burger wrote:I've got a situation at work where we're being tasked with monitoring failed logins. We've got logging and audit file creation in place...what I'm looking for is whether or not there's a way to have Xymon monitor those logs/files to look for X entries/occurrences within Y amount of time. The requirement is to try to do as much as this programatically as possible without having to resort to scripting or other measures that may be easily countered by a system admin. Thanks.
list Betsy Schwartz
what I'm looking for is whether or not there's a way to have Xymon monitor those logs/files to look for X entries/occurrences within Y amount of time.
I have not played around with it much - been on my list for a while - but this is a task made for "sec" (Simple Event Correlator) http://simple-evcorr.sourceforge.net/
list Jeremy Laidman
▸
On 21 January 2013 05:44, Betsy Schwartz <user-c61747246f66@xymon.invalid> wrote:
I have not played around with it much - been on my list for a while - but this is a task made for "sec" (Simple Event Correlator)
OMD, why haven't I seen this before! It's been around for a decade and this is the first I've heard of it??? SEC is gold!! Thanks Betsy. J