On Wed, Nov 23, 2011 at 10:06 AM, Elizabeth Schwartz
<user-c61747246f66@xymon.invalid> wrote:
I've got to monitor some very large log files. They're up to a couple
You can use the GNU tail "--retry" and "--follow=name" options to tail
a file even through log rotation. Then pipe it into grep (set
bufferring to per-line if more than one grep is used) and send the
output via "logger" to syslog. If you simply redirect to another file
rather than logger/syslog, you have problems when you want to
zero/rotate the output file, so using logger/syslog is easier that
way.
For example, add this into /etc/inittab:
tail --follow=MMRequest.log | grep "ERROR servicename LotsOfText" |
grep --line-buffered -v "LOG NNNNN servicename LotsOfHTML" | logger -p
user2.info -t logwatch
Note that some versions of init can't handle inittab lines that are
too long, so you'd put it into a script.
but I'm just not getting the ERROR lines in the log. Is this file just
too large and too full of HTML to parse? Any suggestions?
Yes, could be. I've had problems using the "log:" stuff with
fast-moving logfiles.
Also, I've found bugs in the "logfetch" program in some cases of
"trigger" and "ignore".
Cheers
Jeremy