Xymon Mailing List Archive search

Yet another flap

3 messages in this thread

list Ron Cohen · Thu, 25 Apr 2019 15:27:57 +0100 · 
Hi
have one server which keeps flapping on msgs. Spent hours to look for the
obvious culprit (another feeder) until it dawn on me that it must be the
logfetch, since there are no issues with other columns.
And indeed, after running logfetch with --debug=stderr, I discovered that
it is not always running - I mean not triggered at all - which triggered
the flapping status.
I dug here and there, to no avail. Any bright ideas are welcome..
Cheers
Ron
list Japheth Cleaver · Thu, 25 Apr 2019 12:21:03 -0700 ·
▸ quoted from Ron Cohen
On 4/25/2019 7:27 AM, Ron Cohen wrote:
Hi
have one server which keeps flapping on msgs. Spent hours to look for the obvious culprit (another feeder) until it dawn on me that it must be the logfetch, since there are no issues with other columns.
And indeed, after running logfetch with --debug=stderr, I discovered that it is not always running - I mean not triggered at all - which triggered the flapping status.
I dug here and there, to no avail. Any bright ideas are welcome..
After the initial two runs, it should run and generate with each xymonclient-linux.sh execution. Make sure you're running as the appropriate user and with write privileges into the logfetch.status file when executing it by hand.

For debugging, you can wrap the call in the shell code with whichever output you'd like and it will return in the temporary .msg file as it's constructed. If it's executing, but not doing anything, you could prepend an strace to see if it's short exiting for some reason.

For what should be a last result, you could try clearing that file out to force it to rescan all log files from their start position.

HTH,

-jc
list Ron Cohen · Fri, 26 Apr 2019 11:14:30 +0100 ·

Nailed it. set -xv in the xymonclient.sh and noticed that every other run it can't find the logfetch.$MACHINEDOTS.status, so don't not run logfetch.

Apparently there is another test (silly me!) which collects data from other hosts into the tmp directory, and on cleanup deletes  the *hosts* files, among them the status file.

cheers

Ron

▸ quoted from Japheth Cleaver


On 25/04/2019 20:21, Japheth Cleaver wrote:
On 4/25/2019 7:27 AM, Ron Cohen wrote:
Hi
have one server which keeps flapping on msgs. Spent hours to look for the obvious culprit (another feeder) until it dawn on me that it must be the logfetch, since there are no issues with other columns.
And indeed, after running logfetch with --debug=stderr, I discovered that it is not always running - I mean not triggered at all - which triggered the flapping status.
I dug here and there, to no avail. Any bright ideas are welcome..

After the initial two runs, it should run and generate with each xymonclient-linux.sh execution. Make sure you're running as the appropriate user and with write privileges into the logfetch.status file when executing it by hand.

For debugging, you can wrap the call in the shell code with whichever output you'd like and it will return in the temporary .msg file as it's constructed. If it's executing, but not doing anything, you could prepend an strace to see if it's short exiting for some reason.

For what should be a last result, you could try clearing that file out to force it to rescan all log files from their start position.

HTH,

-jc