Xymon Mailing List Archive search

Can't match alerts.cfg rules

7 messages in this thread

list Jake · Sat, 05 Nov 2016 16:14:37 -0400 ·
 
I have 4.3.27-1.el7.terabithia running on Centos 7, and can't get any
rules in alerts.cfg.to match and fire an alert. 
I have in hosts.cfg: 
192.168.2.31 phone-sales # conn 
In alerts.cfg: 
 HOST=phone-sales COLOR=red
 MAIL user-81168669c6fa@xymon.invalid
## Wildcard rule
 HOST=%^.*$ COLOR=red
 MAIL user-81168669c6fa@xymon.invalid 
Made the wildcard rule a regexp because "HOST=*" wasn't working, and
added the host-specific rule because no wildcard was working. But if I
trigger a red on host phone-sales, in alert.log I see: 
Checking criteria for host 'phone-sales', which is not yet defined; some
alerts may not immediately fire 
What am I missing? I retired a year and a half ago, and have been out of
practice, so feel free to start with "You dummy...."
list Japheth Cleaver · Sat, 5 Nov 2016 23:09:48 -0700 ·
quoted from Jake

On Sat, November 5, 2016 1:14 pm, Jake wrote:

I have 4.3.27-1.el7.terabithia running on Centos 7, and can't get any
rules in alerts.cfg.to match and fire an alert.

I have in hosts.cfg:

192.168.2.31 phone-sales # conn

In alerts.cfg:

 HOST=phone-sales COLOR=red
 MAIL user-81168669c6fa@xymon.invalid
## Wildcard rule
 HOST=%^.*$ COLOR=red
 MAIL user-81168669c6fa@xymon.invalid

Made the wildcard rule a regexp because "HOST=*" wasn't working, and
added the host-specific rule because no wildcard was working. But if I
trigger a red on host phone-sales, in alert.log I see:

Checking criteria for host 'phone-sales', which is not yet defined; some
alerts may not immediately fire

What am I missing? I retired a year and a half ago, and have been out of
practice, so feel free to start with "You dummy...."

Hi,

Can you show the output of xymoncmd xymond_alert --dump-config ? That
would help us see what xymond_alert itself is interpreting things as.

The "Checking criteria for host..." should only apply when xymond_alert
(via loadhosts) hasn't reloaded the hosts.cfg yet (in this instance). That
entry looks normal, though. Does the error get shown every time
xymond_alert starts up?


Regards,
-jc
list Jake · Sun, 06 Nov 2016 16:55:58 -0500 ·
quoted from Japheth Cleaver
 
 On 2016-11-06 1:09, J.C. Cleaver wrote: 
On Sat, November 5, 2016 1:14 pm, Jake wrote:
I have 4.3.27-1.el7.terabithia running on Centos 7, and can't get any rules in alerts.cfg.to match and fire an alert. I have in hosts.cfg: 192.168.2.31 phone-sales # conn In alerts.cfg: HOST=phone-sales COLOR=red MAIL user-81168669c6fa@xymon.invalid ## Wildcard rule HOST=%^.*$ COLOR=red MAIL user-81168669c6fa@xymon.invalid Made the wildcard rule a regexp because "HOST=*" wasn't working, and added the host-specific rule because no wildcard was working. But if I trigger a red on host phone-sales, in alert.log I see: Checking criteria for host 'phone-sales', which is not yet defined; some alerts may not immediately fire What am I missing? I retired a year and a half ago, and have been out of practice, so feel free to start with "You dummy...."
Hi,

Can you show the output of xymoncmd xymond_alert --dump-config ? That
would help us see what xymond_alert itself is interpreting things as.

The "Checking criteria for host..." should only apply when xymond_alert
(via loadhosts) hasn't reloaded the hosts.cfg yet (in this instance). That
entry looks normal, though. Does the error get shown every time
xymond_alert starts up?

Regards,
-jc

[root at hccmon xymon]# xymoncmd xymond_alert --dump-config
 120 HOST=phone-sales COLOR=red  MAIL user-81168669c6fa@xymon.invalid FORMAT=TEXT REPEAT=30 
 123 HOST=%^.* COLOR=red  MAIL user-81168669c6fa@xymon.invalid FORMAT=TEXT REPEAT=30 
Sorry, I was misinterpreting the "Checking criteria..." message as being
triggered by the red code. Perhaps I'm not waiting long enough for the
alert to fire before deciding it's not working. But the message is not
showing every time I "systemctl restart xymonlaunch.service". Will get
back to this in the morning, EST. 
Thanks, 
Jake
list Jake · Mon, 07 Nov 2016 18:37:43 -0500 ·
quoted from Jake
 
On 2016-11-06 16:55, Jake wrote: 
On 2016-11-06 1:09, J.C. Cleaver wrote: 
On Sat, November 5, 2016 1:14 pm, Jake wrote:
I have 4.3.27-1.el7.terabithia running on Centos 7, and can't get any rules in alerts.cfg.to match and fire an alert. I have in hosts.cfg: 192.168.2.31 phone-sales # conn In alerts.cfg: HOST=phone-sales COLOR=red MAIL user-81168669c6fa@xymon.invalid ## Wildcard rule HOST=%^.*$ COLOR=red MAIL user-81168669c6fa@xymon.invalid Made the wildcard rule a regexp because "HOST=*" wasn't working, and added the host-specific rule because no wildcard was working. But if I trigger a red on host phone-sales, in alert.log I see: Checking criteria for host 'phone-sales', which is not yet defined; some alerts may not immediately fire What am I missing? I retired a year and a half ago, and have been out of practice, so feel free to start with "You dummy...." 
Hi,

Can you show the output of xymoncmd xymond_alert --dump-config ? That
would help us see what xymond_alert itself is interpreting things as.

The "Checking criteria for host..." should only apply when xymond_alert
(via loadhosts) hasn't reloaded the hosts.cfg yet (in this instance). That
entry looks normal, though. Does the error get shown every time
xymond_alert starts up?

Regards,
-jc

[root at hccmon xymon]# xymoncmd xymond_alert --dump-config
 120 HOST=phone-sales COLOR=red  MAIL user-81168669c6fa@xymon.invalid FORMAT=TEXT REPEAT=30 
 123 HOST=%^.* COLOR=red  MAIL user-81168669c6fa@xymon.invalid FORMAT=TEXT REPEAT=30 
Sorry, I was misinterpreting the "Checking criteria..." message as being
triggered by the red code. Perhaps I'm not waiting long enough for the
alert to fire before deciding it's not working. But the message is not
showing every time I "systemctl restart xymonlaunch.service". Will get
back to this in the morning, EST. 
Thanks, 
Jake 

/xymon

The "Checking criteria for host..." message does correlate to the red
condition. If I trigger a red, it occurs several times a minute until I
un-trigger it. 
[root at hccmon xymon]# tail -f /var/log/xymon/alert.log 
2016-11-07 18:23:21 Checking criteria for host 'phone-sales', which is
not yet defined; some alerts may not immediately fire 2016-11-07
18:23:46 Received reload request 
2016-11-07 18:23:50 Checking criteria for host 'phone-sales', which is
not yet defined; some alerts may not immediately fire 2016-11-07
18:23:50 Checking criteria for host 'phone-sales', which is not yet
defined; some alerts may not immediately fire 2016-11-07 18:24:25
Checking criteria for host 'phone-sales', which is not yet defined; some
alerts may not immediately fire 2016-11-07 18:24:25 Checking criteria
for host 'phone-sales', which is not yet defined; some alerts may not
immediately fire 2016-11-07 18:24:59 Checking criteria for host
'phone-sales', which is not yet defined; some alerts may not immediately
fire 2016-11-07 18:24:59 Checking criteria for host 'phone-sales', which
is not yet defined; some alerts may not immediately fire 2016-11-07
18:25:29 Checking criteria for host 'phone-sales', which is not yet
defined; some alerts may not immediately fire 2016-11-07 18:25:29
Checking criteria for host 'phone-sales', which is not yet defined; some
alerts may not immediately fire 2016-11-07 18:26:32 Checking criteria
for host 'phone-sales', which is not yet defined; some alerts may not
immediately fire 2016-11-07 18:26:32 Checking criteria for host
'phone-sales', which is not yet defined; some alerts may not immediately
fire 2016-11-07 18:26:36 Checking criteria for host 'phone-sales', which
is not yet defined; some alerts may not immediately fire 2016-11-07
18:26:36 Checking criteria for host 'phone-sales', which is not yet
defined; some alerts may not immediately fire
list Another Xymon User · Mon, 07 Nov 2016 22:48:30 -0500 ·
 
Ah. It's the DNS affinity thing. Small shop, pro bono support, no split
DNS. Putting the host in /etc/hosts made the alert fire. I know what I
need to do. 
-------- Original Message -------- 
		SUBJECT:
 		Re: [Xymon] Can't match alerts.cfg rules

		DATE:
 		2016-11-06 16:55

		FROM:
 		Jake <user-b71bfe4edfbc@xymon.invalid>

		TO:
quoted from Jake

 On 2016-11-06 1:09, J.C. Cleaver wrote: 
On Sat, November 5, 2016 1:14 pm, Jake wrote:
I have 4.3.27-1.el7.terabithia running on Centos 7, and can't get any rules in alerts.cfg.to match and fire an alert. I have in hosts.cfg: 192.168.2.31 phone-sales # conn In alerts.cfg: HOST=phone-sales COLOR=red MAIL user-81168669c6fa@xymon.invalid ## Wildcard rule HOST=%^.*$ COLOR=red MAIL user-81168669c6fa@xymon.invalid Made the wildcard rule a regexp because "HOST=*" wasn't working, and added the host-specific rule because no wildcard was working. But if I trigger a red on host phone-sales, in alert.log I see: Checking criteria for host 'phone-sales', which is not yet defined; some alerts may not immediately fire What am I missing? I retired a year and a half ago, and have been out of practice, so feel free to start with "You dummy...."
Hi,

Can you show the output of xymoncmd xymond_alert --dump-config ? That
would help us see what xymond_alert itself is interpreting things as.

The "Checking criteria for host..." should only apply when xymond_alert
(via loadhosts) hasn't reloaded the hosts.cfg yet (in this instance). That
entry looks normal, though. Does the error get shown every time
xymond_alert starts up?

Regards,
-jc

[root at hccmon xymon]# xymoncmd xymond_alert --dump-config
 120 HOST=phone-sales COLOR=red  MAIL user-81168669c6fa@xymon.invalid FORMAT=TEXT REPEAT=30 
 123 HOST=%^.* COLOR=red  MAIL user-81168669c6fa@xymon.invalid FORMAT=TEXT REPEAT=30 
Sorry, I was misinterpreting the "Checking criteria..." message as being
triggered by the red code. Perhaps I'm not waiting long enough for the
alert to fire before deciding it's not working. But the message is not
showing every time I "systemctl restart xymonlaunch.service". Will get
back to this in the morning, EST. 
Thanks, 
Jake
Attachments (1)
list Jake · Wed, 16 Nov 2016 16:27:56 -0500 ·
 
On 2016-11-07 18:37, Jake wrote:: 
quoted from Jake
On 2016-11-06 16:55, Jake wrote: 
On 2016-11-06 1:09, J.C. Cleaver wrote: 
On Sat, November 5, 2016 1:14 pm, Jake wrote:
I have 4.3.27-1.el7.terabithia running on Centos 7, and can't get any rules in alerts.cfg.to match and fire an alert. I have in hosts.cfg: 192.168.2.31 phone-sales # conn In alerts.cfg: HOST=phone-sales COLOR=red MAIL user-81168669c6fa@xymon.invalid ## Wildcard rule HOST=%^.*$ COLOR=red MAIL user-81168669c6fa@xymon.invalid Made the wildcard rule a regexp because "HOST=*" wasn't working, and added the host-specific rule because no wildcard was working. But if I trigger a red on host phone-sales, in alert.log I see: Checking criteria for host 'phone-sales', which is not yet defined; some alerts may not immediately fire What am I missing? I retired a year and a half ago, and have been out of practice, so feel free to start with "You dummy...." 
Hi,

Can you show the output of xymoncmd xymond_alert --dump-config ? That
would help us see what xymond_alert itself is interpreting things as.

The "Checking criteria for host..." should only apply when xymond_alert
(via loadhosts) hasn't reloaded the hosts.cfg yet (in this instance). That
entry looks normal, though. Does the error get shown every time
xymond_alert starts up?

Regards,
-jc

[root at hccmon xymon]# xymoncmd xymond_alert --dump-config
 120 HOST=phone-sales COLOR=red  MAIL user-81168669c6fa@xymon.invalid FORMAT=TEXT REPEAT=30 
 123 HOST=%^.* COLOR=red  MAIL user-81168669c6fa@xymon.invalid FORMAT=TEXT REPEAT=30 
Sorry, I was misinterpreting the "Checking criteria..." message as being
triggered by the red code. Perhaps I'm not waiting long enough for the
alert to fire before deciding it's not working. But the message is not
showing every time I "systemctl restart xymonlaunch.service". Will get
back to this in the morning, EST. 
Thanks, 
Jake 
/xymon

The "Checking criteria for host..." message does correlate to the red
condition. If I trigger a red, it occurs several times a minute until I
un-trigger it. 
[root at hccmon xymon]# tail -f /var/log/xymon/alert.log 
2016-11-07 18:23:21 Checking criteria for host 'phone-sales', which is
not yet defined; some alerts may not immediately fire 2016-11-07
18:23:46 Received reload request 
2016-11-07 18:23:50 Checking criteria for host 'phone-sales', which is
not yet defined; some alerts may not immediately fire 2016-11-07
18:23:50 Checking criteria for host 'phone-sales', which is not yet
defined; some alerts may not immediately fire 2016-11-07 18:24:25
Checking criteria for host 'phone-sales', which is not yet defined; some
alerts may not immediately fire 2016-11-07 18:24:25 Checking criteria
for host 'phone-sales', which is not yet defined; some alerts may not
immediately fire 2016-11-07 18:24:59 Checking criteria for host
'phone-sales', which is not yet defined; some alerts may not immediately
fire 2016-11-07 18:24:59 Checking criteria for host 'phone-sales', which
is not yet defined; some alerts may not immediately fire 2016-11-07
18:25:29 Checking criteria for host 'phone-sales', which is not yet
defined; some alerts may not immediately fire 2016-11-07 18:25:29
Checking criteria for host 'phone-sales', which is not yet defined; some
alerts may not immediately fire 2016-11-07 18:26:32 Checking criteria
for host 'phone-sales', which is not yet defined; some alerts may not
immediately fire 2016-11-07 18:26:32 Checking criteria for host
'phone-sales', which is not yet defined; some alerts may not immediately
fire 2016-11-07 18:26:36 Checking criteria for host 'phone-sales', which
is not yet defined; some alerts may not immediately fire 2016-11-07
18:26:36 Checking criteria for host 'phone-sales', which is not yet
defined; some alerts may not immediately fire 


I ended up finding out that in my situation, a host in a hosts.cfg group
gives the "not yet defined" log and fails to fire an alert. A host not
in a group works as expected. 
hosts.cfg: 
group Phones 
192.168.2.253 phoneserver # conn 
192.168.2.31 phone-sales # conn 
If I move phone-sales out of the group (putting it right after the Xymon
server), the wildcard alert fires. 
alerts.cfg: 
HOST=* COLOR=red 
MAIL alerts at mydomain 
/etc/hosts: 
192.168.2.31 phone-sales 
dnswitch.conf has "hosts: files dns myhostname" and I can "ping
phone-sales" from the command line. 
I'm tearing my hair out over this. It seems such a simple setup, so I
figure it's something stupid I'm overlooking. Does anyone have any
insight into this? 
Thanks, 
Jake
list Jake · Fri, 18 Nov 2016 16:31:30 -0500 ·
 
Well poke me with a fork. All of my conclusions from testing were
entirely coincidental to moving hosts above the "directory
/etc/xymon/hosts.d" line. Apparently an empty hosts.d is just fine with
xymond itself but causes xymond_alert to miss any hosts below the
include line. Or maybe it just doesn't see anything past that regardless
of the directory contents. Removing the "directory" directive made
everything work and caused the "not yet defined messages" to stop
occurring. 
quoted from Jake
On 2016-11-16 16:27, Jake wrote: 
On 2016-11-07 18:37, Jake wrote:: 
On 2016-11-06 16:55, Jake wrote: 
On 2016-11-06 1:09, J.C. Cleaver wrote: 
On Sat, November 5, 2016 1:14 pm, Jake wrote:
I have 4.3.27-1.el7.terabithia running on Centos 7, and can't get any rules in alerts.cfg.to match and fire an alert. I have in hosts.cfg: 192.168.2.31 phone-sales # conn In alerts.cfg: HOST=phone-sales COLOR=red MAIL user-81168669c6fa@xymon.invalid ## Wildcard rule HOST=%^.*$ COLOR=red MAIL user-81168669c6fa@xymon.invalid Made the wildcard rule a regexp because "HOST=*" wasn't working, and added the host-specific rule because no wildcard was working. But if I trigger a red on host phone-sales, in alert.log I see: Checking criteria for host 'phone-sales', which is not yet defined; some alerts may not immediately fire What am I missing? I retired a year and a half ago, and have been out of practice, so feel free to start with "You dummy...." 
Hi,

Can you show the output of xymoncmd xymond_alert --dump-config ? That
would help us see what xymond_alert itself is interpreting things as.

The "Checking criteria for host..." should only apply when xymond_alert
(via loadhosts) hasn't reloaded the hosts.cfg yet (in this instance). That
entry looks normal, though. Does the error get shown every time
xymond_alert starts up?

Regards,
-jc

[root at hccmon xymon]# xymoncmd xymond_alert --dump-config
 120 HOST=phone-sales COLOR=red  MAIL user-81168669c6fa@xymon.invalid FORMAT=TEXT REPEAT=30 
 123 HOST=%^.* COLOR=red  MAIL user-81168669c6fa@xymon.invalid FORMAT=TEXT REPEAT=30 
Sorry, I was misinterpreting the "Checking criteria..." message as being
triggered by the red code. Perhaps I'm not waiting long enough for the
alert to fire before deciding it's not working. But the message is not
showing every time I "systemctl restart xymonlaunch.service". Will get
back to this in the morning, EST. 
Thanks, 
Jake 
/xymon

The "Checking criteria for host..." message does correlate to the red
condition. If I trigger a red, it occurs several times a minute until I
un-trigger it. 
[root at hccmon xymon]# tail -f /var/log/xymon/alert.log 
2016-11-07 18:23:21 Checking criteria for host 'phone-sales', which is
not yet defined; some alerts may not immediately fire 2016-11-07
18:23:46 Received reload request 
2016-11-07 18:23:50 Checking criteria for host 'phone-sales', which is
not yet defined; some alerts may not immediately fire 2016-11-07
18:23:50 Checking criteria for host 'phone-sales', which is not yet
defined; some alerts may not immediately fire 2016-11-07 18:24:25
Checking criteria for host 'phone-sales', which is not yet defined; some
alerts may not immediately fire 2016-11-07 18:24:25 Checking criteria
for host 'phone-sales', which is not yet defined; some alerts may not
immediately fire 2016-11-07 18:24:59 Checking criteria for host
'phone-sales', which is not yet defined; some alerts may not immediately
fire 2016-11-07 18:24:59 Checking criteria for host 'phone-sales', which
is not yet defined; some alerts may not immediately fire 2016-11-07
18:25:29 Checking criteria for host 'phone-sales', which is not yet
defined; some alerts may not immediately fire 2016-11-07 18:25:29
Checking criteria for host 'phone-sales', which is not yet defined; some
alerts may not immediately fire 2016-11-07 18:26:32 Checking criteria
for host 'phone-sales', which is not yet defined; some alerts may not
immediately fire 2016-11-07 18:26:32 Checking criteria for host
'phone-sales', which is not yet defined; some alerts may not immediately
fire 2016-11-07 18:26:36 Checking criteria for host 'phone-sales', which
is not yet defined; some alerts may not immediately fire 2016-11-07
18:26:36 Checking criteria for host 'phone-sales', which is not yet
defined; some alerts may not immediately fire 


I ended up finding out that in my situation, a host in a hosts.cfg group
gives the "not yet defined" log and fails to fire an alert. A host not
in a group works as expected. 
hosts.cfg: 
group Phones 
192.168.2.253 phoneserver # conn 
192.168.2.31 phone-sales # conn 
If I move phone-sales out of the group (putting it right after the Xymon
server), the wildcard alert fires. 
alerts.cfg: 
HOST=* COLOR=red 
MAIL alerts at mydomain 
/etc/hosts: 
192.168.2.31 phone-sales 
dnswitch.conf has "hosts: files dns myhostname" and I can "ping
phone-sales" from the command line. 
I'm tearing my hair out over this. It seems such a simple setup, so I
figure it's something stupid I'm overlooking. Does anyone have any
insight into this? 
Thanks, 
Jake