ports check for a VIP...
list Ian Diddams
we have a HA mysql db cluster that runs with a VIP (drbd, pacemaker, corosync) we want a ports check for 3306 - but if we configure it for the two systems in the cluster one of them will always be red as its the slave and not running mysqld (it only runs on the master). the clear alternative is to somehow run a 3306 check on the VIP...? but the VIP itself is only an IP in hosts.cfg (so we can conn check its existence etc) and so isnt a client sending reports back to the server. any hints as to how we can acheive such a test ?
list Christoph Zechner
Hi,
▸
On 01/10/2021 14:15, Ian Diddams via Xymon wrote:we have a HA mysql db cluster that runs with a VIP (drbd, pacemaker, corosync) we want a ports check for 3306 - but if we configure it for the two systems in the cluster one of them will always be red as its the slave and not running mysqld (it only runs on the master). the clear alternative is to somehow run a 3306 check on the VIP...? but the VIP itself is only an IP in hosts.cfg (so we can conn check its existence etc) and so isnt a client sending reports back to the server. any hints as to how we can acheive such a test ?
You could try a combo status test, I've implemented these for connection tests, but it should work for port checks too, I guess (have not tried it though). (all paths are for Debian, that's what I am using) I defined a target in /etc/xymon/combo.cfg like this: computer.conn = (computer1.conn || computer2.conn) This returns green if one of the machines is green and also shows both statuses in the web interface. I guess, this should also work with ports... Good luck! Christoph
list Ian Diddams
that's interesting
but presumably it would require something like
VIP V
system A
system B
V.ports = (V.conn && A.ports) ||? (V.conn && B.ports)
but that would need A & B reporting on ports (ie 3306) - but one of those will still always be red though?
I suspect Ive not understood something!
On Friday, 1 October 2021, 13:41:02 BST, Christoph Zechner <user-249716582ccc@xymon.invalid> wrote: Hi,
▸
On 01/10/2021 14:15, Ian Diddams via Xymon wrote:we have a HA mysql db cluster that runs with a VIP (drbd, pacemaker, corosync) we want a ports check for 3306 - but if we configure it for the two systems in the cluster one of them will always be red as its the slave and not running mysqld (it only runs on the master). the clear alternative is to somehow run a 3306 check on the VIP...? but the VIP itself is only an IP in hosts.cfg (so we can conn check its existence etc) and so isnt a client sending reports back to the server. any hints as to how we can acheive such a test ?
You could try a combo status test, I've implemented these for connection tests, but it should work for port checks too, I guess (have not tried it though). (all paths are for Debian, that's what I am using) I defined a target in /etc/xymon/combo.cfg like this: computer.conn = (computer1.conn || computer2.conn) This returns green if one of the machines is green and also shows both statuses in the web interface. I guess, this should also work with ports... Good luck! Christoph
list Christoph Zechner
▸
On 01/10/2021 16:21, Ian Diddams wrote:
that's interesting but presumably it would require something like VIP V system A system B
My example is a bit different than your setup, since I'm only checking for connection and I'm doing this from two different xymon proxies which both report to the main xymon server. I suspect you have to define it like you wrote, yes.
V.ports = (V.conn && A.ports) ||? (V.conn && B.ports)
Do you also need the connection status as well? "ports" and "conn" are two different colums, I don't think mixing them would be wise?
but that would need A & B reporting on ports (ie 3306) - but one of those will still always be red though?
That's why you need the "||", it's an or, so as long as one of your two hosts is green, your overall status is green.
I suspect Ive not understood something!
I don't think so, I fear, my example was not really that great, sorry. As I said, I've never tried it with a ports check, only connection, which works like a charm. Cheers Christoph
▸
On Friday, 1 October 2021, 13:41:02 BST, Christoph Zechner <user-249716582ccc@xymon.invalid> wrote: Hi, On 01/10/2021 14:15, Ian Diddams via Xymon wrote:we have a HA mysql db cluster that runs with a VIP (drbd, pacemaker, corosync) we want a ports check for 3306 - but if we configure it for the two systems in the cluster one of them will always be red as its the slave and not running mysqld (it only runs on the master). the clear alternative is to somehow run a 3306 check on the VIP...? but the VIP itself is only an IP in hosts.cfg (so we can conn check its existence etc) and so isnt a client sending reports back to the server. any hints as to how we can acheive such a test ?You could try a combo status test, I've implemented these for connection tests, but it should work for port checks too, I guess (have not tried it though). (all paths are for Debian, that's what I am using) I defined a target in /etc/xymon/combo.cfg like this: computer.conn = (computer1.conn || computer2.conn) This returns green if one of the machines is green and also shows both statuses in the web interface. I guess, this should also work with ports... Good luck! Christoph
list Christoph Zechner
▸
On 01/10/2021 17:39, Ian Diddams wrote:
Do you also need the connection status as well? "ports" and "conn" are two different colums, I don't think mixing them would be wise?
No - TBH not really
▸
yes - but then presumably that emans the other host is red. And that will just spoawn reports form our monitoring team or at the very least obfuscate the page sending it red for no real reason.
It will not spawn any reports or alarm, as long as one of them is green, everything is fine. So in your case, there would be three statuses: 1) overall status of this check: green if one is green 2) system A red or green 3) system B reg or green There are no alarms as long as one of them is green.
The bottom line is only one server in the cluster has the port in use at any one time and we dont want a "red" simoly becasue that is actually the correct situation. cheers ian
list Ian Diddams
▸
It will not spawn any reports or alarm, as long as one of them is green, everything is fine. So in your case, there would be three statuses:
1) overall status of this check: green if one is green 2) system A red or green 3) system B reg or green
There are no alarms as long as one of them is green.
well having set it up, the combo test is green indeed - but the two servfers that make up the combo test are shwoing * one green (as the port check is green as mysqld is runnoing on that system) * one red? (as the port check is red as mysqld is runnoing on that system) So? we do get a red showing that the monitoring team will respond to. I cant make it "clear" as there are other procs and ports being checked that do need a red if alerting. so unfortunately it doesnt seem to be the solution - but I do appreciate the attem[pt to help :-) cheers ian
list Christoph Zechner
▸
On 04/10/2021 10:25, Ian Diddams wrote:
It will not spawn any reports or alarm, as long as one of them is green, everything is fine. So in your case, there would be three statuses:1) overall status of this check: green if one is green 2) system A red or green 3) system B reg or greenThere are no alarms as long as one of them is green.well having set it up, the combo test is green indeed - but the two servfers that make up the combo test are shwoing * one green (as the port check is green as mysqld is runnoing on that system) * one red (as the port check is red as mysqld is runnoing on that system) So? we do get a red showing that the monitoring team will respond to. I cant make it "clear" as there are other procs and ports being checked that do need a red if alerting.
Oh, I see. That is the way it should work, but I did not anticipate your team responding to the sub-status as well, sorry, my mistake, I could have worded that more clearly... :-/
so unfortunately it doesnt seem to be the solution - but I do appreciate the attem[pt to help :-)
No worries, thanks for trying! Cheers Christoph
cheers ian
list Jeremy Laidman
▸
On Mon, 4 Oct 2021 at 19:25, Ian Diddams via Xymon <xymon at xymon.com> wrote:
well having set it up, the combo test is green indeed - but the two servfers that make up the combo test are shwoing * one green (as the port check is green as mysqld is runnoing on that system) * one red (as the port check is red as mysqld is runnoing on that system) So we do get a red showing that the monitoring team will respond to. I cant make it "clear" as there are other procs and ports being checked that do need a red if alerting. so unfortunately it doesnt seem to be the solution - but I do appreciate the attem[pt to help :-)
You can silence the alarms for the individual tests, in alarms.cfg. J
list Jeremy Laidman
Woops, I mean alerts.cfg.
▸
On Mon, 4 Oct 2021 at 19:55, Jeremy Laidman <user-0608abae5e7c@xymon.invalid> wrote:
On Mon, 4 Oct 2021 at 19:25, Ian Diddams via Xymon <xymon at xymon.com> wrote:well having set it up, the combo test is green indeed - but the two servfers that make up the combo test are shwoing * one green (as the port check is green as mysqld is runnoing on that system) * one red (as the port check is red as mysqld is runnoing on that system) So we do get a red showing that the monitoring team will respond to. I cant make it "clear" as there are other procs and ports being checked that do need a red if alerting. so unfortunately it doesnt seem to be the solution - but I do appreciate the attem[pt to help :-)You can silence the alarms for the individual tests, in alarms.cfg. J
list Ian Diddams
actually, Ive possibly? found a solution.
analysis.cfg
HOST=wp-vldyn-estrela
??????? PORT "LOCAL=%([.:]3306)$" state=LISTEN min=1 color=clear TEXT=MySQL
HOST=wp-vldyn-rafeiro
??????? PORT "LOCAL=%([.:]3306)$" state=LISTEN min=1 color=clear TEXT=MySQL
and now estrela's port check shows a shite unhappy face, and rafeiro's port check shows green
and combo.cfg
mysqldHA.ports = ("wp-vldyn-estrela".ports) || ("wp-vldyn-rafeiro".ports)
which seems to do the trick? - the mysqldHA ports check shows green
intrigiingly the actual output on that combo check shows
(wp-vldyn-estrela.ports)||(wp-vldyn-rafeiro.ports) = (0-0-1)||(0-0-1) = 1
wp
vldyn
estrela.ports
wp
vldyn
rafeiro.ports
i.e. its all showing clear for the individual lines which are also separated where the hostname includes a hyphen...? is that all correct ?
ian
▸
On Monday, 4 October 2021, 09:38:21 BST, Christoph Zechner <user-249716582ccc@xymon.invalid> wrote: On 04/10/2021 10:25, Ian Diddams wrote:? >It will not spawn any reports or alarm, as long as one of them is green, ? >everything is fine. So in your case, there would be three statuses: ? >1) overall status of this check: green if one is green ? >2) system A red or green ? >3) system B reg or green ? >There are no alarms as long as one of them is green. well having set it up, the combo test is green indeed - but the two servfers that make up the combo test are shwoing * one green (as the port check is green as mysqld is runnoing on that system) * one red (as the port check is red as mysqld is runnoing on that system) So? we do get a red showing that the monitoring team will respond to. I cant make it "clear" as there are other procs and ports being checked that do need a red if alerting.
Oh, I see. That is the way it should work, but I did not anticipate your team responding to the sub-status as well, sorry, my mistake, I could have worded that more clearly... :-/
so unfortunately it doesnt seem to be the solution - but I do appreciate the attem[pt to help :-)
No worries, thanks for trying! Cheers Christoph
cheers ian
list Christoph Zechner
▸
On 04/10/2021 14:15, Ian Diddams wrote:
actually, Ive possibly? found a solution.
analysis.cfg
HOST=wp-vldyn-estrela
??????? PORT "LOCAL=%([.:]3306)$" state=LISTEN min=1 color=clear TEXT=MySQL
HOST=wp-vldyn-rafeiro
??????? PORT "LOCAL=%([.:]3306)$" state=LISTEN min=1 color=clear TEXT=MySQL
and now estrela's port check shows a shite unhappy face, and rafeiro's port check shows green
and combo.cfg
mysqldHA.ports = ("wp-vldyn-estrela".ports) || ("wp-vldyn-rafeiro".ports)
which seems to do the trick? - the mysqldHA ports check shows green
intrigiingly the actual output on that combo check shows
(wp-vldyn-estrela.ports)||(wp-vldyn-rafeiro.ports) = (0-0-1)||(0-0-1) = 1
clear wp
clear vldyn
clear estrela.ports
clear wp
clear vldyn
clear rafeiro.ports
i.e. its all showing clear for the individual lines which are also separated where the hostname includes a hyphen...? is that all correct ?Interesting... I'd say this looks correct, yes. Christoph
▸
ian On Monday, 4 October 2021, 09:38:21 BST, Christoph Zechner <user-249716582ccc@xymon.invalid> wrote: On 04/10/2021 10:25, Ian Diddams wrote:? >It will not spawn any reports or alarm, as long as one of them is green, ? >everything is fine. So in your case, there would be three statuses: ? >1) overall status of this check: green if one is green ? >2) system A red or green ? >3) system B reg or green ? >There are no alarms as long as one of them is green. well having set it up, the combo test is green indeed - but the two servfers that make up the combo test are shwoing * one green (as the port check is green as mysqld is runnoing on that system) * one red (as the port check is red as mysqld is runnoing on that system) So? we do get a red showing that the monitoring team will respond to. I cant make it "clear" as there are other procs and ports being checked that do need a red if alerting.Oh, I see. That is the way it should work, but I did not anticipate your team responding to the sub-status as well, sorry, my mistake, I could have worded that more clearly... :-/so unfortunately it doesnt seem to be the solution - but I do appreciate the attem[pt to help :-)No worries, thanks for trying! Cheers Christophcheers ian
list Jeremy Laidman
▸
On Mon, 4 Oct 2021 at 23:15, Ian Diddams via Xymon <xymon at xymon.com> wrote:
intrigiingly the actual output on that combo check shows (wp-vldyn-estrela.ports)||(wp-vldyn-rafeiro.ports) = (0-0-1)||(0-0-1) = 1
<snip>
i.e. its all showing clear for the individual lines which are also separated where the hostname includes a hyphen... is that all correct ?
I don't think so. From the manpage of combo.cfg:"Note: If the expression involves hostnames with a character that is also an operator - e.g. if you have a host "t1-router-newyork.foo.com" with a dash in the hostname - then the operator-character must be escaped with a backslash '\' in the expression, or it will be interpreted as an operator. E.g. like this: nyc.conn = (t1\-router\-nyc.conn || backup\-router\-nyc.conn)" That's why it thinks you have hostnames like "wp" and "vldyn". It's almost certainly not testing what you want it to be testing. Cheers Jeremy
list Ian Diddams
That's great Jeremy - many thanks Now (wp\-vldyn\-estrela.ports)||(wp\-vldyn\-rafeiro.ports) = (1)||(1) = 1 wp-vldyn-estrela.ports wp-vldyn-rafeiro.ports ian
▸
On Wednesday, 6 October 2021, 05:15:11 BST, Jeremy Laidman <user-0608abae5e7c@xymon.invalid> wrote:
On Mon, 4 Oct 2021 at 23:15, Ian Diddams via Xymon <xymon at xymon.com> wrote:
intrigiingly the actual output on that combo check shows
(wp-vldyn-estrela.ports)||(wp-vldyn-rafeiro.ports) = (0-0-1)||(0-0-1) = 1
?<snip>
i.e. its all showing clear for the individual lines which are also separated where the hostname includes a hyphen...? is that all correct ?
I don't think so. From the manpage of combo.cfg:
"Note: If the expression involves hostnames with a character that is also an operator - e.g. if you have a host "t1-router-newyork.foo.com" with a dash in the hostname - then the operator-character must be escaped with a backslash '\' in the expression, or it will be interpreted as an operator. E.g. like this:
?nyc.conn = (t1\-router\-nyc.conn || backup\-router\-nyc.conn)"
That's why it thinks you have hostnames like "wp" and "vldyn". It's almost certainly not testing what you want it to be testing.
CheersJeremy
list Ian Diddams
Purely a sort of FYI I guess...? not sure what was occurring but last Friday early hours the clusetrr had a bit of a tizz and both nodes had no mysqld running and so procs was not green on both nodes...? but the combo check remained green throughout..
▸
On Thursday, 7 October 2021, 09:41:02 BST, Ian Diddams via Xymon <xymon at xymon.com> wrote:
That's great Jeremy - many thanks
Now
(wp\-vldyn\-estrela.ports)||(wp\-vldyn\-rafeiro.ports) = (1)||(1) = 1
wp-vldyn-estrela.ports
wp-vldyn-rafeiro.ports
ian
On Wednesday, 6 October 2021, 05:15:11 BST, Jeremy Laidman <user-0608abae5e7c@xymon.invalid> wrote:
On Mon, 4 Oct 2021 at 23:15, Ian Diddams via Xymon <xymon at xymon.com> wrote:
intrigiingly the actual output on that combo check shows
(wp-vldyn-estrela.ports)||(wp-vldyn-rafeiro.ports) = (0-0-1)||(0-0-1) = 1
?<snip>
i.e. its all showing clear for the individual lines which are also separated where the hostname includes a hyphen...? is that all correct ?
I don't think so. From the manpage of combo.cfg:
"Note: If the expression involves hostnames with a character that is also an operator - e.g. if you have a host "t1-router-newyork.foo.com" with a dash in the hostname - then the operator-character must be escaped with a backslash '\' in the expression, or it will be interpreted as an operator. E.g. like this:
?nyc.conn = (t1\-router\-nyc.conn || backup\-router\-nyc.conn)"
That's why it thinks you have hostnames like "wp" and "vldyn". It's almost certainly not testing what you want it to be testing.
CheersJeremy
list Jeremy Laidman
What was the output on the combo check page, during the outage? On Mon, 11 Oct 2021 at 19:16, Ian Diddams via Xymon <xymon at xymon.com> wrote:
---------- Forwarded message ---------- From: Ian Diddams <user-7fbf34ed5219@xymon.invalid> To: xymon <xymon at xymon.com> Cc: Bcc: Date: Mon, 11 Oct 2021 08:16:02 +0000 (UTC) Subject: Re: [Xymon] ports check for a VIP...
▸
Purely a sort of FYI I guess... not sure what was occurring but last
Friday early hours the clusetrr had a bit of a tizz and both nodes had no
mysqld running and so procs was not green on both nodes... but the combo
check remained green throughout..
On Thursday, 7 October 2021, 09:41:02 BST, Ian Diddams via Xymon <
xymon at xymon.com> wrote:
That's great Jeremy - many thanks
Now
(wp\-vldyn\-estrela.ports)||(wp\-vldyn\-rafeiro.ports) = (1)||(1) = 1
[image: clear] wp-vldyn-estrela.ports
[image: green] wp-vldyn-rafeiro.ports
▸
ian
On Wednesday, 6 October 2021, 05:15:11 BST, Jeremy Laidman <
user-0608abae5e7c@xymon.invalid> wrote:
On Mon, 4 Oct 2021 at 23:15, Ian Diddams via Xymon <xymon at xymon.com>
wrote:
intrigiingly the actual output on that combo check shows
(wp-vldyn-estrela.ports)||(wp-vldyn-rafeiro.ports) = (0-0-1)||(0-0-1) = 1
<snip>
i.e. its all showing clear for the individual lines which are also
separated where the hostname includes a hyphen... is that all correct ?
I don't think so. From the manpage of combo.cfg:
"Note: If the expression involves hostnames with a character that is also
an operator - e.g. if you have a host "t1-router-newyork.foo.com" with a
dash in the hostname - then the operator-character must be escaped with a
backslash '\' in the expression, or it will be interpreted as an operator.
E.g. like this:
nyc.conn = (t1\-router\-nyc.conn || backup\-router\-nyc.conn)"
That's why it thinks you have hostnames like "wp" and "vldyn". It's almost
certainly not testing what you want it to be testing.
Cheers
Jeremy
---------- Forwarded message ----------
From: Ian Diddams via Xymon <xymon at xymon.com>
To: xymon <xymon at xymon.com>
Cc:
Bcc:
Date: Mon, 11 Oct 2021 08:16:02 +0000 (UTC)
Subject: Re: [Xymon] ports check for a VIP...
list Ian Diddams
as reported previously...
▸
(wp\-vldyn\-estrela.ports)||(wp\-vldyn\-rafeiro.ports) = (1)||(1) = 1
wp-vldyn-estrela.ports
wp-vldyn-rafeiro.ports
ian
On Monday, 11 October 2021, 12:44:30 BST, Jeremy Laidman <user-0608abae5e7c@xymon.invalid> wrote:
What was the output on the combo check page, during the outage?
On Mon, 11 Oct 2021 at 19:16, Ian Diddams via Xymon <xymon at xymon.com> wrote:
---------- Forwarded message ----------
From:?Ian Diddams <user-7fbf34ed5219@xymon.invalid>
To:?xymon <xymon at xymon.com>
Cc:?
Bcc:?
Date:?Mon, 11 Oct 2021 08:16:02 +0000 (UTC)
Subject:?Re: [Xymon] ports check for a VIP...
▸
Purely a sort of FYI I guess...? not sure what was occurring but last Friday early hours the clusetrr had a bit of a tizz and both nodes had no mysqld running and so procs was not green on both nodes...? but the combo check remained green throughout..
On Thursday, 7 October 2021, 09:41:02 BST, Ian Diddams via Xymon <xymon at xymon.com> wrote:
That's great Jeremy - many thanks
Now
(wp\-vldyn\-estrela.ports)||(wp\-vldyn\-rafeiro.ports) = (1)||(1) = 1
wp-vldyn-estrela.ports
wp-vldyn-rafeiro.ports
ian
On Wednesday, 6 October 2021, 05:15:11 BST, Jeremy Laidman <user-0608abae5e7c@xymon.invalid> wrote:
On Mon, 4 Oct 2021 at 23:15, Ian Diddams via Xymon <xymon at xymon.com> wrote:
intrigiingly the actual output on that combo check shows
(wp-vldyn-estrela.ports)||(wp-vldyn-rafeiro.ports) = (0-0-1)||(0-0-1) = 1
?<snip>
i.e. its all showing clear for the individual lines which are also separated where the hostname includes a hyphen...? is that all correct ?
I don't think so. From the manpage of combo.cfg:
"Note: If the expression involves hostnames with a character that is also an operator - e.g. if you have a host "t1-router-newyork.foo.com" with a dash in the hostname - then the operator-character must be escaped with a backslash '\' in the expression, or it will be interpreted as an operator. E.g. like this:
?nyc.conn = (t1\-router\-nyc.conn || backup\-router\-nyc.conn)"
That's why it thinks you have hostnames like "wp" and "vldyn". It's almost certainly not testing what you want it to be testing.
CheersJeremy
---------- Forwarded message ----------
From:?Ian Diddams via Xymon <xymon at xymon.com>
To:?xymon <xymon at xymon.com>
Cc:?
Bcc:?
Date:?Mon, 11 Oct 2021 08:16:02 +0000 (UTC)
Subject:?Re: [Xymon] ports check for a VIP...
list Jeremy Laidman
So during the outage, both wp-vldyn-estrelata.ports and wp-vldyn-rafeiro.ports were green? That's what the (1)||(1) means. Is it possible that the two services were down, but just not at the same time? Maybe you can do a snapshot report at the time of the outage, and check the two procs status values? Note that "yellow" is considered non-red, so comes out as a "1" in the boolean calculations (only "red" is "0", unless configured otherwise). Although I imagine if the mysqld process was down, the ports status wouldn't be yellow. Regardless, I recommend you test the combo status by intentionally adding checks that are down, or manually downing services to cause reds. You can simulate this result without actually breaking the service by running something like: xymoncmd xymon 127.1 "status wp-vldyn-estrela.ports red testing" xymoncmd xymon 127.1 "status wp-vldyn-rafeiro.ports red testing" and until the real test runs next, the status will be red. At that point, the combo should also turn red. If it doesn't, you might have an error in your config. After running the above two commands, perhaps run: xymoncmd combostatus --debug --no-update and see how it's parsing the logic. Cheers Jeremy
▸
On Tue, 12 Oct 2021 at 20:12, Ian Diddams <user-7fbf34ed5219@xymon.invalid> wrote:
as reported previously... (wp\-vldyn\-estrela.ports)||(wp\-vldyn\-rafeiro.ports) = (1)||(1) = 1 [image: clear] wp-vldyn-estrela.ports [image: green] wp-vldyn-rafeiro.ports ian On Monday, 11 October 2021, 12:44:30 BST, Jeremy Laidman < user-0608abae5e7c@xymon.invalid> wrote: What was the output on the combo check page, during the outage? On Mon, 11 Oct 2021 at 19:16, Ian Diddams via Xymon <xymon at xymon.com> wrote: ---------- Forwarded message ---------- From: Ian Diddams <user-7fbf34ed5219@xymon.invalid> To: xymon <xymon at xymon.com> Cc: Bcc: Date: Mon, 11 Oct 2021 08:16:02 +0000 (UTC) Subject: Re: [Xymon] ports check for a VIP... Purely a sort of FYI I guess... not sure what was occurring but last Friday early hours the clusetrr had a bit of a tizz and both nodes had no mysqld running and so procs was not green on both nodes... but the combo check remained green throughout.. On Thursday, 7 October 2021, 09:41:02 BST, Ian Diddams via Xymon < xymon at xymon.com> wrote: That's great Jeremy - many thanks Now (wp\-vldyn\-estrela.ports)||(wp\-vldyn\-rafeiro.ports) = (1)||(1) = 1 [image: clear] wp-vldyn-estrela.ports [image: green] wp-vldyn-rafeiro.ports ian On Wednesday, 6 October 2021, 05:15:11 BST, Jeremy Laidman < user-0608abae5e7c@xymon.invalid> wrote: On Mon, 4 Oct 2021 at 23:15, Ian Diddams via Xymon <xymon at xymon.com> wrote: intrigiingly the actual output on that combo check shows (wp-vldyn-estrela.ports)||(wp-vldyn-rafeiro.ports) = (0-0-1)||(0-0-1) = 1 <snip> i.e. its all showing clear for the individual lines which are also separated where the hostname includes a hyphen... is that all correct ? I don't think so. From the manpage of combo.cfg: "Note: If the expression involves hostnames with a character that is also an operator - e.g. if you have a host "t1-router-newyork.foo.com" with a dash in the hostname - then the operator-character must be escaped with a backslash '\' in the expression, or it will be interpreted as an operator. E.g. like this: nyc.conn = (t1\-router\-nyc.conn || backup\-router\-nyc.conn)" That's why it thinks you have hostnames like "wp" and "vldyn". It's almost certainly not testing what you want it to be testing. Cheers Jeremy ---------- Forwarded message ---------- From: Ian Diddams via Xymon <xymon at xymon.com> To: xymon <xymon at xymon.com> Cc: Bcc: Date: Mon, 11 Oct 2021 08:16:02 +0000 (UTC) Subject: Re: [Xymon] ports check for a VIP...
list Ian Diddams
Thanks for your patience Jeremy - Ill try your test suggestions to see what happens...
cheers
ian
On Tuesday, 12 October 2021, 10:48:04 BST, Jeremy Laidman <user-0608abae5e7c@xymon.invalid> wrote:
So during the outage, both wp-vldyn-estrelata.ports?and wp-vldyn-rafeiro.ports were green? That's what the (1)||(1) means.
Is it possible that the two services were down, but just not at the?same time? Maybe you can do a snapshot report at the time of the outage, and check the two procs status values?
Note that "yellow" is considered non-red, so comes out as a "1" in the boolean calculations (only "red" is "0", unless configured otherwise). Although?I imagine if the mysqld process was down, the ports status wouldn't be yellow.
Regardless, I recommend you test the combo status by intentionally adding checks that are down, or manually downing services to cause reds. You can simulate this result without actually breaking the service by running something like:
? xymoncmd xymon 127.1 "status wp-vldyn-estrela.ports red testing"??xymoncmd xymon 127.1 "status wp-vldyn-rafeiro.ports red testing"
▸
and until the real test runs next, the status will be red. At that point, the combo should also turn red. If it doesn't, you might have an error in your config.
After running the above two commands, perhaps run:
?xymoncmd combostatus --debug --no-update
and see how it's parsing the logic.
CheersJeremy
▸
On Tue, 12 Oct 2021 at 20:12, Ian Diddams <user-7fbf34ed5219@xymon.invalid> wrote:
as reported previously...
(wp\-vldyn\-estrela.ports)||(wp\-vldyn\-rafeiro.ports) = (1)||(1) = 1
wp-vldyn-estrela.ports
wp-vldyn-rafeiro.ports
ian
On Monday, 11 October 2021, 12:44:30 BST, Jeremy Laidman <user-0608abae5e7c@xymon.invalid> wrote:
What was the output on the combo check page, during the outage?
On Mon, 11 Oct 2021 at 19:16, Ian Diddams via Xymon <xymon at xymon.com> wrote:
---------- Forwarded message ----------
From:?Ian Diddams <user-7fbf34ed5219@xymon.invalid>
To:?xymon <xymon at xymon.com>
Cc:?
Bcc:?
Date:?Mon, 11 Oct 2021 08:16:02 +0000 (UTC)
Subject:?Re: [Xymon] ports check for a VIP...
Purely a sort of FYI I guess...? not sure what was occurring but last Friday early hours the clusetrr had a bit of a tizz and both nodes had no mysqld running and so procs was not green on both nodes...? but the combo check remained green throughout..
On Thursday, 7 October 2021, 09:41:02 BST, Ian Diddams via Xymon <xymon at xymon.com> wrote:
That's great Jeremy - many thanks
Now
(wp\-vldyn\-estrela.ports)||(wp\-vldyn\-rafeiro.ports) = (1)||(1) = 1
wp-vldyn-estrela.ports
wp-vldyn-rafeiro.ports
ian
On Wednesday, 6 October 2021, 05:15:11 BST, Jeremy Laidman <user-0608abae5e7c@xymon.invalid> wrote:
On Mon, 4 Oct 2021 at 23:15, Ian Diddams via Xymon <xymon at xymon.com> wrote:
intrigiingly the actual output on that combo check shows
(wp-vldyn-estrela.ports)||(wp-vldyn-rafeiro.ports) = (0-0-1)||(0-0-1) = 1
?<snip>
i.e. its all showing clear for the individual lines which are also separated where the hostname includes a hyphen...? is that all correct ?
I don't think so. From the manpage of combo.cfg:
"Note: If the expression involves hostnames with a character that is also an operator - e.g. if you have a host "t1-router-newyork.foo.com" with a dash in the hostname - then the operator-character must be escaped with a backslash '\' in the expression, or it will be interpreted as an operator. E.g. like this:
?nyc.conn = (t1\-router\-nyc.conn || backup\-router\-nyc.conn)"
That's why it thinks you have hostnames like "wp" and "vldyn". It's almost certainly not testing what you want it to be testing.
CheersJeremy
---------- Forwarded message ----------
From:?Ian Diddams via Xymon <xymon at xymon.com>
To:?xymon <xymon at xymon.com>
Cc:?
Bcc:?
Date:?Mon, 11 Oct 2021 08:16:02 +0000 (UTC)
Subject:?Re: [Xymon] ports check for a VIP...
list Ian Diddams
before forcing the double red status i tried out the combostatus command to set a baseline etc... but... $? xymoncmd combostatus --debug --no-update 2021-10-18 13:55:07.958800 xymoncmd: execvp of 'combostatus' failed: No such file or directory trying to sort that out first...
list Ian Diddams
sorted.,... combostatus --debug --no-update does the job