xymonfetch
list Galen Johnson
Hey, I need to set up some clients to use xymonfetch and the man page isn't exactly clear. --server=XYMON.SERVER.IP Defines the IP address of the Xymon server where the collected client messages are forwarded to. By default, messages are sent to the loopback address 127.0.0.1, i.e. to a Xymon server running on the same host as xymonfetch. This implies that I need to have a server instance running...or am I misunderstanding what it's saying? Has anyone done this and could provide a some guidance? thx =G=
list Mike Burger
▸
Hey, I need to set up some clients to use xymonfetch and the man page isn't exactly clear. --server=XYMON.SERVER.IP Defines the IP address of the Xymon server where the collected client messages are forwarded to. By default, messages are sent to the loopback address 127.0.0.1, i.e. to a Xymon server running on the same host as xymonfetch. This implies that I need to have a server instance running...or am I misunderstanding what it's saying? Has anyone done this and could provide a some guidance? thx
Simply, what it means is that if you're running xymonfetch from the Xymon server, itself, then the --server variable could be filled in with the localhost ip. If, however, you're running xymonfetch from a Xymon client, elsewhere, then you should fill the in --server variable with the actual IP of the actual Xymon server. -- Mike Burger http://www.bubbanfriends.org "It's always suicide-mission this, save-the-planet that. No one ever just stops by to say 'hi' anymore." --Colonel Jack O'Neill, SG1 On Sunday, September 8, I'll be participating in the Spokes of Hope ride, to raise money for cancer awareness and make a difference in the lives of cancer victims and their families/friends. If you'd care to donate, please click: http://ow.ly/nIdrC Thank you.
list Galen Johnson
So, really, I just need to worry about setting up the clients to use msgcache? =G=
▸
From: Mike Burger [user-cc5c6e80f4c5@xymon.invalid]
Sent: Wednesday, August 21, 2013 11:23 AM
To: Galen Johnson
Cc: xymon at xymon.com
Subject: Re: [Xymon] xymonfetch
Hey, I need to set up some clients to use xymonfetch and the man page isn't exactly clear. --server=XYMON.SERVER.IP Defines the IP address of the Xymon server where the collected client messages are forwarded to. By default, messages are sent to the loopback address 127.0.0.1, i.e. to a Xymon server running on the same host as xymonfetch. This implies that I need to have a server instance running...or am I misunderstanding what it's saying? Has anyone done this and could provide a some guidance? thx
Simply, what it means is that if you're running xymonfetch from the Xymon server, itself, then the --server variable could be filled in with the localhost ip. If, however, you're running xymonfetch from a Xymon client, elsewhere, then you should fill the in --server variable with the actual IP of the actual Xymon server. -- Mike Burger http://www.bubbanfriends.org "It's always suicide-mission this, save-the-planet that. No one ever just stops by to say 'hi' anymore." --Colonel Jack O'Neill, SG1 On Sunday, September 8, I'll be participating in the Spokes of Hope ride, to raise money for cancer awareness and make a difference in the lives of cancer victims and their families/friends. If you'd care to donate, please click: http://ow.ly/nIdrC Thank you.
list Galen Johnson
I must be doing something wrong. On my client, I have XYMSRV="127.0.0.1" and I've commented out the "DISABLE" in clientlaunch.cfg. I'm seeing content in the /tmp/msg.myhost.txt file that looks like what I would expect. On the server, I've added "pulldata" to the entry in etc/hosts.cfg, edited the etc/tasks.cfg to comment out "DISABLE" and set the server IP to my server's IP as well as tried 127.0.0.1. I see the following in my log: 2013-08-21 18:38:06 Whoops ! Failed to send message (Connection failed) 2013-08-21 18:38:06 -> Could not connect to Xymon daemon at 10.40.1.39:1984 (Connection refused) 2013-08-21 18:38:06 -> Recipient '10.40.1.39', timeout 15 2013-08-21 18:38:06 -> 1st line: 'config hosts.cfg' 2013-08-21 18:38:06 Cannot load hosts.cfg from xymond, code 5 2013-08-21 18:38:06 Failed to load from xymond, reverting to file-load 2013-08-21 18:38:06 Connection lost during connect/write to 10.40.1.39:1984 (req 2): Connection refused 2013-08-21 18:48:43 Caught TERM signal, terminating 2013-08-21 18:48:45 Whoops ! Failed to send message (Connection failed) 2013-08-21 18:48:45 -> Could not connect to Xymon daemon at 10.40.1.39:1984 (Connection refused) 2013-08-21 18:48:45 -> Recipient '10.40.1.39', timeout 15 2013-08-21 18:48:45 -> 1st line: 'config hosts.cfg' 2013-08-21 18:48:45 Cannot load hosts.cfg from xymond, code 5 2013-08-21 18:48:45 Failed to load from xymond, reverting to file-load The server is also not updating anything outside my typical network tests. Anyone have any idea what I may be missing here?
▸
=G=
From: Mike Burger [user-cc5c6e80f4c5@xymon.invalid]
Sent: Wednesday, August 21, 2013 11:23 AM
To: Galen Johnson
Cc: xymon at xymon.com
Subject: Re: [Xymon] xymonfetch
Hey, I need to set up some clients to use xymonfetch and the man page isn't exactly clear. --server=XYMON.SERVER.IP Defines the IP address of the Xymon server where the collected client messages are forwarded to. By default, messages are sent to the loopback address 127.0.0.1, i.e. to a Xymon server running on the same host as xymonfetch. This implies that I need to have a server instance running...or am I misunderstanding what it's saying? Has anyone done this and could provide a some guidance? thx
Simply, what it means is that if you're running xymonfetch from the Xymon server, itself, then the --server variable could be filled in with the localhost ip. If, however, you're running xymonfetch from a Xymon client, elsewhere, then you should fill the in --server variable with the actual IP of the actual Xymon server. -- Mike Burger http://www.bubbanfriends.org "It's always suicide-mission this, save-the-planet that. No one ever just stops by to say 'hi' anymore." --Colonel Jack O'Neill, SG1 On Sunday, September 8, I'll be participating in the Spokes of Hope ride, to raise money for cancer awareness and make a difference in the lives of cancer victims and their families/friends. If you'd care to donate, please click: http://ow.ly/nIdrC Thank you.
list Galen Johnson
OK...I got it to stop giving me the errors referenced below by adding "NEEDS xymond" to the xymonfetch block. I've enable debug and it appears to be connecting and returning data but the server page still only shows the network test tabs. Gonna see if letting it sit for a few cycles makes a difference. =G=
▸
From: Galen Johnson
Sent: Wednesday, August 21, 2013 6:52 PM
To: xymon at xymon.com
Cc: Mike Burger
Subject: RE: [Xymon] xymonfetch
I must be doing something wrong.
On my client, I have XYMSRV="127.0.0.1" and I've commented out the "DISABLE" in clientlaunch.cfg. I'm seeing content in the /tmp/msg.myhost.txt file that looks like what I would expect.
On the server, I've added "pulldata" to the entry in etc/hosts.cfg, edited the etc/tasks.cfg to comment out "DISABLE" and set the server IP to my server's IP as well as tried 127.0.0.1. I see the following in my log:
2013-08-21 18:38:06 Whoops ! Failed to send message (Connection failed)
2013-08-21 18:38:06 -> Could not connect to Xymon daemon at 10.40.1.39:1984 (Connection refused)
2013-08-21 18:38:06 -> Recipient '10.40.1.39', timeout 15
2013-08-21 18:38:06 -> 1st line: 'config hosts.cfg'
2013-08-21 18:38:06 Cannot load hosts.cfg from xymond, code 5
2013-08-21 18:38:06 Failed to load from xymond, reverting to file-load
2013-08-21 18:38:06 Connection lost during connect/write to 10.40.1.39:1984 (req 2): Connection refused
2013-08-21 18:48:43 Caught TERM signal, terminating
2013-08-21 18:48:45 Whoops ! Failed to send message (Connection failed)
2013-08-21 18:48:45 -> Could not connect to Xymon daemon at 10.40.1.39:1984 (Connection refused)
2013-08-21 18:48:45 -> Recipient '10.40.1.39', timeout 15
2013-08-21 18:48:45 -> 1st line: 'config hosts.cfg'
2013-08-21 18:48:45 Cannot load hosts.cfg from xymond, code 5
2013-08-21 18:48:45 Failed to load from xymond, reverting to file-load
The server is also not updating anything outside my typical network tests. Anyone have any idea what I may be missing here?
=G=
From: Mike Burger [user-cc5c6e80f4c5@xymon.invalid]
Sent: Wednesday, August 21, 2013 11:23 AM
To: Galen Johnson
Cc: xymon at xymon.com
Subject: Re: [Xymon] xymonfetch
Hey, I need to set up some clients to use xymonfetch and the man page isn't exactly clear. --server=XYMON.SERVER.IP Defines the IP address of the Xymon server where the collected client messages are forwarded to. By default, messages are sent to the loopback address 127.0.0.1, i.e. to a Xymon server running on the same host as xymonfetch. This implies that I need to have a server instance running...or am I misunderstanding what it's saying? Has anyone done this and could provide a some guidance? thx
Simply, what it means is that if you're running xymonfetch from the Xymon server, itself, then the --server variable could be filled in with the localhost ip. If, however, you're running xymonfetch from a Xymon client, elsewhere, then you should fill the in --server variable with the actual IP of the actual Xymon server. -- Mike Burger http://www.bubbanfriends.org "It's always suicide-mission this, save-the-planet that. No one ever just stops by to say 'hi' anymore." --Colonel Jack O'Neill, SG1 On Sunday, September 8, I'll be participating in the Spokes of Hope ride, to raise money for cancer awareness and make a difference in the lives of cancer victims and their families/friends. If you'd care to donate, please click: http://ow.ly/nIdrC Thank you.
list Jeremy Laidman
▸
On 22 August 2013 08:52, Galen Johnson <user-87f955643e3d@xymon.invalid> wrote:
On my client, I have XYMSRV="127.0.0.1" and I've commented out the "DISABLE" in clientlaunch.cfg. I'm seeing content in the /tmp/msg.myhost.txt file that looks like what I would expect.
Good. I'm assuming you commented out "DISABLED" in the [msgcache] section of clientlaunch.cfg and not somewhere else. At that stage, you should be able to connect from the Xymon server to the client on port 1984 (eg with "telnet client.host.name 1984", perhaps sending it a "pullclient" command also). If this works, then the msgcache process is most likely ready for use.
▸
On the server, I've added "pulldata" to the entry in etc/hosts.cfg, editedthe etc/tasks.cfg to comment out "DISABLE"
Again, "DISABLED" in the [xymonfetch] section?
and set the server IP to my server's IP as well as tried 127.0.0.1.
This is the default, so you can leave out the "--server=127.0.0.1" if you want.
I see the following in my log:
Which log?
2013-08-21 18:38:06 Whoops ! Failed to send message (Connection failed) 2013-08-21 18:38:06 -> Could not connect to Xymon daemon at 10.40.1.39:1984(Connection refused)
▸
2013-08-21 18:38:06 -> Recipient '10.40.1.39', timeout 15
2013-08-21 18:38:06 -> 1st line: 'config hosts.cfg'
2013-08-21 18:38:06 Cannot load hosts.cfg from xymond, code 5
2013-08-21 18:38:06 Failed to load from xymond, reverting to file-load
This implies that the logs are on the Xymon server (10.40.1.39?), but the xymond daemon is not running, or at least not on port 1984. Make sure you can do "telnet 10.40.1.39 1984" on the Xymon server.
▸
2013-08-21 18:38:06 Connection lost during connect/write to 10.40.1.39:1984 (req 2): Connection refused
Ah, this message only comes from xymonfetch. So it looks like xymonfetch is unable to send messages into xymond on port 1984. J
list Jeremy Laidman
▸
On 22 August 2013 09:15, Galen Johnson <user-87f955643e3d@xymon.invalid> wrote:
OK...I got it to stop giving me the errors referenced below by adding "NEEDS xymond" to the xymonfetch block.
?? This was on the Xymon server? And xymond wasn't running? You
shouldn't need the "NEEDS xymond" on a regular xymon server.
▸
I've enable debug and it appears to be connecting and returning data but the server page still only shows the network test tabs. Gonna see if letting it sit for a few cycles makes a difference.
It's as if you're running xymond and xymonfetch on the client as well as msgcache. If so, this is not going to get you anything. J
list Galen Johnson
I think it was starting xymonfetch before xymond was ready. I did get it working. I found the host on the ghost report and added the short name to CLIENT:. Sent from my iPhone
▸
On Aug 21, 2013, at 10:08 PM, "Jeremy Laidman" <user-71895fb2e44c@xymon.invalid<mailto:user-71895fb2e44c@xymon.invalid>> wrote:
On 22 August 2013 09:15, Galen Johnson <user-87f955643e3d@xymon.invalid<mailto:user-87f955643e3d@xymon.invalid>> wrote:
OK...I got it to stop giving me the errors referenced below by adding "NEEDS xymond" to the xymonfetch block.
?? This was on the Xymon server? And xymond wasn't running? You shouldn't need the "NEEDS xymond" on a regular xymon server.
I've enable debug and it appears to be connecting and returning data but the server page still only shows the network test tabs. Gonna see if letting it sit for a few cycles makes a difference.
It's as if you're running xymond and xymonfetch on the client as well as msgcache. If so, this is not going to get you anything.
J
list Paul Root
I'm having issues with xymonfetch.
I have 2 xymon proxies, that also have the xymon server running as a backup to the main server on port 1985. The proxy sends to the main server on port 1984 and to itself on port 1985.
I'm trying to add 2 new machines as clients of these proxy servers. They are outside a firewall that I don't want to open, so I'm doing a fetch. Xymon-client is running on 1984, and msgcache is running.
Client clientlaunch.cfg:
[msgcache]
# DISABLED
ENVFILE $XYMONCLIENTHOME/etc/xymonclient.cfg
CMD $XYMONCLIENTHOME/bin/msgcache --no-daemon --pidfile=$XYMONCLIENTLOGS/msgcache.pid
LOGFILE $XYMONCLIENTLOGS/msgcache.log
Proxy tasks.cfg:
[xymonfetch]
ENVFILE /etc/xymon/xymonserver.cfg
#CMD $XYMONHOME/bin/xymonfetch --server=YOUR.XYMON.SERVER.IP --no-daemon --pidfile=$XYMONSERVERLOGS/xymonfetch.pid
CMD $XYMONHOME/bin/xymonfetch --server=127.0.0.1:1985 --log-interval=60 --id=1 --pidfile=$XYMONSERVERLOGS/xymonfetch.pid
LOGFILE $XYMONSERVERLOGS/xymonfetch.log
Proxy hosts.cfg:
10.6.0.14 remoteclient # pulldata=10.6.0.14:1984 http://198.36.155.243
10.6.0.15 remoteclient2 # pulldata=10.6.0.15:1984
I have a third xymon server, the development server, which is not a proxy, and the server runs on 1984. This pulls the data just fine from the remoteclients.
The xymonfetch.log file gets these errors:
2015-11-11 15:56:24 Connection lost during connect/write to 10.6.0.14:1985 (req 27): Connection refused
2015-11-11 15:56:41 Connection lost during connect/write to 10.6.0.15:1985 (req 28): Connection refused
2015-11-11 15:57:27 Connection lost during connect/write to 10.6.0.14:1985 (req 29): Connection refused
2015-11-11 15:57:44 Connection lost during connect/write to 10.6.0.15:1985 (req 30): Connection refused
The proxy servers are 4.3.10. The development server is 4.3.21. The main server is also 4.3.10, and it runs a fetch without issue, to other clients.
The clients are 4.3.10 (CentOS 5) and 4.3.21 (CentOS 6).
I am working on upgrading the proxy servers in OS (CentOS 5 -> 6) and Xymon (to 4.3.21). But not quite there yet. Another subject, but sending data from a proxy to another proxy does not work. So, when I have everything else, I may end up putting it in place, and hoping the proxy works....
Any ideas here? It seems like such a simple thing.
Paul Root
Lead Engineer
CenturyLink Network Reliability Operations Center
390 Commerce Dr
Woodbury, MN 55125
Direct: (651)312-5207
user-76fdb6883669@xymon.invalid
This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
list Japheth Cleaver
▸
On 11/11/2015 2:04 PM, Root, Paul T wrote:
I’m having issues with xymonfetch. I have 2 xymon proxies, that also have the xymon server running as a backup to the main server on port 1985. The proxy sends to the main server on port 1984 and to itself on port 1985. I’m trying to add 2 new machines as clients of these proxy servers. They are outside a firewall that I don’t want to open, so I’m doing a fetch. Xymon-client is running on 1984, and msgcache is running. Client clientlaunch.cfg: *[msgcache]* *# DISABLED* * ENVFILE $XYMONCLIENTHOME/etc/xymonclient.cfg* * CMD $XYMONCLIENTHOME/bin/msgcache --no-daemon --pidfile=$XYMONCLIENTLOGS/msgcache.pid* * LOGFILE $XYMONCLIENTLOGS/msgcache.log* ** ** Proxy tasks.cfg: *[xymonfetch]* * ENVFILE /etc/xymon/xymonserver.cfg* * #CMD $XYMONHOME/bin/xymonfetch --server=YOUR.XYMON.SERVER.IP --no-daemon --pidfile=$XYMONSERVERLOGS/xymonfetch.pid* * CMD $XYMONHOME/bin/xymonfetch --server=127.0.0.1:1985 --log-interval=60 --id=1 --pidfile=$XYMONSERVERLOGS/xymonfetch.pid* * LOGFILE $XYMONSERVERLOGS/xymonfetch.log* ** Proxy hosts.cfg: *10.6.0.14 remoteclient # pulldata=10.6.0.14:1984 http://198.36.155.243* *10.6.0.15 remoteclient2 # pulldata=10.6.0.15:1984 * ** I have a third xymon server, the development server, which is not a proxy, and the server runs on 1984. This pulls the data just fine from the remoteclients. ** ** The xymonfetch.log file gets these errors: *2015-11-11 15:56:24 Connection lost during connect/write to 10.6.0.14:1985 (req 27): Connection refused* *2015-11-11 15:56:41 Connection lost during connect/write to 10.6.0.15:1985 (req 28): Connection refused* *2015-11-11 15:57:27 Connection lost during connect/write to 10.6.0.14:1985 (req 29): Connection refused* *2015-11-11 15:57:44 Connection lost during connect/write to 10.6.0.15:1985 (req 30): Connection refused* ** ** *The proxy servers are 4.3.10. The development server is 4.3.21. The main server is also 4.3.10, and it runs a fetch without issue, to other clients.* ** *The clients are 4.3.10 (CentOS 5) and 4.3.21 (CentOS 6). * ** *I am working on upgrading the proxy servers in OS (CentOS 5 -> 6) and Xymon (to 4.3.21). But not quite there yet. Another subject, but sending data from a proxy to another proxy does not work. So, when I have everything else, I may end up putting it in place, and hoping the proxy works….* ** ** *Any ideas here? It seems like such a simple thing. * **
Does seem a bit odd. Either xymonfetch is not paying attention to pulldata, or it seems to want to reconnect back to the original host incorrectly for the client (config) reply. Can you try --debug on the xymonfetch process, or alternatively strace it, so we can see at which phase of the transaction that write is failing at? On the second issue, I know xymonproxy is capable of doing proxy-proxy submission, as we used it pretty heavily for that (albeit with caveats at scale when dealing with TCP packet loss), does a 'ping' make it through at all when going on that path? -jc
list Paul Root
It gets even weirder. If I run it without xymoncmd, then it uses port 1984 (the xymonproxy) but then is trying to pull data from the machine the main server pulls from, and doesn’t even exist in the proxy’s hosts.cfg. But when I run with xymoncmd, I get this. To me, it appears to be completely ignoring the command line: $ xymonfetch --server=:1984 --log-interval=60 --id=1 --debug 28843 2015-11-12 08:56:09 Transport setup is: 28843 2015-11-12 08:56:09 xymondportnumber = 1985 28843 2015-11-12 08:56:09 xymonproxyhost = NONE 28843 2015-11-12 08:56:09 xymonproxyport = 0 28843 2015-11-12 08:56:09 Recipient listed as '204.155.140.159' 28843 2015-11-12 08:56:09 Standard protocol on port 1985 28843 2015-11-12 08:56:09 Will connect to address 204.155.140.159 port 1985 28843 2015-11-12 08:56:09 Connect status is 0 28843 2015-11-12 08:56:09 Sent 16 bytes 28843 2015-11-12 08:56:09 Read 14206 bytes 28843 2015-11-12 08:56:09 Closing connection 28843 2015-11-12 08:56:09 Queuing request 1 to 10.6.0.15:1985 for remoteclient2: 'pullclient 1 ' 28843 2015-11-12 08:56:09 Queuing request 2 to 10.6.0.14:1985 for remoteclient: 'pullclient 1 ' 2015-11-12 08:56:09 Connection lost during connect/write to 10.6.0.14:1985 (req 2): Connection refused 28843 2015-11-12 08:56:09 Doing cleanup 28843 2015-11-12 08:56:09 Next poll of remoteclient in 55 seconds 28843 2015-11-12 08:56:09 Request completed: req 2, peer 10.6.0.14:1985, action was 2, type was 0 2015-11-12 08:56:09 Connection lost during connect/write to 10.6.0.15:1985 (req 1): Connection refused 28843 2015-11-12 08:56:09 Doing cleanup 28843 2015-11-12 08:56:09 Next poll of remoteclient2 in 48 seconds 28843 2015-11-12 08:56:09 Request completed: req 1, peer 10.6.0.14:1985, action was 2, type was 0 28843 2015-11-12 08:56:57 Queuing request 3 to 10.6.0.15:1985 for remoteclient2: 'pullclient 1 ' 2015-11-12 08:56:57 Connection lost during connect/write to 10.6.0.15:1985 (req 3): Connection refused 28843 2015-11-12 08:56:57 Doing cleanup 28843 2015-11-12 08:56:57 Next poll of remoteclient2 in 52 seconds 28843 2015-11-12 08:56:57 Request completed: req 3, peer 10.6.0.15:1985, action was 2, type was 0 It makes no difference if I put in –server=127.0.0.1:1984 or –server=204.155.140.159:1984.
▸
From: Japheth Cleaver [mailto:user-87556346d4af@xymon.invalid] Sent: Wednesday, November 11, 2015 6:13 PM To: Root, Paul T; xymon at xymon.com Subject: Re: [Xymon] xymonfetch On 11/11/2015 2:04 PM, Root, Paul T wrote: I’m having issues with xymonfetch. I have 2 xymon proxies, that also have the xymon server running as a backup to the main server on port 1985. The proxy sends to the main server on port 1984 and to itself on port 1985. I’m trying to add 2 new machines as clients of these proxy servers. They are outside a firewall that I don’t want to open, so I’m doing a fetch. Xymon-client is running on 1984, and msgcache is running. Client clientlaunch.cfg: [msgcache] # DISABLED ENVFILE $XYMONCLIENTHOME/etc/xymonclient.cfg CMD $XYMONCLIENTHOME/bin/msgcache --no-daemon --pidfile=$XYMONCLIENTLOGS/msgcache.pid LOGFILE $XYMONCLIENTLOGS/msgcache.log Proxy tasks.cfg: [xymonfetch] ENVFILE /etc/xymon/xymonserver.cfg #CMD $XYMONHOME/bin/xymonfetch --server=YOUR.XYMON.SERVER.IP --no-daemon --pidfile=$XYMONSERVERLOGS/xymonfetch.pid CMD $XYMONHOME/bin/xymonfetch --server=127.0.0.1:1985 --log-interval=60 --id=1 --pidfile=$XYMONSERVERLOGS/xymonfetch.pid LOGFILE $XYMONSERVERLOGS/xymonfetch.log Proxy hosts.cfg: 10.6.0.14 remoteclient # pulldata=10.6.0.14:1984 http://198.36.155.243 10.6.0.15 remoteclient2 # pulldata=10.6.0.15:1984 I have a third xymon server, the development server, which is not a proxy, and the server runs on 1984. This pulls the data just fine from the remoteclients. The xymonfetch.log file gets these errors: 2015-11-11 15:56:24 Connection lost during connect/write to 10.6.0.14:1985 (req 27): Connection refused 2015-11-11 15:56:41 Connection lost during connect/write to 10.6.0.15:1985 (req 28): Connection refused 2015-11-11 15:57:27 Connection lost during connect/write to 10.6.0.14:1985 (req 29): Connection refused 2015-11-11 15:57:44 Connection lost during connect/write to 10.6.0.15:1985 (req 30): Connection refused The proxy servers are 4.3.10. The development server is 4.3.21. The main server is also 4.3.10, and it runs a fetch without issue, to other clients. The clients are 4.3.10 (CentOS 5) and 4.3.21 (CentOS 6). I am working on upgrading the proxy servers in OS (CentOS 5 -> 6) and Xymon (to 4.3.21). But not quite there yet. Another subject, but sending data from a proxy to another proxy does not work. So, when I have everything else, I may end up putting it in place, and hoping the proxy works…. Any ideas here? It seems like such a simple thing. Does seem a bit odd. Either xymonfetch is not paying attention to pulldata, or it seems to want to reconnect back to the original host incorrectly for the client (config) reply. Can you try --debug on the xymonfetch process, or alternatively strace it, so we can see at which phase of the transaction that write is failing at? On the second issue, I know xymonproxy is capable of doing proxy-proxy submission, as we used it pretty heavily for that (albeit with caveats at scale when dealing with TCP packet loss), does a 'ping' make it through at all when going on that path? -jc This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
list Paul Root
The saga continues.
I got my network people to open up 1985 through the firewall to my machines, and then restarted xymon-client on those machines to use 1985 with msgcache. So while that is wrong, at least I have a work around. Also, the fetch=ip:1984 was ignored, it only would talk on port 1985.
Then the xymonfetch on the proxy servers would connect and get data from them.
HOWEVER, the proxy servers will not send the data to the proxy port (1984), only the server port (1985). Which doesn’t get me data to my primary server.
Is it possible to run xymonfetch from a client machine? That client would send to the xymon proxy server.
If I get time today (or maybe over the weekend), I’m going to try to put my new xymon proxy server in place (CentOS6 with 4.3.21). Again the current servers are CentOS 5 with 4.3.10.
[xymonfetch]
ENVFILE /etc/xymon/xymonserver.cfg
#CMD $XYMONHOME/bin/xymonfetch --server=127.0.0.1:1984 --id=1 --pidfile=$XYMONSERVERLOGS/xymonfetch.pid # does not work
CMD $XYMONHOME/bin/xymonfetch --server=204.155.140.159:1984 --id=1 --pidfile=$XYMONSERVERLOGS/xymonfetch.pid # does not work
#CMD $XYMONHOME/bin/xymonfetch id=1 --pidfile=$XYMONSERVERLOGS/xymonfetch.pid # works to server not proxy
LOGFILE $XYMONSERVERLOGS/xymonfetch.log
Xymonfetch.log:
2015-11-13 08:00:19 Invalid client IP: 127.0.0.1:1984 (req 2435)
2015-11-13 08:03:36 Invalid client IP: 127.0.0.1:1984 (req 2443)
2015-11-13 08:15:22 Invalid client IP: 127.0.0.1:1984 (req 2473)
2015-11-13 08:18:46 Invalid client IP: 127.0.0.1:1984 (req 2481)
2015-11-13 08:27:52 Caught TERM signal, terminating
2015-11-13 08:27:52 Caught TERM signal, terminating
2015-11-13 08:27:52 Caught TERM signal, terminating
2015-11-13 08:29:24 Invalid client IP: 204.155.140.159:1984 (req 6)
2015-11-13 08:29:24 Invalid client IP: 204.155.140.159:1984 (req 6)
2015-11-13 08:29:24 Invalid client IP: 204.155.140.159:1984 (req 6)
2015-11-13 08:30:45 Invalid client IP: 204.155.140.159:1984 (req 10)
2015-11-13 08:30:45 Invalid client IP: 204.155.140.159:1984 (req 10)
2015-11-13 08:30:45 Invalid client IP: 204.155.140.159:1984 (req 10)
▸
From: Xymon [mailto:xymon-bounces at xymon.com] On Behalf Of Root, Paul T Sent: Thursday, November 12, 2015 9:14 AM To: 'Japheth Cleaver'; 'xymon at xymon.com' Subject: Re: [Xymon] xymonfetch It gets even weirder. If I run it without xymoncmd, then it uses port 1984 (the xymonproxy) but then is trying to pull data from the machine the main server pulls from, and doesn’t even exist in the proxy’s hosts.cfg. But when I run with xymoncmd, I get this. To me, it appears to be completely ignoring the command line: $ xymonfetch --server=:1984 --log-interval=60 --id=1 --debug 28843 2015-11-12 08:56:09 Transport setup is: 28843 2015-11-12 08:56:09 xymondportnumber = 1985 28843 2015-11-12 08:56:09 xymonproxyhost = NONE 28843 2015-11-12 08:56:09 xymonproxyport = 0 28843 2015-11-12 08:56:09 Recipient listed as '204.155.140.159' 28843 2015-11-12 08:56:09 Standard protocol on port 1985 28843 2015-11-12 08:56:09 Will connect to address 204.155.140.159 port 1985 28843 2015-11-12 08:56:09 Connect status is 0 28843 2015-11-12 08:56:09 Sent 16 bytes 28843 2015-11-12 08:56:09 Read 14206 bytes 28843 2015-11-12 08:56:09 Closing connection 28843 2015-11-12 08:56:09 Queuing request 1 to 10.6.0.15:1985 for remoteclient2: 'pullclient 1 ' 28843 2015-11-12 08:56:09 Queuing request 2 to 10.6.0.14:1985 for remoteclient: 'pullclient 1 ' 2015-11-12 08:56:09 Connection lost during connect/write to 10.6.0.14:1985 (req 2): Connection refused 28843 2015-11-12 08:56:09 Doing cleanup 28843 2015-11-12 08:56:09 Next poll of remoteclient in 55 seconds 28843 2015-11-12 08:56:09 Request completed: req 2, peer 10.6.0.14:1985, action was 2, type was 0 2015-11-12 08:56:09 Connection lost during connect/write to 10.6.0.15:1985 (req 1): Connection refused 28843 2015-11-12 08:56:09 Doing cleanup 28843 2015-11-12 08:56:09 Next poll of remoteclient2 in 48 seconds 28843 2015-11-12 08:56:09 Request completed: req 1, peer 10.6.0.14:1985, action was 2, type was 0 28843 2015-11-12 08:56:57 Queuing request 3 to 10.6.0.15:1985 for remoteclient2: 'pullclient 1 ' 2015-11-12 08:56:57 Connection lost during connect/write to 10.6.0.15:1985 (req 3): Connection refused 28843 2015-11-12 08:56:57 Doing cleanup 28843 2015-11-12 08:56:57 Next poll of remoteclient2 in 52 seconds 28843 2015-11-12 08:56:57 Request completed: req 3, peer 10.6.0.15:1985, action was 2, type was 0 It makes no difference if I put in –server=127.0.0.1:1984 or –server=204.155.140.159:1984. From: Japheth Cleaver [mailto:user-87556346d4af@xymon.invalid] Sent: Wednesday, November 11, 2015 6:13 PM To: Root, Paul T; xymon at xymon.com<mailto:xymon at xymon.com> Subject: Re: [Xymon] xymonfetch On 11/11/2015 2:04 PM, Root, Paul T wrote: I’m having issues with xymonfetch. I have 2 xymon proxies, that also have the xymon server running as a backup to the main server on port 1985. The proxy sends to the main server on port 1984 and to itself on port 1985. I’m trying to add 2 new machines as clients of these proxy servers. They are outside a firewall that I don’t want to open, so I’m doing a fetch. Xymon-client is running on 1984, and msgcache is running. Client clientlaunch.cfg: [msgcache] # DISABLED ENVFILE $XYMONCLIENTHOME/etc/xymonclient.cfg CMD $XYMONCLIENTHOME/bin/msgcache --no-daemon --pidfile=$XYMONCLIENTLOGS/msgcache.pid LOGFILE $XYMONCLIENTLOGS/msgcache.log Proxy tasks.cfg: [xymonfetch] ENVFILE /etc/xymon/xymonserver.cfg #CMD $XYMONHOME/bin/xymonfetch --server=YOUR.XYMON.SERVER.IP --no-daemon --pidfile=$XYMONSERVERLOGS/xymonfetch.pid CMD $XYMONHOME/bin/xymonfetch --server=127.0.0.1:1985 --log-interval=60 --id=1 --pidfile=$XYMONSERVERLOGS/xymonfetch.pid LOGFILE $XYMONSERVERLOGS/xymonfetch.log Proxy hosts.cfg: 10.6.0.14 remoteclient # pulldata=10.6.0.14:1984 http://198.36.155.243 10.6.0.15 remoteclient2 # pulldata=10.6.0.15:1984 I have a third xymon server, the development server, which is not a proxy, and the server runs on 1984. This pulls the data just fine from the remoteclients. The xymonfetch.log file gets these errors: 2015-11-11 15:56:24 Connection lost during connect/write to 10.6.0.14:1985 (req 27): Connection refused 2015-11-11 15:56:41 Connection lost during connect/write to 10.6.0.15:1985 (req 28): Connection refused 2015-11-11 15:57:27 Connection lost during connect/write to 10.6.0.14:1985 (req 29): Connection refused 2015-11-11 15:57:44 Connection lost during connect/write to 10.6.0.15:1985 (req 30): Connection refused The proxy servers are 4.3.10. The development server is 4.3.21. The main server is also 4.3.10, and it runs a fetch without issue, to other clients. The clients are 4.3.10 (CentOS 5) and 4.3.21 (CentOS 6). I am working on upgrading the proxy servers in OS (CentOS 5 -> 6) and Xymon (to 4.3.21). But not quite there yet. Another subject, but sending data from a proxy to another proxy does not work. So, when I have everything else, I may end up putting it in place, and hoping the proxy works…. Any ideas here? It seems like such a simple thing. Does seem a bit odd. Either xymonfetch is not paying attention to pulldata, or it seems to want to reconnect back to the original host incorrectly for the client (config) reply. Can you try --debug on the xymonfetch process, or alternatively strace it, so we can see at which phase of the transaction that write is failing at? On the second issue, I know xymonproxy is capable of doing proxy-proxy submission, as we used it pretty heavily for that (albeit with caveats at scale when dealing with TCP packet loss), does a 'ping' make it through at all when going on that path? -jc This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments. This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
list Japheth Cleaver
Hmm. It's looking like we have a fair amount of crufty code in xymonfetch that might need to be brought up to spec, that might be causing some of the confusion. First, the syntax will need to be just --server=<destinationIP>. The port is hard-coded to use whatever $XYMONDPORT is (from xymonserver.cfg if via xymonlaunch/xymoncmd, but can be altered on the command line with 'env'). Secondly (possible typo), it'll need to be --id=# on the CMD line, not just id=#. I have to admit to not using xymonfetch or msgcache much myself, but xymonproxy itself should be able to handle the pass-through of two-way messages okay for the most part. I think. Can re-run with just the bare IP? I suspect it wasn't trying to connect with what it looked like it was trying to. HTH, -jc
▸
On Fri, November 13, 2015 6:59 am, Root, Paul T wrote:The saga continues. I got my network people to open up 1985 through the firewall to my machines, and then restarted xymon-client on those machines to use 1985 with msgcache. So while that is wrong, at least I have a work around. Also, the fetch=ip:1984 was ignored, it only would talk on port 1985. Then the xymonfetch on the proxy servers would connect and get data from them. HOWEVER, the proxy servers will not send the data to the proxy port
(1984), only the server port (1985). Which doesnât get me data to my
▸
primary server.
Is it possible to run xymonfetch from a client machine? That client would
send to the xymon proxy server.
If I get time today (or maybe over the weekend), Iâm going to try to put
▸
my new xymon proxy server in place (CentOS6 with 4.3.21). Again the
current servers are CentOS 5 with 4.3.10.
[xymonfetch]
ENVFILE /etc/xymon/xymonserver.cfg
#CMD $XYMONHOME/bin/xymonfetch --server=127.0.0.1:1984 --id=1
--pidfile=$XYMONSERVERLOGS/xymonfetch.pid
# does not work
CMD $XYMONHOME/bin/xymonfetch --server=204.155.140.159:1984 --id=1
--pidfile=$XYMONSERVERLOGS/xymonfetch.pid # does not
work
#CMD $XYMONHOME/bin/xymonfetch id=1
--pidfile=$XYMONSERVERLOGS/xymonfetch.pid
#
works to server not proxy
LOGFILE $XYMONSERVERLOGS/xymonfetch.log
Xymonfetch.log:
2015-11-13 08:00:19 Invalid client IP: 127.0.0.1:1984 (req 2435)
2015-11-13 08:03:36 Invalid client IP: 127.0.0.1:1984 (req 2443)
2015-11-13 08:15:22 Invalid client IP: 127.0.0.1:1984 (req 2473)
2015-11-13 08:18:46 Invalid client IP: 127.0.0.1:1984 (req 2481)
2015-11-13 08:27:52 Caught TERM signal, terminating
2015-11-13 08:27:52 Caught TERM signal, terminating
2015-11-13 08:27:52 Caught TERM signal, terminating
2015-11-13 08:29:24 Invalid client IP: 204.155.140.159:1984 (req 6)
2015-11-13 08:29:24 Invalid client IP: 204.155.140.159:1984 (req 6)
2015-11-13 08:29:24 Invalid client IP: 204.155.140.159:1984 (req 6)
2015-11-13 08:30:45 Invalid client IP: 204.155.140.159:1984 (req 10)
2015-11-13 08:30:45 Invalid client IP: 204.155.140.159:1984 (req 10)
2015-11-13 08:30:45 Invalid client IP: 204.155.140.159:1984 (req 10)
From: Xymon [mailto:xymon-bounces at xymon.com] On Behalf Of Root, Paul T
Sent: Thursday, November 12, 2015 9:14 AM
To: 'Japheth Cleaver'; 'xymon at xymon.com'
Subject: Re: [Xymon] xymonfetch
It gets even weirder. If I run it without xymoncmd, then it uses port 1984
(the xymonproxy) but then is trying to pull data from the machine the main
server pulls from, and doesnât even exist in the proxyâs hosts.cfg.
▸
But when I run with xymoncmd, I get this. To me, it appears to be
completely ignoring the command line:
$ xymonfetch --server=:1984 --log-interval=60 --id=1 --debug
28843 2015-11-12 08:56:09 Transport setup is:
28843 2015-11-12 08:56:09 xymondportnumber = 1985
28843 2015-11-12 08:56:09 xymonproxyhost = NONE
28843 2015-11-12 08:56:09 xymonproxyport = 0
28843 2015-11-12 08:56:09 Recipient listed as '204.155.140.159'
28843 2015-11-12 08:56:09 Standard protocol on port 1985
28843 2015-11-12 08:56:09 Will connect to address 204.155.140.159 port
1985
28843 2015-11-12 08:56:09 Connect status is 0
28843 2015-11-12 08:56:09 Sent 16 bytes
28843 2015-11-12 08:56:09 Read 14206 bytes
28843 2015-11-12 08:56:09 Closing connection
28843 2015-11-12 08:56:09 Queuing request 1 to 10.6.0.15:1985 for
remoteclient2: 'pullclient 1
'
28843 2015-11-12 08:56:09 Queuing request 2 to 10.6.0.14:1985 for
remoteclient: 'pullclient 1
'
2015-11-12 08:56:09 Connection lost during connect/write to 10.6.0.14:1985
(req 2): Connection refused
28843 2015-11-12 08:56:09 Doing cleanup
28843 2015-11-12 08:56:09 Next poll of remoteclient in 55 seconds
28843 2015-11-12 08:56:09 Request completed: req 2, peer 10.6.0.14:1985,
action was 2, type was 0
2015-11-12 08:56:09 Connection lost during connect/write to 10.6.0.15:1985
(req 1): Connection refused
28843 2015-11-12 08:56:09 Doing cleanup
28843 2015-11-12 08:56:09 Next poll of remoteclient2 in 48 seconds
28843 2015-11-12 08:56:09 Request completed: req 1, peer 10.6.0.14:1985,
action was 2, type was 0
28843 2015-11-12 08:56:57 Queuing request 3 to 10.6.0.15:1985 for
remoteclient2: 'pullclient 1
'
2015-11-12 08:56:57 Connection lost during connect/write to 10.6.0.15:1985
(req 3): Connection refused
28843 2015-11-12 08:56:57 Doing cleanup
28843 2015-11-12 08:56:57 Next poll of remoteclient2 in 52 seconds
28843 2015-11-12 08:56:57 Request completed: req 3, peer 10.6.0.15:1985,
action was 2, type was 0
It makes no difference if I put in âserver=127.0.0.1:1984 or
âserver=204.155.140.159:1984.
▸
From: Japheth Cleaver [mailto:user-87556346d4af@xymon.invalid]
Sent: Wednesday, November 11, 2015 6:13 PM
To: Root, Paul T; xymon at xymon.com<mailto:xymon at xymon.com>
Subject: Re: [Xymon] xymonfetch
On 11/11/2015 2:04 PM, Root, Paul T wrote:
Iâm having issues with xymonfetch.
▸
I have 2 xymon proxies, that also have the xymon server running as a
backup to the main server on port 1985. The proxy sends to the main
server on port 1984 and to itself on port 1985.
Iâm trying to add 2 new machines as clients of these proxy servers. They
are outside a firewall that I donât want to open, so Iâm doing a
▸
fetch. Xymon-client is running on 1984, and msgcache is running. Client clientlaunch.cfg: [msgcache] # DISABLED ENVFILE $XYMONCLIENTHOME/etc/xymonclient.cfg CMD $XYMONCLIENTHOME/bin/msgcache --no-daemon --pidfile=$XYMONCLIENTLOGS/msgcache.pid LOGFILE $XYMONCLIENTLOGS/msgcache.log Proxy tasks.cfg: [xymonfetch] ENVFILE /etc/xymon/xymonserver.cfg #CMD $XYMONHOME/bin/xymonfetch --server=YOUR.XYMON.SERVER.IP --no-daemon --pidfile=$XYMONSERVERLOGS/xymonfetch.pid CMD $XYMONHOME/bin/xymonfetch --server=127.0.0.1:1985 --log-interval=60 --id=1 --pidfile=$XYMONSERVERLOGS/xymonfetch.pid LOGFILE $XYMONSERVERLOGS/xymonfetch.log Proxy hosts.cfg: 10.6.0.14 remoteclient # pulldata=10.6.0.14:1984 http://198.36.155.243 10.6.0.15 remoteclient2 # pulldata=10.6.0.15:1984 I have a third xymon server, the development server, which is not a proxy, and the server runs on 1984. This pulls the data just fine from the remoteclients. The xymonfetch.log file gets these errors: 2015-11-11 15:56:24 Connection lost during connect/write to 10.6.0.14:1985 (req 27): Connection refused 2015-11-11 15:56:41 Connection lost during connect/write to 10.6.0.15:1985 (req 28): Connection refused 2015-11-11 15:57:27 Connection lost during connect/write to 10.6.0.14:1985 (req 29): Connection refused 2015-11-11 15:57:44 Connection lost during connect/write to 10.6.0.15:1985 (req 30): Connection refused The proxy servers are 4.3.10. The development server is 4.3.21. The main server is also 4.3.10, and it runs a fetch without issue, to other clients. The clients are 4.3.10 (CentOS 5) and 4.3.21 (CentOS 6). I am working on upgrading the proxy servers in OS (CentOS 5 -> 6) and Xymon (to 4.3.21). But not quite there yet. Another subject, but sending data from a proxy to another proxy does not work. So, when I have everything else, I may end up putting it in place, and hoping the proxy
worksâ¦.
▸
Any ideas here? It seems like such a simple thing.
Does seem a bit odd. Either xymonfetch is not paying attention to
pulldata, or it seems to want to reconnect back to the original host
incorrectly for the client (config) reply. Can you try --debug on the
xymonfetch process, or alternatively strace it, so we can see at which
phase of the transaction that write is failing at?
On the second issue, I know xymonproxy is capable of doing proxy-proxy
submission, as we used it pretty heavily for that (albeit with caveats at
scale when dealing with TCP packet loss), does a 'ping' make it through at
all when going on that path?
-jc
This communication is the property of CenturyLink and may contain
confidential or privileged information. Unauthorized use of this
communication is strictly prohibited and may be unlawful. If you have
received this communication in error, please immediately notify the sender
by reply e-mail and destroy all copies of the communication and any
attachments.
This communication is the property of CenturyLink and may contain
confidential or privileged information. Unauthorized use of this
communication is strictly prohibited and may be unlawful. If you have
received this communication in error, please immediately notify the sender
by reply e-mail and destroy all copies of the communication and any
attachments.
list Paul Root
Thanks, I can confirm it ignores the :port. I will try the environment variable trick. That sounds good. Thanks for catching that typo on id.
▸
-----Original Message-----
From: J.C. Cleaver [mailto:user-87556346d4af@xymon.invalid]
Sent: Friday, November 13, 2015 12:37 PM
To: Root, Paul T
Cc: Xymon Mailing List
Subject: RE: [Xymon] xymonfetch
Hmm. It's looking like we have a fair amount of crufty code in xymonfetch
that might need to be brought up to spec, that might be causing some of
the confusion.
First, the syntax will need to be just --server=<destinationIP>. The port
is hard-coded to use whatever $XYMONDPORT is (from xymonserver.cfg if via
xymonlaunch/xymoncmd, but can be altered on the command line with 'env').
Secondly (possible typo), it'll need to be --id=# on the CMD line, not
just id=#.
I have to admit to not using xymonfetch or msgcache much myself, but
xymonproxy itself should be able to handle the pass-through of two-way
messages okay for the most part. I think.
Can re-run with just the bare IP? I suspect it wasn't trying to connect
with what it looked like it was trying to.
HTH,
-jc
On Fri, November 13, 2015 6:59 am, Root, Paul T wrote:The saga continues.
I got my network people to open up 1985 through the firewall to my
machines, and then restarted xymon-client on those machines to use 1985
with msgcache. So while that is wrong, at least I have a work around.
Also, the fetch=ip:1984 was ignored, it only would talk on port 1985.
Then the xymonfetch on the proxy servers would connect and get data from
them.
HOWEVER, the proxy servers will not send the data to the proxy port
(1984), only the server port (1985). Which doesn’t get me data to my
primary server.
Is it possible to run xymonfetch from a client machine? That client would
send to the xymon proxy server.
If I get time today (or maybe over the weekend), I’m going to try to put
my new xymon proxy server in place (CentOS6 with 4.3.21). Again the
current servers are CentOS 5 with 4.3.10.
[xymonfetch]
ENVFILE /etc/xymon/xymonserver.cfg
#CMD $XYMONHOME/bin/xymonfetch --server=127.0.0.1:1984 --id=1
--pidfile=$XYMONSERVERLOGS/xymonfetch.pid
# does not work
CMD $XYMONHOME/bin/xymonfetch --server=204.155.140.159:1984 --id=1
--pidfile=$XYMONSERVERLOGS/xymonfetch.pid # does not
work
#CMD $XYMONHOME/bin/xymonfetch id=1
--pidfile=$XYMONSERVERLOGS/xymonfetch.pid
#
works to server not proxy
LOGFILE $XYMONSERVERLOGS/xymonfetch.log
Xymonfetch.log:
2015-11-13 08:00:19 Invalid client IP: 127.0.0.1:1984 (req 2435)
2015-11-13 08:03:36 Invalid client IP: 127.0.0.1:1984 (req 2443)
2015-11-13 08:15:22 Invalid client IP: 127.0.0.1:1984 (req 2473)
2015-11-13 08:18:46 Invalid client IP: 127.0.0.1:1984 (req 2481)
2015-11-13 08:27:52 Caught TERM signal, terminating
2015-11-13 08:27:52 Caught TERM signal, terminating
2015-11-13 08:27:52 Caught TERM signal, terminating
2015-11-13 08:29:24 Invalid client IP: 204.155.140.159:1984 (req 6)
2015-11-13 08:29:24 Invalid client IP: 204.155.140.159:1984 (req 6)
2015-11-13 08:29:24 Invalid client IP: 204.155.140.159:1984 (req 6)
2015-11-13 08:30:45 Invalid client IP: 204.155.140.159:1984 (req 10)
2015-11-13 08:30:45 Invalid client IP: 204.155.140.159:1984 (req 10)
2015-11-13 08:30:45 Invalid client IP: 204.155.140.159:1984 (req 10)
From: Xymon [mailto:xymon-bounces at xymon.com] On Behalf Of Root, Paul T
Sent: Thursday, November 12, 2015 9:14 AM
To: 'Japheth Cleaver'; 'xymon at xymon.com'
Subject: Re: [Xymon] xymonfetch
It gets even weirder. If I run it without xymoncmd, then it uses port 1984
(the xymonproxy) but then is trying to pull data from the machine the main
server pulls from, and doesn’t even exist in the proxy’s hosts.cfg.
But when I run with xymoncmd, I get this. To me, it appears to be
completely ignoring the command line:
$ xymonfetch --server=:1984 --log-interval=60 --id=1 --debug
28843 2015-11-12 08:56:09 Transport setup is:
28843 2015-11-12 08:56:09 xymondportnumber = 1985
28843 2015-11-12 08:56:09 xymonproxyhost = NONE
28843 2015-11-12 08:56:09 xymonproxyport = 0
28843 2015-11-12 08:56:09 Recipient listed as '204.155.140.159'
28843 2015-11-12 08:56:09 Standard protocol on port 1985
28843 2015-11-12 08:56:09 Will connect to address 204.155.140.159 port
1985
28843 2015-11-12 08:56:09 Connect status is 0
28843 2015-11-12 08:56:09 Sent 16 bytes
28843 2015-11-12 08:56:09 Read 14206 bytes
28843 2015-11-12 08:56:09 Closing connection
28843 2015-11-12 08:56:09 Queuing request 1 to 10.6.0.15:1985 for
remoteclient2: 'pullclient 1
'
28843 2015-11-12 08:56:09 Queuing request 2 to 10.6.0.14:1985 for
remoteclient: 'pullclient 1
'
2015-11-12 08:56:09 Connection lost during connect/write to 10.6.0.14:1985
(req 2): Connection refused
28843 2015-11-12 08:56:09 Doing cleanup
28843 2015-11-12 08:56:09 Next poll of remoteclient in 55 seconds
28843 2015-11-12 08:56:09 Request completed: req 2, peer 10.6.0.14:1985,
action was 2, type was 0
2015-11-12 08:56:09 Connection lost during connect/write to 10.6.0.15:1985
(req 1): Connection refused
28843 2015-11-12 08:56:09 Doing cleanup
28843 2015-11-12 08:56:09 Next poll of remoteclient2 in 48 seconds
28843 2015-11-12 08:56:09 Request completed: req 1, peer 10.6.0.14:1985,
action was 2, type was 0
28843 2015-11-12 08:56:57 Queuing request 3 to 10.6.0.15:1985 for
remoteclient2: 'pullclient 1
'
2015-11-12 08:56:57 Connection lost during connect/write to 10.6.0.15:1985
(req 3): Connection refused
28843 2015-11-12 08:56:57 Doing cleanup
28843 2015-11-12 08:56:57 Next poll of remoteclient2 in 52 seconds
28843 2015-11-12 08:56:57 Request completed: req 3, peer 10.6.0.15:1985,
action was 2, type was 0
It makes no difference if I put in –server=127.0.0.1:1984 or
–server=204.155.140.159:1984.
From: Japheth Cleaver [mailto:user-87556346d4af@xymon.invalid]
Sent: Wednesday, November 11, 2015 6:13 PM
To: Root, Paul T; xymon at xymon.com<mailto:xymon at xymon.com>
Subject: Re: [Xymon] xymonfetch
On 11/11/2015 2:04 PM, Root, Paul T wrote:
I’m having issues with xymonfetch.
I have 2 xymon proxies, that also have the xymon server running as a
backup to the main server on port 1985. The proxy sends to the main
server on port 1984 and to itself on port 1985.
I’m trying to add 2 new machines as clients of these proxy servers. They
are outside a firewall that I don’t want to open, so I’m doing a
fetch. Xymon-client is running on 1984, and msgcache is running.
Client clientlaunch.cfg:
[msgcache]
# DISABLED
ENVFILE $XYMONCLIENTHOME/etc/xymonclient.cfg
CMD $XYMONCLIENTHOME/bin/msgcache --no-daemon
--pidfile=$XYMONCLIENTLOGS/msgcache.pid
LOGFILE $XYMONCLIENTLOGS/msgcache.log
Proxy tasks.cfg:
[xymonfetch]
ENVFILE /etc/xymon/xymonserver.cfg
#CMD $XYMONHOME/bin/xymonfetch --server=YOUR.XYMON.SERVER.IP
--no-daemon --pidfile=$XYMONSERVERLOGS/xymonfetch.pid
CMD $XYMONHOME/bin/xymonfetch --server=127.0.0.1:1985
--log-interval=60 --id=1 --pidfile=$XYMONSERVERLOGS/xymonfetch.pid
LOGFILE $XYMONSERVERLOGS/xymonfetch.log
Proxy hosts.cfg:
10.6.0.14 remoteclient # pulldata=10.6.0.14:1984
http://198.36.155.243
10.6.0.15 remoteclient2 # pulldata=10.6.0.15:1984
I have a third xymon server, the development server, which is not a proxy,
and the server runs on 1984. This pulls the data just fine from the
remoteclients.
The xymonfetch.log file gets these errors:
2015-11-11 15:56:24 Connection lost during connect/write to 10.6.0.14:1985
(req 27): Connection refused
2015-11-11 15:56:41 Connection lost during connect/write to 10.6.0.15:1985
(req 28): Connection refused
2015-11-11 15:57:27 Connection lost during connect/write to 10.6.0.14:1985
(req 29): Connection refused
2015-11-11 15:57:44 Connection lost during connect/write to 10.6.0.15:1985
(req 30): Connection refused
The proxy servers are 4.3.10. The development server is 4.3.21. The main
server is also 4.3.10, and it runs a fetch without issue, to other
clients.
The clients are 4.3.10 (CentOS 5) and 4.3.21 (CentOS 6).
I am working on upgrading the proxy servers in OS (CentOS 5 -> 6) and
Xymon (to 4.3.21). But not quite there yet. Another subject, but sending
data from a proxy to another proxy does not work. So, when I have
everything else, I may end up putting it in place, and hoping the proxy
works….
Any ideas here? It seems like such a simple thing.
Does seem a bit odd. Either xymonfetch is not paying attention to
pulldata, or it seems to want to reconnect back to the original host
incorrectly for the client (config) reply. Can you try --debug on the
xymonfetch process, or alternatively strace it, so we can see at which
phase of the transaction that write is failing at?
On the second issue, I know xymonproxy is capable of doing proxy-proxy
submission, as we used it pretty heavily for that (albeit with caveats at
scale when dealing with TCP packet loss), does a 'ping' make it through at
all when going on that path?
-jc
This communication is the property of CenturyLink and may contain
confidential or privileged information. Unauthorized use of this
communication is strictly prohibited and may be unlawful. If you have
received this communication in error, please immediately notify the sender
by reply e-mail and destroy all copies of the communication and any
attachments.
This communication is the property of CenturyLink and may contain
confidential or privileged information. Unauthorized use of this
communication is strictly prohibited and may be unlawful. If you have
received this communication in error, please immediately notify the sender
by reply e-mail and destroy all copies of the communication and any
attachments.
This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
list Paul Root
So a new hiccup. Putting in the XYMONDPORT=1984, the process pulls the "pulldata" commands out of the hosts.cfg of the main server, not the local proxy. That's a bit crazy. The hosts.cfg file are drastically different between the main server, and the proxies. The proxy is set to send locally first, then to the primary: CMD $XYMONHOME/bin/xymonproxy --server=127.0.0.1:1985,10.0.0.10 --listen=0.0.0.0:1984 --report=$MACHINE.xymonproxy --no-daemon --pidfile=$XYMONSERVERLOGS/xymonproxy.pid I'm going to try to put the pulldata for the two servers I want the proxies to pull, but can't do that right now. More tomorrow. Still haven't had time to put the new proxy in place, also.
▸
-----Original Message-----
From: J.C. Cleaver [mailto:user-87556346d4af@xymon.invalid]
Sent: Friday, November 13, 2015 12:37 PM
To: Root, Paul T
Cc: Xymon Mailing List
Subject: RE: [Xymon] xymonfetch
Hmm. It's looking like we have a fair amount of crufty code in xymonfetch
that might need to be brought up to spec, that might be causing some of
the confusion.
First, the syntax will need to be just --server=<destinationIP>. The port
is hard-coded to use whatever $XYMONDPORT is (from xymonserver.cfg if via
xymonlaunch/xymoncmd, but can be altered on the command line with 'env').
Secondly (possible typo), it'll need to be --id=# on the CMD line, not
just id=#.
I have to admit to not using xymonfetch or msgcache much myself, but
xymonproxy itself should be able to handle the pass-through of two-way
messages okay for the most part. I think.
Can re-run with just the bare IP? I suspect it wasn't trying to connect
with what it looked like it was trying to.
HTH,
-jc
On Fri, November 13, 2015 6:59 am, Root, Paul T wrote:The saga continues.
I got my network people to open up 1985 through the firewall to my
machines, and then restarted xymon-client on those machines to use 1985
with msgcache. So while that is wrong, at least I have a work around.
Also, the fetch=ip:1984 was ignored, it only would talk on port 1985.
Then the xymonfetch on the proxy servers would connect and get data from
them.
HOWEVER, the proxy servers will not send the data to the proxy port
(1984), only the server port (1985). Which doesn’t get me data to my
primary server.
Is it possible to run xymonfetch from a client machine? That client would
send to the xymon proxy server.
If I get time today (or maybe over the weekend), I’m going to try to put
my new xymon proxy server in place (CentOS6 with 4.3.21). Again the
current servers are CentOS 5 with 4.3.10.
[xymonfetch]
ENVFILE /etc/xymon/xymonserver.cfg
#CMD $XYMONHOME/bin/xymonfetch --server=127.0.0.1:1984 --id=1
--pidfile=$XYMONSERVERLOGS/xymonfetch.pid
# does not work
CMD $XYMONHOME/bin/xymonfetch --server=204.155.140.159:1984 --id=1
--pidfile=$XYMONSERVERLOGS/xymonfetch.pid # does not
work
#CMD $XYMONHOME/bin/xymonfetch id=1
--pidfile=$XYMONSERVERLOGS/xymonfetch.pid
#
works to server not proxy
LOGFILE $XYMONSERVERLOGS/xymonfetch.log
Xymonfetch.log:
2015-11-13 08:00:19 Invalid client IP: 127.0.0.1:1984 (req 2435)
2015-11-13 08:03:36 Invalid client IP: 127.0.0.1:1984 (req 2443)
2015-11-13 08:15:22 Invalid client IP: 127.0.0.1:1984 (req 2473)
2015-11-13 08:18:46 Invalid client IP: 127.0.0.1:1984 (req 2481)
2015-11-13 08:27:52 Caught TERM signal, terminating
2015-11-13 08:27:52 Caught TERM signal, terminating
2015-11-13 08:27:52 Caught TERM signal, terminating
2015-11-13 08:29:24 Invalid client IP: 204.155.140.159:1984 (req 6)
2015-11-13 08:29:24 Invalid client IP: 204.155.140.159:1984 (req 6)
2015-11-13 08:29:24 Invalid client IP: 204.155.140.159:1984 (req 6)
2015-11-13 08:30:45 Invalid client IP: 204.155.140.159:1984 (req 10)
2015-11-13 08:30:45 Invalid client IP: 204.155.140.159:1984 (req 10)
2015-11-13 08:30:45 Invalid client IP: 204.155.140.159:1984 (req 10)
From: Xymon [mailto:xymon-bounces at xymon.com] On Behalf Of Root, Paul T
Sent: Thursday, November 12, 2015 9:14 AM
To: 'Japheth Cleaver'; 'xymon at xymon.com'
Subject: Re: [Xymon] xymonfetch
It gets even weirder. If I run it without xymoncmd, then it uses port 1984
(the xymonproxy) but then is trying to pull data from the machine the main
server pulls from, and doesn’t even exist in the proxy’s hosts.cfg.
But when I run with xymoncmd, I get this. To me, it appears to be
completely ignoring the command line:
$ xymonfetch --server=:1984 --log-interval=60 --id=1 --debug
28843 2015-11-12 08:56:09 Transport setup is:
28843 2015-11-12 08:56:09 xymondportnumber = 1985
28843 2015-11-12 08:56:09 xymonproxyhost = NONE
28843 2015-11-12 08:56:09 xymonproxyport = 0
28843 2015-11-12 08:56:09 Recipient listed as '204.155.140.159'
28843 2015-11-12 08:56:09 Standard protocol on port 1985
28843 2015-11-12 08:56:09 Will connect to address 204.155.140.159 port
1985
28843 2015-11-12 08:56:09 Connect status is 0
28843 2015-11-12 08:56:09 Sent 16 bytes
28843 2015-11-12 08:56:09 Read 14206 bytes
28843 2015-11-12 08:56:09 Closing connection
28843 2015-11-12 08:56:09 Queuing request 1 to 10.6.0.15:1985 for
remoteclient2: 'pullclient 1
'
28843 2015-11-12 08:56:09 Queuing request 2 to 10.6.0.14:1985 for
remoteclient: 'pullclient 1
'
2015-11-12 08:56:09 Connection lost during connect/write to 10.6.0.14:1985
(req 2): Connection refused
28843 2015-11-12 08:56:09 Doing cleanup
28843 2015-11-12 08:56:09 Next poll of remoteclient in 55 seconds
28843 2015-11-12 08:56:09 Request completed: req 2, peer 10.6.0.14:1985,
action was 2, type was 0
2015-11-12 08:56:09 Connection lost during connect/write to 10.6.0.15:1985
(req 1): Connection refused
28843 2015-11-12 08:56:09 Doing cleanup
28843 2015-11-12 08:56:09 Next poll of remoteclient2 in 48 seconds
28843 2015-11-12 08:56:09 Request completed: req 1, peer 10.6.0.14:1985,
action was 2, type was 0
28843 2015-11-12 08:56:57 Queuing request 3 to 10.6.0.15:1985 for
remoteclient2: 'pullclient 1
'
2015-11-12 08:56:57 Connection lost during connect/write to 10.6.0.15:1985
(req 3): Connection refused
28843 2015-11-12 08:56:57 Doing cleanup
28843 2015-11-12 08:56:57 Next poll of remoteclient2 in 52 seconds
28843 2015-11-12 08:56:57 Request completed: req 3, peer 10.6.0.15:1985,
action was 2, type was 0
It makes no difference if I put in –server=127.0.0.1:1984 or
–server=204.155.140.159:1984.
From: Japheth Cleaver [mailto:user-87556346d4af@xymon.invalid]
Sent: Wednesday, November 11, 2015 6:13 PM
To: Root, Paul T; xymon at xymon.com<mailto:xymon at xymon.com>
Subject: Re: [Xymon] xymonfetch
On 11/11/2015 2:04 PM, Root, Paul T wrote:
I’m having issues with xymonfetch.
I have 2 xymon proxies, that also have the xymon server running as a
backup to the main server on port 1985. The proxy sends to the main
server on port 1984 and to itself on port 1985.
I’m trying to add 2 new machines as clients of these proxy servers. They
are outside a firewall that I don’t want to open, so I’m doing a
fetch. Xymon-client is running on 1984, and msgcache is running.
Client clientlaunch.cfg:
[msgcache]
# DISABLED
ENVFILE $XYMONCLIENTHOME/etc/xymonclient.cfg
CMD $XYMONCLIENTHOME/bin/msgcache --no-daemon
--pidfile=$XYMONCLIENTLOGS/msgcache.pid
LOGFILE $XYMONCLIENTLOGS/msgcache.log
Proxy tasks.cfg:
[xymonfetch]
ENVFILE /etc/xymon/xymonserver.cfg
#CMD $XYMONHOME/bin/xymonfetch --server=YOUR.XYMON.SERVER.IP
--no-daemon --pidfile=$XYMONSERVERLOGS/xymonfetch.pid
CMD $XYMONHOME/bin/xymonfetch --server=127.0.0.1:1985
--log-interval=60 --id=1 --pidfile=$XYMONSERVERLOGS/xymonfetch.pid
LOGFILE $XYMONSERVERLOGS/xymonfetch.log
Proxy hosts.cfg:
10.6.0.14 remoteclient # pulldata=10.6.0.14:1984
http://198.36.155.243
10.6.0.15 remoteclient2 # pulldata=10.6.0.15:1984
I have a third xymon server, the development server, which is not a proxy,
and the server runs on 1984. This pulls the data just fine from the
remoteclients.
The xymonfetch.log file gets these errors:
2015-11-11 15:56:24 Connection lost during connect/write to 10.6.0.14:1985
(req 27): Connection refused
2015-11-11 15:56:41 Connection lost during connect/write to 10.6.0.15:1985
(req 28): Connection refused
2015-11-11 15:57:27 Connection lost during connect/write to 10.6.0.14:1985
(req 29): Connection refused
2015-11-11 15:57:44 Connection lost during connect/write to 10.6.0.15:1985
(req 30): Connection refused
The proxy servers are 4.3.10. The development server is 4.3.21. The main
server is also 4.3.10, and it runs a fetch without issue, to other
clients.
The clients are 4.3.10 (CentOS 5) and 4.3.21 (CentOS 6).
I am working on upgrading the proxy servers in OS (CentOS 5 -> 6) and
Xymon (to 4.3.21). But not quite there yet. Another subject, but sending
data from a proxy to another proxy does not work. So, when I have
everything else, I may end up putting it in place, and hoping the proxy
works….
Any ideas here? It seems like such a simple thing.
Does seem a bit odd. Either xymonfetch is not paying attention to
pulldata, or it seems to want to reconnect back to the original host
incorrectly for the client (config) reply. Can you try --debug on the
xymonfetch process, or alternatively strace it, so we can see at which
phase of the transaction that write is failing at?
On the second issue, I know xymonproxy is capable of doing proxy-proxy
submission, as we used it pretty heavily for that (albeit with caveats at
scale when dealing with TCP packet loss), does a 'ping' make it through at
all when going on that path?
-jc
This communication is the property of CenturyLink and may contain
confidential or privileged information. Unauthorized use of this
communication is strictly prohibited and may be unlawful. If you have
received this communication in error, please immediately notify the sender
by reply e-mail and destroy all copies of the communication and any
attachments.
This communication is the property of CenturyLink and may contain
confidential or privileged information. Unauthorized use of this
communication is strictly prohibited and may be unlawful. If you have
received this communication in error, please immediately notify the sender
by reply e-mail and destroy all copies of the communication and any
attachments.
This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
list Adam Goryachev
▸
On 13/11/2015 18:55, Root, Paul T wrote:
So a new hiccup. Putting in the XYMONDPORT=1984, the process pulls the "pulldata" commands out of the hosts.cfg of the main server, not the local proxy. That's a bit crazy. The hosts.cfg file are drastically different between the main server, and the proxies.
The proxy is set to send locally first, then to the primary:
CMD $XYMONHOME/bin/xymonproxy --server=127.0.0.1:1985,10.0.0.10 --listen=0.0.0.0:1984 --report=$MACHINE.xymonproxy --no-daemon --pidfile=$XYMONSERVERLOGS/xymonproxy.pid
I'm going to try to put the pulldata for the two servers I want the proxies to pull, but can't do that right now. More tomorrow. Still haven't had time to put the new proxy in place, also.
AFAIK, the proxy will pull the data from the last server you configured, so try reversing the order that the proxy reports to, I am pretty sure this is documented in the xymonproxy manpage. Hope that helps. Regards, Adam
list Paul Root
Thanks,
I try that.
▸
-----Original Message-----
From: Xymon [mailto:xymon-bounces at xymon.com] On Behalf Of Adam Goryachev
Sent: Friday, November 13, 2015 6:38 PM
To: xymon at xymon.com
Subject: Re: [Xymon] xymonfetch
On 13/11/2015 18:55, Root, Paul T wrote:So a new hiccup. Putting in the XYMONDPORT=1984, the process pulls the "pulldata" commands out of the hosts.cfg of the main server, not the local proxy. That's a bit crazy. The hosts.cfg file are drastically different between the main server, and the proxies.
The proxy is set to send locally first, then to the primary:
CMD $XYMONHOME/bin/xymonproxy --server=127.0.0.1:1985,10.0.0.10 --listen=0.0.0.0:1984 --report=$MACHINE.xymonproxy --no-daemon --pidfile=$XYMONSERVERLOGS/xymonproxy.pid
I'm going to try to put the pulldata for the two servers I want the proxies to pull, but can't do that right now. More tomorrow. Still haven't had time to put the new proxy in place, also.
AFAIK, the proxy will pull the data from the last server you configured, so try reversing the order that the proxy reports to, I am pretty sure this is documented in the xymonproxy manpage. Hope that helps. Regards, Adam This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
list Paul Root
I believe this works!
So to summarize the solution is in tasks.cfg:
[xymonproxy]
CMD $XYMONHOME/bin/xymonproxy --server=<main server IP>,127.0.0.1:1985 --listen=0.0.0.0:1984 --report=$MACHINE.xymonproxy --no-daemon --pidfile=$XYMONSERVERLOGS/xymonproxy.pid
[xymonfetch]
CMD XYMONDPORT=1984 $XYMONHOME/bin/xymonfetch --log-interval=60 --server=127.0.0.1 --id=1 --pidfile=$XYMONSERVERLOGS/xymonfetch.pid
And the clients msgcache is on 1984 (I had tried moving it to 1985). The hosts.cfg of the proxy/secondary server has the pulldata field in the comments.
Oh, and I did put the CentOS 6/Xymon 4.3.21 in place, so this works for both 4.3.10 and 4.3.21.
Thanks for all the help.
Paul.
▸
-----Original Message-----
From: Xymon [mailto:xymon-bounces at xymon.com] On Behalf Of Root, Paul T
Sent: Saturday, November 14, 2015 12:07 PM
To: 'Adam Goryachev'; 'xymon at xymon.com'
Subject: Re: [Xymon] xymonfetch
Thanks,
I try that.
-----Original Message-----
From: Xymon [mailto:xymon-bounces at xymon.com] On Behalf Of Adam Goryachev
Sent: Friday, November 13, 2015 6:38 PM
To: xymon at xymon.com
Subject: Re: [Xymon] xymonfetch
On 13/11/2015 18:55, Root, Paul T wrote:So a new hiccup. Putting in the XYMONDPORT=1984, the process pulls the "pulldata" commands out of the hosts.cfg of the main server, not the local proxy. That's a bit crazy. The hosts.cfg file are drastically different between the main server, and the proxies.
The proxy is set to send locally first, then to the primary:
CMD $XYMONHOME/bin/xymonproxy --server=127.0.0.1:1985,10.0.0.10 --listen=0.0.0.0:1984 --report=$MACHINE.xymonproxy --no-daemon --pidfile=$XYMONSERVERLOGS/xymonproxy.pid
I'm going to try to put the pulldata for the two servers I want the proxies to pull, but can't do that right now. More tomorrow. Still haven't had time to put the new proxy in place, also.
AFAIK, the proxy will pull the data from the last server you configured, so try reversing the order that the proxy reports to, I am pretty sure this is documented in the xymonproxy manpage. Hope that helps. Regards, Adam This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments. This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.