Xymon Mailing List Archive search

agents with pulldata - xymonfetch and ssh tunelling

list Pierre L
Thu, 02 Nov 2017 08:56:12 +0000
Message-Id: <user-95ff03cc83de@xymon.invalid>

Hello John,

I ll try to answer your questions:

Have you confirmed msgcache is listening on your remote host?

that's what msgcache log on the client says:

 2017-11-02 09:05:34.064994 Refused message from 127.0.0.1: pullclient 1
 2017-11-02 09:05:34.065025 Rejected pullclient request from 127.0.0.1

 it looks like it is listening since the last time i restarted the
serveice, then I have no logs.

Have you looked in the xymonfetch logfile? This is an option which canbe
defined in tasks.cfg

the xymonfetch log on the xymon server
2017-11-02 08:22:55.644671 Connection lost during connect/write to
127.0.0.1:1235 (req 4925): Connection refused
2017-11-02 08:38:07.598875 Connection lost during connect/write to
127.0.0.1:1235 (req 4957): Connection refused

Have you tried running your xymonfetch manually to see what happens?
This is how I was eventually able to figure out my problems.
    ~/server/bin/xymoncmd ~/server/bin/xymonfetch --id=19 --debug


*xymoncmd xymonfetch --id=19 --debug 23985 2017-11-02 09:02:24.783229  -
deciding if we should be compressing here... 23985 2017-11-02
09:02:24.783370 Using default compression type of zlib 23985 2017-11-02
09:02:24.783373 sendtomany: allservers=0, compression=0 23985 2017-11-02
09:02:24.783381 Will retry 2 times 23985 2017-11-02 09:02:24.783385
Transport setup is: 23985 2017-11-02 09:02:24.783387 xymondportnumber =
1984 23985 2017-11-02 09:02:24.783389 xymonproxyhost = NONE 23985
2017-11-02 09:02:24.783391 xymonproxyport = 0 23985 2017-11-02
09:02:24.783393 Recipient listed as 'xymonserverIP' 23985 2017-11-02
09:02:24.783395 Standard protocol on port 1984 23985 2017-11-02
09:02:24.783400 Will connect to address xymonserverIP   port 1984 23985
2017-11-02 09:02:24.783589 Connect status is 0 23985 2017-11-02
09:02:24.783613  - sent 16 bytes, 0 bytes remaining 23985 2017-11-02
09:02:24.783771 Read 2946 bytes 23985 2017-11-02 09:02:24.783789 Closing
connection 23985 2017-11-02 09:02:24.783817  -> initialize_hostlist 23985
2017-11-02 09:02:24.783820   - freeing default 23985 2017-11-02
09:02:24.783821   - freeing name list (namehead) 23985 2017-11-02
09:02:24.783823   - freeing page list (pghead) 23985 2017-11-02
09:02:24.783825  <- initialize_hostlist 23985 2017-11-02 09:02:24.783854
loadhostnames:checking if this host clienthostname-P02 has been defined
before... 23985 2017-11-02 09:02:24.783861  loadhostnames:adding host
clienthostname-P02 as a new item... = 0x1f8e640 23985 2017-11-02
09:02:24.783865  loadhostnames:build_hosttree - status for that add was 0
23985 2017-11-02 09:02:24.783870  loadhostnames: clientname found was
(null) 23985 2017-11-02 09:02:24.783882  loadhostnames:checking if this
host clienthostname-V01 has been defined before... 23985 2017-11-02
09:02:24.783885  loadhostnames:adding host clienthostname-V01 as a new
item... = 0x1f8e7f0 23985 2017-11-02 09:02:24.783887
loadhostnames:build_hosttree - status for that add was 0 23985 2017-11-02
09:02:24.783890  loadhostnames: clientname found was (null) 23985
2017-11-02 09:02:24.783896  loadhostnames:checking if this host
clienthostname-P01 has been defined before... 23985 2017-11-02
09:02:24.783899  loadhostnames:adding host clienthostname-P01 as a new
item... = 0x1f8e9e0 23985 2017-11-02 09:02:24.783902
loadhostnames:build_hosttree - status for that add was 0 23985 2017-11-02
09:02:24.783904  loadhostnames: clientname found was (null) 23985
2017-11-02 09:02:24.783909  loadhostnames:checking if this host
clienthostname-P01 has been defined before... 23985 2017-11-02
09:02:24.783912  loadhostnames:adding host clienthostname-P01 as a new
item... = 0x1f8ec50 23985 2017-11-02 09:02:24.783914
loadhostnames:build_hosttree - status for that add was 0 23985 2017-11-02
09:02:24.783916  loadhostnames: clientname found was (null) 23985
2017-11-02 09:02:24.783925  loadhostnames:checking if this host
clienthostname-P06 has been defined before... 23985 2017-11-02
09:02:24.783928  loadhostnames:adding host clienthostname-P06 as a new
item... = 0x1f8eef0 23985 2017-11-02 09:02:24.783931
loadhostnames:build_hosttree - status for that add was 0 23985 2017-11-02
09:02:24.783933  loadhostnames: clientname found was (null) 23985
2017-11-02 09:02:24.783938  loadhostnames:checking if this host
clienthostname-P07 has been defined before... 23985 2017-11-02
09:02:24.783941  loadhostnames:adding host clienthostname-P07 as a new
item... = 0x1f8f190 23985 2017-11-02 09:02:24.783943
loadhostnames:build_hosttree - status for that add was 0 23985 2017-11-02
09:02:24.783946  loadhostnames: clientname found was (null) 23985
2017-11-02 09:02:24.783952  loadhostnames:checking if this host
clienthostname-V01 has been defined before... 23985 2017-11-02
09:02:24.783955  loadhostnames:adding host clienthostname-V01 as a new
item... = 0x1f8f470 23985 2017-11-02 09:02:24.783958
loadhostnames:build_hosttree - status for that add was 0 23985 2017-11-02
09:02:24.783960  loadhostnames: clientname found was (null) 23985
2017-11-02 09:02:24.783965  loadhostnames:checking if this host
clienthostname-V01 has been defined before... 23985 2017-11-02
09:02:24.783967  loadhostnames:adding host clienthostname-V01 as a new
item... = 0x1f8f680 23985 2017-11-02 09:02:24.783970
loadhostnames:build_hosttree - status for that add was 0 23985 2017-11-02
09:02:24.783972  loadhostnames: clientname found was (null) 23985
2017-11-02 09:02:24.783977  loadhostnames:checking if this host
clienthostname-P01 has been defined before... 23985 2017-11-02
09:02:24.783980  loadhostnames:adding host clienthostname-P01 as a new
item... = 0x1f8f8c0 23985 2017-11-02 09:02:24.783982
loadhostnames:build_hosttree - status for that add was 0 23985 2017-11-02
09:02:24.783985  loadhostnames: clientname found was (null) 23985
2017-11-02 09:02:24.783989  loadhostnames:checking if this host
clienthostname-P02 has been defined before... 23985 2017-11-02
09:02:24.783992  loadhostnames:adding host clienthostname-P02 as a new
item... = 0x1f8fb10 23985 2017-11-02 09:02:24.783995
loadhostnames:build_hosttree - status for that add was 0 23985 2017-11-02
09:02:24.783997  loadhostnames: clientname found was (null) 23985
2017-11-02 09:02:24.784001  loadhostnames:checking if this host
clienthostname-P01 has been defined before... 23985 2017-11-02
09:02:24.784004  loadhostnames:adding host clienthostname-P01 as a new
item... = 0x1f8fd60 23985 2017-11-02 09:02:24.784006
loadhostnames:build_hosttree - status for that add was 0 23985 2017-11-02
09:02:24.784008  loadhostnames: clientname found was (null) 23985
2017-11-02 09:02:24.784015  loadhostnames:checking if this host
clienthostname-P02 has been defined before... 23985 2017-11-02
09:02:24.784018  loadhostnames:adding host clienthostname-P02 as a new
item... = 0x1f8ffb0 23985 2017-11-02 09:02:24.784020
loadhostnames:build_hosttree - status for that add was 0 23985 2017-11-02
09:02:24.784023  loadhostnames: clientname found was (null) 23985
2017-11-02 09:02:24.784028  loadhostnames:checking if this host
clienthostname-P10 has been defined before... 23985 2017-11-02
09:02:24.784031  loadhostnames:adding host clienthostname-P10 as a new
item... = 0x1f90230 23985 2017-11-02 09:02:24.784034
loadhostnames:build_hosttree - status for that add was 0 23985 2017-11-02
09:02:24.784036  loadhostnames: clientname found was (null) 23985
2017-11-02 09:02:24.784040  loadhostnames:checking if this host
clienthostname-P11 has been defined before... 23985 2017-11-02
09:02:24.784043  loadhostnames:adding host clienthostname-P11 as a new
item... = 0x1f90470 23985 2017-11-02 09:02:24.784045
loadhostnames:build_hosttree - status for that add was 0 23985 2017-11-02
09:02:24.784048  loadhostnames: clientname found was (null) 23985
2017-11-02 09:02:24.784054  loadhostnames:checking if this host
DE0-03IRO-P01 has been defined before... 23985 2017-11-02 09:02:24.784056
loadhostnames:adding host DE0-03IRO-P01 as a new item... = 0x1f906e0 23985
2017-11-02 09:02:24.784170  loadhostnames:build_hosttree - status for that
add was 0 23985 2017-11-02 09:02:24.784174  loadhostnames: clientname found
was (null) 23985 2017-11-02 09:02:24.784181  -> build_hosttree 23985
2017-11-02 09:02:24.784183  -> destroy_hosttree (posix) 23985 2017-11-02
09:02:24.784185  <- destroy_hosttree 23985 2017-11-02 09:02:24.784187
loadhosts:build_hosttree - xtreeAdd to rbhosts for clienthostname-P02 at
0x1f8e640 23985 2017-11-02 09:02:24.784190  loadhosts:build_hosttree -
status for that add was 0 23985 2017-11-02 09:02:24.784192
loadhosts:build_hosttree - walk->clientname for clienthostname-P02 is:
clienthostname-P02 23985 2017-11-02 09:02:24.784194
loadhosts:build_hosttree - xtreeAdd to rbclients for clienthostname-P02 at
0x1f8e640 23985 2017-11-02 09:02:24.784196  loadhosts:build_hosttree -
status for that add was 0 23985 2017-11-02 09:02:24.784202
loadhosts:build_hosttree - xtreeAdd to rbhosts for clienthostname-V01 at
0x1f8e7f0 23985 2017-11-02 09:02:24.784205  loadhosts:build_hosttree -
status for that add was 0 23985 2017-11-02 09:02:24.784207
loadhosts:build_hosttree - walk->clientname for clienthostname-V01 is:
clienthostname-V01 23985 2017-11-02 09:02:24.784209
loadhosts:build_hosttree - xtreeAdd to rbclients for clienthostname-V01 at
0x1f8e7f0 23985 2017-11-02 09:02:24.784211  loadhosts:build_hosttree -
status for that add was 0 23985 2017-11-02 09:02:24.784213
loadhosts:build_hosttree - xtreeAdd to rbhosts for clienthostname-P01 at
0x1f8e9e0 23985 2017-11-02 09:02:24.784216  loadhosts:build_hosttree -
status for that add was 0 23985 2017-11-02 09:02:24.784218
loadhosts:build_hosttree - walk->clientname for clienthostname-P01 is:
clienthostname-P01 23985 2017-11-02 09:02:24.784220
loadhosts:build_hosttree - xtreeAdd to rbclients for clienthostname-P01 at
0x1f8e9e0 23985 2017-11-02 09:02:24.784222  loadhosts:build_hosttree -
status for that add was 0 23985 2017-11-02 09:02:24.784224
loadhosts:build_hosttree - xtreeAdd to rbhosts for clienthostname-P01 at
0x1f8ec50 23985 2017-11-02 09:02:24.784226  loadhosts:build_hosttree -
status for that add was 0 23985 2017-11-02 09:02:24.784228
loadhosts:build_hosttree - walk->clientname for clienthostname-P01 is:
clienthostname-P01 23985 2017-11-02 09:02:24.784230
loadhosts:build_hosttree - xtreeAdd to rbclients for clienthostname-P01 at
0x1f8ec50 23985 2017-11-02 09:02:24.784232  loadhosts:build_hosttree -
status for that add was 0 23985 2017-11-02 09:02:24.784234
loadhosts:build_hosttree - xtreeAdd to rbhosts for clienthostname-P06 at
0x1f8eef0 23985 2017-11-02 09:02:24.784236  loadhosts:build_hosttree -
status for that add was 0 23985 2017-11-02 09:02:24.784238
loadhosts:build_hosttree - walk->clientname for clienthostname-P06 is:
clienthostname-P06 23985 2017-11-02 09:02:24.784240
loadhosts:build_hosttree - xtreeAdd to rbclients for clienthostname-P06 at
0x1f8eef0 23985 2017-11-02 09:02:24.784243  loadhosts:build_hosttree -
status for that add was 0 23985 2017-11-02 09:02:24.784245
loadhosts:build_hosttree - xtreeAdd to rbhosts for clienthostname-P07 at
0x1f8f190 23985 2017-11-02 09:02:24.784247  loadhosts:build_hosttree -
status for that add was 0 23985 2017-11-02 09:02:24.784249
loadhosts:build_hosttree - walk->clientname for clienthostname-P07 is:
clienthostname-P07 23985 2017-11-02 09:02:24.784251
loadhosts:build_hosttree - xtreeAdd to rbclients for clienthostname-P07 at
0x1f8f190 23985 2017-11-02 09:02:24.784253  loadhosts:build_hosttree -
status for that add was 0 23985 2017-11-02 09:02:24.784255
loadhosts:build_hosttree - xtreeAdd to rbhosts for clienthostname-V01 at
0x1f8f470 23985 2017-11-02 09:02:24.784257  loadhosts:build_hosttree -
status for that add was 0 23985 2017-11-02 09:02:24.784259
loadhosts:build_hosttree - walk->clientname for clienthostname-V01 is:
clienthostname-V01 23985 2017-11-02 09:02:24.784261
loadhosts:build_hosttree - xtreeAdd to rbclients for clienthostname-V01 at
0x1f8f470 23985 2017-11-02 09:02:24.784263  loadhosts:build_hosttree -
status for that add was 0 23985 2017-11-02 09:02:24.784265
loadhosts:build_hosttree - xtreeAdd to rbhosts for clienthostname-V01 at
0x1f8f680 23985 2017-11-02 09:02:24.784268  loadhosts:build_hosttree -
status for that add was 0 23985 2017-11-02 09:02:24.784270
loadhosts:build_hosttree - walk->clientname for clienthostname-V01 is:
clienthostname-V01 23985 2017-11-02 09:02:24.784272
loadhosts:build_hosttree - xtreeAdd to rbclients for clienthostname-V01 at
0x1f8f680 23985 2017-11-02 09:02:24.784274  loadhosts:build_hosttree -
status for that add was 0 23985 2017-11-02 09:02:24.784276
loadhosts:build_hosttree - xtreeAdd to rbhosts for clienthostname-P01 at
0x1f8f8c0 23985 2017-11-02 09:02:24.784278  loadhosts:build_hosttree -
status for that add was 0 23985 2017-11-02 09:02:24.784309
loadhosts:build_hosttree - walk->clientname for clienthostname-P01 is:
clienthostname-P01 23985 2017-11-02 09:02:24.784311
loadhosts:build_hosttree - xtreeAdd to rbclients for clienthostname-P01 at
0x1f8f8c0 23985 2017-11-02 09:02:24.784314  loadhosts:build_hosttree -
status for that add was 0 23985 2017-11-02 09:02:24.784317
loadhosts:build_hosttree - xtreeAdd to rbhosts for clienthostname-P02 at
0x1f8fb10 23985 2017-11-02 09:02:24.784319  loadhosts:build_hosttree -
status for that add was 0 23985 2017-11-02 09:02:24.784321
loadhosts:build_hosttree - walk->clientname for clienthostname-P02 is:
clienthostname-P02 23985 2017-11-02 09:02:24.784323
loadhosts:build_hosttree - xtreeAdd to rbclients for clienthostname-P02 at
0x1f8fb10 23985 2017-11-02 09:02:24.784325  loadhosts:build_hosttree -
status for that add was 0 23985 2017-11-02 09:02:24.784327
loadhosts:build_hosttree - xtreeAdd to rbhosts for clienthostname-P01 at
0x1f8fd60 23985 2017-11-02 09:02:24.784330  loadhosts:build_hosttree -
status for that add was 0 23985 2017-11-02 09:02:24.784331
loadhosts:build_hosttree - walk->clientname for clienthostname-P01 is:
clienthostname-P01 23985 2017-11-02 09:02:24.784333
loadhosts:build_hosttree - xtreeAdd to rbclients for clienthostname-P01 at
0x1f8fd60 23985 2017-11-02 09:02:24.784336  loadhosts:build_hosttree -
status for that add was 0 23985 2017-11-02 09:02:24.784338
loadhosts:build_hosttree - xtreeAdd to rbhosts for clienthostname-P02 at
0x1f8ffb0 23985 2017-11-02 09:02:24.784340  loadhosts:build_hosttree -
status for that add was 0 23985 2017-11-02 09:02:24.784342
loadhosts:build_hosttree - walk->clientname for clienthostname-P02 is:
clienthostname-P02 23985 2017-11-02 09:02:24.784344
loadhosts:build_hosttree - xtreeAdd to rbclients for clienthostname-P02 at
0x1f8ffb0 23985 2017-11-02 09:02:24.784430  loadhosts:build_hosttree -
status for that add was 0 23985 2017-11-02 09:02:24.784434
loadhosts:build_hosttree - xtreeAdd to rbhosts for clienthostname-P10 at
0x1f90230 23985 2017-11-02 09:02:24.784436  loadhosts:build_hosttree -
status for that add was 0 23985 2017-11-02 09:02:24.784438
loadhosts:build_hosttree - walk->clientname for clienthostname-P10 is:
clienthostname-P10 23985 2017-11-02 09:02:24.784440
loadhosts:build_hosttree - xtreeAdd to rbclients for clienthostname-P10 at
0x1f90230 23985 2017-11-02 09:02:24.784443  loadhosts:build_hosttree -
status for that add was 0 23985 2017-11-02 09:02:24.784445
loadhosts:build_hosttree - xtreeAdd to rbhosts for clienthostname-P11 at
0x1f90470 23985 2017-11-02 09:02:24.784447  loadhosts:build_hosttree -
status for that add was 0 23985 2017-11-02 09:02:24.784449
loadhosts:build_hosttree - walk->clientname for clienthostname-P11 is:
clienthostname-P11 23985 2017-11-02 09:02:24.784451
loadhosts:build_hosttree - xtreeAdd to rbclients for clienthostname-P11 at
0x1f90470 23985 2017-11-02 09:02:24.784453  loadhosts:build_hosttree -
status for that add was 0 23985 2017-11-02 09:02:24.784466  <-
build_hosttree 23985 2017-11-02 09:02:24.784507 Queuing request 1 to
127.0.0.1:1235 <http://127.0.0.1:1235>; for clienthostname-P02: 'pullclient
19*


*' 23985 2017-11-02 09:02:24.784592 Sent 14 bytes to 127.0.0.1:1235
<http://127.0.0.1:1235>; (req 1) 23985 2017-11-02 09:02:24.786253 Done
reading data from 127.0.0.1:1235 <http://127.0.0.1:1235>; (req 1) 23985
2017-11-02 09:02:24.786264 Doing cleanup 23985 2017-11-02 09:02:24.786268
Next poll of clienthostname-P02 in 68 seconds 23985 2017-11-02
09:02:24.786271 Request completed: req 1, peer 127.0.0.1:1235
<http://127.0.0.1:1235>;, action was 2, type was 0 23985 2017-11-02
09:03:32.854479 Queuing request 2 to 127.0.0.1:1235 <http://127.0.0.1:1235>;
for clienthostname-P02: 'pullclient 19*


*' 23985 2017-11-02 09:03:32.854671 Sent 14 bytes to 127.0.0.1:1235
<http://127.0.0.1:1235>; (req 2) 23985 2017-11-02 09:03:32.857518 Done
reading data from 127.0.0.1:1235 <http://127.0.0.1:1235>; (req 2) 23985
2017-11-02 09:03:32.857528 Doing cleanup 23985 2017-11-02 09:03:32.857531
Next poll of clienthostname-P02 in 53 seconds 23985 2017-11-02
09:03:32.857534 Request completed: req 2, peer 127.0.0.1:1235
<http://127.0.0.1:1235>;, action was 2, type was 0 23985 2017-11-02
09:04:25.914428 Queuing request 3 to 127.0.0.1:1235 <http://127.0.0.1:1235>;
for clienthostname-P02: 'pullclient 19    *

Looks like it is ok but on my server html page I do not see the results


What version of xymon are you running?
4.3.28-1.el7.terabithia


Le mar. 31 oct. 2017 à 18:08, John Thurston <user-ce4d79d99bab@xymon.invalid> a
écrit :
On 10/31/2017 8:37 AM, Pierre L wrote:
Hello,

I am trying to set up xymonfetch/msgcache with ssh tunnelling (because
security does not want to open flows on port 1984).

I have enabled xymonfetch (in tasks.cfg) and added pulldata (in
hosts.cfg) on server side and also msgcache (in clientlaunch.cfg)
and XYMSRV=127.0.0.1 (in xymonclient.cfg).

I have done a ssh tunnel (ssh -L 1234:127.0.0.1:1984
 IP_of_the_client) from the xymon server. My
tunnel is ok but xymon logs still show that nothing happens, no pulldata.

Is there something else I should do to enable this function?
I spent some time making xymonfetch/msgcache for for us. Some questions
for you:

Have you confirmed msgcache is listening on your remote host?

Have you looked in the xymonfetch logfile? This is an option which can
be defined in tasks.cfg

Have you tried running your xymonfetch manually to see what happens?
This is how I was eventually able to figure out my problems.

    ~/server/bin/xymoncmd ~/server/bin/xymonfetch --id=19 --debug

That will go through your hosts.cfg looking for pulldata tags. Those
hosts will then be asked for data. You may find that xymonfetch is not
querying the server:port combination you expect. You may find xymonfetch
is asking for data, and your client has not put anything into the msgcache.

What version of xymon are you running? Prior to [something pretty
recent], xymonfetch did not honor the "port" portion of the pulldata
tag. It would only query on port 1984. Look in the list archives, and
you will find my posts on this subject.


--
    Do things because you should, not just because you can.

John Thurston    XXX-XXX-XXXX <(907)%20465-8591>
user-ce4d79d99bab@xymon.invalid
Department of Administration
State of Alaska