Encryption 🔗 link

On Wed, 23 Aug 2023 at 22:02, Stef Coene <user-dbffe946c0f4@xymon.invalid> wrote:

Hi,

We solved the encryption by using a wget and/or curl script alternative
for the xymon client. It's a drop-in replacement.
We have a setup script that checks if wget or cups exists and it creates
a symlink for the xymon command to the script that works.
The script uses a username and password to connect to xymoncgimsg.cgi
over https to send the data.
We use 1 username / paswoord for all clients but with some scripting you
can give each client it's own username / password.

If wget or cups is not available (some old AIX servers can not connect
to a https server...), the good old binary is used without encryption :(

I can document it somewhere if anyone is interested.
It's not that complicated.


For the Windows clients we also use https as much as possible.

I have some patches for the Powershell client that allows for testing a
new XML file. We want to manage the XML files centrally and want to
avoid mistakes that can disable the client. So this allows us to test a
new config file before overwriting the old one.
I also added the ping test command for this and did some changes so data
can be send to multiple Xymon servers.
I will create some patches and send them to the mailinglist.


Stef

On Wed, August 23, 2023 16:41, Jeremy Laidman wrote:

Stef

Nice idea on the Powershell patch. We don't monitor Windows hosts where I
use Xymon, but others in this list might benefit.

Is the PS client able to use https natively, or do you need to install
curl.exe on those hosts?

I'm interested in the wget/curl option. Have you had an opportunity to
compare performance impact on the server running xymond and the CGI script
vs the native TCP/1984 method? This is perhaps my biggest concern, knowing
that xymond is written with high-performance socket handling, and although
Apache also has code optimised thusly, the CGI script execution might
become a bottleneck. I fear stunnel and similar solutions would suffer
from
the same problem, due to (I assume) being written for low socket turn-over
environments.

Does the CGI provide the client local data (from client-local.cfg) to the
client?

I'd be interested to see a HOWTO on this, even though it seems simple
enough. If the CGI option works out for us, we'll be able to cut over a
large proportion of hosts.

However we have a sizeable proportion of hosts that cannot make
connections
to the Xymon server, and so we have to do funky things like ssh from
xymons
server to client, setup a reverse tunnel and adjust XYMSRV to 127.0.0.1.
It's encrypted, but it's not particularly clean, and we have to all sorts
of extra hacks to support dual Xymon servers (so that the client message
tempfile for one Xymon server isn't stomped on by the second instance
running for the other Xymon server, and so the reverse tunnels don't
interfere with each others' listening ports). I'm investigating using
msgcache/xymonfetch option designed for server->client connections, but
it's non-trivial to have it work through an ssh tunnel, and so far I've
not
succeeded.

J

On Wed, 23 Aug 2023 at 22:02, Stef Coene <user-dbffe946c0f4@xymon.invalid> wrote:

Hi,

We solved the encryption by using a wget and/or curl script alternative
for the xymon client. It's a drop-in replacement.
We have a setup script that checks if wget or cups exists and it creates
a symlink for the xymon command to the script that works.
The script uses a username and password to connect to xymoncgimsg.cgi
over https to send the data.
We use 1 username / paswoord for all clients but with some scripting you
can give each client it's own username / password.

If wget or cups is not available (some old AIX servers can not connect
to a https server...), the good old binary is used without encryption :(

I can document it somewhere if anyone is interested.
It's not that complicated.


For the Windows clients we also use https as much as possible.

I have some patches for the Powershell client that allows for testing a
new XML file. We want to manage the XML files centrally and want to
avoid mistakes that can disable the client. So this allows us to test a
new config file before overwriting the old one.
I also added the ping test command for this and did some changes so data
can be send to multiple Xymon servers.
I will create some patches and send them to the mailinglist.


Stef

wrote:

On Wed, August 23, 2023 16:41, Jeremy Laidman wrote:

Stef

Nice idea on the Powershell patch. We don't monitor Windows hosts where I
use Xymon, but others in this list might benefit.

Is the PS client able to use https natively, or do you need to install
curl.exe on those hosts?

I'm interested in the wget/curl option. Have you had an opportunity to
compare performance impact on the server running xymond and the CGI
script
vs the native TCP/1984 method? This is perhaps my biggest concern,
knowing
that xymond is written with high-performance socket handling, and
although
Apache also has code optimised thusly, the CGI script execution might
become a bottleneck. I fear stunnel and similar solutions would suffer
from
the same problem, due to (I assume) being written for low socket
turn-over
environments.

Does the CGI provide the client local data (from client-local.cfg) to the
client?

I'd be interested to see a HOWTO on this, even though it seems simple
enough. If the CGI option works out for us, we'll be able to cut over a
large proportion of hosts.

However we have a sizeable proportion of hosts that cannot make
connections
to the Xymon server, and so we have to do funky things like ssh from
xymons
server to client, setup a reverse tunnel and adjust XYMSRV to 127.0.0.1.
It's encrypted, but it's not particularly clean, and we have to all sorts
of extra hacks to support dual Xymon servers (so that the client message
tempfile for one Xymon server isn't stomped on by the second instance
running for the other Xymon server, and so the reverse tunnels don't
interfere with each others' listening ports). I'm investigating using
msgcache/xymonfetch option designed for server->client connections, but
it's non-trivial to have it work through an ssh tunnel, and so far I've
not
succeeded.

The two-way communication for the client message is indeed an obstacle,
since a full conversation with xymond is necessary, but only for really
high-concurrency situations.

The Terabithia RPM has a STATUSMODE option that leveraged a
bb-compatibility hack in xymond which allowed client messages to be
one-way. This was updated to a SUBMITMODE option ~4.3.19 with some server
patches to accept new types of client(submit/config) commands. This was
added into the 4.x branch at https://sourceforge.net/p/xymon/code/7852/
and should be working in 4.4

Basically, both of these (SUBMITMODE preferred now) make client messages
much more efficient for processing, either by the .cgi or by xymonproxy
(which does much the same thing). This also means the xymon binary can be
forked by xymonclient.sh when submitting the message (we don't need a
response), which helps if you're sending to multiple servers at once.


As to how to actually download the client-local.cfg snippet? In the RPM,
this is handled via a separate client task that once every 30m downloads
client-local.cfg instead (with a null client body, so the connection is
just like any other single-line command). I need to double-check that this
script made it into SF, however.

A better recommendation in a truly large farm would probably be to have
this managed out-of-band by a config mgmt system, but lowering the
frequency of two-way comms to 30m seemed a reasonable middle ground.

Unfortunately, yeah; the encryption, SSL, and massive-scale performance
questions are all a bit intertwined still.


Regards,
-jc

J

On Wed, 23 Aug 2023 at 22:02, Stef Coene <user-dbffe946c0f4@xymon.invalid> wrote:

Hi,

We solved the encryption by using a wget and/or curl script alternative
for the xymon client. It's a drop-in replacement.
We have a setup script that checks if wget or cups exists and it creates
a symlink for the xymon command to the script that works.
The script uses a username and password to connect to xymoncgimsg.cgi
over https to send the data.
We use 1 username / paswoord for all clients but with some scripting you
can give each client it's own username / password.

If wget or cups is not available (some old AIX servers can not connect
to a https server...), the good old binary is used without encryption :(

I can document it somewhere if anyone is interested.
It's not that complicated.


For the Windows clients we also use https as much as possible.

I have some patches for the Powershell client that allows for testing a
new XML file. We want to manage the XML files centrally and want to
avoid mistakes that can disable the client. So this allows us to test a
new config file before overwriting the old one.
I also added the ping test command for this and did some changes so data
can be send to multiple Xymon servers.
I will create some patches and send them to the mailinglist.


Stef

On 2023-08-23 13:57, Stef Coene wrote:

Hi,

We solved the encryption by using a wget and/or curl script alternative for the xymon client. It's a drop-in replacement.
We have a setup script that checks if wget or cups exists and it creates a symlink for the xymon command to the script that works.
The script uses a username and password to connect to xymoncgimsg.cgi over https to send the data.
We use 1 username / paswoord for all clients but with some scripting you can give each client it's own username / password.

If wget or cups is not available (some old AIX servers can not connect to a https server...), the good old binary is used without encryption :(

I can document it somewhere if anyone is interested.
It's not that complicated.


For the Windows clients we also use https as much as possible.

I have some patches for the Powershell client that allows for testing a new XML file. We want to manage the XML files centrally and want to avoid mistakes that can disable the client. So this allows us to test a new config file before overwriting the old one.
I also added the ping test command for this and did some changes so data can be send to multiple Xymon servers.
I will create some patches and send them to the mailinglist.


Stef