Technical questions regarding message flow 🔗 link

I have some technical questions in regard to how the actual message flow between server / clients take place.

The first questions relate to network traffic.  My intent is to monitor a large number of clients (getting close to 9,000) so this is going to be a question I need to fully understand before I go too far. What does the message look like and how big is it (excluding message text) ?  Is it XML?  How is the connection made between the client and host?  Are sockets made? How does the server handle large volumes of connections.  One thing I have read is that any bottle necks will be more likely with disk I/O than network traffic, is this fair?

Secondly, what is the process of message flow between client and server?  I understand how the xymonclient is configured to run at a defined interval (configured in clientlaunch.cfg), it gathers data from its logs and sends the data to the server.  The server saves this in its /data/hist path along with any rrd data.  The server uses this data to display its web pages. The next part I'm not sure I understand correctly.... The sever then sends a response to the client that contains data that instructs the client what information it wants next time (this is taken from hosts.cfg?)  The client then uses this response to determine which information to collect and send the next time the client is launched.   So, aside from the server's ping test, all traffic is initiated by the client?

Thank you for any help.

-- 
Michael Beatty

On 12 December 2012 02:17, Michael Beatty <user-4aea7c115850@xymon.invalid>wrote:

No, it's raw text.  When you send a status message from the command-line or
in a script, the content of the text is sent to the server.  There's no
encoding, no compression, no conversion, no (added) headers, nothing but
what you send.  So a shell-based Xymon client clone can be constructed
using telnet like so:

( echo "MSG"; sleep 1 ) | telnet $HOST $PORT

And the message on the wire looks like (depending on the test and message
type):

status the,host,name.conn Thu Dec 25 00:00:01 UTC 2012
bla bla
yada yada

In standard Xymon/BB messages, there's no text sent back from the server.
 A "central" mode client is different - see later.  There are different
message types, which you can read about in the man page for the "xymon"
command.

 How is the connection made between the client and host?  Are sockets made?


Yes, as per above.  Also see other examples:
http://www.xymon.com/xymon/help/xymon-tips.html#noinstall

Probably, but I can't comment with any authority.  I know that the Xymon
daemon (xymond) keeps a lot of stuff in memory, with periodic snapshots
written to disk, and so this probably improves performance significantly
compared to Big Brother.

There are potentially two parts to the client-server interaction.  A client
can have its own "ext" scripts that connect to the server and send an
update on a single client-side test.  This is not the default, and would
need to be added on top of a standard Xymon client installation.  For
instance, my systems all have checks on the SMART disk status, which can
only be done on the client side, and are not included in the standard
client data.

The rest of the data sent to the server are concatenated into one long
"client" message.  This is all driven by the "xymonclient.sh" script, which
in turn runs xymonclient-linux.sh (or whichever) and logfetch.  The client
connects to the server and sends a client data message looking like so:

client the,host,name.linux linux
[date]
Thu Dec 25 00:00:01 UTC 2012
[uname]
Linux the.host.name 2.6.16.10-1.2.3-smp x86_64
...

This is exactly what you see when you click on the "Client data available"
link at the bottom of some of the status pages (eg "conn").

Now, if the client is in "local mode", that's the end.

However, in "central mode", the client script wait for Xymon to send back
some logfetch info.  This is stored in the temporary logfetch configuration
file, typically /tmp/logfetch.<name of server>.cfg.  This contains the
sections of client-local.cfg on the server that are relevant to the client.
 You can see the logfetch configuration by catting the temporary file, or
you can fetch it yourself with the following command:

printf "[dummy]\ntesting\n" | $XYMON $XYMSRV 'client/dummyID
name,of,host.linux linux'

Each time the xymonclient.sh script runs, it launches logfetch, which uses
this temporary logfetch configuration to determine what to append to the
client message to be sent back to Xymon.  On each run, the configuration
being used is from the last time xymonclient.sh was run, which is why there
is a delay (of 5 to 10 minutes) in changes to client-local.cfg being picked
up on the client.

The client knows nothing of the contents of hosts.cfg.  (I think there's a
way it can get it with a "hostinfo" message, but I don't believe any of the
standard client scripts and binaries use this.)  The content of the client
message is pretty much governed by the commands in the xymonclient-linux.sh
(or whatever) script, and logwatch and its configuration derived from
client-local.cfg.

Well, aside from the other network tests (ssh, telnet, http, etc) and other
"ext" tests, yes that is correct.

J