Monitoring network traffic
list Rolf Schrittenlocher
Hi, first thanks to all contributing to xymon and the mailing list - we profit from your work for many years up to now! Our challenge at moment is how to monitor traffic quantity in/out in order to detect suspicious activities on Solaris 10. Is there are way to do this with xymon? greetings and thanks Rolf Rolf Schrittenlocher Bibliotheksmanagementsystem IT | IT-Services (ITS) Universit?tsbibliothek Johann Christian Senckenberg Goethe-Universit?t Frankfurt | Campus Bockenheim Zentralbibliothek | Freimannplatz 1 60325 Frankfurt am Main | GERMANY Telefon Sammelnummer +49 (0)69 798 28830 Telefon pers?nlich +49 (0)69 798 28908 E-Mail: user-64314bfd1eb5@xymon.invalid E-Mail (pers?nlich) user-c8b69be9a15a@xymon.invalid Website: https://www.ub.uni-frankfurt.de
list Axel Beckert
Hi Rolf, Schrittenlocher, Rolf schrieb am Thu, Apr 04, 2024 at 07:45:58AM +0000:
▸
Our challenge at moment is how to monitor traffic quantity in/out in order to detect suspicious activities on Solaris 10. Is there are way to do this with xymon?
Definitely. ;-) For our own use (in a university, too :-) and published via Debian's hobbit-plugins package, I've written a plugin simply called "net" which can check many network interface characteristics including monitoring network traffic (calculating bytes/second average from the rx/tx difference of 10 seconds), but so far it's just for Linux and uses common Linux commandline tools and /proc/ links: https://salsa.debian.org/debian/hobbit-plugins/-/blob/master/src/usr/lib/xymon/client/ext/net (It also uses the Hobbit.pm Perl module from the same package: https://salsa.debian.org/debian/hobbit-plugins/-/blob/master/src/usr/share/perl5/Hobbit.pm) It though shouldn't be too hard to adapt it to some Solaris commandline tools and their output. I'm just not sure how to convert the /proc/ stuff. Maybe there's a Linux compat mode like in FreeBSD? (Haven't touched any Solaris for like 20 years or so, back when I was a student.) Regards, Axel -- PGP: 2FF9CD59612616B5 /~\ Plain Text Ribbon Campaign, http://arc.pasp.de/ Mail: user-bc188e45dae4@xymon.invalid \ / Gegen HTML in E-Mails und Usenet Mail+Jabber: user-0064bde8d49d@xymon.invalid X https://axel.beckert.ch/ / \ I love long mails: https://email.is-not-s.ms/
list Rolf Schrittenlocher
Hi, thanks Axel. I just saw that "trends" shows network traffic. So the data is already collected and available on the server. xymon server is Linux, only the clients are Solaris. So someone can tell me how I can access the data either with a client script or on server side? kind regards
▸
Rolf Rolf Schrittenlocher Bibliotheksmanagementsystem IT | IT-Services (ITS) Universit?tsbibliothek Johann Christian Senckenberg Goethe-Universit?t Frankfurt | Campus Bockenheim Zentralbibliothek | Freimannplatz 1 60325 Frankfurt am Main | GERMANY Telefon Sammelnummer +49 (0)69 798 28830 Telefon pers?nlich +49 (0)69 798 28908 E-Mail: user-64314bfd1eb5@xymon.invalid E-Mail (pers?nlich) user-c8b69be9a15a@xymon.invalid Website: https://www.ub.uni-frankfurt.de Von: Axel Beckert <user-bc188e45dae4@xymon.invalid> Gesendet: Donnerstag, 4. April 2024 10:17 An: Schrittenlocher, Rolf Cc: Xymon at xymon.com Betreff: Re: [Xymon] Monitoring network traffic Hi Rolf, Schrittenlocher, Rolf schrieb am Thu, Apr 04, 2024 at 07:45:58AM +0000:
Our challenge at moment is how to monitor traffic quantity in/out in order to detect suspicious activities on Solaris 10. Is there are way to do this with xymon?
Definitely. ;-) For our own use (in a university, too :-) and published via Debian's hobbit-plugins package, I've written a plugin simply called "net" which can check many network interface characteristics including monitoring network traffic (calculating bytes/second average from the rx/tx difference of 10 seconds), but so far it's just for Linux and uses common Linux commandline tools and /proc/ links: https://salsa.debian.org/debian/hobbit-plugins/-/blob/master/src/usr/lib/xymon/client/ext/net (It also uses the Hobbit.pm Perl module from the same package: https://salsa.debian.org/debian/hobbit-plugins/-/blob/master/src/usr/share/perl5/Hobbit.pm) It though shouldn't be too hard to adapt it to some Solaris commandline tools and their output. I'm just not sure how to convert the /proc/ stuff. Maybe there's a Linux compat mode like in FreeBSD? (Haven't touched any Solaris for like 20 years or so, back when I was a student.) Regards, Axel -- PGP: 2FF9CD59612616B5 /~\ Plain Text Ribbon Campaign, http://arc.pasp.de/ Mail: user-bc188e45dae4@xymon.invalid \ / Gegen HTML in E-Mails und Usenet Mail+Jabber: user-0064bde8d49d@xymon.invalid X https://axel.beckert.ch/ / \ I love long mails: https://email.is-not-s.ms/
list Axel Beckert
Hi Rolf, Schrittenlocher, Rolf schrieb am Thu, Apr 04, 2024 at 08:29:54AM +0000:
▸
I just saw that "trends" shows network traffic. So the data is already collected and available on the server.
Yes, that data comes from the generic data collection (like process list, load, uptime, etc.) each client sends. There's just no alerting on traffic thresholds possible. That's one of the metrics for which my plugin can warn or alert (with measurements and comparisons done on the client side, though).
▸
So someone can tell me how I can access the data either with a client script or on server side?
Sorry, not out of my mind. I mostly know how to parse hosts.cfg and extract parameters and flags from there. The man page xymon(1) shows quite some ways to extract data from the server, except that I was not able to extract anything useful related to trends, netstat or ifstat. An example of how to work with server data might be our ircbot plugin at https://salsa.debian.org/debian/hobbit-plugins/-/blob/master/src/usr/lib/xymon/server/ext/ircbot But it also just uses "xymoncmd xymon xymondboard" and "xymoncmd xymon query" to fetch data from the server and that one doesn't seem to work with data or trends. The only way I currently see is to use the "xymoncmd xymon clientlog $hostname" command which fetches the latest raw client message including e.g. the "ifconfig" output. It also has a "netstat" section which e.g. looks like this: ---8<--- ? [netstat] Ip: Forwarding: 1 867497 total packets received 0 forwarded 0 incoming packets discarded 853230 incoming packets delivered 835141 requests sent out 225 outgoing packets dropped Icmp: 21635 ICMP messages received 2 input ICMP message failed ICMP input histogram: destination unreachable: 650 echo requests: 20985 92359 ICMP messages sent 0 ICMP messages failed ICMP output histogram: destination unreachable: 651 echo requests: 70723 echo replies: 20985 IcmpMsg: InType3: 650 InType8: 20985 OutType0: 20985 OutType3: 651 OutType8: 70723 Tcp: 23811 active connection openings 2102 passive connection openings 4 failed connection attempts 1911 connection resets received 21 connections established 1007911 segments received 1268414 segments sent out 176 segments retransmitted 0 bad segments received 853 resets sent Udp: 53387 packets received 649 packets to unknown port received 0 packet receive errors 53372 packets sent 0 receive buffer errors 0 send buffer errors IgnoredMulti: 5414 UdpLite: TcpExt: 1 resets received for embryonic SYN_RECV sockets 12258 TCP sockets finished time wait in fast timer 17779 delayed acks sent 16 delayed acks further delayed because of locked socket Quick ack mode was activated 96 times 71063 packet headers predicted 142582 acknowledgments not containing data payload received 546613 predicted acknowledgments TCPSackRecovery: 43 Detected reordering 2106 times using SACK Detected reordering 36 times using time stamp 2 congestion windows fully recovered without slow start 35 congestion windows partially recovered using Hoe heuristic TCPDSACKUndo: 4 1 congestion windows recovered without slow start after partial ack TCPLostRetransmit: 69 67 fast retransmits 1 retransmits in slow start TCPTimeouts: 87 TCPLossProbes: 26 TCPLossProbeRecovery: 4 TCPBacklogCoalesce: 2432 TCPDSACKOldSent: 96 TCPDSACKRecv: 53 120 connections reset due to unexpected data 13 connections reset due to early user close 7 connections aborted due to timeout TCPDSACKIgnoredNoUndo: 43 TCPSackShifted: 194 TCPSackMerged: 34 TCPSackShiftFallback: 4424 TCPRcvCoalesce: 66342 TCPOFOQueue: 352 TCPChallengeACK: 1 TCPAutoCorking: 28376 TCPFromZeroWindowAdv: 32 TCPToZeroWindowAdv: 32 TCPWantZeroWindowAdv: 323 TCPSynRetrans: 21 TCPOrigDataSent: 1019410 TCPHystartTrainDetect: 656 TCPHystartTrainCwnd: 50284 TCPACKSkippedSynRecv: 11 TCPWinProbe: 1 TCPKeepAlive: 26 TCPDelivered: 1042042 TCPAckCompressed: 109 TcpTimeoutRehash: 80 TcpDuplicateDataRehash: 15 TCPDSACKRecvSegs: 63 IpExt: InMcastPkts: 1579 OutMcastPkts: 4 InBcastPkts: 5414 InOctets: 217038821 OutOctets: 653115273 InMcastOctets: 50528 OutMcastOctets: 160 InBcastOctets: 1360064 InNoECTPkts: 909738 MPTcpExt: Sctp: 0 Current Associations 0 Active Associations 0 Passive Associations 0 Number of Aborteds 0 Number of Graceful Terminations 0 Number of Out of Blue packets 0 Number of Packets with invalid Checksum 0 Number of control chunks sent 0 Number of ordered chunks sent 0 Number of Unordered chunks sent 0 Number of control chunks received 0 Number of ordered chunks received 0 Number of Unordered chunks received 0 Number of messages fragmented 0 Number of messages reassembled 0 Number of SCTP packets sent 0 Number of SCTP packets received [?] ? --->8--- But you would need to parse the data interesting for you out of this yourself. Hope this helps nevertheless.
▸
Regards, Axel -- PGP: 2FF9CD59612616B5 /~\ Plain Text Ribbon Campaign, http://arc.pasp.de/ Mail: user-bc188e45dae4@xymon.invalid \ / Gegen HTML in E-Mails und Usenet Mail+Jabber: user-0064bde8d49d@xymon.invalid X https://axel.beckert.ch/ / \ I love long mails: https://email.is-not-s.ms/
list Josh Luthman
The clientlog includes [netstat] which has a snapshot of activity in text The trends puts it in a pretty graph stored in rrd. On Thu, Apr 4, 2024 at 4:30?AM Schrittenlocher, Rolf <
▸
user-c8b69be9a15a@xymon.invalid> wrote:
Hi, thanks Axel. I just saw that "trends" shows network traffic. So the data is already collected and available on the server. xymon server is Linux, only the clients are Solaris. So someone can tell me how I can access the data either with a client script or on server side? kind regards Rolf Rolf Schrittenlocher Bibliotheksmanagementsystem IT | IT-Services (ITS) Universit?tsbibliothek Johann Christian Senckenberg Goethe-Universit?t Frankfurt | Campus Bockenheim Zentralbibliothek | Freimannplatz 1 60325 Frankfurt am Main | GERMANY Telefon Sammelnummer +49 (0)69 798 28830 Telefon pers?nlich +49 (0)69 798 28908 E-Mail: user-64314bfd1eb5@xymon.invalid E-Mail (pers?nlich) user-c8b69be9a15a@xymon.invalid Website: https://www.ub.uni-frankfurt.de *Von:* Axel Beckert <user-bc188e45dae4@xymon.invalid> *Gesendet:* Donnerstag, 4. April 2024 10:17 *An:* Schrittenlocher, Rolf *Cc:* Xymon at xymon.com *Betreff:* Re: [Xymon] Monitoring network traffic Hi Rolf, Schrittenlocher, Rolf schrieb am Thu, Apr 04, 2024 at 07:45:58AM +0000:Our challenge at moment is how to monitor traffic quantity in/out in order to detect suspicious activities on Solaris 10. Is there are way to do this with xymon?Definitely. ;-) For our own use (in a university, too :-) and published via Debian's hobbit-plugins package, I've written a plugin simply called "net" which can check many network interface characteristics including monitoring network traffic (calculating bytes/second average from the rx/tx difference of 10 seconds), but so far it's just for Linux and uses common Linux commandline tools and /proc/ links: https://salsa.debian.org/debian/hobbit-plugins/-/blob/master/src/usr/lib/xymon/client/ext/net (It also uses the Hobbit.pm Perl module from the same package: https://salsa.debian.org/debian/hobbit-plugins/-/blob/master/src/usr/share/perl5/Hobbit.pm ) It though shouldn't be too hard to adapt it to some Solaris commandline tools and their output. I'm just not sure how to convert the /proc/ stuff. Maybe there's a Linux compat mode like in FreeBSD? (Haven't touched any Solaris for like 20 years or so, back when I was a student.) Regards, Axel -- PGP: 2FF9CD59612616B5 /~\ Plain Text Ribbon Campaign, http://arc.pasp.de/ Mail: user-bc188e45dae4@xymon.invalid \ / Gegen HTML in E-Mails und Usenet Mail+Jabber: user-0064bde8d49d@xymon.invalid X https://axel.beckert.ch/ / \ I love long mails: https://email.is-not-s.ms/
list Norbert Kriegenburg
Hi,
I created a server side script for all the *nix servers where I extract the
network info from the clientlog.
The script identifies all server with a ssh column (this is clearly a *nix
server) and then loops over all these targets to create a "nic" column with
interface info.
Nothing to configure especially, a new *nic server will be automatically
identified and get the column with detailed info and some graphs.
Some snippets to get the idea:
# grab all client info
get_all_info(){
$XYMONBIN localhost "clientlog $TARGET"
}
ALLINFO=`get_all_info`
##################################################
# grab the nic details
get_nic_info(){
echo "$ALLINFO" | \
$NAWK '/^\[ifconfig/,/^\[route/' | \
$GREP -v "^\["
}
##################################################
# grab the route
get_route_info(){
echo "$ALLINFO" | \
$NAWK '/^\[route/,/^\[netstat/' | \
$GREP -v "^\["
}
##################################################
# grab the ports
get_ports_info(){
ALLPORTS=`echo "$ALLINFO" | \
$NAWK '/^\[ports/,/^\[ifstat/' | \
$GREP -v "^\["`
PORTSTATUS=`echo "$ALLPORTS" | \
$NAWK '/^tcp/{print $NF}' | \
$SORT -u`
for stat in $PORTSTATUS
do
NUM=`echo "$ALLPORTS" | \
$NAWK 'BEGIN{i=0}
/'$stat'/{i++};BEGIN{i=0}
END{print i}'`
echo "tcp ports in status $stat: $NUM"
done
}
# create the output to send to xymon
echo "<h4>interface info</h4>"
get_nic_info
echo "<h4>route info</h4>"
get_route_info
echo "<h4>active tcp connections</h4>"
get_ports_info
showgraph ifstat_kB
All these data are then send to the xymon server daemon and create a nic
column.
A complete run over 500 servers will take approx. 60 secs (but you can run
more scripts in parallel if needed).
HTH
Norbert
Am Do., 4. Apr. 2024 um 19:21 Uhr schrieb Josh Luthman <
user-4c45a83f15cb@xymon.invalid>:
▸
The clientlog includes [netstat] which has a snapshot of activity in text The trends puts it in a pretty graph stored in rrd. On Thu, Apr 4, 2024 at 4:30?AM Schrittenlocher, Rolf < user-c8b69be9a15a@xymon.invalid> wrote:Hi, thanks Axel. I just saw that "trends" shows network traffic. So the data is already collected and available on the server. xymon server is Linux, only the clients are Solaris. So someone can tell me how I can access the data either with a client script or on server side? kind regards Rolf Rolf Schrittenlocher Bibliotheksmanagementsystem IT | IT-Services (ITS) Universit?tsbibliothek Johann Christian Senckenberg Goethe-Universit?t Frankfurt | Campus Bockenheim Zentralbibliothek | Freimannplatz 1 60325 Frankfurt am Main | GERMANY Telefon Sammelnummer +49 (0)69 798 28830 Telefon pers?nlich +49 (0)69 798 28908 E-Mail: user-64314bfd1eb5@xymon.invalid E-Mail (pers?nlich) user-c8b69be9a15a@xymon.invalid Website: https://www.ub.uni-frankfurt.de *Von:* Axel Beckert <user-bc188e45dae4@xymon.invalid> *Gesendet:* Donnerstag, 4. April 2024 10:17 *An:* Schrittenlocher, Rolf *Cc:* Xymon at xymon.com *Betreff:* Re: [Xymon] Monitoring network traffic Hi Rolf, Schrittenlocher, Rolf schrieb am Thu, Apr 04, 2024 at 07:45:58AM +0000:Our challenge at moment is how to monitor traffic quantity in/out in order to detect suspicious activities on Solaris 10. Is there are way to do this with xymon?Definitely. ;-) For our own use (in a university, too :-) and published via Debian's hobbit-plugins package, I've written a plugin simply called "net" which can check many network interface characteristics including monitoring network traffic (calculating bytes/second average from the rx/tx difference of 10 seconds), but so far it's just for Linux and uses common Linux commandline tools and /proc/ links: https://salsa.debian.org/debian/hobbit-plugins/-/blob/master/src/usr/lib/xymon/client/ext/net (It also uses the Hobbit.pm Perl module from the same package: https://salsa.debian.org/debian/hobbit-plugins/-/blob/master/src/usr/share/perl5/Hobbit.pm ) It though shouldn't be too hard to adapt it to some Solaris commandline tools and their output. I'm just not sure how to convert the /proc/ stuff. Maybe there's a Linux compat mode like in FreeBSD? (Haven't touched any Solaris for like 20 years or so, back when I was a student.) Regards, Axel -- PGP: 2FF9CD59612616B5 /~\ Plain Text Ribbon Campaign, http://arc.pasp.de/ Mail: user-bc188e45dae4@xymon.invalid \ / Gegen HTML in E-Mails und Usenet Mail+Jabber: user-0064bde8d49d@xymon.invalid X https://axel.beckert.ch/ / \ I love long mails: https://email.is-not-s.ms/
list Rolf Schrittenlocher
Hi, @Josh : Yes I saw it, I hoped there's an easy way to reuse the data used for the trends presentation @Norbert : Thanks's a lot that helps a lot. I'll adapt it to our needs
▸
Kind regards Rolf Rolf Schrittenlocher Bibliotheksmanagementsystem IT | IT-Services (ITS) Universit?tsbibliothek Johann Christian Senckenberg Goethe-Universit?t Frankfurt | Campus Bockenheim Zentralbibliothek | Freimannplatz 1 60325 Frankfurt am Main | GERMANY Telefon Sammelnummer +49 (0)69 798 28830 Telefon pers?nlich +49 (0)69 798 28908 E-Mail: user-64314bfd1eb5@xymon.invalid E-Mail (pers?nlich) user-c8b69be9a15a@xymon.invalid Website: https://www.ub.uni-frankfurt.de Von: nor krie <user-ff2afb5e635f@xymon.invalid> Gesendet: Donnerstag, 4. April 2024 23:27 An: Josh Luthman Cc: Schrittenlocher, Rolf; Xymon at xymon.com Betreff: Re: [Xymon] Monitoring network traffic Hi, I created a server side script for all the *nix servers where I extract the network info from the clientlog. The script identifies all server with a ssh column (this is clearly a *nix server) and then loops over all these targets to create a "nic" column with interface info. Nothing to configure especially, a new *nic server will be automatically identified and get the column with detailed info and some graphs. Some snippets to get the idea: # grab all client info get_all_info(){ $XYMONBIN localhost "clientlog $TARGET" } ALLINFO=`get_all_info` ################################################## # grab the nic details get_nic_info(){ echo "$ALLINFO" | \ $NAWK '/^\[ifconfig/,/^\[route/' | \ $GREP -v "^\[" } ################################################## # grab the route get_route_info(){ echo "$ALLINFO" | \ $NAWK '/^\[route/,/^\[netstat/' | \ $GREP -v "^\[" } ################################################## # grab the ports get_ports_info(){ ALLPORTS=`echo "$ALLINFO" | \ $NAWK '/^\[ports/,/^\[ifstat/' | \ $GREP -v "^\["` PORTSTATUS=`echo "$ALLPORTS" | \ $NAWK '/^tcp/{print $NF}' | \ $SORT -u` for stat in $PORTSTATUS do NUM=`echo "$ALLPORTS" | \ $NAWK 'BEGIN{i=0} /'$stat'/{i++};BEGIN{i=0} END{print i}'` echo "tcp ports in status $stat: $NUM" done } # create the output to send to xymon echo "<h4>interface info</h4>" get_nic_info echo "<h4>route info</h4>" get_route_info echo "<h4>active tcp connections</h4>" get_ports_info showgraph ifstat_kB All these data are then send to the xymon server daemon and create a nic column. A complete run over 500 servers will take approx. 60 secs (but you can run more scripts in parallel if needed). HTH Norbert
Am Do., 4. Apr. 2024 um 19:21 Uhr schrieb Josh Luthman <user-4c45a83f15cb@xymon.invalid<mailto:user-4c45a83f15cb@xymon.invalid>>:
▸
The clientlog includes [netstat] which has a snapshot of activity in text
The trends puts it in a pretty graph stored in rrd.
On Thu, Apr 4, 2024 at 4:30?AM Schrittenlocher, Rolf <user-c8b69be9a15a@xymon.invalid<mailto:user-c8b69be9a15a@xymon.invalid>> wrote:
Hi,
thanks Axel. I just saw that "trends" shows network traffic. So the data is already collected and available on the server. xymon server is Linux, only the clients are Solaris. So someone can tell me how I can access the data either with a client script or on server side?
kind regards
Rolf
Rolf Schrittenlocher
Bibliotheksmanagementsystem IT | IT-Services (ITS)
Universit?tsbibliothek Johann Christian Senckenberg
Goethe-Universit?t Frankfurt | Campus Bockenheim
Zentralbibliothek | Freimannplatz 1
60325 Frankfurt am Main | GERMANY
Telefon Sammelnummer +49 (0)69 798 28830
Telefon pers?nlich +49 (0)69 798 28908
E-Mail: user-64314bfd1eb5@xymon.invalid<mailto:user-64314bfd1eb5@xymon.invalid> E-Mail (pers?nlich) user-c8b69be9a15a@xymon.invalid<mailto:user-c8b69be9a15a@xymon.invalid> Website: https://www.ub.uni-frankfurt.de
▸
Von: Axel Beckert <user-bc188e45dae4@xymon.invalid<mailto:user-bc188e45dae4@xymon.invalid>>
Gesendet: Donnerstag, 4. April 2024 10:17
An: Schrittenlocher, Rolf
Cc: Xymon at xymon.com<mailto:Xymon at xymon.com>
Betreff: Re: [Xymon] Monitoring network traffic
Hi Rolf,
Schrittenlocher, Rolf schrieb am Thu, Apr 04, 2024 at 07:45:58AM +0000:Our challenge at moment is how to monitor traffic quantity in/out in order to detect suspicious activities on Solaris 10. Is there are way to do this with xymon?
Definitely. ;-) For our own use (in a university, too :-) and published via Debian's hobbit-plugins package, I've written a plugin simply called "net" which can check many network interface characteristics including monitoring network traffic (calculating bytes/second average from the rx/tx difference of 10 seconds), but so far it's just for Linux and uses common Linux commandline tools and /proc/ links: https://salsa.debian.org/debian/hobbit-plugins/-/blob/master/src/usr/lib/xymon/client/ext/net (It also uses the Hobbit.pm Perl module from the same package: https://salsa.debian.org/debian/hobbit-plugins/-/blob/master/src/usr/share/perl5/Hobbit.pm) It though shouldn't be too hard to adapt it to some Solaris commandline tools and their output. I'm just not sure how to convert the /proc/ stuff. Maybe there's a Linux compat mode like in FreeBSD? (Haven't touched any Solaris for like 20 years or so, back when I was a student.) Regards, Axel -- PGP: 2FF9CD59612616B5 /~\ Plain Text Ribbon Campaign, http://arc.pasp.de/
Mail: user-bc188e45dae4@xymon.invalid<mailto:user-bc188e45dae4@xymon.invalid> \ / Gegen HTML in E-Mails und Usenet Mail+Jabber: user-0064bde8d49d@xymon.invalid<mailto:user-0064bde8d49d@xymon.invalid> X https://axel.beckert.ch/ / \ I love long mails: https://email.is-not-s.ms/
list Jeremy Ruffer
Hi Rolf, You could try using rrdfetch to get the data that Trends uses. HTH Jeremy
▸
------ Original Message ------
From: "Schrittenlocher, Rolf" <user-c8b69be9a15a@xymon.invalid>
To: "nor krie" <user-ff2afb5e635f@xymon.invalid>; "Josh Luthman" <user-4c45a83f15cb@xymon.invalid>
Cc: "Xymon at xymon.com" <Xymon at xymon.com>
Sent: 05/04/2024 05:32:01
Subject: Re: [Xymon] Monitoring network traffic
Hi, @Josh : Yes I saw it, I hoped there's an easy way to reuse the data used for the trends presentation @Norbert : Thanks's a lot that helps a lot. I'll adapt it to our needs Kind regards Rolf Rolf Schrittenlocher Bibliotheksmanagementsystem IT | IT-Services (ITS) Universit?tsbibliothek Johann Christian Senckenberg Goethe-Universit?t Frankfurt | Campus Bockenheim Zentralbibliothek | Freimannplatz 1 60325 Frankfurt am Main | GERMANY Telefon Sammelnummer +49 (0)69 798 28830 Telefon pers?nlich +49 (0)69 798 28908 E-Mail: user-64314bfd1eb5@xymon.invalid E-Mail (pers?nlich) user-c8b69be9a15a@xymon.invalid Website: https://www.ub.uni-frankfurt.de Von: nor krie <user-ff2afb5e635f@xymon.invalid> Gesendet: Donnerstag, 4. April 2024 23:27 An: Josh Luthman Cc: Schrittenlocher, Rolf; Xymon at xymon.com Betreff: Re: [Xymon] Monitoring network traffic Hi, I created a server side script for all the *nix servers where I extract the network info from the clientlog. The script identifies all server with a ssh column (this is clearly a *nix server) and then loops over all these targets to create a "nic" column with interface info. Nothing to configure especially, a new *nic server will be automatically identified and get the column with detailed info and some graphs. Some snippets to get the idea: # grab all client info get_all_info(){ $XYMONBIN localhost "clientlog $TARGET" } ALLINFO=`get_all_info` ################################################## # grab the nic details get_nic_info(){ echo "$ALLINFO" | \ $NAWK '/^\[ifconfig/,/^\[route/' | \ $GREP -v "^\[" } ################################################## # grab the route get_route_info(){ echo "$ALLINFO" | \ $NAWK '/^\[route/,/^\[netstat/' | \ $GREP -v "^\[" } ################################################## # grab the ports get_ports_info(){ ALLPORTS=`echo "$ALLINFO" | \ $NAWK '/^\[ports/,/^\[ifstat/' | \ $GREP -v "^\["` PORTSTATUS=`echo "$ALLPORTS" | \ $NAWK '/^tcp/{print $NF}' | \ $SORT -u` for stat in $PORTSTATUS do NUM=`echo "$ALLPORTS" | \ $NAWK 'BEGIN{i=0} /'$stat'/{i++};BEGIN{i=0} END{print i}'` echo "tcp ports in status $stat: $NUM" done } # create the output to send to xymon echo "<h4>interface info</h4>" get_nic_info echo "<h4>route info</h4>" get_route_info echo "<h4>active tcp connections</h4>" get_ports_info showgraph ifstat_kB All these data are then send to the xymon server daemon and create a nic column. A complete run over 500 servers will take approx. 60 secs (but you can run more scripts in parallel if needed). HTH Norbert Am Do., 4. Apr. 2024 um 19:21 Uhr schrieb Josh Luthman <user-4c45a83f15cb@xymon.invalid>:The clientlog includes [netstat] which has a snapshot of activity in text The trends puts it in a pretty graph stored in rrd. On Thu, Apr 4, 2024 at 4:30?AM Schrittenlocher, Rolf <user-c8b69be9a15a@xymon.invalid> wrote:Hi, thanks Axel. I just saw that "trends" shows network traffic. So the data is already collected and available on the server. xymon server is Linux, only the clients are Solaris. So someone can tell me how I can access the data either with a client script or on server side? kind regards Rolf Rolf Schrittenlocher Bibliotheksmanagementsystem IT | IT-Services (ITS) Universit?tsbibliothek Johann Christian Senckenberg Goethe-Universit?t Frankfurt | Campus Bockenheim Zentralbibliothek | Freimannplatz 1 60325 Frankfurt am Main | GERMANY Telefon Sammelnummer +49 (0)69 798 28830 Telefon pers?nlich +49 (0)69 798 28908 E-Mail: user-64314bfd1eb5@xymon.invalid E-Mail (pers?nlich) user-c8b69be9a15a@xymon.invalid Website: https://www.ub.uni-frankfurt.de Von: Axel Beckert <user-bc188e45dae4@xymon.invalid> Gesendet: Donnerstag, 4. April 2024 10:17 An: Schrittenlocher, Rolf
Cc:Xymon at xymon.com
▸
Betreff: Re: [Xymon] Monitoring network traffic Hi Rolf, Schrittenlocher, Rolf schrieb am Thu, Apr 04, 2024 at 07:45:58AM +0000:Our challenge at moment is how to monitor traffic quantity in/out in order to detect suspicious activities on Solaris 10. Is there are way to do this with xymon?Definitely. ;-) For our own use (in a university, too :-) and published via Debian's hobbit-plugins package, I've written a plugin simply called "net" which can check many network interface characteristics including monitoring network traffic (calculating bytes/second average from the rx/tx difference of 10 seconds), but so far it's just for Linux and uses common Linux commandline tools and /proc/ links: https://salsa.debian.org/debian/hobbit-plugins/-/blob/master/src/usr/lib/xymon/client/ext/net (It also uses the Hobbit.pm Perl module from the same package: https://salsa.debian.org/debian/hobbit-plugins/-/blob/master/src/usr/share/perl5/Hobbit.pm) It though shouldn't be too hard to adapt it to some Solaris commandline tools and their output. I'm just not sure how to convert the /proc/ stuff. Maybe there's a Linux compat mode like in FreeBSD? (Haven't touched any Solaris for like 20 years or so, back when I was a student.) Regards, Axel -- PGP: 2FF9CD59612616B5 /~\ Plain Text Ribbon Campaign, http://arc.pasp.de/ <http://arc.pasp.de/>; Mail: user-bc188e45dae4@xymon.invalid \ / Gegen HTML in E-Mails und Usenet Mail+Jabber: user-0064bde8d49d@xymon.invalid X https://axel.beckert.ch/ / \ I love long mails: https://email.is-not-s.ms/
list Jeremy Laidman
Check out the DS option in analysis.cfg. This can perform a threshold operation on an RRD file value. J
▸
On Fri, 5 Apr 2024, 19:46 Jeremy Ruffer, <user-6d8e227afca3@xymon.invalid> wrote:
Hi Rolf, You could try using rrdfetch to get the data that Trends uses. HTH Jeremy ------ Original Message ------ From: "Schrittenlocher, Rolf" <user-c8b69be9a15a@xymon.invalid> To: "nor krie" <user-ff2afb5e635f@xymon.invalid>; "Josh Luthman" < user-4c45a83f15cb@xymon.invalid> Cc: "Xymon at xymon.com" <Xymon at xymon.com> Sent: 05/04/2024 05:32:01 Subject: Re: [Xymon] Monitoring network traffic Hi, @Josh : Yes I saw it, I hoped there's an easy way to reuse the data used for the trends presentation @Norbert : Thanks's a lot that helps a lot. I'll adapt it to our needs Kind regards Rolf Rolf Schrittenlocher Bibliotheksmanagementsystem IT | IT-Services (ITS) Universit?tsbibliothek Johann Christian Senckenberg Goethe-Universit?t Frankfurt | Campus Bockenheim Zentralbibliothek | Freimannplatz 1 60325 Frankfurt am Main | GERMANY Telefon Sammelnummer +49 (0)69 798 28830 Telefon pers?nlich +49 (0)69 798 28908 E-Mail: user-64314bfd1eb5@xymon.invalid E-Mail (pers?nlich) user-c8b69be9a15a@xymon.invalid Website: https://www.ub.uni-frankfurt.de *Von:* nor krie <user-ff2afb5e635f@xymon.invalid> *Gesendet:* Donnerstag, 4. April 2024 23:27 *An:* Josh Luthman *Cc:* Schrittenlocher, Rolf; Xymon at xymon.com *Betreff:* Re: [Xymon] Monitoring network traffic Hi, I created a server side script for all the *nix servers where I extract the network info from the clientlog. The script identifies all server with a ssh column (this is clearly a *nix server) and then loops over all these targets to create a "nic" column with interface info. Nothing to configure especially, a new *nic server will be automatically identified and get the column with detailed info and some graphs. Some snippets to get the idea: # grab all client info get_all_info(){ $XYMONBIN localhost "clientlog $TARGET" } ALLINFO=`get_all_info` ################################################## # grab the nic details get_nic_info(){ echo "$ALLINFO" | \ $NAWK '/^\[ifconfig/,/^\[route/' | \ $GREP -v "^\[" } ################################################## # grab the route get_route_info(){ echo "$ALLINFO" | \ $NAWK '/^\[route/,/^\[netstat/' | \ $GREP -v "^\[" } ################################################## # grab the ports get_ports_info(){ ALLPORTS=`echo "$ALLINFO" | \ $NAWK '/^\[ports/,/^\[ifstat/' | \ $GREP -v "^\["` PORTSTATUS=`echo "$ALLPORTS" | \ $NAWK '/^tcp/{print $NF}' | \ $SORT -u` for stat in $PORTSTATUS do NUM=`echo "$ALLPORTS" | \ $NAWK 'BEGIN{i=0} /'$stat'/{i++};BEGIN{i=0} END{print i}'` echo "tcp ports in status $stat: $NUM" done } # create the output to send to xymon echo "<h4>interface info</h4>" get_nic_info echo "<h4>route info</h4>" get_route_info echo "<h4>active tcp connections</h4>" get_ports_info showgraph ifstat_kB All these data are then send to the xymon server daemon and create a nic column. A complete run over 500 servers will take approx. 60 secs (but you can run more scripts in parallel if needed). HTH Norbert Am Do., 4. Apr. 2024 um 19:21 Uhr schrieb Josh Luthman < user-4c45a83f15cb@xymon.invalid>:The clientlog includes [netstat] which has a snapshot of activity in text The trends puts it in a pretty graph stored in rrd. On Thu, Apr 4, 2024 at 4:30?AM Schrittenlocher, Rolf < user-c8b69be9a15a@xymon.invalid> wrote:Hi, thanks Axel. I just saw that "trends" shows network traffic. So the data is already collected and available on the server. xymon server is Linux, only the clients are Solaris. So someone can tell me how I can access the data either with a client script or on server side? kind regards Rolf Rolf Schrittenlocher Bibliotheksmanagementsystem IT | IT-Services (ITS) Universit?tsbibliothek Johann Christian Senckenberg Goethe-Universit?t Frankfurt | Campus Bockenheim Zentralbibliothek | Freimannplatz 1 60325 Frankfurt am Main | GERMANY Telefon Sammelnummer +49 (0)69 798 28830 Telefon pers?nlich +49 (0)69 798 28908 E-Mail: user-64314bfd1eb5@xymon.invalid E-Mail (pers?nlich) user-c8b69be9a15a@xymon.invalid Website: https://www.ub.uni-frankfurt.de *Von:* Axel Beckert <user-bc188e45dae4@xymon.invalid> *Gesendet:* Donnerstag, 4. April 2024 10:17 *An:* Schrittenlocher, Rolf *Cc:* Xymon at xymon.com *Betreff:* Re: [Xymon] Monitoring network traffic Hi Rolf, Schrittenlocher, Rolf schrieb am Thu, Apr 04, 2024 at 07:45:58AM +0000:Our challenge at moment is how to monitor traffic quantity in/out in order to detect suspicious activities on Solaris 10. Is there are way to do this with xymon?Definitely. ;-) For our own use (in a university, too :-) and published via Debian's hobbit-plugins package, I've written a plugin simply called "net" which can check many network interface characteristics including monitoring network traffic (calculating bytes/second average from the rx/tx difference of 10 seconds), but so far it's just for Linux and uses common Linux commandline tools and /proc/ links: https://salsa.debian.org/debian/hobbit-plugins/-/blob/master/src/usr/lib/xymon/client/ext/net (It also uses the Hobbit.pm Perl module from the same package: https://salsa.debian.org/debian/hobbit-plugins/-/blob/master/src/usr/share/perl5/Hobbit.pm ) It though shouldn't be too hard to adapt it to some Solaris commandline tools and their output. I'm just not sure how to convert the /proc/ stuff. Maybe there's a Linux compat mode like in FreeBSD? (Haven't touched any Solaris for like 20 years or so, back when I was a student.) Regards, Axel -- PGP: 2FF9CD59612616B5 /~\ Plain Text Ribbon Campaign, http://arc.pasp.de/ Mail: user-bc188e45dae4@xymon.invalid \ / Gegen HTML in E-Mails und Usenet Mail+Jabber: user-0064bde8d49d@xymon.invalid X https://axel.beckert.ch/ / \ I love long mails: https://email.is-not-s.ms/
list Rolf Schrittenlocher
Good morning, thanks, I found DS on the man page (the explanations in analysis.cfg's comments doent't show it). As far as I understand it is unfortunately not suitable: "NOTE: This rule uses the raw data value from a client to examine the rules. So this type of test is only really suitable for datasets that are of the "GAUGE" type. It cannot be used meaningfully for datasets that use "COUNTER" or "DERIVE" - e.g. the datasets that are used to capture network packet traffic - because the data stored in the RRD for COUNTER-based datasets undergo a transformation (calculation) when going into the RRD. Xymon does not have direct access to the calculated data." Bad luck, cheers
▸
Rolf Rolf Schrittenlocher Bibliotheksmanagementsystem IT | IT-Services (ITS) Universit?tsbibliothek Johann Christian Senckenberg Goethe-Universit?t Frankfurt | Campus Bockenheim Zentralbibliothek | Freimannplatz 1 60325 Frankfurt am Main | GERMANY Telefon Sammelnummer +49 (0)69 798 28830 Telefon pers?nlich +49 (0)69 798 28908 E-Mail: user-64314bfd1eb5@xymon.invalid E-Mail (pers?nlich) user-c8b69be9a15a@xymon.invalid Website: https://www.ub.uni-frankfurt.de Von: Xymon <xymon-bounces at xymon.com> im Auftrag von Jeremy Laidman <user-0608abae5e7c@xymon.invalid> Gesendet: Sonntag, 7. April 2024 11:06 An: xymon at xymon.com Betreff: Re: [Xymon] Monitoring network traffic Check out the DS option in analysis.cfg. This can perform a threshold operation on an RRD file value. J On Fri, 5 Apr 2024, 19:46 Jeremy Ruffer, <user-6d8e227afca3@xymon.invalid<mailto:user-6d8e227afca3@xymon.invalid>> wrote: Hi Rolf, You could try using rrdfetch to get the data that Trends uses. HTH Jeremy ------ Original Message ------ From: "Schrittenlocher, Rolf" <user-c8b69be9a15a@xymon.invalid<mailto:user-c8b69be9a15a@xymon.invalid>> To: "nor krie" <user-ff2afb5e635f@xymon.invalid<mailto:user-ff2afb5e635f@xymon.invalid>>; "Josh Luthman" <user-4c45a83f15cb@xymon.invalid<mailto:user-4c45a83f15cb@xymon.invalid>> Cc: "Xymon at xymon.com<mailto:Xymon at xymon.com>" <Xymon at xymon.com<mailto:Xymon at xymon.com>> Sent: 05/04/2024 05:32:01 Subject: Re: [Xymon] Monitoring network traffic Hi, @Josh : Yes I saw it, I hoped there's an easy way to reuse the data used for the trends presentation @Norbert : Thanks's a lot that helps a lot. I'll adapt it to our needs Kind regards Rolf Rolf Schrittenlocher Bibliotheksmanagementsystem IT | IT-Services (ITS) Universit?tsbibliothek Johann Christian Senckenberg Goethe-Universit?t Frankfurt | Campus Bockenheim Zentralbibliothek | Freimannplatz 1 60325 Frankfurt am Main | GERMANY Telefon Sammelnummer +49 (0)69 798 28830 Telefon pers?nlich +49 (0)69 798 28908 E-Mail: user-64314bfd1eb5@xymon.invalid<mailto:user-64314bfd1eb5@xymon.invalid> E-Mail (pers?nlich) user-c8b69be9a15a@xymon.invalid<mailto:user-c8b69be9a15a@xymon.invalid> Website: https://www.ub.uni-frankfurt.de Von: nor krie <user-ff2afb5e635f@xymon.invalid<mailto:user-ff2afb5e635f@xymon.invalid>> Gesendet: Donnerstag, 4. April 2024 23:27 An: Josh Luthman Cc: Schrittenlocher, Rolf; Xymon at xymon.com<mailto:Xymon at xymon.com> Betreff: Re: [Xymon] Monitoring network traffic Hi, I created a server side script for all the *nix servers where I extract the network info from the clientlog. The script identifies all server with a ssh column (this is clearly a *nix server) and then loops over all these targets to create a "nic" column with interface info. Nothing to configure especially, a new *nic server will be automatically identified and get the column with detailed info and some graphs. Some snippets to get the idea: # grab all client info get_all_info(){ $XYMONBIN localhost "clientlog $TARGET" } ALLINFO=`get_all_info` ################################################## # grab the nic details get_nic_info(){ echo "$ALLINFO" | \ $NAWK '/^\[ifconfig/,/^\[route/' | \ $GREP -v "^\[" } ################################################## # grab the route get_route_info(){ echo "$ALLINFO" | \ $NAWK '/^\[route/,/^\[netstat/' | \ $GREP -v "^\[" } ################################################## # grab the ports get_ports_info(){ ALLPORTS=`echo "$ALLINFO" | \ $NAWK '/^\[ports/,/^\[ifstat/' | \ $GREP -v "^\["` PORTSTATUS=`echo "$ALLPORTS" | \ $NAWK '/^tcp/{print $NF}' | \ $SORT -u` for stat in $PORTSTATUS do NUM=`echo "$ALLPORTS" | \ $NAWK 'BEGIN{i=0} /'$stat'/{i++};BEGIN{i=0} END{print i}'` echo "tcp ports in status $stat: $NUM" done } # create the output to send to xymon echo "<h4>interface info</h4>" get_nic_info echo "<h4>route info</h4>" get_route_info echo "<h4>active tcp connections</h4>" get_ports_info showgraph ifstat_kB All these data are then send to the xymon server daemon and create a nic column. A complete run over 500 servers will take approx. 60 secs (but you can run more scripts in parallel if needed). HTH Norbert Am Do., 4. Apr. 2024 um 19:21 Uhr schrieb Josh Luthman <user-4c45a83f15cb@xymon.invalid<mailto:user-4c45a83f15cb@xymon.invalid>>: The clientlog includes [netstat] which has a snapshot of activity in text The trends puts it in a pretty graph stored in rrd. On Thu, Apr 4, 2024 at 4:30?AM Schrittenlocher, Rolf <user-c8b69be9a15a@xymon.invalid<mailto:user-c8b69be9a15a@xymon.invalid>> wrote: Hi, thanks Axel. I just saw that "trends" shows network traffic. So the data is already collected and available on the server. xymon server is Linux, only the clients are Solaris. So someone can tell me how I can access the data either with a client script or on server side? kind regards Rolf Rolf Schrittenlocher Bibliotheksmanagementsystem IT | IT-Services (ITS) Universit?tsbibliothek Johann Christian Senckenberg Goethe-Universit?t Frankfurt | Campus Bockenheim Zentralbibliothek | Freimannplatz 1 60325 Frankfurt am Main | GERMANY Telefon Sammelnummer +49 (0)69 798 28830 Telefon pers?nlich +49 (0)69 798 28908 E-Mail: user-64314bfd1eb5@xymon.invalid<mailto:user-64314bfd1eb5@xymon.invalid> E-Mail (pers?nlich) user-c8b69be9a15a@xymon.invalid<mailto:user-c8b69be9a15a@xymon.invalid> Website: https://www.ub.uni-frankfurt.de Von: Axel Beckert <user-bc188e45dae4@xymon.invalid<mailto:user-bc188e45dae4@xymon.invalid>> Gesendet: Donnerstag, 4. April 2024 10:17 An: Schrittenlocher, Rolf Cc: Xymon at xymon.com<mailto:Xymon at xymon.com> Betreff: Re: [Xymon] Monitoring network traffic Hi Rolf, Schrittenlocher, Rolf schrieb am Thu, Apr 04, 2024 at 07:45:58AM +0000:
Our challenge at moment is how to monitor traffic quantity in/out in order to detect suspicious activities on Solaris 10. Is there are way to do this with xymon?
Definitely. ;-) For our own use (in a university, too :-) and published via Debian's hobbit-plugins package, I've written a plugin simply called "net" which can check many network interface characteristics including monitoring network traffic (calculating bytes/second average from the rx/tx difference of 10 seconds), but so far it's just for Linux and uses common Linux commandline tools and /proc/ links: https://salsa.debian.org/debian/hobbit-plugins/-/blob/master/src/usr/lib/xymon/client/ext/net (It also uses the Hobbit.pm Perl module from the same package: https://salsa.debian.org/debian/hobbit-plugins/-/blob/master/src/usr/share/perl5/Hobbit.pm) It though shouldn't be too hard to adapt it to some Solaris commandline tools and their output. I'm just not sure how to convert the /proc/ stuff. Maybe there's a Linux compat mode like in FreeBSD? (Haven't touched any Solaris for like 20 years or so, back when I was a student.) Regards, Axel -- PGP: 2FF9CD59612616B5 /~\ Plain Text Ribbon Campaign, http://arc.pasp.de/ Mail: user-bc188e45dae4@xymon.invalid<mailto:user-bc188e45dae4@xymon.invalid> \ / Gegen HTML in E-Mails und Usenet Mail+Jabber: user-0064bde8d49d@xymon.invalid<mailto:user-0064bde8d49d@xymon.invalid> X https://axel.beckert.ch/ / \ I love long mails: https://email.is-not-s.ms/