problem with receiving bbwin data
list Phil Crooker
I've been having difficulties setting up the bbwin client (ver 0.13 going to xymon 4.3.10 server) running on win7 sp1, 2008r2 and win2012 boxes in central mode. I've had zero response from the bbwin-help forum for this, so please bear with me.
For background: I can't get the bbwin client to stop sending all the logs, it ignores any maxdata parameters I use. Eg:
eventlog:system:1024
It sends everything anyway. If I use log:system:1024, the bbwin client throws an error that it can't find the system log file. It does find the file without the maxdata parameter.
This aside, I do need to get this working as I have to start monitoring 10 new windows servers and have to monitor the event logs, so I can't just stop sending them as has been suggested. Yes I could do some powershell scripts and send them via bbwincmd but the bbwin client is made for this task.....
So, looking at this from the other side, the xymon server appears to be resetting the session after about 22MB of data has been sent (I know, this is ludicrous, but it is windows). Nothing in the xymond logs (except for the occasional data flooding error ("1st line client", always)); on the client side it reports it can't send the data to the xymon server.
I've set the MAXMSG_* to quite silly levels:
# ipcs
------ Shared Memory Segments --------
key shmid owner perms bytes nattch status
0x01034be7 16908288 xymon 600 102400000 2
0x02034be7 16941057 xymon 600 102400000 2
0x03034be7 16973826 xymon 600 102400000 2
0x04034be7 17006595 xymon 600 102400000 2
0x05034be7 17039364 xymon 600 262144 1
0x06034be7 17072133 xymon 600 32768 1
0x07034be7 17104902 xymon 600 102400000 2
0x08034be7 17137671 xymon 600 102400000 2
0x09034be7 17170440 xymon 600 131072 1
Anything up to and including this size has no effect on the problem.
Looking at the tcpdump stream, the bbwin client sends data normally with regular ACKs from xymond till around that 22MB mark then xymond responds with a FIN packet, then with RST packets and the session shuts down. Nothing in the packets themselves indicate what the problem is.
If anyone can help with this, please, it would be great.
thanks, Phil
Please consider the environment before printing this e-mail
This message from ORIX Australia may contain confidential and/or privileged information. If you are not the intended recipient, any use, disclosure or copying of this message (or of any attachments to it) is not authorised. If you have received this message in error, please notify the sender immediately and delete the message and any attachments from your system. Please inform the sender if you do not wish to receive further communications by email. ORIX handles personal information according to a Privacy Policy that is consistent with the National Privacy Principles. Please let us know if you would like a copy.
It is also available at www.orix.com.au<http://www.orix.com.au>;
list David Baldwin
Phil, Just found a fix that works for me :)
▸
I've been having difficulties setting up the bbwin client (ver 0.13
going to xymon 4.3.10 server) running on win7 sp1, 2008r2 and win2012
boxes in central mode. I've had zero response from the bbwin-help
forum for this, so please bear with me.
For background: I can't get the bbwin client to stop sending all the
logs, it ignores any maxdata parameters I use. Eg:
eventlog:system:1024
It sends everything anyway. If I use log:system:1024, the bbwin client
throws an error that it can't find the system log file. It does find
the file without the maxdata parameter.
One problem with the settings in client-local.cfg on the xymon server is that they only propagate to the client after a successful sending of a client message. The mechanics is that after the client message has been sent, BBWin does a shutdown on the write socket and then reads from the read socket until EOF. If the write socket is never shutdown because the server breaks the connection due to flooding, then the client-local.cfg section will never be sent to the client. I've previously attempted to manually create C:\Program Files(x86)\BBWin\tmp\clientlocal.cfg to trick it, but with limited success. Just played around with it again - with client reporting to my primary server I just got it going - previously I was reporting to 2 servers. The trick seems to be in having logfetch.status also created. I then modified my BBWin.cfg back to 2 servers and it broke again. Maybe the 2nd server is responding differently. It is undefined which server's clientlocal.cfg to believe anyway :) Swapped the order of the servers around and it seems stable. If the server just discarded all the flooding data beyond MAXMSG_* and then returned the client config section anyway maybe it could be made to work, but there may be good reasons why that would not be sensible.
▸
This aside, I do need to get this working as I have to start
monitoring 10 new windows servers and have to monitor the event logs,
so I can't just stop sending them as has been suggested. Yes I could
do some powershell scripts and send them via bbwincmd but the bbwin
client is made for this task.....
So, looking at this from the other side, the xymon server appears to
be resetting the session after about 22MB of data has been sent (I
know, this is ludicrous, but it is windows). Nothing in the xymond
logs (except for the occasional data flooding error ("1st line
client", always)); on the client side it reports it can't send the
data to the xymon server.
Depending on what auditting you have enabled, 22MB is very easy to achieve! Turn on Security event logging including success and failure and it's almost guaranteed :) David.
▸
I've set the MAXMSG_* to quite silly levels:
# ipcs
------ Shared Memory Segments --------
key shmid owner perms bytes
nattch status
0x01034be7 16908288 xymon 600 102400000 2
0x02034be7 16941057 xymon 600 102400000 2
0x03034be7 16973826 xymon 600 102400000 2
0x04034be7 17006595 xymon 600 102400000 2
0x05034be7 17039364 xymon 600 262144 1
0x06034be7 17072133 xymon 600 32768 1
0x07034be7 17104902 xymon 600 102400000 2
0x08034be7 17137671 xymon 600 102400000 2
0x09034be7 17170440 xymon 600 131072 1
Anything up to and including this size has no effect on the problem.
Looking at the tcpdump stream, the bbwin client sends data normally
with regular ACKs from xymond till around that 22MB mark then xymond
responds with a FIN packet, then with RST packets and the session
shuts down. Nothing in the packets themselves indicate what the
problem is.
If anyone can help with this, please, it would be great.
thanks, Phil
*Please consider the environment before printing this e-mail*
This message from ORIX Australia may contain confidential and/or
privileged information. If you are not the intended recipient, any
use, disclosure or copying of this message (or of any attachments to
it) is not authorised. If you have received this message in error,
please notify the sender immediately and delete the message and any
attachments from your system. Please inform the sender if you do not
wish to receive further communications by email. ORIX handles personal
information according to a Privacy Policy that is consistent with the
National Privacy Principles. Please let us know if you would like a copy.
It is also available at www.orix.com.au <http://www.orix.com.au>;
-- David Baldwin - Senior Systems Administrator (Datacentres + Networks) Information and Communication Technology Services Australian Sports Commission http://ausport.gov.au Tel 02 62147830 Fax 02 62141830 PO Box 176 Belconnen ACT 2616 user-cbbf693f2c89@xymon.invalid Leverrier Street Bruce ACT 2617 Keep up to date with what's happening in Australian sport visit http://www.ausport.gov.au This message is intended for the addressee named and may contain confidential and privileged information. If you are not the intended recipient please note that any form of distribution, copying or use of this communication or the information in it is strictly prohibited and may be unlawful. If you receive this message in error, please delete it and notify the sender.
list Carl Inglis
I'm not a C++ programmer, but if I'm reading the code correctly then the maxdata parameter is ignored in the eventlog scanning function (AgentMsgs::Run in msgs.cpp), and also in Session::Execute in EventLog.cpp Note it's a big if, and I'd be happier if someone with C++ experience could give a more reliable assessment. Regards, Carl Carl Inglis Systems Administrator Rakon UK Limited Dowsett House, Sadler Road, Lincoln LN6 3RS, United Kingdom Tel: +XX (X)XXXX XXXXXX | Fax: +XX (X) XXXX XXXXXX | Mob: +44 (0) 7786 552915 user-96685bdc864b@xymon.invalid | www.rakon.com [The Queens Awards for Enterprise 2012] [Rakon Logo] This message together with any attachments contains confidential information and may be subject to privilege. If you are not the intended recipient you may not distribute it in any way, you must notify the sender immediately and delete any copies of the message along with its attachments. Rakon UK Ltd is a limited company registered in England and Wales. Registered Office: Dowsett House, Sadler Road, Lincoln LN6 3RS Company Registration Number: 5128090. Please be aware that Rakon UK Limited may monitor email traffic data including the date, time, subject line, sender and recipients for the purposes of security and usage monitoring. Automated monitoring systems may also be applied to ascertain whether incoming/outgoing emails are likely to contain viruses, other destructive devices or inappropriate content.
▸
From: Xymon [mailto:xymon-bounces at xymon.com] On Behalf Of Phil Crooker Sent: 10 July 2013 07:24 To: xymon at xymon.com Subject: [Xymon] problem with receiving bbwin data I've been having difficulties setting up the bbwin client (ver 0.13 going to xymon 4.3.10 server) running on win7 sp1, 2008r2 and win2012 boxes in central mode. I've had zero response from the bbwin-help forum for this, so please bear with me. For background: I can't get the bbwin client to stop sending all the logs, it ignores any maxdata parameters I use. Eg: eventlog:system:1024 It sends everything anyway. If I use log:system:1024, the bbwin client throws an error that it can't find the system log file. It does find the file without the maxdata parameter. This aside, I do need to get this working as I have to start monitoring 10 new windows servers and have to monitor the event logs, so I can't just stop sending them as has been suggested. Yes I could do some powershell scripts and send them via bbwincmd but the bbwin client is made for this task..... So, looking at this from the other side, the xymon server appears to be resetting the session after about 22MB of data has been sent (I know, this is ludicrous, but it is windows). Nothing in the xymond logs (except for the occasional data flooding error ("1st line client", always)); on the client side it reports it can't send the data to the xymon server. I've set the MAXMSG_* to quite silly levels: # ipcs ------ Shared Memory Segments -------- key shmid owner perms bytes nattch status 0x01034be7 16908288 xymon 600 102400000 2 0x02034be7 16941057 xymon 600 102400000 2 0x03034be7 16973826 xymon 600 102400000 2 0x04034be7 17006595 xymon 600 102400000 2 0x05034be7 17039364 xymon 600 262144 1 0x06034be7 17072133 xymon 600 32768 1 0x07034be7 17104902 xymon 600 102400000 2 0x08034be7 17137671 xymon 600 102400000 2 0x09034be7 17170440 xymon 600 131072 1 Anything up to and including this size has no effect on the problem. Looking at the tcpdump stream, the bbwin client sends data normally with regular ACKs from xymond till around that 22MB mark then xymond responds with a FIN packet, then with RST packets and the session shuts down. Nothing in the packets themselves indicate what the problem is. If anyone can help with this, please, it would be great. thanks, Phil Please consider the environment before printing this e-mail This message from ORIX Australia may contain confidential and/or privileged information. If you are not the intended recipient, any use, disclosure or copying of this message (or of any attachments to it) is not authorised. If you have received this message in error, please notify the sender immediately and delete the message and any attachments from your system. Please inform the sender if you do not wish to receive further communications by email. ORIX handles personal information according to a Privacy Policy that is consistent with the National Privacy Principles. Please let us know if you would like a copy. It is also available at www.orix.com.au<http://www.orix.com.au>;
list Scot Kreienkamp
If the eventlog data being transmitted is too large to transmit until the clientlocal.cfg is sent back and populated in BBWin, then why not disable the eventlog client in BBWin until it successfully sends 2-3 status messages? By that time the clientlocal.cfg will be populated on the client and you can enable the eventlog reader again and everything should work fine. Scot Kreienkamp
▸
From: Xymon [mailto:xymon-bounces at xymon.com] On Behalf Of David Baldwin Sent: Wednesday, July 10, 2013 4:19 AM To: Phil Crooker Cc: xymon at xymon.com Subject: Re: [Xymon] problem with receiving bbwin data Phil, Just found a fix that works for me :) I've been having difficulties setting up the bbwin client (ver 0.13 going to xymon 4.3.10 server) running on win7 sp1, 2008r2 and win2012 boxes in central mode. I've had zero response from the bbwin-help forum for this, so please bear with me. For background: I can't get the bbwin client to stop sending all the logs, it ignores any maxdata parameters I use. Eg: eventlog:system:1024 It sends everything anyway. If I use log:system:1024, the bbwin client throws an error that it can't find the system log file. It does find the file without the maxdata parameter. One problem with the settings in client-local.cfg on the xymon server is that they only propagate to the client after a successful sending of a client message. The mechanics is that after the client message has been sent, BBWin does a shutdown on the write socket and then reads from the read socket until EOF. If the write socket is never shutdown because the server breaks the connection due to flooding, then the client-local.cfg section will never be sent to the client. I've previously attempted to manually create C:\Program Files(x86)\BBWin\tmp\clientlocal.cfg to trick it, but with limited success. Just played around with it again - with client reporting to my primary server I just got it going - previously I was reporting to 2 servers. The trick seems to be in having logfetch.status also created. I then modified my BBWin.cfg back to 2 servers and it broke again. Maybe the 2nd server is responding differently. It is undefined which server's clientlocal.cfg to believe anyway :) Swapped the order of the servers around and it seems stable. If the server just discarded all the flooding data beyond MAXMSG_* and then returned the client config section anyway maybe it could be made to work, but there may be good reasons why that would not be sensible. This aside, I do need to get this working as I have to start monitoring 10 new windows servers and have to monitor the event logs, so I can't just stop sending them as has been suggested. Yes I could do some powershell scripts and send them via bbwincmd but the bbwin client is made for this task..... So, looking at this from the other side, the xymon server appears to be resetting the session after about 22MB of data has been sent (I know, this is ludicrous, but it is windows). Nothing in the xymond logs (except for the occasional data flooding error ("1st line client", always)); on the client side it reports it can't send the data to the xymon server. Depending on what auditting you have enabled, 22MB is very easy to achieve! Turn on Security event logging including success and failure and it's almost guaranteed :) David. I've set the MAXMSG_* to quite silly levels: # ipcs ------ Shared Memory Segments -------- key shmid owner perms bytes nattch status 0x01034be7 16908288 xymon 600 102400000 2 0x02034be7 16941057 xymon 600 102400000 2 0x03034be7 16973826 xymon 600 102400000 2 0x04034be7 17006595 xymon 600 102400000 2 0x05034be7 17039364 xymon 600 262144 1 0x06034be7 17072133 xymon 600 32768 1 0x07034be7 17104902 xymon 600 102400000 2 0x08034be7 17137671 xymon 600 102400000 2 0x09034be7 17170440 xymon 600 131072 1 Anything up to and including this size has no effect on the problem. Looking at the tcpdump stream, the bbwin client sends data normally with regular ACKs from xymond till around that 22MB mark then xymond responds with a FIN packet, then with RST packets and the session shuts down. Nothing in the packets themselves indicate what the problem is. If anyone can help with this, please, it would be great. thanks, Phil Please consider the environment before printing this e-mail This message from ORIX Australia may contain confidential and/or privileged information. If you are not the intended recipient, any use, disclosure or copying of this message (or of any attachments to it) is not authorised. If you have received this message in error, please notify the sender immediately and delete the message and any attachments from your system. Please inform the sender if you do not wish to receive further communications by email. ORIX handles personal information according to a Privacy Policy that is consistent with the National Privacy Principles. Please let us know if you would like a copy. It is also available at www.orix.com.au<http://www.orix.com.au>; -- David Baldwin - Senior Systems Administrator (Datacentres + Networks) Information and Communication Technology Services Australian Sports Commission http://ausport.gov.au Tel 02 62147830 Fax 02 62141830 PO Box 176 Belconnen ACT 2616
user-cbbf693f2c89@xymon.invalid<mailto:user-cbbf693f2c89@xymon.invalid> Leverrier Street Bruce ACT 2617 Keep up to date with what's happening in Australian sport visit www.ausport.gov.au<http://www.ausport.gov.au>; This message is intended for the addressee named and may contain confidential and privileged information. If you are not the intended recipient please note that any form of distribution, copying or use of this communication or the information in it is strictly prohibited and may be unlawful. If you receive this message in error, please delete it and notify the sender. This message is intended only for the individual or entity to which it is addressed. It may contain privileged, confidential information which is exempt from disclosure under applicable laws. If you are not the intended recipient, please note that you are strictly prohibited from disseminating or distributing this information (other than to the intended recipient) or copying this information. If you have received this communication in error, please notify us immediately by e-mail or by telephone at the above number. Thank you.
list Phil Crooker
Thanks for the reply.
Before continuing, what is supposed to be in logfetch.status? and in clientversion.cfg?
So, I have one server with the msgs.dll out of the config. It has been running for many days with the client updating fine. I redid the xymon client-local.cfg, stopped bbwin, deleted \tmp\clientlocal.cfg and restarted it (as it wouldn't update), the \tmp\clientlocal.cfg now reads what the client-local has:
eventlog:application:1024
ignore BigBrotherXymonClient
eventlog:system:1025
eventlog:security:10240
I let that run a cycle, it still updates OK. I then put in the msgs.dll statement in BBWin.cfg; that registers and BBWin.log shows:
2013/07/11 10:42:55 [DEBUG]: [bbwinupdate]: can't open C:\Program Files (x86)\BBWin\etc\clientversion.cfg : The system cannot find the file specified.
2013/07/11 10:42:55 [DEBUG]: [cpu]: cpu review started
2013/07/11 10:42:55 [DEBUG]: [cpu]: cpu review ended
2013/07/11 10:42:55 [DEBUG]: [disk]: drive string A:\
2013/07/11 10:42:55 [DEBUG]: [disk]: drive string C:\
2013/07/11 10:42:55 [DEBUG]: [disk]: drive string D:\
2013/07/11 10:42:55 [DEBUG]: [disk]: drive string E:\
2013/07/11 10:42:55 [DEBUG]: [filesystem]: start AgentFileSystem::InitCentralMode
2013/07/11 10:42:55 [DEBUG]: [filesystem]: checking file C:\Program Files (x86)\BBWin\tmp\clientlocal.cfg
2013/07/11 10:42:55 [DEBUG]: [filesystem]: reading file C:\Program Files (x86)\BBWin\tmp\clientlocal.cfg
2013/07/11 10:42:55 [DEBUG]: [msgs]: start AgentMsgs::InitCentralMode
2013/07/11 10:42:55 [DEBUG]: [msgs]: checking file C:\Program Files (x86)\BBWin\tmp\clientlocal.cfg
2013/07/11 10:42:55 [DEBUG]: [msgs]: reading file C:\Program Files (x86)\BBWin\tmp\clientlocal.cfg
2013/07/11 10:42:55 [DEBUG]: [msgs]: will use maxdata size : 1024
2013/07/11 10:42:55 [DEBUG]: [msgs]: will ignore : BigBrotherXymonClient
2013/07/11 10:42:55 [DEBUG]: [msgs]: will use maxdata size : 1025
2013/07/11 10:42:55 [DEBUG]: [msgs]: will use maxdata size : 10240
and no more updates. The .\tmp\msgs.hostname.txt file is 88MB.
thanks, Phil
▸
From: Scot Kreienkamp Sent: Wednesday, 10 July 2013 10:19 PM To: David Baldwin; Phil Crooker Cc: xymon at xymon.com Subject: RE: [Xymon] problem with receiving bbwin data If the eventlog data being transmitted is too large to transmit until the clientlocal.cfg is sent back and populated in BBWin, then why not disable the eventlog client in BBWin until it successfully sends 2-3 status messages? By that time the clientlocal.cfg will be populated on the client and you can enable the eventlog reader again and everything should work fine. Scot Kreienkamp From: Xymon [mailto:xymon-bounces at xymon.com] On Behalf Of David Baldwin Sent: Wednesday, July 10, 2013 4:19 AM To: Phil Crooker Cc: xymon at xymon.com Subject: Re: [Xymon] problem with receiving bbwin data Phil, Just found a fix that works for me :) I've been having difficulties setting up the bbwin client (ver 0.13 going to xymon 4.3.10 server) running on win7 sp1, 2008r2 and win2012 boxes in central mode. I've had zero response from the bbwin-help forum for this, so please bear with me. For background: I can't get the bbwin client to stop sending all the logs, it ignores any maxdata parameters I use. Eg: eventlog:system:1024 It sends everything anyway. If I use log:system:1024, the bbwin client throws an error that it can't find the system log file. It does find the file without the maxdata parameter. One problem with the settings in client-local.cfg on the xymon server is that they only propagate to the client after a successful sending of a client message. The mechanics is that after the client message has been sent, BBWin does a shutdown on the write socket and then reads from the read socket until EOF. If the write socket is never shutdown because the server breaks the connection due to flooding, then the client-local.cfg section will never be sent to the client. I've previously attempted to manually create C:\Program Files(x86)\BBWin\tmp\clientlocal.cfg to trick it, but with limited success. Just played around with it again - with client reporting to my primary server I just got it going - previously I was reporting to 2 servers. The trick seems to be in having logfetch.status also created. I then modified my BBWin.cfg back to 2 servers and it broke again. Maybe the 2nd server is responding differently. It is undefined which server's clientlocal.cfg to believe anyway :) Swapped the order of the servers around and it seems stable. If the server just discarded all the flooding data beyond MAXMSG_* and then returned the client config section anyway maybe it could be made to work, but there may be good reasons why that would not be sensible. This aside, I do need to get this working as I have to start monitoring 10 new windows servers and have to monitor the event logs, so I can't just stop sending them as has been suggested. Yes I could do some powershell scripts and send them via bbwincmd but the bbwin client is made for this task..... So, looking at this from the other side, the xymon server appears to be resetting the session after about 22MB of data has been sent (I know, this is ludicrous, but it is windows). Nothing in the xymond logs (except for the occasional data flooding error ("1st line client", always)); on the client side it reports it can't send the data to the xymon server. Depending on what auditting you have enabled, 22MB is very easy to achieve! Turn on Security event logging including success and failure and it's almost guaranteed :) David. I've set the MAXMSG_* to quite silly levels: # ipcs ------ Shared Memory Segments -------- key shmid owner perms bytes nattch status 0x01034be7 16908288 xymon 600 102400000 2 0x02034be7 16941057 xymon 600 102400000 2 0x03034be7 16973826 xymon 600 102400000 2 0x04034be7 17006595 xymon 600 102400000 2 0x05034be7 17039364 xymon 600 262144 1 0x06034be7 17072133 xymon 600 32768 1 0x07034be7 17104902 xymon 600 102400000 2 0x08034be7 17137671 xymon 600 102400000 2 0x09034be7 17170440 xymon 600 131072 1 Anything up to and including this size has no effect on the problem. Looking at the tcpdump stream, the bbwin client sends data normally with regular ACKs from xymond till around that 22MB mark then xymond responds with a FIN packet, then with RST packets and the session shuts down. Nothing in the packets themselves indicate what the problem is. If anyone can help with this, please, it would be great. thanks, Phil Please consider the environment before printing this e-mail This message from ORIX Australia may contain confidential and/or privileged information. If you are not the intended recipient, any use, disclosure or copying of this message (or of any attachments to it) is not authorised. If you have received this message in error, please notify the sender immediately and delete the message and any attachments from your system. Please inform the sender if you do not wish to receive further communications by email. ORIX handles personal information according to a Privacy Policy that is consistent with the National Privacy Principles. Please let us know if you would like a copy. It is also available at www.orix.com.au<http://www.orix.com.au>; -- David Baldwin - Senior Systems Administrator (Datacentres + Networks) Information and Communication Technology Services Australian Sports Commission http://ausport.gov.au Tel 02 62147830 Fax 02 62141830 PO Box 176 Belconnen ACT 2616 user-cbbf693f2c89@xymon.invalid<mailto:user-cbbf693f2c89@xymon.invalid> Leverrier Street Bruce ACT 2617 Keep up to date with what's happening in Australian sport visit www.ausport.gov.au<http://www.ausport.gov.au>; This message is intended for the addressee named and may contain confidential and privileged information. If you are not the intended recipient please note that any form of distribution, copying or use of this communication or the information in it is strictly prohibited and may be unlawful. If you receive this message in error, please delete it and notify the sender. This message is intended only for the individual or entity to which it is addressed. It may contain privileged, confidential information which is exempt from disclosure under applicable laws. If you are not the intended recipient, please note that you are strictly prohibited from disseminating or distributing this information (other than to the intended recipient) or copying this information. If you have received this communication in error, please notify us immediately by e-mail or by telephone at the above number. Thank you. Please consider the environment before printing this e-mail This message from ORIX Australia may contain confidential and/or privileged information. If you are not the intended recipient, any use, disclosure or copying of this message (or of any attachments to it) is not authorised. If you have received this message in error, please notify the sender immediately and delete the message and any attachments from your system. Please inform the sender if you do not wish to receive further communications by email. ORIX handles personal information according to a Privacy Policy that is consistent with the National Privacy Principles. Please let us know if you would like a copy. It is also available at www.orix.com.au<http://www.orix.com.au>;
list David Baldwin
Phil,
▸
Thanks for the reply. Before continuing, what is supposed to be in logfetch.status? and in clientversion.cfg?
logfetch.status will possibly only exist if you are checking any regular logfiles - e.g. if client-local.cfg contains something like: [win32] log:c:\WINDOWS\WindowsUpdate.log:20480 clientlocal.cfg will contain the relevant section similar to your 4 line example below. clientversion.cfg doesn't exist on my systems As BBWin runs, check the size of clientlocal.cfg - if it gets wiped, try saving the contents back again. Also, updating BBWin.cfg should cause BBWin to run immediately without restarting the service. I haven't played around with commenting out msgs.dll David.
▸
So, I have one server with the msgs.dll out of the config. It has been
running for many days with the client updating fine. I redid the xymon
client-local.cfg, stopped bbwin, deleted \tmp\clientlocal.cfg and
restarted it (as it wouldn't update), the \tmp\clientlocal.cfg now
reads what the client-local has:
eventlog:application:1024
ignore BigBrotherXymonClient
eventlog:system:1025
eventlog:security:10240
I let that run a cycle, it still updates OK. I then put in the
msgs.dll statement in BBWin.cfg; that registers and BBWin.log shows:
2013/07/11 10:42:55 [DEBUG]: [bbwinupdate]: can't open C:\Program
Files (x86)\BBWin\etc\clientversion.cfg : The system cannot find the
file specified.
2013/07/11 10:42:55 [DEBUG]: [cpu]: cpu review started
2013/07/11 10:42:55 [DEBUG]: [cpu]: cpu review ended
2013/07/11 10:42:55 [DEBUG]: [disk]: drive string A:\
2013/07/11 10:42:55 [DEBUG]: [disk]: drive string C:\
2013/07/11 10:42:55 [DEBUG]: [disk]: drive string D:\
2013/07/11 10:42:55 [DEBUG]: [disk]: drive string E:\
2013/07/11 10:42:55 [DEBUG]: [filesystem]: start
AgentFileSystem::InitCentralMode
2013/07/11 10:42:55 [DEBUG]: [filesystem]: checking file C:\Program
Files (x86)\BBWin\tmp\clientlocal.cfg
2013/07/11 10:42:55 [DEBUG]: [filesystem]: reading file C:\Program
Files (x86)\BBWin\tmp\clientlocal.cfg
2013/07/11 10:42:55 [DEBUG]: [msgs]: start AgentMsgs::InitCentralMode
2013/07/11 10:42:55 [DEBUG]: [msgs]: checking file C:\Program Files
(x86)\BBWin\tmp\clientlocal.cfg
2013/07/11 10:42:55 [DEBUG]: [msgs]: reading file C:\Program Files
(x86)\BBWin\tmp\clientlocal.cfg
2013/07/11 10:42:55 [DEBUG]: [msgs]: will use maxdata size : 1024
2013/07/11 10:42:55 [DEBUG]: [msgs]: will ignore : BigBrotherXymonClient
2013/07/11 10:42:55 [DEBUG]: [msgs]: will use maxdata size : 1025
2013/07/11 10:42:55 [DEBUG]: [msgs]: will use maxdata size : 10240
and no more updates. The .\tmp\msgs.hostname.txt file is 88MB.
thanks, Phil
*From:* Scot Kreienkamp
*Sent:* Wednesday, 10 July 2013 10:19 PM
*To:* David Baldwin; Phil Crooker
*Cc:* xymon at xymon.com
*Subject:* RE: [Xymon] problem with receiving bbwin data
If the eventlog data being transmitted is too large to transmit until
the clientlocal.cfg is sent back and populated in BBWin, then why not
disable the eventlog client in BBWin until it successfully sends 2-3
status messages? By that time the clientlocal.cfg will be populated
on the client and you can enable the eventlog reader again and
everything should work fine.
*Scot Kreienkamp *
*From:*Xymon [mailto:xymon-bounces at xymon.com] *On Behalf Of *David Baldwin
▸
*Sent:* Wednesday, July 10, 2013 4:19 AM *To:* Phil Crooker *Cc:* xymon at xymon.com *Subject:* Re: [Xymon] problem with receiving bbwin data Phil, Just found a fix that works for me :) I've been having difficulties setting up the bbwin client (ver 0.13 going to xymon 4.3.10 server) running on win7 sp1, 2008r2 and win2012 boxes in central mode. I've had zero response from the bbwin-help forum for this, so please bear with me. For background: I can't get the bbwin client to stop sending all the logs, it ignores any maxdata parameters I use. Eg: eventlog:system:1024 It sends everything anyway. If I use log:system:1024, the bbwin client throws an error that it can't find the system log file. It does find the file without the maxdata parameter. One problem with the settings in client-local.cfg on the xymon server is that they only propagate to the client after a successful sending of a client message. The mechanics is that after the client message has been sent, BBWin does a shutdown on the write socket and then reads from the read socket until EOF. If the write socket is never shutdown because the server breaks the connection due to flooding, then the client-local.cfg section will never be sent to the client. I've previously attempted to manually create C:\Program Files(x86)\BBWin\tmp\clientlocal.cfg to trick it, but with limited success. Just played around with it again - with client reporting to my primary server I just got it going - previously I was reporting to 2 servers. The trick seems to be in having logfetch.status also created. I then modified my BBWin.cfg back to 2 servers and it broke again. Maybe the 2nd server is responding differently. It is undefined which server's clientlocal.cfg to believe anyway :) Swapped the order of the servers around and it seems stable. If the server just discarded all the flooding data beyond MAXMSG_* and then returned the client config section anyway maybe it could be made to work, but there may be good reasons why that would not be sensible. This aside, I do need to get this working as I have to start monitoring 10 new windows servers and have to monitor the event logs, so I can't just stop sending them as has been suggested. Yes I could do some powershell scripts and send them via bbwincmd but the bbwin client is made for this task..... So, looking at this from the other side, the xymon server appears to be resetting the session after about 22MB of data has been sent (I know, this is ludicrous, but it is windows). Nothing in the xymond logs (except for the occasional data flooding error ("1st line client", always)); on the client side it reports it can't send the data to the xymon server. Depending on what auditting you have enabled, 22MB is very easy to achieve! Turn on Security event logging including success and failure and it's almost guaranteed :) David. I've set the MAXMSG_* to quite silly levels: # ipcs ------ Shared Memory Segments -------- key shmid owner perms bytes nattch status 0x01034be7 16908288 xymon 600 102400000 2 0x02034be7 16941057 xymon 600 102400000 2 0x03034be7 16973826 xymon 600 102400000 2 0x04034be7 17006595 xymon 600 102400000 2 0x05034be7 17039364 xymon 600 262144 1 0x06034be7 17072133 xymon 600 32768 1 0x07034be7 17104902 xymon 600 102400000 2 0x08034be7 17137671 xymon 600 102400000 2 0x09034be7 17170440 xymon 600 131072 1 Anything up to and including this size has no effect on the problem. Looking at the tcpdump stream, the bbwin client sends data normally with regular ACKs from xymond till around that 22MB mark then xymond responds with a FIN packet, then with RST packets and the session shuts down. Nothing in the packets themselves indicate what the problem is. If anyone can help with this, please, it would be great. thanks, Phil *Please consider the environment before printing this e-mail* This message from ORIX Australia may contain confidential and/or privileged information. If you are not the intended recipient, any use, disclosure or copying of this message (or of any attachments to it) is not authorised. If you have received this message in error, please notify the sender immediately and delete the message and any attachments from your system. Please inform the sender if you do not wish to receive further communications by email. ORIX handles personal information according to a Privacy Policy that is consistent with the National Privacy Principles. Please let us know if you would like a copy. It is also available at www.orix.com.au <http://www.orix.com.au>; -- David Baldwin - Senior Systems Administrator (Datacentres + Networks) Information and Communication Technology Services Australian Sports Commission http://ausport.gov.au Tel 02 62147830 Fax 02 62141830 PO Box 176 Belconnen ACT 2616
user-cbbf693f2c89@xymon.invalid <mailto:user-cbbf693f2c89@xymon.invalid> Leverrier Street Bruce ACT 2617 Keep up to date with what's happening in Australian sport visit www.ausport.gov.au <http://www.ausport.gov.au>;
▸
This message is intended for the addressee named and may contain confidential and privileged information. If you are not the intended recipient please note that any form of distribution, copying or use of this communication or the information in it is strictly prohibited and may be unlawful. If you receive this message in error, please delete it and notify the sender. This message is intended only for the individual or entity to which it is addressed. It may contain privileged, confidential information which is exempt from disclosure under applicable laws. If you are not the intended recipient, please note that you are strictly prohibited from disseminating or distributing this information (other than to the intended recipient) or copying this information. If you have received this communication in error, please notify us immediately by e-mail or by telephone at the above number. Thank you. *Please consider the environment before printing this e-mail* This message from ORIX Australia may contain confidential and/or privileged information. If you are not the intended recipient, any use, disclosure or copying of this message (or of any attachments to it) is not authorised. If you have received this message in error, please notify the sender immediately and delete the message and any attachments from your system. Please inform the sender if you do not wish to receive further communications by email. ORIX handles personal information according to a Privacy Policy that is consistent with the National Privacy Principles. Please let us know if you would like a copy. It is also available at www.orix.com.au <http://www.orix.com.au>;
-- David Baldwin - Senior Systems Administrator (Datacentres + Networks) Information and Communication Technology Services Australian Sports Commission http://ausport.gov.au Tel 02 62147830 Fax 02 62141830 PO Box 176 Belconnen ACT 2616 user-cbbf693f2c89@xymon.invalid Leverrier Street Bruce ACT 2617
list Phil Crooker
Thanks, David.... Yes, I don't have clientversion.cfg either - I tried putting in "0.13" in a file of that name in \etc, the client says it reads it and no more error, but who knows what this does or doesn't do... Anyway, on to the rest. As I described in the previous email, I did let the clientlocal.cfg update successfully. But as soon as I add the msgs.dll agent bbwin still ignores the maxdata parameter, tries to send everything and xymond drops the connection before completion (this latest time at 31MB). Do you have this working? Does it actually reduce the size of the msgs sent? thanks, Phil
▸
From: David Baldwin Sent: Thursday, 11 July 2013 11:28 AM To: Phil Crooker Cc: xymon at xymon.com Subject: Re: [Xymon] problem with receiving bbwin data Phil, Thanks for the reply. Before continuing, what is supposed to be in logfetch.status? and in clientversion.cfg? logfetch.status will possibly only exist if you are checking any regular logfiles - e.g. if client-local.cfg contains something like: [win32] log:c:\WINDOWS\WindowsUpdate.log:20480 clientlocal.cfg will contain the relevant section similar to your 4 line example below. clientversion.cfg doesn't exist on my systems As BBWin runs, check the size of clientlocal.cfg - if it gets wiped, try saving the contents back again. Also, updating BBWin.cfg should cause BBWin to run immediately without restarting the service. I haven't played around with commenting out msgs.dll David. So, I have one server with the msgs.dll out of the config. It has been running for many days with the client updating fine. I redid the xymon client-local.cfg, stopped bbwin, deleted \tmp\clientlocal.cfg and restarted it (as it wouldn't update), the \tmp\clientlocal.cfg now reads what the client-local has: eventlog:application:1024 ignore BigBrotherXymonClient eventlog:system:1025 eventlog:security:10240 I let that run a cycle, it still updates OK. I then put in the msgs.dll statement in BBWin.cfg; that registers and BBWin.log shows: 2013/07/11 10:42:55 [DEBUG]: [bbwinupdate]: can't open C:\Program Files (x86)\BBWin\etc\clientversion.cfg : The system cannot find the file specified. 2013/07/11 10:42:55 [DEBUG]: [cpu]: cpu review started 2013/07/11 10:42:55 [DEBUG]: [cpu]: cpu review ended 2013/07/11 10:42:55 [DEBUG]: [disk]: drive string A:\ 2013/07/11 10:42:55 [DEBUG]: [disk]: drive string C:\ 2013/07/11 10:42:55 [DEBUG]: [disk]: drive string D:\ 2013/07/11 10:42:55 [DEBUG]: [disk]: drive string E:\ 2013/07/11 10:42:55 [DEBUG]: [filesystem]: start AgentFileSystem::InitCentralMode 2013/07/11 10:42:55 [DEBUG]: [filesystem]: checking file C:\Program Files (x86)\BBWin\tmp\clientlocal.cfg 2013/07/11 10:42:55 [DEBUG]: [filesystem]: reading file C:\Program Files (x86)\BBWin\tmp\clientlocal.cfg 2013/07/11 10:42:55 [DEBUG]: [msgs]: start AgentMsgs::InitCentralMode 2013/07/11 10:42:55 [DEBUG]: [msgs]: checking file C:\Program Files (x86)\BBWin\tmp\clientlocal.cfg 2013/07/11 10:42:55 [DEBUG]: [msgs]: reading file C:\Program Files (x86)\BBWin\tmp\clientlocal.cfg 2013/07/11 10:42:55 [DEBUG]: [msgs]: will use maxdata size : 1024 2013/07/11 10:42:55 [DEBUG]: [msgs]: will ignore : BigBrotherXymonClient 2013/07/11 10:42:55 [DEBUG]: [msgs]: will use maxdata size : 1025 2013/07/11 10:42:55 [DEBUG]: [msgs]: will use maxdata size : 10240 and no more updates. The .\tmp\msgs.hostname.txt file is 88MB. thanks, Phil From: Scot Kreienkamp Sent: Wednesday, 10 July 2013 10:19 PM To: David Baldwin; Phil Crooker Cc: xymon at xymon.com<mailto:xymon at xymon.com> Subject: RE: [Xymon] problem with receiving bbwin data If the eventlog data being transmitted is too large to transmit until the clientlocal.cfg is sent back and populated in BBWin, then why not disable the eventlog client in BBWin until it successfully sends 2-3 status messages? By that time the clientlocal.cfg will be populated on the client and you can enable the eventlog reader again and everything should work fine. Scot Kreienkamp From: Xymon [mailto:xymon-bounces at xymon.com] On Behalf Of David Baldwin Sent: Wednesday, July 10, 2013 4:19 AM To: Phil Crooker Cc: xymon at xymon.com<mailto:xymon at xymon.com> Subject: Re: [Xymon] problem with receiving bbwin data Phil, Just found a fix that works for me :) I've been having difficulties setting up the bbwin client (ver 0.13 going to xymon 4.3.10 server) running on win7 sp1, 2008r2 and win2012 boxes in central mode. I've had zero response from the bbwin-help forum for this, so please bear with me. For background: I can't get the bbwin client to stop sending all the logs, it ignores any maxdata parameters I use. Eg: eventlog:system:1024 It sends everything anyway. If I use log:system:1024, the bbwin client throws an error that it can't find the system log file. It does find the file without the maxdata parameter. One problem with the settings in client-local.cfg on the xymon server is that they only propagate to the client after a successful sending of a client message. The mechanics is that after the client message has been sent, BBWin does a shutdown on the write socket and then reads from the read socket until EOF. If the write socket is never shutdown because the server breaks the connection due to flooding, then the client-local.cfg section will never be sent to the client. I've previously attempted to manually create C:\Program Files(x86)\BBWin\tmp\clientlocal.cfg to trick it, but with limited success. Just played around with it again - with client reporting to my primary server I just got it going - previously I was reporting to 2 servers. The trick seems to be in having logfetch.status also created. I then modified my BBWin.cfg back to 2 servers and it broke again. Maybe the 2nd server is responding differently. It is undefined which server's clientlocal.cfg to believe anyway :) Swapped the order of the servers around and it seems stable. If the server just discarded all the flooding data beyond MAXMSG_* and then returned the client config section anyway maybe it could be made to work, but there may be good reasons why that would not be sensible. This aside, I do need to get this working as I have to start monitoring 10 new windows servers and have to monitor the event logs, so I can't just stop sending them as has been suggested. Yes I could do some powershell scripts and send them via bbwincmd but the bbwin client is made for this task..... So, looking at this from the other side, the xymon server appears to be resetting the session after about 22MB of data has been sent (I know, this is ludicrous, but it is windows). Nothing in the xymond logs (except for the occasional data flooding error ("1st line client", always)); on the client side it reports it can't send the data to the xymon server. Depending on what auditting you have enabled, 22MB is very easy to achieve! Turn on Security event logging including success and failure and it's almost guaranteed :) David. I've set the MAXMSG_* to quite silly levels: # ipcs ------ Shared Memory Segments -------- key shmid owner perms bytes nattch status 0x01034be7 16908288 xymon 600 102400000 2 0x02034be7 16941057 xymon 600 102400000 2 0x03034be7 16973826 xymon 600 102400000 2 0x04034be7 17006595 xymon 600 102400000 2 0x05034be7 17039364 xymon 600 262144 1 0x06034be7 17072133 xymon 600 32768 1 0x07034be7 17104902 xymon 600 102400000 2 0x08034be7 17137671 xymon 600 102400000 2 0x09034be7 17170440 xymon 600 131072 1 Anything up to and including this size has no effect on the problem. Looking at the tcpdump stream, the bbwin client sends data normally with regular ACKs from xymond till around that 22MB mark then xymond responds with a FIN packet, then with RST packets and the session shuts down. Nothing in the packets themselves indicate what the problem is. If anyone can help with this, please, it would be great. thanks, Phil Please consider the environment before printing this e-mail This message from ORIX Australia may contain confidential and/or privileged information. If you are not the intended recipient, any use, disclosure or copying of this message (or of any attachments to it) is not authorised. If you have received this message in error, please notify the sender immediately and delete the message and any attachments from your system. Please inform the sender if you do not wish to receive further communications by email. ORIX handles personal information according to a Privacy Policy that is consistent with the National Privacy Principles. Please let us know if you would like a copy. It is also available at www.orix.com.au<http://www.orix.com.au>; -- David Baldwin - Senior Systems Administrator (Datacentres + Networks) Information and Communication Technology Services Australian Sports Commission http://ausport.gov.au Tel 02 62147830 Fax 02 62141830 PO Box 176 Belconnen ACT 2616 user-cbbf693f2c89@xymon.invalid<mailto:user-cbbf693f2c89@xymon.invalid> Leverrier Street Bruce ACT 2617 Keep up to date with what's happening in Australian sport visit www.ausport.gov.au<http://www.ausport.gov.au>; This message is intended for the addressee named and may contain confidential and privileged information. If you are not the intended recipient please note that any form of distribution, copying or use of this communication or the information in it is strictly prohibited and may be unlawful. If you receive this message in error, please delete it and notify the sender. This message is intended only for the individual or entity to which it is addressed. It may contain privileged, confidential information which is exempt from disclosure under applicable laws. If you are not the intended recipient, please note that you are strictly prohibited from disseminating or distributing this information (other than to the intended recipient) or copying this information. If you have received this communication in error, please notify us immediately by e-mail or by telephone at the above number. Thank you. Please consider the environment before printing this e-mail This message from ORIX Australia may contain confidential and/or privileged information. If you are not the intended recipient, any use, disclosure or copying of this message (or of any attachments to it) is not authorised. If you have received this message in error, please notify the sender immediately and delete the message and any attachments from your system. Please inform the sender if you do not wish to receive further communications by email. ORIX handles personal information according to a Privacy Policy that is consistent with the National Privacy Principles. Please let us know if you would like a copy. It is also available at www.orix.com.au<http://www.orix.com.au>; -- David Baldwin - Senior Systems Administrator (Datacentres + Networks) Information and Communication Technology Services Australian Sports Commission http://ausport.gov.au Tel 02 62147830 Fax 02 62141830 PO Box 176 Belconnen ACT 2616 user-cbbf693f2c89@xymon.invalid<mailto:user-cbbf693f2c89@xymon.invalid> Leverrier Street Bruce ACT 2617 Please consider the environment before printing this e-mail This message from ORIX Australia may contain confidential and/or privileged information. If you are not the intended recipient, any use, disclosure or copying of this message (or of any attachments to it) is not authorised. If you have received this message in error, please notify the sender immediately and delete the message and any attachments from your system. Please inform the sender if you do not wish to receive further communications by email. ORIX handles personal information according to a Privacy Policy that is consistent with the National Privacy Principles. Please let us know if you would like a copy. It is also available at www.orix.com.au<http://www.orix.com.au>;
list David Baldwin
Phil,
▸
Thanks, David.... Yes, I don't have clientversion.cfg either - I tried putting in "0.13" in a file of that name in \etc, the client says it reads it and no more error, but who knows what this does or doesn't do... Anyway, on to the rest. As I described in the previous email, I did let the clientlocal.cfg update successfully. But as soon as I add the msgs.dll agent bbwin still ignores the maxdata parameter, tries to send everything and xymond drops the connection before completion (this latest time at 31MB). Do you have this working? Does it actually reduce the size of the msgs sent?
I have it working, but I don't actually use BBWin for event log reporting - I forward all my event logs using Snare to a central syslog server and have my own checking against that server-side - see http://xymonton.org/monitors:winevtmsgs.pl My client-local.cfg for Windows clients includes the following to filter out everything client-side. You may be able to play with the ignore rules to tune it a bit more to your needs while still reducing the size of the report. By checking the BBWin\tmp\msg.<clientname>.txt file you should see what is taking up all the space. log:eventlog_security:10240 ignore .* ignore . eventlog:security:10240 ignore handle ignore .* ignore . eventlog:System:10240 ignore .* ignore . eventlog:application:10240 ignore .* ignore . eventlog:directory service:10240 ignore .* ignore . David.
▸
thanks, Phil *From:* David Baldwin *Sent:* Thursday, 11 July 2013 11:28 AM *To:* Phil Crooker *Cc:* xymon at xymon.com *Subject:* Re: [Xymon] problem with receiving bbwin data Phil,Thanks for the reply. Before continuing, what is supposed to be in logfetch.status? and in clientversion.cfg?logfetch.status will possibly only exist if you are checking any regular logfiles - e.g. if client-local.cfg contains something like: [win32] log:c:\WINDOWS\WindowsUpdate.log:20480 clientlocal.cfg will contain the relevant section similar to your 4 line example below. clientversion.cfg doesn't exist on my systems As BBWin runs, check the size of clientlocal.cfg - if it gets wiped, try saving the contents back again. Also, updating BBWin.cfg should cause BBWin to run immediately without restarting the service. I haven't played around with commenting out msgs.dll David.So, I have one server with the msgs.dll out of the config. It has been running for many days with the client updating fine. I redid the xymon client-local.cfg, stopped bbwin, deleted \tmp\clientlocal.cfg and restarted it (as it wouldn't update), the \tmp\clientlocal.cfg now reads what the client-local has: eventlog:application:1024 ignore BigBrotherXymonClient eventlog:system:1025 eventlog:security:10240 I let that run a cycle, it still updates OK. I then put in the msgs.dll statement in BBWin.cfg; that registers and BBWin.log shows: 2013/07/11 10:42:55 [DEBUG]: [bbwinupdate]: can't open C:\Program Files (x86)\BBWin\etc\clientversion.cfg : The system cannot find the file specified. 2013/07/11 10:42:55 [DEBUG]: [cpu]: cpu review started 2013/07/11 10:42:55 [DEBUG]: [cpu]: cpu review ended 2013/07/11 10:42:55 [DEBUG]: [disk]: drive string A:\ 2013/07/11 10:42:55 [DEBUG]: [disk]: drive string C:\ 2013/07/11 10:42:55 [DEBUG]: [disk]: drive string D:\ 2013/07/11 10:42:55 [DEBUG]: [disk]: drive string E:\ 2013/07/11 10:42:55 [DEBUG]: [filesystem]: start AgentFileSystem::InitCentralMode 2013/07/11 10:42:55 [DEBUG]: [filesystem]: checking file C:\Program Files (x86)\BBWin\tmp\clientlocal.cfg 2013/07/11 10:42:55 [DEBUG]: [filesystem]: reading file C:\Program Files (x86)\BBWin\tmp\clientlocal.cfg 2013/07/11 10:42:55 [DEBUG]: [msgs]: start AgentMsgs::InitCentralMode 2013/07/11 10:42:55 [DEBUG]: [msgs]: checking file C:\Program Files (x86)\BBWin\tmp\clientlocal.cfg 2013/07/11 10:42:55 [DEBUG]: [msgs]: reading file C:\Program Files (x86)\BBWin\tmp\clientlocal.cfg 2013/07/11 10:42:55 [DEBUG]: [msgs]: will use maxdata size : 1024 2013/07/11 10:42:55 [DEBUG]: [msgs]: will ignore : BigBrotherXymonClient 2013/07/11 10:42:55 [DEBUG]: [msgs]: will use maxdata size : 1025 2013/07/11 10:42:55 [DEBUG]: [msgs]: will use maxdata size : 10240 and no more updates. The .\tmp\msgs.hostname.txt file is 88MB. thanks, Phil *From:* Scot Kreienkamp *Sent:* Wednesday, 10 July 2013 10:19 PM *To:* David Baldwin; Phil Crooker *Cc:* xymon at xymon.com *Subject:* RE: [Xymon] problem with receiving bbwin data If the eventlog data being transmitted is too large to transmit until the clientlocal.cfg is sent back and populated in BBWin, then why not disable the eventlog client in BBWin until it successfully sends 2-3 status messages? By that time the clientlocal.cfg will be populated on the client and you can enable the eventlog reader again and everything should work fine. *Scot Kreienkamp * *From:*Xymon [mailto:xymon-bounces at xymon.com] *On Behalf Of *David Baldwin *Sent:* Wednesday, July 10, 2013 4:19 AM *To:* Phil Crooker *Cc:* xymon at xymon.com *Subject:* Re: [Xymon] problem with receiving bbwin data Phil, Just found a fix that works for me :) I've been having difficulties setting up the bbwin client (ver 0.13 going to xymon 4.3.10 server) running on win7 sp1, 2008r2 and win2012 boxes in central mode. I've had zero response from the bbwin-help forum for this, so please bear with me. For background: I can't get the bbwin client to stop sending all the logs, it ignores any maxdata parameters I use. Eg: eventlog:system:1024 It sends everything anyway. If I use log:system:1024, the bbwin client throws an error that it can't find the system log file. It does find the file without the maxdata parameter. One problem with the settings in client-local.cfg on the xymon server is that they only propagate to the client after a successful sending of a client message. The mechanics is that after the client message has been sent, BBWin does a shutdown on the write socket and then reads from the read socket until EOF. If the write socket is never shutdown because the server breaks the connection due to flooding, then the client-local.cfg section will never be sent to the client. I've previously attempted to manually create C:\Program Files(x86)\BBWin\tmp\clientlocal.cfg to trick it, but with limited success. Just played around with it again - with client reporting to my primary server I just got it going - previously I was reporting to 2 servers. The trick seems to be in having logfetch.status also created. I then modified my BBWin.cfg back to 2 servers and it broke again. Maybe the 2nd server is responding differently. It is undefined which server's clientlocal.cfg to believe anyway :) Swapped the order of the servers around and it seems stable. If the server just discarded all the flooding data beyond MAXMSG_* and then returned the client config section anyway maybe it could be made to work, but there may be good reasons why that would not be sensible. This aside, I do need to get this working as I have to start monitoring 10 new windows servers and have to monitor the event logs, so I can't just stop sending them as has been suggested. Yes I could do some powershell scripts and send them via bbwincmd but the bbwin client is made for this task..... So, looking at this from the other side, the xymon server appears to be resetting the session after about 22MB of data has been sent (I know, this is ludicrous, but it is windows). Nothing in the xymond logs (except for the occasional data flooding error ("1st line client", always)); on the client side it reports it can't send the data to the xymon server. Depending on what auditting you have enabled, 22MB is very easy to achieve! Turn on Security event logging including success and failure and it's almost guaranteed :) David. I've set the MAXMSG_* to quite silly levels: # ipcs ------ Shared Memory Segments -------- key shmid owner perms bytes nattch status 0x01034be7 16908288 xymon 600 102400000 2 0x02034be7 16941057 xymon 600 102400000 2 0x03034be7 16973826 xymon 600 102400000 2 0x04034be7 17006595 xymon 600 102400000 2 0x05034be7 17039364 xymon 600 262144 1 0x06034be7 17072133 xymon 600 32768 1 0x07034be7 17104902 xymon 600 102400000 2 0x08034be7 17137671 xymon 600 102400000 2 0x09034be7 17170440 xymon 600 131072 1 Anything up to and including this size has no effect on the problem. Looking at the tcpdump stream, the bbwin client sends data normally with regular ACKs from xymond till around that 22MB mark then xymond responds with a FIN packet, then with RST packets and the session shuts down. Nothing in the packets themselves indicate what the problem is. If anyone can help with this, please, it would be great. thanks, Phil
-- David Baldwin - Senior Systems Administrator (Datacentres + Networks) Information and Communication Technology Services Australian Sports Commission http://ausport.gov.au Tel 02 62147830 Fax 02 62141830 PO Box 176 Belconnen ACT 2616 user-cbbf693f2c89@xymon.invalid Leverrier Street Bruce ACT 2617 Keep up to date with what's happening in Australian sport visit http://www.ausport.gov.au This message is intended for the addressee named and may contain confidential and privileged information. If you are not the intended recipient please note that any form of distribution, copying or use of this communication or the information in it is strictly prohibited and may be unlawful. If you receive this message in error, please delete it and notify the sender.
list Phil Crooker
Thanks again. Yes, snare / winevtmsgs.pl looks like a good work around for this issue.... cheers, P
▸
From: David Baldwin
Sent: Thursday, 11 July 2013 2:45 PM
To: Phil Crooker
Cc: xymon at xymon.com
Subject: Re: [Xymon] problem with receiving bbwin data
Phil,Thanks, David.... Yes, I don't have clientversion.cfg either - I tried putting in "0.13" in a file of that name in \etc, the client says it reads it and no more error, but who knows what this does or doesn't do... Anyway, on to the rest. As I described in the previous email, I did let the clientlocal.cfg update successfully. But as soon as I add the msgs.dll agent bbwin still ignores the maxdata parameter, tries to send everything and xymond drops the connection before completion (this latest time at 31MB). Do you have this working? Does it actually reduce the size of the msgs sent?
I have it working, but I don't actually use BBWin for event log reporting - I forward all my event logs using Snare to a central syslog server and have my own checking against that server-side - see http://xymonton.org/monitors:winevtmsgs.pl My client-local.cfg for Windows clients includes the following to filter out everything client-side. You may be able to play with the ignore rules to tune it a bit more to your needs while still reducing the size of the report. By checking the BBWin\tmp\msg.<clientname>.txt file you should see what is taking up all the space. log:eventlog_security:10240 ignore .* ignore . eventlog:security:10240 ignore handle ignore .* ignore . eventlog:System:10240 ignore .* ignore . eventlog:application:10240 ignore .* ignore . eventlog:directory service:10240 ignore .* ignore . David.
thanks, Phil *From:* David Baldwin *Sent:* Thursday, 11 July 2013 11:28 AM *To:* Phil Crooker *Cc:* xymon at xymon.com *Subject:* Re: [Xymon] problem with receiving bbwin data Phil,Thanks for the reply. Before continuing, what is supposed to be in logfetch.status? and in clientversion.cfg?logfetch.status will possibly only exist if you are checking any regular logfiles - e.g. if client-local.cfg contains something like: [win32] log:c:\WINDOWS\WindowsUpdate.log:20480 clientlocal.cfg will contain the relevant section similar to your 4 line example below. clientversion.cfg doesn't exist on my systems As BBWin runs, check the size of clientlocal.cfg - if it gets wiped, try saving the contents back again. Also, updating BBWin.cfg should cause BBWin to run immediately without restarting the service. I haven't played around with commenting out msgs.dll David.So, I have one server with the msgs.dll out of the config. It has been running for many days with the client updating fine. I redid the xymon client-local.cfg, stopped bbwin, deleted \tmp\clientlocal.cfg and restarted it (as it wouldn't update), the \tmp\clientlocal.cfg now reads what the client-local has: eventlog:application:1024 ignore BigBrotherXymonClient eventlog:system:1025 eventlog:security:10240 I let that run a cycle, it still updates OK. I then put in the msgs.dll statement in BBWin.cfg; that registers and BBWin.log shows: 2013/07/11 10:42:55 [DEBUG]: [bbwinupdate]: can't open C:\Program Files (x86)\BBWin\etc\clientversion.cfg : The system cannot find the file specified. 2013/07/11 10:42:55 [DEBUG]: [cpu]: cpu review started 2013/07/11 10:42:55 [DEBUG]: [cpu]: cpu review ended 2013/07/11 10:42:55 [DEBUG]: [disk]: drive string A:\ 2013/07/11 10:42:55 [DEBUG]: [disk]: drive string C:\ 2013/07/11 10:42:55 [DEBUG]: [disk]: drive string D:\ 2013/07/11 10:42:55 [DEBUG]: [disk]: drive string E:\ 2013/07/11 10:42:55 [DEBUG]: [filesystem]: start AgentFileSystem::InitCentralMode 2013/07/11 10:42:55 [DEBUG]: [filesystem]: checking file C:\Program Files (x86)\BBWin\tmp\clientlocal.cfg 2013/07/11 10:42:55 [DEBUG]: [filesystem]: reading file C:\Program Files (x86)\BBWin\tmp\clientlocal.cfg 2013/07/11 10:42:55 [DEBUG]: [msgs]: start AgentMsgs::InitCentralMode 2013/07/11 10:42:55 [DEBUG]: [msgs]: checking file C:\Program Files (x86)\BBWin\tmp\clientlocal.cfg 2013/07/11 10:42:55 [DEBUG]: [msgs]: reading file C:\Program Files (x86)\BBWin\tmp\clientlocal.cfg 2013/07/11 10:42:55 [DEBUG]: [msgs]: will use maxdata size : 1024 2013/07/11 10:42:55 [DEBUG]: [msgs]: will ignore : BigBrotherXymonClient 2013/07/11 10:42:55 [DEBUG]: [msgs]: will use maxdata size : 1025 2013/07/11 10:42:55 [DEBUG]: [msgs]: will use maxdata size : 10240 and no more updates. The .\tmp\msgs.hostname.txt file is 88MB. thanks, Phil *From:* Scot Kreienkamp *Sent:* Wednesday, 10 July 2013 10:19 PM *To:* David Baldwin; Phil Crooker *Cc:* xymon at xymon.com *Subject:* RE: [Xymon] problem with receiving bbwin data If the eventlog data being transmitted is too large to transmit until the clientlocal.cfg is sent back and populated in BBWin, then why not disable the eventlog client in BBWin until it successfully sends 2-3 status messages? By that time the clientlocal.cfg will be populated on the client and you can enable the eventlog reader again and everything should work fine. *Scot Kreienkamp * *From:*Xymon [mailto:xymon-bounces at xymon.com] *On Behalf Of *David Baldwin *Sent:* Wednesday, July 10, 2013 4:19 AM *To:* Phil Crooker *Cc:* xymon at xymon.com *Subject:* Re: [Xymon] problem with receiving bbwin data Phil, Just found a fix that works for me :) I've been having difficulties setting up the bbwin client (ver 0.13 going to xymon 4.3.10 server) running on win7 sp1, 2008r2 and win2012 boxes in central mode. I've had zero response from the bbwin-help forum for this, so please bear with me. For background: I can't get the bbwin client to stop sending all the logs, it ignores any maxdata parameters I use. Eg: eventlog:system:1024 It sends everything anyway. If I use log:system:1024, the bbwin client throws an error that it can't find the system log file. It does find the file without the maxdata parameter. One problem with the settings in client-local.cfg on the xymon server is that they only propagate to the client after a successful sending of a client message. The mechanics is that after the client message has been sent, BBWin does a shutdown on the write socket and then reads from the read socket until EOF. If the write socket is never shutdown because the server breaks the connection due to flooding, then the client-local.cfg section will never be sent to the client. I've previously attempted to manually create C:\Program Files(x86)\BBWin\tmp\clientlocal.cfg to trick it, but with limited success. Just played around with it again - with client reporting to my primary server I just got it going - previously I was reporting to 2 servers. The trick seems to be in having logfetch.status also created. I then modified my BBWin.cfg back to 2 servers and it broke again. Maybe the 2nd server is responding differently. It is undefined which server's clientlocal.cfg to believe anyway :) Swapped the order of the servers around and it seems stable. If the server just discarded all the flooding data beyond MAXMSG_* and then returned the client config section anyway maybe it could be made to work, but there may be good reasons why that would not be sensible. This aside, I do need to get this working as I have to start monitoring 10 new windows servers and have to monitor the event logs, so I can't just stop sending them as has been suggested. Yes I could do some powershell scripts and send them via bbwincmd but the bbwin client is made for this task..... So, looking at this from the other side, the xymon server appears to be resetting the session after about 22MB of data has been sent (I know, this is ludicrous, but it is windows). Nothing in the xymond logs (except for the occasional data flooding error ("1st line client", always)); on the client side it reports it can't send the data to the xymon server. Depending on what auditting you have enabled, 22MB is very easy to achieve! Turn on Security event logging including success and failure and it's almost guaranteed :) David. I've set the MAXMSG_* to quite silly levels: # ipcs ------ Shared Memory Segments -------- key shmid owner perms bytes nattch status 0x01034be7 16908288 xymon 600 102400000 2 0x02034be7 16941057 xymon 600 102400000 2 0x03034be7 16973826 xymon 600 102400000 2 0x04034be7 17006595 xymon 600 102400000 2 0x05034be7 17039364 xymon 600 262144 1 0x06034be7 17072133 xymon 600 32768 1 0x07034be7 17104902 xymon 600 102400000 2 0x08034be7 17137671 xymon 600 102400000 2 0x09034be7 17170440 xymon 600 131072 1 Anything up to and including this size has no effect on the problem. Looking at the tcpdump stream, the bbwin client sends data normally with regular ACKs from xymond till around that 22MB mark then xymond responds with a FIN packet, then with RST packets and the session shuts down. Nothing in the packets themselves indicate what the problem is. If anyone can help with this, please, it would be great. thanks, Phil
-- David Baldwin - Senior Systems Administrator (Datacentres + Networks) Information and Communication Technology Services Australian Sports Commission http://ausport.gov.au Tel 02 62147830 Fax 02 62141830 PO Box 176 Belconnen ACT 2616 user-cbbf693f2c89@xymon.invalid Leverrier Street Bruce ACT 2617 Keep up to date with what's happening in Australian sport visit http://www.ausport.gov.au This message is intended for the addressee named and may contain confidential and privileged information. If you are not the intended recipient please note that any form of distribution, copying or use of this communication or the information in it is strictly prohibited and may be unlawful. If you receive this message in error, please delete it and notify the sender. Please consider the environment before printing this e-mail This message from ORIX Australia may contain confidential and/or privileged information. If you are not the intended recipient, any use, disclosure or copying of this message (or of any attachments to it) is not authorised. If you have received this message in error, please notify the sender immediately and delete the message and any attachments from your system. Please inform the sender if you do not wish to receive further communications by email. ORIX handles personal information according to a Privacy Policy that is consistent with the National Privacy Principles. Please let us know if you would like a copy. It is also available at www.orix.com.au<http://www.orix.com.au>;
list David Baldwin
Phil,
Thanks again. Yes, snare / winevtmsgs.pl looks like a good work around for this issue....
Let me know if you try it out in anger. I'm not aware of anyone who actually uses it, and I've probably made some updates. I've also got a mass of localised rules for event log filtering since the severity levels, etc can be pretty bogus - errors that can be safely ignored and "informational" messages that you really need to be alerted on... David.
▸
cheers, P From: David Baldwin Sent: Thursday, 11 July 2013 2:45 PM To: Phil Crooker Cc: xymon at xymon.com Subject: Re: [Xymon] problem with receiving bbwin data Phil,Thanks, David.... Yes, I don't have clientversion.cfg either - I tried putting in "0.13" in a file of that name in \etc, the client says it reads it and no more error, but who knows what this does or doesn't do... Anyway, on to the rest. As I described in the previous email, I did let the clientlocal.cfg update successfully. But as soon as I add the msgs.dll agent bbwin still ignores the maxdata parameter, tries to send everything and xymond drops the connection before completion (this latest time at 31MB). Do you have this working? Does it actually reduce the size of the msgs sent?I have it working, but I don't actually use BBWin for event log reporting - I forward all my event logs using Snare to a central syslog server and have my own checking against that server-side - see http://xymonton.org/monitors:winevtmsgs.pl My client-local.cfg for Windows clients includes the following to filter out everything client-side. You may be able to play with the ignore rules to tune it a bit more to your needs while still reducing the size of the report. By checking the BBWin\tmp\msg.<clientname>.txt file you should see what is taking up all the space. log:eventlog_security:10240 ignore .* ignore . eventlog:security:10240 ignore handle ignore .* ignore . eventlog:System:10240 ignore .* ignore . eventlog:application:10240 ignore .* ignore . eventlog:directory service:10240 ignore .* ignore . David.thanks, Phil *From:* David Baldwin *Sent:* Thursday, 11 July 2013 11:28 AM *To:* Phil Crooker *Cc:* xymon at xymon.com *Subject:* Re: [Xymon] problem with receiving bbwin data Phil,Thanks for the reply. Before continuing, what is supposed to be in logfetch.status? and in clientversion.cfg?logfetch.status will possibly only exist if you are checking any regular logfiles - e.g. if client-local.cfg contains something like: [win32] log:c:\WINDOWS\WindowsUpdate.log:20480 clientlocal.cfg will contain the relevant section similar to your 4 line example below. clientversion.cfg doesn't exist on my systems As BBWin runs, check the size of clientlocal.cfg - if it gets wiped, try saving the contents back again. Also, updating BBWin.cfg should cause BBWin to run immediately without restarting the service. I haven't played around with commenting out msgs.dll David.So, I have one server with the msgs.dll out of the config. It has been running for many days with the client updating fine. I redid the xymon client-local.cfg, stopped bbwin, deleted \tmp\clientlocal.cfg and restarted it (as it wouldn't update), the \tmp\clientlocal.cfg now reads what the client-local has: eventlog:application:1024 ignore BigBrotherXymonClient eventlog:system:1025 eventlog:security:10240 I let that run a cycle, it still updates OK. I then put in the msgs.dll statement in BBWin.cfg; that registers and BBWin.log shows: 2013/07/11 10:42:55 [DEBUG]: [bbwinupdate]: can't open C:\Program Files (x86)\BBWin\etc\clientversion.cfg : The system cannot find the file specified. 2013/07/11 10:42:55 [DEBUG]: [cpu]: cpu review started 2013/07/11 10:42:55 [DEBUG]: [cpu]: cpu review ended 2013/07/11 10:42:55 [DEBUG]: [disk]: drive string A:\ 2013/07/11 10:42:55 [DEBUG]: [disk]: drive string C:\ 2013/07/11 10:42:55 [DEBUG]: [disk]: drive string D:\ 2013/07/11 10:42:55 [DEBUG]: [disk]: drive string E:\ 2013/07/11 10:42:55 [DEBUG]: [filesystem]: start AgentFileSystem::InitCentralMode 2013/07/11 10:42:55 [DEBUG]: [filesystem]: checking file C:\Program Files (x86)\BBWin\tmp\clientlocal.cfg 2013/07/11 10:42:55 [DEBUG]: [filesystem]: reading file C:\Program Files (x86)\BBWin\tmp\clientlocal.cfg 2013/07/11 10:42:55 [DEBUG]: [msgs]: start AgentMsgs::InitCentralMode 2013/07/11 10:42:55 [DEBUG]: [msgs]: checking file C:\Program Files (x86)\BBWin\tmp\clientlocal.cfg 2013/07/11 10:42:55 [DEBUG]: [msgs]: reading file C:\Program Files (x86)\BBWin\tmp\clientlocal.cfg 2013/07/11 10:42:55 [DEBUG]: [msgs]: will use maxdata size : 1024 2013/07/11 10:42:55 [DEBUG]: [msgs]: will ignore : BigBrotherXymonClient 2013/07/11 10:42:55 [DEBUG]: [msgs]: will use maxdata size : 1025 2013/07/11 10:42:55 [DEBUG]: [msgs]: will use maxdata size : 10240 and no more updates. The .\tmp\msgs.hostname.txt file is 88MB. thanks, Phil *From:* Scot Kreienkamp *Sent:* Wednesday, 10 July 2013 10:19 PM *To:* David Baldwin; Phil Crooker *Cc:* xymon at xymon.com *Subject:* RE: [Xymon] problem with receiving bbwin data If the eventlog data being transmitted is too large to transmit until the clientlocal.cfg is sent back and populated in BBWin, then why not disable the eventlog client in BBWin until it successfully sends 2-3 status messages? By that time the clientlocal.cfg will be populated on the client and you can enable the eventlog reader again and everything should work fine. *Scot Kreienkamp * *From:*Xymon [mailto:xymon-bounces at xymon.com] *On Behalf Of *David Baldwin *Sent:* Wednesday, July 10, 2013 4:19 AM *To:* Phil Crooker *Cc:* xymon at xymon.com *Subject:* Re: [Xymon] problem with receiving bbwin data Phil, Just found a fix that works for me :) I've been having difficulties setting up the bbwin client (ver 0.13 going to xymon 4.3.10 server) running on win7 sp1, 2008r2 and win2012 boxes in central mode. I've had zero response from the bbwin-help forum for this, so please bear with me. For background: I can't get the bbwin client to stop sending all the logs, it ignores any maxdata parameters I use. Eg: eventlog:system:1024 It sends everything anyway. If I use log:system:1024, the bbwin client throws an error that it can't find the system log file. It does find the file without the maxdata parameter. One problem with the settings in client-local.cfg on the xymon server is that they only propagate to the client after a successful sending of a client message. The mechanics is that after the client message has been sent, BBWin does a shutdown on the write socket and then reads from the read socket until EOF. If the write socket is never shutdown because the server breaks the connection due to flooding, then the client-local.cfg section will never be sent to the client. I've previously attempted to manually create C:\Program Files(x86)\BBWin\tmp\clientlocal.cfg to trick it, but with limited success. Just played around with it again - with client reporting to my primary server I just got it going - previously I was reporting to 2 servers. The trick seems to be in having logfetch.status also created. I then modified my BBWin.cfg back to 2 servers and it broke again. Maybe the 2nd server is responding differently. It is undefined which server's clientlocal.cfg to believe anyway :) Swapped the order of the servers around and it seems stable. If the server just discarded all the flooding data beyond MAXMSG_* and then returned the client config section anyway maybe it could be made to work, but there may be good reasons why that would not be sensible. This aside, I do need to get this working as I have to start monitoring 10 new windows servers and have to monitor the event logs, so I can't just stop sending them as has been suggested. Yes I could do some powershell scripts and send them via bbwincmd but the bbwin client is made for this task..... So, looking at this from the other side, the xymon server appears to be resetting the session after about 22MB of data has been sent (I know, this is ludicrous, but it is windows). Nothing in the xymond logs (except for the occasional data flooding error ("1st line client", always)); on the client side it reports it can't send the data to the xymon server. Depending on what auditting you have enabled, 22MB is very easy to achieve! Turn on Security event logging including success and failure and it's almost guaranteed :) David. I've set the MAXMSG_* to quite silly levels: # ipcs ------ Shared Memory Segments -------- key shmid owner perms bytes nattch status 0x01034be7 16908288 xymon 600 102400000 2 0x02034be7 16941057 xymon 600 102400000 2 0x03034be7 16973826 xymon 600 102400000 2 0x04034be7 17006595 xymon 600 102400000 2 0x05034be7 17039364 xymon 600 262144 1 0x06034be7 17072133 xymon 600 32768 1 0x07034be7 17104902 xymon 600 102400000 2 0x08034be7 17137671 xymon 600 102400000 2 0x09034be7 17170440 xymon 600 131072 1 Anything up to and including this size has no effect on the problem. Looking at the tcpdump stream, the bbwin client sends data normally with regular ACKs from xymond till around that 22MB mark then xymond responds with a FIN packet, then with RST packets and the session shuts down. Nothing in the packets themselves indicate what the problem is. If anyone can help with this, please, it would be great. thanks, Phil-- David Baldwin - Senior Systems Administrator (Datacentres + Networks) Information and Communication Technology Services Australian Sports Commission http://ausport.gov.au Tel 02 62147830 Fax 02 62141830 PO Box 176 Belconnen ACT 2616 user-cbbf693f2c89@xymon.invalid Leverrier Street Bruce ACT 2617 Keep up to date with what's happening in Australian sport visit http://www.ausport.gov.au This message is intended for the addressee named and may contain confidential and privileged information. If you are not the intended recipient please note that any form of distribution, copying or use of this communication or the information in it is strictly prohibited and may be unlawful. If you receive this message in error, please delete it and notify the sender. Please consider the environment before printing this e-mail This message from ORIX Australia may contain confidential and/or privileged information. If you are not the intended recipient, any use, disclosure or copying of this message (or of any attachments to it) is not authorised. If you have received this message in error, please notify the sender immediately and delete the message and any attachments from your system. Please inform the sender if you do not wish to receive further communications by email. ORIX handles personal information according to a Privacy Policy that is consistent with the National Privacy Principles. Please let us know if you would like a copy. It is also available at www.orix.com.au<http://www.orix.com.au>;
-- David Baldwin - Senior Systems Administrator (Datacentres + Networks) Information and Communication Technology Services Australian Sports Commission http://ausport.gov.au Tel 02 62147830 Fax 02 62141830 PO Box 176 Belconnen ACT 2616 user-cbbf693f2c89@xymon.invalid Leverrier Street Bruce ACT 2617
list Martin Blomdahl
Hi Have a look at the last post in this thread http://sourceforge.net/p/bbwin/discussion/460874/thread/04a00deb/?limit=50 I have tested it, IT WORKS , and it explains HOW ! Martin ( who have been struggling with this a lot, whithout success)
▸
Phil,Thanks, David.... Yes, I don't have clientversion.cfg either - I tried putting in "0.13" in a file of that name in \etc, the client says it reads it and no more error, but who knows what this does or doesn't do... Anyway, on to the rest. As I described in the previous email, I did let the clientlocal.cfg update successfully. But as soon as I add the msgs.dll agent bbwin still ignores the maxdata parameter, tries to send everything and xymond drops the connection before completion (this latest time at 31MB). Do you have this working? Does it actually reduce the size of the msgs sent?
I have it working, but I don't actually use BBWin for event log reporting - I forward all my event logs using Snare to a central syslog server and have my own checking against that server-side - see http://xymonton.org/monitors:winevtmsgs.pl My client-local.cfg for Windows clients includes the following to filter out everything client-side. You may be able to play with the ignore rules to tune it a bit more to your needs while still reducing the size of the report. By checking the BBWin\tmp\msg.<clientname>.txt file you should see what is taking up all the space. log:eventlog_security:10240 ignore .* ignore . eventlog:security:10240 ignore handle ignore .* ignore . eventlog:System:10240 ignore .* ignore . eventlog:application:10240 ignore .* ignore . eventlog:directory service:10240 ignore .* ignore . David.
thanks, Phil -- *From:* David Baldwin *Sent:* Thursday, 11 July 2013 11:28 AM *To:* Phil Crooker *Cc:* xymon at xymon.com *Subject:* Re: [Xymon] problem with receiving bbwin data Phil,Thanks for the reply. Before continuing, what is supposed to be in logfetch.status? and in clientversion.cfg?logfetch.status will possibly only exist if you are checking any regular logfiles - e.g. if client-local.cfg contains something like: [win32] log:c:\WINDOWS\WindowsUpdate.log:20480 clientlocal.cfg will contain the relevant section similar to your 4 line example below. clientversion.cfg doesn't exist on my systems As BBWin runs, check the size of clientlocal.cfg - if it gets wiped, try saving the contents back again. Also, updating BBWin.cfg should cause BBWin to run immediately without restarting the service. I haven't played around with commenting out msgs.dll David.So, I have one server with the msgs.dll out of the config. It has been running for many days with the client updating fine. I redid the xymon client-local.cfg, stopped bbwin, deleted \tmp\clientlocal.cfg and restarted it (as it wouldn't update), the \tmp\clientlocal.cfg now reads what the client-local has: eventlog:application:1024 ignore BigBrotherXymonClient eventlog:system:1025 eventlog:security:10240 I let that run a cycle, it still updates OK. I then put in the msgs.dll statement in BBWin.cfg; that registers and BBWin.log shows: 2013/07/11 10:42:55 [DEBUG]: [bbwinupdate]: can't open C:\Program Files (x86)\BBWin\etc\clientversion.cfg : The system cannot find the file specified. 2013/07/11 10:42:55 [DEBUG]: [cpu]: cpu review started 2013/07/11 10:42:55 [DEBUG]: [cpu]: cpu review ended 2013/07/11 10:42:55 [DEBUG]: [disk]: drive string A:\ 2013/07/11 10:42:55 [DEBUG]: [disk]: drive string C:\ 2013/07/11 10:42:55 [DEBUG]: [disk]: drive string D:\ 2013/07/11 10:42:55 [DEBUG]: [disk]: drive string E:\ 2013/07/11 10:42:55 [DEBUG]: [filesystem]: start AgentFileSystem::InitCentralMode 2013/07/11 10:42:55 [DEBUG]: [filesystem]: checking file C:\Program Files (x86)\BBWin\tmp\clientlocal.cfg 2013/07/11 10:42:55 [DEBUG]: [filesystem]: reading file C:\Program Files (x86)\BBWin\tmp\clientlocal.cfg 2013/07/11 10:42:55 [DEBUG]: [msgs]: start AgentMsgs::InitCentralMode 2013/07/11 10:42:55 [DEBUG]: [msgs]: checking file C:\Program Files (x86)\BBWin\tmp\clientlocal.cfg 2013/07/11 10:42:55 [DEBUG]: [msgs]: reading file C:\Program Files (x86)\BBWin\tmp\clientlocal.cfg 2013/07/11 10:42:55 [DEBUG]: [msgs]: will use maxdata size : 1024 2013/07/11 10:42:55 [DEBUG]: [msgs]: will ignore : BigBrotherXymonClient 2013/07/11 10:42:55 [DEBUG]: [msgs]: will use maxdata size : 1025 2013/07/11 10:42:55 [DEBUG]: [msgs]: will use maxdata size : 10240 and no more updates. The .\tmp\msgs.hostname.txt file is 88MB. thanks, Phil --- *From:* Scot Kreienkamp *Sent:* Wednesday, 10 July 2013 10:19 PM *To:* David Baldwin; Phil Crooker *Cc:* xymon at xymon.com *Subject:* RE: [Xymon] problem with receiving bbwin data If the eventlog data being transmitted is too large to transmit until the clientlocal.cfg is sent back and populated in BBWin, then why not disable the eventlog client in BBWin until it successfully sends 2-3 status messages? By that time the clientlocal.cfg will be populated on the client and you can enable the eventlog reader again and everything should work fine. *Scot Kreienkamp * *From:*Xymon [mailto:xymon-bounces at xymon.com] *On Behalf Of *David Baldwin *Sent:* Wednesday, July 10, 2013 4:19 AM *To:* Phil Crooker *Cc:* xymon at xymon.com *Subject:* Re: [Xymon] problem with receiving bbwin data Phil, Just found a fix that works for me :) I've been having difficulties setting up the bbwin client (ver 0.13 going to xymon 4.3.10 server) running on win7 sp1, 2008r2 and win2012 boxes in central mode. I've had zero response from the bbwin-help forum for this, so please bear with me. For background: I can't get the bbwin client to stop sending all the logs, it ignores any maxdata parameters I use. Eg: eventlog:system:1024 It sends everything anyway. If I use log:system:1024, the bbwin client throws an error that it can't find the system log file. It does find the file without the maxdata parameter. One problem with the settings in client-local.cfg on the xymon server is that they only propagate to the client after a successful sending of a client message. The mechanics is that after the client message has been sent, BBWin does a shutdown on the write socket and then reads from the read socket until EOF. If the write socket is never shutdown because the server breaks the connection due to flooding, then the client-local.cfg section will never be sent to the client. I've previously attempted to manually create C:\Program Files(x86)\BBWin\tmp\clientlocal.cfg to trick it, but with limited success. Just played around with it again - with client reporting to my primary server I just got it going - previously I was reporting to 2 servers. The trick seems to be in having logfetch.status also created. I then modified my BBWin.cfg back to 2 servers and it broke again. Maybe the 2nd server is responding differently. It is undefined which server's clientlocal.cfg to believe anyway :) Swapped the order of the servers around and it seems stable. If the server just discarded all the flooding data beyond MAXMSG_* and then returned the client config section anyway maybe it could be made to work, but there may be good reasons why that would not be sensible. This aside, I do need to get this working as I have to start monitoring 10 new windows servers and have to monitor the event logs, so I can't just stop sending them as has been suggested. Yes I could do some powershell scripts and send them via bbwincmd but the bbwin client is made for this task..... So, looking at this from the other side, the xymon server appears to be resetting the session after about 22MB of data has been sent (I know, this is ludicrous, but it is windows). Nothing in the xymond logs (except for the occasional data flooding error ("1st line client", always)); on the client side it reports it can't send the data to the xymon server. Depending on what auditting you have enabled, 22MB is very easy to achieve! Turn on Security event logging including success and failure and it's almost guaranteed :) David. I've set the MAXMSG_* to quite silly levels: # ipcs ------ Shared Memory Segments -------- key shmid owner perms bytes nattch status 0x01034be7 16908288 xymon 600 102400000 2 0x02034be7 16941057 xymon 600 102400000 2 0x03034be7 16973826 xymon 600 102400000 2 0x04034be7 17006595 xymon 600 102400000 2 0x05034be7 17039364 xymon 600 262144 1 0x06034be7 17072133 xymon 600 32768 1 0x07034be7 17104902 xymon 600 102400000 2 0x08034be7 17137671 xymon 600 102400000 2 0x09034be7 17170440 xymon 600 131072 1 Anything up to and including this size has no effect on the problem. Looking at the tcpdump stream, the bbwin client sends data normally with regular ACKs from xymond till around that 22MB mark then xymond responds with a FIN packet, then with RST packets and the session shuts down. Nothing in the packets themselves indicate what the problem is. If anyone can help with this, please, it would be great. thanks, Phil
-- David Baldwin - Senior Systems Administrator (Datacentres + Networks) Information and Communication Technology Services Australian Sports Commission http://ausport.gov.au Tel 02 62147830 Fax 02 62141830 PO Box 176 Belconnen ACT 2616 user-cbbf693f2c89@xymon.invalid Leverrier Street Bruce ACT 2617 Keep up to date with what's happening in Australian sport visit http://www.ausport.gov.au This message is intended for the addressee named and may contain confidential and privileged information. If you are not the intended recipient please note that any form of distribution, copying or use of this communication or the information in it is strictly prohibited and may be unlawful. If you receive this message in error, please delete it and notify the sender.
list Phil Crooker
Hi Martin, Yes, I've seen that post and used those settings and they don't work - eg I tried setting the maxdata to say 1024 bytes and the client definitely sends more than that. It is sending 80-odd MB when I've limited it to under 20K (we are generating a lot of logs at the moment). If it isn't too much trouble, could you pls try setting maxdata to a small number and see whether it is truncated on the respective msgs page? You'll have to check the local copy of clientlocal.cfg (under \tmp) to make sure it has updated. I've had to shut down the client, delete that file and then restart it to force it. cheers, Phil
▸
From: Martin Blomdahl Sent: Thursday, 11 July 2013 6:43 PM To: David Baldwin; Phil Crooker Cc: xymon at xymon.com Subject: Re: [Xymon] problem with receiving bbwin data Hi Have a look at the last post in this thread http://sourceforge.net/p/bbwin/discussion/460874/thread/04a00deb/?limit=50 I have tested it, IT WORKS , and it explains HOW ! Martin ( who have been struggling with this a lot, whithout success) Phil,
Thanks, David.... Yes, I don't have clientversion.cfg either - I tried putting in "0.13" in a file of that name in \etc, the client says it reads it and no more error, but who knows what this does or doesn't do... Anyway, on to the rest. As I described in the previous email, I did let the clientlocal.cfg update successfully. But as soon as I add the msgs.dll agent bbwin still ignores the maxdata parameter, tries to send everything and xymond drops the connection before completion (this latest time at 31MB). Do you have this working? Does it actually reduce the size of the msgs sent?
I have it working, but I don't actually use BBWin for event log reporting - I forward all my event logs using Snare to a central syslog server and have my own checking against that server-side - see http://xymonton.org/monitors:winevtmsgs.pl My client-local.cfg for Windows clients includes the following to filter out everything client-side. You may be able to play with the ignore rules to tune it a bit more to your needs while still reducing the size of the report. By checking the BBWin\tmp\msg.<clientname>.txt file you should see what is taking up all the space. log:eventlog_security:10240 ignore .* ignore . eventlog:security:10240 ignore handle ignore .* ignore . eventlog:System:10240 ignore .* ignore . eventlog:application:10240 ignore .* ignore . eventlog:directory service:10240 ignore .* ignore . David.
thanks, Phil -- *From:* David Baldwin *Sent:* Thursday, 11 July 2013 11:28 AM *To:* Phil Crooker *Cc:* xymon at xymon.com *Subject:* Re: [Xymon] problem with receiving bbwin data Phil,Thanks for the reply. Before continuing, what is supposed to be in logfetch.status? and in clientversion.cfg?logfetch.status will possibly only exist if you are checking any regular logfiles - e.g. if client-local.cfg contains something like: [win32] log:c:\WINDOWS\WindowsUpdate.log:20480 clientlocal.cfg will contain the relevant section similar to your 4 line example below. clientversion.cfg doesn't exist on my systems As BBWin runs, check the size of clientlocal.cfg - if it gets wiped, try saving the contents back again. Also, updating BBWin.cfg should cause BBWin to run immediately without restarting the service. I haven't played around with commenting out msgs.dll David.So, I have one server with the msgs.dll out of the config. It has been running for many days with the client updating fine. I redid the xymon client-local.cfg, stopped bbwin, deleted \tmp\clientlocal.cfg and restarted it (as it wouldn't update), the \tmp\clientlocal.cfg now reads what the client-local has: eventlog:application:1024 ignore BigBrotherXymonClient eventlog:system:1025 eventlog:security:10240 I let that run a cycle, it still updates OK. I then put in the msgs.dll statement in BBWin.cfg; that registers and BBWin.log shows: 2013/07/11 10:42:55 [DEBUG]: [bbwinupdate]: can't open C:\Program Files (x86)\BBWin\etc\clientversion.cfg : The system cannot find the file specified. 2013/07/11 10:42:55 [DEBUG]: [cpu]: cpu review started 2013/07/11 10:42:55 [DEBUG]: [cpu]: cpu review ended 2013/07/11 10:42:55 [DEBUG]: [disk]: drive string A:\ 2013/07/11 10:42:55 [DEBUG]: [disk]: drive string C:\ 2013/07/11 10:42:55 [DEBUG]: [disk]: drive string D:\ 2013/07/11 10:42:55 [DEBUG]: [disk]: drive string E:\ 2013/07/11 10:42:55 [DEBUG]: [filesystem]: start AgentFileSystem::InitCentralMode 2013/07/11 10:42:55 [DEBUG]: [filesystem]: checking file C:\Program Files (x86)\BBWin\tmp\clientlocal.cfg 2013/07/11 10:42:55 [DEBUG]: [filesystem]: reading file C:\Program Files (x86)\BBWin\tmp\clientlocal.cfg 2013/07/11 10:42:55 [DEBUG]: [msgs]: start AgentMsgs::InitCentralMode 2013/07/11 10:42:55 [DEBUG]: [msgs]: checking file C:\Program Files (x86)\BBWin\tmp\clientlocal.cfg 2013/07/11 10:42:55 [DEBUG]: [msgs]: reading file C:\Program Files (x86)\BBWin\tmp\clientlocal.cfg 2013/07/11 10:42:55 [DEBUG]: [msgs]: will use maxdata size : 1024 2013/07/11 10:42:55 [DEBUG]: [msgs]: will ignore : BigBrotherXymonClient 2013/07/11 10:42:55 [DEBUG]: [msgs]: will use maxdata size : 1025 2013/07/11 10:42:55 [DEBUG]: [msgs]: will use maxdata size : 10240 and no more updates. The .\tmp\msgs.hostname.txt file is 88MB. thanks, Phil --- *From:* Scot Kreienkamp *Sent:* Wednesday, 10 July 2013 10:19 PM *To:* David Baldwin; Phil Crooker *Cc:* xymon at xymon.com *Subject:* RE: [Xymon] problem with receiving bbwin data If the eventlog data being transmitted is too large to transmit until the clientlocal.cfg is sent back and populated in BBWin, then why not disable the eventlog client in BBWin until it successfully sends 2-3 status messages? By that time the clientlocal.cfg will be populated on the client and you can enable the eventlog reader again and everything should work fine. *Scot Kreienkamp * *From:*Xymon [mailto:xymon-bounces at xymon.com] *On Behalf Of *David Baldwin *Sent:* Wednesday, July 10, 2013 4:19 AM *To:* Phil Crooker *Cc:* xymon at xymon.com *Subject:* Re: [Xymon] problem with receiving bbwin data Phil, Just found a fix that works for me :) I've been having difficulties setting up the bbwin client (ver 0.13 going to xymon 4.3.10 server) running on win7 sp1, 2008r2 and win2012 boxes in central mode. I've had zero response from the bbwin-help forum for this, so please bear with me. For background: I can't get the bbwin client to stop sending all the logs, it ignores any maxdata parameters I use. Eg: eventlog:system:1024 It sends everything anyway. If I use log:system:1024, the bbwin client throws an error that it can't find the system log file. It does find the file without the maxdata parameter. One problem with the settings in client-local.cfg on the xymon server is that they only propagate to the client after a successful sending of a client message. The mechanics is that after the client message has been sent, BBWin does a shutdown on the write socket and then reads from the read socket until EOF. If the write socket is never shutdown because the server breaks the connection due to flooding, then the client-local.cfg section will never be sent to the client. I've previously attempted to manually create C:\Program Files(x86)\BBWin\tmp\clientlocal.cfg to trick it, but with limited success. Just played around with it again - with client reporting to my primary server I just got it going - previously I was reporting to 2 servers. The trick seems to be in having logfetch.status also created. I then modified my BBWin.cfg back to 2 servers and it broke again. Maybe the 2nd server is responding differently. It is undefined which server's clientlocal.cfg to believe anyway :) Swapped the order of the servers around and it seems stable. If the server just discarded all the flooding data beyond MAXMSG_* and then returned the client config section anyway maybe it could be made to work, but there may be good reasons why that would not be sensible. This aside, I do need to get this working as I have to start monitoring 10 new windows servers and have to monitor the event logs, so I can't just stop sending them as has been suggested. Yes I could do some powershell scripts and send them via bbwincmd but the bbwin client is made for this task..... So, looking at this from the other side, the xymon server appears to be resetting the session after about 22MB of data has been sent (I know, this is ludicrous, but it is windows). Nothing in the xymond logs (except for the occasional data flooding error ("1st line client", always)); on the client side it reports it can't send the data to the xymon server. Depending on what auditting you have enabled, 22MB is very easy to achieve! Turn on Security event logging including success and failure and it's almost guaranteed :) David. I've set the MAXMSG_* to quite silly levels: # ipcs ------ Shared Memory Segments -------- key shmid owner perms bytes nattch status 0x01034be7 16908288 xymon 600 102400000 2 0x02034be7 16941057 xymon 600 102400000 2 0x03034be7 16973826 xymon 600 102400000 2 0x04034be7 17006595 xymon 600 102400000 2 0x05034be7 17039364 xymon 600 262144 1 0x06034be7 17072133 xymon 600 32768 1 0x07034be7 17104902 xymon 600 102400000 2 0x08034be7 17137671 xymon 600 102400000 2 0x09034be7 17170440 xymon 600 131072 1 Anything up to and including this size has no effect on the problem. Looking at the tcpdump stream, the bbwin client sends data normally with regular ACKs from xymond till around that 22MB mark then xymond responds with a FIN packet, then with RST packets and the session shuts down. Nothing in the packets themselves indicate what the problem is. If anyone can help with this, please, it would be great. thanks, Phil
-- David Baldwin - Senior Systems Administrator (Datacentres + Networks) Information and Communication Technology Services Australian Sports Commission http://ausport.gov.au Tel 02 62147830 Fax 02 62141830 PO Box 176 Belconnen ACT 2616 user-cbbf693f2c89@xymon.invalid Leverrier Street Bruce ACT 2617 Keep up to date with what's happening in Australian sport visit http://www.ausport.gov.au This message is intended for the addressee named and may contain confidential and privileged information. If you are not the intended recipient please note that any form of distribution, copying or use of this communication or the information in it is strictly prohibited and may be unlawful. If you receive this message in error, please delete it and notify the sender. Please consider the environment before printing this e-mail This message from ORIX Australia may contain confidential and/or privileged information. If you are not the intended recipient, any use, disclosure or copying of this message (or of any attachments to it) is not authorised. If you have received this message in error, please notify the sender immediately and delete the message and any attachments from your system. Please inform the sender if you do not wish to receive further communications by email. ORIX handles personal information according to a Privacy Policy that is consistent with the National Privacy Principles. Please let us know if you would like a copy. It is also available at www.orix.com.au<http://www.orix.com.au>;
list Martin Blomdahl
Phil You are absolutely right, the maxdata does not work, but everything else do. I got at first 10MB from my DC, but after putting a lot of ignore clauses, I got It down to 1MB. So that part works. (hadent come so far previous) I have an consultant who is good on C++, I will let him have a look at it, with low priority, in August. Cheers, Martin -----Ursprungligt meddelande----- Från: Phil Crooker [mailto:user-e8e31cd73303@xymon.invalid] Skickat: den 12 juli 2013 01:45 Till: Blomdahl, Martin; David Baldwin Kopia: xymon at xymon.com Ämne: RE: [Xymon] problem with receiving bbwin data
▸
Hi Martin, Yes, I've seen that post and used those settings and they don't work - eg I tried setting the maxdata to say 1024 bytes and the client definitely sends more than that. It is sending 80-odd MB when I've limited it to under 20K (we are generating a lot of logs at the moment). If it isn't too much trouble, could you pls try setting maxdata to a small number and see whether it is truncated on the respective msgs page? You'll have to check the local copy of clientlocal.cfg (under \tmp) to make sure it has updated. I've had to shut down the client, delete that file and then restart it to force it. cheers, Phil From: Martin Blomdahl Sent: Thursday, 11 July 2013 6:43 PM To: David Baldwin; Phil Crooker Cc: xymon at xymon.com Subject: Re: [Xymon] problem with receiving bbwin data Hi Have a look at the last post in this thread http://sourceforge.net/p/bbwin/discussion/460874/thread/04a00deb/?limit=50 I have tested it, IT WORKS , and it explains HOW ! Martin ( who have been struggling with this a lot, whithout success) Phil,
Thanks, David.... Yes, I don't have clientversion.cfg either - I tried putting in "0.13" in a file of that name in \etc, the client says it reads it and no more error, but who knows what this does or doesn't do... Anyway, on to the rest. As I described in the previous email, I did let the clientlocal.cfg update successfully. But as soon as I add the msgs.dll agent bbwin still ignores the maxdata parameter, tries to send everything and xymond drops the connection before completion (this latest time at 31MB). Do you have this working? Does it actually reduce the size of the msgs sent?
I have it working, but I don't actually use BBWin for event log reporting - I forward all my event logs using Snare to a central syslog server and have my own checking against that server-side - see http://xymonton.org/monitors:winevtmsgs.pl My client-local.cfg for Windows clients includes the following to filter out everything client-side. You may be able to play with the ignore rules to tune it a bit more to your needs while still reducing the size of the report. By checking the BBWin\tmp\msg.<clientname>.txt file you should see what is taking up all the space. log:eventlog_security:10240 ignore .* ignore . eventlog:security:10240 ignore handle ignore .* ignore . eventlog:System:10240 ignore .* ignore . eventlog:application:10240 ignore .* ignore . eventlog:directory service:10240 ignore .* ignore . David.
thanks, Phil -- *From:* David Baldwin *Sent:* Thursday, 11 July 2013 11:28 AM *To:* Phil Crooker *Cc:* xymon at xymon.com *Subject:* Re: [Xymon] problem with receiving bbwin data Phil,Thanks for the reply. Before continuing, what is supposed to be in logfetch.status? and in clientversion.cfg?logfetch.status will possibly only exist if you are checking any regular logfiles - e.g. if client-local.cfg contains something like: [win32] log:c:\WINDOWS\WindowsUpdate.log:20480 clientlocal.cfg will contain the relevant section similar to your 4 line example below. clientversion.cfg doesn't exist on my systems As BBWin runs, check the size of clientlocal.cfg - if it gets wiped, try saving the contents back again. Also, updating BBWin.cfg should cause BBWin to run immediately without restarting the service. I haven't played around with commenting out msgs.dll David.So, I have one server with the msgs.dll out of the config. It has been running for many days with the client updating fine. I redid the xymon client-local.cfg, stopped bbwin, deleted \tmp\clientlocal.cfg and restarted it (as it wouldn't update), the \tmp\clientlocal.cfg now reads what the client-local has: eventlog:application:1024 ignore BigBrotherXymonClient eventlog:system:1025 eventlog:security:10240 I let that run a cycle, it still updates OK. I then put in the msgs.dll statement in BBWin.cfg; that registers and BBWin.log shows: 2013/07/11 10:42:55 [DEBUG]: [bbwinupdate]: can't open C:\Program Files (x86)\BBWin\etc\clientversion.cfg : The system cannot find the file specified. 2013/07/11 10:42:55 [DEBUG]: [cpu]: cpu review started 2013/07/11 10:42:55 [DEBUG]: [cpu]: cpu review ended 2013/07/11 10:42:55 [DEBUG]: [disk]: drive string A:\ 2013/07/11 10:42:55 [DEBUG]: [disk]: drive string C:\ 2013/07/11 10:42:55 [DEBUG]: [disk]: drive string D:\ 2013/07/11 10:42:55 [DEBUG]: [disk]: drive string E:\ 2013/07/11 10:42:55 [DEBUG]: [filesystem]: start AgentFileSystem::InitCentralMode 2013/07/11 10:42:55 [DEBUG]: [filesystem]: checking file C:\Program Files (x86)\BBWin\tmp\clientlocal.cfg 2013/07/11 10:42:55 [DEBUG]: [filesystem]: reading file C:\Program Files (x86)\BBWin\tmp\clientlocal.cfg 2013/07/11 10:42:55 [DEBUG]: [msgs]: start AgentMsgs::InitCentralMode 2013/07/11 10:42:55 [DEBUG]: [msgs]: checking file C:\Program Files (x86)\BBWin\tmp\clientlocal.cfg 2013/07/11 10:42:55 [DEBUG]: [msgs]: reading file C:\Program Files (x86)\BBWin\tmp\clientlocal.cfg 2013/07/11 10:42:55 [DEBUG]: [msgs]: will use maxdata size : 1024 2013/07/11 10:42:55 [DEBUG]: [msgs]: will ignore : BigBrotherXymonClient 2013/07/11 10:42:55 [DEBUG]: [msgs]: will use maxdata size : 1025 2013/07/11 10:42:55 [DEBUG]: [msgs]: will use maxdata size : 10240 and no more updates. The .\tmp\msgs.hostname.txt file is 88MB. thanks, Phil --- *From:* Scot Kreienkamp *Sent:* Wednesday, 10 July 2013 10:19 PM *To:* David Baldwin; Phil Crooker *Cc:* xymon at xymon.com *Subject:* RE: [Xymon] problem with receiving bbwin data If the eventlog data being transmitted is too large to transmit until the clientlocal.cfg is sent back and populated in BBWin, then why not disable the eventlog client in BBWin until it successfully sends 2-3 status messages? By that time the clientlocal.cfg will be populated on the client and you can enable the eventlog reader again and everything should work fine. *Scot Kreienkamp * *From:*Xymon [mailto:xymon-bounces at xymon.com] *On Behalf Of *David Baldwin *Sent:* Wednesday, July 10, 2013 4:19 AM *To:* Phil Crooker *Cc:* xymon at xymon.com *Subject:* Re: [Xymon] problem with receiving bbwin data Phil, Just found a fix that works for me :) I've been having difficulties setting up the bbwin client (ver 0.13 going to xymon 4.3.10 server) running on win7 sp1, 2008r2 and win2012 boxes in central mode. I've had zero response from the bbwin-help forum for this, so please bear with me. For background: I can't get the bbwin client to stop sending all the logs, it ignores any maxdata parameters I use. Eg: eventlog:system:1024 It sends everything anyway. If I use log:system:1024, the bbwin client throws an error that it can't find the system log file. It does find the file without the maxdata parameter. One problem with the settings in client-local.cfg on the xymon server is that they only propagate to the client after a successful sending of a client message. The mechanics is that after the client message has been sent, BBWin does a shutdown on the write socket and then reads from the read socket until EOF. If the write socket is never shutdown because the server breaks the connection due to flooding, then the client-local.cfg section will never be sent to the client. I've previously attempted to manually create C:\Program Files(x86)\BBWin\tmp\clientlocal.cfg to trick it, but with limited success. Just played around with it again - with client reporting to my primary server I just got it going - previously I was reporting to 2 servers. The trick seems to be in having logfetch.status also created. I then modified my BBWin.cfg back to 2 servers and it broke again. Maybe the 2nd server is responding differently. It is undefined which server's clientlocal.cfg to believe anyway :) Swapped the order of the servers around and it seems stable. If the server just discarded all the flooding data beyond MAXMSG_* and then returned the client config section anyway maybe it could be made to work, but there may be good reasons why that would not be sensible. This aside, I do need to get this working as I have to start monitoring 10 new windows servers and have to monitor the event logs, so I can't just stop sending them as has been suggested. Yes I could do some powershell scripts and send them via bbwincmd but the bbwin client is made for this task..... So, looking at this from the other side, the xymon server appears to be resetting the session after about 22MB of data has been sent (I know, this is ludicrous, but it is windows). Nothing in the xymond logs (except for the occasional data flooding error ("1st line client", always)); on the client side it reports it can't send the data to the xymon server. Depending on what auditting you have enabled, 22MB is very easy to achieve! Turn on Security event logging including success and failure and it's almost guaranteed :) David. I've set the MAXMSG_* to quite silly levels: # ipcs ------ Shared Memory Segments -------- key shmid owner perms bytes nattch status 0x01034be7 16908288 xymon 600 102400000 2 0x02034be7 16941057 xymon 600 102400000 2 0x03034be7 16973826 xymon 600 102400000 2 0x04034be7 17006595 xymon 600 102400000 2 0x05034be7 17039364 xymon 600 262144 1 0x06034be7 17072133 xymon 600 32768 1 0x07034be7 17104902 xymon 600 102400000 2 0x08034be7 17137671 xymon 600 102400000 2 0x09034be7 17170440 xymon 600 131072 1 Anything up to and including this size has no effect on the problem. Looking at the tcpdump stream, the bbwin client sends data normally with regular ACKs from xymond till around that 22MB mark then xymond responds with a FIN packet, then with RST packets and the session shuts down. Nothing in the packets themselves indicate what the problem is. If anyone can help with this, please, it would be great. thanks, Phil
-- David Baldwin - Senior Systems Administrator (Datacentres + Networks) Information and Communication Technology Services Australian Sports Commission http://ausport.gov.au Tel 02 62147830 Fax 02 62141830 PO Box 176 Belconnen ACT 2616 user-cbbf693f2c89@xymon.invalid Leverrier Street Bruce ACT 2617 Keep up to date with what's happening in Australian sport visit http://www.ausport.gov.au This message is intended for the addressee named and may contain confidential and privileged information. If you are not the intended recipient please note that any form of distribution, copying or use of this communication or the information in it is strictly prohibited and may be unlawful. If you receive this message in error, please delete it and notify the sender. Please consider the environment before printing this e-mail This message from ORIX Australia may contain confidential and/or privileged information. If you are not the intended recipient, any use, disclosure or copying of this message (or of any attachments to it) is not authorised. If you have received this message in error, please notify the sender immediately and delete the message and any attachments from your system. Please inform the sender if you do not wish to receive further communications by email. ORIX handles personal information according to a Privacy Policy that is consistent with the National Privacy Principles. Please let us know if you would like a copy. It is also available at www.orix.com.au<http://www.orix.com.au>;
list Phil Crooker
Hi Martin, Thanks so much for confirming this. I'll try the ignore statements. It would be wonderful to get the maxdata working - I was thinking of asking Etienne about it - the company is so dependent on xymon now. Also there is David Baldwin's perl script (and snare), so some options now. cheers, P
▸
From: Martin Blomdahl Sent: Friday, 12 July 2013 11:44 PM To: David Baldwin; Phil Crooker Cc: xymon at xymon.com Subject: Re: [Xymon] problem with receiving bbwin data Phil You are absolutely right, the maxdata does not work, but everything else do. I got at first 10MB from my DC, but after putting a lot of ignore clauses, I got It down to 1MB. So that part works. (hadent come so far previous) I have an consultant who is good on C++, I will let him have a look at it, with low priority, in August. Cheers, Martin -----Ursprungligt meddelande----- Från: Phil Crooker [mailto:user-e8e31cd73303@xymon.invalid] Skickat: den 12 juli 2013 01:45 Till: Blomdahl, Martin; David Baldwin Kopia: xymon at xymon.com Ämne: RE: [Xymon] problem with receiving bbwin data Hi Martin, Yes, I've seen that post and used those settings and they don't work - eg I tried setting the maxdata to say 1024 bytes and the client definitely sends more than that. It is sending 80-odd MB when I've limited it to under 20K (we are generating a lot of logs at the moment). If it isn't too much trouble, could you pls try setting maxdata to a small number and see whether it is truncated on the respective msgs page? You'll have to check the local copy of clientlocal.cfg (under \tmp) to make sure it has updated. I've had to shut down the client, delete that file and then restart it to force it. cheers, Phil From: Martin Blomdahl Sent: Thursday, 11 July 2013 6:43 PM To: David Baldwin; Phil Crooker Cc: xymon at xymon.com Subject: Re: [Xymon] problem with receiving bbwin data Hi Have a look at the last post in this thread http://sourceforge.net/p/bbwin/discussion/460874/thread/04a00deb/?limit=50 I have tested it, IT WORKS , and it explains HOW ! Martin ( who have been struggling with this a lot, whithout success) Phil,
Thanks, David.... Yes, I don't have clientversion.cfg either - I tried putting in "0.13" in a file of that name in \etc, the client says it reads it and no more error, but who knows what this does or doesn't do... Anyway, on to the rest. As I described in the previous email, I did let the clientlocal.cfg update successfully. But as soon as I add the msgs.dll agent bbwin still ignores the maxdata parameter, tries to send everything and xymond drops the connection before completion (this latest time at 31MB). Do you have this working? Does it actually reduce the size of the msgs sent?
I have it working, but I don't actually use BBWin for event log reporting - I forward all my event logs using Snare to a central syslog server and have my own checking against that server-side - see http://xymonton.org/monitors:winevtmsgs.pl My client-local.cfg for Windows clients includes the following to filter out everything client-side. You may be able to play with the ignore rules to tune it a bit more to your needs while still reducing the size of the report. By checking the BBWin\tmp\msg.<clientname>.txt file you should see what is taking up all the space. log:eventlog_security:10240 ignore .* ignore . eventlog:security:10240 ignore handle ignore .* ignore . eventlog:System:10240 ignore .* ignore . eventlog:application:10240 ignore .* ignore . eventlog:directory service:10240 ignore .* ignore . David.
thanks, Phil -- *From:* David Baldwin *Sent:* Thursday, 11 July 2013 11:28 AM *To:* Phil Crooker *Cc:* xymon at xymon.com *Subject:* Re: [Xymon] problem with receiving bbwin data Phil,Thanks for the reply. Before continuing, what is supposed to be in logfetch.status? and in clientversion.cfg?logfetch.status will possibly only exist if you are checking any regular logfiles - e.g. if client-local.cfg contains something like: [win32] log:c:\WINDOWS\WindowsUpdate.log:20480 clientlocal.cfg will contain the relevant section similar to your 4 line example below. clientversion.cfg doesn't exist on my systems As BBWin runs, check the size of clientlocal.cfg - if it gets wiped, try saving the contents back again. Also, updating BBWin.cfg should cause BBWin to run immediately without restarting the service. I haven't played around with commenting out msgs.dll David.So, I have one server with the msgs.dll out of the config. It has been running for many days with the client updating fine. I redid the xymon client-local.cfg, stopped bbwin, deleted \tmp\clientlocal.cfg and restarted it (as it wouldn't update), the \tmp\clientlocal.cfg now reads what the client-local has: eventlog:application:1024 ignore BigBrotherXymonClient eventlog:system:1025 eventlog:security:10240 I let that run a cycle, it still updates OK. I then put in the msgs.dll statement in BBWin.cfg; that registers and BBWin.log shows: 2013/07/11 10:42:55 [DEBUG]: [bbwinupdate]: can't open C:\Program Files (x86)\BBWin\etc\clientversion.cfg : The system cannot find the file specified. 2013/07/11 10:42:55 [DEBUG]: [cpu]: cpu review started 2013/07/11 10:42:55 [DEBUG]: [cpu]: cpu review ended 2013/07/11 10:42:55 [DEBUG]: [disk]: drive string A:\ 2013/07/11 10:42:55 [DEBUG]: [disk]: drive string C:\ 2013/07/11 10:42:55 [DEBUG]: [disk]: drive string D:\ 2013/07/11 10:42:55 [DEBUG]: [disk]: drive string E:\ 2013/07/11 10:42:55 [DEBUG]: [filesystem]: start AgentFileSystem::InitCentralMode 2013/07/11 10:42:55 [DEBUG]: [filesystem]: checking file C:\Program Files (x86)\BBWin\tmp\clientlocal.cfg 2013/07/11 10:42:55 [DEBUG]: [filesystem]: reading file C:\Program Files (x86)\BBWin\tmp\clientlocal.cfg 2013/07/11 10:42:55 [DEBUG]: [msgs]: start AgentMsgs::InitCentralMode 2013/07/11 10:42:55 [DEBUG]: [msgs]: checking file C:\Program Files (x86)\BBWin\tmp\clientlocal.cfg 2013/07/11 10:42:55 [DEBUG]: [msgs]: reading file C:\Program Files (x86)\BBWin\tmp\clientlocal.cfg 2013/07/11 10:42:55 [DEBUG]: [msgs]: will use maxdata size : 1024 2013/07/11 10:42:55 [DEBUG]: [msgs]: will ignore : BigBrotherXymonClient 2013/07/11 10:42:55 [DEBUG]: [msgs]: will use maxdata size : 1025 2013/07/11 10:42:55 [DEBUG]: [msgs]: will use maxdata size : 10240 and no more updates. The .\tmp\msgs.hostname.txt file is 88MB. thanks, Phil --- *From:* Scot Kreienkamp *Sent:* Wednesday, 10 July 2013 10:19 PM *To:* David Baldwin; Phil Crooker *Cc:* xymon at xymon.com *Subject:* RE: [Xymon] problem with receiving bbwin data If the eventlog data being transmitted is too large to transmit until the clientlocal.cfg is sent back and populated in BBWin, then why not disable the eventlog client in BBWin until it successfully sends 2-3 status messages? By that time the clientlocal.cfg will be populated on the client and you can enable the eventlog reader again and everything should work fine. *Scot Kreienkamp * *From:*Xymon [mailto:xymon-bounces at xymon.com] *On Behalf Of *David Baldwin *Sent:* Wednesday, July 10, 2013 4:19 AM *To:* Phil Crooker *Cc:* xymon at xymon.com *Subject:* Re: [Xymon] problem with receiving bbwin data Phil, Just found a fix that works for me :) I've been having difficulties setting up the bbwin client (ver 0.13 going to xymon 4.3.10 server) running on win7 sp1, 2008r2 and win2012 boxes in central mode. I've had zero response from the bbwin-help forum for this, so please bear with me. For background: I can't get the bbwin client to stop sending all the logs, it ignores any maxdata parameters I use. Eg: eventlog:system:1024 It sends everything anyway. If I use log:system:1024, the bbwin client throws an error that it can't find the system log file. It does find the file without the maxdata parameter. One problem with the settings in client-local.cfg on the xymon server is that they only propagate to the client after a successful sending of a client message. The mechanics is that after the client message has been sent, BBWin does a shutdown on the write socket and then reads from the read socket until EOF. If the write socket is never shutdown because the server breaks the connection due to flooding, then the client-local.cfg section will never be sent to the client. I've previously attempted to manually create C:\Program Files(x86)\BBWin\tmp\clientlocal.cfg to trick it, but with limited success. Just played around with it again - with client reporting to my primary server I just got it going - previously I was reporting to 2 servers. The trick seems to be in having logfetch.status also created. I then modified my BBWin.cfg back to 2 servers and it broke again. Maybe the 2nd server is responding differently. It is undefined which server's clientlocal.cfg to believe anyway :) Swapped the order of the servers around and it seems stable. If the server just discarded all the flooding data beyond MAXMSG_* and then returned the client config section anyway maybe it could be made to work, but there may be good reasons why that would not be sensible. This aside, I do need to get this working as I have to start monitoring 10 new windows servers and have to monitor the event logs, so I can't just stop sending them as has been suggested. Yes I could do some powershell scripts and send them via bbwincmd but the bbwin client is made for this task..... So, looking at this from the other side, the xymon server appears to be resetting the session after about 22MB of data has been sent (I know, this is ludicrous, but it is windows). Nothing in the xymond logs (except for the occasional data flooding error ("1st line client", always)); on the client side it reports it can't send the data to the xymon server. Depending on what auditting you have enabled, 22MB is very easy to achieve! Turn on Security event logging including success and failure and it's almost guaranteed :) David. I've set the MAXMSG_* to quite silly levels: # ipcs ------ Shared Memory Segments -------- key shmid owner perms bytes nattch status 0x01034be7 16908288 xymon 600 102400000 2 0x02034be7 16941057 xymon 600 102400000 2 0x03034be7 16973826 xymon 600 102400000 2 0x04034be7 17006595 xymon 600 102400000 2 0x05034be7 17039364 xymon 600 262144 1 0x06034be7 17072133 xymon 600 32768 1 0x07034be7 17104902 xymon 600 102400000 2 0x08034be7 17137671 xymon 600 102400000 2 0x09034be7 17170440 xymon 600 131072 1 Anything up to and including this size has no effect on the problem. Looking at the tcpdump stream, the bbwin client sends data normally with regular ACKs from xymond till around that 22MB mark then xymond responds with a FIN packet, then with RST packets and the session shuts down. Nothing in the packets themselves indicate what the problem is. If anyone can help with this, please, it would be great. thanks, Phil
-- David Baldwin - Senior Systems Administrator (Datacentres + Networks) Information and Communication Technology Services Australian Sports Commission http://ausport.gov.au Tel 02 62147830 Fax 02 62141830 PO Box 176 Belconnen ACT 2616 user-cbbf693f2c89@xymon.invalid Leverrier Street Bruce ACT 2617 Keep up to date with what's happening in Australian sport visit http://www.ausport.gov.au This message is intended for the addressee named and may contain confidential and privileged information. If you are not the intended recipient please note that any form of distribution, copying or use of this communication or the information in it is strictly prohibited and may be unlawful. If you receive this message in error, please delete it and notify the sender. Please consider the environment before printing this e-mail This message from ORIX Australia may contain confidential and/or privileged information. If you are not the intended recipient, any use, disclosure or copying of this message (or of any attachments to it) is not authorised. If you have received this message in error, please notify the sender immediately and delete the message and any attachments from your system. Please inform the sender if you do not wish to receive further communications by email. ORIX handles personal information according to a Privacy Policy that is consistent with the National Privacy Principles. Please let us know if you would like a copy. It is also available at www.orix.com.au<http://www.orix.com.au>; Please consider the environment before printing this e-mail This message from ORIX Australia may contain confidential and/or privileged information. If you are not the intended recipient, any use, disclosure or copying of this message (or of any attachments to it) is not authorised. If you have received this message in error, please notify the sender immediately and delete the message and any attachments from your system. Please inform the sender if you do not wish to receive further communications by email. ORIX handles personal information according to a Privacy Policy that is consistent with the National Privacy Principles. Please let us know if you would like a copy. It is also available at www.orix.com.au<http://www.orix.com.au>;