Feature Request ~ logfetch
list Michael Beatty
I see the comments made in logfetch.c /* Is it ok for these to be hardcoded ? */ #define MAXCHECK 102400 /* When starting, dont look at more than 100 KB of data */ #define MAXMINUTES 30 #define POSCOUNT ((MAXMINUTES / 5) + 1) #define LINES_AROUND_TRIGGER 5 My answer to this would be, "no". Due to various reasons, it is not desirable to run my client more than once every hour. Therefore, the msgs check for log files is almost useless unless I change these values (and honestly, I'm not even sure that would work as I haven't seen if this would impact anything else). If I ran my client every hour, I would lose 50% of my log data. I would like to see where the log check would be dependent on how long of an interval your client is run as opposed to a hard coded 30 minutes. -- Michael Beatty
list Ryan Novosielski
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
▸
On 02/25/2013 03:55 PM, Michael Beatty wrote:I see the comments made in logfetch.c /* Is it ok for these to be hardcoded ? */ #define MAXCHECK 102400 /* When starting, dont look at more than 100 KB of data */ #define MAXMINUTES 30 #define POSCOUNT ((MAXMINUTES / 5) + 1) #define LINES_AROUND_TRIGGER 5 My answer to this would be, "no". Due to various reasons, it is not desirable to run my client more than once every hour. Therefore, the msgs check for log files is almost useless unless I change these values (and honestly, I'm not even sure that would work as I haven't seen if this would impact anything else). If I ran my client every hour, I would lose 50% of my log data. I would like to see where the log check would be dependent on how long of an interval your client is run as opposed to a hard coded 30 minutes.
I personally have no need for this change at the moment, but I agree that no it's not OK for them to be hardcoded. Seems to be an easy enough change to make and there's virtually no benefit (that I can think of) to hard coding. - -- - ---- _ _ _ _ ___ _ _ _ |Y#| | | |\/| | \ |\ | | |Ryan Novosielski - Sr. Systems Programmer |$&| |__| | | |__/ | \| _| |user-ae4522577e16@xymon.invalid - 973/972.0922 (2-0922) \__/ Univ. of Med. and Dent.|IST/EI-Academic Svcs. - ADMC 450, Newark -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with undefined - http://www.enigmail.net/ iEYEARECAAYFAlEr22YACgkQmb+gadEcsb5S4gCgrYiBbt8dpOpTIHUMU+ZAnTpk fKAAoM4KnoMvConjfpfopXSuWcvkf8zC =p+0J -----END PGP SIGNATURE-----
list Japheth Cleaver
▸
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 02/25/2013 03:55 PM, Michael Beatty wrote:I see the comments made in logfetch.c /* Is it ok for these to be hardcoded ? */ #define MAXCHECK 102400 /* When starting, dont look at more than 100 KB of data */ #define MAXMINUTES 30 #define POSCOUNT ((MAXMINUTES / 5) + 1) #define LINES_AROUND_TRIGGER 5 My answer to this would be, "no". Due to various reasons, it is not desirable to run my client more than once every hour. Therefore, the msgs check for log files is almost useless unless I change these values (and honestly, I'm not even sure that would work as I haven't seen if this would impact anything else). If I ran my client every hour, I would lose 50% of my log data. I would like to see where the log check would be dependent on how long of an interval your client is run as opposed to a hard coded 30 minutes.I personally have no need for this change at the moment, but I agree that no it's not OK for them to be hardcoded. Seems to be an easy enough change to make and there's virtually no benefit (that I can think of) to hard coding.
That does seem a little suspect. There definitely needs to be *some* sort
of safety max_buffer size, but perhaps in this day of disk space out the
yin-yang it warrants an increase.
The POSCOUNT/MAXMINUTES values are compile time defines, but there might
be a better way to derive from the last run how far back to go... And the
"current" run interval (except when using CRONDATE) is put into the
environment at runtime by to xymonlaunch.
-jc
list Henrik Størner
On Mon, 25 Feb 2013 15:55:06 -0500, Michael Beatty
▸
<user-4aea7c115850@xymon.invalid> wrote:I see the comments made in logfetch.c /* Is it ok for these to be hardcoded ? */ #define MAXCHECK 102400 /* When starting, dont look at more than 100
KB of data */ #define MAXMINUTES 30 #define POSCOUNT ((MAXMINUTES / 5) + 1) #define LINES_AROUND_TRIGGER 5 My answer to this would be, "no". Due to various reasons, it is not desirable to run my client more than once every hour. Therefore, the msgs check for log files is almost useless unless I change these values (and honestly, I'm not even sure that would work as I haven't seen if this would impact anything else). If I ran my client every hour, I would lose 50% of my log data. I would like to see where the log check would be dependent on how long of an interval your client is run as opposed to a hard coded 30 minutes.
OK, just to clarify what these are for: - MAXCHECK is only used when the client has not run before, to avoid searching gigabytes of log data. It does not affect how log files are processed during normal client-cycles - that is controlled by the logsize setting in client-local.cfg. - MAXMINUTES is only used to calculate POSCOUNT. - POSCOUNT determines how many of the previous client cycles the client will include in the client-data as "current logfile data". What this means is that the "30" is really a red herring - what matters is that the client will send data from the logfile generated during the last 7 times the client ran. So if the client runs only once an hour, then you'll have logdata from the past 6 hours included in the client data. Regards, Henrik
list Yuki Matsushita
▸
(2013/03/06 22:55), user-ce4a2c883f75@xymon.invalid wrote:
- MAXCHECK is only used when the client has not run before, to avoid searching gigabytes of log data. It does not affect how log files are processed during normal client-cycles - that is controlled by the logsize setting in client-local.cfg.
Would that be right?
I read logfetch.c and it seems comparing bufsz to MAXCHECK is performed
always.
Is it valid if the logsize setting in client-local.cfg was over MAXCHECK?
In addition, I found a typo in line 251.
247: /* See if this is a trigger line */
248: if (logdef->triggercount) {
249: int i, match = 0;
250:
251: for (i=0; ((i < logdef->ignorecount) && !match); i++) {
"ignorecount" should be "triggercount".
Regards,
Yuki
--
Yuki Matsushita
list Jeremy Laidman
On 1 July 2013 15:57, Yuki Matsushita <user-f21c28c3bb60@xymon.invalid> wrote:
In addition, I found a typo in line 251.
This! Domo arigato Yuki-san! Yes, this has been bugging me for almost 2 years (when I reported the resultant segfault it to the List). I haven't been able to use any "trigger" clauses in all that time. It's still broken in the current trunk (rev 7257). I'm guessing, from reading the code, that I could work around this by always have exactly the same number of "trigger" lines as "ignore" lines - perhaps padding out with bogus "trigger" or "ignore" lines. But obviously would be better to have it fixed. Henrik, do you need a patch submitted for this one-line fix? J
list Yuki Matsushita
▸
(2013/08/09 16:44), Jeremy Laidman wrote:
On 1 July 2013 15:57, Yuki Matsushita <user-f21c28c3bb60@xymon.invalid> wrote:In addition, I found a typo in line 251.This! Domo arigato Yuki-san! Yes, this has been bugging me for almost 2 years (when I reported the resultant segfault it to the List). I haven't been able to use any "trigger" clauses in all that time. It's still broken in the current trunk (rev 7257).
Thanks for your response, Jeremy. Dou itashimashite.
▸
I'm guessing, from reading the code, that I could work around this by always have exactly the same number of "trigger" lines as "ignore" lines - perhaps padding out with bogus "trigger" or "ignore" lines.
Thanks for analysing this issue. I think so too. Regards, Yuki
list Jeremy Laidman
▸
On 9 August 2013 21:05, Yuki Matsushita <user-f21c28c3bb60@xymon.invalid> wrote:
I'm guessing, from reading the code, that I could work around this by always have exactly the same number of "trigger" lines as "ignore" lines-perhaps padding out with bogus "trigger" or "ignore" lines.Thanks for analysing this issue. I think so too.
Yes, and no. It now goes past the code with the typo. However, it now
dumps core when it actually uses the trigger and the number of bytes in
matching lines is large. gdb tells me that it fails in line 315, in the
logdata() function:
memmove(triggerendpos, skipend, bytesleft);
If I push out the filesize limit to the max (1024k) then this line doesn't
get executed and no more segfault. But I think I'm just being lucky that
the volume of matching lines is less than 100k, and if it reached 100k, I'd
be getting the segfault again.
So, still a problem, just a different one. And this code is waaaay too
complicated for me to grok.
J