Looking for "page too old" script
list Josh Luthman
I have a tsch script that does some things, manipulates and image and such and posts it on a web site. Occasionally, I have found a bug or two where the image stops updating. I don't know this until it is too late. Is there any sort of technique or custom script that will poll a website and see if something is to old? I'm at a bit of a loss as to how to correctly indicate a problem and hoping there is something I can start from already. Thanks in advance! Josh Luthman Office: XXX-XXX-XXXX Direct: XXX-XXX-XXXX XXXX Wayne St Suite XXXX Troy, OH XXXXX
list Ralph Mitchell
I wrote something like that when I was with EDS. It grabs an image and compares the md5sum to the md5sum of a stored copy of the image. If it's different, sound the alarm. I can probably dredge that up this evening when I get home. In the meantime, it would not be too hard to reconstruct. If it's just the image that isn't being updated, then grab a copy with curl: curl -s -S -L -o image.jpg http://www.server.com/img.jpg and md5sum it. Compare that to the md5sum of the last image you fetched. If different, you can then send a red report, otherwise send green. If the page itself is not being update, just have the tcsh script insert a timestamp as a comment, then grab the page in much the same way and grep the timestamp from it. If the timestamp varies from real time by several minutes, sound the alarm. Ralph Mitchell On Fri, Jul 29, 2011 at 10:40 AM, Josh Luthman
▸
<user-4c45a83f15cb@xymon.invalid>wrote:
I have a tsch script that does some things, manipulates and image and such and posts it on a web site. Occasionally, I have found a bug or two where the image stops updating. I don't know this until it is too late. Is there any sort of technique or custom script that will poll a website and see if something is to old? I'm at a bit of a loss as to how to correctly indicate a problem and hoping there is something I can start from already. Thanks in advance! Josh Luthman Office: XXX-XXX-XXXX Direct: XXX-XXX-XXXX XXXX Wayne St Suite XXXX Troy, OH XXXXX
list Paul Root
Couldn't you monitor the image file modification date on the server? Or have the script send a message to a log file on success and monitor in logs? Paul Root - Engineer III - Qwest is becoming CenturyLink
▸
From: xymon-bounces at xymon.com [mailto:xymon-bounces at xymon.com] On Behalf Of Josh Luthman
Sent: Friday, July 29, 2011 9:41 AM
To: xymon at xymon.com
Subject: [Xymon] Looking for "page too old" script
I have a tsch script that does some things, manipulates and image and such and posts it on a web site. Occasionally, I have found a bug or two where the image stops updating. I don't know this until it is too late. Is there any sort of technique or custom script that will poll a website and see if something is to old? I'm at a bit of a loss as to how to correctly indicate a problem and hoping there is something I can start from already.
Thanks in advance!
Josh Luthman
Office: XXX-XXX-XXXX
Direct: XXX-XXX-XXXX
XXXX Wayne St
Suite XXXX
Troy, OH XXXXX
This communication is the property of Qwest and may contain confidential or
privileged information. Unauthorized use of this communication is strictly
prohibited and may be unlawful. If you have received this communication
in error, please immediately notify the sender by reply e-mail and destroy
all copies of the communication and any attachments.
list Henrik Størner
▸
On 29-07-2011 17:00, Ralph Mitchell wrote:
I wrote something like that when I was with EDS. It grabs an image and compares the md5sum to the md5sum of a stored copy of the image. If it's different, sound the alarm. I can probably dredge that up this evening when I get home. In the meantime, it would not be too hard to reconstruct.
Xymon has this built-in. 10.0.0.1 www # cont;http://www/img.jpg;#md5:83299f5516985377e3911fd95d0b0038 Regards, Henrik
list Tim McCloskey
couple random thoughts: Would HTTP cache control headers help in this regard? You can poll a page via wget or some other mechanism and look at the page headers, that won't tell you if the image is old but if the page is regenerated on each image update it may work... If you have shell access to the server:/image.file you could test the mac times... Or, toss in some logic to catch the bugz :)
▸
From: xymon-bounces at xymon.com [xymon-bounces at xymon.com] On Behalf Of Josh Luthman [user-4c45a83f15cb@xymon.invalid]
Sent: Friday, July 29, 2011 7:40 AM
To: xymon at xymon.com
Subject: [Xymon] Looking for "page too old" script
I have a tsch script that does some things, manipulates and image and such and posts it on a web site. Occasionally, I have found a bug or two where the image stops updating. I don't know this until it is too late. Is there any sort of technique or custom script that will poll a website and see if something is to old? I'm at a bit of a loss as to how to correctly indicate a problem and hoping there is something I can start from already.
Thanks in advance!
Josh Luthman
Office: XXX-XXX-XXXX
Direct: XXX-XXX-XXXX
XXXX Wayne St
Suite XXXX
Troy, OH XXXXX
list Josh Luthman
The page is set to expire immediately and usually the cache problems are the browser/client. At least in my experience. On hobbit-clients.cfg for the host (it has proc tests already) I added: FILE /var/www/foo/bar/output.png red mtime<1800 #go red if file older than 30 minutes How do I get the client to send the details about this file? From reading the man page I need to specify client-local.cfg but I am not sure what goes in this. I'm pretty sure I want: file:/var/www/foo/bar/output.png Do I put this under [linux]? If I do this, will it only have the clients with an appropriate FILE check or ask this file of every linux box?
▸
Josh Luthman
Office: XXX-XXX-XXXX
Direct: XXX-XXX-XXXX
XXXX Wayne St
Suite XXXX
Troy, OH XXXXX
▸
On Fri, Jul 29, 2011 at 11:58 AM, Tim McCloskey <user-440820cc07d6@xymon.invalid> wrote:
couple random thoughts: Would HTTP cache control headers help in this regard? You can poll a page via wget or some other mechanism and look at the page headers, that won't tell you if the image is old but if the page is regenerated on each image update it may work... If you have shell access to the server:/image.file you could test the mac times... Or, toss in some logic to catch the bugz :) From: xymon-bounces at xymon.com [xymon-bounces at xymon.com] On Behalf Of Josh Luthman [user-4c45a83f15cb@xymon.invalid] Sent: Friday, July 29, 2011 7:40 AM To: xymon at xymon.com Subject: [Xymon] Looking for "page too old" script I have a tsch script that does some things, manipulates and image and such and posts it on a web site. Occasionally, I have found a bug or two where the image stops updating. I don't know this until it is too late. Is there any sort of technique or custom script that will poll a website and see if something is to old? I'm at a bit of a loss as to how to correctly indicate a problem and hoping there is something I can start from already. Thanks in advance! Josh Luthman Office: XXX-XXX-XXXX Direct: XXX-XXX-XXXX XXXX Wayne St Suite XXXX Troy, OH XXXXX
list Tim McCloskey
If your system is setup to store the client data on the central server you can have something like the following in hobbit-clients.cfg. HOST=somehost GROUP=optional-some-group FILE /some/path/tofile.ext SIZE<55M and SIZE>0 red Client-local can take a host name too -- I think....
▸
From: Josh Luthman [user-4c45a83f15cb@xymon.invalid]
Sent: Friday, July 29, 2011 11:24 AM
To: Tim McCloskey
Cc: xymon at xymon.com
Subject: Re: [Xymon] Looking for "page too old" script
The page is set to expire immediately and usually the cache problems are the browser/client. At least in my experience.
On hobbit-clients.cfg for the host (it has proc tests already) I added:
FILE /var/www/foo/bar/output.png red mtime<1800 #go red if file older than 30 minutes
How do I get the client to send the details about this file? From reading the man page I need to specify client-local.cfg but I am not sure what goes in this. I'm pretty sure I want:
file:/var/www/foo/bar/output.png
Do I put this under [linux]? If I do this, will it only have the clients with an appropriate FILE check or ask this file of every linux box?
Josh Luthman
Office: XXX-XXX-XXXX
Direct: XXX-XXX-XXXX
XXXX Wayne St
Suite XXXX
Troy, OH XXXXX
On Fri, Jul 29, 2011 at 11:58 AM, Tim McCloskey <user-440820cc07d6@xymon.invalid<mailto:user-440820cc07d6@xymon.invalid>> wrote:
couple random thoughts:
Would HTTP cache control headers help in this regard? You can poll a page via wget or some other mechanism and look at the page headers, that won't tell you if the image is old but if the page is regenerated on each image update it may work... If you have shell access to the server:/image.file you could test the mac times... Or, toss in some logic to catch the bugz :)
From: xymon-bounces at xymon.com<mailto:xymon-bounces at xymon.com> [xymon-bounces at xymon.com<mailto:xymon-bounces at xymon.com>] On Behalf Of Josh Luthman [user-4c45a83f15cb@xymon.invalid<mailto:user-4c45a83f15cb@xymon.invalid>]
Sent: Friday, July 29, 2011 7:40 AM
To: xymon at xymon.com<mailto:xymon at xymon.com>
Subject: [Xymon] Looking for "page too old" script
I have a tsch script that does some things, manipulates and image and such and posts it on a web site. Occasionally, I have found a bug or two where the image stops updating. I don't know this until it is too late. Is there any sort of technique or custom script that will poll a website and see if something is to old? I'm at a bit of a loss as to how to correctly indicate a problem and hoping there is something I can start from already.
Thanks in advance!
Josh Luthman
Office: XXX-XXX-XXXX<tel:XXX-XXX-XXXX>
Direct: XXX-XXX-XXXX<tel:XXX-XXX-XXXX>
XXXX Wayne St
Suite XXXX
Troy, OH XXXXX
list Josh Luthman
Tim, I have that right now. But I don't see how the server is going to know what the file meta data is unless it's sent to the server somehow. At this time, probably about half an hour after I added it, the test is still white. I'm almost 100% sure hobbit-clients.cfg changes doesn't require a restart.
▸
Josh Luthman
Office: XXX-XXX-XXXX
Direct: XXX-XXX-XXXX
XXXX Wayne St
Suite XXXX
Troy, OH XXXXX
▸
On Fri, Jul 29, 2011 at 2:34 PM, Tim McCloskey <user-440820cc07d6@xymon.invalid> wrote:
If your system is setup to store the client data on the central server you can have something like the following in hobbit-clients.cfg. HOST=somehost GROUP=optional-some-group FILE /some/path/tofile.ext SIZE<55M and SIZE>0 red Client-local can take a host name too -- I think.... From: Josh Luthman [user-4c45a83f15cb@xymon.invalid] Sent: Friday, July 29, 2011 11:24 AM To: Tim McCloskey Cc: xymon at xymon.com Subject: Re: [Xymon] Looking for "page too old" script The page is set to expire immediately and usually the cache problems are the browser/client. At least in my experience. On hobbit-clients.cfg for the host (it has proc tests already) I added: FILE /var/www/foo/bar/output.png red mtime<1800 #go red if file older than 30 minutes How do I get the client to send the details about this file? From reading the man page I need to specify client-local.cfg but I am not sure what goes in this. I'm pretty sure I want: file:/var/www/foo/bar/output.png Do I put this under [linux]? If I do this, will it only have the clients with an appropriate FILE check or ask this file of every linux box? Josh Luthman Office: XXX-XXX-XXXX Direct: XXX-XXX-XXXX XXXX Wayne St Suite XXXX Troy, OH XXXXX
On Fri, Jul 29, 2011 at 11:58 AM, Tim McCloskey <user-440820cc07d6@xymon.invalid<mailto:
▸
user-440820cc07d6@xymon.invalid>> wrote:
couple random thoughts:
Would HTTP cache control headers help in this regard? You can poll a page
via wget or some other mechanism and look at the page headers, that won't
tell you if the image is old but if the page is regenerated on each image
update it may work... If you have shell access to the server:/image.file you
could test the mac times... Or, toss in some logic to catch the bugz :)
From: xymon-bounces at xymon.com<mailto:xymon-bounces at xymon.com> [
xymon-bounces at xymon.com<mailto:xymon-bounces at xymon.com>] On Behalf Of Josh
Luthman [user-4c45a83f15cb@xymon.invalid<mailto:user-4c45a83f15cb@xymon.invalid>]
Sent: Friday, July 29, 2011 7:40 AM
To: xymon at xymon.com<mailto:xymon at xymon.com>
Subject: [Xymon] Looking for "page too old" script
I have a tsch script that does some things, manipulates and image and such
and posts it on a web site. Occasionally, I have found a bug or two where
the image stops updating. I don't know this until it is too late. Is there
any sort of technique or custom script that will poll a website and see if
something is to old? I'm at a bit of a loss as to how to correctly indicate
a problem and hoping there is something I can start from already.
Thanks in advance!
Josh Luthman
Office: XXX-XXX-XXXX<tel:XXX-XXX-XXXX>
Direct: XXX-XXX-XXXX<tel:XXX-XXX-XXXX>
XXXX Wayne St
Suite XXXX
Troy, OH XXXXX
list Tim McCloskey
Do I put this under [linux]? If I do this, will it only have the clients with an appropriate FILE check or ask this file of every linux box?
Just took another look at a working config....here's a couple notes. You don't need the whole [linux] class, you can use a [hostname] in client-local.cfg ... [your-hostname] file:/var/www/foo/bar/output.png ... The stanza above with a matching FILE directive from hobbit-clients.cfg (both server side) works for me, on 4.2.0. If you have this setup - sounds like you do - and it's not working you can Try touching the logfetch.$foo.cfg and logfetch.$foo.status files in the tmp dir on the client (create them however you need, just make sure to chown them to your hobbit user). You should not need to populate the files, just create them. Example file... cat someclienthostname:~hobbit/client/tmp/logfetch.someclienthostname.cfg log:/some/path/to/somefile:10240 file:/bin/su Tim
list Josh Luthman
Got it working! Thanks!!! FILE /var/www/inxwireless/network/output.png red mtime<1800 This is correct. From what the documentation states, of which I may be understanding it incorrectly, this is backwards. Initially I did mtime>1800 based on what I read on the man page. The test went red "File was modified 31 seconds ago - should be >1800"
▸
Josh Luthman
Office: XXX-XXX-XXXX
Direct: XXX-XXX-XXXX
XXXX Wayne St
Suite XXXX
Troy, OH XXXXX
▸
On Fri, Jul 29, 2011 at 4:31 PM, Tim McCloskey <user-440820cc07d6@xymon.invalid> wrote:
Do I put this under [linux]? If I do this, will it only have the clients with an appropriate FILE check or ask this file of every linux box?Just took another look at a working config....here's a couple notes. You don't need the whole [linux] class, you can use a [hostname] in client-local.cfg ... [your-hostname] file:/var/www/foo/bar/output.png ... The stanza above with a matching FILE directive from hobbit-clients.cfg (both server side) works for me, on 4.2.0. If you have this setup - sounds like you do - and it's not working you can Try touching the logfetch.$foo.cfg and logfetch.$foo.status files in the tmp dir on the client (create them however you need, just make sure to chown them to your hobbit user). You should not need to populate the files, just create them. Example file... cat someclienthostname:~hobbit/client/tmp/logfetch.someclienthostname.cfg log:/some/path/to/somefile:10240 file:/bin/su Tim
list Henrik Størner
▸
On 01-08-2011 17:46, Josh Luthman wrote:
Got it working! Thanks!!! FILE /var/www/inxwireless/network/output.png red mtime<1800 This is correct. From what the documentation states, of which I may be understanding it incorrectly, this is backwards.
I must admit I had my doubts when coding this. But the idea is to read it as it is written: "File foo must have been modified LESS THAN 1800 seconds ago". Regards, Henrik
list Josh Luthman
The code makes more sense to me then the sentence explaining it in the man page. That's just me. Either way you only have two choices so it won't be a big mistake.
▸
On Aug 7, 2011 8:20 AM, "Henrik Størner" <user-ce4a2c883f75@xymon.invalid> wrote:On 01-08-2011 17:46, Josh Luthman wrote:Got it working! Thanks!!! FILE /var/www/inxwireless/network/output.png red mtime<1800 This is correct. From what the documentation states, of which I may be understanding it incorrectly, this is backwards.I must admit I had my doubts when coding this. But the idea is to read it as it is written: "File foo must have been modified LESS THAN 1800 seconds ago". Regards, Henrik
list Galen Johnson
▸
On 08/07/2011 01:53 PM, Josh Luthman wrote:
The code makes more sense to me then the sentence explaining it in the man page. That's just me. Either way you only have two choices so it won't be a big mistake. On Aug 7, 2011 8:20 AM, "Henrik Størner" <user-ce4a2c883f75@xymon.invalid <mailto:user-ce4a2c883f75@xymon.invalid>> wrote:On 01-08-2011 17:46, Josh Luthman wrote:Got it working! Thanks!!! FILE /var/www/inxwireless/network/output.png red mtime<1800 This is correct. From what the documentation states, of which I may be understanding it incorrectly, this is backwards.I must admit I had my doubts when coding this. But the idea is to read it as it is written: "File foo must have been modified LESS THAN 1800 seconds ago". Regards, Henrik
This would make sense if the file is being updated every 30 minutes. However, you may want to give it a 5 minute buffer to avoid timing false alarms, like "mtime<2100". NB: I would have read this as "Turn red if the file is less than 30 minutes old" which would seem backwards but I'm reluctant to contradict the author of the program :-). =G=