Testing external web through a proxy
list Vernon Everett
Hi all I am trying to test our interent connectivity by checking Google through our proxy. (Once I get this right, I want to start testing some of our external web pages in the same manner) I have configured this in bb-hosts 0.0.0.0 www.google.com # noconn http://hobbituser:user-1b7da96bb10f@xymon.invalid:8080/http://www.google.com And I get this. yellow<http://hobbit/hobbit/gifs/yellow.gif>; http://www.google.com.au/ - Proxy Authentication Required HTTP/1.1 407 Proxy Authentication Required ( The ISA Server requires authorization to fulfill the request. Access to the Web Proxy filter is denied. ) Via: 1.1 PXYHOSTNAME Proxy-Authenticate: Negotiate Proxy-Authenticate: Kerberos Proxy-Authenticate: NTLM Connection: Keep-Alive Proxy-Connection: Keep-Alive Pragma: no-cache Cache-Control: no-cache Content-Type: text/html Content-Length: 4118 Seconds: 0.02 Any assistance appreciated. Regards Vernon NOTICE: This email and any attachments are confidential. They may contain legally privileged information or copyright material. You must not read, copy, use or disclose them without authorisation. If you are not an intended recipient, please contact us at once by return email and then delete both messages and all attachments.
list Ralph Mitchell
Looks like your proxy requires authentication via Kerberos or NTLM or
GSS-Negotiate. Dunno if Xymon can do those. Luckily the curl tool
(httpp://curl.haxx.se) *can* do those... I haven't had to use them, but it
should not be too hard to bang out an ext script to do the check. Something
along these lines:
#!/bin/sh
curl -x http://ourproxy.ourdomain.com:8080 \
--proxy-anyauth -U hobbituser:hobbitpassword \
-s -S -L -o /dev/null http://www.google.com/
if [ "$?" -eq "0" ]; then
COLOR=green
MESSAGE="Pinged Google OK"
else
COLOR=RED
MESSAGE="Failed to reach Google"
fi
$BB $BBDISP "status www,google,com.home $COLOR `date`
$MESSAGE"
Obviously that can be expanded quite a lot... :) Drop it somewhere on your
Xymon server (maybe in server/ext) and set up a block in
server/etc/hobbitlaunch.cfg to fire it off as often as you want. You may
need to give the full path for curl, which is probably /usr/bin/curl.
Ralph Mitchell
On Fri, Nov 21, 2008 at 1:30 AM, Everett, Vernon <
▸
user-9da1a1882f49@xymon.invalid> wrote:
Hi all I am trying to test our interent connectivity by checking Google through our proxy. (Once I get this right, I want to start testing some of our external web pages in the same manner) I have configured this in bb-hosts
0.0.0.0 *www.google.com* <http://www.google.com>; # noconn • http://hobbituser:user-1b7da96bb10f@xymon.invalid:8080/http://www.google.com *<http://hobbituser:user-1b7da96bb10f@xymon.invalid:8080/http://www.google.com>; And I get this. *yellow* <http://hobbit/hobbit/gifs/yellow.gif>; * http://www.google.com.au/* <http://www.google.com.au/>; - Proxy
▸
Authentication Required
HTTP/1.1 407 Proxy Authentication Required ( The ISA Server requires
authorization to fulfill the request. Access to the Web Proxy filter is
denied. )
Via: 1.1 PXYHOSTNAME
Proxy-Authenticate: Negotiate
Proxy-Authenticate: Kerberos
Proxy-Authenticate: NTLM
Connection: Keep-Alive
Proxy-Connection: Keep-Alive
Pragma: no-cache
Cache-Control: no-cache
Content-Type: text/html
Content-Length: 4118
Seconds: 0.02
Any assistance appreciated.
Regards
Vernon
NOTICE: This email and any attachments are confidential.
They may contain legally privileged information or
copyright material. You must not read, copy, use or
disclose them without authorisation. If you are not an
intended recipient, please contact us at once by return
email and then delete both messages and all attachments.
list Jason Hand
I am just looking for what you think is the best way to exclude a few clients from reporting problems during regular 3-time a week maintenance periods where these are naturally going to be off or not reporting. How would I set that in the best way? Thanks, Jason
list Ulric Eriksson
▸
On Fri, 21 Nov 2008, Jason Hand wrote:
I am just looking for what you think is the best way to exclude a few clients from reporting problems during regular 3-time a week maintenance periods where these are naturally going to be off or not reporting. How would I set that in the best way?
Set up a cron job to disable them at those times. 0 1 * * mon,wed,fri /home/hobbit/server/bin/bb yourhobbitserver "disable yourhobbitclient 60 Scheduled reboot" Ulric
list Jason Hand
These are a variety of monitored entities including windows machines and web sites. How would that work as you describe? Do I just substitute the days, "yourhobbitserver" with the name or localhost, "yourhobbitclient" with the hostname of the client to disable, "60" with number of minutes to disable it, and then the "Scheduled reboot" would be the cause? And this cron job would run on the Hobbit server right? Thanks , Jason -----Original Message----- From: Ulric Eriksson [mailto:user-de31148ebe0c@xymon.invalid] Sent: Friday, November 21, 2008 6:58 PM To: user-ae9b8668bcde@xymon.invalid Subject: Re: [hobbit] Recommendation on setting regular maintenance exclusion for reporting
▸
On Fri, 21 Nov 2008, Jason Hand wrote:
I am just looking for what you think is the best way to exclude a few clients from reporting problems during regular 3-time a week maintenance periods where these are naturally going to be off or not reporting. How would I set that in the best way?
Set up a cron job to disable them at those times. 0 1 * * mon,wed,fri /home/hobbit/server/bin/bb yourhobbitserver "disable yourhobbitclient 60 Scheduled reboot" Ulric
9:37 AM
list Alan Sparks
Can't you just use the DOWNTIME specification in bb-hosts for those clients? -Alan
▸
Jason Hand wrote:These are a variety of monitored entities including windows machines and web sites. How would that work as you describe? Do I just substitute the days, "yourhobbitserver" with the name or localhost, "yourhobbitclient" with the hostname of the client to disable, "60" with number of minutes to disable it, and then the "Scheduled reboot" would be the cause? And this cron job would run on the Hobbit server right? Thanks , Jason -----Original Message----- From: Ulric Eriksson [mailto:user-de31148ebe0c@xymon.invalid] Sent: Friday, November 21, 2008 6:58 PM To: user-ae9b8668bcde@xymon.invalid Subject: Re: [hobbit] Recommendation on setting regular maintenance exclusion for reporting On Fri, 21 Nov 2008, Jason Hand wrote:I am just looking for what you think is the best way to exclude a few clients from reporting problems during regular 3-time a week maintenance periods where these are naturally going to be off or notreporting.How would I set that in the best way?Set up a cron job to disable them at those times. 0 1 * * mon,wed,fri /home/hobbit/server/bin/bb yourhobbitserver "disable yourhobbitclient 60 Scheduled reboot" Ulric 9:37 AM
list Ulric Eriksson
▸
On Fri, 21 Nov 2008, Jason Hand wrote:
These are a variety of monitored entities including windows machines and web sites. How would that work as you describe? Do I just substitute the days, "yourhobbitserver" with the name or localhost, "yourhobbitclient" with the hostname of the client to disable, "60" with number of minutes to disable it, and then the "Scheduled reboot" would be the cause?
Right. Actually, "yourhobbitclient" would be, for example: web1,example,com.cpu to disable just the cpu test on the client web1.example.com, or web1,example,com.* to disable all tests on the same client.
And this cron job would run on the Hobbit server right?
Not necessarily, but that would be a good place to run it. Ulric
list Jason Hand
That worked like a charm. I had not used that spec before but that was exactly what I was looking for. Thanks for the suggestion. -Jason
▸
-----Original Message-----
From: Alan Sparks [mailto:user-8f2174fd8b66@xymon.invalid] Sent: Friday, November 21, 2008 7:28 PM
To: user-ae9b8668bcde@xymon.invalid
Subject: Re: [hobbit] Recommendation on setting regular maintenance
exclusion for reporting
Can't you just use the DOWNTIME specification in bb-hosts for those clients?
-Alan
Jason Hand wrote:These are a variety of monitored entities including windows machines and web sites. How would that work as you describe? Do I just substitute the days, "yourhobbitserver" with the name or localhost, "yourhobbitclient" with the hostname of the client to disable, "60" with number of minutes to disable it, and then the "Scheduled reboot" would be the cause? And this cron job would run on the Hobbit server right? Thanks , Jason -----Original Message----- From: Ulric Eriksson [mailto:user-de31148ebe0c@xymon.invalid] Sent: Friday, November 21, 2008 6:58 PM To: user-ae9b8668bcde@xymon.invalid Subject: Re: [hobbit] Recommendation on setting regular maintenance exclusion for reporting On Fri, 21 Nov 2008, Jason Hand wrote:I am just looking for what you think is the best way to exclude a few clients from reporting problems during regular 3-time a week maintenance periods where these are naturally going to be off or notreporting.How would I set that in the best way?Set up a cron job to disable them at those times. 0 1 * * mon,wed,fri /home/hobbit/server/bin/bb yourhobbitserver "disable yourhobbitclient 60 Scheduled reboot" Ulric
11/21/2008
9:37 AM
9:37 AM
list Vernon Everett
Hi Ralph
This script works a treat.
Thanks
Vernon
From: Ralph Mitchell [mailto:user-00a5e44c48c0@xymon.invalid]
Sent: Saturday, 22 November 2008 12:23 AM
To: user-ae9b8668bcde@xymon.invalid
Subject: Re: [hobbit] Testing external web through a proxy
Looks like your proxy requires authentication via Kerberos or NTLM or GSS-Negotiate. Dunno if Xymon can do those. Luckily the curl tool (httpp://curl.haxx.se<http://curl.haxx.se>;) *can* do those... I haven't had to use them, but it should not be too hard to bang out an ext script to do the check. Something along these lines:
▸
#!/bin/sh curl -x http://ourproxy.ourdomain.com:8080 \ --proxy-anyauth -U hobbituser:hobbitpassword \ -s -S -L -o /dev/null http://www.google.com/ if [ "$?" -eq "0" ]; then COLOR=green MESSAGE="Pinged Google OK" else COLOR=RED MESSAGE="Failed to reach Google" fi $BB $BBDISP "status www,google,com.home $COLOR `date` $MESSAGE" Obviously that can be expanded quite a lot... :) Drop it somewhere on your Xymon server (maybe in server/ext) and set up a block in server/etc/hobbitlaunch.cfg to fire it off as often as you want. You may need to give the full path for curl, which is probably /usr/bin/curl. Ralph Mitchell On Fri, Nov 21, 2008 at 1:30 AM, Everett, Vernon <user-9da1a1882f49@xymon.invalid<mailto:user-9da1a1882f49@xymon.invalid>> wrote: Hi all I am trying to test our interent connectivity by checking Google through our proxy. (Once I get this right, I want to start testing some of our external web pages in the same manner) I have configured this in bb-hosts
0.0.0.0<http://0.0.0.0>; www.google.com<http://www.google.com>; # noconn http://hobbituser:user-1b7da96bb10f@xymon.invalid:8080/http://www.google.com
▸
And I get this. yellow<http://hobbit/hobbit/gifs/yellow.gif>; http://www.google.com.au/ - Proxy Authentication Required HTTP/1.1 407 Proxy Authentication Required ( The ISA Server requires authorization to fulfill the request. Access to the Web Proxy filter is denied. ) Via: 1.1 PXYHOSTNAME Proxy-Authenticate: Negotiate Proxy-Authenticate: Kerberos Proxy-Authenticate: NTLM Connection: Keep-Alive Proxy-Connection: Keep-Alive Pragma: no-cache Cache-Control: no-cache Content-Type: text/html Content-Length: 4118 Seconds: 0.02 Any assistance appreciated. Regards Vernon NOTICE: This email and any attachments are confidential. They may contain legally privileged information or copyright material. You must not read, copy, use or disclose them without authorisation. If you are not an intended recipient, please contact us at once by return email and then delete both messages and all attachments. NOTICE: This email and any attachments are confidential. They may contain legally privileged information or copyright material. You must not read, copy, use or disclose them without authorisation. If you are not an intended recipient, please contact us at once by return email and then delete both messages and all attachments.
list Ralph Mitchell
Heh - dude, that wasn't supposed to be the *whole* script... :) If that one was useful, here's something that may be a bit better. I added a timeout for curl, otherwise it'll wait for a long time if the server doesn't respond. Also, this format produces a message formatted just like Xymon's internal http test, including the time taken to grab the headers. You could extract the "Seconds" line to determine if the response is a bit slow. With a bit more magic in Xymon's configs, you could even graph it... Ralph Mitchell ============= cut here ============ #!/bin/sh TESTHOST=www.google.com TESTURL=http://$TESTHOST/ TIMEOUT=30 # Grab *just* the headers, simulating Xymon's builtin http check MESSAGE=`curl -x http://ourproxy.ourdomain.com:8080 \ --proxy-anyauth -U hobbituser:hobbitpassword \ -m $TIMEOUT \ -w 'Seconds: %{time_total}\n' \ -s -S -L -I $TESTURL | $GREP -v Set-Cookie` if [ "$?" -eq "0" ]; then COLOR=green else COLOR=red fi # convert dots to commas in the hostname MACHINE=`echo $TESTHOST | $SED -e 's/\./\,/g' $BB $BBDISP "status $MACHINE.home $COLOR `date` $MESSAGE" ============= cut here ============ On Sun, Nov 23, 2008 at 9:04 PM, Everett, Vernon <
▸
user-9da1a1882f49@xymon.invalid> wrote:
Hi Ralph This script works a treat. Thanks Vernon *From:* Ralph Mitchell [mailto:user-00a5e44c48c0@xymon.invalid] *Sent:* Saturday, 22 November 2008 12:23 AM *To:* user-ae9b8668bcde@xymon.invalid *Subject:* Re: [hobbit] Testing external web through a proxy Looks like your proxy requires authentication via Kerberos or NTLM or GSS-Negotiate. Dunno if Xymon can do those. Luckily the curl tool (httpp://curl.haxx.se) *can* do those... I haven't had to use them, but it should not be too hard to bang out an ext script to do the check. Something along these lines: #!/bin/sh curl -x http://ourproxy.ourdomain.com:8080 \ --proxy-anyauth -U hobbituser:hobbitpassword \ -s -S -L -o /dev/null http://www.google.com/ if [ "$?" -eq "0" ]; then COLOR=green MESSAGE="Pinged Google OK" else COLOR=RED MESSAGE="Failed to reach Google" fi $BB $BBDISP "status www,google,com.home $COLOR `date` $MESSAGE" Obviously that can be expanded quite a lot... :) Drop it somewhere on your Xymon server (maybe in server/ext) and set up a block in server/etc/hobbitlaunch.cfg to fire it off as often as you want. You may need to give the full path for curl, which is probably /usr/bin/curl. Ralph Mitchell On Fri, Nov 21, 2008 at 1:30 AM, Everett, Vernon < user-9da1a1882f49@xymon.invalid> wrote:Hi all I am trying to test our interent connectivity by checking Google through our proxy. (Once I get this right, I want to start testing some of our external web pages in the same manner) I have configured this in bb-hosts 0.0.0.0 *www.google.com* <http://www.google.com>; # noconn * http://hobbituser:user-1b7da96bb10f@xymon.invalid:8080/http://www.google.com *<http://hobbituser:user-1b7da96bb10f@xymon.invalid:8080/http://www.google.com>; And I get this. *yellow* <http://hobbit/hobbit/gifs/yellow.gif>; * http://www.google.com.au/* <http://www.google.com.au/>; - Proxy Authentication Required HTTP/1.1 407 Proxy Authentication Required ( The ISA Server requires authorization to fulfill the request. Access to the Web Proxy filter is denied. ) Via: 1.1 PXYHOSTNAME Proxy-Authenticate: Negotiate Proxy-Authenticate: Kerberos Proxy-Authenticate: NTLM Connection: Keep-Alive Proxy-Connection: Keep-Alive Pragma: no-cache Cache-Control: no-cache Content-Type: text/html Content-Length: 4118 Seconds: 0.02 Any assistance appreciated. Regards Vernon NOTICE: This email and any attachments are confidential. They may contain legally privileged information or copyright material. You must not read, copy, use or disclose them without authorisation. If you are not an intended recipient, please contact us at once by return email and then delete both messages and all attachments.NOTICE: This email and any attachments are confidential. They may contain legally privileged information or copyright material. You must not read, copy, use or disclose them without authorisation. If you are not an intended recipient, please contact us at once by return email and then delete both messages and all attachments.
list Vernon Everett
Of course it wasn't the whole script, but it still worked. :-) I then tweaked it a little to make it work better for our environment, and added a few features, like the time_total. I haven't implemented a timeout value though, although that's probably a good idea. What you gave me was a skeleton, and after I added a bit of flesh, I ended up with something quite workable. Which worked great. Thanks :-) Here's one that Henrik might need to address though. If I use curl to replace the standard http test, then I can't get it to graph unless I use the NCV feature. However, I am reluctant to try this, because it might break other HTTP tests that are working well (on internal sites) Is it OK to start graphing this using NCV, or is there a way to get the custom curl-based http test to inject the data points into the standard http graphing system? Cheers
▸
Vernon
From: Ralph Mitchell [mailto:user-00a5e44c48c0@xymon.invalid]
Sent: Tuesday, 25 November 2008 1:04 AM
To: user-ae9b8668bcde@xymon.invalid
Subject: Re: [hobbit] Testing external web through a proxy
Heh - dude, that wasn't supposed to be the *whole* script... :)
If that one was useful, here's something that may be a bit better. I added a timeout for curl, otherwise it'll wait for a long time if the server doesn't respond. Also, this format produces a message formatted just like Xymon's internal http test, including the time taken to grab the headers. You could extract the "Seconds" line to determine if the response is a bit slow. With a bit more magic in Xymon's configs, you could even graph it...
Ralph Mitchell
============= cut here ============
#!/bin/sh
TESTHOST=www.google.com<http://www.google.com>;
▸
TESTURL=http://$TESTHOST/ TIMEOUT=30 # Grab *just* the headers, simulating Xymon's builtin http check MESSAGE=`curl -x http://ourproxy.ourdomain.com:8080 \ --proxy-anyauth -U hobbituser:hobbitpassword \ -m $TIMEOUT \ -w 'Seconds: %{time_total}\n' \ -s -S -L -I $TESTURL | $GREP -v Set-Cookie` if [ "$?" -eq "0" ]; then COLOR=green else COLOR=red fi # convert dots to commas in the hostname MACHINE=`echo $TESTHOST | $SED -e 's/\./\,/g' $BB $BBDISP "status $MACHINE.home $COLOR `date` $MESSAGE" ============= cut here ============ On Sun, Nov 23, 2008 at 9:04 PM, Everett, Vernon <user-9da1a1882f49@xymon.invalid<mailto:user-9da1a1882f49@xymon.invalid>> wrote: Hi Ralph This script works a treat. Thanks Vernon From: Ralph Mitchell [mailto:user-00a5e44c48c0@xymon.invalid<mailto:user-00a5e44c48c0@xymon.invalid>] Sent: Saturday, 22 November 2008 12:23 AM To: user-ae9b8668bcde@xymon.invalid<mailto:user-ae9b8668bcde@xymon.invalid> Subject: Re: [hobbit] Testing external web through a proxy Looks like your proxy requires authentication via Kerberos or NTLM or GSS-Negotiate. Dunno if Xymon can do those. Luckily the curl tool (httpp://curl.haxx.se<http://curl.haxx.se>;) *can* do those... I haven't had to use them, but it should not be too hard to bang out an ext script to do the check. Something along these lines: #!/bin/sh curl -x http://ourproxy.ourdomain.com:8080 \ --proxy-anyauth -U hobbituser:hobbitpassword \ -s -S -L -o /dev/null http://www.google.com/ if [ "$?" -eq "0" ]; then COLOR=green MESSAGE="Pinged Google OK" else COLOR=RED MESSAGE="Failed to reach Google" fi $BB $BBDISP "status www,google,com.home $COLOR `date` $MESSAGE" Obviously that can be expanded quite a lot... :) Drop it somewhere on your Xymon server (maybe in server/ext) and set up a block in server/etc/hobbitlaunch.cfg to fire it off as often as you want. You may need to give the full path for curl, which is probably /usr/bin/curl. Ralph Mitchell On Fri, Nov 21, 2008 at 1:30 AM, Everett, Vernon <user-9da1a1882f49@xymon.invalid<mailto:user-9da1a1882f49@xymon.invalid>> wrote: Hi all I am trying to test our interent connectivity by checking Google through our proxy. (Once I get this right, I want to start testing some of our external web pages in the same manner) I have configured this in bb-hosts 0.0.0.0<http://0.0.0.0>; www.google.com<http://www.google.com>; # noconn http://hobbituser:user-1b7da96bb10f@xymon.invalid:8080/http://www.google.com And I get this. yellow<http://hobbit/hobbit/gifs/yellow.gif>; http://www.google.com.au/ - Proxy Authentication Required HTTP/1.1 407 Proxy Authentication Required ( The ISA Server requires authorization to fulfill the request. Access to the Web Proxy filter is denied. ) Via: 1.1 PXYHOSTNAME Proxy-Authenticate: Negotiate Proxy-Authenticate: Kerberos Proxy-Authenticate: NTLM Connection: Keep-Alive Proxy-Connection: Keep-Alive Pragma: no-cache Cache-Control: no-cache Content-Type: text/html Content-Length: 4118 Seconds: 0.02 Any assistance appreciated. Regards Vernon NOTICE: This email and any attachments are confidential. They may contain legally privileged information or copyright material. You must not read, copy, use or disclose them without authorisation. If you are not an intended recipient, please contact us at once by return email and then delete both messages and all attachments. NOTICE: This email and any attachments are confidential. They may contain legally privileged information or copyright material. You must not read, copy, use or disclose them without authorisation. If you are not an intended recipient, please contact us at once by return email and then delete both messages and all attachments. NOTICE: This email and any attachments are confidential. They may contain legally privileged information or copyright material. You must not read, copy, use or disclose them without authorisation. If you are not an intended recipient, please contact us at once by return email and then delete both messages and all attachments.