URL monitoring 🔗 link

I know that hobbit can monitor URLs, of course.

I have a need to monitor a constantly changing list of URLs, which could
number in the hundreds or thousands.

Will hobbit scale to this level, or should I instead write something custom
using wget, which can take a file containing a list of URLs as input?

Thanks for your suggestions, and thanks for hobbit.

| Dan |

On Wed, May 31, 2006 at 12:57:42PM -0700, Dan Simoes wrote:

I'm obviously biased, but I think you'll find Hobbit a lot more
efficient at testing your URL's than any script using wget.

Now, the way Hobbit's URL testing reports results might not be what you
want - it is after all designed to report data to a Hobbit server. I
don't know what you plan to do with all of your test results ... But
I think it should be possible to tweak the network test tool to report
in a format suitable for what you need.


Regards,
Henrik

Thanks for your reply Henrik.
I'm all for avoiding reinventing the wheel :)

Do you have any suggestions on how to mate a rapidly changing list of URLs
with the bb-hosts file?
Also, I assume that hobbit, like bb, requires a restart whenever bb-hosts
changes?

Finally, has anyone compiled a list of differences between bb and hobbit,
sort of like "Hobbit for BB admins"?  I've encountered some minor
differences here and there, which is not unexpected.

| Dan |

On 5/31/06, Henrik Stoerner <user-ce4a2c883f75@xymon.invalid> wrote:

On Wed, May 31, 2006 at 01:34:48PM -0700, Dan Simoes wrote:

How rapid is "rapid" ? Once or twice a day, or several times a minute ?

My immediate thought is to put an "include urlhosts.txt" in the bb-hosts
file, and then have some script which updates the "urlhosts.txt" file
with your dynamic list of URL's and the "hosts" you want to map them to.
I don't know if all of your URL's pertain to the same host(s), but if
you have several hundred then you may want to split them up in some 
way that makes sense for your setup. Remember, "hosts" (entries in the
bb-hosts file) need not be real, physical servers. They are really just
entities that you can connect with a series of tests.

So come up with a script to generate the urlhosts.txt file in bb-hosts
format, like
   0.0.0.0   web1   # noconn http://web1.foo.com/ https://web1.foo.com/breakingnews/
   0.0.0.0   web2   # noconn http://....

Nope. Hobbit automatically reloads the bb-hosts file every 10 minutes,
or when you send it a HUP signal.

I guess there are some differences, but I've been embroiled in Hobbit
for so long I've lost track of how BB did things. So someone else will
have to answer that one.


Regards,
Henrik