Xymon Mailing List Archive search

[feature request] --follow http redirect support come back

4 messages in this thread

list Charles Goyard · Wed, 7 Jun 2006 17:15:25 +0200 ·
Hi all,

this is my first post here, so I'll introduce myself briefly : I'm a
sysadmin for 4 or 5 years, I live in Paris, France. I built up a Big
Brother + bbgen 2.x server at our company, taking care of more than 400
hosts. bbgen as proven so useful that we not want to drop Big Brother in
favour of Hobbit (even thought I dislike Tolkien writings and like
Orwell ones).

However, there's a catch. Since bbgen 3.x, the libcurl has been dropped
for bbtest-net, and the ability to follow Redirects for http
disappeared. Of course, I need it for some major websites. I have
several options :

(1)/ ask Henrik to implement a straightforward 302 support ;

(2) implement it myself into bbtest-net ;

(3) write an external module based on bbtest-net 2.x to perform the http
tests. This makes a "--no-http" switch into hobbit useful, so I can keep
my URLs into the regular bb-hosts file.

I just had a look at the source code, and (2) is out of my skills. (3)
is a reasonable option. But keeping the hobbit as standard and clean as
possible is important as well, so... (1) Henrik, have you got any plans of
making the http redirect back to work ? (please please please :)).

Regards,

-- 
Charles Goyard - user-98f9625a7a59@xymon.invalid - (+33) 1 45 38 01 31
list Henrik Størner · Thu, 8 Jun 2006 22:28:37 +0200 ·
quoted from Charles Goyard
On Wed, Jun 07, 2006 at 05:15:25PM +0200, Charles Goyard wrote:
Since bbgen 3.x, the libcurl has been dropped
for bbtest-net, and the ability to follow Redirects for http
disappeared. Of course, I need it for some major websites. I have
several options :
[snip]
(1) Henrik, have you got any plans of
making the http redirect back to work ? (please please please :)).
I do, but it won't happen in the near future. So you may want to
try out the custom script mechanism until I get around to it.

My hope is that this will be a rather easy thing to do, once I
get around to rewriting the network tester so that it can engage
in a real dialogue. Currently the network tester connects,
sends something, and grabs the response. Period. It won't look
at the response and go "hey, I need to send another request"
and then do that. This is also causing problems for e.g. SMTP
checks, because the current default Hobbit setup is actually
not SMTP protocol compliant - and a few SMTP servers out there
actually complain about it, causing Hobbit to think they are
out of order, when in fact they are running fine.

So - the plan is to put some more advanced communications
mechanism into the network tester, and with that in place it
should be easier to provide the redirect-support you need.


Regards,
Henrik
list Adam Goryachev · Fri, 09 Jun 2006 09:39:23 +1000 ·
quoted from Henrik Størner
On Thu, 2006-06-08 at 22:28 +0200, Henrik Stoerner wrote:
My hope is that this will be a rather easy thing to do, once I
get around to rewriting the network tester so that it can engage
in a real dialogue. Currently the network tester connects,
sends something, and grabs the response. Period. It won't look
at the response and go "hey, I need to send another request"
and then do that. This is also causing problems for e.g. SMTP
checks, because the current default Hobbit setup is actually
not SMTP protocol compliant - and a few SMTP servers out there
actually complain about it, causing Hobbit to think they are
out of order, when in fact they are running fine.

So - the plan is to put some more advanced communications
mechanism into the network tester, and with that in place it
should be easier to provide the redirect-support you need.
I have been thinking recently (well, since I screwed up the pop server
last weekend such that it passed the hobbit pop test, but no user could
collect any email), that it would be really nice to extend the network
tests to engage in a "real" test.

ie, the SMTP test would actually send an email
the POP test would actually retrieve an email
etc...

Once, many moons ago, I did write some utils to do these things with BB,
in fact, it actually went one step further. The email which was sent via
SMTP had to show up on the POP test within a certain time frame or else
it would also alarm. Since we had a seperate farm for POP and SMTP, this
meant we could confirm there were no delays between the two sides etc...

Just my 5c worth...

If I get some time, I'll go and write some ext scripts to do this stuff,
but it doesn't look likely at the moment.

However, I have recently added tinydns statistics graphing recently, so
here is how it was done:
Install tinystats as per the installation instructions
I'm still using BB client on my two old DNS servers and hobbit on the
two new ones, so this ext script works on both:
tinystats.sh
#!/bin/bash

CAT=/bin/cat
CUT=/usr/bin/cut

STATFILE=/var/dns/tinydns/log/main/tinystats.out
TMPSTAT=$BBTMP/tinystats.$$
TMPSTATout=$BBTMP/tinystats.out.$$
TEST=dnsnum

${TAIL} -1 ${STATFILE} > ${TMPSTAT}
${HEAD} -1 ${STATFILE} >> ${TMPSTAT}

for i in 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
do
        stats="`${CAT} ${TMPSTAT}`"
        output=`echo "$stats" | ${CUT} -d: -f${i}`
        echo ${output} | ${TR} " " ":" >> $TMPSTATout
done

${BB} ${BBDISP} "status ${MACHINE}.${TEST} green `${DATE}` DNS
Statistics
`cat ${TMPSTATout}`"

${RM} -f ${TMPSTAT} ${TMPSTATout}

PS, it is a crap script, but it works for me, if anyone can suggest a
better way to pull out the data, I'd appreciate it :)

add "dnsnum=ncv" to the end of your TEST2RRD string in hobbitserver.cfg
add "dnsnum" to the end of your GRAPHS string in hobbitserver.cfg
add the following to your hobbitgraph.cfg
[dnsnum]
        TITLE TinyDNS Utilization
        YAXIS Stats
        DEF:A=dnsnum.rrd:a:AVERAGE
        DEF:NS=dnsnum.rrd:ns:AVERAGE
        DEF:CNAME=dnsnum.rrd:cname:AVERAGE
        DEF:SOA=dnsnum.rrd:soa:AVERAGE
        DEF:PTR=dnsnum.rrd:ptr:AVERAGE
        DEF:MX=dnsnum.rrd:mx:AVERAGE
        DEF:TXT=dnsnum.rrd:txt:AVERAGE
        DEF:AAAA=dnsnum.rrd:aaaa:AVERAGE
        DEF:ANY=dnsnum.rrd:any:AVERAGE
        DEF:OTHER=dnsnum.rrd:other:AVERAGE
        DEF:NOTAUTH=dnsnum.rrd:notauth:AVERAGE
        DEF:NOTIMPL=dnsnum.rrd:notimpl:AVERAGE
        DEF:BADCLASS=dnsnum.rrd:badclass:AVERAGE
        LINE2:A#0000FF:A
        LINE2:NS#000088:NS
        LINE2:CNAME#00FF00:CNAME
        LINE2:SOA#008800:SOA
        LINE2:PTR#FF0000:PTR
        LINE2:MX#880000:MX
        LINE2:TXT#FF00FF:TXT
        LINE2:AAAA#880088:AAAA
        LINE2:ANY#FFFF00:ANY
        LINE2:OTHER#555555:O
        LINE2:NOTAUTH#0088FF:NOTAUTH
        LINE2:NOTIMPL#008888:NOTIMPL
        LINE2:BADCLASS#00FFFF:BADCLASS
        COMMENT:\n
        GPRINT:A:LAST:A     \: %5.1lf (cur)
        GPRINT:A:MAX:A    \: %5.1lf (max)
        GPRINT:A:MIN:A     \: %5.1lf (min)
        GPRINT:A:AVERAGE:A     \: %5.1lf (avg)\n
        GPRINT:ANY:LAST:ANY   \: %5.1lf (cur)
        GPRINT:ANY:MAX:ANY  \: %5.1lf (max)
        GPRINT:ANY:MIN:ANY   \: %5.1lf (min)
        GPRINT:ANY:AVERAGE:ANY   \: %5.1lf (avg)\n
        GPRINT:MX:LAST:MX    \: %5.1lf (cur)
        GPRINT:MX:MAX:MX   \: %5.1lf (max)
        GPRINT:MX:MIN:MX    \: %5.1lf (min)
        GPRINT:MX:AVERAGE:MX    \: %5.1lf (avg)\n
        GPRINT:OTHER:LAST:OTHER     \: %5.1lf (cur)
        GPRINT:OTHER:MAX:OTHER    \: %5.1lf (max)
        GPRINT:OTHER:MIN:OTHER     \: %5.1lf (min)
        GPRINT:OTHER:AVERAGE:OTHER     \: %5.1lf (avg)\n
        GPRINT:PTR:LAST:PTR   \: %5.1lf (cur)
        GPRINT:PTR:MAX:PTR  \: %5.1lf (max)
        GPRINT:PTR:MIN:PTR   \: %5.1lf (min)
        GPRINT:PTR:AVERAGE:PTR   \: %5.1lf (avg)\n
        GPRINT:SOA:LAST:SOA   \: %5.1lf (cur)
        GPRINT:SOA:MAX:SOA  \: %5.1lf (max)
        GPRINT:SOA:MIN:SOA   \: %5.1lf (min)
        GPRINT:SOA:AVERAGE:SOA   \: %5.1lf (avg)\n

BTW, that needs further adjustment, I'd like to split it into two
graphs, one for error, and one for 'normal'. Also, it doesn't show data
for the various stats that all showed as 0 on my servers. Also, you may
want more GPRINT lines, I just copied/modified the bind ones....

That it all....

Apart from the above graphing things, the only other thing I haven't
worked out/looked up is how to get the graph to show up on the dnsnum
column, currently it is only on the trends column.

Hope that helps someone else out there.

Regards,
Adam
list Charles Goyard · Tue, 27 Jun 2006 09:06:08 +0200 ·
quoted from Henrik Størner
Henrik Stoerner a écrit :
On Wed, Jun 07, 2006 at 05:15:25PM +0200, Charles Goyard wrote:
Since bbgen 3.x, the libcurl has been dropped
for bbtest-net, and the ability to follow Redirects for http
disappeared. Of course, I need it for some major websites. I have
several options :
[snip]
(1) Henrik, have you got any plans of
making the http redirect back to work ? (please please please :)).
I do, but it won't happen in the near future. So you may want to
try out the custom script mechanism until I get around to it.
Hi all,

I made a mixed hobbit/bbgen 2.x setup. It's basically a big deletion of code
in bbtest-net2 (the diff looks like a bunch of "-" lines) and a brand new "--nohttp" for
bbtest-net4.

The code is so clean it's been a pleasure to poke around, thanks again
Henrik

--
Charles