Critical Systems page displaying "Internal Server Error"
list Gary Baluha
It seems that the Critical Systems page has decided to stop working on me. Any time I load the page, it just displays an Internal Server Error. I enabled the --debug option, and in the apache error logs, I see the following message: [Sun Dec 02 11:11:46 2007] [error] [client x.x.x.x] malformed header from script. Bad header=HOST=: hobbit-nkview.sh "HOST" happens to be the first entry in the hobbit-nkview.cfg file. I have tried a couple of changes to the config file to see what would happen. I first tried deleting the entire contents of the configuration file, and still get the Internal Server Error. When this happens, I don't see any entry added to the apache error log. If I go back to the original hobbit-nkview.cfg file and just delete the first entry, the error above changes to "HOSTB", where HOSTB was the host below HOST, but is now the first. Essentially, I have determined that whatever is the first entry in the hobbit-nkview.cfg file is the host that appears as the "Bad header=" in the error above. The hobbit-nkview.cfg file has not changed in at least a week, and the Critical Systems page was working as of only 3 days ago. Any idea what's going on? I have seen the "Internal Server Error" problem come up before, but it would usually be fine on the next reload of the page. Also, the HOST that the error message is complaining about doesn't have any non-green alerts, so it appears to be some problem with the parsing of the configuration file. Is there additional logging I can turn on? I tried enabling the --debug for the bb-display module, but that just produced a huge amount of logging, but nothing that helped to narrow down the problem.
list Gary Baluha
I seem to be on a roll with weird problems lately. I just found out that the Critical Systems page doesn't load at all if I zero out the hobbit-nkview.cfg file. At least, not in a typical browser. If I curl or lynx the critical systems page from the command line, it loads okay. It just doesn't load in a graphical browser.
▸
On Dec 2, 2007 1:42 PM, Gary Baluha <user-ae3e15c22de1@xymon.invalid> wrote:
It seems that the Critical Systems page has decided to stop working on me. Any time I load the page, it just displays an Internal Server Error. I enabled the --debug option, and in the apache error logs, I see the following message: [Sun Dec 02 11:11:46 2007] [error] [client x.x.x.x ] malformed header from script. Bad header=HOST=: hobbit-nkview.sh "HOST" happens to be the first entry in the hobbit-nkview.cfg file. I have tried a couple of changes to the config file to see what would happen. I first tried deleting the entire contents of the configuration file, and still get the Internal Server Error. When this happens, I don't see any entry added to the apache error log. If I go back to the original hobbit-nkview.cfg file and just delete the first entry, the error above changes to "HOSTB", where HOSTB was the host below HOST, but is now the first. Essentially, I have determined that whatever is the first entry in the hobbit-nkview.cfg file is the host that appears as the "Bad header=" in the error above. The hobbit-nkview.cfg file has not changed in at least a week, and the Critical Systems page was working as of only 3 days ago. Any idea what's going on? I have seen the "Internal Server Error" problem come up before, but it would usually be fine on the next reload of the page. Also, the HOST that the error message is complaining about doesn't have any non-green alerts, so it appears to be some problem with the parsing of the configuration file. Is there additional logging I can turn on? I tried enabling the --debug for the bb-display module, but that just produced a huge amount of logging, but nothing that helped to narrow down the problem.
list Josh Luthman
Can you confirm that these are all similar... -> cat /home/shire/cgi-bin/hobbit-nkview.sh #!/bin/sh # This is a wrapper for the Hobbit hobbit-nkview.cgi script . /home/shire/server/etc/hobbitcgi.cfg exec /home/shire/server/bin/hobbit-nkview.cgi $CGI_NKVIEW_OPTS -> grep CGI_NKVIEW_OPTS hobbitcgi.cfg CGI_NKVIEW_OPTS="--env=/home/shire/server/etc/hobbitserver.cfg" Josh
▸
On 12/3/07, Gary Baluha <user-ae3e15c22de1@xymon.invalid> wrote:I seem to be on a roll with weird problems lately. I just found out that the Critical Systems page doesn't load at all if I zero out the hobbit-nkview.cfg file. At least, not in a typical browser. If I curl or lynx the critical systems page from the command line, it loads okay. It just doesn't load in a graphical browser. On Dec 2, 2007 1:42 PM, Gary Baluha <user-ae3e15c22de1@xymon.invalid> wrote:It seems that the Critical Systems page has decided to stop working on me. Any time I load the page, it just displays an Internal Server Error. I enabled the --debug option, and in the apache error logs, I see the following message: [Sun Dec 02 11:11:46 2007] [error] [client x.x.x.x ] malformed header from script. Bad header=HOST=: hobbit-nkview.sh "HOST" happens to be the first entry in the hobbit-nkview.cfg file. I have tried a couple of changes to the config file to see what would happen. I first tried deleting the entire contents of the configuration file, and still get the Internal Server Error. When this happens, I don't see any entry added to the apache error log. If I go back to the original hobbit-nkview.cfg file and just delete the first entry, the error above changes to "HOSTB", where HOSTB was the host below HOST, but is now the first. Essentially, I have determined that whatever is the first entry in the hobbit-nkview.cfg file is the host that appears as the "Bad header=" in the error above. The hobbit-nkview.cfg file has not changed in at least a week, and the Critical Systems page was working as of only 3 days ago. Any idea what's going on? I have seen the "Internal Server Error" problem come up before, but it would usually be fine on the next reload of the page. Also, the HOST that the error message is complaining about doesn't have any non-green alerts, so it appears to be some problem with the parsing of the configuration file. Is there additional logging I can turn on? I tried enabling the --debug for the bb-display module, but that just produced a huge amount of logging, but nothing that helped to narrow down the problem.
--
Josh Luthman
Office: XXX-XXX-XXXX
Direct: XXX-XXX-XXXX
XXXX Wayne St
Suite XXXX
Troy, OH XXXXX
Those who don't understand UNIX are condemned to reinvent it, poorly.
--- Henry Spencer
list Gary Baluha
Yep, looks the same. I turned on ScriptLog in the apache configuration, and got some interesting output. %% [Mon Dec 03 15:34:21 2007] GET /hobbit-cgi/hobbit-nkview.sh HTTP/1.1 %% 500 /var/hobbit/cgi-bin/hobbit-nkview.sh %request Host: hobbit.hidden.com User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20070309 Firefox/2.0.0.3 Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9 ,text/plain;q=0.8,image/png,*/*;q=0.5 Accept-Language: en-us,en;q=0.5 Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 300 Connection: keep-alive Referer: http://hobbit.hidden.com/hobbit/bbnk Cookie: pagepath=; host= Cache-Control: max-age=0 %response HOST= <snip> So at least now, I see where that "header=HOST=" line in the error_log file is coming from.
▸
On Dec 3, 2007 3:33 PM, Josh Luthman <user-4c45a83f15cb@xymon.invalid> wrote:
Can you confirm that these are all similar... -> cat /home/shire/cgi-bin/hobbit-nkview.sh #!/bin/sh # This is a wrapper for the Hobbit hobbit-nkview.cgi script . /home/shire/server/etc/hobbitcgi.cfg exec /home/shire/server/bin/hobbit-nkview.cgi $CGI_NKVIEW_OPTS -> grep CGI_NKVIEW_OPTS hobbitcgi.cfg CGI_NKVIEW_OPTS="--env=/home/shire/server/etc/hobbitserver.cfg" Josh On 12/3/07, Gary Baluha <user-ae3e15c22de1@xymon.invalid> wrote:I seem to be on a roll with weird problems lately. I just found out that the Critical Systems page doesn't load at all if I zero out the hobbit-nkview.cfg file. At least, not in a typical browser. If I curl or lynx the critical systems page from the command line, it loads okay. It just doesn't load in a graphical browser. On Dec 2, 2007 1:42 PM, Gary Baluha < user-ae3e15c22de1@xymon.invalid> wrote:It seems that the Critical Systems page has decided to stop working on me. Any time I load the page, it just displays an Internal Server Error. I enabled the --debug option, and in the apache error logs, I see the following message: [Sun Dec 02 11:11:46 2007] [error] [client x.x.x.x ] malformed header from script. Bad header=HOST=: hobbit-nkview.sh "HOST" happens to be the first entry in the hobbit-nkview.cfg file. I have tried a couple of changes to the config file to see what would happen. I first tried deleting the entire contents of the configuration file, and still get the Internal Server Error. When this happens, I don't see any entry added to the apache error log. If I go back to the original hobbit-nkview.cfg file and just delete the first entry, the error above changes to "HOSTB", where HOSTB was the host below HOST, but is now the first. Essentially, I have determined that whatever is the first entry in the hobbit-nkview.cfg file is the host that appears as the "Bad header=" in the error above. The hobbit-nkview.cfg file has not changed in at least a week, and the Critical Systems page was working as of only 3 days ago. Any idea what's going on? I have seen the "Internal Server Error" problem come up before, but it would usually be fine on the next reload of the page. Also, the HOST that the error message is complaining about doesn't have any non-green alerts, so it appears to be some problem with the parsing of the configuration file. Is there additional logging I can turn on? I tried enabling the --debug for the bb-display module, but that just produced a huge amount of logging, but nothing that helped to narrow down the problem.-- Josh Luthman Office: XXX-XXX-XXXX Direct: XXX-XXX-XXXX XXXX Wayne St Suite XXXX Troy, OH XXXXX Those who don't understand UNIX are condemned to reinvent it, poorly. --- Henry Spencer
list Gary Baluha
It appears there was a permissions issue. The critical systems page is working again.
▸
On Dec 3, 2007 3:48 PM, Gary Baluha <user-ae3e15c22de1@xymon.invalid> wrote:
Yep, looks the same. I turned on ScriptLog in the apache configuration, and got some interesting output. %% [Mon Dec 03 15:34:21 2007] GET /hobbit-cgi/hobbit-nkview.sh HTTP/1.1
%% 500 /var/hobbit/cgi-bin/hobbit- nkview.sh
▸
%request
Host: hobbit.hidden.com
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3)
Gecko/20070309 Firefox/2.0.0.3
Accept: text/xml,application/xml,application/xhtml+xml,text/html;q= 0.9
▸
,text/plain;q=0.8,image/png,*/*;q=0.5 Accept-Language: en-us,en;q=0.5 Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 300 Connection: keep-alive Referer: http://hobbit.hidden.com/hobbit/bbnk Cookie: pagepath=; host= Cache-Control: max-age=0 %response HOST= <snip> So at least now, I see where that "header=HOST=" line in the error_log file is coming from. On Dec 3, 2007 3:33 PM, Josh Luthman <user-4c45a83f15cb@xymon.invalid> wrote:Can you confirm that these are all similar... -> cat /home/shire/cgi-bin/hobbit-nkview.sh #!/bin/sh # This is a wrapper for the Hobbit hobbit-nkview.cgi script . /home/shire/server/etc/hobbitcgi.cfg exec /home/shire/server/bin/hobbit-nkview.cgi $CGI_NKVIEW_OPTS -> grep CGI_NKVIEW_OPTS hobbitcgi.cfg CGI_NKVIEW_OPTS="--env=/home/shire/server/etc/hobbitserver.cfg" Josh On 12/3/07, Gary Baluha <user-ae3e15c22de1@xymon.invalid> wrote:I seem to be on a roll with weird problems lately. I just found out that the Critical Systems page doesn't load at all if I zero out the hobbit-nkview.cfg file. At least, not in a typical browser. If I curl or lynx the critical systems page from the command line, it loads okay. It just doesn't load in a graphical browser. On Dec 2, 2007 1:42 PM, Gary Baluha < user-ae3e15c22de1@xymon.invalid> wrote:It seems that the Critical Systems page has decided to stop working on me. Any time I load the page, it just displays an Internal Server Error. I enabled the --debug option, and in the apache error logs, I see the following message: [Sun Dec 02 11:11:46 2007] [error] [client x.x.x.x ] malformed header from script. Bad header=HOST=: hobbit-nkview.sh "HOST" happens to be the first entry in the hobbit-nkview.cfg file. I have tried a couple of changes to the config file to see what would happen. I first tried deleting the entire contents of the configuration file, and still get the Internal Server Error. When this happens, I don't see any entry added to the apache error log. If I go back to the original hobbit-nkview.cfg file and just delete the first entry, the error above changes to "HOSTB", where HOSTB was the host below HOST, but is now the first. Essentially, I have determined that whatever is the first entry in the hobbit-nkview.cfg file is the host that appears as the "Bad header=" in the error above. The hobbit-nkview.cfg file has not changed in at least a week, and the Critical Systems page was working as of only 3 days ago. Any idea what's going on? I have seen the "Internal Server Error" problem come up before, but it would usually be fine on the next reload of the page. Also, the HOST that the error message is complaining about doesn't have any non-green alerts, so it appears to be some problem with the parsing of the configuration file. Is there additional logging I can turn on? I tried enabling the --debug for the bb-display module, but that just produced a huge amount of logging, but nothing that helped to narrow down the problem.-- Josh Luthman Office: XXX-XXX-XXXX Direct: XXX-XXX-XXXX XXXX Wayne St Suite XXXX Troy, OH XXXXX Those who don't understand UNIX are condemned to reinvent it, poorly. --- Henry Spencer
list Josh Luthman
Didn't the apache httpd log say there was a permissions issue instead of the internal server error?
▸
On 12/4/07, Gary Baluha <user-ae3e15c22de1@xymon.invalid> wrote:It appears there was a permissions issue. The critical systems page is working again. On Dec 3, 2007 3:48 PM, Gary Baluha <user-ae3e15c22de1@xymon.invalid> wrote:Yep, looks the same. I turned on ScriptLog in the apache configuration, and got some interesting output. %% [Mon Dec 03 15:34:21 2007] GET /hobbit-cgi/hobbit-nkview.sh HTTP/1.1 %% 500 /var/hobbit/cgi-bin/hobbit- nkview.sh %request Host: hobbit.hidden.com User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20070309 Firefox/2.0.0.3 Accept: text/xml,application/xml,application/xhtml+xml,text/html;q= 0.9 ,text/plain;q=0.8,image/png,*/*;q=0.5 Accept-Language: en-us,en;q=0.5 Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 300 Connection: keep-alive Referer: http://hobbit.hidden.com/hobbit/bbnk Cookie: pagepath=; host= Cache-Control: max-age=0 %response HOST= <snip> So at least now, I see where that "header=HOST=" line in the error_log file is coming from. On Dec 3, 2007 3:33 PM, Josh Luthman <user-4c45a83f15cb@xymon.invalid> wrote:Can you confirm that these are all similar... -> cat /home/shire/cgi-bin/hobbit-nkview.sh #!/bin/sh # This is a wrapper for the Hobbit hobbit-nkview.cgi script . /home/shire/server/etc/hobbitcgi.cfg exec /home/shire/server/bin/hobbit-nkview.cgi $CGI_NKVIEW_OPTS -> grep CGI_NKVIEW_OPTS hobbitcgi.cfg CGI_NKVIEW_OPTS="--env=/home/shire/server/etc/hobbitserver.cfg" Josh On 12/3/07, Gary Baluha <user-ae3e15c22de1@xymon.invalid> wrote:I seem to be on a roll with weird problems lately. I just found out that the Critical Systems page doesn't load at all if I zero out the hobbit-nkview.cfg file. At least, not in a typical browser. If I curl or lynx the critical systems page from the command line, it loads okay. It just doesn't load in a graphical browser. On Dec 2, 2007 1:42 PM, Gary Baluha < user-ae3e15c22de1@xymon.invalid> wrote:It seems that the Critical Systems page has decided to stop working on me. Any time I load the page, it just displays an Internal Server Error. I enabled the --debug option, and in the apache error logs, I see the following message: [Sun Dec 02 11:11:46 2007] [error] [client x.x.x.x ] malformed header from script. Bad header=HOST=: hobbit-nkview.sh
"HOST" happens to be the first entry in the hobbit-nkview.cfgfile. I have tried a couple of changes to the config file to see what would
▸
happen. I first tried deleting the entire contents of the configuration file, and still get the Internal Server Error. When this happens, I don't see any entry added to the apache error log. If I go back to the original hobbit-nkview.cfg file and just delete the first entry, the error above changes to "HOSTB", where HOSTB was the host below HOST, but is now the first. Essentially, I have determined that whatever is the first entry in the hobbit-nkview.cfg file is the host that appears as the "Bad header=" in the error above. The hobbit-nkview.cfg file has not changed in at least a week, and the Critical Systems page was working as of only 3 days ago. Any idea what's going on? I have seen the "Internal Server Error" problem come up before, but it would usually be fine on the next reload of the page. Also, the HOST that the error message is complaining about doesn't have any non-green alerts, so it appears to be some problem with the parsing of the configuration file. Is there additional logging I can turn on? I tried enabling the --debug for the bb-display module, but that just produced a huge amount of logging, but nothing that helped to narrow down the problem.-- Josh Luthman Office: XXX-XXX-XXXX Direct: XXX-XXX-XXXX XXXX Wayne St Suite XXXX Troy, OH XXXXX Those who don't understand UNIX are condemned to reinvent it, poorly. --- Henry Spencer
-- Josh Luthman Office: XXX-XXX-XXXX Direct: XXX-XXX-XXXX XXXX Wayne St Suite XXXX Troy, OH XXXXX Those who don't understand UNIX are condemned to reinvent it, poorly. --- Henry Spencer