Errors for acknowledge alerts in 4.3.18 🔗 link

On 20 June 2015 at 04:48, Nicole Beck <user-80034b0579c6@xymon.invalid> wrote:

 Hello,

I just upgraded from 4.2.3 to 4.3.18, and we are getting errors when
acknowledging alerts. I saw other people wrote to the list with similar
problems, but I didn't really see a solution (or I just missed it.) So I
apologize in advance for the duplicate question. Any help would be
appreciated.


 This is the error we see in the web interface:


 The server encountered an internal error or misconfiguration and was
unable to complete your request.

Please contact the server administrator, user-d313ab318eb0@xymon.invalid and inform them of
the time the error occurred, and anything you might have done that may have
caused the error.

More information about this error may be available in the server error log


 This is in the Apache error log:

[Fri Jun 19 14:37:16 2015] [error] [client xxx.xxx.xxx.xxx] Premature end
of script headers: acknowledge.sh, referer:
https://hobbit-test.syr.edu/xymon-ackcgi/acknowledge.sh


 The messages file indicates it is creating a core file:

Jun 19 14:39:59 hobbit-test kernel: acknowledge.cgi[8882]: segfault at 0
ip 0000000000402b4f sp 00007fffae050310 error 4 in
acknowledge.cgi[400000+25000]
Jun 19 14:39:59 hobbit-test abrtd: Directory
'ccpp-2015-06-19-14:39:59-8882' creation detected
Jun 19 14:39:59 hobbit-test abrt[8883]: Saved core dump of pid 8882
(/usr/local/xymon/xymon-4.3.18/server/bin/acknowledge.cgi) to
/var/spool/abrt/ccpp-2015-06-19-14:39:59-8882 (819200 bytes)
Jun 19 14:39:59 hobbit-test abrtd: Executable
'/usr/local/xymon/xymon-4.3.18/server/bin/acknowledge.cgi' doesn't belong
to any package and ProcessUnpackaged is set to 'no'


 Since we want a certain group of people to be able to acknowledge
alerts, I copied the acknowledge.sh script/link to another cgi directory:


 [root at hobbit-test xymon-current]# ls -al cgi-ack
total 208
drwxr-xr-x 2 xymon xymon   4096 Jun 19 13:32 .
drwxr-xr-x 8 xymon xymon   4096 Mar  5 10:28 ..
lrwxrwxrwx 1 xymon xymon     49 Jun 19 13:32 acknowledge.sh ->
/usr/local/xymon/xymon-current/server/bin/cgiwrap


 This is a portion of the httpd.conf file:


 ScriptAlias /xymon-ackcgi/ "/usr/local/xymon/xymon-current/cgi-ack/"
<Directory "/usr/local/xymon/xymon-current/cgi-ack">
    AllowOverride None
    Options ExecCGI FollowSymLinks Includes
    Order allow,deny
    Allow from all

    Require group ackers
</Directory>

 Thanks,
Nicole

From: Jeremy Laidman <user-71895fb2e44c@xymon.invalid>
Sent: Wednesday, July 1, 2015 11:14 PM
To: Nicole Beck
Cc: xymon at xymon.com
Subject: Re: [Xymon] Errors for acknowledge alerts in 4.3.18

Nicole

I can't answer your question, but it might help if you can post a backtrace.  You can refer to the following thread for an example of how to do this:

http://lists.xymon.com/archive/2014-October/040455.html

Cheers
Jeremy


On 20 June 2015 at 04:48, Nicole Beck <user-80034b0579c6@xymon.invalid<mailto:user-80034b0579c6@xymon.invalid>> wrote:

Hello,

I just upgraded from 4.2.3 to 4.3.18, and we are getting errors when acknowledging alerts. I saw other people wrote to the list with similar problems, but I didn't really see a solution (or I just missed it.) So I apologize in advance for the duplicate question. Any help would be appreciated.


This is the error we see in the web interface:


The server encountered an internal error or misconfiguration and was unable to complete your request.

Please contact the server administrator, user-d313ab318eb0@xymon.invalid and inform them of the time the error occurred, and anything you might have done that may have caused the error.

More information about this error may be available in the server error log


This is in the Apache error log:

[Fri Jun 19 14:37:16 2015] [error] [client xxx.xxx.xxx.xxx] Premature end of script headers: acknowledge.sh, referer: https://hobbit-test.syr.edu/xymon-ackcgi/acknowledge.sh


The messages file indicates it is creating a core file:

Jun 19 14:39:59 hobbit-test kernel: acknowledge.cgi[8882]: segfault at 0 ip 0000000000402b4f sp 00007fffae050310 error 4 in acknowledge.cgi[400000+25000]
Jun 19 14:39:59 hobbit-test abrtd: Directory 'ccpp-2015-06-19-14:39:59-8882' creation detected
Jun 19 14:39:59 hobbit-test abrt[8883]: Saved core dump of pid 8882 (/usr/local/xymon/xymon-4.3.18/server/bin/acknowledge.cgi) to /var/spool/abrt/ccpp-2015-06-19-14:39:59-8882 (819200 bytes)
Jun 19 14:39:59 hobbit-test abrtd: Executable '/usr/local/xymon/xymon-4.3.18/server/bin/acknowledge.cgi' doesn't belong to any package and ProcessUnpackaged is set to 'no'


Since we want a certain group of people to be able to acknowledge alerts, I copied the acknowledge.sh script/link to another cgi directory:


[root at hobbit-test xymon-current]# ls -al cgi-ack
total 208
drwxr-xr-x 2 xymon xymon   4096 Jun 19 13:32 .
drwxr-xr-x 8 xymon xymon   4096 Mar  5 10:28 ..
lrwxrwxrwx 1 xymon xymon     49 Jun 19 13:32 acknowledge.sh -> /usr/local/xymon/xymon-current/server/bin/cgiwrap


This is a portion of the httpd.conf file:


ScriptAlias /xymon-ackcgi/ "/usr/local/xymon/xymon-current/cgi-ack/"
<Directory "/usr/local/xymon/xymon-current/cgi-ack">
    AllowOverride None
    Options ExecCGI FollowSymLinks Includes
    Order allow,deny
    Allow from all

    Require group ackers
</Directory>


Thanks,
Nicole

From: Xymon [mailto:xymon-bounces at xymon.com] On Behalf Of Nicole Beck
Sent: Thursday, July 02, 2015 10:48 AM
To: Jeremy Laidman
Cc: xymon at xymon.com
Subject: Re: [Xymon] Errors for acknowledge alerts in 4.3.18


Hi Jeremy,

I'm not that familiar with working with core files, but here is the output. Hopefully I did it right.


Thanks,

Nicole


[root at hobbit-test ccpp-2015-07-02-10:41:42-16666]# gdb /usr/local/xymon/xymon-current/server/bin/acknowledge.cgi coredump
GNU gdb (GDB) Red Hat Enterprise Linux (7.2-75.el6)
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>;
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>;...
Reading symbols from /usr/local/xymon/xymon-4.3.18/server/bin/acknowledge.cgi...done.
[New Thread 16666]
Missing separate debuginfo for
Try: yum --enablerepo='*-debug*' install /usr/lib/debug/.build-id/ee/208590e0e20612ddabb0a20acafc7f9bd06e8e
Reading symbols from /usr/lib64/libssl.so.10...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libssl.so.10
Reading symbols from /usr/lib64/libcrypto.so.10...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libcrypto.so.10
Reading symbols from /lib64/librt.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib64/librt.so.1
Reading symbols from /lib64/libpcre.so.0...(no debugging symbols found)...done.
Loaded symbols for /lib64/libpcre.so.0
Reading symbols from /lib64/libc.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib64/libc.so.6
Reading symbols from /lib64/libgssapi_krb5.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/libgssapi_krb5.so.2
Reading symbols from /lib64/libkrb5.so.3...(no debugging symbols found)...done.
Loaded symbols for /lib64/libkrb5.so.3
Reading symbols from /lib64/libcom_err.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/libcom_err.so.2
Reading symbols from /lib64/libk5crypto.so.3...(no debugging symbols found)...done.
Loaded symbols for /lib64/libk5crypto.so.3
Reading symbols from /lib64/libdl.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/libdl.so.2
Reading symbols from /lib64/libz.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib64/libz.so.1
Reading symbols from /lib64/libpthread.so.0...(no debugging symbols found)...done.
[Thread debugging using libthread_db enabled]
Loaded symbols for /lib64/libpthread.so.0
Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/ld-linux-x86-64.so.2
Reading symbols from /lib64/libkrb5support.so.0...(no debugging symbols found)...done.
Loaded symbols for /lib64/libkrb5support.so.0
Reading symbols from /lib64/libkeyutils.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib64/libkeyutils.so.1
Reading symbols from /lib64/libresolv.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/libresolv.so.2
Reading symbols from /lib64/libselinux.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib64/libselinux.so.1
Core was generated by `/usr/local/xymon/xymon-current/server/bin/acknowledge.cgi --env=/usr/local/xymo'.
Program terminated with signal 11, Segmentation fault.
#0  0x0000000000402b4f in main (argc=<value optimized out>,
    argv=<value optimized out>) at acknowledge.c:392
392                             if (strncmp(awalk->period, "hour", 4) == 0) awalk->validity *= 60;
Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.132.el6.x86_64 keyutils-libs-1.4-4.el6.x86_64 krb5-libs-1.10.3-15.el6_5.1.x86_64 libcom_err-1.41.12-18.el6.x86_64 libselinux-2.0.94-5.3.el6_4.1.x86_64 openssl-1.0.1e-16.el6_5.7.x86_64 pcre-7.8-6.el6.x86_64 zlib-1.2.3-29.el6.x86_64
(gdb) bt
#0  0x0000000000402b4f in main (argc=<value optimized out>,
    argv=<value optimized out>) at acknowledge.c:392

From: Jeremy Laidman <user-71895fb2e44c@xymon.invalid<mailto:user-71895fb2e44c@xymon.invalid>>
Sent: Wednesday, July 1, 2015 11:14 PM
To: Nicole Beck
Cc: xymon at xymon.com<mailto:xymon at xymon.com>
Subject: Re: [Xymon] Errors for acknowledge alerts in 4.3.18

Nicole

I can't answer your question, but it might help if you can post a backtrace.  You can refer to the following thread for an example of how to do this:

http://lists.xymon.com/archive/2014-October/040455.html

Cheers
Jeremy


On 20 June 2015 at 04:48, Nicole Beck <user-80034b0579c6@xymon.invalid<mailto:user-80034b0579c6@xymon.invalid>> wrote:

Hello,

I just upgraded from 4.2.3 to 4.3.18, and we are getting errors when acknowledging alerts. I saw other people wrote to the list with similar problems, but I didn't really see a solution (or I just missed it.) So I apologize in advance for the duplicate question. Any help would be appreciated.


This is the error we see in the web interface:


The server encountered an internal error or misconfiguration and was unable to complete your request.

More information about this error may be available in the server error log


This is in the Apache error log:

[Fri Jun 19 14:37:16 2015] [error] [client xxx.xxx.xxx.xxx] Premature end of script headers: acknowledge.sh, referer: https://hobbit-test.syr.edu/xymon-ackcgi/acknowledge.sh


The messages file indicates it is creating a core file:

Jun 19 14:39:59 hobbit-test kernel: acknowledge.cgi[8882]: segfault at 0 ip 0000000000402b4f sp 00007fffae050310 error 4 in acknowledge.cgi[400000+25000]
Jun 19 14:39:59 hobbit-test abrtd: Directory 'ccpp-2015-06-19-14:39:59-8882' creation detected
Jun 19 14:39:59 hobbit-test abrt[8883]: Saved core dump of pid 8882 (/usr/local/xymon/xymon-4.3.18/server/bin/acknowledge.cgi) to /var/spool/abrt/ccpp-2015-06-19-14:39:59-8882 (819200 bytes)
Jun 19 14:39:59 hobbit-test abrtd: Executable '/usr/local/xymon/xymon-4.3.18/server/bin/acknowledge.cgi' doesn't belong to any package and ProcessUnpackaged is set to 'no'


Since we want a certain group of people to be able to acknowledge alerts, I copied the acknowledge.sh script/link to another cgi directory:


[root at hobbit-test xymon-current]# ls -al cgi-ack
total 208
drwxr-xr-x 2 xymon xymon   4096 Jun 19 13:32 .
drwxr-xr-x 8 xymon xymon   4096 Mar  5 10:28 ..
lrwxrwxrwx 1 xymon xymon     49 Jun 19 13:32 acknowledge.sh -> /usr/local/xymon/xymon-current/server/bin/cgiwrap


This is a portion of the httpd.conf file:


ScriptAlias /xymon-ackcgi/ "/usr/local/xymon/xymon-current/cgi-ack/"
<Directory "/usr/local/xymon/xymon-current/cgi-ack">
    AllowOverride None
    Options ExecCGI FollowSymLinks Includes
    Order allow,deny
    Allow from all

    Require group ackers
</Directory>

Thanks,
Nicole

On 03/07/2015 1:07 AM, "Nicole Beck" <user-80034b0579c6@xymon.invalid> wrote:

I probably should add that I have removed the “—no-pin” option from the

CGI_ACK_OPTS in cgioptions.cfg.

I'm not that familiar with working with core files, but here is the
output. Hopefully I did it right.

Program terminated with signal 11, Segmentation fault.
#0  0x0000000000402b4f in main (argc=<value optimized out>,
    argv=<value optimized out>) at acknowledge.c:392
392                             if (strncmp(awalk->period, "hour", 4) ==

0) awalk->validity *= 60;

From: Jeremy Laidman [mailto:user-71895fb2e44c@xymon.invalid]
Sent: Thursday, July 02, 2015 4:14 PM
To: Nicole Beck; user-87556346d4af@xymon.invalid
Cc: xymon at xymon.com
Subject: RE: [Xymon] Errors for acknowledge alerts in 4.3.18


On 03/07/2015 1:07 AM, "Nicole Beck" <user-80034b0579c6@xymon.invalid<mailto:user-80034b0579c6@xymon.invalid>> wrote:

I probably should add that I have removed the “—no-pin” option from the CGI_ACK_OPTS in cgioptions.cfg.

Unlikely to be related to the problem.

I'm not that familiar with working with core files, but here is the output. Hopefully I did it right.

Yes, you've done well.

Program terminated with signal 11, Segmentation fault.
#0  0x0000000000402b4f in main (argc=<value optimized out>,
    argv=<value optimized out>) at acknowledge.c:392
392                             if (strncmp(awalk->period, "hour", 4) == 0) awalk->validity *= 60;

Ah. This is a bug that is triggered when specifying the ack validity period with the string "hour". I don't know the fix, but a work around might be to specify validity without units (I'm guessing it defaults to minutes).

JC, has this bug been fixed in later versions?

J