semop failed, Invalid argument 🔗 link

I have been battling this semop failed message for a couple of months on multiple servers.  On some servers it is only occasionally, on this particular server it is happened over 90 times in a single day (on multiple occasions).

Then, there are days it runs fine.  I have upgraded to 4.3.30 on this particular server though it has happened using 4.3.21 & 4.3.28.

Reboots do not solve it.

It is so bad that I have a cron script running every 10 minutes looking for the errors in the xymod.log file and restarting Xymon automatically.
Xymond.log:
2019-11-01 11:33:12.197864 Setup complete
2019-11-01 11:45:07.029630 semop failed, Invalid argument
2019-11-01 11:45:07.029669 Dropping message due to semaphore error
2019-11-01 11:45:07.083973 semop failed, Invalid argument
2019-11-01 11:45:07.083982 Dropping message due to semaphore error
....  <hundreds to thousands of these messages>

Ubuntu 16.04LTS
Xymon 4.3.30

Thanks,
John
Upcoming PTO:
John Rothlisberger
IT Strategy, Infrastructure & Security - Technology Growth Platform
TGP for Business Process Outsourcing
Accenture
XXX.XXX.XXXX office


This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited. Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy. Your privacy is important to us. Accenture uses your personal data only in compliance with data protection laws. For further information on how Accenture processes your personal data, please see our privacy statement at https://www.accenture.com/us-en/privacy-policy.

www.accenture.com

Does anyone have any helpful ideas?

Thanks,
John

From: Rothlisberger, John R. <user-7adce57665bb@xymon.invalid>
Sent: Friday, November 1, 2019 7:31 AM
To: xymon at xymon.com
Subject: semop failed, Invalid argument

I have been battling this semop failed message for a couple of months on multiple servers.  On some servers it is only occasionally, on this particular server it is happened over 90 times in a single day (on multiple occasions).

Then, there are days it runs fine.  I have upgraded to 4.3.30 on this particular server though it has happened using 4.3.21 & 4.3.28.

Reboots do not solve it.

It is so bad that I have a cron script running every 10 minutes looking for the errors in the xymod.log file and restarting Xymon automatically.
Xymond.log:
2019-11-01 11:33:12.197864 Setup complete
2019-11-01 11:45:07.029630 semop failed, Invalid argument
2019-11-01 11:45:07.029669 Dropping message due to semaphore error
2019-11-01 11:45:07.083973 semop failed, Invalid argument
2019-11-01 11:45:07.083982 Dropping message due to semaphore error
....  <hundreds to thousands of these messages>

Ubuntu 16.04LTS
Xymon 4.3.30

Thanks,
John
Upcoming PTO:
John Rothlisberger
IT Strategy, Infrastructure & Security - Technology Growth Platform
TGP for Business Process Outsourcing
Accenture
XXX.XXX.XXXX office


This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited. Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy. Your privacy is important to us. Accenture uses your personal data only in compliance with data protection laws. For further information on how Accenture processes your personal data, please see our privacy statement at https://www.accenture.com/us-en/privacy-policy.

www.accenture.com<http://www.accenture.com>;

On Tue, 2019-11-12 at 17:03 +0000, Rothlisberger, John R. via Xymon wrote:

...

Not really I'm afraid.

The actual error is the invalid argument bit. It refers to a call to 'semop'. A
very quick look at the xymond code seems to indicate that it would be the
semaphore id that it is objecting to - basically it doesn't exist or the id is
less than zero.

How that came about though, I have no idea. It may be that the code is being
called when it shouldn't be - perhaps a lack of argument checking. It would
require much further checking of the code to see what is happening. Maybe
running xymond is debug mode will give more information.


John.

--
John Horne | Senior Operations Analyst | Technology and Information Services
University of Plymouth | Drake Circus | Plymouth | Devon | PL4 8AA | UK
[http://www.plymouth.ac.uk/images/email_footer.gif]<http://www.plymouth.ac.uk/worldclass>;

This email and any files with it are confidential and intended solely for the use of the recipient to whom it is addressed. If you are not the intended recipient then copying, distribution or other use of the information contained is strictly prohibited and you should not rely on it. If you have received this email in error please let the sender know immediately and delete it from your system(s). Internet emails are not necessarily secure. While we take every care, University of Plymouth accepts no responsibility for viruses and it is your responsibility to scan emails and their attachments. University of Plymouth does not accept responsibility for any changes made after it was sent. Nothing in this email or its attachments constitutes an order for goods or services unless accompanied by an official order form.