xymonnet additional capability -- pause
list Shawn Heisey
Over ten years ago, I asked on this list how I could get rid of the "incorrect pipelining message" logged by postfix every time the smtp, smtps, or submission tests are done. https://lists.xymon.com/oldarchive/2010/11/msg00207.html The fix that I claim worked (multiple send commands) was for a job I had at the time. Now I am have a personal mail server (that also runs xymon) and that fix I mentioned so long ago is not working. I tweaked the code to allow a "pause" action in protocols.cfg and after modifying protocols.cfg to utilize it, I have eliminated the "incorrect command pipelining" message in mail.log. Here's the patch: --- xymon-4.3.28/lib/netservices.c 2017-01-05 19:00:06.000000000 -0700 +++ pause-xymon-4.3.28/lib/netservices.c 2021-05-29 12:51:09.717461323 -0600 @@ -259,6 +259,10 @@ } } } + else if (strncmp(l, "pause ", 6) == 0) { + int pausetime = atoi(skipwhitespace(l+5)); + sleep(pausetime); + } } if (fd) stackfclose(fd); Here's the new definitions I created in protocols.cfg: [smtp] pause 2 send "ehlo xymonnet.localdomain\r\n" pause 2 send "mail\r\n" pause 2 send "quit\r\n" expect "220" options banner port 25 [smtps] pause 2 send "ehlo xymonnet.localdomain\r\n" pause 2 send "mail\r\n" pause 2 send "quit\r\n" expect "220" options ssl,banner # No default port-number assignment for smtps - nonstandard according to IANA [submission|msa] pause 2 send "ehlo xymonnet.localdomain\r\n" pause 2 send "mail\r\n" pause 2 send "quit\r\n" expect "220" options banner port 587 And this is now what I see in mail.log (testing smtps and submission: May 29 13:51:15 bilbo postfix/submission/smtpd[16324]: connect from bilbo.elyograg.org[172.31.8.104] May 29 13:51:15 bilbo postfix/submission/smtpd[16324]: disconnect from bilbo.elyograg.org[172.31.8.104] quit=1 commands=1 May 29 13:51:15 bilbo postfix/smtps/smtpd[16325]: connect from bilbo.elyograg.org[172.31.8.104] May 29 13:51:15 bilbo postfix/smtps/smtpd[16325]: disconnect from bilbo.elyograg.org[172.31.8.104] quit=1 commands=1 Interesting thing here is that it's not actually pausing. Which probably means that I implemented it incorrectly. My training on C is ancient and I'm very rusty. But even though it doesn't pause, the error is gone, simply because each smtp command is now sent in a separate packet, which appears to *sometimes* be enough "delay" for postfix to not complain about pipelining. What happens now is occasionally I will get a yellow status on smtps with the xymon UI saying "Service smtps on bilbo.elyograg.org is not OK : Unexpected service response". So I think what I will do before submitting a patch is implement a "null" action (which will do nothing) as well as a "pause" action, and get some help from the real C developers here for making "pause" behave as advertised. Can somebody point me to some instructions on properly creating and submitting a patch? Also, if I could get some info on any other files I need to modify (man pages, readme files, etc), I would really appreciate it. Thanks, Shawn
list Ralph Mitchell
As far as the sleep() thing goes, you may be running into this, from the
'man 3 sleep' man page:
BUGS
sleep() may be implemented using SIGALRM; mixing calls to
alarm(2) and
sleep() is a bad idea.
I don't know how you'd fix it. Hopefully this gives you a starting point.
Ralph Mitchell
▸
On Sat, May 29, 2021 at 4:11 PM Shawn Heisey <user-5d0d01dba542@xymon.invalid> wrote:
Over ten years ago, I asked on this list how I could get rid of the "incorrect pipelining message" logged by postfix every time the smtp, smtps, or submission tests are done. https://lists.xymon.com/oldarchive/2010/11/msg00207.html The fix that I claim worked (multiple send commands) was for a job I had at the time. Now I am have a personal mail server (that also runs xymon) and that fix I mentioned so long ago is not working. I tweaked the code to allow a "pause" action in protocols.cfg and after modifying protocols.cfg to utilize it, I have eliminated the "incorrect command pipelining" message in mail.log. Here's the patch: --- xymon-4.3.28/lib/netservices.c 2017-01-05 19:00:06.000000000 -0700 +++ pause-xymon-4.3.28/lib/netservices.c 2021-05-29 12:51:09.717461323 -0600 @@ -259,6 +259,10 @@ } } } + else if (strncmp(l, "pause ", 6) == 0) { + int pausetime = atoi(skipwhitespace(l+5)); + sleep(pausetime); + } } if (fd) stackfclose(fd); Here's the new definitions I created in protocols.cfg: [smtp] pause 2 send "ehlo xymonnet.localdomain\r\n" pause 2 send "mail\r\n" pause 2 send "quit\r\n" expect "220" options banner port 25 [smtps] pause 2 send "ehlo xymonnet.localdomain\r\n" pause 2 send "mail\r\n" pause 2 send "quit\r\n" expect "220" options ssl,banner # No default port-number assignment for smtps - nonstandard according to IANA [submission|msa] pause 2 send "ehlo xymonnet.localdomain\r\n" pause 2 send "mail\r\n" pause 2 send "quit\r\n" expect "220" options banner port 587 And this is now what I see in mail.log (testing smtps and submission: May 29 13:51:15 bilbo postfix/submission/smtpd[16324]: connect from bilbo.elyograg.org[172.31.8.104] May 29 13:51:15 bilbo postfix/submission/smtpd[16324]: disconnect from bilbo.elyograg.org[172.31.8.104] quit=1 commands=1 May 29 13:51:15 bilbo postfix/smtps/smtpd[16325]: connect from bilbo.elyograg.org[172.31.8.104] May 29 13:51:15 bilbo postfix/smtps/smtpd[16325]: disconnect from bilbo.elyograg.org[172.31.8.104] quit=1 commands=1 Interesting thing here is that it's not actually pausing. Which probably means that I implemented it incorrectly. My training on C is ancient and I'm very rusty. But even though it doesn't pause, the error is gone, simply because each smtp command is now sent in a separate packet, which appears to *sometimes* be enough "delay" for postfix to not complain about pipelining. What happens now is occasionally I will get a yellow status on smtps with the xymon UI saying "Service smtps on bilbo.elyograg.org is not OK : Unexpected service response". So I think what I will do before submitting a patch is implement a "null" action (which will do nothing) as well as a "pause" action, and get some help from the real C developers here for making "pause" behave as advertised. Can somebody point me to some instructions on properly creating and submitting a patch? Also, if I could get some info on any other files I need to modify (man pages, readme files, etc), I would really appreciate it. Thanks, Shawn
list Jeremy Laidman
Shawn An alternative to "waiting a tad" might be to flush the queue. Apparently this can be done by disabling the Nagle algorithm that is designed to collect multiple chunks in a single TCP segment. It appears that you can disable Nagle and re-enable it again, to effectively flush the output queue. A bit more info here: https://stackoverflow.com/questions/855544/is-there-a-way-to-flush-a-posix-socket So you might do something like this (although please note that I'm nto a C programmer): + else if (strncmp(l, "flush", 5) == 0) { + int flag=0; + setsockopt(sock, IPPROTO_TCP, TCP_NODELAY, (char *) &flag, sizeof(int)); + int flag=1; + setsockopt(sock, IPPROTO_TCP, TCP_NODELAY, (char *) &flag, sizeof(int)); + } Having said that, I don't think this is going to work. The netservices.c file you're trying to patch is only used when reading the config file. This is probably why your delay is not affecting the connection - it's probably delaying only the reading of the configuration file. I think the file that does the sending is contest.c. I can't see a simple way of disabling and re-enabling Nagle (as above) at a specific stage in the dialogue. However it might be possible, notwithstanding the bug that Ralf mentioned, to introduce a sleep at the point that data are written (eg after socket(write)). Cheers Jeremy
▸
On Sun, 30 May 2021 at 10:19, Ralph M <user-00a5e44c48c0@xymon.invalid> wrote:
As far as the sleep() thing goes, you may be running into this, from the 'man 3 sleep' man page: BUGS sleep() may be implemented using SIGALRM; mixing calls to alarm(2) and sleep() is a bad idea. I don't know how you'd fix it. Hopefully this gives you a starting point. Ralph Mitchell On Sat, May 29, 2021 at 4:11 PM Shawn Heisey <user-5d0d01dba542@xymon.invalid> wrote:Over ten years ago, I asked on this list how I could get rid of the "incorrect pipelining message" logged by postfix every time the smtp, smtps, or submission tests are done. https://lists.xymon.com/oldarchive/2010/11/msg00207.html The fix that I claim worked (multiple send commands) was for a job I had at the time. Now I am have a personal mail server (that also runs xymon) and that fix I mentioned so long ago is not working. I tweaked the code to allow a "pause" action in protocols.cfg and after modifying protocols.cfg to utilize it, I have eliminated the "incorrect command pipelining" message in mail.log. Here's the patch: --- xymon-4.3.28/lib/netservices.c 2017-01-05 19:00:06.000000000 -0700 +++ pause-xymon-4.3.28/lib/netservices.c 2021-05-29 12:51:09.717461323 -0600 @@ -259,6 +259,10 @@ } } } + else if (strncmp(l, "pause ", 6) == 0) { + int pausetime = atoi(skipwhitespace(l+5)); + sleep(pausetime); + } } if (fd) stackfclose(fd); Here's the new definitions I created in protocols.cfg: [smtp] pause 2 send "ehlo xymonnet.localdomain\r\n" pause 2 send "mail\r\n" pause 2 send "quit\r\n" expect "220" options banner port 25 [smtps] pause 2 send "ehlo xymonnet.localdomain\r\n" pause 2 send "mail\r\n" pause 2 send "quit\r\n" expect "220" options ssl,banner # No default port-number assignment for smtps - nonstandard according to IANA [submission|msa] pause 2 send "ehlo xymonnet.localdomain\r\n" pause 2 send "mail\r\n" pause 2 send "quit\r\n" expect "220" options banner port 587 And this is now what I see in mail.log (testing smtps and submission: May 29 13:51:15 bilbo postfix/submission/smtpd[16324]: connect from bilbo.elyograg.org[172.31.8.104] May 29 13:51:15 bilbo postfix/submission/smtpd[16324]: disconnect from bilbo.elyograg.org[172.31.8.104] quit=1 commands=1 May 29 13:51:15 bilbo postfix/smtps/smtpd[16325]: connect from bilbo.elyograg.org[172.31.8.104] May 29 13:51:15 bilbo postfix/smtps/smtpd[16325]: disconnect from bilbo.elyograg.org[172.31.8.104] quit=1 commands=1 Interesting thing here is that it's not actually pausing. Which probably means that I implemented it incorrectly. My training on C is ancient and I'm very rusty. But even though it doesn't pause, the error is gone, simply because each smtp command is now sent in a separate packet, which appears to *sometimes* be enough "delay" for postfix to not complain about pipelining. What happens now is occasionally I will get a yellow status on smtps with the xymon UI saying "Service smtps on bilbo.elyograg.org is not OK : Unexpected service response". So I think what I will do before submitting a patch is implement a "null" action (which will do nothing) as well as a "pause" action, and get some help from the real C developers here for making "pause" behave as advertised. Can somebody point me to some instructions on properly creating and submitting a patch? Also, if I could get some info on any other files I need to modify (man pages, readme files, etc), I would really appreciate it. Thanks, Shawn
list Michael T Pins
▸
On Sat, May 29, 2021 at 4:11 PM Shawn Heisey <user-5d0d01dba542@xymon.invalid> wrote:
Over ten years ago, I asked on this list how I could get rid of the "incorrect pipelining message" logged by postfix every time the smtp, smtps, or submission tests are done.
https://lists.xymon.com/oldarchive/2010/11/msg00207.html
The fix that I claim worked (multiple send commands) was for a job I had at the time. Now I am have a personal mail server (that also runs xymon) and that fix I mentioned so long ago is not working.
I tweaked the code to allow a "pause" action in protocols.cfg and after modifying protocols.cfg to utilize it, I have eliminated the "incorrect command pipelining" message in mail.log. Here's the patch:
[ code elided ] A much simpler method that works for me, talking to both postfix and sendmail, is simply adding an expect to the test in protocols.cfg: $ diff protocols.cfg.bak protocols.cfg 70c70,72 < send "mail\r\nquit\r\n" ---
send "mail\r\n expect "501" quit\r\n"
list Michael T Pins
▸
I wrote:
On Sat, May 29, 2021 at 4:11 PM Shawn Heisey <user-5d0d01dba542@xymon.invalid> wrote:
Over ten years ago, I asked on this list how I could get rid of the "incorrect pipelining message" logged by postfix every time the smtp, smtps, or submission tests are done.
https://lists.xymon.com/oldarchive/2010/11/msg00207.html
The fix that I claim worked (multiple send commands) was for a job I had at the time. Now I am have a personal mail server (that also runs xymon) and that fix I mentioned so long ago is not working.
I tweaked the code to allow a "pause" action in protocols.cfg and after modifying protocols.cfg to utilize it, I have eliminated the "incorrect command pipelining" message in mail.log. Here's the patch:
[ code elided ]
A much simpler method that works for me, talking to both postfix and sendmail, is simply adding an expect to the test in protocols.cfg:
$ diff protocols.cfg.bak protocols.cfg 70c70,72 < send "mail\r\nquit\r\n" ---send "mail\r\n expect "501" quit\r\n"
Or, if you want to be more "correct": $ diff protocols.cfg.bak protocols.cfg 70c70,72 < send "mail\r\nquit\r\n" ---
send "ehlo xymonnet\r\n expect "501" quit\r\n"
76c78,80 < send "mail\r\nquit\r\n" ---
send "ehlo xymonnet\r\n expect "501" quit\r\n"