Possible bug in protocol tests
list Shawn Heisey
I had a thread on the mailing list about this previously. In the recent past, I had to rebuild my xymon server and now I'm ready to declare that I think the behavior I'm seeing is a bug. http://lists.xymon.com/oldarchive/2010/11/msg00189.html The original problem is that the default smtp settings which come with xymon in protocols.cfg produce a warning/error in the postfix log about improper command pipelining, so I am adjusting the xymon config to keep that from happening. Here's a transcript of a session with my mail server: root at nexus2:~# telnet localhost 25 Trying ::1... Connected to localhost. Escape character is '^]'. 220 nexus2.REDACTED.com ESMTP Postfix (Ubuntu) mail 503 5.5.1 Error: send HELO/EHLO first quit 221 2.0.0 Bye Connection closed by foreign host. The following config duplicates that transcript, but it generates a yellow status on a 4.3.14 server: [smtp] expect "220" send "mail\r\n" expect "503" send "quit\r\n" expect "221" options banner port 25 With the following config, I get a green status, but in fact I should get red, because "220" is *not* in the response to the quit command: [smtp] send "mail\r\n" expect "503" send "quit\r\n" expect "220" options banner port 25 Here's the bug, which might be two separate bugs: The first group of settings above should produce a green status, the second group of settings should produce a red status. In protocols.cfg, "expect" does not work as expected. I've got no idea why the first case comes out as yellow. For the second case, the 'expect "220"' line is passing because of the 220 in the initial banner from the server. That line should fail. Thanks, Shawn
list Mark Felder
▸
On Tue, Jul 28, 2015, at 09:26, Shawn Heisey wrote:
I had a thread on the mailing list about this previously. In the recent past, I had to rebuild my xymon server and now I'm ready to declare that I think the behavior I'm seeing is a bug. http://lists.xymon.com/oldarchive/2010/11/msg00189.html The original problem is that the default smtp settings which come with xymon in protocols.cfg produce a warning/error in the postfix log about improper command pipelining, so I am adjusting the xymon config to keep that from happening. Here's a transcript of a session with my mail server: root at nexus2:~# telnet localhost 25 Trying ::1... Connected to localhost. Escape character is '^]'. 220 nexus2.REDACTED.com ESMTP Postfix (Ubuntu) mail 503 5.5.1 Error: send HELO/EHLO first quit 221 2.0.0 Bye Connection closed by foreign host. The following config duplicates that transcript, but it generates a yellow status on a 4.3.14 server: [smtp] expect "220" send "mail\r\n" expect "503" send "quit\r\n" expect "221" options banner port 25 With the following config, I get a green status, but in fact I should get red, because "220" is *not* in the response to the quit command: [smtp] send "mail\r\n" expect "503" send "quit\r\n" expect "220" options banner port 25 Here's the bug, which might be two separate bugs: The first group of settings above should produce a green status, the second group of settings should produce a red status. In protocols.cfg, "expect" does not work as expected. I've got no idea why the first case comes out as yellow. For the second case, the 'expect "220"' line is passing because of the 220 in the initial banner from the server. That line should fail.
You're right. I would expect that "expect" would only match once per entry. If you needed an "expect 220" at the beginning and another in response to your "quit", you should have to declare it twice. It seems the way expect is processing the data doesn't match reasonable assumptions.
list Japheth Cleaver
▸
On Wed, July 29, 2015 10:16 am, Mark Felder wrote:
On Tue, Jul 28, 2015, at 09:26, Shawn Heisey wrote:I had a thread on the mailing list about this previously. In the recent past, I had to rebuild my xymon server and now I'm ready to declare that I think the behavior I'm seeing is a bug. http://lists.xymon.com/oldarchive/2010/11/msg00189.html The original problem is that the default smtp settings which come with xymon in protocols.cfg produce a warning/error in the postfix log about improper command pipelining, so I am adjusting the xymon config to keep that from happening. Here's a transcript of a session with my mail server: root at nexus2:~# telnet localhost 25 Trying ::1... Connected to localhost. Escape character is '^]'. 220 nexus2.REDACTED.com ESMTP Postfix (Ubuntu) mail 503 5.5.1 Error: send HELO/EHLO first quit 221 2.0.0 Bye Connection closed by foreign host. The following config duplicates that transcript, but it generates a yellow status on a 4.3.14 server: [smtp] expect "220" send "mail\r\n" expect "503" send "quit\r\n" expect "221" options banner port 25 With the following config, I get a green status, but in fact I should get red, because "220" is *not* in the response to the quit command: [smtp] send "mail\r\n" expect "503" send "quit\r\n" expect "220" options banner port 25 Here's the bug, which might be two separate bugs: The first group of settings above should produce a green status, the second group of settings should produce a red status. In protocols.cfg, "expect" does not work as expected. I've got no idea why the first case comes out as yellow. For the second case, the 'expect "220"' line is passing because of the 220 in the initial banner from the server. That line should fail.You're right. I would expect that "expect" would only match once per entry. If you needed an "expect 220" at the beginning and another in response to your "quit", you should have to declare it twice. It seems the way expect is processing the data doesn't match reasonable assumptions.
As near as I can tell, this is the way things had always worked. It's not
really a full expect send-response syntax. Although 'send' handling is
performed while the connection is open (obviously), the 'expect'
evaluation seems to only be present at the end when decide_color is done.
The re-architecting of this, including having a full send-expect dialog is
something that Henrik had been working on in xymonnet2 (trunk).
To be honest, I'm a little curious how the multiple "send" lines are
actually being sent. I can see them being inserted into a list, but I
don't see anywhere that they're being rotated through line by line.
I'll see if there's any shortcut that can be performed here, but I don't
believe this will be that easy to fix with the current architecture. (We
would also probably need for enforce an alternating
send/expect/send/expect syntax in the config as a result.)
Regards,
-jc
list Shawn Heisey
▸
On 7/29/2015 12:33 PM, J.C. Cleaver wrote:
I'll see if there's any shortcut that can be performed here, but I don't believe this will be that easy to fix with the current architecture. (We would also probably need for enforce an alternating send/expect/send/expect syntax in the config as a result.)
Based on the behavior, I figured the engine was pretty simple and a fix probably would involve some significant work, possibly a complete redesign. SMTP is a protocol that is specifically designed to NOT work with the brute force approach, and I think that the current protocol engine can't do much beyond brute force. If Henrik is already working on a redesigned network protocol engine, then there's not much more to be said, except to wish him the best of luck with it. Thanks, Shawn
list Phil Crooker
Just out of curiosity, I tried this to eliminate the 503 error:
[smtp2]
send "helo hostname\r\nquit\r\n"
expect "250"
options banner
port 25
Which produces:
220 server.domain.com ESMTP Postfix
250 server.domain.com
221 2.0.0 Bye
So, this fixes the 503 error but the test fails to yellow as xymonnet appears only to respond to the initial 220 response. OK, so if I remove the banner option I get this, just the single line:
220 server.domain.com ESMTP Postfix
And the 'expect "250"' still fails (to reset the web display I had to set expect to 220 to get green, then back to 250). So it appears all the responses other than the initial connection response are considered "banner" and not part of the evaluation.
▸
From: Xymon <xymon-bounces at xymon.com> on behalf of Shawn Heisey <user-5d0d01dba542@xymon.invalid>
Sent: Thursday, 30 July 2015 8:04 AM
To: xymon at xymon.com
Subject: Re: [Xymon] Possible bug in protocol tests
On 7/29/2015 12:33 PM, J.C. Cleaver wrote:I'll see if there's any shortcut that can be performed here, but I don't believe this will be that easy to fix with the current architecture. (We would also probably need for enforce an alternating send/expect/send/expect syntax in the config as a result.)
Based on the behavior, I figured the engine was pretty simple and a fix probably would involve some significant work, possibly a complete redesign. SMTP is a protocol that is specifically designed to NOT work with the brute force approach, and I think that the current protocol engine can't do much beyond brute force. If Henrik is already working on a redesigned network protocol engine, then there's not much more to be said, except to wish him the best of luck with it. Thanks, Shawn