Subject: Re: [Emailcore] Ticket #13: G.7.7. Does the 'first digit only' and/or non-listed reply code text need clarification?

From: Viktor Dukhovni <[email-masked]>
Date: Mon, 22 Mar 2021 15:19:46 -0400

On Tue, Jan 19, 2021 at 07:31:50AM -0800, Dave Crocker wrote:

> Given the normative term in the second sentence it seems odd -- and
> possibly problematic -- to have no normative term in the first sentence.
> 
> I assume the intention here is:
> 
>      Sender-SMTP MUST first test the whole 3 digit reply code it
> receives, as well as any accompanying supplemental codes or information
> (see RFC 3463 and RFC 5248). If the full reply code or additional
> information is not recognized, Sender-SMTP MUST use the first digit
> (severity indication) of a reply code it receives

I'm rather late to the party, this was posted Jan 19th, but perhaps
better late than never...  To whit, I have no idea what the above
mandate to 'test' the whole 3 digit reply actually means.

If some hypothetical MTA divides the whole 3 digits by 100, and then
bases its actions on the quotient, is that conformant?

    https://github.com/vdukhovni/postfix/blob/master/postfix/src/smtp/smtp_proto.c#L1980

    if (resp->code / 100 != 2) {
        smtp_mesg_fail(state, STR(iter->host), resp,
                       "host %s said: %s (in reply to %s)",
                       session->namaddr,
                       translit(resp->str, "\n", " "),
                       xfer_request[SMTP_STATE_MAIL]);
        mail_from_rejected = 1;
    }

What exactly is such testing expected to entail?  With negative replies
the only effect of the extra digits on the above MTA is that 50x syntax
errors are used to pessimistically guess that the protocol state
machines may be out of sync between client and server, and so the
connection should not be reused for additional messages:

    https://github.com/vdukhovni/postfix/blob/master/postfix/src/smtp/smtp_trouble.c#L189-L208

-- 
    Viktor.

---

From: Ned Freed <[email-masked]>
Date: Wed, 24 Mar 2021 13:21:06 -0700 (PDT)

> mandate to 'test' the whole 3 digit reply actually means.

I don't know what it's supposed to mean, but I do know that there are
implementations that take this to heart and then get things badly wrong: At
least one widely used code base fails with an error if the code isn't one that's
listed in RFC5321.

>     https://github.com/vdukhovni/postfix/blob/master/postfix/src/smtp/smtp_trouble.c#L189-L208

I think the intent is to check for things like forwarding addresses and
act on them if possible. But even if you agree that this sort of thing
is worth supporting - and I don't - the fact remains that the emphasis
needs to be on first digit processing, not on the rest of the line.

The alternate text I would like to see is something like:

  Sender-SMTP MUST first test the first digit (severity indication)
  of the reply code it receives, it MUST act in a fashion
  consistent with that code, and it MUST be able to handle all
  syntactically valid reply codes. Sender-SMTP MAY examine and use any
  any accompanying supplemental codes or information to refine any
  actions it performs (see RFC 3463 and  RFC 5248).

Regardless of what text we use, we need to get away from the notion that clients
absolutely have to do sophisticated processing of the entire line, while
recognizing that some clients do need and will perform such processing.

				Ned

---

From: Viktor Dukhovni <[email-masked]>
Date: Wed, 24 Mar 2021 17:28:17 -0400

> 
> 
> 
> 
> 
> On Mar 24, 2021, at 4:21 PM, Ned Freed <[email-masked]> wrote:
> 

+1.  I think this is actually clear and actionable.

> recognizing that some clients do need and will perform such processing.

That's fine.  They should however know to be prepared to find that
the digits after the first are often ad-hoc approximations or even
just wildly off-base from the RFC and/or the actual reason why the
server is responding with that [245]XX code.

-- 
	Viktor.

---

From: John C Klensin <[email-masked]>
Date: Thu, 25 Mar 2021 00:27:21 -0400

(As individual)

--On Wednesday, March 24, 2021 13:21 -0700 Ned Freed
<[email-masked]> wrote:

>...
> I think the intent is to check for things like forwarding
> addresses and act on them if possible. But even if you agree
> that this sort of thing is worth supporting - and I don't -
> the fact remains that the emphasis needs to be on first digit
> processing, not on the rest of the line.
> 
>   Sender-SMTP MUST first test the first digit (severity
> indication)   of the reply code it receives, it MUST act in a
> fashion   consistent with that code, and it MUST be able to
> handle all   syntactically valid reply codes. Sender-SMTP MAY
> examine and use any   any accompanying supplemental codes or
> information to refine any   actions it performs (see RFC 3463
> and  RFC 5248).

I rather like this.

> Regardless of what text we use, we need to get away from the
> notion that clients absolutely have to do sophisticated
> processing of the entire line, while recognizing that some
> clients do need and will perform such processing.

Agreed.  

    john

---

From: Laura Atkins <[email-masked]>
Date: Thu, 25 Mar 2021 10:53:54 +0000

> On 24 Mar 2021, at 21:28, Viktor Dukhovni <[email-masked]> wrote:
> 
>> 
>> 
>> 
>> 
>> 

I like this language, too. 

> server is responding with that [245]XX code.

Yup. eg: 452 this message will never succeed. 

laura 

-- 
Having an Email Crisis?  We can help! 800 823-9674 

Laura Atkins
Word to the Wise
[email-masked]
(650) 437-0741		

Email Delivery Blog: https://wordtothewise.com/blog

---

From: Alessandro Vesely <[email-masked]>
Date: Thu, 25 Mar 2021 13:19:36 +0100

On Thu 25/Mar/2021 05:27:21 +0100 John C Klensin (As individual) wrote:
> --On Wednesday, March 24, 2021 13:21 -0700 Ned Freed wrote:
> 
>> 

strncmp(response, "421", 3)==0 has to come before *response=='4'.

According to previous threads, *response=='4' allows a client to try RCPT 
commands for another 100-N recipients.  strncmp(response, "421", 3)==0 suggests 
to quit the transaction sooner.

BTW, Section 3.8 allows 421 at any time a server must forcibly shut down. 
However, Section 4.3.2 only contemplates:

       RCPT

          S: 250, 251 (but see Section 3.4 for discussion of 251 and 551)
          E: 550, 551, 552, 553, 450, 451, 452, 503, 455, 555

Best
Ale
--

---

From: Ned Freed <[email-masked]>
Date: Thu, 25 Mar 2021 08:09:58 -0700 (PDT)

> strncmp(response, "421", 3)==0 has to come before *response=='4'.

In fact it's exactly the opposite: The client needs the ability to treat 421 as
a generic temporary error. Without that ability pipelining is effectively
forbidden by the standard, because now you are saying that you have to check
every command response as it arrives, so you can tell whether or not it's OK to
proceed.

And in the case of 421, this is completely unnecessary. If a 421 does arrive,
and if, as the standard requires, the connection is then closed, it doesn't
matter whether the client has sent a gazillion more RCPT TOs, DATA, BDAT,
another 10 transactions, whatever. All of that data is discarded, in effect with
additional 421 codes.

At this point - and only at this point - does it make sense for the client to
make decisions about what it's going to do on subsequent attempts to transfer
the message. I for one don't believe that you can safely conclude that the
reason for a 421 response to a RCPT TO is that you sent too many recipients, but
if a client wants to implement that strategy, that's its choice.

Of course it's possible that the 421 isn't followed by a disconnect. In
this case the server is operating outside what the standard specifies.
In this case the meaning of 421 is effectively unknown, and it's even
less clear what the client should do - other than treat it as a temporary
error.

451 is actually the more interesting case - it's the code that should be
used when a recipient limit has been hit. But once again the freedom to treat it
as a generic temporary error during the current session is needed in order to
allow pipelining. And since 452 does have other meansing, it's not at all
clear to me that the right thing for a client to do is assume subsequent
RCPT TOs will be rejected with the same error. Why not send them and
find out?

But if you want to make that assumption, that's also fine. Nothing in the
suggested text is inconsistent with using a 451 as an indication to stop sending
more RCPT TOs.

> commands for another 100-N recipients.

Absolutely, and that has to include 421 and 451 response. See above.

>  strncmp(response, "421", 3)==0 suggests

I disagree that's what 421 suggests. 451 might suggest that, but it most
certainly doesn't require it. What is required is that 4YZ be treated as a
temporary error. Nothing more, nothing less.

The nasty case is when you start getting 4YZ errors in response to RCPT TOs and
then get a 4YZ error in response to DATA/BDAT. When this happens it makes some
sense for the client to conclude that a recipient limit was hit and to limit the
number of recipients it tries next time to the number of RCPT TOs that were
accepted prior to the first 4YZ.

But this is a case of a client dealing with bad server behavior, and for that
reason it is one where if I implemented this I would base the choice of how to
proceed on the first digit only.

>           E: 550, 551, 552, 553, 450, 451, 452, 503, 455, 555

421 isn't listed as a repsonse to anything in that section. There's no
point to doing so since it can be sent at any time and therefore valid
in any state where a command response is expected.

					Ned

---

From: Alessandro Vesely <[email-masked]>
Date: Thu, 25 Mar 2021 18:36:33 +0100

On Thu 25/Mar/2021 16:09:58 +0100 Ned Freed wrote:
> On Thu 25/Mar/2021 13:19:36 +0100 Alessandro Vesely wrote:

Pipelining may have its own reasons, rules, and methods.  In fact, it allows 
clients to send commands before looking at the responses.  Whether a client 
doesn't look at the first byte of the response, rather than not looking at the 
whole code, sounds quite irrelevant.

> another 10 transactions, whatever. All of that data is discarded, in effect with

The server can tear down the connection before the client acknowledges a 421. 
Isn't it still preferable to issue a QUIT?

Perhaps, a non-pipelining client can:

    if (*response=='4')
    {
       // react to special codes
       if (strncmp(response, "421", 3)==0)
          do_quit();

       // react to generic temporary errors
       // ...
    }

> if a client wants to implement that strategy, that's its choice.

A client can also send unrecognizable commands.  I'd call it a TCP client, but 
hardly an SMTP client.

> less clear what the client should do - other than treat it as a temporary

A dialog between the deaf and the dumb?

> Absolutely, and that has to include 421 and 451 response. See above.

What is doing a non-pipelining client that continues to send RCPT commands 
after getting a 421?  Is it testing the server behavior?  Is it deaf?

>>  strncmp(response, "421", 3)==0 suggests to quit the transaction sooner.
> 
> I disagree that's what 421 suggests.

Do you?

> 451 might suggest that, but it most certainly doesn't require it.

Surely you must be joking.  451 tells me the last RCPT I sent must be retried 
later.  If there are more recipients, I should still try each one of them. 
Replying QUIT because of a 451 is nonsensical, especially if some previous 
recipients were accepted.

> What is required is that 4YZ be treated as a temporary error. Nothing more,
> nothing less.

For requirements, maybe.  (Although the current wording still says to consider 
the whole code first.)  Yet, a client MAY be smarter than minimal requirements.

> proceed on the first digit only.

That depends on the YZ.  If the last code was a 421 due to an unanticipated 
event, or a 450 due to policy reasons, such as waiting for a verdict about 
something found in the message content, inferring a recipient limit is 
excessively smart.

> 421 isn't listed as a response to anything in that section. There's no

Ack, thank you.

Best
Ale
--

---

From: Ned Freed <[email-masked]>
Date: Thu, 25 Mar 2021 12:16:14 -0700 (PDT)

I'm afraid none of this makes the slightest bit of sense to me, so I'm 
done trying to respond to the overall argument.

> Isn't it still preferable to issue a QUIT?

The answer is it's irrelevant what the client does at this point - the service
is supposed to be closing the connection, so whatever the client sends will be
dropped on the floor, regardless of whether it's a QUIT or some pipelined
command.

And if the server doesn't do that, I will reiterate the point others have
repeatedly made: When a server misbehaves in a rather obvious way, trusting that
the reply codes it sends are sensible when it's other behavior is not makes very
little sense.

				Ned