Subject: [Emailcore] rfc5321bis-05 preview and heads-up

From: John C Klensin <[email-masked]>
Date: Fri, 22 Oct 2021 19:26:47 -0400

Hi.

As people have probably guessed from notes I have posted to this
list within the last hour or two, I'm trying to wrap up
rfc5321bis-05 with the expectation of getting it done and posted
over the weekend.   Anyone having uncompleted assignments
(either self-imposed or WG-assigned) for work that should be
incorporated in that version should try to get it up RSN.   If
such messages appear, I will incorporate their content if time
permits and they seem uncontroversial but, even if I don't,
getting text posted now would probably allow useful discussion
before we meet at IETF 112.

best,
   john

---

From: Alessandro Vesely <[email-masked]>
Date: Sun, 24 Oct 2021 16:30:54 +0200

Hi,

On Sat 23/Oct/2021 01:26:47 +0200 John C Klensin wrote:
> 
> If such messages appear, I will incorporate their content if time permits
> and they seem uncontroversial but, even if I don't, getting text posted now
> would probably allow useful discussion before we meet at IETF 112.

May I re-re-propose text for ticket #19, comment 4?  It's for Section 4.1.4, 
replacing the paragraph that contains CREF13 with the following two-liner:

      An SMTP server MAY verify the validity of the domain name argument in
      the EHLO command.   See [A/S] for further discussion.

In fact, w.r.t. previous text proposals, verifying that the domain name 
argument resolves to the other side of the TCP connection is not the only type 
of verification that servers carry out.  With the advent of MPTCP it might 
change even further.

Best
Ale
--

---

From: John C Klensin <[email-masked]>
Date: Sun, 24 Oct 2021 11:01:10 -0400

--On Sunday, October 24, 2021 16:30 +0200 Alessandro Vesely
<[email-masked]> wrote:

>> If such messages appear, I will incorporate their content if
>> time permits and they seem uncontroversial but, even if I
>> don't, getting text posted now would probably allow useful
>> discussion before we meet at IETF 112.
> 
> 
> May I re-re-propose text for ticket #19, comment 4?  It's for
> Section 4.1.4, replacing the paragraph that contains CREF13
> with the following two-liner:
> 
> 

Mostly already done in -05 (which I'm getting ready to post).
There is a broader question about what to say about forward
referencing to the A/S and how to say it that may be better for
a discussion during the meeting than trying to sort it out in
email.   

I do wonder about the upper-case "MAY" in this case because I'm
not sure it communicates anything, but I'm not going to change
it unless the WG decides it is an issue.
 
> In fact, w.r.t. previous text proposals, verifying that the
> domain name argument resolves to the other side of the TCP
> connection is not the only type of verification that servers
> carry out.  With the advent of MPTCP it might change even
> further.

First, that is one of the reasons I question "MAY" (see above).
Second, please reread Sections 7.8 and 7.9.  They appear to me
to cover other types of verification and no one has proposed
opening them yet.  Finally, unlike 821, 5321 (and 5321bis so
far) are specifications for SMTP over TCP.  If you think we
should be considering MTCP (or any other transport-level
protocol), propose a charter change and consider what would be
needed to keep these documents on the path to Internet Standard.

best,
   john

---

From: Alessandro Vesely <[email-masked]>
Date: Sun, 24 Oct 2021 17:43:12 +0200

On Sun 24/Oct/2021 17:01:10 +0200 John C Klensin wrote:
> --On Sunday, October 24, 2021 16:30 +0200 Alessandro Vesely <[email-masked]> wrote:
>>> If such messages appear, I will incorporate their content [...]
>> 

Great!

> First, that is one of the reasons I question "MAY" (see above).

It is stated that way in RFC 5321.  If we want to further retract the rest of 
that paragraph, we could add:

    However, if the verification fails, the server MAY refuse to
    accept a message on that basis.

> needed to keep these documents on the path to Internet Standard.

While sticking to TCP transport, Multipath TCP is designed to be fully 
compatible with it.  If it will eventually make it to main stream kernel, all 
servers which install it will be actually running MPTCP, possibly unwittingly.

Best
Ale
--

---

From: Pete Resnick <[email-masked]>
Date: Sun, 24 Oct 2021 13:12:50 -0500

On 24 Oct 2021, at 10:01, John C Klensin wrote:

> I do wonder about the upper-case "MAY" in this case because I'm not 
> sure it communicates anything, but I'm not going to change it unless 
> the WG decides it is an issue.

I am reminded of the printed instructions I got from the doctor after 
being on anesthetic:

"DO NOT COOK USING THE STOVE; YOU MAY START A FIRE AND INJURE YOURSELF"

I was not, AFAICT, being given permission to start a fire or injure 
myself.

My take is that MAY in standards should be used to express permission to 
the actor, and more importantly to express a warning to someone 
interacting with the actor that this option might occur. Mostly, that's 
important when the actor is sending something in the protocol stream 
that the recipient is going to have to deal with. In this case, we are 
saying it is permissible for a server receiving the command to do (or 
not do) the verifications, so we are warning sender of the command that 
it might be verified (or not). (*shrug*) I guess that it's good for the 
sender of the command to know that they best put something reasonable in 
there, but that usually dictates a MUST or SHOULD on the sending side, 
not a MAY on the receiving side. But all that to say: "Whatever." It's 
generally superfluous.

pr

---

From: Dave Crocker <[email-masked]>
Date: Sun, 24 Oct 2021 11:19:11 -0700

On 10/24/2021 11:12 AM, Pete Resnick wrote:
> Mostly, that's important when the actor is sending something in the 
> protocol stream that the recipient is going to have to deal with. In 
> this case, we are saying it is permissible for a server receiving the 
> command to do (or not do) the verifications, so we are warning sender of 
> the command that it might be verified (or not).

The problem is that it's normative language that a) does not pertain to 
an actual protocol component, and b) does not specify something that can 
reasonably be controlled by the presence or absence of the MAY.

Servers can and do reject sessions for all sorts of reasons.  Giving 
them "permission" for this one case is in the noise, at best.  That is, 
it's meaningless.  It's the sort of text that looks intuitively 
reasonable but is not actually useful.

To the extent that the goal is to give notice about some of the reasons 
a server might choose to reject a message or a connection -- beyond the 
definition of response codes -- that makes more sense to have in a 
document for operational guidance.

d/
-- 
Dave Crocker
Brandenburg InternetWorking
bbiw.net

---

From: John C Klensin <[email-masked]>
Date: Sun, 24 Oct 2021 15:02:13 -0400

FWIW, I agree with both Dave and Pete, and considerations
similar to their different ones were the reason I identified the
issue.  Given RFC 8174, I could apply the principle of minimum
change and just drop that to "may" and we could then go about
our business.   Or I/we could figure out how to rewrite things
to tell SMTP clients that they need to be careful and orderly
about that information because some server might try to verify
it and behave in a way that would not be conducive to the
message getting through if the verification failed (an extension
on Pete's observation).

As editor, I have no opinion at the moment other than reluctance
to start messing with this unless the WG clearly wants to.  Too
late for -05 (and hence our meeting at IETF 112) in any event.

best,
  john

--On Sunday, October 24, 2021 11:19 -0700 Dave Crocker
<[email-masked]> wrote:

>> Mostly, that's important when the actor is sending something
>> in the  protocol stream that the recipient is going to have
>> to deal with. In  this case, we are saying it is permissible
>> for a server receiving the  command to do (or not do) the
>> verifications, so we are warning sender of  the command that
>> it might be verified (or not).
> 
> 
> The problem is that it's normative language that a) does not
> pertain to an actual protocol component, and b) does not
> specify something that can reasonably be controlled by the
> presence or absence of the MAY.
> 
> Servers can and do reject sessions for all sorts of reasons.
> Giving them "permission" for this one case is in the noise, at
> best.  That is, it's meaningless.  It's the sort of text that
> looks intuitively reasonable but is not actually useful.
> 
> To the extent that the goal is to give notice about some of
> the reasons a server might choose to reject a message or a
> connection -- beyond the definition of response codes -- that
> makes more sense to have in a document for operational
> guidance.
> 
> d/

---

From: Alessandro Vesely <[email-masked]>
Date: Mon, 25 Oct 2021 11:59:55 +0200

On Sun 24/Oct/2021 20:19:11 +0200 Dave Crocker wrote:
>> In this case, we are saying it is permissible for a server receiving the
>> the command that it might be verified (or not). >
> Servers can and do reject sessions for all sorts of reasons.  Giving them 
> "permission" for this one case is in the noise, at best.  That is, it's 
> meaningless.  It's the sort of text that looks intuitively reasonable but is 
> not actually useful.

The point is to stress that the domain name argument in the EHLO command is an 
essential identifier.  That stress is needed because, in the past, phrases like 
"the server MUST NOT refuse to accept a message on that basis" have been used 
to infer that one can write whatever it likes in the EHLO command.  Because of 
this reason, for example, DMARC discards the domain name given here.

Domain alignment becomes more and more important to sort out email traffic, and 
reliability of Received: chains depends on the domain name given in EHLO.  It 
can be verified according to SPF or iprev methods, defined in their own RFCs.

To verify that the domain name resolves to the actual IP address of the remote 
connection is an obvious method which, AFAIK, is not described in any standard 
document.  RFC1123 mentions that such test is possible, but postpones it to 
offline checking of Received: fields:

          DISCUSSION:
               Including both the source host and the IP source address
               in the Received: line may provide enough information for
               tracking illicit mail sources and eliminate a need to
               explicitly verify the HELO parameter.

Perhaps, RFC5321-bis could specify in the last paragraph of Section 2.3.5 that 
the resolved domain address must match the connecting IP.  Or perhaps the A/S 
could fully describe this method and assign an Authentication-Results: keyword 
to it.  For Section 4.1.4, IMHO, there is no reason why generic verifications 
of the domain name argument should be limited to this method only.

At any rate, since a server can reject an invalid EHLO, the next question is 
going to be what its consequences are.  I think we don't want to fail the EHLO 
command itself.  Should the spec say that rejections shall reveal themselves 
when/if transactions are attempted?

Best
Ale
--

---

From: Alexey Melnikov <[email-masked]>
Date: Mon, 25 Oct 2021 13:01:12 +0100

> On 24 Oct 2021, at 20:02, John C Klensin <[email-masked]> wrote:
> 
> our business.

I think lowercase may is the right thing here.

>   Or I/we could figure out how to rewrite things
> Emailcore mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/emailcore

---

From: Jeremy Harris <[email-masked]>
Date: Mon, 25 Oct 2021 16:05:46 +0100

On 25/10/2021 10:59, Alessandro Vesely wrote:
> At any rate, since a server can reject an invalid EHLO, the next question is going to be what its consequences are.  I think we don't want to fail the EHLO command itself.

I think you will find that this does indeed happen.
-- 
Cheers,
   Jeremy

---

From: Michael Peddemors <[email-masked]>
Date: Mon, 25 Oct 2021 08:21:30 -0700

On 2021-10-25 8:05 a.m., Jeremy Harris wrote:
>> At any rate, since a server can reject an invalid EHLO, the next 
>> question is going to be what its consequences are.  I think we don't 
>> want to fail the EHLO command itself.
> 

Yes, we fail at the EHLO stage under certain circumstances.
It's the obvious approach for an obvious threat. And of course, all the 
broken 'bots' out there.

However, remember the concept of 'invalid' EHLO has to be flexible 
enough to allow different MTA's and operators to have their own 
definition of 'invalid', and it should NOT be too explicit.

Eg, there are valid use cases where the EHLO does not match the public 
PTR, and instead represents an internal networking definition of the 
'server name'.

But we should be able to agree that at least the EHLO should be a FQDN 
for MTA->MTA traffic, and that we have to be much more flexible on 
MTU-MTA traffic, but it should never be undefined.. eg 127.0.0.1.  Every 
MTU should have a host name for instance.

But 'invalid' COULD simply be a regex definition that the operator 
chooses to use.  In theory, (not recommending this) an operator could 
consider any use of a specific TLD as 'invalid' to their operations.

But it should be defined as 'fixed', eg a MTU or MTA must NOT use a 
random value for EHLO.

(Just love that bot that keeps doing that, trying to pretend it's 
different MTU's behind a CGN, use of random EHLO is an automatic block 
in our world)

-- 
"Catch the Magic of Linux..."
------------------------------------------------------------------------
Michael Peddemors, President/CEO LinuxMagic Inc.
Visit us at http://www.linuxmagic.com @linuxmagic
A Wizard IT Company - For More Info http://www.wizard.ca
"LinuxMagic" a Registered TradeMark of Wizard Tower TechnoServices Ltd.
------------------------------------------------------------------------
604-682-0300 Beautiful British Columbia, Canada

This email and any electronic data contained are confidential and intended
solely for the use of the individual or entity to which they are addressed.
Please note that any views or opinions presented in this email are solely
those of the author and are not intended to represent those of the company.

---

From: John C Klensin <[email-masked]>
Date: Mon, 25 Oct 2021 11:40:38 -0400

Ale,

(personal opinion, no hat).

It seems to me that, whether your ideas are correct or
incorrect, you are headed down a new path.  First, remember that
imposing new requirements in 5321bis takes us very close to the
line at which we'd need to reset to Proposed Standard, rather
than going to full standard.  As I interpret the rules (again
personal opinion), such new requirements would be justified in a
document going to Internet Standard only if there were clear
proof that the absence of those requirements was causing
interoperability harm, not that they would make a different
protocol work better.  

FWIW, I think the WG's charter prohibits even getting into that
type of issue as far as 5321bis is concerned.  I'd appreciate it
if the WG Chairs would speak to that.

If one sets that concern aside and wants to charge ahead on the
subject, then this is, as others have pointed out, really a
requirement on what SMTP-senders can put into that field, not
something we should be trying to affect by vague comments about
what the server might do (much less playing around with how we
spell "MaY" or "maY" (presumably different degrees of
non-requirement :-( ).  However, that would probably push the
Internet Standard requirement (and the charter) even further.

So, two suggestions, with the understanding that, as a WG
participant (even more than as an editor), I want this work,
especially the 5321bis/5322bis parts, done and done at a higher
rate and with more active participation by more people than I'm
seeing, 

(1) Go silent on this subject until 5321bis is under control,
then push for text in the A/S.  I don't know whether that
document could reasonably impose new requirements either, but it
could certainly explain why better quality EHLO arguments would
benefit both client and server and strongly encourage them.

or, if you want to fix 5321bis...

(2) Generate an I-D, right now (ideally making the posting
deadline but otherwise begging Murray for permission for a late
submission), in Proposed Standard form that modifies RFC 5321 to
specify whatever you think needs to be specified, updating the
SPF/ DKIM/ etc. documents as needed to clarify the relationships
between the EHLO domain or address-literal and other fields.
Make sure it describes current practice rather than speculating.
Convince Murray (or Francesca) to AD-sponsor it or to add it to
the charter of this WG.  For the latter case, convince the WG to
put 5321bis on hold until your draft is discussed and processed.
If you can get it approved as PS and this WG continues at the
pace we seem to have adopted, we could easily be still fussing
with 5321bis when your six months runs out, you write an
implementation report, and then argue that the substance of the
I-D should be folded into 5321bis.

In either case -- and this may be affecting my personal
skepticism -- I'm concerned both about the number of
environments in which those who configure SMTP services may not
have nearly as much control over those responsible for the local
DMS zone as we might like, about clusters of machines that share
functions but not addresses, and about an assortment of edge
cases that might lead to more use of address literals if we
impose more rigid requirements on EHLO domains.  I think that
would be a step backwards; YMMD.

But, right now, in this forum and on this thread, I don't see
the discussion as leading anywhere useful wrt 5321bis.  Again,
just my personal opinion.

best,
   john

--On Monday, October 25, 2021 11:59 +0200 Alessandro Vesely
<[email-masked]> wrote:

>>> In this case, we are saying it is permissible for a server
>>> receiving the command to do (or not do) the verifications,
>>> so we are warning sender of the command that it might be
>>> verified (or not). >
>> Servers can and do reject sessions for all sorts of
>> reasons.  Giving them  "permission" for this one case is in
>> the noise, at best.  That is, it's  meaningless.  It's the
>> sort of text that looks intuitively reasonable but is  not
>> actually useful.
> 
> 
> The point is to stress that the domain name argument in the
> EHLO command is an essential identifier.  That stress is
> needed because, in the past, phrases like "the server MUST NOT
> refuse to accept a message on that basis" have been used to
> infer that one can write whatever it likes in the EHLO
> command.  Because of this reason, for example, DMARC discards
> the domain name given here.
> 
> Domain alignment becomes more and more important to sort out
> email traffic, and reliability of Received: chains depends on
> the domain name given in EHLO.  It can be verified according
> to SPF or iprev methods, defined in their own RFCs.
> 
> To verify that the domain name resolves to the actual IP
> address of the remote connection is an obvious method which,
> AFAIK, is not described in any standard document.  RFC1123
> mentions that such test is possible, but postpones it to
> Perhaps, RFC5321-bis could specify in the last paragraph of
> Section 2.3.5 that the resolved domain address must match the
> connecting IP.  Or perhaps the A/S could fully describe this
> method and assign an Authentication-Results: keyword to it.
> For Section 4.1.4, IMHO, there is no reason why generic
> verifications of the domain name argument should be limited to
> this method only.
> 
> At any rate, since a server can reject an invalid EHLO, the
> next question is going to be what its consequences are.  I
> think we don't want to fail the EHLO command itself.  Should
> the spec say that rejections shall reveal themselves when/if
> transactions are attempted?
> 
> 
> Best
> Ale
> --

---

From: Alessandro Vesely <[email-masked]>
Date: Mon, 25 Oct 2021 19:58:37 +0200

John,

far from me the idea to explore new paths, let alone new requirements.

Let me recall that there was a long discussion about checking the EHLO argument 
on the [ietf-smtp] mailing list, starting 18 September 2020.  Its conclusion 
was that a new edition of SMTP should have relaxed the requirement level of 
servers reaction to bad EHLO arguments from MUST NOT to SHOULD NOT.  When 
[emailcore] started, I summarized that thread here:
https://mailarchive.ietf.org/arch/msg/emailcore/rX4A8_mILuijOt3RzeSMXAq5op0/

I refer that message in order to clarify that the issue is not extraneous to 
the original path of 5321bis.  It's been here since the start.

Subsequently, the WG decided that rather than relaxing the requirement, the 
bulk of the paragraph had better be moved to a newly conceived A/S.  So far, so 
good.  However, that way the paragraph sounds much more permissive than the 
originally proposed SHOULD NOT.  Its -05 version implies that the EHLO argument 
can actually be invalid.  The derailment point is there.  Indeed, lacking 
further specification, a naive reader would derive that an invalid argument 
deserves a 5xx response to EHLO.  Don't you agree?

IOW, while SHOULD NOT would have left it to the infringer to devise an 
interoperable behavior, letting servers free to choose their behavior in the 
face of a bad EHLO argument can be problematic.  An invalid argument can be the 
result of erroneous configuration or temporary hiccups, besides malicious 
operations.  Shouldn't a well specified standard say how to handle such cases?

IMHO, the spec could say something around the line of accepting EHLO or HELO 
whatever the argument and whatever the validation method, but then reject any 
transaction if the validation failed.  I won't attempt to word it any better, 
let alone write an I-D.

Please just ignore this thread if it looks useless.  It looks relevant to me, 
but I may be wrong.

Best
Ale

On Mon 25/Oct/2021 17:40:38 +0200 John C Klensin wrote:
> --On Monday, October 25, 2021 11:59 +0200 Alessandro Vesely <[email-masked]> wrote:
>>>> verified (or not).