Subject: [apps-discuss] Pete Resnick's Yes on draft-ietf-appsawg-malformed-mail-10: (with COMMENT)

From: "Pete Resnick" <[email-masked]>
Date: Wed, 20 Nov 2013 14:35:33 -0800

Pete Resnick has entered the following ballot position for
draft-ietf-appsawg-malformed-mail-10: Yes

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)

Please refer to http://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.

The document, along with other ballot positions, can be found here:
http://datatracker.ietf.org/doc/draft-ietf-appsawg-malformed-mail/

----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

Nothing that would stop me from endorsing this document going forward,
but please do take the following into consideration:

1.1 - The 5th paragraph seems redundant with previous paragraphs in this
section. The last paragraph seems redundant with section 1.2. Suggest
striking.

4 - It seems worth pointing out somewhere in this section that the
prepending of Received fields is the safest thing to do if changes must
be made to the message to pass information between modules.

7.1 - "A message using an obsolete header syntax" You might consider
adding a direct reference to 5322 section 4 to define what's meant by
"obsolete".

7.1.6 - Why is the second example not obviously better? I have a hard
time imagining circumstances where an unterminated quoted-string that
contains an angle-bracketed thing that looks like an addr-spec is in fact
a local part.

7.4 - "acceptance grammar" is a weird construction, not used in 5322.
Suggest "obsolete syntax" (with the reference to section 4) instead.

7.5 - Third paragraph: Reference to DKIM would be useful.
Fourth paragraph: I find the word "enacted" a bit weird. I suggest
changing "can be enacted" to "can be used" or "strategies can be used"
What's the difference between 3 & 4? Or maybe I don't know what "compound
instance" means in 3.

7.5.3 - What's the harm in more than one Return-Path? Only one of
interest is the top-most.

---

Finally, a gedankenexperiment, or maybe fodder for a real experiment:
What would happen if, upon receiving a malformed message that was
determined to not be otherwise malicious, a receiving SMTP system both
returned a 5xx to the message *and* processed and delivered the message
(i.e., give the receiver what they want, but push back on folks who
generate crap)? Would it help? (I am not asking for a discussion of this
in the document. Just an interesting thought.)

---

From: "John Levine" <[email-masked]>
Date: 21 Nov 2013 02:28:57 -0000

>generate crap)? Would it help?

I've been doing reject and process on my spamtraps for years.  It
doesn't seem to have broken anything, but it also doesn't seem that
anyone's taken the hint.

R's,
John

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Thu, 21 Nov 2013 06:18:41 -0800

Hi Pete,

> but please do take the following into consideration:
[...]

Thanks for the suggestions.  I'll take a run at these after the telechat
today.

> in the document. Just an interesting thought.)
If there's information available about this that can get WG consensus (and
one person has already chimed in on apps-discuss), we can add a sentence or
two about it as well.

-MSK

---

From: Ned Freed <[email-masked]>
Date: Thu, 21 Nov 2013 18:52:48 -0800 (PST)

> doesn't seem to have broken anything, but it also doesn't seem that

I'm not sure we want to open this particular door in a standardized
way. The potential for user astonishment here is very large...

				Ned

---

From: John R Levine <[email-masked]>
Date: Thu, 21 Nov 2013 22:09:56 -0500 (EST)

> way. The potential for user astonishment here is very large...

The particular reason I do it on the spamtraps is to deter the "nobody 
ever complained so you must have opted in" crowd.  I agree that it's 
probably not a great idea for anything that might be considered real mail.

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
Please consider the environment before reading this e-mail.

---

From: Ned Freed <[email-masked]>
Date: Fri, 22 Nov 2013 07:13:51 -0800 (PST)

> ever complained so you must have opted in" crowd.

Makes sense.

Another case where this comes up is certain types of compliance archiving.

> I agree that it's

				Ned

---

From: Dave Crocker <[email-masked]>
Date: Fri, 22 Nov 2013 07:27:24 -0800

>> probably not a great idea for anything that might be considered real

Would there be any benefit in a short BCP about this, including when to 
use it and when not to?

d/

-- 
Dave Crocker
Brandenburg InternetWorking
bbiw.net

---

From: John R Levine <[email-masked]>
Date: Fri, 22 Nov 2013 11:12:51 -0500 (EST)

> Would there be any benefit in a short BCP about this, including when to use 
> it and when not to?

What I'm doing is pretty exotic.  If we can't find some other people doing 
reject and process, the best I can offer is an HSG* rather than BCP.

R's,
John

* - Highly Speculative Guess

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Fri, 22 Nov 2013 11:39:04 -0800

All of this works for me except:

> interest is the top-most.

The issue here is that some components pick the last one, and some pick the
first, in general.  More often this happen with From, but Return-Path is
not special in this regard.

-MSK

---

From: Ned Freed <[email-masked]>
Date: Fri, 22 Nov 2013 17:37:50 -0800 (PST)

> reject and process, the best I can offer is an HSG* rather than BCP.

And in regards to compliance archiving, a BCP is probably a good idea, but I'm
not the one to write it. You need someone with substantial familiarity with
both the legal landscape and actual deployment experience.

I also note that now may not be the best time for such a document.

				Ned

---

From: Ned Freed <[email-masked]>
Date: Sat, 23 Nov 2013 09:04:20 -0800 (PST)

> not special in this regard.

FWIW, one thing an MDA can do is check and see if the top Return-path: agrees
with the current MAIL FROM, and if it does don't add a redundant Return-path:
field.

				Ned

---

From: Pete Resnick <[email-masked]>
Date: Tue, 26 Nov 2013 11:44:14 -0600

I know Barry sent through his approval, but I wanted to send you my 
comments just in case there is room to change during AUTH48. (I was sick 
with a cold over the weekend; sorry for the delay.)

On 11/23/13 11:04 AM, Ned Freed wrote:

> Murray wrote:
>> On Wed, Nov 20, 2013 at 2:35 PM, Pete Resnick<[email-masked]>wrote:
>>

Your new text says:

    Where a change to content between modules is unavoidable, adding
    trace data (such as prepending a standard Received field) will at
    least allow tracing of the handling by modules that actually see
    different input.

I think this is misplaced, and I think you missed the point. Your 
paragraph seems to say you *want* to insert trace data, and that 
prepending Received fields is one way to do it. I think the main point 
of the section is generally correct: You *don't* want to be adding trace 
data. What I meant for you to add was that, if you do need to add trace 
data, then the *only* recommended way to do it is to prepend Received 
fields. I would change the paragraph to:

    Where a change to content between modules is unavoidable, for example
    to add trace data, the safest way to do so is to prepend Received
    fields with the appropriate information.

And I'd move it right after paragraph 2.

>>> contains an angle-bracketed thing that looks like an addr-spec is in fact

Your change here doesn't address my comment. You still say, "leaves 
software with no obvious "good" interpretation". I disagree. I think the 
second *is* an obvious good interpretation.

>>> 7.5 -
>>> What's the difference between 3&  4? Or maybe I don't know what "compound
>>> instance" means in 3.

You never did answer that question.

> with the current MAIL FROM, and if it does don't add a redundant Return-path:

Yep. And I worry about the fact that 7.5.3 pretty directly contradicts 
[MAIL] on this point without mentioning it. 5322 clearly allows multiple 
Return-Path fields, so long as they're block prepended with Received 
fields. Maybe that's now considered a problem, but it seems like you'd 
want to mention that this is a change. And I'm still not clear on the 
harm given that Return-Path is supposed to be trace and ignored by end 
systems (unlike From, which is clearly going to be used).

pr

-- 
Pete Resnick<http://www.qualcomm.com/~presnick/>
Qualcomm Technologies, Inc. - +1 (858)651-4478

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Tue, 26 Nov 2013 10:23:42 -0800

> with a cold over the weekend; sorry for the delay.)

No objection to working on this during AUTH48.

>>> On Wed, Nov 20, 2013 at 2:35 PM, Pete Resnick<[email-masked]
>>> >wrote:
>>>
> paragraph seems to say you *want* to insert trace data, and that prepending
> Received fields is one way to do it. I think the main point of the section
> is generally correct: You *don't* want to be adding trace data. What I
> meant for you to add was that, if you do need to add trace data, then the
> *only* recommended way to do it is to prepend Received fields. I would
> change the paragraph to:
>

OK by me.

> second *is* an obvious good interpretation.

Will revisit.  I think I agreed (it's not in front of me now), but I guess
I didn't capture it properly.

> You never did answer that question.

I thought I clarified that in the latest draft by explaining what was meant
by "compound instance".

> fields. Maybe that's now considered a problem, but it seems like you'd want
> to mention that this is a change. And I'm still not clear on the harm given
> that Return-Path is supposed to be trace and ignored by end systems (unlike
> From, which is clearly going to be used).
>
>
My recollection of Return-Path is that there's always zero or one, and it's
typically added at the top only during local delivery; any instance of it
in transit is basically wrong and subject to removal.  I'll have to revisit
this as well if that's wrong.

Thanks for persisting on these points.

-MSK

---

From: Ned Freed <[email-masked]>
Date: Tue, 26 Nov 2013 22:04:27 -0800 (PST)

> this as well if that's wrong.

Apologies, I should have caught this sooner. It's zero or one per block of
Received/Resent-*/Return-path fields. So yes, you can have more than one of
them. And that means section 7.5.3 is incorrect when it asserts that a message
with more than one is invalid in the first sentence. Perhaps a change to
something like this is in order:

   While legitimate messages can contain more than one Return-Path
   header field, such usage is often an error rather that a valid
   message containing multiple header field blocks as described in
   sections 3.6 of [RFC 5322]. Accordingly, when a message containing
   multiple Return-path: header fields is encountered, all but the
   topmost one is to be disregarded, as it is most likely to have
   been added nearest to the mailbox that received that message.

I think that should cover it.

That said, I've always regarded the notion that such blocks can be reliably
distinguished as nothing but specious nonsense, and that fact that RFC 5322
doesn't contain language recommending against such usage as a problematic
aspect of that specification.

The problem is fundamental: Aside from the dependency of Return-path: fields on
Received: fields (but not vice versa), all of the fields in one of these blocks
are optional. And all the rules only say that fielde must be prepended; they
say nothing about which of the various fields in a given block have to be
prepended first. It is therefore perfectly valid in a one step MSA/MDA situation
to prepend Return-path: first, then Received:, then Resent-*. Or any other
order, and that plus the optionality of all the field creates a situation where
there is no way to determine the boundaries between blocks with any degree of
relability. 

It follows that if you expect the recipient of your message to be able to
consume such information reliably it's a really good idea to remove all
instances of these fields other than your own on submission.

But all this goes far beyond the scope of the present draft, which is 
concerned with malformed mail, and as Pete correctly points out, messages
containing multiple Return-path: fields may be ill-advised but are not
necessarily malformed. Although in practice a lot of them are, because another
thing that happens frequently is that when messages are resent all Received:
fields are removed (because if you don't your mail tends to get caught by loop
checks) but the Return-path: fields are retained. And that's then a malformed
message according to the rules.

				Ned

P.S. I claim no special insight into any of this. Marshall Rose patiently
explained this issue to me many years ago.

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Tue, 26 Nov 2013 23:22:15 -0800

>> And I'd move it right after paragraph 2.
Understood, but I think yours implies that one would change content
specifically to add trace data, and not for some other reason.  What I'm
talking about is the capabilities of the modules to hand data among
themselves without having to modify the content itself; if for some reason
that's simply not possible, then at least make it look like a hop occurred
by adding trace data.

So how about:

Where a change to content between modules is unavoidable, it is a good idea
to add standard trace data to indicate a "visible" handoff between modules
has occurred.  The only advisable way to do this is to prepend Received
fields with the appropriate information, as described in Section 3.6.7 of
[MAIL].

Will make Ned's proposed change as well.  Here's a diff from -10 (which the
IESG approved) to a proposed -12, which incorporates everything to date,
including the above:

http://www.blackops.org/~msk/draft-ietf-appsawg-malformed-mail-from--10.diff.html

-MSK

---

From: Pete Resnick <[email-masked]>
Date: Wed, 27 Nov 2013 09:36:54 -0600

On 11/27/13 12:04 AM, Ned Freed wrote:

> Perhaps a change to something like this is in order:
>

wfm.

> aspect of that specification.

Yeah, well, I was still bright-eyed and bushy-tailed when I wrote that 
text. Grumpy-old me of today would have said different things.

> there is no way to determine the boundaries between blocks with any degree of

Actually, that's not quite true. The spec is clear that the Return-Path: 
is always on top of it's associated Received: fields. And though not 
crystal clear, it certainly implies that the Resent-Fields: get 
prepended *before* being reintroduced into the transport system, which 
is what ought to be prepending the Received: and Return-Path: fields. 
But yes, ambiguity abounds.

In any event, I agree with Ned that this is discussion of spilled milk 
that has gone over the bridge or under the dam or something. The fix he 
suggests above is perfectly fine for this document.

pr

---

From: Pete Resnick <[email-masked]>
Date: Wed, 27 Nov 2013 09:46:21 -0600

Everything in the diff you sent looks great. Just one outstanding bit:

On 11/26/13 12:23 PM, Murray S. Kucherawy wrote:
> I thought I clarified that in the latest draft by explaining what was 
> meant by "compound instance".

Could you give me an example of 3 (a compound instance) which is not a 4?

pr

---

From: Ned Freed <[email-masked]>
Date: Thu, 28 Nov 2013 08:25:52 -0800 (PST)

(IESG dropped from this since they have better things to do than discuss
message format arcana.)

> is always on top of it's associated Received: fields.

I'm skeptical of this claim. RFC 5322 section 3.6 states:

   It is important to note that the header fields are not guaranteed to
   be in a particular order.  They may appear in any order, and they
   have been known to be reordered occasionally when transported over
   the Internet.  However, for the purposes of this specification,
   header fields SHOULD NOT be reordered when a message is transported
   or transformed.  More importantly, the trace header fields and resent
   header fields MUST NOT be reordered, and SHOULD be kept in blocks
   prepended to the message.  See sections 3.6.6 and 3.6.7 for more
   information.

Seems to me this says that trace fields can't be reordered once they are
generated, but at the same time this appears to provide carte blanche for a
given operation adding trace fields to do it in any order it pleases.

And even if you believe that the ABNF showing that return-path comes first
within a given block overrides this text, there's also the fact that a header
"block" is itsef an ill-defined notion. I can easily envision an implementation
that adds return-path and one received field at one step, followed by
another process that adds another received field to log some additional
processing done to the message.

Since the specification never comes out and says that a given
create/submit/relay/deliver step MUST only add a single block of trace fields,
you've completely lost the ability to call an implementation incompliant when
it meets the technical requirements of the ABNF by generating multiple blocks
during one of these operations.

> And though not  crystal clear, it certainly implies that the Resent-Fields: get 
> prepended *before* being reintroduced into the transport system, which  is what
> ought to be prepending the Received: and Return-Path: fields.

The situation with resent-* is, if anything, worse. It would be one thing if
resent-* were associated exclusively with a particular sort of manual message
submission operation, which necessarily occurs after final delivery and thus if
everything is working perfectly will result in a clear ordering of blocks.

But RFC 5322 doesn't restrict resent-* fields to that role. And that means
that you cannot count on a delivery event having occurred prior to the
"reintroduction" event that's adding the resent-* fields.

Just as one example, sieve redirect can easily be seen to meet the criteria of
RFC 5322 section 3.6.6 irrespective of whether or not it's done prior to final
deliver, and in practice some sieve implementations operating before final
delivery do add resent- fields. (We provide it as an option in our
implementation.) How this then maps onto a particular set of block additions to
the message is anyone's guess.

				Ned

---

From: Ned Freed <[email-masked]>
Date: Fri, 29 Nov 2013 09:39:52 -0800 (PST)

> Could you give me an example of 3 (a compound instance) which is not a 4?

I don't know of any in RFC 5322, but I've seen content-disposition information
scattered across multiple instances of the header. But not often enough to
bother coding any kind of support to handle it.

That said, "3 but not 4" would appear to be sufficiently obscure that I have no
problem with combining them.

				Ned

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sat, 30 Nov 2013 20:31:23 -0800

Concur.  I'll send that out momentarily to you guys and, if no objections,
I'll send it to the RFC Editor to replace -11.

-MSK

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sat, 30 Nov 2013 20:33:00 -0800

I'll send it Monday unless I hear otherwise.