Subject: Re: [Emailcore] 'obsolete' considered misleading

From: Pete Resnick <[email-masked]>
Date: Tue, 04 May 2021 11:45:03 -0500

Just trying to close the loop on this:

Dave, Keith, Alessandro: Since you were the three who voiced concerns. 
Is the below from Ned and John acceptable?

pr

On 27 Apr 2021, at 18:00, Ned Freed wrote:

>> --On Tuesday, April 27, 2021 17:51 -0500 Pete Resnick
>> <[email-masked]> wrote:
>
>>> Answering a couple of items in this reply:
>>>
>>> On 26 Apr 2021, at 19:47, Keith Moore wrote:
>>>
>>>> Regarding the use of the word "obsolete" to describe once
>>>> common  syntax
>>>> that is less favored today: Consider "legacy" as an
>>>> alternative, and  try
>>>> to make it clear that readers need to continue to support
>>>> that syntax indefnintely.   I agree that "obsolete" conveys
>>>> the wrong  impression.
>>>
>>> I grant the point that "legacy" (or, as I mentioned before,
>>> "accept-only", or probably other choices) would have probably
>>> been better than "obsolete". But I still haven't seen an
>>> answer to: Is there a proposal to change the terminology
>>> and/or the ABNF element names? Or is there a proposal to
>>> better explain in section 1.2.3 (as quoted by Alessandro
>>> below) that we chose a misleading or confusing name for these
>>> things, and we apologize for that, but we're sticking with it
>>> to avoid even more confusion? Or is there some other proposal
>>> for what you all want done in the document? This editor needs
>>> guidance to come up with some proposed text.
>
>> I'm nervous about changes either introducing errors or being
>> interpreted as more than than are.  Consequently my personal
>> preference is that we add an explanation to section 1.2.3 along
>> the lines you describe, probably add a cross-reference to that
>> explanation early in Section 4, and then move on.
>
> +1
>
> I'll again repeat that we've managed something fairly remarkable here: 
> A message
> written in 1980 is almost certain to work with current agents. Do we 
> really want
> to compromise this?
>
> Try reading an X.400-1984 message with an 1988-only version sometime.
>
> 				Ned

-- 
Pete Resnick https://www.episteme.net/
All connections to the world are tenuous at best

---

From: Keith Moore <[email-masked]>
Date: Tue, 4 May 2021 17:29:45 -0400

> Is the below from Ned and John acceptable?
I probably have a slight preference for changing the names of ABNF 
productions. However "legacy" was just a suggestion and one made without 
a lot of thought about how much if anything should be changed.   I can 
certainly live with Ned & John's suggestion instead.

Keith

---

From: Dave Crocker <[email-masked]>
Date: Tue, 4 May 2021 14:36:43 -0700

> Is the below from Ned and John acceptable?

My original posting was to raise a concern.  Poor documentation language 
encourages poor documentation understanding.  I was seeking discussion 
of the concern.

Some of the reactions seem to have mistaken that for a demand that there 
be massive documentation, syntax or semantic changes, or to have found 
fault that the expression of concern did not also including specific 
recommendations.

I wanted to get some agreement about the nature of the issue, before 
making more specific suggestions.

1. The word 'obsolete' has very clear and specific semantics.  The 
multi-decade reality of this specification makes obvious that the term 
wasn't and isn't appropriate.  Whether a distinction between generation 
and interpretation syntaxes was or remains appropriate is quite a 
separate matter.  But the terminology is, as I noted, misleading (at 
best.)

Happily(?) The last paragraph of Section 3.1 provides clear and precise 
normative guidance about the actual /use/ of the unfortunately-prefaced 
ABNF.  This permits arguing for a minimal revision with an annotation, 
given the modest goals of the current effort.  So, a highlighted noted 
that comments on the nature of the terminology choice is not 
unreasonable. For example, after that paragraph:

    NOTE: These rules have been classed as obsolete by virtue of their 
no longer being valid to use for message generation. Originally, it was 
thought that it might also be eventually possible to drop the rules from 
the specification, but support for old messages have remains a concern.

2. Ale's note suggesting text attempts to distinguish interpreting 
archived messages versus incoming ones.  This draws am operationally 
bright line that has not been evident and might not be intended:

      Email parsing for incoming mail is not required to support the
      OBS-* rules.

That's the operational effect of his language.  Is that really what folk 
find acceptable?

(Ironically, that interpretation and effect really does make the OBS-* 
labeling choice appropriate...)

If that's really what folk want, I'll offer some tweaks to his proposed 
text:

NEW:
   Section 4 of this document specifies syntax rules that are labeled
   "obsolete".  Section 3 includes reference to these obsolete
   syntactic elements. They have appeared in earlier versions of this
   specification or have previously been widely used in Internet
   messages.  However, these syntactic elements MUST NOT be used when
   generating messages conforming to the current specification and MAY
   NOT be used when interpreting received messages.

   Syntactic rules beginning with "obs-" have been determined to be non-
   interoperable or to cause significant problems for recipients of
   messages. However, old mail archives exist and are being being
   actively preserved.  Therefore, parsers that are applied to archives
   that might contain older messages SHOULD support the obs- elements.

There are two landmines here.

One has been longstanding:  The spec required supporting the obs- rules 
by interpreters, while also saying that the rules are obs- because they 
cause problems for interpreters.  (The fact that the nature of the 
problems wasn't explained is secondary.)'

    This pill will cause problems but you must swallow it...

The other landmine is incoming mail that contains an embedded messages 
from an archive.

This, of course, suggests that the distinction between incoming and 
archive is a convenient -- but problematic -- myth, since there is no 
obvious labeling of mail to distinguish what version of the 
specification it conforms to.

d/

-- 
Dave Crocker
Brandenburg InternetWorking
bbiw.net

---

From: Keith Moore <[email-masked]>
Date: Tue, 4 May 2021 18:20:21 -0400

On 5/4/21 5:36 PM, Dave Crocker wrote:

> That's the operational effect of his language.  Is that really what 
> folk find acceptable?

I do not find it acceptable.   Quite the opposite, actually.  In general 
conforming MUAs and most other mail handling programs need be able to 
parse and deal with email messages containing obs- constructs.

It might be tempting to try to carve out specific exceptions, such as 
saying that MUAs or MSAs are free to reject new messages that contain 
obs- constructs at the time of origination.   But mostly I think it's a 
Bad Idea to try to accommodate these exceptions, because there are 
exceptions to the exceptions.  (like resending an old message, or 
forwarding an old message inside a new message).   I strongly suspect 
that we're better off if there's a single grammar for parsing all email 
messages, regardless of presumed age, than to try to enumerate the 
specific cases in which an obs- construct should never appear.   Simpler 
rules make for fewer bugs, and the existing rules are complicated enough 
already.   (Getting rid of obs- constructs doesn't make the rules 
simpler, it makes them more complicated.)

> One has been longstanding:  The spec required supporting the obs- 
> rules by interpreters, while also saying that the rules are obs- 
> because they cause problems for interpreters.  (The fact that the 
> nature of the problems wasn't explained is secondary.)'

IMO trying to save interpreters the burden of parsing obs- constructs 
was always shortsighted, and should now be abandoned as a goal.    I 
agree with others that being able to handle 40+ year old messages with 
standard tools is a huge win, one which will only become more useful 
over time.

If we still think that obs- constructs are sometimes not implemented 
correctly, the thing to do at this point is to develop, identify, and/or 
promote open source implementations that do implement them correctly, or 
at least without major failures.

> specification it conforms to.

IMO it's not even a convenient myth, since believing in the myth causes 
more problems than it solves.

Keith

---

From: Alessandro Vesely <[email-masked]>
Date: Wed, 5 May 2021 14:35:31 +0200

On Wed 05/May/2021 00:20:21 +0200 Keith Moore wrote:
>> 2. Ale's note suggesting text attempts to distinguish interpreting archived 
>> messages versus incoming ones.  This draws am operationally bright line that 
>> has not been evident and might not be intended:
>>
>> That's the operational effect of his language.  Is that really what folk find 
>> acceptable?
> 
> conforming MUAs and most other mail handling programs need be able to parse and 
> deal with email messages containing obs- constructs.

There are very many programs which happen to do email, and email addresses in 
particular, without having to fully understand message format.

> It might be tempting to try to carve out specific exceptions, such as saying 
> that MUAs or MSAs are free to reject new messages that contain obs- constructs 
> at the time of origination.   But mostly I think it's a Bad Idea to try to 
> accommodate these exceptions, because there are exceptions to the exceptions.  
> (like resending an old message, or forwarding an old message inside a new 
> message).

Resending an old message should imply either wrapping it inside a 
message/rfc822 attachment or reformatting it.  MUAs do reformat messages as 
users edit them.  Mediators reformat some header fields anyway, like 
MIME-Version, Content-Transfer-Encoding.

I think if you need to resend a message, old or new, making sure that all of 
its format is left intact, you're better off sealing it inside a zip.

> I strongly suspect that we're better off if there's a single grammar for
> parsing all email messages, regardless of presumed age, than to try to
> enumerate the specific cases in which an obs- construct should never 
> appear.

We already have syntax compartments.  For example, SMTP says:

    for maximum interoperability, a host that expects to receive mail
    SHOULD avoid defining mailboxes where the Local-part requires (or
    uses) the Quoted-string form or where the Local-part is case-
    sensitive.

Hosts that expect to receive messages is an operational category akin to the 
one of messages that expect to be sent.

>> This, of course, suggests that the distinction between incoming and archive 
>> is a convenient -- but problematic -- myth, since there is no obvious 
>> labeling of mail to distinguish what version of the specification it conforms 
>> to.
> 
> IMO it's not even a convenient myth, since believing in the myth causes more 
> problems than it solves.

Obsolete messages don't have to be a myth.  Yet they exist.

Best
Ale
--

---

From: Ned Freed <[email-masked]>
Date: Wed, 05 May 2021 08:05:31 -0700 (PDT)

> particular, without having to fully understand message format.

While true, I fail to see the relevance. Programs that don't need to parse
messages will continue to not need to parse messages no matter what we say or do
with the obsolete syntax.

> message/rfc822 attachment or reformatting it.

First, resending/forwarding is not generating.  The restrictions on message
generation do not and should not apply. If this is in any unclear, it needs to
clarified.

The reason this is important is conversion of obsolete syntax to modern syntax
in full generality is close to, if not actually, a gateway problem. Gateways are
hard to code and even harder to code correctly. Absent a specification for how
to do the conversion, this is not something we want to encourage.

Second, the entire point of using message/rfc822 is to identify the message as a
message so receiving agents can parse and process it properly. As such, it
doesn't remove any requirement for processing obsolete syntax. If anything
it's the exact opposite.

> MUAs do reformat messages as

Yes, and one only has to look at some of the HTML that results from multiple
MUAs having had a crack at the content to see what a wonderful idea this is.

That said, composing a reply usually involves extraction of information from the
original message. And in some cases, like extraction of addresses from message
headers, this does necessitate a limited form of conversion form obsolete to
modern syntax.

However, while problems with address extraction are common, IME use of
obsolete syntax has nothing to do with it. It's far more likely that
something that's allowed by the current syntax like

   "[email-masked]" <[email-masked]>

is going to cause a MUA to do Bad Things than, say, something like

    ned . [email-masked]

that is obsolete. 

In any case, the need for such conversion This is unfortunate, but nothing we
say or do now is going to make the who knows how many billions of messages
stored in archives go away.

> its format is left intact, you're better off sealing it inside a zip.

Hmm. Well, all I can say is I forward old messages pretty regularly to people
for whom "zip" refers to something on their clothes. They are about as likely to
be able to deal with a zip file unassisted as I am to run a marathon tomorrow.

I doubt very much I'm alone in this.

>     uses) the Quoted-string form or where the Local-part is case-

This falls far short of defining a compartment. In fact it really
doesn't belong in the standard at all; it's the sort of thing that
needs to be moved to the applicability statement.

				Ned

---

From: Steffen Nurpmeso <[email-masked]>
Date: Wed, 05 May 2021 18:44:17 +0200

Ned Freed wrote in
 <[email-masked]>:
 |> On Wed 05/May/2021 00:20:21 +0200 Keith Moore wrote:
 |>> On 5/4/21 5:36 PM, Dave Crocker wrote:
 ...
 |> Resending an old message should imply either wrapping it inside a
 |> message/rfc822 attachment or reformatting it.
 |
 |First, resending/forwarding is not generating.  The restrictions on message
 |generation do not and should not apply. If this is in any unclear, \
 |it needs to
 |clarified.

I concur.  We strictly only add Resend-* headers and ship the rest
"as-is".  (Having said that, it may change in the future when the
software parses a mail as an object tree that is then "dumped to
wire".  But the current state is decade old behaviour, and things
like DKIM etc. are built upon cheap production over plain RFC 5322
message format, if i understood and recall correctly, and as
opposed to extraction of content-encodings and normalization.)

  ...
 |In any case, the need for such conversion This is unfortunate, but \
 |nothing we
 |say or do now is going to make the who knows how many billions of messages
 |stored in archives go away.
  ...

(I for one never understood the discussion as such.  MUAs also
have to face MBOX From_ lines with obscured mail addresses a bit
in order to function, which is not even obsolete.  And what is
unclear about "obsolete", i always thought of that as, as has been
said in this thread already, "do not generate, but could be seen
in the wild".)

--steffen
|
|Der Kragenbaer,                The moon bear,
|der holt sich munter           he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

---

From: Dave Crocker <[email-masked]>
Date: Wed, 5 May 2021 09:48:13 -0700

On 5/5/2021 9:44 AM, Steffen Nurpmeso wrote:
> We strictly only add Resend-* headers and ship the rest
> "as-is".  
...
>       And what is

Shipping text that conforms to the obs- rules is the same as generating 
that text.  There is no systems-level distinction.

d/

---

From: Steffen Nurpmeso <[email-masked]>
Date: Wed, 05 May 2021 20:22:57 +0200

Dave Crocker wrote in
 <[email-masked]>:
 |On 5/5/2021 9:44 AM, Steffen Nurpmeso wrote:
 |> We strictly only add Resend-* headers and ship the rest
 |> "as-is".  
 |...
 |>       And what is
 |> unclear about "obsolete", i always thought of that as, as has been
 |> said in this thread already, "do not generate, but could be seen
 |> in the wild".)
 |
 |Shipping text that conforms to the obs- rules is the same as generating 
 |that text.  There is no systems-level distinction.

To me..  I do not expect a RFC to represent literate programming
in the sense of CWEB, but in the academic field cross-references,
footnotes, indices and other forms of context seem higly respected
and appreciated, i always rejoice when reading a book of Oxford
University Press for example (Ivanhoe!) and so it is here.
Leaving it all entirely up to some lonely ABNF (or wherever that
sails to) near the end of a document may not be sufficient to
understand the troubles programs could be faced with in real life.

I admire a word of experience or context more than pages of the
most detailed ABNF, even if contents are now hyperlinked (shall
that be true).  If you look at the list of roman emperors[1], ie
the coins and busts, and compare from Augustus to Caracalla, and
what comes thereafter .. and after, .. i do not see improvement
and refinement by reduction, but only loss of culture and spirit.

  [1] https://en.wikipedia.org/wiki/List_of_Roman_emperors

--steffen
|

---

From: Alessandro Vesely <[email-masked]>
Date: Wed, 5 May 2021 20:34:23 +0200

On Wed 05/May/2021 17:05:31 +0200 Ned Freed wrote:
>>> conforming MUAs and most other mail handling programs need be able to parse 
>>> and deal with email messages containing obs- constructs.
> 

A compliant MTA has to recognize and strip off source routes.  But why should a 
web form that validates email addresses care about obs-angle-addr?

> generation do not and should not apply. If this is in any unclear, it needs to

That's clear.  So is the observation that shipping text that conforms to the 
obs- rules is the same as generating that text.

> to do the conversion, this is not something we want to encourage.

I don't think one needs to convert obs- to current.  Yet, if you tinker with 
obs- messages, some parts may happen to be converted.

> doesn't remove any requirement for processing obsolete syntax. If anything

On the one hand, a parser acting on email message attachment may happen to be 
applied on obsolete content.  As such, it MUST be able to interpret obs- 
elements, according to the text I proposed, or, better, it SHOULD support the 
obs- elements, according to Dave's tweak.

OTOH, we have MIME types for anything.  How about message/obs-822?

> MUAs having had a crack at the content to see what a wonderful idea this is.

Agreed!

> be able to deal with a zip file unassisted as I am to run a marathon tomorrow.

:-)

If you send that message to the Museum of Historical Mail Formats, they 
probably can deflate the zip.  Otherwise, there is no point in striving to 
maintain the exact format the message had when it was sent at the time.

> needs to be moved to the applicability statement.

It's still in rfc5321bis-02.

Best
Ale
--

---

From: Keith Moore <[email-masked]>
Date: Wed, 5 May 2021 14:40:56 -0400

On 5/5/21 8:35 AM, Alessandro Vesely wrote:

>> I do not find it acceptable.   Quite the opposite, actually.  In 
>> general conforming MUAs and most other mail handling programs need be 
>> able to parse and deal with email messages containing obs- constructs.
>
> There are very many programs which happen to do email, and email 
> addresses in particular, without having to fully understand message 
> format.

Yes, that's true.   Many things can be properly implemented without 
having to do any significant parsing.    I guess I should say instead 
that mail handling programs should not break or incorrectly implement a 
feature because a message contains obs- constructs.

>> obs- constructs at the time of origination.   But mostly I think it's 
>> a Bad Idea to try to accommodate these exceptions, because there are 
>> forwarding an old message inside a new message).
>
>
> message/rfc822 attachment or reformatting it.  MUAs do reformat 
> messages as users edit them.  Mediators reformat some header fields 
> anyway, like MIME-Version, Content-Transfer-Encoding.
>
> I think if you need to resend a message, old or new, making sure that 
> all of its format is left intact, you're better off sealing it inside 
> a zip.

Emphatically disagree.    We should not be trying to change how people 
use email, or disable features that people have long found valuable.  
And there are sometimes good reasons for leaving the original message 
format intact, such as when you receive a complex message that has been 
misdirected and you want to send it to the right place.   A resent 
message and a forwarded message are semantically different, and 
forwarding when you mean to resend degrades the message.   Also some UAs 
still don't read attached messages as functionally as they do top-level 
messages - they might not implement reply to an attached message, for 
instance.

>    uses) the Quoted-string form or where the Local-part is case-

I thought we were talking about the message format, not best operational 
practice for assigning mailbox names.

More generally, what an implementation should support is a very 
different topic than what we'd recommend that an operator do.

> Hosts that expect to receive messages is an operational category akin 
> to the one of messages that expect to be sent.

disagree.

>>> archive is a convenient -- but problematic -- myth, since there is 
>>> no obvious labeling of mail to distinguish what version of the 
>> IMO it's not even a convenient myth, since believing in the myth 
>> causes more problems than it solves.
>
>

Mostly disagree.   We've never declared a particular kind of message to 
be obsolete, and as a practical matter every decent MUA needs to be able 
to deal with old messages.   I might well consider rfc733 messages to be 
obsolete, since none of the updates to rfc822 ever required support for 
"user at host" addresses. But (nearly) anything that conforms to RFC822 
and later should still be supported, to the extent feasible.   We don't 
expect pre-DNS hostnames to work, or DNS names that no longer are in DNS 
to be resolvable.  But that doesn't make the messages unreadable.

Maybe we really do need to change the obs- symbols in the ABNF to 
clarify this situation.

Keith

---

From: Ned Freed <[email-masked]>
Date: Wed, 05 May 2021 13:18:38 -0700 (PDT)

> that text.  There is no systems-level distinction.

Absolutely, and that is the reason why transport agents have to be able to
accept obsolete syntax.

But this isn't why we restrict newly generated material to a subset of the
overall syntax. We do that because we believed generating only this subset
improves overall interoperability.

In hindsight I wish we had been a bit more, um, liberal in what we 
chose to move to the obsolete side of things. But that's water
under the bridge, and given the charter of this group the focus
needs to be on how best to describe the syntax we have.

				Ned

---

From: Ned Freed <[email-masked]>
Date: Wed, 05 May 2021 11:54:18 -0700 (PDT)

> web form that validates email addresses care about obs-angle-addr?

I'm skeptical that most web forms accept anything beyond addr-spec, but even so,
I take your point: While it seems unlikely that J. Random User is going to
directly enter an obsolete syntax address as anything other than a mistake,
people can and do cut and paste from various sources.

And addr-spec has its own obsolete elements.

My immediate thought would be to try and cover this point in the applicability
statement: What piece of grammer is appropriate for a web form accepting an
email address to use?

> obs- rules is the same as generating that text.

I'm afraid I have to disagree. It's completely inappropriate to apply generating
rules to a forwarded messsage within that text, for example. The term "generate"
is used for a reason: It applies to stuff you *generate*.

The relationship of MIME to obsolete syntax is definitely something that
will need to be covered in any future revision of MIME.

> obs- messages, some parts may happen to be converted.

You do if you believe everything you submit to the transport infrastructure
has to be treated as if you generated it.

> OTOH, we have MIME types for anything.  How about message/obs-822?

An interesting idea, and had the obs concept existed at the time MIME was
created I for one would have given it serious consideration. Unfortunately that
was never possible given that MIME (RFCs 1341-1342) significantly predate the
first revision of the message format and SMTP documents (RFC 2821-2822).

Given the cost and benefits, I don't think a strong enough case can be made at
this late date for adding such a type. FWIW, IME message/global uptake hasn't
been all that great, and the reasons for a separate type there are pretty
compelling.

> maintain the exact format the message had when it was sent at the time.

Now you have me thinking about the message format used once upon a time by Sun
workstations, which tar'ed up the parts. Or the one used by NeXT computers,
which used some flavor of zip.

And Microsoft's TNEF, of course. Never forget TNEF. There's stuff out there
that still sends it.

> It's still in rfc5321bis-02.

A move should be considered, since it's clearly interoperability advice.

				Ned

---

From: John C Klensin <[email-masked]>
Date: Wed, 05 May 2021 21:49:17 -0400

--On Wednesday, May 5, 2021 11:54 -0700 Ned Freed
<[email-masked]> wrote:

>...
>> A compliant MTA has to recognize and strip off source routes.
>> But why should a web form that validates email addresses care
>> about obs-angle-addr?
 
> I'm skeptical that most web forms accept anything beyond
> addr-spec, but even so, I take your point: While it seems
> unlikely that J. Random User is going to directly enter an
> obsolete syntax address as anything other than a mistake,

> And addr-spec has its own obsolete elements.

And, based on purely anecdotal observations (but many of them),
the far bigger problem with sloppily designed or implemented web
forms involves rejecting as "invalid addresses" strings that
contain perfectly valid characters.  The most frequent problems
seem to involve "+", but I've seen issues with "=", "/", and
even "." to say nothing of assorted constructions using quotes
and assumptions that local parts are case insensitive (see below
for more about that).  None of that interacts with so-called
obsolete forms or "obs-" syntax productions, but I think all
they lead to the same place, specifically... 
 
> My immediate thought would be to try and cover this point in
> the applicability statement: What piece of grammer is
> appropriate for a web form accepting an email address to use?

Either couched as web form-specific or as a section about common
mistakes, the problems they cause, and how competent
implementations deal with them.  While web forms pose special
"opportunities" as you and others have pointed out, most of the
problems can be encountered by MUAs that are trying to adapt to
copying or pasting bits of syntax from ancient or fussy messages
(or elderly users with what we now consider bad habits).   FWIW,
you may remember that I/we tried to address some of the same
issues in RFC 3696.  While I'd rate that document as a
resounding failure, using bits of the portions of it that would
fall into this WG's scope as a starting point for some of the
A/S and then obsoleting all or part of that old effort would
probably be helpful.

>...
>> > The reason this is important is conversion of obsolete
>> > syntax to modern syntax in full generality is close to, if
>> > not actually, a gateway problem. Gateways are hard to code
>> > and even harder to code correctly. Absent a specification
>> > for how to do the conversion, this is not something we want
>> > to encourage.
> 
>> I don't think one needs to convert obs- to current.  Yet, if
>> you tinker with obs- messages, some parts may happen to be
>> converted.
> 

And the reality is that, absent either treating every instance
of obs- as a gateway situation (as you pointed out) and/or the
arrival of the protocol police, implementations will handle the
edge cases however they handle them.  It would be nice to expect
complete consistency but it is unrealistic.  And, even if the
protocol police show up, implementations that decide to transmit
the obs- stuff will just say "well, they have to accept it, so
we are not _really_ doing anything wrong.  Been there... all too
many times and I'm sure you and others have too.

My conclusion is that making significant changes is likely to
create more confusion and cause more problems than the changes
will actually solve.  It would give us additional ability to
point to particular implementers and their users and say "you
are a non-conforming evildoer", but the historical evidence
suggests that the worst offenders won't care.  Trying to split
hairs to get the explanations exactly right just won't help ...
and I certainly hope the WG has more important issues to spend
time on.  

And that, I think, takes us back to where we were a while ago,
only, I hope, with more understanding of what the A/S should say
(and I hope someone is making notes).  Specifically, 5322bis
gets a paragraph explaining our poor choice of terminology and
then we move on, concentrating on explanations of good behavior,
bad behavior, and the risks of the latter in the A/S.

>...
>> >> I think if you need to resend a message, old or new,
>> >> making sure that all of its format is left intact, you're
>> >> better off sealing it inside a zip.
>> > 
>> > Hmm. Well, all I can say is I forward old messages pretty
>> > regularly to people for whom "zip" refers to something on
>> > their clothes. They are about as likely to be able to deal
>> > with a zip file unassisted as I am to run a marathon
>> > tomorrow.

To say nothing of the anti-malware systems who think that the
only reason to send a compressed file or archive is get assorted
forms of evil past them.  If one of those intervenes, the users
of whom you speak are likely to not see the message at all
unless they make a habit of searching trash folders for things
that they might have expected by can't decode.

>...
>> If you send that message to the Museum of Historical Mail
>> Formats, they probably can deflate the zip.  Otherwise, there
>> is no point in striving to maintain the exact format the
>> message had when it was sent at the time.

> Now you have me thinking about the message format used once
> upon a time by Sun workstations, which tar'ed up the parts. Or
> the one used by NeXT computers, which used some flavor of zip.

> And Microsoft's TNEF, of course. Never forget TNEF. There's
> stuff out there that still sends it.

All problems MIME was supposed to solve/ eliminate, of course.
NeXT and Sun may have taken care of themselves, but I received a
message that was dependent of TNEF last week.  I have observed
that facing in the direction of Redmond, pointing one or more
fingers, and shouting "evil violation of standards" has not been
successful in bringing about improvements or even in getting the
messages decoded.

>> >>> I strongly suspect that we're better off if there's a
>> >>> single grammar for parsing all email messages, regardless
>> >>> of presumed age, than to try to enumerate the specific
>> >>> cases in which an obs- construct should never appear.
>> > 
>> >>     for maximum interoperability, a host that expects
>> >> to receive mail     SHOULD avoid defining mailboxes
>> >> where the Local-part requires (or     uses) the
>> >> Quoted-string form or where the Local-part is case-    
>> >> sensitive.
>> > 
>> > This falls far short of defining a compartment. In fact it
>> > really doesn't belong in the standard at all; it's the sort
>> > of thing that needs to be moved to the applicability
>> > statement.
> 

Concur.  It is definitely not the only text in 5321/5321bis that
was put there to keep people out of trouble with others who fail
to carefully follow the specs.  Without looking, I'm not sure
how many of those cases can be identified and moved to another
document in a surgically clean way.  If, as I expect, the answer
turns out to be "some but not all", the WG will need to decide
what to do about that and, in particular, whether it is ok to
fix the easy ones and leave the others (including the ones no
one spots and calls out). But, in principle, I'd be happy to
have every bit of it taken out and moved to the A/S.

best,
    john

---

From: John C Klensin <[email-masked]>
Date: Wed, 05 May 2021 22:26:54 -0400

Hi.  I have already semi-covered this, but want to address a
different part of it and use a new subject line to keep it from
getting lost.

--On Wednesday, May 5, 2021 14:40 -0400 Keith Moore
<[email-masked]> wrote:

>>    for maximum interoperability, a host that expects to
>> receive mail    SHOULD avoid defining mailboxes where the
>> Local-part requires (or    uses) the Quoted-string form or
>> where the Local-part is case-    sensitive.
> 
> I thought we were talking about the message format, not best
> operational practice for assigning mailbox names.

And that comment is not a syntax compartment or really about
syntax at all.  It does absolutely nothing to change the
requirements that implementations support full Quoted-string
syntax and case-sensitive local parts.  Perhaps there should be
been more explanation, but what is does -- all it does -- is to
tell mail host operators (not SMTP implementers) that there are
things out there (enough of them to be a problem) that do not
understand the subtleties associated with those forms and so,
depending on them is probably a bad idea.  If I (or the WG, had
been inclined to write more test, we would even have explained
that, e.g., 
treating [email-masked] as different from [email-masked] and
rejecting one of them as a bad address is probably undesirable
but treating them as separate addresses and routing them to
different mailboxes (and probably people) is likely worse.
And, in that context, the "things" that can get confused are not
just mail system implementations.  They can be other software
entirely (including web forms) or even human-type users.

Per the other note, this really is A/S material buried in the
Technical Specification.   It is not unusual to combine the two,
so doing so in 5321 and its predecessors is not some sort of
error, but, if we are doing a separate A/S document, it may make
sense to move some or all statements of that sort.

I don't know if it is all a candidate for moving to the A/S (see
the comments in my earlier note about surgically clean removal),
but it is worth noting that all of Section 7.8 of RFC 5321 (and,
so far, draft-ietf-emailcore-rfc5321bis) is also about
operations and configuration choices.  It does not change the
requirements for an implementation except in one way that is
only implicit: that language suggests that smart implementers
will design enough configuration options or "knobs" into their
implementations to either automate the sorts of adjustments that
section specifies, allow making them on a per-host or
per-installation basis (ideally in real time rather than forcing
the SMTP code to be restarted), or both.  Should that have been
more clear?  Perhaps although the argument against it is that
SMTP is supposed to be about what happens on the wire rather
than the fine details of implementations.

> More generally, what an implementation should support is a
> very different topic than what we'd recommend that an operator
> do.

Exactly.

best,
   john

---

From: Todd Herr <[email-masked]>
Date: Thu, 6 May 2021 22:21:40 -0400

As chair...

It seems we were close to consensus earlier this week on the idea of adding
some text to section 1.2.3 explaining that "obsolete" wasn't really the
correct word to use, with a back reference to it in section 4, but now the
thread has wandered off that path. I'm going to try to bring it back to the
path, because I think the thread has run its course, and I'd like to bring
this to a conclusion and move on.

Let me propose some text to see if we can't nudge this across the line.

Here is the current paragraph from section 1.2.3:

   Section 4 of this document specifies an "obsolete" syntax.  There are

   references in section 3 to these obsolete syntactic elements.  The

   rules of the obsolete syntax are elements that have appeared in

   earlier versions of this specification or have previously been widely

   used in Internet messages.  As such, these elements MUST be

   interpreted by parsers of messages in order to be conformant to this

   specification.  However, since items in this syntax have been

   determined to be non-interoperable or to cause significant problems

   for recipients of messages, they MUST NOT be generated by creators of

   conformant messages.

And I propose as a starting point adding the following paragraph after that
one:

   Note: The dictionary definition of "obsolete" is "no longer in use or

   no longer useful". While this specification mandates that these syntactic

   elements no longer be generated, it also mandates that conformant parsers

   be able to support them. The reason for this latter requirement is that

   there are long-established sites on the internet with mail archives that

   go back decades, archives with messages containing these elements. While

   these archives may only be mined occasionally, they are nonetheless still

   in use, making "obsolete" the incorrect term to describe these elements.

   Later efforts to revise this specification contemplated changing the term

   to "legacy" or something that would more accurately describe the
elements,

   but such a change was rejected due to fears that it would result in

   unnecessary confusion, especially among long-time users and implementers
of

   the specification.

Have at it...

-- 

*Todd Herr* | Sr. Technical Program Manager
*e:* [email-masked]
*m:* 703.220.4153

This email and all data transmitted with it contains confidential and/or
proprietary information intended solely for the use of individual(s)
authorized to receive it. If you are not an intended and authorized
recipient you are hereby notified of any use, disclosure, copying or
distribution of the information included in this transmission is prohibited
and may be unlawful. Please immediately notify the sender by replying to
this email and then delete it from your system.

---

From: Dave Crocker <[email-masked]>
Date: Thu, 6 May 2021 19:25:53 -0700

> Have at it...

Your proposed text is quite good.

d/

---

From: Keith Moore <[email-masked]>
Date: Thu, 6 May 2021 23:31:16 -0400

On 5/6/21 10:21 PM, Todd Herr wrote:

>      but such a change was rejected due tofears that it would result in
>
>      unnecessary confusion, especially amonglong-time users and
>     implementers of
>
>        the specification.
>
>
Suggest instead:

    no longer useful". While this specification mandates that these
    syntactic

    elements no longer be generated, it also mandates that conformant
    parsers

    be able to support them. One reason for this latter requirement is that

    ^^^

    these archives may only be mined occasionally, they are nonetheless
    still

and add something like:

        Another, similar, reason for this requirement is that many
    people have
        decades-old messages in their personal message stores, and for
    various
        reasons it is occasionally useful to not only read such messages
    but
        also resend or forward them to others.

    Later efforts to revise this specification contemplated changing the
    term

      but such a change was rejected due tofears that it would result in

      unnecessary confusion, especially amonglong-time users and
    implementers of

      the specification.

---

From: John C Klensin <[email-masked]>
Date: Fri, 07 May 2021 00:54:37 -0400

--On Thursday, May 6, 2021 22:21 -0400 Todd Herr
<todd.herr=[email-masked]> wrote:

> As chair...
> 
> It seems we were close to consensus earlier this week on the
> idea of adding some text to section 1.2.3 explaining that
> "obsolete" wasn't really the correct word to use, with a back
> reference to it in section 4, but now the thread has wandered
> off that path. I'm going to try to bring it back to the path,
> because I think the thread has run its course, and I'd like to
> bring this to a conclusion and move on.
> 

> And I propose as a starting point adding the following
> paragraph after that one:

>  Note: The dictionary definition of "obsolete" is "no longer
>  in use or no longer useful". While this specification
>  mandates that these syntactic elements no longer be
>  generated, it also mandates that conformant parsers be able
>  to support them. The reason for this latter requirement is
>  that there are long-established sites on the internet with
>  mail archives that go back decades, archives with messages
>  containing these elements. While these archives may only be
>  mined occasionally, they are nonetheless still in use,
>  making "obsolete" the incorrect term to describe these
>  elements.

>  Later efforts to revise this specification contemplated
>  changing the term to "legacy" or something that would more
>  accurately describe the elements, but such a change was
>  rejected due to fears that it would result in unnecessary
>  confusion, especially among long-time users and implementers
>  of the specification. 

> Have at it...

Todd,

I am ok with your text and wish we could accept it and move on.
However, if we are trying to be absolutely correct, Keith is
correct: some of the personal collections and archives are as
important (at least to them and potential historians) as the
more public or formally maintained ones that your text refers
to.  His text doesn't quite spell it out, but there is no reason
why some of those collections have to be online ("one the
internet" (sic)) either. If we want to recognize that, a much
shorter and possibly more effective fix would be to change...

Old:
	The reason for this latter requirement is that there are
	long-established sites...  

Proposed:
	Reasons for this latter requirement include the
	existence of both decades-old personal and
	organizational archives and long-established sites...  

Noting the possibility that we could now generate another long
thread of messages in group editing and nit-picking of text, my
suggestion is that Pete confirm that he understands and accepts
the gist of the requested change and that we turn the finer
details over to editor's discretion and, in the long run,
whatever can be worked out with the RFC Editor.

thanks,
    john

---

From: Ned Freed <[email-masked]>
Date: Thu, 06 May 2021 20:12:58 -0700 (PDT)

Seems fine to me.

				Ned

---

From: Alessandro Vesely <[email-masked]>
Date: Fri, 7 May 2021 13:20:46 +0200

> Your proposed text is quite good.

+1, since the map and the terrain disagree...

Best
Ale
--

---

From: Keith Moore <[email-masked]>
Date: Fri, 7 May 2021 07:37:37 -0400

> whatever can be worked out with the RFC Editor.
I'd be ok with this.

Keith

---

From: Pete Resnick <[email-masked]>
Date: Fri, 07 May 2021 10:29:53 -0500

>> whatever can be worked out with the RFC Editor.

I do believe I understand the gist, and I'm happy to take the text here 
and wordsmith it as appropriate. I'll look to put something both in 
1.2.3 and at least a back pointer at the top of section 4.

pr

---

From: Todd Herr <[email-masked]>
Date: Fri, 7 May 2021 11:41:14 -0400

> 1.2.3 and at least a back pointer at the top of section 4.
Thanks, Pete (and everyone who contributed).

Let's call this thread closed and move on to other matters.

As chair...
--

---

From: Dave Crocker <[email-masked]>
Date: Fri, 7 May 2021 08:42:39 -0700

> 1.2.3 and at least a back pointer at the top of section 4.

Now I'm confused.

Todd offered precise language.  Ned, Alessandro and I express solid 
support for that language.

John says he is ok with the text but then proceeds to explain that he 
really isn't.

Keith expresses a desire for you to do separate wordsmithing.

That is 4 people (but arguably 5) like an existing set of words.

Two others suggest a task that has only generic parameters, with no 
clear sense of deficiencies in the existing, proposed text.

And the conclusion is that you should do separate wordsmithing?

d/

---

From: Todd Herr <[email-masked]>
Date: Fri, 7 May 2021 12:16:01 -0400

> And the conclusion is that you should do separate wordsmithing?
I believe the only wordsmithing or even substantive changes to the text I
proposed was in regards to calling out two distinct types of old mail
stores rather than the one that was in my original text.

My text referred to "long-established sites on the internet with mail
archives that go back decades" and may be mined, and Keith suggested adding
"many people have decades-old messages in their personal message stores,
and for various reasons it is occasionally useful to not only read such
messages but also resend or forward them to others" as a second example of
mail stores that contain mail messages that have obsolete syntax.

There is consensus that "obsolete" is the wrong word to use, and there is
consensus to add a paragraph explaining why it's wrong but won't be
changed. There is language for this paragraph in the combination of what I
wrote and what Keith added to what I wrote:

   Another, similar, reason for this requirement is that many people have
   decades-old messages in their personal message stores, and for various
   reasons it is occasionally useful to not only read such messages but
   also resend or forward them to others.

   the specification.

I see nothing factually incorrect in that language, and I don't expect that
any wordsmithing Pete might do will alter the message being conveyed here,
which is "Obsolete is the wrong word, because reasons, but we didn't change
it, because confusion."

When the document gets to the RFC Editor part of the process, it could very
well see a change made that combines the two examples (long-established
sites that mine old messages and people who read old messages) into one
more generic example, perhaps drawing on or similar to what John wrote
("Reasons
for this latter requirement include the existence of both decades-old
personal and organizational archives and long-established sites... ") or
perhaps Pete will make such a change himself, but again, that won't change
the overall message of "Obsolete wrong word because reasons, didn't change
because confusion."

--

---

From: Pete Resnick <[email-masked]>
Date: Fri, 07 May 2021 12:36:15 -0500

On 7 May 2021, at 10:42, Dave Crocker wrote:

>> I do believe I understand the gist, and I'm happy to take the text 
>> here and wordsmith it as appropriate. I'll look to put something both 
>> in 1.2.3 and at least a back pointer at the top of section 4.
>
> Now I'm confused.

I hope Todd's followup message helped to unconfuse you, as I think it's 

> support for that language.

Yep. Got that part.

> John says...

Actually Keith first says that he's OK with the text, but wants a 
clarification that mail archives are not the one and only reason for 
parsing these "obs-" forms, and suggested some language.

> John says he is ok with the text but then proceeds to explain that he 

I don't think that's true. John says he agrees with Keith's addition and 
gives some slightly different proposed wording for that addition. He 
then suggests that I simply have a go at integrating the addition, 
which, as Todd says, doesn't disagree with anything he wrote.

> Keith expresses a desire for you to do separate wordsmithing.

I would rephrase to say, Keith expresses agreement that I integrate the 
additional text.

> That is 4 people (but arguably 5) like an existing set of words.

Yep, I like them too. And save the word "the", and adding an additional 
example, two others seem to like them too.

> clear sense of deficiencies in the existing, proposed text.

They did give a clear description of the deficiency (identifying 
archives as the only reason), and provided text to clarify.

> And the conclusion is that you should do separate wordsmithing?

My conclusion was that integrating Keith and/or John's examples was 
something I was able to figure out without substantively changing 
anything Todd wrote and that I was willing to do that.

I hope that makes my intent clear.

There's no action you asked me to take or not take in your message, so I 
will go forward with my plan unless you or others say otherwise.

pr

---

From: Dave Crocker <[email-masked]>
Date: Fri, 7 May 2021 10:39:40 -0700

On 5/7/2021 10:36 AM, Pete Resnick wrote:
> 
> I hope Todd's followup message helped to unconfuse you,

sure.

d/

---

From: Ned Freed <[email-masked]>
Date: Fri, 07 May 2021 08:15:38 -0700 (PDT)

+1

				Ned