Subject: [Emailcore] Navigating 5321bis (and predecessors)

From: John C Klensin <[email-masked]>
Date: Fri, 01 Jan 2021 15:18:28 -0500

Hi.

In trying to look for some other things today, I was reminded of
an issue that I think we should be clear about.   RFC 2821, and
hence 5321 and, in its present form, 5321bis, are badly
organized and show the results of basically being a merge of
three separate documents written at very different times.  Those
documents -- and consequently different sections of 2821 and
5321 -- depending a bit on how you count, show the different
writing styles and perhaps the motivations of as many as seven
different authors.  The result is that it is over-long and it is
hard to find things.  As a specific example from yesterday, Dave
wrote

	"with apologies to all, for my reading comprehension,
	limitations, but would someone point to the place in RFC
	5321 where the semantics of the FOR clause are
	explained, and especially in terms of it's coming from a
	RCPT TO?"

The implicit question of whether that portion of the
specification appears and where to find it turns out to be hard
to answer, much harder than it ideally should be, and that
problem is by no means limited to a particular clause in a
particular trace header.

We ended up in this uncomfortable situation because of a
decision at the time of DRUMS, reaffirmed at the time of YAM,
that the disadvantages of doing things that way were outweighed
by the risks of making inadvertent and problematic changes in
the process of a rewrite.  It doesn't make much difference then
and shouldn't now, but I came to conclude (along with many
others) that it was the right decision even though I hated it
then, still do, and my dislike gets stronger every time someone
says "I dare you to find where it says that" or "probably this
paragraph should be somewhere else".

I'm adding a warning note about this to 5321bis-02.  The WG
should decide whether it should be rewritten or removed prior to
Last Call or publication.

Two questions I'd like WG participants to think about:

(1) How much reorganizing and rewriting do we want to do and
what are our stopping rules?  Moving a paragraph from one place
to another (as in Erratum 1851; Ticket #28) seems rather easy
and safe.  Changing section titles and/or rewriting their
introductory paragraphs may be a bit more risky.  Or, in
principle, we could reverse the decision above and start on a
large-scale rewrite.  WHatever we choose, I think it would help
to have some sort of consensus about our target sooner rather
than later.

(2) Are there ways that we can mitigate the problems with
finding things.  The index in 5321 (and carried forward into
5321bis) was created as part of a compromise with the IESG after
the Last Call closed on what became 5321.  Because of how <iref>
elements are handled in xml2rfc v2, those index entries should
up as pointing to page numbers (as do the index entries in RFC
7749.  Presumably xml2rfc v3 will show section numbers.  Do we
want to retain the index?  If so, are there additional things
that could and should be indexed?   In particular, it occurs to
me that, with some hours of work, I could index keywords from
section titles.  Would that be worth doing in terms of finding
information or is the table of contents sufficient?   Other
ideas?

thanks,
   john

---

From: Dave Crocker <[email-masked]>
Date: Fri, 1 Jan 2021 15:56:55 -0800

On 1/1/2021 12:18 PM, John C Klensin wrote:
> what are our stopping rules?

None of the first and as little as is reasonable of the latter.  The 
purpose of the current effort is to expedite elevation to full Standard. 
  Any effort that is 'substantial' puts that at risk.

In the case of the particular issue I raised about 'FOR', it makes sense 
to add some text that defines its semantics.  It doesn't make sense to 
make a major effort of -- or resulting from -- that.

d/

-- 
Dave Crocker
Brandenburg InternetWorking
bbiw.net

---

From: John C Klensin <[email-masked]>
Date: Fri, 01 Jan 2021 21:45:54 -0500

--On Friday, January 1, 2021 15:56 -0800 Dave Crocker
<[email-masked]> wrote:

> None of the first and as little as is reasonable of the
> latter.  The purpose of the current effort is to expedite
> elevation to full Standard.   Any effort that is 'substantial'
> puts that at risk.

Speaking personally, agreed.  However, I note that we have
already moved one paragraph and that counts as "reorganizing",
at least as I used that term.  I would summarize the principle
as "don't open cans of worms", but that interpretation of your
comment has some interesting implications.

> In the case of the particular issue I raised about 'FOR', it
> makes sense to add some text that defines its semantics.  It
> doesn't make sense to make a major effort of -- or resulting
> from -- that.

An interesting example because, as soon as we try to pin down
the semantics, we probably should address the question of
whether, in today's privacy-sensitive world, supplying that
clause at all is a good idea [1].   It may be relevant to a
discussion of that topic that, while hop-by-hop transport
encryption would hide that information on the wire, message
headers (including trace fields) need to be in cleartext for
SMTP trace fields/ time stamps to function properly.

To make the situation a little more entertaining, RFC 6409 is
not explicit as to whether the submission server is required to
insert a time-stamp field although the requirement in the third
paragraph of Section 8 certainly implies that.  Because a
submission server is explicitly allowed to resolve local aliases
(important for SMTPUTF8 and I have no idea what people on this
list call others in the privacy of their local environments),
exposing one of those aliases in a FOR field might be a bad idea
even if it had the syntax of a Mailbox name.

In other words, while I would favor putting in some words about
the semantics of the FOR clause (and any other clauses of a
time-stamp field whose semantics are missing or vague), we need
to understand that even something that seemingly trivial might
open one of those cans of worms.

    john

[1] Note that aspect of the FOR clause is already addressed,
albeit with a bit of handwaving, in Section 7.6 _and_ I don't
even know exactly what the last sentence of that section means
given that a given Received: field can only accommodate at most
one FOR clause with exactly one Mailbox (now noted in -02).

---

From: Alessandro Vesely <[email-masked]>
Date: Thu, 7 Jan 2021 10:25:32 +0100

On Fri 01/Jan/2021 21:18:28 +0100 John C Klensin wrote:
> (1) How much reorganizing and rewriting do we want to do and what are our
> stopping rules?

Minimal reorganizing and rewriting, but not less.

> (2) Are there ways that we can mitigate the problems with finding things.
> [...]  Would that be worth doing in terms of finding information or is the
> table of contents sufficient?   Other ideas?

I think an index is useless nowadays, since page numbers are not a univocal 
reference anymore (they only appear in PDFs and are presumably going to depend 
on paper formats.)

Rather, I'd insert [See also <xref>] notes wherever appropriate.

Best
Ale
--

---

From: John C Klensin <[email-masked]>
Date: Sat, 09 Jan 2021 17:24:04 -0500

--On Thursday, January 7, 2021 10:25 +0100 Alessandro Vesely
<[email-masked]> wrote:

>> finding things. [...]  Would that be worth doing in terms of
>> finding information or is the table of contents sufficient?
>> Other ideas?
> 
> I think an index is useless nowadays, since page numbers are
> not a univocal reference anymore (they only appear in PDFs and
> are presumably going to depend on paper formats.)

I hope it is sorted out quickly --certainly before 5321bis is
ready-- but the xml2rfc vocabulary defines index ("<iref>")
elements which presumably means it is someone's problem (not
ours) to implement them in a way that is sensible for all
relevant output formats.  My personal opinion is that will mean
index entries have to end up targeting section numbers, not page
numbers, but I wait to see what others come up with.   However,
unless there is a proposal and community consensus to declare
that markup element useless and/or obsolete, I think we get to
assume that they will work in some meaningful way.

> Rather, I'd insert [See also <xref>] notes wherever

As I think I mentioned in an earlier note, that is already
underway.  I welcome pointers to additional places where such
cross-references would be helpful.

best,
   john

---

From: "John Levine" <[email-masked]>
Date: 10 Jan 2021 03:21:35 -0000

In article <306A848BA9E6BE7CC38465E3@PSB>,
>> are presumably going to depend on paper formats.)

Indices don't have to point to oage numbers.  They can refer to
section numbers or unsing this new-fangled web thing, they can
link directly to the relevant sections.

R's,
John
-- 
Regards,
John Levine, [email-masked], Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. https://jl.ly

---

From: John C Klensin <[email-masked]>
Date: Sat, 09 Jan 2021 22:41:22 -0500

--On Sunday, January 10, 2021 03:21 +0000 John Levine
<[email-masked]> wrote:

>>> not a univocal reference anymore (they only appear in PDFs
>>> and are presumably going to depend on paper formats.)
> 

That is more or less what the portion of that note I actually
wrote (as distinct from the quote from Alessandro above).  Did
that not catch up with you and the list?

    john

---

From: Julian Reschke <[email-masked]>
Date: Sun, 10 Jan 2021 07:26:08 +0100

Am 09.01.2021 um 23:24 schrieb John C Klensin:
> ...

+1, but note <https://trac.tools.ietf.org/tools/xml2rfc/trac/ticket/418>
-- index generation currently is broken in xml2rfc (in v3 mode).

Best regards, Julian

---

From: John C Klensin <[email-masked]>
Date: Sun, 10 Jan 2021 02:03:50 -0500

--On Sunday, January 10, 2021 07:26 +0100 Julian Reschke
<[email-masked]> wrote:

>> relevant output formats.  My personal opinion is that will
>> mean index entries have to end up targeting section numbers,
>> not page numbers, but I wait to see what others come up with.
>> However, unless there is a proposal and community consensus
>> to declare that markup element useless and/or obsolete, I
>> think we get to assume that they will work in some meaningful
>> way.
>> ...
> 
> +1, but note
> <https://trac.tools.ietf.org/tools/xml2rfc/trac/ticket/418>

That had been called to my attention after I noticed that
converting the (v2) xml for draft-ietf-emailcore-rfc5321bis-01
(which carries forward the index from RFC 5321) to v3 produced
perfectly good <iref> entries but that processing it into either
text or PDF produced some rather badly damaged output without
any index targets.

My perspective as editor is that the index was added to RFC 5321
after IETF Last Call and at the insistence of the IESG.  Unless
the IESG wants to undo that decision (and undo it RSN rather
than during or after IETF LC when the document gets there), I
hope the problem is fixed before the WG finishes its work.  The
fix should ideally reference section numbers, not page numbers,
for obvious reasons.  Otherwise either the RPC is going to have
a really interesting problem on its hands or this WG should stop
its work until the tools are ready.

So I agree with your end of December priority change from
"minor" to "major".

    john

---

From: Julian Reschke <[email-masked]>
Date: Sun, 10 Jan 2021 14:09:40 +0100

> link directly to the relevant sections.

Index entries should hyperlink where possible (HTML, PDF).

In unpaginated formats, the displayed value for each link can be the
section number (and is and has been that way in all processors *except*
xml2rfc's v3 mode).

In paginated formats (HTML in paged media mode, PDF), the displayed
value can be indeed a page number.

Best regards, Julian