Subject: Re: #1080 - MIME parameters for Injection-Info and Archive header field need more text/updated syntax

From: "Charles Lindsey" <[email-masked]>
Date: Mon, 25 Jul 2005 11:49:32 GMT

This ticket was created by Alexey, but not announced to this list. For the
full text, see https://rt.psg.com/Ticket/Display.html?id=1080.

However, that text is a little dated, and the problem has already been
mentioned on this list. Essentially, it is that the current syntax
introducing the various <parameter>s in Injection-Info fails to account
for various features of RFC 2045, for example the necessity to encapsulate
in a <quoted-string>.

Also, the problem no longer arises with the Archive header field, since we
no longer define the "filename" parameter there.

Here is a proposed text to replace the present start of USEFOR 3.2.14:

   injection-info  =  "Injection-Info:" SP [CFWS] path-identity
                      [CFWS] *( ";" parameter ) CRLF

     NOTE: The syntax of <parameter> ([RFC2045] as amended by
     [RFC2231]), taken in conjunction with the folding rules of [RFC
     822], effectively allows [CFWS] to occur both before and after the
     <parameter>, and also on either side of its "=".

   This standard defines <parameter>s for use with this header field as
   listed in the following table. At most one occurrence of each is
   allowed.

   <attribute>                           syntax of <value>
   -------------------------------------------------------

   "posting-host"                        <host-value>
   "posting-account"                     any <value>
   "sender"                              <sender-value>
   "logging-data"                        any <value>
   "mail-complaints-to"                  <address-list>

   where

   host-value      =  dot-atom / [ dot-atom ":" ]
                      ( IPv4address / IPv6address ) ;  see [RFC 3986]
   sender-value    =  mailbox / "verified"

     NOTE: Since each <host-value>, <sender-value> or <address-list>
     also has to be a syntactically correct <value>, it will often be
     necessary to encapsulate is as a <quoted-string>, for example:

     sender = "\"Joe Q. Public\" <[email-masked]>"

   Additionally, <parameter>s whose <attribute> starts with "x-" MAY be
   used where the defined ones appear to be insufficient, but other
   unlisted <parameter>s SHOULD NOT be used unless defined in extensions
   to this standard.

     NOTE: Should Non-ASCII characters be required in any <value>, the
     mechanisms described in Section 4 of [RFC 2231] are available.
     However, it is unlikely that the more gibbous mechanisms of Section
     3 will be needed, given the possibility of folding within
     <quoted-string>s and the lack of any limit on the length of a
     header line short of the maximum 998 characters.

If that last paragraph is a bit too gibbous for your taste, no huge harm
would arise from omitting it.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

---

From: Frank Ellermann <[email-masked]>
Date: Tue, 26 Jul 2005 09:23:21 +0200

Charles Lindsey wrote:

> https://rt.psg.com/Ticket/Display.html?id=1080.

Apparently down.

>                    [CFWS] *( ";" parameter ) RLF
 
> [RFC2231]), taken in conjunction with the folding rules of
> [RFC 822], effectively allows [CFWS] to occur both before
> and after the <parameter>, and also on either side of its "="

None of these parameters cries for I18N, therefore I propose 
to use the SPF syntax <key-value-list> bypassing <parameter>.

>              ( IPv4address / IPv6address ) ;  see [RFC 3986]

How about <dot-atom-text> here ?  Unless you want the [CFWS]
of <dot-atom> for consistency with other alues.  It's better
readable if you use groupings as recommended by 2234(bis):

  host-value = dot-atom / 
               ( [dot-atom ":"] (IPv4address / IPv6address) )

It doesn't have square brackets this way, we could roll our
own IPaddress for this purpose:

  host-value = dot-atom / ( [dot-atom ":"] "[" IP-address "]" )
  IP-address = ( IPv4address / IPv6address ) ;  see [RFC 3986]

> sender-value    =  mailbox / "verified"

If you want to inherit 2822 [CFWS] for all values, you should
also allow it for "verified":

  sender-value    =  mailbox / ( [CFWS] "verified" [CFWS] )

>      sender = "\"Joe Q. Public\" <[email-masked]>"

Ugh.  We end up with <quoted-string>s within a <quoted-string>
and stuff like sender = "<\"\\\\\"@example.com>".

I've no idea how to avoid it... :-(  This *NIXism has to be
explained, maybe a second example, <"\\"@example.com> results
in sender = "<\"\\\\\"@example.com>".

> NOTE: Should Non-ASCII characters be required in any <value>,
> the mechanisms described in Section 4 of [RFC 2231] are
> available.

That would be very confusing for <mailbox>, where Section 5 or
rather 2047 could also do the trick.  Get rid of <parameter>,
I really don't like it.

> However, it is unlikely that the more gibbous mechanisms of
> Section 3 will be needed

That doesn't help.  An implementor can't ignore 2231 Section 3
based on "unlikely" and "gibbous".  Avoid <parameter> at all
costs, even dropping the complete Injection-Info is an option
if <key-value-pair> doesn't work.  We don't need I18N in the
Injection-Info beyond 2047.
                            Bye, Frank

---

From: Bruce Lilly <[email-masked]>
Date: Tue, 26 Jul 2005 08:52:22 -0400

> > and after the <parameter>, and also on either side of its "="

Add reference to the Errata page, s/822/2822/.
 
> to use the SPF syntax <key-value-list> bypassing <parameter>.

SPF = not a standard of any kind, certainly not a normative
reference (and cannot be because it would be an improper
downward reference), and SPF is HARMFUL.

RFC 2047/2231/errata isn't solely about i18n, it also addresses
long parameters which may need to be split and language-tagging.

> >              ( IPv4address / IPv6address ) ;  see [RFC 3986]

A host name can be up to 255 octets, and there are recommended line
length limits << 255.

> > sender-value    =  mailbox / "verified"

A mailbox can be >> 255 octets (local-part, '<', '@', '>', CFWS in
addition to domain name).
 
> rather 2047 could also do the trick.

No. RFC 2047 is applicable in precisely 3 contexts: an unstructured
field (not applicable to this case), in a comment in a structured
field (also not applicable in this case), and as a word in a phrase
in a structured field (possibly applicable for a display name, but
not for local-part or domain [of course, the domain should use the
on-the-wire IDN format which is comprised solely of letters, digits
and hyphens plus dots; local-parts remain to be specified]).

and Frank remarked:

> An implementor can't ignore 2231 Section 3
> based on "unlikely" and "gibbous".

True.

> Avoid <parameter> at all 
> if <key-value-pair> doesn't work.

There was a proposal to use a parameter-less (Received-like) syntax
some time ago, but there was no traction.  Feel free to try that
again, Frank, if you think it will fly now.

> We don't need I18N in the 

"any <value>" might conceivably need i18n, as well as splitting of long
values and/or language-tagging.  The length issues related to mailbox
are even wore germane to address-list, as a single address can be a
named group, and an address-list can contain multiple comma-separated
addresses, possibly with interspersed CFWS.

---

From: Frank Ellermann <[email-masked]>
Date: Tue, 26 Jul 2005 17:14:21 +0200

> Add reference to the Errata page, s/822/2822/.

It's about 822, we just discussed these issues on the 822 list,
2231 and 2045 were written before 2822.

>> to use the SPF syntax <key-value-list> bypassing <parameter>

> reference

I'm not talking about references, what a strange idea, neither
normative nor informative, and also no acknowledgements.  It's
just a place to grab the <key-value-list> idea, and use it for
the Injection-Info values.

> SPF is HARMFUL.

For spammers, they feel forced to respect it.  Poor bastards.

> RFC 2047/2231/errata isn't solely about i18n, it also
> addresses long parameters which may need to be split and
> language-tagging.
[...]
> A host name can be up to 255 octets, and there are
> recommended line length limits << 255.

FQDNs are used everywhere in header fields, why should that be
different for the posting-host value ?

You only need 2231 folding where embedded WSP can be sigificant
(file names) or when you hit the 997 limit (excl. URLs, where
you can get away with dummy inserted FWS by just specifying it)

> A mailbox can be >> 255 octets (local-part, '<', '@', '>',
> CFWS in addition to domain name).

No WSP delimited part exceeds 997, otherwise it won't work in
header fields like From, Sender, or Reply-To.  Where does this
magic 255 come from, AFAIK not 2045 / 2231 / 2822, and if you
are a Pascal-fan I'd be surprised.

>>> Should Non-ASCII characters be required in any <value>,

>> That would be very confusing for <mailbox>, where Section 5
>> or rather 2047 could also do the trick.

> No. RFC 2047 is applicable in precisely 3 contexts: an
> unstructured field (not applicable to this case), in a
> comment in a structured field (also not applicable in this
> case), and as a word in a phrase in a structured field
> (possibly applicable for a display name, but not for
> local-part or domain

Only a display name.  Local part or domain are plain US ASCII.

Just take the complete valid <mailbox> and stuff it as is into
the value of the sender-parameter, ready.  A <mailbox> contains
at least "@", therefore this value has to be a quoted string.

It starts with a DQUOTE, and it ends with an unescaped DQUOTE.
Any DQUOTE within it has to be escaped (\"), therefore also
all backslashes within it have to be escaped (\\).  Other stuff
works as is, insert FWS when necessary and where it's allowed.

While a formerly encoded word sits within a quoted string it is
no more encoded word, it's just ASCII with peculiar "?" and "="
characters.  After you have upacked it (undoing any escapes) it
is again a valid encoded word.  Assuming you started with a
valid encoded word, it's a garbage in / garbage out procedure.

Where's the problem ?  If that's too shaky for your tastes we
could restrict the sender-value to <addr-spec>, nobody needs
any <display-name> in the Injection-Info.

> local-parts remain to be specified

They are already specified, but not in RfC 3696 chapter 3.

> There was a proposal to use a parameter-less (Received-like)
> syntax some time ago, but there was no traction.

Yes, the stuff copied from the weird Received-SPF header field.

> Feel free to try that again, Frank, if you think it will fly
> now.

 injection-info = "Injection-Info:" SP [CFWS] path-identity
                  [CFWS inj-value-list] CRLF

 inj-value-list = inj-value-pair
                  *( ";" [CFWS] inj-value-pair ) [";"]

 inj-value-pair = inj-key [CFWS] "=" (dot-atom / quoted-string)

 inj-key        = "posting-host" / "posting-account" /
                  "sender" / "logging-data" /
                  "mail-complaints-to" /
                  ( "x-" *(ALPHA / DIGIT / "-") ALPHA / DIGIT )

 inj-posting-host = <etc. as in usefor-05, names starting with
                    "inj-" followed by the name of the inj-key>

> The length issues related to mailbox are even wore germane to
> address-list, as a single address can be a named group, and
> an address-list can contain multiple comma-separated

Nothing forces you to put a value in one line, or to fold it
at places where it can't be folded.  The normal folding works:

[...] mail-complaints-to = "
 insert folded escaped address-list here
 ";sender = "
 insert folded escaped mailbox here
 " ;x-foo = bar

As soon as there's an optional [FWS] at both ends it should
work.  A less ugly style:

[...] mail-complaints-to =
 "insert folded escaped
 address-list
 between DQUOTEs"; sender =
 "insert folded escaped
 mailbox between DQUOTEs"; x-foo = bar

We'd be in trouble if logging-data or posting-account exceeds
995 characters, but otherwise I don't see the problem.  Bye

---

From: "Charles Lindsey" <[email-masked]>
Date: Wed, 27 Jul 2005 11:13:17 GMT

>> https://rt.psg.com/Ticket/Display.html?id=1080.

Note that the intent of this ticket is to arrive at a way of setting out
the structure of the Injection-Info header field in USEFOR that makes it
clear that it follows all the rules established by RFC 2045/2231.

It is NOT intended to challenge what I think is the (possibly reluctant)
consenus that these things are going to look and behave just like MIME
<parameter>s. If you want to do that, then start another ticket for it;
but we have already been around that route.

So, in particular, I made no change to the existing definitions of
<host-value> and <sender-value), though your suggestions concerning them
look interesting.

>of <dot-atom> for consistency with other alues.

Yes, I might buy that idea - it would certainly reduce unnecessary
clutter.

>  It's better

Yes, I just copied Ken's ABNF as written. I have changed my working copy
of this text.

>also allow it for "verified":

More clutter. Better to s/mailbox/addr-spec/, though even that still
leaves the problem.

>and stuff like sender = "<\"\\\\\"@example.com>".

Yes, I put the example there to warn people. But the whole of USEFOR is
written on the (unlikely) assumption that all readers have a perfect
understanding of RFC 2822, and it would spoil the fun to give them too
many clues :-( .

>rather 2047 could also do the trick.

Actually, it was more "posting account" and "logging data" that I had in
mind. And it is not at all clear that you are allowed to use 2047 stuff
inside a <phrase> that is inside a <parameter>.

>based on "unlikely" and "gibbous".

This header will be generated automatically by injecting agents, who can
easily avoid the Section 3 stuff if they put their minds to it. Hence it
is most "unlikely" that they will find a case where they really need it.

As for decoding it in the unlikely event that it does happen, this will
mostly fall to humans or ad hoc scripts. The humans will be mightily miffed
and send rude complaints to the perpetrators. The ad hoc scripts will
probably just ignore those cases and report to their human masters (q.v.).

---

From: Frank Ellermann <[email-masked]>
Date: Wed, 27 Jul 2005 14:45:41 +0200

Charles Lindsey wrote:

> the intent of this ticket is to arrive at a way of setting
> out the structure of the Injection-Info header field in
> USEFOR that makes it clear that it follows all the rules
> established by RFC 2045/2231.

> It is NOT intended to challenge what I think is the (possibly
> reluctant) consenus that these things are going to look and
> behave just like MIME <parameter>s. If you want to do that,
> then start another ticket for it;

Two tickets for the same issue make no sense, something with
the Injection-Info is not yet ready.

It can be solved somehow with 2231-parameters.  I doubt that it
will get many "fans" (e.g. scripts trying to extract the abuse
address).  It can be solved somehow with key-value-pairs, but
Bruce doubts that folding values only because they "happen" to
allow the insertion of FWS somewhere is sound engineering.

He has a point there.  Specifying why it works for the special
cases of mailbox and address-list is an odd idea.  Admittedly
a _bad_ idea.  Not worse than 2231, but I'm not sure that it's
really better.  If someboday adds an x-filename parameter just
for the fun of it, it breaks without 2231-parameters... :-(

Or Russ' x-complaintsurl, works with 2231, could work without,
but not without an explicit rule to add / remove FWS if needed.

> we have already been around that route.

Yes, the same Russ said that he hates CFWS and 2231 (or that's
how I remember it).

> Better to s/mailbox/addr-spec/, though even that still
> leaves the problem.

At least it eliminates the unnecessary <display-name>, that's
the job of From: or Sender:, not an Injection-Info: parameter.

>> stuff like sender = "<\"\\\\\"@example.com>".
> Yes, I put the example there to warn people.

Your example doesn't include \, and for readers without some
background knowledge with *NIX shells or similar tools it's
not necessarily obvious that they've to escape both " and \.

Use a better example, e.g. something from RfC 3696 chapter 3:

             Fred\ [email-masked]

That's AFAIK wrong, it should be:  "Fred\ Bloggs"@example.com
resulting in a: sender = "\"Fred\\ Bloggs\"@example.com"
Add both forms, original address and correponding sender value.

> USEFOR is written on the (unlikely) assumption that all
> readers have a perfect understanding of RFC 2822, and it
> would spoil the fun to give them too many clues :-( .

Nowhere in 2822 did "you" (see the list of 2822 credits) try
to put a <quoted-string> into another <quoted-string>.  This
madness is a USEFOR-speciality, because unlike 2822 we don't
have the good sense to be independent of MIME.  It's 100% our
fault when we try this stunt.

> And it is not at all clear that you are allowed to use 2047
> stuff inside a <phrase> that is inside a <parameter>.

Any legal <quoted-string> is okay, there's no rule that it
cannot contain some =?..?..?..?= sub-strings.  They are no
<encoded-word>s within a <quoted-string>.  And the sub-string
\"Fred\\ Bloggs\" within a <quoted-string> is also not the
original <local-part> "Fred\ Bloggs", but it's legal and okay.

>> An implementor can't ignore 2231 Section 3 based on
>> "unlikely" and "gibbous".

> This header will be generated automatically by injecting
> agents, who can easily avoid the Section 3 stuff if they put
> their minds to it.  Hence it is most "unlikely" that they
> will find a case where they really need it.

Nevertheless it's not illegal, an implementor trying to extract
the abuse address has to get it right, funny remarks why 90% of
the necessary code are "unlikely" or "gibbous" wont't speed up
the conformance testing by a single man month.

                        Bye, Frank

---

From: "Charles Lindsey" <[email-masked]>
Date: Wed, 27 Jul 2005 11:27:36 GMT

>Add reference to the Errata page, s/822/2822/.

Why? RFC 2045/2231 rely on RFC 822 for their folding rules. That NOTE is
my interpretation of how you would view them in RFC 2822 terms (I also got
Keith Moore to confirm that I had interpreted correctly). The NOTE is not
normative, of course.

>A host name can be up to 255 octets, and there are recommended line

There are no such limits in our drafts. In the unlikely event that such a
host name arises, it is better to leave the long line than to start using
the RFC 2231 splitting.

>True.

False. See my reply to Frank. A generating implementor can easily avoid
Section 3. Accepting implementors don't exist (except for ad hoc scripts).

More to the point is whether you agree that my method of setting out the
structure of Injection-Info achieves its object of integrating these
<parameter>s within the overall structure of MIME parameters as set out in
2045/2231.

---

From: Bruce Lilly <[email-masked]>
Date: Wed, 27 Jul 2005 09:56:38 -0400

> 2231 and 2045 were written before 2822.

822 does not define anything called "CFWS", much less "[CFWS]". Those
are defined in 2822.

> the Injection-Info values.

Since our "document uses a cite by reference methodology, rather than
repeating the contents of other" documents, what precisely are you
suggesting if not a reference?

> For spammers, they feel forced to respect it.  Poor bastards.

No, embraced by spammers, as it lends them an air of legitimacy.  Harmful
to legitimate senders and to recipients; i.e. to the mail system.  As
discussed in detail elsewhere, and which need not be repeated here.
Suffice to say that SPF isn't a good model unless one wants a document
that will be relegated to Experimental status.
 
> different for the posting-host value ?

Parameter values, unlike domains in other parts of structured fields,
must be quoted if folded.
 
> (file names)

No, quoting works for that.

> or when you hit the 997 limit

That is a hard limit; there is also a 78 octet line length recommended
limit (particularly for something which is intended primarily for human
consumption, which seems to be Charles' assertion).

> (excl. URLs, where  

If and only if specified in the (necessarily) structured field syntax.

RFC 2231 fragmentation might not be *necessary* in some circumstances, but
it can still be *used* to split long values.  Without introducing
ambiguity.

> Where does this
> magic 255 come from, AFAIK not 2045 / 2231 / 2822

As we're talking about domain names, see RFC 1034 (sect. 3.1) and RFC 1035
(sect. 2.3.4) as well as RFC 1123 (sect. 2.1).

> Only a display name.  Local part or domain are plain US ASCII.

Domain names are supposed to be LDH, though some IDN advocates claim
that 8-bit stuff (i.e. not encoded) are equally valid [IMO, that's for
display (a la the stuff that results from decoding 2047 encoded-words),
with actual on-the-wire content limited to LDH].  Local-part i18n hasn't
yet been standardized, so one cannot definitively state what constraints
might apply.

> at least "@", therefore this value has to be a quoted string.

Or encoded.  Mere quoting can introduce ambiguities, and does not
address the length problem.
 
> works as is, insert FWS when necessary and where it's allowed.

That relies on particular interpretation by parsers:
  filename="foo bar"
means that the space character is part of the parameter value.  You
seem to want to have spaces (and folding) thrown away in some cases;
how do you propose to distinguish (in a parser) the cases where the
space is an integral part of the value vs. where it should be discarded?
 
> Where's the problem ?

That relies on even more implementation specifics, and is unlikely to
be interoperable; the context in the field is not unstructured, a
comment, or a phrase (it is a parameter value), so 2047 encoding
doesn't apply.
 
> If that's too shaky for your tastes we

That still doesn't address the domain name length issue, or CFWS
permitted in addr-spec vs. interpretation of embedded, quoted spaces.

We could define a structured field that avoids the issues raised by
trying to put such things in parameters.

> They are already specified, but not in RfC 3696 chapter 3.

That Informational document does not specify i18n for local-parts.
 
> Yes, the stuff copied from the weird Received-SPF header field.

No, I was not referring to anything related to SPF cruft.
 
>                   [CFWS inj-value-list] CRLF

So in this case you think that
  Injection-Info: 
   (continuation line) foo.example.com
(i.e. SP on first line followed immediately by line folding) is OK?
Inconsistency detected.
 
>                     "inj-" followed by the name of the inj-key>

That appears something like a parameter list but suffers from several
deficiencies:
1. unlike something based strictly on 2045/2231/errata, there are no
   existing implementations which could be adapted
2. if intended to be human-readable ("to assist in tracing"), it fails
   to provide for specification of charset and language (BCP 18),
   particularly for logging-data and display name parts of address-lists
3. it doesn't address the length issue and related ambiguity regarding
   quoted FWS
4. the optional trailing semicolon is a poor design choice; it raises
   ambiguity about whether the field was prematurely truncated
5. it is poorly specified; "quoted-string" implies one thing whereas
   the details in the usefor specification referenced state syntax
   which is not specifically a quoted-string (nor dot-atom).  See the
   concise and clear syntax of the Received field (recently posted
   separately) for a parameter-less syntax which specifies the necessary
   alternatives (and also provides for a time-stamp).
 
> at places where it can't be folded.  The normal folding works:

See above re. ambiguity of interpretation of quoted SP (and folding)
(also display names which are an integral mandatory part of a named
group).
 
>  insert folded escaped mailbox here

That's begging for interoperability problems.  No existing parameter
in any defined field specifies a value containing an address-list --
for good reasons as noted above.

---

From: Bruce Lilly <[email-masked]>
Date: Wed, 27 Jul 2005 10:25:11 -0400

> resulting in a: sender = "\"Fred\\ Bloggs\"@example.com"

See the errata page, http://www.rfc-editor.org/cgi-bin/errata.pl

> the conformance testing by a single man month.

An implementer of a parser necessarily must handle everything that
is legal to generate.  Display names in a parameter has no precedent;
it is uncharted territory and is prone to interoperability problems.
Unlike an earlier proposal for a Complaints-To field with a URI (no
parameters, and mailto URIs are widely implemented).

---

From: Frank Ellermann <[email-masked]>
Date: Thu, 28 Jul 2005 07:48:22 +0200

[...]
> See the errata page,
> http://www.rfc-editor.org/cgi-bin/errata.pl

Oh, John fixed it less than three weeks ago, good.  Apparently
it still works if the author submits errata.  Makes sense, bye

---

From: Frank Ellermann <[email-masked]>
Date: Thu, 28 Jul 2005 09:33:41 +0200

Charles Lindsey wrote:

> More to the point is whether you agree that my method of
> setting out the structure of Injection-Info achieves its
> object of integrating these <parameter>s within the overall
> structure of MIME parameters as set out in 2045/2231.

Let's finish that business, state of the art from my POV:

- <inj-key-value-list> was rejected => 2231-parameters

- replace <mailbox> by <addr-spec> for the sender
- maybe replace <address-list> by a new <addr-spec-list>
  (we need no <display-name> or <group>, we have CFWS)

- maybe replace <dot-atom> by <dot-atom-text> for the host
- use 2234 grouping for "host colon IP" (already agreed)
- do we want the IP in square brackets ?  Copy whatever a
  2821 Received header field does in a "from"-value.

- remove nonsense remarks about "unlikely" and "gibbous"
- remove useless remarks about I18N and non-ASCII, those
  who want it will find it in 2231 and 3066bis
- update the example to show both " => \" and \ => \\,
  add another example showing @ => must be quoted string

- declare #1080 state "text proposed" and let's move on

                            Bye, Frank

---

From: Frank Ellermann <[email-masked]>
Date: Thu, 28 Jul 2005 10:32:20 +0200

Bruce Lilly wrote:

> 822 does not define anything called "CFWS", much less
> "[CFWS]". Those are defined in 2822.

Please propose a complete Injection-Info, I've just retracted
my proposal.  TTBOMK we inherit tons of CFWS from 2822, incl.
leading and trailing [CFWS] for several values.  We could add
it everywhere for consistency, or remove it everywhere if that
is possible.

Don't forget [CFWS] before and after "verified" in your idea,
and everywhere between the name and the "=".

> Since our "document uses a cite by reference methodology,
> rather than repeating the contents of other" documents, what
> precisely are you suggesting if not a reference?

Anything that's not handled by reference (or differs) has to
be specified, by copying similar constructs like the RfC 3834
Auto-Submitted etc.

==============================================================
 [OT SPF issues]
> No, embraced by spammers

Instruct Ned's spammer, apparently a hard case of rule #3, it's
not yet completely "adopted" by spammers unfortunately (where
adoption means stop forging)

> Harmful to legitimate senders

Hogwash, based on 1,000 backscatter mails per day for 6 months
in 2004 it saved me from giving up "my" xyzzy vanity host, and
if you ever changed your primary address you know what that's
about.  Now 11 months without this backscatter, if you ever had
a modem you might also know why I couldn't afford to handle the
about 11 * 30 * 1000 = 330,000 backscatter mails.

> As discussed in detail elsewhere, and which need not be
> repeated here.

You spread FUD in public, I answer in public.  I have a FAIL
policy for about 15 months now.

> Suffice to say that SPF isn't a good model unless one wants
> a document that will be relegated to Experimental status.

1.000.000 domains (not my estimate) or 85% of all mails (also
not my estimate, I only contributed to reduce this from 95% to
85%) are no "experiment".  That's only a trick to abuse these
policies for another incompatible "anti-phishing" scheme.

The latter is really "experimental".  They found no volunteers,
that's why they want to steal the installed base, and needed
the IESG to "legalize" this blatant abuse as an "experiment".
==============================================================

>> FQDNs are used everywhere in header fields, why should that
>> be different for the posting-host value ?

> Parameter values, unlike domains in other parts of structured
> fields, must be quoted if folded.

Yes, but there are many [FWS] in the unquoted version, just fold
it as needed, and then quote it with embedded FWS:

   foo = "b
 ar"

> No, quoting works for that.

In file names WSP is sigificant, you cannot freely replace SP by
FWS, or insert FWS whereeveer you need it.  Therefore you have to
use enumerated quoted strings:

    foo*0 = "b"
    foo*1 = "ar"

> RFC 2231 fragmentation might not be *necessary* in some circumstances,
> but it can still be *used* to split long values.  Without introducing
> ambiguity.

Okay, let's say I buy one "bar", if we'll have security considerations
as proposed elsewhere.  Not "unlikely" plus "gibbous", it's a real PITA.

> As we're talking about domain names, see RFC 1034 (sect. 3.1) and RFC
> 1035 (sect. 2.3.4) as well as RFC 1123 (sect. 2.1).

I'm pretty sure that these standards don't talk about >> 255 for FQDNs.
Forget it, 255 is nothing we need to worry about, and 998 > 255 is
clear for all implementors.

> some IDN advocates claim hat 8-bit stuff (i.e. not encoded) are
> equally> valid

We're not doing I18N for NetNews, the old usefor-07 failed, that idea
was dropped and left as an exercise for a future USEFOR2 WG.  It's a
good strategy, officially we're updating 1036, repeating the complete
Internet history of two decades in one giant jump cannot work.  Some
of us are wooried about UUCP hosts "wolf" and "cafe", others have
serious questions about a "Subject: cmsg", adding I18N is no option,
let's forget it.

> Or encoded.

ACK, I tend to forget this because nobody uses it (and I would see
it, my UA doesn't support 2231):

    title*0*=us-ascii'en'This%20is%20even%20more%20
    title*1*=%2A%2A%2Afun%2A%2A%2A%20
    title*2="isn't it!"

>   filename="foo bar"

ACK, see above and in another article.  That won't fly with my ad-hoc
idea <key-value-pair> instead of <parameter>.

> We could define a structured field that avoids the issues raised
> by trying to put such things in parameters.

If you have a good idea _without_ 2231 I'm very interested.

> (i.e. SP on first line followed immediately by line folding) is OK?

Rough WG consensus, some proposals to fix [CFWS] were rejected.
It is addressed in prose, for any trailing [CFWS] even in 2822.

> 1. unlike something based strictly on 2045/2231/errata, there are
>    no existing implementations which could be adapted

It's the same as 2045 without 2231 (or that was the design idea,
my bad if I didn't get it right).

> 2. if intended to be human-readable ("to assist in tracing"), it
>    fails to provide for specification of charset and language (BCP
>    18), particularly for logging-data

The logging-data is readable for those who insert it, for everybody
else it's opaque.

>    and display name parts of address-lists

Sub-strings reflecting 2047-words are allowed, and the display-name
is irrelevant at best.

> 3.

ACK, see above, x-filename won't work in all weird cases.

> 4. the optional trailing semicolon is a poor design choice

Yes, screw it, I copied the SPF syntax, or as you said "SPF-cruft".

> See the concise and clear syntax of the Received field (recently
> posted separately) for a parameter-less syntax

Yes, I saw it.  It was the 2822 view, not the 2821 view.  Here we
need both, but spliiting these views is a good idea.  Just propose
something.  Complete ABNF for both views if it avoids 2231.  If it
doesn't we already have Charles' version plus some minor nits.  Bye

---

From: "Charles Lindsey" <[email-masked]>
Date: Thu, 28 Jul 2005 13:47:44 GMT

>Yes, the same Russ said that he hates CFWS and 2231 (or that's

Well we all hate 2231, but Russ did also say that he was no longer going
to oppose the use of MIME-style parameters in Injection-Info, and I think
that sealed the consensus to which we are now working.

Also, we agreed to do away with Complaints-To, and to stick with just mail
addresses for complaints (with room for a future extension parameter for
other possibilities). So I don't think it helps much for people to keep
going back over that ground again.

---

From: Bruce Lilly <[email-masked]>
Date: Fri, 29 Jul 2005 10:47:46 -0400

On Thu July 28 2005 04:32, Frank Ellermann wrote:
> my proposal.

I proposed something over a year ago, based on the Received field syntax.
If you want ABNF, take the 2822 Received field syntax, change the field
name and optionally drop the semicolon and timestamp.

It's been discussed off-list with the WG Chair; at last discussion
some months ago, it didn't seem worth pursuing.

> be specified, by copying similar constructs like the RfC 3834

Since we're committed (sect. 1.6) not to rewrite syntax, that leaves
a reference citation.  Incidentally, 3834 uses MIME 2045/2231/errata
syntax.

[dropping off-topic stuff because it's off-topic and shouldn't have
been mentioned in the first place]
 
> it as needed, and then quote it with embedded FWS:

Not likely to be interoperable. Please reread RFCs 2017, 2045, and 2231.
Note that 2017 applies to a specific parameter to a specific field when
used with a specific media type, and imposes several special constraints
(e.g. 40-character limit).  Note that 2231 provides an alternate,
uniform mechanism for handling long parameters and gives an example which
precisely covers the RFC 2017 case.

> FWS, or insert FWS whereeveer you need it.

Nor can one do so in any other parameter without specific specification
of such syntax (see RFC 2017 for an example).

> Therefore you have to 

That is quite different from your earlier example, which contained
embedded CR LF SP in the parameter value.

> I'm pretty sure that these standards don't talk about >> 255 for FQDNs.

You asked where the 255 number came from.  It's the limit on a fully-
qualified domain name length.

> clear for all implementors.

255 >> 78, and RFC 2822 gives a recommended maximum line length of 78
octets per line (not including CRLF) [we cite 2822 as normative].  If
you want to use 2017-like syntax, 255 >> 40.
 
> We're not doing I18N for

newsgroup names.  We address I18N via MIME for suitable header field
content (unstructured fields, words in phrases, in comments).  I18N
for domain names has been specified and applies to domain names where
they may appear (in LDH format).

> If you have a good idea _without_ 2231 I'm very interested.

See above (and 2822 ABNF for Received).
 
> It is addressed in prose, for any trailing [CFWS] even in 2822.

No, there's no whitespace-only continuation line there; all perfectly
valid per 2822.  But there's only an SP after the colon on the first
line, in accordance with your proposed ABNF but contradicting the
unjustified BCP 14 imperative mandated non-whitespace on every line.
 
> my bad if I didn't get it right).

See 2017 for the additional things that need to be specified for
implementations to discard embedded FWS.  That would of course
require new code in implementations.
 
> The logging-data is readable for those who insert it, for everybody

Not much point in transmitting it then; leave it in logs on the
originating system, referenced by message-id and/or host plus timestamp.
 
> Sub-strings reflecting 2047-words are allowed,

Not clear; would have to be quoted, and encoded-words aren't allowed in
quotes-strings.  Not likely to be interoperable.

> and the display-name 

It's specified in mailbox, address, address-list...
 
> need both,

No, 2821 is specific to SMTP; irrelevant to our work.

---

From: "Charles Lindsey" <[email-masked]>
Date: Fri, 29 Jul 2005 14:23:00 GMT

>- replace <mailbox> by <addr-spec> for the sender

I might buy that.

>  (we need no <display-name> or <group>, we have CFWS)

But that is departing from the usual syntax as use in Reply-To and the
like. And a <display -name> ("AOL Abuse Desk") could actually be useful.
Moreover, it is useful to have a few folding points in it somewhere, just
in case the <address-list> get inconveniently long.

>- maybe replace <dot-atom> by <dot-atom-text> for the host

Yes, I think so

>  2821 Received header field does in a "from"-value.

No, do whatever is currently done in NNTP-Posting-Host, which is no square
brackets AFAIK (also that is our present syntax).

>- remove nonsense remarks about "unlikely" and "gibbous"

If people want that.

>  who want it will find it in 2231 and 3066bis

Maybe. It is a bit more useful that the line-folding gibbousness.

>  add another example showing @ => must be quoted string

Those examples are just plain ugly. At least the "Joe Q. Public" is a case
that might actually arise, and one where your average reader can probably
figure out what problem you are trying to solve.

Anyway, nobody has spoken against my basic methodology, so that seems
sort of accepted.

---

From: Frank Ellermann <[email-masked]>
Date: Fri, 29 Jul 2005 19:42:48 +0200

> No, do whatever is currently done in NNTP-Posting-Host,
> which is no square brackets AFAIK (also that is our present
> syntax).

Okay.  Reason I stumbled over it:  It starts (optionally) with
<host> ":" followed by the IP.  Hard to read for an IPv6, e.g.
need.more.cafe:cafe:cafe::cafe for a future gTLD cafe.  OTOH
need.more.cafe:[cafe:cafe::cafe] could be clearer.

>> - remove nonsense remarks about "unlikely" and "gibbous"

Necessity, implementor's POV.  You can't suggest in a standard
(between the lines) that it will be never as bad as it is, in
reality it will be even worse, all those strange errors.

> Those examples are just plain ugly.

I took it from 3696 (or now its errata as Bruce pointed out),
maybe you see a better example.  IMHO we need two cases, a nice
address to show "needs quotes because it has an '@'", and an
ugly case showing how to escape quotes and backslash.

> nobody has spoken against my basic methodology, so that seems
> sort of accepted.

Reluctantly, yes, unless Bruce finds a way around 2231 based on
his "do it like Received" idea.  Still hoping...  Bye, Frank

---

From: Frank Ellermann <[email-masked]>
Date: Fri, 29 Jul 2005 20:40:41 +0200

> I proposed something over a year ago, based on the Received

Did I miss, forget it, or was it incomplete ?  Your article
some days ago was "only" the general structure, but for the
Injection-Info we need also the specific details, the most
complex part is probably mail-complaints-to.

Can't judge it without seeing it, e.g. how you want to say that
joe sender is an invalid display name because it contains the
keyword sender (or similar).

> It's been discussed off-list with the WG Chair; at last
> discussion some months ago, it didn't seem worth pursuing.

You see Charles' proposal, if you really have something better
please post it.

> [dropping off-topic stuff because it's off-topic and
>  shouldn't have been mentioned in the first place]

ACK, and after all it didn't work as a method to avoid the
2231-parameter idea in Injection-Info, I skip also this stuff.

> Please reread RFCs 2017, 2045, and 2231.

Come on, I did that more than a year ago after you told me so,
and more than once.

> (e.g. 40-character limit).

BTW unnecessary, I hate new magic numbers without good reasons.
 
> You asked where the 255 number came from.  It's the limit on
> a fully-qualified domain name length.

Yes, I didn't get your point about 255 > 78, because 255 < 998.
The 78 is "only" a SHOULD, I'd ignore it e.g. for long URLs in
the body of an article.

>> The logging-data is readable for those who insert it, for
>> everybody else it's opaque.
 
> Not much point in transmitting it then; leave it in logs on
> the originating system, referenced by message-id and/or host
> plus timestamp.

If servers can solve it _without_ local data they'll do it, I
guess that's the idea of the logging-data.  Not only technical
reasons, also legal issues (privacy, any log file that doesn't
exist can't violate a law or similar problems)

> Not clear; would have to be quoted,

Yes. = and ? are no specials, but if there's a blank => quoted.

> and encoded-words aren't allowed in quotes-strings.

No, = and ? are allowed in quoted-strings, in any combination.

>> It was the 2822 view, not the 2821 view.  Here we need both,

See the first lines for what I meant, your propsal has also to
cover the details, like 2821 does with "for", "via", "by", etc,
- but here it would be "sender", "logging-data", etc.  Bye