Subject: [apps-discuss] draft-kucherawy-rfc5451bis

From: "Murray S. Kucherawy" <[email-masked]>
Date: Thu, 14 Mar 2013 15:40:06 -0400

I've uploaded an update to draft-kucherawy-rfc5451bis for review.  It
incorporates the feedback I've received over the last few months since the
last version.

https://datatracker.ietf.org/doc/draft-kucherawy-rfc5451bis/

To recap: RFC5451 defines a standard message header field for recording the
results of things like DKIM and SPF by border MTAs so internal things like
MUAs don't have to repeat those checks.  This is an update draft based on
operational experience since the proposed standard was originally published
in 2009.

So, two things:

1) Reviewers, please!  I think this is pretty much ready to go as it's been
around since last June with some reviews, but nothing lately.  A final
glance by some of the apps community (especially mail people) would be
grand.

2) I'd like to process this through APPSAWG.  Salvatore would have to
shepherd it since I can't shepherd my own document.  Is that a reasonable
path for this work?  There isn't an active working group doing SMTP things,
and we know our ADs would prefer to avoid sponsoring things if possible.

Thanks,
-MSK, participatorially

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Thu, 14 Mar 2013 16:09:04 -0400

Reviewers might find this diff from RFC5451 to the current version to be
helpful:

http://www.blackops.org/~msk/draft-kucherawy-rfc5451bis-from-rfc5451.diff.html

-MSK

On Thu, Mar 14, 2013 at 3:40 PM, Murray S. Kucherawy <[email-masked]>wrote:

> To recap: RFC5451 defines a standard message header field for recording
> the results of things like DKIM and SPF by border MTAs so internal things
> like MUAs don't have to repeat those checks.  This is an update draft based
> on operational experience since the proposed standard was originally
> published in 2009.
>
> So, two things:
>
> 1) Reviewers, please!  I think this is pretty much ready to go as it's
> been around since last June with some reviews, but nothing lately.  A final

---

From: Dave Cridland <[email-masked]>
Date: Fri, 15 Mar 2013 11:34:03 +0000

> https://datatracker.ietf.org/doc/draft-kucherawy-rfc5451bis/
I've not seen this document before, nor its predecessor, so I'm casting a
fresh pair of eyes on it. This may result in my not having understood the
draft, but I'll pretend that's the draft's fault rather than it possibly
being my own idiocy.

For the most part, this document looks fine and solid.

Two items concern me in a major way; one further item has me mildly nervous.

First, the minor one:

The header includes definition of a version ABNF production optionally
following the authentication server's identity. It took me some time to
find the note of how to treat this; I didn't see any examples with it
present either.

Second, the first major one:

The examples tend to imply, for the most part, a canonical form to the
header, which places CFWS is only particular places. In particular, all the
examples are of the form auth-srv-id "; " method "=" result, with some
optional properties and/or reason, with one single exception (which
includes a comment in lieu of a reason). It's not immediately apparent to a
comparatively novice reader that the following header field is valid:

Authentication-Results: foo.example.net (My mailserver) / (I am number) 1
(You see) ; dkim (Because I like it) /2 (Two yay) = (wait for it) pass ;
policy (And a dot can go here) . (Like that) fruit (which surprised me) =
(because I was not expecting it) banana

I appreciate it's always nicer to only write pretty examples, but given the
specification, providing a nasty one is probably needed, lest people expect
to take a whitespace delimited token and split it on '='. For some reason I
particularly wasn't expecting the CFWS around the dot. (Maybe that's
because I'm out of practise with header parsing).

In any case, highlighting this with an example seems appropriate and useful.

Thirdly, one more major concern:

You'll note that I included a version for the method in my example above.
The definition for this, from the comments in the ABNF, suggests that this
relates to the version of RFC 5451 and successors used, but this seems
unlikely - I'm also at a loss as to what a receiving implementation should
do with such a method, since the handling of version seems to be specified
only for the version number after the auth-srv-id.

I have a vague suspicion that the correct thing to do here may be to
deprecate the method version entirely, but I might be missing something -
still, it doesn't appear to relate to the method, but the header field.

Dave.

---

From: Alessandro Vesely <[email-masked]>
Date: Sat, 16 Mar 2013 09:52:12 +0100

On Fri 15/Mar/2013 12:34:03 +0100 Dave Cridland wrote:
> 
> Authentication-Results: foo.example.net (My mailserver) / (I am
> number) 1 (You see) ; dkim (Because I like it) /2 (Two yay) = (wait
> for it) pass ; policy (And a dot can go here) . (Like that) fruit
> (which surprised me) = (because I was not expecting it) banana

My understanding is that the above is not valid.  Nuking three chars,
the first "/" and the "1" from autherv-id, which should be dot-atom,
and then the ";" between "pass" and "policy", which shouldn't be
there, I can parse it with my new Perl function, and get:

$VAR1 = {
          'dkim' => [
                      {
                        'policy_fruit' => 'banana',
                        'result' => 'pass'
                      }
                    ]
        };

(I s/./_/ for ease of use.)

I agree the grammar is laid down too lazily.  For example, dot-atom
allows equal signs to be part of property, e.g. "fru=it", which would
be unparsable.

---

From: Alessandro Vesely <[email-masked]>
Date: Sat, 16 Mar 2013 09:59:47 +0100

> the first "/" and the "1" from autherv-id, which should be dot-atom,

Oops, I overlooked that "[ CFWS version ]" in authres-header.  Thus
you helped me catch a bug.  Thanks!

> _______________________________________________
> apps-discuss mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/apps-discuss
>

---

From: Dave Cridland <[email-masked]>
Date: Sat, 16 Mar 2013 18:43:37 +0000

> you helped me catch a bug.  Thanks!

That was even the one that was properly specified... it's the other one I'm
more concerned with.

Dave.

---

From: Alessandro Vesely <[email-masked]>
Date: Sun, 17 Mar 2013 10:18:17 +0100

On Sat 16/Mar/2013 19:43:37 +0100 Dave Cridland wrote:
> That was even the one that was properly specified... it's the other
> one I'm more concerned with.

"Properly" might be something of an overstatement.  Section 2.3 says:

 For tracing and debugging purposes, the authentication identifier MAY
 instead be the hostname of the MTA performing the authentication
 check whose result is being reported.  This is also useful for
 another purpose, as described in Section 4.  Moreover, some
 implementations have considered appending a delimiter such as "/" and
 following it with useful transport tracing data such as the [SMTP]
 queue ID or a timestamp.

Informal discussions like that tend to be ambiguous because "/" is
allowed in dot-atom.  Hence one doesn't know whether the text means
something like:

  server.example.com/Q304b5 / 1

or:

  server.example.com / 197813

---

From: Dave Cridland <[email-masked]>
Date: Sun, 17 Mar 2013 11:34:59 +0000

Oh, well spotted. I'd not seen that one.

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Fri, 22 Mar 2013 12:21:21 -0700

Hi Dave, thanks for the review.   Comments below.

> For the most part, this document looks fine and solid.

Hooray!

> find the note of how to treat this; I didn't see any examples with it

I've updated one of the examples to include a version.

> I appreciate it's always nicer to only write pretty examples, but given
> the specification, providing a nasty one is probably needed, lest people
> expect to take a whitespace delimited token and split it on '='. For some
> reason I particularly wasn't expecting the CFWS around the dot. (Maybe
> that's because I'm out of practise with header parsing).
>

Copied to the draft, with some adjustments to make it compliant and fit the
width limit.

> still, it doesn't appear to relate to the method, but the header field.
The method version refers to the method itself, which is specified
elsewhere; the authserv-id version refers to this draft and thus the syntax
of this header field.  The idea is that if you find a version after an
authserv-id that you can't handle, you stop trying to parse right away
because what follows might not be what you expect.  In terms of a method
version, the parser should ignore a method result if the version of the
version is not supported in case the semantics of the result have a
different meaning.  If, for example, DKIM v2 (were such a thing to exist)
yielded a "pass" for different reasons than v1 did, a consumer of this
field might not want the altered semantics to apply.  This is a way to
indicate this and let the consumer decide.

Is your reply to this "You should say that?"  :-)

-MSK

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Fri, 22 Mar 2013 12:22:43 -0700

I've revised the grammar in the next draft to avoid this ambiguity.  Let me
know if it works.  Essentially, "dot-atom" has been replaced by "value",
which disallows the case you've illustrated here.

---

From: Alessandro Vesely <[email-masked]>
Date: Sat, 23 Mar 2013 19:56:08 +0100

On Fri 22/Mar/2013 20:22:43 +0100 Murray S. Kucherawy wrote:
> I've revised the grammar in the next draft to avoid this ambiguity.  Let
> me know if it works.  Essentially, "dot-atom" has been replaced by
> "value", which disallows the case you've illustrated here.

Hm... that way it becomes possible to have "dkim"="pass".  I'd beg for
"token", which is somewhat simpler, but it allows dots.  Wouldn't it be
possible to have a grammar that matches the actual use?  I mean, e.g.:

authserv-id = domain-name
     method = Keyword [ [CFWS] "/" [CFWS] version ]
     result = Keyword
   property = Keyword

"Keyword" is defined in RFC 5321, as well as "domain-name".  (It seems
to be useless to recall the definition from RFC 6376, as the domain
literal alternative that was in RFC 2821 has gone away.)

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sat, 23 Mar 2013 13:31:22 -0700

What would be wrong with "dkim"="pass"?

The main difference between "token" and "value" is that the latter permits
quoted strings.  I couldn't think of a good reason to proscribe those, and
it's not like it makes parsing any more difficult; things that can parse
"value" have been around for years.

-MSK

---

From: Alessandro Vesely <[email-masked]>
Date: Sun, 24 Mar 2013 13:04:07 +0100

There's nothing intrinsically wrong with "value", albeit it is sometimes
a nuisance not to have a canonical form.  The point is how many A-R
parsers will need to be updated to comply with it, since "dot-atom"
didn't admit quotes.

Another point is about extensions.  I proposed "Keyword" because it
matches all the registered methods, and thus looks like what may seem to
be the spirit of the spec.  If 5451bis will say "value", then the
appointed expert will have no reason to oppose methods with long names
and properties, comprising spaces and punctuation like some sort of
filenames which require composition assistance in order to be spelled
correctly.  Is that what we want?

On Sat 23/Mar/2013 21:31:22 +0100 Murray S. Kucherawy wrote:
> The main difference between "token" and "value" is that the latter
> permits quoted strings.  I couldn't think of a good reason to proscribe
> those, and it's not like it makes parsing any more difficult; things
> that can parse "value" have been around for years.
> 
> -MSK
> 
> 
> On Sat, Mar 23, 2013 at 11:56 AM, Alessandro Vesely <[email-masked]
> <mailto:[email-masked]>> wrote:
> 
>     [email-masked] <mailto:[email-masked]>

---

From: Scott Kitterman <[email-masked]>
Date: Sun, 24 Mar 2013 08:43:11 -0400

I tried out "auth"="pass" and my implementation chokes on it.  I'd strongly 
prefer not to have to change it and having a single canonical form is 
definitely better.  So my answer to Alessandro's question about how many 
parsers would have to be updated is "at least one".

Scott K

On Sunday, March 24, 2013 01:04:07 PM Alessandro Vesely wrote:
> >     > I've revised the grammar in the next draft to avoid this
> >     
> >     ambiguity.  Let
> >

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sun, 24 Mar 2013 14:06:41 -0700

I'm not in favour of changing to a grammar that matches current use to the
exclusion of other uses without a good reason to do so.  Since in the prose
we talk about authserv-id being a domain name in the typical case but
basically also say it could be anything the operator wants to use, I think
"value" is the right thing there.  We also shouldn't be making changes that
aren't either improvements to the prose or bug fixes to the normative parts.

Keyword is probably fine for the other three you mentioned.  I'll make that
change.  This is obviously a bug fix since, as you pointed out, "dot-atom"
permits ambiguous productions.

-MSK

---

From: Alessandro Vesely <[email-masked]>
Date: Mon, 25 Mar 2013 13:19:30 +0100

On Sun 24/Mar/2013 22:06:41 +0100 Murray S. Kucherawy wrote:
> I'm not in favour of changing to a grammar that matches current use to
> the exclusion of other uses without a good reason to do so.  Since in
> the prose we talk about authserv-id being a domain name in the typical
> case but basically also say it could be anything the operator wants to
> use, I think "value" is the right thing there.

Besides the parsability point (quotes), the intended use of this piece
of data is rather overloaded.  Let me recap some requirements:

1. Identification of the ADMD for assessing trust relationships
   (and possibly also for POP3 MUAs to send back complaints,
   per Section 5.3 of RFC 6650),
2. uniqueness of the identifier,
3. similar in syntax to a fully-qualified domain name,
4. transport tracing data such as the SMTP queue ID or a timestamp
   (semantic hints would also go here, e.g. "all signatures checked
   but failures not reported" vs "checking stops on first success"),
5. version identification,
6. examples of valid authentication identifiers are "example.com",
   "mail.example.org", "ms1.newyork.example.com", and "example-auth".

Bullets #3 and #6 are compatible with "domain-name".  By using domain
names in the traditional ways, even though they are not required to be
in the DNS, operators can easily implement #1 and #2.  Conversely,
we'd need production rules to parse a "domain-name" inside the rest of
the "value" if the grammar were relaxed that way.

Alternatively, we can meet #4 and #5 by relaxing or extending the
"version".  For example:

     authres-header = "Authentication-Results:" [CFWS] authserv-id
              [ [CFWS] separator [CFWS] ex-version ]
              ( [CFWS] ";" [CFWS] "none" / 1*resinfo ) [CFWS] CRLF

     artext  = ALPHA / DIGIT /  ; RFC 5322 atext without "="
               "-" / "_" /
               separator

     separator =  "!" / "#" /   ; dot-atom characters that cannot
                  "$" / "%" /   ; be part of a domain-name
                  "&" / "'" /
                  "*" / "+" /
                      / "/" /
                        "?" /
                  "^" /
                  "`" / "{" /
                  "|" / "}" /
                  "~"

     ar-dot-text = 1*artext *("." 1*artext)

     ex-version = [CFWS] ar-dot-text [CFWS]

That way, we can be compatible with the current dot-atom, still allow
free use, and define the id.  It is grammatically involved, as bug
fixes often are.

Semantically, allowing "anything the operator wants to use" conflicts
with the requirement that a parser explicitly knows it, in the new
Section 2.4.  IMHO, unrecognized ex-versions could just be ignored, as
the format is determined by the field name.

> We also shouldn't be making changes that aren't either improvements
> to the prose or bug fixes to the normative parts.
> 
> Keyword is probably fine for the other three you mentioned.  I'll make
> that change.  This is obviously a bug fix since, as you pointed out,
> "dot-atom" permits ambiguous productions.

Thanks

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Mon, 25 Mar 2013 11:54:25 -0700

>    per Section 5.3 of RFC 6650),

I don't see a reference to this work there.

> 2. uniqueness of the identifier,

This is key for identifying your own, but it doesn't speak to a necessary
syntax.

> 3. similar in syntax to a fully-qualified domain name,

There's nothing saying an ADMD can't use a simple word or even a single
punctuation here (so long as it fits the grammar).  Ideally it's most
useful when it's in this form, but an ADMD could use "---" as an
authserv-id and still get results useful to it.  A complete sentence would
also be valid.

I can't think of a good reason to restrict this to look like a domain name
if it doesn't actually represent a single DNS domain entity.  One ADMD
could be doing authentication service for lots of domains, but it's still
one ADMD.  If I run a service that does email authentication for tons of
domains, why could I not use "Murray's Authentication Service" for an
authserv-id?  Or even the empty string?  As long as the producer and
consumer are consistent in their uses of it, all of your stated
requirements are met.

>    but failures not reported" vs "checking stops on first success"),

This also doesn't appear to me to demand a specific format (as above).

> 5. version identification,

That's not part of the authserv-id itself.

>    "mail.example.org", "ms1.newyork.example.com", and "example-auth".

The last string in that list is not what most people think of as a
fully-qualified domain name, but as you point out, it's perfectly
acceptable.

>      artext  = ALPHA / DIGIT /  ; RFC 5322 atext without "="

I think we're wandering off into some bikeshedding here.  I'm curious to
know if anyone else supports this more complicated notion.  I certainly
don't agree with making the "separator" be anything other than "/".

The overall grammar is pretty clean right now: break on unescaped
semicolons and you get paragraphs of useful information; the first
paragraph is the authserv-id stuff, and everything else is "method=result"
followed by relevant method parameters; any unescaped "/" followed by a
number is a version.  Is there a reason to complicate that?

> the format is determined by the field name.

How does it conflict?

-MSK

---

From: Alessandro Vesely <[email-masked]>
Date: Tue, 26 Mar 2013 13:41:11 +0100

On Mon 25/Mar/2013 19:54:25 +0100 Murray S. Kucherawy wrote:
>>> Since in the prose we talk about authserv-id being a domain
>>> name in the typical case but basically also say it could be
>>> anything the operator wants to use, I think "value" is the 
>>> right thing there.
>> 

That possibility was voiced in the ASRG and elsewhere, but never
formalized --which is why I put it in parentheses.  AIUI, the
authentication server identifies the ADMD responsible for receiving a
message.  The means of identification, usually a registered domain
name, is consistent with the way each method identifies the ADMD(s)
responsible for /sending/ the message.

> This is key for identifying your own, but it doesn't speak to a
> necessary syntax.

I'm dumbfounded by that statement.  How can a piece of software be
configured by an ADMD so as to determine whether A-Rs "have been added
within its trust boundary"[Section 5] if that software is unable to
distinguish the server id from, say, the queue id?

> There's nothing saying an ADMD can't use a simple word or even a
> single punctuation here (so long as it fits the grammar).  Ideally
> it's most useful when it's in this form, but an ADMD could use "---"
> as an authserv-id and still get results useful to it.

"---" doesn't match "domain-name".

> A complete sentence would also be valid.

If you use "value", yes.  However, the methods to determine whether
two sentences identify the same ADMD are not part of the common practices.

> I can't think of a good reason to restrict this to look like a domain
> name if it doesn't actually represent a single DNS domain entity.

Well, one could use, e.g, verifier-X.host-Y.my-ADMD.example.  In that
case, those common practices might suggest that Z.my-ADMD.example
belongs to the same ADMD.  RFC 5451 does not impose such method,
presumably to allow implementers to experiment different ways.

> One ADMD could be doing authentication service for lots of domains,
> but it's still one ADMD.

It's their choice whether to masquerade or not.

> If I run a service that does email authentication for tons of
> domains, why could I not use "Murray's Authentication Service" for
> an authserv-id?  Or even the empty string?  As long as the producer
> and consumer are consistent in their uses of it, all of your stated
> requirements are met.

In principle, I agree.  However, producers and consumers can consist
of software developed by several independent programmers.  Those
agents can interoperate only if programmers agree on a common syntax,
so that agents can be configured in ways compatible with one another.

> This also doesn't appear to me to demand a specific format (as above).

It is unspecified how a consumer would use a queue id or a verifier's
configuration version.  Consonant producers and consumers can carve
out a space where to pass useful data from one to the other.  The
important point is that they don't step on another agent's toes by
invading the id proper.

> That's not part of the authserv-id itself.

Right, it's similar to #4 in this respect.

>> 6. examples of valid authentication identifiers are "example.com"

Yes, albeit that may preclude A-Rs produced that way to cross domain
boundaries.

>> "version".  [example omitted]
> 
> I think we're wandering off into some bikeshedding here.  I'm curious
> to know if anyone else supports this more complicated notion.  I
> certainly don't agree with making the "separator" be anything other
> than "/".

The text is not stringent:

   appending a delimiter such as "/" and following it with useful
   transport tracing data

I'd prefer a fixed separator too.  How about this:

   "Authentication-Results:" [CFWS] domain-name
              [ [CFWS] "/" [CFWS] dot-atom ]
?

> paragraph is the authserv-id stuff, and everything else is
> "method=result" followed by relevant method parameters; any unescaped
> "/" followed by a number is a version.  Is there a reason to
> complicate that?

My attempt is for simplifying, not complicating.  The "value" syntax
would allow:

   Authentication-Results: "Murray's Authentication Service;
      used with permission since version /1. ;-)" / 2;
      dkim = pass header . d = example.com (...)

You'd have to extend the acid test in C.7 to account for that.

>> the format is determined by the field name.

For example, assume my MTA is set up such that I can tell from the
queue id whether a message arrived to port 25 rather than 587, and I'd
like to use that info downstream, e.g. for logging something.  It is a
hack, admittedly, but the A-R producer I use happens to let me
configure it for appending the queue id, so I use that and roll my A-R
consumer hack.  Now, I also run another A-R consumer, for example
spamassassin, which is --will be, hopefully-- able to use existing
results rather than computing them from scratch.  What should it do
when it finds A-R-producer.my-host.my-ADMD/12345?  If that is a
version number that it knows nothing about, it should discard the field.

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Tue, 26 Mar 2013 08:35:28 -0700

> responsible for /sending/ the message.

My point is that RFC6650 Section 5.3 says nothing about allowing POP3 MUAs
to extract a reporting address from the Authentication-Results header
field's authserv-id, so I don't understand why you used it as a reference.
Such was never the intent.  It's fine to talk about it in ASRG or anywhere
else, of course, but that's not in my view a good impetus for now adding
that possible meaning here.

> distinguish the server id from, say, the queue id?

That logic presupposes that queue IDs have a known syntax, but they don't.
I know of one that uses slashes and hyphens in its queue IDs, for example.

But to answer your question directly, any software inside an ADMD seeking
to use this field would be configured to look for a particular string right
after the header field name and up to the first semicolon, and if that
string matches the string it's looking for, then the field can (presumably)
be trusted.

> "---" doesn't match "domain-name".

"domain-name" isn't in the new document except as part of a pvalue.

> two sentences identify the same ADMD are not part of the common practices.

There is no obvious reason to preclude the possibility of not using a
domain-name there either.

> It's their choice whether to masquerade or not.

Indeed.  And there's no reason to constrain such masquerading to
"domain-name", is there?

> so that agents can be configured in ways compatible with one another.

Isn't that the point of publishing a standard, i.e., this entire exercise?

> important point is that they don't step on another agent's toes by

You seem to be advocating for giving the queue ID its own token in the
header field different from the authserv-id in a normative sense.  What for?

> Right, it's similar to #4 in this respect.

Then I don't understand you claimed it as a separate requirement.

> Yes, albeit that may preclude A-Rs produced that way to cross domain

I don't understand.

> My attempt is for simplifying, not complicating.  The "value" syntax
I don't see how the additional grammar you proposed previously (which was
half a page long on my screen) simplifies anything.

> You'd have to extend the acid test in C.7 to account for that.

I'm fine with extending the convoluted example to show this possibility.

> version number that it knows nothing about, it should discard the field.

Attaching the queue ID as part of the authserv-id means you're effectively
using a new authserv-id for every message, which is clearly not the
intent.  It's incumbent on the consumers to figure out what that means.  I
don't think that (or any hack) should be encoded into the document.

But to give you a counter-hack, append the queue ID after a ":" instead of
a "/".

-MSK

---

From: Alessandro Vesely <[email-masked]>
Date: Tue, 26 Mar 2013 20:27:20 +0100

> that possible meaning here.

Point taken.  I mentioned it to stress that there are cases where
authserv-id is consumed outside the ADMD where it was produced.

> I know of one that uses slashes and hyphens in its queue IDs, for example.

Yes,the queue id is just an example.

> string matches the string it's looking for, then the field can (presumably)

It has better be careful to exclude the queue id from the comparison.

>> [...]

A domain name can be used effectively only if the consumer knows it is
a domain name.

> Indeed.  And there's no reason to constrain such masquerading to

There are pros and cons.  Similar arguments hold for the choice of
domain that an ADMD uses as d= to sign messages...

> Isn't that the point of publishing a standard, i.e., this entire exercise?

Yes!  My point is that is a producer is allowed to stuff a queue id
somewhere besides the server id, then the consumer has better to
expect that possibility.

>>>> [...]
> You seem to be advocating for giving the queue ID its own token in
> the header field different from the authserv-id in a normative
> sense.

Exactly!

> What for?

Anything that it might be useful for.  If we think it isn't useful,
then the spec should not allow it.

> Then I don't understand you claimed it as a separate requirement.

They are both specified.  I think it is difficult and useless to have
them both to coexist.  I'd drop the authserv-id's version.

>> Yes, albeit that may preclude A-Rs produced that way to cross domain

It is quite complicated to let trusted domain's A-R pass through.
Using identifiers that are not domain names forces an additional level
of indirection, and I'd figure average admins would rather remove them.

> half a page long on my screen) simplifies anything.

Importing grammar from external sources doesn't make it simpler.

> I'm fine with extending the convoluted example to show this possibility.

And the (possibly minor) incompatibility with RFC 5451?

> don't think that (or any hack) should be encoded into the document.

But then why do you write:

                                              Moreover, some

You can allow or prohibit that, but not both at the same time.

---

From: "John Levine" <[email-masked]>
Date: 26 Mar 2013 20:16:12 -0000

The point of the authserv-id is to allow a system to recognize its own
A-R headers.  I was happy with dot-atom and don't particularly care
about changing it to e MIME-ish value, which I gather is to agree with
some existing implementations.  But I certainly don't see any need to
add further complications, which nobody will implement anyway.

R's,
John

PS: In my implementation the authserv-id is the FQDN of the host on
which the mail server is running.

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Tue, 26 Mar 2013 16:33:17 -0500

> Yes,the queue id is just an example.

So the goal then is to allow the authserv-id to encode arbitrary other
details, such as but not including the queue ID, and that one example has a
completely unspecified format.  This seems like a no-win situation.

> It has better be careful to exclude the queue id from the comparison.

If an ADMD chooses to use the authserv-id to encode details apart from an
ADMD identifier, contrary to its stated purpose, then you're absolutely
correct.  RFC5451 already talks about that.

> A domain name can be used effectively only if the consumer knows it is

Again, there's nothing saying that it has to be a domain name.  It's merely
a string that the consumers expect will be set to a certain thing by border
MTAs within its own ADMD.  I get that it's a convenient construct, and it's
a good example, but there's no operational reason to limit it that way.

> domain that an ADMD uses as d= to sign messages...

Quite right.  Should this document be making that call?

> somewhere besides the server id, then the consumer has better to

Absolutely correct, and also absolutely not something that has to be
specified here beyond making it a "value".

I'd be happy to add text saying this specifically if necessary.

> then the spec should not allow it.

So why doesn't making authserv-id a "value" accommodate that case?

> them both to coexist.  I'd drop the authserv-id's version.

The authserv-id doesn't have a version in the ABNF.  That version is the
version of the A-R specification as a whole.  It's a convenient place to
attach that information (e.g., "this ADMD is using A-R version 'x'"), and
consistent with use of versions on the method tokens.

> of indirection, and I'd figure average admins would rather remove them.

Why is it quite complicated?  Trusted authserv-ids is merely a set of
strings.  If you're looking at an A-R header field whose authserv-id is in
that list, you've decided explicitly that its content can be trusted.
Anything else should either be ignored or removed (depending on which agent
is looking at it).

It only gets complicated if you choose to overload that field with your own
purposes by encoding other data in it.  If some people find that useful,
then great, but I don't think this specification should describe how to do
that beyond not explicitly blocking it.

> And the (possibly minor) incompatibility with RFC 5451?

The specific thing you're talking about, namely the occasional practice of
encoding queue IDs into authserv-ids, is already incompatible with
RFC5451.  This change adapts to current use.

> You can allow or prohibit that, but not both at the same time.

Keep reading; the remainder of that section actively discourages the
practice.

There are a lot of things that standards enable but also say "You could do
this, but there are costs involved."  This is one of them.

-MSK

---

From: Alessandro Vesely <[email-masked]>
Date: Wed, 27 Mar 2013 13:15:23 +0100

> correct.  RFC5451 already talks about that.

Ok, that is starting to make some sense.  Maybe it's me, but reading
RFC 5451 as well as its bis I had the false impression that the spec
not only allows but even encourages such practice.  I'll try and
suggest an alternative text at the bottom.

> Quite right.  Should this document be making that call?

No.  However I'd like a discourse on virtual domains, I don't think
this I-D is the right place to develop it.

>>> What [would transport tracing data and such be useful] for?
>>

It would overkill, considering that we'd tend to discourage
complicated stuff.  A "dot-atom" can have a slash, but perhaps we can
live with that.  Otherwise, a "token" can be fit.

> Anything else should either be ignored or removed (depending on which agent

IMHO, the first paragraph of Section 5 is too mild.  It could be
hardened like so:

OLD
 For security reasons, any MTA conforming to this specification MUST
 delete any discovered instance of this header field that claims, by
 virtue of its authentication service identifier, to have been added
 within its trust boundary and that did not come directly from another
 trusted MTA.

NEW
 For security reasons, any MTA conforming to this specification MUST
 remove or rename any instance of this header field that was not added
 within its trust boundary.  The authentication service identifier
 (authserv-id) is used to identify the agent which added the header
 field, with the limits described in Section 7.1.

If an MTA kills only the A-Rs that it can clearly identify as
forgeries, it would let thru any A-R field bearing some unknown
identifier.  Those identifiers may be forged.  If they match a trusted
id at the next ADMD (MTA or MUA), then if the latter trusts the
former, it has no obvious reason to suspect.  It gets complicated if
one tries to derive trustworthiness from routing considerations.

Renaming rather than removing solves some issues with debugging and
signature verification, so I'd consider it anyway.

I don't understand the "(border or otherwise)", still in the first
paragraph.  If it is an internal MTA it shouldn't delete fields added
by the border MTA of example.com.  The rest of the section becomes
more stringent, which seems to be correct.

> that beyond not explicitly blocking it.

Fine!

>>> I don't think that (or any hack) should be encoded into the
>>> document.
>>

Here's the alternative text I promised above:

                                              Consistently with the
 scope limit outlined in Section 1.2, the grammar does not constrain
 the authentication identifier to be a domain name.  Improper use
 could thus consider appending a delimiter such as "#" and following
 it with useful transport tracing data such as the [SMTP] queue ID or
 a timestamp.

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 27 Mar 2013 13:56:28 +0100

> live with that.  Otherwise, a "token" can be fit.

A quoted string is complicated?  They've been used in email header fields
for quite a long time now.

> one tries to derive trustworthiness from routing considerations.

If consumers are already looking only for A-Rs that use an expected
authserv-id, and ignoring others, what's the harm in letting the other ones
through?

The document already goes as far as saying one can also remove all of them
if it chooses, just to be safe.  Renaming them is probably also fine, but
I'm skeptical about adding that here since you're renaming within a
registered namespace.

> signature verification, so I'd consider it anyway.

True, but the agent removing suspect header fields could also log or
otherwise record them as it does so.

> more stringent, which seems to be correct.

Yeah, I don't know what that means either.  If I can't come up with a good
reason to keep it, I'll remove it in the next version.

>  it with useful transport tracing data such as the [SMTP] queue ID or

Isn't that materially the same as what's already in Section 2.3?

-MSK

---

From: Alessandro Vesely <[email-masked]>
Date: Wed, 27 Mar 2013 19:41:01 +0100

> for quite a long time now.

Yes, sorry for picking the wrong adjective.  The "complication" is
just the annoyance to modify the parser.  It is not that much of a
burden, but asking developers to do so only for accommodating a hack
that we'd discourage anyway is what I'd call overkill.

And, yes, at this point it is bikeshedding...

Another reason to review implementations is the version requirement of
Section 2.4.  Don't you need to say explicitly that the field version
being specified is "1"?

> authserv-id, and ignoring others, what's the harm in letting the other ones

It disables transitive extensions of trust.  The assumption that the
producer has the same list of trusted identities as the consumer
imposes restrictions on the trust boundary.  Substantially, it
requires the trust boundary to be a single ADMD, or a set of ADMDs
where the transitive closure of the trust relationship can be actually
computed.

For example, if you have Gmail and Yahoo accounts, you cannot
configure your MUA to simply trust the corresponding identities unless
both ADMDs vet A-R fields.  Otherwise, someone can send a message
containing a forged trust-me.yahoo.com to your Gmail account, which
would get through.  In a non-transitive trust scenario, your MUA can
trust Yahoo IDs only when they come from Yahoo directly.  For
transitive trust, if Gmail trusts Yahoo and you instruct Yahoo to
forward your mail to Gmail, you may expect to find pristine Yahoo A-Rs
in there, and have your MUA trust them when they come from any
compliant ADMD.

In this case, I think "complicated" is the right adjective.  Do you
have an idea of what percentage of opendkim users configure those
regexps complicatedly?

> I'm skeptical about adding that here since you're renaming within a

Hm... I don't know if registering the renamed header would make sense.

> otherwise record them as it does so.

That needs to be done cleverly to avoid losing the field's position,
and is unpractical for signature verification.  Another possibility is
to comment out the field content.  Like logging, that allows
additional explanations with it.  If the header was signed with the
relaxed algorithm there's no need to worry about added whitespace.

Perhaps the spec can mention the three possibilities, delete, rename,
and comment-out, and then pretend that the term "remove" means any of
such dismissals, when used for the A-R header field in the rest of the
document.  However, five times the I-D uses the term "delete" instead.

> Isn't that materially the same as what's already in Section 2.3?

Materially, yes, but it took me multiple messages to finally grasp the
intended meaning.  Again, maybe it's me, but the discussion starting
at the next paragraph doesn't seem to characterize the stuffing of a
queue id as improper use, and the somewhat relaxed grammar can
complete the misunderstanding.

---

From: SM <[email-masked]>
Date: Wed, 27 Mar 2013 12:20:03 -0700

Hi Alessandro,
>signature verification, so I'd consider it anyway.

I have a vague recollection of this.

The remove operation (milter) was easier.  The issue was about 
security where some unknown party spoofs my "id".  The rename 
(ignoring the "X-" stuff) might create more problems as I'll have to 
decide what to do when such a header is spoofed.

The case of the large provider doing a rename operation is their 
internal decision.  I'll have to apply heuristics to use that header.

Regards,
-sm

---

From: "John Levine" <[email-masked]>
Date: 27 Mar 2013 19:46:58 -0000

>suggest an alternative text at the bottom.

As far as I'm aware, nobody else has read it that way.  When I read
the text, it's clear that it's a stable identifier to let systems
recognize their own A-R headers.

> remove or rename

Definitely not.

R's,
John

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 27 Mar 2013 20:53:32 +0100

> Section 2.4.  Don't you need to say explicitly that the field version

The version is indeed 1, but it is optional in the ABNF.

> where the transitive closure of the trust relationship can be actually

But this isn't a new requirement.  It has always been the case that the
consumers need to know how to identify the instances produced by its own
ADMD.

> in there, and have your MUA trust them when they come from any

I don't think this document (old or new) goes to great lengths to establish
transitive trust mechanisms, and shouldn't unless there are implementations
that do so.  (I know of some that tried and failed, in fact.)  We do
mention that a starting point for doing so would be to sign A-R fields
before they travel onward, and then the ultimate recipient would need to
know (a) that the external ones to be trusted were covered by a valid
signature, and (b) that they contained an authserv-id that can be trusted.

But this is more bikeshedding for, as John put it, something that's
unlikely to be implemented anyway.

> have an idea of what percentage of opendkim users configure those

I don't think there are any regular expression configurations in opendkim
with respect to A-R.  There's one setting that enables the "/queue-id"
hack, if that's what you're talking about.  Either way, no, I have no idea
how many people are using.  I recall only one or two ever discussing it.

> Hm... I don't know if registering the renamed header would make sense.

Indeed it would not.

> relaxed algorithm there's no need to worry about added whitespace.

Any modification, including removing or altering in place, risks clobbering
otherwise valid signatures.  They aren't different in that regard.

> document.  However, five times the I-D uses the term "delete" instead.

Do you know of any extant or proposed implementations of the non-delete
options?  I don't think it's appropriate to revise a Proposed Standard by
peppering it with a giant "maybe".

-MSK

---

From: Alessandro Vesely <[email-masked]>
Date: Thu, 28 Mar 2013 18:08:02 +0100

> The version is indeed 1, but it is optional in the ABNF.

Fine.  But since the spec says:

   if a parser finds a version after an authserv-id
   that it does not explicitly know,

then, a parser designed after 5451bis has to _guess_ that the one it
knows is indeed "1".

>> imposes restrictions on the trust boundary.
> 

My point was that the producer needs to know the list trusted by the
consumers, in order to remove spoofed fields.

> that do so.

I agree that it shouldn't:  Section 1.2 is clear about being agnostic
on the trust boundary.  However, if the local definition of "trust
boundary" is such that a producer doesn't know the whole trust-list,
then the MUST in the first paragraph of Section 5 is not actionable.

>> Do you have an idea of what percentage of opendkim users configure
>> those regexps complicatedly?
> 
> I don't think there are any regular expression configurations in
> opendkim with respect to A-R.

Oops, I thought the patterns in "RemoveARFrom" were regular
expressions rather than plain strings.

> Either way, no, I have no idea how many people are using.  I recall
> only one or two ever discussing it.

Yeah, not worth overworking on that.  Sorry I raised that point.

>>> The document already goes as far as saying one can also remove
>>> all of them if it chooses, just to be safe.

Besides simplicity and security, that choice allows extensibility of
the trust boundary.

> otherwise valid signatures.  They aren't different in that regard.

Renaming by prefix insertion allows recovery at very small risk, with
very simple code:

  dkim_header(dkim, start + dkim_unrename,
     eol - start - dkim_unrename);

where dkim_unrename is either 0 or the length of the prefix inserted
by the upstream agent.  Note that the code above doesn't know whether
the field is actually signed or not.

> peppering it with a giant "maybe".

Courier-MTA does rename.  That will go in Debian and other distros
when released.  I discussed the A-R issue with Sam Varshavchik and was
initially unhappy with the solution he came up with.  Now that I have
experienced configuring and coding my filter to work that way, I must
say he was right:  It's simple, secure, and can be undone at will.

I'm not proposing to standardize renaming, since that should consider
also conventions on what prefix to use, and maybe even how to sign in
the presence of such prefix.  But I think the standard should avoid a
"MUST delete", as dismissals by any other means sort the same effect.

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Fri, 29 Mar 2013 03:24:10 +0100

> then, a parser designed after 5451bis has to _guess_ that the one it

Why?  This hasn't changed in the -bis document either.

> consumers, in order to remove spoofed fields.

This too is unchanged, and hasn't been identified as a burden yet.  The
producers and consumers need to know their own domain names as well (for
the obvious reasons) and that's never been considered a problem.

> then the MUST in the first paragraph of Section 5 is not actionable.

Why would the producer need to know more than one entry in the list?

> the field is actually signed or not.

You're doing this at the wrong layer.  This change would cause the DKIM
validation code to see something different.  It wouldn't cause the version
passed to the end user to be different.  You need to send the instruction
in the other direction, which makes it more complicated.

> "MUST delete", as dismissals by any other means sort the same effect.
I could also argue that renaming has the same effect as deleting; something
looking for "Authentication-Results" as a header field name will simply no
longer find it.  I'll see if I can write up some compromise text for the
next version.

-MSK

---

From: Alessandro Vesely <[email-masked]>
Date: Fri, 29 Mar 2013 11:33:35 +0100

> Why?  This hasn't changed in the -bis document either.

In RFC 5451 it was indicated as a comment to the "version" production
rule, as in the current draft.  That might be slightly inappropriate
since that rule works also for the methods, which have their own
versions.  Since -bis has a section on version tokens, readers may
expect to find that statement there.

BTW, would sender-id have had "2.0" there?

> the obvious reasons) and that's never been considered a problem.

Yeah, producers don't mind extra-ADMD consumers who may want to trust
them.

>>> I don't think this document (old or new) goes to great lengths
>>> to establish transitive trust mechanisms, and shouldn't unless
>>> there are implementations that do so.
>>

That's usual.  Doesn't opendkim take a list of identities and compare
each A-R against it?  If it doesn't have the whole list --and admins
may worry about allowing write access to their remardb to any possible
downstream consumer-- then there's nothing it can do about it.

As John said, the point of the authserv-id is to allow a system to
recognize its own A-R headers.  The "its own" part is what limits free
extensibility of the trust boundary.

> validation code to see something different.

Different from what the rest of the system sees, but hopefully not
different from what the signer signed.

> It wouldn't cause the version passed to the end user to be different.

Yup.

> You need to send the instruction in the other direction, which makes
> it more complicated.

What do you mean?  I cannot tell the sender that I don't trust it, so
it should stop cluttering messages with its A-Rs.  An A-R for the auth
method, for example, can only be written there.

OTOH, it is possible to recognize that a sender/signer is trusted and
unrename its A-Rs after verification too, when changes are committed
to disk.  I'm not going to code such complication, for the time being.

> longer find it.  I'll see if I can write up some compromise text for the

Thank you :-)

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Fri, 29 Mar 2013 20:25:47 +0100

> expect to find that statement there.

The version production is 1*DIGIT.  That's not a comment.

> BTW, would sender-id have had "2.0" there?

No.

> each A-R against it?

Yes, when operating as a consumer of the field (for the purpose of
identifying the ones that need to be deleted), not a producer of it.

>  If it doesn't have the whole list --and admins

It doesn't need the whole list when acting as a producer.  It needs one
string.

> extensibility of the trust boundary.

Consumers get to have a set that they consider "their own".  Producers can
only insert one string there, obviously.

> different from what the signer signed.

Of course it is.  What you're feeding to dkim_header(), by changing it,
will invalidate the signature, if that field was signed.

> to disk.  I'm not going to code such complication, for the time being.

I'm totally confused by what you're getting at here.

dkim_header() tells the DKIM layer what it's signing or verifying.  Why you
would want to change that is beyond me.  What you're trying to do is alter
the name of the field so consumers won't see it.  dkim_header() is
absolutely the wrong way to do that.  Rather, you need to tell the MTA to
do the rename operation, which might actually be a delete+insert
operation.  DKIM has absolutely nothing to do with that.

-MSK