Subject: Re: [dmarc-ietf] draft-levine-dkim-conditional-02

From: "Murray S. Kucherawy" <[email-masked]>
Date: Fri, 2 Oct 2015 05:25:49 -0700

This has been implemented in an experimental branch of OpenDKIM, since
about May.  I haven't released it yet, but it's probably time to do that.

I made a post some time back about changes I made in my implementation
versus the spec.  I'll dig that up too.  For one thing, I apparently
decided I didn't like the term "forward signature" and preferred
"conditional domain", so the new tag is "!cd".  We can argue about that and
the other stuff when I find that post.

If there's generally support for serious consideration of this approach, I
suggest the chairs do a Call For Adoption.

-MSK

On Tue, Sep 29, 2015 at 10:08 AM, John Levine <[email-masked]> wrote:

> I refreshed this draft so it wouldn't expire.  Not very different,
> mostly changed the @fs= to !fs= per Murray's suggestion.
>
> I still think this is the least broken way I've seen to let
> mailing lists coexist with DMARC.
>
> R's,
> John
>
> _______________________________________________
> dmarc mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/dmarc
>

---

From: "John R Levine" <[email-masked]>
Date: 2 Oct 2015 10:42:00 -0400

> the other stuff when I find that post.

Any tag syntax that's OK with you is OK with me so long as it does the 
job.

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
Please consider the environment before reading this e-mail.

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Fri, 2 Oct 2015 20:13:35 -0700

On Fri, Oct 2, 2015 at 7:42 AM, John R Levine <[email-masked]> wrote:

> Any tag syntax that's OK with you is OK with me so long as it does the job.
>

I think "!cd" is the only thing I changed.  Here's my old comment about it
with some open issues mentioned, like "v=2" to which Dave objected:

https://mailarchive.ietf.org/arch/msg/dmarc/WQNRhqdUjM5Kut0BWzENx8aJSO8

-MSK

---

From: "John R Levine" <[email-masked]>
Date: 3 Oct 2015 13:08:58 -0400

> https://mailarchive.ietf.org/arch/msg/dmarc/WQNRhqdUjM5Kut0BWzENx8aJSO8

I suppose the version bump to v=2 is an open issue, but I don't really see 
what the problem is.  Since the current spec says that unknown tags are 
ignored, there's no way to add mandatory tags without a version bump.  I 
believe when this came up before, people looked at common DKIM libraries 
including yours, the perl and python ones, and found that the current code 
would all fail a v=2 signature, so the version bump should do what we want 
it to.

There was a further rather theological argument about whether one can do a 
version bump by publishing a short document that describes the changes, or 
if it requires republishing the entire spec.  I don't understand the 
motivation for the latter, but some people seemed to think it was 
important.

Regards,

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sat, 03 Oct 2015 21:41:18 +0000

I think at least the latter is a minor point since the XML source for
RFC6376 is readily available.  I'm happy to pick up the editor pen again if
needed.

-MSK

---

From: "John R Levine" <[email-masked]>
Date: 3 Oct 2015 18:21:02 -0400

> RFC6376 is readily available.  I'm happy to pick up the editor pen again if

My concern is that once the can is opened, it's hard to keep some of the 
worms from escaping.

Regards,

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sat, 3 Oct 2015 16:46:46 -0700

On Sat, Oct 3, 2015 at 3:21 PM, John R Levine <[email-masked]> wrote:

> My concern is that once the can is opened, it's hard to keep some of the

Can't the charter help us keep the worms in check?  Changes have to be
within scope, for example; some of the old worms are off-topic here.

-MSK

---

From: Dave Crocker <[email-masked]>
Date: Sat, 3 Oct 2015 16:49:06 -0700

On 10/3/2015 10:08 AM, John R Levine wrote:
> I suppose the version bump to v=2 is an open issue, but I don't really
> see what the problem is.  Since the current spec says that unknown tags
> are ignored, there's no way to add mandatory tags without a version
> bump.  I believe when this came up before, people looked at common DKIM
> libraries including yours, the perl and python ones, and found that the
> current code would all fail a v=2 signature, so the version bump should
> do what we want it to.

Claiming that something is mandatory, as part of a version bump, is
meaningless, when the installed base will be using the older version and
ignoring the supposedly-mandatory new feature.

If the installed base can legally ignore a new feature, then it is not
mandatory.

The only way to make a feature mandatory in a new version is to provide
it in a fashion that makes it only available to folks adhering to the
new set of mandatory features.[*]

     FWIW, this is equivalent to saying that adding new
     mandatory features is equivalent to creating a new
     protocol.

So if you want to modify DKIM to change the set of mandatory features,
then specify that the signature use a /different header field name/,
such as DKIM-Signature2.  Only adherents to the new scheme will see this.

If the signer wants legacy folk who only understand older DKIM to also
be able to evaluate a signature, then the signer needs to create two
signatures.

Things get easier if you make the new feature optional, in which case
you don't need a version number...

d/

[*]  There's an exception that takes a long time, which being starting
by making the feature initially optional, then waiting for it to gain
massively widespread adoption, then issuing a declaration that it is now
mandatory.

-- 
Dave Crocker
Brandenburg InternetWorking
bbiw.net

---

From: "John R Levine" <[email-masked]>
Date: 3 Oct 2015 20:07:11 -0400

> you don't need a version number...

If you can figure out how to make signature conditional on another with 
without a version bump, that would be swell.  I've been thinking about it 
for over a year and so far this trick eludes me.

Regards,

---

From: Dave Crocker <[email-masked]>
Date: Sat, 3 Oct 2015 17:22:49 -0700

On 10/3/2015 5:07 PM, John R Levine wrote:
> without a version bump, that would be swell.

1.  I wasn't recommending making the new feature optional.  Merely
noting the considerable benefit that accrues if you can live with it
being optional.

2. I don't really understand your comment.  I'm probably missing the
implications of the larger context you meant to be replying to.

d/

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sat, 3 Oct 2015 17:23:24 -0700

> If the installed base can legally ignore a new feature, then it is not

I think there's possibly an issue with terminology here.

In DKIM v1, a verifier that sees a tag it doesn't understand simply ignores
the tag and continues processing the signature.  There are no "mandatory"
tags other than the few without which a signature can't be evaluated at all
(h, b, bh, d, s, c, and a come to mind).  Something using, for example, the
ones defined in RFC6541 or RFC6651, won't prevent verification by a
verifier that doesn't implement those.

In the proposed DKIM v2, a verifier that sees a tag prefixed with "!" that
it doesn't understand has to fail the signature without further
processing.  It's not mandatory that the tag be present, but it is
mandatory that the verifier has to do something with it or fail.  The
"mandatory" describes the handling of the tag, not its presence.

A v1 verifier would also ignore a v2 signature.

> new set of mandatory features.[*]

Doesn't this do that?

> such as DKIM-Signature2.  Only adherents to the new scheme will see this.

Isn't it true that we've found all current DKIM implementations would
ignore a "v=2" signature?  If so, I'm not understanding the distinction
between "DKIM-Signature: v=2; ..." and a wholly different field name.

> be able to evaluate a signature, then the signer needs to create two

It could do a v=1 and a v=2, certainly.  In fact in the instant proposal,
that's actually necessary.

-MSK

---

From: Dave Crocker <[email-masked]>
Date: Sat, 3 Oct 2015 18:03:02 -0700

On 10/3/2015 5:23 PM, Murray S. Kucherawy wrote:
> On Sat, Oct 3, 2015 at 4:49 PM, Dave Crocker <[email-masked]
> <mailto:[email-masked]>> wrote:
> 
> In DKIM v1, a verifier that sees a tag it doesn't understand simply
> ignores the tag and continues processing the signature.  There are no
> "mandatory" tags other than the few without which a signature can't be
> evaluated at all (h, b, bh, d, s, c, and a come to mind).  

Right.

> Something
> using, for example, the ones defined in RFC6541 or RFC6651, won't
> prevent verification by a verifier that doesn't implement those.
> 
> In the proposed DKIM v2, a verifier that sees a tag prefixed with "!"
> that it doesn't understand has to fail the signature without further

Understood.  However this does not change my point at all.  The language
you are using dictates mandatory behavior by the signature validation
engine.

But it is not really mandatory, since no legacy engine will understand
it.  Rather, they will ignore it.  Being allowed to ignore a parameter
means that the parameter is optional, not mandatory.  Or it means that
you folk are using the term 'mandatory' in a way I've never encountered
before and really don't understand.

> A v1 verifier would also ignore a v2 signature.

I had not noticed -- and still don't quite understand -- what it is
about the new stuff that will cause a legacy engine to fail the
signature validation.  Yes, I see that it will ignore all the !* new
stuff, but that doesn't mean it will fail the evaluation.

However to the extent that v1 engine evaluation failure is a target
outcome, that merely further underscores my point that you are creating
a new and incompatible protocol.

The way to signal that is at the entry point to the evaluation engine,
not buried amidst the parameters given to it.

The entry point is the header-field name.

At base, the spec defines a meta-syntax that appears to entirely miss
the point of creating signer/validator (or any producer/consumer)
compatibility.  The challenge in getting a consumer to process new,
mandatory fields is not syntax.  It's semantics.  And once you get the
consumer to understand the semantics, you don't need the meta-syntax.

>     new set of mandatory features.[*]

No.  Legacy engines will see it ignore it.  That does not match the
'mandatory' semantic being asserted in this spec.

> between "DKIM-Signature: v=2; ..." and a wholly different field name.

Ignoring v=2 is not the issue.  Ignoring the other, new and supposedly
mandatory parameters is the issue.

> It could do a v=1 and a v=2, certainly.  In fact in the instant
> proposal, that's actually necessary.

Which DKIM engine should I apply?  v1 or v2?

That gets answered at dispatch time with header field name.  It gets
answered after that with the version number model.  And a legacy engine
will incorrectly process the v=2 signature as if it were v=1.

d/

---

From: "John Levine" <[email-masked]>
Date: 4 Oct 2015 01:25:53 -0000

>noting the considerable benefit that accrues if you can live with it

You might want to reread my draft, because this discussion has no
relationship I can see to anything it says.

The DKIM spec says a signature is valid if a list of things are true,
e.g, the hash of the message's body matches the bh= value.  But since
the spec says that unknown tags are ignored, there's no way that you
can add a new tag that adds a new condition to the existing ones.  All
you can add are tags that are comments.  You could hypothetically add
a tag that would make an otherwise invalid signature good, e.g.,
here's another place to look for the key, but that seems to me like a
bad idea and nobody I know has done it.

What I want to do is add new optional conditions, this signature is
only good if all of the normal things are true, and these other things
are true, too.  Someone (not me) had the bright idea to use an
indicator, currently an exclamation point, to say that a tag adds a
new condition, so the verifier fails if it can't check all the tags
with that indicator, and we can add more condition tags later without
another version bump.

But now the syntax has changed, so if a signature uses the new kind of
tag, it has to be a v=2 signature.  If a signature doesn't, it's still
v=1, so everything that used to work still works.  If an old verifier
sees a v=2 tag it'll fail, so this fails closed.

This is about this simplest version of upward compatibility I can
imagine.  You could do a hack to put this in an otherwise invalid v=1
signature, e.g., put the body hash in a bh2= field and omit the bh=
so only a verifier that knows about bh2 will succeed, but why bother?
It's ugly and would work in exactly the same places that v=2 does.

R's,
John

---

From: "John Levine" <[email-masked]>
Date: 4 Oct 2015 01:35:36 -0000

>signature validation.

It's in section 3.5 of RFC 6376, the part about the DKIM-Signature
header, where it says:

   v= Version (plain-text; REQUIRED).  This tag defines the version of
      this specification that applies to the signature record.  It MUST
      have the value "1" for implementations compliant with this version
      of DKIM.

      ABNF:

      sig-v-tag       = %x76 [FWS] "=" [FWS] 1*DIGIT

         INFORMATIVE NOTE: DKIM-Signature version numbers may increase
         arithmetically as new versions of this specification are
         released.

People have looked at the code in most existing DKIM verifiers to see
whether the code matches the spec, and the current code does indeed
reject signatures that don't start with v=1.

R's,
John

---

From: Dave Crocker <[email-masked]>
Date: Sat, 3 Oct 2015 18:59:24 -0700

> reject signatures that don't start with v=1.

Yeah, and what's interesting about that is that the spec does not
explicitly specify that signatures without a v=1 need to be failed.

And you said 'most', not 'all'.

Anyhow this wasn't the primary focus of my comments and you haven't
responded to those.

d/

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sat, 3 Oct 2015 19:36:05 -0700

> you are using dictates mandatory behavior by the signature validation

Specifically, by a signature validation engine that understands v=2
signatures.  A v=1-only engine (i.e., the deployed base) is not expected to
change.

> it.  Rather, they will ignore it.

Specifically, they ignore the entire signature, not just the new tag.

  Being allowed to ignore a parameter
> before and really don't understand.

I think "mandatory" applies only to new implementations that support the
new "v=2".

> stuff, but that doesn't mean it will fail the evaluation.

As John pointed out, RFC6376 requires that the signature not pass for any
"v=" value other than 1.  It won't even get as far as considering the "!"
tags.

He said "most" implementations he checked will conform to that.  I thought
it was "all"; perhaps he'd care to elaborate.

> 'mandatory' semantic being asserted in this spec.

"Mandatory" in this context has nothing to do with legacy engines.  The
only "mandatory" that applies to them is that they will not pass a "v=2"
signature.

> mandatory parameters is the issue.

I think it's a core part of the issue.  If a legacy engine ignores "v=2"
signatures, semantics of the mandatory tags are moot to them.

> will incorrectly process the v=2 signature as if it were v=1.

If it doesn't conform to RFC6376 Section 3.5, yes.  So does this reduce to
the problem of accommodating deployed non-compliant verifiers?

-MSK

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sat, 3 Oct 2015 19:38:57 -0700

> explicitly specify that signatures without a v=1 need to be failed.

I believe Section 6.1.1, first paragraph, has that covered.

-MSK

---

From: "John R Levine" <[email-masked]>
Date: 4 Oct 2015 11:35:18 -0400

> it was "all"; perhaps he'd care to elaborate.

All the ones that anyone has checked.  Since the code for places like 
Hotmail isn't available, you'd have to send them some v=2 signatures and 
see what happens.  I haven't done that, although I'd be pretty surprised 
if they interpreted the spec differently from everyone else.

R's,
John

---

From: Terry Zink <[email-masked]>
Date: Thu, 8 Oct 2015 20:29:20 +0000

I'm not sure what Hotmail does currently, but it won't matter in the long run because the email infrastructure is moving over to Office 365. The DKIM code there will interpret a v=2 as an invalid signature.

-- Terry

-----Original Message-----
From: dmarc [mailto:[email-masked]] On Behalf Of John R Levine
Sent: Sunday, October 4, 2015 8:35 AM
To: Murray S. Kucherawy
Cc: [email-masked]
Subject: Re: [dmarc-ietf] draft-levine-dkim-conditional-02

R's,
John

---

From: [email-masked]
Date: Wed, 14 Oct 2015 14:04:36 -0700 (PDT)

Responding to this late, but I think there are some important points not
previously covered.

> ignoring the supposedly-mandatory new feature.

There's always a context to terms like "mandatory". In this case the context
is that of v2 signatures. The term is hardly meaningless in that context.

> If the installed base can legally ignore a new feature, then it is not

The nature of DKIM is such that many signatures end up getting ignored as
part of normal operation. That doesn't make any aspect of the various
mandatory aspects of processing those signatures any less mandatory.

>      mandatory features is equivalent to creating a new

I suppose you can define it that way if you like, but doing so ignores
the substantive overlap both in terms of syntax, processing semantics, and
most important of call, usage. 

> such as DKIM-Signature2.  Only adherents to the new scheme will see this.

This approach has the significant downside that it's more costly in terms
of code. Not a lot more costly, and if John's proposal had significant
associated cost I'd say it's a wash.

But John's proposal isn't costly at all. As a result I'd say that of using a
different header versus building off existing version suport just about doubles
the complexity.

Given this, I need significant to see significant justification - such as
reports that existing implementations don't properly handle DKIM versions - 
before I will support a proposal to use a different field.

Such reports have not appeared AFAIK.

> be able to evaluate a signature, then the signer needs to create two

That's not an argument for using a different field.

				Ned

---

From: Hector Santos <[email-masked]>
Date: Thu, 15 Oct 2015 13:16:19 -0400

> mandatory aspects of processing those signatures any less mandatory.

Its the same old theory since the original conception -- Only the ones 
with a policy will have any payoff, the more stronger, the higher the 
payoff will be. The more relaxed policies are still exploitable making 
it a high waste and overhead in processing.

> most important of call, usage.

For my package, we have to modify the C/C++ signing engine.  If ALT-N 
does not make the change, we have to do it ourselves.  In any case, 
any change requirement will allow the opportunity for the drastic 
changes to be done.  So just get it defined correctly.  Right now, v2 
signatures will fail.   Any we also have to change our C/C++ List 
server code and also our current official release DKIM+ASDP+ATPS based 
package to a DKIM+DMARC+ATPS.   We are not even covering the ATPS 
part. So I have to clean that mess up too perhaps and somehow tie it 
into this !FS= v2 thing.

I don't understand why if we have to make so many changes, we can just 
easily get it right with straight policy logic.  Look up the 3rd party 
authorization.  We should be allowed the opportunity to open that door 
now and allow people to explore it.

     Allowed = 3rdPartyFunction(domain)

and that 3rd party function can anything, including !FS and/or a 
policy lookup.

Whatever.  If the IETF is going to ask people to change their stuff, 
lets make it work this time around.  Levine proposed ADSP before and 
it failed. Now he has this !FS= thing.  I'm willing to give it one 
more shot.

Thanks.

-- 
HLS