Subject: [dmarc-ietf] indeterminisim of ARC-Seal b= value

From: Gene Shuman <[email-masked]>
Date: Fri, 24 Mar 2017 11:17:40 -0700

Hi All,

Based on a conversation I had with Murray the other day, I'd like to revive
a conversation I'd started on the ARC discuss forum a few months back.
Basically there is a subtle issue present in the current ARC draft that
doesn't exist with DKIM that I don't think most people are aware of that
gives implementor a large degree of control over ARC-Seal b= values

The issue is that its possible for two separate arc implementations, given
the exact same message inputs, keys, timestamps, etc to be generating two
different, but equally valid ARC seal hashes.  Basically, the ARC-seal b=
tag *is not deterministic* based on the incoming message.  This hash value
is *implementation dependent.*  I think that we want to think about this,
as this state of affairs does not exist in DKIM.  In DKIM(for a given set
of headers to sign, etc), there are only one valid b= & bh= hashes.

Here are some details.  Message 1 comes in with no arc signature.  The
implementation creates an ARC-Message-Signature tag, which is basically
just a DKIM signature.  Given, timestamps, header list, selector, domain,
etc, there is exactly one way to compute both the b= & bh= tags.  However, *the
*implementation* is free to vary the text representation of this signature*.
There is no sort of canonicalization happening here.  The implementation is
free to order tags arbitrarily, and things like add additional tags &
whitespace.

The reason this matters is because the textual representation of this
signature is used in the ARC-Seal's b= computation, making this value
*arbitrarily
implementation dependent.  *The implementation has a very large degree of
control over the actual value of the ARC-Seal b= hash, and this seems like
a non-trivial security risk.

The obvious solution I see here is to canonicalize the headers going into
the the AS computation.  There are some pretty sensible ways to do this.

---

From: "John Levine" <[email-masked]>
Date: 24 Mar 2017 21:23:04 -0000

In article <[email-masked]> you write:
>different, but equally valid ARC seal hashes.

DKIM does the same thing.  The order of fields in a DKIM-Signature
header is arbitrary, and the b= hash includes that header, so there
are lots of different equivalent DKIM signatures for the same message
and same selector and key.  Verifiers use the DKIM-Signature header in
the message so they get the same answer as the signer, which I would
think would work the same way in ARC-Seal.

Can you explain why you think this is a problem?

R's,
John

---

From: Gene Shuman <[email-masked]>
Date: Sun, 26 Mar 2017 15:23:47 -0700

Ah that had slipped my mind & is a good point.  However, I think the issue
here is generally that ARC is more complex protocol than DKIM and therefore
it's more important to reduce ambiguity & increase standardization while we
have the chance.  I think this is generally a good idea from a security
perspective, however this is mostly relevant with respect to testing &
validation, as ensuring cross-compatibility is a much bigger challenge.
It's even more important than it was with DKIM to have a test suite that
can verify signing behavior.  If we don't agree on any sort of standard, a
test suite will need to select a preferred format for the ARC headers &
will fail all implementations that don't meet this form.  We've discussed
this with Murray, and he agrees with this conclusion.

Our belief is that, given this is a new standard and all of the
implementors are on this list, it makes the most sense to define the
tag/value behavior strictly in the RFC.

Thoughts?

---

From: "John R Levine" <[email-masked]>
Date: 26 Mar 2017 21:32:40 -0500

> this with Murray, and he agrees with this conclusion.

I have to say I don't find that very persuasive.  It's not hard to write 
test code that checks for the hashes and fields in a signature header 
without regard to what order they're in.  I would rather write better test 
code than add fragile extra cruft to every ARC implementation.

As it happens, the IETF is meeting this week in Chicago, and I expect I'll 
see Murray in the next day or two so I can talk to him directly.

R's,
John

---

From: Peter Goldstein <[email-masked]>
Date: Sun, 26 Mar 2017 21:25:55 -0700

John,

> code than add fragile extra cruft to every ARC implementation.

I honestly find this response a little unpersuasive  :).  There are two
points I'd like to raise.

First, the idea that locking down the format of the signature adds 'fragile
extra cruft' seems off to me.  On the contrary, the suggestions in this
thread reduce fragility - by requiring that the set of signed fields (ARC
Message Signatures, ARC Seals) meet a well-defined, strict, and trivially
verifiable format.  This is fairly standard practice in video, security,
etc. protocols - every field has a place, and large scale
re-ordering/re-formatting is not permitted unless there's a compelling
technical reason.

I'm not sure why there's a perceived value in accommodating ambiguity here
- this is a brand new standard with a limited number of actual
implementers.  There's no random person not on this list implementing ARC
(especially given the standard isn't finalized).  And if there is, the
whole idea of providing a globally usable automated test suite is to ensure
that person can easily validate that their implementation will interoperate.

Second, the premise that "it's not hard to write test code..." is simply
not true.  What would be required to actually write such code would be
either a) pick a preferred ordering/formatting for these tags, and only
have the tests pass for that ordering or b) include a complete ARC
verifying system in the test code.  The first is what OpenDKIM did for its
specs (see below).  The latter is possible, but seems ill-advised for a
test suite.

To see that this is the case, it's useful to look at two earlier email
standards and their corresponding test suites.  SPF has a language-agnostic
test suite (http://www.openspf.org/Test_Suite) that makes it easy to test
implementations in any language.  I've done it myself, and it requires
minimal effort - https://github.com/ValiMail/coppertone/tree/master/spec/
open_spf.  I can tell you from personal experience that the test suite was
really helpful in ensuring the implementation was correct in all cases.

There is no corresponding test suite for DKIM, and the ambiguity in
tag/value ordering is the fundamental reason why.  The closest we come are
the tests for OpenDKIM.  If you examine the signing tests for OpenDKIM,
it's clear that they fail your "not hard" requirement.  I can write a
perfectly valid, legal DKIM signing implementation that will fail all of
OpenDKIM's signing tests, because OpenDKIM has a preferred ordering and
formatting for tags.

The OpenDKIM signing tests all check that OpenDKIM produces a signature
header matching a constant string that embodies this preferred ordering.
Any implementation that uses a different ordering/formatting will obviously
fail.  Changing the tag ordering/formatting changes the hashed value in the
header, so it's not just a matter of parsing some tag/value pairs.  And so
short of a full DKIM verification implementation it's impossible to make a
generic test suite that accepts all legal implementations.

Frankly, this is somewhat unfortunate.  But DKIM is a relatively simple
protocol.  ARC is more complicated, with multiple levels of signatures and
more complex state.  Given the substantially more complicated nature of
ARC, it's desirable to have a test suite that will pass any legal
implementation.  Interops are great, but I'd take a bulletproof test suite
over an interop any day of the week.

I won't be at the IETF meeting this week, but I'd be happy to dive into
this in more detail over email or phone if it's helpful.

Best,

Peter

On Sun, Mar 26, 2017 at 7:32 PM, John R Levine <[email-masked]> wrote:

> _______________________________________________
> dmarc mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/dmarc
>

-- 

Bringing Trust to Email

Peter Goldstein | CTO & Co-Founder

[email-masked]
+1.415.793.5783 <(415)%20793-5783>

---

From: Brandon Long <[email-masked]>
Date: Mon, 27 Mar 2017 11:06:20 -0700

What does validating the exact signature generated benefit us over just
verifying that the signature verifies?

I haven't looked at what the tests are for signing, though given the
descriptions  for the verifying tests, I'm not sure it would be obvious.
Can you summarize them?

Brandon

On Mar 26, 2017 9:26 PM, "Peter Goldstein" <[email-masked]> wrote:

> First, the idea that locking down the format of the signature adds
> 'fragile extra cruft' seems off to me.  On the contrary, the suggestions in
> this thread reduce fragility - by requiring that the set of signed fields
> (ARC Message Signatures, ARC Seals) meet a well-defined, strict, and
> trivially verifiable format.  This is fairly standard practice in video,
> security, etc. protocols - every field has a place, and large scale
> minimal effort - https://github.com/ValiMail/
> coppertone/tree/master/spec/open_spf.  I can tell you from personal
> experience that the test suite was really helpful in ensuring the
> implementation was correct in all cases.
>
>> As it happens, the IETF is meeting this week in Chicago, and I expect
>> I'll see Murray in the next day or two so I can talk to him directly.
>>
>>
>> R's,
>> John
>>

---

From: Peter Goldstein <[email-masked]>
Date: Mon, 27 Mar 2017 12:02:46 -0700

Brandon,

How do you validate that the signature verifies?  To do that you need an
entire ARC implementation that is packaged within the test suite (and the
ability for it to run as a service, or make it available in some way to
implementations being tested that are written in different languages).
That's the whole point.

The tests under discussion are validating that with assorted inputs
(original message, DNS configuration, etc.) an ARC implementation produces
a valid output message (correct ARC seal, ARC chain, etc.).  Essentially
we're looking at testing that:

SIGN(INPUTS) => OUTPUT MESSAGE

With the canonicalization being proposed, the OUTPUT MESSAGE becomes a
deterministic result, and verifying it becomes a matter of string
comparison.  This is the success criterion:

OUTPUT MESSAGE == EXPECTED MESSAGE

Without the canonicalization being proposed, it will be necessary to run
verification on the OUTPUT MESSAGE to assess whether the test gives the
expected results for a generic implementation.  In that case this is the
success criterion:

ARC_VALIDATION(OUTPUT MESSAGE, INPUTS) == EXPECTED_ARC_STATE(INPUTS)

We can't use the ARC validation implementation for the library under test,
because it may share bugs with the signing side.  Instead we'll need to
have a bundled ARC validaiton with the test suite.

Does that make sense?

Best,

Peter

On Mon, Mar 27, 2017 at 11:06 AM, Brandon Long <[email-masked]> wrote:

>> I'm not sure why there's a perceived value in accommodating ambiguity
>> here - this is a brand new standard with a limited number of actual
>> test suite (http://www.openspf.org/Test_Suite) that makes it easy to
>> test implementations in any language.  I've done it myself, and it requires

-- 

Bringing Trust to Email

[email-masked]
+1.415.793.5783

---

From: Brandon Long <[email-masked]>
Date: Mon, 27 Mar 2017 16:36:38 -0700

Generating the tests did require an arc impl , however.  I can understand
if this was simple enough to generate them by hand or with a couple lines
of python, but instead we're testing against an unknown impl that we can't
validate is correct.

Brandon

On Mar 27, 2017 12:02 PM, "Peter Goldstein" <[email-masked]> wrote:

Brandon,

Does that make sense?

Best,

Peter

-- 

Bringing Trust to Email

[email-masked]

---

From: Peter Goldstein <[email-masked]>
Date: Mon, 27 Mar 2017 17:24:09 -0700

Sure, generating the output message requires an ARC implementation.  Same
as generating a DKIM signature for validation requires an actual DKIM
implementation.  Or generating an SPF result for a complex situation
generally requires an actual SPF implementation.  Or generating an
ARC-signed input message for the verifying side of the test suite requires
some sort of ARC signing implementation.  But I wouldn't agree with the
inference that "we're testing against an unknown impl that we can't
validate is correct" in any of these cases.

The validity of the test suite derives from the fact that:

1. It was constructed from the RFC, and is built to cover a reasonably
comprehensive set of cases - more complete than any individual developer is
likely to consider.
2. While the output may have been generated with a particular
implementation, the test suite should be validated against a number of
independently developed implementations.  That ensures correctness, and
surfaces discrepancies which may represent ambiguities in the specification
and/or errors in the test suite or one or more of the implementations.
3. Once N implementations have validated, the N+1th implementation is
guaranteed a high degree of interoperability if it also passes the test
suite.
4. If new edge cases are discovered or evolved, interoperability can be
assured by adding them to the test suite and testing existing
implementations.  If there is ambiguity or disagreement in this
circumstance, this is an opportunity for the community to come up with the
agreed-upon right answer for behavior in this new case, and update the test
suite to reflect that.

None of this requires that we consider some unknown implementation that we
can't validate is correct.  The test suite becomes a statement of
consistency among the different implementers and their interpretations of
the RFC.  See the OpenSPF.org test suite for a good example of how this
works.

Having a fully deterministic specification makes this all a lot easier.

As I've stated previously, the alternative to the above is bundling a
reference ARC verifier with the test suite.  That means someone needs to
write such a reference verifier (or repurpose and repackage an existing
one).   In addition, the test suite needs to be updated to ensure the
reference implementation can be used as a service when developing ARC
implementations in any language.  That's more work, and it also makes the
use of the test suite much more complex.

I think tightening up some currently allowed ambiguity in the ARC
specification is a much simpler and much better solution.  I'm not sure why
there's such concern about canonicalizing the format and ordering of some
tag/value pairs.

Best,

Peter

On Mon, Mar 27, 2017 at 4:36 PM, Brandon Long <[email-masked]> wrote:

>>>> I have to say I don't find that very persuasive.  It's not hard to
>>>> write test code that checks for the hashes and fields in a signature header
>>>> I have to say I don't find that very persuasive.  It's not hard to
>>>> write test code that checks for the hashes and fields in a signature header

-- 

Bringing Trust to Email

[email-masked]
+1.415.793.5783

---

From: "John R Levine" <[email-masked]>
Date: 27 Mar 2017 21:21:11 -0500

> there's such concern about canonicalizing the format and ordering of some

If you think that's all it would take to make signature headers perfectly 
identical, you will be deeeply disappointed.  (Take a look at FWS in the 
ABNF and all of the other generic ABNF for message headers.)

I think what we've been saying is that the SMTP mail ecosystem has never 
tried to make stuff bit-for-bit reproducible, and even if you could hammer 
on the spec to make super strict rules for one particular header, it's 
unlikely that the people who are adapting their DKIM code would pay 
attention.

R's,
John

---

From: Peter Goldstein <[email-masked]>
Date: Mon, 27 Mar 2017 19:54:43 -0700

John,

I'm familiar with the definition of the FWS in the ABNF, as well as the
generic definition ABNF for message headers.  I'm also aware of the
challenges with trying to normalize such headers, and how that can impact
email authentication - breaking forwarded DKIM signatures and such.  But
none of that is actually relevant here - we are not interested in arbitrary
messages.

We are interested in creating a set of valid input messages and ensuring
that these messages are signed in a consistent, reliable way by any valid
ARC implementation.  This is an extremely narrow case - so yes, I do think
the above is basically all it would take to make such signature headers
identical.  Perhaps I'll be disappointed, but based on the sample messages
I have on hand from existing implementations, that seems unlikely.

As for the hypothetical developers who will be adapting DKIM libraries to
do ARC signing, we've been talking about them since M3AAWG in October
2015.  So far they haven't materialized.  Honestly, it's not even clear to
me that there are that many DKIM libraries out there to adapt.

Instead we've seen ~5 implementations (Google, AOL, Dkimpy, OpenARC,
MailerQ), with potential support for implementations in 1-2 additional
languages (e.g. Perl) probably driven by one or more of the implementers of
the existing implementations.  It should be relatively easy to coordinate
such a change across the small number of existing implementations.

Best,

Peter

-- 

Bringing Trust to Email

[email-masked]
+1.415.793.5783

---

From: Scott Kitterman <[email-masked]>
Date: Tue, 28 Mar 2017 00:49:17 -0400

What's more difficult to is identify all the ways that things get reordered, 
mangled, etc as they transit the various elements of the mail architecture.  
If you over specify the allowed order, aren't you risking increasing the 
brittleness of the solution.

If test cases are automatically generated, then they are cheap and we 
shouldn't worry about unduly constraining things to keep the number small.  As 
long as, at some point, the test cases are validated against multiple 
implementations, I think it's fine.  If you've got a handful of 
implementations, then the risk of them all having the same bug is pretty low.

In addition to Perl, I'd expect something in Ruby to appear in due course.

Scott K

---

From: Peter Goldstein <[email-masked]>
Date: Mon, 27 Mar 2017 22:30:37 -0700

Scott,

Header reordering is irrelevant to this discussion - the ARC RFC specifies
in some detail how to construct the ARC Seal from the AMS and AAR headers -
https://tools.ietf.org/html/draft-ietf-dmarc-arc-protocol-02#section-5.2 .
And the AMS headers have the same behavior as DKIM - header ordering and
canonicalization is also well-defined, so header re-ordering doesn't impact
the signature.  The discussion at hand is literally about the ordering and
formatting of tag/value pairs within the individual ARC headers.

So the only brittleness question is whether existing mail infrastructure
will potentially alter the internal structure of the AMS header or ARC Seal
header.  That is...unlikely.  We can conclude it's unlikely, because a
similar alteration of a DKIM signature header would break the DKIM
signature of forwarded email.  So if DKIM generally works (and it does), we
should be able to conclude that specifying ordering/formatting of the
tag/value pairs in an AMS/ARC header does not represent an increase in
brittleness.

I'm not really sure I understand the point regarding the # of test cases.
We're not worried about keeping the # of test cases small - we're
interested in ensuring that the same test cases will run successfully
against any valid ARC implementation.  Can you please clarify?

And yes, something will probably be developed in Ruby at some point - but
it's just as likely to be a de novo effort, as it is a modification of an
existing DKIM gem.  There is only 1 pure Ruby DKIM signing gem I'm aware of
- https://github.com/jhawthorn/dkim - and it hasn't had any updates in 2+
years and lacks verification logic.  So it's a less than ideal basis for
developing an ARC gem.

Best,

Peter

On Mon, Mar 27, 2017 at 9:49 PM, Scott Kitterman <[email-masked]>
wrote:

> > >> specification is a much simpler and much better solution.  I'm not
> sure
> > >> why

-- 

Bringing Trust to Email

[email-masked]
+1.415.793.5783

---

From: Scott Kitterman <[email-masked]>
Date: Tue, 28 Mar 2017 08:16:15 -0400

I will confess to not following all the details on a regular basis, but 
doesn't the paragraph you referenced give a required sequence of multiple 
header fields?  Maybe that's not a problem any more (it's been some years 
since I looked), but that definitely used to be something it was risky to rely 
on.

The test case comment was based on my understanding that you wanted 
deterministic ordering because if the ordering wasn't deterministic it would 
be too hard to test.  I think that can be managed.

Scott K

On Monday, March 27, 2017 10:30:37 PM Peter Goldstein wrote:
> > > none of that is actually relevant here - we are not interested in
> > 
> > arbitrary
> > 
> > > messages.
> > > 
> > > ARC implementation.  This is an extremely narrow case - so yes, I do
> > 
> > think
> > 
> > > identical.  Perhaps I'll be disappointed, but based on the sample
> > 
> > messages
> > 
> > > 2015.  So far they haven't materialized.  Honestly, it's not even clear
> > 
> > to
> > 
> > > languages (e.g. Perl) probably driven by one or more of the implementers
> > 
> > of
> > 
> > > >> there's such concern about canonicalizing the format and ordering of
> > 
> > some
> > 
> > > >> tag/value pairs.
> > > > 
> > > > If you think that's all it would take to make signature headers
> > 
> > perfectly
> > 
> > > > identical, you will be deeeply disappointed.  (Take a look at FWS in
> > 
> > the
> > 
> > > > I think what we've been saying is that the SMTP mail ecosystem has
> > 
> > never
> > 
> > > > tried to make stuff bit-for-bit reproducible, and even if you could
> > 
> > hammer
> >

---

From: Peter Goldstein <[email-masked]>
Date: Tue, 28 Mar 2017 07:57:09 -0700

Scott,

> since I looked), but that definitely used to be something it was risky to

No.  Precisely the opposite.  The referenced paragraph in the ARC spec is a
set of instructions of how to create an ARC Seal for a message.  The point
of referencing the instructions is to show that the algorithm is already
resilient to header re-ordering, and nothing being proposed here would
change that.

> be too hard to test.  I think that can be managed.

I think there's a misunderstanding here.  You seem to be under the
impression that supporting the current non-deterministic behavior can be
managed with 'more tests' that can be auto-generated and run against all
implementations.  That's not the issue.

The non-deterministic ordering requires either:

1. Implementations that use different tag/value ordering/formatting cannot
share a test suite.  Signing tests that are valid for one implementation
will not be valid for an implementation using a different tag/value
ordering or formatting.

or

2. The test suite needs to include a 'reference' ARC verifier

#1 explicitly fails your requirement that "test cases are validated against
multiple implementations", unless the implementations use the same
tag/value canonicalization.  This thread is simply proposing that we pick
one such canonicalization, and define it in the RFC, so that all ARC
implementations share the same canonicalization.

I've discussed why I think #2 is a bad idea - it's a lot of work (and it's
unclear who will do it), much more fragile and complex, and yields no
benefit that has been actually identified.

Best,

Peter

On Tue, Mar 28, 2017 at 5:16 AM, Scott Kitterman <[email-masked]>
wrote:

> > > implementations, then the risk of them all having the same bug is
> pretty
> > > low.
> > >
> > > > As for the hypothetical developers who will be adapting DKIM
> libraries
> > > > to

-- 

Bringing Trust to Email

[email-masked]
+1.415.793.5783

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Thu, 30 Mar 2017 03:21:12 -0500

I have to catch up on the rest of this thread still, but I wanted to chime
in here to get started:

> this with Murray, and he agrees with this conclusion.

I agree that it's impossible to design a signer test suite that confirms
correct output, without doing crytpo checks of its own with a known-good
verifier, unless we nail down the syntax the output will use.  For this to
work, we'd have to mandate a lot of things, including at least these:

- the order of the tags as presented to the hash algorithm
- which tags will be present (note that many are not required, including
"t=")
- the specific values they will all contain
- for ARC-Message-Signature, which canonicalization will be used
- the spacing between the tags; since "relaxed" header canonicalization
compresses spaces but does not add them, "a=foo;b=foo" is not the same as
"a=foo; b=foo", but "a=foo;\r\n\tb=foo" is
- similarly, how signature fields will be wrapped (if at all)
- what signing key will be used
- the body content to be signed
- the header content to be signed
- the set of header fields that will be signed (which becomes "h=")

It certainly removes many variables to nail down a test in this way, and
it's faster to run tests that don't require crypto functions or a
known-good verifier to confirm.  To be honest, I can't remember if we ever
considered this sort of approach during the development of DKIM.  I don't
think we did, and after an admittedly brief search I couldn't find anything
in the archives.  I think when canonicalization was developed, it was plain
that tag order wouldn't matter to the verifier, and that was sort of the
end of it.

The obvious counter-argument here is that DKIM has been successful without
ever being strict about what a signature looks like.  Our interop event in
Dallas way-back-when consisted solely of implementer "A" sending mail at
implementer "B" and seeing what passed and what failed, and digging into
the failure cases to see if they're bugs in code or bugs in the
specification.  Independent of interop events, various implementers also
set up auto-responders that would verify an arriving message, then re-sign
it and send it back so the originator can test its verification.  Any
failure compelled us to implement debugging tools and exchange information
so we could work out the kinks, either in the spec or in the
implementation(s), until all participants were consistent.  We reached a
point where we had organically developed a bunch of "good" implementations
based on the fact that they all appeared to agree with each other.

Since the canonicalization algorithms in ARC are the same as the ones in
DKIM, and the tag=value and key syntaxes are also the same, we've got a lot
of concepts and code being recycled here.  I'm not sure how much new
fragility we should really be concerned about.  I'll know more as I
complete my implementation.

Paul's point that video and security protocols are traditionally much more
rigid is of course quite true, and I presume he's thinking of things like
the headers of DNS, IP, TCP, etc.  But I note that those are based on a
different model where fields have fixed sizes and predictable positions.
That's antithetical to email, however, where the fields come in arbitrary
order, and the only way you know what a given header means is that its name
precedes its value.  There can even be duplication.  And from one MTA to
another, header fields can be added, removed, or rearranged.  That's
impossible in a rigid header definition, so it's perhaps no surprise that
this community isn't exactly warm to the idea.

The question, then, is whether this is a desirable path to follow: Is the
value of a quick evaluation of a deterministic signer high enough to
surrender the flexibility of DKIM style of tag spacing and ordering?  And,
perhaps more importantly, what's the cost of being wrong later?

-MSK

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Thu, 30 Mar 2017 03:52:26 -0500

> specs (see below).  The latter is possible, but seems ill-advised for a

OpenDKIM does both.  At some point when it was still "dkim-milter", when it
came time to write tests (because at one point there were none), I mainly
wanted to assure that commits made didn't cause any regressions;
correctness could be affirmed in other ways like at interop events or
through other implementers' auto-responders.  So I picked a tag ordering I
liked, generated a bunch of signatures with it, and then wrote tests that
did two things with them:

1) Confirmed that the signing code produces those specific signatures given
fixed inputs (timestamp, key, selector, domain, canonicalization, etc.).
This was inherently a tautology at the time I generated them of course, but
my goal was just to ensure no regressions were introduced.  I didn't know
for sure that the output was right, but I knew it would be consistent.

2) Since OpenDKIM conveniently includes a verifying capability, there are
some tests that instantiate the library to produce a signature, then
re-instantiate the library to verify it.  Obviously this doesn't prove that
its signatures are correct, but they do prove that its code is consistent
with itself, and again that changes made later don't break verifying.

None of this was done with an eye toward making my test suite applicable to
other implementations.  It's only for my own.  What we're talking about
here is different: coming up with tests that could be applied to any
implementation, not to a specific one, so long as they agree on fixed
syntax and inputs.

> implementation was correct in all cases.

An inherent difference between SPF and DKIM is that the order of the tokens
in SPF is significant.  In DKIM, reordering tags changes nothing about the
meaning of the header field, though obviously it alters the resulting
header hash.  In contrast, the order of tags in SPF is necessarily
specified.

-MSK

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Thu, 30 Mar 2017 03:54:01 -0500

> verifying that the signature verifies?

The most obvious benefit I can think of is that the output of signing is
entirely deterministic.  You could test it with strcmp() on a disconnected
computer, rather than invoking a complete ARC verifier that might need DNS
and would have to invoke crypto functions.

-MSK

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Thu, 30 Mar 2017 03:57:41 -0500

> brittleness of the solution.

I think Peter's proposal (and he can correct me if I'm wrong) is only to
constrain signers.  Verifiers are still expected to handle everything weird
that the infrastructure might do.

> implementations, then the risk of them all having the same bug is pretty

That sort of organic validation of implementations seemed to work for
DKIM.  The unknown here is whether ARC will be as successful with that
model.

-MSK

---

From: [email-masked]
Date: Thu, 30 Mar 2017 07:01:01 -0700 (PDT)

> that the infrastructure might do.

This proposal creates a situation where verifiers are required to support
something that no compliant signer will generate. The result will be that a
test suite with reasonable coverage cannot be generated using compliant
software.

And since most people are not going to bother to create an incompliant signer
specifically for purposes of testing, if anything this will create, rather than
elimnate, interoperability problems.

> DKIM.  The unknown here is whether ARC will be as successful with that

Exactly. And given the success of the approach with DKIM, the burden is
on those proposing to use a different model for ARC to explain why it will
work better.

And note that "this kind of test becomes possible" is not sufficient. All
kinds of tests become possible when you add constraints - the question is
what sort of real world problem will those additional tests find.

				Ned

---

From: Peter Goldstein <[email-masked]>
Date: Thu, 30 Mar 2017 09:10:22 -0700

> verifier, unless we nail down the syntax the output will use.

Great.  :)  I wasn't sure there was agreement on that in the wider thread.

> For this to work, we'd have to mandate a lot of things, including at least
> these:

So while it is true for a given test case you would need to constrain all
of these items, this is a much larger list than would need to be specified
as part of the proposal under consideration.

Many of these items are already constrained by the controlling RFC (ARC or
DKIM) and the other values specified in the signature.  For example, the
order in which different header fields are combined and hashed is defined
as a function of the tag/value pairs in the signature (i.e in DKIM
signature the 'b' value is impacted by the choice of values for the 'a',
'c','h', etc. tag values).  That ordering is therefore an input to the test
case.  The input message content (headers + body), the signing key(s), etc.
would all also be input to a particular test case.

A test suite should handle this by having test cases that exercise the
library with different inputs that give good coverage across allowed
variations.  That has the side benefit of allowing the test suite
implementer(s) to easily develop targeted test cases that cover those
specific variations in isolation, making it easier to find bugs.

That said, some of these items in the list (intertag spacing, required
tags, canonical values for some of the tags) would need to be constrained
as part of this proposal.  This is clearly a change, but it's a change to a
draft standard that has a small # of implementers and should be relatively
easy to lock down.  And I think it's far less of a change than some others
on this list do.

> ever being strict about what a signature looks like.  ...  We reached a

I think the bigger point being argued is that DKIM has been successful
without ever having a global test suite, because that's the impact of not
making this choice (or committing to a global reference implementation).

Which is mostly true, but frankly not entirely true.  I'm aware of at least
two major vendors who have production DKIM signing implementations with
edge case bugs that would've been caught by such a suite.  Those
implementations made it through development, passed whatever implementation
specific test cases the developers built, and made it to production.
That's what a good test suite helps you avoid.

> this community isn't exactly warm to the idea.

I'm assuming that I'm Paul.  :)

I am thinking of protocols like DNS, IP, TCP, but also standards that are
higher up the stack like some video protocols.  The latter standards often
allow large scale re-ordering of the individual chunks and streams of the
video, but usually have byte level constraints on the content of those
chunks.  In my view that's exactly analogous to what I'm proposing here.

I'm also recognizing that, once a standard has a binary component (i.e.
hashes), the level of flexibility that's being claimed no longer actually
exists.  As a specific example, take any email message you like that has
passed DKIM.  Insert a single additional space between any two tags/value
in a passing DKIM signature.  The message now fails DKIM.  That means the
semantic content of the message (and more importantly, how receivers will
treat it) will change.

There seems to be a lot of confusion in this thread about the difference
between strictly specifying fields inside a single new header, and changes
that cross headers.  None of the above mentioned changes to the header set
- adding, deleting, duplicating, etc. - is impacted in any way by the
proposal under discussion.  ARC already needs to handle those cases for the
construction of the ARC seal, and much of the complexity of ARC is around
canonicalizing the header set (specifying an order, eliminating duplicates)
and handling the corner cases in the face of such traditional flexibility.

> perhaps more importantly, what's the cost of being wrong later?

Agreed.  But I'd rephrase it as:

1. What is the value of having a open, globally available test suite for
ARC against which any implementer can test a new implementation or changes
to an existing implementation to ensure compatibility?   This is compared
to the current system of ad hoc interops at industry events.
2. What is the value of the flexibility of DKIM style of tag spacing and
ordering in the context of ARC?
3. What's the cost of sacrificing #2 for #1?

I hope I've made it clear why I think #1 is valuable.

I haven't actually heard much of an argument for #2 in this thread - I'd
love to understand the perceived benefits.

Most of the arguments have centered around #3, and the idea that lots of
ARC implementation will be adopted from existing DKIM implementations,
which don't have this constraint, and that the implementations will ignore
the new constraint.

Frankly I'm skeptical about this - DKIM is made up of relatively simple
components (RSA signing, Base64 encoding, even canonicalization) - that are
mostly available as standalone libraries in many languages.  Moreover, if
you survey the number of existing libraries per language that implement
both DKIM signing and verification, you come up with a pretty small number
for most popular languages (1-3), and many of those projects are moribund.
In many cases it'll be easier to start from scratch, especially given the
bulk of ARC's complexity is in areas that don't really exist in DKIM
(header set ordering by i, etc.).  I know that if I write a Ruby
implementation, that's likely how I'd approach it.

Even if the DKIM code is adopted, we're talking about a very specific set
of change to two isolated areas.  Looking at OpenARC in particular, that
code is already imposing a preferred tag/value ordering and spacing on the
signing side.  It's just buried in the implementation details.
Changing/standardizing it would not be particularly difficult.  Changing
the verifying code to validate the ordering/spacing would also not be
particularly difficult.  That's pretty much all of the work as applies to
that implementation.

Best,

Peter

-- 

Bringing Trust to Email

[email-masked]
+1.415.793.5783

---

From: Peter Goldstein <[email-masked]>
Date: Thu, 30 Mar 2017 10:01:05 -0700

> elimnate, interoperability problems.

Just to be clear, this is not what I was proposing.  Specifically, I don't
think the Robustness Principle
<https://en.wikipedia.org/wiki/Robustness_principle> applies in this case.
Verification should check the RFC defined spacing, ordering, etc. of the
tag/value pairs and reject non-compliant values

As this is a new protocol, and these headers are all new and used
exclusively for ARC, and for the reasons mentioned by Ned, there's no
reason to be lenient in what is accepted.

> on those proposing to use a different model for ARC to explain why it will

A few quick comments on this.

As per my earlier email, I have a lot less confidence in how well this
'seemed to work' than some of the other participants in this thread.  One
of the benefits of DMARC re:DKIM is it has exposed some issues with DKIM
implementations that were not obvious prior to DMARC evaluation.  Without
DMARC reporting some of the problems with certain DKIM implementations were
simply not very visible.  So I don't actually think the DKIM model worked
as well as is being asserted.

Second, the existence of such a test suite would have had material benefits
just within the last ~18 months over which ARC implementations have been
developed.  We know this because of the immediate benefits the current test
suite (which does impose a canonical ordering, spacing, etc.) yielded
almost immediately.

The suite exposed a few minor but significant issues with the Python
implementation that was submitted to dkimpy.  Similarly, it exposed that
the current OpenARC signing functionality is in a much less mature state
than was generally understood.  In terms of RFC definition, it exposed the
fact that the current RFC as written required support for insecure 512 bit
keys - and triggered a discussion on this very list.  In our internal
development it's forced us to look hard at some of the corner cases and
raise them on this list or at interops.  Note that most of these issues had
escaped detection through multiple interops.

None of the above is meant to insult the associated developers or spec
authors.  Their contributions have been essential, and the skill and
dedication they've brought to their tasks is greatly appreciated.  It is
simply meant to demonstrate the material benefit that this kind of testing
has already brought.  This is what automated tests do - bring problems to
light quickly.

So yes, I think this approach works better.  Much better.

Best,

Peter

-- 

Bringing Trust to Email

[email-masked]
+1.415.793.5783

---

From: [email-masked]
Date: Thu, 30 Mar 2017 10:13:54 -0700 (PDT)

> reason to be lenient in what is accepted.

On the contrary, there are very good reasons. People aren't going to magic
up implementations of ARC out of nothing; they will undoubtedly use
existing DKIM code as a starting point. Implementing an ordering
requirement on top of such code could get interesting depending on how
it works.

And given the relationship with DKIM people are going to expect ARC
to work in the same general way, maing such a change a least astonishment
principle violation.

> as well as is being asserted.

Which of these problems had anything to do with field ordering? Which of
these problems would have been exposed by requiring a specific
order?

> So yes, I think this approach works better.  Much better.

Simply put, I don't. I'm strongly opposed to this, to the point where I'm
likely to significantly deprioritize implementation of this proposal because of
it.

I have better things to do that waste my time watching the sorts of bugs this
is going to cause get shaken out.

				Ned

---

From: Seth Blank <[email-masked]>
Date: Thu, 30 Mar 2017 10:34:50 -0700

> what sort of real world problem will those additional tests find.

ARC is fundamentally different than DKIM in several key ways. If we were
only talking about ARC Signing messages, I'd generally agree with the
comments on this list.

However, ARC is fundamentally different. It is about a chain of messages
that validate each other. And ARC's complexity is not in the signing (where
most of this conversation has been focused), but in this chain validation
and the cascading issues that can be created by any member of the chain
doing something slightly wrong. And since *everyone* in the chain except
the initial signer is also a receiver, the final receiver simply can't make
up for mistakes caused by a less savvy intermediary - the chain will
already be broken and unrecoverable.

ARC also has a lot of subtlety that is missable at interops because of
this. For instance, if I'm modifying a message, I need to validate the
incoming chain, then make my modifications, and then sign the outgoing
message. An intermediary doing this wrong (say, validating and then signing
after message modification) isn't a bug a final receiver can fix. And
without a seriously complex interop with a ton of intermediaries and
sending relationships mapped out for testing, it's not likely an error in
any specific implementation here will be discovered until it's too late.

To my understanding, working on the test suite shook these issues out
because they were what caused problems running the test suite at the
interop in the first place. So this is actually a good and healthy part of
the process, and is the natural consequence of the interops. What worked
here for DKIM is insufficient with the nuances of ARC.

What we've been discussing doesn't just guarantee a test suite can be made
that works for everyone (which is huge in itself), but constrains the
problem space for everyone involved in an ARC chain. This is clean and
desirable.

Since the spec isn't finalized, this is the perfect time to impose some
order that isn't necessary for DKIM but is critical when we're talking
about multiple operators working together over multiple hops as part of a
chain of trust.

Seth

-- 

Bringing Trust to Email

Seth Blank | Head of Product for Open Source and Protocols
[email-masked]
+1-415-894-2724 <415-894-2724>

---

From: Brandon Long <[email-masked]>
Date: Thu, 30 Mar 2017 16:00:19 -0700

On Thu, Mar 30, 2017 at 1:21 AM, Murray S. Kucherawy <[email-masked]>
wrote:

>> Ah that had slipped my mind & is a good point.  However, I think the
>> issue here is generally that ARC is more complex protocol than DKIM and
>> therefore it's more important to reduce ambiguity & increase
>> standardization while we have the chance.  I think this is generally a good
>> idea from a security perspective, however this is mostly relevant with
>> respect to testing & validation, as ensuring cross-compatibility is a much
>> bigger challenge.  It's even more important than it was with DKIM to have a
>> test suite that can verify signing behavior.  If we don't agree on any sort
>> of standard, a test suite will need to select a preferred format for the
>> ARC headers & will fail all implementations that don't meet this form.
>> We've discussed this with Murray, and he agrees with this conclusion.
>>
>

This could imply two different things, either "alphabetize them in the
header" like was done for the current test suite, or it could imply a
canonicalization step when passing the header to signing. The latter seems
unlikely to be what we'd want, given we have to sign a whole bunch of ARC
headers, we probably don't want to interpret and sign each one, signing the
string is easier.

And I admit that I immediately disliked the alphabetized list upon seeing
it, the main issue I had was with not having i= at the front, the rest just
seemed ugly (ie, the b/bh tend to be at the end in implementations because
they are long).  That's probably not a valid argument against it, though I
do think that i= at the front is useful for visual grouping without having
to scan the full header to find it.

> - for ARC-Message-Signature, which canonicalization will be used

Does this also mean that my signer has to support all four combinations of
canonicalization?  Right now, it only does relaxed/relaxed.  I have no
intention of every generating anything with simple, and I could just fail
all of those tests, I guess.

> - similarly, how signature fields will be wrapped (if at all)

agreed.  I think some of this also goes to what implementers think look
good.  I also think dkimpy uses certain spacing so that they can wrap it
easier, as opposed to say Google's impl which hard codes most of it and has
more specific or hard coded wrapping.

These are implied by the test already, so no big deal.

> - the set of header fields that will be signed (which becomes "h=")

And the order in which they're signed.  Ie, our DKIM impl, because it was
also a DomainKeys impl, could choose headers in either direction, but
actually chooses them "backwards" for DKIM, requiring kind of a nasty loop
that's also worst case (O(n^2)).  For ARC, I chose to choose them bottom
up, which matches the order to search, and means usually the dual loop has
O(n) instead of O(n^2).  Granted, it's a silly optimization, and doesn't
help with other implementations we have to also handle.

dkimpy just has a fixed list of fields, and adds them mostly in order.
There's also the choice of whether to include a header or not, we removed
content-type from our DKIM list at one point trying to work around
Exchange's DKIM breaking behavior, for example.  Some implementations
double add Subject/etc to prevent duplicates, others don't.

We can probably come to agreements on all of these things... well, maybe.
I don't know how many potential implementations we'll curtail by doing so.
Or maybe some of these won't be specified in the spec, but only for the
test suite, so they'll be options to the signers but not required (ie, list
of headers to prevent duplicates should be N for testing).

There's also the fact that the current drafts basically incorporate all of
this from the DKIM without specifying, if we do otherwise, we have to have
a lot more exposition.  We'll also most likely end up with things that
aren't specified in the spec explicitly but are required by the test suite
(which may be better than the alternative, something ambiguous in the spec
without being explicit elsewhere).

Also, I'm not clear on the statement that these aren't relying on dkim
libraries.  The dkimpy arc impl is clearly depending on shared code between
them, I think Paul mentioned that the AOL one re-used code from their Java
DKIM impl, and I know the Google impl shares a bunch of code.  I don't
think any of this precludes these changes, but they aren't free either.

Brandon

---

From: Dave Crocker <[email-masked]>
Date: Thu, 30 Mar 2017 18:15:33 -0500

> to work in the same general way, maing such a change a least astonishment

+10.

This thread has been active for 6 days.  That is 5.5 days longer than it 
should have lasted, and I'm being generous.

Internet technologies succeed through interoperability testing, not 
conformance tests, or the like.  Conformance testing, and the like, are 
useful for early-stage bug-finding, but not for 'certification'.  This 
point has been a distinguishing feature of Internet technical work since 
before the Internet.

ARC is a small re-casting of DKIM.  DKIM has worked for quite a few 
years.  Some details of ARC need to be different from DKIM, but the 
basic paradigm -- including the model of achieving field operation 
through interoperability testing -- should be the same.

My reading of this too-long message thread is that there is essentially 
no support for making ARC's header-related issues any different from 
DKIM's, so I encourage the chair to shut this thread down.

d/

-- 
Dave Crocker
Brandenburg InternetWorking
bbiw.net

---

From: Seth Blank <[email-masked]>
Date: Thu, 30 Mar 2017 17:10:19 -0700

Dave, If we were only talking about ARC Signing messages, I'd generally
agree with the comments on this list.

Since the spec is not finalized, this is the perfect time to set a
deterministic order for headers that no one has even seen before.

Seth

On Thu, Mar 30, 2017 at 4:15 PM, Dave Crocker <[email-masked]> wrote:

> years.  Some details of ARC need to be different from DKIM, but the basic
> paradigm -- including the model of achieving field operation through
> interoperability testing -- should be the same.
>
> My reading of this too-long message thread is that there is essentially no
> support for making ARC's header-related issues any different from DKIM's,
> so I encourage the chair to shut this thread down.
>
> d/
>
> --
> Dave Crocker

-- 

Bringing Trust to Email

---

From: "HANSEN, TONY L" <[email-masked]>
Date: Fri, 31 Mar 2017 01:11:05 +0000

One of the reasons DKIM was successful in becoming interoperable was not that we had a test suite to force conformance, but instead that we had >multiple< test suites that were able to successfully validate DKIM signatures from multiple implementations. My test suite caught corner cases that Ned’s and Yahoo’s and Google’s did not, and vice versa for each one of them. (My company actually had two totally independent implementations of DKIM so that we could do internal testing against a different platform. It was just prudent to do it that way.)

When there was an issue in someone’s test suite, we were then able to discuss whether it was an issue in the test suite that needed to be fixed, or it was an issue in the implementation. A single test suite is never sufficient – that’s one reason we prefer multiple interoperating implementations in order for something to be considered an internet standard.

By the way, none of us that implemented DKIM test suites had an issue with the header order being determined by the sender. It makes the validation infinitesimally trickier than having a predetermined header order.

-1000 to the suggestion of having a single conformance test suite that everyone lives and dies by. +1000 to multiple test suites that multiple people write and can be used to test against.

I’m all for you, Seth and Peter, to write a test suite. But nothing you’ve said so far has convinced me that a predetermined header order or a single conformance test suite is worth pursuing. One of many, sure.

                Tony Hansen

From: dmarc <[email-masked]> on behalf of Seth Blank <[email-masked]>
Date: Thursday, March 30, 2017 at 8:10 PM
To: "[email-masked]" <[email-masked]>
Cc: "Murray S. Kucherawy" <[email-masked]>, "[email-masked]" <[email-masked]>, Scott Kitterman <[email-masked]>, Peter Goldstein <[email-masked]>, "[email-masked]" <[email-masked]>
Subject: Re: [dmarc-ietf] indeterminisim of ARC-Seal b= value

Dave, If we were only talking about ARC Signing messages, I'd generally agree with the comments on this list.

However, ARC is fundamentally different. It is about a chain of messages that validate each other. And ARC's complexity is not in the signing (where most of this conversation has been focused), but in this chain validation and the cascading issues that can be created by any member of the chain doing something slightly wrong. And since *everyone* in the chain except the initial signer is also a receiver, the final receiver simply can't make up for mistakes caused by a less savvy intermediary - the chain will already be broken and unrecoverable.

ARC also has a lot of subtlety that is missable at interops because of this. For instance, if I'm modifying a message, I need to validate the incoming chain, then make my modifications, and then sign the outgoing message. An intermediary doing this wrong (say, validating and then signing after message modification) isn't a bug a final receiver can fix. And without a seriously complex interop with a ton of intermediaries and sending relationships mapped out for testing, it's not likely an error in any specific implementation here will be discovered until it's too late.

To my understanding, working on the test suite shook these issues out because they were what caused problems running the test suite at the interop in the first place. So this is actually a good and healthy part of the process, and is the natural consequence of the interops. What worked here for DKIM is insufficient with the nuances of ARC.

What we've been discussing doesn't just guarantee a test suite can be made that works for everyone (which is huge in itself), but constrains the problem space for everyone involved in an ARC chain. This is clean and desirable.

Since the spec is not finalized, this is the perfect time to set a deterministic order for headers that no one has even seen before.

Seth

On Thu, Mar 30, 2017 at 4:15 PM, Dave Crocker <[email-masked]<mailto:[email-masked]>> wrote:
On 3/30/2017 12:13 PM, [email-masked]<mailto:[email-masked]> wrote:
And given the relationship with DKIM people are going to expect ARC
to work in the same general way, maing such a change a least astonishment
principle violation.

+10.

This thread has been active for 6 days.  That is 5.5 days longer than it should have lasted, and I'm being generous.

Internet technologies succeed through interoperability testing, not conformance tests, or the like.  Conformance testing, and the like, are useful for early-stage bug-finding, but not for 'certification'.  This point has been a distinguishing feature of Internet technical work since before the Internet.

ARC is a small re-casting of DKIM.  DKIM has worked for quite a few years.  Some details of ARC need to be different from DKIM, but the basic paradigm -- including the model of achieving field operation through interoperability testing -- should be the same.

My reading of this too-long message thread is that there is essentially no support for making ARC's header-related issues any different from DKIM's, so I encourage the chair to shut this thread down.

d/

--
Dave Crocker
Brandenburg InternetWorking
bbiw.net<https://urldefense.proofpoint.com/v2/url?u=http-3A__bbiw.net&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=Kz8VdgPVctDNSNPJ6PsBaw&m=M5zzV_Ak3GCR8lZhAHLFfMA0SvGqSztWEPI0H6a5Y2E&s=ZX1C85lFDoWiX0J86FUeLGQx37E3YQcIT3LIGyCaeWU&e=>

_______________________________________________
dmarc mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/dmarc<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_dmarc&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=Kz8VdgPVctDNSNPJ6PsBaw&m=M5zzV_Ak3GCR8lZhAHLFfMA0SvGqSztWEPI0H6a5Y2E&s=mJ57Dtkqee8RcYkQnZ9yx6EdyO8ycHhuJTmem3tZxy8&e=>

--

[mage removed by sender. logo for sig file.png]

Bringing Trust to Email

Seth Blank | Head of Product for Open Source and Protocols
[email-masked]<mailto:[email-masked]>
+1-415-894-2724<tel:415-894-2724>

---

From: Seth Blank <[email-masked]>
Date: Thu, 30 Mar 2017 18:45:34 -0700

Yes, +1000 to many test suites.

But no test suite can function without a deterministic order of fields in
the header unless it also includes an ARC implementation for
signing/validating, which is where this conversation started and why we
care so much about discussing this ordering.

On Thu, Mar 30, 2017 at 6:11 PM, HANSEN, TONY L <[email-masked]> wrote:

> One of the reasons DKIM was successful in becoming interoperable was not
> that we had a test suite to force conformance, but instead that we had
> >multiple< test suites that were able to successfully validate DKIM
> signatures from multiple implementations. My test suite caught corner cases
> that Ned’s and Yahoo’s and Google’s did not, and vice versa for each one of
> them. (My company actually had two totally independent implementations of
> DKIM so that we could do internal testing against a different platform. It
> was just prudent to do it that way.)
>
>
>
> When there was an issue in someone’s test suite, we were then able to
> discuss whether it was an issue in the test suite that needed to be fixed,
> or it was an issue in the implementation. A single test suite is never
> sufficient – that’s one reason we prefer multiple interoperating
> implementations in order for something to be considered an internet
> standard.
>
>
>
> By the way, none of us that implemented DKIM test suites had an issue with
> the header order being determined by the sender. It makes the validation
> infinitesimally trickier than having a predetermined header order.
>
>
>
> -1000 to the suggestion of having a single conformance test suite that
> everyone lives and dies by. +1000 to multiple test suites that multiple
> people write and can be used to test against.
>
>
>
> I’m all for you, Seth and Peter, to write a test suite. But nothing you’ve
> said so far has convinced me that a predetermined header order or a single
> conformance test suite is worth pursuing. One of many, sure.
>
>
>
>                 Tony Hansen
>
>
>
> *From: *dmarc <[email-masked]> on behalf of Seth Blank <
> [email-masked]>
> *Date: *Thursday, March 30, 2017 at 8:10 PM
> *To: *"[email-masked]" <[email-masked]>
> *Cc: *"Murray S. Kucherawy" <[email-masked]>, "[email-masked]" <
> [email-masked]>, Scott Kitterman <[email-masked]>, Peter Goldstein <
> [email-masked]>, "[email-masked]" <[email-masked]>
> *Subject: *Re: [dmarc-ietf] indeterminisim of ARC-Seal b= value
>
>
>
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__bbiw.net&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=Kz8VdgPVctDNSNPJ6PsBaw&m=M5zzV_Ak3GCR8lZhAHLFfMA0SvGqSztWEPI0H6a5Y2E&s=ZX1C85lFDoWiX0J86FUeLGQx37E3YQcIT3LIGyCaeWU&e=>
>
>
>
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_dmarc&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=Kz8VdgPVctDNSNPJ6PsBaw&m=M5zzV_Ak3GCR8lZhAHLFfMA0SvGqSztWEPI0H6a5Y2E&s=mJ57Dtkqee8RcYkQnZ9yx6EdyO8ycHhuJTmem3tZxy8&e=>
>
>
>
>
>
> --
>
>
> *Bringing Trust to Email*
>

-- 

Bringing Trust to Email

---

From: Dave Crocker <[email-masked]>
Date: Thu, 30 Mar 2017 21:35:18 -0500

> However, ARC is fundamentally different. It is about a chain of messages

Either you are correct, in which case ARC has been made far too fragile 
to be able to work with any serious degree of reliability at scale,

or you are wrong, in which case the fact of there being a sequence of 
DKIM-ish signatures has the same requirements as for individual 
signatures.

What you have been getting told by a range of folk with quite a lengthy 
history of DKIM and email deployment experience is in line with the latter.

So to the extent that you are sure things really are that fragile, the 
answer is not going to be a test suite or excessively demanding 
algorithms, but a re-thinking of the details, to make the implementation 
and deployment issues simpler.

d/

---

From: "John Levine" <[email-masked]>
Date: 31 Mar 2017 02:54:17 -0000

>DKIM's, so I encourage the chair to shut this thread down.

What he said.  Can we please limit subsequent discussion of testing
about how to test ARC as defined and implemented, not some not-ARC
that is not defined and certainly will not be implemented?

R's,
John

---

From: Seth Blank <[email-masked]>
Date: Thu, 30 Mar 2017 20:41:30 -0700

On Thu, Mar 30, 2017 at 7:35 PM, Dave Crocker <[email-masked]> wrote:
> answer is not going to be a test suite or excessively demanding algorithms,
> but a re-thinking of the details, to make the implementation and deployment
> issues simpler.

Exactly; we are in agreement. Where some discussion after the last interop
panned out was that deterministic behavior in the new headers would remove
this fragility. That's why the conversation was raised to this list instead
of happening off to the side.

If the consensus here is that the matter is not worth pursuing further,
that is fine - I just want to make sure we're all talking about the same
thing.

Seth

On Thu, Mar 30, 2017 at 7:35 PM, Dave Crocker <[email-masked]> wrote:

> Either you are correct, in which case ARC has been made far too fragile to
> be able to work with any serious degree of reliability at scale,
>
> DKIM-ish signatures has the same requirements as for individual signatures.
>
> answer is not going to be a test suite or excessively demanding algorithms,
> but a re-thinking of the details, to make the implementation and deployment
> issues simpler.
>
>
>
> d/
>
> --
> Dave Crocker

-- 

Bringing Trust to Email

---

From: Dave Crocker <[email-masked]>
Date: Fri, 31 Mar 2017 10:55:54 -0500

> that is fine - I just want to make sure we're all talking about the same

Except that 'the matter is not worth pursuing' isn't what I heard anyone 
saying and it definitely wasn't what I meant...

I'm not sure whether it's been presented here sufficiently, but I 
believe your underlying concern is based on observed problems with 
implementation of the current ARC specification.  That is, from 
interoperability testing, the actual use of ARC is proving far too fragile.

If that's true, then there needs to be an effort to a) understand the 
fragility better, and b) consider ways to make ARC more robust.

So while there has been strong push-back against the /solution/ that you 
proferred, I am not clear whether there is working group understanding 
of the motivating concern or with the way to resolve it.

d/

---

From: Seth Blank <[email-masked]>
Date: Fri, 31 Mar 2017 09:13:25 -0700

Thank you, Dave.

What's the best way to proceed from here for the working group?

On Fri, Mar 31, 2017 at 8:55 AM, Dave Crocker <[email-masked]> wrote:

> I'm not sure whether it's been presented here sufficiently, but I believe
> your underlying concern is based on observed problems with implementation
> of the current ARC specification.  That is, from interoperability testing,
> the actual use of ARC is proving far too fragile.
>
> proferred, I am not clear whether there is working group understanding of
> the motivating concern or with the way to resolve it.
>
>
>
> d/
>
> --
> Dave Crocker

-- 

Bringing Trust to Email

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Fri, 31 Mar 2017 15:15:35 -0500

Hi Seth,

Can you describe in detail the fragility you've observed?  I saw Peter
(sorry for "Paul", it was 4am) upthread somewhere say that a few different
projects had various problems found, but I don't recall him saying exactly
what they were.

-MSK