Subject: [dmarc-ietf] Proposed ARC "Experimental Considerations" section

From: Seth Blank <[email-masked]>
Date: Thu, 23 Nov 2017 05:34:50 +0000

Since we're aiming for an Experimental document, it was discussed that it
makes sense to define that experiment. I'm proposing the following section,
and suggest it be the final section after Security Considerations but
before References.

For this section, I'm taking the stance "the best way to get the right
answer online is to give the wrong answer online." The one thing I know is
that what I've proposed below is wrong. It's just a matter of how, why, and
to what degree.

I'm also certain I'm missing critical design decisions, and the success
criteria probably needs some hard watermark (like 90% of mail that
fails/was subject to override now passes due to ARC alone).

Anyway, let's attack this straw man to figure out what the right form of
this section should be.

Seth

--------

16 Experimental Considerations

[[ NOTE TO WORKING GROUP: Should this section be for the IESG only to be
removed by the document editor, or should it stay with the document as long
as it’s experimental? ]]

It must be demonstrated that ARC actually solves the problem it is supposed
to - mainly, that ARC provides an effective signal to a Final Receiver that
allows messages indirectly delivered to properly be rescued after a DMARC
failure.

This section defines what success and failure look like, the protocol
design decisions that influence those criteria, and open issues that the
experiment should shine light on.

16.1 Design Decisions

16.1.1 Trace information is valuable

Because it is unclear exactly what information will be meaningful to Final
Receivers to make delivery decisions, it was decided that the protocol
would lean towards providing more information than less.

16.1.2 Chains can’t be restarted

Originally, it was expected that ARC Chains would be restartable. However,
this turned out to be a deeply complicated and implementation dependent
mechanism, for a use case that wasn’t clear would ever occur. The ARC-Seal
was necessary when restarting a chain to guarantee that the restarter knew
everyone who had been involved in the chain previously.

16.2 Success Criteria

Currently, many receivers have heuristic based overrides in place to
attempt to rescue mail from DMARC failures that they believe have been
delivered indirectly so as not to reject or junk mail their users
legitimately wish to receive.

ARC will be a success if, for ARC Sealed messages, receivers show equal or
better delivery rates than achieved through their overrides.

16.3 Failure Criteria

The intent of ARC is to be at most value-add and at worst benign. If ARC
opens up new vectors for abuse - beyond DKIM replay attacks and other known
issues (see:Security Considerations) inherent in the mail protocols ARC is
built upon - then ARC will be a failure.

16.4 Open Questions

The following open questions are academic and have no clear answer at the
time of the writing of this document. However, the ARC experiment should
gather the necessary data to conclusively answer some or all of them.

16.4.1 Value of ARC Seal

Because the ARC Seal was initially intended to provide context when
restarting a chain, now that restarting the chain is not something ARC
attempts to do, the value of the ARC Seal appears limited.

As part of the ARC experiment, data should be collected to show if the ARC
Seal provides value beyond the ARC Message Signature for either making
delivery decisions or catching malicious actors trying to craft or replay
malicious chains.

16.4.2 DNS Overhead

The longer an ARC Chain, the more queries that are needed to retrieve keys
to validate the Chain. It is not believed this will be a security issue
(see Security Considerations:DNS Attacks), but it is unclear how much
overhead will truly be added.

As part of the ARC experiment, data should be collected to better
understand the DNS impact of ARC Chains.

16.4.3 What trace information is valuable

There are several edge cases where the information in the AAR can make the
difference between message delivery or rejection. For example, if there is
a well known mailing list that ARC Seals but doesn’t do its own initial
DMARC checks, a Final Receiver with this knowledge could make a delivery
decision based upon the authentication information it sees in AAR[1].

Certain trace information in the AAR is useful in the construction of DMARC
reports.

Furthermore, certain large receivers believe the entire set of trace
information would be valuable to feed into machine learning system to shine
light on fraud or determine other signals related to message delivery.

However, it is unclear what trace information will be consistently valuable
for all receivers, regardless of size.

As part of the ARC experiment, data should be collected on what trace
information receivers are using that provide signal that affects
deliverability, and what portions of the trace data are left untouched or
provide no useful information.

---

From: Hector Santos <[email-masked]>
Date: Fri, 24 Nov 2017 11:39:08 -0500

On 11/23/2017 12:34 AM, Seth Blank wrote:
> [[ NOTE TO WORKING GROUP: Should this section be for the IESG only to
> be removed by the document editor, or should it stay with the document
> as long as it’s experimental? ]]
>
> It must be demonstrated that ARC actually solves the problem it is
> supposed to - mainly, that ARC provides an effective signal to a Final
> Receiver that allows messages indirectly delivered to properly be
> rescued after a DMARC failure.

Fwiw .....

DMARC is an DKIM Author Domain Policy System.  A DMARC 
(p=reject/quarantine) policy failure is Author Domain defined.  Hence 
an ARC "signal" to correct this failure must also be Author Domain 
defined, otherwise there will exist a security loop hole.

The DMARC should have an dedicator, i.e. tag extension, that allows 
the Author Domain to publish and declare to the world receivers that 
some sort of successful ARC calculation (TBD) promotes the DMARC 
failure to a "pass" (or non-fail) condition.

Conversely, DMARC should also have an dedicator (by default) that ARC 
(or anything else) should not correct/rescue DMARC mail failures.

To avoid adding DMARC tags,  possibly the first ARC seal created by 
the original DMARC author domain could be a method to trigger the ARC 
corrections.  Something like:

     If the receiver detects a DMARC failure, but also sees
     a valid ARC seal where seal #1 is tied to the author domain,
     then the receiver MAY promote the failure to pass.

-- 
HLS

---

From: Seth Blank <[email-masked]>
Date: Fri, 24 Nov 2017 21:52:49 +0000

On Fri, Nov 24, 2017 at 4:39 PM, Hector Santos <[email-masked]> wrote:
> (p=reject/quarantine) policy failure is Author Domain defined.  Hence an
> ARC "signal" to correct this failure must also be Author Domain defined,
> otherwise there will exist a security loop hole.
>

Agreed with you that the language needs to be less DMARC-specific. ARC
should assist any authentication mechanism that loses signal due to
transmission via an intermediary. I'll clarify the language.

---

From: "Kurt Andersen (b)" <[email-masked]>
Date: Wed, 29 Nov 2017 17:17:56 +0000

> that what I've proposed below is wrong. It's just a matter of how, why, and

While I have a number of issues with the details of the proposal, I'll
tackle those in another thread. The fundamental problem that I have with
the whole "experiment" approach is that it is something like throwing a
baseball into a pool of quiescent sharks. I have no confidence that
"hoping" for people to do something that a random group of people propose
in an RFC will bear any fruit at all. It's not an experiment, it's an
exercise in futility.

--Kurt

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Thu, 30 Nov 2017 23:12:49 -0800

> in an RFC will bear any fruit at all. It's not an experiment, it's an

I suppose that depends on the audience.  If the audience is motivated,
something will happen.

But what would you hope sharks would do with a baseball anyway?

-MSK

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Thu, 30 Nov 2017 23:21:16 -0800

Omitting stylistic nits for now:

> removed by the document editor, or should it stay with the document as long

For what it's worth, RFC6541 was also an experimental RFC I did some time
ago and it had a section that described (albeit briefly) what the
experiment process would be.  The AD sponsoring that document suggested it,
and it seems to have worked.  So I think leaving something like this in the
document through to publication is a good idea.

[...]
> everyone who had been involved in the chain previously.

I think a reader without context won't know what "restarting" a chain means.

[...]
> 16.4 Open Questions

Maybe "Questions To Be Resolved By The Experiment" and then drop the prose.

16.4.1 Value of ARC Seal
> attempts to do, the value of the ARC Seal appears limited.

I don't know what this means without describing "restart" (as above).

> delivery decisions or catching malicious actors trying to craft or replay

... or indeed, beyond DKIM.

> decision based upon the authentication information it sees in AAR[1].

It might also be worth noting in the experiment what ARC provides to those
operators with large knowledge bases, like the one to which you alluded
here, and those without.

> light on fraud or determine other signals related to message delivery.

Given that this involves secret sauce on the part of such operators, the
best they can provide is an anecdote here, not actual data.

-MSK