Subject: Re: [dmarc-ietf] Proposed ARC "Experimental Considerations" section

From: Hector Santos <[email-masked]>
Date: Fri, 01 Dec 2017 08:47:11 -0500

On 11/29/2017 12:17 PM, Kurt Andersen (b) wrote:
>
> While I have a number of issues with the details of the proposal, I'll
> tackle those in another thread. The fundamental problem that I have
> with the whole "experiment" approach is that it is something like
> throwing a baseball into a pool of quiescent sharks. I have no
> confidence that "hoping" for people to do something that a random
> group of people propose in an RFC will bear any fruit at all. It's not
> an experiment, it's an exercise in futility.

Hi Ken,

The MARID experiment between SPF and SENDERID worked with random 
groups of people in the community/industry/market.  The "better 
protocol" eventually prevailed.  So it us possible this experimental 
approach for ARC is doable, albeit it will take a long time.

However, it was pretty clear that SPF/SenderID offered a strong proof 
of concept. It was pretty solid what the intent was, how it was going 
to work, and it was easy to implement. With ARC, we don't have such 
comfort, in my opinion.  Making it a proposed standard isn't going to 
increase its endorsement when it lacks technical merit.

Where does ARC fit in the DKIM Service Architecture, RFC5585?
https://tools.ietf.org/html/rfc5585#section-5

How does it fit with basic "DKIM Author Domain" policy model, in this 
case DMARC?   That isn't clear and until then, I'm afraid it will be 
left in limbo.

-- 
HLS

---

From: Hector Santos <[email-masked]>
Date: Fri, 01 Dec 2017 08:49:40 -0500

Who's Ken?  :)  My apology Kurt.

-- 
HLS

---

From: Dave Crocker <[email-masked]>
Date: Fri, 1 Dec 2017 09:18:14 -0800

Nicely done.  Reads like a diligent effort and seems to indeed cover 
things nicely.  Not that changes won't help -- as the thread already 
suggests -- but has a good template and good content.

As for possible changes...

I agree that it needs to be cast primarily as a DKIM value-added 
mechanism, where reference to DMARC is limited.  Not zero, but not 
wholly dependent.

As for 'experiment', again, the template here casts things useful in 
that regard.  Whether the community will engage with the necessary due 
diligence is always an issue.

But as I've noted before, basing delivery decisions on the 
authentication information done by intermediaries is new territory and 
since it would seem to be an invitation to spoofing, we should treat its 
initial use as needing very careful analysis.  Errr... experimentation.

To the details...

On 11/22/2017 9:34 PM, Seth Blank wrote:
> Since we're aiming for an Experimental document, it was discussed that 
> it makes sense to define that experiment. I'm proposing the following 
> section, and suggest it be the final section after Security 
> Considerations but before References.
> 
> For this section, I'm taking the stance "the best way to get the right 
> answer online is to give the wrong answer online." The one thing I know 
> is that what I've proposed below is wrong. It's just a matter of how, 
> why, and to what degree.
> 
> I'm also certain I'm missing critical design decisions, and the success 
> criteria probably needs some hard watermark (like 90% of mail that 
> fails/was subject to override now passes due to ARC alone).
> 
> Anyway, let's attack this straw man to figure out what the right form of 
> this section should be.
> 
> Seth
> 
> --------
> 
> 16 Experimental Considerations
> 
> [[ NOTE TO WORKING GROUP: Should this section be for the IESG only to be 
> removed by the document editor, or should it stay with the document as 
> long as it’s experimental? ]]

Not sure what is typical.  Having this text in the document means it 
will require reissuing the document when it transitions from 
Experimental to Standards track.  Theoretically, a change in status 
doesn't require a new RFC.  However it's almost always true that the 
document benefits from other changes over that sort of time, so there 
isn't any negative of having the text in the base document.  The 
benefit, of course, is that it is more accessible, to more people.

> It must be demonstrated that ARC actually solves the problem it is 
> supposed to - mainly, that ARC provides an effective signal to a Final 
> Receiver that allows messages indirectly delivered to properly be 
> rescued after a DMARC failure.

      It must be demonstrated that ARC provides utility in recovering 
from a validation failure for a DKIM signature by an originator.  That 
is, can a final receiver perform useful validation of the message 
handling, to determine delivery disposition?  This is especially 
relevant for DMARC processing when the DKIM d= value is aligned with the 
rfc5322.From author domain.

Tossing in a distinct statement about DMARC acknowledged the issue that 
motivated this work, without putting it into the critical path for ARC. /d

> This section defines what success and failure look like, the protocol 
> design decisions that influence those criteria, and open issues that the 
> experiment should shine light on.
> 
> 16.1 Design Decisions
> 
> 16.1.1 Trace information is valuable
> 
> Because it is unclear exactly what information will be meaningful to 
> Final Receivers to make delivery decisions, it was decided that the 
> protocol would lean towards providing more information than less.

Simple, useful bit of design background.  Nice.  /d

> 16.1.2 Chains can’t be restarted
> 
> Originally, it was expected that ARC Chains would be restartable. 
> However, this turned out to be a deeply complicated and implementation 
> dependent mechanism, for a use case that wasn’t clear would ever occur. 
> The ARC-Seal was necessary when restarting a chain to guarantee that the 
> restarter knew everyone who had been involved in the chain previously.

      The ARC-Seal might prove to be unnecessary.  It is provided to 
provide additional analysis by the final receiver.

The issue of restarting, per se, seems a distraction here.  Useful 
history, but not really relevant to this section. /d

> 16.2 Success Criteria
> 
> Currently, many receivers have heuristic based overrides in place to 
> attempt to rescue mail from DMARC failures that they believe have been 
> delivered indirectly so as not to reject or junk mail their users 
> legitimately wish to receive.

      Currently, some receivers apply heuristics to augment the handling 
of DKIM-related validation failures.  ARC is intended to permit more 
reliable and accurate handling of such cases.  So the primary criterion 
for ARC success will be demonstrating a good cost/benefit result for 
that improvement.

> 
> ARC will be a success if, for ARC Sealed messages, receivers show equal 
> or better delivery rates than achieved through their overrides.
> 
> 16.3 Failure Criteria
> 
> The intent of ARC is to be at most value-add and at worst benign. If ARC 
> opens up new vectors for abuse - beyond DKIM replay attacks and other 
> known issues (see:Security Considerations) inherent in the mail 
> protocols ARC is built upon - then ARC will be a failure.

Having 'failure criteria' is interesting.  Hadn't occurred to me but 
seems useful for the template and the explicit reference to ARC possibly 
making a new attack surface seems appropriate. /d

> 16.4 Open Questions

    Questions To Be Resolved By The Experiment

I agree with Murray's suggestion, which means also dropping the 
following paragraph. /d

> The following open questions are academic and have no clear answer at 
> the time of the writing of this document. However, the ARC experiment 
> should gather the necessary data to conclusively answer some or all of them.
> 
> 16.4.1 Value of ARC Seal
> 
> Because the ARC Seal was initially intended to provide context when 
> restarting a chain, now that restarting the chain is not something ARC 
> attempts to do, the value of the ARC Seal appears limited.

I think the above paragraph isn't needed. /d

> As part of the ARC experiment, data should be collected to show if the 
> ARC Seal provides value beyond the ARC Message Signature for either 
> making delivery decisions or catching malicious actors trying to craft 
> or replay malicious chains.

      Data should be collected to show whether the ARC Seal provides 
significant value, beyond that of the ARC Message Signauture or, for 
that matter, beyond simple DKIM signing.

taking Murray's point. /d

> 16.4.2 DNS Overhead
> 
> The longer an ARC Chain, the more queries that are needed to retrieve 
> keys to validate the Chain. It is not believed this will be a security 
> issue (see Security Considerations:DNS Attacks), but it is unclear how 
> much overhead will truly be added.

      A longer ARC Chain requires more queries for key retrieval.  It is 
believed this will not be a security issue...

> As part of the ARC experiment, data should be collected to better 
> understand the DNS impact of ARC Chains.

    DNS impact of  ->  DNS-related costs of creating an evaluating

> 16.4.3 What trace information is valuable
> 
> There are several edge cases where the information in the AAR can make 
> the difference between message delivery or rejection. For example, if 
> there is a well known mailing list that ARC Seals but doesn’t do its own 
> initial DMARC checks, a Final Receiver with this knowledge could make a 
> delivery decision based upon the authentication information it sees in 
> AAR[1].
> 
> Certain trace information in the AAR is useful in the construction of 
> DMARC reports.
> 
> Furthermore, certain large receivers believe the entire set of trace 
> information would be valuable to feed into machine learning system to 
> shine light on fraud or determine other signals related to message delivery.

      Given the proprietary nature of such value-added processing, it is 
likely that the only public reporting on this aspect of the experiment 
will be some form of yes/no assertion by these receivers.

> However, it is unclear what trace information will be consistently 
> valuable for all receivers, regardless of size.
> 
> As part of the ARC experiment, data should be collected on what trace 
> information receivers are using that provide signal that affects 
> deliverability, and what portions of the trace data are left untouched 
> or provide no useful information.

-- 
Dave Crocker
Brandenburg InternetWorking
bbiw.net