Subject: [dmarc-ietf] questions about section 5.1.1 of the new ARC draft

From: Gene Shuman <[email-masked]>
Date: Mon, 24 Jul 2017 17:21:20 -0700

Hi All, I'm going through the new draft.  I definitely approve of the
language changes, although some things wrt section 5.1.1 & the new data
going into AAR's are unclear to me.

So this is my best understanding, and it doesn't seem like it's right:

1. A message comes into an ADMD and does spf/dkim/etc checks & puts these
into AR's as usual.  At some point in there it performs an ARC validation
step.  First question, do the new ams1, as1, etc get added to the AR then?
 i.e do we get another other AR then, that looks like:

Authentication-Resules: list.com; arc=pass ams1.d=domain1 ams1.s=selector1,
as1.d=domain1 as1.d=selector1

Or are the additional fields only added in when the AAR is compiled?

2. The message is modified.

3. The messge is arc-sealed.  All AR's(from list.com) are concatendated,
and source.ip & header.s are added.  Is header.s added to the dkim sections
of the ARs that are getting concatenated?  This is just for dkim signatures
right?  Is ARC code expected to parse this out of dkim signatures?

Also, where does source.ip go?  Into any SPF sections of the AR?  Where
does the ARC code get this information from?

Are we expecting the earlier steps in the ADMD's processing pipeline to
stick this info somewhere(in their AR's?)

I'm fairly confused.  Am I missing something?

=Gene

---

From: "Kurt Andersen (b)" <[email-masked]>
Date: Mon, 24 Jul 2017 17:37:38 -0700

> step.  First question, do the new ams1, as1, etc get added to the AR then?

That depends on whether the message as received bore ARCset-1 on receipt.
If so, then the checking should record the needed info.

> Or are the additional fields only added in when the AAR is compiled?

Maybe (how's that for being definitive?) - there's nothing which prevents
you from sticking it in the "ingress A-R" (or any other header or offline
store) and then it's up to you to consume it appropriately when sealing the
message.

> does the ARC code get this information from?

I think that source.ip makes the most sense within the SPF space, but it
can go anywhere that you can find it when doing the sealing :-)

> stick this info somewhere(in their AR's?)

Yes, otherwise the info won't be available for inclusion in the ARC set
upon egress.

> I'm fairly confused.  Am I missing something?

ARC relies upon an ADMD having some functional (secure) way to transmit
information between ingress and egress with possible message changes in
between. That has been one of the gaps in early implementations which I
tried to address with more explicit language in section 5.3.1 ("the state
of the ARC chain. . .when it arrived at the ADMD") and in the last
paragraph of section 6 ("record the cv state for [use]. . .later. . .within
the same trust boundary").

--Kurt

---

From: Gene Shuman <[email-masked]>
Date: Mon, 24 Jul 2017 18:06:21 -0700

OK thanks.  So if I understanding this correctly, we're suggesting, within
the ARC protocol, that the ways in which both DKIM & SPF stamp their info
into AR's is changed, at least when they're part of an ARC aware pipeline?
Or is this stamping outside of the scope of the SPF & DKIM rfcs & left up
to implementors?  What are the chances we can actually get these changes
into popular DKIM & SPF libraries?  That seems necessary, right?  Unless
some other part of the ADMD is going to go in, and after the fact add this
info to the AR headers?  Who's going to write the software/milters/etc that
do that?  Or am I missing something?

=Gene

---

From: "Kurt Andersen (b)" <[email-masked]>
Date: Tue, 25 Jul 2017 08:05:54 -0700

> do that?  Or am I missing something?

I don't necessarily see this as telling the DKIM/SPF libraries anything per
se (because that is out of scope). I do think that we have some influence
over DMARC (notice the WG name :-) ) but we also don't have any official
influence over the mandatory A-R content as defined in RFC7601. I think
that we can suggest mechanisms for operators to use, subject to their
internal architectures and that A-R and its feeder systems are a fairly
natural target (or ARC sealing upon ingress and putting the enhanced
content directly into the AAR). In any case, DMARC reporting will have to
be able to find this information for reports generated past the first
mediator.

I think that this is the natural consequence of the threads that we've been
having regarding getting source debugging information into the DMARC
reports on mediated mail flows. Did you have something different in mind?

--Kurt

---

From: Scott Kitterman <[email-masked]>
Date: Tue, 25 Jul 2017 16:14:52 +0000

>reports on mediated mail flows. Did you have something different in

Insisting that all results from an ADMD for SPF, DKIM, and DMARC be collapsed into a single AR header field is a huge change from the way AR works today.  AR was designed for each mechanism to be able to add new information about its results.

Yes, AR has, from the beginning, supported integrating multiple results into a single field and some do it, but requiring is an entirely different thing.

What this would mean is that an ARC participant would have to remove and consolidate multiple fields on receipt (the original sender has no requirement to be ARC aware AIUI).

If I'm understanding correctly, this is not a good idea.  ARC should consume AR fields just like everyone else.

Scott K

---

From: "Kurt Andersen (b)" <[email-masked]>
Date: Tue, 25 Jul 2017 09:22:11 -0700

On Tue, Jul 25, 2017 at 9:14 AM, Scott Kitterman <[email-masked]>
wrote:

>
> Insisting that all results from an ADMD for SPF, DKIM, and DMARC be
> collapsed into a single AR header field is a huge change from the way AR
> works today.  AR was designed for each mechanism to be able to add new
> information about its results.
>

There is no requirement to do so. The only thing is that for the AAR, all
of the *locally trustworthy* AR headers get collapsed into a single AAR.

> What this would mean is that an ARC participant would have to remove and
> consolidate multiple fields on receipt (the original sender has no
> requirement to be ARC aware AIUI).
>

An ARC participant should not, upon ingress, pay any heed to AR records
that might have been put there by previous ADMDs so I'm not quite sure what
you are referring to (maybe the language that I used was unclear?)

> If I'm understanding correctly, this is not a good idea.  ARC should
> consume AR fields just like everyone else.
>

Maybe you're thinking of ARC signing on *egress*? Somehow, the signing
instance needs to know about the state of the authentication upon ingress
to the ADMD. AR may work, or may not - it depends on the operator's
implementation. I know of some MTAs that strip AR records without a means
to tell it which ones to leave alone. If those live within the internal
pathway, then AR is not a workable approach for communication from ingress
to egress.

--Kurt

---

From: Gene Shuman <[email-masked]>
Date: Tue, 25 Jul 2017 12:31:38 -0700

So I think I'm still missing something.  Let's imagine an ADMD setup in
something like the following fashion:

inbound -> policyspf(or some other spf check)  -> OpenDKIM -> OpenDMARC ->
OpenARC Validation -> mailing list software -> OpenArc Signature ->
outbound.

Or something like this.  Lets say policyspf, OpenDKIM, OpenDMARC, OpenARC,
all stamp the standard info into various AR headers on ingress.  On egress,
in the OpenARC signature milter, we then consolidate all AR's into the
AAR.  In this stage we also need source.ip & header.s for DKIM signatures.
Where is this information coming from?  Who is recording the source.ip in
this chain?  It's not policy-spf, or anybody further down the chain, as I
don't believe this information is propagated.  Would this need to be a new
piece of software somewhere in the chain adding this information to an
additional AR header, or modifying the one added by policy-spf?

And who is responsible for parsing header.s out of DKIM signatures?
OpenARC?  Or an additional piece of software?  And in addition, it will
need to be put in the dkim, or one of the dkim, sections of the newly
aggregated AAR, which will require more software.

This situation seems very complicated to me, which makes me think that I
don't understand it.  Either that or it is complicated, and possibly
unreasonably so.  I'm not sure.

I do understand, and entirely support, the need/desire for this
information.  But the practical details of how we're doing this elude me.

-Gene

On Tue, Jul 25, 2017 at 9:22 AM, Kurt Andersen (b) <[email-masked]> wrote:

> _______________________________________________
> dmarc mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/dmarc
>
>

---

From: Juri Haberland <[email-masked]>
Date: Tue, 25 Jul 2017 23:44:20 +0200

> OpenARC Validation -> mailing list software -> OpenArc Signature ->

Well, OpenARC validation should be before OpenDMARC...

> additional AR header, or modifying the one added by policy-spf?

All milter see the original IP and port, so OpenArc could report that.

> aggregated AAR, which will require more software.

In my understanding OpenArc would validate the ARC set. OpenDMARC would
need to parse the first AAR if DKIM and SPF fail and ARC passes.

Um, right?

  Juri

---

From: Gene Shuman <[email-masked]>
Date: Tue, 25 Jul 2017 15:01:31 -0700

> All milter see the original IP and port, so OpenArc could report that.

OK that's good to know.  Still, it needs to be parsing an SPF AR &
inserting the source.ip, which is doable, but work.

OpenARC validation is creating no signatures.  Therefore OpenDMARC will
have no access to an new AARs(from the current hop)

OpenARC though, is now expected to parse incoming DKIM signatures as well,
and extract the s= tag,, right? And then rebuild & reinsert this tag into
the AR generated by OpenDKIM?  This is possible, but again, a very non
trivial amount of work, as it's very much outside of the scope of what
we're doing currently.  And also seems kind of crazy.

The situation with non-milter based approaches is even worse.  dkimpy &
mailman don't even have access to source.ip, so that's definitely out.  And
again, dkimpy would need to parse DKIM signatures, extract info & stick it
somewhere.  Again, a bit convoluted.

Brandon, have you thought about this much?

=Gene

---

From: Gene Shuman <[email-masked]>
Date: Tue, 25 Jul 2017 15:09:38 -0700

---

From: "Kurt Andersen (b)" <[email-masked]>
Date: Tue, 25 Jul 2017 15:27:49 -0700

> information.  But the practical details of how we're doing this elude me.

If you have a clearer, cleaner, and less convoluted mechanism to obtain
this information and pass it through the series of intermediaries to get
into the DMARC report, I'm happy to update the spec along those lines :-)

--Kurt

---

From: "Kurt Andersen (b)" <[email-masked]>
Date: Tue, 25 Jul 2017 15:29:47 -0700

> Well, OpenARC validation should be before OpenDMARC...

Well, that depends on your implementation. If you generally evaluate local
policy conditions before checking the primary DMARC mechanisms, then yes;
otherwise, ARC only needs to be evaluated if DMARC does not achieve a PASS
from the primary mechanism or if you expect to break the message's
authentication.

--Kurt

---

From: Juri Haberland <[email-masked]>
Date: Wed, 26 Jul 2017 00:30:37 +0200

> have no access to an new AARs(from the current hop)

OpenARC will create an A-R header, which can be parsed by OpenDMARC.

> we're doing currently.  And also seems kind of crazy.

I don't see the problem:
OpenARC (or whatever) checks the ARC chain and if it passes, the first AAR
header should be parsed by OpenDMARC (or somesuch). That's all.

Reporting (DMARC) is a different matter, though, but let's concentrate on
one problem after the other...

  Juri

---

From: Juri Haberland <[email-masked]>
Date: Wed, 26 Jul 2017 00:36:28 +0200

> from the primary mechanism or if you expect to break the message's

Granted, both ways are possible :)

  Juri

---

From: Gene Shuman <[email-masked]>
Date: Tue, 25 Jul 2017 15:48:19 -0700

I do not have any better ideas.  I just wanted to make sure I understood
the situation correctly atm.

On Tue, Jul 25, 2017 at 3:36 PM, Juri Haberland <[email-masked]>
wrote:

> >> > inbound -> policyspf(or some other spf check)  -> OpenDKIM ->
> OpenDMARC
> >> ->

---

From: Brandon Long <[email-masked]>
Date: Mon, 31 Jul 2017 16:20:25 -0700

Catching up here...

Perhaps the traditional separation of all of the mail authenticators is
what should change, ie if we combined the various handlers into one, then
it becomes much more obvious how the information flow would occur.

For our impl, it's just various calls within the single process, hence why
it's easy to issue a single authres header as well.

Looking at the 5.1.1 section, I do think we should specify that the
source.ip is either in the arc section or the spf section, and not leave
that to random choice.

I also do not like the arc related data as specified.  If the arc chain is
intact, then the information is trivially extracted from the chain, if it's
not intact, then who cares.
Even if we want to include this data, I think the fact that we have a chain
of data could be used, and therefore we would only include the i-1 hop data
in the single aar.

Arc-Authentication-Results: i=3; mx.google.com; arc=pass ams.d=google.com
ams.s=arc as.d=google.com as.s=arc

The spec currently says i can go up to 1024 but unlikely over 50, are we
going to generate:
Arc-Authentication-Results: i=50; mx.google.com;
    arc=pass ams49.d=google.com ams49.s=arc-20160816 as49.d=google.com
ams49.s=arc-20160816
    ams48.d=google.com ams48.s=arc-20160816 as48.d=google.com
ams48.s=arc-20160816
    ams47.d=google.com ams47.s=arc-20160816 as47.d=google.com
ams47.s=arc-20160816
    ams46.d=google.com ams46.s=arc-20160816 as46.d=google.com
ams46.s=arc-20160816
    ams45.d=google.com ams45.s=arc-20160816 as45.d=google.com
ams45.s=arc-20160816
    ams44.d=google.com ams44.s=arc-20160816 as44.d=google.com
ams44.s=arc-20160816
    ams43.d=google.com ams43.s=arc-20160816 as43.d=google.com
ams43.s=arc-20160816
    ams42.d=google.com ams42.s=arc-20160816 as42.d=google.com
ams42.s=arc-20160816
    ams41.d=google.com ams41.s=arc-20160816 as41.d=google.com
ams41.s=arc-20160816
    ams40.d=google.com ams40.s=arc-20160816 as40.d=google.com
ams40.s=arc-20160816
    ams39.d=google.com ams39.s=arc-20160816 as39.d=google.com
ams39.s=arc-20160816
    ams38.d=google.com ams38.s=arc-20160816 as38.d=google.com
ams38.s=arc-20160816
    ams37.d=google.com ams37.s=arc-20160816 as37.d=google.com
ams37.s=arc-20160816
    ams36.d=google.com ams36.s=arc-20160816 as36.d=google.com
ams36.s=arc-20160816
    ams35.d=google.com ams35.s=arc-20160816 as35.d=google.com
ams35.s=arc-20160816
    ams34.d=google.com ams34.s=arc-20160816 as34.d=google.com
ams34.s=arc-20160816
    ams33.d=google.com ams33.s=arc-20160816 as33.d=google.com
ams33.s=arc-20160816
    ams32.d=google.com ams32.s=arc-20160816 as32.d=google.com
ams32.s=arc-20160816
    ams31.d=google.com ams31.s=arc-20160816 as31.d=google.com
ams31.s=arc-20160816
    ams30.d=google.com ams30.s=arc-20160816 as30.d=google.com
ams30.s=arc-20160816
    ams29.d=google.com ams29.s=arc-20160816 as29.d=google.com
ams29.s=arc-20160816
    ams28.d=google.com ams28.s=arc-20160816 as28.d=google.com
ams28.s=arc-20160816
    ams27.d=google.com ams27.s=arc-20160816 as27.d=google.com
ams27.s=arc-20160816
    ams26.d=google.com ams26.s=arc-20160816 as26.d=google.com
ams26.s=arc-20160816
    ams25.d=google.com ams25.s=arc-20160816 as25.d=google.com
ams25.s=arc-20160816
    ams24.d=google.com ams24.s=arc-20160816 as24.d=google.com
ams24.s=arc-20160816
    ams23.d=google.com ams23.s=arc-20160816 as23.d=google.com
ams23.s=arc-20160816
    ams22.d=google.com ams22.s=arc-20160816 as22.d=google.com
ams22.s=arc-20160816
    ams21.d=google.com ams21.s=arc-20160816 as21.d=google.com
ams21.s=arc-20160816
    ams20.d=google.com ams20.s=arc-20160816 as20.d=google.com
ams20.s=arc-20160816
    ams19.d=google.com ams19.s=arc-20160816 as19.d=google.com
ams19.s=arc-20160816
    ams18.d=google.com ams18.s=arc-20160816 as18.d=google.com
ams18.s=arc-20160816
    ams17.d=google.com ams17.s=arc-20160816 as17.d=google.com
ams17.s=arc-20160816
    ams16.d=google.com ams16.s=arc-20160816 as16.d=google.com
ams16.s=arc-20160816
    ams15.d=google.com ams15.s=arc-20160816 as15.d=google.com
ams15.s=arc-20160816
    ams14.d=google.com ams14.s=arc-20160816 as14.d=google.com
ams14.s=arc-20160816
    ams13.d=google.com ams13.s=arc-20160816 as13.d=google.com
ams13.s=arc-20160816
    ams12.d=google.com ams12.s=arc-20160816 as12.d=google.com
ams12.s=arc-20160816
    ams11.d=google.com ams11.s=arc-20160816 as11.d=google.com
ams11.s=arc-20160816
    ams10.d=google.com ams10.s=arc-20160816 as10.d=google.com
ams10.s=arc-20160816
    ams9.d=google.com ams9.s=arc-20160816 as9.d=google.com
ams9.s=arc-20160816
    ams8.d=google.com ams8.s=arc-20160816 as8.d=google.com
ams8.s=arc-20160816
    ams7.d=google.com ams7.s=arc-20160816 as7.d=google.com
ams7.s=arc-20160816
    ams6.d=google.com ams6.s=arc-20160816 as6.d=google.com
ams6.s=arc-20160816
    ams5.d=google.com ams5.s=arc-20160816 as5.d=google.com
ams5.s=arc-20160816
    ams4.d=google.com ams4.s=arc-20160816 as4.d=google.com
ams4.s=arc-20160816
    ams3.d=google.com ams3.s=arc-20160816 as3.d=google.com
ams3.s=arc-20160816
    ams2.d=google.com ams2.s=arc-20160816 as2.d=google.com
ams2.s=arc-20160816
    ams1.d=google.com ams1.s=arc-20160816 as1.d=google.com
ams1.s=arc-20160816

I'd also point out that this is iterative, so the i=49 one is only slightly
smaller and so on.

Brandon

---

From: Gene Shuman <[email-masked]>
Date: Mon, 31 Jul 2017 16:25:25 -0700

I was just talking with Seth about this.  I think the language was poorly
worded, and the intention is not to have a million ams.s, ams.ds in the
aar, but just the ones from the most recent hop.  Is this correct?

Also, I still dont like ARC implementations being asked to parse DKIM
signatures to get header.s.  Nor do I really understand why we want that
info.

On Mon, Jul 31, 2017 at 4:20 PM, Brandon Long <[email-masked]> wrote:

> Even if we want to include this data, I think the fact that we have a
> chain of data could be used, and therefore we would only include the i-1
> hop data in the single aar.
>
> I'd also point out that this is iterative, so the i=49 one is only
> slightly smaller and so on.
>
> Brandon
>
>>> > On Tue, Jul 25, 2017 at 2:44 PM, Juri Haberland <[email-masked]
>>> >
>>> > wrote:

---

From: Brandon Long <[email-masked]>
Date: Mon, 31 Jul 2017 16:31:05 -0700

In that case, I dislike specifying the hop number in the ptype, given the
registry nature of the ptype and property, it seems like they should be
specific and not an enumerated value.

Brandon

On Mon, Jul 31, 2017 at 4:25 PM, Gene Shuman <[email-masked]> wrote:

>> For our impl, it's just various calls within the single process, hence
>> why it's easy to issue a single authres header as well.
>>
>> I also do not like the arc related data as specified.  If the arc chain
>> is intact, then the information is trivially extracted from the chain, if
>> it's not intact, then who cares.

---

From: Gene Shuman <[email-masked]>
Date: Mon, 31 Jul 2017 16:38:42 -0700

I understand exactly why we want header.s.  But I think I might have
incorrectly assumed that getting this from the arc signatures was good
enough.

On Mon, Jul 31, 2017 at 4:31 PM, Brandon Long <[email-masked]> wrote:

>>> Arc-Authentication-Results: i=3; mx.google.com; arc=pass ams.d=
>>> google.com ams.s=arc as.d=google.com as.s=arc
>>>
>>>> I do not have any better ideas.  I just wanted to make sure I
>>>> understood the situation correctly atm.
>>>>

---

From: Gene Shuman <[email-masked]>
Date: Mon, 31 Jul 2017 16:46:07 -0700

But I understand now, simple misunderstanding.

So is this assumption correct?  An AAR should look like this:

Arc-Authentication-Results: i=14; mx.google.com;
    arc=pass ams.d=google.com ams.s=arc-20160816 as.d=google.com
 ams.s=arc-20160816

Maybe with some additional dkim/spf/dmarc info.  This seems reasonable
enough to me

On Mon, Jul 31, 2017 at 4:38 PM, Gene Shuman <[email-masked]> wrote:

>>> I was just talking with Seth about this.  I think the language was
>>> poorly worded, and the intention is not to have a million ams.s, ams.ds in
>>> the aar, but just the ones from the most recent hop.  Is this correct?
>>>
>>>> The spec currently says i can go up to 1024 but unlikely over 50, are
>>>> we going to generate:
>>>>> On Tue, Jul 25, 2017 at 3:36 PM, Juri Haberland <[email-masked]
>>>>> > wrote:
>>>>>

---

From: Gene Shuman <[email-masked]>
Date: Mon, 31 Jul 2017 16:53:15 -0700

Sorry, sent from the wrong email addr.  To repeat:

Arc-Authentication-Results: i=14; mx.google.com;
    arc=pass ams.d=google.com ams.s=arc-20160816 as.d=google.com ams.s=arc-
20160816

On Mon, Jul 31, 2017 at 4:46 PM, Gene Shuman <[email-masked]> wrote:

>>> In that case, I dislike specifying the hop number in the ptype, given
>>> the registry nature of the ptype and property, it seems like they should be
>>>>> Perhaps the traditional separation of all of the mail authenticators
>>>>> is what should change, ie if we combined the various handlers into one,
>>>>> then it becomes much more obvious how the information flow would occur.
>>>>>
>>>>> I also do not like the arc related data as specified.  If the arc
>>>>> chain is intact, then the information is trivially extracted from the
>>>>> chain, if it's not intact, then who cares.
>>>>>> On Tue, Jul 25, 2017 at 3:36 PM, Juri Haberland <
>>>>>> [email-masked]> wrote:
>>>>>>

---

From: Brandon Long <[email-masked]>
Date: Mon, 31 Jul 2017 16:58:00 -0700

>     arc=pass ams.d=google.com ams.s=arc-20160816 as.d=google.com

I wonder if more in keeping with the ptypes, that it would be:

Arc-Authentication-Results: i=14; mx.google.com;
     arc=pass header.ams-d=google.com header.ams-s=arc-20160816
header.as-d=google.com header.as-s=arc-20160816

There are folks on this list who better understand authres who may
have a stronger opinion.

Brandon

>>>>>> is what should change, ie if we combined the various handlers into one, then
>>>>>> chain is intact, then the information is trivially extracted from the chain,
>>>>>> if it's not intact, then who cares.
>>>>>> chain of data could be used, and therefore we would only include the i-1 hop
>>>>>> data in the single aar.
>>>>>>
>>>>>> Arc-Authentication-Results: i=3; mx.google.com; arc=pass
>>>>>> ams.d=google.com ams.s=arc as.d=google.com as.s=arc
>>>>>>
>>>>>>> On Tue, Jul 25, 2017 at 3:36 PM, Juri Haberland
>>>>>>> <[email-masked]> wrote:
>>>>>>>>

---

From: "Kurt Andersen (b)" <[email-masked]>
Date: Mon, 31 Jul 2017 22:03:16 -0700

On Mon, Jul 31, 2017 at 4:58 PM, Brandon Long <[email-masked]> wrote:

> > To repeat:

My interpretation of the conversations in Prague with Seth was that he was
looking to have the (ridiculously) long list of all the chain components;
but I'll take responsibility for misunderstanding that one :-)

I think that what Brandon is proposing for the "only ARC" portion of the
AAR is reasonable and will update the draft accordingly unless I hear
otherwise from other folks.

I'm also in the midst of incorporating responses to Alexy and Dave's inputs.

--Kurt