Subject: [dmarc-ietf] ... and two more tiny nits, while I'm at it

From: Anne Bennett <[email-masked]>
Date: Fri, 16 Jan 2015 19:41:29 -0500

Having just spent several hours poring over this document
(-12), I might as well send my additional minor observations.
I suspect that some of you will consider these items trivial,
but they gave me pause as I went back and forth through several
sections of the text to make sure I understood correctly.  So...

In "6.6.2. Determine Handling Policy", items 3 and 4, it
would be helpful to make it clear whether only "passed" checks
are passed back from SPF and DKIM to DMARC modules, or only
"pass/fail", or all results including temporary errors.

In "6.6.3 Policy discovery", item 3, I think you mean that
the OD must be looked up if AND ONLY IF the set is now empty.
Otherwise, one does run the risk of ending up with several
records, which item 5 implies is erroneous.

Anne.
-- 
Ms. Anne Bennett, Senior Sysadmin, ENCS, Concordia University, Montreal H3G 1M8
[email-masked]                                    +1 514 848-2424 x2285

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Fri, 16 Jan 2015 20:08:36 -0800

Hello Anne,

On Fri, Jan 16, 2015 at 4:41 PM, Anne Bennett <[email-masked]>
wrote:

> [...]

I think all of the points in your three messages are good input for a more
solid specification, but the timing is unfortunate as we just got
publication approval for -12 a week ago.  Making more changes post-approval
would probably not be a good idea, and by my reading none of them rise to
the level of being urgent to correct.

The plan for the DMARC working group is to consider, among other things,
whether it wants to produce a new version of the base document on the
Standards Track (because of its path to publication, -12 will be
Informational).  Your points here are ideal for consideration when the
working group reaches that juncture.

Would the co-chairs object to beginning to track these items using the WG's
tracker?  If and when we do decide to crack open the base document for a
Proposed Standard revision, we'd already have an inventory of topics to
consider.  It would also help to keep the discussion on this list focused
on active topics now that the base draft is "done".

-MSK

---

From: Hector Santos <[email-masked]>
Date: Sat, 17 Jan 2015 12:00:20 -0500

I have two concerns.

It seems you "jumped the gun" to accept the RFC 4408 obsolete idea. Is 
7208 backward compatible or not?  Does DMARC require 7208 operations 
or 4408 operations?

And is this -12 publication "worthy" of even considering for 
implementation?  Or should we wait for the more "solid specification?"

-- 
HLS

On 1/16/2015 11:08 PM, Murray S. Kucherawy wrote:
> Hello Anne,
>
> On Fri, Jan 16, 2015 at 4:41 PM, Anne Bennett <[email-masked]
> <mailto:[email-masked]>> wrote:
>
> I think all of the points in your three messages are good input for a
> more solid specification, but the timing is unfortunate as we just got
> publication approval for -12 a week ago.  Making more changes
> post-approval would probably not be a good idea, and by my reading
> none of them rise to the level of being urgent to correct.
>
> The plan for the DMARC working group is to consider, among other
> things, whether it wants to produce a new version of the base document
> on the Standards Track (because of its path to publication, -12 will
> be Informational).  Your points here are ideal for consideration when
> the working group reaches that juncture.
>
> Would the co-chairs object to beginning to track these items using the
> WG's tracker?  If and when we do decide to crack open the base
> document for a Proposed Standard revision, we'd already have an
> inventory of topics to consider.  It would also help to keep the
> discussion on this list focused on active topics now that the base
> draft is "done".
>
> -MSK
>
>
> _______________________________________________
> dmarc mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/dmarc
>

---

From: Tim Draegen <[email-masked]>
Date: Mon, 19 Jan 2015 09:30:09 -0500

> On Jan 16, 2015, at 11:08 PM, Murray S. Kucherawy <[email-masked]> wrote:
> 
> Would the co-chairs object to beginning to track these items using the WG's tracker?  If and when we do decide to crack open the base document for a Proposed Standard revision, we'd already have an inventory of topics to consider.  It would also help to keep the discussion on this list focused on active topics now that the base draft is "done".

No objection — please do use the WG’s tracker for these items.  Anne’s thorough review will be picked up (and not rediscovered!) if we’ve got an obvious place to start from.

=- Tim

---

From: Tim Draegen <[email-masked]>
Date: Mon, 19 Jan 2015 09:43:40 -0500

> On Jan 17, 2015, at 12:00 PM, Hector Santos <[email-masked]> wrote:
> 
> I have two concerns.
> 
> It seems you "jumped the gun" to accept the RFC 4408 obsolete idea. Is 7208 backward compatible or not?  Does DMARC require 7208 operations or 4408 operations?
> 
> And is this -12 publication "worthy" of even considering for implementation?  Or should we wait for the more "solid specification?"

Hi Hector, if you review:

  https://www.rfc-editor.org/rfc/rfc7208.txt

..you'll see at the very top "Obsoletes: 4408".  If there are discrepancies between 4408 and 7208, it's best to file an errata (erratum?):   http://www.rfc-editor.org/how_to_report.html

DMARC implementations are already in the wild and deployed.  Input to the existing specification will be largely based on working implementations.  You might have your own reasons for waiting for this WG to review the DMARC base draft, but if you want to provide input based on operational & implementation experience, it's probably best to not wait.

=- Tim

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Mon, 19 Jan 2015 20:37:43 -0800

On Mon, Jan 19, 2015 at 6:43 AM, Tim Draegen <[email-masked]> wrote:

> DMARC implementations are already in the wild and deployed.  Input to the
> existing specification will be largely based on working implementations.
> You might have your own reasons for waiting for this WG to review the DMARC
> base draft, but if you want to provide input based on operational &
> implementation experience, it's probably best to not wait.

Indeed, -12 represents what's been deployed, and that was always the intent
of this ISE submission.  One could thus conclude that it is solid enough
for everyone that has already deployed it, and that's not exactly a small
or obscure list.

Still, as has been said before: If there's more work to be done on this
document, the working group is chartered to generate a Standards Track
version using this document as a starting point once its other deliverables
have been completed.  We can record issues we'd like to see addressed in
the tracker if there are some, and, if and when the WG gets there, we'll be
off to the races.

-MSK

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Mon, 19 Jan 2015 20:52:10 -0800

On Mon, Jan 19, 2015 at 6:30 AM, Tim Draegen <[email-masked]> wrote:

> No objection — please do use the WG’s tracker for these items.  Anne’s
> thorough review will be picked up (and not rediscovered!) if we’ve got an
> obvious place to start from.
>

Done for Anne's points, and I'll do so for Jim Fenton's remaining ones as
well.

-MSK

---

From: Anne Bennett <[email-masked]>
Date: Tue, 20 Jan 2015 16:44:16 -0500

Hi, Murray.

MK> I think all of the points in your three messages are good input for a more
MK> solid specification, but the timing is unfortunate as we just got
MK> publication approval for -12 a week ago.

I apologize for my inadvertently poor timing; I was catapulted
into all this last week when my parent domain (also my
Organizational Domain) published an SPF record and a DKIM
record, and we became concerned that they might implement DMARC,
which could have a very negative impact on our mail services
unless we take action quickly and become prepared to publish
our own DMARC record.  Thus, my sudden plunge into all these
RFCs and this draft.  :-/

MK> Making more changes post-approval
MK> would probably not be a good idea, and by my reading none of them rise to
MK> the level of being urgent to correct.

That's certainly not my call to make; I can only give you
a point of view from a fairly small site (about 10K users),
where we're not looking to implement any of these mechanisms
from scratch (we'll use software someone else wrote, mostly),
but we *do* need to understand the implications of our (or
our OD's) publishing DNS records for SPF, DKIM, and/or DMARC.

Since, as pointed out by Tim Draegen, "DMARC implementations are
already in the wild and deployed", one would expect to be able
to determine what those implementations do, based on this spec.
Sadly this hasn't been possible (so far) for me and my colleague
Michael Jack Assels, despite our being two fairly smart cookies,
with nearly a half-century of e-mail experience between us.

I want to emphasize that I think that the documents in question
(at least this draft and RFC7208 - I've barely skimmed RFC6376
on DKIM yet) individually are well written, but trying to
understand them in context together is proving to be quite
a challenge, and the lack of clarity on the HELO issue is
the biggest part of the problem.

We're now resorting to running tests to see how the biggest
gorilla in this jungle (Google) handles all this.  We've just
completed an initial set of tests with SPF records only (no
DMARC), which show that Google uses the MAIL FROM identity but
seems to ignore the HELO identity even in the absence of DMARC,
much to our surprise given RFC7208.

We haven't yet run our with-DMARC tests, though I suspect that
if Google doesn't look at HELO in a "pure SPF" environment, it
probably won't use it in the context of DMARC either.

If indeed it is the case that (at least in the context of
DMARC) the HELO identity is not normally used, I would hope
that introducing a small clarification to that effect could
be done without significantly impeding the progress of this
draft towards publication.  Mind you, I don't know that the
publication constraints are, so perhaps my hope is utterly
in vain!

But on the off-chance that it's not impossible to clarify
this now, and assuming that my growing suspicion that HELO is
ignored is correct, then I would propose:

In section "3.1. Identifier Alignment":

< For example, [DKIM] authenticates the domain that affixed a
< signature to the message, while [SPF] authenticates either
< the domain that appears in the RFC5321.MailFrom portion of
< [SMTP] or the RFC5321.EHLO/HELO domain if the RFC5321.MailFrom
< is null (in the case of Delivery Status Notifications).

> For example, [DKIM] authenticates the domain that affixed a
> signature to the message, while [SPF] can authenticate either
> the domain that appears in the RFC5321.MailFrom portion of
> [SMTP], or the RFC5321.EHLO/HELO domain, or both.

In section "3.1.2. SPF-authenticated Identifiers":

< In relaxed mode, the [SPF]-authenticated domain and
< RFC5322.From domain must have the same Organizational Domain.
< In strict mode, only an exact DNS domain match is considered
< to produce identifier alignment.

> In relaxed mode, the [SPF]-authenticated domain and
> RFC5322.From domain must have the same Organizational Domain.
> In strict mode, only an exact DNS domain match is considered
> to produce identifier alignment.
> 
> Note that the HELO identity is not used in the context of
> DMARC (except when required to "fake" an otherwise null
> reverse-path), even though a "pure SPF" implementation
> according to [SPF] would check the HELO domain.

In "4.1. Authentication Mechanisms":

< o  [SPF], which authenticates the domain found in an
<    [SMTP] MAIL command when it is the authorized domain.

> o  [SPF], which authenticates the domain found in an
>    [SMTP] MAIL command, or the domain found in the
>    [SMTP] HELO/EHLO command if the MAIL command has
>    a null path.  Note that in contradiction to [SPF],
>    in the context of DMARC, the HELO identity is not
>    checked (except in the case of a null path).

Even if it turns out to be impossible to make the above changes
before publication, I'd be grateful to the people on this list
for confirming (or disconfirming!) my understanding that HELO
is not used in the context of DMARC (modulo the null path case).

Anne.

---

From: Franck Martin <[email-masked]>
Date: Tue, 20 Jan 2015 16:14:32 -0600 (CST)

----- Original Message -----
> From: "Anne Bennett" <[email-masked]>
> To: "DMARC list" <[email-masked]>
> Sent: Tuesday, January 20, 2015 1:44:16 PM
> Subject: Re: [dmarc-ietf] ... and two more tiny nits, while I'm at it
> 
> 
> Hi, Murray.
> 

Your confusion on HELO is may be related to the fact that the HELO string is only used when the MAIL-FROM: is empty?

There is some text here:
http://trac.tools.ietf.org/html/rfc7208#section-10.1.3

The HELO string is not evaluated all the time, it is more like a fall back.

Also I have trouble to understand why you may be affected by your OD? What is the relation between your domain and your OD? I suspect this is concordia.ca?

If you look at the public suffix list, google has put a separation at blogspot.com level, so that each hosting domain, can be organizationally separated.

https://publicsuffix.org/list/effective_tld_names.dat

I would suggest you publish a DMARC record in monitoring mode (p=none) so that you get reports (and no impact) and better understand your email infrastructure and what needs to be done.

---

From: Scott Kitterman <[email-masked]>
Date: Tue, 20 Jan 2015 17:29:10 -0500

On Tuesday, January 20, 2015 16:14:32 Franck Martin wrote:
> > MK> would probably not be a good idea, and by my reading none of them rise
> > to MK> the level of being urgent to correct.
> > 

The confusion is that many people think that HELO is only used when Mail From 
is empty.  It's been recommended as a stand alone test in both RFC 4408 and 
that's not changed in RFC 7208.  It's just more obvious now.

You do use postmaster@$HELO when Mail From is empty, which is relevant to 
DMARC use of SPF results.  

You can also use HELO on it's own, which is not.  SPF pass on the HELO 
identity isn't useful as an accept criteria.  SPF fail on the HELO identity 
can, however, be useful as a reject criteria.

Scott K

---

From: Laura Atkins <[email-masked]>
Date: Tue, 20 Jan 2015 14:34:22 -0800

> The HELO string is not evaluated all the time, it is more like a fall back.

7208 actually recommends that the HELO string be evaluated every time. 
http://trac.tools.ietf.org/html/rfc7208#section-2.3

"2.3.  The "HELO" Identity

   It is RECOMMENDED that SPF verifiers not only check the "MAIL FROM"
   identity but also separately check the "HELO" identity by applying
   the check_host() function (Section 4) to the "HELO" identity as the
   <sender>.  Checking "HELO" promotes consistency of results and can
   reduce DNS resource usage."

laura 

-- 
Laura Atkins
Word to the Wise			"The Deliverability Experts!"
Direct: 650 678-3454		Fax: 650 249-1909
@wise_laura
Delivery blog: <http://blog.wordtothewise.com/>

---

From: Franck Martin <[email-masked]>
Date: Tue, 20 Jan 2015 16:38:43 -0600 (CST)

----- Original Message -----
> From: "Scott Kitterman" <[email-masked]>
> To: [email-masked]
> Sent: Tuesday, January 20, 2015 2:29:10 PM
> > Your confusion on HELO is may be related to the fact that the HELO string
> > is
> > only used when the MAIL-FROM: is empty?
> 
Yes, RFC7208 says evaluate both in parallel, but the result of an spf=pass/fail is highly constrained on the success or failure of the MAIL FROM spf test.

I mean, it seems quite rare to find an SPF record on the HELO string but not one the MAIL FROM string

---

From: Scott Kitterman <[email-masked]>
Date: Tue, 20 Jan 2015 17:46:39 -0500

On Tuesday, January 20, 2015 14:34:22 Laura Atkins wrote:
> > is only used when the MAIL-FROM: is empty?
> > 
> > There is some text here:

Approximately the same text existed in RFC 4408 2.1:

>    It is RECOMMENDED that SPF clients not only check the "MAIL FROM"
>    identity, but also separately check the "HELO" identity by applying
>    <sender>.

HELO results are unrelated to DMARC.

Scott K

---

From: Scott Kitterman <[email-masked]>
Date: Tue, 20 Jan 2015 17:49:01 -0500

On Tuesday, January 20, 2015 16:38:43 Franck Martin wrote:
> > > > MK> I think all of the points in your three messages are good input
> > > > for a
> > > > more
> > The confusion is that many people think that HELO is only used when Mail
> > From is empty.  It's been recommended as a stand alone test in both RFC
> > 4408 and that's not changed in RFC 7208.  It's just more obvious now.
> > 
> Yes, RFC7208 says evaluate both in parallel, but the result of an
> spf=pass/fail is highly constrained on the success or failure of the MAIL
> FROM spf test.
> 

Last time I had stats, it was about 10% as common as Mail From oriented 
records.  Much less common, but I wouldn't say rare.  When done this way, 
there isn't a singe "SPF" result, there are two: SPF/Mail From and SPF/HELO.  
Only SPF/Mail From is relevant to DMARC.  

In the case of the message having a null Mail From, you can still get two SPF 
results, they will just be the same.

Scott K

---

From: Anne Bennett <[email-masked]>
Date: Tue, 20 Jan 2015 17:54:22 -0500

Franck Martin <[email-masked]> writes:

> Yes, RFC7208 says evaluate both in parallel, but the result
> of an spf=pass/fail is highly constrained on the success or
> failure of the MAIL FROM spf test.

Actually, it recommends checking the HELO identity first,
because if you get a definite result from that, you may not
have to evaluate the MAIL FROM identity at all.

> I mean, it seems quite rare to find an SPF record on the
> HELO string but not one the MAIL FROM string

I have no stats on how rare it is, but I can show you an
example for one of my home domains:

  # host -t txt quill.porcupine.ca
  quill.porcupine.ca descriptive text "v=spf1 +a -all"

  # host -t txt porcupine.ca
  porcupine.ca descriptive text "v=spf1 +mx ?all"

... that is, if anyone tries to HELO as host "quill" and isn't
coming from my IP address, you can reject that mail out of
hand as a fake ("-all").

However, if the envelope sender is in my domain, then if it
came from my MX, it's almost certainly good (lower its spam
score), but if not, perhaps someone is forwarding their mail,
and I don't want their final receiving ISP to drop my message,
which is why "?all" in that case.

(And yes, forwarded bounce mail originally from my mailer will
fail the "MAIL FROM constructed from HELO" test, but I think I
mostly avoid backscatter, so that's not too serious.)

Anne.

---

From: Terry Zink <[email-masked]>
Date: Tue, 20 Jan 2015 22:55:58 +0000

> http://trac.tools.ietf.org/html/rfc7208#section-2.3

They do say to check it both times but I don't agree with the rationale provided. Expanding on the excerpt that Laura provided:

2.3.  The "HELO" Identity

   reduce DNS resource usage.  

[tzink] How does this reduce DNS resource usage? Aren't we now checking two domains instead of one?

   If a conclusive determination about the
   message can be made based on a check of "HELO", then the use of DNS
   resources to process the typically more complex "MAIL FROM" can be
   avoided.

[tzink] Disagree here, especially the case of shared hosting environment like Office 365. Customers relay spam through us and sometimes that spam will fail SPF checks. Yet, if you checked the SPF record on the HELO string (e.g., na01-bn1-obe.outbound.protection.outlook.com), that would pass an SPF check every time whereas the @paypal.com in the MAIL FROM would fail SPF. Seems like you can only be sure you can trust the HELO when you are sure the sender locks down the outbound mail infrastructure, and that is not the case everywhere.

  Additionally, since SPF records published for "HELO"
   identities refer to a single host, when available, they are a very
   reliable source of host authorization status.  Checking "HELO" before
   "MAIL FROM" is the RECOMMENDED sequence if both are checked.

[tzink] They are for a single host but not necessarily for a single organization or even a series of organizations with the same set of policies. It seems like this RFC does not have a mult-tenant hosted environment in mind, and that is becoming more common.

-- Terry

---

From: "John Levine" <[email-masked]>
Date: 20 Jan 2015 22:57:34 -0000

>HELO results are unrelated to DMARC.

Is that still true when the bounce address is empty?  It's fairly common
to have an NDR with an empty bounce address and 

 From: [email-masked]

Assuming it's not DKIM signed (most NDRs aren't) what's a DMARC user
supposed to do?

R's,
John

---

From: Franck Martin <[email-masked]>
Date: Tue, 20 Jan 2015 17:40:39 -0600 (CST)

----- Original Message -----
> Sent: Tuesday, January 20, 2015 2:49:01 PM

> Only SPF/Mail From is relevant to DMARC.

Why do you say that?

DMARC takes the result from SPF (pass/fail/...) and the string that SPF used for this result be it mail from or helo, to check for alignment.

Did I miss something?

---

From: Franck Martin <[email-masked]>
Date: Tue, 20 Jan 2015 17:43:44 -0600 (CST)

----- Original Message -----
> From: "John Levine" <[email-masked]>
> To: [email-masked]
> Cc: [email-masked]
> Sent: Tuesday, January 20, 2015 2:57:34 PM

The same as a non DMARC user should do...

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Tue, 20 Jan 2015 17:22:27 -0800

> our own DMARC record.  Thus, my sudden plunge into all these

Well, shoot.  Timing notwithstanding, I also apologize if I came across as
dismissing your use case as unimportant.  I thought you were merely
providing reviews as an interested party, not actually addressing a
production problem.

> with nearly a half-century of e-mail experience between us.

I can't speak for most of the operators you're probably dealing with, many
of whom have their own implementations.

I can say that OpenDMARC consumes the Authentication-Results field, or the
Received-SPF field if the former isn't there, but it prefers a result based
on MAIL FROM over one based on HELO if both are present.  But it will use
both.

I'm pretty sure Gmail people are on, or at least following, this list;
hopefully someone there will comment.

> the biggest part of the problem.

This is certainly useful feedback to the WG.  In addition to considering it
as a topic for a Proposed Standard version of the DMARC specification,
there might be a need to explore some kind of interim statement about
what's supposed to happen here (if necessary).

> ignored is correct, then I would propose:

Do people concur with this change, or something close to it?

-MSK

---

From: "John Levine" <[email-masked]>
Date: 21 Jan 2015 02:14:57 -0000

>Do people concur with this change, or something close to it?

I'm OK with it, but to the meta-question, I realize the practical
issues involved with yanking something out of the production queue,
but in this case I wonder if that's not the right thing to do.

There's no great hurry in getting the DMARC document published, since
nothing currently depends on it, and if reasonable people are finding
holes in it that make it hard to write interoperable code, I'd rather
fix the holes than add lengthy errata or recycle later.

R's,
John

---

From: "Stephen J. Turnbull" <[email-masked]>
Date: Wed, 21 Jan 2015 12:56:58 +0900

> fix the holes than add lengthy errata or recycle later.

*If* the existing code bases are interoperable, then it makes sense to
postpone publication to fix the spec to conform to actual practice
(which is its purpose, after all).  Is it worth the effort/editor time
to compose and discuss better language and the calendar time to wait
for confirmation from implementors?

---

From: Scott Kitterman <[email-masked]>
Date: Tue, 20 Jan 2015 23:59:02 -0500

On Tuesday, January 20, 2015 22:55:58 Terry Zink wrote:
> They do say to check it both times but I don't agree with the rationale
> provided. Expanding on the excerpt that Laura provided:
> 
> 2.3.  The "HELO" Identity
> 

Note: This point was actively discussed during the SPFbis working group.

HELO records are virtually always very simple: zero or one additional DNS 
lookup.  If you get a fail result and choose to reject as a result, you never 
have to even retrieve the Mail From record, let alone do all the lookups 
triggered by the terms in the record.

This is for SPF on it's own and not particularly relevant if you want to feed 
SPF results into DMARC.

> [tzink] Disagree here, especially the case of shared hosting environment
> like Office 365. Customers relay spam through us and sometimes that spam
> will fail SPF checks. Yet, if you checked the SPF record on the HELO string
> (e.g., na01-bn1-obe.outbound.protection.outlook.com), that would pass an
> SPF check every time whereas the @paypal.com in the MAIL FROM would fail
> SPF. Seems like you can only be sure you can trust the HELO when you are
> sure the sender locks down the outbound mail infrastructure, and that is
> not the case everywhere.

Agree.  The only definitive result you can get from SPF HELO checks is reject.  
It'll never tell you something is good.

> [tzink] They are for a single host but not necessarily for a single
> organization or even a series of organizations with the same set of
> policies. It seems like this RFC does not have a mult-tenant hosted
> environment in mind, and that is becoming more common.

It doesn't matter.  The HELO name is (should be) particular to that host so it 
works regardless.  Recall that for SPF there's no notion of alignment so 
there's no requirement that the HELO name have anything to do with any other 
identity in the message.

To read RFC 7208 as intended, you have to forget DMARC exists.  It was written 
to stand alone.  It's up to DMARC to explain how it wants to be built on top 
of SPF.

Scott K

---

From: Scott Kitterman <[email-masked]>
Date: Tue, 20 Jan 2015 23:59:58 -0500

> Assuming it's not DKIM signed (most NDRs aren't) what's a DMARC user

Subtle point, but that's not a HELO result, that's a Mail From result 
evaluated using a synthetic Mail From based on HELO.

Scott K

---

From: Scott Kitterman <[email-masked]>
Date: Wed, 21 Jan 2015 00:02:26 -0500

On Tuesday, January 20, 2015 17:40:39 Franck Martin wrote:
> > there isn't a singe "SPF" result, there are two: SPF/Mail From and
> > SPF/HELO. Only SPF/Mail From is relevant to DMARC.
> 
> Why do you say that?
> 
> DMARC takes the result from SPF (pass/fail/...) and the string that SPF used
> for this result be it mail from or helo, to check for alignment.
> 
> Did I miss something?

DMARC takes the SPF result and the Mail From as an input (which in the case of 
a null Mail From is a synthetic Mail From built using HELO, but that's just a 
coincidence).  SPF isn't just a result (pass, fail, etc), it also has a domain 
and a related identity.

Scott K

---

From: Franck Martin <[email-masked]>
Date: Tue, 20 Jan 2015 23:31:45 -0600 (CST)

----- Original Message -----
> Sent: Tuesday, January 20, 2015 9:02:26 PM

If I recall the SPF spec, it specifies a MAIL FROM which is not the RFC5321.mailfrom, but a mix of RFC5321.mailfrom and RFC5321.helo based on which one was used to get the SPF result.

The original SPF spec is quite confusing on that, at least for me. On senderid, I think they call this field mfrom to avoid confusion.

It seems weird that the SPF results would depend more on the HELO than the RFC5321.mailfrom because if spammer.com put a SPF then

mailfrom: [email-masked]
helo spammer.com

would seem quite problematic....

---

From: Scott Kitterman <[email-masked]>
Date: Wed, 21 Jan 2015 00:36:57 -0500

On January 21, 2015 12:31:45 AM EST, Franck Martin <[email-masked]> wrote:
>> > DMARC takes the result from SPF (pass/fail/...) and the string that
>SPF
>> > used
>> DMARC takes the SPF result and the Mail From as an input (which in
>the case
>> of
>> coincidence).  SPF isn't just a result (pass, fail, etc), it also has
>a
>> domain
>> and a related identity.
>> 
>
>If I recall the SPF spec, it specifies a MAIL FROM which is not the
>RFC5321.mailfrom, but a mix of RFC5321.mailfrom and RFC5321.helo based
>on which one was used to get the SPF result.
>
>The original SPF spec is quite confusing on that, at least for me. On
>senderid, I think they call this field mfrom to avoid confusion.
>
>It seems weird that the SPF results would depend more on the HELO than
>the RFC5321.mailfrom because if spammer.com put a SPF then
>

Which is precisely why I said HELO pass doesn't mean anything. HELO is really for rejection on fail. 

Scott K

---

From: Michael Jack Assels <[email-masked]>
Date: Wed, 21 Jan 2015 10:50:08 -0500

On Tue, 20 Jan 2015 16:14:32 CST, 
Franck Martin <[email-masked]> wrote:

> [...]
> Your confusion on HELO is may be related to the fact that the
> HELO string is only used when the MAIL-FROM: is empty?
>
> There is some text here:

It's already been pointed out that
   http://trac.tools.ietf.org/html/rfc7208#section-2.3
RECOMMENDs checking the HELO string first; hence our confusion.
It'a not difficult to imagine a HELO string that passes SPF but
doesn't align with the RFC5322.From with a MAIL FROM domain that
passes SPF *and* aligns, while the DKIM-Signature validates but
doesn't align.  An implementation of DMARC that follows the SPF
recommendation will, as I understand it, produce [SPF:fail,
DKIM:fail], despite the existence of an SPF-valid MAIL FROM
domain that aligns with the RFC5322.From domain.  It seems
to be the consensus that this would be wrong, and that existing
DMARC implementations don't behave that way.  That's fine by me,
but it needs to be said out loud that DMARC implementations
MUST NOT (SHOULD NOT?) follow rfc7208#section-2.3's
recommendation.

> Also I have trouble to understand why you may be affected by
> your OD? What is the relation between your domain and your OD?
> I suspect this is concordia.ca?

Yes, it's Concordia.ca.  Some details:  Concordia.ca is Concordia
University, while encs.concordia.ca is just the Faculty of
Engineering and Computer Science at Concordia, so it's
appropriate that concordia.ca is the OD.  They've delegated 
encs.concordia.ca to us, and we (ENCS staff) manage it ourselves,
with minimal interaction.

> If you look at the public suffix list, google has put a
> separation at blogspot.com level, so that each hosting domain,
> can be organizationally separated.
> 

Are you saying that any owner of a registered domain can
arrange to the suffix list on request, thereby allowing
next-level subdomains to be elevated to ODs?  If that's
right, I'd certainly be keen on suggesting to our
concordia.ca colleagues that they do us that favour.

> I would suggest you publish a DMARC record in monitoring mode
> (p=none) so that you get reports (and no impact) and better
> understand your email infrastructure and what needs to be done.

We'll certainly publish a DMARC record with p=none.

Thanks,

MJA

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 21 Jan 2015 11:12:51 -0800

> fix the holes than add lengthy errata or recycle later.

I am asking the IESG and the ISE what the process is for making such
adjustments now.

Mainly my resistance to further change comes from the fact that we've done
last calls of varying kinds on this document more times than I can count.
It really is time to put the non-IETF version to bed and hand it off, even
with its weaknesses, and let the standards process take it from there.
There's a working group already chartered to do exactly that; in fact, that
was one of the premises of creating that working group.

-MSK

---

From: "John R Levine" <[email-masked]>
Date: 21 Jan 2015 14:18:23 -0500

> was one of the premises of creating that working group.

I was under the impression that the reason this version's going through 
the ISE is that the DMARC group isn't willing to hand change control to 
the IETF.  If they are willing, it makes no sense to do it outside of a 
WG.

I sympathize with your concern about the length of the process, and the 
lack of timely review, but DMARC is unusually complex for a mail thing, 
the implementations have all been from different drafts (did anyone ever 
do reporting by http?) and it's not surprising that there are still places 
where the code doesn't match the text, or two versions of code do 
different things.

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
Please consider the environment before reading this e-mail.

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 21 Jan 2015 11:33:40 -0800

On Wed, Jan 21, 2015 at 11:18 AM, John R Levine <[email-masked]> wrote:

> the ISE is that the DMARC group isn't willing to hand change control to the
> IETF.  If they are willing, it makes no sense to do it outside of a WG.
>

It's a little late to re-hash the path by which we got here now, I think,
and it was a rancorous debate.  The agreement we have with the IESG is to
do it via the path we're now on.

Likewise, I was under the impression that publishing something through the
ISE is deliberately restricted to Informational status specifically because
what the document specifies might have flaws as it hasn't been subjected to
any kind of IETF Review.  I would agree with you if it were on the
Standards Track, but it isn't.

The idea of a process for ensuring that all implementations are based on
-12 (or -13) and thus any two versions of the code do the same thing sounds
dreadfully open-ended.  I'd really like to have the goal posts sit still
for a while.

-MSK

---

From: Hector Santos <[email-masked]>
Date: Wed, 21 Jan 2015 15:43:09 -0500

> RECOMMENDs checking the HELO string first; hence our confusion.

Implementators should know this would be a high DNS waste and overhead 
suggestion.

The optimized approach is to delay any DNS query applications and 
first capture the SMTP process field parameters to determine what 
augmented DNS applications apply, what needs be queried and 
calculated.  For example, SMTP has offered guidance in this area for 
delaying the reverse-path (MAIL FROM) processing until the 
forward-path (RCPT TO)  is known:

    SMTP Section 3.3 Mail Transactions

    .....

    Despite the apparent scope of this requirement, there are
    circumstances in which the acceptability of the reverse-path may
    not be determined until one or more forward-paths (in RCPT
    commands) can be examined.  In those cases, the server MAY
    reasonably accept the reverse-path (with a 250 reply) and then
    report problems after the forward-paths are received and
    examined.  Normally, failures produce 550 or 553 replies.

HELO/EHLO has historically been known to be incorrect for many 
reasons.  At best, the receiver can perform a field syntax check 
including a required bracketed IP Domain literal check to make sure it 
matches the connection IP address before doing anything else and 
allowing the next SMTP state to continue.

The reality is that SMTP receivers are not going to know the DMARC 
status of an incoming anonymous client until the DATA payload is 
received.  It doesn't need DMARC information for SPF -ALL rejection 
policies.   SPF -ALL policy SHOULD be processed immediately before 
DATA.   But I think DMARC wants you further delay SPF processing until 
DATA payload is received (for reporting reasons I suppose).

Since it can be potentially high overhead of getting a payload with no 
DMARC information, SPF receivers may not wait until DATA is received 
to process SPF.

Incidentally,  SENDER-ID [1] resolve this DATA overhead potential 
issue with the SUBMITTER optimization [2] where the PRA (Purported 
Responsible Address) [3] was passed to the MAIL FROM state:

   MAIL FROM:<return-path> SUBMITTER=pra

This tells the SPF processor that the pra DOMAIN is the lookup domain.

DMARC can borrow from such ideas to pass the 5322 Author Domain (DMARC 
domain), and better yet even pass the policy as well to help optimized 
the receiver:

   MAIL FROM:<return-path>[ DMARC=author-domain[ DMARC.P=reject]]

This DMARC ESMTP idea would probably only work if there is a TRUST 
that the that payload is DKIM signed correctly using the problem 
5322.From address.

-- 
HLS

[1] https://tools.ietf.org/html/rfc4406
[2] https://tools.ietf.org/html/rfc4405
[3] https://tools.ietf.org/html/rfc4407

---

From: Hector Santos <[email-masked]>
Date: Wed, 21 Jan 2015 16:18:03 -0500

> have to evaluate the MAIL FROM identity at all.

However, in practice, optimized receivers will wait for a valid local 
RCPT TO (Forward-path) address before exerting any potentially, high 
overhead, wasteful DNS application queries.

For remote RCPT TO address, this generally requires 
authentication/authorization to relay mail. Under the SUBMIT protocol, 
HELO/EHLO can be enforced.  Our package does not enforce it (it was 
relaxed) for PORT 587 operations because AUTH is required anyway and 
in the SOHO market, with NATs around, it was not always possible to 
put a proper machine client host name for the EHLO/HELO field or the 
Domain IP Literal was incorrect (did not match the connection ip).

But for public port 25 operations, there is a high payoff to wait.  We 
realized a 60% savings in DNS lookups because 60% of the RCPT TO were 
invalid!  SMTP RFC5321 even provides insight into using delay strategies:

    .....

Overall, all specs are just guidelines. You don't have to follow them 
exactly where it doesn't make sense.  A RECOMMEND is functionally 
equivalent to a SHOULD, it is not a MUST.

As long as the end result is the same, we should be generally ok so we 
are basically talking about optimization concepts.

-- 
HLS

---

From: Franck Martin <[email-masked]>
Date: Wed, 21 Jan 2015 16:01:53 -0600 (CST)

----- Original Message -----
> Sent: Tuesday, January 20, 2015 9:36:57 PM
>From http://www.openspf.org/Implementations Mail::SPF has passed the test suites

http://search.cpan.org/dist/Mail-SPF/lib/Mail/SPF/Request.pm
"Note: In the case of an empty MAIL FROM SMTP transaction parameter (MAIL FROM:<>), you should perform a check with the helo scope instead."

I think we are going in circles here.

The SPF spec is not clear or there is a disconnect with what's out there, or both...

---

From: Anne Bennett <[email-masked]>
Date: Wed, 21 Jan 2015 17:20:42 -0500

Scott Kitterman <[email-masked]> writes:

> DMARC takes the SPF result and the Mail From as an input
> (which in the case of a null Mail From is a synthetic Mail From
> built using HELO, but that's just a coincidence).  SPF isn't
> just a result (pass, fail, etc), it also has a domain and a
> related identity.

... in other words, aside from its presence in the null return
path case, the HELO identity is *not* used by DMARC.

> on MAIL FROM over one based on HELO if both are present.  But it will use

... in other words, a HELO identity *can* be used by DMARC.

Either I'm misunderstanding both of you, or the implementations differ
on this point.  :-(

Anne.

---

From: Scott Kitterman <[email-masked]>
Date: Wed, 21 Jan 2015 19:21:06 -0500

>Either I'm misunderstanding both of you, or the implementations differ

They differ, but I think it's mostly a theoretical difference. 

Scott K

---

From: Scott Kitterman <[email-masked]>
Date: Wed, 21 Jan 2015 19:19:44 -0500

On January 21, 2015 5:01:53 PM EST, Franck Martin <[email-masked]> wrote:
>> On January 21, 2015 12:31:45 AM EST, Franck Martin
><[email-masked]>
>> wrote:
>> >
>> >> > > Subject: Re: [dmarc-ietf] ... and two more tiny nits, while
>I'm
>> >at it
>> >> > > 
>> >> > > 
>> >> > > 
>> >> > > 
>> >> > > records.  Much less common, but I wouldn't say rare.  When
>done
>> >this way,
>> Which is precisely why I said HELO pass doesn't mean anything. HELO
>is really
>> for rejection on fail.
>> 
>"Note: In the case of an empty MAIL FROM SMTP transaction parameter
>(MAIL FROM:<>), you should perform a check with the helo scope
>instead."
>

By design, SPF doesn't go very far into what receivers should do.  Unlike DMARC, it actively avoids specifying receiver policy. It's not clear what to do because what to do is outside the scope of the specification.

Scott K

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Thu, 22 Jan 2015 10:27:40 -0800

> was one of the premises of creating that working group.

I've consulted with the Area Director sponsoring the document's conflict
review, and the ISE.  Both of them agree that we will only make changes
approved by the ISE and only during AUTH48 at this point, and those will be
limited to correcting serious problems that would prevent current DMARC
implementations from interacting properly.  Anything else can be left to
the DMARC working group on its standards track deliverable.

An argument can be made that this proposed change qualifies under that
definition, so please review it and comment as to whether it satisfies the
defect identified, or whether the change is necessary at all.  I will
assume "yes" unless I hear otherwise.  Again, the diff is here:

http://www.blackops.org/~msk/dmarc/diff.html

-MSK

---

From: Franck Martin <[email-masked]>
Date: Thu, 22 Jan 2015 12:48:03 -0600 (CST)

----- Original Message -----

> From: "Murray S. Kucherawy" <[email-masked]>
> To: [email-masked]
> Sent: Thursday, January 22, 2015 10:27:40 AM
> Subject: Re: [dmarc-ietf] questions on the spec, was ... and two more tiny
> nits, while I'm at it

> On Wed, Jan 21, 2015 at 11:12 AM, Murray S. Kucherawy < [email-masked] >
> wrote:

> > It
> > really is time to put the non-IETF version to bed and hand it off, even
> > with
> > its weaknesses, and let the standards process take it from there. There's a
> > working group already chartered to do exactly that; in fact, that was one
> > of
> > the premises of creating that working group.
> 

> implementations from interacting properly. Anything else can be left to the
> DMARC working group on its standards track deliverable.

> defect identified, or whether the change is necessary at all. I will assume
> "yes" unless I hear otherwise. Again, the diff is here:

> http://www.blackops.org/~msk/dmarc/diff.html

Hold on... 

What is the decision matrix of SPF? 

SPF uses two strings, the RFC5321.mailfrom and the RFC5321.helo. Each string may or may not have an SPF record. What gives the spf status?

---

From: Michael Jack Assels <[email-masked]>
Date: Thu, 22 Jan 2015 15:00:58 -0500

On Thu, 22 Jan 2015 12:48:03 CST, 
Franck Martin <[email-masked]> wrote:

> > Subject: Re: [dmarc-ietf] questions on the spec, was ... and two more tiny nits, while I'm at it
> 
> > > I am asking the IESG and the ISE what the process is for
> > > making such adjustments now.
> > 
>
> > > Mainly my resistance to further change comes from the fact
> > > that we've done last calls of varying kinds on this
> > > document more times than I can count.  It really is time to
> > > put the non-IETF version to bed and hand it off, even with
> > > its weaknesses, and let the standards process take it from
> > > there. There's a working group already chartered to do
> > > exactly that; in fact, that was one of the premises of
> > > creating that working group.
> > 

> > I've consulted with the Area Director sponsoring the
> > document's conflict review, and the ISE. Both of them agree
> > that we will only make changes approved by the ISE and only
> > during AUTH48 at this point, and those will be limited to
> > correcting serious problems that would prevent current DMARC
> > implementations from interacting properly. Anything else can
> > be left to the DMARC working group on its standards track
> > deliverable.
> 
> > An argument can be made that this proposed change qualifies
> > under that definition, so please review it and comment as to
> > whether it satisfies the defect identified, or whether the
> > change is necessary at all. I will assume "yes" unless I hear
> > otherwise. Again, the diff is here:
> 
> SPF uses two strings, the RFC5321.mailfrom and the
> RFC5321.helo. Each string may or may not have an SPF record.
> What gives the spf status? 

As I read RFC7208, it doesn't explicitly provide a decision
matrix, but it does clearly recommend in section 2.3, that
[i]f a conclusive determination about the message can be made
based on a check of "HELO", then the use of DNS resources to
process the typically more complex "MAIL FROM" can be avoided."

Section 2.4 provides that "SPF verifiers MUST check the
[RFC5321.mailfrom] identity if a [RFC5321.helo] check either
has not been performed or has not reached a definitive policy"

I can't think of a way to read that that doesn't imply that
a "pass" or a "fail" on the basis of an SPF record for the
RFC5321.helo indentity (if such a record exists) is the spf
result; otherwise, the result is based on the RFC5321.mailfrom
identity.

I believe that this is not what DMARC implementations actually
do, and that the proposed change to the draft correctly points
out the difference and makes it clear what DMARC does, so if
I had a vote, I'd vote "yes" to the change.

MJA

---

From: "MH Michael Hammer (5304)" <[email-masked]>
Date: Thu, 22 Jan 2015 20:26:31 +0000

I’ve reviewed the diff between -12 and -13 and I’m comfortable with the changes.

Mike

From: dmarc [mailto:[email-masked]] On Behalf Of Murray S. Kucherawy
Sent: Thursday, January 22, 2015 1:28 PM
To: [email-masked]
Subject: Re: [dmarc-ietf] questions on the spec, was ... and two more tiny nits, while I'm at it

On Wed, Jan 21, 2015 at 11:12 AM, Murray S. Kucherawy <[email-masked]<mailto:[email-masked]>> wrote:
I am asking the IESG and the ISE what the process is for making such adjustments now.

Mainly my resistance to further change comes from the fact that we've done last calls of varying kinds on this document more times than I can count.  It really is time to put the non-IETF version to bed and hand it off, even with its weaknesses, and let the standards process take it from there.  There's a working group already chartered to do exactly that; in fact, that was one of the premises of creating that working group.

I've consulted with the Area Director sponsoring the document's conflict review, and the ISE.  Both of them agree that we will only make changes approved by the ISE and only during AUTH48 at this point, and those will be limited to correcting serious problems that would prevent current DMARC implementations from interacting properly.  Anything else can be left to the DMARC working group on its standards track deliverable.
An argument can be made that this proposed change qualifies under that definition, so please review it and comment as to whether it satisfies the defect identified, or whether the change is necessary at all.  I will assume "yes" unless I hear otherwise.  Again, the diff is here:

-MSK

---

From: Franck Martin <[email-masked]>
Date: Thu, 22 Jan 2015 14:46:59 -0600 (CST)

----- Original Message -----
> From: "Michael Jack Assels" <[email-masked]>
> To: [email-masked]
> Sent: Thursday, January 22, 2015 12:00:58 PM

an RFC to reach standard status needs to represent what is out there, I'd like to see more code before I form an opinion.

---

From: Michael Jack Assels <[email-masked]>
Date: Thu, 22 Jan 2015 16:20:35 -0500

On Thu, 22 Jan 2015 14:46:59 CST, 
Franck Martin <[email-masked]> wrote:

> > > [....]
> Ok, but a specific well known implementation does not seem to
> do that: From http://www.openspf.org/Implementations Mail::SPF
> has passed the test suites
> 
> "Note: In the case of an empty MAIL FROM SMTP transaction
> parameter (MAIL FROM:<>), you should perform a check with the
> helo scope instead."

This is what the proposed change says, isn't it?

> an RFC to reach standard status needs to represent what is out
> there, I'd like to see more code before I form an opinion.

I think we've crossed wires here.  I unreservedly agree that
RFC7208 does NOT represent what all DMARC implementations do,
and it may not even represent what all SPF implementations do.
Perhaps RFC7208 needs correction, but given what it says now,
and given that DMARC has an obvious dependency on SPF, it's
important that DMARC's standard should say "contrary to what
RFC7208 recommends, DMARC normally SPF-checks HELO only when
MAIL FROM is <>".

I don't think we're disagreeing about what DMARC does, or even
about what SPF usually does, despite what RFC7208 says.

MJA

---

From: Scott Kitterman <[email-masked]>
Date: Thu, 22 Jan 2015 17:18:56 -0500

On January 22, 2015 3:46:59 PM EST, Franck Martin <[email-masked]> wrote:
>> > > Subject: Re: [dmarc-ietf] questions on the spec, was ... and two
>more
>> > > tiny nits, while I'm at it
>> > 
>an RFC to reach standard status needs to represent what is out there,
>I'd like to see more code before I form an opinion.

Mail::SPF is a library that can be called by various applications. It's not an application which is where such logic would be implemented. You're quoting one library author's recommendations to application developers, not an actual implementation detail. Note:also not yet updated for RFC 7208.

Also, DMARC isn't standards track.

DMARC leverages the Mail From identity, so I don't see how independent HELO checks can be relevant. 

Scott K

---

From: Franck Martin <[email-masked]>
Date: Thu, 22 Jan 2015 16:47:42 -0600 (CST)

----- Original Message -----
> Sent: Thursday, January 22, 2015 1:20:35 PM

Yes, this is my point, seems to me RFC7208 needs an errata... than DMARC... but I'm ok with the proposed language, I just think DMARC should avoid to fix problems with lower protocols...

If you read the previous RFC:
http://trac.tools.ietf.org/html/rfc4408 section 2.2

   [RFC2821] allows the reverse-path to be null (see Section 4.5.5 in
   RFC 2821).  In this case, there is no explicit sender mailbox, and
   such a message can be assumed to be a notification message from the
   mail system itself.  When the reverse-path is null, this document
   defines the "MAIL FROM" identity to be the mailbox composed of the
   localpart "postmaster" and the "HELO" identity (which may or may not
   have been checked separately before).

   SPF clients MUST check the "MAIL FROM" identity.  SPF clients check
   the "MAIL FROM" identity by applying the check_host() function to the
   "MAIL FROM" identity as the <sender>.

so the MAIL FROM entity contains postmaster@helo if the RFC5321.mailfrom is empty...

Now in this document, it says to do the check_host on mail from and helo separately, it says the check_host function will return a result, and that result is deterministic, I see the code. However I reread quickly the spec, it does not say how you would merge both results to a single one: Did SPF passes or not? For instance, RFC7001 expects one result. http://tools.ietf.org/html/rfc7001#section-2.6.2

RFC7208 was written to make the text of the document more suitable for a "Standard" track. The charter was to not modify the protocol. So RFC7208 is not addressing this issue either.

---

From: "Rolf E. Sonneveld" <[email-masked]>
Date: Fri, 23 Jan 2015 00:08:51 +0100

> an RFC to reach standard status needs to represent what is out there,

the current draft is heading for Informational status, not standard 

> I'd like to see more code before I form an opinion.

as from an interoperability point of view it is important to know what 
the various implementations of the current implementors do exactly with 
SPF re. the point raised by Anne (SPF use of RFC5321.MailFrom vs. 
RFC5321.EHLO/HELO).

As for the 'SPF part of DMARC only an 'SPF pass' counts it must be 
crystal clear how an 'SPF pass' is reached. Therefore, if (repeat: if) 
all current implementations of DMARC only use the RFC5321.MailFrom, I'd 
suggest to use RFC2119 terminology here, like:

-13 text:

Note that the RFC5321.HELO identity is not typically used in the
             context of DMARC (except when required to "fake" an 
otherwise null
             reverse-path), even though a "pure SPF" implementation 
according to
             [SPF] would check that identifier.

Suggested text:

If RFC5321.MailFrom identity is present and not null, the RFC5321.HELO 
identity MUST NOT be used in the
             context of DMARC, even though a "pure SPF" implementation 
according to
             [SPF] would check that identifier.

/rolf

---

From: "John Levine" <[email-masked]>
Date: 22 Jan 2015 23:17:28 -0000

>DMARC leverages the Mail From identity, so I don't see how independent HELO checks can be relevant. 

If you look at sections 2.3 and 2.4 of RFC 7208, a reasonable
interpretation is that you check the HELO identity, and if you get a
"definitive policy" result, you're done and return that to the caller.

So a message comes from host mail.provider.com with From:
[email-masked].  The recipient host does an SPF check on
mail.provider.com, it passes, so SPF is done.  DMARC sees that the SPF
domain isn't aligned so it ignores it, and DMARC says it's unaligned,
even though an SPF check of customer.com might have passed.

I can't say whether this is a bug in 7208 or a fundamental flaw in
DMARC, but something is clearly wrong and this does not match what
running code does.  As things are written now, I don't see any way to
demand that SPF look at the MAIL FROM if it likes the HELO.

Fix 1: file an erratum on 7208 to say to switch the order, do the MAIL
FROM check first and only do the HELO check otherwise.  This may match
some running code, I haven't looked.

Fix 2: change 7208 to say that SPF can return multiple results.  Ugh.

R's,
John

---

From: Scott Kitterman <[email-masked]>
Date: Thu, 22 Jan 2015 18:24:21 -0500

On January 22, 2015 5:47:42 PM EST, Franck Martin <[email-masked]> wrote:
>Yes, this is my point, seems to me RFC7208 needs an errata... than
>DMARC... but I'm ok with the proposed language, I just think DMARC
>should avoid to fix problems with lower protocols...
>
>
>Now in this document, it says to do the check_host on mail from and
>helo separately, it says the check_host function will return a result,
>and that result is deterministic, I see the code. However I reread
>quickly the spec, it does not say how you would merge both results to a
>single one: Did SPF passes or not? For instance, RFC7001 expects one
>result. http://tools.ietf.org/html/rfc7001#section-2.6.2
>
>RFC7208 was written to make the text of the document more suitable for
>a "Standard" track. The charter was to not modify the protocol. So
>RFC7208 is not addressing this issue either.

Sigh.

RFC 7208 doesn't say the HELO result determines anything. It says IF (I say again IF) a decision has been reached about message disposition based on the HELO result, there is no requirement to go ahead and do a pointless Mail From check. 

Avoiding a check that has been determined to be pointless is the only change in this area in RFC 7208.

This is much ado about nothing and really, really irrelevant to DMARC. 

Scott K

P.S. No erratum needed

---

From: Scott Kitterman <[email-masked]>
Date: Thu, 22 Jan 2015 18:30:59 -0500

>Fix 2: change 7208 to say that SPF can return multiple results.  Ugh.

4408 and 7208 both suggest multiple calls to check_host() each with a single result. 

If I were configuring and SPF verifier to provide an input to DMARC processing, then I would probably configure it not to reject based on SPF fail.  Then the problem doesn't arise. 

This really is a non-issue. 

Scott K

---

From: Kurt Andersen <[email-masked]>
Date: Thu, 22 Jan 2015 15:35:59 -0800

On Thu, Jan 22, 2015 at 3:30 PM, Scott Kitterman <[email-masked]>
wrote:

> If I were configuring and SPF verifier to provide an input to DMARC
> processing, then I would probably configure it not to reject based on SPF
> fail.  Then the problem doesn't arise.
>

Are you suggesting that the DMARC spec should say that people SHOULD
configure (some would say usurp) SPF in such a way? I seem to recall some
contentious discussions about such usurpation during SPFbis (though I could
be conflating arguments from another context).

--Kurt

---

From: Franck Martin <[email-masked]>
Date: Thu, 22 Jan 2015 17:57:32 -0600 (CST)

----- Original Message -----
> From: "Rolf E. Sonneveld" <[email-masked]>
> To: "Franck Martin" <[email-masked]>, "Michael Jack Assels" <[email-masked]>
> Cc: [email-masked]
> Sent: Thursday, January 22, 2015 3:08:51 PM
> > "Note: In the case of an empty MAIL FROM SMTP transaction parameter (MAIL
> > FROM:<>), you should perform a check with the helo scope instead."
> >

No, no... :P

if you send a bounce, and usually some MTAs have trouble to sign the bounce they generate (not anymore true with postfix but a bit tricky to setup), then the only way to recover the message is to ensure there is an SPF aligned on the string presented by HELO.

So basically, I would say DMARC takes only the result of the check_host for the MAIL FROM entity which contains the postmaster@helo if the RFC5321.mailfrom is empty.

Murray, I think the elegant way in DMARC to refer to RFC7208 is this.

"DMARC uses only the result of the check_host() applied on the MAIL FROM entity as defined by RFC7208. The MAIL FROM entity is the one used for alignment checking.".

If I recall the operational test created by Tim, were checking these cases: http://dmarc-qa.com/ (2.1)

We all ran these tests during inter-op for conformance.

---

From: Terry Zink <[email-masked]>
Date: Fri, 23 Jan 2015 00:13:46 +0000

The way it works in Office 365 is this:

1. When checking SPF, use the domain in the 5321.MailFrom. If it is empty, use the domain in the HELO/EHLO.
2. Use the domain extracted from (1) when doing the DMARC alignment check for SPF.

We don't check both 5321.MailFrom AND HELO/EHLO for SPF. I've never found this to be a problem.

I don't know how it works in Hotmail (although I should) but it is moving over to the Office 365 infrastructure over the next several months anyhow so it will be the same.

-- Terry

---

From: Scott Kitterman <[email-masked]>
Date: Thu, 22 Jan 2015 19:45:38 -0500

On January 22, 2015 7:13:46 PM EST, Terry Zink <[email-masked]> wrote:
>I don't know how it works in Hotmail (although I should) but it is
>moving over to the Office 365 infrastructure over the next several
>months anyhow so it will be the same.
>
>-- Terry
>
>if you send a bounce, and usually some MTAs have trouble to sign the
>bounce they generate (not anymore true with postfix but a bit tricky to
>setup), then the only way to recover the message is to ensure there is
>an SPF aligned on the string presented by HELO.
>
>So basically, I would say DMARC takes only the result of the check_host
>for the MAIL FROM entity which contains the postmaster@helo if the
>RFC5321.mailfrom is empty.
>
>"DMARC uses only the result of the check_host() applied on the MAIL
>FROM entity as defined by RFC7208. The MAIL FROM entity is the one used
>for alignment checking.".
>

This is a perfectly reasonable way to do it.  Spamassassin checks both independently (using Mail::SPF) and applies the results to two different SA tests. 

There's more than one way to do it (Meng Wong is a Perl programmer).

Scott K

---

From: Scott Kitterman <[email-masked]>
Date: Thu, 22 Jan 2015 20:03:28 -0500

>be conflating arguments from another context).

Of course. Section 6.7 discusses this in general terms. If you want to only use SPF as an input to DMARC, then it wouldn't make sense to set up your system to reject mail just based on SPF.

Specifying receiver policy was somewhat contentious in SPFbis.  In the end, RFC7208 specifies almost, if not, exactly the same amount of receiver policy as did RFC4408 (almost none).

Scott K

---

From: [email-masked]
Date: Thu, 22 Jan 2015 17:41:46 -0800 (PST)

> Fix 2: change 7208 to say that SPF can return multiple results.  Ugh.

Filing an erratum for purposes of documenting the issue is fine, but since this
is a substantive change to the protocol it far exceeds the scope of what
approval of an erratum is allowed to do. As such, I believe the best outcome
you can get here would be "held for document update".

				Ned

---

From: Kurt Andersen <[email-masked]>
Date: Thu, 22 Jan 2015 17:59:42 -0800

On Thu, Jan 22, 2015 at 5:03 PM, Scott Kitterman <[email-masked]>
wrote:

> >> processing, then I would probably configure it not to reject based on
> >> SPF fail.  Then the problem doesn't arise.
> >
> >
> Of course. Section 6.7 discusses this in general terms. If you want to
> only use SPF as an input to DMARC, then it wouldn't make sense to set up
> your system to reject mail just based on SPF.
>
> Specifying receiver policy was somewhat contentious in SPFbis.  In the
> end, RFC7208 specifies almost, if not, exactly the same amount of receiver
> policy as did RFC4408 (almost none).
>

I think that the crux of the issue is this:
1) The DMARC spec was written with 4408 as context. That remains true
today, except that in the meantime 7208 was finalized (thanks to SPFbis)
and Murray was seeking to keep up with the times by following the "7208
obsoletes 4408" statement.
2) The key problem is that 7208 changes the checking precedence.  Here's
what the two specs actually say:
4408, section 2.2: "SPF clients MUST check the "MAIL FROM" identity."
7208, section 2.4: "SPF verifiers MUST check the "MAIL FROM" identity if a
"HELO" check either has not been performed or has not reached a definitive
policy. . ."

My recommendation is to take -12 and amend it to change the SPF reference
back to 4408 and let the WG wrestle through how to handle the change that
was introduced in 7208.

--Kurt

---

From: Franck Martin <[email-masked]>
Date: Thu, 22 Jan 2015 20:16:30 -0600 (CST)

----- Original Message -----
> From: [email-masked]
> To: "John Levine" <[email-masked]>
> Cc: [email-masked], [email-masked]
> Sent: Thursday, January 22, 2015 5:41:46 PM
I tried to understand Scott, and I think this is what is missing in RFC7208.

Each check_host() is meant to be done as soon as you have the data to do it.

So after the HELO phase in SMTP RFC5321, you do check_host(HELO identity, IP). You get a result and apply the SPF policy (and disconnect if policy says so)
after the mail from phase, you do check_host(MAIL FROM identity, IP). You get a result and apply the SPF policy (and disconnect if policy says so)

This makes sense then, except when SPF policy on both check_host is not -all

The difficulty then comes when you do these check_host() later in the RFC5321 transaction, as you have to emulate, this serialization.
And then there is still some uncertainty on what to do when both SPF records exist and do not have -all both (or when receivers consider -all as ~all)

I think RFC7208 confused more this serialization, that was implicit in RFC4408, like the elephant in the room. :P

To come back to operational matters, I think SPF implementations just do check_host(MAIL FROM identity, IP) as Terry pointed out, this is why I suggest DMARC spec could just say that: "DMARC uses only the MAIL FROM identity and results of the check_host() of the MAIL FROM identity as defined in RFC7208)"

And we will fix RFC7208 later to match operational deployments, but this ought to be recorded somewhere.

(add reporting to emails, and you get plenty new questions about standardizations :P)

---

From: Franck Martin <[email-masked]>
Date: Thu, 22 Jan 2015 20:21:54 -0600 (CST)

----- Original Message -----

> From: "Kurt Andersen" <[email-masked]>
> To: "Scott Kitterman" <[email-masked]>
> Cc: [email-masked]
> Sent: Thursday, January 22, 2015 5:59:42 PM

> On Thu, Jan 22, 2015 at 5:03 PM, Scott Kitterman < [email-masked] >
> wrote:

> > On January 22, 2015 6:35:59 PM EST, Kurt Andersen < [email-masked] >
> > wrote:
> 
> > >On Thu, Jan 22, 2015 at 3:30 PM, Scott Kitterman < [email-masked] >
> 
> > >wrote:
> 
> > >
> 
> > >configure (some would say usurp) SPF in such a way? I seem to recall
> 
> > >some
> 
> > >contentious discussions about such usurpation during SPFbis (though I
> 
> > >could
> 

> > Of course. Section 6.7 discusses this in general terms. If you want to only
> > use SPF as an input to DMARC, then it wouldn't make sense to set up your
> > system to reject mail just based on SPF.
> 

> > Specifying receiver policy was somewhat contentious in SPFbis. In the end,
> > RFC7208 specifies almost, if not, exactly the same amount of receiver
> > policy

> 1) The DMARC spec was written with 4408 as context. That remains true today,
> except that in the meantime 7208 was finalized (thanks to SPFbis) and Murray
> was seeking to keep up with the times by following the "7208 obsoletes 4408"
> statement.
> 2) The key problem is that 7208 changes the checking precedence. Here's what
> the two specs actually say:

I think this text in 7208 is wrong, if you consider the serialization for the checks implicit in 4408, it should have been written: 
"SPF verifiers MUST check the "MAIL FROM" identity if a "HELO" check either has not been performed or has not reached a definitive fail result that led to the application of the -all policy. . ." 

But I think the fix is a bit more complex to be elegant.

---

From: Scott Kitterman <[email-masked]>
Date: Thu, 22 Jan 2015 21:24:09 -0500

On Thursday, January 22, 2015 17:41:46 Ned Freed wrote:
> > >DMARC leverages the Mail From identity, so I don't see how independent
> > >HELO checks can be relevant.> 
> Filing an erratum for purposes of documenting the issue is fine, but since
> this is a substantive change to the protocol it far exceeds the scope of
> what approval of an erratum is allowed to do. As such, I believe the best
> outcome you can get here would be "held for document update".

Please don't.  There's no error to document.  Order was explicitly discussed 
during SPFbis and is not there by accident.  Just because someone working on 
an unrelated ISE draft thinks it should have been different doesn't create an 
error.

Scott K

---

From: Scott Kitterman <[email-masked]>
Date: Thu, 22 Jan 2015 21:25:49 -0500

On Thursday, January 22, 2015 17:59:42 Kurt Andersen wrote:
> > On January 22, 2015 6:35:59 PM EST, Kurt Andersen <[email-masked]>
> > 
> > wrote:

If you've already rejected the message (e.g. HELO check reached a definitive 
result) the DMARC doesn't care.  There's no relevant change between 4408 and 
7208.

Scott K

---

From: Scott Kitterman <[email-masked]>
Date: Thu, 22 Jan 2015 21:36:31 -0500

On Thursday, January 22, 2015 20:16:30 Franck Martin wrote:
> > nits, while I'm at it> 
> > > >HELO checks can be relevant.
> > > 
> > this
> > approval of an erratum is allowed to do. As such, I believe the best
> So after the HELO phase in SMTP RFC5321, you do check_host(HELO identity,
> IP). You get a result and apply the SPF policy (and disconnect if policy
> says so) after the mail from phase, you do check_host(MAIL FROM identity,
> IP). You get a result and apply the SPF policy (and disconnect if policy
> says so)
> 
> The difficulty then comes when you do these check_host() later in the
> RFC5321 transaction, as you have to emulate, this serialization. And then
> there is still some uncertainty on what to do when both SPF records exist
> and do not have -all both (or when receivers consider -all as ~all)

There is no uncertainty.  What you do it apply local policy.  End of story.  
The only tricky part is you have to decide what local policy is.  This is not 
new.

> I think RFC7208 confused more this serialization, that was implicit in
> RFC4408, like the elephant in the room. :P
> 
> To come back to operational matters, I think SPF implementations just do
> check_host(MAIL FROM identity, IP) as Terry pointed out, this is why I
> suggest DMARC spec could just say that: "DMARC uses only the MAIL FROM
> identity and results of the check_host() of the MAIL FROM identity as
> defined in RFC7208)"
> 

What the code I've written and distributed does is check HELO and if there's 
not a reject (definitive result) store the helo result and then check Mail From 
(using the synthetic postmaster@HELO if needed).  If the message is not 
rejected/deferred then the appropriate result header (SPF Received or auth-
res) is added for both results.  There is a configuration option to only check 
HELO if Mail From is null, but it's not the default.

By default, Postfix (which is what I'm writing for) delays all such checks 
until rcpt to, so no.  There's absolutely no need to do check_host() as soon 
as you have the information.  

IIRC I first wrote that code in about 2007 based on RFC 4408.  The only way it 
didn't do explicitly what 4408 required was that it didn't both doing a Mail 
>From check that would have been pointless since the message was already going 
to be rejected.  We fixed that bug in the spec in RFC 7208 during SPFbis.

I really think this is making a mountain out of the tiniest of molehills.

Recall: The almost complete lack of receiver policy specification in RFC 4408 
(and RFC 7208) is not an oversight.  It was a deliberate design choice.

Scott K

---

From: "John Levine" <[email-masked]>
Date: 23 Jan 2015 03:03:28 -0000

>RFC 7208 doesn't say the HELO result determines anything. It says IF (I say again IF) a decision
>has been reached about message disposition based on the HELO result, there is no requirement to go
>ahead and do a pointless Mail From check. 

While that is certainly one plausible interpretation of the 7208
language, it says "definitive policy result", a phrase that's not
defined anywhere, which may or may not involve a decision about mail
disposition.  A "pass" result certainly strikes me as a definitive
policy result.

>Avoiding a check that has been determined to be pointless is the only change in this area in RFC 7208.

Indeed, and that turns out to be a lot more incompatible than was
appreciated at the time.

R's,
John

---

From: Scott Kitterman <[email-masked]>
Date: Thu, 22 Jan 2015 22:16:58 -0500

On Friday, January 23, 2015 03:03:28 John Levine wrote:
> >RFC 7208 doesn't say the HELO result determines anything. It says IF (I say
> >again IF) a decision has been reached about message disposition based on
> >the HELO result, there is no requirement to go ahead and do a pointless
> >Mail From check.
> 

Pass is an SPF result.  I probably should have added the word local in front 
of policy when I wrote that.  What I wrote above isn't just a random 
interpretation, it's what I meant when I wrote it and what we discussed in the 
working group.

> Indeed, and that turns out to be a lot more incompatible than was

I'm up to accepting that there's some ambiguity in the language, but I don't 
see any actual incompatibility assuming the ambiguity is resolved 
appropriately.

If one changes "definitive policy result" to "definitive local policy result" or 
"definitive receiver policy result" then I think there's no ambiguity.  

I'm still a bit boggled that anyone is confused about this, but obviously they 
are.

Scott K

---

From: Franck Martin <[email-masked]>
Date: Thu, 22 Jan 2015 22:04:59 -0600 (CST)

----- Original Message -----
> Sent: Thursday, January 22, 2015 7:16:58 PM

To try to explain the confusion...

Well, DKIM is easy, DKIM is valid or is not (I'm excluding temp failures due to DNS etc..). The DKIM spec tells what the dkim result MUST be, and then the receiver do whatever with this result.

With SPF, the spf=pass/fail result (as shown in the authentication-result header) is not depending on the sender policy as expressed in the SPF record, but at whatever the receiver policy is...

Usually, a SPEC will tell the receiver what it SHOULD do to define the spf= status, but instead it seems RFC7208 says put whatever result you feel like... Therefore different implementations will produce different results.

I could have an implementation that checks HELO only and check MAILFROM and ignore the last result to put in the SPF result and that would be in accordance to RFC7208.
I could have an implementation that checks HELO and MAILFROM, where an helo pass is better than a MAILFROM fail or softfail, to put SPF=pass as result and that would be still in accordance with RFC7208.

or I'm mistaken?

---

From: Scott Kitterman <[email-masked]>
Date: Thu, 22 Jan 2015 23:41:39 -0500

On Thursday, January 22, 2015 22:04:59 Franck Martin wrote:
> > I'm up to accepting that there's some ambiguity in the language, but I
> > don't see any actual incompatibility assuming the ambiguity is resolved
> > appropriately.
> > 
> Well, DKIM is easy, DKIM is valid or is not (I'm excluding temp failures due
> to DNS etc..). The DKIM spec tells what the dkim result MUST be, and then
> the receiver do whatever with this result.
> 
> With SPF, the spf=pass/fail result (as shown in the authentication-result
> header) is not depending on the sender policy as expressed in the SPF
> record, but at whatever the receiver policy is...

No.  An SPF result is the deterministic.  It's a combination of domain, 
identity, and result.  This is always true and always consistent.

Where the variation is in what the receiver decides to do.  This is exactly 
the same as DKIM.  I think the confusion is that people think SPF does more 
because of the name and because at one time (pre-RFC) it did.  In hind sight, 
we'd have been much better off to keep the original name: Sender Permitted 
From.

> Usually, a SPEC will tell the receiver what it SHOULD do to define the spf=
> status, but instead it seems RFC7208 says put whatever result you feel
> like... Therefore different implementations will produce different results.
> 
> I could have an implementation that checks HELO only and check MAILFROM and
> ignore the last result to put in the SPF result and that would be in
> accordance to RFC7208. I could have an implementation that checks HELO and
> MAILFROM, where an helo pass is better than a MAILFROM fail or softfail, to
> put SPF=pass as result and that would be still in accordance with RFC7208.
> 
> or I'm mistaken?

I think your mistake to insist that there be one and only one defined way to 
deal with SPF results from both HELO and Mail From and there isn't.  Since RFC 
7208 doesn't attempt (except in one special case) to dictate to receivers, 
it's not at all surprising the you fail to find direction on what to do as a 
receiver.

Scott K

---

From: Scott Kitterman <[email-masked]>
Date: Thu, 22 Jan 2015 19:54:18 -0500

>http://www.blackops.org/~msk/dmarc/diff.html

I think what's in the diff is correct and a good clarification. I'm not sure the problem it fixes qualifies as serious, although the longer this thread goes on, the more I lean in that direction. 

Scott K

---

From: Scott Kitterman <[email-masked]>
Date: Thu, 22 Jan 2015 17:21:50 -0500

On January 22, 2015 4:20:35 PM EST, Michael Jack Assels <[email-masked]> wrote:
>more tiny nits, while I'm at it
>> > 

I think that's close. DMARC doesn't do SPF, it uses SPF results. Nothing contrary to RFC 7208 (or 4408).  

Scott K

---

From: Franck Martin <[email-masked]>
Date: Thu, 22 Jan 2015 23:52:58 -0600 (CST)

----- Original Message -----
> Sent: Thursday, January 22, 2015 8:41:39 PM
> > > > >RFC 7208 doesn't say the HELO result determines anything. It says IF
> > > > >(I
> > > > >say
> > > > >

I know the receiver can do whatever of the result, and anyhow the receiver, its rules.

but I'm sorry I don't read anywhere in RFC7208 where f() is defined.

spfresult in (pass,fail,softfail,permfail,tmpfail,none,...)=f(check_host(HELO identity, IP), check_host(MAIL FROM identity,IP))

it may be clear to you, but it is certainly not for me. Would you please define f()?

(note calling MAIL FROM a combination of RFC5321.mailfrom and [email-masked] does not help clarity either).

---

From: Scott Kitterman <[email-masked]>
Date: Fri, 23 Jan 2015 01:14:56 -0500

On Thursday, January 22, 2015 23:52:58 Franck Martin wrote:
> > > > > >Avoiding a check that has been determined to be pointless is the
> > > > > >only
> > > > > >change in this area in RFC 7208.
> > > > > 
> > > > I'm still a bit boggled that anyone is confused about this, but
> > > > obviously
> > > > they
> > > > are.
> > > 
> > because of the name and because at one time (pre-RFC) it did.  In hind
> > sight, we'd have been much better off to keep the original name: Sender
> > Permitted From.
> 
> spfresult in
> (pass,fail,softfail,permfail,tmpfail,none,...)=f(check_host(HELO identity,
> IP), check_host(MAIL FROM identity,IP))
> 
> (note calling MAIL FROM a combination of RFC5321.mailfrom and
> [email-masked] does not help clarity either).

Probably not, but I don't know how else to have dealt with it.

f() doesn't exist.  SPF's check_host() has inputs and an output.  If you call 
it twice then you have two outputs to decide how to aggregate.

The way I've done it typically is something like (let's leave SPF check 
whitelisting out of the equation to keep it relatively simple):

spfresult.helo in 
(pass,fail,softfail,permfail,tmpfail,none,...)=f(check_host(HELO identity, IP)

Apply local policy:

if spfresult.helo == fail, neutral, softfail, permerror then reject
if spfresult.helo == temperror then defer
else no definitive result

if no definitive result:

If Mail From == "" then Mail From = postmaster@HELO identity

spfresult.mailfrom in 
(pass,fail,softfail,permfail,tmpfail,none,...)=f(check_host(Mail From 
identity, IP)

Apply local policy:

if spfresult.mailfrom == fail, permerror then reject
if spfresult.mailfrom == temperror then defer
else no definitive result

If no definitive result, add Received SPF or A-R header fields for both checks 
and complete the SPF check (the messsage may still end up rejected, deferred, 
spamfoldered, dropped, or accepted based on other checks - I wouldn't 
recommend accepting just on SPF).

The follow-on processing might include DMARC, but it's unrelated.

Scott K

Note: This is roughly the default processing the SPF policy server I wrote for 
postfix.  There's lots of ways to do this and most of them wouldn't be wrong.

---

From: Hector Santos <[email-masked]>
Date: Fri, 23 Jan 2015 12:03:54 -0500

On 1/22/2015 8:59 PM, Kurt Andersen wrote:
> today, except that in the meantime 7208 was finalized (thanks to
> SPFbis) and Murray was seeking to keep up with the times by following
> the "7208 obsoletes 4408" statement.
> 2) The key problem is that 7208 changes the checking precedence.
> Here's what the two specs actually say:
> 7208, section 2.4: "SPF verifiers MUST check the "MAIL FROM" identity
> if a "HELO" check either has not been performed or has not reached a
> definitive policy. . ."
>
> My recommendation is to take -12 and amend it to change the SPF
> reference back to 4408 and let the WG wrestle through how to handle
> the change that was introduced in 7208.
>
> --Kurt

+0.8

I believe this -12 rev "jumped the gun" to presume 4408 is obsolete 
when its not in practice and probably won't be for a very long time.

I understand the IETF idea that 7208 updates 4408 but there are 
differences besides the HELO issue and the reality would be there may 
be both type of SPF receivers and that includes support for 
5322.Authentication-Result (AUTH-RES) in 7208, and 4408 which only 
supports 5322.Received-SPF.  7208 was suppose to be compatible to both 
possible consumers of these 5322 reporting headers.

So if DMARC requires AUTH-RES, then it should reference 7208 for this 
key reason and it should also continue to reference 4408 to provide 
design semantics regarding adding DMARC support existing 4408 only 
systems with the official SPF reporting trace header. Its possible for 
DMARC implementations to add support for detecting both reporting 
headers.  That would be an "informational" type of design semantics 
that should be in this document, if not already, i.e, future DMARC 
implementations need to understand that current software SPF modules 
SPF may not be 7208 ready and will A) change it if they can, and/or b) 
add support for SPF 4408 Received-SPF results.

For new developers, starting with this Informational version of DMARC, 
there is a mix of inconsistencies. And until the official IETF 
standard track "solid spec" version is done, this IETF-produced 
informational doc will be the "pseudo" standard to work with.  So it 
should be ready for the "solid spec."

Finally, just look at its ADSP references, with semantics, direct or 
indirect, regarding ADSP, it should probably be removed at this point. 
  The informational doc should be as clean and clear as possible about 
what is expected to come and that includes support for 3rd party 
Policy augmented protocol extensions.

One last thing, it should also add some Migration considerations such as:

   - SPF 4408 to SPF 7208,
   - ADSP to DMARC,
   - Other future software enhancement "Get Ready" considerations.

For future implementations that do not use the open source libraries, 
the "OpenDMarc" or any other 3rd party package, this would be a big 
item.

-- 
HLS

---

From: Hector Santos <[email-masked]>
Date: Fri, 23 Jan 2015 13:06:43 -0500

> We don't check both 5321.MailFrom AND HELO/EHLO for SPF. I've never found this to be a problem.

SPF Software that also supports:

    RFC4405   SUBMITTER SMTP Service Extension
    RFC4406   Sender ID: Authenticating E-Mail
    RFC4407   Purported Responsible Address (PRA)

will also look at the PRA for SPF calculations.  Enable the SMTP 
Extension 4405 and watch many of the sessions using the SUBMITTER 
field in MAIL FROM:

> I don't know how it works in Hotmail (although I should) but it is moving over to the Office 365 infrastructure over the next several months anyhow so it will be the same.

Its good to know that Microsoft is single sourcing it.

-- 
HLS

---

From: Franck Martin <[email-masked]>
Date: Fri, 23 Jan 2015 12:36:45 -0600 (CST)

----- Original Message -----

> http://www.blackops.org/~msk/dmarc/diff.html

So after the whole discussion, I don't think in 4.1 we can say "in contradiction of SPF". DMARC is defining a "local policy" for SPF, which is valid. People implementting DMARC must ensure their SPF implementation follow also this local policy. 

I would cite Terry's email where his SPF follows what DMARC expects, and where Scott said it is a valid implementation of SPF, and Scott implementation of SPF, which is valid too. I would also cite Tim's conformance tests, that matches operational deployment. 

So to be pedantic, suggested text: 
[SPF], which authenticates the HELO identity and the MAIL FROM identity: the domain found in an [SMTP] MAIL 
command, or the domain found in the HELO/EHLO command if the MAIL 
command has a null path. Note that in 
the context of DMARC, this later identity is only used.

---

From: Franck Martin <[email-masked]>
Date: Fri, 23 Jan 2015 12:45:02 -0600 (CST)

----- Original Message -----
> Sent: Thursday, January 22, 2015 10:14:56 PM

First thanks, for describing in details your implementation. Very useful.

> it twice then you have two outputs to decide how to aggregate.

Yes this is where the point is, RFC7208 does not define how to aggregate the results. This is the part which is not clear in the spec, and should be clearly spelled out, me thinks.

---

From: Scott Kitterman <[email-masked]>
Date: Fri, 23 Jan 2015 15:06:56 -0500

On Friday, January 23, 2015 13:06:43 Hector Santos wrote:
> > 1. When checking SPF, use the domain in the 5321.MailFrom. If it is empty,
> > use the domain in the HELO/EHLO. 2. Use the domain extracted from (1)
> > when doing the DMARC alignment check for SPF.
> > 

Hector,

As you know, those are Sender ID, not SPF.  

Scott K

---

From: Scott Kitterman <[email-masked]>
Date: Fri, 23 Jan 2015 15:30:22 -0500

On Friday, January 23, 2015 12:45:02 Franck Martin wrote:
> > > I know the receiver can do whatever of the result, and anyhow the
> > > receiver,
> > > its rules.
> > > 
> > f() doesn't exist.  SPF's check_host() has inputs and an output.  If you
> > call it twice then you have two outputs to decide how to aggregate.
> 
> Yes this is where the point is, RFC7208 does not define how to aggregate the
> results. This is the part which is not clear in the spec, and should be
> clearly spelled out, me thinks.

I guess that's where we'll have to disagree.  I don't know what RFC 7208 could 
have said beyond "How to aggregate SPF HELO and SPF Mail From checks is a 
matter of local policy".  That doesn't really help much.  Personally, I don't 
think they can be successfully aggregated into a single SPF result since they 
are about different things.

Scott K

---

From: Franck Martin <[email-masked]>
Date: Fri, 23 Jan 2015 16:08:26 -0600 (CST)

----- Original Message -----
> Sent: Friday, January 23, 2015 12:30:22 PM
> > > > it may be clear to you, but it is certainly not for me. Would you
> > > > please
> > > > define f()?
> > > > 

No, no, we don't disagree, I did not express myself correctly. I would like to see this text you just wrote in the spec

may be something like:
"Aggregating SPF HELO and SPF Mail From checks is a matter of local policy. This document does not defines a single result nor how to get one, but defines a result for each check."

---

From: Hector Santos <[email-masked]>
Date: Fri, 23 Jan 2015 17:54:34 -0500

> As you know, those are Sender ID, not SPF.

Correct, but as pointed out during SPFBIS, the integration of all the 
protocols is possible as it is with our package.  When the PRA is 
passed via the MAIL FROM SUBMITTER field, this could override or may 
be used in conjunction with 5321.MAIL FROM.

Grant it, we are probably the only SMTP package that supports 
SUBMITTER at the server side so it may not mean much to you, but the 
fact remains, if you turn on the SMTP extension, you will get a high 
rate of clients that issue the MAIL FROM SUBMITTER=pra field.  You 
choose to not consider this high possibility of SUBMITTER SPF sessions 
during SPFBIS.  It may be a consideration for DMARC Informational doc.

Just a quick view of todays SMTP trace logs, here is one of many SMTP 
sessions with the SUBMITTER field.

**************************************************************************
Wildcat! ESMTP Server v7.0.454.4
SMTP log started at Fri, 23 Jan 2015  01:19:19
Connection Time: 20150123 01:19:19  cid: 0000169A tid: 0000159C
SSL-Enabled=YES No-Quit-Cancel=OFF Receiver-Bin=ON
Client IP: 216.98.142.131:59607 (mx131.gbm22.com) Host IP: 
208.247.131.9:25
01:19:19.824 S: 220-winserver.com Wildcat! ESMTP Server v7.0.454.4 ready
01:19:19.824 S: 220-************** WARNING:  FOR AUTHORIZED USE ONLY! 
**********************
01:19:19.825 S: 220-* THIS SYSTEM DO NOT AUTHORIZE THE USE OF ITS 
PROPRIETARY COMPUTERS    *
01:19:19.825 S: 220-* AND COMPUTER NETWORKS TO ACCEPT, TRANSMIT, OR 
DISTRIBUTE UNSOLICITED *
01:19:19.825 S: 220-* BULK E-MAIL SENT FROM THE INTERNET. THIS SYSTEM 
WILL RESTRICT ACCESS *
01:19:19.825 S: 220-* TO CAN-SPAM (US S. 877) COMPLIANT CLIENTS ONLY. 
                      *
01:19:19.825 S: 220 
************************************************************************
01:19:20.039 C: EHLO mx131.gbm22.com
01:19:20.041 S: 250-winserver.com, Hello mx131.gbm22.com, pleased to 
meet you.
01:19:20.041 S: 250-SIZE 10240000
01:19:20.041 S: 250-8BITMIME
01:19:20.041 S: 250-SUBMITTER
01:19:20.041 S: 250-ETRN
01:19:20.041 S: 250-AUTH CRAM-MD5 DIGEST-MD5 LOGIN PLAIN PLAIN-MD5 SHA-1
01:19:20.041 S: 250-AUTH=LOGIN
01:19:20.041 S: 250-HELP
01:19:20.041 S: 250 STARTTLS
01:19:20.294 C: MAIL 
FROM:<support-eric.anderson=[email-masked]> BODY=7BIT 
SUBMITTER=[email-masked]
01:19:20.295 S: 250 
<support-eric.anderson=[email-masked]>... Sender 
validation pending. Continue.
01:19:20.401 C: RCPT TO:<[email-masked]>
01:19:20.402 S: 550 User not a member of domain: 
<[email-masked]>
01:19:20.511 C: QUIT
01:19:20.511 S: 221 closing connection
01:19:20.511 ** Completed. Elapsed Time: 1124 msecs

First, for our receiver, note the delayed Sender validation was 
pending (SPF checking, among other envelope parameters filter checks) 
until RCPT TO is known.

Next, note what the SPF and DMARC records at:

gbresponder.com
         "v=spf1 a mx a:gbresponder.com ip4:71.6.179.3 
ip4:71.6.179.0/24 ip4:167.88.9.128/26 -all"

_dmarc.gbresponder.com
         "v=DMARC1; p=none; rf=afrf; rua=mailto:[email-masked]"

The client domain name mx131.gbm22.com has no record and there was a 
short delay.  Just to illustrate that these are still high overhead items.

Our package will still support SUBMITTER until I have a reason not to. 
I don't think its a big thing because most of the time, the domains 
all align up.

I am not making a case for submitter, just to point that there are 
still operations out there using it, especially at the client side. 
The package with such a client COULD also be supporting the server 
side of submitter SPF processing as well.

A case can be made that the SUBMITTER=PRA is the same as the 5322.From 
field, its possible to explore this as a way to determine the DMARC 
policy before the DATA overhead is transferred.

-- 
HLS

---

From: Michael Jack Assels <[email-masked]>
Date: Fri, 23 Jan 2015 19:06:12 -0500

What seems like ages ago, on Thu, 22 Jan 2015 10:27:40 PST, 

> http://www.blackops.org/~msk/dmarc/diff.html

That sound to me like a call for thumbs up or thumbs down.  I'd
personally like to change the infelicitous word "contradiction"
to "contrast", but rather than argue about changes, I think any
remaining argument should be simply about making the change as
proposed by Murray or not making any change at all.  I've paid
attention to all the points raised, and I still think -13 is
better than -12.  I don't think the floor is open for -14.

MJA

---

From: Franck Martin <[email-masked]>
Date: Fri, 23 Jan 2015 18:15:38 -0600 (CST)

----- Original Message -----
> Sent: Friday, January 23, 2015 4:06:12 PM

> better than -12.  I don't think the floor is open for -14.
Yes "contrast" would make me more happy.