Subject: [spfbis] WGLC: draft-ietf-spfbis-4408bis-14

From: Andrew Sullivan <[email-masked]>
Date: Tue, 9 Apr 2013 02:24:32 -0400

Dear colleagues,

This message initiates a two-week working group last call on
draft-ietf-spfbis-4408bis-14, our only work item.  

The draft has been through 14 revisions, and all open issues in the
tracker are closed.  Please review the document and indicate any
comments you have by sending a note to the mailing list.  If you have
no comments, it is valuable nevertheless to learn that you have read
the document and support its publication.

WGLC will close at 2014-04-24 00:00 UTC.

SM will be the shepherd for this document.

Best regards,

Andrew (for the chairs)

-- 
Andrew Sullivan
[email-masked]

---

From: Scott Kitterman <[email-masked]>
Date: Tue, 16 Apr 2013 21:51:07 -0400

> SM will be the shepherd for this document.

Just as a reminder, we're ~half way through the WGLC period and things have 
been very quiet.  Please don't save all your reviews/comments until the end.

Scott K

---

From: Dotzero <[email-masked]>
Date: Tue, 16 Apr 2013 22:08:12 -0400

I've reviewed the draft and have one "nit" and one question/comment:

In the Abstract the use of ADMD should be followed with
(ADministrative Management Domains)

   Email on the Internet can be forged in a number of ways.  In
   particular, existing protocols place no restriction on what a sending
   host can use as the "MAIL FROM" of a message or the domain given on
   the SMTP HELO/EHLO commands.  This document describes version 1 of
   the Sender Policy Framework (SPF) protocol, whereby an ADMD can
   explicitly authorize the hosts that are allowed to use its domain
   names, and a receiving host can check such authorization.

For section 2.6.7.  Permerror

   A "permerror" result means the domain's published records could not
   be correctly interpreted.  This signals an error condition that
   definitely requires manual intervention to be resolved.

Manual intervention by whom? Does this need to be clarified?

Other than these two relatively minor details, nothing jumped out at me.

Mike

On Tue, Apr 9, 2013 at 2:24 AM, Andrew Sullivan <[email-masked]> wrote:
> _______________________________________________
> spfbis mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/spfbis

---

From: Tim Draegen <[email-masked]>
Date: Tue, 16 Apr 2013 22:10:02 -0400

>   names, and a receiving host can check such authorization.

Oh dude!   Get out of my head!   I'm reviewing the document (about 5 minutes in) and this was my first note!

---

From: Dotzero <[email-masked]>
Date: Tue, 16 Apr 2013 22:42:44 -0400

> Oh dude!   Get out of my head!   I'm reviewing the document (about 5 minutes in) and this was my first note!

At least what I found is relatively minor in the greater scheme of
things. Reading the whole thing thoughtfully is so different from
going back and forth on individual items.

Mike

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Tue, 16 Apr 2013 20:02:51 -0700

Apologies; I do plan to spend time on it this week.  I've been buried with
$DAYJOB, illness, travel, etc. lately and I'm quite behind on non-trivial
IETF matters.

---

From: Tim Draegen <[email-masked]>
Date: Wed, 17 Apr 2013 00:04:03 -0400

Read the document, support its publication.  Notes from review:

- Abstract mentioned ADMD before defined
- extra quote first sentence of 2.2
- 2nd paragraph of 2.5 references "return-path", should be "MAIL FROM"
    - and 10.1.3, except the context references RFC5321 so might be ok
- 10.2: missing ")" in first paragraph
- Grammar nits in Appendix C.  a lot of the bullets have naked references at the
  end of sentences.  Maybe put parenthesis around these?

=- Tim

---

From: S Moonesamy <[email-masked]>
Date: Tue, 16 Apr 2013 21:50:35 -0700

Hi Mike, Tim,
>going back and forth on individual items.

As Scott mentioned, things have been very quiet for this WGLC.  It 
helps if there are people who read the draft as you did above as I 
can determine whether the working group reviewed the draft and is ok with it.

Regards,
S. Moonesamy (as document shepherd)

---

From: Commerco WebMaster <[email-masked]>
Date: Wed, 17 Apr 2013 03:01:02 -0600

Having reviewed the draft again end to end, I had pretty much the same 
minor "nit" as Mike (AKA Dotzero) has already expressed on the list 
regarding expanding the ADMD acronym in the Abstract section.

As the Abstract is the first place in the entire document where the ADMD 
acronym gets used, perhaps it should be expressed as ADministrative 
Management Domain (ADMD) to mirror what exists in Section 2.2.1. 
Originator of RFC5598, from whence the acronym comes, as later 
documented in the 4408bis-14 section 1. Introduction.

Also perhaps the ADMDs (ADministrative Management Domains, see 
[RFC5598]), as it currently exists, should be ADministrative Management 
Domains (ADMDs, see[RFC5598]), closely mirroring the plural as seen in 
Section 2.3. Administrative Actors of RFC5598 with attribution to same 
as already exists in section 1. Introduction of SPFbis-4408bis-14.

Overall, the work product contained in the SPFbis looks solid and all 
participants should be congratulated in helping to bring the SPFbis to 
this point.  I think that special thanks to Editor Scott Kitterman and 
the co-chairs for all their work in keeping the process on track and 
constructive are in order.

Finally, I also agree that the publication of this draft version 
(draft-ietf-spfbis-4408bis-14) should move forward with suggested 
changes at the editor's and co-chair's discretion.

Best,

Alan M.

---

From: Alessandro Vesely <[email-masked]>
Date: Thu, 18 Apr 2013 19:44:13 +0200

On Wed 17/Apr/2013 03:51:07 +0200 Scott Kitterman wrote:
> Just as a reminder, we're ~half way through the WGLC period

Oh, you mean the close date was a typo?

Here's my notes, mostly nits.  I pasted lots of text from the I-D, so
that issues can be evaluated even without getting at the document.

*Abstract*

 Email on the Internet can be forged in a number of ways.  In
 particular, existing protocols place no restriction on [...]

That was good in 2004, now SPF exists already.  A possible replacement
could be, e.g.:

 This document describes version 1 of the Sender Policy Framework
 (SPF) protocol, whereby an Administration Management Domain (ADMD)
 can explicitly authorize the hosts that are allowed to use its domain
 names.  A receiving host can check such authorization based on the IP
 address of the sending client and the "MAIL FROM" of a message or
 the domain given on the SMTP HELO/EHLO commands.

1. *Introduction*

   The current email infrastructure has the property that any host
   injecting mail into the system can use any DNS domain name it wants
   in each of the various identifiers specified by [RFC5321] and
   [RFC5322].

This sounds oldish too.  SPF is not the only email authentication
method.  Perhaps we could use the introduction to explain the differences.

1.2. *check_host()*

 Section 4 introduces an algorithm to evaluate an SPF policy against
 an arriving email transaction.

s/arriving email transaction/incoming email message/?

2.5. *Location of Checks*

 The authorization check SHOULD be performed during the processing of
 the SMTP transaction that sends the mail.

s/sends/receives/?

3. *SPF Records*

Do we wish to give an example of multi-line format in the zone file?

4.3. *Initial Processing*

                                        Internationalized domain names
 MUST be encoded as A-labels, as described in Section 2.3 of
 [RFC5890].on 2.3 of [RFC5890].

s/on 2.3 of [RFC5890].// once.

4.4. *Record Lookup*

 If all DNS lookups that are made return a server failure (RCODE 2),
 or other error (RCODE other than 0 or 3), or time out, then
 check_host() terminates immediately with the result "temperror".

That "all" seems to refer to the SPF and TXT types.  I'd say:

 If the query returns a server failure (RCODE 2),

4.6.4. *DNS Lookup Limits*

 These limits are per mechanism or macro in the record, and are in
 addition to the lookup limits specified above.

Hm... above where?  Perhaps splitting the section into, say:

   4.6.4.1 *Total Limits*, and
   4.6.4.2 *Particular cases*

would allow to specify it more clearly.  If you do so, then move the
following (last two) paragraphs up.

4.7. *Default Result*

OLD
   Note that records SHOULD always use either a "redirect" modifier or
   an "all" mechanism to explicitly terminate processing.
NEW
   The RECOMMENDED practice is to use either a "redirect" modifier or
   an "all" mechanism to explicitly terminate the record.

(A reader has no way to actually "note" that, and modifiers don't
always terminate processing.)

5. *Mechanism Definitions*

OLD
 Designated sender mechanisms are used to designate a set of <ip>
NEW
 Designated-sender mechanisms are used to qualify a set of <ip>

(Just avoiding to explain a term with itself.)

                                       SPF implementations on IPv6
 servers need to handle both "AAAA" and "A" secords, for clients on
 IPv4 mapped IPv6 addresses [RFC4291].

s/secords/records/

5.2. *"include"*

<vspace/>
          <!-- FIXME: prevent automatic line break after '"mx:' -->

I have no suggestions.  Do we need it?

5.5. *"ptr" (do not use)*

                           After many years of SPF deployment
 experience it has been concluded it is unnecessary and more reliable
 alternatives used instead.

Maybe it's me, or the page break, but I think a comma after
"experience" would make it more readable.

5.6. *"ip4" and "ip6"*

Production rules can be set as mentioned in
http://www.ietf.org/mail-archive/web/spfbis/current/msg03266.html

6.1. *redirect: Redirected Query*

There is no statement on whether redirect= can be applied by an
"include" mechanism.

6.2. *exp: Explanation*

 If check_host() results in a "fail" due to a mechanism match (such as
 "-all"), and the "exp" modifier is present, then [...]

This is incomprehensible after the reorganization.  I'd add "and the
verifier is going to reject the message", and a link to Section 8.4 Fail.

8. Result Handling

                     As such, there is no normative requirement for
 message handling in response to any particular result.  This section
 is provided to present a complete picture of the likely cause of each
 result, and where available, the experience gained during
 experimental deployment.

Please remove those two sentences, as they are not true.  We could say
there is no normative requirement for rejecting a message.  But, for
example, for softfail we correctly have normative requirements:

 Receiving software SHOULD NOT reject the message based solely on this
 result, but MAY subject the message to closer scrutiny than normal.

This not holds for 10.2. Receivers too.

8.2. *Neutral*

 A "neutral" result indicates that although a policy for the identity
 was discovered, there is no definite assertion about the (positive or
 negative) about the client.

Remove the first "about the".

8.7. *Permerror*

                                                  It is also possible
 that this result is generated by certain SPF clients due to the input
 arguments having an unexpected format; see Section 4.8.

I'd s/client/verifiers/.  Raw "client" throughout the document refers
to SMTP clients.  "SPF client" is mentioned a few times and could
cause confusion.

There is no mention of "exp".  Is its usage permitted/recommended?

Missing reference to Appendix H.3, but see below.

9.2. *SPF Results in the Authentication-Results Header Field*

 It is, however, possible to add CFWS in the "reason" part of an
 Authentication-Results header field and provide the equivalent
 information, if desired.

s/CFWS/key-value pairs/?

10. Effects on Infrastructure

 a comprehensive list of what such entities SHOULD do in light of this
 document.

Is that /meant/ to be normative?

10.1.2. Administrator's Considerations

 changes are common, it is better to use the less resource intensive
 mechanisms like "ip4" and "ip6" over "a" or "a" or "mx".

s/"a" or "mx"/"a" over "mx"/

 The hostname is generally the identity used in the 5321.HELO/.EHLO
 command.  In the case of messages with a null 5321.MailFrom, this is
 used as the domain for 5321.MailFrom SPF checks, in addition to being
 used in 5321.HELO/.EHLO based SPF checks.

"hostname" is called "domain" in the previous paragraph, and the long
slash-dot form of the names is used.  In addition to reword that, it
may be worth to note that [email-masked] can be used as an
address even if there's no corresponding MX.

10.2. *Receivers*

                       As stated elsewhere in this document, there is
 no normative requirement for specific handling of a message based on
 any SPF result.

See 8. Result Handling above.

11.1. *Processing Limits*

                         Using SPF clients also allows the attacker to
    hide the true source of the attack.

s/clients/verifiers/.

11.5.1. *Recorded Results*

 This information, passed to the receiver in the Received-SPF: or
 Authentication-Results: trace fields, may be returned to the client
 MTA as an SMTP rejection message.  If such an SMTP rejection message

s/may/can/ if needed.

Appendix H. *Local Policy Considerations*

Do we still need to have this text in an appendix?  It seems to be
possible to insert each subsection within the relevant subsection of
Section 8.  IMHO that would improve clarity.

Considerations for temperror are missing, anyway.

Appendix I. *Protocol Status*

s/techincal/technical/

P.S.: what widely deployed extensions to SPF?

Appendix J. *Change History*

    Resolved Section 2.5.7 PermError on invalid domains after macro
    expansion errata in favor of documenting that different clients
    produce different results.

s/clients/verifiers/.

---

From: Scott Kitterman <[email-masked]>
Date: Fri, 19 Apr 2013 00:18:52 -0400

>    names, and a receiving host can check such authorization.

Fixed locally.  I moved it up from the next paragraph to this one.

> Manual intervention by whom? Does this need to be clarified?

Since it's an error in the record, I think it's not something that needs to be 
changed.  Anyone else?

> Other than these two relatively minor details, nothing jumped out at me.
Thanks for the review.

Scott K

---

From: Scott Kitterman <[email-masked]>
Date: Fri, 19 Apr 2013 00:33:04 -0400

> - Abstract mentioned ADMD before defined

Fixed locally.

> - extra quote first sentence of 2.2

Fixed locally.

> - 2nd paragraph of 2.5 references "return-path", should be "MAIL FROM"

Fixed locally.

>     - and 10.1.3, except the context references RFC5321 so might be ok

Fixed locally.  That makes it consistent.

> - 10.2: missing ")" in first paragraph

Fixed locally.

> - Grammar nits in Appendix C.  a lot of the bullets have naked references at
> the end of sentences.  Maybe put parenthesis around these?

Fixed things up locally, so it should be better now.

Thanks for the review.

Scott K

---

From: Scott Kitterman <[email-masked]>
Date: Fri, 19 Apr 2013 00:38:51 -0400

> as already exists in section 1. Introduction of SPFbis-4408bis-14.

I made it plural.

> changes at the editor's and co-chair's discretion.

Thanks for the review,

Scott K

---

From: "John Levine" <[email-masked]>
Date: 20 Apr 2013 19:46:10 -0000

These are mostly nits.  I'm not worrying about typos and minor
grammatical issues, since the skilled staff at the RFC production
center will find them better than I do.

2.1 HELO identity

   Checking "HELO" promotes consistency of results and can reduce DNS resource usage. 

I understand the consistency, but how does doing two lookups rather
than one reduce DNS resource usage?  Suggest just removing the second
clause.

2.4  Checking Authorization

  Without explicit approval of the domain owner, checking other
  identities against SPF version 1 records is NOT RECOMMENDED because
  there are cases that are known to give incorrect results. For example,
  almost all mailing lists rewrite the "MAIL FROM" identity (see Section
  10.3), but some do not change any other identities in the message.
  Documents that define other identities will have to define the method
  for explicit approval.

I realize that this paragraph translates to "Sender-ID sucks", but as
it stands it raises more questions than it answers, e.g., how would a
receiver know what a domain owner approves, and what should I check
then.  Since Sender-ID is dead, I suggest removing it.

  When a mail receiver decides to perform an SPF check, it MUST use a
  correctly-implemented check_host() function (Section 4) evaluated with
  the correct parameters. Although the test as a whole is optional, once
  it has been decided to perform a test it has to be performed as
  specified so that the correct semantics are preserved between
  publisher and receiver.

If people haven't already figured out that they have to implement the
spec correctly, this won't help.  I suggest removing it.

2.6. Results of Evaluation

  An SPF verifier implements something semantically identical to the
  function defined there.

Suggest "semantically equivalent" to match langauge in Sec 4. 

3. SPF Records

   The SPF record is a single string of text. 

Suggest "The SPF record is interpreted as a single string of text" since in
fact the record may well have several strings.

8.4. Fail

Suggest moving everything after the first paragraph to App. H.2. since
it's basically an example.

8.5. Softfail

Suggest moving everything after the first paragraph to a new
subsection of App. H.  Please do our users a favor and remove the bad
advice about highlighting failures in MUAs.

9.2. SPF Results in the Authentication-Results Header Field

Odd bug in the xml, the last para and example are switched in
the html output.

R's,
John

---

From: Scott Kitterman <[email-masked]>
Date: Sat, 20 Apr 2013 18:06 -0400

> Oh, you mean the close date was a typo?

No.  April 17 is roughly half way between April 9 and April 24.

>  the domain given on the SMTP HELO/EHLO commands.

I would rather not rewrite the abstract at this stage.  Yes, SPF has existed 
for a long time, but for many it's been "only an experiment", so I think the 
current text is good.

> method.  Perhaps we could use the introduction to explain the differences.

We did discuss including information about DKIM and how it related to SPF, but 
there was push back. I think that since anything about DKIM is out of scope, 
we ought not get into it.  No where in the existing text does it claim SPF is 
the only solution.

> s/arriving email transaction/incoming email message/?

That seems OK to me, but I didn't write that, so I'll wait for others to chime 
in to change it.

Fixed locally. 
> Do we wish to give an example of multi-line format in the zone file?

No, I think zone file tutorials are out of scope.  There's plenty of 
documentation on making zone files and not everyone uses BIND anyway.

Fixed locally.
>  If the query returns a server failure (RCODE 2),

Fixed locally.  I kept DNS lookup instead of query, but it's all singular now.
> following (last two) paragraphs up.

above is the previous paragraph.

I'm not sure that helps.  I'd like to hear other opinions.
> always terminate processing.)

I got rid of "Note that" and I also changed the word included to provided at 
the end of the paragraph (to avoid possible confusion about the include 
mechanism).
> (Just avoiding to explain a term with itself.)

Fixed using identify in place of designate.

Fixed locally.
> I have no suggestions.  Do we need it?

No.  Not needed, so removed. 
> "experience" would make it more readable.

Added.
> http://www.ietf.org/mail-archive/web/spfbis/current/msg03266.html

I think RFC 4291 is a better source for IPv6 than RFC 3986.  For IPv4, that 
looks like it might be a good idea.  I'd appreciate input from people with 
more ABNF experience than me before I change anything.
> There is no statement on whether redirect= can be applied by an

I don't understand the question?
> verifier is going to reject the message", and a link to Section 8.4 Fail.

When I first looked at this, I agreed with you, but you have to consider that 
when the document is referring to 'returning' the explanation, it's 
check_host() returning it to the calling application.  Check_host() doesn't 
know if the message will be rejected or not. 

I've added "... MUST be returned to the calling application." to the end of 
the paragraph to (hopefully) clarify this.

>  result, but MAY subject the message to closer scrutiny than normal.

The statement is generally true, but as you state not 100% correct.  I've 
added the word comprehensive so that it now reads, "... comprehensive 
normative requirement ..." to make it clearer that this is generally true even 
though there are a few exceptions.

I tried to clarify it there too.  The relevant sentence now reads:

> As stated elsewhere in this document, there is no comprehensive
> normative requirement for specific handling of a message based
> on SPF results.

I think that clears it up.
> Remove the first "about the".

Done locally.
> to SMTP clients.  "SPF client" is mentioned a few times and could

Done locally.

> There is no mention of "exp".  Is its usage permitted/recommended?

No.  Exp is a message from the sending domain back to senders of mail from the 
domain that fails SPF checks.  It's appropriate for receivers to send sensible 
text back with the 550 if they reject due to permerror, but that's not the 
case that exp supports.

> Missing reference to Appendix H.3, but see below.

Fixed.  I do think we need to keep it as has been discussed in reply to your 
later message.
>  Authentication-Results header field and provide the equivalent

msk: I think you provided this text.  Would you please comment?
> Is that /meant/ to be normative?

I think you missed the not.  "... not a comprehensive list of what such 
entities SHOULD do ..."  No.  It's not meant to be normative.
> address even if there's no corresponding MX.

I'm not sure what you're after here, would you please provide text so I can 
better understand?  Also, due to the implicit MX fallback to A, any host with 
an A record can receiver mail, so I don't understand why "evern if there's no 
MX" is relevant to anything.

> See 8. Result Handling above.
Fixed.  See above.

Fixed locally.
>  MTA as an SMTP rejection message.  If such an SMTP rejection message

Made it can.
> Section 8.  IMHO that would improve clarity.

Yes.  I think it needs to be in the appendix as discussed in the other thread.

> Considerations for temperror are missing, anyway.

OK.  Fixed.  I took the permerror considerations and modeled words for 
temperror on them.  I aimed to avoid novelty or controversy.  Let's see if I 
succeeded.  Here's what I added locally:

> H.4.  Policy For SPF Temperror
> 
>    The "temperror" result (see Section 2.6.6) indicates the SPF
>    processing module at the receiver could not retrieve and SPF policy
>    record due to a (probably) transient condition.  This gives no true
>    indication about the authorized use of the data found in the
>    envelope.
>    
>    As with all results, implementers have a choice to make regarding
>    what to do with a message that yields this result.  SMTP allows only
>    a few basic options.
> 
>    Deferring the message is an option, in that it is the one thing a
>    receiver can do to draw attention to the difficulty encountered while
>    protecting itself from messages that do not have a definite SPF
>    result of some kind.  However, if the SPF implementation is defective
>    and returns spurious "temperror" results, only the sender is actively
>    notified of the defect (in the form of mail rejected after it times
>    out of the sending queue), and not the receiver making use of SPF.
>    
>    Because of long queue lifetimes, it is possible that mail will be
>    repeatedly deferred for several days and so any awareness by the
>    sender of a problem could be quite delayed.  If "temperrors" persist
>    for multiple delivery attempts, it might be perferable to treat the
>    error as permanent and reduce the amount of time the message is in
>    transit.
>    
>    The less intrusive handling choice is to deliver the message, perhaps
>    with some kind of annotation of the difficulty encountered and/or
>    logging of a similar nature.  However, this will not be desirable to
>    operators that wish to implement SPF checking as strictly as
>    possible, nor is this sort of passive problem reporting typically
>    effective.
>    
>    There is of course the option placing this choice in the hands of the
>    operator rather than the implementer since this kind of choice is
>    often a matter of local policy rather than a condition with a
>    universal solution, but this adds one more piece of complexity to an
>    already non-trivial environment.
>    
>    Both implementers and operators need to be cautious of all choices
>    and outcomes when handling SPF results.

How's that?

Fixed locally.

> P.S.: what widely deployed extensions to SPF?

As I recall, the two cases where we invoked widely deployed to include things 
in 4408bis (per the charter) are use of Authentication Results in place of 
Received SPF and the "void lookup" processing limit.

Fixed locally.

Thanks for the detailed review.

Scott K

---

From: Scott Kitterman <[email-masked]>
Date: Sat, 20 Apr 2013 18:26:01 -0400

> than one reduce DNS resource usage?  Suggest just removing the second

Generally, the SPF record for HELO requires at most one additional DNS lookup, 
so even if you have a relatively low rejection rate base on these records, you 
can come out ahead on resource expenditure because it's generally an 
inexpensive check compared to Mail From records.

> then.  Since Sender-ID is dead, I suggest removing it.

That's certainly the original reason it got included, but the issue keeps 
coming up.  DMARC was forced to work around this issue very carefully (I think 
they got it right, but it was non-trivial to get there).

There are any number of schemes people have come up with to leverage the 
information in SPF records and I think it's useful to keep the words in there 
about record reuse being a bad idea.  Even though there are no internet 
police, sometimes it actually does help to be able to point to an RFC and say 
"See - it says that's wrong right here".

I'd prefer to keep it.  Let's see what others say.

> spec correctly, this won't help.  I suggest removing it.

It was in RFC 4408 that way, so keeping it is consistent, but I'm OK with 
removing it if we keep the other language in your previous point.

Fixed locally.

Fixed locally.

> advice about highlighting failures in MUAs.

For both your comments on Fail and Softfail, my perspective is the same:

We have been through what goes in the main body and what goes into an appendix 
several times and finally got to something people are roughly happy with.  I 
would propose not having the 4408bis organization discussion again.  If people 
are really unhappy with the organization, I have several comments against it 
as well that I'll submit, but I think that we're better off to leave well 
enough alone.

I found one oddity in the XML.  The output now looks like:

>    As mentioned in Section 9, the Authentication-Results header field is
>    designed to communicate lists of tests a border MTA did and their
>    results.  The specified elements of the field provide less
>    information than the Received-SPF field:
>    
>    Authentication-Results: myhost.example.org; spf=pass
>      smtp.mailfrom=example.net
>    
>    Received-SPF: pass (myhost.example.org: domain of
>     [email-masked] designates 192.0.2.1 as permitted sender)
>        receiver=mybox.example.org; client-ip=192.0.2.1;
>        envelope-from="[email-masked]"; helo=foo.example.com;
>    
>    As an example, an expanded Authentication-Results header field might
>    look like (for a "MAIL FROM" check in this example):
>    
>    Authentication-Results: myhost.example.org; spf=pass
>      reason="client-ip=192.0.2.1; smtp.helo=foo.example.com"
>      smtp.mailfrom=[email-masked]

Is that more like what you were expecting?

Thanks for the review.

Scott K

---

From: "John Levine" <[email-masked]>
Date: 20 Apr 2013 23:32:29 -0000

>inexpensive check compared to Mail From records.

Well, maybe.  If someone says "HELO hotmail.com" as some spamware has
been known to do, you're going to do a lot of extra work before you
look at the Mail From.

>as well that I'll submit, but I think that we're better off to leave well 

If moving this stuff requires a lot of argument, I agree it's not worth it.

>I found one oddity in the XML.  The output now looks like: ...

>Is that more like what you were expecting?

Yup.

R's,
John

---

From: Scott Kitterman <[email-masked]>
Date: Sat, 20 Apr 2013 19:39:52 -0400

On Saturday, April 20, 2013 11:32:29 PM John Levine wrote:
> >>    Checking "HELO" promotes consistency of results and can reduce DNS
> >> 
> >> resource usage.
> >> 
> >Generally, the SPF record for HELO requires at most one additional DNS
> >lookup, so even if you have a relatively low rejection rate base on these
> >records, you can come out ahead on resource expenditure because it's
> >generally an inexpensive check compared to Mail From records.
> 

That's why it says "can reduce", not "does reduce".  In my experience, it's 
generally a savings, but there are certainly cases where it's not.

(snipped the rest since I vote let's not argue).

Scott K

---

From: Alessandro Vesely <[email-masked]>
Date: Sun, 21 Apr 2013 13:51:38 +0200

> No.  April 17 is roughly half way between April 9 and April 24.

Of 2013... ;-)

> I don't understand the question?

Section 6 says:

 These two modifiers MUST NOT appear in a record more than once each.

So it seems it is possible to specify "redirect" in one or more included
records, as well as in redirected record.  Defining the redirect during
a recursive call looks less safe and is not common.  Is it legal?

> MX" is relevant to anything.

Perhaps:

 Domain names can refer to both individual hosts or mail domains.
 Albeit "HELO" identities happen to be individual hosts more frequently
 than "MAIL FROM", either can be used to form an email address, and
 "HELO" can be the only identity available.  A standard SPF record
 for an individual host that is involved in mail sending is:

?

>>    processing module at the receiver could not retrieve and SPF policy
...........................................................^^^
>>    [...]

WFM, albeit lengthy.

---

From: Scott Kitterman <[email-masked]>
Date: Sun, 21 Apr 2013 10:29:05 -0400

>a recursive call looks less safe and is not common.  Is it legal?

Yes. I don't think we need to be explicit about it.  It's rare and I think the existing text is sufficient. 
> for an individual host that is involved in mail sending is:

Generally v=spf1 a -all works. One can also specify the IP address directly with ip4/ip6.  I don't see a need to add text about this.

>>>    processing module at the receiver could not retrieve and SPF

Agreed. If I'd had more time I'd have written something shorter. Given where we are in the process, text highly parallel to the permerror text the group's already reviewed seemed best.

Thanks for reviewing. 

Scott K

---

From: Stuart Gathman <[email-masked]>
Date: Sun, 21 Apr 2013 21:21:42 -0400

On 04/17/2013 12:50 AM, S Moonesamy wrote:
> helps if there are people who read the draft as you did above as I can 
> determine whether the working group reviewed the draft and is ok with it.
Minor nit:

Section 5.1

Mechanisms listed after "all" MUST be ignored.

Sure, section 4.6 says

If there are any syntax errors
    anywhere in the record, check_host() returns immediately with the
    result "permerror", without further interpretation.

But an implementer could misinterpret this as saying the following 
should get Fail rather than PermError:

v=spf1 mx -all foobar

Section 4.6 doesn't make it clear you have to parse everything 
(returning permerror on syntax errors), and only *then* interpret. The 
wording makes it sound like you could parse and interpret one term at a 
time, stopping when you get a match or syntax error.

---

From: Alessandro Vesely <[email-masked]>
Date: Mon, 22 Apr 2013 10:12:36 +0200

On Sun 21/Apr/2013 16:29:05 +0200 Scott Kitterman wrote:
> Generally v=spf1 a -all works. One can also specify the IP address
> directly with ip4/ip6.  I don't see a need to add text about this.

Yeah, no doubt about the record content.  The paragraph quoted from my
previous message was meant to e a possible replacement for:

 The hostname is generally the identity used in the 5321.HELO/.EHLO
 command.  In the case of messages with a null 5321.MailFrom, this is
 used as the domain for 5321.MailFrom SPF checks, in addition to being
 used in 5321.HELO/.EHLO based SPF checks.  The standard SPF record
 for an individual host that is involved in mail processing is:

At least, for uniformity with the rest of the document, replacement of
the terms defined in Sections 1.1.3 and 1.1.4 ought to be done.  The
term "hostname" is used once more in the I-D, and I propose a
replacement it in order to make the text smoother in Section 10.1.2.

Please do what you think is better.  Thank you for editing.

---

From: Scott Kitterman <[email-masked]>
Date: Mon, 22 Apr 2013 10:04:11 -0400

> time, stopping when you get a match or syntax error.

I would tend to favor the MUST be ignored over the returns immediately with a 
permerror (yielding the opposed view of the preferred result from yours).  I 
think this confirms there's an ambiguity in the current draft.

If I take your view that it's better to raise the error (unknown mechanism), 
the perhaps changing the 5.1 language would help.  Adding the sentence before 
in, for more context:

> Mechanisms after "all" will never be tested.  Mechanisms listed after "all"
> MUST be ignored.  

Perhaps if we combine those it helps:

> Mechanisms after "all" MUST not be tested.  Mechanisms listed after "all"
> will be ignored for all purposes except syntax error evaluation. 

Does that help?

Scott K

---

From: Scott Kitterman <[email-masked]>
Date: Mon, 22 Apr 2013 10:26:07 -0400

> previous message was meant to e a possible replacement for:

Sorry about that.  That's what I get  for not looking in the draft for 
context, I didn't realize you were quoting.

> Please do what you think is better.  Thank you for editing.

I think hostname is sufficiently standard that it's OK to use there and 
appropriate because we're generally referring to the name of and individual 
host and not a domain that may have multiple hosts.  

Any other opinions?

Scott K

---

From: Kurt Andersen <[email-masked]>
Date: Mon, 22 Apr 2013 07:40:14 -0700

That leaves the ambiguity of handling -all in embedded (include:) records.
What should be done if the sample foobar mechanism is being referenced in
an included record?

--Kurt

---

From: Scott Kitterman <[email-masked]>
Date: Mon, 22 Apr 2013 10:47:42 -0400

I don't think so.  An included record only gets match/notmatch or some error.  
The qualifier for the 'all' mechanism in the included record doesn't enter into 
it.

Scott K

On Monday, April 22, 2013 07:40:14 AM Kurt Andersen wrote:
> > > > determine whether the working group reviewed the draft and is ok with
> > 
> > it.
> > 
> > > Minor nit:
> > > 
> > > Section 5.1
> > > 
> > permerror (yielding the opposed view of the preferred result from yours).
> > 
> >  I
> > 
> > > Mechanisms after "all" will never be tested.  Mechanisms listed after
> > 
> > "all"
> > 
> > > MUST be ignored.
> >

---

From: "John Levine" <[email-masked]>
Date: 22 Apr 2013 14:54:25 -0000

In article <[email-masked]> you write:
>-=-=-=-=-=-
>-=-=-=-=-=-

At this point, I think we have to say that stuff like that is
undefined, since in reality it does whatever the code does, and it
happens so rarely that nobody cares.

Remember that a standard needs to tell people what to do to interoperate, but
it doesn't have to tell you want to do if someone else screws up.

R's,
John

---

From: Alessandro Vesely <[email-masked]>
Date: Mon, 22 Apr 2013 17:48:33 +0200

>> will be ignored for all purposes except syntax error evaluation. 

Nope, IMHO it's better as is now.  That is:

CURRENT
   If there are any syntax errors
EQUIVALENT-FROM-A-PRAGMATIC-POV
   If any syntax errors are found

See also http://tools.ietf.org/wg/spfbis/trac/ticket/26
and http://www.ietf.org/mail-archive/web/spfbis/current/msg02765.html

---

From: Scott Kitterman <[email-masked]>
Date: Mon, 22 Apr 2013 12:06:41 -0400

On Monday, April 22, 2013 05:48:33 PM Alessandro Vesely wrote:
> > If I take your view that it's better to raise the error (unknown
> > mechanism), the perhaps changing the 5.1 language would help.  Adding the
> > sentence before> 
> > in, for more context:

Right, but how can you find a syntax error in something you MUST ignore?  
That's the conflict that we might want to resolve (and I view resolving a 
conflict within the document as very different than making novel changes to it).

Scott K

---

From: Alessandro Vesely <[email-masked]>
Date: Mon, 22 Apr 2013 18:37:56 +0200

> Right, but how can you find a syntax error in something you MUST ignore?

You have to parse it anyway, as it might be a modifier, e.g.

   "v=spf1 a -all ra=rfc6652"

---

From: Scott Kitterman <[email-masked]>
Date: Mon, 22 Apr 2013 13:18:58 -0400

>    "v=spf1 a -all ra=rfc6652"

That's true, but as soon as I determine it's a mechanism, I ignore it, so the 
ambiguity still exists.

Scott K

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Mon, 22 Apr 2013 11:22:17 -0700

> host and not a domain that may have multiple hosts.

I'd like to see something in OLD vs NEW form before I could say for sure,
but generally I think I agree.

---

From: Alessandro Vesely <[email-masked]>
Date: Tue, 23 Apr 2013 09:59:25 +0200

On Mon 22/Apr/2013 19:18:58 +0200 Scott Kitterman wrote:
>>>>>> "all" MUST be ignored.
>>>>>> Mechanisms after "all" MUST not be tested.  Mechanisms listed after
>>>>>> "all" will be ignored for all purposes except syntax error evaluation.
>>>>> 
>>>>> Does that help?
>>>> 

If you determine it's a valid something, there's no syntax error.

---

From: Scott Kitterman <[email-masked]>
Date: Tue, 23 Apr 2013 05:55:12 -0400

On Tuesday, April 23, 2013 09:59:25 AM Alessandro Vesely wrote:
> > That's true, but as soon as I determine it's a mechanism, I ignore it, so
> > the ambiguity still exists.
> 

Anyone else?

I still think Stuart's point is valid, but I'm not sure the best way to fix it.  
I also think it would only matter in rare cases, but not so rare we can just 
say "Meh, corner case." and move on.

Scott K

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Tue, 23 Apr 2013 11:03:37 -0700

> SM will be the shepherd for this document.

Most of this is nits; except as noted, you could ignore it all and I'd be
happy.  Nevertheless:

Section 1:

The last paragraph makes the claim that domain reputation is likely to be
more accurate than IP reputation.  Is it worthwhile to add a sentence or
two explaining why this is true?

Section 1.1.2:

Suggest replacing the first sentence with "ABNF is defined in [RFC5234], as
are the tokens ALPHA, ..."  As it is now, "ABNF" is not expanded on first
use.

Section 1.1.3:

Replace the last sentence with: "Note that other terms that might
superficially look like the common terms, such as 'reverse-path', are used
only as they are specified in their defining documents."

Section 2.1:

I think someone else already mentioned this, but how does encouraging a
second check decrease DNS resource use?

Also, suggest replacing the first sentence of the second paragraph with
"Note that requirements for the domain presented in the EHLO or HELO
command are not always clear to the sending party, and SPF verifiers MUST
be prepared for is identity to be an IP address literal, or simply be
malformed."   We might even reference the IP address literal ABNF in
RFC5321 here.

Section 2.2:

Add to the first paragraph "...or no HELO check was done." or something of
that ilk.

Section 2.3:

The second paragraph is loaded with negatives "no... neither... nor."
Suggest: "ADMDs can publish SPF records that authorize no hosts to use
their DNS domain names in HELO or MAIL FROM commands during SMTP sessions."

Section 2.4:

Isn't the fourth paragraph implicit in any standards specification?  That
is, do we really need it?

Section 2.5:

I can't remember if I suggested it previously, but an appendix of RFC5451
gives a pretty thorough treatment of the notion of doing mail
authentication checks during the SMTP session versus later.  It might be
helpful as an informative reference here.

Section 2.6.7:

Does "permerror" also result when permanent DNS errors occur?

Section 3.1:

s/alternate/alternative/

Section 4.1:

The list of check_host() inputs have been repeated here as in Section 2.4.
Do we need both lists?

Section 4.4:

s/suggests on an algorithm/suggests an algorithm/

Section 4.6.4:

I imagine you're going to say that 10 is the limit imposed by most
implementations, but shouldn't we say that there should be a finite,
perhaps configurable limit, and operational experience has shown that 10 is
a reasonable default?

Why 20 seconds for the overall run time?

Section 4.8:

In the third paragraph, a couple of sentences start with "Domain" when you
actually mean it as a literal parameter name.  I suggest using "domain"
(including the quotes) instead.

The "Some implementations ..." sentence seems to be malformed.  I can't
parse it.

Section 5.2:

s/Check_host/check_host/ (it's not a word that starts sentences and thus
needs capitalization)

Missing end-quote at the end of bullet 3.

The second-last paragraph is weirdly wrapped.  Is the XML ok there?

Section 5.4:

Should the limits stuff in here be removed, and simply reference Section
4.6.4 instead?

Section 5.5:

The SHOULD NOT in here is puzzling.  Does it interfere with
interoperability to use it?  It was my impression that it's possible to get
the same functionality some other way, but not that using "ptr" actually
was a bad idea for interoperability reasons.

Again with the limits stuff, redundant to Section 4.6.4.

Section 5.6:

For IPv4, shouldn't the CIDR be 1*2DIGIT?  For IPv6, shouldn't the CIDR be
1*3DIGIT?

Section 6.2:

It just occurred to me that in the earlier sections (2 or 4, in
particular), it's clear that check_host() returns one of the enumerated
result, but it's never made explicit that it also outputs an explanation
string.  We might want to mention this earlier than we do now so it's not a
surprise later.

The SHOULD in the fifth paragraph is questionable: How does it describe an
aspect of interoperability?

Last paragraph, s/exp=/exp/ 2x.

Section 7.1:

ABNF rules say literal strings in productions are case-insensitive.  That
means the macro-letter set actually implicitly includes all of the capital
letter equivalents.  If we want to constrain it to lowercase only, we need
to use hex notation.

Section 7.3:

"This macro SHOULD NOT be used...." should be in its own paragraph.  Also
the second sentence should be "See Section xxx for discussion."

At the end of this section, there are three paragraphs that all start
"Note:".  I don't think those tags add anything and suggest they be removed.

Section 8:

Second paragraph, s/, and where/ and, where/

Section 8.7:

Don't the last two sentences say the same thing?

Section 9.1:

Same point about the case-insensitivity of ABNF here.

The "SPF verifiers MUST make sure..." doesn't seem actionable to me.  Do we
really expect Received-SPF generators to be able to identify and exclude
malicious data?

Section 10:

s/document/protocol/ (or specification) x3

Section 10.1.1:

s/required/necessary/

Section 10.1.2:

Add "at the cost of a DNS query per receiver" to the end of the first
sentence.

Section 10.1.3:

s/to hostname/to the hostname/, s/of domain/of the domain/

Section 10.2:

Missing close parenthesis in the first paragraph.

Section 10.3:

Errant period in the first sentence.

s/Address/address/

Section 11.1:

Aren't the first and third bullets describing the same attack?

Missing parenthesis at the end of the fourth bullet.

Section 11.3:

Clobbering the DNS at a receiver could also result in delivery of malicious
mail if the receiver is configured to pass on "tempfail" and just add a
header field.

Is that second bullet true?  It appears to claim that IP address spoofing
is effectively impossible.

Section 11.5:

This appears to be an incomplete thought.  At a minimum, it doesn't flow
nicely into the next subsection.  Maybe add a sentence to indicate that
SMTP itself doesn't do anything to validate those parameters?

Section 11.5.2:

Remove commas around the section reference.

Appendix A:

Any changes made to ABNF above need to be reflected down here.

Appendix B.3:

Suggest a reference to RFC5782.

Appendix C:

s/implmentation/implementation/

Appendix G:

Fourth paragraph: As I mentioned above, RFC5451 has an appendix that talks
at length about why you want to do things like SPF checks at the border
rather than inside.

Appendix I:

The second paragraph claims we're documenting extensions since 4408.  Were
there any?

-MSK

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Tue, 23 Apr 2013 11:21:15 -0700

"v=spf1 -all foobar" is syntactically invalid.  I agree with Stuart that it
should result in "permerror" and not "fail".  Thus, I think 4.6 has it
right as-is.

Adding text to say anything after "all" MUST be ignored is a good idea as
well, though they have to be syntactically valid anyway.  Mentally to me,
this is the same as C and C++ in how they define the short-circuit logic of
"if" statements.  For example, in "if (foo || bar)", if "foo" is true, C
and C++ specify that "bar" will not be evaluated.  But if the expression at
"bar" is actually a syntax error, the whole thing won't compile even if
everything up to there is fine.

-MSK

On Tue, Apr 23, 2013 at 2:55 AM, Scott Kitterman <[email-masked]> wrote:

> > >>>> and
> http://www.ietf.org/mail-archive/web/spfbis/current/msg02765.html
> > >>>

---

From: Hector Santos <[email-masked]>
Date: Tue, 23 Apr 2013 17:25:05 -0400

I believe that would be a FAIL on our left to right natural string 
parser, -ALL would terminate the processing, never see foobar.

I think its correct logic to expect of all.  Don't assume processing 
would be a two-pass parser:

   Pass One: Check Validity of entire line
   Pass Two: Process each string word as its extracted from the line.

I would think the easiest, fastest is just a one pass parser.

Now if the junk is before the EOL (end of line), that is a different 
situation - permerror.

I don't think it would be correct to assume all will read this as a 
permerror when there is a terminator before foobar/junk.

---

From: "John Levine" <[email-masked]>
Date: 24 Apr 2013 00:26:29 -0000

>perhaps configurable limit, and operational experience has shown that 10 is

If everyone doesn't have the same limit, an SPF check might fail at a
site with a lower limit and the identical check would succeed at one
with a higher limit.  The limit has been 10 for a long time and I
don't see any reason to change it now.

>The "Some implementations ..." sentence seems to be malformed.  I can't

That whole paragraph could be replaced by "the result of evaluating
check_host() with a syntactically invalid domain is undefined."

>For IPv4, shouldn't the CIDR be 1*2DIGIT?  For IPv6, shouldn't the CIDR be

What's wrong with 10.1.0.0/016 ?

>is effectively impossible.

Ever since TCP stacks started randomizing sequence numbers in the
mid-1990s, spoofing a TCP session has been implausibly hard.  Spoofing
UDP is all too easy, but that's not what we're talking about here.

R's,
John

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Tue, 23 Apr 2013 21:10:15 -0700

> don't see any reason to change it now.

I'd rather we say something like this.  Otherwise some neophyte will read
this and think "What's so special about 10?"

-MSK

---

From: Scott Kitterman <[email-masked]>
Date: Wed, 24 Apr 2013 00:33:51 -0400

On Tuesday, April 23, 2013 09:10:15 PM Murray S. Kucherawy wrote:
> > >perhaps configurable limit, and operational experience has shown that 10
> > 
> > is
> > 
> > >a reasonable default?
> > 

OK.  Would you please propose text?

Scott K

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Tue, 23 Apr 2013 22:21:21 -0700

Sure, add to the end of 4.6.4:

The number 10 used as the limit above was chosen during early
implementations and has shown to produce reasonable results both in terms
of giving complete answers when used properly and preventing abuse
otherwise.  The choice was largely arbitrary, but it is now used as the
mandatory limit so that SPF results are consistent across implementations.

(I'm guessing that history is correct; feel free to make corrections as
needed.)

---

From: Stuart Gathman <[email-masked]>
Date: Wed, 24 Apr 2013 12:28:31 -0400

Long ago, Nostradamus foresaw that on 04/22/2013 10:54 AM, John Levine
> it doesn't have to tell you want to do if someone else screws up.
It wasn't undefined for rfc4408, and is in the test suite.  It isn't
undefined for rfc4408bis either - I'm just trying to prevent a few
boneheaded implementations.

---

From: Stuart Gathman <[email-masked]>
Date: Wed, 24 Apr 2013 13:28:11 -0400

Long ago, Nostradamus foresaw that on 04/23/2013 05:25 PM, Hector Santos
> I would think the easiest, fastest is just a one pass parser.
You always have to parse the whole line, regardless, because you have to
find any modifiers after "all".  You can use a one-pass parser, but you
will have to just note a match, and keep parsing until the end.

Furthermore, as discussed for rfc4408, a guiding principle of SPF is for
results to be deterministic as much as possible.  Permerror or not
depending on the connection is not good.

---

From: Scott Kitterman <[email-masked]>
Date: Wed, 24 Apr 2013 13:32:55 -0400

> depending on the connection is not good.

Definitely.

Scott K

---

From: S Moonesamy <[email-masked]>
Date: Wed, 24 Apr 2013 14:02:32 -0700

Hello,

As a reminder, the WGLC for draft-ietf-spfbis-4408bis-14 will close 
at 2014-04-24 00:00 UTC [1].

Regards,

1. http://www.ietf.org/mail-archive/web/spfbis/current/msg03347.html

---

From: Stuart Gathman <[email-masked]>
Date: Wed, 24 Apr 2013 17:47:06 -0400

On 04/24/2013 05:02 PM, S Moonesamy wrote:
> As a reminder, the WGLC for draft-ietf-spfbis-4408bis-14 will close at 
> 2014-04-24 00:00 UTC [1].
   I'm not sure we need another whole year!

---

From: S Moonesamy <[email-masked]>
Date: Wed, 24 Apr 2013 14:52:25 -0700

Hi Stuart,
>   I'm not sure we need another whole year!

That should be April 24, 2013 00:00 UTC.  Another whole year is too 
much for me. :-)

Regards,
S. Moonesamy