Subject: [spfbis] I-D Action: draft-ietf-spfbis-4408bis-10.txt

From: [email-masked]
Date: Thu, 07 Feb 2013 00:11:51 -0800

A New Internet-Draft is available from the on-line Internet-Drafts directories.
 This draft is a work item of the SPF Update Working Group of the IETF.

	Title           : Sender Policy Framework (SPF) for Authorizing Use of Domains in Email, Version 1
	Author(s)       : Scott Kitterman
	Filename        : draft-ietf-spfbis-4408bis-10.txt
	Pages           : 73
	Date            : 2013-02-07

Abstract:
   Email on the Internet can be forged in a number of ways.  In
   particular, existing protocols place no restriction on what a sending
   host can use as the "MAIL FROM" of a message or the domain given on
   the SMTP HELO/EHLO commands.  This document describes version 1 of
   the Sender Policy Framework (SPF) protocol, whereby an ADMD can
   explicitly authorize the hosts that are allowed to use its domain
   names, and a receiving host can check such authorization.

   This document obsoletes RFC4408.

The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-spfbis-4408bis

There's also a htmlized version available at:
http://tools.ietf.org/html/draft-ietf-spfbis-4408bis-10

A diff from the previous version is available at:
http://www.ietf.org/rfcdiff?url2=draft-ietf-spfbis-4408bis-10

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

---

From: Scott Kitterman <[email-masked]>
Date: Thu, 07 Feb 2013 03:18:06 -0500

On Thursday, February 07, 2013 12:11:51 AM [email-masked] wrote:
> A New Internet-Draft is available from the on-line Internet-Drafts
> directories. This draft is a work item of the SPF Update Working Group of
> the IETF.
> 
>         Title           : Sender Policy Framework (SPF) for Authorizing Use
> of Domains in Email, Version 1 Author(s)       : Scott Kitterman

I believe this update addresses these open issues in the tracker:

#36 	RFC 2119 key words
#38 	Deprecated
#43 	New requirement for mx: or ptr records
#45 	Section 5.5 steps
#46 	DNS reply in Section 5.7
#48 	Section 8.1 - macro defs
#52 	Section 8.1 *Macro Definitions
#33 	Marked Failed Mail Exposure

http://trac.tools.ietf.org/wg/spfbis/trac/report/1

Please review.  I'll give people a bit of time to digest the revised draft and 
raise concerns if they have them before I mark the issues as fixed.  Thanks to 
those who helped with the updates.

Scott K

---

From: S Moonesamy <[email-masked]>
Date: Thu, 07 Feb 2013 00:31:24 -0800

>Please review.  I'll give people a bit of time to digest the revised draft and

Please note that Issue #29 [1] was closed a few days ago.

Regards,
S. Moonesamy

1. http://trac.tools.ietf.org/wg/spfbis/trac/ticket/29

---

From: Scott Kitterman <[email-masked]>
Date: Thu, 07 Feb 2013 03:38:01 -0500

> 1. http://trac.tools.ietf.org/wg/spfbis/trac/ticket/29

Yes, because the text about which there was a specific complaint that caused it 
to be reopened no longer exists.  I went through and closed a number of issues 
earlier in the week since their status in the tracker was not correct.  Those 
actions are unrelated to this new revision.

Scott K

---

From: "Hector Santos" <[email-masked]>
Date: Thu, 7 Feb 2013 21:48:53 -0500

Section 2.3, to the last paragraph, it added the 30 days sentence:

   When changing SPF records, care has to be taken to ensure that there
   is a transition period so that the old policy remains valid until all
   legitimate email can reasonably expect to have been checked.  This
   can be as much as 30 days.

Where does this 30 days come from? RFC4408 does not have this.

Does this include MUA user delays (i.e. returning from vacation)?

I ask because I find it unreasonable that anyone will wait 30 days to deliver mail, thats a long time, and if so, how would one go about designing or adding additional delivery strategies to minimize potential policy conflicts with different receivers?

Is there a 30 day caching of the last SPF record before comparing and applying any new record available?  IOW, for example, a lookup is done, a lexical compare is done against the cached record if any. If different, then we need to keep both around but only apply the old one for 30 days?

Anyway, I see no justification for this 30 day that seems to include MUA delay considerations where it may be doing SPF verifications.  Seems to bit much to expect, MUAs expected to cache dual records [1], etc.

The higher the days, the more it seems it would be a problem since the range of days covers a longer time period regarding factoring delivery strategies that will minimize potential policy conflicts with different receivers.  A lot can happen within 30 days, like sending tons of email, multiple changes in fine tunning SPF reocrds, etc. 

I would think just talking/reference about SMTP suggested guidelines of 4-5 days for delivery retry delays is appropriate or take out the sentence (keep like RFC4408 here).

--
HLS

1] SMTP has a recommended 4-5 days retry suggestion. I believe SPF is about SMTP to SMTP transports so the SPF-Receiver host will do the checking and that will be within 4-5 days of getting the mail. At best, with marking operations enabled, the Received-SPF (or Auth-Res now) header will be added for the MUA to potentially see. This is deployment assumption that the the receiver has a Process-But-Do-Nothing local policy concept, i.e. its only MARKING SPF result, its taking no action at the backend and puts all the mail in the user's pickup bin  for POP3.  This means the MUA will need to have a SMTP Received Time Stamp trace line (already there) but also recording of the old SPF record stored in the RFC5322 header. Otherwise we expect the MUA to do this double recording!  This is a time-shifted problem, that was also available with DKIM when the deployment was not yet done at receivers. A proposed solution was offered for DKIM. 

Partial DKIM Verifier Support using a DKIM-Received Trace Headerhttp://tools.ietf.org/html/draft-santos-dkim-rcvd-00
A similar one would be needed for SPF with such a long delay expected with MUA doing the verification - not the user's ISP (backend) SMTP receiver.

----- Original Message ----- 
From: "Scott Kitterman" <[email-masked]>
To: <[email-masked]>
Sent: Thursday, February 07, 2013 3:18 AM
Subject: Re: [spfbis] I-D Action: draft-ietf-spfbis-4408bis-10.txt

> _______________________________________________
> spfbis mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/spfbis
>

---

From: Scott Kitterman <[email-masked]>
Date: Thu, 07 Feb 2013 23:06:53 -0500

On Thursday, February 07, 2013 09:48:53 PM Hector Santos wrote:
> I ask because I find it unreasonable that anyone will wait 30 days to
> deliver mail, thats a long time, and if so, how would one go about
> designing or adding additional delivery strategies to minimize potential
> policy conflicts with different receivers?
> 
> Is there a 30 day caching of the last SPF record before comparing and
> applying any new record available?  IOW, for example, a lookup is done, a
> lexical compare is done against the cached record if any. If different,
> then we need to keep both around but only apply the old one for 30 days?
> 
> Anyway, I see no justification for this 30 day that seems to include MUA
> delay considerations where it may be doing SPF verifications.  Seems to bit
> much to expect, MUAs expected to cache dual records [1], etc.
> 
> The higher the days, the more it seems it would be a problem since the range
> of days covers a longer time period regarding factoring delivery strategies
> that will minimize potential policy conflicts with different receivers.  A
> lot can happen within 30 days, like sending tons of email, multiple changes
> in fine tunning SPF reocrds, etc.
> 
> I would think just talking/reference about SMTP suggested guidelines of 4-5
> days for delivery retry delays is appropriate or take out the sentence
> (keep like RFC4408 here).
> 
> --
> HLS
> 
> 1] SMTP has a recommended 4-5 days retry suggestion. I believe SPF is about
> SMTP to SMTP transports so the SPF-Receiver host will do the checking and
> that will be within 4-5 days of getting the mail. At best, with marking
> operations enabled, the Received-SPF (or Auth-Res now) header will be added
> for the MUA to potentially see. This is deployment assumption that the the
> receiver has a Process-But-Do-Nothing local policy concept, i.e. its only
> MARKING SPF result, its taking no action at the backend and puts all the
> mail in the user's pickup bin  for POP3.  This means the MUA will need to
> have a SMTP Received Time Stamp trace line (already there) but also
> recording of the old SPF record stored in the RFC5322 header. Otherwise we
> expect the MUA to do this double recording!  This is a time-shifted
> problem, that was also available with DKIM when the deployment was not yet
> done at receivers. A proposed solution was offered for DKIM.
> 
> Partial DKIM Verifier Support using a DKIM-Received Trace
> Headerhttp://tools.ietf.org/html/draft-santos-dkim-rcvd-00 A similar one
> would be needed for SPF with such a long delay expected with MUA doing the
> verification - not the user's ISP (backend) SMTP receiver.

RFC 4408 says:

>    When changing SPF records, care must be taken to ensure that there is
>    a transition period so that the old policy remains valid until all
>    legitimate E-Mail has been checked.

There was feedback suggesting this was too generic and we ought to make it 
more specific/actionable.  30 days was pulled from the original Microsoft 
Caller-ID for Email specification.  I couldn't find any other recommendation.

Whether you like it or not, people do do post-SMTP SPF validation, including 
in MUAs.  A recommendation that just caters to fully connected SMTP operations 
will be too narrow.

I'm not wedded to 30 days and am open to suggestions, but I do think it's 
better to make a more specific recommendation than RFC 4408 did and I don't 
think 4-5 days is long enough.

Scott K

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Thu, 7 Feb 2013 20:35:56 -0800

If you were to characterize the 30 days as a "consensus estimate" or some
such regarding a reasonable maximum time one might expect mail to take to
be either delivered or returned as undeliverable, it would appear less
arbitrary.

That assumes, of course, that such is the consensus.  I for one would be
fine with saying so; my own experience is that the average is substantially
shorter, but I have configured systems (in the distant past, admittedly)
that had a 28-day timeout.

-MSK

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Thu, 7 Feb 2013 20:42:25 -0800

I should add that I don't support the notion of recording in the header the
SPF record or DKIM key data that were discovered by the receiving MTA.
That stuff is supposed to be evaluate at receipt time, not later by the MUA
when it gets around to doing so.  There was a lengthy appendix added to
RFC5451 that talked about all the reasons you don't want to do it that way.

The point of the "30 days" statement is to remind operators that mail can
be delayed in transit.   Suppose, for example, you have an SPF record that
authorizes some third party to send mail on your behalf, but that contract
is ending so you plan to remove the authorization.  If at time T0 that
third party sends something you want to get out, then at T1 you revoke that
authorization, and then at T2 the actual delivery attempt occurs, the mail
will get rejected.  The point this is trying to make is that the time
between T0 and T2 could be longer than the operator might realize.

-MSK

On Thu, Feb 7, 2013 at 8:35 PM, Murray S. Kucherawy <[email-masked]>wrote:

> On Thu, Feb 7, 2013 at 8:06 PM, Scott Kitterman <[email-masked]>wrote:
>

---

From: S Moonesamy <[email-masked]>
Date: Thu, 07 Feb 2013 21:22:02 -0800

Hi Murray,
At 20:35 07-02-2013, Murray S. Kucherawy wrote:
>If you were to characterize the 30 days as a "consensus estimate" or 
>some such regarding a reasonable maximum time one might expect mail 
>to take to be either delivered or returned as undeliverable, it 
>would appear less arbitrary.

See 
http://www.ietf.org/mail-archive/web/spfbis/current/msg02559.html 
and http://www.ietf.org/mail-archive/web/spfbis/current/msg01842.html 
for the "30 days" text.

Regards,
S. Moonesamy

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Thu, 7 Feb 2013 21:55:59 -0800

Thanks for the reminder.  Scott, we should make sure we didn't drop any of
the things in the second thread in the long period since it petered out.

However, the fact that 30 days is what the older Caller ID drafts said
doesn't mean it's not an arbitrary number.  I think something more general
(see upthread) is probably more appropriate.

-MSK

On Thu, Feb 7, 2013 at 9:22 PM, S Moonesamy <[email-masked]> wrote:

> See http://www.ietf.org/mail-**archive/web/spfbis/current/**msg02559.html<http://www.ietf.org/mail-archive/web/spfbis/current/msg02559.html>and
> http://www.ietf.org/mail-**archive/web/spfbis/current/**msg01842.html<http://www.ietf.org/mail-archive/web/spfbis/current/msg01842.html>for the "30 days" text.
>
> Regards,
> S. Moonesamy
>

---

From: Scott Kitterman <[email-masked]>
Date: Fri, 08 Feb 2013 00:59:01 -0500

I don't have any objection to going back to the general advice in 4408.  I'm 
not convinced we can do better.

Scott K

On Thursday, February 07, 2013 09:55:59 PM Murray S. Kucherawy wrote:
> > http://www.ietf.org/mail-**archive/web/spfbis/current/**msg02559.html<htt
> > p://www.ietf.org/mail-archive/web/spfbis/current/msg02559.html>and
> > http://www.ietf.org/mail-**archive/web/spfbis/current/**msg01842.html<htt
> > p://www.ietf.org/mail-archive/web/spfbis/current/msg01842.html>for the "30
> > days" text.
> > 
> > Regards,
> > S. Moonesamy

---

From: "John Levine" <[email-masked]>
Date: 8 Feb 2013 06:12:09 -0000

>>would appear less arbitrary.

The reality is that we're pulling it out of our rears.

RFC 5321 says that SMTP deliveries can take five days, and my
understanding has always been that you should be prepared for mail to
trickle in for up to a week.  But do we have any idea about the actual
practices of people who do off line SPF verification?  Do they wait a
week? A month? A year?

In DKIM (yes, I know this isn't DKIM) we made a concrete decision that
DKIM isn't for archival verification of messages, it's for checking
messages in transit, and in that way SPF is similar.  I would also
remember that SPF records depend on MX, A, and other records that have
their own update schedules.

My preference would be to point to 5321 for how long a message might
be in transit, and to note that while it's not forbidden to do offline
checks, the closer to the time of receipt you do the checks, the more
likely you are to get the answer that the domain owner intended.

---

From: "Hector Santos" <[email-masked]>
Date: Fri, 8 Feb 2013 07:49:29 -0500

+1

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Fri, 8 Feb 2013 07:40:23 -0800

Works for me.

---

From: "Hector Santos" <[email-masked]>
Date: Fri, 8 Feb 2013 11:21:54 -0500

I think if anything needs to be stated, is possibly an example sentence illustrating a type of SPF policy change which could affect existing mail in the delivery/outbound queue(s).

Offhand, its only an issue when the IP::DOMAIN LMAP/SPF association changes - not an issue for minor corrections and policy updates where the IP::DOMAIN association does not change.  Like I could decide after BIS is published to change all our current PTR SPF usage to specific IP4 rules. A big change for our DNS records maintenance procedure (someone needs to be aware IP/Machine changes requires updating SPF records), but overall, the IP::DOMAIN will be effectively the same so bo problem for emails in the loop.  This can help ease readers who now see the concerns of PTR usage and feel ok the change will not hurt the mail.

   The general SMTP guideline 4-5 days as part of a typical mail delivery retry strategies

--
HLS

----- Original Message ----- 
From: "Murray S. Kucherawy" <[email-masked]>
To: "John Levine" <[email-masked]>
Cc: <[email-masked]>; "SM" <[email-masked]>
Sent: Friday, February 08, 2013 10:40 AM
Subject: Re: [spfbis] Section 2.3 - Genesis of 30 days delays?

>> likely you are to get the answer that the domain owner intended.

--------------------------------------------------------------------------------

---

From: Scott Kitterman <[email-masked]>
Date: Fri, 08 Feb 2013 11:53:09 -0500

On Thursday, February 07, 2013 03:18:06 AM Scott Kitterman wrote:
> Please review.  I'll give people a bit of time to digest the revised draft
> and raise concerns if they have them before I mark the issues as fixed. 
> Thanks to those who helped with the updates.

I only got feedback on #36, so I'll leave that open and close the rest of the 
listed issues.  I'll address #36 and the results of the Section 2.3 discussion 
in -11.  I'll give specific feedback on the issue #36 comments once I've had 
time to digest them.

Thanks,

Scott K

---

From: Scott Kitterman <[email-masked]>
Date: Sun, 10 Feb 2013 00:35:05 -0500

> likely you are to get the answer that the domain owner intended.

OK.  Here's what I've committed locally for -11:

         When changing SPF records, care has to be taken to ensure that there
         is a transition period so that the old policy remains valid until
         all legitimate email can reasonably expect to have been checked.
         <xref target="RFC5321"/> discusses how long a message might be in
         transit.  While offline checks are possible, the closer to the
         original transmission time checks are performed, the more likely they
         are to get an SPF result that matches the sending ADMD intent at the
         time the message was sent.

Scott K

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sat, 9 Feb 2013 22:29:47 -0800

WFM, especially if the reference could include a Section.

-MSK

---

From: Scott Kitterman <[email-masked]>
Date: Sun, 10 Feb 2013 01:37:45 -0500

I'm just heading to bed now, so if someone wants to look it up, I'll be glad 
to include it.

Scott K

On Saturday, February 09, 2013 10:29:47 PM Murray S. Kucherawy wrote:
> >          When changing SPF records, care has to be taken to ensure that
> > 
> > there
> > 
> >          original transmission time checks are performed, the more likely
> > 
> > they
> > 
> >          are to get an SPF result that matches the sending ADMD intent at
> > 
> > the
> >

---

From: "John Levine" <[email-masked]>
Date: 10 Feb 2013 06:49:39 -0000

>I'm just heading to bed now, so if someone wants to look it up, I'll be glad 

RFC 5321 sec 4.5.4.1

---

From: Scott Kitterman <[email-masked]>
Date: Sun, 10 Feb 2013 22:04:12 -0500

On Sunday, February 10, 2013 06:49:39 AM John Levine wrote:
> sec 4.5.4.1

Thanks.  Applied locally for -11.

Scott K