Subject: [spfbis] Suggestion for an additional deliverable

From: "Murray S. Kucherawy" <[email-masked]>
Date: Fri, 23 Dec 2011 12:47:30 -0800

I believe this falls within the current scope, but it would involve either an additional draft deliverable or an additional section in the spfbis document.

There are two errata against RFC5451: http://www.rfc-editor.org/errata_search.php?rfc=5451

The first of these is a change to what Authentication-Results uses to report SPF failures.  In particular, SPF itself uses "fail", but A-R uses "hardfail" to report that case.  Since RFC5451 set up IANA registries for results of authentication methods, fixing this amounts to just an IANA action and maybe an "Updates" against RFC5451.

I believe we could do this in spfbis easily as it amounts to nothing more than a correction, and those are in scope.  Although this modifies a document outside of our current plan, it is a germane modification to this work.

If there's objection, I can seek to do it as an AD-sponsored individual submission or maybe through APPSAWG.

Comments?

-MSK

---

From: Dave CROCKER <[email-masked]>
Date: Fri, 23 Dec 2011 13:49:59 -0800

> If there’s objection, I can seek to do it as an AD-sponsored individual
> submission or maybe through APPSAWG.

I don't understand.  Unless I missed it, the draft charter does not cite RFC 
5451, reporting or header fields.  How does modifying a reporting specification 
fall within the current draft charter?

d/

-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

---

From: SM <[email-masked]>
Date: Fri, 23 Dec 2011 13:41:31 -0800

Hi Murray,
At 12:47 23-12-2011, Murray S. Kucherawy wrote:
>I believe we could do this in spfbis easily as it amounts to nothing 
>more than a correction, and those are in scope.  Although this 
>modifies a document outside of our current plan, it is a germane 
>modification to this work.

I agree that it would be easy work.

>If there's objection, I can seek to do it as an AD-sponsored 
>individual submission or maybe through APPSAWG.

I suggest that you defer this until after the core work is 
completed.  There will always be a good reason for adding 
deliverables.  It's better not to go down that path.

Regards,
-sm

---

From: Scott Kitterman <[email-masked]>
Date: Fri, 23 Dec 2011 18:25:54 -0500

>deliverables.  It's better not to go down that path.

One possible reason to deal with this as core work would be if we wanted to explore deprecating the Received SPF header field in favor of the Authentication Results header field.

SPF Received is in wide use, so it's not a candidate for removal, but we might start in that direction if we could make a few adjustments to Authentication Results. It would be difficult to do this immediately after we finish 4408bis.

Scott K

---

From: Commerco WebMaster <[email-masked]>
Date: Fri, 23 Dec 2011 16:44:03 -0700

The errata items for RFC 5451 look correct. The comment from D. Stussy 
expresses the correct current expected SPF response per RFC 4408.

Murray, since RFC 5451 looks like it is authored by you, would you 
entertain adjusting 2.4.2 (the SPF and Sender-ID results) to reflect the 
SPF responses per RFC 4408?  Hardfail is not mentioned in RFC 4408 
(http://www.rfc-editor.org/rfc/rfc4408.txt), so I'm not sure how it got 
into RFC 5451 (unless it is part of the Sender-ID fork, which I think is 
outside the scope of the charter for SPFbis).

Best,

Alan M.

On 12/23/2011 1:47 PM, Murray S. Kucherawy wrote:
> I believe this falls within the current scope, but it would involve
> either an additional draft deliverable or an additional section in the
> spfbis document.
>
> There are two errata against RFC5451:
> http://www.rfc-editor.org/errata_search.php?rfc=5451
>
> The first of these is a change to what Authentication-Results uses to
> report SPF failures. In particular, SPF itself uses “fail”, but A-R uses
> “hardfail” to report that case. Since RFC5451 set up IANA registries for
> results of authentication methods, fixing this amounts to just an IANA
> action and maybe an “Updates” against RFC5451.
>
> more than a correction, and those are in scope. Although this modifies a
> document outside of our current plan, it is a germane modification to
> this work.
>
> _______________________________________________
> spfbis mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/spfbis

---

From: Hector Santos <[email-masked]>
Date: Fri, 23 Dec 2011 22:40:07 -0500

I can support adding AUTH-RES support to SPF-BIS charter. This is more 
in line with SPF-BIS work. Received-SPF is a SHOULD, thus is already 
optional for SPF implementations and it would not be a protocol change 
to offer an alternative 5322 reporting option for AUTH-RES or both. 
Since these headers are generally are for internal mail bots, there is 
already the possibility that other headers are using to pass more 
useful information for the internal mail processors.

However, for domain SPF policies using -ALL (hard fail), when the 
result is a FAIL, the PAYLOAD will not be downloaded and there is NO 
5322 header added, no Received-SPF.  So by the time mail is 
downloaded, there will only be for results:

     result = = "Pass" / SoftFail" / "Neutral" /
                "None" / "TempError" / "PermError"

This is critical important and one of the concerns I have with the 
mindset change with SPF-BIS now focusing on RFC5322 when it was never 
a major part of SPF other than for adding a Receiver-SPF header.

What that means is, while its possible for an local policy to not 
reject the SPF failures at SMTP, and include a ReceivedSPF: Fail 
header, this defeats the operational importance of SPF a hard fail 
policy offers to avoid  Accept/Bounce attacks.

So I can support the support of AUTH-RES, but not with a focus of 
reporting to force payload downloads for the purpose of reporting 
AUTH-RES fail reports.

-- 
Hector Santos, CTO
http://www.santronics.com
http://santronics.blogspot.com

Murray S. Kucherawy wrote:
> ------------------------------------------------------------------------
>

---

From: Dotzero <[email-masked]>
Date: Fri, 23 Dec 2011 23:24:40 -0500

On Fri, Dec 23, 2011 at 10:40 PM, Hector Santos <[email-masked]> wrote:
> I can support adding AUTH-RES support to SPF-BIS charter. This is more in
> line with SPF-BIS work. Received-SPF is a SHOULD, thus is already optional
> for SPF implementations and it would not be a protocol change to offer an
> alternative 5322 reporting option for AUTH-RES or both. Since these headers
> are generally are for internal mail bots, there is already the possibility
> that other headers are using to pass more useful information for the
> internal mail processors.
>
> However, for domain SPF policies using -ALL (hard fail), when the result is
> a FAIL, the PAYLOAD will not be downloaded and there is NO 5322 header
> added, no Received-SPF.  So by the time mail is downloaded, there will only
> be for results:
>

Is this true? RFC4408 states:

2.5.4.  Fail

   A "Fail" result is an explicit statement that the client is not
   authorized to use the domain in the given identity.  The checking
   software can choose to mark the mail based on this or to reject the
   mail outright.

So rejecting is one option but marking is an option as well. This
would mean that "Fail" could conceivably be a reported option.

> This is critical important and one of the concerns I have with the mindset
> change with SPF-BIS now focusing on RFC5322 when it was never a major part
> of SPF other than for adding a Receiver-SPF header.
>

This is not necessarily a function of RFC5322.

> What that means is, while its possible for an local policy to not reject the
> SPF failures at SMTP, and include a ReceivedSPF: Fail header, this defeats
> the operational importance of SPF a hard fail policy offers to avoid
>  Accept/Bounce attacks.
>

I have to disagree with you on this Hector. My experience with private
policy assertions based on SPF failures (as well as DKIM failures)
coupled with auth failure FBLs has convinced me that there is quite a
bit of utility in capturing the payload. I guess where you stand
depends on where you sit.

> So I can support the support of AUTH-RES, but not with a focus of reporting
> to force payload downloads for the purpose of reporting AUTH-RES fail
> reports.
>

Downloading the payload has always been an option in RFC4408 as
indicated in 2.5.4.

>> I believe this falls within the current scope, but it would involve either
>> an additional draft deliverable or an additional section in the spfbis
>> document.
>>
>> report SPF failures.  In particular, SPF itself uses "fail", but A-R uses
>> "hardfail" to report that case.  Since RFC5451 set up IANA registries for
>> action and maybe an "Updates" against RFC5451.
>>
>> I believe we could do this in spfbis easily as it amounts to nothing more
>> than a correction, and those are in scope.  Although this modifies a
>> document outside of our current plan, it is a germane modification to this
>> work.
>>

---

From: Hector Santos <[email-masked]>
Date: Sat, 24 Dec 2011 00:17:04 -0500

Dotzero wrote:

>> added, no Received-SPF. �So by the time mail is downloaded, there will only
>> be for results:
>>
>> � �result = = "Pass" / SoftFail" / "Neutral" /
>> � � � � � � � "None" / "TempError" / "PermError"
>>
> 

It is true if the mail is rejected which is often the cause with the 
deterministic SPF protocol.

> would mean that "Fail" could conceivably be a reported option.

Yes, see below.

>> �Accept/Bounce attacks.

And the history and how one has integrated newer stuff.

> Downloading the payload has always been an option in RFC4408 as

It was left open ended because of the history.

The language in the original specs leading up to RFC 4408  was first 
considered when implementations were only mostly possible with 
ACCEPTED mail system. But as more implementations came on board and 
offered or changed to do more dynamic SMTP level processing, this 
altered the mind set.

Keep in mind SPF was based on two other LMAP protocols called DMP and 
RMX. Both used SMTP level parameters for the LMAP domain::ip 
association assertion. We supported and explored DMP since RMX was 
more complex.  In fact, DMP was deprecated long ago, and finally 
removed in our WCSAP product early this year.  See Example Below.

This was specifically highlight during the SPF-DISCUSS and MARID 
because there were two (actually four now) kinds of operations:

   - Original, Always Accept, especially for high scale operation needs.

   - With better/faster hardware, multi-thread OSes, more modem SMTP 
software
     began to do hooks at the SMTP state machine level.  This capability
     was limited to certain SMTP software, it was not universal.

   - Many of the original setups, that did have the capability but
     would not scale back then, with the better hardware, they could 
feasibility
     do more of smtp level work, and

   - finally, among all, even if the software had dynamic SMTP level 
shimming
     capabilities, it MAY NOT included the more complex shimming at the
     DATA state.   This because more important as DKIM was coming 
along and
     newer software began to add the FILTERING scripts to the DATA level
     to offer a EOD response.

In short, SPF is written to allow people to implement it for both 
types of operations - Dynamic SMTP vs POST SMTP mail processing 
because the industry was changing very rapidly with better and faster, 
multi-core hardware during the 2000 decade. What systems were not 
capable of doing before and had to scale out, were not able to SCALE 
UP and add more integration of backend hosting databases and do more 
dynamic processing of the mail.  This is very important to combat the 
hugely problematic Accept/Bounce problem.

There was no doubt the direction took on a SMTP level deterministic 
result for HARD FAILS. If this was never the case (SPF clearly 
required the payload in all cases, or you believed that is how most 
systems worked) then the SPF/SENDER-ID history would be very 
different.  That was the whole battle with the issue with SENDER-ID - 
not requiring or be dependent on the payload.

So yes, there are things to clean up in 4408 with semantics when a 
Received-SPF: FAIL is possible but most likely will never be used or 
seen.  Most of the language in 4408 was based originally on POST SMTP 
processing because that is how people explored it.  But it quickly 
changed to DYNAMIC SMTP and the language was altered, not not always 
consistent.  I saw that back then, but didn't think it was worth the 
time to get that cleaned up.

Just consider, then to support SPF at the POST SMTP level, you need 
three parameters passed to the RFC5322 payload as headers or as META data.

   CIP IP                   (Connecting IP)
   RPD Return Path Domain   (5321.MAIL FROM)
   CDN Client Domain        (5321.EHLO/HELO)

The RPD was required with SMTP receivers with the

   Return-Path:

So POST-SMTP SPF processor can get that piece of data.  But keep in 
mind that this COULD BE removed before the mail is finally posted. 
This is UUCP/SLIP history here related to this, the SPF would have to 
be done before it was finally posted.

But where would the POST-SMTP SPF processor get the CIP and CDN?

The only possibility would be a SMTP required Received: hop line.  But 
not critical because an implementation can use proprietary META header 
for the post processing before the final post is done, like:

   ENV-IP:  1.2.3.4
   ENV-CDN: mail.foobar.com
   ENV-FROM: [email-masked]
   ENV-TO: [email-masked]
   ENV-DATA:
   {RFC5322-PAYLOAD}

The point is this is also material to codify in SPF-BIS to finally get 
to do modes of operations consistent for all results regardless if 
payload is downloaded or not.

But for reporting, it must be understood that fail at SMTP MAY NOT 
include any payload available to add any Received-SPF or AUTH-RES 
header to pass on to anyone.

DMP EXAMPLE:

Here is how DMP work which predated SPF using DMP records still existing:

  nslookup -query=txt 9.131.247.208.in-addr._smtp-client.santronics.com

  9.131.247.208.in-addr._smtp-client.santronics.com

       text =  "dmp=allow"

Change the IP and repeat:

  nslookup -query=txt 8.131.247.208.in-addr._smtp-client.santronics.com

  8.131.247.208.in-addr._smtp-client.santronics.com

       text = "dmp=deny"

I really liked DMP because it was designed to work with wild cards, 
but it also required you to add lots of TXT records and that is SPF 
offered over DMP. Otherwise, the concepts are the same.

OVERALL - Accepting the PAYLOAD with SPF fails, is the odd exception 
for pure and true SPF operations that 100% honor a domains SPF hard 
fail policy.  If a SPF receiver disregards the DOMAIN policy, then the 
legal responsibility now passed on to the RECEIVER because it 
intentionally ignored a DETERMINISTIC hard failure.

---

From: Hector Santos <[email-masked]>
Date: Sat, 24 Dec 2011 00:51:37 -0500

As a follow up,

> Downloading the payload has always been an option in RFC4408 as

But consider the text in 2.5:

    Although invalid, malformed, or non-existent domains cause SPF checks
    to return "None" because no SPF record can be found, it has long been
    the policy of many MTAs to reject E-Mail from such domains,
    especially in the case of invalid "MAIL FROM".  In order to prevent
    the circumvention of SPF records, rejecting E-Mail from invalid
    domains should be considered.

In 2.5.5.  SoftFail

    A "SoftFail" result should be treated as somewhere between a "Fail"
    and a "Neutral".  The domain believes the host is not authorized but
    is not willing to make that strong of a statement.  Receiving
    software SHOULD NOT reject the message based solely on this result,
    but MAY subject the message to closer scrutiny than normal.

Ask Scott, this became an OPTION with implementations to force a 
rejection due to the wide abuse over the 2000 decade experience.   We 
offer SPF options such as these:

   Accept-SPF-Pass      True            ; if false, continue testing
   Accept-SPF-SoftFail  FALSE           ; if false, continue testing
   Accept-SPF-Neutral   FALSE           ; if false, continue testing

   rescode_spf_fail     550         ; rejected by SPF Lookup
   rescode_spf_softfail 550         ; rejected by SPF Lookup

and that last one was added because of operational experience.

In 10.5.  Untrusted Information Sources

it says:

    SPF uses information supplied by third parties, such as the "HELO"
    domain name, the "MAIL FROM" address, and SPF records.  This
    information is then passed to the receiver in the Received-SPF: mail
    headers and possibly returned to the client MTA in the form of an
    SMTP rejection message.

This is only possible with a ACCEPT mail first operation in order to 
create a bounce.  A SMTP Level reject will not have a DSN or BOUNCE so 
its impossible to have an Receiver-SPF at this point.   The sender 
could add it when it does the bounce, but I doubt it will add a 
Receiver-SPF unless there was some SMTP negotiated information in the 
55x response where the sender can determine it was a SPF rejection.

Overall, while RFC4408 has history to allow for payload acceptance 
before SPF processing was done, it is not the dominant mode and main 
focus of implementing SPF today as a means to avoid the need to accept 
the payloads in the first place.

This is the kind of stuff we need to clean up in the RFC 4408, because 
the semantics are mixed with chicken vs egg design issues.

---

From: Alessandro Vesely <[email-masked]>
Date: Sat, 24 Dec 2011 14:39:03 +0100

On 24/Dec/11 00:25, Scott Kitterman wrote:
> One possible reason to deal with this as core work would be if we
> wanted to explore deprecating the Received SPF header field in
> favor of the Authentication Results header field.

We don't need to deprecate the former in order to allow the latter.

An implementation may be configured to add either header field,
independently of each other.  A-R needs some configuration anyway,
e.g. for how to set the "authserv-id" token.

> SPF Received is in wide use, so it's not a candidate for removal,

+1

> but we might start in that direction if we could make a few
> adjustments to Authentication Results.

I don't think we need a separate document for establishing allowing
A-R instead of or in addition to R-SPF.  However, it would be
impractical to have a large document such as 4408bis update RFC 5451,
because readers will have a hard time spotting what does the update
consist of.  It would be much better to produce a 5451bis, and/or
somehow bless the errata meanwhile.

jm2c

---

From: Scott Kitterman <[email-masked]>
Date: Sat, 24 Dec 2011 19:19:50 -0500

> reporting to force payload downloads for the purpose of reporting

One of the rules for new modifiers in SPF is that they cannot change the SPF 
result.  Whatever this extension does, the output it produces is not an SPF 
result, it's some kind of other result and should be reported in a way that 
doesn't confuse it with an SPF result.  The extension draft should probably 
cover how to report it's results since it needs a different authenticatoin 
type.

Scott K

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sat, 24 Dec 2011 21:07:29 -0800

> -----Original Message-----
> From: Dave CROCKER [mailto:[email-masked]]
> Sent: Friday, December 23, 2011 1:50 PM
> To: Murray S. Kucherawy
> Cc: [email-masked]
> Subject: Re: [spfbis] Suggestion for an additional deliverable
> 
> I don't understand.  Unless I missed it, the draft charter does not
> cite RFC 5451, reporting or header fields.  How does modifying a
> reporting specification fall within the current draft charter?

There's some movement toward replacing the Received-SPF header field with the one defined in RFC5451 as the standard way to relay the results of SPF evaluation on a message.  If that's the consensus desire, then it seems  to me that resolving this erratum is a reasonable thing to include in the overall "clean up and publish" effort that the charter seeks to execute.

We don't have to; I can always do it as an individual submission.  It just seems a good fit.  Or maybe it's time to consider moving RFC5451 to Internet Standard since it's been more than six months and there are ample implementations of it now, which I would consider doing through APPSAWG or as an individual.

-MSK

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sat, 24 Dec 2011 21:08:21 -0800

> From: [email-masked] [mailto:[email-masked]] On Behalf Of SM
> Sent: Friday, December 23, 2011 1:42 PM
> To: [email-masked]
> I suggest that you defer this until after the core work is completed.
> There will always be a good reason for adding deliverables.  It's
> better not to go down that path.

I suggest that it is "core work" given that the goal here is to clean up and republish SPF, and there's some consideration for the idea of dropping Received-SPF and replacing it with Authentication-Results.

-MSK

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sat, 24 Dec 2011 21:10:01 -0800

> From: [email-masked] [mailto:[email-masked]] On Behalf Of Commerco WebMaster
> Sent: Friday, December 23, 2011 3:44 PM
> To: [email-masked]
> entertain adjusting 2.4.2 (the SPF and Sender-ID results) to reflect
> the SPF responses per RFC 4408?  Hardfail is not mentioned in RFC 4408
> into RFC 5451 (unless it is part of the Sender-ID fork, which I think
> is outside the scope of the charter for SPFbis).

I'd be happy to do that.  Fortunately we can tackle it as a simple IANA change and not a full republication of RFC5451.  Since the change is specific to SPF, I thought this might be a reasonable thing to add in the initial effort of spfbis.

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sat, 24 Dec 2011 21:17:39 -0800

> From: [email-masked] [mailto:[email-masked]] On Behalf Of Alessandro Vesely
> Sent: Saturday, December 24, 2011 5:39 AM
> To: [email-masked]
> > wanted to explore deprecating the Received SPF header field in favor
> > of the Authentication Results header field.
> 

While true, it does also mean that an agent which adds one or the other needs to pick the one that downstream agents (i.e., MUAs) are most likely to use.  Having two standards to do the same thing doesn't help interoperability.

> > adjustments to Authentication Results.

I suggest we work toward convergence and eventually deprecate Received-SPF.  If we can do that directly in RFC4408bis by fixing up this one errata item, then we should do so.

> I don't think we need a separate document for establishing allowing A-R
> instead of or in addition to R-SPF.  However, it would be impractical
> to have a large document such as 4408bis update RFC 5451, because
> readers will have a hard time spotting what does the update consist of.
> It would be much better to produce a 5451bis, and/or somehow bless the
> errata meanwhile.

I disagree; RFC4408bis can update RFC5451 just to make this one correction, which amounts to a simple IANA action.  RFC5451 would thereafter show as "Updated by RFC4408bis", which means "If you're implementing RFC5451, you really should go read RFC4408bis as well".  The corrected definition will then appear in two places: RFC4408bis, and the registry of A-R method results.

RFC5451 has two errata against it, one of which is minor (a change to an example), and the other would be resolved by this action.  No need to do an RFC5451bis just for this.

Nice and neat.

-MSK

---

From: Scott Kitterman <[email-masked]>
Date: Sun, 25 Dec 2011 02:03:21 -0500

On Saturday, December 24, 2011 09:07:29 PM Murray S. Kucherawy wrote:
> There's some movement toward replacing the Received-SPF header field with
> the one defined in RFC5451 as the standard way to relay the results of SPF
> evaluation on a message.  If that's the consensus desire, then it seems  to
> me that resolving this erratum is a reasonable thing to include in the
> overall "clean up and publish" effort that the charter seeks to execute.

I don't think the RFC 5451 Authentication Results header field is currently a 
fully suitable replacement for Received-SPF.  It is close, but I think there 
is work to be done beyond the eratta.  Not a lot, but some.  I think it's 
sensible to fold adding investigating this as a core task and then using 
4408bis to update 5451 if there is consensus to go in this direction.

Scott K

---

From: Dave CROCKER <[email-masked]>
Date: Sun, 25 Dec 2011 13:14:10 -0800

On 12/24/2011 9:07 PM, Murray S. Kucherawy wrote:
> There's some movement toward replacing the Received-SPF header field

Absent an emergency, I'd recommend deferring this until the core work is done. 
That's not a comment on the benefit of the added task, but of working group focus.

There might be some debate about having that spec fit within this working group 
at all, given that it's scope is broader than SPF, but since the actual work is 
SPF-related, we can probably fold it in.

But, really, I think we'll do best by maintaining focus on the core work of the 
charter, first.

d/

-- 

   Dave Crocker

---

From: Hector Santos <[email-masked]>
Date: Sun, 25 Dec 2011 23:45:38 -0500

> cover how to report it's results since it needs a different authentication 
> type.

+1, I think you are touching base with what we all expected would 
inevitably need to do once these protocols since MARID got to a point 
were we could it all together and had the data to evaluate and 
optimize the process - Basically establishing what would be the new 
persistent assertions both deterministic and heuristics systems could 
yse to get a low/zero false positive high payoff by combining all the 
email technologies, in particular SPF, DKIM and DKIM POLICY.  IMO, the 
deterministic policies were easy in concept to understand, but it was 
the indeterministic results that needed to be dealt with - items 
closely related to a common issue with both IMO - 3rd party or middle 
ware related issues.  With SPF, authorizing the transition point 
(multi-hops) issue and with DKIM, the authorization of the 3rd party 
signature.

I have some text I wanted to post regarding this but will wait until I 
get a chance to review your draft and perhaps (or not) introduce it in 
my comments - haven't had the time yet to read it but will sometime 
early this week. :)

---

From: Alessandro Vesely <[email-masked]>
Date: Mon, 26 Dec 2011 19:42:31 +0100

On 25/Dec/11 06:17, Murray S. Kucherawy wrote:
>> From: ietf.org On Behalf Of Alessandro Vesely
>> 
> While true, it does also mean that an agent which adds one or the
> other needs to pick the one that downstream agents (i.e., MUAs) are
> most likely to use.  Having two standards to do the same thing
> doesn't help interoperability.

It is also a question of modularity.  The mail filter that sets
Authentication Results needs to access the whole body of a message,
and that might be the first time it gets called.  In such cases,
Received-SPF can be considered a convenient standard place to store
the result of an SPF evaluation, temporarily if the filter will later
translate it into an A-R field --on a shared-object architecture like
Sendmail one could use core memory, but that's not always the case.

For example, I get these:

> "If you're implementing RFC5451, you really should go read RFC4408bis as well".

Isn't it a pain in the neck to have to learn the whole SPF spec for
someone who just wishes to implement, say, the auth method for his MSA?

---

From: Stuart Gathman <[email-masked]>
Date: Mon, 26 Dec 2011 17:20:38 -0500

Long ago, Nostradamus foresaw that on 12/24/2011 12:17 AM, Hector Santos
> not requiring or be dependent on the payload.

Checking SPF dynamically is technically not a problem.  The problem for
a small business is that a sender will publish a policy which gives a
FAIL result, but still expect the email to be delivered.  If we were
gmail or aol, they would fix their problem and move on, but they will
not do so for a small business.  Hence, a small business must maintain a
database of overrides to sender policy.  Often, this database will
include "substitute" SPF record reverse engineered from the sending
patterns of the broken email domain.   When reporting SPF results for
such domains, an implementation MUST report the ACTUAL SPF result, and
not the "customer is never wrong" policy driven result.  Hence, any
Received-SPF or Authentication-Results header must be able to report a
FAIL result.

P.S.  Should 4408bis address when/if to send DSNs based on SPF results?

---

From: Alessandro Vesely <[email-masked]>
Date: Wed, 28 Dec 2011 11:50:29 +0100

On 26/Dec/11 23:20, Stuart Gathman wrote:
> 
> Hence, a small business must maintain a database of overrides to
> sender policy.  Often, this database will include "substitute" SPF
> record reverse engineered from the sending patterns of the broken
> email domain.   When reporting SPF results for such domains, an
> implementation MUST report the ACTUAL SPF result, and not the
> "customer is never wrong" policy driven result.

Agreed.  However, well established policies (e.g. best-guess) deserve
to be reported adequately too --especially for empty MAIL FROMs.

> Hence, any Received-SPF or Authentication-Results header must be
> able to report a FAIL result.

Yes.  But for result-based processing, that Hector calls post-SMTP,
authentication is successful on "pass".  Neutral, fail, or whatever
else are just not authenticated.  IOW, the important point is to bring
out /positive/ results.  Is that correct?

> P.S.  Should 4408bis address when/if to send DSNs based on SPF results?

Given the similarity between DSN and SPF-reporting, that discussion
may open up the possibility of a nice summing-up if it will be
addressed in the other deliverable.

---

From: Hector Santos <[email-masked]>
Date: Wed, 28 Dec 2011 08:57:21 -0500

> P.S.  Should 4408bis address when/if to send DSNs based on SPF results?

I like to view this as what is required in the first place.

If an Online SMTP rejection is performed (for any reason), there is no 
DNS or bounce  possibility at this node.  The sender, or rather the 
hop (receiver) that did the initial 250 acceptance is required per 
SMTP to do a DNS/BOUNCE.

That has nothing to do with SPF or any other email technology, and to 
keep with persistent engineering,  SPF should not change it and if so, 
only by a standard POLICY concept that we have endorsed.

That is where SPF policies come in to play:

    FAIL     - A Domain Policy to REJECT, don't question it.
    SOFT     - It looks wrong, but may be rejected is not automatic
    NEUTRAL  - "I have no clue what I am doing."

FAIL is a 100% deterministic and the DOMAIN is telling you, the sender 
information its wrong, don't accept it.  The other two are 
indeterminate conditions.

Any debates regarding this strong hard FAIL SPF policy is not related 
to the technical merits but one that begins to question the DOMAIN 
decision to use it.  Thats is an entirely different concept. We often 
hear people say the domain doesn't know what its doing. That is was a 
poor or a mistake that a domain has a -ALL SPF policy.  Well, I take 
the position giving domains the benefit of the doubt, one that 
included an explicit series of steps to add a -ALL to the SPF record.

I agree there are times where you may need to whitelist a hard SPF 
policy because it took a path where the domain::ip assertion fails. 
But these are known transactions,  they are not anonymous. This is the 
part of the protocol awareness of knowing the difference. Its not an 
reason to ignore the hard policy and accept all mail anyway.   So I 
view this white listing is part the SPF protocol local policy option 
to TEST for SPF result, record and accept the mail only, but it is an 
EXCEPTION, not the NORM to do this.

Look at this way, if SPF had this framework where you are just 
TEST/RECORD/ACCEPT the payload anyway, I sincerely doubt it would have 
been successful or even get the huge and super fast industry wide 
endorsement it did.  I know I and many others would not had wasted our 
time when at the time the #1 problem was the huge ACCEPT/BOUNCE 
dilemma that the 2003 SORBIG eVirus highlighted to the world. That is 
what give a SPF a spark and got MARID started.  There was no dominant 
position that was saying "Oh, lets test, record and accept the mail 
anyway."  No.  We were looking at many LMAP ideas and SPF was one of 
them for a new standard transport level based technology for immediate 
online SMTP level rejection.  All the other payload stuff were 
interesting but it was very different and we need to remember that 
with SPF.  The opponents of SPF has longed question the idea of an 
hard deterministic rejection protocol and also Receivers that were 
100% ok with the idea of honoring them.  I prefer not to get into this 
debate of questioning DOMAIN decision to use hard deterministic 
policies or receivers that choose to honor it by simply following the 
SPF protocol immediate reject semantics.   Yet, there is no hard rule 
that says a local policy can do a Test/Record/Accept setup.

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 28 Dec 2011 20:19:02 -0800

> Sent: Monday, December 26, 2011 10:43 AM
> To: [email-masked]
> Authentication Results needs to access the whole body of a message, and
> that might be the first time it gets called.

Why's that?  An Authentication-Results field could be added to a message as soon as SPF completes evaluation, and it doesn't require a full body download.

> Return-Path: <[email-masked]>
> Authentication-Results: wmail.tana.it;
>   spf=pass smtp.mailfrom=ietf.org;
>   dkim=pass header.i=@ietf.org

There's no requirement that they be combined.  At my site I run sid-milter and opendkim, and they attach those independently.

> someone who just wishes to implement, say, the auth method for his MSA?

I was just explaining what the "Updates" would mean.  We could leave it off if that's how it might be interpreted.

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 28 Dec 2011 20:20:11 -0800

> From: [email-masked] [mailto:[email-masked]] On Behalf Of Stuart Gathman
> Sent: Monday, December 26, 2011 2:21 PM
> To: [email-masked]

Sure, if the community has operational experience in that area that warrants making some statements about it.

---

From: Scott Kitterman <[email-masked]>
Date: Wed, 28 Dec 2011 23:51:04 -0500

On Wednesday, December 28, 2011 08:20:11 PM Murray S. Kucherawy wrote:
> > From: [email-masked] [mailto:[email-masked]] On Behalf
> > Of Stuart Gathman Sent: Monday, December 26, 2011 2:21 PM
> > To: [email-masked]

I think it's something for a BCP that might be part of follow on work after 
4408bis.

Scott K

---

From: Alessandro Vesely <[email-masked]>
Date: Thu, 29 Dec 2011 14:03:28 +0100

On 29/Dec/11 05:19, Murray S. Kucherawy wrote:
> Why's that?  An Authentication-Results field could be added to a
> message as soon as SPF completes evaluation, and it doesn't require
> a full body download.

Courier-MTA checks SPF natively /and/ calls filters after full body
download.  So its users cannot have reject-on-fail /and/
filter-handled SPF at the same time.

> There's no requirement that they be combined.

Correct.  Assuming that next release of Courier-MTA offers a choice,
the only advantage of combining them will consist in specifying the
"authserv-id" in just one config file.

---

From: Douglas Otis <[email-masked]>
Date: Thu, 29 Dec 2011 10:59:22 -0800

> I think it's something for a BCP that might be part of follow on work after
Is there any interest in assuring use of mx mechanism will work properly 
with domains publishing more than 5 MX records?  Many current SPF 
libraries process a maximum of 5 rather than the specified 10 to 
circumvent a possible 100 DNS transactional sequence.  The impact is 
this may cause intermittent failures difficult to troubleshoot.  What 
makes this contentious is possible macro expansion of local-parts 
against cached SPF records.

-Doug

---

From: Hector Santos <[email-masked]>
Date: Fri, 30 Dec 2011 21:11:05 -0500

Douglas Otis wrote:

> with domains publishing more than 5 MX records?  

Can you illustrate the MX concern and whether there is an SPF problem 
here?

> Many current SPF 

Doug, its been what?  nearly 9, 10 years? I have yet to see a 
situation that was not managed with the SPF implementation, especially 
with recursive parts where limits are placed.

I have yet to see or hear of any exploit that circumvents SPF 
implementations.  Its all theoretical with the presumption the 
implementation is not aware of aggressive abuse.

---

From: Hector Santos <[email-masked]>
Date: Fri, 30 Dec 2011 21:21:05 -0500

> I think it's something for a BCP that might be part of follow on work after 

In terms of SPF, a "BCP" is to reject HARD failures and/or mark them, 
record and save if you wish, but overall it is hurting users if a HARD 
failure was passed to them or begin to think about sending SPF 
determined rejection as BOUNCES - thats goes against its whole purpose 
in email life.

If we begin to argue that fundamental point, then SPF is under attack 
for a framework and mindset change in this small and isolated group 
with leaders that are 100% against SPF and that concerns me.

PS: Happy New Years :)