Subject: Re: [dmarc-ietf] dmarc Digest, Vol 11, Issue 8

From: Vlatko Salaj <[email-masked]>
Date: Fri, 14 Feb 2014 02:01:49 -0800 (PST)

On Friday, February 14, 2014 9:26 AM, "[email-masked]" <[email-masked]> wrote:

> 1. That a given IP address has a substantial, sustained message volume;
  large and persistent enough to perform meaningful statistical analysis on.
> 2. That the complaint rate for messages from that IP address is
        consistently
  negligible.
> 3. That messages from that address routinely fail authentication
        from a wide
  range of domains that otherwise pass when they deliver directly.

these heuristics don't account for a forwarder that fails on 2nd point, which
is quite frequent situation on shared hosting; nor do they account for a
forwarder failing on 1st point.

however, heuristics looking at SRS-enabled forwarder DMARC-enabled message would
solve even such scenarios.

> If you mean that the mere use of SRS by a sender could usefully be
    treated
> by a receiver as a signal that the sender is reliable

no, i don't. even DMARC itself doesn't offer that.

> if you trust someone to do
    SRS honestly and competently then you can probably
> exempt them from
    DMARC processing anyway, meaning that SRS adds nothing.

no. u use SRS heuristics to determine why SPF/DMARC have failed in the 1st
place; so other way around. however, i'm not suggesting we should extend DMARC
spec to include that. i do suggest we mention it as a useful technique when
deploying a forwarder meant to observe DMARC as much as possible, among other
things.

>> given that main purpose of DMARC is to help major mailbox receivers identify
>> reliable email from major email senders,
> Not quite; it's to help Domain Owners of any size - not just major
    ones - and
> receivers co-operate on keeping spoofed mail out of
    inboxes.

sure, in some ideal future where every domain deploys DMARC. it doesn't do that
currently, and may never, considering all DMARC failures as a protocol, and
reservation on its usefulness by many developers, thought as a mere trinket for
major domain owners against phishing.

not that i have anything against another protocol to help us fight such things.

but let's be realistic. DMARC, with its reliance on "from:" field, is basically
broken for most, but cleanest of message transaction paths, under which its
usefulness remains in area of what i mentioned earlier, and mostly only that.

personally, i never saw a domain of a small company that has such a clean
message transaction path. i even keep getting errors from ietf.org itself, as
well as isoc.

DMARC just adds more mess there. i'm trying to suggest something to help combat
that, while keeping specs as they are now.

>> imo, SPF spec did fail to do that, for whatever reason, and maybe we should
>> try not to miss such opportunity again.
> If you are implying that the absence of SRS from the SPF spec is any
    part of
> the reason for the non-adoption of SRS by forwarders then
    you are mistaken.

ofc it is, at least partly. if u provide no guidance on official specs, nobody
will deploy it. that's the main reason SRS isn't largely deployed on SPF-enabled
forwarders. thry have no idea about it, and even if they do, software they use
doesn't.

>> current draft does talk about ADSP quite a bit, all while ADSP seems even
>> less relevant to DMARC than SRS.
> DMARC is very close to ADSP

close, but otherwise, technically, completely irrelevant.

even mentioning it is a junk data nobody should waste time on while reading
DMARC spec. ADSP-centered text in DMARC draft is one page length, and more,
all while providing no useful information on
 DMARC deployment in any way,
but instead moves topic to a protocol 
participants SHOULD NOT use and which
has been made obsolete by it. waste of spec space, imo.

that was my impression every time i reread the doc. it should be removed from
draft, or replaced with a simple:

DMARC doesn't use ADSP, and it is not recommended that participating entities
deploy or account for ADSP. DMARC builds on ADSP capabilities and efficiently
surpasses it.

quite enough. maybe it would make more space to talk about DMARC failures and
how to alleviate them.

-- 

Vlatko Salaj aka goodone
http://goodone.tk

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Fri, 14 Feb 2014 07:24:04 -0800

> deploying a forwarder meant to observe DMARC as much as possible, among

There's a bit of a slippery slope here, for two reasons:

(1) If some sites implement SRS checking and others don't, then a sender
gets a mixed experience when some receivers see the messages as failing
DMARC while others don't.  On the flipside, that might be true already even
without SRS.

(2) Nudging people in the direction of considering SRS suggests we should
also start mentioning DKIM extensions that further complicate things, like
the experimental ATPS.  I'm not sure of the benefit.

I think there are ample valid reasons why SRS has not been widely adopted
to date, and it's been around as long as SPF has.  I doubt the advent of
DMARC is going to change any of that.

but let's be realistic. DMARC, with its reliance on "from:" field, is
> basically
> message transaction path. i even keep getting errors from ietf.orgitself, as
> well as isoc.
>

The data I've seen suggest otherwise.  If DKIM signatures are passing the
vast majority of the time, which more than one party here has claimed, then
the transaction paths (be they direct or indirect) aren't all that
unclean.  That goes for large and small senders and receivers alike.

> forwarders. thry have no idea about it, and even if they do, software they

I'll go the other direction: I think including SRS as an even optional part
of SPF would have hindered SPF adoption.  Keeping them apart was absolutely
the right move.

> has been made obsolete by it. waste of spec space, imo.

I don't agree here either.  An obvious question that comes up when talking
about DMARC, or any new work, is "What problem is this solving?", often
followed by "But doesn't other-protocol-X, which already exists, already do
that?"  The text we have there answers the question.  It's possible we
could be more brief, but I don't agree that it's "no useful information" to
the general reader.

-MSK

---

From: Vlatko Salaj <[email-masked]>
Date: Fri, 14 Feb 2014 11:14:49 -0800 (PST)

On Friday, February 14, 2014 4:24 PM, Murray S. Kucherawy <[email-masked]> wrote:

> (2) Nudging people in the direction of considering SRS suggests we should also
> start mentioning DKIM extensions that further complicate things, like the
> experimental ATPS. I'm not sure of the benefit.

ATSP solves another scenario, and has little to no connection to DMARC. one can
use it with DMARC or not, changes little.

on the other hand, SRS solves situations in which a domain owner is forced to
use "none" DMARC policy cause its messages go through forwarders. a situation
which basically trashes all benefits DMARC, SPF and DKIM provide for such domain.

if forwarders have SRS enabled, such domain owner would at least be able to rely
on SPF evaluation on receivers' side, which would pass, even if DMARC doesn't.

that's why this problem is connected to DMARC. protocol as it is creates
additional mess for forwarders, but suggests no proper solution in any way.

DMARC draft should include such info, whatever it is. and SRS is such info.

also, SRS is not alone. mentioning, for example, that mailing list providers
should do their best not to break DKIM in their processing should also be added
as guidance in DMARC specs. and i'm sure there are some other suggestions
for solving DMARC inefficiencies that should go into this category too.

if u waste so much draft space to compare DMARC to ADSP, it's reasonable to
include at least some info on dealing with grief DMARC requirements would
introduce in the email world.

>> basically broken for most, but cleanest of message transaction paths,
> The data I've seen suggest otherwise.  If DKIM signatures are passing the vast
> majority of the time, which more than one party here has claimed, then the
> transaction paths (be they direct or indirect) aren't all that unclean.
> That goes for large and small senders and receivers alike.

i'm not sure that goes for all types of senders. i didn't see any DKIM stats
that actually try do differentiate between major senders, like fb, paypal
and small senders like millions of companies running on cpanel shared hosts.

however, stats about DKIM signatures say nothing about DKIM DMARC alignment,
which numbers are probably quite lower. so, even if DKIM is saved, alignments?

> of SPF would have hindered SPF adoption. Keeping them apart was absolutely the
> right move.

how so? considering SRS is required by, and only by, forwarders... i see little
reason for SPF adoption to be affected.

but, we will never know, will we? actually, if SPF wasn't so broken by
forwarders in the 1st place, maybe we would have no need for DKIM or DMARC now.
but that's what you get when you have microsoft involved into specs discussion.
at least back then.

>> ADSP-centered text in DMARC draft is one page length, and more, all while
>> providing no useful information on DMARC deployment in any way, but instead
>> moves topic to a protocol participants SHOULD NOT use and which has been
>> made obsolete by it. waste of spec space, imo.
> It's possible we could be more brief

i'm quite sure many ppl would be quite thankful for that.

-- 
Vlatko Salaj aka goodone
http://goodone.tk

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Fri, 14 Feb 2014 11:36:05 -0800

> use it with DMARC or not, changes little.

I think they solve different scenarios, but both are arguably important to
some sub-community of the email world.  In that sense, they look the same
to me.  It may not be that ATPS solves any of your problems, but the ATPS
community (small as it is) can similarly say that SRS doesn't solve theirs.

on the other hand, SRS solves situations in which a domain owner is forced
> to

ATPS solves situations in which the domain owner is forced to use "none"
DMARC policy because its messages are signed by third-parties, a situation
which basically trashes all benefits DMARC, SPF and DKIM provide for such a
domain.  :-)

> DMARC draft should include such info, whatever it is. and SRS is such info.

So the same argument could be made for ATPS, or any other logic that
augments the core of what DKIM provides.  And ATPS has the advantage of
being documented in an RFC.

> for solving DMARC inefficiencies that should go into this category too.

A lot of this is written down in RFC6377, at least as it pertains to DKIM.
Perhaps DMARC should reference that as well.

> introduce in the email world.

I get that you think this is wasted space.  Does anyone else concur?  Does
someone want to propose simpler text to replace what's there?

> and small senders like millions of companies running on cpanel shared

The statistics OpenDKIM collects at
http://www.opendkim.org/stats/report.html#passfail don't include feeds from
any major receivers, but rather a few smaller ones.  It still concurs with
the general anecdotal data of an over 90% pass rate.  If you have
statistics to the contrary, I'm sure we'd love to see them.

> which numbers are probably quite lower. so, even if DKIM is saved,

Wouldn't that be visible in the DMARC aggregate reports?

> reason for SPF adoption to be affected.

My recollection of most of the SPF and IETF community at the time was that
SRS didn't get a favorable reaction.  It was seen as ugly and the problem
space was not large enough to warrant the added complexity.  Including it
in what became RFC4408 wouldn't have done it any favors.

> but that's what you get when you have microsoft involved into specs

SPF has not been around as long as forwarders have been, so I think making
the argument that this is forwarders' fault is a bit silly.  I also think
casting blame for past problems isn't going to help here.  We need to focus
on moving forward from where we are now.

> i'm quite sure many ppl would be quite thankful for that.
I saw that you proposed some brief text to replace what's there.  What do
others think?

-MSK

---

From: Vlatko Salaj <[email-masked]>
Date: Fri, 14 Feb 2014 13:51:42 -0800 (PST)

On Friday, February 14, 2014 8:36 PM, Murray S. Kucherawy <[email-masked]> wrote:

> It may not be that ATPS solves any of your problems, but the ATPS community
> (small as it is) can similarly say that SRS doesn't solve theirs.

well, i do not have any proposal on how to fix ATPS-induced DKIM
DMARC-alignment fail. it simply seems DMARC and ATPS are mutually
exclusive, as presently defined. maybe ATPS should be revised to account for
DMARC compatibility.

however, that's not the case with SRS. it affects nothing in DMARC processing,
just helps solve some of its shortcomings; shortcomings introduced with
same alignment requirements, but on SPF side. and we can't exclude SPF,
or revise it; it's an essential part of spec as defined already.

ATPS doesn't solve DMARC shortcomings, it makes them.

also, two points: 1. domain owners control ATPS deployment; 2. they have an
alternative that works with DMARC.

however, domain owners have no control over forwarders, nor they have much of
an alternative.

maybe spec should have a note about incompatibility with ATPS. but that's it.

or is there a better idea flying around?

>> should do their best not to break DKIM in their processing should also be
>> added as guidance in DMARC specs. and i'm sure there are some other
>> suggestions for solving DMARC inefficiencies that should go into this
>> category too.

actually, that may not be a bad idea.

> Wouldn't that be visible in the DMARC aggregate reports?

where? i'm not aware of any publicly released stats, beside those big numbers
last year. but those were more of an ad, than real statistical data.

> SRS didn't get a favorable reaction.

are you sure that wasn't about microsoft's version of it, later part of
sender id spec? asaik, plain SRS wasn't hated, it was merely a special case
handling and thus less important to SPF.

> the argument that this is forwarders' fault is a bit silly.
my point was that cause SPF was never finished accounting for forwarders, mainly
of bad blood caused by sender id, it was, unfortunately, broken from the start.

and yes, forwarders do introduce quite a bit of grief into email world. but, its
late to think about that problem now. we can, however, devise workarounds.

let me ask a public question: was there any intention at all by original DMARC
authors to consider replacing SPF with SenderID, or to use PRA instead of
"from:" header alignment? while controversial in many ways, not everything
in that algorithm was useless.

-- 
Vlatko Salaj aka goodone
http://goodone.tk

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Fri, 14 Feb 2014 17:41:50 -0800

> exclusive, as presently defined. maybe ATPS should be revised to account

Are we talking about the same thing?  ATPS doesn't induce DKIM failures; it
suggests a way to indicate a message signer can be endorsed by an author,
saying "treat this signature as if it were my own".  That brings an
unaligned DKIM signature into alignment, in DMARC terms.  I don't see how
it can deteriorate things.  Rather, its problem is that it's a corner case
with limited adoption.

> maybe spec should have a note about incompatibility with ATPS. but that's
...but they aren't incompatible.

> last year. but those were more of an ad, than real statistical data.

Anyone receiving large amounts of mail and generating reports based on them
(Yahoo, for example) has enough data to tell you what percentage of DKIM
signatures are passing, and what percentage of those are DMARC-aligned.

> handling and thus less important to SPF.

Yes, I'm sure.  And I don't recall there being multiple versions of SRS
back then.  I could be wrong, but it could also be the case that people
only knew about one version.  I also don't recall a version of SRS put
forward by Microsoft.

> late to think about that problem now. we can, however, devise workarounds.

The people working on it had a choice: A version of SPF that didn't deal
with forwarding (which is what we got), or a version of SPF that did deal
with forwarding in a manner the community at the time felt was
unpalatable.  It appears to have chosen the former, for better or worse.

SPF (the document, not so much the protocol) was just revised to change its
status from Experimental to Proposed Standard.  SRS didn't come up there
either; the consensus supported a version that continued not to deal with
forwarding directly.

let me ask a public question: was there any intention at all by original
> DMARC

Including Sender-ID support was considered and discarded, though DMARC is
also extensible to include it later if it turns out to be desirable.
However, Sender-ID appears to be drifting into history, so we instead went
with the algorithm that's there now.

-MSK

---

From: Vlatko Salaj <[email-masked]>
Date: Sat, 15 Feb 2014 00:45:48 -0800 (PST)

> ...but they aren't incompatible.

i only skimmed ATPS specs, so maybe i misunderstood something. however, as far
as i see it, the way it's defined is incompatible with DMARC.

ATPS changes "d=" tag to 3rd party, while "from:" header remains 1st party.
essentially, this breaks DKIM alignment, in DMARC terms. the only way ATPS
could be compatible is as a DMARC policy override, similarly to the way
ATPS defines compatibility with ADSP.

so, mutually exclusive. imo, since ATPS standard is still in its making,
it should be changed to account for DMARC. however, i'm not sure if that's
possible, without making significant changes to DKIM as well.

or just use it as a policy override.

> signatures are passing, and what percentage of those are DMARC-aligned.

reports i'm dealing with go around 10%. however, my pool is too small to
be considered deal-breaking. however, in my experience, DMARC alignment
is a b*atch.

> SPF (the document, not so much the protocol) was just revised to change
> its status from Experimental to Proposed Standard. SRS didn't come up
> there either; the consensus supported a version that continued not to deal
> with forwarding directly.

i guess that answers a question i woke up today with: would any major
mailbox provider be interested into adding SRS into their forwarding
logic? considering how they look at SPF as annoyance as it is [yahoo
not publishing any SPF records, for example], no wonder they don't
like making that protocol even more hard to deploy.

> only knew about one version. I also don't recall a version of SRS put

SRS was made to deal with forwarders, in SPF terms. however, before SPF was
finished, microsoft worked on expanding it to something we know now as
SenderID. SenderID used plain SPF with no SRS, but instead introduced its
own type of dealing with forwarders, mailing lists, u name it - called PRA
[Purported Responsible Address]. considering microsoft pushed for its version
quite a bit, and how SRS made SPF more difficult to implement, SRS never got
enough interests or attention, it was left sidetracked, pushed away by a new
promising idea - PRA.

however, PRA was so bad at the end, the entire working group ceased
functioning cause of conflicts, causing riples between major mailbox owners...

and now SenderID is dead, SPFs is easily broken in real world and SRS never
got any ietf work.

pity actually. but history always repeats itself now and then, i guess.

-- 
Vlatko Salaj aka goodone
http://goodone.tk

---

From: "Rolf E. Sonneveld" <[email-masked]>
Date: Sat, 15 Feb 2014 14:26:27 +0100

On 02/15/2014 09:45 AM, Vlatko Salaj wrote:
> it should be changed to account for DMARC.

One can argue whether it is correct to require to change ATPS, which has 
(Expirimental) RFC status, to accomodate for DMARC, which has no IETF 
status yet.

>   however, i'm not sure if that's

Not sure why you (Murray) mention Yahoo here, but in general it would 
help us when the major ESPs would share their statistics on things like 
DMARC alignment, SPF pass/fail rate, DKIM pass/fail rate, DKIM+SPF 
fail/fail rate etc. The most interesting question here being: what 
percentage of legitimate mail fails both SPF and DKIM (DMARC).

> like making that protocol even more hard to deploy.

Forget about SRS. To paraphrase John Levines statement about the budget 
that receivers have to fix the problems of the sender:

"The total budget at all _forwarders_ for solving senders' problems is $0."

For senders and receivers there can be (some) mutual interest in solving 
problems at either side (this is true for 'transactional' mail), but 
there is no mutual interest between senders and forwarders and there's 
no mutual interest between forwarders and receivers.

/rolf

---

From: Vlatko Salaj <[email-masked]>
Date: Sat, 15 Feb 2014 06:20:14 -0800 (PST)

On Saturday, February 15, 2014 2:26 PM, Rolf E. Sonneveld <[email-masked]> wrote:

> problems at either side (this is true for 'transactional' mail), but there is
> no mutual interest between senders and forwarders and there's no mutual
> interest between forwarders and receivers.

i beg to differ. there's barely a single forwarder online that's not connected
to either party in some way, and many are controlled, to a degree, by them.

so, basically, forwarders' problems are either receivers problems or senders
problems. however, solutions usually don't offer removing forwarders out of the
message path, for whatever reason. i've seen it many times.

so, we should work on protocols helping us to overcome whatever problems
forwarding brings into equation.

on the side note, i have contacted SRS main author about moving SRS to
IETF process. we'll see how that goes.

-- 
Vlatko Salaj aka goodone
http://goodone.tk

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sat, 15 Feb 2014 16:02:17 -0800

> ATPS defines compatibility with ADSP.

Correct, but a DKIM verifier that encounters proper ATPS use would simply
report to the DMARC module that the first party signed the message, and
alignment is thus not violated.  In the same way, an SPF module could
report a "pass" where SRS tells it so, even if pure SPF were to fail.
Thus, to me, they're the same kind of thing.

> possible, without making significant changes to DKIM as well.

It doesn't make sense to modify ATPS, which is a DKIM adjunct, to account
for DMARC, which is a layer above.

> or just use it as a policy override.

That's closer to what really happens.

> like making that protocol even more hard to deploy.

I don't think they see SPF as an annoyance, but I also get the impression
that the forwarding issue is not enough of a pain point to roll out another
chunk of infrastructure.  (Don't make the mistake of underestimating the
cost in that for large operators.)

> enough interests or attention, it was left sidetracked, pushed away by a

That sounds closer to the history I remember, specifically the "made SPF
more difficult to implement".  The payoff for the added complexity didn't
seem to be there.

> pity actually. but history always repeats itself now and then, i guess.

I think you just explained why DMARC went with the (SPF || DKIM) logic.

-MSK

---

From: Vlatko Salaj <[email-masked]>
Date: Sun, 16 Feb 2014 03:04:05 -0800 (PST)

On Sunday, February 16, 2014 1:02 AM, Murray S. Kucherawy <[email-masked]> wrote:

> alignment is thus not violated.

if, and only if, we change DMARC specs to accommodate. and section 3.1.4.1.
is 1st in the line. we need to remove all "d=" references, and just stick
with: "DKIM authenticated domain".

but, yes, this seems right way to go, and doesn't seem a big problem to fix.

> In the same way, an SPF module could report a "pass" where SRS tells it so,
> even if pure SPF were to fail. Thus, to me, they're the same kind of thing.

it's not completely similar to ATPS. even when we use SRS, SPF alignment will
fail. however, SPF will pass, and this way, a receiver has additional data to
evaluate such sending host as a probable reliable forwarder. it helps much in
situations when receiver has little useful data to make such evaluation using
other mechanisms, or evaluation between mechanisms is contradicting.

SRS helps with DMARC policy override, as Franck concluded previously.

> I don't think they see SPF as an annoyance, but I also get the impression that
> the forwarding issue is not enough of a pain point to roll out another chunk
> of infrastructure.

they should. all of major ESPs provide forwarding as a service, thus breaking
more than one protocol. and i am not sure if any of them tries to accommodate
for such issues, while doing so.

-- 
Vlatko Salaj aka goodone
http://goodone.tk

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Mon, 17 Feb 2014 08:18:46 -0800

> SRS helps with DMARC policy override, as Franck concluded previously.

This has devolved into hair-splitting now.  Both of them provide
information for DMARC policy overrides.  You can either modify your SPF or
DKIM module to return a modified result based on the SRS or ATPS,
respectively, or the information can be provided separately.  The result is
the same: alignment where there was none previously.

> for such issues, while doing so.

Perhaps, but again, it has not created enough of a problem to warrant
action.  Action is expensive.  If more people complained about it, the
answer might be different.  But, apparently, they don't.

-MSK

---

From: Franck Martin <[email-masked]>
Date: Mon, 17 Feb 2014 11:16:04 -0600 (CST)

----- Original Message -----

From: "Murray S. Kucherawy" <[email-masked]> 
To: "Vlatko Salaj" <[email-masked]> 
Cc: [email-masked] 
Sent: Monday, February 17, 2014 8:18:46 AM 
Subject: Re: [dmarc-ietf] SRS helps SPF/DMARC 

No, I said, it may help, but this is for each receiver to decide how it applies overrides at the sole risk of the receiver. Any override opens a security hole, therefore this is discouraged and should not be standardized. 

I think you need to implement DMARC, apply some of these techniques, and come back to us with some data, experience. 

_______________________________________________ 
dmarc mailing list 
[email-masked] 
https://www.ietf.org/mailman/listinfo/dmarc