Subject: [dmarc-ietf] Rethinking DMARC for PSDs

From: "Douglas E. Foster" <[email-masked]>
Date: Sun, 7 Apr 2019 20:02:33 -0400

I understand how much work it takes to create consensus toward an IETF 
standard, but I suggest that the problem needs to be re-examined because 
DMARC for PSDs seems to be neither the sufficient solution nor the 
necessary one.
  
 The problem:
  	Spammers use non-existent domains to achieve identity spoofing, such as 
tax.example.gov.uk 
 This is primarily a reception problem, because many recipient mail filters 
are not equipped to block this type of fraud.  However, domain owners are 
also concerned about protecting their reputation from fraudulent material 
sent using their identity.
  
 These two sender-receiver combinations are already protected from the 
problem:
  	Recipients with DMARC enforcement enabled if the message comes from a 
parent domain with DMARC-enforcing policy. 	Recipients who filter for 
non-existent domains, regardless of parent domain or receiver DMARC 
capability. 
 These three sender-receiver combinations currently have no protection:
  	Any recipient when the sender does not request DMARC enforcement (and 
non-existent domain filtering is also off.) 	Any recipient who lacks both 
DMARC enforcement capability and non-existent domain filtering.. 	Any 
recipient when the parent domain is a PSD (and non-existent domain 
filtering is also off.) 
 The DMARC for PSD standard will only protect the last category of 
unprotected recipients.
  
 What can we assume about legitimate usage of non-existent subdomains?
  	Non-existent subdomains support application-generated messages, not 
personal ones. 	Some messages will be transactional, where the recipient 
expects the message and will whitelist as necessary to ensure delivery. 
	Some message will be marketing related, as a tool to support data 
collection related to specific marketing campaigns.  The recipient probably 
does not expect the traffic, and is not likely to whitelist the traffic.   
The sender relies on the probability that most recipients do not filter on 
non-existent domains, so their non-standard business practice has minimal 
consequences for the sender. 	Possibly, some government cyber-warfare or 
cyber-intelligence operations use this feature.   Whether this is 
legitimate or not depends on the government involved.   It is unlikely that 
the recipient will consider the usage appropriate. 
 Is there justification for IETF to tolerate or tacitly permit messages 
from non-existent domains?
  	No.   Before SPF, there was no method for a sender to announce a 
transmit-only domain, but that problem was solved a long time ago.   Based 
on existing IETF standards, it is reasonable to conclude that: 	 		If a 
domain has not published either an MX record or a SPF record, then it is 
not a legitimate source of email.   Obviously, domains with no NS record 
are also included 	

 Given all of the above, I hope the real solution is clear:
  	We need to encourage recipients to block messages from non-existent 
domains (any domain that has no MX or SPF record published.)  Whitellisting 
should be used for exceptions.  Domain registration is not difficult and 
involves minimal cost, so there is no real justification for legitimate 
senders to maintain sloppy business practices. 
 Implementation methods:
  	Obtain a quorum of major mail-receiving organizations to announce that 
they will no longer accept messages from domains that are non-existent or 
not mail-enabled via an MX or SPF record  (effective no later than 
mm/dd/yyyy).  Exceptions require application on  their registered-sender 
site, with justification.   I expect that Google GMail and Microsoft 
LiveMail would be sufficient to be considered a quorum, and I assume they 
are represented on this distribution list. .The government domains could 
decide whether the best political strategy is to join the initial 
announcement or to mimic it shortly thereafter.
	  	Encourage mail filter vendors to provide non-existent domain filtering 
capability in their products, with the ability to confirm whitelisting as 
requested by the system administrator.   .The major cloud-based email 
filtering services can add this protection to their clients, if it is not 
already present in their products.   This brings another large group of 
recipients under protection 
	  	To ensure that every recipient can implement non-existent domain 
filtering, it would be beneficial if non-existent domain checking becomes 
implemented as a Reputation Block List:   When an email filter requests an 
RBL check on the domain name, the RBL checks for existence of at least one 
MX or SPF record.  If none is found a reject address is returned.  Since 
every email filtering product that I have encountered has support for RBL 
checking, this provides potential coverage for all recipients everywhere, 
and quickly.
	  	If IETF wants to provide a standards-based mechanism for mail-enabling 
a non-existent subdomain, this could be done as follow-up, using TXT 
records in the parent domain.  Once promulgated, it would only be useful 
for senders who know that their recipients have email filters which apply 
the new standard.   Outside a limited distribution group, this is not 
something that the sender can ever know.   Consequently, I do not see this 
as a particularly useful undertaking, but I have no objection to it.   
Those who do not want to depend on recipient whitelisting should register 
their domains and publish an SPF record.   
 It seems like this approach should be able to solve the whole problem in 
less than 6 months.
  
 Have I misunderstood the problem, or misunderstood your solution to it?
  
 Doug Foster

---

From: "John Levine" <[email-masked]>
Date: 7 Apr 2019 20:50:44 -0400

In article <[email-masked]> you write:
>are not equipped to block this type of fraud. ..

Right, and we can stop right there.

A decent spam filter will treat a nonexistent From: domain or envelope
bounce address as extremely suspicious and send the message into spam
folder purgatory.  If someone's filters aren't doing that, it is
unlikely that they're paying much if any attention to DMARC, and no
amount of fiddling with DMARC will make any difference.

My mail server rejects anything with a non-existent bounce address at
SMTP time and I don't think it's ever rejected anything my users would
want.

The solution to this problem is for mail systems to fix their filters,
not to invent yet another mail-breaking hack that they won't use
anyway.

R's,
John

---

From: Scott Kitterman <[email-masked]>
Date: Sun, 07 Apr 2019 21:59:33 -0400

> not to invent yet another mail-breaking hack that they won't use

Which mail breaking hack is that?  Since PSD DMARC almost entirely applies to 
domains that don't send mail, I don't think it breaks anything.  It is in part 
a tool to make hard rejects easier for receivers that don't typically reject 
solely due to non-existence and in part a tool to provide feedback to PSD 
operators so they can understand patters of abuse in their namespace.

As I understand it, rejecting mail from non-existent domains is a long 
standing, well-known tool for receivers.  I hear you saying it works for you 
in your circumstances, but that doesn't mean it scales.  Given that rejecting 
non-existent domains is a well established option, but not everyone does it, 
what basis for optimism do you have  that 'fix their filters' will change 
anything?

If fixing filters was enough, would anyone bothered to have published:

$ dig txt _dmarc.gov.uk +short
"v=DMARC1\;p=reject\;sp=none\;adkim=s\;aspf=s\;fo=1\;rua=mailto:dmarc-
[email-masked]\;ruf=mailto:[email-masked]"

All PSD DMARC would do is make that record apply to domains lower in the tree 
without their own DMARC record.  It's not that complicated.

Fielding of DMARC did a huge amount of damage to the e-mail ecosystem that I'm 
not convinced it will ever fully recover from, but PSD DMARC doesn't add to 
it.

Scott K

---

From: "Douglas E. Foster" <[email-masked]>
Date: Mon, 8 Apr 2019 07:02:26 -0400

Mr Levine brings up the valid point that there are a lot of mail filters 
with inadequate capabilities.   I determined that my two products have 
inexcusable weaknesses, so I went shopping.   
 I had only these rudimentary requirements:
  	IP filtering 	Reverse DNS filtering 	Multi-factor whitelisting or some 
other safe mechanism for handling SPF policy mistakes. 	DMARC policy 
enforcement 	A secure-email method for outbound messages with sensitive 
content 	No domain spoofing by the product that is supposed to protect from 
domain spoofing. 
 I was blown away when I discovered:
  	Products that could not do IP filtering. 	Products that could do DMARC 
enforcement but not Reverse DNS filtering, and the reverse. 	Vendors who 
had no idea what multi-factor whitelisting meant. 	High-end vendors who did 
domain spoofing in their secure-email solution.  (They have been notified.) 
   
 Along the way, I was referred to three industry-analyst reports, and was 
equally disappointed that the reports showed no evidence of a theoretical 
framework on which to judge vendor differences.  At least one analyst 
declared the industry "mature", which struck me as odd given the damage 
done by WannaCry and the obvious problems in the products I have been 
examining.
  
 I currently have exactly ONE vendor that is probably adequate, but I 
curtailed the discussion when I realized his costs were way higher than 
what I can interest management in spending.   Overall, it appears product 
vendors have no idea what is actually needed and why.
  
 Since bad email filters are the problem, why is there no IETF working 
group to define the expected behavior of email filters?    More 
importantly, can we start one NOW?
  
 I have been preparing to submit an email filter evaluation RFC as an 
individual, since this group seemed uninterested after an earlier post.   
But it would be better as a working group document, and it might become 
standards-worthy.
  
 Doug Foster

----------------------------------------
 From: "John Levine" <[email-masked]>
Sent: Sunday, April 7, 2019 8:51 PM
To: [email-masked]
Cc: [email-masked]
Subject: Re: [dmarc-ietf] Rethinking DMARC for PSDs   
In article <[email-masked]> you 

R's,
John

---

From: "Douglas E. Foster" <[email-masked]>
Date: Mon, 8 Apr 2019 07:10:49 -0400

Have the national CIRT groups made an issue about needing to block 
non-existent domains?
  
 Because a spammer can create a non-existent government agency like 
"irs.audit.gov", this email weakness becomes a national security issue and 
should be handled as a CVE.    This should get the vendors moving.  Has it 
been done?
  
 If not, perhaps Mr Levy would be willing to start the process on behalf of 
uk.gov.
  
 Doug Foster

----------------------------------------
 From: "Scott Kitterman" <[email-masked]>
Sent: Sunday, April 7, 2019 10:00 PM
To: [email-masked]
Subject: Re: [dmarc-ietf] Rethinking DMARC for PSDs   
> not to invent yet another mail-breaking hack that they won't use

Which mail breaking hack is that? Since PSD DMARC almost entirely applies 
to
domains that don't send mail, I don't think it breaks anything. It is in 
part
a tool to make hard rejects easier for receivers that don't typically 
reject
solely due to non-existence and in part a tool to provide feedback to PSD
operators so they can understand patters of abuse in their namespace.

As I understand it, rejecting mail from non-existent domains is a long
standing, well-known tool for receivers. I hear you saying it works for 
you
in your circumstances, but that doesn't mean it scales. Given that 
rejecting
non-existent domains is a well established option, but not everyone does 
it,
what basis for optimism do you have that 'fix their filters' will change
anything?

All PSD DMARC would do is make that record apply to domains lower in the 
tree
without their own DMARC record. It's not that complicated.

Fielding of DMARC did a huge amount of damage to the e-mail ecosystem that 
I'm
not convinced it will ever fully recover from, but PSD DMARC doesn't add 
to
it.

Scott K

_______________________________________________
dmarc mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/dmarc

---

From: Jeremy Harris <[email-masked]>
Date: Mon, 8 Apr 2019 12:21:17 +0100

On 08/04/2019 12:02, Douglas E. Foster wrote:
>   	[...] 	A secure-email method for outbound messages with sensitive 
> content 
Rudimentary?   How secure; what's your threat model?
Those two things often don't go together.

(I should declare an interest: Exim developer.)

-- 
Jeremy

---

From: "Douglas E. Foster" <[email-masked]>
Date: Mon, 8 Apr 2019 08:46:15 -0400

Your question confirms my core argument:   We do not have consensus around 
email filter requirements because IETF has focused on sender behavior 
rather than recipient behavior.
  
 To the specifics:
  	By "secure email relay", I am referring to Zixmail-type functioinality.  
It is complex code, generally involving a cloud server (although I would 
prefer keeping it on an appliance under my control).   It is the least bad 
option for demonstrating regulatory compliance when transmitting sensitive 
information to an ad-hoc recipient.  Most organizations need it.   Further 
discussion on this point probably belongs in the MEDUP discussion. 
 Use Cases
  	I receive hositle mail from "server7.badexample.com".   I want to block 
the IP address because I know it is a bad source.   But I also want to 
blacklist the other servers at this organization, so I want hostname 
filtering to block *.badexample.com.  I actually want to block anything 
from this organization, so I really want to block this entity anytime it 
appears in a HELO/EHLO name, a reply-to name, a list header name, a URL 
field, or a message-id field.   But for simplicity in a low-end product 
that meets my budget, I will settle for IP and ReverseDNS filtering.
	  	DMARC has three components:  detecting and enforcing sender DMARC 
policy, collecting feedback information and transmitting it to the sender, 
and interpreting feedback received for the sender domain.   In a low-end 
product, I am only expecting policy enforcement, not the other two 
components.   From a coding standpoint, it seems only slightly more complex 
than SPF enforcement or DKIM signature checking.   From a regulatory 
standpoint, it is mandatory.  If PayPal and Gmail have gone to the trouble 
to request DMARC enforcement, and I am devastated by malware because I did 
not check DMARC information,, then I am guilty of reckless negligence.   So 
yes, DMARC enforcement is essential.
	  	Multi-factor whitelisting is needed because whitelisting (give this 
message high trust because the sender is trustworthy) is only safe if the 
sender is authenticated (the message really came from the trusted sender).  
Sender authenticator requires multiple attributes.

	Example:  CompanyA.com uses HostingSevice.com.   I want to ensure that 
messages from CompanyA.com are not blocked based on spam scoring 
heuristics.   Siingle-factor whitelisting says that I can whitelist the 
sender (and expose myself to atacks from anyone spoofing their domain) or I 
can whitelist the source server (and expose myself to attacks from any 
other client domain at HostingService.com).    When ReverseDNS is not an 
option, i have no feasible way to whitelist HostingService.com, because I 
have no way to know all of their servers and keep the list maintained over 
time.   A related problem is that I should be able to do granular 
whitelisting, for example to ignore only spam scoring while retaining other 
tests.   Many of the examined products provide all-or-nothing whitelist 
mechanisms.
	  	SPF corrections are a related issue, and multi-factor whitelisting 
would be a tolerable but inferior tool for handling the problem.  What is 
really needed is a local override mechanism that can be expressed in SPF 
terms.   SPF omissions can include (a) primary domain omitted a server from 
its list, (b) included domain omitted a server from its list, or (c) an 
entire domain is omitted from the list.   Email filters should provide 
constructs to handle all three;  None of the corrections require 
whitelisting.   I do not necessarily want to guarantee delivery, I just 
want the SPF sender authentication function to be peerformed on accurate 
source data.  
 On EXIM in particular:
  
 My secondary spam filter is based on EXIM, with a vendor-supplied wrapper 
and user interface.   Because it is secondary, I am protected from some of 
the products weaknesses.
  	Other system administrators routinely complain that when the message 
From-Address is there own domain, the message is not blocked.   The product 
does not fitler on message From -Address, and my research into Exim 
capabilities indicated that the Exim filtering process could not even 
extract the message From-Address to reference in filtering.
	  	The product based on Exim cannot do ReverseDNS filtering.   It looked 
like I might be able to simulate the capability with Exim filter syntax, 
but I gave up because I could not decipher how the vendor implemented its 
wrapper.
	  	The logging is a summary of all of the chatter between sending and 
receiving systems.   The logs entries need to be assembled to present a 
single data record about what happened to this message.   The vendor's GUI 
provides a crude approximation of this process,, but the GUI information 
cannot be exported.  I finally wrote a bunch of complex code to turn the 
log detail into a message data record.
	  	The product based on Exim has only two SPF options:   Off or Block.   
When it is off, it does not check or log SPF results.  When it is on, it 
blocks the message so there is no data on which to evaluate false 
positives.   If a false positive is detected despite the difficulties, the 
product only supports single-factor whitelisting, although it does provide 
granularity by feature.
	  	Since email filtering is a heuristic process,, the message review 
features are as important as the message filtering features.  It should be 
possible to enable any filtering rule in two modes: 	 		without enforcement 
(learn mode) so that effectiveness and false positives can be evaluated and 
corrected prospectively, before the policy is enforced. 		with enforcement 
(policy mode) with sufficient data collected so that false positives can be 
detected and corrected retrospectively, after the policy is enforced. 	

 Consequently, I stand by the assertion that my feature list is the bare 
minimum for anything that claims to be a spam filter.  It is not a complete 
list.  For example RBL checking is also mandatory, but I have not seen that 
as an omitted feature, so it was omitted.   Nothing in this list requires 
proprietary databases that only the richest vendors can offer.
  
 Doug Foster

----------------------------------------
 From: "Jeremy Harris" <[email-masked]>
Sent: Monday, April 8, 2019 7:21 AM
To: [email-masked]
Subject: Re: [dmarc-ietf] Rethinking DMARC for PSDs   

--
Jeremy

---

From: "John R Levine" <[email-masked]>
Date: 8 Apr 2019 09:41:44 -0400

> importantly, can we start one NOW?

It's kind of outside the IETF's expertise -- there are people in the IETF 
(not in this WG) who believe that DNSBLs were a failed experiment in the 
1990s and nobody uses them any more.

There's also the perverse incentive that if you define the way filters 
work, you're giving bad guys a road map to avoid them.

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
Please consider the environment before reading this e-mail. https://jl.ly

---

From: "Douglas E. Foster" <[email-masked]>
Date: Mon, 8 Apr 2019 09:50:08 -0400

RBLs are alive and well.
  
 I understand the risk of helping the bad guys, but I think the evidence is 
that silence is hurting the good guys more than the bad guys. 
  
 My focus is on defining a framework for discussing product capabilities, 
while leaving room for vendors to add value above the minimum 
configuration.
  
 Demonstrating email legitimacy is essentially a protocol between sender 
and receiver.   IETF has only defined one half of the protocol, so why 
should we be surprised that the conversation is broken.   Expertise can be 
assembled; the task is desperately needed.
  
 In the interim, I am open to recommendations for good spam filters.   I 
have been trying to avoid disparaging the bad ones by name in a public 
forum.
  
 Doug Foster

----------------------------------------
 From: "John R Levine" <[email-masked]>
Sent: Monday, April 8, 2019 9:41 AM
To: "Douglas E. Foster" <[email-masked]>
Cc: [email-masked]
Subject: Re: [dmarc-ietf] Rethinking DMARC for PSDs   

Regards,

---

From: Ken O'Driscoll <[email-masked]>
Date: Mon, 08 Apr 2019 15:27:55 +0100

On Mon, 2019-04-08 at 09:50 -0400, Douglas E. Foster wrote:
[...snip...]
> while leaving room for vendors to add value above the minimum

That sounds more like what organisations such as Gartner are there for as
opposed to the IEFT. And they do a fairly good job in framing vendor
selection processes.

> should we be surprised that the conversation is broken.   Expertise can
> be assembled; the task is desperately needed.

That's not exactly true. The protocols do explain how a receiver should
implement them from a technical perspective. That ensures that there is no
divergence between how say a SPF record is interpreted on a product by
product basis. What they don't do, and what you appear to be alluding to,
is proscribe how receivers should filter spam, and more specifically forged
sender addresses from non-existent domains.

The IETF don't make such proscriptions because a) it's off-mission and b)
there is no one size fits all spam filter configuration.

What works for a small office would be unsuitable for a large corporation,
what works for a large corporation would be unsuitable for a hospital with
a similar number of users and in turn, dealing with millions of users
requires a completely different approach all together.

> have been trying to avoid disparaging the bad ones by name in a public

This list isn't the appropriate place for that discussion as it does not
pertain to DMARC, despite the fact DMARC might be baked into some spam
filtering solutions.

Your starting point should be proper functional requirements specific to
your organisation.

Ken.

---

From: Alessandro Vesely <[email-masked]>
Date: Wed, 10 Apr 2019 18:55:54 +0200

I changed the subject to make it clear that this is not about mail filtering in
general.  It is about Scott's draft.

> SMTP time and I don't think it's ever rejected anything my users would

Me too.  However, I don't reject nxdomain in From:.  I have the option, but I
disabled it because some mailing lists have (had?) authors with addresses like
[email-masked].

That said, rethinking boils down to consider if it would suffice to look up the
PSD's _dmarc record only in case of non-existing domains.  Existing domains can
be forced to publish adequate DMARC records by forging a suitable PSD's policy.
 That way, old-fashioned mailing lists can continue to use picturesque From:'s
as long as they are based on traditional TLDs (assuming .com won't publish a
strict DMARC policy.)  And no central repository.

jm2c
Ale
--