Subject: Re: [apps-discuss] Please review these 2 I-D on IPv6 and SMTP.

From: Andy Davidson <[email-masked]>
Date: Tue, 28 Jan 2014 11:35:45 +0000

Hi, Frank, apps --

> http://datatracker.ietf.org/doc/draft-martin-smtp-ipv6-to-ipv4-fallback/

Please don't implement this type of behaviour in SMTP.

- There are spam filtering techniques which have no logical equivalent in domain based reputation, for example "I wish to take action X on all mail from domestic internet access subscribers."

- We can not make the assumption that it is possible to fall back from an IPv6 transport to an IPv4 transport in our future.  The end goal, whilst distant, is to turn that old stuff off.

-- 
Andy Davidson
CTO, Allegro Networks Ltd | Connectivity You Control 
T. +44 161 200 1610       | www.allegro.net

---

From: John Leslie <[email-masked]>
Date: Tue, 28 Jan 2014 08:18:59 -0500

> Please don't implement this type of behaviour in SMTP.

   Out of context, it's hard to distinguish what "this type of behaviour"
refers to...

   As things stand now, many tools for spam-control don't work in
IPv6-land.

   While I sincerely hope we can fix that, and I'm not particularly
fond of this draft, I don't see how we can avoid some mail-servers
refusing IPv6 connections by default.

   How we progress from here deserves discussion, IMHO.

> - There are spam filtering techniques which have no logical equivalent
>   in domain based reputation, for example "I wish to take action X on
>   all mail from domestic internet access subscribers."

   True. IMHO that doesn't work too well in IPv4-land, and the problem
looks worse for IPv6...

> - We can not make the assumption that it is possible to fall back from
>   an IPv6 transport to an IPv4 transport in our future.

   True.

>   The end goal, whilst distant, is to turn that old stuff off.

   I don't expect to live that long. :^(

   (BTW, realage.com suggests that's 25 years.)

--
John Leslie <[email-masked]>

---

From: Franck Martin <[email-masked]>
Date: Tue, 28 Jan 2014 12:34:23 -0600 (CST)

As data points:
-rbldnsd now supports IPv6 since June 2013. It used to be a patch.
-Domain based blocking lists for email is sketchy at best. For instance, spamassassin does not have a single rule using domain based blocking lists. I'm not talking about URL checking, but if this email comes from that domain, I don't want it.

More inline...

----- Original Message -----
> From: "John Leslie" <[email-masked]>
> To: "Andy Davidson" <[email-masked]>
> Cc: "Franck Martin" <[email-masked]>, [email-masked]
> Sent: Tuesday, January 28, 2014 5:18:59 AM
> Subject: [apps-discuss] draft-martin-smtp-ipv6-to-ipv4-fallback
> 

Yes, it is similar to greylisting in some ways, it is making software do what they were not really designed for, but it works. Gmail tends to send by default to the spam folder non-authenticated emails received over IPv6. We do this fallback, and so far it is working well. A few people questioned why so many transfail in their logs and quickly added their IPv6 space to their SPF records (being able to send to gmail helps too).
>    How we progress from here deserves discussion, IMHO.

Yes, I prefer to have it documented, even if this may take some time to progress, or even if it is to close a door.

>    True. IMHO that doesn't work too well in IPv4-land, and the problem

I'm not sure what is the use case here but submission is better than whitelisting based on the source IP for SMTP. If you are talking about domestic internet access.

> >   an IPv6 transport to an IPv4 transport in our future.

As long as there is IPv4... and this is app specific. Web browsers fall back to IPv4 already.

I think we should not assume that the IPv6/IPv4 selection can only be solved at the network stack level. It is there for unaware applications.

>    (BTW, realage.com suggests that's 25 years.)

I'm not holding my breath either....

---

From: "John Levine" <[email-masked]>
Date: 28 Jan 2014 20:10:31 -0000

>> Please don't implement this type of behaviour in SMTP.

Please don't even try to implement this type of behavior in SMTP.
I've looked at a variety of MTAs and it simply is not possible to do
fallback from v6 to v4 in a reliable way, despite what the draft
implies.

More to the point, if people are going to upgrade their sending
software, I would much rather encourage them to sign their outgoing
mail with DKIM so v6 hosts will accept it, rather than to implement
this crock.

R's,
John

---

From: Simon Perreault <[email-masked]>
Date: Tue, 28 Jan 2014 15:26:31 -0500

> fallback from v6 to v4 in a reliable way, despite what the draft

Why?

> mail with DKIM so v6 hosts will accept it, rather than to implement

Why?

Simon
-- 
DTN made easy, lean, and smart --> http://postellation.viagenie.ca
NAT64/DNS64 open-source        --> http://ecdysis.viagenie.ca
STUN/TURN server               --> http://numb.viagenie.ca

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Tue, 28 Jan 2014 14:07:33 -0800

> mail with DKIM so v6 hosts will accept it, rather than to implement
I think the claim is that most available MTA code now does some kind of
reasonable fallback despite the ambiguity in the email RFCs.  Thus, there
wouldn't be much in the way of upgrades needed, and the point is to codify
existing behavior to ensure a coherent path forward.

What counter-evidence exists?  That would be useful information to include
here.

-MSK

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Tue, 28 Jan 2014 23:20:49 +0100

On Tuesday, January 28, 2014 11:07:33 PM CEST, Murray S. Kucherawy wrote:
> I think the claim is that most available MTA code now does some 
> kind of reasonable fallback despite the ambiguity in the email 
> RFCs.  Thus, there wouldn't be much in the way of upgrades 
> needed, and the point is to codify existing behavior to ensure a 
> coherent path forward.

"Use 4 instead of 6" isn't a way forward, it's a way backward. "Use 6 
instead of 4" is forward.

Arnt

---

From: Kurt Andersen <[email-masked]>
Date: Tue, 28 Jan 2014 14:23:12 -0800

> instead of 4" is forward.

It's only "use 4 until you can figure out how to use 6 properly" :-)

--Kurt

---

From: "John R Levine" <[email-masked]>
Date: 28 Jan 2014 17:29:23 -0500

> reasonable fallback despite the ambiguity in the email RFCs.

I've taken a look, and it's not consistent, in particular the way they 
respond to 4xx at the end of data, which is the only place a receiver can 
respond to the absence of DKIM.  You can come up with hacks that sorta 
kinda work, but they're not what I would want to standardize.  As you may 
recall from previous discussions on other lists, some MTAs will try other 
IPs, some will keep retrying the same IP that gave them the 4xx.

I suggested a grosse hackque in which the receiving MTA remembers sending 
MTAs to which it's given the back-off return code, and refuses connections 
from them for a while, to try and force them to use other MXes.  But I 
really don't think we want to go there.

More to the point, this draft suggests adding a new SMTP status code and 
ESMTP keywords, which sending MTAs would add new code to interpret.  If 
people are going to add new code to their sending MTAs, why in the world 
would you want them to do this hack, rather than just DKIM signing all 
their mail so the v6 systems will accept it, no hack required.  We already 
know how to do that.  Note that any DKIM signature will do to get MTAs to 
accept mail, no need to match the headers.

Regards,
John Levine, [email-masked], Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. http://jl.ly

---

From: Franck Martin <[email-masked]>
Date: Tue, 28 Jan 2014 17:48:49 -0600 (CST)

----- Original Message -----
> From: "Terry Zink" <[email-masked]>
> To: "Franck Martin" <[email-masked]>, "John Leslie" <[email-masked]>
> Cc: [email-masked]
> Sent: Tuesday, January 28, 2014 1:45:35 PM
> Subject: RE: draft-martin-smtp-ipv6-to-ipv4-fallback
> 
> 
> According to a Google blog post
> (http://googleonlinesecurity.blogspot.com/2013/12/internet-wide-efforts-to-fight-email.html),
> 91% of all messages today are validated with either SPF or DKIM. They don't
> say how many pass the PTR requirement, but we can use this as an estimate of
> how much email traffic it would affect. This statistic applies to number of
> emails, not to domains, so there may be a long tail of small senders who do
> not do basic email authentication.
> 

And it is in this long tail that the badness happens...

---

From: "John Levine" <[email-masked]>
Date: 29 Jan 2014 01:02:27 -0000

>And it is in this long tail that the badness happens...

Right.  It's nuts to imagine that these small senders, with their
dusty old Windows and linux boxes, would ever implement this fallback
hack.

If they'd ever upgraded their MTAs, they'd have SPF and DKIM working
already.

R's,
John

---

From: Terry Zink <[email-masked]>
Date: Tue, 28 Jan 2014 21:45:35 +0000

> Gmail tends to send by default to the spam folder non-authenticated emails received over IPv6.

Correct. Google's requirements are published here: https://support.google.com/mail/answer/81126?hl=en under "Authentication & Identification" wherein they state that the sending IP must have  PTR record and it should match the forward DNS resolution of the hostname in the PTR, and that the sending domain should pass SPF or DKIM.

>>  We can not make the assumption that it is possible to fall back from

According to a Google blog post (http://googleonlinesecurity.blogspot.com/2013/12/internet-wide-efforts-to-fight-email.html), 91% of all messages today are validated with either SPF or DKIM. They don't say how many pass the PTR requirement, but we can use this as an estimate of how much email traffic it would affect. This statistic applies to number of emails, not to domains, so there may be a long tail of small senders who do not do basic email authentication.

-- Terry

-----Original Message-----
From: apps-discuss [mailto:[email-masked]] On Behalf Of Franck Martin
Sent: Tuesday, January 28, 2014 10:34 AM
To: John Leslie
Cc: [email-masked]
Subject: Re: [apps-discuss] draft-martin-smtp-ipv6-to-ipv4-fallback

As data points:

More inline...

----- Original Message -----

I'm not holding my breath either....
_______________________________________________
apps-discuss mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/apps-discuss

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Wed, 29 Jan 2014 09:01:08 +0100

> It's only "use 4 until you can figure out how to use 6 properly" :-)

In that case it's a standstill. The way forward would be "here's what 
you're missing to use 6 properly". In someone's example yesterday, "use 
DKIM and I'll accept mail from you".

Arnt

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Wed, 29 Jan 2014 10:13:32 +0100

As an example of how that might be:

   If the server advertises the capability REJECTIONDETAILS, then certain
   bracketed keywords in 4xx and 5xx responses to MAIL FROM, RCPT TO and
   DATA mean as follows:

     [MISSINGDKIM]  The lack of a DKIM signature is (part of) the reason
                    for rejecting this transaction.

     ...

In some ways this would be like RFC 5530 (for IMAP), which has been 
surprisingly widely deployed. I like something along these lines a great 
deal better than "go back to ipv4".

Arnt

---

From: Ned Freed <[email-masked]>
Date: Wed, 29 Jan 2014 06:43:18 -0800 (PST)

> deal better than "go back to ipv4".

Completely unnecessary. All you need to do is assign an enhanced status
code (5.7.x) for this purpose.

The absolute last thing SMTP needs is a third status code field in responses.

				Ned

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Wed, 29 Jan 2014 15:57:46 +0100

> Completely unnecessary. All you need to do is assign an enhanced status

You're right of course. Stupid of me.

My point stands, though: It's better to point out the problems with the v6 
connection than to specify an RFC with a transtion from v6 to v4.

Arnt

---

From: Ned Freed <[email-masked]>
Date: Wed, 29 Jan 2014 07:04:06 -0800 (PST)

> connection than to specify an RFC with a transtion from v6 to v4.

Quite right. And did we miss an opportunity to assign a similar code for
SPF failures in the recent revision to that document? I think we did...
damn it.

				Ned

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Wed, 29 Jan 2014 16:20:58 +0100

> SPF failures in the recent revision to that document? I think we did...

Well, it doesn't cost more to publish an RFC that defines three codes (SPF, 
DKIM and PTR) than just one ;)

I'm not sure I really like it. But it definitely is better than the thing 
that started this thread.

Arnt

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 29 Jan 2014 08:18:52 -0800

I'll take this on if you two (and hopefully others) will support such a
draft with reviews and comments.

On Wed, Jan 29, 2014 at 7:20 AM, Arnt Gulbrandsen
<[email-masked]>wrote:

> Well, it doesn't cost more to publish an RFC that defines three codes
> (SPF, DKIM and PTR) than just one ;)
>

---

From: Andre Luis <[email-masked]>
Date: Wed, 29 Jan 2014 17:31:51 -0200

I think mutch profiable create RFCs for normalize the parameters of
protocols of th internet get away in the benefits of self have an Internet
regulated
I proporse an discussion for the preparation of an RFC in this one sense

2014/1/29 Murray S. Kucherawy <[email-masked]>

> On Wed, Jan 29, 2014 at 7:20 AM, Arnt Gulbrandsen <
> [email-masked]> wrote:
>
>> Ned Freed answers me:
>>

---

From: Franck Martin <[email-masked]>
Date: Wed, 29 Jan 2014 14:06:49 -0600 (CST)

----- Original Message -----
> From: "Ned Freed" <[email-masked]>
> To: "Arnt Gulbrandsen" <[email-masked]>
> Sent: Wednesday, January 29, 2014 6:43:18 AM

Agreed on the creation of Enhanced Status code for SPF, DKIM, PTR, however this is not read by MTAs, and not acted upon, this is read later by "bounce processors" to, for instance, update the status of the emails in a sending list. The MTA acts on 4xx or 5xx, I don't know any that acts synchronously on enhanced status codes.

The problem of rejecting a message (permfail) due to say failed DKIM, is that you reject a message that would have been perfectly accepted if it was sent over IPv4. The mailbox receiver now gets complains because "mum cannot send me emails", therefore the mailbox receivers disable IPv6 SMTP, or let anything in and hope for the best.

I'm not sure that receivers are willing to permfail on non spam emails because of technicalities. This draft allows to not permfails emails, but ensure mandatory domain reputation over IPv6.

I'm cool about this ID to be informational and not propose new protocols, I needed mainly to document this behavior, but after talking to some folks it was suggested the path to standard was better. 

Anyhow, I'm listening to the list traffic. We have an opportunity to kind of mandate authenticated emails over IPv6, rather than tell dummies they should authenticate their emails and hope for the best.

---

From: "John Levine" <[email-masked]>
Date: 29 Jan 2014 20:28:17 -0000

In article <CAL0qLwYSaxvs+-P48MZr0=[email-masked]> you write:
> [ draft to define "you need DKIM" and other codes ]

Sure.  At worst it's harmless, at best it'll help people diagnose
their mail problems.

Regards,

---

From: "John Levine" <[email-masked]>
Date: 29 Jan 2014 20:29:58 -0000

>Agreed on the creation of Enhanced Status code for SPF, DKIM, PTR, however this is not read by
>MTAs, and not acted upon, ...

In this case, it's read by people looking at the logs and trying to
diagnose their mail problems.

>I'm not sure that receivers are willing to permfail on non spam emails because of
>technicalities. This draft allows to not permfails emails, but ensure mandatory domain
>reputation over IPv6.

Um, why do you think MTAs can't put enhanced status codes on 4xx rejections?

R's,
John

---

From: Franck Martin <[email-masked]>
Date: Wed, 29 Jan 2014 16:23:22 -0600 (CST)

----- Original Message -----
> From: "John Levine" <[email-masked]>
> To: [email-masked]
> Cc: [email-masked]
> Sent: Wednesday, January 29, 2014 12:29:58 PM

Thanks for rephrasing

> Um, why do you think MTAs can't put enhanced status codes on 4xx rejections?

I don't:
http://tools.ietf.org/html/draft-martin-smtp-ipv6-to-ipv4-fallback-00#section-5.1

---

From: Terry Zink <[email-masked]>
Date: Wed, 29 Jan 2014 22:39:18 +0000

> Well, it doesn't cost more to publish an RFC that defines three codes (SPF, DKIM and PTR) 
> than just one ;)

Wouldn't this require only two codes for IPv6, and not three:

1) x.x.x Message does not have a PTR
2) x.x.x Message does not pass either SPF or DKIM

A message can pass SPF and not DKIM, or DKIM but not SPF, and still be accepted by the email receiver. Therefore, a third one is not required individually (one for no SPF and one for no DKIM).

-- Terry

-----Original Message-----
From: apps-discuss [mailto:[email-masked]] On Behalf Of Arnt Gulbrandsen
Sent: Wednesday, January 29, 2014 7:21 AM
To: [email-masked]
Subject: Re: [apps-discuss] draft-martin-smtp-ipv6-to-ipv4-fallback

Ned Freed answers me:
> Quite right. And did we miss an opportunity to assign a similar code 
> for SPF failures in the recent revision to that document? I think we did...
> damn it.

Well, it doesn't cost more to publish an RFC that defines three codes (SPF, DKIM and PTR) than just one ;)

I'm not sure I really like it. But it definitely is better than the thing that started this thread.

Arnt

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Thu, 30 Jan 2014 09:01:43 +0100

> 2) x.x.x Message does not pass either SPF or DKIM

Neither of them are strictly required, x.7.0 already covers any and all 
policy reasons. They're there to inform senders what their problem is. "You 
need to fix either your DKIM or your SPF, but I'm not going to tell you 
which" is, IMO, an offensively bad error message.

> A message can pass SPF and not DKIM, or DKIM but not SPF, and 
> still be accepted by the email receiver.

I don't understand your point.

Suppose a site uses a Spamhaus list, a content scanner, DKIM and SPF, and 
that it'll reject if any of the four sources say no (simplistic, I know). 
If I understand you correctly, you think the site should report "rejected 
for miscellaneous policy reasons" if either of the first two fail and 
"rejected for DKIM/SPF-related reasons" if the latter two fail. Why the 
half-generic, half-specific response in the latter case? It's neither fish 
nor fowl.

Arnt

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Thu, 30 Jan 2014 09:17:39 +0100

Franck Martin wrote:
> I'm not sure that receivers are willing to permfail on non spam 
> emails because of technicalities. This draft allows to not 
> permfails emails, but ensure mandatory domain reputation over 
> IPv6.

That's already allowed.

Consider a more well-defined failure case: An MTA has two IP addresses and 
the SPF policy for the message it wants to send allows only one of the two.

Your draft handles case where the disallowed IP address is an IPv6 address 
and the IPv4 address is allowed. But what you tell clients to do is already 
possible: The MTA can try from its other address just as easily without 
your draft as with. "Hm, I got an x.7.x failure, I wonder if that's related 
to the IP address or to the message? I'd better try again from the other IP 
address."

For that matter, it can also try from its other address if the server 
response is "your IP address fails SPF", just as if the response were "try 
from IPv4".

Arnt

---

From: "John Levine" <[email-masked]>
Date: 30 Jan 2014 16:49:45 -0000

>which" is, IMO, an offensively bad error message.

I think you're missing a key point here: v6 MTAs will generally accept
mail if it passes *either* SPF or DKIM.  So this message is telling
you that your message failed both of them, and you can fix either one
to get the mail accepted.

>that it'll reject if any of the four sources say no (simplistic, I know). 

Not simplistic, 100% completely totally unrealistic.

R's,
John

---

From: "John Levine" <[email-masked]>
Date: 30 Jan 2014 16:51:34 -0000

>the SPF policy for the message it wants to send allows only one of the two.

There's not much we can do if people insist on publishing SPF records
that don't correctly describe their mail sending policies.

R's,
John

---

From: Franck Martin <[email-masked]>
Date: Thu, 30 Jan 2014 13:30:10 -0600 (CST)

----- Original Message -----
> From: "Arnt Gulbrandsen" <[email-masked]>
> Sent: Thursday, January 30, 2014 12:17:39 AM
Allowed yes, implemented no

MTA's do not read x.7.x extended smtp codes, they read 4xx and 5xx and act on them.

This draft just throws a 421 and disconnect, telling the sender, this receiver is having existential issues, don't try it for a little while, try another one.

http://tools.ietf.org/html/rfc5321#section-4.2.3

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Thu, 30 Jan 2014 12:57:24 -0800

As I read it, Sections 4.5.4.1 and 5.1 of RFC5321 don't specify a
particular retry behavior that clients have to use in the face of a 421 or
an interrupted connection.  Specifically, "try another one" is not
necessarily how all deployed clients will respond.  I think you're only
guaranteed that if the lowest precedence MX is completely unreachable, the
client will try at least one other.

-MSK

On Thu, Jan 30, 2014 at 11:30 AM, Franck Martin <[email-masked]>wrote:

> This draft just throws a 421 and disconnect, telling the sender, this
> receiver is having existential issues, don't try it for a little while, try
> another one.
>

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Thu, 30 Jan 2014 22:42:57 +0100

Murray S. Kucherawy wrote:
> As I read it, Sections 4.5.4.1 and 5.1 of RFC5321 don't specify 
> a particular retry behavior that clients have to use in the face 
> of a 421 or an interrupted connection.  Specifically, "try 
> another one" is not necessarily how all deployed clients will 
> respond.  I think you're only guaranteed that if the lowest 
> precedence MX is completely unreachable, the client will try at 
> least one other.

I wasn't trying to say that unextended clients do the right thing. Rather, 
I tried to say that an MTA can get the same functionality as in 
draft-martin-smtp-ipv6-to-ipv4-fallback even without server extension. It 
just has to fall back to its other address(es) if the server sends x.7.x 
instead of falling back based on the draft-...-fallback extension.

Of course that requires implementation work, and of course it would lead to 
some futile retries.

Arnt