Subject: [apps-discuss] Better NDR

From: Franck Martin <[email-masked]>
Date: Tue, 27 May 2014 18:55:45 -0500 (CDT)

I was wondering, if anyone has worked or has pointers on making better NDR. 

I think the tendency nowaday is to reject emails during the SMTP transaction. The sending MTA then produces a NDR based on this SMTP one line. This email is for the consumption of the human who tried to send the email. 

Many humans cannot make any sense of this. Could we pass back something like a fully formated multipart email than a one line error? Something in plain "english"?

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Tue, 27 May 2014 16:58:58 -0700

On Tue, May 27, 2014 at 4:55 PM, Franck Martin <[email-masked]>wrote:

> I think the tendency nowaday is to reject emails during the SMTP
> transaction. The sending MTA then produces a NDR based on this SMTP one
> line. This email is for the consumption of the human who tried to send the
> email.
>
> Many humans cannot make any sense of this. Could we pass back something
> like a fully formated multipart email than a one line error? Something in
> plain "english"?
>
>
Isn't this what the text/plain part of a DSN is for?

-MSK

---

From: "John Levine" <[email-masked]>
Date: 28 May 2014 00:19:55 -0000

>Isn't this what the text/plain part of a DSN is for?

Yes, of course.

Also see https://bounce.io/

R's,
John

---

From: Jacob R Rideout <[email-masked]>
Date: Tue, 27 May 2014 20:21:06 -0400

> like a fully formated multipart email than a one line error? Something in

Take a look a Bounce.io.

---

From: Franck Martin <[email-masked]>
Date: Tue, 27 May 2014 19:30:52 -0500 (CDT)

----- Original Message -----

> From: "Murray S. Kucherawy" <[email-masked]>
> To: "Franck Martin" <[email-masked]>
> Cc: "IETF Apps Discuss" <[email-masked]>
> Sent: Tuesday, May 27, 2014 4:58:58 PM
> Subject: Re: [apps-discuss] Better NDR

> On Tue, May 27, 2014 at 4:55 PM, Franck Martin < [email-masked] >
> wrote:

> > transaction.
> > The sending MTA then produces a NDR based on this SMTP one line. This email
> > is for the consumption of the human who tried to send the email.
> 

> > like
> > a fully formated multipart email than a one line error? Something in plain
> > "english"?
> 

> Isn't this what the text/plain part of a DSN is for?

This is if the receiver decides to send a bounce as an email, with all the backscatter problems that may result. 

If the receiver gets an SMTP error "500 mailbox invalid", then the receiver needs to translate that into something readable by humans, the interpretation by the sender may loose the original meaning of the receiver. This is a simple example, but there are more complicated one. 

Gmail tries to pack the error message on several lines by using /r/n in the error text. 

Tracking what error message really means is complex, just trying to figure out if it is a hard or soft bounce, is complicated with no opensource library to do that... 

It would make more sense, if the receiver would pass a meaningful message during the SMTP transaction.

---

From: Franck Martin <[email-masked]>
Date: Tue, 27 May 2014 19:34:49 -0500 (CDT)

corrected sender vs receiver 

> If the sender gets an SMTP error "500 mailbox invalid", then the sender needs
> to translate that into something readable by humans, the interpretation by
> the sender may loose the original meaning of the receiver. This is a simple
> example, but there are more complicated one.

> Tracking what error message really means is complex, just trying to figure
> out if it is a hard or soft bounce, is complicated with no opensource
> library to do that...

> It would make more sense, if the receiver would pass a meaningful long
> message during the SMTP transaction.

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Tue, 27 May 2014 17:35:10 -0700

On Tue, May 27, 2014 at 5:30 PM, Franck Martin <[email-masked]>wrote:

> ------------------------------
>
>
> If the receiver gets an SMTP error "500 mailbox invalid", then the
> receiver needs to translate that into something readable by humans, the
> interpretation by the sender may loose the original meaning of the
> receiver. This is a simple example, but there are more complicated one.
>

Either way, a DSN is generated, whether it's by the client or the server.
I think your question reduces to how useful to an end user the text part of
that is.

> out if it is a hard or soft bounce, is complicated with no opensource

Isn't the first digit of the SMTP reply code all you need to figure out if
it's a permanent or temporary failure?

If the issue is parsing a DSN for that information, then that's the stuff
that's supposed to be in the second MIME part.  The problem is that not all
DSN generators follow the RFC.

> during the SMTP transaction.

Enhanced status codes were added to make the SMTP reply more meaningful.
Beyond that is the text part, which isn't really "safe" for standardization.

So is your question really: How do we encourage DSN generators to translate
SMTP replies and enhanced status codes into something more useful to end
users?  Or do you want something more fine-grained than the codes we
already have?

-MSK

---

From: Franck Martin <[email-masked]>
Date: Tue, 27 May 2014 19:51:23 -0500 (CDT)

----- Original Message -----

> Sent: Tuesday, May 27, 2014 5:35:10 PM

> On Tue, May 27, 2014 at 5:30 PM, Franck Martin < [email-masked] >
> wrote:

> > If the receiver gets an SMTP error "500 mailbox invalid", then the receiver
> > needs to translate that into something readable by humans, the
> > receiver.
> > This is a simple example, but there are more complicated one.
> 

> Either way, a DSN is generated, whether it's by the client or the server. I
> think your question reduces to how useful to an end user the text part of
> that is.

yes, but then you know there is html too... The marketing types, prefer html... 

> it's a permanent or temporary failure?

permanent/temporary failures is different of soft/hard bounce. 

"500 Your email contains spam" is a permanent failure but a soft bounce. 

> DSN generators follow the RFC.

you think asynchronous bounce. bounce from the receiver. The sender is supposed to generate a nice message for the user based on the receiver hint. You can do like bounce.io and intercept this email and rewrite it for the end user... but this is not what I'm talking about 

> users? Or do you want something more fine-grained than the codes we already
> have?

The second part ;) 

Crudely: 
... 
C: /r/n./r/n 
S: 500 5.7.0 email contains spam 
S: 500 Content-type: text/html;charset:UTF-8 
S: 500 <h1> email system at example.com</h1> 
S: 500 <p> Thanks for emailing us but blah... </p> 
.....

---

From: Ned Freed <[email-masked]>
Date: Tue, 27 May 2014 17:51:36 -0700 (PDT)

> Isn't this what the text/plain part of a DSN is for?

It doesn't have to be a text/plain. It can be a text/html, multipart/related,
multipart/alternative, etc.

But also note that the second part of a DSN or MDN contains the information
necessary for the user agent to provide its own "view" of the report. This 
is supposed to allow for internationalization, although in practice few
if any user agents actually bother.

				Ned

---

From: Ned Freed <[email-masked]>
Date: Tue, 27 May 2014 17:58:09 -0700 (PDT)

> backscatter problems that may result.

And pray tell how do you think reports of nondelivery are provided for
legitimate emails rejected by ingress MTAs? The egress MTA on the other end
generates a DSN, that's how. 

I don't know where this notion that backscatter means DSNs are bad in all
circumstances has come from, but it's nonsensical. There are places where it's
not only completely appropriate for a DSN to be generated, it's essential to
the proper operation of the email service.

Indeed, the only place were the user agent gets something other than a full DSN
as a response is when an error arises during submission. (And in some cases not
even then.) In this case it's up to the MSA to generate a sensible message and
for the user agent to place that error message in the proper context.

> needs to translate that into something readable by humans, the interpretation
> by the sender may loose the original meaning of the receiver. This is a simple

Well, first, I'd argue that "mailbox invalid" is a poor message choice for an
SMTP error. "There's no user here with that address", "No such user", even
"unknown address" would be a lot better. And since such messages are sometimes
displayed without proper context, it's best to include the address you're
talking about.

But this ignores the ability to put additional explanatory text in the DSN. Or,
in the case of SUBMIT, for the user agent to provide it.

> Gmail tries to pack the error message on several lines by using /r/n in the

SMTP has always allowed multiline respsonses. Their handling is another matter,
and so is misuse of multiline responses to provide "formatted" responses.

> out if it is a hard or soft bounce, is complicated with no opensource library
> to do that...

The main problem here is an implementation quality issue. If whoever generates
the SMTP error botches it and puts in crappy text like the example you gave and
either doesn't put in enhanced status code or puts in an unnecesarily vague
one, if whoever generates the DSN does a poor job of it, either by using a
nonstandard format or by barely meeting the requirements for the standard one,
and if the user agent that shows the error to the user either handles the DSN
badly or fails to handle a submit error well, things don't work well.

There are a lot of levels where things can go wrong. But there are also a lot
of levels for things to get fixed up.

Beyond that, well, the existence of bounce.io kind of shows this is a
nontrivial issue.

But beyond the need to define additional enhanced status codes, which we've
been somewhat remiss in doing, I'm not seeing an IETF issue here.

> during the SMTP transaction.

Exactly.

				Ned