Subject: [apps-discuss] Multipart/report, draft-kucherawy-rfc3462bis-01.txt

From: Alessandro Vesely <[email-masked]>
Date: Wed, 27 Jul 2011 15:34:00 +0200

On 27/Jul/11 07:26, [email-masked] wrote:
> A New Internet-Draft is available from the on-line Internet-Drafts directories.
> 	Filename        : draft-kucherawy-rfc3462bis-01.txt

The new paragraph tries to explain a difficult concept:

 User agents and gateways need to be able to determine automatically
 that a message is a mail system report and need to be processed as
 such.  Placing the multipart/report as the outermost content provides
 a mechanism whereby an auto-processor can detect through parsing the
 message header that the message is a report.  Therefore, when used to
 generate a report, the multipart/report content-type MUST be the top-
 level MIME content type for any report message.  However, when
 forwarding or otherwise packaging a previously-generated report, this
 constraint is not in effect.

I'm puzzled about the "and gateways" part.  Is it correct?  IMHO, it
is just the MUA, or possibly the MDA, who may want to relate a DSN to
the message that triggered it.

In general, there is a relationship between the address that a report
is being delivered to and and its "active" status, expressed by having
multipart/report as the topmost MIME entity.  In order to process
reports, an agent must assume that the delivered-to address is the one
featured in smtp.mailfrom (for DSN) or in some FBLs/abuse-mailboxes
(for ARF).  To avoid fooling agents, reports should be wrapped inside
some other MIME entity when they are sent to other addresses (Ned said
"Indeed, I would actually like to require such
things NOT be sent at top level.")

Question: in case an ARF processor forwards an abuse (mis)report to
the original author, is it correct to keep multipart/report as the top
Content-Type?

jm2c

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 27 Jul 2011 06:47:27 -0700

> -----Original Message-----
> From: Alessandro Vesely [mailto:[email-masked]]
> Sent: Wednesday, July 27, 2011 6:34 AM
> To: Murray S. Kucherawy
> Cc: apps-discuss
> Subject: Multipart/report, draft-kucherawy-rfc3462bis-01.txt
> 

"User agents and gateways" is copied from the existing RFC.  It's not part of what I'm seeking to change here.  I can change it if there's consensus to do so, of course.

> the original author, is it correct to keep multipart/report as the top

That's exactly what this change is attempting to address.  The old text disallows forwarding of multipart/report altogether, other than re-injecting it into the mail stream with exactly the same MIME structure it had originally; that is, it can't be wrapped inside multipart/anything or message/rfc822.  It's that restriction that's being removed.

However, this morning I realized that, in the procedural shuffle in the last few days, this change has lost sight of the original goal which is to enable the origination of multipart/reports without the restriction altogether.  The MARF working group is seeking to converge OMA's SpamRep specification (which has its own media type that can contain many reports) with its own, the latter of which uses multipart/report.  Therefore, this change is a step in the right direction but it doesn't get all the way there.

It's my understanding that Keith would like the text that's in this draft or something close to it, Ned would be satisfied with either outcome, but MARF needs to go further.  So that's where we are now.

I guess then I'd like Keith to comment on this idea: What about changing the MUST in that paragraph to a SHOULD?

-MSK

---

From: [email-masked]
Date: Wed, 27 Jul 2011 15:55:55 -0700 (PDT)

> >  constraint is not in effect.

This is not acceptable as far as I'm concerned. For one thing, it uses
various new terms like "mail system report" and "report message" that
AFAIK aren't well defined. And even if I assume definitions for these terms
I think the author probably meant, the result still isn't what I'd like to see.

Multipart/report is, or should be, a very general construct used to report
pretty much anything relating to messages or perhaps even more general MIME
content.

The restriction that generated DSNs and MDNs (two specific types of report)
need to be placed at the top-level is what's needed here. Nothing more. We
should not attempt to guess at possible future uses of the report framework.
This sort of anticipatory restriction is what gets us into trouble with great
regularity, and we need to stop doing it.

In fact the best way to handle this would be to put the restriction into the
DSN and MDN specifications that build on the multipart/report framework, and
say something like, "Uses of the multipart/report container format MAY restrict
the contexts in which a particular report type can appear."

> > I'm puzzled about the "and gateways" part.  Is it correct?

Yes. A proper gateway to X.400, for example, will turn a top-level DSN into an
X.400 delivery report and vice versa. (It would nice if the same applied to
MDNs and X.400 read receipts, but the semantics are so dissimilar that
conversion is infeasible.)

> > IMHO, it

Sure, but not all user agents operate in the Internet email world. This way
X.400 user agents (and like it or not, these still exist) get the report in a
format they can actually process.

> "User agents and gateways" is copied from the existing RFC.  It's not part of
> what I'm seeking to change here.  I can change it if there's consensus to do
> so, of course.

> That's exactly what this change is attempting to address.  The old text
> disallows forwarding of multipart/report altogether, other than re-injecting it
> into the mail stream with exactly the same MIME structure it had originally;
> that is, it can't be wrapped inside multipart/anything or message/rfc822.  It's
> that restriction that's being removed.

It's a little more subtle than that. See above.

> However, this morning I realized that, in the procedural shuffle in the last
> few days, this change has lost sight of the original goal which is to enable
> the origination of multipart/reports without the restriction altogether.  The
> MARF working group is seeking to converge OMA's SpamRep specification (which
> has its own media type that can contain many reports) with its own, the latter
> of which uses multipart/report.  Therefore, this change is a step in the right
> direction but it doesn't get all the way there.

I'm not seeing a problem with MARF and the change I'm proposing.

> It's my understanding that Keith would like the text that's in this draft or
> something close to it, Ned would be satisfied with either outcome, but MARF
> needs to go further.  So that's where we are now.

> MUST in that paragraph to a SHOULD?

I'm entirely against that. It would relax the conditions on generating
DSNs and MDNs and I do not think we should be doing that.

				Ned

---

From: Tony Hansen <[email-masked]>
Date: Thu, 28 Jul 2011 00:27:20 -0400

On 7/27/2011 9:34 AM, Alessandro Vesely wrote:
>   User agents and gateways need to be able to determine ...
>

Gateways are odd beasts: they often act like both MUAs and MDAs.

     Tony Hansen
     [email-masked]

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 27 Jul 2011 21:32:56 -0700

> From: Ned Freed [mailto:[email-masked]] On Behalf Of ned+apps- [email-masked]
> Sent: Wednesday, July 27, 2011 3:56 PM
> To: Murray S. Kucherawy
> Cc: Alessandro Vesely; apps-discuss
> Subject: Re: [apps-discuss] Multipart/report, draft-kucherawy-rfc3462bis-01.txt
> 

You might be right that they're not well defined, but they're not new.  They're copied directly from RFC3462.

> the contexts in which a particular report type can appear."

I'm happy to take those on as well as individual submissions in support of the MARF work, assuming such work is limited to these small changes doesn't become a kitchen sink for a massive collection of updates to these three RFCs.  If it becomes a bigger update project, I think we're talking about something YAM-like, or at least something that should land in APPSAWG.

And to Barry and/or the ADs: The RFCs we're talking about are all Draft Standard.  Could these changes be considered clarifications and not require recycles at Proposed Standard, or do we doom them to that by making these changes?

-MSK

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Thu, 28 Jul 2011 08:26:19 -0700

> From: [email-masked] [mailto:[email-masked]] On Behalf Of Murray S. Kucherawy
> Sent: Wednesday, July 27, 2011 9:33 PM
> To: [email-masked]
> Cc: apps-discuss
> And to Barry and/or the ADs: The RFCs we're talking about are all Draft
> Standard.  Could these changes be considered clarifications and not
> require recycles at Proposed Standard, or do we doom them to that by
> making these changes?

...and RFC3464 is showing updated by two other documents.  I imagine we'd have to sort out how to resolve all of that, so maybe this is a more heavyweight change than I'd hoped.

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Thu, 28 Jul 2011 11:01:27 -0700

> I'm happy to take those on as well as individual submissions in support
> of the MARF work, assuming such work is limited to these small changes
> doesn't become a kitchen sink for a massive collection of updates to
> these three RFCs.  If it becomes a bigger update project, I think we're
> talking about something YAM-like, or at least something that should
> land in APPSAWG.

Well, I started down this road and was delighted to find this sort of text in both the DSN and MDN documents:

3.  Format of a Message Disposition Notification

   A message disposition notification is a MIME message with a top-level
   content-type of multipart/report (defined in [RFC-REPORT]).  When
   multipart/report content is used to transmit an MDN: ...

So it seems removing the restriction entirely from RFC3462 is fine, because it's basically repeated here.  Hopefully that can just proceed then, and enables progress for MARF.

The remaining issue, I think, is what Keith wants to see, which is an assertion that at generation time the above is required, but not when an existing one is forwarded inside a multipart.  I believe that will require only a slight wording change to DSN and MDN text shown above, but will also thus require publishing an update (clarification?) to two Draft Standards, one of which has updates from other documents.

---

From: Ned Freed <[email-masked]>
Date: Thu, 28 Jul 2011 11:10:02 -0700 (PDT)

> ...and RFC3464 is showing updated by two other documents.  I imagine we'd
> have to sort out how to resolve all of that, so maybe this is a more
> heavyweight change than I'd hoped.

I have to ask: So what? So let's say we have to do a 3462bis that resets
things to proposed.

Why is this a big deal? We do it, six months later we advance the
specification back to draft - this should be easy since we only need
to review the consequences, if any, of the change we're making.

				Ned

P.S. It would be a little more difficult if revisions to the DSN and MDN
specs were needed, but that does not appear to be the case.

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Thu, 28 Jul 2011 11:18:53 -0700

> From: Ned Freed [mailto:[email-masked]]
> Sent: Thursday, July 28, 2011 11:10 AM
> To: Murray S. Kucherawy
> Cc: apps-discuss

Maybe my impression that recycling is something we prefer to avoid is more inflated than it should be.  If that's the case, then it looks like this is the path forward, at least for 3462.

If we think the DSN and MDN specs are fine as-is and don't need stronger language about the format used at generation time, then I guess we're close to done other than initiating publication.

---

From: Dave CROCKER <[email-masked]>
Date: Thu, 28 Jul 2011 14:22:46 -0400

> to review the consequences, if any, of the change we're making.

The transaction cost of having an additional process through the IETF/IESG is 
never trivial.

I know you know this; I don't know why you think it's a small deal.

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

---

From: Alessandro Vesely <[email-masked]>
Date: Thu, 28 Jul 2011 21:19:36 +0200

On 28/Jul/11 00:55, [email-masked] wrote:
> need to be placed at the top-level is what's needed here. Nothing more.

IMHO "generated" is not very clear here.  I roughly understand it as
referring to the act of issuing a report, automatically or not.

BTW, ARF reports, as currently used, probably have to be at top level
as well.

> the contexts in which a particular report type can appear."

Does that mean top-level implies "active"?

> [moved] We should not attempt to guess at possible future uses of
> the report framework. This sort of anticipatory restriction is what
> gets us into trouble with great regularity, and we need to stop
> doing it.

In general, I agree.  But here we are introducing a split, or
categorization, to distinguish "active", "generated", or "top-level"
from other kinds of report transmission.  How can we do so without
affecting future report types as well?

> conversion is infeasible.)

I'm not familiar with X.400, but this turns out to be a clarifying
element.  Top-level reports may be converted.  In pseudo-code

  if (/^Content-Type: *multipart\/report.*report-type=([:value:]+)/ih)
  // assume h stands for header only, and :value: catches token or
  // quoted-string appropriately
  {
    switch ($1)
    {
      case "delivery-status":
      // convert
      // ...
    }
  }

The task of preserving gateway functionality allows to define the
behavior without shredding much more light on the semantics of
"top-level" vs. "wrapped".

>>> the original author, is it correct to keep multipart/report as the top

I'm sorry I badly expressed this point.  Consider that the original
author probably has a copy of the reported message in her "Sent"
folder.  In this particular case, an agent could do some meaningful
work with reports.  Would it make sense to send the report as
"active", so as to trigger any agent that may process it?

jm2c

---

From: Ned Freed <[email-masked]>
Date: Thu, 28 Jul 2011 14:40:56 -0700 (PDT)

> The transaction cost of having an additional process through the IETF/IESG is

Actually, I can point at any number of cases where it was pretty painless. In
particular, small revisions to existing widely deployed standards seem to fare
pretty well. Of course there are exceptions, like the revision to the Sieve
base spec, where someone gets a hair up their butt and files a totally
inappropriate DISCUSS, but one things for sure: Doing nothing guarantees
nothing will be done.

> I know you know this; I don't know why you think it's a small deal.

Perhaps because it *should* be a small deal. And to the extent it isn't, I'm
not sure I see the value in trying to accomodate brokenness.

				Ned

---

From: Barry Leiba <[email-masked]>
Date: Thu, 28 Jul 2011 18:01:46 -0400

> not sure I see the value in trying to accomodate brokenness.

At least part of Dave's point, setting aside issues related to
complaints and over-tweaking and DISCUSS ballots, is that there's a
*real* cost to getting any document through that involves:

-- Someone taking on the task of editing the document and making the
case for it.
-- A document shepherd and reviewers to make sure it's ready to go.
-- An AD's time to manage the process.
-- An IETF-wide last call.
-- Usually, several directorate reviews; at least GenART and SecDir
for everything.
-- IESG review, which requires ballots (and, therefore, presumably,
review) from at least ten ADs.
-- Time on a telechat, even if it's only a minute or two.
-- Review by IANA, and communication with the authors.
-- RFC Editor processing, which is never insignificant.

I might have missed something, but I think I got most of it.  Even the
most lightweight version of all that is more expensive than most of us
think about.

Barry

---

From: Ned Freed <[email-masked]>
Date: Thu, 28 Jul 2011 15:05:32 -0700 (PDT)

> most lightweight version of all that is more expensive than most of us

No, it's exactly as expensive as I assumed it would be.

But given that this is a draft standard and the rules say removing a
restriction means recycle at proposed, what alternative would you suggest?
Creating a new document that removes the restriction, getting that approved at
proposed, then moving that to draft, then revising at  draft is actually more
steps and more work. Or perhaps you would prefer to make a nice big mess by
creating a new subtype like, say,
multipart/report-without-restriction-because-IETF-processes-are-too-heavy-to-fix-stuff?

I'm sorry, but tying ourselves into knots trying to avoid doing the right
thing is expensive in its own right - vastly more expensive IMNSHO than
the process you're trying to avoid.

				Ned

---

From: Tony Hansen <[email-masked]>
Date: Thu, 28 Jul 2011 18:41:07 -0400

The MDN spec needs an update for other reasons. Now that YAM is dead, I 
might be persuaded to crack it open.

     Tony

On 7/28/2011 2:01 PM, Murray S. Kucherawy wrote:
> _______________________________________________
> apps-discuss mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/apps-discuss

---

From: Ned Freed <[email-masked]>
Date: Thu, 28 Jul 2011 15:41:04 -0700 (PDT)

> Does that mean top-level implies "active"?

You know, I was thinking about that as well. It's clear than active DSNs and
MDNs have to be top-level but that doesn't mean the converse is necessarily
true.

Operationally at least, I think the answer is something along the lines of,
"it's active if it correlates with a message you sent, otherwise not". 

I'm not sure if this warrants mention in the spec or not.

> affecting future report types as well?

Well, the obvious thing is to say that all report types need to specify
the contexts in which they can appear and what they mean in those contexts.

> I'm not familiar with X.400,

Consider yourself fortunate in that regard.

> but this turns out to be a clarifying

>   // quoted-string appropriately

Yep, that look about right. It also has to be outermost level because in X.400
a delivery report is not, properly speaking, a message, so you cannot include
other message-ish stuff in it.

> "active", so as to trigger any agent that may process it?

I don't see why not, but then again I'm not entirely clear on all the ins
and outs of ARF usage.

				Ned

---

From: Alessandro Vesely <[email-masked]>
Date: Sat, 30 Jul 2011 12:39:57 +0200

> MDNs have to be top-level but that doesn't mean the converse is necessarily

Plus: it would allow an ABNF-like definition of "active",
minus: inactive messages would then have to be wrapped.

> "it's active if it correlates with a message you sent, otherwise not". 

+1, it clarifies what reporting is all about, linking the active
status with the destination address (and is quite complaisant to data
protection principles in this respect.)  Let me try to reword the
first paragraph of Section 3:

   The multipart/report MIME media type is a general "family" or
   "container" type for electronic mail reports of any kind.  A mail
   report presents an account of events occurred to another mail
   message, which is called the original message, returned message, or
   reported message.  A report message, i.e. a message carrying a mail
   report, is considered "active" when it is destined to an entity
   who actively contributed to the sending of the original message,
   including authors, relays, gateways, delivery agents, and link
   operators.  Mail processing programs can benefit of using a single
   media type for all kinds of reports, and detect active reports
   correlated with sent messages.

> I'm not sure if this warrants mention in the spec or not.

Nor I.  It may ease the wording of the top-level requirement, though.

jm2c

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sat, 30 Jul 2011 21:22:22 -0700

> From: [email-masked] [mailto:[email-masked]] On Behalf Of Alessandro Vesely
> Sent: Saturday, July 30, 2011 3:40 AM
> To: [email-masked]
> [...]

I think this is better stated in the DSN and MDN updates we're now considering, and maybe in ARF if we decide to update it.  It's not clear that all uses of multipart/report forever and ever will need to make this distinction, and we're talking about removing the top-level restriction from it entirely now anyway.

It sounds like the path forward is to remove the restriction from multipart/report altogether, which is safe because it's already stated on MDN and DSN as well.  This would satisfy MARF's needs.  We can then look at clarifying DSN and MDN if/when we feel doing that would be useful.