Subject: [EAI] Minutes from Prague

From: Harald Tveit Alvestrand <[email-masked]>
Date: Fri, 20 Apr 2007 22:48:57 +0200

Due to my inattention, I have let this slip right until the submission 
deadline.
So the attached file has been submitted as minutes without review from the 
WG.

My apologies.

          Harald Alvestrand
_______________________________________________
IMA mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ima

---

From: "Charles Lindsey" <[email-masked]>
Date: Tue, 24 Apr 2007 10:48:23 +0100

Seems I sent this just to Harald, and not the list, which explains why he  
replied to me only :-( .

------- Forwarded message -------
From: "Charles Lindsey" <[email-masked]>
To: "Harald Tveit Alvestrand" <[email-masked]>
Subject: Re: [EAI] Minutes from Prague
Date: Mon, 23 Apr 2007 11:34:50 +0100

> So the attached file has been submitted as minutes without review from  

And from the minutes:

..................
DSN
draft-ietf-eai-dsn-00
---------------------

Issue: Should message encoding be application/utf8smtp or message/utf8smtp
Discussion: MIME allows 8-bit types, but explicitly prohibits 8-bit message
types for message/*
Argument in favor of message/utf8smtp: the object is a message.
Argument against message/utf8smtp: may break 7-bit implementations
Argument for application/utf8smtp: will work with everything, since
application/<unknown> is supposed to be treated as application/octet-stream
Argument against application/utf8smtp: automated DSN processors might barf
on 8-bit content
Argument against (soft) message/utf8smtp: the use will leak out, e.g.,
multipart/digest
Resolution: do message/utf8smtp; fix broken implementations (if any)
...................

I don't understand the proposed resolution here. It violates a very
explicit "EXPRESSLY FORBIDDEN" in RFC 2045, so you can hardly describe an
implementation that does not expect to see 8bit where 8bit is expressly
forbidden as "broken" and "to be fixed". So what am I missing?

And the dicussion completely failed to consider the third possibility of
using message/rfc822, downgraded as necessary before it is allowed to be
seen in the non-UTF8SMTP universe (but presumably upgraded again for
display by recipients so capable).

Moreover, since we seemed to have abandoned any HDR=UFT8SMTP parameter in
SMTP, downgraders are going to have to recognize and deal with any 8bit
headers contained within a message/rfc822, just as they are going to have
to recognize and deal with them within mime body headerts within any
multipart.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131                       
   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

---

From: Frank Ellermann <[email-masked]>
Date: Tue, 24 Apr 2007 15:51:03 +0200

Charles Lindsey wrote:

> downgraders are going to have to recognize and deal with any 8bit
> headers contained within a message/rfc822

By definition there are no 8bit header fields in a message/rfc822,
downgraders and others would look for UTF-8 in header field bodies
if they need or wish to identify a superset of the message/rfc822
format dubbed as message/utf-8.

And John apparently wants to limit this to some precisely defined
places in known header fields, instead of taking the simpler "any
UTF-8 can be fine for message/utf-8, nobody cares about the syntax
in transit" approach.

What you're talking about is apparently the UNKNOWN-8BIT seen in
"some" (syntactically invalid) message/rfc822 today, without EAI.

We've to solve the first problem, valid message/utf-8.  We're not
forced to solve the second problem, UNKNOWN-8BIT in a header of
something that's certainly syntactically invalid, with or without
EAI.  Implementation details, a harsh recipe is "reject", but it
might be the best solution.  A really *odd* recipe would be "just
turn off the 8th bit", IMO that deserves a MUST NOT.  And a very
sophisticated downgrade implementation might try to preseve any
UNKNOWN-8BIT somehow, for critical header fields where it knows
the structure (otherwise using RFC 2047 could fail miserably).

> just as they are going to have to recognize and deal with them
> within mime body headerts within any multipart.

There is no UNKNOWN-8BIT in valid MIME part headers.  There are
also no addresses in MIME part headers, or rather I'm not aware
of any addresses in any Content-* header field, unless you'd put
them in a comment.  So why is this a topic for this WG (= EAI) ?

Frank

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Thu, 26 Apr 2007 23:02:42 +0200

> the structure (otherwise using RFC 2047 could fail miserably).

I believe that both PP and Exim defaulted to "strip the 8th bit" for quite 
a while. I'm not sure what current mailers default to.

            Harald

---

From: "Charles Lindsey" <[email-masked]>
Date: Fri, 27 Apr 2007 14:14:50 +0100

> format dubbed as message/utf-8.

No existing 8BITMIME downgrader is going to "look for UTF-8 in header  
field bodies" whether it is dubbed as message/utf-8 or as  
message/anything-else, so such things must never be seen outside the  
UTF8SMTP universe.
> UTF-8 can be fine for message/utf-8, nobody cares about the syntax

I think you would have to accept whatever our utf8headers document allows  
- no more and no less, so that excludes any UNKNOWN-8BIT (so garbage-in ->  
garbage-out would apply, since detecting that case is more trouble than  
any expected benefit).

> them in a comment.  So why is this a topic for this WG (= EAI) ?

Indeed. The only things currently proposed to be allowed in Content-*  
headers are UTF-8 in <comment>s (rare, but possible) and UTF-8 on the RHS  
of <paramater>s (for which RFC 2231 can be used).

---

From: Frank Ellermann <[email-masked]>
Date: Sat, 28 Apr 2007 09:45:51 +0200

Harald Tveit Alvestrand wrote:

> I believe that both PP and Exim defaulted to "strip the 8th bit"
> for quite a while. I'm not sure what current mailers default to.

Yesterday I saw that it's a MAY in RFC 2821, may strip MSB, or may
reject the message as FUBAR.

For my application/message idea I thought about how that could be
done by a an 8to7-script:  The input is an 8BITMIME message/rfc822.

The algorithm 8to7( x ) would work something like this:

1: Split x into header and body
2: Strip 8th bit in the header (it's supposed to be ASCII)
3: Check if content-type is multipart/* (5) or message/* (6)

4: non-composite part
4.1: For no CTE:   strip 8th bit in the part, return
4.2: For CTE 7bit: strip 8th bit in the part, return
4.3: For CTE B64:  strip 8th bit in the part, return
4.4: For CTE QP:   strip 8th bit in the part, return
4.5: For CTE 8bit replace CTE B64, encode and return
4.6: For CTE binary or unknown CTEs abort with error

5: multipart/*
5.1: For CTE binary or unknown CTEs abort with error
5.2: For CTE B64 or QP abort with error <= EXPRESSLY FORBIDDEN
5.3: For CTE 7bit: strip 8th bit in the part, return
5.4: For CTE 8bit replace CTE 7bit
5.4.1: strip 8th bit in the MIME part header
5.4.2: For each part do
5.4.2.1: for a multipart use the procedure in step 5, iterate
5.4.2.2: for message/rfc822 use procedure for step 1, iterate
5.4.2.3: if it's a non-composite part (no message/*)
5.4.2.3.1: strip 8th bit in the MIME part header
5.4.2.3.2: use procedure in step 4, iterate (next part at 5.4.2)
5.4.2.4: convert message/* to an application/message, iterate

5.5: No more parts in the multipart/* at 5.4.2, return

6: message/*
6.1: For CTE binary or unknown CTEs abort with error
6.2: For CTE B64 or QP abort with error <= EXPRESSLY FORBIDDEN
6.3: For CTE 7bit: strip 8th bit in the part, return
6.4: for a message/rfc822 use the procedure in step 1, return
6.5: convert message/* to an application/message, return

The "conversion" to application/message could be to grab all
lines between the boundaries, B64 encode them as is, insert
a MIME part header Content-Type: application/message, CTE B64,
no parameter, no other Content-*, no UTF-8 comments ;-), keep
the encoded original message as "body" of this part, ready.

If minimal MIME conformance requires to support message/partial
and/or message/external-body add the missing steps.  Shall we
try this application/message idea as special EAI contribution
to the existing solutions for "8BITMIME to 7bit" gateways ?

Frank

---

From: "Charles Lindsey" <[email-masked]>
Date: Mon, 30 Apr 2007 11:58:07 +0100

> reject the message as FUBAR.

Clearly, bouncing the message as FUBAR is always the safe option.

But I think it better, if you are determined to try to continue with  
delivery, just to send the 8bits regardless rather than to strip the MSB  
(this is what sendmail does AFAICT).

Stripping the MSB clearly damages the message, certainly creating garbage  
of what may or may not have been garbage in the first place. Sending the  
8bits regardless may well result in garbage at the far end (the far end  
might even truncate the MSB). But the garbage seen in this case will  
certainly be no worse than the garbage seen in the strip-MSB case (it may  
even show up as the same garbage). But there still remains a (small)  
possibility that it will show up as something sensible, or at least  
something that a determined recipient can disentangle manually.

So it is a choice between being definitely worse off and probably/possibly  
being worse off.

And it is strictly of-topic for this group, though possibly interesting  
for understanding the environment in which we have to operate.

---

From: John C Klensin <[email-masked]>
Date: Mon, 30 Apr 2007 14:35:27 -0400

--On Monday, 30 April, 2007 11:58 +0100 Charles Lindsey
<[email-masked]> wrote:

>>> I believe that both PP and Exim defaulted to "strip the 8th
>>> bit" for quite a while. I'm not sure what current mailers
>>> default to.
>> 
>> Yesterday I saw that it's a MAY in RFC 2821, may strip MSB,
>> or may reject the message as FUBAR.
> 
> But I think it better, if you are determined to try to
> continue with delivery, just to send the 8bits regardless
> rather than to strip the MSB (this is what sendmail does
> AFAICT).

But this is exactly the "just-send-8" option that was rejected
during the original MIME work, rejected again when MIME advanced
to Draft, and rejected again by DRUMS (and maybe a few other
times).  Even if you were right, it would be too late.

>....
> And it is strictly of-topic for this group, though possibly
> interesting for understanding the environment in which we have
> to operate.

So, why are you bringing it up here?  And why am bothering to
respond to it, rather than considering it an indication that I
should start filtering your mail out as frequently containing
off-topic discussions?  (You can, and probably should, treat
both questions as rhetorical.)

       john