Subject: Re: UTF-8 and literal packets

From: "Hal Finney" <[email-masked]>
Date: Wed, 24 Mar 2004 09:38:50 -0800

Jon Callas writes:
> We would like to have a modification to the literal packet, where there 
> is a type 'u' packet which is identical to the type 't' packet except 
> that in this packet the implementation is saying, "By gum, I *know* 
> this really, really contains UTF-8 in it. Trust me. Really."

So how would we define appropriate behavior with respect to the
't' and 'u' packets, on receipt and on creation?  Are we deprecating
't' on creation, and we SHOULD use a 'u' (and UTF-8 of course)?
And then on receipt, what should (or SHOULD) we do?

Hal

---

From: Werner Koch <[email-masked]>
Date: Wed, 24 Mar 2004 20:08:40 +0100

> And then on receipt, what should (or SHOULD) we do?

I'd say don't deprecate anything, the 'u' is merely a flag that the
message is for sure utf-8 encoded.  It's up to the MUA[1] to create
and display a message correctly, that flag might be helpful.  Maybe a
"If it is definitely known that the message is utf-8 encoded, the 'u'
flag SHOULD be used."

  Werner

[1] or better up to the text editor or whatever OpenPGP aware
    application, as MUAs should use PGP/MIME

---

From: David Shaw <[email-masked]>
Date: Wed, 24 Mar 2004 14:49:39 -0500

> And then on receipt, what should (or SHOULD) we do?

The way I look at it is that the rule for 't' is to take text and
canonicalize the line endings.  The rule for 'u' is to take text and
canonicalize the line endings and the encoding.  When receiving a
message of either sort, you decanonicalize, doing whatever is
appropriate for your platform.

If possible, use 'u'.  If you can't, whether for lack of information
on the original character set, or even just a minimal implementation
that doesn't have character set recoding ability, then use 't'.  't'
should not be deprecated.

David

---

From: Jon Callas <[email-masked]>
Date: Wed, 24 Mar 2004 15:03:34 -0800

I would rather have the description in the RFC say what it means, 
rather than what to do with it. There are a number of reasons for that, 
chief among them that we don't have all the answers.

I'm working now on an OS with native support for umpteen character sets 
and encodings, one of which is UTF-8. There's a "Text Encoding" menu so 
I can select one should things not be displayed correctly. In my case, 
my MUA has the option of setting a property on the window as to what 
character set it is, and then it's all magically handled.

Let's also not forget that while OpenPGP gets used a whole lot for 
mail, it is an object security standard, not a mail security standard.

I can conceive of instances where whatever agent handles the text, it 
may know it's not UTF-8, but have no idea what it is, nor how to 
canonicalize. In that case, it should do the best job it can. I can 
also conceive of applications that may not have a good way to 
canonicalize, either. In any event, we are not wise enough to tell the 
implementer what to do.

What it means simply is that when you get such a literal packet, you 
know the text is in UTF-8. Do the right thing with said text. If you 
have a blob of text and you know it's not in UTF-8, then canonicalize, 
or use the 't' flag, or even use the 'b' flag.

	Jon