Subject: [ISSUE] UTF-8 CRLF

From: David Shaw <[email-masked]>
Date: Tue, 26 Oct 2004 16:47:58 -0400

The last sentence of section 5.9 reads:

  Text data is stored with <CR><LF> text endings (i.e. network-normal
  line endings).  These should be converted to native line endings by
  the receiving software.

Suggest to add:

  For the 'u' UTF8 literal packet, the minimal UTF8 encoding for the
  <CR><LF> line endings SHOULD be used.  That is, 0x0D 0x0A and not
  0xC0 0x8D 0xC0 0x8A or other multibyte encodings.

Rationale:

This would be a kindness for those implementations that will not be
doing UTF8->local conversions.  Lacking a UTF8 decoder, those
implementations cannot tell that "0xC0 0x8D" or other encodings is
identical to 0x0D.  If senders are careful in this regard, then the
non-UTF8 implementations can at least get the line endings right.

David

---

From: Florian Weimer <[email-masked]>
Date: Tue, 26 Oct 2004 23:03:23 +0200

>   0xC0 0x8D 0xC0 0x8A or other multibyte encodings.

This isn't valid UTF-8.  A UTF-8 implementation MUST NOT decode these
octets, but MUST flag an error.  The most recent UTF-8 RFC is quite
explicit in this regard.

The UTF-8 issue I mentioned previously arises because Unicode has
additional characters with line-ending semantics.  There used to be a
Unicode Technical Report on this topic, but it has been superseded by
section 5.8 in Unicode 4.0:

  <http://www.unicode.org/versions/Unicode4.0.0/ch05.pdf>

---

From: David Shaw <[email-masked]>
Date: Tue, 26 Oct 2004 17:19:50 -0400

> octets, but MUST flag an error.  The most recent UTF-8 RFC is quite

My mistake.  I had missed that change.  I withdraw the suggestion.

David

---

From: Jon Callas <[email-masked]>
Date: Tue, 26 Oct 2004 14:29:46 -0700

>   0xC0 0x8D 0xC0 0x8A or other multibyte encodings.

Done.

	Jon

---

From: Jon Callas <[email-masked]>
Date: Tue, 26 Oct 2004 14:39:29 -0700

> My mistake.  I had missed that change.  I withdraw the suggestion.

And I backed out the change.

	Jon