Subject: RE: [Syslog] #5 - character encoding (was: Consensus?)

From: "Rainer Gerhards" <[email-masked]>
Date: Tue, 29 Nov 2005 17:26:38 +0100

Chris & WG,

> > #5 Character encoding in MSG: due to my proof-of-concept
> >   implementation, I have raised the (ugly) question if we need
> >   to allow encodings other than UTF-8. Please note that this
> >   question arises from needs introduced by e.g. POSIX. So we
> >   can't easily argue them away by whishful thinking ;)
> >
> > Not even discussed yet.
> 
> I haven't reviewed that yet.  However, I'll note that 
> allowing different 
> encoding can be accomplished in the future as long as we establish a 
> default encoding and a way to identify it in our current work.

I have read a little in the mailing archive. Please note that in 2000 it
was consensus that the MSG part may contain encodings other then
US-ASCII. Follow this threat:

http://www.syslog.cc/ietf/autoarc/msg00127.html

This discussion lead to RFC 3164 saying "other encodings MAY be used".
While this was observed behaviour, we need still to be aware that the
POSIX (and glibc) API places the restrictions on us that we simply do
not know the character encoding used by the application. As such, no
*nix syslogd can be programmed to be compliant to syslog-protocol if we
demand UTF-8 exclusively.

I propose that we RECOMMEND UTF-8 that MUST start with the Unicode Byte
Order Mask (BOM) if used. If the MSG part does not start with the BOM,
it may be any encoding just as in RFC 3164. I do not see any alternative
to this.

Rainer

_______________________________________________
Syslog mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/syslog

---

From: Chris Lonvick <[email-masked]>
Date: Tue, 29 Nov 2005 10:22:28 -0800 (PST)

Hi Rainer,

Why don't we look at it from the other direction?  We could state that any 
encoding is acceptable - for ease-of-use/migration with existing syslog 
implementations.  It is RECOMMENDED that UTF-8 be used.  When it is 
used, an SD-ID element will be REQUIRED.  e.g. - [enc="utf-8" lang="en"]

Thoughts?

All:  Let's discuss this and close this issue.

Thanks,
Chris

---

From: Darren Reed <[email-masked]>
Date: Wed, 30 Nov 2005 05:44:43 +1100 (EST)

> used, an SD-ID element will be REQUIRED.  e.g. - [enc="utf-8" lang="en"]

I think this is a very sensible approach.

Darren

---

From: "Andrew Ross" <[email-masked]>
Date: Thu, 1 Dec 2005 00:19:37 +1300

>used, an SD-ID element will be REQUIRED.  e.g. - [enc="utf-8" lang="en"]

I like that idea too.

So, if no SD-ID encoding element is specified, then we must assume US-ASCII
and deal with it accordingly??

Cheers

Andrew