Subject: RE: [Syslog] New direction and proposed charter

From: "Steve Chang \(schang99\)" <[email-masked]>
Date: Thu, 24 Nov 2005 02:48:12 -0800

Rainer:

> -----Original Message-----
> From: [email-masked]
[mailto:[email-masked]]
> On Behalf Of Rainer Gerhards
> Sent: Thursday, November 24, 2005 1:25 AM
> To: Balazs Scheidler
> Cc: [email-masked]
> Subject: RE: [Syslog] New direction and proposed charter
> 
> > On Thu, 2005-11-24 at 09:36 +0100, Rainer Gerhards wrote:
> > > Anton:
> >
> > > So I wonder if it wouldn't be wiser to accept that CLR here
> > and disallow
> > > NUL. After all, I can not see a valid use case for it
> > either... (in the
> > > sample you provided it honestly believe the sender should
> > not send a NUL
> > > octet but a "\u0000" string).
> > >
> >
> > \u0000 is a NUL octet at least in its canonical encoding.
> 
> OK, looks like I need to dig up the ASCII table. My fault I didn't do
it
> in the first place.
> 
> Anston's sample was
> 
>  ...MyApp pid123 MALFORMED_INPUT: Bad input received from client:
> [aaa\u0000]
> 
> For simplicity, let me strip the rest and just look at that part
> 
> \u0000
> 
> I think the sender of the sample message should not encode it as
> 
> NUL (0x00)
> 
> but instead as
> 
> 0x5c-0x75-0x30-0x30-0x30-0x30 (\-u-0-0-0-0)
> 
> Encoding it as NUL in an otherwise human-readable message makes no
sense
> to me.

>From the perspective of formatting a piece of device internal raw data 
into an outgoing syslog message and considering the CPU cycles needed to
encode a NULL character into some other form to keep the whole MSG
intact,  
I would say prepend the length of MSG (or say, add a fixed 4 digits byte
length of the MSG between SD-IDs and MSG) will be much easier for the
sender.  With this, I think the new IETF syslog compliant receiver
should also have easier time fetching in the complete MSG, regardless
it's a plain text, XML-formated, or even binary data.
[** side note: a fixed 4 digit MSG length can handle up to 9999 bytes.
Big enough for foreseeable future. Also, the first MSG can be appended
with other feature specific information before it gets sent out. And
hence updating the final MSG length can be easier.] 

As to what will happen to the old syslog receivers receiving the message
will NULL character?  The message will probably be cut short right at
reading the NULL character as you have explained. And I don't think
there will be a good compromised solution for it. 

At the sender side, the overhead in checking for a NULL character in any
of the incoming raw message and encode it to other form is just not
practical. I would rather not have to pay that overhead if possible.
Besides, if mandated, you will be limiting how MSG is to be used.

I would say it should be sufficient to just explicitly point out the
danger of having the NUL in the MSG part in the new IETF syslog
protocol.
And you can be kind to add some suggestions as to how the syslog message
initial sending device can choose to do to handle this situation
themselves. 
Vendors should make their own decisions on this issue.

With this caveat, we get to keep the flexibility of having the MSG part 
in plain text, or in binary for the IETF syslog protocol.

It's tradeoff for WG to debate.

> I am questioning if there is any legitimate case where an actual NUL
> needs to be part of MSG.
> 
> > UTF-8 allows
> > the same character to be encoded in multiple ways, see a
> > description on
> > the encoding format:
> >
> > http://www1.tip.nl/~t876506/utf8tbl.html
> >
> > And I think it would be wise to require the canonical encoding to be
> > used and treat all other encodings of the same characters as errors.
> > this is a large can of worms, lot's of security problems arised from
> > this issue. (see the IIS problems couple of years ago)
> 
> "Shortest possible form" thanksfully is already required in
> syslog-protocol-15.
> 
> Rainer
> 
> _______________________________________________
> Syslog mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/syslog

_______________________________________________
Syslog mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/syslog

---

From: Rainer Gerhards <[email-masked]>
Date: Thu, 24 Nov 2005 12:50:25 +0100

Steve,

no reply, but a question very important to me. What do you consider a
valid use case for the US-ASCII NUL character inside MSG? If I had a
good sample, I'd probably have a better time understanding why this is
important...

Rainer

---

From: "Andrew Ross" <[email-masked]>
Date: Fri, 25 Nov 2005 11:04:03 +1300

Rainer,

FWIIW, I've seen Netscreen, NetGear and some LinkSys devices send a Null
character at the end of each message. Not all versions of the firmware would
include the Null at the end which leads me to believe it may be a bug in
some releases of the firmware.

When sending syslog over TCP, Netscreen firewalls will delimit each message
with LF NULL.

This probably won't have any impact on the 'CLR' since it is the very last
character in the message, but I thought I would raise the issue with you as
a real-life case of a Null character being used.

Cheers

Andrew

---

From: Darren Reed <[email-masked]>
Date: Fri, 25 Nov 2005 10:00:26 +1100 (EST)

> a real-life case of a Null character being used.

Buggy software does not equate to a "real-life case".

But to judge whether or not it is a bug requires knowing what their
protocol spec is.

In this case, it is a delimeter and that is not something that requires
escaping.  In fact, it would be a bug to escape it if it were being used
in this manner.

Darren