Subject: RE: [Syslog] #3 NUL octets, #4 binary data, #8 octet-counting

From: "Andrew Ross" <[email-masked]>
Date: Thu, 1 Dec 2005 01:19:18 +1300

Rainer,

That sounds good to me at this stage, and it keeps the door open. I would
prefer to see all binary data encoded in some "safe" format like base64. It
just makes logging the data to file much easier. For instance, if the binary
data contained a LF character, when it was logged to disk, it would end up
as two separate messages when opened in notepad etc.

Also, if we are to transport syslog over TCP at some stage, we need to keep
a delimiter character free from use in the message. Again, a LF would be a
good choice for this delimiter.

Cheers

Andrew

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
On Behalf Of Rainer Gerhards
Sent: Wednesday, 30 November 2005 9:26 p.m.
To: [email-masked]
Subject: [Syslog] #3 NUL octets, #4 binary data, #8 octet-counting

Hi WG,

I have received notes via private mail telling me there seem to be some
existing (and eventually soon upcoming) valid use cases for binary data
in syslog. I think there is no point in arguing whether that's fortunate
or not. It simply looks like that's the way it is. I do not like the
idea of breaking existing use cases for syslog (because that will only
lead to implementors ignoring the spec and the story of syslog
inconsistencies continues...). As such, I think we need to provide at
least some minimal support for it (aka "not outlaw it").

At first, this implies that NUL octets may be present in the message.

I propose that we write text that discourages the use of NUL, but allows
it if needed. That text should also allow, but discourage, a receiver to
modify messages containing NUL. With that, we allow the use case, but do
not make it a "show stopper" for implementing compliant software. This
would also be pretty much in sync with what we currently find in
practice, so it is already expected behaviour. Finally, such text would
caution implementors that when NUL octets are present, chancs are high
that eventually present digitial signatures will be broken. In my point
of view, that's fair and efficient.

Chris proposal for #5 (character encoding) also provides an elegant
solution for binary data. We can use something like:

[enc="binary"]

or

[enc="base-64"]

I do NOT intend to specify this - I think it should be in the scope of a
separate document specifying the use of binary data. Then would also be
the right time to discuss all issues that arise out of it. For now, I
just would like to keep the door open.

Finally, I propose to extend Chris format so that the message size can
be conveyed. This has been brought up several times and I think a clean
solution is now obvious:

[enc="utf-8" lang="en" size="MSG-size-in-octets"]

MSG-size-in-octets would be the size of the MSG part (just that!) in
octets. Counting just the MSG part is sufficient, as the rest of the
message consists of fields properly delimited. The size is probably most
useful for binary data.

Please comment.

Rainer

_______________________________________________
Syslog mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/syslog

_______________________________________________
Syslog mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/syslog

---

From: "Rainer Gerhards" <[email-masked]>
Date: Wed, 30 Nov 2005 21:12:57 +0100

Anton:

> Specifying the encoding makes sense to me.  This way we can 
> state that only certain encoding support is required, but not 
> preclude other options.  
> 
> We are still ok with always having UTF-8 in SD values, right? 

Yes, I was talking exclusively about MSG. I expect SD values to be
written by newer software only. There is no POSIX API for it, so we
can't have an issue with that API ;)

Rainer
> We need this for foreign usernames.  We have discussed this before.
> 
> Thanks,
> Anton.  
> 
> > From: [email-masked]
> > [mailto:[email-masked]] On Behalf Of Rainer Gerhards
> > Sent: Wednesday, November 30, 2005 3:26 AM
> > To: [email-masked]
> > I have received notes via private mail telling me there seem
> > to be some existing (and eventually soon upcoming) valid use 
> > cases for binary data in syslog. I think there is no point in 
> > arguing whether that's fortunate or not. It simply looks like 
> > that's the way it is. I do not like the idea of breaking 
> > existing use cases for syslog (because that will only lead to 
> > implementors ignoring the spec and the story of syslog 
> > inconsistencies continues...). As such, I think we need to 
> > provide at least some minimal support for it (aka "not outlaw it").
> > 
> > I propose that we write text that discourages the use of NUL,
> > but allows it if needed. That text should also allow, but 
> > discourage, a receiver to modify messages containing NUL. 
> > With that, we allow the use case, but do not make it a "show 
> > stopper" for implementing compliant software. This would also 
> > be pretty much in sync with what we currently find in 
> > practice, so it is already expected behaviour. Finally, such 
> > text would caution implementors that when NUL octets are 
> > present, chancs are high that eventually present digitial 
> > signatures will be broken. In my point of view, that's fair 
> > and efficient.
> > 
> > Chris proposal for #5 (character encoding) also provides an
> > elegant solution for binary data. We can use something like:
> > 
> > [enc="binary"]
> > 
> > or
> > 
> > [enc="base-64"]
> > 
> > I do NOT intend to specify this - I think it should be in the
> > scope of a separate document specifying the use of binary 
> > data. Then would also be the right time to discuss all issues 
> > that arise out of it. For now, I just would like to keep the 
> > door open.
> > 
> > Finally, I propose to extend Chris format so that the message
> > size can be conveyed. This has been brought up several times 
> > and I think a clean solution is now obvious:
> > 
> > MSG-size-in-octets would be the size of the MSG part (just
> > that!) in octets. Counting just the MSG part is sufficient, 
> > as the rest of the message consists of fields properly 
> > delimited. The size is probably most useful for binary data.
> > 
> > Please comment.
> > 
> > Rainer
> > 
> > [email-masked] https://www1.ietf.org/mailman/listinfo/syslog
> > 
>