Subject: RE: [Syslog] New direction and proposed charter

From: "Rainer Gerhards" <[email-masked]>
Date: Thu, 24 Nov 2005 14:01:25 +0100

Glenn,

> Now the question is : are there any RFC3164 compliant devices
> (relays and syslogd's) and applications.

I have to admit that I have written at least one compliant to RFC 3164.
But my feeling is that I am very lonley with that. In one project, we
had to make RFC 3164 not only optional but to disable it by default,
because as soon as you enable it, everything is broken. 

Also, I have done a lot of testing with different syslogds yesterday.
None of this quite popular systems is compliant. I neither know any
compliant device. Anton has backed this point of view (for example, he
adds Solaris as being non-compliant).

RFC 3164 is an informational document whichs utility was primarily to
describe security issues. It is good in that. However, it does not even
remotely describe what BSD syslog is about (a good indication of that
might be that BSD syslog works totally different from RFC 3164...).

I question that it makes any sense to modify our efforts to take care of
an informational RFC that doesn't even describe what is current
technology. Don't take me wrong: I myself based a lot of my decisions on
RFC 3164. Now, after extended testing, I thinkt that was a great
failure. When it comes to the observed format, RFC 3164 should tell
about <PRI> only. Other than that, there are no similarities in observed
syslog behaviour. Sorry for the harsh words, but if you go to the lab,
that's where you end up (notice my own disappointment yesterday...).

> If there are then we gain some and lose very little. [ Correct
> me here if am wrong in saying that we lose very little. I may
> have missed something]
> 

No sure on "lose very little". I am tired of taklking theory. I will
implement something and then tell. I guess we will loose some things, at
least in ease of parsing. And I still think we do not gain anything at
all...

> I will agree that many implementations are not RFC3164 compliant.
> But, it will be difficult to convince IESG folks or, anyone for
> that matter, that there are no RFC3164 compliant devices or
> applications at all. [ Note that the applications matter too!]

OK, let's poll the list: who has implemented RFC 3164 including relay
behaviour described there?

I have two candidates, MonitorWare/WinSyslog (some core component) on
Windows and rsyslogd on *nix. MonitorWare is the one with
default-disabled rfc3164. rsyslogd uses a much enhanced parser (non
compliant) to get its work done. rsyslogd would probably benefit from
your suggestion. However, I do not think this to be worthwhile enough,
because a) there are currently too few deployed to make it matter and b)
the next version will natively support-syslog protocol and is an
extremely easy upgrade.

> My suggestion is- avoid changes wherever we can,

I agree on this, but in that specific case I really think its not worth
it. RFC 3164 creates a false impression of format understanding. Other
than <PRI>, there is noting in common about existing syslog
implementations.

Rainer
> 
> Cheers
> 
> Glenn
> 
> Rainer Gerhards wrote:
> > This is really disappointing...
> > 
> > I have done further testing with more syslogds to confirm 
> the initial
> > findings. The more different programs / versions I try, the 
> more a mess
> > this gets. OK, we knew FreeBSD syslogd does not include the 
> hostname.
> > Next I tested with some Windows products, which looked 
> promising. Then I
> > looked at the sysklogd source. It is the standard Linux 
> package, so if
> > it works, a lot would be won. The source tells me that at least the
> > timestamp should correctly be extracted. Then I actually 
> tried it out
> > with a Debian system - the timestamp was ignored. Checking 
> that source
> > tree I see that *that* sysklogd does deliberately ignore 
> the data and
> > always pulls the local date (or ist it a bug - who 
> cares...). When it
> > comes to relaying it is even stranger: sysklogd does 
> neither relay the
> > timestamp nor the host part. At that point, I have stopped further
> > analysis, because I think I would be able to find another 
> good number of
> > variants of syslog behaviour.
> > 
> > Conclusion:
> > 
> > 1. There is no point in trying to preserve backward
> >    compatibility besides sticking with the <PRI>. Everything other
> >    than <PRI> is handled differently by different implementations 
> >    and/or versions.
> > 
> > 2. We can NOT expect that relaying over existing syslog 
> >    implementations will ever work. Please note that this would 
> >    also have broken syslog-sign without specifically implemented
> >    daemons.
> > 
> > As such, I revert back to proposing 
> > 
> > <PRI>VERSION SP TIMESTAMP SP HOSTNAME SP APP-NAME SP PROCID 
> SP MSGID SP
> > [SD-ID]s SP MSG
> > 
> > or, somewhat incorrect but shorter:
> > 
> > <PRI>VERSION TIMESTAMP HOSTNAME APP-NAME PROCID MSGID [SD-ID]s MSG
> > 
> > Please note that I have added the MSGID to the header.
> > 
> > Rainer
> > 
> > 
> >>-----Original Message-----
> >>From: [email-masked] 
> >>[mailto:[email-masked]] On Behalf Of Rainer Gerhards
> >>Sent: Wednesday, November 23, 2005 3:04 PM
> >>To: Glenn Mansfield Keeni; [email-masked]
> >>Subject: RE: [Syslog] New direction and proposed charter
> >>
> >>Glenn,
> >>
> >>very interesting approach with the timestamp. I think your 
> >>ideas can be
> >>the key to maintaining a lot of backwards compatibility by still
> >>retaining new functionality.
> >>
> >>First some bad news: I am not sure if by "BSD syslog" you 
> are refering
> >>to RFC 3164 or a specific distribution of BSD. I have 
> created a small
> >>script to test out your recommendation. I used FreeBSD stock 
> >>syslogd as
> >>the receiver.
> >>
> >>It did NOT work as expected. There are two reasons
> >>
> >>a) (that) BSD syslogd takes the sender always from the system 
> >>that send
> >>it
> >>b) even worse, when relaying, it puts "Forwarded from 
> >><hostname>: " into
> >>
> >>   the hostname part (yes, with all that spaces)
> >>
> >>So while the idea sounds excellent, it does not work with 
> >>stock FreeBSD
> >>syslogd. I am not sure about other BSD variants, nor have I 
> >>checked the
> >>sysklogd package. I believe it will have less issues in this regard.
> >>   
> >>This was the message I sent (via perl script):
> >>
> >>"<148>Oct 11 22:14:15 mymachine.example.com 1 ID47 2003T.003Z 
> >> 'su root'
> >>failed for lonvick on /dev/pts/8"
> >>
> >>this was the raw message received after being relayed once 
> by FreeBSD
> >>stock syslogd:
> >>
> >>"<148>Oct 11 22:14:15 Forwarded from 172.19.2.7: 
> >>mymachine.example.com 1
> >>ID47 2003T.003Z  'su root' failed for lonvick on /dev/pts/8"
> >>
> >>As you can see, the message is somewhat distorted - 
> definitely enough
> >>for digital signatures to be broken. [Implementor's 
> >>side-note: This can
> >>be fixed on a syslog-application layer level, far beyond the 
> >>IETF scope.
> >>It's straightforward and easy to do, so it will probably happen.]
> >>
> >>Even though this actual sample seems not to work, it paves 
> >>the way to a
> >>very elegant compatibility solution. The key is to add the extra
> >>information (e.g. Timestamp) in a different place. At least I was so
> >>focussed on fields at whole that I did not notice this 
> possibility. I
> >>have experiemented a bit more with Glenn's proposal, shuffeling some
> >>fields. The result was this:
> >>
> >><PRI>BSD_syslog_timestamp FQDN TAG "@#"VERSION MSGID 
> >>Remainder_Timestamp
> >>[SD-ID]s MSG
> >>
> >>or in an actual sample:
> >>
> >>"<148>Oct 11 22:14:15 mymachine.example.com su[4711]: @#1 ID47
> >>2003T.003Z [SD-IDs] 'su root' failed for lonvick on /dev/pts/8"
> >>
> >>I have used the BSD timestamp and FAQN as Glenn suggested. 
> >>Then, I have
> >>added the "TAG" again. If we think in the spirit of my mail 
> >>this morning
> >>on syslog & non-IETF standards, it would not really hurt if we
> >>standardize TAG instead of two fields. If I would like to retain the
> >>APP-NAME and PROCESSID, I could do the following ABNF:
> >>
> >>TAG = APP-NAME ["[" PROCESSID "]"] ":"
> >>
> >>The side-effect of this is that almost all syslog-messages currently
> >>emited comply with that format. So I suggest that we 
> strongly consider
> >>joining these two again. Out of the sudden, we have the 
> "old" header,
> >>but it is parsable by a hyptothetical new syslogd. Next, I 
> have used a
> >>trick from syslog-sign. I have changed the VERSION from a 
> number to a
> >>number plus a cookie. The version would now be "@#1". I do 
> not care if
> >>the cookie is "@#" or something else. The key point is that 
> >>it, together
> >>with the version should be very unlikely to exist at that place in
> >>old-style syslog. That would allow a "new" receiver to differentiate
> >>between old and new style syslog messages. The rest of the 
> message is
> >>just applying Glenn's proposal again: it has the MSG, the 
> >>missing parts
> >>of the timestamp, SD-IDs and MSG. The Remainder_Timestamp 
> >>looks strange.
> >>We might like it, we might like something else. That's easy 
> to change
> >>and discuss. It's the concept that matters right now, not the exact
> >>format.
> >>
> >>If we take the outlined route, we would be able to extend the syslog
> >>protocol with as much backward compatibility as is possible in a
> >>not-yet-standardized world. I find this very desirable. I 
> >>think we even
> >>have good chances that many existing "old" syslogds would relay such
> >>messages without changing them, thus keeping digital 
> >>signatures intact.
> >>The required text changes for syslog-protocol should be moderate.
> >>
> >>I strongly propose we go in that direction.
> >>
> >>Rainer
> >>
> >>>-----Original Message-----
> >>>From: [email-masked] 
> >>>[mailto:[email-masked]] On Behalf Of Glenn 
> >>>Mansfield Keeni
> >>>Sent: Wednesday, November 23, 2005 1:39 PM
> >>>To: [email-masked]
> >>>Subject: Re: [Syslog] New direction and proposed charter
> >>>
> >>>Chris/Rainer,
> >>>
> >>>
> >>>>we continue to use <PRI>... at the start of syslog 
> >>>
> >>>messages.  This will
> >>>
> >>>>allow current receivers to continue to receive messages and 
> >>>
> >>>put them in
> >>>
> >>>>the right bins.  Does anyone disagree with this?
> >>>
> >>>Complete agreement.
> >>>
> >>>>
> >>>>The WG has agreed to use the timestamp Rainer has in the current
> >>>>syslog-protocol.
> >>>
> >>>In principle I agree with the timestamp format. It is good.
> >>>I may have missed the discussion on this matter,  in that 
> >>
> >>case please
> >>
> >>>accept my apologies and ignore the rest of the mail.
> >>>
> >>>To get existing BSD syslog devices specifically relays into 
> >>>the compatibility
> >>>fold it WILL be good idea to keep the timestamp in two parts
> >>>
> >>>       RainerTimestamp =  BSD_syslog_timestamp  + 
> RemainderTimestamp
> >>>
> >>>
> >>>>One possibility would be to assemble a syslog message as:
> >>>>
> >>>><PRI>TIMESTAMP FQDN VERSION MSGID [SD-ID]s MSG
> >>>
> >>>In the context of what has been said above about the 
> >>>timestamp. I would
> >>>suggest
> >>>  <PRI>BSD_syslog_timestamp FQDN VERSION MSGID 
> >>>Remainder_Timestamp [SD-ID]s MSG
> >>>
> >>>That would allow existing BSD-syslog relays to handle the new 
> >>>syslog protocol
> >>>in a transparent manner. Everything from VERSION to the end 
> >>>is treated as "message".
> >>>
> >>>We do not lose information.
> >>>     The Remainder_timestamp carries it - in a slightly less 
> >>>convenient place
> >>>     though.
> >>>
> >>>On the other hand if we insist on using RainerTimestamp, 
> >>>existing BSD_syslog
> >>>relays will relay the message as
> >>>
> >>><PRI>BSD_syslog_timestamp FQDN RainerTimestamp FQDN VERSION 
> >>>MSGID [SD-ID]s MSG
> >>>
> >>>The message does get distorted to some extent.
> >>>
> >>>
> >>>>If we can agree to this then I suspect that we can have a working
> >>>>document within Rainer's timeframe.  I'll propose the 
> >>>
> >>>following charter
> >>>
> >>>>to keep us focused.
> >>>>
> >>>>-------- Proposed Charter  --------------
> >>>>
> >>>>Syslog is a de-facto standard for logging system events.  
> >>>
> >>>However, the
> >>>
> >>>>protocol component of this event logging system has not 
> >>>
> >>>been formally
> >>>
> >>>>documented.  While the protocol has been very useful and 
> >>>
> >>>scalable, it
> >>>
> >>>>has some known security problems which were documented in 
> >>
> >>RFC 3164.
> >>
> >>>>The goal of this working group is to address the security and
> >>>>integrity problems of the existing syslog mechanism while 
> >>>
> >>>not breaking
> >>>
> >>>>backwards compatibility.  The most obvious problems that 
> >>
> >>need to be
> >>
> >>>>addressed in the syslog protocol are the timestamp, which has not
> >>>>formally included a means to indicate the year, and the 
> >>>
> >>>identification
> >>>
> >>>>of the source which has been a hostname without a qualified domain
> >>>>name.  Additionally, a version, some type of message 
> >>>
> >>>indicator, and a
> >>>
> >>>>means to convey structured data will be included in the protocol.
> >>>>
> >>>>syslog has traditionally been transported over UDP and this WG has
> >>>>already defined RFC 3195 for the reliable transport for the syslog
> >>>>messages.  The WG will separate the UDP transport from the 
> >>>
> >>>protocol so
> >>>
> >>>>that others may define additional transports in the future.
> >>>>
> >>>>- A standard will be produced that formally documents the syslog
> >>>>protocol.  A mechanism will also be defined in this specification
> >>>>that will provide a means to convey structured data.
> >>>>
> >>>>- A standard will be produced that documents the UDP transport for
> >>>>syslog.
> >>>>
> >>>>- A standard will be produced that documents a mechanism to sign
> >>>>syslog messages to provide integrity checking and source
> >>>>authentication.
> >>>>
> >>>>- A MIB definition for syslog will be produced.
> >>>>
> >>>>-------------------------------------------
> >>>>
> >>>>
> >>>>PLEASE review this and respond.
> >>>>
> >>>
> >>>Glenn
> >>>
> >>>
> >>>_______________________________________________
> >>>Syslog mailing list
> >>>[email-masked]
> >>>https://www1.ietf.org/mailman/listinfo/syslog
> >>>
> >>
> >>_______________________________________________
> >>Syslog mailing list
> >>[email-masked]
> >>https://www1.ietf.org/mailman/listinfo/syslog
> >>
> > 
> > 
> 
> 
> 
> 
> _______________________________________________
> Syslog mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/syslog
> 

_______________________________________________
Syslog mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/syslog

---

From: "Andrew Ross" <[email-masked]>
Date: Fri, 25 Nov 2005 11:20:15 +1300

Rainer,

I agree that 3164 is only really valid with respect to the <PRI>. When we
implemented it in Kiwi Syslog we found no device actually used the 3164
format exactly. Sometimes the hostname was there, sometimes not. Having to
write parsing code to work out if a hostname was actually a TAG or not was a
huge processing overhead. We now disable 3164 parsing by default.

Cheers

Andrew