Subject: RE: [Syslog] timeline

From: Miao Fuyou <[email-masked]>
Date: Tue, 15 Aug 2006 09:06:39 +0800

Hi, Rainer,

A new implementation could rely on byte-counting only and then delete LF
from the frame(appplication knows exactly where the LF is), it may not force
us to use escapes. For LF, I think it is difficult to get 100% compatibility
for a legacy implementation to comply TLS-transport without any change to
the code. At least, some imlementation may need to change CR LF to LF
because some implementations use CR LF rather than LF. So, it may be ok to
add several LOC to delete FRAME-LEN SP from the frame. 

I still prefer byte-counting only to byte-counting+LF even if it is a
feasible tradeoff.  

Miao

> -----Original Message-----
> From: Rainer Gerhards [mailto:[email-masked]] 
> Sent: Monday, August 14, 2006 10:18 PM
> To: Miao Fuyou
> Subject: RE: [Syslog] timeline
> 
> We should not go byte-counting + LF. This is the worst choice: it 
> 
> A) breaks compatibility
> B) Forces us to use escapes
> 
> So we get the bad of both worlds, without any benefits.
> 
> Rainer 
> 
> > -----Original Message-----
> > From: Miao Fuyou [mailto:[email-masked]]
> > Sent: Monday, August 14, 2006 12:58 AM
> > To: 'Anton Okmianski (aokmians)'; 'David Harrington'; 
> [email-masked]
> > Subject: RE: [Syslog] timeline
> > 
> > 
> > My vote: byte-counting only > byte-counting + LF > LF
>  
> 

_______________________________________________
Syslog mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/syslog

---

From: "Rainer Gerhards" <[email-masked]>
Date: Tue, 15 Aug 2006 06:40:48 +0200

Miao,

I am actually concerned about backward compatibility with existing code
*without* the need to upgrade any of that code. As you know, deployed
software tends to stick.

If we use just LF, existing, deployed technology (e.g. syslog-ng with
stunnel) would be able to understand a message sent from a "new style"
syslogd. Having the octet count in front of the message removes that
ability, as the old syslogd will no longer see the <pri> at the start of
the message.

I agree that it is trivial to modify code to take care for the octet
counter. But this is not my concern. My concern is that I would like to
achive as good as possible compatibility with existing deployed (aka
"unmodified") technology. I should have been more specific on that.
Sorry for the omission...

I am also unaware of any implementation that mandates CR LF over just
LF. Could you let me know which ones are these?

Rainer 

> Sent: Monday, August 14, 2006 7:07 PM
> To: Rainer Gerhards
> Cc: [email-masked]

---

From: Miao Fuyou <[email-masked]>
Date: Tue, 15 Aug 2006 14:59:28 +0800

Rainer,

Stunnel is a secure wrapper for TCP stream. Actually delimiting Syslog is
done in the TCP part rather than TLS (or stunnel) part in Syslog-ng with
stunnel. One can use stunnel to secure any Syslog TCP transport, such as
rsyslog and kiwisyslog, and kiwisyslog does use CRLF for delimiting
(http://www.kiwisyslog.com/whats_new_syslog.htm). 

Stunnel implementation is different from Syslog TLS transport, and I don' t
think it is the exact implementation of Syslog TLS transport. I have not
been aware of a Syslog implementation in TLS-transport style till now. So,
most of the implementation may be modified, slightly or heavily, to existing
code to get it comply to the specification. 

Miao

> Sent: Tuesday, August 15, 2006 12:41 PM
> To: Miao Fuyou
> Cc: [email-masked]
> *without* the need to upgrade any of that code. As you know, 
> deployed software tends to stick.
> 
> syslogd. Having the octet count in front of the message 
> removes that ability, as the old syslogd will no longer see 
> the <pri> at the start of the message.
> 
> I agree that it is trivial to modify code to take care for 
> the octet counter. But this is not my concern. My concern is 
> that I would like to achive as good as possible compatibility 
> with existing deployed (aka
> I am also unaware of any implementation that mandates CR LF 
> over just LF. Could you let me know which ones are these?
> 
> Rainer 
> 
> > A new implementation could rely on byte-counting only and 
> then delete 
> > LF from the frame(appplication knows exactly where the LF 
> is), it may 
> > not force us to use escapes. For LF, I think it is difficult to get 
> > 100% compatibility for a legacy implementation to comply 
> TLS-transport 
> > without any change to the code. At least, some 
> imlementation may need 
> > to change CR LF to LF because some implementations use CR LF rather 
> > than LF. So, it may be ok to add several LOC to delete FRAME-LEN SP 
> > from the frame.
> >

---

From: "Andrew Ross" <[email-masked]>
Date: Tue, 15 Aug 2006 19:59:26 +1200

Just to clarify, Kiwi Syslog V8 currently sends CRLF as the TCP delimiter,
but will accept LF, CR, CRLF, LFCR and NULL as valid delimiters on the
incoming stream. We will be changing our sending delimiter to LF in the near
future to make it more compatible with syslog-ng etc.

Cheers

Andrew

Rainer,

Miao

---

From: Balazs Scheidler <[email-masked]>
Date: Tue, 15 Aug 2006 13:26:56 +0200

> future to make it more compatible with syslog-ng etc.

syslog-ng sends LF, but accepts CRLF.

-- 
Bazsi

---

From: Balazs Scheidler <[email-masked]>
Date: Tue, 15 Aug 2006 13:28:31 +0200

> syslog-ng sends LF, but accepts CRLF.

Ughh let me clarify. So syslog-ng sends LF in its messages, but accepts
LF, CRLF and NUL characters in received messages.

-- 
Bazsi

---

From: "David Harrington" <[email-masked]>
Date: Tue, 15 Aug 2006 13:24:32 -0400

Hi Rainer,

[speaking as co-chair]
Can we change the subject line to "byte-counting vs special character"
for this topic so it is easier to find comments on this topic as
compared to other things in the timeline? That will make it nuch
easier for the chairs.

This WG has already gotten stuck, and had WG progress stall, trying to
provide backwards compatibility to a bunch of incompatible
implementations. I argued on this list before becoming co-chair that
backwarsd compatibility may not be achieveable for some features and
we need to stop getting hung up on it. Sometimes to build a good
standard, you need to choose something that will break some existing
implementations. 

I raised this concern with Chris when I started as co-chair. I do not
want to see backwards compatibility arguments stall progress again. I
made sure this was reflected in the timeline, which says that by
Friday (if you decide to use a special character) you must reach
consensus on which special character to use.
[/speaking as co-chair]

[speaking as contributor]
I like the argument that the LF solution will not break existing
implementations, but I would like to know this is actually true. Have
you actually tested this against multiple implementations, or is it a
theoretical argument?

I know you have tested a number of other proposed ways of doing things
against multiple implementations to try to verify backwards
compatibility. Have you actually tested multiple existing
implementations with the LF and found that they do continue to work
without significant problems? Can you tell the WG which ones you have
tested? Are there implementations that break when using this solution?

In a different message you argue that syslog-sign only needs to
support -protocol because the implementations will need to have new
code added anyway, and the added complexity of backwards compatibility
to rfc3164 implementations is not needed.

You only mention one implementation explicitly that provides
syslog/tls, syslog-ng over stunnel. Would all the other
implementations need to be modified to support a TLS substrate anyway,
so it would not make a difference to them which special character was
chosen or if byte-counting was chosen? Which syslog/tls
implementations will be impacted by our choice here? 

Ideally, only the implementations that support a feature need to pay
the price for the feature.

A syslog message delineator will only be needed in implemntations that
add support for syslog/tls (or other streams). Whether one supports
byte-counting or a special character, only implementations that
support multiple messages in a stream, e.g. syslog/tls, should need to
be modified to support the choice. 

Personally I find the octet-counting cleaner because previous
discussions on terminators showed that available implementations are
highly inconsistent with their special characters. 

Since byte-counting happens outside the syslog message, and only for
syslog in a delimited stream coming in over the syslog/tls port, it
would not impact existing code, only code that supports syslog/tls.
New syslog/tls code would be needed to extract each "normal" syslog
message from the stream and then send it on for processing through the
existing parser. 

If message originators start escaping special characters in the
message content, existing receivers may need to be modified to
un-escape the characters. Think about applications, such as intrusion
detection systems, that search for special patterns in syslog content
- those could be broken by having the content changed by escaping
characters in the middle of the content.

What happens at a store-then-forward relay? Does the relay store the
escaped characters or does it un-escape them first? Does it escape
them again when it forwards the message? Does it escape them if it
un-escaped them on input? Does it escape them if it did not un-escape
them on input? What if the sender included some escaped characaters
(like '\') for other purposes? Should the relay unescape them as well?
Will it know to re-escape them on output? What should the ultimate
reciever expect to receive?

Byte-counting looks way less complex to me than choosing a special
character and escaping/unescaping special characters in the content
during the sending/forwarding/recieving process. (And my opinion has
nothing to do with the H**** in my email address.)

dbh

> Sent: Tuesday, August 15, 2006 12:41 AM
> To: Miao Fuyou
> Cc: [email-masked]
deployed
> > then delete LF

---

From: "Rainer Gerhards" <[email-masked]>
Date: Tue, 15 Aug 2006 16:26:29 +0200

Miao,

Rsyslog accepts CRLF, and can even be configured to emit it.

I have no access to a lab. Could you please clarify where exactly
-transport-tls is unable to work with currently existing deployments
using stunnel? I would greatly appreciate insight as I can not try it
out for a while (and I am also unable to do any in-depth analysis).

Thanks,
Rainer 

> Sent: Tuesday, August 15, 2006 12:59 AM
> To: Rainer Gerhards
> Cc: [email-masked]

---

From: "Rainer Gerhards" <[email-masked]>
Date: Wed, 16 Aug 2006 05:12:51 +0200

<inline> 

> From: David Harrington [mailto:[email-masked]] 
> Sent: Tuesday, August 15, 2006 11:25 AM
> To: Rainer Gerhards
> Cc: [email-masked]
> Subject: byte-counting vs special character
> 
> Hi Rainer,
> 
> [speaking as co-chair]

Sure

> consensus on which special character to use.

As you know, I have originally opted for octet-couting. However, I am
trying to describe what made me think this is a sub-optimal solution. I
think backwards compatibilty can easily be brought in here. However, I
do not think this is a vital point. I can live with octet-couting and
would not object it if it is the WG consensus to use it. So far, I've
not seen a clear consesus for either.

What I would find very unfortunate is using both escapes and
octet-counting (for the reasons outlined in previous mails - gains
nothing, costs many).

To fulfil your requirement, I propse that we use the backslash character
("\") as escape if we should decide to go the "escape way".
 
> you actually tested this against multiple implementations, or is it a

No, it's a practical one, but based on plain tcp transport seen in the
wild.

I know for sure that Cisco PIX, syslog-ng, rsyslog,
WinSyslog/MonitorWare and most probably Kiwi syslog support LF as
terminator (Kiwi with a leading CR as Andrew has pointed out). All of
these solutions interop via ssl if used with stunnel.

> tested? Are there implementations that break when using this solution?

All quoted above rely on the LF. If it is not present, at least most of
them will merge syslog records (I can not at this moment verify the
later for all one mentioned because I have no lab access).

So far, my understanding was that stunnel is basically compatible to
what -transport-tls specifies. Miao has questions that this morning.
Again, I can not verify at this time (not even having a -transport-tls
implementation at hand). I have asked Miao for details where he sees the
incompatibility. When he replies, his argument might actually settle the
whole issue and proove me wrong.

> to rfc3164 implementations is not needed.

I see a big difference here. To support -sign, all new implementations
are needed. -tls (at least in my current view) does not come with this
restriction.
 
> implementations will be impacted by our choice here? 

Stunnel is a wrapper. It works with anything that runs on tcp. I have
mentioned syslog-ng because it is the best known (and nothing I am
involved with). All implementations I have quoted work with stunnel and
are interoperable in that way (again, Kiwi only 98% sure).
 
> Personally I find the octet-counting cleaner 

I have argued much in the same way and still find byte-counting the
technically best solution.

> because previous

I think it impacts existing deployed solutions using syslog with
stunnel.

> characters in the middle of the content.

LF is currently not seen in log files. A problem could occur by escaping
\, but it seems to be seldomly used.

> reciever expect to receive?

Existing receivers should never unescape and escape. -transport-tls
receivers MUST unescape and escape the message when forwarding. The
escape is just a transfer encoding, so it is not present in the actual
message.
 
> nothing to do with the H**** in my email address.)

I agree octet-couting is easier. As I said, it was my preferred
soltution, too. What made me change my mind was backwards compatibility.
The added complexity IMHO is a small price to pay for immediately being
compatible to everything that is deployed today. If I am wrong here
(Miao's answer...), escaping would be a big mistake. Until this is
proven, I stick with it as my preferrence. I do not, however, object
octet-couting if that is consensus.

Rainer

---

From: "Rainer Gerhards" <[email-masked]>
Date: Wed, 16 Aug 2006 17:39:21 +0200

Miao,

I need to leave, thus short: To the best of my knowledge, a native
sender can talk to an stunnel-enhanced receiver. I am pretty sure I have
tested this, but I can not verify due to missing lab abilities right
now. Maybe somebody else could try it out. AFAIK the stunnel manual also
describes that scenario.

This ability is my #1 reason why I think escaping is superiour to
octet-couting.

Rainer

> Sent: Wednesday, August 16, 2006 4:01 AM
> To: Rainer Gerhards
> Cc: [email-masked]
> As you know, in stunnel/syslog model, a syslog sender sends 
> syslog message
> in TCP transport to a local port of the stunnel client, it 
> then forwards it
> to the stunnel server on another host in a TLS secure tunnel. 
> After stunnel
> server received the message it sends the message to the 
> receiver on the same
> host.  Basically stunnel is a port forwarding facility for 
> application(any
> application in TCP transport, not only syslog), and stunnel client and
> server are both stand-alone applications co-locating on same host with
> syslog sender or receiver application. 
> 
> Transport-tls requires a secure transport mechanism for 
> Syslog, that means
> the TLS transport is part of the sender/receiver application 
> rather than
> another stand-alone application. So, I believe stunnel/syslog 
> is not the
> exact implementation for transport-tls. 
> 
> I have not checked whether a transport-TLS sender can works 
> well with a
> "stunnelized" receiver or vice versa because there are not available
> transport-tls implementation currently. It may be possible, 
> however, is such
> interaction capability a requirement for transport-tls?
> 
> Thanks!
> Miao
> 
> > Sent: Tuesday, August 15, 2006 10:26 PM
> > To: Miao Fuyou
> > Cc: [email-masked]
> > I have no access to a lab. Could you please clarify where 
> > exactly -transport-tls is unable to work with currently 
> > existing deployments using stunnel? I would greatly 
> > appreciate insight as I can not try it out for a while (and I 
> > am also unable to do any in-depth analysis).
> > 
> > Thanks,
> > Rainer 
> > 
> > > Stunnel is a secure wrapper for TCP stream. Actually 
> > delimiting Syslog 
> > > is done in the TCP part rather than TLS (or stunnel) part 
> > in Syslog-ng 
> > > with stunnel. One can use stunnel to secure any Syslog TCP 
> > transport, 
> > > such as rsyslog and kiwisyslog, and kiwisyslog does use CRLF for 
> > > delimiting (http://www.kiwisyslog.com/whats_new_syslog.htm).
> > > 
> > > Stunnel implementation is different from Syslog TLS 
> > transport, and I 
> > > don' t think it is the exact implementation of Syslog TLS 
> transport.
> > > I have not
> > > been aware of a Syslog implementation in TLS-transport 
> > style till now. 
> > > So, most of the implementation may be modified, slightly or 
> > heavily, 
> > > to existing code to get it comply to the specification.
> > > 
> > > Miao
> > > 
> > > > I am actually concerned about backward compatibility with 
> > existing 
> > > > code
> > removes that 
> > > > ability, as the old syslogd will no longer see the <pri> at the 
> > > > start of the message.
> > > > 
> > > > counter. But this is not my concern. My concern is that I 
> > would like 
> > > > to achive as good as possible compatibility with existing 
> > deployed 
> > > > (aka
> > > > I am also unaware of any implementation that mandates 
> CR LF over 
> > > > just LF. Could you let me know which ones are these?
> > > > 
> > > > Rainer
> > > >

---

From: Miao Fuyou <[email-masked]>
Date: Wed, 16 Aug 2006 18:01:23 +0800

Rainer,

As you know, in stunnel/syslog model, a syslog sender sends syslog message
in TCP transport to a local port of the stunnel client, it then forwards it
to the stunnel server on another host in a TLS secure tunnel. After stunnel
server received the message it sends the message to the receiver on the same
host.  Basically stunnel is a port forwarding facility for application(any
application in TCP transport, not only syslog), and stunnel client and
server are both stand-alone applications co-locating on same host with
syslog sender or receiver application. 

Transport-tls requires a secure transport mechanism for Syslog, that means
the TLS transport is part of the sender/receiver application rather than
another stand-alone application. So, I believe stunnel/syslog is not the
exact implementation for transport-tls. 

I have not checked whether a transport-TLS sender can works well with a
"stunnelized" receiver or vice versa because there are not available
transport-tls implementation currently. It may be possible, however, is such
interaction capability a requirement for transport-tls?

Thanks!
Miao

> > don' t think it is the exact implementation of Syslog TLS transport.
> > I have not
> > > I am also unaware of any implementation that mandates CR LF over

---

From: Carson Gaspar <[email-masked]>
Date: Wed, 16 Aug 2006 12:32:46 -0700

Escaping precludes no-configuration backwards compatibility, as no legacy 
syslog-over-tcp code does escaping. So if you want to interoperate with 
existing code, you must have a "don't escape or expect escapes" switch in 
your code. If you're going to do that, just have a "LF mode vs byte-count 
mode" switch. This whole backwards compat argument is bogus, iff we decide 
to escape embedded LF instead of forbidding it. And I have yet to see 
anyone argue for LF on any grounds except backwards compatibility.

-- 
Carson

---

From: "Rainer Gerhards" <[email-masked]>
Date: Thu, 17 Aug 2006 02:08:01 +0200

Carson,

Legacy code does not contain LF in messages. It is advised that
new-style syslog also does not contain control characters (though it now
is allowed).

Thus the argument is valid. Again, I do not object octet-couting (I
actually introduced the idea ;)) but find it the second best-solution.
Maybe we should do a simple poll - some have already voiced their
choices...

Rainer 

> From: Carson Gaspar [mailto:[email-masked]] 
> Sent: Wednesday, August 16, 2006 1:33 PM
> To: [email-masked]
> Subject: Re: [Syslog] RE: byte-counting vs special character
>

---

From: Carson Gaspar <[email-masked]>
Date: Wed, 16 Aug 2006 17:21:20 -0700

> new-style syslog also does not contain control characters (though it now

Legacy code _will_ emit doubled escape characters (unless the escape 
character selected is forbidden in legacy code). They will then be 
mis-interpreted by new code.

New code _will_ double the escape character when sending it to legacy code, 
mangling the message.

You can't have it both ways. Either you forbid LF and use it as a EOM 
marker and get backwards compatibility, or you allow LF (via whatever 
mechanism, escaping or octet counting are equivalent) and break backwards 
compatibility. Statements that "doubled escape characters are unlikely in 
legacy messages" are bogus. Either they are _impossible_, or they will 
happen eventually.

If you want to bow at the alter of the installed base, forbid LF in 
messages. If not, then octet counting is technically superior. This 
wishy-washy middle ground is just a terrible idea.

-- 
Carson

---

From: "David Harrington" <[email-masked]>
Date: Wed, 16 Aug 2006 21:03:41 -0400

Hi Rainer,

The IETF doesn't vote.
The chairs will determine consensus on or after the Aug 18 deadline
for this decision.

David Harrington
[email-masked] 
[email-masked]
[email-masked]
co-chair, Syslog WG 

> Sent: Wednesday, August 16, 2006 8:08 PM
> To: Carson Gaspar
> Cc: [email-masked]
> Subject: RE: [Syslog] RE: byte-counting vs special character
> 
> Carson,
>

---

From: "Rainer Gerhards" <[email-masked]>
Date: Thu, 17 Aug 2006 03:18:56 +0200

Caspar - since when does legacy code do escaping? (Even if I've
overlooked it, we might use a different escape character...)

Anyhow - the "whishy-washy" blew the WGs work last year. Ignoring legacy
compatibility trashed roughly a year of work. I don't want to see this
happen again. I'll post a more elaborate message on last years failure
of the syslog WG due to ignoring backward compatibility as soon as I am
able to do so...

Rainer 

> Sent: Wednesday, August 16, 2006 6:21 PM
> To: [email-masked]

---

From: Carson Gaspar <[email-masked]>
Date: Wed, 16 Aug 2006 18:55:28 -0700

> overlooked it, we might use a different escape character...)

It doesn't. However, if the escape character is legal in legacy code, it 
can appear twice. Which will be converted to a single escape character by a 

>> mis-interpreted by new code.

Note the "unless".

> of the syslog WG due to ignoring backward compatibility as soon as I am

Great. So forbid LF in the message and drop the escaping thing. I'm fine 
with that. Adding an escape character and claiming backwards compatibility 
is a sham, unless you can _guarantee_ that the escape character is never 
emitted by legacy code. The appearance of compatibility without the 
substance is useless.

-- 
Carson

---

From: "Rainer Gerhards" <[email-masked]>
Date: Thu, 17 Aug 2006 04:41:49 +0200

> Sent: Wednesday, August 16, 2006 7:55 PM
> To: [email-masked]

No - a new receiver checks the version field and detects that it is a
legacy message. This is what the header is designed for. As it knows it
is legacy, it won't unescape...

Rainer

---

From: "tom.petch" <[email-masked]>
Date: Wed, 16 Aug 2006 19:07:34 +0200

----- Original Message -----
From: "David Harrington" <[email-masked]>
To: "'Rainer Gerhards'" <[email-masked]>
Cc: <[email-masked]>
Sent: Tuesday, August 15, 2006 7:24 PM
Subject: [Syslog] byte-counting vs special character

><snip>>
Turning the argument around, how many implementations have you got and have
tested for interoperability that use byte counting for syslog?

As Rainer's posts and earlier work as documented on his web site show, there is
an awful lot of syslog out there, more pre-existing, interoperable
implementation than in any other activity of the IETF I have been involved with.
For me, this overcomes any technical, architectural considerations of
'betterness' and says we must go with our best understanding of the existing
marketplace (I thought differently when I started:-(

Other participants on this list may only represent a little of the implemented
code but it is still an awful lot of pre-existing implementation.

Tom Petch

>/syslog

---

From: Balazs Scheidler <[email-masked]>
Date: Thu, 17 Aug 2006 14:19:22 +0200

> anyone argue for LF on any grounds except backwards compatibility.

As I said in a private mail to you, no we don't need that switch. LF is
escaped as a sequence of two characters '\' and 'n'. This way escaped LF
characters will not affect protocol processing, the only issue is that
LFs in the message will be written to the disk in a slightly different
format. But adding the fact that current TCP senders are not transparent
wrt LFs this is not a big deal.

- old sender - new receiver 
    => works, because current syslog-TCP senders strip LFs off the
message, either they replace it with space or forward multiple messages

- new sender - old receiver
    => works, because the old receiver does not care about the "\n"
string in the message, although it will not unescape it when it writes
it to disk

-- 
Bazsi

---

From: Chris Lonvick <[email-masked]>
Date: Thu, 17 Aug 2006 07:21:16 -0700 (PDT)

Hi,

> string in the message, although it will not unescape it when it writes

What's going to happen with syslog-sign and/or other mechanisms that will 
look at the packet and create a hash of it?  It sounds like everything 
will be acceptable if a new receiver gets it and does the re-conversion 
before anything looks at the contents.  However, an old receiver will 
continue to keep the \n which will mess up syslog-sign.  Is that correct?

Also, what's going to happen to a new receiver that receives a legitimate 
"\n" as in an original message send:
    <PRI>... BOM The offending characters are \n
Will the receiver convert that into LF?

Thanks,
Chris

---

From: Chris Lonvick <[email-masked]>
Date: Thu, 17 Aug 2006 07:22:35 -0700 (PDT)

Hi,

I agree; we don't want a vote here.  We want strong technical reasons for 
making a decision.

Thanks,
Chris

---

From: "David Harrington" <[email-masked]>
Date: Thu, 17 Aug 2006 11:54:56 -0400

Hi Tom,

[speaking as a contributor]

Rainer asserted that using LF would not break existing
implementations. It does not appear to me that we have reached
consensus that this assertion is true. I personally have concerns
because this WG has already found that many syslog implementations are
not compatible, especially on the point of having reserved/escaped
control characters for special use. I asked if Rainer could provide
any empirical support for his assertion, or whether his assertion was
theoretical. 

I have not asserted that a message containing an octet count would
work with existing implementations; only syslog implementations that
were updated to support the not-yet-defined <syslog/tls> standard
would use octet counting. 

Legacy messages without an octet-count would continue to be sent to
the legacy syslog port, presumably one by one or delineated using
whatever mechanism the legacy implementation currently supports. 

As I see it, a syslog/tls implementation would prepend an octet-count
field to each syslog message, concatenate the messages into a stream,
and send the concatenated messages to the new <syslog/tls> port. An
implementation that supports the syslog/tls standard would strip off
the prepending octet-count and extract each counted syslog message
from the stream. Each extracted message could then be processed using
the exact same parser code as would be used for messages received at
the legacy syslog port. 

Using an octet-count encapsulation technique like this means the
existing syslog message is not touched in any way, and the presence of
an octet-counted stream delivered to a special port expecting the
octet-counted stream, cannot interfere with any existing assumptions
about the message format or content. 

Note that the syslog messages encapsulated by the octet-count might be
able to be of any syslog format. The legacy parser implementation
(whichever of the many incompatible variations) would not be changed.
If the ability to carry different formats of syslog messages is
desired, an additional field should probably be prepended to each
syslog message in the stream that identified which syslog parser
should be used to parse it. I am not a supporter of this approach
because I would rather see implementations move to the syslog-protocol
standard, but for those who value backwards compatibility above all
else, this might be an interesting property.

David Harrington
[email-masked] 
[email-masked]
[email-masked]

> From: tom.petch [mailto:[email-masked]] 
> Sent: Wednesday, August 16, 2006 1:08 PM
> To: David Harrington
> Cc: [email-masked]
> Subject: Re: [Syslog] byte-counting vs special character
>

---

From: Balazs Scheidler <[email-masked]>
Date: Thu, 17 Aug 2006 22:12:32 +0200

> I agree; we don't want a vote here.  We want strong technical reasons for 

As I see:

We have a technically superior (we all seem to agree on this one)
solution that breaks compatibility. Compatibility is important, this is
documented in our charter.

If we are to break compatibility we definitely need to have important
incentives to do so. As I see the byte counter in itself is not
important enough, I would simply forbid LF in messages as a compromise
if that is acceptable to others.

Thus we should not simply stop at adding the byte counter. There are
other features currently missing from the protocol which can only be
implemented with an incompatible change. An important example is
application layer acknowledgements. (not the complex one in
syslog/COOKED but a simpler scheme) What I'm afraid about this path is
that we need to deliver something soon.

-- 
Bazsi