Subject: RE: [Syslog] delineated datagrams

From: "Rainer Gerhards" <[email-masked]>
Date: Thu, 6 Jul 2006 16:16:47 +0200

Tom,

I think your and Anton's commetn below so far is the most important
comment on -transport-tls technical issues (assuming that the
certificate issue can relatively easy be fixed by specifying a cipher
suite).

IMHO the comment applies to any shim layer for stream protocols, not
just TLS. I think a compromise to get something like -transport-tls
going is that we might actually define LF to be the end of record marker
and require the message inside the "frame" to escape LF. That would
require two  characters to be escaped (of for the LF and one for the
escape character itself). Such a compromise would also be consistent
with what is currently done in practice. And, indeed, we could avoid the
byte-counting. That would fully bring us in sync to what is done today
(syslog-ng, cisco Pix, rsyslog, Kiwi Syslog, MonitorWare to name a few).
I still consider this to be a non-perfect framing, but I think it would
be acceptable. In the medium term, we should look into a more
sophisticated framing, maybe borrowed from NETCONF. But that should come
after we have delivered something. If that might be a compromise, I
could quickly draft an I-D that *documents* the way TCP based stream
transport is used today. -transport-tls would then just need to add
description of TLS handshaking. Or the I-D could be informational
describing what can be found in practice. I do not know if that would
have any effect on the patent office's decision (but I can claim
publically-available previous work, around Jan 2003 - see
http://adiscon.org/specs/selp-2003-01-15.txt).

For the legal folks: I have running implementations and public documents
describing the method outlined above. It is fraudulent if somebody
claims he has invented the method I have described here, at least if he
hasn't invented it roughly 10 years or so ago.

Rainer 

> -----Original Message-----
> From: Tom Petch [mailto:[email-masked]] 
> Sent: Thursday, June 22, 2006 11:47 AM
> To: Anton Okmianski (aokmians); [email-masked]
> Subject: Re: [Syslog] delineated datagrams
> 
> <inline>
> ----- Original Message -----
> From: "Anton Okmianski (aokmians)" <[email-masked]>
> To: "Tom Petch" <[email-masked]>; <[email-masked]>
> Sent: Tuesday, June 20, 2006 8:18 PM
> Subject: RE: [Syslog] delineated datagrams
> 
> 
> Tom:
> 
> I think these are valid concerns. They span different layers:
> 
> 1. If we only care about network-layer reliability (as in 
> byte insert/remove
> examples): client/server can be recommended to reset 
> connection every so often.
> Decent corruption detection is already part of TCP/IP and 
> layer 2 protocols.
> 
> 2. If we care about app-layer reliability (as in 
> encode/decode error examples):
> app-level ACK. As in RFC 3195, for example.  This certainly 
> expands the scope
> quite a bit beyond just secure transport.
> 
> Anton
> 
> My concern was in between, the shim between TCP and syslog 
> that is TLS.
> 
> With UDP transport, a Relay receives a datagram and sends it 
> on its way, no
> processing required; probability of corruption small enough to ignore,
> consequences when it does, one corrupted packet.
> 
> With TLS, the packet must be decrypted, unpadded, 
> decompressed split into
> 'datagrams' on the basis of a string of decimal digits and 
> then re-packaged to
> send on its way.  If a byte is lost here, chances are the 
> string of decimal
> digits will point into the middle of a valid string of 
> decimal digits, which
> will give a truncated second 'datagram' and point into who knows what.
> Eventually, the pointer will not point to a string of decimal 
> digits and the
> Relay will realise it is lost.  By then, it may have 
> forwarded several corrupted
> or truncated 'datagrams'.  And the chance of recovering sync 
> is, IMO, nil; tear
> down the connection and set it up again.
> 
> By contrast, there are protocols which rely on detecting 
> sync, use it initially,
> error detect and error recover with it.  Of course, they have 
> more to go on than
> a string of decimal digits, they have structures which are 
> distinctive in the
> context of the datastream (I would for example expect to 
> recover sync with BER.1
> encoding of SNMP, although I don't know anyone who does that).
> 
> So my thinking is should we have more than a string of 
> decimal digits, like
> </length=137>
> or something else that is obviously invalid so that errors 
> are likely to be
> detected immediately and the Relay has a chance of scanning 
> the stream to
> recover sync.
> 
> You may recall we have had discussions of length v end of 
> record marker before
> (and yes, I do like end of record markers:-)
> 
> Tom Petch
> 
> Anton.
> 
> > -----Original Message-----
> > From: Tom Petch [mailto:[email-masked]]
> > Sent: Tuesday, June 20, 2006 8:43 AM
> > To: [email-masked]
> > Subject: Re: [Syslog] delineated datagrams
> > wasdraft-ietf-syslog-transport-tls-01.txt
> >
> > I wonder if others share my concern about the lack of
> > robustness in the way in
> > which datagrams are delineated in the stream protocol (a TCP
> > rather than a TLS
> > issue).
> >
> > The system works as long as
> >  - the frame length is encoded perfectly
> >  - the frame length is decoded perfectly
> >  - no bytes are inserted or removed in error
> > which is doubtless true in some networks, but I would prefer
> > not to rely on it.
> >
> > So, when an error occurs, can the Collector/Relay detect 
> it?  Can the
> > Collector/Relay recover synch?  If not, what does the
> > Collector/Relay do?
> >
> > There is very little redundancy in the definition of frame
> > length, and syslog
> > messages have very little structure to help the application,
> > so I think that
> > this is an issue we should address.
> >
> > Tom Petch
> >
> > ----- Original Message -----
> > From: "David B Harrington" <[email-masked]>
> > To: <[email-masked]>
> > Sent: Tuesday, May 09, 2006 4:26 PM
> > Subject: [Syslog] draft-ietf-syslog-transport-tls-01.txt
> >
> >
> > Hi,
> >
> > A new revision of the syslog/TLS draft is available.
> > 
> http://www.ietf.org/internet-drafts/draft-ietf-syslog-transport-tls-01
> > .txt
> >
> > We need reviewers.
> > Can we get
> > 1) a person to check the grammar?
> > 2) a person to check the syslog technical parts?
> > 3) a person to check compatibility with the other WG documents?
> > 4) a person to check the TLS technical parts?
> >
> > We also need general reviews of the document by multiple people.
> >
> > Thanks,
> > David Harrington
> > co-chair, Syslog WG
> > [email-masked]
> >
> >
> > _______________________________________________
> > Syslog mailing list
> > [email-masked]
> > https://www1.ietf.org/mailman/listinfo/syslog
> >
> >
> > _______________________________________________
> > Syslog mailing list
> > [email-masked]
> > https://www1.ietf.org/mailman/listinfo/syslog
> >
> 
> 
> _______________________________________________
> Syslog mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/syslog
> 

_______________________________________________
Syslog mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/syslog

---

From: "Rainer Gerhards" <[email-masked]>
Date: Thu, 20 Jul 2006 16:01:42 +0200

WG,

now with the consensus on moving forward with -transport-tls, I would
like to re-state my thoughts on delineated datagrams:

I think a compromise to get -transport-tls going is that we might
actually define LF to be the end of record marker and require the
message inside the "frame" to escape LF. That would require two
characters to be escaped (of for the LF and one for the escape character
itself). Such a compromise would also be consistent with what is
currently done in practice. And, indeed, we could avoid the
byte-counting. That would fully bring us in sync to what is done today
(syslog-ng, cisco Pix, rsyslog, Kiwi Syslog, MonitorWare to name a few).
I still consider this to be a non-perfect framing, but I think it would
be acceptable. In the medium term, we should look into a more
sophisticated framing, maybe borrowed from NETCONF. But that should come
after we have delivered something. 

A byte-count could be prefixed to each record, but that would break
compatibility with existing implementations (this is not absolutely
necessary, but I consider it a very-nice-to-have, especially if the
price for it is low).

Comments appreciated,
Rainer

> From: Rainer Gerhards [mailto:[email-masked]] 
> Sent: Thursday, July 06, 2006 4:17 PM
> To: Tom Petch; [email-masked]