Subject: RE: [Syslog] RE: Message format

From: "Rainer Gerhards" <[email-masked]>
Date: Wed, 23 Nov 2005 10:00:24 +0100

Andrew,

on the size: Though I have some concerns, I can agree with your point of
view. In fact, one of the syslog-protocol revisions had a mechanism
called multi-part messages. This could be utilized. Maybe we should do a
separate spec on "large size messages". That wouldn't be too much effort
and be a truely optional feature (I even think we could simply carry
over the text from that draft version).

What I am currently concerned about is putting this size issue to a
rest. I think the compromise is good enough, especially as we do NOT yet
specify a plain tcp mapping (we even don't know if we will ever get
consensus to do that). That means the currently proposed text keeps the
options open to do anything we might later decide. And it acutally puts
a hard limit to roughly 64K by the fact that only UDP is supported. As I
have outlined in another mail, that practical limit seems to be more 4K
than 64K.

Given that situation, I strongly suggest not to get another round of max
size discussion.

I think we urgently need now a consensus on the lowest denominator and
get that consensus published. Else we will be discussing and discussing
but never achive any milestone. I like the approach of baby-steps, which
will give us something usable after each step. The core thing we need to
do is have a format specification including layered architecture) that
allows us to build on. Then, I think, we can focus on specific issues.

Rainer

> -----Original Message-----
> From: Andrew Ross [mailto:[email-masked]] 
> Sent: Wednesday, November 23, 2005 9:51 AM
> To: Rainer Gerhards
> Subject: RE: [Syslog] RE: Message format
> 
> 
> >> Mapping over UDP should be limited to a single message per packet.
> >I agree on that. If we need an ultra-compact UDP delivery, we could
> >later add it in a separate transport mapping.
> 
> Yes, good idea. I doubt anyone will ever want to do this, or 
> at least go to
> the effort of trying to get it drafted into an RFC ;-)
> 
> >> When mapping over plain TCP I believe we should limit the 
> >> total message size
> >> to 65507 bytes (to keep it compatible with UDP) and delimit 
> >> each message
> >> stream with an LF, or CRLF. Either delimiter would work for me.
> 
> >I would prefer not to restart the size discussion at this 
> point. I think
> >the current compromise (everyone must support 2K, anyone 
> might support
> >as much as he likes) is sufficient for most, if not all, 
> cases. I would
> >not like to see an application to become non-compliant just 
> because it
> >needs to transmit 65508 bytes inside a message.
> 
> <SOAPBOX>
> I realise this should have been brought up earlier in the 
> draft process,
> however, I would really like to see a limit on the message 
> size so that it
> is directly compatible with UDP. If we allow an opened ended 
> message size,
> people *will* use it for non syslog related things. I feel 
> that any message
> longer than will fit into a UDP packet should be broken into 
> two or more
> separate messages by the sender, even if sent over TCP. This 
> allows me to
> allocate a maximum known buffer size for incoming TCP 
> messages. There is a
> potential for huge messages filling the memory and memory 
> buffer overflows
> happening if the messages are not limited in size. "Syslog" 
> is meant to be a
> human readable system log message. Anything longer (including 
> binary crash
> dumps or other things people misuse syslog for) should be broken into
> separate messages by the sender, or sent over a different protocol.
> </SOAPBOX>
> 
> I think we should keep syslog simple and flexible, but not at 
> the expense of
> making it handle things it was never meant for. If a message 
> needs to be
> broken into many chunks, the SD-ID tags could be used to tie all the
> messages together again by the parser. The syslog receiver or 
> relay will
> just handle them as separate messages and not even know they 
> were split.
> This makes things so much simpler.
> 
> Cheers
> 
> Andrew
> 
> 

_______________________________________________
Syslog mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/syslog

---

From: "Andrew Ross" <[email-masked]>
Date: Thu, 24 Nov 2005 12:07:45 +1300

Hi Rainer,

I concur on the message size issue. Let's leave it as it stands until we get
to a TCP mapping. Can you mention in your document that the real life max
UDP size was found to be 4K bytes? I think this is a valuable finding and
would save a lot of hair pulling on the part of implementers.

Cheers

Andrew

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
On Behalf Of Rainer Gerhards
Sent: Wednesday, 23 November 2005 10:00 p.m.
To: [email-masked]
Cc: [email-masked]
Subject: RE: [Syslog] RE: Message format

Andrew,

Rainer