Subject: RE: [Syslog] #2, max message size

From: "Rainer Gerhards" <[email-masked]>
Date: Wed, 30 Nov 2005 18:00:45 +0100

Darren,

I violently object your argument and leave it up to the rest of the WG
what should be done. I respect your argument, but I do not like to
re-iterate this forever.

One time we have discussed this was in October 2004:

http://www.syslog.cc/ietf/autoarc/msg01289.html

If you browse the archive, you will find several other ocasions where
this was discussed.

What's your actual recommendation? Do not say anything and leave it the
implementor to guess what is OK? After all, its the implementor's
problem if the message can not be transmitted. Or do you prefer to
define an API where the lower layer tells the upper layer what
capabilities it has? Maybe MTU discovery? Or an app-level ack? I think
all of these options have already been discussed during the past years.
What is now in is a (fragile) consensus, but if only you do not like it,
I think we should go ahead and leave an unhappy Darren behind. If only I
insist on it, we can also go ahead and leave an unhappy Rainer behind.
But we need to go ahead!

Rainer 

> -----Original Message-----
> From: Darren Reed [mailto:[email-masked]] 
> Sent: Wednesday, November 30, 2005 4:49 PM
> To: Rainer Gerhards
> Cc: [email-masked]
> Subject: Re: [Syslog] #2, max message size
> 
> > Darren,
> > 
> > > The only place a message size limit should be specified is in 
> > > a transport
> > > mapping.  If it's in -15 then it should be removed.  Limits 
> > > of all sizes
> > > and types do nothing but contribute to aging of a protocol.
> > 
> > -protocol-15 is a compromise after a very long discussion. It says:
> > 
> > -----
> >    A receiver MUST be able to accept messages up to and 
> including 480
> >    octets in length.  For interoperability reasons, all receiver
> >    implementations SHOULD be able to accept messages up to 
> and including
> >    2,048 octets in length.
> > 
> >    If a receiver receives a message with a length larger than 2,048
> >    octets, or larger than it supports, the receiver MAY discard the
> >    message or truncate the payload.
> > -----
> > 
> > I think this text is useful. It keeps the door open for any size
> > messages while still allowing it to be restricted by the transport
> > mappings and individual implementations (e.g. on low-end embedded
> > devices). It cautions implementors against being too 
> verbose but also
> > sets a lower limit that each implementation can assume to 
> be received.
> > 
> > I think we should continue to use this text. Do you agree?
> 
> No.  That text doesn't belong in this draft.
> 
> Darren
> 

_______________________________________________
Syslog mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/syslog

---

From: "Moehrke, John \(GE Healthcare\)" <[email-masked]>
Date: Wed, 30 Nov 2005 11:13:05 -0600

Shouldn't the MTU be defined by the binding to the transport? I fail to
see why the protocol, unbound to a transport, needs to have a limit. 

> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of Rainer Gerhards
> Sent: Wednesday, November 30, 2005 11:01 AM
> To: Darren Reed
> Cc: [email-masked]
> Subject: RE: [Syslog] #2, max message size
> 
> Darren,
>

---

From: "Moehrke, John \(GE Healthcare\)" <[email-masked]>
Date: Wed, 30 Nov 2005 12:51:12 -0600

I understand the arguments that have been used, yes. I understand that
in the end there will be a small practical MTU when transmitting over
UDP (I would like it to be the 4k that your experiments determined, not
2K, or worse 480). I don't argue with that truth. I very simply fail to
see why this is a syslog-protocol layer and not a transport-binding
specification. 

Side comment, not necessary for the document at hand: I also don't
understand what I would do different if I could determine (in your words
negotiate) what the MTU actually was. I have a message that needs to be
sent, it will either make it or not. Knowing a dynamically determined
MTU is of no value as I can't make my message smaller, it is what it is.
If it gets fragmented, I hope it will be reassembled. If it is big, I
hope that the receiver offers up a big enough buffer to it's socket
library. It is all hope at this point. I am ok with hope, I just don't
want you to limit my ability to hope.

John

> From: Rainer Gerhards [mailto:[email-masked]] 
> Sent: Wednesday, November 30, 2005 11:37 AM
> To: Moehrke, John (GE Healthcare)
> Cc: [email-masked]
> John,
> the issue is the simplex nature of syslog. With syslog (other 
> than with
> almost all other protocols), you send a message and need to 
> *hope* that
> the recipient can receive it. There is also no negotiation 
> phase. So you
> need to send it blindly. For example, if you send an IHE 2K 
> message, but
> the receiver does just support 1K, you will loose 1K without 
> knowing it.
> As such, it is vital to know at least a "safe size" that you 
> can use and
> know that it can be processed (if it makes it to the recipient). We
> selected 480 octets because that fits into an unfragmented 
> UDP packet in
> any case. I know that's too few for IHE, but for network 
> management - an
> important use case for syslog - this is very often sufficient. This is
> the reason why a simplex protocol like syslog (send it and 
> forget it ;))
> should provide some guidance about lower limits. Keep in mind 
> that there
> is NO upper limit - this is done in the transport mappings and the
> implementations. So if you need 32K for IHE, that is perfectly well
> (-transport-UDP, I think, suppport 64K - but you know the practical
> limits).
> 
> Does this clarify?
> 
> Rainer
> 
> > From: Moehrke, John (GE Healthcare) 
> [mailto:[email-masked]] 
> > Sent: Wednesday, November 30, 2005 6:13 PM
> > To: Rainer Gerhards; Darren Reed
> > Cc: [email-masked]

---

From: "Rainer Gerhards" <[email-masked]>
Date: Wed, 30 Nov 2005 21:47:56 +0100

John,

> I understand the arguments that have been used, yes. I 
> understand that in the end there will be a small practical 
> MTU when transmitting over UDP (I would like it to be the 4k 
> that your experiments determined, not 2K, or worse 480). I 
> don't argue with that truth. I very simply fail to see why 
> this is a syslog-protocol layer and not a transport-binding 
> specification. 
> 
> Side comment, not necessary for the document at hand: I also 
> don't understand what I would do different if I could 
> determine (in your words
> negotiate) what the MTU actually was.

At least three options
- abbreviate things (in human-readable text)
- truncate things you know to be less important
- do application level segmentation

> I have a message that 
> needs to be sent, it will either make it or not. Knowing a 
> dynamically determined MTU is of no value as I can't make my 
> message smaller, it is what it is. If it gets fragmented, I 
> hope it will be reassembled. If it is big, I hope that the 
> receiver offers up a big enough buffer to it's socket 
> library. It is all hope at this point. I am ok with hope, I 
> just don't want you to limit my ability to hope.

-protocol-15 does not limit your abilit to hope. It limits your worst
case, because you know what minimum length support you can expect. ;)

Rainer

---

From: "Moehrke, John \(GE Healthcare\)" <[email-masked]>
Date: Wed, 30 Nov 2005 15:33:39 -0600

Rainer,

Are you suggesting that these 'options' be done in real-time, or at
my-application design time? 

At my-application design time, I don't necessarily know what size
strings will come out For example: I might want to have a simple string
that indicates that a specified user logged in at a specified
workstation. For things like "username", I might have quite a long dn
(1024), and the workstation might also have a very long FQDN (255). Am I
to not have this type of a string because it might very clearly be more
than 480 characters? The likelihood of these two values being this big
is low, so I will probably go ahead. 

Even during real-time, why should I preemptively truncate. I figure send
it. It just might make it through normal (Section 8.3). Trying to add
the logic to more intelligently truncate seems unlikely.

The instructions in section 6.1 seem to have more to do with internal
design than any interoperability. Section 8.3 even says go ahead and try
to send the packet, making it even more clear this has little to do with
interoperability. It is this 'instructions on how to design your syslog
receiver' that I object to being normative. There is no such
instructions in other protocols that I have reviewed, and I will readily
admit that although I am intimate with dozens there are millions of
protocols that I don't know at all.

It is these reasons why I recommend that the content of section 6.1/8.3
be moved to the udp-protocol binding, and be included as guidelines and
not normative text. Yes, I understand that if it is in this layer, the
application writer may never see it. That is just as likely of a risk as
any of the others mentioned in this string of emails.

All that said, I will bow to the will of the more committed. I really am
a very interested observer, far more interested in this standard being
written and implemented than I am worried about the standard being
written in any specific way. 

John