From mailman-bounces@ietf.org Fri Oct 1 05:43:29 2004 Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id FAA11149 for ; Fri, 1 Oct 2004 05:43:29 -0400 (EDT) Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CDK5G-0002vM-97 for syslog-archive@ietf.org; Fri, 01 Oct 2004 05:52:14 -0400 Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CDJSh-0004Qz-C0 for syslog-archive@ietf.org; Fri, 01 Oct 2004 05:12:23 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Subject: lists.ietf.org mailing list memberships reminder From: mailman-owner@ietf.org To: syslog-archive@ietf.org X-No-Archive: yes Message-ID: Date: Fri, 01 Oct 2004 05:02:14 -0400 Precedence: bulk X-BeenThere: mailman@lists.ietf.org X-Mailman-Version: 2.1.5 List-Id: Mailman site list X-List-Administrivia: yes Sender: mailman-bounces@ietf.org Errors-To: mailman-bounces@ietf.org X-Spam-Score: 0.3 (/) X-Scan-Signature: 3e15cc4fdc61d7bce84032741d11c8e5 Content-Transfer-Encoding: 7bit This is a reminder, sent out once a month, about your lists.ietf.org mailing list memberships. It includes your subscription info and how to use it to change it or unsubscribe from a list. You can visit the URLs to change your membership status or configuration, including unsubscribing, setting digest-style delivery or disabling delivery altogether (e.g., for a vacation), and so on. In addition to the URL interfaces, you can also use email to make such changes. For more info, send a message to the '-request' address of the list (for example, mailman-request@lists.ietf.org) containing just the word 'help' in the message body, and an email message will be sent to you with instructions. ********************************************************************** NOTE WELL: Any submission to the IETF intended by the Contributor for publication as all or part of an IETF Internet-Draft or RFC and any statement made within the context of an IETF activity is considered an "IETF Contribution". Such statements include oral statements in IETF sessions, as well as written and electronic communications made at any time or place, which are addressed to: o the IETF plenary session, o any IETF working group or portion thereof, o the IESG, or any member thereof on behalf of the IESG, o the IAB or any member thereof on behalf of the IAB, o any IETF mailing list, including the IETF list itself, any working group or design team list, or any other list functioning under IETF auspices, o the RFC Editor or the Internet-Drafts function All IETF Contributions are subject to the rules of RFC 3667 and RFC 3668. Statements made outside of an IETF session, mailing list or other function, that are clearly not intended to be input to an IETF activity, group or function, are not IETF Contributions in the context of this notice. Please consult RFC 3667 for details. ******************************************************************************* If you have questions, problems, comments, etc, send them to mailman-owner@lists.ietf.org. Thanks! Passwords for syslog-archive@ietf.org: List Password // URL ---- -------- syslog@lists.ietf.org abzuka https://www1.ietf.org/mailman/options/syslog/syslog-archive%40ietf.org From mailman-bounces@willers.employees.org Fri Oct 1 08:12:32 2004 Received: from willers.employees.org (willers.employees.org [192.83.249.36]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id IAA22583 for ; Fri, 1 Oct 2004 08:12:32 -0400 (EDT) Received: from willers.employees.org (localhost.employees.org [127.0.0.1]) by willers.employees.org (Postfix) with ESMTP id 983D45C822 for ; Fri, 1 Oct 2004 05:02:00 -0700 (PDT) MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Subject: www.employees.org mailing list memberships reminder From: mailman-owner@willers.employees.org To: syslog-archive@ietf.org X-No-Archive: yes Message-ID: Date: Fri, 01 Oct 2004 05:00:46 -0700 Precedence: bulk X-BeenThere: mailman@www.employees.org X-Mailman-Version: 2.1.4 List-Id: mailman.www.employees.org X-List-Administrivia: yes Sender: mailman-bounces@willers.employees.org Errors-To: mailman-bounces@willers.employees.org Content-Transfer-Encoding: 7bit This is a reminder, sent out once a month, about your www.employees.org mailing list memberships. It includes your subscription info and how to use it to change it or unsubscribe from a list. You can visit the URLs to change your membership status or configuration, including unsubscribing, setting digest-style delivery or disabling delivery altogether (e.g., for a vacation), and so on. In addition to the URL interfaces, you can also use email to make such changes. For more info, send a message to the '-request' address of the list (for example, mailman-request@www.employees.org) containing just the word 'help' in the message body, and an email message will be sent to you with instructions. If you have questions, problems, comments, etc, send them to mailman-owner@www.employees.org. Thanks! Passwords for syslog-archive@lists.ietf.org: List Password // URL ---- -------- syslog-sec@www.employees.org widuza http://www.employees.org/mailman/options/syslog-sec/syslog-archive%40lists.ietf.org From syslog-sec-bounces@willers.employees.org Tue Oct 5 17:12:58 2004 Received: from willers.employees.org (willers.employees.org [192.83.249.36]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA12630 for ; Tue, 5 Oct 2004 17:12:57 -0400 (EDT) Received: from willers.employees.org (localhost.employees.org [127.0.0.1]) by willers.employees.org (Postfix) with ESMTP id 69D855C765; Tue, 5 Oct 2004 14:12:58 -0700 (PDT) X-Original-To: syslog-sec@employees.org Delivered-To: syslog-sec@employees.org Received: from sj-iport-1.cisco.com (sj-iport-1-in.cisco.com [171.71.176.70]) by willers.employees.org (Postfix) with ESMTP id 8ED585C724 for ; Tue, 5 Oct 2004 13:41:28 -0700 (PDT) Received: from sj-core-1.cisco.com (171.71.177.237) by sj-iport-1.cisco.com with ESMTP; 05 Oct 2004 13:49:01 -0700 X-BrightmailFiltered: true Received: from flask.cisco.com (IDENT:mirapoint@flask.cisco.com [161.44.122.62]) by sj-core-1.cisco.com (8.12.10/8.12.6) with ESMTP id i95KfMEE017116; Tue, 5 Oct 2004 13:41:23 -0700 (PDT) Received: from aokmiansw2k ([161.44.64.231]) by flask.cisco.com (MOS 3.4.6-GR) with ESMTP id AMB53711; Tue, 5 Oct 2004 16:41:23 -0400 (EDT) From: "Anton Okmianski" To: "'Rainer Gerhards'" , , Subject: RE: [Syslog-sec] Sturctured Data & Unicode Date: Tue, 5 Oct 2004 16:41:26 -0400 Message-ID: <000201c4ab1b$af814d90$e7402ca1@amer.cisco.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.6626 Importance: Normal In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA02FF5C@grfint2.intern.adiscon.com> X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4939.300 X-Mailman-Approved-At: Tue, 05 Oct 2004 14:12:57 -0700 X-BeenThere: syslog-sec@www.employees.org X-Mailman-Version: 2.1.4 Precedence: list List-Id: syslog-sec.www.employees.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: syslog-sec-bounces@willers.employees.org Errors-To: syslog-sec-bounces@willers.employees.org Content-Transfer-Encoding: quoted-printable Rainer: I second David's opinion. I don't see how control characters in tag names are any different than control characters in message content which we allow. =20 BTW, I am back from 2 week vacation. Sorry for a long silence on the message size and fragmentation issue. I am going to revisit it this week. I am consulting with more people.=20 Thanks, Anton.=20 > -----Original Message----- > From: syslog-sec-bounces@willers.employees.org=20 > [mailto:syslog-sec-bounces@willers.employees.org] On Behalf=20 > Of Rainer Gerhards > Sent: Monday, September 20, 2004 12:48 PM > To: ietfdbh@comcast.net; syslog-sec@employees.org > Subject: RE: [Syslog-sec] Sturctured Data & Unicode >=20 >=20 > David, >=20 > I mostly agree with you. I have updated the draft in your=20 > spirit. There is one issue left - that is control characters=20 > in the structured data names. I would like to make sure that=20 > only printable characters are used for names. But what does=20 > printable mean in Unicode? As of my understanding, stating=20 > something like "printable Unicode characters" does not really=20 > make sense, as what printable is largely depends on the=20 > installed character sets. Or should I simply outrule the=20 > US-ASCII control character range - but this leaves e.g.=20 > Japanese width control characters open... >=20 > I would appreciate any advise you can offer. >=20 > Many thanks for all of your great help!=20 >=20 > Rainer >=20 > > -----Original Message----- > > From: David B Harrington [mailto:ietfdbh@comcast.net] > > Sent: Monday, September 20, 2004 5:35 PM > > To: Rainer Gerhards; syslog-sec@employees.org > > Subject: RE: [Syslog-sec] Sturctured Data & Unicode > >=20 > > Hi, > >=20 > > If you are suggesting what I think you are suggesting, I=20 > disagree with=20 > > this approach. > >=20 > > What I hear you saying is, we will allow, say, a Japanese=20 > operator to=20 > > express **values** in a Japanese langauge, but they need to learn a=20 > > US-ASCII based language to read the **name** for the data=20 > point. This=20 > > strikes me as being rather biased in favor of people that use=20 > > languages that are US-ASCII based. It would be easier for that=20 > > Japanese operator to be able to design their name-value=20 > pairs fully in=20 > > Japanese, both name and value. > >=20 > > I also don't see why such a restriction is necessary. There is no=20 > > extra cost borne by US-ASCII language based users for allowing a=20 > > Unicode format, since US-ASCII is a subset of Unicode and no extra > > bits on the wire are required for their US-ASCII name-value=20 > pairs when=20 > > written in Unicode format. > >=20 > > If you want to ensure that IETF-STANDARD name-value pairs are=20 > > understandable to the largest community, then requiring that IETF=20 > > standards-track names be specified in US-ASCII, that would=20 > make some=20 > > sense, but I think it probably should go further and=20 > require that the=20 > > names be in English, just as it is a requirement that=20 > Internet-Drafts=20 > > and RFCs be written in English. But this should NOT mean=20 > that the name=20 > > field, if it can also contain names that are not standards-track,=20 > > should be restricted to support only US-ASCII format. > >=20 > > I am also a bit concerned that requiring names to only support=20 > > US-ASCII will lead implementors to only support US-ASCII for the=20 > > values as well, and that would be unfair to a large, and growing,=20 > > percentage of Internet users. > >=20 > > So I tend to disagree with the suggestion to limit the name to=20 > > US-ASCII. > >=20 > > David Harrington > > dbharrington@comcast.net > >=20 > > -----Original Message----- > > From: syslog-sec-bounces@www.employees.org > > [mailto:syslog-sec-bounces@www.employees.org] On Behalf Of Rainer=20 > > Gerhards > > Sent: Monday, September 20, 2004 9:47 AM > > To: syslog-sec@employees.org > > Subject: [Syslog-sec] Sturctured Data & Unicode > >=20 > > Hi list, > >=20 > > STRUCTURED-DATA is currently specified as All-Unicode.=20 > STRUCTURED-DATA=20 > > consists of structured data elements. Each of them consists=20 > of a name=20 > > and name-value pairs. > >=20 > > After going through some scenarios, I think we do NOT need=20 > to support=20 > > full Unicode in the name tags. So I propose we allow=20 > Unicode only in=20 > > the values and limit the name tags to US-ASCII. If nobody=20 > objects, I=20 > > will edit the draft in this way. > >=20 > > Thanks, > > Rainer > > _______________________________________________ > > Syslog-sec mailing list > > Syslog-sec@www.employees.org=20 > > http://www.employees.org/mailman/listinfo/syslog-sec > >=20 > >=20 > >=20 > _______________________________________________ > Syslog-sec mailing list > Syslog-sec@www.employees.org=20 > http://www.employees.org/mailman/listinfo/syslog-sec >=20 _______________________________________________ Syslog-sec mailing list Syslog-sec@www.employees.org http://www.employees.org/mailman/listinfo/syslog-sec From syslog-sec-bounces@willers.employees.org Tue Oct 12 21:16:34 2004 Received: from willers.employees.org (willers.employees.org [192.83.249.36]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA05514 for ; Tue, 12 Oct 2004 21:16:33 -0400 (EDT) Received: from willers.employees.org (localhost.employees.org [127.0.0.1]) by willers.employees.org (Postfix) with ESMTP id 5F6865C763; Tue, 12 Oct 2004 18:16:33 -0700 (PDT) X-Original-To: syslog-sec@employees.org Delivered-To: syslog-sec@employees.org Received: from sj-iport-1.cisco.com (sj-iport-1-in.cisco.com [171.71.176.70]) by willers.employees.org (Postfix) with ESMTP id A27B25C78D for ; Tue, 12 Oct 2004 14:25:16 -0700 (PDT) Received: from sj-core-1.cisco.com (171.71.177.237) by sj-iport-1.cisco.com with ESMTP; 12 Oct 2004 14:34:03 -0700 X-BrightmailFiltered: true Received: from flask.cisco.com (IDENT:mirapoint@flask.cisco.com [161.44.122.62]) by sj-core-1.cisco.com (8.12.10/8.12.6) with ESMTP id i9CLP1k2013044 for ; Tue, 12 Oct 2004 14:25:02 -0700 (PDT) Received: from aokmiansw2k ([161.44.64.231]) by flask.cisco.com (MOS 3.4.6-GR) with ESMTP id AMF82113; Tue, 12 Oct 2004 17:25:02 -0400 (EDT) Message-Id: <200410122125.AMF82113@flask.cisco.com> From: "Anton Okmianski" To: Date: Tue, 12 Oct 2004 17:25:02 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Office Outlook, Build 11.0.5510 Thread-Index: AcSwoe8xQ3+YxqsUQ8iCTkpgGQfmqg== X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4939.300 X-Mailman-Approved-At: Tue, 12 Oct 2004 18:16:32 -0700 Subject: [Syslog-sec] UDP message size issue proposal X-BeenThere: syslog-sec@www.employees.org X-Mailman-Version: 2.1.4 Precedence: list List-Id: syslog-sec.www.employees.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: syslog-sec-bounces@willers.employees.org Errors-To: syslog-sec-bounces@willers.employees.org Content-Transfer-Encoding: quoted-printable Hi! Sorry for a long delay on this issue - I was on a 2 week vacation. I = have spoken with a number of TCP/UDP/IP experts regarding the sizing = issue. I am ready to propose the following changes: 1. Syslog-protocol will state that the max message will be defined by = the transport layer. =20 2. Syslog-transport-udp will support messages up to maximum UDP datagram = size of 64K. UDP is a very bad choice for large message transmissions, = so it does not make sense for us to stretch it by adding our own = fragmentation without other transmission control features such as found = in TCP. 3. Syslog-transport-udp will rely on IP fragmentation and we will get = rid of "proprietary" fragmentation which was designed to handle messages = over 64K and deal with various non-compliant network hosts. =20 4. Syslog-transport-udp will recommend sending messages within the = boundaries of prevalent MTU size on a given network to be safe. It will = recommend Ethernet's 1500 bytes less headers and will draw reader = attention to the minimum MTU size hosts on the network are required to = support for IPv6 (576 bytes) and IPv6 (1280 bytes). =20 5. Path MTU discovery may not work robustly and some TCP/IP stacks may = not support UDP packets of full 64K size and truncate them. We will = mention this and bite this bullet. We should not restrict the protocol = due to incompliant implementations because it is a moving target and = penalizes compliant implementations with extra overhead. The above size = recommendation would partially deal with this, but leave the final = choice up to the administrator. =20 6. We will get rid of most syslog transport headers for UDP as they will = no longer be needed. The only thing that will be left is the transport = protocol version prefixed to every syslog message. Should we even = bother with that? =20 This is a major change to the syslog-transport-udp. I'd like to get = positive feedback before I proceed with this update. The best part is = that if we all agree on the above, the next draft will be 1/3 of the = size -- easier read for you. :) Thanks, Anton.=20 =20 _______________________________________________ Syslog-sec mailing list Syslog-sec@www.employees.org http://www.employees.org/mailman/listinfo/syslog-sec From syslog-sec-bounces@willers.employees.org Wed Oct 13 09:08:24 2004 Received: from willers.employees.org (willers.employees.org [192.83.249.36]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA24412 for ; Wed, 13 Oct 2004 09:08:23 -0400 (EDT) Received: from willers.employees.org (localhost.employees.org [127.0.0.1]) by willers.employees.org (Postfix) with ESMTP id C01255C764; Wed, 13 Oct 2004 06:08:19 -0700 (PDT) X-Original-To: syslog-sec@employees.org Delivered-To: syslog-sec@employees.org Received: from mail.hq.adiscon.com (mail.hq.adiscon.com [217.6.190.188]) by willers.employees.org (Postfix) with ESMTP id 58EF25C725 for ; Wed, 13 Oct 2004 02:13:34 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by mail.hq.adiscon.com (Postfix) with ESMTP id 152E29C757; Wed, 13 Oct 2004 11:30:28 +0200 (CEST) Received: from mail.hq.adiscon.com ([127.0.0.1]) by localhost (grfdeb [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 15420-08; Wed, 13 Oct 2004 11:30:05 +0200 (CEST) Received: from grfint2.intern.adiscon.com (grfint2 [172.19.0.6]) by mail.hq.adiscon.com (Postfix) with ESMTP id 5B53B9C755; Wed, 13 Oct 2004 11:30:05 +0200 (CEST) Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: [Syslog-sec] UDP message size issue proposal X-MimeOLE: Produced By Microsoft Exchange V6.5.7226.0 Date: Wed, 13 Oct 2004 11:17:06 +0200 Message-ID: <577465F99B41C842AAFBE9ED71E70ABA030179@grfint2.intern.adiscon.com> Thread-Topic: [Syslog-sec] UDP message size issue proposal Thread-Index: AcSwoe8xQ3+YxqsUQ8iCTkpgGQfmqgAYjRlA From: "Rainer Gerhards" To: "Anton Okmianski" , X-Virus-Scanned: by amavisd-new-20030616-p7 (Debian) at adiscon.com X-Mailman-Approved-At: Wed, 13 Oct 2004 06:08:18 -0700 X-BeenThere: syslog-sec@www.employees.org X-Mailman-Version: 2.1.4 Precedence: list List-Id: syslog-sec.www.employees.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: syslog-sec-bounces@willers.employees.org Errors-To: syslog-sec-bounces@willers.employees.org Content-Transfer-Encoding: quoted-printable Anton, I agree to your conclusion. Although I am the one who always voted for larger sizes, the discussion has shown that this yields to unnecessary complexity. I can satisfy my needs with a vendor-extension structured data element, which I think shows the flexibility of the new drafts. So I support your move. I would tend to even remove the transport version header. If there is need to, we could always include that in a later release (e.g. "v1", which would make it clearly distingshable from the current frame format). I see little need to do so, though. Regarding -protocol, I think we should still keep an upper limit in the spec. Even I don't see any reason why a syslog message should go above 16MB. So for -protocol, I propose the following new text: #### 5.1 Message Length The maximum length of any syslog message is 16,777,216 octets. Any receiver receiving a larger message MUST discard the message. A diagnostic message SHOULD be logged in this case. A receiver MUST be able to accept messages that are 480 octets (or less) in length. A receiver SHOULD be able to accept messages that are 65,535 octets (or less) in length. It is RECOMMENDED that receivers be able to accept messages up to the maximum message length of 16,777,216 octets. If a receiver receives a message within the maximum length, but with a length larger than it handles, the receiver MAY discard or truncate it. A transport mapping MAY define a maximum length below the one specified here. Each transport mapping MUST support a maximum size of 480 octets or more. #### If there is agreement on this, I can post a new version of -protocol. That version will also include some changes made thanks to Andrew Ross comments (sent via private mail). I have finished this version. It is available for immediate posting, but I would like to wait for agreement on the size issue (at least if that can be reached quickly). Rainer > -----Original Message----- > From: syslog-sec-bounces@www.employees.org=20 > [mailto:syslog-sec-bounces@www.employees.org] On Behalf Of=20 > Anton Okmianski > Sent: Tuesday, October 12, 2004 11:25 PM > To: syslog-sec@employees.org > Subject: [Syslog-sec] UDP message size issue proposal >=20 > Hi! >=20 > Sorry for a long delay on this issue - I was on a 2 week=20 > vacation. I have spoken with a number of TCP/UDP/IP experts=20 > regarding the sizing issue. I am ready to propose the=20 > following changes: >=20 > 1. Syslog-protocol will state that the max message will be=20 > defined by the transport layer. =20 >=20 > 2. Syslog-transport-udp will support messages up to maximum=20 > UDP datagram size of 64K. UDP is a very bad choice for large=20 > message transmissions, so it does not make sense for us to=20 > stretch it by adding our own fragmentation without other=20 > transmission control features such as found in TCP. >=20 > 3. Syslog-transport-udp will rely on IP fragmentation and we=20 > will get rid of "proprietary" fragmentation which was=20 > designed to handle messages over 64K and deal with various=20 > non-compliant network hosts. =20 >=20 > 4. Syslog-transport-udp will recommend sending messages=20 > within the boundaries of prevalent MTU size on a given=20 > network to be safe. It will recommend Ethernet's 1500 bytes=20 > less headers and will draw reader attention to the minimum=20 > MTU size hosts on the network are required to support for=20 > IPv6 (576 bytes) and IPv6 (1280 bytes). =20 >=20 > 5. Path MTU discovery may not work robustly and some TCP/IP=20 > stacks may not support UDP packets of full 64K size and=20 > truncate them. We will mention this and bite this bullet. =20 > We should not restrict the protocol due to incompliant=20 > implementations because it is a moving target and penalizes=20 > compliant implementations with extra overhead. The above size=20 > recommendation would partially deal with this, but leave the=20 > final choice up to the administrator. =20 >=20 > 6. We will get rid of most syslog transport headers for UDP=20 > as they will no longer be needed. The only thing that will=20 > be left is the transport protocol version prefixed to every=20 > syslog message. Should we even bother with that? =20 >=20 > This is a major change to the syslog-transport-udp. I'd like=20 > to get positive feedback before I proceed with this update. =20 > The best part is that if we all agree on the above, the next=20 > draft will be 1/3 of the size -- easier read for you. :) >=20 > Thanks, > Anton.=20 > =20 >=20 > _______________________________________________ > Syslog-sec mailing list > Syslog-sec@www.employees.org > http://www.employees.org/mailman/listinfo/syslog-sec >=20 _______________________________________________ Syslog-sec mailing list Syslog-sec@www.employees.org http://www.employees.org/mailman/listinfo/syslog-sec From syslog-sec-bounces@willers.employees.org Wed Oct 13 09:08:47 2004 Received: from willers.employees.org (willers.employees.org [192.83.249.36]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA24448 for ; Wed, 13 Oct 2004 09:08:47 -0400 (EDT) Received: from willers.employees.org (localhost.employees.org [127.0.0.1]) by willers.employees.org (Postfix) with ESMTP id 97B195C7B4; Wed, 13 Oct 2004 06:08:20 -0700 (PDT) X-Original-To: syslog-sec@employees.org Delivered-To: syslog-sec@employees.org Received: from mail.hq.adiscon.com (mail.hq.adiscon.com [217.6.190.188]) by willers.employees.org (Postfix) with ESMTP id 1B5B45C725 for ; Wed, 13 Oct 2004 02:16:12 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by mail.hq.adiscon.com (Postfix) with ESMTP id AD42E9C757 for ; Wed, 13 Oct 2004 11:33:16 +0200 (CEST) Received: from mail.hq.adiscon.com ([127.0.0.1]) by localhost (grfdeb [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 15531-02 for ; Wed, 13 Oct 2004 11:32:54 +0200 (CEST) Received: from grfint2.intern.adiscon.com (grfint2 [172.19.0.6]) by mail.hq.adiscon.com (Postfix) with ESMTP id AB85A9C755 for ; Wed, 13 Oct 2004 11:32:54 +0200 (CEST) Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-MimeOLE: Produced By Microsoft Exchange V6.5.7226.0 Date: Wed, 13 Oct 2004 11:19:55 +0200 Message-ID: <577465F99B41C842AAFBE9ED71E70ABA03017A@grfint2.intern.adiscon.com> Thread-Topic: syslog-interntional Thread-Index: AcSxBc2iqdRAukYvTnylqzvI7/u6vA== From: "Rainer Gerhards" To: X-Virus-Scanned: by amavisd-new-20030616-p7 (Debian) at adiscon.com X-Mailman-Approved-At: Wed, 13 Oct 2004 06:08:18 -0700 Subject: [Syslog-sec] syslog-interntional X-BeenThere: syslog-sec@www.employees.org X-Mailman-Version: 2.1.4 Precedence: list List-Id: syslog-sec.www.employees.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: syslog-sec-bounces@willers.employees.org Errors-To: syslog-sec-bounces@willers.employees.org Content-Transfer-Encoding: quoted-printable Hi List, this is more or less a housekeeping call. I think -protocol addresses all internationalization issues now. We still lists syslog-international on the WG home page. I think now would be the right time to abandon that project, as there is no more need for it. Rainer _______________________________________________ Syslog-sec mailing list Syslog-sec@www.employees.org http://www.employees.org/mailman/listinfo/syslog-sec From syslog-sec-bounces@willers.employees.org Wed Oct 13 10:46:30 2004 Received: from willers.employees.org (willers.employees.org [192.83.249.36]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA04224 for ; Wed, 13 Oct 2004 10:46:30 -0400 (EDT) Received: from willers.employees.org (localhost.employees.org [127.0.0.1]) by willers.employees.org (Postfix) with ESMTP id 664C75C7CD; Wed, 13 Oct 2004 07:46:26 -0700 (PDT) X-Original-To: syslog-sec@employees.org Delivered-To: syslog-sec@employees.org Received: from rtp-iport-1.cisco.com (rtp-iport-1.cisco.com [64.102.122.148]) by willers.employees.org (Postfix) with ESMTP id 992085C722 for ; Wed, 13 Oct 2004 07:37:20 -0700 (PDT) Received: from rtp-core-2.cisco.com (64.102.124.13) by rtp-iport-1.cisco.com with ESMTP; 13 Oct 2004 10:57:15 -0400 X-BrightmailFiltered: true Received: from flask.cisco.com (IDENT:mirapoint@flask.cisco.com [161.44.122.62]) by rtp-core-2.cisco.com (8.12.10/8.12.6) with ESMTP id i9DEbHKm016896; Wed, 13 Oct 2004 10:37:17 -0400 (EDT) Received: from aokmiansw2k ([161.44.64.231]) by flask.cisco.com (MOS 3.4.6-GR) with ESMTP id AMG24306; Wed, 13 Oct 2004 10:37:15 -0400 (EDT) Message-Id: <200410131437.AMG24306@flask.cisco.com> From: "Anton Okmianski" To: "'Rainer Gerhards'" Subject: RE: [Syslog-sec] UDP message size issue proposal Date: Wed, 13 Oct 2004 10:37:15 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook, Build 11.0.5510 Thread-Index: AcSwoe8xQ3+YxqsUQ8iCTkpgGQfmqgAYjRlAAAr8laA= X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4939.300 In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA030179@grfint2.intern.adiscon.com> X-Mailman-Approved-At: Wed, 13 Oct 2004 07:46:25 -0700 Cc: syslog-sec@employees.org X-BeenThere: syslog-sec@www.employees.org X-Mailman-Version: 2.1.4 Precedence: list List-Id: syslog-sec.www.employees.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: syslog-sec-bounces@willers.employees.org Errors-To: syslog-sec-bounces@willers.employees.org Content-Transfer-Encoding: 7bit Rainer: Glad you liked the latest proposal. I received two other votes for it as well (from Andrew Ross and John Moehrke) and no disagreements. So, I will go ahead with the proposal. I will remove version field from udp transport as well. I agree that in the unlikely scenario that we would need a new version of UDP transport, we can add the version field at the very front which does not conflict with syslog-protocol headers. One clarification question on suggested text below. > #### > 5.1 Message Length > > The maximum length of any syslog message is 16,777,216 octets. Any > receiver receiving a larger message MUST discard the message. A > diagnostic message SHOULD be logged in this case. > > A receiver MUST be able to accept messages that are 480 octets (or > less) in length. A receiver SHOULD be able to accept messages that > are 65,535 octets (or less) in length. It is RECOMMENDED that > receivers be able to accept messages up to the maximum > message length > of 16,777,216 octets. > > If a receiver receives a message within the maximum > length, but with > a length larger than it handles, the receiver MAY discard > or truncate > it. > > A transport mapping MAY define a maximum length below the one > specified here. Each transport mapping MUST support a maximum size > of 480 octets or more. > #### You really meant: "Each transport mapping MUST support a *minimum* size of 480 octets or more." Or actually, this should probably be: "Each transport mapping MUST support transmission of messages of at least 480 octets". Note, we are not placing a requirement on sender or receiver here. Just saying that transport protocol must be able to handle messages of this size. Right? Anton. > > If there is agreement on this, I can post a new version of -protocol. > That version will also include some changes made thanks to > Andrew Ross comments (sent via private mail). I have finished > this version. It is available for immediate posting, but I > would like to wait for agreement on the size issue (at least > if that can be reached quickly). > > Rainer > > > -----Original Message----- > > From: syslog-sec-bounces@www.employees.org > > [mailto:syslog-sec-bounces@www.employees.org] On Behalf Of Anton > > Okmianski > > Sent: Tuesday, October 12, 2004 11:25 PM > > To: syslog-sec@employees.org > > Subject: [Syslog-sec] UDP message size issue proposal > > > > Hi! > > > > Sorry for a long delay on this issue - I was on a 2 week > vacation. I > > have spoken with a number of TCP/UDP/IP experts regarding > the sizing > > issue. I am ready to propose the following changes: > > > > 1. Syslog-protocol will state that the max message will be > defined by > > the transport layer. > > > > 2. Syslog-transport-udp will support messages up to maximum UDP > > datagram size of 64K. UDP is a very bad choice for large message > > transmissions, so it does not make sense for us to stretch it by > > adding our own fragmentation without other transmission control > > features such as found in TCP. > > > > 3. Syslog-transport-udp will rely on IP fragmentation and > we will get > > rid of "proprietary" fragmentation which was designed to handle > > messages over 64K and deal with various > > non-compliant network hosts. > > > > 4. Syslog-transport-udp will recommend sending messages within the > > boundaries of prevalent MTU size on a given network to be safe. It > > will recommend Ethernet's 1500 bytes less headers and will > draw reader > > attention to the minimum MTU size hosts on the network are > required to > > support for > > IPv6 (576 bytes) and IPv6 (1280 bytes). > > > > 5. Path MTU discovery may not work robustly and some TCP/IP > stacks may > > not support UDP packets of full 64K size and truncate them. > We will > > mention this and bite this bullet. > > We should not restrict the protocol due to incompliant > implementations > > because it is a moving target and penalizes compliant > implementations > > with extra overhead. The above size recommendation would partially > > deal with this, but leave the > > final choice up to the administrator. > > > > 6. We will get rid of most syslog transport headers for UDP as they > > will no longer be needed. The only thing that will be left is the > > transport protocol version prefixed to every syslog > message. Should > > we even bother with that? > > > > This is a major change to the syslog-transport-udp. I'd > like to get > > positive feedback before I proceed with this update. > > The best part is that if we all agree on the above, the next draft > > will be 1/3 of the size -- easier read for you. :) > > > > Thanks, > > Anton. > > > > > > _______________________________________________ > > Syslog-sec mailing list > > Syslog-sec@www.employees.org > > http://www.employees.org/mailman/listinfo/syslog-sec > > > _______________________________________________ Syslog-sec mailing list Syslog-sec@www.employees.org http://www.employees.org/mailman/listinfo/syslog-sec From syslog-sec-bounces@willers.employees.org Wed Oct 13 10:47:00 2004 Received: from willers.employees.org (willers.employees.org [192.83.249.36]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA04333 for ; Wed, 13 Oct 2004 10:47:00 -0400 (EDT) Received: from willers.employees.org (localhost.employees.org [127.0.0.1]) by willers.employees.org (Postfix) with ESMTP id 2E31B5C7EE; Wed, 13 Oct 2004 07:46:27 -0700 (PDT) X-Original-To: syslog-sec@employees.org Delivered-To: syslog-sec@employees.org Received: from sccrmhc12.comcast.net (sccrmhc12.comcast.net [204.127.202.56]) by willers.employees.org (Postfix) with ESMTP id 582D15C7A6 for ; Wed, 13 Oct 2004 07:38:14 -0700 (PDT) Received: from djyxpy41 (h00104b8ce2a3.ne.client2.attbi.com[24.128.104.220]) by comcast.net (sccrmhc12) with SMTP id <20041013143813012003bqvce>; Wed, 13 Oct 2004 14:38:13 +0000 From: "David B Harrington" To: "'Rainer Gerhards'" , "'Anton Okmianski'" , Subject: RE: [Syslog-sec] UDP message size issue proposal Date: Wed, 13 Oct 2004 10:38:09 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook, Build 11.0.5510 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1441 In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA030179@grfint2.intern.adiscon.com> Thread-Index: AcSwoe8xQ3+YxqsUQ8iCTkpgGQfmqgAYjRlAAAkZb6A= Message-Id: <20041013143814.582D15C7A6@willers.employees.org> X-Mailman-Approved-At: Wed, 13 Oct 2004 07:46:25 -0700 X-BeenThere: syslog-sec@www.employees.org X-Mailman-Version: 2.1.4 Precedence: list Reply-To: ietfdbh@comcast.net List-Id: syslog-sec.www.employees.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: syslog-sec-bounces@willers.employees.org Errors-To: syslog-sec-bounces@willers.employees.org Content-Transfer-Encoding: 7bit Hi, The RFC2119 keyword usage needs to be tightened up. I suggest: >From RFC2119: "MUST This word, or the terms "REQUIRED" or "SHALL", mean that the definition is an absolute requirement of the specification." "SHOULD This word, or the adjective "RECOMMENDED", mean that there may exist valid reasons in particular circumstances to ignore a particular item, but the full implications must be understood and carefully weighed before choosing a different course." MUST is used to specify compliance requirements. Your text " A receiver MUST be able to accept messages that are 480 octets (or less) in length." could be interpreted to mean that I am compliant if I accept only messages of 4 bytes, since 4 bytes would fit within your "480 octets (or less)" requirement. "up to and including 480 octets" is clearer (and consistent with the the same restriction in RFC3417). SHOULD and RECOMMENDED are equivalent. So I suggest that " A receiver SHOULD be able to accept messages that are 65,535 octets (or less) in length. It is RECOMMENDED that receivers be able to accept messages up to the maximum message length of 16,777,216 octets." be reduced to " A receiver SHOULD be able to accept messages up to the maximum message length of 16,777,216 octets." If they cannot support that size, which would be a valid reason to ignore that requirement, they should understand the implications. We don't need the intermediate SHOULD 64k size in the standard, we only need the minimum size and the maximum size. If the WG wants to make the 64k the standard maximum, yet allow implementors to support up to 16,777,216 octets, then the text should specify a SHOULD for 64k, and a MAY for 16,777,216 bytes. Can a receiver truncate the message, or just the payload of the message? If the WG ever s=chooses to add something to the end of the message, such as for security, the text as written would prevent that. I would also suggest reordering this text to move from minimum to maximum: #### 5.1 Message Length A receiver MUST be able to accept messages up to and including 480 octets in length. A receiver SHOULD be able to accept messages up to and including 65,535 octets in length. If a receiver receives a message with a length larger than it supports, the receiver MAY discard the message or truncate the payload. A receiver MAY accept messages up to the maximum message length of 16,777,216 octets. The maximum length of any syslog message is 16,777,216 octets. Any receiver receiving a larger message MUST discard the message. A diagnostic message SHOULD be logged in this case. A transport mapping MAY define a maximum length below the one specified here. Each transport mapping MUST support at least the minimum size of 480 octets, and not exceed the maximum of 16,777,216 octets. #### Why are we specifying a maximum length allowed, and why MUST a larger message be discarded? It is not a matter of interoperability if I ACCEPT a message larger than 16,777,216 octets. Since I can truncate any message to my maxiumum supported size, it isn't even a problem if somebody SENDS me a message larger than that size. So why even mention such a limit? What if ten years from now, a new transport is defined that efficiently moves messages largee than this, and somebody wants to support messages larger than 16,777,216 octets? Why MUST this be prevented? If the WG chooses 64k as the STANDARD maximum, why isn't a diagnostic message required for messages that exceed the standard maximum? Why is it REQUIRED only for messages above 16,777,216 octets? Does this impact on-the-wire interoperability? Wouldn't it make sense to log any message received greater than that supported by the receiver, so the operator realizes that a message was received but the message may have been truncated or discarded? David Harrington dbharrington@comcast.net -----Original Message----- From: syslog-sec-bounces@www.employees.org [mailto:syslog-sec-bounces@www.employees.org] On Behalf Of Rainer Gerhards Sent: Wednesday, October 13, 2004 5:17 AM To: Anton Okmianski; syslog-sec@employees.org Subject: RE: [Syslog-sec] UDP message size issue proposal Anton, I agree to your conclusion. Although I am the one who always voted for larger sizes, the discussion has shown that this yields to unnecessary complexity. I can satisfy my needs with a vendor-extension structured data element, which I think shows the flexibility of the new drafts. So I support your move. I would tend to even remove the transport version header. If there is need to, we could always include that in a later release (e.g. "v1", which would make it clearly distingshable from the current frame format). I see little need to do so, though. Regarding -protocol, I think we should still keep an upper limit in the spec. Even I don't see any reason why a syslog message should go above 16MB. So for -protocol, I propose the following new text: #### 5.1 Message Length The maximum length of any syslog message is 16,777,216 octets. Any receiver receiving a larger message MUST discard the message. A diagnostic message SHOULD be logged in this case. A receiver MUST be able to accept messages that are 480 octets (or less) in length. A receiver SHOULD be able to accept messages that are 65,535 octets (or less) in length. It is RECOMMENDED that receivers be able to accept messages up to the maximum message length of 16,777,216 octets. If a receiver receives a message within the maximum length, but with a length larger than it handles, the receiver MAY discard or truncate it. A transport mapping MAY define a maximum length below the one specified here. Each transport mapping MUST support a maximum size of 480 octets or more. #### If there is agreement on this, I can post a new version of -protocol. That version will also include some changes made thanks to Andrew Ross comments (sent via private mail). I have finished this version. It is available for immediate posting, but I would like to wait for agreement on the size issue (at least if that can be reached quickly). Rainer > -----Original Message----- > From: syslog-sec-bounces@www.employees.org > [mailto:syslog-sec-bounces@www.employees.org] On Behalf Of Anton > Okmianski > Sent: Tuesday, October 12, 2004 11:25 PM > To: syslog-sec@employees.org > Subject: [Syslog-sec] UDP message size issue proposal > > Hi! > > Sorry for a long delay on this issue - I was on a 2 week vacation. I > have spoken with a number of TCP/UDP/IP experts regarding the sizing > issue. I am ready to propose the following changes: > > 1. Syslog-protocol will state that the max message will be defined by > the transport layer. > > 2. Syslog-transport-udp will support messages up to maximum UDP > datagram size of 64K. UDP is a very bad choice for large message > transmissions, so it does not make sense for us to stretch it by > adding our own fragmentation without other transmission control > features such as found in TCP. > > 3. Syslog-transport-udp will rely on IP fragmentation and we will get > rid of "proprietary" fragmentation which was designed to handle > messages over 64K and deal with various > non-compliant network hosts. > > 4. Syslog-transport-udp will recommend sending messages within the > boundaries of prevalent MTU size on a given network to be safe. It > will recommend Ethernet's 1500 bytes less headers and will draw reader > attention to the minimum MTU size hosts on the network are required to > support for > IPv6 (576 bytes) and IPv6 (1280 bytes). > > 5. Path MTU discovery may not work robustly and some TCP/IP stacks may > not support UDP packets of full 64K size and truncate them. We will > mention this and bite this bullet. > We should not restrict the protocol due to incompliant implementations > because it is a moving target and penalizes compliant implementations > with extra overhead. The above size recommendation would partially > deal with this, but leave the > final choice up to the administrator. > > 6. We will get rid of most syslog transport headers for UDP as they > will no longer be needed. The only thing that will be left is the > transport protocol version prefixed to every syslog message. Should > we even bother with that? > > This is a major change to the syslog-transport-udp. I'd like to get > positive feedback before I proceed with this update. > The best part is that if we all agree on the above, the next draft > will be 1/3 of the size -- easier read for you. :) > > Thanks, > Anton. > > > _______________________________________________ > Syslog-sec mailing list > Syslog-sec@www.employees.org > http://www.employees.org/mailman/listinfo/syslog-sec > _______________________________________________ Syslog-sec mailing list Syslog-sec@www.employees.org http://www.employees.org/mailman/listinfo/syslog-sec _______________________________________________ Syslog-sec mailing list Syslog-sec@www.employees.org http://www.employees.org/mailman/listinfo/syslog-sec From syslog-sec-bounces@willers.employees.org Wed Oct 13 11:09:51 2004 Received: from willers.employees.org (willers.employees.org [192.83.249.36]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA06254 for ; Wed, 13 Oct 2004 11:09:50 -0400 (EDT) Received: from willers.employees.org (localhost.employees.org [127.0.0.1]) by willers.employees.org (Postfix) with ESMTP id AB13F5C7B6; Wed, 13 Oct 2004 08:09:48 -0700 (PDT) X-Original-To: syslog-sec@employees.org Delivered-To: syslog-sec@employees.org Received: from sccrmhc11.comcast.net (sccrmhc11.comcast.net [204.127.202.55]) by willers.employees.org (Postfix) with ESMTP id 337145C735 for ; Wed, 13 Oct 2004 08:09:39 -0700 (PDT) Received: from djyxpy41 (h00104b8ce2a3.ne.client2.attbi.com[24.128.104.220]) by comcast.net (sccrmhc11) with SMTP id <2004101315093801100k54gbe>; Wed, 13 Oct 2004 15:09:38 +0000 From: "David B Harrington" To: "'Rainer Gerhards'" , "'Anton Okmianski'" , Date: Wed, 13 Oct 2004 11:09:37 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook, Build 11.0.5510 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1441 In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA030179@grfint2.intern.adiscon.com> Thread-Index: AcSwoe8xQ3+YxqsUQ8iCTkpgGQfmqgAYjRlAAAuIk6A= Message-Id: <20041013150939.337145C735@willers.employees.org> Subject: [Syslog-sec] Maximum message size X-BeenThere: syslog-sec@www.employees.org X-Mailman-Version: 2.1.4 Precedence: list Reply-To: ietfdbh@comcast.net List-Id: syslog-sec.www.employees.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: syslog-sec-bounces@willers.employees.org Errors-To: syslog-sec-bounces@willers.employees.org Content-Transfer-Encoding: 7bit Hi, I am concerned that we are in danger of promoting uses of syslog that will defeat its current usefulness as a human-readable log of system activity. Do network operators really believe the 16,777,216 octet size is a needed improvement to syslog, or are we designing this size in explicitly for the vendors of applications who want to use syslog as a programmatic interface? I believe Syslog should be designed to meet the needs of operators. Most of the discussion ssems to be from vendors of syslog receivers or applications. Other protocols such as FTP might be better used for such a specialty use case. Does allowing messages of 16,777,216 octets to be accumulated within the system log make it harder to use some widely-available minimal text editors, like Notepad, to view the system log? Will having huge application-specific messages in the system log make it harder for an operator to locate useful troubleshooting messages within the system log? Can the information in such a huge message fit within an 80x25 screen, when an operator is troubleshooting network problems and needs to use a bare-bones terminal attached to the serial port of a device to view the system log? Syslog was designing to be an operator's tool, and Syslog should be designed to meet the needs of operators. Will allowing messages of this size negatively impact its usefulness in the primary use case? What do network **operators** think about the need for messages of this size? Have we deliberately asked them their opinion on this by, for example, taking a survey or going to NANOG to ask them? I don't think we should support such large messages in the system logging protocol. David Harrington dbharrington@comcast.net -----Original Message----- From: syslog-sec-bounces@www.employees.org [mailto:syslog-sec-bounces@www.employees.org] On Behalf Of Rainer Gerhards Sent: Wednesday, October 13, 2004 5:17 AM To: Anton Okmianski; syslog-sec@employees.org Subject: RE: [Syslog-sec] UDP message size issue proposal Anton, I agree to your conclusion. Although I am the one who always voted for larger sizes, the discussion has shown that this yields to unnecessary complexity. I can satisfy my needs with a vendor-extension structured data element, which I think shows the flexibility of the new drafts. So I support your move. I would tend to even remove the transport version header. If there is need to, we could always include that in a later release (e.g. "v1", which would make it clearly distingshable from the current frame format). I see little need to do so, though. Regarding -protocol, I think we should still keep an upper limit in the spec. Even I don't see any reason why a syslog message should go above 16MB. So for -protocol, I propose the following new text: #### 5.1 Message Length The maximum length of any syslog message is 16,777,216 octets. Any receiver receiving a larger message MUST discard the message. A diagnostic message SHOULD be logged in this case. A receiver MUST be able to accept messages that are 480 octets (or less) in length. A receiver SHOULD be able to accept messages that are 65,535 octets (or less) in length. It is RECOMMENDED that receivers be able to accept messages up to the maximum message length of 16,777,216 octets. If a receiver receives a message within the maximum length, but with a length larger than it handles, the receiver MAY discard or truncate it. A transport mapping MAY define a maximum length below the one specified here. Each transport mapping MUST support a maximum size of 480 octets or more. #### If there is agreement on this, I can post a new version of -protocol. That version will also include some changes made thanks to Andrew Ross comments (sent via private mail). I have finished this version. It is available for immediate posting, but I would like to wait for agreement on the size issue (at least if that can be reached quickly). Rainer > -----Original Message----- > From: syslog-sec-bounces@www.employees.org > [mailto:syslog-sec-bounces@www.employees.org] On Behalf Of Anton > Okmianski > Sent: Tuesday, October 12, 2004 11:25 PM > To: syslog-sec@employees.org > Subject: [Syslog-sec] UDP message size issue proposal > > Hi! > > Sorry for a long delay on this issue - I was on a 2 week vacation. I > have spoken with a number of TCP/UDP/IP experts regarding the sizing > issue. I am ready to propose the following changes: > > 1. Syslog-protocol will state that the max message will be defined by > the transport layer. > > 2. Syslog-transport-udp will support messages up to maximum UDP > datagram size of 64K. UDP is a very bad choice for large message > transmissions, so it does not make sense for us to stretch it by > adding our own fragmentation without other transmission control > features such as found in TCP. > > 3. Syslog-transport-udp will rely on IP fragmentation and we will get > rid of "proprietary" fragmentation which was designed to handle > messages over 64K and deal with various > non-compliant network hosts. > > 4. Syslog-transport-udp will recommend sending messages within the > boundaries of prevalent MTU size on a given network to be safe. It > will recommend Ethernet's 1500 bytes less headers and will draw reader > attention to the minimum MTU size hosts on the network are required to > support for > IPv6 (576 bytes) and IPv6 (1280 bytes). > > 5. Path MTU discovery may not work robustly and some TCP/IP stacks may > not support UDP packets of full 64K size and truncate them. We will > mention this and bite this bullet. > We should not restrict the protocol due to incompliant implementations > because it is a moving target and penalizes compliant implementations > with extra overhead. The above size recommendation would partially > deal with this, but leave the > final choice up to the administrator. > > 6. We will get rid of most syslog transport headers for UDP as they > will no longer be needed. The only thing that will be left is the > transport protocol version prefixed to every syslog message. Should > we even bother with that? > > This is a major change to the syslog-transport-udp. I'd like to get > positive feedback before I proceed with this update. > The best part is that if we all agree on the above, the next draft > will be 1/3 of the size -- easier read for you. :) > > Thanks, > Anton. > > > _______________________________________________ > Syslog-sec mailing list > Syslog-sec@www.employees.org > http://www.employees.org/mailman/listinfo/syslog-sec > _______________________________________________ Syslog-sec mailing list Syslog-sec@www.employees.org http://www.employees.org/mailman/listinfo/syslog-sec _______________________________________________ Syslog-sec mailing list Syslog-sec@www.employees.org http://www.employees.org/mailman/listinfo/syslog-sec From syslog-sec-bounces@willers.employees.org Wed Oct 13 13:15:39 2004 Received: from willers.employees.org (willers.employees.org [192.83.249.36]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA16745 for ; Wed, 13 Oct 2004 13:15:38 -0400 (EDT) Received: from willers.employees.org (localhost.employees.org [127.0.0.1]) by willers.employees.org (Postfix) with ESMTP id 4B5EB5C7E3; Wed, 13 Oct 2004 10:15:38 -0700 (PDT) X-Original-To: syslog-sec@employees.org Delivered-To: syslog-sec@employees.org Received: from mlvv9n2x.shs.siemens.com (ns.shs.siemens.com [64.46.248.225]) by willers.employees.org (Postfix) with ESMTP id E8D6D5C72D for ; Wed, 13 Oct 2004 09:56:23 -0700 (PDT) Received: from mlvv9m1x.shs.siemens.com (unknown [165.226.204.11]) by mlvv9n2x.shs.siemens.com (Postfix) with ESMTP id 1015368077 for ; Wed, 13 Oct 2004 12:56:23 -0400 (EDT) Received: from MLVV9MBA.ww005.siemens.net (mlvv9mba.smshsc.net [165.226.204.44]) by mlvv9m1x.shs.siemens.com (8.12.11/8.12.10) with ESMTP id i9DGuMMm004303 for ; Wed, 13 Oct 2004 12:56:22 -0400 Received: from mlvexc01.smshsc.net ([165.226.249.73]) by 165.226.204.44 with InterScan Messaging Security Suite; Wed, 13 Oct 2004 12:54:52 -0400 Received: by MLVEXC01 with Internet Mail Service (5.5.2657.72) id <45GXDPQV>; Wed, 13 Oct 2004 12:56:37 -0400 Message-ID: From: Marshall Glen To: "'ietfdbh@comcast.net'" Subject: RE: [Syslog-sec] Maximum message size Date: Wed, 13 Oct 2004 12:56:19 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2657.72) Content-Type: text/plain X-Mailman-Approved-At: Wed, 13 Oct 2004 10:15:37 -0700 Cc: syslog-sec@employees.org X-BeenThere: syslog-sec@www.employees.org X-Mailman-Version: 2.1.4 Precedence: list List-Id: syslog-sec.www.employees.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: syslog-sec-bounces@willers.employees.org Errors-To: syslog-sec-bounces@willers.employees.org While I'd agree that a 16M octet size is extreme, we do need to accommodate larger syslog messages. See RFC3881 for an example. Communication of healthcare application security audit data to a central repository is the key use case for RFC3881. The end-users are healthcare security and privacy officers, and they will need aggregated data from multiple IT-enable healthcare systems to assess security conformance and track patterns of inappropriate data access. The design of an XML schema to meet the end-user information needs leads to a message size on the order of a few kilobytes. It needs to be sent reliably and securely to a central repository for after-the-fact analysis. Best regards, Glen Marshall -----Original Message----- From: David B Harrington [mailto:ietfdbh@comcast.net] Sent: Wednesday, October 13, 2004 11:10 AM To: 'Rainer Gerhards'; 'Anton Okmianski'; syslog-sec@employees.org Subject: [Syslog-sec] Maximum message size Hi, I am concerned that we are in danger of promoting uses of syslog that will defeat its current usefulness as a human-readable log of system activity. Do network operators really believe the 16,777,216 octet size is a needed improvement to syslog, or are we designing this size in explicitly for the vendors of applications who want to use syslog as a programmatic interface? I believe Syslog should be designed to meet the needs of operators. Most of the discussion ssems to be from vendors of syslog receivers or applications. Other protocols such as FTP might be better used for such a specialty use case. Does allowing messages of 16,777,216 octets to be accumulated within the system log make it harder to use some widely-available minimal text editors, like Notepad, to view the system log? Will having huge application-specific messages in the system log make it harder for an operator to locate useful troubleshooting messages within the system log? Can the information in such a huge message fit within an 80x25 screen, when an operator is troubleshooting network problems and needs to use a bare-bones terminal attached to the serial port of a device to view the system log? Syslog was designing to be an operator's tool, and Syslog should be designed to meet the needs of operators. Will allowing messages of this size negatively impact its usefulness in the primary use case? What do network **operators** think about the need for messages of this size? Have we deliberately asked them their opinion on this by, for example, taking a survey or going to NANOG to ask them? I don't think we should support such large messages in the system logging protocol. David Harrington dbharrington@comcast.net -----Original Message----- From: syslog-sec-bounces@www.employees.org [mailto:syslog-sec-bounces@www.employees.org] On Behalf Of Rainer Gerhards Sent: Wednesday, October 13, 2004 5:17 AM To: Anton Okmianski; syslog-sec@employees.org Subject: RE: [Syslog-sec] UDP message size issue proposal Anton, I agree to your conclusion. Although I am the one who always voted for larger sizes, the discussion has shown that this yields to unnecessary complexity. I can satisfy my needs with a vendor-extension structured data element, which I think shows the flexibility of the new drafts. So I support your move. I would tend to even remove the transport version header. If there is need to, we could always include that in a later release (e.g. "v1", which would make it clearly distingshable from the current frame format). I see little need to do so, though. Regarding -protocol, I think we should still keep an upper limit in the spec. Even I don't see any reason why a syslog message should go above 16MB. So for -protocol, I propose the following new text: #### 5.1 Message Length The maximum length of any syslog message is 16,777,216 octets. Any receiver receiving a larger message MUST discard the message. A diagnostic message SHOULD be logged in this case. A receiver MUST be able to accept messages that are 480 octets (or less) in length. A receiver SHOULD be able to accept messages that are 65,535 octets (or less) in length. It is RECOMMENDED that receivers be able to accept messages up to the maximum message length of 16,777,216 octets. If a receiver receives a message within the maximum length, but with a length larger than it handles, the receiver MAY discard or truncate it. A transport mapping MAY define a maximum length below the one specified here. Each transport mapping MUST support a maximum size of 480 octets or more. #### If there is agreement on this, I can post a new version of -protocol. That version will also include some changes made thanks to Andrew Ross comments (sent via private mail). I have finished this version. It is available for immediate posting, but I would like to wait for agreement on the size issue (at least if that can be reached quickly). Rainer > -----Original Message----- > From: syslog-sec-bounces@www.employees.org > [mailto:syslog-sec-bounces@www.employees.org] On Behalf Of Anton > Okmianski > Sent: Tuesday, October 12, 2004 11:25 PM > To: syslog-sec@employees.org > Subject: [Syslog-sec] UDP message size issue proposal > > Hi! > > Sorry for a long delay on this issue - I was on a 2 week vacation. I > have spoken with a number of TCP/UDP/IP experts regarding the sizing > issue. I am ready to propose the following changes: > > 1. Syslog-protocol will state that the max message will be defined by > the transport layer. > > 2. Syslog-transport-udp will support messages up to maximum UDP > datagram size of 64K. UDP is a very bad choice for large message > transmissions, so it does not make sense for us to stretch it by > adding our own fragmentation without other transmission control > features such as found in TCP. > > 3. Syslog-transport-udp will rely on IP fragmentation and we will get > rid of "proprietary" fragmentation which was designed to handle > messages over 64K and deal with various > non-compliant network hosts. > > 4. Syslog-transport-udp will recommend sending messages within the > boundaries of prevalent MTU size on a given network to be safe. It > will recommend Ethernet's 1500 bytes less headers and will draw reader > attention to the minimum MTU size hosts on the network are required to > support for > IPv6 (576 bytes) and IPv6 (1280 bytes). > > 5. Path MTU discovery may not work robustly and some TCP/IP stacks may > not support UDP packets of full 64K size and truncate them. We will > mention this and bite this bullet. > We should not restrict the protocol due to incompliant implementations > because it is a moving target and penalizes compliant implementations > with extra overhead. The above size recommendation would partially > deal with this, but leave the > final choice up to the administrator. > > 6. We will get rid of most syslog transport headers for UDP as they > will no longer be needed. The only thing that will be left is the > transport protocol version prefixed to every syslog message. Should > we even bother with that? > > This is a major change to the syslog-transport-udp. I'd like to get > positive feedback before I proceed with this update. > The best part is that if we all agree on the above, the next draft > will be 1/3 of the size -- easier read for you. :) > > Thanks, > Anton. > > > _______________________________________________ > Syslog-sec mailing list > Syslog-sec@www.employees.org > http://www.employees.org/mailman/listinfo/syslog-sec > _______________________________________________ Syslog-sec mailing list Syslog-sec@www.employees.org http://www.employees.org/mailman/listinfo/syslog-sec _______________________________________________ Syslog-sec mailing list Syslog-sec@www.employees.org http://www.employees.org/mailman/listinfo/syslog-sec ------------------------------------------------------------------------------- This message and any included attachments are from Siemens Medical Solutions USA, Inc. and are intended only for the addressee(s). The information contained herein may include trade secrets or privileged or otherwise confidential information. Unauthorized review, forwarding, printing, copying, distributing, or using such information is strictly prohibited and may be unlawful. If you received this message in error, or have reason to believe you are not authorized to receive it, please promptly delete this message and notify the sender by e-mail with a copy to Central.SecurityOffice@shs.siemens.com Thank you _______________________________________________ Syslog-sec mailing list Syslog-sec@www.employees.org http://www.employees.org/mailman/listinfo/syslog-sec From syslog-sec-bounces@willers.employees.org Wed Oct 13 21:10:59 2004 Received: from willers.employees.org (willers.employees.org [192.83.249.36]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA01151 for ; Wed, 13 Oct 2004 21:10:58 -0400 (EDT) Received: from willers.employees.org (localhost.employees.org [127.0.0.1]) by willers.employees.org (Postfix) with ESMTP id 4C7F65C735; Wed, 13 Oct 2004 18:10:55 -0700 (PDT) X-Original-To: syslog-sec@employees.org Delivered-To: syslog-sec@employees.org Received: from relay.pair.com (relay.pair.com [209.68.1.20]) by willers.employees.org (Postfix) with SMTP id 093A95C7C4 for ; Wed, 13 Oct 2004 17:35:46 -0700 (PDT) Received: (qmail 44215 invoked from network); 14 Oct 2004 00:35:42 -0000 Received: from 219-88-140-76.airnet.net.nz (HELO Kiwi2000) (219.88.140.76) by relay.pair.com with SMTP; 14 Oct 2004 00:35:42 -0000 X-pair-Authenticated: 219.88.140.76 From: "Andrew Ross" To: , "'Rainer Gerhards'" , "'Anton Okmianski'" , Subject: RE: [Syslog-sec] Maximum message size Date: Thu, 14 Oct 2004 13:35:40 +1300 Organization: Kiwi Enterprises Message-ID: <001501c4b185$bc84c100$1101a8c0@Kiwi2000> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.2616 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1441 In-Reply-To: <20041013150939.337145C735@willers.employees.org> Importance: Normal X-Mailman-Approved-At: Wed, 13 Oct 2004 18:10:54 -0700 X-BeenThere: syslog-sec@www.employees.org X-Mailman-Version: 2.1.4 Precedence: list Reply-To: andrew@kiwisyslog.com List-Id: syslog-sec.www.employees.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: syslog-sec-bounces@willers.employees.org Errors-To: syslog-sec-bounces@willers.employees.org Content-Transfer-Encoding: 7bit David, You make a good point on the end use of syslog messages. I'm not a big fan of large messages, I'd be happy limiting it to 64K even over TCP. It makes it readable by humans and parsable by standard file parsing programs. Some standard text editors will only allow a max line length of 4096 characters. If we are to assume that a "syslog" message is intended to be displayed on a console, logged to a file and maybe included in some sort of paging/e-mail alert system, then the message size really needs to be kept fairly short. Rainer's experience is that some devices/apps will send very large diagnostic messages of around 1MB in size. I can understand the need in this case to be able to handle large message sizes. However, I don't plan on accepting messages any larger than 64K in my implementation of -protocol. A client would have to make a REALLY good case for me to want to accept larger messages, especially 16MB in size. FTP, or even HTTP POST would be the best solution in this case. Syslog messages are meant for displaying, logging to disk and alerting. Not large binary style core dump info or database style records. I would like to see a note in the -protocol spec that encourages implementers to only send small messages that adhere to the original spirit of syslog. Cheers Andrew -----Original Message----- From: syslog-sec-bounces@www.employees.org [mailto:syslog-sec-bounces@www.employees.org] On Behalf Of David B Harrington Sent: Thursday, 14 October 2004 4:10 a.m. To: 'Rainer Gerhards'; 'Anton Okmianski'; syslog-sec@employees.org Subject: [Syslog-sec] Maximum message size Hi, I am concerned that we are in danger of promoting uses of syslog that will defeat its current usefulness as a human-readable log of system activity. Do network operators really believe the 16,777,216 octet size is a needed improvement to syslog, or are we designing this size in explicitly for the vendors of applications who want to use syslog as a programmatic interface? I believe Syslog should be designed to meet the needs of operators. Most of the discussion ssems to be from vendors of syslog receivers or applications. Other protocols such as FTP might be better used for such a specialty use case. Does allowing messages of 16,777,216 octets to be accumulated within the system log make it harder to use some widely-available minimal text editors, like Notepad, to view the system log? Will having huge application-specific messages in the system log make it harder for an operator to locate useful troubleshooting messages within the system log? Can the information in such a huge message fit within an 80x25 screen, when an operator is troubleshooting network problems and needs to use a bare-bones terminal attached to the serial port of a device to view the system log? Syslog was designing to be an operator's tool, and Syslog should be designed to meet the needs of operators. Will allowing messages of this size negatively impact its usefulness in the primary use case? What do network **operators** think about the need for messages of this size? Have we deliberately asked them their opinion on this by, for example, taking a survey or going to NANOG to ask them? I don't think we should support such large messages in the system logging protocol. David Harrington dbharrington@comcast.net -----Original Message----- From: syslog-sec-bounces@www.employees.org [mailto:syslog-sec-bounces@www.employees.org] On Behalf Of Rainer Gerhards Sent: Wednesday, October 13, 2004 5:17 AM To: Anton Okmianski; syslog-sec@employees.org Subject: RE: [Syslog-sec] UDP message size issue proposal Anton, I agree to your conclusion. Although I am the one who always voted for larger sizes, the discussion has shown that this yields to unnecessary complexity. I can satisfy my needs with a vendor-extension structured data element, which I think shows the flexibility of the new drafts. So I support your move. I would tend to even remove the transport version header. If there is need to, we could always include that in a later release (e.g. "v1", which would make it clearly distingshable from the current frame format). I see little need to do so, though. Regarding -protocol, I think we should still keep an upper limit in the spec. Even I don't see any reason why a syslog message should go above 16MB. So for -protocol, I propose the following new text: #### 5.1 Message Length The maximum length of any syslog message is 16,777,216 octets. Any receiver receiving a larger message MUST discard the message. A diagnostic message SHOULD be logged in this case. A receiver MUST be able to accept messages that are 480 octets (or less) in length. A receiver SHOULD be able to accept messages that are 65,535 octets (or less) in length. It is RECOMMENDED that receivers be able to accept messages up to the maximum message length of 16,777,216 octets. If a receiver receives a message within the maximum length, but with a length larger than it handles, the receiver MAY discard or truncate it. A transport mapping MAY define a maximum length below the one specified here. Each transport mapping MUST support a maximum size of 480 octets or more. #### If there is agreement on this, I can post a new version of -protocol. That version will also include some changes made thanks to Andrew Ross comments (sent via private mail). I have finished this version. It is available for immediate posting, but I would like to wait for agreement on the size issue (at least if that can be reached quickly). Rainer > -----Original Message----- > From: syslog-sec-bounces@www.employees.org > [mailto:syslog-sec-bounces@www.employees.org] On Behalf Of Anton > Okmianski > Sent: Tuesday, October 12, 2004 11:25 PM > To: syslog-sec@employees.org > Subject: [Syslog-sec] UDP message size issue proposal > > Hi! > > Sorry for a long delay on this issue - I was on a 2 week vacation. I > have spoken with a number of TCP/UDP/IP experts regarding the sizing > issue. I am ready to propose the following changes: > > 1. Syslog-protocol will state that the max message will be defined by > the transport layer. > > 2. Syslog-transport-udp will support messages up to maximum UDP > datagram size of 64K. UDP is a very bad choice for large message > transmissions, so it does not make sense for us to stretch it by > adding our own fragmentation without other transmission control > features such as found in TCP. > > 3. Syslog-transport-udp will rely on IP fragmentation and we will get > rid of "proprietary" fragmentation which was designed to handle > messages over 64K and deal with various > non-compliant network hosts. > > 4. Syslog-transport-udp will recommend sending messages within the > boundaries of prevalent MTU size on a given network to be safe. It > will recommend Ethernet's 1500 bytes less headers and will draw reader > attention to the minimum MTU size hosts on the network are required to > support for > IPv6 (576 bytes) and IPv6 (1280 bytes). > > 5. Path MTU discovery may not work robustly and some TCP/IP stacks may > not support UDP packets of full 64K size and truncate them. We will > mention this and bite this bullet. > We should not restrict the protocol due to incompliant implementations > because it is a moving target and penalizes compliant implementations > with extra overhead. The above size recommendation would partially > deal with this, but leave the > final choice up to the administrator. > > 6. We will get rid of most syslog transport headers for UDP as they > will no longer be needed. The only thing that will be left is the > transport protocol version prefixed to every syslog message. Should > we even bother with that? > > This is a major change to the syslog-transport-udp. I'd like to get > positive feedback before I proceed with this update. > The best part is that if we all agree on the above, the next draft > will be 1/3 of the size -- easier read for you. :) > > Thanks, > Anton. > > > _______________________________________________ > Syslog-sec mailing list > Syslog-sec@www.employees.org > http://www.employees.org/mailman/listinfo/syslog-sec > _______________________________________________ Syslog-sec mailing list Syslog-sec@www.employees.org http://www.employees.org/mailman/listinfo/syslog-sec _______________________________________________ Syslog-sec mailing list Syslog-sec@www.employees.org http://www.employees.org/mailman/listinfo/syslog-sec _______________________________________________ Syslog-sec mailing list Syslog-sec@www.employees.org http://www.employees.org/mailman/listinfo/syslog-sec From syslog-sec-bounces@willers.employees.org Thu Oct 14 07:30:52 2004 Received: from willers.employees.org (willers.employees.org [192.83.249.36]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id HAA03842 for ; Thu, 14 Oct 2004 07:30:51 -0400 (EDT) Received: from willers.employees.org (localhost.employees.org [127.0.0.1]) by willers.employees.org (Postfix) with ESMTP id 46A985C724; Thu, 14 Oct 2004 04:30:51 -0700 (PDT) X-Original-To: syslog-sec@employees.org Delivered-To: syslog-sec@employees.org Received: from mail.hq.adiscon.com (mail.hq.adiscon.com [217.6.190.188]) by willers.employees.org (Postfix) with ESMTP id 5D70F5C72F for ; Thu, 14 Oct 2004 04:30:44 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by mail.hq.adiscon.com (Postfix) with ESMTP id 26E739C757 for ; Thu, 14 Oct 2004 13:47:54 +0200 (CEST) Received: from mail.hq.adiscon.com ([127.0.0.1]) by localhost (grfdeb [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 17106-05 for ; Thu, 14 Oct 2004 13:47:30 +0200 (CEST) Received: from grfint2.intern.adiscon.com (grfint2 [172.19.0.6]) by mail.hq.adiscon.com (Postfix) with ESMTP id A4C609C755 for ; Thu, 14 Oct 2004 13:47:30 +0200 (CEST) Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-MimeOLE: Produced By Microsoft Exchange V6.5.7226.0 Date: Thu, 14 Oct 2004 13:34:24 +0200 Message-ID: <577465F99B41C842AAFBE9ED71E70ABA03019A@grfint2.intern.adiscon.com> Thread-Topic: Maximum message size Thread-Index: AcSwoe8xQ3+YxqsUQ8iCTkpgGQfmqgAYjRlAAAuIk6AAIvA8sA== From: "Rainer Gerhards" To: X-Virus-Scanned: by amavisd-new-20030616-p7 (Debian) at adiscon.com Subject: [Syslog-sec] RE: Maximum message size X-BeenThere: syslog-sec@www.employees.org X-Mailman-Version: 2.1.4 Precedence: list List-Id: syslog-sec.www.employees.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: syslog-sec-bounces@willers.employees.org Errors-To: syslog-sec-bounces@willers.employees.org Content-Transfer-Encoding: quoted-printable David and all, many thanks for all the good comments. I agree that 16 MB for a single message is far more than actually should be used. Let's look a bit at the history of this: Initially, we noticed that the 1024 octet limit for current syslog is too short. I think there is still general agreement on this. We had a good discussion about the size issue in September 2003. A link to an archive is here: http://www.syslog.cc/ietf/autoarc/msg00855.html The general issue is that 1K is too short for many applications. For example, Marshall Glen mentions the healthcare needs (RFC3881) for a larger message. As far as I know, the IHE initiative currently standardizes on a maximum XML stream size of 32K (which is then supposed to be transmitted via syslog). I think a message size limit of 64K would probably be suffcient for a long time to come (and leaves some headroom for message content increases). Besides that, I myself identified a need - in very rare occasions - to send information larger than 64K. In extreme cases, this information could be as large as one megabyte. Again, these were cases that might happen once in a year. All of this, of course, only if an operator *really* wants to submit that large information. So the initial approach and suggestion was to introduce a way to build "message groups". That would have been a way to split "oversize messages" (as they were called then) into multiple syslog messages. They key to that concept was that each single message could be kept within a reasonable limit (it would have worked even with 1K max message size). It would only have been a way to transmit very large messages in those very seldom cases that they would actually be used. Some time later, we came to the conclusion that it should be possible for a single syslog message to be of the largest imaginable size. It was argued that fragmentation is a transport issue and as such application-level "fragmentation" (the message groups) would not do good. We agreed to that. Even some time later, we needed to define a maximum message size, just to limit it somewhere (if I recall correctly, that was a transport requirement). We picked 16MB=20 just as a number that we thought would never have be reached and such be sufficient for all imaginable time to come. As far as I was concerend, that decision was driven by the fact that in the past there were often limits that nobody expected to reach - and yet they wer quickly reached. Som 16MB was picked more or less as "larger than anybody will need anytime in the forseeable future". We did not even have a big discussion on that limit. I think this are the key points on how we reached this limit. Now, thanks to your call and looking the whole situation, I really think we've gone overboard. That 16MB limit is something nobody intends to use. Even a lower - but high limit - is extremely unlikely. I'd expect to see a 1MB "message" in maybe 0.0001% of the cases - if at all (but I expect to see e.g. 70KB "message" in maybe 0.001% of the cases). Also, I have read your proposal to set a maxium size for the message, but allow an implementation to go beyond that. Maybe something like this: #### 5.1 Message Length A receiver MUST be able to accept messages up to and including 480 octets in length.=20 A receiver SHOULD be able to accept messages up to and including 65,535 octets in length.=20 If a receiver receives a message with a length larger than it supports, the receiver MAY=20 discard the message or truncate the payload. A receiver MAY accept messages larger than 65,535 octets. #### I've dropped the wording that transports MAY support only a smaller max size - I think UDP was the most critical in this regard and it obviously handles 64K. So I assume this clause is not really needed if we go for 64K as the max size. With that clause, we would limit the size for all interop cases, but leave a window open so that the message length could be extended if there is a real need AND the operator and software vendor wants this. I assume nobody will implement this burden if there is no market need to do so. But if it is, it clould at least be implemented in a standards-compliant way. If I look back at the original requirement - provide a way to transmit very rare oversize messages, I think we can safely implement this the bounds of the current proposals, even if the size limit does not go beyond 64K. If somebody needs it, this can be easily done via an experimental (or vendor-specific) SD-ID. Again, I do not think somebody will accept this burden without market need. In the light of this, I, too, would recommend that we limit the message size to 64K. I would prefer, however, that we keep the maximum open as in the suggested wording above. Rainer > -----Original Message----- > From: David B Harrington [mailto:ietfdbh@comcast.net]=20 > Sent: Wednesday, October 13, 2004 5:10 PM > To: Rainer Gerhards; 'Anton Okmianski'; syslog-sec@employees.org > Subject: Maximum message size=20 >=20 > Hi, >=20 > I am concerned that we are in danger of promoting uses of syslog that > will defeat its current usefulness as a human-readable log of system > activity. >=20 > Do network operators really believe the 16,777,216 octet size is a > needed improvement to syslog, or are we designing this size in > explicitly for the vendors of applications who want to use syslog as a > programmatic interface? I believe Syslog should be designed to meet > the needs of operators. Most of the discussion ssems to be from > vendors of syslog receivers or applications. Other protocols such as > FTP might be better used for such a specialty use case.=20 >=20 > Does allowing messages of 16,777,216 octets to be accumulated within > the system log make it harder to use some widely-available minimal > text editors, like Notepad, to view the system log? Will having huge > application-specific messages in the system log make it harder for an > operator to locate useful troubleshooting messages within the system > log? Can the information in such a huge message fit within an 80x25 > screen, when an operator is troubleshooting network problems and needs > to use a bare-bones terminal attached to the serial port of a device > to view the system log?=20 >=20 > Syslog was designing to be an operator's tool, and Syslog should be > designed to meet the needs of operators. Will allowing messages of > this size negatively impact its usefulness in the primary use case? =20 >=20 > What do network **operators** think about the need for messages of > this size? Have we deliberately asked them their opinion on this by, > for example, taking a survey or going to NANOG to ask them? >=20 > I don't think we should support such large messages in the system > logging protocol. >=20 >=20 > David Harrington > dbharrington@comcast.net >=20 >=20 >=20 > -----Original Message----- > From: syslog-sec-bounces@www.employees.org > [mailto:syslog-sec-bounces@www.employees.org] On Behalf Of Rainer > Gerhards > Sent: Wednesday, October 13, 2004 5:17 AM > To: Anton Okmianski; syslog-sec@employees.org > Subject: RE: [Syslog-sec] UDP message size issue proposal >=20 > Anton, >=20 > I agree to your conclusion. >=20 > Although I am the one who always voted for larger sizes, the > discussion has shown that this yields to unnecessary complexity. I can > satisfy my needs with a vendor-extension structured data element, > which I think shows the flexibility of the new drafts. >=20 > So I support your move. I would tend to even remove the transport > version header. If there is need to, we could always include that in a > later release (e.g. "v1", which would make it clearly distingshable > from the current frame format). I see little need to do so, though. >=20 > Regarding -protocol, I think we should still keep an upper limit in > the spec. Even I don't see any reason why a syslog message should go > above 16MB. So for -protocol, I propose the following new text: >=20 > #### > 5.1 Message Length >=20 > The maximum length of any syslog message is 16,777,216 octets. Any > receiver receiving a larger message MUST discard the message. A > diagnostic message SHOULD be logged in this case. >=20 > A receiver MUST be able to accept messages that are 480 octets (or > less) in length. A receiver SHOULD be able to accept messages that > are 65,535 octets (or less) in length. It is RECOMMENDED that > receivers be able to accept messages up to the maximum message > length > of 16,777,216 octets. >=20 > If a receiver receives a message within the maximum length, but > with > a length larger than it handles, the receiver MAY discard or > truncate > it. >=20 > A transport mapping MAY define a maximum length below the one > specified here. Each transport mapping MUST support a maximum size > of 480 octets or more. > #### >=20 > If there is agreement on this, I can post a new version of -protocol. > That version will also include some changes made thanks to Andrew Ross > comments (sent via private mail). I have finished this version. It is > available for immediate posting, but I would like to wait for > agreement on the size issue (at least if that can be reached quickly). >=20 > Rainer >=20 > > -----Original Message----- > > From: syslog-sec-bounces@www.employees.org > > [mailto:syslog-sec-bounces@www.employees.org] On Behalf Of Anton=20 > > Okmianski > > Sent: Tuesday, October 12, 2004 11:25 PM > > To: syslog-sec@employees.org > > Subject: [Syslog-sec] UDP message size issue proposal > >=20 > > Hi! > >=20 > > Sorry for a long delay on this issue - I was on a 2 week vacation. > I=20 > > have spoken with a number of TCP/UDP/IP experts regarding the sizing >=20 > > issue. I am ready to propose the following changes: > >=20 > > 1. Syslog-protocol will state that the max message will be defined > by=20 > > the transport layer. > >=20 > > 2. Syslog-transport-udp will support messages up to maximum UDP=20 > > datagram size of 64K. UDP is a very bad choice for large message=20 > > transmissions, so it does not make sense for us to stretch it by=20 > > adding our own fragmentation without other transmission control=20 > > features such as found in TCP. > >=20 > > 3. Syslog-transport-udp will rely on IP fragmentation and we will > get=20 > > rid of "proprietary" fragmentation which was designed to handle=20 > > messages over 64K and deal with various > > non-compliant network hosts. =20 > >=20 > > 4. Syslog-transport-udp will recommend sending messages within the=20 > > boundaries of prevalent MTU size on a given network to be safe. It=20 > > will recommend Ethernet's 1500 bytes less headers and will draw > reader=20 > > attention to the minimum MTU size hosts on the network are required > to=20 > > support for > > IPv6 (576 bytes) and IPv6 (1280 bytes). =20 > >=20 > > 5. Path MTU discovery may not work robustly and some TCP/IP stacks > may=20 > > not support UDP packets of full 64K size and truncate them. We will >=20 > > mention this and bite this bullet. > > We should not restrict the protocol due to incompliant > implementations=20 > > because it is a moving target and penalizes compliant > implementations=20 > > with extra overhead. The above size recommendation would partially=20 > > deal with this, but leave the > > final choice up to the administrator. =20 > >=20 > > 6. We will get rid of most syslog transport headers for UDP as they=20 > > will no longer be needed. The only thing that will be left is the=20 > > transport protocol version prefixed to every syslog message. Should >=20 > > we even bother with that? > >=20 > > This is a major change to the syslog-transport-udp. I'd like to get >=20 > > positive feedback before I proceed with this update. > > The best part is that if we all agree on the above, the next draft=20 > > will be 1/3 of the size -- easier read for you. :) > >=20 > > Thanks, > > Anton.=20 > > =20 > >=20 > > _______________________________________________ > > Syslog-sec mailing list > > Syslog-sec@www.employees.org > > http://www.employees.org/mailman/listinfo/syslog-sec > >=20 > _______________________________________________ > Syslog-sec mailing list > Syslog-sec@www.employees.org > http://www.employees.org/mailman/listinfo/syslog-sec >=20 >=20 >=20 _______________________________________________ Syslog-sec mailing list Syslog-sec@www.employees.org http://www.employees.org/mailman/listinfo/syslog-sec From syslog-sec-bounces@willers.employees.org Thu Oct 14 08:53:16 2004 Received: from willers.employees.org (willers.employees.org [192.83.249.36]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id IAA10522 for ; Thu, 14 Oct 2004 08:53:16 -0400 (EDT) Received: from willers.employees.org (localhost.employees.org [127.0.0.1]) by willers.employees.org (Postfix) with ESMTP id 4E2045C7D1; Thu, 14 Oct 2004 05:53:16 -0700 (PDT) X-Original-To: syslog-sec@employees.org Delivered-To: syslog-sec@employees.org Received: from sccrmhc13.comcast.net (sccrmhc13.comcast.net [204.127.202.64]) by willers.employees.org (Postfix) with ESMTP id B879E5C78F for ; Thu, 14 Oct 2004 05:53:04 -0700 (PDT) Received: from djyxpy41 (h00104b8ce2a3.ne.client2.attbi.com[24.128.104.220]) by comcast.net (sccrmhc13) with SMTP id <20041014125303016000oih2e>; Thu, 14 Oct 2004 12:53:03 +0000 From: "David B Harrington" To: "'Rainer Gerhards'" , Subject: RE: [Syslog-sec] RE: Maximum message size Date: Thu, 14 Oct 2004 08:53:01 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook, Build 11.0.5510 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1441 In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA03019A@grfint2.intern.adiscon.com> Thread-Index: AcSwoe8xQ3+YxqsUQ8iCTkpgGQfmqgAYjRlAAAuIk6AAIvA8sAAJc8ug Message-Id: <20041014125304.B879E5C78F@willers.employees.org> X-BeenThere: syslog-sec@www.employees.org X-Mailman-Version: 2.1.4 Precedence: list Reply-To: ietfdbh@comcast.net List-Id: syslog-sec.www.employees.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: syslog-sec-bounces@willers.employees.org Errors-To: syslog-sec-bounces@willers.employees.org Content-Transfer-Encoding: 7bit Hi Rainer, Thanks for making the changes. I'd like to make one point a little more strenuously: "For interoperability reasons, all receiver implementations SHOULD be able to accept messages up to and including 65,535 octets in length." Also, I think the last two sentences could be combined into one - "If a receiver receives a message with a length larger than 65,535 octets, or larger than it supports, the receiver MAY discard the message or truncate the payload." dbh -----Original Message----- From: syslog-sec-bounces@www.employees.org [mailto:syslog-sec-bounces@www.employees.org] On Behalf Of Rainer Gerhards Sent: Thursday, October 14, 2004 7:34 AM To: syslog-sec@employees.org Subject: [Syslog-sec] RE: Maximum message size David and all, many thanks for all the good comments. I agree that 16 MB for a single message is far more than actually should be used. Let's look a bit at the history of this: Initially, we noticed that the 1024 octet limit for current syslog is too short. I think there is still general agreement on this. We had a good discussion about the size issue in September 2003. A link to an archive is here: http://www.syslog.cc/ietf/autoarc/msg00855.html The general issue is that 1K is too short for many applications. For example, Marshall Glen mentions the healthcare needs (RFC3881) for a larger message. As far as I know, the IHE initiative currently standardizes on a maximum XML stream size of 32K (which is then supposed to be transmitted via syslog). I think a message size limit of 64K would probably be suffcient for a long time to come (and leaves some headroom for message content increases). Besides that, I myself identified a need - in very rare occasions - to send information larger than 64K. In extreme cases, this information could be as large as one megabyte. Again, these were cases that might happen once in a year. All of this, of course, only if an operator *really* wants to submit that large information. So the initial approach and suggestion was to introduce a way to build "message groups". That would have been a way to split "oversize messages" (as they were called then) into multiple syslog messages. They key to that concept was that each single message could be kept within a reasonable limit (it would have worked even with 1K max message size). It would only have been a way to transmit very large messages in those very seldom cases that they would actually be used. Some time later, we came to the conclusion that it should be possible for a single syslog message to be of the largest imaginable size. It was argued that fragmentation is a transport issue and as such application-level "fragmentation" (the message groups) would not do good. We agreed to that. Even some time later, we needed to define a maximum message size, just to limit it somewhere (if I recall correctly, that was a transport requirement). We picked 16MB just as a number that we thought would never have be reached and such be sufficient for all imaginable time to come. As far as I was concerend, that decision was driven by the fact that in the past there were often limits that nobody expected to reach - and yet they wer quickly reached. Som 16MB was picked more or less as "larger than anybody will need anytime in the forseeable future". We did not even have a big discussion on that limit. I think this are the key points on how we reached this limit. Now, thanks to your call and looking the whole situation, I really think we've gone overboard. That 16MB limit is something nobody intends to use. Even a lower - but high limit - is extremely unlikely. I'd expect to see a 1MB "message" in maybe 0.0001% of the cases - if at all (but I expect to see e.g. 70KB "message" in maybe 0.001% of the cases). Also, I have read your proposal to set a maxium size for the message, but allow an implementation to go beyond that. Maybe something like this: #### 5.1 Message Length A receiver MUST be able to accept messages up to and including 480 octets in length. A receiver SHOULD be able to accept messages up to and including 65,535 octets in length. If a receiver receives a message with a length larger than it supports, the receiver MAY discard the message or truncate the payload. A receiver MAY accept messages larger than 65,535 octets. #### I've dropped the wording that transports MAY support only a smaller max size - I think UDP was the most critical in this regard and it obviously handles 64K. So I assume this clause is not really needed if we go for 64K as the max size. With that clause, we would limit the size for all interop cases, but leave a window open so that the message length could be extended if there is a real need AND the operator and software vendor wants this. I assume nobody will implement this burden if there is no market need to do so. But if it is, it clould at least be implemented in a standards-compliant way. If I look back at the original requirement - provide a way to transmit very rare oversize messages, I think we can safely implement this the bounds of the current proposals, even if the size limit does not go beyond 64K. If somebody needs it, this can be easily done via an experimental (or vendor-specific) SD-ID. Again, I do not think somebody will accept this burden without market need. In the light of this, I, too, would recommend that we limit the message size to 64K. I would prefer, however, that we keep the maximum open as in the suggested wording above. Rainer > -----Original Message----- > From: David B Harrington [mailto:ietfdbh@comcast.net] > Sent: Wednesday, October 13, 2004 5:10 PM > To: Rainer Gerhards; 'Anton Okmianski'; syslog-sec@employees.org > Subject: Maximum message size > > Hi, > > I am concerned that we are in danger of promoting uses of syslog that > will defeat its current usefulness as a human-readable log of system > activity. > > Do network operators really believe the 16,777,216 octet size is a > needed improvement to syslog, or are we designing this size in > explicitly for the vendors of applications who want to use syslog as a > programmatic interface? I believe Syslog should be designed to meet > the needs of operators. Most of the discussion ssems to be from > vendors of syslog receivers or applications. Other protocols such as > FTP might be better used for such a specialty use case. > > Does allowing messages of 16,777,216 octets to be accumulated within > the system log make it harder to use some widely-available minimal > text editors, like Notepad, to view the system log? Will having huge > application-specific messages in the system log make it harder for an > operator to locate useful troubleshooting messages within the system > log? Can the information in such a huge message fit within an 80x25 > screen, when an operator is troubleshooting network problems and needs > to use a bare-bones terminal attached to the serial port of a device > to view the system log? > > Syslog was designing to be an operator's tool, and Syslog should be > designed to meet the needs of operators. Will allowing messages of > this size negatively impact its usefulness in the primary use case? > > What do network **operators** think about the need for messages of > this size? Have we deliberately asked them their opinion on this by, > for example, taking a survey or going to NANOG to ask them? > > I don't think we should support such large messages in the system > logging protocol. > > > David Harrington > dbharrington@comcast.net > > > > -----Original Message----- > From: syslog-sec-bounces@www.employees.org > [mailto:syslog-sec-bounces@www.employees.org] On Behalf Of Rainer > Gerhards > Sent: Wednesday, October 13, 2004 5:17 AM > To: Anton Okmianski; syslog-sec@employees.org > Subject: RE: [Syslog-sec] UDP message size issue proposal > > Anton, > > I agree to your conclusion. > > Although I am the one who always voted for larger sizes, the > discussion has shown that this yields to unnecessary complexity. I can > satisfy my needs with a vendor-extension structured data element, > which I think shows the flexibility of the new drafts. > > So I support your move. I would tend to even remove the transport > version header. If there is need to, we could always include that in a > later release (e.g. "v1", which would make it clearly distingshable > from the current frame format). I see little need to do so, though. > > Regarding -protocol, I think we should still keep an upper limit in > the spec. Even I don't see any reason why a syslog message should go > above 16MB. So for -protocol, I propose the following new text: > > #### > 5.1 Message Length > > The maximum length of any syslog message is 16,777,216 octets. Any > receiver receiving a larger message MUST discard the message. A > diagnostic message SHOULD be logged in this case. > > A receiver MUST be able to accept messages that are 480 octets (or > less) in length. A receiver SHOULD be able to accept messages that > are 65,535 octets (or less) in length. It is RECOMMENDED that > receivers be able to accept messages up to the maximum message > length > of 16,777,216 octets. > > If a receiver receives a message within the maximum length, but > with > a length larger than it handles, the receiver MAY discard or > truncate > it. > > A transport mapping MAY define a maximum length below the one > specified here. Each transport mapping MUST support a maximum size > of 480 octets or more. > #### > > If there is agreement on this, I can post a new version of -protocol. > That version will also include some changes made thanks to Andrew Ross > comments (sent via private mail). I have finished this version. It is > available for immediate posting, but I would like to wait for > agreement on the size issue (at least if that can be reached quickly). > > Rainer > > > -----Original Message----- > > From: syslog-sec-bounces@www.employees.org > > [mailto:syslog-sec-bounces@www.employees.org] On Behalf Of Anton > > Okmianski > > Sent: Tuesday, October 12, 2004 11:25 PM > > To: syslog-sec@employees.org > > Subject: [Syslog-sec] UDP message size issue proposal > > > > Hi! > > > > Sorry for a long delay on this issue - I was on a 2 week vacation. > I > > have spoken with a number of TCP/UDP/IP experts regarding the sizing > > > issue. I am ready to propose the following changes: > > > > 1. Syslog-protocol will state that the max message will be defined > by > > the transport layer. > > > > 2. Syslog-transport-udp will support messages up to maximum UDP > > datagram size of 64K. UDP is a very bad choice for large message > > transmissions, so it does not make sense for us to stretch it by > > adding our own fragmentation without other transmission control > > features such as found in TCP. > > > > 3. Syslog-transport-udp will rely on IP fragmentation and we will > get > > rid of "proprietary" fragmentation which was designed to handle > > messages over 64K and deal with various > > non-compliant network hosts. > > > > 4. Syslog-transport-udp will recommend sending messages within the > > boundaries of prevalent MTU size on a given network to be safe. It > > will recommend Ethernet's 1500 bytes less headers and will draw > reader > > attention to the minimum MTU size hosts on the network are required > to > > support for > > IPv6 (576 bytes) and IPv6 (1280 bytes). > > > > 5. Path MTU discovery may not work robustly and some TCP/IP stacks > may > > not support UDP packets of full 64K size and truncate them. We will > > > mention this and bite this bullet. > > We should not restrict the protocol due to incompliant > implementations > > because it is a moving target and penalizes compliant > implementations > > with extra overhead. The above size recommendation would partially > > deal with this, but leave the > > final choice up to the administrator. > > > > 6. We will get rid of most syslog transport headers for UDP as they > > will no longer be needed. The only thing that will be left is the > > transport protocol version prefixed to every syslog message. Should > > > we even bother with that? > > > > This is a major change to the syslog-transport-udp. I'd like to get > > > positive feedback before I proceed with this update. > > The best part is that if we all agree on the above, the next draft > > will be 1/3 of the size -- easier read for you. :) > > > > Thanks, > > Anton. > > > > > > _______________________________________________ > > Syslog-sec mailing list > > Syslog-sec@www.employees.org > > http://www.employees.org/mailman/listinfo/syslog-sec > > > _______________________________________________ > Syslog-sec mailing list > Syslog-sec@www.employees.org > http://www.employees.org/mailman/listinfo/syslog-sec > > > _______________________________________________ Syslog-sec mailing list Syslog-sec@www.employees.org http://www.employees.org/mailman/listinfo/syslog-sec _______________________________________________ Syslog-sec mailing list Syslog-sec@www.employees.org http://www.employees.org/mailman/listinfo/syslog-sec From syslog-sec-bounces@willers.employees.org Thu Oct 14 09:21:36 2004 Received: from willers.employees.org (willers.employees.org [192.83.249.36]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA13081 for ; Thu, 14 Oct 2004 09:21:35 -0400 (EDT) Received: from willers.employees.org (localhost.employees.org [127.0.0.1]) by willers.employees.org (Postfix) with ESMTP id 04B785C7BC; Thu, 14 Oct 2004 06:21:36 -0700 (PDT) X-Original-To: syslog-sec@employees.org Delivered-To: syslog-sec@employees.org Received: from sccrmhc12.comcast.net (sccrmhc12.comcast.net [204.127.202.56]) by willers.employees.org (Postfix) with ESMTP id 807805C78D for ; Thu, 14 Oct 2004 06:21:30 -0700 (PDT) Received: from djyxpy41 (h00104b8ce2a3.ne.client2.attbi.com[24.128.104.220]) by comcast.net (sccrmhc12) with SMTP id <2004101413212501200g06hpe>; Thu, 14 Oct 2004 13:21:25 +0000 From: "David B Harrington" To: "'Rainer Gerhards'" , Subject: RE: [Syslog-sec] RE: Maximum message size Date: Thu, 14 Oct 2004 09:21:24 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook, Build 11.0.5510 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1441 In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA03019A@grfint2.intern.adiscon.com> Thread-Index: AcSwoe8xQ3+YxqsUQ8iCTkpgGQfmqgAYjRlAAAuIk6AAIvA8sAALriig Message-Id: <20041014132130.807805C78D@willers.employees.org> X-BeenThere: syslog-sec@www.employees.org X-Mailman-Version: 2.1.4 Precedence: list Reply-To: ietfdbh@comcast.net List-Id: syslog-sec.www.employees.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: syslog-sec-bounces@willers.employees.org Errors-To: syslog-sec-bounces@willers.employees.org Content-Transfer-Encoding: 7bit Hi, I have concerns about the requirements for maximum message size, and would like to propose an alternate approach. Most syslog messages currently are less than 1K in size. There is consensus that 1K is typically acceptable, but it is a bit restrictive in some circunstances and the size should be increased. I agree with that consensus. I disagree that the size needs to be increased 64 times to accommodate normal usage, and I recognize that some valid, but abnormal, usages do require larger sizes. I think it is an unnecessary burden on receiver implementors to be required to receive 64K messages when 98% of all syslog messages are likely to be <1K, and 99.9% will be under <2K, just to support those unusual cases where messages may be as large as 32K or 70K. I dislike this approach because if all receivers SHOULD support 64k messages, that will tend to encourage implementors of message generators to make their messages large and wordy rather than succinct, which could have a negative impact on the primary use case for syslog. If we set the required message support to be smaller, most implementors will keep their messages smaller. I suggest that the required maximum size be more on the order of 2k, 4k, or 8k rather than 64k. This reduced receiver requirement should be accommpanied by advice that some operational environments, such as that specified in RFC3881, may require support for larger messages, and reciever implementors MAY support larger messages or MAY make the maximum message size a value that is configurable by the administrator of the system, based on their operational needs. This would keep most syslog messages small, yet accommodate special operational needs such as health care security auditing. David Harrington dbharrington@comcast.net -----Original Message----- From: syslog-sec-bounces@www.employees.org [mailto:syslog-sec-bounces@www.employees.org] On Behalf Of Rainer Gerhards Sent: Thursday, October 14, 2004 7:34 AM To: syslog-sec@employees.org Subject: [Syslog-sec] RE: Maximum message size David and all, many thanks for all the good comments. I agree that 16 MB for a single message is far more than actually should be used. Let's look a bit at the history of this: Initially, we noticed that the 1024 octet limit for current syslog is too short. I think there is still general agreement on this. We had a good discussion about the size issue in September 2003. A link to an archive is here: http://www.syslog.cc/ietf/autoarc/msg00855.html The general issue is that 1K is too short for many applications. For example, Marshall Glen mentions the healthcare needs (RFC3881) for a larger message. As far as I know, the IHE initiative currently standardizes on a maximum XML stream size of 32K (which is then supposed to be transmitted via syslog). I think a message size limit of 64K would probably be suffcient for a long time to come (and leaves some headroom for message content increases). Besides that, I myself identified a need - in very rare occasions - to send information larger than 64K. In extreme cases, this information could be as large as one megabyte. Again, these were cases that might happen once in a year. All of this, of course, only if an operator *really* wants to submit that large information. So the initial approach and suggestion was to introduce a way to build "message groups". That would have been a way to split "oversize messages" (as they were called then) into multiple syslog messages. They key to that concept was that each single message could be kept within a reasonable limit (it would have worked even with 1K max message size). It would only have been a way to transmit very large messages in those very seldom cases that they would actually be used. Some time later, we came to the conclusion that it should be possible for a single syslog message to be of the largest imaginable size. It was argued that fragmentation is a transport issue and as such application-level "fragmentation" (the message groups) would not do good. We agreed to that. Even some time later, we needed to define a maximum message size, just to limit it somewhere (if I recall correctly, that was a transport requirement). We picked 16MB just as a number that we thought would never have be reached and such be sufficient for all imaginable time to come. As far as I was concerend, that decision was driven by the fact that in the past there were often limits that nobody expected to reach - and yet they wer quickly reached. Som 16MB was picked more or less as "larger than anybody will need anytime in the forseeable future". We did not even have a big discussion on that limit. I think this are the key points on how we reached this limit. Now, thanks to your call and looking the whole situation, I really think we've gone overboard. That 16MB limit is something nobody intends to use. Even a lower - but high limit - is extremely unlikely. I'd expect to see a 1MB "message" in maybe 0.0001% of the cases - if at all (but I expect to see e.g. 70KB "message" in maybe 0.001% of the cases). Also, I have read your proposal to set a maxium size for the message, but allow an implementation to go beyond that. Maybe something like this: #### 5.1 Message Length A receiver MUST be able to accept messages up to and including 480 octets in length. A receiver SHOULD be able to accept messages up to and including 65,535 octets in length. If a receiver receives a message with a length larger than it supports, the receiver MAY discard the message or truncate the payload. A receiver MAY accept messages larger than 65,535 octets. #### I've dropped the wording that transports MAY support only a smaller max size - I think UDP was the most critical in this regard and it obviously handles 64K. So I assume this clause is not really needed if we go for 64K as the max size. With that clause, we would limit the size for all interop cases, but leave a window open so that the message length could be extended if there is a real need AND the operator and software vendor wants this. I assume nobody will implement this burden if there is no market need to do so. But if it is, it clould at least be implemented in a standards-compliant way. If I look back at the original requirement - provide a way to transmit very rare oversize messages, I think we can safely implement this the bounds of the current proposals, even if the size limit does not go beyond 64K. If somebody needs it, this can be easily done via an experimental (or vendor-specific) SD-ID. Again, I do not think somebody will accept this burden without market need. In the light of this, I, too, would recommend that we limit the message size to 64K. I would prefer, however, that we keep the maximum open as in the suggested wording above. Rainer > -----Original Message----- > From: David B Harrington [mailto:ietfdbh@comcast.net] > Sent: Wednesday, October 13, 2004 5:10 PM > To: Rainer Gerhards; 'Anton Okmianski'; syslog-sec@employees.org > Subject: Maximum message size > > Hi, > > I am concerned that we are in danger of promoting uses of syslog that > will defeat its current usefulness as a human-readable log of system > activity. > > Do network operators really believe the 16,777,216 octet size is a > needed improvement to syslog, or are we designing this size in > explicitly for the vendors of applications who want to use syslog as a > programmatic interface? I believe Syslog should be designed to meet > the needs of operators. Most of the discussion ssems to be from > vendors of syslog receivers or applications. Other protocols such as > FTP might be better used for such a specialty use case. > > Does allowing messages of 16,777,216 octets to be accumulated within > the system log make it harder to use some widely-available minimal > text editors, like Notepad, to view the system log? Will having huge > application-specific messages in the system log make it harder for an > operator to locate useful troubleshooting messages within the system > log? Can the information in such a huge message fit within an 80x25 > screen, when an operator is troubleshooting network problems and needs > to use a bare-bones terminal attached to the serial port of a device > to view the system log? > > Syslog was designing to be an operator's tool, and Syslog should be > designed to meet the needs of operators. Will allowing messages of > this size negatively impact its usefulness in the primary use case? > > What do network **operators** think about the need for messages of > this size? Have we deliberately asked them their opinion on this by, > for example, taking a survey or going to NANOG to ask them? > > I don't think we should support such large messages in the system > logging protocol. > > > David Harrington > dbharrington@comcast.net > > > > -----Original Message----- > From: syslog-sec-bounces@www.employees.org > [mailto:syslog-sec-bounces@www.employees.org] On Behalf Of Rainer > Gerhards > Sent: Wednesday, October 13, 2004 5:17 AM > To: Anton Okmianski; syslog-sec@employees.org > Subject: RE: [Syslog-sec] UDP message size issue proposal > > Anton, > > I agree to your conclusion. > > Although I am the one who always voted for larger sizes, the > discussion has shown that this yields to unnecessary complexity. I can > satisfy my needs with a vendor-extension structured data element, > which I think shows the flexibility of the new drafts. > > So I support your move. I would tend to even remove the transport > version header. If there is need to, we could always include that in a > later release (e.g. "v1", which would make it clearly distingshable > from the current frame format). I see little need to do so, though. > > Regarding -protocol, I think we should still keep an upper limit in > the spec. Even I don't see any reason why a syslog message should go > above 16MB. So for -protocol, I propose the following new text: > > #### > 5.1 Message Length > > The maximum length of any syslog message is 16,777,216 octets. Any > receiver receiving a larger message MUST discard the message. A > diagnostic message SHOULD be logged in this case. > > A receiver MUST be able to accept messages that are 480 octets (or > less) in length. A receiver SHOULD be able to accept messages that > are 65,535 octets (or less) in length. It is RECOMMENDED that > receivers be able to accept messages up to the maximum message > length > of 16,777,216 octets. > > If a receiver receives a message within the maximum length, but > with > a length larger than it handles, the receiver MAY discard or > truncate > it. > > A transport mapping MAY define a maximum length below the one > specified here. Each transport mapping MUST support a maximum size > of 480 octets or more. > #### > > If there is agreement on this, I can post a new version of -protocol. > That version will also include some changes made thanks to Andrew Ross > comments (sent via private mail). I have finished this version. It is > available for immediate posting, but I would like to wait for > agreement on the size issue (at least if that can be reached quickly). > > Rainer > > > -----Original Message----- > > From: syslog-sec-bounces@www.employees.org > > [mailto:syslog-sec-bounces@www.employees.org] On Behalf Of Anton > > Okmianski > > Sent: Tuesday, October 12, 2004 11:25 PM > > To: syslog-sec@employees.org > > Subject: [Syslog-sec] UDP message size issue proposal > > > > Hi! > > > > Sorry for a long delay on this issue - I was on a 2 week vacation. > I > > have spoken with a number of TCP/UDP/IP experts regarding the sizing > > > issue. I am ready to propose the following changes: > > > > 1. Syslog-protocol will state that the max message will be defined > by > > the transport layer. > > > > 2. Syslog-transport-udp will support messages up to maximum UDP > > datagram size of 64K. UDP is a very bad choice for large message > > transmissions, so it does not make sense for us to stretch it by > > adding our own fragmentation without other transmission control > > features such as found in TCP. > > > > 3. Syslog-transport-udp will rely on IP fragmentation and we will > get > > rid of "proprietary" fragmentation which was designed to handle > > messages over 64K and deal with various > > non-compliant network hosts. > > > > 4. Syslog-transport-udp will recommend sending messages within the > > boundaries of prevalent MTU size on a given network to be safe. It > > will recommend Ethernet's 1500 bytes less headers and will draw > reader > > attention to the minimum MTU size hosts on the network are required > to > > support for > > IPv6 (576 bytes) and IPv6 (1280 bytes). > > > > 5. Path MTU discovery may not work robustly and some TCP/IP stacks > may > > not support UDP packets of full 64K size and truncate them. We will > > > mention this and bite this bullet. > > We should not restrict the protocol due to incompliant > implementations > > because it is a moving target and penalizes compliant > implementations > > with extra overhead. The above size recommendation would partially > > deal with this, but leave the > > final choice up to the administrator. > > > > 6. We will get rid of most syslog transport headers for UDP as they > > will no longer be needed. The only thing that will be left is the > > transport protocol version prefixed to every syslog message. Should > > > we even bother with that? > > > > This is a major change to the syslog-transport-udp. I'd like to get > > > positive feedback before I proceed with this update. > > The best part is that if we all agree on the above, the next draft > > will be 1/3 of the size -- easier read for you. :) > > > > Thanks, > > Anton. > > > > > > _______________________________________________ > > Syslog-sec mailing list > > Syslog-sec@www.employees.org > > http://www.employees.org/mailman/listinfo/syslog-sec > > > _______________________________________________ > Syslog-sec mailing list > Syslog-sec@www.employees.org > http://www.employees.org/mailman/listinfo/syslog-sec > > > _______________________________________________ Syslog-sec mailing list Syslog-sec@www.employees.org http://www.employees.org/mailman/listinfo/syslog-sec _______________________________________________ Syslog-sec mailing list Syslog-sec@www.employees.org http://www.employees.org/mailman/listinfo/syslog-sec From syslog-sec-bounces@willers.employees.org Thu Oct 14 10:36:13 2004 Received: from willers.employees.org (willers.employees.org [192.83.249.36]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA24705 for ; Thu, 14 Oct 2004 10:36:12 -0400 (EDT) Received: from willers.employees.org (localhost.employees.org [127.0.0.1]) by willers.employees.org (Postfix) with ESMTP id 06E875C7CE; Thu, 14 Oct 2004 07:36:11 -0700 (PDT) X-Original-To: syslog-sec@employees.org Delivered-To: syslog-sec@employees.org Received: from sj-iport-4.cisco.com (sj-iport-4.cisco.com [171.68.10.86]) by willers.employees.org (Postfix) with ESMTP id D411D5C720 for ; Thu, 14 Oct 2004 07:36:03 -0700 (PDT) Received: from sj-core-1.cisco.com (171.71.177.237) by sj-iport-4.cisco.com with ESMTP; 14 Oct 2004 07:36:33 -0700 X-BrightmailFiltered: true Received: from flask.cisco.com (IDENT:mirapoint@flask.cisco.com [161.44.122.62]) by sj-core-1.cisco.com (8.12.10/8.12.6) with ESMTP id i9EEZwk4022202; Thu, 14 Oct 2004 07:36:01 -0700 (PDT) Received: from aokmiansw2k ([161.44.64.231]) by flask.cisco.com (MOS 3.4.6-GR) with ESMTP id AMH03455; Thu, 14 Oct 2004 10:35:57 -0400 (EDT) Message-Id: <200410141435.AMH03455@flask.cisco.com> From: "Anton Okmianski" To: , "'Rainer Gerhards'" , Subject: RE: [Syslog-sec] RE: Maximum message size Date: Thu, 14 Oct 2004 10:35:56 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook, Build 11.0.5510 Thread-Index: AcSwoe8xQ3+YxqsUQ8iCTkpgGQfmqgAYjRlAAAuIk6AAIvA8sAAJc8ugAAU2HwA= X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4939.300 In-Reply-To: <20041014125304.B879E5C78F@willers.employees.org> X-BeenThere: syslog-sec@www.employees.org X-Mailman-Version: 2.1.4 Precedence: list List-Id: syslog-sec.www.employees.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: syslog-sec-bounces@willers.employees.org Errors-To: syslog-sec-bounces@willers.employees.org Content-Transfer-Encoding: 7bit Rainer: I agree with a SHOULD on 64k support on receivers and MAY on larger sizes. I think we need to remove or change this sentence: "A receiver MUST be able to accept messages up to and including 480 octets in length". 1. Is this an absolute requirement? We keep thinking about general purpose receiver libraries. But it does not have to be general purpose. If I have two specific applications with their own syslog sender/receiver implementations (how hard is it to send/receive a udp message) and they exchange syslog messages which are known not to exceed 400 bytes, should my receiver be required to support 8k messages? 2. Why 480? If this is intended to ensure basic interoperability, then at least this should be a size greater than 1K (size previous syslog RFC observed). Maybe 4k or 8k. At least it is a better arbitrary number. So, I would support either removing the MUST requirement all together or at least changing it to a more reasonable size. The latter approach would be more limiting for implementations, but will insure interoperability for general-purpose syslog libraries if this is the overriding goal. Thanks, Anton. > -----Original Message----- > From: syslog-sec-bounces@willers.employees.org > [mailto:syslog-sec-bounces@willers.employees.org] On Behalf > Of David B Harrington > Sent: Thursday, October 14, 2004 8:53 AM > To: 'Rainer Gerhards'; syslog-sec@employees.org > Subject: RE: [Syslog-sec] RE: Maximum message size > > Hi Rainer, > > Thanks for making the changes. > > I'd like to make one point a little more strenuously: > "For interoperability reasons, all receiver implementations > SHOULD be able to accept messages up to and including 65,535 > octets in length." > > Also, I think the last two sentences could be combined into > one - "If a receiver receives a message with a length larger > than 65,535 octets, or larger than it supports, the receiver > MAY discard the message or truncate the payload." > > dbh > > -----Original Message----- > From: syslog-sec-bounces@www.employees.org > [mailto:syslog-sec-bounces@www.employees.org] On Behalf Of > Rainer Gerhards > Sent: Thursday, October 14, 2004 7:34 AM > To: syslog-sec@employees.org > Subject: [Syslog-sec] RE: Maximum message size > > David and all, > > many thanks for all the good comments. > > I agree that 16 MB for a single message is far more than > actually should be used. Let's look a bit at the history of this: > > Initially, we noticed that the 1024 octet limit for current > syslog is too short. I think there is still general agreement > on this. We had a good discussion about the size issue in > September 2003. A link to an archive is here: > > http://www.syslog.cc/ietf/autoarc/msg00855.html > > The general issue is that 1K is too short for many > applications. For example, Marshall Glen mentions the > healthcare needs (RFC3881) for a larger message. As far as I > know, the IHE initiative currently standardizes on a maximum > XML stream size of 32K (which is then supposed to be > transmitted via syslog). I think a message size limit of 64K > would probably be suffcient for a long time to come (and > leaves some headroom for message content increases). > > Besides that, I myself identified a need - in very rare > occasions - to send information larger than 64K. In extreme > cases, this information could be as large as one megabyte. > Again, these were cases that might happen once in a year. All > of this, of course, only if an operator > *really* wants to submit that large information. So the > initial approach and suggestion was to introduce a way to > build "message groups". That would have been a way to split > "oversize messages" (as they were called > then) into multiple syslog messages. They key to that concept > was that each single message could be kept within a > reasonable limit (it would have worked even with 1K max > message size). It would only have been a way to transmit very > large messages in those very seldom cases that they would > actually be used. > > Some time later, we came to the conclusion that it should be > possible for a single syslog message to be of the largest > imaginable size. It was argued that fragmentation is a > transport issue and as such application-level "fragmentation" > (the message groups) would not do good. We agreed to that. > > Even some time later, we needed to define a maximum message > size, just to limit it somewhere (if I recall correctly, that > was a transport requirement). We picked 16MB just as a number > that we thought would never have be reached and such be > sufficient for all imaginable time to come. As far as I was > concerend, that decision was driven by the fact that in the > past there were often limits that nobody expected to reach - > and yet they wer quickly reached. > Som 16MB was picked more or less as "larger than anybody will > need anytime in the forseeable future". We did not even have > a big discussion on that limit. > > I think this are the key points on how we reached this limit. > Now, thanks to your call and looking the whole situation, I > really think we've gone overboard. That 16MB limit is > something nobody intends to use. Even a lower - but high > limit - is extremely unlikely. I'd expect to see a 1MB > "message" in maybe 0.0001% of the cases - if at all (but I > expect to see e.g. 70KB "message" in maybe 0.001% of the cases). > > Also, I have read your proposal to set a maxium size for the > message, but allow an implementation to go beyond that. Maybe > something like > this: > > #### > 5.1 Message Length > > A receiver MUST be able to accept messages up to and > including 480 octets in length. > > A receiver SHOULD be able to accept messages up to and including > 65,535 octets in length. > If a receiver receives a message with a length larger than > it supports, the receiver MAY > discard the message or truncate the payload. > > A receiver MAY accept messages larger than 65,535 octets. > #### > > I've dropped the wording that transports MAY support only a > smaller max size - I think UDP was the most critical in this > regard and it obviously handles 64K. So I assume this clause > is not really needed if we go for 64K as the max size. > > With that clause, we would limit the size for all interop > cases, but leave a window open so that the message length > could be extended if there is a real need AND the operator > and software vendor wants this. > I assume nobody will implement this burden if there is no > market need to do so. But if it is, it clould at least be > implemented in a standards-compliant way. > > If I look back at the original requirement - provide a way to > transmit very rare oversize messages, I think we can safely > implement this the bounds of the current proposals, even if > the size limit does not go beyond 64K. If somebody needs it, > this can be easily done via an experimental (or > vendor-specific) SD-ID. Again, I do not think somebody will > accept this burden without market need. > > In the light of this, I, too, would recommend that we limit > the message size to 64K. I would prefer, however, that we > keep the maximum open as in the suggested wording above. > > Rainer > > > -----Original Message----- > > From: David B Harrington [mailto:ietfdbh@comcast.net] > > Sent: Wednesday, October 13, 2004 5:10 PM > > To: Rainer Gerhards; 'Anton Okmianski'; syslog-sec@employees.org > > Subject: Maximum message size > > > > Hi, > > > > I am concerned that we are in danger of promoting uses of syslog > that > > will defeat its current usefulness as a human-readable log of system > > > activity. > > > > Do network operators really believe the 16,777,216 octet size is a > > needed improvement to syslog, or are we designing this size in > > explicitly for the vendors of applications who want to use syslog as > a > > programmatic interface? I believe Syslog should be designed to meet > > the needs of operators. Most of the discussion ssems to be from > > vendors of syslog receivers or applications. Other protocols such as > > > FTP might be better used for such a specialty use case. > > > > Does allowing messages of 16,777,216 octets to be accumulated within > > > the system log make it harder to use some widely-available minimal > > text editors, like Notepad, to view the system log? Will having huge > > > application-specific messages in the system log make it harder for > an > > operator to locate useful troubleshooting messages within the system > > > log? Can the information in such a huge message fit within an 80x25 > > screen, when an operator is troubleshooting network problems and > needs > > to use a bare-bones terminal attached to the serial port of a device > > > to view the system log? > > > > Syslog was designing to be an operator's tool, and Syslog should be > > designed to meet the needs of operators. Will allowing messages of > > this size negatively impact its usefulness in the primary use case? > > > > What do network **operators** think about the need for messages of > > this size? Have we deliberately asked them their opinion on this by, > > > for example, taking a survey or going to NANOG to ask them? > > > > I don't think we should support such large messages in the system > > logging protocol. > > > > > > David Harrington > > dbharrington@comcast.net > > > > > > > > -----Original Message----- > > From: syslog-sec-bounces@www.employees.org > > [mailto:syslog-sec-bounces@www.employees.org] On Behalf Of Rainer > > Gerhards > > Sent: Wednesday, October 13, 2004 5:17 AM > > To: Anton Okmianski; syslog-sec@employees.org > > Subject: RE: [Syslog-sec] UDP message size issue proposal > > > > Anton, > > > > I agree to your conclusion. > > > > Although I am the one who always voted for larger sizes, the > > discussion has shown that this yields to unnecessary complexity. I > can > > satisfy my needs with a vendor-extension structured data element, > > which I think shows the flexibility of the new drafts. > > > > So I support your move. I would tend to even remove the transport > > version header. If there is need to, we could always include that in > a > > later release (e.g. "v1", which would make it clearly distingshable > > from the current frame format). I see little need to do so, though. > > > > Regarding -protocol, I think we should still keep an upper limit in > > the spec. Even I don't see any reason why a syslog message should go > > > above 16MB. So for -protocol, I propose the following new text: > > > > #### > > 5.1 Message Length > > > > The maximum length of any syslog message is 16,777,216 octets. > Any > > receiver receiving a larger message MUST discard the message. A > > diagnostic message SHOULD be logged in this case. > > > > A receiver MUST be able to accept messages that are 480 octets > (or > > less) in length. A receiver SHOULD be able to accept messages > that > > are 65,535 octets (or less) in length. It is RECOMMENDED that > > receivers be able to accept messages up to the maximum message > > length > > of 16,777,216 octets. > > > > If a receiver receives a message within the maximum length, but > > with > > a length larger than it handles, the receiver MAY discard or > > truncate > > it. > > > > A transport mapping MAY define a maximum length below the one > > specified here. Each transport mapping MUST support a maximum > size > > of 480 octets or more. > > #### > > > > If there is agreement on this, I can post a new version of > -protocol. > > That version will also include some changes made thanks to Andrew > Ross > > comments (sent via private mail). I have finished this version. It > is > > available for immediate posting, but I would like to wait for > > agreement on the size issue (at least if that can be reached > quickly). > > > > Rainer > > > > > -----Original Message----- > > > From: syslog-sec-bounces@www.employees.org > > > [mailto:syslog-sec-bounces@www.employees.org] On Behalf Of Anton > > > Okmianski > > > Sent: Tuesday, October 12, 2004 11:25 PM > > > To: syslog-sec@employees.org > > > Subject: [Syslog-sec] UDP message size issue proposal > > > > > > Hi! > > > > > > Sorry for a long delay on this issue - I was on a 2 week vacation. > > I > > > have spoken with a number of TCP/UDP/IP experts regarding the > sizing > > > > > issue. I am ready to propose the following changes: > > > > > > 1. Syslog-protocol will state that the max message will be defined > > by > > > the transport layer. > > > > > > 2. Syslog-transport-udp will support messages up to maximum UDP > > > datagram size of 64K. UDP is a very bad choice for large message > > > transmissions, so it does not make sense for us to stretch it by > > > adding our own fragmentation without other transmission control > > > features such as found in TCP. > > > > > > 3. Syslog-transport-udp will rely on IP fragmentation and we will > > get > > > rid of "proprietary" fragmentation which was designed to handle > > > messages over 64K and deal with various > > > non-compliant network hosts. > > > > > > 4. Syslog-transport-udp will recommend sending messages within the > > > > boundaries of prevalent MTU size on a given network to be safe. It > > > > will recommend Ethernet's 1500 bytes less headers and will draw > > reader > > > attention to the minimum MTU size hosts on the network are > required > > to > > > support for > > > IPv6 (576 bytes) and IPv6 (1280 bytes). > > > > > > 5. Path MTU discovery may not work robustly and some TCP/IP stacks > > may > > > not support UDP packets of full 64K size and truncate them. We > will > > > > > mention this and bite this bullet. > > > We should not restrict the protocol due to incompliant > > implementations > > > because it is a moving target and penalizes compliant > > implementations > > > with extra overhead. The above size recommendation would partially > > > > deal with this, but leave the > > > final choice up to the administrator. > > > > > > 6. We will get rid of most syslog transport headers for UDP as > they > > > will no longer be needed. The only thing that will be left is the > > > > transport protocol version prefixed to every syslog message. > Should > > > > > we even bother with that? > > > > > > This is a major change to the syslog-transport-udp. I'd like to > get > > > > > positive feedback before I proceed with this update. > > > The best part is that if we all agree on the above, the next draft > > > > will be 1/3 of the size -- easier read for you. :) > > > > > > Thanks, > > > Anton. > > > > > > > > > _______________________________________________ > > > Syslog-sec mailing list > > > Syslog-sec@www.employees.org > > > http://www.employees.org/mailman/listinfo/syslog-sec > > > > > _______________________________________________ > > Syslog-sec mailing list > > Syslog-sec@www.employees.org > > http://www.employees.org/mailman/listinfo/syslog-sec > > > > > > > _______________________________________________ > Syslog-sec mailing list > Syslog-sec@www.employees.org > http://www.employees.org/mailman/listinfo/syslog-sec > > > _______________________________________________ > Syslog-sec mailing list > Syslog-sec@www.employees.org > http://www.employees.org/mailman/listinfo/syslog-sec > _______________________________________________ Syslog-sec mailing list Syslog-sec@www.employees.org http://www.employees.org/mailman/listinfo/syslog-sec From syslog-sec-bounces@willers.employees.org Thu Oct 14 10:56:24 2004 Received: from willers.employees.org (willers.employees.org [192.83.249.36]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA27363 for ; Thu, 14 Oct 2004 10:56:24 -0400 (EDT) Received: from willers.employees.org (localhost.employees.org [127.0.0.1]) by willers.employees.org (Postfix) with ESMTP id 4DA305C7FE; Thu, 14 Oct 2004 07:56:25 -0700 (PDT) X-Original-To: syslog-sec@employees.org Delivered-To: syslog-sec@employees.org Received: from mail.hq.adiscon.com (mail.hq.adiscon.com [217.6.190.188]) by willers.employees.org (Postfix) with ESMTP id AB75F5C7CB for ; Thu, 14 Oct 2004 07:56:18 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by mail.hq.adiscon.com (Postfix) with ESMTP id 2F9DC9C757; Thu, 14 Oct 2004 17:13:29 +0200 (CEST) Received: from mail.hq.adiscon.com ([127.0.0.1]) by localhost (grfdeb [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 17368-10; Thu, 14 Oct 2004 17:13:02 +0200 (CEST) Received: from grfint2.intern.adiscon.com (grfint2 [172.19.0.6]) by mail.hq.adiscon.com (Postfix) with ESMTP id 8DB439C755; Thu, 14 Oct 2004 17:13:02 +0200 (CEST) Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: [Syslog-sec] RE: Maximum message size X-MimeOLE: Produced By Microsoft Exchange V6.5.7226.0 Date: Thu, 14 Oct 2004 16:59:50 +0200 Message-ID: <577465F99B41C842AAFBE9ED71E70ABA03019D@grfint2.intern.adiscon.com> Thread-Topic: [Syslog-sec] RE: Maximum message size Thread-Index: AcSwoe8xQ3+YxqsUQ8iCTkpgGQfmqgAYjRlAAAuIk6AAIvA8sAAJc8ugAAU2HwAAAS1K8A== From: "Rainer Gerhards" To: "Anton Okmianski" , , X-Virus-Scanned: by amavisd-new-20030616-p7 (Debian) at adiscon.com X-BeenThere: syslog-sec@www.employees.org X-Mailman-Version: 2.1.4 Precedence: list List-Id: syslog-sec.www.employees.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: syslog-sec-bounces@willers.employees.org Errors-To: syslog-sec-bounces@willers.employees.org Content-Transfer-Encoding: quoted-printable Hello Anton: the 480 octet limit stems back to your work and proposal in -transport-udp. I thought you found out that more than 480 bytes can not be transmitted "reliably" (whatever this means for UDP). Of course, I can remove this and we could simply state that senders send data and receivers receive data and that the receiver MAY receive any size or not. But wouldn't it make sense to have one size that I can count on to be received? The intension of the 480 octets was to provide a known message size that every implementation actually MUST support - so if an applications intends to send a message that MUST be received fully, that application knows a maximum size. If you mean that with the upcoming version of -transport-udp the 480 octet can safely be replaced by 1K or 2K then we of course can change this. I, however, think that a minimal burden should be placed on an implementation. I would tend to call a syslog receiver that is only capable of receiving 400 octets to be non-compliant. But, OK, maybe we should say this is OK. So what limit is it then? Is 200 octets compliant? 100? 50? Whatever it is, I think we should require one size... If we do not require this, one could argue writing/parsing a correct header is also too much of a burden for a given scenario... Rainer > -----Original Message----- > From: Anton Okmianski [mailto:aokmians@cisco.com]=20 > Sent: Thursday, October 14, 2004 4:36 PM > To: ietfdbh@comcast.net; Rainer Gerhards; syslog-sec@employees.org > Subject: RE: [Syslog-sec] RE: Maximum message size=20 >=20 > Rainer: >=20 > I agree with a SHOULD on 64k support on receivers and MAY on larger > sizes. =20 >=20 > I think we need to remove or change this sentence: "A receiver MUST be > able to accept messages up to and including 480 octets in length". =20 >=20 > 1. Is this an absolute requirement? We keep thinking about general > purpose receiver libraries. But it does not have to be general > purpose. If I have two specific applications with their own syslog > sender/receiver implementations (how hard is it to send/receive a udp > message) and they exchange syslog messages which are known not to > exceed 400 bytes, should my receiver be required to support 8k > messages?=20 >=20 > 2. Why 480? If this is intended to ensure basic interoperability, > then at least this should be a size greater than 1K (size previous > syslog RFC observed). Maybe 4k or 8k. At least it is a better > arbitrary number. =20 >=20 > So, I would support either removing the MUST requirement all together > or at least changing it to a more reasonable size. The latter approach > would be more limiting for implementations, but will insure > interoperability for general-purpose syslog libraries if this is the > overriding goal.=20 >=20 > Thanks, > Anton.=20 >=20 > > -----Original Message----- > > From: syslog-sec-bounces@willers.employees.org=20 > > [mailto:syslog-sec-bounces@willers.employees.org] On Behalf=20 > > Of David B Harrington > > Sent: Thursday, October 14, 2004 8:53 AM > > To: 'Rainer Gerhards'; syslog-sec@employees.org > > Subject: RE: [Syslog-sec] RE: Maximum message size=20 > >=20 > > Hi Rainer, > >=20 > > Thanks for making the changes. > >=20 > > I'd like to make one point a little more strenuously: > > "For interoperability reasons, all receiver implementations=20 > > SHOULD be able to accept messages up to and including 65,535=20 > > octets in length." > >=20 > > Also, I think the last two sentences could be combined into=20 > > one - "If a receiver receives a message with a length larger=20 > > than 65,535 octets, or larger than it supports, the receiver=20 > > MAY discard the message or truncate the payload." > >=20 > > dbh > >=20 > > -----Original Message----- > > From: syslog-sec-bounces@www.employees.org > > [mailto:syslog-sec-bounces@www.employees.org] On Behalf Of=20 > > Rainer Gerhards > > Sent: Thursday, October 14, 2004 7:34 AM > > To: syslog-sec@employees.org > > Subject: [Syslog-sec] RE: Maximum message size=20 > >=20 > > David and all, > >=20 > > many thanks for all the good comments. > >=20 > > I agree that 16 MB for a single message is far more than=20 > > actually should be used. Let's look a bit at the history of this: > >=20 > > Initially, we noticed that the 1024 octet limit for current=20 > > syslog is too short. I think there is still general agreement=20 > > on this. We had a good discussion about the size issue in=20 > > September 2003. A link to an archive is here: > >=20 > > http://www.syslog.cc/ietf/autoarc/msg00855.html > >=20 > > The general issue is that 1K is too short for many=20 > > applications. For example, Marshall Glen mentions the=20 > > healthcare needs (RFC3881) for a larger message. As far as I=20 > > know, the IHE initiative currently standardizes on a maximum=20 > > XML stream size of 32K (which is then supposed to be=20 > > transmitted via syslog). I think a message size limit of 64K=20 > > would probably be suffcient for a long time to come (and=20 > > leaves some headroom for message content increases). > >=20 > > Besides that, I myself identified a need - in very rare=20 > > occasions - to send information larger than 64K. In extreme=20 > > cases, this information could be as large as one megabyte.=20 > > Again, these were cases that might happen once in a year. All=20 > > of this, of course, only if an operator > > *really* wants to submit that large information. So the=20 > > initial approach and suggestion was to introduce a way to=20 > > build "message groups". That would have been a way to split=20 > > "oversize messages" (as they were called > > then) into multiple syslog messages. They key to that concept=20 > > was that each single message could be kept within a=20 > > reasonable limit (it would have worked even with 1K max=20 > > message size). It would only have been a way to transmit very=20 > > large messages in those very seldom cases that they would=20 > > actually be used. > >=20 > > Some time later, we came to the conclusion that it should be=20 > > possible for a single syslog message to be of the largest=20 > > imaginable size. It was argued that fragmentation is a=20 > > transport issue and as such application-level "fragmentation"=20 > > (the message groups) would not do good. We agreed to that. > >=20 > > Even some time later, we needed to define a maximum message=20 > > size, just to limit it somewhere (if I recall correctly, that=20 > > was a transport requirement). We picked 16MB just as a number=20 > > that we thought would never have be reached and such be=20 > > sufficient for all imaginable time to come. As far as I was=20 > > concerend, that decision was driven by the fact that in the=20 > > past there were often limits that nobody expected to reach -=20 > > and yet they wer quickly reached. > > Som 16MB was picked more or less as "larger than anybody will=20 > > need anytime in the forseeable future". We did not even have=20 > > a big discussion on that limit. > >=20 > > I think this are the key points on how we reached this limit.=20 > > Now, thanks to your call and looking the whole situation, I=20 > > really think we've gone overboard. That 16MB limit is=20 > > something nobody intends to use. Even a lower - but high=20 > > limit - is extremely unlikely. I'd expect to see a 1MB=20 > > "message" in maybe 0.0001% of the cases - if at all (but I=20 > > expect to see e.g. 70KB "message" in maybe 0.001% of the cases). > >=20 > > Also, I have read your proposal to set a maxium size for the=20 > > message, but allow an implementation to go beyond that. Maybe=20 > > something like > > this: > >=20 > > #### > > 5.1 Message Length > >=20 > > A receiver MUST be able to accept messages up to and=20 > > including 480 octets in length.=20 > >=20 > > A receiver SHOULD be able to accept messages up to and including > > 65,535 octets in length.=20 > > If a receiver receives a message with a length larger than=20 > > it supports, the receiver MAY=20 > > discard the message or truncate the payload. > >=20 > > A receiver MAY accept messages larger than 65,535 octets. > > #### > >=20 > > I've dropped the wording that transports MAY support only a=20 > > smaller max size - I think UDP was the most critical in this=20 > > regard and it obviously handles 64K. So I assume this clause=20 > > is not really needed if we go for 64K as the max size. > >=20 > > With that clause, we would limit the size for all interop=20 > > cases, but leave a window open so that the message length=20 > > could be extended if there is a real need AND the operator=20 > > and software vendor wants this. > > I assume nobody will implement this burden if there is no=20 > > market need to do so. But if it is, it clould at least be=20 > > implemented in a standards-compliant way. > >=20 > > If I look back at the original requirement - provide a way to=20 > > transmit very rare oversize messages, I think we can safely=20 > > implement this the bounds of the current proposals, even if=20 > > the size limit does not go beyond 64K. If somebody needs it,=20 > > this can be easily done via an experimental (or=20 > > vendor-specific) SD-ID. Again, I do not think somebody will=20 > > accept this burden without market need. > >=20 > > In the light of this, I, too, would recommend that we limit=20 > > the message size to 64K. I would prefer, however, that we=20 > > keep the maximum open as in the suggested wording above. > >=20 > > Rainer > >=20 > > > -----Original Message----- > > > From: David B Harrington [mailto:ietfdbh@comcast.net] > > > Sent: Wednesday, October 13, 2004 5:10 PM > > > To: Rainer Gerhards; 'Anton Okmianski'; syslog-sec@employees.org > > > Subject: Maximum message size > > >=20 > > > Hi, > > >=20 > > > I am concerned that we are in danger of promoting uses of syslog > > that=20 > > > will defeat its current usefulness as a human-readable log of > system > >=20 > > > activity. > > >=20 > > > Do network operators really believe the 16,777,216 octet size is a >=20 > > > needed improvement to syslog, or are we designing this size in=20 > > > explicitly for the vendors of applications who want to use syslog > as > > a=20 > > > programmatic interface? I believe Syslog should be designed to > meet=20 > > > the needs of operators. Most of the discussion ssems to be from=20 > > > vendors of syslog receivers or applications. Other protocols such > as > >=20 > > > FTP might be better used for such a specialty use case. > > >=20 > > > Does allowing messages of 16,777,216 octets to be accumulated > within > >=20 > > > the system log make it harder to use some widely-available minimal >=20 > > > text editors, like Notepad, to view the system log? Will having > huge > >=20 > > > application-specific messages in the system log make it harder for > > an=20 > > > operator to locate useful troubleshooting messages within the > system > >=20 > > > log? Can the information in such a huge message fit within an > 80x25=20 > > > screen, when an operator is troubleshooting network problems and > > needs=20 > > > to use a bare-bones terminal attached to the serial port of a > device > >=20 > > > to view the system log? > > >=20 > > > Syslog was designing to be an operator's tool, and Syslog should > be=20 > > > designed to meet the needs of operators. Will allowing messages > of=20 > > > this size negatively impact its usefulness in the primary use > case? > > >=20 > > > What do network **operators** think about the need for messages of >=20 > > > this size? Have we deliberately asked them their opinion on this > by, > >=20 > > > for example, taking a survey or going to NANOG to ask them? > > >=20 > > > I don't think we should support such large messages in the system=20 > > > logging protocol. > > >=20 > > >=20 > > > David Harrington > > > dbharrington@comcast.net > > >=20 > > >=20 > > >=20 > > > -----Original Message----- > > > From: syslog-sec-bounces@www.employees.org > > > [mailto:syslog-sec-bounces@www.employees.org] On Behalf Of Rainer=20 > > > Gerhards > > > Sent: Wednesday, October 13, 2004 5:17 AM > > > To: Anton Okmianski; syslog-sec@employees.org > > > Subject: RE: [Syslog-sec] UDP message size issue proposal > > >=20 > > > Anton, > > >=20 > > > I agree to your conclusion. > > >=20 > > > Although I am the one who always voted for larger sizes, the=20 > > > discussion has shown that this yields to unnecessary complexity. I > > can=20 > > > satisfy my needs with a vendor-extension structured data element,=20 > > > which I think shows the flexibility of the new drafts. > > >=20 > > > So I support your move. I would tend to even remove the transport=20 > > > version header. If there is need to, we could always include that > in > > a=20 > > > later release (e.g. "v1", which would make it clearly > distingshable=20 > > > from the current frame format). I see little need to do so, > though. > > >=20 > > > Regarding -protocol, I think we should still keep an upper limit > in=20 > > > the spec. Even I don't see any reason why a syslog message should > go > >=20 > > > above 16MB. So for -protocol, I propose the following new text: > > >=20 > > > #### > > > 5.1 Message Length > > >=20 > > > The maximum length of any syslog message is 16,777,216 octets. > > Any > > > receiver receiving a larger message MUST discard the message. > A > > > diagnostic message SHOULD be logged in this case. > > >=20 > > > A receiver MUST be able to accept messages that are 480 octets > > (or > > > less) in length. A receiver SHOULD be able to accept messages > > that > > > are 65,535 octets (or less) in length. It is RECOMMENDED that > > > receivers be able to accept messages up to the maximum message=20 > > > length > > > of 16,777,216 octets. > > >=20 > > > If a receiver receives a message within the maximum length, but >=20 > > > with > > > a length larger than it handles, the receiver MAY discard or=20 > > > truncate > > > it. > > >=20 > > > A transport mapping MAY define a maximum length below the one > > > specified here. Each transport mapping MUST support a maximum > > size > > > of 480 octets or more. > > > #### > > >=20 > > > If there is agreement on this, I can post a new version of > > -protocol. > > > That version will also include some changes made thanks to Andrew > > Ross=20 > > > comments (sent via private mail). I have finished this version. It > > is=20 > > > available for immediate posting, but I would like to wait for=20 > > > agreement on the size issue (at least if that can be reached > > quickly). > > >=20 > > > Rainer > > >=20 > > > > -----Original Message----- > > > > From: syslog-sec-bounces@www.employees.org > > > > [mailto:syslog-sec-bounces@www.employees.org] On Behalf Of Anton >=20 > > > > Okmianski > > > > Sent: Tuesday, October 12, 2004 11:25 PM > > > > To: syslog-sec@employees.org > > > > Subject: [Syslog-sec] UDP message size issue proposal > > > >=20 > > > > Hi! > > > >=20 > > > > Sorry for a long delay on this issue - I was on a 2 week > vacation. > > > I > > > > have spoken with a number of TCP/UDP/IP experts regarding the > > sizing > > >=20 > > > > issue. I am ready to propose the following changes: > > > >=20 > > > > 1. Syslog-protocol will state that the max message will be > defined > > > by > > > > the transport layer. > > > >=20 > > > > 2. Syslog-transport-udp will support messages up to maximum UDP=20 > > > > datagram size of 64K. UDP is a very bad choice for large message >=20 > > > > transmissions, so it does not make sense for us to stretch it by >=20 > > > > adding our own fragmentation without other transmission control=20 > > > > features such as found in TCP. > > > >=20 > > > > 3. Syslog-transport-udp will rely on IP fragmentation and we > will > > > get > > > > rid of "proprietary" fragmentation which was designed to handle=20 > > > > messages over 64K and deal with various > > > > non-compliant network hosts. =20 > > > >=20 > > > > 4. Syslog-transport-udp will recommend sending messages within > the > >=20 > > > > boundaries of prevalent MTU size on a given network to be safe. > It > >=20 > > > > will recommend Ethernet's 1500 bytes less headers and will draw > > > reader > > > > attention to the minimum MTU size hosts on the network are > > required > > > to > > > > support for > > > > IPv6 (576 bytes) and IPv6 (1280 bytes). =20 > > > >=20 > > > > 5. Path MTU discovery may not work robustly and some TCP/IP > stacks > > > may > > > > not support UDP packets of full 64K size and truncate them. We > > will > > >=20 > > > > mention this and bite this bullet. > > > > We should not restrict the protocol due to incompliant > > > implementations > > > > because it is a moving target and penalizes compliant > > > implementations > > > > with extra overhead. The above size recommendation would > partially > >=20 > > > > deal with this, but leave the > > > > final choice up to the administrator. =20 > > > >=20 > > > > 6. We will get rid of most syslog transport headers for UDP as > > they=20 > > > > will no longer be needed. The only thing that will be left is > the > >=20 > > > > transport protocol version prefixed to every syslog message. > > Should > > >=20 > > > > we even bother with that? > > > >=20 > > > > This is a major change to the syslog-transport-udp. I'd like to > > get > > >=20 > > > > positive feedback before I proceed with this update. > > > > The best part is that if we all agree on the above, the next > draft > >=20 > > > > will be 1/3 of the size -- easier read for you. :) > > > >=20 > > > > Thanks, > > > > Anton.=20 > > > > =20 > > > >=20 > > > > _______________________________________________ > > > > Syslog-sec mailing list > > > > Syslog-sec@www.employees.org > > > > http://www.employees.org/mailman/listinfo/syslog-sec > > > >=20 > > > _______________________________________________ > > > Syslog-sec mailing list > > > Syslog-sec@www.employees.org > > > http://www.employees.org/mailman/listinfo/syslog-sec > > >=20 > > >=20 > > >=20 > > _______________________________________________ > > Syslog-sec mailing list > > Syslog-sec@www.employees.org > > http://www.employees.org/mailman/listinfo/syslog-sec > >=20 > >=20 > > _______________________________________________ > > Syslog-sec mailing list > > Syslog-sec@www.employees.org > > http://www.employees.org/mailman/listinfo/syslog-sec > >=20 >=20 >=20 _______________________________________________ Syslog-sec mailing list Syslog-sec@www.employees.org http://www.employees.org/mailman/listinfo/syslog-sec From syslog-sec-bounces@willers.employees.org Thu Oct 14 12:04:32 2004 Received: from willers.employees.org (willers.employees.org [192.83.249.36]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA03174 for ; Thu, 14 Oct 2004 12:04:31 -0400 (EDT) Received: from willers.employees.org (localhost.employees.org [127.0.0.1]) by willers.employees.org (Postfix) with ESMTP id E18245C77D; Thu, 14 Oct 2004 09:04:32 -0700 (PDT) X-Original-To: syslog-sec@employees.org Delivered-To: syslog-sec@employees.org Received: from sccrmhc12.comcast.net (sccrmhc12.comcast.net [204.127.202.56]) by willers.employees.org (Postfix) with ESMTP id B8A4A5C76F for ; Thu, 14 Oct 2004 09:04:26 -0700 (PDT) Received: from djyxpy41 (h00104b8ce2a3.ne.client2.attbi.com[24.128.104.220]) by comcast.net (sccrmhc12) with SMTP id <2004101416042501200ft3qte>; Thu, 14 Oct 2004 16:04:25 +0000 From: "David B Harrington" To: "'Anton Okmianski'" , "'Rainer Gerhards'" , Subject: RE: [Syslog-sec] RE: Maximum message size Date: Thu, 14 Oct 2004 12:04:24 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook, Build 11.0.5510 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1441 In-Reply-To: <200410141435.AMH03455@flask.cisco.com> Thread-Index: AcSwoe8xQ3+YxqsUQ8iCTkpgGQfmqgAYjRlAAAuIk6AAIvA8sAAJc8ugAAU2HwAAA5bRkA== Message-Id: <20041014160426.B8A4A5C76F@willers.employees.org> X-BeenThere: syslog-sec@www.employees.org X-Mailman-Version: 2.1.4 Precedence: list Reply-To: ietfdbh@comcast.net List-Id: syslog-sec.www.employees.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: syslog-sec-bounces@willers.employees.org Errors-To: syslog-sec-bounces@willers.employees.org Content-Transfer-Encoding: 7bit Keep in mind that the IETF is defining standards for cross-vendor interoperability, not providing an umbrella for everybody's implementation choices. You can certainly build your implementation to support only 400 bytes if that is what your vendor-specific usage demands; but you just cannot claim compliance to the IETF standard for syslog. I believe the 480 text should remain as the minimum size standard for an IETF compliant vendor-interoperable implementation. dbh -----Original Message----- From: Anton Okmianski [mailto:aokmians@cisco.com] Sent: Thursday, October 14, 2004 10:36 AM To: ietfdbh@comcast.net; 'Rainer Gerhards'; syslog-sec@employees.org Subject: RE: [Syslog-sec] RE: Maximum message size Rainer: I agree with a SHOULD on 64k support on receivers and MAY on larger sizes. I think we need to remove or change this sentence: "A receiver MUST be able to accept messages up to and including 480 octets in length". 1. Is this an absolute requirement? We keep thinking about general purpose receiver libraries. But it does not have to be general purpose. If I have two specific applications with their own syslog sender/receiver implementations (how hard is it to send/receive a udp message) and they exchange syslog messages which are known not to exceed 400 bytes, should my receiver be required to support 8k messages? 2. Why 480? If this is intended to ensure basic interoperability, then at least this should be a size greater than 1K (size previous syslog RFC observed). Maybe 4k or 8k. At least it is a better arbitrary number. So, I would support either removing the MUST requirement all together or at least changing it to a more reasonable size. The latter approach would be more limiting for implementations, but will insure interoperability for general-purpose syslog libraries if this is the overriding goal. Thanks, Anton. > -----Original Message----- > From: syslog-sec-bounces@willers.employees.org > [mailto:syslog-sec-bounces@willers.employees.org] On Behalf Of David B > Harrington > Sent: Thursday, October 14, 2004 8:53 AM > To: 'Rainer Gerhards'; syslog-sec@employees.org > Subject: RE: [Syslog-sec] RE: Maximum message size > > Hi Rainer, > > Thanks for making the changes. > > I'd like to make one point a little more strenuously: > "For interoperability reasons, all receiver implementations SHOULD be > able to accept messages up to and including 65,535 octets in length." > > Also, I think the last two sentences could be combined into one - "If > a receiver receives a message with a length larger than 65,535 octets, > or larger than it supports, the receiver MAY discard the message or > truncate the payload." > > dbh > > -----Original Message----- > From: syslog-sec-bounces@www.employees.org > [mailto:syslog-sec-bounces@www.employees.org] On Behalf Of Rainer > Gerhards > Sent: Thursday, October 14, 2004 7:34 AM > To: syslog-sec@employees.org > Subject: [Syslog-sec] RE: Maximum message size > > David and all, > > many thanks for all the good comments. > > I agree that 16 MB for a single message is far more than actually > should be used. Let's look a bit at the history of this: > > Initially, we noticed that the 1024 octet limit for current syslog is > too short. I think there is still general agreement on this. We had a > good discussion about the size issue in September 2003. A link to an > archive is here: > > http://www.syslog.cc/ietf/autoarc/msg00855.html > > The general issue is that 1K is too short for many applications. For > example, Marshall Glen mentions the healthcare needs (RFC3881) for a > larger message. As far as I know, the IHE initiative currently > standardizes on a maximum XML stream size of 32K (which is then > supposed to be transmitted via syslog). I think a message size limit > of 64K would probably be suffcient for a long time to come (and leaves > some headroom for message content increases). > > Besides that, I myself identified a need - in very rare occasions - to > send information larger than 64K. In extreme cases, this information > could be as large as one megabyte. > Again, these were cases that might happen once in a year. All of this, > of course, only if an operator > *really* wants to submit that large information. So the initial > approach and suggestion was to introduce a way to build "message > groups". That would have been a way to split "oversize messages" (as > they were called > then) into multiple syslog messages. They key to that concept was that > each single message could be kept within a reasonable limit (it would > have worked even with 1K max message size). It would only have been a > way to transmit very large messages in those very seldom cases that > they would actually be used. > > Some time later, we came to the conclusion that it should be possible > for a single syslog message to be of the largest imaginable size. It > was argued that fragmentation is a transport issue and as such > application-level "fragmentation" > (the message groups) would not do good. We agreed to that. > > Even some time later, we needed to define a maximum message size, just > to limit it somewhere (if I recall correctly, that was a transport > requirement). We picked 16MB just as a number that we thought would > never have be reached and such be sufficient for all imaginable time > to come. As far as I was concerend, that decision was driven by the > fact that in the past there were often limits that nobody expected to > reach - and yet they wer quickly reached. > Som 16MB was picked more or less as "larger than anybody will need > anytime in the forseeable future". We did not even have a big > discussion on that limit. > > I think this are the key points on how we reached this limit. > Now, thanks to your call and looking the whole situation, I really > think we've gone overboard. That 16MB limit is something nobody > intends to use. Even a lower - but high limit - is extremely unlikely. > I'd expect to see a 1MB "message" in maybe 0.0001% of the cases - if > at all (but I expect to see e.g. 70KB "message" in maybe 0.001% of the > cases). > > Also, I have read your proposal to set a maxium size for the message, > but allow an implementation to go beyond that. Maybe something like > this: > > #### > 5.1 Message Length > > A receiver MUST be able to accept messages up to and including 480 > octets in length. > > A receiver SHOULD be able to accept messages up to and including > 65,535 octets in length. > If a receiver receives a message with a length larger than it > supports, the receiver MAY > discard the message or truncate the payload. > > A receiver MAY accept messages larger than 65,535 octets. > #### > > I've dropped the wording that transports MAY support only a smaller > max size - I think UDP was the most critical in this regard and it > obviously handles 64K. So I assume this clause is not really needed if > we go for 64K as the max size. > > With that clause, we would limit the size for all interop cases, but > leave a window open so that the message length could be extended if > there is a real need AND the operator and software vendor wants this. > I assume nobody will implement this burden if there is no market need > to do so. But if it is, it clould at least be implemented in a > standards-compliant way. > > If I look back at the original requirement - provide a way to transmit > very rare oversize messages, I think we can safely implement this the > bounds of the current proposals, even if the size limit does not go > beyond 64K. If somebody needs it, this can be easily done via an > experimental (or > vendor-specific) SD-ID. Again, I do not think somebody will accept > this burden without market need. > > In the light of this, I, too, would recommend that we limit the > message size to 64K. I would prefer, however, that we keep the maximum > open as in the suggested wording above. > > Rainer > > > -----Original Message----- > > From: David B Harrington [mailto:ietfdbh@comcast.net] > > Sent: Wednesday, October 13, 2004 5:10 PM > > To: Rainer Gerhards; 'Anton Okmianski'; syslog-sec@employees.org > > Subject: Maximum message size > > > > Hi, > > > > I am concerned that we are in danger of promoting uses of syslog > that > > will defeat its current usefulness as a human-readable log of system > > > activity. > > > > Do network operators really believe the 16,777,216 octet size is a > > needed improvement to syslog, or are we designing this size in > > explicitly for the vendors of applications who want to use syslog as > a > > programmatic interface? I believe Syslog should be designed to meet > > the needs of operators. Most of the discussion ssems to be from > > vendors of syslog receivers or applications. Other protocols such as > > > FTP might be better used for such a specialty use case. > > > > Does allowing messages of 16,777,216 octets to be accumulated within > > > the system log make it harder to use some widely-available minimal > > text editors, like Notepad, to view the system log? Will having huge > > > application-specific messages in the system log make it harder for > an > > operator to locate useful troubleshooting messages within the system > > > log? Can the information in such a huge message fit within an 80x25 > > screen, when an operator is troubleshooting network problems and > needs > > to use a bare-bones terminal attached to the serial port of a device > > > to view the system log? > > > > Syslog was designing to be an operator's tool, and Syslog should be > > designed to meet the needs of operators. Will allowing messages of > > this size negatively impact its usefulness in the primary use case? > > > > What do network **operators** think about the need for messages of > > this size? Have we deliberately asked them their opinion on this by, > > > for example, taking a survey or going to NANOG to ask them? > > > > I don't think we should support such large messages in the system > > logging protocol. > > > > > > David Harrington > > dbharrington@comcast.net > > > > > > > > -----Original Message----- > > From: syslog-sec-bounces@www.employees.org > > [mailto:syslog-sec-bounces@www.employees.org] On Behalf Of Rainer > > Gerhards > > Sent: Wednesday, October 13, 2004 5:17 AM > > To: Anton Okmianski; syslog-sec@employees.org > > Subject: RE: [Syslog-sec] UDP message size issue proposal > > > > Anton, > > > > I agree to your conclusion. > > > > Although I am the one who always voted for larger sizes, the > > discussion has shown that this yields to unnecessary complexity. I > can > > satisfy my needs with a vendor-extension structured data element, > > which I think shows the flexibility of the new drafts. > > > > So I support your move. I would tend to even remove the transport > > version header. If there is need to, we could always include that in > a > > later release (e.g. "v1", which would make it clearly distingshable > > from the current frame format). I see little need to do so, though. > > > > Regarding -protocol, I think we should still keep an upper limit in > > the spec. Even I don't see any reason why a syslog message should go > > > above 16MB. So for -protocol, I propose the following new text: > > > > #### > > 5.1 Message Length > > > > The maximum length of any syslog message is 16,777,216 octets. > Any > > receiver receiving a larger message MUST discard the message. A > > diagnostic message SHOULD be logged in this case. > > > > A receiver MUST be able to accept messages that are 480 octets > (or > > less) in length. A receiver SHOULD be able to accept messages > that > > are 65,535 octets (or less) in length. It is RECOMMENDED that > > receivers be able to accept messages up to the maximum message > > length > > of 16,777,216 octets. > > > > If a receiver receives a message within the maximum length, but > > with > > a length larger than it handles, the receiver MAY discard or > > truncate > > it. > > > > A transport mapping MAY define a maximum length below the one > > specified here. Each transport mapping MUST support a maximum > size > > of 480 octets or more. > > #### > > > > If there is agreement on this, I can post a new version of > -protocol. > > That version will also include some changes made thanks to Andrew > Ross > > comments (sent via private mail). I have finished this version. It > is > > available for immediate posting, but I would like to wait for > > agreement on the size issue (at least if that can be reached > quickly). > > > > Rainer > > > > > -----Original Message----- > > > From: syslog-sec-bounces@www.employees.org > > > [mailto:syslog-sec-bounces@www.employees.org] On Behalf Of Anton > > > Okmianski > > > Sent: Tuesday, October 12, 2004 11:25 PM > > > To: syslog-sec@employees.org > > > Subject: [Syslog-sec] UDP message size issue proposal > > > > > > Hi! > > > > > > Sorry for a long delay on this issue - I was on a 2 week vacation. > > I > > > have spoken with a number of TCP/UDP/IP experts regarding the > sizing > > > > > issue. I am ready to propose the following changes: > > > > > > 1. Syslog-protocol will state that the max message will be defined > > by > > > the transport layer. > > > > > > 2. Syslog-transport-udp will support messages up to maximum UDP > > > datagram size of 64K. UDP is a very bad choice for large message > > > transmissions, so it does not make sense for us to stretch it by > > > adding our own fragmentation without other transmission control > > > features such as found in TCP. > > > > > > 3. Syslog-transport-udp will rely on IP fragmentation and we will > > get > > > rid of "proprietary" fragmentation which was designed to handle > > > messages over 64K and deal with various > > > non-compliant network hosts. > > > > > > 4. Syslog-transport-udp will recommend sending messages within the > > > > boundaries of prevalent MTU size on a given network to be safe. It > > > > will recommend Ethernet's 1500 bytes less headers and will draw > > reader > > > attention to the minimum MTU size hosts on the network are > required > > to > > > support for > > > IPv6 (576 bytes) and IPv6 (1280 bytes). > > > > > > 5. Path MTU discovery may not work robustly and some TCP/IP stacks > > may > > > not support UDP packets of full 64K size and truncate them. We > will > > > > > mention this and bite this bullet. > > > We should not restrict the protocol due to incompliant > > implementations > > > because it is a moving target and penalizes compliant > > implementations > > > with extra overhead. The above size recommendation would partially > > > > deal with this, but leave the > > > final choice up to the administrator. > > > > > > 6. We will get rid of most syslog transport headers for UDP as > they > > > will no longer be needed. The only thing that will be left is the > > > > transport protocol version prefixed to every syslog message. > Should > > > > > we even bother with that? > > > > > > This is a major change to the syslog-transport-udp. I'd like to > get > > > > > positive feedback before I proceed with this update. > > > The best part is that if we all agree on the above, the next draft > > > > will be 1/3 of the size -- easier read for you. :) > > > > > > Thanks, > > > Anton. > > > > > > > > > _______________________________________________ > > > Syslog-sec mailing list > > > Syslog-sec@www.employees.org > > > http://www.employees.org/mailman/listinfo/syslog-sec > > > > > _______________________________________________ > > Syslog-sec mailing list > > Syslog-sec@www.employees.org > > http://www.employees.org/mailman/listinfo/syslog-sec > > > > > > > _______________________________________________ > Syslog-sec mailing list > Syslog-sec@www.employees.org > http://www.employees.org/mailman/listinfo/syslog-sec > > > _______________________________________________ > Syslog-sec mailing list > Syslog-sec@www.employees.org > http://www.employees.org/mailman/listinfo/syslog-sec > _______________________________________________ Syslog-sec mailing list Syslog-sec@www.employees.org http://www.employees.org/mailman/listinfo/syslog-sec From syslog-sec-bounces@willers.employees.org Thu Oct 14 12:29:01 2004 Received: from willers.employees.org (willers.employees.org [192.83.249.36]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA04898 for ; Thu, 14 Oct 2004 12:29:00 -0400 (EDT) Received: from willers.employees.org (localhost.employees.org [127.0.0.1]) by willers.employees.org (Postfix) with ESMTP id A01D15C7A9; Thu, 14 Oct 2004 09:29:02 -0700 (PDT) X-Original-To: syslog-sec@employees.org Delivered-To: syslog-sec@employees.org Received: from mail.hq.adiscon.com (mail.hq.adiscon.com [217.6.190.188]) by willers.employees.org (Postfix) with ESMTP id A07945C775 for ; Thu, 14 Oct 2004 09:28:56 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by mail.hq.adiscon.com (Postfix) with ESMTP id 9854E9C757; Thu, 14 Oct 2004 18:46:07 +0200 (CEST) Received: from mail.hq.adiscon.com ([127.0.0.1]) by localhost (grfdeb [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 17640-04; Thu, 14 Oct 2004 18:45:44 +0200 (CEST) Received: from grfint2.intern.adiscon.com (grfint2 [172.19.0.6]) by mail.hq.adiscon.com (Postfix) with ESMTP id 241619C755; Thu, 14 Oct 2004 18:45:44 +0200 (CEST) Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: [Syslog-sec] RE: Maximum message size X-MimeOLE: Produced By Microsoft Exchange V6.5.7226.0 Date: Thu, 14 Oct 2004 18:32:35 +0200 Message-ID: <577465F99B41C842AAFBE9ED71E70ABA0301A0@grfint2.intern.adiscon.com> Thread-Topic: [Syslog-sec] RE: Maximum message size Thread-Index: AcSwoe8xQ3+YxqsUQ8iCTkpgGQfmqgAYjRlAAAuIk6AAIvA8sAALriigAAdyA+A= From: "Rainer Gerhards" To: , X-Virus-Scanned: by amavisd-new-20030616-p7 (Debian) at adiscon.com X-BeenThere: syslog-sec@www.employees.org X-Mailman-Version: 2.1.4 Precedence: list List-Id: syslog-sec.www.employees.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: syslog-sec-bounces@willers.employees.org Errors-To: syslog-sec-bounces@willers.employees.org Content-Transfer-Encoding: quoted-printable David, I see your point. I have no objections in limiting the message size to 2K as long as we keep the door open for senders/receivers supporting larger message sizes. Then the market can decide ... and if there is need for larger sizes, an application can implement it and still claim compliance to the syslog standard. For example, it might be a competitive advantage to support 32K messages, so that a product can be used for the healthcare environment. As such, a vendor might want to implement it. As long as we do not disallow this, the small size is not a problem. And, yes, I agree it will encourage implementors to create short messages. So I am now proposing the following text: #### 5.1 Message Length A receiver MUST be able to accept messages up to and including 480 octets in length. For interoperability reasons, all receiver implementations=20 SHOULD be able to accept messages up to and including 2,048 octets=20 in length. If a receiver receives a message with a length larger than 2,048 octets, or larger than it supports, the receiver MAY discard the message or truncate the payload. #### =20 Rainer > -----Original Message----- > From: David B Harrington [mailto:ietfdbh@comcast.net]=20 > Sent: Thursday, October 14, 2004 3:21 PM > To: Rainer Gerhards; syslog-sec@employees.org > Subject: RE: [Syslog-sec] RE: Maximum message size=20 >=20 > Hi, >=20 > I have concerns about the requirements for maximum message size, and > would like to propose an alternate approach. >=20 > Most syslog messages currently are less than 1K in size. There is > consensus that 1K is typically acceptable, but it is a bit restrictive > in some circunstances and the size should be increased. I agree with > that consensus. I disagree that the size needs to be increased 64 > times to accommodate normal usage, and I recognize that some valid, > but abnormal, usages do require larger sizes. >=20 > I think it is an unnecessary burden on receiver implementors to be > required to receive 64K messages when 98% of all syslog messages are > likely to be <1K, and 99.9% will be under <2K, just to support those > unusual cases where messages may be as large as 32K or 70K. I dislike > this approach because if all receivers SHOULD support 64k messages, > that will tend to encourage implementors of message generators to make > their messages large and wordy rather than succinct, which could have > a negative impact on the primary use case for syslog. If we set the > required message support to be smaller, most implementors will keep > their messages smaller.=20 >=20 > I suggest that the required maximum size be more on the order of 2k, > 4k, or 8k rather than 64k. This reduced receiver requirement should be > accommpanied by advice that some operational environments, such as > that specified in RFC3881, may require support for larger messages, > and reciever implementors MAY support larger messages or MAY make the > maximum message size a value that is configurable by the administrator > of the system, based on their operational needs. >=20 > This would keep most syslog messages small, yet accommodate special > operational needs such as health care security auditing. >=20 > David Harrington > dbharrington@comcast.net >=20 >=20 > -----Original Message----- > From: syslog-sec-bounces@www.employees.org > [mailto:syslog-sec-bounces@www.employees.org] On Behalf Of Rainer > Gerhards > Sent: Thursday, October 14, 2004 7:34 AM > To: syslog-sec@employees.org > Subject: [Syslog-sec] RE: Maximum message size=20 >=20 > David and all, >=20 > many thanks for all the good comments. >=20 > I agree that 16 MB for a single message is far more than actually > should be used. Let's look a bit at the history of this: >=20 > Initially, we noticed that the 1024 octet limit for current syslog is > too short. I think there is still general agreement on this. We had a > good discussion about the size issue in September 2003. A link to an > archive is here: >=20 > http://www.syslog.cc/ietf/autoarc/msg00855.html >=20 > The general issue is that 1K is too short for many applications. For > example, Marshall Glen mentions the healthcare needs (RFC3881) for a > larger message. As far as I know, the IHE initiative currently > standardizes on a maximum XML stream size of 32K (which is then > supposed to be transmitted via syslog). I think a message size limit > of 64K would probably be suffcient for a long time to come (and leaves > some headroom for message content increases). >=20 > Besides that, I myself identified a need - in very rare occasions - to > send information larger than 64K. In extreme cases, this information > could be as large as one megabyte. Again, these were cases that might > happen once in a year. All of this, of course, only if an operator > *really* wants to submit that large information. So the initial > approach and suggestion was to introduce a way to build "message > groups". That would have been a way to split "oversize messages" (as > they were called > then) into multiple syslog messages. They key to that concept was that > each single message could be kept within a reasonable limit (it would > have worked even with 1K max message size). It would only have been a > way to transmit very large messages in those very seldom cases that > they would actually be used. >=20 > Some time later, we came to the conclusion that it should be possible > for a single syslog message to be of the largest imaginable size. It > was argued that fragmentation is a transport issue and as such > application-level "fragmentation" (the message groups) would not do > good. We agreed to that. >=20 > Even some time later, we needed to define a maximum message size, just > to limit it somewhere (if I recall correctly, that was a transport > requirement). We picked 16MB just as a number that we thought would > never have be reached and such be sufficient for all imaginable time > to come. As far as I was concerend, that decision was driven by the > fact that in the past there were often limits that nobody expected to > reach - and yet they wer quickly reached. > Som 16MB was picked more or less as "larger than anybody will need > anytime in the forseeable future". We did not even have a big > discussion on that limit. >=20 > I think this are the key points on how we reached this limit. Now, > thanks to your call and looking the whole situation, I really think > we've gone overboard. That 16MB limit is something nobody intends to > use. Even a lower - but high limit - is extremely unlikely. I'd expect > to see a 1MB "message" in maybe 0.0001% of the cases - if at all (but > I expect to see e.g. 70KB "message" in maybe 0.001% of the cases). >=20 > Also, I have read your proposal to set a maxium size for the message, > but allow an implementation to go beyond that. Maybe something like > this: >=20 > #### > 5.1 Message Length >=20 > A receiver MUST be able to accept messages up to and including 480 > octets in length.=20 >=20 > A receiver SHOULD be able to accept messages up to and including > 65,535 octets in length.=20 > If a receiver receives a message with a length larger than it > supports, the receiver MAY=20 > discard the message or truncate the payload. >=20 > A receiver MAY accept messages larger than 65,535 octets. > #### >=20 > I've dropped the wording that transports MAY support only a smaller > max size - I think UDP was the most critical in this regard and it > obviously handles 64K. So I assume this clause is not really needed if > we go for 64K as the max size. >=20 > With that clause, we would limit the size for all interop cases, but > leave a window open so that the message length could be extended if > there is a real need AND the operator and software vendor wants this. > I assume nobody will implement this burden if there is no market need > to do so. But if it is, it clould at least be implemented in a > standards-compliant way. >=20 > If I look back at the original requirement - provide a way to transmit > very rare oversize messages, I think we can safely implement this the > bounds of the current proposals, even if the size limit does not go > beyond 64K. If somebody needs it, this can be easily done via an > experimental (or vendor-specific) SD-ID. Again, I do not think > somebody will accept this burden without market need. >=20 > In the light of this, I, too, would recommend that we limit the > message size to 64K. I would prefer, however, that we keep the maximum > open as in the suggested wording above. >=20 > Rainer >=20 > > -----Original Message----- > > From: David B Harrington [mailto:ietfdbh@comcast.net] > > Sent: Wednesday, October 13, 2004 5:10 PM > > To: Rainer Gerhards; 'Anton Okmianski'; syslog-sec@employees.org > > Subject: Maximum message size > >=20 > > Hi, > >=20 > > I am concerned that we are in danger of promoting uses of syslog > that=20 > > will defeat its current usefulness as a human-readable log of system >=20 > > activity. > >=20 > > Do network operators really believe the 16,777,216 octet size is a=20 > > needed improvement to syslog, or are we designing this size in=20 > > explicitly for the vendors of applications who want to use syslog as > a=20 > > programmatic interface? I believe Syslog should be designed to meet=20 > > the needs of operators. Most of the discussion ssems to be from=20 > > vendors of syslog receivers or applications. Other protocols such as >=20 > > FTP might be better used for such a specialty use case. > >=20 > > Does allowing messages of 16,777,216 octets to be accumulated within >=20 > > the system log make it harder to use some widely-available minimal=20 > > text editors, like Notepad, to view the system log? Will having huge >=20 > > application-specific messages in the system log make it harder for > an=20 > > operator to locate useful troubleshooting messages within the system >=20 > > log? Can the information in such a huge message fit within an 80x25=20 > > screen, when an operator is troubleshooting network problems and > needs=20 > > to use a bare-bones terminal attached to the serial port of a device >=20 > > to view the system log? > >=20 > > Syslog was designing to be an operator's tool, and Syslog should be=20 > > designed to meet the needs of operators. Will allowing messages of=20 > > this size negatively impact its usefulness in the primary use case? > >=20 > > What do network **operators** think about the need for messages of=20 > > this size? Have we deliberately asked them their opinion on this by, >=20 > > for example, taking a survey or going to NANOG to ask them? > >=20 > > I don't think we should support such large messages in the system=20 > > logging protocol. > >=20 > >=20 > > David Harrington > > dbharrington@comcast.net > >=20 > >=20 > >=20 > > -----Original Message----- > > From: syslog-sec-bounces@www.employees.org > > [mailto:syslog-sec-bounces@www.employees.org] On Behalf Of Rainer=20 > > Gerhards > > Sent: Wednesday, October 13, 2004 5:17 AM > > To: Anton Okmianski; syslog-sec@employees.org > > Subject: RE: [Syslog-sec] UDP message size issue proposal > >=20 > > Anton, > >=20 > > I agree to your conclusion. > >=20 > > Although I am the one who always voted for larger sizes, the=20 > > discussion has shown that this yields to unnecessary complexity. I > can=20 > > satisfy my needs with a vendor-extension structured data element,=20 > > which I think shows the flexibility of the new drafts. > >=20 > > So I support your move. I would tend to even remove the transport=20 > > version header. If there is need to, we could always include that in > a=20 > > later release (e.g. "v1", which would make it clearly distingshable=20 > > from the current frame format). I see little need to do so, though. > >=20 > > Regarding -protocol, I think we should still keep an upper limit in=20 > > the spec. Even I don't see any reason why a syslog message should go >=20 > > above 16MB. So for -protocol, I propose the following new text: > >=20 > > #### > > 5.1 Message Length > >=20 > > The maximum length of any syslog message is 16,777,216 octets. > Any > > receiver receiving a larger message MUST discard the message. A > > diagnostic message SHOULD be logged in this case. > >=20 > > A receiver MUST be able to accept messages that are 480 octets > (or > > less) in length. A receiver SHOULD be able to accept messages > that > > are 65,535 octets (or less) in length. It is RECOMMENDED that > > receivers be able to accept messages up to the maximum message=20 > > length > > of 16,777,216 octets. > >=20 > > If a receiver receives a message within the maximum length, but=20 > > with > > a length larger than it handles, the receiver MAY discard or=20 > > truncate > > it. > >=20 > > A transport mapping MAY define a maximum length below the one > > specified here. Each transport mapping MUST support a maximum > size > > of 480 octets or more. > > #### > >=20 > > If there is agreement on this, I can post a new version of > -protocol. > > That version will also include some changes made thanks to Andrew > Ross=20 > > comments (sent via private mail). I have finished this version. It > is=20 > > available for immediate posting, but I would like to wait for=20 > > agreement on the size issue (at least if that can be reached > quickly). > >=20 > > Rainer > >=20 > > > -----Original Message----- > > > From: syslog-sec-bounces@www.employees.org > > > [mailto:syslog-sec-bounces@www.employees.org] On Behalf Of Anton=20 > > > Okmianski > > > Sent: Tuesday, October 12, 2004 11:25 PM > > > To: syslog-sec@employees.org > > > Subject: [Syslog-sec] UDP message size issue proposal > > >=20 > > > Hi! > > >=20 > > > Sorry for a long delay on this issue - I was on a 2 week vacation. > > I > > > have spoken with a number of TCP/UDP/IP experts regarding the > sizing > >=20 > > > issue. I am ready to propose the following changes: > > >=20 > > > 1. Syslog-protocol will state that the max message will be defined > > by > > > the transport layer. > > >=20 > > > 2. Syslog-transport-udp will support messages up to maximum UDP=20 > > > datagram size of 64K. UDP is a very bad choice for large message=20 > > > transmissions, so it does not make sense for us to stretch it by=20 > > > adding our own fragmentation without other transmission control=20 > > > features such as found in TCP. > > >=20 > > > 3. Syslog-transport-udp will rely on IP fragmentation and we will > > get > > > rid of "proprietary" fragmentation which was designed to handle=20 > > > messages over 64K and deal with various > > > non-compliant network hosts. =20 > > >=20 > > > 4. Syslog-transport-udp will recommend sending messages within the >=20 > > > boundaries of prevalent MTU size on a given network to be safe. It >=20 > > > will recommend Ethernet's 1500 bytes less headers and will draw > > reader > > > attention to the minimum MTU size hosts on the network are > required > > to > > > support for > > > IPv6 (576 bytes) and IPv6 (1280 bytes). =20 > > >=20 > > > 5. Path MTU discovery may not work robustly and some TCP/IP stacks > > may > > > not support UDP packets of full 64K size and truncate them. We > will > >=20 > > > mention this and bite this bullet. > > > We should not restrict the protocol due to incompliant > > implementations > > > because it is a moving target and penalizes compliant > > implementations > > > with extra overhead. The above size recommendation would partially >=20 > > > deal with this, but leave the > > > final choice up to the administrator. =20 > > >=20 > > > 6. We will get rid of most syslog transport headers for UDP as > they=20 > > > will no longer be needed. The only thing that will be left is the >=20 > > > transport protocol version prefixed to every syslog message. > Should > >=20 > > > we even bother with that? > > >=20 > > > This is a major change to the syslog-transport-udp. I'd like to > get > >=20 > > > positive feedback before I proceed with this update. > > > The best part is that if we all agree on the above, the next draft >=20 > > > will be 1/3 of the size -- easier read for you. :) > > >=20 > > > Thanks, > > > Anton.=20 > > > =20 > > >=20 > > > _______________________________________________ > > > Syslog-sec mailing list > > > Syslog-sec@www.employees.org > > > http://www.employees.org/mailman/listinfo/syslog-sec > > >=20 > > _______________________________________________ > > Syslog-sec mailing list > > Syslog-sec@www.employees.org > > http://www.employees.org/mailman/listinfo/syslog-sec > >=20 > >=20 > >=20 > _______________________________________________ > Syslog-sec mailing list > Syslog-sec@www.employees.org > http://www.employees.org/mailman/listinfo/syslog-sec >=20 >=20 >=20 _______________________________________________ Syslog-sec mailing list Syslog-sec@www.employees.org http://www.employees.org/mailman/listinfo/syslog-sec From syslog-sec-bounces@willers.employees.org Thu Oct 14 13:54:34 2004 Received: from willers.employees.org (willers.employees.org [192.83.249.36]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA11473 for ; Thu, 14 Oct 2004 13:54:33 -0400 (EDT) Received: from willers.employees.org (localhost.employees.org [127.0.0.1]) by willers.employees.org (Postfix) with ESMTP id E75575C7E0; Thu, 14 Oct 2004 10:54:33 -0700 (PDT) X-Original-To: syslog-sec@employees.org Delivered-To: syslog-sec@employees.org Received: from sj-iport-2.cisco.com (sj-iport-2-in.cisco.com [171.71.176.71]) by willers.employees.org (Postfix) with ESMTP id C5DB75C736 for ; Thu, 14 Oct 2004 10:54:24 -0700 (PDT) Received: from sj-core-2.cisco.com (171.71.177.254) by sj-iport-2.cisco.com with ESMTP; 14 Oct 2004 11:00:58 -0700 Received: from flask.cisco.com (IDENT:mirapoint@flask.cisco.com [161.44.122.62]) by sj-core-2.cisco.com (8.12.10/8.12.6) with ESMTP id i9EHsEYJ022855; Thu, 14 Oct 2004 10:54:14 -0700 (PDT) Received: from aokmiansw2k ([161.44.64.231]) by flask.cisco.com (MOS 3.4.6-GR) with ESMTP id AMH23075; Thu, 14 Oct 2004 13:54:14 -0400 (EDT) Message-Id: <200410141754.AMH23075@flask.cisco.com> From: "Anton Okmianski" To: "'Rainer Gerhards'" , , Subject: RE: [Syslog-sec] RE: Maximum message size Date: Thu, 14 Oct 2004 13:54:14 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook, Build 11.0.5510 Thread-Index: AcSwoe8xQ3+YxqsUQ8iCTkpgGQfmqgAYjRlAAAuIk6AAIvA8sAAJc8ugAAU2HwAAAS1K8AAF2flA X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4939.300 In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA03019D@grfint2.intern.adiscon.com> X-BeenThere: syslog-sec@www.employees.org X-Mailman-Version: 2.1.4 Precedence: list List-Id: syslog-sec.www.employees.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: syslog-sec-bounces@willers.employees.org Errors-To: syslog-sec-bounces@willers.employees.org Content-Transfer-Encoding: 7bit Rainer: The ~480 byte limit was on the size of packet, not datagram or message which can be fragmented into multiple packets. Besides, for IPv6 it was ~1160. In order to avoid requiring people to implement "proprietary" transport fragmentation we did say that implementations must only support messages of size which does not require this fragmentation. But we don't do that fragmentation anymore and rely on IP fragmentation. If the intention of this MUST requirement is a baseline interoperability and we know that existing applications may be sending messages as large as 1K, then it seems to me the minimum required support should be at least that big. The transport may provide its own recommendations. For example, udp transport will *recommend* that messages do not exceed the smallest MTU size on the network, which could be smaller or larger, but that will be just a recommendation. For TCP, it is not as big of an issue, for example, because it handles loss of segments. I don't think such transport layer requirements should be in the protocol which describes the content of the messages. Cheers, Anton. > -----Original Message----- > From: Rainer Gerhards [mailto:rgerhards@hq.adiscon.com] > Sent: Thursday, October 14, 2004 11:00 AM > To: Anton Okmianski; ietfdbh@comcast.net; syslog-sec@employees.org > Subject: RE: [Syslog-sec] RE: Maximum message size > > Hello Anton: > > the 480 octet limit stems back to your work and proposal in > -transport-udp. I thought you found out that more than 480 > bytes can not be transmitted "reliably" (whatever this means for UDP). > > Of course, I can remove this and we could simply state that > senders send data and receivers receive data and that the > receiver MAY receive any size or not. But wouldn't it make > sense to have one size that I can count on to be received? > > The intension of the 480 octets was to provide a known > message size that every implementation actually MUST support > - so if an applications intends to send a message that MUST > be received fully, that application knows a maximum size. > > If you mean that with the upcoming version of -transport-udp > the 480 octet can safely be replaced by 1K or 2K then we of > course can change this. > > I, however, think that a minimal burden should be placed on > an implementation. I would tend to call a syslog receiver > that is only capable of receiving 400 octets to be > non-compliant. But, OK, maybe we should say this is OK. So > what limit is it then? Is 200 octets compliant? 100? 50? > Whatever it is, I think we should require one size... If we > do not require this, one could argue writing/parsing a > correct header is also too much of a burden for a given scenario... > > Rainer > > > -----Original Message----- > > From: Anton Okmianski [mailto:aokmians@cisco.com] > > Sent: Thursday, October 14, 2004 4:36 PM > > To: ietfdbh@comcast.net; Rainer Gerhards; syslog-sec@employees.org > > Subject: RE: [Syslog-sec] RE: Maximum message size > > > > Rainer: > > > > I agree with a SHOULD on 64k support on receivers and MAY on larger > > sizes. > > > > I think we need to remove or change this sentence: "A > receiver MUST be > > able to accept messages up to and including 480 octets in length". > > > > 1. Is this an absolute requirement? We keep thinking about general > > purpose receiver libraries. But it does not have to be general > > purpose. If I have two specific applications with their own syslog > > sender/receiver implementations (how hard is it to > send/receive a udp > > message) and they exchange syslog messages which are known not to > > exceed 400 bytes, should my receiver be required to support 8k > > messages? > > > > 2. Why 480? If this is intended to ensure basic interoperability, > > then at least this should be a size greater than 1K (size previous > > syslog RFC observed). Maybe 4k or 8k. At least it is a better > > arbitrary number. > > > > So, I would support either removing the MUST requirement > all together > > or at least changing it to a more reasonable size. The > latter approach > > would be more limiting for implementations, but will insure > > interoperability for general-purpose syslog libraries if > this is the > > overriding goal. > > > > Thanks, > > Anton. > > > > > -----Original Message----- > > > From: syslog-sec-bounces@willers.employees.org > > > [mailto:syslog-sec-bounces@willers.employees.org] On > Behalf Of David > > > B Harrington > > > Sent: Thursday, October 14, 2004 8:53 AM > > > To: 'Rainer Gerhards'; syslog-sec@employees.org > > > Subject: RE: [Syslog-sec] RE: Maximum message size > > > > > > Hi Rainer, > > > > > > Thanks for making the changes. > > > > > > I'd like to make one point a little more strenuously: > > > "For interoperability reasons, all receiver > implementations SHOULD > > > be able to accept messages up to and including 65,535 octets in > > > length." > > > > > > Also, I think the last two sentences could be combined into one - > > > "If a receiver receives a message with a length larger > than 65,535 > > > octets, or larger than it supports, the receiver MAY discard the > > > message or truncate the payload." > > > > > > dbh > > > > > > -----Original Message----- > > > From: syslog-sec-bounces@www.employees.org > > > [mailto:syslog-sec-bounces@www.employees.org] On Behalf Of Rainer > > > Gerhards > > > Sent: Thursday, October 14, 2004 7:34 AM > > > To: syslog-sec@employees.org > > > Subject: [Syslog-sec] RE: Maximum message size > > > > > > David and all, > > > > > > many thanks for all the good comments. > > > > > > I agree that 16 MB for a single message is far more than actually > > > should be used. Let's look a bit at the history of this: > > > > > > Initially, we noticed that the 1024 octet limit for > current syslog > > > is too short. I think there is still general agreement on > this. We > > > had a good discussion about the size issue in September > 2003. A link > > > to an archive is here: > > > > > > http://www.syslog.cc/ietf/autoarc/msg00855.html > > > > > > The general issue is that 1K is too short for many > applications. For > > > example, Marshall Glen mentions the healthcare needs > (RFC3881) for a > > > larger message. As far as I know, the IHE initiative currently > > > standardizes on a maximum XML stream size of 32K (which is then > > > supposed to be transmitted via syslog). I think a message > size limit > > > of 64K would probably be suffcient for a long time to come (and > > > leaves some headroom for message content increases). > > > > > > Besides that, I myself identified a need - in very rare > occasions - > > > to send information larger than 64K. In extreme cases, this > > > information could be as large as one megabyte. > > > Again, these were cases that might happen once in a year. All of > > > this, of course, only if an operator > > > *really* wants to submit that large information. So the initial > > > approach and suggestion was to introduce a way to build "message > > > groups". That would have been a way to split "oversize > messages" (as > > > they were called > > > then) into multiple syslog messages. They key to that concept was > > > that each single message could be kept within a > reasonable limit (it > > > would have worked even with 1K max message size). It > would only have > > > been a way to transmit very large messages in those very seldom > > > cases that they would actually be used. > > > > > > Some time later, we came to the conclusion that it should be > > > possible for a single syslog message to be of the largest > imaginable > > > size. It was argued that fragmentation is a transport > issue and as > > > such application-level "fragmentation" > > > (the message groups) would not do good. We agreed to that. > > > > > > Even some time later, we needed to define a maximum message size, > > > just to limit it somewhere (if I recall correctly, that was a > > > transport requirement). We picked 16MB just as a number that we > > > thought would never have be reached and such be > sufficient for all > > > imaginable time to come. As far as I was concerend, that decision > > > was driven by the fact that in the past there were often > limits that > > > nobody expected to reach - and yet they wer quickly reached. > > > Som 16MB was picked more or less as "larger than anybody > will need > > > anytime in the forseeable future". We did not even have a big > > > discussion on that limit. > > > > > > I think this are the key points on how we reached this limit. > > > Now, thanks to your call and looking the whole situation, > I really > > > think we've gone overboard. That 16MB limit is something nobody > > > intends to use. Even a lower - but high limit - is extremely > > > unlikely. I'd expect to see a 1MB "message" in maybe > 0.0001% of the > > > cases - if at all (but I expect to see e.g. 70KB > "message" in maybe > > > 0.001% of the cases). > > > > > > Also, I have read your proposal to set a maxium size for the > > > message, but allow an implementation to go beyond that. Maybe > > > something like > > > this: > > > > > > #### > > > 5.1 Message Length > > > > > > A receiver MUST be able to accept messages up to and including > > > 480 octets in length. > > > > > > A receiver SHOULD be able to accept messages up to and > including > > > 65,535 octets in length. > > > If a receiver receives a message with a length larger than it > > > supports, the receiver MAY > > > discard the message or truncate the payload. > > > > > > A receiver MAY accept messages larger than 65,535 octets. > > > #### > > > > > > I've dropped the wording that transports MAY support only > a smaller > > > max size - I think UDP was the most critical in this > regard and it > > > obviously handles 64K. So I assume this clause is not > really needed > > > if we go for 64K as the max size. > > > > > > With that clause, we would limit the size for all interop > cases, but > > > leave a window open so that the message length could be > extended if > > > there is a real need AND the operator and software vendor wants > > > this. > > > I assume nobody will implement this burden if there is no market > > > need to do so. But if it is, it clould at least be > implemented in a > > > standards-compliant way. > > > > > > If I look back at the original requirement - provide a way to > > > transmit very rare oversize messages, I think we can safely > > > implement this the bounds of the current proposals, even > if the size > > > limit does not go beyond 64K. If somebody needs it, this can be > > > easily done via an experimental (or > > > vendor-specific) SD-ID. Again, I do not think somebody > will accept > > > this burden without market need. > > > > > > In the light of this, I, too, would recommend that we limit the > > > message size to 64K. I would prefer, however, that we keep the > > > maximum open as in the suggested wording above. > > > > > > Rainer > > > > > > > -----Original Message----- > > > > From: David B Harrington [mailto:ietfdbh@comcast.net] > > > > Sent: Wednesday, October 13, 2004 5:10 PM > > > > To: Rainer Gerhards; 'Anton Okmianski'; syslog-sec@employees.org > > > > Subject: Maximum message size > > > > > > > > Hi, > > > > > > > > I am concerned that we are in danger of promoting uses of syslog > > > that > > > > will defeat its current usefulness as a human-readable log of > > system > > > > > > > activity. > > > > > > > > Do network operators really believe the 16,777,216 > octet size is a > > > > > > needed improvement to syslog, or are we designing this size in > > > > explicitly for the vendors of applications who want to > use syslog > > as > > > a > > > > programmatic interface? I believe Syslog should be designed to > > meet > > > > the needs of operators. Most of the discussion ssems to be from > > > > vendors of syslog receivers or applications. Other > protocols such > > as > > > > > > > FTP might be better used for such a specialty use case. > > > > > > > > Does allowing messages of 16,777,216 octets to be accumulated > > within > > > > > > > the system log make it harder to use some > widely-available minimal > > > > > > text editors, like Notepad, to view the system log? Will having > > huge > > > > > > > application-specific messages in the system log make it > harder for > > > an > > > > operator to locate useful troubleshooting messages within the > > system > > > > > > > log? Can the information in such a huge message fit within an > > 80x25 > > > > screen, when an operator is troubleshooting network problems and > > > needs > > > > to use a bare-bones terminal attached to the serial port of a > > device > > > > > > > to view the system log? > > > > > > > > Syslog was designing to be an operator's tool, and Syslog should > > be > > > > designed to meet the needs of operators. Will allowing messages > > of > > > > this size negatively impact its usefulness in the primary use > > case? > > > > > > > > What do network **operators** think about the need for > messages of > > > > > > this size? Have we deliberately asked them their opinion on this > > by, > > > > > > > for example, taking a survey or going to NANOG to ask them? > > > > > > > > I don't think we should support such large messages in > the system > > > > logging protocol. > > > > > > > > > > > > David Harrington > > > > dbharrington@comcast.net > > > > > > > > > > > > > > > > -----Original Message----- > > > > From: syslog-sec-bounces@www.employees.org > > > > [mailto:syslog-sec-bounces@www.employees.org] On Behalf > Of Rainer > > > > Gerhards > > > > Sent: Wednesday, October 13, 2004 5:17 AM > > > > To: Anton Okmianski; syslog-sec@employees.org > > > > Subject: RE: [Syslog-sec] UDP message size issue proposal > > > > > > > > Anton, > > > > > > > > I agree to your conclusion. > > > > > > > > Although I am the one who always voted for larger sizes, the > > > > discussion has shown that this yields to unnecessary > complexity. I > > > can > > > > satisfy my needs with a vendor-extension structured > data element, > > > > which I think shows the flexibility of the new drafts. > > > > > > > > So I support your move. I would tend to even remove the > transport > > > > version header. If there is need to, we could always > include that > > in > > > a > > > > later release (e.g. "v1", which would make it clearly > > distingshable > > > > from the current frame format). I see little need to do so, > > though. > > > > > > > > Regarding -protocol, I think we should still keep an upper limit > > in > > > > the spec. Even I don't see any reason why a syslog > message should > > go > > > > > > > above 16MB. So for -protocol, I propose the following new text: > > > > > > > > #### > > > > 5.1 Message Length > > > > > > > > The maximum length of any syslog message is > 16,777,216 octets. > > > Any > > > > receiver receiving a larger message MUST discard the message. > > A > > > > diagnostic message SHOULD be logged in this case. > > > > > > > > A receiver MUST be able to accept messages that are > 480 octets > > > (or > > > > less) in length. A receiver SHOULD be able to > accept messages > > > that > > > > are 65,535 octets (or less) in length. It is > RECOMMENDED that > > > > receivers be able to accept messages up to the > maximum message > > > > length > > > > of 16,777,216 octets. > > > > > > > > If a receiver receives a message within the maximum > length, but > > > > > > with > > > > a length larger than it handles, the receiver MAY discard or > > > > truncate > > > > it. > > > > > > > > A transport mapping MAY define a maximum length below the one > > > > specified here. Each transport mapping MUST support > a maximum > > > size > > > > of 480 octets or more. > > > > #### > > > > > > > > If there is agreement on this, I can post a new version of > > > -protocol. > > > > That version will also include some changes made thanks > to Andrew > > > Ross > > > > comments (sent via private mail). I have finished this > version. It > > > is > > > > available for immediate posting, but I would like to wait for > > > > agreement on the size issue (at least if that can be reached > > > quickly). > > > > > > > > Rainer > > > > > > > > > -----Original Message----- > > > > > From: syslog-sec-bounces@www.employees.org > > > > > [mailto:syslog-sec-bounces@www.employees.org] On > Behalf Of Anton > > > > > > > Okmianski > > > > > Sent: Tuesday, October 12, 2004 11:25 PM > > > > > To: syslog-sec@employees.org > > > > > Subject: [Syslog-sec] UDP message size issue proposal > > > > > > > > > > Hi! > > > > > > > > > > Sorry for a long delay on this issue - I was on a 2 week > > vacation. > > > > I > > > > > have spoken with a number of TCP/UDP/IP experts regarding the > > > sizing > > > > > > > > > issue. I am ready to propose the following changes: > > > > > > > > > > 1. Syslog-protocol will state that the max message will be > > defined > > > > by > > > > > the transport layer. > > > > > > > > > > 2. Syslog-transport-udp will support messages up to > maximum UDP > > > > > datagram size of 64K. UDP is a very bad choice for > large message > > > > > > > transmissions, so it does not make sense for us to > stretch it by > > > > > > > adding our own fragmentation without other > transmission control > > > > > features such as found in TCP. > > > > > > > > > > 3. Syslog-transport-udp will rely on IP fragmentation and we > > will > > > > get > > > > > rid of "proprietary" fragmentation which was designed > to handle > > > > > messages over 64K and deal with various > > > > > non-compliant network hosts. > > > > > > > > > > 4. Syslog-transport-udp will recommend sending messages within > > the > > > > > > > > boundaries of prevalent MTU size on a given network > to be safe. > > It > > > > > > > > will recommend Ethernet's 1500 bytes less headers and > will draw > > > > reader > > > > > attention to the minimum MTU size hosts on the network are > > > required > > > > to > > > > > support for > > > > > IPv6 (576 bytes) and IPv6 (1280 bytes). > > > > > > > > > > 5. Path MTU discovery may not work robustly and some TCP/IP > > stacks > > > > may > > > > > not support UDP packets of full 64K size and truncate > them. We > > > will > > > > > > > > > mention this and bite this bullet. > > > > > We should not restrict the protocol due to incompliant > > > > implementations > > > > > because it is a moving target and penalizes compliant > > > > implementations > > > > > with extra overhead. The above size recommendation would > > partially > > > > > > > > deal with this, but leave the > > > > > final choice up to the administrator. > > > > > > > > > > 6. We will get rid of most syslog transport headers for UDP as > > > they > > > > > will no longer be needed. The only thing that will be left is > > the > > > > > > > > transport protocol version prefixed to every syslog message. > > > Should > > > > > > > > > we even bother with that? > > > > > > > > > > This is a major change to the syslog-transport-udp. > I'd like to > > > get > > > > > > > > > positive feedback before I proceed with this update. > > > > > The best part is that if we all agree on the above, the next > > draft > > > > > > > > will be 1/3 of the size -- easier read for you. :) > > > > > > > > > > Thanks, > > > > > Anton. > > > > > > > > > > > > > > > _______________________________________________ > > > > > Syslog-sec mailing list > > > > > Syslog-sec@www.employees.org > > > > > http://www.employees.org/mailman/listinfo/syslog-sec > > > > > > > > > _______________________________________________ > > > > Syslog-sec mailing list > > > > Syslog-sec@www.employees.org > > > > http://www.employees.org/mailman/listinfo/syslog-sec > > > > > > > > > > > > > > > _______________________________________________ > > > Syslog-sec mailing list > > > Syslog-sec@www.employees.org > > > http://www.employees.org/mailman/listinfo/syslog-sec > > > > > > > > > _______________________________________________ > > > Syslog-sec mailing list > > > Syslog-sec@www.employees.org > > > http://www.employees.org/mailman/listinfo/syslog-sec > > > > > > > > _______________________________________________ Syslog-sec mailing list Syslog-sec@www.employees.org http://www.employees.org/mailman/listinfo/syslog-sec From syslog-sec-bounces@willers.employees.org Mon Oct 18 08:58:57 2004 Received: from willers.employees.org (willers.employees.org [192.83.249.36]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id IAA13111 for ; Mon, 18 Oct 2004 08:58:56 -0400 (EDT) Received: from willers.employees.org (localhost.employees.org [127.0.0.1]) by willers.employees.org (Postfix) with ESMTP id D10D65C725; Mon, 18 Oct 2004 05:58:51 -0700 (PDT) X-Original-To: syslog-sec@employees.org Delivered-To: syslog-sec@employees.org Received: from mail.hq.adiscon.com (mail.hq.adiscon.com [217.6.190.188]) by willers.employees.org (Postfix) with ESMTP id 5DCF15C72E for ; Mon, 18 Oct 2004 01:54:56 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by mail.hq.adiscon.com (Postfix) with ESMTP id 9A3EC9C757; Mon, 18 Oct 2004 11:12:16 +0200 (CEST) Received: from mail.hq.adiscon.com ([127.0.0.1]) by localhost (grfdeb [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 22021-09; Mon, 18 Oct 2004 11:11:52 +0200 (CEST) Received: from grfint2.intern.adiscon.com (grfint2 [172.19.0.6]) by mail.hq.adiscon.com (Postfix) with ESMTP id 9A43F9C755; Mon, 18 Oct 2004 11:11:52 +0200 (CEST) Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: [Syslog-sec] RE: Maximum message size X-MimeOLE: Produced By Microsoft Exchange V6.5.7226.0 Date: Mon, 18 Oct 2004 10:58:19 +0200 Message-ID: <577465F99B41C842AAFBE9ED71E70ABA0301C7@grfint2.intern.adiscon.com> Thread-Topic: [Syslog-sec] RE: Maximum message size Thread-Index: AcSwoe8xQ3+YxqsUQ8iCTkpgGQfmqgAYjRlAAAuIk6AAIvA8sAAJc8ugAAU2HwAAAS1K8AAF2flAALansiA= From: "Rainer Gerhards" To: "Anton Okmianski" , , X-Virus-Scanned: by amavisd-new-20030616-p7 (Debian) at adiscon.com X-Mailman-Approved-At: Mon, 18 Oct 2004 05:58:51 -0700 X-BeenThere: syslog-sec@www.employees.org X-Mailman-Version: 2.1.4 Precedence: list List-Id: syslog-sec.www.employees.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: syslog-sec-bounces@willers.employees.org Errors-To: syslog-sec-bounces@willers.employees.org Content-Transfer-Encoding: quoted-printable Anton: I understand the source of the 480 byte limit you outlined in -transport. Even though we have now decided that we can use UDP fragementation, I still think the fact that 480 is the only safe number still persists. So I do not see any value in requiring an application to support messages of up to 1K when we know this might not work. I agree, it looks like it has worked (at least most of the time) so far, but why introduce a problem if there is no need to? Maybe we had no issues in the past because messages were most often below 480 octets. Also, I do not think that we can say the minimum to be supported size is 1K in protocol and then allow a transport mapping to specify something lower. If it is 1K, then it MUST be 1K in all cases. Else I can not see how we will be able to ensure interoperability. I am of the strong opinion we need to know a maximum size that is guaranteed to be received. I do not see the value in raising the minimum to be supported size to 1K. Maybe I am overlooking the obvious, but I think we deliberatley introduce problems if we go above that. And, yes, I am refering to IPv4 only, because I am looking at the weakest transport (which, in this case, is also the most commonly used). Rainer > -----Original Message----- > From: Anton Okmianski [mailto:aokmians@cisco.com]=20 > Sent: Thursday, October 14, 2004 7:54 PM > To: Rainer Gerhards; ietfdbh@comcast.net; syslog-sec@employees.org > Subject: RE: [Syslog-sec] RE: Maximum message size=20 >=20 > Rainer: >=20 > The ~480 byte limit was on the size of packet, not datagram or message > which can be fragmented into multiple packets. Besides, for IPv6 it > was ~1160. In order to avoid requiring people to implement > "proprietary" transport fragmentation we did say that implementations > must only support messages of size which does not require this > fragmentation. But we don't do that fragmentation anymore and rely on > IP fragmentation. =20 >=20 > If the intention of this MUST requirement is a baseline > interoperability and we know that existing applications may be sending > messages as large as 1K, then it seems to me the minimum required > support should be at least that big. =20 >=20 > The transport may provide its own recommendations. For example, udp > transport will *recommend* that messages do not exceed the smallest > MTU size on the network, which could be smaller or larger, but that > will be just a recommendation. For TCP, it is not as big of an issue, > for example, because it handles loss of segments. I don't think such > transport layer requirements should be in the protocol which describes > the content of the messages. =20 >=20 > Cheers, >=20 > Anton.=20 >=20 > > -----Original Message----- > > From: Rainer Gerhards [mailto:rgerhards@hq.adiscon.com]=20 > > Sent: Thursday, October 14, 2004 11:00 AM > > To: Anton Okmianski; ietfdbh@comcast.net; syslog-sec@employees.org > > Subject: RE: [Syslog-sec] RE: Maximum message size=20 > >=20 > > Hello Anton: > >=20 > > the 480 octet limit stems back to your work and proposal in=20 > > -transport-udp. I thought you found out that more than 480=20 > > bytes can not be transmitted "reliably" (whatever this means for > UDP). > >=20 > > Of course, I can remove this and we could simply state that=20 > > senders send data and receivers receive data and that the=20 > > receiver MAY receive any size or not. But wouldn't it make=20 > > sense to have one size that I can count on to be received? > >=20 > > The intension of the 480 octets was to provide a known=20 > > message size that every implementation actually MUST support=20 > > - so if an applications intends to send a message that MUST=20 > > be received fully, that application knows a maximum size. > >=20 > > If you mean that with the upcoming version of -transport-udp=20 > > the 480 octet can safely be replaced by 1K or 2K then we of=20 > > course can change this. > >=20 > > I, however, think that a minimal burden should be placed on=20 > > an implementation. I would tend to call a syslog receiver=20 > > that is only capable of receiving 400 octets to be=20 > > non-compliant. But, OK, maybe we should say this is OK. So=20 > > what limit is it then? Is 200 octets compliant? 100? 50?=20 > > Whatever it is, I think we should require one size... If we=20 > > do not require this, one could argue writing/parsing a=20 > > correct header is also too much of a burden for a given scenario... > >=20 > > Rainer > >=20 > > > -----Original Message----- > > > From: Anton Okmianski [mailto:aokmians@cisco.com] > > > Sent: Thursday, October 14, 2004 4:36 PM > > > To: ietfdbh@comcast.net; Rainer Gerhards; syslog-sec@employees.org > > > Subject: RE: [Syslog-sec] RE: Maximum message size > > >=20 > > > Rainer: > > >=20 > > > I agree with a SHOULD on 64k support on receivers and MAY on > larger > > > sizes. =20 > > >=20 > > > I think we need to remove or change this sentence: "A=20 > > receiver MUST be=20 > > > able to accept messages up to and including 480 octets in length". > > >=20 > > > 1. Is this an absolute requirement? We keep thinking about > general=20 > > > purpose receiver libraries. But it does not have to be general=20 > > > purpose. If I have two specific applications with their own > syslog=20 > > > sender/receiver implementations (how hard is it to=20 > > send/receive a udp > > > message) and they exchange syslog messages which are known not to=20 > > > exceed 400 bytes, should my receiver be required to support 8k=20 > > > messages? > > >=20 > > > 2. Why 480? If this is intended to ensure basic interoperability, >=20 > > > then at least this should be a size greater than 1K (size previous >=20 > > > syslog RFC observed). Maybe 4k or 8k. At least it is a better=20 > > > arbitrary number. > > >=20 > > > So, I would support either removing the MUST requirement=20 > > all together=20 > > > or at least changing it to a more reasonable size. The=20 > > latter approach=20 > > > would be more limiting for implementations, but will insure=20 > > > interoperability for general-purpose syslog libraries if=20 > > this is the=20 > > > overriding goal. > > >=20 > > > Thanks, > > > Anton.=20 > > >=20 > > > > -----Original Message----- > > > > From: syslog-sec-bounces@willers.employees.org > > > > [mailto:syslog-sec-bounces@willers.employees.org] On=20 > > Behalf Of David=20 > > > > B Harrington > > > > Sent: Thursday, October 14, 2004 8:53 AM > > > > To: 'Rainer Gerhards'; syslog-sec@employees.org > > > > Subject: RE: [Syslog-sec] RE: Maximum message size > > > >=20 > > > > Hi Rainer, > > > >=20 > > > > Thanks for making the changes. > > > >=20 > > > > I'd like to make one point a little more strenuously: > > > > "For interoperability reasons, all receiver=20 > > implementations SHOULD=20 > > > > be able to accept messages up to and including 65,535 octets in=20 > > > > length." > > > >=20 > > > > Also, I think the last two sentences could be combined into one > -=20 > > > > "If a receiver receives a message with a length larger=20 > > than 65,535=20 > > > > octets, or larger than it supports, the receiver MAY discard the >=20 > > > > message or truncate the payload." > > > >=20 > > > > dbh > > > >=20 > > > > -----Original Message----- > > > > From: syslog-sec-bounces@www.employees.org > > > > [mailto:syslog-sec-bounces@www.employees.org] On Behalf Of > Rainer=20 > > > > Gerhards > > > > Sent: Thursday, October 14, 2004 7:34 AM > > > > To: syslog-sec@employees.org > > > > Subject: [Syslog-sec] RE: Maximum message size > > > >=20 > > > > David and all, > > > >=20 > > > > many thanks for all the good comments. > > > >=20 > > > > I agree that 16 MB for a single message is far more than > actually=20 > > > > should be used. Let's look a bit at the history of this: > > > >=20 > > > > Initially, we noticed that the 1024 octet limit for=20 > > current syslog=20 > > > > is too short. I think there is still general agreement on=20 > > this. We=20 > > > > had a good discussion about the size issue in September=20 > > 2003. A link=20 > > > > to an archive is here: > > > >=20 > > > > http://www.syslog.cc/ietf/autoarc/msg00855.html > > > >=20 > > > > The general issue is that 1K is too short for many=20 > > applications. For=20 > > > > example, Marshall Glen mentions the healthcare needs=20 > > (RFC3881) for a=20 > > > > larger message. As far as I know, the IHE initiative currently=20 > > > > standardizes on a maximum XML stream size of 32K (which is then=20 > > > > supposed to be transmitted via syslog). I think a message=20 > > size limit=20 > > > > of 64K would probably be suffcient for a long time to come (and=20 > > > > leaves some headroom for message content increases). > > > >=20 > > > > Besides that, I myself identified a need - in very rare=20 > > occasions -=20 > > > > to send information larger than 64K. In extreme cases, this=20 > > > > information could be as large as one megabyte. > > > > Again, these were cases that might happen once in a year. All of >=20 > > > > this, of course, only if an operator > > > > *really* wants to submit that large information. So the initial=20 > > > > approach and suggestion was to introduce a way to build "message >=20 > > > > groups". That would have been a way to split "oversize=20 > > messages" (as=20 > > > > they were called > > > > then) into multiple syslog messages. They key to that concept > was=20 > > > > that each single message could be kept within a=20 > > reasonable limit (it=20 > > > > would have worked even with 1K max message size). It=20 > > would only have=20 > > > > been a way to transmit very large messages in those very seldom=20 > > > > cases that they would actually be used. > > > >=20 > > > > Some time later, we came to the conclusion that it should be=20 > > > > possible for a single syslog message to be of the largest=20 > > imaginable=20 > > > > size. It was argued that fragmentation is a transport=20 > > issue and as=20 > > > > such application-level "fragmentation" > > > > (the message groups) would not do good. We agreed to that. > > > >=20 > > > > Even some time later, we needed to define a maximum message > size,=20 > > > > just to limit it somewhere (if I recall correctly, that was a=20 > > > > transport requirement). We picked 16MB just as a number that we=20 > > > > thought would never have be reached and such be=20 > > sufficient for all=20 > > > > imaginable time to come. As far as I was concerend, that > decision=20 > > > > was driven by the fact that in the past there were often=20 > > limits that=20 > > > > nobody expected to reach - and yet they wer quickly reached. > > > > Som 16MB was picked more or less as "larger than anybody=20 > > will need=20 > > > > anytime in the forseeable future". We did not even have a big=20 > > > > discussion on that limit. > > > >=20 > > > > I think this are the key points on how we reached this limit.=20 > > > > Now, thanks to your call and looking the whole situation,=20 > > I really=20 > > > > think we've gone overboard. That 16MB limit is something nobody=20 > > > > intends to use. Even a lower - but high limit - is extremely=20 > > > > unlikely. I'd expect to see a 1MB "message" in maybe=20 > > 0.0001% of the=20 > > > > cases - if at all (but I expect to see e.g. 70KB=20 > > "message" in maybe=20 > > > > 0.001% of the cases). > > > >=20 > > > > Also, I have read your proposal to set a maxium size for the=20 > > > > message, but allow an implementation to go beyond that. Maybe=20 > > > > something like > > > > this: > > > >=20 > > > > #### > > > > 5.1 Message Length > > > >=20 > > > > A receiver MUST be able to accept messages up to and > including=20 > > > > 480 octets in length. > > > >=20 > > > > A receiver SHOULD be able to accept messages up to and=20 > > including > > > > 65,535 octets in length.=20 > > > > If a receiver receives a message with a length larger than it >=20 > > > > supports, the receiver MAY > > > > discard the message or truncate the payload. > > > >=20 > > > > A receiver MAY accept messages larger than 65,535 octets. > > > > #### > > > >=20 > > > > I've dropped the wording that transports MAY support only=20 > > a smaller=20 > > > > max size - I think UDP was the most critical in this=20 > > regard and it=20 > > > > obviously handles 64K. So I assume this clause is not=20 > > really needed=20 > > > > if we go for 64K as the max size. > > > >=20 > > > > With that clause, we would limit the size for all interop=20 > > cases, but=20 > > > > leave a window open so that the message length could be=20 > > extended if=20 > > > > there is a real need AND the operator and software vendor wants=20 > > > > this. > > > > I assume nobody will implement this burden if there is no market >=20 > > > > need to do so. But if it is, it clould at least be=20 > > implemented in a=20 > > > > standards-compliant way. > > > >=20 > > > > If I look back at the original requirement - provide a way to=20 > > > > transmit very rare oversize messages, I think we can safely=20 > > > > implement this the bounds of the current proposals, even=20 > > if the size=20 > > > > limit does not go beyond 64K. If somebody needs it, this can be=20 > > > > easily done via an experimental (or > > > > vendor-specific) SD-ID. Again, I do not think somebody=20 > > will accept=20 > > > > this burden without market need. > > > >=20 > > > > In the light of this, I, too, would recommend that we limit the=20 > > > > message size to 64K. I would prefer, however, that we keep the=20 > > > > maximum open as in the suggested wording above. > > > >=20 > > > > Rainer > > > >=20 > > > > > -----Original Message----- > > > > > From: David B Harrington [mailto:ietfdbh@comcast.net] > > > > > Sent: Wednesday, October 13, 2004 5:10 PM > > > > > To: Rainer Gerhards; 'Anton Okmianski'; > syslog-sec@employees.org > > > > > Subject: Maximum message size > > > > >=20 > > > > > Hi, > > > > >=20 > > > > > I am concerned that we are in danger of promoting uses of > syslog > > > > that > > > > > will defeat its current usefulness as a human-readable log of > > > system > > > >=20 > > > > > activity. > > > > >=20 > > > > > Do network operators really believe the 16,777,216=20 > > octet size is a > > >=20 > > > > > needed improvement to syslog, or are we designing this size in >=20 > > > > > explicitly for the vendors of applications who want to=20 > > use syslog > > > as > > > > a > > > > > programmatic interface? I believe Syslog should be designed to > > > meet > > > > > the needs of operators. Most of the discussion ssems to be > from=20 > > > > > vendors of syslog receivers or applications. Other=20 > > protocols such > > > as > > > >=20 > > > > > FTP might be better used for such a specialty use case. > > > > >=20 > > > > > Does allowing messages of 16,777,216 octets to be accumulated > > > within > > > >=20 > > > > > the system log make it harder to use some=20 > > widely-available minimal > > >=20 > > > > > text editors, like Notepad, to view the system log? Will > having > > > huge > > > >=20 > > > > > application-specific messages in the system log make it=20 > > harder for > > > > an > > > > > operator to locate useful troubleshooting messages within the > > > system > > > >=20 > > > > > log? Can the information in such a huge message fit within an > > > 80x25 > > > > > screen, when an operator is troubleshooting network problems > and > > > > needs > > > > > to use a bare-bones terminal attached to the serial port of a > > > device > > > >=20 > > > > > to view the system log? > > > > >=20 > > > > > Syslog was designing to be an operator's tool, and Syslog > should > > > be > > > > > designed to meet the needs of operators. Will allowing > messages > > > of > > > > > this size negatively impact its usefulness in the primary use > > > case? > > > > >=20 > > > > > What do network **operators** think about the need for=20 > > messages of > > >=20 > > > > > this size? Have we deliberately asked them their opinion on > this > > > by, > > > >=20 > > > > > for example, taking a survey or going to NANOG to ask them? > > > > >=20 > > > > > I don't think we should support such large messages in=20 > > the system=20 > > > > > logging protocol. > > > > >=20 > > > > >=20 > > > > > David Harrington > > > > > dbharrington@comcast.net > > > > >=20 > > > > >=20 > > > > >=20 > > > > > -----Original Message----- > > > > > From: syslog-sec-bounces@www.employees.org > > > > > [mailto:syslog-sec-bounces@www.employees.org] On Behalf=20 > > Of Rainer=20 > > > > > Gerhards > > > > > Sent: Wednesday, October 13, 2004 5:17 AM > > > > > To: Anton Okmianski; syslog-sec@employees.org > > > > > Subject: RE: [Syslog-sec] UDP message size issue proposal > > > > >=20 > > > > > Anton, > > > > >=20 > > > > > I agree to your conclusion. > > > > >=20 > > > > > Although I am the one who always voted for larger sizes, the=20 > > > > > discussion has shown that this yields to unnecessary=20 > > complexity. I > > > > can > > > > > satisfy my needs with a vendor-extension structured=20 > > data element,=20 > > > > > which I think shows the flexibility of the new drafts. > > > > >=20 > > > > > So I support your move. I would tend to even remove the=20 > > transport=20 > > > > > version header. If there is need to, we could always=20 > > include that > > > in > > > > a > > > > > later release (e.g. "v1", which would make it clearly > > > distingshable > > > > > from the current frame format). I see little need to do so, > > > though. > > > > >=20 > > > > > Regarding -protocol, I think we should still keep an upper > limit > > > in > > > > > the spec. Even I don't see any reason why a syslog=20 > > message should > > > go > > > >=20 > > > > > above 16MB. So for -protocol, I propose the following new > text: > > > > >=20 > > > > > #### > > > > > 5.1 Message Length > > > > >=20 > > > > > The maximum length of any syslog message is=20 > > 16,777,216 octets. > > > > Any > > > > > receiver receiving a larger message MUST discard the > message. > > > A > > > > > diagnostic message SHOULD be logged in this case. > > > > >=20 > > > > > A receiver MUST be able to accept messages that are=20 > > 480 octets > > > > (or > > > > > less) in length. A receiver SHOULD be able to=20 > > accept messages > > > > that > > > > > are 65,535 octets (or less) in length. It is=20 > > RECOMMENDED that > > > > > receivers be able to accept messages up to the=20 > > maximum message=20 > > > > > length > > > > > of 16,777,216 octets. > > > > >=20 > > > > > If a receiver receives a message within the maximum=20 > > length, but > > >=20 > > > > > with > > > > > a length larger than it handles, the receiver MAY discard > or=20 > > > > > truncate > > > > > it. > > > > >=20 > > > > > A transport mapping MAY define a maximum length below the > one > > > > > specified here. Each transport mapping MUST support=20 > > a maximum > > > > size > > > > > of 480 octets or more. > > > > > #### > > > > >=20 > > > > > If there is agreement on this, I can post a new version of > > > > -protocol. > > > > > That version will also include some changes made thanks=20 > > to Andrew > > > > Ross > > > > > comments (sent via private mail). I have finished this=20 > > version. It > > > > is > > > > > available for immediate posting, but I would like to wait for=20 > > > > > agreement on the size issue (at least if that can be reached > > > > quickly). > > > > >=20 > > > > > Rainer > > > > >=20 > > > > > > -----Original Message----- > > > > > > From: syslog-sec-bounces@www.employees.org > > > > > > [mailto:syslog-sec-bounces@www.employees.org] On=20 > > Behalf Of Anton > > >=20 > > > > > > Okmianski > > > > > > Sent: Tuesday, October 12, 2004 11:25 PM > > > > > > To: syslog-sec@employees.org > > > > > > Subject: [Syslog-sec] UDP message size issue proposal > > > > > >=20 > > > > > > Hi! > > > > > >=20 > > > > > > Sorry for a long delay on this issue - I was on a 2 week > > > vacation. > > > > > I > > > > > > have spoken with a number of TCP/UDP/IP experts regarding > the > > > > sizing > > > > >=20 > > > > > > issue. I am ready to propose the following changes: > > > > > >=20 > > > > > > 1. Syslog-protocol will state that the max message will be > > > defined > > > > > by > > > > > > the transport layer. > > > > > >=20 > > > > > > 2. Syslog-transport-udp will support messages up to=20 > > maximum UDP=20 > > > > > > datagram size of 64K. UDP is a very bad choice for=20 > > large message > > >=20 > > > > > > transmissions, so it does not make sense for us to=20 > > stretch it by > > >=20 > > > > > > adding our own fragmentation without other=20 > > transmission control=20 > > > > > > features such as found in TCP. > > > > > >=20 > > > > > > 3. Syslog-transport-udp will rely on IP fragmentation and we > > > will > > > > > get > > > > > > rid of "proprietary" fragmentation which was designed=20 > > to handle=20 > > > > > > messages over 64K and deal with various > > > > > > non-compliant network hosts. =20 > > > > > >=20 > > > > > > 4. Syslog-transport-udp will recommend sending messages > within > > > the > > > >=20 > > > > > > boundaries of prevalent MTU size on a given network=20 > > to be safe. > > > It > > > >=20 > > > > > > will recommend Ethernet's 1500 bytes less headers and=20 > > will draw > > > > > reader > > > > > > attention to the minimum MTU size hosts on the network are > > > > required > > > > > to > > > > > > support for > > > > > > IPv6 (576 bytes) and IPv6 (1280 bytes). =20 > > > > > >=20 > > > > > > 5. Path MTU discovery may not work robustly and some TCP/IP > > > stacks > > > > > may > > > > > > not support UDP packets of full 64K size and truncate=20 > > them. We > > > > will > > > > >=20 > > > > > > mention this and bite this bullet. > > > > > > We should not restrict the protocol due to incompliant > > > > > implementations > > > > > > because it is a moving target and penalizes compliant > > > > > implementations > > > > > > with extra overhead. The above size recommendation would > > > partially > > > >=20 > > > > > > deal with this, but leave the > > > > > > final choice up to the administrator. =20 > > > > > >=20 > > > > > > 6. We will get rid of most syslog transport headers for UDP > as > > > > they > > > > > > will no longer be needed. The only thing that will be left > is > > > the > > > >=20 > > > > > > transport protocol version prefixed to every syslog message. > > > > Should > > > > >=20 > > > > > > we even bother with that? > > > > > >=20 > > > > > > This is a major change to the syslog-transport-udp. =20 > > I'd like to > > > > get > > > > >=20 > > > > > > positive feedback before I proceed with this update. > > > > > > The best part is that if we all agree on the above, the next > > > draft > > > >=20 > > > > > > will be 1/3 of the size -- easier read for you. :) > > > > > >=20 > > > > > > Thanks, > > > > > > Anton.=20 > > > > > > =20 > > > > > >=20 > > > > > > _______________________________________________ > > > > > > Syslog-sec mailing list > > > > > > Syslog-sec@www.employees.org > > > > > > http://www.employees.org/mailman/listinfo/syslog-sec > > > > > >=20 > > > > > _______________________________________________ > > > > > Syslog-sec mailing list > > > > > Syslog-sec@www.employees.org > > > > > http://www.employees.org/mailman/listinfo/syslog-sec > > > > >=20 > > > > >=20 > > > > >=20 > > > > _______________________________________________ > > > > Syslog-sec mailing list > > > > Syslog-sec@www.employees.org > > > > http://www.employees.org/mailman/listinfo/syslog-sec > > > >=20 > > > >=20 > > > > _______________________________________________ > > > > Syslog-sec mailing list > > > > Syslog-sec@www.employees.org > > > > http://www.employees.org/mailman/listinfo/syslog-sec > > > >=20 > > >=20 > > >=20 > >=20 >=20 >=20 _______________________________________________ Syslog-sec mailing list Syslog-sec@www.employees.org http://www.employees.org/mailman/listinfo/syslog-sec From syslog-sec-bounces@willers.employees.org Thu Oct 21 08:45:46 2004 Received: from willers.employees.org (willers.employees.org [192.83.249.36]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id IAA19191 for ; Thu, 21 Oct 2004 08:45:45 -0400 (EDT) Received: from willers.employees.org (localhost.employees.org [127.0.0.1]) by willers.employees.org (Postfix) with ESMTP id 7D6A45C72A; Thu, 21 Oct 2004 05:45:44 -0700 (PDT) X-Original-To: syslog-sec@employees.org Delivered-To: syslog-sec@employees.org Received: from sj-iport-1.cisco.com (sj-iport-1-in.cisco.com [171.71.176.70]) by willers.employees.org (Postfix) with ESMTP id 8195A5C730 for ; Thu, 21 Oct 2004 05:43:49 -0700 (PDT) Received: from sj-core-1.cisco.com (171.71.177.237) by sj-iport-1.cisco.com with ESMTP; 21 Oct 2004 05:54:08 -0700 X-BrightmailFiltered: true Received: from edison.cisco.com (edison.cisco.com [171.71.180.109]) by sj-core-1.cisco.com (8.12.10/8.12.6) with ESMTP id i9LChgk2022908 for ; Thu, 21 Oct 2004 05:43:42 -0700 (PDT) Received: from localhost (clonvick@localhost) by edison.cisco.com (8.8.6 (PHNE_14041)/CISCO.SERVER.1.2) with ESMTP id FAA04863 for ; Thu, 21 Oct 2004 05:43:47 -0700 (PDT) Date: Thu, 21 Oct 2004 05:43:47 -0700 (PDT) From: Chris Lonvick To: syslog-sec@employees.org Subject: RE: [Syslog-sec] RE: Maximum message size In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA0301C7@grfint2.intern.adiscon.com> Message-ID: References: <577465F99B41C842AAFBE9ED71E70ABA0301C7@grfint2.intern.adiscon.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Mailman-Approved-At: Thu, 21 Oct 2004 05:45:43 -0700 X-BeenThere: syslog-sec@www.employees.org X-Mailman-Version: 2.1.4 Precedence: list List-Id: syslog-sec.www.employees.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: syslog-sec-bounces@willers.employees.org Errors-To: syslog-sec-bounces@willers.employees.org Hi All, Do we have rough consensus on this? We can still request a slot at the upcoming IETF to discuss this. Thanks, Chris On Mon, 18 Oct 2004, Rainer Gerhards wrote: > Anton: > > I understand the source of the 480 byte limit you outlined in > -transport. Even though we have now decided that we can use UDP > fragementation, I still think the fact that 480 is the only safe number > still persists. > > So I do not see any value in requiring an application to support > messages of up to 1K when we know this might not work. I agree, it looks > like it has worked (at least most of the time) so far, but why introduce > a problem if there is no need to? Maybe we had no issues in the past > because messages were most often below 480 octets. > > Also, I do not think that we can say the minimum to be supported size is > 1K in protocol and then allow a transport mapping to specify something > lower. If it is 1K, then it MUST be 1K in all cases. Else I can not see > how we will be able to ensure interoperability. I am of the strong > opinion we need to know a maximum size that is guaranteed to be > received. > > I do not see the value in raising the minimum to be supported size to > 1K. Maybe I am overlooking the obvious, but I think we deliberatley > introduce problems if we go above that. > > And, yes, I am refering to IPv4 only, because I am looking at the > weakest transport (which, in this case, is also the most commonly used). > > Rainer > > > -----Original Message----- > > From: Anton Okmianski [mailto:aokmians@cisco.com] > > Sent: Thursday, October 14, 2004 7:54 PM > > To: Rainer Gerhards; ietfdbh@comcast.net; syslog-sec@employees.org > > Subject: RE: [Syslog-sec] RE: Maximum message size > > > > Rainer: > > > > The ~480 byte limit was on the size of packet, not datagram or message > > which can be fragmented into multiple packets. Besides, for IPv6 it > > was ~1160. In order to avoid requiring people to implement > > "proprietary" transport fragmentation we did say that implementations > > must only support messages of size which does not require this > > fragmentation. But we don't do that fragmentation anymore and rely on > > IP fragmentation. > > > > If the intention of this MUST requirement is a baseline > > interoperability and we know that existing applications may be sending > > messages as large as 1K, then it seems to me the minimum required > > support should be at least that big. > > > > The transport may provide its own recommendations. For example, udp > > transport will *recommend* that messages do not exceed the smallest > > MTU size on the network, which could be smaller or larger, but that > > will be just a recommendation. For TCP, it is not as big of an issue, > > for example, because it handles loss of segments. I don't think such > > transport layer requirements should be in the protocol which describes > > the content of the messages. > > > > Cheers, > > > > Anton. > > > > > -----Original Message----- > > > From: Rainer Gerhards [mailto:rgerhards@hq.adiscon.com] > > > Sent: Thursday, October 14, 2004 11:00 AM > > > To: Anton Okmianski; ietfdbh@comcast.net; syslog-sec@employees.org > > > Subject: RE: [Syslog-sec] RE: Maximum message size > > > > > > Hello Anton: > > > > > > the 480 octet limit stems back to your work and proposal in > > > -transport-udp. I thought you found out that more than 480 > > > bytes can not be transmitted "reliably" (whatever this means for > > UDP). > > > > > > Of course, I can remove this and we could simply state that > > > senders send data and receivers receive data and that the > > > receiver MAY receive any size or not. But wouldn't it make > > > sense to have one size that I can count on to be received? > > > > > > The intension of the 480 octets was to provide a known > > > message size that every implementation actually MUST support > > > - so if an applications intends to send a message that MUST > > > be received fully, that application knows a maximum size. > > > > > > If you mean that with the upcoming version of -transport-udp > > > the 480 octet can safely be replaced by 1K or 2K then we of > > > course can change this. > > > > > > I, however, think that a minimal burden should be placed on > > > an implementation. I would tend to call a syslog receiver > > > that is only capable of receiving 400 octets to be > > > non-compliant. But, OK, maybe we should say this is OK. So > > > what limit is it then? Is 200 octets compliant? 100? 50? > > > Whatever it is, I think we should require one size... If we > > > do not require this, one could argue writing/parsing a > > > correct header is also too much of a burden for a given scenario... > > > > > > Rainer > > > > > > > -----Original Message----- > > > > From: Anton Okmianski [mailto:aokmians@cisco.com] > > > > Sent: Thursday, October 14, 2004 4:36 PM > > > > To: ietfdbh@comcast.net; Rainer Gerhards; syslog-sec@employees.org > > > > Subject: RE: [Syslog-sec] RE: Maximum message size > > > > > > > > Rainer: > > > > > > > > I agree with a SHOULD on 64k support on receivers and MAY on > > larger > > > > sizes. > > > > > > > > I think we need to remove or change this sentence: "A > > > receiver MUST be > > > > able to accept messages up to and including 480 octets in length". > > > > > > > > 1. Is this an absolute requirement? We keep thinking about > > general > > > > purpose receiver libraries. But it does not have to be general > > > > purpose. If I have two specific applications with their own > > syslog > > > > sender/receiver implementations (how hard is it to > > > send/receive a udp > > > > message) and they exchange syslog messages which are known not to > > > > exceed 400 bytes, should my receiver be required to support 8k > > > > messages? > > > > > > > > 2. Why 480? If this is intended to ensure basic interoperability, > > > > > > then at least this should be a size greater than 1K (size previous > > > > > > syslog RFC observed). Maybe 4k or 8k. At least it is a better > > > > arbitrary number. > > > > > > > > So, I would support either removing the MUST requirement > > > all together > > > > or at least changing it to a more reasonable size. The > > > latter approach > > > > would be more limiting for implementations, but will insure > > > > interoperability for general-purpose syslog libraries if > > > this is the > > > > overriding goal. > > > > > > > > Thanks, > > > > Anton. > > > > > > > > > -----Original Message----- > > > > > From: syslog-sec-bounces@willers.employees.org > > > > > [mailto:syslog-sec-bounces@willers.employees.org] On > > > Behalf Of David > > > > > B Harrington > > > > > Sent: Thursday, October 14, 2004 8:53 AM > > > > > To: 'Rainer Gerhards'; syslog-sec@employees.org > > > > > Subject: RE: [Syslog-sec] RE: Maximum message size > > > > > > > > > > Hi Rainer, > > > > > > > > > > Thanks for making the changes. > > > > > > > > > > I'd like to make one point a little more strenuously: > > > > > "For interoperability reasons, all receiver > > > implementations SHOULD > > > > > be able to accept messages up to and including 65,535 octets in > > > > > length." > > > > > > > > > > Also, I think the last two sentences could be combined into one > > - > > > > > "If a receiver receives a message with a length larger > > > than 65,535 > > > > > octets, or larger than it supports, the receiver MAY discard the > > > > > > > message or truncate the payload." > > > > > > > > > > dbh > > > > > > > > > > -----Original Message----- > > > > > From: syslog-sec-bounces@www.employees.org > > > > > [mailto:syslog-sec-bounces@www.employees.org] On Behalf Of > > Rainer > > > > > Gerhards > > > > > Sent: Thursday, October 14, 2004 7:34 AM > > > > > To: syslog-sec@employees.org > > > > > Subject: [Syslog-sec] RE: Maximum message size > > > > > > > > > > David and all, > > > > > > > > > > many thanks for all the good comments. > > > > > > > > > > I agree that 16 MB for a single message is far more than > > actually > > > > > should be used. Let's look a bit at the history of this: > > > > > > > > > > Initially, we noticed that the 1024 octet limit for > > > current syslog > > > > > is too short. I think there is still general agreement on > > > this. We > > > > > had a good discussion about the size issue in September > > > 2003. A link > > > > > to an archive is here: > > > > > > > > > > http://www.syslog.cc/ietf/autoarc/msg00855.html > > > > > > > > > > The general issue is that 1K is too short for many > > > applications. For > > > > > example, Marshall Glen mentions the healthcare needs > > > (RFC3881) for a > > > > > larger message. As far as I know, the IHE initiative currently > > > > > standardizes on a maximum XML stream size of 32K (which is then > > > > > supposed to be transmitted via syslog). I think a message > > > size limit > > > > > of 64K would probably be suffcient for a long time to come (and > > > > > leaves some headroom for message content increases). > > > > > > > > > > Besides that, I myself identified a need - in very rare > > > occasions - > > > > > to send information larger than 64K. In extreme cases, this > > > > > information could be as large as one megabyte. > > > > > Again, these were cases that might happen once in a year. All of > > > > > > > this, of course, only if an operator > > > > > *really* wants to submit that large information. So the initial > > > > > approach and suggestion was to introduce a way to build "message > > > > > > > groups". That would have been a way to split "oversize > > > messages" (as > > > > > they were called > > > > > then) into multiple syslog messages. They key to that concept > > was > > > > > that each single message could be kept within a > > > reasonable limit (it > > > > > would have worked even with 1K max message size). It > > > would only have > > > > > been a way to transmit very large messages in those very seldom > > > > > cases that they would actually be used. > > > > > > > > > > Some time later, we came to the conclusion that it should be > > > > > possible for a single syslog message to be of the largest > > > imaginable > > > > > size. It was argued that fragmentation is a transport > > > issue and as > > > > > such application-level "fragmentation" > > > > > (the message groups) would not do good. We agreed to that. > > > > > > > > > > Even some time later, we needed to define a maximum message > > size, > > > > > just to limit it somewhere (if I recall correctly, that was a > > > > > transport requirement). We picked 16MB just as a number that we > > > > > thought would never have be reached and such be > > > sufficient for all > > > > > imaginable time to come. As far as I was concerend, that > > decision > > > > > was driven by the fact that in the past there were often > > > limits that > > > > > nobody expected to reach - and yet they wer quickly reached. > > > > > Som 16MB was picked more or less as "larger than anybody > > > will need > > > > > anytime in the forseeable future". We did not even have a big > > > > > discussion on that limit. > > > > > > > > > > I think this are the key points on how we reached this limit. > > > > > Now, thanks to your call and looking the whole situation, > > > I really > > > > > think we've gone overboard. That 16MB limit is something nobody > > > > > intends to use. Even a lower - but high limit - is extremely > > > > > unlikely. I'd expect to see a 1MB "message" in maybe > > > 0.0001% of the > > > > > cases - if at all (but I expect to see e.g. 70KB > > > "message" in maybe > > > > > 0.001% of the cases). > > > > > > > > > > Also, I have read your proposal to set a maxium size for the > > > > > message, but allow an implementation to go beyond that. Maybe > > > > > something like > > > > > this: > > > > > > > > > > #### > > > > > 5.1 Message Length > > > > > > > > > > A receiver MUST be able to accept messages up to and > > including > > > > > 480 octets in length. > > > > > > > > > > A receiver SHOULD be able to accept messages up to and > > > including > > > > > 65,535 octets in length. > > > > > If a receiver receives a message with a length larger than it > > > > > > > supports, the receiver MAY > > > > > discard the message or truncate the payload. > > > > > > > > > > A receiver MAY accept messages larger than 65,535 octets. > > > > > #### > > > > > > > > > > I've dropped the wording that transports MAY support only > > > a smaller > > > > > max size - I think UDP was the most critical in this > > > regard and it > > > > > obviously handles 64K. So I assume this clause is not > > > really needed > > > > > if we go for 64K as the max size. > > > > > > > > > > With that clause, we would limit the size for all interop > > > cases, but > > > > > leave a window open so that the message length could be > > > extended if > > > > > there is a real need AND the operator and software vendor wants > > > > > this. > > > > > I assume nobody will implement this burden if there is no market > > > > > > > need to do so. But if it is, it clould at least be > > > implemented in a > > > > > standards-compliant way. > > > > > > > > > > If I look back at the original requirement - provide a way to > > > > > transmit very rare oversize messages, I think we can safely > > > > > implement this the bounds of the current proposals, even > > > if the size > > > > > limit does not go beyond 64K. If somebody needs it, this can be > > > > > easily done via an experimental (or > > > > > vendor-specific) SD-ID. Again, I do not think somebody > > > will accept > > > > > this burden without market need. > > > > > > > > > > In the light of this, I, too, would recommend that we limit the > > > > > message size to 64K. I would prefer, however, that we keep the > > > > > maximum open as in the suggested wording above. > > > > > > > > > > Rainer > > > > > > > > > > > -----Original Message----- > > > > > > From: David B Harrington [mailto:ietfdbh@comcast.net] > > > > > > Sent: Wednesday, October 13, 2004 5:10 PM > > > > > > To: Rainer Gerhards; 'Anton Okmianski'; > > syslog-sec@employees.org > > > > > > Subject: Maximum message size > > > > > > > > > > > > Hi, > > > > > > > > > > > > I am concerned that we are in danger of promoting uses of > > syslog > > > > > that > > > > > > will defeat its current usefulness as a human-readable log of > > > > system > > > > > > > > > > > activity. > > > > > > > > > > > > Do network operators really believe the 16,777,216 > > > octet size is a > > > > > > > > > > needed improvement to syslog, or are we designing this size in > > > > > > > > explicitly for the vendors of applications who want to > > > use syslog > > > > as > > > > > a > > > > > > programmatic interface? I believe Syslog should be designed to > > > > meet > > > > > > the needs of operators. Most of the discussion ssems to be > > from > > > > > > vendors of syslog receivers or applications. Other > > > protocols such > > > > as > > > > > > > > > > > FTP might be better used for such a specialty use case. > > > > > > > > > > > > Does allowing messages of 16,777,216 octets to be accumulated > > > > within > > > > > > > > > > > the system log make it harder to use some > > > widely-available minimal > > > > > > > > > > text editors, like Notepad, to view the system log? Will > > having > > > > huge > > > > > > > > > > > application-specific messages in the system log make it > > > harder for > > > > > an > > > > > > operator to locate useful troubleshooting messages within the > > > > system > > > > > > > > > > > log? Can the information in such a huge message fit within an > > > > 80x25 > > > > > > screen, when an operator is troubleshooting network problems > > and > > > > > needs > > > > > > to use a bare-bones terminal attached to the serial port of a > > > > device > > > > > > > > > > > to view the system log? > > > > > > > > > > > > Syslog was designing to be an operator's tool, and Syslog > > should > > > > be > > > > > > designed to meet the needs of operators. Will allowing > > messages > > > > of > > > > > > this size negatively impact its usefulness in the primary use > > > > case? > > > > > > > > > > > > What do network **operators** think about the need for > > > messages of > > > > > > > > > > this size? Have we deliberately asked them their opinion on > > this > > > > by, > > > > > > > > > > > for example, taking a survey or going to NANOG to ask them? > > > > > > > > > > > > I don't think we should support such large messages in > > > the system > > > > > > logging protocol. > > > > > > > > > > > > > > > > > > David Harrington > > > > > > dbharrington@comcast.net > > > > > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > From: syslog-sec-bounces@www.employees.org > > > > > > [mailto:syslog-sec-bounces@www.employees.org] On Behalf > > > Of Rainer > > > > > > Gerhards > > > > > > Sent: Wednesday, October 13, 2004 5:17 AM > > > > > > To: Anton Okmianski; syslog-sec@employees.org > > > > > > Subject: RE: [Syslog-sec] UDP message size issue proposal > > > > > > > > > > > > Anton, > > > > > > > > > > > > I agree to your conclusion. > > > > > > > > > > > > Although I am the one who always voted for larger sizes, the > > > > > > discussion has shown that this yields to unnecessary > > > complexity. I > > > > > can > > > > > > satisfy my needs with a vendor-extension structured > > > data element, > > > > > > which I think shows the flexibility of the new drafts. > > > > > > > > > > > > So I support your move. I would tend to even remove the > > > transport > > > > > > version header. If there is need to, we could always > > > include that > > > > in > > > > > a > > > > > > later release (e.g. "v1", which would make it clearly > > > > distingshable > > > > > > from the current frame format). I see little need to do so, > > > > though. > > > > > > > > > > > > Regarding -protocol, I think we should still keep an upper > > limit > > > > in > > > > > > the spec. Even I don't see any reason why a syslog > > > message should > > > > go > > > > > > > > > > > above 16MB. So for -protocol, I propose the following new > > text: > > > > > > > > > > > > #### > > > > > > 5.1 Message Length > > > > > > > > > > > > The maximum length of any syslog message is > > > 16,777,216 octets. > > > > > Any > > > > > > receiver receiving a larger message MUST discard the > > message. > > > > A > > > > > > diagnostic message SHOULD be logged in this case. > > > > > > > > > > > > A receiver MUST be able to accept messages that are > > > 480 octets > > > > > (or > > > > > > less) in length. A receiver SHOULD be able to > > > accept messages > > > > > that > > > > > > are 65,535 octets (or less) in length. It is > > > RECOMMENDED that > > > > > > receivers be able to accept messages up to the > > > maximum message > > > > > > length > > > > > > of 16,777,216 octets. > > > > > > > > > > > > If a receiver receives a message within the maximum > > > length, but > > > > > > > > > > with > > > > > > a length larger than it handles, the receiver MAY discard > > or > > > > > > truncate > > > > > > it. > > > > > > > > > > > > A transport mapping MAY define a maximum length below the > > one > > > > > > specified here. Each transport mapping MUST support > > > a maximum > > > > > size > > > > > > of 480 octets or more. > > > > > > #### > > > > > > > > > > > > If there is agreement on this, I can post a new version of > > > > > -protocol. > > > > > > That version will also include some changes made thanks > > > to Andrew > > > > > Ross > > > > > > comments (sent via private mail). I have finished this > > > version. It > > > > > is > > > > > > available for immediate posting, but I would like to wait for > > > > > > agreement on the size issue (at least if that can be reached > > > > > quickly). > > > > > > > > > > > > Rainer > > > > > > > > > > > > > -----Original Message----- > > > > > > > From: syslog-sec-bounces@www.employees.org > > > > > > > [mailto:syslog-sec-bounces@www.employees.org] On > > > Behalf Of Anton > > > > > > > > > > > Okmianski > > > > > > > Sent: Tuesday, October 12, 2004 11:25 PM > > > > > > > To: syslog-sec@employees.org > > > > > > > Subject: [Syslog-sec] UDP message size issue proposal > > > > > > > > > > > > > > Hi! > > > > > > > > > > > > > > Sorry for a long delay on this issue - I was on a 2 week > > > > vacation. > > > > > > I > > > > > > > have spoken with a number of TCP/UDP/IP experts regarding > > the > > > > > sizing > > > > > > > > > > > > > issue. I am ready to propose the following changes: > > > > > > > > > > > > > > 1. Syslog-protocol will state that the max message will be > > > > defined > > > > > > by > > > > > > > the transport layer. > > > > > > > > > > > > > > 2. Syslog-transport-udp will support messages up to > > > maximum UDP > > > > > > > datagram size of 64K. UDP is a very bad choice for > > > large message > > > > > > > > > > > transmissions, so it does not make sense for us to > > > stretch it by > > > > > > > > > > > adding our own fragmentation without other > > > transmission control > > > > > > > features such as found in TCP. > > > > > > > > > > > > > > 3. Syslog-transport-udp will rely on IP fragmentation and we > > > > will > > > > > > get > > > > > > > rid of "proprietary" fragmentation which was designed > > > to handle > > > > > > > messages over 64K and deal with various > > > > > > > non-compliant network hosts. > > > > > > > > > > > > > > 4. Syslog-transport-udp will recommend sending messages > > within > > > > the > > > > > > > > > > > > boundaries of prevalent MTU size on a given network > > > to be safe. > > > > It > > > > > > > > > > > > will recommend Ethernet's 1500 bytes less headers and > > > will draw > > > > > > reader > > > > > > > attention to the minimum MTU size hosts on the network are > > > > > required > > > > > > to > > > > > > > support for > > > > > > > IPv6 (576 bytes) and IPv6 (1280 bytes). > > > > > > > > > > > > > > 5. Path MTU discovery may not work robustly and some TCP/IP > > > > stacks > > > > > > may > > > > > > > not support UDP packets of full 64K size and truncate > > > them. We > > > > > will > > > > > > > > > > > > > mention this and bite this bullet. > > > > > > > We should not restrict the protocol due to incompliant > > > > > > implementations > > > > > > > because it is a moving target and penalizes compliant > > > > > > implementations > > > > > > > with extra overhead. The above size recommendation would > > > > partially > > > > > > > > > > > > deal with this, but leave the > > > > > > > final choice up to the administrator. > > > > > > > > > > > > > > 6. We will get rid of most syslog transport headers for UDP > > as > > > > > they > > > > > > > will no longer be needed. The only thing that will be left > > is > > > > the > > > > > > > > > > > > transport protocol version prefixed to every syslog message. > > > > > Should > > > > > > > > > > > > > we even bother with that? > > > > > > > > > > > > > > This is a major change to the syslog-transport-udp. > > > I'd like to > > > > > get > > > > > > > > > > > > > positive feedback before I proceed with this update. > > > > > > > The best part is that if we all agree on the above, the next > > > > draft > > > > > > > > > > > > will be 1/3 of the size -- easier read for you. :) > > > > > > > > > > > > > > Thanks, > > > > > > > Anton. > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > > > > > > Syslog-sec mailing list > > > > > > > Syslog-sec@www.employees.org > > > > > > > http://www.employees.org/mailman/listinfo/syslog-sec > > > > > > > > > > > > > _______________________________________________ > > > > > > Syslog-sec mailing list > > > > > > Syslog-sec@www.employees.org > > > > > > http://www.employees.org/mailman/listinfo/syslog-sec > > > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > > > > Syslog-sec mailing list > > > > > Syslog-sec@www.employees.org > > > > > http://www.employees.org/mailman/listinfo/syslog-sec > > > > > > > > > > > > > > > _______________________________________________ > > > > > Syslog-sec mailing list > > > > > Syslog-sec@www.employees.org > > > > > http://www.employees.org/mailman/listinfo/syslog-sec > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > Syslog-sec mailing list > Syslog-sec@www.employees.org > http://www.employees.org/mailman/listinfo/syslog-sec > _______________________________________________ Syslog-sec mailing list Syslog-sec@www.employees.org http://www.employees.org/mailman/listinfo/syslog-sec From syslog-sec-bounces@willers.employees.org Thu Oct 21 10:40:19 2004 Received: from willers.employees.org (willers.employees.org [192.83.249.36]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA07673 for ; Thu, 21 Oct 2004 10:40:18 -0400 (EDT) Received: from willers.employees.org (localhost.employees.org [127.0.0.1]) by willers.employees.org (Postfix) with ESMTP id 8AB195C735; Thu, 21 Oct 2004 07:40:19 -0700 (PDT) X-Original-To: syslog-sec@employees.org Delivered-To: syslog-sec@employees.org Received: from rtp-iport-2.cisco.com (rtp-iport-2.cisco.com [64.102.122.149]) by willers.employees.org (Postfix) with ESMTP id DE7305C796 for ; Thu, 21 Oct 2004 07:24:24 -0700 (PDT) Received: from rtp-core-1.cisco.com (64.102.124.12) by rtp-iport-2.cisco.com with ESMTP; 21 Oct 2004 10:24:24 -0400 X-BrightmailFiltered: true Received: from flask.cisco.com (IDENT:mirapoint@flask.cisco.com [161.44.122.62]) by rtp-core-1.cisco.com (8.12.10/8.12.6) with ESMTP id i9LEOKxT004114; Thu, 21 Oct 2004 10:24:21 -0400 (EDT) Received: from aokmiansw2k ([161.44.64.231]) by flask.cisco.com (MOS 3.4.6-GR) with ESMTP id AML52489; Thu, 21 Oct 2004 10:24:20 -0400 (EDT) Message-Id: <200410211424.AML52489@flask.cisco.com> From: "Anton Okmianski" To: "'Chris Lonvick'" , Subject: RE: [Syslog-sec] RE: Maximum message size Date: Thu, 21 Oct 2004 10:24:19 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook, Build 11.0.5510 Thread-Index: AcS3a+90AcdCGH0sRdSOacXEmHTX1QADDIcQ In-Reply-To: X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4939.300 X-Mailman-Approved-At: Thu, 21 Oct 2004 07:40:18 -0700 X-BeenThere: syslog-sec@www.employees.org X-Mailman-Version: 2.1.4 Precedence: list List-Id: syslog-sec.www.employees.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: syslog-sec-bounces@willers.employees.org Errors-To: syslog-sec-bounces@willers.employees.org Content-Transfer-Encoding: 7bit Rainer, Chris, all: My vote is for -protocol to say that any syslog *transport* MUST support transferring messages of at least 1Kb (or 2 Kb) and recommend that they support 64Kb. Udp-transport will, in turn, recommend that applications don't send message with sizes which exceed the MTU on a given network to avoid fragmentation and provide Ethernet MTU (less headers) as a guide. So, if somebody' network has the old MTUs of 576 bytes, we would still handle it, they will just need to set-up their TCP-IP stack with appropriate MTU (assuming MTU Discovery does not work) and there will be fragmentation for messages greater than this size. We could also say that *receivers* SHOULD support messages of at least 1Kb (or 2Kb). I would not set any MUST requirements on the receivers. Note the difference between requirements on receivers and transport protocol. Rainer, please get the consensus on this on the list and let's move on one way or the other. I will be happy with whatever the majority consensus is. Thanks, Anton. > -----Original Message----- > From: syslog-sec-bounces@willers.employees.org > [mailto:syslog-sec-bounces@willers.employees.org] On Behalf > Of Chris Lonvick > Sent: Thursday, October 21, 2004 8:44 AM > To: syslog-sec@employees.org > Subject: RE: [Syslog-sec] RE: Maximum message size > > Hi All, > > Do we have rough consensus on this? We can still request a > slot at the upcoming IETF to discuss this. > > Thanks, > Chris > > On Mon, 18 Oct 2004, Rainer Gerhards wrote: > > > Anton: > > > > I understand the source of the 480 byte limit you outlined in > > -transport. Even though we have now decided that we can use UDP > > fragementation, I still think the fact that 480 is the only safe > > number still persists. > > > > So I do not see any value in requiring an application to support > > messages of up to 1K when we know this might not work. I agree, it > > looks like it has worked (at least most of the time) so > far, but why > > introduce a problem if there is no need to? Maybe we had no > issues in > > the past because messages were most often below 480 octets. > > > > Also, I do not think that we can say the minimum to be > supported size > > is 1K in protocol and then allow a transport mapping to specify > > something lower. If it is 1K, then it MUST be 1K in all > cases. Else I > > can not see how we will be able to ensure interoperability. I am of > > the strong opinion we need to know a maximum size that is > guaranteed > > to be received. > > > > I do not see the value in raising the minimum to be > supported size to > > 1K. Maybe I am overlooking the obvious, but I think we deliberatley > > introduce problems if we go above that. > > > > And, yes, I am refering to IPv4 only, because I am looking at the > > weakest transport (which, in this case, is also the most > commonly used). > > > > Rainer > > > > > -----Original Message----- > > > From: Anton Okmianski [mailto:aokmians@cisco.com] > > > Sent: Thursday, October 14, 2004 7:54 PM > > > To: Rainer Gerhards; ietfdbh@comcast.net; syslog-sec@employees.org > > > Subject: RE: [Syslog-sec] RE: Maximum message size > > > > > > Rainer: > > > > > > The ~480 byte limit was on the size of packet, not datagram or > > > message which can be fragmented into multiple packets. > Besides, for > > > IPv6 it was ~1160. In order to avoid requiring people to > implement > > > "proprietary" transport fragmentation we did say that > > > implementations must only support messages of size which does not > > > require this fragmentation. But we don't do that fragmentation > > > anymore and rely on IP fragmentation. > > > > > > If the intention of this MUST requirement is a baseline > > > interoperability and we know that existing applications may be > > > sending messages as large as 1K, then it seems to me the minimum > > > required support should be at least that big. > > > > > > The transport may provide its own recommendations. For > example, udp > > > transport will *recommend* that messages do not exceed > the smallest > > > MTU size on the network, which could be smaller or > larger, but that > > > will be just a recommendation. For TCP, it is not as big of an > > > issue, for example, because it handles loss of segments. I don't > > > think such transport layer requirements should be in the protocol > > > which describes the content of the messages. > > > > > > Cheers, > > > > > > Anton. > > > > > > > -----Original Message----- > > > > From: Rainer Gerhards [mailto:rgerhards@hq.adiscon.com] > > > > Sent: Thursday, October 14, 2004 11:00 AM > > > > To: Anton Okmianski; ietfdbh@comcast.net; > syslog-sec@employees.org > > > > Subject: RE: [Syslog-sec] RE: Maximum message size > > > > > > > > Hello Anton: > > > > > > > > the 480 octet limit stems back to your work and proposal in > > > > -transport-udp. I thought you found out that more than > 480 bytes > > > > can not be transmitted "reliably" (whatever this means for > > > UDP). > > > > > > > > Of course, I can remove this and we could simply state that > > > > senders send data and receivers receive data and that > the receiver > > > > MAY receive any size or not. But wouldn't it make sense to have > > > > one size that I can count on to be received? > > > > > > > > The intension of the 480 octets was to provide a known message > > > > size that every implementation actually MUST support > > > > - so if an applications intends to send a message that MUST be > > > > received fully, that application knows a maximum size. > > > > > > > > If you mean that with the upcoming version of > -transport-udp the > > > > 480 octet can safely be replaced by 1K or 2K then we of > course can > > > > change this. > > > > > > > > I, however, think that a minimal burden should be placed on an > > > > implementation. I would tend to call a syslog receiver that is > > > > only capable of receiving 400 octets to be > non-compliant. But, OK, > > > > maybe we should say this is OK. So what limit is it > then? Is 200 > > > > octets compliant? 100? 50? > > > > Whatever it is, I think we should require one size... > If we do not > > > > require this, one could argue writing/parsing a correct > header is > > > > also too much of a burden for a given scenario... > > > > > > > > Rainer > > > > > > > > > -----Original Message----- > > > > > From: Anton Okmianski [mailto:aokmians@cisco.com] > > > > > Sent: Thursday, October 14, 2004 4:36 PM > > > > > To: ietfdbh@comcast.net; Rainer Gerhards; > > > > > syslog-sec@employees.org > > > > > Subject: RE: [Syslog-sec] RE: Maximum message size > > > > > > > > > > Rainer: > > > > > > > > > > I agree with a SHOULD on 64k support on receivers and MAY on > > > larger > > > > > sizes. > > > > > > > > > > I think we need to remove or change this sentence: "A > > > > receiver MUST be > > > > > able to accept messages up to and including 480 > octets in length". > > > > > > > > > > 1. Is this an absolute requirement? We keep thinking about > > > general > > > > > purpose receiver libraries. But it does not have to > be general > > > > > purpose. If I have two specific applications with their own > > > syslog > > > > > sender/receiver implementations (how hard is it to > > > > send/receive a udp > > > > > message) and they exchange syslog messages which are > known not > > > > > to exceed 400 bytes, should my receiver be required > to support > > > > > 8k messages? > > > > > > > > > > 2. Why 480? If this is intended to ensure basic > > > > > interoperability, > > > > > > > > then at least this should be a size greater than 1K (size > > > > > previous > > > > > > > > syslog RFC observed). Maybe 4k or 8k. At least it > is a better > > > > > arbitrary number. > > > > > > > > > > So, I would support either removing the MUST requirement > > > > all together > > > > > or at least changing it to a more reasonable size. The > > > > latter approach > > > > > would be more limiting for implementations, but will insure > > > > > interoperability for general-purpose syslog libraries if > > > > this is the > > > > > overriding goal. > > > > > > > > > > Thanks, > > > > > Anton. > > > > > > > > > > > -----Original Message----- > > > > > > From: syslog-sec-bounces@willers.employees.org > > > > > > [mailto:syslog-sec-bounces@willers.employees.org] On > > > > Behalf Of David > > > > > > B Harrington > > > > > > Sent: Thursday, October 14, 2004 8:53 AM > > > > > > To: 'Rainer Gerhards'; syslog-sec@employees.org > > > > > > Subject: RE: [Syslog-sec] RE: Maximum message size > > > > > > > > > > > > Hi Rainer, > > > > > > > > > > > > Thanks for making the changes. > > > > > > > > > > > > I'd like to make one point a little more strenuously: > > > > > > "For interoperability reasons, all receiver > > > > implementations SHOULD > > > > > > be able to accept messages up to and including > 65,535 octets > > > > > > in length." > > > > > > > > > > > > Also, I think the last two sentences could be combined into > > > > > > one > > > - > > > > > > "If a receiver receives a message with a length larger > > > > than 65,535 > > > > > > octets, or larger than it supports, the receiver > MAY discard > > > > > > the > > > > > > > > > message or truncate the payload." > > > > > > > > > > > > dbh > > > > > > > > > > > > -----Original Message----- > > > > > > From: syslog-sec-bounces@www.employees.org > > > > > > [mailto:syslog-sec-bounces@www.employees.org] On Behalf Of > > > Rainer > > > > > > Gerhards > > > > > > Sent: Thursday, October 14, 2004 7:34 AM > > > > > > To: syslog-sec@employees.org > > > > > > Subject: [Syslog-sec] RE: Maximum message size > > > > > > > > > > > > David and all, > > > > > > > > > > > > many thanks for all the good comments. > > > > > > > > > > > > I agree that 16 MB for a single message is far more than > > > actually > > > > > > should be used. Let's look a bit at the history of this: > > > > > > > > > > > > Initially, we noticed that the 1024 octet limit for > > > > current syslog > > > > > > is too short. I think there is still general agreement on > > > > this. We > > > > > > had a good discussion about the size issue in September > > > > 2003. A link > > > > > > to an archive is here: > > > > > > > > > > > > http://www.syslog.cc/ietf/autoarc/msg00855.html > > > > > > > > > > > > The general issue is that 1K is too short for many > > > > applications. For > > > > > > example, Marshall Glen mentions the healthcare needs > > > > (RFC3881) for a > > > > > > larger message. As far as I know, the IHE > initiative currently > > > > > > standardizes on a maximum XML stream size of 32K (which is > > > > > > then supposed to be transmitted via syslog). I > think a message > > > > size limit > > > > > > of 64K would probably be suffcient for a long time to come > > > > > > (and leaves some headroom for message content increases). > > > > > > > > > > > > Besides that, I myself identified a need - in very rare > > > > occasions - > > > > > > to send information larger than 64K. In extreme cases, this > > > > > > information could be as large as one megabyte. > > > > > > Again, these were cases that might happen once in a > year. All > > > > > > of > > > > > > > > > this, of course, only if an operator > > > > > > *really* wants to submit that large information. So the > > > > > > initial approach and suggestion was to introduce a way to > > > > > > build "message > > > > > > > > > groups". That would have been a way to split "oversize > > > > messages" (as > > > > > > they were called > > > > > > then) into multiple syslog messages. They key to > that concept > > > was > > > > > > that each single message could be kept within a > > > > reasonable limit (it > > > > > > would have worked even with 1K max message size). It > > > > would only have > > > > > > been a way to transmit very large messages in those very > > > > > > seldom cases that they would actually be used. > > > > > > > > > > > > Some time later, we came to the conclusion that it > should be > > > > > > possible for a single syslog message to be of the largest > > > > imaginable > > > > > > size. It was argued that fragmentation is a transport > > > > issue and as > > > > > > such application-level "fragmentation" > > > > > > (the message groups) would not do good. We agreed to that. > > > > > > > > > > > > Even some time later, we needed to define a maximum message > > > size, > > > > > > just to limit it somewhere (if I recall correctly, > that was a > > > > > > transport requirement). We picked 16MB just as a > number that > > > > > > we thought would never have be reached and such be > > > > sufficient for all > > > > > > imaginable time to come. As far as I was concerend, that > > > decision > > > > > > was driven by the fact that in the past there were often > > > > limits that > > > > > > nobody expected to reach - and yet they wer quickly reached. > > > > > > Som 16MB was picked more or less as "larger than anybody > > > > will need > > > > > > anytime in the forseeable future". We did not even > have a big > > > > > > discussion on that limit. > > > > > > > > > > > > I think this are the key points on how we reached > this limit. > > > > > > Now, thanks to your call and looking the whole situation, > > > > I really > > > > > > think we've gone overboard. That 16MB limit is something > > > > > > nobody intends to use. Even a lower - but high limit - is > > > > > > extremely unlikely. I'd expect to see a 1MB > "message" in maybe > > > > 0.0001% of the > > > > > > cases - if at all (but I expect to see e.g. 70KB > > > > "message" in maybe > > > > > > 0.001% of the cases). > > > > > > > > > > > > Also, I have read your proposal to set a maxium > size for the > > > > > > message, but allow an implementation to go beyond > that. Maybe > > > > > > something like > > > > > > this: > > > > > > > > > > > > #### > > > > > > 5.1 Message Length > > > > > > > > > > > > A receiver MUST be able to accept messages up to and > > > including > > > > > > 480 octets in length. > > > > > > > > > > > > A receiver SHOULD be able to accept messages up to and > > > > including > > > > > > 65,535 octets in length. > > > > > > If a receiver receives a message with a length > larger than > > > > > > it > > > > > > > > > supports, the receiver MAY > > > > > > discard the message or truncate the payload. > > > > > > > > > > > > A receiver MAY accept messages larger than 65,535 octets. > > > > > > #### > > > > > > > > > > > > I've dropped the wording that transports MAY support only > > > > a smaller > > > > > > max size - I think UDP was the most critical in this > > > > regard and it > > > > > > obviously handles 64K. So I assume this clause is not > > > > really needed > > > > > > if we go for 64K as the max size. > > > > > > > > > > > > With that clause, we would limit the size for all interop > > > > cases, but > > > > > > leave a window open so that the message length could be > > > > extended if > > > > > > there is a real need AND the operator and software vendor > > > > > > wants this. > > > > > > I assume nobody will implement this burden if there is no > > > > > > market > > > > > > > > > need to do so. But if it is, it clould at least be > > > > implemented in a > > > > > > standards-compliant way. > > > > > > > > > > > > If I look back at the original requirement - > provide a way to > > > > > > transmit very rare oversize messages, I think we can safely > > > > > > implement this the bounds of the current proposals, even > > > > if the size > > > > > > limit does not go beyond 64K. If somebody needs it, > this can > > > > > > be easily done via an experimental (or > > > > > > vendor-specific) SD-ID. Again, I do not think somebody > > > > will accept > > > > > > this burden without market need. > > > > > > > > > > > > In the light of this, I, too, would recommend that we limit > > > > > > the message size to 64K. I would prefer, however, > that we keep > > > > > > the maximum open as in the suggested wording above. > > > > > > > > > > > > Rainer > > > > > > > > > > > > > -----Original Message----- > > > > > > > From: David B Harrington [mailto:ietfdbh@comcast.net] > > > > > > > Sent: Wednesday, October 13, 2004 5:10 PM > > > > > > > To: Rainer Gerhards; 'Anton Okmianski'; > > > syslog-sec@employees.org > > > > > > > Subject: Maximum message size > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > I am concerned that we are in danger of promoting uses of > > > syslog > > > > > > that > > > > > > > will defeat its current usefulness as a > human-readable log > > > > > > > of > > > > > system > > > > > > > > > > > > > activity. > > > > > > > > > > > > > > Do network operators really believe the 16,777,216 > > > > octet size is a > > > > > > > > > > > > needed improvement to syslog, or are we designing > this size > > > > > > > in > > > > > > > > > > explicitly for the vendors of applications who want to > > > > use syslog > > > > > as > > > > > > a > > > > > > > programmatic interface? I believe Syslog should > be designed > > > > > > > to > > > > > meet > > > > > > > the needs of operators. Most of the discussion ssems to be > > > from > > > > > > > vendors of syslog receivers or applications. Other > > > > protocols such > > > > > as > > > > > > > > > > > > > FTP might be better used for such a specialty use case. > > > > > > > > > > > > > > Does allowing messages of 16,777,216 octets to be > > > > > > > accumulated > > > > > within > > > > > > > > > > > > > the system log make it harder to use some > > > > widely-available minimal > > > > > > > > > > > > text editors, like Notepad, to view the system log? Will > > > having > > > > > huge > > > > > > > > > > > > > application-specific messages in the system log make it > > > > harder for > > > > > > an > > > > > > > operator to locate useful troubleshooting messages within > > > > > > > the > > > > > system > > > > > > > > > > > > > log? Can the information in such a huge message > fit within > > > > > > > an > > > > > 80x25 > > > > > > > screen, when an operator is troubleshooting > network problems > > > and > > > > > > needs > > > > > > > to use a bare-bones terminal attached to the > serial port of > > > > > > > a > > > > > device > > > > > > > > > > > > > to view the system log? > > > > > > > > > > > > > > Syslog was designing to be an operator's tool, and Syslog > > > should > > > > > be > > > > > > > designed to meet the needs of operators. Will allowing > > > messages > > > > > of > > > > > > > this size negatively impact its usefulness in the primary > > > > > > > use > > > > > case? > > > > > > > > > > > > > > What do network **operators** think about the need for > > > > messages of > > > > > > > > > > > > this size? Have we deliberately asked them their > opinion on > > > this > > > > > by, > > > > > > > > > > > > > for example, taking a survey or going to NANOG to > ask them? > > > > > > > > > > > > > > I don't think we should support such large messages in > > > > the system > > > > > > > logging protocol. > > > > > > > > > > > > > > > > > > > > > David Harrington > > > > > > > dbharrington@comcast.net > > > > > > > > > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > From: syslog-sec-bounces@www.employees.org > > > > > > > [mailto:syslog-sec-bounces@www.employees.org] On Behalf > > > > Of Rainer > > > > > > > Gerhards > > > > > > > Sent: Wednesday, October 13, 2004 5:17 AM > > > > > > > To: Anton Okmianski; syslog-sec@employees.org > > > > > > > Subject: RE: [Syslog-sec] UDP message size issue proposal > > > > > > > > > > > > > > Anton, > > > > > > > > > > > > > > I agree to your conclusion. > > > > > > > > > > > > > > Although I am the one who always voted for larger > sizes, the > > > > > > > discussion has shown that this yields to unnecessary > > > > complexity. I > > > > > > can > > > > > > > satisfy my needs with a vendor-extension structured > > > > data element, > > > > > > > which I think shows the flexibility of the new drafts. > > > > > > > > > > > > > > So I support your move. I would tend to even remove the > > > > transport > > > > > > > version header. If there is need to, we could always > > > > include that > > > > > in > > > > > > a > > > > > > > later release (e.g. "v1", which would make it clearly > > > > > distingshable > > > > > > > from the current frame format). I see little need > to do so, > > > > > though. > > > > > > > > > > > > > > Regarding -protocol, I think we should still keep an upper > > > limit > > > > > in > > > > > > > the spec. Even I don't see any reason why a syslog > > > > message should > > > > > go > > > > > > > > > > > > > above 16MB. So for -protocol, I propose the following new > > > text: > > > > > > > > > > > > > > #### > > > > > > > 5.1 Message Length > > > > > > > > > > > > > > The maximum length of any syslog message is > > > > 16,777,216 octets. > > > > > > Any > > > > > > > receiver receiving a larger message MUST discard the > > > message. > > > > > A > > > > > > > diagnostic message SHOULD be logged in this case. > > > > > > > > > > > > > > A receiver MUST be able to accept messages that are > > > > 480 octets > > > > > > (or > > > > > > > less) in length. A receiver SHOULD be able to > > > > accept messages > > > > > > that > > > > > > > are 65,535 octets (or less) in length. It is > > > > RECOMMENDED that > > > > > > > receivers be able to accept messages up to the > > > > maximum message > > > > > > > length > > > > > > > of 16,777,216 octets. > > > > > > > > > > > > > > If a receiver receives a message within the maximum > > > > length, but > > > > > > > > > > > > with > > > > > > > a length larger than it handles, the receiver > MAY discard > > > or > > > > > > > truncate > > > > > > > it. > > > > > > > > > > > > > > A transport mapping MAY define a maximum > length below the > > > one > > > > > > > specified here. Each transport mapping MUST support > > > > a maximum > > > > > > size > > > > > > > of 480 octets or more. > > > > > > > #### > > > > > > > > > > > > > > If there is agreement on this, I can post a new version of > > > > > > -protocol. > > > > > > > That version will also include some changes made thanks > > > > to Andrew > > > > > > Ross > > > > > > > comments (sent via private mail). I have finished this > > > > version. It > > > > > > is > > > > > > > available for immediate posting, but I would like to wait > > > > > > > for agreement on the size issue (at least if that can be > > > > > > > reached > > > > > > quickly). > > > > > > > > > > > > > > Rainer > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > From: syslog-sec-bounces@www.employees.org > > > > > > > > [mailto:syslog-sec-bounces@www.employees.org] On > > > > Behalf Of Anton > > > > > > > > > > > > > Okmianski > > > > > > > > Sent: Tuesday, October 12, 2004 11:25 PM > > > > > > > > To: syslog-sec@employees.org > > > > > > > > Subject: [Syslog-sec] UDP message size issue proposal > > > > > > > > > > > > > > > > Hi! > > > > > > > > > > > > > > > > Sorry for a long delay on this issue - I was on a 2 week > > > > > vacation. > > > > > > > I > > > > > > > > have spoken with a number of TCP/UDP/IP experts > regarding > > > the > > > > > > sizing > > > > > > > > > > > > > > > issue. I am ready to propose the following changes: > > > > > > > > > > > > > > > > 1. Syslog-protocol will state that the max > message will be > > > > > defined > > > > > > > by > > > > > > > > the transport layer. > > > > > > > > > > > > > > > > 2. Syslog-transport-udp will support messages up to > > > > maximum UDP > > > > > > > > datagram size of 64K. UDP is a very bad choice for > > > > large message > > > > > > > > > > > > > transmissions, so it does not make sense for us to > > > > stretch it by > > > > > > > > > > > > > adding our own fragmentation without other > > > > transmission control > > > > > > > > features such as found in TCP. > > > > > > > > > > > > > > > > 3. Syslog-transport-udp will rely on IP > fragmentation and > > > > > > > > we > > > > > will > > > > > > > get > > > > > > > > rid of "proprietary" fragmentation which was designed > > > > to handle > > > > > > > > messages over 64K and deal with various non-compliant > > > > > > > > network hosts. > > > > > > > > > > > > > > > > 4. Syslog-transport-udp will recommend sending messages > > > within > > > > > the > > > > > > > > > > > > > > boundaries of prevalent MTU size on a given network > > > > to be safe. > > > > > It > > > > > > > > > > > > > > will recommend Ethernet's 1500 bytes less headers and > > > > will draw > > > > > > > reader > > > > > > > > attention to the minimum MTU size hosts on the > network are > > > > > > required > > > > > > > to > > > > > > > > support for > > > > > > > > IPv6 (576 bytes) and IPv6 (1280 bytes). > > > > > > > > > > > > > > > > 5. Path MTU discovery may not work robustly and some > > > > > > > > TCP/IP > > > > > stacks > > > > > > > may > > > > > > > > not support UDP packets of full 64K size and truncate > > > > them. We > > > > > > will > > > > > > > > > > > > > > > mention this and bite this bullet. > > > > > > > > We should not restrict the protocol due to incompliant > > > > > > > implementations > > > > > > > > because it is a moving target and penalizes compliant > > > > > > > implementations > > > > > > > > with extra overhead. The above size recommendation would > > > > > partially > > > > > > > > > > > > > > deal with this, but leave the final choice up to the > > > > > > > > administrator. > > > > > > > > > > > > > > > > 6. We will get rid of most syslog transport headers for > > > > > > > > UDP > > > as > > > > > > they > > > > > > > > will no longer be needed. The only thing that will be > > > > > > > > left > > > is > > > > > the > > > > > > > > > > > > > > transport protocol version prefixed to every > syslog message. > > > > > > Should > > > > > > > > > > > > > > > we even bother with that? > > > > > > > > > > > > > > > > This is a major change to the syslog-transport-udp. > > > > I'd like to > > > > > > get > > > > > > > > > > > > > > > positive feedback before I proceed with this update. > > > > > > > > The best part is that if we all agree on the above, the > > > > > > > > next > > > > > draft > > > > > > > > > > > > > > will be 1/3 of the size -- easier read for you. :) > > > > > > > > > > > > > > > > Thanks, > > > > > > > > Anton. > > > > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > > > > > > > Syslog-sec mailing list > > > > > > > > Syslog-sec@www.employees.org > > > > > > > > http://www.employees.org/mailman/listinfo/syslog-sec > > > > > > > > > > > > > > > _______________________________________________ > > > > > > > Syslog-sec mailing list > > > > > > > Syslog-sec@www.employees.org > > > > > > > http://www.employees.org/mailman/listinfo/syslog-sec > > > > > > > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > > > > > Syslog-sec mailing list > > > > > > Syslog-sec@www.employees.org > > > > > > http://www.employees.org/mailman/listinfo/syslog-sec > > > > > > > > > > > > > > > > > > _______________________________________________ > > > > > > Syslog-sec mailing list > > > > > > Syslog-sec@www.employees.org > > > > > > http://www.employees.org/mailman/listinfo/syslog-sec > > > > > > > > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > Syslog-sec mailing list > > Syslog-sec@www.employees.org > > http://www.employees.org/mailman/listinfo/syslog-sec > > > _______________________________________________ > Syslog-sec mailing list > Syslog-sec@www.employees.org > http://www.employees.org/mailman/listinfo/syslog-sec > _______________________________________________ Syslog-sec mailing list Syslog-sec@www.employees.org http://www.employees.org/mailman/listinfo/syslog-sec From syslog-sec-bounces@willers.employees.org Thu Oct 21 11:20:23 2004 Received: from willers.employees.org (willers.employees.org [192.83.249.36]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA12419 for ; Thu, 21 Oct 2004 11:20:23 -0400 (EDT) Received: from willers.employees.org (localhost.employees.org [127.0.0.1]) by willers.employees.org (Postfix) with ESMTP id 66F645C78C; Thu, 21 Oct 2004 08:20:24 -0700 (PDT) X-Original-To: syslog-sec@employees.org Delivered-To: syslog-sec@employees.org Received: from sccrmhc12.comcast.net (sccrmhc12.comcast.net [204.127.202.56]) by willers.employees.org (Postfix) with ESMTP id 92DBE5C722 for ; Thu, 21 Oct 2004 08:11:12 -0700 (PDT) Received: from djyxpy41 (h00104b8ce2a3.ne.client2.attbi.com[24.128.104.220]) by comcast.net (sccrmhc12) with SMTP id <2004102115110501200g0p0re>; Thu, 21 Oct 2004 15:11:05 +0000 From: "David B Harrington" To: "'Anton Okmianski'" , "'Chris Lonvick'" , Subject: RE: [Syslog-sec] RE: Maximum message size Date: Thu, 21 Oct 2004 11:11:04 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook, Build 11.0.5510 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1441 Thread-Index: AcS3a+90AcdCGH0sRdSOacXEmHTX1QADDIcQAAEUeSA= In-Reply-To: <200410211424.AML52489@flask.cisco.com> Message-Id: <20041021151112.92DBE5C722@willers.employees.org> X-Mailman-Approved-At: Thu, 21 Oct 2004 08:20:22 -0700 X-BeenThere: syslog-sec@www.employees.org X-Mailman-Version: 2.1.4 Precedence: list Reply-To: ietfdbh@comcast.net List-Id: syslog-sec.www.employees.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: syslog-sec-bounces@willers.employees.org Errors-To: syslog-sec-bounces@willers.employees.org Content-Transfer-Encoding: 7bit Hi, The reason SNMP has a 480 minimum-maximum has to do with difficult delivery problems in a broken network. SNMP uses UDP and has this 480 restriction deliberately to deliberately avoid session overhead and message fragmentation. In a network being troubleshot, the likelihood of getting one single-packet message delivered successfully is higher than getting three message fragments delivered successfully, so using a 1K minimum-maximum may prevent the operator from getting some critical information about the problem, whereas using a 480 minimum-maximum might get that information to the operator. Which again raises the issue of the original use case for syslog. Is syslog meant to be a protocol useful for troubleshooting devices over a network? If the WG decides this is no longer an important use case, then I suggest the WG should have a long hard discussion with operators to see if they agree with this decision. If syslog is to remain a primary tool for operators to troubleshoot broekn networks, then it should be designed with broken-network delivery problems in mind. It should avoid encouraging implementors to use a minimum-maximum messages size that would require fragmentation. dbh co-chair SNMPv3 WG, concluded -----Original Message----- From: syslog-sec-bounces@www.employees.org [mailto:syslog-sec-bounces@www.employees.org] On Behalf Of Anton Okmianski Sent: Thursday, October 21, 2004 10:24 AM To: 'Chris Lonvick'; syslog-sec@employees.org Subject: RE: [Syslog-sec] RE: Maximum message size Rainer, Chris, all: My vote is for -protocol to say that any syslog *transport* MUST support transferring messages of at least 1Kb (or 2 Kb) and recommend that they support 64Kb. Udp-transport will, in turn, recommend that applications don't send message with sizes which exceed the MTU on a given network to avoid fragmentation and provide Ethernet MTU (less headers) as a guide. So, if somebody' network has the old MTUs of 576 bytes, we would still handle it, they will just need to set-up their TCP-IP stack with appropriate MTU (assuming MTU Discovery does not work) and there will be fragmentation for messages greater than this size. We could also say that *receivers* SHOULD support messages of at least 1Kb (or 2Kb). I would not set any MUST requirements on the receivers. Note the difference between requirements on receivers and transport protocol. Rainer, please get the consensus on this on the list and let's move on one way or the other. I will be happy with whatever the majority consensus is. Thanks, Anton. > -----Original Message----- > From: syslog-sec-bounces@willers.employees.org > [mailto:syslog-sec-bounces@willers.employees.org] On Behalf Of Chris > Lonvick > Sent: Thursday, October 21, 2004 8:44 AM > To: syslog-sec@employees.org > Subject: RE: [Syslog-sec] RE: Maximum message size > > Hi All, > > Do we have rough consensus on this? We can still request a slot at > the upcoming IETF to discuss this. > > Thanks, > Chris > > On Mon, 18 Oct 2004, Rainer Gerhards wrote: > > > Anton: > > > > I understand the source of the 480 byte limit you outlined in > > -transport. Even though we have now decided that we can use UDP > > fragementation, I still think the fact that 480 is the only safe > > number still persists. > > > > So I do not see any value in requiring an application to support > > messages of up to 1K when we know this might not work. I agree, it > > looks like it has worked (at least most of the time) so > far, but why > > introduce a problem if there is no need to? Maybe we had no > issues in > > the past because messages were most often below 480 octets. > > > > Also, I do not think that we can say the minimum to be > supported size > > is 1K in protocol and then allow a transport mapping to specify > > something lower. If it is 1K, then it MUST be 1K in all > cases. Else I > > can not see how we will be able to ensure interoperability. I am of > > the strong opinion we need to know a maximum size that is > guaranteed > > to be received. > > > > I do not see the value in raising the minimum to be > supported size to > > 1K. Maybe I am overlooking the obvious, but I think we deliberatley > > introduce problems if we go above that. > > > > And, yes, I am refering to IPv4 only, because I am looking at the > > weakest transport (which, in this case, is also the most > commonly used). > > > > Rainer > > > > > -----Original Message----- > > > From: Anton Okmianski [mailto:aokmians@cisco.com] > > > Sent: Thursday, October 14, 2004 7:54 PM > > > To: Rainer Gerhards; ietfdbh@comcast.net; syslog-sec@employees.org > > > Subject: RE: [Syslog-sec] RE: Maximum message size > > > > > > Rainer: > > > > > > The ~480 byte limit was on the size of packet, not datagram or > > > message which can be fragmented into multiple packets. > Besides, for > > > IPv6 it was ~1160. In order to avoid requiring people to > implement > > > "proprietary" transport fragmentation we did say that > > > implementations must only support messages of size which does not > > > require this fragmentation. But we don't do that fragmentation > > > anymore and rely on IP fragmentation. > > > > > > If the intention of this MUST requirement is a baseline > > > interoperability and we know that existing applications may be > > > sending messages as large as 1K, then it seems to me the minimum > > > required support should be at least that big. > > > > > > The transport may provide its own recommendations. For > example, udp > > > transport will *recommend* that messages do not exceed > the smallest > > > MTU size on the network, which could be smaller or > larger, but that > > > will be just a recommendation. For TCP, it is not as big of an > > > issue, for example, because it handles loss of segments. I don't > > > think such transport layer requirements should be in the protocol > > > which describes the content of the messages. > > > > > > Cheers, > > > > > > Anton. > > > > > > > -----Original Message----- > > > > From: Rainer Gerhards [mailto:rgerhards@hq.adiscon.com] > > > > Sent: Thursday, October 14, 2004 11:00 AM > > > > To: Anton Okmianski; ietfdbh@comcast.net; > syslog-sec@employees.org > > > > Subject: RE: [Syslog-sec] RE: Maximum message size > > > > > > > > Hello Anton: > > > > > > > > the 480 octet limit stems back to your work and proposal in > > > > -transport-udp. I thought you found out that more than > 480 bytes > > > > can not be transmitted "reliably" (whatever this means for > > > UDP). > > > > > > > > Of course, I can remove this and we could simply state that > > > > senders send data and receivers receive data and that > the receiver > > > > MAY receive any size or not. But wouldn't it make sense to have > > > > one size that I can count on to be received? > > > > > > > > The intension of the 480 octets was to provide a known message > > > > size that every implementation actually MUST support > > > > - so if an applications intends to send a message that MUST be > > > > received fully, that application knows a maximum size. > > > > > > > > If you mean that with the upcoming version of > -transport-udp the > > > > 480 octet can safely be replaced by 1K or 2K then we of > course can > > > > change this. > > > > > > > > I, however, think that a minimal burden should be placed on an > > > > implementation. I would tend to call a syslog receiver that is > > > > only capable of receiving 400 octets to be > non-compliant. But, OK, > > > > maybe we should say this is OK. So what limit is it > then? Is 200 > > > > octets compliant? 100? 50? > > > > Whatever it is, I think we should require one size... > If we do not > > > > require this, one could argue writing/parsing a correct > header is > > > > also too much of a burden for a given scenario... > > > > > > > > Rainer > > > > > > > > > -----Original Message----- > > > > > From: Anton Okmianski [mailto:aokmians@cisco.com] > > > > > Sent: Thursday, October 14, 2004 4:36 PM > > > > > To: ietfdbh@comcast.net; Rainer Gerhards; > > > > > syslog-sec@employees.org > > > > > Subject: RE: [Syslog-sec] RE: Maximum message size > > > > > > > > > > Rainer: > > > > > > > > > > I agree with a SHOULD on 64k support on receivers and MAY on > > > larger > > > > > sizes. > > > > > > > > > > I think we need to remove or change this sentence: "A > > > > receiver MUST be > > > > > able to accept messages up to and including 480 > octets in length". > > > > > > > > > > 1. Is this an absolute requirement? We keep thinking about > > > general > > > > > purpose receiver libraries. But it does not have to > be general > > > > > purpose. If I have two specific applications with their own > > > syslog > > > > > sender/receiver implementations (how hard is it to > > > > send/receive a udp > > > > > message) and they exchange syslog messages which are > known not > > > > > to exceed 400 bytes, should my receiver be required > to support > > > > > 8k messages? > > > > > > > > > > 2. Why 480? If this is intended to ensure basic > > > > > interoperability, > > > > > > > > then at least this should be a size greater than 1K (size > > > > > previous > > > > > > > > syslog RFC observed). Maybe 4k or 8k. At least it > is a better > > > > > arbitrary number. > > > > > > > > > > So, I would support either removing the MUST requirement > > > > all together > > > > > or at least changing it to a more reasonable size. The > > > > latter approach > > > > > would be more limiting for implementations, but will insure > > > > > interoperability for general-purpose syslog libraries if > > > > this is the > > > > > overriding goal. > > > > > > > > > > Thanks, > > > > > Anton. > > > > > > > > > > > -----Original Message----- > > > > > > From: syslog-sec-bounces@willers.employees.org > > > > > > [mailto:syslog-sec-bounces@willers.employees.org] On > > > > Behalf Of David > > > > > > B Harrington > > > > > > Sent: Thursday, October 14, 2004 8:53 AM > > > > > > To: 'Rainer Gerhards'; syslog-sec@employees.org > > > > > > Subject: RE: [Syslog-sec] RE: Maximum message size > > > > > > > > > > > > Hi Rainer, > > > > > > > > > > > > Thanks for making the changes. > > > > > > > > > > > > I'd like to make one point a little more strenuously: > > > > > > "For interoperability reasons, all receiver > > > > implementations SHOULD > > > > > > be able to accept messages up to and including > 65,535 octets > > > > > > in length." > > > > > > > > > > > > Also, I think the last two sentences could be combined into > > > > > > one > > > - > > > > > > "If a receiver receives a message with a length larger > > > > than 65,535 > > > > > > octets, or larger than it supports, the receiver > MAY discard > > > > > > the > > > > > > > > > message or truncate the payload." > > > > > > > > > > > > dbh > > > > > > > > > > > > -----Original Message----- > > > > > > From: syslog-sec-bounces@www.employees.org > > > > > > [mailto:syslog-sec-bounces@www.employees.org] On Behalf Of > > > Rainer > > > > > > Gerhards > > > > > > Sent: Thursday, October 14, 2004 7:34 AM > > > > > > To: syslog-sec@employees.org > > > > > > Subject: [Syslog-sec] RE: Maximum message size > > > > > > > > > > > > David and all, > > > > > > > > > > > > many thanks for all the good comments. > > > > > > > > > > > > I agree that 16 MB for a single message is far more than > > > actually > > > > > > should be used. Let's look a bit at the history of this: > > > > > > > > > > > > Initially, we noticed that the 1024 octet limit for > > > > current syslog > > > > > > is too short. I think there is still general agreement on > > > > this. We > > > > > > had a good discussion about the size issue in September > > > > 2003. A link > > > > > > to an archive is here: > > > > > > > > > > > > http://www.syslog.cc/ietf/autoarc/msg00855.html > > > > > > > > > > > > The general issue is that 1K is too short for many > > > > applications. For > > > > > > example, Marshall Glen mentions the healthcare needs > > > > (RFC3881) for a > > > > > > larger message. As far as I know, the IHE > initiative currently > > > > > > standardizes on a maximum XML stream size of 32K (which is > > > > > > then supposed to be transmitted via syslog). I > think a message > > > > size limit > > > > > > of 64K would probably be suffcient for a long time to come > > > > > > (and leaves some headroom for message content increases). > > > > > > > > > > > > Besides that, I myself identified a need - in very rare > > > > occasions - > > > > > > to send information larger than 64K. In extreme cases, this > > > > > > information could be as large as one megabyte. > > > > > > Again, these were cases that might happen once in a > year. All > > > > > > of > > > > > > > > > this, of course, only if an operator > > > > > > *really* wants to submit that large information. So the > > > > > > initial approach and suggestion was to introduce a way to > > > > > > build "message > > > > > > > > > groups". That would have been a way to split "oversize > > > > messages" (as > > > > > > they were called > > > > > > then) into multiple syslog messages. They key to > that concept > > > was > > > > > > that each single message could be kept within a > > > > reasonable limit (it > > > > > > would have worked even with 1K max message size). It > > > > would only have > > > > > > been a way to transmit very large messages in those very > > > > > > seldom cases that they would actually be used. > > > > > > > > > > > > Some time later, we came to the conclusion that it > should be > > > > > > possible for a single syslog message to be of the largest > > > > imaginable > > > > > > size. It was argued that fragmentation is a transport > > > > issue and as > > > > > > such application-level "fragmentation" > > > > > > (the message groups) would not do good. We agreed to that. > > > > > > > > > > > > Even some time later, we needed to define a maximum message > > > size, > > > > > > just to limit it somewhere (if I recall correctly, > that was a > > > > > > transport requirement). We picked 16MB just as a > number that > > > > > > we thought would never have be reached and such be > > > > sufficient for all > > > > > > imaginable time to come. As far as I was concerend, that > > > decision > > > > > > was driven by the fact that in the past there were often > > > > limits that > > > > > > nobody expected to reach - and yet they wer quickly reached. > > > > > > Som 16MB was picked more or less as "larger than anybody > > > > will need > > > > > > anytime in the forseeable future". We did not even > have a big > > > > > > discussion on that limit. > > > > > > > > > > > > I think this are the key points on how we reached > this limit. > > > > > > Now, thanks to your call and looking the whole situation, > > > > I really > > > > > > think we've gone overboard. That 16MB limit is something > > > > > > nobody intends to use. Even a lower - but high limit - is > > > > > > extremely unlikely. I'd expect to see a 1MB > "message" in maybe > > > > 0.0001% of the > > > > > > cases - if at all (but I expect to see e.g. 70KB > > > > "message" in maybe > > > > > > 0.001% of the cases). > > > > > > > > > > > > Also, I have read your proposal to set a maxium > size for the > > > > > > message, but allow an implementation to go beyond > that. Maybe > > > > > > something like > > > > > > this: > > > > > > > > > > > > #### > > > > > > 5.1 Message Length > > > > > > > > > > > > A receiver MUST be able to accept messages up to and > > > including > > > > > > 480 octets in length. > > > > > > > > > > > > A receiver SHOULD be able to accept messages up to and > > > > including > > > > > > 65,535 octets in length. > > > > > > If a receiver receives a message with a length > larger than > > > > > > it > > > > > > > > > supports, the receiver MAY > > > > > > discard the message or truncate the payload. > > > > > > > > > > > > A receiver MAY accept messages larger than 65,535 octets. > > > > > > #### > > > > > > > > > > > > I've dropped the wording that transports MAY support only > > > > a smaller > > > > > > max size - I think UDP was the most critical in this > > > > regard and it > > > > > > obviously handles 64K. So I assume this clause is not > > > > really needed > > > > > > if we go for 64K as the max size. > > > > > > > > > > > > With that clause, we would limit the size for all interop > > > > cases, but > > > > > > leave a window open so that the message length could be > > > > extended if > > > > > > there is a real need AND the operator and software vendor > > > > > > wants this. > > > > > > I assume nobody will implement this burden if there is no > > > > > > market > > > > > > > > > need to do so. But if it is, it clould at least be > > > > implemented in a > > > > > > standards-compliant way. > > > > > > > > > > > > If I look back at the original requirement - > provide a way to > > > > > > transmit very rare oversize messages, I think we can safely > > > > > > implement this the bounds of the current proposals, even > > > > if the size > > > > > > limit does not go beyond 64K. If somebody needs it, > this can > > > > > > be easily done via an experimental (or > > > > > > vendor-specific) SD-ID. Again, I do not think somebody > > > > will accept > > > > > > this burden without market need. > > > > > > > > > > > > In the light of this, I, too, would recommend that we limit > > > > > > the message size to 64K. I would prefer, however, > that we keep > > > > > > the maximum open as in the suggested wording above. > > > > > > > > > > > > Rainer > > > > > > > > > > > > > -----Original Message----- > > > > > > > From: David B Harrington [mailto:ietfdbh@comcast.net] > > > > > > > Sent: Wednesday, October 13, 2004 5:10 PM > > > > > > > To: Rainer Gerhards; 'Anton Okmianski'; > > > syslog-sec@employees.org > > > > > > > Subject: Maximum message size > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > I am concerned that we are in danger of promoting uses of > > > syslog > > > > > > that > > > > > > > will defeat its current usefulness as a > human-readable log > > > > > > > of > > > > > system > > > > > > > > > > > > > activity. > > > > > > > > > > > > > > Do network operators really believe the 16,777,216 > > > > octet size is a > > > > > > > > > > > > needed improvement to syslog, or are we designing > this size > > > > > > > in > > > > > > > > > > explicitly for the vendors of applications who want to > > > > use syslog > > > > > as > > > > > > a > > > > > > > programmatic interface? I believe Syslog should > be designed > > > > > > > to > > > > > meet > > > > > > > the needs of operators. Most of the discussion ssems to be > > > from > > > > > > > vendors of syslog receivers or applications. Other > > > > protocols such > > > > > as > > > > > > > > > > > > > FTP might be better used for such a specialty use case. > > > > > > > > > > > > > > Does allowing messages of 16,777,216 octets to be > > > > > > > accumulated > > > > > within > > > > > > > > > > > > > the system log make it harder to use some > > > > widely-available minimal > > > > > > > > > > > > text editors, like Notepad, to view the system log? Will > > > having > > > > > huge > > > > > > > > > > > > > application-specific messages in the system log make it > > > > harder for > > > > > > an > > > > > > > operator to locate useful troubleshooting messages within > > > > > > > the > > > > > system > > > > > > > > > > > > > log? Can the information in such a huge message > fit within > > > > > > > an > > > > > 80x25 > > > > > > > screen, when an operator is troubleshooting > network problems > > > and > > > > > > needs > > > > > > > to use a bare-bones terminal attached to the > serial port of > > > > > > > a > > > > > device > > > > > > > > > > > > > to view the system log? > > > > > > > > > > > > > > Syslog was designing to be an operator's tool, and Syslog > > > should > > > > > be > > > > > > > designed to meet the needs of operators. Will allowing > > > messages > > > > > of > > > > > > > this size negatively impact its usefulness in the primary > > > > > > > use > > > > > case? > > > > > > > > > > > > > > What do network **operators** think about the need for > > > > messages of > > > > > > > > > > > > this size? Have we deliberately asked them their > opinion on > > > this > > > > > by, > > > > > > > > > > > > > for example, taking a survey or going to NANOG to > ask them? > > > > > > > > > > > > > > I don't think we should support such large messages in > > > > the system > > > > > > > logging protocol. > > > > > > > > > > > > > > > > > > > > > David Harrington > > > > > > > dbharrington@comcast.net > > > > > > > > > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > From: syslog-sec-bounces@www.employees.org > > > > > > > [mailto:syslog-sec-bounces@www.employees.org] On Behalf > > > > Of Rainer > > > > > > > Gerhards > > > > > > > Sent: Wednesday, October 13, 2004 5:17 AM > > > > > > > To: Anton Okmianski; syslog-sec@employees.org > > > > > > > Subject: RE: [Syslog-sec] UDP message size issue proposal > > > > > > > > > > > > > > Anton, > > > > > > > > > > > > > > I agree to your conclusion. > > > > > > > > > > > > > > Although I am the one who always voted for larger > sizes, the > > > > > > > discussion has shown that this yields to unnecessary > > > > complexity. I > > > > > > can > > > > > > > satisfy my needs with a vendor-extension structured > > > > data element, > > > > > > > which I think shows the flexibility of the new drafts. > > > > > > > > > > > > > > So I support your move. I would tend to even remove the > > > > transport > > > > > > > version header. If there is need to, we could always > > > > include that > > > > > in > > > > > > a > > > > > > > later release (e.g. "v1", which would make it clearly > > > > > distingshable > > > > > > > from the current frame format). I see little need > to do so, > > > > > though. > > > > > > > > > > > > > > Regarding -protocol, I think we should still keep an upper > > > limit > > > > > in > > > > > > > the spec. Even I don't see any reason why a syslog > > > > message should > > > > > go > > > > > > > > > > > > > above 16MB. So for -protocol, I propose the following new > > > text: > > > > > > > > > > > > > > #### > > > > > > > 5.1 Message Length > > > > > > > > > > > > > > The maximum length of any syslog message is > > > > 16,777,216 octets. > > > > > > Any > > > > > > > receiver receiving a larger message MUST discard the > > > message. > > > > > A > > > > > > > diagnostic message SHOULD be logged in this case. > > > > > > > > > > > > > > A receiver MUST be able to accept messages that are > > > > 480 octets > > > > > > (or > > > > > > > less) in length. A receiver SHOULD be able to > > > > accept messages > > > > > > that > > > > > > > are 65,535 octets (or less) in length. It is > > > > RECOMMENDED that > > > > > > > receivers be able to accept messages up to the > > > > maximum message > > > > > > > length > > > > > > > of 16,777,216 octets. > > > > > > > > > > > > > > If a receiver receives a message within the maximum > > > > length, but > > > > > > > > > > > > with > > > > > > > a length larger than it handles, the receiver > MAY discard > > > or > > > > > > > truncate > > > > > > > it. > > > > > > > > > > > > > > A transport mapping MAY define a maximum > length below the > > > one > > > > > > > specified here. Each transport mapping MUST support > > > > a maximum > > > > > > size > > > > > > > of 480 octets or more. > > > > > > > #### > > > > > > > > > > > > > > If there is agreement on this, I can post a new version of > > > > > > -protocol. > > > > > > > That version will also include some changes made thanks > > > > to Andrew > > > > > > Ross > > > > > > > comments (sent via private mail). I have finished this > > > > version. It > > > > > > is > > > > > > > available for immediate posting, but I would like to wait > > > > > > > for agreement on the size issue (at least if that can be > > > > > > > reached > > > > > > quickly). > > > > > > > > > > > > > > Rainer > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > From: syslog-sec-bounces@www.employees.org > > > > > > > > [mailto:syslog-sec-bounces@www.employees.org] On > > > > Behalf Of Anton > > > > > > > > > > > > > Okmianski > > > > > > > > Sent: Tuesday, October 12, 2004 11:25 PM > > > > > > > > To: syslog-sec@employees.org > > > > > > > > Subject: [Syslog-sec] UDP message size issue proposal > > > > > > > > > > > > > > > > Hi! > > > > > > > > > > > > > > > > Sorry for a long delay on this issue - I was on a 2 week > > > > > vacation. > > > > > > > I > > > > > > > > have spoken with a number of TCP/UDP/IP experts > regarding > > > the > > > > > > sizing > > > > > > > > > > > > > > > issue. I am ready to propose the following changes: > > > > > > > > > > > > > > > > 1. Syslog-protocol will state that the max > message will be > > > > > defined > > > > > > > by > > > > > > > > the transport layer. > > > > > > > > > > > > > > > > 2. Syslog-transport-udp will support messages up to > > > > maximum UDP > > > > > > > > datagram size of 64K. UDP is a very bad choice for > > > > large message > > > > > > > > > > > > > transmissions, so it does not make sense for us to > > > > stretch it by > > > > > > > > > > > > > adding our own fragmentation without other > > > > transmission control > > > > > > > > features such as found in TCP. > > > > > > > > > > > > > > > > 3. Syslog-transport-udp will rely on IP > fragmentation and > > > > > > > > we > > > > > will > > > > > > > get > > > > > > > > rid of "proprietary" fragmentation which was designed > > > > to handle > > > > > > > > messages over 64K and deal with various non-compliant > > > > > > > > network hosts. > > > > > > > > > > > > > > > > 4. Syslog-transport-udp will recommend sending messages > > > within > > > > > the > > > > > > > > > > > > > > boundaries of prevalent MTU size on a given network > > > > to be safe. > > > > > It > > > > > > > > > > > > > > will recommend Ethernet's 1500 bytes less headers and > > > > will draw > > > > > > > reader > > > > > > > > attention to the minimum MTU size hosts on the > network are > > > > > > required > > > > > > > to > > > > > > > > support for > > > > > > > > IPv6 (576 bytes) and IPv6 (1280 bytes). > > > > > > > > > > > > > > > > 5. Path MTU discovery may not work robustly and some > > > > > > > > TCP/IP > > > > > stacks > > > > > > > may > > > > > > > > not support UDP packets of full 64K size and truncate > > > > them. We > > > > > > will > > > > > > > > > > > > > > > mention this and bite this bullet. > > > > > > > > We should not restrict the protocol due to incompliant > > > > > > > implementations > > > > > > > > because it is a moving target and penalizes compliant > > > > > > > implementations > > > > > > > > with extra overhead. The above size recommendation would > > > > > partially > > > > > > > > > > > > > > deal with this, but leave the final choice up to the > > > > > > > > administrator. > > > > > > > > > > > > > > > > 6. We will get rid of most syslog transport headers for > > > > > > > > UDP > > > as > > > > > > they > > > > > > > > will no longer be needed. The only thing that will be > > > > > > > > left > > > is > > > > > the > > > > > > > > > > > > > > transport protocol version prefixed to every > syslog message. > > > > > > Should > > > > > > > > > > > > > > > we even bother with that? > > > > > > > > > > > > > > > > This is a major change to the syslog-transport-udp. > > > > I'd like to > > > > > > get > > > > > > > > > > > > > > > positive feedback before I proceed with this update. > > > > > > > > The best part is that if we all agree on the above, the > > > > > > > > next > > > > > draft > > > > > > > > > > > > > > will be 1/3 of the size -- easier read for you. :) > > > > > > > > > > > > > > > > Thanks, > > > > > > > > Anton. > > > > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > > > > > > > Syslog-sec mailing list > > > > > > > > Syslog-sec@www.employees.org > > > > > > > > http://www.employees.org/mailman/listinfo/syslog-sec > > > > > > > > > > > > > > > _______________________________________________ > > > > > > > Syslog-sec mailing list > > > > > > > Syslog-sec@www.employees.org > > > > > > > http://www.employees.org/mailman/listinfo/syslog-sec > > > > > > > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > > > > > Syslog-sec mailing list > > > > > > Syslog-sec@www.employees.org > > > > > > http://www.employees.org/mailman/listinfo/syslog-sec > > > > > > > > > > > > > > > > > > _______________________________________________ > > > > > > Syslog-sec mailing list > > > > > > Syslog-sec@www.employees.org > > > > > > http://www.employees.org/mailman/listinfo/syslog-sec > > > > > > > > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > Syslog-sec mailing list > > Syslog-sec@www.employees.org > > http://www.employees.org/mailman/listinfo/syslog-sec > > > _______________________________________________ > Syslog-sec mailing list > Syslog-sec@www.employees.org > http://www.employees.org/mailman/listinfo/syslog-sec > _______________________________________________ Syslog-sec mailing list Syslog-sec@www.employees.org http://www.employees.org/mailman/listinfo/syslog-sec _______________________________________________ Syslog-sec mailing list Syslog-sec@www.employees.org http://www.employees.org/mailman/listinfo/syslog-sec From syslog-sec-bounces@willers.employees.org Thu Oct 21 11:53:39 2004 Received: from willers.employees.org (willers.employees.org [192.83.249.36]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA15808 for ; Thu, 21 Oct 2004 11:53:38 -0400 (EDT) Received: from willers.employees.org (localhost.employees.org [127.0.0.1]) by willers.employees.org (Postfix) with ESMTP id 5B6715C7B1; Thu, 21 Oct 2004 08:53:37 -0700 (PDT) X-Original-To: syslog-sec@employees.org Delivered-To: syslog-sec@employees.org Received: from mail.hq.adiscon.com (mail.hq.adiscon.com [217.6.190.188]) by willers.employees.org (Postfix) with ESMTP id EB8935C791 for ; Thu, 21 Oct 2004 08:51:37 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by mail.hq.adiscon.com (Postfix) with ESMTP id 35C3F9C761; Thu, 21 Oct 2004 18:09:15 +0200 (CEST) Received: from mail.hq.adiscon.com ([127.0.0.1]) by localhost (grfdeb [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 27251-09; Thu, 21 Oct 2004 18:08:50 +0200 (CEST) Received: from grfint2.intern.adiscon.com (grfint2 [172.19.0.6]) by mail.hq.adiscon.com (Postfix) with ESMTP id CCBEB9C755; Thu, 21 Oct 2004 18:08:50 +0200 (CEST) Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: [Syslog-sec] RE: Maximum message size X-MimeOLE: Produced By Microsoft Exchange V6.5.7226.0 Date: Thu, 21 Oct 2004 17:54:52 +0200 Message-ID: <577465F99B41C842AAFBE9ED71E70ABA030266@grfint2.intern.adiscon.com> Thread-Topic: [Syslog-sec] RE: Maximum message size Thread-Index: AcS3a+90AcdCGH0sRdSOacXEmHTX1QADDIcQAAEUeSAAAiy3QA== From: "Rainer Gerhards" To: , "Anton Okmianski" , X-Virus-Scanned: by amavisd-new-20030616-p7 (Debian) at adiscon.com X-Mailman-Approved-At: Thu, 21 Oct 2004 08:53:36 -0700 X-BeenThere: syslog-sec@www.employees.org X-Mailman-Version: 2.1.4 Precedence: list List-Id: syslog-sec.www.employees.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: syslog-sec-bounces@willers.employees.org Errors-To: syslog-sec-bounces@willers.employees.org Content-Transfer-Encoding: quoted-printable Hi all, I share David's concerns. I still think limiting to 480 octets is necessary to support the broken-network use case that David outlined. Actually, this is one use case, but I think a very important one. For these type of alert messages, 480 octets is far then enough. Traditional syslog messages, at least in UNIX environments, are typically even below 100 octets. There is another use case, which is transmittal of logging information during normal operation (like in healthcare and other auditing environments). Here, this size is often too small. But by allowing an application to support sending/receiving larger message sizes, we support this scenario, too. I think we have created a good compromise by the wording below: #### 5.1 Message Length A receiver MUST be able to accept messages up to and including 480 octets in length. For interoperability reasons, all receiver implementations=20 SHOULD be able to accept messages up to and including 2,048 octets=20 in length. If a receiver receives a message with a length larger than 2,048 octets, or larger than it supports, the receiver MAY discard the message or truncate the payload. ####=20 Actively participating in the discussion were Anton and David so far. As Anton said he can live with this way, too, I think this is the current rough concensus. If anybody violently OBJECTS the given wording, please make yourself heared NOW.=20 Otherwise, I will see that I can change the draft tomorrow so that it can (hopefully) be submitted before the meeting cutoff date. Rainer > -----Original Message----- > From: syslog-sec-bounces@www.employees.org=20 > [mailto:syslog-sec-bounces@www.employees.org] On Behalf Of=20 > David B Harrington > Sent: Thursday, October 21, 2004 5:11 PM > To: 'Anton Okmianski'; 'Chris Lonvick'; syslog-sec@employees.org > Subject: RE: [Syslog-sec] RE: Maximum message size=20 >=20 > Hi, >=20 > The reason SNMP has a 480 minimum-maximum has to do with difficult > delivery problems in a broken network. SNMP uses UDP and has this 480 > restriction deliberately to deliberately avoid session overhead and > message fragmentation. In a network being troubleshot, the likelihood > of getting one single-packet message delivered successfully is higher > than getting three message fragments delivered successfully, so using > a 1K minimum-maximum may prevent the operator from getting some > critical information about the problem, whereas using a 480 > minimum-maximum might get that information to the operator. >=20 > Which again raises the issue of the original use case for syslog. Is > syslog meant to be a protocol useful for troubleshooting devices over > a network? If the WG decides this is no longer an important use case, > then I suggest the WG should have a long hard discussion with > operators to see if they agree with this decision. >=20 > If syslog is to remain a primary tool for operators to troubleshoot > broekn networks, then it should be designed with broken-network > delivery problems in mind. It should avoid encouraging implementors to > use a minimum-maximum messages size that would require fragmentation. >=20 > dbh > co-chair SNMPv3 WG, concluded >=20 > -----Original Message----- > From: syslog-sec-bounces@www.employees.org > [mailto:syslog-sec-bounces@www.employees.org] On Behalf Of Anton > Okmianski > Sent: Thursday, October 21, 2004 10:24 AM > To: 'Chris Lonvick'; syslog-sec@employees.org > Subject: RE: [Syslog-sec] RE: Maximum message size=20 >=20 > Rainer, Chris, all: >=20 > My vote is for -protocol to say that any syslog *transport* MUST > support transferring messages of at least 1Kb (or 2 Kb) and recommend > that they support 64Kb. Udp-transport will, in turn, recommend that > applications don't send message with sizes which exceed the MTU on a > given network to avoid fragmentation and provide Ethernet MTU (less > headers) as a guide.=20 >=20 > So, if somebody' network has the old MTUs of 576 bytes, we would still > handle it, they will just need to set-up their TCP-IP stack with > appropriate MTU (assuming MTU Discovery does not work) and there will > be fragmentation for messages greater than this size. =20 >=20 > We could also say that *receivers* SHOULD support messages of at least > 1Kb (or 2Kb). =20 > I would not set any MUST requirements on the receivers. Note the > difference between requirements on receivers and transport protocol. >=20 > Rainer, please get the consensus on this on the list and let's move on > one way or the other. I will be happy with whatever the majority > consensus is. >=20 > Thanks, > Anton.=20 >=20 >=20 >=20 > > -----Original Message----- > > From: syslog-sec-bounces@willers.employees.org > > [mailto:syslog-sec-bounces@willers.employees.org] On Behalf Of Chris >=20 > > Lonvick > > Sent: Thursday, October 21, 2004 8:44 AM > > To: syslog-sec@employees.org > > Subject: RE: [Syslog-sec] RE: Maximum message size > >=20 > > Hi All, > >=20 > > Do we have rough consensus on this? We can still request a slot at=20 > > the upcoming IETF to discuss this. > >=20 > > Thanks, > > Chris > >=20 > > On Mon, 18 Oct 2004, Rainer Gerhards wrote: > >=20 > > > Anton: > > > > > > I understand the source of the 480 byte limit you outlined in=20 > > > -transport. Even though we have now decided that we can use UDP=20 > > > fragementation, I still think the fact that 480 is the only safe=20 > > > number still persists. > > > > > > So I do not see any value in requiring an application to support=20 > > > messages of up to 1K when we know this might not work. I agree, it >=20 > > > looks like it has worked (at least most of the time) so > > far, but why > > > introduce a problem if there is no need to? Maybe we had no > > issues in > > > the past because messages were most often below 480 octets. > > > > > > Also, I do not think that we can say the minimum to be > > supported size > > > is 1K in protocol and then allow a transport mapping to specify=20 > > > something lower. If it is 1K, then it MUST be 1K in all > > cases. Else I > > > can not see how we will be able to ensure interoperability. I am > of=20 > > > the strong opinion we need to know a maximum size that is > > guaranteed > > > to be received. > > > > > > I do not see the value in raising the minimum to be > > supported size to > > > 1K. Maybe I am overlooking the obvious, but I think we > deliberatley=20 > > > introduce problems if we go above that. > > > > > > And, yes, I am refering to IPv4 only, because I am looking at the=20 > > > weakest transport (which, in this case, is also the most > > commonly used). > > > > > > Rainer > > > > > > > -----Original Message----- > > > > From: Anton Okmianski [mailto:aokmians@cisco.com] > > > > Sent: Thursday, October 14, 2004 7:54 PM > > > > To: Rainer Gerhards; ietfdbh@comcast.net; > syslog-sec@employees.org > > > > Subject: RE: [Syslog-sec] RE: Maximum message size > > > > > > > > Rainer: > > > > > > > > The ~480 byte limit was on the size of packet, not datagram or=20 > > > > message which can be fragmented into multiple packets. > > Besides, for > > > > IPv6 it was ~1160. In order to avoid requiring people to > > implement > > > > "proprietary" transport fragmentation we did say that=20 > > > > implementations must only support messages of size which does > not=20 > > > > require this fragmentation. But we don't do that fragmentation=20 > > > > anymore and rely on IP fragmentation. > > > > > > > > If the intention of this MUST requirement is a baseline=20 > > > > interoperability and we know that existing applications may be=20 > > > > sending messages as large as 1K, then it seems to me the minimum >=20 > > > > required support should be at least that big. > > > > > > > > The transport may provide its own recommendations. For > > example, udp > > > > transport will *recommend* that messages do not exceed > > the smallest > > > > MTU size on the network, which could be smaller or > > larger, but that > > > > will be just a recommendation. For TCP, it is not as big of an=20 > > > > issue, for example, because it handles loss of segments. I > don't=20 > > > > think such transport layer requirements should be in the > protocol=20 > > > > which describes the content of the messages. > > > > > > > > Cheers, > > > > > > > > Anton. > > > > > > > > > -----Original Message----- > > > > > From: Rainer Gerhards [mailto:rgerhards@hq.adiscon.com] > > > > > Sent: Thursday, October 14, 2004 11:00 AM > > > > > To: Anton Okmianski; ietfdbh@comcast.net; > > syslog-sec@employees.org > > > > > Subject: RE: [Syslog-sec] RE: Maximum message size > > > > > > > > > > Hello Anton: > > > > > > > > > > the 480 octet limit stems back to your work and proposal in=20 > > > > > -transport-udp. I thought you found out that more than > > 480 bytes > > > > > can not be transmitted "reliably" (whatever this means for > > > > UDP). > > > > > > > > > > Of course, I can remove this and we could simply state that=20 > > > > > senders send data and receivers receive data and that > > the receiver > > > > > MAY receive any size or not. But wouldn't it make sense to > have=20 > > > > > one size that I can count on to be received? > > > > > > > > > > The intension of the 480 octets was to provide a known message >=20 > > > > > size that every implementation actually MUST support > > > > > - so if an applications intends to send a message that MUST be >=20 > > > > > received fully, that application knows a maximum size. > > > > > > > > > > If you mean that with the upcoming version of > > -transport-udp the > > > > > 480 octet can safely be replaced by 1K or 2K then we of > > course can > > > > > change this. > > > > > > > > > > I, however, think that a minimal burden should be placed on an >=20 > > > > > implementation. I would tend to call a syslog receiver that is >=20 > > > > > only capable of receiving 400 octets to be > > non-compliant. But, OK, > > > > > maybe we should say this is OK. So what limit is it > > then? Is 200 > > > > > octets compliant? 100? 50? > > > > > Whatever it is, I think we should require one size...=20 > > If we do not > > > > > require this, one could argue writing/parsing a correct > > header is > > > > > also too much of a burden for a given scenario... > > > > > > > > > > Rainer > > > > > > > > > > > -----Original Message----- > > > > > > From: Anton Okmianski [mailto:aokmians@cisco.com] > > > > > > Sent: Thursday, October 14, 2004 4:36 PM > > > > > > To: ietfdbh@comcast.net; Rainer Gerhards;=20 > > > > > > syslog-sec@employees.org > > > > > > Subject: RE: [Syslog-sec] RE: Maximum message size > > > > > > > > > > > > Rainer: > > > > > > > > > > > > I agree with a SHOULD on 64k support on receivers and MAY on > > > > larger > > > > > > sizes. > > > > > > > > > > > > I think we need to remove or change this sentence: "A > > > > > receiver MUST be > > > > > > able to accept messages up to and including 480 > > octets in length". > > > > > > > > > > > > 1. Is this an absolute requirement? We keep thinking about > > > > general > > > > > > purpose receiver libraries. But it does not have to > > be general > > > > > > purpose. If I have two specific applications with their own > > > > syslog > > > > > > sender/receiver implementations (how hard is it to > > > > > send/receive a udp > > > > > > message) and they exchange syslog messages which are > > known not > > > > > > to exceed 400 bytes, should my receiver be required > > to support > > > > > > 8k messages? > > > > > > > > > > > > 2. Why 480? If this is intended to ensure basic=20 > > > > > > interoperability, > > > > > > > > > > then at least this should be a size greater than 1K (size=20 > > > > > > previous > > > > > > > > > > syslog RFC observed). Maybe 4k or 8k. At least it > > is a better > > > > > > arbitrary number. > > > > > > > > > > > > So, I would support either removing the MUST requirement > > > > > all together > > > > > > or at least changing it to a more reasonable size. The > > > > > latter approach > > > > > > would be more limiting for implementations, but will insure=20 > > > > > > interoperability for general-purpose syslog libraries if > > > > > this is the > > > > > > overriding goal. > > > > > > > > > > > > Thanks, > > > > > > Anton. > > > > > > > > > > > > > -----Original Message----- > > > > > > > From: syslog-sec-bounces@willers.employees.org > > > > > > > [mailto:syslog-sec-bounces@willers.employees.org] On > > > > > Behalf Of David > > > > > > > B Harrington > > > > > > > Sent: Thursday, October 14, 2004 8:53 AM > > > > > > > To: 'Rainer Gerhards'; syslog-sec@employees.org > > > > > > > Subject: RE: [Syslog-sec] RE: Maximum message size > > > > > > > > > > > > > > Hi Rainer, > > > > > > > > > > > > > > Thanks for making the changes. > > > > > > > > > > > > > > I'd like to make one point a little more strenuously: > > > > > > > "For interoperability reasons, all receiver > > > > > implementations SHOULD > > > > > > > be able to accept messages up to and including > > 65,535 octets > > > > > > > in length." > > > > > > > > > > > > > > Also, I think the last two sentences could be combined > into=20 > > > > > > > one > > > > - > > > > > > > "If a receiver receives a message with a length larger > > > > > than 65,535 > > > > > > > octets, or larger than it supports, the receiver > > MAY discard > > > > > > > the > > > > > > > > > > > message or truncate the payload." > > > > > > > > > > > > > > dbh > > > > > > > > > > > > > > -----Original Message----- > > > > > > > From: syslog-sec-bounces@www.employees.org > > > > > > > [mailto:syslog-sec-bounces@www.employees.org] On Behalf Of > > > > Rainer > > > > > > > Gerhards > > > > > > > Sent: Thursday, October 14, 2004 7:34 AM > > > > > > > To: syslog-sec@employees.org > > > > > > > Subject: [Syslog-sec] RE: Maximum message size > > > > > > > > > > > > > > David and all, > > > > > > > > > > > > > > many thanks for all the good comments. > > > > > > > > > > > > > > I agree that 16 MB for a single message is far more than > > > > actually > > > > > > > should be used. Let's look a bit at the history of this: > > > > > > > > > > > > > > Initially, we noticed that the 1024 octet limit for > > > > > current syslog > > > > > > > is too short. I think there is still general agreement on > > > > > this. We > > > > > > > had a good discussion about the size issue in September > > > > > 2003. A link > > > > > > > to an archive is here: > > > > > > > > > > > > > > http://www.syslog.cc/ietf/autoarc/msg00855.html > > > > > > > > > > > > > > The general issue is that 1K is too short for many > > > > > applications. For > > > > > > > example, Marshall Glen mentions the healthcare needs > > > > > (RFC3881) for a > > > > > > > larger message. As far as I know, the IHE > > initiative currently > > > > > > > standardizes on a maximum XML stream size of 32K (which is >=20 > > > > > > > then supposed to be transmitted via syslog). I=20 > > think a message > > > > > size limit > > > > > > > of 64K would probably be suffcient for a long time to come >=20 > > > > > > > (and leaves some headroom for message content increases). > > > > > > > > > > > > > > Besides that, I myself identified a need - in very rare > > > > > occasions - > > > > > > > to send information larger than 64K. In extreme cases, > this=20 > > > > > > > information could be as large as one megabyte. > > > > > > > Again, these were cases that might happen once in a=20 > > year. All=20 > > > > > > > of > > > > > > > > > > > this, of course, only if an operator > > > > > > > *really* wants to submit that large information. So the=20 > > > > > > > initial approach and suggestion was to introduce a way to=20 > > > > > > > build "message > > > > > > > > > > > groups". That would have been a way to split "oversize > > > > > messages" (as > > > > > > > they were called > > > > > > > then) into multiple syslog messages. They key to=20 > > that concept > > > > was > > > > > > > that each single message could be kept within a > > > > > reasonable limit (it > > > > > > > would have worked even with 1K max message size). It > > > > > would only have > > > > > > > been a way to transmit very large messages in those very=20 > > > > > > > seldom cases that they would actually be used. > > > > > > > > > > > > > > Some time later, we came to the conclusion that it=20 > > should be=20 > > > > > > > possible for a single syslog message to be of the largest > > > > > imaginable > > > > > > > size. It was argued that fragmentation is a transport > > > > > issue and as > > > > > > > such application-level "fragmentation" > > > > > > > (the message groups) would not do good. We agreed to that. > > > > > > > > > > > > > > Even some time later, we needed to define a maximum > message > > > > size, > > > > > > > just to limit it somewhere (if I recall correctly,=20 > > that was a=20 > > > > > > > transport requirement). We picked 16MB just as a=20 > > number that=20 > > > > > > > we thought would never have be reached and such be > > > > > sufficient for all > > > > > > > imaginable time to come. As far as I was concerend, that > > > > decision > > > > > > > was driven by the fact that in the past there were often > > > > > limits that > > > > > > > nobody expected to reach - and yet they wer quickly > reached. > > > > > > > Som 16MB was picked more or less as "larger than anybody > > > > > will need > > > > > > > anytime in the forseeable future". We did not even=20 > > have a big=20 > > > > > > > discussion on that limit. > > > > > > > > > > > > > > I think this are the key points on how we reached=20 > > this limit. > > > > > > > Now, thanks to your call and looking the whole situation, > > > > > I really > > > > > > > think we've gone overboard. That 16MB limit is something=20 > > > > > > > nobody intends to use. Even a lower - but high limit - is=20 > > > > > > > extremely unlikely. I'd expect to see a 1MB=20 > > "message" in maybe > > > > > 0.0001% of the > > > > > > > cases - if at all (but I expect to see e.g. 70KB > > > > > "message" in maybe > > > > > > > 0.001% of the cases). > > > > > > > > > > > > > > Also, I have read your proposal to set a maxium=20 > > size for the=20 > > > > > > > message, but allow an implementation to go beyond=20 > > that. Maybe=20 > > > > > > > something like > > > > > > > this: > > > > > > > > > > > > > > #### > > > > > > > 5.1 Message Length > > > > > > > > > > > > > > A receiver MUST be able to accept messages up to and > > > > including > > > > > > > 480 octets in length. > > > > > > > > > > > > > > A receiver SHOULD be able to accept messages up to and > > > > > including > > > > > > > 65,535 octets in length. > > > > > > > If a receiver receives a message with a length=20 > > larger than=20 > > > > > > > it > > > > > > > > > > > supports, the receiver MAY > > > > > > > discard the message or truncate the payload. > > > > > > > > > > > > > > A receiver MAY accept messages larger than 65,535 > octets. > > > > > > > #### > > > > > > > > > > > > > > I've dropped the wording that transports MAY support only > > > > > a smaller > > > > > > > max size - I think UDP was the most critical in this > > > > > regard and it > > > > > > > obviously handles 64K. So I assume this clause is not > > > > > really needed > > > > > > > if we go for 64K as the max size. > > > > > > > > > > > > > > With that clause, we would limit the size for all interop > > > > > cases, but > > > > > > > leave a window open so that the message length could be > > > > > extended if > > > > > > > there is a real need AND the operator and software vendor=20 > > > > > > > wants this. > > > > > > > I assume nobody will implement this burden if there is no=20 > > > > > > > market > > > > > > > > > > > need to do so. But if it is, it clould at least be > > > > > implemented in a > > > > > > > standards-compliant way. > > > > > > > > > > > > > > If I look back at the original requirement -=20 > > provide a way to=20 > > > > > > > transmit very rare oversize messages, I think we can > safely=20 > > > > > > > implement this the bounds of the current proposals, even > > > > > if the size > > > > > > > limit does not go beyond 64K. If somebody needs it,=20 > > this can=20 > > > > > > > be easily done via an experimental (or > > > > > > > vendor-specific) SD-ID. Again, I do not think somebody > > > > > will accept > > > > > > > this burden without market need. > > > > > > > > > > > > > > In the light of this, I, too, would recommend that we > limit=20 > > > > > > > the message size to 64K. I would prefer, however,=20 > > that we keep=20 > > > > > > > the maximum open as in the suggested wording above. > > > > > > > > > > > > > > Rainer > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > From: David B Harrington [mailto:ietfdbh@comcast.net] > > > > > > > > Sent: Wednesday, October 13, 2004 5:10 PM > > > > > > > > To: Rainer Gerhards; 'Anton Okmianski'; > > > > syslog-sec@employees.org > > > > > > > > Subject: Maximum message size > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > I am concerned that we are in danger of promoting uses > of > > > > syslog > > > > > > > that > > > > > > > > will defeat its current usefulness as a=20 > > human-readable log=20 > > > > > > > > of > > > > > > system > > > > > > > > > > > > > > > activity. > > > > > > > > > > > > > > > > Do network operators really believe the 16,777,216 > > > > > octet size is a > > > > > > > > > > > > > > needed improvement to syslog, or are we designing=20 > > this size=20 > > > > > > > > in > > > > > > > > > > > > explicitly for the vendors of applications who want to > > > > > use syslog > > > > > > as > > > > > > > a > > > > > > > > programmatic interface? I believe Syslog should=20 > > be designed=20 > > > > > > > > to > > > > > > meet > > > > > > > > the needs of operators. Most of the discussion ssems to > be > > > > from > > > > > > > > vendors of syslog receivers or applications. Other > > > > > protocols such > > > > > > as > > > > > > > > > > > > > > > FTP might be better used for such a specialty use case. > > > > > > > > > > > > > > > > Does allowing messages of 16,777,216 octets to be=20 > > > > > > > > accumulated > > > > > > within > > > > > > > > > > > > > > > the system log make it harder to use some > > > > > widely-available minimal > > > > > > > > > > > > > > text editors, like Notepad, to view the system log? Will > > > > having > > > > > > huge > > > > > > > > > > > > > > > application-specific messages in the system log make it > > > > > harder for > > > > > > > an > > > > > > > > operator to locate useful troubleshooting messages > within=20 > > > > > > > > the > > > > > > system > > > > > > > > > > > > > > > log? Can the information in such a huge message=20 > > fit within=20 > > > > > > > > an > > > > > > 80x25 > > > > > > > > screen, when an operator is troubleshooting=20 > > network problems > > > > and > > > > > > > needs > > > > > > > > to use a bare-bones terminal attached to the=20 > > serial port of=20 > > > > > > > > a > > > > > > device > > > > > > > > > > > > > > > to view the system log? > > > > > > > > > > > > > > > > Syslog was designing to be an operator's tool, and > Syslog > > > > should > > > > > > be > > > > > > > > designed to meet the needs of operators. Will allowing > > > > messages > > > > > > of > > > > > > > > this size negatively impact its usefulness in the > primary=20 > > > > > > > > use > > > > > > case? > > > > > > > > > > > > > > > > What do network **operators** think about the need for > > > > > messages of > > > > > > > > > > > > > > this size? Have we deliberately asked them their=20 > > opinion on > > > > this > > > > > > by, > > > > > > > > > > > > > > > for example, taking a survey or going to NANOG to=20 > > ask them? > > > > > > > > > > > > > > > > I don't think we should support such large messages in > > > > > the system > > > > > > > > logging protocol. > > > > > > > > > > > > > > > > > > > > > > > > David Harrington > > > > > > > > dbharrington@comcast.net > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > From: syslog-sec-bounces@www.employees.org > > > > > > > > [mailto:syslog-sec-bounces@www.employees.org] On Behalf > > > > > Of Rainer > > > > > > > > Gerhards > > > > > > > > Sent: Wednesday, October 13, 2004 5:17 AM > > > > > > > > To: Anton Okmianski; syslog-sec@employees.org > > > > > > > > Subject: RE: [Syslog-sec] UDP message size issue > proposal > > > > > > > > > > > > > > > > Anton, > > > > > > > > > > > > > > > > I agree to your conclusion. > > > > > > > > > > > > > > > > Although I am the one who always voted for larger=20 > > sizes, the=20 > > > > > > > > discussion has shown that this yields to unnecessary > > > > > complexity. I > > > > > > > can > > > > > > > > satisfy my needs with a vendor-extension structured > > > > > data element, > > > > > > > > which I think shows the flexibility of the new drafts. > > > > > > > > > > > > > > > > So I support your move. I would tend to even remove the > > > > > transport > > > > > > > > version header. If there is need to, we could always > > > > > include that > > > > > > in > > > > > > > a > > > > > > > > later release (e.g. "v1", which would make it clearly > > > > > > distingshable > > > > > > > > from the current frame format). I see little need=20 > > to do so, > > > > > > though. > > > > > > > > > > > > > > > > Regarding -protocol, I think we should still keep an > upper > > > > limit > > > > > > in > > > > > > > > the spec. Even I don't see any reason why a syslog > > > > > message should > > > > > > go > > > > > > > > > > > > > > > above 16MB. So for -protocol, I propose the following > new > > > > text: > > > > > > > > > > > > > > > > #### > > > > > > > > 5.1 Message Length > > > > > > > > > > > > > > > > The maximum length of any syslog message is > > > > > 16,777,216 octets. > > > > > > > Any > > > > > > > > receiver receiving a larger message MUST discard the > > > > message. > > > > > > A > > > > > > > > diagnostic message SHOULD be logged in this case. > > > > > > > > > > > > > > > > A receiver MUST be able to accept messages that are > > > > > 480 octets > > > > > > > (or > > > > > > > > less) in length. A receiver SHOULD be able to > > > > > accept messages > > > > > > > that > > > > > > > > are 65,535 octets (or less) in length. It is > > > > > RECOMMENDED that > > > > > > > > receivers be able to accept messages up to the > > > > > maximum message > > > > > > > > length > > > > > > > > of 16,777,216 octets. > > > > > > > > > > > > > > > > If a receiver receives a message within the maximum > > > > > length, but > > > > > > > > > > > > > > with > > > > > > > > a length larger than it handles, the receiver=20 > > MAY discard > > > > or > > > > > > > > truncate > > > > > > > > it. > > > > > > > > > > > > > > > > A transport mapping MAY define a maximum=20 > > length below the > > > > one > > > > > > > > specified here. Each transport mapping MUST support > > > > > a maximum > > > > > > > size > > > > > > > > of 480 octets or more. > > > > > > > > #### > > > > > > > > > > > > > > > > If there is agreement on this, I can post a new version > of > > > > > > > -protocol. > > > > > > > > That version will also include some changes made thanks > > > > > to Andrew > > > > > > > Ross > > > > > > > > comments (sent via private mail). I have finished this > > > > > version. It > > > > > > > is > > > > > > > > available for immediate posting, but I would like to > wait=20 > > > > > > > > for agreement on the size issue (at least if that can be >=20 > > > > > > > > reached > > > > > > > quickly). > > > > > > > > > > > > > > > > Rainer > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > > From: syslog-sec-bounces@www.employees.org > > > > > > > > > [mailto:syslog-sec-bounces@www.employees.org] On > > > > > Behalf Of Anton > > > > > > > > > > > > > > > Okmianski > > > > > > > > > Sent: Tuesday, October 12, 2004 11:25 PM > > > > > > > > > To: syslog-sec@employees.org > > > > > > > > > Subject: [Syslog-sec] UDP message size issue proposal > > > > > > > > > > > > > > > > > > Hi! > > > > > > > > > > > > > > > > > > Sorry for a long delay on this issue - I was on a 2 > week > > > > > > vacation. > > > > > > > > I > > > > > > > > > have spoken with a number of TCP/UDP/IP experts=20 > > regarding > > > > the > > > > > > > sizing > > > > > > > > > > > > > > > > > issue. I am ready to propose the following changes: > > > > > > > > > > > > > > > > > > 1. Syslog-protocol will state that the max=20 > > message will be > > > > > > defined > > > > > > > > by > > > > > > > > > the transport layer. > > > > > > > > > > > > > > > > > > 2. Syslog-transport-udp will support messages up to > > > > > maximum UDP > > > > > > > > > datagram size of 64K. UDP is a very bad choice for > > > > > large message > > > > > > > > > > > > > > > transmissions, so it does not make sense for us to > > > > > stretch it by > > > > > > > > > > > > > > > adding our own fragmentation without other > > > > > transmission control > > > > > > > > > features such as found in TCP. > > > > > > > > > > > > > > > > > > 3. Syslog-transport-udp will rely on IP=20 > > fragmentation and=20 > > > > > > > > > we > > > > > > will > > > > > > > > get > > > > > > > > > rid of "proprietary" fragmentation which was designed > > > > > to handle > > > > > > > > > messages over 64K and deal with various non-compliant=20 > > > > > > > > > network hosts. > > > > > > > > > > > > > > > > > > 4. Syslog-transport-udp will recommend sending > messages > > > > within > > > > > > the > > > > > > > > > > > > > > > > boundaries of prevalent MTU size on a given network > > > > > to be safe. > > > > > > It > > > > > > > > > > > > > > > > will recommend Ethernet's 1500 bytes less headers and > > > > > will draw > > > > > > > > reader > > > > > > > > > attention to the minimum MTU size hosts on the=20 > > network are > > > > > > > required > > > > > > > > to > > > > > > > > > support for > > > > > > > > > IPv6 (576 bytes) and IPv6 (1280 bytes). > > > > > > > > > > > > > > > > > > 5. Path MTU discovery may not work robustly and some=20 > > > > > > > > > TCP/IP > > > > > > stacks > > > > > > > > may > > > > > > > > > not support UDP packets of full 64K size and truncate > > > > > them. We > > > > > > > will > > > > > > > > > > > > > > > > > mention this and bite this bullet. > > > > > > > > > We should not restrict the protocol due to incompliant > > > > > > > > implementations > > > > > > > > > because it is a moving target and penalizes compliant > > > > > > > > implementations > > > > > > > > > with extra overhead. The above size recommendation > would > > > > > > partially > > > > > > > > > > > > > > > > deal with this, but leave the final choice up to the=20 > > > > > > > > > administrator. > > > > > > > > > > > > > > > > > > 6. We will get rid of most syslog transport headers > for=20 > > > > > > > > > UDP > > > > as > > > > > > > they > > > > > > > > > will no longer be needed. The only thing that will be >=20 > > > > > > > > > left > > > > is > > > > > > the > > > > > > > > > > > > > > > > transport protocol version prefixed to every=20 > > syslog message. > > > > > > > Should > > > > > > > > > > > > > > > > > we even bother with that? > > > > > > > > > > > > > > > > > > This is a major change to the syslog-transport-udp. > > > > > I'd like to > > > > > > > get > > > > > > > > > > > > > > > > > positive feedback before I proceed with this update. > > > > > > > > > The best part is that if we all agree on the above, > the=20 > > > > > > > > > next > > > > > > draft > > > > > > > > > > > > > > > > will be 1/3 of the size -- easier read for you. :) > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > Anton. > > > > > > > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > > > > > > > > Syslog-sec mailing list > > > > > > > > > Syslog-sec@www.employees.org=20 > > > > > > > > > http://www.employees.org/mailman/listinfo/syslog-sec > > > > > > > > > > > > > > > > > _______________________________________________ > > > > > > > > Syslog-sec mailing list > > > > > > > > Syslog-sec@www.employees.org=20 > > > > > > > > http://www.employees.org/mailman/listinfo/syslog-sec > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > > > > > > Syslog-sec mailing list > > > > > > > Syslog-sec@www.employees.org > > > > > > > http://www.employees.org/mailman/listinfo/syslog-sec > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > > > > > > Syslog-sec mailing list > > > > > > > Syslog-sec@www.employees.org > > > > > > > http://www.employees.org/mailman/listinfo/syslog-sec > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > > Syslog-sec mailing list > > > Syslog-sec@www.employees.org > > > http://www.employees.org/mailman/listinfo/syslog-sec > > > > > _______________________________________________ > > Syslog-sec mailing list > > Syslog-sec@www.employees.org > > http://www.employees.org/mailman/listinfo/syslog-sec > >=20 >=20 > _______________________________________________ > Syslog-sec mailing list > Syslog-sec@www.employees.org > http://www.employees.org/mailman/listinfo/syslog-sec >=20 >=20 > _______________________________________________ > Syslog-sec mailing list > Syslog-sec@www.employees.org > http://www.employees.org/mailman/listinfo/syslog-sec >=20 _______________________________________________ Syslog-sec mailing list Syslog-sec@www.employees.org http://www.employees.org/mailman/listinfo/syslog-sec From syslog-sec-bounces@willers.employees.org Fri Oct 22 16:55:48 2004 Received: from willers.employees.org (willers.employees.org [192.83.249.36]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA14212 for ; Fri, 22 Oct 2004 16:55:47 -0400 (EDT) Received: from willers.employees.org (localhost.employees.org [127.0.0.1]) by willers.employees.org (Postfix) with ESMTP id 10D8B5C786; Fri, 22 Oct 2004 13:55:46 -0700 (PDT) X-Original-To: syslog-sec@employees.org Delivered-To: syslog-sec@employees.org Received: from ietf.org (odin.ietf.org [132.151.1.176]) by willers.employees.org (Postfix) with ESMTP id DF9495C720 for ; Fri, 22 Oct 2004 13:12:44 -0700 (PDT) Received: from CNRI.Reston.VA.US (localhost [127.0.0.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA06529; Fri, 22 Oct 2004 16:12:43 -0400 (EDT) Message-Id: <200410222012.QAA06529@ietf.org> Mime-Version: 1.0 Content-Type: Multipart/Mixed; Boundary="NextPart" To: i-d-announce@ietf.org From: Internet-Drafts@ietf.org Date: Fri, 22 Oct 2004 16:12:42 -0400 X-Mailman-Approved-At: Fri, 22 Oct 2004 13:55:44 -0700 Cc: syslog-sec@employees.org Subject: [Syslog-sec] I-D ACTION:draft-ietf-syslog-protocol-07.txt X-BeenThere: syslog-sec@www.employees.org X-Mailman-Version: 2.1.4 Precedence: list List-Id: syslog-sec.www.employees.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: syslog-sec-bounces@willers.employees.org Errors-To: syslog-sec-bounces@willers.employees.org --NextPart A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the Security Issues in Network Event Logging Working Group of the IETF. Title : The syslog Protocol Author(s) : R. Gerhards Filename : draft-ietf-syslog-protocol-07.txt Pages : 38 Date : 2004-10-22 This document describes the syslog protocol which is used to convey event notification messages. It describes a layered architecture for an easily extensible syslog protocol. It also describes the basic message format and structured elements used to provide meta-information about the message. A URL for this Internet-Draft is: http://www.ietf.org/internet-drafts/draft-ietf-syslog-protocol-07.txt To remove yourself from the I-D Announcement list, send a message to i-d-announce-request@ietf.org with the word unsubscribe in the body of the message. You can also visit https://www1.ietf.org/mailman/listinfo/I-D-announce to change your subscription settings. Internet-Drafts are also available by anonymous FTP. Login with the username "anonymous" and a password of your e-mail address. After logging in, type "cd internet-drafts" and then "get draft-ietf-syslog-protocol-07.txt". A list of Internet-Drafts directories can be found in http://www.ietf.org/shadow.html or ftp://ftp.ietf.org/ietf/1shadow-sites.txt Internet-Drafts can also be obtained by e-mail. Send a message to: mailserv@ietf.org. In the body type: "FILE /internet-drafts/draft-ietf-syslog-protocol-07.txt". NOTE: The mail server at ietf.org can return the document in MIME-encoded form by using the "mpack" utility. To use this feature, insert the command "ENCODING mime" before the "FILE" command. To decode the response(s), you will need "munpack" or a MIME-compliant mail reader. Different MIME-compliant mail readers exhibit different behavior, especially when dealing with "multipart" MIME messages (i.e. documents which have been split up into multiple messages), so check your local documentation on how to manipulate these messages. Below is the data which will enable a MIME compliant mail reader implementation to automatically retrieve the ASCII version of the Internet-Draft. --NextPart Content-Type: Multipart/Alternative; Boundary="OtherAccess" --OtherAccess Content-Type: Message/External-body; access-type="mail-server"; server="mailserv@ietf.org" Content-Type: text/plain Content-ID: <2004-10-22162047.I-D@ietf.org> ENCODING mime FILE /internet-drafts/draft-ietf-syslog-protocol-07.txt --OtherAccess Content-Type: Message/External-body; name="draft-ietf-syslog-protocol-07.txt"; site="ftp.ietf.org"; access-type="anon-ftp"; directory="internet-drafts" Content-Type: text/plain Content-ID: <2004-10-22162047.I-D@ietf.org> --OtherAccess-- --NextPart Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline Content-Transfer-Encoding: 7bit _______________________________________________ Syslog-sec mailing list Syslog-sec@www.employees.org http://www.employees.org/mailman/listinfo/syslog-sec --NextPart-- From syslog-sec-bounces@willers.employees.org Thu Oct 28 22:18:18 2004 Received: from willers.employees.org (willers.employees.org [192.83.249.36]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id WAA22666 for ; Thu, 28 Oct 2004 22:18:17 -0400 (EDT) Received: from willers.employees.org (localhost.employees.org [127.0.0.1]) by willers.employees.org (Postfix) with ESMTP id 7E6DE5C730; Thu, 28 Oct 2004 19:18:17 -0700 (PDT) X-Original-To: syslog-sec@employees.org Delivered-To: syslog-sec@employees.org Received: from sj-iport-2.cisco.com (sj-iport-2-in.cisco.com [171.71.176.71]) by willers.employees.org (Postfix) with ESMTP id 7DB0F5C73B for ; Thu, 28 Oct 2004 15:17:12 -0700 (PDT) Received: from sj-core-5.cisco.com (171.71.177.238) by sj-iport-2.cisco.com with ESMTP; 28 Oct 2004 15:26:32 -0700 Received: from flask.cisco.com (IDENT:mirapoint@flask.cisco.com [161.44.122.62]) by sj-core-5.cisco.com (8.12.10/8.12.6) with ESMTP id i9SMHB6s024467 for ; Thu, 28 Oct 2004 15:17:11 -0700 (PDT) Received: from aokmiansw2k ([161.44.64.231]) by flask.cisco.com (MOS 3.4.6-GR) with ESMTP id AMQ46650; Thu, 28 Oct 2004 18:17:09 -0400 (EDT) Message-Id: <200410282217.AMQ46650@flask.cisco.com> From: "Anton Okmianski" To: Date: Thu, 28 Oct 2004 18:17:09 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Office Outlook, Build 11.0.5510 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4939.300 Thread-Index: AcS9O91rXdb7DIMDShGfsHZnUv5zLg== X-Mailman-Approved-At: Thu, 28 Oct 2004 19:18:16 -0700 Subject: [Syslog-sec] Required transport X-BeenThere: syslog-sec@www.employees.org X-Mailman-Version: 2.1.4 Precedence: list List-Id: syslog-sec.www.employees.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: syslog-sec-bounces@willers.employees.org Errors-To: syslog-sec-bounces@willers.employees.org Content-Transfer-Encoding: quoted-printable Hi! I am doing final edits to the new revision of udp transport draft. I = want to return to one issue we touched upon before. We have stated this = in the past:=20 "This transport protocol is REQUIRED for all syslog protocol = implementations." I am not 100% comfortable with this. I am looking at syslog as a = protocol which is often embedded in the application, not general purpose = re-usable syslog libraries which some people have in mind. If my = application only want to send syslog messages via some TCP mapping, why = should it be required to implement the UDP part which it does not intend = to use? Same thing on the receiver. I understand the use case of general = purpose syslog receiver. However, if I have special purpose application = to listen to faults from a given other application, why must I implement = udp mapping? =20 I have indeed worked on applications which fall into the above two = scenarios. The sending one did indeed use UDP. But if we have chosen = TCP, I don't see how the UDP transport requirement would have applied to = us. =20 Any sentiments against striking this requirement? Thanks, Anton.=20 _______________________________________________ Syslog-sec mailing list Syslog-sec@www.employees.org http://www.employees.org/mailman/listinfo/syslog-sec From syslog-sec-bounces@willers.employees.org Fri Oct 29 13:44:13 2004 Received: from willers.employees.org (willers.employees.org [192.83.249.36]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA18719 for ; Fri, 29 Oct 2004 13:44:13 -0400 (EDT) Received: from willers.employees.org (localhost.employees.org [127.0.0.1]) by willers.employees.org (Postfix) with ESMTP id 8C47C5C72B; Fri, 29 Oct 2004 10:44:12 -0700 (PDT) X-Original-To: syslog-sec@employees.org Delivered-To: syslog-sec@employees.org Received: from sccrmhc11.comcast.net (sccrmhc11.comcast.net [204.127.202.55]) by willers.employees.org (Postfix) with ESMTP id E2D2E5C766 for ; Fri, 29 Oct 2004 07:45:37 -0700 (PDT) Received: from djyxpy41 (h00104b8ce2a3.ne.client2.attbi.com[24.128.104.220]) by comcast.net (sccrmhc11) with SMTP id <200410291445360110030pd3e>; Fri, 29 Oct 2004 14:45:36 +0000 From: "David B Harrington" To: "'Anton Okmianski'" , Subject: RE: [Syslog-sec] Required transport Date: Fri, 29 Oct 2004 10:45:39 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook, Build 11.0.5510 Thread-Index: AcS9O91rXdb7DIMDShGfsHZnUv5zLgAgk/4w X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1441 In-Reply-To: <200410282217.AMQ46650@flask.cisco.com> Message-Id: <20041029144537.E2D2E5C766@willers.employees.org> X-Mailman-Approved-At: Fri, 29 Oct 2004 10:44:11 -0700 X-BeenThere: syslog-sec@www.employees.org X-Mailman-Version: 2.1.4 Precedence: list Reply-To: ietfdbh@comcast.net List-Id: syslog-sec.www.employees.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: syslog-sec-bounces@willers.employees.org Errors-To: syslog-sec-bounces@willers.employees.org Content-Transfer-Encoding: 7bit Hi, We are in the business of defining standards to ensure **interoperability**. If we do not choose a transport that is required for compliance, then two implementations compliant to the standard, where one uses UDP and the other uses TCP, would not be interoperable. The goal of this group is to define a standard set of features to be included in implementations to guarantee interoperability between **any** two independent implementations compliant to this standard. In the scenario you discuss, the implemention is for a "special purpose application". The implementor doesn't care about interoperability with **any** other implementations, only other **specific-application** implementations, so there is a different set of interoperability requirements for that special application (TCP, special messages, etc.). The implementor does NOT need to develop to the IETF standard for general-purpose syslog interoperability, and therefore doesn't need to implement UDP support. They just cannot claim compliance to the IETF standard if they don't. But that should not be very important because their main claim is that they provide the special-application functionality. Our job is not to develop standards that fit the architectures of specific-purpose applications so marketing departments can claim IETF standards-compliance, it is to develop a minimum set of compliance rules to ensure interoperability between independent implementations. Selecting one transport and making it a requirement is an important part of the standard. David Harrington dbharrington@comcast.net -----Original Message----- From: syslog-sec-bounces@www.employees.org [mailto:syslog-sec-bounces@www.employees.org] On Behalf Of Anton Okmianski Sent: Thursday, October 28, 2004 6:17 PM To: syslog-sec@employees.org Subject: [Syslog-sec] Required transport Hi! I am doing final edits to the new revision of udp transport draft. I want to return to one issue we touched upon before. We have stated this in the past: "This transport protocol is REQUIRED for all syslog protocol implementations." I am not 100% comfortable with this. I am looking at syslog as a protocol which is often embedded in the application, not general purpose re-usable syslog libraries which some people have in mind. If my application only want to send syslog messages via some TCP mapping, why should it be required to implement the UDP part which it does not intend to use? Same thing on the receiver. I understand the use case of general purpose syslog receiver. However, if I have special purpose application to listen to faults from a given other application, why must I implement udp mapping? I have indeed worked on applications which fall into the above two scenarios. The sending one did indeed use UDP. But if we have chosen TCP, I don't see how the UDP transport requirement would have applied to us. Any sentiments against striking this requirement? Thanks, Anton. _______________________________________________ Syslog-sec mailing list Syslog-sec@www.employees.org http://www.employees.org/mailman/listinfo/syslog-sec _______________________________________________ Syslog-sec mailing list Syslog-sec@www.employees.org http://www.employees.org/mailman/listinfo/syslog-sec From syslog-sec-bounces@willers.employees.org Fri Oct 29 14:13:55 2004 Received: from willers.employees.org (willers.employees.org [192.83.249.36]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA20963 for ; Fri, 29 Oct 2004 14:13:54 -0400 (EDT) Received: from willers.employees.org (localhost.employees.org [127.0.0.1]) by willers.employees.org (Postfix) with ESMTP id BBE2C5C7BC; Fri, 29 Oct 2004 11:13:54 -0700 (PDT) X-Original-To: syslog-sec@employees.org Delivered-To: syslog-sec@employees.org Received: from rtp-iport-1.cisco.com (rtp-iport-1.cisco.com [64.102.122.148]) by willers.employees.org (Postfix) with ESMTP id E1A3B5C7B8 for ; Fri, 29 Oct 2004 11:01:52 -0700 (PDT) Received: from rtp-core-2.cisco.com (64.102.124.13) by rtp-iport-1.cisco.com with ESMTP; 29 Oct 2004 14:24:39 -0400 X-BrightmailFiltered: true Received: from flask.cisco.com (IDENT:mirapoint@flask.cisco.com [161.44.122.62]) by rtp-core-2.cisco.com (8.12.10/8.12.6) with ESMTP id i9TI1mg3017002; Fri, 29 Oct 2004 14:01:51 -0400 (EDT) Received: from aokmiansw2k ([161.44.64.231]) by flask.cisco.com (MOS 3.4.6-GR) with ESMTP id AMR02820; Fri, 29 Oct 2004 14:01:43 -0400 (EDT) Message-Id: <200410291801.AMR02820@flask.cisco.com> From: "Anton Okmianski" To: , Subject: RE: [Syslog-sec] Required transport Date: Fri, 29 Oct 2004 14:01:43 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook, Build 11.0.5510 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4939.300 In-Reply-To: <3ehbbq$8rjlt@sj-inbound-e.cisco.com> Thread-Index: AcS9O91rXdb7DIMDShGfsHZnUv5zLgAgk/4wAAI2ssA= X-Mailman-Approved-At: Fri, 29 Oct 2004 11:13:53 -0700 X-BeenThere: syslog-sec@www.employees.org X-Mailman-Version: 2.1.4 Precedence: list List-Id: syslog-sec.www.employees.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: syslog-sec-bounces@willers.employees.org Errors-To: syslog-sec-bounces@willers.employees.org Content-Transfer-Encoding: 7bit David: > We are in the business of defining standards to ensure > **interoperability**. If we do not choose a transport that is > required for compliance, then two implementations compliant > to the standard, where one uses UDP and the other uses TCP, > would not be interoperable. > > The goal of this group is to define a standard set of > features to be included in implementations to guarantee > interoperability between > **any** two independent implementations compliant to this standard. I think this will still be assured. Example: RFC A - syslog protocol RFC B - udp mapping RFC C - tcp mapping If somebody says they are complaint to RFC A, it only means that they follow message format. If somebody says that they are complaint with RFC B, then it means they support UDP transport for syslog. So, I don't see any ambiguity or interoperability issues between implementation claiming to support a given RFC. Why should support for RFC C require support for RFC B? Here is a practical analogy. My mail application can retrieve email using POP3, IMAP or MS Exchange protocol. MIME, which defines mail format, does not say that you MUST use it over POP3 to ensure interoperability. It is a choice of application vendors and administrators what they will implement. This works pretty interoperably as far as I can tell since we read email from each other. And it opened the possibility for various alternatives. > In the scenario you discuss, the implemention is for a > "special purpose application". The implementor doesn't care > about interoperability with **any** other implementations, only other > **specific-application** implementations, so there is a > different set of interoperability requirements for that > special application (TCP, special messages, etc.). The > implementor does NOT need to develop to the IETF standard for > general-purpose syslog interoperability, and therefore > doesn't need to implement UDP support. They just cannot claim > compliance to the IETF standard if they don't. But that > should not be very important because their main claim is that > they provide the special-application functionality. In the case I was mentioning an application cares about interoperating with standard syslog loggers, but only with ones that support a TCP transport RFC. Very simple qualification. Obviously, administrator will need to ensure that the syslog implementation on the other side implements TCP binding RFC if they want to use an application like this. > Our job is not to develop standards that fit the > architectures of specific-purpose applications so marketing > departments can claim IETF standards-compliance, it is to > develop a minimum set of compliance rules to ensure > interoperability between independent implementations. > Selecting one transport and making it a requirement is an > important part of the standard. All applications are special-purpose. I am not talking about accommodating some strange scenario. But rather what would be a pretty standard use case where an application would want to support just one transport for reasons of reliability, for example. Initially, vendors may choose to support UDP, but as other transport mappings become prevalent, they should not be required to be stuck to legacy transport. What you are saying is that if vendor wants to use just TCP and claim compliance to RFC C, it must implement UDP and RFC B. I see absolutely no practical point in this if application does not intend on using UDP transport. I also don't see a need for featuring UDP as a preferred (ie "standard") transport, while we know all of its deficiencies. In fact, I can see a danger in this. It may drag syslog protocol back instead of allowing it to evolve to better transports. I think our goal for this effort was specifically to make syslog protocol transport-independent. Kinda like SOAP messages which can be sent over whatever transport. Mandating a UDP mapping IMO defeats this purpose, even if it provides for possibilities for optional extensions. Anton. _______________________________________________ Syslog-sec mailing list Syslog-sec@www.employees.org http://www.employees.org/mailman/listinfo/syslog-sec From syslog-sec-bounces@willers.employees.org Fri Oct 29 14:46:53 2004 Received: from willers.employees.org (willers.employees.org [192.83.249.36]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA23208 for ; Fri, 29 Oct 2004 14:46:52 -0400 (EDT) Received: from willers.employees.org (localhost.employees.org [127.0.0.1]) by willers.employees.org (Postfix) with ESMTP id 898865C79B; Fri, 29 Oct 2004 11:46:52 -0700 (PDT) X-Original-To: syslog-sec@employees.org Delivered-To: syslog-sec@employees.org Received: from sccrmhc13.comcast.net (sccrmhc13.comcast.net [204.127.202.64]) by willers.employees.org (Postfix) with ESMTP id C49995C79D for ; Fri, 29 Oct 2004 11:36:06 -0700 (PDT) Received: from djyxpy41 (h00104b8ce2a3.ne.client2.attbi.com[24.128.104.220]) by comcast.net (sccrmhc13) with SMTP id <200410291836050160015l0ee>; Fri, 29 Oct 2004 18:36:05 +0000 From: "David B Harrington" To: "'Anton Okmianski'" , Subject: RE: [Syslog-sec] Required transport Date: Fri, 29 Oct 2004 14:36:09 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook, Build 11.0.5510 Thread-Index: AcS9O91rXdb7DIMDShGfsHZnUv5zLgAgk/4wAAI2ssAABypB4A== X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1441 In-Reply-To: <200410291801.AMR02820@flask.cisco.com> Message-Id: <20041029183606.C49995C79D@willers.employees.org> X-Mailman-Approved-At: Fri, 29 Oct 2004 11:46:51 -0700 X-BeenThere: syslog-sec@www.employees.org X-Mailman-Version: 2.1.4 Precedence: list Reply-To: ietfdbh@comcast.net List-Id: syslog-sec.www.employees.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: syslog-sec-bounces@willers.employees.org Errors-To: syslog-sec-bounces@willers.employees.org Content-Transfer-Encoding: 7bit Hi Anton, I think you'll find the IESG will expect the WG to select one transport as the interoperability baseline for the standard. I could of course be wrong, so I suggest the area directors be consulted to see what expectations they have on this issue. dbh -----Original Message----- From: Anton Okmianski [mailto:aokmians@cisco.com] Sent: Friday, October 29, 2004 2:02 PM To: ietfdbh@comcast.net; syslog-sec@employees.org Subject: RE: [Syslog-sec] Required transport David: > We are in the business of defining standards to ensure > **interoperability**. If we do not choose a transport that is required > for compliance, then two implementations compliant to the standard, > where one uses UDP and the other uses TCP, would not be interoperable. > > The goal of this group is to define a standard set of features to be > included in implementations to guarantee interoperability between > **any** two independent implementations compliant to this standard. I think this will still be assured. Example: RFC A - syslog protocol RFC B - udp mapping RFC C - tcp mapping If somebody says they are complaint to RFC A, it only means that they follow message format. If somebody says that they are complaint with RFC B, then it means they support UDP transport for syslog. So, I don't see any ambiguity or interoperability issues between implementation claiming to support a given RFC. Why should support for RFC C require support for RFC B? Here is a practical analogy. My mail application can retrieve email using POP3, IMAP or MS Exchange protocol. MIME, which defines mail format, does not say that you MUST use it over POP3 to ensure interoperability. It is a choice of application vendors and administrators what they will implement. This works pretty interoperably as far as I can tell since we read email from each other. And it opened the possibility for various alternatives. > In the scenario you discuss, the implemention is for a "special > purpose application". The implementor doesn't care about > interoperability with **any** other implementations, only other > **specific-application** implementations, so there is a different set > of interoperability requirements for that special application (TCP, > special messages, etc.). The implementor does NOT need to develop to > the IETF standard for general-purpose syslog interoperability, and > therefore doesn't need to implement UDP support. They just cannot > claim compliance to the IETF standard if they don't. But that should > not be very important because their main claim is that they provide > the special-application functionality. In the case I was mentioning an application cares about interoperating with standard syslog loggers, but only with ones that support a TCP transport RFC. Very simple qualification. Obviously, administrator will need to ensure that the syslog implementation on the other side implements TCP binding RFC if they want to use an application like this. > Our job is not to develop standards that fit the architectures of > specific-purpose applications so marketing departments can claim IETF > standards-compliance, it is to develop a minimum set of compliance > rules to ensure interoperability between independent implementations. > Selecting one transport and making it a requirement is an important > part of the standard. All applications are special-purpose. I am not talking about accommodating some strange scenario. But rather what would be a pretty standard use case where an application would want to support just one transport for reasons of reliability, for example. Initially, vendors may choose to support UDP, but as other transport mappings become prevalent, they should not be required to be stuck to legacy transport. What you are saying is that if vendor wants to use just TCP and claim compliance to RFC C, it must implement UDP and RFC B. I see absolutely no practical point in this if application does not intend on using UDP transport. I also don't see a need for featuring UDP as a preferred (ie "standard") transport, while we know all of its deficiencies. In fact, I can see a danger in this. It may drag syslog protocol back instead of allowing it to evolve to better transports. I think our goal for this effort was specifically to make syslog protocol transport-independent. Kinda like SOAP messages which can be sent over whatever transport. Mandating a UDP mapping IMO defeats this purpose, even if it provides for possibilities for optional extensions. Anton. _______________________________________________ Syslog-sec mailing list Syslog-sec@www.employees.org http://www.employees.org/mailman/listinfo/syslog-sec From syslog-sec-bounces@willers.employees.org Fri Oct 29 14:49:16 2004 Received: from willers.employees.org (willers.employees.org [192.83.249.36]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA23438 for ; Fri, 29 Oct 2004 14:49:15 -0400 (EDT) Received: from willers.employees.org (localhost.employees.org [127.0.0.1]) by willers.employees.org (Postfix) with ESMTP id E2C915C7A6; Fri, 29 Oct 2004 11:49:15 -0700 (PDT) X-Original-To: syslog-sec@employees.org Delivered-To: syslog-sec@employees.org Received: from sj-iport-2.cisco.com (sj-iport-2-in.cisco.com [171.71.176.71]) by willers.employees.org (Postfix) with ESMTP id A0C4E5C76C for ; Fri, 29 Oct 2004 11:49:01 -0700 (PDT) Received: from sj-core-1.cisco.com (171.71.177.237) by sj-iport-2.cisco.com with ESMTP; 29 Oct 2004 11:58:31 -0700 Received: from edison.cisco.com (edison.cisco.com [171.71.180.109]) by sj-core-1.cisco.com (8.12.10/8.12.6) with ESMTP id i9TImuom018077; Fri, 29 Oct 2004 11:48:57 -0700 (PDT) Received: from localhost (clonvick@localhost) by edison.cisco.com (8.8.6 (PHNE_14041)/CISCO.SERVER.1.2) with ESMTP id LAA14103; Fri, 29 Oct 2004 11:48:58 -0700 (PDT) Date: Fri, 29 Oct 2004 11:48:58 -0700 (PDT) From: Chris Lonvick To: David B Harrington Subject: RE: [Syslog-sec] Required transport In-Reply-To: <20041029183606.C49995C79D@willers.employees.org> Message-ID: References: <20041029183606.C49995C79D@willers.employees.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Mailman-Approved-At: Fri, 29 Oct 2004 11:49:14 -0700 Cc: syslog-sec@employees.org X-BeenThere: syslog-sec@www.employees.org X-Mailman-Version: 2.1.4 Precedence: list List-Id: syslog-sec.www.employees.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: syslog-sec-bounces@willers.employees.org Errors-To: syslog-sec-bounces@willers.employees.org Hi, On Fri, 29 Oct 2004, David B Harrington wrote: > Hi Anton, > > I think you'll find the IESG will expect the WG to select one > transport as the interoperability baseline for the standard. I'm certain so as well. > I could of course be wrong, so I suggest the area directors be > consulted to see what expectations they have on this issue. I'll double check while at the IETF. Thanks, Chris _______________________________________________ Syslog-sec mailing list Syslog-sec@www.employees.org http://www.employees.org/mailman/listinfo/syslog-sec