Date: Tue, 2 Mar 2004 17:07:50 -0800 (PST) From: Chris Lonvick Subject: Meeting Today Hi Folks, I've updated our Agenda for today's meeting. http://www.employees.org/~lonvick/Seoul_agenda/agenda.html This contains all of the presentations and our overall purpose for meeting at this IETF. I don't mean to hog the microphone but I'll be giving the presentations for Jon, Rainer and Anton. I'm still looking for a scribe and jabberer. Please send me email to let me know if you can do either. Also, we're in Gardenia-A2. That's the room next to the Terminal Room and is easy to miss. Thanks, Chris ------------------------------ Date: Wed, 3 Mar 2004 16:02:15 -0800 (PST) From: Chris Lonvick Subject: Minutes from the syslog meeting at the 59th IETF Hi Folks, Below are the minutes from the syslog WG meeting held yesterday here at the IETF. I wish to sincerely thank Richard Graveman for these superb notes. These notes are consistent with my recollection of the meeting and if no one objects, I will post them to the IESG as official. Apologies but no one in the room had jabber on their machine so we missed that. Thanks, Chris - ---notes as taken by Richard Graveman--- Security Issues in Network Event Logging WG (syslog WG; Sec Area) Notes from Richard Graveman, RFG Security. Chair: Chris Lonvick The ML is at syslog-sec@employees.org. Additional information is available at http://www.employees.org/~lonvick/Seoul_agenda/agenda.html. Issue tracking is in use. I. Review of Scope and Charter (Chris) 10 min. There were no changes to the agenda. The agenda and on-line information were reviewed. The work is to secure the transport of logs. Content is out of scope. They produced RFCs 3164 and 3195. Protocol, UDP transport, sign, device MIB, and Internationalization drafts are in progress. II. Update of "syslog Protocol" ID (Rainer or proxy) 30 min. draft-ietf-syslog-protocol-03.txt The three layers are application, protocol, and transport. 3164 is the two lower layers. 3195 touches the application. Sign and International are mostly the two higher layers. A new concept for aligning documents within these layers was proposed, which leads to six documents instead of four. UDP and 3195; protocol, and then international and sign, etc. at the application layer. Protocol describes devices, relays, collectors, protocol layers, simplex communications, common header, structured data, classical free-form messages, and security considerations. There is no two-way communications, except for RFC 3195 which is over TCP. End-to-end (duplex) would be an entirely different protocol. Transport layer mapping: UDP being written; others, including TCP, later. The protocol ID is not completely backward compatible. Non-compliant messages will be treated as messages without a header. WG consensus that a solid base is more important than backwards compatibility. The new HEADER has more fields, including a version number. TIMESTAMP is better defined. John Kelsey introduced structured data elements. Used for extended functionality. Assigned by IANA. Extensible. The ID s defined are msgpart, time, and origin, with more to come. Time zones were debated. IP addresses are preferable to hostnames. Multi-part messaging: 1280 is the maximum message size. A message segmentation scheme exists. Downstream message splitting is proposed. Semantics questions were on facility (issue 5) and TAG (issue 16). Issue 13 states that syslog-protocol currently dictates many details. There is a tradeoff between ambiguity and security. The notes could be moved to an Informational RFC. There was no objection to this. Relay operations may be described in the -protocol document or described briefly with the rest deferred. It will be left for now. Message size was discussed: current maximum is 1280. Removing this restriction was proposed. It could become a transport issue. Comments and feedback are solicited. See http://www.syslog.cc/ietf/protocol.html. III. Introduction of "syslog Transport" ID (Anton or proxy) 15 min. draft-ietf-syslog-transport-udp-00.html http://www.employees.org/~lonvick/transport/draft-ietf-syslog-transport-udp-00.txt This is a standards track ID to replace 3164. It defines UDP over port 514 as MUST. One datagram per syslog message. One datagram per msg part for multi-part. UDP checksums RECOMMENDED. IP fragmentation SHOULD be avoided. Recommends 548 bytes maximum message size to avoid fragmentation. (S. Bellovin drew a comparison with the DNS message size limit of 512. Reliability is a separate section: lost, corrupted, congestion control, un-sequenced delivery, and increased risk of loss with IP fragmentation. Many sec considerations: authenticity, authentication, forgery, eavesdropping, replay, unreliability, no prioritization, DoS, and covert channels. There was no objection to this approach, which simplifies the syslog-protocol document. IV. Update of "syslog-sign" ID (Jon or proxy) 15 min. draft-ietf-syslog-sign-13 Slightly on hold while the protocol issues are resolved. Parts can be removed based on the above protocol and transport. IANA considerations need to be done. The current document is transport independent. It can be used with classic syslog or home-grown applications. No objection to this. V. Update of "syslog-device-mib" ID (Glenn Mansfield Keeni) 30 min. draft-ietf-syslog-device-mib-05 This started with modest ambitions and became more complicated. After more reflection, the current -05 draft emerged. The name "device" is historical. Purpose is to monitor syslog operation: message stats, received, processed, relayed. Security considerations need to address READONLY threats and other threats. The following issues are resolved: Issue 1: Written, and feedback is requested. Issue 2: Make the MIB generic (now BSD centric) Issue 3: Do not overlap with HR-MIB. provide mapping to hrSWRunTable. Remove syslogParamsProcessStatus (or make it readOnly), add syslogProcReference. The syslogProcReference OBJECT-TYPE was shown. Issue 4: Add SNMP notification. The MIB will not address syslog over SNMP. Issue 5: Add "no DNS lookup" Issue 6: Use the MIB to control and configure syslog. The suggestion was to do this with ScriptMIB. The syslog.conf semantics are not part of this, but a script can be transferred to a host, which can deal with syslog.conf. Much complexity The next draft is scheduled for April: DESCRIPTION clauses, REFERENCE clauses, editorial items. VI. Wrap-up and review of decisions made (Chris) 10 min. There were no additional items. There was no objection to the plan to proceed as described in the discussion items. Meeting adjourned at 16:34. - ---end ------------------------------ Date: Wed, 3 Mar 2004 16:09:11 -0800 (PST) From: Chris Lonvick Subject: RE: Meeting Today Hi Anton, On Wed, 3 Mar 2004, Anton Okmianski wrote: > Chris, Rainer: > > Your presentation references syslog-international as future work. Does > this still hold true? Not since the meeting was held. We've been exploring the idea of separating syslog-protocol and syslog-transport from syslog-sign. I asked (several months ago) for us to look closely at that and decide if this was a good way to go at IETF 59. The group in the room had no objection to this procedure and it appears on the mailing list as if everyone is in agreement. At this point, I can say that our official plan is to progress syslog-protocol and syslog-transport. Once those are sufficiently underway, Jon can take a look at syslog-sign and modify as necessary. We can also look closely at syslog-internation along the way and see if it can be incorporated into syslog-protocol. Thanks, Chris > > I thought by specifying Unicode and UTF8 we will already handle > internationalization in syslog-protocol. The only current exceptions I > can think of are maybe support for Unicode in structured elements (which > is easy to allow) and international DNS names which we did not > explicitly mention in -protocol. > > If these are the only things left for internationalization of -protocol, > then maybe we should just include it -protocol. This should not expand > it by more than 1-5 paragraphs, and if it saves us from doing a whole > separate ID, I think it is worth it. We can either support the > "punnycode" (ASCII encoding of Unicode DNS names used in DNS) or allow > Unicode UTF8 in HOSTNAME. > > For punnycode details see RFCs 3490, 3491, 3492 and 3454. But I am not > clear we should support punnycode. This is something that DNS supports > in order to be able to re-use existing DNS server/client implementations > which don't support Unicode. I think it would be a good idea to just > stick to Unicode and insure we meet the length requirements of > international DNS names (I don't know what they are but can find out). > > > Are there other areas of internationalization for -protocol? > > Thanks, > Anton. > > > > > -----Original Message----- > > From: owner-syslog-sec@employees.org > > [mailto:owner-syslog-sec@employees.org] On Behalf Of Chris Lonvick > > Sent: Tuesday, March 02, 2004 8:08 PM > > To: syslog-sec@employees.org > > Subject: Meeting Today > > > > > > Hi Folks, > > > > I've updated our Agenda for today's meeting. > > > http://www.employees.org/~lonvick/Seoul_agenda/agenda.html > > This contains all of the presentations and our overall purpose for > meeting at this IETF. I don't mean to hog the microphone but I'll be > giving the presentations for Jon, Rainer and Anton. > > I'm still looking for a scribe and jabberer. Please send me email to > let me know if you can do either. Also, we're in Gardenia-A2. That's > the room next to the Terminal Room and is easy to miss. > > Thanks, > Chris > > ------------------------------ Date: Wed, 3 Mar 2004 17:23:09 -0800 (PST) From: Chris Lonvick Subject: WG Session Summary (fwd) Hi Folks, Below is a request from Russ Housley (Security AD) for us to produce a brief note about the decisions and action plans resulting from our meeting. Below that is my summary. - ---------- Forwarded message ---------- Date: Sun, 29 Feb 2004 21:25:34 -0500 From: Russ Housley To: sec-chairs@ietf.org Subject: WG Session Summary Security Area WG Chairs, In the past, WG chairs have been asked to produce a 2-3 paragraph summary of what happened during their WG sessions. The ADs used these to review status, especially if we were unable to attend the whole session. Some WG chairs have suggested that it would be good to have a summary to remind those that ended up with action items from the WG meeting. For the Seoul IETF, the idea is to make the summary a bit more useful to both the ADs and the community as a whole. The intent is to be able to quickly summarize what decisions the WG made (with more detail to follow in the minutes), remind those with action items that they have action items (instead of waiting with such reminders until just before the next IETF meeting), and help keep the charter milestones more visible to the WG. What we would like to see is a fairly short (no more than a page of text) summary that focuses on important decisions made during the meeting, and what is expected to happen in the 3-4 months before the next meeting. The summary would be sent to the WG list as well as the ADs. I would like to have the summary before the SAAG session. Russ - ----my response---- Discussion and Decisions - - The WG decided that the fundamental format for syslog messages needed to be addressed in a separate document rather than being lumped into the syslog-sign ID http://www.ietf.org/internet-drafts/draft-ietf-syslog-sign-13.txt From this, Rainer Gerhards has been working on that document with much enthusiasm and progress. http://www.ietf.org/internet-drafts/draft-ietf-syslog-protocol-03.txt The consensus in the room agreed with the consensus on the mailing list so this will be our official direction. - - The WG also asked that the transport mapping of syslog be addressed in a separate document. This document will REQUIRE a mapping of syslog (as currently defined in Rainer's ID) to UDP and will leave the door open for other efforts to try their own mappings. Anton Okmianski has written a draft of this document but did not get it submitted before the cutoff date: http://www.employees.org/~lonvick/transport/draft-ietf-syslog-transport-udp-00.html The tricky part of this will be that syslog-protocol and syslog-transport-udp reference each other and will need to be submitted together. I don't really believe that this is going to be a problem. A revision of this document will probably include a reference to DTLS when that is available. The consensus in the room agreed with the consensus on the mailing list so this will be our official direction. - - When these documents are finished, it will be an easy edit to get the syslog-sign ID to conform with the syslog-protocol document. It may then be sent around to Last Call as the mechanisms defined in the ID have the general consensus of the WG. - - Glenn Mansfield polled the WG and found that the consensus was to remove the "configuration tables" from the syslog-device-mib ID http://www.ietf.org/internet-drafts/draft-ietf-syslog-device-mib-05.txt I believe that this was what Bert Wijnen was suggesting at our last meeting, and I also believe that this has consensus in the WG at this time. The room was polled with no objection to this. Actions Items - - Rainer will continue to progress syslog-protocol. - - Anton will continue to progress syslog-transport-udp. - - Jon will wait until after those documents are submitted to the IESG and will then revise syslog-sign to reflect the direction taken in syslog-sign and syslog-transport-udp. - - Glenn will continue to progress syslog-device-mib. Longer Term - - The WG asked to include a document on "internationalizing" syslog messages. Rainer started that but found that it brought up issues that needed to be addressed in the base syslog protocol. The WG concurred so it is on the back burner until the syslog-protocol becomes stable. http://www.ietf.org/internet-drafts/draft-ietf-syslog-international-00.txt - - Once we get the syslog-protocol document to a finished state, I can ask Marshall Rose and Darren New to look at revising RFC 3195. The only wildcard in that is that it is being considered as a transport for the Netconf work. I invited the chairs of that WG to speak at the upcoming syslog WG meeting if they had anything they'd like to share, but they declined. - ---end Thanks, Chris ------------------------------ Date: Wed, 3 Mar 2004 13:59:17 -0500 From: Anton Okmianski Subject: RE: Meeting Today Chris, Rainer: Your presentation references syslog-international as future work. Does this still hold true? I thought by specifying Unicode and UTF8 we will already handle internationalization in syslog-protocol. The only current exceptions I can think of are maybe support for Unicode in structured elements (which is easy to allow) and international DNS names which we did not explicitly mention in -protocol. If these are the only things left for internationalization of -protocol, then maybe we should just include it -protocol. This should not expand it by more than 1-5 paragraphs, and if it saves us from doing a whole separate ID, I think it is worth it. We can either support the "punnycode" (ASCII encoding of Unicode DNS names used in DNS) or allow Unicode UTF8 in HOSTNAME. For punnycode details see RFCs 3490, 3491, 3492 and 3454. But I am not clear we should support punnycode. This is something that DNS supports in order to be able to re-use existing DNS server/client implementations which don't support Unicode. I think it would be a good idea to just stick to Unicode and insure we meet the length requirements of international DNS names (I don't know what they are but can find out). Are there other areas of internationalization for -protocol? Thanks, Anton. > -----Original Message----- > From: owner-syslog-sec@employees.org > [mailto:owner-syslog-sec@employees.org] On Behalf Of Chris Lonvick > Sent: Tuesday, March 02, 2004 8:08 PM > To: syslog-sec@employees.org > Subject: Meeting Today > > > Hi Folks, > > I've updated our Agenda for today's meeting. > http://www.employees.org/~lonvick/Seoul_agenda/agenda.html This contains all of the presentations and our overall purpose for meeting at this IETF. I don't mean to hog the microphone but I'll be giving the presentations for Jon, Rainer and Anton. I'm still looking for a scribe and jabberer. Please send me email to let me know if you can do either. Also, we're in Gardenia-A2. That's the room next to the Terminal Room and is easy to miss. Thanks, Chris ------------------------------ Date: Thu, 4 Mar 2004 14:53:25 +0100 From: Rainer Gerhards Subject: RE: Meeting Today Actually, I tend to agree to Anton, that it is not a complicated matter now. However, we have still a good number of issues open in -protocol. Basically, protocol includes everything that is needed to support international messages, but there may be some additional things be required so that everyone is satisfied (I am not saything this is the case, I am just speculating!). I have specifically not mentioned punnycode because I don't like the idea to build on this still highly political topic. I've just said "valid DNS name" and I think that should cover punnycode. The length field - out of my mind - was a max of 255 chars, which will probably first cause DNS protocol issues than syslog. So - in short - I'd like to finish -protocol WITHOUT looking too much at internationalization. Once this is done, we could re-evaluate and see if we need to address some specifics and - if so - we could do this in - -international. I think there are great chances -international will NOT be needed. Rainer > -----Original Message----- > From: Chris Lonvick [mailto:clonvick@cisco.com] > Sent: Thursday, March 04, 2004 1:09 AM > To: Anton Okmianski > Cc: syslog-sec@employees.org > Subject: RE: Meeting Today > > Hi Anton, > > On Wed, 3 Mar 2004, Anton Okmianski wrote: > > > Chris, Rainer: > > > > Your presentation references syslog-international as future > work. Does > > this still hold true? > > Not since the meeting was held. We've been exploring the idea of > separating syslog-protocol and syslog-transport from > syslog-sign. I asked > (several months ago) for us to look closely at that and > decide if this was > a good way to go at IETF 59. The group in the room had no > objection to > this procedure and it appears on the mailing list as if everyone is in > agreement. At this point, I can say that our official plan > is to progress > syslog-protocol and syslog-transport. Once those are sufficiently > underway, Jon can take a look at syslog-sign and modify as > necessary. We > can also look closely at syslog-internation along the way and > see if it > can be incorporated into syslog-protocol. > > Thanks, > Chris > > > > > > I thought by specifying Unicode and UTF8 we will already handle > > internationalization in syslog-protocol. The only current > exceptions I > > can think of are maybe support for Unicode in structured > elements (which > > is easy to allow) and international DNS names which we did not > > explicitly mention in -protocol. > > > > If these are the only things left for internationalization > of -protocol, > > then maybe we should just include it -protocol. This should > not expand > > it by more than 1-5 paragraphs, and if it saves us from > doing a whole > > separate ID, I think it is worth it. We can either support the > > "punnycode" (ASCII encoding of Unicode DNS names used in > DNS) or allow > > Unicode UTF8 in HOSTNAME. > > > > For punnycode details see RFCs 3490, 3491, 3492 and 3454. > But I am not > > clear we should support punnycode. This is something that > DNS supports > > in order to be able to re-use existing DNS server/client > implementations > > which don't support Unicode. I think it would be a good idea to just > > stick to Unicode and insure we meet the length requirements of > > international DNS names (I don't know what they are but can > find out). > > > > > > Are there other areas of internationalization for -protocol? > > > > Thanks, > > Anton. > > > > > > > > > -----Original Message----- > > > From: owner-syslog-sec@employees.org > > > [mailto:owner-syslog-sec@employees.org] On Behalf Of Chris Lonvick > > > Sent: Tuesday, March 02, 2004 8:08 PM > > > To: syslog-sec@employees.org > > > Subject: Meeting Today > > > > > > > > > Hi Folks, > > > > > > I've updated our Agenda for today's meeting. > > > > > http://www.employees.org/~lonvick/Seoul_agenda/agenda.html > > > > This contains all of the presentations and our overall purpose for > > meeting at this IETF. I don't mean to hog the microphone > but I'll be > > giving the presentations for Jon, Rainer and Anton. > > > > I'm still looking for a scribe and jabberer. Please send > me email to > > let me know if you can do either. Also, we're in > Gardenia-A2. That's > > the room next to the Terminal Room and is easy to miss. > > > > Thanks, > > Chris > > > > > > ------------------------------ Date: Tue, 09 Mar 2004 15:58:04 -0500 From: Internet-Drafts@ietf.org Subject: I-D ACTION:draft-ietf-syslog-transport-udp-00.txt A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the Security Issues in Network Event Logging Working Group of the IETF. Title : Transmission of syslog messages over UDP Author(s) : A. Okmianski Filename : draft-ietf-syslog-transport-udp-00.txt Pages : 17 Date : 2004-3-9 This document describes the UDP transport for the syslog protocol. A URL for this Internet-Draft is: http://www.ietf.org/internet-drafts/draft-ietf-syslog-transport-udp-00.txt To remove yourself from the IETF Announcement list, send a message to ietf-announce-request with the word unsubscribe in the body of the message. Internet-Drafts are also available by anonymous FTP. Login with the username "anonymous" and a password of your e-mail address. After logging in, type "cd internet-drafts" and then "get draft-ietf-syslog-transport-udp-00.txt". A list of Internet-Drafts directories can be found in http://www.ietf.org/shadow.html or ftp://ftp.ietf.org/ietf/1shadow-sites.txt Internet-Drafts can also be obtained by e-mail. Send a message to: mailserv@ietf.org. In the body type: "FILE /internet-drafts/draft-ietf-syslog-transport-udp-00.txt". NOTE: The mail server at ietf.org can return the document in MIME-encoded form by using the "mpack" utility. To use this feature, insert the command "ENCODING mime" before the "FILE" command. To decode the response(s), you will need "munpack" or a MIME-compliant mail reader. Different MIME-compliant mail readers exhibit different behavior, especially when dealing with "multipart" MIME messages (i.e. documents which have been split up into multiple messages), so check your local documentation on how to manipulate these messages. Below is the data which will enable a MIME compliant mail reader implementation to automatically retrieve the ASCII version of the Internet-Draft. ------------------------------ Date: Wed, 10 Mar 2004 07:13:17 -0800 (PST) From: Chris Lonvick Subject: New Milestones and Dates Hi Everyone, I've updated the WG web page: http://www.employees.org/~lonvick/index.shtml In it, and on the IETF WG page, the due dates are a bit past due and don't reflect the way in which we are now proceeding. I'd like to propose these new Milestones and Dates. - --- Jul 04 Submit Syslog Protocol to IESG for consideration as a PROPOSED STANDARD Jul 04 Submit Syslog Transport Mapping to IESG for consideration as a PROPOSED STANDARD. Jul 04 Submit Syslog Device MIB to IESG for consideration as a PROPOSED STANDARD. Oct 04 Submit Syslog Authentication Protocol to IESG for consideration as a PROPOSED STANDARD. Oct 04 Submit Syslog Internationalization to IESG for consideration as a PROPOSED STANDARD. Apr 05 Revise drafts as necessary to advance these Internet-Drafts to Standards Track RFCs. - --- This will give us about 3 months to resolve issues in syslog-protocol and syslog-transport-udp so they may be submitted together. This will also give Glenn the same amount of time but it looked like many of his issues were resolved during the meeting. After that, we'll have the next 3 months for Jon to align syslog-sign with syslog-protocol/transport, and for us to decide upon how we will address syslog-international. Once those are done, we can move forward with 3195bis and see about moving the documents to Draft Standards. Please send in your comments about these dates. Are we being too optimistic? Do the ID authors/editors need more time? Thanks, Chris ------------------------------ Date: Wed, 10 Mar 2004 08:44:41 -0800 (PST) From: Chris Lonvick Subject: New ID: syslog-transport-udp Hi Folks, Anton has submited the syslog-transport-udp ID and it is in the ID repository: http://www.ietf.org/internet-drafts/draft-ietf-syslog-transport-udp-00.txt Please look it over and provide comments to the mailing list. Thanks, Chris ------------------------------ Date: Thu, 11 Mar 2004 15:46:34 +0100 From: Rainer Gerhards Subject: RE: New Milestones and Dates Chris, as far as syslog-protocol is concerned, I think these are realistic milestones/dates. I am sure we can meet mid-year if nothing really bad shows up. Rainer > -----Original Message----- > From: Chris Lonvick [mailto:clonvick@cisco.com] > Sent: Wednesday, March 10, 2004 4:13 PM > To: syslog-sec@employees.org > Subject: New Milestones and Dates > > Hi Everyone, > > I've updated the WG web page: > http://www.employees.org/~lonvick/index.shtml > In it, and on the IETF WG page, the due dates are a bit past due and > don't reflect the way in which we are now proceeding. I'd > like to propose > these new Milestones and Dates. > > --- > > Jul 04 Submit Syslog Protocol to IESG for consideration as a PROPOSED > STANDARD > > Jul 04 Submit Syslog Transport Mapping to IESG for consideration as a > PROPOSED STANDARD. > > Jul 04 Submit Syslog Device MIB to IESG for consideration as > a PROPOSED > STANDARD. > > Oct 04 Submit Syslog Authentication Protocol to IESG for > consideration as > a PROPOSED STANDARD. > > Oct 04 Submit Syslog Internationalization to IESG for > consideration as > a PROPOSED STANDARD. > > Apr 05 Revise drafts as necessary to advance these Internet-Drafts to > Standards Track RFCs. > > --- > > This will give us about 3 months to resolve issues in > syslog-protocol and > syslog-transport-udp so they may be submitted together. This > will also > give Glenn the same amount of time but it looked like many of > his issues > were resolved during the meeting. After that, we'll have the next 3 > months for Jon to align syslog-sign with > syslog-protocol/transport, and > for us to decide upon how we will address syslog-international. Once > those are done, we can move forward with 3195bis and see > about moving the > documents to Draft Standards. > > Please send in your comments about these dates. Are we being too > optimistic? Do the ID authors/editors need more time? > > Thanks, > Chris > > ------------------------------ Date: Mon, 29 Mar 2004 10:08:09 -0800 (PST) From: Lonvick Subject: syslog-protocol issue 7: enterprise-id - ---------- Forwarded message ---------- Date: Mon, 29 Mar 2004 18:24:21 +0200 From: Rainer Gerhards To: syslog-sec@employees.org Subject: syslog-protocol issue 7: enterprise-id Hi WG, sorry for the long silence. Other stuff got in the way... Anyhow, I am back on "syslog track" and would like to sort out the remaining issues. In issue 7, we discuss if we should include the enterprise id in the header. As a refersher, you may want to review http://www.syslog.cc/ietf/protocol/issue7.html Based on the (few) feedback so far, I think this is not a mandatory data element. As such, I propose we a) remove it from the required header b) move it as an optional element into the "origin" SD-ID (7.3) This would still allow to provide it, but it would no longer be required. I think this is a good compromise. Please comment. Rainer ------------------------------ Date: Mon, 29 Mar 2004 18:24:21 +0200 From: Rainer Gerhards Subject: syslog-protocol issue 7: enterprise-id Hi WG, sorry for the long silence. Other stuff got in the way... Anyhow, I am back on "syslog track" and would like to sort out the remaining issues. In issue 7, we discuss if we should include the enterprise id in the header. As a refersher, you may want to review http://www.syslog.cc/ietf/protocol/issue7.html Based on the (few) feedback so far, I think this is not a mandatory data element. As such, I propose we a) remove it from the required header b) move it as an optional element into the "origin" SD-ID (7.3) This would still allow to provide it, but it would no longer be required. I think this is a good compromise. Please comment. Rainer ------------------------------ Date: Tue, 30 Mar 2004 18:58:53 +0200 From: "ons-huis.net!ALbert" Subject: some (well a lot ) review remarks Hello Rainer, Anton, WG Last weeks I have reviewed some of the current draft documents. I had planned to spend more time on it, coming month. But, plans are changed. It will be very busy, which is good .. But also means less tome for syslog. I have one important "issue": I think it would be a great improvement if some parts of Rainer's document would move to Anton's. I think the protocol document should NOT specify how to break messages in parts. It should assume the transport layer can "transport" 'long messages'. The transport layer, should when a long message can't be send in once, spilt in is several parts. I think it should be possible to write another 'transport' document, which describes how 'long, new, better' messages can be transported by rfc-3164 syslog messages. Note: I'm NOT saying we should make an RFC for this. Only that, if we could write it, the separation between transport and "upper" layers are better. Note 2: That document should describe how to 'shrink' the longer headers into the old headers, etc. Not by putting the complete message into the payload. - -------------- Aside from the point above, I will include a raider long list of (personal) notes 'asis'. Rainer, please use those notes when relevant, Skip the others. I'm sorry, I don't have more time to rewrite my notes to a good review. It is either me to throw away them, of give another that option.:-) Hope it helps a bit. =========================================================== - -- Groetjes, - --ALbert Mietus Private mail to: albert at ons-huis dot net Business mail to: albert dot mietus at PTS dot nl Spam: Just don't do it! Thrust me, I will not order! Hello WG, This is my comment on Rainer's syslog-protcol draft 04. I have been very passive on following this mailinglist; (to busy:-) But on receiving a request to review I decided to do so. But I apologise if I raise issues that are discussed already. I didn't read them. This week I started, after a log time, to study the current draft document(s). Before continuing, let me say I like the idea of splitting syslog in several layers. I say so, in the hope the individual sub-specifications will become short. My comment is split into 3 parts, of which you are reading part 1 now. Part 1 is about the "idea" (the design). In a separate mail I will comment on the text of the document; in the hope I will become clearer, cleaner and shorter. The 3rd mail will contain some details and bit that didn't fit in the others. Possible, I will send more comment in futures mail; it will depend on the amount of time I have now, and then. - ------------------------------------------------------- This review is becoming (very) long, I really hope all comments are worth reading. I have tried to express my thoughts about is a well as possible, given the time I have... General Comment on the idea/design of syslog-protocol ===================================================== * This idea is GOOD! H2 Architecture ================ Traditional, syslog knows Devices, Collectors and Relays. I would like to add two 'things'. One I would like to call a *Generator* , the other an *Runner*. *Generator* As we can see in e.g. most Unix implementations, it is the application that knows WHAT to log, transmit that to the system (the log device, the syslogdaemon) that know HOW to log it. Also on embedded systems, this can be seen. Historically, the combination of (a part of the) application and the "system" form the log-device. I would like to split this, now this opportunity exits. The part that is build-in in the application, (in C: the lines syslog(..."hai there");) can be called the (LOG-)Generator. The communication between generator and Device is system/platform depended. On Unix systems usually the log-device, on embedded systems a function-call, and on windows log-events can be used. By introducing this Generator, we (can) make clear this private/dedicated communication exist; and is allowed. We also make clear the Generator is syslog-protocol INDEPENDED. The function of the Device, know becomes clear: it get log-message (-events) and transport them. It also does some bookkeeping, like timestamping, adding crypto (for -sign), etc. *Runner* A LOG-runner is a other kind of syslog-thing, which is frequently used. Without a proper place in the architecture. Whereas a relay (should) forwards syslog-messages without knowledge of the semantics of the message, the Runner does. The most simple life-form of a runner is a filter. It "relays" messages, but only when the are important. On Unix there a several of these in perl, grep-scripts etc. Formally, the are knot relays (I think). A more complex runner, is a "program" that receives log-messages, CHANGES them, and send (or stores) the result. Examples: statically analyses, Intrusion detection, etc. Both kind of Runners are useful, frequently used, but the not part of the architecture. And as we try to make syslog "better", we better add them and make sure out standards can "deal" with them. Otherwise, non-RFC compliance log-programs will be standard. H4 Syslog format ================ 412 enterpriseID - ----------------- I don't see any reason to include an enterpriseID; not into the header. (When needed, it can be used in the structured MSG part) Currently, it is just a number. It will be unused, misused or will lead to a lot of (operational) management. I'm afraid for the latter, as in H4.1.3 is suggested that the semantic of the Facility can be enterpriceID depended. Also, it is required to use the "IANA assigned vendor" number. This implies open-source/free/non-commercial are 'ruled out' as the often will not to so. Last, should the number of the Generator-vendor, the system-vendor of the device-vendor be used? (See above about generator/device) This is not clear to me. And whatever one is chosen, it will be hard to implement. Not using the defacto (Unix) logging api's! 413 Facility - ------------- Although, at first sight, liked the idea of "a terrible lot" of facilities. The current is wrong, as I see it. Aside from the problems mentioned above, more then a million facilities will mean relays can't be managed! The set of facilities, which will be seen in a (major) network, expressed as numbers, will be are more or less at random. Which implies very long complex and unmanageable configfiles (or MIB's) for each LOG-router! As we heave learn form routing IPv4, hierarchical structuring is needed. I think, extending the set of facilities is good. But I can't imagine more the say 1000 are ever needed. So, my counter-proposition is: * Make facilities (as a number) structured * Limit the number of facilities to a manageable number * Keep the format such that extending the allowed numbers is possible A Facility then is still a number, at least 3 (or 4) digits long. A longer number means the it is an extended facility. They have to be assign by IANA. Facilities of length 3 (4) MUST have the format '(K)KLM', where 'K' (or 'KK') indicated the kind of facility; 'L' give a sub indication and 'M' is *SITE* configurable (so, by the local sysadmin, see example below). The 'K/KK' is based on the RFC3164 facilities, clean up and extended. Those numbers can be IANA assigned. L can be chosen by the (generator) vendor, and `M' by the admin. 'M' defaults to 0 (zero), and applications/vendors MAY give the possibility to set that digit. Example: For mail, there will be an K specified, let say 1. Then all mail-log will have the format '1XY', which is easily routable. It will do for small sites. Some vendors, like "sendmail" (only 1 process) will probably use only one value for L e.g. '0'; others, like "postfix" (several processes) can use multiple values, like '1', '2' and '3'. When supported by both sendmail and postfix, the local sysadmin can add (change) the M-digit, such that mail-systems on the border, and internal ones use another facility. In all cases, the local sysadmin can either use simple routing-rules, like 1** (for all mail), or 10* for sendmail and 1[1-2]*, 13* for postfix, or even more complex. Now, the sysadmin has a choice, and can keep it manageable. Note: the 0 for sendmail and 1-4 for postfix are "by example". However, we can add a "rule" that '0' shall be used when only 1 L-value is used, and when several values are used, zero should be skipped. Also, I would like to prescribe/reserve "9" for local additions. (on all digits). The (K)KLM idea is used a suggestion to improve, IF this idea is accepted, THEN we can discuss variations like 3 or 4 digits, where to save mappings (IANA of this rfc), etc. 5 Structured data ================== In short: I thing having the option of structured data is a nice option. But lets keep it simple. The current one is to complex, it has to be as we need 4 pages to describe it. Also, I find those pages hard to study (given the time I had :-). Also it can lead to not implementing it. Programmers, especially there bosses, don't have a lot of time! More positive: The main reason why structured data is complex (currently) comes down into 1 problems: 1) It isn't part of the main-design (the ABNF on page 9) 2) The "structure" can start anywhere in MSG Both are easy to solve: Ad 2) Specify that the structured part ALWAYS START directly after the header. Ad 1) We need to introduce it where it belongs. In an optional field on page 9 Let give it a try ( I also use the "improvement" on the ABNF of my other mail; it saves typing) (Also I "forget" the SP parts, for now. Just the idea) SYSLOG-MSG = HEADER DATA HEADER = VERSIONING PRIO ID // See other mail DATA = [ *STR-DATA ] MSG STR-DATA = see below MSG = free format Given this ABNF, the structured data ALWAYS comes (in RFC3164 notation) at the start of the MSG-part (of in new ABNF: before the free format MSG). This implies receivers always have (as last resort) the option to see everything after the header as free-format. And just store/forward it. It implies the start of STR-DATA is simple to find: Its starts directly after the header, or directly after another STR-DATA This implies to, we have the option to start STR-DATA with '<' which is more usable and XML-alike. The complex long "[@#" cookie isn't needed anymore. However, we free to use it. My personal vote is for the (XML) < > style. See also my other posting about details of structured-data. 6 Multi-Part Messages ===================== There are some mistakes in the this part, but I like the general idea. However, I fell spliting/reassembly is done in other protocols too. Maybe we can use/reference a (de facto) standard? I don't know an RFC which we can use, but I'm sure there must be one! Second, it to complex, and to long (to read). I have studied it, but I'm not sure I do understand it. Some details about which I find hard/wrong/dislike MP-timestamp - ------------ I do not like having several messages having the same TIMESTAMP 62 SD-ID receiving an optional STR-DATA ======================================= This must be a mistake! In the 3rd paragraph is stated the a receiver sometimes MUST NOT parse a STR-DATA of a log-message that is received. However, when the option Multi-part is not implemented, is doesn't now this! Hello WG, This is part 2 (of 3) of my comment on the 3th draft-syslog-protocol. See the introduction at my posting 1@3. This one contains comment on the text of the document; to help to clarify, and shorten the document. It does NOT contain comment on the "idea" (design); se posting 1@3. - ------------------------------------------------------- Implementation hints ==================== The current RFC contains a lot of valuable hints for programmers, like the one about time-secfrac (Yes I introduced it, at least the bug:-)! Currently the are scattered around the document, making the document long and more complex to read for non-programmers. I would suggest to move all of them to a new chapter, at the end of the document (after the current chapter 9). ABNF (Chapter 4) ================ I think we can clarify the syntax, by "unflattening" RGC-3164 syslog uses some field and subfields which are nice when introducing syslog to others. The "understand" names as priority. So, keep it structured. I give it a try (please correct the syntax of the ABNF, as I not writing it daily anymore) SYSLOG-MSG = HEADER DATA HEADER = VERSIONING PRIO ID DATA = [ STR-DATA ] MSG VERSIONING = "V" VERSION VERSION = 1*3 DIGIT PRIO = '<' FACILITY '.' SEVERITY '>' // See 2@3 for notation change ID = TIMESTAMP SP HOSTNAME SP TAG STR-DATA = see elsewhere MSG = free format (etc) Now we have meaningful field, that can be used to. E.g the ID field (see other mail) make each message unique 5.1 Format (typo?) ================== On page 20, the paragraph starting with "The structured data element MUST ..." is confusing. The 2nd last line say _no space_ is allowed, the last one says one or more space are. Is this a typo? Or it is unclear (at least to me) Structures data, ID-length ========================== I don't see why we should limit the several field to 64 positions. I agree this will normally suffice. But so will 32, or 16, of any other number. 64 is "to big" to use as a fix-sized ("reserved") space for programmer's, database fields, etc. (to big == spoil to much bytes on huge logs). So dynamic field need to be used anyhow. Then there is no need for a trivial maximum. Note, there is a maximum anyhow, given by the size of a single log-message. That will do for "short term allocation". Removing this limit, make the rdc cleaner and smaller. Structured data, spaces ======================= I would like to have all line about SP (spaces) in chapter 5 removed. The point about 0,1 or more spaces is not relevant. In general, syslog uses SP to separate field (when needed). And allows them in MSG-part. The syntax and semantics of STR-DATA is does not depend on the amount of space. Nor is it harder to read. Implementing receivers even become easier when spaces (in STR-DATA) can be skipped (while { if 'sp' then skip } ) instead of checking the correct number, and doing something if wrong ! Proposal: Allow spaces anywhere, but inside SD-ID (see note) and SD-PARAM. SP in SD-VALUE is allowed (already), but not a separator. Prescribe (at least 1) SP between each param-value pair. *Note: as SP between '[#@'(or '<') and the SP-ID itself is probably not a good idea, but not a problem. We can fix is, by moving the fixed string in the ABNF: STR-DATA = STR-START ... STR-END STR-START = "[#@" SD-ID ; or '<' STR-END = "]" ; or '>' ... = as before, SP are allowed. Note2: Doing so allows for format line for human reading, which is handy Example This example is simple to parse, both for humans and computers. This change will make chapter-5 shorter, I think! Last, I think any whitespace should be allowed instead of SP (eg. TAB) Chapter-5, MSG ============== I suggest, but only as a detail, the text of chapter-5 should be part of 4.2 Hello WG, This is part 3 (of 3) of my comment on the 3th draft-syslog-protocol. It only contains short remark on details, and bit that didn't fit in part 1 (idea/design) or part 2 (the document itself). See posting 1@3 for more introduction - ------------------------------------------------------- 413/314 FACILITY/SEVERITY ========================== In this draft, both facility and severity are numbers. Even with my suggestion, the are 'just numbers'. And numbers are hard to read for humans. Especially when the are a lot of them. Most people will forget which column contains which number. Therefore, I think it is better to use the (verbose)notation used by most syslog implementations: "'<' facility '.' severity '>'" Both facility and severity are numbers (at least in the wire). Collectors (viewers) can translate those numbers into there names. But still use this format. And Even without them, it is easier to read the first (new) then the second (current) line: V1 0 <888.4> 2003-010-11T22:14:13.003Z new.Formated. ... V1 0 888 4 2003-010-11T22:14:13.003Z old.Formated. ... Note: I agree, we should not the tricky "8 times F plus S" notation. I'm not suggesting that! Just insert a dot and the angles. 4151 timestamp, without time ============================ There are 'devices' as meant in this section which haven't an idea of TIME. So it is a good idea this section. Often, those devices can store ("know") a few bit of information. Therefore, I would like to change this fixed TIMESTAMP, to the same one, but with a sequence number attached; the factional-seconds field can be used for it. Then a timestamp becomes 2001-01-01T00:00:60.Z. As in the current draft, this time doesn't exist. But al least, collectors can (more or less) sort a set of logmessages form 1 device Note: the latter is needed for e.g. syslog-sign 417 TAG ======= I think we need to make the TAG stuctured! All current syslog receivers (collectors, relays) use __PARTS of__ the RFC3164 TAG to route messages. In RFC3164 the TAG is simple and short, so it is quite simple to use it for routing. Note: not the complete TAG is used, only the 'program name', never the PID part. With the new TAG, with an static ID an a dynamic part, similar routing should be possible. At least, the RFC should be clear on it. So, by demanding the static part is "fixed", and make sure that static part can be found. Given the current practice, routing is based (mainly) on the program-name, it would be wise to (at least suggest) how (where) that part can be found. Proposal: Forget native support for VMS/Windows/DOS and even Unix pathnames. And introduce an URI (URL)-alike schema. Where only '/' (not the one form Unix, but from URL's) is used to "path separation" and ':' and '//' are major separators. In this case, the ABNF is simple; only the semantics become a little more complex. Also, it become simple for web-applications to log. The have an URL already. All "old fashioned:-)" application have an URL: ''file://path/to/appl'' already. This is valid on any system. Note: the dynamic part has to be added Note2: for web-applictions, which include a 'hostpart' in the URL: that hostname is NOT the same as the HOSTPART in the header. Frequently (a web-farm) several systems share the same URL, but not the hostname. Then the sysadmin can decide which one to use for routing. Message ID ========== Given the architecture of syslog-networks, messages can be duplicated. But, sometimes messages are related (e.g. with the signatures of syslog-sign). Both the RFC3164 and the current -protocol draft do not have possibilities to unique identify a message. In practice, messages are unique by there hostname, TAG and timestamp. But, we can't trust on this, as it isn't required. I would like to introduce this requirement. It is simple to add to the RFC, and simple to implement. Given the HOSTNAME and TAG, all the implementers have to do is never send a message with the same timestamp. Given the microsecond resolution, this doable. (It does imply some systems have to fake the last digits; I don't see a problem with this. Otherwise we can add a "." SequenceNo to the timestamp Structured data, tokens ======================= Given the current draft, of my counter proposal of it, of structured data, the are (only) 2 kinds of tokens. The IANA controlled ones and the experimental ones; the latter starting with "x-" I think, we can safely add an third: "X-" for private/local/vendor specific tokens. As we can see my e.g. mail, this kind of field will be used a lot. Now we have an option to allow the, without giving the a status of "testing". STR-DATA, can we use it for syslog-sign (or similar)? ===================================================== Just an idea: in a protocol as syslog-sign (here just as example), where messages reference to other messages (now implicit), we could use the STR-DATA to do so? I verified the syntax/semantics of this, and YES, we could do so. This means, I like STR-DATA a lot more :-) It is great. Even when -sign doesn't use it, I (we) can use the same format to present it to the user! 64 MultiPart examples ===================== All examples use rfc3164 headers, shouldn't -protocol headers be used? ------------------------------