From majordomo@mil.doit.wisc.edu Wed Apr 2 01:49:38 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id BAA01818 for ; Wed, 2 Apr 2003 01:49:38 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 190bmB-0006Qr-00 for ipfix-list@mil.doit.wisc.edu; Wed, 02 Apr 2003 00:31:11 -0600 Received: from weird-brew.cisco.com ([144.254.15.118] helo=strange-brew.cisco.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 190bm8-0006Qj-00 for ipfix@net.doit.wisc.edu; Wed, 02 Apr 2003 00:31:09 -0600 Received: from cisco.com (bclaise-isdn-home3.cisco.com [10.49.4.220]) by strange-brew.cisco.com (8.11.6+Sun/8.8.8) with ESMTP id h326V6K26009 for ; Wed, 2 Apr 2003 08:31:07 +0200 (CEST) Message-ID: <3E8A83AA.90602@cisco.com> Date: Wed, 02 Apr 2003 08:31:06 +0200 From: Benoit Claise User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.3) Gecko/20030312 X-Accept-Language: en-us, en MIME-Version: 1.0 To: ipfix@net.doit.wisc.edu Subject: [ipfix] IPr statement for draft-claise-netflow-9-01.txt Content-Type: multipart/alternative; boundary="------------030102010809070409010308" Precedence: bulk Sender: majordomo listserver This is a multi-part message in MIME format. --------------030102010809070409010308 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Hi, The Intellectual Property Rights statement has been added to the IETF web site. http://www.ietf.org/ipr.html -> Cisco's Patent Statement pertaining to draft-claise-netflow-9-01.txt entitled "Cisco Systems NetFlow Services Export Version 9" Regards, Benoit. --------------030102010809070409010308 Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7bit Hi,

The Intellectual Property Rights statement has been added to the IETF web site.
http://www.ietf.org/ipr.html
-> Cisco's Patent Statement pertaining to draft-claise-netflow-9-01.txt entitled "Cisco Systems NetFlow Services Export Version 9"

Regards, Benoit.

--------------030102010809070409010308-- -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu Apr 3 12:23:34 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA01180 for ; Thu, 3 Apr 2003 12:23:34 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19185p-0005QN-00 for ipfix-list@mil.doit.wisc.edu; Thu, 03 Apr 2003 11:01:37 -0600 Received: from ip166.usw12.rb1.bel.nwlink.com ([209.20.253.166] helo=ran.psg.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19185n-0005QF-00 for ipfix@net.doit.wisc.edu; Thu, 03 Apr 2003 11:01:36 -0600 Received: from localhost ([127.0.0.1] helo=ran.psg.com) by ran.psg.com with esmtp (Exim 4.12) id 19185m-0005qZ-00 for ipfix@net.doit.wisc.edu; Thu, 03 Apr 2003 09:01:34 -0800 From: Randy Bush MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Date: Thu, 3 Apr 2003 09:01:34 -0800 To: ipfix@net.doit.wisc.edu Subject: [ipfix] iesg comment on draft-ietf-ipfix-reqs-09.txt Message-Id: Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit comments from iesg reviewers randy --- I see: 5.4. Timestamps The metering process MUST be able to generate timestamps for the first and the last observation of a packet of a flow at the observation point. The timestamp resolution MUST be at least the one of the sysUpTime [RFC1213], which is one centisecond. I think I'd rather see a reference to RFC3418 instead of RFC1213. We can do so by RFC-Editor note. May want to check with authors too. On page 14 I see: 9. type of service octet (in case of IPv4), traffic class octet (in case of IPv6) Is that not better state as 9. DSCP, which is set in the type of service octet (in case of IPv4), or in the traffic class octet (in case of IPv6) --- I would prefer to see the paragraph referencing RFC 2119 in section 2. Last paragraph in section 6.3.3 needs to be reworded. In section 10.2, an important point is left unsaid. If the injection of IPFIX traffic fool the network provided into thinking that a DOS attack is underway, the countermeasures employed by the network provider may actually deny service to real customers. The appendix needs to be moved up in the document. --- Overall, I like it. A couple of things jumped out at me, though. This document goes to a lot of trouble to say exactly what is exported (e.g. what fields, etc), except for two cases: 8. byte counter Which bytes of a packet are counted MUST be defined exactly. 20. multicast replication factor the number of outgoing packets originating from a single incoming multicast packet. This is a dynamic property of multicast flows, that may change over time. For unicast flows it has the constant value 1. The computation of the factor MUST be clearly defined. This document is defining what fields are being reported on; why can't it also define these two things that must also be defined? Is it useful to have different ipfix protocols exporting different measurements? Also, at the end of 6.1 it kind of says "Plus other stuff related to inter-AS routing if you want". It's already in the MAY section; can't they just list some concrete items related to inter-AS routing as 27, 28, ...? I don't think it's useful to say what amounts to "plus anything else that you might feel like adding that you think is relevant", since that doesn't encourage different people to make the same choices. --- This is a good document overall. The applications include usage-based accounting. They should cite RFC 2975, and indicate that the particular niche for usage-based accounting would not be met if reliability was not very high. Section 5.1 reliability requirements do not meet the needs, nor do the timestamping of first and last packet of a flow, because the flow may have been mischaracterized and that is first and last of different flows. The way to satisfy this issue is to caveat the discussion of usage-based accounting with strong words on reliability that counter the comments on lesser reliability and more probabilistic techniques for other applications of ipfix. Requirements on the ipfix implementation - the document is long and I wonder if the working group really meant the protocol's confidentiality and anonymization features to be so optional - SHOULD confidentiality, MAY anonymization. Just for implementation. -30- -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri Apr 4 11:15:01 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA27917 for ; Fri, 4 Apr 2003 11:15:01 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 191TiC-0005NG-00 for ipfix-list@mil.doit.wisc.edu; Fri, 04 Apr 2003 10:06:40 -0600 Received: from tokyo.ccrle.nec.de ([195.37.70.2]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 191TiA-0005N8-00 for ipfix@net.doit.wisc.edu; Fri, 04 Apr 2003 10:06:38 -0600 Received: from venus.office (venus.office [10.1.1.11]) by tokyo.ccrle.nec.de (8.12.9/8.12.8) with ESMTP id h34G6YVI018450; Fri, 4 Apr 2003 18:06:35 +0200 (CEST) Received: from [10.1.1.128] (n-quittek.office [10.1.1.128]) by venus.office (Postfix on SuSE Linux eMail Server 3.0) with ESMTP id BAD4C98228; Fri, 4 Apr 2003 18:04:15 +0200 (CEST) Date: Fri, 04 Apr 2003 18:07:23 +0200 From: Juergen Quittek To: Randy Bush , ipfix@net.doit.wisc.edu Subject: Re: [ipfix] iesg comment on draft-ietf-ipfix-reqs-09.txt Message-ID: <26048235.1049479643@[10.1.1.128]> In-Reply-To: References: X-Mailer: Mulberry/2.1.2 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Hi all, I think we got many good comments. Most of them can be committed quickly. However, there are some that might require some discussion: - Does the Appendix need to be moved up in the document? - Shall we be more clear on byte counter, multicast replication factor, and BGP AS#s? - Do we agree that we need to add a statement on that accounting requirements can only be met with higher reliability? I would answer 'no' on all three questions. Please see some more detailed statement on the comments below. Juergen -- Juergen Quittek quittek@ccrle.nec.de Tel: +49 6221 90511-15 NEC Europe Ltd., Network Laboratories Fax: +49 6221 90511-55 Kurfuersten-Anlage 36, 69115 Heidelberg, Germany http://www.ccrle.nec.de -- Randy Bush wrote on 03 April 2003 09:01 -0800: > > comments from iesg reviewers > > randy > > --- > > I see: > 5.4. Timestamps > > The metering process MUST be able to generate timestamps for the > first and the last observation of a packet of a flow at the > observation point. The timestamp resolution MUST be at least the one > of the sysUpTime [RFC1213], which is one centisecond. > > I think I'd rather see a reference to RFC3418 instead of RFC1213. > We can do so by RFC-Editor note. May want to check with authors too. Certainly we should follow this suggestion. > On page 14 I see: > > > 9. type of service octet (in case of IPv4), traffic class > octet (in case of IPv6) > > Is that not better state as > > 9. DSCP, which is set in the type of service octet (in case of IPv4), > or in the traffic class octet (in case of IPv6) We discussed this before: According to RFC 2474, the DSCP covers only 6 bit of the type of service octet, and in some cases you want to see the entire byte. > > --- > > I would prefer to see the paragraph referencing RFC 2119 in section 2. Makes sense. > Last paragraph in section 6.3.3 needs to be reworded. Let's try to find another wording. > In section 10.2, an important point is left unsaid. If the injection of > IPFIX traffic fool the network provided into thinking that a DOS attack is > underway, the countermeasures employed by the network provider may actually > deny service to real customers. Good point. We should discuss this threat also in some additional text. > The appendix needs to be moved up in the document. Can anybody explain why? > --- > > Overall, I like it. A couple of things jumped out at me, though. > This document goes to a lot of trouble to say exactly what is > exported (e.g. what fields, etc), except for two cases: > > 8. byte counter > Which bytes of a packet are counted MUST be defined exactly. > > 20. multicast replication factor > the number of outgoing packets originating from a single > incoming multicast packet. This is a dynamic property of > multicast flows, that may change over time. For unicast flows > it has the constant value 1. The computation of the factor MUST > be clearly defined. > > This document is defining what fields are being reported on; > why can't it also define these two things that must also be > defined? Is it useful to have different ipfix protocols exporting > different measurements? This is a requirements document. It is a preparation for defining a single IPFIX standard protocol. Therefore, I do not see a problem in leaving some (potentially tricky) issues to be defined by the protocol. Also, this list was intended to name and briefly explain the attributes. For the multicast replication factor, a definition takes several more words: It is a dynamic property. What would be its value, if during the time since the last report there were five outgoing packets per incoming packet and just before the report was made, one receiver disconnected? Would still 4 be correct for this report? Should it be 4.96? Or 5? We has this discussion and found no requirement to prefer one or the other. > > Also, at the end of 6.1 it kind of says "Plus other stuff related to inter-AS > routing if you want". It's already in the MAY section; can't they just > list some concrete items related to inter-AS routing as 27, 28, ...? > I don't think it's useful to say what amounts to "plus anything else that > you might feel like adding that you think is relevant", since that doesn't > encourage different people to make the same choices. Well, the statement was not THAT general. When discussing the BGP issues, we found that source AS#, destination AS#, previous hop AS#, and next hop AS# could be useful. Shall we list all of them as 27, 28, 29, 30? > --- > > This is a good document overall. > > The applications include usage-based accounting. They should cite RFC 2975, > and indicate that the particular niche for usage-based accounting would not > be met if reliability was not very high. Section 5.1 reliability requirements > do not meet the needs, nor do the timestamping of first and last packet of > a flow, because the flow may have been mischaracterized and that is first and > last of different flows. The way to satisfy this issue is to caveat the > discussion of usage-based accounting with strong words on reliability that > counter the comments on lesser reliability and more probabilistic techniques > for other applications of ipfix. Citing RFC 2975 is fine. But the WG could not agree on a statement, that this niche would not be met if reliability was not very high. We found non-neglectible counter-examples. I think there will be consensus on a statement like "Some accounting applications require high reliability". > Requirements on the ipfix implementation - the document is long and I wonder if > the working group really meant the protocol's confidentiality and anonymization > features to be so optional - SHOULD confidentiality, MAY anonymization. > Just for implementation. Sorry, what is the proposal here? Juergen > -30- > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Sat Apr 5 00:13:58 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id AAA18575 for ; Sat, 5 Apr 2003 00:13:57 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 191fqg-0006Bj-00 for ipfix-list@mil.doit.wisc.edu; Fri, 04 Apr 2003 23:04:14 -0600 Received: from psg.com ([147.28.0.62]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 191fqe-0006Ba-00 for ipfix@net.doit.wisc.edu; Fri, 04 Apr 2003 23:04:12 -0600 Received: from localhost ([127.0.0.1] helo=roam.psg.com) by psg.com with esmtp (Exim 3.36 #1) id 191fqb-0004pC-01; Fri, 04 Apr 2003 21:04:10 -0800 Received: from localhost ([127.0.0.1] helo=roam.psg.com ident=randy) by roam.psg.com with esmtp (Exim 4.12) id 191YJx-000FJC-00; Fri, 04 Apr 2003 13:01:58 -0800 From: Randy Bush MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Date: Fri, 4 Apr 2003 13:01:57 -0800 To: Juergen Quittek Cc: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] iesg comment on draft-ietf-ipfix-reqs-09.txt References: <26048235.1049479643@[10.1.1.128]> Message-Id: Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit >> 9. DSCP, which is set in the type of service octet (in case of IPv4), >> or in the traffic class octet (in case of IPv6) > We discussed this before: According to RFC 2474, the DSCP covers only 6 > bit of the type of service octet, and in some cases you want to see the > entire byte. it is not clear that saying this would damage the document >> The appendix needs to be moved up in the document. > Can anybody explain why? because appendices are more comfortable above references, copyright, and authors' addresses? >> This document is defining what fields are being reported on; >> why can't it also define these two things that must also be >> defined? Is it useful to have different ipfix protocols exporting >> different measurements? > This is a requirements document. It is a preparation for > defining a single IPFIX standard protocol. Therefore, I do not > see a problem in leaving some (potentially tricky) issues to be > defined by the protocol. is there a danger in making very clear that all measurements of the same phenomonon should produce the same result? > Also, this list was intended to name and briefly explain the > attributes. For the multicast replication factor, a definition > takes several more words: It is a dynamic property. What would be > its value, if during the time since the last report there were > five outgoing packets per incoming packet and just before the > report was made, one receiver disconnected? Would still 4 be > correct for this report? Should it be 4.96? Or 5? We has this > discussion and found no requirement to prefer one or the other. any harm in stating this? > I think there will be consensus on a statement like "Some > accounting applications require high reliability". i think our not making this more loudly and clearly explicit early on led us down enough ratholes already. so i would suggest making it quite clear that real accounting would require more than ipfix is promising to provide. >> Requirements on the ipfix implementation - the document is long and I >> wonder if the working group really meant the protocol's confidentiality >> and anonymization features to be so optional - SHOULD confidentiality, >> MAY anonymization. Just for implementation. > Sorry, what is the proposal here? maybe make confidentiality and anonymization mandatory to implement but not mandatory to use? randy -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Sun Apr 6 23:02:51 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id XAA12640 for ; Sun, 6 Apr 2003 23:02:50 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 192MXW-00040z-00 for ipfix-list@mil.doit.wisc.edu; Sun, 06 Apr 2003 21:39:18 -0500 Received: from tokyo.ccrle.nec.de ([195.37.70.2]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 192MXV-00040q-00 for ipfix@net.doit.wisc.edu; Sun, 06 Apr 2003 21:39:17 -0500 Received: from venus.office (venus.office [10.1.1.11]) by tokyo.ccrle.nec.de (8.12.9/8.12.8) with ESMTP id h372dAVI069709; Mon, 7 Apr 2003 04:39:10 +0200 (CEST) Received: from [10.1.1.27] (dial03.office [10.1.1.27]) by venus.office (Postfix on SuSE Linux eMail Server 3.0) with ESMTP id EDBAC9964E; Mon, 7 Apr 2003 04:36:26 +0200 (CEST) Date: Mon, 07 Apr 2003 04:40:00 +0200 From: Juergen Quittek To: Randy Bush Cc: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] iesg comment on draft-ietf-ipfix-reqs-09.txt Message-ID: <192274726.1049690399@[10.1.1.27]> In-Reply-To: References: X-Mailer: Mulberry/2.1.2 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit -- Randy Bush wrote on 04 April 2003 13:01 -0800: > >>> 9. DSCP, which is set in the type of service octet (in case of IPv4), >>> or in the traffic class octet (in case of IPv6) >> We discussed this before: According to RFC 2474, the DSCP covers only 6 >> bit of the type of service octet, and in some cases you want to see the >> entire byte. > > it is not clear that saying this would damage the document Should not be a problem to minimize damage when clarifying this. >>> The appendix needs to be moved up in the document. >> Can anybody explain why? > > because appendices are more comfortable above references, copyright, > and authors' addresses? Thanks. Moving it up will be easy. >>> This document is defining what fields are being reported on; >>> why can't it also define these two things that must also be >>> defined? Is it useful to have different ipfix protocols exporting >>> different measurements? >> This is a requirements document. It is a preparation for >> defining a single IPFIX standard protocol. Therefore, I do not >> see a problem in leaving some (potentially tricky) issues to be >> defined by the protocol. > > is there a danger in making very clear that all measurements of the same > phenomonon should produce the same result? No, not at all. But the reviewer requested to solve open questions (specify attribute semantics more clearly). This takes time and would delay this document again. Also, there is the info & data model document, which has exactly the goal of specifying attributes in more detail. >> Also, this list was intended to name and briefly explain the >> attributes. For the multicast replication factor, a definition >> takes several more words: It is a dynamic property. What would be >> its value, if during the time since the last report there were >> five outgoing packets per incoming packet and just before the >> report was made, one receiver disconnected? Would still 4 be >> correct for this report? Should it be 4.96? Or 5? We has this >> discussion and found no requirement to prefer one or the other. > > any harm in stating this? No. But it would also perfectly fit in the info & data model document. >> I think there will be consensus on a statement like "Some >> accounting applications require high reliability". > > i think our not making this more loudly and clearly explicit early > on led us down enough ratholes already. so i would suggest making > it quite clear that real accounting would require more than ipfix > is promising to provide. Fine. Let's be more explicit. >>> Requirements on the ipfix implementation - the document is long and I >>> wonder if the working group really meant the protocol's confidentiality >>> and anonymization features to be so optional - SHOULD confidentiality, >>> MAY anonymization. Just for implementation. >> Sorry, what is the proposal here? > > maybe make confidentiality and anonymization mandatory to implement > but not mandatory to use? This would be a significant change. Most of the other requests of the reviewers concern clarifications or editorial changes, but this one addresses the requirements itself. We should exchange pro and con arguments on stricter requirements for confidentiality and anonymization on the list. > randy > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Apr 8 20:08:25 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA08925 for ; Tue, 8 Apr 2003 20:08:24 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 1932os-0000LG-00 for ipfix-list@mil.doit.wisc.edu; Tue, 08 Apr 2003 18:48:02 -0500 Received: from atlrel6.hp.com ([156.153.255.205]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 1932oq-0000L9-00 for ipfix@net.doit.wisc.edu; Tue, 08 Apr 2003 18:48:00 -0500 Received: from xatlrelay1.atl.hp.com (xatlrelay1.atl.hp.com [15.45.89.190]) by atlrel6.hp.com (Postfix) with ESMTP id 9ABDA1C010E9; Tue, 8 Apr 2003 19:47:59 -0400 (EDT) Received: from xatlbh4.atl.hp.com (xatlbh4.atl.hp.com [15.45.89.189]) by xatlrelay1.atl.hp.com (Postfix) with ESMTP id 566F61C000B2; Tue, 8 Apr 2003 19:47:59 -0400 (EDT) Received: by xatlbh4.atl.hp.com with Internet Mail Service (5.5.2655.55) id <2JX05MYK>; Tue, 8 Apr 2003 19:47:59 -0400 Message-ID: <1D3D2C371FCBD947A7897FABBD3533A566B99F@xsun01.ptp.hp.com> From: "MEYER,JEFFREY D (HP-Cupertino,ex1)" To: "'Juergen Quittek'" , Randy Bush , ipfix@net.doit.wisc.edu Subject: RE: [ipfix] iesg comment on draft-ietf-ipfix-reqs-09.txt Date: Tue, 8 Apr 2003 19:47:58 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2655.55) Content-Type: text/plain; charset="iso-8859-1" Precedence: bulk Sender: majordomo listserver Hi, As to the last point, (accounting requirements), if we are talking about carrier grade accounting for billing purposes, then definitely NFv9 continues to have the same problems as all previous versions, i.e. if a packet is lost in transit it's lost, too bad. The fact that many carriers do bill with NF doesn't mean they are happy about this behavior. So, I do think it is important to point this out. Since the lack of addressing this requirement implies IPDR will need to go ahead independently to address this important requirement. Also I'd include references to the two RFC's I mentioned earlier in the references section as they include some useful discussions of prior learnings in this area: RFC2975 - Introduction to Accounting Management RFC2924 - Accounting Attributes and Record Formats Regards, Jeff Meyer > -----Original Message----- > From: Juergen Quittek [mailto:quittek@ccrle.nec.de] > Sent: Friday, April 04, 2003 8:07 AM > To: Randy Bush; ipfix@net.doit.wisc.edu > Subject: Re: [ipfix] iesg comment on draft-ietf-ipfix-reqs-09.txt > > > Hi all, > > I think we got many good comments. Most of them can be > committed quickly. > However, there are some that might require some discussion: > > - Does the Appendix need to be moved up in the document? > - Shall we be more clear on byte counter, multicast > replication factor, > and BGP AS#s? > - Do we agree that we need to add a statement on that > accounting requirements > can only be met with higher reliability? > > I would answer 'no' on all three questions. Please see some > more detailed > statement on the comments below. > > Juergen > -- > Juergen Quittek quittek@ccrle.nec.de Tel: +49 > 6221 90511-15 > NEC Europe Ltd., Network Laboratories Fax: +49 > 6221 90511-55 > Kurfuersten-Anlage 36, 69115 Heidelberg, Germany > http://www.ccrle.nec.de > > > -- Randy Bush wrote on 03 April 2003 09:01 -0800: > > > > comments from iesg reviewers > > > > randy > > > > --- > > > > I see: > > 5.4. Timestamps > > > > The metering process MUST be able to generate > timestamps for the > > first and the last observation of a packet of a flow at the > > observation point. The timestamp resolution MUST be at > least the one > > of the sysUpTime [RFC1213], which is one centisecond. > > > > I think I'd rather see a reference to RFC3418 instead of RFC1213. > > We can do so by RFC-Editor note. May want to check with authors too. > > Certainly we should follow this suggestion. > > > On page 14 I see: > > > > > > 9. type of service octet (in case of IPv4), traffic class > > octet (in case of IPv6) > > > > Is that not better state as > > > > 9. DSCP, which is set in the type of service octet > (in case of IPv4), > > or in the traffic class octet (in case of IPv6) > > We discussed this before: According to RFC 2474, the DSCP > covers only 6 bit > of the type of service octet, and in some cases you want to > see the entire byte. > > > > > --- > > > > I would prefer to see the paragraph referencing RFC 2119 > in section 2. > > Makes sense. > > > Last paragraph in section 6.3.3 needs to be reworded. > > Let's try to find another wording. > > > In section 10.2, an important point is left unsaid. If > the injection of > > IPFIX traffic fool the network provided into thinking that > a DOS attack is > > underway, the countermeasures employed by the network > provider may actually > > deny service to real customers. > > Good point. We should discuss this threat also in some > additional text. > > > The appendix needs to be moved up in the document. > > Can anybody explain why? > > > --- > > > > Overall, I like it. A couple of things jumped out at me, though. > > This document goes to a lot of trouble to say exactly what is > > exported (e.g. what fields, etc), except for two cases: > > > > 8. byte counter > > Which bytes of a packet are counted MUST be > defined exactly. > > > > 20. multicast replication factor > > the number of outgoing packets originating from a single > > incoming multicast packet. This is a dynamic property of > > multicast flows, that may change over time. For > unicast flows > > it has the constant value 1. The computation of > the factor MUST > > be clearly defined. > > > > This document is defining what fields are being reported on; > > why can't it also define these two things that must also be > > defined? Is it useful to have different ipfix protocols exporting > > different measurements? > > This is a requirements document. It is a preparation for defining > a single IPFIX standard protocol. Therefore, I do not see a problem in > leaving some (potentially tricky) issues to be defined by the > protocol. > > Also, this list was intended to name and briefly explain the > attributes. > For the multicast replication factor, a definition takes > several more words: > It is a dynamic property. What would be its value, if during > the time since > the last report there were five outgoing packets per incoming > packet and > just before the report was made, one receiver disconnected? > Would still 4 be correct for this report? Should it be 4.96? Or 5? > We has this discussion and found no requirement to prefer one > or the other. > > > > > Also, at the end of 6.1 it kind of says "Plus other stuff > related to inter-AS > > routing if you want". It's already in the MAY section; > can't they just > > list some concrete items related to inter-AS routing as 27, 28, ...? > > I don't think it's useful to say what amounts to "plus > anything else that > > you might feel like adding that you think is relevant", > since that doesn't > > encourage different people to make the same choices. > > Well, the statement was not THAT general. When discussing > the BGP issues, we > found that source AS#, destination AS#, previous hop AS#, and > next hop AS# could > be useful. Shall we list all of them as 27, 28, 29, 30? > > > --- > > > > This is a good document overall. > > > > The applications include usage-based accounting. They > should cite RFC 2975, > > and indicate that the particular niche for usage-based > accounting would not > > be met if reliability was not very high. Section 5.1 > reliability requirements > > do not meet the needs, nor do the timestamping of first and > last packet of > > a flow, because the flow may have been mischaracterized and > that is first and > > last of different flows. The way to satisfy this issue is > to caveat the > > discussion of usage-based accounting with strong words on > reliability that > > counter the comments on lesser reliability and more > probabilistic techniques > > for other applications of ipfix. > > Citing RFC 2975 is fine. > > But the WG could not agree on a statement, that this niche > would not be met > if reliability was not very high. We found non-neglectible > counter-examples. > > I think there will be consensus on a statement like "Some > accounting applications > require high reliability". > > > Requirements on the ipfix implementation - the document is > long and I wonder if > > the working group really meant the protocol's > confidentiality and anonymization > > features to be so optional - SHOULD confidentiality, MAY > anonymization. > > Just for implementation. > > Sorry, what is the proposal here? > > Juergen > > > -30- > > > > > > -- > > Help mailto:majordomo@net.doit.wisc.edu and say > "help" in message body > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe ipfix" in message body > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" > in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri Apr 11 00:38:43 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id AAA17802 for ; Fri, 11 Apr 2003 00:38:43 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 193pwc-0005I2-00 for ipfix-list@mil.doit.wisc.edu; Thu, 10 Apr 2003 23:15:18 -0500 Received: from eng4.oar.net ([192.148.244.24]) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 193pwa-0005HX-00 for ipfix@net.doit.wisc.edu; Thu, 10 Apr 2003 23:15:16 -0500 Received: (qmail 16751 invoked by uid 4454); 11 Apr 2003 04:15:10 -0000 Date: Fri, 11 Apr 2003 00:15:10 -0400 From: Mark Fullmer To: "MEYER,JEFFREY D \(HP-Cupertino,ex1\)" Cc: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] iesg comment on draft-ietf-ipfix-reqs-09.txt Message-ID: <20030411001510.A16691@net.ohio-state.edu> References: <1D3D2C371FCBD947A7897FABBD3533A566B99F@xsun01.ptp.hp.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: <1D3D2C371FCBD947A7897FABBD3533A566B99F@xsun01.ptp.hp.com>; from jeff.meyer2@hp.com on Tue, Apr 08, 2003 at 07:47:58PM -0400 Precedence: bulk Sender: majordomo listserver Why are we rehashing this again? NetFlow v9 can run over a reliable transport such as TCP or SCTP. The only part that's missing is a way for the collector to inform the exporter at failover time what flows to resend, which _must_ be an optional feature for the exporter. Consider: Exporter sends a "I expect a hello every N seconds" to the collector as an option, using the existing NetFlow v9 option template. Collector sees the above message and ACK's with the highest sequence number PDU it has processed every N seconds for each observation domain. Collector could even use the same packet format. If the exporter doesn't get an ACK in N+holdown seconds it switches to the secondary and retransmits PDU's that have not been ACK'd, iff it has the resources to buffer them. There's no negotiation, this feature is entirely controlled by the exporter (routers probably won't send this option, stand alone probes that have the resources probably will), and it's done within the current framework of NetFlow v9. NetFlow v9 != IPFIX. There will be some _small_ changes. The burden is going to be on the collector to support multiple transports, a reliable transport would allow something like above, an unreliable transport obviously won't. mark On Tue, Apr 08, 2003 at 07:47:58PM -0400, MEYER,JEFFREY D (HP-Cupertino,ex1) wrote: > Hi, > > As to the last point, (accounting requirements), if we are talking about > carrier grade accounting for billing purposes, then definitely NFv9 > continues > to have the same problems as all previous versions, i.e. if a packet is lost > in transit it's lost, too bad. The fact that many carriers do bill with NF > doesn't mean they are happy about this behavior. > > So, I do think it is important to point this out. Since the lack of > addressing > this requirement implies IPDR will need to go ahead independently to address > this important requirement. > > Also I'd include references to the two RFC's I mentioned earlier in the > references section as they include some useful discussions of prior > learnings > in this area: > > RFC2975 - Introduction to Accounting Management > RFC2924 - Accounting Attributes and Record Formats > > Regards, > > Jeff Meyer > > > -----Original Message----- > > From: Juergen Quittek [mailto:quittek@ccrle.nec.de] > > Sent: Friday, April 04, 2003 8:07 AM > > To: Randy Bush; ipfix@net.doit.wisc.edu > > Subject: Re: [ipfix] iesg comment on draft-ietf-ipfix-reqs-09.txt > > > > > > Hi all, > > > > I think we got many good comments. Most of them can be > > committed quickly. > > However, there are some that might require some discussion: > > > > - Does the Appendix need to be moved up in the document? > > - Shall we be more clear on byte counter, multicast > > replication factor, > > and BGP AS#s? > > - Do we agree that we need to add a statement on that > > accounting requirements > > can only be met with higher reliability? > > > > I would answer 'no' on all three questions. Please see some > > more detailed > > statement on the comments below. > > > > Juergen > > -- > > Juergen Quittek quittek@ccrle.nec.de Tel: +49 > > 6221 90511-15 > > NEC Europe Ltd., Network Laboratories Fax: +49 > > 6221 90511-55 > > Kurfuersten-Anlage 36, 69115 Heidelberg, Germany > > http://www.ccrle.nec.de > > > > > > -- Randy Bush wrote on 03 April 2003 09:01 -0800: > > > > > > comments from iesg reviewers > > > > > > randy > > > > > > --- > > > > > > I see: > > > 5.4. Timestamps > > > > > > The metering process MUST be able to generate > > timestamps for the > > > first and the last observation of a packet of a flow at the > > > observation point. The timestamp resolution MUST be at > > least the one > > > of the sysUpTime [RFC1213], which is one centisecond. > > > > > > I think I'd rather see a reference to RFC3418 instead of RFC1213. > > > We can do so by RFC-Editor note. May want to check with authors too. > > > > Certainly we should follow this suggestion. > > > > > On page 14 I see: > > > > > > > > > 9. type of service octet (in case of IPv4), traffic class > > > octet (in case of IPv6) > > > > > > Is that not better state as > > > > > > 9. DSCP, which is set in the type of service octet > > (in case of IPv4), > > > or in the traffic class octet (in case of IPv6) > > > > We discussed this before: According to RFC 2474, the DSCP > > covers only 6 bit > > of the type of service octet, and in some cases you want to > > see the entire byte. > > > > > > > > --- > > > > > > I would prefer to see the paragraph referencing RFC 2119 > > in section 2. > > > > Makes sense. > > > > > Last paragraph in section 6.3.3 needs to be reworded. > > > > Let's try to find another wording. > > > > > In section 10.2, an important point is left unsaid. If > > the injection of > > > IPFIX traffic fool the network provided into thinking that > > a DOS attack is > > > underway, the countermeasures employed by the network > > provider may actually > > > deny service to real customers. > > > > Good point. We should discuss this threat also in some > > additional text. > > > > > The appendix needs to be moved up in the document. > > > > Can anybody explain why? > > > > > --- > > > > > > Overall, I like it. A couple of things jumped out at me, though. > > > This document goes to a lot of trouble to say exactly what is > > > exported (e.g. what fields, etc), except for two cases: > > > > > > 8. byte counter > > > Which bytes of a packet are counted MUST be > > defined exactly. > > > > > > 20. multicast replication factor > > > the number of outgoing packets originating from a single > > > incoming multicast packet. This is a dynamic property of > > > multicast flows, that may change over time. For > > unicast flows > > > it has the constant value 1. The computation of > > the factor MUST > > > be clearly defined. > > > > > > This document is defining what fields are being reported on; > > > why can't it also define these two things that must also be > > > defined? Is it useful to have different ipfix protocols exporting > > > different measurements? > > > > This is a requirements document. It is a preparation for defining > > a single IPFIX standard protocol. Therefore, I do not see a problem in > > leaving some (potentially tricky) issues to be defined by the > > protocol. > > > > Also, this list was intended to name and briefly explain the > > attributes. > > For the multicast replication factor, a definition takes > > several more words: > > It is a dynamic property. What would be its value, if during > > the time since > > the last report there were five outgoing packets per incoming > > packet and > > just before the report was made, one receiver disconnected? > > Would still 4 be correct for this report? Should it be 4.96? Or 5? > > We has this discussion and found no requirement to prefer one > > or the other. > > > > > > > > Also, at the end of 6.1 it kind of says "Plus other stuff > > related to inter-AS > > > routing if you want". It's already in the MAY section; > > can't they just > > > list some concrete items related to inter-AS routing as 27, 28, ...? > > > I don't think it's useful to say what amounts to "plus > > anything else that > > > you might feel like adding that you think is relevant", > > since that doesn't > > > encourage different people to make the same choices. > > > > Well, the statement was not THAT general. When discussing > > the BGP issues, we > > found that source AS#, destination AS#, previous hop AS#, and > > next hop AS# could > > be useful. Shall we list all of them as 27, 28, 29, 30? > > > > > --- > > > > > > This is a good document overall. > > > > > > The applications include usage-based accounting. They > > should cite RFC 2975, > > > and indicate that the particular niche for usage-based > > accounting would not > > > be met if reliability was not very high. Section 5.1 > > reliability requirements > > > do not meet the needs, nor do the timestamping of first and > > last packet of > > > a flow, because the flow may have been mischaracterized and > > that is first and > > > last of different flows. The way to satisfy this issue is > > to caveat the > > > discussion of usage-based accounting with strong words on > > reliability that > > > counter the comments on lesser reliability and more > > probabilistic techniques > > > for other applications of ipfix. > > > > Citing RFC 2975 is fine. > > > > But the WG could not agree on a statement, that this niche > > would not be met > > if reliability was not very high. We found non-neglectible > > counter-examples. > > > > I think there will be consensus on a statement like "Some > > accounting applications > > require high reliability". > > > > > Requirements on the ipfix implementation - the document is > > long and I wonder if > > > the working group really meant the protocol's > > confidentiality and anonymization > > > features to be so optional - SHOULD confidentiality, MAY > > anonymization. > > > Just for implementation. > > > > Sorry, what is the proposal here? > > > > Juergen > > > > > -30- > > > > > > > > > -- > > > Help mailto:majordomo@net.doit.wisc.edu and say > > "help" in message body > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > "unsubscribe ipfix" in message body > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > > > -- > > Help mailto:majordomo@net.doit.wisc.edu and say "help" > > in message body > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe ipfix" in message body > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri Apr 11 09:14:31 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA11431 for ; Fri, 11 Apr 2003 09:14:30 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 193y98-0001qq-00 for ipfix-list@mil.doit.wisc.edu; Fri, 11 Apr 2003 08:00:46 -0500 Received: from [209.202.115.138] (helo=kanmx1.ca.alcatel.com) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 193y96-0001qi-00 for ipfix@net.doit.wisc.edu; Fri, 11 Apr 2003 08:00:44 -0500 Received: (qmail 15771 invoked from network); 11 Apr 2003 13:09:08 -0000 Received: from unknown (HELO alcatel.com) (138.120.51.110) by kanmx1.ca.alcatel.com with SMTP; 11 Apr 2003 13:09:08 -0000 Message-ID: <3E96BC76.75C7588A@alcatel.com> Date: Fri, 11 Apr 2003 09:00:39 -0400 From: Mark Thibodeau X-Mailer: Mozilla 4.78 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: Mark Fullmer , ipfix@net.doit.wisc.edu Subject: Re: [ipfix] iesg comment on draft-ietf-ipfix-reqs-09.txt References: <1D3D2C371FCBD947A7897FABBD3533A566B99F@xsun01.ptp.hp.com> <20030411001510.A16691@net.ohio-state.edu> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Hi Mark, Good points. But, why do you say "routers probably won't send this option"? Shouldn't the behavior of the router be similar to that of the standalone probe? I.e. shouldn't the router handle collector failover? For the router to detect a collector failure it will also need these "hello's". Cheers, Mark Mark Fullmer wrote: > Why are we rehashing this again? NetFlow v9 can run over a reliable transport > such as TCP or SCTP. The only part that's missing is a way for the collector > to inform the exporter at failover time what flows to resend, which _must_ > be an optional feature for the exporter. > > Consider: > > Exporter sends a "I expect a hello every N seconds" to the collector > as an option, using the existing NetFlow v9 option template. > > Collector sees the above message and ACK's with the highest sequence > number PDU it has processed every N seconds for each observation domain. > Collector could even use the same packet format. > > If the exporter doesn't get an ACK in N+holdown seconds it switches to > the secondary and retransmits PDU's that have not been ACK'd, iff it > has the resources to buffer them. > > There's no negotiation, this feature is entirely controlled by the exporter > (routers probably won't send this option, stand alone probes that have > the resources probably will), and it's done within the current framework > of NetFlow v9. > > NetFlow v9 != IPFIX. There will be some _small_ changes. > > The burden is going to be on the collector to support multiple transports, > a reliable transport would allow something like above, an unreliable > transport obviously won't. > > mark > > On Tue, Apr 08, 2003 at 07:47:58PM -0400, MEYER,JEFFREY D (HP-Cupertino,ex1) wrote: > > Hi, > > > > As to the last point, (accounting requirements), if we are talking about > > carrier grade accounting for billing purposes, then definitely NFv9 > > continues > > to have the same problems as all previous versions, i.e. if a packet is lost > > in transit it's lost, too bad. The fact that many carriers do bill with NF > > doesn't mean they are happy about this behavior. > > > > So, I do think it is important to point this out. Since the lack of > > addressing > > this requirement implies IPDR will need to go ahead independently to address > > this important requirement. > > > > Also I'd include references to the two RFC's I mentioned earlier in the > > references section as they include some useful discussions of prior > > learnings > > in this area: > > > > RFC2975 - Introduction to Accounting Management > > RFC2924 - Accounting Attributes and Record Formats > > > > Regards, > > > > Jeff Meyer > > > > > -----Original Message----- > > > From: Juergen Quittek [mailto:quittek@ccrle.nec.de] > > > Sent: Friday, April 04, 2003 8:07 AM > > > To: Randy Bush; ipfix@net.doit.wisc.edu > > > Subject: Re: [ipfix] iesg comment on draft-ietf-ipfix-reqs-09.txt > > > > > > > > > Hi all, > > > > > > I think we got many good comments. Most of them can be > > > committed quickly. > > > However, there are some that might require some discussion: > > > > > > - Does the Appendix need to be moved up in the document? > > > - Shall we be more clear on byte counter, multicast > > > replication factor, > > > and BGP AS#s? > > > - Do we agree that we need to add a statement on that > > > accounting requirements > > > can only be met with higher reliability? > > > > > > I would answer 'no' on all three questions. Please see some > > > more detailed > > > statement on the comments below. > > > > > > Juergen > > > -- > > > Juergen Quittek quittek@ccrle.nec.de Tel: +49 > > > 6221 90511-15 > > > NEC Europe Ltd., Network Laboratories Fax: +49 > > > 6221 90511-55 > > > Kurfuersten-Anlage 36, 69115 Heidelberg, Germany > > > http://www.ccrle.nec.de > > > > > > > > > -- Randy Bush wrote on 03 April 2003 09:01 -0800: > > > > > > > > comments from iesg reviewers > > > > > > > > randy > > > > > > > > --- > > > > > > > > I see: > > > > 5.4. Timestamps > > > > > > > > The metering process MUST be able to generate > > > timestamps for the > > > > first and the last observation of a packet of a flow at the > > > > observation point. The timestamp resolution MUST be at > > > least the one > > > > of the sysUpTime [RFC1213], which is one centisecond. > > > > > > > > I think I'd rather see a reference to RFC3418 instead of RFC1213. > > > > We can do so by RFC-Editor note. May want to check with authors too. > > > > > > Certainly we should follow this suggestion. > > > > > > > On page 14 I see: > > > > > > > > > > > > 9. type of service octet (in case of IPv4), traffic class > > > > octet (in case of IPv6) > > > > > > > > Is that not better state as > > > > > > > > 9. DSCP, which is set in the type of service octet > > > (in case of IPv4), > > > > or in the traffic class octet (in case of IPv6) > > > > > > We discussed this before: According to RFC 2474, the DSCP > > > covers only 6 bit > > > of the type of service octet, and in some cases you want to > > > see the entire byte. > > > > > > > > > > > --- > > > > > > > > I would prefer to see the paragraph referencing RFC 2119 > > > in section 2. > > > > > > Makes sense. > > > > > > > Last paragraph in section 6.3.3 needs to be reworded. > > > > > > Let's try to find another wording. > > > > > > > In section 10.2, an important point is left unsaid. If > > > the injection of > > > > IPFIX traffic fool the network provided into thinking that > > > a DOS attack is > > > > underway, the countermeasures employed by the network > > > provider may actually > > > > deny service to real customers. > > > > > > Good point. We should discuss this threat also in some > > > additional text. > > > > > > > The appendix needs to be moved up in the document. > > > > > > Can anybody explain why? > > > > > > > --- > > > > > > > > Overall, I like it. A couple of things jumped out at me, though. > > > > This document goes to a lot of trouble to say exactly what is > > > > exported (e.g. what fields, etc), except for two cases: > > > > > > > > 8. byte counter > > > > Which bytes of a packet are counted MUST be > > > defined exactly. > > > > > > > > 20. multicast replication factor > > > > the number of outgoing packets originating from a single > > > > incoming multicast packet. This is a dynamic property of > > > > multicast flows, that may change over time. For > > > unicast flows > > > > it has the constant value 1. The computation of > > > the factor MUST > > > > be clearly defined. > > > > > > > > This document is defining what fields are being reported on; > > > > why can't it also define these two things that must also be > > > > defined? Is it useful to have different ipfix protocols exporting > > > > different measurements? > > > > > > This is a requirements document. It is a preparation for defining > > > a single IPFIX standard protocol. Therefore, I do not see a problem in > > > leaving some (potentially tricky) issues to be defined by the > > > protocol. > > > > > > Also, this list was intended to name and briefly explain the > > > attributes. > > > For the multicast replication factor, a definition takes > > > several more words: > > > It is a dynamic property. What would be its value, if during > > > the time since > > > the last report there were five outgoing packets per incoming > > > packet and > > > just before the report was made, one receiver disconnected? > > > Would still 4 be correct for this report? Should it be 4.96? Or 5? > > > We has this discussion and found no requirement to prefer one > > > or the other. > > > > > > > > > > > Also, at the end of 6.1 it kind of says "Plus other stuff > > > related to inter-AS > > > > routing if you want". It's already in the MAY section; > > > can't they just > > > > list some concrete items related to inter-AS routing as 27, 28, ...? > > > > I don't think it's useful to say what amounts to "plus > > > anything else that > > > > you might feel like adding that you think is relevant", > > > since that doesn't > > > > encourage different people to make the same choices. > > > > > > Well, the statement was not THAT general. When discussing > > > the BGP issues, we > > > found that source AS#, destination AS#, previous hop AS#, and > > > next hop AS# could > > > be useful. Shall we list all of them as 27, 28, 29, 30? > > > > > > > --- > > > > > > > > This is a good document overall. > > > > > > > > The applications include usage-based accounting. They > > > should cite RFC 2975, > > > > and indicate that the particular niche for usage-based > > > accounting would not > > > > be met if reliability was not very high. Section 5.1 > > > reliability requirements > > > > do not meet the needs, nor do the timestamping of first and > > > last packet of > > > > a flow, because the flow may have been mischaracterized and > > > that is first and > > > > last of different flows. The way to satisfy this issue is > > > to caveat the > > > > discussion of usage-based accounting with strong words on > > > reliability that > > > > counter the comments on lesser reliability and more > > > probabilistic techniques > > > > for other applications of ipfix. > > > > > > Citing RFC 2975 is fine. > > > > > > But the WG could not agree on a statement, that this niche > > > would not be met > > > if reliability was not very high. We found non-neglectible > > > counter-examples. > > > > > > I think there will be consensus on a statement like "Some > > > accounting applications > > > require high reliability". > > > > > > > Requirements on the ipfix implementation - the document is > > > long and I wonder if > > > > the working group really meant the protocol's > > > confidentiality and anonymization > > > > features to be so optional - SHOULD confidentiality, MAY > > > anonymization. > > > > Just for implementation. > > > > > > Sorry, what is the proposal here? > > > > > > Juergen > > > > > > > -30- > > > > > > > > > > > > -- > > > > Help mailto:majordomo@net.doit.wisc.edu and say > > > "help" in message body > > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > > "unsubscribe ipfix" in message body > > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > > > > > > > -- > > > Help mailto:majordomo@net.doit.wisc.edu and say "help" > > > in message body > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > "unsubscribe ipfix" in message body > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > > -- > > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe ipfix" in message body > > Archive http://ipfix.doit.wisc.edu/archive/ > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- **************************************************** * Mark Thibodeau, P Eng. * 670 RSP Development - Firmware Designer * Alcatel CID - (613) 784-5375 * Fax - (613) 599-3696 * mark.thibodeau@alcatel.com **************************************************** -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri Apr 11 12:38:47 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA20378 for ; Fri, 11 Apr 2003 12:38:46 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 1941Km-0006Fx-00 for ipfix-list@mil.doit.wisc.edu; Fri, 11 Apr 2003 11:25:00 -0500 Received: from palrel12.hp.com ([156.153.255.237]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 1941Kk-0006Fb-00 for ipfix@net.doit.wisc.edu; Fri, 11 Apr 2003 11:24:58 -0500 Received: from xparelay2.ptp.hp.com (xparelay2.ptp.hp.com [15.1.28.65]) by palrel12.hp.com (Postfix) with ESMTP id E09CD1C02340; Fri, 11 Apr 2003 09:24:57 -0700 (PDT) Received: from xpabh1.ptp.hp.com (xpabh1.ptp.hp.com [15.1.28.60]) by xparelay2.ptp.hp.com (Postfix) with ESMTP id 805BF1C00AD9; Fri, 11 Apr 2003 09:24:57 -0700 (PDT) Received: by xpabh1.ptp.hp.com with Internet Mail Service (5.5.2655.55) id <2S4ZGTWS>; Fri, 11 Apr 2003 09:24:57 -0700 Message-ID: <1D3D2C371FCBD947A7897FABBD3533A566B9CC@xsun01.ptp.hp.com> From: "MEYER,JEFFREY D (HP-Cupertino,ex1)" To: "'Mark Fullmer'" , "MEYER,JEFFREY D (HP-Cupertino,ex1)" Cc: ipfix@net.doit.wisc.edu Subject: RE: [ipfix] iesg comment on draft-ietf-ipfix-reqs-09.txt Date: Fri, 11 Apr 2003 09:24:55 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2655.55) Content-Type: text/plain; charset="iso-8859-1" Precedence: bulk Sender: majordomo listserver Hi, I'm not rehashing this, simply pointing out that a reliable transport is not the same thing as guaranteed delivery. For some reason people don't seem to accept this. Don't take my word for it, see RFC2975 section 2.1.5 "Application acknowledgements". Since NF/IPFIX as defined today is unidirectional, the server has NO way of knowing what may have been lost in transit. One can wish or pretend this isn't a problem, but it just won't go away. For some reasons, providers who bill on this accounting information like to have a higher level of confidence that things don't just "fall on the floor". Note: I'm resigned to the fact that IPFIX will not enable this capability, and one could argue that this is a different class of accounting (or telemetry), and shouldn't be bothered with such details. Fine. My only point is that there is a significant market out there that does care about this, and RADIUS and Diameter are too high overhead to address. So a middle ground between IPFIX's apparent charter and the AAA groups is needed. Hence, IPDR. Alternatively, this could be flagged as "issue #1" in the joint work on creating an IPFIX protocol. Regards, Jeff Meyer > -----Original Message----- > From: Mark Fullmer [mailto:maf@eng.oar.net] > Sent: Thursday, April 10, 2003 9:15 PM > To: MEYER,JEFFREY D (HP-Cupertino,ex1) > Cc: ipfix@net.doit.wisc.edu > Subject: Re: [ipfix] iesg comment on draft-ietf-ipfix-reqs-09.txt > > > Why are we rehashing this again? NetFlow v9 can run over a > reliable transport > such as TCP or SCTP. The only part that's missing is a way > for the collector > to inform the exporter at failover time what flows to resend, > which _must_ > be an optional feature for the exporter. > > Consider: > > Exporter sends a "I expect a hello every N seconds" to the > collector > as an option, using the existing NetFlow v9 option template. > > Collector sees the above message and ACK's with the highest sequence > number PDU it has processed every N seconds for each > observation domain. > Collector could even use the same packet format. > > If the exporter doesn't get an ACK in N+holdown seconds it > switches to > the secondary and retransmits PDU's that have not been ACK'd, iff it > has the resources to buffer them. > > There's no negotiation, this feature is entirely controlled > by the exporter > (routers probably won't send this option, stand alone probes that have > the resources probably will), and it's done within the > current framework > of NetFlow v9. > > NetFlow v9 != IPFIX. There will be some _small_ changes. > > The burden is going to be on the collector to support > multiple transports, > a reliable transport would allow something like above, an unreliable > transport obviously won't. > > mark > > On Tue, Apr 08, 2003 at 07:47:58PM -0400, MEYER,JEFFREY D > (HP-Cupertino,ex1) wrote: > > Hi, > > > > As to the last point, (accounting requirements), if we are > talking about > > carrier grade accounting for billing purposes, then definitely NFv9 > > continues > > to have the same problems as all previous versions, i.e. if > a packet is lost > > in transit it's lost, too bad. The fact that many carriers > do bill with NF > > doesn't mean they are happy about this behavior. > > > > So, I do think it is important to point this out. Since the lack of > > addressing > > this requirement implies IPDR will need to go ahead > independently to address > > this important requirement. > > > > Also I'd include references to the two RFC's I mentioned > earlier in the > > references section as they include some useful discussions of prior > > learnings > > in this area: > > > > RFC2975 - Introduction to Accounting Management > > RFC2924 - Accounting Attributes and Record Formats > > > > Regards, > > > > Jeff Meyer > > > > > -----Original Message----- > > > From: Juergen Quittek [mailto:quittek@ccrle.nec.de] > > > Sent: Friday, April 04, 2003 8:07 AM > > > To: Randy Bush; ipfix@net.doit.wisc.edu > > > Subject: Re: [ipfix] iesg comment on draft-ietf-ipfix-reqs-09.txt > > > > > > > > > Hi all, > > > > > > I think we got many good comments. Most of them can be > > > committed quickly. > > > However, there are some that might require some discussion: > > > > > > - Does the Appendix need to be moved up in the document? > > > - Shall we be more clear on byte counter, multicast > > > replication factor, > > > and BGP AS#s? > > > - Do we agree that we need to add a statement on that > > > accounting requirements > > > can only be met with higher reliability? > > > > > > I would answer 'no' on all three questions. Please see some > > > more detailed > > > statement on the comments below. > > > > > > Juergen > > > -- > > > Juergen Quittek quittek@ccrle.nec.de Tel: +49 > > > 6221 90511-15 > > > NEC Europe Ltd., Network Laboratories Fax: +49 > > > 6221 90511-55 > > > Kurfuersten-Anlage 36, 69115 Heidelberg, Germany > > > http://www.ccrle.nec.de > > > > > > > > > -- Randy Bush wrote on 03 April 2003 09:01 -0800: > > > > > > > > comments from iesg reviewers > > > > > > > > randy > > > > > > > > --- > > > > > > > > I see: > > > > 5.4. Timestamps > > > > > > > > The metering process MUST be able to generate > > > timestamps for the > > > > first and the last observation of a packet of a flow at the > > > > observation point. The timestamp resolution MUST be at > > > least the one > > > > of the sysUpTime [RFC1213], which is one centisecond. > > > > > > > > I think I'd rather see a reference to RFC3418 instead > of RFC1213. > > > > We can do so by RFC-Editor note. May want to check with > authors too. > > > > > > Certainly we should follow this suggestion. > > > > > > > On page 14 I see: > > > > > > > > > > > > 9. type of service octet (in case of IPv4), traffic class > > > > octet (in case of IPv6) > > > > > > > > Is that not better state as > > > > > > > > 9. DSCP, which is set in the type of service octet > > > (in case of IPv4), > > > > or in the traffic class octet (in case of IPv6) > > > > > > We discussed this before: According to RFC 2474, the DSCP > > > covers only 6 bit > > > of the type of service octet, and in some cases you want to > > > see the entire byte. > > > > > > > > > > > --- > > > > > > > > I would prefer to see the paragraph referencing RFC 2119 > > > in section 2. > > > > > > Makes sense. > > > > > > > Last paragraph in section 6.3.3 needs to be reworded. > > > > > > Let's try to find another wording. > > > > > > > In section 10.2, an important point is left unsaid. If > > > the injection of > > > > IPFIX traffic fool the network provided into thinking that > > > a DOS attack is > > > > underway, the countermeasures employed by the network > > > provider may actually > > > > deny service to real customers. > > > > > > Good point. We should discuss this threat also in some > > > additional text. > > > > > > > The appendix needs to be moved up in the document. > > > > > > Can anybody explain why? > > > > > > > --- > > > > > > > > Overall, I like it. A couple of things jumped out at > me, though. > > > > This document goes to a lot of trouble to say exactly what is > > > > exported (e.g. what fields, etc), except for two cases: > > > > > > > > 8. byte counter > > > > Which bytes of a packet are counted MUST be > > > defined exactly. > > > > > > > > 20. multicast replication factor > > > > the number of outgoing packets originating > from a single > > > > incoming multicast packet. This is a dynamic > property of > > > > multicast flows, that may change over time. For > > > unicast flows > > > > it has the constant value 1. The computation of > > > the factor MUST > > > > be clearly defined. > > > > > > > > This document is defining what fields are being reported on; > > > > why can't it also define these two things that must also be > > > > defined? Is it useful to have different ipfix > protocols exporting > > > > different measurements? > > > > > > This is a requirements document. It is a preparation for defining > > > a single IPFIX standard protocol. Therefore, I do not see > a problem in > > > leaving some (potentially tricky) issues to be defined by the > > > protocol. > > > > > > Also, this list was intended to name and briefly explain the > > > attributes. > > > For the multicast replication factor, a definition takes > > > several more words: > > > It is a dynamic property. What would be its value, if during > > > the time since > > > the last report there were five outgoing packets per incoming > > > packet and > > > just before the report was made, one receiver disconnected? > > > Would still 4 be correct for this report? Should it be 4.96? Or 5? > > > We has this discussion and found no requirement to prefer one > > > or the other. > > > > > > > > > > > Also, at the end of 6.1 it kind of says "Plus other stuff > > > related to inter-AS > > > > routing if you want". It's already in the MAY section; > > > can't they just > > > > list some concrete items related to inter-AS routing as > 27, 28, ...? > > > > I don't think it's useful to say what amounts to "plus > > > anything else that > > > > you might feel like adding that you think is relevant", > > > since that doesn't > > > > encourage different people to make the same choices. > > > > > > Well, the statement was not THAT general. When discussing > > > the BGP issues, we > > > found that source AS#, destination AS#, previous hop AS#, and > > > next hop AS# could > > > be useful. Shall we list all of them as 27, 28, 29, 30? > > > > > > > --- > > > > > > > > This is a good document overall. > > > > > > > > The applications include usage-based accounting. They > > > should cite RFC 2975, > > > > and indicate that the particular niche for usage-based > > > accounting would not > > > > be met if reliability was not very high. Section 5.1 > > > reliability requirements > > > > do not meet the needs, nor do the timestamping of first and > > > last packet of > > > > a flow, because the flow may have been mischaracterized and > > > that is first and > > > > last of different flows. The way to satisfy this issue is > > > to caveat the > > > > discussion of usage-based accounting with strong words on > > > reliability that > > > > counter the comments on lesser reliability and more > > > probabilistic techniques > > > > for other applications of ipfix. > > > > > > Citing RFC 2975 is fine. > > > > > > But the WG could not agree on a statement, that this niche > > > would not be met > > > if reliability was not very high. We found non-neglectible > > > counter-examples. > > > > > > I think there will be consensus on a statement like "Some > > > accounting applications > > > require high reliability". > > > > > > > Requirements on the ipfix implementation - the document is > > > long and I wonder if > > > > the working group really meant the protocol's > > > confidentiality and anonymization > > > > features to be so optional - SHOULD confidentiality, MAY > > > anonymization. > > > > Just for implementation. > > > > > > Sorry, what is the proposal here? > > > > > > Juergen > > > > > > > -30- > > > > > > > > > > > > -- > > > > Help mailto:majordomo@net.doit.wisc.edu and say > > > "help" in message body > > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > > "unsubscribe ipfix" in message body > > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > > > > > > > -- > > > Help mailto:majordomo@net.doit.wisc.edu and say "help" > > > in message body > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > "unsubscribe ipfix" in message body > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > > -- > > Help mailto:majordomo@net.doit.wisc.edu and say > "help" in message body > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe ipfix" in message body > > Archive http://ipfix.doit.wisc.edu/archive/ > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Sat Apr 12 20:13:07 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA08157 for ; Sat, 12 Apr 2003 20:13:06 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 194ULv-0004La-00 for ipfix-list@mil.doit.wisc.edu; Sat, 12 Apr 2003 18:24:07 -0500 Received: from eng4.oar.net ([192.148.244.24]) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 194ULt-0004LT-00 for ipfix@net.doit.wisc.edu; Sat, 12 Apr 2003 18:24:05 -0500 Received: (qmail 29604 invoked by uid 4454); 12 Apr 2003 23:24:05 -0000 Date: Sat, 12 Apr 2003 19:24:05 -0400 From: Mark Fullmer To: Mark Thibodeau Cc: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] iesg comment on draft-ietf-ipfix-reqs-09.txt Message-ID: <20030412192404.A29592@net.ohio-state.edu> References: <1D3D2C371FCBD947A7897FABBD3533A566B99F@xsun01.ptp.hp.com> <20030411001510.A16691@net.ohio-state.edu> <3E96BC76.75C7588A@alcatel.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: <3E96BC76.75C7588A@alcatel.com>; from mark.thibodeau@alcatel.com on Fri, Apr 11, 2003 at 09:00:39AM -0400 Precedence: bulk Sender: majordomo listserver It depends on the transport. SCTP has a heartbeat option which unlike TCP's keepalive is easy for an application to change. mark On Fri, Apr 11, 2003 at 09:00:39AM -0400, Mark Thibodeau wrote: > Hi Mark, > > Good points. > > But, why do you say "routers probably won't send this option"? Shouldn't the behavior > of the router be similar to that of the standalone probe? I.e. shouldn't the router > handle collector failover? For the router to detect a collector failure it will also > need these "hello's". > > Cheers, > Mark > > Mark Fullmer wrote: > > > Why are we rehashing this again? NetFlow v9 can run over a reliable transport > > such as TCP or SCTP. The only part that's missing is a way for the collector > > to inform the exporter at failover time what flows to resend, which _must_ > > be an optional feature for the exporter. > > > > Consider: > > > > Exporter sends a "I expect a hello every N seconds" to the collector > > as an option, using the existing NetFlow v9 option template. > > > > Collector sees the above message and ACK's with the highest sequence > > number PDU it has processed every N seconds for each observation domain. > > Collector could even use the same packet format. > > > > If the exporter doesn't get an ACK in N+holdown seconds it switches to > > the secondary and retransmits PDU's that have not been ACK'd, iff it > > has the resources to buffer them. > > > > There's no negotiation, this feature is entirely controlled by the exporter > > (routers probably won't send this option, stand alone probes that have > > the resources probably will), and it's done within the current framework > > of NetFlow v9. > > > > NetFlow v9 != IPFIX. There will be some _small_ changes. > > > > The burden is going to be on the collector to support multiple transports, > > a reliable transport would allow something like above, an unreliable > > transport obviously won't. > > > > mark > > > > On Tue, Apr 08, 2003 at 07:47:58PM -0400, MEYER,JEFFREY D (HP-Cupertino,ex1) wrote: > > > Hi, > > > > > > As to the last point, (accounting requirements), if we are talking about > > > carrier grade accounting for billing purposes, then definitely NFv9 > > > continues > > > to have the same problems as all previous versions, i.e. if a packet is lost > > > in transit it's lost, too bad. The fact that many carriers do bill with NF > > > doesn't mean they are happy about this behavior. > > > > > > So, I do think it is important to point this out. Since the lack of > > > addressing > > > this requirement implies IPDR will need to go ahead independently to address > > > this important requirement. > > > > > > Also I'd include references to the two RFC's I mentioned earlier in the > > > references section as they include some useful discussions of prior > > > learnings > > > in this area: > > > > > > RFC2975 - Introduction to Accounting Management > > > RFC2924 - Accounting Attributes and Record Formats > > > > > > Regards, > > > > > > Jeff Meyer > > > > > > > -----Original Message----- > > > > From: Juergen Quittek [mailto:quittek@ccrle.nec.de] > > > > Sent: Friday, April 04, 2003 8:07 AM > > > > To: Randy Bush; ipfix@net.doit.wisc.edu > > > > Subject: Re: [ipfix] iesg comment on draft-ietf-ipfix-reqs-09.txt > > > > > > > > > > > > Hi all, > > > > > > > > I think we got many good comments. Most of them can be > > > > committed quickly. > > > > However, there are some that might require some discussion: > > > > > > > > - Does the Appendix need to be moved up in the document? > > > > - Shall we be more clear on byte counter, multicast > > > > replication factor, > > > > and BGP AS#s? > > > > - Do we agree that we need to add a statement on that > > > > accounting requirements > > > > can only be met with higher reliability? > > > > > > > > I would answer 'no' on all three questions. Please see some > > > > more detailed > > > > statement on the comments below. > > > > > > > > Juergen > > > > -- > > > > Juergen Quittek quittek@ccrle.nec.de Tel: +49 > > > > 6221 90511-15 > > > > NEC Europe Ltd., Network Laboratories Fax: +49 > > > > 6221 90511-55 > > > > Kurfuersten-Anlage 36, 69115 Heidelberg, Germany > > > > http://www.ccrle.nec.de > > > > > > > > > > > > -- Randy Bush wrote on 03 April 2003 09:01 -0800: > > > > > > > > > > comments from iesg reviewers > > > > > > > > > > randy > > > > > > > > > > --- > > > > > > > > > > I see: > > > > > 5.4. Timestamps > > > > > > > > > > The metering process MUST be able to generate > > > > timestamps for the > > > > > first and the last observation of a packet of a flow at the > > > > > observation point. The timestamp resolution MUST be at > > > > least the one > > > > > of the sysUpTime [RFC1213], which is one centisecond. > > > > > > > > > > I think I'd rather see a reference to RFC3418 instead of RFC1213. > > > > > We can do so by RFC-Editor note. May want to check with authors too. > > > > > > > > Certainly we should follow this suggestion. > > > > > > > > > On page 14 I see: > > > > > > > > > > > > > > > 9. type of service octet (in case of IPv4), traffic class > > > > > octet (in case of IPv6) > > > > > > > > > > Is that not better state as > > > > > > > > > > 9. DSCP, which is set in the type of service octet > > > > (in case of IPv4), > > > > > or in the traffic class octet (in case of IPv6) > > > > > > > > We discussed this before: According to RFC 2474, the DSCP > > > > covers only 6 bit > > > > of the type of service octet, and in some cases you want to > > > > see the entire byte. > > > > > > > > > > > > > > --- > > > > > > > > > > I would prefer to see the paragraph referencing RFC 2119 > > > > in section 2. > > > > > > > > Makes sense. > > > > > > > > > Last paragraph in section 6.3.3 needs to be reworded. > > > > > > > > Let's try to find another wording. > > > > > > > > > In section 10.2, an important point is left unsaid. If > > > > the injection of > > > > > IPFIX traffic fool the network provided into thinking that > > > > a DOS attack is > > > > > underway, the countermeasures employed by the network > > > > provider may actually > > > > > deny service to real customers. > > > > > > > > Good point. We should discuss this threat also in some > > > > additional text. > > > > > > > > > The appendix needs to be moved up in the document. > > > > > > > > Can anybody explain why? > > > > > > > > > --- > > > > > > > > > > Overall, I like it. A couple of things jumped out at me, though. > > > > > This document goes to a lot of trouble to say exactly what is > > > > > exported (e.g. what fields, etc), except for two cases: > > > > > > > > > > 8. byte counter > > > > > Which bytes of a packet are counted MUST be > > > > defined exactly. > > > > > > > > > > 20. multicast replication factor > > > > > the number of outgoing packets originating from a single > > > > > incoming multicast packet. This is a dynamic property of > > > > > multicast flows, that may change over time. For > > > > unicast flows > > > > > it has the constant value 1. The computation of > > > > the factor MUST > > > > > be clearly defined. > > > > > > > > > > This document is defining what fields are being reported on; > > > > > why can't it also define these two things that must also be > > > > > defined? Is it useful to have different ipfix protocols exporting > > > > > different measurements? > > > > > > > > This is a requirements document. It is a preparation for defining > > > > a single IPFIX standard protocol. Therefore, I do not see a problem in > > > > leaving some (potentially tricky) issues to be defined by the > > > > protocol. > > > > > > > > Also, this list was intended to name and briefly explain the > > > > attributes. > > > > For the multicast replication factor, a definition takes > > > > several more words: > > > > It is a dynamic property. What would be its value, if during > > > > the time since > > > > the last report there were five outgoing packets per incoming > > > > packet and > > > > just before the report was made, one receiver disconnected? > > > > Would still 4 be correct for this report? Should it be 4.96? Or 5? > > > > We has this discussion and found no requirement to prefer one > > > > or the other. > > > > > > > > > > > > > > Also, at the end of 6.1 it kind of says "Plus other stuff > > > > related to inter-AS > > > > > routing if you want". It's already in the MAY section; > > > > can't they just > > > > > list some concrete items related to inter-AS routing as 27, 28, ...? > > > > > I don't think it's useful to say what amounts to "plus > > > > anything else that > > > > > you might feel like adding that you think is relevant", > > > > since that doesn't > > > > > encourage different people to make the same choices. > > > > > > > > Well, the statement was not THAT general. When discussing > > > > the BGP issues, we > > > > found that source AS#, destination AS#, previous hop AS#, and > > > > next hop AS# could > > > > be useful. Shall we list all of them as 27, 28, 29, 30? > > > > > > > > > --- > > > > > > > > > > This is a good document overall. > > > > > > > > > > The applications include usage-based accounting. They > > > > should cite RFC 2975, > > > > > and indicate that the particular niche for usage-based > > > > accounting would not > > > > > be met if reliability was not very high. Section 5.1 > > > > reliability requirements > > > > > do not meet the needs, nor do the timestamping of first and > > > > last packet of > > > > > a flow, because the flow may have been mischaracterized and > > > > that is first and > > > > > last of different flows. The way to satisfy this issue is > > > > to caveat the > > > > > discussion of usage-based accounting with strong words on > > > > reliability that > > > > > counter the comments on lesser reliability and more > > > > probabilistic techniques > > > > > for other applications of ipfix. > > > > > > > > Citing RFC 2975 is fine. > > > > > > > > But the WG could not agree on a statement, that this niche > > > > would not be met > > > > if reliability was not very high. We found non-neglectible > > > > counter-examples. > > > > > > > > I think there will be consensus on a statement like "Some > > > > accounting applications > > > > require high reliability". > > > > > > > > > Requirements on the ipfix implementation - the document is > > > > long and I wonder if > > > > > the working group really meant the protocol's > > > > confidentiality and anonymization > > > > > features to be so optional - SHOULD confidentiality, MAY > > > > anonymization. > > > > > Just for implementation. > > > > > > > > Sorry, what is the proposal here? > > > > > > > > Juergen > > > > > > > > > -30- > > > > > > > > > > > > > > > -- > > > > > Help mailto:majordomo@net.doit.wisc.edu and say > > > > "help" in message body > > > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > > > "unsubscribe ipfix" in message body > > > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > > > > > > > > > > > -- > > > > Help mailto:majordomo@net.doit.wisc.edu and say "help" > > > > in message body > > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > > "unsubscribe ipfix" in message body > > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > > > > > -- > > > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > "unsubscribe ipfix" in message body > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > -- > > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe ipfix" in message body > > Archive http://ipfix.doit.wisc.edu/archive/ > > -- > > **************************************************** > * Mark Thibodeau, P Eng. > * 670 RSP Development - Firmware Designer > * Alcatel CID - (613) 784-5375 > * Fax - (613) 599-3696 > * mark.thibodeau@alcatel.com > **************************************************** > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Sat Apr 12 20:23:02 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA08289 for ; Sat, 12 Apr 2003 20:23:01 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 194Ubk-0004et-00 for ipfix-list@mil.doit.wisc.edu; Sat, 12 Apr 2003 18:40:28 -0500 Received: from eng4.oar.net ([192.148.244.24]) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 194Ubi-0004em-00 for ipfix@net.doit.wisc.edu; Sat, 12 Apr 2003 18:40:26 -0500 Received: (qmail 29646 invoked by uid 4454); 12 Apr 2003 23:40:25 -0000 Date: Sat, 12 Apr 2003 19:40:25 -0400 From: Mark Fullmer To: "MEYER,JEFFREY D \(HP-Cupertino,ex1\)" Cc: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] iesg comment on draft-ietf-ipfix-reqs-09.txt Message-ID: <20030412194025.B29592@net.ohio-state.edu> References: <1D3D2C371FCBD947A7897FABBD3533A566B9CC@xsun01.ptp.hp.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: <1D3D2C371FCBD947A7897FABBD3533A566B9CC@xsun01.ptp.hp.com>; from jeff.meyer2@hp.com on Fri, Apr 11, 2003 at 09:24:55AM -0700 Precedence: bulk Sender: majordomo listserver You're referring to in-flight data that the exporter has sent with a write()/sendto() that the collector hasn't processed yet, correct? The issue being the exporter can not know the data has made it to the collector without application level ACK's? My read of the bidirectional argument stemmed from people wanting IPFIX to include the ability to configure the export and metering process. Assuming a reliable transport that guarantees in-order delivery of records, do you feel an optional periodic keep-alive from the collector which includes the packet sequence number as an indicator of processed (stored to disk, whatever) data sufficient? This doesn't preclude an implementation from not using the keep-alive's and/or using an unreliable transport. mark On Fri, Apr 11, 2003 at 09:24:55AM -0700, MEYER,JEFFREY D (HP-Cupertino,ex1) wrote: > Hi, > > I'm not rehashing this, simply pointing out that a reliable > transport is not the same thing as guaranteed delivery. For > some reason people don't seem to accept this. Don't take my > word for it, see RFC2975 section 2.1.5 "Application acknowledgements". > > Since NF/IPFIX as defined today is unidirectional, the server > has NO way of knowing what may have been lost in transit. > > One can wish or pretend this isn't a problem, but it just > won't go away. > > For some reasons, providers who bill on this accounting information > like to have a higher level of confidence that things don't just > "fall on the floor". > > > Note: I'm resigned to the fact that IPFIX will not enable this > capability, and one could argue that this is a different class > of accounting (or telemetry), and shouldn't be bothered with > such details. Fine. My only point is that there is a significant > market out there that does care about this, and RADIUS and Diameter > are too high overhead to address. So a middle ground between > IPFIX's apparent charter and the AAA groups is needed. Hence, IPDR. > > Alternatively, this could be flagged as "issue #1" in the > joint work on creating an IPFIX protocol. > > Regards, > > Jeff Meyer > > > -----Original Message----- > > From: Mark Fullmer [mailto:maf@eng.oar.net] > > Sent: Thursday, April 10, 2003 9:15 PM > > To: MEYER,JEFFREY D (HP-Cupertino,ex1) > > Cc: ipfix@net.doit.wisc.edu > > Subject: Re: [ipfix] iesg comment on draft-ietf-ipfix-reqs-09.txt > > > > > > Why are we rehashing this again? NetFlow v9 can run over a > > reliable transport > > such as TCP or SCTP. The only part that's missing is a way > > for the collector > > to inform the exporter at failover time what flows to resend, > > which _must_ > > be an optional feature for the exporter. > > > > Consider: > > > > Exporter sends a "I expect a hello every N seconds" to the > > collector > > as an option, using the existing NetFlow v9 option template. > > > > Collector sees the above message and ACK's with the highest sequence > > number PDU it has processed every N seconds for each > > observation domain. > > Collector could even use the same packet format. > > > > If the exporter doesn't get an ACK in N+holdown seconds it > > switches to > > the secondary and retransmits PDU's that have not been ACK'd, iff it > > has the resources to buffer them. > > > > There's no negotiation, this feature is entirely controlled > > by the exporter > > (routers probably won't send this option, stand alone probes that have > > the resources probably will), and it's done within the > > current framework > > of NetFlow v9. > > > > NetFlow v9 != IPFIX. There will be some _small_ changes. > > > > The burden is going to be on the collector to support > > multiple transports, > > a reliable transport would allow something like above, an unreliable > > transport obviously won't. > > > > mark > > > > On Tue, Apr 08, 2003 at 07:47:58PM -0400, MEYER,JEFFREY D > > (HP-Cupertino,ex1) wrote: > > > Hi, > > > > > > As to the last point, (accounting requirements), if we are > > talking about > > > carrier grade accounting for billing purposes, then definitely NFv9 > > > continues > > > to have the same problems as all previous versions, i.e. if > > a packet is lost > > > in transit it's lost, too bad. The fact that many carriers > > do bill with NF > > > doesn't mean they are happy about this behavior. > > > > > > So, I do think it is important to point this out. Since the lack of > > > addressing > > > this requirement implies IPDR will need to go ahead > > independently to address > > > this important requirement. > > > > > > Also I'd include references to the two RFC's I mentioned > > earlier in the > > > references section as they include some useful discussions of prior > > > learnings > > > in this area: > > > > > > RFC2975 - Introduction to Accounting Management > > > RFC2924 - Accounting Attributes and Record Formats > > > > > > Regards, > > > > > > Jeff Meyer > > > > > > > -----Original Message----- > > > > From: Juergen Quittek [mailto:quittek@ccrle.nec.de] > > > > Sent: Friday, April 04, 2003 8:07 AM > > > > To: Randy Bush; ipfix@net.doit.wisc.edu > > > > Subject: Re: [ipfix] iesg comment on draft-ietf-ipfix-reqs-09.txt > > > > > > > > > > > > Hi all, > > > > > > > > I think we got many good comments. Most of them can be > > > > committed quickly. > > > > However, there are some that might require some discussion: > > > > > > > > - Does the Appendix need to be moved up in the document? > > > > - Shall we be more clear on byte counter, multicast > > > > replication factor, > > > > and BGP AS#s? > > > > - Do we agree that we need to add a statement on that > > > > accounting requirements > > > > can only be met with higher reliability? > > > > > > > > I would answer 'no' on all three questions. Please see some > > > > more detailed > > > > statement on the comments below. > > > > > > > > Juergen > > > > -- > > > > Juergen Quittek quittek@ccrle.nec.de Tel: +49 > > > > 6221 90511-15 > > > > NEC Europe Ltd., Network Laboratories Fax: +49 > > > > 6221 90511-55 > > > > Kurfuersten-Anlage 36, 69115 Heidelberg, Germany > > > > http://www.ccrle.nec.de > > > > > > > > > > > > -- Randy Bush wrote on 03 April 2003 09:01 -0800: > > > > > > > > > > comments from iesg reviewers > > > > > > > > > > randy > > > > > > > > > > --- > > > > > > > > > > I see: > > > > > 5.4. Timestamps > > > > > > > > > > The metering process MUST be able to generate > > > > timestamps for the > > > > > first and the last observation of a packet of a flow at the > > > > > observation point. The timestamp resolution MUST be at > > > > least the one > > > > > of the sysUpTime [RFC1213], which is one centisecond. > > > > > > > > > > I think I'd rather see a reference to RFC3418 instead > > of RFC1213. > > > > > We can do so by RFC-Editor note. May want to check with > > authors too. > > > > > > > > Certainly we should follow this suggestion. > > > > > > > > > On page 14 I see: > > > > > > > > > > > > > > > 9. type of service octet (in case of IPv4), traffic class > > > > > octet (in case of IPv6) > > > > > > > > > > Is that not better state as > > > > > > > > > > 9. DSCP, which is set in the type of service octet > > > > (in case of IPv4), > > > > > or in the traffic class octet (in case of IPv6) > > > > > > > > We discussed this before: According to RFC 2474, the DSCP > > > > covers only 6 bit > > > > of the type of service octet, and in some cases you want to > > > > see the entire byte. > > > > > > > > > > > > > > --- > > > > > > > > > > I would prefer to see the paragraph referencing RFC 2119 > > > > in section 2. > > > > > > > > Makes sense. > > > > > > > > > Last paragraph in section 6.3.3 needs to be reworded. > > > > > > > > Let's try to find another wording. > > > > > > > > > In section 10.2, an important point is left unsaid. If > > > > the injection of > > > > > IPFIX traffic fool the network provided into thinking that > > > > a DOS attack is > > > > > underway, the countermeasures employed by the network > > > > provider may actually > > > > > deny service to real customers. > > > > > > > > Good point. We should discuss this threat also in some > > > > additional text. > > > > > > > > > The appendix needs to be moved up in the document. > > > > > > > > Can anybody explain why? > > > > > > > > > --- > > > > > > > > > > Overall, I like it. A couple of things jumped out at > > me, though. > > > > > This document goes to a lot of trouble to say exactly what is > > > > > exported (e.g. what fields, etc), except for two cases: > > > > > > > > > > 8. byte counter > > > > > Which bytes of a packet are counted MUST be > > > > defined exactly. > > > > > > > > > > 20. multicast replication factor > > > > > the number of outgoing packets originating > > from a single > > > > > incoming multicast packet. This is a dynamic > > property of > > > > > multicast flows, that may change over time. For > > > > unicast flows > > > > > it has the constant value 1. The computation of > > > > the factor MUST > > > > > be clearly defined. > > > > > > > > > > This document is defining what fields are being reported on; > > > > > why can't it also define these two things that must also be > > > > > defined? Is it useful to have different ipfix > > protocols exporting > > > > > different measurements? > > > > > > > > This is a requirements document. It is a preparation for defining > > > > a single IPFIX standard protocol. Therefore, I do not see > > a problem in > > > > leaving some (potentially tricky) issues to be defined by the > > > > protocol. > > > > > > > > Also, this list was intended to name and briefly explain the > > > > attributes. > > > > For the multicast replication factor, a definition takes > > > > several more words: > > > > It is a dynamic property. What would be its value, if during > > > > the time since > > > > the last report there were five outgoing packets per incoming > > > > packet and > > > > just before the report was made, one receiver disconnected? > > > > Would still 4 be correct for this report? Should it be 4.96? Or 5? > > > > We has this discussion and found no requirement to prefer one > > > > or the other. > > > > > > > > > > > > > > Also, at the end of 6.1 it kind of says "Plus other stuff > > > > related to inter-AS > > > > > routing if you want". It's already in the MAY section; > > > > can't they just > > > > > list some concrete items related to inter-AS routing as > > 27, 28, ...? > > > > > I don't think it's useful to say what amounts to "plus > > > > anything else that > > > > > you might feel like adding that you think is relevant", > > > > since that doesn't > > > > > encourage different people to make the same choices. > > > > > > > > Well, the statement was not THAT general. When discussing > > > > the BGP issues, we > > > > found that source AS#, destination AS#, previous hop AS#, and > > > > next hop AS# could > > > > be useful. Shall we list all of them as 27, 28, 29, 30? > > > > > > > > > --- > > > > > > > > > > This is a good document overall. > > > > > > > > > > The applications include usage-based accounting. They > > > > should cite RFC 2975, > > > > > and indicate that the particular niche for usage-based > > > > accounting would not > > > > > be met if reliability was not very high. Section 5.1 > > > > reliability requirements > > > > > do not meet the needs, nor do the timestamping of first and > > > > last packet of > > > > > a flow, because the flow may have been mischaracterized and > > > > that is first and > > > > > last of different flows. The way to satisfy this issue is > > > > to caveat the > > > > > discussion of usage-based accounting with strong words on > > > > reliability that > > > > > counter the comments on lesser reliability and more > > > > probabilistic techniques > > > > > for other applications of ipfix. > > > > > > > > Citing RFC 2975 is fine. > > > > > > > > But the WG could not agree on a statement, that this niche > > > > would not be met > > > > if reliability was not very high. We found non-neglectible > > > > counter-examples. > > > > > > > > I think there will be consensus on a statement like "Some > > > > accounting applications > > > > require high reliability". > > > > > > > > > Requirements on the ipfix implementation - the document is > > > > long and I wonder if > > > > > the working group really meant the protocol's > > > > confidentiality and anonymization > > > > > features to be so optional - SHOULD confidentiality, MAY > > > > anonymization. > > > > > Just for implementation. > > > > > > > > Sorry, what is the proposal here? > > > > > > > > Juergen > > > > > > > > > -30- > > > > > > > > > > > > > > > -- > > > > > Help mailto:majordomo@net.doit.wisc.edu and say > > > > "help" in message body > > > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > > > "unsubscribe ipfix" in message body > > > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > > > > > > > > > > > -- > > > > Help mailto:majordomo@net.doit.wisc.edu and say "help" > > > > in message body > > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > > "unsubscribe ipfix" in message body > > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > > > > > -- > > > Help mailto:majordomo@net.doit.wisc.edu and say > > "help" in message body > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > "unsubscribe ipfix" in message body > > > Archive http://ipfix.doit.wisc.edu/archive/ > > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Sun Apr 13 23:23:47 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id XAA11231 for ; Sun, 13 Apr 2003 23:23:46 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 194uEO-0006Lu-00 for ipfix-list@mil.doit.wisc.edu; Sun, 13 Apr 2003 22:02:04 -0500 Received: from atlrel7.hp.com ([156.153.255.213]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 194uEL-0006Lc-00 for ipfix@net.doit.wisc.edu; Sun, 13 Apr 2003 22:02:02 -0500 Received: from xatlrelay2.atl.hp.com (xatlrelay2.atl.hp.com [15.45.89.191]) by atlrel7.hp.com (Postfix) with ESMTP id 5B5181C01228; Sun, 13 Apr 2003 23:02:01 -0400 (EDT) Received: from xatlbh2.atl.hp.com (xatlbh2.atl.hp.com [15.45.89.187]) by xatlrelay2.atl.hp.com (Postfix) with ESMTP id 3D2221C000AC; Sun, 13 Apr 2003 23:02:01 -0400 (EDT) Received: by xatlbh2.atl.hp.com with Internet Mail Service (5.5.2655.55) id <2LS9Y8XG>; Sun, 13 Apr 2003 23:02:01 -0400 Message-ID: <1D3D2C371FCBD947A7897FABBD3533A566B9D9@xsun01.ptp.hp.com> From: "MEYER,JEFFREY D (HP-Cupertino,ex1)" To: "'Mark Fullmer'" , "MEYER,JEFFREY D (HP-Cupertino,ex1)" Cc: ipfix@net.doit.wisc.edu Subject: RE: [ipfix] iesg comment on draft-ietf-ipfix-reqs-09.txt Date: Sun, 13 Apr 2003 23:01:59 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2655.55) Content-Type: text/plain; charset="iso-8859-1" Precedence: bulk Sender: majordomo listserver Mark, You got it. In-flight data's disposition is unknown to the exporter if the collector fails. Application acks or heartbeats are the only way to make this information unambiguous at the exporter side. Note well, however, that some flow records may arrive at collector without having had an ack flow back to the exporter. This means that the exporter SHOULD (or MUST) flag any "retransmissions" of data to collector #2 as potentially being duplicates. Without such a flag the process of duplicate detection becomes computationally much more costly. Finally, I agree that there are a class of applications that really don't care if a "few" usage records don't make it. So making the acks optional is a probably reasonable (and is the current status of the IPFIX protocol). Regards, Jeff Meyer > -----Original Message----- > From: Mark Fullmer [mailto:maf@eng.oar.net] > Sent: Saturday, April 12, 2003 4:40 PM > To: MEYER,JEFFREY D (HP-Cupertino,ex1) > Cc: ipfix@net.doit.wisc.edu > Subject: Re: [ipfix] iesg comment on draft-ietf-ipfix-reqs-09.txt > > > You're referring to in-flight data that the exporter has sent > with a write()/sendto() > that the collector hasn't processed yet, correct? The issue > being the exporter > can not know the data has made it to the collector without > application level ACK's? > > My read of the bidirectional argument stemmed from people > wanting IPFIX to include > the ability to configure the export and metering process. > > Assuming a reliable transport that guarantees in-order > delivery of records, > do you feel an optional periodic keep-alive from the > collector which includes > the packet sequence number as an indicator of processed > (stored to disk, whatever) > data sufficient? This doesn't preclude an implementation > from not using the > keep-alive's and/or using an unreliable transport. > > mark > > On Fri, Apr 11, 2003 at 09:24:55AM -0700, MEYER,JEFFREY D > (HP-Cupertino,ex1) wrote: > > Hi, > > > > I'm not rehashing this, simply pointing out that a reliable > > transport is not the same thing as guaranteed delivery. For > > some reason people don't seem to accept this. Don't take my > > word for it, see RFC2975 section 2.1.5 "Application > acknowledgements". > > > > Since NF/IPFIX as defined today is unidirectional, the server > > has NO way of knowing what may have been lost in transit. > > > > One can wish or pretend this isn't a problem, but it just > > won't go away. > > > > For some reasons, providers who bill on this accounting > information > > like to have a higher level of confidence that things don't just > > "fall on the floor". > > > > > > Note: I'm resigned to the fact that IPFIX will not enable this > > capability, and one could argue that this is a different class > > of accounting (or telemetry), and shouldn't be bothered with > > such details. Fine. My only point is that there is a significant > > market out there that does care about this, and RADIUS and Diameter > > are too high overhead to address. So a middle ground between > > IPFIX's apparent charter and the AAA groups is needed. Hence, IPDR. > > > > Alternatively, this could be flagged as "issue #1" in the > > joint work on creating an IPFIX protocol. > > > > Regards, > > > > Jeff Meyer > > > > > -----Original Message----- > > > From: Mark Fullmer [mailto:maf@eng.oar.net] > > > Sent: Thursday, April 10, 2003 9:15 PM > > > To: MEYER,JEFFREY D (HP-Cupertino,ex1) > > > Cc: ipfix@net.doit.wisc.edu > > > Subject: Re: [ipfix] iesg comment on draft-ietf-ipfix-reqs-09.txt > > > > > > > > > Why are we rehashing this again? NetFlow v9 can run over a > > > reliable transport > > > such as TCP or SCTP. The only part that's missing is a way > > > for the collector > > > to inform the exporter at failover time what flows to resend, > > > which _must_ > > > be an optional feature for the exporter. > > > > > > Consider: > > > > > > Exporter sends a "I expect a hello every N seconds" to the > > > collector > > > as an option, using the existing NetFlow v9 option template. > > > > > > Collector sees the above message and ACK's with the > highest sequence > > > number PDU it has processed every N seconds for each > > > observation domain. > > > Collector could even use the same packet format. > > > > > > If the exporter doesn't get an ACK in N+holdown seconds it > > > switches to > > > the secondary and retransmits PDU's that have not been > ACK'd, iff it > > > has the resources to buffer them. > > > > > > There's no negotiation, this feature is entirely controlled > > > by the exporter > > > (routers probably won't send this option, stand alone > probes that have > > > the resources probably will), and it's done within the > > > current framework > > > of NetFlow v9. > > > > > > NetFlow v9 != IPFIX. There will be some _small_ changes. > > > > > > The burden is going to be on the collector to support > > > multiple transports, > > > a reliable transport would allow something like above, an > unreliable > > > transport obviously won't. > > > > > > mark > > > > > > On Tue, Apr 08, 2003 at 07:47:58PM -0400, MEYER,JEFFREY D > > > (HP-Cupertino,ex1) wrote: > > > > Hi, > > > > > > > > As to the last point, (accounting requirements), if we are > > > talking about > > > > carrier grade accounting for billing purposes, then > definitely NFv9 > > > > continues > > > > to have the same problems as all previous versions, i.e. if > > > a packet is lost > > > > in transit it's lost, too bad. The fact that many carriers > > > do bill with NF > > > > doesn't mean they are happy about this behavior. > > > > > > > > So, I do think it is important to point this out. > Since the lack of > > > > addressing > > > > this requirement implies IPDR will need to go ahead > > > independently to address > > > > this important requirement. > > > > > > > > Also I'd include references to the two RFC's I mentioned > > > earlier in the > > > > references section as they include some useful > discussions of prior > > > > learnings > > > > in this area: > > > > > > > > RFC2975 - Introduction to Accounting Management > > > > RFC2924 - Accounting Attributes and Record Formats > > > > > > > > Regards, > > > > > > > > Jeff Meyer > > > > > > > > > -----Original Message----- > > > > > From: Juergen Quittek [mailto:quittek@ccrle.nec.de] > > > > > Sent: Friday, April 04, 2003 8:07 AM > > > > > To: Randy Bush; ipfix@net.doit.wisc.edu > > > > > Subject: Re: [ipfix] iesg comment on > draft-ietf-ipfix-reqs-09.txt > > > > > > > > > > > > > > > Hi all, > > > > > > > > > > I think we got many good comments. Most of them can be > > > > > committed quickly. > > > > > However, there are some that might require some discussion: > > > > > > > > > > - Does the Appendix need to be moved up in the document? > > > > > - Shall we be more clear on byte counter, multicast > > > > > replication factor, > > > > > and BGP AS#s? > > > > > - Do we agree that we need to add a statement on that > > > > > accounting requirements > > > > > can only be met with higher reliability? > > > > > > > > > > I would answer 'no' on all three questions. Please see some > > > > > more detailed > > > > > statement on the comments below. > > > > > > > > > > Juergen > > > > > -- > > > > > Juergen Quittek quittek@ccrle.nec.de Tel: +49 > > > > > 6221 90511-15 > > > > > NEC Europe Ltd., Network Laboratories Fax: +49 > > > > > 6221 90511-55 > > > > > Kurfuersten-Anlage 36, 69115 Heidelberg, Germany > > > > > http://www.ccrle.nec.de > > > > > > > > > > > > > > > -- Randy Bush wrote on 03 April 2003 09:01 -0800: > > > > > > > > > > > > comments from iesg reviewers > > > > > > > > > > > > randy > > > > > > > > > > > > --- > > > > > > > > > > > > I see: > > > > > > 5.4. Timestamps > > > > > > > > > > > > The metering process MUST be able to generate > > > > > timestamps for the > > > > > > first and the last observation of a packet of > a flow at the > > > > > > observation point. The timestamp resolution MUST be at > > > > > least the one > > > > > > of the sysUpTime [RFC1213], which is one centisecond. > > > > > > > > > > > > I think I'd rather see a reference to RFC3418 instead > > > of RFC1213. > > > > > > We can do so by RFC-Editor note. May want to check with > > > authors too. > > > > > > > > > > Certainly we should follow this suggestion. > > > > > > > > > > > On page 14 I see: > > > > > > > > > > > > > > > > > > 9. type of service octet (in case of IPv4), > traffic class > > > > > > octet (in case of IPv6) > > > > > > > > > > > > Is that not better state as > > > > > > > > > > > > 9. DSCP, which is set in the type of service octet > > > > > (in case of IPv4), > > > > > > or in the traffic class octet (in case of IPv6) > > > > > > > > > > We discussed this before: According to RFC 2474, the DSCP > > > > > covers only 6 bit > > > > > of the type of service octet, and in some cases you want to > > > > > see the entire byte. > > > > > > > > > > > > > > > > > --- > > > > > > > > > > > > I would prefer to see the paragraph referencing RFC 2119 > > > > > in section 2. > > > > > > > > > > Makes sense. > > > > > > > > > > > Last paragraph in section 6.3.3 needs to be reworded. > > > > > > > > > > Let's try to find another wording. > > > > > > > > > > > In section 10.2, an important point is left unsaid. If > > > > > the injection of > > > > > > IPFIX traffic fool the network provided into thinking that > > > > > a DOS attack is > > > > > > underway, the countermeasures employed by the network > > > > > provider may actually > > > > > > deny service to real customers. > > > > > > > > > > Good point. We should discuss this threat also in some > > > > > additional text. > > > > > > > > > > > The appendix needs to be moved up in the document. > > > > > > > > > > Can anybody explain why? > > > > > > > > > > > --- > > > > > > > > > > > > Overall, I like it. A couple of things jumped out at > > > me, though. > > > > > > This document goes to a lot of trouble to say > exactly what is > > > > > > exported (e.g. what fields, etc), except for two cases: > > > > > > > > > > > > 8. byte counter > > > > > > Which bytes of a packet are counted MUST be > > > > > defined exactly. > > > > > > > > > > > > 20. multicast replication factor > > > > > > the number of outgoing packets originating > > > from a single > > > > > > incoming multicast packet. This is a dynamic > > > property of > > > > > > multicast flows, that may change over time. For > > > > > unicast flows > > > > > > it has the constant value 1. The computation of > > > > > the factor MUST > > > > > > be clearly defined. > > > > > > > > > > > > This document is defining what fields are being reported on; > > > > > > why can't it also define these two things that must also be > > > > > > defined? Is it useful to have different ipfix > > > protocols exporting > > > > > > different measurements? > > > > > > > > > > This is a requirements document. It is a preparation > for defining > > > > > a single IPFIX standard protocol. Therefore, I do not see > > > a problem in > > > > > leaving some (potentially tricky) issues to be defined by the > > > > > protocol. > > > > > > > > > > Also, this list was intended to name and briefly explain the > > > > > attributes. > > > > > For the multicast replication factor, a definition takes > > > > > several more words: > > > > > It is a dynamic property. What would be its value, if during > > > > > the time since > > > > > the last report there were five outgoing packets per incoming > > > > > packet and > > > > > just before the report was made, one receiver disconnected? > > > > > Would still 4 be correct for this report? Should it > be 4.96? Or 5? > > > > > We has this discussion and found no requirement to prefer one > > > > > or the other. > > > > > > > > > > > > > > > > > Also, at the end of 6.1 it kind of says "Plus other stuff > > > > > related to inter-AS > > > > > > routing if you want". It's already in the MAY section; > > > > > can't they just > > > > > > list some concrete items related to inter-AS routing as > > > 27, 28, ...? > > > > > > I don't think it's useful to say what amounts to "plus > > > > > anything else that > > > > > > you might feel like adding that you think is relevant", > > > > > since that doesn't > > > > > > encourage different people to make the same choices. > > > > > > > > > > Well, the statement was not THAT general. When discussing > > > > > the BGP issues, we > > > > > found that source AS#, destination AS#, previous hop AS#, and > > > > > next hop AS# could > > > > > be useful. Shall we list all of them as 27, 28, 29, 30? > > > > > > > > > > > --- > > > > > > > > > > > > This is a good document overall. > > > > > > > > > > > > The applications include usage-based accounting. They > > > > > should cite RFC 2975, > > > > > > and indicate that the particular niche for usage-based > > > > > accounting would not > > > > > > be met if reliability was not very high. Section 5.1 > > > > > reliability requirements > > > > > > do not meet the needs, nor do the timestamping of first and > > > > > last packet of > > > > > > a flow, because the flow may have been mischaracterized and > > > > > that is first and > > > > > > last of different flows. The way to satisfy this issue is > > > > > to caveat the > > > > > > discussion of usage-based accounting with strong words on > > > > > reliability that > > > > > > counter the comments on lesser reliability and more > > > > > probabilistic techniques > > > > > > for other applications of ipfix. > > > > > > > > > > Citing RFC 2975 is fine. > > > > > > > > > > But the WG could not agree on a statement, that this niche > > > > > would not be met > > > > > if reliability was not very high. We found non-neglectible > > > > > counter-examples. > > > > > > > > > > I think there will be consensus on a statement like "Some > > > > > accounting applications > > > > > require high reliability". > > > > > > > > > > > Requirements on the ipfix implementation - the document is > > > > > long and I wonder if > > > > > > the working group really meant the protocol's > > > > > confidentiality and anonymization > > > > > > features to be so optional - SHOULD confidentiality, MAY > > > > > anonymization. > > > > > > Just for implementation. > > > > > > > > > > Sorry, what is the proposal here? > > > > > > > > > > Juergen > > > > > > > > > > > -30- > > > > > > > > > > > > > > > > > > -- > > > > > > Help mailto:majordomo@net.doit.wisc.edu and say > > > > > "help" in message body > > > > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > > > > "unsubscribe ipfix" in message body > > > > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > > > > > > > > > > > > > > > -- > > > > > Help mailto:majordomo@net.doit.wisc.edu and say "help" > > > > > in message body > > > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > > > "unsubscribe ipfix" in message body > > > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > > > > > > > > -- > > > > Help mailto:majordomo@net.doit.wisc.edu and say > > > "help" in message body > > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > > "unsubscribe ipfix" in message body > > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" > in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed Apr 16 10:54:24 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA04535 for ; Wed, 16 Apr 2003 10:54:23 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 195o0E-0000qd-00 for ipfix-list@mil.doit.wisc.edu; Wed, 16 Apr 2003 09:35:10 -0500 Received: from weird-brew.cisco.com ([144.254.15.118] helo=strange-brew.cisco.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 195o0C-0000qV-00 for ipfix@net.doit.wisc.edu; Wed, 16 Apr 2003 09:35:09 -0500 Received: from cisco.com (dhcp-peg3-vl30-144-254-7-246.cisco.com [144.254.7.246]) by strange-brew.cisco.com (8.11.6+Sun/8.8.8) with ESMTP id h3GEZ5R23215 for ; Wed, 16 Apr 2003 16:35:05 +0200 (CEST) Message-ID: <3E9D6A18.9070901@cisco.com> Date: Wed, 16 Apr 2003 16:35:04 +0200 From: Benoit Claise User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.3) Gecko/20030312 X-Accept-Language: en-us, en MIME-Version: 1.0 To: ipfix@net.doit.wisc.edu Subject: [ipfix] Variable length fields data type Content-Type: multipart/alternative; boundary="------------030405070701090009010101" Precedence: bulk Sender: majordomo listserver This is a multi-part message in MIME format. --------------030405070701090009010101 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Dear all, During the last IETF, in both IPFIX and PSAMP WGs, it was correctly noted that the current basis protocol for IPFIX has got a limitation regarding the export of variable length data types. So for example for the export of a text or for the export of packet sample (whose length can vary in PSAMP). Here is a proposal: in the template Flowset (template definition), let's put FFFF for the length of the variable length data types. Then, the actual length is encoded in data Flowset (flow records) as follows: - if length is < 255 bytes, it stored in 1 byte - if length is >=255 bytes, 255 is stored in the first byte, and the actual 16 bit length is stored in the next 2 bytes. The cases of length of >= 255 will be very rear. Even PSAMP packet fragments should not have a length above 255 bytes, otherwise we would report just too much paylod and we would have privacy issues. And even in these cases, one extra byte will represent <= 1/255-th part of data record. Why FFFF and not 0? The reason for this is that FFFF is a truely invalid value which should be caught in collectors, by good implementations, today. For 0 this isn't necessary the case as it's meaning is "just" pointless, but not a course of concern for the collectors. Basically FFFF should make the collector (current implementations) discard the template as being invalid. Note also that some option templates data types could potentially have a length of 0. Do we need a length coded on more than 16 bits? I don't think so! Even if PSAMP would decide to report the full packet (not permitted by RFC 2804 btw), the packet length will never need more than 16 bits to be coded as the length in an IP packet is coded on 16 bits. Note also that the Length field in the Data FlowSet Format is coded on 16 bits; so can't exceed a variable length of 64 k - 8 bytes (In the Data FlowSet, the FlowSet ID + the FlowSet Length + 1 byte for 255 + the 16 bits variable length) So the advantages of this proposal are: - code the length on 1 byte for most of the cases - use the 8 bits to code the length, so with a max of 254 bytes And not using the Most Significant Bit for the length: first bit reserved (if 0, then length max of 7 bits = 127 bytes) first bit is 1, then the length is coded on 14 bits max; 7 bits from the 1st bytes and 7 last bits from the second bytes. So max length = 16k for 2 bytes Note: with flow records containing variable length fields, the collecting process will have to decode each flow in the FlowSet before determining how many flow records are part of the Data Flowset ... due to the fact that there is a length field in the data flowsetBut, with flow records not containing variable length fields, the collecting process can deduce the number of the number of flow records in a Data Flowset (by dividing the flow set length by record length) at the time of the packet reception. Any feedback? Regards, Benoit _ _ --------------030405070701090009010101 Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7bit Dear all,

During the last IETF, in both IPFIX and PSAMP WGs, it was correctly noted that the current basis protocol for IPFIX has got a limitation regarding the export of variable length data types. So for example for the export of a text or for the export of packet sample (whose length can vary in PSAMP).

Here is a proposal: in the template Flowset (template definition), let's put FFFF for the length of the variable length data types.
Then, the actual length is encoded in data Flowset (flow records) as follows:
- if length is < 255 bytes, it stored in 1 byte
- if length is >=255 bytes, 255 is stored in the first byte, and the actual
16 bit length is stored in the next 2 bytes.

The cases of length of >= 255 will be very rear. Even PSAMP packet fragments should not have a length above 255 bytes, otherwise we would report just too much paylod and we would have privacy issues.
And even in these cases, one extra byte will represent <= 1/255-th part of data record.

Why FFFF and not 0?
The reason for this is that FFFF is a truely invalid value which should be
caught in collectors, by good implementations, today. For 0 this isn't
necessary the case as it's meaning is "just" pointless, but not a course
of concern for the collectors.
Basically FFFF should make the collector (current implementations) discard the template as being invalid.
Note also that some option templates data types could potentially have a length of 0.

Do we need a length coded on more than 16 bits?
I don't think so!
Even if PSAMP would decide to report the full packet (not permitted by RFC 2804 btw), the packet length will never need more than 16 bits to be coded as the length in an IP packet is
coded on 16 bits. Note also that the Length field in the Data FlowSet
Format is coded on 16 bits; so can't exceed a variable length of 64 k -
8 bytes (In the Data FlowSet, the FlowSet ID + the FlowSet Length + 1 byte for 255 + the
16 bits variable length)

So the advantages of this proposal are:
- code the length on 1 byte for most of the cases
- use the 8 bits to code the length, so with a max of 254 bytes
And not using the Most Significant Bit for the length:
first bit reserved (if 0, then length max of 7 bits = 127 bytes)
first bit is 1, then the length is coded on 14 bits max; 7 bits from
the 1st bytes and 7 last bits from the second bytes. So max length = 16k for 2 bytes

Note: with flow records containing variable length fields, the collecting process will have to decode each flow in the FlowSet before determining how many flow records are part of the Data Flowset ... due to the fact that there is a length field in the data flowsetBut, with flow records not containing variable length fields, the collecting process can deduce the number of the number of flow records in a Data Flowset (by dividing the flow set length by record length) at the time of the packet reception.

Any feedback?

Regards, Benoit

--------------030405070701090009010101-- -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed Apr 16 16:55:28 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA16610 for ; Wed, 16 Apr 2003 16:55:28 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 195tq2-0000ZC-00 for ipfix-list@mil.doit.wisc.edu; Wed, 16 Apr 2003 15:49:02 -0500 Received: from eng4.oar.net ([192.148.244.24]) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 195tq0-0000Z3-00 for ipfix@net.doit.wisc.edu; Wed, 16 Apr 2003 15:49:00 -0500 Received: (qmail 52612 invoked by uid 4454); 16 Apr 2003 20:49:00 -0000 Date: Wed, 16 Apr 2003 16:49:00 -0400 From: Mark Fullmer To: Benoit Claise Cc: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Variable length fields data type Message-ID: <20030416164859.A52559@net.ohio-state.edu> References: <3E9D6A18.9070901@cisco.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: <3E9D6A18.9070901@cisco.com>; from bclaise@cisco.com on Wed, Apr 16, 2003 at 04:35:04PM +0200 Precedence: bulk Sender: majordomo listserver It's MTU barriers that need to be avoided. Is there anything that can move more than a 9000 byte GigE Jumbo frame? I don't think privacy issues should play a role in defending the 16K limit. An implementation would artifically limit the snap length to something much much smaller than 16K if privacy were an issue. The work done to save one byte when the field length is < 254 seems a bit excessive...It probably doesn't matter one way or the other. mark On Wed, Apr 16, 2003 at 04:35:04PM +0200, Benoit Claise wrote: > Dear all, > > During the last IETF, in both IPFIX and PSAMP WGs, it was correctly > noted that the current basis protocol for IPFIX has got a limitation > regarding the export of variable length data types. So for example for > the export of a text or for the export of packet sample (whose length > can vary in PSAMP). > > Here is a proposal: in the template Flowset (template definition), let's put FFFF for the length of the variable length data types. > Then, the actual length is encoded in data Flowset (flow records) as follows: > - if length is < 255 bytes, it stored in 1 byte > - if length is >=255 bytes, 255 is stored in the first byte, and the actual > 16 bit length is stored in the next 2 bytes. > > The cases of length of >= 255 will be very rear. Even PSAMP packet fragments should not have a length above 255 bytes, otherwise we would report just too much paylod and we would have privacy issues. > And even in these cases, one extra byte will represent <= 1/255-th part of data record. > > Why FFFF and not 0? > The reason for this is that FFFF is a truely invalid value which should be > caught in collectors, by good implementations, today. For 0 this isn't > necessary the case as it's meaning is "just" pointless, but not a course > of concern for the collectors. > Basically FFFF should make the collector (current implementations) discard the template as being invalid. > Note also that some option templates data types could potentially have a length of 0. > > Do we need a length coded on more than 16 bits? > I don't think so! > Even if PSAMP would decide to report the full packet (not permitted by RFC 2804 btw), the packet length will never need more than 16 bits to be coded as the length in an IP packet is > coded on 16 bits. Note also that the Length field in the Data FlowSet > Format is coded on 16 bits; so can't exceed a variable length of 64 k - > 8 bytes (In the Data FlowSet, the FlowSet ID + the FlowSet Length + 1 byte for 255 + the > 16 bits variable length) > > So the advantages of this proposal are: > - code the length on 1 byte for most of the cases > - use the 8 bits to code the length, so with a max of 254 bytes > And not using the Most Significant Bit for the length: > first bit reserved (if 0, then length max of 7 bits = 127 bytes) > first bit is 1, then the length is coded on 14 bits max; 7 bits from > the 1st bytes and 7 last bits from the second bytes. So max length = 16k for 2 bytes > > Note: with flow records containing variable length fields, the > collecting process will have to decode each flow in the FlowSet before > determining how many flow records are part of the Data Flowset ... due > to the fact that there is a length field in the data flowsetBut, with > flow records not containing variable length fields, the collecting > process can deduce the number of the number of flow records in a Data > Flowset (by dividing the flow set length by record length) at the time > of the packet reception. > > Any feedback? > > Regards, Benoit > _ > _ > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed Apr 16 17:19:32 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA17259 for ; Wed, 16 Apr 2003 17:19:31 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 195uBI-00011Y-00 for ipfix-list@mil.doit.wisc.edu; Wed, 16 Apr 2003 16:11:00 -0500 Received: from palrel11.hp.com ([156.153.255.246]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 195uBG-00011S-00 for ipfix@net.doit.wisc.edu; Wed, 16 Apr 2003 16:10:58 -0500 Received: from xparelay2.ptp.hp.com (xparelay2.ptp.hp.com [15.1.28.65]) by palrel11.hp.com (Postfix) with ESMTP id B481B1C01A04; Wed, 16 Apr 2003 14:10:57 -0700 (PDT) Received: from xpabh3.ptp.hp.com (xpabh3.ptp.hp.com [15.1.28.63]) by xparelay2.ptp.hp.com (Postfix) with ESMTP id 8C51F1C000BB; Wed, 16 Apr 2003 14:10:57 -0700 (PDT) Received: by xpabh3.ptp.hp.com with Internet Mail Service (5.5.2655.55) id <2JYWMYL9>; Wed, 16 Apr 2003 14:10:57 -0700 Message-ID: <1D3D2C371FCBD947A7897FABBD3533A566B9F3@xsun01.ptp.hp.com> From: "MEYER,JEFFREY D (HP-Cupertino,ex1)" To: "'Benoit Claise'" , ipfix@net.doit.wisc.edu Subject: RE: [ipfix] Variable length fields data type Date: Wed, 16 Apr 2003 14:10:47 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2655.55) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C3045C.A6DB2D30" Precedence: bulk Sender: majordomo listserver This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C3045C.A6DB2D30 Content-Type: text/plain; charset="iso-8859-1" Hi, I'd suggest an alternate approach with regards to the templates. Today the templates carry two pieces of information for each attribute in a flow record: attribute identifier, an integer value from a single namespace, and length. One can still send two factors in the template, but convey more information by sending: attribute identifier, an integer value and a "type code." The type space I would recommend is the following: boolean int8 uint8 int16 uint16 int32 uint32 int64 uint64 ? debateable real32 real64 utf8 octetString ipV4Addr ipV6Addr Each type would have a unique (two octet) identifier, and the lengths are assumed fixed for each type, with the exception of utf8 and octetString. So the encoding would not change in the flow record, and the encoding for utf8 and octetString would use 2 octets (or 4) length field. This helps address extension, where a collector implementation may not recognize the attribute id, but would at least be able to render the field in a more intelligible manner. I.e. just having a length doesn't tell me if I have a signed or unsigned integer or an ipV4Addr. Distinguishing between arbitrary byte sequences (octetString) and printable strings (utf8) would also aid in presentation. For boolean I would encode this as an octet with a value of 0 or 1, to avoid bit level alignment. This would also help the data model. I.e. you define how each of the various primitive types are encoded once and each attribute simply indicates what type it is. Today's document has lot's of repeating text describing how each attribute is encoded. Good engineering practice says that lots of copy paste should be addressed through indirection to a central definition. Regards, Jeff Meyer -----Original Message----- From: Benoit Claise [mailto:bclaise@cisco.com] Sent: Wednesday, April 16, 2003 7:35 AM To: ipfix@net.doit.wisc.edu Subject: [ipfix] Variable length fields data type Dear all, During the last IETF, in both IPFIX and PSAMP WGs, it was correctly noted that the current basis protocol for IPFIX has got a limitation regarding the export of variable length data types. So for example for the export of a text or for the export of packet sample (whose length can vary in PSAMP). Here is a proposal: in the template Flowset (template definition), let's put FFFF for the length of the variable length data types. Then, the actual length is encoded in data Flowset (flow records) as follows: - if length is < 255 bytes, it stored in 1 byte - if length is >=255 bytes, 255 is stored in the first byte, and the actual 16 bit length is stored in the next 2 bytes. The cases of length of >= 255 will be very rear. Even PSAMP packet fragments should not have a length above 255 bytes, otherwise we would report just too much paylod and we would have privacy issues. And even in these cases, one extra byte will represent <= 1/255-th part of data record. Why FFFF and not 0? The reason for this is that FFFF is a truely invalid value which should be caught in collectors, by good implementations, today. For 0 this isn't necessary the case as it's meaning is "just" pointless, but not a course of concern for the collectors. Basically FFFF should make the collector (current implementations) discard the template as being invalid. Note also that some option templates data types could potentially have a length of 0. Do we need a length coded on more than 16 bits? I don't think so! Even if PSAMP would decide to report the full packet (not permitted by RFC 2804 btw), the packet length will never need more than 16 bits to be coded as the length in an IP packet is coded on 16 bits. Note also that the Length field in the Data FlowSet Format is coded on 16 bits; so can't exceed a variable length of 64 k - 8 bytes (In the Data FlowSet, the FlowSet ID + the FlowSet Length + 1 byte for 255 + the 16 bits variable length) So the advantages of this proposal are: - code the length on 1 byte for most of the cases - use the 8 bits to code the length, so with a max of 254 bytes And not using the Most Significant Bit for the length: first bit reserved (if 0, then length max of 7 bits = 127 bytes) first bit is 1, then the length is coded on 14 bits max; 7 bits from the 1st bytes and 7 last bits from the second bytes. So max length = 16k for 2 bytes Note: with flow records containing variable length fields, the collecting process will have to decode each flow in the FlowSet before determining how many flow records are part of the Data Flowset ... due to the fact that there is a length field in the data flowsetBut, with flow records not containing variable length fields, the collecting process can deduce the number of the number of flow records in a Data Flowset (by dividing the flow set length by record length) at the time of the packet reception. Any feedback? Regards, Benoit ------_=_NextPart_001_01C3045C.A6DB2D30 Content-Type: text/html; charset="iso-8859-1"

Hi,

I'd suggest an alternate approach with regards to the templates.

Today the templates carry two pieces of information for each attribute in a flow record: attribute identifier, an integer value from a single namespace, and length.

One can still send two factors in the template, but convey more information by sending: attribute identifier, an integer value and a "type code." The type space I would recommend is the following:

boolean

int8

uint8

int16

uint16

int32

uint32

int64

uint64 ? debateable

real32

real64

utf8

octetString

ipV4Addr

ipV6Addr

Each type would have a unique (two octet) identifier, and the lengths are assumed fixed for each type, with the exception of utf8 and octetString. So the encoding would not change in the flow record, and the encoding for utf8 and octetString would use 2 octets (or 4) length field.

This helps address extension, where a collector implementation may not recognize the attribute id, but would at least be able to render the field in a more intelligible manner. I.e. just having a length doesn't tell me if I have a signed or unsigned integer or an ipV4Addr.

Distinguishing between arbitrary byte sequences (octetString) and printable strings (utf8) would also aid in presentation.

For boolean I would encode this as an octet with a value of 0 or 1, to avoid bit level alignment.

This would also help the data model. I.e. you define how each of the various primitive types are encoded once and each attribute simply indicates what type it is. Today's document has lot's of repeating text describing how each attribute is encoded. Good engineering practice says that lots of copy paste should be addressed through indirection to a central definition.

Regards,

Jeff Meyer

-----Original Message-----
From: Benoit Claise [mailto:bclaise@cisco.com]
Sent: Wednesday, April 16, 2003 7:35 AM
To: ipfix@net.doit.wisc.edu
Subject: [ipfix] Variable length fields data type

Dear all,

During the last IETF, in both IPFIX and PSAMP WGs, it was correctly noted that the current basis protocol for IPFIX has got a limitation regarding the export of variable length data types. So for example for the export of a text or for the export of packet sample (whose length can vary in PSAMP).
Here is a proposal: in the template Flowset (template definition), let's put FFFF for the length of the variable length data types.
Then, the actual length is encoded in data Flowset (flow records) as follows:
- if length is < 255 bytes, it stored in 1 byte
- if length is >=255 bytes, 255 is stored in the first byte, and the actual
  16 bit length is stored in the next 2 bytes.

The cases of length of >= 255 will be very rear. Even PSAMP packet fragments should not have a length above 255 bytes, otherwise we would report just too much paylod and we would have privacy issues.
And even in these cases, one extra byte will represent <= 1/255-th part of data record.

Why FFFF and not 0?
The reason for this is that FFFF is a truely invalid value which should be
caught in collectors, by good implementations, today. For 0 this isn't
necessary the case as it's meaning is "just" pointless, but not a course
of concern for the collectors. 
Basically FFFF should make the collector (current implementations) discard the template as being invalid.
Note also that some option templates data types could potentially have a length of 0.

Do we need a length coded on more than 16 bits?
I don't think so!
Even if PSAMP would decide to report the full packet (not permitted by RFC 2804 btw), the packet length will never need more than 16 bits to be coded as the length in an IP packet is
coded on 16 bits. Note also that the Length field in the Data FlowSet
Format is coded on 16 bits; so can't exceed a variable length of 64 k -
8 bytes (In the Data FlowSet, the FlowSet ID + the FlowSet Length + 1 byte for 255 + the
16 bits variable length)

So the advantages of this proposal are:
- code the length on 1 byte for most of the cases
- use the 8 bits to code the length, so with a max of 254 bytes
  And not using the Most Significant Bit for the length:
    first bit reserved (if 0, then length max of 7 bits = 127 bytes)
    first bit is 1, then the length is coded on 14 bits max; 7 bits from
    the 1st bytes and 7 last bits from the second bytes. So max length = 16k for 2 bytes
Note: with flow records containing variable length fields, the collecting process will have to decode each flow in the FlowSet before determining how many flow records are part of the Data Flowset ... due to the fact that there is a length field in the data flowsetBut, with flow records not containing variable length fields, the collecting process can deduce the number of the number of flow records in a Data Flowset (by dividing the flow set length by record length) at the time of the packet reception.

Any feedback?

Regards, Benoit

------_=_NextPart_001_01C3045C.A6DB2D30-- -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed Apr 16 19:15:48 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA20936 for ; Wed, 16 Apr 2003 19:15:48 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 195w2q-0003Vy-00 for ipfix-list@mil.doit.wisc.edu; Wed, 16 Apr 2003 18:10:24 -0500 Received: from eng4.oar.net ([192.148.244.24]) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 195w2p-0003Vp-00 for ipfix@net.doit.wisc.edu; Wed, 16 Apr 2003 18:10:23 -0500 Received: (qmail 53644 invoked by uid 4454); 16 Apr 2003 23:10:21 -0000 Date: Wed, 16 Apr 2003 19:10:21 -0400 From: Mark Fullmer To: "MEYER,JEFFREY D \(HP-Cupertino,ex1\)" Cc: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Variable length fields data type Message-ID: <20030416191021.A53476@net.ohio-state.edu> References: <1D3D2C371FCBD947A7897FABBD3533A566B9F3@xsun01.ptp.hp.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: <1D3D2C371FCBD947A7897FABBD3533A566B9F3@xsun01.ptp.hp.com>; from jeff.meyer2@hp.com on Wed, Apr 16, 2003 at 02:10:47PM -0700 Precedence: bulk Sender: majordomo listserver If the encoding length is not specified in the template, an unrecognized "type code" received by the collector would result in a template that can not be processed -- more correctly the data records could not be decoded. With the existing Type and Length style encoding unrecognized fields can be ignored and skipped over. For example, if EthernetAddress was used by the exporter and unknown to the collector, the offset of the next field could not be computed because the collector doesn't know how many bytes an EthernetAddress is. mark On Wed, Apr 16, 2003 at 02:10:47PM -0700, MEYER,JEFFREY D (HP-Cupertino,ex1) wrote: > Hi, > > I'd suggest an alternate approach with regards to the templates. > > Today the templates carry two pieces of information for each attribute in > a flow record: attribute identifier, an integer value from a single > namespace, and length. > > One can still send two factors in the template, but convey more > information by sending: attribute identifier, an integer value and a "type > code." The type space I would recommend is the following: > > boolean > int8 > uint8 > int16 > uint16 > int32 > uint32 > int64 > uint64 ? debateable > real32 > real64 > utf8 > octetString > ipV4Addr > ipV6Addr > > Each type would have a unique (two octet) identifier, and the lengths are > assumed fixed for each type, with the exception of utf8 and octetString. So > the encoding would not change in the flow record, and the encoding for utf8 > and octetString would use 2 octets (or 4) length field. > > This helps address extension, where a collector implementation may not > recognize the attribute id, but would at least be able to render the field > in a more intelligible manner. I.e. just having a length doesn't tell me if > I have a signed or unsigned integer or an ipV4Addr. > > Distinguishing between arbitrary byte sequences (octetString) and > printable strings (utf8) would also aid in presentation. > > For boolean I would encode this as an octet with a value of 0 or 1, to > avoid bit level alignment. > > This would also help the data model. I.e. you define how each of the > various primitive types are encoded once and each attribute simply indicates > what type it is. Today's document has lot's of repeating text describing > how each attribute is encoded. Good engineering practice says that lots of > copy paste should be addressed through indirection to a central definition. > > Regards, > > Jeff Meyer > > -----Original Message----- > From: Benoit Claise [mailto:bclaise@cisco.com] > Sent: Wednesday, April 16, 2003 7:35 AM > To: ipfix@net.doit.wisc.edu > Subject: [ipfix] Variable length fields data type > > > Dear all, > > During the last IETF, in both IPFIX and PSAMP WGs, it was correctly noted > that the current basis protocol for IPFIX has got a limitation regarding the > export of variable length data types. So for example for the export of a > text or for the export of packet sample (whose length can vary in PSAMP). > > Here is a proposal: in the template Flowset (template definition), let's put > FFFF for the length of the variable length data types. > > Then, the actual length is encoded in data Flowset (flow records) as > follows: > > - if length is < 255 bytes, it stored in 1 byte > > - if length is >=255 bytes, 255 is stored in the first byte, and the actual > > 16 bit length is stored in the next 2 bytes. > > > > The cases of length of >= 255 will be very rear. Even PSAMP packet fragments > should not have a length above 255 bytes, otherwise we would report just too > much paylod and we would have privacy issues. > > And even in these cases, one extra byte will represent <= 1/255-th part of > data record. > > > > Why FFFF and not 0? > > The reason for this is that FFFF is a truely invalid value which should be > > caught in collectors, by good implementations, today. For 0 this isn't > > necessary the case as it's meaning is "just" pointless, but not a course > > of concern for the collectors. > > Basically FFFF should make the collector (current implementations) discard > the template as being invalid. > > Note also that some option templates data types could potentially have a > length of 0. > > > > Do we need a length coded on more than 16 bits? > > I don't think so! > > Even if PSAMP would decide to report the full packet (not permitted by RFC > 2804 btw), the packet length will never need more than 16 bits to be coded > as the length in an IP packet is > > coded on 16 bits. Note also that the Length field in the Data FlowSet > > Format is coded on 16 bits; so can't exceed a variable length of 64 k - > > 8 bytes (In the Data FlowSet, the FlowSet ID + the FlowSet Length + 1 byte > for 255 + the > > 16 bits variable length) > > > > So the advantages of this proposal are: > > - code the length on 1 byte for most of the cases > > - use the 8 bits to code the length, so with a max of 254 bytes > > And not using the Most Significant Bit for the length: > > first bit reserved (if 0, then length max of 7 bits = 127 bytes) > > first bit is 1, then the length is coded on 14 bits max; 7 bits from > > the 1st bytes and 7 last bits from the second bytes. So max length = 16k > for 2 bytes > Note: with flow records containing variable length fields, the collecting > process will have to decode each flow in the FlowSet before determining how > many flow records are part of the Data Flowset ... due to the fact that > there is a length field in the data flowsetBut, with flow records not > containing variable length fields, the collecting process can deduce the > number of the number of flow records in a Data Flowset (by dividing the > flow set length by record length) at the time of the packet reception. > > Any feedback? > > Regards, Benoit > > > > > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed Apr 16 19:27:30 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA21156 for ; Wed, 16 Apr 2003 19:27:30 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 195wER-0003la-00 for ipfix-list@mil.doit.wisc.edu; Wed, 16 Apr 2003 18:22:23 -0500 Received: from palrel10.hp.com ([156.153.255.245]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 195wEP-0003lT-00 for ipfix@net.doit.wisc.edu; Wed, 16 Apr 2003 18:22:21 -0500 Received: from xparelay1.ptp.hp.com (xparelay1.ptp.hp.com [15.1.28.62]) by palrel10.hp.com (Postfix) with ESMTP id 8D0851C00C32; Wed, 16 Apr 2003 16:22:20 -0700 (PDT) Received: from xpabh1.ptp.hp.com (xpabh1.ptp.hp.com [15.1.28.60]) by xparelay1.ptp.hp.com (Postfix) with ESMTP id 64EA71004BB2; Wed, 16 Apr 2003 16:22:20 -0700 (PDT) Received: by xpabh1.ptp.hp.com with Internet Mail Service (5.5.2655.55) id <2S4ZYR28>; Wed, 16 Apr 2003 16:22:20 -0700 Message-ID: <1D3D2C371FCBD947A7897FABBD3533A566B9F6@xsun01.ptp.hp.com> From: "MEYER,JEFFREY D (HP-Cupertino,ex1)" To: "'Mark Fullmer'" , "MEYER,JEFFREY D (HP-Cupertino,ex1)" Cc: ipfix@net.doit.wisc.edu Subject: RE: [ipfix] Variable length fields data type Date: Wed, 16 Apr 2003 16:22:17 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2655.55) Content-Type: text/plain; charset="iso-8859-1" Precedence: bulk Sender: majordomo listserver Hi, Obviously you need to fix the type codes up front. Ethernet address would be contained in an octetString. Or we decide up front that MACAddr is a special type (6 bytes) in order to save 2 bytes. I suppose if you think that there's going to be lots of new and important fixed length fields which aren't basic numeric quantities, and you're concerned with the 2 byte overhead, you could continue with the NFv9 approach. However, if you think the majority of the data items you'll have in the future fall into the existing categories, you use octetString, as your back door for any subsequent extensions. No ambiguity, things decode fine... jeff > -----Original Message----- > From: Mark Fullmer [mailto:maf@eng.oar.net] > Sent: Wednesday, April 16, 2003 4:10 PM > To: MEYER,JEFFREY D (HP-Cupertino,ex1) > Cc: ipfix@net.doit.wisc.edu > Subject: Re: [ipfix] Variable length fields data type > > > If the encoding length is not specified in the template, an > unrecognized > "type code" received by the collector would result in a template that > can not be processed -- more correctly the data records could not be > decoded. With the existing Type and Length style encoding > unrecognized > fields can be ignored and skipped over. > > For example, if EthernetAddress was used by the exporter and > unknown to > the collector, the offset of the next field could not be computed > because the collector doesn't know how many bytes an > EthernetAddress is. > > mark > > On Wed, Apr 16, 2003 at 02:10:47PM -0700, MEYER,JEFFREY D > (HP-Cupertino,ex1) wrote: > > Hi, > > > > I'd suggest an alternate approach with regards to the templates. > > > > Today the templates carry two pieces of information for > each attribute in > > a flow record: attribute identifier, an integer value from a single > > namespace, and length. > > > > One can still send two factors in the template, but convey more > > information by sending: attribute identifier, an integer > value and a "type > > code." The type space I would recommend is the following: > > > > boolean > > int8 > > uint8 > > int16 > > uint16 > > int32 > > uint32 > > int64 > > uint64 ? debateable > > real32 > > real64 > > utf8 > > octetString > > ipV4Addr > > ipV6Addr > > > > Each type would have a unique (two octet) identifier, and > the lengths are > > assumed fixed for each type, with the exception of utf8 and > octetString. So > > the encoding would not change in the flow record, and the > encoding for utf8 > > and octetString would use 2 octets (or 4) length field. > > > > This helps address extension, where a collector > implementation may not > > recognize the attribute id, but would at least be able to > render the field > > in a more intelligible manner. I.e. just having a length > doesn't tell me if > > I have a signed or unsigned integer or an ipV4Addr. > > > > Distinguishing between arbitrary byte sequences (octetString) and > > printable strings (utf8) would also aid in presentation. > > > > For boolean I would encode this as an octet with a value > of 0 or 1, to > > avoid bit level alignment. > > > > This would also help the data model. I.e. you define how > each of the > > various primitive types are encoded once and each attribute > simply indicates > > what type it is. Today's document has lot's of repeating > text describing > > how each attribute is encoded. Good engineering practice > says that lots of > > copy paste should be addressed through indirection to a > central definition. > > > > Regards, > > > > Jeff Meyer > > > > -----Original Message----- > > From: Benoit Claise [mailto:bclaise@cisco.com] > > Sent: Wednesday, April 16, 2003 7:35 AM > > To: ipfix@net.doit.wisc.edu > > Subject: [ipfix] Variable length fields data type > > > > > > Dear all, > > > > During the last IETF, in both IPFIX and PSAMP WGs, it was > correctly noted > > that the current basis protocol for IPFIX has got a > limitation regarding the > > export of variable length data types. So for example for > the export of a > > text or for the export of packet sample (whose length can > vary in PSAMP). > > > > Here is a proposal: in the template Flowset (template > definition), let's put > > FFFF for the length of the variable length data types. > > > > Then, the actual length is encoded in data Flowset (flow records) as > > follows: > > > > - if length is < 255 bytes, it stored in 1 byte > > > > - if length is >=255 bytes, 255 is stored in the first > byte, and the actual > > > > 16 bit length is stored in the next 2 bytes. > > > > > > > > The cases of length of >= 255 will be very rear. Even PSAMP > packet fragments > > should not have a length above 255 bytes, otherwise we > would report just too > > much paylod and we would have privacy issues. > > > > And even in these cases, one extra byte will represent <= > 1/255-th part of > > data record. > > > > > > > > Why FFFF and not 0? > > > > The reason for this is that FFFF is a truely invalid value > which should be > > > > caught in collectors, by good implementations, today. For 0 > this isn't > > > > necessary the case as it's meaning is "just" pointless, but > not a course > > > > of concern for the collectors. > > > > Basically FFFF should make the collector (current > implementations) discard > > the template as being invalid. > > > > Note also that some option templates data types could > potentially have a > > length of 0. > > > > > > > > Do we need a length coded on more than 16 bits? > > > > I don't think so! > > > > Even if PSAMP would decide to report the full packet (not > permitted by RFC > > 2804 btw), the packet length will never need more than 16 > bits to be coded > > as the length in an IP packet is > > > > coded on 16 bits. Note also that the Length field in the > Data FlowSet > > > > Format is coded on 16 bits; so can't exceed a variable > length of 64 k - > > > > 8 bytes (In the Data FlowSet, the FlowSet ID + the FlowSet > Length + 1 byte > > for 255 + the > > > > 16 bits variable length) > > > > > > > > So the advantages of this proposal are: > > > > - code the length on 1 byte for most of the cases > > > > - use the 8 bits to code the length, so with a max of 254 bytes > > > > And not using the Most Significant Bit for the length: > > > > first bit reserved (if 0, then length max of 7 bits = 127 bytes) > > > > first bit is 1, then the length is coded on 14 bits > max; 7 bits from > > > > the 1st bytes and 7 last bits from the second bytes. So > max length = 16k > > for 2 bytes > > Note: with flow records containing variable length fields, > the collecting > > process will have to decode each flow in the FlowSet before > determining how > > many flow records are part of the Data Flowset ... due to > the fact that > > there is a length field in the data flowsetBut, with flow > records not > > containing variable length fields, the collecting process > can deduce the > > number of the number of flow records in a Data Flowset > (by dividing the > > flow set length by record length) at the time of the packet > reception. > > > > Any feedback? > > > > Regards, Benoit > > > > > > > > > > > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu Apr 17 06:21:24 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id GAA14562 for ; Thu, 17 Apr 2003 06:21:24 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 1966E0-0001zP-00 for ipfix-list@mil.doit.wisc.edu; Thu, 17 Apr 2003 05:02:36 -0500 Received: from weird-brew.cisco.com ([144.254.15.118] helo=strange-brew.cisco.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 1966Dx-0001zK-00 for ipfix@net.doit.wisc.edu; Thu, 17 Apr 2003 05:02:34 -0500 Received: from cisco.com (dhcp-peg3-vl30-144-254-7-246.cisco.com [144.254.7.246]) by strange-brew.cisco.com (8.11.6+Sun/8.8.8) with ESMTP id h3HA2VR11360; Thu, 17 Apr 2003 12:02:31 +0200 (CEST) Message-ID: <3E9E7BB6.4010400@cisco.com> Date: Thu, 17 Apr 2003 12:02:30 +0200 From: Benoit Claise User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.3) Gecko/20030312 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Mark Fullmer CC: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Variable length fields data type References: <3E9D6A18.9070901@cisco.com> <20030416164859.A52559@net.ohio-state.edu> In-Reply-To: <20030416164859.A52559@net.ohio-state.edu> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Mark, >It's MTU barriers that need to be avoided. > Why? Why not let IP fragmentation do its job? >Is there anything that can >move more than a 9000 byte GigE Jumbo frame? > What about 16 Mbit/s Token Ring, 17914 bytes? I don't think this is a good idea to limit the max. size of a variable length data type assuming a layer 2 technology. What technologiy/MTU will we have in the future? > >I don't think privacy issues should play a role in defending the 16K >limit. An implementation would artifically limit the snap length to >something much much smaller than 16K if privacy were an issue. > Agreed. > >The work done to save one byte when the field length is < 254 seems >a bit excessive...It probably doesn't matter one way or the other. > Well, it comes from the experience that having a full metering process (non sampled) could produce a lot of data! Regards, Benoit. > >mark > >On Wed, Apr 16, 2003 at 04:35:04PM +0200, Benoit Claise wrote: > > >>Dear all, >> >>During the last IETF, in both IPFIX and PSAMP WGs, it was correctly >>noted that the current basis protocol for IPFIX has got a limitation >>regarding the export of variable length data types. So for example for >>the export of a text or for the export of packet sample (whose length >>can vary in PSAMP). >> >>Here is a proposal: in the template Flowset (template definition), let's put FFFF for the length of the variable length data types. >>Then, the actual length is encoded in data Flowset (flow records) as follows: >>- if length is < 255 bytes, it stored in 1 byte >>- if length is >=255 bytes, 255 is stored in the first byte, and the actual >> 16 bit length is stored in the next 2 bytes. >> >>The cases of length of >= 255 will be very rear. Even PSAMP packet fragments should not have a length above 255 bytes, otherwise we would report just too much paylod and we would have privacy issues. >>And even in these cases, one extra byte will represent <= 1/255-th part of data record. >> >>Why FFFF and not 0? >>The reason for this is that FFFF is a truely invalid value which should be >>caught in collectors, by good implementations, today. For 0 this isn't >>necessary the case as it's meaning is "just" pointless, but not a course >>of concern for the collectors. >>Basically FFFF should make the collector (current implementations) discard the template as being invalid. >>Note also that some option templates data types could potentially have a length of 0. >> >>Do we need a length coded on more than 16 bits? >>I don't think so! >>Even if PSAMP would decide to report the full packet (not permitted by RFC 2804 btw), the packet length will never need more than 16 bits to be coded as the length in an IP packet is >>coded on 16 bits. Note also that the Length field in the Data FlowSet >>Format is coded on 16 bits; so can't exceed a variable length of 64 k - >>8 bytes (In the Data FlowSet, the FlowSet ID + the FlowSet Length + 1 byte for 255 + the >>16 bits variable length) >> >>So the advantages of this proposal are: >>- code the length on 1 byte for most of the cases >>- use the 8 bits to code the length, so with a max of 254 bytes >> And not using the Most Significant Bit for the length: >> first bit reserved (if 0, then length max of 7 bits = 127 bytes) >> first bit is 1, then the length is coded on 14 bits max; 7 bits from >> the 1st bytes and 7 last bits from the second bytes. So max length = 16k for 2 bytes >> >>Note: with flow records containing variable length fields, the >>collecting process will have to decode each flow in the FlowSet before >>determining how many flow records are part of the Data Flowset ... due >>to the fact that there is a length field in the data flowsetBut, with >>flow records not containing variable length fields, the collecting >>process can deduce the number of the number of flow records in a Data >>Flowset (by dividing the flow set length by record length) at the time >>of the packet reception. >> >>Any feedback? >> >>Regards, Benoit >>_ >>_ >> >> >> -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu Apr 17 11:46:20 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA25776 for ; Thu, 17 Apr 2003 11:46:20 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 196BOM-00019f-00 for ipfix-list@mil.doit.wisc.edu; Thu, 17 Apr 2003 10:33:38 -0500 Received: from eng4.oar.net ([192.148.244.24]) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 196BOL-00019Z-00 for ipfix@net.doit.wisc.edu; Thu, 17 Apr 2003 10:33:37 -0500 Received: (qmail 58157 invoked by uid 4454); 17 Apr 2003 15:33:36 -0000 Date: Thu, 17 Apr 2003 11:33:36 -0400 From: Mark Fullmer To: Benoit Claise Cc: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Variable length fields data type Message-ID: <20030417113336.A58054@net.ohio-state.edu> References: <3E9D6A18.9070901@cisco.com> <20030416164859.A52559@net.ohio-state.edu> <3E9E7BB6.4010400@cisco.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: <3E9E7BB6.4010400@cisco.com>; from bclaise@cisco.com on Thu, Apr 17, 2003 at 12:02:30PM +0200 Precedence: bulk Sender: majordomo listserver On Thu, Apr 17, 2003 at 12:02:30PM +0200, Benoit Claise wrote: > Mark, > > >It's MTU barriers that need to be avoided. > > > Why? Why not let IP fragmentation do its job? The maximum size packet that can be sampled will be largest frame that can be put on the wire, not the maximum sized IP packet, which would be fragmented. > >Is there anything that can > >move more than a 9000 byte GigE Jumbo frame? > > > What about 16 Mbit/s Token Ring, 17914 bytes? Isn't this a good argument that the 16K limit is too small then? > > I don't think this is a good idea to limit the max. size of a variable length data type assuming a layer 2 technology. > What technologiy/MTU will we have in the future? My point was that layer2 MTU's are much less than the the 64K limit of an IP datagram and that a device will be sampling from layer2 where IP datagrams > MTU would be fragmented to something smaller. > >The work done to save one byte when the field length is < 254 seems > >a bit excessive...It probably doesn't matter one way or the other. > > > Well, it comes from the experience that having a full metering process > (non sampled) could produce a lot of data! It's just 8 bits! mark -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu Apr 17 11:51:00 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA26126 for ; Thu, 17 Apr 2003 11:51:00 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 196BX9-0001Kn-00 for ipfix-list@mil.doit.wisc.edu; Thu, 17 Apr 2003 10:42:43 -0500 Received: from eng4.oar.net ([192.148.244.24]) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 196BX8-0001Kh-00 for ipfix@net.doit.wisc.edu; Thu, 17 Apr 2003 10:42:42 -0500 Received: (qmail 58230 invoked by uid 4454); 17 Apr 2003 15:42:42 -0000 Date: Thu, 17 Apr 2003 11:42:42 -0400 From: Mark Fullmer To: "MEYER,JEFFREY D \(HP-Cupertino,ex1\)" Cc: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Variable length fields data type Message-ID: <20030417114242.B58054@net.ohio-state.edu> References: <1D3D2C371FCBD947A7897FABBD3533A566B9F6@xsun01.ptp.hp.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: <1D3D2C371FCBD947A7897FABBD3533A566B9F6@xsun01.ptp.hp.com>; from jeff.meyer2@hp.com on Wed, Apr 16, 2003 at 04:22:17PM -0700 Precedence: bulk Sender: majordomo listserver IMHO, the current v9 Type and Length templates should stay as is. mark On Wed, Apr 16, 2003 at 04:22:17PM -0700, MEYER,JEFFREY D (HP-Cupertino,ex1) wrote: > Hi, > > Obviously you need to fix the type codes up front. Ethernet address > would be contained in an octetString. Or we decide up front that MACAddr > is a special type (6 bytes) in order to save 2 bytes. > > I suppose if you think that there's going to be lots of new and important > fixed length fields which aren't basic numeric quantities, and you're > concerned with the 2 byte overhead, you could continue with the NFv9 > approach. However, if you think the majority of the data items you'll > have in the future fall into the existing categories, you use octetString, > as your back door for any subsequent extensions. No ambiguity, things > decode fine... > > > jeff > > > -----Original Message----- > > From: Mark Fullmer [mailto:maf@eng.oar.net] > > Sent: Wednesday, April 16, 2003 4:10 PM > > To: MEYER,JEFFREY D (HP-Cupertino,ex1) > > Cc: ipfix@net.doit.wisc.edu > > Subject: Re: [ipfix] Variable length fields data type > > > > > > If the encoding length is not specified in the template, an > > unrecognized > > "type code" received by the collector would result in a template that > > can not be processed -- more correctly the data records could not be > > decoded. With the existing Type and Length style encoding > > unrecognized > > fields can be ignored and skipped over. > > > > For example, if EthernetAddress was used by the exporter and > > unknown to > > the collector, the offset of the next field could not be computed > > because the collector doesn't know how many bytes an > > EthernetAddress is. > > > > mark > > > > On Wed, Apr 16, 2003 at 02:10:47PM -0700, MEYER,JEFFREY D > > (HP-Cupertino,ex1) wrote: > > > Hi, > > > > > > I'd suggest an alternate approach with regards to the templates. > > > > > > Today the templates carry two pieces of information for > > each attribute in > > > a flow record: attribute identifier, an integer value from a single > > > namespace, and length. > > > > > > One can still send two factors in the template, but convey more > > > information by sending: attribute identifier, an integer > > value and a "type > > > code." The type space I would recommend is the following: > > > > > > boolean > > > int8 > > > uint8 > > > int16 > > > uint16 > > > int32 > > > uint32 > > > int64 > > > uint64 ? debateable > > > real32 > > > real64 > > > utf8 > > > octetString > > > ipV4Addr > > > ipV6Addr > > > > > > Each type would have a unique (two octet) identifier, and > > the lengths are > > > assumed fixed for each type, with the exception of utf8 and > > octetString. So > > > the encoding would not change in the flow record, and the > > encoding for utf8 > > > and octetString would use 2 octets (or 4) length field. > > > > > > This helps address extension, where a collector > > implementation may not > > > recognize the attribute id, but would at least be able to > > render the field > > > in a more intelligible manner. I.e. just having a length > > doesn't tell me if > > > I have a signed or unsigned integer or an ipV4Addr. > > > > > > Distinguishing between arbitrary byte sequences (octetString) and > > > printable strings (utf8) would also aid in presentation. > > > > > > For boolean I would encode this as an octet with a value > > of 0 or 1, to > > > avoid bit level alignment. > > > > > > This would also help the data model. I.e. you define how > > each of the > > > various primitive types are encoded once and each attribute > > simply indicates > > > what type it is. Today's document has lot's of repeating > > text describing > > > how each attribute is encoded. Good engineering practice > > says that lots of > > > copy paste should be addressed through indirection to a > > central definition. > > > > > > Regards, > > > > > > Jeff Meyer > > > > > > -----Original Message----- > > > From: Benoit Claise [mailto:bclaise@cisco.com] > > > Sent: Wednesday, April 16, 2003 7:35 AM > > > To: ipfix@net.doit.wisc.edu > > > Subject: [ipfix] Variable length fields data type > > > > > > > > > Dear all, > > > > > > During the last IETF, in both IPFIX and PSAMP WGs, it was > > correctly noted > > > that the current basis protocol for IPFIX has got a > > limitation regarding the > > > export of variable length data types. So for example for > > the export of a > > > text or for the export of packet sample (whose length can > > vary in PSAMP). > > > > > > Here is a proposal: in the template Flowset (template > > definition), let's put > > > FFFF for the length of the variable length data types. > > > > > > Then, the actual length is encoded in data Flowset (flow records) as > > > follows: > > > > > > - if length is < 255 bytes, it stored in 1 byte > > > > > > - if length is >=255 bytes, 255 is stored in the first > > byte, and the actual > > > > > > 16 bit length is stored in the next 2 bytes. > > > > > > > > > > > > The cases of length of >= 255 will be very rear. Even PSAMP > > packet fragments > > > should not have a length above 255 bytes, otherwise we > > would report just too > > > much paylod and we would have privacy issues. > > > > > > And even in these cases, one extra byte will represent <= > > 1/255-th part of > > > data record. > > > > > > > > > > > > Why FFFF and not 0? > > > > > > The reason for this is that FFFF is a truely invalid value > > which should be > > > > > > caught in collectors, by good implementations, today. For 0 > > this isn't > > > > > > necessary the case as it's meaning is "just" pointless, but > > not a course > > > > > > of concern for the collectors. > > > > > > Basically FFFF should make the collector (current > > implementations) discard > > > the template as being invalid. > > > > > > Note also that some option templates data types could > > potentially have a > > > length of 0. > > > > > > > > > > > > Do we need a length coded on more than 16 bits? > > > > > > I don't think so! > > > > > > Even if PSAMP would decide to report the full packet (not > > permitted by RFC > > > 2804 btw), the packet length will never need more than 16 > > bits to be coded > > > as the length in an IP packet is > > > > > > coded on 16 bits. Note also that the Length field in the > > Data FlowSet > > > > > > Format is coded on 16 bits; so can't exceed a variable > > length of 64 k - > > > > > > 8 bytes (In the Data FlowSet, the FlowSet ID + the FlowSet > > Length + 1 byte > > > for 255 + the > > > > > > 16 bits variable length) > > > > > > > > > > > > So the advantages of this proposal are: > > > > > > - code the length on 1 byte for most of the cases > > > > > > - use the 8 bits to code the length, so with a max of 254 bytes > > > > > > And not using the Most Significant Bit for the length: > > > > > > first bit reserved (if 0, then length max of 7 bits = 127 bytes) > > > > > > first bit is 1, then the length is coded on 14 bits > > max; 7 bits from > > > > > > the 1st bytes and 7 last bits from the second bytes. So > > max length = 16k > > > for 2 bytes > > > Note: with flow records containing variable length fields, > > the collecting > > > process will have to decode each flow in the FlowSet before > > determining how > > > many flow records are part of the Data Flowset ... due to > > the fact that > > > there is a length field in the data flowsetBut, with flow > > records not > > > containing variable length fields, the collecting process > > can deduce the > > > number of the number of flow records in a Data Flowset > > (by dividing the > > > flow set length by record length) at the time of the packet > > reception. > > > > > > Any feedback? > > > > > > Regards, Benoit > > > > > > > > > > > > > > > > > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu Apr 17 11:53:51 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA26335 for ; Thu, 17 Apr 2003 11:53:51 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 196BdC-0001UW-00 for ipfix-list@mil.doit.wisc.edu; Thu, 17 Apr 2003 10:48:58 -0500 Received: from palrel10.hp.com ([156.153.255.245]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 196BdB-0001UP-00 for ipfix@net.doit.wisc.edu; Thu, 17 Apr 2003 10:48:57 -0500 Received: from xparelay2.ptp.hp.com (xparelay2.ptp.hp.com [15.1.28.65]) by palrel10.hp.com (Postfix) with ESMTP id 735051C0219D; Thu, 17 Apr 2003 08:48:56 -0700 (PDT) Received: from xpabh1.ptp.hp.com (xpabh1.ptp.hp.com [15.1.28.60]) by xparelay2.ptp.hp.com (Postfix) with ESMTP id 41CAE1C00A95; Thu, 17 Apr 2003 08:48:56 -0700 (PDT) Received: by xpabh1.ptp.hp.com with Internet Mail Service (5.5.2655.55) id <2S4Z5VQ5>; Thu, 17 Apr 2003 08:48:56 -0700 Message-ID: <1D3D2C371FCBD947A7897FABBD3533A566B9FA@xsun01.ptp.hp.com> From: "MEYER,JEFFREY D (HP-Cupertino,ex1)" To: "'Mark Fullmer'" , "MEYER,JEFFREY D (HP-Cupertino,ex1)" Cc: ipfix@net.doit.wisc.edu Subject: RE: [ipfix] Variable length fields data type Date: Thu, 17 Apr 2003 08:48:54 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2655.55) Content-Type: text/plain; charset="iso-8859-1" Precedence: bulk Sender: majordomo listserver Because... > -----Original Message----- > From: Mark Fullmer [mailto:maf@eng.oar.net] > Sent: Thursday, April 17, 2003 8:43 AM > To: MEYER,JEFFREY D (HP-Cupertino,ex1) > Cc: ipfix@net.doit.wisc.edu > Subject: Re: [ipfix] Variable length fields data type > > > IMHO, the current v9 Type and Length templates should stay as is. > > mark > > On Wed, Apr 16, 2003 at 04:22:17PM -0700, MEYER,JEFFREY D > (HP-Cupertino,ex1) wrote: > > Hi, > > > > Obviously you need to fix the type codes up front. Ethernet address > > would be contained in an octetString. Or we decide up > front that MACAddr > > is a special type (6 bytes) in order to save 2 bytes. > > > > I suppose if you think that there's going to be lots of new > and important > > fixed length fields which aren't basic numeric quantities, > and you're > > concerned with the 2 byte overhead, you could continue with the NFv9 > > approach. However, if you think the majority of the data > items you'll > > have in the future fall into the existing categories, you > use octetString, > > as your back door for any subsequent extensions. No > ambiguity, things > > decode fine... > > > > > > jeff > > > > > -----Original Message----- > > > From: Mark Fullmer [mailto:maf@eng.oar.net] > > > Sent: Wednesday, April 16, 2003 4:10 PM > > > To: MEYER,JEFFREY D (HP-Cupertino,ex1) > > > Cc: ipfix@net.doit.wisc.edu > > > Subject: Re: [ipfix] Variable length fields data type > > > > > > > > > If the encoding length is not specified in the template, an > > > unrecognized > > > "type code" received by the collector would result in a > template that > > > can not be processed -- more correctly the data records > could not be > > > decoded. With the existing Type and Length style encoding > > > unrecognized > > > fields can be ignored and skipped over. > > > > > > For example, if EthernetAddress was used by the exporter and > > > unknown to > > > the collector, the offset of the next field could not be computed > > > because the collector doesn't know how many bytes an > > > EthernetAddress is. > > > > > > mark > > > > > > On Wed, Apr 16, 2003 at 02:10:47PM -0700, MEYER,JEFFREY D > > > (HP-Cupertino,ex1) wrote: > > > > Hi, > > > > > > > > I'd suggest an alternate approach with regards to the > templates. > > > > > > > > Today the templates carry two pieces of information for > > > each attribute in > > > > a flow record: attribute identifier, an integer value > from a single > > > > namespace, and length. > > > > > > > > One can still send two factors in the template, but > convey more > > > > information by sending: attribute identifier, an integer > > > value and a "type > > > > code." The type space I would recommend is the following: > > > > > > > > boolean > > > > int8 > > > > uint8 > > > > int16 > > > > uint16 > > > > int32 > > > > uint32 > > > > int64 > > > > uint64 ? debateable > > > > real32 > > > > real64 > > > > utf8 > > > > octetString > > > > ipV4Addr > > > > ipV6Addr > > > > > > > > Each type would have a unique (two octet) identifier, and > > > the lengths are > > > > assumed fixed for each type, with the exception of utf8 and > > > octetString. So > > > > the encoding would not change in the flow record, and the > > > encoding for utf8 > > > > and octetString would use 2 octets (or 4) length field. > > > > > > > > This helps address extension, where a collector > > > implementation may not > > > > recognize the attribute id, but would at least be able to > > > render the field > > > > in a more intelligible manner. I.e. just having a length > > > doesn't tell me if > > > > I have a signed or unsigned integer or an ipV4Addr. > > > > > > > > Distinguishing between arbitrary byte sequences > (octetString) and > > > > printable strings (utf8) would also aid in presentation. > > > > > > > > For boolean I would encode this as an octet with a value > > > of 0 or 1, to > > > > avoid bit level alignment. > > > > > > > > This would also help the data model. I.e. you define how > > > each of the > > > > various primitive types are encoded once and each attribute > > > simply indicates > > > > what type it is. Today's document has lot's of repeating > > > text describing > > > > how each attribute is encoded. Good engineering practice > > > says that lots of > > > > copy paste should be addressed through indirection to a > > > central definition. > > > > > > > > Regards, > > > > > > > > Jeff Meyer > > > > > > > > -----Original Message----- > > > > From: Benoit Claise [mailto:bclaise@cisco.com] > > > > Sent: Wednesday, April 16, 2003 7:35 AM > > > > To: ipfix@net.doit.wisc.edu > > > > Subject: [ipfix] Variable length fields data type > > > > > > > > > > > > Dear all, > > > > > > > > During the last IETF, in both IPFIX and PSAMP WGs, it was > > > correctly noted > > > > that the current basis protocol for IPFIX has got a > > > limitation regarding the > > > > export of variable length data types. So for example for > > > the export of a > > > > text or for the export of packet sample (whose length can > > > vary in PSAMP). > > > > > > > > Here is a proposal: in the template Flowset (template > > > definition), let's put > > > > FFFF for the length of the variable length data types. > > > > > > > > Then, the actual length is encoded in data Flowset > (flow records) as > > > > follows: > > > > > > > > - if length is < 255 bytes, it stored in 1 byte > > > > > > > > - if length is >=255 bytes, 255 is stored in the first > > > byte, and the actual > > > > > > > > 16 bit length is stored in the next 2 bytes. > > > > > > > > > > > > > > > > The cases of length of >= 255 will be very rear. Even PSAMP > > > packet fragments > > > > should not have a length above 255 bytes, otherwise we > > > would report just too > > > > much paylod and we would have privacy issues. > > > > > > > > And even in these cases, one extra byte will represent <= > > > 1/255-th part of > > > > data record. > > > > > > > > > > > > > > > > Why FFFF and not 0? > > > > > > > > The reason for this is that FFFF is a truely invalid value > > > which should be > > > > > > > > caught in collectors, by good implementations, today. For 0 > > > this isn't > > > > > > > > necessary the case as it's meaning is "just" pointless, but > > > not a course > > > > > > > > of concern for the collectors. > > > > > > > > Basically FFFF should make the collector (current > > > implementations) discard > > > > the template as being invalid. > > > > > > > > Note also that some option templates data types could > > > potentially have a > > > > length of 0. > > > > > > > > > > > > > > > > Do we need a length coded on more than 16 bits? > > > > > > > > I don't think so! > > > > > > > > Even if PSAMP would decide to report the full packet (not > > > permitted by RFC > > > > 2804 btw), the packet length will never need more than 16 > > > bits to be coded > > > > as the length in an IP packet is > > > > > > > > coded on 16 bits. Note also that the Length field in the > > > Data FlowSet > > > > > > > > Format is coded on 16 bits; so can't exceed a variable > > > length of 64 k - > > > > > > > > 8 bytes (In the Data FlowSet, the FlowSet ID + the FlowSet > > > Length + 1 byte > > > > for 255 + the > > > > > > > > 16 bits variable length) > > > > > > > > > > > > > > > > So the advantages of this proposal are: > > > > > > > > - code the length on 1 byte for most of the cases > > > > > > > > - use the 8 bits to code the length, so with a max of 254 bytes > > > > > > > > And not using the Most Significant Bit for the length: > > > > > > > > first bit reserved (if 0, then length max of 7 bits > = 127 bytes) > > > > > > > > first bit is 1, then the length is coded on 14 bits > > > max; 7 bits from > > > > > > > > the 1st bytes and 7 last bits from the second bytes. So > > > max length = 16k > > > > for 2 bytes > > > > Note: with flow records containing variable length fields, > > > the collecting > > > > process will have to decode each flow in the FlowSet before > > > determining how > > > > many flow records are part of the Data Flowset ... due to > > > the fact that > > > > there is a length field in the data flowsetBut, with flow > > > records not > > > > containing variable length fields, the collecting process > > > can deduce the > > > > number of the number of flow records in a Data Flowset > > > (by dividing the > > > > flow set length by record length) at the time of the packet > > > reception. > > > > > > > > Any feedback? > > > > > > > > Regards, Benoit > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > Help mailto:majordomo@net.doit.wisc.edu and say > "help" in message body > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe ipfix" in message body > > Archive http://ipfix.doit.wisc.edu/archive/ > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri Apr 25 12:13:40 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA13413 for ; Fri, 25 Apr 2003 12:13:40 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 1995T1-0005Wd-00 for ipfix-list@mil.doit.wisc.edu; Fri, 25 Apr 2003 10:50:27 -0500 Received: from tokyo.ccrle.nec.de ([195.37.70.2]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 1995Sy-0005WT-00 for ipfix@net.doit.wisc.edu; Fri, 25 Apr 2003 10:50:25 -0500 Received: from venus.office (venus.office [10.1.1.11]) by tokyo.ccrle.nec.de (8.12.9/8.12.8) with ESMTP id h3PFoOVI002993 for ; Fri, 25 Apr 2003 17:50:24 +0200 (CEST) Received: from ccrle.nec.de (molina.office [10.1.1.126]) by venus.office (Postfix on SuSE Linux eMail Server 3.0) with ESMTP id 53DB5BD52 for ; Fri, 25 Apr 2003 17:44:44 +0200 (CEST) Message-ID: <3EA95941.57BFEE44@ccrle.nec.de> Date: Fri, 25 Apr 2003 17:50:25 +0200 From: Maurizio Molina X-Mailer: Mozilla 4.77 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: ipfix@net.doit.wisc.edu Subject: [ipfix] export packet length Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Hi all, I hope not to rise an issue already discussed in the WG. If yes, I apologise in advance. If an IPFIX export packet is carried over TCP, it's not unusual that it gets split across multiple TCP segments. At the collector side, on the contrary, the implementation of the parsing is much simpler if only full export packet are parsed. If the information about the WHOLE export packet were contained in the header, it would be simple to wait until all the bytes composing an export packets are read from the TCP socket before beginning the parsing. Unfortunately, this information isn't currently contained in Netflow 9 packet header (I suppose, due to Netflow's 9 "bias" towards UDP...). Therefore, a parsing implementation having to deal with this "fragentation" problem without this information needs to be much more complicated (e.g. getting step by step the lenght of each flow set, see if it's all there, parse it and go on....). In summary: if IPFIX has to be carried over tcp, having the length of the export packet in the packet header would be beneficial. Regards, Maurizio -- Maurizio Molina Research Staff member Network Laboratories Heidelberg NEC Europe Ltd. Kurfuersten-Anlage 36, 69115 Heidelberg, Germany. Tel. (49)6221 90511-18 Fax: (49)6221 90511-55 e-mail: molina@ccrle.nec.de Web: www.ccrle.nec.de -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri Apr 25 12:42:08 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA14200 for ; Fri, 25 Apr 2003 12:42:07 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19960V-0006JZ-00 for ipfix-list@mil.doit.wisc.edu; Fri, 25 Apr 2003 11:25:03 -0500 Received: from zrc2s0jx.nortelnetworks.com ([47.103.122.112]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19960T-0006JR-00 for ipfix@net.doit.wisc.edu; Fri, 25 Apr 2003 11:25:01 -0500 Received: from zsc3c028.us.nortel.com (zsc3c028.us.nortel.com [47.81.138.28]) by zrc2s0jx.nortelnetworks.com (Switch-2.2.6/Switch-2.2.0) with ESMTP id h3PGOqI10383; Fri, 25 Apr 2003 11:24:52 -0500 (CDT) Received: by zsc3c028.us.nortel.com with Internet Mail Service (5.5.2653.19) id <2R1CDW4Y>; Fri, 25 Apr 2003 09:24:53 -0700 Message-ID: <0A11633F61BD9F40B43ABCC694004F93F18EEC@zsc3c026.us.nortel.com> From: "Reinaldo Penno" To: "'Maurizio Molina'" , ipfix@net.doit.wisc.edu Subject: RE: [ipfix] export packet length Date: Fri, 25 Apr 2003 09:24:51 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C30B47.32055660" Precedence: bulk Sender: majordomo listserver This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C30B47.32055660 Content-Type: text/plain; charset="ISO-8859-1" That's a good idea. Other protocols like HTTP and SIP make it mandatory to include the Content-Length header (which tell the length of the payload) when carried over TCP. It helps a lot when dealing with fragmentation. Regards, > -----Original Message----- > From: Maurizio Molina [mailto:molina@ccrle.nec.de] > Sent: Friday, April 25, 2003 11:50 AM > To: ipfix@net.doit.wisc.edu > Subject: [ipfix] export packet length > > > Hi all, > I hope not to rise an issue already discussed in the WG. If > yes, I apologise in advance. If an IPFIX export packet is > carried over TCP, it's not unusual that it gets split across > multiple TCP segments. At the collector side, on the > contrary, the implementation of the parsing is much simpler > if only full export packet are parsed. If the information > about the WHOLE export packet were contained in the header, > it would be simple to wait until all the bytes composing an > export packets are read from the TCP socket before beginning > the parsing. Unfortunately, this information isn't currently > contained in Netflow 9 packet header (I suppose, due to > Netflow's 9 "bias" towards UDP...). Therefore, a parsing > implementation having to deal with this "fragentation" > problem without this information needs to be much more > complicated (e.g. getting step by step the lenght of each > flow set, see if it's all there, parse it and go on....). In > summary: if IPFIX has to be carried over tcp, having the > length of the export packet in the packet header would be > beneficial. Regards, Maurizio > > > -- > Maurizio Molina > Research Staff member > Network Laboratories Heidelberg > NEC Europe Ltd. > Kurfuersten-Anlage 36, 69115 Heidelberg, Germany. > Tel. (49)6221 90511-18 Fax: (49)6221 90511-55 > e-mail: molina@ccrle.nec.de > Web: www.ccrle.nec.de > > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" > in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ > ------_=_NextPart_001_01C30B47.32055660 Content-Type: text/html; charset="ISO-8859-1" Content-Transfer-Encoding: quoted-printable RE: [ipfix] export packet length

That's a good idea.

Other protocols like HTTP and SIP make it mandatory = to include the Content-Length header (which tell the length of the = payload) when carried over TCP. It helps a lot when dealing with = fragmentation.

Regards,

> -----Original Message-----
> From: Maurizio Molina [mailto:molina@ccrle.nec.de] =
> Sent: Friday, April 25, 2003 11:50 AM
> To: ipfix@net.doit.wisc.edu
> Subject: [ipfix] export packet length
>
>
> Hi all,
> I hope not to rise an issue already discussed = in the WG. If
> yes, I apologise in advance. If an IPFIX export = packet is
> carried over TCP, it's not unusual that it gets = split across
> multiple TCP segments. At the collector side, = on the
> contrary, the implementation of the parsing is = much simpler
> if only full export packet are parsed. If the = information
> about the WHOLE export packet were contained in = the header,
> it would be simple to wait until all the bytes = composing an
> export packets are read from the TCP socket = before beginning
> the parsing. Unfortunately, this information = isn't currently
> contained in Netflow 9 packet header (I = suppose, due to
> Netflow's 9 "bias" towards UDP...). = Therefore, a parsing
> implementation having to deal with this = "fragentation"
> problem without this information needs to be = much more
> complicated (e.g. getting step by step the = lenght of each
> flow set, see if it's all there, parse it and = go on....). In
> summary: if IPFIX has to be carried over tcp, = having the
> length of the export packet in the packet = header would be
> beneficial. Regards, Maurizio
>
>
> --
> Maurizio Molina
> Research Staff member
> Network Laboratories Heidelberg
> NEC Europe Ltd.
> Kurfuersten-Anlage 36, 69115 Heidelberg, = Germany.
> Tel. (49)6221 90511-18 Fax: (49)6221 = 90511-55
> e-mail: molina@ccrle.nec.de
> Web: www.ccrle.nec.de
>
>
>
> --
> Help = mailto:majordomo@net.doit.wi= sc.edu and say "help"
> in message body
> Unsubscribe mailto:majordomo@net.doit.wi= sc.edu and say
> "unsubscribe ipfix" in message body
> Archive http://ipfix.doit.wisc.edu/archive/
>

------_=_NextPart_001_01C30B47.32055660-- -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Sun Apr 27 14:41:35 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA28949 for ; Sun, 27 Apr 2003 14:41:35 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 199qlT-0004pz-00 for ipfix-list@mil.doit.wisc.edu; Sun, 27 Apr 2003 13:20:39 -0500 Received: from smtp801.mail.sc5.yahoo.com ([66.163.168.180]) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 199qlS-0004ps-00 for ipfix@net.doit.wisc.edu; Sun, 27 Apr 2003 13:20:38 -0500 Received: from adsl-64-164-1-234.dsl.snfc21.pacbell.net (HELO yahoo.com) (p?ludemann@64.164.1.234 with plain) by smtp-sbc-v1.mail.vip.sc5.yahoo.com with SMTP; 27 Apr 2003 18:20:36 -0000 Message-ID: <3EAC1F74.9000900@yahoo.com> Date: Sun, 27 Apr 2003 11:20:36 -0700 From: Peter Ludemann User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2.1) Gecko/20030225 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Reinaldo Penno CC: "'Maurizio Molina'" , ipfix@net.doit.wisc.edu Subject: Re: [ipfix] export packet length References: <0A11633F61BD9F40B43ABCC694004F93F18EEC@zsc3c026.us.nortel.com> In-Reply-To: <0A11633F61BD9F40B43ABCC694004F93F18EEC@zsc3c026.us.nortel.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Reinaldo/Maurizio: Benoit's scheme tries to save bytes, so it would be counter-productive to also have a length for the entire packet when it is not needed. Here are some implementation details ... If you're using Java, DataInputStream.readFully() will do what you want for each flowSet: // the calls to "new" can be avoided for efficiency byte [] lengthBytes = new byte[2]; stream.ReadFully(lengthByte2); int length = ((lengthByte[0]&0xff) << 8) + (lengthByte[1]&0xff); byte [] flowSet = new byte[length]; stream.ReadFully(flowset, 0, length); // blocks until all bytes for flowSet are available There are C++ libraries that have similar capabilities. (By the way, don't just copy this code; there are some subtletities that require more complexity; but the same subtleties exist if you have a length for the entire NetFlow packet.) If you're using UDP, instead of depending on IP fragmentation, you might want to use something like IBM's "variable block spanned" scheme for long records split over shorter blocks (basically, it splits a string into as many segments as needed, with a length for each segment and a flag indicating whether it's the last segment or not). Again, this could be done on a per-flowSet basis. Of course, this doesn't work properly if a UDP packet gets dropped; but neither would IP-fragments. (If your conclusion from all this is that UDP is more trouble than it's worth, I'd agree with you.) - peter PS: Benoit's 14-bit encoding of string lengths isn't very complicated to handle; and it does save a byte!: int strLen = buf[pos++]&0xff; if (strLen > 0x7f) { strLen = ((strLen&0x7f) << 7) + (buf[pos++] & 0x7f); } Reinaldo Penno wrote: > That's a good idea. > > Other protocols like HTTP and SIP make it mandatory to include the > Content-Length header (which tell the length of the payload) when > carried over TCP. It helps a lot when dealing with fragmentation. > > Regards, > > > -----Original Message----- > > From: Maurizio Molina [mailto:molina@ccrle.nec.de] > > Sent: Friday, April 25, 2003 11:50 AM > > To: ipfix@net.doit.wisc.edu > > Subject: [ipfix] export packet length > > > > > > Hi all, > > I hope not to rise an issue already discussed in the WG. If > > yes, I apologise in advance. If an IPFIX export packet is > > carried over TCP, it's not unusual that it gets split across > > multiple TCP segments. At the collector side, on the > > contrary, the implementation of the parsing is much simpler > > if only full export packet are parsed. If the information > > about the WHOLE export packet were contained in the header, > > it would be simple to wait until all the bytes composing an > > export packets are read from the TCP socket before beginning > > the parsing. Unfortunately, this information isn't currently > > contained in Netflow 9 packet header (I suppose, due to > > Netflow's 9 "bias" towards UDP...). Therefore, a parsing > > implementation having to deal with this "fragentation" > > problem without this information needs to be much more > > complicated (e.g. getting step by step the lenght of each > > flow set, see if it's all there, parse it and go on....). In > > summary: if IPFIX has to be carried over tcp, having the > > length of the export packet in the packet header would be > > beneficial. Regards, Maurizio > > > > > > -- > > Maurizio Molina > > Research Staff member > > Network Laboratories Heidelberg > > NEC Europe Ltd. > > Kurfuersten-Anlage 36, 69115 Heidelberg, Germany. > > Tel. (49)6221 90511-18 Fax: (49)6221 90511-55 > > e-mail: molina@ccrle.nec.de > > Web: www.ccrle.nec.de > > > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Sun Apr 27 22:47:03 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id WAA07311 for ; Sun, 27 Apr 2003 22:47:02 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 199yZJ-00073y-00 for ipfix-list@mil.doit.wisc.edu; Sun, 27 Apr 2003 21:40:37 -0500 Received: from eng4.oar.net ([192.148.244.24]) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 199yZH-00073q-00 for ipfix@net.doit.wisc.edu; Sun, 27 Apr 2003 21:40:35 -0500 Received: (qmail 13841 invoked by uid 4454); 28 Apr 2003 02:40:34 -0000 Date: Sun, 27 Apr 2003 22:40:34 -0400 From: Mark Fullmer To: Peter Ludemann Cc: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] export packet length Message-ID: <20030427224034.A13697@net.ohio-state.edu> References: <0A11633F61BD9F40B43ABCC694004F93F18EEC@zsc3c026.us.nortel.com> <3EAC1F74.9000900@yahoo.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: <3EAC1F74.9000900@yahoo.com>; from p_ludemann@yahoo.com on Sun, Apr 27, 2003 at 11:20:36AM -0700 Precedence: bulk Sender: majordomo listserver On Sun, Apr 27, 2003 at 11:20:36AM -0700, Peter Ludemann wrote: > Reinaldo/Maurizio: > > Benoit's scheme tries to save bytes, so it would be counter-productive > to also have a length for the entire packet when it is not needed. Here > are some implementation details ... There are other ways to save bytes. SysUpTime and Unix Secs which are currently sent with every export PDU can go away when a connection oriented protocol like TCP or SCTP are used. The SysUptime : Unix_Secs mapping, which should be SysUpTime : {Unix_Secs,Unix_Msecs} only needs to be sent once per Source ID per connection. The current Count header field could be changed to Length.... mark -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Sun Apr 27 23:54:13 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id XAA08209 for ; Sun, 27 Apr 2003 23:54:13 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 199zZD-0000Ya-00 for ipfix-list@mil.doit.wisc.edu; Sun, 27 Apr 2003 22:44:35 -0500 Received: from mailhost2.auckland.ac.nz ([130.216.191.4]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 199zZA-0000YS-00 for ipfix@net.doit.wisc.edu; Sun, 27 Apr 2003 22:44:33 -0500 Received: from mailhost.auckland.ac.nz (IDENT:mirapoint@mailhost [130.216.191.61]) by mailhost2.auckland.ac.nz (8.12.9/8.12.9/8.12.9-ua) with ESMTP id h3S3iTcV022356 for ; Mon, 28 Apr 2003 15:44:30 +1200 (NZST) Received: from hotlava.auckland.ac.nz (hotlava.auckland.ac.nz [130.216.191.123]) by mailhost.auckland.ac.nz (Mirapoint Messaging Server MOS 3.3.2-CR) with ESMTP id APG37252; Mon, 28 Apr 2003 15:44:29 +1200 (NZST) Received: (from apache@localhost) by hotlava.auckland.ac.nz (8.11.6/8.11.6) id h3S3iTj01454 for ipfix@net.doit.wisc.edu; Mon, 28 Apr 2003 15:44:29 +1200 X-Authentication-Warning: hotlava.auckland.ac.nz: apache set sender to n.brownlee@auckland.ac.nz using -f Received: from nebbiolo.itss.auckland.ac.nz (nebbiolo.itss.auckland.ac.nz [130.216.4.167]) by hotlava.auckland.ac.nz (Horde) with HTTP for ; Mon, 28 Apr 2003 15:44:29 +1200 Message-ID: <1051501469.add1860ad3a45@hotlava.auckland.ac.nz> Date: Mon, 28 Apr 2003 15:44:29 +1200 From: Nevil Brownlee To: ipfix@net.doit.wisc.edu Subject: [ipfix] Editors for IPFIX drafts MIME-Version: 1.0 Content-Type: text/plain; charset="ISO-8859-1" Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Internet Messaging Program (IMP) 4.0-cvs X-Originating-IP: 130.216.4.167 Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Hello all: Following the IPFIX meeting at the San Francisco IETF we asked for volunteers to edit the IPFIX drafts. The current list is: Applicability: Tanja Zseby, Reinaldo Penno Architecture: Ganesh Sadasivan, Nevil Brownlee Information Model: Paul Calato, Jeff Meyer, Juergen Quittek Protocol: Mark Fulmer, Paul Calato, Reinaldo Penno At this stage we already have old (expired) versions of the Architecture (-02.txt) and Data Model (-01.txt) drafts on the IPFIX web page; would the editors of all four drafts please email me an initial version of their new drafts to published as placeholders within the next few weeks. After that, the editors will work on developing better versions of the drafts which can be discussed on the IPFIX list and submitted by mid-June, i.e. in plenty of time for the Vienna IETF meeting in July. Cheers, Nevil PS: Good to see some real discussion on the list. Lets concentrate on producing text the draft editors can use! ----------------------------------------------------------------------- Nevil Brownlee Director, Technology Development Phone: +64 9 373 7599 x88941 ITSS, The University of Auckland FAX: +64 9 373 7021 Private Bag 92019, Auckland, New Zealand ------------------------------------------------- This mail sent through University of Auckland http://www.auckland.ac.nz/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon Apr 28 10:56:11 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA03713 for ; Mon, 28 Apr 2003 10:56:11 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19A9ux-0000FM-00 for ipfix-list@mil.doit.wisc.edu; Mon, 28 Apr 2003 09:47:43 -0500 Received: from tokyo.ccrle.nec.de ([195.37.70.2]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19A9uv-0000FA-00 for ipfix@net.doit.wisc.edu; Mon, 28 Apr 2003 09:47:41 -0500 Received: from venus.office (venus.office [10.1.1.11]) by tokyo.ccrle.nec.de (8.12.9/8.12.8) with ESMTP id h3SElbVI091496 for ; Mon, 28 Apr 2003 16:47:38 +0200 (CEST) Received: from ccrle.nec.de (molina.office [10.1.1.126]) by venus.office (Postfix on SuSE Linux eMail Server 3.0) with ESMTP id C2AFCDFEA for ; Mon, 28 Apr 2003 16:41:29 +0200 (CEST) Message-ID: <3EAD3F0B.88DCE050@ccrle.nec.de> Date: Mon, 28 Apr 2003 16:47:39 +0200 From: Maurizio Molina X-Mailer: Mozilla 4.77 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 Cc: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] export packet length References: <0A11633F61BD9F40B43ABCC694004F93F18EEC@zsc3c026.us.nortel.com> <3EAC1F74.9000900@yahoo.com> <20030427224034.A13697@net.ohio-state.edu> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Mark Fullmer wrote: > On Sun, Apr 27, 2003 at 11:20:36AM -0700, Peter Ludemann wrote: > > Reinaldo/Maurizio: > > > > Benoit's scheme tries to save bytes, so it would be counter-productive > > to also have a length for the entire packet when it is not needed. Here > > are some implementation details ... > > There are other ways to save bytes. > > SysUpTime and Unix Secs which are currently sent with every export PDU can > go away when a connection oriented protocol like TCP or SCTP are used. > > The SysUptime : Unix_Secs mapping, which should be > SysUpTime : {Unix_Secs,Unix_Msecs} only needs to be sent once per > Source ID per connection. > > The current Count header field could be changed to Length.... Mark, being a 16 bit counter it would limit the export packet size to 65536 bytes. Don't you see it as a constraint? Maurizio > > > mark > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon Apr 28 11:09:53 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA03985 for ; Mon, 28 Apr 2003 11:09:52 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19AA4i-0000Ox-00 for ipfix-list@mil.doit.wisc.edu; Mon, 28 Apr 2003 09:57:48 -0500 Received: from atlrel6.hp.com ([156.153.255.205]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19AA4g-0000Oo-00 for ipfix@net.doit.wisc.edu; Mon, 28 Apr 2003 09:57:47 -0500 Received: from xatlrelay2.atl.hp.com (xatlrelay2.atl.hp.com [15.45.89.191]) by atlrel6.hp.com (Postfix) with ESMTP id 741E41C01B7E; Mon, 28 Apr 2003 10:57:46 -0400 (EDT) Received: from xatlbh1.atl.hp.com (xatlbh1.atl.hp.com [15.45.89.186]) by xatlrelay2.atl.hp.com (Postfix) with ESMTP id 2AF911C000A1; Mon, 28 Apr 2003 10:57:46 -0400 (EDT) Received: by xatlbh1.atl.hp.com with Internet Mail Service (5.5.2655.55) id ; Mon, 28 Apr 2003 10:57:45 -0400 Message-ID: <1D3D2C371FCBD947A7897FABBD3533A566BA68@xsun01.ptp.hp.com> From: "MEYER,JEFFREY D (HP-Cupertino,ex1)" To: "'Nevil Brownlee'" , ipfix@net.doit.wisc.edu Subject: RE: [ipfix] Editors for IPFIX drafts Date: Mon, 28 Apr 2003 10:57:43 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2655.55) Content-Type: text/plain; charset="iso-8859-1" Precedence: bulk Sender: majordomo listserver Hi, My particular interest on working on the information model would be on developing a more formal and standard way of describing the information elements which are used in IPFIX communication. The modelling of information elements should ideally be done using a formal description language, so that tools can be constructed which do not require humans to transcribe this information. I would propose the use of XML-Schema to provide this formal description. Through the use of XSL processors, prose based descriptions and tables can be produced if these are considered desireable. NOTE: describing the information model formally does NOT mean that encoding and transport for IPFIX need to change in any way. It simply means that rather than having as the base description of information items some prose in a document, you instead have a machine readable representation. As an example of a formal (but currently incomplete) information model for IPFIX, see: http://www.ipdr.org/documents/ipfix/ipfixService-20020902.xml I would recommend making such a representation the normative form of the IPFIX information model, much like SNMP MIBs. Please let me know if people see value in this exercise. Regards, Jeff Meyer > -----Original Message----- > From: Nevil Brownlee [mailto:n.brownlee@auckland.ac.nz] > Sent: Sunday, April 27, 2003 8:44 PM > To: ipfix@net.doit.wisc.edu > Subject: [ipfix] Editors for IPFIX drafts > > > > Hello all: > > Following the IPFIX meeting at the San Francisco IETF we asked for > volunteers to edit the IPFIX drafts. The current list is: > > Applicability: Tanja Zseby, Reinaldo Penno > > Architecture: Ganesh Sadasivan, Nevil Brownlee > > Information Model: Paul Calato, Jeff Meyer, Juergen Quittek > > Protocol: Mark Fulmer, Paul Calato, Reinaldo Penno > > At this stage we already have old (expired) versions of the > Architecture (-02.txt) and Data Model (-01.txt) drafts on the IPFIX > web page; would the editors of all four drafts please email > me an initial > version of their new drafts to published as placeholders > within the next > few weeks. > > After that, the editors will work on developing better versions of the > drafts which can be discussed on the IPFIX list and submitted > by mid-June, > i.e. in plenty of time for the Vienna IETF meeting in July. > > Cheers, Nevil > > PS: Good to see some real discussion on the list. Lets concentrate on > producing text the draft editors can use! > > -------------------------------------------------------------- > --------- > Nevil Brownlee Director, Technology Development > Phone: +64 9 373 7599 x88941 ITSS, The University of Auckland > FAX: +64 9 373 7021 Private Bag 92019, Auckland, New Zealand > > > ------------------------------------------------- > This mail sent through University of Auckland > http://www.auckland.ac.nz/ > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" > in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon Apr 28 11:22:23 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA04313 for ; Mon, 28 Apr 2003 11:22:23 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19AAEO-0000hC-00 for ipfix-list@mil.doit.wisc.edu; Mon, 28 Apr 2003 10:07:48 -0500 Received: from tokyo.ccrle.nec.de ([195.37.70.2]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19AAEL-0000h4-00 for ipfix@net.doit.wisc.edu; Mon, 28 Apr 2003 10:07:46 -0500 Received: from venus.office (venus.office [10.1.1.11]) by tokyo.ccrle.nec.de (8.12.9/8.12.8) with ESMTP id h3SF7jVI093131 for ; Mon, 28 Apr 2003 17:07:45 +0200 (CEST) Received: from ccrle.nec.de (molina.office [10.1.1.126]) by venus.office (Postfix on SuSE Linux eMail Server 3.0) with ESMTP id DDA5D4009D for ; Mon, 28 Apr 2003 17:01:36 +0200 (CEST) Message-ID: <3EAD43C2.A98E4FA8@ccrle.nec.de> Date: Mon, 28 Apr 2003 17:07:46 +0200 From: Maurizio Molina X-Mailer: Mozilla 4.77 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 Cc: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] export packet length References: <0A11633F61BD9F40B43ABCC694004F93F18EEC@zsc3c026.us.nortel.com> <3EAC1F74.9000900@yahoo.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Peter Ludemann wrote: > Reinaldo/Maurizio: > > Benoit's scheme tries to save bytes, so it would be counter-productive > to also have a length for the entire packet when it is not needed. Here > are some implementation details ... > > If you're using Java, DataInputStream.readFully() will do what you want > for each flowSet: > // the calls to "new" can be avoided for efficiency > byte [] lengthBytes = new byte[2]; > stream.ReadFully(lengthByte2); > int length = ((lengthByte[0]&0xff) << 8) + (lengthByte[1]&0xff); > byte [] flowSet = new byte[length]; > stream.ReadFully(flowset, 0, length); // blocks until all bytes for > flowSet are available > > There are C++ libraries that have similar capabilities. Also in a standard C implemenatation you can still live with the information currently contained in the header (basically using the number of bytes returned from the socket reading function). It's just more complex and prone to errors. I was suggesting that 4 bytes more per each export packet would make life easier for implementations over TCP. Regards, Maurizio > (By the way, > don't just copy this code; there are some subtletities that require more > complexity; but the same subtleties exist if you have a length for the > entire NetFlow packet.) > > If you're using UDP, instead of depending on IP fragmentation, you might > want to use something like IBM's "variable block spanned" scheme for > long records split over shorter blocks (basically, it splits a string > into as many segments as needed, with a length for each segment and a > flag indicating whether it's the last segment or not). Again, this could > be done on a per-flowSet basis. Of course, this doesn't work properly if > a UDP packet gets dropped; but neither would IP-fragments. > > (If your conclusion from all this is that UDP is more trouble than it's > worth, I'd agree with you.) > > - peter > > PS: Benoit's 14-bit encoding of string lengths isn't very complicated to > handle; and it does save a byte!: > int strLen = buf[pos++]&0xff; > if (strLen > 0x7f) { strLen = ((strLen&0x7f) << 7) + (buf[pos++] > & 0x7f); } > > Reinaldo Penno wrote: > > > That's a good idea. > > > > Other protocols like HTTP and SIP make it mandatory to include the > > Content-Length header (which tell the length of the payload) when > > carried over TCP. It helps a lot when dealing with fragmentation. > > > > Regards, > > > > > -----Original Message----- > > > From: Maurizio Molina [mailto:molina@ccrle.nec.de] > > > Sent: Friday, April 25, 2003 11:50 AM > > > To: ipfix@net.doit.wisc.edu > > > Subject: [ipfix] export packet length > > > > > > > > > Hi all, > > > I hope not to rise an issue already discussed in the WG. If > > > yes, I apologise in advance. If an IPFIX export packet is > > > carried over TCP, it's not unusual that it gets split across > > > multiple TCP segments. At the collector side, on the > > > contrary, the implementation of the parsing is much simpler > > > if only full export packet are parsed. If the information > > > about the WHOLE export packet were contained in the header, > > > it would be simple to wait until all the bytes composing an > > > export packets are read from the TCP socket before beginning > > > the parsing. Unfortunately, this information isn't currently > > > contained in Netflow 9 packet header (I suppose, due to > > > Netflow's 9 "bias" towards UDP...). Therefore, a parsing > > > implementation having to deal with this "fragentation" > > > problem without this information needs to be much more > > > complicated (e.g. getting step by step the lenght of each > > > flow set, see if it's all there, parse it and go on....). In > > > summary: if IPFIX has to be carried over tcp, having the > > > length of the export packet in the packet header would be > > > beneficial. Regards, Maurizio > > > > > > > > > -- > > > Maurizio Molina > > > Research Staff member > > > Network Laboratories Heidelberg > > > NEC Europe Ltd. > > > Kurfuersten-Anlage 36, 69115 Heidelberg, Germany. > > > Tel. (49)6221 90511-18 Fax: (49)6221 90511-55 > > > e-mail: molina@ccrle.nec.de > > > Web: www.ccrle.nec.de > > > > > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon Apr 28 11:41:41 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA05059 for ; Mon, 28 Apr 2003 11:41:41 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19AATB-00012S-00 for ipfix-list@mil.doit.wisc.edu; Mon, 28 Apr 2003 10:23:05 -0500 Received: from tokyo.ccrle.nec.de ([195.37.70.2]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19AAT9-00012F-00 for ipfix@net.doit.wisc.edu; Mon, 28 Apr 2003 10:23:03 -0500 Received: from venus.office (venus.office [10.1.1.11]) by tokyo.ccrle.nec.de (8.12.9/8.12.8) with ESMTP id h3SFMsVI094108; Mon, 28 Apr 2003 17:22:54 +0200 (CEST) Received: from [10.1.1.128] (n-quittek.office [10.1.1.128]) by venus.office (Postfix on SuSE Linux eMail Server 3.0) with ESMTP id 5F4C29F696; Mon, 28 Apr 2003 17:16:45 +0200 (CEST) Date: Mon, 28 Apr 2003 17:24:17 +0200 From: Juergen Quittek To: "MEYER,JEFFREY D (HP-Cupertino,ex1)" , "'Nevil Brownlee'" , ipfix@net.doit.wisc.edu Subject: RE: [ipfix] Editors for IPFIX drafts Message-ID: <30577938.1051550657@[10.1.1.128]> In-Reply-To: <1D3D2C371FCBD947A7897FABBD3533A566BA68@xsun01.ptp.hp.com> References: <1D3D2C371FCBD947A7897FABBD3533A566BA68@xsun01.ptp.hp.com> X-Mailer: Mulberry/2.1.2 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Jeff, I do see a value in such a formal description. But there is a trafe-off. I also see a degradation of human readability. You loose means of structuring your document text when using XML, for example grouping attributes that are related. (Of course you could do so in XML, but it is much harder to find the heading "Port-related Attributes" in an XML document, because headings cannot be well highlighted and numbering of headings is not well supported. Anyway, I like the idea of an XML encoding of the model, but I we should not go this way if we cannot find a good way of getting it hunam readable (maybe by applying formatting rules). An option would also be adding the XML code to the document as an appendix. Juergen -- MEYER,JEFFREY D (HP-Cupertino,ex1) wrote on 28 April 2003 10:57 -0400: > Hi, > > My particular interest on working on the information model would > be on developing a more formal and standard way of describing the > information elements which are used in IPFIX communication. > > The modelling of information elements should ideally be done > using a formal description language, so that tools can be constructed > which do not require humans to transcribe this information. > > I would propose the use of XML-Schema to provide this formal > description. Through the use of XSL processors, prose based descriptions > and tables can be produced if these are considered desireable. > > NOTE: describing the information model formally does NOT mean > that encoding and transport for IPFIX need to change in any way. It > simply means that rather than having as the base description of information > items some prose in a document, you instead have a machine readable > representation. > > > As an example of a formal (but currently incomplete) information > model for IPFIX, see: > http://www.ipdr.org/documents/ipfix/ipfixService-20020902.xml > > I would recommend making such a representation the normative form > of the IPFIX information model, much like SNMP MIBs. > > > Please let me know if people see value in this exercise. > > Regards, > > Jeff Meyer > >> -----Original Message----- >> From: Nevil Brownlee [mailto:n.brownlee@auckland.ac.nz] >> Sent: Sunday, April 27, 2003 8:44 PM >> To: ipfix@net.doit.wisc.edu >> Subject: [ipfix] Editors for IPFIX drafts >> >> >> >> Hello all: >> >> Following the IPFIX meeting at the San Francisco IETF we asked for >> volunteers to edit the IPFIX drafts. The current list is: >> >> Applicability: Tanja Zseby, Reinaldo Penno >> >> Architecture: Ganesh Sadasivan, Nevil Brownlee >> >> Information Model: Paul Calato, Jeff Meyer, Juergen Quittek >> >> Protocol: Mark Fulmer, Paul Calato, Reinaldo Penno >> >> At this stage we already have old (expired) versions of the >> Architecture (-02.txt) and Data Model (-01.txt) drafts on the IPFIX >> web page; would the editors of all four drafts please email >> me an initial >> version of their new drafts to published as placeholders >> within the next >> few weeks. >> >> After that, the editors will work on developing better versions of the >> drafts which can be discussed on the IPFIX list and submitted >> by mid-June, >> i.e. in plenty of time for the Vienna IETF meeting in July. >> >> Cheers, Nevil >> >> PS: Good to see some real discussion on the list. Lets concentrate on >> producing text the draft editors can use! >> >> -------------------------------------------------------------- >> --------- >> Nevil Brownlee Director, Technology Development >> Phone: +64 9 373 7599 x88941 ITSS, The University of Auckland >> FAX: +64 9 373 7021 Private Bag 92019, Auckland, New Zealand >> >> >> ------------------------------------------------- >> This mail sent through University of Auckland >> http://www.auckland.ac.nz/ >> >> -- >> Help mailto:majordomo@net.doit.wisc.edu and say "help" >> in message body >> Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >> "unsubscribe ipfix" in message body >> Archive http://ipfix.doit.wisc.edu/archive/ >> > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon Apr 28 12:19:13 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA05967 for ; Mon, 28 Apr 2003 12:19:12 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19ABIV-0002Fj-00 for ipfix-list@mil.doit.wisc.edu; Mon, 28 Apr 2003 11:16:07 -0500 Received: from bay8-f27.bay8.hotmail.com ([64.4.27.27] helo=hotmail.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19ABIU-0002FT-00 for ipfix@net.doit.wisc.edu; Mon, 28 Apr 2003 11:16:06 -0500 Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC; Mon, 28 Apr 2003 09:16:03 -0700 Received: from 57.250.229.136 by by8fd.bay8.hotmail.msn.com with HTTP; Mon, 28 Apr 2003 16:16:02 GMT X-Originating-IP: [57.250.229.136] X-Originating-Email: [elkou141061@hotmail.com] From: "M. ELK" To: quittek@ccrle.nec.de, jeff.meyer2@hp.com, n.brownlee@auckland.ac.nz, ipfix@net.doit.wisc.edu Subject: RE: [ipfix] Editors for IPFIX drafts Date: Mon, 28 Apr 2003 16:16:02 +0000 Mime-Version: 1.0 Content-Type: text/plain; format=flowed Message-ID: X-OriginalArrivalTime: 28 Apr 2003 16:16:03.0075 (UTC) FILETIME=[76D01530:01C30DA1] Precedence: bulk Sender: majordomo listserver Hi i am interested in the work for IPFIX group as a netw operator , we are not programmer so have no knwoledge in XML format and we do not need to spend time to learn XML just to be able to read IPFIX doc . So pls keep it in a human readable format and XML format can go to an annex . Brgds >From: Juergen Quittek >To: "MEYER,JEFFREY D (HP-Cupertino,ex1)" , "'Nevil >Brownlee'" , ipfix@net.doit.wisc.edu >Subject: RE: [ipfix] Editors for IPFIX drafts >Date: Mon, 28 Apr 2003 17:24:17 +0200 > >Jeff, > >I do see a value in such a formal description. But there is a trafe-off. > >I also see a degradation of human readability. You loose means of >structuring your document text when using XML, for example grouping >attributes that are related. (Of course you could do so in XML, but it is >much harder to find the heading "Port-related Attributes" in an XML >document, >because headings cannot be well highlighted and numbering of headings is >not well supported. > >Anyway, I like the idea of an XML encoding of the model, but I we should >not go this way if we cannot find a good way of getting it hunam readable >(maybe by applying formatting rules). > >An option would also be adding the XML code to the document as an appendix. > > Juergen > > >-- MEYER,JEFFREY D (HP-Cupertino,ex1) wrote on 28 April 2003 10:57 -0400: > >>Hi, >> >> My particular interest on working on the information model would >>be on developing a more formal and standard way of describing the >>information elements which are used in IPFIX communication. >> >> The modelling of information elements should ideally be done >>using a formal description language, so that tools can be constructed >>which do not require humans to transcribe this information. >> >> I would propose the use of XML-Schema to provide this formal >>description. Through the use of XSL processors, prose based descriptions >>and tables can be produced if these are considered desireable. >> >> NOTE: describing the information model formally does NOT mean >>that encoding and transport for IPFIX need to change in any way. It >>simply means that rather than having as the base description of >>information >>items some prose in a document, you instead have a machine readable >>representation. >> >> >> As an example of a formal (but currently incomplete) information >>model for IPFIX, see: >>http://www.ipdr.org/documents/ipfix/ipfixService-20020902.xml >> >> I would recommend making such a representation the normative form >>of the IPFIX information model, much like SNMP MIBs. >> >> >> Please let me know if people see value in this exercise. >> >>Regards, >> >> Jeff Meyer >> >>>-----Original Message----- >>>From: Nevil Brownlee [mailto:n.brownlee@auckland.ac.nz] >>>Sent: Sunday, April 27, 2003 8:44 PM >>>To: ipfix@net.doit.wisc.edu >>>Subject: [ipfix] Editors for IPFIX drafts >>> >>> >>> >>>Hello all: >>> >>>Following the IPFIX meeting at the San Francisco IETF we asked for >>>volunteers to edit the IPFIX drafts. The current list is: >>> >>> Applicability: Tanja Zseby, Reinaldo Penno >>> >>> Architecture: Ganesh Sadasivan, Nevil Brownlee >>> >>> Information Model: Paul Calato, Jeff Meyer, Juergen Quittek >>> >>> Protocol: Mark Fulmer, Paul Calato, Reinaldo Penno >>> >>>At this stage we already have old (expired) versions of the >>>Architecture (-02.txt) and Data Model (-01.txt) drafts on the IPFIX >>>web page; would the editors of all four drafts please email >>>me an initial >>>version of their new drafts to published as placeholders >>>within the next >>>few weeks. >>> >>>After that, the editors will work on developing better versions of the >>>drafts which can be discussed on the IPFIX list and submitted >>>by mid-June, >>>i.e. in plenty of time for the Vienna IETF meeting in July. >>> >>>Cheers, Nevil >>> >>>PS: Good to see some real discussion on the list. Lets concentrate on >>> producing text the draft editors can use! >>> >>>-------------------------------------------------------------- >>>--------- >>> Nevil Brownlee Director, Technology Development >>> Phone: +64 9 373 7599 x88941 ITSS, The University of Auckland >>> FAX: +64 9 373 7021 Private Bag 92019, Auckland, New Zealand >>> >>> >>>------------------------------------------------- >>>This mail sent through University of Auckland >>>http://www.auckland.ac.nz/ >>> >>>-- >>>Help mailto:majordomo@net.doit.wisc.edu and say "help" >>>in message body >>>Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >>>"unsubscribe ipfix" in message body >>>Archive http://ipfix.doit.wisc.edu/archive/ >>> >> >>-- >>Help mailto:majordomo@net.doit.wisc.edu and say "help" in message >>body >>Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >>"unsubscribe ipfix" in message body >>Archive http://ipfix.doit.wisc.edu/archive/ > > > >-- >Help mailto:majordomo@net.doit.wisc.edu and say "help" in message >body >Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >"unsubscribe ipfix" in message body >Archive http://ipfix.doit.wisc.edu/archive/ _________________________________________________________________ Tired of spam? Get advanced junk mail protection with MSN 8. http://join.msn.com/?page=features/junkmail -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon Apr 28 12:24:19 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA06108 for ; Mon, 28 Apr 2003 12:24:19 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19ABN4-0002O9-00 for ipfix-list@mil.doit.wisc.edu; Mon, 28 Apr 2003 11:20:50 -0500 Received: from palrel11.hp.com ([156.153.255.246]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19ABN2-0002O0-00 for ipfix@net.doit.wisc.edu; Mon, 28 Apr 2003 11:20:48 -0500 Received: from xparelay1.ptp.hp.com (xparelay1.ptp.hp.com [15.1.28.62]) by palrel11.hp.com (Postfix) with ESMTP id 104AC1C00E60; Mon, 28 Apr 2003 09:20:48 -0700 (PDT) Received: from xpabh2.ptp.hp.com (xpabh2.ptp.hp.com [15.1.28.61]) by xparelay1.ptp.hp.com (Postfix) with ESMTP id 877A71004B9C; Mon, 28 Apr 2003 09:20:44 -0700 (PDT) Received: by xpabh2.ptp.hp.com with Internet Mail Service (5.5.2655.55) id ; Mon, 28 Apr 2003 09:20:44 -0700 Message-ID: <1D3D2C371FCBD947A7897FABBD3533A566BA6B@xsun01.ptp.hp.com> From: "MEYER,JEFFREY D (HP-Cupertino,ex1)" To: "'Juergen Quittek'" , "MEYER,JEFFREY D (HP-Cupertino,ex1)" , "'Nevil Brownlee'" , ipfix@net.doit.wisc.edu Subject: RE: [ipfix] Editors for IPFIX drafts Date: Mon, 28 Apr 2003 09:20:41 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2655.55) Content-Type: text/plain; charset="iso-8859-1" Precedence: bulk Sender: majordomo listserver Juergen, Thanks for the response. I agree XML in and of itself is marginally human readable. I think that basic tools such as Xalan (http://xml.apache.org/), can be used to "transform" a base XML document into a variety of formats, including formatting consistent with RFC's. Note that all the IPDR RFC's were written in XML using Marshall Rose's xml2rfc tools (http://xml.resource.org/). So the model I would picture is having the Normative XML data model as an appendix, and having generated information from this data model populate some of the body sections to aid human readers. Regards, Jeff Meyer > -----Original Message----- > From: Juergen Quittek [mailto:quittek@ccrle.nec.de] > Sent: Monday, April 28, 2003 8:24 AM > To: MEYER,JEFFREY D (HP-Cupertino,ex1); 'Nevil Brownlee'; > ipfix@net.doit.wisc.edu > Subject: RE: [ipfix] Editors for IPFIX drafts > > > Jeff, > > I do see a value in such a formal description. But there is a > trafe-off. > > I also see a degradation of human readability. You loose means of > structuring your document text when using XML, for example grouping > attributes that are related. (Of course you could do so in > XML, but it is > much harder to find the heading "Port-related Attributes" in > an XML document, > because headings cannot be well highlighted and numbering of > headings is > not well supported. > > Anyway, I like the idea of an XML encoding of the model, but > I we should > not go this way if we cannot find a good way of getting it > hunam readable > (maybe by applying formatting rules). > > An option would also be adding the XML code to the document > as an appendix. > > Juergen > > > -- MEYER,JEFFREY D (HP-Cupertino,ex1) wrote on 28 April 2003 > 10:57 -0400: > > > Hi, > > > > My particular interest on working on the information model would > > be on developing a more formal and standard way of describing the > > information elements which are used in IPFIX communication. > > > > The modelling of information elements should ideally be done > > using a formal description language, so that tools can be > constructed > > which do not require humans to transcribe this information. > > > > I would propose the use of XML-Schema to provide this formal > > description. Through the use of XSL processors, prose > based descriptions > > and tables can be produced if these are considered desireable. > > > > NOTE: describing the information model formally does NOT mean > > that encoding and transport for IPFIX need to change in any way. It > > simply means that rather than having as the base > description of information > > items some prose in a document, you instead have a machine readable > > representation. > > > > > > As an example of a formal (but currently incomplete) information > > model for IPFIX, see: > > http://www.ipdr.org/documents/ipfix/ipfixService-20020902.xml > > > > I would recommend making such a representation the normative form > > of the IPFIX information model, much like SNMP MIBs. > > > > > > Please let me know if people see value in this exercise. > > > > Regards, > > > > Jeff Meyer > > > >> -----Original Message----- > >> From: Nevil Brownlee [mailto:n.brownlee@auckland.ac.nz] > >> Sent: Sunday, April 27, 2003 8:44 PM > >> To: ipfix@net.doit.wisc.edu > >> Subject: [ipfix] Editors for IPFIX drafts > >> > >> > >> > >> Hello all: > >> > >> Following the IPFIX meeting at the San Francisco IETF we asked for > >> volunteers to edit the IPFIX drafts. The current list is: > >> > >> Applicability: Tanja Zseby, Reinaldo Penno > >> > >> Architecture: Ganesh Sadasivan, Nevil Brownlee > >> > >> Information Model: Paul Calato, Jeff Meyer, Juergen Quittek > >> > >> Protocol: Mark Fulmer, Paul Calato, Reinaldo Penno > >> > >> At this stage we already have old (expired) versions of the > >> Architecture (-02.txt) and Data Model (-01.txt) drafts on the IPFIX > >> web page; would the editors of all four drafts please email > >> me an initial > >> version of their new drafts to published as placeholders > >> within the next > >> few weeks. > >> > >> After that, the editors will work on developing better > versions of the > >> drafts which can be discussed on the IPFIX list and submitted > >> by mid-June, > >> i.e. in plenty of time for the Vienna IETF meeting in July. > >> > >> Cheers, Nevil > >> > >> PS: Good to see some real discussion on the list. Lets > concentrate on > >> producing text the draft editors can use! > >> > >> -------------------------------------------------------------- > >> --------- > >> Nevil Brownlee Director, Technology > Development > >> Phone: +64 9 373 7599 x88941 ITSS, The University > of Auckland > >> FAX: +64 9 373 7021 Private Bag 92019, Auckland, > New Zealand > >> > >> > >> ------------------------------------------------- > >> This mail sent through University of Auckland > >> http://www.auckland.ac.nz/ > >> > >> -- > >> Help mailto:majordomo@net.doit.wisc.edu and say "help" > >> in message body > >> Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > >> "unsubscribe ipfix" in message body > >> Archive http://ipfix.doit.wisc.edu/archive/ > >> > > > > -- > > Help mailto:majordomo@net.doit.wisc.edu and say > "help" in message body > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe ipfix" in message body > > Archive http://ipfix.doit.wisc.edu/archive/ > > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon Apr 28 16:33:41 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA13768 for ; Mon, 28 Apr 2003 16:33:40 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19AFCr-0007Om-00 for ipfix-list@mil.doit.wisc.edu; Mon, 28 Apr 2003 15:26:33 -0500 Received: from ip2-pal-focal.netli.com ([66.243.52.2] helo=smtp.netli.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19AFCp-0007Of-00 for ipfix@net.doit.wisc.edu; Mon, 28 Apr 2003 15:26:32 -0500 Received: (qmail 22678 invoked by uid 84); 28 Apr 2003 20:26:31 -0000 Received: from vlm@netli.com by l3-1 with qmail-scanner-0.96 (uvscan: v4.1.40/v4121. . Clean. Processed in 0.132339 secs); 28 Apr 2003 20:26:31 -0000 Received: from unknown (HELO netli.com) (172.17.1.52) by mx01-pal-lan.netli.lan with SMTP; 28 Apr 2003 20:26:30 -0000 Message-ID: <3EAD8E81.5020708@netli.com> Date: Mon, 28 Apr 2003 13:26:41 -0700 From: Lev Walkin Organization: Netli, Inc. User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.3) Gecko/20030424 X-Accept-Language: ru, en-us, en MIME-Version: 1.0 To: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] export packet length References: <0A11633F61BD9F40B43ABCC694004F93F18EEC@zsc3c026.us.nortel.com> <3EAC1F74.9000900@yahoo.com> <3EAD43C2.A98E4FA8@ccrle.nec.de> In-Reply-To: <3EAD43C2.A98E4FA8@ccrle.nec.de> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Maurizio Molina wrote: > > Peter Ludemann wrote: > > >>Reinaldo/Maurizio: >> >>Benoit's scheme tries to save bytes, so it would be counter-productive >>to also have a length for the entire packet when it is not needed. Here >>are some implementation details ... >> >>If you're using Java, DataInputStream.readFully() will do what you want >>for each flowSet: >> // the calls to "new" can be avoided for efficiency >> byte [] lengthBytes = new byte[2]; >> stream.ReadFully(lengthByte2); >> int length = ((lengthByte[0]&0xff) << 8) + (lengthByte[1]&0xff); >> byte [] flowSet = new byte[length]; >> stream.ReadFully(flowset, 0, length); // blocks until all bytes for >>flowSet are available >> >>There are C++ libraries that have similar capabilities. > > > Also in a standard C implemenatation you can still live with the information > currently contained in the header (basically using the number of bytes > returned from the socket reading function). It's just more complex and prone > to errors. By the way, some stacks (notably, some versions of Linux) could silently truncate the large _outgoing_ UDP packet. On the remote end of communication one would receive a perfectly good UDP data, but with the contents effectively less then expected. The default credit for the UDP packets with zero checksum (they're allowed to pass as the correct ones in the majority of the IP stacks) doesn't really help anything. > I was suggesting that 4 bytes more per each export packet would make life > easier for implementations over TCP. As in the DNS protocol (1035): it has a plain structure over UDP and an extended one (with the length prepended) when transferred over TCP. -- Lev Walkin vlm@netli.com -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon Apr 28 16:55:30 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA14304 for ; Mon, 28 Apr 2003 16:55:30 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19AFbg-00005q-00 for ipfix-list@mil.doit.wisc.edu; Mon, 28 Apr 2003 15:52:12 -0500 Received: from tokyo.ccrle.nec.de ([195.37.70.2]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19AFbe-00005h-00 for ipfix@net.doit.wisc.edu; Mon, 28 Apr 2003 15:52:10 -0500 Received: from venus.office (venus.office [10.1.1.11]) by tokyo.ccrle.nec.de (8.12.9/8.12.8) with ESMTP id h3SKpxVI002596; Mon, 28 Apr 2003 22:52:01 +0200 (CEST) Received: from [10.1.1.26] (dial02.office [10.1.1.26]) by venus.office (Postfix on SuSE Linux eMail Server 3.0) with ESMTP id 66F6EA049C; Mon, 28 Apr 2003 22:45:47 +0200 (CEST) Date: Mon, 28 Apr 2003 22:53:19 +0200 From: Juergen Quittek To: "MEYER,JEFFREY D (HP-Cupertino,ex1)" , "'Nevil Brownlee'" , ipfix@net.doit.wisc.edu Subject: RE: [ipfix] Editors for IPFIX drafts Message-ID: <16107501.1051570398@[10.1.1.26]> In-Reply-To: <1D3D2C371FCBD947A7897FABBD3533A566BA6B@xsun01.ptp.hp.com> References: <1D3D2C371FCBD947A7897FABBD3533A566BA6B@xsun01.ptp.hp.com> X-Mailer: Mulberry/2.1.2 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Jeff, -- MEYER,JEFFREY D (HP-Cupertino,ex1) wrote on 28 April 2003 09:20 -0700: > > Juergen, > > Thanks for the response. > > I agree XML in and of itself is marginally human readable. > > I think that basic tools such as Xalan (http://xml.apache.org/), can > be used to "transform" a base XML document into a variety of formats, > including formatting consistent with RFC's. Note that all the IPDR > RFC's were written in XML using Marshall Rose's xml2rfc tools > (http://xml.resource.org/). I also like Marshall's tools. If Paul agrees, we should use them for the data model document. > So the model I would picture is having the Normative XML data model > as an appendix, and having generated information from this data model > populate some of the body sections to aid human readers. This looks like a reasonable way to go. Juergen > Regards, > > Jeff Meyer > >> -----Original Message----- >> From: Juergen Quittek [mailto:quittek@ccrle.nec.de] >> Sent: Monday, April 28, 2003 8:24 AM >> To: MEYER,JEFFREY D (HP-Cupertino,ex1); 'Nevil Brownlee'; >> ipfix@net.doit.wisc.edu >> Subject: RE: [ipfix] Editors for IPFIX drafts >> >> >> Jeff, >> >> I do see a value in such a formal description. But there is a >> trafe-off. >> >> I also see a degradation of human readability. You loose means of >> structuring your document text when using XML, for example grouping >> attributes that are related. (Of course you could do so in >> XML, but it is >> much harder to find the heading "Port-related Attributes" in >> an XML document, >> because headings cannot be well highlighted and numbering of >> headings is >> not well supported. >> >> Anyway, I like the idea of an XML encoding of the model, but >> I we should >> not go this way if we cannot find a good way of getting it >> hunam readable >> (maybe by applying formatting rules). >> >> An option would also be adding the XML code to the document >> as an appendix. >> >> Juergen >> >> >> -- MEYER,JEFFREY D (HP-Cupertino,ex1) wrote on 28 April 2003 >> 10:57 -0400: >> >> > Hi, >> > >> > My particular interest on working on the information model would >> > be on developing a more formal and standard way of describing the >> > information elements which are used in IPFIX communication. >> > >> > The modelling of information elements should ideally be done >> > using a formal description language, so that tools can be >> constructed >> > which do not require humans to transcribe this information. >> > >> > I would propose the use of XML-Schema to provide this formal >> > description. Through the use of XSL processors, prose >> based descriptions >> > and tables can be produced if these are considered desireable. >> > >> > NOTE: describing the information model formally does NOT mean >> > that encoding and transport for IPFIX need to change in any way. It >> > simply means that rather than having as the base >> description of information >> > items some prose in a document, you instead have a machine readable >> > representation. >> > >> > >> > As an example of a formal (but currently incomplete) information >> > model for IPFIX, see: >> > http://www.ipdr.org/documents/ipfix/ipfixService-20020902.xml >> > >> > I would recommend making such a representation the normative form >> > of the IPFIX information model, much like SNMP MIBs. >> > >> > >> > Please let me know if people see value in this exercise. >> > >> > Regards, >> > >> > Jeff Meyer >> > >> >> -----Original Message----- >> >> From: Nevil Brownlee [mailto:n.brownlee@auckland.ac.nz] >> >> Sent: Sunday, April 27, 2003 8:44 PM >> >> To: ipfix@net.doit.wisc.edu >> >> Subject: [ipfix] Editors for IPFIX drafts >> >> >> >> >> >> >> >> Hello all: >> >> >> >> Following the IPFIX meeting at the San Francisco IETF we asked for >> >> volunteers to edit the IPFIX drafts. The current list is: >> >> >> >> Applicability: Tanja Zseby, Reinaldo Penno >> >> >> >> Architecture: Ganesh Sadasivan, Nevil Brownlee >> >> >> >> Information Model: Paul Calato, Jeff Meyer, Juergen Quittek >> >> >> >> Protocol: Mark Fulmer, Paul Calato, Reinaldo Penno >> >> >> >> At this stage we already have old (expired) versions of the >> >> Architecture (-02.txt) and Data Model (-01.txt) drafts on the IPFIX >> >> web page; would the editors of all four drafts please email >> >> me an initial >> >> version of their new drafts to published as placeholders >> >> within the next >> >> few weeks. >> >> >> >> After that, the editors will work on developing better >> versions of the >> >> drafts which can be discussed on the IPFIX list and submitted >> >> by mid-June, >> >> i.e. in plenty of time for the Vienna IETF meeting in July. >> >> >> >> Cheers, Nevil >> >> >> >> PS: Good to see some real discussion on the list. Lets >> concentrate on >> >> producing text the draft editors can use! >> >> >> >> -------------------------------------------------------------- >> >> --------- >> >> Nevil Brownlee Director, Technology >> Development >> >> Phone: +64 9 373 7599 x88941 ITSS, The University >> of Auckland >> >> FAX: +64 9 373 7021 Private Bag 92019, Auckland, >> New Zealand >> >> >> >> >> >> ------------------------------------------------- >> >> This mail sent through University of Auckland >> >> http://www.auckland.ac.nz/ >> >> >> >> -- >> >> Help mailto:majordomo@net.doit.wisc.edu and say "help" >> >> in message body >> >> Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >> >> "unsubscribe ipfix" in message body >> >> Archive http://ipfix.doit.wisc.edu/archive/ >> >> >> > >> > -- >> > Help mailto:majordomo@net.doit.wisc.edu and say >> "help" in message body >> > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >> > "unsubscribe ipfix" in message body >> > Archive http://ipfix.doit.wisc.edu/archive/ >> >> -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon Apr 28 17:09:53 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA14693 for ; Mon, 28 Apr 2003 17:09:52 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19AFni-0000QD-00 for ipfix-list@mil.doit.wisc.edu; Mon, 28 Apr 2003 16:04:38 -0500 Received: from tokyo.ccrle.nec.de ([195.37.70.2]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19AFng-0000Q8-00 for ipfix@net.doit.wisc.edu; Mon, 28 Apr 2003 16:04:36 -0500 Received: from venus.office (venus.office [10.1.1.11]) by tokyo.ccrle.nec.de (8.12.9/8.12.8) with ESMTP id h3SL4aVI002734 for ; Mon, 28 Apr 2003 23:04:36 +0200 (CEST) Received: from [10.1.1.26] (dial02.office [10.1.1.26]) by venus.office (Postfix on SuSE Linux eMail Server 3.0) with ESMTP id 49410400D4 for ; Mon, 28 Apr 2003 22:58:24 +0200 (CEST) Date: Mon, 28 Apr 2003 23:06:00 +0200 From: Juergen Quittek To: ipfix@net.doit.wisc.edu Subject: [ipfix] drawing the line between protocol and data model document Message-ID: <16869116.1051571160@[10.1.1.26]> X-Mailer: Mulberry/2.1.2 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Hi all, When shaping the planned documents, we have to draw a line between the protocol and the data model. Definitely, the data model document defines all field types, but does it also define basic data types, such as '16 bit unsigned integer'? Maybe we should have a set of basic data types defined in the protocol document and restrict the data model doc to these types? This would further structure the data model. Any thoughts? Juergen -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Apr 29 06:30:06 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id GAA11670 for ; Tue, 29 Apr 2003 06:30:06 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19AS3e-0002Ku-00 for ipfix-list@mil.doit.wisc.edu; Tue, 29 Apr 2003 05:09:54 -0500 Received: from sj-core-1.cisco.com ([171.71.177.237]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19AS3b-0002Kl-00 for ipfix@net.doit.wisc.edu; Tue, 29 Apr 2003 05:09:51 -0500 Received: from cisco.com (mrwint.cisco.com [64.103.71.48]) by sj-core-1.cisco.com (8.12.6/8.12.6) with ESMTP id h3TA9ljE017744 for ; Tue, 29 Apr 2003 03:09:48 -0700 (PDT) Received: from cisco.com (edin-comm-vl10-dhcp22.cisco.com [144.254.112.42]) by cisco.com (8.8.8/2.6/Cisco List Logging/8.8.8) with ESMTP id LAA24095; Tue, 29 Apr 2003 11:09:43 +0100 (BST) Message-ID: <3EAE4EFE.1050809@cisco.com> Date: Tue, 29 Apr 2003 11:07:58 +0100 From: Stewart Bryant Reply-To: stbryant@cisco.com Organization: Cisco Systems, Inc. User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.2) Gecko/20030208 Netscape/7.02 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Benoit Claise CC: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Variable length fields data type References: <3E9D6A18.9070901@cisco.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit > Even if PSAMP would decide to report the full packet > (not permitted by RFC 2804 btw), the packet length will > never need more than 16 bits to be coded as the length > in an IP packet is coded on 16 bits. I hope that this does not degenerate into a rat-hole, but I don't think that RFC2804 actually says this. RFC2804 just says that the IETF will not standardise wiretapping (which has all sorts of legal complexities). Stewart -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Apr 29 11:54:49 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA28675 for ; Tue, 29 Apr 2003 11:54:49 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19AX82-0001a3-00 for ipfix-list@mil.doit.wisc.edu; Tue, 29 Apr 2003 10:34:46 -0500 Received: from eng4.oar.net ([192.148.244.24]) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 19AX80-0001Zx-00 for ipfix@net.doit.wisc.edu; Tue, 29 Apr 2003 10:34:44 -0500 Received: (qmail 22207 invoked by uid 4454); 29 Apr 2003 15:34:43 -0000 Date: Tue, 29 Apr 2003 11:34:43 -0400 From: Mark Fullmer To: Maurizio Molina Cc: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] export packet length Message-ID: <20030429113443.A22123@net.ohio-state.edu> References: <0A11633F61BD9F40B43ABCC694004F93F18EEC@zsc3c026.us.nortel.com> <3EAC1F74.9000900@yahoo.com> <20030427224034.A13697@net.ohio-state.edu> <3EAD3F0B.88DCE050@ccrle.nec.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: <3EAD3F0B.88DCE050@ccrle.nec.de>; from molina@ccrle.nec.de on Mon, Apr 28, 2003 at 04:47:39PM +0200 Precedence: bulk Sender: majordomo listserver On Mon, Apr 28, 2003 at 04:47:39PM +0200, Maurizio Molina wrote: > > > > The current Count header field could be changed to Length.... > > Mark, > being a 16 bit counter it would limit the export packet size to 65536 bytes. > Don't you see it as a constraint? > Maurizio It limits the PDU size to 64K. How this translates to packets depends on the transport. I don't think this is an issue. mark -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Apr 29 14:53:39 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA05486 for ; Tue, 29 Apr 2003 14:53:39 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19AaB5-0005kd-00 for ipfix-list@mil.doit.wisc.edu; Tue, 29 Apr 2003 13:50:07 -0500 Received: from atlrel7.hp.com ([156.153.255.213]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19AaB3-0005kX-00 for ipfix@net.doit.wisc.edu; Tue, 29 Apr 2003 13:50:05 -0500 Received: from xatlrelay2.atl.hp.com (xatlrelay2.atl.hp.com [15.45.89.191]) by atlrel7.hp.com (Postfix) with ESMTP id 3155C1C0227C; Tue, 29 Apr 2003 14:50:05 -0400 (EDT) Received: from xatlbh4.atl.hp.com (xatlbh4.atl.hp.com [15.45.89.189]) by xatlrelay2.atl.hp.com (Postfix) with ESMTP id CF9AA1C000AF; Tue, 29 Apr 2003 14:50:04 -0400 (EDT) Received: by xatlbh4.atl.hp.com with Internet Mail Service (5.5.2655.55) id ; Tue, 29 Apr 2003 14:50:04 -0400 Message-ID: <1D3D2C371FCBD947A7897FABBD3533A566BA7E@xsun01.ptp.hp.com> From: "MEYER,JEFFREY D (HP-Cupertino,ex1)" To: "'Juergen Quittek'" , ipfix@net.doit.wisc.edu Subject: RE: [ipfix] drawing the line between protocol and data model docu ment Date: Tue, 29 Apr 2003 14:50:02 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2655.55) Content-Type: text/plain; charset="iso-8859-1" Precedence: bulk Sender: majordomo listserver Juergen, I agree that the data model should have a well defined type system. And that that typing system should reflect the requirements. This is in part why I was hoping that the protocol itself would be more explicit here. (See my mail http://ipfix.doit.wisc.edu/archive/1604.html) There is also a type list in that mail. However, I would suggest using the names from XML-Schema part 2 Data types, as this would facilitate creating a formal/normative definition, versus reinventing names for the types. Here are the type names from XML-Schema/IPDR: boolean - true/false (may be encoded in byte) byte unsignedByte short unsignedShort int - 32-bit signed long - 64-bit signed unsignedInt unsignedLong float double dateTime - 32-bit integer representing seconds since EPOCH (1/1/1970 0:00 GMT) string - UTF-8 formatted text ipdr:dateTimeMsec - 64-bit integer representing msecs since EPOCH ipdr:ipV4Addr ipdr:ipV6Addr ipdr:UUID hexBinary - arbitrary sequence of octets If the protocol itself has a basic "octetString" (or from XML, hexBinary) payload, then this can be used to carry new classes of information in the future w/o rejiggering the protocol itself. In addition other "derived" types may be created from the base types. For instance in IPDR, the ipdr:dateTimeMsec is derived from long. Regards, Jeff Meyer > -----Original Message----- > From: Juergen Quittek [mailto:quittek@ccrle.nec.de] > Sent: Monday, April 28, 2003 2:06 PM > To: ipfix@net.doit.wisc.edu > Subject: [ipfix] drawing the line between protocol and data model > document > > > Hi all, > > When shaping the planned documents, we have to draw a line > between the protocol and the data model. > > Definitely, the data model document defines all field types, > but does it also define basic data types, such as '16 bit > unsigned integer'? > > Maybe we should have a set of basic data types defined in > the protocol document and restrict the data model doc to these types? > This would further structure the data model. > > Any thoughts? > > Juergen > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" > in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Apr 29 18:06:54 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA13246 for ; Tue, 29 Apr 2003 18:06:53 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19Ad2L-0001zC-00 for ipfix-list@mil.doit.wisc.edu; Tue, 29 Apr 2003 16:53:17 -0500 Received: from eng4.oar.net ([192.148.244.24]) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 19Ad2K-0001z4-00 for ipfix@net.doit.wisc.edu; Tue, 29 Apr 2003 16:53:16 -0500 Received: (qmail 23900 invoked by uid 4454); 29 Apr 2003 21:53:15 -0000 Date: Tue, 29 Apr 2003 17:53:15 -0400 From: Mark Fullmer To: "MEYER,JEFFREY D \(HP-Cupertino,ex1\)" Cc: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] drawing the line between protocol and data model docu ment Message-ID: <20030429175315.A23821@net.ohio-state.edu> References: <1D3D2C371FCBD947A7897FABBD3533A566BA7E@xsun01.ptp.hp.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: <1D3D2C371FCBD947A7897FABBD3533A566BA7E@xsun01.ptp.hp.com>; from jeff.meyer2@hp.com on Tue, Apr 29, 2003 at 02:50:02PM -0400 Precedence: bulk Sender: majordomo listserver IMHO crippling the protocol to serve the needs of a documentation tool is just silly. It would be possible to use both a Type name and a Length instead of the existing Length, which would allow a collector to process unknown Types. With the existing v9 protocol a collector can ignore an unknown attribute, or possibly just pass it on to post processing tools which might understand it. Your suggested changes no longer allow a collector to ignore unknown attributes if the Type is also unknown. Resorting to encoding all new Types after the initial protocol spec as octet strings is (IMHO) not acceptable. I'm still trying to understand what the benefits to explicit Types sent in templates are. mark On Tue, Apr 29, 2003 at 02:50:02PM -0400, MEYER,JEFFREY D (HP-Cupertino,ex1) wrote: > Juergen, > > I agree that the data model should have a well defined type > system. And that that typing system should reflect the requirements. > > This is in part why I was hoping that the protocol itself would > be more explicit here. (See my mail > http://ipfix.doit.wisc.edu/archive/1604.html) > > There is also a type list in that mail. However, I would suggest > using the names from XML-Schema part 2 Data types, as this would > facilitate creating a formal/normative definition, versus reinventing > names for the types. > > Here are the type names from XML-Schema/IPDR: > > boolean - true/false (may be encoded in byte) > byte > unsignedByte > short > unsignedShort > int - 32-bit signed > long - 64-bit signed > unsignedInt > unsignedLong > float > double > dateTime - 32-bit integer representing seconds since EPOCH > (1/1/1970 0:00 GMT) > string - UTF-8 formatted text > ipdr:dateTimeMsec - 64-bit integer representing msecs since EPOCH > ipdr:ipV4Addr > ipdr:ipV6Addr > ipdr:UUID > hexBinary - arbitrary sequence of octets > > If the protocol itself has a basic "octetString" (or from XML, hexBinary) > payload, > then this can be used to carry new classes of information in the future w/o > rejiggering the protocol itself. > > In addition other "derived" types may be created from the base types. For > instance in IPDR, the ipdr:dateTimeMsec is derived from long. > > Regards, > > Jeff Meyer > > > -----Original Message----- > > From: Juergen Quittek [mailto:quittek@ccrle.nec.de] > > Sent: Monday, April 28, 2003 2:06 PM > > To: ipfix@net.doit.wisc.edu > > Subject: [ipfix] drawing the line between protocol and data model > > document > > > > > > Hi all, > > > > When shaping the planned documents, we have to draw a line > > between the protocol and the data model. > > > > Definitely, the data model document defines all field types, > > but does it also define basic data types, such as '16 bit > > unsigned integer'? > > > > Maybe we should have a set of basic data types defined in > > the protocol document and restrict the data model doc to these types? > > This would further structure the data model. > > > > Any thoughts? > > > > Juergen > > > > -- > > Help mailto:majordomo@net.doit.wisc.edu and say "help" > > in message body > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe ipfix" in message body > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Apr 29 18:37:43 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA15091 for ; Tue, 29 Apr 2003 18:37:43 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19AdXI-0002jG-00 for ipfix-list@mil.doit.wisc.edu; Tue, 29 Apr 2003 17:25:16 -0500 Received: from atlrel7.hp.com ([156.153.255.213]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19AdXG-0002j8-00 for ipfix@net.doit.wisc.edu; Tue, 29 Apr 2003 17:25:14 -0500 Received: from xatlrelay1.atl.hp.com (xatlrelay1.atl.hp.com [15.45.89.190]) by atlrel7.hp.com (Postfix) with ESMTP id 5E80D1C01EF1; Tue, 29 Apr 2003 18:25:14 -0400 (EDT) Received: from xatlbh3.atl.hp.com (xatlbh3.atl.hp.com [15.45.89.188]) by xatlrelay1.atl.hp.com (Postfix) with ESMTP id 4834E1C000B2; Tue, 29 Apr 2003 18:25:14 -0400 (EDT) Received: by xatlbh3.atl.hp.com with Internet Mail Service (5.5.2655.55) id ; Tue, 29 Apr 2003 18:25:14 -0400 Message-ID: <1D3D2C371FCBD947A7897FABBD3533A566BA84@xsun01.ptp.hp.com> From: "MEYER,JEFFREY D (HP-Cupertino,ex1)" To: "'Mark Fullmer'" , "MEYER,JEFFREY D (HP-Cupertino,ex1)" Cc: ipfix@net.doit.wisc.edu Subject: RE: [ipfix] drawing the line between protocol and data model docu ment Date: Tue, 29 Apr 2003 18:25:11 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2655.55) Content-Type: text/plain; charset="iso-8859-1" Precedence: bulk Sender: majordomo listserver Mark, I'll try to clarify: Concept #1: The information model and the encoding are separate. A well defined mapping determines how an information model is to be encoded for a specific use (e.g. IPFIX). The encoding may take whatever approach allows both ends to do something useful with the data. As you point out there is some useful stuff which can be done with the length alone. Concept #2: Derived typing. By defining a base set of types, which must be understood, derived types can be defined in terms of these base types. For instance Diameter does this. This enables a core set of types to be defined and coded to, but also enables richer types later on with an appropriate fallback for systems which aren't aware of the extensions. The benefit I see of explicit typing is that instead of seeing that a field is of length "4 bytes" something which will be seen A LOT. Is that we can instead differentiate between IPAddr, Integer, unsigned integer, and float. This seems handy to me. Your argument seems centered around the long term potential of adding lots of fixed length fields whose base length is not 4,8 or 16 bytes long, for instance MACAddr (although if we agree on going the type route, I'd recommend adding MACAddr and then you have 1,2,4,6,8 and 16 bytes covered. I guess I don't foresee a lot more fixed length fields coming down the pipe in the IPFIX domain. Hence my desire to make it more explicit for the types which do occur all the time and are likely points of extension. WHY? Because then a tool could at least present these in a meaningful way. As opposed to incorrectly formatting a float type as an int. But hey, either way will work. If the goal is to limit the amount of changes, I'll (grudgingly) go along. I haven't heard any opinions from anyone but you in either direction. So, at this point I don't see a quorom either way... Regards, Jeff Meyer > -----Original Message----- > From: Mark Fullmer [mailto:maf@eng.oar.net] > Sent: Tuesday, April 29, 2003 2:53 PM > To: MEYER,JEFFREY D (HP-Cupertino,ex1) > Cc: ipfix@net.doit.wisc.edu > Subject: Re: [ipfix] drawing the line between protocol and data model > docu ment > > > IMHO crippling the protocol to serve the needs of a documentation tool > is just silly. > > It would be possible to use both a Type name and a Length instead of > the existing Length, which would allow a collector to process > unknown Types. > > With the existing v9 protocol a collector can ignore an > unknown attribute, > or possibly just pass it on to post processing tools which > might understand > it. > > Your suggested changes no longer allow a collector to ignore unknown > attributes if the Type is also unknown. Resorting to encoding all > new Types after the initial protocol spec as octet strings is (IMHO) > not acceptable. > > I'm still trying to understand what the benefits to explicit > Types sent > in templates are. > > mark > > On Tue, Apr 29, 2003 at 02:50:02PM -0400, MEYER,JEFFREY D > (HP-Cupertino,ex1) wrote: > > Juergen, > > > > I agree that the data model should have a well defined type > > system. And that that typing system should reflect the > requirements. > > > > This is in part why I was hoping that the protocol itself would > > be more explicit here. (See my mail > > http://ipfix.doit.wisc.edu/archive/1604.html) > > > > There is also a type list in that mail. However, I would suggest > > using the names from XML-Schema part 2 Data types, as this would > > facilitate creating a formal/normative definition, versus > reinventing > > names for the types. > > > > Here are the type names from XML-Schema/IPDR: > > > > boolean - true/false (may be encoded in byte) > > byte > > unsignedByte > > short > > unsignedShort > > int - 32-bit signed > > long - 64-bit signed > > unsignedInt > > unsignedLong > > float > > double > > dateTime - 32-bit integer representing seconds since EPOCH > > (1/1/1970 0:00 GMT) > > string - UTF-8 formatted text > > ipdr:dateTimeMsec - 64-bit integer representing msecs > since EPOCH > > ipdr:ipV4Addr > > ipdr:ipV6Addr > > ipdr:UUID > > hexBinary - arbitrary sequence of octets > > > > If the protocol itself has a basic "octetString" (or from > XML, hexBinary) > > payload, > > then this can be used to carry new classes of information > in the future w/o > > rejiggering the protocol itself. > > > > In addition other "derived" types may be created from the > base types. For > > instance in IPDR, the ipdr:dateTimeMsec is derived from long. > > > > Regards, > > > > Jeff Meyer > > > > > -----Original Message----- > > > From: Juergen Quittek [mailto:quittek@ccrle.nec.de] > > > Sent: Monday, April 28, 2003 2:06 PM > > > To: ipfix@net.doit.wisc.edu > > > Subject: [ipfix] drawing the line between protocol and data model > > > document > > > > > > > > > Hi all, > > > > > > When shaping the planned documents, we have to draw a line > > > between the protocol and the data model. > > > > > > Definitely, the data model document defines all field types, > > > but does it also define basic data types, such as '16 bit > > > unsigned integer'? > > > > > > Maybe we should have a set of basic data types defined in > > > the protocol document and restrict the data model doc to > these types? > > > This would further structure the data model. > > > > > > Any thoughts? > > > > > > Juergen > > > > > > -- > > > Help mailto:majordomo@net.doit.wisc.edu and say "help" > > > in message body > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > "unsubscribe ipfix" in message body > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > > -- > > Help mailto:majordomo@net.doit.wisc.edu and say > "help" in message body > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe ipfix" in message body > > Archive http://ipfix.doit.wisc.edu/archive/ > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/