Requirements draft issues

From majordomo@mil.doit.wisc.edu Tue Oct 1 00:35:52 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id AAA07050 for ; Tue, 1 Oct 2002 00:35:52 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17wEcd-0004L5-00 for ipfix-list@mil.doit.wisc.edu; Mon, 30 Sep 2002 23:26:59 -0500 Received: from h65s138a81n47.user.nortelnetworks.com ([47.81.138.65] helo=zsc3s004.nortelnetworks.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 17wEcb-0004KN-00 for ipfix-req@net.doit.wisc.edu; Mon, 30 Sep 2002 23:26:57 -0500 Received: from zsc4c000.us.nortel.com (zsc4c000.us.nortel.com [47.81.138.47]) by zsc3s004.nortelnetworks.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id g914QDE04500; Mon, 30 Sep 2002 21:26:14 -0700 (PDT) Received: by zsc4c000.us.nortel.com with Internet Mail Service (5.5.2653.19) id ; Mon, 30 Sep 2002 21:26:13 -0700 Message-ID: <7B802811BE77D51189910002A55CFD2C0411A7B8@zsc3c032.us.nortel.com> From: "Reinaldo Penno" To: Juergen Quittek , ipfix-req@net.doit.wisc.edu Subject: [ipfix-req] Requirements draft issues - Reinaldo 1 Date: Mon, 30 Sep 2002 21:26:12 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C26902.ACAED428" Precedence: bulk Sender: majordomo listserver This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C26902.ACAED428 Content-Type: text/plain; charset="iso-8859-1" Issue Number : Reinaldo 1 Issue: I think there are some important packet fields/scenarios that should be included in the requirements draft. 1 - We discussed a long time ago to include the VPN-ID as one of the parameters exported. If Ipfix will be used in VPN environments for accouting, QoS, monitoring, etc, this field is a must. 2 - There is mention to intrusion detection/security as one of the applications for IPfix, but there is no requirement to export NATed flows (before and after). In a device that supports NAT (traditional, layer 7 interception, whatever) and it's going to use IPfix for security/intrusion detection, dealing with NAT is must. There is no way any intrusion detection device will find attacker/attacked properly without both sides of a NAT flow. Not to mention accouting, billing, etc. Proposed Modification : I would propose to include both items on the requirements draft. ------_=_NextPart_001_01C26902.ACAED428 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Requirements draft issues - Reinaldo 1

Issue Number : Reinaldo 1

Issue:

I think there are some important packet = fields/scenarios that should be included in the requirements draft. =

1 - We discussed a long time ago to include the = VPN-ID as one of the parameters exported. If Ipfix will be used in VPN = environments for accouting, QoS, monitoring, etc, this field is a = must.

2 - There is mention to intrusion detection/security = as one of the applications for IPfix, but there is no requirement to = export NATed flows (before and after).

In a device that supports NAT (traditional, layer 7 = interception, whatever) and it's going to use IPfix for = security/intrusion detection, dealing with NAT is must. There is no way = any intrusion detection device will find attacker/attacked properly = without both sides of a NAT flow. Not to mention accouting, billing, = etc.

Proposed Modification :

I would propose to include both items on the = requirements draft.

------_=_NextPart_001_01C26902.ACAED428-- -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Oct 1 00:36:17 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id AAA07063 for ; Tue, 1 Oct 2002 00:36:17 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17wEci-0004LV-00 for ipfix-list@mil.doit.wisc.edu; Mon, 30 Sep 2002 23:27:04 -0500 Received: from zrc2s0jx.nortelnetworks.com ([47.103.122.112]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 17wEcg-0004KS-00 for ipfix-req@net.doit.wisc.edu; Mon, 30 Sep 2002 23:27:02 -0500 Received: from zsc4c000.us.nortel.com (zsc4c000.us.nortel.com [47.81.138.47]) by zrc2s0jx.nortelnetworks.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id g914QbU26373; Mon, 30 Sep 2002 23:26:37 -0500 (CDT) Received: by zsc4c000.us.nortel.com with Internet Mail Service (5.5.2653.19) id ; Mon, 30 Sep 2002 21:26:23 -0700 Message-ID: <7B802811BE77D51189910002A55CFD2C0411A7B9@zsc3c032.us.nortel.com> From: "Reinaldo Penno" To: Juergen Quittek , ipfix-req@net.doit.wisc.edu Subject: [ipfix-req] Requirements draft issues - Reinaldo 2 Date: Mon, 30 Sep 2002 21:26:15 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C26902.AE0AEB7C" Precedence: bulk Sender: majordomo listserver This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C26902.AE0AEB7C Content-Type: text/plain; charset="iso-8859-1" Issue Number : Reinaldo 2 Issue: In the beggining of section 5.1 it is stated that "This is based on the requirements specified in the requirement document [2]." But I couldn't find a peer of some of the requirements from architecture draft on the requirements draft. Will these be included on the requirements draft? Excluded from the architecture? For example.... 5.1.2. IPFIX Protocol on IPFIX Device (At Export Process) 1. Ability to detect loss of connectivity with the collector and trigger the appropriate action (eg. a switch over to an alternate collector.) <--- 2. Optionally export flow records to multiple collectors. 3. Optionally re-transmit lost flow records. <--- 4. Exchange control information from the collector, monitor export process and detect any overload in the process of exporting. <--- P Proposed Solution: I would suggest to just take out the requirement section from the architecture draft and incorporate the relevant parts into the requirements draft to avoid confusion and repetition. If people do not agree it would be good if the editors of both documents would put them side by side and compare the wording. ------_=_NextPart_001_01C26902.AE0AEB7C Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Requirements draft issues - Reinaldo 2

Issue Number : Reinaldo 2

Issue:

In the beggining of section 5.1 it is stated that =

"This is based on the requirements specified in = the requirement
document [2]."

But I couldn't find a peer of some of the = requirements from architecture draft on the requirements draft. Will = these be included on the requirements draft? Excluded from the = architecture? For example....

5.1.2. IPFIX Protocol on IPFIX Device (At Export = Process)

     1. Ability to detect loss of = connectivity with the collector and
        trigger = the appropriate action (eg. a switch over to an
        alternate = collector.) <---
     2. Optionally export flow = records to multiple collectors.
     3. Optionally re-transmit = lost flow records. <---
     4. Exchange control = information from the collector, monitor export
        process = and detect any overload in the process of exporting. <--- P

Proposed Solution:

I would suggest to just take out the requirement = section from the architecture draft and incorporate the relevant parts = into the requirements draft to avoid confusion and repetition. If = people do not agree it would be good if the editors of both documents = would put them side by side and compare the wording.

------_=_NextPart_001_01C26902.AE0AEB7C-- -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Oct 1 07:40:43 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id HAA09616 for ; Tue, 1 Oct 2002 07:40:43 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17wLD7-0001v4-00 for ipfix-list@mil.doit.wisc.edu; Tue, 01 Oct 2002 06:29:05 -0500 Received: from odd-brew.cisco.com ([144.254.15.119] helo=strange-brew.cisco.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 17wLD4-0001tx-00 for ipfix-req@net.doit.wisc.edu; Tue, 01 Oct 2002 06:29:03 -0500 Received: from cisco.com (bclaise-isdn-home.cisco.com [10.49.4.218]) by strange-brew.cisco.com (8.11.6+Sun/8.8.8) with ESMTP id g91B6b728322; Tue, 1 Oct 2002 13:06:37 +0200 (CEST) Message-ID: <3D9981BD.1050002@cisco.com> Date: Tue, 01 Oct 2002 13:06:37 +0200 From: Benoit Claise User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.1) Gecko/20020815 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Reinaldo Penno CC: Juergen Quittek , ipfix-req@net.doit.wisc.edu Subject: Re: [ipfix-req] Requirements draft issues - Reinaldo 2 References: <7B802811BE77D51189910002A55CFD2C0411A7B9@zsc3c032.us.nortel.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Reinaldo, > Issue Number : Reinaldo 2 > > Issue: > > In the beggining of section 5.1 it is stated that > > "This is based on the requirements specified in the requirement > document [2]." > You are referring to http://www.ietf.org/internet-drafts/draft-ietf-ipfix-architecture-02.txt But the architecture draft work is on hold until we have consensus on the requirement draft and selected the candidate protocol. Regards, Benoit. > But I couldn't find a peer of some of the requirements from > architecture draft on the requirements draft. Will these be included > on the requirements draft? Excluded from the architecture? For example.... > > 5.1.2. IPFIX Protocol on IPFIX Device (At Export Process) > > 1. Ability to detect loss of connectivity with the collector and > trigger the appropriate action (eg. a switch over to an > alternate collector.) <--- > 2. Optionally export flow records to multiple collectors. > 3. Optionally re-transmit lost flow records. <--- > 4. Exchange control information from the collector, monitor export > process and detect any overload in the process of exporting. > <--- P > > > Proposed Solution: > > I would suggest to just take out the requirement section from the > architecture draft and incorporate the relevant parts into the > requirements draft to avoid confusion and repetition. If people do not > agree it would be good if the editors of both documents would put them > side by side and compare the wording. > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Oct 1 11:04:07 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA21465 for ; Tue, 1 Oct 2002 11:04:07 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17wOQM-0000b2-00 for ipfix-list@mil.doit.wisc.edu; Tue, 01 Oct 2002 09:54:58 -0500 Received: from [64.95.122.60] (helo=RS-SC-EXC4.rs.riverstonenet.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 17wOQJ-0000a5-00; Tue, 01 Oct 2002 09:54:56 -0500 Received: from riverstonenet.com ([134.141.180.71]) by RS-SC-EXC4.rs.riverstonenet.com with Microsoft SMTPSVC(5.0.2195.4905); Tue, 1 Oct 2002 07:54:23 -0700 Message-ID: <3D99B6AF.5B2FD5E1@riverstonenet.com> Date: Tue, 01 Oct 2002 10:52:31 -0400 From: calato@riverstonenet.com X-Mailer: Mozilla 4.75C-CCK-MCD NSCPCD475 [en] (X11; U; SunOS 5.6 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: Reinaldo Penno CC: req , ipfix-eval-team@net.doit.wisc.edu Subject: Re: [ipfix-req] IPFIX Requirement draft status - section 6.3.2 References: <7B802811BE77D51189910002A55CFD2C04119EF5@zsc3c032.us.nortel.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 01 Oct 2002 14:54:23.0578 (UTC) FILETIME=[6E266BA0:01C2695A] Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit I agree, we need more clarity on this issue. Here are the 4 items I see being implied when we discuss reliability: 1. Congestion awareness. 2. Message ordering 3. Transport reliability 4. Application layer reliability Since we do not have infinite storage capacity a fifth one is... 5. Tracking data loss (statistics) #1 is required by the IETF. #2 depends on the protocol (i.e. do things break if messages are not processed in order.) With #3 a certain amount of data may be lost when the connection goes down but given a valid connection the data has guaranteed delivery. With #4, the application needs to acknowledge receipt of messages. Unacknowledged messages are resent. This one also creates a problem of weeding out duplicate data since mesasges may be resent. We would also need to define the semantics of the Ack. Does it only mean message received or does it mean message recieved and processed (i.e. persisted in case of failure). Also, since different applications require different levels of reliability, is this aspect tunable? The main reason, I think, for making it tunable would be if there is a significant performance difference between the levels. I'll stop here for comments from the list. Paul > Reinaldo Penno wrote: > > Hello, > > I'm looking to understand/really clarify this requirement. It seems > ambiguous...at least to me. I do not understand what is the > meaning/purpose of "-->abscence<-- of reliability....MUST be > indicated". Shouldn't we just rephrase that by just saying that the > protocol MUST be reliable, provide in-sequence delivery and > retransmission? > > It seems to me we should have 2 options: > > 1. The IPfix protocol runs over TCP or SCTP > 2. The ipfix protocol does not run over TCP or TCP but provide a > service equal or in excess to what is provided by TCP/SCTP in terms of > reliability, congestion awareness and in-sequence delivery. > > Reading this text I feel there is a third option which is: > > 3. "the protocol should run over TCP or SCTP, but you can sidestep > this requirement by putting a sequence number on the application layer > and/or providing 'abscence' of reliability"... > > Or maybe one should ask what's the definition of a reliable protocol > as far as the ipfix protocol is concerned... > > "6.3.2. Reliability > > Absence of reliability of the data transfer, for example caused by > loss or reordering of packets containing flow information, MUST be > indicated. > > Please note that if an unreliable transport protocol is used, > reliability can be provided by higher layers. In such a case only > lack of overall reliability MUST be indicated. For example > reordering > could be dealt with by adding a sequence number to each packet." > > regards, > > Reinaldo -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Oct 1 12:04:23 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA24217 for ; Tue, 1 Oct 2002 12:04:22 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17wPK8-0002LR-00 for ipfix-list@mil.doit.wisc.edu; Tue, 01 Oct 2002 10:52:36 -0500 Received: from [64.95.122.60] (helo=RS-SC-EXC4.rs.riverstonenet.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 17wPK6-0002Ks-00; Tue, 01 Oct 2002 10:52:34 -0500 Received: from riverstonenet.com ([134.141.180.71]) by RS-SC-EXC4.rs.riverstonenet.com with Microsoft SMTPSVC(5.0.2195.4905); Tue, 1 Oct 2002 08:52:00 -0700 Message-ID: <3D99C430.A73244C5@riverstonenet.com> Date: Tue, 01 Oct 2002 11:50:08 -0400 From: calato@riverstonenet.com X-Mailer: Mozilla 4.75C-CCK-MCD NSCPCD475 [en] (X11; U; SunOS 5.6 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: Reinaldo Penno CC: eval-team , ipfixx Subject: [ipfix] Re: Comments on LFAP Version 5.0 References: <7B802811BE77D51189910002A55CFD2C04119EF8@zsc3c032.us.nortel.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 01 Oct 2002 15:52:01.0348 (UTC) FILETIME=[7B244040:01C26962] Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit > Reinaldo Penno wrote: > > Hello paul, > > during the process of evaluating LFAP I read the base + data model > draft again. I have some general suggestions/questions (I'm doing this > for all candidates...). > > a) In section 2 there is a typo > > "then the CCE MUSST terminate the TCP"... > > b) > > I think on the text below you mean > "...reachable from the CCE" > Actually I do mean FAS. The CCE might be able to reach the FAS but another FAS may not be able to reach it. I think the example is if the IP address is NATed, the FAS cannot contact the other server given that IP address. > 4.1 FAS IP Address IE > This is the IP Address of a device running a Flow Accounting > Server > that may or may not be reachable from the FAS. The CCE uses this > IE > in the Connection Request message. IE format is: > > c) In section 4.2 there is a typo > > "The record descriptor consist of two fields the first > field is the length of the fixed information field. The second > is" > > Maybe it should be ....of two fields. The first field is the lenght of > the fixed information field, the second..." Yep. > > d) In section 5.11 there is a typo.. > > "The FAS sends a Keep Alive > (KA) messge so the CCE can determine " > > s/messge/message > > e) > > In regards to 5.12 (below), why the CCE cannot use DR also? How the > CCE terminates a lfap session gracefully (perhaps indicating some > error code if needed?) > Since the CCE initiates the connection it is the one which needs to be redirected. I'm not sure what it means to redirect the listener. The LFAP server, at the moment, does not need to know why the CCE disconnected. If there is a need for the info, I would use an AR message to send it, not the DR. > "5.12 Disconnect Request (DR) Message > > A DR message is sent only by the FAS to request the CCE > disconnect. > It can be sent when the FAS is in the Send State or Connected > State. DR messages may contain one optional IE which is the FAS > IP > Address IE. The FAS may suggest an alternative FAS by supplying > the > FAS IP Address IE. Status is set to SUCCESS in DR messages. > > The Optional IE that can be transmitted in a DR message is a: > FAS IP Address IE" > > f) Would you be so kind to give a concrete example on how the lfap > packet would look like when the mulitple record format explained in > section 4.2 is used? This IE is mentioned in this section and nowhere > else in both drafts (base and data model). Except when you say in the > data model "The multiple record IE MUST NOT be used.". Is this IE > being used or not? Today it is mostly used in the FUN message. For example, Note - FIXED-START, FIXED-END, FORMAT-START, FORMAT-END, RECORD-START, RECORD-END do not exist in the message they were added for readability type=multi-record, len = 84 Fixed-Info-len = 16 Record-format-len = 12 ___ FIXED START____ type = time, len = 4, value = 9:00 type = state, len = 4, value = active ____FIXED-END_____ ____FORMAT-START__ type = flow id, len = 12 type = bytes, len = 8 type = packets, len = 8 ____FORMAT-END____ ____RECORD-START__ 1.1.5 3773 45 ____RECORD-END___ ____RECORD-START_ 1.1.67 67990 145 ____RECORD-END___ ... So each flow is updated with time, state, flowid, bytes, packets. The time and state are listed only once, the rest are given for each flow. There was restriction on using multiple record format in the FAR message. It was supposed to be lifted in LFAP V5 but that change didn't make it in. I'll need to do an update to fix that. > > g) I liked the detailed explanations on the protocol machinery, and > actually liked the application level intelligence that lfap provides > but IMHO I feel something similar could be done in respect to examples > of packet formats in order to help understand the draft. Is it > possible to give some concrete examples on the packet format? > > For example (lfap header + mandatory IEs), (lfap header + mandatory > IEs + some optional IEs) for some flows? Some examples of what I'm > talking about can be found in section 10 of > draft-bclaise-netflow-9-00.txt. Yes, I can do that. I'll work on it this week and send out the examples to the list. > > h) In regards the Flow ID IE in the data model I could ask you for > clarifications, but I wouldn't know what to ask for, it's just that it > seems this concept deserves more wording to better understand it. For > example, how is the flow identifier exactly prefix choosen? What > happens in a fail-over scenario (CCE crashes, FAS crashes, both crash, > etc) in regards to this Flow ID IE? I can add more text on that and provide an example of how it is done today. For now, here are the details. Flow ID = Prefix(8 bytes) + CCE ID (4 bytes) Basically the FAS picks a globally unique 8 byte prefix. Today that is done by using the FAS server's IP address in the first 4 bytes. The second 4 bytes is a monotonically increasing number. The initial value is picked at random. There are other ways to pick a unique prefix, this is how it's done today. The CCE ID is a monotonically increasing number starting at 1. This 12 byte value uniquely identifies a flow across all FAS's and all CCE's. If the CCE crashes, it will get a new prefix from the FAS. It will start its CCE ID from 1 and increment from there. If the FAS crashes, the CCE can continue to use the prefix. Existing flows keep their flow ID and are merely reported to the new FAS. New flows also operate as normal (i.e. just increment the CCE-ID). When the FAS comes back up, the last prefix given out is in persisted storage, so it continues to hand out prefixes in order. If they both crash, the FAS and CCE operate as described above. Paul > > regards, > > Reinaldo -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Oct 1 12:14:39 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA24848 for ; Tue, 1 Oct 2002 12:14:39 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17wPXv-0002pC-00 for ipfix-list@mil.doit.wisc.edu; Tue, 01 Oct 2002 11:06:51 -0500 Received: from dplonka by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17wPXt-0002p7-00 for ipfix@net.doit.wisc.edu; Tue, 01 Oct 2002 11:06:49 -0500 Date: Tue, 1 Oct 2002 11:06:49 -0500 From: Dave Plonka To: ipfix@net.doit.wisc.edu Subject: [ipfix] I-D ACTION:draft-ietf-ipfix-reqs-06.txt Message-ID: <20021001110649.B27495@doit.wisc.edu> Reply-To: plonka@doit.wisc.edu Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0.1i Organization: UW-Madison, DoIT, Network Services X-VMS-Error: %SYSTEM-W-CONFQUAL, conflicting qualifiers X-Shakespearean-Insult: Thou frothy beetle-headed bugbear Precedence: bulk Sender: majordomo listserver IPFIX folks, Below please find the announcement regarding the availability of the updated requirements draft. The following link to it now appears on the IPFIX web site index: http://www.ietf.org/internet-drafts/draft-ietf-ipfix-reqs-06.txt I have also placed it here: http://ipfix.doit.wisc.edu/req/draft-ietf-ipfix-reqs-06.txt Dave ----- Forwarded message from owner-ipfix@net.doit.wisc.edu ----- To: IETF-Announce: ; Cc: ipfix@net.doit.wisc.edu From: Internet-Drafts@ietf.org Reply-to: Internet-Drafts@ietf.org Subject: I-D ACTION:draft-ietf-ipfix-reqs-06.txt Date: Tue, 01 Oct 2002 07:35:49 -0400 Sender: nsyracus@cnri.reston.va.us --NextPart A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the IP Flow Information Export Working Group of the IETF. Title : Requirements for IP Flow Information Export Author(s) : J. Quittek et al. Filename : draft-ietf-ipfix-reqs-06.txt Pages : 29 Date : 2002-9-30 This memo defines requirements for the export of measured IP flow information out of routers, traffic measurement probes and middleboxes. A URL for this Internet-Draft is: http://www.ietf.org/internet-drafts/draft-ietf-ipfix-reqs-06.txt To remove yourself from the IETF Announcement list, send a message to ietf-announce-request with the word unsubscribe in the body of the message. Internet-Drafts are also available by anonymous FTP. Login with the username "anonymous" and a password of your e-mail address. After logging in, type "cd internet-drafts" and then "get draft-ietf-ipfix-reqs-06.txt". A list of Internet-Drafts directories can be found in http://www.ietf.org/shadow.html or ftp://ftp.ietf.org/ietf/1shadow-sites.txt Internet-Drafts can also be obtained by e-mail. Send a message to: mailserv@ietf.org. In the body type: "FILE /internet-drafts/draft-ietf-ipfix-reqs-06.txt". NOTE: The mail server at ietf.org can return the document in MIME-encoded form by using the "mpack" utility. To use this feature, insert the command "ENCODING mime" before the "FILE" command. To decode the response(s), you will need "munpack" or a MIME-compliant mail reader. Different MIME-compliant mail readers exhibit different behavior, especially when dealing with "multipart" MIME messages (i.e. documents which have been split up into multiple messages), so check your local documentation on how to manipulate these messages. Below is the data which will enable a MIME compliant mail reader implementation to automatically retrieve the ASCII version of the Internet-Draft. --NextPart Content-Type: Multipart/Alternative; Boundary="OtherAccess" --OtherAccess Content-Type: Message/External-body; access-type="mail-server"; server="mailserv@ietf.org" Content-Type: text/plain Content-ID: <2002-9-30144601.I-D@ietf.org> ENCODING mime FILE /internet-drafts/draft-ietf-ipfix-reqs-06.txt --OtherAccess Content-Type: Message/External-body; name="draft-ietf-ipfix-reqs-06.txt"; site="ftp.ietf.org"; access-type="anon-ftp"; directory="internet-drafts" Content-Type: text/plain Content-ID: <2002-9-30144601.I-D@ietf.org> --OtherAccess-- --NextPart-- ----- End forwarded message ----- -- plonka@doit.wisc.edu http://net.doit.wisc.edu/~plonka ARS:N9HZF Madison, WI -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Oct 1 12:18:51 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA25147 for ; Tue, 1 Oct 2002 12:18:51 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17wPcM-0002wH-00 for ipfix-list@mil.doit.wisc.edu; Tue, 01 Oct 2002 11:11:26 -0500 Received: from [64.95.122.60] (helo=RS-SC-EXC4.rs.riverstonenet.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 17wPcK-0002vM-00 for ipfix-req@net.doit.wisc.edu; Tue, 01 Oct 2002 11:11:24 -0500 Received: from riverstonenet.com ([134.141.180.71]) by RS-SC-EXC4.rs.riverstonenet.com with Microsoft SMTPSVC(5.0.2195.4905); Tue, 1 Oct 2002 09:10:49 -0700 Message-ID: <3D99C89A.8763F153@riverstonenet.com> Date: Tue, 01 Oct 2002 12:08:58 -0400 From: calato@riverstonenet.com X-Mailer: Mozilla 4.75C-CCK-MCD NSCPCD475 [en] (X11; U; SunOS 5.6 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: Ganesh Sadasivan CC: req Subject: Re: [ipfix-req] Re: IPFIX Requirements Questions References: <3D6BC9AD.CAC153C8@riverstonenet.com> <3D761701.E5986839@riverstonenet.com> <3D926ED7.55A3E8E@cisco.com> <3D9316A7.54B16699@riverstonenet.com> <3D932A40.D96DD195@cisco.com> <3D933258.B9BD4EA0@riverstonenet.com> <3D9357CB.4DC32290@cisco.com> <3D936678.639BBEE6@riverstonenet.com> <3D987831.5B277F0B@cisco.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 01 Oct 2002 16:10:50.0479 (UTC) FILETIME=[1C27F7F0:01C26965] Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Ganesh Sadasivan wrote: > > Just taking the topic of DP alone. > > > > > > > > > > > I think this points to an area of weakness for us (IPFIX). We are > > > > having a hard time having a discussion. > > > > > > > > Lets say that DP = "Distinguising Properties". Where DP is > > > > the set of fields and functions, etc... that are used > > > > to define a flow. Lets assume for now that the set of possible > > > > fields and functions, etc... is fully defined somewhere > > > > (it's not, but assume it is for now.). > > > > > > > > If we require that each flow can be mapped to its DP > > > > then the discussion moves to how that happens. > > > > > > > > There are many ways. Here are a few which come to mind... > > > > > > > > > > Does it matter how the DP is identified & send (whether with each flow or split > > > across flow & control or otherwise) is sent, from a req. doc perspective? > > > As long as we define DP (which is what you are suggesting to do) and tell that DP > > > must give enough information so that the full flow context can be derived at the > > > collector, will it not solve this issue? > > > > > > > Yes it will. I gave example to make sure we were on the > > same page. They would not go in the req doc. > > > > > > > > > > 1) send the DP with each flow. > > > > 2) send the DP with an ID. Then each flow contains > > > > a DP-ID field. > > > > 3) Send the DP for noraml operation and the DP for > > > > overload situation. Then only state changes need > > > > to be delivered. > > > > > > > > #3 above seems to be what you are talking about. > > > > > > > > However, I think we need to define DP as a term and the set of > > > > possible fields and functions first. Add the requirement of mapping > > > > a flow to a DP. > > > > > > As I see DP is a function of {1.metering process(s) employed, 2. exporter state, > > > 3. fields collected}. > > > > I don't see a direct relationship #1 and #2/#3. For example > > I could have a DP of source and dest IP but collect (i.e. report) > > Src/Dst AS number. Also, the state may change from normal > > to overload with no change in DP but rather a dropping of > > additional flows. > > It depends on the overload policy. > * If it is such that say more aggregation (by using less fields) is done in > the case of an overload (which effecively reduces the # of records > exported) then there is a relation between 1 & fields exported (may not > be the fields collected as I mentioned above). > * If there are fields extracted in the packet path which are not directly > derived from the packet (like AS #) in the case of an overload, one may > not collect these fields in the packet path itself. > * When the exporter state is changed to overload, the metering process > may be changed from sampling interval X to Y. That is why I said "direct" relationship. There may be a secondary relationship but one does not drive the other. Thus, state is not a defining feature of DP. Paul > > Thanks > Ganesh > > > > > > > I think DP stands pretty much on its own. > > > In other words > > fields collected and the state are not part of the DP > > definition. Those things may influence which DP is used > > but they are not part of the DP definition itself. > > > > But I think we are getting a little ahead of the game. > > We need to define DP and its fields and functions. > > > > Once we have consensus that we need to define DP and its > > fields and functions and we agree we need to map flow > > to DP, then we can kick off the discussion. > > > > > There is a furthur inter dependency between > > > * 2&1 (with the exporter state change metering could change), > > > * 2 & 3 (with the exporter state change fields collected could change), > > > * 1 & 3 (fields are input tometering functions). > > > > > > Am I missing anything? > > > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Oct 1 13:03:42 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA26978 for ; Tue, 1 Oct 2002 13:03:41 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17wQFW-00045j-00 for ipfix-list@mil.doit.wisc.edu; Tue, 01 Oct 2002 11:51:54 -0500 Received: from [64.95.122.60] (helo=RS-SC-EXC4.rs.riverstonenet.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 17wQFU-00044y-00; Tue, 01 Oct 2002 11:51:52 -0500 Received: from riverstonenet.com ([134.141.180.71]) by RS-SC-EXC4.rs.riverstonenet.com with Microsoft SMTPSVC(5.0.2195.4905); Tue, 1 Oct 2002 09:51:18 -0700 Message-ID: <3D99D216.5391B6AD@riverstonenet.com> Date: Tue, 01 Oct 2002 12:49:26 -0400 From: calato@riverstonenet.com X-Mailer: Mozilla 4.75C-CCK-MCD NSCPCD475 [en] (X11; U; SunOS 5.6 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: Peter Ludemann CC: Reinaldo Penno , req , ipfix-eval-team@net.doit.wisc.edu Subject: Re: [ipfix-req] IPFIX Requirement draft status - section 5.8 References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 01 Oct 2002 16:51:19.0049 (UTC) FILETIME=[C3B25790:01C2696A] Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit I think maybe you are using to narrow a definition of the term "protocol". The IPFIX protocol specification needs to cover more than the message format and message exchange. Paul -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Oct 1 15:19:44 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA02133 for ; Tue, 1 Oct 2002 15:19:43 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17wSR1-0000dS-00 for ipfix-list@mil.doit.wisc.edu; Tue, 01 Oct 2002 14:11:55 -0500 Received: from sj-msg-core-3.cisco.com ([171.70.157.152]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 17wSR0-0000ch-00 for ipfix-req@net.doit.wisc.edu; Tue, 01 Oct 2002 14:11:54 -0500 Received: from mira-sjcd-1.cisco.com (IDENT:mirapoint@mira-sjcd-1.cisco.com [171.69.43.44]) by sj-msg-core-3.cisco.com (8.12.2/8.12.2) with ESMTP id g91J3faa013337; Tue, 1 Oct 2002 12:11:16 -0700 (PDT) Received: from cisco.com (dhcp-171-71-137-27.cisco.com [171.71.137.27]) by mira-sjcd-1.cisco.com (Mirapoint Messaging Server MOS 3.1.0.66-GA) with ESMTP id AAJ00687; Tue, 1 Oct 2002 12:01:11 -0700 (PDT) Message-ID: <3D99F0BD.2B362D26@cisco.com> Date: Tue, 01 Oct 2002 12:00:14 -0700 From: Ganesh Sadasivan X-Mailer: Mozilla 4.76 [en]C-CCK-MCD (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: calato@riverstonenet.com CC: req Subject: Re: [ipfix-req] Re: IPFIX Requirements Questions References: <3D6BC9AD.CAC153C8@riverstonenet.com> <3D761701.E5986839@riverstonenet.com> <3D926ED7.55A3E8E@cisco.com> <3D9316A7.54B16699@riverstonenet.com> <3D932A40.D96DD195@cisco.com> <3D933258.B9BD4EA0@riverstonenet.com> <3D9357CB.4DC32290@cisco.com> <3D936678.639BBEE6@riverstonenet.com> <3D987831.5B277F0B@cisco.com> <3D99C89A.8763F153@riverstonenet.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit calato@riverstonenet.com wrote: > Ganesh Sadasivan wrote: > > > > Just taking the topic of DP alone. > > > > > > > > > > > > > > > I think this points to an area of weakness for us (IPFIX). We are > > > > > having a hard time having a discussion. > > > > > > > > > > Lets say that DP = "Distinguising Properties". Where DP is > > > > > the set of fields and functions, etc... that are used > > > > > to define a flow. Lets assume for now that the set of possible > > > > > fields and functions, etc... is fully defined somewhere > > > > > (it's not, but assume it is for now.). > > > > > > > > > > If we require that each flow can be mapped to its DP > > > > > then the discussion moves to how that happens. > > > > > > > > > > There are many ways. Here are a few which come to mind... > > > > > > > > > > > > > Does it matter how the DP is identified & send (whether with each flow or split > > > > across flow & control or otherwise) is sent, from a req. doc perspective? > > > > As long as we define DP (which is what you are suggesting to do) and tell that DP > > > > must give enough information so that the full flow context can be derived at the > > > > collector, will it not solve this issue? > > > > > > > > > > Yes it will. I gave example to make sure we were on the > > > same page. They would not go in the req doc. > > > > > > > > > > > > > 1) send the DP with each flow. > > > > > 2) send the DP with an ID. Then each flow contains > > > > > a DP-ID field. > > > > > 3) Send the DP for noraml operation and the DP for > > > > > overload situation. Then only state changes need > > > > > to be delivered. > > > > > > > > > > #3 above seems to be what you are talking about. > > > > > > > > > > However, I think we need to define DP as a term and the set of > > > > > possible fields and functions first. Add the requirement of mapping > > > > > a flow to a DP. > > > > > > > > As I see DP is a function of {1.metering process(s) employed, 2. exporter state, > > > > 3. fields collected}. > > > > > > I don't see a direct relationship #1 and #2/#3. For example > > > I could have a DP of source and dest IP but collect (i.e. report) > > > Src/Dst AS number. Also, the state may change from normal > > > to overload with no change in DP but rather a dropping of > > > additional flows. > > > > It depends on the overload policy. > > * If it is such that say more aggregation (by using less fields) is done in > > the case of an overload (which effecively reduces the # of records > > exported) then there is a relation between 1 & fields exported (may not > > be the fields collected as I mentioned above). > > * If there are fields extracted in the packet path which are not directly > > derived from the packet (like AS #) in the case of an overload, one may > > not collect these fields in the packet path itself. > > * When the exporter state is changed to overload, the metering process > > may be changed from sampling interval X to Y. > > That is why I said "direct" relationship. There may be > a secondary relationship but one does not drive the > other. I think I am getting a sense of what you are speaking. Let me re-state it in my own words and you can correct me if I'm wrong. - A flow is distinguished by the metering process(es) done on the packet, what information is collected out from these operations and any other auxiliary operations done using the collected information. - The metering process(es) itself may change based on the exporter state in which case it results in an entirely different flow. Thanks Ganesh -ganesh > > > Thus, state is not a defining feature of DP. > > Paul > > > > > Thanks > > Ganesh > > > > > > > > > > > I think DP stands pretty much on its own. > > > > > In other words > > > fields collected and the state are not part of the DP > > > definition. Those things may influence which DP is used > > > but they are not part of the DP definition itself. > > > > > > But I think we are getting a little ahead of the game. > > > We need to define DP and its fields and functions. > > > > > > Once we have consensus that we need to define DP and its > > > fields and functions and we agree we need to map flow > > > to DP, then we can kick off the discussion. > > > > > > > There is a furthur inter dependency between > > > > * 2&1 (with the exporter state change metering could change), > > > > * 2 & 3 (with the exporter state change fields collected could change), > > > > * 1 & 3 (fields are input tometering functions). > > > > > > > > Am I missing anything? > > > > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Oct 1 20:38:34 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA10299 for ; Tue, 1 Oct 2002 20:38:33 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17wXO1-0000y9-00 for ipfix-list@mil.doit.wisc.edu; Tue, 01 Oct 2002 19:29:09 -0500 Received: from h65s138a81n47.user.nortelnetworks.com ([47.81.138.65] helo=zsc3s004.nortelnetworks.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 17wXNz-0000xl-00 for ipfix-req@net.doit.wisc.edu; Tue, 01 Oct 2002 19:29:07 -0500 Received: from zrc2s0jx.nortelnetworks.com (zrc2s0jx.nortelnetworks.com [47.103.122.112]) by zsc3s004.nortelnetworks.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id g920SYE19356; Tue, 1 Oct 2002 17:28:34 -0700 (PDT) Received: from zsc4c000.us.nortel.com (zsc4c000.us.nortel.com [47.81.138.47]) by zrc2s0jx.nortelnetworks.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id g920SjV11168; Tue, 1 Oct 2002 19:28:46 -0500 (CDT) Received: by zsc4c000.us.nortel.com with Internet Mail Service (5.5.2653.19) id ; Tue, 1 Oct 2002 17:28:31 -0700 Message-ID: <7B802811BE77D51189910002A55CFD2C0419F77E@zsc3c032.us.nortel.com> From: "Reinaldo Penno" To: Juergen Quittek , ipfix-req@net.doit.wisc.edu Subject: [ipfix-req] Requirements draft issues - Reinaldo 3 Date: Tue, 1 Oct 2002 17:28:32 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C269AA.A303366E" Precedence: bulk Sender: majordomo listserver This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C269AA.A303366E Content-Type: text/plain; charset="iso-8859-1" Issue Number : Reinaldo 3 Issue: I'm looking to understand/really clarify this requirement. It seems ambiguous...at least to me. I do not understand what is the meaning/purpose of "-->abscence<-- of reliability....MUST be indicated". Shouldn't we just rephrase that by just saying that the protocol MUST be reliable, provide in-sequence delivery and retransmission? It seems to me we should have 2 options: 1. The IPfix protocol runs over TCP or SCTP 2. The ipfix protocol does not run over TCP or TCP but provide a service equal or in excess to what is provided by TCP/SCTP in terms of reliability, congestion awareness and in-sequence delivery. Reading this text I feel there is a third option which is: 3. "the protocol should run over TCP or SCTP, but you can sidestep this requirement by putting a sequence number on the application layer and/or providing 'abscence' of reliability"... Or maybe one should ask what's the definition of a reliable protocol as far as the ipfix protocol is concerned... "6.3.2. Reliability Absence of reliability of the data transfer, for example caused by loss or reordering of packets containing flow information, MUST be indicated. Please note that if an unreliable transport protocol is used, reliability can be provided by higher layers. In such a case only lack of overall reliability MUST be indicated. For example reordering could be dealt with by adding a sequence number to each packet." Proposed Solution: clean up the text, clarify what is a reliable protocol/what to expect from one, possibly incorporate the two options suggested above. ------_=_NextPart_001_01C269AA.A303366E Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Requirements draft issues - Reinaldo 3

Issue Number : Reinaldo 3

Issue:

I'm looking to understand/really clarify this = requirement. It seems ambiguous...at least to me. I do not understand = what is the meaning/purpose of "-->abscence<-- of = reliability....MUST be indicated". Shouldn't we just rephrase that = by just saying that the protocol MUST be reliable, provide in-sequence = delivery and retransmission?

It seems to me we should have 2 options:

1. The IPfix protocol runs over TCP or SCTP
2. The ipfix protocol does not run over TCP or TCP = but provide a service equal or in excess to what is provided by = TCP/SCTP in terms of reliability, congestion awareness and in-sequence = delivery.

Reading this text I feel there is a third option = which is:

3. "the protocol should run over TCP or SCTP, = but you can sidestep this requirement by putting a sequence number on = the application layer and/or providing 'abscence' of = reliability"...

Or maybe one should ask what's the definition of a = reliable protocol as far as the ipfix protocol is concerned...

"6.3.2. Reliability

   Absence of reliability of the data = transfer, for example caused by
   loss or reordering of packets = containing flow information, MUST be
   indicated.

   Please note that if an unreliable = transport protocol is used,
   reliability can be provided by higher = layers. In such a case only
   lack of overall reliability MUST be = indicated. For example reordering
   could be dealt with by adding a = sequence number to each packet."

Proposed Solution:

clean up the text, clarify what is a reliable = protocol/what to expect from one, possibly incorporate the two options = suggested above.

------_=_NextPart_001_01C269AA.A303366E-- -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Oct 1 23:51:14 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id XAA14621 for ; Tue, 1 Oct 2002 23:51:14 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17waPY-0005ua-00 for ipfix-list@mil.doit.wisc.edu; Tue, 01 Oct 2002 22:42:57 -0500 Received: from h65s138a81n47.user.nortelnetworks.com ([47.81.138.65] helo=zsc3s004.nortelnetworks.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 17waPW-0005u1-00; Tue, 01 Oct 2002 22:42:54 -0500 Received: from zrc2s0jx.nortelnetworks.com (zrc2s0jx.nortelnetworks.com [47.103.122.112]) by zsc3s004.nortelnetworks.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id g923gLE26436; Tue, 1 Oct 2002 20:42:21 -0700 (PDT) Received: from zsc4c000.us.nortel.com (zsc4c000.us.nortel.com [47.81.138.47]) by zrc2s0jx.nortelnetworks.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id g923gVV22398; Tue, 1 Oct 2002 22:42:32 -0500 (CDT) Received: by zsc4c000.us.nortel.com with Internet Mail Service (5.5.2653.19) id ; Tue, 1 Oct 2002 20:42:17 -0700 Message-ID: <7B802811BE77D51189910002A55CFD2C0419F82D@zsc3c032.us.nortel.com> From: "Reinaldo Penno" To: calato@riverstonenet.com, Peter Ludemann Cc: req , ipfix-eval-team@net.doit.wisc.edu Subject: RE: [ipfix-req] IPFIX Requirement draft status - section 5.8 Date: Tue, 1 Oct 2002 20:42:18 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C269C5.B4D5B676" Precedence: bulk Sender: majordomo listserver This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C269C5.B4D5B676 Content-Type: text/plain; charset="iso-8859-1" Hello Paul, I think that the ipfix *Architecture* needs to cover more than the protocol. But the protocol should cover only..the protocol. Maybe the right word is "solution"... I mean, the solution/architecture ( IPfix internal device implementation + IPfix protocol) must meet certain requirements (metering, observations domains, scalability, etc, etc)... But the evaluation of the protocol part IMO shouldn't take into consideration metering requirements such as the one I posted. Let's see if there are other opinions on this besides Peter's. regards, Reinaldo > -----Original Message----- > From: calato@riverstonenet.com [mailto:calato@riverstonenet.com] > Sent: Tuesday, October 01, 2002 9:49 AM > To: Peter Ludemann > Cc: Penno, Reinaldo [BL60:0430:EXCH]; req; > ipfix-eval-team@net.doit.wisc.edu > Subject: Re: [ipfix-req] IPFIX Requirement draft status - section 5.8 > > > > I think maybe you are using to narrow a definition of the > term "protocol". The IPFIX protocol specification needs to > cover more than the message format and message exchange. > > Paul > ------_=_NextPart_001_01C269C5.B4D5B676 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable RE: [ipfix-req] IPFIX Requirement draft status - section = 5.8

Hello Paul,

I think that the ipfix *Architecture* needs to cover = more than the protocol. But the protocol should cover only..the = protocol. Maybe the right word is "solution"...

I mean, the solution/architecture ( IPfix internal = device implementation + IPfix protocol) must meet certain requirements = (metering, observations domains, scalability, etc, etc)...

But the evaluation of the protocol part IMO shouldn't = take into consideration metering requirements such as the one I posted. = Let's see if there are other opinions on this besides Peter's. =

regards,

Reinaldo

> -----Original Message-----
> From: calato@riverstonenet.com [mailto:calato@riverstonenet.com= ]
> Sent: Tuesday, October 01, 2002 9:49 AM
> To: Peter Ludemann
> Cc: Penno, Reinaldo [BL60:0430:EXCH]; = req;
> ipfix-eval-team@net.doit.wisc.edu
> Subject: Re: [ipfix-req] IPFIX Requirement = draft status - section 5.8
>
>
>
> I think maybe you are using to narrow a = definition of the
> term "protocol". The IPFIX protocol = specification needs to
> cover more than the message format and message = exchange.
>
> Paul
>

------_=_NextPart_001_01C269C5.B4D5B676-- -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed Oct 2 02:10:23 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id CAA26108 for ; Wed, 2 Oct 2002 02:10:23 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17wcaD-0001V7-00 for ipfix-list@mil.doit.wisc.edu; Wed, 02 Oct 2002 01:02:05 -0500 Received: from palrel12.hp.com ([156.153.255.237]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 17wcaB-0001UG-00; Wed, 02 Oct 2002 01:02:03 -0500 Received: from hpcuhe.cup.hp.com (hpcuhe.cup.hp.com [15.0.80.203]) by palrel12.hp.com (Postfix) with ESMTP id 2FC47E00775; Tue, 1 Oct 2002 23:01:49 -0700 (PDT) Received: from simail.cup.hp.com (imail@sim.cup.hp.com [15.16.123.26]) by hpcuhe.cup.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) with ESMTP id XAA19781; Tue, 1 Oct 2002 23:01:43 -0700 (PDT) Received: from cup.hp.com ([15.244.160.26]) by simail.cup.hp.com (InterMail vM.5.01.03.01 201-253-122-118-101-20010319) with ESMTP id <20021002063240.HBDD18196.simail.cup.hp.com@cup.hp.com>; Tue, 1 Oct 2002 23:32:40 -0700 Message-ID: <3D9A8B91.30005@cup.hp.com> Date: Tue, 01 Oct 2002 23:00:49 -0700 From: Jeff Meyer User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:0.9.2) Gecko/20010726 Netscape6/6.1 X-Accept-Language: en-us MIME-Version: 1.0 To: Reinaldo Penno Cc: calato@riverstonenet.com, Peter Ludemann , req , ipfix-eval-team@net.doit.wisc.edu Subject: Re: [ipfix-req] IPFIX Requirement draft status - section 5.8 References: <7B802811BE77D51189910002A55CFD2C0419F82D@zsc3c032.us.nortel.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Reinaldo, I completely agree that there are requirement aspects that have nothing to do with the export protocol. Given the focus on protocols, changing non-protocol requirements now only confuses things. Regards, Jeff Meyer Reinaldo Penno wrote: > Hello Paul, > > I think that the ipfix *Architecture* needs to cover more than the > protocol. But the protocol should cover only..the protocol. Maybe the > right word is "solution"... > > I mean, the solution/architecture ( IPfix internal device implementation > + IPfix protocol) must meet certain requirements (metering, observations > domains, scalability, etc, etc)... > > But the evaluation of the protocol part IMO shouldn't take into > consideration metering requirements such as the one I posted. Let's see > if there are other opinions on this besides Peter's. > > regards, > > Reinaldo > > > -----Original Message----- > > From: calato@riverstonenet.com [mailto:calato@riverstonenet.com] > > Sent: Tuesday, October 01, 2002 9:49 AM > > To: Peter Ludemann > > Cc: Penno, Reinaldo [BL60:0430:EXCH]; req; > > ipfix-eval-team@net.doit.wisc.edu > > Subject: Re: [ipfix-req] IPFIX Requirement draft status - section 5.8 > > > > > > > > I think maybe you are using to narrow a definition of the > > term "protocol". The IPFIX protocol specification needs to > > cover more than the message format and message exchange. > > > > Paul > > > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed Oct 2 02:41:10 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id CAA10767 for ; Wed, 2 Oct 2002 02:41:10 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17wd5E-0002IB-00 for ipfix-list@mil.doit.wisc.edu; Wed, 02 Oct 2002 01:34:08 -0500 Received: from mailhub.xacct.com ([204.253.100.25] helo=xacct.com) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 17wd5C-0002I6-00 for ipfix-eval@net.doit.wisc.edu; Wed, 02 Oct 2002 01:34:06 -0500 Received: (qmail 21687 invoked from network); 2 Oct 2002 06:33:58 -0000 Received: from usmail.xacct.com (204.253.100.12) by mailhub.xacct.com with SMTP; 2 Oct 2002 06:33:58 -0000 Received: from peter ([192.168.254.163]) by usmail.xacct.com (8.11.2/8.11.2) with SMTP id g926b6s08951; Tue, 1 Oct 2002 23:37:06 -0700 From: "Peter Ludemann" To: "Jeff Meyer" , Subject: RE: [ipfix-eval] Discussion of Candidate Protocols Date: Tue, 1 Oct 2002 23:34:04 -0700 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 In-Reply-To: <3D98D86D.3060104@cup.hp.com> Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit My comments on Jeff Meyer's comments (hopefully this discussion won't get indented too many levels by those who rebut me). - IPDR - much richer and more formal extensibility model, leveraging industry standard XML-Schema. Compactness as good as other candidates (after negotiation of "templates"/"descriptors"), simple set of operations. I would argue that a hierarchical format is a Bad Thing. History repeats itself; and the XML people are finding out what old-timers already knew: that there were good reasons why relational databases supplanted hierarchical and network databases on the 1970s ... if anybody wishes me to bore them with the details, just ask ... or read other people's opinions such as http://www.dbazine.com/pascal4.html - Diameter - less compact due to AVP encoding model. Extensibility model formalized, but not provided as a machine readable document. Overhead of acking makes this seem too heavyweight for the requirements as specified. Finally the requirement for a recorder to support the SCTP protocol seems prohibitive, since this may require kernel support lacking on several OS's. - LFAP - has a limited (numeric id based) extensibility model. Protocol is relatively simple. The encoding is tag/len/value meaning it will be less compact than IPDR or NFv9. Compactness is a Good Thing when sending thousands of records per second. This is not just an issue of bandwidth; it is also an issue of processing time -- if you know exactly how a record is laid out, you can process it faster. In some cases, you can avoid copy operations, which can be a significant cost at high data rates. I don't see how tag/len/value encoding can have as high performance as externally defined formats. BTW, doesn't the Diameter spec say that it can use UDP, TCP, or SCTP? (I would love to require SCTP for CRANE; but as it is not wide-spread yet, we must allow the possibility of TCP for now.) - NFv9 - compact, but extensibility model (templates) don't provide the set of information which IPDR does. Relatively simple. - CRANE - proprietary vendor format. More complex than is needed for IPFIX use case (20 different operation codes). Template based (like NFv9), has similar lack of formalism in actual definition of extensions. "Proprietary vendor format" is irrelevant. Netscape, LFAP, NetFlow will all be released under IETF rules if they are adopted as standards. The alleged complexity in CRANE is to provide the necessary level of reliability and flexibility in the templates. One specific design goal was to allow the exporter to be informed of which fields were needed, so that it could adjust its processing accordingly (for our PacketSight probe, we can see roughly 3:1 performance change, depending on the amount of metrics and attributes that are collected). If these are left out, the number of operation codes drops by 1/3. Remove the operations for multiplexing sessions on a connection (which would be unnecessary if we could mandate SCTP) and we're down to 11 operations. 4 of these are for flow control; 3 are for status reporting. Anyway, the number of operations is irrelevant (most of the template operations have similar content) ... the tricky thing in implementing CRANE is getting the reliability and fail-over definitions just right. As to "lack of formalism", which appears to be a sin shared by all by IPDR ... all I can say is that I have taken enough graduate-level courses in formal semantics to have become deeply sceptical of any claim of a "formal model". (Anybody remember the 2nd Algol-68 Report?) In the end, everything boils down to agreements about the meanings of words. And there's nothing stopping future standards that define precisely the meanings of "upstreamByteCount", "applicationResponseTime", etc. which can then be used by everybody. (Or some kind of registration process, such as IANA's.) - peter -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed Oct 2 02:47:11 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id CAA10867 for ; Wed, 2 Oct 2002 02:47:10 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17wd9G-0002OG-00 for ipfix-list@mil.doit.wisc.edu; Wed, 02 Oct 2002 01:38:18 -0500 Received: from mailhub.xacct.com ([204.253.100.25] helo=xacct.com) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 17wd9E-0002O5-00 for ipfix-req@net.doit.wisc.edu; Wed, 02 Oct 2002 01:38:16 -0500 Received: (qmail 21816 invoked from network); 2 Oct 2002 06:38:08 -0000 Received: from usmail.xacct.com (204.253.100.12) by mailhub.xacct.com with SMTP; 2 Oct 2002 06:38:08 -0000 Received: from peter ([192.168.254.163]) by usmail.xacct.com (8.11.2/8.11.2) with SMTP id g926fEs08999; Tue, 1 Oct 2002 23:41:15 -0700 From: "Peter Ludemann" To: "Reinaldo Penno" , Cc: "req" , Subject: RE: [ipfix-req] IPFIX Requirement draft status - section 5.8 Date: Tue, 1 Oct 2002 23:38:12 -0700 Message-ID: MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_0074_01C269A3.9B401890" X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 In-Reply-To: <7B802811BE77D51189910002A55CFD2C0419F82D@zsc3c032.us.nortel.com> Precedence: bulk Sender: majordomo listserver This is a multi-part message in MIME format. ------=_NextPart_000_0074_01C269A3.9B401890 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit RE: [ipfix-req] IPFIX Requirement draft status - section 5.8I would like to make a stronger statement than has been attributed to me: a.. The protocol should cover only the issues of delivering data from the exporter to the collector. It must be expressive enough to contain all the data defined in the data model. b.. The data model should cover the identification of the metrics and attributes, their meanings, and how they are grouped together. c.. The architecture should cover issues such as where data are observed, what data are required and what are optional, when reliability is needed, how the data can be manipulated (e.g., calculating bandwidth), etc., etc. This is classic computer science "divide and conquer". Each layer from protocol, to data model, to architecture, is more complex and more likely to cause argument. So, I have confined my comments to the protocol, even though we also have a "probe" and would like to see standardization of its metrics and attributes. In the end, even the most formal models end up with words describing bits. (And, yes, I would include denotational semantics and related esoterica in this sweeping statement.) So, there must be precise agreement on the definitions of the data types and the standard names used. As a simple example, from our PacketSight probe, does "response time" for HTTP flows mean: a.. TCP-level response (time between seeing the SYN and ACKs) b.. TCP/application-level response (time between sending the SYN and sending the first GET) c.. application-level response (time between the first or last packet of an HTTP GET and the first or last packet of the response or of the first or last packet of the payload) [6 possibilities] d.. how should application-level response be interpreted for status responses such as 3xx (redirection), 404 (file not found), or 5xx (server error)? e.. does the definition take into account out-of-order or duplicate packets? f.. what is the application-level definition with persistent connections? g.. etc., etc. (which is why I've been leaving myself out of the discussions -- I just don't have time to discuss these interesting issues.) We anticipate standard names for some measurements being defined in the future; but we also wish a protocol that is completely open, so that people can define whatever they want. In a sense, it is like a high performance ODBC, which only requires that both client and server agree on the meanings of the record fields. By the way, we intend to use CRANE for more than just IP flow export ... other candidate uses include collecting telephony CDRs, QoS metrics (formerly done with SNMP), web server logs, etc. - peter -----Original Message----- From: Reinaldo Penno [mailto:rpenno@nortelnetworks.com] Sent: Tuesday, October 01, 2002 8:42 PM To: calato@riverstonenet.com; Peter Ludemann Cc: req; ipfix-eval-team@net.doit.wisc.edu Subject: RE: [ipfix-req] IPFIX Requirement draft status - section 5.8 Hello Paul, I think that the ipfix *Architecture* needs to cover more than the protocol. But the protocol should cover only..the protocol. Maybe the right word is "solution"... I mean, the solution/architecture ( IPfix internal device implementation + IPfix protocol) must meet certain requirements (metering, observations domains, scalability, etc, etc)... But the evaluation of the protocol part IMO shouldn't take into consideration metering requirements such as the one I posted. Let's see if there are other opinions on this besides Peter's. regards, Reinaldo > -----Original Message----- > From: calato@riverstonenet.com [mailto:calato@riverstonenet.com] > Sent: Tuesday, October 01, 2002 9:49 AM > To: Peter Ludemann > Cc: Penno, Reinaldo [BL60:0430:EXCH]; req; > ipfix-eval-team@net.doit.wisc.edu > Subject: Re: [ipfix-req] IPFIX Requirement draft status - section 5.8 > > > > I think maybe you are using to narrow a definition of the > term "protocol". The IPFIX protocol specification needs to > cover more than the message format and message exchange. > > Paul > ------=_NextPart_000_0074_01C269A3.9B401890 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable RE: [ipfix-req] IPFIX Requirement draft status - = section 5.8

I=20 would like to make a stronger statement than has been attributed to=20 me:

The=20 protocol should cover only the issues of delivering data from = the=20 exporter to the collector. It must be expressive enough to contain all = the=20 data defined in the data model.
The=20 data model should cover the identification of the metrics and = attributes,=20 their meanings, and how they are grouped together.
The=20 architecture should cover issues such as where data are observed, what = data=20 are required and what are optional, when reliability is needed, how = the data=20 can be manipulated (e.g., calculating bandwidth), etc.,=20 etc.

This is classic computer science = "divide and=20 conquer".

Each layer from protocol, to data model, to = architecture, is=20 more complex and more likely to cause argument. So, I have confined=20 my comments to the protocol, even though we also have a = "probe" and=20 would like to see standardization of its metrics and=20 attributes.

In=20 the end, even the most formal models end up with words describing bits. = (And,=20 yes, I would include denotational semantics and related esoterica in = this=20 sweeping statement.) So, there must be precise agreement on the = definitions of=20 the data types and the standard names used. As a simple example, from = our=20 PacketSight probe, does "response time" for HTTP flows = mean:

TCP-level response (time between seeing the SYN and=20 ACKs)
TCP/application-level response (time between sending the SYN = and=20 sending the first GET)
application-level response = (time=20 between the first or last packet of an HTTP GET and the = first or=20 last packet of the response or of the first or last packet of the = payload) [6=20 possibilities]
how=20 should application-level response be interpreted for status responses = such as=20 3xx (redirection), 404 (file not found), or 5xx (server=20 error)?
does the = definition take into=20 account out-of-order or duplicate packets?
what=20 is the application-level definition with persistent=20 connections?
etc., etc.

(which is why I've been leaving myself out of the discussions = -- I just=20 don't have time to discuss these interesting = issues.)

We=20 anticipate standard names for some measurements being defined in the = future; but=20 we also wish a protocol that is completely open, so that people can = define=20 whatever they want. In a sense, it is like a high performance ODBC, = which only=20 requires that both client and server agree on the meanings of the record = fields.

By=20 the way, we intend to use CRANE for more than just IP flow export ...=20 other candidate uses include collecting telephony CDRs, QoS metrics = (formerly done with SNMP), web server logs, etc.

-=20 peter

-----Original Message-----
From: Reinaldo Penno=20 [mailto:rpenno@nortelnetworks.com]
Sent: Tuesday, October = 01, 2002=20 8:42 PM
To: calato@riverstonenet.com; Peter = Ludemann
Cc:=20 req; ipfix-eval-team@net.doit.wisc.edu
Subject: RE: = [ipfix-req]=20 IPFIX Requirement draft status - section 5.8

Hello Paul,

I think that the ipfix *Architecture* needs to cover = more than=20 the protocol. But the protocol should cover only..the protocol. Maybe = the=20 right word is "solution"...

I mean, the solution/architecture ( IPfix internal = device=20 implementation + IPfix protocol) must meet certain requirements = (metering,=20 observations domains, scalability, etc, etc)...

But the evaluation of the protocol part IMO = shouldn't take=20 into consideration metering requirements such as the one I posted. = Let's see=20 if there are other opinions on this besides Peter's.

regards,

Reinaldo

> -----Original Message-----
>=20 From: calato@riverstonenet.com [mailto:calato@riverstonenet.com<= /A>]=20
> Sent: Tuesday, October 01, 2002 9:49 AM =
> To: Peter Ludemann
> Cc: = Penno,=20 Reinaldo [BL60:0430:EXCH]; req;
>=20 ipfix-eval-team@net.doit.wisc.edu
> = Subject: Re:=20 [ipfix-req] IPFIX Requirement draft status - section 5.8 =
>
>
>=20
> I think maybe you are using to narrow a = definition of the
> term "protocol". The = IPFIX=20 protocol specification needs to
> cover = more than=20 the message format and message exchange.
>=20
> Paul
>=20

------=_NextPart_000_0074_01C269A3.9B401890-- -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed Oct 2 02:57:19 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id CAA10992 for ; Wed, 2 Oct 2002 02:57:19 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17wdLG-0002ls-00 for ipfix-list@mil.doit.wisc.edu; Wed, 02 Oct 2002 01:50:42 -0500 Received: from mailhub.xacct.com ([204.253.100.25] helo=xacct.com) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 17wdLC-0002ll-00 for ipfix-req@net.doit.wisc.edu; Wed, 02 Oct 2002 01:50:38 -0500 Received: (qmail 22056 invoked from network); 2 Oct 2002 06:50:30 -0000 Received: from usmail.xacct.com (204.253.100.12) by mailhub.xacct.com with SMTP; 2 Oct 2002 06:50:30 -0000 Received: from peter ([192.168.254.163]) by usmail.xacct.com (8.11.2/8.11.2) with SMTP id g926rds09083; Tue, 1 Oct 2002 23:53:39 -0700 From: "Peter Ludemann" To: , "Reinaldo Penno" Cc: "req" , Subject: RE: [ipfix-req] IPFIX Requirement draft status - section 6.3.2 Date: Tue, 1 Oct 2002 23:50:37 -0700 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 In-Reply-To: <3D99B6AF.5B2FD5E1@riverstonenet.com> Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit calato@riverstonenet.com wrote Tuesday, October 01, 2002 7:53 AM (with various [snip]s): > 3. Transport reliability > 4. Application layer reliability > 5. Tracking data loss (statistics) > > With #3 a certain amount of data may be lost when the > connection goes down but given a valid connection the > data has guaranteed delivery. And there must be sufficient information so that de-duplication can be done at the receiving end (it is impossible to design fail-over such that no duplicates occur). > With #4, the application needs to acknowledge receipt > of messages .... Does [ACK] only mean message received > or does it mean message recieved and processed (i.e. persisted > in case of failure). I am not sure that this is part of the protocol (for example, the TCP spec does not make this clear; but all TCP implementations that I know of send ACK only at the socket level -- but there is nothing stopping somebody from implementing TCP such that the ACK happens when the data have been fully processed ... in fact, the existence of such implementations would have greatly simplified our work). However, a useful implementation would ACK only when the data have been processed in such a way as to allow recovery after a crash. Otherwise, there's not much point in having a reliable protocol: ordinary TCP would suffice. > Also, since different applications require different levels of > reliability, is this aspect tunable? The main reason, I think, for > making it tunable would be if there is a significant performance > difference between the levels. The protocol must not prevent an efficient reliability mechanism at the receiver. I think it is sufficient that the protocol allows a delay of reception of ACKs (e.g., records 1-100 are sent, and then ACK for 1-50 is received). - peter -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed Oct 2 03:22:32 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id DAA11404 for ; Wed, 2 Oct 2002 03:22:31 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17wdiU-0003QK-00 for ipfix-list@mil.doit.wisc.edu; Wed, 02 Oct 2002 02:14:42 -0500 Received: from [64.95.122.60] (helo=agile.yagosys.com) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 17wdiS-0003Pg-00 for ipfix-eval@net.doit.wisc.edu; Wed, 02 Oct 2002 02:14:40 -0500 Received: (qmail 25572 invoked by uid 10041); 2 Oct 2002 07:14:09 -0000 Date: Wed, 2 Oct 2002 00:14:09 -0700 From: Michael MacFaden To: ipfix-eval@net.doit.wisc.edu Subject: Re: [ipfix-eval] Discussion of Candidate Protocols Message-ID: <20021002071409.GY1995@riverstonenet.com> References: <3D98D86D.3060104@cup.hp.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4i X-Operating-System: GNU/Linux 2.4.18 Precedence: bulk Sender: majordomo listserver On Tue, Oct 01, 2002 at 11:34:04PM -0700, Peter Ludemann wrote: >Compactness is a Good Thing when sending thousands of records per >second. This is not just an issue of bandwidth; it is also an issue >of processing time -- if you know exactly how a record is laid out, >you can process it faster. In some cases, you can avoid copy operations, >which can be a significant cost at high data rates. I don't see how >tag/len/value encoding can have as high performance as externally >defined formats. Having spent a few years debugging snmp agents/mgmt apps, I believe compact formats may imply a tradeoff with debugging and diagnosis capability. Unless your modified tcpdump is capable of caching the template and its offsets, one can't simply figure out from a given pdu if something is askew. Detecting encoding errors (off by one, etc) in such protocols I think will be rather difficult. If one reads the LFAP protocol spec, specifically the List IE, one can see that even AVPs values can be compacted by being able to use repeat counts. BGP over TCP seems to be doing just fine scaling with its avp encoding scheme to send thousands of route entries. I would really like to see us figure out the overall "bandwidth usage" and "cpu/memory usage" in a real network protocol than judging just one aspect such as compact encoding. Factors such as protocol statefullness and cpu time to process any given PDU come into play and may overshadow any one aspect of the the initial (useful) criteria posted. Since all the prospective protocols have existed for some time in their current form in real networks, it shouldn't be hard to gather some real data. Dave Plonka probably already has some sort of test bed/sample input :-) Regards, Mike MacFaden -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed Oct 2 04:37:39 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id EAA12535 for ; Wed, 2 Oct 2002 04:37:39 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17weqV-00059r-00 for ipfix-list@mil.doit.wisc.edu; Wed, 02 Oct 2002 03:27:03 -0500 Received: from mailhub.fokus.gmd.de ([193.174.154.14]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 17weqS-00059S-00; Wed, 02 Oct 2002 03:27:00 -0500 Received: from fokus.gmd.de (dhcp229 [195.37.78.229]) by mailhub.fokus.gmd.de (8.11.6/8.11.6) with ESMTP id g928OCe00789; Wed, 2 Oct 2002 10:24:12 +0200 (MEST) Message-ID: <3D9AAC79.4070501@fokus.gmd.de> Date: Wed, 02 Oct 2002 10:21:13 +0200 From: Sebastian Zander Organization: Fraunhofer FOKUS User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; de-DE; rv:0.9.4.1) Gecko/20020508 Netscape6/6.2.3 X-Accept-Language: de-DE MIME-Version: 1.0 To: Reinaldo Penno CC: calato@riverstonenet.com, Peter Ludemann , req , ipfix-eval-team@net.doit.wisc.edu Subject: Re: [ipfix-req] IPFIX Requirement draft status - section 5.8 References: <7B802811BE77D51189910002A55CFD2C0419F82D@zsc3c032.us.nortel.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Hi Reinaldo, the requirements draft lists requirments for IP flow export. It is not limited to protocol requirements. I completly agree to you that some of the requirments are not applicable for the protocol but I assume that the eval team will not consider those. Cheers, Sebastian Reinaldo Penno wrote: > Hello Paul, > > I think that the ipfix *Architecture* needs to cover more than the > protocol. But the protocol should cover only..the protocol. Maybe the > right word is "solution"... > > I mean, the solution/architecture ( IPfix internal device implementation > + IPfix protocol) must meet certain requirements (metering, observations > domains, scalability, etc, etc)... > > But the evaluation of the protocol part IMO shouldn't take into > consideration metering requirements such as the one I posted. Let's see > if there are other opinions on this besides Peter's. > > regards, > > Reinaldo > > > -----Original Message----- > > From: calato@riverstonenet.com [mailto:calato@riverstonenet.com] > > Sent: Tuesday, October 01, 2002 9:49 AM > > To: Peter Ludemann > > Cc: Penno, Reinaldo [BL60:0430:EXCH]; req; > > ipfix-eval-team@net.doit.wisc.edu > > Subject: Re: [ipfix-req] IPFIX Requirement draft status - section 5.8 > > > > > > > > I think maybe you are using to narrow a definition of the > > term "protocol". The IPFIX protocol specification needs to > > cover more than the message format and message exchange. > > > > Paul > > > -- Sebastian Zander E-mail: zander@fokus.fhg.de FhI FOKUS / Global Networking (GloNe) Tel: +49-30-3463-7287 Kaiserin-Augusta-Allee 31 Fax: +49-30-3463-8287 D-10589 Berlin, Germany www.fokus.fhg.de/usr/sebastian.zander -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed Oct 2 04:48:01 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id EAA12663 for ; Wed, 2 Oct 2002 04:48:01 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17wf1I-0005V0-00 for ipfix-list@mil.doit.wisc.edu; Wed, 02 Oct 2002 03:38:12 -0500 Received: from mailhub.xacct.com ([204.253.100.25] helo=xacct.com) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 17wf1G-0005Ut-00 for ipfix-eval@net.doit.wisc.edu; Wed, 02 Oct 2002 03:38:10 -0500 Received: (qmail 24609 invoked from network); 2 Oct 2002 08:38:02 -0000 Received: from usmail.xacct.com (204.253.100.12) by mailhub.xacct.com with SMTP; 2 Oct 2002 08:38:02 -0000 Received: from peter ([192.168.254.163]) by usmail.xacct.com (8.11.2/8.11.2) with SMTP id g928fCs09874; Wed, 2 Oct 2002 01:41:12 -0700 From: "Peter Ludemann" To: "Michael MacFaden" , Subject: RE: [ipfix-eval] Discussion of Candidate Protocols Date: Wed, 2 Oct 2002 01:38:09 -0700 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 In-Reply-To: <20021002071409.GY1995@riverstonenet.com> Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Michael MacFaden wrote Wednesday, October 02, 2002 12:14 AM > Having spent a few years debugging snmp agents/mgmt apps, > I believe compact formats may imply a tradeoff with > debugging and diagnosis capability. Agree ... (optional) MD5 checksums would have saved me a few grey hairs. One unnamed embedded OS's faulty TCP stacks didn't would have been detected much faster with this ... > Unless your modified tcpdump is capable of caching the template and > its offsets, one can't simply figure out from a given pdu > if something is askew. > Detecting encoding errors (off by one, etc) in such protocols > I think will be rather difficult. I have a Perl script that you might enjoy. But it only works with CRANE. :-) [Hint: do this in two steps: first step extracts records from the stream and deals with issues such as retransmits; second step interprets the records] > If one reads the LFAP protocol spec, specifically the List IE, > one can see that even AVPs values can be compacted by > being able to use repeat counts. BGP over TCP seems to be doing just > fine scaling with its avp encoding scheme to send thousands of > route entries. Agreed. However, we have encountered some situations where saving a single copy operation per output recordcan make all the difference between being able to deliver the data and not. It's highly unlikely that an application will keep data in AVP form, so AVP requires copies. If the protocol packs the data into something that a C programmer might do with a "struct", then memory-to-memory copying can be avoided. > I would really like to see us figure out the overall "bandwidth usage" > and "cpu/memory usage" in a real network protocol than judging just > one aspect such as compact encoding. > Factors such as protocol statefullness and cpu time to process > any given PDU come into play and may overshadow any one aspect of > the the initial (useful) criteria posted. My experience is that once you go past a few thousand records per second, life becomes miserable in determining processing bottlenecks. For example, timeliness of ACKs can be important (especially when ACKs are constrained by persistency issues). When you get into the 10Ks/sec range, a certain amount of purple magic smoke helps. > Since all the prospective protocols have existed for some time in > their current form in real networks, it shouldn't be hard to gather > some real data. Dave Plonka probably already has some sort of test > bed/sample input :-) Well, I don't think he has CRANE, unless he has implemented it from scratch. :-) As things stand, even I can't give you a maximum throughput number right now for CRANEbecause my testbed is transitioning to a "new improved" collector. Also, your mileage will vary, depending a lot on length of records, type of data (string? numbers? and delicate tuning issues on the TCP stacks. (Please, nobody mention VPNs and "long fat pipes" and "TCP selective acknowledgment" ... http://www.erg.abdn.ac.uk/public_html/research/satcom/tcp-evol.html) - peter -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed Oct 2 05:35:29 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id FAA13294 for ; Wed, 2 Oct 2002 05:35:29 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17wfkk-0006l1-00 for ipfix-list@mil.doit.wisc.edu; Wed, 02 Oct 2002 04:25:10 -0500 Received: from mailhub.fokus.gmd.de ([193.174.154.14]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 17wfkh-0006ko-00 for ipfix-eval@net.doit.wisc.edu; Wed, 02 Oct 2002 04:25:08 -0500 Received: from fokus.gmd.de (dhcp229 [195.37.78.229]) by mailhub.fokus.gmd.de (8.11.6/8.11.6) with ESMTP id g929HSe12444; Wed, 2 Oct 2002 11:17:28 +0200 (MEST) Message-ID: <3D9AB8F5.4070209@fokus.gmd.de> Date: Wed, 02 Oct 2002 11:14:29 +0200 From: Sebastian Zander Organization: Fraunhofer FOKUS User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; de-DE; rv:0.9.4.1) Gecko/20020508 Netscape6/6.2.3 X-Accept-Language: de-DE MIME-Version: 1.0 To: Peter Ludemann CC: Jeff Meyer , ipfix-eval@net.doit.wisc.edu Subject: Re: [ipfix-eval] Discussion of Candidate Protocols References: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit First, for me as an advocate I find it somewhat inappropriate to comment on other advocate drafts because I am biased of course ;). It would be a good thing if there are other people not directly releated to the advocates who have some opinions on the protocols. But this is only my personal feeling. Peter Ludemann wrote: > My comments on Jeff Meyer's comments (hopefully this discussion won't get > indented too many levels by those who rebut me). Lets find out by adding another level :) > - IPDR - much richer and more formal extensibility model, > leveraging industry standard XML-Schema. > Compactness as good as other candidates (after > negotiation of "templates"/"descriptors"), simple > set of operations. > > I would argue that a hierarchical format is a Bad Thing. History repeats > itself; and the XML people are finding out what old-timers already knew: > that there were good reasons why relational databases supplanted > hierarchical and network databases on the 1970s ... if anybody wishes me to > bore them with the details, just ask ... or read other people's opinions > such as http://www.dbazine.com/pascal4.html > > - Diameter - less compact due to AVP encoding model. Extensibility > model formalized, but not provided as a machine readable > document. Overhead of acking makes this seem too heavyweight > for the requirements as specified. Finally the requirement > for a recorder to support the SCTP protocol seems prohibitive, > since this may require kernel support lacking on several OS's. The Diameter message definition is of course machine readable. Without the acking there is no application layer reliability which is IMHO a bad idea for accounting. Since one important application for IPFIX is accounting I am wondering how how this can be done reliably without. Maybe there could be a mode without acking for those who don't need it. You are right that the SCTP requirement is somewhat prohibitive but I would love to see SCTP become wide-spead. If SCTP should not make it than probably Diameter will be run mostly over TCP. > - LFAP - has a limited (numeric id based) extensibility model. > Protocol is relatively simple. The encoding is tag/len/value > meaning it will be less compact than IPDR or NFv9. > > Compactness is a Good Thing when sending thousands of records per > second. This is not just an issue of bandwidth; it is also an issue > of processing time -- if you know exactly how a record is laid out, > you can process it faster. In some cases, you can avoid copy operations, > which can be a significant cost at high data rates. I don't see how > tag/len/value encoding can have as high performance as externally > defined formats. Compactness is important but not everything. Human readability is often a nice feature. HTTP, RTSP, SIP are following this philosophy. And an HTTP server needs to process as much transactions/s as possible. On the other hand maybe the IPFIX protocol will be so simple that this is not an issue. However parsing the records is not the complete protocol. There is other stuff do to such as security etc. BTW templates could be used with Diameter as well. Templates and data can be encoded as compound AVPs avoiding tag/len overhead. Most IPFIX attributes are not yet defined within Diameter anyway. But again I think the record format is only a part of the protocol. There is other stuff like the message exchange, security, vendor extensibility mechanism, capability exchange etc. > BTW, doesn't the Diameter spec say that it can use UDP, TCP, or SCTP? It can use TCP, SCTP but not UDP. > (I would love to require SCTP for CRANE; but as it is not wide-spread > yet, we must allow the possibility of TCP for now.) Probably this is true for Diameter as well > - NFv9 - compact, but extensibility model (templates) don't > provide the set of information which IPDR does. Relatively > simple. > > - CRANE - proprietary vendor format. More complex than is > needed for IPFIX use case (20 different operation codes). > Template based (like NFv9), has similar lack of formalism > in actual definition of extensions. > > "Proprietary vendor format" is irrelevant. Netscape, LFAP, NetFlow will > all be released under IETF rules if they are adopted as standards. > > The alleged complexity in CRANE is to provide the necessary level of > reliability and flexibility in the templates. One specific design goal > was to allow the exporter to be informed of which fields were needed, > so that it could adjust its processing accordingly (for our PacketSight > probe, we can see roughly 3:1 performance change, depending on the > amount of metrics and attributes that are collected). If these are left > out, the number of operation codes drops by 1/3. Remove the operations > for multiplexing sessions on a connection (which would be unnecessary > if we could mandate SCTP) and we're down to 11 operations. 4 of these > are for flow control; 3 are for status reporting. > > Anyway, the number of operations is irrelevant (most of the template > operations have similar content) ... the tricky thing in implementing > CRANE is getting the reliability and fail-over definitions just right. > > As to "lack of formalism", which appears to be a sin shared by > all by IPDR ... all I can say is that I have taken enough > graduate-level courses in formal semantics to have become > deeply sceptical of any claim of a "formal model". (Anybody remember > the 2nd Algol-68 Report?) In the end, everything boils down to > agreements about the meanings of words. And there's nothing > stopping future standards that define precisely the meanings > of "upstreamByteCount", "applicationResponseTime", etc. which can > then be used by everybody. (Or some kind of registration process, > such as IANA's.) > > - peter > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ > > Cheers, Sebastian -- Sebastian Zander E-mail: zander@fokus.fhg.de FhI FOKUS / Global Networking (GloNe) Tel: +49-30-3463-7287 Kaiserin-Augusta-Allee 31 Fax: +49-30-3463-8287 D-10589 Berlin, Germany www.fokus.fhg.de/usr/sebastian.zander -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed Oct 2 08:40:50 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id IAA17072 for ; Wed, 2 Oct 2002 08:40:50 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17wicL-00059C-00 for ipfix-list@mil.doit.wisc.edu; Wed, 02 Oct 2002 07:28:41 -0500 Received: from [64.95.122.60] (helo=RS-SC-EXC4.rs.riverstonenet.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 17wicH-00058L-00; Wed, 02 Oct 2002 07:28:37 -0500 Received: from riverstonenet.com ([134.141.180.89]) by RS-SC-EXC4.rs.riverstonenet.com with Microsoft SMTPSVC(5.0.2195.4905); Wed, 2 Oct 2002 05:28:04 -0700 Message-ID: <3D9AE5E5.A9C17CFE@riverstonenet.com> Date: Wed, 02 Oct 2002 08:26:13 -0400 From: calato@riverstonenet.com X-Mailer: Mozilla 4.75C-CCK-MCD NSCPCD475 [en] (X11; U; SunOS 5.6 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: Reinaldo Penno CC: Peter Ludemann , req , ipfix-eval-team@net.doit.wisc.edu Subject: Re: [ipfix-req] IPFIX Requirement draft status - section 5.8 References: <7B802811BE77D51189910002A55CFD2C0419F82D@zsc3c032.us.nortel.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 02 Oct 2002 12:28:04.0694 (UTC) FILETIME=[27F0C760:01C26A0F] Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit > Reinaldo Penno wrote: > > Hello Paul, > > I think that the ipfix *Architecture* needs to cover more than the > protocol. But the protocol should cover only..the protocol. Maybe the > right word is "solution"... > > I mean, the solution/architecture ( IPfix internal device > implementation + IPfix protocol) must meet certain requirements > (metering, observations domains, scalability, etc, etc)... > > But the evaluation of the protocol part IMO shouldn't take into > consideration metering requirements such as the one I posted. Let's > see if there are other opinions on this besides Peter's. Agreed. Your choice of terms works for me as well. Basically "messages & message formats" == "protocol" and "protocol + processing/semantics" == "architecture". Paul > > regards, > > Reinaldo > > > -----Original Message----- > > From: calato@riverstonenet.com [mailto:calato@riverstonenet.com] > > Sent: Tuesday, October 01, 2002 9:49 AM > > To: Peter Ludemann > > Cc: Penno, Reinaldo [BL60:0430:EXCH]; req; > > ipfix-eval-team@net.doit.wisc.edu > > Subject: Re: [ipfix-req] IPFIX Requirement draft status - section > 5.8 > > > > > > > > I think maybe you are using to narrow a definition of the > > term "protocol". The IPFIX protocol specification needs to > > cover more than the message format and message exchange. > > > > Paul > > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed Oct 2 08:53:47 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id IAA17716 for ; Wed, 2 Oct 2002 08:53:47 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17wipk-0005Vw-00 for ipfix-list@mil.doit.wisc.edu; Wed, 02 Oct 2002 07:42:32 -0500 Received: from [64.95.122.60] (helo=RS-SC-EXC4.rs.riverstonenet.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 17wipg-0005VA-00; Wed, 02 Oct 2002 07:42:28 -0500 Received: from riverstonenet.com ([134.141.180.89]) by RS-SC-EXC4.rs.riverstonenet.com with Microsoft SMTPSVC(5.0.2195.4905); Wed, 2 Oct 2002 05:41:55 -0700 Message-ID: <3D9AE924.5E1DA94C@riverstonenet.com> Date: Wed, 02 Oct 2002 08:40:04 -0400 From: calato@riverstonenet.com X-Mailer: Mozilla 4.75C-CCK-MCD NSCPCD475 [en] (X11; U; SunOS 5.6 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: Peter Ludemann CC: Reinaldo Penno , req , ipfix-eval-team@net.doit.wisc.edu Subject: Re: [ipfix-req] IPFIX Requirement draft status - section 6.3.2 References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 02 Oct 2002 12:41:55.0714 (UTC) FILETIME=[17445E20:01C26A11] Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Peter Ludemann wrote: > > calato@riverstonenet.com wrote Tuesday, October 01, 2002 7:53 AM > (with various [snip]s): > > > 3. Transport reliability > > 4. Application layer reliability > > 5. Tracking data loss (statistics) > > > > With #3 a certain amount of data may be lost when the > > connection goes down but given a valid connection the > > data has guaranteed delivery. > > And there must be sufficient information so that de-duplication > can be done at the receiving end (it is impossible to design > fail-over such that no duplicates occur). > > > With #4, the application needs to acknowledge receipt > > of messages .... Does [ACK] only mean message received > > or does it mean message recieved and processed (i.e. persisted > > in case of failure). > > I am not sure that this is part of the protocol (for example, the > TCP spec does not make this clear; but all TCP implementations that > I know of send ACK only at the socket level -- but there is nothing > stopping somebody from implementing TCP such that the ACK happens > when the data have been fully processed ... in fact, the existence > of such implementations would have greatly simplified our work). > > However, a useful implementation would ACK only when the data have > been processed in such a way as to allow recovery after a crash. > Otherwise, there's not much point in having a reliable protocol: > ordinary TCP would suffice. This is why we need to define it as part of the protocol. Otherwise some implementations will ack as soon as the message is seen and others will ack after the message is persisted. The former doesn't provide any crash protection and thus, as you stated, might as well just use TCP. > > > Also, since different applications require different levels of > > reliability, is this aspect tunable? The main reason, I think, for > > making it tunable would be if there is a significant performance > > difference between the levels. > > The protocol must not prevent an efficient reliability mechanism at > the receiver. I think it is sufficient that the protocol allows a > delay of reception of ACKs (e.g., records 1-100 are sent, and then > ACK for 1-50 is received). Right, but my question is do we define one level or reliability for all or allow multiple levels. One level for all IMHO significantly reduces the specification, implementation and inter operability issues. However, allowing multiples may allow significant performance increases when the extra reliability is not needed. I'm not sure which is the best way to go. Paul > > - peter -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed Oct 2 09:05:00 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA18392 for ; Wed, 2 Oct 2002 09:05:00 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17wj2q-0005w9-00 for ipfix-list@mil.doit.wisc.edu; Wed, 02 Oct 2002 07:56:04 -0500 Received: from [64.95.122.60] (helo=RS-SC-EXC4.rs.riverstonenet.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 17wj2n-0005vm-00 for ipfix-eval@net.doit.wisc.edu; Wed, 02 Oct 2002 07:56:01 -0500 Received: from riverstonenet.com ([134.141.180.89]) by RS-SC-EXC4.rs.riverstonenet.com with Microsoft SMTPSVC(5.0.2195.4905); Wed, 2 Oct 2002 05:55:28 -0700 Message-ID: <3D9AEC51.BA498740@riverstonenet.com> Date: Wed, 02 Oct 2002 08:53:37 -0400 From: calato@riverstonenet.com X-Mailer: Mozilla 4.75C-CCK-MCD NSCPCD475 [en] (X11; U; SunOS 5.6 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: Peter Ludemann CC: Michael MacFaden , ipfix-eval@net.doit.wisc.edu Subject: Re: [ipfix-eval] Discussion of Candidate Protocols References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 02 Oct 2002 12:55:29.0016 (UTC) FILETIME=[FC086780:01C26A12] Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Peter Ludemann wrote: > > Michael MacFaden wrote Wednesday, October 02, 2002 12:14 AM > > > Having spent a few years debugging snmp agents/mgmt apps, > > I believe compact formats may imply a tradeoff with > > debugging and diagnosis capability. > > Agree ... (optional) MD5 checksums would have saved me a few > grey hairs. One unnamed embedded OS's faulty TCP stacks didn't > would have been detected much faster with this ... > > > Unless your modified tcpdump is capable of caching the template and > > its offsets, one can't simply figure out from a given pdu > > if something is askew. > > Detecting encoding errors (off by one, etc) in such protocols > > I think will be rather difficult. > > I have a Perl script that you might enjoy. But it only works > with CRANE. :-) > [Hint: do this in two steps: first step extracts records from > the stream and deals with issues such as retransmits; second > step interprets the records] > > > If one reads the LFAP protocol spec, specifically the List IE, > > one can see that even AVPs values can be compacted by > > being able to use repeat counts. BGP over TCP seems to be doing just > > fine scaling with its avp encoding scheme to send thousands of > > route entries. > > Agreed. > > However, we have encountered some situations where saving a single > copy operation per output recordcan make all the difference between > being able to deliver the data and not. It's highly unlikely that > an application will keep data in AVP form, so AVP requires copies. > If the protocol packs the data into something that a C programmer > might do with a "struct", then memory-to-memory copying can be > avoided. Having done a fair amount of tuning on a collector I find it interesting that your performance hinged on a memory copy. In my experince performance was limited by how fast the disk writes are. > > > I would really like to see us figure out the overall "bandwidth usage" > > and "cpu/memory usage" in a real network protocol than judging just > > one aspect such as compact encoding. > > Factors such as protocol statefullness and cpu time to process > > any given PDU come into play and may overshadow any one aspect of > > the the initial (useful) criteria posted. > > My experience is that once you go past a few thousand records per > second, life becomes miserable in determining processing bottlenecks. > For example, timeliness of ACKs can be important (especially when > ACKs are constrained by persistency issues). When you get into the > 10Ks/sec range, a certain amount of purple magic smoke helps. > > > Since all the prospective protocols have existed for some time in > > their current form in real networks, it shouldn't be hard to gather > > some real data. Dave Plonka probably already has some sort of test > > bed/sample input :-) > > Well, I don't think he has CRANE, unless he has implemented it from > scratch. :-) As things stand, even I can't give you a maximum throughput > number right now for CRANEbecause my testbed is transitioning to a > "new improved" collector. Also, your mileage will vary, depending a > lot on length of records, type of data (string? numbers? and delicate > tuning issues on the TCP stacks. (Please, nobody mention VPNs and > "long fat pipes" and "TCP selective acknowledgment" ... > http://www.erg.abdn.ac.uk/public_html/research/satcom/tcp-evol.html) > > - peter > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed Oct 2 09:09:47 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA18691 for ; Wed, 2 Oct 2002 09:09:47 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17wj8o-00065f-00 for ipfix-list@mil.doit.wisc.edu; Wed, 02 Oct 2002 08:02:14 -0500 Received: from [64.95.122.60] (helo=RS-SC-EXC4.rs.riverstonenet.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 17wj8m-00064n-00 for ipfix-eval@net.doit.wisc.edu; Wed, 02 Oct 2002 08:02:12 -0500 Received: from riverstonenet.com ([134.141.180.89]) by RS-SC-EXC4.rs.riverstonenet.com with Microsoft SMTPSVC(5.0.2195.4905); Wed, 2 Oct 2002 06:01:39 -0700 Message-ID: <3D9AEDC2.14852887@riverstonenet.com> Date: Wed, 02 Oct 2002 08:59:46 -0400 From: calato@riverstonenet.com X-Mailer: Mozilla 4.75C-CCK-MCD NSCPCD475 [en] (X11; U; SunOS 5.6 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: Sebastian Zander CC: Peter Ludemann , Jeff Meyer , ipfix-eval@net.doit.wisc.edu Subject: Re: [ipfix-eval] Discussion of Candidate Protocols References: <3D9AB8F5.4070209@fokus.gmd.de> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 02 Oct 2002 13:01:40.0255 (UTC) FILETIME=[D94EFAF0:01C26A13] Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Sebastian Zander wrote: > > First, for me as an advocate I find it somewhat inappropriate to comment on > other advocate drafts because I am biased of course ;). It would be a good > thing if there are other people not directly releated to the advocates who > have some opinions on the protocols. But this is only my personal feeling. For the most part I have refrained for the same reason. Unfortunately the advocates are likely the ones in the best position to comment. So we've taken 5 strong voices out of the discussion. At some point we'll we'll need to open the flood gates. Maybe after the evals come out. Paul > > Peter Ludemann wrote: > > > My comments on Jeff Meyer's comments (hopefully this discussion won't get > > indented too many levels by those who rebut me). > > Lets find out by adding another level :) > > > > - IPDR - much richer and more formal extensibility model, > > leveraging industry standard XML-Schema. > > Compactness as good as other candidates (after > > negotiation of "templates"/"descriptors"), simple > > set of operations. > > > > I would argue that a hierarchical format is a Bad Thing. History repeats > > itself; and the XML people are finding out what old-timers already knew: > > that there were good reasons why relational databases supplanted > > hierarchical and network databases on the 1970s ... if anybody wishes me to > > bore them with the details, just ask ... or read other people's opinions > > such as http://www.dbazine.com/pascal4.html > > > > - Diameter - less compact due to AVP encoding model. Extensibility > > model formalized, but not provided as a machine readable > > document. Overhead of acking makes this seem too heavyweight > > for the requirements as specified. Finally the requirement > > for a recorder to support the SCTP protocol seems prohibitive, > > since this may require kernel support lacking on several OS's. > > The Diameter message definition is of course machine readable. Without > the acking there is no application layer reliability which is IMHO a bad > idea for accounting. Since one important application for IPFIX is accounting > I am wondering how how this can be done reliably without. Maybe there > could be a mode without acking for those who don't need it. > > You are right that the SCTP requirement is somewhat prohibitive but I > would love to see SCTP become wide-spead. If SCTP should not make it than > probably Diameter will be run mostly over TCP. > > > > - LFAP - has a limited (numeric id based) extensibility model. > > Protocol is relatively simple. The encoding is tag/len/value > > meaning it will be less compact than IPDR or NFv9. > > > > Compactness is a Good Thing when sending thousands of records per > > second. This is not just an issue of bandwidth; it is also an issue > > of processing time -- if you know exactly how a record is laid out, > > you can process it faster. In some cases, you can avoid copy operations, > > which can be a significant cost at high data rates. I don't see how > > tag/len/value encoding can have as high performance as externally > > defined formats. > > Compactness is important but not everything. Human readability is often a nice > feature. HTTP, RTSP, SIP are following this philosophy. And an HTTP server > needs to process as much transactions/s as possible. On the other hand maybe > the IPFIX protocol will be so simple that this is not an issue. However > parsing the records is not the complete protocol. There is other stuff > do to such as security etc. > > BTW templates could be used with Diameter as well. Templates and data can be > encoded as compound AVPs avoiding tag/len overhead. Most IPFIX attributes > are not yet defined within Diameter anyway. But again I think the record > format is only a part of the protocol. There is other stuff like the message > exchange, security, vendor extensibility mechanism, capability exchange etc. > > > BTW, doesn't the Diameter spec say that it can use UDP, TCP, or SCTP? > > It can use TCP, SCTP but not UDP. > > > (I would love to require SCTP for CRANE; but as it is not wide-spread > > yet, we must allow the possibility of TCP for now.) > > Probably this is true for Diameter as well > > > > - NFv9 - compact, but extensibility model (templates) don't > > provide the set of information which IPDR does. Relatively > > simple. > > > > - CRANE - proprietary vendor format. More complex than is > > needed for IPFIX use case (20 different operation codes). > > Template based (like NFv9), has similar lack of formalism > > in actual definition of extensions. > > > > "Proprietary vendor format" is irrelevant. Netscape, LFAP, NetFlow will > > all be released under IETF rules if they are adopted as standards. > > > > The alleged complexity in CRANE is to provide the necessary level of > > reliability and flexibility in the templates. One specific design goal > > was to allow the exporter to be informed of which fields were needed, > > so that it could adjust its processing accordingly (for our PacketSight > > probe, we can see roughly 3:1 performance change, depending on the > > amount of metrics and attributes that are collected). If these are left > > out, the number of operation codes drops by 1/3. Remove the operations > > for multiplexing sessions on a connection (which would be unnecessary > > if we could mandate SCTP) and we're down to 11 operations. 4 of these > > are for flow control; 3 are for status reporting. > > > > Anyway, the number of operations is irrelevant (most of the template > > operations have similar content) ... the tricky thing in implementing > > CRANE is getting the reliability and fail-over definitions just right. > > > > As to "lack of formalism", which appears to be a sin shared by > > all by IPDR ... all I can say is that I have taken enough > > graduate-level courses in formal semantics to have become > > deeply sceptical of any claim of a "formal model". (Anybody remember > > the 2nd Algol-68 Report?) In the end, everything boils down to > > agreements about the meanings of words. And there's nothing > > stopping future standards that define precisely the meanings > > of "upstreamByteCount", "applicationResponseTime", etc. which can > > then be used by everybody. (Or some kind of registration process, > > such as IANA's.) > > > > - peter > > > > > > -- > > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe ipfix" in message body > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > Cheers, > > Sebastian > > -- > Sebastian Zander E-mail: zander@fokus.fhg.de > FhI FOKUS / Global Networking (GloNe) Tel: +49-30-3463-7287 > Kaiserin-Augusta-Allee 31 Fax: +49-30-3463-8287 > D-10589 Berlin, Germany www.fokus.fhg.de/usr/sebastian.zander > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed Oct 2 11:23:30 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA24415 for ; Wed, 2 Oct 2002 11:23:30 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17wlEK-0001mF-00 for ipfix-list@mil.doit.wisc.edu; Wed, 02 Oct 2002 10:16:04 -0500 Received: from mailhost2.auckland.ac.nz ([130.216.1.4]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 17wlEB-0001lx-00 for ipfix-eval@net.doit.wisc.edu; Wed, 02 Oct 2002 10:15:55 -0500 Received: from mailhost.auckland.ac.nz (IDENT:mirapoint@mailhost [130.216.191.61]) by mailhost2.auckland.ac.nz (8.9.2/8.9.2/8.9.2-ua) with ESMTP id DAA10981 for ; Thu, 3 Oct 2002 03:15:52 +1200 (NZST) Received: from postbox.auckland.ac.nz (postbox.auckland.ac.nz [130.216.191.126]) by mailhost.auckland.ac.nz (Mirapoint Messaging Server MOS 3.2.1.6-EA) with ESMTP id AIJ78144; Thu, 3 Oct 2002 03:15:51 +1200 (NZST) Received: from localhost (hotlava.auckland.ac.nz [130.216.191.123]) by postbox.auckland.ac.nz (8.11.6/8.11.6) with ESMTP id g92FFpJ10987 for ; Thu, 3 Oct 2002 03:15:51 +1200 Received: from 129.69.30.65 ([129.69.30.65]) by hotlava.auckland.ac.nz (IMP) with HTTP for ; Thu, 3 Oct 2002 03:15:51 +1200 Message-ID: <1033571751.3d9b0da788782@hotlava.auckland.ac.nz> Date: Thu, 3 Oct 2002 03:15:51 +1200 From: Nevil Brownlee To: ipfix-eval@net.doit.wisc.edu Subject: [ipfix-eval] Re: Discussion of Candidate Protocols MIME-Version: 1.0 Content-Type: text/plain Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Internet Messaging Program (IMP) 4.0-cvs X-Originating-IP: 129.69.30.65 Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Hi all: Sebastian and Paul have made very polite remarks about the advocates holding back from discussing the candidate protocols. But there's no need for that! Do please feel free to comment on the other proposals, robust debate on the list will help everyone understand the differences between the protocols. Cheers, Nevil ----------------------------------------------------------------------- Nevil Brownlee Director, Technology Development Phone: +64 9 373 7599 x8941 ITSS, The University of Auckland FAX: +64 9 373 7021 Private Bag 92019, Auckland, New Zealand ------------------------------------------------- This mail sent through IMP: http://horde.org/imp/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed Oct 2 23:52:58 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id XAA16299 for ; Wed, 2 Oct 2002 23:52:58 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17wwna-0000Vu-00 for ipfix-list@mil.doit.wisc.edu; Wed, 02 Oct 2002 22:37:14 -0500 Received: from [208.253.219.211] (helo=narus.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 17wwnY-0000VF-00 for ipfix-eval@net.doit.wisc.edu; Wed, 02 Oct 2002 22:37:12 -0500 Received: from franklin.narus.com (franklin.narus.com [192.168.7.140]) by narus.com (8.11.6/8.11.6) with ESMTP id g933YWu05534; Wed, 2 Oct 2002 20:34:33 -0700 Received: by franklin.narus.com with Internet Mail Service (5.5.2655.55) id <4CNBFRCR>; Wed, 2 Oct 2002 20:34:33 -0700 Message-ID: <580E532D9F7A9B4BAE8A130848E0DDA702083A03@franklin.narus.com> From: Stas Khirman To: "'Peter Ludemann'" , Jeff Meyer , ipfix-eval@net.doit.wisc.edu Subject: RE: [ipfix-eval] Discussion of Candidate Protocols Date: Wed, 2 Oct 2002 20:34:28 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2655.55) Content-Type: text/plain; charset="iso-8859-1" Precedence: bulk Sender: majordomo listserver Peter Ludemann wrote: > My comments on Jeff Meyer's comments (hopefully this > discussion won't get > indented too many levels by those who rebut me). > > - IPDR - much richer and more formal extensibility model, > leveraging industry standard XML-Schema. > Compactness as good as other candidates (after > negotiation of "templates"/"descriptors"), simple > set of operations. > > I would argue that a hierarchical format is a Bad Thing. > History repeats > itself; and the XML people are finding out what old-timers > already knew: > that there were good reasons why relational databases supplanted > hierarchical and network databases on the 1970s ... if > anybody wishes me to > bore them with the details, just ask ... or read other > people's opinions > such as http://www.dbazine.com/pascal4.html I do not think that relational/XML database example is valid at this point - even good old DB applications aren't build on top of one big flat table. In general, telecommunication applications are dealing with complex structural objects. It is why virtually all Telco standards are based on structural data/signal representations (old good ASN.1!). IPDR.org had recognized that we are not dealing with a "flat" objects - it is why structural ( I prefer this term over "hierarchical") representation are suggested. XML is chosen as a convenient tool to express and validate this structure. BTW, earliest IPDR draft used traditional ASN.1 approach, but it had been replaced by XML - myriad debugging, testing, support, processing and so on tools are available (many for free). Binary encoded IPDR ( as suggested in IETF submission) is free from major XML obstacles (transmission overhead and parsing performance) but retain major XML advantages (structure, extensibility, possibility for verification and wide set of tools to support). Bottom line: IPDR suggest to use structural, formal , extensible model to complex objects reports. XML is just a tool to express this formalization (one of many!!). > > - Diameter - less compact due to AVP encoding model. > Extensibility > model formalized, but not provided as a machine readable > document. Overhead of acking makes this seem too heavyweight > for the requirements as specified. Finally the requirement > for a recorder to support the SCTP protocol seems prohibitive, > since this may require kernel support lacking on several OS's. > > - LFAP - has a limited (numeric id based) extensibility model. > Protocol is relatively simple. The encoding is tag/len/value > meaning it will be less compact than IPDR or NFv9. > > Compactness is a Good Thing when sending thousands of records per > second. This is not just an issue of bandwidth; it is also an issue > of processing time -- if you know exactly how a record is laid out, > you can process it faster. In some cases, you can avoid copy > operations, > which can be a significant cost at high data rates. I don't see how > tag/len/value encoding can have as high performance as externally > defined formats. > Hm, I do not think that memcopy operation could have a significant impact. Let's assume we are dealing with 10,000 records per second 100 bytes each. It is mean that in worst case we have to copy 1Mbyte - 250,000 copy operations per second on 32 bit processor. I'm sure it will take only a fraction of one percent on any normal CPU, am I wrong? BTW, majority of TCP stack implementations are doing few memory copies anyway. > BTW, doesn't the Diameter spec say that it can use UDP, TCP, or SCTP? > (I would love to require SCTP for CRANE; but as it is not wide-spread > yet, we must allow the possibility of TCP for now.) > > - NFv9 - compact, but extensibility model (templates) don't > provide the set of information which IPDR does. Relatively > simple. > > - CRANE - proprietary vendor format. More complex than is > needed for IPFIX use case (20 different operation codes). > Template based (like NFv9), has similar lack of formalism > in actual definition of extensions. > > "Proprietary vendor format" is irrelevant. Netscape, LFAP, > NetFlow will > all be released under IETF rules if they are adopted as standards. > Availability of the open-source tested multi-vendor implementation is not a negligible reason for protocol choice. I have no detailed information about DIAMETER, but IPDR implementation is tested, corrected, extended and adopted by collective effort of something like 20-30 vendors ( Jeff , please correct me if I'm wrong in numbers). > The alleged complexity in CRANE is to provide the necessary level of > reliability and flexibility in the templates. One specific design goal > was to allow the exporter to be informed of which fields were needed, > so that it could adjust its processing accordingly (for our > PacketSight > probe, we can see roughly 3:1 performance change, depending on the > amount of metrics and attributes that are collected). If > these are left > out, the number of operation codes drops by 1/3. Remove the operations > for multiplexing sessions on a connection (which would be unnecessary > if we could mandate SCTP) and we're down to 11 operations. 4 of these > are for flow control; 3 are for status reporting. Indeed, CRANE capability to eliminate some of attributes is a nice feature, however it is not clear why it have to be done using in-band signaling. It certainly overcomplicates protocol and make it hard manageable when someone deal with one to many ( multiple consumers) communication. Also, many accounting applications have to use "store and forward" model which breaks in-band configuration approach. In IPDR case, configuration could be expressed as a specific XML-Schema and delivered to IPDR producer out-band ( it is not clear if this configuration step have to be standardized at all - any ideas are very welcome). Transmitted IPDR records always refer to specification used to produce them - you could store records for years without fear to lost association with a specification. IPDR ability to specify in machine/human readable format structure of transmitted information is a crucial feature when you deal with a financially sensitive information. This feature provide consuming application with abilities to verify, store , convert, process or transform into another format without fear of misconfiguration or version incompatibility. (Elimination of the external registration process is one more small, but nice advantage - just remember RADIUS port conflict problem.) Disclaimer: Being a author of early IPDR drafts, I have to be considered as a VERY biased IPDR advocate. Regards Stas Khirman -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu Oct 3 12:28:25 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA22793 for ; Thu, 3 Oct 2002 12:28:24 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17x8UR-0005lJ-00 for ipfix-list@mil.doit.wisc.edu; Thu, 03 Oct 2002 11:06:15 -0500 Received: from palrel12.hp.com ([156.153.255.237]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 17x8UP-0005lA-00 for ipfix-eval@net.doit.wisc.edu; Thu, 03 Oct 2002 11:06:13 -0500 Received: from hpcuhe.cup.hp.com (hpcuhe.cup.hp.com [15.0.80.203]) by palrel12.hp.com (Postfix) with ESMTP id 542D2E00EDD; Thu, 3 Oct 2002 09:06:12 -0700 (PDT) Received: from simail.cup.hp.com (imail@sim.cup.hp.com [15.16.123.26]) by hpcuhe.cup.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) with ESMTP id JAA17663; Thu, 3 Oct 2002 09:06:07 -0700 (PDT) Received: from cup.hp.com ([15.16.124.96]) by simail.cup.hp.com (InterMail vM.5.01.03.01 201-253-122-118-101-20010319) with ESMTP id <20021003163711.HCWJ18196.simail.cup.hp.com@cup.hp.com>; Thu, 3 Oct 2002 09:37:11 -0700 Message-ID: <3D9C6B6C.6010803@cup.hp.com> Date: Thu, 03 Oct 2002 09:08:12 -0700 From: Jeff Meyer User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:0.9.4.1) Gecko/20020508 Netscape6/6.2.3 X-Accept-Language: en-us MIME-Version: 1.0 To: Peter Ludemann Cc: ipfix-eval@net.doit.wisc.edu Subject: Re: [ipfix-eval] Discussion of Candidate Protocols References: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Hi, In this message I just wanted to respond the the discussion around "hierarchical formats". Since the comment indicates the author has not actually read IPDR... The current version of IPDR does in fact require that data be delivered in First Normal Form (just like Relational DB's). IPDR specifies a SUBSET of the rich (overly so?) vocabulary of XML Schema. So, if you look at the schema proposed for IPFIX (in Appendix A of the Advocacy draft), you will see that all of the values are laid out side by side, no nested structures, no repeating of fields. Also to clarify. The proposed protocol uses Compact IPDR format. The wire protocol IS NOT XML, rather XML is simply used to model the data which is sent. So in the example above, the record representing a flow would have a descriptor identifying the type of IPDRRecord being encoded followed by the integer values one next to the other (i.e. 4 bytes/ field). Now this modelling has the benefit that it ALSO defines an equivalent XML encoding for the same compact content. And in addition, since it is FNF, the definition can also be used to define how this information is to be stored in a DB. So you have one description language, based on XML Schema, a wire protocol, a binary format, an XML format and a DB mapper, in a single document. Show me how that is done w/ the other proposed drafts? Regards, Jeff Meyer P.S. I am in the process of defining extensions to IPDR to enable more structured data. (i.e. repeating fields and fields which are not of a primitive type). However this capability does not appear to be required for IPFIX. The bottom line however is that flat data Peter Ludemann wrote: > My comments on Jeff Meyer's comments (hopefully this discussion won't get > indented too many levels by those who rebut me). > > - IPDR - much richer and more formal extensibility model, > leveraging industry standard XML-Schema. > Compactness as good as other candidates (after > negotiation of "templates"/"descriptors"), simple > set of operations. > > I would argue that a hierarchical format is a Bad Thing. History repeats > itself; and the XML people are finding out what old-timers already knew: > that there were good reasons why relational databases supplanted > hierarchical and network databases on the 1970s ... if anybody wishes me to > bore them with the details, just ask ... or read other people's opinions > such as http://www.dbazine.com/pascal4.html -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu Oct 3 14:02:03 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA28224 for ; Thu, 3 Oct 2002 14:02:02 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17xA7p-0000O5-00 for ipfix-list@mil.doit.wisc.edu; Thu, 03 Oct 2002 12:51:01 -0500 Received: from palrel10.hp.com ([156.153.255.245]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 17xA7n-0000Nu-00 for ipfix-eval@net.doit.wisc.edu; Thu, 03 Oct 2002 12:50:59 -0500 Received: from hpcuhe.cup.hp.com (hpcuhe.cup.hp.com [15.0.80.203]) by palrel10.hp.com (Postfix) with ESMTP id 8851EC00460 for ; Thu, 3 Oct 2002 10:50:58 -0700 (PDT) Received: from simail.cup.hp.com (imail@sim.cup.hp.com [15.16.123.26]) by hpcuhe.cup.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) with ESMTP id KAA22615 for ; Thu, 3 Oct 2002 10:50:53 -0700 (PDT) Received: from cup.hp.com ([15.16.124.96]) by simail.cup.hp.com (InterMail vM.5.01.03.01 201-253-122-118-101-20010319) with ESMTP id <20021003182158.HDAZ18196.simail.cup.hp.com@cup.hp.com> for ; Thu, 3 Oct 2002 11:21:58 -0700 Message-ID: <3D9C83FB.2060605@cup.hp.com> Date: Thu, 03 Oct 2002 10:52:59 -0700 From: Jeff Meyer User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:0.9.4.1) Gecko/20020508 Netscape6/6.2.3 X-Accept-Language: en-us MIME-Version: 1.0 Cc: ipfix-eval@net.doit.wisc.edu Subject: Re: [ipfix-eval] Discussion of Candidate Protocols References: <3D9C6B6C.6010803@cup.hp.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Hi, I didn't complete my closing remark before hitting send... The bottom line however is that flat data representations (First Normal Form) do have benefits in the ability to map directly to a relational DB table, and as such, can simplify the processing model for operating on large sets of events. When possible, mapping recorded usage information to a flat structure is going to increase the likelihood that off the shelf tools can operate on it (think Excel spreadsheet). -- Jeff Jeff Meyer wrote: > Hi, > > In this message I just wanted to respond the the > discussion around "hierarchical formats". Since > the comment indicates the author has not actually > read IPDR... > > The current version of IPDR does in fact require > that data be delivered in First Normal Form (just > like Relational DB's). > > IPDR specifies a SUBSET of the rich (overly so?) > vocabulary of XML Schema. > > So, if you look at the schema proposed for IPFIX > (in Appendix A of the Advocacy draft), you will see > that all of the values are laid out side by side, > no nested structures, no repeating of fields. > > > > > > > > > > > > > > > > > > > > > > > > > > Also to clarify. The proposed protocol uses Compact > IPDR format. The wire protocol IS NOT XML, rather XML > is simply used to model the data which is sent. So > in the example above, the record representing > a flow would have a descriptor identifying the type > of IPDRRecord being encoded followed by the > integer values one next to the other (i.e. 4 bytes/ > field). > > Now this modelling has the benefit that it ALSO > defines an equivalent XML encoding for the same > compact content. And in addition, since it is FNF, > the definition can also be used to define how this > information is to be stored in a DB. > > So you have one description language, based on > XML Schema, a wire protocol, a binary format, > an XML format and a DB mapper, in a single document. > > > Show me how that is done w/ the other proposed > drafts? > > > Regards, > > Jeff Meyer > > P.S. I am in the process of defining extensions to > IPDR to enable more structured data. (i.e. repeating > fields and fields which are not of a primitive type). > However this capability does not appear to be required > for IPFIX. > > The bottom line however is that flat data > > > > Peter Ludemann wrote: > >> My comments on Jeff Meyer's comments (hopefully this discussion won't get >> indented too many levels by those who rebut me). >> >> - IPDR - much richer and more formal extensibility model, >> leveraging industry standard XML-Schema. >> Compactness as good as other candidates (after >> negotiation of "templates"/"descriptors"), simple >> set of operations. >> >> I would argue that a hierarchical format is a Bad Thing. History repeats >> itself; and the XML people are finding out what old-timers already knew: >> that there were good reasons why relational databases supplanted >> hierarchical and network databases on the 1970s ... if anybody wishes >> me to >> bore them with the details, just ask ... or read other people's opinions >> such as http://www.dbazine.com/pascal4.html > > > > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message > body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu Oct 3 15:58:35 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA03081 for ; Thu, 3 Oct 2002 15:58:33 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17xBzY-0003JN-00 for ipfix-list@mil.doit.wisc.edu; Thu, 03 Oct 2002 14:50:36 -0500 Received: from [64.95.122.60] (helo=agile.yagosys.com) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 17xBzW-0003Io-00 for ipfix-eval@net.doit.wisc.edu; Thu, 03 Oct 2002 14:50:34 -0500 Received: (qmail 29351 invoked by uid 10041); 3 Oct 2002 19:50:03 -0000 Date: Thu, 3 Oct 2002 12:50:03 -0700 From: Michael MacFaden To: ipfix-eval@net.doit.wisc.edu Subject: Re: [ipfix-eval] Discussion of Candidate Protocols Message-ID: <20021003195003.GE6062@riverstonenet.com> References: <3D9C6B6C.6010803@cup.hp.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3D9C6B6C.6010803@cup.hp.com> User-Agent: Mutt/1.4i X-Operating-System: GNU/Linux 2.4.18 Precedence: bulk Sender: majordomo listserver On Thu, Oct 03, 2002 at 09:08:12AM -0700, Jeff Meyer wrote: > So you have one description language, based on >XML Schema, a wire protocol, a binary format, >an XML format and a DB mapper, in a single document. > > > Show me how that is done w/ the other proposed >drafts? My main issue with such a system is do we *require* such complexity if a simple numbering of fields will do just fine? I'd really like to see the IPIFIX protocol do one thing and do it very well. Delivering IP header information, as has been done for a very long time now with NetFlow and others, only really needs a clear definition of is being collected, how it is collected, and how to signal when the collector can't collect a paritcular piece of data ala RDBMS null indicator. The LFAP data spec clearly defines what is being collected and like NetFlow are purpose built targeted protocols. They not generic data movers. This makes them simple as can be, consistent and extremely interoperable. Regards, Mike MacFaden -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu Oct 3 17:42:02 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA06110 for ; Thu, 3 Oct 2002 17:42:02 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17xDZh-0005tu-00 for ipfix-list@mil.doit.wisc.edu; Thu, 03 Oct 2002 16:32:01 -0500 Received: from pool-129-44-37-17.ny325.east.verizon.net ([129.44.37.17] helo=newyork.qosient.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 17xDZe-0005ta-00 for ipfix-eval@net.doit.wisc.edu; Thu, 03 Oct 2002 16:31:58 -0500 Received: from SET (set [192.168.0.161]) by newyork.qosient.com (8.11.6/8.11.0) with ESMTP id g93LVvO23006; Thu, 3 Oct 2002 17:31:57 -0400 Reply-To: From: "Carter Bullard" To: "'Michael MacFaden'" , Subject: RE: [ipfix-eval] Discussion of Candidate Protocols Date: Thu, 3 Oct 2002 17:31:48 -0400 Organization: QoSient, LLC Message-ID: <5C8959A16A71B449AE793CF52FBBED6607E61C@ptah.newyork.qosient.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.3416 Importance: Normal In-Reply-To: <5C8959A16A71B449AE793CF52FBBED660BCEB9@ptah.newyork.qosient.com> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Gentle people, My existing flow record generators provide jitter and loss information in flow data records. How do I do that with an LFAP or netflow strategy? Carter Carter Bullard QoSient, LLC 300 E. 56th Street, Suite 18K New York, New York 10022 carter@qosient.com Phone +1 212 588-9133 Fax +1 212 588-9134 http://qosient.com -----Original Message----- From: majordomo listserver [mailto:majordomo@mil.doit.wisc.edu] On Behalf Of Michael MacFaden Sent: Thursday, October 03, 2002 3:50 PM To: ipfix-eval@net.doit.wisc.edu Subject: Re: [ipfix-eval] Discussion of Candidate Protocols On Thu, Oct 03, 2002 at 09:08:12AM -0700, Jeff Meyer wrote: > So you have one description language, based on >XML Schema, a wire protocol, a binary format, >an XML format and a DB mapper, in a single document. > > > Show me how that is done w/ the other proposed >drafts? My main issue with such a system is do we *require* such complexity if a simple numbering of fields will do just fine? I'd really like to see the IPIFIX protocol do one thing and do it very well. Delivering IP header information, as has been done for a very long time now with NetFlow and others, only really needs a clear definition of is being collected, how it is collected, and how to signal when the collector can't collect a paritcular piece of data ala RDBMS null indicator. The LFAP data spec clearly defines what is being collected and like NetFlow are purpose built targeted protocols. They not generic data movers. This makes them simple as can be, consistent and extremely interoperable. Regards, Mike MacFaden -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu Oct 3 18:19:10 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA06954 for ; Thu, 3 Oct 2002 18:19:10 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17xEBI-0006xK-00 for ipfix-list@mil.doit.wisc.edu; Thu, 03 Oct 2002 17:10:52 -0500 Received: from [64.95.122.60] (helo=agile.yagosys.com) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 17xEBG-0006wh-00 for ipfix-eval@net.doit.wisc.edu; Thu, 03 Oct 2002 17:10:50 -0500 Received: (qmail 18838 invoked by uid 10041); 3 Oct 2002 22:10:20 -0000 Date: Thu, 3 Oct 2002 15:10:20 -0700 From: Michael MacFaden To: ipfix-eval@net.doit.wisc.edu Subject: Re: [ipfix-eval] Discussion of Candidate Protocols Message-ID: <20021003221019.GJ6062@riverstonenet.com> References: <5C8959A16A71B449AE793CF52FBBED660BCEB9@ptah.newyork.qosient.com> <5C8959A16A71B449AE793CF52FBBED6607E61C@ptah.newyork.qosient.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5C8959A16A71B449AE793CF52FBBED6607E61C@ptah.newyork.qosient.com> User-Agent: Mutt/1.4i X-Operating-System: GNU/Linux 2.4.18 Precedence: bulk Sender: majordomo listserver On Thu, Oct 03, 2002 at 05:31:48PM -0400, Carter Bullard wrote: >My existing flow record generators provide jitter and loss >information in flow data records. How do I do that with an >LFAP or netflow strategy? For LFAP, build your collector as one normally would and then just follow section 2.45 of http://www.ietf.org/internet-drafts/draft-riverstone-lfap-data-01.txt My point was/is simply this. We should consider the feature/complexity tradeoff for each criteria considered in context of the problem being solved. My personal belief is that IPFIX does not stand for GDMP (Generic Data Mover Protocol). Regards, Mike MacFaden -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu Oct 3 20:15:32 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA08987 for ; Thu, 3 Oct 2002 20:15:32 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17xG0R-00022w-00 for ipfix-list@mil.doit.wisc.edu; Thu, 03 Oct 2002 19:07:47 -0500 Received: from pool-129-44-37-17.ny325.east.verizon.net ([129.44.37.17] helo=newyork.qosient.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 17xG0P-00022q-00 for ipfix-eval@net.doit.wisc.edu; Thu, 03 Oct 2002 19:07:45 -0500 Received: from SET (set [192.168.0.161]) by newyork.qosient.com (8.11.6/8.11.0) with ESMTP id g9407iO23166; Thu, 3 Oct 2002 20:07:44 -0400 Reply-To: From: "Carter Bullard" To: "'Michael MacFaden'" , Subject: RE: [ipfix-eval] Discussion of Candidate Protocols Date: Thu, 3 Oct 2002 20:07:35 -0400 Organization: QoSient, LLC Message-ID: <5C8959A16A71B449AE793CF52FBBED6607A45C@ptah.newyork.qosient.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.3416 Importance: Normal In-Reply-To: <5C8959A16A71B449AE793CF52FBBED660BCF51@ptah.newyork.qosient.com> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Hey Mike, Thanks, but I want to note that netflow cannot support much in the way of extensions, so simply describing existing practice and defining a standard to support those observations is not going to do the job for my customers. In writing this response I found myself criticizing LFAP quite a bit, and I didn't really want to do that, but I'm going to leave some of my issues in hopes that it will help the process. I hope that some do find it useful, and I welcome any comment. One question. None of the candidates are perfect. If we adopt one, what will be the process that we go through to 'correct' any issues? I'm sure it has been described, but could someone paraphrase? Thanks, Carter Comments on LFAP as IPFIX candidate. I don't see that LFAP is a solution to the flow record transport problems that I worry about, which is the efficient, reliable and secure transport of any vendors flow records, and I'm particularly concerned with at least one of the basic architectural positions that LFAP takes, the protocol complexity and its Information Model. The biggest issue with the architecture is timestamps. My understanding reading the lfap-eval, is that timestamps relate to the flow cache rather than the wire-line timestamps of the packet stream that is being monitored. If true, I believe that this is huge problem, as it will introduce a lot of timestamp variance that cannot be adjusted for, rendering the data very limited in its usefulness. The IPPM framework devotes a large amount of time and effort describing how a monitor should deal with timestamps of packets, and an IPFIX monitor should comply with the IPPM framework on this. Anything less would be networking 'malpractice', if there ever could be such a thing. I think the protocol has too many message types. Don't need a separate turn for VR/VRA, and AR/ARA you should be able to present the VR/CR/AR information elements in a single message and then get a single response. The concept that you can support the division of FAR and FUN messages generate great potential problems. If I connect at some arbitrary point and don't receive the FAR message, how do I process FUN messages, which don't have flow descriptors? With regard to the Information Model, I sense that I would put my entire record in a single vendor specific IE and leave it at that. While a drastic statement, its not too far from the truth. Just to innumerate a few problems I have with LFAP's basic data definitions, and this is not an exhaustive set, of course. Best time granularity of 10's of milliseconds (1/100th of a sec) is not good, uSec minimum, nanoSec preferable. The requirement that the FAS/CCE's must be globally unique, and then you start adding ones to them on roll over (how do you ensure that the FAS is still globally unique?) is a big issue (locally unique?) The concepts of the Flow Qualifier checkpoint and priority are completely proprietary, they should be vendor specific OIDs? 64 bit counters for packets and bytes in every record, when over 90% of all flows have less than 256 packets and 8K of bytes in the entire flow? We should have 1, 2, 4 and 8 byte counter IEs as needed, don't you think? The IpId field is 16 bits but the IE to report it is 64 bits long. Why not a 32 bit field for those special values that are 8-16 bits long? Why not have a composite IE, like an IP header attribute IE, which could have the IpId, Tos, TTL, and option indicators all in a single IE? Why not an IE for the classic 5-tuple flow? Minor points, but important points for my products. Carter -----Original Message----- From: majordomo listserver [mailto:majordomo@mil.doit.wisc.edu] On Behalf Of Michael MacFaden Sent: Thursday, October 03, 2002 6:10 PM To: ipfix-eval@net.doit.wisc.edu Subject: Re: [ipfix-eval] Discussion of Candidate Protocols On Thu, Oct 03, 2002 at 05:31:48PM -0400, Carter Bullard wrote: >My existing flow record generators provide jitter and loss information >in flow data records. How do I do that with an LFAP or netflow >strategy? For LFAP, build your collector as one normally would and then just follow section 2.45 of http://www.ietf.org/internet-drafts/draft-riverstone-lfap-data-01.txt My point was/is simply this. We should consider the feature/complexity tradeoff for each criteria considered in context of the problem being solved. My personal belief is that IPFIX does not stand for GDMP (Generic Data Mover Protocol). Regards, Mike MacFaden -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri Oct 4 05:30:56 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id FAA27006 for ; Fri, 4 Oct 2002 05:30:55 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17xOZy-0007ch-00 for ipfix-list@mil.doit.wisc.edu; Fri, 04 Oct 2002 04:17:02 -0500 Received: from mailhub.fokus.gmd.de ([193.174.154.14]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 17xOZw-0007cH-00 for ipfix-eval@net.doit.wisc.edu; Fri, 04 Oct 2002 04:17:00 -0500 Received: from fokus.gmd.de (dhcp229 [195.37.78.229]) by mailhub.fokus.gmd.de (8.11.6/8.11.6) with ESMTP id g9499Oe04494; Fri, 4 Oct 2002 11:09:24 +0200 (MEST) Message-ID: <3D9D5A10.2030906@fokus.gmd.de> Date: Fri, 04 Oct 2002 11:06:24 +0200 From: Sebastian Zander Organization: Fraunhofer FOKUS User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; de-DE; rv:0.9.4.1) Gecko/20020508 Netscape6/6.2.3 X-Accept-Language: de-DE MIME-Version: 1.0 To: Jeff Meyer CC: Peter Ludemann , ipfix-eval@net.doit.wisc.edu Subject: Re: [ipfix-eval] Discussion of Candidate Protocols References: <3D9C6B6C.6010803@cup.hp.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Jeff Meyer wrote: [...] > > So you have one description language, based on > XML Schema, a wire protocol, a binary format, > an XML format and a DB mapper, in a single document. > > > Show me how that is done w/ the other proposed > drafts? Jeff, its very nice to have these things but I don't think its a good idea to have them in one document. IMHO the main focus here is on the protocol. Working on all the things at once will only slow down the standardization process. I am also wondering if the IETF would ever standardize things like binary formats. Probably not. Furthermore I am not convinced that it is a good idea to standardize things like DB mapper or binary formats. E.g. there exist a number of different formats and some people may want to continue using them. But informational RFCs could cover all these things. To summarize my point: the evaluation should focus on the protocol and its capabilities and not on things storage formats etc. BTW Diameter has a proposed XML definition as well (draft-frascone-aaa-xml-dictionary-00.html). Another proposal was to use SMIng (draft-schoenw-sming-dia-00.txt). Cheers, Sebastian -- Sebastian Zander E-mail: zander@fokus.fhg.de FhI FOKUS / Global Networking (GloNe) Tel: +49-30-3463-7287 Kaiserin-Augusta-Allee 31 Fax: +49-30-3463-8287 D-10589 Berlin, Germany www.fokus.fhg.de/usr/sebastian.zander -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri Oct 4 12:30:14 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA12107 for ; Fri, 4 Oct 2002 12:30:13 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17xVB2-0004E1-00 for ipfix-list@mil.doit.wisc.edu; Fri, 04 Oct 2002 11:19:44 -0500 Received: from [64.95.122.60] (helo=RS-SC-EXC4.rs.riverstonenet.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 17xVB0-0004DL-00 for ipfix-eval@net.doit.wisc.edu; Fri, 04 Oct 2002 11:19:42 -0500 Received: from riverstonenet.com ([134.141.180.93]) by RS-SC-EXC4.rs.riverstonenet.com with Microsoft SMTPSVC(5.0.2195.4905); Fri, 4 Oct 2002 09:19:07 -0700 Message-ID: <3D9DBF0A.1DCD90FE@riverstonenet.com> Date: Fri, 04 Oct 2002 12:17:14 -0400 From: calato@riverstonenet.com X-Mailer: Mozilla 4.75C-CCK-MCD NSCPCD475 [en] (X11; U; SunOS 5.6 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: carter@qosient.com CC: "'Michael MacFaden'" , ipfix-eval@net.doit.wisc.edu Subject: Re: [ipfix-eval] Discussion of Candidate Protocols References: <5C8959A16A71B449AE793CF52FBBED6607A45C@ptah.newyork.qosient.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 04 Oct 2002 16:19:07.0824 (UTC) FILETIME=[C3D60B00:01C26BC1] Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Carter Bullard wrote: > > Hey Mike, > Thanks, but I want to note that netflow cannot support > much in the way of extensions, so simply describing existing > practice and defining a standard to support those observations > is not going to do the job for my customers. > > In writing this response I found myself criticizing LFAP > quite a bit, and I didn't really want to do that, but I'm going to > leave some of my issues in hopes that it will help the process. > I hope that some do find it useful, and I welcome any comment. > > One question. None of the candidates are perfect. If we > adopt one, what will be the process that we go through to > 'correct' any issues? I'm sure it has been described, but > could someone paraphrase? I assumed the adopted protocol would be a starting point and need modification to better suit IPFIX needs. Just my 2 cents. More below... > > Thanks, > > Carter > > Comments on LFAP as IPFIX candidate. > > I don't see that LFAP is a solution to the flow record transport > problems that I worry about, which is the efficient, reliable > and secure transport of any vendors flow records, and I'm > particularly concerned with at least one of the basic architectural > positions that LFAP takes, the protocol complexity and > its Information Model. It is a bit difficult to address these general comments so I'll stick to addressing specific items you mentioned below. If you care to elaborate on any of the above I'll do my best to clarify. > > The biggest issue with the architecture is timestamps. My > understanding reading the lfap-eval, is that timestamps > relate to the flow cache rather than the wire-line timestamps of > the packet stream that is being monitored. Not true at all. In the eval I mentioned that timestamps need some clarification and here is why. There are 2 types. 1) cache timeout 2) wire-line timestamps. My main point is that we (IPFIX) need to support both types and we need to know which which one is being used. But both can be handled via LFAP. > If true, I believe > that this is huge problem, as it will introduce a lot of > timestamp variance that cannot be adjusted for, rendering > the data very limited in its usefulness. The IPPM framework > devotes a large amount of time and effort describing how a monitor > should deal with timestamps of packets, and an IPFIX monitor > should comply with the IPPM framework on this. Anything less > would be networking 'malpractice', if there ever could be such > a thing. > > I think the protocol has too many message types. Don't need > a separate turn for VR/VRA, and AR/ARA you should be able to > present the VR/CR/AR information elements in a single message and > then get a single response. When you are negotiating version number you cannot do anything until the version is agreed upon. Otherwise you cannot reliably decode the message formats. Hence VR/VRA first. There are a couple of benefits of the CR message. First it provides a list of other servers known to the device. **IF** the servers attempt to do any load balancing the device can now be redirected to another known server early on. The CR is also here for security purposes. You don't want to allow someone to do a DOS attack on your sever by connecting and blasting it with messages. The protocol message sequence was designed to help with that problem. The AR/ARA then allows an exchange of information and was designed for flexibility and extensibility. The amount of time it takes to send the messages is a non-issue since it is only done once at session start and is infinitesimal when compared to the rest of the session. So there are good reasons for the message exchange. It is a balance between simplicity, extensibility and security. Big topics handled by 3 little messages. > > The concept that you can support the division of FAR and FUN > messages generate great potential problems. If I connect at some > arbitrary point and don't receive the FAR message, how do I > process FUN messages, which don't have flow descriptors? You are absolutely correct. This is probably the biggest difference for LFAP. As I mentioned in the eval, I think the LFAP spec should be modified to allow measured attributes (bytes, packets, etc...) in the FAR. This is a trivial change and provides the ability to send complete flows in one message. This is the easy problem to solve. The problem with the one shot messages is storage. The device now needs to store all the attributes until it is time to send the message. When you have millions of flows this can be a huge problem! Or if the device is small it can be a large problem. And as the number of attributes to report grows it makes the problem worse. Solving this problem is much more difficult than allowing LFAP to send one shot messages. LFAP has solved this problem by splitting up the attributes and allowing them to be reported independently. The answer to your specific question "you" don't connect to the device, the device connects to you. But that just moves the problem, what happens if the device looses the connection to one server and connects to another. You could resend all the FAR's but that would create a lot of extra burden on the device. So we chose to let the servers handle the failover case. FUN records with no descriptior are sent to a single sever. Similarly, FAR records which don't have a closing FUN are sent to the single server. (FYI - long lived flows, may have several FUNs.) In this way only failover flows need to get resolved and that should be a fraction of the total number of flows. > > With regard to the Information Model, I sense that I would > put my entire record in a single vendor specific IE and leave it > at that. While a drastic statement, its not too far from the > truth. I understand the statement but I don't understand how it fits in the IPFIX context. IPFIX is trying to standardize 2 things... 1) The data that is exported 2) The way in which data is exported. If you have to put all your data in the vendor-specific field then IPFIX only accomplished 1 of those 2 things. LFAP's data model provides a better starting point than most but is by no means perfect. If adopted, I'm sure it will need to be modified. > > Just to innumerate a few problems I have with LFAP's basic > data definitions, and this is not an exhaustive set, of course. > > Best time granularity of 10's of milliseconds (1/100th of a sec) > is not good, uSec minimum, nanoSec preferable. No problem. 1/100th of a second is good enough for cache timeout. The definition for wire-line timestamps can be nanoSec. > > The requirement that the FAS/CCE's must be globally unique, and then > you start adding ones to them on roll over (how do you ensure that > the FAS is still globally unique?) is a big issue (locally unique?) I'm not sure what you mean by a unique FAS/CCE. What part are you referencing? > > The concepts of the Flow Qualifier checkpoint and priority are > completely proprietary, they should be vendor specific OIDs? You may be right on that. > > 64 bit counters for packets and bytes in every record, when over > 90% of all flows have less than 256 packets and 8K of bytes in the > entire flow? We should have 1, 2, 4 and 8 byte counter IEs as > needed, don't you think? I would agree to that. > > The IpId field is 16 bits but the IE to report it is 64 bits long. > Why not a 32 bit field for those special values that are 8-16 bits > long? Why not have a composite IE, like an IP header attribute > IE, which could have the IpId, Tos, TTL, and option indicators all > in a single IE? Why not an IE for the classic 5-tuple flow? > > Minor points, but important points for my products. All good points. As I said LFAP has a good data model starting point but would need the input of people such as yourself. Paul > > Carter > > > -----Original Message----- > From: majordomo listserver [mailto:majordomo@mil.doit.wisc.edu] On > Behalf Of Michael MacFaden > Sent: Thursday, October 03, 2002 6:10 PM > To: ipfix-eval@net.doit.wisc.edu > Subject: Re: [ipfix-eval] Discussion of Candidate Protocols > > On Thu, Oct 03, 2002 at 05:31:48PM -0400, Carter Bullard wrote: > >My existing flow record generators provide jitter and loss information > >in flow data records. How do I do that with an LFAP or netflow > >strategy? > > For LFAP, build your collector as one normally would and > then just follow section 2.45 of > http://www.ietf.org/internet-drafts/draft-riverstone-lfap-data-01.txt > > My point was/is simply this. > We should consider the feature/complexity tradeoff > for each criteria considered in context of the problem > being solved. > > My personal belief is that IPFIX does not stand for > GDMP (Generic Data Mover Protocol). > > Regards, > Mike MacFaden > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message > body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe > ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri Oct 4 12:57:01 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA12877 for ; Fri, 4 Oct 2002 12:57:01 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17xVeH-00050P-00 for ipfix-list@mil.doit.wisc.edu; Fri, 04 Oct 2002 11:49:57 -0500 Received: from mailhub.xacct.com ([204.253.100.25] helo=xacct.com) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 17xVeE-00050H-00 for ipfix-eval@net.doit.wisc.edu; Fri, 04 Oct 2002 11:49:55 -0500 Received: (qmail 31378 invoked from network); 4 Oct 2002 16:49:52 -0000 Received: from usmail.xacct.com (204.253.100.12) by mailhub.xacct.com with SMTP; 4 Oct 2002 16:49:52 -0000 Received: from peter (inside.us.xacct.com [204.253.100.102]) by usmail.xacct.com (8.11.2/8.11.2) with SMTP id g94Gqus05246; Fri, 4 Oct 2002 09:52:56 -0700 From: "Peter Ludemann" To: , "Sebastian Zander" Cc: "Jeff Meyer" , Subject: RE: [ipfix-eval] Discussion of Candidate Protocols Date: Fri, 4 Oct 2002 09:49:45 -0700 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 In-Reply-To: <3D9AEDC2.14852887@riverstonenet.com> Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit calato@riverstonenet.com wrote Wednesday, October 02, 2002 6:00 AM: > Unfortunately the advocates are likely the ones in the best > position to comment. So we've taken 5 strong voices out of the > discussion. At some point we'll we'll need to open the > flood gates. Maybe after the evals come out. I too have restrained, but no more! Diogenes-like, I shall carry my lantern, searching for Truth. Alas, my Silicon Valley bathtub carries a million dollar mortgage, so I must work 36 hours a day at XACCT to keep the bankers away, but that won't bias my opinions. I think that my biases should be obvious (I come from a world where protocols are a means to an end and the end is usually a big database) but if anybody wants a more extensive list of my background and biases, I'll be happy to provide them. - peter ludemann principal engineer xacct technologies -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri Oct 4 13:02:35 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA13184 for ; Fri, 4 Oct 2002 13:02:35 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17xVk3-00059R-00 for ipfix-list@mil.doit.wisc.edu; Fri, 04 Oct 2002 11:55:55 -0500 Received: from palrel13.hp.com ([156.153.255.238]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 17xVk1-00059L-00 for ipfix-eval@net.doit.wisc.edu; Fri, 04 Oct 2002 11:55:53 -0500 Received: from hpcuhe.cup.hp.com (hpcuhe.cup.hp.com [15.0.80.203]) by palrel13.hp.com (Postfix) with ESMTP id 5CEA2400224; Fri, 4 Oct 2002 09:55:50 -0700 (PDT) Received: from simail.cup.hp.com (imail@sim.cup.hp.com [15.16.123.26]) by hpcuhe.cup.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) with ESMTP id JAA15577; Fri, 4 Oct 2002 09:55:45 -0700 (PDT) Received: from cup.hp.com ([15.16.124.96]) by simail.cup.hp.com (InterMail vM.5.01.03.01 201-253-122-118-101-20010319) with ESMTP id <20021004172655.HEKH18196.simail.cup.hp.com@cup.hp.com>; Fri, 4 Oct 2002 10:26:55 -0700 Message-ID: <3D9DC88F.9060507@cup.hp.com> Date: Fri, 04 Oct 2002 09:57:51 -0700 From: Jeff Meyer User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:0.9.4.1) Gecko/20020508 Netscape6/6.2.3 X-Accept-Language: en-us MIME-Version: 1.0 To: Sebastian Zander Cc: Peter Ludemann , ipfix-eval@net.doit.wisc.edu Subject: Re: [ipfix-eval] Discussion of Candidate Protocols References: <3D9C6B6C.6010803@cup.hp.com> <3D9D5A10.2030906@fokus.gmd.de> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Hi, There seems to be some confusion over the separation between a formal, machine readable description of the data model (as IPDR defines), and the requirements put on an implementation (especially in an IPFIX Middlebox). Presumably folks are familiar with SNMP and the separation between what a MIB defines and how the protocol operates. Basically IPDR's use of XML schema is producing a MIB for sets of accounting information. Why not use MIB's? Well, SNMP is really geared towards providing views/access into a current system state. Historical information tends to be global or other coarse grain counters. Fine grain counters in SNMP are hindered by SNMP's OID indexing model (i.e. you end up with something like RMON, a clever use of SNMP, but not much fun to code to). IP Flow information is just a specifically defined set of metrics which are being collected based on certain observed network traffic. It just so happens that this can produce a huge amount of info, which often cannot be stored on the device itself for any period of time. For this reason, using SNMP for gathering flow info is probably a bad idea, and I'm glad to see no one signed up as the SNMPv3 advocate ;-) But the essential concept of MIBs, which SMIng is also looking at revising is important. The extensibility model which MIBs provided for SNMP is what has led to its large scale adoption for many problem domains. It just so happens that the profile of flow exporting is not appropriate for the SNMP domain. However, the statement that IP flow is somehow unique in its data requirements and as such a more generalized "data mover" is somehow problematic, is just plain wrong. As a mediation product vendor I see lots of devices with lots of vendor specific protocols. And there are several which have the same protocol requirements as IPFIX. Just different data models. Consider a middlebox which looks not only at flows, but also at protocols above layer 4. Would it not make sense for the additional information produced by this to use the same basic protocol as IPFIX? IPFIX does identify extensibility as a requirement, is there some bounded set of extensions which are envisioned? If not, then a trivial numeric id scheme is a pretty lousy namespace to work from. Diameter's vendorId/ avaCode model is not much better, but at least has some partitioning. The only possible unique aspect I could picture is based on observing NFv2,5,7. In all of these versions, all data items were fixed width (typically 32-bit integers). So... If IPFIX wants to say that only fixed with data values may be sent via the IPFIX protocol, then I'd say, maybe you could optimize beyond IPDR for that. A minimal implementation of IPDR on a middlebox would NOT need to have any XML knowledge whatsoever. As a producer, the system could merely hardcode the information model produced (or make it configurable). The encoding itself follows XDR format, which has worked alright for NFS and RPC, and is the precursor to the encoding for DCE and CORBA encoding. It is a very simple model. Read RFC 1832. An implementation for what IPDR requires is pretty trivial. IPDR members have access to opensource for this in both Java and C. And as Stas pointed out in an earlier memo there are > 10 implementations out there. The other benefits from IPDR which I cited, such as XML mapping, DB, binary file format, etc. are just that, additional benefits. They ARE NOT REQUIRED to be implemented. Regards, Jeff Meyer Sebastian Zander wrote: > Jeff Meyer wrote: > > [...] > >> >> So you have one description language, based on >> XML Schema, a wire protocol, a binary format, >> an XML format and a DB mapper, in a single document. >> >> >> Show me how that is done w/ the other proposed >> drafts? > > > Jeff, > > > its very nice to have these things but I don't think its > a good idea to have them in one document. IMHO the main > focus here is on the protocol. Working on all the things > at once will only slow down the standardization process. > I am also wondering if the IETF would ever standardize > things like binary formats. Probably not. > > Furthermore I am not convinced that it is a good idea to > standardize things like DB mapper or binary formats. E.g. > there exist a number of different formats and some people > may want to continue using them. But informational RFCs > could cover all these things. > > To summarize my point: the evaluation should focus on the > protocol and its capabilities and not on things storage > formats etc. > > BTW Diameter has a proposed XML definition as well > (draft-frascone-aaa-xml-dictionary-00.html). Another proposal > was to use SMIng (draft-schoenw-sming-dia-00.txt). > > Cheers, > > Sebastian > > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri Oct 4 13:17:47 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA13822 for ; Fri, 4 Oct 2002 13:17:46 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17xVlM-0005C6-00 for ipfix-list@mil.doit.wisc.edu; Fri, 04 Oct 2002 11:57:16 -0500 Received: from [64.95.122.60] (helo=RS-SC-EXC4.rs.riverstonenet.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 17xVlK-0005Al-00 for ipfix-eval@net.doit.wisc.edu; Fri, 04 Oct 2002 11:57:14 -0500 Received: from riverstonenet.com ([134.141.180.93]) by RS-SC-EXC4.rs.riverstonenet.com with Microsoft SMTPSVC(5.0.2195.4905); Fri, 4 Oct 2002 09:56:40 -0700 Message-ID: <3D9DC7D7.D5F3320D@riverstonenet.com> Date: Fri, 04 Oct 2002 12:54:47 -0400 From: calato@riverstonenet.com X-Mailer: Mozilla 4.75C-CCK-MCD NSCPCD475 [en] (X11; U; SunOS 5.6 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: Peter Ludemann CC: Sebastian Zander , Jeff Meyer , ipfix-eval@net.doit.wisc.edu Subject: Re: [ipfix-eval] Discussion of Candidate Protocols References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 04 Oct 2002 16:56:41.0087 (UTC) FILETIME=[02E2B0F0:01C26BC7] Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Peter Ludemann wrote: > > calato@riverstonenet.com wrote Wednesday, October 02, 2002 6:00 AM: > > > Unfortunately the advocates are likely the ones in the best > > position to comment. So we've taken 5 strong voices out of the > > discussion. At some point we'll we'll need to open the > > flood gates. Maybe after the evals come out. > > I too have restrained, but no more! Diogenes-like, I shall > carry my lantern, searching for Truth. Alas, my Silicon Valley > bathtub carries a million dollar mortgage, so I must work 36 hours > a day at XACCT to keep the bankers away, but that won't bias > my opinions. > > I think that my biases should be obvious (I come from a world > where protocols are a means to an end and the end is usually > a big database) but if anybody wants a more extensive list of > my background and biases, I'll be happy to provide them. I think the mortgage information will suffice :-) > > - peter ludemann > principal engineer > xacct technologies -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri Oct 4 14:03:50 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA15224 for ; Fri, 4 Oct 2002 14:03:49 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17xWh7-0006ZV-00 for ipfix-list@mil.doit.wisc.edu; Fri, 04 Oct 2002 12:56:57 -0500 Received: from mailhub.lawrence.edu ([143.44.65.14]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 17xWh6-0006ZO-00 for ipfix-eval@net.doit.wisc.edu; Fri, 04 Oct 2002 12:56:56 -0500 Received: from lawrence.edu ([143.44.97.19]) by lawrence.edu (PMDF V6.1-1 #30572) with ESMTPA id <0H3G01E6HXY11Y@lawrence.edu> for ipfix-eval@net.doit.wisc.edu; Fri, 04 Oct 2002 12:58:50 -0500 (CDT) Date: Fri, 04 Oct 2002 12:56:55 -0500 From: Robert Lowe Subject: Re: [ipfix-eval] Discussion of Candidate Protocols To: Jeff Meyer Cc: Sebastian Zander , Peter Ludemann , ipfix-eval@net.doit.wisc.edu Message-id: <3D9DD667.110220E2@lawrence.edu> Organization: Lawrence University MIME-version: 1.0 X-Mailer: Mozilla 4.77 [en] (Windows NT 5.0; U) Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7bit X-Accept-Language: en References: <3D9C6B6C.6010803@cup.hp.com> <3D9D5A10.2030906@fokus.gmd.de> <3D9DC88F.9060507@cup.hp.com> Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Jeff Meyer wrote: Attention: lurker factor in play... > An implementation > for what IPDR requires is pretty trivial. IPDR > members have access to opensource for this in > both Java and C. The common definition of opensource includes descriptives such as "publicly-available" and "no-strings-attached". You seem like a pretty good evangelist for IPDR, but once someone starts passing the collection plate, using the word "opensource" only sullies the good name of the sacred cow of 21st century computing with the marketeer's comparison to a members-only reference implementation. Some words don't have increasing degrees of comparison. You can be dead, but you can't be 'deader', and in my book, open is open, not open for a fee. Grammer lesson and rant off... -Robert -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri Oct 4 15:15:59 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA17670 for ; Fri, 4 Oct 2002 15:15:59 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17xXjZ-0000kf-00 for ipfix-list@mil.doit.wisc.edu; Fri, 04 Oct 2002 14:03:33 -0500 Received: from ctron-dnm.enterasys.com ([12.25.1.120] ident=firewall-user) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 17xXjX-0000kW-00 for ipfix-eval@net.doit.wisc.edu; Fri, 04 Oct 2002 14:03:31 -0500 Received: (from uucp@localhost) by ctron-dnm.enterasys.com (8.8.7/8.8.7) id PAA24045 for ; Fri, 4 Oct 2002 15:15:11 -0400 (EDT) Received: from unknown(134.141.77.96) by ctron-dnm.enterasys.com via smap (4.1) id xma024024; Fri, 4 Oct 02 15:14:52 -0400 Received: by cnc-exc1.enterasys.com with Internet Mail Service (5.5.2653.19) id ; Fri, 4 Oct 2002 14:57:53 -0400 Message-ID: <6D745637A7E0F94DA070743C55CDA9BA256D5A@NHROCMBX1.ets.enterasys.com> From: "Harrington, David" To: ipfix-eval@net.doit.wisc.edu Subject: RE: [ipfix-eval] Discussion of Candidate Protocols Date: Fri, 4 Oct 2002 15:03:15 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C26BD8.62564A23" Precedence: bulk Sender: majordomo listserver This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C26BD8.62564A23 Content-Type: text/plain A question: Are there intellectual property claims against IPDR? There probably are for all the protocols proposed; I just don't remember seeing anything mentioned and want to make sure all the cards are on the table. dbh > -----Original Message----- > From: Robert Lowe [mailto:robert.h.lowe@lawrence.edu] > Sent: Friday, October 04, 2002 1:57 PM > To: Jeff Meyer > Cc: Sebastian Zander; Peter Ludemann; ipfix-eval@net.doit.wisc.edu > Subject: Re: [ipfix-eval] Discussion of Candidate Protocols > > > Jeff Meyer wrote: > > Attention: lurker factor in play... > > > An implementation > > for what IPDR requires is pretty trivial. IPDR > > members have access to opensource for this in > > both Java and C. > > The common definition of opensource includes descriptives such > as "publicly-available" and "no-strings-attached". You seem > like a pretty good evangelist for IPDR, but once someone starts > passing the collection plate, using the word "opensource" only > sullies the good name of the sacred cow of 21st century computing > with the marketeer's comparison to a members-only reference > implementation. Some words don't have increasing degrees of > comparison. You can be dead, but you can't be 'deader', and in > my book, open is open, not open for a fee. Grammer lesson and > rant off... > > -Robert > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" > in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ > ------_=_NextPart_001_01C26BD8.62564A23 Content-Type: text/html Content-Transfer-Encoding: quoted-printable RE: [ipfix-eval] Discussion of Candidate Protocols

A question: Are there intellectual property claims = against IPDR? There probably are for all the protocols proposed; I just = don't remember seeing anything mentioned and want to make sure all the = cards are on the table.

dbh

> -----Original Message-----
> From: Robert Lowe [mailto:robert.h.lowe@lawrence= .edu]
> Sent: Friday, October 04, 2002 1:57 PM
> To: Jeff Meyer
> Cc: Sebastian Zander; Peter Ludemann; = ipfix-eval@net.doit.wisc.edu
> Subject: Re: [ipfix-eval] Discussion of = Candidate Protocols
>
>
> Jeff Meyer wrote:
>
> Attention: lurker factor in play...
>
> > An implementation
> > for what IPDR requires is pretty = trivial. IPDR
> > members have access to opensource for this = in
> > both Java and C.
>
> The common definition of opensource includes = descriptives such
> as "publicly-available" and = "no-strings-attached". You seem
> like a pretty good evangelist for IPDR, but = once someone starts
> passing the collection plate, using the word = "opensource" only
> sullies the good name of the sacred cow of 21st = century computing
> with the marketeer's comparison to a = members-only reference
> implementation. Some words don't have = increasing degrees of
> comparison. You can be dead, but you = can't be 'deader', and in
> my book, open is open, not open for a = fee. Grammer lesson and
> rant off...
>
> -Robert
>
> --
> Help = mailto:majordomo@net.doit.wi= sc.edu and say "help"
> in message body
> Unsubscribe mailto:majordomo@net.doit.wi= sc.edu and say
> "unsubscribe ipfix" in message = body
> Archive http://ipfix.doit.wisc.edu/archive/
>

------_=_NextPart_001_01C26BD8.62564A23-- -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri Oct 4 15:51:21 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA19006 for ; Fri, 4 Oct 2002 15:51:21 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17xYEi-0001fj-00 for ipfix-list@mil.doit.wisc.edu; Fri, 04 Oct 2002 14:35:44 -0500 Received: from [64.95.122.60] (helo=RS-SC-EXC4.rs.riverstonenet.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 17xYEg-0001f5-00 for ipfix-eval@net.doit.wisc.edu; Fri, 04 Oct 2002 14:35:42 -0500 Received: from riverstonenet.com ([134.141.180.93]) by RS-SC-EXC4.rs.riverstonenet.com with Microsoft SMTPSVC(5.0.2195.4905); Fri, 4 Oct 2002 12:35:08 -0700 Message-ID: <3D9DECFC.87159370@riverstonenet.com> Date: Fri, 04 Oct 2002 15:33:16 -0400 From: calato@riverstonenet.com X-Mailer: Mozilla 4.75C-CCK-MCD NSCPCD475 [en] (X11; U; SunOS 5.6 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: "Harrington, David" CC: ipfix-eval@net.doit.wisc.edu Subject: Re: [ipfix-eval] Discussion of Candidate Protocols References: <6D745637A7E0F94DA070743C55CDA9BA256D5A@NHROCMBX1.ets.enterasys.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 04 Oct 2002 19:35:09.0428 (UTC) FILETIME=[264C6340:01C26BDD] Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit > "Harrington, David" wrote: > > A question: Are there IPDR? There > probably are for all the protocols proposed; I just don't remember > seeing anything mentioned and want to make sure all the cards are on > the table. > Just FYI, there are no intellectual property claims against LFAP. Paul > dbh > > > -----Original Message----- > > From: Robert Lowe [mailto:robert.h.lowe@lawrence.edu] > > Sent: Friday, October 04, 2002 1:57 PM > > To: Jeff Meyer > > Cc: Sebastian Zander; Peter Ludemann; ipfix-eval@net.doit.wisc.edu > > Subject: Re: [ipfix-eval] Discussion of Candidate Protocols > > > > > > Jeff Meyer wrote: > > > > Attention: lurker factor in play... > > > > > An implementation > > > for what IPDR requires is pretty trivial. IPDR > > > members have access to opensource for this in > > > both Java and C. > > > > The common definition of opensource includes descriptives such > > as "publicly-available" and "no-strings-attached". You seem > > like a pretty good evangelist for IPDR, but once someone starts > > passing the collection plate, using the word "opensource" only > > sullies the good name of the sacred cow of 21st century computing > > with the marketeer's comparison to a members-only reference > > implementation. Some words don't have increasing degrees of > > comparison. You can be dead, but you can't be 'deader', and in > > my book, open is open, not open for a fee. Grammer lesson and > > rant off... > > > > -Robert > > > > -- > > Help mailto:majordomo@net.doit.wisc.edu and say "help" > > in message body > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe ipfix" in message body > > Archive http://ipfix.doit.wisc.edu/archive/ > > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri Oct 4 16:01:10 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA19429 for ; Fri, 4 Oct 2002 16:01:10 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17xYVi-00022Z-00 for ipfix-list@mil.doit.wisc.edu; Fri, 04 Oct 2002 14:53:18 -0500 Received: from palrel10.hp.com ([156.153.255.245]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 17xYVg-00022S-00 for ipfix-eval@net.doit.wisc.edu; Fri, 04 Oct 2002 14:53:16 -0500 Received: from hpcuhe.cup.hp.com (hpcuhe.cup.hp.com [15.0.80.203]) by palrel10.hp.com (Postfix) with ESMTP id C152DC00164; Fri, 4 Oct 2002 12:53:13 -0700 (PDT) Received: from simail.cup.hp.com (imail@sim.cup.hp.com [15.16.123.26]) by hpcuhe.cup.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) with ESMTP id MAA25960; Fri, 4 Oct 2002 12:53:08 -0700 (PDT) Received: from cup.hp.com ([15.16.124.96]) by simail.cup.hp.com (InterMail vM.5.01.03.01 201-253-122-118-101-20010319) with ESMTP id <20021004202419.HERQ18196.simail.cup.hp.com@cup.hp.com>; Fri, 4 Oct 2002 13:24:19 -0700 Message-ID: <3D9DF222.1000703@cup.hp.com> Date: Fri, 04 Oct 2002 12:55:14 -0700 From: Jeff Meyer User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:0.9.4.1) Gecko/20020508 Netscape6/6.2.3 X-Accept-Language: en-us MIME-Version: 1.0 To: Robert Lowe Cc: Sebastian Zander , Peter Ludemann , ipfix-eval@net.doit.wisc.edu, protocol@ipdr.metratech.com Subject: Re: [ipfix-eval] Discussion of Candidate Protocols References: <3D9C6B6C.6010803@cup.hp.com> <3D9D5A10.2030906@fokus.gmd.de> <3D9DC88F.9060507@cup.hp.com> <3D9DD667.110220E2@lawrence.edu> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Robert, Good point. This is currently a reference implementation. Although I've advocated to move it to complete opensource, e.g. SourceForge, this has not happened, YET. Regards, Jeff Meyer Robert Lowe wrote: > Jeff Meyer wrote: > > Attention: lurker factor in play... > > >>An implementation >>for what IPDR requires is pretty trivial. IPDR >>members have access to opensource for this in >>both Java and C. >> > > The common definition of opensource includes descriptives such > as "publicly-available" and "no-strings-attached". You seem > like a pretty good evangelist for IPDR, but once someone starts > passing the collection plate, using the word "opensource" only > sullies the good name of the sacred cow of 21st century computing > with the marketeer's comparison to a members-only reference > implementation. Some words don't have increasing degrees of > comparison. You can be dead, but you can't be 'deader', and in > my book, open is open, not open for a fee. Grammer lesson and > rant off... > > -Robert > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri Oct 4 16:05:29 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA19542 for ; Fri, 4 Oct 2002 16:05:28 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17xYad-0002AS-00 for ipfix-list@mil.doit.wisc.edu; Fri, 04 Oct 2002 14:58:23 -0500 Received: from palrel12.hp.com ([156.153.255.237]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 17xYab-0002AM-00 for ipfix-eval@net.doit.wisc.edu; Fri, 04 Oct 2002 14:58:21 -0500 Received: from hpcuhe.cup.hp.com (hpcuhe.cup.hp.com [15.0.80.203]) by palrel12.hp.com (Postfix) with ESMTP id 5BD5BE005A6; Fri, 4 Oct 2002 12:58:20 -0700 (PDT) Received: from simail.cup.hp.com (imail@sim.cup.hp.com [15.16.123.26]) by hpcuhe.cup.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) with ESMTP id MAA26179; Fri, 4 Oct 2002 12:58:15 -0700 (PDT) Received: from cup.hp.com ([15.16.124.96]) by simail.cup.hp.com (InterMail vM.5.01.03.01 201-253-122-118-101-20010319) with ESMTP id <20021004202926.HERV18196.simail.cup.hp.com@cup.hp.com>; Fri, 4 Oct 2002 13:29:26 -0700 Message-ID: <3D9DF355.2010306@cup.hp.com> Date: Fri, 04 Oct 2002 13:00:21 -0700 From: Jeff Meyer User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:0.9.4.1) Gecko/20020508 Netscape6/6.2.3 X-Accept-Language: en-us MIME-Version: 1.0 To: "Harrington, David" Cc: ipfix-eval@net.doit.wisc.edu Subject: Re: [ipfix-eval] Discussion of Candidate Protocols References: <6D745637A7E0F94DA070743C55CDA9BA256D5A@NHROCMBX1.ets.enterasys.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit David, Section 1.3 in the IPDR Advocacy draft states: 1.3 Patent Protection There are no known patents which apply to or affect the right to freely use this technology. I have been assured that this is the case. Regards, Jeff Meyer Harrington, David wrote: > A question: Are there intellectual property claims against IPDR? There > probably are for all the protocols proposed; I just don't remember > seeing anything mentioned and want to make sure all the cards are on the > table. > > dbh > > > -----Original Message----- > > From: Robert Lowe [mailto:robert.h.lowe@lawrence.edu] > > Sent: Friday, October 04, 2002 1:57 PM > > To: Jeff Meyer > > Cc: Sebastian Zander; Peter Ludemann; ipfix-eval@net.doit.wisc.edu > > Subject: Re: [ipfix-eval] Discussion of Candidate Protocols > > > > > > Jeff Meyer wrote: > > > > Attention: lurker factor in play... > > > > > An implementation > > > for what IPDR requires is pretty trivial. IPDR > > > members have access to opensource for this in > > > both Java and C. > > > > The common definition of opensource includes descriptives such > > as "publicly-available" and "no-strings-attached". You seem > > like a pretty good evangelist for IPDR, but once someone starts > > passing the collection plate, using the word "opensource" only > > sullies the good name of the sacred cow of 21st century computing > > with the marketeer's comparison to a members-only reference > > implementation. Some words don't have increasing degrees of > > comparison. You can be dead, but you can't be 'deader', and in > > my book, open is open, not open for a fee. Grammer lesson and > > rant off... > > > > -Robert > > > > -- > > Help mailto:majordomo@net.doit.wisc.edu and say "help" > > in message body > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe ipfix" in message body > > Archive http://ipfix.doit.wisc.edu/archive/ > > > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri Oct 4 17:49:33 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA23257 for ; Fri, 4 Oct 2002 17:49:32 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17xaBy-0004jx-00 for ipfix-list@mil.doit.wisc.edu; Fri, 04 Oct 2002 16:41:02 -0500 Received: from palrel11.hp.com ([156.153.255.246]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 17xaBw-0004jj-00 for ipfix-eval@net.doit.wisc.edu; Fri, 04 Oct 2002 16:41:00 -0500 Received: from hpcuhe.cup.hp.com (hpcuhe.cup.hp.com [15.0.80.203]) by palrel11.hp.com (Postfix) with ESMTP id 6F17A600285; Fri, 4 Oct 2002 14:40:59 -0700 (PDT) Received: from simail.cup.hp.com (imail@sim.cup.hp.com [15.16.123.26]) by hpcuhe.cup.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) with ESMTP id OAA03135; Fri, 4 Oct 2002 14:40:54 -0700 (PDT) Received: from cup.hp.com ([15.16.124.96]) by simail.cup.hp.com (InterMail vM.5.01.03.01 201-253-122-118-101-20010319) with ESMTP id <20021004221205.HEUZ18196.simail.cup.hp.com@cup.hp.com>; Fri, 4 Oct 2002 15:12:05 -0700 Message-ID: <3D9E0B64.8020404@cup.hp.com> Date: Fri, 04 Oct 2002 14:43:00 -0700 From: Jeff Meyer User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:0.9.4.1) Gecko/20020508 Netscape6/6.2.3 X-Accept-Language: en-us MIME-Version: 1.0 To: Sebastian Zander Cc: Peter Ludemann , ipfix-eval@net.doit.wisc.edu Subject: Re: [ipfix-eval] Discussion of Candidate Protocols References: <3D9C6B6C.6010803@cup.hp.com> <3D9D5A10.2030906@fokus.gmd.de> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Sebastian, Diameter's XML draft is pretty close to IPDR, thanks for the pointer. The trick used by IPDR, however, is to write directly using XML-Schema's directives for defining information models, and reuse XML-Schema's existing data type set rather than inventing new names. Diameter uses the older DTD definition syntax to define a grammar from scratch. The benefit of the IPDR model is that you not only have a description language (for mapping to various serialization protocols, e.g. Compact IPDR...), but a validateable XML representation of that data immediately falls out, if you are interested in having an XML representation. Note that the document itself only contains descriptive information about the information items (elements and groups of elements). How one maps from this information set to Compact IPDR or DB tables, is established by convention, documented in a separate RFC/spec. So, I don't think your concern about putting too much in the document is an issue. I don't understand your comment: > I am also wondering if the IETF would ever standardize > things like binary formats. Probably not. Diameter protocol messages define a binary payload, as do many other IETF protocols? Regards, Jeff Meyer Sebastian Zander wrote: > Jeff Meyer wrote: > > [...] > >> >> So you have one description language, based on >> XML Schema, a wire protocol, a binary format, >> an XML format and a DB mapper, in a single document. >> >> >> Show me how that is done w/ the other proposed >> drafts? > > > Jeff, > > > its very nice to have these things but I don't think its > a good idea to have them in one document. IMHO the main > focus here is on the protocol. Working on all the things > at once will only slow down the standardization process. > I am also wondering if the IETF would ever standardize > things like binary formats. Probably not. > > Furthermore I am not convinced that it is a good idea to > standardize things like DB mapper or binary formats. E.g. > there exist a number of different formats and some people > may want to continue using them. But informational RFCs > could cover all these things. > > To summarize my point: the evaluation should focus on the > protocol and its capabilities and not on things storage > formats etc. > > BTW Diameter has a proposed XML definition as well > (draft-frascone-aaa-xml-dictionary-00.html). Another proposal > was to use SMIng (draft-schoenw-sming-dia-00.txt). > > Cheers, > > Sebastian > > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri Oct 4 17:50:58 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA23293 for ; Fri, 4 Oct 2002 17:50:57 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17xaAm-0004ha-00 for ipfix-list@mil.doit.wisc.edu; Fri, 04 Oct 2002 16:39:48 -0500 Received: from ctron-dnm.enterasys.com ([12.25.1.120] ident=firewall-user) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 17xaAj-0004hQ-00 for ipfix-eval@net.doit.wisc.edu; Fri, 04 Oct 2002 16:39:45 -0500 Received: (from uucp@localhost) by ctron-dnm.enterasys.com (8.8.7/8.8.7) id RAA00031 for ; Fri, 4 Oct 2002 17:51:25 -0400 (EDT) Received: from unknown(134.141.77.96) by ctron-dnm.enterasys.com via smap (4.1) id xma000023; Fri, 4 Oct 02 17:50:34 -0400 Received: by cnc-exc1.enterasys.com with Internet Mail Service (5.5.2653.19) id ; Fri, 4 Oct 2002 17:33:34 -0400 Message-ID: <6D745637A7E0F94DA070743C55CDA9BA075831@NHROCMBX1.ets.enterasys.com> From: "Harrington, David" To: ipfix-eval@net.doit.wisc.edu Subject: RE: [ipfix-eval] Discussion of Candidate Protocols Date: Fri, 4 Oct 2002 17:38:56 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C26BEE.70EB4478" Precedence: bulk Sender: majordomo listserver This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C26BEE.70EB4478 Content-Type: text/plain Hi, Let me be clear that I am advocating no particular solution. I am trying to cut through any hype and understand the technical pros and cons of each approach. Am I biased? You decide. I don't believe so, and want to dispel any bias assumptions. I am a network management technology analyst for the CTO of Enterasys (which used to be Cabletron). 1) I analyzed LFAP in 1999(?) for its potential as an industry standard and found it lacking. Paul has since addressed most of the points I raised, and we've had many discussions about its pros and cons. It has both. 2) I was a member of the IPDR and reviewed their early specs. I did not find it compelling. Its development in a fee-based members-only forum limited the industry's ability to review it as it developed. There were obviously many talented individuals involved, but some companies had a larger influence in its design, which makes me believe it may not be very vendor-neutral. It included a number of people from telco companies, where usage accounting and billing has a long history. I pushed for a while to have them submit it to the IETF, but they resisted. As a long-time IETFer, I didn't see much justification for not working within an open standards organization like the IETF. 3) I have some serious reservations about Netflow's technical design, especially after talking candidly with many Cisco developers at IETF meetings. Nonetheless, I promoted the replacement of LFAP with Netflow in Enterasys devices because of Netflow's wide-spread deployment and third-party application support. We did not want to be in the LFAP accounting application business. Cisco is our primary competitor, and I would like to see them not control the flow accounting standard. I believe Netflow has both pros and cons technically. 4) Xacct sought out Enterasys as a potential partner in 2000(?). I did the technology analysis on CRANE for the due diligence. My largest concern was the proprietary nature of the protocol. Just like IPDR, it was developed in a members-only agreement and was denied the opportunity for wide-spread industry review. This was done to capture the intellectaul property rights before making it public. Another major concern was the push for standardizing "buckets for transporting proprietary data in"; I don't believe that transporting a lot of proprietary data models really helps standardization and interoperability. CRANE has pros and cons. 5) I co-authored RFC2975 "Introduction to Internet Accounting" which discusses many of the requirements identified here. 6) As the co-chair of SNMPv3 WG, I am glad nobody proposed SNMP as an IPFIX candidate because it would likely not be very satisfactory for flow accounting purposes. SNMP has pros and cons, but I don't think SNMP (as currently defined) is well suited to this particular task. I am not advocating any particular solution, and I believe all the proposals offer some benefits for the stated purpose. That being said, I have chosen to reply to Jeff's mail because it uses SNMP for comparison and I fail to fully understand the point being made. Comments inline. dbh > -----Original Message----- > From: Jeff Meyer [mailto:jeffm@cup.hp.com] > Sent: Friday, October 04, 2002 12:58 PM > To: Sebastian Zander > Cc: Peter Ludemann; ipfix-eval@net.doit.wisc.edu > Subject: Re: [ipfix-eval] Discussion of Candidate Protocols > > > Hi, > > There seems to be some confusion over the separation > between a formal, machine readable description of > the data model (as IPDR defines), and the requirements put > on an implementation (especially in an IPFIX Middlebox). > > Presumably folks are familiar with SNMP and the > separation between what a MIB defines and how the > protocol operates. > > Basically IPDR's use of XML schema is producing a > MIB for sets of accounting information. > So IPDR provides a formal machine readable description of a data model just as SNMP does. The difference is IPDR uses an XML schema and MIBs use SMI schemas. OK. > > Why not use MIB's? Well, SNMP is really geared towards > providing views/access into a current system state. Historical > information tends to be global or other coarse grain > counters. Fine grain counters in SNMP are hindered by > SNMP's OID indexing model (i.e. you end up with something > like RMON, a clever use of SNMP, but not much fun to > code to). You make these assertions about SNMP, without any supporting analysis. You seem to do this as the setup for a comparison with IPDR's XML schema, but you don't follow through. When you say SNMP is geared towards providing views/access into a current system state, are you talking about the SNMP protocol being geared to this, or is it just that currently existing SNMP mibs seem geared to this task? You talk about historical information being global or coarse-grain counters. Again, are you talking about the SNMP protocol constraining this, or is this an artifact of the mibs that have been defined? How does XML do this differently? If we can model this in XML, couldn't we also model this in a mib? (Does this mean you're advocating SNMP as a viable solution? ;-) I am missing the point about how OIDs hinder fine-grained counters. An OID is used to name an object such as a counter. I don't think the naming has a significant effect on how fine-grained the counter is. Are you asserting that somehow XML-naming better supports fine-grained counters? Can you explain further? > > IP Flow information is just a specifically defined > set of metrics which are being collected based on > certain observed network traffic. SNMP mibs typically are also a specifically defined set of metrics which are being collected based on certain observed network traffic, such as bytes in/out or the RMON flow statistics, or other events. So IP Flow information is just a specifically defined set of metrics, like a MIB. Are you advocating we solve the problem by writing a mib module for IP Flows? > It just so happens > that this can produce a huge amount of info, which > often cannot be stored on the device itself for any > period of time. For this reason, using SNMP for gathering > flow info is probably a bad idea, and I'm glad to > see no one signed up as the SNMPv3 advocate ;-) Off-loading large amounts of information quickly could be done using SNMP Informs. I certainly agree that using SNMP to gather flow info is probably a bad idea if you mean gathering it up and waiting for applications to poll for it. I would have concerns about the efficiency of SNMP Informs compared to a protocol designed to handle off-loading large quantities of flow information quickly. See RFC2975. As you pointed out, SNMP makes a clear separation between the data modeling and the transport mechanism. I wouldn't recommend SNMPv3 for IPFIX primarily because of the transport constraints associated with 484 byte messages over UDP, lack of support for streaming data, and repetitive OID attribute identifiers, not because of the data modeling capabilities. I don't see that an XML data model addresses those problems. > > > But the essential concept of MIBs, which SMIng is also > looking at revising is important. I disagree that SMIng is looking at revising the essential concept of MIBs. They are looking at revising the SMI to improve ease-of-use and add object-orientation. The essential concept of MIBs as "data models that are extensible without protocol changes" remains the same. > The extensibility > model which MIBs provided for SNMP is what has led > to its large scale adoption for many problem domains. Agreed. An extensible data model has been important to SNMP's large scale adoption, and I believe an extensible data model should be supported by IPFIX. Keep in mind however, that there are trade-offs. An extensible data model adds overhead and complexity, which may reduce the efficiency of an extensible IPFIX. A vendor-extensible data modeling language also leads to many vendor extensions which greatly reduces standardization and the ability to aggregate/compare/contrast information from multiple vendors. This has been a major factor in SNMP applications all offering similar limited sets of functionality, and the large number of applications that are little more than mib browsers - device vendors do not provide solid support for fully-compliant industry standard mib implementations, but rather offer lots of proprietary mib data using a standardized modeling language and a standardized transport mechanism. The more data elements that can be named, the longer the names need to be to differentiate them. OIDs work well in SNMP to provide a data naming space that can be extended as needed without naming conflicts. Integers work well in LFAP to name a limited set of data elements. XML is extensible, and will suffer the tradeoffs of long names and lots of proprietary extensions just like SNMP. > > It just so happens that the profile of flow exporting > is not appropriate for the SNMP domain. Here you lose me. What is the "profile" you refer to that makes flow exporting inappropriate for the SNMP domain? How does IPDR deal with those same profile characteristics differently than SNMP and the other IPFIX proposals? > > > However, the statement that IP flow is somehow unique > in its data requirements and as such a more generalized > "data mover" is somehow problematic, is just plain wrong. If your argument is that IP flow is just another data model and that a generalized "data mover" is acceptable, then are you advocating we model IP flow in a MIB and move the data using SNMP? If your argument is that IP flow is just another data model and that a generalized "data mover" is acceptable as long as it meets performance requirements, then would a MIB transported over another protocol be acceptable? or would a mib transported over an SNMP that supported TCP, larger messages, and OID suppression/compression be acceptable? If so, you might want to check on some of the work being done by the EOS WG. > > As a mediation product vendor I see lots of devices > with lots of vendor specific protocols. And there > are several which have the same protocol requirements > as IPFIX. Just different data models. > > Consider a middlebox which looks not only at flows, > but also at protocols above layer 4. Would it not > make sense for the additional information produced > by this to use the same basic protocol as IPFIX? So you're arguing that if IPFIX already solves the protocol problems associated with fast-offloading of bulk data, it should be reused for transporting any type of data? Let me make sure I follow your logic: You're arguing that we should keep the data model and protocol separate. You're arguing that we should reuse rather than develop more task-specific protocols. Then shouldn't you also be advocating using existing mechanisms for data modeling of formal, machine readable descriptions of the data model, such as the SMI rather than XML? > > IPFIX does identify extensibility as a requirement, > is there some bounded set of extensions which are > envisioned? > One of the real strengths of SNMP's MIB approach is the allocation of a partitioned namespace, because industry standard data models were developed, and because the enterprise-specific namespace encouraged vendors to use SNMP. One of the real strengths of SNMP's MIB approach is the specification of a common data modeling language, with very limited syntax options, that allows machine-parsing of any mib and display of any mib information. One of the real weaknesses of SNMP's MIB approach is the allocation of an enterprise-specific namespace, because vendors used it to develop proprietary data models which hindered interoperability across vendors. I think it is very important to understand the tradeoffs involved in deciding how extensible something should be. > If not, then a trivial numeric id scheme is a pretty > lousy namespace to work from. Diameter's vendorId/ > avaCode model is not much better, but at least has > some partitioning. SNMP's OID hierarchy has been found to be a good namespace to work from for its extensibility aspects and partitioning, and it has lots of room for extension. Are you advocating SNMP again? tsk, tsk. ;-) > > > The only possible unique aspect I could picture is > based on observing NFv2,5,7. In all of these versions, > all data items were fixed width (typically 32-bit > integers). So... If IPFIX wants to say that only > fixed with data values may be sent via the IPFIX > protocol, then I'd say, maybe you could optimize > beyond IPDR for that. > > > A minimal implementation of IPDR on a middlebox > would NOT need to have any XML knowledge whatsoever. > As a producer, the system could merely hardcode the > information model produced (or make it configurable). > > The encoding itself follows XDR format, which has > worked alright for NFS and RPC, and is the precursor > to the encoding for DCE and CORBA encoding. It is > a very simple model. Read RFC 1832. An implementation > for what IPDR requires is pretty trivial. IPDR > members have access to opensource for this in > both Java and C. And as Stas pointed out in an > earlier memo there are > 10 implementations out > there. > > The other benefits from IPDR which I cited, such > as XML mapping, DB, binary file format, etc. are > just that, additional benefits. They ARE NOT > REQUIRED to be implemented. > > > Regards, > > Jeff Meyer > > > > > Sebastian Zander wrote: > > > Jeff Meyer wrote: > > > > [...] > > > >> > >> So you have one description language, based on > >> XML Schema, a wire protocol, a binary format, > >> an XML format and a DB mapper, in a single document. > >> > >> > >> Show me how that is done w/ the other proposed > >> drafts? > > > > > > Jeff, > > > > > > its very nice to have these things but I don't think its > > a good idea to have them in one document. IMHO the main > > focus here is on the protocol. Working on all the things > > at once will only slow down the standardization process. > > I am also wondering if the IETF would ever standardize > > things like binary formats. Probably not. > > > > Furthermore I am not convinced that it is a good idea to > > standardize things like DB mapper or binary formats. E.g. > > there exist a number of different formats and some people > > may want to continue using them. But informational RFCs > > could cover all these things. > > > > To summarize my point: the evaluation should focus on the > > protocol and its capabilities and not on things storage > > formats etc. > > > > BTW Diameter has a proposed XML definition as well > > (draft-frascone-aaa-xml-dictionary-00.html). Another proposal > > was to use SMIng (draft-schoenw-sming-dia-00.txt). > > > > Cheers, > > > > Sebastian > > > > > > > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" > in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ > ------_=_NextPart_001_01C26BEE.70EB4478 Content-Type: text/html Content-Transfer-Encoding: quoted-printable RE: [ipfix-eval] Discussion of Candidate Protocols

Hi,

Let me be clear that I am advocating no particular = solution. I am trying to cut through any hype and understand the = technical pros and cons of each approach.

Am I biased? You decide. I don't believe so, and want = to dispel any bias assumptions.
I am a network management technology analyst for the = CTO of Enterasys (which used to be Cabletron).

1) I analyzed LFAP in 1999(?) for its potential as an = industry standard and found it lacking. Paul has since addressed most = of the points I raised, and we've had many discussions about its pros = and cons. It has both.

2) I was a member of the IPDR and reviewed their = early specs. I did not find it compelling. Its development in a = fee-based members-only forum limited the industry's ability to review = it as it developed. There were obviously many talented individuals = involved, but some companies had a larger influence in its design, = which makes me believe it may not be very vendor-neutral. It included a = number of people from telco companies, where usage accounting and = billing has a long history. I pushed for a while to have them submit it = to the IETF, but they resisted. As a long-time IETFer, I didn't see = much justification for not working within an open standards = organization like the IETF.

3) I have some serious reservations about Netflow's = technical design, especially after talking candidly with many Cisco = developers at IETF meetings. Nonetheless, I promoted the replacement of = LFAP with Netflow in Enterasys devices because of Netflow's wide-spread = deployment and third-party application support. We did not want to be = in the LFAP accounting application business. Cisco is our primary = competitor, and I would like to see them not control the flow = accounting standard. I believe Netflow has both pros and cons = technically.

4) Xacct sought out Enterasys as a potential partner = in 2000(?). I did the technology analysis on CRANE for the due = diligence. My largest concern was the proprietary nature of the = protocol. Just like IPDR, it was developed in a members-only agreement = and was denied the opportunity for wide-spread industry review. This = was done to capture the intellectaul property rights before making it = public. Another major concern was the push for standardizing = "buckets for transporting proprietary data in"; I don't = believe that transporting a lot of proprietary data models really helps = standardization and interoperability. CRANE has pros and = cons.

5) I co-authored RFC2975 "Introduction to = Internet Accounting" which discusses many of the requirements = identified here.

6) As the co-chair of SNMPv3 WG, I am glad nobody = proposed SNMP as an IPFIX candidate because it would likely not be very = satisfactory for flow accounting purposes. SNMP has pros and cons, but = I don't think SNMP (as currently defined) is well suited to this = particular task.

I am not advocating any particular solution, and I = believe all the proposals offer some benefits for the stated = purpose.

That being said, I have chosen to reply to Jeff's = mail because it uses SNMP for comparison and I fail to fully understand = the point being made.

Comments inline.

dbh

> -----Original Message-----
> From: Jeff Meyer [mailto:jeffm@cup.hp.com]
> Sent: Friday, October 04, 2002 12:58 PM
> To: Sebastian Zander
> Cc: Peter Ludemann; = ipfix-eval@net.doit.wisc.edu
> Subject: Re: [ipfix-eval] Discussion of = Candidate Protocols
>
>
> Hi,
>
>    There seems to be some = confusion over the separation
> between a formal, machine readable description = of
> the data model (as IPDR defines), and the = requirements put
> on an implementation (especially in an IPFIX = Middlebox).
>
>    Presumably folks are familiar = with SNMP and the
> separation between what a MIB defines and how = the
> protocol operates.
>
>    Basically IPDR's use of XML = schema is producing a
> MIB for sets of accounting information.
>

So IPDR provides a formal machine readable = description of a data model just as SNMP does. The difference is IPDR = uses an XML schema and MIBs use SMI schemas.

OK.

>
> Why not use MIB's? = Well, SNMP is really geared towards
> providing views/access into a current system = state. Historical
> information tends to be global or other coarse = grain
> counters. Fine grain counters in SNMP are = hindered by
> SNMP's OID indexing model (i.e. you end up with = something
> like RMON, a clever use of SNMP, but not much = fun to
> code to).

You make these assertions about SNMP, without any = supporting analysis. You seem to do this as the setup for a comparison = with IPDR's XML schema, but you don't follow through.

When you say SNMP is geared towards providing = views/access into a current system state, are you talking about the = SNMP protocol being geared to this, or is it just that currently = existing SNMP mibs seem geared to this task?

You talk about historical information being global or = coarse-grain counters. Again, are you talking about the SNMP protocol = constraining this, or is this an artifact of the mibs that have been = defined? How does XML do this differently? If we can model this in XML, = couldn't we also model this in a mib? (Does this mean you're advocating = SNMP as a viable solution? ;-)

I am missing the point about how OIDs hinder = fine-grained counters. An OID is used to name an object such as a = counter. I don't think the naming has a significant effect on how = fine-grained the counter is. Are you asserting that somehow XML-naming = better supports fine-grained counters?

Can you explain further?
>
> IP Flow information is just a = specifically defined
> set of metrics which are being collected based = on
> certain observed network traffic. =

SNMP mibs typically are also a specifically defined = set of metrics which are being collected based on certain observed = network traffic, such as bytes in/out or the RMON flow statistics, or = other events. So IP Flow information is just a specifically defined set = of metrics, like a MIB. Are you advocating we solve the problem by = writing a mib module for IP Flows?

> It just so happens
> that this can produce a huge amount of info, = which
> often cannot be stored on the device itself for = any
> period of time. For this reason, using = SNMP for gathering
> flow info is probably a bad idea, and I'm glad = to
> see no one signed up as the SNMPv3 advocate = ;-)

Off-loading large amounts of information quickly = could be done using SNMP Informs. I certainly agree that using SNMP to = gather flow info is probably a bad idea if you mean gathering it up and = waiting for applications to poll for it. I would have concerns about = the efficiency of SNMP Informs compared to a protocol designed to = handle off-loading large quantities of flow information quickly. See = RFC2975.

As you pointed out, SNMP makes a clear separation = between the data modeling and the transport mechanism. I wouldn't = recommend SNMPv3 for IPFIX primarily because of the transport = constraints associated with 484 byte messages over UDP, lack of support = for streaming data, and repetitive OID attribute identifiers, not = because of the data modeling capabilities.

I don't see that an XML data model addresses those = problems.
>
>
> But the essential concept of = MIBs, which SMIng is also
> looking at revising is important.

I disagree that SMIng is looking at revising the = essential concept of MIBs. They are looking at revising the SMI to = improve ease-of-use and add object-orientation. The essential concept = of MIBs as "data models that are extensible without protocol = changes" remains the same.

> The extensibility
> model which MIBs provided for SNMP is what has = led
> to its large scale adoption for many problem = domains.

Agreed. An extensible data model has been important = to SNMP's large scale adoption, and I believe an extensible data model = should be supported by IPFIX. Keep in mind however, that there = are trade-offs. An extensible data model adds overhead and complexity, = which may reduce the efficiency of an extensible IPFIX.

A vendor-extensible data modeling language also leads = to many vendor extensions which greatly reduces standardization and the = ability to aggregate/compare/contrast information from multiple = vendors. This has been a major factor in SNMP applications all offering = similar limited sets of functionality, and the large number of = applications that are little more than mib browsers - device vendors do = not provide solid support for fully-compliant industry standard mib = implementations, but rather offer lots of proprietary mib data using a = standardized modeling language and a standardized transport = mechanism.

The more data elements that can be named, the longer = the names need to be to differentiate them. OIDs work well in SNMP to = provide a data naming space that can be extended as needed without = naming conflicts. Integers work well in LFAP to name a limited set of = data elements.

XML is extensible, and will suffer the tradeoffs of = long names and lots of proprietary extensions just like SNMP.

>
> It just so happens that the = profile of flow exporting
> is not appropriate for the SNMP domain.

Here you lose me. What is the "profile" you = refer to that makes flow exporting inappropriate for the SNMP domain? = How does IPDR deal with those same profile characteristics differently = than SNMP and the other IPFIX proposals?

>
>
> However, the statement that = IP flow is somehow unique
> in its data requirements and as such a more = generalized
> "data mover" is somehow problematic, = is just plain wrong.

If your argument is that IP flow is just another data = model and that a generalized "data mover" is acceptable, then = are you advocating we model IP flow in a MIB and move the data using = SNMP?

If your argument is that IP flow is just another data = model and that a generalized "data mover" is acceptable as = long as it meets performance requirements, then would a MIB transported = over another protocol be acceptable? or would a mib transported over an = SNMP that supported TCP, larger messages, and OID = suppression/compression be acceptable? If so, you might want to check = on some of the work being done by the EOS WG.

>
> As a mediation product vendor = I see lots of devices
> with lots of vendor specific protocols. = And there
> are several which have the same protocol = requirements
> as IPFIX. Just different data = models.
>
> Consider a middlebox which = looks not only at flows,
> but also at protocols above layer 4. = Would it not
> make sense for the additional information = produced
> by this to use the same basic protocol as = IPFIX?

So you're arguing that if IPFIX already solves the = protocol problems associated with fast-offloading of bulk data, it = should be reused for transporting any type of data?

Let me make sure I follow your logic:
You're arguing that we should keep the data model = and protocol separate.
You're arguing that we should reuse rather than = develop more task-specific protocols.

Then shouldn't you also be advocating using existing = mechanisms for data modeling of formal, machine readable descriptions = of the data model, such as the SMI rather than XML?

>
> IPFIX does identify = extensibility as a requirement,
> is there some bounded set of extensions which = are
> envisioned?
>

One of the real strengths of SNMP's MIB approach is = the allocation of a partitioned namespace, because industry standard = data models were developed, and because the enterprise-specific = namespace encouraged vendors to use SNMP.

One of the real strengths of SNMP's MIB approach is = the specification of a common data modeling language, with very limited = syntax options, that allows machine-parsing of any mib and display of = any mib information.

One of the real weaknesses of SNMP's MIB approach is = the allocation of an enterprise-specific namespace, because vendors = used it to develop proprietary data models which hindered = interoperability across vendors.

I think it is very important to understand the = tradeoffs involved in deciding how extensible something should = be.

> If not, then a trivial numeric = id scheme is a pretty
> lousy namespace to work from. Diameter's = vendorId/
> avaCode model is not much better, but at least = has
> some partitioning.

SNMP's OID hierarchy has been found to be a good = namespace to work from for its extensibility aspects and partitioning, = and it has lots of room for extension. Are you advocating SNMP again? = tsk, tsk. ;-)

>
>
>    The only possible unique = aspect I could picture is
> based on observing NFv2,5,7. In all of = these versions,
> all data items were fixed width (typically = 32-bit
> integers). So... If IPFIX wants to = say that only
> fixed with data values may be sent via the = IPFIX
> protocol, then I'd say, maybe you could = optimize
> beyond IPDR for that.
>
>
>    A minimal implementation of = IPDR on a middlebox
> would NOT need to have any XML knowledge = whatsoever.
> As a producer, the system could merely hardcode = the
> information model produced (or make it = configurable).
>
>    The encoding itself follows = XDR format, which has
> worked alright for NFS and RPC, and is the = precursor
> to the encoding for DCE and CORBA = encoding. It is
> a very simple model. Read RFC 1832. = An implementation
> for what IPDR requires is pretty trivial. = IPDR
> members have access to opensource for this = in
> both Java and C. And as Stas pointed out = in an
> earlier memo there are > 10 implementations = out
> there.
>
>    The other benefits from IPDR = which I cited, such
> as XML mapping, DB, binary file format, etc. = are
> just that, additional benefits. They ARE = NOT
> REQUIRED to be implemented.
>
>
> Regards,
>
>    Jeff Meyer
>
>
>
>
> Sebastian Zander wrote:
>
> > Jeff Meyer wrote:
> >
> > [...]
> >
> >>
> >>   So you have one = description language, based on
> >> XML Schema, a wire protocol, a binary = format,
> >> an XML format and a DB mapper, in a = single document.
> >>
> >>
> >>   Show me how that is done = w/ the other proposed
> >> drafts?
> >
> >
> > Jeff,
> >
> >
> > its very nice to have these things but I = don't think its
> > a good idea to have them in one document. = IMHO the main
> > focus here is on the protocol. Working on = all the things
> > at once will only slow down the = standardization process.
> > I am also wondering if the IETF would ever = standardize
> > things like binary formats. Probably = not.
> >
> > Furthermore I am not convinced that it is = a good idea to
> > standardize things like DB mapper or = binary formats. E.g.
> > there exist a number of different formats = and some people
> > may want to continue using them. But = informational RFCs
> > could cover all these things.
> >
> > To summarize my point: the evaluation = should focus on the
> > protocol and its capabilities and not on = things storage
> > formats etc.
> >
> > BTW Diameter has a proposed XML definition = as well
> > = (draft-frascone-aaa-xml-dictionary-00.html). Another proposal
> > was to use SMIng = (draft-schoenw-sming-dia-00.txt).
> >
> > Cheers,
> >
> > Sebastian
> >
> >
>
>
>
>
> --
> Help        = mailto:majordomo@net.doit.wi= sc.edu and say "help"
> in message body
> Unsubscribe mailto:majordomo@net.doit.wi= sc.edu and say
> "unsubscribe ipfix" in message = body
> Archive     http://ipfix.doit.wisc.edu/archive/
>

------_=_NextPart_001_01C26BEE.70EB4478-- -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri Oct 4 18:30:20 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA24422 for ; Fri, 4 Oct 2002 18:30:20 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17xaqA-0005lb-00 for ipfix-list@mil.doit.wisc.edu; Fri, 04 Oct 2002 17:22:34 -0500 Received: from ctron-dnm.enterasys.com ([12.25.1.120] ident=firewall-user) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 17xaq8-0005lV-00 for ipfix-eval@net.doit.wisc.edu; Fri, 04 Oct 2002 17:22:32 -0500 Received: (from uucp@localhost) by ctron-dnm.enterasys.com (8.8.7/8.8.7) id SAA00535; Fri, 4 Oct 2002 18:27:34 -0400 (EDT) Received: from unknown(134.141.77.96) by ctron-dnm.enterasys.com via smap (4.1) id xma000527; Fri, 4 Oct 02 18:26:45 -0400 Received: by cnc-exc1.enterasys.com with Internet Mail Service (5.5.2653.19) id ; Fri, 4 Oct 2002 18:09:46 -0400 Message-ID: <6D745637A7E0F94DA070743C55CDA9BA256D5F@NHROCMBX1.ets.enterasys.com> From: "Harrington, David" To: "'Jeff Meyer'" Cc: ipfix-eval@net.doit.wisc.edu Subject: RE: [ipfix-eval] Discussion of Candidate Protocols Date: Fri, 4 Oct 2002 18:15:06 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C26BF3.0B2310CF" Precedence: bulk Sender: majordomo listserver This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C26BF3.0B2310CF Content-Type: text/plain Hi Jeff, Since IPDR is forum based, rather than a vendor proposal, have you been assured that all the members of the forum will submit IPR Notices to the IETF stating this? or that a joint notice from IPDR.org will be filed? dbh > -----Original Message----- > From: Jeff Meyer [mailto:jeffm@cup.hp.com] > Sent: Friday, October 04, 2002 4:00 PM > To: Harrington, David > Cc: ipfix-eval@net.doit.wisc.edu > Subject: Re: [ipfix-eval] Discussion of Candidate Protocols > > > David, > > Section 1.3 in the IPDR Advocacy draft states: > > 1.3 Patent Protection > > There are no known patents which apply to or affect the > right to freely use this technology. > > I have been assured that this is the case. > > Regards, > > Jeff Meyer > > > Harrington, David wrote: > > > A question: Are there intellectual property claims against > IPDR? There > > probably are for all the protocols proposed; I just don't remember > > seeing anything mentioned and want to make sure all the > cards are on the > > table. > > > > dbh > > > > > -----Original Message----- > > > From: Robert Lowe [mailto:robert.h.lowe@lawrence.edu] > > > Sent: Friday, October 04, 2002 1:57 PM > > > To: Jeff Meyer > > > Cc: Sebastian Zander; Peter Ludemann; > ipfix-eval@net.doit.wisc.edu > > > Subject: Re: [ipfix-eval] Discussion of Candidate Protocols > > > > > > > > > Jeff Meyer wrote: > > > > > > Attention: lurker factor in play... > > > > > > > An implementation > > > > for what IPDR requires is pretty trivial. IPDR > > > > members have access to opensource for this in > > > > both Java and C. > > > > > > The common definition of opensource includes descriptives such > > > as "publicly-available" and "no-strings-attached". You seem > > > like a pretty good evangelist for IPDR, but once someone starts > > > passing the collection plate, using the word "opensource" only > > > sullies the good name of the sacred cow of 21st century computing > > > with the marketeer's comparison to a members-only reference > > > implementation. Some words don't have increasing degrees of > > > comparison. You can be dead, but you can't be 'deader', and in > > > my book, open is open, not open for a fee. Grammer lesson and > > > rant off... > > > > > > -Robert > > > > > > -- > > > Help mailto:majordomo@net.doit.wisc.edu and say "help" > > > in message body > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > "unsubscribe ipfix" in message body > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > > > ------_=_NextPart_001_01C26BF3.0B2310CF Content-Type: text/html Content-Transfer-Encoding: quoted-printable RE: [ipfix-eval] Discussion of Candidate Protocols

Hi Jeff,

Since IPDR is forum based, rather than a vendor = proposal, have you been assured that all the members of the forum will = submit IPR Notices to the IETF stating this? or that a joint notice = from IPDR.org will be filed?

dbh

> -----Original Message-----
> From: Jeff Meyer [mailto:jeffm@cup.hp.com]
> Sent: Friday, October 04, 2002 4:00 PM
> To: Harrington, David
> Cc: ipfix-eval@net.doit.wisc.edu
> Subject: Re: [ipfix-eval] Discussion of = Candidate Protocols
>
>
> David,
>
>    Section 1.3 in the IPDR = Advocacy draft states:
>
>      1.3 Patent = Protection
>
>     There are no known = patents which apply to or affect the
>     right to freely use = this technology.
>
>    I have been assured that this = is the case.
>
> Regards,
>
>    Jeff Meyer
>
>
> Harrington, David wrote:
>
> > A question: Are there intellectual = property claims against
> IPDR? There
> > probably are for all the protocols = proposed; I just don't remember
> > seeing anything mentioned and want to make = sure all the
> cards are on the
> > table.
> >
> > dbh
> >
> > > -----Original = Message-----
> > > From: Robert Lowe [mailto:robert.h.lowe@lawrence= .edu]
> > > Sent: Friday, October 04, 2002 = 1:57 PM
> > > To: Jeff Meyer
> > > Cc: Sebastian Zander; Peter = Ludemann;
> ipfix-eval@net.doit.wisc.edu
> > > Subject: Re: [ipfix-eval] = Discussion of Candidate Protocols
> > >
> > >
> > > Jeff Meyer wrote:
> > >
> > > Attention: lurker factor in = play...
> > >
> > > > An implementation
> > > > for what IPDR requires is = pretty trivial. IPDR
> > > > members have access to = opensource for this in
> > > > both Java and C.
> > >
> > > The common definition of = opensource includes descriptives such
> > > as = "publicly-available" and = "no-strings-attached". You seem
> > > like a pretty good evangelist = for IPDR, but once someone starts
> > > passing the collection plate, = using the word "opensource" only
> > > sullies the good name of the = sacred cow of 21st century computing
> > > with the marketeer's comparison = to a members-only reference
> > > implementation. Some = words don't have increasing degrees of
> > > comparison. You can be = dead, but you can't be 'deader', and in
> > > my book, open is open, not open = for a fee. Grammer lesson and
> > > rant off...
> > >
> > > -Robert
> > >
> > > --
> > > = Help        mailto:majordomo@net.doit.wi= sc.edu and say "help"
> > > in message body
> > > Unsubscribe mailto:majordomo@net.doit.wi= sc.edu and say
> > > "unsubscribe ipfix" = in message body
> > > Archive     = http://ipfix.doit.wisc.edu/archive/
> > >
> >
>
>
>

------_=_NextPart_001_01C26BF3.0B2310CF-- -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri Oct 4 18:37:16 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA24722 for ; Fri, 4 Oct 2002 18:37:15 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17xayC-0005yH-00 for ipfix-list@mil.doit.wisc.edu; Fri, 04 Oct 2002 17:30:52 -0500 Received: from babar.switch.ch ([130.59.4.2]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 17xayA-0005x5-00; Fri, 04 Oct 2002 17:30:50 -0500 Received: from babar.switch.ch (localhost [IPv6:::1]) by babar.switch.ch (8.12.2+Sun/8.12.2) with ESMTP id g94MUIGU018113; Sat, 5 Oct 2002 00:30:18 +0200 (MEST) Received: (from leinen@localhost) by babar.switch.ch (8.12.2+Sun/8.12.2/Submit) id g94MUI4m018110; Sat, 5 Oct 2002 00:30:18 +0200 (MEST) X-Authentication-Warning: babar.switch.ch: leinen set sender to simon@limmat.switch.ch using -f To: ipfix-req@net.doit.wisc.edu Cc: Reinaldo Penno , ipfix-eval-team@net.doit.wisc.edu Subject: [ipfix-req] Data Export Reliability Requirement [was: Re: IPFIX Requirement draft status - section 6.3.2] References: <7B802811BE77D51189910002A55CFD2C04119EF5@zsc3c032.us.nortel.com> X-Face: 1Nk*r=:$IBBb8|TyRB'2WSY6u:BzMO7N)#id#-4_}MsU5?vTI?dez|JiutW4sKBLjp.l7,F 7QOld^hORRtpCUj)!cP]gtK_SyK5FW(+o"!or:v^C^]OxX^3+IPd\z,@ttmwYVO7l`6OXXYR` From: Simon Leinen In-Reply-To: <7B802811BE77D51189910002A55CFD2C04119EF5@zsc3c032.us.nortel.com> Date: 05 Oct 2002 00:30:17 +0200 Message-ID: Lines: 53 User-Agent: Gnus/5.09 (Gnus v5.9.0) Emacs/21.2.90 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Precedence: bulk Sender: majordomo listserver On Mon, 30 Sep 2002 03:00:03 -0700, "Reinaldo Penno" said: > I'm looking to understand/really clarify this requirement. It seems > ambiguous...at least to me. I do not understand what is the > meaning/purpose of "-->abscence<-- of reliability....MUST be > indicated". Yes, that just sounds weird. (High) reliability should be a goal for the exporting process, not something that can be readily measured, or even less is "present" or "absent" at a given time during the operation of an IPFIX implementation. reqs-06> Absence of reliability of the data transfer, for example caused by reqs-06> loss or reordering of packets containing flow information, MUST be reqs-06> indicated. Why not write something like this instead: "If some flow information data cannot be delivered to the collector in a timely manner, the amount of missing flow information data MUST be indicated to the collector. Possible reasons for failure to deliver data include loss of packets on the network path, or resource shortage at the exporter." I removed packet reordering as an example for reasons to lose data because I don't think it's that relevant in practice. From operational experience, I think overflows at the exporter are a much more likely source of data loss. An indication of the amount of data lost is more useful than an unspecific indication of "absence of reliability" and should be relatively easy to provide. reqs-06> Please note that if an unreliable transport protocol is used, reqs-06> reliability can be provided by higher layers. In such a case reqs-06> only lack of overall reliability MUST be indicated. And end-to-end purist might argue that even if the underlying transport protocol is used, reliability must be provided at higher layers too. See the "application layer ACK" debate on the ipfix-req list a few days ago. "Reliability can be supported by suitable transport protocols. In any case only end-to-end flow information loss between exporter and collector need be indicated." reqs-06> For example reordering could be dealt with by adding a reqs-06> sequence number to each packet. Or order may not be relevant since the exact arrival time of a piece of flow information doesn't matter - the timestamps in the flow record will provide sufficient sequence information for the collector. -- Simon. -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri Oct 4 19:39:37 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA25980 for ; Fri, 4 Oct 2002 19:39:36 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17xbob-0007K5-00 for ipfix-list@mil.doit.wisc.edu; Fri, 04 Oct 2002 18:25:01 -0500 Received: from [208.253.219.211] (helo=narus.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 17xboY-0007JS-00 for ipfix-eval@net.doit.wisc.edu; Fri, 04 Oct 2002 18:24:58 -0500 Received: from franklin.narus.com (franklin.narus.com [192.168.7.140]) by narus.com (8.11.6/8.11.6) with ESMTP id g94NM1u18141; Fri, 4 Oct 2002 16:22:01 -0700 Received: by franklin.narus.com with Internet Mail Service (5.5.2655.55) id <4CNBFRWW>; Fri, 4 Oct 2002 16:22:01 -0700 Message-ID: <580E532D9F7A9B4BAE8A130848E0DDA702083A1D@franklin.narus.com> From: Stas Khirman To: "'Harrington, David'" , "'Jeff Meyer'" , ipfix-eval@net.doit.wisc.edu Cc: "'scotton@compuserve.com'" , "'kensarno@ipdr.org'" , "'aheintz@ipdr.org'" Subject: RE: [ipfix-eval] Discussion of Candidate Protocols Date: Fri, 4 Oct 2002 16:22:01 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2655.55) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C26BFC.D7801020" Precedence: bulk Sender: majordomo listserver This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C26BFC.D7801020 Content-Type: text/plain; charset="iso-8859-1" I think that some clarification have to be made: Every and each IPDR participating companies signed "release of intellectual rights" document. At least it was a primary condition for participation ( and I hope that IPDR.org administrative staff have all paperwork in order). Certainly, I am not lawyer, but on my humble opinion all IPDR rights belong today to IPDR.org and not to any separate member-company. Submitting IPDR draft to IETF, IPDR.org is shows readiness to share all intellectual rights with IETF and (automatically) with a general public (I think IETF follows GPL policy). I'm forwarding this message to few IPDR key people and appreciate if they provide IPFIX with a more formal information and legal questions clarification . Stas -----Original Message----- From: Harrington, David [mailto:dbh@enterasys.com] Sent: Friday, October 04, 2002 3:15 PM To: 'Jeff Meyer' Cc: ipfix-eval@net.doit.wisc.edu Subject: RE: [ipfix-eval] Discussion of Candidate Protocols Hi Jeff, Since IPDR is forum based, rather than a vendor proposal, have you been assured that all the members of the forum will submit IPR Notices to the IETF stating this? or that a joint notice from IPDR.org will be filed? dbh > -----Original Message----- > From: Jeff Meyer [ mailto:jeffm@cup.hp.com ] > Sent: Friday, October 04, 2002 4:00 PM > To: Harrington, David > Cc: ipfix-eval@net.doit.wisc.edu > Subject: Re: [ipfix-eval] Discussion of Candidate Protocols > > > David, > > Section 1.3 in the IPDR Advocacy draft states: > > 1.3 Patent Protection > > There are no known patents which apply to or affect the > right to freely use this technology. > > I have been assured that this is the case. > > Regards, > > Jeff Meyer > > > Harrington, David wrote: > > > A question: Are there intellectual property claims against > IPDR? There > > probably are for all the protocols proposed; I just don't remember > > seeing anything mentioned and want to make sure all the > cards are on the > > table. > > > > dbh > > > > > -----Original Message----- > > > From: Robert Lowe [ mailto:robert.h.lowe@lawrence.edu ] > > > Sent: Friday, October 04, 2002 1:57 PM > > > To: Jeff Meyer > > > Cc: Sebastian Zander; Peter Ludemann; > ipfix-eval@net.doit.wisc.edu > > > Subject: Re: [ipfix-eval] Discussion of Candidate Protocols > > > > > > > > > Jeff Meyer wrote: > > > > > > Attention: lurker factor in play... > > > > > > > An implementation > > > > for what IPDR requires is pretty trivial. IPDR > > > > members have access to opensource for this in > > > > both Java and C. > > > > > > The common definition of opensource includes descriptives such > > > as "publicly-available" and "no-strings-attached". You seem > > > like a pretty good evangelist for IPDR, but once someone starts > > > passing the collection plate, using the word "opensource" only > > > sullies the good name of the sacred cow of 21st century computing > > > with the marketeer's comparison to a members-only reference > > > implementation. Some words don't have increasing degrees of > > > comparison. You can be dead, but you can't be 'deader', and in > > > my book, open is open, not open for a fee. Grammer lesson and > > > rant off... > > > > > > -Robert > > > > > > -- > > > Help mailto:majordomo@net.doit.wisc.edu and say "help" > > > in message body > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > "unsubscribe ipfix" in message body > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > > > ------_=_NextPart_001_01C26BFC.D7801020 Content-Type: text/html; charset="iso-8859-1" RE: [ipfix-eval] Discussion of Candidate Protocols

I think that some clarification have to be made: Every and each IPDR participating companies signed "release of intellectual rights" document. At least it was a primary condition for participation ( and I hope that IPDR.org administrative staff have all paperwork in order).

Certainly, I am not lawyer, but on my humble opinion all IPDR rights belong today to IPDR.org and not to any separate member-company. Submitting IPDR draft to IETF, IPDR.org is shows readiness to share all intellectual rights with IETF and (automatically) with a general public (I think IETF follows GPL policy).

I'm forwarding this message to few IPDR key people and appreciate if they provide IPFIX with a more formal information and legal questions clarification .

Stas

-----Original Message-----
From: Harrington, David [mailto:dbh@enterasys.com]
Sent: Friday, October 04, 2002 3:15 PM
To: 'Jeff Meyer'
Cc: ipfix-eval@net.doit.wisc.edu
Subject: RE: [ipfix-eval] Discussion of Candidate Protocols

Hi Jeff,

Since IPDR is forum based, rather than a vendor proposal, have you been assured that all the members of the forum will submit IPR Notices to the IETF stating this? or that a joint notice from IPDR.org will be filed?

dbh

> -----Original Message-----
> From: Jeff Meyer [mailto:jeffm@cup.hp.com]
> Sent: Friday, October 04, 2002 4:00 PM
> To: Harrington, David
> Cc: ipfix-eval@net.doit.wisc.edu
> Subject: Re: [ipfix-eval] Discussion of Candidate Protocols
>
>
> David,
>
>    Section 1.3 in the IPDR Advocacy draft states:
>
>      1.3 Patent Protection
>
>     There are no known patents which apply to or affect the
>     right to freely use this technology.
>
>    I have been assured that this is the case.
>
> Regards,
>
>    Jeff Meyer
>
>
> Harrington, David wrote:
>
> > A question: Are there intellectual property claims against
> IPDR? There
> > probably are for all the protocols proposed; I just don't remember
> > seeing anything mentioned and want to make sure all the
> cards are on the
> > table.
> >
> > dbh
> >
> > > -----Original Message-----
> > > From: Robert Lowe [mailto:robert.h.lowe@lawrence.edu]
> > > Sent: Friday, October 04, 2002 1:57 PM
> > > To: Jeff Meyer
> > > Cc: Sebastian Zander; Peter Ludemann;
> ipfix-eval@net.doit.wisc.edu
> > > Subject: Re: [ipfix-eval] Discussion of Candidate Protocols
> > >
> > >
> > > Jeff Meyer wrote:
> > >
> > > Attention: lurker factor in play...
> > >
> > > > An implementation
> > > > for what IPDR requires is pretty trivial. IPDR
> > > > members have access to opensource for this in
> > > > both Java and C.
> > >
> > > The common definition of opensource includes descriptives such
> > > as "publicly-available" and "no-strings-attached". You seem
> > > like a pretty good evangelist for IPDR, but once someone starts
> > > passing the collection plate, using the word "opensource" only
> > > sullies the good name of the sacred cow of 21st century computing
> > > with the marketeer's comparison to a members-only reference
> > > implementation. Some words don't have increasing degrees of
> > > comparison. You can be dead, but you can't be 'deader', and in
> > > my book, open is open, not open for a fee. Grammer lesson and
> > > rant off...
> > >
> > > -Robert
> > >
> > > --
> > > Help        mailto:majordomo@net.doit.wisc.edu and say "help"
> > > in message body
> > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say
> > > "unsubscribe ipfix" in message body
> > > Archive     http://ipfix.doit.wisc.edu/archive/
> > >
> >
>
>
>

------_=_NextPart_001_01C26BFC.D7801020-- -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri Oct 4 19:57:45 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA26336 for ; Fri, 4 Oct 2002 19:57:45 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17xcDx-0000Av-00 for ipfix-list@mil.doit.wisc.edu; Fri, 04 Oct 2002 18:51:13 -0500 Received: from babar.switch.ch ([130.59.4.2]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 17xcDu-0000A6-00 for ipfix-req@net.doit.wisc.edu; Fri, 04 Oct 2002 18:51:10 -0500 Received: from babar.switch.ch (localhost [IPv6:::1]) by babar.switch.ch (8.12.2+Sun/8.12.2) with ESMTP id g94NocGU018247; Sat, 5 Oct 2002 01:50:38 +0200 (MEST) Received: (from leinen@localhost) by babar.switch.ch (8.12.2+Sun/8.12.2/Submit) id g94Nobxg018244; Sat, 5 Oct 2002 01:50:37 +0200 (MEST) X-Authentication-Warning: babar.switch.ch: leinen set sender to simon@limmat.switch.ch using -f To: "Reinaldo Penno" Cc: req Subject: Re: [ipfix-req] Requirements drafts, NATed flows and VPNs References: <7B802811BE77D51189910002A55CFD2C0411A1DC@zsc3c032.us.nortel.com> X-Face: 1Nk*r=:$IBBb8|TyRB'2WSY6u:BzMO7N)#id#-4_}MsU5?vTI?dez|JiutW4sKBLjp.l7,F 7QOld^hORRtpCUj)!cP]gtK_SyK5FW(+o"!or:v^C^]OxX^3+IPd\z,@ttmwYVO7l`6OXXYR` From: Simon Leinen In-Reply-To: <7B802811BE77D51189910002A55CFD2C0411A1DC@zsc3c032.us.nortel.com> Date: 05 Oct 2002 01:50:37 +0200 Message-ID: Lines: 41 User-Agent: Gnus/5.09 (Gnus v5.9.0) Emacs/21.2.90 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Precedence: bulk Sender: majordomo listserver [ipfix-eval-team removed from list of recipients, since this seems purely a requirements issue.] I am opposed to this proposal because it opens a can of worms if we start to try to accomodate arbitrary services and middleboxes. What about those cool rate shapers that fiddle with TCP window sizes? If they support IPFIX, the window-fiddle-factor field ist a must! What about "cookie-cutting" layer 4-7 switches? The cookie field is a must! VoIP switches? etc. etc. Please let's try to stick with the core goal of looking at IP (v4&v6) packets and the "transport" protocol or next header. Anything else should be optional and left to other documents. The IPFIX protocol just should support enough extensibility to allow this. My counter-proposal would be to drop the MPLS label requirement from the requirements draft. -- Simon. On Mon, 30 Sep 2002 10:26:36 -0700, "Reinaldo Penno" said: > I think there are some important packet fields/scenarios that should > be included in the requirements draft. > 1 - We discussed a long time ago to include the VPN-ID as one of the > parameters exported. If Ipfix will be used in VPN environments for > accouting, QoS, monitoring, etc, this field is a must. > 2 - There is mention to intrusion detection/security as one of the > applications for IPfix, but there is no requirement to export NATed > flows (before and after). > In a device that supports NAT (traditional, layer 7 interception, > whatever) and it's going to use IPfix for security/intrusion > detection, dealing with NAT is must. There is no way a intrusion > detection device will find attacker/attacked properly without both > sides of a NAT flow. Not to mention accouting, billing, etc. > I would propose to include both items on the requirements draft. -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri Oct 4 20:11:51 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA26582 for ; Fri, 4 Oct 2002 20:11:50 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17xcPX-0000Tx-00 for ipfix-list@mil.doit.wisc.edu; Fri, 04 Oct 2002 19:03:11 -0500 Received: from rip.psg.com ([147.28.0.39]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 17xcPW-0000Tr-00 for ipfix-req@net.doit.wisc.edu; Fri, 04 Oct 2002 19:03:10 -0500 Received: from localhost ([127.0.0.1] helo=rip.psg.com.psg.com) by rip.psg.com with esmtp (Exim 4.10) id 17xcPT-0004B2-00; Fri, 04 Oct 2002 17:03:07 -0700 From: Randy Bush MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit To: Simon Leinen Cc: "Reinaldo Penno" , req Subject: Re: [ipfix-req] Requirements drafts, NATed flows and VPNs References: <7B802811BE77D51189910002A55CFD2C0411A1DC@zsc3c032.us.nortel.com> Message-Id: Date: Fri, 04 Oct 2002 17:03:07 -0700 Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit > Please let's try to stick with the core goal of looking at IP (v4&v6) > packets and the "transport" protocol or next header. i think that was the intent of this wg's charter. randy, with ad hat on -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri Oct 4 20:20:47 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA26701 for ; Fri, 4 Oct 2002 20:20:47 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17xcWE-0000iL-00 for ipfix-list@mil.doit.wisc.edu; Fri, 04 Oct 2002 19:10:06 -0500 Received: from palrel13.hp.com ([156.153.255.238]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 17xcWD-0000iF-00 for ipfix-eval@net.doit.wisc.edu; Fri, 04 Oct 2002 19:10:05 -0500 Received: from hpcuhe.cup.hp.com (hpcuhe.cup.hp.com [15.0.80.203]) by palrel13.hp.com (Postfix) with ESMTP id 733AB400393; Fri, 4 Oct 2002 17:10:04 -0700 (PDT) Received: from simail.cup.hp.com (imail@sim.cup.hp.com [15.16.123.26]) by hpcuhe.cup.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) with ESMTP id RAA11994; Fri, 4 Oct 2002 17:09:59 -0700 (PDT) Received: from cup.hp.com ([15.16.124.96]) by simail.cup.hp.com (InterMail vM.5.01.03.01 201-253-122-118-101-20010319) with ESMTP id <20021005004110.HEZF18196.simail.cup.hp.com@cup.hp.com>; Fri, 4 Oct 2002 17:41:10 -0700 Message-ID: <3D9E2E54.7040608@cup.hp.com> Date: Fri, 04 Oct 2002 17:12:05 -0700 From: Jeff Meyer User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:0.9.4.1) Gecko/20020508 Netscape6/6.2.3 X-Accept-Language: en-us MIME-Version: 1.0 To: "Harrington, David" , ipfix-eval@net.doit.wisc.edu, Aron Heintz Subject: Re: [ipfix-eval] Discussion of Candidate Protocols References: <6D745637A7E0F94DA070743C55CDA9BA256D5F@NHROCMBX1.ets.enterasys.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit David, I believe the legal agreement signed by IPDR members passes the rights to IPDR itself. IPDR's board has approved this submission activity. I'll forward this onto the IPDR President for further clarification. -- Jeff Harrington, David wrote: > Hi Jeff, > > Since IPDR is forum based, rather than a vendor proposal, have you been > assured that all the members of the forum will submit IPR Notices to the > IETF stating this? or that a joint notice from IPDR.org will be filed? > > dbh > > > -----Original Message----- > > From: Jeff Meyer [mailto:jeffm@cup.hp.com] > > Sent: Friday, October 04, 2002 4:00 PM > > To: Harrington, David > > Cc: ipfix-eval@net.doit.wisc.edu > > Subject: Re: [ipfix-eval] Discussion of Candidate Protocols > > > > > > David, > > > > Section 1.3 in the IPDR Advocacy draft states: > > > > 1.3 Patent Protection > > > > There are no known patents which apply to or affect the > > right to freely use this technology. > > > > I have been assured that this is the case. > > > > Regards, > > > > Jeff Meyer > > > > > > Harrington, David wrote: > > > > > A question: Are there intellectual property claims against > > IPDR? There > > > probably are for all the protocols proposed; I just don't remember > > > seeing anything mentioned and want to make sure all the > > cards are on the > > > table. > > > > > > dbh > > > > > > > -----Original Message----- > > > > From: Robert Lowe [mailto:robert.h.lowe@lawrence.edu] > > > > Sent: Friday, October 04, 2002 1:57 PM > > > > To: Jeff Meyer > > > > Cc: Sebastian Zander; Peter Ludemann; > > ipfix-eval@net.doit.wisc.edu > > > > Subject: Re: [ipfix-eval] Discussion of Candidate Protocols > > > > > > > > > > > > Jeff Meyer wrote: > > > > > > > > Attention: lurker factor in play... > > > > > > > > > An implementation > > > > > for what IPDR requires is pretty trivial. IPDR > > > > > members have access to opensource for this in > > > > > both Java and C. > > > > > > > > The common definition of opensource includes descriptives such > > > > as "publicly-available" and "no-strings-attached". You seem > > > > like a pretty good evangelist for IPDR, but once someone starts > > > > passing the collection plate, using the word "opensource" only > > > > sullies the good name of the sacred cow of 21st century computing > > > > with the marketeer's comparison to a members-only reference > > > > implementation. Some words don't have increasing degrees of > > > > comparison. You can be dead, but you can't be 'deader', and in > > > > my book, open is open, not open for a fee. Grammer lesson and > > > > rant off... > > > > > > > > -Robert > > > > > > > > -- > > > > Help mailto:majordomo@net.doit.wisc.edu and say "help" > > > > in message body > > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > > "unsubscribe ipfix" in message body > > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > > > > > > > > > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri Oct 4 21:44:51 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA28089 for ; Fri, 4 Oct 2002 21:44:51 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17xdso-0002qm-00 for ipfix-list@mil.doit.wisc.edu; Fri, 04 Oct 2002 20:37:30 -0500 Received: from h65s138a81n47.user.nortelnetworks.com ([47.81.138.65] helo=zsc3s004.nortelnetworks.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 17xdsm-0002pp-00 for ipfix-req@net.doit.wisc.edu; Fri, 04 Oct 2002 20:37:28 -0500 Received: from zrc2s0jx.nortelnetworks.com (zrc2s0jx.nortelnetworks.com [47.103.122.112]) by zsc3s004.nortelnetworks.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id g951atY04083; Fri, 4 Oct 2002 18:36:55 -0700 (PDT) Received: from zsc4c000.us.nortel.com (zsc4c000.us.nortel.com [47.81.138.47]) by zrc2s0jx.nortelnetworks.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id g951b6q10338; Fri, 4 Oct 2002 20:37:07 -0500 (CDT) Received: by zsc4c000.us.nortel.com with Internet Mail Service (5.5.2653.19) id ; Fri, 4 Oct 2002 18:36:52 -0700 Message-ID: <7B802811BE77D51189910002A55CFD2C0426C5EF@zsc3c032.us.nortel.com> From: "Reinaldo Penno" To: Simon Leinen Cc: req Subject: RE: [ipfix-req] Requirements drafts, NATed flows and VPNs Date: Fri, 4 Oct 2002 18:36:52 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C26C0F.AE89F542" Precedence: bulk Sender: majordomo listserver This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C26C0F.AE89F542 Content-Type: text/plain; charset="iso-8859-1" that's fair...but the premise was the statement in the draft that IPfix could be used for intrusion detection. Given that intrusion detection devices get all their data from port mirroring; the limited subset of data provided by IPfix + issues such as the lack of NAT visibility (which are directly related to the statement) makes me wonder if we shouldn't take that statement out. well, somebody could always argue that IPfix can do some intrusion detection...but I really do not think it's a good fit. The "some" argument could be expanded to include all kinds of applications. I could also agree we should drop the MPLS label requirement. Otherwise one should ask why not export information about other tunneling protocols such as L2TP/GRE/IPSEC information also. regards, Reinaldo > -----Original Message----- > From: Simon Leinen [mailto:simon@limmat.switch.ch] > Sent: Friday, October 04, 2002 4:51 PM > To: Penno, Reinaldo [BL60:0430:EXCH] > Cc: req > Subject: Re: [ipfix-req] Requirements drafts, NATed flows and VPNs > > > [ipfix-eval-team removed from list of recipients, since this seems > purely a requirements issue.] > > I am opposed to this proposal because it opens a can of worms if we > start to try to accomodate arbitrary services and middleboxes. > > What about those cool rate shapers that fiddle with TCP window sizes? > If they support IPFIX, the window-fiddle-factor field ist a must! > What about "cookie-cutting" layer 4-7 switches? The cookie field is a > must! VoIP switches? etc. etc. > > Please let's try to stick with the core goal of looking at IP (v4&v6) > packets and the "transport" protocol or next header. > > Anything else should be optional and left to other documents. The > IPFIX protocol just should support enough extensibility to allow this. > > My counter-proposal would be to drop the MPLS label requirement from > the requirements draft. > -- > Simon. > > On Mon, 30 Sep 2002 10:26:36 -0700, "Reinaldo Penno" > said: > > I think there are some important packet fields/scenarios that should > > be included in the requirements draft. > > > 1 - We discussed a long time ago to include the VPN-ID as one of the > > parameters exported. If Ipfix will be used in VPN environments for > > accouting, QoS, monitoring, etc, this field is a must. > > > 2 - There is mention to intrusion detection/security as one of the > > applications for IPfix, but there is no requirement to export NATed > > flows (before and after). > > > In a device that supports NAT (traditional, layer 7 interception, > > whatever) and it's going to use IPfix for security/intrusion > > detection, dealing with NAT is must. There is no way a intrusion > > detection device will find attacker/attacked properly without both > > sides of a NAT flow. Not to mention accouting, billing, etc. > > > I would propose to include both items on the requirements draft. > ------_=_NextPart_001_01C26C0F.AE89F542 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable RE: [ipfix-req] Requirements drafts, NATed flows and = VPNs

that's fair...but the premise was the statement in = the draft that IPfix could be used for intrusion detection. Given that = intrusion detection devices get all their data from port mirroring; the = limited subset of data provided by IPfix + issues such as the lack of = NAT visibility (which are directly related to the statement) makes me = wonder if we shouldn't take that statement out.

well, somebody could always argue that IPfix can do = some intrusion detection...but I really do not think it's a good fit. = The "some" argument could be expanded to include all kinds of = applications.

I could also agree we should drop the MPLS label = requirement. Otherwise one should ask why not export information about = other tunneling protocols such as L2TP/GRE/IPSEC information also. =

regards,

Reinaldo

> -----Original Message-----
> From: Simon Leinen [mailto:simon@limmat.switch.ch= ]
> Sent: Friday, October 04, 2002 4:51 PM
> To: Penno, Reinaldo [BL60:0430:EXCH]
> Cc: req
> Subject: Re: [ipfix-req] Requirements drafts, = NATed flows and VPNs
>
>
> [ipfix-eval-team removed from list of = recipients, since this seems
> purely a requirements issue.]
>
> I am opposed to this proposal because it opens = a can of worms if we
> start to try to accomodate arbitrary services = and middleboxes.
>
> What about those cool rate shapers that fiddle = with TCP window sizes?
> If they support IPFIX, the window-fiddle-factor = field ist a must!
> What about "cookie-cutting" layer 4-7 = switches? The cookie field is a
> must! VoIP switches? etc. etc.
>
> Please let's try to stick with the core goal of = looking at IP (v4&v6)
> packets and the "transport" protocol = or next header.
>
> Anything else should be optional and left to = other documents. The
> IPFIX protocol just should support enough = extensibility to allow this.
>
> My counter-proposal would be to drop the MPLS = label requirement from
> the requirements draft.
> --
> Simon.
>
> On Mon, 30 Sep 2002 10:26:36 -0700, = "Reinaldo Penno"
> <rpenno@nortelnetworks.com> said:
> > I think there are some important packet = fields/scenarios that should
> > be included in the requirements = draft.
>
> > 1 - We discussed a long time ago to = include the VPN-ID as one of the
> > parameters exported. If Ipfix will be used = in VPN environments for
> > accouting, QoS, monitoring, etc, this = field is a must.
>
> > 2 - There is mention to intrusion = detection/security as one of the
> > applications for IPfix, but there is no = requirement to export NATed
> > flows (before and after).
>
> > In a device that supports NAT = (traditional, layer 7 interception,
> > whatever) and it's going to use IPfix for = security/intrusion
> > detection, dealing with NAT is must. There = is no way a intrusion
> > detection device will find = attacker/attacked properly without both
> > sides of a NAT flow. Not to mention = accouting, billing, etc.
>
> > I would propose to include both items on = the requirements draft.
>

------_=_NextPart_001_01C26C0F.AE89F542-- -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri Oct 4 21:55:48 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA28151 for ; Fri, 4 Oct 2002 21:55:48 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17xdzs-0002y4-00 for ipfix-list@mil.doit.wisc.edu; Fri, 04 Oct 2002 20:44:48 -0500 Received: from h65s138a81n47.user.nortelnetworks.com ([47.81.138.65] helo=zsc3s004.nortelnetworks.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 17xdzq-0002xn-00; Fri, 04 Oct 2002 20:44:46 -0500 Received: from zrc2s0jx.nortelnetworks.com (zrc2s0jx.nortelnetworks.com [47.103.122.112]) by zsc3s004.nortelnetworks.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id g951iDY04217; Fri, 4 Oct 2002 18:44:13 -0700 (PDT) Received: from zsc4c000.us.nortel.com (zsc4c000.us.nortel.com [47.81.138.47]) by zrc2s0jx.nortelnetworks.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id g951iOq10466; Fri, 4 Oct 2002 20:44:25 -0500 (CDT) Received: by zsc4c000.us.nortel.com with Internet Mail Service (5.5.2653.19) id ; Fri, 4 Oct 2002 18:44:10 -0700 Message-ID: <7B802811BE77D51189910002A55CFD2C0426C5F2@zsc3c032.us.nortel.com> From: "Reinaldo Penno" To: Simon Leinen , ipfix-req@net.doit.wisc.edu Cc: ipfix-eval-team@net.doit.wisc.edu Subject: [ipfix-req] RE: Data Export Reliability Requirement [was: Re: IPFIX Requireme nt draft status - section 6.3.2] Date: Fri, 4 Oct 2002 18:44:10 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C26C10.B3AE8712" Precedence: bulk Sender: majordomo listserver This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C26C10.B3AE8712 Content-Type: text/plain; charset="iso-8859-1" > -----Original Message----- > From: Simon Leinen [mailto:simon@limmat.switch.ch] > Sent: Friday, October 04, 2002 3:30 PM > To: ipfix-req@net.doit.wisc.edu > Cc: Penno, Reinaldo [BL60:0430:EXCH]; > ipfix-eval-team@net.doit.wisc.edu > Subject: Data Export Reliability Requirement [was: Re: IPFIX > Requirement > draft status - section 6.3.2] > > > On Mon, 30 Sep 2002 03:00:03 -0700, "Reinaldo Penno" > said: > > I'm looking to understand/really clarify this requirement. It seems > > ambiguous...at least to me. I do not understand what is the > > meaning/purpose of "-->abscence<-- of reliability....MUST be > > indicated". > > Yes, that just sounds weird. (High) reliability should be a goal for > the exporting process, not something that can be readily measured, or > even less is "present" or "absent" at a given time during the > operation of an IPFIX implementation. > > reqs-06> Absence of reliability of the data transfer, for > example caused by > reqs-06> loss or reordering of packets containing flow > information, MUST be > reqs-06> indicated. > > Why not write something like this instead: > > "If some flow information data cannot be delivered to the collector > in a timely manner, the amount of missing flow information data > MUST be indicated to the collector. Possible reasons for failure > to deliver data include loss of packets on the network path, or > resource shortage at the exporter." But how the exporter would know which records were lost in the network unless we have (application layer) ACKs? > > I removed packet reordering as an example for reasons to lose data > because I don't think it's that relevant in practice. From > operational experience, I think overflows at the exporter are a much > more likely source of data loss. > > An indication of the amount of data lost is more useful than an > unspecific indication of "absence of reliability" and should be > relatively easy to provide. > > reqs-06> Please note that if an unreliable transport protocol is used, > reqs-06> reliability can be provided by higher layers. In such a case > reqs-06> only lack of overall reliability MUST be indicated. > > And end-to-end purist might argue that even if the underlying > transport protocol is used, reliability must be provided at higher > layers too. See the "application layer ACK" debate on the ipfix-req > list a few days ago. > > "Reliability can be supported by suitable transport protocols. In > any case only end-to-end flow information loss between exporter and > collector need be indicated." > > reqs-06> For example reordering could be dealt with by adding a > reqs-06> sequence number to each packet. when I read this whole section of the requirement draft I'm not sure what are "exactly" the requirements. It's vague, full of loopholes due to the "CANs" associated with specific situations... > > Or order may not be relevant since the exact arrival time of a piece > of flow information doesn't matter - the timestamps in the flow record > will provide sufficient sequence information for the collector. > -- > Simon. > ------_=_NextPart_001_01C26C10.B3AE8712 Content-Type: text/html; charset="iso-8859-1" RE: Data Export Reliability Requirement [was: Re: IPFIX Requirement draft status - section 6.3.2]

> -----Original Message-----
> From: Simon Leinen [mailto:simon@limmat.switch.ch]
> Sent: Friday, October 04, 2002 3:30 PM
> To: ipfix-req@net.doit.wisc.edu
> Cc: Penno, Reinaldo [BL60:0430:EXCH];
> ipfix-eval-team@net.doit.wisc.edu
> Subject: Data Export Reliability Requirement [was: Re: IPFIX
> Requirement
> draft status - section 6.3.2]
>
>
> On Mon, 30 Sep 2002 03:00:03 -0700, "Reinaldo Penno"
> <rpenno@nortelnetworks.com> said:
> > I'm looking to understand/really clarify this requirement. It seems
> > ambiguous...at least to me. I do not understand what is the
> > meaning/purpose of "-->abscence<-- of reliability....MUST be
> > indicated".
>
> Yes, that just sounds weird. (High) reliability should be a goal for
> the exporting process, not something that can be readily measured, or
> even less is "present" or "absent" at a given time during the
> operation of an IPFIX implementation.
>
> reqs-06> Absence of reliability of the data transfer, for
> example caused by
> reqs-06> loss or reordering of packets containing flow
> information, MUST be
> reqs-06> indicated.
>
> Why not write something like this instead:
>
>   "If some flow information data cannot be delivered to the collector
>    in a timely manner, the amount of missing flow information data
>    MUST be indicated to the collector. Possible reasons for failure
>    to deliver data include loss of packets on the network path, or
>    resource shortage at the exporter."

But how the exporter would know which records were lost in the network unless we have (application layer) ACKs?

>
> I removed packet reordering as an example for reasons to lose data
> because I don't think it's that relevant in practice. From
> operational experience, I think overflows at the exporter are a much
> more likely source of data loss.
>
> An indication of the amount of data lost is more useful than an
> unspecific indication of "absence of reliability" and should be
> relatively easy to provide.
>
> reqs-06> Please note that if an unreliable transport protocol is used,
> reqs-06> reliability can be provided by higher layers. In such a case
> reqs-06> only lack of overall reliability MUST be indicated.
>
> And end-to-end purist might argue that even if the underlying
> transport protocol is used, reliability must be provided at higher
> layers too. See the "application layer ACK" debate on the ipfix-req
> list a few days ago.
>
>   "Reliability can be supported by suitable transport protocols. In
>   any case only end-to-end flow information loss between exporter and
>   collector need be indicated."
>
> reqs-06> For example reordering could be dealt with by adding a
> reqs-06> sequence number to each packet.

when I read this whole section of the requirement draft I'm not sure what are "exactly" the requirements. It's vague, full of loopholes due to the "CANs" associated with specific situations...

>
> Or order may not be relevant since the exact arrival time of a piece
> of flow information doesn't matter - the timestamps in the flow record
> will provide sufficient sequence information for the collector.
> --
> Simon.
>

------_=_NextPart_001_01C26C10.B3AE8712-- -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Sat Oct 5 05:59:03 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id FAA12857 for ; Sat, 5 Oct 2002 05:59:02 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17xlSE-0006o4-00 for ipfix-list@mil.doit.wisc.edu; Sat, 05 Oct 2002 04:42:34 -0500 Received: from babar.switch.ch ([130.59.4.2]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 17xlSB-0006mt-00; Sat, 05 Oct 2002 04:42:31 -0500 Received: from babar.switch.ch (localhost [IPv6:::1]) by babar.switch.ch (8.12.2+Sun/8.12.2) with ESMTP id g959fxGU018601; Sat, 5 Oct 2002 11:41:59 +0200 (MEST) Received: (from leinen@localhost) by babar.switch.ch (8.12.2+Sun/8.12.2/Submit) id g959fwiI018598; Sat, 5 Oct 2002 11:41:58 +0200 (MEST) X-Authentication-Warning: babar.switch.ch: leinen set sender to simon@limmat.switch.ch using -f To: "Reinaldo Penno" Cc: ipfix-req@net.doit.wisc.edu, ipfix-eval-team@net.doit.wisc.edu Subject: [ipfix-req] Re: Data Export Reliability Requirement [was: Re: IPFIX Requireme nt draft status - section 6.3.2] References: <7B802811BE77D51189910002A55CFD2C0426C5F2@zsc3c032.us.nortel.com> X-Face: 1Nk*r=:$IBBb8|TyRB'2WSY6u:BzMO7N)#id#-4_}MsU5?vTI?dez|JiutW4sKBLjp.l7,F 7QOld^hORRtpCUj)!cP]gtK_SyK5FW(+o"!or:v^C^]OxX^3+IPd\z,@ttmwYVO7l`6OXXYR` From: Simon Leinen In-Reply-To: <7B802811BE77D51189910002A55CFD2C0426C5F2@zsc3c032.us.nortel.com> Date: 05 Oct 2002 11:41:58 +0200 Message-ID: Lines: 41 User-Agent: Gnus/5.09 (Gnus v5.9.0) Emacs/21.2.90 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Precedence: bulk Sender: majordomo listserver On Fri, 4 Oct 2002 18:44:10 -0700 , "Reinaldo Penno" said: >> From: Simon Leinen [mailto:simon@limmat.switch.ch] >> "If some flow information data cannot be delivered to the collector >> in a timely manner, the amount of missing flow information data >> MUST be indicated to the collector. Possible reasons for failure >> to deliver data include loss of packets on the network path, or >> resource shortage at the exporter." > But how the exporter would know which records were lost in the > network unless we have (application layer) ACKs? I see - my "MUST be indicated" is confusing too. The exporter wouldn't need to know, but the collector must be able to determine that something was lost, and "how much", where I intentionally leave open the accuracy - the collector won't be able to reconstruct everything anyway. So what about this: "If some flow information data cannot be delivered to the collector in a timely manner, the collector MUST be able to estimate the amount of data missing. Possible reasons for failure to deliver data include loss of packets on the network path, or resource shortage at the exporter." As an example, the flow export mechanism I use sends me packets of flow records, and each packet has a flow record sequence number. So I can determine exactly how many flows I have missed. This information is important for diagnosis of situations where there is information loss (some, but not all of this information loss would have been prevented by using a reliable transport protocol). For my particular applications, I'm mostly interested in the byte amounts of metered traffic. So if a flow export protocol had sequence numbers representing the cumulative total bytes metered, that would be very useful to me - I really would like to know what percentage of traffic could not be accounted for. (In addition to that, I would still want to know how many flow records I missed.) -- Simon Leinen simon@babar.switch.ch SWITCH http://www.switch.ch/misc/leinen/ Computers hate being anthropomorphized. -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Sat Oct 5 12:58:12 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA19118 for ; Sat, 5 Oct 2002 12:58:11 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17xs7H-0002tQ-00 for ipfix-list@mil.doit.wisc.edu; Sat, 05 Oct 2002 11:49:23 -0500 Received: from mailhub.xacct.com ([204.253.100.25] helo=xacct.com) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 17xs7F-0002tJ-00 for ipfix-eval@net.doit.wisc.edu; Sat, 05 Oct 2002 11:49:21 -0500 Received: (qmail 16969 invoked from network); 5 Oct 2002 16:49:15 -0000 Received: from usmail.xacct.com (204.253.100.12) by mailhub.xacct.com with SMTP; 5 Oct 2002 16:49:15 -0000 Received: from peter ([192.168.254.163]) by usmail.xacct.com (8.11.2/8.11.2) with SMTP id g95GqMs26977; Sat, 5 Oct 2002 09:52:23 -0700 From: "Peter Ludemann" To: , "'Michael MacFaden'" , Subject: RE: [ipfix-eval] Discussion of Candidate Protocols Date: Sat, 5 Oct 2002 09:49:16 -0700 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 In-Reply-To: <5C8959A16A71B449AE793CF52FBBED6607A45C@ptah.newyork.qosient.com> Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Carter Bullard wrote Thursday, October 03, 2002 5:08 PM: > I don't see that LFAP is a solution to the flow record transport > problems that I worry about, which is the efficient, reliable > and secure transport of any vendors flow records, and I'm > particularly concerned with at least one of the basic architectural > positions that LFAP takes, the protocol complexity and > its Information Model. As a general comment, could we leave out the Information Model in these discussion, unless we're discussing data types. I thought that we were just discussing the protocols and not the semantics of the data (i.e., the form, not the content). > The biggest issue with the architecture is timestamps. My > understanding reading the lfap-eval, is that timestamps > relate to the flow cache rather than the wire-line timestamps of > the packet stream that is being monitored. ... Anything less > would be networking 'malpractice', if there ever could be such > a thing. This is an example of a statement that does not belong in this discussion, but does belong in a discussion on the data model, information model, or architecture. IMHO. ... > With regard to the Information Model, I sense that I would > put my entire record in a single vendor specific IE and leave it > at that. While a drastic statement, its not too far from the > truth. This is true of any protocol that supports a "blob" data type, although it's arguably violating the spirit of the specification. ... > Best time granularity of 10's of milliseconds (1/100th of a sec) > is not good, uSec minimum, nanoSec preferable. Agreed ... for our PacketSight probe, we use nanoseconds. I should add that the CRANE specification is lacking a few useful time types also. We hadn't decided on the full list, so left them out of the draft. (e.g., a nanosecond time that uses two 32-bit values [seconds, nanoseconds] instead of a 64-bit value). Adding them to CRANE is trivial. ... > 64 bit counters for packets and bytes in every record, when over > 90% of all flows have less than 256 packets and 8K of bytes in the > entire flow? We should have 1, 2, 4 and 8 byte counter IEs as > needed, don't you think? This is why we provided the full range of 8, 16, 32, 64 bit counters, both signed and unsigned. (We were also debating whether these would be described as gauges or counters but decided that that would be left to the semantics of the fields; this decision might be revisited in the future.) Also, we decided to output in host byte order, not in network byte order, so that little-endian machines wouldn't pay a performance penalty (the assumption being that the receiver has more CPU and also has fewer realtime constraints). - peter -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Sat Oct 5 14:38:56 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA20942 for ; Sat, 5 Oct 2002 14:38:55 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17xthc-0005FX-00 for ipfix-list@mil.doit.wisc.edu; Sat, 05 Oct 2002 13:31:00 -0500 Received: from mailhub.xacct.com ([204.253.100.25] helo=xacct.com) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 17xthZ-0005FQ-00 for ipfix-req@net.doit.wisc.edu; Sat, 05 Oct 2002 13:30:57 -0500 Received: (qmail 19386 invoked from network); 5 Oct 2002 18:30:51 -0000 Received: from usmail.xacct.com (204.253.100.12) by mailhub.xacct.com with SMTP; 5 Oct 2002 18:30:51 -0000 Received: from peter ([192.168.254.163]) by usmail.xacct.com (8.11.2/8.11.2) with SMTP id g95IY1s03843; Sat, 5 Oct 2002 11:34:01 -0700 From: "Peter Ludemann" To: Cc: Subject: [ipfix-req] RE: Data Export Reliability Requirement [was: Re: IPFIX Requirement draft status - section 6.3.2] Date: Sat, 5 Oct 2002 11:30:55 -0700 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 In-Reply-To: <7B802811BE77D51189910002A55CFD2C0426C5F2@zsc3c032.us.nortel.com> Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit This note is in two parts: comments on reliability requirements as for the export protocol and comments on how data loss is reported. I've added snippets on how we do this in CRANE and suggest that the other protocol experts describe how they support these functionalities. At the protocol level, the minimal requirement is that there be some kind of ACK from the collector to the exporter (as Reinaldo Penno noted). We also need some kind of unique key that allows de-duplication at the collector. [In CRANE, we use the exporter "boot time" + 32-bit record sequence number; the de-duplicator takes multiple data streams from the CRANE collectors and de-duplicates them.] (I assume that everyone agrees that de-duplication is required at the collector.) For a high-availability solution, we should have the ability to send to multiple collectors and for the exporter to be able to switch from one collector to another. The timing issues for fail-over are outside the protocol; but the capabilities must be part of it. [In our CRANE implementation, we use a timeout on ACKs for switching to the alternate receiver. We intend to improve this in future with some kind of load balancing by observing the rate at which data is being ACKed. It turns out to be a bit tricky to get this working well for both high-speed transmission (3000+ rec/sec) and low-speed (1 rec/sec or slower). We only send an ACK only when the data has been persisted - this behavior is outside the scope of the protocol itself and therefore should be a SHOULD, not a MUST in the standard.] However, there is still the possibility of data loss: the exporter may have data areas fill up before an ACK arrives. The exporter must therefore throw away some data. The exporter knows AT LEAST how many records were received (it is possible that more were received, but the ACKs were lost). So, if the exporter is forced to throw away data, it can report about what it threw away. (There are other causes of data loss than running of memory: such as running out of CPU, or acting as a proxy and losing data; these can all be covered by a similar mechanism.) There will inevitably be some information loss in reporting data loss. The minimal information that the exporter needs to keep is: number of records not ACKed, totals for the metrics that were not ACKed. The information that are kept about data loss are application-dependent -- which totals are important? is some high-level aggregation on the exporter possible? etc. For a specific data model (e.g., the standard data exported by Cisco routers using NetFlow v8), a separate specification could state exactly what must be reported for data loss. [Note: NetFlow v8 is only being used as an example of data *content* - because the protocol lacks reliability, the collector can only report statistics such as number of records that were lost, detected by sequence number gaps.] The separate specification could also state the rules for throwing away data: e.g., throw away oldest, newest, smallest, etc. As an example of data loss reporting, suppose that we are exporting a single metric and that the statistics on the exporter are: # flow records sent (not counting retransmitted) 1000 # flow records ACKed 900 sum of metric all records 10000 sum of metric in all ACKed records 8000 From which we derive the following: average metric per record 10 average metric per ACKed record 8.889 average metric per un-ACKed record 20 And on the receiver, we have: # flow records received (de-duplicated) 950 sum of metric in all received records 9200 average metric per received record 9.684 The receiver notes that it did not receive 50 records; it can estimate the sum of metric in these as 50 * 20 = 1000, for an estimated total of the metric of 10200. (Something fancier could be done by using variance calculations, combined with other metrics.) [In CRANE, we provide two mechanisms for reporting loss: - the exporter can define an additional set of fields that it exports periodically with data loss values; - a status request from the collector, also reported as a set of fields. The content of these records is defined by the exporter, using the general template mechanism.] So, to sum up: - require application-level ACKs in the protocol (SHOULD be after data are persisted) - require enable de-duplication of received records - require mechanism for reporting data loss in the exporter - the details of this mechanism are outside the scope of the protocol except that the protocol must be general enough to allow them - individual information models (which use the flow export protocol) SHOULD describe data loss reporting - peter -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Sat Oct 5 14:41:19 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA21096 for ; Sat, 5 Oct 2002 14:41:19 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17xtlZ-0005PC-00 for ipfix-list@mil.doit.wisc.edu; Sat, 05 Oct 2002 13:35:05 -0500 Received: from mailhub.xacct.com ([204.253.100.25] helo=xacct.com) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 17xtlX-0005P6-00 for ipfix-eval@net.doit.wisc.edu; Sat, 05 Oct 2002 13:35:03 -0500 Received: (qmail 19492 invoked from network); 5 Oct 2002 18:34:57 -0000 Received: from usmail.xacct.com (204.253.100.12) by mailhub.xacct.com with SMTP; 5 Oct 2002 18:34:57 -0000 Received: from peter ([192.168.254.163]) by usmail.xacct.com (8.11.2/8.11.2) with SMTP id g95Ic6s03875; Sat, 5 Oct 2002 11:38:07 -0700 From: "Peter Ludemann" To: "Harrington, David" , Subject: RE: [ipfix-eval] Discussion of Candidate Protocols Date: Sat, 5 Oct 2002 11:35:01 -0700 Message-ID: MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_0016_01C26C63.3DDD4A70" X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 In-Reply-To: <6D745637A7E0F94DA070743C55CDA9BA075831@NHROCMBX1.ets.enterasys.com> Precedence: bulk Sender: majordomo listserver This is a multi-part message in MIME format. ------=_NextPart_000_0016_01C26C63.3DDD4A70 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit RE: [ipfix-eval] Discussion of Candidate ProtocolsDavid Harrington wrote Friday, October 04, 2002 2:39 PM : 4) Xacct sought out Enterasys as a potential partner in 2000(?). I did the technology analysis on CRANE for the due diligence. My largest concern was the proprietary nature of the protocol. Just like IPDR, it was developed in a members-only agreement and was denied the opportunity for wide-spread industry review. This was done to capture the intellectaul property rights before making it public. Another major concern was the push for standardizing "buckets for transporting proprietary data in"; I don't believe that transporting a lot of proprietary data models really helps standardization and interoperability. CRANE has pros and cons. The CRANE spec will not be "proprietary" if adopted, so do you have any other significant objections to it (there have been some improvements since 2000)? Even if it isn't adopted, I still appreciate criticism, so that I can improve it as a product. - peter ------=_NextPart_000_0016_01C26C63.3DDD4A70 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable RE: [ipfix-eval] Discussion of Candidate = Protocols

David Harrington wrote Friday, October 04, 2002 = 2:39=20 PM :

4) Xacct sought out Enterasys as a potential partner = in=20 2000(?). I did the technology analysis on CRANE for the due diligence. = My=20 largest concern was the proprietary nature of the protocol. Just like = IPDR, it=20 was developed in a members-only agreement and was denied the = opportunity for=20 wide-spread industry review. This was done to capture the intellectaul = property rights before making it public. Another major concern was the = push=20 for standardizing "buckets for transporting proprietary data in"; I = don't=20 believe that transporting a lot of proprietary data models really = helps=20 standardization and interoperability. CRANE has pros and cons.

<= /FONT>

The CRANE=20 spec will not be "proprietary" if adopted, so do=20 you have any other significant objections to it (there have been some=20 improvements since 2000)? Even if it isn't adopted, I still appreciate=20 criticism, so that I can improve it as a = product.

-=20 peter

------=_NextPart_000_0016_01C26C63.3DDD4A70-- -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Sat Oct 5 15:02:40 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA21565 for ; Sat, 5 Oct 2002 15:02:40 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17xu5V-0005ts-00 for ipfix-list@mil.doit.wisc.edu; Sat, 05 Oct 2002 13:55:41 -0500 Received: from mailhub.xacct.com ([204.253.100.25] helo=xacct.com) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 17xu5T-0005tm-00 for ipfix-eval@net.doit.wisc.edu; Sat, 05 Oct 2002 13:55:40 -0500 Received: (qmail 19683 invoked from network); 5 Oct 2002 18:55:33 -0000 Received: from usmail.xacct.com (204.253.100.12) by mailhub.xacct.com with SMTP; 5 Oct 2002 18:55:33 -0000 Received: from peter ([192.168.254.163]) by usmail.xacct.com (8.11.2/8.11.2) with SMTP id g95Iwis04614 for ; Sat, 5 Oct 2002 11:58:44 -0700 From: "Peter Ludemann" To: Subject: RE: [ipfix-eval] Discussion of Candidate Protocols Date: Sat, 5 Oct 2002 11:55:38 -0700 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 In-Reply-To: <3D9DC88F.9060507@cup.hp.com> Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Jeff Meyer wrote Friday, October 04, 2002 9:58 AM: > There seems to be some confusion over the separation > between a formal, machine readable description of > the data model (as IPDR defines), and the requirements put > on an implementation (especially in an IPFIX Middlebox). YES ... so, can we all agree that data model issues are not part of this discussion? [snip] > However, the statement that IP flow is somehow unique > in its data requirements and as such a more generalized > "data mover" is somehow problematic, is just plain wrong. YES! > As a mediation product vendor I see lots of devices > with lots of vendor specific protocols. And there > are several which have the same protocol requirements > as IPFIX. Just different data models. YES! The data models need to also be aligned; but just dealing with all the protocols is a headache, especially with the unreliable protocols such as NetFlow and SNMP (RADIUS is an "almost reliable" protocol and therefore isn't quite so difficult). [snip] > ... If IPFIX wants to say that only > fixed with data values may be sent via the IPFIX > protocol, then I'd say, maybe you could optimize > beyond IPDR for that. CRANE imposes only the header overhead on each record. The data are also sent in the "native" form for the exporter (I'll comment on XDR later). However, I think that a numeric-only protocol is not sufficient. As soon as metrics are collected at the application level, people want attributes, such as URL path, response status, payload type, etc. which are either difficult or impossible to encode as number. > A minimal implementation of IPDR on a middlebox > would NOT need to have any XML knowledge whatsoever. > As a producer, the system could merely hardcode the > information model produced (or make it configurable). Likewise for CRANE, the box could simply reject all requests to limit the fields that are exported, by hardcoding the templates. However, processing the templates is somewhat easier in CRANE because of their fixed layout, not requiring an XML engine (yes, I know that an XML engine can be quite compact; but it's not common on network elements, at least not now). > The encoding itself follows XDR format, which has > worked alright for NFS and RPC, and is the precursor > to the encoding for DCE and CORBA encoding. We looked at XDR and rejected it because it can be a little "heavy" for CPU-constrained devices (e.g., CSU/DSU, "cable routers", etc.). We prefer exporting in the "native" format (big- or little-endian) with template flags to allow decoding at the receiver. Of course, we don't prevent export in XDR (big-endian) form. - peter -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Sat Oct 5 15:14:41 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA21668 for ; Sat, 5 Oct 2002 15:14:41 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17xuHP-0006EF-00 for ipfix-list@mil.doit.wisc.edu; Sat, 05 Oct 2002 14:07:59 -0500 Received: from mailhub.xacct.com ([204.253.100.25] helo=xacct.com) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 17xuHN-0006EA-00 for ipfix-eval@net.doit.wisc.edu; Sat, 05 Oct 2002 14:07:57 -0500 Received: (qmail 19817 invoked from network); 5 Oct 2002 19:07:51 -0000 Received: from usmail.xacct.com (204.253.100.12) by mailhub.xacct.com with SMTP; 5 Oct 2002 19:07:51 -0000 Received: from peter ([192.168.254.163]) by usmail.xacct.com (8.11.2/8.11.2) with SMTP id g95JB0s04689; Sat, 5 Oct 2002 12:11:01 -0700 From: "Peter Ludemann" To: Cc: "Stas Khirman" , Subject: RE: [ipfix-eval] Discussion of Candidate Protocols Date: Sat, 5 Oct 2002 12:07:55 -0700 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 In-Reply-To: <580E532D9F7A9B4BAE8A130848E0DDA702083A03@franklin.narus.com> Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Stas Khirman wrote Wednesday, October 02, 2002 8:34 PM: > Hm, I do not think that memcopy operation could have a significant impact. > Let's assume we are dealing with 10,000 records per second 100 bytes each. > It is mean that in worst case we have to copy 1Mbyte - 250,000 copy > operations per second on 32 bit processor. I'm sure it will take only a > fraction of one percent on any normal CPU, am I wrong? BTW, > majority of TCP > stack implementations are doing few memory copies anyway. And some TCP implementations are zero-copy. All I can tell you is that one of our partners had memory bandwidth issues getting data off the collection board into main memory for transmission; and that minimizing memory-to-memory operations was important. calato@riverstonenet.com wrote Wednesday, October 02, 2002 5:54 AM: > Having done a fair amount of tuning on a collector I find > it interesting that your performance hinged on a memory > copy. In my experince performance was limited by how fast the > disk writes are. I agree on the collector performance tends to be constrained by I/O. But collectors are often bigger boxes (also doing other things with the collected data, such as aggregating, correlating, filtering, etc.) and have less stringent real-time requirements. Usually reliable sources have told me that certain vendors' switches quietly stop collecting flow statistics when CPU exceeds 70% or so. These same sources have therefore installed probes to collect the data. So, minimising CPU usage on the network element is important! - peter -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon Oct 7 06:58:02 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id GAA12978 for ; Mon, 7 Oct 2002 06:58:02 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17yVQL-0000kC-00 for ipfix-list@mil.doit.wisc.edu; Mon, 07 Oct 2002 05:47:41 -0500 Received: from babar.switch.ch ([130.59.4.2]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 17yVQJ-0000jZ-00 for ipfix-eval@net.doit.wisc.edu; Mon, 07 Oct 2002 05:47:39 -0500 Received: from babar.switch.ch (localhost [IPv6:::1]) by babar.switch.ch (8.12.2+Sun/8.12.2) with ESMTP id g97Al7GU025376; Mon, 7 Oct 2002 12:47:07 +0200 (MEST) Received: (from leinen@localhost) by babar.switch.ch (8.12.2+Sun/8.12.2/Submit) id g97Al4cH025373; Mon, 7 Oct 2002 12:47:04 +0200 (MEST) X-Authentication-Warning: babar.switch.ch: leinen set sender to simon@limmat.switch.ch using -f To: calato@riverstonenet.com Cc: Peter Ludemann , Michael MacFaden , ipfix-eval@net.doit.wisc.edu Subject: Re: [ipfix-eval] Discussion of Candidate Protocols References: <3D9AEC51.BA498740@riverstonenet.com> X-Face: 1Nk*r=:$IBBb8|TyRB'2WSY6u:BzMO7N)#id#-4_}MsU5?vTI?dez|JiutW4sKBLjp.l7,F 7QOld^hORRtpCUj)!cP]gtK_SyK5FW(+o"!or:v^C^]OxX^3+IPd\z,@ttmwYVO7l`6OXXYR` From: Simon Leinen In-Reply-To: <3D9AEC51.BA498740@riverstonenet.com> Date: 07 Oct 2002 12:47:04 +0200 Message-ID: Lines: 28 User-Agent: Gnus/5.09 (Gnus v5.9.0) Emacs/21.2.90 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Precedence: bulk Sender: majordomo listserver On Wed, 02 Oct 2002 08:53:37 -0400, calato@riverstonenet.com said: > Peter Ludemann wrote: >> However, we have encountered some situations where saving a single >> copy operation per output recordcan make all the difference between >> being able to deliver the data and not. It's highly unlikely that >> an application will keep data in AVP form, so AVP requires copies. >> If the protocol packs the data into something that a C programmer >> might do with a "struct", then memory-to-memory copying can be >> avoided. > Having done a fair amount of tuning on a collector I find it > interesting that your performance hinged on a memory copy. In > my experince performance was limited by how fast the disk > writes are. Some collectors will write large amounts of data to disk - if this is the main bottleneck I'd assume that they use the accounting data without too heavy processing (or they need a better approach to disk I/O, such as direct I/O, scatter/gather interfaces, faster disks etc.). Unnecessary copying should be avoided mainly because memory bandwidth doesn't seem to grow as fast as other performance dimensions. Personally I think this is more of an issue than for example byte-order optimizations. Regards, -- Simon. -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon Oct 7 10:19:14 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA20197 for ; Mon, 7 Oct 2002 10:19:14 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17yYbD-0001N3-00 for ipfix-list@mil.doit.wisc.edu; Mon, 07 Oct 2002 09:11:07 -0500 Received: from [64.95.122.60] (helo=RS-SC-EXC4.rs.riverstonenet.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 17yYbA-0001La-00; Mon, 07 Oct 2002 09:11:04 -0500 Received: from riverstonenet.com ([134.141.180.78]) by RS-SC-EXC4.rs.riverstonenet.com with Microsoft SMTPSVC(5.0.2195.4905); Mon, 7 Oct 2002 07:10:31 -0700 Message-ID: <3DA19565.5E831D0D@riverstonenet.com> Date: Mon, 07 Oct 2002 10:08:37 -0400 From: calato@riverstonenet.com X-Mailer: Mozilla 4.75C-CCK-MCD NSCPCD475 [en] (X11; U; SunOS 5.6 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: Simon Leinen CC: ipfix-req@net.doit.wisc.edu, Reinaldo Penno , ipfix-eval-team@net.doit.wisc.edu Subject: Re: [ipfix-req] Data Export Reliability Requirement[was: Re: IPFIX Requirement draft status - section 6.3.2] References: <7B802811BE77D51189910002A55CFD2C04119EF5@zsc3c032.us.nortel.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 07 Oct 2002 14:10:31.0785 (UTC) FILETIME=[4BF52D90:01C26E0B] Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Simon Leinen wrote: > > On Mon, 30 Sep 2002 03:00:03 -0700, "Reinaldo Penno" said: > > I'm looking to understand/really clarify this requirement. It seems > > ambiguous...at least to me. I do not understand what is the > > meaning/purpose of "-->abscence<-- of reliability....MUST be > > indicated". > > Yes, that just sounds weird. (High) reliability should be a goal for > the exporting process, not something that can be readily measured, or > even less is "present" or "absent" at a given time during the > operation of an IPFIX implementation. > > reqs-06> Absence of reliability of the data transfer, for example caused by > reqs-06> loss or reordering of packets containing flow information, MUST be > reqs-06> indicated. > > Why not write something like this instead: > > "If some flow information data cannot be delivered to the collector > in a timely manner, the amount of missing flow information data > MUST be indicated to the collector. Possible reasons for failure > to deliver data include loss of packets on the network path, or > resource shortage at the exporter." I like this. One question, why did you add "in a timely manner". What not just leave it at "cannot be delivered". Also, at some point need a clear definition of what "missing flow information data" really means. Is it a byte and packet count, is it IPFIX messages? Both? > > I removed packet reordering as an example for reasons to lose data > because I don't think it's that relevant in practice. From > operational experience, I think overflows at the exporter are a much > more likely source of data loss. Too early to make this call. It could be that packet reordering does cause a loss of data. But that analysis can't happen until we have a protocol. > > An indication of the amount of data lost is more useful than an > unspecific indication of "absence of reliability" and should be > relatively easy to provide. Agreed. > > reqs-06> Please note that if an unreliable transport protocol is used, > reqs-06> reliability can be provided by higher layers. In such a case > reqs-06> only lack of overall reliability MUST be indicated. > > And end-to-end purist might argue that even if the underlying > transport protocol is used, reliability must be provided at higher > layers too. See the "application layer ACK" debate on the ipfix-req > list a few days ago. > > "Reliability can be supported by suitable transport protocols. In > any case only end-to-end flow information loss between exporter and > collector need be indicated." > > reqs-06> For example reordering could be dealt with by adding a > reqs-06> sequence number to each packet. > > Or order may not be relevant since the exact arrival time of a piece > of flow information doesn't matter - the timestamps in the flow record > will provide sufficient sequence information for the collector. The thing I don't like about going down this road is the potential storage and processing requirements needed at the collector. Also, how do I know that when processing a timestamp at 10:00 that another one is coming for 9:59? Paul > -- > Simon. > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon Oct 7 10:32:50 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA20920 for ; Mon, 7 Oct 2002 10:32:50 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17yYp8-0001yb-00 for ipfix-list@mil.doit.wisc.edu; Mon, 07 Oct 2002 09:25:30 -0500 Received: from [64.95.122.60] (helo=RS-SC-EXC4.rs.riverstonenet.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 17yYp6-0001x2-00; Mon, 07 Oct 2002 09:25:28 -0500 Received: from riverstonenet.com ([134.141.180.78]) by RS-SC-EXC4.rs.riverstonenet.com with Microsoft SMTPSVC(5.0.2195.4905); Mon, 7 Oct 2002 07:24:56 -0700 Message-ID: <3DA198C6.CC26BCCD@riverstonenet.com> Date: Mon, 07 Oct 2002 10:23:02 -0400 From: calato@riverstonenet.com X-Mailer: Mozilla 4.75C-CCK-MCD NSCPCD475 [en] (X11; U; SunOS 5.6 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: Simon Leinen CC: Reinaldo Penno , ipfix-req@net.doit.wisc.edu, ipfix-eval-team@net.doit.wisc.edu Subject: Re: [ipfix-req] Re: Data Export Reliability Requirement [was: Re: IPFIX Requireme nt draft status - section 6.3.2] References: <7B802811BE77D51189910002A55CFD2C0426C5F2@zsc3c032.us.nortel.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 07 Oct 2002 14:24:57.0015 (UTC) FILETIME=[4FACCC70:01C26E0D] Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Simon Leinen wrote: > > On Fri, 4 Oct 2002 18:44:10 -0700 , "Reinaldo Penno" said: > >> From: Simon Leinen [mailto:simon@limmat.switch.ch] > >> "If some flow information data cannot be delivered to the collector > >> in a timely manner, the amount of missing flow information data > >> MUST be indicated to the collector. Possible reasons for failure > >> to deliver data include loss of packets on the network path, or > >> resource shortage at the exporter." > > > But how the exporter would know which records were lost in the > > network unless we have (application layer) ACKs? > > I see - my "MUST be indicated" is confusing too. The exporter > wouldn't need to know, but the collector must be able to determine > that something was lost, and "how much", where I intentionally leave > open the accuracy - the collector won't be able to reconstruct > everything anyway. So what about this: > > "If some flow information data cannot be delivered to the collector > in a timely manner, the collector MUST be able to estimate the > amount of data missing. Possible reasons for failure to deliver > data include loss of packets on the network path, or resource > shortage at the exporter." Not sure I like that. How about when the exporter starts talking to another collector. How about in overload mode when the collector is just dropping flows. I think the exporter is in the best position to estimate how much data was lost. Once more of the protocol is nailed down we can better define how to measure loss. > > As an example, the flow export mechanism I use sends me packets of > flow records, and each packet has a flow record sequence number. So I > can determine exactly how many flows I have missed. This information > is important for diagnosis of situations where there is information > loss (some, but not all of this information loss would have been > prevented by using a reliable transport protocol). > > For my particular applications, I'm mostly interested in the byte > amounts of metered traffic. So if a flow export protocol had sequence > numbers representing the cumulative total bytes metered, that would be > very useful to me - I really would like to know what percentage of > traffic could not be accounted for. (In addition to that, I would > still want to know how many flow records I missed.) > -- > Simon Leinen simon@babar.switch.ch > SWITCH http://www.switch.ch/misc/leinen/ > > Computers hate being anthropomorphized. > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon Oct 7 10:33:02 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA20971 for ; Mon, 7 Oct 2002 10:33:01 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17yYm1-0001qj-00 for ipfix-list@mil.doit.wisc.edu; Mon, 07 Oct 2002 09:22:17 -0500 Received: from odd-brew.cisco.com ([144.254.15.119] helo=strange-brew.cisco.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 17yYlz-0001pD-00; Mon, 07 Oct 2002 09:22:15 -0500 Received: from cisco.com (bclaise-isdn-home3.cisco.com [10.49.4.220]) by strange-brew.cisco.com (8.11.6+Sun/8.8.8) with ESMTP id g97ELg721863; Mon, 7 Oct 2002 16:21:42 +0200 (CEST) Message-ID: <3DA19874.7040702@cisco.com> Date: Mon, 07 Oct 2002 16:21:40 +0200 From: Benoit Claise User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.1) Gecko/20020815 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Simon Leinen CC: ipfix-req@net.doit.wisc.edu, Reinaldo Penno , ipfix-eval-team@net.doit.wisc.edu Subject: Re: [ipfix-req] Data Export Reliability Requirement [was: Re: IPFIX Requirement draft status - section 6.3.2] References: <7B802811BE77D51189910002A55CFD2C04119EF5@zsc3c032.us.nortel.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Reinaldo, Paul, Peter and Simon, [I've been trying to compile all ideas in order to reply] 6.3.2. Reliability Absence of reliability of the data transfer, for example caused by loss or reordering of packets containing flow information, MUST be indicated. Please note that if an unreliable transport protocol is used, reliability can be provided by higher layers. In such a case only lack of overall reliability MUST be indicated. For example reordering could be dealt with by adding a sequence number to each packet. As Peter pointed out, the idea behing this requirement is that: the connection between the exporting process and collecting process could be broken for a certain time; slow ACK, no routing, etc... or the router could have some problems: queue full So we must at least indicate to the collecting process that some data where lost somewhere (on the exporting process or along the path) Paul divided into a few categories: 1. Congestion awareness. 2. Message ordering 3. Transport reliability 4. Application layer reliability 5. Tracking data loss (statistics) 1. Taken into consideration by the section 6.3.1 "Congestion Awareness" 2. Paul is right: depends on the protocol. Not sure we need a special section for that 3. The transport protocol reliability would be a completely new requirement, no even part of the charter. Unless you only mean by this: when the transport connection breaks. If this is the case, then see #4. 4. If you want ACK at the application level, this would be a completely new requirement. Actually the goal of section 6.3.2 "Reliability" was to take care of that issue, without requiring ACK at the application level (which I personally think is not possible due to the amount of traffic exported). Simon's comment on removing the "message ordering" from section 6.3.2 makes sense (see #2 where the message ordering depends on the transport protocol). I vote for Simon's proposal. "If some flow information data cannot be delivered to the collector in a timely manner, the amount of missing flow information data MUST be indicated to the collector. Possible reasons for failure to deliver data include loss of packets on the network path, or resource shortage at the exporter." 5. I think this is taken into consideration by the text in #4. What do you think? Regards, Benoit >On Mon, 30 Sep 2002 03:00:03 -0700, "Reinaldo Penno" said: > > >>I'm looking to understand/really clarify this requirement. It seems >>ambiguous...at least to me. I do not understand what is the >>meaning/purpose of "-->abscence<-- of reliability....MUST be >>indicated". >> >> > >Yes, that just sounds weird. (High) reliability should be a goal for >the exporting process, not something that can be readily measured, or >even less is "present" or "absent" at a given time during the >operation of an IPFIX implementation. > >reqs-06> Absence of reliability of the data transfer, for example caused by >reqs-06> loss or reordering of packets containing flow information, MUST be >reqs-06> indicated. > >Why not write something like this instead: > > "If some flow information data cannot be delivered to the collector > in a timely manner, the amount of missing flow information data > MUST be indicated to the collector. Possible reasons for failure > to deliver data include loss of packets on the network path, or > resource shortage at the exporter." > >I removed packet reordering as an example for reasons to lose data >because I don't think it's that relevant in practice. From >operational experience, I think overflows at the exporter are a much >more likely source of data loss. > >An indication of the amount of data lost is more useful than an >unspecific indication of "absence of reliability" and should be >relatively easy to provide. > >reqs-06> Please note that if an unreliable transport protocol is used, >reqs-06> reliability can be provided by higher layers. In such a case >reqs-06> only lack of overall reliability MUST be indicated. > >And end-to-end purist might argue that even if the underlying >transport protocol is used, reliability must be provided at higher >layers too. See the "application layer ACK" debate on the ipfix-req >list a few days ago. > > "Reliability can be supported by suitable transport protocols. In > any case only end-to-end flow information loss between exporter and > collector need be indicated." > >reqs-06> For example reordering could be dealt with by adding a >reqs-06> sequence number to each packet. > >Or order may not be relevant since the exact arrival time of a piece >of flow information doesn't matter - the timestamps in the flow record >will provide sufficient sequence information for the collector. > > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon Oct 7 10:47:10 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA21751 for ; Mon, 7 Oct 2002 10:47:10 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17yYyf-0002GU-00 for ipfix-list@mil.doit.wisc.edu; Mon, 07 Oct 2002 09:35:21 -0500 Received: from odd-brew.cisco.com ([144.254.15.119] helo=strange-brew.cisco.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 17yYyd-0002FT-00; Mon, 07 Oct 2002 09:35:20 -0500 Received: from cisco.com (bclaise-isdn-home3.cisco.com [10.49.4.220]) by strange-brew.cisco.com (8.11.6+Sun/8.8.8) with ESMTP id g97EYg706676; Mon, 7 Oct 2002 16:34:42 +0200 (CEST) Message-ID: <3DA19B80.80600@cisco.com> Date: Mon, 07 Oct 2002 16:34:40 +0200 From: Benoit Claise User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.1) Gecko/20020815 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Simon Leinen CC: Reinaldo Penno , ipfix-req@net.doit.wisc.edu, ipfix-eval-team@net.doit.wisc.edu Subject: Re: [ipfix-req] Re: Data Export Reliability Requirement [was: Re: IPFIX Requireme nt draft status - section 6.3.2] References: <7B802811BE77D51189910002A55CFD2C0426C5F2@zsc3c032.us.nortel.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit All, [In my previous email, I thought I took the latest email in the thread, I was wrong. Sorry!] >On Fri, 4 Oct 2002 18:44:10 -0700 , "Reinaldo Penno" said: > > >>>From: Simon Leinen [mailto:simon@limmat.switch.ch] >>>"If some flow information data cannot be delivered to the collector >>>in a timely manner, the amount of missing flow information data >>>MUST be indicated to the collector. Possible reasons for failure >>>to deliver data include loss of packets on the network path, or >>>resource shortage at the exporter." >>> >>> > > > >>But how the exporter would know which records were lost in the >>network unless we have (application layer) ACKs? >> >> > >I see - my "MUST be indicated" is confusing too. The exporter >wouldn't need to know, but the collector must be able to determine >that something was lost, and "how much", where I intentionally leave >open the accuracy - the collector won't be able to reconstruct >everything anyway. So what about this: > > "If some flow information data cannot be delivered to the collector > in a timely manner, the collector MUST be able to estimate the > amount of data missing. Possible reasons for failure to deliver > data include loss of packets on the network path, or resource > shortage at the exporter." > >As an example, the flow export mechanism I use sends me packets of >flow records, and each packet has a flow record sequence number. So I >can determine exactly how many flows I have missed. This information >is important for diagnosis of situations where there is information >loss (some, but not all of this information loss would have been >prevented by using a reliable transport protocol). > The sequence ID per observation domain on the exporter is exactly what we use in NetFlow to detect we lost some export packets. And we don't need application level ACK. Regards, Benoit. > >For my particular applications, I'm mostly interested in the byte >amounts of metered traffic. So if a flow export protocol had sequence >numbers representing the cumulative total bytes metered, that would be >very useful to me - I really would like to know what percentage of >traffic could not be accounted for. (In addition to that, I would >still want to know how many flow records I missed.) > > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon Oct 7 10:49:06 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA21800 for ; Mon, 7 Oct 2002 10:49:05 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17yZ0M-0002JF-00 for ipfix-list@mil.doit.wisc.edu; Mon, 07 Oct 2002 09:37:06 -0500 Received: from [64.95.122.60] (helo=RS-SC-EXC4.rs.riverstonenet.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 17yZ0K-0002Hz-00 for ipfix-eval@net.doit.wisc.edu; Mon, 07 Oct 2002 09:37:04 -0500 Received: from riverstonenet.com ([134.141.180.78]) by RS-SC-EXC4.rs.riverstonenet.com with Microsoft SMTPSVC(5.0.2195.4905); Mon, 7 Oct 2002 07:36:32 -0700 Message-ID: <3DA19B7E.7DF5421A@riverstonenet.com> Date: Mon, 07 Oct 2002 10:34:38 -0400 From: calato@riverstonenet.com X-Mailer: Mozilla 4.75C-CCK-MCD NSCPCD475 [en] (X11; U; SunOS 5.6 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: Peter Ludemann CC: ipfix-eval@net.doit.wisc.edu Subject: Re: [ipfix-eval] Discussion of Candidate Protocols References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 07 Oct 2002 14:36:32.0836 (UTC) FILETIME=[EE6AA840:01C26E0E] Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Peter Ludemann wrote: > > Jeff Meyer wrote Friday, October 04, 2002 9:58 AM: > > > There seems to be some confusion over the separation > > between a formal, machine readable description of > > the data model (as IPDR defines), and the requirements put > > on an implementation (especially in an IPFIX Middlebox). > > YES ... so, can we all agree that data model issues are not > part of this discussion? No. We have no interoperability without a data model. > > [snip] > > > However, the statement that IP flow is somehow unique > > in its data requirements and as such a more generalized > > "data mover" is somehow problematic, is just plain wrong. > > YES! No. Generalized data movers are not the problem. Defining a protocol that is well defined enough so that many device vendors can export data and many application vendors can process the data regardless of the device is problematic if all you define is a general purpose data mover. > > > As a mediation product vendor I see lots of devices > > with lots of vendor specific protocols. And there > > are several which have the same protocol requirements > > as IPFIX. Just different data models. > > YES! > > The data models need to also be aligned; but just dealing > with all the protocols is a headache, especially with the > unreliable protocols such as NetFlow and SNMP (RADIUS is > an "almost reliable" protocol and therefore isn't quite > so difficult). > > [snip] > > > ... If IPFIX wants to say that only > > fixed with data values may be sent via the IPFIX > > protocol, then I'd say, maybe you could optimize > > beyond IPDR for that. > > CRANE imposes only the header overhead on each record. > The data are also sent in the "native" form for the > exporter (I'll comment on XDR later). > > However, I think that a numeric-only protocol is not > sufficient. As soon as metrics are collected at the > application level, people want attributes, such as > URL path, response status, payload type, etc. which > are either difficult or impossible to encode as number. I'll just quote Randy's message... >> Please let's try to stick with the core goal of looking at IP (v4&v6) >> packets and the "transport" protocol or next header. > > i think that was the intent of this wg's charter. > > randy, with ad hat on Paul > > > A minimal implementation of IPDR on a middlebox > > would NOT need to have any XML knowledge whatsoever. > > As a producer, the system could merely hardcode the > > information model produced (or make it configurable). > > Likewise for CRANE, the box could simply reject all > requests to limit the fields that are exported, by > hardcoding the templates. However, processing the > templates is somewhat easier in CRANE because of their > fixed layout, not requiring an XML engine (yes, I know > that an XML engine can be quite compact; but it's not > common on network elements, at least not now). > > > The encoding itself follows XDR format, which has > > worked alright for NFS and RPC, and is the precursor > > to the encoding for DCE and CORBA encoding. > > We looked at XDR and rejected it because it can be a > little "heavy" for CPU-constrained devices (e.g., CSU/DSU, > "cable routers", etc.). We prefer exporting in the > "native" format (big- or little-endian) with template > flags to allow decoding at the receiver. Of course, > we don't prevent export in XDR (big-endian) form. > > - peter > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon Oct 7 11:03:46 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA22217 for ; Mon, 7 Oct 2002 11:03:46 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17yZCU-0002g8-00 for ipfix-list@mil.doit.wisc.edu; Mon, 07 Oct 2002 09:49:38 -0500 Received: from [64.95.122.60] (helo=RS-SC-EXC4.rs.riverstonenet.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 17yZCS-0002fN-00 for ipfix-eval@net.doit.wisc.edu; Mon, 07 Oct 2002 09:49:36 -0500 Received: from riverstonenet.com ([134.141.180.78]) by RS-SC-EXC4.rs.riverstonenet.com with Microsoft SMTPSVC(5.0.2195.4905); Mon, 7 Oct 2002 07:49:04 -0700 Message-ID: <3DA19E6E.A018857B@riverstonenet.com> Date: Mon, 07 Oct 2002 10:47:10 -0400 From: calato@riverstonenet.com X-Mailer: Mozilla 4.75C-CCK-MCD NSCPCD475 [en] (X11; U; SunOS 5.6 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: Simon Leinen CC: Peter Ludemann , Michael MacFaden , ipfix-eval@net.doit.wisc.edu Subject: Re: [ipfix-eval] Discussion of Candidate Protocols References: <3D9AEC51.BA498740@riverstonenet.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 07 Oct 2002 14:49:05.0190 (UTC) FILETIME=[AEDAC460:01C26E10] Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Simon Leinen wrote: > > On Wed, 02 Oct 2002 08:53:37 -0400, calato@riverstonenet.com said: > > Peter Ludemann wrote: > >> However, we have encountered some situations where saving a single > >> copy operation per output recordcan make all the difference between > >> being able to deliver the data and not. It's highly unlikely that > >> an application will keep data in AVP form, so AVP requires copies. > >> If the protocol packs the data into something that a C programmer > >> might do with a "struct", then memory-to-memory copying can be > >> avoided. > > > Having done a fair amount of tuning on a collector I find it > > interesting that your performance hinged on a memory copy. In > > my experince performance was limited by how fast the disk > > writes are. > > Some collectors will write large amounts of data to disk - if this is > the main bottleneck I'd assume that they use the accounting data > without too heavy processing (or they need a better approach to disk > I/O, such as direct I/O, scatter/gather interfaces, faster disks etc.). I don't follow. Are you saying I/O isn't typically the bottleneck at the collector? > > Unnecessary copying should be avoided mainly because memory bandwidth > doesn't seem to grow as fast as other performance dimensions. > Personally I think this is more of an issue than for example > byte-order optimizations. I think there are lots of other issues that come into play before memory copies do. malloc and free are typically to big culprits of CPU usage. A design that causes lots of task switching is another. Your program needs to be pretty finely tuned before memory copy shows up, assuming the app is not just plain silly about memory copies. > > Regards, > -- > Simon. -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon Oct 7 11:42:13 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA23664 for ; Mon, 7 Oct 2002 11:42:12 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17yZkx-0003ee-00 for ipfix-list@mil.doit.wisc.edu; Mon, 07 Oct 2002 10:25:15 -0500 Received: from odd-brew.cisco.com ([144.254.15.119] helo=strange-brew.cisco.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 17yZkv-0003dO-00; Mon, 07 Oct 2002 10:25:13 -0500 Received: from cisco.com (bclaise-isdn-home3.cisco.com [10.49.4.220]) by strange-brew.cisco.com (8.11.6+Sun/8.8.8) with ESMTP id g97FO0729419; Mon, 7 Oct 2002 17:24:00 +0200 (CEST) Message-ID: <3DA1A710.9080308@cisco.com> Date: Mon, 07 Oct 2002 17:24:00 +0200 From: Benoit Claise User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.1) Gecko/20020815 X-Accept-Language: en-us, en MIME-Version: 1.0 To: calato@riverstonenet.com CC: Simon Leinen , Reinaldo Penno , ipfix-req@net.doit.wisc.edu, ipfix-eval-team@net.doit.wisc.edu Subject: Re: [ipfix-req] Re: Data Export Reliability Requirement [was: Re: IPFIX Requireme nt draft status - section 6.3.2] References: <7B802811BE77D51189910002A55CFD2C0426C5F2@zsc3c032.us.nortel.com> <3DA198C6.CC26BCCD@riverstonenet.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Paul, >Simon Leinen wrote: > > >>On Fri, 4 Oct 2002 18:44:10 -0700 , "Reinaldo Penno" said: >> >> >>>>From: Simon Leinen [mailto:simon@limmat.switch.ch] >>>>"If some flow information data cannot be delivered to the collector >>>>in a timely manner, the amount of missing flow information data >>>>MUST be indicated to the collector. Possible reasons for failure >>>>to deliver data include loss of packets on the network path, or >>>>resource shortage at the exporter." >>>> >>>> >>>But how the exporter would know which records were lost in the >>>network unless we have (application layer) ACKs? >>> >>> >>I see - my "MUST be indicated" is confusing too. The exporter >>wouldn't need to know, but the collector must be able to determine >>that something was lost, and "how much", where I intentionally leave >>open the accuracy - the collector won't be able to reconstruct >>everything anyway. So what about this: >> >> "If some flow information data cannot be delivered to the collector >> in a timely manner, the collector MUST be able to estimate the >> amount of data missing. Possible reasons for failure to deliver >> data include loss of packets on the network path, or resource >> shortage at the exporter." >> I even prefer this proposal compared to the previous one from Simon. >> >> > > Not sure I like that. How about when the exporter starts > talking to another collector. How about in overload mode > when the collector is just dropping flows. > What is the issue, you could add "overload on the collector" in the list of example? What is important is the collector must know the amount of data missing, whereever it comes from: exporter, path or collector. If you send this information from the exporter, you could possibly not report missing information. Ex: - the transport session breaks, what happened to the flow records which were sent? - the transport protocol acknowlegde some packets but they are not handled correctly by the collector (CPU too busy) BTW, I agree with you Paul about the "timely manner" which is confusing and maybe not necessary. Regards, Benoit. > > > I think the exporter is in the best position to estimate how > much data was lost. Once more of the protocol is nailed down > we can better define how to measure loss. > > >>As an example, the flow export mechanism I use sends me packets of >>flow records, and each packet has a flow record sequence number. So I >>can determine exactly how many flows I have missed. This information >>is important for diagnosis of situations where there is information >>loss (some, but not all of this information loss would have been >>prevented by using a reliable transport protocol). >> >>For my particular applications, I'm mostly interested in the byte >>amounts of metered traffic. So if a flow export protocol had sequence >>numbers representing the cumulative total bytes metered, that would be >>very useful to me - I really would like to know what percentage of >>traffic could not be accounted for. (In addition to that, I would >>still want to know how many flow records I missed.) >>-- >>Simon Leinen simon@babar.switch.ch >>SWITCH http://www.switch.ch/misc/leinen/ >> >> Computers hate being anthropomorphized. >> >>-- >>Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body >>Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >>"unsubscribe ipfix" in message body >>Archive http://ipfix.doit.wisc.edu/archive/ >> >> > >-- >Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body >Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >"unsubscribe ipfix" in message body >Archive http://ipfix.doit.wisc.edu/archive/ > > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon Oct 7 12:00:10 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA24492 for ; Mon, 7 Oct 2002 12:00:09 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17yaC3-0004R9-00 for ipfix-list@mil.doit.wisc.edu; Mon, 07 Oct 2002 10:53:15 -0500 Received: from [64.95.122.60] (helo=RS-SC-EXC4.rs.riverstonenet.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 17yaC1-0004QO-00; Mon, 07 Oct 2002 10:53:13 -0500 Received: from riverstonenet.com ([134.141.180.78]) by RS-SC-EXC4.rs.riverstonenet.com with Microsoft SMTPSVC(5.0.2195.4905); Mon, 7 Oct 2002 08:52:39 -0700 Message-ID: <3DA1AD55.8573BDE0@riverstonenet.com> Date: Mon, 07 Oct 2002 11:50:45 -0400 From: calato@riverstonenet.com X-Mailer: Mozilla 4.75C-CCK-MCD NSCPCD475 [en] (X11; U; SunOS 5.6 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: Benoit Claise CC: Simon Leinen , Reinaldo Penno , ipfix-req@net.doit.wisc.edu, ipfix-eval-team@net.doit.wisc.edu Subject: Re: [ipfix-req] Re: Data Export Reliability Requirement [was: Re:IPFIX Requireme nt draft status - section 6.3.2] References: <7B802811BE77D51189910002A55CFD2C0426C5F2@zsc3c032.us.nortel.com> <3DA198C6.CC26BCCD@riverstonenet.com> <3DA1A710.9080308@cisco.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 07 Oct 2002 15:52:40.0582 (UTC) FILETIME=[91016260:01C26E19] Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Benoit Claise wrote: > > Paul, > > >Simon Leinen wrote: > > > > > >>On Fri, 4 Oct 2002 18:44:10 -0700 , "Reinaldo Penno" said: > >> > >> > >>>>From: Simon Leinen [mailto:simon@limmat.switch.ch] > >>>>"If some flow information data cannot be delivered to the collector > >>>>in a timely manner, the amount of missing flow information data > >>>>MUST be indicated to the collector. Possible reasons for failure > >>>>to deliver data include loss of packets on the network path, or > >>>>resource shortage at the exporter." > >>>> > >>>> > >>>But how the exporter would know which records were lost in the > >>>network unless we have (application layer) ACKs? > >>> > >>> > >>I see - my "MUST be indicated" is confusing too. The exporter > >>wouldn't need to know, but the collector must be able to determine > >>that something was lost, and "how much", where I intentionally leave > >>open the accuracy - the collector won't be able to reconstruct > >>everything anyway. So what about this: > >> > >> "If some flow information data cannot be delivered to the collector > >> in a timely manner, the collector MUST be able to estimate the > >> amount of data missing. Possible reasons for failure to deliver > >> data include loss of packets on the network path, or resource > >> shortage at the exporter." > >> > I even prefer this proposal compared to the previous one from Simon. > > >> > >> > > > > Not sure I like that. How about when the exporter starts > > talking to another collector. How about in overload mode > > when the collector is just dropping flows. > > > What is the issue, you could add "overload on the collector" in the list > of example? > What is important is the collector must know the amount of data missing, > whereever it comes from: exporter, path or collector. > If you send this information from the exporter, you could possibly not > report missing information. > Ex: > - the transport session breaks, what happened to the flow records which > were sent? Exaclty, see below. > - the transport protocol acknowlegde some packets but they are not > handled correctly by the collector (CPU too busy) Again, I think it is difficult to describe how to measure loss without a protocol definition to analyze. But my view is that we will be in a **better** position (not perfect) to design a solution by measuring data loss from the exporter's perspective rather than the collector. The reason being the exporter talks to many collectors and thus has the common view, not the collector. So lets assume we have a simple message level ACK for every 100 messages. I have an outstanding ACK when the device looses connectivity to the collector. So the device resends the 100 messages to collector #2. No need to report any data loss. If on the other hand if the device never resends those messages (because of a design choice) then the device needs to report the potential loss to the new collector (or how ever that info is to be retrieved, maybe SNMP). If we examine this from the collector perspective, lets assume it never receives any of the 100 messages that were outstanding. How does it even know to report a loss. If somehow it detected the loss, how does it know the messages were resent? If the collector recieved the messages but just the ACK got lost and the exporter resent the messages, it didin't report a loss anyway. If the exporter didn't resend then it falsely reported the loss. But I think this is preferrable to the collector scenario where no loss was reported when there was one. If I as the administrator know that the potential loss is the upper bound then the operator can make an informed decision to look into the matter futher or not. I don't think we can design a 100% guaranteed solution but one from the exporter's perspective I think lends itself to better possibilities. Paul > > BTW, I agree with you Paul about the "timely manner" which is confusing > and maybe not necessary. > > Regards, Benoit. > > > > > > > I think the exporter is in the best position to estimate how > > much data was lost. Once more of the protocol is nailed down > > we can better define how to measure loss. > > > > > >>As an example, the flow export mechanism I use sends me packets of > >>flow records, and each packet has a flow record sequence number. So I > >>can determine exactly how many flows I have missed. This information > >>is important for diagnosis of situations where there is information > >>loss (some, but not all of this information loss would have been > >>prevented by using a reliable transport protocol). > >> > >>For my particular applications, I'm mostly interested in the byte > >>amounts of metered traffic. So if a flow export protocol had sequence > >>numbers representing the cumulative total bytes metered, that would be > >>very useful to me - I really would like to know what percentage of > >>traffic could not be accounted for. (In addition to that, I would > >>still want to know how many flow records I missed.) > >>-- > >>Simon Leinen simon@babar.switch.ch > >>SWITCH http://www.switch.ch/misc/leinen/ > >> > >> Computers hate being anthropomorphized. > >> > >>-- > >>Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > >>Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > >>"unsubscribe ipfix" in message body > >>Archive http://ipfix.doit.wisc.edu/archive/ > >> > >> > > > >-- > >Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > >Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > >"unsubscribe ipfix" in message body > >Archive http://ipfix.doit.wisc.edu/archive/ > > > > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon Oct 7 18:59:06 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA09178 for ; Mon, 7 Oct 2002 18:59:06 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17ygJ9-0001Wd-00 for ipfix-list@mil.doit.wisc.edu; Mon, 07 Oct 2002 17:24:59 -0500 Received: from mailhub.xacct.com ([204.253.100.25] helo=xacct.com) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 17ygJ7-0001WX-00 for ipfix-eval@net.doit.wisc.edu; Mon, 07 Oct 2002 17:24:57 -0500 Received: (qmail 531 invoked from network); 7 Oct 2002 22:24:54 -0000 Received: from usmail.xacct.com (204.253.100.12) by mailhub.xacct.com with SMTP; 7 Oct 2002 22:24:54 -0000 Received: from peter (inside.us.xacct.com [204.253.100.102]) by usmail.xacct.com (8.11.2/8.11.2) with SMTP id g97MS5s31951 for ; Mon, 7 Oct 2002 15:28:05 -0700 From: "Peter Ludemann" To: Subject: [ipfix-eval] RE: Performance (was: Discussion of Candidate Protocols) Date: Mon, 7 Oct 2002 15:24:57 -0700 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 In-Reply-To: Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit (This is a fairly minor point but I'd just like to have my "last word" :-) Simon Leinen wrote Monday, October 07, 2002 3:47 AM: > Some collectors will write large amounts of data to disk - if this is > the main bottleneck I'd assume that they use the accounting data > without too heavy processing (or they need a better approach to disk > I/O, such as direct I/O, scatter/gather interfaces, faster disks etc.). Yes. Especially scatter/gather. > Unnecessary copying should be avoided mainly because memory bandwidth > doesn't seem to grow as fast as other performance dimensions. > Personally I think this is more of an issue than for example > byte-order optimizations. If the exporter must do byte-swapping it must do copying. That's why we avoided XDR. calato@riverstonenet.com wrote Monday, October 07, 2002 7:47 AM: > I think there are lots of other issues that come into > play before memory copies do. malloc and free are typically > to big culprits of CPU usage. A design that causes lots > of task switching is another. Your program needs to be > pretty finely tuned before memory copy shows up, assuming > the app is not just plain silly about memory copies. Which is why Un*x kernels keep pools of memory, use mbufs, etc. Anybody who uses malloc/free in high-performance code needs some re-education. But I'm talking about already highly tuned situations, running on small boxes (no fans, low power CPUs) ... we've encountered situations where bus bandwidth is a significant performance issue, and there's no point in putting in roadblocks (such as potential byte-swapping by forcing use of XDR or forcing field copies because the output format doesn't fit in an obvious way to the internal data structures). Even in high-performance situations, export of measurements is treated as an extra cost, to be minimised. As I recall, on telephone switches the goal was for operational measurements to take less than 5% of the system CPU. - peter -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Oct 8 03:07:04 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id DAA02020 for ; Tue, 8 Oct 2002 03:07:03 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17yoFF-0007YG-00 for ipfix-list@mil.doit.wisc.edu; Tue, 08 Oct 2002 01:53:29 -0500 Received: from mailhub.xacct.com ([204.253.100.25] helo=xacct.com) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 17yoFD-0007Y8-00 for ipfix-eval@net.doit.wisc.edu; Tue, 08 Oct 2002 01:53:27 -0500 Received: (qmail 15700 invoked from network); 8 Oct 2002 06:53:22 -0000 Received: from usmail.xacct.com (204.253.100.12) by mailhub.xacct.com with SMTP; 8 Oct 2002 06:53:22 -0000 Received: from peter ([192.168.254.163]) by usmail.xacct.com (8.11.2/8.11.2) with SMTP id g986uXs25171 for ; Mon, 7 Oct 2002 23:56:34 -0700 From: "Peter Ludemann" To: Subject: [ipfix-eval] RE: Discussion of Candidate Protocols - data model; broadcast vs reliable Date: Mon, 7 Oct 2002 23:53:25 -0700 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 In-Reply-To: <3DA19B7E.7DF5421A@riverstonenet.com> Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit This note discusses two things: - whether we need a data model to define the protocol (I claim that we do not) - cost of UDP broadcast vs minimalist reliable transport (I claim that there is very little difference) calato@riverstonenet.com wrote Monday, October 07, 2002 7:35 AM: > Peter Ludemann wrote: [snip] > > YES ... so, can we all agree that data model issues are not > > part of this discussion? > > No. We have no interoperability without a data model. I agree that we need a data model. I don't see why it should affect the discussion of the export protocol. The only requirements on the export protocol are that it exhibit the desired reliability and efficiency; and that it be able to transport all the data types required by the data models. If we can agree on all the data types, then we can define the protocol; this does not require defining the entire data model. [After all, SNMP is defined independently of the various MIBs - which correspond to the data model] > No. Generalized data movers are not the problem. > Defining a protocol that is well defined enough so > that many device vendors can export data and many > application vendors can process the data regardless > of the device is problematic if all you define is a > general purpose data mover. But deciding on an adequate "generalized data mover" is the point of this evaluation process, I thought. I expect that there will be multiple data models, for the various kinds of generalized devices. (e.g., for our probe, we've defined three generic groupings of data: volume metrics; QoS metrics; usage metrics and attributes -- similarly, SNMP's enterprise MIBs and RADIUS's vendor-specific attributes). There is one place where the nature of the data might intersect with the protocol -- the high-volume metrics such as exported by NetFlow; and where the claim is that a reliable protocol is an unnecessary overhead. So, let me discuss this a bit more. The argument is that with high volume export it is preferable to do multi-cast UDP with sequence numbers; data loss can be detected by noting which sequence numbers are missing ... and reliability can be increased by having multiple receivers and de-duplicating what they receive. For such a situation, the reliable protocol's *implementation* would not keep a queue of data to retransmit on fail-over; it would only keep enough buffer to deal with TCP's or SCTP's flow control and to handle bursts of records. ACKs would be used only to notice when data are not received at the other end and to cause fail-over. TCP "write would block" also causes fail-over. I would argue that the cost of exporting this way is very similar to that of the UDP broadcast. And on the receiving end, it is *much* easier to handle than a UDP-broadcast, which also needs stronger hardware because of the higher de-duplication load. Can be about the same for both: - data copying (if scatter/gather is used) - protocol overhead (sequence number, template ID, etc.) The possible extra cost of the "reliable" export version is: - timer for noticing when ACK isn't received [trivial cost] - TCP vs UDP (little difference with modern implementations) - TCP retransmit with packet loss (typically very low) - cost of fail-over when no ACK received or write would block The savings and benefits of the "reliable" version are: - fewer packet writes (broadcast requires 2x as many packets; and TCP can pack more efficiently because records can span packets) - lower network traffic (which adds to reliability) - smaller machines for receiving - more accurate estimate of loss due to lack of reliability - can handle congestion [As an aside, even with the UDP-broadcast version, the exporter ought to compute totals for the various metrics and output those from time to time, to provide a more accurate understanding of data loss. Perhaps such totals are already available in the MIBs, but there remains the issue of how to correlate with the sequence numbers of the exported data.] - peter -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Oct 8 06:36:11 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id GAA05421 for ; Tue, 8 Oct 2002 06:36:11 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 17yrXz-00009M-00 for ipfix-list@mil.doit.wisc.edu; Tue, 08 Oct 2002 05:25:03 -0500 Received: from odd-brew.cisco.com ([144.254.15.119] helo=strange-brew.cisco.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 17yrXw-000088-00; Tue, 08 Oct 2002 05:25:00 -0500 Received: from cisco.com (dhcp-peg3-vl30-144-254-7-121.cisco.com [144.254.7.121]) by strange-brew.cisco.com (8.11.6+Sun/8.8.8) with ESMTP id g98AOH702521; Tue, 8 Oct 2002 12:24:18 +0200 (CEST) Message-ID: <3DA2B251.2060700@cisco.com> Date: Tue, 08 Oct 2002 12:24:17 +0200 From: Benoit Claise