From majordomo@mil.doit.wisc.edu Fri Mar 1 05:15:57 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id FAA04512 for ; Fri, 1 Mar 2002 05:15:56 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16gjlX-0002Qb-00 for ipfix-list@mil.doit.wisc.edu; Fri, 01 Mar 2002 03:55:51 -0600 Received: from mailhub.fokus.gmd.de ([193.174.154.14]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16gjlU-0002QO-00 for ipfix@net.doit.wisc.edu; Fri, 01 Mar 2002 03:55:48 -0600 Received: from fokus.fhg.de (dhcp187 [195.37.78.187]) by mailhub.fokus.gmd.de (8.11.6/8.11.6) with ESMTP id g219t6e09014; Fri, 1 Mar 2002 10:55:08 +0100 (MET) Message-ID: <3C7F4ED9.D826E88A@fokus.fhg.de> Date: Fri, 01 Mar 2002 10:50:17 +0100 From: Sebastian Zander Organization: FhI FOKUS X-Mailer: Mozilla 4.75 [de] (Windows NT 5.0; U) X-Accept-Language: de MIME-Version: 1.0 To: n.brownlee@auckland.ac.nz CC: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Current WG activities References: <200202271700.GAA09103@mailhost.auckland.ac.nz> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Hi Nevil, n.brownlee@auckland.ac.nz schrieb: [...] > > 1) The Arch and Data version 00 drafts are now published (another WG > Goal met). > > 2) The charter says we'll "select a protocol," which means we need to > work on - i.e. discuss further on the list - the selection criteria > for that protocol. > > 3) It seems to me that discussing aspects like exporter and collector > configuration methods is helpful; it would be good to see some > consensus on this emerge. > > 4) In selecting a protocol, we'll pick the one which comes closest to > meeting all the selection criteria. *If* the chosen one could be > *slightly* modified or extended to meet all the criteria, we may > have to consider including those modifications in the final IPFIX > Arcitecture document. Are there any official plans yet on how the selection process will work? Will it be similar to that happended in the AAA WG where an evaluation team checked the proposals vs. the requirements producing an I-D? -- Sebastian -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri Mar 1 09:55:08 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA28217 for ; Fri, 1 Mar 2002 09:55:08 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16go5A-0002PR-00 for ipfix-list@mil.doit.wisc.edu; Fri, 01 Mar 2002 08:32:24 -0600 Received: from mailhub.fokus.gmd.de ([193.174.154.14]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16go57-0002PJ-00 for ipfix-req@net.doit.wisc.edu; Fri, 01 Mar 2002 08:32:21 -0600 Received: from fokus.fhg.de (dhcp206 [195.37.78.206]) by mailhub.fokus.gmd.de (8.11.6/8.11.6) with ESMTP id g21EWIe25055 for ; Fri, 1 Mar 2002 15:32:18 +0100 (MET) Message-ID: <3C7F911F.1090306@fokus.fhg.de> Date: Fri, 01 Mar 2002 15:33:03 +0100 From: Tanja Zseby Organization: FhI FOKUS User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:0.9.4) Gecko/20011019 Netscape6/6.2 X-Accept-Language: en-us MIME-Version: 1.0 To: IPFIX Requirements <" ipfix-req"@net.doit.wisc.edu> Subject: [ipfix-req] updated version of requirements draft Content-Type: multipart/mixed; boundary="------------010708070700020904010606" Precedence: bulk Sender: majordomo listserver This is a multi-part message in MIME format. --------------010708070700020904010606 Content-Type: multipart/alternative; boundary="------------010103020303070804050708" --------------010103020303070804050708 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Hi all, we submitted an updated version of the requirements document (see attached document). It contains only small changes, mainly in the terminology section. Regards, Tanja -------- Original Message -------- Subject: please post draft draft-ietf-ipfix-reqs-02.txt Date: Fri, 01 Mar 2002 15:23:56 +0100 From: Tanja Zseby Organization: FhI FOKUS To: internet-drafts@ietf.org CC: ipfix-chairs@net.doit.wisc.edu Please post the attached document draft-ietf-ipfix-reqs-02.txt as an Internet Draft. Thank you Tanja -- Dipl.-Ing. Tanja Zseby FhI FOKUS/Global Networking Email: zseby@fokus.fhg.de Kaiserin-Augusta-Allee 31 Phone: +49-30-3463-7153 D-10589 Berlin, Germany Fax: +49-30-3463-8153 -------------------------------------------------------------------------------------- "Living on earth is expensive but it includes a free trip around the sun." (Anonymous) -------------------------------------------------------------------------------------- -- Dipl.-Ing. Tanja Zseby FhI FOKUS/Global Networking Email: zseby@fokus.fhg.de Kaiserin-Augusta-Allee 31 Phone: +49-30-3463-7153 D-10589 Berlin, Germany Fax: +49-30-3463-8153 -------------------------------------------------------------------------------------- "Living on earth is expensive but it includes a free trip around the sun." (Anonymous) -------------------------------------------------------------------------------------- --------------010103020303070804050708 Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7bit Hi all,

we submitted an updated version of the requirements document (see attached document). It contains only small changes, mainly in the terminology section.

Regards,
Tanja


-------- Original Message --------
Subject: please post draft draft-ietf-ipfix-reqs-02.txt
Date: Fri, 01 Mar 2002 15:23:56 +0100
From: Tanja Zseby <zseby@fokus.fhg.de>
Organization: FhI FOKUS
To: internet-drafts@ietf.org
CC: ipfix-chairs@net.doit.wisc.edu 


Please post the attached document

                draft-ietf-ipfix-reqs-02.txt

as an Internet Draft.

Thank you
Tanja

-- 
Dipl.-Ing. Tanja Zseby			    	      	
FhI FOKUS/Global Networking			Email: zseby@fokus.fhg.de	
Kaiserin-Augusta-Allee 31				Phone: +49-30-3463-7153
D-10589 Berlin, Germany				Fax:   +49-30-3463-8153
-------------------------------------------------------------------------------------- 
"Living on earth is expensive but it includes a free trip around the sun." (Anonymous)
--------------------------------------------------------------------------------------



-- 
Dipl.-Ing. Tanja Zseby			    	      	
FhI FOKUS/Global Networking			Email: zseby@fokus.fhg.de	
Kaiserin-Augusta-Allee 31				Phone: +49-30-3463-7153
D-10589 Berlin, Germany				Fax:   +49-30-3463-8153
-------------------------------------------
------------------------------------------- 
"Living on earth is expensive but it includes a free trip around the sun." (Anonymous)
--------------------------------------------------------------------------------------
--------------010103020303070804050708-- --------------010708070700020904010606 Content-Type: text/plain; name="draft-ietf-ipfix-reqs-02.txt" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="draft-ietf-ipfix-reqs-02.txt" Content-Transfer-Encoding: 7bit Internet Draft J. Quittek NEC Europe Ltd. Expires: August 2002 T. Zseby FhI FOKUS B. Claise Cisco Systems (Editors) March 2002 Requirements for IP Flow Information Export Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Abstract This memo defines requirements for the export of measured IP flow information out of routers, traffic measurement probes and middleboxes. Conventions used in this document The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119. Quittek et al. draft-ietf-ipfix-reqs-02.txt [Page 1] Internet Draft IPFIX Requirements March 2002 Table of Content 1. Introduction 2. Terminology 2.1. IP Traffic Flows 2.2. Observation Point 2.3. Metering Process 2.4. Flow Record 2.5. Export Process 2.6. Flow Information Collector or Collector 2.7. IPFIX device 3. Applications Requiring IP Flow Information Export 3.1 Usage-based Accounting 3.2 Traffic Profiling 3.3 Traffic Engineering 3.4 Attack/Intrusion Detection 3.5 QoS Monitoring 4. Distinguishing Flows 4.1. Interfaces 4.2. IP Header Fields 4.3. Transport Header Fields 4.4. MPLS Label 4.5. DiffServ Code Point 4.6. Header Compression and Encryption 5. Metering Process 5.1. Reliability 5.2. Sampling 5.3. Overload Behavior 5.4. Timestamps 5.5. Time Synchronization 5.6. Timeout 5.7. Ignore Port Copy 6. Data Export 6.1. Information Model 6.2. Data Model 6.3. Data Transfer 6.3.1. Congestion Awareness 6.3.2. Reliability 6.3.3. Security 6.4. Push and Pull Mode Reporting 6.5. Regular Reporting Interval 6.6. Notification on Specific Events 6.7. Anonymization 7. Configuration 8. General Requirements 8.1. Openness 8.2. Scalability concerning measuring devices 8.3. Several Data Collectors Quittek et al. draft-ietf-ipfix-reqs-02.txt [Page 2] Internet Draft IPFIX Requirements March 2002 9. Security Considerations 10. Acknowledgments 11. References 12. Authors' Addresses Appendix: Derivation of Requirements form Target Applications 1. Introduction There are several applications that require flow-based IP traffic measurements. Such measurements could be performed by a router while forwarding the traffic, by a middlebox [MIDBOXTAX], or by a traffic measurement probe attached to a line or a monitored port. This memo defines requirements for exporting traffic flow information out of these boxes for further processing by applications located on other devices. In section 2 a selection of such applications is presented. The following sections list requirements derived from these applications. 2. Terminology 2.1. IP Traffic Flows There are several definitions of the term 'flow' being used by the Internet community. Within this document we use the following one: A flow is defined as a set of packets passing an observation point in the network during a certain time interval. All packets belonging to a particular flow have a set of common properties. Each property is defined as the result of applying a function to the values of: 1. one or more of packet header fields (eg. destination IP address) 2. one or more properties of the packet itself (eg. packet length) 3. one or more of fields derived from packet treatment (eg. AS number) A packet is defined to belong to a flow if it completely satisfies all the defined properties of the flow. This definition covers the range from a flow containing all packets observed at a network interface to a flow consisting of just a single packet between two applications with a specific sequence number. Please note that the flow definition does not match a general application-level end-to-end stream. However, an application may derive properties of application-level streams by processing measured flow data. Quittek et al. draft-ietf-ipfix-reqs-02.txt [Page 3] Internet Draft IPFIX Requirements March 2002 2.2. Observation Point The observation point is a location in the network where IP packets can be observed. Examples are a line to which a probe is attached, a shared medium, such as an Ethernet-based LAN, a single port of a router, or a set of interfaces (physical or logical) of a router. 2.3. Metering Process A metering process is a set of actions performed on packets observed at an observation point in order to map them to a flow. A metering process can include functions for timestamping, classification and sampling of packets. Typically, the metering process also includes maintainance of flow records, computation of flow statistics, and detection of flow expiration. 2.4. Flow Record A flow record contains information about a specific flow that was metered at an observation point. A flow record contains measured properties of the flow (e.g. the total number of bytes of all packets of the flow) and usually characteristic properties of the flow (e.g. source IP address). 2.5. Export Process The export process sends flow records to one or more collectors. 2.6. Collector The collector receives flow records from one or more export processes. The collector might process or store received flow record, but these actions are out of the scope of this document. 2.7. IPFIX device A device hosting at least a flow information export process. Typically, corresponding Observation points, metering processes, and export processes are co-located at this device, for example at a router. Quittek et al. draft-ietf-ipfix-reqs-02.txt [Page 4] Internet Draft IPFIX Requirements March 2002 3. Applications Requiring IP Flow Information Export The following list contains a selection of applications requiring IP flow information export. Because requirements for flow export listed in further sections below are derived from these applications, their selection is crucial. The goal of this requirements document is not to cover all possible applications with all their flow export requirements, but to cover applications which are considered to be of significant importance in today's or future IP networks, and for which requirements can be met with reasonable technical effort. Please note, that the described applications can have a large number of differing implementations. Requirement details or the weighting of requirements could differ for specific implementations. Therefore we derive the requirements from the general functionality of the selected applications. Furthermore, the list of applications should lead to a better understanding of the requirements which is particularly important when designing or implementing a traffic flow measuring device. 3.1 Usage-based Accounting Several new business models for selling IP service and IP-based services are currently under investigation. Beyond flat rate services which do not need accounting, accounting for these models can be based on time or volume. Accounting data can serve as input for billing systems. Accounting can be performed per user or per user group, it can be performed just for basic IP service or individually per high-level service and/or per content type delivered. For advanced/future services, accounting may also be performed per class of service, per application, per time of day, per used (label switched) path, etc. 3.2 Traffic Profiling Traffic profiling is a process of characterizing IP flows and flow aggregates by using a model that represents key parameters of the flow such as flow duration, volume, time and burstiness. It is a prerequisite for network planning, network dimensioning, trend analysis, developing business models, and other activities. It heavily depends on the particular traffic profiling objective(s) what statistics and accuracy are required from the measurements. Typical information needed for traffic profiling are the distribution of used services and protocols in the network, the amount of packets of a specific type (e.g. percentage of IPv6 packets) and specific flow profiles. Since objectives for traffic profiling can vary, this application Quittek et al. draft-ietf-ipfix-reqs-02.txt [Page 5] Internet Draft IPFIX Requirements March 2002 requires a high flexibility of the measurement infrastructure, especially regarding the options for measurement configuration and packet classification. 3.3 Traffic Engineering Traffic Engineering (TE) comprises methods for measurement, modeling, characterization and control of a network. The goal of TE is the optimization of network resource utilization and traffic performance [RFC2702]. Since control and administrative reaction to measurement results requires access to the involved network nodes, TE mechanisms and the required measurement function usually are performed within one administrative domain. Typical parameters required for TE are link utilization, load between specific network nodes, number, size and entry/exit points of the active flows and routing information. 3.4 Attack/Intrusion Detection Capturing of flow information plays an important role for network security, both for detection of security violation, and for subsequent defense. In case of a Denial of Service (DOS) attack, flow monitoring can allow detection of unusual load situations or suspicious flows. In a second step, flow analysis can be performed in order to gather information about the attacking flows, and for deriving a defense strategy. Intrusion detection is a potentially more demanding application which would not only look at specific characteristics of flows, but that may also use a stateful packet flow analysis for detecting specific, suspicious activities, or unusually frequent activities. Such activities may be characterized by specific communication patterns, detectable by characteristic sequences of certain packet types. 3.5 QoS Monitoring QoS monitoring is the non-intrusive (passive) measurement of quality parameters for single flows or traffic aggregates. In contrast to intrusive (active) measurements, non-intrusive measurements utilize the existing traffic in the network for QoS analysis. Since no test traffic is sent, non-intrusive measurements can only be applied in situations where the traffic of interest is already present in the network. One example application is the validation of QoS parameters negotiated in a service level specification (SLS). Non-intrusive measurements cannot provide the kind of controllable experiments that can be achieved with active measurements. On the other hand non-intrusive measurements do not suffer from undesired side effects caused by sending test traffic (e.g. additional load, Quittek et al. draft-ietf-ipfix-reqs-02.txt [Page 6] Internet Draft IPFIX Requirements March 2002 potential differences in treatment of test traffic and real customer traffic) QoS monitoring often requires the correlation of data from multiple measurement instances (e.g. for measuring one-way metrics). This requires proper clock synchronization of the involved measurement instances. For some measurements packet events at the different measurement points must be correlated. For this, the provisioning of post-processing functions (e.g. the generation of packet IDs) at the measurement instances would be useful. Furthermore, QoS monitoring can lead to a huge amount of measurement result data. Therefore it would highly benefit from mechanisms to reduce the measurement data, like aggregation of results and flow sampling. 4. Distinguishing Flows Packets are mapped to flows by evaluating their properties. Packets with common properties are considered to belong to the same flow. A packet showing at least one difference in the set of properties is considered to belong to a different flow. The following subsections list a set of properties which a metering process MUST, SHOULD, or MAY be able to evaluate for mapping packets to flows. Please note that requiring the ability to evaluate a certain property does not imply that this property must be evaluated for each packet. In other words, compliant with IPFIX means that the metering process in general must be able, via its configuration, to somehow support to distinguish flows via all the MUST fields, even if in certain circumstance/for certain applications, only a subset of the MUST fields is needed and only a subset of the MUST fields is effectively used to distinguish flows. Which combination of properties is evaluated for a particular measurement and how these properties are evaluated depends on the configuration of the IPFIX device. The configured choice of evaluated properties strongly depends on the environment and purpose of the measurement and on the information required by the collector. For specific deployments, only a subset of the REQUIRED properties listed below could be used to distinguish flows, in order to aggregate the flow records and reduce the number of flow records exported. On the other hand, some other deployments will require to distinguish flows by some extra parameters, like for example the TTL field of the IP header or the BGP Autonomous Systems. Quittek et al. draft-ietf-ipfix-reqs-02.txt [Page 7] Internet Draft IPFIX Requirements March 2002 4.1. Interfaces The metering process MUST be able to separate flows by the incoming interface or by the outgoing interface or by both of them. 4.2. IP Header Fields The metering process MUST, SHOULD, or MAY be able to separate flows by the following fields of the IP header as indicated. 1. source IP address (MUST) 2. destination IP address (MUST) 3. transport protocol type (TCP,UDP,ICMP,...) (MUST) 4. IP version number (SHOULD) This requirement only applies if the device is supporting more than one version of IP. For source address and destination address separating by full match MUST be supported as well as separation by a partial match (for example subnet masking). 4.3. Transport Header Fields The metering process MUST be able to separate flows by the port numbers of the transport header in case of TCP or UDP being used as transport protocol. Both, source and destination port number MUST be supported for distinguishing flows, individually as well as in combination. 4.4. MPLS Label If the metering process supports Multiprotocol Label Switching (MPLS, see [RFC3031]), then the measuring device MUST be able to separate flows by the MPLS label. 4.5. DiffServ Code Point If the IPFIX device supports Differentiated Services (DiffServ) then the metering process MUST be able to separate flows by the DiffServ Code Point (DSCP, see [RFC2474]). Quittek et al. draft-ietf-ipfix-reqs-02.txt [Page 8] Internet Draft IPFIX Requirements March 2002 4.6. Header Compression and Encryption If header compression or encryption is used, the metering process might not be able to access all header fields. In such a case only observation points at end points of header compression or of packet encryption are expected to meet the requirements stated in this section 4. 5. Metering Process The following are requirements for the metering process. All measurements MUST be conducted from the point of view of the observation point. 5.1. Reliability The metering process MUST either be reliable or missing reliability MUST be known and indicated. The metering process is reliable, if each packet passing the observation point is measured according to the configuration of the metering process. If, e.g. due to some overload, not all passing packets can be included into the metering process, then the metering process MUST be able to detect this failure and to report about it. 5.2. Sampling The metering process MAY support measuring traffic by packet sampling. If sampling is supported the sampling method and its parameters MUST be well defined. If sampling parameters are changed during operation, this MUST be indicated to all collectors receiving the affected flow records. 5.3. Overload Behavior In case of an overload, e. g. lack of memory or processing power, the metering process MAY change in order to cope with the lack of resources. Possible reactions include: - Reduce the number of flow accounts. This can be achieved by more coarse grained flow measurement or by a restriction of the flow accounts to a subset of the set of original ones. - Switch to sampling packets before they are processed by the meter or - if sampling is already performed - reduce the sampling rate. - Stop metering. Overload behavior is not restricted to the three options listed Quittek et al. draft-ietf-ipfix-reqs-02.txt [Page 9] Internet Draft IPFIX Requirements March 2002 above. But in any case, the overload behavior MUST be clearly defined and the collector MUST be able to distinguish the flow records exported before and after the metering process behavior change. For example in the case of switching to sampling, the collector MUST be able to distinguish the flow records generated with sampling from the flow records generated without sampling and the sampling method and all its parameters MUST be known or indicated. 5.4. Timestamps The metering process MUST be able to generate a timestamp for each observed packet. Please note that section 5.1 requires to offer reporting a timestamp for the first and the last observed packet of a flow. Therefore, the metering process MUST be able to store at least two timestamps per flow. 5.5. Time Synchronization Different metering process(es) and collector(s) SHOULD be time synchronized between each other. Using NTP or GPS are possible ways of achieving this. Selecting and deploying a method for time synchronization is not in the scope of IPFIX. 5.6. Timeout The metering process MUST be able to detect flow timeout. A flow is considered to be timed out if no packet of this flow has been observed for a given timeout interval. The metering process MAY support means for detecting the end of a flow before a time out occurs, for example by detecting the FIN or RST bits in a TCP connection. 5.7. Ignore Port Copy The metering process MAY be able to ignore packets which are generated by a port copy function acting at the same device. 6. Data Export The following are requirements for exporting measured flow data out of the IPFIX device. Beside requirements on the data transfer, we separate requirements concerning the information model from requirements concerning the data model. Furthermore, we list requirements on reporting times and events and on anonymization of records. Quittek et al. draft-ietf-ipfix-reqs-02.txt [Page 10] Internet Draft IPFIX Requirements March 2002 6.1. Information Model The information model for the flow information export is the list of attributes of a flow to be contained in the report (including the semantics of the attributes). This section lists attributes an export process MUST or MAY be able to report. This does not imply that a exported flow records MUST contain all REQUIRED attributes, but that it MUST be possible to configure the device in a way that all of the REQUIRED attributes are contained in the flow records for each measured flow. In other words, compliant with IPFIX means that the box in general must be able, via its configuration, to somehow support to report all the MUST fields, even if in certain circumstance/for certain applications, only a subset of the MUST fields is needed and only a subset of the MUST fields is effectively reported. Beyond that, the device might offer to report also further attributes not mentioned here. A particular flow record may contain some of the "REQUIRED" attributes as well as some additional ones, for example covering future technologies. The measuring device MUST be able to report the following attributes for each measured flow: 1. IP version number This requirement only applies if the device is supporting more than one version of IP. 2. source IP address 3. destination IP address 4. transport protocol type 5. source TCP/UDP port number 6. destination TCP/UDP port number 7. packet counter If a packet is fragmented, each fragment is counted as an individual packet. 8. byte counter 9. in case of IPv4: Type of Service 10. in case of IPv6: Flow Label 11. if BGP is supported: BGP AS# 12. if MPLS is supported: MPLS label 13. if DiffServ is supported: DSCP 14. timestamp of the first packet observed 15. timestamp of the last packet observed 16. if sampling is used: sampling method 17. if sampling is used: sampling parameters 18. unique identifier of the observation point Quittek et al. draft-ietf-ipfix-reqs-02.txt [Page 11] Internet Draft IPFIX Requirements March 2002 19. unique identifier of the export process The measuring device MAY be able to report the following attributes for each measured flow: 20. Time To Live 21. IP header flags 22. TCP header flags 23. dropped packet counter If a packet is fragmented, each fragment MUST be counted as an individual packet. This requirements does not apply to probes where the value of this counter is always zero. 24. fragmented packet counter counter of all packets for which the fragmented bit is set in the IP header 25. multicast replication factor the number of outgoing packets originating from a single incoming multicast packet 6.2 Data Model The data model describes how information is represented in flow records. The data model used for exporting flow information MAY be flexible concerning the flow attributes contained in flow records. A flexible record format would offer the possibility of defining records in a flexible (customizable) way regarding the number and type of contained attributes. The data model MUST be extensible for future attributes to be added. Even if a set of attributes is fixed in the flow record, the data model MUST provide a way of extending the record by configuration or for certain implementations. The Data Model SHOULD be independent of the underlying transport protocol, i.e. the data transfer. 6.3. Data Transfer Requirements for the data transfer include reliability and security requirements. These requirements do not apply to the measuring device alone, but also to the transport network. Consequently, the export process does not necessarily have to guarantee that all requirements are met. Particularly if the security requirements are already guaranteed by the network used for data transfer, then these requirements do not have to be considered anymore by the export process. Therefore, these requirements are OPTIONAL for the export process, although they may be REQUIRED for the data transfer as specified in the appendix. Quittek et al. draft-ietf-ipfix-reqs-02.txt [Page 12] Internet Draft IPFIX Requirements March 2002 6.3.1. Congestion Awareness For the data transfer a congestion aware protocol MUST be supported. 6.3.2. Reliability Absence of reliability of the data transfer MUST be indicated covering packet loss and packet reordering. Please note that if an unreliable transport protocol is used, reliability can be provided by higher layers. In such a case only lack of overall reliability MUST be indicated. For example reordering could be dealt with by adding a sequence number to each packet. 6.3.3. Security Confidentiality of transferred IPFIX data SHOULD be ensured. Integrity of transferred IPFIX data MUST be ensured. Authenticity of transferred IPFIX data MUST be ensured. 6.4. Push and Pull Mode Reporting In general, there are two ways of deciding on reporting times: push mode and pull mode. In push mode, the export process decides without an external trigger on when to send a report on measured flows. In pull mode, sending a report is triggered by an explicit request from a collector. The measuring device MUST support push mode reporting, it MAY support pull mode reporting. 6.5. Regular Reporting Interval The export process SHOULD be capable of reporting measured traffic data regularly according to a given interval length. 6.6. Notification on Specific Events The export process MAY be capable of sending notifications to a consumer of measure data, if a specific event occurs. Such an event might be the arrival of the first packet of a new flow, or the termination of a flow after flow timeout. Quittek et al. draft-ietf-ipfix-reqs-02.txt [Page 13] Internet Draft IPFIX Requirements March 2002 6.7. Anonymization The export process MAY be capable of anonymizing source and destination IP addresses in flow data before exporting them. It MAY support anonymization of port numbers and other fields. Please note that anonymization is not originally an application requirement, but derived from general requirements for treatment of traffic within a network. 7. Configuration The IPFIX device MUST provide a way of configuring the traffic measurement and the traffic data export. The following parameters SHOULD be configurable: 1. specification of the observation point, e.g. a list of interfaces to be monitored. 2. specifications of flows to be measured 3. reporting data format Specifying the reporting data format SHOULD include a selection of attributes to be reported for each flow. 4. flow timeouts The following parameters MAY be configurable: 5. notifications 6. sampling method and parameters, if feature is supported 7. flow anonymization, if feature is supported If configuration is done remotely, the IPFIX device SHOULD support security of the configuration including confidentiality, integrity and authenticity. The means used for remote configuration of IPFIX devices are out of the scope of this document. 8. General Requirements 8.1. Openness IPFIX specifications SHOULD be open to future technologies. This includes extensibility of configuration of measurement and reporting as well as extensibility of the reporting information model and data model. Quittek et al. draft-ietf-ipfix-reqs-02.txt [Page 14] Internet Draft IPFIX Requirements March 2002 8.2. Scalability concerning measuring devices Data collection from hundreds of different IPFIX devices MUST be supported. The collector MUST be able to distinguish several hundred IPFIX devices by their identifiers. 8.3. Several Collectors The exporting process MAY be able to export flow information to more than one collector. 9. Security Considerations This document describes requirements for IP Flow Information Export (IPFIX). It therefore also states the required security features for a future IPFIX protocol. Nevertheless, the suggestion of solutions for achieving the security properties is out of scope of this document and will be part of future IPFIX documents that specify IPFIX architecture and protocol. Like other requirements, the security requirements differ for the considered applications. The incentive to modify collected for accounting or intrusion detection for instance is usually higher than the incentive to change data collected for traffic profiling. Therefore the required security features are listed per application. Furthermore, the security requirements also differ with regard to the environment in which an IPFIX protocol is used (e.g. intra- or inter- domain). Some of these issues are part of the IPFIX architecture and with this out of scope of this document. Therefore this document also tries to consider security threats that can only occur in an insecure environment (e.g. where it can not be excluded, that an attacker might gain access to the network). Several security hazards also occur if the IPFIX device is configured remotely (e.g. access to the measurement process, forgery of configuration information, etc.). Nevertheless, this document specifies only what parameters SHOULD or MAY be configurable for the IPFIX device. It does not deal with requirements for a protocol for measurement configuration. Therefore security considerations regarding the measurement configuration are out of scope of this document. The following potential security hazards for an IPFIX protocol can be identified: - Disclosure of IP flow information data It may be required to keep measurement records confidential Quittek et al. draft-ietf-ipfix-reqs-02.txt [Page 15] Internet Draft IPFIX Requirements March 2002 between the involved parties. Observation of IP flow information data gives an attacker information about the active flows in the network, communication endpoints and traffic patterns. This information can not only be used to spy out user behavior but also to plan and conceal future attacks. Therefore the requirements document recommends to ensure the confidentiality of the transferred data. This can be done for instance by encryption. Furthermore, features for anonymization may be provided by the future IPFIX protocol. With this communication endpoints can be kept confidential. Anonymization is also a useful feature to allow measurements (e.g. by a third party) without violating privacy protection. - Forgery of exported IP flow information data Especially for applications like accounting or intrusion detection there are strong incentives (e.g. save money or prevent detection of an attack) to forge exported IP flow information records. This can be done either by altering data on the path or by exporting records from a device that pretends to be the IPFIX device. In order to make the IPFIX protocol resistant against such attacks this document requires to ensure authenticity and integrity of the data for the IPFIX data transfer. Special caution is required if security applications rely on IPFIX data With forgery of exported IP flow information data it is possible to trick on security applications. With this it can be for instance possible to pretend that a DoS attack happens without even launching a real attack. - Denial of Service (DoS) attacks DoS attacks on routers or other middleboxes that have the IPFIX protocol implemented would also affect the IPFIX protocol and impair the sending of IPFIX records. Nevertheless, since such hazards are not induced specifically by the IPFIX protocol the prevention of such attacks is out of scope of this document. IPFIX itself causes the following potential hazards for DoS attacks. It is always possible to overload the IPFIX device if it expects the reception of traffic. For IPFIX this can occur in two cases. First, if the protocol supports the pull mode (which is one option in this document) and expects requests. Secondly, if data is expected for remote measurement configuration. The first case could be prevented by ensuring authenticity for IPFIX requests. The second case should be addressed for the specification of an IPFIX remote Quittek et al. draft-ietf-ipfix-reqs-02.txt [Page 16] Internet Draft IPFIX Requirements March 2002 configuration mechanism and therefore is out of scope of this document. Also IPFIX collectors can become target of an DoS attack. This can be done by sending IPFIX data from a malicious device that pretends to be an IPFIX device. This can be prevented by ensuring authenticity of IPFIX data as stated in this document. It is also possible that collectors are flooded with IPFIX record from an authorized IPFIX devices for which the configuration was altered. Furthermore, malicious configuration or forgery of exported data can cause a loss or re-direction of flow information (e.g. if destination addresses for flow records are modified). This can lead to a disruption of upper layer services (accounting, intrusion detection, etc.) due to lack of IPFIX records. This can be prevented by controlling configuration access and by ensuring the integrity of exported data. 10. Acknowledgments We like to thank all the people contributing to the requirements discussion on the mailing list for a lot of valuable comments. 11. References [MIDBOXTAX] B. Carpenter, "Middleboxes: taxonomy and issues", RFC 3234, February 2002. [RFC3031] E. Rosen, A. Viswanathan, R. Callon, "Multiprotocol Label Switching Architecture", RFC 3031, January 2001. [RFC2474] K. Nichols, S. Blake, F. Baker, D. Black, "Definition of the Differentiated Services Field (DS Field) in the IPv4 and IPv6 Headers", RFC 2474, December 1998. [RFC2702] D. Awduche, J. Malcolm, J. Agogbua, M. O'Dell, J. McManus, "Requirements for Traffic Engineering Over MPLS", RFC 2702, September 1999. [RFC2274] U. Blumenthal, B. Wijnen "User-based Security Model (USM) for version 3 of the Simple Network Management Protocol (SNMPv3), RFC 2274, January 1998. Quittek et al. draft-ietf-ipfix-reqs-02.txt [Page 17] Internet Draft IPFIX Requirements March 2002 12. List of Authors Juergen Quittek NEC Europe Ltd., Network Laboratories Adenauerplatz 6 69115 Heidelberg Germany Phone: +49 6221 90511-15 EMail: quittek@ccrle.nec.de Tanja Zseby Fraunhofer Institute for Open Communication Systems (FOKUS) Kaiserin-Augusta-Allee 31 10589 Berlin Germany Phone: +49 30 3463 7153 Email: zseby@fokus.fhg.de Georg Carle Fraunhofer Institute for Open Communication Systems (FOKUS) Kaiserin-Augusta-Allee 31 10589 Berlin Germany Phone: +49 30 3463 7149 Email: carle@fokus.fhg.de Sebastian Zander Fraunhofer Institute for Open Communication Systems (FOKUS) Kaiserin-Augusta-Allee 31 10589 Berlin Germany Phone: +49 30 3463 7287 Email: zander@fokus.fhg.de Quittek et al. draft-ietf-ipfix-reqs-02.txt [Page 18] Internet Draft IPFIX Requirements March 2002 Benoit Claise Cisco Systems De Kleetlaan 6a b1 1831 Diegem Belgium Phone: +32 2 704 5622 Email: bclaise@cisco.com K.C. Norseth Enterasys Networks 2691 S. Decker Lake Lane Salt Lake City, Utah 84119 USA Phone: +1 801 887 9823 Email: knorseth@enterasys.com Quittek et al. draft-ietf-ipfix-reqs-02.txt [Page 19] Internet Draft IPFIX Requirements March 2002 Appendix: Derivation of Requirements form Target Applications The following table documents, how the requirements stated in sections 3-7 are derived from requirements of the applications listed in section 2. Used abbreviations: M = MUST S = SHOULD O = MAY (OPTIONAL) - = DONT CARE -----------------------------------------------------------------------. IPFIX | ----------------------------------------------------------------. | E: QoS Monitoring | | ----------------------------------------------------------. | | D: Attack/Intrusion Detection | | | ----------------------------------------------------. | | | C: Traffic Engineering | | | | ----------------------------------------------. | | | | B: Traffic Profiling | | | | | ----------------------------------------. | | | | | A: Usage-based Accounting | | | | | | ----------------------------------. | | | | | | | | | | | | | | Sect. | Requirement | A | B | C | D | E | IPFIX| |-------+-------------------------+-----+-----+-----+-----+-----+------| | 4. | DISTINGUISHING FLOWS | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 4. | Combination of | M | M | M | M | M | M | | | required attributes | | | | | | | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 4.1. | in/out IF | S | M | M | S | S | M | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 4.2. | src/dst address | M | M | M | M | M | M | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 4.2. | Masking of IP adresses | M | M | M | M | M | M | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 4.2. | transport protocol | M | M | - | M | M | M | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 4.2. | version field | - | S | S | O | O | S | | | | | | (b) | | | | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 4.3. | src/dst port | M | M | - | M | M | M | |-------+-------------------------+-----+-----+-----+-----+-----+------| Quittek et al. draft-ietf-ipfix-reqs-02.txt [Page 20] Internet Draft IPFIX Requirements March 2002 | Sect. | Requirement | A | B | C | D | E | IPFIX| |-------+-------------------------+-----+-----+-----+-----+-----+------| | 4.4. | MPLS label (a) | S | S | M | O | S | M | | | | | | (c) | | | | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 4.5. | DSCP (a) | M | S | M | O | M | M | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 5. | METERING PROCESS | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 5.1. | Reliability | M | S | S | S | S | | |-------+-------------------------+-----+-----+-----+-----+-----+ M | | 5.1. | Indication of | - | M | M | M | M | | | | missing reliability | | | | | | | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 5.2. | Sampling (g) | O | O | O | O | O | O | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 5.2. | Dynamic sampling | O | O | O | O | O | O | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 5.4. | Timestamping at | M | O | O | S | M | M | | | measurement device | | | | | | | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 5.5. | Time synchronization | S | S | S | S | S | S | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 5.6. | Flow timeout | M | S | - | O | O | M | | | | (d) | | | | | | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 5.7. | Ignore port copy | O | O | O | O | O | O | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6. | DATA EXPORT | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.1. | INFORMATION MODEL | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.1. | IP Version | - | M | M | O | O | M | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.1. | src/dst address | M | M | M | M | M | M | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.1. | transport protocol | M | M | - | M | M | M | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.1. | src/dst transport | M | M | - | M | M | M | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.1. | Packet counter (h) | S | M | M | S | S | M | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.1. | Byte counter | M | M | M | S | S | M | |-------+-------------------------+-----+-----+-----+-----+-----+------| Quittek et al. draft-ietf-ipfix-reqs-02.txt [Page 21] Internet Draft IPFIX Requirements March 2002 | Sect. | Requirement | A | B | C | D | E | IPFIX| |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.1. | Dropped Packet | O | M | M | S | M | M | | | Counter (h,i) | | | | | | | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.1. | ToS Byte | M | S | M | O | M | M | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.1. | Flow Label | M | S | M | O | M | M | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.1. | BGP AS# | - | S | M | - | - | M | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.1. | MPLS label (a) | S | S | M | O | S | M | | | | | | (c) | | | | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.1. | DSCP (a) | M | S | M | O | M | M | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.1. | Timestamps for | M | O | O | S | S | M | | | first/last packet | | | | | | | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.1. | Sampling methods | M | M | M | M | M | M | | | & parameters (k) | | | | | | | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.1. | observation point | M | M | M | M | M | M | | | identifier | | | | | | | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.1. | measuring device | M | M | M | M | M | M | | | identity | | | | | | | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.1. | TTL | O | O | O | O | O | O | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.1. | IP header flags | - | O | O | O | O | O | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.1. | TCP header flags | - | O | O | O | - | O | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.1. | Fragment counter | - | O | O | O | O | O | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.1. | Multicast | O | O | O | - | - | O | | | replication factor | | | | | | | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.1. | Flow configuration | O | O | O | O | O | O | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.2. | DATA MODEL | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.2. | Flexibility | O | O | O | O | O | O | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.2. | Extensibility | M | M | M | M | M | M | |-------+-------------------------+-----+-----+-----+-----+-----+------| Quittek et al. draft-ietf-ipfix-reqs-02.txt [Page 22] Internet Draft IPFIX Requirements March 2002 | Sect. | Requirement | A | B | C | D | E | IPFIX| |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.3. | DATA TRANSFER | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.3.1.| Congestion aware | M | M | M | M | M | M | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.3.2.| Reliability | M | S | S | S | S | M | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.3.3.| Confidentiality | S | S | S | S | S | S | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.3.4.| Integrity | M | M | M | M | M | M | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.3.5.| Authenticity | M | M | M | M | M | M | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.4. | REPORTING TIMES | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.4. | Push mode | M | O | O | M | S | M | | | | | (e) | (e) | |(e,f)| | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.4. | Pull mode | O | O | O | O | O | O | | | | | (e) | (e) | | (e) | | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.4.1.| Regular Interval | S | S | S | S | S | S | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.6. | Notifications | O | O | O | O | O | O | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.7. | Anonymization | O | O | O | O | O | O | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 7. | CONFIGURATION | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 7. | Config Measurement | M | M | M | M | M | M | | | & Data Export | | | | | | | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 7. | Config Observation Point| S | S | S | S | S | S | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 7. | Config Flow | S | S | S | S | S | S | | | Specifications | | | | | | | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 7. | Config Report | S | S | S | S | S | S | | | Data Format | | | | | | | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 7. | Config Flow Timeouts | S | S | S | S | O | S | |-------+-------------------------+-----+-----+-----+-----+-----+------| Quittek et al. draft-ietf-ipfix-reqs-02.txt [Page 23] Internet Draft IPFIX Requirements March 2002 | Sect. | Requirement | A | B | C | D | E | IPFIX| |-------+-------------------------+-----+-----+-----+-----+-----+------| | 7. | Config | O | O | O | O | O | O | | | Notifications | | | | | | | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 7. | Config Sampling | O | O | O | O | O | O | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 7. | Config Anonymization | O | O | O | O | O | O | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 7. | Config Security | O | O | O | O | O | O | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 8. | GENERAL REQUIREMENTS | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 8.1. | Openness | S | S | S | S | S | S | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 8.2. | Scalability: | | | | | | | | | data collection | M | S | M | O | S | M | | | from hundreds of | | | | | | | | | measurement devices | | | | | | | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 8.3. | Several Collectors | O | O | O | O | O | O | |-------+-------------------------+-----+-----+-----+-----+-----+------| Remarks: (a) If feature is supported. (b) The differentiation of IPv4 and IPv6 is for TE of importance. So we tended to make this a MUST. Nevertheless, a SHOULD seems to be sufficient to perform most TE tasks and allows us to have a SHOULD for IPFIX instead of a MUST. (c) For TE in an MPLS network the label is essential. Therefore a MUST is given here leading to a MUST in IPFIX. (d) Precise time-based accounting requires reaction to a flow timeout. (e) Either push or pull has to be supported. (f) Required, in order to immediately report drop indications for SLA validation. (g) If sampling is supported the parameters and methods MUST be well defined. (h) If a packet is fragmented, each fragment is counted as an individual packet. (i) Only if measurement is done on data path i.e. has access to forwarding decision. (k) If sampling is used. Quittek et al. draft-ietf-ipfix-reqs-02.txt [Page 24] --------------010708070700020904010606-- -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri Mar 1 12:26:35 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA08413 for ; Fri, 1 Mar 2002 12:26:35 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16gqX9-0005hF-00 for ipfix-list@mil.doit.wisc.edu; Fri, 01 Mar 2002 11:09:27 -0600 Received: from dplonka by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16gqX6-0005h5-00 for ipfix-req@net.doit.wisc.edu; Fri, 01 Mar 2002 11:09:24 -0600 Date: Fri, 1 Mar 2002 11:09:24 -0600 From: Dave Plonka To: ipfix-req@net.doit.wisc.edu Subject: Re: [ipfix-req] URL for updated requirements draft Message-ID: <20020301110924.A21573@doit.wisc.edu> Reply-To: plonka@doit.wisc.edu References: <3C7F911F.1090306@fokus.fhg.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0.1i In-Reply-To: <3C7F911F.1090306@fokus.fhg.de>; from zseby@fokus.gmd.de on Fri, Mar 01, 2002 at 03:33:03PM +0100 Organization: UW-Madison, DoIT, Network Services X-VMS-Error: %SYSTEM-W-SHARTOOBIG, new version of shareable image too big X-Shakespearean-Insult: Thou fawning ill-breeding giglet Precedence: bulk Sender: majordomo listserver I've also placed the update document on the IPFIX site and linked it from the index. You can find it here: http://ipfix.doit.wisc.edu/req/ (I think it will be a while before it's on the IETF site sine we're pass the cut-off date for IETF 53.) Dave On Fri, Mar 01, 2002 at 03:33:03PM +0100, Tanja Zseby wrote: > > Hi all, > we submitted an updated version of the requirements document (see > attached document). It contains only small changes, mainly in the > terminology section. > Regards, > Tanja -- plonka@doit.wisc.edu http://net.doit.wisc.edu/~plonka ARS:N9HZF Madison, WI -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Sun Mar 3 17:46:46 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA00646 for ; Sun, 3 Mar 2002 17:46:46 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16heE0-0003mS-00 for ipfix-list@mil.doit.wisc.edu; Sun, 03 Mar 2002 16:13:00 -0600 Received: from eng4.oar.net ([192.148.244.24]) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 16heDy-0003mK-00 for ipfix-req@net.doit.wisc.edu; Sun, 03 Mar 2002 16:12:58 -0600 Received: (qmail 32416 invoked by uid 4454); 3 Mar 2002 22:12:38 -0000 Date: Sun, 3 Mar 2002 17:12:38 -0500 From: Mark Fullmer To: Peter Ludemann Cc: ipfix-req@net.doit.wisc.edu Subject: Re: [ipfix-req] Collector Failover as a requirement Message-ID: <20020303171238.A31778@net.ohio-state.edu> References: <20020223235437.A84125@net.ohio-state.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: ; from peter.ludemann@xacct.com on Mon, Feb 25, 2002 at 12:21:29AM -0800 Precedence: bulk Sender: majordomo listserver (sorry for the late reply...) On Mon, Feb 25, 2002 at 12:21:29AM -0800, Peter Ludemann wrote: > > In a normal deployment, with the collector close to the sender, "reliable" > isn't more resource-intensive. And TCP is a reasonable choice; the > latest NFS is done over TCP because it's as efficient as using > UDP and avoids re-inventing wheels for things like congestion control. > When a connection is broken, TCP uses longer and longer intervals when > retransmitting, so the execution overhead remains small. In an _ideal_ deployment the collector is directly connected to the exporter. Many times this is not the case. Rack space in co-lo facilities may not be availabe, the cost of maintaining collectors around the country may be too high, routers may not have the space for a linecard with a LAN interface for the collector, etc. Most NetFlow deployments I've seen use centralized collectors... The buffer space for implementing exports over TCP may not be available on the hardware -- ie routers that have been augmented with flow export capability. Flow exports can be very bursty and require a lot of socket buffer space that is not available to some vendor implementations. Depending on the application a small percentage of lost flow records may not be an issue. Example, one router exported around 70,000,000 flows in a day (a few days ago). The collector stats show that around 19,000 were lost, an export record rate of around 2.7x10^-4, or a little over 600 packets. For traffic engineering purposes this doesn't matter, especially since the flows are generated from a sample rate of 1/100. If this were an issue the collector could be directly connected to the router to reduce the loss. For an IDS or other applications lost export records may be more of an issue. An IDS exporter would probably have more resources and be able to use TCP, a disk based backing store, or some other methods to make the exporter -> collector connection more reliable. There are two camps of vendors here, one which has the resources to implement TCP or some other form of retransmits for the flow exports, and one that does not. For IPFIX to be well adopted the export mechanism needs to satisfy both. This isn't difficult with a separate control and export mechanism, and can easly be added to draft-gsadasiv-ipfix-proposal-00.txt. > One more thing about unreliable protocols ... they're attractive in theory > but in practice we find ourselves building workarounds to make them > seem reliable (this applies to both NetFlow and SNMP). Any data delivery > method that avoids reliability simply pushes things on to another > part of the system. If using a reliable and congestion aware protocol were that big of an issue for the applications NetFlow is used for today it would not be so widely deployed. IPFIX needs to take this into account while being flexible enough to support a "better" (TCP/SCTP) transport for the exports. > > > o make the control connection in draft-gsadasiv-ipfix-proposal-00.txt > > required and TCP. This removes the need for retransmits or > > periodic updates of templates to the collector. When the collector > > boots up it won't have to wait for templates. This also can > > stop the router from generating flows to a non existant collector. > > I don't think that only TCP should be required. I would also allow SCTP > ... it's better than TCP for fail-over, but not widely available yet. TCP with a keepalive mechanism will work fine. Requiring SCTP is an unnecessary deployment barrier. char:~% uname -r 4.4-RELEASE char:~% man -k sctp sctp: nothing appropriate (yes, I know there are SCTP implementation for FreeBSD, Linux, etc) They're not widely available/integrated/burned in yet. > > o Implement a keepalive mechanism where the collector will ACK the > > highest sequence number flow record it has processed per (any?) > > "observation domain" The keepalive can be optional -- ie the > > exporter can send a keepalive of N to inform the collector it > > expects a keepalive every N seconds or 0 to disable it. Default > > disabled. > > This is essentially the way that CRANE works. However, I don't think > that a keepalive gives any advantage ... the sender should just time > out and failover if it doesn't receive a data ACK in time (there's no > need for keepalives when no data are being sent). [Downstream > deduplication will always be needed for handling failures because > ACKs can get lost.] Acking every data packet is a lot different from a keepalive... mark -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Sun Mar 3 17:51:46 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA00680 for ; Sun, 3 Mar 2002 17:51:42 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16hePc-0003zc-00 for ipfix-list@mil.doit.wisc.edu; Sun, 03 Mar 2002 16:25:00 -0600 Received: from eng4.oar.net ([192.148.244.24]) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 16hePa-0003zX-00 for ipfix-req@net.doit.wisc.edu; Sun, 03 Mar 2002 16:24:58 -0600 Received: (qmail 32458 invoked by uid 4454); 3 Mar 2002 22:24:39 -0000 Date: Sun, 3 Mar 2002 17:24:39 -0500 From: Mark Fullmer To: Kevin Zhang Cc: Peter Ludemann , ipfix-req@net.doit.wisc.edu Subject: Re: [ipfix-req] Collector Failover as a requirement Message-ID: <20020303172439.B31778@net.ohio-state.edu> References: <20020223235437.A84125@net.ohio-state.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: ; from kevin.zhang@xacct.com on Mon, Feb 25, 2002 at 10:28:43PM -0500 Precedence: bulk Sender: majordomo listserver On Mon, Feb 25, 2002 at 10:28:43PM -0500, Kevin Zhang wrote: > This is a very good idea. keepalive can be piggybacked through ACKs from collectors. This will not only serve as indication of collector fault conditions as well as congestion, it can also improve transport reliability. This should be done on the IPFIX protocol layer whether TCP or UDP is used as the transport protocol. Yes, a keepalive will be required regardless of the flow export (data) channel to support collector failover. This complicates multicast a little bit since the multicast collectors will need some way to access the templates. Multicast clients may need to connect to the exporter for this information... > > > Simply counting the lost flow records may not be enough as values associated with each record are different (e.g. start of a flow, end of a flow, interim records). We should really base our design over a reliable transport layer that offers congestion control. UDP can optionally be used for limited configurations. Ideally the exporter will periodically send down other information on the control channel such flows dropped internally (to transmit buffer, etc), packet size distributions, etc. draft-gsadasiv-ipfix-proposal-00 seems to allude to this. mark -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Sun Mar 3 17:55:25 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA00714 for ; Sun, 3 Mar 2002 17:55:25 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16hefo-0004Ja-00 for ipfix-list@mil.doit.wisc.edu; Sun, 03 Mar 2002 16:41:44 -0600 Received: from eng4.oar.net ([192.148.244.24]) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 16hefm-0004JS-00 for ipfix-req@net.doit.wisc.edu; Sun, 03 Mar 2002 16:41:42 -0600 Received: (qmail 32516 invoked by uid 4454); 3 Mar 2002 22:41:23 -0000 Date: Sun, 3 Mar 2002 17:41:23 -0500 From: Mark Fullmer To: Ganesh Sadasivan Cc: ipfix-req@net.doit.wisc.edu Subject: Re: [ipfix-req] Collector Failover as a requirement Message-ID: <20020303174123.A32465@net.ohio-state.edu> References: <3C7A82AA.BD814C45@cisco.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: <3C7A82AA.BD814C45@cisco.com>; from gsadasiv@cisco.com on Mon, Feb 25, 2002 at 10:30:03AM -0800 Precedence: bulk Sender: majordomo listserver On Mon, Feb 25, 2002 at 10:30:03AM -0800, Ganesh Sadasivan wrote: > > > o Change the sequence number in draft-gsadasiv-ipfix-proposal-00.txt > > > to number of flows instead of number of packets so that the collector > > > can count lost flow records as in NetFlow v5,6,7,8. > > There are other means to achive the same in the new protocol. Basically the > flowsets tell the length and using the size of each record in the flowset, one > can calculate the # of records. I've read through the draft again and I don't see how a collector can figure out how many flow exports it missed when a packet is dropped. The size of the record and the flowset(s) are unknown for a missing packet. mark -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Sun Mar 3 23:05:01 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id XAA05215 for ; Sun, 3 Mar 2002 23:05:00 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16hjMO-0002Xz-00 for ipfix-list@mil.doit.wisc.edu; Sun, 03 Mar 2002 21:42:00 -0600 Received: from sj-msg-core-4.cisco.com ([171.71.163.10]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16hjMM-0002XD-00 for ipfix-req@net.doit.wisc.edu; Sun, 03 Mar 2002 21:41:58 -0600 Received: from mira-sjcd-1.cisco.com (mira-sjcd-1.cisco.com [171.69.43.44]) by sj-msg-core-4.cisco.com (8.11.3/8.9.1) with ESMTP id g243fRT19152; Sun, 3 Mar 2002 19:41:27 -0800 (PST) Received: from cisco.com ([10.25.5.227]) by mira-sjcd-1.cisco.com (Mirapoint) with ESMTP id ABJ08311; Sun, 3 Mar 2002 19:41:55 -0800 (PST) Message-ID: <3C82ECE6.4ECC6C36@cisco.com> Date: Sun, 03 Mar 2002 19:41:27 -0800 From: Ganesh Sadasivan X-Mailer: Mozilla 4.76 [en]C-CCK-MCD (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: Mark Fullmer CC: ipfix-req@net.doit.wisc.edu Subject: Re: [ipfix-req] Collector Failover as a requirement References: <3C7A82AA.BD814C45@cisco.com> <20020303174123.A32465@net.ohio-state.edu> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Hi Mark, Mark Fullmer wrote: > On Mon, Feb 25, 2002 at 10:30:03AM -0800, Ganesh Sadasivan wrote: > > > > > o Change the sequence number in draft-gsadasiv-ipfix-proposal-00.txt > > > > to number of flows instead of number of packets so that the collector > > > > can count lost flow records as in NetFlow v5,6,7,8. > > > > There are other means to achive the same in the new protocol. Basically the > > flowsets tell the length and using the size of each record in the flowset, one > > can calculate the # of records. > > I've read through the draft again and I don't see how a collector > can figure out how many flow exports it missed when a packet > is dropped. The size of the record and the flowset(s) are > unknown for a missing packet. An export packet can have records of more than one flow set (type). Probably a cumulative flow record # per flow set is what is necessary to figure the loss of flow records per type. Getting a general count I guess is not much useful. This is my initial thought. Will get back to you with a clearer answer. Ganesh > > > mark -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon Mar 4 02:22:04 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id CAA16208 for ; Mon, 4 Mar 2002 02:22:04 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16hmZL-0006vM-00 for ipfix-list@mil.doit.wisc.edu; Mon, 04 Mar 2002 01:07:35 -0600 Received: from usmail.xacct.com ([204.253.100.12]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16hmZJ-0006uj-00 for ipfix-req@net.doit.wisc.edu; Mon, 04 Mar 2002 01:07:33 -0600 Received: from peter (cpe-66-87-83-93.ca.sprintbbd.net [66.87.83.93]) by usmail.xacct.com (8.11.2/8.11.2) with SMTP id g2478Rl14840; Sun, 3 Mar 2002 23:08:28 -0800 From: "Peter Ludemann" To: "Mark Fullmer" , "Kevin Zhang" Cc: Subject: RE: [ipfix-req] Collector Failover as a requirement Date: Sun, 3 Mar 2002 23:06:59 -0800 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 In-Reply-To: <20020303172439.B31778@net.ohio-state.edu> Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Keepalives are not needed with TCP; ACKs with timeouts are needed (TCP's ACKs are not sufficient). In a multicast situation, the ACKs can be kept on a per-receiver basis (but why do you need this? failover can be done other ways). As to lost data ... with multiple output "templates", one could output information about losses in a separate template (which begs the question about losing loss data).. Alternatively, the information could be available in a MIB. - peter > -----Original Message----- > From: majordomo listserver [mailto:majordomo@mil.doit.wisc.edu]On Behalf > Of Mark Fullmer > Sent: Sunday, March 03, 2002 2:25 PM > To: Kevin Zhang > Cc: Peter Ludemann; ipfix-req@net.doit.wisc.edu > Subject: Re: [ipfix-req] Collector Failover as a requirement > > > On Mon, Feb 25, 2002 at 10:28:43PM -0500, Kevin Zhang wrote: > > > This is a very good idea. keepalive can be piggybacked through > ACKs from collectors. This will not only serve as indication of > collector fault conditions as well as congestion, it can also > improve transport reliability. This should be done on the IPFIX > protocol layer whether TCP or UDP is used as the transport protocol. > > Yes, a keepalive will be required regardless of the flow export > (data) channel to support collector failover. This complicates > multicast a little bit since the multicast collectors will need > some way to access the templates. Multicast clients may need > to connect to the exporter for this information... > > > > > > Simply counting the lost flow records may not be enough as > values associated with each record are different (e.g. start of a > flow, end of a flow, interim records). We should really base our > design over a reliable transport layer that offers congestion > control. UDP can optionally be used for limited configurations. > > Ideally the exporter will periodically send down other > information on the control channel such flows dropped > internally (to transmit buffer, etc), packet size > distributions, etc. draft-gsadasiv-ipfix-proposal-00 > seems to allude to this. > > mark > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in > message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon Mar 4 13:05:13 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA06629 for ; Mon, 4 Mar 2002 13:05:13 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16hwJ3-0006j2-00 for ipfix-list@mil.doit.wisc.edu; Mon, 04 Mar 2002 11:31:25 -0600 Received: from host3.riverstonenet.com ([63.113.148.3] helo=exc-sc1.yagosys.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16hwJ1-0006iC-00 for ipfix-data@net.doit.wisc.edu; Mon, 04 Mar 2002 11:31:23 -0600 Received: from riverstonenet.com (134.141.180.86 [134.141.180.86]) by exc-sc1.yagosys.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13) id 1PZZB8CG; Mon, 4 Mar 2002 09:30:04 -0800 Message-ID: <3C83AF1C.E7491AC@riverstonenet.com> Date: Mon, 04 Mar 2002 12:30:04 -0500 From: calato@riverstonenet.com X-Mailer: Mozilla 4.75C-CCK-MCD NSCPCD475 [en] (X11; U; SunOS 5.6 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: Reinaldo Penno CC: "Norseth, KC" , ipfix-data@net.doit.wisc.edu Subject: Re: [ipfix-data] Data doc References: <4F973E944951D311B6660008C7917CF008CBBDEB@zsc4c008.us.nortel.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Sorry for the delay, I was on vacation the latter part of the week. I don't agree with the section on middle boxes. As was said before, we'll try and come to an agreement at the next meeting. Paul Reinaldo Penno wrote: > > Hello KC, Paul > > here it goes the sections I promised in one single doc. > > regards, > > Reinaldo > > > > Name: draft-ietf-ipfix-data-reinaldo-00.txt > draft-ietf-ipfix-data-reinaldo-00.txt Type: Plain Text (text/plain) > Encoding: quoted-printable -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon Mar 4 13:24:56 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA07821 for ; Mon, 4 Mar 2002 13:24:55 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16hwuw-0007hb-00 for ipfix-list@mil.doit.wisc.edu; Mon, 04 Mar 2002 12:10:34 -0600 Received: from rip.psg.com ([147.28.0.39]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16hwuu-0007gr-00 for ipfix@net.doit.wisc.edu; Mon, 04 Mar 2002 12:10:32 -0600 Received: from randy by rip.psg.com with local (Exim 4.00) id 16hwuP-000PJM-00; Mon, 04 Mar 2002 10:10:01 -0800 From: Randy Bush MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit To: Allison Mankin Cc: Ganesh Sadasivan , Will Eatherton , blaise@cisco.com, ipfix@net.doit.wisc.edu, sob@harvard.edu, bwijnen@lucent.com Subject: [ipfix] Re: FW: congestion awareness References: <3C83B2EB.77231684@cisco.com> Message-Id: Date: Mon, 04 Mar 2002 10:10:01 -0800 Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit > The language about recovery from loss must be omitted > if this is the "no loss" case - there's really no middle > ground - either the protocol supplies reliability or it > tolerates loss. If the former, it needs to be congestion- > avoiding in the Internet. completely agree. and note i have moved this discussion to the mailing list. to quote comments from this ever-ongoing, and inappropriately private, discussion with a vendor. > in fact, isps today collect netflow data over the open internet. > it is not some weird possibility, but fact. we may not like it. > we may think it inadvisable. blah blah. but providers DO it. > > and they will continue to do it, whether we like it, advise it, > ... and, since they will transport over the open internet, it > MUST be congestion friendly. > > as the charter says, and for the reasons repeatedly discussed, it > is *extremely* unlikely that anything will make it through the > process which is not congestion friendly. of course, when the data are not going over the internet, then you may use whatever protocol you wish. but, since it is not internet, there is no need to discuss it in the ietf, is there? congestion control will be mandatory to implement. congestion control will be mandatory to use when transportng over the internet. how you *automatically* detect that you are not transporting on the internet, and *guaranty* congestion-friendly transport when you are on the internet, is left as an exercise for those wanting to take liberties. as allison points out, the charter is fairly clear on this, but the current drafts do not match it. these will need to become charter, and congestion control, compliant before these documents go to the iesg or we are wasting a lot of people's time. randy -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon Mar 4 14:21:24 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA15020 for ; Mon, 4 Mar 2002 14:21:24 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16hxlu-0000yw-00 for ipfix-list@mil.doit.wisc.edu; Mon, 04 Mar 2002 13:05:18 -0600 Received: from mailhost.auckland.ac.nz ([130.216.1.4]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16hxls-0000yo-00 for ipfix@net.doit.wisc.edu; Mon, 04 Mar 2002 13:05:16 -0600 Received: from auckland.ac.nz (root@ccu1.auckland.ac.nz [130.216.3.1]) by mailhost.auckland.ac.nz (8.9.2/8.9.2/8.9.2-ua) with ESMTP id IAA05742 for ; Tue, 5 Mar 2002 08:05:10 +1300 (NZDT) Message-Id: <200203041905.IAA05742@mailhost.auckland.ac.nz> Date: Mon, 4 Mar 2002 11:07:16 -0800 (PST) From: n.brownlee@auckland.ac.nz Subject: [ipfix] Congestion awareness, reliability To: ipfix@net.doit.wisc.edu MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Precedence: bulk Sender: majordomo listserver Hello all: Our charter says that whatever protocol we select for IPFIX transport, it must be congestion-aware. In thinking about this, it seems to me that: * When reliable transport is needed, IPFIX could use TCP or SCTP. SCTP has the advantage that it's designed not only to provide reliable transport, but also to handle fail-over between collectors. This could be an elegant way to provide that for IPFIX - no sense in re-inventing this stuff! What do others think about using SCTP? * For unreliable transport, we still have to use a congestion-aware protocol. Right now it's not too obvious what is the best way to do that. When I asked the Transport Area Directors about this, Allison said that "the TSV ADs are not encouraging unreliable SCTP, but we have a congestion controlled UDP technique,the Congestion Manager, RFC 3124, and we have work starting on a distinct transport to do more coherent congestion avoiding unreliable transport (draft-kohler-dcp-*)" I know that lots of people use UDP now with NetFlow, mostly on 'private' networks - that's fine. After all, if you have really large volumes of traffic to measure, you'll probably engineer a system which does as much as possible of its data reduction close to the exporter. For an IPFIX standard, we must assume that some users will use unreliable transport over the Internet, which is why the standard has to use a congestion-aware protocol. So, maybe WG folk could read kohler's DCP drafts and comment on DCP as a transport mechanism for iPFIX? And/or RFC 3124, Congestion Manager?? Cheers, Nevil ----------------------------------------------------------------------- Nevil Brownlee Director, Technology Development Phone: +64 9 373 7599 x8941 ITSS, The University of Auckland FAX: +64 9 373 7021 Private Bag 92019, Auckland, New Zealand -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon Mar 4 17:11:23 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA26713 for ; Mon, 4 Mar 2002 17:11:23 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16i0MU-0004TO-00 for ipfix-list@mil.doit.wisc.edu; Mon, 04 Mar 2002 15:51:14 -0600 Received: from sj-msg-core-3.cisco.com ([171.70.157.152]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16i0MS-0004SV-00 for ipfix@net.doit.wisc.edu; Mon, 04 Mar 2002 15:51:12 -0600 Received: from postal.cisco.com (postal.cisco.com [171.71.160.26]) by sj-msg-core-3.cisco.com (8.11.3/8.9.1) with ESMTP id g24LoU309869; Mon, 4 Mar 2002 13:50:30 -0800 (PST) Received: from WILLW2K1 (sjc-vpn3-560.cisco.com [10.21.66.48]) by postal.cisco.com (Mirapoint) with SMTP id AAN06280; Mon, 4 Mar 2002 13:50:51 -0800 (PST) From: "Will Eatherton" To: "Randy Bush" , "Allison Mankin" Cc: "Ganesh Sadasivan" , , , , Subject: RE: [ipfix] Re: FW: congestion awareness Date: Mon, 4 Mar 2002 13:50:21 -0800 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) In-Reply-To: X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4807.1700 Importance: Normal Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Rand and group, I was hoping to educate myself in this thread on the reasons for requiring that there be no option of using IPFIX without congestion control if an ISP and vendor so desire. At the bottom of the reasons was not concern in the general case (I provided several scenarios and got no counter arguments) but only concern for the cases of ISP/Vendor stupidity and case of router's being compromised. TCP is essentially a way to try and provider boundries in those cases (though the boundry seems a bit superficial). Intranet use of IPFIX is not to be covered in any way, and that should be made very clear (also makes IPFIX irrelevant to a lot of folks). With all this understood I will terminate my concern for this topic and simply suggest that these points be well documented. Will > -----Original Message----- > From: majordomo listserver [mailto:majordomo@mil.doit.wisc.edu]On Behalf > Of Randy Bush > Sent: Monday, March 04, 2002 10:10 AM > To: Allison Mankin > Cc: Ganesh Sadasivan; Will Eatherton; blaise@cisco.com; > ipfix@net.doit.wisc.edu; sob@harvard.edu; bwijnen@lucent.com > Subject: [ipfix] Re: FW: congestion awareness > > > > The language about recovery from loss must be omitted > > if this is the "no loss" case - there's really no middle > > ground - either the protocol supplies reliability or it > > tolerates loss. If the former, it needs to be congestion- > > avoiding in the Internet. > > completely agree. and note i have moved this discussion > to the mailing list. > > to quote comments from this ever-ongoing, and inappropriately > private, discussion with a vendor. > > > in fact, isps today collect netflow data over the open internet. > > it is not some weird possibility, but fact. we may not like it. > > we may think it inadvisable. blah blah. but providers DO it. > > > > and they will continue to do it, whether we like it, advise it, > > ... and, since they will transport over the open internet, it > > MUST be congestion friendly. > > > > as the charter says, and for the reasons repeatedly discussed, it > > is *extremely* unlikely that anything will make it through the > > process which is not congestion friendly. > > of course, when the data are not going over the internet, then you > may use whatever protocol you wish. but, since it is not internet, > there is no need to discuss it in the ietf, is there? > > congestion control will be mandatory to implement. congestion > control will be mandatory to use when transportng over the > internet. how you *automatically* detect that you are not > transporting on the internet, and *guaranty* congestion-friendly > transport when you are on the internet, is left as an exercise for > those wanting to take liberties. > > as allison points out, the charter is fairly clear on this, but the > current drafts do not match it. these will need to become charter, > and congestion control, compliant before these documents go to the > iesg or we are wasting a lot of people's time. > > randy > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in > message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon Mar 4 17:22:13 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA27430 for ; Mon, 4 Mar 2002 17:22:12 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16i0bj-0004rz-00 for ipfix-list@mil.doit.wisc.edu; Mon, 04 Mar 2002 16:06:59 -0600 Received: from rip.psg.com ([147.28.0.39]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16i0bh-0004rN-00 for ipfix@net.doit.wisc.edu; Mon, 04 Mar 2002 16:06:57 -0600 Received: from randy by rip.psg.com with local (Exim 4.00) id 16i0b7-000ACq-00; Mon, 04 Mar 2002 14:06:21 -0800 From: Randy Bush MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit To: "Will Eatherton" Cc: "Allison Mankin" , "Ganesh Sadasivan" , , , , Subject: RE: [ipfix] Re: FW: congestion awareness References: Message-Id: Date: Mon, 04 Mar 2002 14:06:21 -0800 Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit > Intranet use of IPFIX is not to be covered in any way, and that should be > made very clear (also makes IPFIX irrelevant to a lot of folks). > >> congestion control will be mandatory to implement. congestion >> control will be mandatory to use when transportng over the >> internet. how you *automatically* detect that you are not >> transporting on the internet, and *guaranty* congestion-friendly >> transport when you are on the internet, is left as an exercise for >> those wanting to take liberties. randy -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Mar 5 01:33:53 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id BAA10817 for ; Tue, 5 Mar 2002 01:33:53 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16i8AK-0006tD-00 for ipfix-list@mil.doit.wisc.edu; Tue, 05 Mar 2002 00:11:12 -0600 Received: from eng4.oar.net ([192.148.244.24]) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 16i8AG-0006t3-00 for ipfix-req@net.doit.wisc.edu; Tue, 05 Mar 2002 00:11:08 -0600 Received: (qmail 40866 invoked by uid 4454); 5 Mar 2002 06:10:46 -0000 Date: Tue, 5 Mar 2002 01:10:46 -0500 From: Mark Fullmer To: Peter Ludemann Cc: Kevin Zhang , ipfix-req@net.doit.wisc.edu Subject: Re: [ipfix-req] Collector Failover as a requirement Message-ID: <20020305011046.A40777@net.ohio-state.edu> References: <20020303172439.B31778@net.ohio-state.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: ; from peter.ludemann@xacct.com on Sun, Mar 03, 2002 at 11:06:59PM -0800 Precedence: bulk Sender: majordomo listserver On Sun, Mar 03, 2002 at 11:06:59PM -0800, Peter Ludemann wrote: > Keepalives are not needed with TCP; ACKs with timeouts are needed (TCP's > ACKs are not sufficient). In a multicast situation, the ACKs can be kept on > a per-receiver basis (but why do you need this? failover can be done other > ways). The context of my original message was using TCP for the control channel. Regardless, a periodic keepalive is still needed even if the collector is ACKing messages from the exporter. Consider the case when the exporter is not sending any data. It's also important for the collector to be able to detect a dead exporter so it can mark the data period as such instead of reporting no flow activity. Reliable multicast is an entirely different topic. I'm assuming a simple case where the exporter is using unreliable (no ACK's) UDP to a multicast address. With the current NetFlow (v1,5,6,7,8.x) multicast is a non issue since the templates are static C struct's. The exports packets can be sent (actually reflected from unicast since Cisco doesn't implement this) out unmodified. I've used multicast NetFlow to reduce fanout when multiple collectors are processing the same data, not for failover. > As to lost data ... with multiple output "templates", one could output > information about losses in a separate template (which begs the question > about losing loss data).. Alternatively, the information could be available > in a MIB. Or as a field in the template... mark -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Mar 5 10:43:39 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA05136 for ; Tue, 5 Mar 2002 10:43:38 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16iGeQ-000521-00 for ipfix-list@mil.doit.wisc.edu; Tue, 05 Mar 2002 09:14:50 -0600 Received: from host3.riverstonenet.com ([63.113.148.3] helo=exc-sc1.yagosys.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16iGeO-00051R-00 for ipfix@net.doit.wisc.edu; Tue, 05 Mar 2002 09:14:48 -0600 Received: from riverstonenet.com (134.141.180.76 [134.141.180.76]) by exc-sc1.yagosys.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13) id 1PZZCNQZ; Tue, 5 Mar 2002 07:13:27 -0800 Message-ID: <3C84E096.70E419DF@riverstonenet.com> Date: Tue, 05 Mar 2002 10:13:27 -0500 From: calato@riverstonenet.com X-Mailer: Mozilla 4.75C-CCK-MCD NSCPCD475 [en] (X11; U; SunOS 5.6 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: Randy Bush CC: Allison Mankin , Ganesh Sadasivan , Will Eatherton , blaise@cisco.com, ipfix@net.doit.wisc.edu, sob@harvard.edu, bwijnen@lucent.com Subject: Re: [ipfix] Re: FW: congestion awareness References: <3C83B2EB.77231684@cisco.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Randy Bush wrote: > > > The language about recovery from loss must be omitted > > if this is the "no loss" case - there's really no middle > > ground - either the protocol supplies reliability or it > > tolerates loss. If the former, it needs to be congestion- > > avoiding in the Internet. > > completely agree. and note i have moved this discussion > to the mailing list. > > to quote comments from this ever-ongoing, and inappropriately > private, discussion with a vendor. > > > in fact, isps today collect netflow data over the open internet. > > it is not some weird possibility, but fact. we may not like it. > > we may think it inadvisable. blah blah. but providers DO it. > > > > and they will continue to do it, whether we like it, advise it, > > ... and, since they will transport over the open internet, it > > MUST be congestion friendly. > > > > as the charter says, and for the reasons repeatedly discussed, it > > is *extremely* unlikely that anything will make it through the > > process which is not congestion friendly. > > of course, when the data are not going over the internet, then you > may use whatever protocol you wish. but, since it is not internet, > there is no need to discuss it in the ietf, is there? Forgive the dumb question, I haven't been an IETF'er for very long, but are all efforts in the IETF only concered with the Internet. I know "Internet" is in the name, but for some reason I thought it had since become broader. I for one, did not realize that IPFIX was only concerned with the transport of IP flows over the Internet. I thought it was to report IP flows in any environment, private or public. Is that not the case? > > congestion control will be mandatory to implement. congestion > control will be mandatory to use when transportng over the > internet. how you *automatically* detect that you are not > transporting on the internet, and *guaranty* congestion-friendly > transport when you are on the internet, is left as an exercise for > those wanting to take liberties. > > as allison points out, the charter is fairly clear on this, but the > current drafts do not match it. these will need to become charter, > and congestion control, compliant before these documents go to the > iesg or we are wasting a lot of people's time. > > randy > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Mar 5 15:28:07 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA26365 for ; Tue, 5 Mar 2002 15:28:06 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16iL3Y-0003U6-00 for ipfix-list@mil.doit.wisc.edu; Tue, 05 Mar 2002 13:57:04 -0600 Received: from mailhost.auckland.ac.nz ([130.216.1.4]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16iL3W-0003Ti-00 for ipfix@net.doit.wisc.edu; Tue, 05 Mar 2002 13:57:02 -0600 Received: from auckland.ac.nz (root@ccu1.auckland.ac.nz [130.216.3.1]) by mailhost.auckland.ac.nz (8.9.2/8.9.2/8.9.2-ua) with ESMTP id IAA01579; Wed, 6 Mar 2002 08:56:35 +1300 (NZDT) Message-Id: <200203051956.IAA01579@mailhost.auckland.ac.nz> Date: Tue, 5 Mar 2002 11:58:36 -0800 (PST) From: n.brownlee@auckland.ac.nz Subject: Re: [ipfix] Re: FW: congestion awareness To: calato@riverstonenet.com cc: randy@psg.com, mankin@psg.com, gsadasiv@cisco.com, will@cisco.com, blaise@cisco.com, ipfix@net.doit.wisc.edu, sob@harvard.edu, bwijnen@lucent.com In-Reply-To: <3C84E096.70E419DF@riverstonenet.com> MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Precedence: bulk Sender: majordomo listserver Hi Paul: > Forgive the dumb question, I haven't been an IETF'er > for very long, but are all efforts in the IETF > only concered with the Internet. I know "Internet" is > in the name, but for some reason I thought it had since > become broader. > > I for one, did not realize that IPFIX was only concerned > with the transport of IP flows over the Internet. I thought > it was to report IP flows in any environment, private or > public. Is that not the case? The point being made was that the IETF sets standards for the Internet. Having such a standard doesn't prevent anyone using non-standard protocols (such as non-congestion-aware variants of IPFIX) anywhere. What we really want for IPFIX is a congestion-aware protocol which everyone will want to use everywhere! So, what's your opinion on reliable vs unreliable transport for IPFIX? Would TCP be OK? How about SCTP?? Cheers, Nevil ----------------------------------------------------------------------- Nevil Brownlee Director, Technology Development Phone: +64 9 373 7599 x8941 ITSS, The University of Auckland FAX: +64 9 373 7021 Private Bag 92019, Auckland, New Zealand -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed Mar 6 09:28:46 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA09675 for ; Wed, 6 Mar 2002 09:28:46 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16icAH-0005w2-00 for ipfix-list@mil.doit.wisc.edu; Wed, 06 Mar 2002 08:13:09 -0600 Received: from dplonka by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16icAF-0005vv-00 for ipfix-req@net.doit.wisc.edu; Wed, 06 Mar 2002 08:13:07 -0600 Date: Wed, 6 Mar 2002 08:13:07 -0600 From: Dave Plonka To: ipfix-req@net.doit.wisc.edu Subject: [ipfix-req] I-D ACTION:draft-ietf-ipfix-reqs-02.txt Message-ID: <20020306081307.A22375@doit.wisc.edu> Reply-To: plonka@doit.wisc.edu Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0.1i Organization: UW-Madison, DoIT, Network Services X-VMS-Error: %SYSTEM-F-INHCHMK, inhibited CHMKernel trap, code=00000000, PC=000004CC, PS=7FED6580 X-Shakespearean-Insult: Thou gorbellied common-kissing canker-blossom Precedence: bulk Sender: majordomo listserver Below please find the announcement regarding the availability of the updated requirements draft. The following links to it now appear in the IPFIX web site index: http://www.ietf.org/internet-drafts/draft-ietf-ipfix-reqs-02.txt http://ipfix.doit.wisc.edu/req/draft-ietf-ipfix-reqs-02.txt Dave ----- Forwarded message from owner-ipfix@net.doit.wisc.edu ----- To: IETF-Announce: ; Cc: ipfix@net.doit.wisc.edu From: Internet-Drafts@ietf.org Reply-to: Internet-Drafts@ietf.org Subject: I-D ACTION:draft-ietf-ipfix-reqs-02.txt Date: Wed, 06 Mar 2002 08:15:34 -0500 Sender: nsyracus@cnri.reston.va.us --NextPart A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the IP Flow Information Export Working Group of the IETF. Title : Requirements for IP Flow Information Export Author(s) : J. Quittek et al. Filename : draft-ietf-ipfix-reqs-02.txt Pages : 24 Date : 05-Mar-02 This memo defines requirements for the export of measured IP flow information out of routers, traffic measurement probes and middleboxes. A URL for this Internet-Draft is: http://www.ietf.org/internet-drafts/draft-ietf-ipfix-reqs-02.txt To remove yourself from the IETF Announcement list, send a message to ietf-announce-request with the word unsubscribe in the body of the message. Internet-Drafts are also available by anonymous FTP. Login with the username "anonymous" and a password of your e-mail address. After logging in, type "cd internet-drafts" and then "get draft-ietf-ipfix-reqs-02.txt". A list of Internet-Drafts directories can be found in http://www.ietf.org/shadow.html or ftp://ftp.ietf.org/ietf/1shadow-sites.txt Internet-Drafts can also be obtained by e-mail. Send a message to: mailserv@ietf.org. In the body type: "FILE /internet-drafts/draft-ietf-ipfix-reqs-02.txt". NOTE: The mail server at ietf.org can return the document in MIME-encoded form by using the "mpack" utility. To use this feature, insert the command "ENCODING mime" before the "FILE" command. To decode the response(s), you will need "munpack" or a MIME-compliant mail reader. Different MIME-compliant mail readers exhibit different behavior, especially when dealing with "multipart" MIME messages (i.e. documents which have been split up into multiple messages), so check your local documentation on how to manipulate these messages. Below is the data which will enable a MIME compliant mail reader implementation to automatically retrieve the ASCII version of the Internet-Draft. --NextPart Content-Type: Multipart/Alternative; Boundary="OtherAccess" --OtherAccess Content-Type: Message/External-body; access-type="mail-server"; server="mailserv@ietf.org" Content-Type: text/plain Content-ID: <20020305134211.I-D@ietf.org> ENCODING mime FILE /internet-drafts/draft-ietf-ipfix-reqs-02.txt --OtherAccess Content-Type: Message/External-body; name="draft-ietf-ipfix-reqs-02.txt"; site="ftp.ietf.org"; access-type="anon-ftp"; directory="internet-drafts" Content-Type: text/plain Content-ID: <20020305134211.I-D@ietf.org> --OtherAccess-- --NextPart-- ----- End forwarded message ----- -- plonka@doit.wisc.edu http://net.doit.wisc.edu/~plonka ARS:N9HZF Madison, WI -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed Mar 6 09:29:00 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA09696 for ; Wed, 6 Mar 2002 09:29:00 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16ic5Z-0005op-00 for ipfix-list@mil.doit.wisc.edu; Wed, 06 Mar 2002 08:08:17 -0600 Received: from dplonka by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16ic5W-0005oj-00 for ipfix-app@net.doit.wisc.edu; Wed, 06 Mar 2002 08:08:14 -0600 Date: Wed, 6 Mar 2002 08:08:14 -0600 From: Dave Plonka To: ipfix-app@net.doit.wisc.edu Subject: [ipfix-app] [ipfix-arch] URL for applicability draft Message-ID: <20020306080814.B20979@doit.wisc.edu> Reply-To: plonka@doit.wisc.edu References: <3C753EC1.4010603@fokus.fhg.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0.1i In-Reply-To: <3C753EC1.4010603@fokus.fhg.de>; from zseby@fokus.gmd.de on Thu, Feb 21, 2002 at 07:38:57PM +0100 Organization: UW-Madison, DoIT, Network Services X-VMS-Error: %SYSTEM-F-BADATTRIB, bad attribute control list X-Shakespearean-Insult: Thou mangled boil-brained death-token Precedence: bulk Sender: majordomo listserver I've also placed the Applicability draft on the IPFIX site and linked it from the index. You can find it here: http://ipfix.doit.wisc.edu/app/ At this point, this document is a work-in-progress and has not been submitted as an I-D. Dave -- plonka@doit.wisc.edu http://net.doit.wisc.edu/~plonka ARS:N9HZF Madison, WI -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed Mar 6 09:49:04 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA10738 for ; Wed, 6 Mar 2002 09:49:04 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16icRN-0006LZ-00 for ipfix-list@mil.doit.wisc.edu; Wed, 06 Mar 2002 08:30:49 -0600 Received: from mgw-dax2.ext.nokia.com ([63.78.179.217]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16icRL-0006LQ-00 for ipfix-app@net.doit.wisc.edu; Wed, 06 Mar 2002 08:30:47 -0600 Received: from davir02nok.americas.nokia.com (davir02nok.americas.nokia.com [172.18.242.85]) by mgw-dax2.ext.nokia.com (Switch-2.1.0/Switch-2.1.0) with ESMTP id g26EWsx06264 for ; Wed, 6 Mar 2002 08:32:55 -0600 (CST) Received: from daebh001.NOE.Nokia.com (unverified) by davir02nok.americas.nokia.com (Content Technologies SMTPRS 4.2.5) with ESMTP id ; Wed, 6 Mar 2002 08:30:44 -0600 Received: from bsebe001.NOE.Nokia.com ([172.19.160.13]) by daebh001.NOE.Nokia.com with Microsoft SMTPSVC(5.0.2195.3779); Wed, 6 Mar 2002 08:30:44 -0600 content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C1C51B.7FB2F98D" X-MimeOLE: Produced By Microsoft Exchange V6.0.5762.3 Subject: [ipfix-app] RE: [ipfix-arch] applicability document Date: Wed, 6 Mar 2002 09:30:44 -0500 Message-ID: Thread-Topic: [ipfix-arch] applicability document Thread-Index: AcG7DQfFEhm7nncEQJqY4eP3YaPTjwKDqw2Q From: To: , Cc: X-OriginalArrivalTime: 06 Mar 2002 14:30:44.0972 (UTC) FILETIME=[804292C0:01C1C51B] Precedence: bulk Sender: majordomo listserver This is a multi-part message in MIME format. ------_=_NextPart_001_01C1C51B.7FB2F98D Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hello Tanja and Penno, =20 Sorry, i was not able to access my email for more than a week. Just = started reading the old emails . I went through the applicability = document, it is good and it has some remarks like " who contributed this section in = architecture " in Intrusion detection section.=20 =20 FYI, i worked on that section for architecture document, if you want = to expand that section more to address the relationship with IDWG = framework i can take that. =20 I know this is too late, we can add that for version 02. =20 Let me know who is editing this document. Thanks and Regards Ramg ------_=_NextPart_001_01C1C51B.7FB2F98D Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
 Hello Tanja and = Penno,
 
Sorry,=20 i was not able to access my email for more than a week. Just = started =20 reading the old emails . I went through the  applicability = document, it is=20 good
and it=20 has some remarks like " who contributed this section in architecture=20 " in Intrusion detection section. =
 
FYI, i=20 worked on that section for architecture document,  = if you want =20 to expand that section more to address the relationship with IDWG = framework i=20 can take that.
 
I know=20 this is too late, we can  add that for version = 02.
 
Let me=20 know who is editing this document.
Thanks and = Regards
Ramg
------_=_NextPart_001_01C1C51B.7FB2F98D-- -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed Mar 6 11:09:53 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA14995 for ; Wed, 6 Mar 2002 11:09:52 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16idhB-0000Ar-00 for ipfix-list@mil.doit.wisc.edu; Wed, 06 Mar 2002 09:51:13 -0600 Received: from mailhub.fokus.gmd.de ([193.174.154.14]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16idh9-0000Ai-00 for ipfix-app@net.doit.wisc.edu; Wed, 06 Mar 2002 09:51:11 -0600 Received: from fokus.fhg.de (dhcp206 [195.37.78.206]) by mailhub.fokus.gmd.de (8.11.6/8.11.6) with ESMTP id g26FoxR12441; Wed, 6 Mar 2002 16:50:59 +0100 (MET) Message-ID: <3C863B07.2070004@fokus.fhg.de> Date: Wed, 06 Mar 2002 16:51:35 +0100 From: Tanja Zseby Organization: FhI FOKUS User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:0.9.4) Gecko/20011019 Netscape6/6.2 X-Accept-Language: en-us MIME-Version: 1.0 To: ram.gopal@nokia.com CC: zseby@fokus.gmd.de, reinaldo_penno@nortelnetworks.com, ipfix-app@net.doit.wisc.edu Subject: Re: [ipfix-app] RE: [ipfix-arch] applicability document References: Content-Type: multipart/alternative; boundary="------------060402090400010306030504" Precedence: bulk Sender: majordomo listserver --------------060402090400010306030504 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Hi Ram, we have not yet submitted the applicability document officially as an I-D (too many sections missing). Currently only Reinaldo and I are working on this document. Therefore I would definitely appreciate if you contribute to the intrusion detection section. If you send me some text, I can include it into the next version. Regards Tanja P.S: Will you be in Minneapolis ? ram.gopal@nokia.com wrote: > Hello Tanja and Penno, > > > > Sorry, i was not able to access my email for more than a week. Just > started reading the old emails . I went through the applicability > document, it is good > > and it has some remarks like " who contributed this section in > architecture " in Intrusion detection section. > > > > FYI, i worked on that section for architecture document, if you want > to expand that section more to address the relationship with IDWG > framework i can take that. > > > > I know this is too late, we can add that for version 02. > > > > Let me know who is editing this document. > > Thanks and Regards > > Ramg > -- Dipl.-Ing. Tanja Zseby FhI FOKUS/Global Networking Email: zseby@fokus.fhg.de Kaiserin-Augusta-Allee 31 Phone: +49-30-3463-7153 D-10589 Berlin, Germany Fax: +49-30-3463-8153 -------------------------------------------------------------------------------------- "Living on earth is expensive but it includes a free trip around the sun." (Anonymous) -------------------------------------------------------------------------------------- --------------060402090400010306030504 Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7bit Hi Ram,

we have not yet submitted the applicability document officially as an I-D (too many sections missing).
Currently only Reinaldo and I are working on this document.  Therefore I would definitely appreciate if you contribute to the intrusion detection section.  If you send me some text, I can include it into the next version.

Regards
Tanja
P.S: Will you be in Minneapolis ?

ram.gopal@nokia.com wrote:
 Hello Tanja and Penno,
 
Sorry, i was not able to access my email for more than a week. Just started  reading the old emails . I went through the  applicability document, it is good
and it has some remarks like " who contributed this section in architecture " in Intrusion detection section.
 
FYI, i worked on that section for architecture document,  if you want  to expand that section more to address the relationship with IDWG framework i can take that.
 
I know this is too late, we can  add that for version 02.
 
Let me know who is editing this document.
Thanks and Regards
Ramg

-- 
Dipl.-Ing. Tanja Zseby			    	      	
FhI FOKUS/Global Networking			Email: zseby@fokus.fhg.de	
Kaiserin-Augusta-Allee 31				Phone: +49-30-3463-7153
D-10589 Berlin, Germany				Fax:   +49-30-3463-8153
-------------------------------------------------------------------------------------- 
"Living on earth is expensive but it includes a free trip around the sun." (Anonymous)
--------------------------------------------------------------------------------------

--------------060402090400010306030504-- -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed Mar 6 11:29:54 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA16296 for ; Wed, 6 Mar 2002 11:29:53 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16idvQ-0000WL-00 for ipfix-list@mil.doit.wisc.edu; Wed, 06 Mar 2002 10:05:56 -0600 Received: from mgw-x2.nokia.com ([131.228.20.22]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16idvO-0000WF-00 for ipfix-app@net.doit.wisc.edu; Wed, 06 Mar 2002 10:05:54 -0600 Received: from esvir04nok.ntc.nokia.com (esvir04nokt.ntc.nokia.com [172.21.143.36]) by mgw-x2.nokia.com (Switch-2.1.0/Switch-2.1.0) with ESMTP id g26G62Z05797 for ; Wed, 6 Mar 2002 18:06:02 +0200 (EET) Received: from esebh004.NOE.Nokia.com (unverified) by esvir04nok.ntc.nokia.com (Content Technologies SMTPRS 4.2.5) with ESMTP id ; Wed, 6 Mar 2002 18:05:51 +0200 Received: from daebh002.NOE.Nokia.com ([172.18.242.232]) by esebh004.NOE.Nokia.com with Microsoft SMTPSVC(5.0.2195.3779); Wed, 6 Mar 2002 18:05:51 +0200 Received: from bsebe001.NOE.Nokia.com ([172.19.160.13]) by daebh002.NOE.Nokia.com with Microsoft SMTPSVC(5.0.2195.3779); Wed, 6 Mar 2002 10:04:35 -0600 content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C1C528.9B860FC7" X-MimeOLE: Produced By Microsoft Exchange V6.0.5762.3 Subject: RE: [ipfix-app] RE: [ipfix-arch] applicability document Date: Wed, 6 Mar 2002 11:04:34 -0500 Message-ID: Thread-Topic: [ipfix-app] RE: [ipfix-arch] applicability document Thread-Index: AcHFJrssVb5y2vrQSl2Z41tea2D/qQAAqI8A From: To: Cc: , X-OriginalArrivalTime: 06 Mar 2002 16:04:35.0151 (UTC) FILETIME=[9C1BB5F0:01C1C528] Precedence: bulk Sender: majordomo listserver This is a multi-part message in MIME format. ------_=_NextPart_001_01C1C528.9B860FC7 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hello Tanja, =20 Thanks, I will send it by Friday. I will be there on Minneapolis. =20 Regards Ramg =20 -----Original Message----- From: ext Tanja Zseby [mailto:zseby@fokus.gmd.de] Sent: Wednesday, March 06, 2002 10:52 AM To: Gopal Ram (NRC/Boston) Cc: zseby@fokus.gmd.de; reinaldo_penno@nortelnetworks.com; = ipfix-app@net.doit.wisc.edu Subject: Re: [ipfix-app] RE: [ipfix-arch] applicability document Hi Ram, we have not yet submitted the applicability document officially as an = I-D (too many sections missing). Currently only Reinaldo and I are working on this document. Therefore I = would definitely appreciate if you contribute to the intrusion detection = section. If you send me some text, I can include it into the next = version. Regards Tanja P.S: Will you be in Minneapolis ?=20 ram.gopal@nokia.com wrote: Hello Tanja and Penno, =20 Sorry, i was not able to access my email for more than a week. Just = started reading the old emails . I went through the applicability = document, it is good and it has some remarks like " who contributed this section in = architecture " in Intrusion detection section.=20 =20 FYI, i worked on that section for architecture document, if you want = to expand that section more to address the relationship with IDWG = framework i can take that. =20 I know this is too late, we can add that for version 02. =20 Let me know who is editing this document. Thanks and Regards Ramg --=20 Dipl.-Ing. Tanja Zseby =09 FhI FOKUS/Global Networking Email: zseby@fokus.fhg.de=09 Kaiserin-Augusta-Allee 31 Phone: +49-30-3463-7153 D-10589 Berlin, Germany Fax: +49-30-3463-8153 -------------------------------------------------------------------------= -------------=20 "Living on earth is expensive but it includes a free trip around the = sun." (Anonymous) -------------------------------------------------------------------------= ------------- ------_=_NextPart_001_01C1C528.9B860FC7 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
Hello=20 Tanja,
 
Thanks, I will send it  by Friday.
I will=20 be there on Minneapolis.
 
Regards
Ramg
 
-----Original Message-----
From: ext Tanja Zseby=20 [mailto:zseby@fokus.gmd.de]
Sent: Wednesday, March 06, 2002 = 10:52=20 AM
To: Gopal Ram (NRC/Boston)
Cc: = zseby@fokus.gmd.de;=20 reinaldo_penno@nortelnetworks.com;=20 ipfix-app@net.doit.wisc.edu
Subject: Re: [ipfix-app] RE:=20 [ipfix-arch] applicability document

Hi = Ram,

we have=20 not yet submitted the applicability document officially as an I-D (too = many=20 sections missing).
Currently only Reinaldo and I are working on = this=20 document.  Therefore I would definitely appreciate if you = contribute to=20 the intrusion detection section.  If you send me some text, I can = include=20 it into the next version.

Regards
Tanja
P.S: Will you be = in=20 Minneapolis ?

ram.gopal@nokia.com wrote:
 Hello Tanja and = Penno,
 
Sorry, i was not able to access my email = for more=20 than a week. Just started  reading the old emails . I went = through=20 the  applicability document, it is good
and it has some remarks like " who = contributed this=20 section in architecture " in Intrusion detection = section.=20
 
FYI, i worked on that section for = architecture=20 document,  if you want  to expand that section = more to=20 address the relationship with IDWG framework i can take=20 that.
 
I=20 know this is too late, we can  add that for version=20 02.
 
Let me know who is editing this=20 document.
Thanks and Regards
Ramg

--=20
Dipl.-Ing. Tanja Zseby			    	      =09
FhI FOKUS/Global Networking			Email: zseby@fokus.fhg.de=09
Kaiserin-Augusta-Allee 31				Phone: +49-30-3463-7153
D-10589 Berlin, Germany				Fax:   +49-30-3463-8153
-------------------------------------------------------------------------=
-------------=20
"Living on earth is expensive but it includes a free trip around the =
sun." (Anonymous)
-------------------------------------------------------------------------=
-------------

------_=_NextPart_001_01C1C528.9B860FC7-- -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu Mar 7 14:36:21 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA17770 for ; Thu, 7 Mar 2002 14:36:16 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16j39N-0004rf-00 for ipfix-list@mil.doit.wisc.edu; Thu, 07 Mar 2002 13:02:01 -0600 Received: from mailhost.auckland.ac.nz ([130.216.1.4]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16j39K-0004r8-00 for ipfix@net.doit.wisc.edu; Thu, 07 Mar 2002 13:01:58 -0600 Received: from auckland.ac.nz (root@ccu1.auckland.ac.nz [130.216.3.1]) by mailhost.auckland.ac.nz (8.9.2/8.9.2/8.9.2-ua) with ESMTP id IAA25066 for ; Fri, 8 Mar 2002 08:01:49 +1300 (NZDT) Message-Id: <200203071901.IAA25066@mailhost.auckland.ac.nz> Date: Thu, 7 Mar 2002 11:03:46 -0800 (PST) From: n.brownlee@auckland.ac.nz Subject: [ipfix] RTFM compared with IPFIX To: ipfix@net.doit.wisc.edu MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Precedence: bulk Sender: majordomo listserver Hi everyone: I promised KC Norseth I'd write some notes comparing RTFM with IPFIX, and here - at last - they are. They're intended, after discussion and editing, to go into the IPFIX Architecture draft. Comments, improvements, etc. please ... Cheers, Nevil RTFM compared with IPFIX (A) Definition of 'flow' RTFM and IPFIX both use the same definition of flow; a flow is a set of packets which share a common set of end-point address attribute values. A flow is therefore completely specified by that set of values, together with an inactivity timeout. A flow is considered to have ended when no packets are seen for at least the inactivity time. RTFM flows are bidirectional, which has given rise to some confusion. At the simplest level, a flow information exporter may achieve this by maintaining two unidirectional flows, one for each direcion. To export bidirectional flow information, e.g. to- and from- packet counts, for a flow from A to B, the exporter has only to search its flow table to find the matching flow from B to A. RTFM, however, takes bi-directionality a stage further, by including in the RTFM architecture [RFC 2722] a fully-detailed algorithm for realtime matching of the two directions of a flow. This was done for two reasons, to reduce the memory required to store each flow (common address attributes for each direction), and to allow for attributes which required fine detail for the two directions, e.g. short-term bit rate distributions [RFC 2724]. ** So far there has been no suggestion that IPFIX should do this. (B) Configuration and Management The RTFM architecture specifies a complete system for gathering flow information. It defines three entities, - Meters are very similar to IPFIX exporters. - Meter Readers are very similar to IPFIX collectors. - Managers co-ordinate the activities of meters and meter readers, and download configuration to them. Note that the whole RTFM system is asynchronous, many readers may collector flow data from a meter, and any reader may collect flow data from many meters. Rulesets allow the user to specify which flows are of interest, which are the source and destination ends of each flow, and what level of address granularity is required in the metered flows. For example, on may select all packets from 192.168/16, but build flow information for 192.168/24. RTFM selection is done by testing under masks, and the masks do not have to use consective ones from the left. Non-contiguous masks were considered important for handling some OSI protocols, but the' need for that has diminished considerably. The RTFM approach is based on RMON, in that if a user wants to collect flow data for some particular set of flows, this can be achieved by writing a ruleset, i.e. an SRL program [RFC 2723], to specify what flows are of interest, requesting a manager to download that ruleset to a meter, and requesting the manager to have a meter reader collect the flow data at specified intervals. The details of how the manager communicates this information to meters and meter readers is not specified in the architecture. RTFM has a Meter MIB [RFC 2720], which is a standard which can be used to configure a meter, but nothing is said about how to configure a meter reader. The extent to which IPFIX should specify how meters or exporters should be configured is, at this stage, an open question. Clearly a collector needs some way to be sure of what it's collecting, e.g. by receiving 'templates' from the meter. RTFM and IPFIX both leave parts of the system unspecified. For RTFM flow data to be useful one must know the ruleset used to configure the meter, but a user can specify the ruleset. For IPFIX one knows what the data is from the templates, but we have yet to determine whether in-band configuration will be supported. (C) Data Model details (C.1) Count in one bucket Within a ruleset, a packet may only be counted on one bucket, i.e. it may only be included in one flow. This means that the meter does not have to keep track of overlapping flows - if such aggregation is required, it must be done after the raw flow data has been read by a meter reader. >From time to time one may wish to collect flow data for different levels of aggreation at the same time. RTFM allows a meter to run several rulesets at the same time, and meter readers must specify which rulesets they are collecting data from. The 'count in one bucket' rule, together with the ability to run multiple rulesets, has proved very simple and effective in practice. (C.2) Counter wrapping For its packet- and byte-count attributes RTFM uses continuously- incrementing 64-bit counters, which are never reset. This makes asynchronous meter reading easy, any reader simply has to remember its previous reading and compute the difference. The only caveat is that the meter should be read often enough to avoid situations when the counter has cycled more than once between readings. (C.3) Sampling issues RTFM provides 1 out of N sampling as a configuration option, so that some metering interfaces may only process every Nth packet. The RTFM Arcitecture [RFC 2722] does not discuss the statistical implications of this, merely saying that users will need to satisfy themselves that sampling makes sense in their environment. RTFM makes no provision for flow sampling. Recently there has been a lot of interest in flow sampling schemes which favour the 'most important' flows, perhaps we need to consider this for IPFIX. At this stage, however, the effect of simpler schemes (e.g. 1 out of N flow sampling) is onlypoorly understood. (D) Application/transport protocol RTFM has a standards-track Meter MIB [RFC 2720], which can be used both to configure a meter and to read flow data from it. The MIB provides a way to read lists of attributes with a single Object Identifier (clled a 'package'), which dramatically reduces the SNMP overhead for flow data collection. NeTraMet, a widely-used open-source RTFM implementation, uses SNMPv2C for configuration and data collection. SNMP, of course, normally uses UDP as its transport protocol. Since RTFM requires a reliable flow data transport system, an RTFM meter reader must time out and resend unanswered SNMP requests. Apart from being clumsy, this can limit the maximum data transfer rate from meter to meter reader. SNMP over TCP would be a better approach, but that is currently an IRTF project. On the other hand, RTFM does not specify an application protocol in its architecture, leaving this as an implementation issue. For example, a team at IBM Research implemented a RTFM meter and meter reader in a single host, with the reader storing the flow data directly into a large database system. Simlarly, many NeTraMet user run the meter and meter reader on the same host system. A need for high flow data rates highlights the need for careful systems design when building a flow data collection system. When data rates are high, and it is not possible to use a high level of aggregation, then it makes sense to have the collectors very close to their exporters. Once the data is safely on a dedicated host machine, large volumes of it can be moved using 'background' techniques such as FTP. The RTFM architecture only specifies a pull model for getting data out of a meter. To implement push mode data transfer would require specification of triggers to indicate when data should be sent for each flow. ------------------------------------------------------------- Nevil Brownlee Internet Researcher Phone: (858) 534 8338 CAIDA, San Diego -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu Mar 7 22:48:08 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id WAA12329 for ; Thu, 7 Mar 2002 22:48:07 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16jAk0-0007HT-00 for ipfix-list@mil.doit.wisc.edu; Thu, 07 Mar 2002 21:08:20 -0600 Received: from usmail.xacct.com ([204.253.100.12]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16jAjz-0007Gi-00 for ipfix@net.doit.wisc.edu; Thu, 07 Mar 2002 21:08:19 -0600 Received: from Kevinz (1Cust45.tnt15.dca5.da.uu.net [63.10.143.45]) by usmail.xacct.com (8.11.2/8.11.2) with SMTP id g2839El32660; Thu, 7 Mar 2002 19:09:15 -0800 Reply-To: From: "Kevin Zhang" To: "Ipfix@Net. Doit. Wisc. Edu" Cc: "Ganesh Sadasivan" , "KC' 'Norseth" Subject: [ipfix] ipfix-arch Section 4.2 comments Date: Thu, 7 Mar 2002 22:07:15 -0500 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by ietf.org id WAA12329 Hi All, Please see my comments and proposals related to section 4.2 of the IPFIX architecture draft. I would like to see my proposed text included in the next architecture draft. Thanks, Kevin 4.2. IPFIX protocol The information that needs to be exported from the exporting device can be classified into the following categories: [kz] The IPFIX protocol support bi-directional information exchanges. I recommend change the above sentence to "The information that needs to be exchanged between exporting and collecting devices can be classified into the following categories" [kz] 1 Control Information - This includes the flow type definition, selection criteria for packets within the flow etc. [kz] Using template is a key feature that proposed by both Netflow and CRANE. It will greatly improve efficiency of bandwidth consumption for data export and data processing (parsing) at devices. Templates are important control information which should be negotiated. Propose to change the above sentence to - "Control Information - Control information is exchanged to facilitate correct interpretation of IP flow information. Control information may include templates and template identifiers, flow type definition, packet selection criteria, etc. [kz] 2 Flow record - This includes data records corresponding to the various observed flows at each of the observation point. [kz] propose to change the above sentence to - "Flow record - This includes data records that describe some aspects of observed IP flows at the observation points of interest"[kz] IPFIX protocol at the exporting device does the following: 1 Encode the control information into self-describing structures. 2 Encode the flows measured at the exporting device into flow records. [kz] propose to change the above sentence to - "2 Encode the flow information obtained at the exporting device into flow records" [kz] 3 Packetize the flow records and/or control information into export packets. 4 Use the underlying transport layer to send the export packets to the collector. [kz] propose to add - "5 Support dynamical procedures to enable redundant collector configurations" and "6 Retransmit packets upon reception of an indication of packet loss or collecting device failure" [kz] IPFIX protocol at the collecting device is responsible for 1 Receive and store the control information. 2 Decode and store the flow records using the control information. [kz] propose to add - "3 Indicate to the exporting device of any gaps of the received flow records" and "4 Indicate to the exporting device of any processing and/or storage congestion at the collecting device" [kz] [kz] below are duplicated information in the current draft, I suggest to remove it. [---- The information that needs to be exported from the IPFIX device can be classified into the following categories: * Control Information - This includes the flow type definition, selection criteria for packets within the flow etc. * Flow records. At the IPFIX device, the protocol functionality may be split between Observation domain and exporter process. IPFIX protocol at the IPFIX device does the following: 1 Encode the control information into self-describing templates. 2 Encode the flows measured at the exporting device into flow records. 3 Packetize the flow records and/or control information into export packets. 4 Use the underlying transport layer to send the export packets to the collector. IPFIX protocol at the collector is responsible for 1 Receive and store the control information. 2 Decode and store the flow records using the control information. ----] ÿñÞ–™šŠ[hþf£¢·hšçzßÝ¢+ÿÂ+ýçnjwlk/ázZÿŠyž²Æ yºÉIì¹»®&Þ™¨¥¶æj:+v‰¨þw­ýÚ"·ü"±ÏÞvæ§vƲþéì¹»®&ÞŠ—âÇø§™ë,j›¡Ü€­Èb½èm¶Ÿÿþ*_‹Ý¢+ÿÂ+ýçnýªÜ†+Þ From majordomo@mil.doit.wisc.edu Thu Mar 7 22:48:16 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id WAA12340 for ; Thu, 7 Mar 2002 22:48:16 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16jAk5-0007Hb-00 for ipfix-list@mil.doit.wisc.edu; Thu, 07 Mar 2002 21:08:25 -0600 Received: from usmail.xacct.com ([204.253.100.12]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16jAk3-0007Gk-00 for ipfix@net.doit.wisc.edu; Thu, 07 Mar 2002 21:08:23 -0600 Received: from Kevinz (1Cust45.tnt15.dca5.da.uu.net [63.10.143.45]) by usmail.xacct.com (8.11.2/8.11.2) with SMTP id g2839Ml32663; Thu, 7 Mar 2002 19:09:23 -0800 Reply-To: From: "Kevin Zhang" To: "Ipfix@Net. Doit. Wisc. Edu" Cc: "Ganesh Sadasivan" , "KC' 'Norseth" Subject: [ipfix] ipfix-arch Section 3 comments Date: Thu, 7 Mar 2002 22:07:22 -0500 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by ietf.org id WAA12340 Hi All, I have a couple of comments related to flow type and template definitions. See if you can agree with my proposed changes. Thanks, Kevin Flow Type: A function F that would take input as a set of keys and the output would be one or more flows depending on the combination of values for the set of keys. [kz] this is a bit confusing, I am proposing the following "Flow Type: A flow type represents some properties of a flow, it is derived by applying a function on a set of keys. With identical keys as inputs to the function, only a single flow type shall be obtained." [kz] Template: Templates is a set of {type, length} ordered pairs, used to completely identify the structure and semantics of a particular information that needs to be communicated from the exporter to the collector. Each template is uniquely identified by a Template ID. [kz] The definition of Template was significantly changed from our previous draft, the current definition looks more like an IE coding. Propose the following definition for Template and Data Record Spec. " Template: A Template is an ordered list of Data Record Spec., it completely defines the structure of multiple data records that may be logically related (e.g. through flow definitions). It specifies the data type, meaning, and location of the data records in such a set. A template is typically negotiated between the exporter and collectors; an exporter uniquely identifies each template by a Template ID. Data Record Spec.: A Data Record Spec. defines a data record that an exporter may export. The specification may consist of the description, length, and the data type of the data record. (e.g. 'Byte Count' can be a key that is an 32 bit unsigned integer) An exporting device typically defines keys." [kz] ÿñÞ–™šŠ[hþf£¢·hšçzßÝ¢+ÿÂ+ýçnjwlk/ázZÿŠyž²Æ yºÉIì¹»®&Þ™¨¥¶æj:+v‰¨þw­ýÚ"·ü"±ÏÞvæ§vƲþéì¹»®&ÞŠ—âÇø§™ë,j›¡Ü€­Èb½èm¶Ÿÿþ*_‹Ý¢+ÿÂ+ýçnýªÜ†+Þ From majordomo@mil.doit.wisc.edu Thu Mar 7 23:36:29 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id XAA13682 for ; Thu, 7 Mar 2002 23:36:29 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16jBm9-0000ox-00 for ipfix-list@mil.doit.wisc.edu; Thu, 07 Mar 2002 22:14:37 -0600 Received: from sj-msg-core-2.cisco.com ([171.69.24.11]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16jBm7-0000nP-00 for ipfix@net.doit.wisc.edu; Thu, 07 Mar 2002 22:14:35 -0600 Received: from mira-sjcd-1.cisco.com (mira-sjcd-1.cisco.com [171.69.43.44]) by sj-msg-core-2.cisco.com (8.11.3/8.9.1) with ESMTP id g284DJh03348; Thu, 7 Mar 2002 20:13:19 -0800 (PST) Received: from cisco.com (dhcp-171-71-126-95.cisco.com [171.71.126.95]) by mira-sjcd-1.cisco.com (Mirapoint) with ESMTP id ABK47064; Thu, 7 Mar 2002 20:13:47 -0800 (PST) Message-ID: <3C883A5E.7241DF12@cisco.com> Date: Thu, 07 Mar 2002 20:13:18 -0800 From: Ganesh Sadasivan X-Mailer: Mozilla 4.76 [en]C-CCK-MCD (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: kevin.zhang@xacct.com CC: "Ipfix@Net. Doit. Wisc. Edu" , "KC' 'Norseth" Subject: [ipfix] Re: ipfix-arch Section 3 comments References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Hi Kevin, Kevin Zhang wrote: > Hi All, > > I have a couple of comments related to flow type and template definitions. See if you can agree with my proposed changes. > > Thanks, > > Kevin > > Flow Type: > A function F that would take input as a set of keys and the output > would be one or more flows depending on the combination of values > for the set of keys. > > [kz] this is a bit confusing, I am proposing the following > "Flow Type: > A flow type represents some properties of a flow, it is derived by applying a function on a set of keys. With identical keys as inputs to the function, only a single flow type shall be obtained." [kz] A flow type generates a class of flows. Infact this is a problem of distributing definitions across docs. The data model doc describes with examples what a flow type is. You can imagine this as a function f(x1,x2..). Based on different values of tuple (x1, x2..) the function returns values which are flows. Now for 2 different tuples, the function can return the same flow or different flows. It depends on what the function does. > > > Template: > Templates is a set of {type, length} ordered pairs, used to > completely identify the structure and semantics of a particular > information that needs to be communicated from the exporter to the > collector. Each template is uniquely identified by a Template ID. The above defintion is inline with the proposed data model of {field type, size}. The semantics (.i.e meaning) is automatically understood based on this information and so is the structure (what you refer to as "location"). > > [kz] The definition of Template was significantly changed from our previous draft, the current definition looks more like an IE coding. Propose the following definition for Template and Data Record Spec. > " Template: > A Template is an ordered list of Data Record Spec., it completely defines the structure of multiple data records that may be logically related (e.g. through flow definitions). It specifies the data type, meaning, and location of the data records in such a set. A template is typically negotiated between the exporter and collectors; an exporter uniquely identifies each template by a Template ID. "negotiation" is not part of template defintion. > > > Data Record Spec.: > A Data Record Spec. defines a data record that an exporter may export. The specification may consist of the description, length, and the data type of the data record. (e.g. 'Byte Count' can be a key that is an 32 bit unsigned integer) An exporting device typically defines keys." [kz] This statement talks about an implementation. Thanks Ganesh -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu Mar 7 23:36:32 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id XAA13693 for ; Thu, 7 Mar 2002 23:36:32 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16jBm4-0000or-00 for ipfix-list@mil.doit.wisc.edu; Thu, 07 Mar 2002 22:14:32 -0600 Received: from sj-msg-core-1.cisco.com ([171.71.163.11]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16jBm2-0000nN-00 for ipfix@net.doit.wisc.edu; Thu, 07 Mar 2002 22:14:30 -0600 Received: from mira-sjcd-1.cisco.com (mira-sjcd-1.cisco.com [171.69.43.44]) by sj-msg-core-1.cisco.com (8.11.3/8.9.1) with ESMTP id g284DNd04953; Thu, 7 Mar 2002 20:13:23 -0800 (PST) Received: from cisco.com (dhcp-171-71-126-95.cisco.com [171.71.126.95]) by mira-sjcd-1.cisco.com (Mirapoint) with ESMTP id ABK47066; Thu, 7 Mar 2002 20:13:51 -0800 (PST) Message-ID: <3C883A62.EE2B07DC@cisco.com> Date: Thu, 07 Mar 2002 20:13:22 -0800 From: Ganesh Sadasivan X-Mailer: Mozilla 4.76 [en]C-CCK-MCD (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: kevin.zhang@xacct.com CC: "Ipfix@Net. Doit. Wisc. Edu" , "KC' 'Norseth" Subject: [ipfix] Re: ipfix-arch Section 4.2 comments References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Hi Kevin, The general comment is to refer to http://ipfix.doit.wisc.edu/arch/draft-ietf-ipfix-architecture-00f.txt. This is the updated version. Please see inline. Kevin Zhang wrote: > Hi All, > > Please see my comments and proposals related to section 4.2 of the IPFIX architecture draft. I would like to see my proposed text included in the next architecture draft. > > Thanks, > > Kevin > > 4.2. IPFIX protocol > > The information that needs to be exported from the exporting device > can be classified into the following categories: > [kz] The IPFIX protocol support bi-directional information exchanges. I recommend change the above sentence to > "The information that needs to be exchanged between exporting and collecting devices can be classified into the following categories" [kz] This has been moved to section 3. Note that the control info & data are as defined from the exporter side. > > > 1 Control Information - This includes the flow type definition, > selection criteria for packets within the flow etc. > [kz] Using template is a key feature that proposed by both Netflow and CRANE. It will greatly improve efficiency of bandwidth consumption for data export and data processing (parsing) at devices. Templates are important control information which should be negotiated. Propose to change the above sentence to - Template negotiation & infact the whole section on negotiation is still up for discussion. > > "Control Information - Control information is exchanged to facilitate correct interpretation of IP flow information. Control information may include templates and template identifiers, flow type definition, packet selection criteria, etc. [kz] template includes template ids (this is more the specifics). Does the sentence "This includes the flow type definition, selection criteria for packets within the flow." not cover what you are telling above? Flow type definition for export transaltes to templates. > > > 2 Flow record - This includes data records corresponding to > the various observed flows at each of the observation point. > [kz] propose to change the above sentence to - > "Flow record - This includes data records that describe some aspects of observed IP flows at the observation points of interest"[kz] Can you elaborate on what you mean by the above sentence? Does it mean that the flows are measured & only subset of this information is sent across? > > > IPFIX protocol at the exporting device does the following: > > 1 Encode the control information into self-describing structures. > 2 Encode the flows measured at the exporting device into flow > records. > [kz] propose to change the above sentence to - > "2 Encode the flow information obtained at the exporting device into flow records" [kz] "flow information" == {values of fields within flow} is it the intention? If so I guess "flows" by defintion means this. > > > 3 Packetize the flow records and/or control information into > export packets. > 4 Use the underlying transport layer to send the export packets > to the collector. > [kz] propose to add - > "5 Support dynamical procedures to enable redundant collector configurations" and Please explain . > > "6 Retransmit packets upon reception of an indication of packet loss or collecting device failure" [kz] This behavior is still up for discussion. Basically if the underlying transport is reliable then IPFIX protocol is transparent to this. Just to quote from the req. spec: "Absence of reliability of the data transfer MUST be indicated covering packet loss and packet reordering.". It does not talk about retransmission. > > > IPFIX protocol at the collecting device is responsible for > > 1 Receive and store the control information. > 2 Decode and store the flow records using the control > information. > [kz] propose to add - > "3 Indicate to the exporting device of any gaps of the received flow records" and This is also still not discussed completly. The above statement from req. spec holds good here also. > > "4 Indicate to the exporting device of any processing and/or storage congestion at the collecting device" [kz] This is not in scope for IPFIX protocol. > > > > [kz] below are duplicated information in the current draft, I suggest to remove it. Refer to the updated draft.It has been removed. > > > [---- > The information that needs to be exported from the IPFIX device can > be classified into the following categories: > > * Control Information - This includes the flow type definition, > selection criteria for packets within the flow etc. > * Flow records. > > At the IPFIX device, the protocol functionality may be split between > Observation domain and exporter process. IPFIX protocol at the IPFIX > device does the following: > > 1 Encode the control information into self-describing templates. > 2 Encode the flows measured at the exporting device into flow > records. > 3 Packetize the flow records and/or control information into > export packets. > 4 Use the underlying transport layer to send the export packets > to the collector. > > IPFIX protocol at the collector is responsible for > 1 Receive and store the control information. > 2 Decode and store the flow records using the control > information. > ----] -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri Mar 8 11:29:21 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA13611 for ; Fri, 8 Mar 2002 11:29:21 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16jMwm-0001qe-00 for ipfix-list@mil.doit.wisc.edu; Fri, 08 Mar 2002 10:10:20 -0600 Received: from usmail.xacct.com ([204.253.100.12]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16jMwk-0001pW-00 for ipfix@net.doit.wisc.edu; Fri, 08 Mar 2002 10:10:18 -0600 Received: from Kevinz (1Cust32.tnt16.dca5.da.uu.net [63.10.64.32]) by usmail.xacct.com (8.11.2/8.11.2) with SMTP id g28GB2l03207; Fri, 8 Mar 2002 08:11:02 -0800 Reply-To: From: "Kevin Zhang" To: "Ganesh Sadasivan" Cc: "Ipfix@Net. Doit. Wisc. Edu" , "KC' 'Norseth" Subject: RE: [ipfix] Re: ipfix-arch Section 4.2 comments Date: Fri, 8 Mar 2002 11:09:04 -0500 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) Importance: Normal In-Reply-To: <3C883A62.EE2B07DC@cisco.com> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by ietf.org id LAA13611 Hi Ganesh, Please see my comments inserted, I will use -00f in the future as the reference. Thanks, Kevin > -----Original Message----- > From: majordomo listserver [mailto:majordomo@mil.doit.wisc.edu]On Behalf > Of Ganesh Sadasivan > Sent: Thursday, March 07, 2002 11:13 PM > To: kevin.zhang@xacct.com > Cc: Ipfix@Net. Doit. Wisc. Edu; KC' 'Norseth > Subject: [ipfix] Re: ipfix-arch Section 4.2 comments > > > Hi Kevin, > The general comment is to refer to > > http://ipfix.doit.wisc.edu/arch/draft-ietf-ipfix-architecture-00f. > txt. This is the > updated version. Please see inline. > > > Kevin Zhang wrote: > > > Hi All, > > > > Please see my comments and proposals related to section 4.2 of > the IPFIX architecture draft. I would like to see my proposed > text included in the next architecture draft. > > > > Thanks, > > > > Kevin > > > > 4.2. IPFIX protocol > > > > The information that needs to be exported from the exporting device > > can be classified into the following categories: > > [kz] The IPFIX protocol support bi-directional information > exchanges. I recommend change the above sentence to > > "The information that needs to be exchanged between exporting > and collecting devices can be classified into the following > categories" [kz] > > This has been moved to section 3. Note that the control info & data are > as defined from the exporter side. > > > > > > > 1 Control Information - This includes the flow type definition, > > selection criteria for packets within the flow etc. > > [kz] Using template is a key feature that proposed by both > Netflow and CRANE. It will greatly improve efficiency of > bandwidth consumption for data export and data processing > (parsing) at devices. Templates are important control information > which should be negotiated. Propose to change the above sentence to - > > Template negotiation & infact the whole section on negotiation is > still up for discussion. > > > > > "Control Information - Control information is exchanged to > facilitate correct interpretation of IP flow information. > Control information may include templates and template > identifiers, flow type definition, packet selection criteria, etc. [kz] > > template includes template ids (this is more the specifics). Does > the sentence > "This includes the flow type definition, selection criteria for > packets within the flow." > not cover what you are telling above? Flow type definition for > export transaltes to > templates. > Template governs how the data records are put in an IPFIX user message payload, so that only flow information needs to be exported instead of (tag, length, value) type of coding. This allows an efficient way to export as well as processing flow information. This concept was in three previous drafts from the design team, and I believe it is essential for IPFIX. Template negotiation allows the correct context being setup at IPFIX end-points, which should be part of control information exchanges. > > > > > > > 2 Flow record - This includes data records corresponding to > > the various observed flows at each of the observation point. > > [kz] propose to change the above sentence to - > > "Flow record - This includes data records that describe some > aspects of observed IP flows at the observation points of interest"[kz] > > Can you elaborate on what you mean by the above sentence? Does it > mean that the > flows are measured & only subset of this information is sent across? Yes, only the required flow information needs to be sent to the collecting device. Though the exporting device has the final say on what it can and wants to export. > > > > > > > IPFIX protocol at the exporting device does the following: > > > > 1 Encode the control information into self-describing structures. > > 2 Encode the flows measured at the exporting device into flow > > records. > > [kz] propose to change the above sentence to - > > "2 Encode the flow information obtained at the exporting device > into flow records" [kz] > > "flow information" == {values of fields within flow} is it the > intention? > If so I guess "flows" by defintion means this. > "flow information" = some property of the flow, which the exporting device wants to export. It may be a field (e.g. ip address) in a flow, or it can be derived from a flow (e.g. timestamp). > > > > > > 3 Packetize the flow records and/or control information into > > export packets. > > 4 Use the underlying transport layer to send the export packets > > to the collector. > > [kz] propose to add - > > "5 Support dynamical procedures to enable redundant collector > configurations" and > > Please explain . > > > > > "6 Retransmit packets upon reception of an indication of packet > loss or collecting device failure" [kz] > > This behavior is still up for discussion. Basically if the > underlying transport is > reliable then IPFIX protocol is transparent to this. Just to > quote from the req. > spec: > "Absence of reliability of the data transfer MUST be indicated > covering packet loss and packet reordering.". > It does not talk about retransmission. > I am quoting our charter - "Ensure that the flow export system is reliable in that it will minimize the likelihood of flow data being lost due to resource constraints in the exporter or receiver and to accurately report such loss if it occurs" All of my proposals are trying to "minimize" the likelihood of data loss, whether it happens during transmission, processing, or other factors. This applies to the comments I had below. > > > > > > IPFIX protocol at the collecting device is responsible for > > > > 1 Receive and store the control information. > > 2 Decode and store the flow records using the control > > information. > > [kz] propose to add - > > "3 Indicate to the exporting device of any gaps of the received > flow records" and > > This is also still not discussed completly. The above statement > from req. spec > holds good here also. > > > > > > "4 Indicate to the exporting device of any processing and/or > storage congestion at the collecting device" [kz] > > This is not in scope for IPFIX protocol. > > > > > > > > > > [kz] below are duplicated information in the current draft, I > suggest to remove it. > > Refer to the updated draft.It has been removed. > > > > > > > [---- > > The information that needs to be exported from the IPFIX device can > > be classified into the following categories: > > > > * Control Information - This includes the flow type definition, > > selection criteria for packets within the flow etc. > > * Flow records. > > > > At the IPFIX device, the protocol functionality may be split between > > Observation domain and exporter process. IPFIX protocol at the IPFIX > > device does the following: > > > > 1 Encode the control information into self-describing templates. > > 2 Encode the flows measured at the exporting device into flow > > records. > > 3 Packetize the flow records and/or control information into > > export packets. > > 4 Use the underlying transport layer to send the export packets > > to the collector. > > > > IPFIX protocol at the collector is responsible for > > 1 Receive and store the control information. > > 2 Decode and store the flow records using the control > > information. > > ----] > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in > message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ > > ÿñÞ–™šŠ[hþf£¢·hšçzßÝ¢+ÿÂ+ýçnjwlk/ázZÿŠyž²Æ yºÉIì¹»®&Þ™¨¥¶æj:+v‰¨þw­ýÚ"·ü"±ÏÞvæ§vƲþéì¹»®&ÞŠ—âÇø§™ë,j›¡Ü€­Èb½èm¶Ÿÿþ*_‹Ý¢+ÿÂ+ýçnýªÜ†+Þ From majordomo@mil.doit.wisc.edu Fri Mar 8 23:11:56 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id XAA19833 for ; Fri, 8 Mar 2002 23:11:55 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16jXvr-00032c-00 for ipfix-list@mil.doit.wisc.edu; Fri, 08 Mar 2002 21:54:07 -0600 Received: from mailhost.auckland.ac.nz ([130.216.1.4]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16jXvp-00032S-00 for ipfix@net.doit.wisc.edu; Fri, 08 Mar 2002 21:54:05 -0600 Received: from auckland.ac.nz (root@ccu1.auckland.ac.nz [130.216.3.1]) by mailhost.auckland.ac.nz (8.9.2/8.9.2/8.9.2-ua) with ESMTP id QAA06321 for ; Sat, 9 Mar 2002 16:53:58 +1300 (NZDT) Message-Id: <200203090353.QAA06321@mailhost.auckland.ac.nz> Date: Fri, 8 Mar 2002 19:55:50 -0800 (PST) From: n.brownlee@auckland.ac.nz Subject: [ipfix] Mailing list digest, Friday 8 March To: ipfix@net.doit.wisc.edu MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Precedence: bulk Sender: majordomo listserver IPFIX Co-chairs list of IPFIX issues Nevil Brownlee & Dave Plonka, Fri 8 Mar 02 ++ marks each issue ** WG co-chairs comment - Mailing list discussion items Contributors names are listed in full following the list of issues ++ Congestion-aware protocols ** Need to reach consensus on two major points: a. We know we need reliable transport. Do we also need unreliable? b. Given consesnus on (a), how do we provide congestion awareness? - Nevil, 4 Mar: Note setting out IESG / WH chair position on this issue. In brief, if IPFIX doesn't mandate congestion- aware transport, the WG RFCs won't get approved. ++ Collector Failover ** We need a control session handling keepalives. Much of this discussion hinges on the TCP vs UDP debate! - Paul, 27 Feb: Need to handle 'lost data' no matter what! - Mark, 25 Feb: UDP with retransmits as an alternative to TCP or SCTP. A TCP for keepalives could be enough? - Paul/Ganesh, 25 Feb: Not convinced that UDP can be ruled out; TCP on its own doesn't provide 'application level reliability.' - Peter, 25 Feb: 'Reliable' transport is better and simpler. "Any data delivery method that avoids reliability simply pushes things on to another part of the system." Use SCTP or TCP; data ACKs are enough, don;t need keepalives as well. - Mark, 23 Feb: Suggestions: use mandatory TCP control connection, require collector to send periodic ACKS to exporter. - Peter, 23 Feb: TCP keepalive not enough for exporter to detect crash of collector. Need to either use SCTP or have a failure detection mechanism. ------------------------------------------------------------- ++ Counter wrapping, long-running flows ** Support for both absolute counts (counters) and deltas, no clear consensus. - Benoit, 22 Feb: Close flow and export it when it nears counter wrap, restart as a new flow. - Ganesh, 21 Feb: Export flow records when counters go past a threshold, Keep IEs for absolute counts or delta counts, whichever is needed. - Peter, 21 Feb: In our experience, it's better to output deltas. Absolute values are needed only if you have an unreliable data exporter. - KC, 21 Feb: Ability to send a flow record at any interval due to a counter rollover needs to be a selection criteria. - Reinaldo, 19 Feb: Could someone elaborate on this? ++ Configuration: protocol ** Out of scope, i.e. WG should not consider this until we have the Arch and Data drafts well under way. - KC, 20 Feb: not doing that, trying to select best of existing protocols - Randy, 20 Feb: reminder, out of scope - Reinaldo, 20 Feb, IPFIX config protocol needed, MIB OK as well - Fred, 20 Feb: NetFlow v9: good summary posting 'bi-directional config' = collector can ask exporter for reqd flows Fred doesn't think it would be useful, lots of IOS development, implications for reliability, security, etc. Config via an 'IPFIX Configuration MIB' would be better. - Juergen/Reinaldo/Paul/Benoit/Ram, 14 Feb: 'Not trying to standardise meter, therefore don't specify how to configure one.' - Mike, 13 Feb: Configuration and MIBs: "A MIB module is a concise guide to implementors in their design of a configuration interface (CLI, XML, XYZ, whatever) to manage a given techology" - Mark, 12 Feb: Comments on exporter-collector session management, session IDs. - Will, 11 Feb: Doesn't want 'SHOULD support in-band configuration.' ++ Configuration: Content Negotiation - Christian/Peter/Juergen, ~19 Feb: Collector should be able to request a subset of all available flow attributes, exporter should indicate which attributes it can send. - Benoit, 19 Feb: This is pure configuration. You tell the exporter which fields to export by defining a template. ++ Configuration: In-band Configuration - Christian/Peter/Juergen: This is the ability for a collector to ask an exporter for a particular set of flows ++ Applicability document ** Material currently in Arch draft. Discuss at Minneapolis meeting. Charter mentions it to support Architecture draft, so its OK to start work on this, but don't let it distract from other WG drafts! Use ipfix-app list alias for discussion. - Carter, 20 Feb: OK, but don't go too far off topic - KC, 20 Feb: Renaldo, Kevin & Tanja contributed most of this material in Arch, they'd be good editors - Tanja, Feb 19: 'Applicability' contents proposal - Proposed by KC, support from Benoit ++ Middleboxes document *** Not in scope, material currently in Arch draft. Do we need a separate document? Or put it in Applicability draft? ++ Selecting a Flow Info Export protocol ** Our charter says we'll select a protocol, however we'll probably need to look (small) improvements to an existing protocol - 19 Feb: KC had intended to list 'overview' descriptions of existing protocols in Arch draft. Paul thinks this is premature, we should list 'selection criteria' there instead. KC agrees, meantime (so as not to loose the contributed text from B/G and Kevin) we'll shift it into appendices. - Randy, 18 Feb: realistically, i think you will have to modify any of the existing protocols to meet robust needs. and yes, i think security should be a consideration. [ note that snmpv3 control and tcp export over ipsec may meet some of the more difficult issues you raise ] - David, 18 Feb: Selection criteria are needed for this! ++ Defining a flow - Paul, 19 Feb: Selection process chooses packets to consider, flow definition rules determine which unique flows to export, i.e. flow definition rules may define sub-flows. Collector may wish to know the selection cfriteria. - Reinaldo, 19 Feb: Is a subset of a flow (i.e. one formed by adding another filter function) not 'just another' flow? ++ Overload conditions - KC/Robert, 13 Feb: Should exporter or collector be allowed to hanle overload by changing sampling rate? Or is it better to loose packets instead? ++ Security Considerations - Cliff, 12 Feb: text for Arch draft B/G = Benoit & Ganesh Benoit Calise, Cisco Carter Bullard, QOSient Christian Gilby, Nortel Cliff Wang, SmartPipes David Harrington, Enterasys Fred True, AT&T Research Ganesh Sadavisan, Cisco Jean-christophe Martin, Sun Juergen Quittek, NEC KC Norseth, Enterasys Kevin Zhang, Xacct Man M Li, Nokia Mark Fullmer, OARnet Paul Calato, Riverstone Peter Ludemann, Xacct Ram Gopal, Nokia Randy Bush (AD), AT&T Reinaldo Penno, Nortel Sebastian Zander, Fraunhofer Institute FOKUS Simon Leinen, Switch Tanja Zseby, Fraunhofer Institute FOKUS Venkatraman G, Infosys Technologies Will Eatherton, Cisco ++Other comments Robert Lowe , 15 Feb: The lurker opinion poll (no vote)... ;-) In-band configuration -- not now. Negotiation -- limited (but that discussion is promised to follow). Benoit, I think this was a very perceptive comment. It seems we are at times confusing the two. Peter Phaal , 13 Feb: The IPFIX charter doesn't address the algorithms used by the flow collector, but there are system-wide issues that should be explored. Design choices made in the IPFIX protocol specification can have a fundamental effect on the capabilities of traffic management systems collecting IPFIX data (scalability, latency, accuracy, multi-point correlation, etc.) ----------------------------------------------------------------------- Nevil Brownlee Director, Technology Development Phone: +64 9 373 7599 x8941 ITSS, The University of Auckland FAX: +64 9 373 7021 Private Bag 92019, Auckland, New Zealand -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri Mar 8 23:14:01 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id XAA19919 for ; Fri, 8 Mar 2002 23:14:01 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16jY1r-0003ED-00 for ipfix-list@mil.doit.wisc.edu; Fri, 08 Mar 2002 22:00:19 -0600 Received: from [130.216.191.4] (helo=mailhost.auckland.ac.nz) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16jY1p-0003Dw-00 for ipfix@net.doit.wisc.edu; Fri, 08 Mar 2002 22:00:17 -0600 Received: from auckland.ac.nz (root@ccu1.auckland.ac.nz [130.216.3.1]) by mailhost.auckland.ac.nz (8.9.2/8.9.2/8.9.2-ua) with ESMTP id RAA06736 for ; Sat, 9 Mar 2002 17:00:11 +1300 (NZDT) Message-Id: <200203090400.RAA06736@mailhost.auckland.ac.nz> Date: Fri, 8 Mar 2002 20:02:03 -0800 (PST) From: n.brownlee@auckland.ac.nz Subject: [ipfix] Agenda for the Minneapolis meeting To: ipfix@net.doit.wisc.edu MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Precedence: bulk Sender: majordomo listserver The IPFIX meeting at IETF 53 in Minneapolis is scheduled for Monday evening, i.e. 1930-2200 on 18 March 2002 Agenda: 1. Administriva 2. Overview of WG charter, current status 3. WG process issues (Co-chairs). How will we: - Develop the drafts? - Establish selection criteria for 'IPFIX protocol?' - Identify candidate protocols? - Select the IPFIX protocol? 4. Review of WG documents, highlighing current issues, e.g. configuration vs some configuration (ie template uploads) vs no configuration. (Document Editors) - Requirements - Architecture - Data Model - Applicability (mentioned in our charter, though not as a specific goal) 5. Protocol selection criteria (Co-chairs). We've been developing these in the Architecture draft. Review / discussion of progress so far 6. Other items - "Flow Measurements and Abilene" (Mark Fulmer) Any other items, offers of short talks about experiences with flow measurement, or anything else relevant to IPFIX, please send email to one of the co-chairs, i.e. to Nevil Brownlee or Dave Plonka Homework . . . * The IPFIX charter is at http://www.ietf.org/html.charters/ipfix-charter.html * Our current oublished Internet Drafts are Requirements: draft-ietf-ipfix-reqs-02.txt Architecture: draft-ietf-ipfix-architecture-00.txt Data Model: draft-ietf-ipfix-data-00.txt * The IPFIX web site is http://www.ietf.org/html.charters/ipfix-charter.html In its 'IPFIX-related Internet Drafts and RFCs' section there are URLs for more recent versions of the Architecture draft, and a preliminary version of an Applicability draft. ----------------------------------------------------------------------- Nevil Brownlee Director, Technology Development Phone: +64 9 373 7599 x8941 ITSS, The University of Auckland FAX: +64 9 373 7021 Private Bag 92019, Auckland, New Zealand -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Sat Mar 9 20:04:45 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA20120 for ; Sat, 9 Mar 2002 20:04:45 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16jrIt-0006iu-00 for ipfix-list@mil.doit.wisc.edu; Sat, 09 Mar 2002 18:35:11 -0600 Received: from zrc2s0jx.nortelnetworks.com ([47.103.122.112]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16jrIr-0006hq-00 for ipfix@net.doit.wisc.edu; Sat, 09 Mar 2002 18:35:09 -0600 Received: from zsc4c000.us.nortel.com (zsc4c000.us.nortel.com [47.81.138.47]) by zrc2s0jx.nortelnetworks.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id g2A0Xxu26106; Sat, 9 Mar 2002 18:33:59 -0600 (CST) Received: by zsc4c000.us.nortel.com with Internet Mail Service (5.5.2653.19) id ; Sat, 9 Mar 2002 16:33:55 -0800 Message-ID: <7B802811BE77D51189910002A55CFD2C01471999@zsc3c032.us.nortel.com> From: "Reinaldo Penno" To: n.brownlee@auckland.ac.nz, ipfix@net.doit.wisc.edu Subject: RE: [ipfix] Mailing list digest, Friday 8 March Date: Sat, 9 Mar 2002 16:34:04 -0800 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C1C7CB.47E17150" Precedence: bulk Sender: majordomo listserver This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C1C7CB.47E17150 Content-Type: text/plain; charset="ISO-8859-1" Hello Nevil/KC/All, Some comments on this digest... First of all I suggest we put together a small team of volunteers to come up with a presentation (doesn't even need to be a document) explaning what is in-band configuration, capabilities negotiation, content negotiation and other so we can start discussing based on a well understood terminology set. Even with the middlebox material out of the arch draft (today is in the applicability doc) we need to put something there to suggest how to behave in face of these services. I suggest something like: "An exporter may be involved in providing services that require application level intelligence and/or transform or filter content such as middlebox and OPES respectively. When involved in services that transform or filter content such as (but not limited to) NAT, Request Routing and Policing the exporter MUST be able to indicate on the flow record what changes were made to the packet." other comments inline... >-----Original Message----- >From: n.brownlee@auckland.ac.nz [mailto:n.brownlee@auckland.ac.nz] >Sent: Friday, March 08, 2002 7:56 PM >To: ipfix@net.doit.wisc.edu >Subject: [ipfix] Mailing list digest, Friday 8 March > > >IPFIX Co-chairs list of IPFIX issues > >Nevil Brownlee & Dave Plonka, Fri 8 Mar 02 > >++ marks each issue >** WG co-chairs comment > - Mailing list discussion items > > Contributors names are listed in full following the list of issues > >++ Congestion-aware protocols > ** Need to reach consensus on two major points: > a. We know we need reliable transport. Do we > also need unreliable? > b. Given consesnus on (a), how do we provide > congestion awareness? My vote is for a reliable protocol. Congestion awareness is already built-in on TCP ands SCTP. > - Nevil, 4 Mar: Note setting out IESG / WH chair position on > this issue. In brief, if IPFIX doesn't mandate congestion- > aware transport, the WG RFCs won't get approved. > >++ Collector Failover > ** We need a control session handling keepalives. > Much of this discussion hinges on the TCP vs UDP debate! > - Paul, 27 Feb: Need to handle 'lost data' no matter what! > - Mark, 25 Feb: UDP with retransmits as an alternative to TCP > or SCTP. A TCP for keepalives could be enough? > - Paul/Ganesh, 25 Feb: Not convinced that UDP can be ruled out; > TCP on its own doesn't provide 'application level reliability.' > - Peter, 25 Feb: 'Reliable' transport is better and simpler. "Any > data delivery method that avoids reliability simply >pushes things on > to another part of the system." Use SCTP or TCP; data ACKs > are enough, don;t need keepalives as well. > - Mark, 23 Feb: Suggestions: use mandatory TCP control connection, > require collector to send periodic ACKS to exporter. > - Peter, 23 Feb: TCP keepalive not enough for exporter to detect > crash of collector. Need to either use SCTP or have a failure > detection mechanism. I'm not sure what is my position in the collector failover debate. I would tend to vote in favor of SCTP instead of assuming certain things that TCP does not provide, such as keepalives, periodic acks, etc. Otherwise it would be better to put it on the application layer and have a special health check mechanism that does not only check if Layers 1-4 are working but also the application itself, just like L7 switches today. > > >------------------------------------------------------------- > I tought we reached a consensus on this. It was to support absolute counts (counters) and deltas. >++ Counter wrapping, long-running flows > ** Support for both absolute counts (counters) and deltas, no clear > consensus. > - Benoit, 22 Feb: Close flow and export it when it nears >counter wrap, > restart as a new flow. > - Ganesh, 21 Feb: Export flow records when counters go past >a threshold, > Keep IEs for absolute counts or delta counts, whichever is needed. > - Peter, 21 Feb: In our experience, it's better to output >deltas. Absolute > values are needed only if you have an unreliable data exporter. > - KC, 21 Feb: Ability to send a flow record at any interval >due to a > counter rollover needs to be a selection criteria. > - Reinaldo, 19 Feb: Could someone elaborate on this? > > This configuration/negotiation part goes back to my original suggestion. >++ Configuration: protocol > ** Out of scope, i.e. WG should not consider this until we have > the Arch and Data drafts well under way. > - KC, 20 Feb: not doing that, trying to select best of >existing protocols > - Randy, 20 Feb: reminder, out of scope > - Reinaldo, 20 Feb, IPFIX config protocol needed, MIB OK as well > - Fred, 20 Feb: NetFlow v9: good summary posting > 'bi-directional config' = collector can ask exporter >for reqd flows > Fred doesn't think it would be useful, lots of IOS development, > implications for reliability, security, etc. > Config via an 'IPFIX Configuration MIB' would be better. > - Juergen/Reinaldo/Paul/Benoit/Ram, 14 Feb: 'Not trying to >standardise > meter, therefore don't specify how to configure one.' > - Mike, 13 Feb: Configuration and MIBs: "A MIB module is a >concise guide > to implementors in their design of a configuration >interface (CLI, XML, > XYZ, whatever) to manage a given techology" > - Mark, 12 Feb: Comments on exporter-collector session management, > session IDs. > - Will, 11 Feb: Doesn't want 'SHOULD support in-band configuration.' > >++ Configuration: Content Negotiation > - Christian/Peter/Juergen, ~19 Feb: Collector should be >able to request > a subset of all available flow attributes, exporter >should indicate > which attributes it can send. > - Benoit, 19 Feb: This is pure configuration. You tell the exporter > which fields to export by defining a template. > >++ Configuration: In-band Configuration > - Christian/Peter/Juergen: This is the ability for a collector to > ask an exporter for a particular set of flows > > ------_=_NextPart_001_01C1C7CB.47E17150 Content-Type: text/html; charset="ISO-8859-1" Content-Transfer-Encoding: quoted-printable RE: [ipfix] Mailing list digest, Friday 8 March

Hello Nevil/KC/All,

Some comments on this digest...

First of all I suggest we put together a small team = of volunteers to come up with a presentation (doesn't even need to be a = document) explaning what is in-band configuration, capabilities = negotiation, content negotiation and other so we can start discussing = based on a well understood terminology set.

Even with the middlebox material out of the arch = draft (today is in the applicability doc) we need to put something = there to suggest how to behave in face of these services. I suggest = something like:

"An exporter may be involved in providing = services that require
application level intelligence and/or transform or = filter content such
as middlebox and OPES respectively.

When involved in services that transform or filter = content such as (but not limited to)
NAT, Request Routing and Policing the exporter MUST = be able to indicate on the flow
record what changes were made to the = packet."

other comments inline...

>-----Original Message-----
>From: n.brownlee@auckland.ac.nz [mailto:n.brownlee@auckland.ac.= nz]
>Sent: Friday, March 08, 2002 7:56 PM
>To: ipfix@net.doit.wisc.edu
>Subject: [ipfix] Mailing list digest, Friday 8 = March
>
>
>IPFIX Co-chairs list of IPFIX issues
>
>Nevil Brownlee & Dave Plonka, Fri 8 Mar = 02
>
>++ marks each issue
>** WG co-chairs comment
> - Mailing list discussion items
>
>   Contributors names are listed in = full following the list of issues
>
>++ Congestion-aware protocols
>   ** Need to reach consensus on two = major points:
>       a. We know = we need reliable transport.  Do we
>          = also need unreliable?
>       b. Given = consesnus on (a), how do we provide
>          = congestion awareness?

My vote is for a reliable protocol. Congestion = awareness is already built-in on TCP ands SCTP.

>   - Nevil, 4 Mar: Note setting out = IESG / WH chair position on
>     this issue.  In = brief, if IPFIX doesn't mandate congestion-
>     aware transport, the WG = RFCs won't get approved.
>
>++ Collector Failover
>   ** We need a control session = handling keepalives.
>      Much of this = discussion hinges on the TCP vs UDP debate!
>   - Paul, 27 Feb: Need to handle = 'lost data' no matter what!
>   - Mark, 25 Feb: UDP with = retransmits as an alternative to TCP
>     or SCTP.  A TCP = for keepalives could be enough?
>   - Paul/Ganesh, 25 Feb: Not = convinced that UDP can be ruled out;
>     TCP on its own doesn't = provide 'application level reliability.'
>   - Peter, 25 Feb: 'Reliable' = transport is better and simpler. "Any
>     data delivery method = that avoids reliability simply
>pushes things on
>     to another part of the = system."  Use SCTP or TCP; data ACKs
>     are enough, don;t need = keepalives as well.
>   - Mark, 23 Feb: Suggestions: use = mandatory TCP control connection,
>     require collector to = send periodic ACKS to exporter.
>   - Peter, 23 Feb: TCP keepalive not = enough for exporter to detect
>     crash of = collector.  Need to either use SCTP or have a failure
>     detection = mechanism.

I'm not sure what is my position in the collector = failover debate. I would tend to vote in favor of SCTP instead of = assuming certain things that TCP does not provide, such as keepalives, = periodic acks, etc.

Otherwise it would be better to put it on the = application layer and have a special health check mechanism that does = not only check if Layers 1-4 are working but also the application = itself, just like L7 switches today.

>
>
>-----------------------------------------------------------= --
>

I tought we reached a consensus on this. It was to = support absolute counts (counters) and deltas.

>++ Counter wrapping, long-running flows
>   ** Support for both absolute counts = (counters) and deltas, no clear
>      consensus.
>   - Benoit, 22 Feb: Close flow and = export it when it nears
>counter wrap,
>     restart as a new = flow.
>   - Ganesh, 21 Feb: Export flow = records when counters go past
>a threshold,
>     Keep IEs for absolute = counts or delta counts, whichever is needed.
>   - Peter, 21 Feb: In our experience, = it's better to output
>deltas. Absolute
>     values are needed only = if you have an unreliable data exporter.
>   - KC, 21 Feb: Ability to send a = flow record at any interval
>due to a
>     counter rollover needs = to be a selection criteria.
>   - Reinaldo, 19 Feb: Could someone = elaborate on this?
>
>

This configuration/negotiation part goes back to my = original suggestion.

>++ Configuration: protocol
>   ** Out of scope, i.e. WG should not = consider this until we have
>      the Arch and Data = drafts well under way.
>   - KC, 20 Feb: not doing that, = trying to select best of
>existing protocols
>   - Randy, 20 Feb: reminder, out of = scope
>   - Reinaldo, 20 Feb, IPFIX config = protocol needed, MIB OK as well
>   - Fred, 20 Feb: NetFlow v9: good = summary posting
>       = 'bi-directional config' =3D collector can ask exporter
>for reqd flows
>       Fred = doesn't think it would be useful, lots of IOS development,
>       = implications for reliability, security, etc.
>       Config via = an 'IPFIX Configuration MIB' would be better.
>   - Juergen/Reinaldo/Paul/Benoit/Ram, = 14 Feb: 'Not trying to
>standardise
>     meter, therefore don't = specify how to configure one.'
>   - Mike, 13 Feb: Configuration and = MIBs: "A MIB module is a
>concise guide
>     to implementors in = their design of a configuration
>interface (CLI, XML,
>     XYZ, whatever) to = manage a given techology"
>   - Mark, 12 Feb: Comments on = exporter-collector session management,
>     session IDs.
>   - Will, 11 Feb: Doesn't want = 'SHOULD support in-band configuration.'
>
>++ Configuration: Content Negotiation
>   - Christian/Peter/Juergen, ~19 Feb: = Collector should be
>able to request
>     a subset of all = available flow attributes, exporter
>should indicate
>     which  attributes = it can send.
>   - Benoit, 19 Feb: This is pure = configuration.  You tell the exporter
>     which fields to export = by defining a template.
>
>++ Configuration: In-band Configuration
>   - Christian/Peter/Juergen: This is = the ability for a collector to
>     ask an exporter for a = particular set of flows
>
>

------_=_NextPart_001_01C1C7CB.47E17150-- -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Sun Mar 10 23:55:40 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id XAA21021 for ; Sun, 10 Mar 2002 23:55:39 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16kHX8-0006C9-00 for ipfix-list@mil.doit.wisc.edu; Sun, 10 Mar 2002 22:35:38 -0600 Received: from mailhub.xacct.com ([204.253.100.25]) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 16kHX5-00066n-00 for ipfix@net.doit.wisc.edu; Sun, 10 Mar 2002 22:35:35 -0600 Received: (qmail 3286 invoked from network); 11 Mar 2002 04:34:59 -0000 Received: from unknown (HELO usmail.xacct.com) (204.253.100.12) by mailhub.us.xacct.com with SMTP; 11 Mar 2002 04:34:58 -0000 Received: from Kevinz (1Cust122.tnt4.dca5.da.uu.net [65.229.19.122]) by usmail.xacct.com (8.11.2/8.11.2) with SMTP id g2B4Yom28321; Sun, 10 Mar 2002 20:34:52 -0800 Reply-To: From: "Kevin Zhang" To: Cc: Subject: RE: [ipfix] Mailing list digest, Friday 8 March Date: Sun, 10 Mar 2002 23:34:27 -0500 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) Importance: Normal In-Reply-To: <200203090353.QAA06321@mailhost.auckland.ac.nz> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by ietf.org id XAA21021 Hi All, Please see my comments inserted. Thanks, Kevin > -----Original Message----- > From: majordomo listserver [mailto:majordomo@mil.doit.wisc.edu]On Behalf > Of n.brownlee@auckland.ac.nz > Sent: Friday, March 08, 2002 10:56 PM > To: ipfix@net.doit.wisc.edu > Subject: [ipfix] Mailing list digest, Friday 8 March > > > IPFIX Co-chairs list of IPFIX issues > > Nevil Brownlee & Dave Plonka, Fri 8 Mar 02 > > ++ marks each issue > ** WG co-chairs comment > - Mailing list discussion items > > Contributors names are listed in full following the list of issues > > ++ Congestion-aware protocols > ** Need to reach consensus on two major points: > a. We know we need reliable transport. Do we > also need unreliable? > b. Given consesnus on (a), how do we provide > congestion awareness? > - Nevil, 4 Mar: Note setting out IESG / WH chair position on > this issue. In brief, if IPFIX doesn't mandate congestion- > aware transport, the WG RFCs won't get approved. I support using a reliable transport protocol (e.g. TCP or SCTP), this takes care of both reliability and congestion awareness requirements outlined in our charter. > ++ Collector Failover > ** We need a control session handling keepalives. > Much of this discussion hinges on the TCP vs UDP debate! > - Paul, 27 Feb: Need to handle 'lost data' no matter what! > - Mark, 25 Feb: UDP with retransmits as an alternative to TCP > or SCTP. A TCP for keepalives could be enough? > - Paul/Ganesh, 25 Feb: Not convinced that UDP can be ruled out; > TCP on its own doesn't provide 'application level reliability.' > - Peter, 25 Feb: 'Reliable' transport is better and simpler. "Any > data delivery method that avoids reliability simply pushes things on > to another part of the system." Use SCTP or TCP; data ACKs > are enough, don;t need keepalives as well. > - Mark, 23 Feb: Suggestions: use mandatory TCP control connection, > require collector to send periodic ACKS to exporter. > - Peter, 23 Feb: TCP keepalive not enough for exporter to detect > crash of collector. Need to either use SCTP or have a failure > detection mechanism. > In my mind, IPFIX protocol can be viewed as an application running over a transport protocol (e.g. TCP, SCTP, UDP). It is a user of transport service provided by the lower layer. Among the transport protocols, the services can be summarized as - UDP: unreliable, not congestion aware, optional checksum to provide packet integrity check. TCP: reliable, in-sequence delivery, congestion aware, mandatory checksum to provide packet integrity, lacks support for quick failover/failback. SCTP: reliable, in-sequence delivery, congestion aware, provide packet integrity check, support quick failover/failback (e.g. using heartbeat messages). The options in front of us are - 1) IPFIX over UDP: This requires the IPFIX protocol to replicate many capabilities that can be provided by TCP and SCTP. This may result in a rather complicated IPFIX protocol. 2) IPFIX over TCP: IPFIX needs to implement IPFIX layer acknowledgment to achieve 'application level reliability', and 'heartbeat' procedures to support fast failover/failback. 3) IPFIX over SCTP: IPFIX needs to implement IPFIX layer acknowledgment to achieve 'application level reliability'. It appears to me IPFIX over TCP is a good compromise. As we need to have IPFIX layer acknowledgment, 'heartbeat' messages can be piggybacked or sent periodically for detecting collector/link failures. This can also simplify the interaction between the IPFIX and transport layer; as failover detection is handled by IPFIX layer, we do not need to worry about passing signals back and forth between IPFIX and SCTP. > > ------------------------------------------------------------- > > ++ Counter wrapping, long-running flows > ** Support for both absolute counts (counters) and deltas, no clear > consensus. > - Benoit, 22 Feb: Close flow and export it when it nears counter wrap, > restart as a new flow. > - Ganesh, 21 Feb: Export flow records when counters go past a > threshold, > Keep IEs for absolute counts or delta counts, whichever is needed. > - Peter, 21 Feb: In our experience, it's better to output > deltas. Absolute > values are needed only if you have an unreliable data exporter. > - KC, 21 Feb: Ability to send a flow record at any interval due to a > counter rollover needs to be a selection criteria. > - Reinaldo, 19 Feb: Could someone elaborate on this? > I did not follow this thread very closely. But if we close flow and restart each time near a counter wrap, are we violating our flow definition already? I am also not sure about the implications of this choice on applications; it appears there may result in may flows (or sessions) contains identical number of packets. > > ++ Configuration: protocol > ** Out of scope, i.e. WG should not consider this until we have > the Arch and Data drafts well under way. > - KC, 20 Feb: not doing that, trying to select best of > existing protocols > - Randy, 20 Feb: reminder, out of scope > - Reinaldo, 20 Feb, IPFIX config protocol needed, MIB OK as well > - Fred, 20 Feb: NetFlow v9: good summary posting > 'bi-directional config' = collector can ask exporter for reqd flows > Fred doesn't think it would be useful, lots of IOS development, > implications for reliability, security, etc. > Config via an 'IPFIX Configuration MIB' would be better. > - Juergen/Reinaldo/Paul/Benoit/Ram, 14 Feb: 'Not trying to standardise > meter, therefore don't specify how to configure one.' > - Mike, 13 Feb: Configuration and MIBs: "A MIB module is a > concise guide > to implementors in their design of a configuration interface > (CLI, XML, > XYZ, whatever) to manage a given techology" > - Mark, 12 Feb: Comments on exporter-collector session management, > session IDs. > - Will, 11 Feb: Doesn't want 'SHOULD support in-band configuration.' > > ++ Configuration: Content Negotiation > - Christian/Peter/Juergen, ~19 Feb: Collector should be able to request > a subset of all available flow attributes, exporter should indicate > which attributes it can send. > - Benoit, 19 Feb: This is pure configuration. You tell the exporter > which fields to export by defining a template. > I support content negotiation, exporter and collectors should at least negotiate a template that governs what are the exported attributes, and the export format. The collector should be allowed to indicate its requirements, and the exporter should try to meet them; though the exporter will ultimately determine what and how to export. > ++ Configuration: In-band Configuration > - Christian/Peter/Juergen: This is the ability for a collector to > ask an exporter for a particular set of flows > This seems to be the same as content negotiation, I support it. > > ++ Applicability document > ** Material currently in Arch draft. Discuss at Minneapolis meeting. > Charter mentions it to support Architecture draft, so its > OK to start work on this, but don't let it distract from other > WG drafts! Use ipfix-app list alias for discussion. > - Carter, 20 Feb: OK, but don't go too far off topic > - KC, 20 Feb: Renaldo, Kevin & Tanja contributed most of this material > in Arch, they'd be good editors > - Tanja, Feb 19: 'Applicability' contents proposal > - Proposed by KC, support from Benoit > > ++ Middleboxes document > *** Not in scope, material currently in Arch draft. > Do we need a separate document? Or put it in Applicability draft? > > > ++ Selecting a Flow Info Export protocol > ** Our charter says we'll select a protocol, however we'll probably > need to look (small) improvements to an existing protocol > - 19 Feb: KC had intended to list 'overview' descriptions of existing > protocols in Arch draft. Paul thinks this is premature, we > should list > 'selection criteria' there instead. KC agrees, meantime (so as not > to loose the contributed text from B/G and Kevin) we'll shift it into > appendices. > - Randy, 18 Feb: realistically, i think you will have to modify any of > the existing protocols to meet robust needs. and yes, i think > security should be a consideration. [ note that snmpv3 control and > tcp export over ipsec may meet some of the more difficult issues you > raise ] > - David, 18 Feb: Selection criteria are needed for this! > > > ++ Defining a flow > - Paul, 19 Feb: Selection process chooses packets to consider, > flow definition rules determine which unique flows to export, > i.e. flow definition rules may define sub-flows. Collector may > wish to know the selection cfriteria. > - Reinaldo, 19 Feb: Is a subset of a flow (i.e. one formed by adding > another filter function) not 'just another' flow? > > ++ Overload conditions > - KC/Robert, 13 Feb: Should exporter or collector be allowed to hanle > overload by changing sampling rate? Or is it better to loose > packets instead? > > ++ Security Considerations > - Cliff, 12 Feb: text for Arch draft > > > B/G = Benoit & Ganesh > > Benoit Calise, Cisco > Carter Bullard, QOSient > Christian Gilby, Nortel > Cliff Wang, SmartPipes > David Harrington, Enterasys > Fred True, AT&T Research > Ganesh Sadavisan, Cisco > Jean-christophe Martin, Sun > Juergen Quittek, NEC > KC Norseth, Enterasys > Kevin Zhang, Xacct > Man M Li, Nokia > Mark Fullmer, OARnet > Paul Calato, Riverstone > Peter Ludemann, Xacct > Ram Gopal, Nokia > Randy Bush (AD), AT&T > Reinaldo Penno, Nortel > Sebastian Zander, Fraunhofer Institute FOKUS > Simon Leinen, Switch > Tanja Zseby, Fraunhofer Institute FOKUS > Venkatraman G, Infosys Technologies > Will Eatherton, Cisco > > > ++Other comments > > Robert Lowe , 15 Feb: > The lurker opinion poll (no vote)... ;-) > In-band configuration -- not now. > Negotiation -- limited (but that discussion is promised to follow). > Benoit, I think this was a very perceptive comment. It seems we are > at times confusing the two. > > Peter Phaal , 13 Feb: > The IPFIX charter doesn't address the algorithms used by the flow > collector, but there are system-wide issues that should be explored. > Design choices made in the IPFIX protocol specification can have a > fundamental effect on the capabilities of traffic management systems > collecting IPFIX data (scalability, latency, accuracy, multi-point > correlation, etc.) > > > ----------------------------------------------------------------------- > Nevil Brownlee Director, Technology Development > Phone: +64 9 373 7599 x8941 ITSS, The University of Auckland > FAX: +64 9 373 7021 Private Bag 92019, Auckland, New Zealand > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in > message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ > > > ÿñÞ–™šŠ[hþf£¢·hšçzßÝ¢+ÿÂ+ýçnjwlk/ázZÿŠyž²Æ yºÉIì¹»®&Þ™¨¥¶æj:+v‰¨þw­ýÚ"·ü"±ÏÞvæ§vƲþéì¹»®&ÞŠ—âÇø§™ë,j›¡Ü€­Èb½èm¶Ÿÿþ*_‹Ý¢+ÿÂ+ýçnýªÜ†+Þ From majordomo@mil.doit.wisc.edu Mon Mar 11 10:49:48 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA12128 for ; Mon, 11 Mar 2002 10:49:47 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16kRe5-0006ep-00 for ipfix-list@mil.doit.wisc.edu; Mon, 11 Mar 2002 09:23:29 -0600 Received: from sj-msg-core-4.cisco.com ([171.71.163.10]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16kRe3-0006e7-00 for ipfix@net.doit.wisc.edu; Mon, 11 Mar 2002 09:23:27 -0600 Received: from mira-sjcd-1.cisco.com (mira-sjcd-1.cisco.com [171.69.43.44]) by sj-msg-core-4.cisco.com (8.11.3/8.9.1) with ESMTP id g2BFMuT07235; Mon, 11 Mar 2002 07:22:56 -0800 (PST) Received: from cisco.com (dhcp-171-71-137-36.cisco.com [171.71.137.36]) by mira-sjcd-1.cisco.com (Mirapoint) with ESMTP id ABN20397; Mon, 11 Mar 2002 07:23:25 -0800 (PST) Message-ID: <3C8CCBD0.1AE2458E@cisco.com> Date: Mon, 11 Mar 2002 07:22:56 -0800 From: Ganesh Sadasivan X-Mailer: Mozilla 4.76 [en]C-CCK-MCD (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: n.brownlee@auckland.ac.nz CC: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Mailing list digest, Friday 8 March References: <200203090353.QAA06321@mailhost.auckland.ac.nz> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit n.brownlee@auckland.ac.nz wrote: > IPFIX Co-chairs list of IPFIX issues > > Nevil Brownlee & Dave Plonka, Fri 8 Mar 02 > > ++ marks each issue > ** WG co-chairs comment > - Mailing list discussion items > > Contributors names are listed in full following the list of issues > > ++ Congestion-aware protocols > ** Need to reach consensus on two major points: > a. We know we need reliable transport. Do we > also need unreliable? > b. Given consesnus on (a), how do we provide > congestion awareness? > - Nevil, 4 Mar: Note setting out IESG / WH chair position on > this issue. In brief, if IPFIX doesn't mandate congestion- > aware transport, the WG RFCs won't get approved. So the conclusion is to use a congestion aware protocol or use DCP with external congestion control - correct? > > > > ++ Collector Failover > ** We need a control session handling keepalives. > Much of this discussion hinges on the TCP vs UDP debate! > - Paul, 27 Feb: Need to handle 'lost data' no matter what! > - Mark, 25 Feb: UDP with retransmits as an alternative to TCP > or SCTP. A TCP for keepalives could be enough? > - Paul/Ganesh, 25 Feb: Not convinced that UDP can be ruled out; > TCP on its own doesn't provide 'application level reliability.' > - Peter, 25 Feb: 'Reliable' transport is better and simpler. "Any > data delivery method that avoids reliability simply pushes things on > to another part of the system." Use SCTP or TCP; data ACKs > are enough, don;t need keepalives as well. > - Mark, 23 Feb: Suggestions: use mandatory TCP control connection, > require collector to send periodic ACKS to exporter. > - Peter, 23 Feb: TCP keepalive not enough for exporter to detect > crash of collector. Need to either use SCTP or have a failure > detection mechanism. > > > ------------------------------------------------------------- > > ++ Counter wrapping, long-running flows > ** Support for both absolute counts (counters) and deltas, no clear > consensus. > - Benoit, 22 Feb: Close flow and export it when it nears counter wrap, > restart as a new flow. > - Ganesh, 21 Feb: Export flow records when counters go past a threshold, > Keep IEs for absolute counts or delta counts, whichever is needed. > - Peter, 21 Feb: In our experience, it's better to output deltas. Absolute > values are needed only if you have an unreliable data exporter. > - KC, 21 Feb: Ability to send a flow record at any interval due to a > counter rollover needs to be a selection criteria. > - Reinaldo, 19 Feb: Could someone elaborate on this? > > ++ Configuration: protocol > ** Out of scope, i.e. WG should not consider this until we have > the Arch and Data drafts well under way. > - KC, 20 Feb: not doing that, trying to select best of existing protocols > - Randy, 20 Feb: reminder, out of scope > - Reinaldo, 20 Feb, IPFIX config protocol needed, MIB OK as well > - Fred, 20 Feb: NetFlow v9: good summary posting > 'bi-directional config' = collector can ask exporter for reqd flows > Fred doesn't think it would be useful, lots of IOS development, > implications for reliability, security, etc. > Config via an 'IPFIX Configuration MIB' would be better. > - Juergen/Reinaldo/Paul/Benoit/Ram, 14 Feb: 'Not trying to standardise > meter, therefore don't specify how to configure one.' > - Mike, 13 Feb: Configuration and MIBs: "A MIB module is a concise guide > to implementors in their design of a configuration interface (CLI, XML, > XYZ, whatever) to manage a given techology" > - Mark, 12 Feb: Comments on exporter-collector session management, > session IDs. > - Will, 11 Feb: Doesn't want 'SHOULD support in-band configuration.' > > ++ Configuration: Content Negotiation > - Christian/Peter/Juergen, ~19 Feb: Collector should be able to request > a subset of all available flow attributes, exporter should indicate > which attributes it can send. > - Benoit, 19 Feb: This is pure configuration. You tell the exporter > which fields to export by defining a template. There is a good amount of discussions in http://ipfix.doit.wisc.edu/archive/0753.html on both Content Negotiation and In-band Configuration. > > > ++ Configuration: In-band Configuration > - Christian/Peter/Juergen: This is the ability for a collector to > ask an exporter for a particular set of flows > > ++ Applicability document > ** Material currently in Arch draft. Discuss at Minneapolis meeting. > Charter mentions it to support Architecture draft, so its > OK to start work on this, but don't let it distract from other > WG drafts! Use ipfix-app list alias for discussion. > - Carter, 20 Feb: OK, but don't go too far off topic > - KC, 20 Feb: Renaldo, Kevin & Tanja contributed most of this material > in Arch, they'd be good editors > - Tanja, Feb 19: 'Applicability' contents proposal > - Proposed by KC, support from Benoit > > ++ Middleboxes document > *** Not in scope, material currently in Arch draft. > Do we need a separate document? Or put it in Applicability draft? > > ++ Selecting a Flow Info Export protocol > ** Our charter says we'll select a protocol, however we'll probably > need to look (small) improvements to an existing protocol > - 19 Feb: KC had intended to list 'overview' descriptions of existing > protocols in Arch draft. Paul thinks this is premature, we should list > 'selection criteria' there instead. KC agrees, meantime (so as not > to loose the contributed text from B/G and Kevin) we'll shift it into > appendices. > - Randy, 18 Feb: realistically, i think you will have to modify any of > the existing protocols to meet robust needs. and yes, i think > security should be a consideration. [ note that snmpv3 control and > tcp export over ipsec may meet some of the more difficult issues you > raise ] > - David, 18 Feb: Selection criteria are needed for this! > > ++ Defining a flow > - Paul, 19 Feb: Selection process chooses packets to consider, > flow definition rules determine which unique flows to export, > i.e. flow definition rules may define sub-flows. Collector may > wish to know the selection cfriteria. > - Reinaldo, 19 Feb: Is a subset of a flow (i.e. one formed by adding > another filter function) not 'just another' flow? > > ++ Overload conditions > - KC/Robert, 13 Feb: Should exporter or collector be allowed to hanle > overload by changing sampling rate? Or is it better to loose > packets instead? > > > ++ Security Considerations > - Cliff, 12 Feb: text for Arch draft > > B/G = Benoit & Ganesh > > Benoit Calise, Cisco > Carter Bullard, QOSient > Christian Gilby, Nortel > Cliff Wang, SmartPipes > David Harrington, Enterasys > Fred True, AT&T Research > Ganesh Sadavisan, Cisco > Jean-christophe Martin, Sun > Juergen Quittek, NEC > KC Norseth, Enterasys > Kevin Zhang, Xacct > Man M Li, Nokia > Mark Fullmer, OARnet > Paul Calato, Riverstone > Peter Ludemann, Xacct > Ram Gopal, Nokia > Randy Bush (AD), AT&T > Reinaldo Penno, Nortel > Sebastian Zander, Fraunhofer Institute FOKUS > Simon Leinen, Switch > Tanja Zseby, Fraunhofer Institute FOKUS > Venkatraman G, Infosys Technologies > Will Eatherton, Cisco > > ++Other comments > > Robert Lowe , 15 Feb: > The lurker opinion poll (no vote)... ;-) > In-band configuration -- not now. > Negotiation -- limited (but that discussion is promised to follow). > Benoit, I think this was a very perceptive comment. It seems we are > at times confusing the two. > > Peter Phaal , 13 Feb: > The IPFIX charter doesn't address the algorithms used by the flow > collector, but there are system-wide issues that should be explored. > Design choices made in the IPFIX protocol specification can have a > fundamental effect on the capabilities of traffic management systems > collecting IPFIX data (scalability, latency, accuracy, multi-point > correlation, etc.) > > ----------------------------------------------------------------------- > Nevil Brownlee Director, Technology Development > Phone: +64 9 373 7599 x8941 ITSS, The University of Auckland > FAX: +64 9 373 7021 Private Bag 92019, Auckland, New Zealand > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon Mar 11 12:33:21 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA16635 for ; Mon, 11 Mar 2002 12:33:21 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16kTMg-00015i-00 for ipfix-list@mil.doit.wisc.edu; Mon, 11 Mar 2002 11:13:38 -0600 Received: from host3.riverstonenet.com ([63.113.148.3] helo=exc-sc1.yagosys.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16kTMd-00014i-00 for ipfix@net.doit.wisc.edu; Mon, 11 Mar 2002 11:13:35 -0600 Received: from riverstonenet.com (134.141.180.109 [134.141.180.109]) by exc-sc1.yagosys.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13) id 1PZZFF97; Mon, 11 Mar 2002 09:12:08 -0800 Message-ID: <3C8CE567.861AE644@riverstonenet.com> Date: Mon, 11 Mar 2002 12:12:07 -0500 From: calato@riverstonenet.com X-Mailer: Mozilla 4.75C-CCK-MCD NSCPCD475 [en] (X11; U; SunOS 5.6 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: n.brownlee@auckland.ac.nz CC: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Mailing list digest, Friday 8 March References: <200203090353.QAA06321@mailhost.auckland.ac.nz> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit n.brownlee@auckland.ac.nz wrote: > > IPFIX Co-chairs list of IPFIX issues > > Nevil Brownlee & Dave Plonka, Fri 8 Mar 02 > > ++ marks each issue > ** WG co-chairs comment > - Mailing list discussion items > > Contributors names are listed in full following the list of issues > > ++ Congestion-aware protocols > ** Need to reach consensus on two major points: > a. We know we need reliable transport. Do we > also need unreliable? > b. Given consesnus on (a), how do we provide > congestion awareness? > - Nevil, 4 Mar: Note setting out IESG / WH chair position on > this issue. In brief, if IPFIX doesn't mandate congestion- > aware transport, the WG RFCs won't get approved. I believe there are a large number of real world deployment cases where IPFIX is not going over the Internet. In fact, it is likely to be the majority of deployment cases. Is there disagreement on this point? If not, we ought to decide the best way to solve the problem first, then decide what that means for us as an IETF group; not the other way around. > > ++ Collector Failover > ** We need a control session handling keepalives. > Much of this discussion hinges on the TCP vs UDP debate! > - Paul, 27 Feb: Need to handle 'lost data' no matter what! > - Mark, 25 Feb: UDP with retransmits as an alternative to TCP > or SCTP. A TCP for keepalives could be enough? > - Paul/Ganesh, 25 Feb: Not convinced that UDP can be ruled out; > TCP on its own doesn't provide 'application level reliability.' > - Peter, 25 Feb: 'Reliable' transport is better and simpler. "Any > data delivery method that avoids reliability simply pushes things on > to another part of the system." Use SCTP or TCP; data ACKs > are enough, don;t need keepalives as well. > - Mark, 23 Feb: Suggestions: use mandatory TCP control connection, > require collector to send periodic ACKS to exporter. > - Peter, 23 Feb: TCP keepalive not enough for exporter to detect > crash of collector. Need to either use SCTP or have a failure > detection mechanism. > > ------------------------------------------------------------- > > ++ Counter wrapping, long-running flows > ** Support for both absolute counts (counters) and deltas, no clear > consensus. > - Benoit, 22 Feb: Close flow and export it when it nears counter wrap, > restart as a new flow. > - Ganesh, 21 Feb: Export flow records when counters go past a threshold, > Keep IEs for absolute counts or delta counts, whichever is needed. > - Peter, 21 Feb: In our experience, it's better to output deltas. Absolute > values are needed only if you have an unreliable data exporter. > - KC, 21 Feb: Ability to send a flow record at any interval due to a > counter rollover needs to be a selection criteria. > - Reinaldo, 19 Feb: Could someone elaborate on this? > > ++ Configuration: protocol > ** Out of scope, i.e. WG should not consider this until we have > the Arch and Data drafts well under way. > - KC, 20 Feb: not doing that, trying to select best of existing protocols > - Randy, 20 Feb: reminder, out of scope > - Reinaldo, 20 Feb, IPFIX config protocol needed, MIB OK as well > - Fred, 20 Feb: NetFlow v9: good summary posting > 'bi-directional config' = collector can ask exporter for reqd flows > Fred doesn't think it would be useful, lots of IOS development, > implications for reliability, security, etc. > Config via an 'IPFIX Configuration MIB' would be better. > - Juergen/Reinaldo/Paul/Benoit/Ram, 14 Feb: 'Not trying to standardise > meter, therefore don't specify how to configure one.' > - Mike, 13 Feb: Configuration and MIBs: "A MIB module is a concise guide > to implementors in their design of a configuration interface (CLI, XML, > XYZ, whatever) to manage a given techology" > - Mark, 12 Feb: Comments on exporter-collector session management, > session IDs. > - Will, 11 Feb: Doesn't want 'SHOULD support in-band configuration.' > > ++ Configuration: Content Negotiation > - Christian/Peter/Juergen, ~19 Feb: Collector should be able to request > a subset of all available flow attributes, exporter should indicate > which attributes it can send. > - Benoit, 19 Feb: This is pure configuration. You tell the exporter > which fields to export by defining a template. > > ++ Configuration: In-band Configuration > - Christian/Peter/Juergen: This is the ability for a collector to > ask an exporter for a particular set of flows > > ++ Applicability document > ** Material currently in Arch draft. Discuss at Minneapolis meeting. > Charter mentions it to support Architecture draft, so its > OK to start work on this, but don't let it distract from other > WG drafts! Use ipfix-app list alias for discussion. > - Carter, 20 Feb: OK, but don't go too far off topic > - KC, 20 Feb: Renaldo, Kevin & Tanja contributed most of this material > in Arch, they'd be good editors > - Tanja, Feb 19: 'Applicability' contents proposal > - Proposed by KC, support from Benoit > > ++ Middleboxes document > *** Not in scope, material currently in Arch draft. > Do we need a separate document? Or put it in Applicability draft? > > ++ Selecting a Flow Info Export protocol > ** Our charter says we'll select a protocol, however we'll probably > need to look (small) improvements to an existing protocol > - 19 Feb: KC had intended to list 'overview' descriptions of existing > protocols in Arch draft. Paul thinks this is premature, we should list > 'selection criteria' there instead. KC agrees, meantime (so as not > to loose the contributed text from B/G and Kevin) we'll shift it into > appendices. > - Randy, 18 Feb: realistically, i think you will have to modify any of > the existing protocols to meet robust needs. and yes, i think > security should be a consideration. [ note that snmpv3 control and > tcp export over ipsec may meet some of the more difficult issues you > raise ] > - David, 18 Feb: Selection criteria are needed for this! > > ++ Defining a flow > - Paul, 19 Feb: Selection process chooses packets to consider, > flow definition rules determine which unique flows to export, > i.e. flow definition rules may define sub-flows. Collector may > wish to know the selection cfriteria. > - Reinaldo, 19 Feb: Is a subset of a flow (i.e. one formed by adding > another filter function) not 'just another' flow? > > ++ Overload conditions > - KC/Robert, 13 Feb: Should exporter or collector be allowed to hanle > overload by changing sampling rate? Or is it better to loose > packets instead? > > ++ Security Considerations > - Cliff, 12 Feb: text for Arch draft > > B/G = Benoit & Ganesh > > Benoit Calise, Cisco > Carter Bullard, QOSient > Christian Gilby, Nortel > Cliff Wang, SmartPipes > David Harrington, Enterasys > Fred True, AT&T Research > Ganesh Sadavisan, Cisco > Jean-christophe Martin, Sun > Juergen Quittek, NEC > KC Norseth, Enterasys > Kevin Zhang, Xacct > Man M Li, Nokia > Mark Fullmer, OARnet > Paul Calato, Riverstone > Peter Ludemann, Xacct > Ram Gopal, Nokia > Randy Bush (AD), AT&T > Reinaldo Penno, Nortel > Sebastian Zander, Fraunhofer Institute FOKUS > Simon Leinen, Switch > Tanja Zseby, Fraunhofer Institute FOKUS > Venkatraman G, Infosys Technologies > Will Eatherton, Cisco > > ++Other comments > > Robert Lowe , 15 Feb: > The lurker opinion poll (no vote)... ;-) > In-band configuration -- not now. > Negotiation -- limited (but that discussion is promised to follow). > Benoit, I think this was a very perceptive comment. It seems we are > at times confusing the two. > > Peter Phaal , 13 Feb: > The IPFIX charter doesn't address the algorithms used by the flow > collector, but there are system-wide issues that should be explored. > Design choices made in the IPFIX protocol specification can have a > fundamental effect on the capabilities of traffic management systems > collecting IPFIX data (scalability, latency, accuracy, multi-point > correlation, etc.) > > ----------------------------------------------------------------------- > Nevil Brownlee Director, Technology Development > Phone: +64 9 373 7599 x8941 ITSS, The University of Auckland > FAX: +64 9 373 7021 Private Bag 92019, Auckland, New Zealand > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon Mar 11 13:41:06 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA18901 for ; Mon, 11 Mar 2002 13:41:01 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16kUFM-0002Bf-00 for ipfix-list@mil.doit.wisc.edu; Mon, 11 Mar 2002 12:10:08 -0600 Received: from mailhost.auckland.ac.nz ([130.216.1.4]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16kUFK-0002Ba-00 for ipfix@net.doit.wisc.edu; Mon, 11 Mar 2002 12:10:06 -0600 Received: from auckland.ac.nz (root@ccu1.auckland.ac.nz [130.216.3.1]) by mailhost.auckland.ac.nz (8.9.2/8.9.2/8.9.2-ua) with ESMTP id HAA07236 for ; Tue, 12 Mar 2002 07:10:00 +1300 (NZDT) Message-Id: <200203111810.HAA07236@mailhost.auckland.ac.nz> Date: Mon, 11 Mar 2002 10:11:48 -0800 (PST) From: n.brownlee@auckland.ac.nz Subject: Re: [ipfix] Mailing list digest, Friday 8 March To: ipfix@net.doit.wisc.edu In-Reply-To: <3C8CE567.861AE644@riverstonenet.com> MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Precedence: bulk Sender: majordomo listserver Hi Paul: On 11 Mar, calato@riverstonenet.com wrote: > >> - Nevil, 4 Mar: Note setting out IESG / WH chair position on >> this issue. In brief, if IPFIX doesn't mandate congestion- >> aware transport, the WG RFCs won't get approved. > > I believe there are a large number of real world deployment > cases where IPFIX is not going over the Internet. In fact, > it is likely to be the majority of deployment cases. Is there > disagreement on this point? > > If not, we ought to decide the best way to solve the problem > first, then decide what that means for us as an IETF group; > not the other way around. As an IETF WG we're honour-bound to come up with an architecture/ protocol/whatever that's good for the Internet overall, that's why we have to use congestion-aware transport. 'Solving the problem,' in a non-Internet setting, is not ----------------------------------------------------------------------- Nevil Brownlee Director, Technology Development Phone: +64 9 373 7599 x8941 ITSS, The University of Auckland FAX: +64 9 373 7021 Private Bag 92019, Auckland, New Zealand -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon Mar 11 13:57:34 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA19288 for ; Mon, 11 Mar 2002 13:57:34 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16kUeO-0002hw-00 for ipfix-list@mil.doit.wisc.edu; Mon, 11 Mar 2002 12:36:00 -0600 Received: from mailhub.xacct.com ([204.253.100.25]) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 16kUeK-0002hH-00 for ipfix@net.doit.wisc.edu; Mon, 11 Mar 2002 12:35:56 -0600 Received: (qmail 22460 invoked from network); 11 Mar 2002 18:35:22 -0000 Received: from unknown (HELO usmail.xacct.com) (204.253.100.12) by mailhub.us.xacct.com with SMTP; 11 Mar 2002 18:35:22 -0000 Received: from Kevinz (1Cust95.tnt15.dca5.da.uu.net [63.10.143.95]) by usmail.xacct.com (8.11.2/8.11.2) with SMTP id g2BIZGm02256; Mon, 11 Mar 2002 10:35:17 -0800 Reply-To: From: "Kevin Zhang" To: Cc: , Subject: RE: [ipfix] Mailing list digest, Friday 8 March Date: Mon, 11 Mar 2002 13:34:55 -0500 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 In-Reply-To: <3C8CE567.861AE644@riverstonenet.com> Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by ietf.org id NAA19288 Hi Paul, I have to disagree with you, please see my comments. Thanks, Kevin > -----Original Message----- > From: majordomo listserver [mailto:majordomo@mil.doit.wisc.edu]On Behalf > Of calato@riverstonenet.com > Sent: Monday, March 11, 2002 12:12 PM > To: n.brownlee@auckland.ac.nz > Cc: ipfix@net.doit.wisc.edu > Subject: Re: [ipfix] Mailing list digest, Friday 8 March > > > n.brownlee@auckland.ac.nz wrote: > > > > IPFIX Co-chairs list of IPFIX issues > > > > Nevil Brownlee & Dave Plonka, Fri 8 Mar 02 > > > > ++ marks each issue > > ** WG co-chairs comment > > - Mailing list discussion items > > > > Contributors names are listed in full following the list of issues > > > > ++ Congestion-aware protocols > > ** Need to reach consensus on two major points: > > a. We know we need reliable transport. Do we > > also need unreliable? > > b. Given consesnus on (a), how do we provide > > congestion awareness? > > - Nevil, 4 Mar: Note setting out IESG / WH chair position on > > this issue. In brief, if IPFIX doesn't mandate congestion- > > aware transport, the WG RFCs won't get approved. > > > I believe there are a large number of real world deployment > cases where IPFIX is not going over the Internet. In fact, > it is likely to be the majority of deployment cases. Is there > disagreement on this point? > > If not, we ought to decide the best way to solve the problem > first, then decide what that means for us as an IETF group; > not the other way around. > > I believe CRANE protocol (http://search.ietf.org/internet-drafts/draft-kzhang-crane-protocol-02.txt) is considered as a candidate for IPFIX. We have several deployments with tier one carriers, which the exporter and collectors communicate through WAN. Furthermore, potential IPFIX exporting devices may be routers, probes, and mid-boxes as outlined in our charter; I do not agree that the need of high-speed router (may have limited on-device flow processing capability) should over-ride all the other needs. > > > > > ++ Collector Failover > > ** We need a control session handling keepalives. > > Much of this discussion hinges on the TCP vs UDP debate! > > - Paul, 27 Feb: Need to handle 'lost data' no matter what! > > - Mark, 25 Feb: UDP with retransmits as an alternative to TCP > > or SCTP. A TCP for keepalives could be enough? > > - Paul/Ganesh, 25 Feb: Not convinced that UDP can be ruled out; > > TCP on its own doesn't provide 'application level reliability.' > > - Peter, 25 Feb: 'Reliable' transport is better and simpler. "Any > > data delivery method that avoids reliability simply pushes > things on > > to another part of the system." Use SCTP or TCP; data ACKs > > are enough, don;t need keepalives as well. > > - Mark, 23 Feb: Suggestions: use mandatory TCP control connection, > > require collector to send periodic ACKS to exporter. > > - Peter, 23 Feb: TCP keepalive not enough for exporter to detect > > crash of collector. Need to either use SCTP or have a failure > > detection mechanism. > > > > ------------------------------------------------------------- > > > > ++ Counter wrapping, long-running flows > > ** Support for both absolute counts (counters) and deltas, no clear > > consensus. > > - Benoit, 22 Feb: Close flow and export it when it nears > counter wrap, > > restart as a new flow. > > - Ganesh, 21 Feb: Export flow records when counters go past > a threshold, > > Keep IEs for absolute counts or delta counts, whichever is needed. > > - Peter, 21 Feb: In our experience, it's better to output > deltas. Absolute > > values are needed only if you have an unreliable data exporter. > > - KC, 21 Feb: Ability to send a flow record at any interval due to a > > counter rollover needs to be a selection criteria. > > - Reinaldo, 19 Feb: Could someone elaborate on this? > > > > ++ Configuration: protocol > > ** Out of scope, i.e. WG should not consider this until we have > > the Arch and Data drafts well under way. > > - KC, 20 Feb: not doing that, trying to select best of > existing protocols > > - Randy, 20 Feb: reminder, out of scope > > - Reinaldo, 20 Feb, IPFIX config protocol needed, MIB OK as well > > - Fred, 20 Feb: NetFlow v9: good summary posting > > 'bi-directional config' = collector can ask exporter for > reqd flows > > Fred doesn't think it would be useful, lots of IOS development, > > implications for reliability, security, etc. > > Config via an 'IPFIX Configuration MIB' would be better. > > - Juergen/Reinaldo/Paul/Benoit/Ram, 14 Feb: 'Not trying to > standardise > > meter, therefore don't specify how to configure one.' > > - Mike, 13 Feb: Configuration and MIBs: "A MIB module is a > concise guide > > to implementors in their design of a configuration > interface (CLI, XML, > > XYZ, whatever) to manage a given techology" > > - Mark, 12 Feb: Comments on exporter-collector session management, > > session IDs. > > - Will, 11 Feb: Doesn't want 'SHOULD support in-band configuration.' > > > > ++ Configuration: Content Negotiation > > - Christian/Peter/Juergen, ~19 Feb: Collector should be able > to request > > a subset of all available flow attributes, exporter should indicate > > which attributes it can send. > > - Benoit, 19 Feb: This is pure configuration. You tell the exporter > > which fields to export by defining a template. > > > > ++ Configuration: In-band Configuration > > - Christian/Peter/Juergen: This is the ability for a collector to > > ask an exporter for a particular set of flows > > > > ++ Applicability document > > ** Material currently in Arch draft. Discuss at Minneapolis meeting. > > Charter mentions it to support Architecture draft, so its > > OK to start work on this, but don't let it distract from other > > WG drafts! Use ipfix-app list alias for discussion. > > - Carter, 20 Feb: OK, but don't go too far off topic > > - KC, 20 Feb: Renaldo, Kevin & Tanja contributed most of > this material > > in Arch, they'd be good editors > > - Tanja, Feb 19: 'Applicability' contents proposal > > - Proposed by KC, support from Benoit > > > > ++ Middleboxes document > > *** Not in scope, material currently in Arch draft. > > Do we need a separate document? Or put it in > Applicability draft? > > > > ++ Selecting a Flow Info Export protocol > > ** Our charter says we'll select a protocol, however we'll probably > > need to look (small) improvements to an existing protocol > > - 19 Feb: KC had intended to list 'overview' descriptions of existing > > protocols in Arch draft. Paul thinks this is premature, > we should list > > 'selection criteria' there instead. KC agrees, meantime (so as not > > to loose the contributed text from B/G and Kevin) we'll > shift it into > > appendices. > > - Randy, 18 Feb: realistically, i think you will have to > modify any of > > the existing protocols to meet robust needs. and yes, i think > > security should be a consideration. [ note that snmpv3 control and > > tcp export over ipsec may meet some of the more difficult > issues you > > raise ] > > - David, 18 Feb: Selection criteria are needed for this! > > > > ++ Defining a flow > > - Paul, 19 Feb: Selection process chooses packets to consider, > > flow definition rules determine which unique flows to export, > > i.e. flow definition rules may define sub-flows. Collector may > > wish to know the selection cfriteria. > > - Reinaldo, 19 Feb: Is a subset of a flow (i.e. one formed by adding > > another filter function) not 'just another' flow? > > > > ++ Overload conditions > > - KC/Robert, 13 Feb: Should exporter or collector be allowed to hanle > > overload by changing sampling rate? Or is it better to loose > > packets instead? > > > > ++ Security Considerations > > - Cliff, 12 Feb: text for Arch draft > > > > B/G = Benoit & Ganesh > > > > Benoit Calise, Cisco > > Carter Bullard, QOSient > > Christian Gilby, Nortel > > Cliff Wang, SmartPipes > > David Harrington, Enterasys > > Fred True, AT&T Research > > Ganesh Sadavisan, Cisco > > Jean-christophe Martin, Sun > > Juergen Quittek, NEC > > KC Norseth, Enterasys > > Kevin Zhang, Xacct > > Man M Li, Nokia > > Mark Fullmer, OARnet > > Paul Calato, Riverstone > > Peter Ludemann, Xacct > > Ram Gopal, Nokia > > Randy Bush (AD), AT&T > > Reinaldo Penno, Nortel > > Sebastian Zander, Fraunhofer Institute FOKUS > > Simon Leinen, Switch > > Tanja Zseby, Fraunhofer Institute FOKUS > > Venkatraman G, Infosys Technologies > > Will Eatherton, Cisco > > > > ++Other comments > > > > Robert Lowe , 15 Feb: > > The lurker opinion poll (no vote)... ;-) > > In-band configuration -- not now. > > Negotiation -- limited (but that discussion is promised to follow). > > Benoit, I think this was a very perceptive comment. It seems we are > > at times confusing the two. > > > > Peter Phaal , 13 Feb: > > The IPFIX charter doesn't address the algorithms used by the flow > > collector, but there are system-wide issues that should be explored. > > Design choices made in the IPFIX protocol specification can have a > > fundamental effect on the capabilities of traffic management systems > > collecting IPFIX data (scalability, latency, accuracy, multi-point > > correlation, etc.) > > > > ----------------------------------------------------------------------- > > Nevil Brownlee Director, Technology Development > > Phone: +64 9 373 7599 x8941 ITSS, The University of Auckland > > FAX: +64 9 373 7021 Private Bag 92019, Auckland, New Zealand > > > > -- > > Help mailto:majordomo@net.doit.wisc.edu and say "help" > in message body > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe ipfix" in message body > > Archive http://ipfix.doit.wisc.edu/archive/ > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in > message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ > > > ÿñÞ–™šŠ[hþf£¢·hšçzßÝ¢+ÿÂ+ýçnjwlk/ázZÿŠyž²Æ yºÉIì¹»®&Þ™¨¥¶æj:+v‰¨þw­ýÚ"·ü"±ÏÞvæ§vƲþéì¹»®&ÞŠ—âÇø§™ë,j›¡Ü€­Èb½èm¶Ÿÿþ*_‹Ý¢+ÿÂ+ýçnýªÜ†+Þ From majordomo@mil.doit.wisc.edu Mon Mar 11 14:09:12 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA19570 for ; Mon, 11 Mar 2002 14:09:11 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16kUuK-0002zq-00 for ipfix-list@mil.doit.wisc.edu; Mon, 11 Mar 2002 12:52:28 -0600 Received: from host3.riverstonenet.com ([63.113.148.3] helo=exc-sc1.yagosys.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16kUuH-0002z2-00 for ipfix@net.doit.wisc.edu; Mon, 11 Mar 2002 12:52:25 -0600 Received: from riverstonenet.com (134.141.180.109 [134.141.180.109]) by exc-sc1.yagosys.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13) id 1PZZFHGV; Mon, 11 Mar 2002 10:51:03 -0800 Message-ID: <3C8CFC95.9F845DC7@riverstonenet.com> Date: Mon, 11 Mar 2002 13:51:01 -0500 From: calato@riverstonenet.com X-Mailer: Mozilla 4.75C-CCK-MCD NSCPCD475 [en] (X11; U; SunOS 5.6 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: n.brownlee@auckland.ac.nz CC: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Mailing list digest, Friday 8 March References: <200203111810.HAA07236@mailhost.auckland.ac.nz> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit n.brownlee@auckland.ac.nz wrote: > > Hi Paul: > > On 11 Mar, calato@riverstonenet.com wrote: > > > >> - Nevil, 4 Mar: Note setting out IESG / WH chair position on > >> this issue. In brief, if IPFIX doesn't mandate congestion- > >> aware transport, the WG RFCs won't get approved. > > > > I believe there are a large number of real world deployment > > cases where IPFIX is not going over the Internet. In fact, > > it is likely to be the majority of deployment cases. Is there > > disagreement on this point? > > > > If not, we ought to decide the best way to solve the problem > > first, then decide what that means for us as an IETF group; > > not the other way around. > > As an IETF WG we're honour-bound to come up with an architecture/ > protocol/whatever that's good for the Internet overall, that's why > we have to use congestion-aware transport. > 'Solving the problem,' > in a non-Internet setting, is not You are missing my point. If the problem space is 95% in the non-internet setting and we solve it for the internet setting, we solve the wrong problem. What's the point of that? We should not ignore our most prevalent deployment scenario because it doesn't fit nicely within IETF borders. We should say this is how to "best" solve our problem space and then decide what to do next. It would certainly be good if we can come up with something that works in both settings. I don't see why we can't accomplish that. > > ----------------------------------------------------------------------- > Nevil Brownlee Director, Technology Development > Phone: +64 9 373 7599 x8941 ITSS, The University of Auckland > FAX: +64 9 373 7021 Private Bag 92019, Auckland, New Zealand > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon Mar 11 14:16:58 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA19797 for ; Mon, 11 Mar 2002 14:16:57 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16kV1O-00039T-00 for ipfix-list@mil.doit.wisc.edu; Mon, 11 Mar 2002 12:59:46 -0600 Received: from host3.riverstonenet.com ([63.113.148.3] helo=exc-sc1.yagosys.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16kV1L-00038p-00 for ipfix@net.doit.wisc.edu; Mon, 11 Mar 2002 12:59:44 -0600 Received: from riverstonenet.com (134.141.180.109 [134.141.180.109]) by exc-sc1.yagosys.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13) id 1PZZFHJG; Mon, 11 Mar 2002 10:58:17 -0800 Message-ID: <3C8CFE48.B26EF1DF@riverstonenet.com> Date: Mon, 11 Mar 2002 13:58:17 -0500 From: calato@riverstonenet.com X-Mailer: Mozilla 4.75C-CCK-MCD NSCPCD475 [en] (X11; U; SunOS 5.6 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: kevin.zhang@xacct.com CC: ipfix@net.doit.wisc.edu, n.brownlee@auckland.ac.nz Subject: Re: [ipfix] Mailing list digest, Friday 8 March References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Kevin Zhang wrote: > > Hi Paul, > > I have to disagree with you, please see my comments. > > Thanks, > > Kevin > > > -----Original Message----- > > From: majordomo listserver [mailto:majordomo@mil.doit.wisc.edu]On Behalf > > Of calato@riverstonenet.com > > Sent: Monday, March 11, 2002 12:12 PM > > To: n.brownlee@auckland.ac.nz > > Cc: ipfix@net.doit.wisc.edu > > Subject: Re: [ipfix] Mailing list digest, Friday 8 March > > > > > > n.brownlee@auckland.ac.nz wrote: > > > > > > IPFIX Co-chairs list of IPFIX issues > > > > > > Nevil Brownlee & Dave Plonka, Fri 8 Mar 02 > > > > > > ++ marks each issue > > > ** WG co-chairs comment > > > - Mailing list discussion items > > > > > > Contributors names are listed in full following the list of issues > > > > > > ++ Congestion-aware protocols > > > ** Need to reach consensus on two major points: > > > a. We know we need reliable transport. Do we > > > also need unreliable? > > > b. Given consesnus on (a), how do we provide > > > congestion awareness? > > > - Nevil, 4 Mar: Note setting out IESG / WH chair position on > > > this issue. In brief, if IPFIX doesn't mandate congestion- > > > aware transport, the WG RFCs won't get approved. > > > > > > I believe there are a large number of real world deployment > > cases where IPFIX is not going over the Internet. In fact, > > it is likely to be the majority of deployment cases. Is there > > disagreement on this point? > > > > If not, we ought to decide the best way to solve the problem > > first, then decide what that means for us as an IETF group; > > not the other way around. > > > > > I believe CRANE protocol (http://search.ietf.org/internet-drafts/draft-kzhang-crane-protocol-02.txt) is considered as a candidate for IPFIX. We have several deployments with tier one carriers, which the exporter and collectors communicate through WAN. Furthermore, potential IPFIX exporting devices may be routers, probes, and mid-boxes as outlined in our charter; I do not agree that the need of high-speed router (may have limited on-device flow processing capability) should over-ride all the other needs. I agree 100%. And the opposite is also true, why would we ignore intra-net deployments and only consider Inter-net deployments? Especially when Intra-net is more prevalent (I didn't say exclusive, just more prevalent). BTW - I don't think WAN == Internet. > > > > > > > > > ++ Collector Failover > > > ** We need a control session handling keepalives. > > > Much of this discussion hinges on the TCP vs UDP debate! > > > - Paul, 27 Feb: Need to handle 'lost data' no matter what! > > > - Mark, 25 Feb: UDP with retransmits as an alternative to TCP > > > or SCTP. A TCP for keepalives could be enough? > > > - Paul/Ganesh, 25 Feb: Not convinced that UDP can be ruled out; > > > TCP on its own doesn't provide 'application level reliability.' > > > - Peter, 25 Feb: 'Reliable' transport is better and simpler. "Any > > > data delivery method that avoids reliability simply pushes > > things on > > > to another part of the system." Use SCTP or TCP; data ACKs > > > are enough, don;t need keepalives as well. > > > - Mark, 23 Feb: Suggestions: use mandatory TCP control connection, > > > require collector to send periodic ACKS to exporter. > > > - Peter, 23 Feb: TCP keepalive not enough for exporter to detect > > > crash of collector. Need to either use SCTP or have a failure > > > detection mechanism. > > > > > > ------------------------------------------------------------- > > > > > > ++ Counter wrapping, long-running flows > > > ** Support for both absolute counts (counters) and deltas, no clear > > > consensus. > > > - Benoit, 22 Feb: Close flow and export it when it nears > > counter wrap, > > > restart as a new flow. > > > - Ganesh, 21 Feb: Export flow records when counters go past > > a threshold, > > > Keep IEs for absolute counts or delta counts, whichever is needed. > > > - Peter, 21 Feb: In our experience, it's better to output > > deltas. Absolute > > > values are needed only if you have an unreliable data exporter. > > > - KC, 21 Feb: Ability to send a flow record at any interval due to a > > > counter rollover needs to be a selection criteria. > > > - Reinaldo, 19 Feb: Could someone elaborate on this? > > > > > > ++ Configuration: protocol > > > ** Out of scope, i.e. WG should not consider this until we have > > > the Arch and Data drafts well under way. > > > - KC, 20 Feb: not doing that, trying to select best of > > existing protocols > > > - Randy, 20 Feb: reminder, out of scope > > > - Reinaldo, 20 Feb, IPFIX config protocol needed, MIB OK as well > > > - Fred, 20 Feb: NetFlow v9: good summary posting > > > 'bi-directional config' = collector can ask exporter for > > reqd flows > > > Fred doesn't think it would be useful, lots of IOS development, > > > implications for reliability, security, etc. > > > Config via an 'IPFIX Configuration MIB' would be better. > > > - Juergen/Reinaldo/Paul/Benoit/Ram, 14 Feb: 'Not trying to > > standardise > > > meter, therefore don't specify how to configure one.' > > > - Mike, 13 Feb: Configuration and MIBs: "A MIB module is a > > concise guide > > > to implementors in their design of a configuration > > interface (CLI, XML, > > > XYZ, whatever) to manage a given techology" > > > - Mark, 12 Feb: Comments on exporter-collector session management, > > > session IDs. > > > - Will, 11 Feb: Doesn't want 'SHOULD support in-band configuration.' > > > > > > ++ Configuration: Content Negotiation > > > - Christian/Peter/Juergen, ~19 Feb: Collector should be able > > to request > > > a subset of all available flow attributes, exporter should indicate > > > which attributes it can send. > > > - Benoit, 19 Feb: This is pure configuration. You tell the exporter > > > which fields to export by defining a template. > > > > > > ++ Configuration: In-band Configuration > > > - Christian/Peter/Juergen: This is the ability for a collector to > > > ask an exporter for a particular set of flows > > > > > > ++ Applicability document > > > ** Material currently in Arch draft. Discuss at Minneapolis meeting. > > > Charter mentions it to support Architecture draft, so its > > > OK to start work on this, but don't let it distract from other > > > WG drafts! Use ipfix-app list alias for discussion. > > > - Carter, 20 Feb: OK, but don't go too far off topic > > > - KC, 20 Feb: Renaldo, Kevin & Tanja contributed most of > > this material > > > in Arch, they'd be good editors > > > - Tanja, Feb 19: 'Applicability' contents proposal > > > - Proposed by KC, support from Benoit > > > > > > ++ Middleboxes document > > > *** Not in scope, material currently in Arch draft. > > > Do we need a separate document? Or put it in > > Applicability draft? > > > > > > ++ Selecting a Flow Info Export protocol > > > ** Our charter says we'll select a protocol, however we'll probably > > > need to look (small) improvements to an existing protocol > > > - 19 Feb: KC had intended to list 'overview' descriptions of existing > > > protocols in Arch draft. Paul thinks this is premature, > > we should list > > > 'selection criteria' there instead. KC agrees, meantime (so as not > > > to loose the contributed text from B/G and Kevin) we'll > > shift it into > > > appendices. > > > - Randy, 18 Feb: realistically, i think you will have to > > modify any of > > > the existing protocols to meet robust needs. and yes, i think > > > security should be a consideration. [ note that snmpv3 control and > > > tcp export over ipsec may meet some of the more difficult > > issues you > > > raise ] > > > - David, 18 Feb: Selection criteria are needed for this! > > > > > > ++ Defining a flow > > > - Paul, 19 Feb: Selection process chooses packets to consider, > > > flow definition rules determine which unique flows to export, > > > i.e. flow definition rules may define sub-flows. Collector may > > > wish to know the selection cfriteria. > > > - Reinaldo, 19 Feb: Is a subset of a flow (i.e. one formed by adding > > > another filter function) not 'just another' flow? > > > > > > ++ Overload conditions > > > - KC/Robert, 13 Feb: Should exporter or collector be allowed to hanle > > > overload by changing sampling rate? Or is it better to loose > > > packets instead? > > > > > > ++ Security Considerations > > > - Cliff, 12 Feb: text for Arch draft > > > > > > B/G = Benoit & Ganesh > > > > > > Benoit Calise, Cisco > > > Carter Bullard, QOSient > > > Christian Gilby, Nortel > > > Cliff Wang, SmartPipes > > > David Harrington, Enterasys > > > Fred True, AT&T Research > > > Ganesh Sadavisan, Cisco > > > Jean-christophe Martin, Sun > > > Juergen Quittek, NEC > > > KC Norseth, Enterasys > > > Kevin Zhang, Xacct > > > Man M Li, Nokia > > > Mark Fullmer, OARnet > > > Paul Calato, Riverstone > > > Peter Ludemann, Xacct > > > Ram Gopal, Nokia > > > Randy Bush (AD), AT&T > > > Reinaldo Penno, Nortel > > > Sebastian Zander, Fraunhofer Institute FOKUS > > > Simon Leinen, Switch > > > Tanja Zseby, Fraunhofer Institute FOKUS > > > Venkatraman G, Infosys Technologies > > > Will Eatherton, Cisco > > > > > > ++Other comments > > > > > > Robert Lowe , 15 Feb: > > > The lurker opinion poll (no vote)... ;-) > > > In-band configuration -- not now. > > > Negotiation -- limited (but that discussion is promised to follow). > > > Benoit, I think this was a very perceptive comment. It seems we are > > > at times confusing the two. > > > > > > Peter Phaal , 13 Feb: > > > The IPFIX charter doesn't address the algorithms used by the flow > > > collector, but there are system-wide issues that should be explored. > > > Design choices made in the IPFIX protocol specification can have a > > > fundamental effect on the capabilities of traffic management systems > > > collecting IPFIX data (scalability, latency, accuracy, multi-point > > > correlation, etc.) > > > > > > ----------------------------------------------------------------------- > > > Nevil Brownlee Director, Technology Development > > > Phone: +64 9 373 7599 x8941 ITSS, The University of Auckland > > > FAX: +64 9 373 7021 Private Bag 92019, Auckland, New Zealand > > > > > > -- > > > Help mailto:majordomo@net.doit.wisc.edu and say "help" > > in message body > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > "unsubscribe ipfix" in message body > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > -- > > Help mailto:majordomo@net.doit.wisc.edu and say "help" in > > message body > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe ipfix" in message body > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > > > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon Mar 11 14:43:29 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA20480 for ; Mon, 11 Mar 2002 14:43:29 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16kVOI-0003g5-00 for ipfix-list@mil.doit.wisc.edu; Mon, 11 Mar 2002 13:23:26 -0600 Received: from ctron-dnm.enterasys.com ([12.25.1.120] helo=ctron-dnm.ctron.com ident=firewall-user) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16kVOG-0003fw-00 for ipfix@net.doit.wisc.edu; Mon, 11 Mar 2002 13:23:24 -0600 Received: (from uucp@localhost) by ctron-dnm.ctron.com (8.8.7/8.8.7) id OAA24354; Mon, 11 Mar 2002 14:33:06 -0500 (EST) Received: from unknown(134.141.77.96) by ctron-dnm.ctron.com via smap (4.1) id xma024281; Mon, 11 Mar 02 14:32:11 -0500 Received: by cnc-exc1.enterasys.com with Internet Mail Service (5.5.2653.19) id ; Mon, 11 Mar 2002 14:20:57 -0500 Message-ID: <59358A738F45D51186A30008C74CE250DA07AC@slc-exc1.ctron.com> From: "Norseth, KC" To: "'Reinaldo Penno'" Cc: ipfix@net.doit.wisc.edu Subject: RE: [ipfix] Mailing list digest, Friday 8 March Date: Mon, 11 Mar 2002 14:21:46 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C1C931.FC076C90" Precedence: bulk Sender: majordomo listserver This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C1C931.FC076C90 Content-Type: text/plain Hi Reinaldo, I was mostly unavailable for the last week, and am now starting to read the mail. For issues we want to discuss as part of the archecture, lets put them in. That is being worked on now. Ganesh and I will be putting together the arch presentation this week, but for issues that are not yet fully defined, we have a place to put them. There are several issues we need to clarify and they need to be done this meeting. -----Original Message----- From: Reinaldo Penno [mailto:reinaldo_penno@nortelnetworks.com] Sent: Saturday, March 09, 2002 5:34 PM To: n.brownlee@auckland.ac.nz; ipfix@net.doit.wisc.edu Subject: RE: [ipfix] Mailing list digest, Friday 8 March Hello Nevil/KC/All, Some comments on this digest... First of all I suggest we put together a small team of volunteers to come up with a presentation (doesn't even need to be a document) explaning what is in-band configuration, capabilities negotiation, content negotiation and other so we can start discussing based on a well understood terminology set. Even with the middlebox material out of the arch draft (today is in the applicability doc) we need to put something there to suggest how to behave in face of these services. I suggest something like: Even if we don't have a "team" to do this, if we can come up with a couple of slides on these items for discussion, we can put it in the arch presentation. I want to see closure at this conference. "An exporter may be involved in providing services that require application level intelligence and/or transform or filter content such as middlebox and OPES respectively. When involved in services that transform or filter content such as (but not limited to) NAT, Request Routing and Policing the exporter MUST be able to indicate on the flow record what changes were made to the packet." Agreed. other comments inline... >-----Original Message----- >From: n.brownlee@auckland.ac.nz [mailto:n.brownlee@auckland.ac.nz ] >Sent: Friday, March 08, 2002 7:56 PM >To: ipfix@net.doit.wisc.edu >Subject: [ipfix] Mailing list digest, Friday 8 March > > >IPFIX Co-chairs list of IPFIX issues > >Nevil Brownlee & Dave Plonka, Fri 8 Mar 02 > >++ marks each issue >** WG co-chairs comment > - Mailing list discussion items > > Contributors names are listed in full following the list of issues > >++ Congestion-aware protocols > ** Need to reach consensus on two major points: > a. We know we need reliable transport. Do we > also need unreliable? > b. Given consesnus on (a), how do we provide > congestion awareness? My vote is for a reliable protocol. Congestion awareness is already built-in on TCP ands SCTP. A Completly unreliable protocol is dangerous. I don't want to explain to my customers that they cannot rely on getting all their flow information from the router, UDP can also be made more reliable with some handshaking. > - Nevil, 4 Mar: Note setting out IESG / WH chair position on > this issue. In brief, if IPFIX doesn't mandate congestion- > aware transport, the WG RFCs won't get approved. > >++ Collector Failover > ** We need a control session handling keepalives. > Much of this discussion hinges on the TCP vs UDP debate! > - Paul, 27 Feb: Need to handle 'lost data' no matter what! > - Mark, 25 Feb: UDP with retransmits as an alternative to TCP > or SCTP. A TCP for keepalives could be enough? > - Paul/Ganesh, 25 Feb: Not convinced that UDP can be ruled out; > TCP on its own doesn't provide 'application level reliability.' > - Peter, 25 Feb: 'Reliable' transport is better and simpler. "Any > data delivery method that avoids reliability simply >pushes things on > to another part of the system." Use SCTP or TCP; data ACKs > are enough, don;t need keepalives as well. > - Mark, 23 Feb: Suggestions: use mandatory TCP control connection, > require collector to send periodic ACKS to exporter. > - Peter, 23 Feb: TCP keepalive not enough for exporter to detect > crash of collector. Need to either use SCTP or have a failure > detection mechanism. I'm not sure what is my position in the collector failover debate. I would tend to vote in favor of SCTP instead of assuming certain things that TCP does not provide, such as keepalives, periodic acks, etc. Otherwise it would be better to put it on the application layer and have a special health check mechanism that does not only check if Layers 1-4 are working but also the application itself, just like L7 switches today. > > >------------------------------------------------------------- > I tought we reached a consensus on this. It was to support absolute counts (counters) and deltas. >++ Counter wrapping, long-running flows > ** Support for both absolute counts (counters) and deltas, no clear > consensus. > - Benoit, 22 Feb: Close flow and export it when it nears >counter wrap, > restart as a new flow. > - Ganesh, 21 Feb: Export flow records when counters go past >a threshold, > Keep IEs for absolute counts or delta counts, whichever is needed. > - Peter, 21 Feb: In our experience, it's better to output >deltas. Absolute > values are needed only if you have an unreliable data exporter. > - KC, 21 Feb: Ability to send a flow record at any interval >due to a > counter rollover needs to be a selection criteria. > - Reinaldo, 19 Feb: Could someone elaborate on this? > > This configuration/negotiation part goes back to my original suggestion. >++ Configuration: protocol > ** Out of scope, i.e. WG should not consider this until we have > the Arch and Data drafts well under way. > - KC, 20 Feb: not doing that, trying to select best of >existing protocols > - Randy, 20 Feb: reminder, out of scope > - Reinaldo, 20 Feb, IPFIX config protocol needed, MIB OK as well > - Fred, 20 Feb: NetFlow v9: good summary posting > 'bi-directional config' = collector can ask exporter >for reqd flows > Fred doesn't think it would be useful, lots of IOS development, > implications for reliability, security, etc. > Config via an 'IPFIX Configuration MIB' would be better. > - Juergen/Reinaldo/Paul/Benoit/Ram, 14 Feb: 'Not trying to >standardise > meter, therefore don't specify how to configure one.' > - Mike, 13 Feb: Configuration and MIBs: "A MIB module is a >concise guide > to implementors in their design of a configuration >interface (CLI, XML, > XYZ, whatever) to manage a given techology" > - Mark, 12 Feb: Comments on exporter-collector session management, > session IDs. > - Will, 11 Feb: Doesn't want 'SHOULD support in-band configuration.' > >++ Configuration: Content Negotiation > - Christian/Peter/Juergen, ~19 Feb: Collector should be >able to request > a subset of all available flow attributes, exporter >should indicate > which attributes it can send. > - Benoit, 19 Feb: This is pure configuration. You tell the exporter > which fields to export by defining a template. > >++ Configuration: In-band Configuration > - Christian/Peter/Juergen: This is the ability for a collector to > ask an exporter for a particular set of flows > > ------_=_NextPart_001_01C1C931.FC076C90 Content-Type: text/html Message
Hi Reinaldo,
 
I was mostly unavailable for the last week, and am now starting to read the mail. 
 
For issues we want to discuss as part of the archecture, lets put them in. That is being worked on now.  Ganesh and I will be putting together the arch presentation this week, but for issues that are not yet fully defined, we have a place to put them.  There are several issues we need to clarify and they need to be done this meeting.
 
 -----Original Message-----
From: Reinaldo Penno [mailto:reinaldo_penno@nortelnetworks.com]
Sent: Saturday, March 09, 2002 5:34 PM
To: n.brownlee@auckland.ac.nz; ipfix@net.doit.wisc.edu
Subject: RE: [ipfix] Mailing list digest, Friday 8 March

Hello Nevil/KC/All,

Some comments on this digest...

First of all I suggest we put together a small team of volunteers to come up with a presentation (doesn't even need to be a document) explaning what is in-band configuration, capabilities negotiation, content negotiation and other so we can start discussing based on a well understood terminology set.

Even with the middlebox material out of the arch draft (today is in the applicability doc) we need to put something there to suggest how to behave in face of these services. I suggest something like: 

Even if we don't have a "team" to do this, if we can come up with a couple of slides on these items for discussion, we can put it in the arch presentation.  I want to see closure at this conference.

"An exporter may be involved in providing services that require
application level intelligence and/or transform or filter content such
as middlebox and OPES respectively.

When involved in services that transform or filter content such as (but not limited to)
NAT, Request Routing and Policing the exporter MUST be able to indicate on the flow
record what changes were made to the packet."  

Agreed. 

other comments inline...

>-----Original Message-----
>From: n.brownlee@auckland.ac.nz [mailto:n.brownlee@auckland.ac.nz]
>Sent: Friday, March 08, 2002 7:56 PM
>To: ipfix@net.doit.wisc.edu
>Subject: [ipfix] Mailing list digest, Friday 8 March
>
>
>IPFIX Co-chairs list of IPFIX issues
>
>Nevil Brownlee & Dave Plonka, Fri 8 Mar 02
>
>++ marks each issue
>** WG co-chairs comment
> - Mailing list discussion items
>
>   Contributors names are listed in full following the list of issues
>
>++ Congestion-aware protocols
>   ** Need to reach consensus on two major points:
>       a. We know we need reliable transport.  Do we
>          also need unreliable?
>       b. Given consesnus on (a), how do we provide
>          congestion awareness?

My vote is for a reliable protocol. Congestion awareness is already built-in on TCP ands SCTP.  

A Completly unreliable protocol is dangerous. I don't want to explain to my customers that they cannot rely on getting all their flow information from the router,  UDP can also be made more reliable with some handshaking. 

>   - Nevil, 4 Mar: Note setting out IESG / WH chair position on
>     this issue.  In brief, if IPFIX doesn't mandate congestion-
>     aware transport, the WG RFCs won't get approved.
>
>++ Collector Failover
>   ** We need a control session handling keepalives.
>      Much of this discussion hinges on the TCP vs UDP debate!
>   - Paul, 27 Feb: Need to handle 'lost data' no matter what!
>   - Mark, 25 Feb: UDP with retransmits as an alternative to TCP
>     or SCTP.  A TCP for keepalives could be enough?
>   - Paul/Ganesh, 25 Feb: Not convinced that UDP can be ruled out;
>     TCP on its own doesn't provide 'application level reliability.'
>   - Peter, 25 Feb: 'Reliable' transport is better and simpler. "Any
>     data delivery method that avoids reliability simply
>pushes things on
>     to another part of the system."  Use SCTP or TCP; data ACKs
>     are enough, don;t need keepalives as well.
>   - Mark, 23 Feb: Suggestions: use mandatory TCP control connection,
>     require collector to send periodic ACKS to exporter.
>   - Peter, 23 Feb: TCP keepalive not enough for exporter to detect
>     crash of collector.  Need to either use SCTP or have a failure
>     detection mechanism.

I'm not sure what is my position in the collector failover debate. I would tend to vote in favor of SCTP instead of assuming certain things that TCP does not provide, such as keepalives, periodic acks, etc.

Otherwise it would be better to put it on the application layer and have a special health check mechanism that does not only check if Layers 1-4 are working but also the application itself, just like L7 switches today.

>
>
>-------------------------------------------------------------
>

I tought we reached a consensus on this. It was to support absolute counts (counters) and deltas.

>++ Counter wrapping, long-running flows
>   ** Support for both absolute counts (counters) and deltas, no clear
>      consensus.
>   - Benoit, 22 Feb: Close flow and export it when it nears
>counter wrap,
>     restart as a new flow.
>   - Ganesh, 21 Feb: Export flow records when counters go past
>a threshold,
>     Keep IEs for absolute counts or delta counts, whichever is needed.
>   - Peter, 21 Feb: In our experience, it's better to output
>deltas. Absolute
>     values are needed only if you have an unreliable data exporter.
>   - KC, 21 Feb: Ability to send a flow record at any interval
>due to a
>     counter rollover needs to be a selection criteria.
>   - Reinaldo, 19 Feb: Could someone elaborate on this?
>
>

This configuration/negotiation part goes back to my original suggestion.

>++ Configuration: protocol
>   ** Out of scope, i.e. WG should not consider this until we have
>      the Arch and Data drafts well under way.
>   - KC, 20 Feb: not doing that, trying to select best of
>existing protocols
>   - Randy, 20 Feb: reminder, out of scope
>   - Reinaldo, 20 Feb, IPFIX config protocol needed, MIB OK as well
>   - Fred, 20 Feb: NetFlow v9: good summary posting
>       'bi-directional config' = collector can ask exporter
>for reqd flows
>       Fred doesn't think it would be useful, lots of IOS development,
>       implications for reliability, security, etc.
>       Config via an 'IPFIX Configuration MIB' would be better.
>   - Juergen/Reinaldo/Paul/Benoit/Ram, 14 Feb: 'Not trying to
>standardise
>     meter, therefore don't specify how to configure one.'
>   - Mike, 13 Feb: Configuration and MIBs: "A MIB module is a
>concise guide
>     to implementors in their design of a configuration
>interface (CLI, XML,
>     XYZ, whatever) to manage a given techology"
>   - Mark, 12 Feb: Comments on exporter-collector session management,
>     session IDs.
>   - Will, 11 Feb: Doesn't want 'SHOULD support in-band configuration.'
>
>++ Configuration: Content Negotiation
>   - Christian/Peter/Juergen, ~19 Feb: Collector should be
>able to request
>     a subset of all available flow attributes, exporter
>should indicate
>     which  attributes it can send.
>   - Benoit, 19 Feb: This is pure configuration.  You tell the exporter
>     which fields to export by defining a template.
>
>++ Configuration: In-band Configuration
>   - Christian/Peter/Juergen: This is the ability for a collector to
>     ask an exporter for a particular set of flows
>
>

------_=_NextPart_001_01C1C931.FC076C90-- -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon Mar 11 19:31:49 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA00758 for ; Mon, 11 Mar 2002 19:31:48 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16kZuh-00029o-00 for ipfix-list@mil.doit.wisc.edu; Mon, 11 Mar 2002 18:13:11 -0600 Received: from mailhub.xacct.com ([204.253.100.25]) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 16kZuf-00028o-00 for ipfix@net.doit.wisc.edu; Mon, 11 Mar 2002 18:13:10 -0600 Received: (qmail 32228 invoked from network); 12 Mar 2002 00:12:36 -0000 Received: from unknown (HELO usmail.xacct.com) (204.253.100.12) by mailhub.us.xacct.com with SMTP; 12 Mar 2002 00:12:36 -0000 Received: from peter (cpe-66-87-83-93.ca.sprintbbd.net [66.87.83.93]) by usmail.xacct.com (8.11.2/8.11.2) with SMTP id g2C0Ccm10758; Mon, 11 Mar 2002 16:12:39 -0800 From: "Peter Ludemann" To: , Cc: Subject: RE: [ipfix] Mailing list digest, Friday 8 March Date: Mon, 11 Mar 2002 16:12:36 -0800 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Importance: Normal In-Reply-To: <3C8CFC95.9F845DC7@riverstonenet.com> Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Based on my experience, congestion-awareness is useful even in a LAN. What happens if the receiver can't keep up with the sender for a short period? You don't want to be flooding the receiver with packets that it can't handle. TCP's capabilities turned out to be quite adequate for the congestion handling, combined with a record-level ACK for time-out and fail-over to another receiver. As I mentioned earlier, in such situations TCP's overheads are barely more than UDP's; but TCP avoids having to reinvent a congestion-awareness wheel. And if you don't use a congestion-aware transport, you'll just have to put it in yourself (based on my experience with testing in the field). It is an educating experience to deploy a system inside a Tier 1 backbone network ... packets often take longer/stranger paths than one would expect; routing tables getting screwed up and connectivity lost; OC3 pipes connected by a T1; etc. (Or are you discussing something other than the need for congestion-awareness?) - peter ----- Peter Ludemann +1.408.330.5732 Chief Architect +1.650.248.3973 (mobile) XACCT Technologies peter.ludemann@xacct.com 2900 Lakeside Drive http://www.xacct.com Santa Clara CA 95054 > -----Original Message----- > From: majordomo listserver [mailto:majordomo@mil.doit.wisc.edu]On Behalf > Of calato@riverstonenet.com > Sent: Monday, March 11, 2002 10:51 AM > To: n.brownlee@auckland.ac.nz > Cc: ipfix@net.doit.wisc.edu > Subject: Re: [ipfix] Mailing list digest, Friday 8 March > > > n.brownlee@auckland.ac.nz wrote: > > > > Hi Paul: > > > > On 11 Mar, calato@riverstonenet.com wrote: > > > > > >> - Nevil, 4 Mar: Note setting out IESG / WH chair position on > > >> this issue. In brief, if IPFIX doesn't mandate congestion- > > >> aware transport, the WG RFCs won't get approved. > > > > > > I believe there are a large number of real world deployment > > > cases where IPFIX is not going over the Internet. In fact, > > > it is likely to be the majority of deployment cases. Is there > > > disagreement on this point? > > > > > > If not, we ought to decide the best way to solve the problem > > > first, then decide what that means for us as an IETF group; > > > not the other way around. > > > > As an IETF WG we're honour-bound to come up with an architecture/ > > protocol/whatever that's good for the Internet overall, that's why > > we have to use congestion-aware transport. > > 'Solving the problem,' > > in a non-Internet setting, is not > > You are missing my point. If the problem space is 95% > in the non-internet setting and we solve it for the > internet setting, we solve the wrong problem. What's > the point of that? > > We should not ignore our most prevalent deployment > scenario because it doesn't fit nicely within IETF > borders. We should say this is how to "best" solve > our problem space and then decide what to do next. > > It would certainly be good if we can come up with > something that works in both settings. I don't > see why we can't accomplish that. > > > > > ----------------------------------------------------------------------- > > Nevil Brownlee Director, Technology Development > > Phone: +64 9 373 7599 x8941 ITSS, The University of Auckland > > FAX: +64 9 373 7021 Private Bag 92019, Auckland, New Zealand > > > > -- > > Help mailto:majordomo@net.doit.wisc.edu and say "help" > in message body > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe ipfix" in message body > > Archive http://ipfix.doit.wisc.edu/archive/ > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in > message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon Mar 11 20:07:41 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA01270 for ; Mon, 11 Mar 2002 20:07:41 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16kaTy-0002qB-00 for ipfix-list@mil.doit.wisc.edu; Mon, 11 Mar 2002 18:49:38 -0600 Received: from host3.riverstonenet.com ([63.113.148.3] helo=exc-sc1.yagosys.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16kaTv-0002pa-00 for ipfix@net.doit.wisc.edu; Mon, 11 Mar 2002 18:49:36 -0600 Received: from riverstonenet.com (134.141.180.109 [134.141.180.109]) by exc-sc1.yagosys.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13) id 1PZZFMQV; Mon, 11 Mar 2002 16:48:12 -0800 Message-ID: <3C8D5049.54C1C864@riverstonenet.com> Date: Mon, 11 Mar 2002 19:48:09 -0500 From: calato@riverstonenet.com X-Mailer: Mozilla 4.75C-CCK-MCD NSCPCD475 [en] (X11; U; SunOS 5.6 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: Peter Ludemann CC: n.brownlee@auckland.ac.nz, ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Mailing list digest, Friday 8 March References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit I have no problem with TCP. In fact, it is my preferred choice (LFAP also runs over TCP). I have an issue with basing the decision on design criteria that I believe are highly suspect. It will only lead to "wrong" choices down the road. If we say that UDP is out because it is not Inter-net friendly and most of the deployments are Intra-net, that logic seems flawed, even if it arrives at the "correct" answer. Just to push the other side, more comments inline... Peter Ludemann wrote: > > Based on my experience, congestion-awareness is useful even in a LAN. What > happens if the receiver can't keep up with the sender for a short period? > You don't want to be flooding the receiver with packets that it can't > handle. You lose some records. In a lot of applications, that is perfectly acceptable. Even with TCP, if the Collector backs up long enough the exporter will have to drop some information. In either case, IPFIX will need to define and code for a data loss situation. > > TCP's capabilities turned out to be quite adequate for the congestion > handling, combined with a record-level ACK for time-out and fail-over to > another receiver. Record level ACK's can be done with UDP. In a lot of cases there is no congestion issue. > > As I mentioned earlier, in such situations TCP's overheads are barely more > than UDP's; Maybe someone else can comment on the TCP vs UDP overhead. This seems to be the crux of the issue. If TCP and UDP are roughly the same in terms of overhead, speed, etc... then TCP is a clear winner. If they are not roughly equal, then maybe both will be needed. > but TCP avoids having to reinvent a wheel. > And if you don't use a congestion-aware transport, you'll just have to put > it in yourself (based on my experience with testing in the field). Not if you don't congestion-awareness. > > It is an educating experience to deploy a system inside a Tier 1 backbone > network ... packets often take longer/stranger paths than one would expect; > routing tables getting screwed up and connectivity lost; OC3 pipes connected > by a T1; etc. > > (Or are you discussing something other than the need for > congestion-awareness?) > > - peter > > ----- > Peter Ludemann +1.408.330.5732 > Chief Architect +1.650.248.3973 (mobile) > XACCT Technologies peter.ludemann@xacct.com > 2900 Lakeside Drive http://www.xacct.com > Santa Clara CA 95054 > > > -----Original Message----- > > From: majordomo listserver [mailto:majordomo@mil.doit.wisc.edu]On Behalf > > Of calato@riverstonenet.com > > Sent: Monday, March 11, 2002 10:51 AM > > To: n.brownlee@auckland.ac.nz > > Cc: ipfix@net.doit.wisc.edu > > Subject: Re: [ipfix] Mailing list digest, Friday 8 March > > > > > > n.brownlee@auckland.ac.nz wrote: > > > > > > Hi Paul: > > > > > > On 11 Mar, calato@riverstonenet.com wrote: > > > > > > > >> - Nevil, 4 Mar: Note setting out IESG / WH chair position on > > > >> this issue. In brief, if IPFIX doesn't mandate congestion- > > > >> aware transport, the WG RFCs won't get approved. > > > > > > > > I believe there are a large number of real world deployment > > > > cases where IPFIX is not going over the Internet. In fact, > > > > it is likely to be the majority of deployment cases. Is there > > > > disagreement on this point? > > > > > > > > If not, we ought to decide the best way to solve the problem > > > > first, then decide what that means for us as an IETF group; > > > > not the other way around. > > > > > > As an IETF WG we're honour-bound to come up with an architecture/ > > > protocol/whatever that's good for the Internet overall, that's why > > > we have to use congestion-aware transport. > > > 'Solving the problem,' > > > in a non-Internet setting, is not > > > > You are missing my point. If the problem space is 95% > > in the non-internet setting and we solve it for the > > internet setting, we solve the wrong problem. What's > > the point of that? > > > > We should not ignore our most prevalent deployment > > scenario because it doesn't fit nicely within IETF > > borders. We should say this is how to "best" solve > > our problem space and then decide what to do next. > > > > It would certainly be good if we can come up with > > something that works in both settings. I don't > > see why we can't accomplish that. > > > > > > > > ----------------------------------------------------------------------- > > > Nevil Brownlee Director, Technology Development > > > Phone: +64 9 373 7599 x8941 ITSS, The University of Auckland > > > FAX: +64 9 373 7021 Private Bag 92019, Auckland, New Zealand > > > > > > -- > > > Help mailto:majordomo@net.doit.wisc.edu and say "help" > > in message body > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > "unsubscribe ipfix" in message body > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > -- > > Help mailto:majordomo@net.doit.wisc.edu and say "help" in > > message body > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe ipfix" in message body > > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Mar 12 04:00:38 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id EAA18178 for ; Tue, 12 Mar 2002 04:00:38 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16khom-0004dm-00 for ipfix-list@mil.doit.wisc.edu; Tue, 12 Mar 2002 02:39:36 -0600 Received: from mailhub.xacct.com ([204.253.100.25]) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 16khok-0004d1-00 for ipfix@net.doit.wisc.edu; Tue, 12 Mar 2002 02:39:34 -0600 Received: (qmail 9702 invoked from network); 12 Mar 2002 08:39:00 -0000 Received: from unknown (HELO usmail.xacct.com) (204.253.100.12) by mailhub.us.xacct.com with SMTP; 12 Mar 2002 08:39:00 -0000 Received: from peter (cpe-66-87-83-93.ca.sprintbbd.net [66.87.83.93]) by usmail.xacct.com (8.11.2/8.11.2) with SMTP id g2C8d2m15194; Tue, 12 Mar 2002 00:39:03 -0800 From: "Peter Ludemann" To: Cc: , Subject: Congestion awareness, reliability (was: RE: [ipfix] Mailing list digest, Friday 8 March) Date: Tue, 12 Mar 2002 00:39:00 -0800 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) Importance: Normal In-Reply-To: <3C8D5049.54C1C864@riverstonenet.com> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit It seems that there's no firm consensus on reliable/non-reliable and whether we need congestion awareness. My belief is that reliability and congestion-awareness cost little and gain a lot; and that the "benefits" of UDP are marginal at best: - load-balancing by multi-cast -- but almost the same thing can be done by having multiple output streams with TCP; - silent loss of data with UDP when you don't care much about the data. (did I miss any?) Additional comments inline (I found a few references on efficiency of TCP vs UDP). > -----Original Message----- > From: calato [mailto:calato]On Behalf Of calato@riverstonenet.com > Sent: Monday, March 11, 2002 4:48 PM > To: Peter Ludemann > Cc: n.brownlee@auckland.ac.nz; ipfix@net.doit.wisc.edu > Subject: Re: [ipfix] Mailing list digest, Friday 8 March > > > > I have no problem with TCP. In fact, it is my preferred > choice (LFAP also runs over TCP). I have an issue with basing > the decision on design criteria that I believe are highly > suspect. It will only lead to "wrong" choices down the road. > > If we say that UDP is out because it is not Inter-net friendly > and most of the deployments are Intra-net, that logic seems > flawed, even if it arrives at the "correct" answer. > > Just to push the other side, more comments inline... > > Peter Ludemann wrote: > > > > Based on my experience, congestion-awareness is useful even in > a LAN. What > > happens if the receiver can't keep up with the sender for a > short period? > > You don't want to be flooding the receiver with packets that it can't > > handle. > > You lose some records. In a lot of applications, that > is perfectly acceptable. Even with TCP, if the Collector > backs up long enough the exporter will have to drop some > information. In either case, IPFIX will need to define and code > for a data loss situation. But with TCP (or some other "reliable" transport), you know which records you've lost and can compensate in some (e.g., aggregate the lost values and send them later). With UDP, you don't know, unless you have ACKs, in which case you're just reinventing a partial TCP without retransmission and congestion control (and you'd also need to handle duplicates and out-of-orders). > > > > TCP's capabilities turned out to be quite adequate for the congestion > > handling, combined with a record-level ACK for time-out and fail-over to > > another receiver. > > Record level ACK's can be done with UDP. In a lot of cases > there is no congestion issue. > > > > > As I mentioned earlier, in such situations TCP's overheads are > barely more > > than UDP's; > > > Maybe someone else can comment on the TCP vs UDP overhead. > This seems to be the crux of the issue. If TCP and UDP are > roughly the same in terms of overhead, speed, etc... then > TCP is a clear winner. If they are not roughly equal, then > maybe both will be needed. I don't have numbers available; but I do notice that NFS Version 3 is over TCP (RFC 1813, 3010). Presumably efficiency isn't a problem. As a thought experiment, on a local network without congestion, TCP won't be retransmitting or reordering packets; there'll be the ACKs coming back (possibly piggybacked on the record-level ACKs for the data protocol), so at the network level, performance should be about the same and only slight more CPU usage. Java is much more efficient with TCP than UDP, by the way. RFC 3010 references Macklem, Rick, "Lessons Learned Tuning the 4.3BSD Reno Implementation of the NFS Protocol", which I found at http://sunsite.nstu.nsk.su/sun/inform/whitepapers.html. Graph 6 shows that TCP is about 20% more expensive in CPU than UDP (and this paper is over 10 years old, so implementations have probably improved) -- it's not clear to me whether this number takes into account the acknowledgment work done outside the transport with UDP. My interpretation is that on a more powerful machine than a microVAX, the congestion control of TCP will more than offset CPU overhead and that TCP is a winner. Additional references ... http://www.ietf.org/proceedings/98dec/43rd-ietf-98dec-142.html Brent Callaghan discussed the NFS version 4 effort. He recalled that NFS was developed for UDP because TCP performance was pretty poor in the target environment, Ethernet LANs. NFS mount options were added for controlling timeouts and transfer sizes. The adaptive timeouts worked okay, but the transfer size adjustment didn't. As NFS started being used over WANs, it evolved towards also being run over TCP, certainly a win on congested WANs, and fitting better with NFSv3's potentially very big reads and writes. The NFSv4 Transport Wish List: As fast as UDP on LANs; optimized for fast RPC transactions (fast binding, reduced TIME-WAIT); integrated record marking. and this: http://www.frontiertech.com/supernfs/WHITE-P/client_opt.htm Since TCP is a stateful protocol, it transfers a large part of the responsibility for guaranteeing the completion of file requests from the client to the network, with a concomitant increase in reliability and performance. The efficient exploitation of this, however, requires careful client and server design, for which no standards have yet been devised. In addition, for similar reasons, NFS over TCP offers superior performance when several routers ("hops") lie between the client and server. TCP handles multiple "hopping", which is a characteristic of the Internet, much more efficiently. > > but TCP avoids having to reinvent a wheel. > > And if you don't use a congestion-aware transport, you'll just > have to put > > it in yourself (based on my experience with testing in the field). > > Not if you don't congestion-awareness. See the comment above on TCP, as to why congestion-awareness is a Good Thing on WANs. As to LANs, it's surprising how badly some sysadmins can be at configuring a network; not to mention the amusing effect of a hub or switch that develops a "glitch". > > > > It is an educating experience to deploy a system inside a Tier > 1 backbone > > network ... packets often take longer/stranger paths than one > would expect; > > routing tables getting screwed up and connectivity lost; OC3 > pipes connected > > by a T1; etc. > > > > (Or are you discussing something other than the need for > > congestion-awareness?) > > > > - peter > > > > ----- > > Peter Ludemann +1.408.330.5732 > > Chief Architect +1.650.248.3973 (mobile) > > XACCT Technologies peter.ludemann@xacct.com > > 2900 Lakeside Drive http://www.xacct.com > > Santa Clara CA 95054 > > > > > -----Original Message----- > > > From: majordomo listserver > [mailto:majordomo@mil.doit.wisc.edu]On Behalf > > > Of calato@riverstonenet.com > > > Sent: Monday, March 11, 2002 10:51 AM > > > To: n.brownlee@auckland.ac.nz > > > Cc: ipfix@net.doit.wisc.edu > > > Subject: Re: [ipfix] Mailing list digest, Friday 8 March > > > > > > > > > n.brownlee@auckland.ac.nz wrote: > > > > > > > > Hi Paul: > > > > > > > > On 11 Mar, calato@riverstonenet.com wrote: > > > > > > > > > >> - Nevil, 4 Mar: Note setting out IESG / WH chair position on > > > > >> this issue. In brief, if IPFIX doesn't mandate congestion- > > > > >> aware transport, the WG RFCs won't get approved. > > > > > > > > > > I believe there are a large number of real world deployment > > > > > cases where IPFIX is not going over the Internet. In fact, > > > > > it is likely to be the majority of deployment > cases. Is there > > > > > disagreement on this point? > > > > > > > > > > If not, we ought to decide the best way to solve the problem > > > > > first, then decide what that means for us as an IETF group; > > > > > not the other way around. > > > > > > > > As an IETF WG we're honour-bound to come up with an architecture/ > > > > protocol/whatever that's good for the Internet overall, that's why > > > > we have to use congestion-aware transport. > > > > 'Solving the problem,' > > > > in a non-Internet setting, is not > > > > > > You are missing my point. If the problem space is 95% > > > in the non-internet setting and we solve it for the > > > internet setting, we solve the wrong problem. What's > > > the point of that? > > > > > > We should not ignore our most prevalent deployment > > > scenario because it doesn't fit nicely within IETF > > > borders. We should say this is how to "best" solve > > > our problem space and then decide what to do next. > > > > > > It would certainly be good if we can come up with > > > something that works in both settings. I don't > > > see why we can't accomplish that. > > > > > > > > > > > > ----------------------------------------------------------------------- > > > > Nevil Brownlee Director, Technology Development > > > > Phone: +64 9 373 7599 x8941 ITSS, The University of Auckland > > > > FAX: +64 9 373 7021 Private Bag 92019, Auckland, New Zealand -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Mar 12 04:30:53 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id EAA18421 for ; Tue, 12 Mar 2002 04:30:53 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16kiM1-0005RO-00 for ipfix-list@mil.doit.wisc.edu; Tue, 12 Mar 2002 03:13:57 -0600 Received: from mail-green.research.att.com ([135.207.30.103]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16kiLz-0005R6-00 for ipfix@net.doit.wisc.edu; Tue, 12 Mar 2002 03:13:55 -0600 Received: from alliance.research.att.com (alliance.research.att.com [135.207.26.26]) by mail-green.research.att.com (Postfix) with ESMTP id ED1761E078; Tue, 12 Mar 2002 04:13:51 -0500 (EST) Received: from windsor.research.att.com (windsor.research.att.com [135.207.26.46]) by alliance.research.att.com (8.8.7/8.8.7) with ESMTP id EAA20658; Tue, 12 Mar 2002 04:13:50 -0500 (EST) Date: Tue, 12 Mar 2002 04:13:50 -0500 (EST) From: Fred True To: Peter Ludemann Cc: , , Subject: Re: Congestion awareness, reliability (was: RE: [ipfix] Mailing list digest, Friday 8 March) In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Precedence: bulk Sender: majordomo listserver > But with TCP (or some other "reliable" transport), you know which > records you've lost and can compensate in some (e.g., aggregate > the lost values and send them later). With UDP, you don't know, unless > you have ACKs, You know which records you've lost in UDP if the datagrams contain atomic serial numbers, as they do in the current Cisco Netflow export format. This is the way we currently detect and account for loss in our netflow applications (you know how many flow records you lose, but of course you do not know anything about what they contained; but you wouldn't with TCP either). -fred -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Mar 12 09:47:32 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA23643 for ; Tue, 12 Mar 2002 09:47:32 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16kn9k-0006AF-00 for ipfix-list@mil.doit.wisc.edu; Tue, 12 Mar 2002 08:21:37 -0600 Received: from bru-cse-222.cisco.com ([144.254.8.48]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16kn9i-00066c-00 for ipfix@net.doit.wisc.edu; Tue, 12 Mar 2002 08:21:34 -0600 Received: (from bclaise@localhost) by bru-cse-222.cisco.com (8.10.2+Sun/CA/950118) id g2CEFkC12365; Tue, 12 Mar 2002 15:15:46 +0100 (CET) Date: Tue, 12 Mar 2002 15:15:46 +0100 (CET) From: Benoit Claise Message-Id: <200203121415.g2CEFkC12365@bru-cse-222.cisco.com> To: n.brownlee@auckland.ac.nz, calato@riverstonenet.com Subject: Re: [ipfix] Mailing list digest, Friday 8 March Cc: ipfix@net.doit.wisc.edu X-Sun-Charset: US-ASCII Precedence: bulk Sender: majordomo listserver > > > > On 11 Mar, calato@riverstonenet.com wrote: > > > > > >> - Nevil, 4 Mar: Note setting out IESG / WH chair position on > > >> this issue. In brief, if IPFIX doesn't mandate congestion- > > >> aware transport, the WG RFCs won't get approved. > > > > > > I believe there are a large number of real world deployment > > > cases where IPFIX is not going over the Internet. In fact, > > > it is likely to be the majority of deployment cases. Is there > > > disagreement on this point? > > > > > > If not, we ought to decide the best way to solve the problem > > > first, then decide what that means for us as an IETF group; > > > not the other way around. > > > > As an IETF WG we're honour-bound to come up with an architecture/ > > protocol/whatever that's good for the Internet overall, that's why > > we have to use congestion-aware transport. > > 'Solving the problem,' > > in a non-Internet setting, is not > > You are missing my point. If the problem space is 95% > in the non-internet setting and we solve it for the > internet setting, we solve the wrong problem. What's > the point of that? > > We should not ignore our most prevalent deployment > scenario because it doesn't fit nicely within IETF > borders. We should say this is how to "best" solve > our problem space and then decide what to do next. Paul, I agree 100% with you! The direction the IPFIX WG is currently force to take is: - congestion aware. What for? for the 5 % customers which are going to export over the internet - reliable transport because we have almost no other option What for? reliability is mainly needed for billing purposes but not for traffic profiling, traffic engineering, ddos detection, user profiling, capacicity planning, etc.. so again for a subset of the applications using the IPFIX data So just to be "honorable", we're fixing the problem of a probe (no memory problem/not exporting a lot of data) exporting billing information to a remote end of the internet. And we're forcing everybody else to apply the same protocol or ... follow the do-what-you-want-in-your-intranet procedure. I'm just a little dissapointed... I thought that the IETF was based on consensus and running code. Regards, benoit. > > It would certainly be good if we can come up with > something that works in both settings. I don't > see why we can't accomplish that. > > > > > ----------------------------------------------------------------------- > > Nevil Brownlee Director, Technology Development > > Phone: +64 9 373 7599 x8941 ITSS, The University of Auckland > > FAX: +64 9 373 7021 Private Bag 92019, Auckland, New Zealand > > > > -- > > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe ipfix" in message body > > Archive http://ipfix.doit.wisc.edu/archive/ > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Mar 12 12:55:28 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA29125 for ; Tue, 12 Mar 2002 12:55:27 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16kqAf-0002a0-00 for ipfix-list@mil.doit.wisc.edu; Tue, 12 Mar 2002 11:34:45 -0600 Received: from mailhub.xacct.com ([204.253.100.25]) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 16kqAd-0002ZC-00 for ipfix@net.doit.wisc.edu; Tue, 12 Mar 2002 11:34:43 -0600 Received: (qmail 26102 invoked from network); 12 Mar 2002 17:34:07 -0000 Received: from unknown (HELO usmail.xacct.com) (204.253.100.12) by mailhub.us.xacct.com with SMTP; 12 Mar 2002 17:34:07 -0000 Received: from peter ([204.253.100.102]) by usmail.xacct.com (8.11.2/8.11.2) with SMTP id g2CHYDm19766; Tue, 12 Mar 2002 09:34:13 -0800 From: "Peter Ludemann" To: "Fred True" Cc: , , Subject: RE: Congestion awareness, reliability (was: RE: [ipfix] Mailing list digest, Friday 8 March) Date: Tue, 12 Mar 2002 09:34:10 -0800 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) Importance: Normal In-Reply-To: X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit > > But with TCP (or some other "reliable" transport), you know which > > records you've lost and can compensate in some (e.g., aggregate > > the lost values and send them later). With UDP, you don't know, unless > > you have ACKs, > > You know which records you've lost in UDP if the datagrams contain atomic > serial numbers, as they do in the current Cisco Netflow export format. > This is the way we currently detect and account for loss in our netflow > applications (you know how many flow records you lose, but of course you > do not know anything about what they contained; but you wouldn't with TCP > either). > > -fred But only the *sender* knows what was in the lost data (e.g., the total number of bytes that it saw); and UDP doesn't communicate that back to the sender. So, yes, the receiver can know how many records were lost but it has no idea of what data was lost. With TCP, you know which records you couldn't send, so you could calculate aggregates of the data that wasn't sent (and send these aggregates later, present them in a MIB, etc.) - peter > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in > message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Mar 12 13:01:23 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA29408 for ; Tue, 12 Mar 2002 13:01:22 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16kqEO-0002fC-00 for ipfix-list@mil.doit.wisc.edu; Tue, 12 Mar 2002 11:38:36 -0600 Received: from mailhub.xacct.com ([204.253.100.25]) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 16kqEL-0002eb-00 for ipfix@net.doit.wisc.edu; Tue, 12 Mar 2002 11:38:33 -0600 Received: (qmail 26291 invoked from network); 12 Mar 2002 17:37:56 -0000 Received: from unknown (HELO usmail.xacct.com) (204.253.100.12) by mailhub.us.xacct.com with SMTP; 12 Mar 2002 17:37:56 -0000 Received: from Kevinz (sdn-ar-007njpendP017.dialsprint.net [63.178.186.25]) by usmail.xacct.com (8.11.2/8.11.2) with SMTP id g2CHbdm19846; Tue, 12 Mar 2002 09:37:42 -0800 Reply-To: From: "Kevin Zhang" To: Cc: "Benoit Claise" , , Subject: RE: [ipfix] Mailing list digest, Friday 8 March Date: Tue, 12 Mar 2002 12:37:10 -0500 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) In-Reply-To: <200203121415.g2CEFkC12365@bru-cse-222.cisco.com> Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by ietf.org id NAA29408 > -----Original Message----- > From: majordomo listserver [mailto:majordomo@mil.doit.wisc.edu]On Behalf > Of Benoit Claise > Sent: Tuesday, March 12, 2002 9:16 AM > To: n.brownlee@auckland.ac.nz; calato@riverstonenet.com > Cc: ipfix@net.doit.wisc.edu > Subject: Re: [ipfix] Mailing list digest, Friday 8 March > > > > > > > > On 11 Mar, calato@riverstonenet.com wrote: > > > > > > > >> - Nevil, 4 Mar: Note setting out IESG / WH chair position on > > > >> this issue. In brief, if IPFIX doesn't mandate congestion- > > > >> aware transport, the WG RFCs won't get approved. > > > > > > > > I believe there are a large number of real world deployment > > > > cases where IPFIX is not going over the Internet. In fact, > > > > it is likely to be the majority of deployment cases. Is there > > > > disagreement on this point? > > > > > > > > If not, we ought to decide the best way to solve the problem > > > > first, then decide what that means for us as an IETF group; > > > > not the other way around. > > > > > > As an IETF WG we're honour-bound to come up with an architecture/ > > > protocol/whatever that's good for the Internet overall, that's why > > > we have to use congestion-aware transport. > > > 'Solving the problem,' > > > in a non-Internet setting, is not > > > > You are missing my point. If the problem space is 95% > > in the non-internet setting and we solve it for the > > internet setting, we solve the wrong problem. What's > > the point of that? > > > > We should not ignore our most prevalent deployment > > scenario because it doesn't fit nicely within IETF > > borders. We should say this is how to "best" solve > > our problem space and then decide what to do next. > > Paul, I agree 100% with you! > > The direction the IPFIX WG is currently force to take is: > - congestion aware. > What for? for the 5 % customers which are going to export over > the internet > - reliable transport because we have almost no other option > What for? reliability is mainly needed for billing purposes but > not for traffic profiling, traffic engineering, ddos detection, > user profiling, capacicity planning, etc.. > so again for a subset of the applications using the IPFIX data > > So just to be "honorable", we're fixing the problem of a probe > (no memory problem/not exporting a lot of data) exporting billing > information to a remote end of the internet. And we're forcing > everybody else to apply the same protocol or ... follow the > do-what-you-want-in-your-intranet procedure. > > I'm just a little dissapointed... I thought that the IETF was > based on consensus and running code. > > Regards, benoit. > I believe the core of the problem is whether a router is able and willing to buffer one rto+delta worth of flow records in order to allow for any reliability, regardless the transport is UDP or TCP. The design or limitation of a router should not dominate the direction of IPFIX. In terms of current deployment, from our experience, reliability is the most important requirement from carriers. The reason why exporting over WAN is relatively few is exactly due to the fact that there's no standard approach, which the IPFIX WG is addressing. The choices in front of us is whether to address real customer needs that makes IPFIX highly relevant and benefit the industry as a whole, or standardizing some vendor's existing solution that has a very limited audience. In terms of consensus, my observation is that there is at least as much support for a reliable transport as TCP. Currently, CRANE and LFAF are running over TCP (running code). Thanks, Kevin > > > > > > > > It would certainly be good if we can come up with > > something that works in both settings. I don't > > see why we can't accomplish that. > > > > > > > > > ----------------------------------------------------------------------- > > > Nevil Brownlee Director, Technology Development > > > Phone: +64 9 373 7599 x8941 ITSS, The University of Auckland > > > FAX: +64 9 373 7021 Private Bag 92019, Auckland, New Zealand > > > > > > -- > > > Help mailto:majordomo@net.doit.wisc.edu and say "help" > in message body > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > "unsubscribe ipfix" in message body > > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > -- > > Help mailto:majordomo@net.doit.wisc.edu and say "help" > in message body > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe ipfix" in message body > > Archive http://ipfix.doit.wisc.edu/archive/ > > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in > message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ > > ÿñÞ–™šŠ[hþf£¢·hšçzßÝ¢+ÿÂ+ýçnjwlk/ázZÿŠyž²Æ yºÉIì¹»®&Þ™¨¥¶æj:+v‰¨þw­ýÚ"·ü"±ÏÞvæ§vƲþéì¹»®&ÞŠ—âÇø§™ë,j›¡Ü€­Èb½èm¶Ÿÿþ*_‹Ý¢+ÿÂ+ýçnýªÜ†+Þ From majordomo@mil.doit.wisc.edu Tue Mar 12 13:10:06 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA29630 for ; Tue, 12 Mar 2002 13:10:06 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16kqUk-00033E-00 for ipfix-list@mil.doit.wisc.edu; Tue, 12 Mar 2002 11:55:30 -0600 Received: from h-135-207-30-103.research.att.com ([135.207.30.103] helo=mail-green.research.att.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16kqUh-000339-00 for ipfix@net.doit.wisc.edu; Tue, 12 Mar 2002 11:55:28 -0600 Received: from alliance.research.att.com (alliance.research.att.com [135.207.26.26]) by mail-green.research.att.com (Postfix) with ESMTP id CDDFF1E007; Tue, 12 Mar 2002 12:55:26 -0500 (EST) Received: from windsor.research.att.com (windsor.research.att.com [135.207.26.46]) by alliance.research.att.com (8.8.7/8.8.7) with ESMTP id MAA29197; Tue, 12 Mar 2002 12:55:26 -0500 (EST) Date: Tue, 12 Mar 2002 12:55:26 -0500 (EST) From: Fred True To: Peter Ludemann Cc: , , Subject: RE: Congestion awareness, reliability (was: RE: [ipfix] Mailing list digest, Friday 8 March) In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Precedence: bulk Sender: majordomo listserver > But only the *sender* knows what was in the lost data (e.g., the total > number of bytes that it saw); and UDP doesn't communicate that back to > the sender. So, yes, the receiver can know how many records were lost > but it has no idea of what data was lost. With TCP, you know which > records you couldn't send, so you could calculate aggregates of the > data that wasn't sent (and send these aggregates later, present them > in a MIB, etc.) True; however Cisco's netflow (roughly) does this too, although it is only available via CLI. It keeps the number of bytes, pkts, etc. exported via netflow in a router-side table, which can be re-set or queried via cli. We have used this to fill in for loss; however it is somewhat difficult to manage in tight sync with the exported data stream. Still, I think the use of TCP is not the only solution to this problem (and for the sender's tcp stack to have a hook into secondary aggregation sounds like a burden that not many vendors are going to want to shoulder). -fred -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Mar 12 14:30:56 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA03018 for ; Tue, 12 Mar 2002 14:30:56 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16krTb-0004LJ-00 for ipfix-list@mil.doit.wisc.edu; Tue, 12 Mar 2002 12:58:23 -0600 Received: from mail.riverstonenet.com ([63.113.148.10] helo=riverstonenet.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16krTZ-0004Ka-00 for ipfix@net.doit.wisc.edu; Tue, 12 Mar 2002 12:58:21 -0600 Received: from agile.yagosys.com by riverstonenet.com (8.9.3+Sun/SMI-SVR4-Yago) id KAA13366; Tue, 12 Mar 2002 10:57:48 -0800 (PST) Received: from agile.yagosys.com (agile.yagosys.com [127.0.0.1]) by agile.yagosys.com (8.12.0/8.12.0) with ESMTP id g2CIvl9t001428 for ; Tue, 12 Mar 2002 10:57:47 -0800 Received: (from mrm@localhost) by agile.yagosys.com (8.12.0/8.12.0/Submit) id g2CIvlrc001427 for ipfix@net.doit.wisc.edu; Tue, 12 Mar 2002 10:57:47 -0800 Date: Tue, 12 Mar 2002 10:57:47 -0800 From: Michael MacFaden To: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Mailing list digest, Friday 8 March Message-ID: <20020312105747.C1398@riverstonenet.com> References: <200203121415.g2CEFkC12365@bru-cse-222.cisco.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <200203121415.g2CEFkC12365@bru-cse-222.cisco.com>; from bclaise@cisco.com on Tue, Mar 12, 2002 at 03:15:46PM +0100 X-Operating-System: GNU/Linux 2.4.18 Precedence: bulk Sender: majordomo listserver On Tue, Mar 12, 2002 at 03:15:46PM +0100, Benoit Claise wrote: > What for? reliability is mainly needed for billing purposes but not for > traffic profiling, traffic engineering, ddos detection, user profiling, capacicity planning, etc.. > so again for a subset of the applications using the IPFIX data When I last spoke with Daniel McRobb who wrote 'cflowd' when he was a CAIDA, he wanted NetFlow over TCP not UDP.....And he certainly wasn't interested in doing 'billing apps' at the time. So I think this issue can't be divided this neatly. Mike MacFaden -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Mar 12 15:07:39 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA04738 for ; Tue, 12 Mar 2002 15:07:39 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16ksGG-0005LG-00 for ipfix-list@mil.doit.wisc.edu; Tue, 12 Mar 2002 13:48:40 -0600 Received: from sj-msg-core-3.cisco.com ([171.70.157.152]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16ksGD-0005Kf-00 for ipfix@net.doit.wisc.edu; Tue, 12 Mar 2002 13:48:37 -0600 Received: from postal.cisco.com (postal.cisco.com [171.71.160.26]) by sj-msg-core-3.cisco.com (8.11.3/8.9.1) with ESMTP id g2CJlx326936; Tue, 12 Mar 2002 11:47:59 -0800 (PST) Received: from WILLW2K1 (dhcp-171-71-139-122.cisco.com [171.71.139.122]) by postal.cisco.com (Mirapoint) with SMTP id AAP14102; Tue, 12 Mar 2002 11:48:24 -0800 (PST) From: "Will Eatherton" To: , Cc: Subject: RE: [ipfix] Mailing list digest, Friday 8 March Date: Tue, 12 Mar 2002 11:47:51 -0800 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) In-Reply-To: <3C8CFC95.9F845DC7@riverstonenet.com> Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4807.1700 Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Paul Calato, I agree with your point below completely. However, my understanding is that we have been told that if we don't narrow to congestion-aware/internet scenario then we should pack up IPFIX and dissolve the group. I dont like this stance, but this issue isn't worth losing the benefits of a standardized IPFIX (even if 95% of it's use will be non-compliant non-internet use :) Will > > You are missing my point. If the problem space is 95% > in the non-internet setting and we solve it for the > internet setting, we solve the wrong problem. What's > the point of that? > > We should not ignore our most prevalent deployment > scenario because it doesn't fit nicely within IETF > borders. We should say this is how to "best" solve > our problem space and then decide what to do next. > > It would certainly be good if we can come up with > something that works in both settings. I don't > see why we can't accomplish that. > > > > > ----------------------------------------------------------------------- > > Nevil Brownlee Director, Technology Development > > Phone: +64 9 373 7599 x8941 ITSS, The University of Auckland > > FAX: +64 9 373 7021 Private Bag 92019, Auckland, New Zealand > > > > -- > > Help mailto:majordomo@net.doit.wisc.edu and say "help" > in message body > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe ipfix" in message body > > Archive http://ipfix.doit.wisc.edu/archive/ > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in > message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Mar 12 15:17:55 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA05231 for ; Tue, 12 Mar 2002 15:17:55 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16ksLp-0005TG-00 for ipfix-list@mil.doit.wisc.edu; Tue, 12 Mar 2002 13:54:25 -0600 Received: from [216.167.117.195] (helo=www.inmon.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16ksLn-0005TA-00 for ipfix@net.doit.wisc.edu; Tue, 12 Mar 2002 13:54:23 -0600 Received: from phaalpc (w086.z065104045.sjc-ca.dsl.cnc.net [65.104.45.86]) by www.inmon.com (8.9.3/(dn/norelay)) with SMTP id OAA08990; Tue, 12 Mar 2002 14:54:19 -0500 Reply-To: From: "Peter Phaal" To: "'Michael MacFaden'" , Subject: RE: [ipfix] Mailing list digest, Friday 8 March Date: Tue, 12 Mar 2002 11:53:34 -0800 Message-ID: <002901c1c9ff$98f94140$3200000a@xo.com> MIME-Version: 1.0 Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook CWS, Build 9.0.2416 (9.0.2911.0) X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 In-Reply-To: <20020312105747.C1398@riverstonenet.com> Importance: Normal Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit > -----Original Message----- > From: majordomo listserver > [mailto:majordomo@mil.doit.wisc.edu]On Behalf > Of Michael MacFaden > Sent: Tuesday, March 12, 2002 10:58 AM > To: ipfix@net.doit.wisc.edu > Subject: Re: [ipfix] Mailing list digest, Friday 8 March > > > On Tue, Mar 12, 2002 at 03:15:46PM +0100, Benoit Claise wrote: > > What for? reliability is mainly needed for billing > purposes but not for > > traffic profiling, traffic engineering, ddos detection, > user profiling, capacicity planning, etc.. > > so again for a subset of the applications using the IPFIX data > > When I last spoke with Daniel McRobb who wrote 'cflowd' when > he was a CAIDA, he wanted > NetFlow over TCP not UDP.....And he certainly wasn't > interested in doing 'billing apps' at the time. > Another way to look at the need for reliable transport is to consider how heavily the data is aggregated. If you have fine grain flow data (e.g. NetFlow 5) then the loss of a record isn't as serious and can more easily be compensated for than the loss of a highly aggregated flow record (e.g. NetFlow 8). A reliable transport is desireable for highly aggregated flow data. The cost of the extra protocol overhead and buffering in this case is minimal since there isn't very much data being transported. In the case of fine grain flows the data rates can be very high and the overhead of using a reliable transport is not as easy to justify, especially since the loss of some data can be relatively easily compensated for. There seems to be a loose correspondance between highly aggregated data and billing and fine grain data and traffic engineering, DoS detection, etc. It seems that there are clear enough differences that one should be able to select a transport that suits the aggregation scheme and application, rather than picking a single transport and limiting IPFIX to one or other application area. Why not support both TCP and UDP and include a requirement that if IPFIX is being used across the Internet backbone then TCP MUST/SHOULD be used? Peter ---------------------- Peter Phaal InMon Corp. Peter_Phaal@inmon.com -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Mar 12 16:16:28 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA07880 for ; Tue, 12 Mar 2002 16:16:27 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16ktK0-0006mD-00 for ipfix-list@mil.doit.wisc.edu; Tue, 12 Mar 2002 14:56:36 -0600 Received: from sj-msg-core-2.cisco.com ([171.69.24.11]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16ktJx-0006lV-00 for ipfix@net.doit.wisc.edu; Tue, 12 Mar 2002 14:56:33 -0600 Received: from mira-sjcd-2.cisco.com (mira-sjcd-2.cisco.com [171.69.43.46]) by sj-msg-core-2.cisco.com (8.11.3/8.9.1) with ESMTP id g2CKu2h27422; Tue, 12 Mar 2002 12:56:02 -0800 (PST) Received: from cisco.com (dhcp-171-71-137-20.cisco.com [171.71.137.20]) by mira-sjcd-2.cisco.com (Mirapoint) with ESMTP id ACF66843; Tue, 12 Mar 2002 12:49:14 -0800 (PST) Message-ID: <3C8E6B62.4DB96BBF@cisco.com> Date: Tue, 12 Mar 2002 12:56:02 -0800 From: Peram Marimuthu X-Mailer: Mozilla 4.76 [en]C-CCK-MCD (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: Peter_Phaal@inmon.com CC: "'Michael MacFaden'" , ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Mailing list digest, Friday 8 March References: <002901c1c9ff$98f94140$3200000a@xo.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Peter Phaal wrote: > A reliable transport is desireable for highly aggregated flow data. The cost > of the extra protocol overhead and buffering in this case is minimal since > there isn't very much data being transported. > > In the case of fine grain flows the data rates can be very high and the > overhead of using a reliable transport is not as easy to justify, especially > since the loss of some data can be relatively easily compensated for. > > There seems to be a loose correspondance between highly aggregated data and > billing and fine grain data and traffic engineering, DoS detection, etc. It > seems that there are clear enough differences that one should be able to > select a transport that suits the aggregation scheme and application, rather > than picking a single transport and limiting IPFIX to one or other > application area. > > Why not support both TCP and UDP and include a requirement that if IPFIX is > being used across the Internet backbone then TCP MUST/SHOULD be used? > > Peter > ---------------------- > Peter Phaal > InMon Corp. > > Peter_Phaal@inmon.com > Well said. However some people are out of touch with reality and are mandating things that will never get implemented. There will be so many non-compliant implementations, but who cares Peram -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Mar 12 17:40:28 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA10746 for ; Tue, 12 Mar 2002 17:40:28 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16kugu-0000t7-00 for ipfix-list@mil.doit.wisc.edu; Tue, 12 Mar 2002 16:24:20 -0600 Received: from mailhost.auckland.ac.nz ([130.216.1.4]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16kugr-0000sy-00 for ipfix@net.doit.wisc.edu; Tue, 12 Mar 2002 16:24:18 -0600 Received: from auckland.ac.nz (root@ccu1.auckland.ac.nz [130.216.3.1]) by mailhost.auckland.ac.nz (8.9.2/8.9.2/8.9.2-ua) with ESMTP id LAA06742; Wed, 13 Mar 2002 11:24:05 +1300 (NZDT) Message-Id: <200203122224.LAA06742@mailhost.auckland.ac.nz> Date: Tue, 12 Mar 2002 14:25:51 -0800 (PST) From: n.brownlee@auckland.ac.nz Subject: [ipfix] Re: congestion awareness, reliability To: ipfix@net.doit.wisc.edu cc: Peram Marimuthu In-Reply-To: <3C8E6B62.4DB96BBF@cisco.com> MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Precedence: bulk Sender: majordomo listserver Hello all: Bickering about UDP is clearly a rathole, lets stop spending list time on it. IPFIX is concerned about producing a clear, understandable and easily implementable standard for exporting flow information. The data model, and how that data is mapped into whatever application protocol we select is what we should be focussing our design efforts on. More to the point, there's clearly strong support for reliable transport, and the ability to recover from collector or exporter failures. Seems to me that SCTP could provide both of these features, wheras if we use TCP we have to habdle failover within the IPFIX application protocol. Anyone got opinions on this notion? Nevil ----------------------------------------------------------------------- Nevil Brownlee Director, Technology Development Phone: +64 9 373 7599 x8941 ITSS, The University of Auckland FAX: +64 9 373 7021 Private Bag 92019, Auckland, New Zealand -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Mar 12 18:11:22 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA11610 for ; Tue, 12 Mar 2002 18:11:22 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16kvAt-0001Xe-00 for ipfix-list@mil.doit.wisc.edu; Tue, 12 Mar 2002 16:55:19 -0600 Received: from mailhub.xacct.com ([204.253.100.25]) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 16kvAr-0001Wg-00 for ipfix@net.doit.wisc.edu; Tue, 12 Mar 2002 16:55:17 -0600 Received: (qmail 2574 invoked from network); 12 Mar 2002 22:54:40 -0000 Received: from unknown (HELO usmail.xacct.com) (204.253.100.12) by mailhub.us.xacct.com with SMTP; 12 Mar 2002 22:54:40 -0000 Received: from peter ([204.253.100.102]) by usmail.xacct.com (8.11.2/8.11.2) with SMTP id g2CMsnm00689; Tue, 12 Mar 2002 14:54:49 -0800 From: "Peter Ludemann" To: , Cc: "Peram Marimuthu" Subject: RE: [ipfix] Re: congestion awareness, reliability Date: Tue, 12 Mar 2002 14:54:46 -0800 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) Importance: Normal In-Reply-To: <200203122224.LAA06742@mailhost.auckland.ac.nz> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit I agree that SCTP is the preferred transport; unfortunately it isn't as universally available. So, I would suggest a protocol that could use *either* TCP or SCTP as a transport. Interestingly enough, the protocol doesn't need to specify windows, time-outs, fail-overs, multi-homing, etc.; that can be left to the implementation. The only thing [I think] that the protocol needs for TCP and wouldn't need for SCTP is record-level acknowledgment. For this, the protocol specification merely needs to state that if SCTP (or similar) transport is used, the receiver does not need to send acknowledgments. (The protocol might also allow messages such as "heartbeat".) We could also state that TCP is an interim transport solution and is intended to eventually be replaced by SCTP everywhere. BTW, for those who really want non-reliable transfer, see http://www.ietf.org/internet-drafts/draft-ietf-tsvwg-usctp-00.txt, which extends SCTP to allow unreliable delivery. - peter > -----Original Message----- > From: majordomo listserver [mailto:majordomo@mil.doit.wisc.edu]On Behalf > Of n.brownlee@auckland.ac.nz > Sent: Tuesday, March 12, 2002 2:26 PM > To: ipfix@net.doit.wisc.edu > Cc: Peram Marimuthu > Subject: [ipfix] Re: congestion awareness, reliability > > > > Hello all: > > Bickering about UDP is clearly a rathole, lets stop spending list > time on it. > > IPFIX is concerned about producing a clear, understandable and easily > implementable standard for exporting flow information. The data model, > and how that data is mapped into whatever application protocol we select > is what we should be focussing our design efforts on. > > More to the point, there's clearly strong support for reliable > transport, and the ability to recover from collector or exporter > failures. Seems to me that SCTP could provide both of these features, > wheras if we use TCP we have to habdle failover within the IPFIX > application protocol. Anyone got opinions on this notion? > > Nevil > > ----------------------------------------------------------------------- > Nevil Brownlee Director, Technology Development > Phone: +64 9 373 7599 x8941 ITSS, The University of Auckland > FAX: +64 9 373 7021 Private Bag 92019, Auckland, New Zealand > > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in > message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed Mar 13 09:23:57 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA06093 for ; Wed, 13 Mar 2002 09:23:57 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16l9IU-0007aD-00 for ipfix-list@mil.doit.wisc.edu; Wed, 13 Mar 2002 08:00:06 -0600 Received: from ctron-dnm.enterasys.com ([12.25.1.120] helo=ctron-dnm.ctron.com ident=firewall-user) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16l9IR-0007Zp-00 for ipfix@net.doit.wisc.edu; Wed, 13 Mar 2002 08:00:03 -0600 Received: (from uucp@localhost) by ctron-dnm.ctron.com (8.8.7/8.8.7) id JAA00445; Wed, 13 Mar 2002 09:09:46 -0500 (EST) Received: from unknown(134.141.77.96) by ctron-dnm.ctron.com via smap (4.1) id xma000419; Wed, 13 Mar 02 09:09:04 -0500 Received: by cnc-exc1.enterasys.com with Internet Mail Service (5.5.2653.19) id ; Wed, 13 Mar 2002 08:57:48 -0500 Message-ID: <59358A738F45D51186A30008C74CE250DA07BF@slc-exc1.ctron.com> From: "Norseth, KC" To: "'Peram Marimuthu'" Cc: ipfix@net.doit.wisc.edu Subject: Why we are doing this (was RE: [ipfix] Mailing list digest, Frida y 8 March) Date: Wed, 13 Mar 2002 08:58:39 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C1CA97.2DB19610" Precedence: bulk Sender: majordomo listserver This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C1CA97.2DB19610 Content-Type: text/plain |-----Original Message----- |From: Peram Marimuthu [mailto:peram@cisco.com] |Sent: Tuesday, March 12, 2002 1:56 PM |To: Peter_Phaal@inmon.com |Cc: 'Michael MacFaden'; ipfix@net.doit.wisc.edu |Subject: Re: [ipfix] Mailing list digest, Friday 8 March | | |Well said. However some people are out of touch with reality |and are mandating things that will never get implemented. |There will be so many non-compliant implementations, but who cares We do care! That was the purpose of forming this WG, that we provide a common protocol that people can follow. Since a non-compliant implementation would not work with the industry, people will not want to implement a non-compliant one. With that said, we want this protocol to be well defined and implementable. No one wants to have something that they have to reverse engineer to verify that they are complaint with a de-facto standard. Any of the current protocols being looked at could do the job for IPFIX, it is a matter of which one would do the best, be implementable, and has the concensus of the industry. My two cents worth. K.C. ------_=_NextPart_001_01C1CA97.2DB19610 Content-Type: text/html Content-Transfer-Encoding: quoted-printable Why we are doing this (was RE: [ipfix] Mailing list digest, = Friday 8 March)

|-----Original Message-----
|From: Peram Marimuthu [mailto:peram@cisco.com]
|Sent: Tuesday, March 12, 2002 1:56 PM
|To: Peter_Phaal@inmon.com
|Cc: 'Michael MacFaden'; = ipfix@net.doit.wisc.edu
|Subject: Re: [ipfix] Mailing list digest, Friday 8 = March
|
<SNIP>
|
|Well said. However some people are out of touch = with reality
|and are mandating things that will never get = implemented.
|There will be so many non-compliant = implementations, but who cares

We do care!  That was the purpose of forming = this WG, that we provide a common protocol that people can = follow.  Since a non-compliant implementation would not work with = the industry, people will not want to implement a non-compliant = one.

With that said, we want this protocol to be well = defined and implementable.  No one wants to have something that = they have to reverse engineer to verify that they are complaint with a = de-facto standard. 

Any of the current protocols being looked at could do = the job for IPFIX, it is a matter of which one would do the best, be = implementable, and has the concensus of the industry.

My two cents worth.

K.C.

------_=_NextPart_001_01C1CA97.2DB19610-- -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed Mar 13 09:24:02 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA06094 for ; Wed, 13 Mar 2002 09:23:57 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16l9O7-0007m1-00 for ipfix-list@mil.doit.wisc.edu; Wed, 13 Mar 2002 08:05:55 -0600 Received: from ctron-dnm.enterasys.com ([12.25.1.120] helo=ctron-dnm.ctron.com ident=firewall-user) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16l9O5-0007ls-00 for ipfix@net.doit.wisc.edu; Wed, 13 Mar 2002 08:05:53 -0600 Received: (from uucp@localhost) by ctron-dnm.ctron.com (8.8.7/8.8.7) id JAA01041; Wed, 13 Mar 2002 09:15:27 -0500 (EST) Received: from unknown(134.141.77.96) by ctron-dnm.ctron.com via smap (4.1) id xma000972; Wed, 13 Mar 02 09:15:10 -0500 Received: by cnc-exc1.enterasys.com with Internet Mail Service (5.5.2653.19) id ; Wed, 13 Mar 2002 09:03:54 -0500 Message-ID: <59358A738F45D51186A30008C74CE250DA07C0@slc-exc1.ctron.com> From: "Norseth, KC" To: "'Peter Ludemann'" , n.brownlee@auckland.ac.nz, ipfix@net.doit.wisc.edu Cc: Peram Marimuthu Subject: RE: [ipfix] Re: congestion awareness, reliability Date: Wed, 13 Mar 2002 09:04:45 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C1CA98.07B9C760" Precedence: bulk Sender: majordomo listserver This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C1CA98.07B9C760 Content-Type: text/plain |-----Original Message----- |From: Peter Ludemann [mailto:peter.ludemann@xacct.com] |Sent: Tuesday, March 12, 2002 3:55 PM |To: n.brownlee@auckland.ac.nz; ipfix@net.doit.wisc.edu |Cc: Peram Marimuthu |Subject: RE: [ipfix] Re: congestion awareness, reliability | | |I agree that SCTP is the preferred transport; unfortunately it |isn't as universally available. | |So, I would suggest a protocol that could use *either* TCP or |SCTP as a transport. Interestingly enough, the protocol |doesn't need to specify windows, time-outs, fail-overs, |multi-homing, etc.; that can be left to the implementation. |The only thing [I think] that the protocol needs for TCP and |wouldn't need for SCTP is record-level acknowledgment. For |this, the protocol specification merely needs to state that if |SCTP (or similar) transport is used, the receiver does not |need to send acknowledgments. (The protocol might also allow |messages such as "heartbeat".) AGREED! |We could also state that TCP is an interim transport solution |and is intended to eventually be replaced by SCTP everywhere. Famous last words - interim solution. Some legacy products may have IPFIX implemented, but will never have SCTP implemented. As long as SCTP was not forced, this would still be ok. |BTW, for those who really want non-reliable transfer, see |http://www.ietf.org/internet-drafts/draft-ietf-tsvwg-usctp-00.t |xt, which extends SCTP to allow unreliable delivery. | |- peter | |> -----Original Message----- |> From: majordomo listserver [mailto:majordomo@mil.doit.wisc.edu]On |> Behalf Of n.brownlee@auckland.ac.nz |> Sent: Tuesday, March 12, 2002 2:26 PM |> To: ipfix@net.doit.wisc.edu |> Cc: Peram Marimuthu |> Subject: [ipfix] Re: congestion awareness, reliability |> |> |> |> Hello all: |> |> Bickering about UDP is clearly a rathole, lets stop spending |list time |> on it. |> |> IPFIX is concerned about producing a clear, understandable |and easily |> implementable standard for exporting flow information. The data |> model, and how that data is mapped into whatever application |protocol |> we select is what we should be focussing our design efforts on. |> |> More to the point, there's clearly strong support for reliable |> transport, and the ability to recover from collector or exporter |> failures. Seems to me that SCTP could provide both of these |features, |> wheras if we use TCP we have to habdle failover within the IPFIX |> application protocol. Anyone got opinions on this notion? |> |> Nevil |> |> |----------------------------------------------------------------------- |> Nevil Brownlee Director, Technology Development |> Phone: +64 9 373 7599 x8941 ITSS, The University of Auckland |> FAX: +64 9 373 7021 Private Bag 92019, Auckland, New Zealand |> |> |> -- |> Help mailto:majordomo@net.doit.wisc.edu and say "help" in |> message body |> Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe |> ipfix" in message body |> Archive http://ipfix.doit.wisc.edu/archive/ | | |-- |Help mailto:majordomo@net.doit.wisc.edu and say "help" |in message body |Unsubscribe mailto:majordomo@net.doit.wisc.edu and say |"unsubscribe ipfix" in message body |Archive http://ipfix.doit.wisc.edu/archive/ | ------_=_NextPart_001_01C1CA98.07B9C760 Content-Type: text/html Content-Transfer-Encoding: quoted-printable RE: [ipfix] Re: congestion awareness, reliability

|-----Original Message-----
|From: Peter Ludemann [mailto:peter.ludemann@xacct.com= ]
|Sent: Tuesday, March 12, 2002 3:55 PM
|To: n.brownlee@auckland.ac.nz; = ipfix@net.doit.wisc.edu
|Cc: Peram Marimuthu
|Subject: RE: [ipfix] Re: congestion awareness, = reliability
|
|
|I agree that SCTP is the preferred transport; = unfortunately it
|isn't as universally available.
|
|So, I would suggest a protocol that could use = *either* TCP or
|SCTP as a transport. Interestingly enough, the = protocol
|doesn't need to specify windows, time-outs,  = fail-overs,
|multi-homing, etc.; that can be left to the = implementation.
|The only thing [I think] that the protocol needs = for TCP and
|wouldn't need for SCTP is record-level = acknowledgment. For
|this, the protocol specification merely needs to = state that if
|SCTP (or similar) transport is used, the receiver = does not
|need to send acknowledgments. (The protocol might = also allow
|messages such as "heartbeat".)

AGREED!

|We could also state that TCP is an interim transport = solution
|and is intended to eventually be replaced by SCTP = everywhere.

Famous last words - interim solution.

Some legacy products may have IPFIX implemented, but = will never have SCTP implemented.  As long as SCTP was not forced, = this would still be ok.

|BTW, for those who really want non-reliable = transfer, see
|http://www.ietf.org/internet-drafts/draft-ietf-tsvwg-u= sctp-00.t
|xt, which extends SCTP to allow unreliable = delivery.
|
|- peter
|
|> -----Original Message-----
|> From: majordomo listserver [mailto:majordomo@mil.doit.wi= sc.edu]On
|> Behalf Of n.brownlee@auckland.ac.nz
|> Sent: Tuesday, March 12, 2002 2:26 PM
|> To: ipfix@net.doit.wisc.edu
|> Cc: Peram Marimuthu
|> Subject: [ipfix] Re: congestion awareness, = reliability
|>
|>
|>
|> Hello all:
|>
|> Bickering about UDP is clearly a rathole, lets = stop spending
|list time
|> on it.
|>
|> IPFIX is concerned about producing a clear, = understandable
|and easily
|> implementable standard for exporting flow = information.  The data
|> model, and how that data is mapped into = whatever application
|protocol
|> we select is what we should be focussing our = design efforts on.
|>
|> More to the point, there's clearly strong = support for reliable
|> transport, and the ability to recover from = collector or exporter
|> failures.  Seems to me that SCTP could = provide both of these
|features,
|> wheras if we use TCP we have to habdle = failover within the IPFIX
|> application protocol.  Anyone got = opinions on this notion?
|>
|> Nevil
|>
|>
|--------------------------------------------------------------= ---------
|>    Nevil = Brownlee          &nbs= p;        Director, Technology = Development
|>    Phone: +64 9 373 7599 = x8941      ITSS, The University of = Auckland
|>    FAX: +64 9 373 = 7021      Private Bag 92019, Auckland, New = Zealand
|>
|>
|> --
|> Help        = mailto:majordomo@net.doit.wi= sc.edu and say "help" in
|> message body
|> Unsubscribe mailto:majordomo@net.doit.wi= sc.edu and say "unsubscribe
|> ipfix" in message body
|> Archive     http://ipfix.doit.wisc.edu/archive/
|
|
|--
|Help        mailto:majordomo@net.doit.wi= sc.edu and say "help"
|in message body
|Unsubscribe mailto:majordomo@net.doit.wi= sc.edu and say
|"unsubscribe ipfix" in message = body
|Archive     http://ipfix.doit.wisc.edu/archive/
|

------_=_NextPart_001_01C1CA98.07B9C760-- -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed Mar 13 09:31:42 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA06402 for ; Wed, 13 Mar 2002 09:31:42 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16l9Xh-0000DC-00 for ipfix-list@mil.doit.wisc.edu; Wed, 13 Mar 2002 08:15:49 -0600 Received: from host3.riverstonenet.com ([63.113.148.3] helo=exc-sc1.yagosys.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16l9Xf-0000CQ-00 for ipfix@net.doit.wisc.edu; Wed, 13 Mar 2002 08:15:47 -0600 Received: from riverstonenet.com (134.141.180.92 [134.141.180.92]) by exc-sc1.yagosys.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13) id 1PZZGFF6; Wed, 13 Mar 2002 06:14:18 -0800 Message-ID: <3C8F5EBA.FDF93@riverstonenet.com> Date: Wed, 13 Mar 2002 09:14:18 -0500 From: calato@riverstonenet.com X-Mailer: Mozilla 4.75C-CCK-MCD NSCPCD475 [en] (X11; U; SunOS 5.6 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: Peter Ludemann CC: n.brownlee@auckland.ac.nz, ipfix@net.doit.wisc.edu Subject: Re: Congestion awareness, reliability (was: RE: [ipfix] Mailing list digest, Friday 8 March) References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Peter Ludemann wrote: > > It seems that there's no firm consensus on reliable/non-reliable > and whether we need congestion awareness. My belief is that > reliability and congestion-awareness cost little and gain a lot; > and that the "benefits" of UDP are marginal at best: > - load-balancing by multi-cast -- but almost the same thing can > be done by having multiple output streams with TCP; > - silent loss of data with UDP when you don't care much > about the data. > (did I miss any?) > > Additional comments inline (I found a few references on efficiency > of TCP vs UDP). > > > -----Original Message----- > > From: calato [mailto:calato]On Behalf Of calato@riverstonenet.com > > Sent: Monday, March 11, 2002 4:48 PM > > To: Peter Ludemann > > Cc: n.brownlee@auckland.ac.nz; ipfix@net.doit.wisc.edu > > Subject: Re: [ipfix] Mailing list digest, Friday 8 March > > > > > > > > I have no problem with TCP. In fact, it is my preferred > > choice (LFAP also runs over TCP). I have an issue with basing > > the decision on design criteria that I believe are highly > > suspect. It will only lead to "wrong" choices down the road. > > > > If we say that UDP is out because it is not Inter-net friendly > > and most of the deployments are Intra-net, that logic seems > > flawed, even if it arrives at the "correct" answer. > > > > Just to push the other side, more comments inline... > > > > Peter Ludemann wrote: > > > > > > Based on my experience, congestion-awareness is useful even in > > a LAN. What > > > happens if the receiver can't keep up with the sender for a > > short period? > > > You don't want to be flooding the receiver with packets that it can't > > > handle. > > > > You lose some records. In a lot of applications, that > > is perfectly acceptable. Even with TCP, if the Collector > > backs up long enough the exporter will have to drop some > > information. In either case, IPFIX will need to define and code > > for a data loss situation. > > But with TCP (or some other "reliable" transport), you know which > records you've lost and can compensate in some (e.g., aggregate > the lost values and send them later). With UDP, you don't know, unless > you have ACKs, in which case you're just reinventing a partial TCP > without retransmission and congestion control (and you'd also need to > handle duplicates and out-of-orders). If you have no application level ACK and you loose a TCP connection you still do not know which of the last n records sent made it to the other side. So then you start putting in application level ACKs for TCP, which could also be used for UDP. > > > > > > > TCP's capabilities turned out to be quite adequate for the congestion > > > handling, combined with a record-level ACK for time-out and fail-over to > > > another receiver. > > > > Record level ACK's can be done with UDP. In a lot of cases > > there is no congestion issue. > > > > > > > > As I mentioned earlier, in such situations TCP's overheads are > > barely more > > > than UDP's; > > > > > > Maybe someone else can comment on the TCP vs UDP overhead. > > This seems to be the crux of the issue. If TCP and UDP are > > roughly the same in terms of overhead, speed, etc... then > > TCP is a clear winner. If they are not roughly equal, then > > maybe both will be needed. > > I don't have numbers available; but I do notice that NFS Version 3 > is over TCP (RFC 1813, 3010). Presumably efficiency isn't a problem. > As a thought experiment, on a local network without congestion, > TCP won't be retransmitting or reordering packets; there'll be > the ACKs coming back (possibly piggybacked on the record-level > ACKs for the data protocol), so at the network level, performance > should be about the same and only slight more CPU usage. > > Java is much more efficient with TCP than UDP, by the way. > > RFC 3010 references Macklem, Rick, "Lessons Learned Tuning the > 4.3BSD Reno Implementation of the NFS Protocol", which I found > at http://sunsite.nstu.nsk.su/sun/inform/whitepapers.html. > Graph 6 shows that TCP is about 20% more expensive in CPU than > UDP (and this paper is over 10 years old, so implementations > have probably improved) -- it's not clear to me whether this > number takes into account the acknowledgment work done outside > the transport with UDP. My interpretation is that on a more > powerful machine than a microVAX, the congestion control of TCP > will more than offset CPU overhead and that TCP is a winner. > > Additional references ... > http://www.ietf.org/proceedings/98dec/43rd-ietf-98dec-142.html > > Brent Callaghan discussed the NFS version 4 effort. He recalled > that NFS was developed for UDP because TCP performance was > pretty poor in the target environment, Ethernet LANs. NFS mount > options were added for controlling timeouts and transfer sizes. > The adaptive timeouts worked okay, but the transfer size adjustment > didn't. As NFS started being used over WANs, it evolved towards > also being run over TCP, certainly a win on congested WANs, and > fitting better with NFSv3's potentially very big reads and writes. > The NFSv4 Transport Wish List: As fast as UDP on LANs; optimized > for fast RPC transactions (fast binding, reduced TIME-WAIT); > integrated record marking. > > and this: > http://www.frontiertech.com/supernfs/WHITE-P/client_opt.htm > > Since TCP is a stateful protocol, it transfers a large part > of the responsibility for guaranteeing the completion of > file requests from the client to the network, with a > concomitant increase in reliability and performance. The > efficient exploitation of this, however, requires careful > client and server design, for which no standards have yet > been devised. In addition, for similar reasons, NFS over > TCP offers superior performance when several routers ("hops") > lie between the client and server. TCP handles multiple "hopping", > which is a characteristic of the Internet, much more efficiently. Good references. I'll read them over as soon as I can. > > > > but TCP avoids having to reinvent a wheel. > > > And if you don't use a congestion-aware transport, you'll just > > have to put > > > it in yourself (based on my experience with testing in the field). > > > > Not if you don't congestion-awareness. > > See the comment above on TCP, as to why congestion-awareness > is a Good Thing on WANs. As to LANs, it's surprising how badly some > sysadmins can be at configuring a network; not to mention the > amusing effect of a hub or switch that develops a "glitch". > > > > > > > It is an educating experience to deploy a system inside a Tier > > 1 backbone > > > network ... packets often take longer/stranger paths than one > > would expect; > > > routing tables getting screwed up and connectivity lost; OC3 > > pipes connected > > > by a T1; etc. > > > > > > (Or are you discussing something other than the need for > > > congestion-awareness?) > > > > > > - peter > > > > > > ----- > > > Peter Ludemann +1.408.330.5732 > > > Chief Architect +1.650.248.3973 (mobile) > > > XACCT Technologies peter.ludemann@xacct.com > > > 2900 Lakeside Drive http://www.xacct.com > > > Santa Clara CA 95054 > > > > > > > -----Original Message----- > > > > From: majordomo listserver > > [mailto:majordomo@mil.doit.wisc.edu]On Behalf > > > > Of calato@riverstonenet.com > > > > Sent: Monday, March 11, 2002 10:51 AM > > > > To: n.brownlee@auckland.ac.nz > > > > Cc: ipfix@net.doit.wisc.edu > > > > Subject: Re: [ipfix] Mailing list digest, Friday 8 March > > > > > > > > > > > > n.brownlee@auckland.ac.nz wrote: > > > > > > > > > > Hi Paul: > > > > > > > > > > On 11 Mar, calato@riverstonenet.com wrote: > > > > > > > > > > > >> - Nevil, 4 Mar: Note setting out IESG / WH chair position on > > > > > >> this issue. In brief, if IPFIX doesn't mandate congestion- > > > > > >> aware transport, the WG RFCs won't get approved. > > > > > > > > > > > > I believe there are a large number of real world deployment > > > > > > cases where IPFIX is not going over the Internet. In fact, > > > > > > it is likely to be the majority of deployment > > cases. Is there > > > > > > disagreement on this point? > > > > > > > > > > > > If not, we ought to decide the best way to solve the problem > > > > > > first, then decide what that means for us as an IETF group; > > > > > > not the other way around. > > > > > > > > > > As an IETF WG we're honour-bound to come up with an architecture/ > > > > > protocol/whatever that's good for the Internet overall, that's why > > > > > we have to use congestion-aware transport. > > > > > 'Solving the problem,' > > > > > in a non-Internet setting, is not > > > > > > > > You are missing my point. If the problem space is 95% > > > > in the non-internet setting and we solve it for the > > > > internet setting, we solve the wrong problem. What's > > > > the point of that? > > > > > > > > We should not ignore our most prevalent deployment > > > > scenario because it doesn't fit nicely within IETF > > > > borders. We should say this is how to "best" solve > > > > our problem space and then decide what to do next. > > > > > > > > It would certainly be good if we can come up with > > > > something that works in both settings. I don't > > > > see why we can't accomplish that. > > > > > > > > > > > > > > > > ----------------------------------------------------------------------- > > > > > Nevil Brownlee Director, Technology Development > > > > > Phone: +64 9 373 7599 x8941 ITSS, The University of Auckland > > > > > FAX: +64 9 373 7021 Private Bag 92019, Auckland, New Zealand -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed Mar 13 09:35:14 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA06537 for ; Wed, 13 Mar 2002 09:35:13 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16l9aX-0000IQ-00 for ipfix-list@mil.doit.wisc.edu; Wed, 13 Mar 2002 08:18:45 -0600 Received: from host3.riverstonenet.com ([63.113.148.3] helo=exc-sc1.yagosys.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16l9aV-0000Hk-00 for ipfix@net.doit.wisc.edu; Wed, 13 Mar 2002 08:18:43 -0600 Received: from riverstonenet.com (134.141.180.92 [134.141.180.92]) by exc-sc1.yagosys.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13) id 1PZZGFGG; Wed, 13 Mar 2002 06:17:19 -0800 Message-ID: <3C8F5F6F.9C16E835@riverstonenet.com> Date: Wed, 13 Mar 2002 09:17:19 -0500 From: calato@riverstonenet.com X-Mailer: Mozilla 4.75C-CCK-MCD NSCPCD475 [en] (X11; U; SunOS 5.6 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: Peter Ludemann CC: Fred True , n.brownlee@auckland.ac.nz, ipfix@net.doit.wisc.edu Subject: Re: Congestion awareness, reliability (was: RE: [ipfix] Mailing list digest, Friday 8 March) References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Peter Ludemann wrote: > > > > But with TCP (or some other "reliable" transport), you know which > > > records you've lost and can compensate in some (e.g., aggregate > > > the lost values and send them later). With UDP, you don't know, unless > > > you have ACKs, > > > > You know which records you've lost in UDP if the datagrams contain atomic > > serial numbers, as they do in the current Cisco Netflow export format. > > This is the way we currently detect and account for loss in our netflow > > applications (you know how many flow records you lose, but of course you > > do not know anything about what they contained; but you wouldn't with TCP > > either). > > > > -fred > > But only the *sender* knows what was in the lost data (e.g., the total > number > of bytes that it saw); and UDP doesn't communicate that back to the sender. > So, yes, the receiver can know how many records were lost but it has no idea > of what data was lost. With TCP, you know which records you couldn't send, > so you could calculate aggregates of the data that wasn't sent (and send > these > aggregates later, present them in a MIB, etc.) How do you know what records got lost? Some number of records are in the TCP pipe and if you loose the TCP connection you don't know which ones made it and which ones didn't. > > - peter > > > -- > > Help mailto:majordomo@net.doit.wisc.edu and say "help" in > > message body > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe ipfix" in message body > > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed Mar 13 10:14:00 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA07541 for ; Wed, 13 Mar 2002 10:13:59 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16lAC9-0001Br-00 for ipfix-list@mil.doit.wisc.edu; Wed, 13 Mar 2002 08:57:37 -0600 Received: from host3.riverstonenet.com ([63.113.148.3] helo=exc-sc1.yagosys.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16lAC7-00019y-00 for ipfix@net.doit.wisc.edu; Wed, 13 Mar 2002 08:57:35 -0600 Received: from riverstonenet.com (134.141.180.92 [134.141.180.92]) by exc-sc1.yagosys.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13) id 1PZZGGH4; Wed, 13 Mar 2002 06:56:12 -0800 Message-ID: <3C8F688B.A565747D@riverstonenet.com> Date: Wed, 13 Mar 2002 09:56:11 -0500 From: calato@riverstonenet.com X-Mailer: Mozilla 4.75C-CCK-MCD NSCPCD475 [en] (X11; U; SunOS 5.6 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: n.brownlee@auckland.ac.nz CC: ipfix@net.doit.wisc.edu, Peram Marimuthu Subject: Re: [ipfix] Re: congestion awareness, reliability References: <200203122224.LAA06742@mailhost.auckland.ac.nz> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit n.brownlee@auckland.ac.nz wrote: > > Hello all: > > Bickering about UDP is clearly a rathole, lets stop spending list > time on it. I don't see it as bickering about UDP, I see it as trying to get to an understanding of what we mean by reliability, when do we need it and when don't we need it. Peter Phaal made some excellent points on this in his mail http://ipfx.doit.wisc.edu/list/ipfx/archive/0831.html We are also getting to an understanding of when and where we need congestion control. Seems like healthy discussion to me. > > IPFIX is concerned about producing a clear, understandable and easily > implementable standard for exporting flow information. The data model, > and how that data is mapped into whatever application protocol we select > is what we should be focussing our design efforts on. > > More to the point, there's clearly strong support for reliable > transport, I have not seen that on this list. I have seen reliability asked for but the defintion of that is not clear to me. It could mean a reliable transport or it could mean no loss of data transported, they are not the same. I have also seen support for congestion aware transport in some situations. > and the ability to recover from collector or exporter failures. > Seems to me that SCTP could provide both of these features, > wheras if we use TCP we have to habdle failover within the IPFIX > application protocol. Anyone got opinions on this notion? > > Nevil > > ----------------------------------------------------------------------- > Nevil Brownlee Director, Technology Development > Phone: +64 9 373 7599 x8941 ITSS, The University of Auckland > FAX: +64 9 373 7021 Private Bag 92019, Auckland, New Zealand > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed Mar 13 11:48:07 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA10848 for ; Wed, 13 Mar 2002 11:48:07 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16lBbr-0003C4-00 for ipfix-list@mil.doit.wisc.edu; Wed, 13 Mar 2002 10:28:15 -0600 Received: from sj-msg-core-3.cisco.com ([171.70.157.152]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16lBbp-0003BD-00 for ipfix@net.doit.wisc.edu; Wed, 13 Mar 2002 10:28:13 -0600 Received: from mira-sjcd-2.cisco.com (mira-sjcd-2.cisco.com [171.69.43.46]) by sj-msg-core-3.cisco.com (8.11.3/8.9.1) with ESMTP id g2DGRX313775; Wed, 13 Mar 2002 08:27:33 -0800 (PST) Received: from cisco.com (dhcp-171-71-137-20.cisco.com [171.71.137.20]) by mira-sjcd-2.cisco.com (Mirapoint) with ESMTP id ACF86975; Wed, 13 Mar 2002 08:20:51 -0800 (PST) Message-ID: <3C8F7DFA.57397ED@cisco.com> Date: Wed, 13 Mar 2002 08:27:38 -0800 From: Peram Marimuthu X-Mailer: Mozilla 4.76 [en]C-CCK-MCD (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: calato@riverstonenet.com CC: n.brownlee@auckland.ac.nz, ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Re: congestion awareness, reliability References: <200203122224.LAA06742@mailhost.auckland.ac.nz> <3C8F688B.A565747D@riverstonenet.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit I agree with Paul. Ganesh presented some arguments at the last IETF on why TCP will be troublesome for high-end platforms. I have seen *no* arguments or analysis to say why this is not a problem. All I hear is "TCP or close your working group". TCP or SCTP will work for some scenarios and is required for some problems. But to mandate it for all scenarios and all problems just shows that people with power are not willing to listen. Peram calato@riverstonenet.com wrote: > n.brownlee@auckland.ac.nz wrote: > > > > Hello all: > > > > Bickering about UDP is clearly a rathole, lets stop spending list > > time on it. > > I don't see it as bickering about UDP, I see it as > trying to get to an understanding of what we mean > by reliability, when do we need it and when don't > we need it. Peter Phaal made some excellent points > on this in his mail > > http://ipfx.doit.wisc.edu/list/ipfx/archive/0831.html > > We are also getting to an understanding of when and > where we need congestion control. Seems like healthy > discussion to me. > > > > > IPFIX is concerned about producing a clear, understandable and easily > > implementable standard for exporting flow information. The data model, > > and how that data is mapped into whatever application protocol we select > > is what we should be focussing our design efforts on. > > > > More to the point, there's clearly strong support for reliable > > transport, > > I have not seen that on this list. I have seen reliability > asked for but the defintion of that is not clear to me. It > could mean a reliable transport or it could mean no loss of > data transported, they are not the same. > > I have also seen support for congestion aware transport > in some situations. > > > and the ability to recover from collector or exporter failures. > > Seems to me that SCTP could provide both of these features, > > wheras if we use TCP we have to habdle failover within the IPFIX > > application protocol. Anyone got opinions on this notion? > > > > Nevil > > > > ----------------------------------------------------------------------- > > Nevil Brownlee Director, Technology Development > > Phone: +64 9 373 7599 x8941 ITSS, The University of Auckland > > FAX: +64 9 373 7021 Private Bag 92019, Auckland, New Zealand > > > > -- > > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > "unsubscribe ipfix" in message body > > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed Mar 13 12:03:27 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA11277 for ; Wed, 13 Mar 2002 12:03:27 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16lBu3-0003dx-00 for ipfix-list@mil.doit.wisc.edu; Wed, 13 Mar 2002 10:47:03 -0600 Received: from host3.riverstonenet.com ([63.113.148.3] helo=exc-sc1.yagosys.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16lBu1-0003d5-00 for ipfix@net.doit.wisc.edu; Wed, 13 Mar 2002 10:47:01 -0600 Received: from riverstonenet.com (134.141.180.92 [134.141.180.92]) by exc-sc1.yagosys.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13) id 1PZZGHNR; Wed, 13 Mar 2002 08:45:38 -0800 Message-ID: <3C8F8231.6B41A386@riverstonenet.com> Date: Wed, 13 Mar 2002 11:45:37 -0500 From: calato@riverstonenet.com X-Mailer: Mozilla 4.75C-CCK-MCD NSCPCD475 [en] (X11; U; SunOS 5.6 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: Peram Marimuthu CC: n.brownlee@auckland.ac.nz, ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Re: congestion awareness, reliability References: <200203122224.LAA06742@mailhost.auckland.ac.nz> <3C8F688B.A565747D@riverstonenet.com> <3C8F7DFA.57397ED@cisco.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Peram Marimuthu wrote: > > I agree with Paul. Ganesh presented some arguments at the last IETF on > why TCP will be troublesome for high-end platforms. I have to disagree with that statement. Ganesh presented some calculations. From what I recall a lot of the assumptions that went into those calculations were called into question. > I have seen *no* arguments > or analysis to say why this is not a problem. All I hear is "TCP or close your > working group". > > TCP or SCTP will work for some scenarios and is required for some problems. > But to mandate it for all scenarios and all problems just shows that people with > power are not willing to listen. > > Peram > > calato@riverstonenet.com wrote: > > > n.brownlee@auckland.ac.nz wrote: > > > > > > Hello all: > > > > > > Bickering about UDP is clearly a rathole, lets stop spending list > > > time on it. > > > > I don't see it as bickering about UDP, I see it as > > trying to get to an understanding of what we mean > > by reliability, when do we need it and when don't > > we need it. Peter Phaal made some excellent points > > on this in his mail > > > > http://ipfx.doit.wisc.edu/list/ipfx/archive/0831.html > > > > We are also getting to an understanding of when and > > where we need congestion control. Seems like healthy > > discussion to me. > > > > > > > > IPFIX is concerned about producing a clear, understandable and easily > > > implementable standard for exporting flow information. The data model, > > > and how that data is mapped into whatever application protocol we select > > > is what we should be focussing our design efforts on. > > > > > > More to the point, there's clearly strong support for reliable > > > transport, > > > > I have not seen that on this list. I have seen reliability > > asked for but the defintion of that is not clear to me. It > > could mean a reliable transport or it could mean no loss of > > data transported, they are not the same. > > > > I have also seen support for congestion aware transport > > in some situations. > > > > > and the ability to recover from collector or exporter failures. > > > Seems to me that SCTP could provide both of these features, > > > wheras if we use TCP we have to habdle failover within the IPFIX > > > application protocol. Anyone got opinions on this notion? > > > > > > Nevil > > > > > > ----------------------------------------------------------------------- > > > Nevil Brownlee Director, Technology Development > > > Phone: +64 9 373 7599 x8941 ITSS, The University of Auckland > > > FAX: +64 9 373 7021 Private Bag 92019, Auckland, New Zealand > > > > > > -- > > > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > "unsubscribe ipfix" in message body > > > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed Mar 13 12:22:41 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA11949 for ; Wed, 13 Mar 2002 12:22:41 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16lCE0-000457-00 for ipfix-list@mil.doit.wisc.edu; Wed, 13 Mar 2002 11:07:40 -0600 Received: from sj-msg-core-1.cisco.com ([171.71.163.11]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16lCDz-00044W-00 for ipfix@net.doit.wisc.edu; Wed, 13 Mar 2002 11:07:39 -0600 Received: from mira-sjcd-2.cisco.com (mira-sjcd-2.cisco.com [171.69.43.46]) by sj-msg-core-1.cisco.com (8.11.3/8.9.1) with ESMTP id g2DH77d25201; Wed, 13 Mar 2002 09:07:07 -0800 (PST) Received: from cisco.com (dhcp-171-71-137-20.cisco.com [171.71.137.20]) by mira-sjcd-2.cisco.com (Mirapoint) with ESMTP id ACF87884; Wed, 13 Mar 2002 09:00:10 -0800 (PST) Message-ID: <3C8F8730.7E67AA08@cisco.com> Date: Wed, 13 Mar 2002 09:06:57 -0800 From: Peram Marimuthu X-Mailer: Mozilla 4.76 [en]C-CCK-MCD (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: calato@riverstonenet.com CC: n.brownlee@auckland.ac.nz, ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Re: congestion awareness, reliability References: <200203122224.LAA06742@mailhost.auckland.ac.nz> <3C8F688B.A565747D@riverstonenet.com> <3C8F7DFA.57397ED@cisco.com> <3C8F8231.6B41A386@riverstonenet.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit calato@riverstonenet.com wrote: > Peram Marimuthu wrote: > > > > Ganesh presented some arguments at the last IETF on > > why TCP will be troublesome for high-end platforms. > > I have to disagree with that statement. Ganesh presented > some calculations. From what I recall a lot of the > assumptions that went into those calculations were > called into question. > Ok. If I am right, Ganesh's presentation said that we need 6.8Gbytes of memory and 30% CPU utilization to support Netflow (with TCP) on a 40G router. Let us assume that he is completely wrong and the requirement is just 1/5th. Even then we need a buffer of about 1.4Gbytes. Now, what I am looking for is a presentation with the right calculations and acceptable assumptions that shows buffer requirements for a 40G box. Assume 100 ms RTT, 50Kbps as bandwidth per flow, 40 bytes as flow record size, path mtu is 1500 bytes and you will have to support 1 retransmission. Convince that buffering requirements are not very high and TCP will work here. Thanks Peram -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed Mar 13 12:41:29 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA12656 for ; Wed, 13 Mar 2002 12:41:29 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16lCVf-0004Uo-00 for ipfix-list@mil.doit.wisc.edu; Wed, 13 Mar 2002 11:25:55 -0600 Received: from mailhub.xacct.com ([204.253.100.25]) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 16lCVc-0004Ty-00 for ipfix@net.doit.wisc.edu; Wed, 13 Mar 2002 11:25:52 -0600 Received: (qmail 22865 invoked from network); 13 Mar 2002 17:25:20 -0000 Received: from unknown (HELO usmail.xacct.com) (204.253.100.12) by mailhub.us.xacct.com with SMTP; 13 Mar 2002 17:25:20 -0000 Received: from peter (cpe-66-87-83-93.ca.sprintbbd.net [66.87.83.93]) by usmail.xacct.com (8.11.2/8.11.2) with SMTP id g2DHPNm14700; Wed, 13 Mar 2002 09:25:23 -0800 From: "Peter Ludemann" To: Cc: "Fred True" , , Subject: RE: Congestion awareness, reliability (was: RE: [ipfix] Mailing list digest, Friday 8 March) Date: Wed, 13 Mar 2002 09:25:20 -0800 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) Importance: Normal In-Reply-To: <3C8F5F6F.9C16E835@riverstonenet.com> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit I think I'll be the first person in this mailing group to say ... I was wrong. (That the sender would know what records were lost if TCP is used. Even SCTP can't give a 100% answer for that.) But my other arguments for reliable transport still are valid, I think. Has anybody found other references about CPU cost for using TCP vs UDP? All I can say is that when I use CRANE for sending 3000 records per second (over TCP), I can barely see the CPU meter move. By the way, our company has spent a lot of time and money in making NetFlow reception as reliable as possible. We did this because *customers* wanted it and paid us money for it. That's the kind of "rough consensus" that I believe in. (Running code we've already got.) - peter > -----Original Message----- > From: calato [mailto:calato]On Behalf Of calato@riverstonenet.com > Sent: Wednesday, March 13, 2002 6:17 AM > To: Peter Ludemann > Cc: Fred True; n.brownlee@auckland.ac.nz; ipfix@net.doit.wisc.edu > Subject: Re: Congestion awareness, reliability (was: RE: [ipfix] Mailing > list digest, Friday 8 March) > > > Peter Ludemann wrote: > > > > > > But with TCP (or some other "reliable" transport), you know which > > > > records you've lost and can compensate in some (e.g., aggregate > > > > the lost values and send them later). With UDP, you don't > know, unless > > > > you have ACKs, > > > > > > You know which records you've lost in UDP if the datagrams > contain atomic > > > serial numbers, as they do in the current Cisco Netflow export format. > > > This is the way we currently detect and account for loss in > our netflow > > > applications (you know how many flow records you lose, but of > course you > > > do not know anything about what they contained; but you > wouldn't with TCP > > > either). > > > > > > -fred > > > > But only the *sender* knows what was in the lost data (e.g., the total > > number > > of bytes that it saw); and UDP doesn't communicate that back to > the sender. > > So, yes, the receiver can know how many records were lost but > it has no idea > > of what data was lost. With TCP, you know which records you > couldn't send, > > so you could calculate aggregates of the data that wasn't sent (and send > > these > > aggregates later, present them in a MIB, etc.) > > How do you know what records got lost? Some number of > records are in the TCP pipe and if you loose the TCP > connection you don't know which ones made it and which > ones didn't. > > > > > > > - peter > > > > > -- > > > Help mailto:majordomo@net.doit.wisc.edu and say "help" in > > > message body > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > "unsubscribe ipfix" in message body > > > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed Mar 13 15:27:53 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA18158 for ; Wed, 13 Mar 2002 15:27:53 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16lF4s-000086-00 for ipfix-list@mil.doit.wisc.edu; Wed, 13 Mar 2002 14:10:26 -0600 Received: from mail.riverstonenet.com ([63.113.148.10] helo=riverstonenet.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16lF4p-000076-00 for ipfix@net.doit.wisc.edu; Wed, 13 Mar 2002 14:10:23 -0600 Received: from agile.yagosys.com by riverstonenet.com (8.9.3+Sun/SMI-SVR4-Yago) id MAA14786; Wed, 13 Mar 2002 12:09:52 -0800 (PST) Received: from agile.yagosys.com (agile.yagosys.com [127.0.0.1]) by agile.yagosys.com (8.12.0/8.12.0) with ESMTP id g2DK9qFA020694 for ; Wed, 13 Mar 2002 12:09:52 -0800 Received: (from mrm@localhost) by agile.yagosys.com (8.12.0/8.12.0/Submit) id g2DK9paP020692 for ipfix@net.doit.wisc.edu; Wed, 13 Mar 2002 12:09:51 -0800 Date: Wed, 13 Mar 2002 12:09:51 -0800 From: Michael MacFaden To: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Re: congestion awareness, reliability Message-ID: <20020313120951.D5177@riverstonenet.com> References: <3C8E6B62.4DB96BBF@cisco.com> <200203122224.LAA06742@mailhost.auckland.ac.nz> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <200203122224.LAA06742@mailhost.auckland.ac.nz>; from n.brownlee@auckland.ac.nz on Tue, Mar 12, 2002 at 02:25:51PM -0800 X-Operating-System: GNU/Linux 2.4.18 Precedence: bulk Sender: majordomo listserver On Tue, Mar 12, 2002 at 02:25:51PM -0800, n.brownlee@auckland.ac.nz wrote: >More to the point, there's clearly strong support for reliable >transport, and the ability to recover from collector or exporter >failures. Seems to me that SCTP could provide both of these features, >wheras if we use TCP we have to habdle failover within the IPFIX >application protocol. Anyone got opinions on this notion? If we want to be able to use any arbitrary transport, then detection of failed connections or applications must be done in the protocol. I remember some bizarre behaviors in Microsoft platforms where the tcp session was still up but the application had long since died... I suggest the time to detect failure must be deterministic so a complaint implementation can be measured. Finally, determining what to do on failure could take some time to develop but is well worth it. I've seen the following in existing protocols: 1) Just detect you've lost data, if running unicast, you are SOL. if running multicast, then make collectors figure out who has the most complete set of data. 2) Detect you've lost data and then keep simple aggregate byte/pdu counters available via MIB or CLI to track how much has been lost. 3) Keep a queue of records (telco's demand such data spooled to disk for later file xfer) and wait till connection is established, and perform tail drops when queue is full. On reconnection, schedule xfer of the existing queued elements and any new data. Very nice to have congestion aware transport here when you have two streams going.. 4) Attempt to switch to an aggregate method to keep tx queue from filling up too fast hoping another connection will be available. Mike MacFaden -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed Mar 13 17:02:17 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA21296 for ; Wed, 13 Mar 2002 17:02:16 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16lGTk-0002A0-00 for ipfix-list@mil.doit.wisc.edu; Wed, 13 Mar 2002 15:40:12 -0600 Received: from sj-msg-core-3.cisco.com ([171.70.157.152]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16lGTh-00028c-00 for ipfix@net.doit.wisc.edu; Wed, 13 Mar 2002 15:40:09 -0600 Received: from mira-sjcd-1.cisco.com (mira-sjcd-1.cisco.com [171.69.43.44]) by sj-msg-core-3.cisco.com (8.11.3/8.9.1) with ESMTP id g2DLdV322222; Wed, 13 Mar 2002 13:39:31 -0800 (PST) Received: from cisco.com (dhcp-171-71-137-36.cisco.com [171.71.137.36]) by mira-sjcd-1.cisco.com (Mirapoint) with ESMTP id ABO06034; Wed, 13 Mar 2002 13:40:00 -0800 (PST) Message-ID: <3C8FC711.4B9BDD18@cisco.com> Date: Wed, 13 Mar 2002 13:39:30 -0800 From: Ganesh Sadasivan X-Mailer: Mozilla 4.76 [en]C-CCK-MCD (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: calato@riverstonenet.com CC: Peram Marimuthu , n.brownlee@auckland.ac.nz, ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Re: congestion awareness, reliability References: <200203122224.LAA06742@mailhost.auckland.ac.nz> <3C8F688B.A565747D@riverstonenet.com> <3C8F7DFA.57397ED@cisco.com> <3C8F8231.6B41A386@riverstonenet.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit calato@riverstonenet.com wrote: > Peram Marimuthu wrote: > > > > I agree with Paul. Ganesh presented some arguments at the last IETF on > > why TCP will be troublesome for high-end platforms. > > I have to disagree with that statement. Ganesh presented > some calculations. From what I recall a lot of the > assumptions that went into those calculations were > called into question. The main question I heard was the # of memory copies and I guess the numbers would be similar for UDP. But in terms of protocol processing TCP is much more intensive when compared to UDP (checksum calculation, ack processing, RTT adjustment). Agreed that there are many schemes to make TCP efficient - zero-copy, checksum offloading, interrupt coalesing etc. many of which require special h/w if the CPU utilization needs to remain low. But this cannot be assumed if IPFIX were to go into existing deployments. > > > > I have seen *no* arguments > > or analysis to say why this is not a problem. All I hear is "TCP or close your > > working group". > > > > TCP or SCTP will work for some scenarios and is required for some problems. > > But to mandate it for all scenarios and all problems just shows that people with > > power are not willing to listen. > > > > Peram > > > > calato@riverstonenet.com wrote: > > > > > n.brownlee@auckland.ac.nz wrote: > > > > > > > > Hello all: > > > > > > > > Bickering about UDP is clearly a rathole, lets stop spending list > > > > time on it. > > > > > > I don't see it as bickering about UDP, I see it as > > > trying to get to an understanding of what we mean > > > by reliability, when do we need it and when don't > > > we need it. Peter Phaal made some excellent points > > > on this in his mail > > > > > > http://ipfx.doit.wisc.edu/list/ipfx/archive/0831.html > > > > > > We are also getting to an understanding of when and > > > where we need congestion control. Seems like healthy > > > discussion to me. > > > > > > > > > > > IPFIX is concerned about producing a clear, understandable and easily > > > > implementable standard for exporting flow information. The data model, > > > > and how that data is mapped into whatever application protocol we select > > > > is what we should be focussing our design efforts on. > > > > > > > > More to the point, there's clearly strong support for reliable > > > > transport, > > > > > > I have not seen that on this list. I have seen reliability > > > asked for but the defintion of that is not clear to me. It > > > could mean a reliable transport or it could mean no loss of > > > data transported, they are not the same. > > > > > > I have also seen support for congestion aware transport > > > in some situations. > > > > > > > and the ability to recover from collector or exporter failures. > > > > Seems to me that SCTP could provide both of these features, > > > > wheras if we use TCP we have to habdle failover within the IPFIX > > > > application protocol. Anyone got opinions on this notion? > > > > > > > > Nevil > > > > > > > > ----------------------------------------------------------------------- > > > > Nevil Brownlee Director, Technology Development > > > > Phone: +64 9 373 7599 x8941 ITSS, The University of Auckland > > > > FAX: +64 9 373 7021 Private Bag 92019, Auckland, New Zealand > > > > > > > > -- > > > > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > > > > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > > > > "unsubscribe ipfix" in message body > > > > Archive http://ipfix.doit.wisc.edu/archive/ > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu Mar 14 12:01:47 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA22661 for ; Thu, 14 Mar 2002 12:01:46 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16lYJd-0006TS-00 for ipfix-list@mil.doit.wisc.edu; Thu, 14 Mar 2002 10:42:57 -0600 Received: from sj-msg-core-3.cisco.com ([171.70.157.152]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16lYJa-0006So-00 for ipfix@net.doit.wisc.edu; Thu, 14 Mar 2002 10:42:54 -0600 Received: from mira-sjcd-2.cisco.com (mira-sjcd-2.cisco.com [171.69.43.46]) by sj-msg-core-3.cisco.com (8.11.3/8.9.1) with ESMTP id g2EGgB317857; Thu, 14 Mar 2002 08:42:12 -0800 (PST) Received: from cisco.com (dhcp-171-71-137-20.cisco.com [171.71.137.20]) by mira-sjcd-2.cisco.com (Mirapoint) with ESMTP id ACG16653; Thu, 14 Mar 2002 08:35:29 -0800 (PST) Message-ID: <3C90D2E8.9C8BB9CE@cisco.com> Date: Thu, 14 Mar 2002 08:42:16 -0800 From: Peram Marimuthu X-Mailer: Mozilla 4.76 [en]C-CCK-MCD (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: kevin.zhang@us.xacct.com CC: calato@riverstonenet.com, n.brownlee@auckland.ac.nz, ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Re: congestion awareness, reliability References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Kevin, "kevin.zhang" wrote: > > > Ok. If I am right, Ganesh's presentation said that we need 6.8Gbytes > > of memory and 30% CPU utilization to support Netflow (with TCP) > > on a 40G router. > > Let us assume that he is completely wrong and the requirement is > > just 1/5th. > > Even then we need a buffer of about 1.4Gbytes. > > > > Now, what I am looking for is a presentation with the right calculations > > and acceptable assumptions that shows buffer requirements for a 40G box. > > Assume 100 ms RTT, 50Kbps as bandwidth per flow, 40 bytes as flow record > > size, path mtu is 1500 bytes and you will have to support 1 > > retransmission. > > Convince that buffering requirements are not very high and TCP > > will work here. > > > > Thanks > > Peram > > > > Paul's recollection is accurate. I remembered about the presentation assumed that a TCP SEND needs three memory copies, which might be arguable. > > As engineers, we encounter these problems (like Ganesh presented) everyday. Our job is to design a solution that meets customer's requirement given the constraints of the system (System engineering 101). I doubt an application will be interested in each IP packets routed by your high end router, many mechanisms (described in the architecture draft) such as filtering, aggregating, efficient data representation, etc. can significantly reduce the amount of exported data as well as memory consumption. > > If IPFIX is deployed over LAN, the RTT would be much smaller, maybe in 10ms range; this is 10 fold reduction of the memory from your presentation. > > Thanks, > > Kevin A 40G router is a mid-range router, not a high end. We have been shipping 160G boxes for quite some time and you dont even want to do calculations for TCP for this router. Now the argument "filtering, aggregating, efficient data representation, etc. can significantly reduce the amount of exported data as well as memory consumption" does not solve everyone's problem. If someone is able to do Netflow on 40G routers and you tell him that he has "filter, aggregate etc" to use IPfix guess what he is going to do. Even on a LAN, if you do a 10 fold reduction, the buffer requirements are very high. Peram -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri Mar 15 09:25:47 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA29603 for ; Fri, 15 Mar 2002 09:25:46 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16lsMT-0005d4-00 for ipfix-list@mil.doit.wisc.edu; Fri, 15 Mar 2002 08:07:13 -0600 Received: from host3.riverstonenet.com ([63.113.148.3] helo=exc-sc1.yagosys.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16lsMQ-0005c6-00 for ipfix@net.doit.wisc.edu; Fri, 15 Mar 2002 08:07:10 -0600 Received: from riverstonenet.com (134.141.180.106 [134.141.180.106]) by exc-sc1.yagosys.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13) id 1PZZHHC2; Fri, 15 Mar 2002 06:05:45 -0800 Message-ID: <3C91FFB7.C832151A@riverstonenet.com> Date: Fri, 15 Mar 2002 09:05:43 -0500 From: calato@riverstonenet.com X-Mailer: Mozilla 4.75C-CCK-MCD NSCPCD475 [en] (X11; U; SunOS 5.6 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: Peram Marimuthu CC: kevin.zhang@us.xacct.com, n.brownlee@auckland.ac.nz, ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Re: congestion awareness, reliability References: <3C90D2E8.9C8BB9CE@cisco.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Peram Marimuthu wrote: > > Kevin, > > "kevin.zhang" wrote: > > > > > > Ok. If I am right, Ganesh's presentation said that we need 6.8Gbytes > > > of memory and 30% CPU utilization to support Netflow (with TCP) > > > on a 40G router. > > > Let us assume that he is completely wrong and the requirement is > > > just 1/5th. > > > Even then we need a buffer of about 1.4Gbytes. > > > > > > Now, what I am looking for is a presentation with the right calculations > > > and acceptable assumptions that shows buffer requirements for a 40G box. > > > Assume 100 ms RTT, 50Kbps as bandwidth per flow, 40 bytes as flow record > > > size, path mtu is 1500 bytes and you will have to support 1 > > > retransmission. > > > Convince that buffering requirements are not very high and TCP > > > will work here. > > > > > > Thanks > > > Peram > > > > > > > Paul's recollection is accurate. I remembered about the presentation assumed that a TCP SEND needs three memory copies, which might be arguable. > > > > As engineers, we encounter these problems (like Ganesh presented) everyday. Our job is to design a solution that meets customer's requirement given the constraints of the system (System engineering 101). I doubt an application will be interested in each IP packets routed by your high end router, many mechanisms (described in the architecture draft) such as filtering, aggregating, efficient data representation, etc. can significantly reduce the amount of exported data as well as memory consumption. > > > > If IPFIX is deployed over LAN, the RTT would be much smaller, maybe in 10ms range; this is 10 fold reduction of the memory from your presentation. > > > > Thanks, > > > > Kevin > > A 40G router is a mid-range router, not a high end. We have been shipping 160G boxes for quite some time > and you dont even want to do calculations for TCP for this router. Now the argument > "filtering, aggregating, efficient data representation, etc. can significantly reduce the amount of exported data as well as memory consumption" does not solve everyone's problem. If someone is able to do > Netflow on 40G routers and you tell him that he has "filter, aggregate etc" to use IPfix guess > what he is going to do. > > Even on a LAN, if you do a 10 fold reduction, the buffer requirements are very high. Just a question, if we take the numbers at face value, can even UDP handle the load? Also, is there a pointer to Ganesh's presentation? I couldn't find it. > > Peram -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri Mar 15 13:25:50 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA06696 for ; Fri, 15 Mar 2002 13:25:48 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16lw1H-0003GX-00 for ipfix-list@mil.doit.wisc.edu; Fri, 15 Mar 2002 12:01:35 -0600 Received: from sj-msg-core-4.cisco.com ([171.71.163.10]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16lw1F-0003GL-00 for ipfix@net.doit.wisc.edu; Fri, 15 Mar 2002 12:01:33 -0600 Received: from mira-sjcd-1.cisco.com (mira-sjcd-1.cisco.com [171.69.43.44]) by sj-msg-core-4.cisco.com (8.11.3/8.9.1) with ESMTP id g2FI10T25831; Fri, 15 Mar 2002 10:01:00 -0800 (PST) Received: from cisco.com (dhcp-171-71-137-36.cisco.com [171.71.137.36]) by mira-sjcd-1.cisco.com (Mirapoint) with ESMTP id ABO67383; Fri, 15 Mar 2002 10:01:26 -0800 (PST) Message-ID: <3C9236D8.FFBD8DD4@cisco.com> Date: Fri, 15 Mar 2002 10:00:56 -0800 From: Ganesh Sadasivan X-Mailer: Mozilla 4.76 [en]C-CCK-MCD (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: calato@riverstonenet.com CC: Peram Marimuthu , kevin.zhang@us.xacct.com, n.brownlee@auckland.ac.nz, ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Re: congestion awareness, reliability References: <3C90D2E8.9C8BB9CE@cisco.com> <3C91FFB7.C832151A@riverstonenet.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit You can it in http://ipfix.doit.wisc.edu/IETF52/ calato@riverstonenet.com wrote: > Also, is there a pointer to Ganesh's presentation? I couldn't > find it. > > > > > > > Peram > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed Mar 20 10:16:51 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA01891 for ; Wed, 20 Mar 2002 10:16:51 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16nhUG-0005iY-00 for ipfix-list@mil.doit.wisc.edu; Wed, 20 Mar 2002 08:54:48 -0600 Received: from dplonka by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16nhUC-0005i9-00; Wed, 20 Mar 2002 08:54:44 -0600 Date: Wed, 20 Mar 2002 08:54:41 -0600 From: Dave Plonka To: ipfix@net.doit.wisc.edu Cc: Randy Bush , Bert Wijnen Subject: [ipfix] DRAFT IPFIX WG minutes, IETF53 Message-ID: <20020320085441.A21562@doit.wisc.edu> Reply-To: plonka@doit.wisc.edu Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0.1i Organization: UW-Madison, DoIT, Network Services X-VMS-Error: %SYSTEM-E-SUBSTRERR, substring error, PC=00000000, PS=00000560 X-Shakespearean-Insult: Thou vain common-kissing pigeon-egg Precedence: bulk Sender: majordomo listserver IPFIX folks, Please review the DRAFT minutes of the most recent meeting, below. Please send any comments/suggestions/corrections to the list. Thanks, Dave P.S. The slide shows (and also the draft minutes) are available at: http://ipfix.doit.wisc.edu/IETF53/ ---------------------------------------------------------------------- DRAFT minutes of the IP Flow Information eXport (IPFIX) WG IETF 53, Minneapolis, Monday March 18, 2002 78 people in attendance Reported by Greg Ruth (scribe), Dave Plonka & Nevil Brownlee (co-chairs) The meeting agenda is available here: http://ipfix.doit.wisc.edu/IETF53/IETF53_IPFIX_agenda.sdd http://ipfix.doit.wisc.edu/IETF53/IETF53_IPFIX_agenda.ppt Nevil gave a brief overview of the IPFIX goals, commenting particularly that the working group is not trying to invent anything from scratch and that the IPFIX system must utilize a congestion-aware transport protocol. Regarding the milestones, we have completed our first drafts of architecture, requirements and data model on schedule. Mark Fullmer of OARnet, and author of flow-tools, did a short presentation entitled Abilene NetFlow Deployment, which the co-chairs introduced as an example of a large-scale flow-based measurement system. The presentation is available at: http://ipfix.doit.wisc.edu/IETF53/abilene-netflow.ppt The measurement system currently collects about 350 million flows per day, and processes them using multiple kinds of analysis. Mark noted some security issues included that current NetFlow authentication is based solely on source IP. While spoofed flows can be detected by sequence numbers, this is not often done in practice. Mark's tools can be found here: http://www.splintered.net/sw/flow-tools/ The next item on the agenda was to review the WG documents. 1. IPFIX Applicability - Tanja Zseby, FhG FOKUS http://ipfix.doit.wisc.edu/IETF53/IPFIX-applicability.ppt The chairs mentioned that while this document is not yet listed as a WG milestone, it will be required for making IPFIX a standard. Tanja noted these document objectives: a) show how target applications can use IPFIX; b) describe relations and potential interfaces to other frameworks/working groups (RTFM, IPPM, AAA, etc.) c) the document is expected to address at least these applications: Accounting, traffic profiling, TE, Intrusion detection, QoS Monitoring, Deployment of Sampling in IPFIX The document is available here and will be published as an Internet Draft soon: http://ipfix.doit.wisc.edu/app/ipfix-applicability-v1.txt 2. IPFIX Requirements - Juergen Quittek, NEC http://ipfix.doit.wisc.edu/IETF53/IPFIX-reqs-IETF53.ppt Juergen mentioned the following updates: a) added section 2 on terminology - IP traffic flows, observation point, metering process, flow record, exporting process, collecting process b) replaced "measuring device" by either "metering process", "export process" or "IPFIX device" as appropriate c) removed "network surveillance" section d) added "overload behavior" section There are some changes that need to be done for the next (and probably final) draft: consistency checks, add "observation domain", remove "IPFIX device", rephrase Security Considerations Next version to appear May 1 - expect WG last call at that time 3. IPFIX Information Model (formerly Data Model) - K.C. Norseth http://ipfix.doit.wisc.edu/IETF53/ipfix_data_ietf53.ppt KC's overview consisted of the following points: a) This document was created by a design team: equipment vendors, applications vendors and users b) flow types and flow definitions c) selection criteria for a flow d) packet encoding format e) information elements Outstanding issues include: a) need to clarify more elements b) counters (absolute vs. Delta - may have both) c) source & destination address types Next steps are: a) complete/refine the flow definition b) add missing information elements c) detailed definition of elements (type, size, detailed description) Discussion: Should we have just one document instead of a info/data model and architecture (??) Consesus was that it was reasonable to have one seperate document listing the information elements, which may need to be updated more often than the architecture document. Should some topics be moved out of the data model (back) into the architecture document? As the data model document matures, we may revisit whether the defintion of the flow belongs in this document or in the architecture document. 4. IPFIX Architecture - KC Norseth http://ipfix.doit.wisc.edu/IETF53/ipfix_arch_ietf53.ppt The architecture document was also developed by the design team. KC presented an overview of the elements of an architecture including: collector, IPFIX device, etc. The IPFIX Protocol as implemented in the exporting device will encode the control information into self-describing structures, encode the flows measured, and packetize the flow records. It will use the underlying transport layer to send the export packets to the collector. The IPFIX Protocol in the collecting device will receive and store the control information, decode and store flow records using the control information. The architecture document currently refers to separate control and data streams. The control stream being a reliable path to exchange control information, monitor connectivity and other failures, and the data stream to send the IPFIX export payload itself. At this point, a number of attendees took issue with the use of the word "control" because it can be confusing. In response it was noted that, in this context, "control [information]" refers to the portion of IPFIX which communicates the structure of the exported data. Consensus was that both the words "control" and "stream" are inappropriate. "Data description" was suggested as an alternative. [Note that the architure document currently contains a section on "Selection Criteria" for the IPFIX application-level protocol. This section will ultimately be replaced by one describing the IPFIX protocol, as the result of the selection process (see below).] KC mentioned that the co-chairs said that all the references to UDP as transport must be removed from the architecture document. This triggered a discussion about transport protocols. The result of that was that it is IETF consensus (RFC 2914), not just an IESG policy, that standard protocols must use congestion-aware transport. The proposed DCP transport protocol may be satisfactory (since it accommodates congestion handling). The transport area director noted that good implementations of TCP and SCTP should be perfectly satisfactory for our purposes at this time. It was asked what "reliability" means for the IPFIX application-level protocol. In response, it was noted that IPFIX's reliability requirement isn't necessarily satisfied by simply using a reliable transport protocol. Reliability is a multifaceted issue. Different degrees of reliability for different kinds of records. It was suggested that IPFIX consider the Reliable Server Pooling WG effort and its documents: http://www.ietf.org/html.charters/rserpool-charter.html Regarding capability negotiation, consensus amongst attendees (by a hum) was that neither exporter configuration nor capabilities negotiation between collectors and exporters should not be addressed by IPFIX. The chairs will follow-up with a call on the mailing list on this topic. Also, the chairs said that explicit references to template-based application-level protocols need to be removed from the architecture document, since the IPFIX requirements do not exclude other candidate self-describing data encoding methods. The chairs turned the discussion to WG process issues, specifically they proposed an evaluation and selection process for the IPFIX application-level protocol. The proposal is that we will have an "evaluation team" consisting of an odd number of people having experience in disciplines including router hardware, probes, application software, and end use of flow-based measurements. The chairs will select the members from among the interested respondents. The evaluation team will consider candidate protocols which (a) have a working implementation, (b) are documented by an Internet Draft or RFC, and (c) have a IPFIX participant advocating the selection of that protocol and willing to produce a brief document, such as an I-D, explaining how the protocol meets the selection criteria. The evaluation team will interact with the advocates by responding to the advocacy documents with lists of issues/deficiencies and may ask additional questions of the advocates. They will also produce a report according within a WG-defined timeframe proposing their selection and identifying suggested improvements as necessary. The evaluation report will be discussed and approved by mailing list consensus. Subsequent discussion did not offer any improvements to the proposed process. In reponse to a question about whether multiple IPFIX application protocols could be selected, the chairs said that the goal is to settle on a single protocol and that protocols unencumbered by intellectual property issues will be preferred. Lastly, WG milestones were reviewed. The editors of each document agreed to produce their next revision by May 1. The chairs plan to solicit for candidate protocol advocates and appoint an evaluation team by May 1, and call for advocacy reports by June 1. The evaluation team will present a status report at the next IETF WG meeting in July. -- $Id: minutes.txt,v 1.1 2002/03/20 14:36:33 dplonka Exp $ -- plonka@doit.wisc.edu http://net.doit.wisc.edu/~plonka ARS:N9HZF Madison, WI -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed Mar 20 10:46:23 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA02744 for ; Wed, 20 Mar 2002 10:46:22 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16nhxH-0006Sg-00 for ipfix-list@mil.doit.wisc.edu; Wed, 20 Mar 2002 09:24:47 -0600 Received: from ctron-dnm.enterasys.com ([12.25.1.120] helo=ctron-dnm.ctron.com ident=firewall-user) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16nhxF-0006SZ-00 for ipfix@net.doit.wisc.edu; Wed, 20 Mar 2002 09:24:45 -0600 Received: (from uucp@localhost) by ctron-dnm.ctron.com (8.8.7/8.8.7) id KAA08630 for ; Wed, 20 Mar 2002 10:34:33 -0500 (EST) Received: from unknown(134.141.77.96) by ctron-dnm.ctron.com via smap (4.1) id xma008608; Wed, 20 Mar 02 10:33:59 -0500 Received: by cnc-exc1.enterasys.com with Internet Mail Service (5.5.2653.19) id ; Wed, 20 Mar 2002 10:22:29 -0500 Message-ID: <59358A738F45D51186A30008C74CE250DA07F1@slc-exc1.ctron.com> From: "Norseth, KC" To: ipfix@net.doit.wisc.edu Subject: [ipfix] Selection Criteria Date: Wed, 20 Mar 2002 10:21:49 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C1D022.F45AA050" Precedence: bulk Sender: majordomo listserver This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C1D022.F45AA050 Content-Type: text/plain Hello folks, As we discussed the selection criteria for evaluating the protocols, I would like more input. If there is some more selection criteria that we need to have that has not been addressed, mail it to the group. One criteria that needs to be added for example, is the Intellectual property rights. Simply stated, if the protocol owner is not willing to stick to the IETF IPR, we will not consider it. Remember, if the protocol cannot comply by one of the selection criteria, it is not down and out, it is a checkmark against it. It is still possible this could be something that the protocol can be tweaked to make it conform or if it is something that just cannot be met. My goal is to have the next draft ready for submission at the end of next week. This week or early next week is when I want the input. Changes that came from the conference will be included in this release. K.C. ------_=_NextPart_001_01C1D022.F45AA050 Content-Type: text/html Content-Transfer-Encoding: quoted-printable Selection Criteria

Hello folks,

As we discussed the selection criteria for evaluating = the protocols, I would like more input.  If there is some more = selection criteria that we need to have that has not been addressed, = mail it to the group.  One criteria that needs to be added for = example, is the Intellectual property rights. Simply stated, if the = protocol owner is not willing to stick to the IETF IPR, we will not = consider it.  Remember, if the protocol cannot comply by one of = the selection criteria, it is not down and out, it is a checkmark = against it.  It is still possible this could be something that the = protocol can be tweaked to make it conform or if it is something that = just cannot be met. 

My goal is to have the next draft ready for = submission at the end of next week.  This week or early next week = is when I want the input.  Changes that came from the conference = will be included in this release.

K.C.

------_=_NextPart_001_01C1D022.F45AA050-- -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Wed Mar 20 11:02:33 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA03318 for ; Wed, 20 Mar 2002 11:02:32 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16niFV-0006sf-00 for ipfix-list@mil.doit.wisc.edu; Wed, 20 Mar 2002 09:43:37 -0600 Received: from mailhost.auckland.ac.nz ([130.216.191.4]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16niFT-0006sY-00 for ipfix@net.doit.wisc.edu; Wed, 20 Mar 2002 09:43:36 -0600 Received: from auckland.ac.nz (bluebottle.itss.auckland.ac.nz [130.216.4.28]) by mailhost.auckland.ac.nz (8.9.2/8.9.2/8.9.2-ua) with ESMTP id DAA23255; Thu, 21 Mar 2002 03:43:25 +1200 (NZST) Message-Id: <200203201543.DAA23255@mailhost.auckland.ac.nz> Date: Wed, 20 Mar 2002 07:44:59 -0800 (PST) From: n.brownlee@auckland.ac.nz Subject: Re: [ipfix] DRAFT IPFIX WG minutes, IETF53 To: plonka@doit.wisc.edu cc: ipfix@net.doit.wisc.edu In-Reply-To: <20020320085441.A21562@doit.wisc.edu> MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Precedence: bulk Sender: majordomo listserver Oops, one typo in the minutes: > Regarding capability negotiation, consensus amongst attendees (by a > hum) was that neither exporter configuration nor capabilities > negotiation between collectors and exporters should not be addressed delete this 'not' ^^^ > by IPFIX. The chairs will follow-up with a call on the mailing > list on this topic. Cheers, Nevil ----------------------------------------------------------------------- Nevil Brownlee Director, Technology Development Phone: +64 9 373 7599 x8941 ITSS, The University of Auckland FAX: +64 9 373 7021 Private Bag 92019, Auckland, New Zealand -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu Mar 21 12:59:48 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA20145 for ; Thu, 21 Mar 2002 12:59:47 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16o6MY-0003zr-00 for ipfix-list@mil.doit.wisc.edu; Thu, 21 Mar 2002 11:28:30 -0600 Received: from noc-e.ietf53.cw.net ([166.63.177.40]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16o6MW-0003zk-00 for ipfix@net.doit.wisc.edu; Thu, 21 Mar 2002 11:28:28 -0600 Received: from BETA (BETA.ietf53.cw.net [166.63.187.167]) by noc-e.ietf53.cw.net (Postfix) with ESMTP id CC9B46A911 for ; Thu, 21 Mar 2002 11:28:27 -0600 (CST) Date: Thu, 21 Mar 2002 18:31:09 +0100 From: Juergen Quittek To: ipfix@net.doit.wisc.edu Subject: [ipfix] Terminology: metering process Message-ID: <91720907.1016735469@BETA> X-Mailer: Mulberry/2.1.2 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Tanja and myself have discussed the definition of the term 'metering process', particularly considering hierarchical systems as the one Ganesh contributed to the architecture (draft-ietf-ipfix-architecture-00.txt, page 5) and as the ones we showed on the requirement slides at Minneapolis (http://ipfix.doit.wisc.edu/IETF53/IPFIX-reqs-IETF53.ppt, page 10). The old definition in the draft-ietf-ipfix-reqs-01.txt is: 2.3. Metering Process A metering process is a set of actions performed on packets observed at an observation point in order to map them to a flow. A metering process can include functions for timestamping, classification and sampling of packets. Typically, the metering process also includes maintainance of flow records, computation of flow statistics, and detection of flow expiration. We propose to replace it by the following definition: 2.3. Metering Process The metering process generates flow records. Input to the process are IP packets observed at an observation point. The metering process consists of a set of functions that includes packet header capturing, timestamping, sampling, classifying, and maintaining flow records. The maintenance of flow records may include creating new records, updating existing ones, computing flow statistics, deriving further flow properties, detecting flow expiration, passing flows record to the exporting process, and deleting flow records. The sampling function and the classifying function may be realized as trivial functions, for example one-in-one sampling or mapping all packets to the same single class. They may be applied more than once with different parameters. The figure below shows the sequence in which the functions are applied. packet header capturing | timestamping | v +----->+ | | | sampling | | | classifying | | +------+ | maintaining flow records | v Figure 1: Functions of the metering process Cheers, Juergen -- Juergen Quittek quittek@ccrle.nec.de Tel: +49 6221 90511-15 NEC Europe Ltd., Network Laboratories Fax: +49 6221 90511-55 Adenauerplatz 6, 69115 Heidelberg, Germany http://www.ccrle.nec.de -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu Mar 21 14:26:30 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA22464 for ; Thu, 21 Mar 2002 14:26:30 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16o7uz-0006B2-00 for ipfix-list@mil.doit.wisc.edu; Thu, 21 Mar 2002 13:08:09 -0600 Received: from sj-msg-core-1.cisco.com ([171.71.163.11]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16o7ux-0006A8-00 for ipfix@net.doit.wisc.edu; Thu, 21 Mar 2002 13:08:07 -0600 Received: from mira-sjcd-1.cisco.com (mira-sjcd-1.cisco.com [171.69.43.44]) by sj-msg-core-1.cisco.com (8.11.3/8.9.1) with ESMTP id g2LJ7Yd28985; Thu, 21 Mar 2002 11:07:34 -0800 (PST) Received: from cisco.com (dhcp-171-71-137-115.cisco.com [171.71.137.115]) by mira-sjcd-1.cisco.com (Mirapoint) with ESMTP id ABQ29113; Thu, 21 Mar 2002 11:07:59 -0800 (PST) Message-ID: <3C9A2F70.F30B7308@cisco.com> Date: Thu, 21 Mar 2002 11:07:28 -0800 From: Ganesh Sadasivan X-Mailer: Mozilla 4.76 [en]C-CCK-MCD (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: Juergen Quittek CC: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Terminology: metering process References: <91720907.1016735469@BETA> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Hi Jurgene, See inline. Juergen Quittek wrote: > Tanja and myself have discussed the definition of the term > 'metering process', particularly considering hierarchical > systems as the one Ganesh contributed to the architecture > (draft-ietf-ipfix-architecture-00.txt, page 5) and as the > ones we showed on the requirement slides at Minneapolis (http://ipfix.doit.wisc.edu/IETF53/IPFIX-reqs-IETF53.ppt, page 10). > > The old definition in the draft-ietf-ipfix-reqs-01.txt is: > > 2.3. Metering Process > A metering process is a set of actions performed on packets observed > at an observation point in order to map them to a flow. A metering > process can include functions for timestamping, classification and > sampling of packets. Typically, the metering process also includes > maintainance of flow records, computation of flow statistics, and > detection of flow expiration. > > We propose to replace it by the following definition: > > 2.3. Metering Process > The metering process generates flow records. Input to the process > are IP packets observed at an observation point. The metering process > consists of a set of functions that includes packet header capturing, > timestamping, sampling, classifying, and maintaining flow records. > > The maintenance of flow records may include creating new records, > updating existing ones, computing flow statistics, deriving further > flow properties, detecting flow expiration, passing flows record to > the exporting process, and deleting flow records. > > The sampling function and the classifying function may be realized as Don't we need to define sampling & classifying before refering to it here? Looks a little confusing to me. > > trivial functions, for example one-in-one sampling or mapping all packets What do you mean by "trivial functions"? Aren't Sampling & classifying optional. In the figure don't you need a direct path from "timestamping" to "maintaining flow records" ? Thanks Ganesh > > to the same single class. They may be applied more than once with different > parameters. The figure below shows the sequence in which the functions > are applied. > > packet header capturing > | > timestamping > | > v > +----->+ > | | > | sampling > | | > | classifying > | | > +------+ > | > maintaining flow records > | > v > > Figure 1: Functions of the metering process > > Cheers, > > Juergen > -- > Juergen Quittek quittek@ccrle.nec.de Tel: +49 6221 90511-15 > NEC Europe Ltd., Network Laboratories Fax: +49 6221 90511-55 > Adenauerplatz 6, 69115 Heidelberg, Germany http://www.ccrle.nec.de > > -- > Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > "unsubscribe ipfix" in message body > Archive http://ipfix.doit.wisc.edu/archive/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu Mar 21 18:00:08 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA28830 for ; Thu, 21 Mar 2002 18:00:08 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16oB31-0002bi-00 for ipfix-list@mil.doit.wisc.edu; Thu, 21 Mar 2002 16:28:40 -0600 Received: from noc-e.ietf53.cw.net ([166.63.177.40]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16oB2z-0002bc-00 for ipfix@net.doit.wisc.edu; Thu, 21 Mar 2002 16:28:37 -0600 Received: from BETA (BETA.ietf53.cw.net [166.63.187.167]) by noc-e.ietf53.cw.net (Postfix) with ESMTP id B34E86A90E; Thu, 21 Mar 2002 16:28:31 -0600 (CST) Date: Thu, 21 Mar 2002 23:31:13 +0100 From: Juergen Quittek To: Ganesh Sadasivan Cc: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Terminology: metering process Message-ID: <109725116.1016753473@BETA> In-Reply-To: <3C9A2F70.F30B7308@cisco.com> References: <3C9A2F70.F30B7308@cisco.com> X-Mailer: Mulberry/2.1.2 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Ganesh, --On 21 March 2002 11:07 -0800 Ganesh Sadasivan wrote: > Hi Jurgene, > See inline. > > Juergen Quittek wrote: [...] >> The sampling function and the classifying function may be realized as > > Don't we need to define sampling & classifying before refering to it here? > Looks a little confusing to me. Concerning packet sampling I am not sure, because this might be sufficiently understood term already. Concerning classifying I agree. >> >> trivial functions, for example one-in-one sampling or mapping all packets > > What do you mean by "trivial functions"? Aren't Sampling & classifying optional. OK, we have two alternatives of phrasing: either we declare them optional or we say they may degenerate to trivial functions. The effect - at least for sampling - would be the same. What would be your preference? > In the figure don't you need a direct path from "timestamping" to > "maintaining flow records" ? How should this work? we first need to sample and classify before we know which flow record to update. Maybe we should explain timestamping as 'adding a timestamp to the captured packet header'? Cheers, Juergen > > Thanks > Ganesh > >> >> to the same single class. They may be applied more than once with different >> parameters. The figure below shows the sequence in which the functions >> are applied. >> >> packet header capturing >> | >> timestamping >> | >> v >> +----->+ >> | | >> | sampling >> | | >> | classifying >> | | >> +------+ >> | >> maintaining flow records >> | >> v >> >> Figure 1: Functions of the metering process >> >> Cheers, >> >> Juergen >> -- >> Juergen Quittek quittek@ccrle.nec.de Tel: +49 6221 90511-15 >> NEC Europe Ltd., Network Laboratories Fax: +49 6221 90511-55 >> Adenauerplatz 6, 69115 Heidelberg, Germany http://www.ccrle.nec.de >> >> -- >> Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body >> Unsubscribe mailto:majordomo@net.doit.wisc.edu and say >> "unsubscribe ipfix" in message body >> Archive http://ipfix.doit.wisc.edu/archive/ > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri Mar 22 13:06:43 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA27672 for ; Fri, 22 Mar 2002 13:06:43 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16oT39-0004XN-00 for ipfix-list@mil.doit.wisc.edu; Fri, 22 Mar 2002 11:41:59 -0600 Received: from sj-msg-core-2.cisco.com ([171.69.24.11]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16oT37-0004Wn-00 for ipfix@net.doit.wisc.edu; Fri, 22 Mar 2002 11:41:57 -0600 Received: from mira-sjcd-1.cisco.com (mira-sjcd-1.cisco.com [171.69.43.44]) by sj-msg-core-2.cisco.com (8.11.3/8.9.1) with ESMTP id g2MHfQY24916; Fri, 22 Mar 2002 09:41:26 -0800 (PST) Received: from cisco.com (dhcp-171-71-137-115.cisco.com [171.71.137.115]) by mira-sjcd-1.cisco.com (Mirapoint) with ESMTP id ABQ59248; Fri, 22 Mar 2002 09:41:57 -0800 (PST) Message-ID: <3C9B6CC6.3C6C06AE@cisco.com> Date: Fri, 22 Mar 2002 09:41:26 -0800 From: Ganesh Sadasivan X-Mailer: Mozilla 4.76 [en]C-CCK-MCD (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: Juergen Quittek CC: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Terminology: metering process References: <3C9A2F70.F30B7308@cisco.com> <109725116.1016753473@BETA> Content-Type: multipart/alternative; boundary="------------1E16AF2331F07C591E1CB000" Precedence: bulk Sender: majordomo listserver --------------1E16AF2331F07C591E1CB000 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Jurgene, Juergen Quittek wrote: > Ganesh, > > --On 21 March 2002 11:07 -0800 Ganesh Sadasivan wrote: > > > Hi Jurgene, > > See inline. > > > > Juergen Quittek wrote: > [...] > >> The sampling function and the classifying function may be realized as > > > > Don't we need to define sampling & classifying before refering to it here? > > Looks a little confusing to me. > > Concerning packet sampling I am not sure, because this might be > sufficiently understood term already. Concerning classifying I agree. The packet headers are subjected to one of or a combination of classification based on some criteria and sampling based on some algorithm.We can just mention the special cases in the figure. This can can effective represent any combination of Si & Fi defined in the data model. packet header capturing | timestamping | v +----->+ | | | sampling (1:1 in case of no sampling) | | | classification (NULL when No criteria) | | +------+ | maintaining flow records | v Does this look ok to you? Thanks Ganesh > > > > >> > >> trivial functions, for example one-in-one sampling or mapping all packets > > > > What do you mean by "trivial functions"? Aren't Sampling & classifying optional. > > OK, we have two alternatives of phrasing: either we declare them optional > or we say they may degenerate to trivial functions. The effect - at least > for sampling - would be the same. What would be your preference? > > > In the figure don't you need a direct path from "timestamping" to > > "maintaining flow records" ? > > How should this work? we first need to sample and classify before > we know which flow record to update. Maybe we should explain timestamping > as 'adding a timestamp to the captured packet header'? > > Cheers, > > Juergen > > > > > Thanks > > Ganesh > > > >> > >> to the same single class. They may be applied more than once with different > >> parameters. The figure below shows the sequence in which the functions > >> are applied. > >> > >> packet header capturing > >> | > >> timestamping > >> | > >> v > >> +----->+ > >> | | > >> | sampling > >> | | > >> | classifying > >> | | > >> +------+ > >> | > >> maintaining flow records > >> | > >> v > >> > >> Figure 1: Functions of the metering process > >> > >> Cheers, > >> > >> Juergen > >> -- > >> Juergen Quittek quittek@ccrle.nec.de Tel: +49 6221 90511-15 > >> NEC Europe Ltd., Network Laboratories Fax: +49 6221 90511-55 > >> Adenauerplatz 6, 69115 Heidelberg, Germany http://www.ccrle.nec.de > >> > >> -- > >> Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body > >> Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > >> "unsubscribe ipfix" in message body > >> Archive http://ipfix.doit.wisc.edu/archive/ > > --------------1E16AF2331F07C591E1CB000 Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7bit  
Jurgene,

Juergen Quittek wrote:

Ganesh,

--On 21 March 2002 11:07 -0800 Ganesh Sadasivan <gsadasiv@cisco.com> wrote:

> Hi Jurgene,
>   See inline.
>
> Juergen Quittek wrote:
[...]
>>    The sampling function and the classifying function may be realized as
>
> Don't we need to define sampling & classifying before refering to it here?
> Looks a little confusing to me.

Concerning packet sampling I am not sure, because this might be
sufficiently understood term already. Concerning classifying I agree.

The packet headers are subjected to one of or a combination of
classification based on some criteria and sampling based on some
algorithm.We can just mention the special cases in the figure.
This can can effective represent any combination of Si & Fi defined
in the data model.

                      packet header capturing
                                 |
                         timestamping
                                 |
                                 v
                          +----->+
                          |      |
                          |   sampling (1:1 in case of no sampling)
                          |      |
                          | classification (NULL when No criteria)
                          |      |
                          +------+
                                 |
                     maintaining flow records
                                 |
                                 v
Does this look ok to you?
Thanks
Ganesh

 
 

>>
>>    trivial functions, for example one-in-one sampling or mapping all packets
>
> What do you mean by "trivial functions"? Aren't Sampling & classifying optional.

OK, we have two alternatives of phrasing: either we declare them optional
or we say they may degenerate to trivial functions. The effect - at least
for sampling - would be the same. What would be your preference?

> In the figure don't you need a direct path from "timestamping" to
> "maintaining flow records" ?

How should this work? we first need to sample and classify before
we know which flow record to update. Maybe we should explain timestamping
as 'adding a timestamp to the captured packet header'?

Cheers,

    Juergen

>
> Thanks
> Ganesh
>
>>
>>    to the same single class. They may be applied more than once with different
>>    parameters. The figure below shows the sequence in which the functions
>>    are applied.
>>
>>                         packet header capturing
>>                                   |
>>                              timestamping
>>                                   |
>>                                   v
>>                            +----->+
>>                            |      |
>>                            |   sampling
>>                            |      |
>>                            | classifying
>>                            |      |
>>                            +------+
>>                                   |
>>                        maintaining flow records
>>                                   |
>>                                   v
>>
>>               Figure 1: Functions of the metering process
>>
>> Cheers,
>>
>>     Juergen
>> --
>> Juergen Quittek     quittek@ccrle.nec.de     Tel: +49 6221 90511-15
>> NEC Europe Ltd.,    Network Laboratories     Fax: +49 6221 90511-55
>> Adenauerplatz 6, 69115 Heidelberg, Germany   http://www.ccrle.nec.de
>>
>> --
>> Help        mailto:majordomo@net.doit.wisc.edu and say "help" in message body
>> Unsubscribe mailto:majordomo@net.doit.wisc.edu and say
>> "unsubscribe ipfix" in message body
>> Archive     http://ipfix.doit.wisc.edu/archive/
>

--------------1E16AF2331F07C591E1CB000-- -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri Mar 22 15:26:34 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA01014 for ; Fri, 22 Mar 2002 15:26:34 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16oVNZ-0006ER-00 for ipfix-list@mil.doit.wisc.edu; Fri, 22 Mar 2002 14:11:13 -0600 Received: from odd-brew.cisco.com ([144.254.15.119] helo=strange-brew.cisco.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16oVNW-0006DX-00 for ipfix@net.doit.wisc.edu; Fri, 22 Mar 2002 14:11:10 -0600 Received: from cisco.com (sbhardwa-w2k-1.cisco.com [128.107.163.25]) by strange-brew.cisco.com (8.8.8+Sun/8.8.8) with ESMTP id VAA28375; Fri, 22 Mar 2002 21:07:56 +0100 (MET) Message-ID: <3C9B8F1B.39EB2B77@cisco.com> Date: Fri, 22 Mar 2002 21:07:55 +0100 From: Benoit Claise X-Mailer: Mozilla 4.76 [en]C-CCK-MCD (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: "Norseth, KC" CC: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Selection Criteria References: <59358A738F45D51186A30008C74CE250DA07F1@slc-exc1.ctron.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Hi, 2 more selection criterias should be: - extensible in terms of information model - flexible, i.e. export of self describing structure (the keyword template being banned) Regards, Benoit "Norseth, KC" wrote: > > > Hello folks, > > As we discussed the selection criteria for evaluating the protocols, I would > like more input. If there is some more selection criteria that we need to > have that has not been addressed, mail it to the group. One criteria that > needs to be added for example, is the Intellectual property rights. Simply > stated, if the protocol owner is not willing to stick to the IETF IPR, we will > not consider it. Remember, if the protocol cannot comply by one of the > selection criteria, it is not down and out, it is a checkmark against it. It > is still possible this could be something that the protocol can be tweaked to > make it conform or if it is something that just cannot be met. > > My goal is to have the next draft ready for submission at the end of next > week. This week or early next week is when I want the input. Changes that > came from the conference will be included in this release. > > K.C. -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri Mar 22 15:54:07 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA01518 for ; Fri, 22 Mar 2002 15:54:07 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16oVoY-0006nw-00 for ipfix-list@mil.doit.wisc.edu; Fri, 22 Mar 2002 14:39:06 -0600 Received: from c001-h001.c001.snv.cp.net ([209.228.32.115] helo=c001.snv.cp.net) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 16oVoW-0006nj-00 for ipfix@net.doit.wisc.edu; Fri, 22 Mar 2002 14:39:04 -0600 Received: (cpmta 19918 invoked from network); 22 Mar 2002 12:38:32 -0800 Received: from 24.221.253.53 (HELO kcn) by smtp.register-admin.com (209.228.32.115) with SMTP; 22 Mar 2002 12:38:32 -0800 X-Sent: 22 Mar 2002 20:38:32 GMT Message-ID: <00b701c1d1e1$ada05fe0$850f880a@kcn> From: "K.C. Norseth" To: "Benoit Claise" Cc: References: <59358A738F45D51186A30008C74CE250DA07F1@slc-exc1.ctron.com> <3C9B8F1B.39EB2B77@cisco.com> Subject: Re: [ipfix] Selection Criteria Date: Fri, 22 Mar 2002 13:39:34 -0700 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Got it. We will want to be more precise on those 2 items. If I miss on the wording, you can correct it. I am not sure what you mean the keyword template being banned, you mean ban it or what? This was discussed and we need to be careful on how we use the word. It can mean we pass the record definition. Template can still be used and not pass the structure layout. Ganesh and I were going to talk on Tuesday to work on the next rev, including all the things discussed in the meeting. By Friday, I want to have encorporated everything from the conference, as well as these things. Friday, we will have something for the group to see. K.C. ----- Original Message ----- From: "Benoit Claise" To: "Norseth, KC" Cc: Sent: Friday, March 22, 2002 1:07 PM Subject: Re: [ipfix] Selection Criteria | Hi, | | 2 more selection criterias should be: | - extensible in terms of information model | - flexible, i.e. export of self describing structure (the keyword template being | banned) | | Regards, Benoit | | "Norseth, KC" wrote: | | > | > | > Hello folks, | > | > As we discussed the selection criteria for evaluating the protocols, I would | > like more input. If there is some more selection criteria that we need to | > have that has not been addressed, mail it to the group. One criteria that | > needs to be added for example, is the Intellectual property rights. Simply | > stated, if the protocol owner is not willing to stick to the IETF IPR, we will | > not consider it. Remember, if the protocol cannot comply by one of the | > selection criteria, it is not down and out, it is a checkmark against it. It | > is still possible this could be something that the protocol can be tweaked to | > make it conform or if it is something that just cannot be met. | > | > My goal is to have the next draft ready for submission at the end of next | > week. This week or early next week is when I want the input. Changes that | > came from the conference will be included in this release. | > | > K.C. | | | -- | Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body | Unsubscribe mailto:majordomo@net.doit.wisc.edu and say | "unsubscribe ipfix" in message body | Archive http://ipfix.doit.wisc.edu/archive/ | -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Fri Mar 22 20:45:28 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA05549 for ; Fri, 22 Mar 2002 20:45:27 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16oaMY-0005Eu-00 for ipfix-list@mil.doit.wisc.edu; Fri, 22 Mar 2002 19:30:30 -0600 Received: from odd-brew.cisco.com ([144.254.15.119] helo=strange-brew.cisco.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16oaMV-0005Dj-00 for ipfix@net.doit.wisc.edu; Fri, 22 Mar 2002 19:30:27 -0600 Received: from cisco.com (dhcp-171-71-137-58.cisco.com [171.71.137.58]) by strange-brew.cisco.com (8.8.8+Sun/8.8.8) with ESMTP id CAA26678; Sat, 23 Mar 2002 02:29:45 +0100 (MET) Message-ID: <3C9BDA7F.AA1CF954@cisco.com> Date: Sat, 23 Mar 2002 02:29:35 +0100 From: Benoit Claise X-Mailer: Mozilla 4.76 [en]C-CCK-MCD (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: "K.C. Norseth" CC: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Selection Criteria References: <59358A738F45D51186A30008C74CE250DA07F1@slc-exc1.ctron.com> <3C9B8F1B.39EB2B77@cisco.com> <00b701c1d1e1$ada05fe0$850f880a@kcn> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit "K.C. Norseth" wrote: > Got it. We will want to be more precise on those 2 items. If I miss on the > wording, you can correct it. > I am not sure what you mean the keyword template being banned, you mean ban > it or what? This was discussed and we need to be careful on how we use the > word. Well, when I used the "template" keyword during the WG meeting, I was told that I could not say that one of the selection criteria must be: based on template. The given reason was: template is implementation specific, which I beg to differ... Because when I look at the template definition from the architecture draft, it doesn't refer to any specific implementation. Regards, Benoit > It can mean we pass the record definition. Template can still be > used and not pass the structure layout. > > Ganesh and I were going to talk on Tuesday to work on the next rev, > including all the things discussed in the meeting. By Friday, I want to > have encorporated everything from the conference, as well as these things. > Friday, we will have something for the group to see. > > K.C. > > ----- Original Message ----- > From: "Benoit Claise" > To: "Norseth, KC" > Cc: > Sent: Friday, March 22, 2002 1:07 PM > Subject: Re: [ipfix] Selection Criteria > > | Hi, > | > | 2 more selection criterias should be: > | - extensible in terms of information model > | - flexible, i.e. export of self describing structure (the keyword template > being > | banned) > | > | Regards, Benoit > | > | "Norseth, KC" wrote: > | > | > > | > > | > Hello folks, > | > > | > As we discussed the selection criteria for evaluating the protocols, I > would > | > like more input. If there is some more selection criteria that we need > to > | > have that has not been addressed, mail it to the group. One criteria > that > | > needs to be added for example, is the Intellectual property rights. > Simply > | > stated, if the protocol owner is not willing to stick to the IETF IPR, > we will > | > not consider it. Remember, if the protocol cannot comply by one of the > | > selection criteria, it is not down and out, it is a checkmark against > it. It > | > is still possible this could be something that the protocol can be > tweaked to > | > make it conform or if it is something that just cannot be met. > | > > | > My goal is to have the next draft ready for submission at the end of > next > | > week. This week or early next week is when I want the input. Changes > that > | > came from the conference will be included in this release. > | > > | > K.C. > | > | > | -- > | Help mailto:majordomo@net.doit.wisc.edu and say "help" in message > body > | Unsubscribe mailto:majordomo@net.doit.wisc.edu and say > | "unsubscribe ipfix" in message body > | Archive http://ipfix.doit.wisc.edu/archive/ > | -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu Mar 28 21:26:00 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA21914 for ; Thu, 28 Mar 2002 21:26:00 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16qlm1-0001bY-00 for ipfix-list@mil.doit.wisc.edu; Thu, 28 Mar 2002 20:05:49 -0600 Received: from tokyo.ccrle.nec.de ([195.37.70.2]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16qllx-0001ao-00 for ipfix-arch@net.doit.wisc.edu; Thu, 28 Mar 2002 20:05:46 -0600 Received: from wallace.heidelberg.ccrle.nec.de (root@wallace.heidelberg.ccrle.nec.de [192.168.102.1]) by tokyo.ccrle.nec.de (8.11.6/8.11.6) with ESMTP id g2T25DQ25453; Fri, 29 Mar 2002 03:05:13 +0100 (CET) (envelope-from quittek@ccrle.nec.de) Received: from [192.168.102.31] ([192.168.102.31]) by wallace.heidelberg.ccrle.nec.de (8.9.3/8.9.3/SuSE Linux 8.9.3-0.1) with ESMTP id DAA32554; Fri, 29 Mar 2002 03:05:10 +0100 Date: Fri, 29 Mar 2002 03:08:00 +0100 From: Juergen Quittek To: knorseth@enterasys.com cc: ipfix-arch@net.doit.wisc.edu Subject: [ipfix-arch] revised terminology Message-ID: <12205450.1017371280@[192.168.102.31]> X-Mailer: Mulberry/2.1.2 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Hi KC, Here is a revised version of the terminology for the architecture document (as promised in Minneapolis). Most of the changes result from discussions with Tanja. The definitions of IPFIX device, template, control stream and data stream are unchanged, although I am not yet completely satisfied with them. Further discussion on all terms seems to be ahead. Cheers, Juergen -- Juergen Quittek quittek@ccrle.nec.de Tel: +49 6221 90511-15 NEC Europe Ltd., Network Laboratories Fax: +49 6221 90511-55 Adenauerplatz 6, 69115 Heidelberg, Germany http://www.ccrle.nec.de Terminology * IP Traffic Flows A flow is defined as a set of packets passing an observation domain in the network during a certain time interval. All packets belonging to a particular flow have a set of common properties. Each property is defined as the result of applying a function to the values of: 1. one or more of packet header fields (eg. destination IP address) 2. one or more properties of the packet itself (eg. packet length) 3. one or more of fields derived from packet treatment (eg. AS number) A packet is defined to belong to a flow if it completely satisfies all the defined properties of the flow. This definition covers the range from a flow containing all packets observed at a network interface to a flow consisting of just a single packet between two applications with a specific sequence number. Please note that the flow definition does not match a general application-level end-to-end stream. However, an application may derive properties of application-level streams by processing measured flow data. * Observation Point The observation point is a location in the network where IP packets can be observed. Examples are a line to which a probe is attached, a shared medium, such as an Ethernet-based LAN, a single port of a router, or the routing engine of a router. * Observation Domain A single IP traffic flow may contain packets observed at several different observation points, for example at several ports of a line card or at different probes. The observation domain of a flow is the set of observation points at which the packets of this flow have been or might have been observed. The minimal observation domain contains just a single observation point. * Metering Process The metering process generates flow records. Input to the process are IP packets observed in an observation domain. The metering process consists of a set of functions that includes packet header capturing, timestamping, sampling, classifying, and maintaining flow records. Sampling and classifying may be performed repeatedly (with different parameters). The sampling function selects a subset of the received set of items (packet headers with timestamps or flow records) to be passed further to the classifying function. The sampling function may be trivial by passing all items, which is equivalent to applying no sampling at all. The classifying function maps its input to flows. There are two extreme classifying functions: one mapping each packet to different flows and one mapping all packets to the same flow. Maintenance of flow records may include creating new records, updating existing ones, computing flow statistics, deriving further flow properties, detecting flow expiration, passing flows record to the exporting process, and deleting flow records. The figure below shows the sequence in which the functions are applied. packet header capturing | timestamping | v +----->+ | | | sampling | | | classifying | | +------+ | maintaining flow records | v Figure 1: Functions of the metering process * Flow Record A flow record contains information about a specific flow that was metered in an observation domain. A flow record contains measured properties of the flow (e.g. the total number of bytes of all packets of the flow) and usually characteristic properties of the flow (e.g. source IP address). * Exporting Process The exporting process sends flow records to one or more collectors. The flow records are generated by one or more metering processes. * Collecting Process The collecting process receives flow records from one or more exporting processes. The collecting process might store received flow record or further process them, but these actions are out of the scope of this document. * IPFIX Device: A device hosting at least an observation point, a metering process and a flow information export process. Typically, corresponding observation point(s), metering process(es), and exporter process(es) are co-located at this device, for example, at a router. * Template: Templates is a set of {type, length} ordered pairs, used to completely identify the structure and semantics of a particular information that needs to be communicated from the IPFIX Device to the collector. Each template is uniquely identified by a Template ID. * Control Stream, Data Stream: The information that needs to be exported from the IPFIX device can be classified into the following categories: - Control Information : This includes the flow type definition, selection criteria for packets within the flow. This is also called as Control Stream. - Flow record : This includes data records corresponding to the various observed flows at each of the observation point. This is also called as Data Stream. -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Thu Mar 28 22:49:26 2002 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id WAA25353 for ; Thu, 28 Mar 2002 22:49:26 -0500 (EST) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 16qnAB-0003TX-00 for ipfix-list@mil.doit.wisc.edu; Thu, 28 Mar 2002 21:34:51 -0600 Received: from tokyo.ccrle.nec.de ([195.37.70.2]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 16qnAA-0003TO-00 for ipfix-arch@net.doit.wisc.edu; Thu, 28 Mar 2002 21:34:50 -0600 Received: from wallace.heidelberg.ccrle.nec.de (root@wallace.heidelberg.ccrle.nec.de [192.168.102.1]) by tokyo.ccrle.nec.de (8.11.6/8.11.6) with ESMTP id g2T3YIQ26351; Fri, 29 Mar 2002 04:34:18 +0100 (CET) (envelope-from quittek@ccrle.nec.de) Received: from [192.168.102.31] ([192.168.102.31]) by wallace.heidelberg.ccrle.nec.de (8.9.3/8.9.3/SuSE Linux 8.9.3-0.1) with ESMTP id EAA00659; Fri, 29 Mar 2002 04:34:16 +0100 Date: Fri, 29 Mar 2002 04:37:07 +0100 From: Juergen Quittek To: knorseth@enterasys.com cc: ipfix-arch@net.doit.wisc.edu Subject: [ipfix-arch] IPFIX devices Message-ID: <17551698.1017376626@[192.168.102.31]> X-Mailer: Mulberry/2.1.2 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Hi KC, below please find a suggestion for a new section (or subsection) of the architecture document discussing IPFIX-related devices. I call them 'IPFIX-related devices' because not all of them match our definiton of 'IPFIX devices' in the terminology section. Maybe we should discuss this definition again based on the list of 'IPFIX-related devices'? Cheers, Juergen -- Juergen Quittek quittek@ccrle.nec.de Tel: +49 6221 90511-15 NEC Europe Ltd., Network Laboratories Fax: +49 6221 90511-55 Adenauerplatz 6, 69115 Heidelberg, Germany http://www.ccrle.nec.de IPFIX-related Devices This section discusses several IPFIX-related devices with different internal structures. They are all composed of one or more of the following elements: observation point (O), full or partial metering process (M), exporting process (E), collecting process (C). +---+ +-----+ +---------+ +---------+ | E-+-> | E--+-> | E----+-> <-+--E E--+-> | | | | | | | / \ | | | | | | M | | M | | M M | | M M | | | | | /|\ | | /|\ /|\ | | /|\ /|\ | | O | | OOO | | OOO OOO | | OOO OOO | +---+ +-----+ +---------+ +---------+ Probe Basic Complex Multiple Router Router Exporters +---+ +---+ +---+ | E-+-> | E-+-> | E-+------------->---+ | | | | | | | | | +---+ +-+-----+ +-+-+ | M | | M | | E-+------->-+-C-M-E-+-> | | | | | | | | | | +---+ +-+-----+ +-+-+ +-+-+ | O | | M | | E-+->---+ | | | | +---+ | | | | | | | M | +-+-+ | O | | M | | | | | | | +---+ | | | +-----+ | O | | O | | O | ->-+-C-E-+-> +---+ +---+ +---+ +-----+ Protocol Remote Concentrator Proxy Converter Observation Figure X: IPFIX-related Devices A very simple device is a probe. It contains on a single observation point, a single metering process, and a single exporting process. For this device the observation domain contains a single observation point only. A basic router extends this structure by multiple observation point. Here an exported flow record may still have an observation domain containing a single observation point. But also Observation domains containing all observation points of the routers are possible. A more complex router may host more than one metering process, for example one per line card. Please note that an observation domain is restricted to the observation points connected to a single metering process. An observation domain containing all observation points of this router is not possible with this structure. Alternatively, a complex router may use different exporting processes for flow records generated by different metering processes. A protocol converter makes use of a metering process that can be accessed only by another protocol than the one defined for IPFIX, for example the SNMP and the Meter MIB module [RFC2720]. Then the exporter receives flow record from a remote metering process and exports these records using the IPFIX protocol. Another choice is remote packet observation. Packet header captured at an observation point may be exported as raw data to a device hosting metering process and exproting process. An intermediate structure between protocol converter and remote observation (not shown in the Figure) would be a split metering process, for example performing timestaping and sampling at the device hosting the observation point and performing packet classification at another device hosting the exporting process. A concentrator receives flow records via the IPFIX protocol, merges them into higher aggregated flows and exports the resulting flows again using the IPFIX protocol. Please note that for the final flow records the observation domain potentially contains observation points at all first level devices. The metering process of the final flow records is composed by the (partial) metering processes at the first level devices and the partial metering process at the concentrator. Finally, a very simple IPFIX-related device is a proxy. It just receives flow records using the IPFIX protocol and sends them further using the same protocol. A proxy might be useful for traversing firewalls or other gateways. -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/