From bclaise@cisco.com Fri Jun 1 04:10:39 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D8E5121F8622 for ; Fri, 1 Jun 2012 04:10:39 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0 X-Spam-Level: X-Spam-Status: No, score=x tagged_above=-999 required=5 tests=[] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id pFbO+tv2dDie for ; Fri, 1 Jun 2012 04:10:39 -0700 (PDT) Received: from av-tac-bru.cisco.com (weird-brew.cisco.com [144.254.15.118]) by ietfa.amsl.com (Postfix) with ESMTP id B256E21F8619 for ; Fri, 1 Jun 2012 04:10:37 -0700 (PDT) X-TACSUNS: Virus Scanned Received: from strange-brew.cisco.com (localhost.cisco.com [127.0.0.1]) by av-tac-bru.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id q51AcPdq020690; Fri, 1 Jun 2012 12:38:26 +0200 (CEST) Received: from [10.60.67.90] (ams-bclaise-8919.cisco.com [10.60.67.90]) by strange-brew.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id q51AcIkw001014; Fri, 1 Jun 2012 12:38:18 +0200 (CEST) Message-ID: <4FC89B99.40107@cisco.com> Date: Fri, 01 Jun 2012 12:38:17 +0200 From: Benoit Claise User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:12.0) Gecko/20120428 Thunderbird/12.0.1 MIME-Version: 1.0 To: "ipfix@ietf.org" , draft-ietf-ipfix-flow-selection-tech@tools.ietf.org References: <4FC74398.50805@cisco.com> In-Reply-To: <4FC74398.50805@cisco.com> Content-Type: multipart/alternative; boundary="------------060801090201060907020606" Cc: ipfix-chairs@tools.ietf.org Subject: [IPFIX] New AD review of draft-ietf-ipfix-flow-selection-tech-10.txt X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Jun 2012 11:10:40 -0000 This is a multi-part message in MIME format. --------------060801090201060907020606 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Dear authors, I'm performing the (new) AD review of draft-ietf-ipfix-flow-selection-tech-10.txt Lucky you, an extra pair of eyes specifically looking at your draft If some points have been discussed already on the mailing list, let me know. I have to admit that I have not been following the latest iterations of this draft. IMHO, this document needs some more work... I don't think that this document is really in line with the other Intermediate Processes documents: http://tools.ietf.org/html/rfc6235 http://tools.ietf.org/html/draft-ietf-ipfix-a9n-03 Note that I might have some more comments once all the points in this email are addressed, as there are many ;-) However, I'm available for a conf. call to clarify my points if you want to See in-line. > Internet Engineering Task Force S. D'Antonio > Internet-Draft University of Napoli > Intended status: Standards Track "Parthenope" > Expires: October 25, 2012 T. Zseby > CAIDA/FhG FOKUS > C. Henke > Tektronix Communication Berlin > L. Peluso > University of Napoli > April 23, 2012 > > > Flow Selection Techniques > draft-ietf-ipfix-flow-selection-tech-11.txt > > Abstract > > Flow selection is the process of selecting a subset of flows from all > observed flows. The Flow Selection Process may be located at an > observation point, or on an IPFIX Mediator. Flow selection reduces > the effort of post-processing flow data and transferring Flow > Records. This document describes motivations for flow selection and > presents flow selection techniques. It provides an information model > for configuring flow selection techniques and discusses what > information about a flow selection process should be exported. - Be consistent with terms capitalization. Flow, Observation Point, for example, are not capitalized in the abstract The following paragraph is good and consistent with the other IPFIX documents, but make sure you include all the terms that you need. This document is consistent with the terminology introduced in [RFC5101], [RFC5470], [RFC5475] and [RFC3917]. As in [RFC5101] and [RFC5476], the first letter of each IPFIX-specific and PSAMP-specific term is capitalized along with the flow selection specific terms defined here. > > Requirements Language > > The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", > "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this > document are to be interpreted as described in RFC 2119 [RFC2119]. > > Status of this Memo > > This Internet-Draft is submitted in full conformance with the > provisions of BCP 78 and BCP 79. > > Internet-Drafts are working documents of the Internet Engineering > Task Force (IETF). Note that other groups may also distribute > working documents as Internet-Drafts. The list of current Internet- > Drafts is at http://datatracker.ietf.org/drafts/current/. > > Internet-Drafts are draft documents valid for a maximum of six months > and may be updated, replaced, or obsoleted by other documents at any > time. It is inappropriate to use Internet-Drafts as reference > material or to cite them other than as "work in progress." > > This Internet-Draft will expire on October 25, 2012. > > > > D'Antonio, et al. Expires October 25, 2012 [Page 1] > > Internet-Draft Flow Selection Techniques April 2012 > > > Copyright Notice > > Copyright (c) 2012 IETF Trust and the persons identified as the > document authors. All rights reserved. > > This document is subject to BCP 78 and the IETF Trust's Legal > Provisions Relating to IETF Documents > (http://trustee.ietf.org/license-info) in effect on the date of > publication of this document. Please review these documents > carefully, as they describe your rights and restrictions with respect > to this document. Code Components extracted from this document must > include Simplified BSD License text as described in Section 4.e of > the Trust Legal Provisions and are provided without warranty as > described in the Simplified BSD License. > > This document may contain material from IETF Documents or IETF > Contributions published or made publicly available before November > 10, 2008. The person(s) controlling the copyright in some of this > material may not have granted the IETF Trust the right to allow > modifications of such material outside the IETF Standards Process. > Without obtaining an adequate license from the person(s) controlling > the copyright in such materials, this document may not be modified > outside the IETF Standards Process, and derivative works of it may > not be created outside the IETF Standards Process, except to format > it for publication as an RFC or to translate it into languages other > than English. > > > > > > > > > > > > > > > > > > > > > > > > > > D'Antonio, et al. Expires October 25, 2012 [Page 2] > > Internet-Draft Flow Selection Techniques April 2012 > > > Table of Contents > > 1. Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 > 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4 > 3. Difference between Flow Selection and Packet Selection . . . . 7 > 4. Flow selection as a Function in the IPFIX Architecture . . . . 8 > 4.1. Flow selection during the Metering Process . . . . . . . . 10 > 4.2. Flow selection during the Exporting Process . . . . . . . 10 > 4.3. Flow selection as a function of the IPFIX Mediator . . . . 10 > 5. Flow Selection Techniques . . . . . . . . . . . . . . . . . . 11 > 5.1. Flow Filtering . . . . . . . . . . . . . . . . . . . . . . 11 > 5.1.1. Property Match Filtering . . . . . . . . . . . . . . . 11 > 5.1.2. Hash-based Flow Filtering . . . . . . . . . . . . . . 12 > 5.2. Flow Sampling . . . . . . . . . . . . . . . . . . . . . . 12 > 5.2.1. Systematic sampling . . . . . . . . . . . . . . . . . 12 > 5.2.2. Random Sampling . . . . . . . . . . . . . . . . . . . 13 > 5.3. Flow-state Dependent Flow Selection . . . . . . . . . . . 13 > 5.4. Flow-state Dependent Packet Selection . . . . . . . . . . 14 > 6. Configuration of Flow Selection Techniques . . . . . . . . . . 14 > 6.1. Flow Selection Parameters . . . . . . . . . . . . . . . . 16 > 6.2. Description of Flow-state Dependent Packet Selection . . . 18 > 7. Information Model for Flow Selection Configuration and > Reporting . . . . . . . . . . . . . . . . . . . . . . . . . . 18 > 7.1. flowSelectorAlgorithm . . . . . . . . . . . . . . . . . . 20 > 7.2. flowSelectedOctetDeltaCount . . . . . . . . . . . . . . . 21 > 7.3. flowSelectedPacketDeltaCount . . . . . . . . . . . . . . . 21 > 7.4. flowSelectedFlowDeltaCount . . . . . . . . . . . . . . . . 21 > 7.5. selectorIDTotalFlowsObserved . . . . . . . . . . . . . . . 22 > 7.6. selectorIDTotalFlowsSelected . . . . . . . . . . . . . . . 22 > 7.7. samplingFlowInterval . . . . . . . . . . . . . . . . . . . 22 > 7.8. samplingFlowSpace . . . . . . . . . . . . . . . . . . . . 23 > 7.9. flowSamplingTimeInterval . . . . . . . . . . . . . . . . . 23 > 7.10. flowSamplingTimeSpace . . . . . . . . . . . . . . . . . . 24 > 7.11. hashFlowDomain . . . . . . . . . . . . . . . . . . . . . . 24 > 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 24 > 8.1. Registration of Information Elements . . . . . . . . . . . 24 > 8.2. Registration of Object Identifier . . . . . . . . . . . . 32 > 9. Security Considerations . . . . . . . . . . . . . . . . . . . 32 > 10. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 34 > 11. References . . . . . . . . . . . . . . . . . . . . . . . . . . 34 > 11.1. Normative References . . . . . . . . . . . . . . . . . . . 34 > 11.2. Informative References . . . . . . . . . . . . . . . . . . 34 > Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 35 Don't you have to include the non-normative XML in the appendix, as it was done for RFC5102, RFC5103? > > > > > > > > > D'Antonio, et al. Expires October 25, 2012 [Page 3] > > Internet-Draft Flow Selection Techniques April 2012 > > > 1. Scope > > This document describes flow selection techniques for network traffic > measurements. A flow is defined as a set of packets with common > properties as described in [RFC5101]. Flow selection can be done to > limit the resource demands for capturing, storing, exporting and > post-processing of Flow Records. It also can be used to select a > particular set of flows that are of interest to a specific > application. This document provides a categorization of flow > selection techniques and describes configuration and reporting > parameters for them. In order to be compliant with this document, at > least one of the flow selection schemes MUST be implemented. That > means that the configuration parameters as well as the reporting > Information Elements for this particular scheme MUST be supported. Last sentence could be fine, but express that the way to configure the Intermediate Flow Selection Process is out of scope of this document. > > This document also addresses configuration and reporting parameters > for flow-state dependent packet selection as described in [RFC5475], > although this technique is categorized as packet selection. The > reason is that flow-state dependent packet selection techniques often > aim at the reduction of resources for flow capturing and flow > processing. Furthermore, they were only briefly discussed in Not sure what "they" refers to > [RFC5475]. Therefore we included configuration and reporting > considerations for such techniques in this document. please remove "we", "us", "our" from the draft. > > > 2. Terminology > > This document is consistent with the terminology introduced in > [RFC5101], [RFC5470], [RFC5475] and [RFC3917]. As in [RFC5101] and > [RFC5476], the first letter of each IPFIX-specific and PSAMP-specific > term is capitalized along with the flow selection specific terms > defined here. > > * Packet Classification > > Packet Classification is a process by which packets are mapped to > specific Flow Records based on packet properties or external > properties (e.g. interface). The properties (e.g. header > information, packet content, AS number) make up the Flow Key. In > case a Flow Record for a specific Flow Key already exists the Flow > Record is updated, otherwise a new Flow Record is created. How is this different that the Metering Process (RFC5101)? Metering Process The Metering Process generates Flow Records. Inputs to the process are packet headers and characteristics observed at an Observation Point, and packet treatment at the Observation Point (for example, the selected output interface). The Metering Process consists of a set of functions that includes packet header capturing, timestamping, sampling, classifying, and maintaining Flow Records. The maintenance of Flow Records may include creating new records, updating existing ones, computing Flow statistics, deriving further Flow properties, detecting Flow expiration, passing Flow Records to the Exporting Process, and deleting Flow Records. What is the connection with the Metering Process? Figure 1 seems to suggest that Packet Classification is a subset of the Metering Process... > > * Packet Aggregation Process > > In the IPFIX Metering Process the Packet Aggregation Process > aggregates packet data into flow data and forms the Flow Records. How is this different from the Metering Process? > After the aggregation step only the aggregated flow information is > available. Information about individual packets is lost. > > > > D'Antonio, et al. Expires October 25, 2012 [Page 4] > > Internet-Draft Flow Selection Techniques April 2012 > > > * Flow Selection Process > > A Flow Selection Process takes Flow Records as its input and > selects a subset of this set as its output. A Flow Selection > Process MAY run in several places within the IPFIX architecture. > A Flow Selection Process MAY be part of an IPFIX Metering Process, > Exporting Process or as an Intermediate Selection Process as > defined for the IPFIX Mediator [RFC6183]. If you look at the RFC6235, you will see Intermediate Anonymization Process: An intermediate process that takes Data Records and transforms them into Anonymized Data Records. To be perfect correct, the correct way is like it's done in http://tools.ietf.org/html/draft-ietf-ipfix-a9n-03 Intermediate Aggregation Process: an Intermediate Process as in [RFC6183 ] that aggregates records, based upon a set of Flow Keys or functions applied to fields from the record. What should be done here is Intermediate Flow Selection Process: an Intermediate Process as in [RFC6183 ] that ... Btw, you will see that "Intermediate Flow Selection Process" is already defined in the figure where you get it right. Finally, your sentence "A Flow Selection Process MAY be part of an IPFIX Metering Process, Exporting Process or as an Intermediate Selection Process as defined for the IPFIX Mediator [RFC6183]. ": - must contain "A Intermediate Flow Selection Process MAY be part of an IPFIX Metering Process ..." - must be reflected in figure 1, where I only see the Intermediate Flow Selection Process in the IPFIX Mediator. Could also be present in Exporting Process and in the Metering Process. As figure 1 is the only figure, all possibilities must be displayed. See as an example http://tools.ietf.org/html/draft-ietf-ipfix-a9n-03 figure 1 > > * Flow Selection State > > A Flow Selection Process SHOULD maintain state information for use normally, we try to avoid MAY/SHOULD/MUST in definition > by the Flow Selector. At a given time, the Flow Selection State > may depend on flows and packets observed at and before that time, > as well as other variables. Examples include: > > (i) sequence number of packets and accounted Flow Records; > > (ii) number of selected flows; > > (iii) number of observed flows; > > (iv) current flow cache occupancy; > > (v) flow specific counters, lower and upper bounds; > > (vi) flow selection timeout intervals. > > * Flow Selector > > A Flow Selector defines the action of a Flow Selection Process on > a single flow of its input. The Flow Selector can make use of the > following information in order to establish whether a flow has to > be selected or not: > > (i) the content of the Flow Record; > > (ii) any state information related to the Metering Process or > Exporting Process; > > (iii) any Flow Selection State that may be maintained by the > Flow Selection Process. I see many terms: Packet Classification, Packet Aggregation Process, Flow Selection Process, Flow Selection State, Intermediate Process, Intermediate Selection Process, etc... Please have a new figure that put combines all these terms. Something such as the figure in section 3.1 from http://tools.ietf.org/html/rfc5474 It might be your figure 1, but I don't even see those terms in there, or figure 3 in http://tools.ietf.org/html/draft-ietf-ipfix-configuration-model-10 > * Complete Flow > > A Complete Flow consists of all the packets that enter the Flow > Selection Process within the flow time-out interval, and which > belong to the same flow as defined by the flow definition in > > > > D'Antonio, et al. Expires October 25, 2012 [Page 5] > > Internet-Draft Flow Selection Techniques April 2012 > > > [RFC5470]. For this definition only packets that arrive at the > Flow Selection Process are considered. That means, packets that > are not observed at the Flow Selection Process because of prior > packet selection or packet loss are not considered as belonging to > the Complete Flow. what does the second sentence add? > > * Flow Filtering > > Flow Filtering selects flows based on a deterministic function on > the Flow Record content, Flow Selection State, external properties > (e.g. ingress interface) or external events (e.g violated Access > Control List). If the relevant parts of the Flow Record content > can already be observed at packet level (e.g. Flow Keys from > packet header fields) Flow Filtering can be performed at packet > level by Property Match Filtering as described in [RFC5475]. > > * Hash-based Flow Filtering > > Hash-based Flow Filtering is a deterministic flow filter function > that selects flows based on a Hash Function. The Hash Function is > calculated over parts of the Flow Record content or external > properties which are called the Hash Domain. If the hash value > falls into a predefined Hash Selection Range the flow is selected. > Hash-based Flow Filtering can already applied at packet level, in > which case the Hash Domain MUST contain the Flow Key of the > packet. In case Hash-based Flow Filtering is used to select the > same subset of flows at different observation points, the Hash > Domain MUST comprise parts of the packet or flow thar are > invariant on the packet/flow path. Also refer to the according > Trajectory Sampling Application Example on packet level in > [RFC5475] > > * Flow-state Dependent Flow Selection > > Flow-state Dependent Flow Selection is a selection function that > selects or drops flows based on the current Flow Selection State. > The selection can be either deterministic, random or non-uniform > random. > > * Flow-state Dependent Packet Selection > > Flow-state Dependent Packet Selection is a selection function that > selects or drops packets based on the current Flow Selection > State. The selection can be either deterministic, random or non- > uniform random. Flow-state Dependent Packet Selection can be used > to prefer the selection of packets belonging to specific flows. > For example the selection probability of packets belonging to > flows that are already within the Flow Cache may be higher than > > > > D'Antonio, et al. Expires October 25, 2012 [Page 6] > > Internet-Draft Flow Selection Techniques April 2012 > > > for packets that have not been recorded yet. > > * Flow Sampling > > Flow Sampling selects flows based on Flow Record sequence or > arrival times (e.g. entry in flow cache, arrival time at Exporter > or Mediator). The selection can be systematic (e.g. every n-th > flow) or based on a random function (e.g. select each Flow Record > with probability p, or randomly select n out of N Flow Records). > > > 3. Difference between Flow Selection and Packet Selection > > Flow selection differs from packet selection described in [RFC5475]. > Packet selection techniques consider packets as the basic element and > the parent population consists of all packets observed at an > observation point. In contrast to this the basic elements in flow > selection are the flows. The parent population consists of all > observed flows and the selection process operates on the flows. The > major characteristics of flow selection are the following: > > - Flow selection takes flows as basic elements. For packet > selection, packets are considered as basic elements. > > - Flow selection can only take place after Packet > Classification, because the classification rules determine to > which flow a packet belongs. Packet selection can be applied > before and after Packet Classification. I don't understand the last sentence. > > - Flow selection operates on Complete Flows. That means that > after the Flow Selection Process either all packets of the > flow are kept or all packets of the flow are discarded. That > means that if the flow selection is preceded by a packet > selection process the Complete Flow consists only of the > packets that were not discarded during the packet selection. > > There are some techniques that are difficult to unambiguously > categorize into one of the categories. Here we give some guidance > how to categorize such techniques: > > - Techniques that can be considered as both packet and flow > selection: some packet selection techniques result in the > selection of Complete Flows and therefore can be considered > as packet or as flow selection at the same time. An example > is Property Match Filtering of all packets to a specific > destination address. If flows are defined based on > destination addresses, such a packet selection also results > in a flow selection and can be considered as packet or flow > > > > D'Antonio, et al. Expires October 25, 2012 [Page 7] > > Internet-Draft Flow Selection Techniques April 2012 > > > selection. > > - Flow-state Dependent Packet Selection (as described in > [RFC5475]): there exist techniques that select packets based > on the flow state, e.g. based on the number of already > observed packets belonging to the flow. Examples of these > techniques from the literature are "Sample and Hold" [EsVa01] > "Fast Filtered Sampling" [MSZC10] or the "Sticky Sampling" > algorithm presented in [MaMo02]. Such techniques can be used > to influence which flows are captured (e.g. increase the > selection of packets belonging to large flows) and reduce the > number of flows that need to be stored in the flow cache. > Nevertheless, such techniques do not necessarily select > Complete Flows, because they do not ensure that all packets > of a selected flow are captured. Therefore Flow-state > Dependent Packet Selection methods that do not ensure that > either all or no packets of a flow are selected strictly > speaking have to be considered as packet selection techniques > and not as flow selection techniques. > > > 4. Flow selection as a Function in the IPFIX Architecture > > Figure 1 shows the IPFIX reference model as defined in [RFC5470] and > shows the Packet Classification and Packet Aggregation Process in the > Metering Process. > > > > > > > > > > > > > > > > > > > > > > > > > > D'Antonio, et al. Expires October 25, 2012 [Page 8] > > Internet-Draft Flow Selection Techniques April 2012 > > > Packet(s) coming in to Observation Point(s) > | | > v v > +----------------+---------------------------+ +-----+-------+ > | Metering Process | | | > | | | | > | packet header capturing | | | > | | |...| Metering | > | timestamping | | Process N | > | | | | | > | packet sampling | | | > | | | | | > | (packet classification) | | | > | | | | | > | packet filtering* | | | > | | | | | > | (packet aggregation)* | | | > | | | | | > +--------|-----------------------------------+ +-----|-------+ > Flow Records Flow Records | | > +----------------------+----------------------+ > | > +----------------------|-----------------+ > | Exporting Process* | > +----------------------+-----------------+ > | IPFIX (Flow Records) > v > +-------------------------|-----------------------+ > | IPFIX Mediator | | > | v | > | Collecting Process(es) | > | | | > | Intermediate Flow Selection Process (*) | > | | | > | Exporting Process(es) (*) | > +-------------------------|-----------------------+ > v > IPFIX > > (*) indicates where flow selection can take place. > > Figure 1: Flow selection in the IPFIX Architecture Please express the physical boundary between the Exporter and the IPFIX Mediator Why is packet classification in brackets? (packet aggregation), do you mean the Intermediate Aggregation Process in http://tools.ietf.org/html/draft-ietf-ipfix-a9n-03? > > In contrast to packet selection, flow selection is always applied > after the packets are classified into flows. Flows can be selected > at different stages of the measurement chain: > > > > > D'Antonio, et al. Expires October 25, 2012 [Page 9] > > Internet-Draft Flow Selection Techniques April 2012 > > > 1. during the Metering Process > > 2. during Exporting Process(es) > > 3. during an Intermediate Selection Process on a Mediator It's not "during" but "in". Please display the 3 points above in the figure > > 4.1. Flow selection during the Metering Process > > In the Packet Aggregation Process the packet information is used to > update the Flow Records in the flow cache. Flow selection that is > applied before aggregation equals a packet selection process. The > flow still consists of individual packets. Those are then selected > based on the classification information, i.e. based on the flow they > belong to. Flow selection before aggregation can be based on the > fields of the Flow Key (also on a hash value over these fields), but > not based on characteristics that are only available after packet > aggregation (e.g. flow size, flow duration). Flow selection during > the Metering Process is applied to reduce resources for all > succeeding processes or to select specific flows of interest in case > such flow characteristics are already observable at packet level > (e.g. flows to specific IP addresses). In contrast, Flow-state > Dependent Packet Selection is a packet selection method, because it > does not necessarily select Complete Flows. > > 4.2. Flow selection during the Exporting Process > > The Flow Selection Process at the Exporter is similar to an > Intermediate Selection Process as described in [RFC6183] and works on > Flow records. Flow selection during the Exporting Process can > therefore also depend on flow characteristics that are only visible > after the aggregation of packets, such as flow size and flow > duration. Why can't this be done in the Metering Process as well? > The Exporting Process may implement policies for exporting > only a subset of the Flow Records which have been stored in the > system memory in order to unload flow export and flow post- > processing. Flow selection during the Exporting Process may select > only the subset of Flow Records which are of interest to the users > application, or select only as many Flow Records as can be handled by > the available resources (e.g. limited export link capacity). > > 4.3. Flow selection as a function of the IPFIX Mediator > > As shown in Figure 1, flow selection can be performed as an Please rewrite these section 4.* based on the Intermediate Flow Selection Process, and not flow selection, which is not defined. > Intermediate Process within an IPFIX Mediator [RFC6183]. The > Intermediate Selection Process takes Flow Record stream as its input > and selects Flow Records from a sequence based upon criteria- > evaluated record values. The Intermediate Selection Process can > again apply a flow selection technique to obtain flows of interest to > the application. Further, the Intermediate Selection Process can > > > > D'Antonio, et al. Expires October 25, 2012 [Page 10] > > Internet-Draft Flow Selection Techniques April 2012 > > > base its selection decision on the correlation of data from different > observation points, or most of the time Exporters (which btw, you should show this in the figure) See figure A in RFC 6183 > e.g. by only selecting flows that were at least > recorded on two observation points. > > > 5. Flow Selection Techniques > > A flow selection technique selects either all or none of the packets I see sometimes flow selection, sometimes flow selection technique, sometimes selection technique, sometimes flow selection scheme, sometimesflow selection methods, sometimes Flow Selection Process which should really be the Intermediate Flow Selection Process... It's difficult to read the draft, as it misses some cohesion: this clearly shows that different parts of the document have been written by different persons. Please review the entire document with a fresh mind, as if you would discover for the first time, and you will see what I mean. The entire draft needs some improvements on that matter. > of a flow, otherwise the technique has to be considered as packet > selection. We distinguish between Flow Filtering and Flow Sampling. > > 5.1. Flow Filtering > > Flow Filtering is a deterministic function on the IPFIX Flow Record > content. If the relevant flow characteristics are already observable > at packet level (e.g. Flow Keys), Flow Filtering can be applied > before aggregation at packet level. In order to be compliant with > this document, at least the Property Match Filtering MUST be > implemented. This contradicts. In order to be compliant with this document, at least one of the flow selection schemes MUST be implemented. > > 5.1.1. Property Match Filtering > > Property Match Filtering can be performed similarly to Property Match > Filtering for packet selection described in [RFC5475]. The > difference is that, instead of packet fields, Flow Record fields are > here used to derive the selection decision. Property Match Filtering > is typically used to select a specific subset of the flows that are > of interest to a particular application (e.g. all flows to a specific > destination, all large flows, etc.). Properties on which the > filtering is based can be Flow Keys, Flow Timestamps, or Per-Flow > Counters described in [RFC5102]. Examples of properties are the flow > size in bytes, the number of packets in the flow, the observation > time of the first or last packet, or the maximum packet length. An > example is to select flows with more than a threshold number of > observed octets. The selection criteria can be a specific value, a > set of specific values, or an interval. For example, a flow is > selected if destinationIPv4Address and the total number of packets of > the flow equal two predefined values. Property Match Filtering can > be applied during the Metering Process if the properties are already > observable at the packet level (e.g. Flow Key fields). For example, > a flow is selected if sourceIPv4Address and sourceIPv4PrefixLength > equal, respectively, two specific values. > > There are content-based Property Match Filtering techniques that > require a computation on the current flow cache. An example is the > selection of the largest flows or a percentage of flows with the > longest lifetime. This type of Property Match Filtering is also used > in flow selection techniques that react to external events (e.g. > > > > D'Antonio, et al. Expires October 25, 2012 [Page 11] > > Internet-Draft Flow Selection Techniques April 2012 > > > resource constraint). For example when the flow cache is full, the > Flow Record with the lowest flow volume per current flow life time > may be deleted. > > 5.1.2. Hash-based Flow Filtering > > Hash-based Flow Filtering uses a Hash Function h to map the Flow Key > c onto a Hash Range R. A flow is selected if the hash value h(c) is > within the Hash Selection Range S, which is a subset of R. Hash-based > Flow Filtering can be used to emulate a random sampling process but > still enable the correlation between selected flow subsets at > different observation points. Hash-based Flow Filtering is similar > to Hash-based Packet Selection, and in fact is identical when Hash- > based Packet Selection uses the Flow Key that defines the flow as the > hash input. Nevertheless there may be the incentive to apply Hash- > based Flow Filtering not on the packet level during the Metering > Process, for example when the size of the selection range and > therefore the sampling probability is dependent on the number of > observed flows. > > 5.2. Flow Sampling > > Flow Sampling operates on Flow Record sequence or arrival times. It > can use either a systematic or a random function for the selection > process. Flow Sampling usually aims at the selection of a > representative subset of all flows in order to estimate > characteristics of the whole set (e.g. mean flow size in the > network). > > 5.2.1. Systematic sampling > > Systematic sampling is a deterministic selection function. > Systematic sampling may be a periodic selection of the N-th Flow > Record which arrives at the Exporting or Intermediate Selection > Process. Intermediate Flow Selection Process > Systematic sampling MAY be applied during the Metering > Process. An example would be to create, besides the Flow cache of > selected flows, an additional data structure that saves the Flow Keys > of the flows that are not selected. The selection of a flow would > then be based on the first packet of a flow. Everytime a packet > belonging to a new flow (which is neither in the data structure of > the selected or not selected flows) arrives at the measurement point, what is a measurement point? > a counter is increased. In case the counter is increased to a > multiple of N a new flow cache entry is created, and in case the > counter is not a multiple of N the Flow Key is added to the data > structure for not selected flows. > > Systematic sampling can also be time-based. Time-based systematic > sampling is applied by only creating flows that are observed between flows -> Flows all over the doc. > > > > D'Antonio, et al. Expires October 25, 2012 [Page 12] > > Internet-Draft Flow Selection Techniques April 2012 > > > time-based start and stop triggers. The time interval may be applied > at packet level during the Metering Process or after aggregation on > flow level, e.g. by selecting a flow arriving at the Exporting > Process every n seconds. > > 5.2.2. Random Sampling > > Random flow sampling is based on a random process which requires the > calculation of random numbers. One can differentiate between n-out-N > and probabilistic flow sampling. > > 5.2.2.1. n-out-of-N Flow Sampling > > In n-out-of-N Sampling, n elements are selected out of the parent > population that consists of N elements. One example would be to > generate n different random numbers in the range [1,N] and select all > flows that have a flow position equal to one of the random numbers. > > 5.2.2.2. Probabilistic Flow Sampling > > In probabilistic Sampling, the decision whether or not a flow is > selected is made in accordance with a predefined selection > probability. For probabilistic Sampling, the Sample Size can vary > for different trials. The selection probability does not necessarily > have to be the same for each flow. Therefore, we distinguish between > uniform probabilistic sampling (with the same selection probability > for all flows) and non-uniform probabilistic sampling (where the > selection probability can vary for different flows). For non-uniform > probabilistic Flow Sampling the sampling probability may be adjusted > according to the Flow Record content. An example would be to > increase the selection probability of large volume flows over small > volume flows as described in the Smart Sampling technique [DuLT01]. > > 5.3. Flow-state Dependent Flow Selection > > Flow-state Dependent Flow Selection can be a deterministic or random > flow selection process based on the Flow Record content and the flow > state which may be kept additionally for each of the flows. External > processes may update counters, bounds and timers for each of the Flow > Records and the Flow Selection Process utilises this information for > the selection decision. A review of Flow-state Dependent Flow > Selection techniques that aim at the selection of the most frequent > items by keeping additional flow state information can be found in > [CoHa08]. Flow-state Dependent Flow Selection can only be applied > after packet aggregation, when a packet has been assigned to a flow. > The selection process then decides based upon the flow state for each > flow if it is kept in the flow cache or not. Two Flow State > Dependent Flow Selection Algorithms are here described: > > > > D'Antonio, et al. Expires October 25, 2012 [Page 13] > > Internet-Draft Flow Selection Techniques April 2012 > > > The frequent algorithm [KaPS03] is a technique that aims at the > selection of all flows that at least exceed a 1/k fraction of the > Observed Packet Stream. to come back to one of my previous point, please add Observed Packet Stream to the extra figure > The algorithm has only a flow cache of size > k-1 and each flow in the cache has an additional counter. The > counter is incremented each time a packet belonging to the flow in > the flow cache is observed. In case the observed packet does not > belong to any flow all counters are decremented and if any of the > flow counters has a value of zero the flow is replaced with a flow > formed from the new packet. > > Lossy counting is a selection technique that identifies all flows > whose packet count exceeds a certain percentage of the whole observed > packet stream (e.g. 5% of all packets) with a certain estimation > error e. Lossy counting separates the observed packet stream in > windows of size N=1/e, where N is an amount of consecutive packets. > For each observed flow an additional counter will be held in the flow > state. The counter is incremented each time a packet belonging to > the flow is observed and all counters are decremented at the end of > each window and all flows with a counter of zero are removed from the > flow cache. > > 5.4. Flow-state Dependent Packet Selection > > Flow-state Dependent Packet Selection is not a flow selection > technique but a packet selection technique. Nevertheless we will > describe configuration and reporting parameters for this technique in > this document. An example is the "Sample and Hold" algorithm > [EsVa01] that tries to prefer large volume flows in the selection. > When a packet arrives it is selected when a Flow Record for this > packet already exists. In case there is no Flow Record, the packet > is selected by a certain probability that is dependent on the packet > size. > > > 6. Configuration of Flow Selection Techniques > > This section describes the configuration parameters of the flow > selection techniques presented above. It provides the basis for an > information model to be adopted in order to configure the Flow > Selection Process within an IPFIX Device. The actual information > model with the Information Elements (IEs) for the configuration is > described together with the reporting IEs in section 7. The > following table gives an overview of the defined selection > techniques, where they can be applied and what their input parameters > are. Depending on where the flow selection techniques are applied > different input parameters can be configured. > > Overview of Flow Selection Techniques: > > > > D'Antonio, et al. Expires October 25, 2012 [Page 14] > > Internet-Draft Flow Selection Techniques April 2012 > > > +------------------+-----------------+------------------------------+ > | Location | Selection | Selection Input | > | | Method | | > +------------------+-----------------+------------------------------+ > | During the | Flow-state | packet sampling | location is not "during" but "in" > | Metering Process | Dependent | probabilities, Flow | > | based on Packets | Packet | Selection State, packet | > | | Selection | properties | I still don't get why "in the Metering Process" means, on packets? > +------------------+-----------------+------------------------------+ should > | | Property Match | Flow record IEs, Selection | > | | Flow Filtering | Interval | > +------------------+-----------------+------------------------------+ > | | Hash-based Flow | selection range, Hash | > | | Filtering | Function, Flow Key, (seed) | > +------------------+-----------------+------------------------------+ > | | Time-based | flow position (derived from | > | | Systematic Flow | arrival time of packets), | > | | Sampling | flow selection state | > +------------------+-----------------+------------------------------+ > | | Sequence-based | flow position (derived from | > | | Systematic Flow | packet position), flow | > | | Sampling | selection state | > +------------------+-----------------+------------------------------+ > | | Random Flow | random number generator or | > | | Sampling | list and packet position, | > | | | flow state | > +------------------+-----------------+------------------------------+ This location above applies to the 6 column, right? so make a big cell out for "in the Metering Process based on packets" Same remark below. > | Exporting / | Property Match | Flow Record content, filter | > | Intermediate | Flow Filtering | function | > | Selection | | | > | Process | | | > +------------------+-----------------+------------------------------+ > | | Hash-based Flow | selection range, Hash | > | | Filtering | Function, hash input (Flow | > | | | Keys and other flow | > | | | properties) | > +------------------+-----------------+------------------------------+ > | | Flow-state | flow state parameters, | > | | Dependent Flow | random number generator or | > | | Selection | list | > +------------------+-----------------+------------------------------+ > | | Time-based | flow arrival time, flow | > | | Systematic Flow | state | > | | Sampling | | > +------------------+-----------------+------------------------------+ > | | Sequence-based | flow position, flow state | > | | Systematic Flow | | > | | Sampling | | > > > > D'Antonio, et al. Expires October 25, 2012 [Page 15] > > Internet-Draft Flow Selection Techniques April 2012 > > > | | Random Flow | random number generator or | > | | Sampling | list and flow position, flow | > | | | state | > +------------------+-----------------+------------------------------+ Not sure what the "flow position". Below, you use "Spacing" for this concept. > > Table 1: Overview of Flow Selection Techniques > > 6.1. Flow Selection Parameters > > In this section, we define what parameters are required to describe > the most common Flow Selection techniques. You see, yet another new term, which is not in the terminology: Flow Selection And on the top of my head, it was not defined in any other documents referenced in the terminology > > Flow Selection Parameters: > > For Property Match Filtering: > > - Information Element as specified in [iana-ipfix-assignments]): > Specifies the Information Element which is used as the property > in the filter expression. > > - Selection Value or Value Interval: > Specifies the value or interval of the filter expression. > Packets and Flow Record that have a value equal to the Selection > Value or within the Interval will be selected. > > For Hash-based Flow Filtering: > > - Hash Domain: > Specifies the bits from the packet or flow which are taken as the > hash input to the Hash Function. > > - Hash Function: > Specifies the name of the Hash Function that is used to calculate > the hash value. Possible Hash Functions are BOB [RFC5475], IPSX > [RFC5475], CRC-32 [Bra75] > > - Hash Selection Range: > Flows that have a hash value within the Hash Selection Range are > selected. The Hash Selection Range can be a value interval or > arbitrary hash values within the Hash Range of the Hash Function. > > - Random Seed or Initializer Value: > Some Hash Functions require an initializing value. In order to > make the selection decision more secure one can choose a random > seed that configures the hash function. > > For Flow-state Dependent Flow Selection: > > > > > D'Antonio, et al. Expires October 25, 2012 [Page 16] > > Internet-Draft Flow Selection Techniques April 2012 > > > - frequency threshold: > Specifies the frequency threshold s for flow state dependent flow > selection techniques that try to find the most frequent items > within a dataset. All flows which exceed the defined threshold > will be selected. > > - accuracy parameter: > specifies the accuracy parameter e for techniques that deal with > the frequent items problems. The accuracy parameter defines the > maximum error, i.e. no flows that have a true frequency less than > ( s - e) N are selected, where s is the frequency threshold and N > is the total number of packets. > > The above list of parameters for Flow-state Dependent Flow Selection > techniques is suitable for the presented frequent item and lossy > counting algorithms. Nevertheless a variety of techniques exist with > very specific parameters which are not defined here. > > For Systematic time-based Flow Sampling: > > - Interval length (in usec) > Defines the length of the sampling interval during which flows > are selected. > > - Spacing (in usec) > The spacing parameter defines the spacing in usec between the end > of one sampling interval and the start of the next succeeding > interval. > > For Systematic count-based Flow Sampling: > > - Interval length > Defines the number of flows that are selected within the sampling > interval. > > - Spacing > The spacing parameter defines the spacing in number of observed > flows between the end of one sampling interval and the start of > the next succeeding interval. > > For random n-out-of-N Flow Sampling: > > - Population Size N > The Population Size N is the number of all flows in the > Population from which the sample is drawn. > > > > > > > D'Antonio, et al. Expires October 25, 2012 [Page 17] > > Internet-Draft Flow Selection Techniques April 2012 > > > - Sampling Size n > The sampling size n is the number of flows that are randomly > drawn from the population N. > > For probabilistic Flow Sampling: > > - Sampling probability p > The sampling probability p defines the probability by which each > of the observed flows is selected. > > 6.2. Description of Flow-state Dependent Packet Selection > > The configuration of Flow-state Dependent Packet Selection has not > been described in [RFC5475] therefore the parameters are defined > here: > > For Flow-state Dependent Packet Selection: > > - packet selection probability per possible flow state interval > Defines multiple {flow interval, packet selection probability} > value pairs that configure the sampling probability depending on > the current flow state. > > - additional parameters > For the configuration of flow state dependent packet selection > additional parameters or packet properties may be required, e.g. > the packet size ([EsVa01]) > > > 7. Information Model for Flow Selection Configuration and Reporting > > In this section we describe Information Elements (IEs) that MUST be This section specifies ... > exported by a flow selection process in order to support the Flow Selection Process > interpretation of measurement results from flow measurements where > only some flows are selected. "from flow measurements where only some flows are selected. " Do you need that? Isn't it by default what a Flow Selection Process does? > The information is mainly used to > report how many packets and flows have been observed in total and how > many of them were selected. This helps for instance to calculate the > Attained Selection Fraction (see also [RFC5476]), which is an > important parameter to provide an accuracy statement. The IEs can > provide reporting information about Flow Records, packets or bytes. > The reported metrics are total number of elements and the number of > selected elements. From this the number of dropped elements can be > derived. All counters SHOULD be exported and reset when a new > measurement interval starts. I disagree here. It depends which IEs you use: deltaCounter or totalCounter Search for those two terms in http://www.iana.org/assignments/ipfix/ipfix.xml > > List of Flow Selection Information Elements: > > > > > > D'Antonio, et al. Expires October 25, 2012 [Page 18] > > Internet-Draft Flow Selection Techniques April 2012 > > > +------+-------------------------+-------+--------------------------+ > | ID | Name | ID | Name | > +------+-------------------------+-------+--------------------------+ > | 301 | selectionSequenceID | 302 | selectorID | > +------+-------------------------+-------+--------------------------+ > | TBD1 | flowSelectorAlgorithm | 1 | octetDeltaCount | > +------+-------------------------+-------+--------------------------+ > | TBD2 | flowSelectedOctetDeltaC | 2 | packetDeltaCount | > | | ount | | | > +------+-------------------------+-------+--------------------------+ > | TBD3 | flowSelectedPacketDelta | 3 | originalFlowsPresent | > | | Count | | | > +------+-------------------------+-------+--------------------------+ > | TBD4 | flowSelectedFlowDeltaCo | TBD5 | selectorIDTotalFlowsObse | > | | unt | | rved | > +------+-------------------------+-------+--------------------------+ > | TBD6 | selectorIDTotalFlowsSel | TBD7 | samplingFlowInterval | > | | ected | | | > +------+-------------------------+-------+--------------------------+ > | TBD8 | samplingFlowSpace | 309 | samplingSize | > +------+-------------------------+-------+--------------------------+ > | 310 | samplingPopulation | 311 | samplingProbability | > +------+-------------------------+-------+--------------------------+ > | TBD9 | flowSamplingTimeInterva | TBD10 | flowSamplingTimeSpace | > | | l | | | > +------+-------------------------+-------+--------------------------+ > | 326 | digestHashValue | TBD11 | hashFlowOffset | > +------+-------------------------+-------+--------------------------+ > | TBD1 | hashFlowSize | 329 | hashOutputRangeMin | > | 2 | | | | > +------+-------------------------+-------+--------------------------+ > | 330 | hashOutputRangeMax | 331 | hashSelectedRangeMin | > +------+-------------------------+-------+--------------------------+ > | 332 | hashSelectedRangeMax | 333 | hashDigestOutput | > +------+-------------------------+-------+--------------------------+ > | 334 | hashInitialiserValue | 320 | absoluteError | > +------+-------------------------+-------+--------------------------+ > | 321 | relativeError | 336 | upperCILimit | > +------+-------------------------+-------+--------------------------+ > | 337 | lowerCILimit | 338 | confidenceLevel | > +------+-------------------------+-------+--------------------------+ > > Table 2: Flow Selection Information Elements > > > > > > > > > D'Antonio, et al. Expires October 25, 2012 [Page 19] > > Internet-Draft Flow Selection Techniques April 2012 > > > 7.1. flowSelectorAlgorithm > > Description: > > This Information Element identifies the flow selection > method(e.g., Filtering, Sampling) that is applied by the Flow > Selection Process. Most of these methods have parameters as > decribed in Section 6. Further Information Elements are needed to > fully specify packet selection with these methods and all their > parameters. Why do you speak about "packet selection" in the flowSelectorAlgorithm? > Further method identifiers may be added to the list > below. It might be necessary to define new Information Elements > to specify their parameters. The flowSelectorAlgorithm registry > is maintained by IANA. New assignments for the registry will be > administered by IANA and are subject to Expert Review [RFC5226]. > The registry can be updated when specifications of the new > method(s) and any new Information Elements are provided. > > > +----+------------------------+--------------------------+ > | ID | Method | Parameters | > +----+------------------------+--------------------------+ > | 1 | Systematic count-based | flowSamplingInterval | > | | Sampling | flowSamplingSpace | > +----+------------------------+--------------------------+ > | 2 | Systematic time-based | flowSamplingTimeInterval | > | | Sampling | flowSamplingTimeSpace | > +----+------------------------+--------------------------+ > | 3 | Random n-out-of-N | samplingSize | > | | Sampling | samplingPopulation | > +----+------------------------+--------------------------+ > | 4 | Uniform probabilistic | samplingProbability | > | | Sampling | | > +----+------------------------+--------------------------+ > | 5 | Property Match | Information Element | > | | Filtering | Value Range | > +----+------------------------+--------------------------+ > | Hash-based Filtering | hashInitialiserValue | > +----+------------------------+ hashFlowDomain | > | 6 | using BOB | hashSelectedRangeMin | > +----+------------------------+ hashSelectedRangeMax | > | 7 | using IPSX | hashOutputRangeMin | > +----+------------------------+ hashOutputRangeMax | > | 8 | using CRC | | > +----+------------------------+--------------------------+ > | 9 | Flow State Dependent | No agreed Parameters | > | | Flow Selection | | > +----+------------------------+--------------------------+ > > > > > D'Antonio, et al. Expires October 25, 2012 [Page 20] > > Internet-Draft Flow Selection Techniques April 2012 > > > Abstract Data Type: unsigned16 > > ElementId: TBD1 > > Data Type Semantics: identifier > > Status: Proposed > > 7.2. flowSelectedOctetDeltaCount > > Description: > > This Information Element specifies the volume in octets of all > flows that are selected during the Flow Selection Process since > the previous report. > > Abstract Data Type: unsigned64 > > ElementId: TBD2 > > Units: Octets > > Status: Proposed > > 7.3. flowSelectedPacketDeltaCount > > Description: > > This Information Element specifies the volume in packets of all > flows that were selected during the Flow Selection Process since > the previous report. > > Abstract Data Type: unsigned64 > > ElementId: TBD3 > > Units: Packets > > Status: Proposed > > 7.4. flowSelectedFlowDeltaCount > > Description: > > This Information Element specifies the number of Flows that were > selected during the Flow Selection Process since the last report. > > Abstract Data Type: unsigned64 > > > > D'Antonio, et al. Expires October 25, 2012 [Page 21] > > Internet-Draft Flow Selection Techniques April 2012 > > > ElementId: TBD4 > > Units: Flows > > Status: Proposed > > 7.5. selectorIDTotalFlowsObserved > > Description: > > This Information Element specifies the total number of flows > observed by a Selector, for a specific value of SelectorId. This > Information Element should be used in an Options Template scoped > to the observation to which it refers. See Section 3.4.2.1 of the > IPFIX protocol document [RFC5101] . > > Abstract Data Type: unsigned64 > > ElementId: TBD5 > > Units: Flows > > Status: Proposed > > 7.6. selectorIDTotalFlowsSelected > > Description: > > This Information Element specifies the total number of flows > selected by a Selector, for a specific value of SelectorId. This > Information Element should be used in an Options Template scoped > to the observation to which it refers. See Section 3.4.2.1 of the > IPFIX protocol document [RFC5101]. > > Abstract Data Type: unsigned64 > > ElementId: TBD6 > > Units: Flows > > Status: Proposed > > 7.7. samplingFlowInterval > > Description: > > This Information Element specifies the number of flows that are > consecutively sampled. A value of 100 means that 100 consecutive > > > > D'Antonio, et al. Expires October 25, 2012 [Page 22] > > Internet-Draft Flow Selection Techniques April 2012 > > > flows are sampled. For example, this Information Element may be > used to describe the configuration of a systematic count-based > Sampling Selector. > > Abstract Data Type: unsigned64 > > ElementId: TBD7 > > Units: Flows > > Status: Proposed > > 7.8. samplingFlowSpace > > Description: > > This Information Element specifies the number of flows between two > "samplingFlowInterval"s. A value of 100 means that the next > interval starts 100 flows (which are not sampled) after the > current "samplingFlowInterval" is over. For example, this > Information Element may be used to describe the configuration of a > systematic count-based Sampling Selector. The text above mentions: For Systematic count-based Flow Sampling: - Interval length Defines the number of flows that are selected within the sampling interval. - Spacing The spacing parameter defines the spacing in number of observed flows between the end of one sampling interval and the start of the next succeeding interval. So be consistent between "Space", "Spacing", and position (see one of my previous comments" > Abstract Data Type: unsigned64 > > ElementId: TBD8 > > Units: Flows > > Status: Proposed > > 7.9. flowSamplingTimeInterval > > Description: > > This Information Element specifies the time interval in > microseconds during which all arriving flows are sampled. For > example, this Information Element may be used to describe the > configuration of a systematic time-based Sampling Selector. The text above mentions: For Systematic time-based Flow Sampling: - Interval length (in usec) Defines the length of the sampling interval during which flows are selected. - Spacing (in usec) The spacing parameter defines the spacing in usec between the end of one sampling interval and the start of the next succeeding interval. So be consistent between "Space", "Spacing", and position (see one of my previous comments" > > Abstract Data Type: unsigned64 > > ElementId: TBD9 > > Units: microseconds > > Status: Proposed > > > > > D'Antonio, et al. Expires October 25, 2012 [Page 23] > > Internet-Draft Flow Selection Techniques April 2012 > > > 7.10. flowSamplingTimeSpace > > Description: > > This Information Element specifies the time interval in > microseconds between two "flowSamplingTimeInterval"s. A value of > 100 means that the next interval starts 100 microseconds (during > which no flows are sampled) after the current > "flowsamplingTimeInterval" is over. For example, this Information > Element may used to describe the configuration of a systematic > time-based Sampling Selector. Same remark. > > Abstract Data Type: unsigned64 > > ElementId: TBD10 > > Units: microseconds > > Status: Proposed > > 7.11. hashFlowDomain > > Description: > > This Information Element specifies the Information Elements that > are used by the Hash-based flow Selection Selector as the Hash > Domain. > > Abstract Data Type: unsigned16 > > ElementId: TBD11 > > Data Type Semantics: identifier > > Status: Proposed > > > 8. IANA Considerations > > 8.1. Registration of Information Elements > > IANA will register the following IEs in the IPFIX Information > Elements registry at http://www.iana.org/assignments/ipfix/ipfix.xml: > > > > > > > > > D'Antonio, et al. Expires October 25, 2012 [Page 24] > > Internet-Draft Flow Selection Techniques April 2012 > > > +-----+----------------+--------+---------+-------+-----------------+ > | Val | Name | Data | Data | Statu | Description | > | ue | | Type | Type | s | | > | | | | Semanti | | | > | | | | cs | | | > +-----+----------------+--------+---------+-------+-----------------+ > | 1 | flowSelectorAl | unsign | identif | Propo | This | > | | gorithm | ed16 | ier | sed | Information | > | | | | | | Element | > | | | | | | identifies the | > | | | | | | flow selection | > | | | | | | method(e.g., | > | | | | | | Filtering, | > | | | | | | Sampling) that | > | | | | | | is applied by | > | | | | | | the Flow | > | | | | | | Selection | > | | | | | | Process | > +-----+----------------+--------+---------+-------+-----------------+ > | 2 | flowSelectedOc | unsign | Octets | Propo | This | > | | tetDeltaCount | ed64 | | sed | Information | > | | | | | | Element | > | | | | | | specifies the | > | | | | | | volume in | > | | | | | | octets of all | > | | | | | | flows that are | > | | | | | | selected during | > | | | | | | the Flow | > | | | | | | Selection | > | | | | | | Process since | > | | | | | | the previous | > | | | | | | report. | > +-----+----------------+--------+---------+-------+-----------------+ > | 3 | flowSelectedPa | unsign | Packets | Propo | This | > | | cketDeltaCount | ed64 | | sed | Information | > | | | | | | Element | > | | | | | | specifies the | > | | | | | | volume in | > | | | | | | packets of all | > | | | | | | flows that were | > | | | | | | selected during | > | | | | | | the Flow | > | | | | | | Selection | > | | | | | | Process since | > | | | | | | the previous | > | | | | | | report. | > +-----+----------------+--------+---------+-------+-----------------+ > > > > > D'Antonio, et al. Expires October 25, 2012 [Page 25] > > Internet-Draft Flow Selection Techniques April 2012 > > > +-----+----------------+--------+---------+-------+-----------------+ > | 4 | flowSelectedFl | unsign | Flows | Propo | This | > | | owDeltaCount | ed64 | | sed | Information | > | | | | | | Element | > | | | | | | specifies the | > | | | | | | number of Flows | > | | | | | | that were | > | | | | | | selected during | > | | | | | | the Flow | > | | | | | | Selection | > | | | | | | Process since | > | | | | | | the last | > | | | | | | report. | > +-----+----------------+--------+---------+-------+-----------------+ > | 5 | selectorIDTota | unsign | Flows | Propo | This | > | | lFlowsObserved | ed64 | | sed | Information | > | | | | | | Element | > | | | | | | specifies the | > | | | | | | total number of | > | | | | | | flows observed | > | | | | | | by a Selector, | > | | | | | | for a specific | > | | | | | | value of | > | | | | | | SelectorId. | > | | | | | | This | > | | | | | | Information | > | | | | | | Element should | > | | | | | | be used in an | > | | | | | | Options | > | | | | | | Template scoped | > | | | | | | to the | > | | | | | | observation to | > | | | | | | which it | > | | | | | | refers. See | > | | | | | | Section 3.4.2.1 | > | | | | | | of the IPFIX | > | | | | | | protocol | > | | | | | | document | > | | | | | | [RFC5101] | > +-----+----------------+--------+---------+-------+-----------------+ > > > > > > > > > > > > D'Antonio, et al. Expires October 25, 2012 [Page 26] > > Internet-Draft Flow Selection Techniques April 2012 > > > +-----+----------------+--------+---------+-------+-----------------+ > | 6 | selectorIDTota | unsign | Flows | Propo | This | > | | lFlowsSelected | ed64 | | sed | Information | > | | | | | | Element | > | | | | | | specifies the | > | | | | | | total number of | > | | | | | | flows selected | > | | | | | | by a Selector, | > | | | | | | for a specific | > | | | | | | value of | > | | | | | | SelectorId. | > | | | | | | This | > | | | | | | Information | > | | | | | | Element should | > | | | | | | be used in an | > | | | | | | Options | > | | | | | | Template scoped | > | | | | | | to the | > | | | | | | observation to | > | | | | | | which it | > | | | | | | refers. See | > | | | | | | Section 3.4.2.1 | > | | | | | | of the IPFIX | > | | | | | | protocol | > | | | | | | document | > | | | | | | [RFC5101]. | > +-----+----------------+--------+---------+-------+-----------------+ > > > > > > > > > > > > > > > > > > > > > > > > > D'Antonio, et al. Expires October 25, 2012 [Page 27] > > Internet-Draft Flow Selection Techniques April 2012 > > > +-----+----------------+--------+---------+-------+-----------------+ > | 7 | samplingFlowIn | unsign | Flows | Propo | This | > | | terval | ed64 | | sed | Information | > | | | | | | Element | > | | | | | | specifies the | > | | | | | | number of flows | > | | | | | | that are | > | | | | | | consecutively | > | | | | | | sampled. A | > | | | | | | value of 100 | > | | | | | | means that 100 | > | | | | | | consecutive | > | | | | | | flows are | > | | | | | | sampled. For | > | | | | | | example, this | > | | | | | | Information | > | | | | | | Element may be | > | | | | | | used to | > | | | | | | describe the | > | | | | | | configuration | > | | | | | | of a systematic | > | | | | | | count-based | > | | | | | | Sampling | > | | | | | | Selector. | > +-----+----------------+--------+---------+-------+-----------------+ > > > > > > > > > > > > > > > > > > > > > > > > > > > D'Antonio, et al. Expires October 25, 2012 [Page 28] > > Internet-Draft Flow Selection Techniques April 2012 > > > +-----+----------------+--------+---------+-------+-----------------+ > | 8 | samplingFlowSp | unsign | Flows | Propo | This | > | | ace | ed64 | | sed | Information | > | | | | | | Element | > | | | | | | specifies the | > | | | | | | number of flows | > | | | | | | between two | > | | | | | | "samplingFlowIn | > | | | | | | terval"s. A | > | | | | | | value of 100 | > | | | | | | means that the | > | | | | | | next interval | > | | | | | | starts 100 | > | | | | | | flows (which | > | | | | | | are not | > | | | | | | sampled) after | > | | | | | | the current | > | | | | | | "samplingFlowI | > | | | | | | nterval" is ove | > | | | | | | r.For example, | > | | | | | | this | > | | | | | | Information | > | | | | | | Element may b | > | | | | | | e used to | > | | | | | | describe the | > | | | | | | configuration | > | | | | | | of a systemat | > | | | | | | iccount-based | > | | | | | | Sampling | > | | | | | | Selector. | > +-----+----------------+--------+---------+-------+-----------------+ > > > > > > > > > > > > > > > > > > > > > D'Antonio, et al. Expires October 25, 2012 [Page 29] > > Internet-Draft Flow Selection Techniques April 2012 > > > +-----+----------------+--------+---------+-------+-----------------+ > | 9 | flowSamplingTi | unsign | microse | Propo | This | > | | meInterval | ed64 | conds | sed | Information | > | | | | | | Element | > | | | | | | specifies the | > | | | | | | time interval | > | | | | | | in microseconds | > | | | | | | during which | > | | | | | | all arriving | > | | | | | | flows are | > | | | | | | sampled. For | > | | | | | | example, this | > | | | | | | Information | > | | | | | | Element may be | > | | | | | | used to | > | | | | | | describe the | > | | | | | | configuration | > | | | | | | of a systematic | > | | | | | | time-based | > | | | | | | Sampling | > | | | | | | Selector. | > +-----+----------------+--------+---------+-------+-----------------+ > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > D'Antonio, et al. Expires October 25, 2012 [Page 30] > > Internet-Draft Flow Selection Techniques April 2012 > > > +-----+----------------+--------+---------+-------+-----------------+ > | 10 | flowSamplingTi | unsign | microse | Propo | This | > | | meSpace | ed64 | conds | sed | Information | > | | | | | | Element | > | | | | | | specifies the | > | | | | | | time interval | > | | | | | | in microseconds | > | | | | | | between two | > | | | | | | "flowSamplingTi | > | | | | | | meInterval"s. | > | | | | | | Avalue of 100 | > | | | | | | means that the | > | | | | | | next interval | > | | | | | | starts 100 | > | | | | | | microseconds | > | | | | | | (during which | > | | | | | | no flows are | > | | | | | | sampled) after | > | | | | | | the current | > | | | | | | "flowsamplingT | > | | | | | | imeInterval" is | > | | | | | | over. For | > | | | | | | example, this | > | | | | | | Information | > | | | | | | Element may | > | | | | | | used to | > | | | | | | describe the | > | | | | | | configuration | > | | | | | | of a systemat | > | | | | | | ictime-based | > | | | | | | Sampling | > | | | | | | Selector. | > +-----+----------------+--------+---------+-------+-----------------+ > | 11 | hashFlowDomain | unsign | identif | Propo | This | > | | | ed16 | ier | sed | Information | > | | | | | | Element | > | | | | | | specifies the | > | | | | | | Information | > | | | | | | Elements that | > | | | | | | are used by the | > | | | | | | Hash-based flow | > | | | | | | Selection | > | | | | | | Selector as the | > | | | | | | Hash Domain. | > +-----+----------------+--------+---------+-------+-----------------+ > > Table 3: Flow Selection methods > > > > > D'Antonio, et al. Expires October 25, 2012 [Page 31] > > Internet-Draft Flow Selection Techniques April 2012 > > > 8.2. Registration of Object Identifier > > IANA will register the following OID in the IPFIX-SELECTOR-MIB > Functions sub-registry at http://www.iana.org/assignments/smi-numbers > according to the procedures set forth in [I-D.dkcm-ipfix-rfc5815bis] Should be http://tools.ietf.org/html/draft-ietf-ipfix-rfc5815bis-03 Btw, this draft is right now in AUTH48 See http://www.rfc-editor.org/authors/rfc6632.txt http://www.rfc-editor.org/authors/rfc6615.txt Btw, you must mention that you want a new entry under Sub-registry Name: IPFIX-SELECTOR-MIB Functions See http://tools.ietf.org/html/draft-ietf-ipfix-psamp-mib-04#section-8 for an example > > +---------+-----------------------+---------------------+-----------+ > | Decimal | Name | Description | Reference | > +---------+-----------------------+---------------------+-----------+ > | 1 | flowSelectorAlgorithm | This Object | [RFCyyyy] | > | | | Identifier | | > | | | identifies the flow | | > | | | selection method | | > | | | (e.g., Filtering, | | > | | | Sampling) that is | | > | | | applied by the Flow | | > | | | Selection Process | | > +---------+-----------------------+---------------------+-----------+ > > Table 4: Information Elements to be registered You can't use the value 1. IANA will decide what is the next available value. > > Editor's Note (to be removed prior to publication): the RFC editor is > asked to replace "yyyy" in this document by the number of the RFC > when the assignment has been made. > > > 9. Security Considerations > > The described flow sampling techniques and the hash-based flow > filtering technique aim at the selection of a representative subset > of flows in order to make an accurate estimation of the population. Not always, right? For example, it's not the goal of "Property Match Filtering" > An adversary may have incentives to influence the selection of flows, > for example to circumvent accounting. > > Security considerations concerning the choice of a Hash Function for > Hash-based Packet Selection have been discussed in Section 6.2.3 of > [RFC5475] and are also appropriate for Hash-Based Flow Selection. > This section discussed a number of potential attacks to craft Streams I see Observed Packet Stream and Packet Stream in RFC5475, I see Data Stream in RFC5470, but not Stream alone. I guess you want to say Packet Stream > that are disproportionately detected and/or discover the Hash > Function parameters, the vulnerabilities of different Hash Functions > to these attacks, and practices to minimize these vulnerabilities. > > For other sampling approaches a user can gain knowledge about the > start and stop triggers in time-based systematic Sampling, e.g., by > sending test packets. This knowledge might allow users to modify > their send schedule in a way that their packets are > disproportionately selected or not selected. For random Sampling, a > cryptographically strong random number generator should be used in > > > > D'Antonio, et al. Expires October 25, 2012 [Page 32] > > Internet-Draft Flow Selection Techniques April 2012 > > > order to prevent that an advisory can predict the selection decision > [GoRe07]. > > Further security threats can occur when Sampling parameters are > configured or communicated to other entities. The protocol(s) for > the configuration and reporting of Sampling parameters are out of > scope of this document. Nevertheless, a set of initial requirements > for future configuration and reporting protocols are stated below: > > 1. Protection against disclosure of configuration information: Flow Flow -> Flow Sampling > selection configuration information describes the selection > process and its parameters. This information can be useful to > attackers. Attackers may craft packets that never fit the > selection criteria in order to prevent flows to be seen by the > selection process. They can also craft a lot of packets that fit > the selection criteria and overload or bias subsequent processes. > Therefore any transmission of configuration data (e.g., to > configure a process or to report its actual status) should be > protected by encryption. > > 2. Protection against modification of configuration information: If > wrong configuration information is sent to the flow selection > process, it can lead to a malfunction of the selection process. > Also if wrong configuration information is reported from the > selection process to other processes it can lead to wrong > estimations at subsequent processes. Therefore any protocol that > transmits configuration information should prevent that an > attacker can modify configuration information. Data integrity > can be achieved by authenticating the data. > > 3. Protection against malicious nodes sending configuration > information: The remote configuration of flow selection methods > should be protected against access by unauthorized nodes. This > can be achieved by access control lists at the selection devices selection devices? Do you mean IPFIX Exporter? > and source authentication. The reporting of configuration data > from a selection process has to be protected in the same way. > That means that also protocols that report configuration data > from the selection process to other processes need to protect > against unauthorized nodes reporting configuration information. > > The security threats that originate from communicating configuration > information to and from selection processes cannot be assessed solely > with the information given in this document. A further more detailed > assessment of security threats is necessary when a specific protocol > for the configuration or reporting configuration data is proposed. > > > > > > > D'Antonio, et al. Expires October 25, 2012 [Page 33] > > Internet-Draft Flow Selection Techniques April 2012 > > > 10. Acknowledgments > > We would like to thank the IPFIX group, especially Brian Trammell, > Paul Aitken and Benoit Claise for fruitful discussions and for > proofreading the document. > > > 11. References > > 11.1. Normative References > > [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate > Requirement Levels", BCP 14, RFC 2119, March 1997. > > [RFC5101] Claise, B., "Specification of the IP Flow Information > Export (IPFIX) Protocol for the Exchange of IP Traffic > Flow Information", RFC 5101, January 2008. > > [RFC5102] Quittek, J., Bryant, S., Claise, B., Aitken, P., and J. > Meyer, "Information Model for IP Flow Information Export", > RFC 5102, January 2008. > > [RFC5475] Zseby, T., Molina, M., Duffield, N., Niccolini, S., and F. > Raspall, "Sampling and Filtering Techniques for IP Packet > Selection", RFC 5475, March 2009. > > [RFC5476] Claise, B., Johnson, A., and J. Quittek, "Packet Sampling > (PSAMP) Protocol Specifications", RFC 5476, March 2009. > > 11.2. Informative References > > [Bra75] Brayer, K., "Evaluation of 32 Degree Polynomials in Error > Detection on the SATIN IV Autovon Error Patterns", > National Technical Information Service p.74, August 1975. > > [CoHa08] Cormode, G. and M. Hadjieleftheriou, "Finding frequent > items in data streams", Journal, Proceedings of the Very > Large DataBase Endowment VLDB Endowment, Volume 1 Issue 2, > August 2008, August 2008. > > [DuLT01] Duffield, N., Lund, C., and M. Thorup, "Charging from > Sampled Network Usage", ACM Internet Measurement Workshop > IMW 2001, San Francisco, USA, November 2001. > > [EsVa01] Estan, C. and G,. Varghese, "New Directions in Traffic > Measurement and Accounting: Focusing on the Elephants, > Ignoring the Mice", ACM SIGCOMM Internet Measurement > Workshop 2001, San Francisco (CA), November 2001. > > > > D'Antonio, et al. Expires October 25, 2012 [Page 34] > > Internet-Draft Flow Selection Techniques April 2012 > > > [KaPS03] Karp, R., Papadimitriou, C., and S. S. Shenker, "A simple > algorithm for finding frequent elements in sets and > bags.", ACM Transactions on Database Systems, Volume 28, > 51-55, 2003, March 2003. > > [MSZC10] Mai, J., Sridharan, A., Zang, H., and C. Chuah, "Fast > Filtered Sampling", Computer Networks Volume 54, Issue 11, > Pages 1885-1898, ISSN 1389-1286, January 2010. > > [MaMo02] Manku, G. and R. Motwani, "Approximate Frequency Counts > over Data Streams", Proceedings of the International > Conference on Very large DataBases (VLDB) pages 346--357, > 2002, Hong Kong, China, 2002. > > [RFC3917] Quittek, J., Zseby, T., Claise, B., and S. Zander, > "Requirements for IP Flow Information Export (IPFIX)", > RFC 3917, October 2004. > > [RFC5226] Narten, T. and H. Alvestrand, "Guidelines for Writing an > IANA Considerations Section in RFCs", BCP 26, RFC 5226, > May 2008. > > [RFC5470] Sadasivan, G., Brownlee, N., Claise, B., and J. Quittek, > "Architecture for IP Flow Information Export", RFC 5470, > March 2009. > > [RFC6183] Kobayashi, A., Claise, B., Muenz, G., and K. Ishibashi, > "IP Flow Information Export (IPFIX) Mediation: Framework", > RFC 6183, April 2011. > > [iana-ipfix-assignments] > "IP Flow Information Export Information Elements", 2007, > . > > > Authors' Addresses > > Salvatore D'Antonio > University of Napoli "Parthenope" > Centro Direzionale di Napoli Is. C4 > Naples 80143 > Italy > > Phone: +39 081 5476766 > Email: salvatore.dantonio@uniparthenope.it > > > > > > > D'Antonio, et al. Expires October 25, 2012 [Page 35] > > Internet-Draft Flow Selection Techniques April 2012 > > > Tanja Zseby > CAIDA/FhG FOKUS > San Diego Supercomputer Center (SDSC) > University of California, San Diego (UCSD) > 9500 Gilman Drive > La Jolla CA 92093-0505 > USA > > Email: tanja@caida.org > > > Christian Henke > Tektronix Communication Berlin > Wohlrabedamm 32 > Berlin 13629 > Germany > > Phone: +49 17 2323 8717 > Email: christian.henke@tektronix.com > > > Lorenzo Peluso > University of Napoli > Via Claudio 21 > Napoli 80125 > Italy > > Phone: +39 081 7683821 > Email: lorenzo.peluso@unina.it > > > > > > > > > > > > > > > > > > > > > > > D'Antonio, et al. Expires October 25, 2012 [Page 36] > > --------------060801090201060907020606 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Dear authors,

I'm performing the (new) AD review of  draft-ietf-ipfix-flow-selection-tech-10.txt
Lucky you, an extra pair of eyes specifically looking at your draft

If some points have been discussed already on the mailing list, let me know. I have to admit that I have not been following the latest iterations of this draft.

IMHO, this document needs some more work...
I don't think that this document is really in line with the other Intermediate Processes documents:
    http://tools.ietf.org/html/rfc6235
    http://tools.ietf.org/html/draft-ietf-ipfix-a9n-03
Note that I might have some more comments once all the points in this email are addressed, as there are many ;-)
However, I'm available for a conf. call to clarify my points if you want to

See in-line.

Internet Engineering Task Force                             S. D'Antonio
Internet-Draft                                      University of Napoli
Intended status: Standards Track                            "Parthenope"
Expires: October 25, 2012                                       T. Zseby
                                                         CAIDA/FhG FOKUS
                                                                C. Henke
                                          Tektronix Communication Berlin
                                                               L. Peluso
                                                    University of Napoli
                                                          April 23, 2012


                       Flow Selection Techniques
              draft-ietf-ipfix-flow-selection-tech-11.txt

Abstract

   Flow selection is the process of selecting a subset of flows from all
   observed flows.  The Flow Selection Process may be located at an
   observation point, or on an IPFIX Mediator.  Flow selection reduces
   the effort of post-processing flow data and transferring Flow
   Records.  This document describes motivations for flow selection and
   presents flow selection techniques.  It provides an information model
   for configuring flow selection techniques and discusses what
   information about a flow selection process should be exported.
- Be consistent with terms capitalization. Flow, Observation Point, for example, are not capitalized in the abstract
The following paragraph is good and consistent with the other IPFIX documents, but make sure you include all the terms that you need.
   This document is consistent with the terminology introduced in
   [RFC5101], [RFC5470], [RFC5475] and [RFC3917].  As in [RFC5101] and
   [RFC5476], the first letter of each IPFIX-specific and PSAMP-specific
   term is capitalized along with the flow selection specific terms
   defined here.

Requirements Language

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in RFC 2119 [RFC2119].

Status of this Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at http://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on October 25, 2012.



D'Antonio, et al.       Expires October 25, 2012                [Page 1]

Internet-Draft          Flow Selection Techniques             April 2012


Copyright Notice

   Copyright (c) 2012 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

   This document may contain material from IETF Documents or IETF
   Contributions published or made publicly available before November
   10, 2008.  The person(s) controlling the copyright in some of this
   material may not have granted the IETF Trust the right to allow
   modifications of such material outside the IETF Standards Process.
   Without obtaining an adequate license from the person(s) controlling
   the copyright in such materials, this document may not be modified
   outside the IETF Standards Process, and derivative works of it may
   not be created outside the IETF Standards Process, except to format
   it for publication as an RFC or to translate it into languages other
   than English.

























D'Antonio, et al.       Expires October 25, 2012                [Page 2]

Internet-Draft          Flow Selection Techniques             April 2012


Table of Contents

   1.  Scope  . . . . . . . . . . . . . . . . . . . . . . . . . . . .  4
   2.  Terminology  . . . . . . . . . . . . . . . . . . . . . . . . .  4
   3.  Difference between Flow Selection and Packet Selection . . . .  7
   4.  Flow selection as a Function in the IPFIX Architecture . . . .  8
     4.1.  Flow selection during the Metering Process . . . . . . . . 10
     4.2.  Flow selection during the Exporting Process  . . . . . . . 10
     4.3.  Flow selection as a function of the IPFIX Mediator . . . . 10
   5.  Flow Selection Techniques  . . . . . . . . . . . . . . . . . . 11
     5.1.  Flow Filtering . . . . . . . . . . . . . . . . . . . . . . 11
       5.1.1.  Property Match Filtering . . . . . . . . . . . . . . . 11
       5.1.2.  Hash-based Flow Filtering  . . . . . . . . . . . . . . 12
     5.2.  Flow Sampling  . . . . . . . . . . . . . . . . . . . . . . 12
       5.2.1.  Systematic sampling  . . . . . . . . . . . . . . . . . 12
       5.2.2.  Random Sampling  . . . . . . . . . . . . . . . . . . . 13
     5.3.  Flow-state Dependent Flow Selection  . . . . . . . . . . . 13
     5.4.  Flow-state Dependent Packet Selection  . . . . . . . . . . 14
   6.  Configuration of Flow Selection Techniques . . . . . . . . . . 14
     6.1.  Flow Selection Parameters  . . . . . . . . . . . . . . . . 16
     6.2.  Description of Flow-state Dependent Packet Selection . . . 18
   7.  Information Model for Flow Selection Configuration and
       Reporting  . . . . . . . . . . . . . . . . . . . . . . . . . . 18
     7.1.  flowSelectorAlgorithm  . . . . . . . . . . . . . . . . . . 20
     7.2.  flowSelectedOctetDeltaCount  . . . . . . . . . . . . . . . 21
     7.3.  flowSelectedPacketDeltaCount . . . . . . . . . . . . . . . 21
     7.4.  flowSelectedFlowDeltaCount . . . . . . . . . . . . . . . . 21
     7.5.  selectorIDTotalFlowsObserved . . . . . . . . . . . . . . . 22
     7.6.  selectorIDTotalFlowsSelected . . . . . . . . . . . . . . . 22
     7.7.  samplingFlowInterval . . . . . . . . . . . . . . . . . . . 22
     7.8.  samplingFlowSpace  . . . . . . . . . . . . . . . . . . . . 23
     7.9.  flowSamplingTimeInterval . . . . . . . . . . . . . . . . . 23
     7.10. flowSamplingTimeSpace  . . . . . . . . . . . . . . . . . . 24
     7.11. hashFlowDomain . . . . . . . . . . . . . . . . . . . . . . 24
   8.  IANA Considerations  . . . . . . . . . . . . . . . . . . . . . 24
     8.1.  Registration of Information Elements . . . . . . . . . . . 24
     8.2.  Registration of Object Identifier  . . . . . . . . . . . . 32
   9.  Security Considerations  . . . . . . . . . . . . . . . . . . . 32
   10. Acknowledgments  . . . . . . . . . . . . . . . . . . . . . . . 34
   11. References . . . . . . . . . . . . . . . . . . . . . . . . . . 34
     11.1. Normative References . . . . . . . . . . . . . . . . . . . 34
     11.2. Informative References . . . . . . . . . . . . . . . . . . 34
   Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 35
Don't you have to include the non-normative XML in the appendix, as it was done for RFC5102, RFC5103?








D'Antonio, et al.       Expires October 25, 2012                [Page 3]

Internet-Draft          Flow Selection Techniques             April 2012


1.  Scope

   This document describes flow selection techniques for network traffic
   measurements.  A flow is defined as a set of packets with common
   properties as described in [RFC5101].  Flow selection can be done to
   limit the resource demands for capturing, storing, exporting and
   post-processing of Flow Records.  It also can be used to select a
   particular set of flows that are of interest to a specific
   application.  This document provides a categorization of flow
   selection techniques and describes configuration and reporting
   parameters for them.  In order to be compliant with this document, at
   least one of the flow selection schemes MUST be implemented.  That
   means that the configuration parameters as well as the reporting
   Information Elements for this particular scheme MUST be supported.
Last sentence could be fine, but express that the way to configure the Intermediate Flow Selection Process is out of scope of this document.

   This document also addresses configuration and reporting parameters
   for flow-state dependent packet selection as described in [RFC5475],
   although this technique is categorized as packet selection.  The
   reason is that flow-state dependent packet selection techniques often
   aim at the reduction of resources for flow capturing and flow
   processing.  Furthermore, they were only briefly discussed in
Not sure what "they" refers to
   [RFC5475].  Therefore we included configuration and reporting
   considerations for such techniques in this document.

please remove "we", "us", "our" from the draft.


2.  Terminology

   This document is consistent with the terminology introduced in
   [RFC5101], [RFC5470], [RFC5475] and [RFC3917].  As in [RFC5101] and
   [RFC5476], the first letter of each IPFIX-specific and PSAMP-specific
   term is capitalized along with the flow selection specific terms
   defined here.

   * Packet Classification

      Packet Classification is a process by which packets are mapped to
      specific Flow Records based on packet properties or external
      properties (e.g. interface).  The properties (e.g. header
      information, packet content, AS number) make up the Flow Key. In
      case a Flow Record for a specific Flow Key already exists the Flow
      Record is updated, otherwise a new Flow Record is created.

How is this different that the Metering Process (RFC5101)?
   Metering Process

      The Metering Process generates Flow Records.  Inputs to the
      process are packet headers and characteristics observed at an
      Observation Point, and packet treatment at the Observation Point
      (for example, the selected output interface).

      The Metering Process consists of a set of functions that includes
      packet header capturing, timestamping, sampling, classifying, and
      maintaining Flow Records.

      The maintenance of Flow Records may include creating new records,
      updating existing ones, computing Flow statistics, deriving
      further Flow properties, detecting Flow expiration, passing Flow
      Records to the Exporting Process, and deleting Flow Records.
What is the connection with the Metering Process?
Figure 1 seems to suggest that Packet Classification is a subset of the Metering Process...



   * Packet Aggregation Process

      In the IPFIX Metering Process the Packet Aggregation Process
      aggregates packet data into flow data and forms the Flow Records.
How is this different from the Metering Process?
      After the aggregation step only the aggregated flow information is
      available.  Information about individual packets is lost.



D'Antonio, et al.       Expires October 25, 2012                [Page 4]

Internet-Draft          Flow Selection Techniques             April 2012


   * Flow Selection Process

      A Flow Selection Process takes Flow Records as its input and
      selects a subset of this set as its output.  A Flow Selection
      Process MAY run in several places within the IPFIX architecture.
      A Flow Selection Process MAY be part of an IPFIX Metering Process,
      Exporting Process or as an Intermediate Selection Process as
      defined for the IPFIX Mediator [RFC6183].

If you look at the RFC6235, you will see 
   Intermediate Anonymization Process:   An intermediate process that
      takes Data Records and transforms them into Anonymized Data
      Records.
To be perfect correct, the correct way is like it's done in http://tools.ietf.org/html/draft-ietf-ipfix-a9n-03

   Intermediate Aggregation Process:   an Intermediate Process as in
      [RFC6183] that aggregates records, based upon a set of Flow Keys
      or functions applied to fields from the record.
What should be done here is 
Intermediate Flow Selection Process: an Intermediate Process as in
      [RFC6183] that ...
Btw, you will see that "Intermediate Flow Selection Process" is already defined in the figure where you get it right.

Finally, your sentence "A Flow Selection Process MAY be part of an IPFIX Metering Process,  Exporting Process or as an Intermediate Selection Process as
defined for the IPFIX Mediator [RFC6183]. ":
- must contain "A Intermediate Flow Selection Process MAY be part of an IPFIX Metering Process ..."
- must be reflected in figure 1, where I only see the Intermediate Flow Selection Process in the IPFIX Mediator. Could also be present in Exporting Process and in the Metering Process. As figure 1 is the only figure, all possibilities must be displayed. See as an example http://tools.ietf.org/html/draft-ietf-ipfix-a9n-03 figure 1


   * Flow Selection State

      A Flow Selection Process SHOULD maintain state information for use
normally, we try to avoid MAY/SHOULD/MUST in definition
      by the Flow Selector.  At a given time, the Flow Selection State
      may depend on flows and packets observed at and before that time,
      as well as other variables.  Examples include:

        (i)   sequence number of packets and accounted Flow Records;

        (ii)  number of selected flows;

        (iii) number of observed flows;

        (iv)  current flow cache occupancy;

        (v)   flow specific counters, lower and upper bounds;

        (vi)  flow selection timeout intervals.

   * Flow Selector

      A Flow Selector defines the action of a Flow Selection Process on
      a single flow of its input.  The Flow Selector can make use of the
      following information in order to establish whether a flow has to
      be selected or not:

        (i)   the content of the Flow Record;

        (ii)  any state information related to the Metering Process or
              Exporting Process;

        (iii) any Flow Selection State that may be maintained by the
              Flow Selection Process.
I see many terms: 
Packet Classification, Packet Aggregation Process, Flow Selection Process, Flow Selection State, Intermediate Process, Intermediate Selection Process, etc...
Please have a new figure that put combines all these terms. Something such as the figure in section 3.1 from http://tools.ietf.org/html/rfc5474
It might be your figure 1, but I don't even see those terms in there, or figure 3 in http://tools.ietf.org/html/draft-ietf-ipfix-configuration-model-10
   * Complete Flow

      A Complete Flow consists of all the packets that enter the Flow
      Selection Process within the flow time-out interval, and which
      belong to the same flow as defined by the flow definition in



D'Antonio, et al.       Expires October 25, 2012                [Page 5]

Internet-Draft          Flow Selection Techniques             April 2012


      [RFC5470].  For this definition only packets that arrive at the
      Flow Selection Process are considered.  That means, packets that
      are not observed at the Flow Selection Process because of prior
      packet selection or packet loss are not considered as belonging to
      the Complete Flow.
what does the second sentence add?

   * Flow Filtering

      Flow Filtering selects flows based on a deterministic function on
      the Flow Record content, Flow Selection State, external properties
      (e.g. ingress interface) or external events (e.g violated Access
      Control List).  If the relevant parts of the Flow Record content
      can already be observed at packet level (e.g.  Flow Keys from
      packet header fields) Flow Filtering can be performed at packet
      level by Property Match Filtering as described in [RFC5475].

   * Hash-based Flow Filtering

      Hash-based Flow Filtering is a deterministic flow filter function
      that selects flows based on a Hash Function.  The Hash Function is
      calculated over parts of the Flow Record content or external
      properties which are called the Hash Domain.  If the hash value
      falls into a predefined Hash Selection Range the flow is selected.
      Hash-based Flow Filtering can already applied at packet level, in
      which case the Hash Domain MUST contain the Flow Key of the
      packet.  In case Hash-based Flow Filtering is used to select the
      same subset of flows at different observation points, the Hash
      Domain MUST comprise parts of the packet or flow thar are
      invariant on the packet/flow path.  Also refer to the according
      Trajectory Sampling Application Example on packet level in
      [RFC5475]

   * Flow-state Dependent Flow Selection

      Flow-state Dependent Flow Selection is a selection function that
      selects or drops flows based on the current Flow Selection State.
      The selection can be either deterministic, random or non-uniform
      random.

   * Flow-state Dependent Packet Selection

      Flow-state Dependent Packet Selection is a selection function that
      selects or drops packets based on the current Flow Selection
      State.  The selection can be either deterministic, random or non-
      uniform random.  Flow-state Dependent Packet Selection can be used
      to prefer the selection of packets belonging to specific flows.
      For example the selection probability of packets belonging to
      flows that are already within the Flow Cache may be higher than



D'Antonio, et al.       Expires October 25, 2012                [Page 6]

Internet-Draft          Flow Selection Techniques             April 2012


      for packets that have not been recorded yet.

   * Flow Sampling

      Flow Sampling selects flows based on Flow Record sequence or
      arrival times (e.g. entry in flow cache, arrival time at Exporter
      or Mediator).  The selection can be systematic (e.g. every n-th
      flow) or based on a random function (e.g. select each Flow Record
      with probability p, or randomly select n out of N Flow Records).


3.  Difference between Flow Selection and Packet Selection

   Flow selection differs from packet selection described in [RFC5475].
   Packet selection techniques consider packets as the basic element and
   the parent population consists of all packets observed at an
   observation point.  In contrast to this the basic elements in flow
   selection are the flows.  The parent population consists of all
   observed flows and the selection process operates on the flows.  The
   major characteristics of flow selection are the following:

   -       Flow selection takes flows as basic elements.  For packet
           selection, packets are considered as basic elements.

   -       Flow selection can only take place after Packet
           Classification, because the classification rules determine to
           which flow a packet belongs.  Packet selection can be applied
           before and after Packet Classification.
I don't understand the last sentence.

   -       Flow selection operates on Complete Flows.  That means that
           after the Flow Selection Process either all packets of the
           flow are kept or all packets of the flow are discarded.  That
           means that if the flow selection is preceded by a packet
           selection process the Complete Flow consists only of the
           packets that were not discarded during the packet selection.

   There are some techniques that are difficult to unambiguously
   categorize into one of the categories.  Here we give some guidance
   how to categorize such techniques:

   -       Techniques that can be considered as both packet and flow
           selection: some packet selection techniques result in the
           selection of Complete Flows and therefore can be considered
           as packet or as flow selection at the same time.  An example
           is Property Match Filtering of all packets to a specific
           destination address.  If flows are defined based on
           destination addresses, such a packet selection also results
           in a flow selection and can be considered as packet or flow



D'Antonio, et al.       Expires October 25, 2012                [Page 7]

Internet-Draft          Flow Selection Techniques             April 2012


           selection.

   -       Flow-state Dependent Packet Selection (as described in
           [RFC5475]): there exist techniques that select packets based
           on the flow state, e.g. based on the number of already
           observed packets belonging to the flow.  Examples of these
           techniques from the literature are "Sample and Hold" [EsVa01]
           "Fast Filtered Sampling" [MSZC10] or the "Sticky Sampling"
           algorithm presented in [MaMo02].  Such techniques can be used
           to influence which flows are captured (e.g. increase the
           selection of packets belonging to large flows) and reduce the
           number of flows that need to be stored in the flow cache.
           Nevertheless, such techniques do not necessarily select
           Complete Flows, because they do not ensure that all packets
           of a selected flow are captured.  Therefore Flow-state
           Dependent Packet Selection methods that do not ensure that
           either all or no packets of a flow are selected strictly
           speaking have to be considered as packet selection techniques
           and not as flow selection techniques.


4.  Flow selection as a Function in the IPFIX Architecture

   Figure 1 shows the IPFIX reference model as defined in [RFC5470] and
   shows the Packet Classification and Packet Aggregation Process in the
   Metering Process.

























D'Antonio, et al.       Expires October 25, 2012                [Page 8]

Internet-Draft          Flow Selection Techniques             April 2012


                       Packet(s) coming in to Observation Point(s)
                         |                                     |
                         v                                     v
        +----------------+---------------------------+   +-----+-------+
        |          Metering Process                  |   |             |
        |                                            |   |             |
        |   packet header capturing                  |   |             |
        |        |                                   |...| Metering    |
        |   timestamping                             |   | Process N   |
        |        |                                   |   |             |
        |   packet sampling                          |   |             |
        |        |                                   |   |             |
        |   (packet classification)                  |   |             |
        |        |                                   |   |             |
        |   packet filtering*                        |   |             |
        |        |                                   |   |             |
        |   (packet aggregation)*                    |   |             |
        |        |                                   |   |             |
        +--------|-----------------------------------+   +-----|-------+
            Flow Records                                   Flow Records
                 |                                             |
                 +----------------------+----------------------+
                                        |
                 +----------------------|-----------------+
                 | Exporting Process*                     |
                 +----------------------+-----------------+
                                        |  IPFIX (Flow Records)
                                        v
              +-------------------------|-----------------------+
              |  IPFIX Mediator         |                       |
              |                         v                       |
              |               Collecting Process(es)            |
              |                         |                       |
              |      Intermediate Flow Selection Process (*)    |
              |                         |                       |
              |               Exporting Process(es) (*)             |
              +-------------------------|-----------------------+
                                        v
                                      IPFIX

         (*) indicates where flow selection can take place.

            Figure 1: Flow selection in the IPFIX Architecture
Please express the physical boundary between the Exporter and the IPFIX Mediator

Why is packet classification in brackets?
(packet aggregation), do you mean the Intermediate Aggregation Process in http://tools.ietf.org/html/draft-ietf-ipfix-a9n-03?


   In contrast to packet selection, flow selection is always applied
   after the packets are classified into flows.  Flows can be selected
   at different stages of the measurement chain:




D'Antonio, et al.       Expires October 25, 2012                [Page 9]

Internet-Draft          Flow Selection Techniques             April 2012


   1.  during the Metering Process

   2.  during Exporting Process(es)

   3.  during an Intermediate Selection Process on a Mediator
It's not "during" but "in".
Please display the 3 points above in the figure


4.1.  Flow selection during the Metering Process

   In the Packet Aggregation Process the packet information is used to
   update the Flow Records in the flow cache.  Flow selection that is
   applied before aggregation equals a packet selection process.  The
   flow still consists of individual packets.  Those are then selected
   based on the classification information, i.e. based on the flow they
   belong to.  Flow selection before aggregation can be based on the
   fields of the Flow Key (also on a hash value over these fields), but
   not based on characteristics that are only available after packet
   aggregation (e.g. flow size, flow duration).  Flow selection during
   the Metering Process is applied to reduce resources for all
   succeeding processes or to select specific flows of interest in case
   such flow characteristics are already observable at packet level
   (e.g. flows to specific IP addresses).  In contrast, Flow-state
   Dependent Packet Selection is a packet selection method, because it
   does not necessarily select Complete Flows.

4.2.  Flow selection during the Exporting Process

   The Flow Selection Process at the Exporter is similar to an
   Intermediate Selection Process as described in [RFC6183] and works on
   Flow records.  Flow selection during the Exporting Process can
   therefore also depend on flow characteristics that are only visible
   after the aggregation of packets, such as flow size and flow
   duration. 
Why can't this be done in the Metering Process as well?
The Exporting Process may implement policies for exporting
   only a subset of the Flow Records which have been stored in the
   system memory in order to unload flow export and flow post-
   processing.  Flow selection during the Exporting Process may select
   only the subset of Flow Records which are of interest to the users
   application, or select only as many Flow Records as can be handled by
   the available resources (e.g. limited export link capacity).

4.3.  Flow selection as a function of the IPFIX Mediator

   As shown in Figure 1, flow selection can be performed as an
Please rewrite these section 4.* based on the Intermediate Flow Selection Process, and not flow selection, which is not defined.
   Intermediate Process within an IPFIX Mediator [RFC6183].  The
   Intermediate Selection Process takes Flow Record stream as its input
   and selects Flow Records from a sequence based upon criteria-
   evaluated record values.  The Intermediate Selection Process can
   again apply a flow selection technique to obtain flows of interest to
   the application.  Further, the Intermediate Selection Process can



D'Antonio, et al.       Expires October 25, 2012               [Page 10]

Internet-Draft          Flow Selection Techniques             April 2012


   base its selection decision on the correlation of data from different
   observation points,
or most of the time Exporters (which btw, you should show this in the figure)
See figure A in RFC 6183
e.g. by only selecting flows that were at least
   recorded on two observation points.



5.  Flow Selection Techniques

   A flow selection technique selects either all or none of the packets
I see sometimes flow selection, sometimes flow selection technique, sometimes selection technique, sometimes flow selection scheme, sometimesflow selection methods, sometimes Flow Selection Process which should really be the Intermediate Flow Selection Process...
It's difficult to read the draft, as it misses some cohesion: this clearly shows that different parts of the document have been written by different persons.
Please review the entire document with a fresh mind, as if you would discover for the first time, and you will see what I mean.
The entire draft needs some improvements on that matter.

   of a flow, otherwise the technique has to be considered as packet
   selection.  We distinguish between Flow Filtering and Flow Sampling.

5.1.  Flow Filtering

   Flow Filtering is a deterministic function on the IPFIX Flow Record
   content.  If the relevant flow characteristics are already observable
   at packet level (e.g.  Flow Keys), Flow Filtering can be applied
   before aggregation at packet level.  In order to be compliant with
   this document, at least the Property Match Filtering MUST be
   implemented.
This contradicts.
   In order to be compliant with this document, at
   least one of the flow selection schemes MUST be implemented.

5.1.1.  Property Match Filtering

   Property Match Filtering can be performed similarly to Property Match
   Filtering for packet selection described in [RFC5475].  The
   difference is that, instead of packet fields, Flow Record fields are
   here used to derive the selection decision.  Property Match Filtering
   is typically used to select a specific subset of the flows that are
   of interest to a particular application (e.g. all flows to a specific
   destination, all large flows, etc.).  Properties on which the
   filtering is based can be Flow Keys, Flow Timestamps, or Per-Flow
   Counters described in [RFC5102].  Examples of properties are the flow
   size in bytes, the number of packets in the flow, the observation
   time of the first or last packet, or the maximum packet length.  An
   example is to select flows with more than a threshold number of
   observed octets.  The selection criteria can be a specific value, a
   set of specific values, or an interval.  For example, a flow is
   selected if destinationIPv4Address and the total number of packets of
   the flow equal two predefined values.  Property Match Filtering can
   be applied during the Metering Process if the properties are already
   observable at the packet level (e.g.  Flow Key fields).  For example,
   a flow is selected if sourceIPv4Address and sourceIPv4PrefixLength
   equal, respectively, two specific values.

   There are content-based Property Match Filtering techniques that
   require a computation on the current flow cache.  An example is the
   selection of the largest flows or a percentage of flows with the
   longest lifetime.  This type of Property Match Filtering is also used
   in flow selection techniques that react to external events (e.g.



D'Antonio, et al.       Expires October 25, 2012               [Page 11]

Internet-Draft          Flow Selection Techniques             April 2012


   resource constraint).  For example when the flow cache is full, the
   Flow Record with the lowest flow volume per current flow life time
   may be deleted.

5.1.2.  Hash-based Flow Filtering

   Hash-based Flow Filtering uses a Hash Function h to map the Flow Key
   c onto a Hash Range R. A flow is selected if the hash value h(c) is
   within the Hash Selection Range S, which is a subset of R. Hash-based
   Flow Filtering can be used to emulate a random sampling process but
   still enable the correlation between selected flow subsets at
   different observation points.  Hash-based Flow Filtering is similar
   to Hash-based Packet Selection, and in fact is identical when Hash-
   based Packet Selection uses the Flow Key that defines the flow as the
   hash input.  Nevertheless there may be the incentive to apply Hash-
   based Flow Filtering not on the packet level during the Metering
   Process, for example when the size of the selection range and
   therefore the sampling probability is dependent on the number of
   observed flows.

5.2.  Flow Sampling

   Flow Sampling operates on Flow Record sequence or arrival times.  It
   can use either a systematic or a random function for the selection
   process.  Flow Sampling usually aims at the selection of a
   representative subset of all flows in order to estimate
   characteristics of the whole set (e.g. mean flow size in the
   network).

5.2.1.  Systematic sampling

   Systematic sampling is a deterministic selection function.
   Systematic sampling may be a periodic selection of the N-th Flow
   Record which arrives at the Exporting or Intermediate Selection
   Process. 
Intermediate Flow Selection Process
Systematic sampling MAY be applied during the Metering
   Process.  An example would be to create, besides the Flow cache of
   selected flows, an additional data structure that saves the Flow Keys
   of the flows that are not selected.  The selection of a flow would
   then be based on the first packet of a flow.  Everytime a packet
   belonging to a new flow (which is neither in the data structure of
   the selected or not selected flows) arrives at the measurement point,
what is a measurement point?
   a counter is increased.  In case the counter is increased to a
   multiple of N a new flow cache entry is created, and in case the
   counter is not a multiple of N the Flow Key is added to the data
   structure for not selected flows.

   Systematic sampling can also be time-based.  Time-based systematic
   sampling is applied by only creating flows that are observed between
flows -> Flows all over the doc.



D'Antonio, et al.       Expires October 25, 2012               [Page 12]

Internet-Draft          Flow Selection Techniques             April 2012


   time-based start and stop triggers.  The time interval may be applied
   at packet level during the Metering Process or after aggregation on
   flow level, e.g. by selecting a flow arriving at the Exporting
   Process every n seconds.

5.2.2.  Random Sampling

   Random flow sampling is based on a random process which requires the
   calculation of random numbers.  One can differentiate between n-out-N
   and probabilistic flow sampling.

5.2.2.1.  n-out-of-N Flow Sampling

   In n-out-of-N Sampling, n elements are selected out of the parent
   population that consists of N elements.  One example would be to
   generate n different random numbers in the range [1,N] and select all
   flows that have a flow position equal to one of the random numbers.

5.2.2.2.  Probabilistic Flow Sampling

   In probabilistic Sampling, the decision whether or not a flow is
   selected is made in accordance with a predefined selection
   probability.  For probabilistic Sampling, the Sample Size can vary
   for different trials.  The selection probability does not necessarily
   have to be the same for each flow.  Therefore, we distinguish between
   uniform probabilistic sampling (with the same selection probability
   for all flows) and non-uniform probabilistic sampling (where the
   selection probability can vary for different flows).  For non-uniform
   probabilistic Flow Sampling the sampling probability may be adjusted
   according to the Flow Record content.  An example would be to
   increase the selection probability of large volume flows over small
   volume flows as described in the Smart Sampling technique [DuLT01].

5.3.  Flow-state Dependent Flow Selection

   Flow-state Dependent Flow Selection can be a deterministic or random
   flow selection process based on the Flow Record content and the flow
   state which may be kept additionally for each of the flows.  External
   processes may update counters, bounds and timers for each of the Flow
   Records and the Flow Selection Process utilises this information for
   the selection decision.  A review of Flow-state Dependent Flow
   Selection techniques that aim at the selection of the most frequent
   items by keeping additional flow state information can be found in
   [CoHa08].  Flow-state Dependent Flow Selection can only be applied
   after packet aggregation, when a packet has been assigned to a flow.
   The selection process then decides based upon the flow state for each
   flow if it is kept in the flow cache or not.  Two Flow State
   Dependent Flow Selection Algorithms are here described:



D'Antonio, et al.       Expires October 25, 2012               [Page 13]

Internet-Draft          Flow Selection Techniques             April 2012


   The frequent algorithm [KaPS03] is a technique that aims at the
   selection of all flows that at least exceed a 1/k fraction of the
   Observed Packet Stream. 
to come back to one of my previous point, please add Observed Packet Stream to the extra figure
The algorithm has only a flow cache of size
   k-1 and each flow in the cache has an additional counter.  The
   counter is incremented each time a packet belonging to the flow in
   the flow cache is observed.  In case the observed packet does not
   belong to any flow all counters are decremented and if any of the
   flow counters has a value of zero the flow is replaced with a flow
   formed from the new packet.

   Lossy counting is a selection technique that identifies all flows
   whose packet count exceeds a certain percentage of the whole observed
   packet stream (e.g. 5% of all packets) with a certain estimation
   error e.  Lossy counting separates the observed packet stream in
   windows of size N=1/e, where N is an amount of consecutive packets.
   For each observed flow an additional counter will be held in the flow
   state.  The counter is incremented each time a packet belonging to
   the flow is observed and all counters are decremented at the end of
   each window and all flows with a counter of zero are removed from the
   flow cache.

5.4.  Flow-state Dependent Packet Selection

   Flow-state Dependent Packet Selection is not a flow selection
   technique but a packet selection technique.  Nevertheless we will
   describe configuration and reporting parameters for this technique in
   this document.  An example is the "Sample and Hold" algorithm
   [EsVa01] that tries to prefer large volume flows in the selection.
   When a packet arrives it is selected when a Flow Record for this
   packet already exists.  In case there is no Flow Record, the packet
   is selected by a certain probability that is dependent on the packet
   size.


6.  Configuration of Flow Selection Techniques

   This section describes the configuration parameters of the flow
   selection techniques presented above.  It provides the basis for an
   information model to be adopted in order to configure the Flow
   Selection Process within an IPFIX Device.  The actual information
   model with the Information Elements (IEs) for the configuration is
   described together with the reporting IEs in section 7.  The
   following table gives an overview of the defined selection
   techniques, where they can be applied and what their input parameters
   are.  Depending on where the flow selection techniques are applied
   different input parameters can be configured.

   Overview of Flow Selection Techniques:



D'Antonio, et al.       Expires October 25, 2012               [Page 14]

Internet-Draft          Flow Selection Techniques             April 2012


   +------------------+-----------------+------------------------------+
   | Location         | Selection       | Selection Input              |
   |                  | Method          |                              |
   +------------------+-----------------+------------------------------+
   | During the       | Flow-state      | packet sampling              |
location is not "during" but "in"
   | Metering Process | Dependent       | probabilities, Flow          |
   | based on Packets | Packet          | Selection State, packet      |
   |                  | Selection       | properties                   |
I still don't get why "in the Metering Process" means, on packets?
   +------------------+-----------------+------------------------------+
should
   |                  | Property Match  | Flow record IEs, Selection   |
   |                  | Flow Filtering  | Interval                     |
   +------------------+-----------------+------------------------------+
   |                  | Hash-based Flow | selection range, Hash        |
   |                  | Filtering       | Function, Flow Key, (seed)   |
   +------------------+-----------------+------------------------------+
   |                  | Time-based      | flow position (derived from  |
   |                  | Systematic Flow | arrival time of packets),    |
   |                  | Sampling        | flow selection state         |
   +------------------+-----------------+------------------------------+
   |                  | Sequence-based  | flow position (derived from  |
   |                  | Systematic Flow | packet position), flow       |
   |                  | Sampling        | selection state              |
   +------------------+-----------------+------------------------------+
   |                  | Random Flow     | random number generator or   |
   |                  | Sampling        | list and packet position,    |
   |                  |                 | flow state                   |
   +------------------+-----------------+------------------------------+
This location above applies to the 6 column, right? so make a big cell out for "in the Metering Process based on packets"
Same remark below.

   | Exporting /      | Property Match  | Flow Record content, filter  |
   | Intermediate     | Flow Filtering  | function                     |
   | Selection        |                 |                              |
   | Process          |                 |                              |
   +------------------+-----------------+------------------------------+
   |                  | Hash-based Flow | selection range, Hash        |
   |                  | Filtering       | Function, hash input (Flow   |
   |                  |                 | Keys and other flow          |
   |                  |                 | properties)                  |
   +------------------+-----------------+------------------------------+
   |                  | Flow-state      | flow state parameters,       |
   |                  | Dependent Flow  | random number generator or   |
   |                  | Selection       | list                         |
   +------------------+-----------------+------------------------------+
   |                  | Time-based      | flow arrival time, flow      |
   |                  | Systematic Flow | state                        |
   |                  | Sampling        |                              |
   +------------------+-----------------+------------------------------+
   |                  | Sequence-based  | flow position, flow state    |
   |                  | Systematic Flow |                              |
   |                  | Sampling        |                              |



D'Antonio, et al.       Expires October 25, 2012               [Page 15]

Internet-Draft          Flow Selection Techniques             April 2012


   |                  | Random Flow     | random number generator or   |
   |                  | Sampling        | list and flow position, flow |
   |                  |                 | state                        |
   +------------------+-----------------+------------------------------+
Not sure what the "flow position".
Below, you use "Spacing" for this concept.

               Table 1: Overview of Flow Selection Techniques

6.1.  Flow Selection Parameters

   In this section, we define what parameters are required to describe
   the most common Flow Selection techniques.
You see, yet another new term, which is not in the terminology: Flow Selection
And on the top of my head, it was not defined in any other documents referenced in the terminology

   Flow Selection Parameters:

   For Property Match Filtering:

   -   Information Element as specified in [iana-ipfix-assignments]):
       Specifies the Information Element which is used as the property
       in the filter expression.

   -   Selection Value or Value Interval:
       Specifies the value or interval of the filter expression.
       Packets and Flow Record that have a value equal to the Selection
       Value or within the Interval will be selected.

   For Hash-based Flow Filtering:

   -   Hash Domain:
       Specifies the bits from the packet or flow which are taken as the
       hash input to the Hash Function.

   -   Hash Function:
       Specifies the name of the Hash Function that is used to calculate
       the hash value.  Possible Hash Functions are BOB [RFC5475], IPSX
       [RFC5475], CRC-32 [Bra75]

   -   Hash Selection Range:
       Flows that have a hash value within the Hash Selection Range are
       selected.  The Hash Selection Range can be a value interval or
       arbitrary hash values within the Hash Range of the Hash Function.

   -   Random Seed or Initializer Value:
       Some Hash Functions require an initializing value.  In order to
       make the selection decision more secure one can choose a random
       seed that configures the hash function.

   For Flow-state Dependent Flow Selection:




D'Antonio, et al.       Expires October 25, 2012               [Page 16]

Internet-Draft          Flow Selection Techniques             April 2012


   -   frequency threshold:
       Specifies the frequency threshold s for flow state dependent flow
       selection techniques that try to find the most frequent items
       within a dataset.  All flows which exceed the defined threshold
       will be selected.

   -   accuracy parameter:
       specifies the accuracy parameter e for techniques that deal with
       the frequent items problems.  The accuracy parameter defines the
       maximum error, i.e. no flows that have a true frequency less than
       ( s - e) N are selected, where s is the frequency threshold and N
       is the total number of packets.

   The above list of parameters for Flow-state Dependent Flow Selection
   techniques is suitable for the presented frequent item and lossy
   counting algorithms.  Nevertheless a variety of techniques exist with
   very specific parameters which are not defined here.

   For Systematic time-based Flow Sampling:

   -   Interval length (in usec)
       Defines the length of the sampling interval during which flows
       are selected.

   -   Spacing (in usec)
       The spacing parameter defines the spacing in usec between the end
       of one sampling interval and the start of the next succeeding
       interval.

   For Systematic count-based Flow Sampling:

   -   Interval length
       Defines the number of flows that are selected within the sampling
       interval.

   -   Spacing
       The spacing parameter defines the spacing in number of observed
       flows between the end of one sampling interval and the start of
       the next succeeding interval.

   For random n-out-of-N Flow Sampling:

   -   Population Size N
       The Population Size N is the number of all flows in the
       Population from which the sample is drawn.






D'Antonio, et al.       Expires October 25, 2012               [Page 17]

Internet-Draft          Flow Selection Techniques             April 2012


   -   Sampling Size n
       The sampling size n is the number of flows that are randomly
       drawn from the population N.

   For probabilistic Flow Sampling:

   -   Sampling probability p
       The sampling probability p defines the probability by which each
       of the observed flows is selected.

6.2.  Description of Flow-state Dependent Packet Selection

   The configuration of Flow-state Dependent Packet Selection has not
   been described in [RFC5475] therefore the parameters are defined
   here:

   For Flow-state Dependent Packet Selection:

   -   packet selection probability per possible flow state interval
       Defines multiple {flow interval, packet selection probability}
       value pairs that configure the sampling probability depending on
       the current flow state.

   -   additional parameters
       For the configuration of flow state dependent packet selection
       additional parameters or packet properties may be required, e.g.
       the packet size ([EsVa01])


7.  Information Model for Flow Selection Configuration and Reporting

   In this section we describe Information Elements (IEs) that MUST be
This section specifies ...
   exported by a flow selection process in order to support the
Flow Selection Process
   interpretation of measurement results from flow measurements where
   only some flows are selected. 
"from flow measurements where only some flows are selected. "
Do you need that?  Isn't it by default what a Flow Selection Process does?
The information is mainly used to
   report how many packets and flows have been observed in total and how
   many of them were selected.  This helps for instance to calculate the
   Attained Selection Fraction (see also [RFC5476]), which is an
   important parameter to provide an accuracy statement.  The IEs can
   provide reporting information about Flow Records, packets or bytes.
   The reported metrics are total number of elements and the number of
   selected elements.  From this the number of dropped elements can be
   derived.  All counters SHOULD be exported and reset when a new
   measurement interval starts.
I disagree here. It depends which IEs you use: deltaCounter or totalCounter
Search for those two terms in http://www.iana.org/assignments/ipfix/ipfix.xml


   List of Flow Selection Information Elements:





D'Antonio, et al.       Expires October 25, 2012               [Page 18]

Internet-Draft          Flow Selection Techniques             April 2012


   +------+-------------------------+-------+--------------------------+
   | ID   | Name                    | ID    | Name                     |
   +------+-------------------------+-------+--------------------------+
   | 301  | selectionSequenceID     | 302   | selectorID               |
   +------+-------------------------+-------+--------------------------+
   | TBD1 | flowSelectorAlgorithm   | 1     | octetDeltaCount          |
   +------+-------------------------+-------+--------------------------+
   | TBD2 | flowSelectedOctetDeltaC | 2     | packetDeltaCount         |
   |      | ount                    |       |                          |
   +------+-------------------------+-------+--------------------------+
   | TBD3 | flowSelectedPacketDelta | 3     | originalFlowsPresent     |
   |      | Count                   |       |                          |
   +------+-------------------------+-------+--------------------------+
   | TBD4 | flowSelectedFlowDeltaCo | TBD5  | selectorIDTotalFlowsObse |
   |      | unt                     |       | rved                     |
   +------+-------------------------+-------+--------------------------+
   | TBD6 | selectorIDTotalFlowsSel | TBD7  | samplingFlowInterval     |
   |      | ected                   |       |                          |
   +------+-------------------------+-------+--------------------------+
   | TBD8 | samplingFlowSpace       | 309   | samplingSize             |
   +------+-------------------------+-------+--------------------------+
   | 310  | samplingPopulation      | 311   | samplingProbability      |
   +------+-------------------------+-------+--------------------------+
   | TBD9 | flowSamplingTimeInterva | TBD10 | flowSamplingTimeSpace    |
   |      | l                       |       |                          |
   +------+-------------------------+-------+--------------------------+
   | 326  | digestHashValue         | TBD11 | hashFlowOffset           |
   +------+-------------------------+-------+--------------------------+
   | TBD1 | hashFlowSize            | 329   | hashOutputRangeMin       |
   | 2    |                         |       |                          |
   +------+-------------------------+-------+--------------------------+
   | 330  | hashOutputRangeMax      | 331   | hashSelectedRangeMin     |
   +------+-------------------------+-------+--------------------------+
   | 332  | hashSelectedRangeMax    | 333   | hashDigestOutput         |
   +------+-------------------------+-------+--------------------------+
   | 334  | hashInitialiserValue    | 320   | absoluteError            |
   +------+-------------------------+-------+--------------------------+
   | 321  | relativeError           | 336   | upperCILimit             |
   +------+-------------------------+-------+--------------------------+
   | 337  | lowerCILimit            | 338   | confidenceLevel          |
   +------+-------------------------+-------+--------------------------+

               Table 2: Flow Selection Information Elements








D'Antonio, et al.       Expires October 25, 2012               [Page 19]

Internet-Draft          Flow Selection Techniques             April 2012


7.1.  flowSelectorAlgorithm

   Description:

      This Information Element identifies the flow selection
      method(e.g., Filtering, Sampling) that is applied by the Flow
      Selection Process.  Most of these methods have parameters as
      decribed in Section 6.  Further Information Elements are needed to
      fully specify packet selection with these methods and all their
      parameters. 
Why do you speak about "packet selection" in the flowSelectorAlgorithm?
Further method identifiers may be added to the list
      below.  It might be necessary to define new Information Elements
      to specify their parameters.  The flowSelectorAlgorithm registry
      is maintained by IANA.  New assignments for the registry will be
      administered by IANA and are subject to Expert Review [RFC5226].
      The registry can be updated when specifications of the new
      method(s) and any new Information Elements are provided.


   +----+------------------------+--------------------------+
   | ID |        Method          |      Parameters          |
   +----+------------------------+--------------------------+
   | 1  | Systematic count-based | flowSamplingInterval     |
   |    | Sampling               | flowSamplingSpace        |
   +----+------------------------+--------------------------+
   | 2  | Systematic time-based  | flowSamplingTimeInterval |
   |    | Sampling               | flowSamplingTimeSpace    |
   +----+------------------------+--------------------------+
   | 3  | Random n-out-of-N      | samplingSize             |
   |    | Sampling               | samplingPopulation       |
   +----+------------------------+--------------------------+
   | 4  | Uniform probabilistic  | samplingProbability      |
   |    | Sampling               |                          |
   +----+------------------------+--------------------------+
   | 5  | Property Match         | Information Element      |
   |    | Filtering              | Value Range              |
   +----+------------------------+--------------------------+
   |   Hash-based Filtering      | hashInitialiserValue     |
   +----+------------------------+ hashFlowDomain           |
   | 6  | using BOB              | hashSelectedRangeMin     |
   +----+------------------------+ hashSelectedRangeMax     |
   | 7  | using IPSX             | hashOutputRangeMin       |
   +----+------------------------+ hashOutputRangeMax       |
   | 8  | using CRC              |                          |
   +----+------------------------+--------------------------+
   | 9  | Flow State Dependent   | No agreed Parameters     |
   |    | Flow Selection         |                          |
   +----+------------------------+--------------------------+




D'Antonio, et al.       Expires October 25, 2012               [Page 20]

Internet-Draft          Flow Selection Techniques             April 2012


   Abstract Data Type: unsigned16

   ElementId: TBD1

   Data Type Semantics: identifier

   Status: Proposed

7.2.  flowSelectedOctetDeltaCount

   Description:

      This Information Element specifies the volume in octets of all
      flows that are selected during the Flow Selection Process since
      the previous report.

   Abstract Data Type: unsigned64

   ElementId: TBD2

   Units: Octets

   Status: Proposed

7.3.  flowSelectedPacketDeltaCount

   Description:

      This Information Element specifies the volume in packets of all
      flows that were selected during the Flow Selection Process since
      the previous report.

   Abstract Data Type: unsigned64

   ElementId: TBD3

   Units: Packets

   Status: Proposed

7.4.  flowSelectedFlowDeltaCount

   Description:

      This Information Element specifies the number of Flows that were
      selected during the Flow Selection Process since the last report.

   Abstract Data Type: unsigned64



D'Antonio, et al.       Expires October 25, 2012               [Page 21]

Internet-Draft          Flow Selection Techniques             April 2012


   ElementId: TBD4

   Units: Flows

   Status: Proposed

7.5.  selectorIDTotalFlowsObserved

   Description:

      This Information Element specifies the total number of flows
      observed by a Selector, for a specific value of SelectorId.  This
      Information Element should be used in an Options Template scoped
      to the observation to which it refers.  See Section 3.4.2.1 of the
      IPFIX protocol document [RFC5101] .

   Abstract Data Type: unsigned64

   ElementId: TBD5

   Units: Flows

   Status: Proposed

7.6.  selectorIDTotalFlowsSelected

   Description:

      This Information Element specifies the total number of flows
      selected by a Selector, for a specific value of SelectorId.  This
      Information Element should be used in an Options Template scoped
      to the observation to which it refers.  See Section 3.4.2.1 of the
      IPFIX protocol document [RFC5101].

   Abstract Data Type: unsigned64

   ElementId: TBD6

   Units: Flows

   Status: Proposed

7.7.  samplingFlowInterval

   Description:

      This Information Element specifies the number of flows that are
      consecutively sampled.  A value of 100 means that 100 consecutive



D'Antonio, et al.       Expires October 25, 2012               [Page 22]

Internet-Draft          Flow Selection Techniques             April 2012


      flows are sampled.  For example, this Information Element may be
      used to describe the configuration of a systematic count-based
      Sampling Selector.

   Abstract Data Type: unsigned64

   ElementId: TBD7

   Units: Flows

   Status: Proposed

7.8.  samplingFlowSpace

   Description:

      This Information Element specifies the number of flows between two
      "samplingFlowInterval"s.  A value of 100 means that the next
      interval starts 100 flows (which are not sampled) after the
      current "samplingFlowInterval" is over.  For example, this
      Information Element may be used to describe the configuration of a
      systematic count-based Sampling Selector.
The text above mentions:

   For Systematic count-based Flow Sampling:

   -   Interval length
       Defines the number of flows that are selected within the sampling
       interval.

   -   Spacing
       The spacing parameter defines the spacing in number of observed
       flows between the end of one sampling interval and the start of
       the next succeeding interval.

So be consistent between "Space", "Spacing", and position (see one of my previous comments"



   Abstract Data Type: unsigned64

   ElementId: TBD8

   Units: Flows

   Status: Proposed

7.9.  flowSamplingTimeInterval

   Description:

      This Information Element specifies the time interval in
      microseconds during which all arriving flows are sampled.  For
      example, this Information Element may be used to describe the
      configuration of a systematic time-based Sampling Selector.
The text above mentions:
   For Systematic time-based Flow Sampling:

   -   Interval length (in usec)
       Defines the length of the sampling interval during which flows
       are selected.

   -   Spacing (in usec)
       The spacing parameter defines the spacing in usec between the end
       of one sampling interval and the start of the next succeeding
       interval.

So be consistent between "Space", "Spacing", and position (see one of my previous comments"

   Abstract Data Type: unsigned64

   ElementId: TBD9

   Units: microseconds

   Status: Proposed




D'Antonio, et al.       Expires October 25, 2012               [Page 23]

Internet-Draft          Flow Selection Techniques             April 2012


7.10.  flowSamplingTimeSpace

   Description:

      This Information Element specifies the time interval in
      microseconds between two "flowSamplingTimeInterval"s.  A value of
      100 means that the next interval starts 100 microseconds (during
      which no flows are sampled) after the current
      "flowsamplingTimeInterval" is over.  For example, this Information
      Element may used to describe the configuration of a systematic
      time-based Sampling Selector.
Same remark.

   Abstract Data Type: unsigned64

   ElementId: TBD10

   Units: microseconds

   Status: Proposed

7.11.  hashFlowDomain

   Description:

      This Information Element specifies the Information Elements that
      are used by the Hash-based flow Selection Selector as the Hash
      Domain.

   Abstract Data Type: unsigned16

   ElementId: TBD11

   Data Type Semantics: identifier

   Status: Proposed


8.  IANA Considerations

8.1.  Registration of Information Elements

   IANA will register the following IEs in the IPFIX Information
   Elements registry at http://www.iana.org/assignments/ipfix/ipfix.xml:








D'Antonio, et al.       Expires October 25, 2012               [Page 24]

Internet-Draft          Flow Selection Techniques             April 2012


   +-----+----------------+--------+---------+-------+-----------------+
   | Val | Name           | Data   | Data    | Statu | Description     |
   | ue  |                | Type   | Type    | s     |                 |
   |     |                |        | Semanti |       |                 |
   |     |                |        | cs      |       |                 |
   +-----+----------------+--------+---------+-------+-----------------+
   | 1   | flowSelectorAl | unsign | identif | Propo | This            |
   |     | gorithm        | ed16   | ier     | sed   | Information     |
   |     |                |        |         |       | Element         |
   |     |                |        |         |       | identifies the  |
   |     |                |        |         |       | flow selection  |
   |     |                |        |         |       | method(e.g.,    |
   |     |                |        |         |       | Filtering,      |
   |     |                |        |         |       | Sampling) that  |
   |     |                |        |         |       | is applied by   |
   |     |                |        |         |       | the Flow        |
   |     |                |        |         |       | Selection       |
   |     |                |        |         |       | Process         |
   +-----+----------------+--------+---------+-------+-----------------+
   | 2   | flowSelectedOc | unsign | Octets  | Propo | This            |
   |     | tetDeltaCount  | ed64   |         | sed   | Information     |
   |     |                |        |         |       | Element         |
   |     |                |        |         |       | specifies the   |
   |     |                |        |         |       | volume in       |
   |     |                |        |         |       | octets of all   |
   |     |                |        |         |       | flows that are  |
   |     |                |        |         |       | selected during |
   |     |                |        |         |       | the Flow        |
   |     |                |        |         |       | Selection       |
   |     |                |        |         |       | Process since   |
   |     |                |        |         |       | the previous    |
   |     |                |        |         |       | report.         |
   +-----+----------------+--------+---------+-------+-----------------+
   | 3   | flowSelectedPa | unsign | Packets | Propo | This            |
   |     | cketDeltaCount | ed64   |         | sed   | Information     |
   |     |                |        |         |       | Element         |
   |     |                |        |         |       | specifies the   |
   |     |                |        |         |       | volume in       |
   |     |                |        |         |       | packets of all  |
   |     |                |        |         |       | flows that were |
   |     |                |        |         |       | selected during |
   |     |                |        |         |       | the Flow        |
   |     |                |        |         |       | Selection       |
   |     |                |        |         |       | Process since   |
   |     |                |        |         |       | the previous    |
   |     |                |        |         |       | report.         |
   +-----+----------------+--------+---------+-------+-----------------+




D'Antonio, et al.       Expires October 25, 2012               [Page 25]

Internet-Draft          Flow Selection Techniques             April 2012


   +-----+----------------+--------+---------+-------+-----------------+
   | 4   | flowSelectedFl | unsign | Flows   | Propo | This            |
   |     | owDeltaCount   | ed64   |         | sed   | Information     |
   |     |                |        |         |       | Element         |
   |     |                |        |         |       | specifies the   |
   |     |                |        |         |       | number of Flows |
   |     |                |        |         |       | that were       |
   |     |                |        |         |       | selected during |
   |     |                |        |         |       | the Flow        |
   |     |                |        |         |       | Selection       |
   |     |                |        |         |       | Process since   |
   |     |                |        |         |       | the last        |
   |     |                |        |         |       | report.         |
   +-----+----------------+--------+---------+-------+-----------------+
   | 5   | selectorIDTota | unsign | Flows   | Propo | This            |
   |     | lFlowsObserved | ed64   |         | sed   | Information     |
   |     |                |        |         |       | Element         |
   |     |                |        |         |       | specifies the   |
   |     |                |        |         |       | total number of |
   |     |                |        |         |       | flows observed  |
   |     |                |        |         |       | by a Selector,  |
   |     |                |        |         |       | for a specific  |
   |     |                |        |         |       | value of        |
   |     |                |        |         |       | SelectorId.     |
   |     |                |        |         |       | This            |
   |     |                |        |         |       | Information     |
   |     |                |        |         |       | Element should  |
   |     |                |        |         |       | be used in an   |
   |     |                |        |         |       | Options         |
   |     |                |        |         |       | Template scoped |
   |     |                |        |         |       | to the          |
   |     |                |        |         |       | observation to  |
   |     |                |        |         |       | which it        |
   |     |                |        |         |       | refers.  See    |
   |     |                |        |         |       | Section 3.4.2.1 |
   |     |                |        |         |       | of the IPFIX    |
   |     |                |        |         |       | protocol        |
   |     |                |        |         |       | document        |
   |     |                |        |         |       | [RFC5101]       |
   +-----+----------------+--------+---------+-------+-----------------+











D'Antonio, et al.       Expires October 25, 2012               [Page 26]

Internet-Draft          Flow Selection Techniques             April 2012


   +-----+----------------+--------+---------+-------+-----------------+
   | 6   | selectorIDTota | unsign | Flows   | Propo | This            |
   |     | lFlowsSelected | ed64   |         | sed   | Information     |
   |     |                |        |         |       | Element         |
   |     |                |        |         |       | specifies the   |
   |     |                |        |         |       | total number of |
   |     |                |        |         |       | flows selected  |
   |     |                |        |         |       | by a Selector,  |
   |     |                |        |         |       | for a specific  |
   |     |                |        |         |       | value of        |
   |     |                |        |         |       | SelectorId.     |
   |     |                |        |         |       | This            |
   |     |                |        |         |       | Information     |
   |     |                |        |         |       | Element should  |
   |     |                |        |         |       | be used in an   |
   |     |                |        |         |       | Options         |
   |     |                |        |         |       | Template scoped |
   |     |                |        |         |       | to the          |
   |     |                |        |         |       | observation to  |
   |     |                |        |         |       | which it        |
   |     |                |        |         |       | refers.  See    |
   |     |                |        |         |       | Section 3.4.2.1 |
   |     |                |        |         |       | of the IPFIX    |
   |     |                |        |         |       | protocol        |
   |     |                |        |         |       | document        |
   |     |                |        |         |       | [RFC5101].      |
   +-----+----------------+--------+---------+-------+-----------------+
























D'Antonio, et al.       Expires October 25, 2012               [Page 27]

Internet-Draft          Flow Selection Techniques             April 2012


   +-----+----------------+--------+---------+-------+-----------------+
   | 7   | samplingFlowIn | unsign | Flows   | Propo | This            |
   |     | terval         | ed64   |         | sed   | Information     |
   |     |                |        |         |       | Element         |
   |     |                |        |         |       | specifies the   |
   |     |                |        |         |       | number of flows |
   |     |                |        |         |       | that are        |
   |     |                |        |         |       | consecutively   |
   |     |                |        |         |       | sampled.  A     |
   |     |                |        |         |       | value of 100    |
   |     |                |        |         |       | means that 100  |
   |     |                |        |         |       | consecutive     |
   |     |                |        |         |       | flows are       |
   |     |                |        |         |       | sampled.  For   |
   |     |                |        |         |       | example, this   |
   |     |                |        |         |       | Information     |
   |     |                |        |         |       | Element may be  |
   |     |                |        |         |       | used to         |
   |     |                |        |         |       | describe the    |
   |     |                |        |         |       | configuration   |
   |     |                |        |         |       | of a systematic |
   |     |                |        |         |       | count-based     |
   |     |                |        |         |       | Sampling        |
   |     |                |        |         |       | Selector.       |
   +-----+----------------+--------+---------+-------+-----------------+


























D'Antonio, et al.       Expires October 25, 2012               [Page 28]

Internet-Draft          Flow Selection Techniques             April 2012


   +-----+----------------+--------+---------+-------+-----------------+
   | 8   | samplingFlowSp | unsign | Flows   | Propo | This            |
   |     | ace            | ed64   |         | sed   | Information     |
   |     |                |        |         |       | Element         |
   |     |                |        |         |       | specifies the   |
   |     |                |        |         |       | number of flows |
   |     |                |        |         |       | between two     |
   |     |                |        |         |       | "samplingFlowIn |
   |     |                |        |         |       | terval"s.  A    |
   |     |                |        |         |       |  value of 100   |
   |     |                |        |         |       |  means that the |
   |     |                |        |         |       |  next interval  |
   |     |                |        |         |       |  starts 100     |
   |     |                |        |         |       |  flows (which   |
   |     |                |        |         |       |  are not        |
   |     |                |        |         |       |  sampled) after |
   |     |                |        |         |       |  the current    |
   |     |                |        |         |       |  "samplingFlowI |
   |     |                |        |         |       | nterval" is ove |
   |     |                |        |         |       | r.For example,  |
   |     |                |        |         |       |   this          |
   |     |                |        |         |       |   Information   |
   |     |                |        |         |       |   Element may b |
   |     |                |        |         |       | e used to       |
   |     |                |        |         |       |   describe the  |
   |     |                |        |         |       |   configuration |
   |     |                |        |         |       |   of a systemat |
   |     |                |        |         |       | iccount-based   |
   |     |                |        |         |       |   Sampling      |
   |     |                |        |         |       |   Selector.     |
   +-----+----------------+--------+---------+-------+-----------------+




















D'Antonio, et al.       Expires October 25, 2012               [Page 29]

Internet-Draft          Flow Selection Techniques             April 2012


   +-----+----------------+--------+---------+-------+-----------------+
   | 9   | flowSamplingTi | unsign | microse | Propo | This            |
   |     | meInterval     | ed64   | conds   | sed   | Information     |
   |     |                |        |         |       | Element         |
   |     |                |        |         |       | specifies the   |
   |     |                |        |         |       | time interval   |
   |     |                |        |         |       | in microseconds |
   |     |                |        |         |       | during which    |
   |     |                |        |         |       | all arriving    |
   |     |                |        |         |       | flows are       |
   |     |                |        |         |       | sampled.  For   |
   |     |                |        |         |       | example, this   |
   |     |                |        |         |       | Information     |
   |     |                |        |         |       | Element may be  |
   |     |                |        |         |       | used to         |
   |     |                |        |         |       | describe the    |
   |     |                |        |         |       | configuration   |
   |     |                |        |         |       | of a systematic |
   |     |                |        |         |       | time-based      |
   |     |                |        |         |       | Sampling        |
   |     |                |        |         |       | Selector.       |
   +-----+----------------+--------+---------+-------+-----------------+





























D'Antonio, et al.       Expires October 25, 2012               [Page 30]

Internet-Draft          Flow Selection Techniques             April 2012


   +-----+----------------+--------+---------+-------+-----------------+
   | 10  | flowSamplingTi | unsign | microse | Propo | This            |
   |     | meSpace        | ed64   | conds   | sed   | Information     |
   |     |                |        |         |       | Element         |
   |     |                |        |         |       | specifies the   |
   |     |                |        |         |       | time interval   |
   |     |                |        |         |       | in microseconds |
   |     |                |        |         |       | between two     |
   |     |                |        |         |       | "flowSamplingTi |
   |     |                |        |         |       | meInterval"s.   |
   |     |                |        |         |       | Avalue of 100   |
   |     |                |        |         |       |  means that the |
   |     |                |        |         |       |  next interval  |
   |     |                |        |         |       |  starts 100     |
   |     |                |        |         |       |  microseconds   |
   |     |                |        |         |       |  (during which  |
   |     |                |        |         |       |  no flows are   |
   |     |                |        |         |       |  sampled) after |
   |     |                |        |         |       |  the current    |
   |     |                |        |         |       |  "flowsamplingT |
   |     |                |        |         |       | imeInterval" is |
   |     |                |        |         |       |   over.  For    |
   |     |                |        |         |       |   example, this |
   |     |                |        |         |       |   Information   |
   |     |                |        |         |       |   Element may   |
   |     |                |        |         |       |   used to       |
   |     |                |        |         |       |   describe the  |
   |     |                |        |         |       |   configuration |
   |     |                |        |         |       |   of a systemat |
   |     |                |        |         |       | ictime-based    |
   |     |                |        |         |       |   Sampling      |
   |     |                |        |         |       |   Selector.     |
   +-----+----------------+--------+---------+-------+-----------------+
   | 11  | hashFlowDomain | unsign | identif | Propo | This            |
   |     |                | ed16   | ier     | sed   | Information     |
   |     |                |        |         |       | Element         |
   |     |                |        |         |       | specifies the   |
   |     |                |        |         |       | Information     |
   |     |                |        |         |       | Elements that   |
   |     |                |        |         |       | are used by the |
   |     |                |        |         |       | Hash-based flow |
   |     |                |        |         |       | Selection       |
   |     |                |        |         |       | Selector as the |
   |     |                |        |         |       | Hash Domain.    |
   +-----+----------------+--------+---------+-------+-----------------+

                      Table 3: Flow Selection methods




D'Antonio, et al.       Expires October 25, 2012               [Page 31]

Internet-Draft          Flow Selection Techniques             April 2012


8.2.  Registration of Object Identifier

   IANA will register the following OID in the IPFIX-SELECTOR-MIB
   Functions sub-registry at http://www.iana.org/assignments/smi-numbers
   according to the procedures set forth in [I-D.dkcm-ipfix-rfc5815bis]
Should be http://tools.ietf.org/html/draft-ietf-ipfix-rfc5815bis-03
Btw, this draft is right now in AUTH48
See http://www.rfc-editor.org/authors/rfc6632.txt
http://www.rfc-editor.org/authors/rfc6615.txt

Btw,  you must  mention that you want a new entry under 
Sub-registry Name: IPFIX-SELECTOR-MIB Functions
See http://tools.ietf.org/html/draft-ietf-ipfix-psamp-mib-04#section-8 for an example



   +---------+-----------------------+---------------------+-----------+
   | Decimal | Name                  | Description         | Reference |
   +---------+-----------------------+---------------------+-----------+
   | 1       | flowSelectorAlgorithm | This Object         | [RFCyyyy] |
   |         |                       | Identifier          |           |
   |         |                       | identifies the flow |           |
   |         |                       | selection method    |           |
   |         |                       | (e.g., Filtering,   |           |
   |         |                       | Sampling) that is   |           |
   |         |                       | applied by the Flow |           |
   |         |                       | Selection Process   |           |
   +---------+-----------------------+---------------------+-----------+

              Table 4: Information Elements to be registered
You can't use the value 1. IANA will decide what is the next available value.

   Editor's Note (to be removed prior to publication): the RFC editor is
   asked to replace "yyyy" in this document by the number of the RFC
   when the assignment has been made.


9.  Security Considerations

   The described flow sampling techniques and the hash-based flow
   filtering technique aim at the selection of a representative subset
   of flows in order to make an accurate estimation of the population.
Not always, right?  For example, it's not the goal of "Property Match Filtering"
   An adversary may have incentives to influence the selection of flows,
   for example to circumvent accounting.

   Security considerations concerning the choice of a Hash Function for
   Hash-based Packet Selection have been discussed in Section 6.2.3 of
   [RFC5475] and are also appropriate for Hash-Based Flow Selection.
   This section discussed a number of potential attacks to craft Streams
I see Observed Packet Stream and Packet Stream in RFC5475,  I see Data Stream in RFC5470, but not Stream alone.
I guess you want to say Packet Stream
   that are disproportionately detected and/or discover the Hash
   Function parameters, the vulnerabilities of different Hash Functions
   to these attacks, and practices to minimize these vulnerabilities.

   For other sampling approaches a user can gain knowledge about the
   start and stop triggers in time-based systematic Sampling, e.g., by
   sending test packets.  This knowledge might allow users to modify
   their send schedule in a way that their packets are
   disproportionately selected or not selected.  For random Sampling, a
   cryptographically strong random number generator should be used in



D'Antonio, et al.       Expires October 25, 2012               [Page 32]

Internet-Draft          Flow Selection Techniques             April 2012


   order to prevent that an advisory can predict the selection decision
   [GoRe07].

   Further security threats can occur when Sampling parameters are
   configured or communicated to other entities.  The protocol(s) for
   the configuration and reporting of Sampling parameters are out of
   scope of this document.  Nevertheless, a set of initial requirements
   for future configuration and reporting protocols are stated below:

   1.  Protection against disclosure of configuration information: Flow
Flow -> Flow Sampling
       selection configuration information describes the selection
       process and its parameters.  This information can be useful to
       attackers.  Attackers may craft packets that never fit the
       selection criteria in order to prevent flows to be seen by the
       selection process.  They can also craft a lot of packets that fit
       the selection criteria and overload or bias subsequent processes.
       Therefore any transmission of configuration data (e.g., to
       configure a process or to report its actual status) should be
       protected by encryption.

   2.  Protection against modification of configuration information: If
       wrong configuration information is sent to the flow selection
       process, it can lead to a malfunction of the selection process.
       Also if wrong configuration information is reported from the
       selection process to other processes it can lead to wrong
       estimations at subsequent processes.  Therefore any protocol that
       transmits configuration information should prevent that an
       attacker can modify configuration information.  Data integrity
       can be achieved by authenticating the data.

   3.  Protection against malicious nodes sending configuration
       information: The remote configuration of flow selection methods
       should be protected against access by unauthorized nodes.  This
       can be achieved by access control lists at the selection devices
selection devices? Do you mean IPFIX Exporter?
       and source authentication.  The reporting of configuration data
       from a selection process has to be protected in the same way.
       That means that also protocols that report configuration data
       from the selection process to other processes need to protect
       against unauthorized nodes reporting configuration information.

   The security threats that originate from communicating configuration
   information to and from selection processes cannot be assessed solely
   with the information given in this document.  A further more detailed
   assessment of security threats is necessary when a specific protocol
   for the configuration or reporting configuration data is proposed.






D'Antonio, et al.       Expires October 25, 2012               [Page 33]

Internet-Draft          Flow Selection Techniques             April 2012


10.  Acknowledgments

   We would like to thank the IPFIX group, especially Brian Trammell,
   Paul Aitken and Benoit Claise for fruitful discussions and for
   proofreading the document.


11.  References

11.1.  Normative References

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119, March 1997.

   [RFC5101]  Claise, B., "Specification of the IP Flow Information
              Export (IPFIX) Protocol for the Exchange of IP Traffic
              Flow Information", RFC 5101, January 2008.

   [RFC5102]  Quittek, J., Bryant, S., Claise, B., Aitken, P., and J.
              Meyer, "Information Model for IP Flow Information Export",
              RFC 5102, January 2008.

   [RFC5475]  Zseby, T., Molina, M., Duffield, N., Niccolini, S., and F.
              Raspall, "Sampling and Filtering Techniques for IP Packet
              Selection", RFC 5475, March 2009.

   [RFC5476]  Claise, B., Johnson, A., and J. Quittek, "Packet Sampling
              (PSAMP) Protocol Specifications", RFC 5476, March 2009.

11.2.  Informative References

   [Bra75]    Brayer, K., "Evaluation of 32 Degree Polynomials in Error
              Detection on the SATIN IV Autovon Error Patterns",
              National Technical Information Service p.74, August 1975.

   [CoHa08]   Cormode, G. and M. Hadjieleftheriou, "Finding frequent
              items in data streams", Journal, Proceedings of the Very
              Large DataBase Endowment VLDB Endowment, Volume 1 Issue 2,
              August 2008, August 2008.

   [DuLT01]   Duffield, N., Lund, C., and M. Thorup, "Charging from
              Sampled Network Usage", ACM Internet Measurement Workshop
              IMW 2001, San Francisco, USA, November 2001.

   [EsVa01]   Estan, C. and G,. Varghese, "New Directions in Traffic
              Measurement and Accounting: Focusing on the Elephants,
              Ignoring the Mice", ACM SIGCOMM Internet Measurement
              Workshop 2001, San Francisco (CA), November 2001.



D'Antonio, et al.       Expires October 25, 2012               [Page 34]

Internet-Draft          Flow Selection Techniques             April 2012


   [KaPS03]   Karp, R., Papadimitriou, C., and S. S. Shenker, "A simple
              algorithm for finding frequent elements in sets and
              bags.", ACM Transactions on Database Systems, Volume 28,
              51-55, 2003, March 2003.

   [MSZC10]   Mai, J., Sridharan, A., Zang, H., and C. Chuah, "Fast
              Filtered Sampling", Computer Networks Volume 54, Issue 11,
              Pages 1885-1898, ISSN 1389-1286, January 2010.

   [MaMo02]   Manku, G. and R. Motwani, "Approximate Frequency Counts
              over Data Streams", Proceedings of the International
              Conference on Very large DataBases (VLDB) pages 346--357,
              2002, Hong Kong, China, 2002.

   [RFC3917]  Quittek, J., Zseby, T., Claise, B., and S. Zander,
              "Requirements for IP Flow Information Export (IPFIX)",
              RFC 3917, October 2004.

   [RFC5226]  Narten, T. and H. Alvestrand, "Guidelines for Writing an
              IANA Considerations Section in RFCs", BCP 26, RFC 5226,
              May 2008.

   [RFC5470]  Sadasivan, G., Brownlee, N., Claise, B., and J. Quittek,
              "Architecture for IP Flow Information Export", RFC 5470,
              March 2009.

   [RFC6183]  Kobayashi, A., Claise, B., Muenz, G., and K. Ishibashi,
              "IP Flow Information Export (IPFIX) Mediation: Framework",
              RFC 6183, April 2011.

   [iana-ipfix-assignments]
              "IP Flow Information Export Information Elements", 2007,
              <http://www.iana.org/assignments/ipfix/ipfix.xml>.


Authors' Addresses

   Salvatore D'Antonio
   University of Napoli "Parthenope"
   Centro Direzionale di Napoli Is. C4
   Naples  80143
   Italy

   Phone: +39 081 5476766
   Email: salvatore.dantonio@uniparthenope.it






D'Antonio, et al.       Expires October 25, 2012               [Page 35]

Internet-Draft          Flow Selection Techniques             April 2012


   Tanja Zseby
   CAIDA/FhG FOKUS
   San Diego Supercomputer Center (SDSC)
   University of California, San Diego (UCSD)
   9500 Gilman Drive
   La Jolla  CA 92093-0505
   USA

   Email: tanja@caida.org


   Christian Henke
   Tektronix Communication Berlin
   Wohlrabedamm 32
   Berlin  13629
   Germany

   Phone: +49 17 2323 8717
   Email: christian.henke@tektronix.com


   Lorenzo Peluso
   University of Napoli
   Via Claudio 21
   Napoli  80125
   Italy

   Phone: +39 081 7683821
   Email: lorenzo.peluso@unina.it






















D'Antonio, et al.       Expires October 25, 2012               [Page 36]



--------------060801090201060907020606-- From paitken@cisco.com Tue Jun 5 08:29:11 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C866521F8656 for ; Tue, 5 Jun 2012 08:29:11 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.599 X-Spam-Level: X-Spam-Status: No, score=-10.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id cLSyz+ntbYf0 for ; Tue, 5 Jun 2012 08:29:11 -0700 (PDT) Received: from ams-iport-3.cisco.com (ams-iport-3.cisco.com [144.254.224.146]) by ietfa.amsl.com (Postfix) with ESMTP id C37F621F8682 for ; Tue, 5 Jun 2012 08:29:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=paitken@cisco.com; l=877; q=dns/txt; s=iport; t=1338910150; x=1340119750; h=message-id:date:from:mime-version:to:subject:references: in-reply-to:content-transfer-encoding; bh=kTqv8UM/LmnePZNJhE3d3NmE8wYSSBWuCkrYTCG2kuA=; b=Cc55CQmwRJjSceEq0HGRolYnRs2EbTTZun5OTc+98C7hYg92QKa5FJlv pne+vtzuqRTFQn0N8zk/IFc8MIXLXww7MjQiDb1zxX4i8bMDYB5WShevQ uaeGWlzTBZPp+vtOXk67G1RKctcSbsyEhDPbdbZ72EPFSbn2m7iOTJvDI o=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Av4EAGYlzk+Q/khM/2dsb2JhbABFtDmBB4IZAQEEEgElMw0RLBYPCQMCAQIBRQYNCAEBHodpC5ctoAqRIwOVG4EPhEGIQIFmgmE X-IronPort-AV: E=Sophos;i="4.75,718,1330905600"; d="scan'208";a="5401987" Received: from ams-core-3.cisco.com ([144.254.72.76]) by ams-iport-3.cisco.com with ESMTP; 05 Jun 2012 15:29:09 +0000 Received: from [10.61.102.112] (dhcp-10-61-102-112.cisco.com [10.61.102.112]) by ams-core-3.cisco.com (8.14.5/8.14.5) with ESMTP id q55FT9j9010441 for ; Tue, 5 Jun 2012 15:29:09 GMT Message-ID: <4FCE25CB.5020508@cisco.com> Date: Tue, 05 Jun 2012 16:29:15 +0100 From: Paul Aitken User-Agent: Mozilla/5.0 (X11; Linux i686; rv:12.0) Gecko/20120430 Thunderbird/12.0.1 MIME-Version: 1.0 To: IETF IPFIX Working Group References: <4FCE149B.8010802@cisco.com> <4FCE168A.6020603@cisco.com> In-Reply-To: <4FCE168A.6020603@cisco.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: [IPFIX] exporting ranges in IPFIX X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Jun 2012 15:29:11 -0000 Dear IPFIX experts, As far as I know, IPFIX doesn't have a generic mechanism for reporting ranges. I'm looking for a way to report bulk port allocation per section 5 of http://tools.ietf.org/html/draft-ietf-behave-lsn-requirements-05. This method would be useful for reporting port ranges in draft-tsou-behave-natx4-log-reduction-02 and draft-bajko-pripaddrassign-04. So I propose to request three new IPFIX Information Elements: Port block start: 16 bits Port block step size 8 bits Number of ports in block 16 bits However, there could be better ways to export a "range" which don't require three new IEs each time. Do you forsee a need for a such a mechanism? Shall I proceed with my request to IANA? If so, should I write a short ID explaining how the three IEs should be used together? Thanks, P. From andrewf@plixer.com Tue Jun 5 08:35:28 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 35DAF21F86A1 for ; Tue, 5 Jun 2012 08:35:28 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KQzz4G5zlpe9 for ; Tue, 5 Jun 2012 08:35:27 -0700 (PDT) Received: from smtp.plixer.com (smtp.plixer.com [66.186.184.193]) by ietfa.amsl.com (Postfix) with ESMTP id 48C6921F86B0 for ; Tue, 5 Jun 2012 08:35:27 -0700 (PDT) Received: from [10.100.1.132] ([66.186.184.193]) by smtp.plixer.com over TLS secured channel with Microsoft SMTPSVC(6.0.3790.4675); Tue, 5 Jun 2012 11:35:26 -0400 Message-ID: <4FCE273E.4040500@plixer.com> Date: Tue, 05 Jun 2012 11:35:26 -0400 From: Andrew Feren User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:15.0) Gecko/15.0 Thunderbird/15.0a1 MIME-Version: 1.0 To: Paul Aitken References: <4FCE149B.8010802@cisco.com> <4FCE168A.6020603@cisco.com> <4FCE25CB.5020508@cisco.com> In-Reply-To: <4FCE25CB.5020508@cisco.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 05 Jun 2012 15:35:26.0419 (UTC) FILETIME=[D4606230:01CD4330] Cc: IETF IPFIX Working Group Subject: Re: [IPFIX] exporting ranges in IPFIX X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Jun 2012 15:35:28 -0000 Hi Paul, This seems useful, but out of curiousity, why "Number of ports in block" instead of "Port block end". -Andrew On 06/05/2012 11:29 AM, Paul Aitken wrote: > Dear IPFIX experts, > > As far as I know, IPFIX doesn't have a generic mechanism for reporting > ranges. > > I'm looking for a way to report bulk port allocation per section 5 of > http://tools.ietf.org/html/draft-ietf-behave-lsn-requirements-05. > > This method would be useful for reporting port ranges in > draft-tsou-behave-natx4-log-reduction-02 and > draft-bajko-pripaddrassign-04. > > So I propose to request three new IPFIX Information Elements: > > Port block start: 16 bits > Port block step size 8 bits > Number of ports in block 16 bits > > > However, there could be better ways to export a "range" which don't > require three new IEs each time. Do you forsee a need for a such a > mechanism? > > Shall I proceed with my request to IANA? If so, should I write a short > ID explaining how the three IEs should be used together? > > Thanks, > P. > > _______________________________________________ > IPFIX mailing list > IPFIX@ietf.org > https://www.ietf.org/mailman/listinfo/ipfix From paitken@cisco.com Tue Jun 5 08:50:29 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7989721F86FE for ; Tue, 5 Jun 2012 08:50:29 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.599 X-Spam-Level: X-Spam-Status: No, score=-10.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id MwcUI7Gmpgn6 for ; Tue, 5 Jun 2012 08:50:28 -0700 (PDT) Received: from ams-iport-4.cisco.com (ams-iport-4.cisco.com [144.254.224.147]) by ietfa.amsl.com (Postfix) with ESMTP id 6C5D521F86B0 for ; Tue, 5 Jun 2012 08:50:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=paitken@cisco.com; l=1624; q=dns/txt; s=iport; t=1338911428; x=1340121028; h=message-id:date:from:mime-version:to:cc:subject: references:in-reply-to:content-transfer-encoding; bh=uPkigGcVt7WEbN1li8ryLRQO67lVCVTmfJ3MdDkaGwA=; b=RvS516sOyrY8CxEE1AiET1ZU4GM6jTDbEfnkvKzNYue42+vO8lTi3fh4 su7qIYssdiJXNMB5FSQ/mjRpVB075+r9FZ/Fwg/cz5udKsoJtw2qC/4nU cwj9W9tKIpDetBBKIv+aPRk0oMp7q85MQLjAOt6jS139Qmih2OHGB2t+U Q=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Av8EAEMqzk+Q/khR/2dsb2JhbABFtDmBB4IYAQEBAwEBAQEPASUzAwoBBQsLGAkWDwkDAgECARUwBg0BBQIBAR6HZAULly6gCosThhADlRuBD4RBiECBZoJh X-IronPort-AV: E=Sophos;i="4.75,718,1330905600"; d="scan'208";a="5405875" Received: from ams-core-1.cisco.com ([144.254.72.81]) by ams-iport-4.cisco.com with ESMTP; 05 Jun 2012 15:50:27 +0000 Received: from [10.61.102.112] (dhcp-10-61-102-112.cisco.com [10.61.102.112]) by ams-core-1.cisco.com (8.14.5/8.14.5) with ESMTP id q55FoR6l009965; Tue, 5 Jun 2012 15:50:27 GMT Message-ID: <4FCE2AC8.2020504@cisco.com> Date: Tue, 05 Jun 2012 16:50:32 +0100 From: Paul Aitken User-Agent: Mozilla/5.0 (X11; Linux i686; rv:12.0) Gecko/20120430 Thunderbird/12.0.1 MIME-Version: 1.0 To: Andrew Feren References: <4FCE149B.8010802@cisco.com> <4FCE168A.6020603@cisco.com> <4FCE25CB.5020508@cisco.com> <4FCE273E.4040500@plixer.com> In-Reply-To: <4FCE273E.4040500@plixer.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: IETF IPFIX Working Group Subject: Re: [IPFIX] exporting ranges in IPFIX X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Jun 2012 15:50:29 -0000 Andrew, > This seems useful, but out of curiousity, why "Number of ports in > block" instead of "Port block end". Because (step size) x (number of steps) clearly identifies the end, whereas (end - start) might not divide evenly by (size). ie, starting at 1000, with a step size of 10, ending at 1195... the last value is ambiguous. Should we stop before or after 1195? Whereas starting at 1000, with 20 steps of 10, the last value is unambiguous. P. > On 06/05/2012 11:29 AM, Paul Aitken wrote: >> Dear IPFIX experts, >> >> As far as I know, IPFIX doesn't have a generic mechanism for >> reporting ranges. >> >> I'm looking for a way to report bulk port allocation per section 5 of >> http://tools.ietf.org/html/draft-ietf-behave-lsn-requirements-05. >> >> This method would be useful for reporting port ranges in >> draft-tsou-behave-natx4-log-reduction-02 and >> draft-bajko-pripaddrassign-04. >> >> So I propose to request three new IPFIX Information Elements: >> >> Port block start: 16 bits >> Port block step size 8 bits >> Number of ports in block 16 bits >> >> >> However, there could be better ways to export a "range" which don't >> require three new IEs each time. Do you forsee a need for a such a >> mechanism? >> >> Shall I proceed with my request to IANA? If so, should I write a >> short ID explaining how the three IEs should be used together? >> >> Thanks, >> P. >> >> _______________________________________________ >> IPFIX mailing list >> IPFIX@ietf.org >> https://www.ietf.org/mailman/listinfo/ipfix > From internet-drafts@ietf.org Tue Jun 5 13:05:56 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3AD7B21F8670; Tue, 5 Jun 2012 13:05:56 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.559 X-Spam-Level: X-Spam-Status: No, score=-102.559 tagged_above=-999 required=5 tests=[AWL=0.040, BAYES_00=-2.599, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ep4DkAMu6WDV; Tue, 5 Jun 2012 13:05:55 -0700 (PDT) Received: from ietfa.amsl.com (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9E1C321F863B; Tue, 5 Jun 2012 13:05:55 -0700 (PDT) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: internet-drafts@ietf.org To: i-d-announce@ietf.org X-Test-IDTracker: no X-IETF-IDTracker: 4.02 Message-ID: <20120605200555.8844.13603.idtracker@ietfa.amsl.com> Date: Tue, 05 Jun 2012 13:05:55 -0700 Cc: ipfix@ietf.org Subject: [IPFIX] I-D Action: draft-ietf-ipfix-mediation-protocol-01.txt X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Jun 2012 20:05:56 -0000 A New Internet-Draft is available from the on-line Internet-Drafts director= ies. This draft is a work item of the IP Flow Information Export Working Gr= oup of the IETF. Title : Operation of the IP Flow Information Export (IPFIX) Prot= ocol on IPFIX Mediators Author(s) : Benoit Claise Atsushi Kobayashi Brian Trammell Filename : draft-ietf-ipfix-mediation-protocol-01.txt Pages : 26 Date : 2012-06-05 This document specifies the the operation of the IP Flow Information Export (IPFIX) protocol specific to IPFIX Mediators, including Template and Observation Point management, timing considerations, and other Mediator-specific concerns. A URL for this Internet-Draft is: http://www.ietf.org/internet-drafts/draft-ietf-ipfix-mediation-protocol-01.= txt Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ This Internet-Draft can be retrieved at: ftp://ftp.ietf.org/internet-drafts/draft-ietf-ipfix-mediation-protocol-01.t= xt The IETF datatracker page for this Internet-Draft is: https://datatracker.ietf.org/doc/draft-ietf-ipfix-mediation-protocol/ From trammell@tik.ee.ethz.ch Tue Jun 5 13:08:05 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B1D5021F863C for ; Tue, 5 Jun 2012 13:08:05 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -5.999 X-Spam-Level: X-Spam-Status: No, score=-5.999 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, J_CHICKENPOX_41=0.6, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tLaROvdEOyn7 for ; Tue, 5 Jun 2012 13:08:05 -0700 (PDT) Received: from smtp.ee.ethz.ch (smtp.ee.ethz.ch [129.132.2.219]) by ietfa.amsl.com (Postfix) with ESMTP id E0C9721F8542 for ; Tue, 5 Jun 2012 13:08:04 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by smtp.ee.ethz.ch (Postfix) with ESMTP id 99925D9310 for ; Tue, 5 Jun 2012 22:08:00 +0200 (MEST) X-Virus-Scanned: by amavisd-new on smtp.ee.ethz.ch Received: from smtp.ee.ethz.ch ([127.0.0.1]) by localhost (.ee.ethz.ch [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 7dROZjmYDC8h for ; Tue, 5 Jun 2012 22:08:00 +0200 (MEST) Received: from [10.0.1.2] (cust-integra-121-161.antanet.ch [80.75.121.161]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: briant) by smtp.ee.ethz.ch (Postfix) with ESMTPSA id 4452CD9309 for ; Tue, 5 Jun 2012 22:08:00 +0200 (MEST) From: Brian Trammell Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Date: Tue, 5 Jun 2012 22:07:59 +0200 References: <20120605200555.8844.52260.idtracker@ietfa.amsl.com> To: "ipfix@ietf.org Working Group" Message-Id: Mime-Version: 1.0 (Apple Message framework v1278) X-Mailer: Apple Mail (2.1278) Subject: [IPFIX] Fwd: New Version Notification for draft-ietf-ipfix-mediation-protocol-01.txt X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Jun 2012 20:08:05 -0000 Greetings, all, This revision reorganizes the document to present the specifications a = bit better, and distributes the open issues through the document as = [EDITOR'S NOTE]s. The document still requires some serious work and some = editorial effort, but comments on the identified open issues are more = than welcome. Best regards, Brian Begin forwarded message: > From: internet-drafts@ietf.org > Subject: New Version Notification for = draft-ietf-ipfix-mediation-protocol-01.txt > Date: June 5, 2012 10:05:55 PM GMT+02:00 > To: trammell@tik.ee.ethz.ch > Cc: bclaise@cisco.com, akoba@nttv6.net >=20 > A new version of I-D, draft-ietf-ipfix-mediation-protocol-01.txt has = been successfully submitted by Brian Trammell and posted to the IETF = repository. >=20 > Filename: draft-ietf-ipfix-mediation-protocol > Revision: 01 > Title: Operation of the IP Flow Information Export = (IPFIX) Protocol on IPFIX Mediators > Creation date: 2012-06-05 > WG ID: ipfix > Number of pages: 26 >=20 > Abstract: > This document specifies the the operation of the IP Flow Information > Export (IPFIX) protocol specific to IPFIX Mediators, including > Template and Observation Point management, timing considerations, = and > other Mediator-specific concerns. >=20 >=20 >=20 >=20 > The IETF Secretariat From paitken@cisco.com Fri Jun 8 14:06:22 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5C11211E8113 for ; Fri, 8 Jun 2012 14:06:22 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.599 X-Spam-Level: X-Spam-Status: No, score=-10.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fFSk535ABAvF for ; Fri, 8 Jun 2012 14:06:21 -0700 (PDT) Received: from ams-iport-1.cisco.com (ams-iport-1.cisco.com [144.254.224.140]) by ietfa.amsl.com (Postfix) with ESMTP id DBB6021F8692 for ; Fri, 8 Jun 2012 14:06:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=paitken@cisco.com; l=1764; q=dns/txt; s=iport; t=1339189578; x=1340399178; h=message-id:date:from:mime-version:to:subject:references: in-reply-to:content-transfer-encoding; bh=+B60ZNopqgNx5m85IjncAKAl5zCYUr5uV6UAcrK/Ndw=; b=E/Oqej29qDhehQfYdKvpxICMlo1q2EKzIKiSNCmP2r94KZTLbwlfwvA+ dn1lTXJbB8WmhYk+un6Cdszx+0ERNbpw0Me8gAohhaGIAniewHo3EMK58 H8NCHIv5GtuNwWeZcueAjnOxrFJvwvDJLbumqaBIhqxj9FMbWAKMAvcHO Q=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Av8EAJlo0k+Q/khL/2dsb2JhbABFtFeBB4IYAQEBBAEBAQ8BJTMDChELGAkWDwkDAgECARUwBg0GAgEBHodpC5kWn1yRJgOVHoEShEGIQoFmgmE X-IronPort-AV: E=Sophos;i="4.75,738,1330905600"; d="scan'208";a="139385854" Received: from ams-core-2.cisco.com ([144.254.72.75]) by ams-iport-1.cisco.com with ESMTP; 08 Jun 2012 21:06:06 +0000 Received: from [10.61.91.13] (ams3-vpn-dhcp6926.cisco.com [10.61.91.13]) by ams-core-2.cisco.com (8.14.5/8.14.5) with ESMTP id q58L65GS007641 for ; Fri, 8 Jun 2012 21:06:06 GMT Message-ID: <4FD2693E.9090808@cisco.com> Date: Fri, 08 Jun 2012 22:06:06 +0100 From: Paul Aitken User-Agent: Mozilla/5.0 (X11; Linux i686; rv:12.0) Gecko/20120430 Thunderbird/12.0.1 MIME-Version: 1.0 To: IETF IPFIX Working Group References: <4FCE149B.8010802@cisco.com> <4FCE168A.6020603@cisco.com> <4FCE25CB.5020508@cisco.com> In-Reply-To: <4FCE25CB.5020508@cisco.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [IPFIX] exporting ranges in IPFIX X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Jun 2012 21:06:22 -0000 Dear all, I've had two feedbacks requesting a range start / range end mechanism. So I propose to request four fields: Port block start: 16 bits Port block end: 16 bits Port block step size 16 bits Number of ports in block 16 bits These can be reported in whatever way best matches the implementation. eg, { start, step, number }, { start, end, step }, { start, end, number }. The default step size will be 1, so { start, end } indicates a contiguous range. Finally, I've upped the step size from 8 bits to 16 bits to allow step sizes > 255. Any further feedback? Thanks, P. On 05/06/12 16:29, Paul Aitken wrote: > Dear IPFIX experts, > > As far as I know, IPFIX doesn't have a generic mechanism for reporting > ranges. > > I'm looking for a way to report bulk port allocation per section 5 of > http://tools.ietf.org/html/draft-ietf-behave-lsn-requirements-05. > > This method would be useful for reporting port ranges in > draft-tsou-behave-natx4-log-reduction-02 and > draft-bajko-pripaddrassign-04. > > So I propose to request three new IPFIX Information Elements: > > Port block start: 16 bits > Port block step size 8 bits > Number of ports in block 16 bits > > > However, there could be better ways to export a "range" which don't > require three new IEs each time. Do you forsee a need for a such a > mechanism? > > Shall I proceed with my request to IANA? If so, should I write a short > ID explaining how the three IEs should be used together? > > Thanks, > P. > > _______________________________________________ > IPFIX mailing list > IPFIX@ietf.org > https://www.ietf.org/mailman/listinfo/ipfix From trammell@tik.ee.ethz.ch Sun Jun 10 23:36:42 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5D71611E8072 for ; Sun, 10 Jun 2012 23:36:42 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.185 X-Spam-Level: X-Spam-Status: No, score=-4.185 tagged_above=-999 required=5 tests=[BAYES_40=-0.185, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id iLPhfhF83XDX for ; Sun, 10 Jun 2012 23:36:41 -0700 (PDT) Received: from smtp.ee.ethz.ch (smtp.ee.ethz.ch [129.132.2.219]) by ietfa.amsl.com (Postfix) with ESMTP id 9DBAA21F84AF for ; Sun, 10 Jun 2012 23:36:40 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by smtp.ee.ethz.ch (Postfix) with ESMTP id 873EAD9309; Mon, 11 Jun 2012 08:36:39 +0200 (MEST) X-Virus-Scanned: by amavisd-new on smtp.ee.ethz.ch Received: from smtp.ee.ethz.ch ([127.0.0.1]) by localhost (.ee.ethz.ch [127.0.0.1]) (amavisd-new, port 10024) with LMTP id ZRs8vEJMnBbJ; Mon, 11 Jun 2012 08:36:39 +0200 (MEST) Received: from pb-10243.ethz.ch (pb-10243.ethz.ch [82.130.102.152]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: briant) by smtp.ee.ethz.ch (Postfix) with ESMTPSA id 44D23D9305; Mon, 11 Jun 2012 08:36:39 +0200 (MEST) Mime-Version: 1.0 (Apple Message framework v1278) Content-Type: text/plain; charset=us-ascii From: Brian Trammell In-Reply-To: Date: Mon, 11 Jun 2012 08:36:42 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: To: "Romascanu, Dan (Dan)" X-Mailer: Apple Mail (2.1278) Cc: IETF IPFIX Working Group Subject: Re: [IPFIX] WGLC Comments on draft-ietf-ipfix-ie-doctors X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Jun 2012 06:36:42 -0000 Hi, Dan, Many thanks for your review; comments on comments inline. I'm applying = changes to a post-WGLC revision to be submitted shortly... On May 24, 2012, at 5:00 PM, Romascanu, Dan (Dan) wrote: > 2. Section 4.3 talks about the need to designate an expert to advice = to > IANA on Netflow v9 matters. This request is not however mentioned in = the > IANA Considerations section.=20 Fixed.=20 > 3. It is not clear to me what is the goal of the following piece of > process described in Section 4.9:=20 >=20 >> In order to support transition from experimental registration to IANA > registration, the IANA registry provides an optional "enterprise- > specific IE reference" column for each Information Element. In = cases > of promoted enterprise-specific Information Elements, this column in > the registry SHOULD contain the private enterprise and Information > Element numbers of the enterprise-specific version of the = Information > Element. >=20 > If I understand well, an IE shows up in the IANA registry only when a > standard IE is added to the registry. What 'transition' is being > supported here, and how is it supported by a back reference to a > proprietary enterprise registry?=20 The idea is that proprietary enterprise registries are not necessarily = unpublished. The transition envisioned is of experimental or research = applications to production, as in the SIPCLF work, which defined = information elements during the draft stage in PEN 35566 so that initial = implementation could proceed without adding IEs to IANA before the work = published as an RFC (which, indeed, it hasn't been, as it wasn't adopted = by the WG.) > 4. In Section 5.2, the following change in an IE is considered to be > interoperable:=20 >=20 >> it corrects an ambiguity in the Information Element's definition, > which itself leads to non-interoperability (e.g., a prior change > to ipv6ExtensionHeaders) >=20 > How can an IE with a corrected definition be interoperable with an IE > with a definition that is that broken that it leads to > non-interoperability? Brokenly-defined IEs cannot be fixed, they must = be > deprecated and new IEs defined with a different name to start with.=20 This class of change was primarily included as it reflected a revision = which has already been done. The reasoning here is that, as the IE was = originally defined in a broken way, there is no net total negative = interoperability impact from redefining it to correct the ambiguity. = However, this does make the implicit assumption that implementors are = paying attention to information element updates. > 5. Also in 5.2:=20 >=20 >> A non-interoperable Information Element change may also be made if it > can be reasonably assumed in the eyes of the appointed experts that > no unchanged implementation of the Information Element exists; this > can be held to happen if a non-interoperable change to an = Information > Element defined shortly before is proposed to the IPFIX mailing list > by the original proposer of the Information Element, and no = objection > is raised within a reasonable amount of time, to be defined by the > expert reviewers. >=20 > I strongly oppose this. We need to assume that the IANA registry is > universally visible and that people who never read the IPFIX mailing > list can write applications using IEs. Hence there is no 'reasonable' > assumption like the one described here. A non-interoperable change = must > be performed by deprecating the old IE and defining a new IE with a > different name.=20 Put that way, I would tend to agree; we can remove this. > 6. In 5.3:=20 >=20 >> Names > of obsolete Information Elements MAY be reused, but this is NOT > RECOMMENDED, as it may cause confusion among users. >=20 > No, please not! Do not reuse names of obsolete IEs, you can never know > what older applications are being deployed and run.=20 Point, as above; removed. > 7. You may consider to recommend in the Security Considerations = sections > that IETF documents that define new IEs list in their Security > Considerations sections the IEs that have specific security aspects > mentioned in descriptions, or their semantics or their usage can > generate or are prone to security vulnerabilities. =20 Good point; done. Best regards, Brian= From dromasca@avaya.com Mon Jun 11 03:55:08 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id F135821F847E for ; Mon, 11 Jun 2012 03:55:08 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.799 X-Spam-Level: X-Spam-Status: No, score=-102.799 tagged_above=-999 required=5 tests=[AWL=-0.200, BAYES_00=-2.599, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id IT0Ks7or6M14 for ; Mon, 11 Jun 2012 03:55:08 -0700 (PDT) Received: from p-us1-iereast-outbound.us1.avaya.com (p-us1-iereast-outbound.us1.avaya.com [135.11.29.13]) by ietfa.amsl.com (Postfix) with ESMTP id 25AC121F8470 for ; Mon, 11 Jun 2012 03:55:07 -0700 (PDT) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AvgNAJnN1U/GmAcF/2dsb2JhbABFrRGHToEHghgBAQEBAxIeCjEDBgUMBAIBCA0EBAEBAQoGDAsBBgFFCQgBAQQTCBqHaZsZnDOLJ4UJYAObE4oGgmI X-IronPort-AV: E=Sophos;i="4.77,388,1336363200"; d="scan'208";a="13183504" Received: from unknown (HELO co300216-co-erhwest.avaya.com) ([198.152.7.5]) by p-us1-iereast-outbound.us1.avaya.com with ESMTP; 11 Jun 2012 06:52:54 -0400 Received: from unknown (HELO 307622ANEX5.global.avaya.com) ([135.64.140.13]) by co300216-co-erhwest-out.avaya.com with ESMTP; 11 Jun 2012 06:53:32 -0400 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Date: Mon, 11 Jun 2012 12:55:03 +0200 Message-ID: In-Reply-To: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [IPFIX] WGLC Comments on draft-ietf-ipfix-ie-doctors Thread-Index: Ac1HnJLzedY5eux0QBKuW0XwswS2FwAI62LQ References: From: "Romascanu, Dan (Dan)" To: "Brian Trammell" Cc: IETF IPFIX Working Group Subject: Re: [IPFIX] WGLC Comments on draft-ietf-ipfix-ie-doctors X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Jun 2012 10:55:09 -0000 Hi Brian, Thank you for the response. All your responses and proposals seem reasonable to me. The only point I would question is #3 - while now the intent is clarified, wouldn't it be better to have a MAY rather than a SHOULD in '... this column in the registry MAY contain the private enterprise and Information Element numbers of the enterprise-specific version of the Information Element.' Regards, Dan > -----Original Message----- > From: Brian Trammell [mailto:trammell@tik.ee.ethz.ch] > Sent: Monday, June 11, 2012 9:37 AM > To: Romascanu, Dan (Dan) > Cc: IETF IPFIX Working Group > Subject: Re: [IPFIX] WGLC Comments on draft-ietf-ipfix-ie-doctors >=20 > Hi, Dan, >=20 > Many thanks for your review; comments on comments inline. I'm applying > changes to a post-WGLC revision to be submitted shortly... >=20 > On May 24, 2012, at 5:00 PM, Romascanu, Dan (Dan) wrote: >=20 > > 2. Section 4.3 talks about the need to designate an expert to advice > to > > IANA on Netflow v9 matters. This request is not however mentioned in > the > > IANA Considerations section. >=20 > Fixed. >=20 > > 3. It is not clear to me what is the goal of the following piece of > > process described in Section 4.9: > > > >> In order to support transition from experimental registration to > IANA > > registration, the IANA registry provides an optional "enterprise- > > specific IE reference" column for each Information Element. In > cases > > of promoted enterprise-specific Information Elements, this column > in > > the registry SHOULD contain the private enterprise and Information > > Element numbers of the enterprise-specific version of the > Information > > Element. > > > > If I understand well, an IE shows up in the IANA registry only when a > > standard IE is added to the registry. What 'transition' is being > > supported here, and how is it supported by a back reference to a > > proprietary enterprise registry? >=20 > The idea is that proprietary enterprise registries are not necessarily > unpublished. The transition envisioned is of experimental or research > applications to production, as in the SIPCLF work, which defined > information elements during the draft stage in PEN 35566 so that > initial implementation could proceed without adding IEs to IANA before > the work published as an RFC (which, indeed, it hasn't been, as it > wasn't adopted by the WG.) >=20 > > 4. In Section 5.2, the following change in an IE is considered to be > > interoperable: > > > >> it corrects an ambiguity in the Information Element's definition, > > which itself leads to non-interoperability (e.g., a prior change > > to ipv6ExtensionHeaders) > > > > How can an IE with a corrected definition be interoperable with an IE > > with a definition that is that broken that it leads to > > non-interoperability? Brokenly-defined IEs cannot be fixed, they must > be > > deprecated and new IEs defined with a different name to start with. >=20 > This class of change was primarily included as it reflected a revision > which has already been done. The reasoning here is that, as the IE was > originally defined in a broken way, there is no net total negative > interoperability impact from redefining it to correct the ambiguity. > However, this does make the implicit assumption that implementors are > paying attention to information element updates. >=20 > > 5. Also in 5.2: > > > >> A non-interoperable Information Element change may also be made if > it > > can be reasonably assumed in the eyes of the appointed experts that > > no unchanged implementation of the Information Element exists; this > > can be held to happen if a non-interoperable change to an > Information > > Element defined shortly before is proposed to the IPFIX mailing > list > > by the original proposer of the Information Element, and no > objection > > is raised within a reasonable amount of time, to be defined by the > > expert reviewers. > > > > I strongly oppose this. We need to assume that the IANA registry is > > universally visible and that people who never read the IPFIX mailing > > list can write applications using IEs. Hence there is no > 'reasonable' > > assumption like the one described here. A non-interoperable change > must > > be performed by deprecating the old IE and defining a new IE with a > > different name. >=20 > Put that way, I would tend to agree; we can remove this. >=20 > > 6. In 5.3: > > > >> Names > > of obsolete Information Elements MAY be reused, but this is NOT > > RECOMMENDED, as it may cause confusion among users. > > > > No, please not! Do not reuse names of obsolete IEs, you can never > know > > what older applications are being deployed and run. >=20 > Point, as above; removed. >=20 > > 7. You may consider to recommend in the Security Considerations > sections > > that IETF documents that define new IEs list in their Security > > Considerations sections the IEs that have specific security aspects > > mentioned in descriptions, or their semantics or their usage can > > generate or are prone to security vulnerabilities. >=20 > Good point; done. >=20 > Best regards, >=20 > Brian From trammell@tik.ee.ethz.ch Mon Jun 11 04:21:25 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3EB6521F857F for ; Mon, 11 Jun 2012 04:21:25 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -5.392 X-Spam-Level: X-Spam-Status: No, score=-5.392 tagged_above=-999 required=5 tests=[AWL=1.207, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nawuinHWdU28 for ; Mon, 11 Jun 2012 04:21:24 -0700 (PDT) Received: from smtp.ee.ethz.ch (smtp.ee.ethz.ch [129.132.2.219]) by ietfa.amsl.com (Postfix) with ESMTP id 350DF21F850B for ; Mon, 11 Jun 2012 04:21:24 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by smtp.ee.ethz.ch (Postfix) with ESMTP id 6A003D930B; Mon, 11 Jun 2012 13:21:23 +0200 (MEST) X-Virus-Scanned: by amavisd-new on smtp.ee.ethz.ch Received: from smtp.ee.ethz.ch ([127.0.0.1]) by localhost (.ee.ethz.ch [127.0.0.1]) (amavisd-new, port 10024) with LMTP id kmF8Cg+cFp1e; Mon, 11 Jun 2012 13:21:23 +0200 (MEST) Received: from pb-10243.ethz.ch (pb-10243.ethz.ch [82.130.102.152]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: briant) by smtp.ee.ethz.ch (Postfix) with ESMTPSA id 3C27BD9305; Mon, 11 Jun 2012 13:21:23 +0200 (MEST) Mime-Version: 1.0 (Apple Message framework v1278) Content-Type: text/plain; charset=us-ascii From: Brian Trammell In-Reply-To: Date: Mon, 11 Jun 2012 13:21:22 +0200 Content-Transfer-Encoding: 7bit Message-Id: <32047471-4C72-40F3-B9CD-84222DCD512A@tik.ee.ethz.ch> References: To: "Romascanu, Dan (Dan)" X-Mailer: Apple Mail (2.1278) Cc: IETF IPFIX Working Group Subject: Re: [IPFIX] WGLC Comments on draft-ietf-ipfix-ie-doctors X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Jun 2012 11:21:25 -0000 Hi, Dan, A MAY here seems reasonable given the intent; will make the change. Thanks, best regards, Brian On Jun 11, 2012, at 12:55 PM, Romascanu, Dan (Dan) wrote: > Hi Brian, > > Thank you for the response. All your responses and proposals seem > reasonable to me. The only point I would question is #3 - while now the > intent is clarified, wouldn't it be better to have a MAY rather than a > SHOULD in '... this column in the registry MAY contain the private > enterprise and Information Element numbers of the enterprise-specific > version of the Information Element.' > > Regards, > > Dan > > >> -----Original Message----- >> From: Brian Trammell [mailto:trammell@tik.ee.ethz.ch] >> Sent: Monday, June 11, 2012 9:37 AM >> To: Romascanu, Dan (Dan) >> Cc: IETF IPFIX Working Group >> Subject: Re: [IPFIX] WGLC Comments on draft-ietf-ipfix-ie-doctors >> >> Hi, Dan, >> >> Many thanks for your review; comments on comments inline. I'm applying >> changes to a post-WGLC revision to be submitted shortly... >> >> On May 24, 2012, at 5:00 PM, Romascanu, Dan (Dan) wrote: >> >>> 2. Section 4.3 talks about the need to designate an expert to advice >> to >>> IANA on Netflow v9 matters. This request is not however mentioned in >> the >>> IANA Considerations section. >> >> Fixed. >> >>> 3. It is not clear to me what is the goal of the following piece of >>> process described in Section 4.9: >>> >>>> In order to support transition from experimental registration to >> IANA >>> registration, the IANA registry provides an optional "enterprise- >>> specific IE reference" column for each Information Element. In >> cases >>> of promoted enterprise-specific Information Elements, this column >> in >>> the registry SHOULD contain the private enterprise and Information >>> Element numbers of the enterprise-specific version of the >> Information >>> Element. >>> >>> If I understand well, an IE shows up in the IANA registry only when > a >>> standard IE is added to the registry. What 'transition' is being >>> supported here, and how is it supported by a back reference to a >>> proprietary enterprise registry? >> >> The idea is that proprietary enterprise registries are not necessarily >> unpublished. The transition envisioned is of experimental or research >> applications to production, as in the SIPCLF work, which defined >> information elements during the draft stage in PEN 35566 so that >> initial implementation could proceed without adding IEs to IANA before >> the work published as an RFC (which, indeed, it hasn't been, as it >> wasn't adopted by the WG.) >> >>> 4. In Section 5.2, the following change in an IE is considered to be >>> interoperable: >>> >>>> it corrects an ambiguity in the Information Element's definition, >>> which itself leads to non-interoperability (e.g., a prior > change >>> to ipv6ExtensionHeaders) >>> >>> How can an IE with a corrected definition be interoperable with an > IE >>> with a definition that is that broken that it leads to >>> non-interoperability? Brokenly-defined IEs cannot be fixed, they > must >> be >>> deprecated and new IEs defined with a different name to start with. >> >> This class of change was primarily included as it reflected a revision >> which has already been done. The reasoning here is that, as the IE was >> originally defined in a broken way, there is no net total negative >> interoperability impact from redefining it to correct the ambiguity. >> However, this does make the implicit assumption that implementors are >> paying attention to information element updates. >> >>> 5. Also in 5.2: >>> >>>> A non-interoperable Information Element change may also be made if >> it >>> can be reasonably assumed in the eyes of the appointed experts > that >>> no unchanged implementation of the Information Element exists; > this >>> can be held to happen if a non-interoperable change to an >> Information >>> Element defined shortly before is proposed to the IPFIX mailing >> list >>> by the original proposer of the Information Element, and no >> objection >>> is raised within a reasonable amount of time, to be defined by the >>> expert reviewers. >>> >>> I strongly oppose this. We need to assume that the IANA registry is >>> universally visible and that people who never read the IPFIX mailing >>> list can write applications using IEs. Hence there is no >> 'reasonable' >>> assumption like the one described here. A non-interoperable change >> must >>> be performed by deprecating the old IE and defining a new IE with a >>> different name. >> >> Put that way, I would tend to agree; we can remove this. >> >>> 6. In 5.3: >>> >>>> Names >>> of obsolete Information Elements MAY be reused, but this is NOT >>> RECOMMENDED, as it may cause confusion among users. >>> >>> No, please not! Do not reuse names of obsolete IEs, you can never >> know >>> what older applications are being deployed and run. >> >> Point, as above; removed. >> >>> 7. You may consider to recommend in the Security Considerations >> sections >>> that IETF documents that define new IEs list in their Security >>> Considerations sections the IEs that have specific security aspects >>> mentioned in descriptions, or their semantics or their usage can >>> generate or are prone to security vulnerabilities. >> >> Good point; done. >> >> Best regards, >> >> Brian From internet-drafts@ietf.org Mon Jun 11 05:17:49 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0623821F84E1; Mon, 11 Jun 2012 05:17:49 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.599 X-Spam-Level: X-Spam-Status: No, score=-102.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hdrx8R76OAly; Mon, 11 Jun 2012 05:17:48 -0700 (PDT) Received: from ietfa.amsl.com (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8301421F845D; Mon, 11 Jun 2012 05:17:48 -0700 (PDT) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: internet-drafts@ietf.org To: i-d-announce@ietf.org X-Test-IDTracker: no X-IETF-IDTracker: 4.02 Message-ID: <20120611121748.26849.58863.idtracker@ietfa.amsl.com> Date: Mon, 11 Jun 2012 05:17:48 -0700 Cc: ipfix@ietf.org Subject: [IPFIX] I-D Action: draft-ietf-ipfix-ie-doctors-03.txt X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Jun 2012 12:17:49 -0000 A New Internet-Draft is available from the on-line Internet-Drafts director= ies. This draft is a work item of the IP Flow Information Export Working Gr= oup of the IETF. Title : Guidelines for Authors and Reviewers of IPFIX Informatio= n Elements Author(s) : Brian Trammell Benoit Claise Filename : draft-ietf-ipfix-ie-doctors-03.txt Pages : 33 Date : 2012-06-11 This document provides guidelines for the definition of IPFIX Information Elements for addition to the IANA IPFIX Information Element registry, in order to extend the applicability of the IPFIX protocol to new operations and management areas. A URL for this Internet-Draft is: http://www.ietf.org/internet-drafts/draft-ietf-ipfix-ie-doctors-03.txt Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ This Internet-Draft can be retrieved at: ftp://ftp.ietf.org/internet-drafts/draft-ietf-ipfix-ie-doctors-03.txt The IETF datatracker page for this Internet-Draft is: https://datatracker.ietf.org/doc/draft-ietf-ipfix-ie-doctors/ From trammell@tik.ee.ethz.ch Mon Jun 11 05:30:50 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 038C121F858A for ; Mon, 11 Jun 2012 05:30:50 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -5.995 X-Spam-Level: X-Spam-Status: No, score=-5.995 tagged_above=-999 required=5 tests=[AWL=0.603, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bgNKjm49jfQN for ; Mon, 11 Jun 2012 05:30:49 -0700 (PDT) Received: from smtp.ee.ethz.ch (smtp.ee.ethz.ch [129.132.2.219]) by ietfa.amsl.com (Postfix) with ESMTP id 675FE21F855D for ; Mon, 11 Jun 2012 05:30:49 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by smtp.ee.ethz.ch (Postfix) with ESMTP id 6EE6AD930A for ; Mon, 11 Jun 2012 14:30:48 +0200 (MEST) X-Virus-Scanned: by amavisd-new on smtp.ee.ethz.ch Received: from smtp.ee.ethz.ch ([127.0.0.1]) by localhost (.ee.ethz.ch [127.0.0.1]) (amavisd-new, port 10024) with LMTP id WJVDGl+TRj2z for ; Mon, 11 Jun 2012 14:30:48 +0200 (MEST) Received: from pb-10243.ethz.ch (pb-10243.ethz.ch [82.130.102.152]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: briant) by smtp.ee.ethz.ch (Postfix) with ESMTPSA id 40E58D9305 for ; Mon, 11 Jun 2012 14:30:48 +0200 (MEST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Apple Message framework v1278) From: Brian Trammell In-Reply-To: <20120611121748.26849.58863.idtracker@ietfa.amsl.com> Date: Mon, 11 Jun 2012 14:30:47 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20120611121748.26849.58863.idtracker@ietfa.amsl.com> To: IETF IPFIX Working Group X-Mailer: Apple Mail (2.1278) Subject: Re: [IPFIX] I-D Action: draft-ietf-ipfix-ie-doctors-03.txt X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Jun 2012 12:30:50 -0000 Greetings, all, This revision addresses commentary received during WGLC; I believe it = handles all open issues with the document. Best regards, Brian On Jun 11, 2012, at 2:17 PM, internet-drafts@ietf.org wrote: >=20 > A New Internet-Draft is available from the on-line Internet-Drafts = directories. This draft is a work item of the IP Flow Information Export = Working Group of the IETF. >=20 > Title : Guidelines for Authors and Reviewers of IPFIX = Information Elements > Author(s) : Brian Trammell > Benoit Claise > Filename : draft-ietf-ipfix-ie-doctors-03.txt > Pages : 33 > Date : 2012-06-11 >=20 > This document provides guidelines for the definition of IPFIX > Information Elements for addition to the IANA IPFIX Information > Element registry, in order to extend the applicability of the IPFIX > protocol to new operations and management areas. >=20 >=20 > A URL for this Internet-Draft is: > http://www.ietf.org/internet-drafts/draft-ietf-ipfix-ie-doctors-03.txt >=20 > Internet-Drafts are also available by anonymous FTP at: > ftp://ftp.ietf.org/internet-drafts/ >=20 > This Internet-Draft can be retrieved at: > ftp://ftp.ietf.org/internet-drafts/draft-ietf-ipfix-ie-doctors-03.txt >=20 > The IETF datatracker page for this Internet-Draft is: > https://datatracker.ietf.org/doc/draft-ietf-ipfix-ie-doctors/ >=20 > _______________________________________________ > IPFIX mailing list > IPFIX@ietf.org > https://www.ietf.org/mailman/listinfo/ipfix From trammell@tik.ee.ethz.ch Mon Jun 11 07:48:55 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5AC8721F8611 for ; Mon, 11 Jun 2012 07:48:55 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.197 X-Spam-Level: X-Spam-Status: No, score=-6.197 tagged_above=-999 required=5 tests=[AWL=0.402, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8f4GpMsElZwn for ; Mon, 11 Jun 2012 07:48:54 -0700 (PDT) Received: from smtp.ee.ethz.ch (smtp.ee.ethz.ch [129.132.2.219]) by ietfa.amsl.com (Postfix) with ESMTP id 212E121F855D for ; Mon, 11 Jun 2012 07:48:47 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by smtp.ee.ethz.ch (Postfix) with ESMTP id 8DAA5D930A; Mon, 11 Jun 2012 16:48:46 +0200 (MEST) X-Virus-Scanned: by amavisd-new on smtp.ee.ethz.ch Received: from smtp.ee.ethz.ch ([127.0.0.1]) by localhost (.ee.ethz.ch [127.0.0.1]) (amavisd-new, port 10024) with LMTP id EXtgTemt0EWn; Mon, 11 Jun 2012 16:48:46 +0200 (MEST) Received: from pb-10243.ethz.ch (pb-10243.ethz.ch [82.130.102.152]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: briant) by smtp.ee.ethz.ch (Postfix) with ESMTPSA id 47722D9305; Mon, 11 Jun 2012 16:48:46 +0200 (MEST) Mime-Version: 1.0 (Apple Message framework v1278) Content-Type: text/plain; charset=us-ascii From: Brian Trammell In-Reply-To: <4FD2693E.9090808@cisco.com> Date: Mon, 11 Jun 2012 16:48:45 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <496491AA-BE09-4F4E-B45D-F6C3AD8B16C1@tik.ee.ethz.ch> References: <4FCE149B.8010802@cisco.com> <4FCE168A.6020603@cisco.com> <4FCE25CB.5020508@cisco.com> <4FD2693E.9090808@cisco.com> To: Paul Aitken X-Mailer: Apple Mail (2.1278) Cc: IETF IPFIX Working Group Subject: Re: [IPFIX] exporting ranges in IPFIX X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Jun 2012 14:48:55 -0000 Hi, Paul, all, As for adding the port block range IEs, this makes sense as there is (1) = no generic range mechanism for IPFIX and (2) no way to report blocks of = ports semantically consistent with that required by = draft-ietf-behave-lsn-requirements. Two comments, neither particularly IPFIX-specific: First, there is a danger here, of course, that self-inconsistent = information can be reported using either the step size or number of = ports in block for stepped ranges: simply report a step or count greater = than the range. This should of course be treated in the descriptions of = the IEs... Second, it's not clear that the proposed solution covers the = applications envisioned by the draft in question; from section 5:=20 Note that this list is not exhaustive. There is a continuum of behavior that a CGN may choose to implement. For example, a CGN could use scattered port sets of consecutive port sets. The stepped-range thing seems a little hackish to me, but I'm not at all = familiar with how things are done on CGNs. What about disjoint stepped = ranges? What about uneven steps? Hacks are great, but they should at = least address the whole problem, I think. However, I suppose these IEs = plus structured data would handle every reasonable case... if that's the = approach, that should also be mentioned. Cheers, Brian On Jun 8, 2012, at 11:06 PM, Paul Aitken wrote: > Dear all, >=20 > I've had two feedbacks requesting a range start / range end mechanism. = So I propose to request four fields: >=20 > Port block start: 16 bits > Port block end: 16 bits > Port block step size 16 bits > Number of ports in block 16 bits >=20 > These can be reported in whatever way best matches the implementation. >=20 > eg, { start, step, number }, { start, end, step }, { start, end, = number }. >=20 > The default step size will be 1, so { start, end } indicates a = contiguous range. >=20 > Finally, I've upped the step size from 8 bits to 16 bits to allow step = sizes > 255. >=20 > Any further feedback? >=20 > Thanks, > P. >=20 >=20 > On 05/06/12 16:29, Paul Aitken wrote: >> Dear IPFIX experts, >>=20 >> As far as I know, IPFIX doesn't have a generic mechanism for = reporting ranges. >>=20 >> I'm looking for a way to report bulk port allocation per section 5 of = http://tools.ietf.org/html/draft-ietf-behave-lsn-requirements-05. >>=20 >> This method would be useful for reporting port ranges in = draft-tsou-behave-natx4-log-reduction-02 and = draft-bajko-pripaddrassign-04. >>=20 >> So I propose to request three new IPFIX Information Elements: >>=20 >> Port block start: 16 bits >> Port block step size 8 bits >> Number of ports in block 16 bits >>=20 >>=20 >> However, there could be better ways to export a "range" which don't = require three new IEs each time. Do you forsee a need for a such a = mechanism? >>=20 >> Shall I proceed with my request to IANA? If so, should I write a = short ID explaining how the three IEs should be used together? >>=20 >> Thanks, >> P. >>=20 >> _______________________________________________ >> IPFIX mailing list >> IPFIX@ietf.org >> https://www.ietf.org/mailman/listinfo/ipfix >=20 > _______________________________________________ > IPFIX mailing list > IPFIX@ietf.org > https://www.ietf.org/mailman/listinfo/ipfix From ssenthil@cisco.com Mon Jun 11 09:33:07 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A5C7221F85CD for ; Mon, 11 Jun 2012 09:33:07 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.599 X-Spam-Level: X-Spam-Status: No, score=-10.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id U4p6nhGDbJKN for ; Mon, 11 Jun 2012 09:33:06 -0700 (PDT) Received: from rcdn-iport-7.cisco.com (rcdn-iport-7.cisco.com [173.37.86.78]) by ietfa.amsl.com (Postfix) with ESMTP id B9E1021F85C4 for ; Mon, 11 Jun 2012 09:33:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=ssenthil@cisco.com; l=4780; q=dns/txt; s=iport; t=1339432386; x=1340641986; h=date:subject:from:to:cc:message-id:in-reply-to: mime-version:content-transfer-encoding; bh=tVf3m68JsTbVItCdD4XRiL3IeMEzM1Z2X9VcsCXachw=; b=HsTL1+0cCIjr13XhAhnO4wFtfzRjE4gBaw2V//VIqcrqSdEx8d5y6Wlx wAANcQbWowD+4TjnTIJ6saRi4akKJmkENKMc4nnJUZRwssK+D2u3ohVki AZhqrCTdFaB2uE8MAj37O+6n3Ya3zj8tOmcOpx5WO13k+l5wqqeJfhoiW Y=; X-IronPort-AV: E=Sophos;i="4.75,750,1330905600"; d="scan'208";a="91403221" Received: from rcdn-core-1.cisco.com ([173.37.93.152]) by rcdn-iport-7.cisco.com with ESMTP; 11 Jun 2012 16:33:06 +0000 Received: from [10.150.26.7] ([10.150.26.7]) by rcdn-core-1.cisco.com (8.14.5/8.14.5) with ESMTP id q5BGX3nu003169; Mon, 11 Jun 2012 16:33:04 GMT User-Agent: Microsoft-MacOutlook/14.2.2.120421 Date: Mon, 11 Jun 2012 12:33:01 -0400 From: Senthil Sivakumar To: Brian Trammell , Paul Aitken Message-ID: Thread-Topic: [IPFIX] exporting ranges in IPFIX In-Reply-To: <496491AA-BE09-4F4E-B45D-F6C3AD8B16C1@tik.ee.ethz.ch> Mime-version: 1.0 Content-type: text/plain; charset="ISO-8859-1" Content-transfer-encoding: quoted-printable Cc: IETF IPFIX Working Group Subject: Re: [IPFIX] exporting ranges in IPFIX X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Jun 2012 16:33:07 -0000 On 6/11/12 10:48 AM, "Brian Trammell" wrote: >Hi, Paul, all, > >As for adding the port block range IEs, this makes sense as there is (1) >no generic range mechanism for IPFIX and (2) no way to report blocks of >ports semantically consistent with that required by >draft-ietf-behave-lsn-requirements. > >Two comments, neither particularly IPFIX-specific: > >First, there is a danger here, of course, that self-inconsistent >information can be reported using either the step size or number of ports >in block for stepped ranges: simply report a step or count greater than >the range. This should of course be treated in the descriptions of the >IEs... True. There is an effort underway in Behave that is capturing the IE's for NAT logging. And we should have all the constraints and restrictions on what should be reported/collected. > >Second, it's not clear that the proposed solution covers the applications >envisioned by the draft in question; from section 5: > > Note that this list is not exhaustive. There is a continuum of > behavior that a CGN may choose to implement. For example, a CGN > could use scattered port sets of consecutive port sets. It is very difficult to have a generic solution for all possible port allocation mechanisms. There are two goals : one is to reduce the amount of logs sent/received (to avoid per-connection logging) and not to be too predictable that would open one to attacks. That is where the bulk allocation with some arbitrary step size. Of course, it is not fool proof in all respects. > >The stepped-range thing seems a little hackish to me, but I'm not at all >familiar with how things are done on CGNs. What about disjoint stepped >ranges? What about uneven steps? Hacks are great, but they should at >least address the whole problem, I think. However, I suppose these IEs >plus structured data would handle every reasonable case... if that's the >approach, that should also be mentioned. If one wants to have uneven steps or some complex port scattered allocation mechanism then the implementations would have to figure out the necessary information that would required to be synced so that the exporter and collector can work in tandem knowing what port is allocated to which user. We don=B9t have such a mechanism today and when we have one, we might have to come back and ask for additional fields if required. Thanks Senthil > >Cheers, > >Brian > > >On Jun 8, 2012, at 11:06 PM, Paul Aitken wrote: > >> Dear all, >>=20 >> I've had two feedbacks requesting a range start / range end mechanism. >>So I propose to request four fields: >>=20 >> Port block start: 16 bits >> Port block end: 16 bits >> Port block step size 16 bits >> Number of ports in block 16 bits >>=20 >> These can be reported in whatever way best matches the implementation. >>=20 >> eg, { start, step, number }, { start, end, step }, { start, end, number >>}. >>=20 >> The default step size will be 1, so { start, end } indicates a >>contiguous range. >>=20 >> Finally, I've upped the step size from 8 bits to 16 bits to allow step >>sizes > 255. >>=20 >> Any further feedback? >>=20 >> Thanks, >> P. >>=20 >>=20 >> On 05/06/12 16:29, Paul Aitken wrote: >>> Dear IPFIX experts, >>>=20 >>> As far as I know, IPFIX doesn't have a generic mechanism for reporting >>>ranges. >>>=20 >>> I'm looking for a way to report bulk port allocation per section 5 of >>>http://tools.ietf.org/html/draft-ietf-behave-lsn-requirements-05. >>>=20 >>> This method would be useful for reporting port ranges in >>>draft-tsou-behave-natx4-log-reduction-02 and >>>draft-bajko-pripaddrassign-04. >>>=20 >>> So I propose to request three new IPFIX Information Elements: >>>=20 >>> Port block start: 16 bits >>> Port block step size 8 bits >>> Number of ports in block 16 bits >>>=20 >>>=20 >>> However, there could be better ways to export a "range" which don't >>>require three new IEs each time. Do you forsee a need for a such a >>>mechanism? >>>=20 >>> Shall I proceed with my request to IANA? If so, should I write a short >>>ID explaining how the three IEs should be used together? >>>=20 >>> Thanks, >>> P. >>>=20 >>> _______________________________________________ >>> IPFIX mailing list >>> IPFIX@ietf.org >>> https://www.ietf.org/mailman/listinfo/ipfix >>=20 >> _______________________________________________ >> IPFIX mailing list >> IPFIX@ietf.org >> https://www.ietf.org/mailman/listinfo/ipfix > >_______________________________________________ >IPFIX mailing list >IPFIX@ietf.org >https://www.ietf.org/mailman/listinfo/ipfix From wwwrun@rfc-editor.org Mon Jun 11 15:48:47 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9CC9A11E80BB; Mon, 11 Jun 2012 15:48:47 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102 X-Spam-Level: X-Spam-Status: No, score=-102 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, J_CHICKENPOX_93=0.6, NO_RELAYS=-0.001, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nGip72i-e8bE; Mon, 11 Jun 2012 15:48:47 -0700 (PDT) Received: from rfc-editor.org (rfc-editor.org [IPv6:2001:1890:123a::1:2f]) by ietfa.amsl.com (Postfix) with ESMTP id 8AF4611E80BA; Mon, 11 Jun 2012 15:48:45 -0700 (PDT) Received: by rfc-editor.org (Postfix, from userid 30) id 31E8CB1E007; Mon, 11 Jun 2012 15:48:40 -0700 (PDT) To: ietf-announce@ietf.org, rfc-dist@rfc-editor.org From: rfc-editor@rfc-editor.org Message-Id: <20120611224840.31E8CB1E007@rfc-editor.org> Date: Mon, 11 Jun 2012 15:48:40 -0700 (PDT) Cc: ipfix@ietf.org, rfc-editor@rfc-editor.org Subject: [IPFIX] RFC 6615 on Definitions of Managed Objects for IP Flow Information Export X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Jun 2012 22:48:48 -0000 A new Request for Comments is now available in online RFC libraries. RFC 6615 Title: Definitions of Managed Objects for IP Flow Information Export Author: T. Dietz, Ed., A. Kobayashi, B. Claise, G. Muenz Status: Standards Track Stream: IETF Date: June 2012 Mailbox: Thomas.Dietz@neclab.eu, akoba@nttv6.net, bclaise@cisco.com, muenz@net.in.tum.de Pages: 65 Characters: 137274 Obsoletes: RFC5815 I-D Tag: draft-ietf-ipfix-rfc5815bis-03.txt URL: http://www.rfc-editor.org/rfc/rfc6615.txt This document defines managed objects for IP Flow Information eXport (IPFIX). These objects provide information for monitoring IPFIX Exporters and IPFIX Collectors, including basic configuration information. [STANDARDS-TRACK] This document is a product of the IP Flow Information Export Working Group of the IETF. This is now a Proposed Standard Protocol. STANDARDS TRACK: This document specifies an Internet standards track protocol for the Internet community,and requests discussion and suggestions for improvements. Please refer to the current edition of the Internet Official Protocol Standards (STD 1) for the standardization state and status of this protocol. Distribution of this memo is unlimited. This announcement is sent to the IETF-Announce and rfc-dist lists. To subscribe or unsubscribe, see http://www.ietf.org/mailman/listinfo/ietf-announce http://mailman.rfc-editor.org/mailman/listinfo/rfc-dist For searching the RFC series, see http://www.rfc-editor.org/rfcsearch.html. For downloading RFCs, see http://www.rfc-editor.org/rfc.html. Requests for special distribution should be addressed to either the author of the RFC in question, or to rfc-editor@rfc-editor.org. Unless specifically noted otherwise on the RFC itself, all RFCs are for unlimited distribution. The RFC Editor Team Association Management Solutions, LLC From internet-drafts@ietf.org Wed Jun 13 12:55:04 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3E8FB11E80C4; Wed, 13 Jun 2012 12:55:04 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.599 X-Spam-Level: X-Spam-Status: No, score=-102.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lh6fxverwzDQ; Wed, 13 Jun 2012 12:55:03 -0700 (PDT) Received: from ietfa.amsl.com (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8A8E121F854B; Wed, 13 Jun 2012 12:55:03 -0700 (PDT) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: internet-drafts@ietf.org To: i-d-announce@ietf.org X-Test-IDTracker: no X-IETF-IDTracker: 4.20 Message-ID: <20120613195503.4315.19953.idtracker@ietfa.amsl.com> Date: Wed, 13 Jun 2012 12:55:03 -0700 Cc: ipfix@ietf.org Subject: [IPFIX] I-D Action: draft-ietf-ipfix-configuration-model-11.txt X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Jun 2012 19:55:04 -0000 A New Internet-Draft is available from the on-line Internet-Drafts director= ies. This draft is a work item of the IP Flow Information Export Working Group = of the IETF. Title : Configuration Data Model for IPFIX and PSAMP Author(s) : Gerhard Muenz Benoit Claise Paul Aitken Filename : draft-ietf-ipfix-configuration-model-11.txt Pages : 129 Date : 2012-06-13 Abstract: This document specifies a data model for configuring and monitoring Selection Processes, Caches, Exporting Processes, and Collecting Processes of IPFIX and PSAMP compliant Monitoring Devices using the NETCONF protocol. The data model is defined using UML (Unified Modeling Language) class diagrams and formally specified using YANG. The configuration data is encoded in Extensible Markup Language (XML). The IETF datatracker status page for this draft is: https://datatracker.ietf.org/doc/draft-ietf-ipfix-configuration-model There's also a htmlized version available at: http://tools.ietf.org/html/draft-ietf-ipfix-configuration-model-11 A diff from previous version is available at: http://tools.ietf.org/rfcdiff?url2=3Ddraft-ietf-ipfix-configuration-model-11 Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ From muenz@net.in.tum.de Wed Jun 13 14:15:54 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D0F1E21F85A4 for ; Wed, 13 Jun 2012 14:15:53 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -5.649 X-Spam-Level: X-Spam-Status: No, score=-5.649 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HELO_EQ_DE=0.35, J_CHICKENPOX_71=0.6, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Iafu1809tN5G for ; Wed, 13 Jun 2012 14:15:53 -0700 (PDT) Received: from mail-out1.informatik.tu-muenchen.de (mail-out1.informatik.tu-muenchen.de [131.159.0.8]) by ietfa.amsl.com (Postfix) with ESMTP id 13E7921F85A3 for ; Wed, 13 Jun 2012 14:15:52 -0700 (PDT) Received: from [192.168.2.34] (e181139037.adsl.alicedsl.de [85.181.139.37]) by mail.net.in.tum.de (Postfix) with ESMTPSA id EF82C2374740; Wed, 13 Jun 2012 23:15:50 +0200 (CEST) Message-ID: <4FD902EC.6040201@net.in.tum.de> Date: Wed, 13 Jun 2012 23:15:24 +0200 From: Gerhard Muenz User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:12.0) Gecko/20120428 Thunderbird/12.0.1 MIME-Version: 1.0 To: Ron Bonica , RFC Editor References: <20120613195504.4315.91564.idtracker@ietfa.amsl.com> In-Reply-To: <20120613195504.4315.91564.idtracker@ietfa.amsl.com> X-Forwarded-Message-Id: <20120613195504.4315.91564.idtracker@ietfa.amsl.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: IETF IPFIX Working Group Subject: [IPFIX] Fwd: New Version Notification for draft-ietf-ipfix-configuration-model-11.txt X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Jun 2012 21:15:54 -0000 Dear Ron, dear RFC Editors, draft-ietf-ipfix-configuration-model-10 has been in MISSREF*R state for quite a while. Since several changes have been accumulated for AUTH48, I have submitted an updated version -11 of the draft to facilitate the AUTH48 process. This -11 version addresses the following AUTH48 issues: 1) Several typos have been corrected 2) References have been updated: - [W3C.REC-xml-20040204] replaced by [W3C.REC-xml-20081126] as indicated by Peter Saint-Andre in IETF LC - [RFC4347] replaced by [RFC6347] (DTLS) - [I-D.ietf-ipfix-export-per-sctp-stream] replaced by [RFC6526] - [RFC5815] replaced by [RFC6615] (IPFIX MIB) 3) inactiveTimeout renamed idleTimeout inactiveTimeout has been renamed idleTimeout (just like in IPFIX MIB) as agreed on the IPFIX mailing list: http://www.ietf.org/mail-archive/web/ipfix/current/msg06363.html 4) Description of isFlowKey in YANG spec has been corrected 5) Additional state parameters: meteringProcessId, exportingProcessId The addition of these parameters was discussed on the IPFIX mailing list: http://www.ietf.org/mail-archive/web/ipfix/current/msg06072.html The reason why selectorId cannot be added is given here: http://www.ietf.org/mail-archive/web/ipfix/current/msg06194.html Thanks, Gerhard -------- Original Message -------- Subject: New Version Notification for draft-ietf-ipfix-configuration-model-11.txt Date: Wed, 13 Jun 2012 12:55:04 -0700 From: internet-drafts@ietf.org To: muenz@net.in.tum.de CC: bclaise@cisco.com, paitken@cisco.com A new version of I-D, draft-ietf-ipfix-configuration-model-11.txt has been successfully submitted by Gerhard Muenz and posted to the IETF repository. Filename: draft-ietf-ipfix-configuration-model Revision: 11 Title: Configuration Data Model for IPFIX and PSAMP Creation date: 2012-06-12 WG ID: ipfix Number of pages: 129 URL: http://www.ietf.org/internet-drafts/draft-ietf-ipfix-configuration-model-11.txt Status: http://datatracker.ietf.org/doc/draft-ietf-ipfix-configuration-model Htmlized: http://tools.ietf.org/html/draft-ietf-ipfix-configuration-model-11 Diff: http://tools.ietf.org/rfcdiff?url2=draft-ietf-ipfix-configuration-model-11 Abstract: This document specifies a data model for configuring and monitoring Selection Processes, Caches, Exporting Processes, and Collecting Processes of IPFIX and PSAMP compliant Monitoring Devices using the NETCONF protocol. The data model is defined using UML (Unified Modeling Language) class diagrams and formally specified using YANG. The configuration data is encoded in Extensible Markup Language (XML). The IETF Secretariat From paitken@cisco.com Fri Jun 15 14:34:16 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7AC3111E80D9 for ; Fri, 15 Jun 2012 14:34:16 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -5.598 X-Spam-Level: X-Spam-Status: No, score=-5.598 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, GB_SUMOF=5, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2ZlEJQzPg95p for ; Fri, 15 Jun 2012 14:34:07 -0700 (PDT) Received: from ams-iport-4.cisco.com (ams-iport-4.cisco.com [144.254.224.147]) by ietfa.amsl.com (Postfix) with ESMTP id 8354811E8088 for ; Fri, 15 Jun 2012 14:34:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=paitken@cisco.com; l=186098; q=dns/txt; s=iport; t=1339796042; x=1341005642; h=message-id:date:from:mime-version:to:cc:subject; bh=YU95fw8My8ma1OMM1DRoN+jAM7qLjPuhOY1O5KWe1Qs=; b=JaM9NqCbOSHKDC+ycpg97jyP640qMTc7lngSYV1qFy7GOIThvlRqDED7 Ly+fqLcyvmJS8Ig9T8Chpm5MeBgIm/ORAfp0UALSV+Js+hevDdy3LhJIz CMLtHmSC6gRMT4dLOHkN435ya3mKs4DvTVLaR9eQM1910eDlfOiY4C4YJ s=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AgwFADCp20+Q/khN/2dsb2JhbAA7BwOLNqohgQeCGAEBAQMTAQcBDEASBQkfDwwLARwCCi4eAwEFAgEBBRmFb4F1BQuZQ6ARBIs7AQYEDIJ0B4MgA4scigiBEneDS2yHV4FmgmGBVQEI X-IronPort-AV: E=Sophos;i="4.75,780,1330905600"; d="scan'208,217";a="5853907" Received: from ams-core-4.cisco.com ([144.254.72.77]) by ams-iport-4.cisco.com with ESMTP; 15 Jun 2012 21:33:59 +0000 Received: from [10.55.93.37] (dhcp-10-55-93-37.cisco.com [10.55.93.37]) by ams-core-4.cisco.com (8.14.5/8.14.5) with ESMTP id q5FLXsKe023913; Fri, 15 Jun 2012 21:33:54 GMT Message-ID: <4FDBAA42.60003@cisco.com> Date: Fri, 15 Jun 2012 22:33:54 +0100 From: Paul Aitken User-Agent: Mozilla/5.0 (X11; Linux i686; rv:12.0) Gecko/20120430 Thunderbird/12.0.1 MIME-Version: 1.0 To: draft-trammell-ipfix-a9n@tools.ietf.org Content-Type: multipart/alternative; boundary="------------020106010107040308030706" Cc: IETF IPFIX Working Group Subject: [IPFIX] review of draft-trammell-ipfix-a9n-03 X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Jun 2012 21:34:16 -0000 This is a multi-part message in MIME format. --------------020106010107040308030706 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Dear Authors, At long last, here's a review of draft-trammell-ipfix-a9n-03. Thanks for your great patience! P. > IPFIX Working Group B. Trammell > Internet-Draft E. Boschi > Intended status: Standards Track ETH Zurich > Expires: December 31, 2011 A. Wagner > Consecom AG > B. Claise > Cisco Systems, Inc. > June 29, 2011 > > > Exporting Aggregated Flow Data using the IP Flow Information Export > (IPFIX) Protocol > draft-trammell-ipfix-a9n-03.txt > > Abstract > > This document describes the export of aggregated Flow information > using IPFIX. An Aggregated Flow is essentially an IPFIX Flow > representing packets from multiple original Flows sharing some set of > common properties. The document describes Aggregated Flow export > within the framework of IPFIX Mediators and defines an interoperable, > implementation-independent method for Aggregated Flow export. > > Status of this Memo > > This Internet-Draft is submitted in full conformance with the > provisions of BCP 78 and BCP 79. > > Internet-Drafts are working documents of the Internet Engineering > Task Force (IETF). Note that other groups may also distribute > working documents as Internet-Drafts. The list of current Internet- > Drafts is athttp://datatracker.ietf.org/drafts/current/. > > Internet-Drafts are draft documents valid for a maximum of six months > and may be updated, replaced, or obsoleted by other documents at any > time. It is inappropriate to use Internet-Drafts as reference > material or to cite them other than as "work in progress." > > This Internet-Draft will expire on December 31, 2011. > > Copyright Notice > > Copyright (c) 2011 IETF Trust and the persons identified as the > document authors. All rights reserved. > > This document is subject to BCP 78 and the IETF Trust's Legal > Provisions Relating to IETF Documents > (http://trustee.ietf.org/license-info) in effect on the date of > > > > Trammell, et al. Expires December 31, 2011 [Page 1] > > Internet-Draft IPFIX Aggregation June 2011 > > > publication of this document. Please review these documents > carefully, as they describe your rights and restrictions with respect > to this document. Code Components extracted from this document must > include Simplified BSD License text as described in Section 4.e of > the Trust Legal Provisions and are provided without warranty as > described in the Simplified BSD License. > > > Table of Contents > > 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 > 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 5 > 3. Use Cases for IPFIX Aggregation . . . . . . . . . . . . . . . 6 > 4. Architecture for Flow Aggregation . . . . . . . . . . . . . . 6 > 4.1. Aggregation within the IPFIX Architecture . . . . . . . . 7 > 4.2. Intermediate Aggregation Process Architecture . . . . . . 8 > 5. IP Flow Aggregation Operations . . . . . . . . . . . . . . . . 10 > 5.1. Temporal Aggregation through Interval Distribution . . . . 10 > 5.1.1. Distributing Values Across Intervals . . . . . . . . . 11 > 5.1.2. Time Composition . . . . . . . . . . . . . . . . . . . 13 > 5.2. Spatial Aggregation of Flow Keys . . . . . . . . . . . . . 13 > 5.2.1. Counting Distinct Key Values . . . . . . . . . . . . . 15 > 5.2.2. Counting Original Flows . . . . . . . . . . . . . . . 15 > 5.3. Spatial Aggregation of Non-Key Fields . . . . . . . . . . 16 > 5.3.1. Counter Statistics . . . . . . . . . . . . . . . . . . 16 > 5.4. Aggregation Combination . . . . . . . . . . . . . . . . . 17 > 6. Additional Considerations and Special Cases in Flow > Aggregation . . . . . . . . . . . . . . . . . . . . . . . . . 17 > 6.1. Exact versus Approximate Counting during Aggregation . . . 17 > 6.2. Considerations for Aggregation of Sampled Flows . . . . . 17 > 7. Export of Aggregated IP Flows using IPFIX . . . . . . . . . . 17 > 7.1. Time Interval Export . . . . . . . . . . . . . . . . . . . 18 > 7.2. Flow Count Export . . . . . . . . . . . . . . . . . . . . 18 > 7.2.1. originalFlowsPresent . . . . . . . . . . . . . . . . . 18 > 7.2.2. originalFlowsInitiated . . . . . . . . . . . . . . . . 18 > 7.2.3. originalFlowsCompleted . . . . . . . . . . . . . . . . 19 > 7.2.4. originalFlows . . . . . . . . . . . . . . . . . . . . 19 > 7.3. Distinct Host Export . . . . . . . . . . . . . . . . . . . 19 > 7.3.1. distinctCountOfSourceIPv4Address . . . . . . . . . . . 19 > 7.3.2. distinctCountOfDestinationIPv4Address . . . . . . . . 20 > 7.3.3. distinctCountOfSourceIPv6Address . . . . . . . . . . . 20 > 7.3.4. distinctCountOfDestinationIPv6Address . . . . . . . . 20 > 7.4. Aggregate Counter Distribution Export . . . . . . . . . . 20 > 7.4.1. Aggregate Counter Distribution Options Template . . . 21 > 7.4.2. valueDistributionMethod Information Element . . . . . 21 > 8. Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 > 8.1. Traffic Time-Series per Source . . . . . . . . . . . . . . 24 > 8.2. Core Traffic Matrix . . . . . . . . . . . . . . . . . . . 28 > > > > Trammell, et al. Expires December 31, 2011 [Page 2] > > Internet-Draft IPFIX Aggregation June 2011 > > > 8.3. Distinct Source Count per Destination Endpoint . . . . . . 28 > 8.4. Traffic Time-Series per Source with Counter > Distribution . . . . . . . . . . . . . . . . . . . . . . . 29 > 9. Security Considerations . . . . . . . . . . . . . . . . . . . 29 > 10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 29 > 11. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 29 > 12. References . . . . . . . . . . . . . . . . . . . . . . . . . . 30 > 12.1. Normative References . . . . . . . . . . . . . . . . . . . 30 > 12.2. Informative References . . . . . . . . . . . . . . . . . . 30 > Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 31 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Trammell, et al. Expires December 31, 2011 [Page 3] > > Internet-Draft IPFIX Aggregation June 2011 > > > 1. Introduction > > The aggregation of packet data into Flows serves a variety of > different purposes, as noted in the requirements [RFC3917] and > applicability statement [RFC5472] for the IP Flow Information Export > (IPFIX) protocol [RFC5101]. Aggregation beyond the flow level, into > records representing multiple Flows, is a common analysis and data > reduction technique as well, with applicability to large-scale > network data analysis, archiving, and inter-organization exchange. > This applicability in large-scale situations, in particular, led to > the inclusion of aggregation as part of the IPFIX Mediators Problem > Statement [RFC5982], and the definition of an Intermediate > Aggregation Process in the Mediator framework > [I-D.ietf-ipfix-mediators-framework]. > > Aggregation is part of a wide variety of applications, including > traffic matrix calculation, generation of time series data for > visualizations or anomaly detection, or measurement data reduction. There seems to be a missing point here. ie, "Aggregation is part of a wide variety of applications"... used for a specific purpose? > Depending on the keys used for aggregation, it may additionally have > an anonymising affect on the data: for example, aggregation > operations which eliminate IP addresses make it impossible to later > identify nodes using those addresses. There could be enough other information. eg, destination port = 80 could identify a web server. However, "may have an anonymising effect" seems sufficient for the present purpose. > Aggregation as defined and described in this document covers the > applications defined in [RFC5982], including 5.1 "Adjusting Flow > Granularity", 5.4 "Time Composition", and 5.5 "Spatial Composition". > However, this document specifies a more flexible architecture for an > Intermediate Aggregation Process in Section 4.2, which supports a > superset of these applications. So this document isn't just about _export_ as the title and abstract suggest. Perhaps the title needs to change to "Aggregating and Exporting Flow Data using IPFIX" ? The abstract should indicate that the document defines the IAP. > An Intermediate Aggregation Process may be applied to data collected > from multiple Observation Points, as aggregation is natural to apply "aggregation is natural to apply" -> "it's natural to apply aggregation". > for data reduction when concentrating measurement data. This > document specifically does not address the protocol issues that arise > when combining IPFIX data from multiple Observation Points and > exporting from a single Mediator, as these issues are general to > IPFIX Mediation; they are therefore treated in detail in the Mediator Mediator -> Mediation - or, as that text actually says, "Mediations" :-( > Protocol [I-D.claise-ipfix-mediation-protocol] document. > > Since Aggregated Flows as defined in the following section are > essentially Flows, the IPFIX protocol [RFC5101] can be used to > export, and the IPFIX File Format [RFC5655] can be used to store, > aggregated data "as-is"; there are no changes necessary to the > protocol. This document provides a common basis for the application > of IPFIX to the handling of aggregated data, through a detailed > terminology, Intermediate Aggregation Process architecture, and > methods for original Flow counting and counter distribution across > intervals. That last line sounds like something for the abstract. > > > Trammell, et al. Expires December 31, 2011 [Page 4] > > Internet-Draft IPFIX Aggregation June 2011 > > > 2. Terminology > > Terms used in this document that are defined in the Terminology > section of the IPFIX Protocol [RFC5101] document are to be > interpreted as defined there. > > The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", > "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this > document are to be interpreted as described in [RFC2119]. > > In addition, this document defines the following terms > > Aggregated Flow: A Flow, as defined by [RFC5101], derived from a > set of zero or more original Flows within a defined Aggregation > Interval. The two primary differences between a Flow and an > Aggregated Flow are (1) that the time interval of a Flow is > generally derived from information about the timing of the packets > comprising the Flow, while the time interval of an Aggregated Flow > are generally externally imposed; and (2) that an Aggregated Flow > may represent zero packets (i.e., an assertion that no packets > were seen for a given Flow Key in a given time interval). Note That's an oversight in 5101. We should have allowed a Flow to represent zero or more packets. It shouldn't be necessary to define an aggregated flow in order to export that nothing was seen. This should be fixed in 5101bis, and (2) removed above. > that an Aggregated Flow is defined within the context of an > Intermediate Aggregation Process only. once an Aggregated Flow is > exported, it is essentially a Flow as in [RFC5101] and can be > treated as such. > > Intermediate Aggregation Function: A mapping from a set of zero or > more original Flows into a set of Aggregated Flows across one or > more Aggregation Intervals. This function is hosted by an This definition only allows for time based aggregation, and not for spatial aggregation. ie, this allows for "in interval T1 and T2, I saw IP addresses A1, A2, A3, and A4." (ie, addresses are aggregated). Suppose we want to do it the other way, ie: I saw IP address w.x.y.z between T1 and T2, not between T2 and T3, then again between T3 and T4. (ie, time is aggregated). Also see example 8.3, which has no time interval. > Intermediate Aggregation Process, defined below. > > Intermediate Aggregation Process: an Intermediate Process as in > [I-D.ietf-ipfix-mediators-framework] that aggregates records based > upon a set of Flow Keys or functions applied to fields from the > record; this is itself defined in > [I-D.ietf-ipfix-mediators-framework]. > > Aggregation Interval: A time interval imposed upon an Aggregated > Flow. Aggregation Functions may use a regular Aggregation > Interval (e.g. "every five minutes", "every calendar month"), > though regularity is not necessary. Aggregation intervals may > also be derived from the time intervals of the original Flows > being aggregated. > > partially aggregated Flow: A Flow during processing within an > Intermediate Aggregation Process; refers to an intermediate data > structure during aggregation within the Intermediate Aggregation > Process architecture detailed in Section 4.2. Are "partially aggregated Flows" really necessary? > > > Trammell, et al. Expires December 31, 2011 [Page 5] > > Internet-Draft IPFIX Aggregation June 2011 > > > original Flow: A Flow given as input to an Aggregation Function in > order to generate Aggregated Flows. > > contributing Flow: An original Flow that is partially or completely > represented within an Aggregated Flow. Each aggregated Flow is > made up of zero or more contributing Flows, and an original Flow > may contribute to zero or more Aggregated Flows. A "contributing Flow" seems to be little more than an original Flow. Certainly there's no such thing as a "non-contributing Flow". (There could be, but it'd be irrelevant.) So is the distinction useful? Can't you just call these "original Flows" ? > > 3. Use Cases for IPFIX Aggregation > > Aggregation, as a common data analysis method, has many applications. > When used with a regular Aggregation Interval, it generates time > series data from a collection of Flows with discrete intervals. Time > series data is itself useful for a wide variety of analysis tasks, > such as generating input for network anomaly detection systems, or > driving visualizations of volume per time for traffic with specific Fixated on time here. See earlier comments. Specifically, aggregation does not "generates time series data". It depends what is aggregated. Example 8.3 is the only place which acknowledges that time is not required. Yet time isn't special, it's just a field like any other. > characteristics. Traffic matrix calculation from flow data is > inherently an aggregation action, by aggregating the Flow Key down to > input or output interface, address prefix, or autonomous system. > > Irregular or data-dependent Aggregation Intervals and key aggregation > operations can also be used to provide adaptive aggregation of > network flow data. Here, full Flow Records can be kept for Flows of > interest, while Flows deemed "less interesting" to a given > application can be aggregated. For example, in an IPFIX Mediator > equipped with traffic classification capabilities for security > purposes, potentially malicious Flows could be exported directly, > while known-good or probably-good Flows (e.g. normal web browsing) > could be exported simply as time series volumes per web server. > > Note that an Intermediate Aggregation Function which removes > potentially sensitive information as identified in > [I-D.ietf-ipfix-anon] may tend to have an anonymising effect on the > Aggregated Flows, as well; however, any application of aggregation as > part of a data protection scheme should ensure that all the issues > raised in Section 4 of [I-D.ietf-ipfix-anon] are addressed. > > > 4. Architecture for Flow Aggregation > > This section specifies how an Intermediate Aggregation Process fits > into the IPFIX Architecture, and the architecture of the Intermediate > Aggregation Process itself. That's ambiguous (ie, "This section specifies how an Intermediate Aggregation Process fits into ... the architecture of the Intermediate Aggregation Process itself."). Better to say, This section specifies the architecture of the Intermediate Aggregation Process, and how an Intermediate Aggregation Process fits into the IPFIX Architecture. > > > > > > > Trammell, et al. Expires December 31, 2011 [Page 6] > > Internet-Draft IPFIX Aggregation June 2011 > > > 4.1. Aggregation within the IPFIX Architecture > > An Intermediate Aggregation Process may be deployed at three places > within the IPFIX Architecture. While aggregation applications are > most commonly deployed within a Mediator which collects original > Flows from an original Exporter and exports Aggregated Flows, > aggregation can also occur before initial export, or after final > collection, as shown in Figure 1. > > +==========================================+ > | Exporting Process | > +==========================================+ > | | > | (Aggregated Flow Export) | > V | > +=============================+ | > | Mediator | | > +=============================+ | > | | > | (Aggregating Mediator) | "Aggregating Mediator" describes the process, so it should be in the box. > V V > +==========================================+ > | Collecting Process | > +==========================================+ > | > | (Aggregation for Storage) Again, the label is a process which should either be shown in the box or in a subsequent box. > V > +--------------------+ > | IPFIX File Storage | > +--------------------+ > > Figure 1: Potential Aggregation Locations > > The Mediator use case is further shown in Figures A and B in > [I-D.ietf-ipfix-mediators-framework]. > > Aggregation can be applied for either intermediate or final analytic > purposes. In certain circumstances, it may make sense to export > Aggregated Flows directly from an original Exporting Process, for > example, if the Exporting Process is applied to drive a time-series > visualization, or when flow data export bandwidth is restricted and > flow or packet sampling is not an option. Note that this case, where > the Aggregation Process is essentially integrated into the Metering > Process, is essentially covered by the IPFIX architecture [RFC5470]: > the Flow Keys used are simply a subset of those that would normally > be used. A Metering Process in this arrangement MAY choose to > simulate the generation of larger Flows in order to generate original > Flow counts, if the application calls for compatibility with an > > > > Trammell, et al. Expires December 31, 2011 [Page 7] > > Internet-Draft IPFIX Aggregation June 2011 > > > Aggregation Process deployed in a separate location. > > In the specific case that an Aggregation Process is employed for data > reduction for storage purposes, it can take original Flows from a > Collecting Process or File Reader and pass Aggregated Flows to a File > Writer for storage. > > Deployment of an Intermediate Aggregation Process within a Mediator > [RFC5982] is a much more flexible arrangement. Here, the Mediator > consumes original Flows and produces aggregated Flows; this > arrangement is suited to any of the use cases detailed in Section 3. > In a mediator, aggregation can be applied as well to aggregating "can be applied as well" -> "can also be applied" > original Flows from multiple sources into a single stream of > aggregated Flows; the architectural specifics of this arrangement are > not addressed in this document, which is concerned only with the > aggregation operation itself; see > [I-D.claise-ipfix-mediation-protocol] for details. > > The data paths into and out of an Intermediate Aggregation Process > are showin in Figure 2. s/showin/shown/ > packets --+ +- IPFIX Messages -+ > | | | > V V V > +==================+ +====================+ +=============+ > | Metering Process | | Collecting Process | | File Reader | > | | +====================+ +=============+ > | | | original Flows | > | | V V > + - - - - - - - - -+======================================+ > | Intermediate Aggregation Process (IAP) | > +=========================================================+ > | Aggregated Aggregated | > | Flows Flows | > V V > +===================+ +=============+ > | Exporting Process | | File Writer | > +===================+ +=============+ > | | > +------------> IPFIX Messages<----------+ > > Figure 2: Data paths through the aggregation process An IPFIX File Reader consumes IPFIX Files, not IPFIX Messages. An IPFIX File Write produces IPFIX Files, not IPFIX Messages. Surely the process for (re)generating IPFIX messages is: IPFIX Messages -> File Writer -> IPFIX File -> File Reader -> Exporting Process -> IPFIX Messages > 4.2. Intermediate Aggregation Process Architecture > > Within this document, an Intermediate Aggregation Process can be seen > as hosting an Intermediate Aggregation Function composed of four > types of operations on the intermediate results of aggregation, which > > > > Trammell, et al. Expires December 31, 2011 [Page 8] > > Internet-Draft IPFIX Aggregation June 2011 > > > are called partially aggregated Flows in this document, as > illustrated in Figure 3. > > original Flows Note that here you say "original Flows" rather than contributing Flows. See my earlier comment. > | > V > +-----------------------+ > | interval distribution | > +-->| (temporal) |<--+ > | +-----------------------+ | > | | | | | > |(*) |(*) |(*) |(*) |(*) > | | | | | > | V | V | > +-------------------+ | +--------------------+ > | key aggregation | | | value aggregation | > | (spatial) | | | (spatial) | > +-------------------+ | +--------------------+ > ^ | | | ^ > | |(*) | |(*) | > +-------|-------|-------|-------+ > V V V The top half of the figure uses two single-ended arrows (one each way) to connect interval to key / interval to value. However, the lower half uses just one double-ended arrow to link key and value - which is confusing at first, because it looks like a link with no possible input. Either draw two arrows in the lower half, or lose two arrows in the upper half and make the other two upper-half arrows double-ended. > +-------------------------+ > | aggregate combination | > +-------------------------+ > | > V > Aggregated Flows > > (*) partially aggregated Flows > > Figure 3: Conceptual model of aggregation operations > > Interval distribution is a temporal aggregation operation which > imposes an Aggregation Interval on the partially aggregated Flow. > This Aggregation Interval may be regular, irregular, or derived > from the timing of the original Flows themselves. Interval > distribution is discussed in detail in Section 5.1. Must this be done first? > Key aggregation is a spatial aggregation operation which results in > the addition, modification, or deletion of Flow Key fields in the > partially aggregated Flows. New Flow Key fields may be derived > from existing Flow Key fields (e.g., looking up an AS number for > an IP address), or "promoted" from non-Key fields (e.g., when > aggregating Flows by packet count per Flow). Key aggregation can The above sounds like any non-key can be promoted, which isn't true and needs to be discussed (see later). Consider, "or derived from certain specific non-Key fields" ? > also add new non-Key fields derived from Key Fields that are > deleted during key aggregation; mainly counters of unique reduced > keys. Key aggregation is discussed in detail in Section 5.2. > > > > Trammell, et al. Expires December 31, 2011 [Page 9] > > Internet-Draft IPFIX Aggregation June 2011 > > > Value aggregation is a spatial aggregation operation which results > in the addition, modification, or deletion of non-Key fields in > the partially aggregated Flows. These non-Key fields may be > "demoted" from existing Key fields, or derived from existing Key > or non-Key fields. Value aggregation is discussed in detail in > Section 5.3. > > Aggregate combination combines multiple partially aggregated Flows > having undergone interval distribution, key aggregation, and value > aggregation which share Flow Keys and Aggregation Intervals into a > single aggregated Flow per Flow Key and Aggregation Interval. > Aggregate combination is discussed in detail in Section 5.4. > > The first three of these operations may be carried out any number of > times in any order, either on original Flows or on the results of one "in any order" contradicts Figure 3, which distinctly shows that "interval distribution" must be first. > of the Operations (called partially aggregated Flows), with one "Operations" isn't defined. Either define it, or de-capitalise it. > caveat. Since Flows carry their own interval data, any spatial > aggregation operation implies a temporal aggregation operation, so at > least one interval distribution step, even if implicit, is required > by this architecture. This is shown as the first step for the sake > of simplicity in the diagram above. Once all aggregation operations > are complete, aggregate combination ensures that for a given > Aggregation Interval, Flow Key, and Observation Domain, only one Flow > is produced by the Intermediate Aggregation Process. > > > 5. IP Flow Aggregation Operations > > As stated in Section 2, an Aggregated Flow is simply an IPFIX Flow > generated from original Flows by an Aggregation Function. Here, we > detail the operations by which this is achieved within an > Intermediate Aggregation Process. > > 5.1. Temporal Aggregation through Interval Distribution > > Interval distribution imposes a time interval on the resulting > Aggregated Flows. The selection of an interval is specific to the > given aggregation application. Intervals may be derived from the > original Flows themselves (e.g., an interval may be selected to cover > the entire interval containing the set of all Flows sharing a given Avoid "interval ... interval", eg by changing the second occurrence to "time" ? > Key, as in Time Composition describe in Section 5.1.2) or externally Typo, "described". > imposed; in the latter case the externally imposed interval may be > regular (e.g., every five minutes) or irregular (e.g., to allow for > different time resolutions at different times of day, under different > network conditions, or indeed for different sets of original Flows). How does the Aggregated Flow consumer know what the aggregation interval was, especially in the "irregular" case? ie, with regular intervals, a collector can simply store the received Aggregates as eg hourly reports. However, with irregular intervals it may have two minutes here and ten minutes there... which may not be easy to process in a meaningful way? > The length of the imposed interval itself has tradeoffs. Shorter > intervals allow higher resolution aggregated data and, in streaming > > > > Trammell, et al. Expires December 31, 2011 [Page 10] > > Internet-Draft IPFIX Aggregation June 2011 > > > applications, faster reaction time. Longer intervals lead to greater > data reduction and simplified counter distribution. Specifically, *may* lead to greater data reduction, though not necessarily. > counter distribution is greatly simplified by the choice of an > interval longer than the duration of longest original Flow, itself > generally determined by the original Flow's Metering Process active > timeout; in this case an original Flow can contribute to at most two > Aggregated Flows, and the more complex value distribution methods > become inapplicable. This only considers the Metering Process; it fails to consider any delay in the Exporting Process. eg, consider and implementation with 64K cache entries which are checked for exporting at a rate of 1000 entries/second. It will take > 1 minute to check and export the entire cache. Again, consider an implementation which aims to maximise the size of each export packet, building it up a record at a time as records expire from the cache, until the export packet is quite full. If records expire slowly then the mechanism may introduce a significant delay. In effect both of these delays cause flows to appear in later intervals than may be expected. eg, in Figure 4, "Flow A" may appear in interval 1 or interval 2, rather than interval 0. > | | | | > | |<--Flow A-->| | | | > | |<--Flow B-->| | | > | |<-------------Flow C-------------->| | > | | | | > | interval 0 | interval 1 | interval 2 | > > Figure 4: Illustration of interval distribution > > In Figure 4, we illustrate three common possibilities for interval > distribution as applies with regular intervals to a set of three > original Flows. For Flow A, the start and end times lie within the > boundaries of a single interval 0; therefore, Flow A contributes to > only one Aggregated Flow. Flow B, by contrast, has the same duration > but crosses the boundary between intervals 0 and 1; therefore, it > will contribute to two Aggregated Flows, and its counters must be > distributed among these Flows, though in the two-interval case this > can be simplified somewhat simply by picking one of the two > intervals, or proportionally distributing between them. Only Flows > like Flow A and Flow B will be produced when the interval is chosen > to be longer than the duration of longest original Flow, as above. > More complicated is the case of Flow C, which contributes to more > than two Aggregated Flows, and must have its counters distributed > according to some policy as in Section 5.1.1. > > [EDITOR'S NOTE: per Lothar: some implementation guidance here would > be good. specifically, advise that you need multiple rotating > intervals to do this right.] TODO > 5.1.1. Distributing Values Across Intervals > > In general, counters in Aggregated Flows are treated the same as in > any Flow. Each counter is independently is calculated as if it were Typo, "independently is calculated". > derived from the set of packets in the original Flow. For the most > part, when aggregating original Flows into Aggregated Flows, this is > simply done by summation. > > When the Aggregation Interval is guaranteed to be longer than the > longest original Flow, a Flow can cross at most one Interval > > > > Trammell, et al. Expires December 31, 2011 [Page 11] > > Internet-Draft IPFIX Aggregation June 2011 > > > boundary, and will therefore contribute to at most two Aggregated > Flows. Most common in this case is to arbitrarily but consistently > choose to account the original Flow's counters either to the first or > the last aggregated Flow to which it could contribute. "Aggregated Flow" > However, this becomes more complicated when the Aggregation Interval > is shorter than the longest original Flow in the source data. In > such cases, each original Flow can incompletely cover one or more > time intervals, and apply to one or more Aggregated Flows; in this > case, the Aggregation Process must distribute the counters in the > original Flows across the multiple Aggregated Flows. There are > several methods for doing this, listed here in roughly increasing > order of complexity and accuracy; most of these are necessary only in > specialized cases. > > End Interval: The counters for an original Flow are added to the > counters of the appropriate Aggregated Flow containing the end > time of the original Flow. > > Start Interval: The counters for an original Flow are added to the > counters of the appropriate Aggregated Flow containing the start > time of the original Flow. > > Mid Interval: The counters for an original Flow are added to the > counters of a single appropriate Aggregated Flow containing some > timestamp between start and end time of the original Flow. > > Simple Uniform Distribution: Each counter for an original Flow is > divided by the number of time intervals the original Flow covers > (i.e., of appropriate Aggregated Flows sharing the same Flow Key), > and this number is added to each corresponding counter in each > Aggregated Flow. > > Proportional Uniform Distribution: Each counter for an original > Flow is divided by the number of time _units_ the original Flow > covers, to derive a mean count rate. This mean count rate is then > multiplied by the number of time units in the intersection of the > duration of the original Flow and the time interval of each > Aggregated Flow. This is like simple uniform distribution, but > accounts for the fractional portions of a time interval covered by > an original Flow in the first and last time interval. > > Simulated Process: Each counter of the original Flow is distributed > among the intervals of the Aggregated Flows according to some > function the Aggregation Process uses based upon properties of > Flows presumed to be like the original Flow. For example, Flow > Records representing bulk transfer might follow a more or less > proportional uniform distribution, while interactive processes are > > > > Trammell, et al. Expires December 31, 2011 [Page 12] > > Internet-Draft IPFIX Aggregation June 2011 > > > far more bursty. > > Direct: The Aggregation Process has access to the original packet > timings from the packets making up the original Flow, and uses > these to distribute or recalculate the counters. > > A method for exporting the distribution of counters across multiple > Aggregated Flows is detailed in Section 7.4. In any case, counters > MUST be distributed across the multiple Aggregated Flows in such a > way that the total count is preserved, within the limits of accuracy > of the implementation (e.g., inaccuracy introduced by the use of > floating-point numbers is tolerable). This property allows data to > be aggregated and re-aggregated without any loss of original count "without any loss" -> "with only a tolerable loss". Personally, I'd drop the parenthesised FP bit. > information. To avoid confusion in interpretation of the aggregated > data, all the counters for a set of given original Flows SHOULD be > distributed via the same method. > > 5.1.2. Time Composition > > Time Composition as in section 5.4 of [RFC5982] (or interval > combination) is a special case of aggregation, where interval > distribution imposes longer intervals on Flows with matching keys and > "chained" start and end times, without any key reduction, in order to > join long-lived Flows which may have been split (e.g., due to an > active timeout shorter than the Flow.) Here, no Key aggregation is > applied, and the Aggregation Interval is chosen on a per-Flow basis > to cover the interval spanned by the set of aggregated Flows. This > may be applied alone in order to normalize split Flows, or in > combination with other aggregation functions in order to obtain more > accurate original Flow counts. > > 5.2. Spatial Aggregation of Flow Keys > > Key aggregation generates a new Flow Key for the Aggregated Flows > from the original Flow Keys, non-Key fields in the original Flows, or "from the original Flow Key and non-Key fields, or" > from correlation of the original Flow information with some external > source. There are two basic operations here. First, Aggregated Flow > Keys may be derived directly from original Flow Keys through > reduction, or the dropping of fields or precision in the original > Flow Keys. Second, an Aggregated Flow Key may be derived through > replacement, e.g. by removing one or more fields from the original > Flow and replacing them with a fields derived from the removed > fields. Replacement may refer to external information (e.g., IP to > AS number mappings). Replacement need not replace only key fields. > For example, consider an application which aggregates flows by packet > count (i.e., generating an Aggregated Flow for all one-packet Flows, > one for all two-packet Flows, and so on). This application would > promote the packet count to a Flow Key field. Say more about promotion, ie it can only be done on non-key fields which represent a single unique value. eg, counters, time. If a NK field might be any value from a set of >1 values (eg, an IP address, port number, AS, TOS) then it's not suitable for promotion to Key, because there's no way of knowing which other values were or were not observed, and therefore no way of dividing the flow amongst those values. > > > Trammell, et al. Expires December 31, 2011 [Page 13] > > Internet-Draft IPFIX Aggregation June 2011 > > > Key aggregation may also result in the addition of new non-Key fields > to the Aggregated Flows, namely original Flow counters and unique > reduced key counters; these are treated in more detail in > Section 5.2.2 and Section 5.2.1, respectively. I'm sure other non-keys could be derived too, though a good example doesn't immediately spring to mind. > In any key aggregation operation, reduction and/or replacement may be > applied any number of times in any order. Which of these operations > are supported by a given implementation is implementation- and > application-dependent. Key aggregation may aggregate original Flows > with different sets of Flow Key fields; only the Flow Keys of the > resulting Aggregated Flows of any given Key aggregation operation > need contain the same set of fields. I don't understand what "only ... fields" is saying. > Original Flow Key > +---------+---------+----------+----------+-------+-----+ > | src ip4 | dst ip4 | src port | dst port | proto | tos | > +---------+---------+----------+----------+-------+-----+ > | | | | | | > retain mask /24 X X X X > V V > +---------+-------------+ > | src ip4 | dst ip4 /24 | > +---------+-------------+ > Aggregated Flow Key (by source address and destination class-C) > > Figure 5: Illustration of key aggregation by reduction > > Figure 5 illustrates an example reduction operation, aggregation by > source address and destination class C network. Here, the port, > protocol, and type-of-service information is removed from the Flow is -> are ? > Key, the source address is retained, and the destination address is > masked by dropping the low 8 bits. low -> lower ? > Original Flow Key > +---------+---------+----------+----------+-------+-----+ > | src ip4 | dst ip4 | src port | dst port | proto | tos | > +---------+---------+----------+----------+-------+-----+ > | | | | | | > +-------------------+ X X X X > | ASN lookup table | > +-------------------+ > V V > +---------+---------+ > | src asn | dst asn | > +---------+---------+ > Aggregated Flow Key (by source and dest ASN) > > Figure 6: Illustration of key aggregation by reduction and > > > > Trammell, et al. Expires December 31, 2011 [Page 14] > > Internet-Draft IPFIX Aggregation June 2011 > > > replacement > > Figure 6 illustrates an example reduction and replacement operation, > aggregation by source and destination ASN without ASN information > available in the original Flow. Here, the port, protocol, and type- > of-service information is removed from the Flow Key, while the source is -> are? > and destination addresses are run though an IP address to ASN lookup > table, and the Aggregated Flow Key is made up of the resulting source > and destination ASNs. > > 5.2.1. Counting Distinct Key Values > > One common case in aggregation is counting distinct key values that > were reduced away during key aggregation. The most common use case > for this is counting distinct hosts per Flow Key; for example, in > host characterization or anomaly detection, distinct sources per > destination or distinct destinations per source are common metrics. > These new non-Ley fields are added during key aggregation. s/Ley/Key/ > For such applications, Information Elements for distinct counts of > IPv4 and IPv6 addresses are defined in Section 7.3. These are named > distinctCountOf(KeyName). Additional such Information Elements > SHOULD be registered with IANA on an as-needed basis. > > 5.2.2. Counting Original Flows > > When aggregating multiple original Flows into an Aggregated Flow, it > is often useful to know how many original Flows are present in the > Aggregated Flow. This document introduces four new information > elements in Section 7.2 to export these counters. > > There are two possible ways to count original Flows, which we call > here conservative and non-conservative. Conservative flow counting > has the property that each original Flow contributes exactly one to > the total flow count within a set of aggregated Flows. In other > words, conservative flow counters are distributed just as any other > counter during interval distribution, except each original Flow is > assumed to have a flow count of one. When a count for an original > Flow must be distributed across a set of Aggregated Flows, and a > distribution method is used which does not account for that original > Flow completely within a single Aggregated Flow, conservative flow > counting requires a fractional representation. > > By contrast, non-conservative flow counting is used to count how many > contributing Flows are represented in an Aggregated Flow. Flow > counters are not distributed in this case. An original Flow which is > present within N Aggregated Flows would add N to the sum of non- > conservative flow counts, one to each Aggregated Flow. In other > > > > Trammell, et al. Expires December 31, 2011 [Page 15] > > Internet-Draft IPFIX Aggregation June 2011 > > > words, the sum of conservative flow counts over a set of Aggregated > Flows is always equal to the number of original Flows, while the sum > of non-conservative flow counts is strictly greater than or equal to > the number of original Flows. > > For example, consider Flows A, B, and C as illustrated in Figure 4. > Assume that the key aggregation step aggregates the keys of these > three Flows to the same aggregated Flow Key, and that start interval > counter distribution is in effect. The conservative flow count for > interval 0 is 3 (since Flows A, B, and C all begin in this interval), > and for the other two intervals is 0. The non-conservative flow > count for interval 0 is also 3 (due to the presence of Flows A, B, > and C), for interval 1 is 2 (Flows B and C), and for interval 2 is 1 > (Flow C). The sum of the conservative counts 3 + 0 + 0 = 3, the > number of original Flows; while the sum of the non-conservative > counts 3 + 2 + 1 = 6. > > Note that the active and inactive timeouts used to generate original > Flows, as well as the cache policy used to generate those Flows, have > an effect on how meaningful either the conservative or non- > conservative flow count will be during aggregation. In general, all > the original Exporters producing original Flows to be aggregated > SHOULD be aggregated using caches configured identically or > similarly. Original Exporters using the IPFIX Configuration Model > SHOULD be configured to export Flows with equal or similar > activeTimeout and inactiveTimeout configuration values, and the same > cacheMode, as defined in section 4.3 of > [I-D.ietf-ipfix-configuration-model]. It's not defined there since -09 of that draft. > 5.3. Spatial Aggregation of Non-Key Fields > > Aggregation operations may also lead to the addition of value fields > demoted from key fields, or derived from other value fields in the > original Flows. Specific cases of this are treated in the > subsections below. > > 5.3.1. Counter Statistics > > Some applications of aggregation may benefit from computing different > statistics than those native to each non-key field (i.e., union for > flags, sum for counters). For example, minimum and maximum packet > counts per Flow, mean bytes per packet per aggregated Flow, and so > on. Certain Information Elements for these applications are already > provided in the IANA IPFIX Information Elements registry > (http://www.iana.org/assignments/ipfix/ipfix.html (e.g. > minimumIpTotalLength). > > A complete specification of additional aggregate counter statistics > > > > Trammell, et al. Expires December 31, 2011 [Page 16] > > Internet-Draft IPFIX Aggregation June 2011 > > > is outside the scope of this document, and should be added in the > future to the IANA IPFIX Information Elements registry on a per- > application, as-needed basis. > > 5.4. Aggregation Combination > > Interval distribution and key aggregation together may generate > multiple partially aggregated Flows covering the same time interval > with the same Flow Key. The process of combining these partially > aggregated Flows into a single Aggregated Flow is called aggregation > combination. In general, non-Key values from multiple contributing > Flows are combined using the same operation by which values are > combined from packets to form Flows for each Information Element. > Counters are summed, averages are averaged, flags are unioned, and so > on. Although this is common knowledge for us, is it actually specified anywhere in IPFIX? ie, should [IANA-IPFIX] list the relevant aggregation operation for each field? eg, min, max, sum, union, none? > > 6. Additional Considerations and Special Cases in Flow Aggregation > > 6.1. Exact versus Approximate Counting during Aggregation > > In certain circumstances, particularly involving aggregation by > devices with limited resources, and in situations where exact > aggregated counts are less important than relative magnitudes (e.g. > driving graphical displays), counter distribution during key > aggregation may be performed by approximate counting means (e.g. > Bloom filters). The choice to use approximate counting is > implementation- and application-dependent. > > 6.2. Considerations for Aggregation of Sampled Flows > > The accuracy of Aggregated Flows may also be affected by sampling of > the original Flows, or sampling of packets making up the original > Flows. The effect of sampling on flow aggregation is still an open > research question. However, to maximize the comparability of "At the time of writing, the effect of...". This is one of several points in this draft where I'd expect additional citations if it wasn't an IETF document :-) > Aggregated Flows, aggregation of sampled Flows SHOULD only use > original Flows sampled using the same sampling rate and sampling > algorithm, or Flows created from packets sampled using the same > sampling rate and sampling algorithm. For more on packet sampling I think it's also allowable to normalise flows from different samplers / rates before aggregating them. > within IPFIX, see [RFC5476]. For more on Flow sampling within the > IPFIX Mediator Framework, see [I-D.ietf-ipfix-flow-selection-tech]. > > > 7. Export of Aggregated IP Flows using IPFIX > > In general, Aggregated Flows are exported in IPFIX as any normal > Flow. However, certain aspects of Aggregated Flow export benefit > from additional guidelines, or new Information Elements to represent > > > > Trammell, et al. Expires December 31, 2011 [Page 17] > > Internet-Draft IPFIX Aggregation June 2011 > > > aggregation metadata or information generated during aggregation. > These are detailed in the following subsections. Have the new IEs been implemented in released code? We should avoid defining theoretical new IEs (ie, those which nobody has actually implemented, or plans to implement). "Rough consensus and _running code_" :-) > 7.1. Time Interval Export > > Since an Aggregated Flow is simply a Flow, the existing timestamp > Information Elements in the IPFIX Information Model (e.g., > flowStartMilliseconds, flowEndNanoseconds) are sufficient to specify > the time interval for aggregation. Therefore, this document > specifies no new aggregation-specific Information Elements for > exporting time interval information. > > Each Aggregated Flow SHOULD contain both an interval start and > interval end timestamp. If an exporter of Aggregated Flows omits the Disagree. Maybe the aggregation step is to get rid of timestamps. > interval end timestamp from each Aggregated Flow, the time interval > for Aggregated Flows within an Observation Domain and Transport > Session MUST be regular and constant. However, note that this > approach might lead to interoperability problems when exporting > Aggregated Flows to non-aggregation-aware Collecting Processes and > downstream analysis tasks; therefore, an Exporting Process capable of > exporting only interval start timestamps MUST provide a configuration > option to export interval end timestamps as well. I disagree with this time fixation. Time is only one aspect of flows, not the only one. > 7.2. Flow Count Export > > The following four Information Elements are defined to count original > Flows as discussed in Section 5.2.2. > > 7.2.1. originalFlowsPresent > > Description: The non-conservative count of original Flows > contributing to this Aggregated Flow. Non-conservative counts > need not sum to the original count on re-aggregation. > > Abstract Data Type: unsigned64 > > ElementId: TBD1 > > Status: Current > > 7.2.2. originalFlowsInitiated > > Description: The conservative count of original Flows whose first > packet is represented within this Aggregated Flow. Conservative > counts must some to the original count on re-aggregation. some -> sum > > > > > > Trammell, et al. Expires December 31, 2011 [Page 18] > > Internet-Draft IPFIX Aggregation June 2011 > > > Abstract Data Type: unsigned64 > > ElementId: TBD2 > > Status: Current > > 7.2.3. originalFlowsCompleted > > Description: The conservative count of original Flows whose last > packet is represented within this Aggregated Flow. Conservative > counts must some to the original count on re-aggregation. some -> sum > Abstract Data Type: unsigned64 > > ElementId: TBD3 > > Status: Current > > 7.2.4. originalFlows For consistency with the above IEs, name this "originalFlowsContributing" ? > Description: The conservative count of original Flows contributing > to this Aggregated Flow; may be distributed via any of the methods > described in Section 5.1.1. > > Abstract Data Type: float64 > > ElementId: 3 > > Status: Current > > 7.3. Distinct Host Export > > The following four Information Elements represent the distinct counts > of source and destination addresses for IPv4 and IPv6, used to > exporting distinct host counts reduced away during key aggregation. > > 7.3.1. distinctCountOfSourceIPv4Address > > Description: The count of distinct source IPv4 address values for > original Flows contributing to this Aggregated Flow. > > Abstract Data Type: unsigned32 > > ElementId: TBD6 > > > > > > > > Trammell, et al. Expires December 31, 2011 [Page 19] > > Internet-Draft IPFIX Aggregation June 2011 > > > Status: Current > > 7.3.2. distinctCountOfDestinationIPv4Address > > Description: The count of distinct destination IPv4 address values > for original Flows contributing to this Aggregated Flow. > > Abstract Data Type: unsigned32 > > ElementId: TBD7 > > Status: Current > > 7.3.3. distinctCountOfSourceIPv6Address > > Description: The count of distinct source IPv6 address values for > original Flows contributing to this Aggregated Flow. > > Abstract Data Type: unsigned64 > > ElementId: TBD8 > > Status: Current > > 7.3.4. distinctCountOfDestinationIPv6Address > > Description: The count of distinct destination IPv6 address values > for original Flows contributing to this Aggregated Flow. > > Abstract Data Type: unsigned64 > > ElementId: TBD9 > > Status: Current > > 7.4. Aggregate Counter Distribution Export > > When exporting counters distributed among Aggregated Flows, as > described in Section 5.1.1, the Exporting Process MAY export an > Aggregate Counter Distribution Record for each Template describing > Aggregated Flow records; this Options Template is described below. Say "options" earlier. eg, the Exporting Process MAY export an Aggregate Counter Distribution Option Record for each Template describing Aggregated Flow records. The Options Template is described below. > It uses the valueDistributionMethod Information Element, also defined > below. Since in many cases distribution is simple, accounting the > counters from contributing Flows to the first Interval to which they > contribute, this is default situation, for which no Aggregate Counter "this is _the_ default situation" > Distribution Record is necessary; Aggregate Counter Distribution > Records are only applicable in more exotic situations, such as using > an Aggregation Interval smaller than the durations of original Flows. > > > > Trammell, et al. Expires December 31, 2011 [Page 20] > > Internet-Draft IPFIX Aggregation June 2011 > > > 7.4.1. Aggregate Counter Distribution Options Template > > This Options Template defines the Aggregate Counter Distribution > Record, which allows the binding of a value distribution method to a > Template ID. This is used to signal to the Collecting Process how > the counters were distributed. The fields are as below: > > +-------------------------+-----------------------------------------+ > | IE | Description | > +-------------------------+-----------------------------------------+ > | templateId [scope] | The Template ID of the Template | > | | defining the Aggregated Flows to which | > | | this distribution option applies. This | > | | Information Element MUST be defined as | > | | a Scope Field. | > | valueDistributionMethod | The method used to distribute the | > | | counters for the Aggregated Flows | > | | defined by the associated Template. | > +-------------------------+-----------------------------------------+ Presumably the scope could be more specific, eg an "informationElementIndex" could be included. > 7.4.2. valueDistributionMethod Information Element > > Description: A description of the method used to distribute the > counters from contributing Flows into the Aggregated Flow records > described by an associated Template. The method is deemed to > apply to all the non-key Information Elements in the referenced > Template for which value distribution is a valid operation; if the No; it applies to the specified scope, which happens to be a templateId in this case - but may be something else in other uses. eg, vDM could be used with structured data, where the intention would be that it applies only to the current relevant structure, and not to all the non-key elements in some other template. > originalFlowsInitiated and/or originalFlowsCompleted Information > Elements appear in the Template, they are not subject to this > distribution method, as they each infer their own distribution > method. The distribution methods are taken from Section 5.1.1 and > encoded as follows: Please separate the table entries below to make them easier to read. > +-------+-----------------------------------------------------------+ > | Value | Description | > +-------+-----------------------------------------------------------+ What does value = zero mean? > | 1 | Start Interval: The counters for an original Flow are | > | | added to the counters of the appropriate Aggregated Flow | > | | containing the start time of the original Flow. This | > | | should be assumed the default if value distribution | > | | information is not available at a Collecting Process for | > | | an Aggregated Flow. | > | 2 | End Interval: The counters for an original Flow are added | > | | to the counters of the appropriate Aggregated Flow | > | | containing the end time of the original Flow. | > > > > > > > Trammell, et al. Expires December 31, 2011 [Page 21] > > Internet-Draft IPFIX Aggregation June 2011 > > > | 3 | Mid Interval: The counters for an original Flow are added | > | | to the counters of a single appropriate Aggregated Flow | > | | containing some timestamp between start and end time of | > | | the original Flow. | > | 4 | Simple Uniform Distribution: Each counter for an original | > | | Flow is divided by the number of time intervals the | > | | original Flow covers (i.e., of appropriate Aggregated | > | | Flows sharing the same Flow Key), and this number is | > | | added to each corresponding counter in each Aggregated | > | | Flow. | > | 5 | Proportional Uniform Distribution: Each counter for an | > | | original Flow is divided by the number of time _units_ | > | | the original Flow covers, to derive a mean count rate. | > | | This mean count rate is then multiplied by the number of | > | | time units in the intersection of the duration of the | > | | original Flow and the time interval of each Aggregated | > | | Flow. This is like simple uniform distribution, but | > | | accounts for the fractional portions of a time interval | > | | covered by an original Flow in the first and last time | > | | interval. | > | 6 | Simulated Process: Each counter of the original Flow is | > | | distributed among the intervals of the Aggregated Flows | > | | according to some function the Aggregation Process uses | > | | based upon properties of Flows presumed to be like the | > | | original Flow. This is essentially an assertion that the | > | | Aggregation Process has no direct packet timing | > | | information but is nevertheless not using one of the | > | | other simpler distribution methods. The Aggregation | > | | Process specifically makes no assertion as to the | > | | correctness of the simulation. | > | 7 | Direct: The Aggregation Process has access to the | > | | original packet timings from the packets making up the | > | | original Flow, and uses these to distribute or | > | | recalculate the counters. | > +-------+-----------------------------------------------------------+ State whether this can be extended in future, or whether a new IE would be required. From an IE-doctors point of view, this should generally be stated for all list-based IEs, since someone could extend the list and claim support for this IE, while someone else claims support with the original, more limited, list. Clearly the two "supporting" processes would not be interoperable. Expressed another way: if the list is extensible then a new implementation which extends the list invalidates all existing implementations. However, requiring new IEs for each new value may be unreasonable, leading to fragmentation (values 1-7 in IE#1, 8-9 in IE#2, 10-12 in IE#3) and unnecessarily hastening IE space exhaustion. > Abstract Data Type: unsigned8 > > ElementId: TBD5 > > Status: Current > > > 8. Examples > > In these examples, the same data, described by the same template, > will be aggregated multiple different ways; this illustrates the > various different functions which could be implemented by > > > > Trammell, et al. Expires December 31, 2011 [Page 22] > > Internet-Draft IPFIX Aggregation June 2011 > > > Intermediate Aggregation Processes. Templates are shown in iespec s/iespec/IEspec/ > format as introduced in [I-D.trammell-ipfix-ie-doctors]. The source > data format is a simplified flow: timestamps, traditional 5-tuple, > and octet count. The template is shown in Figure 7. > > flowStartMilliseconds > flowEndMilliseconds > sourceIPv4Address > destinationIPv4Address > sourceTransportPort > destinationTransportPort > protocolIdentifier > octetDeltaCount > > Figure 7: Input template for examples > > The data records given as input to the examples in this section are > shown below, in the format "flowStartMilliseconds-flowEndMilliseconds > sourceIPv4Address:sourceTransportPort -> destinationIPv4Address: > destinationTransportPort (protocolIdentifier) octetDeltaCount"; > timestamps are given in H:MM:SS.sss format. > > 9:00:00.138-9:00:00.138 192.0.2.2:47113 -> 192.0.2.131:53 (17) 119 > 9:00:03.246-9:00:03.246 192.0.2.2:22153 -> 192.0.2.131:53 (17) 83 > 9:00:00.478-9:00:03.486 192.0.2.2:52420 -> 198.51.100.2:443 (6) 1637 > 9:00:07.172-9:00:07.172 192.0.2.3:56047 -> 192.0.2.131:53 (17) 111 > 9:00:07.309-9:00:14.861 192.0.2.3:41183 -> 198.51.100.67:80 (6) 16838 > 9:00:03.556-9:00:19.876 192.0.2.2:17606 -> 198.51.100.68:80 (6) 11538 > 9:00:25.210-9:00:25.210 192.0.2.3:47113 -> 192.0.2.131:53 (17) 119 > 9:00:26.358-9:00:30.198 192.0.2.3:48458 -> 198.51.100.133:80 (6) 2973 > 9:00:29.213-9:01:00.061 192.0.2.4:61295 -> 198.51.100.2:443 (6) 8350 > 9:04:00.207-9:04:04.431 203.0.113.3:41256 -> 198.51.100.133:80 (6) 778 > 9:03:59.624-9:04:06.984 203.0.113.3:51662 -> 198.51.100.3:80 (6) 883 > 9:06:56.813-9:06:59.821 203.0.113.3:52572 -> 198.51.100.2:443 (6) 1637 > 9:06:30.565-9:07:00.261 203.0.113.3:49914 -> 197.51.100.133:80 (6) 561 > 9:06:55.160-9:07:05.208 192.0.2.2:50824 -> 198.51.100.2:443 (6) 1899 > 9:06:49.322-9:07:05.322 192.0.2.3:34597 -> 198.51.100.3:80 (6) 1284 > 9:07:05.849-9:07:09.625 203.0.113.3:58907 -> 198.51.100.4:80 (6) 2670 > 9:10:45.161-9:10:45.161 192.0.2.4:22478 -> 192.0.2.131:53 (17) 75 > 9:10:45.209-9:11:01.465 192.0.2.4:49513 -> 198.51.100.68:80 (6) 3374 > 9:10:57.094-9:11:00.614 192.0.2.4:64832 -> 198.51.100.67:80 (6) 138 > 9:10:59.770-9:11:02.842 192.0.2.3:60833 -> 198.51.100.69:443 (6) 2325 > 9:13:53.933-9:14:06.605 192.0.2.2:19638 -> 198.51.100.3:80 (6) 2869 > 9:13:02.864-9:14:08.720 192.0.2.3:40429 -> 198.51.100.4:80 (6) 18289 > > Figure 8: Input data for examples Consider listing the values in aligned columns without the syntax? It fits into 76 chars with double-spaced columns, and is slightly easier to read. Also, good to add column titles to make the subsequent figures easier to follow. Finally, the data are a little out of order. Although it really doesn't matter, in-order might make the example a little easier to follow for pedants who really want to check your data manually, like me. > > > > > Trammell, et al. Expires December 31, 2011 [Page 23] > > Internet-Draft IPFIX Aggregation June 2011 > > > 8.1. Traffic Time-Series per Source > > Aggregating flows by source IP address in time series (i.e., with a > regular interval) can be used in subsequent heavy-hitter analysis and > as a source parameter for statistical anomaly detection techniques. > Here, the Intermediate Aggregation Process imposes an interval, > aggregates the key to remove all key fields other than the source IP > address, then combines the result into a stream of Aggregated Flows. > For simplicity, the imposed interval of 30 minutes is defined to be > larger than the maximum active timeout of the original Flows; counter > distribution will be added to this example below in Section 8.4. "to this example in Section 8.4 below." > In this example the partially aggregated Flows after each conceptual > operation in the Intermediate Aggregation Process are shown. These > are meant to be illustrative of the conceptual operations only, and > not to suggest an implementation (indeed, the example shown here > would not necessarily be the most efficient method for performing > these operations). Subsequent examples will omit the partially > aggregated Flows for brevity. > > The input to this process could be any Flow Record containing a > source IP address and octet counter; consider for this example the > template and data from the introduction. The Intermediate > Aggregation Process would then output records containing just > timestamps, source IP, and octetDeltaCount, as in Figure 9. > > flowStartMilliseconds > flowEndMilliseconds > sourceIPv4Address > octetDeltaCount > > Figure 9: Output template for time series per source > > Assume the goal is to get 5-minute time series of octet counts per For consistency, either say "300s" here or say "5-minutes" in the figure below. > source IP address. The aggregation operations would then be arranged > as in Figure 10. > > > > > > > > > > > > > > > > Trammell, et al. Expires December 31, 2011 [Page 24] > > Internet-Draft IPFIX Aggregation June 2011 > > > original Flows > | > V > +-----------------------+ > | interval distribution | > | * impose uniform | > | 300s time interval | > +-----------------------+ > | > | partially aggregated Flows > V > +------------------------+ > | key aggregation | > | * reduce key to only | > | sourceIPv4Address | > +------------------------+ > | > | partially aggregated Flows > V > +-------------------------+ > | aggregate combination | > | * sum octetDeltaCount | > +-------------------------+ > | > V > Aggregated Flows > > partially aggregated Flows > > Figure 10: Aggregation operations for time series per source > > After the interval distribution step, only the time intervals have > changed; the partially aggregated Flows are shown in Figure 11. Is the interval distribution being calculated on the flow start or flow end timestamp? Although the carefully chosen data give the same result either way, it should be stated. > > > > > > > > > > > > > > > > > > Trammell, et al. Expires December 31, 2011 [Page 25] > > Internet-Draft IPFIX Aggregation June 2011 > > > 9:00:00.000-9:05:00.000 192.0.2.2:47113 -> 192.0.2.131:53 (17) 119 > 9:00:00.000-9:05:00.000 192.0.2.2:22153 -> 192.0.2.131:53 (17) 83 > 9:00:00.000-9:05:00.000 192.0.2.2:52420 -> 198.51.100.2:443 (6) 1637 > 9:00:00.000-9:05:00.000 192.0.2.3:56047 -> 192.0.2.131:53 (17) 111 > 9:00:00.000-9:05:00.000 192.0.2.3:41183 -> 198.51.100.67:80 (6) 16838 > 9:00:00.000-9:05:00.000 192.0.2.2:17606 -> 198.51.100.68:80 (6) 11538 > 9:00:00.000-9:05:00.000 192.0.2.3:47113 -> 192.0.2.131:53 (17) 119 > 9:00:00.000-9:05:00.000 192.0.2.3:48458 -> 198.51.100.133:80 (6) 2973 > 9:00:00.000-9:05:00.000 192.0.2.4:61295 -> 198.51.100.2:443 (6) 8350 > 9:00:00.000-9:05:00.000 203.0.113.3:41256 -> 198.51.100.133:80 (6) 778 > 9:00:00.000-9:05:00.000 203.0.113.3:51662 -> 198.51.100.3:80 (6) 883 > 9:05:00.000-9:10:00.000 203.0.113.3:52572 -> 198.51.100.2:443 (6) 1637 > 9:05:00.000-9:10:00.000 203.0.113.3:49914 -> 197.51.100.133:80 (6) 561 > 9:05:00.000-9:10:00.000 192.0.2.2:50824 -> 198.51.100.2:443 (6) 1899 > 9:05:00.000-9:10:00.000 192.0.2.3:34597 -> 198.51.100.3:80 (6) 1284 > 9:05:00.000-9:10:00.000 203.0.113.3:58907 -> 198.51.100.4:80 (6) 2670 > 9:10:00.000-9:15:00.000 192.0.2.4:22478 -> 192.0.2.131:53 (17) 75 > 9:10:00.000-9:15:00.000 192.0.2.4:49513 -> 198.51.100.68:80 (6) 3374 > 9:10:00.000-9:15:00.000 192.0.2.4:64832 -> 198.51.100.67:80 (6) 138 > 9:10:00.000-9:15:00.000 192.0.2.3:60833 -> 198.51.100.69:443 (6) 2325 > 9:10:00.000-9:15:00.000 192.0.2.2:19638 -> 198.51.100.3:80 (6) 2869 > 9:10:00.000-9:15:00.000 192.0.2.3:40429 -> 198.51.100.4:80 (6) 18289 > > Figure 11: Partially aggregated Flows: intervals imposed Why two timestamps? It's unclear which interval a flow at 9:05:00.000 should be in. Whereas listing a single timestamp (9:00, 9:05, 9:10) is less ambiguous. > After the key aggregation step, all the parts of the flow key except > the source IP address have been discarded, as shown in Figure 12. > This leaves duplicate partially aggregated Flows to be combined the > final operation. > > > > > > > > > > > > > > > > > > > > > > > Trammell, et al. Expires December 31, 2011 [Page 26] > > Internet-Draft IPFIX Aggregation June 2011 > > > 9:00:00.000-9:05:00.000 192.0.2.2 119 > 9:00:00.000-9:05:00.000 192.0.2.2 83 > 9:00:00.000-9:05:00.000 192.0.2.2 1637 > 9:00:00.000-9:05:00.000 192.0.2.3 111 > 9:00:00.000-9:05:00.000 192.0.2.3 16838 > 9:00:00.000-9:05:00.000 192.0.2.2 11538 > 9:00:00.000-9:05:00.000 192.0.2.3 119 > 9:00:00.000-9:05:00.000 192.0.2.3 2973 > 9:00:00.000-9:05:00.000 192.0.2.4 8350 > 9:00:00.000-9:05:00.000 203.0.113.3 778 > 9:00:00.000-9:05:00.000 203.0.113.3 883 > 9:05:00.000-9:10:00.000 203.0.113.3 1637 > 9:05:00.000-9:10:00.000 203.0.113.3 561 > 9:05:00.000-9:10:00.000 192.0.2.2 1899 > 9:05:00.000-9:10:00.000 192.0.2.3 1284 > 9:05:00.000-9:10:00.000 203.0.113.3 2670 > 9:10:00.000-9:15:00.000 192.0.2.4 75 > 9:10:00.000-9:15:00.000 192.0.2.4 3374 > 9:10:00.000-9:15:00.000 192.0.2.4 138 > 9:10:00.000-9:15:00.000 192.0.2.3 2325 > 9:10:00.000-9:15:00.000 192.0.2.2 2869 > 9:10:00.000-9:15:00.000 192.0.2.3 18289 > > Figure 12: Partially aggregated Flows: key aggregation > > Aggregate combination sums the counters per key and interval; the > summations of the first two keys and intervals are shown in detail in > Figure 13. > > 9:00:00.000-9:05:00.000 192.0.2.2 119 > 9:00:00.000-9:05:00.000 192.0.2.2 83 > 9:00:00.000-9:05:00.000 192.0.2.2 1637 > + 9:00:00.000-9:05:00.000 192.0.2.2 11538 > ----- > = 9:00:00.000-9:05:00.000 192.0.2.2 13377 > > 9:00:00.000-9:05:00.000 192.0.2.3 111 > 9:00:00.000-9:05:00.000 192.0.2.3 16838 > 9:00:00.000-9:05:00.000 192.0.2.3 119 > + 9:00:00.000-9:05:00.000 192.0.2.3 2973 > ----- > = 9:00:00.000-9:05:00.000 192.0.2.3 20041 > > Figure 13: Summation during aggregate combination > > Applying this to each set of partially aggregated Flows to produce > the final Aggregated Flows shown in Figure 14m to be exported by the > template in Figure 9. > > > > Trammell, et al. Expires December 31, 2011 [Page 27] > > Internet-Draft IPFIX Aggregation June 2011 > > > 9:00:00.000-9:05:00.000 192.0.2.2 13377 > 9:00:00.000-9:05:00.000 192.0.2.3 20041 > 9:00:00.000-9:05:00.000 192.0.2.4 8350 > 9:00:00.000-9:05:00.000 203.0.113.3 1661 > 9:05:00.000-9:10:00.000 192.0.2.2 1899 > 9:05:00.000-9:10:00.000 192.0.2.3 1284 > 9:05:00.000-9:10:00.000 203.0.113.3 4868 > 9:10:00.000-9:15:00.000 192.0.2.2 2869 > 9:10:00.000-9:15:00.000 192.0.2.3 20594 20594 should be 2325 + 18289 = 20614. > 9:10:00.000-9:15:00.000 192.0.2.4 3587 > > Figure 14: Aggregated Flows As a cross-check, sum(Figure_8) = sum(Figure_11) = 78550, while sum(Figure_14) = 78530 - so there are 20 octets missing. > 8.2. Core Traffic Matrix > > Aggregating flows by source and destination autonomous system number > in time series is used to generate core traffic matrices. The core > traffic matrix provides a view of the state of the routes within a > network, and can be used for long-term planning of changes to network > design based on traffic demand. Here, imposed time intervals are > generally much longer than active flow timeouts. The traffic matrix > is reported in terms of octets, packets, and flows, as each of these > values may have a subtly different effect on capacity planning. > > This example demonstrates key aggregation using derived keys and > Original Flow counting. While some original Flows may be generated > by Exporting Processes on forwarding devices, and therefore contain > the bgpSourceAsNumber and bgpDestinationAsNumber Information > Elements, original Flows from Exporting Processes on dedicated > measurement devices will contain only a destinationIPv[46]Address. > For these flows, the Mediator must look up a next hop AS from a IP to > AS table, replacing source and destination addresses with AS numbers. > > [TODO: complete example. show AS map, output templates, and > processing in IAP.] TODO. > 8.3. Distinct Source Count per Destination Endpoint > > Aggregating flows by destination address and port, and counting > distinct sources aggregated away, can be used as part of passive > service inventory and host characterization approaches. This example > shows aggregation as an analysis technique, performed on source data > stored in an IPFIX File. As the Transport Session in this File is > bounded, removal of all timestamp information allows summarization of > the entire time interval contained within the interval. Removal of > timing information during interval imposition is equivalent to an > infinitely long imposed time interval. This demonstrates both how > infinite intervals work, and how unique counters work. At last, an example which shows that time is _not_ the principal factor in every aggregation - and is not even required. Since there's no temporal aggregation here, there's no interval distribution - which contradicts figure 3. > > > Trammell, et al. Expires December 31, 2011 [Page 28] > > Internet-Draft IPFIX Aggregation June 2011 > > > [TODO: complete example. show output templates and processing in > IAP.] TODO. > 8.4. Traffic Time-Series per Source with Counter Distribution > > Returning to the example in Section 8.1, consider a case where > aggregation by the maximum active timeout, here 30 minutes, is > incompatible with the processing interval, here defined to be 5 > minutes. For this case, flows longer than 5 minutes must have their > counters distributed. This example demonstrates counter distribution > metadata export. > > [TODO: complete example. show output metadata and processing in IAP.] TODO. > > 9. Security Considerations > > [TODO] TODO. > > 10. IANA Considerations > > This document specifies the creation of twelve new IPFIX Information > Elements in the IPFIX Information Element registry located at > http://www.iana.org/assignments/ipfix, as defined in Section 7 above. > IANA has assigned Information Element numbers to these Information > Elements, and entered them into the registry. > > [NOTE for IANA: The text TBDn should be replaced with the respective > assigned Information Element numbers where they appear in this > document. Note that the originalFlows Information Element has been > assigned the number 3, as it is compatible with the corresponding > existing (reserved) NetFlow v9 Information Element. Other > Information Element numbers should be assigned outside the NetFlow V9 > compatibility range, as these Information Elements are not supported > by NetFlow V9.] > > > 11. Acknowledgments > > This work is materially supported by the European Union Seventh > Framework Programme under grant agreement 257315 (DEMONS). > > > 12. References > > > > > > > Trammell, et al. Expires December 31, 2011 [Page 29] > > Internet-Draft IPFIX Aggregation June 2011 > > > 12.1. Normative References > > [RFC5101] Claise, B., "Specification of the IP Flow Information > Export (IPFIX) Protocol for the Exchange of IP Traffic > Flow Information", RFC 5101, January 2008. > > [RFC5102] Quittek, J., Bryant, S., Claise, B., Aitken, P., and J. > Meyer, "Information Model for IP Flow Information Export", > RFC 5102, January 2008. > > 12.2. Informative References > > [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate > Requirement Levels", BCP 14, RFC 2119, March 1997. > > [RFC3917] Quittek, J., Zseby, T., Claise, B., and S. Zander, > "Requirements for IP Flow Information Export (IPFIX)", > RFC 3917, October 2004. > > [RFC5103] Trammell, B. and E. Boschi, "Bidirectional Flow Export > Using IP Flow Information Export (IPFIX)", RFC 5103, > January 2008. > > [RFC5153] Boschi, E., Mark, L., Quittek, J., Stiemerling, M., and P. > Aitken, "IP Flow Information Export (IPFIX) Implementation > Guidelines", RFC 5153, April 2008. > > [RFC5470] Sadasivan, G., Brownlee, N., Claise, B., and J. Quittek, > "Architecture for IP Flow Information Export", RFC 5470, > March 2009. > > [RFC5472] Zseby, T., Boschi, E., Brownlee, N., and B. Claise, "IP > Flow Information Export (IPFIX) Applicability", RFC 5472, > March 2009. > > [RFC5476] Claise, B., Johnson, A., and J. Quittek, "Packet Sampling > (PSAMP) Protocol Specifications", RFC 5476, March 2009. > > [RFC5610] Boschi, E., Trammell, B., Mark, L., and T. Zseby, > "Exporting Type Information for IP Flow Information Export > (IPFIX) Information Elements", RFC 5610, July 2009. > > [RFC5655] Trammell, B., Boschi, E., Mark, L., Zseby, T., and A. > Wagner, "Specification of the IP Flow Information Export > (IPFIX) File Format", RFC 5655, October 2009. > > [RFC5835] Morton, A. and S. Van den Berghe, "Framework for Metric > Composition", RFC 5835, April 2010. > > > > Trammell, et al. Expires December 31, 2011 [Page 30] > > Internet-Draft IPFIX Aggregation June 2011 > > > [RFC5982] Kobayashi, A. and B. Claise, "IP Flow Information Export > (IPFIX) Mediation: Problem Statement", RFC 5982, > August 2010. > > [I-D.ietf-ipfix-anon] > Boschi, E. and B. Trammell, "IP Flow Anonymization > Support", draft-ietf-ipfix-anon-06 (work in progress), > January 2011. > > [I-D.ietf-ipfix-mediators-framework] > Kobayashi, A., Claise, B., Muenz, G., and K. Ishibashi, > "IPFIX Mediation: Framework", > draft-ietf-ipfix-mediators-framework-09 (work in > progress), October 2010. > > [I-D.claise-ipfix-mediation-protocol] > Claise, B., "Specification of the Protocol for IPFIX > Mediations", draft-claise-ipfix-mediation-protocol-03 > (work in progress), February 2011. > > [I-D.trammell-ipfix-ie-doctors] > Trammell, B. and B. Claise, "Guidelines for Authors and > Reviewers of IPFIX Information Elements", > draft-trammell-ipfix-ie-doctors-02 (work in progress), > June 2011. > > [I-D.ietf-ipfix-configuration-model] > Muenz, G., Claise, B., and P. Aitken, "Configuration Data > Model for IPFIX and PSAMP", > draft-ietf-ipfix-configuration-model-09 (work in > progress), March 2011. > > [I-D.ietf-ipfix-flow-selection-tech] > D'Antonio, S., Zseby, T., Henke, C., and L. Peluso, "Flow > Selection Techniques", > draft-ietf-ipfix-flow-selection-tech-06 (work in > progress), May 2011. > > > > > > > > > > > > > > > Trammell, et al. Expires December 31, 2011 [Page 31] > > Internet-Draft IPFIX Aggregation June 2011 > > > Authors' Addresses > > Brian Trammell > Swiss Federal Institute of Technology Zurich > Gloriastrasse 35 > 8092 Zurich > Switzerland > > Phone: +41 44 632 70 13 > Email:trammell@tik.ee.ethz.ch > > > Elisa Boschi > Swiss Federal Institute of Technology Zurich > Gloriastrasse 35 > 8092 Zurich > Switzerland > > Email:boschie@tik.ee.ethz.ch > > > Arno Wagner > Consecom AG > Bleicherweg 64a > 8002 Zurich > Switzerland > > Email:arno@wagner.name > > > Benoit Claise > Cisco Systems, Inc. > De Kleetlaan 6a b1 > 1831 Diagem In existing RFCs, this is "Diegem 1813". > Belgium > > Phone: +32 2 704 5622 > Email:bclaise@cisco.com > > > > > > > > > > > > > > Trammell, et al. Expires December 31, 2011 [Page 32] > --------------020106010107040308030706 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Dear Authors,

At long last, here's a review of draft-trammell-ipfix-a9n-03.

Thanks for your great patience!
P.


IPFIX Working Group                                          B. Trammell
Internet-Draft                                                 E. Boschi
Intended status: Standards Track                              ETH Zurich
Expires: December 31, 2011                                     A. Wagner
                                                             Consecom AG
                                                               B. Claise
                                                     Cisco Systems, Inc.
                                                           June 29, 2011


  Exporting Aggregated Flow Data using the IP Flow Information Export
                            (IPFIX) Protocol
                    draft-trammell-ipfix-a9n-03.txt

Abstract

   This document describes the export of aggregated Flow information
   using IPFIX.  An Aggregated Flow is essentially an IPFIX Flow
   representing packets from multiple original Flows sharing some set of
   common properties.  The document describes Aggregated Flow export
   within the framework of IPFIX Mediators and defines an interoperable,
   implementation-independent method for Aggregated Flow export.

Status of this Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at http://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on December 31, 2011.

Copyright Notice

   Copyright (c) 2011 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of



Trammell, et al.        Expires December 31, 2011               [Page 1]

Internet-Draft              IPFIX Aggregation                  June 2011


   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.


Table of Contents

   1.  Introduction . . . . . . . . . . . . . . . . . . . . . . . . .  4
   2.  Terminology  . . . . . . . . . . . . . . . . . . . . . . . . .  5
   3.  Use Cases for IPFIX Aggregation  . . . . . . . . . . . . . . .  6
   4.  Architecture for Flow Aggregation  . . . . . . . . . . . . . .  6
     4.1.  Aggregation within the IPFIX Architecture  . . . . . . . .  7
     4.2.  Intermediate Aggregation Process Architecture  . . . . . .  8
   5.  IP Flow Aggregation Operations . . . . . . . . . . . . . . . . 10
     5.1.  Temporal Aggregation through Interval Distribution . . . . 10
       5.1.1.  Distributing Values Across Intervals . . . . . . . . . 11
       5.1.2.  Time Composition . . . . . . . . . . . . . . . . . . . 13
     5.2.  Spatial Aggregation of Flow Keys . . . . . . . . . . . . . 13
       5.2.1.  Counting Distinct Key Values . . . . . . . . . . . . . 15
       5.2.2.  Counting Original Flows  . . . . . . . . . . . . . . . 15
     5.3.  Spatial Aggregation of Non-Key Fields  . . . . . . . . . . 16
       5.3.1.  Counter Statistics . . . . . . . . . . . . . . . . . . 16
     5.4.  Aggregation Combination  . . . . . . . . . . . . . . . . . 17
   6.  Additional Considerations and Special Cases in Flow
       Aggregation  . . . . . . . . . . . . . . . . . . . . . . . . . 17
     6.1.  Exact versus Approximate Counting during Aggregation . . . 17
     6.2.  Considerations for Aggregation of Sampled Flows  . . . . . 17
   7.  Export of Aggregated IP Flows using IPFIX  . . . . . . . . . . 17
     7.1.  Time Interval Export . . . . . . . . . . . . . . . . . . . 18
     7.2.  Flow Count Export  . . . . . . . . . . . . . . . . . . . . 18
       7.2.1.  originalFlowsPresent . . . . . . . . . . . . . . . . . 18
       7.2.2.  originalFlowsInitiated . . . . . . . . . . . . . . . . 18
       7.2.3.  originalFlowsCompleted . . . . . . . . . . . . . . . . 19
       7.2.4.  originalFlows  . . . . . . . . . . . . . . . . . . . . 19
     7.3.  Distinct Host Export . . . . . . . . . . . . . . . . . . . 19
       7.3.1.  distinctCountOfSourceIPv4Address . . . . . . . . . . . 19
       7.3.2.  distinctCountOfDestinationIPv4Address  . . . . . . . . 20
       7.3.3.  distinctCountOfSourceIPv6Address . . . . . . . . . . . 20
       7.3.4.  distinctCountOfDestinationIPv6Address  . . . . . . . . 20
     7.4.  Aggregate Counter Distribution Export  . . . . . . . . . . 20
       7.4.1.  Aggregate Counter Distribution Options Template  . . . 21
       7.4.2.  valueDistributionMethod Information Element  . . . . . 21
   8.  Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
     8.1.  Traffic Time-Series per Source . . . . . . . . . . . . . . 24
     8.2.  Core Traffic Matrix  . . . . . . . . . . . . . . . . . . . 28



Trammell, et al.        Expires December 31, 2011               [Page 2]

Internet-Draft              IPFIX Aggregation                  June 2011


     8.3.  Distinct Source Count per Destination Endpoint . . . . . . 28
     8.4.  Traffic Time-Series per Source with Counter
           Distribution . . . . . . . . . . . . . . . . . . . . . . . 29
   9.  Security Considerations  . . . . . . . . . . . . . . . . . . . 29
   10. IANA Considerations  . . . . . . . . . . . . . . . . . . . . . 29
   11. Acknowledgments  . . . . . . . . . . . . . . . . . . . . . . . 29
   12. References . . . . . . . . . . . . . . . . . . . . . . . . . . 30
     12.1. Normative References . . . . . . . . . . . . . . . . . . . 30
     12.2. Informative References . . . . . . . . . . . . . . . . . . 30
   Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 31









































Trammell, et al.        Expires December 31, 2011               [Page 3]

Internet-Draft              IPFIX Aggregation                  June 2011


1.  Introduction

   The aggregation of packet data into Flows serves a variety of
   different purposes, as noted in the requirements [RFC3917] and
   applicability statement [RFC5472] for the IP Flow Information Export
   (IPFIX) protocol [RFC5101].  Aggregation beyond the flow level, into
   records representing multiple Flows, is a common analysis and data
   reduction technique as well, with applicability to large-scale
   network data analysis, archiving, and inter-organization exchange.
   This applicability in large-scale situations, in particular, led to
   the inclusion of aggregation as part of the IPFIX Mediators Problem
   Statement [RFC5982], and the definition of an Intermediate
   Aggregation Process in the Mediator framework
   [I-D.ietf-ipfix-mediators-framework].

   Aggregation is part of a wide variety of applications, including
   traffic matrix calculation, generation of time series data for
   visualizations or anomaly detection, or measurement data reduction.

There seems to be a missing point here.
ie, "Aggregation is part of a wide variety of applications"... used for a specific purpose?


   Depending on the keys used for aggregation, it may additionally have
   an anonymising affect on the data: for example, aggregation
   operations which eliminate IP addresses make it impossible to later
   identify nodes using those addresses.

There could be enough other information. eg, destination port = 80 could identify a web server.
However, "may have an anonymising effect" seems sufficient for the present purpose.


   Aggregation as defined and described in this document covers the
   applications defined in [RFC5982], including 5.1 "Adjusting Flow
   Granularity", 5.4 "Time Composition", and 5.5 "Spatial Composition".
   However, this document specifies a more flexible architecture for an
   Intermediate Aggregation Process in Section 4.2, which supports a
   superset of these applications.

So this document isn't just about export as the title and abstract suggest.

Perhaps the title needs to change to "Aggregating and Exporting Flow Data using IPFIX" ?
The abstract should indicate that the document defines the IAP.


   An Intermediate Aggregation Process may be applied to data collected
   from multiple Observation Points, as aggregation is natural to apply

"aggregation is natural to apply" -> "it's natural to apply aggregation".


   for data reduction when concentrating measurement data.  This
   document specifically does not address the protocol issues that arise
   when combining IPFIX data from multiple Observation Points and
   exporting from a single Mediator, as these issues are general to
   IPFIX Mediation; they are therefore treated in detail in the Mediator

Mediator -> Mediation

- or, as that text actually says, "Mediations" :-(


   Protocol [I-D.claise-ipfix-mediation-protocol] document.

   Since Aggregated Flows as defined in the following section are
   essentially Flows, the IPFIX protocol [RFC5101] can be used to
   export, and the IPFIX File Format [RFC5655] can be used to store,
   aggregated data "as-is"; there are no changes necessary to the
   protocol.  This document provides a common basis for the application
   of IPFIX to the handling of aggregated data, through a detailed
   terminology, Intermediate Aggregation Process architecture, and
   methods for original Flow counting and counter distribution across
   intervals.

That last line sounds like something for the abstract.




Trammell, et al.        Expires December 31, 2011               [Page 4]

Internet-Draft              IPFIX Aggregation                  June 2011


2.  Terminology

   Terms used in this document that are defined in the Terminology
   section of the IPFIX Protocol [RFC5101] document are to be
   interpreted as defined there.

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in [RFC2119].

   In addition, this document defines the following terms

   Aggregated Flow:   A Flow, as defined by [RFC5101], derived from a
      set of zero or more original Flows within a defined Aggregation
      Interval.  The two primary differences between a Flow and an
      Aggregated Flow are (1) that the time interval of a Flow is
      generally derived from information about the timing of the packets
      comprising the Flow, while the time interval of an Aggregated Flow
      are generally externally imposed; and (2) that an Aggregated Flow
      may represent zero packets (i.e., an assertion that no packets
      were seen for a given Flow Key in a given time interval).  Note

That's an oversight in 5101. We should have allowed a Flow to represent zero or more packets.
It shouldn't be necessary to define an aggregated flow in order to export that nothing was seen.

This should be fixed in 5101bis, and (2) removed above.


      that an Aggregated Flow is defined within the context of an
      Intermediate Aggregation Process only. once an Aggregated Flow is
      exported, it is essentially a Flow as in [RFC5101] and can be
      treated as such.

   Intermediate Aggregation Function:   A mapping from a set of zero or
      more original Flows into a set of Aggregated Flows across one or
      more Aggregation Intervals.  This function is hosted by an

This definition only allows for time based aggregation, and not for spatial aggregation.

ie, this allows for "in interval T1 and T2, I saw IP addresses A1, A2, A3, and A4." (ie, addresses are aggregated).

Suppose we want to do it the other way, ie: I saw IP address w.x.y.z between T1 and T2, not between T2 and T3, then again between T3 and T4. (ie, time is aggregated).

Also see example 8.3, which has no time interval.


      Intermediate Aggregation Process, defined below.

   Intermediate Aggregation Process:   an Intermediate Process as in
      [I-D.ietf-ipfix-mediators-framework] that aggregates records based
      upon a set of Flow Keys or functions applied to fields from the
      record; this is itself defined in
      [I-D.ietf-ipfix-mediators-framework].

   Aggregation Interval:   A time interval imposed upon an Aggregated
      Flow.  Aggregation Functions may use a regular Aggregation
      Interval (e.g. "every five minutes", "every calendar month"),
      though regularity is not necessary.  Aggregation intervals may
      also be derived from the time intervals of the original Flows
      being aggregated.

   partially aggregated Flow:   A Flow during processing within an
      Intermediate Aggregation Process; refers to an intermediate data
      structure during aggregation within the Intermediate Aggregation
      Process architecture detailed in Section 4.2.

Are "partially aggregated Flows" really necessary?




Trammell, et al.        Expires December 31, 2011               [Page 5]

Internet-Draft              IPFIX Aggregation                  June 2011


   original Flow:   A Flow given as input to an Aggregation Function in
      order to generate Aggregated Flows.

   contributing Flow:   An original Flow that is partially or completely
      represented within an Aggregated Flow.  Each aggregated Flow is
      made up of zero or more contributing Flows, and an original Flow
      may contribute to zero or more Aggregated Flows.

A "contributing Flow" seems to be little more than an original Flow. Certainly there's no such thing as a "non-contributing Flow". (There could be, but it'd be irrelevant.) So is the distinction useful? Can't you just call these "original Flows" ?



3.  Use Cases for IPFIX Aggregation

   Aggregation, as a common data analysis method, has many applications.
   When used with a regular Aggregation Interval, it generates time
   series data from a collection of Flows with discrete intervals.  Time
   series data is itself useful for a wide variety of analysis tasks,
   such as generating input for network anomaly detection systems, or
   driving visualizations of volume per time for traffic with specific

Fixated on time here. See earlier comments.

Specifically, aggregation does not "generates time series data". It depends what is aggregated.

Example 8.3 is the only place which acknowledges that time is not required. Yet time isn't special, it's just a field like any other.


   characteristics.  Traffic matrix calculation from flow data is
   inherently an aggregation action, by aggregating the Flow Key down to
   input or output interface, address prefix, or autonomous system.

   Irregular or data-dependent Aggregation Intervals and key aggregation
   operations can also be used to provide adaptive aggregation of
   network flow data.  Here, full Flow Records can be kept for Flows of
   interest, while Flows deemed "less interesting" to a given
   application can be aggregated.  For example, in an IPFIX Mediator
   equipped with traffic classification capabilities for security
   purposes, potentially malicious Flows could be exported directly,
   while known-good or probably-good Flows (e.g. normal web browsing)
   could be exported simply as time series volumes per web server.

   Note that an Intermediate Aggregation Function which removes
   potentially sensitive information as identified in
   [I-D.ietf-ipfix-anon] may tend to have an anonymising effect on the
   Aggregated Flows, as well; however, any application of aggregation as
   part of a data protection scheme should ensure that all the issues
   raised in Section 4 of [I-D.ietf-ipfix-anon] are addressed.


4.  Architecture for Flow Aggregation

   This section specifies how an Intermediate Aggregation Process fits
   into the IPFIX Architecture, and the architecture of the Intermediate
   Aggregation Process itself.

That's ambiguous (ie, "This section specifies how an Intermediate Aggregation Process fits into ... the architecture of the Intermediate Aggregation Process itself.").

Better to say,
   This section specifies the architecture of the Intermediate Aggregation Process,
   and how an Intermediate Aggregation Process fits into the IPFIX Architecture.







Trammell, et al.        Expires December 31, 2011               [Page 6]

Internet-Draft              IPFIX Aggregation                  June 2011


4.1.  Aggregation within the IPFIX Architecture

   An Intermediate Aggregation Process may be deployed at three places
   within the IPFIX Architecture.  While aggregation applications are
   most commonly deployed within a Mediator which collects original
   Flows from an original Exporter and exports Aggregated Flows,
   aggregation can also occur before initial export, or after final
   collection, as shown in Figure 1.

     +==========================================+
     | Exporting Process                        |
     +==========================================+
       |                                      |
       |             (Aggregated Flow Export) |
       V                                      |
     +=============================+          |
     | Mediator                    |          |
     +=============================+          |
       |                                      |
       | (Aggregating Mediator)               |

"Aggregating Mediator" describes the process, so it should be in the box.


       V                                      V
     +==========================================+
     | Collecting Process                       |
     +==========================================+
             |
             | (Aggregation for Storage)

Again, the label is a process which should either be shown in the box or in a subsequent box.


             V
     +--------------------+
     | IPFIX File Storage |
     +--------------------+

                 Figure 1: Potential Aggregation Locations

   The Mediator use case is further shown in Figures A and B in
   [I-D.ietf-ipfix-mediators-framework].

   Aggregation can be applied for either intermediate or final analytic
   purposes.  In certain circumstances, it may make sense to export
   Aggregated Flows directly from an original Exporting Process, for
   example, if the Exporting Process is applied to drive a time-series
   visualization, or when flow data export bandwidth is restricted and
   flow or packet sampling is not an option.  Note that this case, where
   the Aggregation Process is essentially integrated into the Metering
   Process, is essentially covered by the IPFIX architecture [RFC5470]:
   the Flow Keys used are simply a subset of those that would normally
   be used.  A Metering Process in this arrangement MAY choose to
   simulate the generation of larger Flows in order to generate original
   Flow counts, if the application calls for compatibility with an



Trammell, et al.        Expires December 31, 2011               [Page 7]

Internet-Draft              IPFIX Aggregation                  June 2011


   Aggregation Process deployed in a separate location.

   In the specific case that an Aggregation Process is employed for data
   reduction for storage purposes, it can take original Flows from a
   Collecting Process or File Reader and pass Aggregated Flows to a File
   Writer for storage.

   Deployment of an Intermediate Aggregation Process within a Mediator
   [RFC5982] is a much more flexible arrangement.  Here, the Mediator
   consumes original Flows and produces aggregated Flows; this
   arrangement is suited to any of the use cases detailed in Section 3.
   In a mediator, aggregation can be applied as well to aggregating

"can be applied as well" -> "can also be applied"


   original Flows from multiple sources into a single stream of
   aggregated Flows; the architectural specifics of this arrangement are
   not addressed in this document, which is concerned only with the
   aggregation operation itself; see
   [I-D.claise-ipfix-mediation-protocol] for details.

   The data paths into and out of an Intermediate Aggregation Process
   are showin in Figure 2.

s/showin/shown/


     packets --+                     +- IPFIX Messages -+
               |                     |                  |
               V                     V                  V
     +==================+ +====================+ +=============+
     | Metering Process | | Collecting Process | | File Reader |
     |                  | +====================+ +=============+
     |                  |            |  original Flows  |
     |                  |            V                  V
     + - - - - - - - - -+======================================+
     |           Intermediate Aggregation Process (IAP)        |
     +=========================================================+
               | Aggregated                  Aggregated |
               | Flows                            Flows |
               V                                        V
     +===================+                       +=============+
     | Exporting Process |                       | File Writer |
     +===================+                       +=============+
               |                                        |
               +------------> IPFIX Messages <----------+

           Figure 2: Data paths through the aggregation process

An IPFIX File Reader consumes IPFIX Files, not IPFIX Messages.
An IPFIX File Write produces IPFIX Files, not IPFIX Messages.

Surely the process for (re)generating IPFIX messages is:

    IPFIX Messages -> File Writer -> IPFIX File -> File Reader -> Exporting Process -> IPFIX Messages 


4.2.  Intermediate Aggregation Process Architecture

   Within this document, an Intermediate Aggregation Process can be seen
   as hosting an Intermediate Aggregation Function composed of four
   types of operations on the intermediate results of aggregation, which



Trammell, et al.        Expires December 31, 2011               [Page 8]

Internet-Draft              IPFIX Aggregation                  June 2011


   are called partially aggregated Flows in this document, as
   illustrated in Figure 3.

                    original Flows

Note that here you say "original Flows" rather than contributing Flows. See my earlier comment.


                          |
                          V
              +-----------------------+
              | interval distribution |
          +-->|      (temporal)       |<--+
          |   +-----------------------+   |
          |       |       |       |       |
          |(*)    |(*)    |(*)    |(*)    |(*)
          |       |       |       |       |
          |       V       |       V       |
   +-------------------+  |  +--------------------+
   |  key aggregation  |  |  |  value aggregation |
   |     (spatial)     |  |  |      (spatial)     |
   +-------------------+  |  +--------------------+
          ^       |       |       |       ^
          |       |(*)    |       |(*)    |
          +-------|-------|-------|-------+
                  V       V       V

The top half of the figure uses two single-ended arrows (one each way) to connect interval to key / interval to value.
However, the lower half uses just one double-ended arrow to link key and value - which is confusing at first, because it looks like a link with no possible input.

Either draw two arrows in the lower half, or lose two arrows in the upper half and make the other two upper-half arrows double-ended.


             +-------------------------+
             |  aggregate combination  |
             +-------------------------+
                          |
                          V
                  Aggregated Flows

   (*) partially aggregated Flows

           Figure 3: Conceptual model of aggregation operations

   Interval distribution  is a temporal aggregation operation which
      imposes an Aggregation Interval on the partially aggregated Flow.
      This Aggregation Interval may be regular, irregular, or derived
      from the timing of the original Flows themselves.  Interval
      distribution is discussed in detail in Section 5.1.

Must this be done first?


   Key aggregation   is a spatial aggregation operation which results in
      the addition, modification, or deletion of Flow Key fields in the
      partially aggregated Flows.  New Flow Key fields may be derived
      from existing Flow Key fields (e.g., looking up an AS number for
      an IP address), or "promoted" from non-Key fields (e.g., when
      aggregating Flows by packet count per Flow).  Key aggregation can

The above sounds like any non-key can be promoted, which isn't true and needs to be discussed (see later).

Consider, "or derived from certain specific non-Key fields" ?


      also add new non-Key fields derived from Key Fields that are
      deleted during key aggregation; mainly counters of unique reduced
      keys.  Key aggregation is discussed in detail in Section 5.2.



Trammell, et al.        Expires December 31, 2011               [Page 9]

Internet-Draft              IPFIX Aggregation                  June 2011


   Value aggregation   is a spatial aggregation operation which results
      in the addition, modification, or deletion of non-Key fields in
      the partially aggregated Flows.  These non-Key fields may be
      "demoted" from existing Key fields, or derived from existing Key
      or non-Key fields.  Value aggregation is discussed in detail in
      Section 5.3.

   Aggregate combination   combines multiple partially aggregated Flows
      having undergone interval distribution, key aggregation, and value
      aggregation which share Flow Keys and Aggregation Intervals into a
      single aggregated Flow per Flow Key and Aggregation Interval.
      Aggregate combination is discussed in detail in Section 5.4.

   The first three of these operations may be carried out any number of
   times in any order, either on original Flows or on the results of one

"in any order" contradicts Figure 3, which distinctly shows that "interval distribution" must be first.


   of the Operations (called partially aggregated Flows), with one

"Operations" isn't defined. Either define it, or de-capitalise it.


   caveat.  Since Flows carry their own interval data, any spatial
   aggregation operation implies a temporal aggregation operation, so at
   least one interval distribution step, even if implicit, is required
   by this architecture.  This is shown as the first step for the sake
   of simplicity in the diagram above.  Once all aggregation operations
   are complete, aggregate combination ensures that for a given
   Aggregation Interval, Flow Key, and Observation Domain, only one Flow
   is produced by the Intermediate Aggregation Process.


5.  IP Flow Aggregation Operations

   As stated in Section 2, an Aggregated Flow is simply an IPFIX Flow
   generated from original Flows by an Aggregation Function.  Here, we
   detail the operations by which this is achieved within an
   Intermediate Aggregation Process.

5.1.  Temporal Aggregation through Interval Distribution

   Interval distribution imposes a time interval on the resulting
   Aggregated Flows.  The selection of an interval is specific to the
   given aggregation application.  Intervals may be derived from the
   original Flows themselves (e.g., an interval may be selected to cover
   the entire interval containing the set of all Flows sharing a given

Avoid "interval ... interval", eg by changing the second occurrence to "time" ?


   Key, as in Time Composition describe in Section 5.1.2) or externally

Typo, "described".


   imposed; in the latter case the externally imposed interval may be
   regular (e.g., every five minutes) or irregular (e.g., to allow for
   different time resolutions at different times of day, under different
   network conditions, or indeed for different sets of original Flows).

How does the Aggregated Flow consumer know what the aggregation interval was, especially in the "irregular" case?
ie, with regular intervals, a collector can simply store the received Aggregates as eg hourly reports.

However, with irregular intervals it may have two minutes here and ten minutes there... which may not be easy to process in a meaningful way?


   The length of the imposed interval itself has tradeoffs.  Shorter
   intervals allow higher resolution aggregated data and, in streaming



Trammell, et al.        Expires December 31, 2011              [Page 10]

Internet-Draft              IPFIX Aggregation                  June 2011


   applications, faster reaction time.  Longer intervals lead to greater
   data reduction and simplified counter distribution.  Specifically,

*may* lead to greater data reduction, though not necessarily.


   counter distribution is greatly simplified by the choice of an
   interval longer than the duration of longest original Flow, itself
   generally determined by the original Flow's Metering Process active
   timeout; in this case an original Flow can contribute to at most two
   Aggregated Flows, and the more complex value distribution methods
   become inapplicable.

This only considers the Metering Process; it fails to consider any delay in the Exporting Process.

eg, consider and implementation with 64K cache entries which are checked for exporting at a rate of 1000 entries/second. It will take > 1 minute to check and export the entire cache.

Again, consider an implementation which aims to maximise the size of each export packet, building it up a record at a time as records expire from the cache, until the export packet is quite full. If records expire slowly then the mechanism may introduce a significant delay.

In effect both of these delays cause flows to appear in later intervals than may be expected. eg, in Figure 4, "Flow A" may appear in interval 1 or interval 2, rather than interval 0.


   |                |                |                |
   | |<--Flow A-->| |                |                |
   |        |<--Flow B-->|           |                |
   |          |<-------------Flow C-------------->|   |
   |                |                |                |
   |   interval 0   |   interval 1   |   interval 2   |

              Figure 4: Illustration of interval distribution

   In Figure 4, we illustrate three common possibilities for interval
   distribution as applies with regular intervals to a set of three
   original Flows.  For Flow A, the start and end times lie within the
   boundaries of a single interval 0; therefore, Flow A contributes to
   only one Aggregated Flow.  Flow B, by contrast, has the same duration
   but crosses the boundary between intervals 0 and 1; therefore, it
   will contribute to two Aggregated Flows, and its counters must be
   distributed among these Flows, though in the two-interval case this
   can be simplified somewhat simply by picking one of the two
   intervals, or proportionally distributing between them.  Only Flows
   like Flow A and Flow B will be produced when the interval is chosen
   to be longer than the duration of longest original Flow, as above.
   More complicated is the case of Flow C, which contributes to more
   than two Aggregated Flows, and must have its counters distributed
   according to some policy as in Section 5.1.1.

   [EDITOR'S NOTE: per Lothar: some implementation guidance here would
   be good. specifically, advise that you need multiple rotating
   intervals to do this right.]

TODO


5.1.1.  Distributing Values Across Intervals

   In general, counters in Aggregated Flows are treated the same as in
   any Flow.  Each counter is independently is calculated as if it were

Typo, "independently is calculated".


   derived from the set of packets in the original Flow.  For the most
   part, when aggregating original Flows into Aggregated Flows, this is
   simply done by summation.

   When the Aggregation Interval is guaranteed to be longer than the
   longest original Flow, a Flow can cross at most one Interval



Trammell, et al.        Expires December 31, 2011              [Page 11]

Internet-Draft              IPFIX Aggregation                  June 2011


   boundary, and will therefore contribute to at most two Aggregated
   Flows.  Most common in this case is to arbitrarily but consistently
   choose to account the original Flow's counters either to the first or
   the last aggregated Flow to which it could contribute.

"Aggregated Flow"


   However, this becomes more complicated when the Aggregation Interval
   is shorter than the longest original Flow in the source data.  In
   such cases, each original Flow can incompletely cover one or more
   time intervals, and apply to one or more Aggregated Flows; in this
   case, the Aggregation Process must distribute the counters in the
   original Flows across the multiple Aggregated Flows.  There are
   several methods for doing this, listed here in roughly increasing
   order of complexity and accuracy; most of these are necessary only in
   specialized cases.

   End Interval:   The counters for an original Flow are added to the
      counters of the appropriate Aggregated Flow containing the end
      time of the original Flow.

   Start Interval:   The counters for an original Flow are added to the
      counters of the appropriate Aggregated Flow containing the start
      time of the original Flow.

   Mid Interval:   The counters for an original Flow are added to the
      counters of a single appropriate Aggregated Flow containing some
      timestamp between start and end time of the original Flow.

   Simple Uniform Distribution:   Each counter for an original Flow is
      divided by the number of time intervals the original Flow covers
      (i.e., of appropriate Aggregated Flows sharing the same Flow Key),
      and this number is added to each corresponding counter in each
      Aggregated Flow.

   Proportional Uniform Distribution:   Each counter for an original
      Flow is divided by the number of time _units_ the original Flow
      covers, to derive a mean count rate.  This mean count rate is then
      multiplied by the number of time units in the intersection of the
      duration of the original Flow and the time interval of each
      Aggregated Flow.  This is like simple uniform distribution, but
      accounts for the fractional portions of a time interval covered by
      an original Flow in the first and last time interval.

   Simulated Process:   Each counter of the original Flow is distributed
      among the intervals of the Aggregated Flows according to some
      function the Aggregation Process uses based upon properties of
      Flows presumed to be like the original Flow.  For example, Flow
      Records representing bulk transfer might follow a more or less
      proportional uniform distribution, while interactive processes are



Trammell, et al.        Expires December 31, 2011              [Page 12]

Internet-Draft              IPFIX Aggregation                  June 2011


      far more bursty.

   Direct:   The Aggregation Process has access to the original packet
      timings from the packets making up the original Flow, and uses
      these to distribute or recalculate the counters.

   A method for exporting the distribution of counters across multiple
   Aggregated Flows is detailed in Section 7.4.  In any case, counters
   MUST be distributed across the multiple Aggregated Flows in such a
   way that the total count is preserved, within the limits of accuracy
   of the implementation (e.g., inaccuracy introduced by the use of
   floating-point numbers is tolerable).  This property allows data to
   be aggregated and re-aggregated without any loss of original count

"without any loss" -> "with only a tolerable loss".

Personally, I'd drop the parenthesised FP bit.


   information.  To avoid confusion in interpretation of the aggregated
   data, all the counters for a set of given original Flows SHOULD be
   distributed via the same method.

5.1.2.  Time Composition

   Time Composition as in section 5.4 of [RFC5982] (or interval
   combination) is a special case of aggregation, where interval
   distribution imposes longer intervals on Flows with matching keys and
   "chained" start and end times, without any key reduction, in order to
   join long-lived Flows which may have been split (e.g., due to an
   active timeout shorter than the Flow.)  Here, no Key aggregation is
   applied, and the Aggregation Interval is chosen on a per-Flow basis
   to cover the interval spanned by the set of aggregated Flows.  This
   may be applied alone in order to normalize split Flows, or in
   combination with other aggregation functions in order to obtain more
   accurate original Flow counts.

5.2.  Spatial Aggregation of Flow Keys

   Key aggregation generates a new Flow Key for the Aggregated Flows
   from the original Flow Keys, non-Key fields in the original Flows, or

"from the original Flow Key and non-Key fields, or"


   from correlation of the original Flow information with some external
   source.  There are two basic operations here.  First, Aggregated Flow
   Keys may be derived directly from original Flow Keys through
   reduction, or the dropping of fields or precision in the original
   Flow Keys.  Second, an Aggregated Flow Key may be derived through
   replacement, e.g. by removing one or more fields from the original
   Flow and replacing them with a fields derived from the removed
   fields.  Replacement may refer to external information (e.g., IP to
   AS number mappings).  Replacement need not replace only key fields.
   For example, consider an application which aggregates flows by packet
   count (i.e., generating an Aggregated Flow for all one-packet Flows,
   one for all two-packet Flows, and so on).  This application would
   promote the packet count to a Flow Key field.

Say more about promotion, ie it can only be done on non-key fields which represent a single unique value. eg, counters, time.
If a NK field might be any value from a set of >1 values (eg, an IP address, port number, AS, TOS) then it's not suitable for promotion to Key, because there's no way of knowing which other values were or were not observed, and therefore no way of dividing the flow amongst those values.




Trammell, et al.        Expires December 31, 2011              [Page 13]

Internet-Draft              IPFIX Aggregation                  June 2011


   Key aggregation may also result in the addition of new non-Key fields
   to the Aggregated Flows, namely original Flow counters and unique
   reduced key counters; these are treated in more detail in
   Section 5.2.2 and Section 5.2.1, respectively.

I'm sure other non-keys could be derived too, though a good example doesn't immediately spring to mind.


   In any key aggregation operation, reduction and/or replacement may be
   applied any number of times in any order.  Which of these operations
   are supported by a given implementation is implementation- and
   application-dependent.  Key aggregation may aggregate original Flows
   with different sets of Flow Key fields; only the Flow Keys of the
   resulting Aggregated Flows of any given Key aggregation operation
   need contain the same set of fields.

I don't understand what "only ... fields" is saying.


   Original Flow Key
   +---------+---------+----------+----------+-------+-----+
   | src ip4 | dst ip4 | src port | dst port | proto | tos |
   +---------+---------+----------+----------+-------+-----+
        |         |         |          |         |      |
     retain   mask /24      X          X         X      X
        V         V
   +---------+-------------+
   | src ip4 | dst ip4 /24 |
   +---------+-------------+
   Aggregated Flow Key (by source address and destination class-C)

          Figure 5: Illustration of key aggregation by reduction

   Figure 5 illustrates an example reduction operation, aggregation by
   source address and destination class C network.  Here, the port,
   protocol, and type-of-service information is removed from the Flow

is -> are ?


   Key, the source address is retained, and the destination address is
   masked by dropping the low 8 bits.

low -> lower ?


   Original Flow Key
   +---------+---------+----------+----------+-------+-----+
   | src ip4 | dst ip4 | src port | dst port | proto | tos |
   +---------+---------+----------+----------+-------+-----+
        |         |         |          |         |      |
   +-------------------+    X          X         X      X
   | ASN lookup table  |
   +-------------------+
        V         V
   +---------+---------+
   | src asn | dst asn |
   +---------+---------+
   Aggregated Flow Key (by source and dest ASN)

        Figure 6: Illustration of key aggregation by reduction and



Trammell, et al.        Expires December 31, 2011              [Page 14]

Internet-Draft              IPFIX Aggregation                  June 2011


                                replacement

   Figure 6 illustrates an example reduction and replacement operation,
   aggregation by source and destination ASN without ASN information
   available in the original Flow.  Here, the port, protocol, and type-
   of-service information is removed from the Flow Key, while the source

is -> are?


   and destination addresses are run though an IP address to ASN lookup
   table, and the Aggregated Flow Key is made up of the resulting source
   and destination ASNs.

5.2.1.  Counting Distinct Key Values

   One common case in aggregation is counting distinct key values that
   were reduced away during key aggregation.  The most common use case
   for this is counting distinct hosts per Flow Key; for example, in
   host characterization or anomaly detection, distinct sources per
   destination or distinct destinations per source are common metrics.
   These new non-Ley fields are added during key aggregation.

s/Ley/Key/

   For such applications, Information Elements for distinct counts of
   IPv4 and IPv6 addresses are defined in Section 7.3.  These are named
   distinctCountOf(KeyName).  Additional such Information Elements
   SHOULD be registered with IANA on an as-needed basis.

5.2.2.  Counting Original Flows

   When aggregating multiple original Flows into an Aggregated Flow, it
   is often useful to know how many original Flows are present in the
   Aggregated Flow.  This document introduces four new information
   elements in Section 7.2 to export these counters.

   There are two possible ways to count original Flows, which we call
   here conservative and non-conservative.  Conservative flow counting
   has the property that each original Flow contributes exactly one to
   the total flow count within a set of aggregated Flows.  In other
   words, conservative flow counters are distributed just as any other
   counter during interval distribution, except each original Flow is
   assumed to have a flow count of one.  When a count for an original
   Flow must be distributed across a set of Aggregated Flows, and a
   distribution method is used which does not account for that original
   Flow completely within a single Aggregated Flow, conservative flow
   counting requires a fractional representation.

   By contrast, non-conservative flow counting is used to count how many
   contributing Flows are represented in an Aggregated Flow.  Flow
   counters are not distributed in this case.  An original Flow which is
   present within N Aggregated Flows would add N to the sum of non-
   conservative flow counts, one to each Aggregated Flow.  In other



Trammell, et al.        Expires December 31, 2011              [Page 15]

Internet-Draft              IPFIX Aggregation                  June 2011


   words, the sum of conservative flow counts over a set of Aggregated
   Flows is always equal to the number of original Flows, while the sum
   of non-conservative flow counts is strictly greater than or equal to
   the number of original Flows.

   For example, consider Flows A, B, and C as illustrated in Figure 4.
   Assume that the key aggregation step aggregates the keys of these
   three Flows to the same aggregated Flow Key, and that start interval
   counter distribution is in effect.  The conservative flow count for
   interval 0 is 3 (since Flows A, B, and C all begin in this interval),
   and for the other two intervals is 0.  The non-conservative flow
   count for interval 0 is also 3 (due to the presence of Flows A, B,
   and C), for interval 1 is 2 (Flows B and C), and for interval 2 is 1
   (Flow C).  The sum of the conservative counts 3 + 0 + 0 = 3, the
   number of original Flows; while the sum of the non-conservative
   counts 3 + 2 + 1 = 6.

   Note that the active and inactive timeouts used to generate original
   Flows, as well as the cache policy used to generate those Flows, have
   an effect on how meaningful either the conservative or non-
   conservative flow count will be during aggregation.  In general, all
   the original Exporters producing original Flows to be aggregated
   SHOULD be aggregated using caches configured identically or
   similarly.  Original Exporters using the IPFIX Configuration Model
   SHOULD be configured to export Flows with equal or similar
   activeTimeout and inactiveTimeout configuration values, and the same
   cacheMode, as defined in section 4.3 of
   [I-D.ietf-ipfix-configuration-model].

It's not defined there since -09 of that draft.


5.3.  Spatial Aggregation of Non-Key Fields

   Aggregation operations may also lead to the addition of value fields
   demoted from key fields, or derived from other value fields in the
   original Flows.  Specific cases of this are treated in the
   subsections below.

5.3.1.  Counter Statistics

   Some applications of aggregation may benefit from computing different
   statistics than those native to each non-key field (i.e., union for
   flags, sum for counters).  For example, minimum and maximum packet
   counts per Flow, mean bytes per packet per aggregated Flow, and so
   on.  Certain Information Elements for these applications are already
   provided in the IANA IPFIX Information Elements registry
   (http://www.iana.org/assignments/ipfix/ipfix.html (e.g.
   minimumIpTotalLength).

   A complete specification of additional aggregate counter statistics



Trammell, et al.        Expires December 31, 2011              [Page 16]

Internet-Draft              IPFIX Aggregation                  June 2011


   is outside the scope of this document, and should be added in the
   future to the IANA IPFIX Information Elements registry on a per-
   application, as-needed basis.

5.4.  Aggregation Combination

   Interval distribution and key aggregation together may generate
   multiple partially aggregated Flows covering the same time interval
   with the same Flow Key. The process of combining these partially
   aggregated Flows into a single Aggregated Flow is called aggregation
   combination.  In general, non-Key values from multiple contributing
   Flows are combined using the same operation by which values are
   combined from packets to form Flows for each Information Element.
   Counters are summed, averages are averaged, flags are unioned, and so
   on.

Although this is common knowledge for us, is it actually specified anywhere in IPFIX?

ie, should [IANA-IPFIX] list the relevant aggregation operation for each field? eg, min, max, sum, union, none?



6.  Additional Considerations and Special Cases in Flow Aggregation

6.1.  Exact versus Approximate Counting during Aggregation

   In certain circumstances, particularly involving aggregation by
   devices with limited resources, and in situations where exact
   aggregated counts are less important than relative magnitudes (e.g.
   driving graphical displays), counter distribution during key
   aggregation may be performed by approximate counting means (e.g.
   Bloom filters).  The choice to use approximate counting is
   implementation- and application-dependent.

6.2.  Considerations for Aggregation of Sampled Flows

   The accuracy of Aggregated Flows may also be affected by sampling of
   the original Flows, or sampling of packets making up the original
   Flows.  The effect of sampling on flow aggregation is still an open
   research question.  However, to maximize the comparability of

"At the time of writing, the effect of...".

This is one of several points in this draft where I'd expect additional citations if it wasn't an IETF document :-)


   Aggregated Flows, aggregation of sampled Flows SHOULD only use
   original Flows sampled using the same sampling rate and sampling
   algorithm, or Flows created from packets sampled using the same
   sampling rate and sampling algorithm.  For more on packet sampling

I think it's also allowable to normalise flows from different samplers / rates before aggregating them.


   within IPFIX, see [RFC5476].  For more on Flow sampling within the
   IPFIX Mediator Framework, see [I-D.ietf-ipfix-flow-selection-tech].


7.  Export of Aggregated IP Flows using IPFIX

   In general, Aggregated Flows are exported in IPFIX as any normal
   Flow.  However, certain aspects of Aggregated Flow export benefit
   from additional guidelines, or new Information Elements to represent



Trammell, et al.        Expires December 31, 2011              [Page 17]

Internet-Draft              IPFIX Aggregation                  June 2011


   aggregation metadata or information generated during aggregation.
   These are detailed in the following subsections.

Have the new IEs been implemented in released code?
We should avoid defining theoretical new IEs (ie, those which nobody has actually implemented, or plans to implement).
"Rough consensus and _running code_" :-)


7.1.  Time Interval Export

   Since an Aggregated Flow is simply a Flow, the existing timestamp
   Information Elements in the IPFIX Information Model (e.g.,
   flowStartMilliseconds, flowEndNanoseconds) are sufficient to specify
   the time interval for aggregation.  Therefore, this document
   specifies no new aggregation-specific Information Elements for
   exporting time interval information.

   Each Aggregated Flow SHOULD contain both an interval start and
   interval end timestamp.  If an exporter of Aggregated Flows omits the

Disagree. Maybe the aggregation step is to get rid of timestamps.


   interval end timestamp from each Aggregated Flow, the time interval
   for Aggregated Flows within an Observation Domain and Transport
   Session MUST be regular and constant.  However, note that this
   approach might lead to interoperability problems when exporting
   Aggregated Flows to non-aggregation-aware Collecting Processes and
   downstream analysis tasks; therefore, an Exporting Process capable of
   exporting only interval start timestamps MUST provide a configuration
   option to export interval end timestamps as well.

I disagree with this time fixation. Time is only one aspect of flows, not the only one.


7.2.  Flow Count Export

   The following four Information Elements are defined to count original
   Flows as discussed in Section 5.2.2.

7.2.1.  originalFlowsPresent

   Description:   The non-conservative count of original Flows
      contributing to this Aggregated Flow.  Non-conservative counts
      need not sum to the original count on re-aggregation.

   Abstract Data Type:   unsigned64

   ElementId:   TBD1

   Status:   Current

7.2.2.  originalFlowsInitiated

   Description:   The conservative count of original Flows whose first
      packet is represented within this Aggregated Flow.  Conservative
      counts must some to the original count on re-aggregation.

some -> sum







Trammell, et al.        Expires December 31, 2011              [Page 18]

Internet-Draft              IPFIX Aggregation                  June 2011


   Abstract Data Type:   unsigned64

   ElementId:   TBD2

   Status:   Current

7.2.3.  originalFlowsCompleted

   Description:   The conservative count of original Flows whose last
      packet is represented within this Aggregated Flow.  Conservative
      counts must some to the original count on re-aggregation.

some -> sum


   Abstract Data Type:   unsigned64

   ElementId:   TBD3

   Status:   Current

7.2.4.  originalFlows

For consistency with the above IEs, name this "originalFlowsContributing" ?


   Description:   The conservative count of original Flows contributing
      to this Aggregated Flow; may be distributed via any of the methods
      described in Section 5.1.1.

   Abstract Data Type:   float64

   ElementId:   3

   Status:   Current

7.3.  Distinct Host Export

   The following four Information Elements represent the distinct counts
   of source and destination addresses for IPv4 and IPv6, used to
   exporting distinct host counts reduced away during key aggregation.

7.3.1.  distinctCountOfSourceIPv4Address

   Description:   The count of distinct source IPv4 address values for
      original Flows contributing to this Aggregated Flow.

   Abstract Data Type:   unsigned32

   ElementId:   TBD6







Trammell, et al.        Expires December 31, 2011              [Page 19]

Internet-Draft              IPFIX Aggregation                  June 2011


   Status:   Current

7.3.2.  distinctCountOfDestinationIPv4Address

   Description:   The count of distinct destination IPv4 address values
      for original Flows contributing to this Aggregated Flow.

   Abstract Data Type:   unsigned32

   ElementId:   TBD7

   Status:   Current

7.3.3.  distinctCountOfSourceIPv6Address

   Description:   The count of distinct source IPv6 address values for
      original Flows contributing to this Aggregated Flow.

   Abstract Data Type:   unsigned64

   ElementId:   TBD8

   Status:   Current

7.3.4.  distinctCountOfDestinationIPv6Address

   Description:   The count of distinct destination IPv6 address values
      for original Flows contributing to this Aggregated Flow.

   Abstract Data Type:   unsigned64

   ElementId:   TBD9

   Status:   Current

7.4.  Aggregate Counter Distribution Export

   When exporting counters distributed among Aggregated Flows, as
   described in Section 5.1.1, the Exporting Process MAY export an
   Aggregate Counter Distribution Record for each Template describing
   Aggregated Flow records; this Options Template is described below.

Say "options" earlier. eg,

   the Exporting Process MAY export an
   Aggregate Counter Distribution Option Record for each Template describing
   Aggregated Flow records. The Options Template is described below.


   It uses the valueDistributionMethod Information Element, also defined
   below.  Since in many cases distribution is simple, accounting the
   counters from contributing Flows to the first Interval to which they
   contribute, this is default situation, for which no Aggregate Counter

"this is _the_ default situation"


   Distribution Record is necessary; Aggregate Counter Distribution
   Records are only applicable in more exotic situations, such as using
   an Aggregation Interval smaller than the durations of original Flows.



Trammell, et al.        Expires December 31, 2011              [Page 20]

Internet-Draft              IPFIX Aggregation                  June 2011


7.4.1.  Aggregate Counter Distribution Options Template

   This Options Template defines the Aggregate Counter Distribution
   Record, which allows the binding of a value distribution method to a
   Template ID.  This is used to signal to the Collecting Process how
   the counters were distributed.  The fields are as below:

   +-------------------------+-----------------------------------------+
   | IE                      | Description                             |
   +-------------------------+-----------------------------------------+
   | templateId [scope]      | The Template ID of the Template         |
   |                         | defining the Aggregated Flows to which  |
   |                         | this distribution option applies.  This |
   |                         | Information Element MUST be defined as  |
   |                         | a Scope Field.                          |
   | valueDistributionMethod | The method used to distribute the       |
   |                         | counters for the Aggregated Flows       |
   |                         | defined by the associated Template.     |
   +-------------------------+-----------------------------------------+

Presumably the scope could be more specific, eg an "informationElementIndex" could be included.


7.4.2.  valueDistributionMethod Information Element

   Description:   A description of the method used to distribute the
      counters from contributing Flows into the Aggregated Flow records
      described by an associated Template.  The method is deemed to
      apply to all the non-key Information Elements in the referenced
      Template for which value distribution is a valid operation; if the

No; it applies to the specified scope, which happens to be a templateId in this case - but may be something else in other uses.

eg, vDM could be used with structured data, where the intention would be that it applies only to the current relevant structure, and not to all the non-key elements in some other template.


      originalFlowsInitiated and/or originalFlowsCompleted Information
      Elements appear in the Template, they are not subject to this
      distribution method, as they each infer their own distribution
      method.  The distribution methods are taken from Section 5.1.1 and
      encoded as follows:

Please separate the table entries below to make them easier to read.


   +-------+-----------------------------------------------------------+
   | Value | Description                                               |
   +-------+-----------------------------------------------------------+

What does value = zero mean?


   | 1     | Start Interval: The counters for an original Flow are     |
   |       | added to the counters of the appropriate Aggregated Flow  |
   |       | containing the start time of the original Flow.  This     |
   |       | should be assumed the default if value distribution       |
   |       | information is not available at a Collecting Process for  |
   |       | an Aggregated Flow.                                       |
   | 2     | End Interval: The counters for an original Flow are added |
   |       | to the counters of the appropriate Aggregated Flow        |
   |       | containing the end time of the original Flow.             |






Trammell, et al.        Expires December 31, 2011              [Page 21]

Internet-Draft              IPFIX Aggregation                  June 2011


   | 3     | Mid Interval: The counters for an original Flow are added |
   |       | to the counters of a single appropriate Aggregated Flow   |
   |       | containing some timestamp between start and end time of   |
   |       | the original Flow.                                        |
   | 4     | Simple Uniform Distribution: Each counter for an original |
   |       | Flow is divided by the number of time intervals the       |
   |       | original Flow covers (i.e., of appropriate Aggregated     |
   |       | Flows sharing the same Flow Key), and this number is      |
   |       | added to each corresponding counter in each Aggregated    |
   |       | Flow.                                                     |
   | 5     | Proportional Uniform Distribution: Each counter for an    |
   |       | original Flow is divided by the number of time _units_    |
   |       | the original Flow covers, to derive a mean count rate.    |
   |       | This mean count rate is then multiplied by the number of  |
   |       | time units in the intersection of the duration of the     |
   |       | original Flow and the time interval of each Aggregated    |
   |       | Flow.  This is like simple uniform distribution, but      |
   |       | accounts for the fractional portions of a time interval   |
   |       | covered by an original Flow in the first and last time    |
   |       | interval.                                                 |
   | 6     | Simulated Process: Each counter of the original Flow is   |
   |       | distributed among the intervals of the Aggregated Flows   |
   |       | according to some function the Aggregation Process uses   |
   |       | based upon properties of Flows presumed to be like the    |
   |       | original Flow.  This is essentially an assertion that the |
   |       | Aggregation Process has no direct packet timing           |
   |       | information but is nevertheless not using one of the      |
   |       | other simpler distribution methods.  The Aggregation      |
   |       | Process specifically makes no assertion as to the         |
   |       | correctness of the simulation.                            |
   | 7     | Direct: The Aggregation Process has access to the         |
   |       | original packet timings from the packets making up the    |
   |       | original Flow, and uses these to distribute or            |
   |       | recalculate the counters.                                 |
   +-------+-----------------------------------------------------------+

State whether this can be extended in future, or whether a new IE would be required.

From an IE-doctors point of view, this should generally be stated for all list-based IEs, since someone could extend the list and claim support for this IE, while someone else claims support with the original, more limited, list. Clearly the two "supporting" processes would not be interoperable.

Expressed another way: if the list is extensible then a new implementation which extends the list invalidates all existing implementations.

However, requiring new IEs for each new value may be unreasonable, leading to fragmentation (values 1-7 in IE#1, 8-9 in IE#2, 10-12 in IE#3) and unnecessarily hastening IE space exhaustion.


   Abstract Data Type:   unsigned8

   ElementId:   TBD5

   Status:   Current


8.  Examples

   In these examples, the same data, described by the same template,
   will be aggregated multiple different ways; this illustrates the
   various different functions which could be implemented by



Trammell, et al.        Expires December 31, 2011              [Page 22]

Internet-Draft              IPFIX Aggregation                  June 2011


   Intermediate Aggregation Processes.  Templates are shown in iespec

s/iespec/IEspec/


   format as introduced in [I-D.trammell-ipfix-ie-doctors].  The source
   data format is a simplified flow: timestamps, traditional 5-tuple,
   and octet count.  The template is shown in Figure 7.

   flowStartMilliseconds
   flowEndMilliseconds
   sourceIPv4Address
   destinationIPv4Address
   sourceTransportPort
   destinationTransportPort
   protocolIdentifier
   octetDeltaCount

                   Figure 7: Input template for examples

   The data records given as input to the examples in this section are
   shown below, in the format "flowStartMilliseconds-flowEndMilliseconds
   sourceIPv4Address:sourceTransportPort -> destinationIPv4Address:
   destinationTransportPort (protocolIdentifier) octetDeltaCount";
   timestamps are given in H:MM:SS.sss format.

9:00:00.138-9:00:00.138 192.0.2.2:47113   -> 192.0.2.131:53   (17)   119
9:00:03.246-9:00:03.246 192.0.2.2:22153   -> 192.0.2.131:53   (17)    83
9:00:00.478-9:00:03.486 192.0.2.2:52420   -> 198.51.100.2:443  (6)  1637
9:00:07.172-9:00:07.172 192.0.2.3:56047   -> 192.0.2.131:53   (17)   111
9:00:07.309-9:00:14.861 192.0.2.3:41183   -> 198.51.100.67:80  (6) 16838
9:00:03.556-9:00:19.876 192.0.2.2:17606   -> 198.51.100.68:80  (6) 11538
9:00:25.210-9:00:25.210 192.0.2.3:47113   -> 192.0.2.131:53   (17)   119
9:00:26.358-9:00:30.198 192.0.2.3:48458   -> 198.51.100.133:80 (6)  2973
9:00:29.213-9:01:00.061 192.0.2.4:61295   -> 198.51.100.2:443  (6)  8350
9:04:00.207-9:04:04.431 203.0.113.3:41256 -> 198.51.100.133:80 (6)   778
9:03:59.624-9:04:06.984 203.0.113.3:51662 -> 198.51.100.3:80   (6)   883
9:06:56.813-9:06:59.821 203.0.113.3:52572 -> 198.51.100.2:443  (6)  1637
9:06:30.565-9:07:00.261 203.0.113.3:49914 -> 197.51.100.133:80 (6)   561
9:06:55.160-9:07:05.208 192.0.2.2:50824   -> 198.51.100.2:443  (6)  1899
9:06:49.322-9:07:05.322 192.0.2.3:34597   -> 198.51.100.3:80   (6)  1284
9:07:05.849-9:07:09.625 203.0.113.3:58907 -> 198.51.100.4:80   (6)  2670
9:10:45.161-9:10:45.161 192.0.2.4:22478   -> 192.0.2.131:53   (17)    75
9:10:45.209-9:11:01.465 192.0.2.4:49513   -> 198.51.100.68:80  (6)  3374
9:10:57.094-9:11:00.614 192.0.2.4:64832   -> 198.51.100.67:80  (6)   138
9:10:59.770-9:11:02.842 192.0.2.3:60833   -> 198.51.100.69:443 (6)  2325
9:13:53.933-9:14:06.605 192.0.2.2:19638   -> 198.51.100.3:80   (6)  2869
9:13:02.864-9:14:08.720 192.0.2.3:40429   -> 198.51.100.4:80   (6) 18289

                     Figure 8: Input data for examples

Consider listing the values in aligned columns without the syntax?
It fits into 76 chars with double-spaced columns, and is slightly easier to read.
Also, good to add column titles to make the subsequent figures easier to follow.

Finally, the data are a little out of order. Although it really doesn't matter, in-order might make the example a little easier to follow for pedants who really want to check your data manually, like me.






Trammell, et al.        Expires December 31, 2011              [Page 23]

Internet-Draft              IPFIX Aggregation                  June 2011


8.1.  Traffic Time-Series per Source

   Aggregating flows by source IP address in time series (i.e., with a
   regular interval) can be used in subsequent heavy-hitter analysis and
   as a source parameter for statistical anomaly detection techniques.
   Here, the Intermediate Aggregation Process imposes an interval,
   aggregates the key to remove all key fields other than the source IP
   address, then combines the result into a stream of Aggregated Flows.
   For simplicity, the imposed interval of 30 minutes is defined to be
   larger than the maximum active timeout of the original Flows; counter
   distribution will be added to this example below in Section 8.4.

"to this example in Section 8.4 below."


   In this example the partially aggregated Flows after each conceptual
   operation in the Intermediate Aggregation Process are shown.  These
   are meant to be illustrative of the conceptual operations only, and
   not to suggest an implementation (indeed, the example shown here
   would not necessarily be the most efficient method for performing
   these operations).  Subsequent examples will omit the partially
   aggregated Flows for brevity.

   The input to this process could be any Flow Record containing a
   source IP address and octet counter; consider for this example the
   template and data from the introduction.  The Intermediate
   Aggregation Process would then output records containing just
   timestamps, source IP, and octetDeltaCount, as in Figure 9.

   flowStartMilliseconds
   flowEndMilliseconds
   sourceIPv4Address
   octetDeltaCount

           Figure 9: Output template for time series per source

   Assume the goal is to get 5-minute time series of octet counts per

For consistency, either say "300s" here or say "5-minutes" in the figure below.


   source IP address.  The aggregation operations would then be arranged
   as in Figure 10.















Trammell, et al.        Expires December 31, 2011              [Page 24]

Internet-Draft              IPFIX Aggregation                  June 2011


                    original Flows
                          |
                          V
              +-----------------------+
              | interval distribution |
              |  * impose uniform     |
              |    300s time interval |
              +-----------------------+
                  |
                  | partially aggregated Flows
                  V
   +------------------------+
   |  key aggregation       |
   |   * reduce key to only |
   |     sourceIPv4Address  |
   +------------------------+
                  |
                  | partially aggregated Flows
                  V
             +-------------------------+
             |  aggregate combination  |
             |   * sum octetDeltaCount |
             +-------------------------+
                          |
                          V
                  Aggregated Flows

   partially aggregated Flows

       Figure 10: Aggregation operations for time series per source

   After the interval distribution step, only the time intervals have
   changed; the partially aggregated Flows are shown in Figure 11.

Is the interval distribution being calculated on the flow start or flow end timestamp?
Although the carefully chosen data give the same result either way, it should be stated.



















Trammell, et al.        Expires December 31, 2011              [Page 25]

Internet-Draft              IPFIX Aggregation                  June 2011


9:00:00.000-9:05:00.000 192.0.2.2:47113   -> 192.0.2.131:53   (17)   119
9:00:00.000-9:05:00.000 192.0.2.2:22153   -> 192.0.2.131:53   (17)    83
9:00:00.000-9:05:00.000 192.0.2.2:52420   -> 198.51.100.2:443  (6)  1637
9:00:00.000-9:05:00.000 192.0.2.3:56047   -> 192.0.2.131:53   (17)   111
9:00:00.000-9:05:00.000 192.0.2.3:41183   -> 198.51.100.67:80  (6) 16838
9:00:00.000-9:05:00.000 192.0.2.2:17606   -> 198.51.100.68:80  (6) 11538
9:00:00.000-9:05:00.000 192.0.2.3:47113   -> 192.0.2.131:53   (17)   119
9:00:00.000-9:05:00.000 192.0.2.3:48458   -> 198.51.100.133:80 (6)  2973
9:00:00.000-9:05:00.000 192.0.2.4:61295   -> 198.51.100.2:443  (6)  8350
9:00:00.000-9:05:00.000 203.0.113.3:41256 -> 198.51.100.133:80 (6)   778
9:00:00.000-9:05:00.000 203.0.113.3:51662 -> 198.51.100.3:80   (6)   883
9:05:00.000-9:10:00.000 203.0.113.3:52572 -> 198.51.100.2:443  (6)  1637
9:05:00.000-9:10:00.000 203.0.113.3:49914 -> 197.51.100.133:80 (6)   561
9:05:00.000-9:10:00.000 192.0.2.2:50824   -> 198.51.100.2:443  (6)  1899
9:05:00.000-9:10:00.000 192.0.2.3:34597   -> 198.51.100.3:80   (6)  1284
9:05:00.000-9:10:00.000 203.0.113.3:58907 -> 198.51.100.4:80   (6)  2670
9:10:00.000-9:15:00.000 192.0.2.4:22478   -> 192.0.2.131:53   (17)    75
9:10:00.000-9:15:00.000 192.0.2.4:49513   -> 198.51.100.68:80  (6)  3374
9:10:00.000-9:15:00.000 192.0.2.4:64832   -> 198.51.100.67:80  (6)   138
9:10:00.000-9:15:00.000 192.0.2.3:60833   -> 198.51.100.69:443 (6)  2325
9:10:00.000-9:15:00.000 192.0.2.2:19638   -> 198.51.100.3:80   (6)  2869
9:10:00.000-9:15:00.000 192.0.2.3:40429   -> 198.51.100.4:80   (6) 18289

         Figure 11: Partially aggregated Flows: intervals imposed

Why two timestamps? It's unclear which interval a flow at 9:05:00.000 should be in.
Whereas listing a single timestamp (9:00, 9:05, 9:10) is less ambiguous.


   After the key aggregation step, all the parts of the flow key except
   the source IP address have been discarded, as shown in Figure 12.
   This leaves duplicate partially aggregated Flows to be combined the
   final operation.






















Trammell, et al.        Expires December 31, 2011              [Page 26]

Internet-Draft              IPFIX Aggregation                  June 2011


   9:00:00.000-9:05:00.000 192.0.2.2      119
   9:00:00.000-9:05:00.000 192.0.2.2       83
   9:00:00.000-9:05:00.000 192.0.2.2     1637
   9:00:00.000-9:05:00.000 192.0.2.3      111
   9:00:00.000-9:05:00.000 192.0.2.3    16838
   9:00:00.000-9:05:00.000 192.0.2.2    11538
   9:00:00.000-9:05:00.000 192.0.2.3      119
   9:00:00.000-9:05:00.000 192.0.2.3     2973
   9:00:00.000-9:05:00.000 192.0.2.4     8350
   9:00:00.000-9:05:00.000 203.0.113.3    778
   9:00:00.000-9:05:00.000 203.0.113.3    883
   9:05:00.000-9:10:00.000 203.0.113.3   1637
   9:05:00.000-9:10:00.000 203.0.113.3    561
   9:05:00.000-9:10:00.000 192.0.2.2     1899
   9:05:00.000-9:10:00.000 192.0.2.3     1284
   9:05:00.000-9:10:00.000 203.0.113.3   2670
   9:10:00.000-9:15:00.000 192.0.2.4       75
   9:10:00.000-9:15:00.000 192.0.2.4     3374
   9:10:00.000-9:15:00.000 192.0.2.4      138
   9:10:00.000-9:15:00.000 192.0.2.3     2325
   9:10:00.000-9:15:00.000 192.0.2.2     2869
   9:10:00.000-9:15:00.000 192.0.2.3    18289

          Figure 12: Partially aggregated Flows: key aggregation

   Aggregate combination sums the counters per key and interval; the
   summations of the first two keys and intervals are shown in detail in
   Figure 13.

     9:00:00.000-9:05:00.000 192.0.2.2      119
     9:00:00.000-9:05:00.000 192.0.2.2       83
     9:00:00.000-9:05:00.000 192.0.2.2     1637
   + 9:00:00.000-9:05:00.000 192.0.2.2    11538
                                          -----
   = 9:00:00.000-9:05:00.000 192.0.2.2    13377

     9:00:00.000-9:05:00.000 192.0.2.3      111
     9:00:00.000-9:05:00.000 192.0.2.3    16838
     9:00:00.000-9:05:00.000 192.0.2.3      119
   + 9:00:00.000-9:05:00.000 192.0.2.3     2973
                                          -----
   = 9:00:00.000-9:05:00.000 192.0.2.3    20041

             Figure 13: Summation during aggregate combination

   Applying this to each set of partially aggregated Flows to produce
   the final Aggregated Flows shown in Figure 14m to be exported by the
   template in Figure 9.



Trammell, et al.        Expires December 31, 2011              [Page 27]

Internet-Draft              IPFIX Aggregation                  June 2011


   9:00:00.000-9:05:00.000 192.0.2.2    13377
   9:00:00.000-9:05:00.000 192.0.2.3    20041
   9:00:00.000-9:05:00.000 192.0.2.4     8350
   9:00:00.000-9:05:00.000 203.0.113.3   1661
   9:05:00.000-9:10:00.000 192.0.2.2     1899
   9:05:00.000-9:10:00.000 192.0.2.3     1284
   9:05:00.000-9:10:00.000 203.0.113.3   4868
   9:10:00.000-9:15:00.000 192.0.2.2     2869
   9:10:00.000-9:15:00.000 192.0.2.3    20594

20594 should be 2325 + 18289 = 20614.


   9:10:00.000-9:15:00.000 192.0.2.4     3587

                        Figure 14: Aggregated Flows

As a cross-check, sum(Figure_8) = sum(Figure_11) = 78550, while sum(Figure_14) = 78530 - so there are 20 octets missing.


8.2.  Core Traffic Matrix

   Aggregating flows by source and destination autonomous system number
   in time series is used to generate core traffic matrices.  The core
   traffic matrix provides a view of the state of the routes within a
   network, and can be used for long-term planning of changes to network
   design based on traffic demand.  Here, imposed time intervals are
   generally much longer than active flow timeouts.  The traffic matrix
   is reported in terms of octets, packets, and flows, as each of these
   values may have a subtly different effect on capacity planning.

   This example demonstrates key aggregation using derived keys and
   Original Flow counting.  While some original Flows may be generated
   by Exporting Processes on forwarding devices, and therefore contain
   the bgpSourceAsNumber and bgpDestinationAsNumber Information
   Elements, original Flows from Exporting Processes on dedicated
   measurement devices will contain only a destinationIPv[46]Address.
   For these flows, the Mediator must look up a next hop AS from a IP to
   AS table, replacing source and destination addresses with AS numbers.

   [TODO: complete example. show AS map, output templates, and
   processing in IAP.]

TODO.


8.3.  Distinct Source Count per Destination Endpoint

   Aggregating flows by destination address and port, and counting
   distinct sources aggregated away, can be used as part of passive
   service inventory and host characterization approaches.  This example
   shows aggregation as an analysis technique, performed on source data
   stored in an IPFIX File.  As the Transport Session in this File is
   bounded, removal of all timestamp information allows summarization of
   the entire time interval contained within the interval.  Removal of
   timing information during interval imposition is equivalent to an
   infinitely long imposed time interval.  This demonstrates both how
   infinite intervals work, and how unique counters work.

At last, an example which shows that time is _not_ the principal factor in every aggregation - and is not even required.

Since there's no temporal aggregation here, there's no interval distribution - which contradicts figure 3.




Trammell, et al.        Expires December 31, 2011              [Page 28]

Internet-Draft              IPFIX Aggregation                  June 2011


   [TODO: complete example. show output templates and processing in
   IAP.]

TODO.


8.4.  Traffic Time-Series per Source with Counter Distribution

   Returning to the example in Section 8.1, consider a case where
   aggregation by the maximum active timeout, here 30 minutes, is
   incompatible with the processing interval, here defined to be 5
   minutes.  For this case, flows longer than 5 minutes must have their
   counters distributed.  This example demonstrates counter distribution
   metadata export.

   [TODO: complete example. show output metadata and processing in IAP.]

TODO.



9.  Security Considerations

   [TODO]

TODO.



10.  IANA Considerations

   This document specifies the creation of twelve new IPFIX Information
   Elements in the IPFIX Information Element registry located at
   http://www.iana.org/assignments/ipfix, as defined in Section 7 above.
   IANA has assigned Information Element numbers to these Information
   Elements, and entered them into the registry.

   [NOTE for IANA: The text TBDn should be replaced with the respective
   assigned Information Element numbers where they appear in this
   document.  Note that the originalFlows Information Element has been
   assigned the number 3, as it is compatible with the corresponding
   existing (reserved) NetFlow v9 Information Element.  Other
   Information Element numbers should be assigned outside the NetFlow V9
   compatibility range, as these Information Elements are not supported
   by NetFlow V9.]


11.  Acknowledgments

   This work is materially supported by the European Union Seventh
   Framework Programme under grant agreement 257315 (DEMONS).


12.  References






Trammell, et al.        Expires December 31, 2011              [Page 29]

Internet-Draft              IPFIX Aggregation                  June 2011


12.1.  Normative References

   [RFC5101]  Claise, B., "Specification of the IP Flow Information
              Export (IPFIX) Protocol for the Exchange of IP Traffic
              Flow Information", RFC 5101, January 2008.

   [RFC5102]  Quittek, J., Bryant, S., Claise, B., Aitken, P., and J.
              Meyer, "Information Model for IP Flow Information Export",
              RFC 5102, January 2008.

12.2.  Informative References

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119, March 1997.

   [RFC3917]  Quittek, J., Zseby, T., Claise, B., and S. Zander,
              "Requirements for IP Flow Information Export (IPFIX)",
              RFC 3917, October 2004.

   [RFC5103]  Trammell, B. and E. Boschi, "Bidirectional Flow Export
              Using IP Flow Information Export (IPFIX)", RFC 5103,
              January 2008.

   [RFC5153]  Boschi, E., Mark, L., Quittek, J., Stiemerling, M., and P.
              Aitken, "IP Flow Information Export (IPFIX) Implementation
              Guidelines", RFC 5153, April 2008.

   [RFC5470]  Sadasivan, G., Brownlee, N., Claise, B., and J. Quittek,
              "Architecture for IP Flow Information Export", RFC 5470,
              March 2009.

   [RFC5472]  Zseby, T., Boschi, E., Brownlee, N., and B. Claise, "IP
              Flow Information Export (IPFIX) Applicability", RFC 5472,
              March 2009.

   [RFC5476]  Claise, B., Johnson, A., and J. Quittek, "Packet Sampling
              (PSAMP) Protocol Specifications", RFC 5476, March 2009.

   [RFC5610]  Boschi, E., Trammell, B., Mark, L., and T. Zseby,
              "Exporting Type Information for IP Flow Information Export
              (IPFIX) Information Elements", RFC 5610, July 2009.

   [RFC5655]  Trammell, B., Boschi, E., Mark, L., Zseby, T., and A.
              Wagner, "Specification of the IP Flow Information Export
              (IPFIX) File Format", RFC 5655, October 2009.

   [RFC5835]  Morton, A. and S. Van den Berghe, "Framework for Metric
              Composition", RFC 5835, April 2010.



Trammell, et al.        Expires December 31, 2011              [Page 30]

Internet-Draft              IPFIX Aggregation                  June 2011


   [RFC5982]  Kobayashi, A. and B. Claise, "IP Flow Information Export
              (IPFIX) Mediation: Problem Statement", RFC 5982,
              August 2010.

   [I-D.ietf-ipfix-anon]
              Boschi, E. and B. Trammell, "IP Flow Anonymization
              Support", draft-ietf-ipfix-anon-06 (work in progress),
              January 2011.

   [I-D.ietf-ipfix-mediators-framework]
              Kobayashi, A., Claise, B., Muenz, G., and K. Ishibashi,
              "IPFIX Mediation: Framework",
              draft-ietf-ipfix-mediators-framework-09 (work in
              progress), October 2010.

   [I-D.claise-ipfix-mediation-protocol]
              Claise, B., "Specification of the Protocol for IPFIX
              Mediations", draft-claise-ipfix-mediation-protocol-03
              (work in progress), February 2011.

   [I-D.trammell-ipfix-ie-doctors]
              Trammell, B. and B. Claise, "Guidelines for Authors and
              Reviewers of IPFIX Information Elements",
              draft-trammell-ipfix-ie-doctors-02 (work in progress),
              June 2011.

   [I-D.ietf-ipfix-configuration-model]
              Muenz, G., Claise, B., and P. Aitken, "Configuration Data
              Model for IPFIX and PSAMP",
              draft-ietf-ipfix-configuration-model-09 (work in
              progress), March 2011.

   [I-D.ietf-ipfix-flow-selection-tech]
              D'Antonio, S., Zseby, T., Henke, C., and L. Peluso, "Flow
              Selection Techniques",
              draft-ietf-ipfix-flow-selection-tech-06 (work in
              progress), May 2011.














Trammell, et al.        Expires December 31, 2011              [Page 31]

Internet-Draft              IPFIX Aggregation                  June 2011


Authors' Addresses

   Brian Trammell
   Swiss Federal Institute of Technology Zurich
   Gloriastrasse 35
   8092 Zurich
   Switzerland

   Phone: +41 44 632 70 13
   Email: trammell@tik.ee.ethz.ch


   Elisa Boschi
   Swiss Federal Institute of Technology Zurich
   Gloriastrasse 35
   8092 Zurich
   Switzerland

   Email: boschie@tik.ee.ethz.ch


   Arno Wagner
   Consecom AG
   Bleicherweg 64a
   8002 Zurich
   Switzerland

   Email: arno@wagner.name


   Benoit Claise
   Cisco Systems, Inc.
   De Kleetlaan 6a b1
   1831 Diagem

In existing RFCs, this is "Diegem 1813".


   Belgium

   Phone: +32 2 704 5622
   Email: bclaise@cisco.com













Trammell, et al.        Expires December 31, 2011              [Page 32]
--------------020106010107040308030706-- From paitken@cisco.com Sat Jun 16 07:56:28 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 81DD021F8564 for ; Sat, 16 Jun 2012 07:56:28 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -8.098 X-Spam-Level: X-Spam-Status: No, score=-8.098 tagged_above=-999 required=5 tests=[AWL=2.501, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TZX7qNpB1lkh for ; Sat, 16 Jun 2012 07:56:27 -0700 (PDT) Received: from ams-iport-1.cisco.com (ams-iport-1.cisco.com [144.254.224.140]) by ietfa.amsl.com (Postfix) with ESMTP id 8D51721F84B4 for ; Sat, 16 Jun 2012 07:56:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=paitken@cisco.com; l=2419; q=dns/txt; s=iport; t=1339858587; x=1341068187; h=message-id:date:from:mime-version:to:subject: content-transfer-encoding; bh=P7VTwC95wGvtfbWusoSwSWd870OVNJ6UmoAhNy/7AEQ=; b=EVuAT2tVLSj8fgvT69vGx/OAJtJPpp1pa9OnDPMLr/b6Vr5vDfD8JJ6i WDMzCxSWN+OWHSdOjmCWq+/WqcDck80tYvJygeqZ9gpsl9sLgIY+l/mGD EoDgNOPPQkH+6CDgwvbXRLbRAo868mCdZ9LgZ6I4K8XgvBKD8Q9VVIiye I=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Av0EAGae3E+Q/khR/2dsb2JhbABFtVOBB4IxASUzDT0WGAMCAQIBPwwBDAgBAR6HaZkEn0CRcwOVJIVUiEOBZoJh X-IronPort-AV: E=Sophos;i="4.75,783,1330905600"; d="scan'208";a="139804509" Received: from ams-core-1.cisco.com ([144.254.72.81]) by ams-iport-1.cisco.com with ESMTP; 16 Jun 2012 14:56:10 +0000 Received: from [10.55.93.37] (dhcp-10-55-93-37.cisco.com [10.55.93.37]) by ams-core-1.cisco.com (8.14.5/8.14.5) with ESMTP id q5GEuAp7026829; Sat, 16 Jun 2012 14:56:10 GMT Message-ID: <4FDC9E8B.1070800@cisco.com> Date: Sat, 16 Jun 2012 15:56:11 +0100 From: Paul Aitken User-Agent: Mozilla/5.0 (X11; Linux i686; rv:12.0) Gecko/20120430 Thunderbird/12.0.1 MIME-Version: 1.0 To: "ipfix-chairs@tools.ietf.org" , ipfix-ads@tools.ietf.org, IETF IPFIX Working Group Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: [IPFIX] new monitoring interval information elements X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 16 Jun 2012 14:56:28 -0000 Dear IPFIX experts, FYI, I'm requesting two new IPFIX Information Elements from IANA to record the start and end of a monitoring interval, which is a period of time during which the Metering Process is running. The existing IPFIX Information Elements provide flow based timestamps, system init, and observation times. There's no indication of the Metering Process, whichmay be running for a period of time during which no flows of interest or relevanceare observed. In fact, many existing elements are described as "The total number of X since the Metering Process (re-)initialization". However, there's currently no indication of when that (re-)initialization was. These new Information Elements enable an IPFIX device to indicate when the Metering Process started and ended. These new elements are particularly useful when the Metering Process is windowed - ie, it runs for a fixed interval, exporting all its observations only at the end of that interval, before optionally starting over. With these new Information Elements, an IPFIX device can readily indicate the start (and end) of each interval. Our present need is only for milliseconds resolution. At this time we don't need the equivalent seconds, microseconds, or nanoseconds elements, although these could be created in future if needs be. Also note the subtle distinction between "monitoring" and "metering": a device could be monitoring traffic for some time, without detecting anything of interest or relevance to be metered. The metering interval could almost be expressed by minFlowStart* to maxFlowEnd*. We've chosen to name these elements "monitoring interval" rather than "metering interval" to clearly express this difference. Name: monitoringIntervalStartMilliSeconds Data Type: dateTimeMilliseconds Sematics: - Description: The absolute timestamp at which the monitoring interval started. A Monitoring interval is the period of time during which the Metering Process is running. Units: milliseconds Name: monitoringIntervalEndMilliSeconds Data Type: dateTimeMilliseconds Sematics: - Description: The absolute timestamp at which the monitoring interval ended. A Monitoring interval is the period of time during which the Metering Process is running. Units: milliseconds Thanks, P. From muenz@net.in.tum.de Sun Jun 17 10:17:51 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3711C21F864A for ; Sun, 17 Jun 2012 10:17:51 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -5.949 X-Spam-Level: X-Spam-Status: No, score=-5.949 tagged_above=-999 required=5 tests=[AWL=0.300, BAYES_00=-2.599, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TaKBjtC-MOPc for ; Sun, 17 Jun 2012 10:17:50 -0700 (PDT) Received: from mail-out1.informatik.tu-muenchen.de (mail-out1.informatik.tu-muenchen.de [131.159.0.8]) by ietfa.amsl.com (Postfix) with ESMTP id 2289721F8634 for ; Sun, 17 Jun 2012 10:17:49 -0700 (PDT) Received: from [192.168.2.34] (e181134053.adsl.alicedsl.de [85.181.134.53]) by mail.net.in.tum.de (Postfix) with ESMTPSA id CF06F201EE22; Sun, 17 Jun 2012 19:17:47 +0200 (CEST) Message-ID: <4FDE112B.40706@net.in.tum.de> Date: Sun, 17 Jun 2012 19:17:31 +0200 From: Gerhard Muenz User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:12.0) Gecko/20120428 Thunderbird/12.0.1 MIME-Version: 1.0 To: Paul Aitken References: <4FDC9E8B.1070800@cisco.com> In-Reply-To: <4FDC9E8B.1070800@cisco.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: ipfix-ads@tools.ietf.org, IETF IPFIX Working Group , "ipfix-chairs@tools.ietf.org" Subject: Re: [IPFIX] new monitoring interval information elements X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 17 Jun 2012 17:17:51 -0000 Paul, Please check if the proposed IEs are compatible with ipfixMeteringProcessCacheDiscontinuityTime in IPFIX-MIB and cacheDiscontinuityTime in IPFIX-CONFIG. Parameters which exist in IPFIX-MIB, IPFIX-CONFIG and as IE should follow the same naming scheme and semantic, if possible. Thanks, Gerhard On 16.06.2012 16:56, Paul Aitken wrote: > Dear IPFIX experts, > > FYI, I'm requesting two new IPFIX Information Elements from IANA to > record the start and end of a monitoring interval, which is a period of > time during which the Metering Process is running. > > The existing IPFIX Information Elements provide flow based timestamps, > system init, and observation times. There's no indication of the > Metering Process, whichmay be running for a period of time during which > no flows of interest or relevanceare observed. > > In fact, many existing elements are described as "The total number of X > since the Metering Process (re-)initialization". However, there's > currently no indication of when that (re-)initialization was. These new > Information Elements enable an IPFIX device to indicate when the > Metering Process started and ended. > > These new elements are particularly useful when the Metering Process is > windowed - ie, it runs for a fixed interval, exporting all its > observations only at the end of that interval, before optionally > starting over. With these new Information Elements, an IPFIX device can > readily indicate the start (and end) of each interval. > > Our present need is only for milliseconds resolution. At this time we > don't need the equivalent seconds, microseconds, or nanoseconds > elements, although these could be created in future if needs be. > > Also note the subtle distinction between "monitoring" and "metering": a > device could be monitoring traffic for some time, without detecting > anything of interest or relevance to be metered. The metering interval > could almost be expressed by minFlowStart* to maxFlowEnd*. We've chosen > to name these elements "monitoring interval" rather than "metering > interval" to clearly express this difference. > > > Name: monitoringIntervalStartMilliSeconds > Data Type: dateTimeMilliseconds > Sematics: - > Description: The absolute timestamp at which the monitoring interval > started. > A Monitoring interval is the period of time during which > the Metering Process is running. > Units: milliseconds > > > Name: monitoringIntervalEndMilliSeconds > Data Type: dateTimeMilliseconds > Sematics: - > Description: The absolute timestamp at which the monitoring interval ended. > A Monitoring interval is the period of time during which > the Metering Process is running. > Units: milliseconds > > > Thanks, > P. > > _______________________________________________ > IPFIX mailing list > IPFIX@ietf.org > https://www.ietf.org/mailman/listinfo/ipfix From n.brownlee@auckland.ac.nz Mon Jun 18 16:04:38 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3050911E80CB for ; Mon, 18 Jun 2012 16:04:38 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.599 X-Spam-Level: X-Spam-Status: No, score=-106.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5jhhPk5oKU6s for ; Mon, 18 Jun 2012 16:04:33 -0700 (PDT) Received: from mx2.auckland.ac.nz (mx2.auckland.ac.nz [130.216.12.44]) by ietfa.amsl.com (Postfix) with ESMTP id 14DFB11E80A0 for ; Mon, 18 Jun 2012 16:04:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=auckland.ac.nz; i=n.brownlee@auckland.ac.nz; q=dns/txt; s=uoa; t=1340060673; x=1371596673; h=message-id:date:from:mime-version:to:cc:subject: references:in-reply-to:content-transfer-encoding; bh=bzOkGqnPWN/jfGgDGuyi2pK/RMR731NksknF+4QMeFQ=; b=adcbZoeDEr5hx9cHzk7d/UpF7FgpdWmttfLQqm/2gtH4DPHN5hEJqjwl XqHvaUG3j8POk+M1P5+yNqWnq6G7GHp92EiA65sy7rpP3ZAXcFTW0glAT 01Idq9s9Dz+ogtZWwYuabX2atEGTdzrtJQCWHYxjiL4qt5lBWE0z3XARt 0=; X-IronPort-AV: E=Sophos;i="4.77,433,1336305600"; d="scan'208";a="128386890" X-Ironport-HAT: UNIVERSITY - $RELAY-THROTTLE X-Ironport-Source: 130.216.38.131 - Outgoing - Outgoing-SSL Received: from nevil-laptop1.sfac.auckland.ac.nz (HELO [130.216.38.131]) ([130.216.38.131]) by mx2-int.auckland.ac.nz with ESMTP; 19 Jun 2012 11:04:31 +1200 Message-ID: <4FDFB3FE.1040300@auckland.ac.nz> Date: Tue, 19 Jun 2012 11:04:30 +1200 From: Nevil Brownlee User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:12.0) Gecko/20120428 Thunderbird/12.0.1 MIME-Version: 1.0 To: iana-prot-param-comment@iana.org References: <201206111234.q5BCYFoR023202@smtp1.lax.icann.org> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: IPFIX Working Group Subject: Re: [IPFIX] [IANA #579421] General Request for Assignment (ipfix) X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Jun 2012 23:04:38 -0000 Hi Amanda: Yes, these two new IEs are OK, they'll be IEs 359 and 360. The draft given as their description is an (AD-sponsored) Individual Submission, so it will become an RFC, but right now it's IESG state is I-D Exists. You could use the I-D as a reference meanwhile, but that will need to be changed once the RFC is published. Cheers, Nevil On 12/06/12 8:50 AM, Amanda Baber via RT wrote: > Hi Nevil, > > This is the first of two new requests from Paul. > > thanks, > > Amanda Baber > IANA > > === > > Contact Name: > Paul Aitken > > Contact Email: > paitken@cisco.com > > Type of Assignment: > IPFIX Information Elements > > Registry: > http://www.iana.org/assignments/ipfix/ipfix.xml#ipfix-information-elements > > Description: > http://tools.ietf.org/id/draft-johnson-ipfix-mib-variable-export-04.txt > > Additional Info: > Please allocate new IPFIX Information Elements for "MIB Object > Identifier Marker" and "MIB Instance Identifier" specified in section > 11.3 of this draft. -- --------------------------------------------------------------------- Nevil Brownlee Computer Science Department | ITS Phone: +64 9 373 7599 x88941 The University of Auckland FAX: +64 9 373 7453 Private Bag 92019, Auckland 1142, New Zealand From n.brownlee@auckland.ac.nz Mon Jun 18 16:08:14 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D34F911E80D0 for ; Mon, 18 Jun 2012 16:08:14 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.599 X-Spam-Level: X-Spam-Status: No, score=-106.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id k7-M8JfkEcbj for ; Mon, 18 Jun 2012 16:08:14 -0700 (PDT) Received: from mx2.auckland.ac.nz (mx2.auckland.ac.nz [130.216.12.44]) by ietfa.amsl.com (Postfix) with ESMTP id A3FF011E80A0 for ; Mon, 18 Jun 2012 16:08:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=auckland.ac.nz; i=n.brownlee@auckland.ac.nz; q=dns/txt; s=uoa; t=1340060894; x=1371596894; h=message-id:date:from:mime-version:to:cc:subject: references:in-reply-to:content-transfer-encoding; bh=Ic3xJR8mROOYRwpQpiToE2yx5j84PdZDkcetFB7TYCs=; b=K91/L0iacNkLLINPFUAXAoPmrobME1tY4n9qli5oMJ3nCenjSzV0UC0k C12A9iswc9k6sXlGtfRTCSXq32N9ft0Tl4fZ46sWiKgwoofVBO5LCvZE9 KRkUi3EyAYhHyavbbq2/7oa10tS7T1lfQZYy+ToJRhEwdhWbh2GDrvKKu A=; X-IronPort-AV: E=Sophos;i="4.77,433,1336305600"; d="scan'208";a="128387937" X-Ironport-HAT: UNIVERSITY - $RELAY-THROTTLE X-Ironport-Source: 130.216.38.131 - Outgoing - Outgoing-SSL Received: from nevil-laptop1.sfac.auckland.ac.nz (HELO [130.216.38.131]) ([130.216.38.131]) by mx2-int.auckland.ac.nz with ESMTP; 19 Jun 2012 11:08:13 +1200 Message-ID: <4FDFB4DC.2010106@auckland.ac.nz> Date: Tue, 19 Jun 2012 11:08:12 +1200 From: Nevil Brownlee User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:12.0) Gecko/20120428 Thunderbird/12.0.1 MIME-Version: 1.0 To: iana-prot-param-comment@iana.org References: <201206111440.q5BEe6ps030247@smtp1.lax.icann.org> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: IPFIX Working Group Subject: Re: [IPFIX] [IANA #579482] General Request for Assignment (ipfix) X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Jun 2012 23:08:15 -0000 Hi again Amanda: These four have been discussed on the IPFIX list, they're OK as submitted. They'll be 361 portRangeStart 362 portRangeEnd 363 portRangeStepSize 364 portRangeNumPorts Cheers, Nevil On 12/06/12 8:51 AM, Amanda Baber via RT wrote: > Hi Nevil, > > Here's Paul's second request. > > thanks, > Amanda > > === > > Contact Name: > Paul Aitken > > Contact Email: > paitken@cisco.com > > Type of Assignment: > IPFIX Information Elements > > Registry: > http://www.iana.org/assignments/ipfix/ipfix.xml#ipfix-information-elements > > Description: > Please allocate the four new fields specified below, which may be used > in various combinations to specify port ranges in IPFIX. > > Additional Info: > Name: portRangeStart > Type: unsigned16 > Semantics: identifier > Description: The port number identifying the start of a range of ports. > A value of zero indicates that the range start is not specified, ie the > range is defined in some other way. > Additional information on defined TCP port numbers can be found at [IANA > registry port-numbers]. > > Name: portRangeEnd > Type: unsigned16 > Semantics: identifier > Description: The port number identifying the end of a range of ports. > A value of zero indicates that the range end is not specified, ie the > range is defined in some other way. > Additional information on defined TCP port numbers can be found at [IANA > registry port-numbers]. > > Name: portRangeStepSize > Type: unsigned16 > Semantics: identifier > Description: The step size in a port range. The default step size is 1, > which indicates contiguous ports. > A value of zero indicates that the step size is not specified, ie the > range is defined in some other way. > > Name: portRangeNumPorts > Type: unsigned16 > Semantics: identifier > Description: The number of ports in a port range. > A value of zero indicates that the number of ports is not specified, ie > the range is defined in some other way. -- --------------------------------------------------------------------- Nevil Brownlee Computer Science Department | ITS Phone: +64 9 373 7599 x88941 The University of Auckland FAX: +64 9 373 7453 Private Bag 92019, Auckland 1142, New Zealand From n.brownlee@auckland.ac.nz Mon Jun 18 16:25:50 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 84C3211E80DB for ; Mon, 18 Jun 2012 16:25:50 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.599 X-Spam-Level: X-Spam-Status: No, score=-106.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FTYEhRC2kOdR for ; Mon, 18 Jun 2012 16:25:49 -0700 (PDT) Received: from mx2.auckland.ac.nz (mx2.auckland.ac.nz [130.216.12.44]) by ietfa.amsl.com (Postfix) with ESMTP id 78EC711E80A0 for ; Mon, 18 Jun 2012 16:25:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=auckland.ac.nz; i=n.brownlee@auckland.ac.nz; q=dns/txt; s=uoa; t=1340061950; x=1371597950; h=message-id:date:from:mime-version:to:cc:subject: references:in-reply-to:content-transfer-encoding; bh=LQAlKR+XF0/BhTlusKFCKurvKdxSwb0+zqiMNHrfqK8=; b=t0DSWlEjQXqLGZaLLpgRrUbwtFp03O7Zqfa8FjLvWASIZSZhizZx4mL3 QYJq8cMREXlUd3JFQkvTb7U6l1UugrmfJhrckKqV8tHmp0labJ5WVn2As RFdnlP4VsXDs/FcedTGSSNEn6QG/vdvtm6u31Go5y/VpNzT2gUxYXHxqp w=; X-IronPort-AV: E=Sophos;i="4.77,433,1336305600"; d="scan'208";a="128392289" X-Ironport-HAT: UNIVERSITY - $RELAY-THROTTLE X-Ironport-Source: 130.216.38.131 - Outgoing - Outgoing-SSL Received: from nevil-laptop1.sfac.auckland.ac.nz (HELO [130.216.38.131]) ([130.216.38.131]) by mx2-int.auckland.ac.nz with ESMTP; 19 Jun 2012 11:25:49 +1200 Message-ID: <4FDFB8FC.6060408@auckland.ac.nz> Date: Tue, 19 Jun 2012 11:25:48 +1200 From: Nevil Brownlee User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:12.0) Gecko/20120428 Thunderbird/12.0.1 MIME-Version: 1.0 To: iana-prot-param-comment@iana.org References: <201206111234.q5BCYFoR023202@smtp1.lax.icann.org> <4FDFB3FE.1040300@auckland.ac.nz> In-Reply-To: <4FDFB3FE.1040300@auckland.ac.nz> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: IPFIX Working Group Subject: Re: [IPFIX] [IANA #579421] General Request for Assignment (ipfix) X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Jun 2012 23:25:50 -0000 Hi again Amanda: Oh, oh, I had confused this draft with another one. draft-johnson-ipfix-mib-variable-export is, of course, an IPFIX WG draft. Sorry to be confusing, Nevil On 19/06/12 11:04 AM, Nevil Brownlee wrote: > > Hi Amanda: > > Yes, these two new IEs are OK, they'll be IEs 359 and 360. > > The draft given as their description is an (AD-sponsored) > Individual Submission, so it will become an RFC, but right > now it's IESG state is I-D Exists. > > You could use the I-D as a reference meanwhile, but that will > need to be changed once the RFC is published. > > Cheers, Nevil > > > On 12/06/12 8:50 AM, Amanda Baber via RT wrote: >> Hi Nevil, >> >> This is the first of two new requests from Paul. >> >> thanks, >> >> Amanda Baber >> IANA >> >> === >> >> Contact Name: >> Paul Aitken >> >> Contact Email: >> paitken@cisco.com >> >> Type of Assignment: >> IPFIX Information Elements >> >> Registry: >> http://www.iana.org/assignments/ipfix/ipfix.xml#ipfix-information-elements >> >> >> Description: >> http://tools.ietf.org/id/draft-johnson-ipfix-mib-variable-export-04.txt >> >> Additional Info: >> Please allocate new IPFIX Information Elements for "MIB Object >> Identifier Marker" and "MIB Instance Identifier" specified in section >> 11.3 of this draft. > > -- --------------------------------------------------------------------- Nevil Brownlee Computer Science Department | ITS Phone: +64 9 373 7599 x88941 The University of Auckland FAX: +64 9 373 7453 Private Bag 92019, Auckland 1142, New Zealand From paitken@cisco.com Mon Jun 25 04:53:04 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id ED00321F8472 for ; Mon, 25 Jun 2012 04:53:04 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -9.349 X-Spam-Level: X-Spam-Status: No, score=-9.349 tagged_above=-999 required=5 tests=[AWL=1.250, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id AA-oPK0kcUYg for ; Mon, 25 Jun 2012 04:53:00 -0700 (PDT) Received: from ams-iport-3.cisco.com (ams-iport-3.cisco.com [144.254.224.146]) by ietfa.amsl.com (Postfix) with ESMTP id 827B621F841B for ; Mon, 25 Jun 2012 04:53:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=paitken@cisco.com; l=682; q=dns/txt; s=iport; t=1340625180; x=1341834780; h=message-id:date:from:mime-version:to:subject:references: in-reply-to:content-transfer-encoding; bh=jTGEV49+R3WMrAfqIstrliU+7BUaApjGEdVNf9pVFrM=; b=kIKlp3B8XgJvRyQiXBtKJE3DpeyxCf2wUMuyDEMJiwBQSF45MhCmHBlx 9Uz/ObjCh9iAQH/VXKqsvWHkUjq2tJ3MShWtcJ3pcsvuAb8OJtf9Bh6lQ 0kyINwZfhOwkkwjc3hVTShVRyvvSzwCAiAXiqd2hg+jlaSJ6B3hVOgMzT o=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Av8EAOBP6E+Q/khN/2dsb2JhbABEhVqwRoEHghkBAQQSARAVQBElAgUWCwICCQMCAQIBRQYNCAEBHodpmQ6NGZIhgSCBIY1igRIDlS6FVohFgWaCYA X-IronPort-AV: E=Sophos;i="4.77,471,1336348800"; d="scan'208";a="6164300" Received: from ams-core-4.cisco.com ([144.254.72.77]) by ams-iport-3.cisco.com with ESMTP; 25 Jun 2012 11:52:56 +0000 Received: from [10.55.88.94] (dhcp-10-55-88-94.cisco.com [10.55.88.94]) by ams-core-4.cisco.com (8.14.5/8.14.5) with ESMTP id q5PBqufA029602 for ; Mon, 25 Jun 2012 11:52:56 GMT Message-ID: <4FE8511B.1030901@cisco.com> Date: Mon, 25 Jun 2012 12:52:59 +0100 From: Paul Aitken User-Agent: Mozilla/5.0 (X11; Linux i686; rv:12.0) Gecko/20120430 Thunderbird/12.0.1 MIME-Version: 1.0 To: IETF IPFIX Working Group References: In-Reply-To: X-Forwarded-Message-Id: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Subject: [IPFIX] new IPFIX information element, "observationDomainName" X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Jun 2012 11:53:05 -0000 Dear all, FYI I've requested a new IPFIX Information Element from IANA, "observationDomainName". This would primarily be used in an option table (mapping a name to each observationDomainId), which allows exporters to give a descriptive name to each observation domain. Collectors can then display the name to help users interpret data in a more meaningful way. Thanks, P. Request: New field, observationDomainName, complementing the existing observationDomainId (#149). Name: observationDomainName Type: string Semantic: - Description: The name of an observation domain identified by an observationDomainId. References: See observationDomainId #149. From trammell@tik.ee.ethz.ch Mon Jun 25 05:28:21 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D6A2C21F8484 for ; Mon, 25 Jun 2012 05:28:21 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.429 X-Spam-Level: X-Spam-Status: No, score=-4.429 tagged_above=-999 required=5 tests=[AWL=-1.826, BAYES_50=0.001, MIME_QP_LONG_LINE=1.396, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vJ1BjeJ1Ol8Y for ; Mon, 25 Jun 2012 05:28:19 -0700 (PDT) Received: from smtp.ee.ethz.ch (smtp.ee.ethz.ch [129.132.2.219]) by ietfa.amsl.com (Postfix) with ESMTP id 04A3921F847C for ; Mon, 25 Jun 2012 05:28:18 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by smtp.ee.ethz.ch (Postfix) with ESMTP id 5F1E8D9307; Mon, 25 Jun 2012 14:28:17 +0200 (MEST) X-Virus-Scanned: by amavisd-new on smtp.ee.ethz.ch Received: from smtp.ee.ethz.ch ([127.0.0.1]) by localhost (.ee.ethz.ch [127.0.0.1]) (amavisd-new, port 10024) with LMTP id yn1hRlI0h7-P; Mon, 25 Jun 2012 14:28:17 +0200 (MEST) Received: from pb-10243.ethz.ch (pb-10243.ethz.ch [82.130.102.152]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: briant) by smtp.ee.ethz.ch (Postfix) with ESMTPSA id 12D74D9304; Mon, 25 Jun 2012 14:28:17 +0200 (MEST) Mime-Version: 1.0 (Apple Message framework v1278) Content-Type: text/plain; charset=iso-8859-1 From: Brian Trammell In-Reply-To: <4FDBAA42.60003@cisco.com> Date: Mon, 25 Jun 2012 14:28:16 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <84E8C12B-C43B-479A-B97E-3F689A93B964@tik.ee.ethz.ch> References: <4FDBAA42.60003@cisco.com> To: Paul Aitken X-Mailer: Apple Mail (2.1278) Cc: IETF IPFIX Working Group Subject: Re: [IPFIX] review of draft-trammell-ipfix-a9n-03 X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Jun 2012 12:28:22 -0000 Hi, Paul, Many, many thanks for your review; though it apparently applies to an = older revision of the document, most of these comments still apply. = Replies on specific points inline; points receiving no reply were either = accepted into the document without comment, or were no longer applicable = to the present revision of the document. These will be incorporated into a new ietf-04 revision of the document = to be published shortly. Again, many thanks, and best regards, Brian On Jun 15, 2012, at 11:33 PM, Paul Aitken wrote: >> 2. Terminology >>=20 >> Terms used in this document that are defined in the Terminology >> section of the IPFIX Protocol [RFC5101] document are to be >> interpreted as defined there. >>=20 >> The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL = NOT", >> "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in = this >> document are to be interpreted as described in [RFC2119]. >>=20 >> In addition, this document defines the following terms >>=20 >> Aggregated Flow: A Flow, as defined by [RFC5101], derived from a >> set of zero or more original Flows within a defined Aggregation >> Interval. The two primary differences between a Flow and an >> Aggregated Flow are (1) that the time interval of a Flow is >> generally derived from information about the timing of the = packets >> comprising the Flow, while the time interval of an Aggregated = Flow >> are generally externally imposed; and (2) that an Aggregated = Flow >> may represent zero packets (i.e., an assertion that no packets >> were seen for a given Flow Key in a given time interval). Note >>=20 >=20 > That's an oversight in 5101. We should have allowed a Flow to = represent zero or more packets. > It shouldn't be necessary to define an aggregated flow in order to = export that nothing was seen. >=20 > This should be fixed in 5101bis, and (2) removed above. Indeed, the strict reading of the definition of Flow in 5101 does not = exclude the zero-packet case ("set of packets" may be empty set) but it = is strongly implied by the example. I've applied a change to the working = version of 5101bis, and removed point 2 here. >>=20 >> partially aggregated Flow: A Flow during processing within an >> Intermediate Aggregation Process; refers to an intermediate = data >> structure during aggregation within the Intermediate = Aggregation >> Process architecture detailed in Section 4.2. >>=20 >=20 > Are "partially aggregated Flows" really necessary? We use them all over the document to explain what's going in inside an = IAP. (Same, kind of, with Contributing Flows...) >>=20 >> 3. Use Cases for IPFIX Aggregation >>=20 >> Aggregation, as a common data analysis method, has many = applications. >> When used with a regular Aggregation Interval, it generates time >> series data from a collection of Flows with discrete intervals. = Time >> series data is itself useful for a wide variety of analysis tasks, >> such as generating input for network anomaly detection systems, or >> driving visualizations of volume per time for traffic with = specific >>=20 >=20 > Fixated on time here. See earlier comments. >=20 > Specifically, aggregation does not "generates time series data". It = depends what is aggregated. >=20 > Example 8.3 is the only place which acknowledges that time is not = required. Yet time isn't special, it's just a field like any other. I have to disagree that "time isn't special": the time interval of a = flow is, in essence, a pseudo key field. The specialness of time is = really the only thing that makes aggregation hard to specify. In any = streaming analysis application -- which pretty much covers anything you = can usefully do on a Mediator -- you can't have meaningful data without = timing. At the very least, the time window from the startup of the IAP = to the present time is an implicit interval. So yeah, the document is kind of fixated on time. That's intentional. = I'd argue that aggregation that doesn't have anything to do with a time = interval is the special case, which can be modeled by discarding = timestamps (an equivalent operation to the imposition of an infinite = interval). However, it wouldn't hurt to be a little more explicit about = this in this section, and to more explicitly state that's what's going = on in the example "without timing." >=20 >> packets --+ +- IPFIX Messages -+ >> | | | >> V V V >> +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ = +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ = +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ >> | Metering Process | | Collecting Process | | File Reader | >> | | +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D+ +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ >> | | | original Flows | >> | | V V >> + - - - - - - - - -+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ >> | Intermediate Aggregation Process (IAP) | >> +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ >> | Aggregated Aggregated | >> | Flows Flows | >> V V >> +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ = +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ >> | Exporting Process | | File Writer | >> +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ = +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ >> | | >> +------------> IPFIX Messages <----------+ >>=20 >> Figure 2: Data paths through the aggregation process >>=20 >=20 > An IPFIX File Reader consumes IPFIX Files, not IPFIX Messages. > An IPFIX File Write produces IPFIX Files, not IPFIX Messages. >=20 > Surely the process for (re)generating IPFIX messages is: >=20 > IPFIX Messages -> File Writer -> IPFIX File -> File Reader -> = Exporting Process -> IPFIX Messages=20 A File is simply a collection of serialized Messages, though. But good = point, the change would make it clearer... >=20 >> 4.2. Intermediate Aggregation Process Architecture >>=20 >> Within this document, an Intermediate Aggregation Process can be = seen >> as hosting an Intermediate Aggregation Function composed of four >> types of operations on the intermediate results of aggregation, = which >>=20 >>=20 >>=20 >> Trammell, et al. Expires December 31, 2011 [Page = 8] >> Internet-Draft IPFIX Aggregation June = 2011 >>=20 >>=20 >> are called partially aggregated Flows in this document, as >> illustrated in Figure 3. >>=20 >> original Flows >>=20 >=20 > Note that here you say "original Flows" rather than contributing = Flows. See my earlier comment. When they show up at the interval distribution step, they're not = contributing yet. "Contributing Flows" only has a meaning within the = context of a specific Aggregated Flow or set thereof.=20 >>=20 >> Interval distribution is a temporal aggregation operation which >> imposes an Aggregation Interval on the partially aggregated = Flow. >> This Aggregation Interval may be regular, irregular, or derived >> from the timing of the original Flows themselves. Interval >> distribution is discussed in detail in Section 5.1. >>=20 >=20 > Must this be done first? No, see below. >> Key aggregation is a spatial aggregation operation which results = in >> the addition, modification, or deletion of Flow Key fields in = the >> partially aggregated Flows. New Flow Key fields may be derived >> from existing Flow Key fields (e.g., looking up an AS number = for >> an IP address), or "promoted" from non-Key fields (e.g., when >> aggregating Flows by packet count per Flow). Key aggregation = can >>=20 >=20 > The above sounds like any non-key can be promoted, which isn't true = and needs to be discussed (see later). >=20 > Consider, "or derived from certain specific non-Key fields" ? Okay... although I'm not convinced there exist non-promoteable fields. = Certainly some of these might not make a whole lot of sense sense, but = you can always count all incoming flows that have some property equal to = each distinct value of that property. >> also add new non-Key fields derived from Key Fields that are >> deleted during key aggregation; mainly counters of unique = reduced >> keys. Key aggregation is discussed in detail in Section 5.2. >>=20 >>=20 >>=20 >> Trammell, et al. Expires December 31, 2011 [Page = 9] >> Internet-Draft IPFIX Aggregation June = 2011 >>=20 >>=20 >> Value aggregation is a spatial aggregation operation which = results >> in the addition, modification, or deletion of non-Key fields in >> the partially aggregated Flows. These non-Key fields may be >> "demoted" from existing Key fields, or derived from existing = Key >> or non-Key fields. Value aggregation is discussed in detail in >> Section 5.3. >>=20 >> Aggregate combination combines multiple partially aggregated = Flows >> having undergone interval distribution, key aggregation, and = value >> aggregation which share Flow Keys and Aggregation Intervals = into a >> single aggregated Flow per Flow Key and Aggregation Interval. >> Aggregate combination is discussed in detail in Section 5.4. >>=20 >> The first three of these operations may be carried out any number = of >> times in any order, either on original Flows or on the results of = one >>=20 >=20 > "in any order" contradicts Figure 3, which distinctly shows that = "interval distribution" must be first. It is intended to show that at least one interval distribution step is = required, as described below. This diagram has undergone significant = changes (everyone has had something they hated about it) since the = reviewed revision. Representing the associative closure of = (interval-distribution[1..x], key-aggregation[0..x], = value-aggregation[0..x]) is difficult. Would it help in understanding if = the diagram appeared _after_ the normative text? >> of the Operations (called partially aggregated Flows), with one >=20 > "Operations" isn't defined. Either define it, or de-capitalise it. done (as opposed to Done. :) ) >> caveat. Since Flows carry their own interval data, any spatial >> aggregation operation implies a temporal aggregation operation, so = at >> least one interval distribution step, even if implicit, is = required >> by this architecture. This is shown as the first step for the = sake >> of simplicity in the diagram above. Once all aggregation = operations >> are complete, aggregate combination ensures that for a given >> Aggregation Interval, Flow Key, and Observation Domain, only one = Flow >> is produced by the Intermediate Aggregation Process. >>=20 >>=20 >> 5. IP Flow Aggregation Operations >>=20 >> As stated in Section 2, an Aggregated Flow is simply an IPFIX Flow >> generated from original Flows by an Aggregation Function. Here, = we >>=20 >> detail the operations by which this is achieved within an >> Intermediate Aggregation Process. >>=20 >> 5.1. Temporal Aggregation through Interval Distribution >>=20 >> Interval distribution imposes a time interval on the resulting >> Aggregated Flows. The selection of an interval is specific to the >> given aggregation application. Intervals may be derived from the >> original Flows themselves (e.g., an interval may be selected to = cover >> the entire interval containing the set of all Flows sharing a = given >> Key, as in Time Composition describe in Section 5.1.2) or = externally >> imposed; in the latter case the externally imposed interval may be >> regular (e.g., every five minutes) or irregular (e.g., to allow = for >> different time resolutions at different times of day, under = different >> network conditions, or indeed for different sets of original = Flows). >>=20 >=20 > How does the Aggregated Flow consumer know what the aggregation = interval was, especially in the "irregular" case? > ie, with regular intervals, a collector can simply store the received = Aggregates as eg hourly reports. >=20 > However, with irregular intervals it may have two minutes here and ten = minutes there... which may not be easy to process in a meaningful way? Indeed; for irregular intervals the post-CP processes will either need = additional information about how the intervals are distributed, or the = analysis will need to be done in a way that is interval-independent. The = point is that flows with irregular imposed intervals are still flows, = and not to constrain applications which for some reason (I would suspect = generally data reduction of "uninteresting" flows through small keys and = big intervals) need to use irregular intervals from running on an "IAP" = as defined. >> The length of the imposed interval itself has tradeoffs. Shorter >> intervals allow higher resolution aggregated data and, in = streaming >>=20 >>=20 >>=20 >> Trammell, et al. Expires December 31, 2011 [Page = 10] >> Internet-Draft IPFIX Aggregation June = 2011 >>=20 >>=20 >> applications, faster reaction time. Longer intervals lead to = greater >> data reduction and simplified counter distribution. Specifically, >>=20 >=20 > *may* lead to greater data reduction, though not necessarily. True. >> counter distribution is greatly simplified by the choice of an >> interval longer than the duration of longest original Flow, itself >> generally determined by the original Flow's Metering Process = active >> timeout; in this case an original Flow can contribute to at most = two >> Aggregated Flows, and the more complex value distribution methods >> become inapplicable. >>=20 >=20 > This only considers the Metering Process; it fails to consider any = delay in the Exporting Process. >=20 > eg, consider and implementation with 64K cache entries which are = checked for exporting at a rate of 1000 entries/second. It will take > 1 = minute to check and export the entire cache. And the timestamps are put on the flows by the EP as opposed to the MP? > Again, consider an implementation which aims to maximise the size of = each export packet, building it up a record at a time as records expire = from the cache, until the export packet is quite full. If records expire = slowly then the mechanism may introduce a significant delay. Again, this will affect the _record_ timestamps? If this is the case, = then the guidance would be to choose intervals larger than the active = timeout plus the MP-EP latency, but this seems to be limited to specific = implementations which separate timing in this way. > In effect both of these delays cause flows to appear in later = intervals than may be expected. eg, in Figure 4, "Flow A" may appear in = interval 1 or interval 2, rather than interval 0. >=20 >=20 >> | | | | >> | |<--Flow A-->| | | | >> | |<--Flow B-->| | | >> | |<-------------Flow C-------------->| | >> | | | | >> | interval 0 | interval 1 | interval 2 | >>=20 >> Figure 4: Illustration of interval distribution >=20 >> from correlation of the original Flow information with some = external >> source. There are two basic operations here. First, Aggregated = Flow >> Keys may be derived directly from original Flow Keys through >> reduction, or the dropping of fields or precision in the original >> Flow Keys. Second, an Aggregated Flow Key may be derived through >> replacement, e.g. by removing one or more fields from the original >> Flow and replacing them with a fields derived from the removed >> fields. Replacement may refer to external information (e.g., IP = to >> AS number mappings). Replacement need not replace only key = fields. >> For example, consider an application which aggregates flows by = packet >> count (i.e., generating an Aggregated Flow for all one-packet = Flows, >> one for all two-packet Flows, and so on). This application would >> promote the packet count to a Flow Key field. >>=20 >=20 > Say more about promotion, ie it can only be done on non-key fields = which represent a single unique value. eg, counters, time. > If a NK field might be any value from a set of >1 values (eg, an IP = address, port number, AS, TOS) then it's not suitable for promotion to = Key, because there's no way of knowing which other values were or were = not observed, and therefore no way of dividing the flow amongst those = values. I'm not sure I follow you here. Why does the potential set of values = (and its difference to the actual set) affect the promotion ability? Can = you give a more detailed example? >>=20 >> Trammell, et al. Expires December 31, 2011 [Page = 13] >> Internet-Draft IPFIX Aggregation June = 2011 >>=20 >>=20 >> Key aggregation may also result in the addition of new non-Key = fields >> to the Aggregated Flows, namely original Flow counters and unique >> reduced key counters; these are treated in more detail in >> Section 5.2.2 and Section 5.2.1, respectively. >>=20 >=20 > I'm sure other non-keys could be derived too, though a good example = doesn't immediately spring to mind. They're also not handled in the draft; we originally thought about going = further down this road but decided to keep the list short and general in = the interests of (1) brevity and (2) IE number space conservation. >> In any key aggregation operation, reduction and/or replacement may = be >> applied any number of times in any order. Which of these = operations >> are supported by a given implementation is implementation- and >> application-dependent. Key aggregation may aggregate original = Flows >> with different sets of Flow Key fields; only the Flow Keys of the >> resulting Aggregated Flows of any given Key aggregation operation >> need contain the same set of fields. >>=20 >=20 > I don't understand what "only ... fields" is saying. The Key Fields defining the input to the operation can be heterogeneous, = because it's only the outputs that need to match; since these operations = can be arbitrarily split and combined, it's only the final key fields = that need to match. This is a confusing sentence, though, and adds = little, so I've decided to cut it. >> 5.4. Aggregation Combination >>=20 >> Interval distribution and key aggregation together may generate >> multiple partially aggregated Flows covering the same time = interval >> with the same Flow Key. The process of combining these partially >> aggregated Flows into a single Aggregated Flow is called = aggregation >> combination. In general, non-Key values from multiple = contributing >> Flows are combined using the same operation by which values are >> combined from packets to form Flows for each Information Element. >> Counters are summed, averages are averaged, flags are unioned, and = so >> on. >>=20 >=20 > Although this is common knowledge for us, is it actually specified = anywhere in IPFIX? >=20 > ie, should [IANA-IPFIX] list the relevant aggregation operation for = each field? eg, min, max, sum, union, none? We thought about this, long ago; however, (1) there may be some = applications wherein non-natural operations on aggregated values would = be useful and (2) specifying in the registry that only one operation was = permissible gets us close to specifying implementation details of the = IAP, which we want to avoid... >> 7. Export of Aggregated IP Flows using IPFIX >>=20 >> In general, Aggregated Flows are exported in IPFIX as any normal >> Flow. However, certain aspects of Aggregated Flow export benefit >> from additional guidelines, or new Information Elements to = represent >>=20 >>=20 >>=20 >> Trammell, et al. Expires December 31, 2011 [Page = 17] >> Internet-Draft IPFIX Aggregation June = 2011 >>=20 >>=20 >> aggregation metadata or information generated during aggregation. >> These are detailed in the following subsections. >>=20 >=20 > Have the new IEs been implemented in released code? > We should avoid defining theoretical new IEs (ie, those which nobody = has actually implemented, or plans to implement). > "Rough consensus and _running code_" :-) These fields are often used as output of rwuniq, for one; when I left = CERT (admittedly, a long time ago now) there were plans IIRC to = introduce IPFIX-based intermediate output for the SiLK analysis tools. I = don't know whether these are still current. >>=20 >> 7.2.4. originalFlows >>=20 >=20 > For consistency with the above IEs, name this = "originalFlowsContributing" ? It was pointed out that this is probably covered by deltaFlowCount (IE = 3); the -ietf rev of the document was changed accordingly. (However, = float64 is not the correct type; have corrected this... >> Description: The conservative count of original Flows = contributing >> to this Aggregated Flow; may be distributed via any of the = methods >> described in Section 5.1.1. >>=20 >> Abstract Data Type: float64 >>=20 >> ElementId: 3 >>=20 >> Status: Current >>=20 >>=20 >>=20 >> 7.4.1. Aggregate Counter Distribution Options Template >>=20 >> This Options Template defines the Aggregate Counter Distribution >> Record, which allows the binding of a value distribution method to = a >> Template ID. This is used to signal to the Collecting Process how >> the counters were distributed. The fields are as below: >>=20 >> = +-------------------------+-----------------------------------------+ >> | IE | Description = | >> = +-------------------------+-----------------------------------------+ >> | templateId [scope] | The Template ID of the Template = | >> | | defining the Aggregated Flows to which = | >> | | this distribution option applies. = This | >> | | Information Element MUST be defined as = | >> | | a Scope Field. = | >> | valueDistributionMethod | The method used to distribute the = | >> | | counters for the Aggregated Flows = | >> | | defined by the associated Template. = | >> = +-------------------------+-----------------------------------------+ >>=20 >=20 > Presumably the scope could be more specific, eg an = "informationElementIndex" could be included. Per-IE indexing would be done as in 5610, presumably, but it's not clear = that it makes sense to distribute the values of separate fields = separately; see the next paragraph. >=20 >> 7.4.2. valueDistributionMethod Information Element >>=20 >> Description: A description of the method used to distribute the >> counters from contributing Flows into the Aggregated Flow = records >> described by an associated Template. The method is deemed to >> apply to all the non-key Information Elements in the referenced >> Template for which value distribution is a valid operation; if = the >>=20 >=20 > No; it applies to the specified scope, which happens to be a = templateId in this case - but may be something else in other uses. Point. However, it is not at _all_ clear that it makes any sense to = distribute one counter one way and another counter another for a given = Aggregated Flow template: when aggregating to Aggregated Flows with = uniform counter distribution, each Aggregated Flow remains independent = as a Flow; otherwise, the resulting Aggregated Flows are dependent among = the set of flows to which a given value may be distributed. > eg, vDM could be used with structured data, where the intention would = be that it applies only to the current relevant structure, and not to = all the non-key elements in some other template. >=20 >> originalFlowsInitiated and/or originalFlowsCompleted = Information >> Elements appear in the Template, they are not subject to this >> distribution method, as they each infer their own distribution >> method. The distribution methods are taken from Section 5.1.1 = and >> encoded as follows: >>=20 >=20 > Please separate the table entries below to make them easier to read. >=20 >=20 >> = +-------+-----------------------------------------------------------+ >> | Value | Description = | >> = +-------+-----------------------------------------------------------+ >>=20 >=20 > What does value =3D zero mean? Missing; will add a "less-than-default" meaning for this (as in = "explictly not distributed according to any consistent policy") >=20 >> | 1 | Start Interval: The counters for an original Flow are = | >> | | added to the counters of the appropriate Aggregated Flow = | >> | | containing the start time of the original Flow. This = | >> | | should be assumed the default if value distribution = | >> | | information is not available at a Collecting Process for = | >> | | an Aggregated Flow. = | >> | 2 | End Interval: The counters for an original Flow are = added | >> | | to the counters of the appropriate Aggregated Flow = | >> | | containing the end time of the original Flow. = | >>=20 >>=20 >>=20 >>=20 >>=20 >>=20 >> Trammell, et al. Expires December 31, 2011 [Page = 21] >> Internet-Draft IPFIX Aggregation June = 2011 >>=20 >>=20 >> | 3 | Mid Interval: The counters for an original Flow are = added | >> | | to the counters of a single appropriate Aggregated Flow = | >> | | containing some timestamp between start and end time of = | >> | | the original Flow. = | >> | 4 | Simple Uniform Distribution: Each counter for an = original | >> | | Flow is divided by the number of time intervals the = | >> | | original Flow covers (i.e., of appropriate Aggregated = | >> | | Flows sharing the same Flow Key), and this number is = | >> | | added to each corresponding counter in each Aggregated = | >> | | Flow. = | >> | 5 | Proportional Uniform Distribution: Each counter for an = | >> | | original Flow is divided by the number of time _units_ = | >> | | the original Flow covers, to derive a mean count rate. = | >> | | This mean count rate is then multiplied by the number of = | >> | | time units in the intersection of the duration of the = | >> | | original Flow and the time interval of each Aggregated = | >> | | Flow. This is like simple uniform distribution, but = | >> | | accounts for the fractional portions of a time interval = | >> | | covered by an original Flow in the first and last time = | >> | | interval. = | >> | 6 | Simulated Process: Each counter of the original Flow is = | >> | | distributed among the intervals of the Aggregated Flows = | >> | | according to some function the Aggregation Process uses = | >> | | based upon properties of Flows presumed to be like the = | >> | | original Flow. This is essentially an assertion that = the | >> | | Aggregation Process has no direct packet timing = | >> | | information but is nevertheless not using one of the = | >> | | other simpler distribution methods. The Aggregation = | >> | | Process specifically makes no assertion as to the = | >> | | correctness of the simulation. = | >> | 7 | Direct: The Aggregation Process has access to the = | >> | | original packet timings from the packets making up the = | >> | | original Flow, and uses these to distribute or = | >> | | recalculate the counters. = | >> = +-------+-----------------------------------------------------------+ >>=20 >=20 > State whether this can be extended in future, or whether a new IE = would be required. >=20 > =46rom an IE-doctors point of view, this should generally be stated = for all list-based IEs, since someone could extend the list and claim = support for this IE, while someone else claims support with the = original, more limited, list. Clearly the two "supporting" processes = would not be interoperable. >=20 > Expressed another way: if the list is extensible then a new = implementation which extends the list invalidates all existing = implementations. >=20 > However, requiring new IEs for each new value may be unreasonable, = leading to fragmentation (values 1-7 in IE#1, 8-9 in IE#2, 10-12 in = IE#3) and unnecessarily hastening IE space exhaustion. The correct way to handle this would be to create a subregistry; the set = is intended to be complete, so we didn't deem a subregistry to be = necessary.=20 >> The data records given as input to the examples in this section = are >> shown below, in the format = "flowStartMilliseconds-flowEndMilliseconds >> sourceIPv4Address:sourceTransportPort -> destinationIPv4Address: >> destinationTransportPort (protocolIdentifier) octetDeltaCount"; >> timestamps are given in H:MM:SS.sss format. >>=20 >> 9:00:00.138-9:00:00.138 192.0.2.2:47113 -> 192.0.2.131:53 (17) = 119 >> 9:00:03.246-9:00:03.246 192.0.2.2:22153 -> 192.0.2.131:53 (17) = 83 >> 9:00:00.478-9:00:03.486 192.0.2.2:52420 -> 198.51.100.2:443 (6) = 1637 >> 9:00:07.172-9:00:07.172 192.0.2.3:56047 -> 192.0.2.131:53 (17) = 111 >> 9:00:07.309-9:00:14.861 192.0.2.3:41183 -> 198.51.100.67:80 (6) = 16838 >> 9:00:03.556-9:00:19.876 192.0.2.2:17606 -> 198.51.100.68:80 (6) = 11538 >> 9:00:25.210-9:00:25.210 192.0.2.3:47113 -> 192.0.2.131:53 (17) = 119 >> 9:00:26.358-9:00:30.198 192.0.2.3:48458 -> 198.51.100.133:80 (6) = 2973 >> 9:00:29.213-9:01:00.061 192.0.2.4:61295 -> 198.51.100.2:443 (6) = 8350 >> 9:04:00.207-9:04:04.431 203.0.113.3:41256 -> 198.51.100.133:80 (6) = 778 >> 9:03:59.624-9:04:06.984 203.0.113.3:51662 -> 198.51.100.3:80 (6) = 883 >> 9:06:56.813-9:06:59.821 203.0.113.3:52572 -> 198.51.100.2:443 (6) = 1637 >> 9:06:30.565-9:07:00.261 203.0.113.3:49914 -> 197.51.100.133:80 (6) = 561 >> 9:06:55.160-9:07:05.208 192.0.2.2:50824 -> 198.51.100.2:443 (6) = 1899 >> 9:06:49.322-9:07:05.322 192.0.2.3:34597 -> 198.51.100.3:80 (6) = 1284 >> 9:07:05.849-9:07:09.625 203.0.113.3:58907 -> 198.51.100.4:80 (6) = 2670 >> 9:10:45.161-9:10:45.161 192.0.2.4:22478 -> 192.0.2.131:53 (17) = 75 >> 9:10:45.209-9:11:01.465 192.0.2.4:49513 -> 198.51.100.68:80 (6) = 3374 >> 9:10:57.094-9:11:00.614 192.0.2.4:64832 -> 198.51.100.67:80 (6) = 138 >> 9:10:59.770-9:11:02.842 192.0.2.3:60833 -> 198.51.100.69:443 (6) = 2325 >> 9:13:53.933-9:14:06.605 192.0.2.2:19638 -> 198.51.100.3:80 (6) = 2869 >> 9:13:02.864-9:14:08.720 192.0.2.3:40429 -> 198.51.100.4:80 (6) = 18289 >>=20 >> Figure 8: Input data for examples >>=20 >=20 > Consider listing the values in aligned columns without the syntax? > It fits into 76 chars with double-spaced columns, and is slightly = easier to read. > Also, good to add column titles to make the subsequent figures easier = to follow. Done. > Finally, the data are a little out of order. Although it really = doesn't matter, in-order might make the example a little easier to = follow for pedants who really want to check your data manually, like me. The out-of-order aspect is intentional; it reflects a realistic export = ordering for the input data. However, this can be handled in the = prose... >> 9:00:00.000-9:05:00.000 192.0.2.2:47113 -> 192.0.2.131:53 (17) = 119 >> 9:00:00.000-9:05:00.000 192.0.2.2:22153 -> 192.0.2.131:53 (17) = 83 >> 9:00:00.000-9:05:00.000 192.0.2.2:52420 -> 198.51.100.2:443 (6) = 1637 >> 9:00:00.000-9:05:00.000 192.0.2.3:56047 -> 192.0.2.131:53 (17) = 111 >> 9:00:00.000-9:05:00.000 192.0.2.3:41183 -> 198.51.100.67:80 (6) = 16838 >> 9:00:00.000-9:05:00.000 192.0.2.2:17606 -> 198.51.100.68:80 (6) = 11538 >> 9:00:00.000-9:05:00.000 192.0.2.3:47113 -> 192.0.2.131:53 (17) = 119 >> 9:00:00.000-9:05:00.000 192.0.2.3:48458 -> 198.51.100.133:80 (6) = 2973 >> 9:00:00.000-9:05:00.000 192.0.2.4:61295 -> 198.51.100.2:443 (6) = 8350 >> 9:00:00.000-9:05:00.000 203.0.113.3:41256 -> 198.51.100.133:80 (6) = 778 >> 9:00:00.000-9:05:00.000 203.0.113.3:51662 -> 198.51.100.3:80 (6) = 883 >> 9:05:00.000-9:10:00.000 203.0.113.3:52572 -> 198.51.100.2:443 (6) = 1637 >> 9:05:00.000-9:10:00.000 203.0.113.3:49914 -> 197.51.100.133:80 (6) = 561 >> 9:05:00.000-9:10:00.000 192.0.2.2:50824 -> 198.51.100.2:443 (6) = 1899 >> 9:05:00.000-9:10:00.000 192.0.2.3:34597 -> 198.51.100.3:80 (6) = 1284 >> 9:05:00.000-9:10:00.000 203.0.113.3:58907 -> 198.51.100.4:80 (6) = 2670 >> 9:10:00.000-9:15:00.000 192.0.2.4:22478 -> 192.0.2.131:53 (17) = 75 >> 9:10:00.000-9:15:00.000 192.0.2.4:49513 -> 198.51.100.68:80 (6) = 3374 >> 9:10:00.000-9:15:00.000 192.0.2.4:64832 -> 198.51.100.67:80 (6) = 138 >> 9:10:00.000-9:15:00.000 192.0.2.3:60833 -> 198.51.100.69:443 (6) = 2325 >> 9:10:00.000-9:15:00.000 192.0.2.2:19638 -> 198.51.100.3:80 (6) = 2869 >> 9:10:00.000-9:15:00.000 192.0.2.3:40429 -> 198.51.100.4:80 (6) = 18289 >>=20 >> Figure 11: Partially aggregated Flows: intervals imposed >>=20 >=20 > Why two timestamps? It's unclear which interval a flow at 9:05:00.000 = should be in. > Whereas listing a single timestamp (9:00, 9:05, 9:10) is less = ambiguous. Notionally, each flow contains the interval to which it is binned. >> 9:00:00.000-9:05:00.000 192.0.2.2 13377 >> 9:00:00.000-9:05:00.000 192.0.2.3 20041 >> 9:00:00.000-9:05:00.000 192.0.2.4 8350 >> 9:00:00.000-9:05:00.000 203.0.113.3 1661 >> 9:05:00.000-9:10:00.000 192.0.2.2 1899 >> 9:05:00.000-9:10:00.000 192.0.2.3 1284 >> 9:05:00.000-9:10:00.000 203.0.113.3 4868 >> 9:10:00.000-9:15:00.000 192.0.2.2 2869 >> 9:10:00.000-9:15:00.000 192.0.2.3 20594 >>=20 >=20 > 20594 should be 2325 + 18289 =3D 20614. >=20 >=20 >> 9:10:00.000-9:15:00.000 192.0.2.4 3587 >>=20 >> Figure 14: Aggregated Flows >>=20 >=20 > As a cross-check, sum(Figure_8) =3D sum(Figure_11) =3D 78550, while = sum(Figure_14) =3D 78530 - so there are 20 octets missing. >=20 Thanks for checking the math here... Should have thrown these into a = calculator but I must admit I seem to recall doing the math in my head = for these... >> 8.3. Distinct Source Count per Destination Endpoint >>=20 >> Aggregating flows by destination address and port, and counting >> distinct sources aggregated away, can be used as part of passive >> service inventory and host characterization approaches. This = example >> shows aggregation as an analysis technique, performed on source = data >> stored in an IPFIX File. As the Transport Session in this File is >> bounded, removal of all timestamp information allows summarization = of >> the entire time interval contained within the interval. Removal = of >> timing information during interval imposition is equivalent to an >> infinitely long imposed time interval. This demonstrates both how >> infinite intervals work, and how unique counters work. >>=20 >=20 > At last, an example which shows that time is _not_ the principal = factor in every aggregation - and is not even required. >=20 > Since there's no temporal aggregation here, there's no interval = distribution - which contradicts figure 3. Ah, but there is temporal aggregation - of all input data to the process = is accounted to the same interval. From paitken@cisco.com Mon Jun 25 08:39:07 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6373721F84B2 for ; Mon, 25 Jun 2012 08:39:07 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -9.765 X-Spam-Level: X-Spam-Status: No, score=-9.765 tagged_above=-999 required=5 tests=[AWL=0.833, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1kgzChanOegc for ; Mon, 25 Jun 2012 08:39:06 -0700 (PDT) Received: from ams-iport-2.cisco.com (ams-iport-2.cisco.com [144.254.224.141]) by ietfa.amsl.com (Postfix) with ESMTP id 3E79E21F84A6 for ; Mon, 25 Jun 2012 08:39:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=paitken@cisco.com; l=4879; q=dns/txt; s=iport; t=1340638742; x=1341848342; h=message-id:date:from:mime-version:to:cc:subject: references:in-reply-to:content-transfer-encoding; bh=OGV9yRa7PaW9M/o02eQhKSeYtfbZO3OPMBPsh/42FcI=; b=QOVdmAY/vSId3J1W7V9HYwRNqVwIB9ZqTjMguWCnNsJtDkXKNkJzmxCm 5J+q8feOL+cBJ337sJNhU+Bj6INGjpXRW53bWkz8HSJ4Xz37cMdzGHFgH 40huKz/DBKnf1x1EYnG/eyd5/2ZCnpz3AnMT5y+aU4ku3WQmmzTJCvQDC A=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Av8EALiF6E+Q/khR/2dsb2JhbABEti2BB4IYAQEBBAEBAQ8BJTMDCgEQCxgJFg8JAwIBAgEVMAYNAQUCAQEeh2kLmHifUwSLM4YCA5UuhVaIRYFmgmA X-IronPort-AV: E=Sophos;i="4.77,471,1336348800"; d="scan'208";a="74832912" Received: from ams-core-1.cisco.com ([144.254.72.81]) by ams-iport-2.cisco.com with ESMTP; 25 Jun 2012 15:39:00 +0000 Received: from [10.55.88.94] (dhcp-10-55-88-94.cisco.com [10.55.88.94]) by ams-core-1.cisco.com (8.14.5/8.14.5) with ESMTP id q5PFd00Q027486; Mon, 25 Jun 2012 15:39:00 GMT Message-ID: <4FE88612.1080702@cisco.com> Date: Mon, 25 Jun 2012 16:38:58 +0100 From: Paul Aitken User-Agent: Mozilla/5.0 (X11; Linux i686; rv:12.0) Gecko/20120430 Thunderbird/12.0.1 MIME-Version: 1.0 To: Gerhard Muenz References: <4FDC9E8B.1070800@cisco.com> <4FDE112B.40706@net.in.tum.de> In-Reply-To: <4FDE112B.40706@net.in.tum.de> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: ipfix-ads@tools.ietf.org, IETF IPFIX Working Group , "ipfix-chairs@tools.ietf.org" Subject: Re: [IPFIX] new monitoring interval information elements X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Jun 2012 15:39:07 -0000 Gerhard, It's not entirely clear what a "discontinuity" is since it's not clearly defined. However, inferring from section 5.8 of RFC 5815, ipfixMeteringProcessCacheDiscontinuityTime and cacheDiscontinuityTime are intended to indicate when the Metering Process was running but wasn't able to meter correctly. The new Information Elements which I'm proposing below aren't related to discontinuities. They address a different set of problems. In this case the Metering Process is functioning correctly, so there are no discontinuities. If there were, they can be expressed with ipfixMeteringProcessCacheDiscontinuityTime and cacheDiscontinuityTime. However, they are not relevant to the discussion here. Looking in IANA's IPFIX Information Element registry, we can see many Information Elements (about 15) defined as "The number of X since the Metering Process (re-)initialization" - so we need to know when that was in order to be able to use these Information Elements properly. These definitions suggest an expectation that the Metering Process will start and may subsequently restart. That could be for many reasons; there's no suggestion that it's an error. The new Information Elements proposed below serve to indicate when the Metering Process (re)started, and when it ended. As far as I know, there are currently no Information Elements for this. Reviewing RFC 5101, I noticed that the "Metering Process Statistics Option Template" contains several fields defined as, "The total number X since the Exporting Process re-initialization", so it seems that an additional Information Element is required for that too, in the case of connectionless export transports. Thanks, P. On 17/06/12 18:17, Gerhard Muenz wrote: > > Paul, > > Please check if the proposed IEs are compatible with > ipfixMeteringProcessCacheDiscontinuityTime in IPFIX-MIB and > cacheDiscontinuityTime in IPFIX-CONFIG. > > Parameters which exist in IPFIX-MIB, IPFIX-CONFIG and as IE should > follow the same naming scheme and semantic, if possible. > > Thanks, > Gerhard > > > On 16.06.2012 16:56, Paul Aitken wrote: >> Dear IPFIX experts, >> >> FYI, I'm requesting two new IPFIX Information Elements from IANA to >> record the start and end of a monitoring interval, which is a period of >> time during which the Metering Process is running. >> >> The existing IPFIX Information Elements provide flow based timestamps, >> system init, and observation times. There's no indication of the >> Metering Process, whichmay be running for a period of time during which >> no flows of interest or relevanceare observed. >> >> In fact, many existing elements are described as "The total number of X >> since the Metering Process (re-)initialization". However, there's >> currently no indication of when that (re-)initialization was. These new >> Information Elements enable an IPFIX device to indicate when the >> Metering Process started and ended. >> >> These new elements are particularly useful when the Metering Process is >> windowed - ie, it runs for a fixed interval, exporting all its >> observations only at the end of that interval, before optionally >> starting over. With these new Information Elements, an IPFIX device can >> readily indicate the start (and end) of each interval. >> >> Our present need is only for milliseconds resolution. At this time we >> don't need the equivalent seconds, microseconds, or nanoseconds >> elements, although these could be created in future if needs be. >> >> Also note the subtle distinction between "monitoring" and "metering": a >> device could be monitoring traffic for some time, without detecting >> anything of interest or relevance to be metered. The metering interval >> could almost be expressed by minFlowStart* to maxFlowEnd*. We've chosen >> to name these elements "monitoring interval" rather than "metering >> interval" to clearly express this difference. >> >> >> Name: monitoringIntervalStartMilliSeconds >> Data Type: dateTimeMilliseconds >> Sematics: - >> Description: The absolute timestamp at which the monitoring interval >> started. >> A Monitoring interval is the period of time during which >> the Metering Process is running. >> Units: milliseconds >> >> >> Name: monitoringIntervalEndMilliSeconds >> Data Type: dateTimeMilliseconds >> Sematics: - >> Description: The absolute timestamp at which the monitoring interval >> ended. >> A Monitoring interval is the period of time during which >> the Metering Process is running. >> Units: milliseconds >> >> >> Thanks, >> P. >> >> _______________________________________________ >> IPFIX mailing list >> IPFIX@ietf.org >> https://www.ietf.org/mailman/listinfo/ipfix From muenz@net.in.tum.de Mon Jun 25 09:51:03 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6E7FA21F852D for ; Mon, 25 Jun 2012 09:51:03 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.099 X-Spam-Level: X-Spam-Status: No, score=-6.099 tagged_above=-999 required=5 tests=[AWL=0.150, BAYES_00=-2.599, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id eOHS3jtcf2EM for ; Mon, 25 Jun 2012 09:51:02 -0700 (PDT) Received: from mail-out1.informatik.tu-muenchen.de (mail-out1.informatik.tu-muenchen.de [131.159.0.8]) by ietfa.amsl.com (Postfix) with ESMTP id C708021F852C for ; Mon, 25 Jun 2012 09:51:00 -0700 (PDT) Received: from [192.168.2.34] (e181142196.adsl.alicedsl.de [85.181.142.196]) by mail.net.in.tum.de (Postfix) with ESMTPSA id A2727200AA48; Mon, 25 Jun 2012 18:50:58 +0200 (CEST) Message-ID: <4FE896DE.6050908@net.in.tum.de> Date: Mon, 25 Jun 2012 18:50:38 +0200 From: Gerhard Muenz User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:13.0) Gecko/20120614 Thunderbird/13.0.1 MIME-Version: 1.0 To: Paul Aitken References: <4FDC9E8B.1070800@cisco.com> <4FDE112B.40706@net.in.tum.de> <4FE88612.1080702@cisco.com> In-Reply-To: <4FE88612.1080702@cisco.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: ipfix-ads@tools.ietf.org, IETF IPFIX Working Group , "ipfix-chairs@tools.ietf.org" Subject: Re: [IPFIX] new monitoring interval information elements X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Jun 2012 16:51:03 -0000 Paul, Ok, I see the difference to discontinuity time. Some further thoughts: The common understanding is that the "time since re-initialization" means sysUpTime. For example, have a look at the description of flowEndSysUpTime: "The relative timestamp of the last packet of this Flow. It indicates the number of milliseconds since the last (re-)initialization of the IPFIX Device (sysUpTime)." Your proposed IEs do not mention neither (re-)initialization nor sysUpTime. They rather seem to specify a time interval which is located somewhere within the sysuptime interval. Hence, the interval does not work to define the time frame of exportedMessageTotalCount, for example. You would need to define a new IE exportedMessageInMonitoringInterval. The different IE descriptions suggest that the time of re-initialization (i.e. sysuptime) can be different for Metering Process, Exporting Process etc. It seems that we need IE to export sysUpTime for each process. Thanks Gerhard On 25.06.2012 17:38, Paul Aitken wrote: > Gerhard, > > It's not entirely clear what a "discontinuity" is since it's not clearly > defined. However, inferring from section 5.8 of RFC 5815, > ipfixMeteringProcessCacheDiscontinuityTime and cacheDiscontinuityTime > are intended to indicate when the Metering Process was running but > wasn't able to meter correctly. > > The new Information Elements which I'm proposing below aren't related to > discontinuities. They address a different set of problems. In this case > the Metering Process is functioning correctly, so there are no > discontinuities. If there were, they can be expressed with > ipfixMeteringProcessCacheDiscontinuityTime and cacheDiscontinuityTime. > However, they are not relevant to the discussion here. > > Looking in IANA's IPFIX Information Element registry, we can see many > Information Elements (about 15) defined as "The number of X since the > Metering Process (re-)initialization" - so we need to know when that was > in order to be able to use these Information Elements properly. > > These definitions suggest an expectation that the Metering Process will > start and may subsequently restart. That could be for many reasons; > there's no suggestion that it's an error. > > The new Information Elements proposed below serve to indicate when the > Metering Process (re)started, and when it ended. As far as I know, there > are currently no Information Elements for this. > > Reviewing RFC 5101, I noticed that the "Metering Process Statistics > Option Template" contains several fields defined as, "The total number X > since the Exporting Process re-initialization", so it seems that an > additional Information Element is required for that too, in the case of > connectionless export transports. > > Thanks, > P. > > > On 17/06/12 18:17, Gerhard Muenz wrote: >> >> Paul, >> >> Please check if the proposed IEs are compatible with >> ipfixMeteringProcessCacheDiscontinuityTime in IPFIX-MIB and >> cacheDiscontinuityTime in IPFIX-CONFIG. >> >> Parameters which exist in IPFIX-MIB, IPFIX-CONFIG and as IE should >> follow the same naming scheme and semantic, if possible. >> >> Thanks, >> Gerhard >> >> >> On 16.06.2012 16:56, Paul Aitken wrote: >>> Dear IPFIX experts, >>> >>> FYI, I'm requesting two new IPFIX Information Elements from IANA to >>> record the start and end of a monitoring interval, which is a period of >>> time during which the Metering Process is running. >>> >>> The existing IPFIX Information Elements provide flow based timestamps, >>> system init, and observation times. There's no indication of the >>> Metering Process, whichmay be running for a period of time during which >>> no flows of interest or relevanceare observed. >>> >>> In fact, many existing elements are described as "The total number of X >>> since the Metering Process (re-)initialization". However, there's >>> currently no indication of when that (re-)initialization was. These new >>> Information Elements enable an IPFIX device to indicate when the >>> Metering Process started and ended. >>> >>> These new elements are particularly useful when the Metering Process is >>> windowed - ie, it runs for a fixed interval, exporting all its >>> observations only at the end of that interval, before optionally >>> starting over. With these new Information Elements, an IPFIX device can >>> readily indicate the start (and end) of each interval. >>> >>> Our present need is only for milliseconds resolution. At this time we >>> don't need the equivalent seconds, microseconds, or nanoseconds >>> elements, although these could be created in future if needs be. >>> >>> Also note the subtle distinction between "monitoring" and "metering": a >>> device could be monitoring traffic for some time, without detecting >>> anything of interest or relevance to be metered. The metering interval >>> could almost be expressed by minFlowStart* to maxFlowEnd*. We've chosen >>> to name these elements "monitoring interval" rather than "metering >>> interval" to clearly express this difference. >>> >>> >>> Name: monitoringIntervalStartMilliSeconds >>> Data Type: dateTimeMilliseconds >>> Sematics: - >>> Description: The absolute timestamp at which the monitoring interval >>> started. >>> A Monitoring interval is the period of time during which >>> the Metering Process is running. >>> Units: milliseconds >>> >>> >>> Name: monitoringIntervalEndMilliSeconds >>> Data Type: dateTimeMilliseconds >>> Sematics: - >>> Description: The absolute timestamp at which the monitoring interval >>> ended. >>> A Monitoring interval is the period of time during which >>> the Metering Process is running. >>> Units: milliseconds >>> >>> >>> Thanks, >>> P. >>> >>> _______________________________________________ >>> IPFIX mailing list >>> IPFIX@ietf.org >>> https://www.ietf.org/mailman/listinfo/ipfix > From paitken@cisco.com Mon Jun 25 10:08:45 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C315521F84A6 for ; Mon, 25 Jun 2012 10:08:45 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -9.974 X-Spam-Level: X-Spam-Status: No, score=-9.974 tagged_above=-999 required=5 tests=[AWL=0.625, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id F3mCs3WVEsqi for ; Mon, 25 Jun 2012 10:08:45 -0700 (PDT) Received: from ams-iport-2.cisco.com (ams-iport-2.cisco.com [144.254.224.141]) by ietfa.amsl.com (Postfix) with ESMTP id 8832021F84A2 for ; Mon, 25 Jun 2012 10:08:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=paitken@cisco.com; l=2710; q=dns/txt; s=iport; t=1340644124; x=1341853724; h=message-id:date:from:mime-version:to:cc:subject: references:in-reply-to:content-transfer-encoding; bh=3UPxMFayE+alzHovIDDIs+JnIVKpTjzIDEnUL1Odbks=; b=QHNAsgrkWSr6ANui9ftbxqB7RF1CISPKlRvbQeqty2xeDkdEKYd9TTWX fm+14pAAbCMdv9YW/ECXntdjVu3xvK9XgDXMb1dKljl12sPrr5LnJxigX 5Ln58JBLOPwcLV6QaTHQD2uA19ugT337JN0BXXrDoAUWQh9h/145P9RcB g=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Av8EANea6E+Q/khM/2dsb2JhbABEtjCBB4IYAQEBAwESASVAAQULCyEWDwkDAgECAUUGDQEHAQEeh2QFmHGfY5E1A5UuhVaIRYFmgmA X-IronPort-AV: E=Sophos;i="4.77,473,1336348800"; d="scan'208";a="74834957" Received: from ams-core-3.cisco.com ([144.254.72.76]) by ams-iport-2.cisco.com with ESMTP; 25 Jun 2012 17:08:40 +0000 Received: from [10.55.88.94] (dhcp-10-55-88-94.cisco.com [10.55.88.94]) by ams-core-3.cisco.com (8.14.5/8.14.5) with ESMTP id q5PH8cUi004969; Mon, 25 Jun 2012 17:08:38 GMT Message-ID: <4FE89B1A.5060301@cisco.com> Date: Mon, 25 Jun 2012 18:08:42 +0100 From: Paul Aitken User-Agent: Mozilla/5.0 (X11; Linux i686; rv:12.0) Gecko/20120430 Thunderbird/12.0.1 MIME-Version: 1.0 To: Gerhard Muenz References: <4FDC9E8B.1070800@cisco.com> <4FDE112B.40706@net.in.tum.de> <4FE88612.1080702@cisco.com> <4FE896DE.6050908@net.in.tum.de> In-Reply-To: <4FE896DE.6050908@net.in.tum.de> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: ipfix-ads@tools.ietf.org, IETF IPFIX Working Group , "ipfix-chairs@tools.ietf.org" Subject: Re: [IPFIX] new monitoring interval information elements X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Jun 2012 17:08:45 -0000 Gerhard, > Ok, I see the difference to discontinuity time. > > Some further thoughts: > > The common understanding is that the "time since re-initialization" > means sysUpTime. For example, have a look at the description of > flowEndSysUpTime: > "The relative timestamp of the last packet of this Flow. It indicates > the number of milliseconds since the last (re-)initialization of the > IPFIX Device (sysUpTime)." Yes, sysUpTime (so specifically, flowStartSysUpTime and flowEndSysUpTime), are times "since the last (re-)initialization of the IPFIX Device". These are not necessarily related to the metering process. Taking some other examples: octetTotalCount, packetTotalCount, droppedOctetTotalCount, droppedPacketTotalCount, observedFlowTotalCount, ignoredPacketTotalCount, ignoredOctetTotalCount, postMCastOctetTotalCount, ... are all of the form: "The total number of X since the Metering Process (re-)initialization for this Observation Point." We could suppose that the Metering Process initialised at system uptime. However, that may not be the case: it could have started any amount of time later. eg, a user could configure a Metering Process at any time. Or, an automated system could start and stop Metering Processes according to different times of day or different network conditions. Without timestamps to tell us, we simply do not know. > Your proposed IEs do not mention neither (re-)initialization nor > sysUpTime. They rather seem to specify a time interval which is > located somewhere within the sysuptime interval. Hence, the interval > does not work to define the time frame of exportedMessageTotalCount, > for example. You would need to define a new IE > exportedMessageInMonitoringInterval. Indeed, because the IEs which I'm specifying here are particular to the Monitoring Process, whereas exportedMessageTotalCount seems to require a time reference for the Exporting Process. It's a related, though different, issue. (ie, the Exporting Process may start from system init, or at any time since then. It may have been started before, or after, the Metering Process. Without a suitable timestamp IE, we simply do not know.) > The different IE descriptions suggest that the time of > re-initialization (i.e. sysuptime) can be different for Metering > Process, Exporting Process etc. Indeed this is so. There is no necessity for these to be the same. > It seems that we need IE to export sysUpTime for each process. We agree. So in principal, you understand what I'm trying to do. However, you would like clearer definitions of the Information Elements? Thanks, P. From muenz@net.in.tum.de Mon Jun 25 10:25:20 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 194D321F84A6 for ; Mon, 25 Jun 2012 10:25:20 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.149 X-Spam-Level: X-Spam-Status: No, score=-6.149 tagged_above=-999 required=5 tests=[AWL=0.100, BAYES_00=-2.599, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XW3UYyw0SgBD for ; Mon, 25 Jun 2012 10:25:19 -0700 (PDT) Received: from mail-out1.informatik.tu-muenchen.de (mail-out1.informatik.tu-muenchen.de [131.159.0.8]) by ietfa.amsl.com (Postfix) with ESMTP id 5067021F8478 for ; Mon, 25 Jun 2012 10:25:13 -0700 (PDT) Received: from [192.168.2.34] (e181142196.adsl.alicedsl.de [85.181.142.196]) by mail.net.in.tum.de (Postfix) with ESMTPSA id 7E350200AA48; Mon, 25 Jun 2012 19:25:08 +0200 (CEST) Message-ID: <4FE89EE0.9020409@net.in.tum.de> Date: Mon, 25 Jun 2012 19:24:48 +0200 From: Gerhard Muenz User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:13.0) Gecko/20120614 Thunderbird/13.0.1 MIME-Version: 1.0 To: Paul Aitken References: <4FDC9E8B.1070800@cisco.com> <4FDE112B.40706@net.in.tum.de> <4FE88612.1080702@cisco.com> <4FE896DE.6050908@net.in.tum.de> <4FE89B1A.5060301@cisco.com> In-Reply-To: <4FE89B1A.5060301@cisco.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: ipfix-ads@tools.ietf.org, IETF IPFIX Working Group , "ipfix-chairs@tools.ietf.org" Subject: Re: [IPFIX] new monitoring interval information elements X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Jun 2012 17:25:20 -0000 Paul, It depends on what kind of interval you want to specify. If you want to specify the (re-)initialization time of the Metering Process, why not name the IE meteringProcessSysUpTime or (if you do not like sysup) meteringProcessInitializationTime? And mention (re-)initialization in the description? Dito for exportingProcessInitializationTime etc. If we need the stop time, you can use meteringProcessStopTime or so. If you want to define a new interval which is not starting at (re-)initialization time but later, then your definitions are ok. But this means that you need to specify new counter elements for this interval. Thanks, Gerhard On 25.06.2012 19:08, Paul Aitken wrote: > Gerhard, > >> Ok, I see the difference to discontinuity time. >> >> Some further thoughts: >> >> The common understanding is that the "time since re-initialization" >> means sysUpTime. For example, have a look at the description of >> flowEndSysUpTime: >> "The relative timestamp of the last packet of this Flow. It indicates >> the number of milliseconds since the last (re-)initialization of the >> IPFIX Device (sysUpTime)." > > Yes, sysUpTime (so specifically, flowStartSysUpTime and > flowEndSysUpTime), are times "since the last (re-)initialization of the > IPFIX Device". > > These are not necessarily related to the metering process. > > Taking some other examples: octetTotalCount, packetTotalCount, > droppedOctetTotalCount, droppedPacketTotalCount, observedFlowTotalCount, > ignoredPacketTotalCount, ignoredOctetTotalCount, > postMCastOctetTotalCount, ... are all of the form: > > "The total number of X since the Metering Process > (re-)initialization for this Observation Point." > > We could suppose that the Metering Process initialised at system uptime. > However, that may not be the case: it could have started any amount of > time later. eg, a user could configure a Metering Process at any time. > Or, an automated system could start and stop Metering Processes > according to different times of day or different network conditions. > Without timestamps to tell us, we simply do not know. > > > >> Your proposed IEs do not mention neither (re-)initialization nor >> sysUpTime. They rather seem to specify a time interval which is >> located somewhere within the sysuptime interval. Hence, the interval >> does not work to define the time frame of exportedMessageTotalCount, >> for example. You would need to define a new IE >> exportedMessageInMonitoringInterval. > > Indeed, because the IEs which I'm specifying here are particular to the > Monitoring Process, whereas exportedMessageTotalCount seems to require a > time reference for the Exporting Process. It's a related, though > different, issue. > > (ie, the Exporting Process may start from system init, or at any time > since then. It may have been started before, or after, the Metering > Process. Without a suitable timestamp IE, we simply do not know.) > > >> The different IE descriptions suggest that the time of >> re-initialization (i.e. sysuptime) can be different for Metering >> Process, Exporting Process etc. > > Indeed this is so. There is no necessity for these to be the same. > > >> It seems that we need IE to export sysUpTime for each process. > > We agree. > > So in principal, you understand what I'm trying to do. However, you > would like clearer definitions of the Information Elements? > > Thanks, > P. > From Quittek@neclab.eu Mon Jun 25 10:26:48 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5267E21F847D; Mon, 25 Jun 2012 10:26:48 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.599 X-Spam-Level: X-Spam-Status: No, score=-102.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id x+mQfZPO1r0k; Mon, 25 Jun 2012 10:26:47 -0700 (PDT) Received: from mailer1.neclab.eu (mailer1.neclab.eu [195.37.70.40]) by ietfa.amsl.com (Postfix) with ESMTP id 6BF9F21F8478; Mon, 25 Jun 2012 10:26:46 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by mailer1.neclab.eu (Postfix) with ESMTP id 9374C101636; Mon, 25 Jun 2012 19:30:55 +0200 (CEST) X-Virus-Scanned: Amavisd on Debian GNU/Linux (netlab.nec.de) Received: from mailer1.neclab.eu ([127.0.0.1]) by localhost (atlas-a.office.hd [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 03KY0lX8ycNO; Mon, 25 Jun 2012 19:30:55 +0200 (CEST) Received: from ENCELADUS.office.hd (enceladus.office.hd [192.168.24.52]) by mailer1.neclab.eu (Postfix) with ESMTP id 6DC4B101635; Mon, 25 Jun 2012 19:30:35 +0200 (CEST) Received: from PALLENE.office.hd ([169.254.1.191]) by ENCELADUS.office.hd ([192.168.24.52]) with mapi id 14.01.0323.003; Mon, 25 Jun 2012 19:27:47 +0200 From: Juergen Quittek To: Ronald Bonica , Benoit Claise Thread-Topic: Write-up for draft-ietf-ipfix-ie-doctors-03 Thread-Index: AQHNUvfW+t/5CncsTUezXnmgw2M/qg== Date: Mon, 25 Jun 2012 17:27:46 +0000 Message-ID: Accept-Language: de-DE, en-US Content-Language: de-DE X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/14.14.0.111121 x-originating-ip: [10.7.0.92] Content-Type: text/plain; charset="us-ascii" Content-ID: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: IESG Secretary , IETF IPFIX Working Group Subject: [IPFIX] Write-up for draft-ietf-ipfix-ie-doctors-03 X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Jun 2012 17:26:48 -0000 Write-up for draft-ietf-ipfix-ie-doctors-03 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D (1) What type of RFC is being requested (BCP, Proposed Standard, Internet Standard, Informational, Experimental, or Historic)? Why is this the proper type of RFC? Is this type of RFC indicated in the title page header? The requested type is BCP as stated in the title page header. (2) The IESG approval announcement includes a Document Announcement Write-Up. Please provide such a Document Announcement Write-Up. Recent examples can be found in the "Action" announcements for approved documents. The approval announcement contains the following sections: Technical Summary This document provides guidelines for the definition of IPFIX Information Elements for addition to the IANA IPFIX Information Element registry, in order to extend the applicability of the IPFIX protocol to new operations and management areas. Working Group Summary This draft has been discussed already at WG sessions and on the mailing list for along time before adopting it as WG work item. There was strong consensus on the need for such a document and on the general content. Details has been discussed extensively. WGLC was conducted in February 2012. Raise issues in the LC Phase were discussed and fixed. Document Quality This BCP for IPFIX IE doctors summarizes experiences gained in several years of dealing with new proposals for IEs. Personnel By default, Benoit Claise would be the responsible area director. But since Benoit Claise is co-author of the draft, the responsible area director is Ron Bonica. Document shepherd is Juergen Quittek. (3) Briefly describe the review of this document that was performed by the Document Shepherd. If this version of the document is not ready for publication, please explain why the document is being forwarded to the IESG. The document shepherd has reviewed the draft and is fully convinced that it is ready for publication. (4) Does the document Shepherd have any concerns about the depth or breadth of the reviews that have been performed? No. (5) Do portions of the document need review from a particular or from broader perspective, e.g., security, operational complexity, AAA, DNS, DHCP, XML, or internationalization? If so, describe the review that took place. No. (6) Describe any specific concerns or issues that the Document Shepherd has with this document that the Responsible Area Director and/or the IESG should be aware of? For example, perhaps he or she is uncomfortable with certain parts of the document, or has concerns whether there really is a need for it. In any event, if the WG has discussed those issues and has indicated that it still wishes to advance the document, detail those concerns here. There are no such issues. (7) Has each author confirmed that any and all appropriate IPR disclosures required for full conformance with the provisions of BCP 78 and BCP 79 have already been filed. If not, explain why. Yes. (8) Has an IPR disclosure been filed that references this document? If so, summarize any WG discussion and conclusion regarding the IPR Disclosures. No. (9) How solid is the WG consensus behind this document? Does it represent the strong concurrence of a few individuals, with others being silent, or does the WG as a whole understand and agree with it? The WG as a whole understand and agree with it. (10) Has anyone threatened an appeal or otherwise indicated extreme discontent? If so, please summarise the areas of conflict in separate email messages to the Responsible Area Director. (It should be in a separate email because this questionnaire is publicly available.) No. (11) Identify any ID nits the Document Shepherd has found in this document. (See http://www.ietf.org/tools/idnits/ and the Internet-Drafts Checklist). Boilerplate checks are not enough; this check needs to be Thorough. The document uses 'NOT RECOMMENDED' as an RFC 2119 keyword, but does not include the phrase in its RFC 2119 key words list. This can be fixed after IETF last call. (12) Describe how the document meets any required formal review criteria, such as the MIB Doctor, media type, and URI type reviews. No further formal review required. (13) Have all references within this document been identified as either normative or informative? Yes. (14) Are there normative references to documents that are not ready for advancement or are otherwise in an unclear state? If such normative references exist, what is the plan for their completion? Yes. There are two: - draft-ietf-ipfix-protocol-rfc5101bis-01 - draft-ietf-ipfix-information-model-rfc5102bis-01 Both are on the IPFIX WG charter and currently progressing in the WG. (15) Are there downward normative references references (see RFC 3967)? If so, list these downward references to support the Area Director in the Last Call procedure. No. (16) Will publication of this document change the status of any existing RFCs? Are those RFCs listed on the title page header, listed in the abstract, and discussed in the introduction? If the RFCs are not listed in the Abstract and Introduction, explain why, and point to the part of the document where the relationship of this document to the other RFCs is discussed. If this information is not in the document, explain why the WG considers it unnecessary. No. (17) Describe the Document Shepherd's review of the IANA considerations section, especially with regard to its consistency with the body of the document. Confirm that all protocol extensions that the document makes are associated with the appropriate reservations in IANA registries. Confirm that any referenced IANA registries have been clearly identified. Confirm that newly created IANA registries include a detailed specification of the initial contents for the registry, that allocations procedures for future registrations are defined, and a reasonable name for the new registry has been suggested (see RFC 5226). The draft defines a process for IANA and extends an IANA registry. Both issues are well documented in the IANA considerations section. (18) List any new IANA registries that require Expert Review for future allocations. Provide any public guidance that the IESG would find useful in selecting the IANA Experts for these new registries. There are no new registries that require expert review requested by this draft. (19) Describe reviews and automated checks performed by the Document Shepherd to validate sections of the document written in a formal language, such as XML code, BNF rules, MIB definitions, etc. None to be done. From internet-drafts@ietf.org Wed Jun 27 00:37:52 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 62ACE21F85E6; Wed, 27 Jun 2012 00:37:52 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.599 X-Spam-Level: X-Spam-Status: No, score=-102.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Esr6CT-a02h8; Wed, 27 Jun 2012 00:37:52 -0700 (PDT) Received: from ietfa.amsl.com (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0018F21F85A5; Wed, 27 Jun 2012 00:37:51 -0700 (PDT) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: internet-drafts@ietf.org To: i-d-announce@ietf.org X-Test-IDTracker: no X-IETF-IDTracker: 4.21 Message-ID: <20120627073751.14170.12188.idtracker@ietfa.amsl.com> Date: Wed, 27 Jun 2012 00:37:51 -0700 Cc: ipfix@ietf.org Subject: [IPFIX] I-D Action: draft-ietf-ipfix-a9n-04.txt X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Jun 2012 07:37:52 -0000 A New Internet-Draft is available from the on-line Internet-Drafts director= ies. This draft is a work item of the IP Flow Information Export Working Group = of the IETF. Title : Flow Aggregation for the IP Flow Information Export (IPF= IX) Protocol Author(s) : Brian Trammell Arno Wagner Benoit Claise Filename : draft-ietf-ipfix-a9n-04.txt Pages : 48 Date : 2012-06-27 Abstract: This document provides a common implementation-independent basis for the interoperable application of the IP Flow Information Export (IPFIX) Protocol to the handling of Aggregated Flows, which are IPFIX Flow representing packets from multiple Original Flows sharing some set of common properties. It does this through a detailed terminology and a descriptive Intermediate Aggregation Process architecture, including a specification of methods for Original Flow counting and counter distribution across intervals. The IETF datatracker status page for this draft is: https://datatracker.ietf.org/doc/draft-ietf-ipfix-a9n There's also a htmlized version available at: http://tools.ietf.org/html/draft-ietf-ipfix-a9n-04 A diff from previous version is available at: http://tools.ietf.org/rfcdiff?url2=3Ddraft-ietf-ipfix-a9n-04 Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ From trammell@tik.ee.ethz.ch Wed Jun 27 00:40:52 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id AC7D121F85A5 for ; Wed, 27 Jun 2012 00:40:52 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.599 X-Spam-Level: X-Spam-Status: No, score=-6.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id RIeCPe4R1Ycx for ; Wed, 27 Jun 2012 00:40:51 -0700 (PDT) Received: from smtp.ee.ethz.ch (smtp.ee.ethz.ch [129.132.2.219]) by ietfa.amsl.com (Postfix) with ESMTP id 8B21421F85E6 for ; Wed, 27 Jun 2012 00:40:51 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by smtp.ee.ethz.ch (Postfix) with ESMTP id AF662D930B for ; Wed, 27 Jun 2012 09:40:50 +0200 (MEST) X-Virus-Scanned: by amavisd-new on smtp.ee.ethz.ch Received: from smtp.ee.ethz.ch ([127.0.0.1]) by localhost (.ee.ethz.ch [127.0.0.1]) (amavisd-new, port 10024) with LMTP id ZpjiGbdzvR8z for ; Wed, 27 Jun 2012 09:40:50 +0200 (MEST) Received: from [10.0.1.2] (cust-integra-121-161.antanet.ch [80.75.121.161]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: briant) by smtp.ee.ethz.ch (Postfix) with ESMTPSA id 71B47D930A for ; Wed, 27 Jun 2012 09:40:50 +0200 (MEST) From: Brian Trammell Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Date: Wed, 27 Jun 2012 09:40:49 +0200 References: <20120627073751.14170.12188.idtracker@ietfa.amsl.com> To: ipfix@ietf.org Message-Id: Mime-Version: 1.0 (Apple Message framework v1278) X-Mailer: Apple Mail (2.1278) Subject: [IPFIX] Fwd: I-D Action: draft-ietf-ipfix-a9n-04.txt X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Jun 2012 07:40:52 -0000 Greetings, all, The -04 revision addresses WGLC comments received to date (see list); = pending no further comment, we consider this revision of the draft ready = for submission to the IESG. Best regards, Brian Begin forwarded message: > From: internet-drafts@ietf.org > Subject: [IPFIX] I-D Action: draft-ietf-ipfix-a9n-04.txt > Date: June 27, 2012 9:37:51 AM GMT+02:00 > To: i-d-announce@ietf.org > Cc: ipfix@ietf.org >=20 >=20 > A New Internet-Draft is available from the on-line Internet-Drafts = directories. > This draft is a work item of the IP Flow Information Export Working = Group of the IETF. >=20 > Title : Flow Aggregation for the IP Flow Information = Export (IPFIX) Protocol > Author(s) : Brian Trammell > Arno Wagner > Benoit Claise > Filename : draft-ietf-ipfix-a9n-04.txt > Pages : 48 > Date : 2012-06-27 >=20 > Abstract: > This document provides a common implementation-independent basis for > the interoperable application of the IP Flow Information Export > (IPFIX) Protocol to the handling of Aggregated Flows, which are = IPFIX > Flow representing packets from multiple Original Flows sharing some > set of common properties. It does this through a detailed > terminology and a descriptive Intermediate Aggregation Process > architecture, including a specification of methods for Original Flow > counting and counter distribution across intervals. >=20 >=20 > The IETF datatracker status page for this draft is: > https://datatracker.ietf.org/doc/draft-ietf-ipfix-a9n >=20 > There's also a htmlized version available at: > http://tools.ietf.org/html/draft-ietf-ipfix-a9n-04 >=20 > A diff from previous version is available at: > http://tools.ietf.org/rfcdiff?url2=3Ddraft-ietf-ipfix-a9n-04 >=20 >=20 > Internet-Drafts are also available by anonymous FTP at: > ftp://ftp.ietf.org/internet-drafts/ >=20 > _______________________________________________ > IPFIX mailing list > IPFIX@ietf.org > https://www.ietf.org/mailman/listinfo/ipfix From internet-drafts@ietf.org Thu Jun 28 05:15:23 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3A49F21F84FF; Thu, 28 Jun 2012 05:15:23 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.599 X-Spam-Level: X-Spam-Status: No, score=-102.599 tagged_above=-999 required=5 tests=[AWL=0.000, BAYES_00=-2.599, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lpQka5pIRs9F; Thu, 28 Jun 2012 05:15:22 -0700 (PDT) Received: from ietfa.amsl.com (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B5D1321F845D; Thu, 28 Jun 2012 05:15:22 -0700 (PDT) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: internet-drafts@ietf.org To: i-d-announce@ietf.org X-Test-IDTracker: no X-IETF-IDTracker: 4.21p1 Message-ID: <20120628121522.8396.53502.idtracker@ietfa.amsl.com> Date: Thu, 28 Jun 2012 05:15:22 -0700 Cc: ipfix@ietf.org Subject: [IPFIX] I-D Action: draft-ietf-ipfix-protocol-rfc5101bis-02.txt X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Jun 2012 12:15:23 -0000 A New Internet-Draft is available from the on-line Internet-Drafts director= ies. This draft is a work item of the IP Flow Information Export Working Group = of the IETF. Title : Specification of the IP Flow Information eXport (IPFIX) = Protocol for the Exchange of Flow Information Author(s) : Benoit Claise Brian Trammell Filename : draft-ietf-ipfix-protocol-rfc5101bis-02.txt Pages : 68 Date : 2012-06-28 Abstract: This document specifies the IP Flow Information Export (IPFIX) protocol that serves for transmitting Traffic Flow information over the network. In order to transmit Traffic Flow information from an Exporting Process to an information Collecting Process, a common representation of flow data and a standard means of communicating them is required. This document describes how the IPFIX Data and Template Records are carried over a number of transport protocols from an IPFIX Exporting Process to an IPFIX Collecting Process. This document obsoletes RFC 5101. The IETF datatracker status page for this draft is: https://datatracker.ietf.org/doc/draft-ietf-ipfix-protocol-rfc5101bis There's also a htmlized version available at: http://tools.ietf.org/html/draft-ietf-ipfix-protocol-rfc5101bis-02 A diff from previous version is available at: http://tools.ietf.org/rfcdiff?url2=3Ddraft-ietf-ipfix-protocol-rfc5101bis-02 Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ From trammell@tik.ee.ethz.ch Thu Jun 28 05:26:33 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 626A621F856F for ; Thu, 28 Jun 2012 05:26:33 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.599 X-Spam-Level: X-Spam-Status: No, score=-6.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kkBrpaBoWZtE for ; Thu, 28 Jun 2012 05:26:32 -0700 (PDT) Received: from smtp.ee.ethz.ch (smtp.ee.ethz.ch [129.132.2.219]) by ietfa.amsl.com (Postfix) with ESMTP id 8231D21F856D for ; Thu, 28 Jun 2012 05:26:29 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by smtp.ee.ethz.ch (Postfix) with ESMTP id 3F932D930B for ; Thu, 28 Jun 2012 14:26:28 +0200 (MEST) X-Virus-Scanned: by amavisd-new on smtp.ee.ethz.ch Received: from smtp.ee.ethz.ch ([127.0.0.1]) by localhost (.ee.ethz.ch [127.0.0.1]) (amavisd-new, port 10024) with LMTP id latfeDBCd6Sg for ; Thu, 28 Jun 2012 14:26:28 +0200 (MEST) Received: from [10.0.1.2] (cust-integra-121-161.antanet.ch [80.75.121.161]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: briant) by smtp.ee.ethz.ch (Postfix) with ESMTPSA id EA652D930A for ; Thu, 28 Jun 2012 14:26:27 +0200 (MEST) From: Brian Trammell Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Date: Thu, 28 Jun 2012 14:26:27 +0200 References: <20120628121522.8396.53502.idtracker@ietfa.amsl.com> To: ipfix@ietf.org Message-Id: <09E3F0A1-27ED-4405-B14C-B0CF371A09FA@tik.ee.ethz.ch> Mime-Version: 1.0 (Apple Message framework v1278) X-Mailer: Apple Mail (2.1278) Subject: [IPFIX] Fwd: I-D Action: draft-ietf-ipfix-protocol-rfc5101bis-02.txt X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Jun 2012 12:26:33 -0000 Greetings, all, 5101-bis rev 02 has been posted. This revision addresses points raised = in discussions on the mailing list after Paris: clarification of the = applicability of reduced-size encoding, clarification of the definition = of Flows in parallel with -a9n, changes to simplified template = management to be more in line with the existing protocol. There are = minor=20 Other points raised after Paris included whether to modify MTI for = TLS/DTLS. On review, the MTI is actually specified per transport (if = SCTP then DTLS, if UDP then DTLS, if TCP then TLS), so the requirement = to implement DTLS over SCTP is only implied by the requirement to = implement SCTP. Given this, and the fact that DTLS over SCTP has = apparently become implementable without having to resort to various = library shenanigans, it would appear that this is neither necessary to = do, nor possible without breaking interop with 5101. We will have to = show DTLS over SCTP, DTLS over UDP, and TLS over TCP interop to advance = the document, though. Please review and comment to the list. Best regards, Brian Begin forwarded message: > From: internet-drafts@ietf.org > Subject: [IPFIX] I-D Action: = draft-ietf-ipfix-protocol-rfc5101bis-02.txt > Date: June 28, 2012 2:15:22 PM GMT+02:00 > To: i-d-announce@ietf.org > Cc: ipfix@ietf.org >=20 >=20 > A New Internet-Draft is available from the on-line Internet-Drafts = directories. > This draft is a work item of the IP Flow Information Export Working = Group of the IETF. >=20 > Title : Specification of the IP Flow Information = eXport (IPFIX) Protocol for the Exchange of Flow Information > Author(s) : Benoit Claise > Brian Trammell > Filename : draft-ietf-ipfix-protocol-rfc5101bis-02.txt > Pages : 68 > Date : 2012-06-28 >=20 > Abstract: > This document specifies the IP Flow Information Export (IPFIX) > protocol that serves for transmitting Traffic Flow information over > the network. In order to transmit Traffic Flow information from an > Exporting Process to an information Collecting Process, a common > representation of flow data and a standard means of communicating > them is required. This document describes how the IPFIX Data and > Template Records are carried over a number of transport protocols > from an IPFIX Exporting Process to an IPFIX Collecting Process. = This > document obsoletes RFC 5101. >=20 >=20 > The IETF datatracker status page for this draft is: > https://datatracker.ietf.org/doc/draft-ietf-ipfix-protocol-rfc5101bis >=20 > There's also a htmlized version available at: > http://tools.ietf.org/html/draft-ietf-ipfix-protocol-rfc5101bis-02 >=20 > A diff from previous version is available at: > = http://tools.ietf.org/rfcdiff?url2=3Ddraft-ietf-ipfix-protocol-rfc5101bis-= 02 >=20 >=20 > Internet-Drafts are also available by anonymous FTP at: > ftp://ftp.ietf.org/internet-drafts/ >=20 > _______________________________________________ > IPFIX mailing list > IPFIX@ietf.org > https://www.ietf.org/mailman/listinfo/ipfix From internet-drafts@ietf.org Thu Jun 28 05:44:09 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 64A4821F85C0; Thu, 28 Jun 2012 05:44:09 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.599 X-Spam-Level: X-Spam-Status: No, score=-102.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id WqkzJtElpKg9; Thu, 28 Jun 2012 05:44:09 -0700 (PDT) Received: from ietfa.amsl.com (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id EA88121F84B8; Thu, 28 Jun 2012 05:44:08 -0700 (PDT) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: internet-drafts@ietf.org To: i-d-announce@ietf.org X-Test-IDTracker: no X-IETF-IDTracker: 4.21p1 Message-ID: <20120628124408.25711.94479.idtracker@ietfa.amsl.com> Date: Thu, 28 Jun 2012 05:44:08 -0700 Cc: ipfix@ietf.org Subject: [IPFIX] I-D Action: draft-ietf-ipfix-information-model-rfc5102bis-02.txt X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Jun 2012 12:44:09 -0000 A New Internet-Draft is available from the on-line Internet-Drafts director= ies. This draft is a work item of the IP Flow Information Export Working Group = of the IETF. Title : Information Model for IP Flow Information eXport (IPFIX) Author(s) : Benoit Claise Brian Trammell Filename : draft-ietf-ipfix-information-model-rfc5102bis-02.txt Pages : 29 Date : 2012-06-28 Abstract: This memo defines an overview of the information model for the IP Flow Information eXport (IPFIX) protocol. It is used by the IPFIX protocol for encoding measured traffic information and information related to the traffic Observation Point, the traffic Metering Process, and the Exporting Process. Although developed for the IPFIX protocol, the model is defined in an open way that easily allows using it in other protocols, interfaces, and applications. This document obsoletes RFC 5102. The IETF datatracker status page for this draft is: https://datatracker.ietf.org/doc/draft-ietf-ipfix-information-model-rfc5102= bis There's also a htmlized version available at: http://tools.ietf.org/html/draft-ietf-ipfix-information-model-rfc5102bis-02 A diff from previous version is available at: http://tools.ietf.org/rfcdiff?url2=3Ddraft-ietf-ipfix-information-model-rfc= 5102bis-02 Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ From trammell@tik.ee.ethz.ch Thu Jun 28 05:51:22 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C119521F8550 for ; Thu, 28 Jun 2012 05:51:21 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.599 X-Spam-Level: X-Spam-Status: No, score=-6.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id SKXycTe1sypN for ; Thu, 28 Jun 2012 05:51:21 -0700 (PDT) Received: from smtp.ee.ethz.ch (smtp.ee.ethz.ch [129.132.2.219]) by ietfa.amsl.com (Postfix) with ESMTP id 7FB4321F854D for ; Thu, 28 Jun 2012 05:51:20 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by smtp.ee.ethz.ch (Postfix) with ESMTP id DAD15D930C for ; Thu, 28 Jun 2012 14:51:19 +0200 (MEST) X-Virus-Scanned: by amavisd-new on smtp.ee.ethz.ch Received: from smtp.ee.ethz.ch ([127.0.0.1]) by localhost (.ee.ethz.ch [127.0.0.1]) (amavisd-new, port 10024) with LMTP id QvgeJQJrXNBf for ; Thu, 28 Jun 2012 14:51:19 +0200 (MEST) Received: from [10.0.1.2] (cust-integra-121-161.antanet.ch [80.75.121.161]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: briant) by smtp.ee.ethz.ch (Postfix) with ESMTPSA id 91098D930A for ; Thu, 28 Jun 2012 14:51:19 +0200 (MEST) From: Brian Trammell Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Date: Thu, 28 Jun 2012 14:51:19 +0200 References: <20120628124408.25711.94479.idtracker@ietfa.amsl.com> To: ipfix@ietf.org Message-Id: <2C05EA55-5AC1-4852-ADCE-5F3B96E6AD1F@tik.ee.ethz.ch> Mime-Version: 1.0 (Apple Message framework v1278) X-Mailer: Apple Mail (2.1278) Subject: [IPFIX] Fwd: I-D Action: draft-ietf-ipfix-information-model-rfc5102bis-02.txt X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Jun 2012 12:51:22 -0000 Greetings, all, The next revision of 5102-bis is now available. The changes bring the = document into line with -ie-doctors, and fix issues with timestamp data = types and data type semantics. There will most likely be one more editorial revision before the = Vancouver deadlines. The largest remaining open issue is a review of = Section 4.1, or its elimination, should we decide it's no longer = important to have in the document. (I'd need some help from a V9 expert = on this point). Best regards, Brian Begin forwarded message: > From: internet-drafts@ietf.org > Subject: [IPFIX] I-D Action: = draft-ietf-ipfix-information-model-rfc5102bis-02.txt > Date: June 28, 2012 2:44:08 PM GMT+02:00 > To: i-d-announce@ietf.org > Cc: ipfix@ietf.org >=20 >=20 > A New Internet-Draft is available from the on-line Internet-Drafts = directories. > This draft is a work item of the IP Flow Information Export Working = Group of the IETF. >=20 > Title : Information Model for IP Flow Information = eXport (IPFIX) > Author(s) : Benoit Claise > Brian Trammell > Filename : = draft-ietf-ipfix-information-model-rfc5102bis-02.txt > Pages : 29 > Date : 2012-06-28 >=20 > Abstract: > This memo defines an overview of the information model for the IP Flow > Information eXport (IPFIX) protocol. It is used by the IPFIX protocol > for encoding measured traffic information and information related to = the > traffic Observation Point, the traffic Metering Process, and the > Exporting Process. Although developed for the IPFIX protocol, the = model > is defined in an open way that easily allows using it in other > protocols, interfaces, and applications. This document obsoletes RFC > 5102. >=20 >=20 > The IETF datatracker status page for this draft is: > = https://datatracker.ietf.org/doc/draft-ietf-ipfix-information-model-rfc510= 2bis >=20 > There's also a htmlized version available at: > = http://tools.ietf.org/html/draft-ietf-ipfix-information-model-rfc5102bis-0= 2 >=20 > A diff from previous version is available at: > = http://tools.ietf.org/rfcdiff?url2=3Ddraft-ietf-ipfix-information-model-rf= c5102bis-02 >=20 >=20 > Internet-Drafts are also available by anonymous FTP at: > ftp://ftp.ietf.org/internet-drafts/ >=20 > _______________________________________________ > IPFIX mailing list > IPFIX@ietf.org > https://www.ietf.org/mailman/listinfo/ipfix From andrjohn@cisco.com Thu Jun 28 16:25:44 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 01DC111E809B for ; Thu, 28 Jun 2012 16:25:44 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.599 X-Spam-Level: X-Spam-Status: No, score=-10.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id dh0BtLZO4wNc for ; Thu, 28 Jun 2012 16:25:43 -0700 (PDT) Received: from mtv-iport-1.cisco.com (mtv-iport-1.cisco.com [173.36.130.12]) by ietfa.amsl.com (Postfix) with ESMTP id 1DA8F11E8088 for ; Thu, 28 Jun 2012 16:25:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=andrjohn@cisco.com; l=6281; q=dns/txt; s=iport; t=1340925943; x=1342135543; h=subject:mime-version:from:in-reply-to:date:cc: content-transfer-encoding:message-id:references:to; bh=IKBqjtlrDsuxxu4gQrWBMq7CKGjmcGF8vXGW+S2no40=; b=aDKDgx63jMN4JgRYBhsz0bwy2+OxrSYB2t5JtfDr4+X+EHoy9SsVQ2ah 6KABiUsHTtRXHtial4PNRbroF59M3M20jQ6Vj1oX9/3lA4sQIfBqyikeo 6HpEftV60CWlCKZE/dfnl4KHMgxvb3ebCKaNfnOjg0FrjdQfjfuC9sS+G c=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Av4EAKPn7E+rRDoJ/2dsb2JhbAA7CrZEgQeCGAEBAQQBAQEPARQTNAsQCxguJzAGEyKHaAELmCOgWwSLNxCFGmADlTKOHYFmgmA X-IronPort-AV: E=Sophos;i="4.77,494,1336348800"; d="scan'208";a="47401341" Received: from mtv-core-4.cisco.com ([171.68.58.9]) by mtv-iport-1.cisco.com with ESMTP; 28 Jun 2012 23:25:42 +0000 Received: from cisco.com (mrwint.cisco.com [64.103.70.36]) by mtv-core-4.cisco.com (8.14.5/8.14.5) with ESMTP id q5SNPfgN003751 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 28 Jun 2012 23:25:42 GMT Received: from ams-andrjohn-8716.cisco.com (ams-andrjohn-8716.cisco.com [10.55.163.39]) by cisco.com (8.14.4+Sun/8.8.8) with ESMTP id q5SNPanP029654; Fri, 29 Jun 2012 00:25:37 +0100 (BST) Mime-Version: 1.0 (Apple Message framework v1278) Content-Type: text/plain; charset=us-ascii From: Andrew Johnson In-Reply-To: <4FE89EE0.9020409@net.in.tum.de> Date: Fri, 29 Jun 2012 00:25:31 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <4FDC9E8B.1070800@cisco.com> <4FDE112B.40706@net.in.tum.de> <4FE88612.1080702@cisco.com> <4FE896DE.6050908@net.in.tum.de> <4FE89B1A.5060301@cisco.com> <4FE89EE0.9020409@net.in.tum.de> To: Gerhard Muenz X-Mailer: Apple Mail (2.1278) Cc: ipfix-ads@tools.ietf.org, IETF IPFIX Working Group , "ipfix-chairs@tools.ietf.org" Subject: Re: [IPFIX] new monitoring interval information elements X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Jun 2012 23:25:44 -0000 Hi Gerhard Thanks for your inputs on this thread. It seems to me that the idea of = "re-initialising" the Metering Process is pretty confusing, in light of = what we're actually doing. I think my mental mode of what is happening = is a bit different from Paul's, so let me start at the beginning, show = how I think of the behaviour, and see what we can come up with to solve = our problem. We would like to split the monitored packet streams into discrete = buckets, based on time. t0 -----> t1 -----> t2 -----> t3 -----> t4 -----> t5 -----> The packet stream is processed by the Metering Process as usual, and = flows form in the cache in the usual manner. However, observations from = one interval won't be merged with observations from a different = interval. Ideally, at the end of each interval, all flows are exported = and flushed from the cache. There are several ways to think about this. One way is to explicitly = flush the cache at the end of each interval (i.e. reset / reinitialise = the Metering Process). The way I see it though, we could just make an = identifier of the current interval a key field and apply the = optimisation of allowing flows from outside the current interval to be = flushed from the cache ASAP, since we know they won't be updated again. Hopefully that gives a bit of a bigger picture. What we are looking for = are two new fields that we can add to the flows, that identify the = beginning at end of the interval they were observed in. In some = circumstances the device may not fully stick to the provided interval = window, so we can't rely on the end of the interval really being the = intervalStart + intervalSize. I was thinking of: observationWindowStart dateTimeMilliseconds The absolute timestamp = of the beginning of the current Observation Window. observationWindowEnd dateTimeMilliseconds The absolute timestamp = of the end of the current Observation Window. However, Paul rightly pointed out to me that IPFIX doesn't have any = concept of an Observation Window, and we're not sure how we can define = that in description of the IE. The idea of re-initialising the Metering = Process may fit in with the current IPFIX model, but I don't think it = properly conveys the behaviour we're after. Kind regards, Andrew On 25 Jun 2012, at 18:24, Gerhard Muenz wrote: > Paul, >=20 > It depends on what kind of interval you want to specify. >=20 > If you want to specify the (re-)initialization time of the Metering = Process, why not name the IE meteringProcessSysUpTime or (if you do not = like sysup) meteringProcessInitializationTime? > And mention (re-)initialization in the description? > Dito for exportingProcessInitializationTime etc. >=20 > If we need the stop time, you can use meteringProcessStopTime or so. >=20 > If you want to define a new interval which is not starting at = (re-)initialization time but later, then your definitions are ok. But = this means that you need to specify new counter elements for this = interval. >=20 > Thanks, > Gerhard >=20 >=20 > On 25.06.2012 19:08, Paul Aitken wrote: >> Gerhard, >>=20 >>> Ok, I see the difference to discontinuity time. >>>=20 >>> Some further thoughts: >>>=20 >>> The common understanding is that the "time since re-initialization" >>> means sysUpTime. For example, have a look at the description of >>> flowEndSysUpTime: >>> "The relative timestamp of the last packet of this Flow. It = indicates >>> the number of milliseconds since the last (re-)initialization of the >>> IPFIX Device (sysUpTime)." >>=20 >> Yes, sysUpTime (so specifically, flowStartSysUpTime and >> flowEndSysUpTime), are times "since the last (re-)initialization of = the >> IPFIX Device". >>=20 >> These are not necessarily related to the metering process. >>=20 >> Taking some other examples: octetTotalCount, packetTotalCount, >> droppedOctetTotalCount, droppedPacketTotalCount, = observedFlowTotalCount, >> ignoredPacketTotalCount, ignoredOctetTotalCount, >> postMCastOctetTotalCount, ... are all of the form: >>=20 >> "The total number of X since the Metering Process >> (re-)initialization for this Observation Point." >>=20 >> We could suppose that the Metering Process initialised at system = uptime. >> However, that may not be the case: it could have started any amount = of >> time later. eg, a user could configure a Metering Process at any = time. >> Or, an automated system could start and stop Metering Processes >> according to different times of day or different network conditions. >> Without timestamps to tell us, we simply do not know. >>=20 >>=20 >>=20 >>> Your proposed IEs do not mention neither (re-)initialization nor >>> sysUpTime. They rather seem to specify a time interval which is >>> located somewhere within the sysuptime interval. Hence, the interval >>> does not work to define the time frame of exportedMessageTotalCount, >>> for example. You would need to define a new IE >>> exportedMessageInMonitoringInterval. >>=20 >> Indeed, because the IEs which I'm specifying here are particular to = the >> Monitoring Process, whereas exportedMessageTotalCount seems to = require a >> time reference for the Exporting Process. It's a related, though >> different, issue. >>=20 >> (ie, the Exporting Process may start from system init, or at any time >> since then. It may have been started before, or after, the Metering >> Process. Without a suitable timestamp IE, we simply do not know.) >>=20 >>=20 >>> The different IE descriptions suggest that the time of >>> re-initialization (i.e. sysuptime) can be different for Metering >>> Process, Exporting Process etc. >>=20 >> Indeed this is so. There is no necessity for these to be the same. >>=20 >>=20 >>> It seems that we need IE to export sysUpTime for each process. >>=20 >> We agree. >>=20 >> So in principal, you understand what I'm trying to do. However, you >> would like clearer definitions of the Information Elements? >>=20 >> Thanks, >> P. >>=20 >=20 >=20 >=20 > _______________________________________________ > IPFIX mailing list > IPFIX@ietf.org > https://www.ietf.org/mailman/listinfo/ipfix From dominik.elsbroek@gmail.com Thu Jun 28 23:59:06 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C82B311E810B for ; Thu, 28 Jun 2012 23:59:06 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.549 X-Spam-Level: X-Spam-Status: No, score=-3.549 tagged_above=-999 required=5 tests=[AWL=0.050, BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mm5oxEb41YM1 for ; Thu, 28 Jun 2012 23:59:06 -0700 (PDT) Received: from mail-ob0-f172.google.com (mail-ob0-f172.google.com [209.85.214.172]) by ietfa.amsl.com (Postfix) with ESMTP id 1FFCA11E80EB for ; Thu, 28 Jun 2012 23:59:06 -0700 (PDT) Received: by obbwc20 with SMTP id wc20so4794099obb.31 for ; Thu, 28 Jun 2012 23:59:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=u78/gA7M6J/CroUoCO6hzTprgTpdZ0MloiK8Pq3PKK0=; b=eTvdkqXKgt85nZWKCleL6qdMHWJB7hl4/C+2TtCwJTB/4luw+UnZAlZe/kYbNSGEVk jBVh7PL0uXA3K8o5k6xwMx2vOtLxuwh4JPkpeixDTFMuv2oe6Mx6kfY+OJ6nPAPf81jS n093kScXjEOB35PKKEeRX+jiNGozUa4+wp3lVagxAIMTyIYRxhy0PfXLa/dqCAuHab8r IlpU7k/8OlOMpmVGHDX+6Tp8jpojMdfjvd+UYlsvfZSnxl80wphBxU3spvzJsr727+1v HCQvr/RVZfdJ8pmVkQ04MjGJks1lkKrEJ/dm9PLvhy8NaFBh5uSBIcGhMPqng4HxhKcs CyNw== MIME-Version: 1.0 Received: by 10.50.46.232 with SMTP id y8mr221886igm.57.1340953145315; Thu, 28 Jun 2012 23:59:05 -0700 (PDT) Received: by 10.50.193.193 with HTTP; Thu, 28 Jun 2012 23:59:05 -0700 (PDT) Date: Fri, 29 Jun 2012 08:59:05 +0200 Message-ID: From: Dominik Elsbroek To: ipfix@ietf.org Content-Type: text/plain; charset=ISO-8859-1 X-Mailman-Approved-At: Fri, 29 Jun 2012 00:06:14 -0700 Subject: [IPFIX] Export Switch Port Identifier X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Jun 2012 06:59:06 -0000 Hi list, I am currently working on an lightweight network intrusion detection system based on IPFIX flow-records. A was seeking for an element that identifies the switch or router port at which the traffic described by flow-record has emerged. This is needed to detect for MAC Address spoofing and might be useful in some more scenarios. Is there such a field? I would expect such a field to be defined in RFC 5102 or any RFC updating RFC 5102? I just found transport layer ports of exporter and collector. Any hints and help are very appreciated. Kind regards, Dominik From tasaxena@cisco.com Fri Jun 29 04:40:29 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 094CB21F86EE for ; Fri, 29 Jun 2012 04:40:29 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.599 X-Spam-Level: X-Spam-Status: No, score=-10.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id RyJoBAYpYy0T for ; Fri, 29 Jun 2012 04:40:27 -0700 (PDT) Received: from rcdn-iport-9.cisco.com (rcdn-iport-9.cisco.com [173.37.86.80]) by ietfa.amsl.com (Postfix) with ESMTP id A5CF921F86A4 for ; Fri, 29 Jun 2012 04:40:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=1147; q=dns/txt; s=iport; t=1340970027; x=1342179627; h=from:to:subject:date:message-id:references:in-reply-to: content-transfer-encoding:mime-version; bh=CvMSlTrnpoLIrxHlJuW/cX9+Z7zg/wClQVv+cMGTxf0=; b=JzBL5kgMfnwuYbGdkPGdWOcFXqEp5BK0KE+CfdJyx3mLc8bIyyVqxoqF fCU2oX7x1brlPFR7MB4Uh8y8GhnlC2Qzyq46Z0SyJru0rzFooHbhGKthQ AaC/Sn1aMb22wDvyUYWhCugxB44/k0NSfsl35bVADZmrjjTEZnqs5fAtF A=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Av8EAAqT7U+tJV2b/2dsb2JhbABFtkuBB4IYAQEBBAEBAQ8BJzQXBAIBCBEEAQELFAkHJwsUCQgCBAESCBqHaQubTKBdBIs3hSpgA6NPgWaCXw X-IronPort-AV: E=Sophos;i="4.77,497,1336348800"; d="scan'208";a="94237897" Received: from rcdn-core-4.cisco.com ([173.37.93.155]) by rcdn-iport-9.cisco.com with ESMTP; 29 Jun 2012 11:40:27 +0000 Received: from xhc-rcd-x02.cisco.com (xhc-rcd-x02.cisco.com [173.37.183.76]) by rcdn-core-4.cisco.com (8.14.5/8.14.5) with ESMTP id q5TBeRPw008902 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Fri, 29 Jun 2012 11:40:27 GMT Received: from xmb-aln-x06.cisco.com ([169.254.1.14]) by xhc-rcd-x02.cisco.com ([173.37.183.76]) with mapi id 14.02.0298.004; Fri, 29 Jun 2012 06:40:27 -0500 From: "Tarun Saxena (tasaxena)" To: Dominik Elsbroek , "ipfix@ietf.org" Thread-Topic: [IPFIX] Export Switch Port Identifier Thread-Index: AQHNVcW1JwQZCXAHqUm+mPJ8IxNeYZcRK7ew Date: Fri, 29 Jun 2012 11:40:26 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [72.163.207.235] x-tm-as-product-ver: SMEX-10.2.0.1135-7.000.1014-19004.005 x-tm-as-result: No--31.426400-8.000000-31 x-tm-as-user-approved-sender: No x-tm-as-user-blocked-sender: No Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [IPFIX] Export Switch Port Identifier X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Jun 2012 11:40:29 -0000 Dominik, By emerged, do you mean "entered" the router/switch or "exited" it? This is= very basic information and is available in ingressInterface/egressinterfac= e fields. Thanks Tarun -----Original Message----- From: ipfix-bounces@ietf.org [mailto:ipfix-bounces@ietf.org] On Behalf Of D= ominik Elsbroek Sent: Friday, June 29, 2012 12:29 PM To: ipfix@ietf.org Subject: [IPFIX] Export Switch Port Identifier Hi list, I am currently working on an lightweight network intrusion detection system based on IPFIX flow-records. A was seeking for an element that identifies the switch or router port at which the traffic described by flow-record has emerged. This is needed to detect for MAC Address spoofing and might be useful in some more scenarios. Is there such a field? I would expect such a field to be defined in RFC 5102 or any RFC updating RFC 5102? I just found transport layer ports of exporter and collector. Any hints and help are very appreciated. Kind regards, Dominik _______________________________________________ IPFIX mailing list IPFIX@ietf.org https://www.ietf.org/mailman/listinfo/ipfix From andrewf@plixer.com Fri Jun 29 07:26:29 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 59F3621F86C1 for ; Fri, 29 Jun 2012 07:26:29 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.598 X-Spam-Level: X-Spam-Status: No, score=-2.598 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zMB-O96KQMr5 for ; Fri, 29 Jun 2012 07:26:28 -0700 (PDT) Received: from smtp.plixer.com (smtp.plixer.com [66.186.184.193]) by ietfa.amsl.com (Postfix) with ESMTP id 2850E21F8737 for ; Fri, 29 Jun 2012 07:26:28 -0700 (PDT) Received: from [10.100.1.132] ([66.186.184.193]) by smtp.plixer.com over TLS secured channel with Microsoft SMTPSVC(6.0.3790.4675); Fri, 29 Jun 2012 10:26:27 -0400 Message-ID: <4FEDBB12.8070501@plixer.com> Date: Fri, 29 Jun 2012 10:26:26 -0400 From: Andrew Feren User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:16.0) Gecko/16.0 Thunderbird/16.0a1 MIME-Version: 1.0 To: ipfix@ietf.org References: In-Reply-To: Content-Type: multipart/alternative; boundary="------------070607060705040708000300" X-OriginalArrivalTime: 29 Jun 2012 14:26:27.0023 (UTC) FILETIME=[2B04ADF0:01CD5603] Subject: Re: [IPFIX] Export Switch Port Identifier X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Jun 2012 14:26:29 -0000 This is a multi-part message in MIME format. --------------070607060705040708000300 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Hi Dominik, Tarun is right about the ingress/egress interface fields. For a current list of information elements your best source is probably IANA ( https://www.iana.org/assignments/ipfix/ipfix.xml) -Andrew On 06/29/2012 07:40 AM, Tarun Saxena (tasaxena) wrote: > Dominik, > > By emerged, do you mean "entered" the router/switch or "exited" it? This is very basic information and is available in ingressInterface/egressinterface fields. > > Thanks > Tarun > > -----Original Message----- > From: ipfix-bounces@ietf.org [mailto:ipfix-bounces@ietf.org] On Behalf Of Dominik Elsbroek > Sent: Friday, June 29, 2012 12:29 PM > To: ipfix@ietf.org > Subject: [IPFIX] Export Switch Port Identifier > > Hi list, > > I am currently working on an lightweight network intrusion detection > system based on IPFIX flow-records. A was seeking for an element that > identifies the switch or router port at which the traffic described by > flow-record has emerged. This is needed to detect for MAC Address > spoofing and might be useful in some more scenarios. > > Is there such a field? I would expect such a field to be defined in > RFC 5102 or any RFC updating RFC 5102? I just found transport layer > ports of exporter and collector. Any hints and help are very > appreciated. > > Kind regards, > Dominik > _______________________________________________ > IPFIX mailing list > IPFIX@ietf.org > https://www.ietf.org/mailman/listinfo/ipfix > _______________________________________________ > IPFIX mailing list > IPFIX@ietf.org > https://www.ietf.org/mailman/listinfo/ipfix --------------070607060705040708000300 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit
Hi Dominik,

Tarun is right about the ingress/egress interface fields.

For a current list of information elements your best source is probably  IANA ( https://www.iana.org/assignments/ipfix/ipfix.xml)

-Andrew

On 06/29/2012 07:40 AM, Tarun Saxena (tasaxena) wrote:
Dominik,

By emerged, do you mean "entered" the router/switch or "exited" it? This is very basic information and is available in ingressInterface/egressinterface fields.

Thanks
Tarun

-----Original Message-----
From: ipfix-bounces@ietf.org [mailto:ipfix-bounces@ietf.org] On Behalf Of Dominik Elsbroek
Sent: Friday, June 29, 2012 12:29 PM
To: ipfix@ietf.org
Subject: [IPFIX] Export Switch Port Identifier

Hi list,

I am currently working on an lightweight network intrusion detection
system based on IPFIX flow-records. A was seeking for an element that
identifies the switch or router port at which the traffic described by
flow-record has emerged. This is needed to detect for MAC Address
spoofing and might be useful in some more scenarios.

Is there such a field? I would expect such a field to be defined in
RFC 5102 or any RFC updating RFC 5102? I just found transport layer
ports of exporter and collector. Any hints and help are very
appreciated.

Kind regards,
Dominik
_______________________________________________
IPFIX mailing list
IPFIX@ietf.org
https://www.ietf.org/mailman/listinfo/ipfix
_______________________________________________
IPFIX mailing list
IPFIX@ietf.org
https://www.ietf.org/mailman/listinfo/ipfix

--------------070607060705040708000300-- From dominik.elsbroek@gmail.com Fri Jun 29 08:08:02 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0B9FE21F861C for ; Fri, 29 Jun 2012 08:08:02 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.562 X-Spam-Level: X-Spam-Status: No, score=-3.562 tagged_above=-999 required=5 tests=[AWL=0.038, BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id pHpRHqZ+ET3j for ; Fri, 29 Jun 2012 08:08:00 -0700 (PDT) Received: from mail-gg0-f172.google.com (mail-gg0-f172.google.com [209.85.161.172]) by ietfa.amsl.com (Postfix) with ESMTP id D59FC21F8615 for ; Fri, 29 Jun 2012 08:07:59 -0700 (PDT) Received: by ggnc4 with SMTP id c4so3138407ggn.31 for ; Fri, 29 Jun 2012 08:07:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=nyGHfGDEBQK5506vfa6rmaTK8ncq2bAAnDr4uZxoQF0=; b=r1dxGtBLwQuz0AaFDcUFRdA4dkCZLqizxSgPYXL9kX6xCwg7pIeJhm23Wk4VKjEvLs z0ovVsFs2VwvNaj8Mk796+Jwn7O6vDv4sudrtL/4AnDKvqv0k1e6F7XKFQazBDMAlQ39 0rIkvKVvsO3ZZ1mcKQozJIqrCx78SIShAZfK1X3HLlbKPv+3tRL1KBiItL+GzWgZWuL1 YTN/0m+OIbN8nIf+Ht7PJFJVBYohpVFlIVm+z3thShrcktsrf18rdOH6RrxVUsQHOud3 UmivyayGJnRYHRQrwiTXl0V2PgCUjG5N8hNYaJxuh8xbRxi3o9V2Q7opdnxow7T2Enhe vXsw== MIME-Version: 1.0 Received: by 10.50.193.196 with SMTP id hq4mr1735304igc.57.1340982478993; Fri, 29 Jun 2012 08:07:58 -0700 (PDT) Received: by 10.50.193.193 with HTTP; Fri, 29 Jun 2012 08:07:58 -0700 (PDT) In-Reply-To: <4FEDBB12.8070501@plixer.com> References: <4FEDBB12.8070501@plixer.com> Date: Fri, 29 Jun 2012 17:07:58 +0200 Message-ID: From: Dominik Elsbroek To: Andrew Feren Content-Type: text/plain; charset=ISO-8859-1 X-Mailman-Approved-At: Fri, 29 Jun 2012 09:09:12 -0700 Cc: ipfix@ietf.org Subject: Re: [IPFIX] Export Switch Port Identifier X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Jun 2012 15:08:02 -0000 Hi Tarun, hi Andrew, thank you very much for your help. Both fields will help me with my work I think. Thank you very much! Dominik On Fri, Jun 29, 2012 at 4:26 PM, Andrew Feren wrote: > Hi Dominik, > > Tarun is right about the ingress/egress interface fields. > > For a current list of information elements your best source is probably > IANA ( https://www.iana.org/assignments/ipfix/ipfix.xml) > > -Andrew > > > On 06/29/2012 07:40 AM, Tarun Saxena (tasaxena) wrote: > > Dominik, > > By emerged, do you mean "entered" the router/switch or "exited" it? This is > very basic information and is available in ingressInterface/egressinterface > fields. > > Thanks > Tarun > > -----Original Message----- > From: ipfix-bounces@ietf.org [mailto:ipfix-bounces@ietf.org] On Behalf Of > Dominik Elsbroek > Sent: Friday, June 29, 2012 12:29 PM > To: ipfix@ietf.org > Subject: [IPFIX] Export Switch Port Identifier > > Hi list, > > I am currently working on an lightweight network intrusion detection > system based on IPFIX flow-records. A was seeking for an element that > identifies the switch or router port at which the traffic described by > flow-record has emerged. This is needed to detect for MAC Address > spoofing and might be useful in some more scenarios. > > Is there such a field? I would expect such a field to be defined in > RFC 5102 or any RFC updating RFC 5102? I just found transport layer > ports of exporter and collector. Any hints and help are very > appreciated. > > Kind regards, > Dominik > _______________________________________________ > IPFIX mailing list > IPFIX@ietf.org > https://www.ietf.org/mailman/listinfo/ipfix > _______________________________________________ > IPFIX mailing list > IPFIX@ietf.org > https://www.ietf.org/mailman/listinfo/ipfix > > > > _______________________________________________ > IPFIX mailing list > IPFIX@ietf.org > https://www.ietf.org/mailman/listinfo/ipfix > From muenz@net.in.tum.de Sat Jun 30 03:00:59 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 023D121F869F for ; Sat, 30 Jun 2012 03:00:59 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.174 X-Spam-Level: X-Spam-Status: No, score=-6.174 tagged_above=-999 required=5 tests=[AWL=0.075, BAYES_00=-2.599, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gQDyz+jmxL7d for ; Sat, 30 Jun 2012 03:00:57 -0700 (PDT) Received: from mail-out1.informatik.tu-muenchen.de (mail-out1.informatik.tu-muenchen.de [131.159.0.8]) by ietfa.amsl.com (Postfix) with ESMTP id 626E221F8674 for ; Sat, 30 Jun 2012 03:00:56 -0700 (PDT) Received: from [192.168.2.34] (e181143108.adsl.alicedsl.de [85.181.143.108]) by mail.net.in.tum.de (Postfix) with ESMTPSA id C7CC420272CE; Sat, 30 Jun 2012 12:00:55 +0200 (CEST) Message-ID: <4FEECE19.4080003@net.in.tum.de> Date: Sat, 30 Jun 2012 11:59:53 +0200 From: Gerhard Muenz User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:13.0) Gecko/20120614 Thunderbird/13.0.1 MIME-Version: 1.0 To: Andrew Johnson References: <4FDC9E8B.1070800@cisco.com> <4FDE112B.40706@net.in.tum.de> <4FE88612.1080702@cisco.com> <4FE896DE.6050908@net.in.tum.de> <4FE89B1A.5060301@cisco.com> <4FE89EE0.9020409@net.in.tum.de> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: ipfix-ads@tools.ietf.org, IETF IPFIX Working Group , "ipfix-chairs@tools.ietf.org" Subject: Re: [IPFIX] new monitoring interval information elements X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Jun 2012 10:00:59 -0000 Andrew, As I pointed out in my previous mail, the window IEs do not seem to be compatible with the descriptions of existing *TotalCount IEs because these refer to the time since (re-)initialization. You could use *DeltaCount, though. Gerhard On 29.06.2012 01:25, Andrew Johnson wrote: > Hi Gerhard > > Thanks for your inputs on this thread. It seems to me that the idea of "re-initialising" the Metering Process is pretty confusing, in light of what we're actually doing. I think my mental mode of what is happening is a bit different from Paul's, so let me start at the beginning, show how I think of the behaviour, and see what we can come up with to solve our problem. > > We would like to split the monitored packet streams into discrete buckets, based on time. > > t0 -----> t1 -----> t2 -----> t3 -----> t4 -----> t5 -----> > > > The packet stream is processed by the Metering Process as usual, and flows form in the cache in the usual manner. However, observations from one interval won't be merged with observations from a different interval. Ideally, at the end of each interval, all flows are exported and flushed from the cache. > > There are several ways to think about this. One way is to explicitly flush the cache at the end of each interval (i.e. reset / reinitialise the Metering Process). The way I see it though, we could just make an identifier of the current interval a key field and apply the optimisation of allowing flows from outside the current interval to be flushed from the cache ASAP, since we know they won't be updated again. > > Hopefully that gives a bit of a bigger picture. What we are looking for are two new fields that we can add to the flows, that identify the beginning at end of the interval they were observed in. In some circumstances the device may not fully stick to the provided interval window, so we can't rely on the end of the interval really being the intervalStart + intervalSize. > > I was thinking of: > > observationWindowStart dateTimeMilliseconds The absolute timestamp of the beginning of the current Observation Window. > observationWindowEnd dateTimeMilliseconds The absolute timestamp of the end of the current Observation Window. > > > However, Paul rightly pointed out to me that IPFIX doesn't have any concept of an Observation Window, and we're not sure how we can define that in description of the IE. The idea of re-initialising the Metering Process may fit in with the current IPFIX model, but I don't think it properly conveys the behaviour we're after. > > > Kind regards, > > Andrew > > > On 25 Jun 2012, at 18:24, Gerhard Muenz wrote: >> Paul, >> >> It depends on what kind of interval you want to specify. >> >> If you want to specify the (re-)initialization time of the Metering Process, why not name the IE meteringProcessSysUpTime or (if you do not like sysup) meteringProcessInitializationTime? >> And mention (re-)initialization in the description? >> Dito for exportingProcessInitializationTime etc. >> >> If we need the stop time, you can use meteringProcessStopTime or so. >> >> If you want to define a new interval which is not starting at (re-)initialization time but later, then your definitions are ok. But this means that you need to specify new counter elements for this interval. >> >> Thanks, >> Gerhard >> >> >> On 25.06.2012 19:08, Paul Aitken wrote: >>> Gerhard, >>> >>>> Ok, I see the difference to discontinuity time. >>>> >>>> Some further thoughts: >>>> >>>> The common understanding is that the "time since re-initialization" >>>> means sysUpTime. For example, have a look at the description of >>>> flowEndSysUpTime: >>>> "The relative timestamp of the last packet of this Flow. It indicates >>>> the number of milliseconds since the last (re-)initialization of the >>>> IPFIX Device (sysUpTime)." >>> >>> Yes, sysUpTime (so specifically, flowStartSysUpTime and >>> flowEndSysUpTime), are times "since the last (re-)initialization of the >>> IPFIX Device". >>> >>> These are not necessarily related to the metering process. >>> >>> Taking some other examples: octetTotalCount, packetTotalCount, >>> droppedOctetTotalCount, droppedPacketTotalCount, observedFlowTotalCount, >>> ignoredPacketTotalCount, ignoredOctetTotalCount, >>> postMCastOctetTotalCount, ... are all of the form: >>> >>> "The total number of X since the Metering Process >>> (re-)initialization for this Observation Point." >>> >>> We could suppose that the Metering Process initialised at system uptime. >>> However, that may not be the case: it could have started any amount of >>> time later. eg, a user could configure a Metering Process at any time. >>> Or, an automated system could start and stop Metering Processes >>> according to different times of day or different network conditions. >>> Without timestamps to tell us, we simply do not know. >>> >>> >>> >>>> Your proposed IEs do not mention neither (re-)initialization nor >>>> sysUpTime. They rather seem to specify a time interval which is >>>> located somewhere within the sysuptime interval. Hence, the interval >>>> does not work to define the time frame of exportedMessageTotalCount, >>>> for example. You would need to define a new IE >>>> exportedMessageInMonitoringInterval. >>> >>> Indeed, because the IEs which I'm specifying here are particular to the >>> Monitoring Process, whereas exportedMessageTotalCount seems to require a >>> time reference for the Exporting Process. It's a related, though >>> different, issue. >>> >>> (ie, the Exporting Process may start from system init, or at any time >>> since then. It may have been started before, or after, the Metering >>> Process. Without a suitable timestamp IE, we simply do not know.) >>> >>> >>>> The different IE descriptions suggest that the time of >>>> re-initialization (i.e. sysuptime) can be different for Metering >>>> Process, Exporting Process etc. >>> >>> Indeed this is so. There is no necessity for these to be the same. >>> >>> >>>> It seems that we need IE to export sysUpTime for each process. >>> >>> We agree. >>> >>> So in principal, you understand what I'm trying to do. However, you >>> would like clearer definitions of the Information Elements? >>> >>> Thanks, >>> P. >>> >> >> >> >> _______________________________________________ >> IPFIX mailing list >> IPFIX@ietf.org >> https://www.ietf.org/mailman/listinfo/ipfix >