Subject: [ipfix] [Fwd: I-D ACTION:draft-dressler-ipfix-aggregation-01.txt]

From: Falko Dressler <[email-masked]>
Date: Sat, 23 Jul 2005 10:55:04 +0200

I'd like to announce the -01 version of the IPFIX aggregation draft.
Since the presentation at the previous IETF, we received only positive 
feedback. Compared to -00, many enhancements and clarifications were 
included. We believe that the draft is now already very mature.
I requested a slot for the Paris meeting to discuss open issues and the 
next steps.

Cheers,
Falko

-------- Original Message --------
Subject: I-D ACTION:draft-dressler-ipfix-aggregation-01.txt
Date: Thu, 21 Jul 2005 15:50:02 -0400
From: [email-masked]
Reply-To: [email-masked]
To: [email-masked]

A New Internet-Draft is available from the on-line Internet-Drafts 
directories.

	Title		: IPFIX Aggregation
	Author(s)	: F. Dressler, et al.
	Filename	: draft-dressler-ipfix-aggregation-01.txt
	Pages		: 18
	Date		: 2005-7-21
	
IPFIX Aggregation describes a methodology for reducing the amount of
    measurement data exchanged between monitoring devices (IPFIX
    exporters) and analyzers (IPFIX collectors).  Using aggregation
    techniques, measurement information of multiple similar flows is
    aggregated into one meta-flow record.  The degree of similarity can
    be customized using aggregation rules.

    To ensure efficient communication of both aggregated flows and the
    aggregation rules used the IPFIX Protocol and IPFIX Information Model
    are slightly extended to allow two new abstract data types and a new
    type of template set.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-dressler-ipfix-aggregation-01.txt

To remove yourself from the I-D Announcement list, send a message to
[email-masked] with the word unsubscribe in the body of 
the message.
You can also visit https://www1.ietf.org/mailman/listinfo/I-D-announce
to change your subscription settings.

Internet-Drafts are also available by anonymous FTP. Login with the username
"anonymous" and a password of your e-mail address. After logging in,
type "cd internet-drafts" and then
	"get draft-dressler-ipfix-aggregation-01.txt".

A list of Internet-Drafts directories can be found in
http://www.ietf.org/shadow.html
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

Internet-Drafts can also be obtained by e-mail.

Send a message to:
	[email-masked].
In the body type:
	"FILE /internet-drafts/draft-dressler-ipfix-aggregation-01.txt".
	
NOTE:	The mail server at ietf.org can return the document in
	MIME-encoded form by using the "mpack" utility.  To use this
	feature, insert the command "ENCODING mime" before the "FILE"
	command.  To decode the response(s), you will need "munpack" or
	a MIME-compliant mail reader.  Different MIME-compliant mail readers
	exhibit different behavior, especially when dealing with
	"multipart" MIME messages (i.e. documents which have been split
	up into multiple messages), so check your local documentation on
	how to manipulate these messages.

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

-- 
Dr.-Ing. Falko Dressler
Computer Networks and Communication Systems
University of Erlangen-Nuremberg, Germany
Phone: +49 9131 85-27914 / Fax: +49 9131 85-27409
EMail: [email-masked] / [email-masked]
WWW: http://www7.informatik.uni-erlangen.de/~dressler/

Network Working Group                                        F. Dressler
Internet-Draft                                                 C. Sommer
Expires: January 21, 2006                         University of Erlangen
                                                                 G. Munz
                                                  University of Tubingen
                                                           July 20, 2005

                           IPFIX Aggregation
               <draft-dressler-ipfix-aggregation-01.txt>

Status of this Memo

   By submitting this Internet-Draft, each author represents that any
   applicable patent or other IPR claims of which he or she is aware
   have been or will be disclosed, and any of which he or she becomes
   aware will be disclosed, in accordance with Section 6 of BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as Internet-
   Drafts.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/ietf/1id-abstracts.txt.

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html.

   This Internet-Draft will expire on January 21, 2006.

Copyright Notice

   Copyright (C) The Internet Society (2005).

Abstract

   IPFIX Aggregation describes a methodology for reducing the amount of
   measurement data exchanged between monitoring devices (IPFIX
   exporters) and analyzers (IPFIX collectors).  Using aggregation
   techniques, measurement information of multiple similar flows is
   aggregated into one meta-flow record.  The degree of similarity can
   be customized using aggregation rules.

Dressler, et al.    draft-dressler-ipfix-aggregation-01.txt     [Page 1]

Internet-Draft              IPFIX Aggregation                  July 2005

   To ensure efficient communication of both aggregated flows and the
   aggregation rules used the IPFIX Protocol and IPFIX Information Model
   are slightly extended to allow two new abstract data types and a new
   type of template set.

Table of Contents

   1.  Introduction . . . . . . . . . . . . . . . . . . . . . . . . .  3

   2.  Aggregation Techniques . . . . . . . . . . . . . . . . . . . .  3
     2.1   Architecture . . . . . . . . . . . . . . . . . . . . . . .  3
     2.2   Methodology  . . . . . . . . . . . . . . . . . . . . . . .  4
       2.2.1   Meta-Flows and Meta-Flow Records . . . . . . . . . . .  4
       2.2.2   Aggregation Rules  . . . . . . . . . . . . . . . . . .  5
       2.2.3   Rule Semantics . . . . . . . . . . . . . . . . . . . .  7
       2.2.4   Special Fields . . . . . . . . . . . . . . . . . . . .  7
       2.2.5   Shared Properties  . . . . . . . . . . . . . . . . . .  8
       2.2.6   Example Rule . . . . . . . . . . . . . . . . . . . . .  8
     2.3   Application Examples . . . . . . . . . . . . . . . . . . .  9
       2.3.1   Charging . . . . . . . . . . . . . . . . . . . . . . .  9
       2.3.2   Intrusion Detection  . . . . . . . . . . . . . . . . .  9

   3.  IPFIX Extensions . . . . . . . . . . . . . . . . . . . . . . . 10
     3.1   ipv4Network  . . . . . . . . . . . . . . . . . . . . . . . 10
     3.2   portRanges . . . . . . . . . . . . . . . . . . . . . . . . 10
     3.3   Data Template  . . . . . . . . . . . . . . . . . . . . . . 11
     3.4   Example  . . . . . . . . . . . . . . . . . . . . . . . . . 14

   4.  Security considerations  . . . . . . . . . . . . . . . . . . . 16

   5.  References . . . . . . . . . . . . . . . . . . . . . . . . . . 16
     5.1   Normative References . . . . . . . . . . . . . . . . . . . 16
     5.2   Informative References . . . . . . . . . . . . . . . . . . 16

       Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . 17

       Intellectual Property and Copyright Statements . . . . . . . . 18

Dressler, et al.    draft-dressler-ipfix-aggregation-01.txt     [Page 2]

Internet-Draft              IPFIX Aggregation                  July 2005

1.  Introduction

   Flow measurement in high-speed large-scale networks easily produces a
   huge amount of data that can not be handled by a central analyzer
   without having been preprocessed.  Flow aggregators alleviate this
   problem by selectively discarding measurement information and merging
   multiple individual flows into a smaller number of meta-flows, thus
   reducing both the number of exported flow records and the amount of
   data carried by each record.  This document proposes how to design
   and implement IPFIX aggregators and introduces appropriate extensions
   to the IPFIX Protocol.

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in [RFC2119].

2.  Aggregation Techniques

2.1  Architecture

   Aggregation of measurement data may take place at two possible
   stages:

   An internal aggregator, as depicted in Figure 1, is implemented as a
   process running in an IPFIX enabled device.  It aggregates raw data
   generated by multiple metering processes and exports them as meta-
   flow records.

   An external aggregator, called concentrator in IPFIX terminology, may
   be used where the deployed monitoring devices cannot be modified to
   incorporate an internal aggregator.  Furthermore, concentrators
   enable cascaded, multi-level aggregation of flow information.  As
   shown in Figure 2, a concentrator receives flow records from
   monitoring devices and/or lower-level concentrators and exports the
   aggregated meta-flow information to higher-level concentrators and/or
   analyzers.

Dressler, et al.    draft-dressler-ipfix-aggregation-01.txt     [Page 3]

Internet-Draft              IPFIX Aggregation                  July 2005

   +------------------------------------------+     +--------------+
   | IPFIX-enabled device       .---.   .---. |     |              |
   | .--------------------.     | A |   |   | | .-->|   Analyzer   |
   | | Metering Process 1 |-.   | g |   | E | | |   |              |
   | `--------------------' |   | g |   | x | | |   +--------------+
   |                        |   | r |   | p |---'
   |           .            '-->| e |   | o | |
   |           .                | g |-->| r | |
   |           .            .-->| a |   | t |---.
   |                        |   | t |   | e | | |   +--------------+
   | .--------------------. |   | o |   | r | | |   |              |
   | | Metering Process N |-'   | r |   |   | | '-->| Concentrator |
   | `--------------------'     `---'   `---' |     |              |
   +------------------------------------------+     +--------------+

                      Figure 1: Internal Aggregation

   +--------------+    +-----------------------+     +--------------+
   |              |    | Concentrator          |     |              |
   | Concentrator |-.  | .---.   .---.   .---. | .-->|   Analyzer   |
   |              | |  | | C |   | A |   |   | | |   |              |
   +--------------+ |  | | o |   | g |   | E | | |   +--------------+
                    '--->| l |   | g |   | x |---'
                       | | l |   | r |   | p | |
                       | | e |-->| e |-->| o | |
                       | | c |   | g |   | r | |
                    .--->| t |   | a |   | t |---.
   +--------------+ |  | | o |   | t |   | e | | |   +--------------+
   |              | |  | | r |   | o |   | r | | |   |              |
   | IPFIX device |-'  | |   |   | r |   |   | | '-->| Concentrator |
   |              |    | `---'   `---'   `---' |     |              |
   +--------------+    +-----------------------+     +--------------+

                      Figure 2: External Aggregation

2.2  Methodology

2.2.1  Meta-Flows and Meta-Flow Records

   We define the term meta-flow as an aggregate of individual flows that
   share a set of common properties.  As an example, flows directed to
   the same destination address can be aggregated into a single meta-
   flow.  Each resulting meta-flow record MAY contain a single counter
   for all packets directed to a given destination within a given time
   interval.  All flow properties that distinguish the aggregated flows
   (e.g. the source address) will not be contained in the meta-flow

Dressler, et al.    draft-dressler-ipfix-aggregation-01.txt     [Page 4]

Internet-Draft              IPFIX Aggregation                  July 2005

   record any more.  The content of the meta-flow record as well as an
   optional set of common properties for all aggregated flows is defined
   using an aggregation rule.  A Data Template, as proposed in
   Section 3.3, MAY be used to define the structure of the meta-flow
   record and to inform the analyzer about the applied aggregation rule
   and the common properties.

2.2.2  Aggregation Rules

   Regarding aggregation methodologies, a simple, rule-based approach is
   proposed.  The aggregator is supplied a list of user-defined
   aggregation rules.  An aggregation rule consists of multiple
   aggregation instructions, one for each IPFIX fields that is to be
   considered by the aggregator.  An aggregation instruction comprises
   the following elements:
   o  The IPFIX field the aggregation instruction refers to (e.g.
      destinationIPv4Address).  Only flows that contain the field
      mentioned will be considered for aggregation in the meta-flow.
   o  An optional pattern for this field that restricts the aggregation
      to those flows that match the given value(s) (e.g. 10.10.0.0/16).
      Only flows that match all patterns of the rule will be aggregated
      in the meta-flow.
   o  One of the field modifiers "discard", "keep", "aggregate", or
      "mask" that specifies how this field is treated by the aggregator
      and if it is included in the meta-flow record or not.

   With this definition of aggregation instructions each rule also
   unambiguosly defines the content of the meta-flow record as well as
   the template to export the meta-flow information.  If and how a field
   is present in the meta-flow record, depends on the field modifier and
   is explained below.  Fields that do not appear in any of the
   aggregation instructions, are not part of the meta-flow record.
   Since the export of a meta-flow record requires an appropriate
   template, a one-to-one relationship between rules and templates can
   be established.

   The following field modifiers are applicable to Flow Keys as defined
   in [I-D.ietf-ipfix-architecture].
   discard:
      The field is not included in the meta-flow records, i.e. meta-
      flows are not distinguishable with respect to this field.
      Incoming flow records with different values for this IPFIX field
      are merged.

Dressler, et al.    draft-dressler-ipfix-aggregation-01.txt     [Page 5]

Internet-Draft              IPFIX Aggregation                  July 2005

   keep:
      The field is included in the meta-flow record, i.e. meta-flows are
      distinguishable with respect to this field.  Incoming flow records
      with different values for this field are not merged, but
      contribute to different meta-flows instead.

   mask/n:
      (Applicable to IP addresses only) The field is included in the
      meta-flow record, but the number of significant bits reduced to n
      bits.  Incoming flow records with IP addresses belonging to the
      same subnet are merged, so meta-flows are distinguishable with
      respect to resulting subnet addresses only.  In accordance with
      the IPFIX Information Model the resulting IP address is
      transferred as one IP prefix field and one IP mask field.

   If an aggregation instruction refers to one of the field types
   mentioned in Section 2.2.4, no pattern must be specified.  The only
   possible field modifier is:
   aggregate:
      The field is included in the meta-flow record.  The determination
      of the value depends on the field type and is specified in
      Section 2.2.4.  As an example, a counter field in a meta-flow
      record is assigned the sum of the corresponding values in the
      incoming flow records.

   Using the Data Template proposed in Section 3.3, the receiving
   collector is informed about the current rule set.  Table 1 summarizes
   the field modifiers and the resulting information in the meta-flow
   records and Data Templates, respectively.

   +----------------+----------------+----------------+----------------+
   | field modifier | pattern        | field in       | fixed-value    |
   |                |                | meta-flow      | field in Data  |
   |                |                | record         | Template       |
   +----------------+----------------+----------------+----------------+
   | discard        | no             | N/A            | N/A            |
   | discard        | yes            | N/A            | yes, contains  |
   |                |                |                | pattern        |
   | keep           | no             | yes            | N/A            |
   | keep           | yes            | yes, if        | yes, contains  |
   |                |                | pattern        | pattern        |
   |                |                | specifies a    |                |
   |                |                | range of       |                |
   |                |                | values         |                |

Dressler, et al.    draft-dressler-ipfix-aggregation-01.txt     [Page 6]

Internet-Draft              IPFIX Aggregation                  July 2005

   | mask           | no             | yes, IP        | N/A            |
   |                |                | network        |                |
   |                |                | address        |                |
   | mask           | yes            | yes, IP        | yes, contains  |
   |                |                | network        | pattern        |
   |                |                | address        |                |
   +----------------+----------------+----------------+----------------+

     Table 1: Relation between field modifiers, meta-flow records, and
                              Data Templates

2.2.3  Rule Semantics

   By default, incoming flow records are checked against every
   aggregation rule.  If a rule matches, i.e. if the flow record
   comprises all required fields and matches all given patterns, the
   field modifiers are applied and the corresponding meta-flow record is
   generated or updated.  Incoming flow records that match multiple
   rules thus contribute to multiple meta-flows.

   In some cases, it is prefered that an incoming flow record that
   matched a certain rule is not checked against other rules in order to
   avoid that this flow contributes to multiple meta-flows.  Therefore,
   it is possible to indicate a preceding rule for each aggregation rule
   that has to be check before.  The current rule is not applied if the
   flow record already matched the preceding rule.  Using the preceding
   rule relationship, rules can be organized in rule chains and rule
   trees where only the first matching rule is applied in every chain or
   branch.  Rules that do not define a preceding rule are checked
   against all incoming flow records and constitute the beginning of a
   rule chain or the root of a rule tree.  Consequently, each flow
   record is counted only once within a single chain or branch, or not
   at all if none of the rules matches.

   The Preceding Rule field in the header of the Data Template (see
   Section 3.3) is used to identify the preceding rule by its Template
   ID.  If set to 0, there is no preceding rule and the rule is checked
   against all incoming records.

2.2.4  Special Fields

   Fields that do not carry information about Flow Keys (see [I-D.ietf-
   ipfix-architecture]) but metering information, such as counters,
   timestamps etc., require special treatment.  For example, the meta-
   flow's start timestamp is set to the minimum of the comprising flows'
   start timestamps.  Packet and byte counters of aggregated flows are
   summed up.

Dressler, et al.    draft-dressler-ipfix-aggregation-01.txt     [Page 7]

Internet-Draft              IPFIX Aggregation                  July 2005

   Table 2 contains an overview of such fields for which a special
   treatment is proposed.  Refer to the IPFIX Information Model
   [I-D.ietf-ipfix-info] for a description of the fields mentioned.

                 +-----------------------+-----------+
                 | Field                 | Set To    |
                 +-----------------------+-----------+
                 | minimumPacketLength   | minimum   |
                 | minimumTtl            | minimum   |
                 | flowStartSeconds      | minimum   |
                 | flowStartMilliSeconds | minimum   |
                 | maximumPacketLength   | maximum   |
                 | maximumTtl            | maximum   |
                 | flowEndSeconds        | maximum   |
                 | flowEndMilliSeconds   | maximum   |
                 | ipv6OptionHeaders     | binary OR |
                 | tcpControlBits        | binary OR |
                 | octetDeltaCount       | sum       |
                 | packetDeltaCount      | sum       |
                 +-----------------------+-----------+

           Table 2: Information with Implicit Aggregation Rules

2.2.5  Shared Properties

   Matching patterns of fields (e.g. source port 80 and destination
   network 10.10.0.0/16 in the previous example) constitute shared
   properties of exported flows.  Such shared properties are equal for
   all meta-flows resulting from the corresponding aggregation rule.
   Shared properties MAY be exported as regular IPFIX fields.  However,
   in order to reduce redundancy and to make patterns distinguishable
   from other fields, they SHOULD be transmitted as fixed-value fields
   using the Data Template proposed in Section 3.3.  As a result, no
   separate transmission of the aggregation instructions needs take
   place.

2.2.6  Example Rule

   Here is an example rule with four aggregation instructions:
   1.  Aggregate
       *  discard sourceTransportPort in 80
       *  keep sourceIPv4Address
       *  mask/24 destinationIPv4Address in 10.10.0.0/16
       *  aggregate packetDeltaCount

   This rule aggregates all flows to a destination address in the subnet
   10.10.0.0/16 with source port equal to 80.  Destination addresses are

Dressler, et al.    draft-dressler-ipfix-aggregation-01.txt     [Page 8]

Internet-Draft              IPFIX Aggregation                  July 2005

   merged according to subnets in 10.10.x.0/24.  The resulting meta-flow
   records comprise the source address, the destination subnet address,
   and the packet counter.

2.3  Application Examples

2.3.1  Charging

   Monitoring and accouting for charging applications require saving
   information about each individual end system.  Further information
   about each particular flow is not required.  Therefore, aggregation
   rules are appropriate if the address of the end system is retained.

   An example ruleset might consist of the following instructions:
   1.  Aggregate
       *  keep destinationIPv4Address in 10.10.0.0/16
       *  aggregate packetDeltaCount
   2.  Aggregate
       *  keep sourceIPv4Address in 10.10.0.0/16
       *  aggregate packetDeltaCount

   Thus, aggregated meta-flows will be created for all inbound and
   outbound traffic of the network managed, separated by direction and
   host.  Protocol and port information will be discarded.

2.3.2  Intrusion Detection

   If monitoring is employed for further analysis in terms of intrusion
   detection, i.e. anomaly detection, rule-based intrusion detection,
   etc., information about used protocols at transport layer as well as
   at application layer is mostly required.  On the other hand, the
   analysis will typically work on the basis of subnetworks instead of
   single hosts because the analysis of individual flows would require
   to much processing power.  Accurate information about the traffic
   between individual end systems is only required if suspicious
   transmissions have already been detected.

   An example ruleset might consist of the following instructions:
   1.  Aggregate
       *  keep destinationIPv4Address in 10.10.0.0/16
       *  mask/24 sourceIPv4Address
       *  keep protocolIdentifier
       *  keep destinationTransportPort
       *  aggregate packetDeltaCount

   Thus, aggregated meta-flows will be created for all packets to hosts
   in a particular network.  The destination port and the protocol will
   be preserved and the source port will be discarded.

Dressler, et al.    draft-dressler-ipfix-aggregation-01.txt     [Page 9]

Internet-Draft              IPFIX Aggregation                  July 2005

3.  IPFIX Extensions

   After having a rule's field modifiers applied, all data records that
   matched a rule comprise the same fields, so for each rule exactly one
   template can be used.  In order to efficiently transmit aggregated
   flows, however, three extensions to IPFIX are required:
   o  New abstract data type "IPNetwork"
   o  New abstract data type "portRanges"
   o  New "Data Template" set

3.1  ipv4Network

   Currently, the transport of IP network information as specified by
   IPFIX is done using separate fields for the network address and the
   corresponding mask.  We propose a new abstract data type ipv4Network
   that represents the common notation of IP networks: address/mask.
   Thus, this data type is build of an unsigned32 for the IPv4 address
   and a single byte for the prefix length, whereas the encoding of the
   IPv4 address corresponds to the definition of the ipv4Address in the
   IPFIX Information Model.

   ipv4Network is required because it offers more flexibility and
   scalability if hugh numbers of IPv4 networks have to be transmitted,
   as ususally done using IPFIX Aggregation.

3.2  portRanges

   As the current draft contains no data type to transmit port lists or
   port ranges, but many applications require port-based aggregation,
   this section defines a new abstract data type for a list of port
   ranges.

   portRanges is a finite length string of unsigned32 values, each
   consisting of an unsigned16 start port followed by an unsigned16 end
   port specification.
   portRanges MAY be used as a variable-length data type as defined in
   [I-D.ietf-ipfix-protocol].
   Data types basing on portRanges MAY also be cast down to unsigned16
   to represent a single Port.  Hence, the transportSourcePort and
   transportDestinationPort data types, currently based on the
   unsigned16 abstract data type, MAY be replaced by portRanges.

Dressler, et al.    draft-dressler-ipfix-aggregation-01.txt    [Page 10]

Internet-Draft              IPFIX Aggregation                  July 2005

      +---------------+--------+---------------------------------+
      | Ports         | Length | Hexadecimal Representation      |
      +---------------+--------+---------------------------------+
      | 80            | 2      | 0050                            |
      | 1:7           | 4      | 0001 0007                       |
      | 80, 443       | 8      | 0050 0050 01BB 01BB             |
      | 1:7, 256:1024 | 8      | 0001 0007 0100 0400             |
      | 20, 80, 443   | 12     | 0014 0014 0050 0050 01BB 01BB   |
      | 1:7, 80, 443  | 12     | 0001 0007 0050 0050 01BB 01BB   |
      +---------------+--------+---------------------------------+

                       Table 3: PortRanges Examples

3.3  Data Template

   When doing rule based aggregation of flows, the resulting meta-flows
   share many field values.  In order to avoid the overhead of the
   repeated transmission of these shared properties in all meta-flow
   records resulting from a given rule, a new template type is
   introduced.  Additionally, the aggregation rule set is provided to
   the analyzer.  Using this information, the collector is able to
   quantify the received data.

   This template type, termed a "Data Template", allows the sender to
   both declare and define common properties of Data Sets based on it.
   The basic format of a Data Template Set is the same as that of a
   Template Set and - for the sake of completeness - is shown in
   Figure 3.  The format of individual template definitions, however,
   differs from that of the standard Template and is shown in Figure 4.

Dressler, et al.    draft-dressler-ipfix-aggregation-01.txt    [Page 11]

Internet-Draft              IPFIX Aggregation                  July 2005

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |          Set ID = 4           |          Length               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                                                               |
   |                        Data Template 1                        |
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                                                               |
   |                              ...                              |
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                                                               |
   |                        Data Template N                        |
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                          Padding (opt)                        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

                    Figure 3: Data Template Set Format

   The Data Template Set field definitions are as follows:
   Set ID
      Type of this template set.  A set ID value of 4 is proposed for
      the Data Template Set.

   Length
      Total length of this set in bytes.

   Padding
      Optional padding to fill the set to a word boundary.  It MUST
      consist of null-bytes and be at most seven bytes in length

Dressler, et al.    draft-dressler-ipfix-aggregation-01.txt    [Page 12]

Internet-Draft              IPFIX Aggregation                  July 2005

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |  Template ID                  |  Field Count                  |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |  Data Count                   |  Preceding Rule               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                                                               |
   |                         Field 1 Type                          |
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                                                               |
   |                              ...                              |
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                                                               |
   |                         Field N Type                          |
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                                                               |
   |                          Data 1 Type                          |
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                                                               |
   |                              ...                              |
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                                                               |
   |                          Data M Type                          |
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                                                               |
   |                          Data 1 Value                         |
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                                                               |
   |                              ...                              |
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                                                               |
   |                          Data M Value                         |
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

                      Figure 4: Data Template Format

   The Data Template field definitions are as follows:

Dressler, et al.    draft-dressler-ipfix-aggregation-01.txt    [Page 13]

Internet-Draft              IPFIX Aggregation                  July 2005

   Template ID
      See the description of a Type 3 Template Set in [I-D.ietf-ipfix-
      protocol]

   Field Count
      Number of regular Fields that will be sent in subsequent Data Sets
      using this Template.

   Data Count
      Number of fixed-value Fields that will be sent in this Template.

   Preceding Rule
      Template ID of the preceding rule that is checked before, or 0 if
      all incoming records are checked against this rule.  When
      organizing aggregation rules in a chain or tree where rules are
      sequentially checked on a first-match basis as proposed in
      Section 2.2.3, the order in which rules are checked needs to be
      made clear to the receiving collector.  Because of the one-to-one
      mapping between rules and template IDs, communicating the order of
      rules is as simple as embedding the preceding rule's Template ID
      into the Data Template(s) of the rule(s) that follow(s).  This way
      the concentrator is implicitly sent the order of the aggregation
      rules within the chain or tree.

   Field N Type
      Type identifier, Type length and an enterprise number (if needed)
      of field N. Refer to [I-D.ietf-ipfix-protocol] for more
      information on Field Types

   Data M Type
      Same as "Field N Type", but used for common properties of all Data
      Sets that use this Template.  Together with Data M Value, a
      similar encoding like TLV is achieved.

   Data M Value
      Bit representation of a common property as would be transmitted in
      a Data Set.

3.4  Example

   In this example we assume the concentrator was given the following
   aggregation rule:
   1.  Aggregate
       *  discard sourceIPv4Address in 10.0.0.0/23
       *  keep destinatonTransportPort

Dressler, et al.    draft-dressler-ipfix-aggregation-01.txt    [Page 14]

Internet-Draft              IPFIX Aggregation                  July 2005

       *  aggregate packetDeltaCount

   We further assume the concentrator receives the following flow
   records:

   +------------+------------+-------------+-------------+-------------+
   | Source IP  | Source     | Destination | Destination | Packets     |
   |            | Port       | IP          | Port        |             |
   +------------+------------+-------------+-------------+-------------+
   | 10.0.0.1   | 64235      | 10.0.0.10   | 80          | 10          |
   | 10.0.1.2   | 64236      | 10.0.0.11   | 110         | 10          |
   | 10.0.0.3   | 64237      | 10.0.0.12   | 80          | 10          |
   | 10.0.2.4   | 64238      | 10.0.0.13   | 80          | 10          |
   | 10.0.2.5   | 64239      | 10.0.0.14   | 80          | 10          |
   +------------+------------+-------------+-------------+-------------+

                          Table 4: Incoming Flows

   Based on the aggregation rule stated above the concentrator would now
   first send a Data Template Set containing the Data Template
   corresponding to the given rule:

                  +--------------+-------------------+
                  | Field        | Value             |
                  +--------------+-------------------+
                  | Template ID  | 10001             |
                  | Field Count  | 2                 |
                  | Data Count   | 2                 |
                  | Reserved     | 0                 |
                  | Field 1 Type | Destination Port  |
                  | Field 2 Type | Packets           |
                  | Data 1 Type  | Source IP Network |
                  | Data 2 Type  | Source IP Mask    |
                  | Data 1 Value | 10.10.0.0         |
                  | Data 2 Value | 23                |
                  +--------------+-------------------+

                        Table 5: Data Template used

Dressler, et al.    draft-dressler-ipfix-aggregation-01.txt    [Page 15]

Internet-Draft              IPFIX Aggregation                  July 2005

   The second Set transmitted now uses this Data Template to send all
   flows that matched the given rule.  Note that the flows' common
   property, a Source IP of 10.0.0.0/23, was already transmitted in the
   template, so besides the packet count only the flows' discriminating
   property, their destination port, is sent in the data records:

                     +------------------+---------+
                     | Destination Port | Packets |
                     +------------------+---------+
                     | 80               | 20      |
                     | 110              | 10      |
                     +------------------+---------+

                         Table 6: Aggregated Flows

4.  Security considerations

   As all methods described in this document are merely variations on
   methods already introduced in [I-D.ietf-ipfix-protocol], the same
   rules regarding exchange of flow information apply.

5.  References

5.1  Normative References

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", March 1997.

5.2  Informative References

   [I-D.ietf-ipfix-architecture]
              Sadasivan, G., Brownlee, N., Claise, B., and J. Quittek,
              "Architecture for IP Flow Information Export",
              draft-ietf-ipfix-architecture-08 (work in progress),
              March 2005.

   [I-D.ietf-ipfix-protocol]
              Claise, B., "IPFIX Protocol Specifications",
              draft-ietf-ipfix-protocol-16 (work in progress),
              June 2005.

   [I-D.ietf-ipfix-info]
              Quittek, J., Bryant, S., Claise, B., and J. Meyer,
              "Information Model for IP Flow Information Export",
              draft-ietf-ipfix-info-07 (work in progress), May 2005.

   [RFC3917]  Quittek, J., Zseby, T., Claise, B., and S. Zander,

Dressler, et al.    draft-dressler-ipfix-aggregation-01.txt    [Page 16]

Internet-Draft              IPFIX Aggregation                  July 2005

              "Requirements for IP Flow Information Export", RFC 3917,
              October 2004.

Authors' Addresses

   Falko Dressler
   University of Erlangen
   Department of Computer Science 7
   Martensstr. 3
   Erlangen  91058
   Germany

   Phone: +49 9131 85-27914
   Email: [email-masked]
   URI:   http://www7.informatik.uni-erlangen.de/

   Christoph Sommer
   University of Erlangen
   Department of Computer Science 7
   Martensstr. 3
   Erlangen  91058
   Germany

   Email: [email-masked]

   Gerhard Munz
   University of Tubingen
   Computer Networks and Internet
   Auf der Morgenstelle 10C
   Tubingen  72076
   Germany

   Phone: +49 7071 29-70534
   Email: [email-masked]
   URI:   http://net.informatik.uni-tuebingen.de/

Dressler, et al.    draft-dressler-ipfix-aggregation-01.txt    [Page 17]

Internet-Draft              IPFIX Aggregation                  July 2005

Intellectual Property Statement

   The IETF takes no position regarding the validity or scope of any
   Intellectual Property Rights or other rights that might be claimed to
   pertain to the implementation or use of the technology described in
   this document or the extent to which any license under such rights
   might or might not be available; nor does it represent that it has
   made any independent effort to identify any such rights.  Information
   on the procedures with respect to rights in RFC documents can be
   found in BCP 78 and BCP 79.

   Copies of IPR disclosures made to the IETF Secretariat and any
   assurances of licenses to be made available, or the result of an
   attempt made to obtain a general license or permission for the use of
   such proprietary rights by implementers or users of this
   specification can be obtained from the IETF on-line IPR repository at
   http://www.ietf.org/ipr.

   The IETF invites any interested party to bring to its attention any
   copyrights, patents or patent applications, or other proprietary
   rights that may cover technology that may be required to implement
   this standard.  Please address the information to the IETF at
   [email-masked].

Disclaimer of Validity

   This document and the information contained herein are provided on an
   "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS
   OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET
   ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED,
   INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE
   INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
   WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Copyright Statement

   Copyright (C) The Internet Society (2005).  This document is subject
   to the rights, licenses and restrictions contained in BCP 78, and
   except as set forth therein, the authors retain all their rights.

Acknowledgment

   Funding for the RFC Editor function is currently provided by the
   Internet Society.

Dressler, et al.    draft-dressler-ipfix-aggregation-01.txt    [Page 18]