From bclaise@cisco.com Tue Oct 2 03:55:53 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A55DC21F8A8F for ; Tue, 2 Oct 2012 03:55:53 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -8.612 X-Spam-Level: X-Spam-Status: No, score=-8.612 tagged_above=-999 required=5 tests=[AWL=1.987, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 9R5KKHHzUqAL for ; Tue, 2 Oct 2012 03:55:53 -0700 (PDT) Received: from av-tac-bru.cisco.com (weird-brew.cisco.com [144.254.15.118]) by ietfa.amsl.com (Postfix) with ESMTP id 000A221F8A8E for ; Tue, 2 Oct 2012 03:55:52 -0700 (PDT) X-TACSUNS: Virus Scanned Received: from strange-brew.cisco.com (localhost.cisco.com [127.0.0.1]) by av-tac-bru.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id q92AtmOW000758; Tue, 2 Oct 2012 12:55:48 +0200 (CEST) Received: from [10.149.12.44] (dhcp-10-149-12-44.cisco.com [10.149.12.44]) by strange-brew.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id q92AtlI2005175; Tue, 2 Oct 2012 12:55:47 +0200 (CEST) Message-ID: <506AC833.3050401@cisco.com> Date: Tue, 02 Oct 2012 12:55:47 +0200 From: Benoit Claise User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:15.0) Gecko/20120907 Thunderbird/15.0.1 MIME-Version: 1.0 To: Paul Aitken , Chris Inacio References: <50587825.9020305@cisco.com> <505B7931.5020404@plixer.com> <50602B53.1060908@cisco.com> <50604F01.8030104@cisco.com> <50605A40.90906@plixer.com> <5060D78A.5060807@cisco.com> <5060DC4D.2010301@cisco.com> <5060DDF9.8000000@cisco.com> <5060E06C.90704@cisco.com> In-Reply-To: <5060E06C.90704@cisco.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: ipfix@ietf.org Subject: Re: [IPFIX] Promotion of Enterprise-Specific IEs to IANA IEs X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Oct 2012 10:55:53 -0000 On 25/09/2012 00:36, Paul Aitken wrote: > Benoit, > >> Instead, just point the collector to the URI with all the >> information. That's way easier for everybody: the exporters and the >> collector. > > My constant point is that there's no way that network operators will > allow this. Time to get the collectors' feedback. We heard from Plixer. We need some more feedback. Chris, since you presented the idea. What are your views? Regards, Benoit. > > Network configuration is locked down, and only changeable during > pre-planned maintenance windows. At such times, only known-good and > pre-tested configurations are rolled out. > > The equivalence option can be tested in isolation in a lab, and the > configuration proven. There's no external influence upon the > configuration. > > Whereas the URI mechanism cannot be tested in isolation because it's > dependent upon the URI content, which could be modified at any time > without notice. > > Also, this introduces a new attack vector upon the collector - ie, by > feeding it malformed or incorrect data through the URI. > > So from a network operator point of view, I wouldn't buy that. > > P. > > From wwwrun@rfc-editor.org Tue Oct 2 16:29:19 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5951021F85C0; Tue, 2 Oct 2012 16:29:19 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.004 X-Spam-Level: X-Spam-Status: No, score=-102.004 tagged_above=-999 required=5 tests=[AWL=-0.004, BAYES_00=-2.599, J_CHICKENPOX_93=0.6, NO_RELAYS=-0.001, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kK4w3R6XpYmb; Tue, 2 Oct 2012 16:29:19 -0700 (PDT) Received: from rfc-editor.org (rfc-editor.org [IPv6:2001:1890:123a::1:2f]) by ietfa.amsl.com (Postfix) with ESMTP id F0B2321F85B8; Tue, 2 Oct 2012 16:29:18 -0700 (PDT) Received: by rfc-editor.org (Postfix, from userid 30) id 183F0B1E003; Tue, 2 Oct 2012 16:24:55 -0700 (PDT) To: ietf-announce@ietf.org, rfc-dist@rfc-editor.org From: rfc-editor@rfc-editor.org Message-Id: <20121002232455.183F0B1E003@rfc-editor.org> Date: Tue, 2 Oct 2012 16:24:55 -0700 (PDT) Cc: ipfix@ietf.org, rfc-editor@rfc-editor.org Subject: [IPFIX] RFC 6727 on Definitions of Managed Objects for Packet Sampling X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Oct 2012 23:29:19 -0000 A new Request for Comments is now available in online RFC libraries. RFC 6727 Title: Definitions of Managed Objects for Packet Sampling Author: T. Dietz, Ed., B. Claise, J. Quittek Status: Standards Track Stream: IETF Date: October 2012 Mailbox: dietz@neclab.eu, bclaise@cisco.com, quittek@neclab.eu Pages: 28 Characters: 55441 Updates/Obsoletes/SeeAlso: None I-D Tag: draft-ietf-ipfix-psamp-mib-06.txt URL: http://www.rfc-editor.org/rfc/rfc6727.txt This memo defines a portion of the Management Information Base (MIB) for use with network management protocols in the Internet community. In particular, it describes extensions to the IPFIX-SELECTOR-MIB module. For IP Flow Information eXport (IPFIX) implementations that use Packet Sampling (PSAMP) techniques, this memo defines the PSAMP- MIB module containing managed objects for providing information on applied packet selection functions and their parameters. [STANDARDS-TRACK] This document is a product of the IP Flow Information Export Working Group of the IETF. This is now a Proposed Standard Protocol. STANDARDS TRACK: This document specifies an Internet standards track protocol for the Internet community,and requests discussion and suggestions for improvements. Please refer to the current edition of the Internet Official Protocol Standards (STD 1) for the standardization state and status of this protocol. Distribution of this memo is unlimited. This announcement is sent to the IETF-Announce and rfc-dist lists. To subscribe or unsubscribe, see http://www.ietf.org/mailman/listinfo/ietf-announce http://mailman.rfc-editor.org/mailman/listinfo/rfc-dist For searching the RFC series, see http://www.rfc-editor.org/rfcsearch.html. For downloading RFCs, see http://www.rfc-editor.org/rfc.html. Requests for special distribution should be addressed to either the author of the RFC in question, or to rfc-editor@rfc-editor.org. Unless specifically noted otherwise on the RFC itself, all RFCs are for unlimited distribution. The RFC Editor Team Association Management Solutions, LLC From wwwrun@rfc-editor.org Tue Oct 2 16:29:34 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0FBD321F8628; Tue, 2 Oct 2012 16:29:34 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.004 X-Spam-Level: X-Spam-Status: No, score=-102.004 tagged_above=-999 required=5 tests=[AWL=-0.004, BAYES_00=-2.599, J_CHICKENPOX_93=0.6, NO_RELAYS=-0.001, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GO9ftkSUliOs; Tue, 2 Oct 2012 16:29:33 -0700 (PDT) Received: from rfc-editor.org (rfc-editor.org [IPv6:2001:1890:123a::1:2f]) by ietfa.amsl.com (Postfix) with ESMTP id 9FBD421F85B8; Tue, 2 Oct 2012 16:29:33 -0700 (PDT) Received: by rfc-editor.org (Postfix, from userid 30) id CF431B1E007; Tue, 2 Oct 2012 16:25:09 -0700 (PDT) To: ietf-announce@ietf.org, rfc-dist@rfc-editor.org From: rfc-editor@rfc-editor.org Message-Id: <20121002232509.CF431B1E007@rfc-editor.org> Date: Tue, 2 Oct 2012 16:25:09 -0700 (PDT) Cc: ipfix@ietf.org, rfc-editor@rfc-editor.org Subject: [IPFIX] RFC 6728 on Configuration Data Model for the IP Flow Information Export (IPFIX) and Packet Sampling (PSAMP) Protocols X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Oct 2012 23:29:34 -0000 A new Request for Comments is now available in online RFC libraries. RFC 6728 Title: Configuration Data Model for the IP Flow Information Export (IPFIX) and Packet Sampling (PSAMP) Protocols Author: G. Muenz, B. Claise, P. Aitken Status: Standards Track Stream: IETF Date: October 2012 Mailbox: muenz@net.in.tum.de, bclaise@cisco.com, paitken@cisco.com Pages: 129 Characters: 279937 Updates/Obsoletes/SeeAlso: None I-D Tag: draft-ietf-ipfix-configuration-model-11.txt URL: http://www.rfc-editor.org/rfc/rfc6728.txt This document specifies a data model for the IP Flow Information Export (IPFIX) and Packet Sampling (PSAMP) protocols. It is for configuring and monitoring Selection Processes, Caches, Exporting Processes, and Collecting Processes of IPFIX- and PSAMP-compliant Monitoring Devices using the Network Configuration Protocol (NETCONF). The data model is defined using UML (Unified Modeling Language) class diagrams and formally specified using YANG. The configuration data is encoded in Extensible Markup Language (XML). [STANDARDS-TRACK] This document is a product of the IP Flow Information Export Working Group of the IETF. This is now a Proposed Standard Protocol. STANDARDS TRACK: This document specifies an Internet standards track protocol for the Internet community,and requests discussion and suggestions for improvements. Please refer to the current edition of the Internet Official Protocol Standards (STD 1) for the standardization state and status of this protocol. Distribution of this memo is unlimited. This announcement is sent to the IETF-Announce and rfc-dist lists. To subscribe or unsubscribe, see http://www.ietf.org/mailman/listinfo/ietf-announce http://mailman.rfc-editor.org/mailman/listinfo/rfc-dist For searching the RFC series, see http://www.rfc-editor.org/rfcsearch.html. For downloading RFCs, see http://www.rfc-editor.org/rfc.html. Requests for special distribution should be addressed to either the author of the RFC in question, or to rfc-editor@rfc-editor.org. Unless specifically noted otherwise on the RFC itself, all RFCs are for unlimited distribution. The RFC Editor Team Association Management Solutions, LLC From internet-drafts@ietf.org Wed Oct 3 07:47:56 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C760621F86C4; Wed, 3 Oct 2012 07:47:56 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.509 X-Spam-Level: X-Spam-Status: No, score=-102.509 tagged_above=-999 required=5 tests=[AWL=0.090, BAYES_00=-2.599, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 53tQU0UGrzE3; Wed, 3 Oct 2012 07:47:56 -0700 (PDT) Received: from ietfa.amsl.com (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7041F21F8546; Wed, 3 Oct 2012 07:47:56 -0700 (PDT) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: internet-drafts@ietf.org To: i-d-announce@ietf.org X-Test-IDTracker: no X-IETF-IDTracker: 4.34 Message-ID: <20121003144756.4800.68013.idtracker@ietfa.amsl.com> Date: Wed, 03 Oct 2012 07:47:56 -0700 Cc: ipfix@ietf.org Subject: [IPFIX] I-D Action: draft-ietf-ipfix-ie-doctors-07.txt X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Oct 2012 14:47:57 -0000 A New Internet-Draft is available from the on-line Internet-Drafts director= ies. This draft is a work item of the IP Flow Information Export Working Group = of the IETF. Title : Guidelines for Authors and Reviewers of IPFIX Informatio= n Elements Author(s) : Brian Trammell Benoit Claise Filename : draft-ietf-ipfix-ie-doctors-07.txt Pages : 33 Date : 2012-10-03 Abstract: This document provides guidelines for how to write definitions of new Information Elements for the IP Flow Information Export (IPFIX) protocol. It provides instructions on using the proper conventions for Information Elements to be registered in the IANA IPFIX Information Element registry, and provides guidelines for expert reviewers to evaluate new registrations. The IETF datatracker status page for this draft is: https://datatracker.ietf.org/doc/draft-ietf-ipfix-ie-doctors There's also a htmlized version available at: http://tools.ietf.org/html/draft-ietf-ipfix-ie-doctors-07 A diff from the previous version is available at: http://www.ietf.org/rfcdiff?url2=3Ddraft-ietf-ipfix-ie-doctors-07 Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ From internet-drafts@ietf.org Wed Oct 3 07:48:04 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 29E8321F86D1; Wed, 3 Oct 2012 07:48:04 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.51 X-Spam-Level: X-Spam-Status: No, score=-102.51 tagged_above=-999 required=5 tests=[AWL=0.089, BAYES_00=-2.599, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id AsP53HUFpXWL; Wed, 3 Oct 2012 07:48:03 -0700 (PDT) Received: from ietfa.amsl.com (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7A21221F86C9; Wed, 3 Oct 2012 07:48:03 -0700 (PDT) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: internet-drafts@ietf.org To: i-d-announce@ietf.org X-Test-IDTracker: no X-IETF-IDTracker: 4.34 Message-ID: <20121003144803.16661.89676.idtracker@ietfa.amsl.com> Date: Wed, 03 Oct 2012 07:48:03 -0700 Cc: ipfix@ietf.org Subject: [IPFIX] I-D Action: draft-ietf-ipfix-information-model-rfc5102bis-06.txt X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Oct 2012 14:48:04 -0000 A New Internet-Draft is available from the on-line Internet-Drafts director= ies. This draft is a work item of the IP Flow Information Export Working Group = of the IETF. Title : Information Model for IP Flow Information eXport (IPFIX) Author(s) : Benoit Claise Brian Trammell Filename : draft-ietf-ipfix-information-model-rfc5102bis-06.txt Pages : 28 Date : 2012-10-03 Abstract: This document provides an overview of the information model for the IP Flow Information eXport (IPFIX) protocol, as defined in the IANA IPFIX Information Element Registry. It is used by the IPFIX Protocol for encoding measured traffic information and information related to the traffic Observation Point, the traffic Metering Process, and the Exporting Process. Although developed for the IPFIX Protocol, the model is defined in an open way that easily allows using it in other protocols, interfaces, and applications. This document obsoletes RFC 5102. The IETF datatracker status page for this draft is: https://datatracker.ietf.org/doc/draft-ietf-ipfix-information-model-rfc5102= bis There's also a htmlized version available at: http://tools.ietf.org/html/draft-ietf-ipfix-information-model-rfc5102bis-06 A diff from the previous version is available at: http://www.ietf.org/rfcdiff?url2=3Ddraft-ietf-ipfix-information-model-rfc51= 02bis-06 Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ From n.brownlee@auckland.ac.nz Wed Oct 3 15:44:55 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8177521F851C for ; Wed, 3 Oct 2012 15:44:54 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.532 X-Spam-Level: X-Spam-Status: No, score=-106.532 tagged_above=-999 required=5 tests=[AWL=0.067, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id oWXeeQ+hz3bw for ; Wed, 3 Oct 2012 15:44:53 -0700 (PDT) Received: from mx2.auckland.ac.nz (mx2.auckland.ac.nz [130.216.12.44]) by ietfa.amsl.com (Postfix) with ESMTP id B53B521F851E for ; Wed, 3 Oct 2012 15:44:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=auckland.ac.nz; i=n.brownlee@auckland.ac.nz; q=dns/txt; s=uoa; t=1349304293; x=1380840293; h=message-id:date:from:mime-version:to:subject: content-transfer-encoding; bh=aZMxo6mwF4RXBuKp/ZKyAMQVAU27cRrFgMPN+kuNoKA=; b=Kuui68g8aqFlmtrdEThpqWseunVC7txNn9KBmfl+na+8YzlQIRlnzNKk onqoH39zuqeEodgHH/iq1q5p4CtRVo91+AuSY+obgW3Y2lC4bHOiyqzNF Bocw7BMmuUHWbp+tC3PO0t7VlVT5JIGQqxSuJ+26QisrP851WKwu8oHUC o=; X-IronPort-AV: E=Sophos;i="4.80,530,1344168000"; d="scan'208";a="148845510" X-Ironport-HAT: UNIVERSITY - $RELAY-THROTTLE X-Ironport-Source: 130.216.38.131 - Outgoing - Outgoing-SSL Received: from nevil-laptop1.sfac.auckland.ac.nz (HELO [130.216.38.131]) ([130.216.38.131]) by mx2-int.auckland.ac.nz with ESMTP; 04 Oct 2012 11:44:52 +1300 Message-ID: <506CBFE3.10607@auckland.ac.nz> Date: Thu, 04 Oct 2012 11:44:51 +1300 From: Nevil Brownlee User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:15.0) Gecko/20120907 Thunderbird/15.0.1 MIME-Version: 1.0 To: IPFIX Working Group Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: [IPFIX] WG Last Call for draft-ietf-ipfix-information-model-rfc5102bis-05.txt X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Oct 2012 22:44:55 -0000 Hi all: The WG Last Call for this I-D starts now, and will run until Monday, 22 October. Do please read the draft, and post your comments to the list. It's important that we can show it has been well-considered/reviewed within the WG, so short comments like "yes, this does clearly describe the IPFIX Information Model as we want it to be" are important and useful! Cheers, Nevil -- --------------------------------------------------------------------- Nevil Brownlee Computer Science Department | ITS Phone: +64 9 373 7599 x88941 The University of Auckland FAX: +64 9 373 7453 Private Bag 92019, Auckland 1142, New Zealand From bclaise@cisco.com Thu Oct 4 07:45:40 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 937CB21F85AA for ; Thu, 4 Oct 2012 07:45:40 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -9.675 X-Spam-Level: X-Spam-Status: No, score=-9.675 tagged_above=-999 required=5 tests=[AWL=2.923, BAYES_00=-2.599, GB_I_LETTER=-2, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id OX25YRzjlYeO for ; Thu, 4 Oct 2012 07:45:39 -0700 (PDT) Received: from av-tac-bru.cisco.com (weird-brew.cisco.com [144.254.15.118]) by ietfa.amsl.com (Postfix) with ESMTP id 8080921F8587 for ; Thu, 4 Oct 2012 07:45:38 -0700 (PDT) X-TACSUNS: Virus Scanned Received: from strange-brew.cisco.com (localhost.cisco.com [127.0.0.1]) by av-tac-bru.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id q94EjZE7020820; Thu, 4 Oct 2012 16:45:35 +0200 (CEST) Received: from [10.60.67.86] (ams-bclaise-8915.cisco.com [10.60.67.86]) by strange-brew.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id q94EjTfA024455; Thu, 4 Oct 2012 16:45:31 +0200 (CEST) Message-ID: <506DA106.5060705@cisco.com> Date: Thu, 04 Oct 2012 16:45:26 +0200 From: Benoit Claise User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:15.0) Gecko/20120907 Thunderbird/15.0.1 MIME-Version: 1.0 To: "ipfix@ietf.org" , draft-ietf-ipfix-flow-selection-tech@tools.ietf.org References: <4FC74398.50805@cisco.com> <4FC89B99.40107@cisco.com> In-Reply-To: <4FC89B99.40107@cisco.com> Content-Type: multipart/alternative; boundary="------------050808070406050201030604" Cc: ipfix-chairs@tools.ietf.org Subject: Re: [IPFIX] New AD review of draft-ietf-ipfix-flow-selection-tech-10.txt X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Oct 2012 14:45:40 -0000 This is a multi-part message in MIME format. --------------050808070406050201030604 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Dear authors, The draft improved quite dramatically. Thanks for that. See in line for some more comments. I removed all unnecessary text. > Dear authors, > > I'm performing the (new) AD review of > draft-ietf-ipfix-flow-selection-tech-10.txt > Lucky you, an extra pair of eyes specifically looking at your draft > > If some points have been discussed already on the mailing list, let me > know. I have to admit that I have not been following the latest > iterations of this draft. > > IMHO, this document needs some more work... > I don't think that this document is really in line with the other > Intermediate Processes documents: > http://tools.ietf.org/html/rfc6235 > http://tools.ietf.org/html/draft-ietf-ipfix-a9n-03 > Note that I might have some more comments once all the points in this > email are addressed, as there are many ;-) > However, I'm available for a conf. call to clarify my points if you > want to > > See in-line. ... >> 8.2. Registration of Object Identifier . . . . . . . . . . . . 32 >> 9. Security Considerations . . . . . . . . . . . . . . . . . . . 32 >> 10. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 34 >> 11. References . . . . . . . . . . . . . . . . . . . . . . . . . . 34 >> 11.1. Normative References . . . . . . . . . . . . . . . . . . . 34 >> 11.2. Informative References . . . . . . . . . . . . . . . . . . 34 >> Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 35 > Don't you have to include the non-normative XML in the appendix, as it > was done for RFC5102, RFC5103? >> > >> 2. Terminology >> >> This document is consistent with the terminology introduced in >> [RFC5101], [RFC5470], [RFC5475] and [RFC3917]. As in [RFC5101] and >> [RFC5476], the first letter of each IPFIX-specific and PSAMP-specific >> term is capitalized along with the flow selection specific terms >> defined here. >> >> * Packet Classification >> >> Packet Classification is a process by which packets are mapped to >> specific Flow Records based on packet properties or external >> properties (e.g. interface). The properties (e.g. header >> information, packet content, AS number) make up the Flow Key. In >> case a Flow Record for a specific Flow Key already exists the Flow >> Record is updated, otherwise a new Flow Record is created. > > How is this different that the Metering Process (RFC5101)? > Metering Process > > The Metering Process generates Flow Records. Inputs to the > process are packet headers and characteristics observed at an > Observation Point, and packet treatment at the Observation Point > (for example, the selected output interface). > > The Metering Process consists of a set of functions that includes > packet header capturing, timestamping, sampling, classifying, and > maintaining Flow Records. > > The maintenance of Flow Records may include creating new records, > updating existing ones, computing Flow statistics, deriving > further Flow properties, detecting Flow expiration, passing Flow > Records to the Exporting Process, and deleting Flow Records. > What is the connection with the Metering Process? > Figure 1 seems to suggest that Packet Classification is a subset of > the Metering Process... not sure that one was answered. > > >> >> * Packet Aggregation Process >> >> In the IPFIX Metering Process the Packet Aggregation Process >> aggregates packet data into flow data and forms the Flow Records. > How is this different from the Metering Process? the "Packet Aggregation Process" is not used in the document. Why do we need it? >> After the aggregation step only the aggregated flow information is >> available. Information about individual packets is lost. >> >> >> > Intermediate Flow Selection Process: an Intermediate Process as in > [RFC6183 ] that ... > > The new definition improved a lot: * Intermediate Flow Selection Process An Intermediate Flow Selection Process takes Flow Records as its input and selects a subset of this set as its output. Intermediate Flow Selection Process is a more general concept than Intermediate Selection Process as defined in [RFC6183 ]. While an Intermediate Selection Process selects Flow Records from a sequence based upon criteria-evaluated Flow record values and passes only those Flow Records that match the criteria, an Intermediate Flow Selection Process selects Flow Records using selection criteria applicable to a larger set of Flow characteristics and information. But is there a reason why this definition can't be based on "intermediate Process" from RFC 6183: Intermediate Process An Intermediate Process takes a record stream as its input from Collecting Processes, Metering Processes, IPFIX File Readers, other Intermediate Processes, or other record sources; performs some transformations on this stream based upon the content of each record, states maintained across multiple records, or other data sources; and passes the transformed record stream as its output to Exporting Processes, IPFIX File Writers, or other Intermediate Processes in order to perform IPFIX Mediation. Typically, an Intermediate Process is hosted by an IPFIX Mediator. Alternatively, an Intermediate Process may be hosted by an Original Exporter. So * Intermediate Flow Selection Process _ An Intermediate Flow Selection Process is__an Intermediate Process as in [_RFC6183 _] that_ takes Flow Records as its input and selects a subset of this set as its output. Intermediate Flow Selection Process is a more general concept than Intermediate Selection Process as defined in [RFC6183 ]. While an Intermediate Selection Process selects Flow Records from a sequence based upon criteria-evaluated Flow record values and passes only those Flow Records that match the criteria, an Intermediate Flow Selection Process selects Flow Records using selection criteria applicable to a larger set of Flow characteristics and information. >> Regarding terminology, I still some instances of "observation point". Should be "Observation Point" ... >> >> 4. Flow selection as a Function in the IPFIX Architecture >> Thanks for your new figure 1. One editorial change: change the + in the left vertical line. +======|========================+ | | | Mediator | | + +-V-------------------+ | | | | Collecting Process | | | + +---------------------+ | | | | Intermediate Flow | | | | | Selection Process | | | + +---------------------+ | | | | Exporting Process | | | + +-|-------------------+ | | +======|========================+ | >> >> 5.1. Flow Filtering >> >> Flow Filtering is a deterministic function on the IPFIX Flow Record >> content. If the relevant flow characteristics are already observable >> at packet level (e.g. Flow Keys), Flow Filtering can be applied >> before aggregation at packet level. In order to be compliant with >> this document, at least the Property Match Filtering MUST be >> implemented. > This contradicts. > In order to be compliant with this document, at > least one of the flow selection schemes MUST be implemented. Actually, wrong cut/paste. This contradicts, in section 1: In order to be compliant with this document, at least the Property Match Filtering MUST be implemented. >> >> 8. IANA Considerations >> >> 8.1. Registration of Information Elements Table 3: Information Elements to be registered, you can't put the value 1, 2, 3, You need TBD1, TBD2, etc... And you must add "IANA Note: please replace TBD1, TBD2, ... with the assigned values, throughout the document." >> >> >> 8.2. Registration of Object Identifier >> RFC 5815 is obsoleted by RFC 6615 What you want is an extra in http://www.iana.org/assignments/smi-numbers, pointing to this RFC: Sub-registry Name: IPFIX-SELECTOR-MIB Functions Reference: [RFC6615] Registration Procedures: Expert Review Prefix: iso.org.dod.internet.mgmt.mib-2.ipfixSelectorMIB.ipfixSelectorObjects.ipfixSelectorFunctions (1.3.6.1.2.1.194.1.1) Decimal Name Description Reference ------- --------------------- --------------------------------- --------- 1 ipfixFuncSelectAll Select everything [RFC6615] 2 psampSampCountBased Systematic Count-based Sampling [RFC6727] 3 psampSampTimeBased Systematic Time-based Sampling [RFC6727] 4 psampSampRandOutOfN Random n-out-of-N Sampling [RFC6727] 5 psampSampUniProb Universal Probabilistic Sampling [RFC6727] 6 psampFiltPropMatch Property Match Filtering [RFC6727] 7 psampFiltHash Hash-based Filtering [RFC6727] So you need TBDx +---------+-----------------------+---------------------+-----------+ | Decimal | Name | Description | Reference | +---------+-----------------------+---------------------+-----------+ | TBDx | flowSelectorAlgorithm | This Object | [RFCyyyy] | | | | Identifier | | | | | identifies the Flow | | | | | selection technique | | | | | (e.g., Filtering, | | | | | Sampling) that is | | | | | applied by the Flow | | | | | Selection Process | | +---------+-----------------------+---------------------+-----------+ Table 4: Object Identifiers to be registered "IANA Note: please replace TBDx with the assigned value, throughout the document." Btw, there is a mismatch between the IANA registry and the table in section 7.1: +----+------------------------+--------------------------+ | ID | Technique | Parameters | +----+------------------------+--------------------------+ | 1 | Systematic count-based | flowSamplingInterval | | | Sampling | flowSamplingSpacing | +----+------------------------+--------------------------+ | 2 | Systematic time-based | flowSamplingTimeInterval | | | Sampling | flowSamplingTimeSpacing | +----+------------------------+--------------------------+ | 3 | Random n-out-of-N | samplingSize | | | Sampling | samplingPopulation | +----+------------------------+--------------------------+ | 4 | Uniform probabilistic | samplingProbability | | | Sampling | | +----+------------------------+--------------------------+ | 5 | Property Match | Information Element | | | Filtering | Value Range | +----+------------------------+--------------------------+ | Hash-based Filtering | hashInitialiserValue | +----+------------------------+ hashFlowDomain | | 6 | using BOB | hashSelectedRangeMin | +----+------------------------+ hashSelectedRangeMax | | 7 | using IPSX | hashOutputRangeMin | +----+------------------------+ hashOutputRangeMax | | 8 | using CRC | | +----+------------------------+--------------------------+ | 9 | Flow-state Dependent | No agreed Parameters | | | Flow Selection | | +----+------------------------+--------------------------+ Also, in this table above, you need "TBDx" instead of 9 - I see "Flow Selection", but this term is not defined. Thanks. Regards, Benoit. --------------050808070406050201030604 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit
Dear authors,

The draft improved quite dramatically.
Thanks for that.
See in line for some more comments. I removed all unnecessary text.

Dear authors,

I'm performing the (new) AD review of  draft-ietf-ipfix-flow-selection-tech-10.txt
Lucky you, an extra pair of eyes specifically looking at your draft

If some points have been discussed already on the mailing list, let me know. I have to admit that I have not been following the latest iterations of this draft.

IMHO, this document needs some more work...
I don't think that this document is really in line with the other Intermediate Processes documents:
    http://tools.ietf.org/html/rfc6235
    http://tools.ietf.org/html/draft-ietf-ipfix-a9n-03
Note that I might have some more comments once all the points in this email are addressed, as there are many ;-)
However, I'm available for a conf. call to clarify my points if you want to

See in-line.
...
     8.2.  Registration of Object Identifier  . . . . . . . . . . . . 32
   9.  Security Considerations  . . . . . . . . . . . . . . . . . . . 32
   10. Acknowledgments  . . . . . . . . . . . . . . . . . . . . . . . 34
   11. References . . . . . . . . . . . . . . . . . . . . . . . . . . 34
     11.1. Normative References . . . . . . . . . . . . . . . . . . . 34
     11.2. Informative References . . . . . . . . . . . . . . . . . . 34
   Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 35
Don't you have to include the non-normative XML in the appendix, as it was done for RFC5102, RFC5103?


2.  Terminology

   This document is consistent with the terminology introduced in
   [RFC5101], [RFC5470], [RFC5475] and [RFC3917].  As in [RFC5101] and
   [RFC5476], the first letter of each IPFIX-specific and PSAMP-specific
   term is capitalized along with the flow selection specific terms
   defined here.

   * Packet Classification

      Packet Classification is a process by which packets are mapped to
      specific Flow Records based on packet properties or external
      properties (e.g. interface).  The properties (e.g. header
      information, packet content, AS number) make up the Flow Key. In
      case a Flow Record for a specific Flow Key already exists the Flow
      Record is updated, otherwise a new Flow Record is created.

How is this different that the Metering Process (RFC5101)?
   Metering Process

      The Metering Process generates Flow Records.  Inputs to the
      process are packet headers and characteristics observed at an
      Observation Point, and packet treatment at the Observation Point
      (for example, the selected output interface).

      The Metering Process consists of a set of functions that includes
      packet header capturing, timestamping, sampling, classifying, and
      maintaining Flow Records.

      The maintenance of Flow Records may include creating new records,
      updating existing ones, computing Flow statistics, deriving
      further Flow properties, detecting Flow expiration, passing Flow
      Records to the Exporting Process, and deleting Flow Records.
What is the connection with the Metering Process?
Figure 1 seems to suggest that Packet Classification is a subset of the Metering Process...
not sure that one was answered.




   * Packet Aggregation Process

      In the IPFIX Metering Process the Packet Aggregation Process
      aggregates packet data into flow data and forms the Flow Records.
How is this different from the Metering Process?
the "Packet Aggregation Process" is not used in the document. Why do we need it?

      After the aggregation step only the aggregated flow information is
      available.  Information about individual packets is lost. 



Intermediate Flow Selection Process: an Intermediate Process as in
      [RFC6183] that ...

The new definition improved a lot:
 * Intermediate Flow Selection Process

      An Intermediate Flow Selection Process takes Flow Records as its
      input and selects a subset of this set as its output.
      Intermediate Flow Selection Process is a more general concept than
      Intermediate Selection Process as defined in [RFC6183].  While an
      Intermediate Selection Process selects Flow Records from a
      sequence based upon criteria-evaluated Flow record values and
      passes only those Flow Records that match the criteria, an
      Intermediate Flow Selection Process selects Flow Records using
      selection criteria applicable to a larger set of Flow
      characteristics and information.
But is there a reason why this definition can't be based on "intermediate Process" from RFC 6183:
Intermediate Process

      An Intermediate Process takes a record stream as its input from
      Collecting Processes, Metering Processes, IPFIX File Readers,
      other Intermediate Processes, or other record sources; performs
      some transformations on this stream based upon the content of each
      record, states maintained across multiple records, or other data
      sources; and passes the transformed record stream as its output to
      Exporting Processes, IPFIX File Writers, or other Intermediate
      Processes in order to perform IPFIX Mediation.  Typically, an
      Intermediate Process is hosted by an IPFIX Mediator.
      Alternatively, an Intermediate Process may be hosted by an
      Original Exporter.
So 

 * Intermediate Flow Selection Process

      An Intermediate Flow Selection Process is an Intermediate Process as in
      [RFC6183] that takes Flow Records as its
      input and selects a subset of this set as its output.
      Intermediate Flow Selection Process is a more general concept than
      Intermediate Selection Process as defined in [RFC6183].  While an
      Intermediate Selection Process selects Flow Records from a
      sequence based upon criteria-evaluated Flow record values and
      passes only those Flow Records that match the criteria, an
      Intermediate Flow Selection Process selects Flow Records using
      selection criteria applicable to a larger set of Flow
      characteristics and information.

 

Regarding terminology, I still some instances of "observation point". Should be "Observation Point"

...


4.  Flow selection as a Function in the IPFIX Architecture
  
Thanks for your new figure 1.
One editorial change: change the + in the left vertical line.

      +======|========================+      |
      |      |  Mediator              |      |
      +    +-V-------------------+    |      |
      |    | Collecting Process  |    |      |
      +    +---------------------+    |      |
      |    | Intermediate Flow   |    |      |
      |    | Selection Process   |    |      |
      +    +---------------------+    |      |
      |    |  Exporting Process  |    |      |
      +    +-|-------------------+    |      |
      +======|========================+      |

5.1.  Flow Filtering

   Flow Filtering is a deterministic function on the IPFIX Flow Record
   content.  If the relevant flow characteristics are already observable
   at packet level (e.g.  Flow Keys), Flow Filtering can be applied
   before aggregation at packet level.  In order to be compliant with
   this document, at least the Property Match Filtering MUST be
   implemented.
This contradicts.
   In order to be compliant with this document, at
   least one of the flow selection schemes MUST be implemented.
Actually, wrong cut/paste.
This contradicts, in section 1:
   In order to be compliant with this document, at
   least the Property Match Filtering MUST be implemented.



8.  IANA Considerations

8.1.  Registration of Information Elements

Table 3: Information Elements to be registered, you can't put the value 1, 2, 3,
You need TBD1, TBD2, etc...
And you must add
"IANA Note: please replace TBD1, TBD2, ... with the assigned values, throughout the document."



8.2.  Registration of Object Identifier


RFC 5815 is obsoleted by RFC 6615

What you want is an extra in http://www.iana.org/assignments/smi-numbers, pointing to this RFC:
 
Sub-registry Name: IPFIX-SELECTOR-MIB Functions
Reference: [RFC6615]
Registration Procedures: Expert Review 
Prefix: iso.org.dod.internet.mgmt.mib-2.ipfixSelectorMIB.ipfixSelectorObjects.ipfixSelectorFunctions 
(1.3.6.1.2.1.194.1.1)

Decimal Name                  Description                       Reference
------- --------------------- --------------------------------- ---------
1       ipfixFuncSelectAll    Select everything                 [RFC6615]
2       psampSampCountBased   Systematic Count-based Sampling   [RFC6727]
3       psampSampTimeBased    Systematic Time-based Sampling    [RFC6727]
4       psampSampRandOutOfN   Random n-out-of-N Sampling        [RFC6727]
5       psampSampUniProb      Universal Probabilistic Sampling  [RFC6727]
6       psampFiltPropMatch    Property Match Filtering          [RFC6727]
7       psampFiltHash         Hash-based Filtering              [RFC6727]
So you need TBDx

   +---------+-----------------------+---------------------+-----------+
   | Decimal | Name                  | Description         | Reference |
   +---------+-----------------------+---------------------+-----------+
   |  TBDx   | flowSelectorAlgorithm | This Object         | [RFCyyyy] |
   |         |                       | Identifier          |           |
   |         |                       | identifies the Flow |           |
   |         |                       | selection technique |           |
   |         |                       | (e.g., Filtering,   |           |
   |         |                       | Sampling) that is   |           |
   |         |                       | applied by the Flow |           |
   |         |                       | Selection Process   |           |
   +---------+-----------------------+---------------------+-----------+

               Table 4: Object Identifiers to be registered

"IANA Note: please replace TBDx with the assigned value, throughout the document."

Btw, there is a mismatch between the IANA registry and the table in section 7.1:
   +----+------------------------+--------------------------+
   | ID |        Technique         |      Parameters          |
   +----+------------------------+--------------------------+
   | 1  | Systematic count-based | flowSamplingInterval     |
   |    | Sampling               | flowSamplingSpacing      |
   +----+------------------------+--------------------------+
   | 2  | Systematic time-based  | flowSamplingTimeInterval |
   |    | Sampling               | flowSamplingTimeSpacing  |
   +----+------------------------+--------------------------+
   | 3  | Random n-out-of-N      | samplingSize             |
   |    | Sampling               | samplingPopulation       |
   +----+------------------------+--------------------------+
   | 4  | Uniform probabilistic  | samplingProbability      |
   |    | Sampling               |                          |
   +----+------------------------+--------------------------+
   | 5  | Property Match         | Information Element      |
   |    | Filtering              | Value Range              |
   +----+------------------------+--------------------------+
   |   Hash-based Filtering      | hashInitialiserValue     |
   +----+------------------------+ hashFlowDomain           |
   | 6  | using BOB              | hashSelectedRangeMin     |
   +----+------------------------+ hashSelectedRangeMax     |
   | 7  | using IPSX             | hashOutputRangeMin       |
   +----+------------------------+ hashOutputRangeMax       |
   | 8  | using CRC              |                          |
   +----+------------------------+--------------------------+
   | 9  | Flow-state Dependent   | No agreed Parameters     |
   |    | Flow Selection         |                          |
   +----+------------------------+--------------------------+

Also, in this table above, you need "TBDx" instead of 9
- I see "Flow Selection", but this term is not defined.

Thanks.


Regards, Benoit.
--------------050808070406050201030604-- From bclaise@cisco.com Fri Oct 5 07:24:34 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id F2C8621F869E for ; Fri, 5 Oct 2012 07:24:33 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -8.711 X-Spam-Level: X-Spam-Status: No, score=-8.711 tagged_above=-999 required=5 tests=[AWL=1.887, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5pRZ8Cb-G0MA for ; Fri, 5 Oct 2012 07:24:32 -0700 (PDT) Received: from av-tac-bru.cisco.com (weird-brew.cisco.com [144.254.15.118]) by ietfa.amsl.com (Postfix) with ESMTP id BD8CE21F866C for ; Fri, 5 Oct 2012 07:24:31 -0700 (PDT) X-TACSUNS: Virus Scanned Received: from strange-brew.cisco.com (localhost.cisco.com [127.0.0.1]) by av-tac-bru.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id q95EOUF7013960 for ; Fri, 5 Oct 2012 16:24:30 +0200 (CEST) Received: from [10.60.67.86] (ams-bclaise-8915.cisco.com [10.60.67.86]) by strange-brew.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id q95EOUHO004277 for ; Fri, 5 Oct 2012 16:24:30 +0200 (CEST) Message-ID: <506EED9E.4030104@cisco.com> Date: Fri, 05 Oct 2012 16:24:30 +0200 From: Benoit Claise User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:15.0) Gecko/20120907 Thunderbird/15.0.1 MIME-Version: 1.0 To: "ipfix@ietf.org" References: <501AFA50.10508@cisco.com> <501FDE8E.9030001@cisco.com> In-Reply-To: <501FDE8E.9030001@cisco.com> Content-Type: multipart/alternative; boundary="------------000804080606090808010304" Subject: [IPFIX] IPFIX interop? X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Oct 2012 14:24:34 -0000 This is a multi-part message in MIME format. --------------000804080606090808010304 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Dear all, At the last IETF Meeting, we discussed the possibility of an IPFIX interop. To give some background information, the goal was mainly to progress RFC5101 to RFC5101bis. However, based on the justification below (My email whose subject was "Re: [IPFIX] RFC 2026 versus RFC6410: progressing a document"), it's no required to test every single features to progress the standards track. So we could progress RFC5101bis. So I guess that the IPFIX interop will not take place. Correct? Regards, Benoit. > Dear all, > > And I'm trying to compare the conditions to progress a document in RFC > 2026 and 6410 > > RFC 2026: > The requirement for at least two independent and interoperable > implementations applies to all of the options and features of the > specification.In cases in which one or more options or features > have not been demonstrated in at least two interoperable > implementations, the specification may advance to the Draft Standard > level only if those options or features are removed. > > RFC 6410 > The IESG, in an IETF-wide Last Call of at least four weeks, confirms > that a document advances from Proposed Standard to Internet Standard. > The request for reclassification is sent to the IESG along with an > explanation of how the criteria have been met. The criteria are: > > (1) There are at least two independent interoperating implementations > with widespread deployment and successful operational experience. > > (2) There are no errata against the specification that would cause a > new implementation to fail to interoperate with deployed ones. > > (3) There are no unused features in the specification that greatly > increase implementation complexity. > > (4) If the technology required to implement the specification > requires patented or otherwise controlled technology, then the > set of implementations must demonstrate at least two independent, > separate and successful uses of the licensing process. > > After confirmation from the IESG, we don't need to test _every single > _feature from the specifications to progress the draft. > However, the 4 points above must be respected. > > Regards, Benoit > > > > > > > _______________________________________________ > IPFIX mailing list > IPFIX@ietf.org > https://www.ietf.org/mailman/listinfo/ipfix --------------000804080606090808010304 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit
Dear all,

At the last IETF Meeting, we discussed the possibility of an IPFIX interop.
To give some background information, the goal was mainly to progress RFC5101 to RFC5101bis.
However, based on the justification below (My email whose subject was "Re: [IPFIX] RFC 2026 versus RFC6410: progressing a document"), it's no required to test every single features to progress the standards track. So we could progress RFC5101bis.
So I guess that the IPFIX interop will not take place. Correct?

Regards, Benoit.

Dear all,

And I'm trying to compare the conditions to progress a document in RFC 2026 and 6410

RFC 2026:
   The requirement for at least two independent and interoperable
   implementations applies to all of the options and features of the
   specification.  In cases in which one or more options or features
   have not been demonstrated in at least two interoperable
   implementations, the specification may advance to the Draft Standard
   level only if those options or features are removed.

RFC 6410
   The IESG, in an IETF-wide Last Call of at least four weeks, confirms
   that a document advances from Proposed Standard to Internet Standard.
   The request for reclassification is sent to the IESG along with an
   explanation of how the criteria have been met.  The criteria are:

   (1) There are at least two independent interoperating implementations
       with widespread deployment and successful operational experience.

   (2) There are no errata against the specification that would cause a
       new implementation to fail to interoperate with deployed ones.

   (3) There are no unused features in the specification that greatly
       increase implementation complexity.

   (4) If the technology required to implement the specification
       requires patented or otherwise controlled technology, then the
       set of implementations must demonstrate at least two independent,
       separate and successful uses of the licensing process.

After confirmation from the IESG, we don't need to test every single feature from the specifications to progress the draft.
However, the 4 points above must be respected.

Regards, Benoit






_______________________________________________
IPFIX mailing list
IPFIX@ietf.org
https://www.ietf.org/mailman/listinfo/ipfix

--------------000804080606090808010304-- From trammell@tik.ee.ethz.ch Fri Oct 5 07:28:34 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 83F7521F8445 for ; Fri, 5 Oct 2012 07:28:34 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.849 X-Spam-Level: X-Spam-Status: No, score=-6.849 tagged_above=-999 required=5 tests=[AWL=-0.250, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id T9hmZRHB1t2g for ; Fri, 5 Oct 2012 07:28:33 -0700 (PDT) Received: from smtp.ee.ethz.ch (smtp.ee.ethz.ch [129.132.2.219]) by ietfa.amsl.com (Postfix) with ESMTP id 7F69521F8444 for ; Fri, 5 Oct 2012 07:28:33 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by smtp.ee.ethz.ch (Postfix) with ESMTP id D46F6D930C; Fri, 5 Oct 2012 16:28:32 +0200 (MEST) X-Virus-Scanned: by amavisd-new on smtp.ee.ethz.ch Received: from smtp.ee.ethz.ch ([127.0.0.1]) by localhost (.ee.ethz.ch [127.0.0.1]) (amavisd-new, port 10024) with LMTP id eWw7fc-EyLnE; Fri, 5 Oct 2012 16:28:32 +0200 (MEST) Received: from [10.0.27.100] (cust-integra-121-161.antanet.ch [80.75.121.161]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: briant) by smtp.ee.ethz.ch (Postfix) with ESMTPSA id 90FF3D9305; Fri, 5 Oct 2012 16:28:32 +0200 (MEST) Mime-Version: 1.0 (Apple Message framework v1283) Content-Type: text/plain; charset=iso-8859-1 From: Brian Trammell In-Reply-To: <506EED9E.4030104@cisco.com> Date: Fri, 5 Oct 2012 16:28:31 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <501AFA50.10508@cisco.com> <501FDE8E.9030001@cisco.com> <506EED9E.4030104@cisco.com> To: Benoit Claise X-Mailer: Apple Mail (2.1283) Cc: "ipfix@ietf.org" Subject: Re: [IPFIX] IPFIX interop? X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Oct 2012 14:28:34 -0000 Hi, Benoit, all, I think we should definitely interop those features we haven't yet, and = have as an action item for myself making sure that all of those features = are supported in my current implementation, but I believe at this point = in time it probably makes sense to try to organize pairwise testing as = opposed to putting together another large everyone-in-a-room event, = especially as it seems as there's not much energy among implementors for = doing one of those again (the last one was in March 2011). If 5101bis' progress doesn't gate on an interop, though, we can do one = at our leisure, no? Best regards, Brian On Oct 5, 2012, at 4:24 PM, Benoit Claise wrote: > Dear all, >=20 > At the last IETF Meeting, we discussed the possibility of an IPFIX = interop. > To give some background information, the goal was mainly to progress = RFC5101 to RFC5101bis. > However, based on the justification below (My email whose subject was = "Re: [IPFIX] RFC 2026 versus RFC6410: progressing a document"), it's no = required to test every single features to progress the standards track. = So we could progress RFC5101bis. > So I guess that the IPFIX interop will not take place. Correct? >=20 > Regards, Benoit. >=20 >> Dear all, >>=20 >> And I'm trying to compare the conditions to progress a document in = RFC 2026 and 6410 >>=20 >> RFC 2026: >> The requirement for at least two independent and interoperable >> implementations applies to all of the options and features of the >> specification. =20 >> In cases in which one or more options or features >> have not been demonstrated in at least two interoperable >> implementations, the specification may advance to the Draft = Standard >> level only if those options or features are removed. >>=20 >>=20 >> RFC 6410 >> The IESG, in an IETF-wide Last Call of at least four weeks, = confirms >> that a document advances from Proposed Standard to Internet = Standard. >> The request for reclassification is sent to the IESG along with an >> explanation of how the criteria have been met. The criteria are: >>=20 >> (1) There are at least two independent interoperating = implementations >> with widespread deployment and successful operational = experience. >>=20 >> =20 >> (2) There are no errata against the specification that would cause a >> new implementation to fail to interoperate with deployed ones. >>=20 >>=20 >> (3) There are no unused features in the specification that greatly >> increase implementation complexity. >>=20 >>=20 >> (4) If the technology required to implement the specification >> requires patented or otherwise controlled technology, then the >> set of implementations must demonstrate at least two = independent, >> separate and successful uses of the licensing process. >>=20 >>=20 >> After confirmation from the IESG, we don't need to test every single = feature from the specifications to progress the draft. >> However, the 4 points above must be respected. >>=20 >> Regards, Benoit >>=20 >>=20 >>=20 >>=20 >>=20 >>=20 >> _______________________________________________ >> IPFIX mailing list >>=20 >> IPFIX@ietf.org >> https://www.ietf.org/mailman/listinfo/ipfix >=20 > _______________________________________________ > IPFIX mailing list > IPFIX@ietf.org > https://www.ietf.org/mailman/listinfo/ipfix From bclaise@cisco.com Fri Oct 5 07:56:56 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4366921F8592 for ; Fri, 5 Oct 2012 07:56:56 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -8.723 X-Spam-Level: X-Spam-Status: No, score=-8.723 tagged_above=-999 required=5 tests=[AWL=1.876, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8h1Kgc7crvKZ for ; Fri, 5 Oct 2012 07:56:55 -0700 (PDT) Received: from av-tac-bru.cisco.com (weird-brew.cisco.com [144.254.15.118]) by ietfa.amsl.com (Postfix) with ESMTP id 69E8921F86B5 for ; Fri, 5 Oct 2012 07:56:55 -0700 (PDT) X-TACSUNS: Virus Scanned Received: from strange-brew.cisco.com (localhost.cisco.com [127.0.0.1]) by av-tac-bru.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id q95EunOq017220; Fri, 5 Oct 2012 16:56:50 +0200 (CEST) Received: from [10.60.67.86] (ams-bclaise-8915.cisco.com [10.60.67.86]) by strange-brew.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id q95Eunuf029316; Fri, 5 Oct 2012 16:56:49 +0200 (CEST) Message-ID: <506EF531.1020408@cisco.com> Date: Fri, 05 Oct 2012 16:56:49 +0200 From: Benoit Claise User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:15.0) Gecko/20120907 Thunderbird/15.0.1 MIME-Version: 1.0 To: Brian Trammell References: <501AFA50.10508@cisco.com> <501FDE8E.9030001@cisco.com> <506EED9E.4030104@cisco.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: "ipfix@ietf.org" Subject: Re: [IPFIX] IPFIX interop? X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Oct 2012 14:56:56 -0000 Hi Brian, all, > Hi, Benoit, all, > > I think we should definitely interop those features we haven't yet, and have as an action item for myself making sure that all of those features are supported in my current implementation, but I believe at this point in time it probably makes sense to try to organize pairwise testing as opposed to putting together another large everyone-in-a-room event, especially as it seems as there's not much energy among implementors for doing one of those again (the last one was in March 2011). > > If 5101bis' progress doesn't gate on an interop, though, we can do one at our leisure, no? Yes. Regards, Benoit. > > Best regards, > > Brian > > > On Oct 5, 2012, at 4:24 PM, Benoit Claise wrote: > >> Dear all, >> >> At the last IETF Meeting, we discussed the possibility of an IPFIX interop. >> To give some background information, the goal was mainly to progress RFC5101 to RFC5101bis. >> However, based on the justification below (My email whose subject was "Re: [IPFIX] RFC 2026 versus RFC6410: progressing a document"), it's no required to test every single features to progress the standards track. So we could progress RFC5101bis. >> So I guess that the IPFIX interop will not take place. Correct? >> >> Regards, Benoit. >> >>> Dear all, >>> >>> And I'm trying to compare the conditions to progress a document in RFC 2026 and 6410 >>> >>> RFC 2026: >>> The requirement for at least two independent and interoperable >>> implementations applies to all of the options and features of the >>> specification. >>> In cases in which one or more options or features >>> have not been demonstrated in at least two interoperable >>> implementations, the specification may advance to the Draft Standard >>> level only if those options or features are removed. >>> >>> >>> RFC 6410 >>> The IESG, in an IETF-wide Last Call of at least four weeks, confirms >>> that a document advances from Proposed Standard to Internet Standard. >>> The request for reclassification is sent to the IESG along with an >>> explanation of how the criteria have been met. The criteria are: >>> >>> (1) There are at least two independent interoperating implementations >>> with widespread deployment and successful operational experience. >>> >>> >>> (2) There are no errata against the specification that would cause a >>> new implementation to fail to interoperate with deployed ones. >>> >>> >>> (3) There are no unused features in the specification that greatly >>> increase implementation complexity. >>> >>> >>> (4) If the technology required to implement the specification >>> requires patented or otherwise controlled technology, then the >>> set of implementations must demonstrate at least two independent, >>> separate and successful uses of the licensing process. >>> >>> >>> After confirmation from the IESG, we don't need to test every single feature from the specifications to progress the draft. >>> However, the 4 points above must be respected. >>> >>> Regards, Benoit >>> >>> >>> >>> >>> >>> >>> _______________________________________________ >>> IPFIX mailing list >>> >>> IPFIX@ietf.org >>> https://www.ietf.org/mailman/listinfo/ipfix >> _______________________________________________ >> IPFIX mailing list >> IPFIX@ietf.org >> https://www.ietf.org/mailman/listinfo/ipfix > > From internet-drafts@ietf.org Fri Oct 5 14:25:17 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5000B21F8703; Fri, 5 Oct 2012 14:25:17 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.499 X-Spam-Level: X-Spam-Status: No, score=-102.499 tagged_above=-999 required=5 tests=[AWL=0.100, BAYES_00=-2.599, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id w4m9hN40qq70; Fri, 5 Oct 2012 14:25:16 -0700 (PDT) Received: from ietfa.amsl.com (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D715A21F8692; Fri, 5 Oct 2012 14:25:16 -0700 (PDT) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: internet-drafts@ietf.org To: i-d-announce@ietf.org X-Test-IDTracker: no X-IETF-IDTracker: 4.34 Message-ID: <20121005212516.1209.55574.idtracker@ietfa.amsl.com> Date: Fri, 05 Oct 2012 14:25:16 -0700 Cc: ipfix@ietf.org Subject: [IPFIX] I-D Action: draft-ietf-ipfix-data-link-layer-monitoring-01.txt X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Oct 2012 21:25:17 -0000 A New Internet-Draft is available from the on-line Internet-Drafts director= ies. This draft is a work item of the IP Flow Information Export Working Group = of the IETF. Title : Information Elements for Data Link Layer Traffic Measure= ment Author(s) : Shingo Kashima Atsushi Kobayashi Paul Aitken Filename : draft-ietf-ipfix-data-link-layer-monitoring-01.txt Pages : 34 Date : 2012-10-05 Abstract: This document describes Information Elements related to data link layer. They are used by the IP Flow Information Export (IPFIX) protocol for encoding measured data link layer traffic information. The IETF datatracker status page for this draft is: https://datatracker.ietf.org/doc/draft-ietf-ipfix-data-link-layer-monitoring There's also a htmlized version available at: http://tools.ietf.org/html/draft-ietf-ipfix-data-link-layer-monitoring-01 A diff from the previous version is available at: http://www.ietf.org/rfcdiff?url2=3Ddraft-ietf-ipfix-data-link-layer-monitor= ing-01 Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ From paitken@cisco.com Fri Oct 5 14:28:06 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 00DDF21F870A for ; Fri, 5 Oct 2012 14:28:06 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.571 X-Spam-Level: X-Spam-Status: No, score=-10.571 tagged_above=-999 required=5 tests=[AWL=0.027, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id p60xrQNVpOse for ; Fri, 5 Oct 2012 14:28:05 -0700 (PDT) Received: from ams-iport-4.cisco.com (ams-iport-4.cisco.com [144.254.224.147]) by ietfa.amsl.com (Postfix) with ESMTP id 8D7C421F8692 for ; Fri, 5 Oct 2012 14:28:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=7088; q=dns/txt; s=iport; t=1349472484; x=1350682084; h=message-id:date:from:mime-version:to:subject; bh=Z3lFKzAWShWDScKIQsztR70xgnGkib2AXABF8+VDjSU=; b=G41GYEFed2E9FSlI/KPOXgltIcS+mPy/gzsY9G7OF92O4wHX1HdpS5Iv l5Soh69BUgc2LCDjCt60hoy2PAMMnBd4862dqm1M75CYv7c9h8QSdAMdd e6d+1jFyJVJF60fDIIhRoRbXIKIoquGrlhFSv5/W45LHi7Vh9TdDlj4Hv Y=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AgkFALlQb1CQ/khN/2dsb2JhbABFi2+zMYEIgjkBZT0WGAMCAQIBSw0IAQEeh2OXWIEooAiRRwOSOoMxhWKIY4Fpgm4 X-IronPort-AV: E=Sophos;i="4.80,541,1344211200"; d="scan'208,217";a="8568873" Received: from ams-core-4.cisco.com ([144.254.72.77]) by ams-iport-4.cisco.com with ESMTP; 05 Oct 2012 21:27:47 +0000 Received: from [10.56.224.29] ([10.56.224.29]) by ams-core-4.cisco.com (8.14.5/8.14.5) with ESMTP id q95LRjvb022460 for ; Fri, 5 Oct 2012 21:27:46 GMT Message-ID: <506F50D2.3040009@cisco.com> Date: Fri, 05 Oct 2012 22:27:46 +0100 From: Paul Aitken User-Agent: Mozilla/5.0 (X11; Linux i686; rv:15.0) Gecko/20120912 Thunderbird/15.0.1 MIME-Version: 1.0 To: IETF IPFIX Working Group Content-Type: multipart/alternative; boundary="------------030001090703030203090901" Subject: [IPFIX] draft-ietf-ipfix-data-link-layer-monitoring-01 X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Oct 2012 21:28:06 -0000 This is a multi-part message in MIME format. --------------030001090703030203090901 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Dear All, I had one action outstanding on this draft since the last WG meeting: One open issue: sectionOfset. Discussion focussed on the problem of having multiple offset IEs in an export record, and the need to have an ofset reference base. Paul will add more text explaining this. The "offset base" issue was already addressed by the text, though I've clarified: If this Information Element is omitted, it defaults to zero (ie, no offset). I've added the text below to address the "multiple offset IEs" issue. Please check whether this satisfies the request. Also, I found an orphaned paragraph (no "t" tag in the XML) at the end of section 2.4. I've fixed the tag so the text now shows. Thanks, P. OLD: 3.4. sectionOffset Description: This Information Element specifies the offset of the packet section (e.g., dataLinkFrameSection, ipHeaderPacketSection, ipPayloadPacketSection, mplsLabelStackSection and mplsPayloadPacketSection). If this Information Element is omitted, it defaults to zero. NEW: 3.4. sectionOffset Description: This Information Element specifies the offset of the packet section (e.g., dataLinkFrameSection, ipHeaderPacketSection, ipPayloadPacketSection, mplsLabelStackSection and mplsPayloadPacketSection). If this Information Element is omitted, it defaults to zero (ie, no offset). If multiple sectionOffset IEs are specified within a single Template, then they apply to the packet section IEs in order. ie, the first sectionOffset applies to the first packet section, etc. Note that the "closest" sectionOffset and packet section IEs within a given Template are not necessarily related. If there are less sectionOffset IEs than packet section IEs, then subsequent packet section IEs have no offset. If there are more sectionOffset IEs than the number of packet section IEs, then the additional sectionOffset IEs are meaningless. END. --------------030001090703030203090901 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Dear All,

I had one action outstanding on this draft since the last WG meeting:

          One open issue: sectionOfset.  Discussion focussed on the problem
          of having multiple offset IEs in an export record, and the need
          to have an ofset reference base.  Paul will add more text explaining
          this.


The "offset base" issue was already addressed by the text, though I've clarified:

    If this Information Element is omitted, it defaults to zero (ie, no offset).


I've added the text below to address the "multiple offset IEs" issue. Please check whether this satisfies the request.

Also, I found an orphaned paragraph (no "t" tag in the XML) at the end of section 2.4. I've fixed the tag so the text now shows.

Thanks,
P.


OLD:
3.4.  sectionOffset
  
   Description:
  
      This Information Element specifies the offset of the packet
      section (e.g., dataLinkFrameSection, ipHeaderPacketSection,
      ipPayloadPacketSection, mplsLabelStackSection and
      mplsPayloadPacketSection).  If this Information Element is
      omitted, it defaults to zero.

NEW:

3.4.  sectionOffset

   Description:
     
      This Information Element specifies the offset of the packet
      section (e.g., dataLinkFrameSection, ipHeaderPacketSection,
      ipPayloadPacketSection, mplsLabelStackSection and
      mplsPayloadPacketSection).  If this Information Element is
      omitted, it defaults to zero (ie, no offset).
      If multiple sectionOffset IEs are specified within a single
      Template, then they apply to the packet section IEs in order. ie,
      the first sectionOffset applies to the first packet section, etc.
      Note that the "closest" sectionOffset and packet section IEs
      within a given Template are not necessarily related.  If there are
      less sectionOffset IEs than packet section IEs, then subsequent
      packet section IEs have no offset.  If there are more
      sectionOffset IEs than the number of packet section IEs, then the
      additional sectionOffset IEs are meaningless.

END.

--------------030001090703030203090901-- From internet-drafts@ietf.org Thu Oct 11 09:49:55 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 88ECD21F8619; Thu, 11 Oct 2012 09:49:55 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.504 X-Spam-Level: X-Spam-Status: No, score=-102.504 tagged_above=-999 required=5 tests=[AWL=0.095, BAYES_00=-2.599, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id BEUwFj5Mgu4G; Thu, 11 Oct 2012 09:49:55 -0700 (PDT) Received: from ietfa.amsl.com (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2B74821F8602; Thu, 11 Oct 2012 09:49:55 -0700 (PDT) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: internet-drafts@ietf.org To: i-d-announce@ietf.org X-Test-IDTracker: no X-IETF-IDTracker: 4.34 Message-ID: <20121011164955.19872.83328.idtracker@ietfa.amsl.com> Date: Thu, 11 Oct 2012 09:49:55 -0700 Cc: ipfix@ietf.org Subject: [IPFIX] I-D Action: draft-ietf-ipfix-a9n-07.txt X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Oct 2012 16:49:55 -0000 A New Internet-Draft is available from the on-line Internet-Drafts director= ies. This draft is a work item of the IP Flow Information Export Working Group = of the IETF. Title : Flow Aggregation for the IP Flow Information Export (IPF= IX) Protocol Author(s) : Brian Trammell Arno Wagner Benoit Claise Filename : draft-ietf-ipfix-a9n-07.txt Pages : 57 Date : 2012-10-11 Abstract: This document provides a common implementation-independent basis for the interoperable application of the IP Flow Information Export (IPFIX) Protocol to the handling of Aggregated Flows, which are IPFIX Flows representing packets from multiple Original Flows sharing some set of common properties. It does this through a detailed terminology and a descriptive Intermediate Aggregation Process architecture, including a specification of methods for Original Flow counting and counter distribution across intervals. The IETF datatracker status page for this draft is: https://datatracker.ietf.org/doc/draft-ietf-ipfix-a9n There's also a htmlized version available at: http://tools.ietf.org/html/draft-ietf-ipfix-a9n-07 A diff from the previous version is available at: http://www.ietf.org/rfcdiff?url2=3Ddraft-ietf-ipfix-a9n-07 Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ From n.brownlee@auckland.ac.nz Sun Oct 14 19:36:34 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2195621F8568 for ; Sun, 14 Oct 2012 19:36:34 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -105.342 X-Spam-Level: X-Spam-Status: No, score=-105.342 tagged_above=-999 required=5 tests=[AWL=-1.157, BAYES_40=-0.185, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id oYp4khtFuwBC for ; Sun, 14 Oct 2012 19:36:29 -0700 (PDT) Received: from mx2.auckland.ac.nz (mx2.auckland.ac.nz [130.216.12.44]) by ietfa.amsl.com (Postfix) with ESMTP id EBAC621F855F for ; Sun, 14 Oct 2012 19:36:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=auckland.ac.nz; i=n.brownlee@auckland.ac.nz; q=dns/txt; s=uoa; t=1350268589; x=1381804589; h=message-id:date:from:mime-version:to:subject: content-transfer-encoding; bh=LguZvszBmowwKd4L6UI9rgqd/emlkkYIDGwRLOe0zwg=; b=Rc031lpZqMek+Of2C98s+r2QcUqlGwsXN0B4XV8Ebwx/Rba2P6ypLGH5 5hBkzTybdSv3aM682rFc7KDiL2P+dTrAi4Xec7KQ+zQEwj034HL8lsHpj EFNC9N1wX0K1uHcr69jwYULG/pu4e+lxFgc6ubrvX8lXXpRNWES8U+fgN A=; X-IronPort-AV: E=Sophos;i="4.80,586,1344168000"; d="scan'208";a="150991793" X-Ironport-HAT: UNIVERSITY - $RELAY-THROTTLE X-Ironport-Source: 130.216.38.131 - Outgoing - Outgoing-SSL Received: from nevil-laptop1.sfac.auckland.ac.nz (HELO [130.216.38.131]) ([130.216.38.131]) by mx2-int.auckland.ac.nz with ESMTP; 15 Oct 2012 15:36:27 +1300 Message-ID: <507B76AA.6020508@auckland.ac.nz> Date: Mon, 15 Oct 2012 15:36:26 +1300 From: Nevil Brownlee User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:16.0) Gecko/20121010 Thunderbird/16.0.1 MIME-Version: 1.0 To: IPFIX Working Group Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: [IPFIX] WG Last Call for draft-ietf-ipfix-data-link-layer-monitoring-01 X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Oct 2012 02:36:34 -0000 HI IPFIXers: Paul has posted this draft, addressing the issue raised at our Meeting in Vancouver. The WG Last Call for it starts now, and will end just after IETF-85, i.e. on Sunday, 11 November. We'll ask the IEEE linklayer folk to comment, but of course we need your feedback too. Please read it through, and comment briefly on the IPFIX list. If you're able to review it, that would be great. However, comments like "I see no problems with this" would also be useful in judging consensus! Cheers, Nevil -- --------------------------------------------------------------------- Nevil Brownlee Computer Science Department | ITS Phone: +64 9 373 7599 x88941 The University of Auckland FAX: +64 9 373 7453 Private Bag 92019, Auckland 1142, New Zealand From n.brownlee@auckland.ac.nz Tue Oct 16 20:20:24 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0874021F8764 for ; Tue, 16 Oct 2012 20:20:24 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.46 X-Spam-Level: X-Spam-Status: No, score=-106.46 tagged_above=-999 required=5 tests=[AWL=0.139, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id l6uwU4hRzw3T for ; Tue, 16 Oct 2012 20:20:23 -0700 (PDT) Received: from mx2.auckland.ac.nz (mx2.auckland.ac.nz [130.216.12.44]) by ietfa.amsl.com (Postfix) with ESMTP id 86B4121F8772 for ; Tue, 16 Oct 2012 20:20:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=auckland.ac.nz; i=n.brownlee@auckland.ac.nz; q=dns/txt; s=uoa; t=1350444023; x=1381980023; h=message-id:date:from:mime-version:to:subject: content-transfer-encoding; bh=X77k/0o+/lj49hbDhlRwY/bXm8p9X6Wml6+A8ZUvPk8=; b=QJIZKapglk4nJL+13QO8sOvm014HT1R4cYxazfpJmvQCQMAeZ24qVXKu H/+pSJkNZPFDZFp+9tjclzuGoZ/d4SGzEFpaH6+cuzrJbAHdfu04NVQE3 JEKwUC1Ww5NFVxAqEuiLzTYo+N3fkFJLrast3M+9h2+uQXBLkXM54RQ+s g=; X-IronPort-AV: E=Sophos;i="4.80,597,1344168000"; d="scan'208";a="151463429" X-Ironport-HAT: UNIVERSITY - $RELAY-THROTTLE X-Ironport-Source: 130.216.38.131 - Outgoing - Outgoing-SSL Received: from nevil-laptop1.sfac.auckland.ac.nz (HELO [130.216.38.131]) ([130.216.38.131]) by mx2-int.auckland.ac.nz with ESMTP; 17 Oct 2012 16:20:00 +1300 Message-ID: <507E23E0.3030803@auckland.ac.nz> Date: Wed, 17 Oct 2012 16:20:00 +1300 From: Nevil Brownlee User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:16.0) Gecko/20121010 Thunderbird/16.0.1 MIME-Version: 1.0 To: IPFIX Working Group Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: [IPFIX] DRAFT agenda for IETF 85 in Atlanta published X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Oct 2012 03:20:24 -0000 Hi all: The -00 version of our agenda for Atlanta is now on https://datatracker.ietf.org/meeting/85/agenda/ipfix/ Please note that the "Other drafts" section is simply copied from our last meeting. If you have material you wish to present, please email me to let me know. That includes items 4(a) and 4(b)! Any other comments or suggestions are, of course, welcome. Cheers, Nevil -- --------------------------------------------------------------------- Nevil Brownlee Computer Science Department | ITS Phone: +64 9 373 7599 x88941 The University of Auckland FAX: +64 9 373 7453 Private Bag 92019, Auckland 1142, New Zealand From trammell@tik.ee.ethz.ch Mon Oct 22 04:09:08 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8A8FC21F849D for ; Mon, 22 Oct 2012 04:09:08 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.466 X-Spam-Level: X-Spam-Status: No, score=-6.466 tagged_above=-999 required=5 tests=[AWL=-0.467, BAYES_00=-2.599, J_CHICKENPOX_37=0.6, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4MuwjkXWLHJP for ; Mon, 22 Oct 2012 04:09:04 -0700 (PDT) Received: from smtp.ee.ethz.ch (smtp.ee.ethz.ch [129.132.2.219]) by ietfa.amsl.com (Postfix) with ESMTP id CDE4F21F8471 for ; Mon, 22 Oct 2012 04:09:03 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by smtp.ee.ethz.ch (Postfix) with ESMTP id 2716ED930B for ; Mon, 22 Oct 2012 13:08:59 +0200 (MEST) X-Virus-Scanned: by amavisd-new on smtp.ee.ethz.ch Received: from smtp.ee.ethz.ch ([127.0.0.1]) by localhost (.ee.ethz.ch [127.0.0.1]) (amavisd-new, port 10024) with LMTP id VTQEF9MR5uBY for ; Mon, 22 Oct 2012 13:08:58 +0200 (MEST) Received: from pb-10243.ethz.ch (pb-10243.ethz.ch [82.130.102.152]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: briant) by smtp.ee.ethz.ch (Postfix) with ESMTPSA id DED02D9308 for ; Mon, 22 Oct 2012 13:08:58 +0200 (MEST) From: Brian Trammell Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Date: Mon, 22 Oct 2012 13:08:57 +0200 Message-Id: <2AF7F2E1-BF78-4087-89B0-92427F39ECA0@tik.ee.ethz.ch> To: IETF IPFIX Working Group Mime-Version: 1.0 (Apple Message framework v1278) X-Mailer: Apple Mail (2.1278) Subject: [IPFIX] Endianness of address types: 5101bis section 6.1.2. X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Oct 2012 11:09:08 -0000 Greetings, all, I've received a comment off-list from an implementor that section 6.1.2 = of 5101 (and 5101-bis) is potentially confusingly worded. The intent = here is that all addresses be _encoded_ in big-endian order, regardless = of how they are handled internally (e.g., Unix programs often use both = host and network byte order for IPv4 addresses: the former for = manipulation, the latter for compatibility with the sockets API). = "6-octet integers" and "16-octet integers", however, are kind of weird = concepts. So I'd suggest the following change: OLD: 6.1.2. Address Types Address types -- macAddress, ipv4Address, and ipv6Address -- MUST be encoded the same way as the integral data types. The macAddress is treated as a 6-octet integer, the ipv4Address as a 4-octet integer, and the ipv6Address as a 16-octet integer. NEW: Address types -- ipv4Address, macAddress, and ipv6Address -- MUST be encoded the same way as unsigned integral data types, as four, six,=20= and sixteen octets in network byte order, respectively. Thoughts? Thanks, Brian From internet-drafts@ietf.org Mon Oct 22 09:34:51 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7349421F8B03; Mon, 22 Oct 2012 09:34:51 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.539 X-Spam-Level: X-Spam-Status: No, score=-102.539 tagged_above=-999 required=5 tests=[AWL=0.060, BAYES_00=-2.599, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1LBNv3u9Vyzw; Mon, 22 Oct 2012 09:34:50 -0700 (PDT) Received: from ietfa.amsl.com (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D8A4421F8AFB; Mon, 22 Oct 2012 09:34:50 -0700 (PDT) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: internet-drafts@ietf.org To: i-d-announce@ietf.org X-Test-IDTracker: no X-IETF-IDTracker: 4.34 Message-ID: <20121022163450.19019.97275.idtracker@ietfa.amsl.com> Date: Mon, 22 Oct 2012 09:34:50 -0700 Cc: ipfix@ietf.org Subject: [IPFIX] I-D Action: draft-ietf-ipfix-mib-variable-export-01.txt X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Oct 2012 16:34:51 -0000 A New Internet-Draft is available from the on-line Internet-Drafts director= ies. This draft is a work item of the IP Flow Information Export Working Group = of the IETF. Title : Exporting MIB Variables using the IPFIX Protocol Author(s) : Benoit Claise Paul Aitken Juergen Schoenwaelder Filename : draft-ietf-ipfix-mib-variable-export-01.txt Pages : 57 Date : 2012-10-22 Abstract: This document specifies a way to complement IPFIX Flow Records with Management Information Base (MIB) objects, avoiding the need to define new IPFIX Information Elements for existing Management Information Base objects that are already fully specified. This method requires an extension to the current IPFIX protocol. New Template Set and Options Template Sets are specified to allow the export of Extended Field Specifiers, which may represent IPFIX Information Elements and Simple Network Management Protocol (SNMP) MIB Objects. The IETF datatracker status page for this draft is: https://datatracker.ietf.org/doc/draft-ietf-ipfix-mib-variable-export There's also a htmlized version available at: http://tools.ietf.org/html/draft-ietf-ipfix-mib-variable-export-01 A diff from the previous version is available at: http://www.ietf.org/rfcdiff?url2=3Ddraft-ietf-ipfix-mib-variable-export-01 Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ From trammell@tik.ee.ethz.ch Mon Oct 22 09:54:06 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B348621F8A0F for ; Mon, 22 Oct 2012 09:54:03 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.73 X-Spam-Level: X-Spam-Status: No, score=-6.73 tagged_above=-999 required=5 tests=[AWL=-0.131, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 35yhZ+-RmRIv for ; Mon, 22 Oct 2012 09:53:43 -0700 (PDT) Received: from smtp.ee.ethz.ch (smtp.ee.ethz.ch [129.132.2.219]) by ietfa.amsl.com (Postfix) with ESMTP id 8F68D21F8B4D for ; Mon, 22 Oct 2012 09:53:37 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by smtp.ee.ethz.ch (Postfix) with ESMTP id A8DA0D930B; Mon, 22 Oct 2012 18:53:35 +0200 (MEST) X-Virus-Scanned: by amavisd-new on smtp.ee.ethz.ch Received: from smtp.ee.ethz.ch ([127.0.0.1]) by localhost (.ee.ethz.ch [127.0.0.1]) (amavisd-new, port 10024) with LMTP id A2LM8dv0ukYG; Mon, 22 Oct 2012 18:53:35 +0200 (MEST) Received: from pb-10243.ethz.ch (pb-10243.ethz.ch [82.130.102.152]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: briant) by smtp.ee.ethz.ch (Postfix) with ESMTPSA id 47D60D930A; Mon, 22 Oct 2012 18:53:35 +0200 (MEST) Mime-Version: 1.0 (Apple Message framework v1278) Content-Type: text/plain; charset=us-ascii From: Brian Trammell In-Reply-To: <507B76AA.6020508@auckland.ac.nz> Date: Mon, 22 Oct 2012 18:53:34 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <026C1D4B-D592-4DA4-85BE-28FBB38FEA43@tik.ee.ethz.ch> References: <507B76AA.6020508@auckland.ac.nz> To: Nevil Brownlee X-Mailer: Apple Mail (2.1278) Cc: IPFIX Working Group Subject: Re: [IPFIX] WG Last Call for draft-ietf-ipfix-data-link-layer-monitoring-01 X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Oct 2012 16:54:07 -0000 Greetings, all, I've done a quick review of = draft-ietf-ipfix-data-link-layer-monitoring-01 for WGLC; I haven't = checked the IE definitions for correctness (I'm not a layer 2 geek, and = will rely on external review from IEEE folks for that). However, I do = have a couple of comments. This draft needs another revision before AD review, mainly to complete = Section 4, to generalize packet section export per the earlier = discussion on the subject (see = http://www.ietf.org/mail-archive/web/ipfix/current/msg06475.html ) -- = Would these changes also obviate the need for dataLinkFrameSize and = dataLinkFrameSection as defined by Sections 3.1 and 3.2?=20 Additional comments, per section: 3.3. dataLinkFrameType Is there any existing registry for these? All I can think of off the = type of my head are the datalink types for libpcap, which are probably = not appropriate, but it would be nice if we had something to refer to = here. Section 8 will need to be more specific about IANA actions with = respect to the registry created by this section.=20 5. Modification of Existing Information Elements Related to VLAN Tag Sections 5.1 through 5.4 will need thorough review by someone deeply = familiar with 802.1q to determine whether the new descriptions for these = Information Elements are interoperable with the existing descriptions, = as per Section 5.2 of IE-DOCTORS. If they are not interoperable, we will = need to create new Information Elements and deprecate the existing ones. 8. IANA Considerations This section needs to be more specific about IANA actions with respect = to the registry created for dataLinkFrameType. In keeping with existing = policy on subregistries, I presume this will be subject to Expert = Review.=20 Cheers, Brian On 15 Oct 2012, at 4:36 , Nevil Brownlee wrote: >=20 > HI IPFIXers: >=20 > Paul has posted this draft, addressing the issue raised at our > Meeting in Vancouver. >=20 > The WG Last Call for it starts now, and will end just after IETF-85, > i.e. on Sunday, 11 November. >=20 > We'll ask the IEEE linklayer folk to comment, but of course we need > your feedback too. Please read it through, and comment briefly on > the IPFIX list. If you're able to review it, that would be great. > However, comments like "I see no problems with this" would also be > useful in judging consensus! >=20 > Cheers, Nevil >=20 > --=20 > --------------------------------------------------------------------- > Nevil Brownlee Computer Science Department | ITS > Phone: +64 9 373 7599 x88941 The University of Auckland > FAX: +64 9 373 7453 Private Bag 92019, Auckland 1142, New Zealand > _______________________________________________ > IPFIX mailing list > IPFIX@ietf.org > https://www.ietf.org/mailman/listinfo/ipfix From andrewf@plixer.com Mon Oct 22 10:39:52 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id BCC0721F889F for ; Mon, 22 Oct 2012 10:39:52 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.199 X-Spam-Level: X-Spam-Status: No, score=-2.199 tagged_above=-999 required=5 tests=[AWL=-0.200, BAYES_00=-2.599, J_CHICKENPOX_37=0.6] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id RUcuqEkvc5NW for ; Mon, 22 Oct 2012 10:39:52 -0700 (PDT) Received: from smtp.plixer.com (smtp.plixer.com [66.186.184.193]) by ietfa.amsl.com (Postfix) with ESMTP id 1A2E721F85B8 for ; Mon, 22 Oct 2012 10:39:51 -0700 (PDT) Received: from [10.100.1.132] ([10.100.1.132]) by smtp.plixer.com over TLS secured channel with Microsoft SMTPSVC(6.0.3790.4675); Mon, 22 Oct 2012 13:39:50 -0400 Message-ID: <508584E6.2050506@plixer.com> Date: Mon, 22 Oct 2012 13:39:50 -0400 From: Andrew Feren User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:19.0) Gecko/19.0 Thunderbird/19.0a1 MIME-Version: 1.0 To: ipfix@ietf.org References: <2AF7F2E1-BF78-4087-89B0-92427F39ECA0@tik.ee.ethz.ch> In-Reply-To: <2AF7F2E1-BF78-4087-89B0-92427F39ECA0@tik.ee.ethz.ch> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 22 Oct 2012 17:39:50.0825 (UTC) FILETIME=[3CED9990:01CDB07C] Subject: Re: [IPFIX] Endianness of address types: 5101bis section 6.1.2. X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Oct 2012 17:39:52 -0000 Hi Brian, I can't say found the original confusing, but if some people did I don't have a problem with the new form. Or even more simply. Address types -- ipv4Address, macAddress, and ipv6Address -- MUST be encoded as four, six, and sixteen octets in network byte order, respectively. -Andrew On 10/22/2012 07:08 AM, Brian Trammell wrote: > Greetings, all, > > I've received a comment off-list from an implementor that section 6.1.2 of 5101 (and 5101-bis) is potentially confusingly worded. The intent here is that all addresses be _encoded_ in big-endian order, regardless of how they are handled internally (e.g., Unix programs often use both host and network byte order for IPv4 addresses: the former for manipulation, the latter for compatibility with the sockets API). "6-octet integers" and "16-octet integers", however, are kind of weird concepts. So I'd suggest the following change: > > OLD: > > 6.1.2. Address Types > > > Address types -- macAddress, ipv4Address, and ipv6Address -- MUST be > encoded the same way as the integral data types. The macAddress is > treated as a 6-octet integer, the ipv4Address as a 4-octet integer, > and the ipv6Address as a 16-octet integer. > > NEW: > > Address types -- ipv4Address, macAddress, and ipv6Address -- MUST be > encoded the same way as unsigned integral data types, as four, six, > and sixteen octets in network byte order, respectively. > > > Thoughts? > > Thanks, > > Brian > > _______________________________________________ > IPFIX mailing list > IPFIX@ietf.org > https://www.ietf.org/mailman/listinfo/ipfix From paitken@cisco.com Mon Oct 22 10:44:13 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1C68621F88C1 for ; Mon, 22 Oct 2012 10:44:13 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.274 X-Spam-Level: X-Spam-Status: No, score=-10.274 tagged_above=-999 required=5 tests=[AWL=-0.275, BAYES_00=-2.599, J_CHICKENPOX_37=0.6, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jg3C3nLqg8bt for ; Mon, 22 Oct 2012 10:44:12 -0700 (PDT) Received: from ams-iport-2.cisco.com (ams-iport-2.cisco.com [144.254.224.141]) by ietfa.amsl.com (Postfix) with ESMTP id B4E1521F894D for ; Mon, 22 Oct 2012 10:44:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=2220; q=dns/txt; s=iport; t=1350927852; x=1352137452; h=message-id:date:from:mime-version:to:cc:subject: references:in-reply-to:content-transfer-encoding; bh=BM/1Yk6JIPPSIMhzLIcgb4j/EDLy+BMfoZRxkIoI1P0=; b=JQjtq+vnHLjTqMKtVQECxyWHOPeUwl5+fKUqd2EaKML1Ar+a3f5yydrx ZuqypE/IyGB4lHk6LqHEETw8E6iCQXJv0zjjRCpROtcYyHAka5UDrrIv6 YPaJwL3zDhsRHHqekydeXJdBJtA5dK7lk7ZzHugJwE+Bq7qOZLB9lEIKj 0=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Av8EAKiFhVCQ/khR/2dsb2JhbABFwReBCIIgAQEBBAEBAQ8BJTYKARALGAkWDwkDAgECARUwBg0BBQIBAR6HYgucEJ91BItfhm8Dkj+DMoVkiGqBa4Jw X-IronPort-AV: E=Sophos;i="4.80,631,1344211200"; d="scan'208";a="77662844" Received: from ams-core-1.cisco.com ([144.254.72.81]) by ams-iport-2.cisco.com with ESMTP; 22 Oct 2012 17:44:10 +0000 Received: from [10.55.90.15] (dhcp-10-55-90-15.cisco.com [10.55.90.15]) by ams-core-1.cisco.com (8.14.5/8.14.5) with ESMTP id q9MHi9uI023257; Mon, 22 Oct 2012 17:44:10 GMT Message-ID: <508585EA.6090404@cisco.com> Date: Mon, 22 Oct 2012 18:44:10 +0100 From: Paul Aitken User-Agent: Mozilla/5.0 (X11; Linux i686; rv:15.0) Gecko/20120912 Thunderbird/15.0.1 MIME-Version: 1.0 To: Brian Trammell References: <2AF7F2E1-BF78-4087-89B0-92427F39ECA0@tik.ee.ethz.ch> <508584E6.2050506@plixer.com> In-Reply-To: <508584E6.2050506@plixer.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: ipfix@ietf.org Subject: Re: [IPFIX] Endianness of address types: 5101bis section 6.1.2. X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Oct 2012 17:44:13 -0000 Brian, +1 to what Andrew said. Additionally, please cite an xref for "network byte order". (Why isn't there an IETF terminology doc for these things?) Else, - RFC 5101 section 3? - RFC 1886 or RFC4122? - draft-newman-network-byte-order from '99 ? Thanks, P. On 22/10/12 18:39, Andrew Feren wrote: > Hi Brian, > > I can't say found the original confusing, but if some people did I > don't have a problem with the new form. Or even more simply. > > Address types -- ipv4Address, macAddress, and ipv6Address -- MUST be > encoded as four, six, and sixteen octets in network byte order, > respectively. > > -Andrew > > On 10/22/2012 07:08 AM, Brian Trammell wrote: >> Greetings, all, >> >> I've received a comment off-list from an implementor that section >> 6.1.2 of 5101 (and 5101-bis) is potentially confusingly worded. The >> intent here is that all addresses be _encoded_ in big-endian order, >> regardless of how they are handled internally (e.g., Unix programs >> often use both host and network byte order for IPv4 addresses: the >> former for manipulation, the latter for compatibility with the >> sockets API). "6-octet integers" and "16-octet integers", however, >> are kind of weird concepts. So I'd suggest the following change: >> >> OLD: >> >> 6.1.2. Address Types >> >> >> Address types -- macAddress, ipv4Address, and ipv6Address -- MUST be >> encoded the same way as the integral data types. The macAddress is >> treated as a 6-octet integer, the ipv4Address as a 4-octet integer, >> and the ipv6Address as a 16-octet integer. >> >> NEW: >> >> Address types -- ipv4Address, macAddress, and ipv6Address -- MUST be >> encoded the same way as unsigned integral data types, as four, six, >> and sixteen octets in network byte order, respectively. >> >> >> Thoughts? >> >> Thanks, >> >> Brian >> >> _______________________________________________ >> IPFIX mailing list >> IPFIX@ietf.org >> https://www.ietf.org/mailman/listinfo/ipfix > > _______________________________________________ > IPFIX mailing list > IPFIX@ietf.org > https://www.ietf.org/mailman/listinfo/ipfix From johnwcrt@au1.ibm.com Mon Oct 22 18:02:35 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2DFD91F0C5F for ; Mon, 22 Oct 2012 18:02:35 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.598 X-Spam-Level: X-Spam-Status: No, score=-6.598 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id iHikX6teuKer for ; Mon, 22 Oct 2012 18:02:30 -0700 (PDT) Received: from e23smtp03.au.ibm.com (e23smtp03.au.ibm.com [202.81.31.145]) by ietfa.amsl.com (Postfix) with ESMTP id 27F711F0C51 for ; Mon, 22 Oct 2012 18:02:29 -0700 (PDT) Received: from /spool/local by e23smtp03.au.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 23 Oct 2012 10:59:43 +1000 Received: from d23relay03.au.ibm.com (202.81.31.245) by e23smtp03.au.ibm.com (202.81.31.209) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Tue, 23 Oct 2012 10:59:41 +1000 Received: from d23av01.au.ibm.com (d23av01.au.ibm.com [9.190.234.96]) by d23relay03.au.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id q9N12HKH40304740 for ; Tue, 23 Oct 2012 12:02:20 +1100 Received: from d23av01.au.ibm.com (loopback [127.0.0.1]) by d23av01.au.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id q9N12HDH001780 for ; Tue, 23 Oct 2012 12:02:17 +1100 Received: from d23mlc03.au.ibm.com (d23mlc03.au.ibm.com [9.190.26.210]) by d23av01.au.ibm.com (8.14.4/8.13.1/NCO v10.0 AVin) with ESMTP id q9N12HvC001777 for ; Tue, 23 Oct 2012 12:02:17 +1100 To: ipfix@ietf.org MIME-Version: 1.0 X-KeepSent: BEE7B680:CE11B7E3-CA257AA0:0001FAB7; type=4; name=$KeepSent X-Mailer: Lotus Notes Release 8.5.3 September 15, 2011 Message-ID: From: John Court Date: Tue, 23 Oct 2012 11:01:31 +1000 X-MIMETrack: Serialize by Router on d23mlc03/23/M/IBM(Release 8.5.3FP2HF29 | July 24, 2012) at 23/10/2012 12:01:34, Serialize complete at 23/10/2012 12:01:34 Content-Type: multipart/alternative; boundary="=_alternative 0005B27A4A257AA0_=" x-cbid: 12102300-6102-0000-0000-0000026B22D8 Subject: [IPFIX] Export of long lived flow information X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Oct 2012 01:02:35 -0000 This is a multipart message in MIME format. --=_alternative 0005B27A4A257AA0_= Content-Type: text/plain; charset="US-ASCII" Hi, I have been a subscriber to the list for a little over a year, and an implementer of IPFIX export for at least one product. This WG has done great work overall ! One area that still has me a little confused even after researching as many of the RFCs as possible including RFC5472 is how to treat export of long lived flows. At the moment I use "DeltaCount" information elements for everything and at specific intervals export long lived flows with the flowEndReason of "flowActiveTimeout". This of course results in multiple flow records for long lived connections over time. Since this situation doesn't seem to be covered explicitly I was hoping someone on the list would point me in the right direction or confirm my assumptions. On thing that is particularly unclear is what to do about flowStart/flowEnd times when sending this type of record. Thanks John Court Senior Software Engineer IBM Security Systems Division IBM Australia Development Laboratory Office: +61 7 5552 4014 Mobile: +61 430 841328 --=_alternative 0005B27A4A257AA0_= Content-Type: text/html; charset="US-ASCII" Hi,

I have been a subscriber to the list for a little over a year, and an implementer of IPFIX export for at least one product.  This WG has done great work overall !

One area that still has me a little confused even after researching as many of the RFCs as possible including RFC5472 is how to treat export of long lived flows.

At the moment I use "DeltaCount" information elements for everything and at specific intervals export long lived flows with the flowEndReason of "flowActiveTimeout".  This of course results in multiple flow records for long lived connections over time.  Since this situation doesn't seem to be covered explicitly I was hoping someone on the list would point me in the right direction or confirm my assumptions.  On thing that is particularly unclear is what to do about flowStart/flowEnd times when sending this type of record.

Thanks

John Court
Senior Software Engineer
IBM Security Systems Division
IBM Australia Development Laboratory
Office:  +61 7 5552 4014
Mobile: +61 430 841328

 --=_alternative 0005B27A4A257AA0_=-- From trammell@tik.ee.ethz.ch Tue Oct 23 00:43:19 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 62E3121F863F for ; Tue, 23 Oct 2012 00:43:19 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.71 X-Spam-Level: X-Spam-Status: No, score=-6.71 tagged_above=-999 required=5 tests=[AWL=-0.111, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FcRTeJTmm03o for ; Tue, 23 Oct 2012 00:43:18 -0700 (PDT) Received: from smtp.ee.ethz.ch (smtp.ee.ethz.ch [129.132.2.219]) by ietfa.amsl.com (Postfix) with ESMTP id 6874B21F863A for ; Tue, 23 Oct 2012 00:43:18 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by smtp.ee.ethz.ch (Postfix) with ESMTP id 6D8C4D9312; Tue, 23 Oct 2012 09:43:17 +0200 (MEST) X-Virus-Scanned: by amavisd-new on smtp.ee.ethz.ch Received: from smtp.ee.ethz.ch ([127.0.0.1]) by localhost (.ee.ethz.ch [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 3C7VHoNng7QK; Tue, 23 Oct 2012 09:43:17 +0200 (MEST) Received: from [10.0.27.100] (cust-integra-121-161.antanet.ch [80.75.121.161]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: briant) by smtp.ee.ethz.ch (Postfix) with ESMTPSA id 2D1F9D930B; Tue, 23 Oct 2012 09:43:17 +0200 (MEST) Mime-Version: 1.0 (Apple Message framework v1283) Content-Type: text/plain; charset=us-ascii From: Brian Trammell In-Reply-To: Date: Tue, 23 Oct 2012 09:43:12 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: To: John Court X-Mailer: Apple Mail (2.1283) Cc: ipfix@ietf.org Subject: Re: [IPFIX] Export of long lived flow information X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Oct 2012 07:43:19 -0000 Hi, John, Your assumptions are basically correct. The IPFIX model of a Metering = Process (MP) makes an implicit assumption that it will be configured or = configurable to export information about long-lived flows every n = minutes through active timeout. =46rom the MP side, this allows periodic = complete flushing of the flow cache.=20 More importantly (IMO), on the Collecting Process (CP) side, it provides = a guarantee that every packet observed and selected for export will be = accounted for within Ta + Tde + Tdc, where Ta is the active timeout, Tde = is the MP to EP delay (how long it takes for an exported flow to make it = from the MP cache, through the Exporting Process (EP), into an IPFIX = Message, which is implementation dependent but typically short) and Tde = is the export delay (OWD from EP to CP). This is important for streaming = process applications, as after this time, the CP and downstream = processes can assume that no further information about the past will = become available. The tradeoff here is that shorter Ta causes more records to be exported = about long flows, and a longer Ta causes a longer delay, which some = streaming applications can't tolerate. In any case, the assumption is that the flow is no longer in the cache = after active timeout, so you don't know when the flow really started. = Since the flowStartTime is the timestamp of the first observed packet = within the record, and flowEndTime is the timestamp of the last observed = packet within the record, the timestamps would then be record-local.... = i.e. a flow less than twice the active timeout (with at least one packet = per idle timeout) would result in two flow records. The first would have = a timestamp range between the first packet of the flow and the last = packet observed before the active timeout; the second between the first = packet observed after the active timeout and the last packet of the = flow. This does, admittedly, make it rather difficult to stitch records for = long flows together -- you can't match timestamps, and essentially need = to simulate the flow cache with a longer active timeout. For = applications requiring a single record per flow, active timeouts can be = set to be practically infinite, with the tradeoff that you never know at = the CP when you'll get a flow with a start time far in the past. Best regards, Brian On Oct 23, 2012, at 3:01 AM, John Court wrote: > Hi,=20 >=20 > I have been a subscriber to the list for a little over a year, and an = implementer of IPFIX export for at least one product. This WG has done = great work overall !=20 >=20 > One area that still has me a little confused even after researching as = many of the RFCs as possible including RFC5472 is how to treat export of = long lived flows.=20 >=20 > At the moment I use "DeltaCount" information elements for everything = and at specific intervals export long lived flows with the flowEndReason = of "flowActiveTimeout". This of course results in multiple flow records = for long lived connections over time. Since this situation doesn't seem = to be covered explicitly I was hoping someone on the list would point me = in the right direction or confirm my assumptions. On thing that is = particularly unclear is what to do about flowStart/flowEnd times when = sending this type of record.=20 >=20 > Thanks=20 >=20 > John Court > Senior Software Engineer > IBM Security Systems Division > IBM Australia Development Laboratory > Office: +61 7 5552 4014 > Mobile: +61 430 841328 >=20 > _______________________________________________ > IPFIX mailing list > IPFIX@ietf.org > https://www.ietf.org/mailman/listinfo/ipfix From trammell@tik.ee.ethz.ch Tue Oct 23 00:52:46 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id BC64721F866B for ; Tue, 23 Oct 2012 00:52:46 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.399 X-Spam-Level: X-Spam-Status: No, score=-6.399 tagged_above=-999 required=5 tests=[AWL=-0.400, BAYES_00=-2.599, J_CHICKENPOX_37=0.6, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id YkNSv9xZZUsb for ; Tue, 23 Oct 2012 00:52:46 -0700 (PDT) Received: from smtp.ee.ethz.ch (smtp.ee.ethz.ch [129.132.2.219]) by ietfa.amsl.com (Postfix) with ESMTP id E2FBC21F8666 for ; Tue, 23 Oct 2012 00:52:45 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by smtp.ee.ethz.ch (Postfix) with ESMTP id 618E9D9312; Tue, 23 Oct 2012 09:52:44 +0200 (MEST) X-Virus-Scanned: by amavisd-new on smtp.ee.ethz.ch Received: from smtp.ee.ethz.ch ([127.0.0.1]) by localhost (.ee.ethz.ch [127.0.0.1]) (amavisd-new, port 10024) with LMTP id vDdEJo5Z5pgw; Tue, 23 Oct 2012 09:52:44 +0200 (MEST) Received: from [10.0.27.100] (cust-integra-121-161.antanet.ch [80.75.121.161]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: briant) by smtp.ee.ethz.ch (Postfix) with ESMTPSA id 0B5F5D930B; Tue, 23 Oct 2012 09:52:44 +0200 (MEST) Mime-Version: 1.0 (Apple Message framework v1283) Content-Type: text/plain; charset=iso-8859-1 From: Brian Trammell In-Reply-To: <508585EA.6090404@cisco.com> Date: Tue, 23 Oct 2012 09:52:43 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <2AF7F2E1-BF78-4087-89B0-92427F39ECA0@tik.ee.ethz.ch> <508584E6.2050506@plixer.com> <508585EA.6090404@cisco.com> To: Paul Aitken X-Mailer: Apple Mail (2.1283) Cc: ipfix@ietf.org Subject: Re: [IPFIX] Endianness of address types: 5101bis section 6.1.2. X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Oct 2012 07:52:46 -0000 Hi, Paul, Hm. The lack of documentation on network byte order is odd, though I = suppose the words "network byte order" are on their own sufficient for = any reasonably competent implementor to know what we're talking about. = So I think the right thing to do is be very clear that everything in = IPFIX is network byte order (as you say, in section 3).=20 Will go with Andrew's language, and review 5101bis to ensure that we're = as clear about byte order as we need to be elsewhere. Many thanks, cheers, Brian On Oct 22, 2012, at 7:44 PM, Paul Aitken wrote: > Brian, >=20 > +1 to what Andrew said. >=20 > Additionally, please cite an xref for "network byte order". >=20 > (Why isn't there an IETF terminology doc for these things?) >=20 > Else, >=20 > - RFC 5101 section 3? > - RFC 1886 or RFC4122? > - draft-newman-network-byte-order from '99 ? >=20 > Thanks, > P. >=20 >=20 > On 22/10/12 18:39, Andrew Feren wrote: >> Hi Brian, >>=20 >> I can't say found the original confusing, but if some people did I = don't have a problem with the new form. Or even more simply. >>=20 >> Address types -- ipv4Address, macAddress, and ipv6Address -- MUST = be >> encoded as four, six, and sixteen octets in network byte order, = respectively. >>=20 >> -Andrew >>=20 >> On 10/22/2012 07:08 AM, Brian Trammell wrote: >>> Greetings, all, >>>=20 >>> I've received a comment off-list from an implementor that section = 6.1.2 of 5101 (and 5101-bis) is potentially confusingly worded. The = intent here is that all addresses be _encoded_ in big-endian order, = regardless of how they are handled internally (e.g., Unix programs often = use both host and network byte order for IPv4 addresses: the former for = manipulation, the latter for compatibility with the sockets API). = "6-octet integers" and "16-octet integers", however, are kind of weird = concepts. So I'd suggest the following change: >>>=20 >>> OLD: >>>=20 >>> 6.1.2. Address Types >>>=20 >>>=20 >>> Address types -- macAddress, ipv4Address, and ipv6Address -- MUST = be >>> encoded the same way as the integral data types. The macAddress = is >>> treated as a 6-octet integer, the ipv4Address as a 4-octet = integer, >>> and the ipv6Address as a 16-octet integer. >>>=20 >>> NEW: >>>=20 >>> Address types -- ipv4Address, macAddress, and ipv6Address -- MUST = be >>> encoded the same way as unsigned integral data types, as four, = six, >>> and sixteen octets in network byte order, respectively. >>>=20 >>>=20 >>> Thoughts? >>>=20 >>> Thanks, >>>=20 >>> Brian >>>=20 >>> _______________________________________________ >>> IPFIX mailing list >>> IPFIX@ietf.org >>> https://www.ietf.org/mailman/listinfo/ipfix >>=20 >> _______________________________________________ >> IPFIX mailing list >> IPFIX@ietf.org >> https://www.ietf.org/mailman/listinfo/ipfix From andrjohn@cisco.com Tue Oct 23 01:42:45 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id BC10721F86C7 for ; Tue, 23 Oct 2012 01:42:45 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -9.999 X-Spam-Level: X-Spam-Status: No, score=-9.999 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, J_CHICKENPOX_64=0.6, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8r45rxgE0Ltm for ; Tue, 23 Oct 2012 01:42:44 -0700 (PDT) Received: from ams-iport-2.cisco.com (ams-iport-2.cisco.com [144.254.224.141]) by ietfa.amsl.com (Postfix) with ESMTP id 007ED21F86CA for ; Tue, 23 Oct 2012 01:42:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=4618; q=dns/txt; s=iport; t=1350981764; x=1352191364; h=subject:mime-version:from:in-reply-to:date:cc: content-transfer-encoding:message-id:references:to; bh=YcZAPOANQOZohFCfye2StQwo96LfdvQOBrDBnHQAduA=; b=cg/sDTPKyg/eog36aeX3hkID34ncssJCCYnNB4LpiBtOylH4q8uoJEOC vXesPJrhLBZ5VtFtBOafqloMs1LP8FYTvufiDgw/qnPZBCzn/QIP8lwhE BODuNLe9Ssb6N0j9A25BK8IX31RqdQM61KS+YuAaMSL/4gKgl7jnOybgM g=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Av4EAC1YhlCQ/khN/2dsb2JhbABEwWOBCIIeAQEBBAEBAQ8BJzQLDgILGC4WETAGARIih2ILnE6PXJBHBASLW4V+YAOVcY5OgWuCcA X-IronPort-AV: E=Sophos;i="4.80,634,1344211200"; d="scan'208";a="77677114" Received: from ams-core-4.cisco.com ([144.254.72.77]) by ams-iport-2.cisco.com with ESMTP; 23 Oct 2012 08:42:30 +0000 Received: from cisco.com (mrwint.cisco.com [64.103.70.36]) by ams-core-4.cisco.com (8.14.5/8.14.5) with ESMTP id q9N8gT1p030553 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 23 Oct 2012 08:42:29 GMT Received: from dhcp-10-147-1-70.cisco.com (dhcp-10-147-1-70.cisco.com [10.147.1.70]) by cisco.com (8.14.4+Sun/8.8.8) with ESMTP id q9N8gQSZ027938; Tue, 23 Oct 2012 09:42:28 +0100 (BST) Mime-Version: 1.0 (Apple Message framework v1278) Content-Type: text/plain; charset=us-ascii From: Andrew Johnson In-Reply-To: Date: Tue, 23 Oct 2012 09:42:21 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <8F849333-9503-4177-BB21-70426C600E93@cisco.com> References: To: Brian Trammell , John Court X-Mailer: Apple Mail (2.1278) Cc: IETF IPFIX Working Group Subject: Re: [IPFIX] Export of long lived flow information X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Oct 2012 08:42:45 -0000 Hi folks I've been thinking for a while now that we should be able to send a = mixture of delta and update records for the same flow, but I haven't = really figured out how to make the semantics obvious to the collector. i.e.: Delta (new record), update, update, update(ends the flow) Perhaps we could use the flow end reason to make things more explicit. = For example, all but the last packet have a flowEndReason of 0 and then = the last one updates the reason. Would this sort of thing be useful? Would collectors be able to work = with it? Cheers, Andrew On 23 Oct 2012, at 08:43, Brian Trammell wrote: > Hi, John, >=20 > Your assumptions are basically correct. The IPFIX model of a Metering = Process (MP) makes an implicit assumption that it will be configured or = configurable to export information about long-lived flows every n = minutes through active timeout. =46rom the MP side, this allows periodic = complete flushing of the flow cache.=20 >=20 > More importantly (IMO), on the Collecting Process (CP) side, it = provides a guarantee that every packet observed and selected for export = will be accounted for within Ta + Tde + Tdc, where Ta is the active = timeout, Tde is the MP to EP delay (how long it takes for an exported = flow to make it from the MP cache, through the Exporting Process (EP), = into an IPFIX Message, which is implementation dependent but typically = short) and Tde is the export delay (OWD from EP to CP). This is = important for streaming process applications, as after this time, the CP = and downstream processes can assume that no further information about = the past will become available. >=20 > The tradeoff here is that shorter Ta causes more records to be = exported about long flows, and a longer Ta causes a longer delay, which = some streaming applications can't tolerate. >=20 > In any case, the assumption is that the flow is no longer in the cache = after active timeout, so you don't know when the flow really started. = Since the flowStartTime is the timestamp of the first observed packet = within the record, and flowEndTime is the timestamp of the last observed = packet within the record, the timestamps would then be record-local.... = i.e. a flow less than twice the active timeout (with at least one packet = per idle timeout) would result in two flow records. The first would have = a timestamp range between the first packet of the flow and the last = packet observed before the active timeout; the second between the first = packet observed after the active timeout and the last packet of the = flow. >=20 > This does, admittedly, make it rather difficult to stitch records for = long flows together -- you can't match timestamps, and essentially need = to simulate the flow cache with a longer active timeout. For = applications requiring a single record per flow, active timeouts can be = set to be practically infinite, with the tradeoff that you never know at = the CP when you'll get a flow with a start time far in the past. >=20 > Best regards, >=20 > Brian >=20 >=20 > On Oct 23, 2012, at 3:01 AM, John Court wrote: >=20 >> Hi,=20 >>=20 >> I have been a subscriber to the list for a little over a year, and an = implementer of IPFIX export for at least one product. This WG has done = great work overall !=20 >>=20 >> One area that still has me a little confused even after researching = as many of the RFCs as possible including RFC5472 is how to treat export = of long lived flows.=20 >>=20 >> At the moment I use "DeltaCount" information elements for everything = and at specific intervals export long lived flows with the flowEndReason = of "flowActiveTimeout". This of course results in multiple flow records = for long lived connections over time. Since this situation doesn't seem = to be covered explicitly I was hoping someone on the list would point me = in the right direction or confirm my assumptions. On thing that is = particularly unclear is what to do about flowStart/flowEnd times when = sending this type of record.=20 >>=20 >> Thanks=20 >>=20 >> John Court >> Senior Software Engineer >> IBM Security Systems Division >> IBM Australia Development Laboratory >> Office: +61 7 5552 4014 >> Mobile: +61 430 841328 >>=20 >> _______________________________________________ >> IPFIX mailing list >> IPFIX@ietf.org >> https://www.ietf.org/mailman/listinfo/ipfix >=20 > _______________________________________________ > IPFIX mailing list > IPFIX@ietf.org > https://www.ietf.org/mailman/listinfo/ipfix From paitken@cisco.com Tue Oct 23 05:25:23 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8B26021F86B1 for ; Tue, 23 Oct 2012 05:25:23 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.253 X-Spam-Level: X-Spam-Status: No, score=-10.253 tagged_above=-999 required=5 tests=[AWL=-0.254, BAYES_00=-2.599, J_CHICKENPOX_37=0.6, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id R-rzxO9dpsqV for ; Tue, 23 Oct 2012 05:25:23 -0700 (PDT) Received: from ams-iport-1.cisco.com (ams-iport-1.cisco.com [144.254.224.140]) by ietfa.amsl.com (Postfix) with ESMTP id 3678E21F86AB for ; Tue, 23 Oct 2012 05:25:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=1021; q=dns/txt; s=iport; t=1350995121; x=1352204721; h=message-id:date:from:mime-version:to:subject: content-transfer-encoding; bh=p3DGErssd+UV3dkosBOlD6Oq2Yh/KXgwkL4XMcifQP0=; b=mzUZdKC0j9Me/oaBUGjH9G2lhtyMapkJfidbcCWsSdVsF3q0c5cWHRsu ZgQcnCYcDiX2n8Qd17b9ZyxXOhKl/ND75ED6IE7kA4W0xa6fVE6H9cAnD HSCxiHYbDhSu98p943gHMZheJvXsfSI12obGrg2PYmZQO+93dFGqAwY7/ g=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Av0EAGeLhlCQ/khL/2dsb2JhbABEwWSBCII3ARQRQD0WGAMCAQIBSw0IAQEeh2KbL4Erj1yQQpI9A5VxhWSIaoFrgnA X-IronPort-AV: E=Sophos;i="4.80,634,1344211200"; d="scan'208";a="145681776" Received: from ams-core-2.cisco.com ([144.254.72.75]) by ams-iport-1.cisco.com with ESMTP; 23 Oct 2012 12:25:20 +0000 Received: from [144.254.153.39] (dhcp-144-254-153-39.cisco.com [144.254.153.39]) by ams-core-2.cisco.com (8.14.5/8.14.5) with ESMTP id q9NCPJwx013669 for ; Tue, 23 Oct 2012 12:25:19 GMT Message-ID: <50868CB1.3080107@cisco.com> Date: Tue, 23 Oct 2012 13:25:21 +0100 From: Paul Aitken User-Agent: Mozilla/5.0 (X11; Linux i686; rv:15.0) Gecko/20120912 Thunderbird/15.0.1 MIME-Version: 1.0 To: IETF IPFIX Working Group Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: [IPFIX] "ID" in Information Element names X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Oct 2012 12:25:23 -0000 I've just been looking at how inconsistently "ID" and "Id" are used for IE names. Since the bulk of the existing names are "...Id", can we assume that the following are all good: vlanId postVlanId classificationEngineId commonPropertiesId observationPointId lineCardId portId meteringProcessId exportingProcessId templateId wlanChannelId flowId observationDomainId dot1qVlanId dot1qCustomerVlanId metroEvcId pseudoWireId postDot1qVlanId postDot1qCustomerVlanId exportSctpStreamId connectionTransactionId selectionSequenceId selectorId informationElementId virtualStationInterfaceId layer2SegmentId And that the following should change? : applicationID -> applicationId natPoolID -> natPoolId Finally, I'm not sure about the following: wlanSSID -> "SSID"seems to be an established term. ingressVRFID -> should it be "VRFId" ? egressVRFID -> should it be "VRFId" ? virtualStationUUID ->should it be "UUId" ? Feedback? Thanks, P. From trammell@tik.ee.ethz.ch Tue Oct 23 05:43:16 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D23DF21F86C0 for ; Tue, 23 Oct 2012 05:43:16 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -5.999 X-Spam-Level: X-Spam-Status: No, score=-5.999 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, J_CHICKENPOX_37=0.6, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id x+dkVcTu9mrn for ; Tue, 23 Oct 2012 05:43:16 -0700 (PDT) Received: from smtp.ee.ethz.ch (smtp.ee.ethz.ch [129.132.2.219]) by ietfa.amsl.com (Postfix) with ESMTP id 120E521F86BD for ; Tue, 23 Oct 2012 05:43:15 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by smtp.ee.ethz.ch (Postfix) with ESMTP id 90D22D9494; Tue, 23 Oct 2012 14:43:13 +0200 (MEST) X-Virus-Scanned: by amavisd-new on smtp.ee.ethz.ch Received: from smtp.ee.ethz.ch ([127.0.0.1]) by localhost (.ee.ethz.ch [127.0.0.1]) (amavisd-new, port 10024) with LMTP id SmHUf-LjJaob; Tue, 23 Oct 2012 14:43:13 +0200 (MEST) Received: from public-docking-etx-3-45.ethz.ch (public-docking-etx-3-45.ethz.ch [129.132.128.45]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: briant) by smtp.ee.ethz.ch (Postfix) with ESMTPSA id 58A6DD93A2; Tue, 23 Oct 2012 14:43:13 +0200 (MEST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) From: Brian Trammell In-Reply-To: <50868CB1.3080107@cisco.com> Date: Tue, 23 Oct 2012 14:43:42 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <2EC7CC46-B95E-47E5-8387-DA79F3937B80@tik.ee.ethz.ch> References: <50868CB1.3080107@cisco.com> To: Paul Aitken X-Mailer: Apple Mail (2.1499) Cc: IETF IPFIX Working Group Subject: Re: [IPFIX] "ID" in Information Element names X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Oct 2012 12:43:16 -0000 Hi, Paul, UUID is an established term too, no? So we should keep that as is. Not sure about VRF. I'd rename applicationId and natPoolId. Cheers, Brian On Oct 23, 2012, at 2:25 PM, Paul Aitken wrote: > I've just been looking at how inconsistently "ID" and "Id" are used = for IE names. >=20 > Since the bulk of the existing names are "...Id", can we assume that = the following are all good: >=20 > vlanId > postVlanId > classificationEngineId > commonPropertiesId > observationPointId > lineCardId > portId > meteringProcessId > exportingProcessId > templateId > wlanChannelId > flowId > observationDomainId > dot1qVlanId > dot1qCustomerVlanId > metroEvcId > pseudoWireId > postDot1qVlanId > postDot1qCustomerVlanId > exportSctpStreamId > connectionTransactionId > selectionSequenceId > selectorId > informationElementId > virtualStationInterfaceId > layer2SegmentId >=20 >=20 > And that the following should change? : >=20 > applicationID -> applicationId > natPoolID -> natPoolId >=20 >=20 > Finally, I'm not sure about the following: >=20 > wlanSSID -> "SSID"seems to be an established term. > ingressVRFID -> should it be "VRFId" ? > egressVRFID -> should it be "VRFId" ? > virtualStationUUID ->should it be "UUId" ? >=20 >=20 >=20 > Feedback? >=20 > Thanks, > P. > _______________________________________________ > IPFIX mailing list > IPFIX@ietf.org > https://www.ietf.org/mailman/listinfo/ipfix From andrewf@plixer.com Tue Oct 23 06:19:21 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D306121F849A for ; Tue, 23 Oct 2012 06:19:21 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.149 X-Spam-Level: X-Spam-Status: No, score=-2.149 tagged_above=-999 required=5 tests=[AWL=-0.150, BAYES_00=-2.599, J_CHICKENPOX_64=0.6] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id BiasCmOrvJ9n for ; Tue, 23 Oct 2012 06:19:21 -0700 (PDT) Received: from smtp.plixer.com (smtp.plixer.com [66.186.184.193]) by ietfa.amsl.com (Postfix) with ESMTP id EC71621F8464 for ; Tue, 23 Oct 2012 06:19:20 -0700 (PDT) Received: from [10.100.1.132] ([10.100.1.132]) by smtp.plixer.com over TLS secured channel with Microsoft SMTPSVC(6.0.3790.4675); Tue, 23 Oct 2012 09:19:20 -0400 Message-ID: <50869957.1020906@plixer.com> Date: Tue, 23 Oct 2012 09:19:19 -0400 From: Andrew Feren User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:19.0) Gecko/19.0 Thunderbird/19.0a1 MIME-Version: 1.0 To: ipfix@ietf.org References: <8F849333-9503-4177-BB21-70426C600E93@cisco.com> In-Reply-To: <8F849333-9503-4177-BB21-70426C600E93@cisco.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 23 Oct 2012 13:19:20.0035 (UTC) FILETIME=[02AA1330:01CDB121] Subject: Re: [IPFIX] Export of long lived flow information X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Oct 2012 13:19:21 -0000 Hi Andrew, I'm not sure I understand your question. How is a delta different than an update? Specific to flowEndReason rather than sending 0 each update should include the reason (e.g. 0x02 active timeout). Do you have some situation in mind where a flow is being exported for an unknown reason? Or is the reason known, but it doesn't fit one of the 5 currently defined? -Andrew On 10/23/2012 04:42 AM, Andrew Johnson wrote: > Hi folks > > I've been thinking for a while now that we should be able to send a mixture of delta and update records for the same flow, but I haven't really figured out how to make the semantics obvious to the collector. > > i.e.: Delta (new record), update, update, update(ends the flow) > > > Perhaps we could use the flow end reason to make things more explicit. For example, all but the last packet have a flowEndReason of 0 and then the last one updates the reason. > > Would this sort of thing be useful? Would collectors be able to work with it? > > > Cheers, Andrew > > > > On 23 Oct 2012, at 08:43, Brian Trammell wrote: >> Hi, John, >> >> Your assumptions are basically correct. The IPFIX model of a Metering Process (MP) makes an implicit assumption that it will be configured or configurable to export information about long-lived flows every n minutes through active timeout. From the MP side, this allows periodic complete flushing of the flow cache. >> >> More importantly (IMO), on the Collecting Process (CP) side, it provides a guarantee that every packet observed and selected for export will be accounted for within Ta + Tde + Tdc, where Ta is the active timeout, Tde is the MP to EP delay (how long it takes for an exported flow to make it from the MP cache, through the Exporting Process (EP), into an IPFIX Message, which is implementation dependent but typically short) and Tde is the export delay (OWD from EP to CP). This is important for streaming process applications, as after this time, the CP and downstream processes can assume that no further information about the past will become available. >> >> The tradeoff here is that shorter Ta causes more records to be exported about long flows, and a longer Ta causes a longer delay, which some streaming applications can't tolerate. >> >> In any case, the assumption is that the flow is no longer in the cache after active timeout, so you don't know when the flow really started. Since the flowStartTime is the timestamp of the first observed packet within the record, and flowEndTime is the timestamp of the last observed packet within the record, the timestamps would then be record-local.... i.e. a flow less than twice the active timeout (with at least one packet per idle timeout) would result in two flow records. The first would have a timestamp range between the first packet of the flow and the last packet observed before the active timeout; the second between the first packet observed after the active timeout and the last packet of the flow. >> >> This does, admittedly, make it rather difficult to stitch records for long flows together -- you can't match timestamps, and essentially need to simulate the flow cache with a longer active timeout. For applications requiring a single record per flow, active timeouts can be set to be practically infinite, with the tradeoff that you never know at the CP when you'll get a flow with a start time far in the past. >> >> Best regards, >> >> Brian >> >> >> On Oct 23, 2012, at 3:01 AM, John Court wrote: >> >>> Hi, >>> >>> I have been a subscriber to the list for a little over a year, and an implementer of IPFIX export for at least one product. This WG has done great work overall ! >>> >>> One area that still has me a little confused even after researching as many of the RFCs as possible including RFC5472 is how to treat export of long lived flows. >>> >>> At the moment I use "DeltaCount" information elements for everything and at specific intervals export long lived flows with the flowEndReason of "flowActiveTimeout". This of course results in multiple flow records for long lived connections over time. Since this situation doesn't seem to be covered explicitly I was hoping someone on the list would point me in the right direction or confirm my assumptions. On thing that is particularly unclear is what to do about flowStart/flowEnd times when sending this type of record. >>> >>> Thanks >>> >>> John Court >>> Senior Software Engineer >>> IBM Security Systems Division >>> IBM Australia Development Laboratory >>> Office: +61 7 5552 4014 >>> Mobile: +61 430 841328 >>> >>> _______________________________________________ >>> IPFIX mailing list >>> IPFIX@ietf.org >>> https://www.ietf.org/mailman/listinfo/ipfix >> _______________________________________________ >> IPFIX mailing list >> IPFIX@ietf.org >> https://www.ietf.org/mailman/listinfo/ipfix > _______________________________________________ > IPFIX mailing list > IPFIX@ietf.org > https://www.ietf.org/mailman/listinfo/ipfix From andrewf@plixer.com Tue Oct 23 06:30:06 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4422221F8704 for ; Tue, 23 Oct 2012 06:30:06 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.119 X-Spam-Level: X-Spam-Status: No, score=-2.119 tagged_above=-999 required=5 tests=[AWL=-0.120, BAYES_00=-2.599, J_CHICKENPOX_37=0.6] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GCmHDIC3w46y for ; Tue, 23 Oct 2012 06:30:05 -0700 (PDT) Received: from smtp.plixer.com (smtp.plixer.com [66.186.184.193]) by ietfa.amsl.com (Postfix) with ESMTP id A616B21F8706 for ; Tue, 23 Oct 2012 06:30:05 -0700 (PDT) Received: from [10.100.1.132] ([10.100.1.132]) by smtp.plixer.com over TLS secured channel with Microsoft SMTPSVC(6.0.3790.4675); Tue, 23 Oct 2012 09:30:05 -0400 Message-ID: <50869BDC.3070708@plixer.com> Date: Tue, 23 Oct 2012 09:30:04 -0400 From: Andrew Feren User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:19.0) Gecko/19.0 Thunderbird/19.0a1 MIME-Version: 1.0 To: ipfix@ietf.org References: <50868CB1.3080107@cisco.com> <2EC7CC46-B95E-47E5-8387-DA79F3937B80@tik.ee.ethz.ch> In-Reply-To: <2EC7CC46-B95E-47E5-8387-DA79F3937B80@tik.ee.ethz.ch> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 23 Oct 2012 13:30:05.0005 (UTC) FILETIME=[8318AFD0:01CDB122] Subject: Re: [IPFIX] "ID" in Information Element names X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Oct 2012 13:30:06 -0000 Hi Paul and Brian, I agree on renaming applicationId and natPoolId and keeping UUID. My personal bias is to also keep VRFID, but I can't claim any evidence that is an established term. I just prefer to keep it upper case, but I won't object either way. -Andrew On 10/23/2012 08:43 AM, Brian Trammell wrote: > Hi, Paul, > > UUID is an established term too, no? So we should keep that as is. > > Not sure about VRF. > > I'd rename applicationId and natPoolId. > > Cheers, > > Brian > > On Oct 23, 2012, at 2:25 PM, Paul Aitken wrote: > >> I've just been looking at how inconsistently "ID" and "Id" are used for IE names. >> >> Since the bulk of the existing names are "...Id", can we assume that the following are all good: >> >> vlanId >> postVlanId >> classificationEngineId >> commonPropertiesId >> observationPointId >> lineCardId >> portId >> meteringProcessId >> exportingProcessId >> templateId >> wlanChannelId >> flowId >> observationDomainId >> dot1qVlanId >> dot1qCustomerVlanId >> metroEvcId >> pseudoWireId >> postDot1qVlanId >> postDot1qCustomerVlanId >> exportSctpStreamId >> connectionTransactionId >> selectionSequenceId >> selectorId >> informationElementId >> virtualStationInterfaceId >> layer2SegmentId >> >> >> And that the following should change? : >> >> applicationID -> applicationId >> natPoolID -> natPoolId >> >> >> Finally, I'm not sure about the following: >> >> wlanSSID -> "SSID"seems to be an established term. >> ingressVRFID -> should it be "VRFId" ? >> egressVRFID -> should it be "VRFId" ? >> virtualStationUUID ->should it be "UUId" ? >> >> >> >> Feedback? >> >> Thanks, >> P. >> _______________________________________________ >> IPFIX mailing list >> IPFIX@ietf.org >> https://www.ietf.org/mailman/listinfo/ipfix > _______________________________________________ > IPFIX mailing list > IPFIX@ietf.org > https://www.ietf.org/mailman/listinfo/ipfix From paitken@cisco.com Tue Oct 23 06:51:03 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3F9C021F86A8 for ; Tue, 23 Oct 2012 06:51:03 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.235 X-Spam-Level: X-Spam-Status: No, score=-10.235 tagged_above=-999 required=5 tests=[AWL=-0.236, BAYES_00=-2.599, J_CHICKENPOX_37=0.6, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id SHXQQDzYKsAs for ; Tue, 23 Oct 2012 06:51:02 -0700 (PDT) Received: from ams-iport-3.cisco.com (ams-iport-3.cisco.com [144.254.224.146]) by ietfa.amsl.com (Postfix) with ESMTP id 454F421F8701 for ; Tue, 23 Oct 2012 06:51:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=2536; q=dns/txt; s=iport; t=1351000262; x=1352209862; h=message-id:date:from:mime-version:to:subject:references: in-reply-to:content-transfer-encoding; bh=g7YYiOCeGDduHzpuOzstjzswAv1td5ZDk0Gp4qWoL+8=; b=T4UcnqxvtVwhaNrRc4Qk/LSQAo3RzQcgx9F0yxqN8ChBfGires+1xJ0f gChlS5VrdbGba1kFmEQaSkIJ6EzUNUl1FHS3Y+OZs29KbMsyR1B699nn0 hubi1WuwnxldO2aAPg4LcXo49yV+wbKVHBZ0SZyKCBKHWTriFfKAmFeLk U=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Av4EACeghlCQ/khM/2dsb2JhbABEwWWBCIIeAQEBBAEBAQ8BFBE2GwsYCSUPAhYwEwYCAQEeh2ILnA2PXJA4BItfgzuDIwOVcYVkiGqBa4Jw X-IronPort-AV: E=Sophos;i="4.80,635,1344211200"; d="scan'208";a="9024200" Received: from ams-core-3.cisco.com ([144.254.72.76]) by ams-iport-3.cisco.com with ESMTP; 23 Oct 2012 13:50:59 +0000 Received: from [144.254.153.39] (dhcp-144-254-153-39.cisco.com [144.254.153.39]) by ams-core-3.cisco.com (8.14.5/8.14.5) with ESMTP id q9NDoxuv003163 for ; Tue, 23 Oct 2012 13:50:59 GMT Message-ID: <5086A0C5.10202@cisco.com> Date: Tue, 23 Oct 2012 14:51:01 +0100 From: Paul Aitken User-Agent: Mozilla/5.0 (X11; Linux i686; rv:15.0) Gecko/20120912 Thunderbird/15.0.1 MIME-Version: 1.0 To: ipfix@ietf.org References: <50868CB1.3080107@cisco.com> <2EC7CC46-B95E-47E5-8387-DA79F3937B80@tik.ee.ethz.ch> <50869BDC.3070708@plixer.com> In-Reply-To: <50869BDC.3070708@plixer.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [IPFIX] "ID" in Information Element names X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Oct 2012 13:51:03 -0000 Thanks Brian and Andrew. I've asked IANA to rename "applicationId" and "natPoolId" for consistency with the other fields. I'll leave wlanSSID, ingressVRFID, egressVRFID, and virtualStationUUID unchanged. P. On 23/10/12 14:30, Andrew Feren wrote: > Hi Paul and Brian, > > I agree on renaming applicationId and natPoolId and keeping UUID. > > My personal bias is to also keep VRFID, but I can't claim any evidence > that is an established term. I just prefer to keep it upper case, but > I won't object either way. > > -Andrew > > > On 10/23/2012 08:43 AM, Brian Trammell wrote: >> Hi, Paul, >> >> UUID is an established term too, no? So we should keep that as is. >> >> Not sure about VRF. >> >> I'd rename applicationId and natPoolId. >> >> Cheers, >> >> Brian >> >> On Oct 23, 2012, at 2:25 PM, Paul Aitken wrote: >> >>> I've just been looking at how inconsistently "ID" and "Id" are used >>> for IE names. >>> >>> Since the bulk of the existing names are "...Id", can we assume that >>> the following are all good: >>> >>> vlanId >>> postVlanId >>> classificationEngineId >>> commonPropertiesId >>> observationPointId >>> lineCardId >>> portId >>> meteringProcessId >>> exportingProcessId >>> templateId >>> wlanChannelId >>> flowId >>> observationDomainId >>> dot1qVlanId >>> dot1qCustomerVlanId >>> metroEvcId >>> pseudoWireId >>> postDot1qVlanId >>> postDot1qCustomerVlanId >>> exportSctpStreamId >>> connectionTransactionId >>> selectionSequenceId >>> selectorId >>> informationElementId >>> virtualStationInterfaceId >>> layer2SegmentId >>> >>> >>> And that the following should change? : >>> >>> applicationID -> applicationId >>> natPoolID -> natPoolId >>> >>> >>> Finally, I'm not sure about the following: >>> >>> wlanSSID -> "SSID"seems to be an established term. >>> ingressVRFID -> should it be "VRFId" ? >>> egressVRFID -> should it be "VRFId" ? >>> virtualStationUUID ->should it be "UUId" ? >>> >>> >>> >>> Feedback? >>> >>> Thanks, >>> P. >>> _______________________________________________ >>> IPFIX mailing list >>> IPFIX@ietf.org >>> https://www.ietf.org/mailman/listinfo/ipfix >> _______________________________________________ >> IPFIX mailing list >> IPFIX@ietf.org >> https://www.ietf.org/mailman/listinfo/ipfix > > _______________________________________________ > IPFIX mailing list > IPFIX@ietf.org > https://www.ietf.org/mailman/listinfo/ipfix From andrjohn@cisco.com Tue Oct 23 07:16:44 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0A84721F8665 for ; Tue, 23 Oct 2012 07:16:44 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -9.999 X-Spam-Level: X-Spam-Status: No, score=-9.999 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, J_CHICKENPOX_64=0.6, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id afw+V+d1lelu for ; Tue, 23 Oct 2012 07:16:43 -0700 (PDT) Received: from ams-iport-2.cisco.com (ams-iport-2.cisco.com [144.254.224.141]) by ietfa.amsl.com (Postfix) with ESMTP id 8474221F8661 for ; Tue, 23 Oct 2012 07:16:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=6363; q=dns/txt; s=iport; t=1351001802; x=1352211402; h=subject:mime-version:from:in-reply-to:date:cc: content-transfer-encoding:message-id:references:to; bh=zHMV1BBRPNrrdu7b2gOnWvBSWbtIjkB6Ny/SBkajioY=; b=Bwyx7v4oFa3YWFh9oFklpapJRxIXYtN5nHFJCQnWYbewC1r0MotHTsfp o1IFUU9B7R4jTE0DABTjeqWxKGE8vNc2lCtUxjTqz9ifwpth6jjoX7k1H qEidQ+3D2VPdVVfcp+20DhCyU+Wy+TdyRXNWbWv+5tu8s4qs6Ayf+AtVP 8=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Av4EAFmmhlCQ/khN/2dsb2JhbABEwWWBCIIeAQEBAwEBAQEPASc0CwUJAgsYLhYRMAYTIodcBgucBo9ckDYEBItbhX5gA5Vxjk6Ba4JwgVkH X-IronPort-AV: E=Sophos;i="4.80,635,1344211200"; d="scan'208";a="77687670" Received: from ams-core-4.cisco.com ([144.254.72.77]) by ams-iport-2.cisco.com with ESMTP; 23 Oct 2012 14:16:39 +0000 Received: from cisco.com (mrwint.cisco.com [64.103.70.36]) by ams-core-4.cisco.com (8.14.5/8.14.5) with ESMTP id q9NEGdGm002917 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 23 Oct 2012 14:16:39 GMT Received: from dhcp-10-147-1-70.cisco.com (dhcp-10-147-1-70.cisco.com [10.147.1.70]) by cisco.com (8.14.4+Sun/8.8.8) with ESMTP id q9NEGb0Q019715; Tue, 23 Oct 2012 15:16:38 +0100 (BST) Mime-Version: 1.0 (Apple Message framework v1278) Content-Type: text/plain; charset=us-ascii From: Andrew Johnson In-Reply-To: <50869957.1020906@plixer.com> Date: Tue, 23 Oct 2012 15:16:32 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <8F849333-9503-4177-BB21-70426C600E93@cisco.com> <50869957.1020906@plixer.com> To: Andrew Feren X-Mailer: Apple Mail (2.1278) Cc: ipfix@ietf.org Subject: Re: [IPFIX] Export of long lived flow information X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Oct 2012 14:16:44 -0000 Hi Andrew When using a permanent cache, the flows can be sent periodically using = update (as opposed to delta) counters. The idea is that the Collector = can be kept up to date without actually ageing out the flow, so there is = no flow end reason. Updates send the latest state of the flow, which is = preferable to deltas because the Exporting Device doesn't need to store = the state of what was last sent (or reset it's internal counters) and if = an update gets lost (e.g. with UDP) then the next update will ensure the = Collector gets the full counts. I am wondering if there is a use case for periodically sending the state = of the flow (using update counters) for long lived flows that will be = aged out at some point in the future. Cheers, Andrew On 23 Oct 2012, at 14:19, Andrew Feren wrote: > Hi Andrew, >=20 > I'm not sure I understand your question. >=20 > How is a delta different than an update? >=20 > Specific to flowEndReason rather than sending 0 each update should = include the reason (e.g. 0x02 active timeout). Do you have some = situation in mind where a flow is being exported for an unknown reason? = Or is the reason known, but it doesn't fit one of the 5 currently = defined? >=20 > -Andrew >=20 > On 10/23/2012 04:42 AM, Andrew Johnson wrote: >> Hi folks >>=20 >> I've been thinking for a while now that we should be able to send a = mixture of delta and update records for the same flow, but I haven't = really figured out how to make the semantics obvious to the collector. >>=20 >> i.e.: Delta (new record), update, update, update(ends the flow) >>=20 >>=20 >> Perhaps we could use the flow end reason to make things more = explicit. For example, all but the last packet have a flowEndReason of = 0 and then the last one updates the reason. >>=20 >> Would this sort of thing be useful? Would collectors be able to work = with it? >>=20 >>=20 >> Cheers, Andrew >>=20 >>=20 >>=20 >> On 23 Oct 2012, at 08:43, Brian Trammell wrote: >>> Hi, John, >>>=20 >>> Your assumptions are basically correct. The IPFIX model of a = Metering Process (MP) makes an implicit assumption that it will be = configured or configurable to export information about long-lived flows = every n minutes through active timeout. =46rom the MP side, this allows = periodic complete flushing of the flow cache. >>>=20 >>> More importantly (IMO), on the Collecting Process (CP) side, it = provides a guarantee that every packet observed and selected for export = will be accounted for within Ta + Tde + Tdc, where Ta is the active = timeout, Tde is the MP to EP delay (how long it takes for an exported = flow to make it from the MP cache, through the Exporting Process (EP), = into an IPFIX Message, which is implementation dependent but typically = short) and Tde is the export delay (OWD from EP to CP). This is = important for streaming process applications, as after this time, the CP = and downstream processes can assume that no further information about = the past will become available. >>>=20 >>> The tradeoff here is that shorter Ta causes more records to be = exported about long flows, and a longer Ta causes a longer delay, which = some streaming applications can't tolerate. >>>=20 >>> In any case, the assumption is that the flow is no longer in the = cache after active timeout, so you don't know when the flow really = started. Since the flowStartTime is the timestamp of the first observed = packet within the record, and flowEndTime is the timestamp of the last = observed packet within the record, the timestamps would then be = record-local.... i.e. a flow less than twice the active timeout (with at = least one packet per idle timeout) would result in two flow records. The = first would have a timestamp range between the first packet of the flow = and the last packet observed before the active timeout; the second = between the first packet observed after the active timeout and the last = packet of the flow. >>>=20 >>> This does, admittedly, make it rather difficult to stitch records = for long flows together -- you can't match timestamps, and essentially = need to simulate the flow cache with a longer active timeout. For = applications requiring a single record per flow, active timeouts can be = set to be practically infinite, with the tradeoff that you never know at = the CP when you'll get a flow with a start time far in the past. >>>=20 >>> Best regards, >>>=20 >>> Brian >>>=20 >>>=20 >>> On Oct 23, 2012, at 3:01 AM, John Court wrote: >>>=20 >>>> Hi, >>>>=20 >>>> I have been a subscriber to the list for a little over a year, and = an implementer of IPFIX export for at least one product. This WG has = done great work overall ! >>>>=20 >>>> One area that still has me a little confused even after researching = as many of the RFCs as possible including RFC5472 is how to treat export = of long lived flows. >>>>=20 >>>> At the moment I use "DeltaCount" information elements for = everything and at specific intervals export long lived flows with the = flowEndReason of "flowActiveTimeout". This of course results in = multiple flow records for long lived connections over time. Since this = situation doesn't seem to be covered explicitly I was hoping someone on = the list would point me in the right direction or confirm my = assumptions. On thing that is particularly unclear is what to do about = flowStart/flowEnd times when sending this type of record. >>>>=20 >>>> Thanks >>>>=20 >>>> John Court >>>> Senior Software Engineer >>>> IBM Security Systems Division >>>> IBM Australia Development Laboratory >>>> Office: +61 7 5552 4014 >>>> Mobile: +61 430 841328 >>>>=20 >>>> _______________________________________________ >>>> IPFIX mailing list >>>> IPFIX@ietf.org >>>> https://www.ietf.org/mailman/listinfo/ipfix >>> _______________________________________________ >>> IPFIX mailing list >>> IPFIX@ietf.org >>> https://www.ietf.org/mailman/listinfo/ipfix >> _______________________________________________ >> IPFIX mailing list >> IPFIX@ietf.org >> https://www.ietf.org/mailman/listinfo/ipfix >=20 > _______________________________________________ > IPFIX mailing list > IPFIX@ietf.org > https://www.ietf.org/mailman/listinfo/ipfix From tasaxena@cisco.com Tue Oct 23 14:35:08 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 468EA11E80DC for ; Tue, 23 Oct 2012 14:35:08 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -9.999 X-Spam-Level: X-Spam-Status: No, score=-9.999 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, J_CHICKENPOX_64=0.6, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id g-TX+0TU0TkH for ; Tue, 23 Oct 2012 14:35:07 -0700 (PDT) Received: from rcdn-iport-9.cisco.com (rcdn-iport-9.cisco.com [173.37.86.80]) by ietfa.amsl.com (Postfix) with ESMTP id 2A1DC11E80D9 for ; Tue, 23 Oct 2012 14:35:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=7236; q=dns/txt; s=iport; t=1351028108; x=1352237708; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=G+DzB7IHjnLiTSrH9eQgPZu3b34WjSi7ocDpn357JI4=; b=jcLpr5aaAiVXzc0Djt/j7m03kqUM9ZAyCjdDb4fza3xb/13Sk0wtkX7s wO40g7ndeAzMDrv1fcrNuGLlQw4raLJafXI2KleV8Oh8UgJTdtEEvk3zs bL7WHloT9SLDWKl9SLUYU6IvR76LJn2GjAkc/54vxhN0ux955JbPM0pXO c=; X-IronPort-AV: E=Sophos;i="4.80,637,1344211200"; d="scan'208";a="131655564" Received: from rcdn-core-2.cisco.com ([173.37.93.153]) by rcdn-iport-9.cisco.com with ESMTP; 23 Oct 2012 21:35:07 +0000 Received: from xhc-rcd-x07.cisco.com (xhc-rcd-x07.cisco.com [173.37.183.81]) by rcdn-core-2.cisco.com (8.14.5/8.14.5) with ESMTP id q9NLZ6cc031053 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Tue, 23 Oct 2012 21:35:06 GMT Received: from xmb-aln-x06.cisco.com ([169.254.1.220]) by xhc-rcd-x07.cisco.com ([173.37.183.81]) with mapi id 14.02.0318.001; Tue, 23 Oct 2012 16:35:06 -0500 From: "Tarun Saxena (tasaxena)" To: "Andrew Johnson (andrjohn)" , Andrew Feren Thread-Topic: [IPFIX] Export of long lived flow information Thread-Index: AQHNsLob1qOOf9AYSkquP+QE7kHVd5fG1ngAgAAQh4CAAE1igIAAD/0AgAAlQjA= Date: Tue, 23 Oct 2012 21:35:06 +0000 Message-ID: References: <8F849333-9503-4177-BB21-70426C600E93@cisco.com> <50869957.1020906@plixer.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.65.72.212] x-tm-as-product-ver: SMEX-10.2.0.1135-7.000.1014-19298.000 x-tm-as-result: No--70.895800-8.000000-31 x-tm-as-user-approved-sender: No x-tm-as-user-blocked-sender: No Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "ipfix@ietf.org" Subject: Re: [IPFIX] Export of long lived flow information X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Oct 2012 21:35:08 -0000 Hello Andrew, Are you proposing a new value for flowEndReason, say 0x6 which denotes that= the flow has not ended (hence not flushed from the cache). The packet/byte= counters are relative to the beginning of the flow and not delta from the = last flow export. What will be the value of flowStartTime and flowEndTime in this case? How will these fields look like when a flow has to be exported due to wrapp= ing around of counters? Forced end? Thanks Tarun -----Original Message----- From: ipfix-bounces@ietf.org [mailto:ipfix-bounces@ietf.org] On Behalf Of A= ndrew Johnson (andrjohn) Sent: Tuesday, October 23, 2012 7:47 PM To: Andrew Feren Cc: ipfix@ietf.org Subject: Re: [IPFIX] Export of long lived flow information Hi Andrew When using a permanent cache, the flows can be sent periodically using upda= te (as opposed to delta) counters. The idea is that the Collector can be k= ept up to date without actually ageing out the flow, so there is no flow en= d reason. Updates send the latest state of the flow, which is preferable t= o deltas because the Exporting Device doesn't need to store the state of wh= at was last sent (or reset it's internal counters) and if an update gets lo= st (e.g. with UDP) then the next update will ensure the Collector gets the = full counts. I am wondering if there is a use case for periodically sending the state of= the flow (using update counters) for long lived flows that will be aged ou= t at some point in the future. Cheers, Andrew On 23 Oct 2012, at 14:19, Andrew Feren wrote: > Hi Andrew, >=20 > I'm not sure I understand your question. >=20 > How is a delta different than an update? >=20 > Specific to flowEndReason rather than sending 0 each update should includ= e the reason (e.g. 0x02 active timeout). Do you have some situation in min= d where a flow is being exported for an unknown reason? Or is the reason k= nown, but it doesn't fit one of the 5 currently defined? >=20 > -Andrew >=20 > On 10/23/2012 04:42 AM, Andrew Johnson wrote: >> Hi folks >>=20 >> I've been thinking for a while now that we should be able to send a mixt= ure of delta and update records for the same flow, but I haven't really fig= ured out how to make the semantics obvious to the collector. >>=20 >> i.e.: Delta (new record), update, update, update(ends the flow) >>=20 >>=20 >> Perhaps we could use the flow end reason to make things more explicit. = For example, all but the last packet have a flowEndReason of 0 and then the= last one updates the reason. >>=20 >> Would this sort of thing be useful? Would collectors be able to work wi= th it? >>=20 >>=20 >> Cheers, Andrew >>=20 >>=20 >>=20 >> On 23 Oct 2012, at 08:43, Brian Trammell wrote: >>> Hi, John, >>>=20 >>> Your assumptions are basically correct. The IPFIX model of a Metering P= rocess (MP) makes an implicit assumption that it will be configured or conf= igurable to export information about long-lived flows every n minutes throu= gh active timeout. From the MP side, this allows periodic complete flushing= of the flow cache. >>>=20 >>> More importantly (IMO), on the Collecting Process (CP) side, it provide= s a guarantee that every packet observed and selected for export will be ac= counted for within Ta + Tde + Tdc, where Ta is the active timeout, Tde is t= he MP to EP delay (how long it takes for an exported flow to make it from t= he MP cache, through the Exporting Process (EP), into an IPFIX Message, whi= ch is implementation dependent but typically short) and Tde is the export d= elay (OWD from EP to CP). This is important for streaming process applicati= ons, as after this time, the CP and downstream processes can assume that no= further information about the past will become available. >>>=20 >>> The tradeoff here is that shorter Ta causes more records to be exported= about long flows, and a longer Ta causes a longer delay, which some stream= ing applications can't tolerate. >>>=20 >>> In any case, the assumption is that the flow is no longer in the cache = after active timeout, so you don't know when the flow really started. Since= the flowStartTime is the timestamp of the first observed packet within the= record, and flowEndTime is the timestamp of the last observed packet withi= n the record, the timestamps would then be record-local.... i.e. a flow les= s than twice the active timeout (with at least one packet per idle timeout)= would result in two flow records. The first would have a timestamp range b= etween the first packet of the flow and the last packet observed before the= active timeout; the second between the first packet observed after the act= ive timeout and the last packet of the flow. >>>=20 >>> This does, admittedly, make it rather difficult to stitch records for l= ong flows together -- you can't match timestamps, and essentially need to s= imulate the flow cache with a longer active timeout. For applications requi= ring a single record per flow, active timeouts can be set to be practically= infinite, with the tradeoff that you never know at the CP when you'll get = a flow with a start time far in the past. >>>=20 >>> Best regards, >>>=20 >>> Brian >>>=20 >>>=20 >>> On Oct 23, 2012, at 3:01 AM, John Court wrote: >>>=20 >>>> Hi, >>>>=20 >>>> I have been a subscriber to the list for a little over a year, and an = implementer of IPFIX export for at least one product. This WG has done gre= at work overall ! >>>>=20 >>>> One area that still has me a little confused even after researching as= many of the RFCs as possible including RFC5472 is how to treat export of l= ong lived flows. >>>>=20 >>>> At the moment I use "DeltaCount" information elements for everything a= nd at specific intervals export long lived flows with the flowEndReason of = "flowActiveTimeout". This of course results in multiple flow records for l= ong lived connections over time. Since this situation doesn't seem to be c= overed explicitly I was hoping someone on the list would point me in the ri= ght direction or confirm my assumptions. On thing that is particularly unc= lear is what to do about flowStart/flowEnd times when sending this type of = record. >>>>=20 >>>> Thanks >>>>=20 >>>> John Court >>>> Senior Software Engineer >>>> IBM Security Systems Division >>>> IBM Australia Development Laboratory >>>> Office: +61 7 5552 4014 >>>> Mobile: +61 430 841328 >>>>=20 >>>> _______________________________________________ >>>> IPFIX mailing list >>>> IPFIX@ietf.org >>>> https://www.ietf.org/mailman/listinfo/ipfix >>> _______________________________________________ >>> IPFIX mailing list >>> IPFIX@ietf.org >>> https://www.ietf.org/mailman/listinfo/ipfix >> _______________________________________________ >> IPFIX mailing list >> IPFIX@ietf.org >> https://www.ietf.org/mailman/listinfo/ipfix >=20 > _______________________________________________ > IPFIX mailing list > IPFIX@ietf.org > https://www.ietf.org/mailman/listinfo/ipfix _______________________________________________ IPFIX mailing list IPFIX@ietf.org https://www.ietf.org/mailman/listinfo/ipfix From johnwcrt@au1.ibm.com Tue Oct 23 15:00:43 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6BE6011E8115 for ; Tue, 23 Oct 2012 15:00:43 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.598 X-Spam-Level: X-Spam-Status: No, score=-6.598 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id oAlaTFXI1-4l for ; Tue, 23 Oct 2012 15:00:42 -0700 (PDT) Received: from e23smtp07.au.ibm.com (e23smtp07.au.ibm.com [202.81.31.140]) by ietfa.amsl.com (Postfix) with ESMTP id 15C5E11E8114 for ; Tue, 23 Oct 2012 15:00:41 -0700 (PDT) Received: from /spool/local by e23smtp07.au.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 24 Oct 2012 07:57:16 +1000 Received: from d23relay04.au.ibm.com (202.81.31.246) by e23smtp07.au.ibm.com (202.81.31.204) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Wed, 24 Oct 2012 07:57:05 +1000 Received: from d23av03.au.ibm.com (d23av03.au.ibm.com [9.190.234.97]) by d23relay04.au.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id q9NLoOXe51642528 for ; Wed, 24 Oct 2012 08:50:25 +1100 Received: from d23av03.au.ibm.com (loopback [127.0.0.1]) by d23av03.au.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id q9NM0LUQ020147 for ; Wed, 24 Oct 2012 09:00:21 +1100 Received: from d23mlc03.au.ibm.com (d23mlc03.au.ibm.com [9.190.26.210]) by d23av03.au.ibm.com (8.14.4/8.13.1/NCO v10.0 AVin) with ESMTP id q9NM0L09020144 for ; Wed, 24 Oct 2012 09:00:21 +1100 To: ipfix@ietf.org MIME-Version: 1.0 X-KeepSent: 96D061AA:F7F6CDD4-CA257AA0:00772818; type=4; name=$KeepSent X-Mailer: Lotus Notes Release 8.5.3 September 15, 2011 Message-ID: From: John Court Date: Wed, 24 Oct 2012 07:59:32 +1000 X-MIMETrack: Serialize by Router on d23mlc03/23/M/IBM(Release 8.5.3FP2HF29 | July 24, 2012) at 24/10/2012 08:59:38, Serialize complete at 24/10/2012 08:59:38 Content-Type: multipart/alternative; boundary="=_alternative 0078DF5E4A257AA0_=" x-cbid: 12102321-0260-0000-0000-00000205EAF7 Subject: Re: [IPFIX] Export of long lived flow information X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Oct 2012 22:00:43 -0000 This is a multipart message in MIME format. --=_alternative 0078DF5E4A257AA0_= Content-Type: text/plain; charset="US-ASCII" Thanks for the interest in resolving this, Andrew, I understand your persistent cache argument. The reason I don't personally use "Total" and DO use "Delta" is more around what the collector does should it not see some of the updates and suddenly gets one that shows large counter values. This could mistakenly result in showing huge traffic over a short period incorrectly. This is particularly true if you do as Brian suggested and are setting the flowStartTime based only on the current record view. Brian, thanks for your detailed explanation. Everything with the exception of flowStartTime was as I am currently doing. I had perhaps mistakenly taken the approach that keeping the flowStartTime as the "conceptual" start rather than for this reporting period would make it easier for the collector to understand what was happening. Not actually sure how I will co-ordinate reseting that time on export yet :-) I would suggest that this sort of situation and Brians explanation be added as an example perhaps in 5101 or even 5472 as it would help with Collector interoperability I am sure :-) Thanks again. John Court Software Engineer IBM Security Systems Division IBM Australia Development Laboratory Office: +61 7 5552 4014 Mobile: +61 430 841328 --=_alternative 0078DF5E4A257AA0_= Content-Type: text/html; charset="US-ASCII" Thanks for the interest in resolving this,

Andrew, I understand your persistent cache argument.  The reason I don't personally use "Total" and DO use "Delta"  is more around what the collector does should it not see some of the updates and suddenly gets one that shows large counter values.  This could mistakenly result in showing huge traffic over a short period incorrectly.  This is particularly true if you do as Brian suggested and are setting the flowStartTime based only on the current record view.

Brian, thanks for your detailed explanation.  Everything with the exception of flowStartTime was as I am currently doing.  I had perhaps mistakenly taken the approach that keeping the flowStartTime as the "conceptual" start rather than for this reporting period would make it easier for the collector to understand what was happening.  Not actually sure how I will co-ordinate reseting that time on export yet :-)

I would suggest that this sort of situation and Brians explanation be added as an example perhaps in 5101 or even 5472 as it would help with Collector interoperability I am sure :-)

Thanks again.


John Court
Software Engineer
IBM Security Systems Division
IBM Australia Development Laboratory
Office:  +61 7 5552 4014
Mobile: +61 430 841328

 --=_alternative 0078DF5E4A257AA0_=-- From johnwcrt@au1.ibm.com Tue Oct 23 15:11:12 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3283D21F847F for ; Tue, 23 Oct 2012 15:11:12 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.598 X-Spam-Level: X-Spam-Status: No, score=-6.598 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FTFmXu88C2G4 for ; Tue, 23 Oct 2012 15:11:11 -0700 (PDT) Received: from e23smtp03.au.ibm.com (e23smtp03.au.ibm.com [202.81.31.145]) by ietfa.amsl.com (Postfix) with ESMTP id E084821F8451 for ; Tue, 23 Oct 2012 15:11:07 -0700 (PDT) Received: from /spool/local by e23smtp03.au.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 24 Oct 2012 08:08:15 +1000 Received: from d23relay03.au.ibm.com (202.81.31.245) by e23smtp03.au.ibm.com (202.81.31.209) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Wed, 24 Oct 2012 08:08:13 +1000 Received: from d23av04.au.ibm.com (d23av04.au.ibm.com [9.190.235.139]) by d23relay03.au.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id q9NMArpf52560090 for ; Wed, 24 Oct 2012 09:10:53 +1100 Received: from d23av04.au.ibm.com (loopback [127.0.0.1]) by d23av04.au.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id q9NMAqVO012341 for ; Wed, 24 Oct 2012 09:10:52 +1100 Received: from d23mlc03.au.ibm.com (d23mlc03.au.ibm.com [9.190.26.210]) by d23av04.au.ibm.com (8.14.4/8.13.1/NCO v10.0 AVin) with ESMTP id q9NMAqss012338 for ; Wed, 24 Oct 2012 09:10:52 +1100 To: ipfix@ietf.org MIME-Version: 1.0 X-KeepSent: 7CC7413C:3DB892A2-CA257AA0:00792F56; type=4; name=$KeepSent X-Mailer: Lotus Notes Release 8.5.3 September 15, 2011 Message-ID: From: John Court Date: Wed, 24 Oct 2012 08:10:03 +1000 X-MIMETrack: Serialize by Router on d23mlc03/23/M/IBM(Release 8.5.3FP2HF29 | July 24, 2012) at 24/10/2012 09:10:09, Serialize complete at 24/10/2012 09:10:09 Content-Type: multipart/alternative; boundary="=_alternative 0079D5E14A257AA0_=" x-cbid: 12102322-6102-0000-0000-0000026CC153 Subject: Re: [IPFIX] Export of long lived flow information X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Oct 2012 22:11:12 -0000 This is a multipart message in MIME format. --=_alternative 0079D5E14A257AA0_= Content-Type: text/plain; charset="US-ASCII" Sorry one last bit is still niggling me. I would think that if a flow was being reported because the Meter or Exporter wanted to reclaim resources, it would use the flowEndReason of "lack of resources 0x05". This gives a clear indication that the flow is no longer being tracked and that any future record for that connection will be with a new flowStartTime. I think that is why I considered it reasonable to maintain the original flowStartTime with the "active timeout 0x02" flowEndReason. Of course reading it now I understand what was in the mind of the authors. If the field had been "flowReportReason" rather than "flowEndReason" perhaps it could have dealt with the concepts updates more easily :-) Thanks again. John Court Software Engineer IBM Security Systems Division IBM Australia Development Laboratory Office: +61 7 5552 4014 Mobile: +61 430 841328 --=_alternative 0079D5E14A257AA0_= Content-Type: text/html; charset="US-ASCII" Sorry one last bit is still niggling me.

I would think that if a flow was being reported because the Meter or Exporter wanted to reclaim resources, it would use the flowEndReason of "lack of resources 0x05".  This gives a clear indication that the flow is no longer being tracked and that any future record for that connection will be with a new flowStartTime.  I think that is why I considered it reasonable to maintain the original flowStartTime with the "active timeout 0x02"  flowEndReason.

Of course reading it now I understand what was in the mind of the authors.  If the field had been "flowReportReason" rather than "flowEndReason" perhaps it could have dealt with the concepts updates more easily :-)

Thanks again.

John Court
Software Engineer
IBM Security Systems Division
IBM Australia Development Laboratory
Office:  +61 7 5552 4014
Mobile: +61 430 841328

 --=_alternative 0079D5E14A257AA0_=-- From trammell@tik.ee.ethz.ch Tue Oct 23 22:19:07 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B0D7421F8BB9 for ; Tue, 23 Oct 2012 22:19:07 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.662 X-Spam-Level: X-Spam-Status: No, score=-6.662 tagged_above=-999 required=5 tests=[AWL=-0.064, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id iN4YIQwobXlD for ; Tue, 23 Oct 2012 22:19:06 -0700 (PDT) Received: from smtp.ee.ethz.ch (smtp.ee.ethz.ch [129.132.2.219]) by ietfa.amsl.com (Postfix) with ESMTP id 6D6D821F8B80 for ; Tue, 23 Oct 2012 22:19:05 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by smtp.ee.ethz.ch (Postfix) with ESMTP id 16FDAD930D; Wed, 24 Oct 2012 07:19:03 +0200 (MEST) X-Virus-Scanned: by amavisd-new on smtp.ee.ethz.ch Received: from smtp.ee.ethz.ch ([127.0.0.1]) by localhost (.ee.ethz.ch [127.0.0.1]) (amavisd-new, port 10024) with LMTP id tIiuuwLDUG-S; Wed, 24 Oct 2012 07:19:02 +0200 (MEST) Received: from [10.0.27.116] (cust-integra-121-161.antanet.ch [80.75.121.161]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: briant) by smtp.ee.ethz.ch (Postfix) with ESMTPSA id B4FB7D930B; Wed, 24 Oct 2012 07:19:02 +0200 (MEST) Content-Type: multipart/alternative; boundary="Apple-Mail=_E83CD7FB-917E-44E6-BB23-1A1B0CE497FB" Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) From: Brian Trammell In-Reply-To: Date: Wed, 24 Oct 2012 07:19:34 +0200 Message-Id: References: To: John Court X-Mailer: Apple Mail (2.1499) Cc: ipfix@ietf.org Subject: Re: [IPFIX] Export of long lived flow information X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Oct 2012 05:19:07 -0000 --Apple-Mail=_E83CD7FB-917E-44E6-BB23-1A1B0CE497FB Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Hi, John, I'll say here that I've never really understood the arguments for export = from persistent caches, and don't recommend the approach unless you have = other overriding application requirements. But that's merely my opinion, = and other people seem to like persistent caches and running total export = (especially router people, who need to keep the cache around for other = reasons anyway), and I guess it's just a sort of point of view thing. My problem with export from persistent caches with updating totals is = that if you want to do anything with the data other than drive per-flow = or per-aggregate displays (a la rrdtool or MRTG), you need to do the = same amount of postprocessing as you would with nonpersistent export = with Ta equal to your desired export interval in order to get "real flow = data", and you lose packet and byte counters per active export interval. = When each subpart of the flow has accurate first/last timestamps, you = get a time series which can give you some idea of the evolution of the = packet and byte rates (is it regular i.e. bulk transfer? is its rate = slowly variable i.e. bulk transfer under congestion control? is it a low = rate highly variable flow? and so on). With persistent cache export you = kind of have the same thing, but (1) you have to subtract to get it and = (2) you have no accurate midpoint timestamp; you can only deduce it from = the export time and your best guess about Tde and Tdc. I'm not sure what you mean about "how to coordinate resetting (start = time) on export"... if you're using non-persistent export, you expire = the flow completely out of the cache on active timeout (with = flowEndReason activeTimeout) and start a new flow record when you see = the first (continuation) packet. If you want to keep ancillary = information about the flow (e.g. deduced AS information if you're not a = packet-forwarding device, other private labeling information that is = expensive to calculate or requires examination of leading payload), you = can keep it around in a secondary cache to be resurrected by flow key = and associated with the new flow record when the continuation packet = comes. As 5101bis is still open, I'll see if I can come up with some suitable = language on the meaning of timestamps therefor (without the = editorializing on persistent export), and propose to the list. Best regards, Brian On Oct 23, 2012, at 11:59 PM, John Court wrote: > Thanks for the interest in resolving this,=20 >=20 > Andrew, I understand your persistent cache argument. The reason I = don't personally use "Total" and DO use "Delta" is more around what the = collector does should it not see some of the updates and suddenly gets = one that shows large counter values. This could mistakenly result in = showing huge traffic over a short period incorrectly. This is = particularly true if you do as Brian suggested and are setting the = flowStartTime based only on the current record view.=20 >=20 > Brian, thanks for your detailed explanation. Everything with the = exception of flowStartTime was as I am currently doing. I had perhaps = mistakenly taken the approach that keeping the flowStartTime as the = "conceptual" start rather than for this reporting period would make it = easier for the collector to understand what was happening. Not actually = sure how I will co-ordinate reseting that time on export yet :-)=20 >=20 > I would suggest that this sort of situation and Brians explanation be = added as an example perhaps in 5101 or even 5472 as it would help with = Collector interoperability I am sure :-)=20 >=20 > Thanks again.=20 >=20 >=20 > John Court > Software Engineer > IBM Security Systems Division > IBM Australia Development Laboratory > Office: +61 7 5552 4014 > Mobile: +61 430 841328 >=20 > _______________________________________________ > IPFIX mailing list > IPFIX@ietf.org > https://www.ietf.org/mailman/listinfo/ipfix --Apple-Mail=_E83CD7FB-917E-44E6-BB23-1A1B0CE497FB Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii Hi, = John,

I'll say here that I've never really understood = the arguments for export from persistent caches, and don't recommend the = approach unless you have other overriding application requirements. But = that's merely my opinion, and other people seem to like persistent = caches and running total export (especially router people, who need to = keep the cache around for other reasons anyway), and I guess it's just a = sort of point of view thing.

My problem with = export from persistent caches with updating totals is that if you want = to do anything with the data other than drive per-flow or per-aggregate = displays (a la rrdtool or MRTG), you need to do the same amount of = postprocessing as you would with nonpersistent export with Ta equal to = your desired export interval in order to get "real flow data", and you = lose packet and byte counters per active export interval. When each = subpart of the flow has accurate first/last timestamps, you get a time = series which can give you some idea of the evolution of the packet and = byte rates (is it regular i.e. bulk transfer? is its rate slowly = variable i.e. bulk transfer under congestion control? is it a low rate = highly variable flow? and so on). With persistent cache export you kind = of have the same thing, but (1) you have to subtract to get it and (2) = you have no accurate midpoint timestamp; you can only deduce it from the = export time and your best guess about Tde and = Tdc.

I'm not sure what you mean about "how to = coordinate resetting (start time) on export"... if you're using = non-persistent export, you expire the flow completely out of the cache = on active timeout (with flowEndReason activeTimeout) and start a new = flow record when you see the first (continuation) packet. If you want to = keep ancillary information about the flow (e.g. deduced AS information = if you're not a packet-forwarding device, other private labeling = information that is expensive to calculate or requires examination of = leading payload), you can keep it around in a secondary cache to be = resurrected by flow key and associated with the new flow record when the = continuation packet comes.

As 5101bis is still = open, I'll see if I can come up with some suitable language on the = meaning of timestamps therefor (without the editorializing on persistent = export), and propose to the list.

Best = regards,

Brian


On Oct 23, 2012, at 11:59 PM, John Court <johnwcrt@au1.ibm.com> = wrote:

Thanks for the = interest in resolving this,

Andrew, I understand your = persistent cache argument.  The reason I don't personally use "Total" and DO use "Delta"  is more around what the collector does should it not see some of the updates and suddenly gets one that shows large counter values.  This could mistakenly result in showing huge traffic over a short period incorrectly.  This is particularly true if you do as Brian suggested and are setting the flowStartTime based = only on the current record view.

Brian, thanks for your detailed = explanation.  Everything with the exception of flowStartTime was as I am = currently doing.  I had perhaps mistakenly taken the approach that keeping = the flowStartTime as the "conceptual" start rather than for this reporting period would make it easier for the collector to understand = what was happening.  Not actually sure how I will co-ordinate reseting that time on export yet :-)

I would suggest that this sort = of situation and Brians explanation be added as an example perhaps in 5101 or even = 5472 as it would help with Collector interoperability I am sure :-)

Thanks again.


John Court
Software Engineer
IBM Security Systems Division
IBM Australia Development Laboratory
Office:  +61 7 5552 4014
Mobile: +61 430 841328

_______________________________________________
IPFIX mailing = list
IPFIX@ietf.org
https://www.ietf.org/= mailman/listinfo/ipfix

= --Apple-Mail=_E83CD7FB-917E-44E6-BB23-1A1B0CE497FB-- From johnwcrt@au1.ibm.com Tue Oct 23 23:02:07 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2F67321F8D44 for ; Tue, 23 Oct 2012 23:02:07 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -8.598 X-Spam-Level: X-Spam-Status: No, score=-8.598 tagged_above=-999 required=5 tests=[AWL=2.000, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id UQ6uJfR8Ku+H for ; Tue, 23 Oct 2012 23:02:05 -0700 (PDT) Received: from e23smtp08.au.ibm.com (e23smtp08.au.ibm.com [202.81.31.141]) by ietfa.amsl.com (Postfix) with ESMTP id BFB4D21F8D43 for ; Tue, 23 Oct 2012 23:02:04 -0700 (PDT) Received: from /spool/local by e23smtp08.au.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 24 Oct 2012 16:01:06 +1000 Received: from d23relay04.au.ibm.com (202.81.31.246) by e23smtp08.au.ibm.com (202.81.31.205) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Wed, 24 Oct 2012 16:01:02 +1000 Received: from d23av01.au.ibm.com (d23av01.au.ibm.com [9.190.234.96]) by d23relay04.au.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id q9O5pqWh52953110 for ; Wed, 24 Oct 2012 16:51:53 +1100 Received: from d23av01.au.ibm.com (loopback [127.0.0.1]) by d23av01.au.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id q9O61oT3027934 for ; Wed, 24 Oct 2012 17:01:50 +1100 Received: from d23mlc03.au.ibm.com (d23mlc03.au.ibm.com [9.190.26.210]) by d23av01.au.ibm.com (8.14.4/8.13.1/NCO v10.0 AVin) with ESMTP id q9O61oHF027931 for ; Wed, 24 Oct 2012 17:01:50 +1100 In-Reply-To: References: To: Brian Trammell MIME-Version: 1.0 X-KeepSent: 30095AE1:689CF5C8-CA257AA1:001FB2C7; type=4; name=$KeepSent X-Mailer: Lotus Notes Release 8.5.3 September 15, 2011 Message-ID: From: John Court Date: Wed, 24 Oct 2012 16:00:58 +1000 X-MIMETrack: Serialize by Router on d23mlc03/23/M/IBM(Release 8.5.3FP2HF29 | July 24, 2012) at 24/10/2012 17:01:07, Serialize complete at 24/10/2012 17:01:07 Content-Type: multipart/alternative; boundary="=_alternative 00211D294A257AA1_=" x-cbid: 12102406-5140-0000-0000-0000023E928F Cc: ipfix@ietf.org Subject: Re: [IPFIX] Export of long lived flow information X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Oct 2012 06:02:07 -0000 This is a multipart message in MIME format. --=_alternative 00211D294A257AA1_= Content-Type: text/plain; charset="US-ASCII" Hi Brian, I suspect I have been mis-interpreting your concept of "persistent caches". From the way you describe it, I would categorise what I export from as a persistent cache. The reason being that for packet processing efficiency reasons, all the "connection" concepts are kept in one place and that includes the accounting (i.e. packets, octets). I only get to see these periodically or on connection close. In IPFIX terminology the meter isn't resetting the flowStartTime or doing actual flow termination outside of detection of "connection" termination. Perhaps it could but that time may be used in other processing, I will have to see. I am not arguing with any of your points. I see the logic behind the operation, I just need to adjust some outputs for long lived connections. Thanks John Court From: Brian Trammell To: John Court/Australia/IBM@IBMAU, Cc: ipfix@ietf.org Date: 24/10/2012 15:19 Subject: Re: [IPFIX] Export of long lived flow information Hi, John, I'll say here that I've never really understood the arguments for export from persistent caches, and don't recommend the approach unless you have other overriding application requirements. But that's merely my opinion, and other people seem to like persistent caches and running total export (especially router people, who need to keep the cache around for other reasons anyway), and I guess it's just a sort of point of view thing. My problem with export from persistent caches with updating totals is that if you want to do anything with the data other than drive per-flow or per-aggregate displays (a la rrdtool or MRTG), you need to do the same amount of postprocessing as you would with nonpersistent export with Ta equal to your desired export interval in order to get "real flow data", and you lose packet and byte counters per active export interval. When each subpart of the flow has accurate first/last timestamps, you get a time series which can give you some idea of the evolution of the packet and byte rates (is it regular i.e. bulk transfer? is its rate slowly variable i.e. bulk transfer under congestion control? is it a low rate highly variable flow? and so on). With persistent cache export you kind of have the same thing, but (1) you have to subtract to get it and (2) you have no accurate midpoint timestamp; you can only deduce it from the export time and your best guess about Tde and Tdc. I'm not sure what you mean about "how to coordinate resetting (start time) on export"... if you're using non-persistent export, you expire the flow completely out of the cache on active timeout (with flowEndReason activeTimeout) and start a new flow record when you see the first (continuation) packet. If you want to keep ancillary information about the flow (e.g. deduced AS information if you're not a packet-forwarding device, other private labeling information that is expensive to calculate or requires examination of leading payload), you can keep it around in a secondary cache to be resurrected by flow key and associated with the new flow record when the continuation packet comes. As 5101bis is still open, I'll see if I can come up with some suitable language on the meaning of timestamps therefor (without the editorializing on persistent export), and propose to the list. Best regards, Brian On Oct 23, 2012, at 11:59 PM, John Court wrote: Thanks for the interest in resolving this, Andrew, I understand your persistent cache argument. The reason I don't personally use "Total" and DO use "Delta" is more around what the collector does should it not see some of the updates and suddenly gets one that shows large counter values. This could mistakenly result in showing huge traffic over a short period incorrectly. This is particularly true if you do as Brian suggested and are setting the flowStartTime based only on the current record view. Brian, thanks for your detailed explanation. Everything with the exception of flowStartTime was as I am currently doing. I had perhaps mistakenly taken the approach that keeping the flowStartTime as the "conceptual" start rather than for this reporting period would make it easier for the collector to understand what was happening. Not actually sure how I will co-ordinate reseting that time on export yet :-) I would suggest that this sort of situation and Brians explanation be added as an example perhaps in 5101 or even 5472 as it would help with Collector interoperability I am sure :-) Thanks again. John Court Software Engineer IBM Security Systems Division IBM Australia Development Laboratory Office: +61 7 5552 4014 Mobile: +61 430 841328 _______________________________________________ IPFIX mailing list IPFIX@ietf.org https://www.ietf.org/mailman/listinfo/ipfix --=_alternative 00211D294A257AA1_= Content-Type: text/html; charset="US-ASCII" Hi Brian,

I suspect I have been mis-interpreting your concept of "persistent caches".  From the way you describe it, I would categorise what I export from as a persistent cache.  The reason being that for packet processing efficiency reasons, all the "connection" concepts are kept in one place and that includes the accounting (i.e. packets, octets).  I only get to see these periodically or on connection close. In IPFIX terminology the meter isn't resetting the flowStartTime or doing actual flow termination outside of detection of "connection" termination.  Perhaps it could but that time may be used in other processing, I will have to see.  

I am not arguing with any of your points.  I see the logic behind the operation, I just need to adjust some outputs for long lived connections.  

Thanks

John Court



From:        Brian Trammell <trammell@tik.ee.ethz.ch>
To:        John Court/Australia/IBM@IBMAU,
Cc:        ipfix@ietf.org
Date:        24/10/2012 15:19
Subject:        Re: [IPFIX] Export of long lived flow information



Hi, John,

I'll say here that I've never really understood the arguments for export from persistent caches, and don't recommend the approach unless you have other overriding application requirements. But that's merely my opinion, and other people seem to like persistent caches and running total export (especially router people, who need to keep the cache around for other reasons anyway), and I guess it's just a sort of point of view thing.

My problem with export from persistent caches with updating totals is that if you want to do anything with the data other than drive per-flow or per-aggregate displays (a la rrdtool or MRTG), you need to do the same amount of postprocessing as you would with nonpersistent export with Ta equal to your desired export interval in order to get "real flow data", and you lose packet and byte counters per active export interval. When each subpart of the flow has accurate first/last timestamps, you get a time series which can give you some idea of the evolution of the packet and byte rates (is it regular i.e. bulk transfer? is its rate slowly variable i.e. bulk transfer under congestion control? is it a low rate highly variable flow? and so on). With persistent cache export you kind of have the same thing, but (1) you have to subtract to get it and (2) you have no accurate midpoint timestamp; you can only deduce it from the export time and your best guess about Tde and Tdc.

I'm not sure what you mean about "how to coordinate resetting (start time) on export"... if you're using non-persistent export, you expire the flow completely out of the cache on active timeout (with flowEndReason activeTimeout) and start a new flow record when you see the first (continuation) packet. If you want to keep ancillary information about the flow (e.g. deduced AS information if you're not a packet-forwarding device, other private labeling information that is expensive to calculate or requires examination of leading payload), you can keep it around in a secondary cache to be resurrected by flow key and associated with the new flow record when the continuation packet comes.

As 5101bis is still open, I'll see if I can come up with some suitable language on the meaning of timestamps therefor (without the editorializing on persistent export), and propose to the list.

Best regards,

Brian


On Oct 23, 2012, at 11:59 PM, John Court <johnwcrt@au1.ibm.com> wrote:

Thanks for the interest in resolving this,

Andrew, I understand your persistent cache argument.  The reason I don't personally use "Total" and DO use "Delta"  is more around what the collector does should it not see some of the updates and suddenly gets one that shows large counter values.  This could mistakenly result in showing huge traffic over a short period incorrectly.  This is particularly true if you do as Brian suggested and are setting the flowStartTime based only on the current record view.

Brian, thanks for your detailed explanation.  Everything with the exception of flowStartTime was as I am currently doing.  I had perhaps mistakenly taken the approach that keeping the flowStartTime as the "conceptual" start rather than for this reporting period would make it easier for the collector to understand what was happening.  Not actually sure how I will co-ordinate reseting that time on export yet :-)

I would suggest that this sort of situation and Brians explanation be added as an example perhaps in 5101 or even 5472 as it would help with Collector interoperability I am sure :-)

Thanks again.


John Court
Software Engineer
IBM Security Systems Division
IBM Australia Development Laboratory
Office:  +61 7 5552 4014
Mobile: +61 430 841328

_______________________________________________
IPFIX mailing list
IPFIX@ietf.org
https://www.ietf.org/mailman/listinfo/ipfix

--=_alternative 00211D294A257AA1_=-- From trammell@tik.ee.ethz.ch Wed Oct 24 00:13:23 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 23AF821F8CF9 for ; Wed, 24 Oct 2012 00:13:23 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.72 X-Spam-Level: X-Spam-Status: No, score=-6.72 tagged_above=-999 required=5 tests=[AWL=-0.122, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7UeY0FGkgSvw for ; Wed, 24 Oct 2012 00:13:21 -0700 (PDT) Received: from smtp.ee.ethz.ch (smtp.ee.ethz.ch [129.132.2.219]) by ietfa.amsl.com (Postfix) with ESMTP id 8236621F8C9F for ; Wed, 24 Oct 2012 00:13:21 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by smtp.ee.ethz.ch (Postfix) with ESMTP id A867CD9314; Wed, 24 Oct 2012 09:13:20 +0200 (MEST) X-Virus-Scanned: by amavisd-new on smtp.ee.ethz.ch Received: from smtp.ee.ethz.ch ([127.0.0.1]) by localhost (.ee.ethz.ch [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 1Fo3RN3Rh-nT; Wed, 24 Oct 2012 09:13:20 +0200 (MEST) Received: from etx-public-dock-188-dhcp.ethz.ch (etx-public-dock-188-dhcp.ethz.ch [82.130.81.188]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: briant) by smtp.ee.ethz.ch (Postfix) with ESMTPSA id 67F40D9307; Wed, 24 Oct 2012 09:13:20 +0200 (MEST) Content-Type: multipart/alternative; boundary="Apple-Mail=_4F749E3B-5FE6-4442-9FC4-8016A75D06B8" Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) From: Brian Trammell In-Reply-To: Date: Wed, 24 Oct 2012 09:13:20 +0200 Message-Id: <26E338E1-DB5E-4270-BE24-9E6294A0FE68@tik.ee.ethz.ch> References: To: John Court X-Mailer: Apple Mail (2.1499) Cc: ipfix@ietf.org Subject: Re: [IPFIX] Export of long lived flow information X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Oct 2012 07:13:23 -0000 --Apple-Mail=_4F749E3B-5FE6-4442-9FC4-8016A75D06B8 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Hi, John, Okay, I understand now... I wasn't really responding directly to you on = this point, just to the general thread on timestamps in MPs with = persistent-cache export, and delta versus total counters. In your case, if you've _already got_ a persistent cache for other = reasons, then by all means use it. :) In that case you just need an = extra timestamp, to keep track of the time of the first packet observed = after the last export, in addition to the first "real" packet, if you = want to export "complete" (delta) flows. Best regards, Brian On Oct 24, 2012, at 8:00 AM, John Court wrote: > Hi Brian,=20 >=20 > I suspect I have been mis-interpreting your concept of "persistent = caches". =46rom the way you describe it, I would categorise what I = export from as a persistent cache. The reason being that for packet = processing efficiency reasons, all the "connection" concepts are kept in = one place and that includes the accounting (i.e. packets, octets). I = only get to see these periodically or on connection close. In IPFIX = terminology the meter isn't resetting the flowStartTime or doing actual = flow termination outside of detection of "connection" termination. = Perhaps it could but that time may be used in other processing, I will = have to see. =20 >=20 > I am not arguing with any of your points. I see the logic behind the = operation, I just need to adjust some outputs for long lived = connections. =20 >=20 > Thanks=20 >=20 > John Court >=20 >=20 >=20 >=20 > From: Brian Trammell =20 > To: John Court/Australia/IBM@IBMAU,=20 > Cc: ipfix@ietf.org=20 > Date: 24/10/2012 15:19=20 > Subject: Re: [IPFIX] Export of long lived flow information=20 >=20 >=20 >=20 > Hi, John,=20 >=20 > I'll say here that I've never really understood the arguments for = export from persistent caches, and don't recommend the approach unless = you have other overriding application requirements. But that's merely my = opinion, and other people seem to like persistent caches and running = total export (especially router people, who need to keep the cache = around for other reasons anyway), and I guess it's just a sort of point = of view thing.=20 >=20 > My problem with export from persistent caches with updating totals is = that if you want to do anything with the data other than drive per-flow = or per-aggregate displays (a la rrdtool or MRTG), you need to do the = same amount of postprocessing as you would with nonpersistent export = with Ta equal to your desired export interval in order to get "real flow = data", and you lose packet and byte counters per active export interval. = When each subpart of the flow has accurate first/last timestamps, you = get a time series which can give you some idea of the evolution of the = packet and byte rates (is it regular i.e. bulk transfer? is its rate = slowly variable i.e. bulk transfer under congestion control? is it a low = rate highly variable flow? and so on). With persistent cache export you = kind of have the same thing, but (1) you have to subtract to get it and = (2) you have no accurate midpoint timestamp; you can only deduce it from = the export time and your best guess about Tde and Tdc.=20 >=20 > I'm not sure what you mean about "how to coordinate resetting (start = time) on export"... if you're using non-persistent export, you expire = the flow completely out of the cache on active timeout (with = flowEndReason activeTimeout) and start a new flow record when you see = the first (continuation) packet. If you want to keep ancillary = information about the flow (e.g. deduced AS information if you're not a = packet-forwarding device, other private labeling information that is = expensive to calculate or requires examination of leading payload), you = can keep it around in a secondary cache to be resurrected by flow key = and associated with the new flow record when the continuation packet = comes.=20 >=20 > As 5101bis is still open, I'll see if I can come up with some suitable = language on the meaning of timestamps therefor (without the = editorializing on persistent export), and propose to the list.=20 >=20 > Best regards,=20 >=20 > Brian=20 >=20 >=20 > On Oct 23, 2012, at 11:59 PM, John Court wrote:=20= >=20 > Thanks for the interest in resolving this,=20 >=20 > Andrew, I understand your persistent cache argument. The reason I = don't personally use "Total" and DO use "Delta" is more around what the = collector does should it not see some of the updates and suddenly gets = one that shows large counter values. This could mistakenly result in = showing huge traffic over a short period incorrectly. This is = particularly true if you do as Brian suggested and are setting the = flowStartTime based only on the current record view.=20 >=20 > Brian, thanks for your detailed explanation. Everything with the = exception of flowStartTime was as I am currently doing. I had perhaps = mistakenly taken the approach that keeping the flowStartTime as the = "conceptual" start rather than for this reporting period would make it = easier for the collector to understand what was happening. Not actually = sure how I will co-ordinate reseting that time on export yet :-)=20 >=20 > I would suggest that this sort of situation and Brians explanation be = added as an example perhaps in 5101 or even 5472 as it would help with = Collector interoperability I am sure :-)=20 >=20 > Thanks again.=20 >=20 >=20 > John Court > Software Engineer > IBM Security Systems Division > IBM Australia Development Laboratory > Office: +61 7 5552 4014 > Mobile: +61 430 841328 >=20 > _______________________________________________ > IPFIX mailing list > IPFIX@ietf.org > https://www.ietf.org/mailman/listinfo/ipfix=20 >=20 --Apple-Mail=_4F749E3B-5FE6-4442-9FC4-8016A75D06B8 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii Hi, = John,

Okay, I understand now... I wasn't really = responding directly to you on this point, just to the general thread on = timestamps in MPs with persistent-cache export, and delta versus total = counters.

In your case, if you've _already got_ = a persistent cache for other reasons, then by all means use it. :) In = that case you just need an extra timestamp, to keep track of the time of = the first packet observed after the last export, in addition to the = first "real" packet, if you want to export "complete" (delta) = flows.

Best = regards,

Brian


=
On Oct 24, 2012, at 8:00 AM, John Court <johnwcrt@au1.ibm.com> = wrote:

Hi Brian,

I suspect I have been = mis-interpreting your concept of "persistent caches".  =46rom the way you describe it, I would categorise what I export from as a persistent cache. =  The reason being that for packet processing efficiency reasons, all the = "connection" concepts are kept in one place and that includes the accounting (i.e. = packets, octets).  I only get to see these periodically or on connection = close. In IPFIX terminology the meter isn't resetting the flowStartTime or = doing actual flow termination outside of detection of "connection" termination.  Perhaps it could but that time may be used in other processing, I will have to see.  

I am not arguing with any of = your points.  I see the logic behind the operation, I just need to adjust some outputs for long lived connections.  

Thanks

John Court



From:   =      Brian Trammell <trammell@tik.ee.ethz.ch>
To:   =      John = Court/Australia/IBM@IBMAU,
Cc:   =      ipfix@ietf.org
Date:   =      24/10/2012 = 15:19
Subject: =        Re: [IPFIX] Export of long lived flow information



Hi, John,

I'll say here that I've never really understood the = arguments for export from persistent caches, and don't recommend the approach = unless you have other overriding application requirements. But that's merely my opinion, and other people seem to like persistent caches and running = total export (especially router people, who need to keep the cache around for other reasons anyway), and I guess it's just a sort of point of view = thing.

My problem with export from persistent caches with = updating totals is that if you want to do anything with the data other than drive per-flow or per-aggregate displays (a la rrdtool or MRTG), you need to do the same amount of postprocessing as you would with nonpersistent = export with Ta equal to your desired export interval in order to get "real flow data", and you lose packet and byte counters per active export interval. When each subpart of the flow has accurate first/last = timestamps, you get a time series which can give you some idea of the evolution of the packet and byte rates (is it regular i.e. bulk transfer? is its rate slowly variable i.e. bulk transfer under congestion control? is it a low rate highly variable flow? and so on). With persistent cache export you kind of have the same thing, but (1) you have to subtract to get it and (2) you have no accurate midpoint timestamp; you can only deduce it from the export time and your best guess about Tde and Tdc.

I'm not sure what you mean about "how to coordinate resetting (start time) on export"... if you're using non-persistent export, you expire the flow completely out of the cache on active = timeout (with flowEndReason activeTimeout) and start a new flow record when you see the first (continuation) packet. If you want to keep ancillary = information about the flow (e.g. deduced AS information if you're not a = packet-forwarding device, other private labeling information that is expensive to = calculate or requires examination of leading payload), you can keep it around in a secondary cache to be resurrected by flow key and associated with the new flow record when the continuation packet comes.

As 5101bis is still open, I'll see if I can come up = with some suitable language on the meaning of timestamps therefor (without = the editorializing on persistent export), and propose to the list.

Best regards,

Brian


On Oct 23, 2012, at 11:59 PM, John Court = <johnwcrt@au1.ibm.com>= wrote:

Thanks for the interest in = resolving this,

Andrew, I understand your persistent cache argument.  The reason I don't personally use "Total" and DO use "Delta"  is more around what the collector does should it not see some of the = updates and suddenly gets one that shows large counter values.  This could mistakenly result in showing huge traffic over a short period = incorrectly.  This is particularly true if you do as Brian suggested and are = setting the flowStartTime based only on the current record view.

Brian, thanks for your detailed explanation.  Everything with the exception of flowStartTime was as I am currently doing.  I had = perhaps mistakenly taken the approach that keeping the flowStartTime as the = "conceptual" start rather than for this reporting period would make it easier for the collector to understand what was happening.  Not actually sure how I will co-ordinate reseting that time on export yet :-)

I would suggest that this sort of situation and Brians explanation be = added as an example perhaps in 5101 or even 5472 as it would help with = Collector interoperability I am sure :-)

Thanks again.


John Court
Software Engineer
IBM Security Systems Division
IBM Australia Development Laboratory
Office:  +61 7 5552 4014
Mobile: +61 430 841328

_______________________________________________
IPFIX mailing list
IPFIX@ietf.org
https://www.ietf.org/mailman/listinfo/ipfix


= --Apple-Mail=_4F749E3B-5FE6-4442-9FC4-8016A75D06B8-- From paitken@cisco.com Wed Oct 24 02:39:46 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2601221F8A89 for ; Wed, 24 Oct 2012 02:39:46 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.519 X-Spam-Level: X-Spam-Status: No, score=-10.519 tagged_above=-999 required=5 tests=[AWL=0.080, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qAxarHVA8+CB for ; Wed, 24 Oct 2012 02:39:45 -0700 (PDT) Received: from ams-iport-2.cisco.com (ams-iport-2.cisco.com [144.254.224.141]) by ietfa.amsl.com (Postfix) with ESMTP id 4568B21F8A62 for ; Wed, 24 Oct 2012 02:39:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=961; q=dns/txt; s=iport; t=1351071585; x=1352281185; h=message-id:date:from:mime-version:to:cc:subject: references:in-reply-to:content-transfer-encoding; bh=Mv+R0lAVUP7TMUeEmTBwjBCHBEII/BFnNB4fB1Jenws=; b=bh+O9bhaRea61K2dwfFZhRSGl78YLwBYWLyVHCv1GGDeG/XV+zk4Lpkx 3mwPk0VDL+ldUg83RpB/+lr1u2kTIai6tIQq1yBm2oXP3VyGFku3YCk1t GNKytQ5JGEdGF3N9dU+mJ4Rw69MDBoKHdg+q8S1cIL4eDn5eehGy/hRIj c=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Av8EABW2h1CQ/khR/2dsb2JhbABEwXyBCIIeAQEBAwESASVAAQULCyEWDwkDAgECAUUGDQEHAQEeh1wGmx6gAJJLA5VzhWSIaoFrgnA X-IronPort-AV: E=Sophos;i="4.80,639,1344211200"; d="scan'208";a="77710362" Received: from ams-core-1.cisco.com ([144.254.72.81]) by ams-iport-2.cisco.com with ESMTP; 24 Oct 2012 09:39:44 +0000 Received: from [144.254.153.39] (dhcp-144-254-153-39.cisco.com [144.254.153.39]) by ams-core-1.cisco.com (8.14.5/8.14.5) with ESMTP id q9O9dhV1007491; Wed, 24 Oct 2012 09:39:44 GMT Message-ID: <5087B761.9090405@cisco.com> Date: Wed, 24 Oct 2012 10:39:45 +0100 From: Paul Aitken User-Agent: Mozilla/5.0 (X11; Linux i686; rv:15.0) Gecko/20120912 Thunderbird/15.0.1 MIME-Version: 1.0 To: Brian Trammell References: <26E338E1-DB5E-4270-BE24-9E6294A0FE68@tik.ee.ethz.ch> In-Reply-To: <26E338E1-DB5E-4270-BE24-9E6294A0FE68@tik.ee.ethz.ch> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: John Court , ipfix@ietf.org Subject: Re: [IPFIX] Export of long lived flow information X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Oct 2012 09:39:46 -0000 Brian, > In your case, if you've _already got_ a persistent cache for other > reasons, then by all means use it. :) In that case you just need an > extra timestamp, to keep track of the time of the first packet > observed after the last export, in addition to the first "real" > packet, if you want to export "complete" (delta) flows. A collector can obtain delta counts from a series of total counts. It may be a little inaccurate if observations were not evenly distributed throughout the interval between total count reports. However this may not matter if the collector is simply reporting aggregates or averages within an interval. eg, if the collector produces hourly reports, it can simply calculate the delta between the current packetTotalCount and the packetTotalCount an hour ago, without being concerned whether the packets arrived at the beginning of the hour, throughout the hour, or at the end of the hour. P. From paitken@cisco.com Wed Oct 24 02:48:28 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 687D721F8BB4 for ; Wed, 24 Oct 2012 02:48:28 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.523 X-Spam-Level: X-Spam-Status: No, score=-10.523 tagged_above=-999 required=5 tests=[AWL=0.075, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id e6eKsC8OGZKa for ; Wed, 24 Oct 2012 02:48:27 -0700 (PDT) Received: from ams-iport-3.cisco.com (ams-iport-3.cisco.com [144.254.224.146]) by ietfa.amsl.com (Postfix) with ESMTP id 559D521F8AE0 for ; Wed, 24 Oct 2012 02:48:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=1569; q=dns/txt; s=iport; t=1351072107; x=1352281707; h=message-id:date:from:mime-version:to:cc:subject: references:in-reply-to; bh=/rvi59v51wEMcgx6Y17SYZWCIMIN6cUJ6FIPdR5F4+Q=; b=jKpqRhik7d+N9czPasWhECjhXX9e8bdbo0zKfQUd13CxXiaffpTty9P4 jLR72fqEK7iCKD+W5QgxpNlLOMmpMWa7g4gsJJiqxKHm7Y8Xur4B96wL4 g2LGO8M9YKgu50rTjWaOnSmNhH/8R5xJc0TunhX4k/A1bDOVp5i4ns/pr 0=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApsFANW4h1CQ/khL/2dsb2JhbABEi3O2CYEIghUJAQEBAwESAWUBBQsLAwEdFg8JAwIBAgFFBg0BBwEBHodcBpshoACSSwOVc4VkiGqBa4Jw X-IronPort-AV: E=Sophos;i="4.80,639,1344211200"; d="scan'208,217";a="9048170" Received: from ams-core-2.cisco.com ([144.254.72.75]) by ams-iport-3.cisco.com with ESMTP; 24 Oct 2012 09:48:26 +0000 Received: from [144.254.153.39] (dhcp-144-254-153-39.cisco.com [144.254.153.39]) by ams-core-2.cisco.com (8.14.5/8.14.5) with ESMTP id q9O9mQAP030378; Wed, 24 Oct 2012 09:48:26 GMT Message-ID: <5087B96B.7020500@cisco.com> Date: Wed, 24 Oct 2012 10:48:27 +0100 From: Paul Aitken User-Agent: Mozilla/5.0 (X11; Linux i686; rv:15.0) Gecko/20120912 Thunderbird/15.0.1 MIME-Version: 1.0 To: John Court References: In-Reply-To: Content-Type: multipart/alternative; boundary="------------080908000104000204030009" Cc: ipfix@ietf.org Subject: Re: [IPFIX] Export of long lived flow information X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Oct 2012 09:48:28 -0000 This is a multi-part message in MIME format. --------------080908000104000204030009 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit John, > I suspect I have been mis-interpreting your concept of "persistent > caches". In a normal cache, the entries are eventually removed - because they've ended, or they've not seen traffic for an amount of time, or they're just too old, or there's simply not enough room in the cache. Whereas in a permanent cache, the entries are never removed. P. --------------080908000104000204030009 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit
John,

I suspect I have been mis-interpreting your concept of "persistent caches".

In a normal cache, the entries are eventually removed - because they've ended, or they've not seen traffic for an amount of time, or they're just too old, or there's simply not enough room in the cache.

Whereas in a permanent cache, the entries are never removed.

P.
--------------080908000104000204030009-- From andrewf@plixer.com Wed Oct 24 09:24:31 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0423A21F86D9 for ; Wed, 24 Oct 2012 09:24:31 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.322 X-Spam-Level: X-Spam-Status: No, score=-2.322 tagged_above=-999 required=5 tests=[AWL=0.124, BAYES_00=-2.599, HTML_MESSAGE=0.001, SARE_SUB_ENC_UTF8=0.152] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fXM46harSHVF for ; Wed, 24 Oct 2012 09:24:29 -0700 (PDT) Received: from smtp.plixer.com (smtp.plixer.com [66.186.184.193]) by ietfa.amsl.com (Postfix) with ESMTP id 78A2321F8673 for ; Wed, 24 Oct 2012 09:24:29 -0700 (PDT) Received: from [10.100.1.132] ([10.100.1.132]) by smtp.plixer.com over TLS secured channel with Microsoft SMTPSVC(6.0.3790.4675); Wed, 24 Oct 2012 12:24:28 -0400 Message-ID: <5088163B.8070004@plixer.com> Date: Wed, 24 Oct 2012 12:24:27 -0400 From: Andrew Feren User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:19.0) Gecko/19.0 Thunderbird/19.0a1 MIME-Version: 1.0 To: IETF IPFIX Working Group Content-Type: multipart/alternative; boundary="------------080708020900020203040405" X-OriginalArrivalTime: 24 Oct 2012 16:24:28.0087 (UTC) FILETIME=[09FE0C70:01CDB204] Subject: [IPFIX] string vs octetArray (for non UTF-8 character sets) X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Oct 2012 16:24:31 -0000 This is a multi-part message in MIME format. --------------080708020900020203040405 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Hi all, I ran into something the other day that doesn't appear to be an issue with any current standard information elements, but given the number of vendors exporting URL information seems like the issue will come up eventually. I originally gave URLs a data type of string as this seemed like the most appropriate type for a human readable string, but then my parser squawked about the following URL in an export http://www.plixer.com/blog/scrutinizer/the-null-scan-you're-being-watched/ In the above URL the "you're" was seen on the wire as "you<92>re" (<92> representing one hex byte). I don't know what character set that is, but Windows thinks it is an apostrophe (or RIGHT SINGLE QUOTATION MARK if we were speaking UTF-8) This got me wondering. What is the right thing to do when monitoring text that is not necessarily UTF-8. a) treat it as an octetArray? This works, but doesn't feel quite right. It seems useful to have a distinction between raw bytes and readable strings. For presentation purposes for example. b) expect the exporter to convert to something that is UTF-8 and still accurately reports what was observed. (For example in this case converting byte in question to an ascii ' (<27>) looks right and is valid UTF-8, but results in a 404, but converting to a real UTF-8 RIGHT SINGLE QUOTATION MARK works) c) define a new data type with character set information. d) whistle and walk away e) something else Thoughts? -Andrew --------------080708020900020203040405 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Hi all,

I ran into something the other day that doesn't appear to be an issue with any current standard information elements, but given the number of vendors exporting URL information seems like the issue will come up eventually.

I originally gave URLs a data type of string as this seemed like the most appropriate type for a human readable string, but then my parser squawked about the following URL in an export
http://www.plixer.com/blog/scrutinizer/the-null-scan-you’re-being-watched/
In the above URL the "you're" was seen on the wire as "you<92>re" (<92> representing one hex byte).  I don't know what character set that is, but Windows thinks it is an apostrophe (or RIGHT SINGLE QUOTATION MARK if we were speaking UTF-8)

This got me wondering.  What is the right thing to do when monitoring text that is not necessarily UTF-8.

a) treat it as an octetArray?  This works, but doesn't feel quite right.  It seems useful to have a distinction between raw bytes and readable strings.  For presentation purposes for example.

b) expect the exporter to convert to something that is UTF-8 and still accurately reports what was observed.  (For example in this case converting byte in question to an ascii ' (<27>) looks right and is valid UTF-8, but results in a 404, but converting to a real UTF-8 RIGHT SINGLE QUOTATION MARK works)

c) define a new data type with character set information.

d) whistle and walk away

e) something else

Thoughts?
-Andrew
--------------080708020900020203040405-- From andrewf@plixer.com Wed Oct 24 11:10:32 2012 Return-Path: X-Original-To: ipfix@ietfa.amsl.com Delivered-To: ipfix@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 85F3C21F8837 for ; Wed, 24 Oct 2012 11:10:32 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.116 X-Spam-Level: X-Spam-Status: No, score=-2.116 tagged_above=-999 required=5 tests=[AWL=-0.118, BAYES_00=-2.599, HTML_MESSAGE=0.001, J_CHICKENPOX_64=0.6] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id yrpZ68uatwGS for ; Wed, 24 Oct 2012 11:10:28 -0700 (PDT) Received: from smtp.plixer.com (smtp.plixer.com [66.186.184.193]) by ietfa.amsl.com (Postfix) with ESMTP id 16CF521F880B for ; Wed, 24 Oct 2012 11:10:24 -0700 (PDT) Received: from [10.100.1.132] ([10.100.1.132]) by smtp.plixer.com over TLS secured channel with Microsoft SMTPSVC(6.0.3790.4675); Wed, 24 Oct 2012 14:10:21 -0400 Message-ID: <50882F0C.5020200@plixer.com> Date: Wed, 24 Oct 2012 14:10:20 -0400 From: Andrew Feren User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:19.0) Gecko/19.0 Thunderbird/19.0a1 MIME-Version: 1.0 To: IETF IPFIX Working Group References: <8F849333-9503-4177-BB21-70426C600E93@cisco.com> <50869957.1020906@plixer.com> In-Reply-To: Content-Type: multipart/alternative; boundary="------------060405050602040100000606" X-OriginalArrivalTime: 24 Oct 2012 18:10:21.0076 (UTC) FILETIME=[D4AB3D40:01CDB212] Subject: Re: [IPFIX] Export of long lived flow information X-BeenThere: ipfix@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: IPFIX WG discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Oct 2012 18:10:32 -0000 This is a multi-part message in MIME format. --------------060405050602040100000606 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Hi Andrew, So we aren't really talking about a mixture of deltas and updates, but totalCounters refreshed at regular intervals and shifting the responsibility of calculating deltas from the EP to the CP. To answer your question though "Would collectors be able to work with it?" There isn't any reason I can see that this can't work, but I'm not sure how many current CPs will do anything useful with the extra totals. A few thoughts in no particular order. * if you are worried about losing packets there are other transports available than UDP. * I'd rather get updates as deltas. * having no updates for a long time can cause data to be inaccurate when trying to diagnose a problem now as there may be many flows that are active, but haven't reported anything. * depending on how long it might be before a total is sent refreshing the totals periodically is probably a good thing -Andrew On 10/23/2012 10:16 AM, Andrew Johnson wrote: > Hi Andrew > > When using a permanent cache, the flows can be sent periodically using update (as opposed to delta) counters. The idea is that the Collector can be kept up to date without actually ageing out the flow, so there is no flow end reason. Updates send the latest state of the flow, which is preferable to deltas because the Exporting Device doesn't need to store the state of what was last sent (or reset it's internal counters) and if an update gets lost (e.g. with UDP) then the next update will ensure the Collector gets the full counts. > > I am wondering if there is a use case for periodically sending the state of the flow (using update counters) for long lived flows that will be aged out at some point in the future. > > > Cheers, Andrew > > > On 23 Oct 2012, at 14:19, Andrew Feren wrote: > >> Hi Andrew, >> >> I'm not sure I understand your question. >> >> How is a delta different than an update? >> >> Specific to flowEndReason rather than sending 0 each update should include the reason (e.g. 0x02 active timeout). Do you have some situation in mind where a flow is being exported for an unknown reason? Or is the reason known, but it doesn't fit one of the 5 currently defined? >> >> -Andrew >> >> On 10/23/2012 04:42 AM, Andrew Johnson wrote: >>> Hi folks >>> >>> I've been thinking for a while now that we should be able to send a mixture of delta and update records for the same flow, but I haven't really figured out how to make the semantics obvious to the collector. >>> >>> i.e.: Delta (new record), update, update, update(ends the flow) >>> >>> >>> Perhaps we could use the flow end reason to make things more explicit. For example, all but the last packet have a flowEndReason of 0 and then the last one updates the reason. >>> >>> Would this sort of thing be useful? Would collectors be able to work with it? >>> >>> >>> Cheers, Andrew >>> >>> >>> >>> On 23 Oct 2012, at 08:43, Brian Trammell wrote: >>>> Hi, John, >>>> >>>> Your assumptions are basically correct. The IPFIX model of a Metering Process (MP) makes an implicit assumption that it will be configured or configurable to export information about long-lived flows every n minutes through active timeout. From the MP side, this allows periodic complete flushing of the flow cache. >>>> >>>> More importantly (IMO), on the Collecting Process (CP) side, it provides a guarantee that every packet observed and selected for export will be accounted for within Ta + Tde + Tdc, where Ta is the active timeout, Tde is the MP to EP delay (how long it takes for an exported flow to make it from the MP cache, through the Exporting Process (EP), into an IPFIX Message, which is implementation dependent but typically short) and Tde is the export delay (OWD from EP to CP). This is important for streaming process applications, as after this time, the CP and downstream processes can assume that no further information about the past will become available. >>>> >>>> The tradeoff here is that shorter Ta causes more records to be exported about long flows, and a longer Ta causes a longer delay, which some streaming applications can't tolerate. >>>> >>>> In any case, the assumption is that the flow is no longer in the cache after active timeout, so you don't know when the flow really started. Since the flowStartTime is the timestamp of the first observed packet within the record, and flowEndTime is the timestamp of the last observed packet within the record, the timestamps would then be record-local.... i.e. a flow less than twice the active timeout (with at least one packet per idle timeout) would result in two flow records. The first would have a timestamp range between the first packet of the flow and the last packet observed before the active timeout; the second between the first packet observed after the active timeout and the last packet of the flow. >>>> >>>> This does, admittedly, make it rather difficult to stitch records for long flows together -- you can't match timestamps, and essentially need to simulate the flow cache with a longer active timeout. For applications requiring a single record per flow, active timeouts can be set to be practically infinite, with the tradeoff that you never know at the CP when you'll get a flow with a start time far in the past. >>>> >>>> Best regards, >>>> >>>> Brian >>>> >>>> >>>> On Oct 23, 2012, at 3:01 AM, John Court wrote: >>>> >>>>> Hi, >>>>> >>>>> I have been a subscriber to the list for a little over a year, and an implementer of IPFIX export for at least one product. This WG has done great work overall ! >>>>> >>>>> One area that still has me a little confused even after researching as many of the RFCs as possible including RFC5472 is how to treat export of long lived flows. >>>>> >>>>> At the moment I use "DeltaCount" information elements for everything and at specific intervals export long lived flows with the flowEndReason of "flowActiveTimeout". This of course results in multiple flow records for long lived connections over time. Since this situation doesn't seem to be covered explicitly I was hoping someone on the list would point me in the right direction or confirm my assumptions. On thing that is particularly unclear is what to do about flowStart/flowEnd times when sending this type of record. >>>>> >>>>> Thanks >>>>> >>>>> John Court >>>>> Senior Software Engineer >>>>> IBM Security Systems Division >>>>> IBM Australia Development Laboratory >>>>> Office: +61 7 5552 4014 >>>>> Mobile: +61 430 841328 >>>>> >>>>> _______________________________________________ >>>>> IPFIX mailing list >>>>> IPFIX@ietf.org >>>>> https://www.ietf.org/mailman/listinfo/ipfix >>>> _______________________________________________ >>>> IPFIX mailing list >>>> IPFIX@ietf.org >>>> https://www.ietf.org/mailman/listinfo/ipfix >>> _______________________________________________ >>> IPFIX mailing list >>> IPFIX@ietf.org >>> https://www.ietf.org/mailman/listinfo/ipfix >> _______________________________________________ >> IPFIX mailing list >> IPFIX@ietf.org >> https://www.ietf.org/mailman/listinfo/ipfix --------------060405050602040100000606 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Hi Andrew,

So we aren't really talking about a mixture of deltas and updates, but totalCounters refreshed at regular intervals and shifting the responsibility of calculating deltas from the EP to the CP.

To answer your question though

"Would collectors be able to work with it?"
There isn't any reason I can see that this can't work, but I'm not sure how many current CPs will do anything useful with the extra totals.

A few thoughts in no particular order.
  • if you are worried about losing packets there are other transports available than UDP.
  • I'd rather get updates as deltas.
  • having no updates for a long time can cause data to be inaccurate when trying to diagnose a problem now as there may be many flows that are active, but haven't reported anything.
  • depending on how long it might be before a total is sent refreshing the totals periodically is probably a good thing
-Andrew


On 10/23/2012 10:16 AM, Andrew Johnson wrote:
Hi Andrew

When using a permanent cache, the flows can be sent periodically using update (as opposed to delta) counters.  The idea is that the Collector can be kept up to date without actually ageing out the flow, so there is no flow end reason.  Updates send the latest state of the flow, which is preferable to deltas because the Exporting Device doesn't need to store the state of what was last sent (or reset it's internal counters) and if an update gets lost (e.g. with UDP) then the next update will ensure the Collector gets the full counts.

I am wondering if there is a use case for periodically sending the state of the flow (using update counters) for long lived flows that will be aged out at some point in the future.


Cheers, Andrew


On 23 Oct 2012, at 14:19, Andrew Feren wrote:


Hi Andrew,

I'm not sure I understand your question.

How is a delta different than an update?

Specific to flowEndReason rather than sending 0 each update should include the reason (e.g. 0x02 active timeout).  Do you have some situation in mind where a flow is being exported for an unknown reason?  Or is the reason known,  but it doesn't fit one of the 5 currently defined?

-Andrew

On 10/23/2012 04:42 AM, Andrew Johnson wrote:

Hi folks

I've been thinking for a while now that we should be able to send a mixture of delta and update records for the same flow, but I haven't really figured out how to make the semantics obvious to the collector.

i.e.:    Delta (new record), update, update, update(ends the flow)


Perhaps we could use the flow end reason to make things more explicit.  For example, all but the last packet have a flowEndReason of 0 and then the last one updates the reason.

Would this sort of thing be useful?  Would collectors be able to work with it?


Cheers, Andrew



On 23 Oct 2012, at 08:43, Brian Trammell wrote:

Hi, John,

Your assumptions are basically correct. The IPFIX model of a Metering Process (MP) makes an implicit assumption that it will be configured or configurable to export information about long-lived flows every n minutes through active timeout. From the MP side, this allows periodic complete flushing of the flow cache.

More importantly (IMO), on the Collecting Process (CP) side, it provides a guarantee that every packet observed and selected for export will be accounted for within Ta + Tde + Tdc, where Ta is the active timeout, Tde is the MP to EP delay (how long it takes for an exported flow to make it from the MP cache, through the Exporting Process (EP), into an IPFIX Message, which is implementation dependent but typically short) and Tde is the export delay (OWD from EP to CP). This is important for streaming process applications, as after this time, the CP and downstream processes can assume that no further information about the past will become available.

The tradeoff here is that shorter Ta causes more records to be exported about long flows, and a longer Ta causes a longer delay, which some streaming applications can't tolerate.

In any case, the assumption is that the flow is no longer in the cache after active timeout, so you don't know when the flow really started. Since the flowStartTime is the timestamp of the first observed packet within the record, and flowEndTime is the timestamp of the last observed packet within the record, the timestamps would then be record-local.... i.e. a flow less than twice the active timeout (with at least one packet per idle timeout) would result in two flow records. The first would have a timestamp range between the first packet of the flow and the last packet observed before the active timeout; the second between the first packet observed after the active timeout and the last packet of the flow.

This does, admittedly, make it rather difficult to stitch records for long flows together -- you can't match timestamps, and essentially need to simulate the flow cache with a longer active timeout. For applications requiring a single record per flow, active timeouts can be set to be practically infinite, with the tradeoff that you never know at the CP when you'll get a flow with a start time far in the past.

Best regards,

Brian


On Oct 23, 2012, at 3:01 AM, John Court wrote:


Hi,

I have been a subscriber to the list for a little over a year, and an implementer of IPFIX export for at least one product.  This WG has done great work overall !

One area that still has me a little confused even after researching as many of the RFCs as possible including RFC5472 is how to treat export of long lived flows.

At the moment I use "DeltaCount" information elements for everything and at specific intervals export long lived flows with the flowEndReason of "flowActiveTimeout".  This of course results in multiple flow records for long lived connections over time.  Since this situation doesn't seem to be covered explicitly I was hoping someone on the list would point me in the right direction or confirm my assumptions.  On thing that is particularly unclear is what to do about flowStart/flowEnd times when sending this type of record.

Thanks

John Court
Senior Software Engineer
IBM Security Systems Division
IBM Australia Development Laboratory
Office:  +61 7 5552 4014
Mobile: +61 430 841328

_______________________________________________
IPFIX mailing list
IPFIX@ietf.org
https://www.ietf.org/mailman/listinfo/ipfix

_______________________________________________
IPFIX mailing list
IPFIX@ietf.org
https://www.ietf.org/mailman/listinfo/ipfix

_______________________________________________
IPFIX mailing list
IPFIX@ietf.org
https://www.ietf.org/mailman/listinfo/ipfix

_______________________________________________
IPFIX mailing list
IPFIX@ietf.org
https://www.ietf.org/mailman/listinfo/ipfix


    

    

  


--------------060405050602040100000606--

From johnwcrt@au1.ibm.com  Wed Oct 24 13:29:52 2012
Return-Path: 
X-Original-To: ipfix@ietfa.amsl.com
Delivered-To: ipfix@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id ABE3621F8A17 for ; Wed, 24 Oct 2012 13:29:52 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -9.098
X-Spam-Level: 
X-Spam-Status: No, score=-9.098 tagged_above=-999 required=5 tests=[AWL=1.500,  BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rG5tYiY1U0Wx for ; Wed, 24 Oct 2012 13:29:52 -0700 (PDT)
Received: from e23smtp08.au.ibm.com (e23smtp08.au.ibm.com [202.81.31.141]) by ietfa.amsl.com (Postfix) with ESMTP id 7941021F8964 for ; Wed, 24 Oct 2012 13:29:49 -0700 (PDT)
Received: from /spool/local by e23smtp08.au.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for  from ; Thu, 25 Oct 2012 06:28:50 +1000
Received: from d23relay04.au.ibm.com (202.81.31.246) by e23smtp08.au.ibm.com (202.81.31.205) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted;  Thu, 25 Oct 2012 06:28:47 +1000
Received: from d23av03.au.ibm.com (d23av03.au.ibm.com [9.190.234.97]) by d23relay04.au.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id q9OKJZ5X41812034 for ; Thu, 25 Oct 2012 07:19:38 +1100
Received: from d23av03.au.ibm.com (loopback [127.0.0.1]) by d23av03.au.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id q9OKTX2p013786 for ; Thu, 25 Oct 2012 07:29:34 +1100
Received: from d23mlc03.au.ibm.com (d23mlc03.au.ibm.com [9.190.26.210]) by d23av03.au.ibm.com (8.14.4/8.13.1/NCO v10.0 AVin) with ESMTP id q9OKTXEW013783; Thu, 25 Oct 2012 07:29:33 +1100
In-Reply-To: <5087B96B.7020500@cisco.com>
References:    <5087B96B.7020500@cisco.com>
To: Paul Aitken 
MIME-Version: 1.0
X-KeepSent: E375B6D9:49AD261E-CA257AA1:00703303; type=4; name=$KeepSent
X-Mailer: Lotus Notes Release 8.5.3 September 15, 2011
Message-ID: 
From: John Court 
Date: Thu, 25 Oct 2012 06:28:43 +1000
X-MIMETrack: Serialize by Router on d23mlc03/23/M/IBM(Release 8.5.3FP2HF29 | July 24, 2012) at 25/10/2012 07:28:50, Serialize complete at 25/10/2012 07:28:50
Content-Type: multipart/alternative; boundary="=_alternative 00708F074A257AA1_="
x-cbid: 12102420-5140-0000-0000-0000023F3E44
Cc: ipfix@ietf.org
Subject: Re: [IPFIX] Export of long lived flow information
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: IPFIX WG discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Wed, 24 Oct 2012 20:29:52 -0000

This is a multipart message in MIME format.
--=_alternative 00708F074A257AA1_=
Content-Type: text/plain; charset="US-ASCII"

Just to be crystal clear on this point of persistent caches.  Even when 
sending "totalCount" fields, the flowStartTime is still relative to the 
current flow record, it doesn't represent the "original" first packet ever 
seen for the flow key in the cache ?  I just want to make sure of the 
semantics of flowStartTime in all cases.

Thanks again for the comments and clarifications
 
John Court
Software Engineer
IBM Security Systems Division
IBM Australia Development Laboratory
Office:  +61 7 5552 4014
Mobile: +61 430 841328





From:   Paul Aitken 
To:     John Court/Australia/IBM@IBMAU, 
Cc:     Brian Trammell , ipfix@ietf.org
Date:   24/10/2012 19:49
Subject:        Re: [IPFIX] Export of long lived flow information



John,

I suspect I have been mis-interpreting your concept of "persistent 
caches".

In a normal cache, the entries are eventually removed - because they've 
ended, or they've not seen traffic for an amount of time, or they're just 
too old, or there's simply not enough room in the cache.

Whereas in a permanent cache, the entries are never removed.

P.

--=_alternative 00708F074A257AA1_=
Content-Type: text/html; charset="US-ASCII"

Just to be crystal clear on this point
of persistent caches.  Even when sending "totalCount" fields,
the flowStartTime is still relative to the current flow record, it doesn't
represent the "original" first packet ever seen for the flow
key in the cache ?  I just want to make sure of the semantics of flowStartTime
in all cases.



Thanks again for the comments and clarifications

  

John Court

Software Engineer

IBM Security Systems Division

IBM Australia Development Laboratory

Office:  +61 7 5552 4014

Mobile: +61 430 841328











From:      
 Paul Aitken <paitken@cisco.com>

To:      
 John Court/Australia/IBM@IBMAU,


Cc:      
 Brian Trammell <trammell@tik.ee.ethz.ch>,
ipfix@ietf.org

Date:      
 24/10/2012 19:49

Subject:    
   Re: [IPFIX]
Export of long lived flow information









John,



I suspect I have been mis-interpreting
your concept of "persistent caches".



In a normal cache, the entries are eventually removed - because they've
ended, or they've not seen traffic for an amount of time, or they're just
too old, or there's simply not enough room in the cache.



Whereas in a permanent cache, the entries are never removed.



P.


--=_alternative 00708F074A257AA1_=--


From muenz@net.in.tum.de  Wed Oct 24 13:34:51 2012
Return-Path: 
X-Original-To: ipfix@ietfa.amsl.com
Delivered-To: ipfix@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 590F821F8B8B for ; Wed, 24 Oct 2012 13:34:51 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -6.249
X-Spam-Level: 
X-Spam-Status: No, score=-6.249 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_MED=-4]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id oOPnhNmNNf9Y for ; Wed, 24 Oct 2012 13:34:50 -0700 (PDT)
Received: from mail-out1.informatik.tu-muenchen.de (mail-out1.informatik.tu-muenchen.de [131.159.0.8]) by ietfa.amsl.com (Postfix) with ESMTP id B378421F8B6C for ; Wed, 24 Oct 2012 13:34:50 -0700 (PDT)
Received: from [192.168.2.36] (g229254134.adsl.alicedsl.de [92.229.254.134]) by mail.net.in.tum.de (Postfix) with ESMTPSA id 7A745189C03B; Wed, 24 Oct 2012 22:34:48 +0200 (CEST)
Message-ID: <508850F7.2080801@net.in.tum.de>
Date: Wed, 24 Oct 2012 22:35:03 +0200
From: Gerhard Muenz 
User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:16.0) Gecko/20121010 Thunderbird/16.0.1
MIME-Version: 1.0
To: John Court 
References:    <5087B96B.7020500@cisco.com> 
In-Reply-To: 
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Cc: ipfix@ietf.org
Subject: Re: [IPFIX] Export of long lived flow information
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: IPFIX WG discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Wed, 24 Oct 2012 20:34:51 -0000

Hi,

flowStartTime is the time of the first packet you are reporting on in 
the given record. So, including totalCount fields implicitly means that 
flowStartTime is the time of the very first packet ever observed for 
this flow.

Now, you can continue discussing what happens if both totalCounts and 
deltaCounts are included in the same record :)

Regards,
Gerhard


On 24.10.2012 22:28, John Court wrote:
> Just to be crystal clear on this point of persistent caches.  Even when
> sending "totalCount" fields, the flowStartTime is still relative to the
> current flow record, it doesn't represent the "original" first packet
> ever seen for the flow key in the cache ?  I just want to make sure of
> the semantics of flowStartTime in all cases.
>
> Thanks again for the comments and clarifications
>
> John Court
> Software Engineer
> IBM Security Systems Division
> IBM Australia Development Laboratory
> Office:  +61 7 5552 4014
> Mobile: +61 430 841328
>
>
>
>
>
> From: Paul Aitken 
> To: John Court/Australia/IBM@IBMAU,
> Cc: Brian Trammell , ipfix@ietf.org
> Date: 24/10/2012 19:49
> Subject: Re: [IPFIX] Export of long lived flow information
> ------------------------------------------------------------------------
>
>
>
> John,
>
> I suspect I have been mis-interpreting your concept of "persistent caches".
>
> In a normal cache, the entries are eventually removed - because they've
> ended, or they've not seen traffic for an amount of time, or they're
> just too old, or there's simply not enough room in the cache.
>
> Whereas in a permanent cache, the entries are never removed.
>
> P.
>
>
> _______________________________________________
> IPFIX mailing list
> IPFIX@ietf.org
> https://www.ietf.org/mailman/listinfo/ipfix
>

From paitken@cisco.com  Wed Oct 24 14:19:08 2012
Return-Path: 
X-Original-To: ipfix@ietfa.amsl.com
Delivered-To: ipfix@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4EF6121F863B for ; Wed, 24 Oct 2012 14:19:08 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -10.528
X-Spam-Level: 
X-Spam-Status: No, score=-10.528 tagged_above=-999 required=5 tests=[AWL=0.071, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fOqBHqa59JX4 for ; Wed, 24 Oct 2012 14:19:07 -0700 (PDT)
Received: from ams-iport-2.cisco.com (ams-iport-2.cisco.com [144.254.224.141]) by ietfa.amsl.com (Postfix) with ESMTP id 6CC5C21F8635 for ; Wed, 24 Oct 2012 14:19:07 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=3346; q=dns/txt; s=iport; t=1351113547; x=1352323147; h=message-id:date:from:mime-version:to:cc:subject: references:in-reply-to:content-transfer-encoding; bh=R0H0QU1HsyKOtVfuYzkvxjJKxu+EXgaxfHbZfgOIabg=; b=HoFrZdd5jXW2qL+4XN1ESKgI+sSYQoctoabnBvuS9HPpL9RV9282MMtc MHTZj4Ki6k7VkYzyEBYIU9/OPGYrZvWUCGct7JpCUIS3/jTHL28LuzpG3 TYEPRwHDwWDmEyWosb1drXd5GNYFm5viCjeufTj2jwsZ52rF7+l1pfyVG g=;
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: AgAFAEBaiFCQ/khM/2dsb2JhbABEwX2BCIIeAQEBBAEBAQ8BJTYKAQ4CCxEDAQIBCRYPCQMCAQIBBBEoCAYBDAEFAgEBHodiC5xnoA0EBItdhm0DlXOFZIhqgWuCcA
X-IronPort-AV: E=Sophos;i="4.80,642,1344211200"; d="scan'208";a="77729362"
Received: from ams-core-3.cisco.com ([144.254.72.76]) by ams-iport-2.cisco.com with ESMTP; 24 Oct 2012 21:19:06 +0000
Received: from [10.55.93.58] (dhcp-10-55-93-58.cisco.com [10.55.93.58]) by ams-core-3.cisco.com (8.14.5/8.14.5) with ESMTP id q9OLJ5qc002569; Wed, 24 Oct 2012 21:19:06 GMT
Message-ID: <50885B49.6050603@cisco.com>
Date: Wed, 24 Oct 2012 22:19:05 +0100
From: Paul Aitken 
User-Agent: Mozilla/5.0 (X11; Linux i686; rv:15.0) Gecko/20120912 Thunderbird/15.0.1
MIME-Version: 1.0
To: Gerhard Muenz , John Court 
References:    <5087B96B.7020500@cisco.com>  <508850F7.2080801@net.in.tum.de>
In-Reply-To: <508850F7.2080801@net.in.tum.de>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Cc: ipfix@ietf.org
Subject: Re: [IPFIX] Export of long lived flow information
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: IPFIX WG discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Wed, 24 Oct 2012 21:19:08 -0000

John, Gerhard,

Looking at the definitions:

octetTotalCount:

          The total number of octets in incoming packets
          for this Flow at the Observation Point since the Metering
          Process (re-)initialization for this Observation Point.

packetTotalCount:

          The total number of incoming packets for this Flow
          at the Observation Point since the Metering Process
          (re-)initialization for this Observation Point.


So even if there's a long gap in the traffic, the intention is that a MP 
which is reporting totalCount fields remembers that it saw the flow before.

So +1 to what Gerhard says: "flowStartTime is the time of the very first 
packet ever observed for this flow." (ie, the "original" first packet).


Whereas when reporting deltaCount fields, the MP may "forget" about the 
flow (by purging the cache entry) - so after the traffic gap the flow 
must start over as if it's entirely new, since the MP retains no 
history. Therefore the flowStartTime must be the first packet in the new 
delta - though, the collector might choose to aggregate this with 
previous deltas, setting the flowStartTime to the earliest reported and 
the flowEndTime to the latest.

This is undoubtedly something that we should capture in one of our 
updated docs.

Thanks for the great question John!

P.


On 24/10/12 21:35, Gerhard Muenz wrote:
>
> Hi,
>
> flowStartTime is the time of the first packet you are reporting on in 
> the given record. So, including totalCount fields implicitly means 
> that flowStartTime is the time of the very first packet ever observed 
> for this flow.
>
> Now, you can continue discussing what happens if both totalCounts and 
> deltaCounts are included in the same record :)
>
> Regards,
> Gerhard
>
>
> On 24.10.2012 22:28, John Court wrote:
>> Just to be crystal clear on this point of persistent caches.  Even when
>> sending "totalCount" fields, the flowStartTime is still relative to the
>> current flow record, it doesn't represent the "original" first packet
>> ever seen for the flow key in the cache ?  I just want to make sure of
>> the semantics of flowStartTime in all cases.
>>
>> Thanks again for the comments and clarifications
>>
>> John Court
>> Software Engineer
>> IBM Security Systems Division
>> IBM Australia Development Laboratory
>> Office:  +61 7 5552 4014
>> Mobile: +61 430 841328
>>
>>
>>
>>
>>
>> From: Paul Aitken 
>> To: John Court/Australia/IBM@IBMAU,
>> Cc: Brian Trammell , ipfix@ietf.org
>> Date: 24/10/2012 19:49
>> Subject: Re: [IPFIX] Export of long lived flow information
>> ------------------------------------------------------------------------
>>
>>
>>
>> John,
>>
>> I suspect I have been mis-interpreting your concept of "persistent 
>> caches".
>>
>> In a normal cache, the entries are eventually removed - because they've
>> ended, or they've not seen traffic for an amount of time, or they're
>> just too old, or there's simply not enough room in the cache.
>>
>> Whereas in a permanent cache, the entries are never removed.
>>
>> P.
>>
>>
>> _______________________________________________
>> IPFIX mailing list
>> IPFIX@ietf.org
>> https://www.ietf.org/mailman/listinfo/ipfix
>>


From andrjohn@cisco.com  Wed Oct 24 14:45:57 2012
Return-Path: 
X-Original-To: ipfix@ietfa.amsl.com
Delivered-To: ipfix@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5231321F8C2A for ; Wed, 24 Oct 2012 14:45:57 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -10.299
X-Spam-Level: 
X-Spam-Status: No, score=-10.299 tagged_above=-999 required=5 tests=[AWL=0.300, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nhlt0w+qizOz for ; Wed, 24 Oct 2012 14:45:56 -0700 (PDT)
Received: from ams-iport-2.cisco.com (ams-iport-2.cisco.com [144.254.224.141]) by ietfa.amsl.com (Postfix) with ESMTP id EEFC721F8C13 for ; Wed, 24 Oct 2012 14:45:53 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=4981; q=dns/txt; s=iport; t=1351115156; x=1352324756; h=subject:mime-version:from:in-reply-to:date:cc: content-transfer-encoding:message-id:references:to; bh=1mIhMqI9TM8iobd0r7r6apfUOWiC/TA2nrPx5KOA9rg=; b=ZdOijTe/m09VHT7LRwVwT6AdRflDO3RskKvPvK+8zMzYefV0JlkPP462 AuIKBywIDAqacBfNo5z1Pd6hG1+7ancJiDJbisRLQBOSeIV4flXUh0D+l pyex6UrnKmmTgw33PAjDSA0CNF+VR86UBlLNpF5ckg8t8zqjxEgX2Mm37 Q=;
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: Av8EAE1giFCQ/khM/2dsb2JhbABEwX+BCIIeAQEBBAEBAQ8BJzQLDgILDgMDAQIBLhYRKAgGEyKHYgucZaALBASLXYYMYQOVc45OgWuCcA
X-IronPort-AV: E=Sophos;i="4.80,642,1344211200"; d="scan'208";a="77729684"
Received: from ams-core-3.cisco.com ([144.254.72.76]) by ams-iport-2.cisco.com with ESMTP; 24 Oct 2012 21:45:52 +0000
Received: from cisco.com (mrwint.cisco.com [64.103.70.36]) by ams-core-3.cisco.com (8.14.5/8.14.5) with ESMTP id q9OLjqaP007577 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 24 Oct 2012 21:45:52 GMT
Received: from ams-andrjohn-8718.cisco.com (ams-andrjohn-8718.cisco.com [10.55.163.41]) by cisco.com (8.14.4+Sun/8.8.8) with ESMTP id q9OLjmKP001550; Wed, 24 Oct 2012 22:45:48 +0100 (BST)
Mime-Version: 1.0 (Apple Message framework v1278)
Content-Type: text/plain; charset=us-ascii
From: Andrew Johnson 
In-Reply-To: <50885B49.6050603@cisco.com>
Date: Wed, 24 Oct 2012 22:45:42 +0100
Content-Transfer-Encoding: quoted-printable
Message-Id: 
References:    <5087B96B.7020500@cisco.com>  <508850F7.2080801@net.in.tum.de> <50885B49.6050603@cisco.com>
To: Paul Aitken 
X-Mailer: Apple Mail (2.1278)
Cc: John Court , ipfix@ietf.org
Subject: Re: [IPFIX] Export of long lived flow information
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: IPFIX WG discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Wed, 24 Oct 2012 21:45:57 -0000

Hi all

Paul, thanks for the terminology correction, I did mean "total" counters =
and was confusing people with "updates".

I was thinking that a mechanism that allowed a non-permanent flow to be =
exported multiple time would be useful.  For example, security =
applications generally want to know about a new flow ASAP, so they can =
act on the information, but a short active timeout values lead to using =
more export bandwidth.  I was thinking we could do something like export =
a report of the flow after the first packet, and then export the final =
version of the flow once the normal timeouts had decided it was over.

I had in mind something like using a delta count, followed by a total =
count.  Reading the below definition of Total counts though, I'm not =
sure that will work, but I think it depends on how we interpret the =
definition of "Flow".  If two records have matching key fields but =
different starting timestamps, are they the same Flow?

I would argue that a single Flow can't have two flowStartTimes, so maybe =
not.  This would mean that we shouldn't reset the flowStartTimes between =
sending reports for the same permanent Flow.


Cheers, Andrew


On 24 Oct 2012, at 22:19, Paul Aitken wrote:
> John, Gerhard,
>=20
> Looking at the definitions:
>=20
> octetTotalCount:
>=20
>         The total number of octets in incoming packets
>         for this Flow at the Observation Point since the Metering
>         Process (re-)initialization for this Observation Point.
>=20
> packetTotalCount:
>=20
>         The total number of incoming packets for this Flow
>         at the Observation Point since the Metering Process
>         (re-)initialization for this Observation Point.
>=20
>=20
> So even if there's a long gap in the traffic, the intention is that a =
MP which is reporting totalCount fields remembers that it saw the flow =
before.
>=20
> So +1 to what Gerhard says: "flowStartTime is the time of the very =
first packet ever observed for this flow." (ie, the "original" first =
packet).
>=20
>=20
> Whereas when reporting deltaCount fields, the MP may "forget" about =
the flow (by purging the cache entry) - so after the traffic gap the =
flow must start over as if it's entirely new, since the MP retains no =
history. Therefore the flowStartTime must be the first packet in the new =
delta - though, the collector might choose to aggregate this with =
previous deltas, setting the flowStartTime to the earliest reported and =
the flowEndTime to the latest.
>=20
> This is undoubtedly something that we should capture in one of our =
updated docs.
>=20
> Thanks for the great question John!
>=20
> P.
>=20
>=20
> On 24/10/12 21:35, Gerhard Muenz wrote:
>>=20
>> Hi,
>>=20
>> flowStartTime is the time of the first packet you are reporting on in =
the given record. So, including totalCount fields implicitly means that =
flowStartTime is the time of the very first packet ever observed for =
this flow.
>>=20
>> Now, you can continue discussing what happens if both totalCounts and =
deltaCounts are included in the same record :)
>>=20
>> Regards,
>> Gerhard
>>=20
>>=20
>> On 24.10.2012 22:28, John Court wrote:
>>> Just to be crystal clear on this point of persistent caches.  Even =
when
>>> sending "totalCount" fields, the flowStartTime is still relative to =
the
>>> current flow record, it doesn't represent the "original" first =
packet
>>> ever seen for the flow key in the cache ?  I just want to make sure =
of
>>> the semantics of flowStartTime in all cases.
>>>=20
>>> Thanks again for the comments and clarifications
>>>=20
>>> John Court
>>> Software Engineer
>>> IBM Security Systems Division
>>> IBM Australia Development Laboratory
>>> Office:  +61 7 5552 4014
>>> Mobile: +61 430 841328
>>>=20
>>>=20
>>>=20
>>>=20
>>>=20
>>> From: Paul Aitken 
>>> To: John Court/Australia/IBM@IBMAU,
>>> Cc: Brian Trammell , ipfix@ietf.org
>>> Date: 24/10/2012 19:49
>>> Subject: Re: [IPFIX] Export of long lived flow information
>>> =
------------------------------------------------------------------------
>>>=20
>>>=20
>>>=20
>>> John,
>>>=20
>>> I suspect I have been mis-interpreting your concept of "persistent =
caches".
>>>=20
>>> In a normal cache, the entries are eventually removed - because =
they've
>>> ended, or they've not seen traffic for an amount of time, or they're
>>> just too old, or there's simply not enough room in the cache.
>>>=20
>>> Whereas in a permanent cache, the entries are never removed.
>>>=20
>>> P.
>>>=20
>>>=20
>>> _______________________________________________
>>> IPFIX mailing list
>>> IPFIX@ietf.org
>>> https://www.ietf.org/mailman/listinfo/ipfix
>>>=20
>=20
> _______________________________________________
> IPFIX mailing list
> IPFIX@ietf.org
> https://www.ietf.org/mailman/listinfo/ipfix


From paitken@cisco.com  Wed Oct 24 15:06:44 2012
Return-Path: 
X-Original-To: ipfix@ietfa.amsl.com
Delivered-To: ipfix@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 01ADB1F0C49 for ; Wed, 24 Oct 2012 15:06:44 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -10.532
X-Spam-Level: 
X-Spam-Status: No, score=-10.532 tagged_above=-999 required=5 tests=[AWL=0.067, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4i6-tkC2gClb for ; Wed, 24 Oct 2012 15:06:43 -0700 (PDT)
Received: from ams-iport-1.cisco.com (ams-iport-1.cisco.com [144.254.224.140]) by ietfa.amsl.com (Postfix) with ESMTP id DAAD21F0425 for ; Wed, 24 Oct 2012 15:06:42 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=1809; q=dns/txt; s=iport; t=1351116403; x=1352326003; h=message-id:date:from:mime-version:to:cc:subject: references:in-reply-to:content-transfer-encoding; bh=p7E7h8dVeAsaXx6Oi20TVHwrPfgWhZYF7zsBv69z5kw=; b=kTzZNnraCL67skk8hEDgSEb3K9wd5ZWyRLVYwUTUWJlGw+9cQoX8PeDH T0wI90+Y0azyXf+gIEN2F+gDR5cPZbOl/MhqPk3nuLvoMFq2HaGTyzc71 mzne+Rj6doAuqECdD4q7XaWytOee1bSLOMPOxxcKBB+pIMKbipxp7U0Aj Q=;
X-IronPort-AV: E=Sophos;i="4.80,642,1344211200"; d="scan'208";a="145769132"
Received: from ams-core-1.cisco.com ([144.254.72.81]) by ams-iport-1.cisco.com with ESMTP; 24 Oct 2012 22:06:42 +0000
Received: from [10.55.93.58] (dhcp-10-55-93-58.cisco.com [10.55.93.58]) by ams-core-1.cisco.com (8.14.5/8.14.5) with ESMTP id q9OM6fWk018357; Wed, 24 Oct 2012 22:06:41 GMT
Message-ID: <5088666F.1090106@cisco.com>
Date: Wed, 24 Oct 2012 23:06:39 +0100
From: Paul Aitken 
User-Agent: Mozilla/5.0 (X11; Linux i686; rv:15.0) Gecko/20120912 Thunderbird/15.0.1
MIME-Version: 1.0
To: Andrew Johnson 
References:    <5087B96B.7020500@cisco.com>  <508850F7.2080801@net.in.tum.de> <50885B49.6050603@cisco.com> 
In-Reply-To: 
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Cc: John Court , ipfix@ietf.org
Subject: Re: [IPFIX] Export of long lived flow information
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: IPFIX WG discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Wed, 24 Oct 2012 22:06:44 -0000

Andrew,

> I was thinking that a mechanism that allowed a non-permanent flow to be exported multiple time would be useful.  For example, security applications generally want to know about a new flow ASAP, so they can act on the information, but a short active timeout values lead to using more export bandwidth.  I was thinking we could do something like export a report of the flow after the first packet, and then export the final version of the flow once the normal timeouts had decided it was over.

I have in the past discussed the idea of exporting a "new flow alert" 
using zero-valued counters in order to make the collector aware that 
we've started monitoring it - so I'm claiming prior art on that.


> I had in mind something like using a delta count, followed by a total count.  Reading the below definition of Total counts though, I'm not sure that will work, but I think it depends on how we interpret the definition of "Flow".  If two records have matching key fields but different starting timestamps, are they the same Flow?

5101 defines:

       A Flow is defined as a set of IP packets passing an Observation
       Point in the network during a certain time interval.


- so it's all about the timestamps :-)


> I would argue that a single Flow can't have two flowStartTimes, so maybe not.

However, two flows with different flowStartTimes can be merged into one 
flow.


> This would mean that we shouldn't reset the flowStartTimes between sending reports for the same permanent Flow.

Definitely. If it's a permanent flow and you're exporting totalCount 
fields - which are measured "since the Metering Process 
(re-)initialization for this Observation Point" - then the flowStartTime 
must surely be the time the first ever packet was observed.

P.

From johnwcrt@au1.ibm.com  Wed Oct 24 18:05:26 2012
Return-Path: 
X-Original-To: ipfix@ietfa.amsl.com
Delivered-To: ipfix@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3431911E80A3 for ; Wed, 24 Oct 2012 18:05:26 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -7.398
X-Spam-Level: 
X-Spam-Status: No, score=-7.398 tagged_above=-999 required=5 tests=[AWL=-0.800, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_MED=-4]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1YV10fO8Po27 for ; Wed, 24 Oct 2012 18:05:25 -0700 (PDT)
Received: from e23smtp03.au.ibm.com (e23smtp03.au.ibm.com [202.81.31.145]) by ietfa.amsl.com (Postfix) with ESMTP id AA49711E80A2 for ; Wed, 24 Oct 2012 18:05:24 -0700 (PDT)
Received: from /spool/local by e23smtp03.au.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for  from ; Thu, 25 Oct 2012 11:02:28 +1000
Received: from d23relay04.au.ibm.com (202.81.31.246) by e23smtp03.au.ibm.com (202.81.31.209) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted;  Thu, 25 Oct 2012 11:02:27 +1000
Received: from d23av03.au.ibm.com (d23av03.au.ibm.com [9.190.234.97]) by d23relay04.au.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id q9P0tAFB47906914 for ; Thu, 25 Oct 2012 11:55:11 +1100
Received: from d23av03.au.ibm.com (loopback [127.0.0.1]) by d23av03.au.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id q9P158c1017382 for ; Thu, 25 Oct 2012 12:05:08 +1100
Received: from d23mlc03.au.ibm.com (d23mlc03.au.ibm.com [9.190.26.210]) by d23av03.au.ibm.com (8.14.4/8.13.1/NCO v10.0 AVin) with ESMTP id q9P158FY017377; Thu, 25 Oct 2012 12:05:08 +1100
In-Reply-To: <5088666F.1090106@cisco.com>
References:    <5087B96B.7020500@cisco.com>  <508850F7.2080801@net.in.tum.de> <50885B49.6050603@cisco.com>  <5088666F.1090106@cisco.com>
To: Paul Aitken 
MIME-Version: 1.0
X-KeepSent: 4B5A9A3A:F88C734E-CA257AA2:0005120F; type=4; name=$KeepSent
X-Mailer: Lotus Notes Release 8.5.3 September 15, 2011
Message-ID: 
From: John Court 
Date: Thu, 25 Oct 2012 11:04:17 +1000
X-MIMETrack: Serialize by Router on d23mlc03/23/M/IBM(Release 8.5.3FP2HF29 | July 24, 2012) at 25/10/2012 12:04:25, Serialize complete at 25/10/2012 12:04:25
Content-Type: multipart/alternative; boundary="=_alternative 0005F3634A257AA2_="
x-cbid: 12102501-6102-0000-0000-0000026E5284
Cc: ipfix@ietf.org
Subject: Re: [IPFIX] Export of long lived flow information
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: IPFIX WG discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Thu, 25 Oct 2012 01:05:26 -0000

This is a multipart message in MIME format.
--=_alternative 0005F3634A257AA2_=
Content-Type: text/plain; charset="US-ASCII"

Yep I think everyone is starting to see the ambiguity that needs to be 
cleared up :-)

Paul,

Definitely. If it's a permanent flow and you're exporting totalCount 
fields - which are measured "since the Metering Process 
(re-)initialization for this Observation Point" - then the flowStartTime 
must surely be the time the first ever packet was observed.

If you take that literally shouldn't that be interpreted to mean that the 
totalCount continues into the next time a connection is up between the 
same flow key ?  Even if a flowEndReason of :

 0x03: end of Flow detected
The Flow was terminated because the Metering Process
detected signals indicating the end of the Flow, for
example, the TCP FIN flag.

That clearly wouldn't be of much use IMO and makes it difficult to see 
what the flowEndReason field semantics mean in that context.  Just 
pointing out that taking that definition literally doesn't give a useful 
answer on its own either :-).  Although maybe that does make sense in a 
router context ?  Can you clarify this some more, perhaps you never intend 
using the flowEndReason IE in your case ?

Thanks





From:   Paul Aitken 
To:     Andrew Johnson , 
Cc:     Gerhard Muenz , John 
Court/Australia/IBM@IBMAU, ipfix@ietf.org
Date:   25/10/2012 08:07
Subject:        Re: [IPFIX] Export of long lived flow information



Andrew,

> I was thinking that a mechanism that allowed a non-permanent flow to be 
exported multiple time would be useful.  For example, security 
applications generally want to know about a new flow ASAP, so they can act 
on the information, but a short active timeout values lead to using more 
export bandwidth.  I was thinking we could do something like export a 
report of the flow after the first packet, and then export the final 
version of the flow once the normal timeouts had decided it was over.

I have in the past discussed the idea of exporting a "new flow alert" 
using zero-valued counters in order to make the collector aware that 
we've started monitoring it - so I'm claiming prior art on that.


> I had in mind something like using a delta count, followed by a total 
count.  Reading the below definition of Total counts though, I'm not sure 
that will work, but I think it depends on how we interpret the definition 
of "Flow".  If two records have matching key fields but different starting 
timestamps, are they the same Flow?

5101 defines:

       A Flow is defined as a set of IP packets passing an Observation
       Point in the network during a certain time interval.


- so it's all about the timestamps :-)


> I would argue that a single Flow can't have two flowStartTimes, so maybe 
not.

However, two flows with different flowStartTimes can be merged into one 
flow.


> This would mean that we shouldn't reset the flowStartTimes between 
sending reports for the same permanent Flow.

Definitely. If it's a permanent flow and you're exporting totalCount 
fields - which are measured "since the Metering Process 
(re-)initialization for this Observation Point" - then the flowStartTime 
must surely be the time the first ever packet was observed.

P.



--=_alternative 0005F3634A257AA2_=
Content-Type: text/html; charset="US-ASCII"

Yep I think everyone is starting to see
the ambiguity that needs to be cleared up :-)



Paul,



Definitely. If it's a permanent flow and you're
exporting totalCount 

fields - which are measured "since the Metering Process 

(re-)initialization for this Observation Point" - then the flowStartTime


must surely be the time the first ever packet was observed.



If you take that literally shouldn't
that be interpreted to mean that the totalCount continues into the next
time a connection is up between the same flow key ?  Even if a flowEndReason
of :



 0x03: end of Flow detected

The Flow was terminated because the Metering
Process

detected signals indicating the end of
the Flow, for

example, the TCP FIN flag.



That clearly wouldn't be of much use
IMO and makes it difficult to see what the flowEndReason field semantics
mean in that context.  Just pointing out that taking that definition
literally doesn't give a useful answer on its own either :-).  Although
maybe that does make sense in a router context ?  Can you clarify
this some more, perhaps you never intend using the flowEndReason IE in
your case ?



Thanks











From:      
 Paul Aitken <paitken@cisco.com>

To:      
 Andrew Johnson <andrjohn@cisco.com>,


Cc:      
 Gerhard Muenz <muenz@net.in.tum.de>,
John Court/Australia/IBM@IBMAU, ipfix@ietf.org

Date:      
 25/10/2012 08:07

Subject:    
   Re: [IPFIX]
Export of long lived flow information









Andrew,



> I was thinking that a mechanism that allowed a non-permanent flow
to be exported multiple time would be useful.  For example, security
applications generally want to know about a new flow ASAP, so they can
act on the information, but a short active timeout values lead to using
more export bandwidth.  I was thinking we could do something like
export a report of the flow after the first packet, and then export the
final version of the flow once the normal timeouts had decided it was over.



I have in the past discussed the idea of exporting a "new flow alert"


using zero-valued counters in order to make the collector aware that 

we've started monitoring it - so I'm claiming prior art on that.





> I had in mind something like using a delta count, followed by a total
count.  Reading the below definition of Total counts though, I'm not
sure that will work, but I think it depends on how we interpret the definition
of "Flow".  If two records have matching key fields but
different starting timestamps, are they the same Flow?



5101 defines:



       A Flow is defined as a set of IP packets passing
an Observation

       Point in the network during a certain time interval.





- so it's all about the timestamps :-)





> I would argue that a single Flow can't have two flowStartTimes, so
maybe not.



However, two flows with different flowStartTimes can be merged into one


flow.





> This would mean that we shouldn't reset the flowStartTimes between
sending reports for the same permanent Flow.



Definitely. If it's a permanent flow and you're exporting totalCount 

fields - which are measured "since the Metering Process 

(re-)initialization for this Observation Point" - then the flowStartTime


must surely be the time the first ever packet was observed.



P.






--=_alternative 0005F3634A257AA2_=--


From muenz@net.in.tum.de  Thu Oct 25 11:26:31 2012
Return-Path: 
X-Original-To: ipfix@ietfa.amsl.com
Delivered-To: ipfix@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4ECEC21F8969 for ; Thu, 25 Oct 2012 11:26:31 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -5.949
X-Spam-Level: 
X-Spam-Status: No, score=-5.949 tagged_above=-999 required=5 tests=[AWL=-0.300, BAYES_00=-2.599, HELO_EQ_DE=0.35, J_CHICKENPOX_54=0.6, RCVD_IN_DNSWL_MED=-4]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 66fTgEmpHJ28 for ; Thu, 25 Oct 2012 11:26:29 -0700 (PDT)
Received: from mail-out1.informatik.tu-muenchen.de (mail-out1.informatik.tu-muenchen.de [131.159.0.8]) by ietfa.amsl.com (Postfix) with ESMTP id 86BB121F88A7 for ; Thu, 25 Oct 2012 11:26:29 -0700 (PDT)
Received: from [192.168.2.36] (g229136115.adsl.alicedsl.de [92.229.136.115]) by mail.net.in.tum.de (Postfix) with ESMTPSA id 3A9A419110DB; Thu, 25 Oct 2012 20:26:25 +0200 (CEST)
Message-ID: <50898454.2000706@net.in.tum.de>
Date: Thu, 25 Oct 2012 20:26:28 +0200
From: Gerhard Muenz 
User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:16.0) Gecko/20121010 Thunderbird/16.0.1
MIME-Version: 1.0
To: John Court 
References:    <5087B96B.7020500@cisco.com>  <508850F7.2080801@net.in.tum.de> <50885B49.6050603@cisco.com>  <5088666F.1090106@cisco.com> 
In-Reply-To: 
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Cc: ipfix@ietf.org
Subject: Re: [IPFIX] Export of long lived flow information
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: IPFIX WG discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Thu, 25 Oct 2012 18:26:31 -0000

Hi,

It seems that some Information Element descriptions need clarification. 
For example, I think that the description of the Information Elements 
flowStart* and flowEnd* should be clarified:

OLD:
The absolute timestamp of the first|last packet of this Flow.
NEW:
The absolute timestamp of the first|last packet accounted in this Flow 
Record.

I assume that this is how it is implemented in most existing 
implementations. Also, I assume that this is the intended meaning.

My understanding is that both, deltaCounts and totalCounts contain the 
number of packets or octets observed in the indicated time interval. So, 
for identical flowStart* and flowEnd* timestamps, the values are the same.

However, the description of totalCounts says that you report the number 
of packets or octets observed for this Flow since re-initialization. So, 
you must never reset the counter for this Flow, even after observing a 
FIN or RST.
If you reset flow counters, or if you remove Flows from your Cache, you 
cannot use totalCounts any more unless you re-initialize the Metering 
Process (e.g. after flushing the entire permanent Cache).

Using totalCounts, the flowStart* timestamp is identical in all Flow 
Records of the same Flow. Also, the collector knows that - for all but 
the first Flow Record of a Flow - the totalCount values include packets 
which were already reported in earlier Flow Records for the same Flow. 
Hence, each new Flow Record of the same Flow is an update of the 
previous ones. Summing up totalCount values in these Flow Records 
results in duplicate counts.
On the other hand, with deltaCounts, the Flow Records refer to distinct 
time intervals. So, you can sum up counters without having duplicates.

Although these subtle differences are not very obvious, the Information 
Element descriptions are quite clear. flowEndReason can be used to 
report some extra information but is not needed to understand the 
meaning of the Flow Records.

Thanks,
Gerhard


On 25.10.2012 03:04, John Court wrote:
> Yep I think everyone is starting to see the ambiguity that needs to be
> cleared up :-)
>
> Paul,
>
> *Definitely. If it's a permanent flow and you're exporting totalCount
> fields - which are measured "since the Metering Process
> (re-)initialization for this Observation Point" - then the flowStartTime
> must surely be the time the first ever packet was observed.*
>
> If you take that literally shouldn't that be interpreted to mean that
> the totalCount continues into the next time a connection is up between
> the same flow key ?  Even if a flowEndReason of :
>
>   0x03: end of Flow detected
> The Flow was terminated because the Metering Process
> detected signals indicating the end of the Flow, for
> example, the TCP FIN flag.
>
> That clearly wouldn't be of much use IMO and makes it difficult to see
> what the flowEndReason field semantics mean in that context.  Just
> pointing out that taking that definition literally doesn't give a useful
> answer on its own either :-).  Although maybe that does make sense in a
> router context ?  Can you clarify this some more, perhaps you never
> intend using the flowEndReason IE in your case ?
>
> Thanks
>
>
>
>
>
> From: Paul Aitken 
> To: Andrew Johnson ,
> Cc: Gerhard Muenz , John Court/Australia/IBM@IBMAU,
> ipfix@ietf.org
> Date: 25/10/2012 08:07
> Subject: Re: [IPFIX] Export of long lived flow information
> ------------------------------------------------------------------------
>
>
>
> Andrew,
>
>  > I was thinking that a mechanism that allowed a non-permanent flow to
> be exported multiple time would be useful.  For example, security
> applications generally want to know about a new flow ASAP, so they can
> act on the information, but a short active timeout values lead to using
> more export bandwidth.  I was thinking we could do something like export
> a report of the flow after the first packet, and then export the final
> version of the flow once the normal timeouts had decided it was over.
>
> I have in the past discussed the idea of exporting a "new flow alert"
> using zero-valued counters in order to make the collector aware that
> we've started monitoring it - so I'm claiming prior art on that.
>
>
>  > I had in mind something like using a delta count, followed by a total
> count.  Reading the below definition of Total counts though, I'm not
> sure that will work, but I think it depends on how we interpret the
> definition of "Flow".  If two records have matching key fields but
> different starting timestamps, are they the same Flow?
>
> 5101 defines:
>
>        A Flow is defined as a set of IP packets passing an Observation
>        Point in the network during a certain time interval.
>
>
> - so it's all about the timestamps :-)
>
>
>  > I would argue that a single Flow can't have two flowStartTimes, so
> maybe not.
>
> However, two flows with different flowStartTimes can be merged into one
> flow.
>
>
>  > This would mean that we shouldn't reset the flowStartTimes between
> sending reports for the same permanent Flow.
>
> Definitely. If it's a permanent flow and you're exporting totalCount
> fields - which are measured "since the Metering Process
> (re-)initialization for this Observation Point" - then the flowStartTime
> must surely be the time the first ever packet was observed.
>
> P.
>
>

From johnwcrt@au1.ibm.com  Thu Oct 25 14:42:06 2012
Return-Path: 
X-Original-To: ipfix@ietfa.amsl.com
Delivered-To: ipfix@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4528521F860F for ; Thu, 25 Oct 2012 14:42:06 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -6.965
X-Spam-Level: 
X-Spam-Status: No, score=-6.965 tagged_above=-999 required=5 tests=[AWL=-0.967, BAYES_00=-2.599, HTML_MESSAGE=0.001, J_CHICKENPOX_54=0.6, RCVD_IN_DNSWL_MED=-4]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id io7amCp4B+Gj for ; Thu, 25 Oct 2012 14:42:05 -0700 (PDT)
Received: from e23smtp05.au.ibm.com (e23smtp05.au.ibm.com [202.81.31.147]) by ietfa.amsl.com (Postfix) with ESMTP id 177A721F867C for ; Thu, 25 Oct 2012 14:42:03 -0700 (PDT)
Received: from /spool/local by e23smtp05.au.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for  from ; Fri, 26 Oct 2012 07:40:06 +1000
Received: from d23relay04.au.ibm.com (202.81.31.246) by e23smtp05.au.ibm.com (202.81.31.211) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted;  Fri, 26 Oct 2012 07:40:04 +1000
Received: from d23av03.au.ibm.com (d23av03.au.ibm.com [9.190.234.97]) by d23relay04.au.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id q9PLVogN66191384 for ; Fri, 26 Oct 2012 08:31:51 +1100
Received: from d23av03.au.ibm.com (loopback [127.0.0.1]) by d23av03.au.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id q9PLfncm029159 for ; Fri, 26 Oct 2012 08:41:49 +1100
Received: from d23mlc03.au.ibm.com (d23mlc03.au.ibm.com [9.190.26.210]) by d23av03.au.ibm.com (8.14.4/8.13.1/NCO v10.0 AVin) with ESMTP id q9PLfnUd029156; Fri, 26 Oct 2012 08:41:49 +1100
In-Reply-To: <50898454.2000706@net.in.tum.de>
References:    <5087B96B.7020500@cisco.com>  <508850F7.2080801@net.in.tum.de> <50885B49.6050603@cisco.com>  <5088666F.1090106@cisco.com>  <50898454.2000706@net.in.tum.de>
To: Gerhard Muenz 
MIME-Version: 1.0
X-KeepSent: 9FA62110:C222915A-CA257AA2:00764D83; type=4; name=$KeepSent
X-Mailer: Lotus Notes Release 8.5.3 September 15, 2011
Message-ID: 
From: John Court 
Date: Fri, 26 Oct 2012 07:40:56 +1000
X-MIMETrack: Serialize by Router on d23mlc03/23/M/IBM(Release 8.5.3FP2HF29 | July 24, 2012) at 26/10/2012 08:41:06, Serialize complete at 26/10/2012 08:41:06
Content-Type: multipart/alternative; boundary="=_alternative 00772B6D4A257AA2_="
x-cbid: 12102521-1396-0000-0000-0000020B7E6F
Cc: ipfix@ietf.org
Subject: Re: [IPFIX] Export of long lived flow information
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: IPFIX WG discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Thu, 25 Oct 2012 21:42:06 -0000

This is a multipart message in MIME format.
--=_alternative 00772B6D4A257AA2_=
Content-Type: text/plain; charset="US-ASCII"

>>OLD:
>>The absolute timestamp of the first|last packet of this Flow.
>>NEW:
>>The absolute timestamp of the first|last packet accounted in this Flow 
>>Record.

For some reason that does make it much clearer to me although that could 
just be because I have had the benefit of all the preceding discussion.

I think that clears up exactly what the statistics represent in a flow 
record.  The only nastiness is that we use terminology of flowEnd* when 
really if you are using totals, conceptually the flow really ISN'T ending 
its just being "Reported".  I am not suggesting changes to anything just 
hopefully making it clear why the mis-interpretations may occur.

For my own purposes its clear I can only use deltas as my work is at the 
application level and only bridges and routers would seem to make sense 
for the "total" counters concept given your definition. For me the 
aggregation into "totals" is up to the collector.

Again thanks for every ones patience it has certainly helped further my 
education and hopefully improve future interoperability of IPFIX devices 
:-)

John Court




From:   Gerhard Muenz 
To:     John Court/Australia/IBM@IBMAU, 
Cc:     Paul Aitken , Andrew Johnson 
, ipfix@ietf.org
Date:   26/10/2012 04:27
Subject:        Re: [IPFIX] Export of long lived flow information




Hi,

It seems that some Information Element descriptions need clarification. 
For example, I think that the description of the Information Elements 
flowStart* and flowEnd* should be clarified:

OLD:
The absolute timestamp of the first|last packet of this Flow.
NEW:
The absolute timestamp of the first|last packet accounted in this Flow 
Record.

I assume that this is how it is implemented in most existing 
implementations. Also, I assume that this is the intended meaning.

My understanding is that both, deltaCounts and totalCounts contain the 
number of packets or octets observed in the indicated time interval. So, 
for identical flowStart* and flowEnd* timestamps, the values are the same.

However, the description of totalCounts says that you report the number 
of packets or octets observed for this Flow since re-initialization. So, 
you must never reset the counter for this Flow, even after observing a 
FIN or RST.
If you reset flow counters, or if you remove Flows from your Cache, you 
cannot use totalCounts any more unless you re-initialize the Metering 
Process (e.g. after flushing the entire permanent Cache).

Using totalCounts, the flowStart* timestamp is identical in all Flow 
Records of the same Flow. Also, the collector knows that - for all but 
the first Flow Record of a Flow - the totalCount values include packets 
which were already reported in earlier Flow Records for the same Flow. 
Hence, each new Flow Record of the same Flow is an update of the 
previous ones. Summing up totalCount values in these Flow Records 
results in duplicate counts.
On the other hand, with deltaCounts, the Flow Records refer to distinct 
time intervals. So, you can sum up counters without having duplicates.

Although these subtle differences are not very obvious, the Information 
Element descriptions are quite clear. flowEndReason can be used to 
report some extra information but is not needed to understand the 
meaning of the Flow Records.

Thanks,
Gerhard


On 25.10.2012 03:04, John Court wrote:
> Yep I think everyone is starting to see the ambiguity that needs to be
> cleared up :-)
>
> Paul,
>
> *Definitely. If it's a permanent flow and you're exporting totalCount
> fields - which are measured "since the Metering Process
> (re-)initialization for this Observation Point" - then the flowStartTime
> must surely be the time the first ever packet was observed.*
>
> If you take that literally shouldn't that be interpreted to mean that
> the totalCount continues into the next time a connection is up between
> the same flow key ?  Even if a flowEndReason of :
>
>   0x03: end of Flow detected
> The Flow was terminated because the Metering Process
> detected signals indicating the end of the Flow, for
> example, the TCP FIN flag.
>
> That clearly wouldn't be of much use IMO and makes it difficult to see
> what the flowEndReason field semantics mean in that context.  Just
> pointing out that taking that definition literally doesn't give a useful
> answer on its own either :-).  Although maybe that does make sense in a
> router context ?  Can you clarify this some more, perhaps you never
> intend using the flowEndReason IE in your case ?
>
> Thanks
>
>
>
>
>
> From: Paul Aitken 
> To: Andrew Johnson ,
> Cc: Gerhard Muenz , John Court/Australia/IBM@IBMAU,
> ipfix@ietf.org
> Date: 25/10/2012 08:07
> Subject: Re: [IPFIX] Export of long lived flow information
> ------------------------------------------------------------------------
>
>
>
> Andrew,
>
>  > I was thinking that a mechanism that allowed a non-permanent flow to
> be exported multiple time would be useful.  For example, security
> applications generally want to know about a new flow ASAP, so they can
> act on the information, but a short active timeout values lead to using
> more export bandwidth.  I was thinking we could do something like export
> a report of the flow after the first packet, and then export the final
> version of the flow once the normal timeouts had decided it was over.
>
> I have in the past discussed the idea of exporting a "new flow alert"
> using zero-valued counters in order to make the collector aware that
> we've started monitoring it - so I'm claiming prior art on that.
>
>
>  > I had in mind something like using a delta count, followed by a total
> count.  Reading the below definition of Total counts though, I'm not
> sure that will work, but I think it depends on how we interpret the
> definition of "Flow".  If two records have matching key fields but
> different starting timestamps, are they the same Flow?
>
> 5101 defines:
>
>        A Flow is defined as a set of IP packets passing an Observation
>        Point in the network during a certain time interval.
>
>
> - so it's all about the timestamps :-)
>
>
>  > I would argue that a single Flow can't have two flowStartTimes, so
> maybe not.
>
> However, two flows with different flowStartTimes can be merged into one
> flow.
>
>
>  > This would mean that we shouldn't reset the flowStartTimes between
> sending reports for the same permanent Flow.
>
> Definitely. If it's a permanent flow and you're exporting totalCount
> fields - which are measured "since the Metering Process
> (re-)initialization for this Observation Point" - then the flowStartTime
> must surely be the time the first ever packet was observed.
>
> P.
>
>



--=_alternative 00772B6D4A257AA2_=
Content-Type: text/html; charset="US-ASCII"

>>OLD:

>>The absolute timestamp of the first|last packet of this Flow.

>>NEW:

>>The absolute timestamp of the first|last packet accounted in this
Flow 

>>Record.



For some reason that does make it much clearer to
me although that could just be because I have had the benefit of all the
preceding discussion.



I think that clears up exactly what the statistics
represent in a flow record.  The only nastiness is that we use terminology
of flowEnd* when really if you are using totals, conceptually the flow
really ISN'T ending its just being "Reported".  I am not
suggesting changes to anything just hopefully making it clear why the mis-interpretations
may occur.



For my own purposes its clear I can only use deltas
as my work is at the application level and only bridges and routers would
seem to make sense for the "total" counters concept given your
definition. For me the aggregation into "totals" is up to the
collector.



Again thanks for every ones patience it has certainly
helped further my education and hopefully improve future interoperability
of IPFIX devices :-)



John Court









From:      
 Gerhard Muenz <muenz@net.in.tum.de>

To:      
 John Court/Australia/IBM@IBMAU,


Cc:      
 Paul Aitken <paitken@cisco.com>,
Andrew Johnson <andrjohn@cisco.com>, ipfix@ietf.org

Date:      
 26/10/2012 04:27

Subject:    
   Re: [IPFIX]
Export of long lived flow information











Hi,



It seems that some Information Element descriptions need clarification.


For example, I think that the description of the Information Elements 

flowStart* and flowEnd* should be clarified:



OLD:

The absolute timestamp of the first|last packet of this Flow.

NEW:

The absolute timestamp of the first|last packet accounted in this Flow


Record.



I assume that this is how it is implemented in most existing 

implementations. Also, I assume that this is the intended meaning.



My understanding is that both, deltaCounts and totalCounts contain the


number of packets or octets observed in the indicated time interval. So,


for identical flowStart* and flowEnd* timestamps, the values are the same.



However, the description of totalCounts says that you report the number


of packets or octets observed for this Flow since re-initialization. So,


you must never reset the counter for this Flow, even after observing a


FIN or RST.

If you reset flow counters, or if you remove Flows from your Cache, you


cannot use totalCounts any more unless you re-initialize the Metering 

Process (e.g. after flushing the entire permanent Cache).



Using totalCounts, the flowStart* timestamp is identical in all Flow 

Records of the same Flow. Also, the collector knows that - for all but


the first Flow Record of a Flow - the totalCount values include packets


which were already reported in earlier Flow Records for the same Flow.


Hence, each new Flow Record of the same Flow is an update of the 

previous ones. Summing up totalCount values in these Flow Records 

results in duplicate counts.

On the other hand, with deltaCounts, the Flow Records refer to distinct


time intervals. So, you can sum up counters without having duplicates.



Although these subtle differences are not very obvious, the Information


Element descriptions are quite clear. flowEndReason can be used to 

report some extra information but is not needed to understand the 

meaning of the Flow Records.



Thanks,

Gerhard





On 25.10.2012 03:04, John Court wrote:

> Yep I think everyone is starting to see the ambiguity that needs to
be

> cleared up :-)

>

> Paul,

>

> *Definitely. If it's a permanent flow and you're exporting totalCount

> fields - which are measured "since the Metering Process

> (re-)initialization for this Observation Point" - then the flowStartTime

> must surely be the time the first ever packet was observed.*

>

> If you take that literally shouldn't that be interpreted to mean that

> the totalCount continues into the next time a connection is up between

> the same flow key ?  Even if a flowEndReason of :

>

>   0x03: end of Flow detected

> The Flow was terminated because the Metering Process

> detected signals indicating the end of the Flow, for

> example, the TCP FIN flag.

>

> That clearly wouldn't be of much use IMO and makes it difficult to
see

> what the flowEndReason field semantics mean in that context.  Just

> pointing out that taking that definition literally doesn't give a
useful

> answer on its own either :-).  Although maybe that does make
sense in a

> router context ?  Can you clarify this some more, perhaps you
never

> intend using the flowEndReason IE in your case ?

>

> Thanks

>

>

>

>

>

> From: Paul Aitken <paitken@cisco.com>

> To: Andrew Johnson <andrjohn@cisco.com>,

> Cc: Gerhard Muenz <muenz@net.in.tum.de>, John Court/Australia/IBM@IBMAU,

> ipfix@ietf.org

> Date: 25/10/2012 08:07

> Subject: Re: [IPFIX] Export of long lived flow information

> ------------------------------------------------------------------------

>

>

>

> Andrew,

>

>  > I was thinking that a mechanism that allowed a non-permanent
flow to

> be exported multiple time would be useful.  For example, security

> applications generally want to know about a new flow ASAP, so they
can

> act on the information, but a short active timeout values lead to
using

> more export bandwidth.  I was thinking we could do something
like export

> a report of the flow after the first packet, and then export the final

> version of the flow once the normal timeouts had decided it was over.

>

> I have in the past discussed the idea of exporting a "new flow
alert"

> using zero-valued counters in order to make the collector aware that

> we've started monitoring it - so I'm claiming prior art on that.

>

>

>  > I had in mind something like using a delta count, followed
by a total

> count.  Reading the below definition of Total counts though,
I'm not

> sure that will work, but I think it depends on how we interpret the

> definition of "Flow".  If two records have matching
key fields but

> different starting timestamps, are they the same Flow?

>

> 5101 defines:

>

>        A Flow is defined as a set of IP packets
passing an Observation

>        Point in the network during a certain time
interval.

>

>

> - so it's all about the timestamps :-)

>

>

>  > I would argue that a single Flow can't have two flowStartTimes,
so

> maybe not.

>

> However, two flows with different flowStartTimes can be merged into
one

> flow.

>

>

>  > This would mean that we shouldn't reset the flowStartTimes
between

> sending reports for the same permanent Flow.

>

> Definitely. If it's a permanent flow and you're exporting totalCount

> fields - which are measured "since the Metering Process

> (re-)initialization for this Observation Point" - then the flowStartTime

> must surely be the time the first ever packet was observed.

>

> P.

>

>






--=_alternative 00772B6D4A257AA2_=--


From paitken@cisco.com  Thu Oct 25 15:56:16 2012
Return-Path: 
X-Original-To: ipfix@ietfa.amsl.com
Delivered-To: ipfix@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8AC3B21F85C2 for ; Thu, 25 Oct 2012 15:56:16 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -10.236
X-Spam-Level: 
X-Spam-Status: No, score=-10.236 tagged_above=-999 required=5 tests=[AWL=-0.237, BAYES_00=-2.599, J_CHICKENPOX_54=0.6, RCVD_IN_DNSWL_HI=-8]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nov3kChWxksz for ; Thu, 25 Oct 2012 15:56:16 -0700 (PDT)
Received: from ams-iport-1.cisco.com (ams-iport-1.cisco.com [144.254.224.140]) by ietfa.amsl.com (Postfix) with ESMTP id C3A3421F887A for ; Thu, 25 Oct 2012 15:56:15 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=566; q=dns/txt; s=iport; t=1351205775; x=1352415375; h=message-id:date:from:mime-version:to:cc:subject: references:in-reply-to:content-transfer-encoding; bh=PWls+efZa2sbyD+BSNTLrZGOp8iHZ+TbNH24F11Ykro=; b=IxtNQOjbnJIqAnmjSdpZIhMHF9KtIJRx/2F8DeVBqDgX1z4oeOvgZoHW kpD32qlX5wSVj6NDqb8+uss1rsu811FigAQpl8Yd1kTgMedqhBp2L238T hn2YhpCcekJz1FhlNDfSYGa80vczBhwD+W76a3EG41Fs165vCUag5ZWN5 c=;
X-IronPort-AV: E=Sophos;i="4.80,650,1344211200"; d="scan'208";a="145833945"
Received: from ams-core-1.cisco.com ([144.254.72.81]) by ams-iport-1.cisco.com with ESMTP; 25 Oct 2012 22:56:14 +0000
Received: from [10.55.94.191] (dhcp-10-55-94-191.cisco.com [10.55.94.191]) by ams-core-1.cisco.com (8.14.5/8.14.5) with ESMTP id q9PMuErt005700; Thu, 25 Oct 2012 22:56:14 GMT
Message-ID: <5089C38E.6090603@cisco.com>
Date: Thu, 25 Oct 2012 23:56:14 +0100
From: Paul Aitken 
User-Agent: Mozilla/5.0 (X11; Linux i686; rv:15.0) Gecko/20120912 Thunderbird/15.0.1
MIME-Version: 1.0
To: Gerhard Muenz 
References:    <5087B96B.7020500@cisco.com>  <508850F7.2080801@net.in.tum.de> <50885B49.6050603@cisco.com>  <5088666F.1090106@cisco.com>  <50898454.2000706@net.in.tum.de>
In-Reply-To: <50898454.2000706@net.in.tum.de>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Cc: John Court , ipfix@ietf.org
Subject: Re: [IPFIX] Export of long lived flow information
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: IPFIX WG discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Thu, 25 Oct 2012 22:56:16 -0000

Gerhard,

If everyone's happy with this clarification, then we should ask IANA to 
update the descriptions in the IPFIX registry.

P.


On 25/10/12 19:26, Gerhard Muenz wrote:
>
> Hi,
>
> It seems that some Information Element descriptions need 
> clarification. For example, I think that the description of the 
> Information Elements flowStart* and flowEnd* should be clarified:
>
> OLD:
> The absolute timestamp of the first|last packet of this Flow.
> NEW:
> The absolute timestamp of the first|last packet accounted in this Flow 
> Record.


From andrewf@plixer.com  Thu Oct 25 17:16:03 2012
Return-Path: 
X-Original-To: ipfix@ietfa.amsl.com
Delivered-To: ipfix@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1946B21F88CB for ; Thu, 25 Oct 2012 17:16:03 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.999
X-Spam-Level: 
X-Spam-Status: No, score=-1.999 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, J_CHICKENPOX_54=0.6]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id EU6tR6BaaRdh for ; Thu, 25 Oct 2012 17:16:02 -0700 (PDT)
Received: from smtp.plixer.com (smtp.plixer.com [66.186.184.193]) by ietfa.amsl.com (Postfix) with ESMTP id 1D96421F88B7 for ; Thu, 25 Oct 2012 17:16:01 -0700 (PDT)
Received: from [192.168.1.37] ([24.34.46.175]) by smtp.plixer.com over TLS secured channel with Microsoft SMTPSVC(6.0.3790.4675);  Thu, 25 Oct 2012 20:16:00 -0400
User-Agent: Microsoft-MacOutlook/14.2.4.120824
Date: Thu, 25 Oct 2012 20:15:50 -0400
From: Andrew Feren 
To: Paul Aitken , Gerhard Muenz 
Message-ID: 
Thread-Topic: [IPFIX] Export of long lived flow information
In-Reply-To: <5089C38E.6090603@cisco.com>
Mime-version: 1.0
Content-type: text/plain; charset="US-ASCII"
Content-transfer-encoding: 7bit
X-OriginalArrivalTime: 26 Oct 2012 00:16:00.0618 (UTC) FILETIME=[14116CA0:01CDB30F]
Cc: John Court , ipfix@ietf.org
Subject: Re: [IPFIX] Export of long lived flow information
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: IPFIX WG discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Fri, 26 Oct 2012 00:16:03 -0000

+1
I have tripped over that wording a few times in the past.  The change from
Flow to  Flow Record is an improvement.


-Andrew


On 10/25/12 6:56 PM, "Paul Aitken"  wrote:

>Gerhard,
>
>If everyone's happy with this clarification, then we should ask IANA to
>update the descriptions in the IPFIX registry.
>
>P.
>
>
>On 25/10/12 19:26, Gerhard Muenz wrote:
>>
>> Hi,
>>
>> It seems that some Information Element descriptions need
>> clarification. For example, I think that the description of the
>> Information Elements flowStart* and flowEnd* should be clarified:
>>
>> OLD:
>> The absolute timestamp of the first|last packet of this Flow.
>> NEW:
>> The absolute timestamp of the first|last packet accounted in this Flow
>> Record.
>
>_______________________________________________
>IPFIX mailing list
>IPFIX@ietf.org
>https://www.ietf.org/mailman/listinfo/ipfix



From trammell@tik.ee.ethz.ch  Fri Oct 26 00:27:26 2012
Return-Path: 
X-Original-To: ipfix@ietfa.amsl.com
Delivered-To: ipfix@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9676F21F8413 for ; Fri, 26 Oct 2012 00:27:26 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -6.353
X-Spam-Level: 
X-Spam-Status: No, score=-6.353 tagged_above=-999 required=5 tests=[AWL=-0.354, BAYES_00=-2.599, J_CHICKENPOX_54=0.6, RCVD_IN_DNSWL_MED=-4]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rztX75spavLy for ; Fri, 26 Oct 2012 00:27:26 -0700 (PDT)
Received: from smtp.ee.ethz.ch (smtp.ee.ethz.ch [129.132.2.219]) by ietfa.amsl.com (Postfix) with ESMTP id 0723D21F81FF for ; Fri, 26 Oct 2012 00:27:25 -0700 (PDT)
Received: from localhost (localhost [127.0.0.1]) by smtp.ee.ethz.ch (Postfix) with ESMTP id 64FB7D9314; Fri, 26 Oct 2012 09:27:24 +0200 (MEST)
X-Virus-Scanned: by amavisd-new on smtp.ee.ethz.ch
Received: from smtp.ee.ethz.ch ([127.0.0.1]) by localhost (.ee.ethz.ch [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 8hDrGVREJcq7; Fri, 26 Oct 2012 09:27:24 +0200 (MEST)
Received: from [10.0.27.100] (cust-integra-121-161.antanet.ch [80.75.121.161]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: briant) by smtp.ee.ethz.ch (Postfix) with ESMTPSA id 00660D930B; Fri, 26 Oct 2012 09:27:23 +0200 (MEST)
Mime-Version: 1.0 (Apple Message framework v1283)
Content-Type: text/plain; charset=us-ascii
From: Brian Trammell 
In-Reply-To: 
Date: Fri, 26 Oct 2012 09:27:22 +0200
Content-Transfer-Encoding: 7bit
Message-Id: <49994E10-6179-4FB7-AD6A-7F7B06178D93@tik.ee.ethz.ch>
References: 
To: Andrew Feren 
X-Mailer: Apple Mail (2.1283)
Cc: John Court , ipfix@ietf.org
Subject: Re: [IPFIX] Export of long lived flow information
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: IPFIX WG discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Fri, 26 Oct 2012 07:27:26 -0000

Another +1.

Cheers, B

On Oct 26, 2012, at 2:15 AM, Andrew Feren wrote:

> +1
> I have tripped over that wording a few times in the past.  The change from
> Flow to  Flow Record is an improvement.
> 
> 
> -Andrew
> 
> 
> On 10/25/12 6:56 PM, "Paul Aitken"  wrote:
> 
>> Gerhard,
>> 
>> If everyone's happy with this clarification, then we should ask IANA to
>> update the descriptions in the IPFIX registry.
>> 
>> P.
>> 
>> 
>> On 25/10/12 19:26, Gerhard Muenz wrote:
>>> 
>>> Hi,
>>> 
>>> It seems that some Information Element descriptions need
>>> clarification. For example, I think that the description of the
>>> Information Elements flowStart* and flowEnd* should be clarified:
>>> 
>>> OLD:
>>> The absolute timestamp of the first|last packet of this Flow.
>>> NEW:
>>> The absolute timestamp of the first|last packet accounted in this Flow
>>> Record.
>> 
>> _______________________________________________
>> IPFIX mailing list
>> IPFIX@ietf.org
>> https://www.ietf.org/mailman/listinfo/ipfix
> 
> 
> _______________________________________________
> IPFIX mailing list
> IPFIX@ietf.org
> https://www.ietf.org/mailman/listinfo/ipfix


From andrjohn@cisco.com  Fri Oct 26 07:24:58 2012
Return-Path: 
X-Original-To: ipfix@ietfa.amsl.com
Delivered-To: ipfix@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5E12E21F85C0 for ; Fri, 26 Oct 2012 07:24:58 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -10.099
X-Spam-Level: 
X-Spam-Status: No, score=-10.099 tagged_above=-999 required=5 tests=[AWL=-0.101, BAYES_00=-2.599, HTML_MESSAGE=0.001, J_CHICKENPOX_54=0.6, RCVD_IN_DNSWL_HI=-8]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 46wp7IzQHS1z for ; Fri, 26 Oct 2012 07:24:56 -0700 (PDT)
Received: from ams-iport-3.cisco.com (ams-iport-3.cisco.com [144.254.224.146]) by ietfa.amsl.com (Postfix) with ESMTP id A7AA821F85B3 for ; Fri, 26 Oct 2012 07:24:55 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=20823; q=dns/txt; s=iport; t=1351261495; x=1352471095; h=subject:mime-version:from:in-reply-to:date:cc:message-id: references:to; bh=zcJebDkq3tqC7MLwNeIQubQgiIcnUxDF3E9xIG6E7hg=; b=Ch5yCCQlLK2PFlFoNQmFOF/X5aCybgx5/sW+DbM2AuM0McW9GEi06cj/ nHH0aDw9iucrZPfyrq4h4RGqFbJKMwb9TXsYR3IY8v9zcvUGnRZ+T4ReL 3RI/orA6F9AK68Axt7TURwzNmb5wPhcW8AV0o4uP/E5iZOEopox44aysu k=;
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: Av4EADucilCQ/khM/2dsb2JhbABEwj6BCIIeAQEBBBIBZhALEQMBAgEuTwgGExYMh2SdMKAZi3GGDWEDkkGDMo5TgWuCcA
X-IronPort-AV: E=Sophos;i="4.80,654,1344211200"; d="scan'208,217";a="9118152"
Received: from ams-core-3.cisco.com ([144.254.72.76]) by ams-iport-3.cisco.com with ESMTP; 26 Oct 2012 14:24:51 +0000
Received: from cisco.com (mrwint.cisco.com [64.103.70.36]) by ams-core-3.cisco.com (8.14.5/8.14.5) with ESMTP id q9QEOpjS012181 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 26 Oct 2012 14:24:51 GMT
Received: from dhcp-10-147-1-70.cisco.com (dhcp-10-147-1-70.cisco.com [10.147.1.70]) by cisco.com (8.14.4+Sun/8.8.8) with ESMTP id q9QEOlgL010268; Fri, 26 Oct 2012 15:24:48 +0100 (BST)
Mime-Version: 1.0 (Apple Message framework v1278)
Content-Type: multipart/alternative; boundary="Apple-Mail=_55713747-87BF-45AF-8EB2-EE9362C98367"
From: Andrew Johnson 
In-Reply-To: 
Date: Fri, 26 Oct 2012 15:24:42 +0100
Message-Id: <04A22782-E8DE-4FFB-A31C-8DC9FAC8FDDE@cisco.com>
References:    <5087B96B.7020500@cisco.com>  <508850F7.2080801@net.in.tum.de> <50885B49.6050603@cisco.com>  <5088666F.1090106@cisco.com>  <50898454.2000706@net.in.tum.de> 
To: John Court 
X-Mailer: Apple Mail (2.1278)
Cc: ipfix@ietf.org
Subject: Re: [IPFIX] Export of long lived flow information
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: IPFIX WG discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Fri, 26 Oct 2012 14:24:58 -0000

--Apple-Mail=_55713747-87BF-45AF-8EB2-EE9362C98367
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=us-ascii

Hi folks

A flow is defined as a set of packets which all have some common =
properties, and the original idea of a flow was based on the common =
5-tuple of IP address, IP protocol and ports.  A Flow Record is formed =
from any observation of packets belonging to a Flow (not all packets =
within the flow are necessarily observed).

If my PC opens an HTTP connection to some server and downloads a simple =
web page, then we'd expect to see one traditional flows (per direction, =
but ignore that for now).  If I open a new HTTP connection to the same =
server, and coincidentally use the same source port, weeks later, is =
that the same flow?

I had always thought of these as two flows, and the flowEndReason =
implies it, but that would introduce some sort of time property as one =
of the common property shared by the packets that make up a flow.  A =
collector might aggregate the two reports, but removing the time =
property is much like any other form of aggregation.

It seems to me that we're using the timeout values of the cache as a =
sort of ill-defined common property, but things get confusing when we =
export total counts, or because the cache is low on resources, etc.  =
Ideally, we'd want to be able to send more than one Flow Record for the =
same Flow and provide enough information for the Collector to =
reconstruct what the Monitoring Process is using to define a Flow.


Cheers, Andrew



On 25 Oct 2012, at 22:40, John Court wrote:
> >>OLD:
> >>The absolute timestamp of the first|last packet of this Flow.
> >>NEW:
> >>The absolute timestamp of the first|last packet accounted in this =
Flow=20
> >>Record.
>=20
> For some reason that does make it much clearer to me although that =
could just be because I have had the benefit of all the preceding =
discussion.=20
>=20
> I think that clears up exactly what the statistics represent in a flow =
record.  The only nastiness is that we use terminology of flowEnd* when =
really if you are using totals, conceptually the flow really ISN'T =
ending its just being "Reported".  I am not suggesting changes to =
anything just hopefully making it clear why the mis-interpretations may =
occur.=20
>=20
> For my own purposes its clear I can only use deltas as my work is at =
the application level and only bridges and routers would seem to make =
sense for the "total" counters concept given your definition. For me the =
aggregation into "totals" is up to the collector.=20
>=20
> Again thanks for every ones patience it has certainly helped further =
my education and hopefully improve future interoperability of IPFIX =
devices :-)=20
>=20
> John Court=20
>=20
>=20
>=20
>=20
> From:        Gerhard Muenz =20
> To:        John Court/Australia/IBM@IBMAU,=20
> Cc:        Paul Aitken , Andrew Johnson =
, ipfix@ietf.org=20
> Date:        26/10/2012 04:27=20
> Subject:        Re: [IPFIX] Export of long lived flow information=20
>=20
>=20
>=20
>=20
> Hi,
>=20
> It seems that some Information Element descriptions need =
clarification.=20
> For example, I think that the description of the Information Elements=20=

> flowStart* and flowEnd* should be clarified:
>=20
> OLD:
> The absolute timestamp of the first|last packet of this Flow.
> NEW:
> The absolute timestamp of the first|last packet accounted in this Flow=20=

> Record.
>=20
> I assume that this is how it is implemented in most existing=20
> implementations. Also, I assume that this is the intended meaning.
>=20
> My understanding is that both, deltaCounts and totalCounts contain the=20=

> number of packets or octets observed in the indicated time interval. =
So,=20
> for identical flowStart* and flowEnd* timestamps, the values are the =
same.
>=20
> However, the description of totalCounts says that you report the =
number=20
> of packets or octets observed for this Flow since re-initialization. =
So,=20
> you must never reset the counter for this Flow, even after observing a=20=

> FIN or RST.
> If you reset flow counters, or if you remove Flows from your Cache, =
you=20
> cannot use totalCounts any more unless you re-initialize the Metering=20=

> Process (e.g. after flushing the entire permanent Cache).
>=20
> Using totalCounts, the flowStart* timestamp is identical in all Flow=20=

> Records of the same Flow. Also, the collector knows that - for all but=20=

> the first Flow Record of a Flow - the totalCount values include =
packets=20
> which were already reported in earlier Flow Records for the same Flow.=20=

> Hence, each new Flow Record of the same Flow is an update of the=20
> previous ones. Summing up totalCount values in these Flow Records=20
> results in duplicate counts.
> On the other hand, with deltaCounts, the Flow Records refer to =
distinct=20
> time intervals. So, you can sum up counters without having duplicates.
>=20
> Although these subtle differences are not very obvious, the =
Information=20
> Element descriptions are quite clear. flowEndReason can be used to=20
> report some extra information but is not needed to understand the=20
> meaning of the Flow Records.
>=20
> Thanks,
> Gerhard
>=20
>=20
> On 25.10.2012 03:04, John Court wrote:
> > Yep I think everyone is starting to see the ambiguity that needs to =
be
> > cleared up :-)
> >
> > Paul,
> >
> > *Definitely. If it's a permanent flow and you're exporting =
totalCount
> > fields - which are measured "since the Metering Process
> > (re-)initialization for this Observation Point" - then the =
flowStartTime
> > must surely be the time the first ever packet was observed.*
> >
> > If you take that literally shouldn't that be interpreted to mean =
that
> > the totalCount continues into the next time a connection is up =
between
> > the same flow key ?  Even if a flowEndReason of :
> >
> >   0x03: end of Flow detected
> > The Flow was terminated because the Metering Process
> > detected signals indicating the end of the Flow, for
> > example, the TCP FIN flag.
> >
> > That clearly wouldn't be of much use IMO and makes it difficult to =
see
> > what the flowEndReason field semantics mean in that context.  Just
> > pointing out that taking that definition literally doesn't give a =
useful
> > answer on its own either :-).  Although maybe that does make sense =
in a
> > router context ?  Can you clarify this some more, perhaps you never
> > intend using the flowEndReason IE in your case ?
> >
> > Thanks
> >
> >
> >
> >
> >
> > From: Paul Aitken 
> > To: Andrew Johnson ,
> > Cc: Gerhard Muenz , John =
Court/Australia/IBM@IBMAU,
> > ipfix@ietf.org
> > Date: 25/10/2012 08:07
> > Subject: Re: [IPFIX] Export of long lived flow information
> > =
------------------------------------------------------------------------
> >
> >
> >
> > Andrew,
> >
> >  > I was thinking that a mechanism that allowed a non-permanent flow =
to
> > be exported multiple time would be useful.  For example, security
> > applications generally want to know about a new flow ASAP, so they =
can
> > act on the information, but a short active timeout values lead to =
using
> > more export bandwidth.  I was thinking we could do something like =
export
> > a report of the flow after the first packet, and then export the =
final
> > version of the flow once the normal timeouts had decided it was =
over.
> >
> > I have in the past discussed the idea of exporting a "new flow =
alert"
> > using zero-valued counters in order to make the collector aware that
> > we've started monitoring it - so I'm claiming prior art on that.
> >
> >
> >  > I had in mind something like using a delta count, followed by a =
total
> > count.  Reading the below definition of Total counts though, I'm not
> > sure that will work, but I think it depends on how we interpret the
> > definition of "Flow".  If two records have matching key fields but
> > different starting timestamps, are they the same Flow?
> >
> > 5101 defines:
> >
> >        A Flow is defined as a set of IP packets passing an =
Observation
> >        Point in the network during a certain time interval.
> >
> >
> > - so it's all about the timestamps :-)
> >
> >
> >  > I would argue that a single Flow can't have two flowStartTimes, =
so
> > maybe not.
> >
> > However, two flows with different flowStartTimes can be merged into =
one
> > flow.
> >
> >
> >  > This would mean that we shouldn't reset the flowStartTimes =
between
> > sending reports for the same permanent Flow.
> >
> > Definitely. If it's a permanent flow and you're exporting totalCount
> > fields - which are measured "since the Metering Process
> > (re-)initialization for this Observation Point" - then the =
flowStartTime
> > must surely be the time the first ever packet was observed.
> >
> > P.
> >
> >
>=20
>=20


--Apple-Mail=_55713747-87BF-45AF-8EB2-EE9362C98367
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
	charset=us-ascii

>>OLD:

>>The absolute timestamp of the first|last packet of this =
Flow.

>>NEW:

>>The absolute timestamp of the first|last packet accounted in =
this
Flow 

>>Record.



For some reason that does make it much clearer =
to
me although that could just be because I have had the benefit of all the
preceding discussion.



I think that clears up exactly what the =
statistics
represent in a flow record.  The only nastiness is that we use =
terminology
of flowEnd* when really if you are using totals, conceptually the flow
really ISN'T ending its just being "Reported".  I am not
suggesting changes to anything just hopefully making it clear why the =
mis-interpretations
may occur.



For my own purposes its clear I can only use =
deltas
as my work is at the application level and only bridges and routers =
would
seem to make sense for the "total" counters concept given your
definition. For me the aggregation into "totals" is up to the
collector.



Again thanks for every ones patience it has =
certainly
helped further my education and hopefully improve future =
interoperability
of IPFIX devices :-)



John Court









From:   =
   
 Gerhard Muenz <muenz@net.in.tum.de>

To:   =
   
 John =
Court/Australia/IBM@IBMAU,


Cc:   =
   
 Paul Aitken <paitken@cisco.com>,
Andrew Johnson <andrjohn@cisco.com>, ipfix@ietf.org

Date:   =
   
 26/10/2012 =
04:27

Subject: =
   
   Re: [IPFIX]
Export of long lived flow information











Hi,



It seems that some Information Element descriptions need clarification.


For example, I think that the description of the Information Elements =


flowStart* and flowEnd* should be clarified:



OLD:

The absolute timestamp of the first|last packet of this Flow.

NEW:

The absolute timestamp of the first|last packet accounted in this Flow


Record.



I assume that this is how it is implemented in most existing 

implementations. Also, I assume that this is the intended meaning.



My understanding is that both, deltaCounts and totalCounts contain the


number of packets or octets observed in the indicated time interval. So,


for identical flowStart* and flowEnd* timestamps, the values are the =
same.



However, the description of totalCounts says that you report the number


of packets or octets observed for this Flow since re-initialization. So,


you must never reset the counter for this Flow, even after observing a


FIN or RST.

If you reset flow counters, or if you remove Flows from your Cache, you


cannot use totalCounts any more unless you re-initialize the Metering =


Process (e.g. after flushing the entire permanent Cache).



Using totalCounts, the flowStart* timestamp is identical in all Flow =


Records of the same Flow. Also, the collector knows that - for all but


the first Flow Record of a Flow - the totalCount values include packets


which were already reported in earlier Flow Records for the same Flow.


Hence, each new Flow Record of the same Flow is an update of the 

previous ones. Summing up totalCount values in these Flow Records 

results in duplicate counts.

On the other hand, with deltaCounts, the Flow Records refer to distinct


time intervals. So, you can sum up counters without having =
duplicates.



Although these subtle differences are not very obvious, the Information


Element descriptions are quite clear. flowEndReason can be used to 

report some extra information but is not needed to understand the 

meaning of the Flow Records.



Thanks,

Gerhard





On 25.10.2012 03:04, John Court wrote:

> Yep I think everyone is starting to see the ambiguity that needs to
be

> cleared up :-)

>

> Paul,

>

> *Definitely. If it's a permanent flow and you're exporting =
totalCount

> fields - which are measured "since the Metering Process

> (re-)initialization for this Observation Point" - then the =
flowStartTime

> must surely be the time the first ever packet was observed.*

>

> If you take that literally shouldn't that be interpreted to mean =
that

> the totalCount continues into the next time a connection is up =
between

> the same flow key ?  Even if a flowEndReason of :

>

>   0x03: end of Flow detected

> The Flow was terminated because the Metering Process

> detected signals indicating the end of the Flow, for

> example, the TCP FIN flag.

>

> That clearly wouldn't be of much use IMO and makes it difficult to
see

> what the flowEndReason field semantics mean in that context. =
 Just

> pointing out that taking that definition literally doesn't give a
useful

> answer on its own either :-).  Although maybe that does make
sense in a

> router context ?  Can you clarify this some more, perhaps you
never

> intend using the flowEndReason IE in your case ?

>

> Thanks

>

>

>

>

>

> From: Paul Aitken <paitken@cisco.com>

> To: Andrew Johnson <andrjohn@cisco.com>,

> Cc: Gerhard Muenz <muenz@net.in.tum.de>, John =
Court/Australia/IBM@IBMAU,

> ipfix@ietf.org

> Date: 25/10/2012 08:07

> Subject: Re: [IPFIX] Export of long lived flow information

> =
------------------------------------------------------------------------
>

>

>

> Andrew,

>

>  > I was thinking that a mechanism that allowed a =
non-permanent
flow to

> be exported multiple time would be useful.  For example, =
security

> applications generally want to know about a new flow ASAP, so they
can

> act on the information, but a short active timeout values lead to
using

> more export bandwidth.  I was thinking we could do something
like export

> a report of the flow after the first packet, and then export the =
final

> version of the flow once the normal timeouts had decided it was =
over.

>

> I have in the past discussed the idea of exporting a "new flow
alert"

> using zero-valued counters in order to make the collector aware =
that

> we've started monitoring it - so I'm claiming prior art on =
that.

>

>

>  > I had in mind something like using a delta count, =
followed
by a total

> count.  Reading the below definition of Total counts though,
I'm not

> sure that will work, but I think it depends on how we interpret =
the

> definition of "Flow".  If two records have matching
key fields but

> different starting timestamps, are they the same Flow?

>

> 5101 defines:

>

>        A Flow is defined as a set of IP packets
passing an Observation

>        Point in the network during a certain =
time
interval.

>

>

> - so it's all about the timestamps :-)

>

>

>  > I would argue that a single Flow can't have two =
flowStartTimes,
so

> maybe not.

>

> However, two flows with different flowStartTimes can be merged into
one

> flow.

>

>

>  > This would mean that we shouldn't reset the =
flowStartTimes
between

> sending reports for the same permanent Flow.

>

> Definitely. If it's a permanent flow and you're exporting =
totalCount

> fields - which are measured "since the Metering Process

> (re-)initialization for this Observation Point" - then the =
flowStartTime

> must surely be the time the first ever packet was observed.

>

> P.

>

>






=

--Apple-Mail=_55713747-87BF-45AF-8EB2-EE9362C98367--

From andrewf@plixer.com  Fri Oct 26 09:23:25 2012
Return-Path: 
X-Original-To: ipfix@ietfa.amsl.com
Delivered-To: ipfix@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6280E21F8611 for ; Fri, 26 Oct 2012 09:23:25 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.402
X-Spam-Level: 
X-Spam-Status: No, score=-2.402 tagged_above=-999 required=5 tests=[AWL=0.197,  BAYES_00=-2.599]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id oXYQZsnmgjSM for ; Fri, 26 Oct 2012 09:23:25 -0700 (PDT)
Received: from smtp.plixer.com (smtp.plixer.com [66.186.184.193]) by ietfa.amsl.com (Postfix) with ESMTP id C153F21F85A7 for ; Fri, 26 Oct 2012 09:23:24 -0700 (PDT)
Received: from [10.100.1.132] ([10.100.1.132]) by smtp.plixer.com over TLS secured channel with Microsoft SMTPSVC(6.0.3790.4675);  Fri, 26 Oct 2012 12:23:23 -0400
Message-ID: <508AB8FB.3060807@plixer.com>
Date: Fri, 26 Oct 2012 12:23:23 -0400
From: Andrew Feren 
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:19.0) Gecko/19.0 Thunderbird/19.0a1
MIME-Version: 1.0
To: Gerhard Muenz 
References:    <5087B96B.7020500@cisco.com>  <508850F7.2080801@net.in.tum.de> <50885B49.6050603@cisco.com>  <5088666F.1090106@cisco.com>  <50898454.2000706@net.in.tum.de>
In-Reply-To: <50898454.2000706@net.in.tum.de>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
X-OriginalArrivalTime: 26 Oct 2012 16:23:23.0577 (UTC) FILETIME=[385E1A90:01CDB396]
Cc: John Court , ipfix@ietf.org
Subject: Re: [IPFIX] Export of long lived flow information
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: IPFIX WG discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Fri, 26 Oct 2012 16:23:25 -0000

Hi Gerhard,

On 10/25/2012 02:26 PM, Gerhard Muenz wrote:
>
> Hi,
>
[ snip ]
>
> My understanding is that both, deltaCounts and totalCounts contain the 
> number of packets or octets observed in the indicated time interval. 
> So, for identical flowStart* and flowEnd* timestamps, the values are 
> the same.
This is my understanding as well.
>
> However, the description of totalCounts says that you report the 
> number of packets or octets observed for this Flow since 
> re-initialization. So, you must never reset the counter for this Flow, 
> even after observing a FIN or RST.
> If you reset flow counters, or if you remove Flows from your Cache, 
> you cannot use totalCounts any more unless you re-initialize the 
> Metering Process (e.g. after flushing the entire permanent Cache).

I can try some tests later, but from what I have seen (and been told) 
many totals being exported are in fact just a delta sent once at the end 
of the flow.  If a later flow had the same IPs, protocol, and ports as 
an earlier flow I'm pretty sure a new start time will be sent rather 
than the the first time that flow was seen since reinitializing the 
metering process.  Or to put it an other way I think deltas are being 
sent, but called totals by the implementation because it seemed like the 
right thing to do for a value being sent once at the end of the flow.

I suspect that totals reporting on the export process (eg 
exportedOctetTotalCount, exportedMessageTotalCount) are, however, 
reported with a start time that is only reset on reinitialization.

-Andrew

From paitken@cisco.com  Fri Oct 26 12:20:56 2012
Return-Path: 
X-Original-To: ipfix@ietfa.amsl.com
Delivered-To: ipfix@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 49EF521F863F for ; Fri, 26 Oct 2012 12:20:56 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -10.524
X-Spam-Level: 
X-Spam-Status: No, score=-10.524 tagged_above=-999 required=5 tests=[AWL=0.075, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id UWQJpU0CpeTN for ; Fri, 26 Oct 2012 12:20:55 -0700 (PDT)
Received: from ams-iport-4.cisco.com (ams-iport-4.cisco.com [144.254.224.147]) by ietfa.amsl.com (Postfix) with ESMTP id 1A64821F860A for ; Fri, 26 Oct 2012 12:20:52 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=2780; q=dns/txt; s=iport; t=1351279253; x=1352488853; h=message-id:date:from:mime-version:to:cc:subject: references:in-reply-to:content-transfer-encoding; bh=wxR6xrUOClD/o/D3c1lzIAAeTcm3eQb8tl/T7alHWoA=; b=acQ45xinbdugSonxZG0bndAGrG2yaiyxOFA7T7DbTCM72Es61lE+iB2B iPE8Q2d0m9/+6FK28Dct1fX0TF4ldjAEsoGL8MqLcqqw4IzOIuJRp4piI DIOui9b9wnQ0VlD31rUpLWbdLcYw5Eg08H62jKFQGaXSgEoGmYKKKPIaP A=;
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: Av8EAPrhilCQ/khL/2dsb2JhbABEwlaBCIIeAQEBAwESASVAAQULCyEWDwkDAgECAUUGDQEHAQEeh14GnGugE5JfA5VzhWaIbYFrgnA
X-IronPort-AV: E=Sophos;i="4.80,654,1344211200";  d="scan'208";a="9135825"
Received: from ams-core-2.cisco.com ([144.254.72.75]) by ams-iport-4.cisco.com with ESMTP; 26 Oct 2012 19:20:51 +0000
Received: from [10.55.82.156] (dhcp-10-55-82-156.cisco.com [10.55.82.156]) by ams-core-2.cisco.com (8.14.5/8.14.5) with ESMTP id q9QJKppm004622; Fri, 26 Oct 2012 19:20:51 GMT
Message-ID: <508AE290.3020902@cisco.com>
Date: Fri, 26 Oct 2012 20:20:48 +0100
From: Paul Aitken 
User-Agent: Mozilla/5.0 (X11; Linux i686; rv:15.0) Gecko/20120912 Thunderbird/15.0.1
MIME-Version: 1.0
To: Andrew Feren 
References:    <5087B96B.7020500@cisco.com>  <508850F7.2080801@net.in.tum.de> <50885B49.6050603@cisco.com>  <5088666F.1090106@cisco.com>  <50898454.2000706@net.in.tum.de> <508AB8FB.3060807@plixer.com>
In-Reply-To: <508AB8FB.3060807@plixer.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Cc: John Court , ipfix@ietf.org
Subject: Re: [IPFIX] Export of long lived flow information
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: IPFIX WG discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Fri, 26 Oct 2012 19:20:56 -0000

Andrew, Gerhard,

>> My understanding is that both, deltaCounts and totalCounts contain 
>> the number of packets or octets observed in the indicated time 
>> interval. So, for identical flowStart* and flowEnd* timestamps, the 
>> values are the same.
> This is my understanding as well.

There has to be a difference between delta and total counts, else we 
wouldn't have both of them!

Suppose we have a permanent cache, so the cache entries never expire.

For a new flow starting at t0 with a first export at t1, the timestamps, 
delta, and total counts are the same.

However with the second export at t2, the total and delta counts are 
different although their timestamps match (they'll both say, "t0 to t2").

With the traditional (non-permanent) cache, the entry would probably 
have been removed at t1 and re-created on a subsequent packet, so at t2 
the delta and total counts would both be equal. However it'd be 
incorrect to report the total count, because that's defined as the total 
number of packets or bytes ..."since the Metering Process 
(re-)initialization for this Observation Point".


>> However, the description of totalCounts says that you report the 
>> number of packets or octets observed for this Flow since 
>> re-initialization. So, you must never reset the counter for this 
>> Flow, even after observing a FIN or RST.
>> If you reset flow counters, or if you remove Flows from your Cache, 
>> you cannot use totalCounts any more unless you re-initialize the 
>> Metering Process (e.g. after flushing the entire permanent Cache).
>
> I can try some tests later, but from what I have seen (and been told) 
> many totals being exported are in fact just a delta sent once at the 
> end of the flow.  If a later flow had the same IPs, protocol, and 
> ports as an earlier flow I'm pretty sure a new start time will be sent 
> rather than the the first time that flow was seen since reinitializing 
> the metering process.

So the MP uses a traditional (non-permanent) cache. In RFC 6728 terms, a 
TimeoutCache or NaturalCache rather than a PermanentCache.


> Or to put it an other way I think deltas are being sent, but called 
> totals by the implementation because it seemed like the right thing to 
> do for a value being sent once at the end of the flow.

The collector could be aggregating deltas to keep running totals.


> I suspect that totals reporting on the export process (eg 
> exportedOctetTotalCount, exportedMessageTotalCount) are, however, 
> reported with a start time that is only reset on reinitialization.

Definitely, because these are defined as "The total number of X that the 
Exporting Process has sent since the Exporting Process 
(re-)initialization ...".

P.


From paitken@cisco.com  Fri Oct 26 13:21:19 2012
Return-Path: 
X-Original-To: ipfix@ietfa.amsl.com
Delivered-To: ipfix@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E6F7A21F85EA for ; Fri, 26 Oct 2012 13:21:19 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -10.599
X-Spam-Level: 
X-Spam-Status: No, score=-10.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Hzg9rnttr90H for ; Fri, 26 Oct 2012 13:21:19 -0700 (PDT)
Received: from rcdn-iport-5.cisco.com (rcdn-iport-5.cisco.com [173.37.86.76]) by ietfa.amsl.com (Postfix) with ESMTP id 2124121F85C1 for ; Fri, 26 Oct 2012 13:21:19 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=2054; q=dns/txt; s=iport; t=1351282879; x=1352492479; h=message-id:date:from:mime-version:to:cc:subject: references:in-reply-to:content-transfer-encoding; bh=WQ+apyrLwPeOOrQfI4zDifTD+BDRKAdR2yYpvSCdg2Y=; b=b2PsJSj8iO10AM2J2Udplh0SiFrq566aE5+B21lQuY2smdmQMA0NbXUR +SGt8ArNH57nmNkJ5zx1qF6Y4jhkPjxGDaJePVCyBPAtsVW9NZaGEfC7L kMQtSLFWVJdPbEk6KDXtjwN8Tvk/dHcFymPNK4CdTR1nSR5ax3uAOf0Sm Y=;
X-IronPort-AV: E=Sophos;i="4.80,656,1344211200"; d="scan'208";a="135834460"
Received: from rcdn-core-5.cisco.com ([173.37.93.156]) by rcdn-iport-5.cisco.com with ESMTP; 26 Oct 2012 20:21:18 +0000
Received: from [10.55.82.156] (dhcp-10-55-82-156.cisco.com [10.55.82.156]) by rcdn-core-5.cisco.com (8.14.5/8.14.5) with ESMTP id q9QKLHKK024277;  Fri, 26 Oct 2012 20:21:17 GMT
Message-ID: <508AF0BD.9070509@cisco.com>
Date: Fri, 26 Oct 2012 21:21:17 +0100
From: Paul Aitken 
User-Agent: Mozilla/5.0 (X11; Linux i686; rv:15.0) Gecko/20120912 Thunderbird/15.0.1
MIME-Version: 1.0
To: Andrew Johnson 
References:    <5087B96B.7020500@cisco.com>  <508850F7.2080801@net.in.tum.de> <50885B49.6050603@cisco.com>  <5088666F.1090106@cisco.com>  <50898454.2000706@net.in.tum.de>  <04A22782-E8DE-4FFB-A31C-8DC9FAC8FDDE@cisco.com>
In-Reply-To: <04A22782-E8DE-4FFB-A31C-8DC9FAC8FDDE@cisco.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Cc: John Court , ipfix@ietf.org
Subject: Re: [IPFIX] Export of long lived flow information
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: IPFIX WG discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Fri, 26 Oct 2012 20:21:20 -0000

Andrew,

> A flow is defined as a set of packets which all have some common 
> properties, and the original idea of a flow was based on the common 
> 5-tuple of IP address, IP protocol and ports.  A Flow Record is formed 
> from any observation of packets belonging to a Flow (not all packets 
> within the flow are necessarily observed).
>
> If my PC opens an HTTP connection to some server and downloads a 
> simple web page, then we'd expect to see one traditional flows (per 
> direction, but ignore that for now).  If I open a new HTTP connection 
> to the same server, and coincidentally use the same source port, weeks 
> later, is that the same flow?

Per RFC 5101:

       A Flow is defined as a set of IP packets passing an Observation
       Point in the network during a certain time interval.


So if you define different time intervals (eg, "last week", "this week", 
and "next week") then yes, you've got two different flows there.

Whereas, if you define a interval which encompasses both of your 
connections, then you've only got one flow. And if your flow is going to 
last for weeks, then I hope you're reporting intermediate values.


> I had always thought of these as two flows, and the flowEndReason 
> implies it, but that would introduce some sort of time property as one 
> of the common property shared by the packets that make up a flow.  A 
> collector might aggregate the two reports, but removing the time 
> property is much like any other form of aggregation.

The time property is built in to the RFC 5101 definition.

P.


> It seems to me that we're using the timeout values of the cache as a 
> sort of ill-defined common property, but things get confusing when we 
> export total counts, or because the cache is low on resources, etc. 
>  Ideally, we'd want to be able to send more than one Flow Record for 
> the same Flow and provide enough information for the Collector to 
> reconstruct what the Monitoring Process is using to define a Flow.
>
>
> Cheers, Andrew


From paitken@cisco.com  Mon Oct 29 04:56:34 2012
Return-Path: 
X-Original-To: ipfix@ietfa.amsl.com
Delivered-To: ipfix@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 703C821F85FA for ; Mon, 29 Oct 2012 04:56:34 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -8.595
X-Spam-Level: 
X-Spam-Status: No, score=-8.595 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8, TRACKER_ID=2.003]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 9nSzlLpu1HWG for ; Mon, 29 Oct 2012 04:56:33 -0700 (PDT)
Received: from ams-iport-3.cisco.com (ams-iport-3.cisco.com [144.254.224.146]) by ietfa.amsl.com (Postfix) with ESMTP id 5557221F85E4 for ; Mon, 29 Oct 2012 04:56:33 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=4425; q=dns/txt; s=iport; t=1351511793; x=1352721393; h=message-id:date:from:mime-version:to:cc:subject: references:in-reply-to; bh=jaCxUv1zijp5Qhl/3D8kKl9JphtnsgBRbSm6cmxVLuQ=; b=VQ5FomhP3gCf/z1nsjpzOnNhvNqgp9CfugPK3sI1Jfws8CYyykPgBnNp mhwNCSDIko+jaBdM6FV/ZjTWuScPzTQZMTs+g+smE4A/I/5CUE4Uh+FMY wyt5ifvFbbG5yU4oIYbXk7Hcb5HzWAUZ88uiuQ1E6W9Renqm+R0CFx5X9 8=;
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: AgQGAKdtjlCQ/khN/2dsb2JhbABEi3e2a4EIgh8BAQQSAWUBEAsEHRYPCQMCAQIBRRMBBwEBHodknCufTI8ugyQDlXSFaYhugWuCbw
X-IronPort-AV: E=Sophos;i="4.80,671,1344211200"; d="scan'208,217";a="9173434"
Received: from ams-core-4.cisco.com ([144.254.72.77]) by ams-iport-3.cisco.com with ESMTP; 29 Oct 2012 11:56:32 +0000
Received: from [10.55.95.150] (dhcp-10-55-95-150.cisco.com [10.55.95.150]) by ams-core-4.cisco.com (8.14.5/8.14.5) with ESMTP id q9TBuUWr015355; Mon, 29 Oct 2012 11:56:30 GMT
Message-ID: <508E6EF2.8000801@cisco.com>
Date: Mon, 29 Oct 2012 11:56:34 +0000
From: Paul Aitken 
User-Agent: Mozilla/5.0 (X11; Linux i686; rv:15.0) Gecko/20120912 Thunderbird/15.0.1
MIME-Version: 1.0
To: ipfix@ietf.org
References: 
In-Reply-To: 
Content-Type: multipart/alternative; boundary="------------000300030107040209010001"
Cc: John Court 
Subject: Re: [IPFIX] Export of long lived flow information
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: IPFIX WG discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Mon, 29 Oct 2012 11:56:34 -0000

This is a multi-part message in MIME format.
--------------000300030107040209010001
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Should we change "this Flow" to "accounted in this Flow Record" 
throughout the registry?

I counted 67 instances. eg:


octetDeltaCount

          The number of octets since the previous report (if any)
          in incoming packets for this Flow at the Observation Point.


tcpControlBits

          TCP control bits observed for packets of this Flow.
          The information is encoded in a set of bit fields.
          For each TCP control bit, there is a bit in this
          set.  A bit is set to 1 if any observed packet of this
          Flow has the corresponding TCP control bit set to 1.
          A value of 0 for a bit indicates that the corresponding
          bit was not set in any of the observed packets
of this Flow.


flowDurationMilliseconds

          The difference in time between the first observed packet
of this Flow and the last observed packet of this Flow.


ingressInterface

            The index of the IP interface where packets of this Flow
            are being received.


egressInterface

            The index of the IP interface where packets of
            this Flow are being sent.

P.

--------------000300030107040209010001
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit


  
    
  
  
    
Should we change "this
        Flow" to "accounted in this Flow
        Record" throughout the registry?

      

      I counted 67 instances. eg:

      

      

      octetDeltaCount

      

               The number of octets since the previous report (if any)

               in incoming packets for this Flow
      at the Observation Point.

      

      

      tcpControlBits

      

               TCP control bits observed for packets of this Flow.

               The information is encoded in a set of bit fields.

               For each TCP control bit, there is a bit in this

               set.  A bit is set to 1 if any observed packet of this

                 Flow has the corresponding TCP control bit set
      to 1.

               A value of 0 for a bit indicates that the corresponding

               bit was not set in any of the observed packets

               of this Flow.

      

      

      flowDurationMilliseconds

      

               The difference in time between the first observed packet

               of this Flow and the last
      observed packet of this Flow.

      

      

      ingressInterface

      

                 The index of the IP interface where packets of this Flow

                 are being received.

      

      

      egressInterface

      

                 The index of the IP interface where packets of

                   this Flow are being sent.

               

      P.

    

  


--------------000300030107040209010001--

From paitken@cisco.com  Mon Oct 29 05:17:02 2012
Return-Path: 
X-Original-To: ipfix@ietfa.amsl.com
Delivered-To: ipfix@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B99E721F864A for ; Mon, 29 Oct 2012 05:17:02 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -9.597
X-Spam-Level: 
X-Spam-Status: No, score=-9.597 tagged_above=-999 required=5 tests=[AWL=1.002,  BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id eQsO5FrqMWRk for ; Mon, 29 Oct 2012 05:17:01 -0700 (PDT)
Received: from ams-iport-4.cisco.com (ams-iport-4.cisco.com [144.254.224.147]) by ietfa.amsl.com (Postfix) with ESMTP id E3FA721F8646 for ; Mon, 29 Oct 2012 05:17:00 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=4621; q=dns/txt; s=iport; t=1351513021; x=1352722621; h=message-id:date:from:mime-version:to:cc:subject: references:in-reply-to; bh=6r7mok7A70y4Mo0sWy7nzi3fA+l7CEm6fESRbD1J+pY=; b=fyqaxt/fskCkb3qR5XVX7GybOCvXIZ+ooN4KGluR/lnmTGr4ZqTdSORR OmUVnhE63MXv0jk8nmSU4o849tvHq4rNMvwqsu+NUpyCGgi18TUlTRaKz 4LQ8abyKumyGyPSGIMebKSHb0NFL/rnmBQ9/ZwKPXb64Z4Wosf3OReGhd M=;
X-IronPort-AV: E=Sophos;i="4.80,671,1344211200"; d="scan'208,217";a="9183668"
Received: from ams-core-3.cisco.com ([144.254.72.76]) by ams-iport-4.cisco.com with ESMTP; 29 Oct 2012 12:16:57 +0000
Received: from [10.55.95.150] (dhcp-10-55-95-150.cisco.com [10.55.95.150]) by ams-core-3.cisco.com (8.14.5/8.14.5) with ESMTP id q9TCGu92032297; Mon, 29 Oct 2012 12:16:56 GMT
Message-ID: <508E73BB.2090009@cisco.com>
Date: Mon, 29 Oct 2012 12:16:59 +0000
From: Paul Aitken 
User-Agent: Mozilla/5.0 (X11; Linux i686; rv:15.0) Gecko/20120912 Thunderbird/15.0.1
MIME-Version: 1.0
To: John Court 
References:    <5087B96B.7020500@cisco.com>  <508850F7.2080801@net.in.tum.de> <50885B49.6050603@cisco.com>  <5088666F.1090106@cisco.com> 
In-Reply-To: 
Content-Type: multipart/alternative; boundary="------------040903010005000108090105"
Cc: ipfix@ietf.org
Subject: Re: [IPFIX] Export of long lived flow information
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: IPFIX WG discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Mon, 29 Oct 2012 12:17:02 -0000

This is a multi-part message in MIME format.
--------------040903010005000108090105
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

John,

> Paul,
>
> *Definitely. If it's a permanent flow and you're exporting totalCount
> fields - which are measured "since the Metering Process
> (re-)initialization for this Observation Point" - then the flowStartTime
> must surely be the time the first ever packet was observed.*
>
> If you take that literally shouldn't that be interpreted to mean that 
> the totalCount continues into the next time a connection is up between 
> the same flow key ?

Yes, that's what it says.


> Even if a flowEndReason of :
>
>  0x03: end of Flow detected
> The Flow was terminated because the Metering Process
> detected signals indicating the end of the Flow, for
> example, the TCP FIN flag.

The definitions of *totalCount are independent of flowEndReason.


> That clearly wouldn't be of much use IMO and makes it difficult to see 
> what the flowEndReason field semantics mean in that context.  Just 
> pointing out that taking that definition literally doesn't give a 
> useful answer on its own either :-).  Although maybe that does make 
> sense in a router context ?  Can you clarify this some more, perhaps 
> you never intend using the flowEndReason IE in your case ?

We do use flowEndReason. However, flowEndReason doesn't logically apply 
to permanent flows: they're permanent, therefore, they don't end.

P.


--------------040903010005000108090105
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit


  
    
  
  
    
John,

      

    

    
Paul,
      

      

      Definitely. If it's a permanent flow and
            you're
            exporting totalCount 

            fields - which are measured "since the Metering Process 

            (re-)initialization for this Observation Point" - then the
            flowStartTime
            

            must surely be the time the first ever packet was observed.
      

      

      If you take that literally
        shouldn't
        that be interpreted to mean that the totalCount continues into
        the next
        time a connection is up between the same flow key ?

    

    Yes, that's what it says.

    

    

    
 Even if a
        flowEndReason
        of :
      

      

       0x03: end of Flow detected
      

      The Flow was terminated because the
        Metering
        Process
      

      detected signals indicating the end
        of
        the Flow, for
      

      example, the TCP FIN flag.
      

    

    

    The definitions of *totalCount are independent of flowEndReason.

    

    

    
That clearly wouldn't
        be of much use
        IMO and makes it difficult to see what the flowEndReason field
        semantics
        mean in that context.  Just pointing out that taking that
        definition
        literally doesn't give a useful answer on its own either :-).
         Although
        maybe that does make sense in a router context ?  Can you
        clarify
        this some more, perhaps you never intend using the flowEndReason
        IE in
        your case ?
      

    

    

    We do use flowEndReason. However, flowEndReason doesn't logically
    apply to permanent flows: they're permanent, therefore, they don't
    end.

    

    P.

    

  


--------------040903010005000108090105--

From andrewf@plixer.com  Mon Oct 29 06:08:42 2012
Return-Path: 
X-Original-To: ipfix@ietfa.amsl.com
Delivered-To: ipfix@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CCA6221F85C0 for ; Mon, 29 Oct 2012 06:08:42 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.595
X-Spam-Level: 
X-Spam-Status: No, score=-0.595 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, TRACKER_ID=2.003]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kR1RNxPt95bG for ; Mon, 29 Oct 2012 06:08:42 -0700 (PDT)
Received: from smtp.plixer.com (smtp.plixer.com [66.186.184.193]) by ietfa.amsl.com (Postfix) with ESMTP id 0435621F853F for ; Mon, 29 Oct 2012 06:08:41 -0700 (PDT)
Received: from [10.100.1.132] ([10.100.1.132]) by smtp.plixer.com over TLS secured channel with Microsoft SMTPSVC(6.0.3790.4675);  Mon, 29 Oct 2012 09:08:40 -0400
Message-ID: <508E7FD8.6000207@plixer.com>
Date: Mon, 29 Oct 2012 09:08:40 -0400
From: Andrew Feren 
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:19.0) Gecko/19.0 Thunderbird/19.0a1
MIME-Version: 1.0
To: Paul Aitken 
References:  <508E6EF2.8000801@cisco.com>
In-Reply-To: <508E6EF2.8000801@cisco.com>
Content-Type: multipart/alternative; boundary="------------070202020409000807040800"
X-OriginalArrivalTime: 29 Oct 2012 13:08:40.0619 (UTC) FILETIME=[84056FB0:01CDB5D6]
Cc: John Court , ipfix@ietf.org
Subject: Re: [IPFIX] [QUAR] Re:  Export of long lived flow information
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: IPFIX WG discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Mon, 29 Oct 2012 13:08:43 -0000

This is a multi-part message in MIME format.
--------------070202020409000807040800
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Hi Paul,

Interesting observation.   The use of flow in these has never tripped me 
up the same way as the text in the original question.

On 10/29/2012 07:56 AM, Paul Aitken wrote:
> Should we change "this Flow" to "accounted in this Flow Record" 
> throughout the registry?
>
> I counted 67 instances. eg:
>
>
> octetDeltaCount
>
>          The number of octets since the previous report (if any)
>          in incoming packets for this Flow at the Observation Point.

For this I think Flow is correct.  I also think this highlights the 
distinction being made by people between Flow and Flow Record.  If a 
Flow is bounded the same way (has the same start and end time) as a Flow 
Record then there can't have been a previous report for this Flow.  The 
Flow and Flow Record would be the same.

>
>
> tcpControlBits
>
>          TCP control bits observed for packets of this Flow.
>          The information is encoded in a set of bit fields.
>          For each TCP control bit, there is a bit in this
>          set.  A bit is set to 1 if any observed packet of this
>          Flow has the corresponding TCP control bit set to 1.
>          A value of 0 for a bit indicates that the corresponding
>          bit was not set in any of the observed packets
>          of this Flow.
>
>
> flowDurationMilliseconds
>
>          The difference in time between the first observed packet
>          of this Flow and the last observed packet of this Flow.

Eep.  Thinking about this is making me rethink my initial +1, but I do 
still think some rewording is needed.  Using Flow Record works for 
deltas, but not for totals.  I'm not sure yet what the right wording is.

>
>
> ingressInterface
>
>            The index of the IP interface where packets of this Flow
>            are being received.
>
>
> egressInterface
>
>            The index of the IP interface where packets of
>            this Flow are being sent.

I think these last two are typically part of what defines a flow and 
won't change with out changing the Flow.  So I guess Flow is as good as 
Flow Record.
>
> P.


--------------070202020409000807040800
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit


  
    
  
  
    Hi Paul,

    

    Interesting observation.   The use of flow in these has never
    tripped me up the same way as the text in the original question.

    

    
On 10/29/2012 07:56 AM, Paul Aitken
      wrote:

    

    

      
      
Should we change "this Flow" to
        "accounted in this Flow Record" throughout the registry?

        

        I counted 67 instances. eg:

        

        

        octetDeltaCount

        

                 The number of octets since the previous report (if any)

                 in incoming packets for this Flow at the Observation
        Point.

      

    

    

    For this I think Flow is correct.  I also think this highlights the
    distinction being made by people between Flow and Flow Record.  If a
    Flow is bounded the same way (has the same start and end time) as a
    Flow Record then there can't have been a previous report for this
    Flow.  The Flow and Flow Record would be the same.

    

    

      
 

        

        tcpControlBits

        

                 TCP control bits observed for packets of this Flow.

                 The information is encoded in a set of bit fields.

                 For each TCP control bit, there is a bit in this

                 set.  A bit is set to 1 if any observed packet of this

                 Flow has the corresponding TCP control bit set to 1.

                 A value of 0 for a bit indicates that the corresponding

                 bit was not set in any of the observed packets

                 of this Flow.

        

        

        flowDurationMilliseconds

        

                 The difference in time between the first observed
        packet

                 of this Flow and the last observed packet of this Flow.

      

    

    

    Eep.  Thinking about this is making me rethink my initial +1, but I
    do still think some rewording is needed.  Using Flow Record works
    for deltas, but not for totals.  I'm not sure yet what the right
    wording is.

    

    

      
 

        

        ingressInterface

        

                   The index of the IP interface where packets of this
        Flow

                   are being received.

        

        

        egressInterface

        

                   The index of the IP interface where packets of

                   this Flow are being sent.

      

    

    

    I think these last two are typically part of what defines a flow and
    won't change with out changing the Flow.  So I guess Flow is as good
    as Flow Record.

    

      
          

        P.

      

    

    

  


--------------070202020409000807040800--

From muenz@net.in.tum.de  Tue Oct 30 04:51:48 2012
Return-Path: 
X-Original-To: ipfix@ietfa.amsl.com
Delivered-To: ipfix@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0C99721F8564 for ; Tue, 30 Oct 2012 04:51:46 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -4.39
X-Spam-Level: 
X-Spam-Status: No, score=-4.39 tagged_above=-999 required=5 tests=[BAYES_20=-0.74, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_MED=-4]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bL2hamEd4oiU for ; Tue, 30 Oct 2012 04:51:45 -0700 (PDT)
Received: from mail-out1.informatik.tu-muenchen.de (mail-out1.informatik.tu-muenchen.de [131.159.0.8]) by ietfa.amsl.com (Postfix) with ESMTP id 7A73221F855E for ; Tue, 30 Oct 2012 04:51:44 -0700 (PDT)
Received: by mail.net.in.tum.de (Postfix, from userid 81) id 9F05818128B2; Tue, 30 Oct 2012 12:51:41 +0100 (CET)
To: Paul Aitken 
X-PHP-Originating-Script: 0:main.inc
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Date: Tue, 30 Oct 2012 12:51:41 +0100
From: Gerhard Muenz 
In-Reply-To: <508AE290.3020902@cisco.com>
References:    <5087B96B.7020500@cisco.com>  <508850F7.2080801@net.in.tum.de> <50885B49.6050603@cisco.com>  <5088666F.1090106@cisco.com>  <50898454.2000706@net.in.tum.de> <508AB8FB.3060807@plixer.com> <508AE290.3020902@cisco.com>
Message-ID: <5b06f4caa55ce88df0f606fb59e19785@net.in.tum.de>
X-Sender: muenz@net.in.tum.de
User-Agent: Roundcube Webmail/0.6
Cc: John Court , ipfix@ietf.org
Subject: Re: [IPFIX] Export of long lived flow information
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: IPFIX WG discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Tue, 30 Oct 2012 11:51:48 -0000

Paul,

On 26.10.2012 20:20, Paul Aitken wrote:
> Andrew, Gerhard,
>
>>> My understanding is that both, deltaCounts and totalCounts contain 
>>> the number of packets or octets observed in the indicated time 
>>> interval. So, for identical flowStart* and flowEnd* timestamps, the 
>>> values are the same.
>> This is my understanding as well.
>
> There has to be a difference between delta and total counts, else we
> wouldn't have both of them!
>
> Suppose we have a permanent cache, so the cache entries never expire.
>
> For a new flow starting at t0 with a first export at t1, the
> timestamps, delta, and total counts are the same.
>
> However with the second export at t2, the total and delta counts are
> different although their timestamps match (they'll both say, "t0 to
> t2").

No, this would contradict the new definition of flowStart* we are just 
discussing.
If delta counts are exported for the interval (t1,t2), then flowStart* 
is t1.
If delta counts are exported for the interval (t0,t2), then flowStart* 
is t0.
If total counts are exported, flowStart is always t0.
These statements hold regardless of which type of cache is used by the 
Metering Process. In general, the information model does not care about 
how the cache is implemented. The exported information just must follow 
the IE definition.

>
> With the traditional (non-permanent) cache, the entry would probably
> have been removed at t1 and re-created on a subsequent packet, so at
> t2 the delta and total counts would both be equal. However it'd be
> incorrect to report the total count, because that's defined as the
> total number of packets or bytes ..."since the Metering Process
> (re-)initialization for this Observation Point".

You must not export total counters in this case because you reset 
counters before re-initialization of the Metering Process.

Thanks,
Gerhard

>
>
>>> However, the description of totalCounts says that you report the 
>>> number of packets or octets observed for this Flow since 
>>> re-initialization. So, you must never reset the counter for this 
>>> Flow, even after observing a FIN or RST.
>>> If you reset flow counters, or if you remove Flows from your Cache, 
>>> you cannot use totalCounts any more unless you re-initialize the 
>>> Metering Process (e.g. after flushing the entire permanent Cache).
>>
>> I can try some tests later, but from what I have seen (and been 
>> told) many totals being exported are in fact just a delta sent once at 
>> the end of the flow.  If a later flow had the same IPs, protocol, and 
>> ports as an earlier flow I'm pretty sure a new start time will be sent 
>> rather than the the first time that flow was seen since reinitializing 
>> the metering process.
>
> So the MP uses a traditional (non-permanent) cache. In RFC 6728
> terms, a TimeoutCache or NaturalCache rather than a PermanentCache.
>
>
>> Or to put it an other way I think deltas are being sent, but called 
>> totals by the implementation because it seemed like the right thing to 
>> do for a value being sent once at the end of the flow.
>
> The collector could be aggregating deltas to keep running totals.
>
>
>> I suspect that totals reporting on the export process (eg 
>> exportedOctetTotalCount, exportedMessageTotalCount) are, however, 
>> reported with a start time that is only reset on reinitialization.
>
> Definitely, because these are defined as "The total number of X that
> the Exporting Process has sent since the Exporting Process
> (re-)initialization ...".
>
> P.


From paitken@cisco.com  Tue Oct 30 05:21:38 2012
Return-Path: 
X-Original-To: ipfix@ietfa.amsl.com
Delivered-To: ipfix@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9A15321F857D for ; Tue, 30 Oct 2012 05:21:38 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -10.098
X-Spam-Level: 
X-Spam-Status: No, score=-10.098 tagged_above=-999 required=5 tests=[AWL=0.501, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id q+qv+1+-owbj for ; Tue, 30 Oct 2012 05:21:38 -0700 (PDT)
Received: from ams-iport-4.cisco.com (ams-iport-4.cisco.com [144.254.224.147]) by ietfa.amsl.com (Postfix) with ESMTP id 9501421F857C for ; Tue, 30 Oct 2012 05:21:37 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=4006; q=dns/txt; s=iport; t=1351599697; x=1352809297; h=message-id:date:from:mime-version:to:cc:subject: references:in-reply-to:content-transfer-encoding; bh=mwuZbpB7GCNG1aBR46u+9BQ97h914jxr4a9f4+qABnw=; b=SeHlhwtLFKB4Z+wevMg4UZ3oL3ZiLsOsWYIQ4cSMmKLEt6NIgDQUZo65 P4BUGmbYWI+L+ZUgJVhxMJC7KYC+D9r03JgRHY+vGKE7kNL7xXH47XlCB 8Csvj0NfIxoVtl6n80aaE4qahZHtohGUIQ2RLNOXGA+wjY4DpLCpRwiho A=;
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: AgkFAIzFj1CQ/khN/2dsb2JhbABEhhi5VYNIgQiCHgEBAQQSARAVQAEQCxgCAgUWCwICCQMCAQIBRQYNAQcBAR6HZJxhjSyCO5A2gSCKVYVKgRMDlXSFaYhugWuCbw
X-IronPort-AV: E=Sophos;i="4.80,679,1344211200";  d="scan'208";a="9211047"
Received: from ams-core-4.cisco.com ([144.254.72.77]) by ams-iport-4.cisco.com with ESMTP; 30 Oct 2012 12:21:33 +0000
Received: from [10.55.92.151] (dhcp-10-55-92-151.cisco.com [10.55.92.151]) by ams-core-4.cisco.com (8.14.5/8.14.5) with ESMTP id q9UCLXcW014463; Tue, 30 Oct 2012 12:21:33 GMT
Message-ID: <508FC64D.3000006@cisco.com>
Date: Tue, 30 Oct 2012 12:21:33 +0000
From: Paul Aitken 
User-Agent: Mozilla/5.0 (X11; Linux i686; rv:15.0) Gecko/20120912 Thunderbird/15.0.1
MIME-Version: 1.0
To: Gerhard Muenz 
References:    <5087B96B.7020500@cisco.com>  <508850F7.2080801@net.in.tum.de> <50885B49.6050603@cisco.com>  <5088666F.1090106@cisco.com>  <50898454.2000706@net.in.tum.de> <508AB8FB.3060807@plixer.com> <508AE290.3020902@cisco.com> <5b06f4caa55ce88df0f606fb59e19785@net.in.tum.de>
In-Reply-To: <5b06f4caa55ce88df0f606fb59e19785@net.in.tum.de>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Cc: John Court , ipfix@ietf.org
Subject: Re: [IPFIX] Export of long lived flow information
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: IPFIX WG discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Tue, 30 Oct 2012 12:21:38 -0000

Gerhard,

I agree with your definitions. Thanks for clarifying.

So what's the next step?

     * Update the IANA definitions?
     * Add clarifications to the WG documents?


Do we need a short presentation at the upcoming WG meeting?

P.


On 30/10/12 11:51, Gerhard Muenz wrote:
> Paul,
>
> On 26.10.2012 20:20, Paul Aitken wrote:
>> Andrew, Gerhard,
>>
>>>> My understanding is that both, deltaCounts and totalCounts contain 
>>>> the number of packets or octets observed in the indicated time 
>>>> interval. So, for identical flowStart* and flowEnd* timestamps, the 
>>>> values are the same.
>>> This is my understanding as well.
>>
>> There has to be a difference between delta and total counts, else we
>> wouldn't have both of them!
>>
>> Suppose we have a permanent cache, so the cache entries never expire.
>>
>> For a new flow starting at t0 with a first export at t1, the
>> timestamps, delta, and total counts are the same.
>>
>> However with the second export at t2, the total and delta counts are
>> different although their timestamps match (they'll both say, "t0 to
>> t2").
>
> No, this would contradict the new definition of flowStart* we are just 
> discussing.
> If delta counts are exported for the interval (t1,t2), then flowStart* 
> is t1.
> If delta counts are exported for the interval (t0,t2), then flowStart* 
> is t0.
> If total counts are exported, flowStart is always t0.
> These statements hold regardless of which type of cache is used by the 
> Metering Process. In general, the information model does not care 
> about how the cache is implemented. The exported information just must 
> follow the IE definition.
>
>>
>> With the traditional (non-permanent) cache, the entry would probably
>> have been removed at t1 and re-created on a subsequent packet, so at
>> t2 the delta and total counts would both be equal. However it'd be
>> incorrect to report the total count, because that's defined as the
>> total number of packets or bytes ..."since the Metering Process
>> (re-)initialization for this Observation Point".
>
> You must not export total counters in this case because you reset 
> counters before re-initialization of the Metering Process.
>
> Thanks,
> Gerhard
>
>>
>>
>>>> However, the description of totalCounts says that you report the 
>>>> number of packets or octets observed for this Flow since 
>>>> re-initialization. So, you must never reset the counter for this 
>>>> Flow, even after observing a FIN or RST.
>>>> If you reset flow counters, or if you remove Flows from your Cache, 
>>>> you cannot use totalCounts any more unless you re-initialize the 
>>>> Metering Process (e.g. after flushing the entire permanent Cache).
>>>
>>> I can try some tests later, but from what I have seen (and been 
>>> told) many totals being exported are in fact just a delta sent once 
>>> at the end of the flow.  If a later flow had the same IPs, protocol, 
>>> and ports as an earlier flow I'm pretty sure a new start time will 
>>> be sent rather than the the first time that flow was seen since 
>>> reinitializing the metering process.
>>
>> So the MP uses a traditional (non-permanent) cache. In RFC 6728
>> terms, a TimeoutCache or NaturalCache rather than a PermanentCache.
>>
>>
>>> Or to put it an other way I think deltas are being sent, but called 
>>> totals by the implementation because it seemed like the right thing 
>>> to do for a value being sent once at the end of the flow.
>>
>> The collector could be aggregating deltas to keep running totals.
>>
>>
>>> I suspect that totals reporting on the export process (eg 
>>> exportedOctetTotalCount, exportedMessageTotalCount) are, however, 
>>> reported with a start time that is only reset on reinitialization.
>>
>> Definitely, because these are defined as "The total number of X that
>> the Exporting Process has sent since the Exporting Process
>> (re-)initialization ...".
>>
>> P.


From trammell@tik.ee.ethz.ch  Tue Oct 30 05:38:57 2012
Return-Path: 
X-Original-To: ipfix@ietfa.amsl.com
Delivered-To: ipfix@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C06B221F852C for ; Tue, 30 Oct 2012 05:38:57 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -6.599
X-Spam-Level: 
X-Spam-Status: No, score=-6.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6KjmIadAFGq2 for ; Tue, 30 Oct 2012 05:38:57 -0700 (PDT)
Received: from smtp.ee.ethz.ch (smtp.ee.ethz.ch [129.132.2.219]) by ietfa.amsl.com (Postfix) with ESMTP id AA62521F8523 for ; Tue, 30 Oct 2012 05:38:56 -0700 (PDT)
Received: from localhost (localhost [127.0.0.1]) by smtp.ee.ethz.ch (Postfix) with ESMTP id D237BD9310; Tue, 30 Oct 2012 13:38:55 +0100 (MET)
X-Virus-Scanned: by amavisd-new on smtp.ee.ethz.ch
Received: from smtp.ee.ethz.ch ([127.0.0.1]) by localhost (.ee.ethz.ch [127.0.0.1]) (amavisd-new, port 10024) with LMTP id rm70XpVN-UnC; Tue, 30 Oct 2012 13:38:55 +0100 (MET)
Received: from pb-10243.ethz.ch (pb-10243.ethz.ch [82.130.102.152]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: briant) by smtp.ee.ethz.ch (Postfix) with ESMTPSA id 83913D9307; Tue, 30 Oct 2012 13:38:55 +0100 (MET)
Mime-Version: 1.0 (Apple Message framework v1283)
Content-Type: text/plain; charset=us-ascii
From: Brian Trammell 
In-Reply-To: <508FC64D.3000006@cisco.com>
Date: Tue, 30 Oct 2012 13:38:54 +0100
Content-Transfer-Encoding: quoted-printable
Message-Id: <03BF5948-51D3-417B-AD8A-5F6B678A9F46@tik.ee.ethz.ch>
References:    <5087B96B.7020500@cisco.com>  <508850F7.2080801@net.in.tum.de> <50885B49.6050603@cisco.com>  <5088666F.1090106@cisco.com>  <50898454.2000706@net.in.tum.de> <508AB8FB.3060807@plixer.com> <508AE290.3020902@cisco.com> <5b06f4caa55ce88df0f606fb59e19785@net.in.tum.de> <508FC64D.3000006@cisco.com>
To: Paul Aitken 
X-Mailer: Apple Mail (2.1283)
Cc: John Court , ipfix@ietf.org
Subject: Re: [IPFIX] Export of long lived flow information
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: IPFIX WG discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Tue, 30 Oct 2012 12:38:57 -0000

Hi, Paul, all,

If someone throws together a quick summary of the position for me I'd be =
happy make a couple of slides and do the presentation in person in =
Atlanta.

=46rom where I sit, it looks like we just go ask IANA to make the =
registry change. Wearing my IE-Doctors-author hat (I can't wear an IE =
Doctor hat, there aren't any yet. :) ), I'd say this revision would be =
covered (and permissible) under point 2 in section 5.2 of ie-doctors.

Best regards,

Brian


On 30 Oct 2012, at 13:21 , Paul Aitken wrote:

> Gerhard,
>=20
> I agree with your definitions. Thanks for clarifying.
>=20
> So what's the next step?
>=20
>    * Update the IANA definitions?
>    * Add clarifications to the WG documents?
>=20
>=20
> Do we need a short presentation at the upcoming WG meeting?
>=20
> P.
>=20
>=20
> On 30/10/12 11:51, Gerhard Muenz wrote:
>> Paul,
>>=20
>> On 26.10.2012 20:20, Paul Aitken wrote:
>>> Andrew, Gerhard,
>>>=20
>>>>> My understanding is that both, deltaCounts and totalCounts contain =
the number of packets or octets observed in the indicated time interval. =
So, for identical flowStart* and flowEnd* timestamps, the values are the =
same.
>>>> This is my understanding as well.
>>>=20
>>> There has to be a difference between delta and total counts, else we
>>> wouldn't have both of them!
>>>=20
>>> Suppose we have a permanent cache, so the cache entries never =
expire.
>>>=20
>>> For a new flow starting at t0 with a first export at t1, the
>>> timestamps, delta, and total counts are the same.
>>>=20
>>> However with the second export at t2, the total and delta counts are
>>> different although their timestamps match (they'll both say, "t0 to
>>> t2").
>>=20
>> No, this would contradict the new definition of flowStart* we are =
just discussing.
>> If delta counts are exported for the interval (t1,t2), then =
flowStart* is t1.
>> If delta counts are exported for the interval (t0,t2), then =
flowStart* is t0.
>> If total counts are exported, flowStart is always t0.
>> These statements hold regardless of which type of cache is used by =
the Metering Process. In general, the information model does not care =
about how the cache is implemented. The exported information just must =
follow the IE definition.
>>=20
>>>=20
>>> With the traditional (non-permanent) cache, the entry would probably
>>> have been removed at t1 and re-created on a subsequent packet, so at
>>> t2 the delta and total counts would both be equal. However it'd be
>>> incorrect to report the total count, because that's defined as the
>>> total number of packets or bytes ..."since the Metering Process
>>> (re-)initialization for this Observation Point".
>>=20
>> You must not export total counters in this case because you reset =
counters before re-initialization of the Metering Process.
>>=20
>> Thanks,
>> Gerhard
>>=20
>>>=20
>>>=20
>>>>> However, the description of totalCounts says that you report the =
number of packets or octets observed for this Flow since =
re-initialization. So, you must never reset the counter for this Flow, =
even after observing a FIN or RST.
>>>>> If you reset flow counters, or if you remove Flows from your =
Cache, you cannot use totalCounts any more unless you re-initialize the =
Metering Process (e.g. after flushing the entire permanent Cache).
>>>>=20
>>>> I can try some tests later, but from what I have seen (and been =
told) many totals being exported are in fact just a delta sent once at =
the end of the flow.  If a later flow had the same IPs, protocol, and =
ports as an earlier flow I'm pretty sure a new start time will be sent =
rather than the the first time that flow was seen since reinitializing =
the metering process.
>>>=20
>>> So the MP uses a traditional (non-permanent) cache. In RFC 6728
>>> terms, a TimeoutCache or NaturalCache rather than a PermanentCache.
>>>=20
>>>=20
>>>> Or to put it an other way I think deltas are being sent, but called =
totals by the implementation because it seemed like the right thing to =
do for a value being sent once at the end of the flow.
>>>=20
>>> The collector could be aggregating deltas to keep running totals.
>>>=20
>>>=20
>>>> I suspect that totals reporting on the export process (eg =
exportedOctetTotalCount, exportedMessageTotalCount) are, however, =
reported with a start time that is only reset on reinitialization.
>>>=20
>>> Definitely, because these are defined as "The total number of X that
>>> the Exporting Process has sent since the Exporting Process
>>> (re-)initialization ...".
>>>=20
>>> P.
>=20
> _______________________________________________
> IPFIX mailing list
> IPFIX@ietf.org
> https://www.ietf.org/mailman/listinfo/ipfix


From bclaise@cisco.com  Tue Oct 30 15:01:08 2012
Return-Path: 
X-Original-To: ipfix@ietfa.amsl.com
Delivered-To: ipfix@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0273F21F85A9 for ; Tue, 30 Oct 2012 15:01:08 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -11.45
X-Spam-Level: 
X-Spam-Status: No, score=-11.45 tagged_above=-999 required=5 tests=[AWL=1.148,  BAYES_00=-2.599, GB_I_LETTER=-2, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4wuJzt9BmDkP for ; Tue, 30 Oct 2012 15:01:06 -0700 (PDT)
Received: from av-tac-bru.cisco.com (weird-brew.cisco.com [144.254.15.118]) by ietfa.amsl.com (Postfix) with ESMTP id B86E221F8587 for ; Tue, 30 Oct 2012 15:01:05 -0700 (PDT)
X-TACSUNS: Virus Scanned
Received: from strange-brew.cisco.com (localhost.cisco.com [127.0.0.1]) by av-tac-bru.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id q9UM13qB028791; Tue, 30 Oct 2012 23:01:03 +0100 (CET)
Received: from [10.60.67.92] (ams-bclaise-89111.cisco.com [10.60.67.92]) by strange-brew.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id q9UM114x010578; Tue, 30 Oct 2012 23:01:02 +0100 (CET)
Message-ID: <50904E1D.7060909@cisco.com>
Date: Tue, 30 Oct 2012 17:01:01 -0500
From: Benoit Claise 
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:16.0) Gecko/20121010 Thunderbird/16.0.1
MIME-Version: 1.0
To: "ipfix@ietf.org" , draft-ietf-ipfix-flow-selection-tech@tools.ietf.org
References: <4FC74398.50805@cisco.com> <4FC89B99.40107@cisco.com> <506DA106.5060705@cisco.com>
In-Reply-To: <506DA106.5060705@cisco.com>
Content-Type: multipart/alternative; boundary="------------030704040609050402090604"
Cc: ipfix-chairs@tools.ietf.org
Subject: Re: [IPFIX] New AD review of draft-ietf-ipfix-flow-selection-tech-10.txt
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: IPFIX WG discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Tue, 30 Oct 2012 22:01:08 -0000

This is a multi-part message in MIME format.
--------------030704040609050402090604
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Dear draft-ietf-ipfix-flow-selection-tech authors,

I was expecting a quick discussion on the very few remaining issues...
That has not happened.
The draft status is: AD evaluation, Revised ID Needed.

Regards, Benoit

> Dear authors,
>
> The draft improved quite dramatically.
> Thanks for that.
> See in line for some more comments. I removed all unnecessary text.
>
>> Dear authors,
>>
>> I'm performing the (new) AD review of 
>> draft-ietf-ipfix-flow-selection-tech-10.txt
>> Lucky you, an extra pair of eyes specifically looking at your draft
>>
>> If some points have been discussed already on the mailing list, let 
>> me know. I have to admit that I have not been following the latest 
>> iterations of this draft.
>>
>> IMHO, this document needs some more work...
>> I don't think that this document is really in line with the other 
>> Intermediate Processes documents:
>> http://tools.ietf.org/html/rfc6235
>> http://tools.ietf.org/html/draft-ietf-ipfix-a9n-03
>> Note that I might have some more comments once all the points in this 
>> email are addressed, as there are many ;-)
>> However, I'm available for a conf. call to clarify my points if you 
>> want to
>>
>> See in-line.
> ...
>>>      8.2.  Registration of Object Identifier  . . . . . . . . . . . 
>>> . 32
>>>    9.  Security Considerations  . . . . . . . . . . . . . . . . . . 
>>> . 32
>>>    10. Acknowledgments  . . . . . . . . . . . . . . . . . . . . . . 
>>> . 34
>>>    11. References . . . . . . . . . . . . . . . . . . . . . . . . . 
>>> . 34
>>>      11.1. Normative References . . . . . . . . . . . . . . . . . . 
>>> . 34
>>>      11.2. Informative References . . . . . . . . . . . . . . . . . 
>>> . 34
>>>    Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 
>>> . 35
>> Don't you have to include the non-normative XML in the appendix, as 
>> it was done for RFC5102, RFC5103?
>>>
>>
>>> 2. Terminology
>>>
>>>    This document is consistent with the terminology introduced in
>>>    [RFC5101], [RFC5470], [RFC5475] and [RFC3917].  As in [RFC5101] and
>>>    [RFC5476], the first letter of each IPFIX-specific and 
>>> PSAMP-specific
>>>    term is capitalized along with the flow selection specific terms
>>>    defined here.
>>>
>>>    * Packet Classification
>>>
>>>       Packet Classification is a process by which packets are mapped to
>>>       specific Flow Records based on packet properties or external
>>>       properties (e.g. interface).  The properties (e.g. header
>>>       information, packet content, AS number) make up the Flow Key. In
>>>       case a Flow Record for a specific Flow Key already exists the 
>>> Flow
>>>       Record is updated, otherwise a new Flow Record is created.
>>
>> How is this different that the Metering Process (RFC5101)?
>>     Metering Process
>>
>>        The Metering Process generates Flow Records.  Inputs to the
>>        process are packet headers and characteristics observed at an
>>        Observation Point, and packet treatment at the Observation Point
>>        (for example, the selected output interface).
>>
>>        The Metering Process consists of a set of functions that includes
>>        packet header capturing, timestamping, sampling, classifying, and
>>        maintaining Flow Records.
>>
>>        The maintenance of Flow Records may include creating new records,
>>        updating existing ones, computing Flow statistics, deriving
>>        further Flow properties, detecting Flow expiration, passing Flow
>>        Records to the Exporting Process, and deleting Flow Records.
>> What is the connection with the Metering Process?
>> Figure 1 seems to suggest that Packet Classification is a subset of 
>> the Metering Process...
> not sure that one was answered.
>
>>
>>
>>>
>>>    * Packet Aggregation Process
>>>
>>>       In the IPFIX Metering Process the Packet Aggregation Process
>>>       aggregates packet data into flow data and forms the Flow Records.
>> How is this different from the Metering Process?
> the "Packet Aggregation Process" is not used in the document. Why do 
> we need it?
>
>>> After the aggregation step only the aggregated flow information is
>>>       available.  Information about individual packets is lost.
>>>
>>>
>>>
>> Intermediate Flow Selection Process: an Intermediate Process as in
>>        [RFC6183  ] that ...
>>
>>
> The new definition improved a lot:
>   * Intermediate Flow Selection Process
>
>        An Intermediate Flow Selection Process takes Flow Records as its
>        input and selects a subset of this set as its output.
>        Intermediate Flow Selection Process is a more general concept than
>        Intermediate Selection Process as defined in [RFC6183  ].  While an
>        Intermediate Selection Process selects Flow Records from a
>        sequence based upon criteria-evaluated Flow record values and
>        passes only those Flow Records that match the criteria, an
>        Intermediate Flow Selection Process selects Flow Records using
>        selection criteria applicable to a larger set of Flow
>        characteristics and information.
> But is there a reason why this definition can't be based on 
> "intermediate Process" from RFC 6183:
>
>     Intermediate Process
>
>            An Intermediate Process takes a record stream as its input from
>            Collecting Processes, Metering Processes, IPFIX File Readers,
>            other Intermediate Processes, or other record sources; performs
>            some transformations on this stream based upon the content of each
>            record, states maintained across multiple records, or other data
>            sources; and passes the transformed record stream as its output to
>            Exporting Processes, IPFIX File Writers, or other Intermediate
>            Processes in order to perform IPFIX Mediation.  Typically, an
>            Intermediate Process is hosted by an IPFIX Mediator.
>            Alternatively, an Intermediate Process may be hosted by an
>            Original Exporter.
>
> So
>
>   * Intermediate Flow Selection Process
>
>       _  An Intermediate Flow Selection Process is__an Intermediate Process as in
>        [_RFC6183  _] that_  takes Flow Records as its
>        input and selects a subset of this set as its output.
>        Intermediate Flow Selection Process is a more general concept than
>        Intermediate Selection Process as defined in [RFC6183  ].  While an
>        Intermediate Selection Process selects Flow Records from a
>        sequence based upon criteria-evaluated Flow record values and
>        passes only those Flow Records that match the criteria, an
>        Intermediate Flow Selection Process selects Flow Records using
>        selection criteria applicable to a larger set of Flow
>        characteristics and information.
>
>>>
>
> Regarding terminology, I still some instances of "observation point". 
> Should be "Observation Point"
>
> ...
>
>>>
>>> 4.  Flow selection as a Function in the IPFIX Architecture
>>>
> Thanks for your new figure 1.
> One editorial change: change the + in the left vertical line.
>
>        +======|========================+      |
>        |      |  Mediator              |      |
>        +    +-V-------------------+    |      |
>        |    | Collecting Process  |    |      |
>        +    +---------------------+    |      |
>        |    | Intermediate Flow   |    |      |
>        |    | Selection Process   |    |      |
>        +    +---------------------+    |      |
>        |    |  Exporting Process  |    |      |
>        +    +-|-------------------+    |      |
>        +======|========================+      |
>        
>>>
>>> 5.1.  Flow Filtering
>>>
>>>    Flow Filtering is a deterministic function on the IPFIX Flow Record
>>>    content.  If the relevant flow characteristics are already 
>>> observable
>>>    at packet level (e.g.  Flow Keys), Flow Filtering can be applied
>>>    before aggregation at packet level.  In order to be compliant with
>>>    this document, at least the Property Match Filtering MUST be
>>>    implemented.
>> This contradicts.
>>     In order to be compliant with this document, at
>>     least one of the flow selection schemes MUST be implemented.
> Actually, wrong cut/paste.
> This contradicts, in section 1:
>     In order to be compliant with this document, at
>     least the Property Match Filtering MUST be implemented.
>
>
>>>
>>> 8.  IANA Considerations
>>>
>>> 8.1.  Registration of Information Elements
>
> Table 3: Information Elements to be registered, you can't put the 
> value 1, 2, 3,
> You need TBD1, TBD2, etc...
> And you must add
> "IANA Note: please replace TBD1, TBD2, ... with the assigned values, 
> throughout the document."
>
>>>
>>>
>>> 8.2.  Registration of Object Identifier
>>>
>
> RFC 5815 is obsoleted by RFC 6615 
>
> What you want is an extra in 
> http://www.iana.org/assignments/smi-numbers, pointing to this RFC:
>
>     Sub-registry Name: IPFIX-SELECTOR-MIB Functions
>     Reference: [RFC6615]
>     Registration Procedures: Expert Review
>     Prefix: iso.org.dod.internet.mgmt.mib-2.ipfixSelectorMIB.ipfixSelectorObjects.ipfixSelectorFunctions
>     (1.3.6.1.2.1.194.1.1)
>
>     Decimal Name                  Description                       Reference
>     ------- --------------------- --------------------------------- ---------
>     1       ipfixFuncSelectAll    Select everything                 [RFC6615]
>     2       psampSampCountBased   Systematic Count-based Sampling   [RFC6727]
>     3       psampSampTimeBased    Systematic Time-based Sampling    [RFC6727]
>     4       psampSampRandOutOfN   Random n-out-of-N Sampling        [RFC6727]
>     5       psampSampUniProb      Universal Probabilistic Sampling  [RFC6727]
>     6       psampFiltPropMatch    Property Match Filtering          [RFC6727]
>     7       psampFiltHash         Hash-based Filtering              [RFC6727]
>
> So you need TBDx
>
>     +---------+-----------------------+---------------------+-----------+
>     | Decimal | Name                  | Description         | Reference |
>     +---------+-----------------------+---------------------+-----------+
>     |  TBDx   | flowSelectorAlgorithm | This Object         | [RFCyyyy] |
>     |         |                       | Identifier          |           |
>     |         |                       | identifies the Flow |           |
>     |         |                       | selection technique |           |
>     |         |                       | (e.g., Filtering,   |           |
>     |         |                       | Sampling) that is   |           |
>     |         |                       | applied by the Flow |           |
>     |         |                       | Selection Process   |           |
>     +---------+-----------------------+---------------------+-----------+
>
>                 Table 4: Object Identifiers to be registered
>
> "IANA Note: please replace TBDx with the assigned value, throughout 
> the document."
>
> Btw, there is a mismatch between the IANA registry and the table in 
> section 7.1:
>     +----+------------------------+--------------------------+
>     | ID |        Technique         |      Parameters          |
>     +----+------------------------+--------------------------+
>     | 1  | Systematic count-based | flowSamplingInterval     |
>     |    | Sampling               | flowSamplingSpacing      |
>     +----+------------------------+--------------------------+
>     | 2  | Systematic time-based  | flowSamplingTimeInterval |
>     |    | Sampling               | flowSamplingTimeSpacing  |
>     +----+------------------------+--------------------------+
>     | 3  | Random n-out-of-N      | samplingSize             |
>     |    | Sampling               | samplingPopulation       |
>     +----+------------------------+--------------------------+
>     | 4  | Uniform probabilistic  | samplingProbability      |
>     |    | Sampling               |                          |
>     +----+------------------------+--------------------------+
>     | 5  | Property Match         | Information Element      |
>     |    | Filtering              | Value Range              |
>     +----+------------------------+--------------------------+
>     |   Hash-based Filtering      | hashInitialiserValue     |
>     +----+------------------------+ hashFlowDomain           |
>     | 6  | using BOB              | hashSelectedRangeMin     |
>     +----+------------------------+ hashSelectedRangeMax     |
>     | 7  | using IPSX             | hashOutputRangeMin       |
>     +----+------------------------+ hashOutputRangeMax       |
>     | 8  | using CRC              |                          |
>     +----+------------------------+--------------------------+
>     | 9  | Flow-state Dependent   | No agreed Parameters     |
>     |    | Flow Selection         |                          |
>     +----+------------------------+--------------------------+
>
> Also, in this table above, you need "TBDx" instead of 9
> - I see "Flow Selection", but this term is not defined.
>
> Thanks.
>
>
> Regards, Benoit.
>
>
> _______________________________________________
> IPFIX mailing list
> IPFIX@ietf.org
> https://www.ietf.org/mailman/listinfo/ipfix


--------------030704040609050402090604
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit


  
    
  
  
    
Dear
      draft-ietf-ipfix-flow-selection-tech authors,

      

      I was expecting a quick discussion on the very few remaining
      issues...

      That has not happened.

      The draft status is: AD evaluation, Revised ID Needed.

      

      Regards, Benoit

      

    

    

      
      
Dear authors,

        

        The draft improved quite dramatically. 

        Thanks for that.

        See in line for some more comments. I removed all unnecessary
        text.

        

      

      

        
        Dear authors,

        

        I'm performing the (new) AD review of 
        draft-ietf-ipfix-flow-selection-tech-10.txt

        Lucky you, an extra pair of eyes specifically looking at your
        draft  

        

        If some points have been discussed already on the mailing list,
        let me know. I have to admit that I have not been following the
        latest iterations of this draft.

        

        IMHO, this document needs some more work... 

        I don't think that this document is really in line with the
        other Intermediate Processes documents: 

            http://tools.ietf.org/html/rfc6235

            http://tools.ietf.org/html/draft-ietf-ipfix-a9n-03

        Note that I might have some more comments once all the points in
        this email are addressed, as there are many ;-)

        However, I'm available for a conf. call to clarify my points if
        you want to 

        

        See in-line. 

      

      ...

      

        

               8.2.  Registration of Object Identifier  . . . . . . . .
          . . . . 32 

             9.  Security Considerations  . . . . . . . . . . . . . . .
          . . . . 32 

             10. Acknowledgments  . . . . . . . . . . . . . . . . . . .
          . . . . 34 

             11. References . . . . . . . . . . . . . . . . . . . . . .
          . . . . 34 

               11.1. Normative References . . . . . . . . . . . . . . .
          . . . . 34 

               11.2. Informative References . . . . . . . . . . . . . .
          . . . . 34 

             Authors' Addresses . . . . . . . . . . . . . . . . . . . .
          . . . . 35 

        

        Don't you have to include the non-normative XML in the appendix,
        as it was done for RFC5102, RFC5103?
        
 

        

        

        
 2. 
          Terminology 

          

             This document is consistent with the terminology introduced
          in 

             [RFC5101], [RFC5470], [RFC5475] and [RFC3917].  As in
          [RFC5101] and 

             [RFC5476], the first letter of each IPFIX-specific and
          PSAMP-specific 

             term is capitalized along with the flow selection specific
          terms 

             defined here. 

        

        
 

             * Packet Classification 

          

                Packet Classification is a process by which packets are
          mapped to 

                specific Flow Records based on packet properties or
          external 

                properties (e.g. interface).  The properties (e.g.
          header 

                information, packet content, AS number) make up the Flow
          Key. In 

                case a Flow Record for a specific Flow Key already
          exists the Flow 

                Record is updated, otherwise a new Flow Record is
          created. 

        

        

        How is this different that the Metering Process (RFC5101)?

        
   Metering Process

      The Metering Process generates Flow Records.  Inputs to the
      process are packet headers and characteristics observed at an
      Observation Point, and packet treatment at the Observation Point
      (for example, the selected output interface).

      The Metering Process consists of a set of functions that includes
      packet header capturing, timestamping, sampling, classifying, and
      maintaining Flow Records.

      The maintenance of Flow Records may include creating new records,
      updating existing ones, computing Flow statistics, deriving
      further Flow properties, detecting Flow expiration, passing Flow
      Records to the Exporting Process, and deleting Flow Records.

        What is the connection with the Metering Process?

        Figure 1 seems to suggest that Packet Classification is a subset
        of the Metering Process...

      

      not sure that one was answered.

      

      
 

        

        
 

             * Packet Aggregation Process 

          

                In the IPFIX Metering Process the Packet Aggregation
          Process 

                aggregates packet data into flow data and forms the Flow
          Records. 

        

        How is this different from the Metering Process?

      

      the "Packet Aggregation Process" is not used in the document. Why
      do we need it?

      

      

        
     

          After the aggregation step only the aggregated flow
          information is 

                available.  Information about individual packets is
          lost. 

          

          

          

        

        
Intermediate Flow Selection Process: an Intermediate Process as in
      [RFC6183] that ...



        

      

      The new definition improved a lot:

      
 * Intermediate Flow Selection Process

      An Intermediate Flow Selection Process takes Flow Records as its
      input and selects a subset of this set as its output.
      Intermediate Flow Selection Process is a more general concept than
      Intermediate Selection Process as defined in [RFC6183].  While an
      Intermediate Selection Process selects Flow Records from a
      sequence based upon criteria-evaluated Flow record values and
      passes only those Flow Records that match the criteria, an
      Intermediate Flow Selection Process selects Flow Records using
      selection criteria applicable to a larger set of Flow
      characteristics and information.

      But is there a reason why this definition can't be based on
      "intermediate Process" from RFC 6183:

      

        
Intermediate Process

      An Intermediate Process takes a record stream as its input from
      Collecting Processes, Metering Processes, IPFIX File Readers,
      other Intermediate Processes, or other record sources; performs
      some transformations on this stream based upon the content of each
      record, states maintained across multiple records, or other data
      sources; and passes the transformed record stream as its output to
      Exporting Processes, IPFIX File Writers, or other Intermediate
      Processes in order to perform IPFIX Mediation.  Typically, an
      Intermediate Process is hosted by an IPFIX Mediator.
      Alternatively, an Intermediate Process may be hosted by an
      Original Exporter.

      

      So 

      

      
 * Intermediate Flow Selection Process

      An Intermediate Flow Selection Process is an Intermediate Process as in
      [RFC6183] that takes Flow Records as its
      input and selects a subset of this set as its output.
      Intermediate Flow Selection Process is a more general concept than
      Intermediate Selection Process as defined in [RFC6183].  While an
      Intermediate Selection Process selects Flow Records from a
      sequence based upon criteria-evaluated Flow record values and
      passes only those Flow Records that match the criteria, an
      Intermediate Flow Selection Process selects Flow Records using
      selection criteria applicable to a larger set of Flow
      characteristics and information.

      

      

        
  

        

      

      

      Regarding terminology, I still some instances of "observation
      point". Should be "Observation Point"

      

      ...

      

      

        
 

          4.  Flow selection as a Function in the IPFIX Architecture 

             

        

      

      Thanks for your new figure 1.

      One editorial change: change the + in the left vertical line.

      

      
      +======|========================+      |
      |      |  Mediator              |      |
      +    +-V-------------------+    |      |
      |    | Collecting Process  |    |      |
      +    +---------------------+    |      |
      |    | Intermediate Flow   |    |      |
      |    | Selection Process   |    |      |
      +    +---------------------+    |      |
      |    |  Exporting Process  |    |      |
      +    +-|-------------------+    |      |
      +======|========================+      |
      

      

        
 

          5.1.  Flow Filtering 

          

             Flow Filtering is a deterministic function on the IPFIX
          Flow Record 

             content.  If the relevant flow characteristics are already
          observable 

             at packet level (e.g.  Flow Keys), Flow Filtering can be
          applied 

             before aggregation at packet level.  In order to be
          compliant with 

             this document, at least the Property Match Filtering MUST
          be 

             implemented. 

        

        This contradicts.

        
   In order to be compliant with this document, at
   least one of the flow selection schemes MUST be implemented.

      

      Actually, wrong cut/paste.

      This contradicts, in section 1:

      
   In order to be compliant with this document, at
   least the Property Match Filtering MUST be implemented.

      

      

      

        
 

          8.  IANA Considerations 

          

          8.1.  Registration of Information Elements 

        

      

      

      Table 3: Information Elements to be registered, you can't put the
      value 1, 2, 3, 

      You need TBD1, TBD2, etc...

      And you must add

      "IANA Note: please replace TBD1, TBD2, ... with the assigned
      values, throughout the document."

      

      

        
 

          

          8.2.  Registration of Object Identifier 

          

        

      

      

      RFC 5815 is obsoleted by RFC 6615

        

        What you want is an extra in http://www.iana.org/assignments/smi-numbers,
        pointing to this RFC:

      
      

        
Sub-registry Name: IPFIX-SELECTOR-MIB Functions
Reference: [RFC6615]
Registration Procedures: Expert Review 
Prefix: iso.org.dod.internet.mgmt.mib-2.ipfixSelectorMIB.ipfixSelectorObjects.ipfixSelectorFunctions 
(1.3.6.1.2.1.194.1.1)

Decimal Name                  Description                       Reference
------- --------------------- --------------------------------- ---------
1       ipfixFuncSelectAll    Select everything                 [RFC6615]
2       psampSampCountBased   Systematic Count-based Sampling   [RFC6727]
3       psampSampTimeBased    Systematic Time-based Sampling    [RFC6727]
4       psampSampRandOutOfN   Random n-out-of-N Sampling        [RFC6727]
5       psampSampUniProb      Universal Probabilistic Sampling  [RFC6727]
6       psampFiltPropMatch    Property Match Filtering          [RFC6727]
7       psampFiltHash         Hash-based Filtering              [RFC6727]

      

      So you need TBDx

      

      
   +---------+-----------------------+---------------------+-----------+
   | Decimal | Name                  | Description         | Reference |
   +---------+-----------------------+---------------------+-----------+
   |  TBDx   | flowSelectorAlgorithm | This Object         | [RFCyyyy] |
   |         |                       | Identifier          |           |
   |         |                       | identifies the Flow |           |
   |         |                       | selection technique |           |
   |         |                       | (e.g., Filtering,   |           |
   |         |                       | Sampling) that is   |           |
   |         |                       | applied by the Flow |           |
   |         |                       | Selection Process   |           |
   +---------+-----------------------+---------------------+-----------+

               Table 4: Object Identifiers to be registered

      

      "IANA Note: please replace TBDx with the assigned value,
      throughout the document."

      

      Btw, there is a mismatch between the IANA registry and the table
      in section 7.1:

      
   +----+------------------------+--------------------------+
   | ID |        Technique         |      Parameters          |
   +----+------------------------+--------------------------+
   | 1  | Systematic count-based | flowSamplingInterval     |
   |    | Sampling               | flowSamplingSpacing      |
   +----+------------------------+--------------------------+
   | 2  | Systematic time-based  | flowSamplingTimeInterval |
   |    | Sampling               | flowSamplingTimeSpacing  |
   +----+------------------------+--------------------------+
   | 3  | Random n-out-of-N      | samplingSize             |
   |    | Sampling               | samplingPopulation       |
   +----+------------------------+--------------------------+
   | 4  | Uniform probabilistic  | samplingProbability      |
   |    | Sampling               |                          |
   +----+------------------------+--------------------------+
   | 5  | Property Match         | Information Element      |
   |    | Filtering              | Value Range              |
   +----+------------------------+--------------------------+
   |   Hash-based Filtering      | hashInitialiserValue     |
   +----+------------------------+ hashFlowDomain           |
   | 6  | using BOB              | hashSelectedRangeMin     |
   +----+------------------------+ hashSelectedRangeMax     |
   | 7  | using IPSX             | hashOutputRangeMin       |
   +----+------------------------+ hashOutputRangeMax       |
   | 8  | using CRC              |                          |
   +----+------------------------+--------------------------+
   | 9  | Flow-state Dependent   | No agreed Parameters     |
   |    | Flow Selection         |                          |
   +----+------------------------+--------------------------+

Also, in this table above, you need "TBDx" instead of 9


      - I see "Flow Selection", but this term is
        not defined.

        

        Thanks.

      

      

      Regards, Benoit.

      

      

      

      
_______________________________________________
IPFIX mailing list
IPFIX@ietf.org
https://www.ietf.org/mailman/listinfo/ipfix


    

    

  


--------------030704040609050402090604--

From iesg-secretary@ietf.org  Tue Oct 30 15:27:54 2012
Return-Path: 
X-Original-To: ipfix@ietfa.amsl.com
Delivered-To: ipfix@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DAA7321F85E1; Tue, 30 Oct 2012 15:27:54 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -102.559
X-Spam-Level: 
X-Spam-Status: No, score=-102.559 tagged_above=-999 required=5 tests=[AWL=0.040, BAYES_00=-2.599, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ie+qRfZGiktx; Tue, 30 Oct 2012 15:27:54 -0700 (PDT)
Received: from ietfa.amsl.com (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4376B21F8422; Tue, 30 Oct 2012 15:27:54 -0700 (PDT)
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
From: The IESG 
To: IETF-Announce 
X-Test-IDTracker: no
X-IETF-IDTracker: 4.35
Message-ID: <20121030222754.8033.44395.idtracker@ietfa.amsl.com>
Date: Tue, 30 Oct 2012 15:27:54 -0700
Cc: ipfix@ietf.org
Subject: [IPFIX] Last Call:  (Flow Aggregation for the IP	Flow Information Export (IPFIX) Protocol) to Proposed Standard
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
Reply-To: ietf@ietf.org
List-Id: IPFIX WG discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Tue, 30 Oct 2012 22:27:55 -0000

The IESG has received a request from the IP Flow Information Export WG
(ipfix) to consider the following document:
- 'Flow Aggregation for the IP Flow Information Export (IPFIX) Protocol'
   as Proposed Standard

The IESG plans to make a decision in the next few weeks, and solicits
final comments on this action. Please send substantive comments to the
ietf@ietf.org mailing lists by 2012-11-13. Exceptionally, comments may be
sent to iesg@ietf.org instead. In either case, please retain the
beginning of the Subject line to allow automated sorting.

Abstract


   This document provides a common implementation-independent basis for
   the interoperable application of the IP Flow Information Export
   (IPFIX) Protocol to the handling of Aggregated Flows, which are IPFIX
   Flows representing packets from multiple Original Flows sharing some
   set of common properties.  It does this through a detailed
   terminology and a descriptive Intermediate Aggregation Process
   architecture, including a specification of methods for Original Flow
   counting and counter distribution across intervals.




The file can be obtained via
http://datatracker.ietf.org/doc/draft-ietf-ipfix-a9n/

IESG discussion can be tracked via
http://datatracker.ietf.org/doc/draft-ietf-ipfix-a9n/ballot/


The following IPR Declarations may be related to this I-D:

   http://datatracker.ietf.org/ipr/1715/
   http://datatracker.ietf.org/ipr/1726/




From paitken@cisco.com  Tue Oct 30 15:28:26 2012
Return-Path: 
X-Original-To: ipfix@ietfa.amsl.com
Delivered-To: ipfix@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9842221F8608 for ; Tue, 30 Oct 2012 15:28:26 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -9.582
X-Spam-Level: 
X-Spam-Status: No, score=-9.582 tagged_above=-999 required=5 tests=[AWL=-0.349, BAYES_00=-2.599, FF_IHOPE_YOU_SINK=2.166, GB_I_LETTER=-2, J_CHICKENPOX_33=0.6, J_CHICKENPOX_37=0.6, RCVD_IN_DNSWL_HI=-8]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id IM3-g4y5D73W for ; Tue, 30 Oct 2012 15:28:22 -0700 (PDT)
Received: from ams-iport-1.cisco.com (ams-iport-1.cisco.com [144.254.224.140]) by ietfa.amsl.com (Postfix) with ESMTP id 8190B21F8433 for ; Tue, 30 Oct 2012 15:28:19 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=75965; q=dns/txt; s=iport; t=1351636100; x=1352845700; h=message-id:date:from:mime-version:to:cc:subject: references:in-reply-to:content-transfer-encoding; bh=wZ5j+Te+uumzE5wzh6lsSoZigeEZb3wWGnpjvOuFbW8=; b=Mzz+9I1KJQVDP/9lxaQMQKtlpTHX6NgADSVyJsP48E27fUoJsBOEaPYi 4Hmb0PURtPqWfbwaAZh9/RcPpCgC4B9euw3VihixJL7qb5KUQM9zhxhK/ 7sjawijUuoMjpTSWNxed4TSln3D5vyt/nLz9xMESqlBm64eRcZ/JP7P4N A=;
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: AgAFAElUkFCQ/khN/2dsb2JhbAA6CsNngQiCHgEBAQMBEgEHAQIbLwQFAgYBBQkCCxIPFg8JAwIBAgEJLg4GAQwBBQIBAQUZhW6BcAYLnDGPZ5AoBItzCQcFhkgDkkODMoEahE+IboFrgm+BWwEIFw
X-IronPort-AV: E=Sophos;i="4.80,683,1344211200"; d="scan'208";a="146045918"
Received: from ams-core-4.cisco.com ([144.254.72.77]) by ams-iport-1.cisco.com with ESMTP; 30 Oct 2012 22:28:14 +0000
Received: from [10.55.92.151] (dhcp-10-55-92-151.cisco.com [10.55.92.151]) by ams-core-4.cisco.com (8.14.5/8.14.5) with ESMTP id q9UMSBqD009747; Tue, 30 Oct 2012 22:28:11 GMT
Message-ID: <5090547C.5020803@cisco.com>
Date: Tue, 30 Oct 2012 22:28:12 +0000
From: Paul Aitken 
User-Agent: Mozilla/5.0 (X11; Linux i686; rv:15.0) Gecko/20120912 Thunderbird/15.0.1
MIME-Version: 1.0
To: IPFIX Working Group , Brian Trammell 
References: <506CBFE3.10607@auckland.ac.nz>
In-Reply-To: <506CBFE3.10607@auckland.ac.nz>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Cc: Nevil Brownlee 
Subject: Re: [IPFIX] WG Last Call for draft-ietf-ipfix-information-model-rfc5102bis-05.txt
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: IPFIX WG discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Tue, 30 Oct 2012 22:28:26 -0000

Nevil, All,

> The WG Last Call for this I-D starts now, and will run until Monday, 
> 22 October.

Sorry I'm late for -05, but hopefully still in time for -06? :-)

So here's a 110% review of rfc5102bis-06.


There are some nits:

   == Missing Reference: 'RFC5101' is mentioned on line 164, but not defined

   == Outdated reference: A later version (-02) exists of
      draft-ietf-ipfix-protocol-rfc5101bis-00

   -- Possible downref: Normative reference to a draft: ref. 'RFC5101bis'

   == Outdated reference: A later version (-07) exists of
      draft-ietf-ipfix-ie-doctors-00

   == Outdated reference: draft-ietf-ipfix-configuration-model has been
      published as RFC 6728

   == Outdated reference: A later version (-02) exists of
      draft-ietf-ipfix-mediation-protocol-00

   == Outdated reference: draft-ietf-ipfix-rfc5815bis has been published as
      RFC 6615


Some IEs are missing from this document, although they are defined in 
IANA's IPFIX registry:

     All the IE's from section 5.8. "Min/Max Flow Properties" of [5102] 
are missing:

         6    tcpControlBits
         25    minimumIpTotalLength
         26    maximumIpTotalLength
         52    minimumTTL
         53    maximumTTL
         64    ipv6ExtensionHeaders
         208    ipv4Options
         209    tcpOptions


     This 5103 IE is missing:

         239    biflowDirection


     The following IEs defined by cisco are listed by IANA, but not in 
this text:

         82    interfaceName
         83    interfaceDescription
         91    mplsTopLabelPrefixLength
         98    postIpDiffServCodePoint
         99    multicastReplicationFactor
         105-127
         225-236
         240+


     The following IEs are not mentioned here, although they are 
detailed in draft-yourtchenko-cisco-ies :

         3, 34, 35, 38, 39, 43, 48-51, 65-69, 84, 87, 89, 92-93, 100, 
101, 102, 103, 104, 94-97.


Please find specific feedback inline:



> Network Working Group                                     B. Claise, Ed.
> Internet Draft                                       Cisco Systems, Inc.
> Obsoletes: 5102                                         B. Trammell, Ed.
> Category: Standards Track                                     ETH Zurich
> Expires: April 6, 2013                                   October 3, 2012
>
>
>          Information Model for IP Flow Information eXport (IPFIX)
>            draft-ietf-ipfix-information-model-rfc5102bis-06.txt
>                                      
>
> Abstract
>
> This document provides an overview of the information model for the IP
> Flow Information eXport (IPFIX) protocol, as defined in the IANA IPFIX
> Information Element Registry. It is used by the IPFIX Protocol for
> encoding measured traffic information and information related to the
> traffic Observation Point, the traffic Metering Process, and the
> Exporting Process. Although developed for the IPFIX Protocol, the model
> is defined in an open way that easily allows using it in other
> protocols, interfaces, and applications. This document obsoletes RFC
> 5102.
>
> Status of This Memo
>
>     This Internet-Draft is submitted in full conformance with the
>     provisions of BCP 78 and BCP 79.
>
>     Internet-Drafts are working documents of the Internet Engineering
>     Task Force (IETF). Note that other groups may also distribute working
>     documents as Internet-Drafts. The list of current Internet-Drafts is
>     at http://datatracker.ietf.org/drafts/current/.
>
>     Internet-Drafts are draft documents valid for a maximum of six months
>     and may be updated, replaced, or obsoleted by other documents at any
>     time. It is inappropriate to use Internet-Drafts as reference
>     material or to cite them other than as "work in progress."
>
>     This Internet-Draft will expire on March 23, 2012.
>
> Copyright Notice
>
>     Copyright (c) 2012 IETF Trust and the persons identified as the
>     document authors. All rights reserved.
>
>     This document is subject to BCP 78 and the IETF Trust's Legal
>     Provisions Relating to IETF Documents
>     (http://trustee.ietf.org/license-info) in effect on the date of
>   
>
>
> Claise, Trammell            Standards Track                     [Page 1]
> 
> Internet-Draft          IPFIX Information Model          October 3, 2012
>
>
>     publication of this document. Please review these documents
>     carefully, as they describe your rights and restrictions with respect
>     to this document. Code Components extracted from this document must
>     include Simplified BSD License text as described in Section 4.e of
>     the Trust Legal Provisions and are provided without warranty as
>     described in the Simplified BSD License.
>
>
> Table of Contents
>
>     1.  Introduction . . . . . . . . . . . . . . . . . . . . . . . . .  3
>       1.1. Changes since RFC 5102  . . . . . . . . . . . . . . . . . .  4
>       1.2. IPFIX Documents Overview  . . . . . . . . . . . . . . . . .  4
>     2.  Properties of IPFIX Protocol Information Elements  . . . . . .  5
>       2.1.  Information Element Specification Template . . . . . . . .  5
>       2.2.  Scope of Information Elements  . . . . . . . . . . . . . .  7
>       2.3.  Naming Conventions for Information Elements  . . . . . . .  8
>     3.  Type Space . . . . . . . . . . . . . . . . . . . . . . . . . .  8
>       3.1.  Abstract Data Types  . . . . . . . . . . . . . . . . . . .  9
>         3.1.1.  unsigned8  . . . . . . . . . . . . . . . . . . . . . .  9
>         3.1.2.  unsigned16 . . . . . . . . . . . . . . . . . . . . . .  9
>         3.1.3.  unsigned32 . . . . . . . . . . . . . . . . . . . . . .  9
>         3.1.4.  unsigned64 . . . . . . . . . . . . . . . . . . . . . .  9
>         3.1.5.  signed8  . . . . . . . . . . . . . . . . . . . . . . .  9
>         3.1.6.  signed16 . . . . . . . . . . . . . . . . . . . . . . .  9
>         3.1.7.  signed32 . . . . . . . . . . . . . . . . . . . . . . .  9
>         3.1.8.  signed64 . . . . . . . . . . . . . . . . . . . . . . .  9
>         3.1.9.  float32  . . . . . . . . . . . . . . . . . . . . . . . 10
>         3.1.10.  float64 . . . . . . . . . . . . . . . . . . . . . . . 10
>         3.1.11.  boolean . . . . . . . . . . . . . . . . . . . . . . . 10
>         3.1.12.  macAddress  . . . . . . . . . . . . . . . . . . . . . 10
>         3.1.13.  octetArray  . . . . . . . . . . . . . . . . . . . . . 10
>         3.1.14.  string  . . . . . . . . . . . . . . . . . . . . . . . 10
>         3.1.15.  dateTimeSeconds . . . . . . . . . . . . . . . . . . . 10
>         3.1.16.  dateTimeMilliseconds  . . . . . . . . . . . . . . . . 10
>         3.1.17.  dateTimeMicroseconds  . . . . . . . . . . . . . . . . 10
>         3.1.18.  dateTimeNanoseconds . . . . . . . . . . . . . . . . . 11
>         3.1.19.  ipv4Address . . . . . . . . . . . . . . . . . . . . . 11
>         3.1.20.  ipv6Address . . . . . . . . . . . . . . . . . . . . . 11
>       3.2.  Data Type Semantics  . . . . . . . . . . . . . . . . . . . 11
>         3.2.1.  quantity . . . . . . . . . . . . . . . . . . . . . . . 11
>         3.2.2.  totalCounter . . . . . . . . . . . . . . . . . . . . . 11
>         3.2.3.  deltaCounter . . . . . . . . . . . . . . . . . . . . . 12
>         3.2.4.  identifier . . . . . . . . . . . . . . . . . . . . . . 12
>         3.2.5.  flags  . . . . . . . . . . . . . . . . . . . . . . . . 12
>     4.  Information Element Identifiers  . . . . . . . . . . . . . . . 12
>       4.1.  NetFlow version 9 compatible Information Element
>             Identifiers  . . . . . . . . . . . . . . . . . . . . . . . 13
>   
>
>
> Claise, Trammell            Standards Track                     [Page 2]
> 
> Internet-Draft          IPFIX Information Model          October 3, 2012
>
>
>     5.  Information Element Categories . . . . . . . . . . . . . . . . 13
>       5.1.  Identifiers  . . . . . . . . . . . . . . . . . . . . . . . 14
>       5.3.  Metering and Exporting Process Statistics  . . . . . . . . 15
>       5.4.  IP Header Fields . . . . . . . . . . . . . . . . . . . . . 15
>       5.5.  Transport Header Fields  . . . . . . . . . . . . . . . . . 16
>       5.6.  Sub-IP Header Fields . . . . . . . . . . . . . . . . . . . 17
>       5.7.  Derived Packet Properties  . . . . . . . . . . . . . . . . 17
>       5.9.  Flow Timestamps  . . . . . . . . . . . . . . . . . . . . . 18
>       5.10.  Per-Flow Counters . . . . . . . . . . . . . . . . . . . . 18
>       5.11.  Miscellaneous Flow Properties . . . . . . . . . . . . . . 19
>       5.12.  Padding . . . . . . . . . . . . . . . . . . . . . . . . . 20
>     6.  Extending the Information Model  . . . . . . . . . . . . . . . 20
>     7.  IANA Considerations  . . . . . . . . . . . . . . . . . . . . . 21
>       7.1.  IPFIX Information Elements . . . . . . . . . . . . . . . . 21
>       7.2.  MPLS Label Type Identifier . . . . . . . . . . . . . . . . 21
>       7.3.  XML Namespace and Schema . . . . . . . . . . . . . . . . . 22
>       7.4.  Addition, Revision, and Deprecation  . . . . . . . . . . . 23
>     8.  Security Considerations  . . . . . . . . . . . . . . . . . . . 23
>     9.  Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 24
>     10.  References  . . . . . . . . . . . . . . . . . . . . . . . . . 24
>       10.1.  Normative References  . . . . . . . . . . . . . . . . . . 24
>       10.2.  Informative References  . . . . . . . . . . . . . . . . . 24
>     Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 28
>
>
>
>
>
> 1.  Introduction
>
>     The IP Flow Information eXport (IPFIX) protocol serves for
>     transmitting information related to measured IP traffic over the

We also have non-IP traffic, eg MPLS and layer 2 information.


>     Internet.  The protocol specification in [RFC5101bis] defines how

Not just "over the Internet", but any IP network.


>     Information Elements are transmitted.  For Information Elements, it
>     specifies the encoding of a set of basic data types.  However, the
>     list of Information Elements that can be transmitted by the protocol,
>     such as Flow attributes (source IP address, number of packets, etc.)
>     and information about the Metering and Exporting Process (packet
>     Observation Point, sampling rate, Flow timeout interval, etc.), is
>     not specified in [RFC5101bis].
>
>     The canonical reference for IPFIX Information Elements the IANA IPFIX
>     Information Element registry [IPFIX-IANA]; the initial values for
>     this registry were provided by [RFC5102].
>
>     This document complements the IPFIX protocol specification by

"the IPFIX protocol specification in [RFC5101bis] by"


>     providing an overview of the IPFIX information model and specifying
>     data types for it. IPFIX-specific terminology used in this document
>   
>
>
> Claise, Trammell            Standards Track                     [Page 3]
> 
> Internet-Draft          IPFIX Information Model          October 3, 2012
>
>
>     is defined in Section 2 of [RFC5101bis]. As in [RFC5101bis], these
>     IPFIX-specific terms have the first letter of a word capitalized when
>     used in this document.
>
>     The use of the term 'information model' is not fully in line with the
>     definition of this term in [RFC3444].  The IPFIX information model
>     does not specify relationships between Information Elements, but also
>     it does not specify a concrete encoding of Information Elements.

The IPFIX encoding is specified in 5101.


>     Besides the encoding used by the IPFIX protocol, other encodings of
>     IPFIX Information Elements can be applied, for example, XML-based
>     encodings.
>
>     The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
>     "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
>     document are to be interpreted as described in [RFC2119].
>
> 1.1. Changes since RFC 5102
>
>     This document obsoletes the Proposed Standard revision of the IPFIX
>     Protocol Specification [RFC5102].  The following changes have been
>     made to this document with respect to the previous document:
>
>        - All outstanding technical and editorial errata filed on the
>     [RFC5102] as of publication time have been corrected
>        - All references into [RFC5101] have been updated to [RFC5101bis],
>     reflecting changes in that document as necessary
>        - Information element definitions have been removed, as the
>     reference for these is now [IPFIX-IANA]; categorizations of
>     information elements as defines in [RFC5102] have been retained in

s/defines/defined/


>     section 5.
>        - The process for modifying [IPFIX-IANA] has been improved, and is
>     now described in [IPFIX-IE-DOCTORS]; Section 6 has been updated
>     accordingly, and a new section 7.3 gives IANA considerations for this
>     process.
>        - Definitions of timestamp data types have been clarified
>        - Appendices A and B have been removed

BTW, the indentation of that section makes it difficult to read. Can you 
get all the text - including the wrapped lines - to the right of the 
bullets?


>
> 1.2. IPFIX Documents Overview
>
>     The IPFIX protocol provides network administrators with access to IP
>     flow information.  The architecture for the export of measured IP
>     flow information out of an IPFIX Exporting Process to a Collecting

I'd be tempted to drop "IP" from "IP flow information" (x2).


>     Process is defined in [RFC5470], per the requirements defined in
>     [RFC3917].  The IPFIX specifications [RFC5101bis] document specifies

"The IPFIX specifications document [RFC5101bis]" ?


>     how IPFIX data records and templates are carried via a number of
>     transport protocols from IPFIX Exporting Processes to IPFIX
>     Collecting Processes.
>
>   
>
>
> Claise, Trammell            Standards Track                     [Page 4]
> 
> Internet-Draft          IPFIX Information Model          October 3, 2012
>
>
>     Four IPFIX optimizations/extensions are currently specified: a
>     bandwidth saving method for the IPFIX protocol in [RFC5473], an
>     efficient method for exporting bidirectional flow in [RFC5103], a

"bidirectional flows", plural.


>     method for the definition and export of complex data structures in
>     [RFC6313], and the specification of the Protocol for IPFIX Mediations
>     [IPFIX-MED-PROTO] based on the IPIFX Mediation Framework [RFC6183].

s/IPIFX/IPFIX/


>
>     IPFIX has a formal description of IPFIX Information Elements, their
>     name, type and additional semantic information, as specified in this
>     document, with the export of the Information Element types specified
>     in [RFC5610].
>
>     [IPFIX-CONF] specifies a data model for configuring and monitoring
>     IPFIX and PSAMP compliant devices using the NETCONF protocol, while
>     the [RFC5815bis] specifies a MIB module for monitoring.

- "the"


>
>     In terms of development, [RFC5153] provides guidelines for the
>     implementation and use of the IPFIX protocol, while [RFC5471]
>     provides guidelines for testing.
>
>     Finally, [RFC5472] describes what type of applications can use the
>     IPFIX protocol and how they can use the information provided.  It
>     furthermore shows how the IPFIX framework relates to other
>     architectures and frameworks.
>
> 2.  Properties of IPFIX Protocol Information Elements
>
> 2.1.  Information Element Specification Template
>
>     Information in messages of the IPFIX protocol is modeled in terms of
>     Information Elements of the IPFIX information model. The IPFIX
>     Information Elements mentioned in Section 5 are specified in [IPFIX-
>     IANA]. For specifying these Information Elements, a template is used
>     that is described below.

At first I misunderstood "template" here. It's all in the context.


>
>     All Information Elements specified for the IPFIX protocol MUST have
>     the following properties defined:
>
>     name - A unique and meaningful name for the Information Element.
>
>     elementId - A numeric identifier of the Information Element.  If this
>        identifier is used without an enterprise identifier (see
>        [RFC5101bis] and enterpriseId below), then it is globally unique
>        and the list of allowed values is administered by IANA.  It is
>        used for compact identification of an Information Element when
>        encoding Templates in the protocol.
>
>     description - The semantics of this Information Element. Describes
>   
>
>
> Claise, Trammell            Standards Track                     [Page 5]
> 
> Internet-Draft          IPFIX Information Model          October 3, 2012
>
>
>        how this Information Element is derived from the Flow or other
>        information available to the observer. Information Elements of
>        dataType string or octetArray which have a length constraints

- "a"


>        (fixed length, minimum and/or maximum length) MUST note these
>        constraints in their description.
>
>     dataType - One of the types listed in Section 3.1 of this document or
>        registered in the IANA IPFIX Information Element Data Types
>        registry. The type space for attributes is constrained to
>        facilitate implementation. The existing type space does however
>        encompass most basic types used in modern programming languages,
>        as well as some derived types (such as ipv4Address) that are
>        common to this domain and useful to distinguish.

At first there seemed to be missing text at the end of this line.


>
>     status - The status of the specification of this Information Element.
>        Allowed values are 'current' and 'deprecated'. All newly-defined
>        Information Elements have 'current' status. The process for moving
>        Information Elements to the 'deprecated' status is defined in
>        Section 5.2 of [IPFIX-IE-DOCTORS].
>
>     Enterprise-specific Information Elements MUST have the following
>     property defined:
>
>     enterpriseId - Enterprises may wish to define Information Elements
>        without registering them with IANA, for example, for
>        enterprise-internal purposes.  For such Information Elements, the
>        Information Element identifier described above is not sufficient
>        when the Information Element is used outside the enterprise.  If
>        specifications of enterprise-specific Information Elements are
>        made public and/or if enterprise-specific identifiers are used by
>        the IPFIX protocol outside the enterprise, then the
>        enterprise-specific identifier MUST be made globally unique by
>        combining it with an enterprise identifier.  Valid values for the
>        enterpriseId are defined by IANA as Structure of Management
>        Information (SMI) network management private enterprise codes.
>        They are defined at http://www.iana.org/assignments/enterprise-
>        numbers.

Move the URL to an I-ref? It's mentioned again below.


>
>     All Information Elements specified for the IPFIX protocol either in
>     this document or by any future extension MAY have the following
>     properties defined:
>
>     dataTypeSemantics - The integral types may be qualified by additional
>        semantic details.  Valid values for the data type semantics are
>        specified in Section 3.2 of this document or in a future extension
>        of the information model.

Section 3.2.1 specifies that "quantity" is the default semantic, so this 
isn't really a MAY.
If semantics is an optional property, why does it have a default?


>
>     units - If the Information Element is a measure of some kind, the
>   
>
>
> Claise, Trammell            Standards Track                     [Page 6]
> 
> Internet-Draft          IPFIX Information Model          October 3, 2012
>
>
>        units identify what the measure is.
>
>     range - Some Information Elements may only be able to take on a
>        restricted set of values that can be expressed as a range (e.g., 0
>        through 511 inclusive).  If this is the case, the valid inclusive
>        range should be specified.

Should we make any comment on the invalidity of values outside the range?


>
>     reference - Identifies additional specifications that more precisely
>        define this item or provide additional context for its use.
>
>
>     The following two Information Element properties are defined to allow
>     the management of an Information Element registry with Information
>     Element definitions that may be updated over time, per the process
>     defined in Section 5.2 of [IPFIX-IE-DOCTORS].
>
>     revision - The revision number of an Information Element, starting at
>        0 for Information Elements at time of definition, and incremented
>        by one for each revision.
>
>     date - The date of the entry of this revision of the Information
>        Element into the registry.
>
>     For Information Elements of the string or octetArray data types which
>     have size limits (minimum and/or maximum size, or fixed length), the
>     limits MUST be defined within the description of the Information
>     Element.

This repeats earlier text in the "description" section.


>
> 2.2.  Scope of Information Elements
>
>     By default, most Information Elements have a scope specified in their
>     definitions.
>
>     o  The Information Elements listed in Sections 5.2 and 5.3, and
>        similar Information Elements in [IPFIX-IANA], have a default of "a
>        specific Metering Process" or of "a specific Exporting Process",
>        respectively.
>
>     o  The Information Elements listed in Sections 5.4-5.11, and similar
>        Information Elements in [IPFIX-IANA], have a scope of "a specific
>        Flow".
>
>     Within Data Records defined by Option Templates, the IPFIX protocol
>     allows further limiting of the Information Element scope.  The new
>     scope is specified by one or more scope fields and defined as the
>     combination of all specified scope values; see Section 3.4.2.1 on
>     IPFIX scopes in [RFC5101bis].
>
>   
>
>
> Claise, Trammell            Standards Track                     [Page 7]
> 
> Internet-Draft          IPFIX Information Model          October 3, 2012
>
>
> 2.3.  Naming Conventions for Information Elements
>
>     The following naming conventions were used for naming Information
>     Elements in this document.  It is recommended that extensions of the
>     model use the same conventions.
>
>     o  Names of Information Elements SHOULD be descriptive.
>
>     o  Names of Information Elements MUST be unique within the IANA
>        registry.   Enterprise-specific Information Elements SHOULD be
>        prefixed with a vendor name.

Unique within IANA's *IPFIX* registry.
Add xref to the registry.


>
>     o  Names of Information Elements MUST start with non-capitalized
>        letters.
>
>     o  Composed names MUST use capital letters for the first letter of
>        each component (except for the first one).  All other letters are
>        non-capitalized, even for acronyms.  Exceptions are made for
>        acronyms containing non-capitalized letters, such as 'IPv4' and
>        'IPv6'.  Examples are sourceMacAddress and destinationIPv4Address.

Combination of the above rules means that IANA will name an IE "foo", 
while the ES equivalent is named "enterpriseFoo".
It's unfortunate that "foo" != "Foo".


>
>     o  Middleboxes [RFC3234] may change Flow properties, such as the
>        Differentiated Service Code Point (DSCP) value or the source IP
>        address.  If an IPFIX Observation Point is located in the path of
>        a Flow before one or more middleboxes that potentially modify
>        packets of the Flow, then it may be desirable to also report Flow
>        properties after the modification performed by the middleboxes.
>        An example is an Observation Point before a packet marker changing
>        a packet's IPv4 Type of Service (TOS) field that is encoded in
>        Information Element ipClassOfService.  Then the value observed and
>        reported by Information Element ipClassOfService is valid at the
>        Observation Point, but not after the packet passed the packet
>        marker.  For reporting the change value of the TOS field, the
>        IPFIX information model uses Information Elements that have a name
>        prefix "post", for example, "postIpClassOfService".  Information
>        Elements with prefix "post" report on Flow properties that are not
>        necessarily observed at the Observation Point, but which are
>        obtained within the Flow's Observation Domain by other means
>        considered to be sufficiently reliable, for example, by analyzing
>        the packet marker's marking tables.
>
> 3.  Type Space
>
>     This section describes the abstract data types that can be used for
>     the specification of IPFIX Information Elements in Section 4.
>     Section 3.1 describes the set of abstract data types.
>
>     Abstract data types unsigned8, unsigned16, unsigned32, unsigned64,
>   
>
>
> Claise, Trammell            Standards Track                     [Page 8]
> 
> Internet-Draft          IPFIX Information Model          October 3, 2012
>
>
>     signed8, signed16, signed32, and signed64 are integral data types.

They're just different sizes of the same types.


>     As described in Section 3.2, their data type semantics can be further
>     specified, for example, by 'totalCounter', 'deltaCounter',
>     'identifier', or 'flags'.
>
> 3.1.  Abstract Data Types
>
>     This section describes the set of valid abstract data types of the
>     IPFIX information model.  Note that further abstract data types may
>     be specified by future extensions of the IPFIX information model.
>
> 3.1.1.  unsigned8
>
>     The type "unsigned8" represents a non-negative integer value in the
>     range of 0 to 255.
>
> 3.1.2.  unsigned16
>
>     The type "unsigned16" represents a non-negative integer value in the
>     range of 0 to 65535.
>
> 3.1.3.  unsigned32
>
>     The type "unsigned32" represents a non-negative integer value in the
>     range of 0 to 4294967295.
>
> 3.1.4.  unsigned64
>
>     The type "unsigned64" represents a non-negative integer value in the
>     range of 0 to 18446744073709551615.
>
> 3.1.5.  signed8
>
>     The type "signed8" represents an integer value in the range of -128
>     to 127.
>
> 3.1.6.  signed16
>
>     The type "signed16" represents an integer value in the range of
>     -32768 to 32767.
>
> 3.1.7.  signed32
>
>     The type "signed32" represents an integer value in the range of
>     -2147483648 to 2147483647.
>
> 3.1.8.  signed64
>
>   
>
>
> Claise, Trammell            Standards Track                     [Page 9]
> 
> Internet-Draft          IPFIX Information Model          October 3, 2012
>
>
>     The type "signed64" represents an integer value in the range of
>     -9223372036854775808 to 9223372036854775807.
>
> 3.1.9.  float32
>
>     The type "float32" corresponds to an IEEE single-precision 32-bit
>     floating point type as defined in [IEEE.754.1985].
>
> 3.1.10.  float64
>
>     The type "float64" corresponds to an IEEE double-precision 64-bit
>     floating point type as defined in [IEEE.754.1985].
>
> 3.1.11.  boolean
>
>     The type "boolean" represents a binary value.  The only allowed
>     values are "true" and "false".
>
> 3.1.12.  macAddress
>
>     The type "macAddress" represents a string of 6 octets.
>
> 3.1.13.  octetArray
>
>     The type "octetArray" represents a finite-length string of octets.
>
> 3.1.14.  string
>
>     The type "string" represents a finite-length string of valid
>     characters from the Unicode character encoding set
>     [ISO.10646-1.1993].  Unicode allows for ASCII [ISO.646.1991] and many
>     other international character sets to be used.
>
> 3.1.15.  dateTimeSeconds
>
>     The data type dateTimeSeconds is an unsigned 32-bit integer
>     representing the number of seconds since the UNIX epoch, 1 January
>     1970 at 00:00 UTC, as defined in [POSIX.1].
>
> 3.1.16.  dateTimeMilliseconds
>
>     The data type dateTimeMilliseconds is an unsigned 64-bit integer
>     containing the number of milliseconds since the UNIX epoch, 1 January
>     1970 at 00:00 UTC, as defined in [POSIX.1].
>
> 3.1.17.  dateTimeMicroseconds
>
>     The type "dateTimeMicroseconds" represents a time value with
>   
>
>
> Claise, Trammell            Standards Track                    [Page 10]
> 
> Internet-Draft          IPFIX Information Model          October 3, 2012
>
>
>     microsecond precision according to the NTP Timestamp format as
>     defined in section 6 of [RFC5905].
>
> 3.1.18.  dateTimeNanoseconds
>
>     The type "dateTimeNanoseconds" represents a time value with
>     nanosecond precision according to the NTP Timestamp format as defined
>     in section 6 of [RFC5905].
>
> 3.1.19.  ipv4Address
>
>     The type "ipv4Address" represents a value of an IPv4 address.

"a value", as if it has many values? What exactly is the "value" of an 
address?

Consider "The type "ipv4Address" represents an IPv4 address." ?


>
> 3.1.20.  ipv6Address
>
>     The type "ipv6Address" represents a value of an IPv6 address.

Similarly.


>
> 3.2.  Data Type Semantics
>
>     This section describes the set of valid data type semantics of the
>     IPFIX information model. A registry of data type semantics is
>     established in [RFC5610]; the restrictions on the use of semantics

Surely IANA is the reference point, rather than 5610?
eg, if new semantics are added, they'll be listed in IANA without 
raising errata against 5610.


>     below are compatible with those specified in section 3.10 of that
>     document. These semantics apply only to numeric types, as noted in
>     the description of each semantic below.
>
>     Further data type semantics may be specified by future extensions of
>     the IPFIX information model.

State the required 5226 action / process for that, eg expert review.
Or, xref the section where that's stated.


>
> 3.2.1.  quantity
>
>     A numeric (integral or floating point) value representing a measured
>     value pertaining to the record. This is distinguished from counters
>     that represent an ongoing measured value whose "odometer" reading is
>     captured as part of a given record. This is the default semantic type
>     of all numeric data types.
>
> 3.2.2.  totalCounter
>
>     An numeric value reporting the value of a counter. Counters are
>     unsigned and wrap back to zero after reaching the limit of the type.
>     For example, an unsigned64 with counter semantics will continue to
>     increment until reaching the value of 2**64 - 1. At this point, the
>     next increment will wrap its value to zero and continue counting from
>     zero. The semantics of a total counter is similar to the semantics of
>     counters used in SNMP, such as Counter32 defined in [RFC2578]. The
>     only difference between total counters and counters used in SNMP is
>     that the total counters have an initial value of 0. A total counter
>   
>
>
> Claise, Trammell            Standards Track                    [Page 11]
> 
> Internet-Draft          IPFIX Information Model          October 3, 2012
>
>
>     counts independently of the export of its value.
>
> 3.2.3.  deltaCounter
>
>     An numeric value reporting the value of a counter. Counters are
>     unsigned and wrap back to zero after reaching the limit of the type.
>     For example, an unsigned64 with counter semantics will continue to
>     increment until reaching the value of 2**64 - 1. At this point, the
>     next increment will wrap its value to zero and continue counting from
>     zero. The semantics of a delta counter is similar to the semantics of
>     counters used in SNMP, such as Counter32 defined in RFC 2578
>     [RFC2578]. The only difference between delta counters and counters
>     used in SNMP is that the delta counters have an initial value of 0. A
>     delta counter is reset to 0 each time its value is exported.

What if the cache entry is removed but not exported (eg, an export 
filter blocks the export) ?
Then the counter was not exported, so it should not be reset to 0?

ie, the reset action is more to do with the cache entry expiring than 
whatever happens to it next.


>
> 3.2.4.  identifier
>
>     An integral value that serves as an identifier. Specifically,
>     mathematical operations on two identifiers (aside from the equality
>     operation) are meaningless. For example, Autonomous System ID 1 *
>     Autonomous System ID 2 is meaningless. Identifiers MUST be one of the
>     signed or unsigned data types.

We could also have non-numeric identifiers, eg wlanSSID is a string 
identifier.


>
> 3.2.5.  flags
>
>     An integral value that represents a set of bit fields. Logical
>     operations are appropriate on such values, but not other mathematical
>     operations. Flags MUST always be of an unsigned data type.
>
> 4.  Information Element Identifiers
>
>     All Information Elements defined in the IANA IPFIX Information
>     Element registry [IPFIX-IANA] have their identifiers assigned by
>     IANA.
>
>     The value of these identifiers is in the range of 1-32767. Within
>     this range, Information Element identifier values in the sub-range of
>     1-127 are compatible with field types used by NetFlow version 9
>     [RFC3954]; Information Element identifiers in this range MUST NOT be
>     assigned unless the Information Element is compatible with the
>     NetFlow version 9 protocol. Such Information Elements may ONLY be
>     requested by a NetFlow v9 expert, to be designated by the IESG.
>
>     In general, IANA will add newly registered Information Elements to
>     the registry, assigning the lowest available Information Element
>     identifier in the range 128-32767.
>
>     Enterprise-specific Information Element identifiers have the same
>   
>
>
> Claise, Trammell            Standards Track                    [Page 12]
> 
> Internet-Draft          IPFIX Information Model          October 3, 2012
>
>
>     range of 1-32767, but they are coupled with an additional enterprise
>     identifier. For enterprise-specific Information Elements, Information
>     Element identifier 0 is also reserved. Enterprise-specific
>     Information Element identifiers can be chosen by an enterprise
>     arbitrarily within the range of 1-32767. The same identifier may be
>     assigned by other enterprises for different purposes; these
>     Information Elements are distinct because the Information Element
>     identifier is coupled with an enterprise identifier.
>
>     Enterprise identifiers MUST be registered as SMI network management
>     private enterprise code numbers with IANA.  The registry can be found
>     at http://www.iana.org/assignments/enterprise-numbers.

Add the URL as an I-ref per earlier comment.


>
> 4.1.  NetFlow version 9 compatible Information Element Identifiers
>
>     Information Elements with identifiers from 1-127 are reserved for
>     compatibility with corresponding fields in NetFlow version 9
>     [RFC3954].

This simply repeats the second paragraph of section 4 above.


>
>
> 5.  Information Element Categories
>
>     This section describes the Information Element category for the IPFIX
>     information model at the time that [RFC5102] was published. Since
>     this category field is not part of the IANA process for assigning new
>     Information Element (even though it has been reused, for example, in

s/Element/Elements/


>     [RFC5103]), the newest Information Elements in IANA [IPFIX-IANA]
>     don't have this classification. The elements are grouped into 12
>     groups according to their semantics and their applicability:

TBD: are categories useful? If not, let's say they're deprecated and not 
discuss them further.


>
>     1.   Identifiers
>     2.   Metering and Exporting Process Configuration
>     3.   Metering and Exporting Process Statistics
>     4.   IP Header Fields
>     5.   Transport Header Fields
>     6.   Sub-IP Header Fields
>     7.   Derived Packet Properties
>     8.   Min/Max Flow Properties
>     9.   Flow Timestamps
>     10.  Per-Flow Counters
>     11.  Miscellaneous Flow Properties
>     12.  Padding
>
>     The Information Elements that are derived from fields of packets or

s/fields of packets/packet fields/


>     from packet treatment, such as the Information Elements in groups
>     4-7, can typically serve as Flow Keys used for mapping packets to
>     Flows.
>
>   
>
>
> Claise, Trammell            Standards Track                    [Page 13]
> 
> Internet-Draft          IPFIX Information Model          October 3, 2012
>
>
>     If they do not serve as Flow Keys, their value may change from packet
>     to packet within a single Flow.  For Information Elements with values
>     that are derived from fields of packets or from packet treatment and

s/fields of packets/packet fields/


>     for which the value may change from packet to packet within a single
>     Flow, the IPFIX information model defines that their value is
>     determined by the first packet observed for the corresponding Flow,
>     unless the description of the Information Element explicitly
>     specifies a different semantics.  This simple rule allows writing all

I don't think it's appropriate for the infomodel to define this.

In some cases, the same IE may be observed in different ways according 
to the implementation.
By the above definition, we'd need multiple IEs.


>     Information Elements related to header fields once when the first
>     packet of the Flow is observed.  For further observed packets of the
>     same Flow, only Flow properties that depend on more than one packet,
>     such as the Information Elements in groups 8-11, need to be updated.

This model is based on an historic and simplistic understanding of the MP.

Today we may not be able to determine all the key fields until some 
variable number of packets have been observed.
eg, consider if a key field is in fragment N > 1.


>
>     Information Elements with a name having the "post" prefix, for
>     example, "postIpClassOfService", do not report properties that were
>     actually observed at the Observation Point, but retrieved by other
>     means within the Observation Domain.  These Information Elements can
>     be used if there are middlebox functions within the Observation
>     Domain changing Flow properties after packets passed the Observation
>     Point.

s/changing/which change/


>
>
> 5.1.  Identifiers
>
>     Information Elements grouped in the table below are identifying
>     components of the IPFIX architecture, of an IPFIX Device, or of the
>     IPFIX protocol.  All of them have an integral abstract data type and
>     data type semantics "identifier" as described in Section 3.2.4.
>
>     Typically, some of them are used for limiting scopes of other
>     Information Elements.  However, other Information Elements MAY be
>     used for limiting scopes.  Note also that all Information Elements
>     listed below MAY be used for other purposes than limiting scopes.
>
>     +-----+---------------------------+-----+---------------------------+
>     |  ID | Name                      |  ID | Name                      |
>     +-----+---------------------------+-----+---------------------------+
>     | 141 | lineCardId                | 148 | flowId                    |
>     | 142 | portId                    | 145 | templateId                |
>     |  10 | ingressInterface          | 149 | observationDomainId       |
>     |  14 | egressInterface           | 138 | observationPointId        |
>     | 143 | meteringProcessId         | 137 | commonPropertiesId        |
>     | 144 | exportingProcessId        |     |                           |
>     +-----+---------------------------+-----+---------------------------+
>
>     See [IPFIX-IANA] for the definitions of these Information Elements.

Instead of repeating this over and over, just say it once in section 5. ?


>
>
>   
>
>
> Claise, Trammell            Standards Track                    [Page 14]
> 
> Internet-Draft          IPFIX Information Model          October 3, 2012
>
>
>     5.2.  Metering and Exporting Process Configuration
>
>     Information Elements in this section describe the configuration of
>     the Metering Process or the Exporting Process.  The set of these
>     Information Elements is listed in the table below.
>
>     +-----+--------------------------+-----+----------------------------+
>     |  ID | Name                     |  ID | Name                       |
>     +-----+--------------------------+-----+----------------------------+
>     | 130 | exporterIPv4Address      | 213 | exportInterface            |
>     | 131 | exporterIPv6Address      | 214 | exportProtocolVersion      |
>     | 217 | exporterTransportPort    | 215 | exportTransportProtocol    |
>     | 211 | collectorIPv4Address     | 216 | collectorTransportPort     |
>     | 212 | collectorIPv6Address     | 173 | flowKeyIndicator           |
>     +-----+--------------------------+-----+----------------------------+
>
>     See [IPFIX-IANA] for the definitions of these Information Elements.
>
>
>
> 5.3.  Metering and Exporting Process Statistics
>
>     Information Elements in this section describe statistics of the
>     Metering Process and/or the Exporting Process.  The set of these
>     Information Elements is listed in the table below.
>
>     +-----+-----------------------------+-----+-------------------------+
>     |  ID | Name                        |  ID | Name                    |
>     +-----+-----------------------------+-----+-------------------------+
>     |  41 | exportedMessageTotalCount   | 165 | ignoredOctetTotalCount  |
>     |  40 | exportedOctetTotalCount     | 166 | notSentFlowTotalCount   |
>     |  42 | exportedFlowRecordTotalCount| 167 | notSentPacketTotalCount |
>     | 163 | observedFlowTotalCount      | 168 | notSentOctetTotalCount  |
>     | 164 | ignoredPacketTotalCount     |     |                         |
>     +-----+-----------------------------+-----+-------------------------+
>
>     See [IPFIX-IANA] for the definitions of these Information Elements.
>
>
>
> 5.4.  IP Header Fields
>
>     Information Elements in this section indicate values of IP header
>     fields or are derived from IP header field values in combination with
>     further information.
>
>     +-----+----------------------------+-----+--------------------------+
>     |  ID | Name                       |  ID | Name                     |
>   
>
>
> Claise, Trammell            Standards Track                    [Page 15]
> 
> Internet-Draft          IPFIX Information Model          October 3, 2012
>
>
>     +-----+----------------------------+-----+--------------------------+
>     |  60 | ipVersion                  | 193 | nextHeaderIPv6           |
>     |   8 | sourceIPv4Address          | 195 | ipDiffServCodePoint      |
>     |  27 | sourceIPv6Address          | 196 | ipPrecedence             |
>     |   9 | sourceIPv4PrefixLength     |   5 | ipClassOfService         |
>     |  29 | sourceIPv6PrefixLength     |  55 | postIpClassOfService     |
>     |  44 | sourceIPv4Prefix           |  31 | flowLabelIPv6            |
>     | 170 | sourceIPv6Prefix           | 206 | isMulticast              |
>     |  12 | destinationIPv4Address     |  54 | fragmentIdentification   |
>     |  28 | destinationIPv6Address     |  88 | fragmentOffset           |
>     |  13 | destinationIPv4PrefixLength| 197 | fragmentFlags            |
>     |  30 | destinationIPv6PrefixLength| 189 | ipHeaderLength           |
>     |  45 | destinationIPv4Prefix      | 207 | ipv4IHL                  |
>     | 169 | destinationIPv6Prefix      | 190 | totalLengthIPv4          |
>     | 192 | ipTTL                      | 224 | ipTotalLength            |
>     |   4 | protocolIdentifier         | 191 | payloadLengthIPv6        |
>     +-----+----------------------------+-----+--------------------------+
>
>     See [IPFIX-IANA] for the definitions of these Information Elements.
>
>
>
> 5.5.  Transport Header Fields
>
>     The set of Information Elements related to transport header fields
>     and length includes the Information Elements listed in the table
>     below.
>
>     +-----+---------------------------+-----+---------------------------+
>     |  ID | Name                      |  ID | Name                      |
>     +-----+---------------------------+-----+---------------------------+
>     |   7 | sourceTransportPort       | 238 | tcpWindowScale            |
>     |  11 | destinationTransportPort  | 187 | tcpUrgentPointer          |
>     | 180 | udpSourcePort             | 188 | tcpHeaderLength           |
>     | 181 | udpDestinationPort        |  32 | icmpTypeCodeIPv4          |
>     | 205 | udpMessageLength          | 176 | icmpTypeIPv4              |
>     | 182 | tcpSourcePort             | 177 | icmpCodeIPv4              |
>     | 183 | tcpDestinationPort        | 139 | icmpTypeCodeIPv6          |
>     | 184 | tcpSequenceNumber         | 178 | icmpTypeIPv6              |
>     | 185 | tcpAcknowledgementNumber  | 179 | icmpCodeIPv6              |
>     | 186 | tcpWindowSize             |  33 | igmpType                  |
>     +-----+---------------------------+-----+---------------------------+
>
>     See [IPFIX-IANA] for the definitions of these Information Elements.
>
>
>
>
>   
>
>
> Claise, Trammell            Standards Track                    [Page 16]
> 
> Internet-Draft          IPFIX Information Model          October 3, 2012
>
>
> 5.6.  Sub-IP Header Fields
>
>     The set of Information Elements related to Sub-IP header fields
>     includes the Information Elements listed in the table below.
>
>     +-----+---------------------------+-----+---------------------------+
>     |  ID | Name                      |  ID | Name                      |
>     +-----+---------------------------+-----+---------------------------+
>     |  56 | sourceMacAddress          | 201 | mplsLabelStackLength      |
>     |  81 | postSourceMacAddress      | 194 | mplsPayloadLength         |
>     |  58 | vlanId                    |  70 | mplsTopLabelStackSection  |
>     |  59 | postVlanId                |  71 | mplsLabelStackSection2    |
>     |  80 | destinationMacAddress     |  72 | mplsLabelStackSection3    |
>     |  57 | postDestinationMacAddress |  73 | mplsLabelStackSection4    |
>     | 146 | wlanChannelId             |  74 | mplsLabelStackSection5    |
>     | 147 | wlanSSID                  |  75 | mplsLabelStackSection6    |
>     | 200 | mplsTopLabelTTL           |  76 | mplsLabelStackSection7    |
>     | 203 | mplsTopLabelExp           |  77 | mplsLabelStackSection8    |
>     | 237 | postMplsTopLabelExp       |  78 | mplsLabelStackSection9    |
>     | 202 | mplsLabelStackDepth       |  79 | mplsLabelStackSection10   |
>     +-----+---------------------------+-----+---------------------------+
>
>     See [IPFIX-IANA] for the definitions of these Information Elements.
>
>
> 5.7.  Derived Packet Properties
>
>     The set of Information Elements derived from packet properties (for
>     example, values of header fields) includes the Information Elements
>     listed in the table below.
>
>     +-----+---------------------------+-----+---------------------------+
>     |  ID | Name                      |  ID | Name                      |
>     +-----+---------------------------+-----+---------------------------+
>     | 204 | ipPayloadLength           |  18 | bgpNextHopIPv4Address     |
>     |  15 | ipNextHopIPv4Address      |  63 | bgpNextHopIPv6Address     |
>     |  62 | ipNextHopIPv6Address      |  46 | mplsTopLabelType          |
>     |  16 | bgpSourceAsNumber         |  47 | mplsTopLabelIPv4Address   |
>     |  17 | bgpDestinationAsNumber    | 140 | mplsTopLabelIPv6Address   |
>     | 128 | bgpNextAdjacentAsNumber   |  90 | mplsVpnRouteDistinguisher |
>     | 129 | bgpPrevAdjacentAsNumber   |     |                           |
>     +-----+---------------------------+-----+---------------------------+
>
>     See [IPFIX-IANA] for the definitions of these Information Elements.
>
>
>
>
>   
>
>
> Claise, Trammell            Standards Track                    [Page 17]
> 
> Internet-Draft          IPFIX Information Model          October 3, 2012
>
>
> 5.9.  Flow Timestamps
>
>     Information Elements in this section are timestamps of events.
>
>     Timestamps flowStartSeconds, flowEndSeconds, flowStartMilliseconds,
>     flowEndMilliseconds, flowStartMicroseconds, flowEndMicroseconds,
>     flowStartNanoseconds, flowEndNanoseconds, and
>     systemInitTimeMilliseconds are absolute and have a well-defined fixed
>     time base, such as, for example, the number of seconds since 0000 UTC
>     Jan 1st 1970.

It's a bit dangerous to give this example, since it could be misread as 
being the actual definition.
xref sections 3.1.15 - 3.1.18 where the time bases are stated.


>
>     Timestamps flowStartDeltaMicroseconds and flowEndDeltaMicroseconds
>     are relative timestamps only valid within the scope of a single
>     IPFIX Message.  They contain the negative time offsets relative to
>     the export time specified in the IPFIX Message Header.  The maximum

In order for the EP to populate *DeltaMicroseconds in a flow record, it 
must first know what Export Time it's going to stamp into the IPFIX 
header, and the flow record must be exported with that given second... 
unless we allow that data may be exported somewhat asynchronously to the 
header timestamping (eg, if there's a queue of outgoing packets at a 
level below the EP, eg in the IP stack). If a flow's export is delayed 
such that the Export Time changes, then these deltas must be 
recalculated. Practically, that may not be possible.

Anyway, why do we have microsecond offsets from a "seconds" time?

In short, these two IEs seem flawed and should be deprecated.


>     time offset that can be encoded by these delta counters is 1 hour, 11
>     minutes, and 34.967295 seconds.
>
>     Timestamps flowStartSysUpTime and flowEndSysUpTime are relative
>     timestamps indicating the time relative to the last
>     (re-)initialization of the IPFIX Device.  For reporting the time
>     of the last (re-)initialization, systemInitTimeMilliseconds can
>     be reported, for example, in Data Records defined by Option
>     Templates.
>
>     +-----+---------------------------+-----+---------------------------+
>     |  ID | Name                      |  ID | Name                      |
>     +-----+---------------------------+-----+---------------------------+
>     | 150 | flowStartSeconds          | 156 | flowStartNanoseconds      |
>     | 151 | flowEndSeconds            | 157 | flowEndNanoseconds        |
>     | 152 | flowStartMilliseconds     | 158 | flowStartDeltaMicroseconds|
>     | 153 | flowEndMilliseconds       | 159 | flowEndDeltaMicroseconds  |
>     | 154 | flowStartMicroseconds     | 160 | systemInitTimeMilliseconds|
>     | 155 | flowEndMicroseconds       |  22 | flowStartSysUpTime        |
>     |     |                           |  21 | flowEndSysUpTime          |
>     +-----+---------------------------+-----+---------------------------+
>
>     See [IPFIX-IANA] for the definitions of these Information Elements.
>
> 5.10.  Per-Flow Counters
>
>     Information Elements in this section are counters all having integer
>     values.  Their values may change for every report they are used in.
>     They cannot serve as part of a Flow Key used for mapping packets to
>     Flows.  However, potentially they can be used for selecting exported

Well, octetDeltaCount could be used to make all packets of the same size 
hash to the same bucket.

More realistically, these could be used when aggregating flows into 
other flows. eg, all the flows with the same number of packets, or all 
the flows with the same TCP SYN count.


>     Flows, for example, by only exporting Flows with more than a
>     threshold number of observed octets.
>
>   
>
>
> Claise, Trammell            Standards Track                    [Page 18]
> 
> Internet-Draft          IPFIX Information Model          October 3, 2012
>
>
>     There are running counters and delta counters.  Delta counters are
>     reset to zero each time their values are exported.  Running counters
>     continue counting independently of the Exporting Process.
>
>     There are per-Flow counters and counters related to the Metering
>     Process and/or the Exporting Process.  Per-Flow counters are Flow
>     properties that potentially change each time a packet belonging to
>     the Flow is observed.  The set of per-Flow counters includes the
>     Information Elements listed in the table below.  Counters related to
>     the Metering Process and/or the Exporting Process are described in
>     Section 5.3.
>
>     +-----+---------------------------+-----+---------------------------+
>     |  ID | Name                      |  ID | Name                      |
>     +-----+---------------------------+-----+---------------------------+
>     |   1 | octetDeltaCount           | 134 | droppedOctetTotalCount    |
>     |  23 | postOctetDeltaCount       | 135 | droppedPacketTotalCount   |
>     | 198 | octetDeltaSumOfSquares    |  19 | postMCastPacketDeltaCount |
>     |  85 | octetTotalCount           |  20 | postMCastOctetDeltaCount  |
>     | 171 | postOctetTotalCount       | 174 | postMCastPacketTotalCount |
>     | 199 | octetTotalSumOfSquares    | 175 | postMCastOctetTotalCount  |
>     |   2 | packetDeltaCount          | 218 | tcpSynTotalCount          |
>     |  24 | postPacketDeltaCount      | 219 | tcpFinTotalCount          |
>     |  86 | packetTotalCount          | 220 | tcpRstTotalCount          |
>     | 172 | postPacketTotalCount      | 221 | tcpPshTotalCount          |
>     | 132 | droppedOctetDeltaCount    | 222 | tcpAckTotalCount          |
>     | 133 | droppedPacketDeltaCount   | 223 | tcpUrgTotalCount          |
>     +-----+---------------------------+-----+---------------------------+
>
>
> See [IPFIX-IANA] for the definitions of these Information Elements.
>
>
> 5.11.  Miscellaneous Flow Properties
>
>     Information Elements in this section describe properties of Flows
>     that are related to Flow start, Flow duration, and Flow termination,
>     but they are not timestamps as the Information Elements in Section
>     5.9 are.
>
>     +-----+---------------------------+-----+---------------------------+
>     |  ID | Name                      |  ID | Name                      |
>     +-----+---------------------------+-----+---------------------------+
>     |  36 | flowActiveTimeout         | 161 | flowDurationMilliseconds  |
>     |  37 | flowIdleTimeout           | 162 | flowDurationMicroseconds  |
>     | 136 | flowEndReason             |  61 | flowDirection             |
>     +-----+---------------------------+-----+---------------------------+
>
>   
>
>
> Claise, Trammell            Standards Track                    [Page 19]
> 
> Internet-Draft          IPFIX Information Model          October 3, 2012
>
>
> See [IPFIX-IANA] for the definitions of these Information Elements.
>
>
>
> 5.12.  Padding
>
>     This section contains a single Information Element that can be used
>     for padding of Flow Records.
>
>     IPFIX implementations may wish to align Information Elements within
>     Data Records or to align entire Data Records to 4-octet or 8-octet
>     boundaries.  This can be achieved by including one or more
>     paddingOctets Information Elements in a Data Record.
>
>     +-----+---------------------------+-----+---------------------------+
>     |  ID | Name                      |  ID | Name                      |
>     +-----+---------------------------+-----+---------------------------+
>     | 210 | paddingOctets             |     |                           |
>     +-----+---------------------------+-----+---------------------------+
>
> See [IPFIX-IANA] for the definitions of these Information Elements.
>
>
>
> 6.  Extending the Information Model
>
>     A key requirement for IPFIX is to allow for extension of the
>     Information Model maintained by IANA. The process for extending the
>     Information Model is described in [IPFIX-IE-DOCTORS], which also
>     provides guidelines for authors and reviewers of new Information
>     Element definitions.
>
>     For new Information Elements, the type space defined in Section 3 can
>     be used. If required, new abstract data types can be added to the
>     subregistry defined in [RFC5610]. New abstract data types MUST be
>     defined in IETF Standards Track documents.

Isn't IANA the master reference for that registry?

What's the policy for adding new IEs? By which I mean, cite one of the 
definitions from section 4.1 of RFC5226.


>
>     Enterprises may wish to define Information Elements without
>     registering them with IANA. IPFIX explicitly supports
>     enterprise-specific Information Elements. Enterprise-specific
>     Information Elements are described in Sections 2.1 and 4; guidelines
>     for using them appear in [IPFIX-IE-DOCTORS].
>
>
>
>
>
>
>   
>
>
> Claise, Trammell            Standards Track                    [Page 20]
> 
> Internet-Draft          IPFIX Information Model          October 3, 2012
>
>
> 7.  IANA Considerations
>
> 7.1.  IPFIX Information Elements
>
> This document refers to Information Elements, for which the Internet
> Assigned Numbers Authority (IANA) has created the IPFIX Information
> Element Registry [IPFIX-IANA]. The columns of this registry must at
> minimum be able to store the information defined in the template in
> Section 2.1; it may contain other information as necessary for the
> management of the registry.
>
> New assignments for IPFIX Information Elements will be administered by

s/will be/are/


> IANA through Expert Review [RFC5226], i.e., review by one of a group of
> experts designated by the IESG. Further considerations for this review
> are specified in [IPFIX-IE-DOCTORS].
>
> Future assignments added to the IPFIX Information Element Registry which
> require subregistries for enumerated values (e.g. section 7.2, below)
> must have those subregistries added simultaneously with the new
> assignment; additions to these subregistries must be subject to Expert
> Review [RFC5226]. Unless specified at assignment time, the experts for
> the subregistry will be the same as for the Information Element registry
> as a whole.
>
> Changes may also be made to the entries in this registry from time to
> time; the process for these changes are specified in [IPFIX-IE-DOCTORS].
>
> [NOTE to IANA: please update the Reference for the IPFIX Information
> Element Registry to refer to this document.]
>
> [NOTE to IANA: on publication of this document, please set the Revision
> of all existing Information Elements to 0.]
>
> [NOTE to IANA: on publication of this document, please set the Date of
> all existing Information Elements to the publication date of this
> document.]
>
> [NOTE to IANA: on publication of this document, please set the Name of
> all existing Reserved Information Elements to "Assigned for NetFlow v9
> compatibility", and the reference to [RFC3954].]

NB this works now that 312 and 315 have been assigned as requested in 
data-link-layer-monitoring, else those would have been incorrectly 
attributed.


>
> 7.2.  MPLS Label Type Identifier
>
> Information Element #46, named mplsTopLabelType, carries MPLS label
> types.  Values for 5 different types have initially been defined.  For
> ensuring extensibility of this information, IANA has created a new
> subregistry for MPLS label types and filled it with the initial list
> from the description Information Element #46, mplsTopLabelType.
>   
>
>
> Claise, Trammell            Standards Track                    [Page 21]
> 
> Internet-Draft          IPFIX Information Model          October 3, 2012
>
>
> New assignments for MPLS label types will be administered by IANA

s/will be/are/


> through Expert Review [RFC5226], i.e., review by one of a group of
> experts designated by an IETF Area Director.  The group of experts must
> double check the label type definitions with already defined label types
> for completeness, accuracy, and redundancy.  The specification of new
> MPLS label types MUST be published using a well-established and
> persistent publication medium.
>
> [NOTE to IANA: please update the Reference for the IPFIX MPLS Label Type
> subregistry to refer to this document.]
>
> 7.3.  XML Namespace and Schema
>
> [IPFIX-XML-SCHEMA] defines an XML schema for IPFIX Information Element
> definitions.  All Information Elements specified in [IPFIX-IANA] are
> defined by this schema.  This schema may also be used for specifying
> further Information Elements in future extensions of the IPFIX
> information model in a machine-readable way.
>
> [IPFIX-XML-SCHEMA] uses URNs to describe an XML namespace and an XML
> schema for IPFIX Information Elements conforming to a registry mechanism
> described in [RFC3688].  Two URI assignments have been made.
>
> 1.  Registration for the IPFIX information model namespace
>      *  URI: urn:ietf:params:xml:ns:ipfix-info
>      *  Registrant Contact: IETF IPFIX Working Group ,
>         as designated by the IESG .
>      *  XML: None.  Namespace URIs do not represent an XML.
>
> 2.  Registration for the IPFIX information model schema
>      *  URI: urn:ietf:params:xml:schema:ipfix-info
>      *  Registrant Contact: IETF IPFIX Working Group ,
>         as designated by the IESG .
>
> Using a machine-readable syntax for the information model enables the
> creation of IPFIX-aware tools that can automatically adapt to
> extensions to the information model, by simply reading updated
> information model specifications.
>
> The wide availability of XML-aware tools and libraries for client
> devices is a primary consideration for this choice.  In particular,
> libraries for parsing XML documents are readily available.  Also,
> mechanisms such as the Extensible Stylesheet Language (XSL) allow for
> transforming a source XML document into other documents.  This
> document was authored in XML and transformed according to [RFC2629].
>
> It should be noted that the use of XML in Exporters, Collectors, or
> other tools is not mandatory for the deployment of IPFIX.  In
>   
>
>
> Claise, Trammell            Standards Track                    [Page 22]
> 
> Internet-Draft          IPFIX Information Model          October 3, 2012
>
>
> particular, Exporting Processes do not produce or consume XML as part
> of their operation.  It is expected that IPFIX Collectors MAY take
> advantage of the machine readability of the information model vs.
> hard coding their behavior or inventing proprietary means for
> accommodating extensions.
>
> [NOTE to IANA: please update the Reference for the the IPFIX
> information model namespace and schema to refer to this document.]
>
> 7.4.  Addition, Revision, and Deprecation
>
> As stated in Section 6, addition, revision, and deletion of Information
> Elements in the IPFIX Information Element registry is subject to a
> process described in [IPFIX-IE-DOCTORS]. The IE-DOCTORS experts mentions

s/mentions/mentioned/


> in this process are to be appointed by the IESG.

When was/will that be done? Where are/will they be listed? How will IANA 
know who they are?


>
> When IANA receives a request to add, revise, or deprecate an Information
> Element in the IPFIX Information Elements Registr, it forwards the

s/Registr/Register/


> request to the IE-DOCTORS experts for review.
>
> When IANA receives an approval for a request to add an Information
> Element definition from the IE-DOCTORS experts, it adds that Information
> Element to the registry. The approved request may include changes from
> the original request.

Changes made by the requester, the experts, or IANA?


P.


>
> When IANA receives an approval for a request to revise an Information
> Element definition from the IE-DOCTORS experts, it changes that
> Information Element's definition in the registry, and updates the
> Revision and Date columns as appropriate. The approved request may
> include changes from the original request. If the original Information
> Element was added to the registry with IETF consensus (i.e., was defined
> by an RFC), the revision will require IETF consensus as well.
>
> When IANA receives an approval for a request to deprecate an Information
> Element definition from the IE-DOCTORS experts, it changes that
> Information Element's definition in the registry, and updates the
> Revision and Date columns as appropriate. The approved request may
> include changes from the original request. If the original Information
> Element was added to the registry with IETF consensus (i.e., was defined
> by an RFC), the deprecation will require IETF consensus as well.
>
>
> 8.  Security Considerations
>
> The IPFIX information model itself does not directly introduce security
> issues.  Rather, it defines a set of attributes that may for privacy or
> business issues be considered sensitive information.
>
>   
>
>
> Claise, Trammell            Standards Track                    [Page 23]
> 
> Internet-Draft          IPFIX Information Model          October 3, 2012
>
>
> For example, exporting values of header fields may make attacks possible
> for the receiver of this information, which would otherwise only be
> possible for direct observers of the reported Flows along the data path.
>
> The underlying protocol used to exchange the information described here
> must therefore apply appropriate procedures to guarantee the integrity
> and confidentiality of the exported information.  Such protocols are
> defined in separate documents, specifically the IPFIX protocol document
> [RFC5101bis].
>
> This document does not specify any Information Element carrying keying
> material.  If future extensions will do so, then appropriate precautions
> need to be taken for properly protecting such sensitive information.
>
> 9.  Acknowledgements
>
> The editors would like to thanks the authors of the RFC5102 [RFC5102],
> as this document is directly based upon this original RFC: Juergen
> Quittek, Stewart Bryant, Paul Aitken, and Jeff Meyer.
>
> 10.  References
>
> 10.1.  Normative References
>
>     [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
>                Requirement Levels", BCP 14, RFC 2119, March 1997.
>
>     [RFC5905]  Mills, D., Delaware, U., Martin, J., Burbank, J. and W.
>                Kasch, "Network Time Protocol Version 4: Protocol and
>                Algorithms Specification", RFC 5905, June 2010
>
>     [RFC5101bis]
>                Claise, B., and B. Trammell, Editors, "Specification of
>                the IP Flow Information eXport (IPFIX) Protocol for the
>                Exchange of IP Traffic Flow Information", draft-ietf-
>                ipfix-protocol-rfc5101bis-00, Work in Progress, November
>                2011.
>
>     [IPFIX-IE-DOCTORS]
>                Trammell, B., and B. Claise, "Guidelines for Authors and
>                Reviewers of IPFIX Information Elements", draft-ietf-
>                ipfix-ie-doctors-00, Work in Progress, November 2011.
>
> 10.2.  Informative References
>
>     [IEEE.754.1985]
>                Institute of Electrical and Electronics Engineers,
>                "Standard for Binary Floating-Point Arithmetic", IEEE
>   
>
>
> Claise, Trammell            Standards Track                    [Page 24]
> 
> Internet-Draft          IPFIX Information Model          October 3, 2012
>
>
>                Standard 754, August 1985.
>
>     [ISO.10646-1.1993]
>                International Organization for Standardization,
>                "Information Technology - Universal Multiple-octet coded
>                Character Set (UCS) - Part 1: Architecture and Basic
>                Multilingual Plane", ISO Standard 10646-1, May 1993.
>
>     [ISO.646.1991]
>                International Organization for Standardization,
>                "Information technology - ISO 7-bit coded character set
>                for information interchange", ISO Standard 646, 1991.
>                
>
>     [POSIX.1]  IEEE 1003.1-2008 - IEEE Standard for Information
>                Technology - Portable Operating System Interface, IEEE,
>                2008.
>
>     [RFC2578]  McCloghrie, K., Perkins, D., and J. Schoenwaelder,
>                "Structure of Management Information Version 2 (SMIv2)",
>                STD 58, RFC 2578, April 1999.
>
>     [RFC2629]  Rose, M., "Writing I-Ds and RFCs using XML", RFC 2629,
>                June 1999.
>
>     [RFC3234]  Carpenter, B. and S. Brim, "Middleboxes: Taxonomy and
>                Issues", RFC 3234, February 2002.
>
>     [RFC3444]  Pras, A. and J. Schoenwaelder, "On the Difference between
>                Information Models and Data Models", RFC 3444, January
>                2003.
>
>     [RFC3688]  Mealling, M., "The IETF XML Registry", BCP 81, RFC 3688,
>                January 2004.
>
>     [RFC3917]  Quittek, J., Zseby, T., Claise, B., and S. Zander,
>                "Requirements for IP Flow Information Export (IPFIX)", RFC
>                3917, October 2004.
>
>     [RFC3954]  Claise, B., Ed., "Cisco Systems NetFlow Services Export
>                Version 9", RFC 3954, October 2004.
>
>     [RFC5102]  Trammell, B., and E. Boschi, "Bidirectional Flow Export
>                Using IP Flow Information Export (IPFIX)", RFC 5103,
>                January 2008.
>
>     [RFC5103]  Quittek, J., Bryant, S. Claise, B., Aitken, P., and J.
>                Meyer, "Information Model for IP Flow Information Export",
>   
>
>
> Claise, Trammell            Standards Track                    [Page 25]
> 
> Internet-Draft          IPFIX Information Model          October 3, 2012
>
>
>                RFC 5102, January 2008.
>
>     [RFC5153]  Boschi, E., Mark, L., Quittek J., and P. Aitken, "IP Flow
>                Information Export (IPFIX) Implementation Guidelines",
>                RFC5153, April 2008.
>
>     [RFC5226]  Narten, T. and H. Alvestrand, "Guidelines for Writing an
>                IANA Considerations Section in RFCs", BCP 26, RFC 5226,
>                May 2008.
>
>     [RFC5470]  Sadasivan, G., Brownlee, N., Claise, B., and J. Quittek,
>                "Architecture for IP Flow Information Export", RFC5470,
>                March 2009.
>
>     [RFC5471]  Schmoll, C., Aitken, P., and B. Claise, "Guidelines for IP
>                Flow Information Export (IPFIX) Testing", RFC5471, March
>                2009.
>
>     [RFC5472]  Zseby, T., Boschi, E., Brownlee, N., and B. Claise, "IP
>                Flow Information Export (IPFIX) Applicability", RFC5472,
>                March 2009.
>
>     [RFC5473]  Boschi, E., Mark, L., and B. Claise, "Reducing Redundancy
>                in IP Flow Information Export (IPFIX) and Packet Sampling
>                (PSAMP) Reports", RFC5473, March 2009.
>
>     [RFC5610]  Boschi, E., Trammell, B., Mark, L., and T. Zseby,
>                "Exporting Type Information for IP Flow Information Export
>                (IPFIX) Information Elements", July 2009.
>
>     [RFC6313]  Claise, B., Dhandapani, G., Aitken, P, and S. Yates,
>                "Export of Structured Data in IP Flow Information Export
>                (IPFIX)", RFC6313, July 2011.
>
>     [RFC6183]  Kobayashi, A., Claise, B., Muenz, G, and K. Ishibashi, "IP
>                Flow Information Export (IPFIX) Mediation: Framework",
>                RFC6183, April 2011.
>
>     [IPFIX-CONF]
>                Muenz, G., Claise, B., and P. Aitken, "Configuration Data
>                Model for IPFIX and PSAMP", draft-ietf-ipfix-
>                configuration-model-10, Work in Progress, July 2011.
>
>     [IPFIX-MED-PROTO]
>                Claise, B., Kobayashi, A., and B. Trammell, "Specification
>                of the Protocol for IPFIX Mediations", draft-ietf-ipfix-
>                mediation-protocol-00, Work in Progress, December 2011.
>
>   
>
>
> Claise, Trammell            Standards Track                    [Page 26]
> 
> Internet-Draft          IPFIX Information Model          October 3, 2012
>
>
>     [RFC5815bis]
>                Dietz, T., Kobayashi, A., Claise, B., and G. Muenz,
>                "Definitions of Managed Objects for IP Flow Information
>                Export", draft-ietf-ipfix-rfc5815bis-01.txt, Work in
>                Progress, January 2012.
>
>     [IPFIX-IANA]
>                http://www.iana.org/assignments/ipfix/ipfix.xml
>
>     [IPFIX-XML-SCHEMA]
>                http://www.iana.org/assignments/xml-
>                registry/schema/ipfix.xsd
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>   
>
>
> Claise, Trammell            Standards Track                    [Page 27]
> 
> Internet-Draft          IPFIX Information Model          October 3, 2012
>
>
> Authors' Addresses
>
>     Benoit Claise
>     Cisco Systems, Inc.
>     De Kleetlaan 6a b1
>     1831 Diegem
>     Belgium
>
>     Phone: +32 2 704 5622
>     EMail: bclaise@cisco.com
>
>
>     Brian Trammell
>     Swiss Federal Institute of Technology Zurich
>     Gloriastrasse 35
>     8092 Zurich
>     Switzerland
>
>     Phone: +41 44 632 70 13
>     EMail: trammell@tik.ee.ethz.ch
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> Claise, Trammell            Standards Track                    [Page 28]


From muenz@net.in.tum.de  Wed Oct 31 00:52:20 2012
Return-Path: 
X-Original-To: ipfix@ietfa.amsl.com
Delivered-To: ipfix@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 442F321F86F3 for ; Wed, 31 Oct 2012 00:52:20 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -5.019
X-Spam-Level: 
X-Spam-Status: No, score=-5.019 tagged_above=-999 required=5 tests=[AWL=0.630,  BAYES_00=-2.599, HELO_EQ_DE=0.35, J_CHICKENPOX_54=0.6, RCVD_IN_DNSWL_MED=-4]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Q1Bnj7y6ev2D for ; Wed, 31 Oct 2012 00:52:19 -0700 (PDT)
Received: from mail-out1.informatik.tu-muenchen.de (mail-out1.informatik.tu-muenchen.de [131.159.0.8]) by ietfa.amsl.com (Postfix) with ESMTP id 003F621F86F1 for ; Wed, 31 Oct 2012 00:52:14 -0700 (PDT)
Received: by mail.net.in.tum.de (Postfix, from userid 81) id 8965119A5899; Wed, 31 Oct 2012 08:52:11 +0100 (CET)
To: Brian Trammell 
X-PHP-Originating-Script: 0:main.inc
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Date: Wed, 31 Oct 2012 08:52:11 +0100
From: Gerhard Muenz 
In-Reply-To: <03BF5948-51D3-417B-AD8A-5F6B678A9F46@tik.ee.ethz.ch>
References:    <5087B96B.7020500@cisco.com>  <508850F7.2080801@net.in.tum.de> <50885B49.6050603@cisco.com>  <5088666F.1090106@cisco.com>  <50898454.2000706@net.in.tum.de> <508AB8FB.3060807@plixer.com> <508AE290.3020902@cisco.com> <5b06f4caa55ce88df0f606fb59e19785@net.in.tum.de> <508FC64D.3000006@cisco.com> <03BF5948-51D3-417B-AD8A-5F6B678A9F46@tik.ee.ethz.ch>
Message-ID: 
X-Sender: muenz@net.in.tum.de
User-Agent: Roundcube Webmail/0.6
Cc: John Court , ipfix@ietf.org
Subject: Re: [IPFIX] Export of long lived flow information
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: IPFIX WG discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Wed, 31 Oct 2012 07:52:20 -0000

Hi all,

I scanned through the IANA registry and briefly checked occurrences of 
"this Flow". I do not think that we need change them in general, except 
for those IEs which define a measurement interval.

So, we have:
- flowStart* (including flowStartDeltaMicroseconds)
- flowEnd* (including flowEndDeltaMicroseconds)
- flowDuration*

My first shot was the following change for absolute flowStart*/flowEnd* 
timestamps:

OLD:
The absolute timestamp of the first|last packet of this Flow.
NEW:
The absolute timestamp of the first|last packet accounted in this Flow 
Record.

We could add a sentence clarifying that the Flow properties reported in 
this Flow Record refer to the given measurement interval.
At the moment, I only find a good "negative" way to describe this:
"If this Information Element is present in a Flow Record, Flow 
properties in this Flow Record do not refer to packets 
preceding|succeeding this timestamp."

Do you have better suggestions?
Or do you think that this clarification is not necessary?

Regards,
Gerhard


On 30.10.2012 13:38, Brian Trammell wrote:
> Hi, Paul, all,
>
> If someone throws together a quick summary of the position for me I'd
> be happy make a couple of slides and do the presentation in person in
> Atlanta.
>
> From where I sit, it looks like we just go ask IANA to make the
> registry change. Wearing my IE-Doctors-author hat (I can't wear an IE
> Doctor hat, there aren't any yet. :) ), I'd say this revision would 
> be
> covered (and permissible) under point 2 in section 5.2 of ie-doctors.
>
> Best regards,
>
> Brian
>
>
> On 30 Oct 2012, at 13:21 , Paul Aitken wrote:
>
>> Gerhard,
>>
>> I agree with your definitions. Thanks for clarifying.
>>
>> So what's the next step?
>>
>>    * Update the IANA definitions?
>>    * Add clarifications to the WG documents?
>>
>>
>> Do we need a short presentation at the upcoming WG meeting?
>>
>> P.
>>
>>
>> On 30/10/12 11:51, Gerhard Muenz wrote:
>>> Paul,
>>>
>>> On 26.10.2012 20:20, Paul Aitken wrote:
>>>> Andrew, Gerhard,
>>>>
>>>>>> My understanding is that both, deltaCounts and totalCounts 
>>>>>> contain the number of packets or octets observed in the indicated 
>>>>>> time interval. So, for identical flowStart* and flowEnd* 
>>>>>> timestamps, the values are the same.
>>>>> This is my understanding as well.
>>>>
>>>> There has to be a difference between delta and total counts, else 
>>>> we
>>>> wouldn't have both of them!
>>>>
>>>> Suppose we have a permanent cache, so the cache entries never 
>>>> expire.
>>>>
>>>> For a new flow starting at t0 with a first export at t1, the
>>>> timestamps, delta, and total counts are the same.
>>>>
>>>> However with the second export at t2, the total and delta counts 
>>>> are
>>>> different although their timestamps match (they'll both say, "t0 
>>>> to
>>>> t2").
>>>
>>> No, this would contradict the new definition of flowStart* we are 
>>> just discussing.
>>> If delta counts are exported for the interval (t1,t2), then 
>>> flowStart* is t1.
>>> If delta counts are exported for the interval (t0,t2), then 
>>> flowStart* is t0.
>>> If total counts are exported, flowStart is always t0.
>>> These statements hold regardless of which type of cache is used by 
>>> the Metering Process. In general, the information model does not care 
>>> about how the cache is implemented. The exported information just 
>>> must follow the IE definition.
>>>
>>>>
>>>> With the traditional (non-permanent) cache, the entry would 
>>>> probably
>>>> have been removed at t1 and re-created on a subsequent packet, so 
>>>> at
>>>> t2 the delta and total counts would both be equal. However it'd be
>>>> incorrect to report the total count, because that's defined as the
>>>> total number of packets or bytes ..."since the Metering Process
>>>> (re-)initialization for this Observation Point".
>>>
>>> You must not export total counters in this case because you reset 
>>> counters before re-initialization of the Metering Process.
>>>
>>> Thanks,
>>> Gerhard
>>>
>>>>
>>>>
>>>>>> However, the description of totalCounts says that you report the 
>>>>>> number of packets or octets observed for this Flow since 
>>>>>> re-initialization. So, you must never reset the counter for this 
>>>>>> Flow, even after observing a FIN or RST.
>>>>>> If you reset flow counters, or if you remove Flows from your 
>>>>>> Cache, you cannot use totalCounts any more unless you 
>>>>>> re-initialize the Metering Process (e.g. after flushing the entire 
>>>>>> permanent Cache).
>>>>>
>>>>> I can try some tests later, but from what I have seen (and been 
>>>>> told) many totals being exported are in fact just a delta sent once 
>>>>> at the end of the flow.  If a later flow had the same IPs, 
>>>>> protocol, and ports as an earlier flow I'm pretty sure a new start 
>>>>> time will be sent rather than the the first time that flow was seen 
>>>>> since reinitializing the metering process.
>>>>
>>>> So the MP uses a traditional (non-permanent) cache. In RFC 6728
>>>> terms, a TimeoutCache or NaturalCache rather than a 
>>>> PermanentCache.
>>>>
>>>>
>>>>> Or to put it an other way I think deltas are being sent, but 
>>>>> called totals by the implementation because it seemed like the 
>>>>> right thing to do for a value being sent once at the end of the 
>>>>> flow.
>>>>
>>>> The collector could be aggregating deltas to keep running totals.
>>>>
>>>>
>>>>> I suspect that totals reporting on the export process (eg 
>>>>> exportedOctetTotalCount, exportedMessageTotalCount) are, however, 
>>>>> reported with a start time that is only reset on reinitialization.
>>>>
>>>> Definitely, because these are defined as "The total number of X 
>>>> that
>>>> the Exporting Process has sent since the Exporting Process
>>>> (re-)initialization ...".
>>>>
>>>> P.
>>
>> _______________________________________________
>> IPFIX mailing list
>> IPFIX@ietf.org
>> https://www.ietf.org/mailman/listinfo/ipfix


From trammell@tik.ee.ethz.ch  Wed Oct 31 01:16:35 2012
Return-Path: 
X-Original-To: ipfix@ietfa.amsl.com
Delivered-To: ipfix@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4112421F870D for ; Wed, 31 Oct 2012 01:16:35 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -6.299
X-Spam-Level: 
X-Spam-Status: No, score=-6.299 tagged_above=-999 required=5 tests=[AWL=-0.300, BAYES_00=-2.599, J_CHICKENPOX_54=0.6, RCVD_IN_DNSWL_MED=-4]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id umHNudCZEj4E for ; Wed, 31 Oct 2012 01:16:34 -0700 (PDT)
Received: from smtp.ee.ethz.ch (smtp.ee.ethz.ch [129.132.2.219]) by ietfa.amsl.com (Postfix) with ESMTP id D9C9F21F870B for ; Wed, 31 Oct 2012 01:16:31 -0700 (PDT)
Received: from localhost (localhost [127.0.0.1]) by smtp.ee.ethz.ch (Postfix) with ESMTP id 27590D9309; Wed, 31 Oct 2012 09:16:30 +0100 (MET)
X-Virus-Scanned: by amavisd-new on smtp.ee.ethz.ch
Received: from smtp.ee.ethz.ch ([127.0.0.1]) by localhost (.ee.ethz.ch [127.0.0.1]) (amavisd-new, port 10024) with LMTP id Pmzi23384yhe; Wed, 31 Oct 2012 09:16:29 +0100 (MET)
Received: from [10.0.27.100] (cust-integra-121-161.antanet.ch [80.75.121.161]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: briant) by smtp.ee.ethz.ch (Postfix) with ESMTPSA id B4E75D9305; Wed, 31 Oct 2012 09:16:29 +0100 (MET)
Mime-Version: 1.0 (Apple Message framework v1283)
Content-Type: text/plain; charset=us-ascii
From: Brian Trammell 
In-Reply-To: 
Date: Wed, 31 Oct 2012 09:16:28 +0100
Content-Transfer-Encoding: quoted-printable
Message-Id: <29FB5FA9-B84E-4B5F-97FF-5F81C826F6D0@tik.ee.ethz.ch>
References:    <5087B96B.7020500@cisco.com>  <508850F7.2080801@net.in.tum.de> <50885B49.6050603@cisco.com>  <5088666F.1090106@cisco.com>  <50898454.2000706@net.in.tum.de> <508AB8FB.3060807@plixer.com> <508AE290.3020902@cisco.com> <5b06f4caa55ce88df0f606fb59e19785@net.in.tum.de> <508FC64D.3000006@cisco.com> <03BF5948-51D3-417B-AD8A-5F6B678A9F46@tik.ee.ethz.ch> 
To: Gerhard Muenz 
X-Mailer: Apple Mail (2.1283)
Cc: John Court , ipfix@ietf.org
Subject: Re: [IPFIX] Export of long lived flow information
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: IPFIX WG discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Wed, 31 Oct 2012 08:16:36 -0000

Hi, Gerhard, all,

Thought about this a bit too, and I can't come up anything better than =
the negative clarification. I might consider changing "preceding" and =
"succeeding" to "before" and "after" to simplify the language for =
non-native speakers, and stick an "observed" in there to make clear =
we're talking about observation time:

If this Information Element is present in a Flow Record, Flow properties =
in this Flow Record do not refer to packets observed before/after this =
timestamp.

It either case, it does make things a lot more explicitly clear, so I'd =
definitely add it to the timestamp/duration IEs.

Thanks, cheers,

Brian

On Oct 31, 2012, at 8:52 AM, Gerhard Muenz wrote:

>=20
> Hi all,
>=20
> I scanned through the IANA registry and briefly checked occurrences of =
"this Flow". I do not think that we need change them in general, except =
for those IEs which define a measurement interval.
>=20
> So, we have:
> - flowStart* (including flowStartDeltaMicroseconds)
> - flowEnd* (including flowEndDeltaMicroseconds)
> - flowDuration*
>=20
> My first shot was the following change for absolute =
flowStart*/flowEnd* timestamps:
>=20
> OLD:
> The absolute timestamp of the first|last packet of this Flow.
> NEW:
> The absolute timestamp of the first|last packet accounted in this Flow =
Record.
>=20
> We could add a sentence clarifying that the Flow properties reported =
in this Flow Record refer to the given measurement interval.
> At the moment, I only find a good "negative" way to describe this:
> "If this Information Element is present in a Flow Record, Flow =
properties in this Flow Record do not refer to packets =
preceding|succeeding this timestamp."
>=20
> Do you have better suggestions?
> Or do you think that this clarification is not necessary?
>=20
> Regards,
> Gerhard
>=20
>=20
> On 30.10.2012 13:38, Brian Trammell wrote:
>> Hi, Paul, all,
>>=20
>> If someone throws together a quick summary of the position for me I'd
>> be happy make a couple of slides and do the presentation in person in
>> Atlanta.
>>=20
>> =46rom where I sit, it looks like we just go ask IANA to make the
>> registry change. Wearing my IE-Doctors-author hat (I can't wear an IE
>> Doctor hat, there aren't any yet. :) ), I'd say this revision would =
be
>> covered (and permissible) under point 2 in section 5.2 of ie-doctors.
>>=20
>> Best regards,
>>=20
>> Brian
>>=20
>>=20
>> On 30 Oct 2012, at 13:21 , Paul Aitken wrote:
>>=20
>>> Gerhard,
>>>=20
>>> I agree with your definitions. Thanks for clarifying.
>>>=20
>>> So what's the next step?
>>>=20
>>>   * Update the IANA definitions?
>>>   * Add clarifications to the WG documents?
>>>=20
>>>=20
>>> Do we need a short presentation at the upcoming WG meeting?
>>>=20
>>> P.
>>>=20
>>>=20
>>> On 30/10/12 11:51, Gerhard Muenz wrote:
>>>> Paul,
>>>>=20
>>>> On 26.10.2012 20:20, Paul Aitken wrote:
>>>>> Andrew, Gerhard,
>>>>>=20
>>>>>>> My understanding is that both, deltaCounts and totalCounts =
contain the number of packets or octets observed in the indicated time =
interval. So, for identical flowStart* and flowEnd* timestamps, the =
values are the same.
>>>>>> This is my understanding as well.
>>>>>=20
>>>>> There has to be a difference between delta and total counts, else =
we
>>>>> wouldn't have both of them!
>>>>>=20
>>>>> Suppose we have a permanent cache, so the cache entries never =
expire.
>>>>>=20
>>>>> For a new flow starting at t0 with a first export at t1, the
>>>>> timestamps, delta, and total counts are the same.
>>>>>=20
>>>>> However with the second export at t2, the total and delta counts =
are
>>>>> different although their timestamps match (they'll both say, "t0 =
to
>>>>> t2").
>>>>=20
>>>> No, this would contradict the new definition of flowStart* we are =
just discussing.
>>>> If delta counts are exported for the interval (t1,t2), then =
flowStart* is t1.
>>>> If delta counts are exported for the interval (t0,t2), then =
flowStart* is t0.
>>>> If total counts are exported, flowStart is always t0.
>>>> These statements hold regardless of which type of cache is used by =
the Metering Process. In general, the information model does not care =
about how the cache is implemented. The exported information just must =
follow the IE definition.
>>>>=20
>>>>>=20
>>>>> With the traditional (non-permanent) cache, the entry would =
probably
>>>>> have been removed at t1 and re-created on a subsequent packet, so =
at
>>>>> t2 the delta and total counts would both be equal. However it'd be
>>>>> incorrect to report the total count, because that's defined as the
>>>>> total number of packets or bytes ..."since the Metering Process
>>>>> (re-)initialization for this Observation Point".
>>>>=20
>>>> You must not export total counters in this case because you reset =
counters before re-initialization of the Metering Process.
>>>>=20
>>>> Thanks,
>>>> Gerhard
>>>>=20
>>>>>=20
>>>>>=20
>>>>>>> However, the description of totalCounts says that you report the =
number of packets or octets observed for this Flow since =
re-initialization. So, you must never reset the counter for this Flow, =
even after observing a FIN or RST.
>>>>>>> If you reset flow counters, or if you remove Flows from your =
Cache, you cannot use totalCounts any more unless you re-initialize the =
Metering Process (e.g. after flushing the entire permanent Cache).
>>>>>>=20
>>>>>> I can try some tests later, but from what I have seen (and been =
told) many totals being exported are in fact just a delta sent once at =
the end of the flow.  If a later flow had the same IPs, protocol, and =
ports as an earlier flow I'm pretty sure a new start time will be sent =
rather than the the first time that flow was seen since reinitializing =
the metering process.
>>>>>=20
>>>>> So the MP uses a traditional (non-permanent) cache. In RFC 6728
>>>>> terms, a TimeoutCache or NaturalCache rather than a =
PermanentCache.
>>>>>=20
>>>>>=20
>>>>>> Or to put it an other way I think deltas are being sent, but =
called totals by the implementation because it seemed like the right =
thing to do for a value being sent once at the end of the flow.
>>>>>=20
>>>>> The collector could be aggregating deltas to keep running totals.
>>>>>=20
>>>>>=20
>>>>>> I suspect that totals reporting on the export process (eg =
exportedOctetTotalCount, exportedMessageTotalCount) are, however, =
reported with a start time that is only reset on reinitialization.
>>>>>=20
>>>>> Definitely, because these are defined as "The total number of X =
that
>>>>> the Exporting Process has sent since the Exporting Process
>>>>> (re-)initialization ...".
>>>>>=20
>>>>> P.
>>>=20
>>> _______________________________________________
>>> IPFIX mailing list
>>> IPFIX@ietf.org
>>> https://www.ietf.org/mailman/listinfo/ipfix


From trammell@tik.ee.ethz.ch  Wed Oct 31 02:29:22 2012
Return-Path: 
X-Original-To: ipfix@ietfa.amsl.com
Delivered-To: ipfix@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 63D8F21F8732 for ; Wed, 31 Oct 2012 02:29:22 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -5.766
X-Spam-Level: 
X-Spam-Status: No, score=-5.766 tagged_above=-999 required=5 tests=[AWL=-0.533, BAYES_00=-2.599, FF_IHOPE_YOU_SINK=2.166, GB_I_LETTER=-2, J_CHICKENPOX_33=0.6, J_CHICKENPOX_37=0.6, RCVD_IN_DNSWL_MED=-4]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bI7v-t-+DzhZ for ; Wed, 31 Oct 2012 02:29:11 -0700 (PDT)
Received: from smtp.ee.ethz.ch (smtp.ee.ethz.ch [129.132.2.219]) by ietfa.amsl.com (Postfix) with ESMTP id E78FE21F8727 for ; Wed, 31 Oct 2012 02:29:10 -0700 (PDT)
Received: from localhost (localhost [127.0.0.1]) by smtp.ee.ethz.ch (Postfix) with ESMTP id 499DFD930A; Wed, 31 Oct 2012 10:29:10 +0100 (MET)
X-Virus-Scanned: by amavisd-new on smtp.ee.ethz.ch
Received: from smtp.ee.ethz.ch ([127.0.0.1]) by localhost (.ee.ethz.ch [127.0.0.1]) (amavisd-new, port 10024) with LMTP id q5TQ06bxUsWG; Wed, 31 Oct 2012 10:29:09 +0100 (MET)
Received: from [10.0.27.100] (cust-integra-121-161.antanet.ch [80.75.121.161]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: briant) by smtp.ee.ethz.ch (Postfix) with ESMTPSA id 2DEAAD9309; Wed, 31 Oct 2012 10:29:09 +0100 (MET)
Mime-Version: 1.0 (Apple Message framework v1283)
Content-Type: text/plain; charset=iso-8859-1
From: Brian Trammell 
In-Reply-To: <5090547C.5020803@cisco.com>
Date: Wed, 31 Oct 2012 10:29:08 +0100
Content-Transfer-Encoding: quoted-printable
Message-Id: 
References: <506CBFE3.10607@auckland.ac.nz> <5090547C.5020803@cisco.com>
To: Paul Aitken 
X-Mailer: Apple Mail (2.1283)
Cc: IPFIX Working Group 
Subject: [IPFIX] Removal of section 5 from RFC5102bis
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: IPFIX WG discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Wed, 31 Oct 2012 09:29:22 -0000

Hi, Paul, all,

A more detailed response to your review follows (thank you!!); however, =
there is a rather important implicit question there that I wanted to put =
to the working group before we meet in Atlanta.

That is: does anyone at all find the content (IE categorization and =
possibly deprecated restrictions) of section 5 of RFC5102bis useful?  =
Categorization was not carried over into the IANA registry (and in any =
case would require redefinition as IPFIX applications move down to layer =
2 and up to layer 7), and Paul makes convincing arguments that most of =
the rest of the content of the section are historic restrictions on =
information model usage.

My proposal is to remove the section completely and replace it with a =
historical note stating that section 5 of 5102 defined the Information =
Model before the IANA registry existed, but that the IANA registry is =
now normative.

Comments?

Thanks, cheers,

Brian



On Oct 30, 2012, at 11:28 PM, Paul Aitken wrote:

> Nevil, All,
>=20
>> The WG Last Call for this I-D starts now, and will run until Monday, =
22 October.
>=20
> Sorry I'm late for -05, but hopefully still in time for -06? :-)
>=20
> So here's a 110% review of rfc5102bis-06.
>=20
>=20
> There are some nits:
>=20
>  =3D=3D Missing Reference: 'RFC5101' is mentioned on line 164, but not =
defined
>=20
>  =3D=3D Outdated reference: A later version (-02) exists of
>     draft-ietf-ipfix-protocol-rfc5101bis-00
>=20
>  -- Possible downref: Normative reference to a draft: ref. =
'RFC5101bis'
>=20
>  =3D=3D Outdated reference: A later version (-07) exists of
>     draft-ietf-ipfix-ie-doctors-00
>=20
>  =3D=3D Outdated reference: draft-ietf-ipfix-configuration-model has =
been
>     published as RFC 6728
>=20
>  =3D=3D Outdated reference: A later version (-02) exists of
>     draft-ietf-ipfix-mediation-protocol-00
>=20
>  =3D=3D Outdated reference: draft-ietf-ipfix-rfc5815bis has been =
published as
>     RFC 6615
>=20
>=20
> Some IEs are missing from this document, although they are defined in =
IANA's IPFIX registry:
>=20
>    All the IE's from section 5.8. "Min/Max Flow Properties" of [5102] =
are missing:
>=20
>        6    tcpControlBits
>        25    minimumIpTotalLength
>        26    maximumIpTotalLength
>        52    minimumTTL
>        53    maximumTTL
>        64    ipv6ExtensionHeaders
>        208    ipv4Options
>        209    tcpOptions
>=20
>=20
>    This 5103 IE is missing:
>=20
>        239    biflowDirection
>=20
>=20
>    The following IEs defined by cisco are listed by IANA, but not in =
this text:
>=20
>        82    interfaceName
>        83    interfaceDescription
>        91    mplsTopLabelPrefixLength
>        98    postIpDiffServCodePoint
>        99    multicastReplicationFactor
>        105-127
>        225-236
>        240+
>=20
>=20
>    The following IEs are not mentioned here, although they are =
detailed in draft-yourtchenko-cisco-ies :
>=20
>        3, 34, 35, 38, 39, 43, 48-51, 65-69, 84, 87, 89, 92-93, 100, =
101, 102, 103, 104, 94-97.
>=20
>=20
> Please find specific feedback inline:
>=20
>=20
>=20
>> Network Working Group                                     B. Claise, =
Ed.
>> Internet Draft                                       Cisco Systems, =
Inc.
>> Obsoletes: 5102                                         B. Trammell, =
Ed.
>> Category: Standards Track                                     ETH =
Zurich
>> Expires: April 6, 2013                                   October 3, =
2012
>>=20
>>=20
>>         Information Model for IP Flow Information eXport (IPFIX)
>>           draft-ietf-ipfix-information-model-rfc5102bis-06.txt
>>                                    =20
>> Abstract
>>=20
>> This document provides an overview of the information model for the =
IP
>> Flow Information eXport (IPFIX) protocol, as defined in the IANA =
IPFIX
>> Information Element Registry. It is used by the IPFIX Protocol for
>> encoding measured traffic information and information related to the
>> traffic Observation Point, the traffic Metering Process, and the
>> Exporting Process. Although developed for the IPFIX Protocol, the =
model
>> is defined in an open way that easily allows using it in other
>> protocols, interfaces, and applications. This document obsoletes RFC
>> 5102.
>>=20
>> Status of This Memo
>>=20
>>    This Internet-Draft is submitted in full conformance with the
>>    provisions of BCP 78 and BCP 79.
>>=20
>>    Internet-Drafts are working documents of the Internet Engineering
>>    Task Force (IETF). Note that other groups may also distribute =
working
>>    documents as Internet-Drafts. The list of current Internet-Drafts =
is
>>    at http://datatracker.ietf.org/drafts/current/.
>>=20
>>    Internet-Drafts are draft documents valid for a maximum of six =
months
>>    and may be updated, replaced, or obsoleted by other documents at =
any
>>    time. It is inappropriate to use Internet-Drafts as reference
>>    material or to cite them other than as "work in progress."
>>=20
>>    This Internet-Draft will expire on March 23, 2012.
>>=20
>> Copyright Notice
>>=20
>>    Copyright (c) 2012 IETF Trust and the persons identified as the
>>    document authors. All rights reserved.
>>=20
>>    This document is subject to BCP 78 and the IETF Trust's Legal
>>    Provisions Relating to IETF Documents
>>    (http://trustee.ietf.org/license-info) in effect on the date of
>> =20
>>=20
>> Claise, Trammell            Standards Track                     [Page =
1]
>>=20
>> Internet-Draft          IPFIX Information Model          October 3, =
2012
>>=20
>>=20
>>    publication of this document. Please review these documents
>>    carefully, as they describe your rights and restrictions with =
respect
>>    to this document. Code Components extracted from this document =
must
>>    include Simplified BSD License text as described in Section 4.e of
>>    the Trust Legal Provisions and are provided without warranty as
>>    described in the Simplified BSD License.
>>=20
>>=20
>> Table of Contents
>>=20
>>    1.  Introduction . . . . . . . . . . . . . . . . . . . . . . . . . =
 3
>>      1.1. Changes since RFC 5102  . . . . . . . . . . . . . . . . . . =
 4
>>      1.2. IPFIX Documents Overview  . . . . . . . . . . . . . . . . . =
 4
>>    2.  Properties of IPFIX Protocol Information Elements  . . . . . . =
 5
>>      2.1.  Information Element Specification Template . . . . . . . . =
 5
>>      2.2.  Scope of Information Elements  . . . . . . . . . . . . . . =
 7
>>      2.3.  Naming Conventions for Information Elements  . . . . . . . =
 8
>>    3.  Type Space . . . . . . . . . . . . . . . . . . . . . . . . . . =
 8
>>      3.1.  Abstract Data Types  . . . . . . . . . . . . . . . . . . . =
 9
>>        3.1.1.  unsigned8  . . . . . . . . . . . . . . . . . . . . . . =
 9
>>        3.1.2.  unsigned16 . . . . . . . . . . . . . . . . . . . . . . =
 9
>>        3.1.3.  unsigned32 . . . . . . . . . . . . . . . . . . . . . . =
 9
>>        3.1.4.  unsigned64 . . . . . . . . . . . . . . . . . . . . . . =
 9
>>        3.1.5.  signed8  . . . . . . . . . . . . . . . . . . . . . . . =
 9
>>        3.1.6.  signed16 . . . . . . . . . . . . . . . . . . . . . . . =
 9
>>        3.1.7.  signed32 . . . . . . . . . . . . . . . . . . . . . . . =
 9
>>        3.1.8.  signed64 . . . . . . . . . . . . . . . . . . . . . . . =
 9
>>        3.1.9.  float32  . . . . . . . . . . . . . . . . . . . . . . . =
10
>>        3.1.10.  float64 . . . . . . . . . . . . . . . . . . . . . . . =
10
>>        3.1.11.  boolean . . . . . . . . . . . . . . . . . . . . . . . =
10
>>        3.1.12.  macAddress  . . . . . . . . . . . . . . . . . . . . . =
10
>>        3.1.13.  octetArray  . . . . . . . . . . . . . . . . . . . . . =
10
>>        3.1.14.  string  . . . . . . . . . . . . . . . . . . . . . . . =
10
>>        3.1.15.  dateTimeSeconds . . . . . . . . . . . . . . . . . . . =
10
>>        3.1.16.  dateTimeMilliseconds  . . . . . . . . . . . . . . . . =
10
>>        3.1.17.  dateTimeMicroseconds  . . . . . . . . . . . . . . . . =
10
>>        3.1.18.  dateTimeNanoseconds . . . . . . . . . . . . . . . . . =
11
>>        3.1.19.  ipv4Address . . . . . . . . . . . . . . . . . . . . . =
11
>>        3.1.20.  ipv6Address . . . . . . . . . . . . . . . . . . . . . =
11
>>      3.2.  Data Type Semantics  . . . . . . . . . . . . . . . . . . . =
11
>>        3.2.1.  quantity . . . . . . . . . . . . . . . . . . . . . . . =
11
>>        3.2.2.  totalCounter . . . . . . . . . . . . . . . . . . . . . =
11
>>        3.2.3.  deltaCounter . . . . . . . . . . . . . . . . . . . . . =
12
>>        3.2.4.  identifier . . . . . . . . . . . . . . . . . . . . . . =
12
>>        3.2.5.  flags  . . . . . . . . . . . . . . . . . . . . . . . . =
12
>>    4.  Information Element Identifiers  . . . . . . . . . . . . . . . =
12
>>      4.1.  NetFlow version 9 compatible Information Element
>>            Identifiers  . . . . . . . . . . . . . . . . . . . . . . . =
13
>> =20
>>=20
>> Claise, Trammell            Standards Track                     [Page =
2]
>>=20
>> Internet-Draft          IPFIX Information Model          October 3, =
2012
>>=20
>>=20
>>    5.  Information Element Categories . . . . . . . . . . . . . . . . =
13
>>      5.1.  Identifiers  . . . . . . . . . . . . . . . . . . . . . . . =
14
>>      5.3.  Metering and Exporting Process Statistics  . . . . . . . . =
15
>>      5.4.  IP Header Fields . . . . . . . . . . . . . . . . . . . . . =
15
>>      5.5.  Transport Header Fields  . . . . . . . . . . . . . . . . . =
16
>>      5.6.  Sub-IP Header Fields . . . . . . . . . . . . . . . . . . . =
17
>>      5.7.  Derived Packet Properties  . . . . . . . . . . . . . . . . =
17
>>      5.9.  Flow Timestamps  . . . . . . . . . . . . . . . . . . . . . =
18
>>      5.10.  Per-Flow Counters . . . . . . . . . . . . . . . . . . . . =
18
>>      5.11.  Miscellaneous Flow Properties . . . . . . . . . . . . . . =
19
>>      5.12.  Padding . . . . . . . . . . . . . . . . . . . . . . . . . =
20
>>    6.  Extending the Information Model  . . . . . . . . . . . . . . . =
20
>>    7.  IANA Considerations  . . . . . . . . . . . . . . . . . . . . . =
21
>>      7.1.  IPFIX Information Elements . . . . . . . . . . . . . . . . =
21
>>      7.2.  MPLS Label Type Identifier . . . . . . . . . . . . . . . . =
21
>>      7.3.  XML Namespace and Schema . . . . . . . . . . . . . . . . . =
22
>>      7.4.  Addition, Revision, and Deprecation  . . . . . . . . . . . =
23
>>    8.  Security Considerations  . . . . . . . . . . . . . . . . . . . =
23
>>    9.  Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . =
24
>>    10.  References  . . . . . . . . . . . . . . . . . . . . . . . . . =
24
>>      10.1.  Normative References  . . . . . . . . . . . . . . . . . . =
24
>>      10.2.  Informative References  . . . . . . . . . . . . . . . . . =
24
>>    Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . =
28
>>=20
>>=20
>>=20
>>=20
>>=20
>> 1.  Introduction
>>=20
>>    The IP Flow Information eXport (IPFIX) protocol serves for
>>    transmitting information related to measured IP traffic over the
>=20
> We also have non-IP traffic, eg MPLS and layer 2 information.
>=20
>=20
>>    Internet.  The protocol specification in [RFC5101bis] defines how
>=20
> Not just "over the Internet", but any IP network.
>=20
>=20
>>    Information Elements are transmitted.  For Information Elements, =
it
>>    specifies the encoding of a set of basic data types.  However, the
>>    list of Information Elements that can be transmitted by the =
protocol,
>>    such as Flow attributes (source IP address, number of packets, =
etc.)
>>    and information about the Metering and Exporting Process (packet
>>    Observation Point, sampling rate, Flow timeout interval, etc.), is
>>    not specified in [RFC5101bis].
>>=20
>>    The canonical reference for IPFIX Information Elements the IANA =
IPFIX
>>    Information Element registry [IPFIX-IANA]; the initial values for
>>    this registry were provided by [RFC5102].
>>=20
>>    This document complements the IPFIX protocol specification by
>=20
> "the IPFIX protocol specification in [RFC5101bis] by"
>=20
>=20
>>    providing an overview of the IPFIX information model and =
specifying
>>    data types for it. IPFIX-specific terminology used in this =
document
>> =20
>>=20
>> Claise, Trammell            Standards Track                     [Page =
3]
>>=20
>> Internet-Draft          IPFIX Information Model          October 3, =
2012
>>=20
>>=20
>>    is defined in Section 2 of [RFC5101bis]. As in [RFC5101bis], these
>>    IPFIX-specific terms have the first letter of a word capitalized =
when
>>    used in this document.
>>=20
>>    The use of the term 'information model' is not fully in line with =
the
>>    definition of this term in [RFC3444].  The IPFIX information model
>>    does not specify relationships between Information Elements, but =
also
>>    it does not specify a concrete encoding of Information Elements.
>=20
> The IPFIX encoding is specified in 5101.
>=20
>=20
>>    Besides the encoding used by the IPFIX protocol, other encodings =
of
>>    IPFIX Information Elements can be applied, for example, XML-based
>>    encodings.
>>=20
>>    The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL =
NOT",
>>    "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in =
this
>>    document are to be interpreted as described in [RFC2119].
>>=20
>> 1.1. Changes since RFC 5102
>>=20
>>    This document obsoletes the Proposed Standard revision of the =
IPFIX
>>    Protocol Specification [RFC5102].  The following changes have been
>>    made to this document with respect to the previous document:
>>=20
>>       - All outstanding technical and editorial errata filed on the
>>    [RFC5102] as of publication time have been corrected
>>       - All references into [RFC5101] have been updated to =
[RFC5101bis],
>>    reflecting changes in that document as necessary
>>       - Information element definitions have been removed, as the
>>    reference for these is now [IPFIX-IANA]; categorizations of
>>    information elements as defines in [RFC5102] have been retained in
>=20
> s/defines/defined/
>=20
>=20
>>    section 5.
>>       - The process for modifying [IPFIX-IANA] has been improved, and =
is
>>    now described in [IPFIX-IE-DOCTORS]; Section 6 has been updated
>>    accordingly, and a new section 7.3 gives IANA considerations for =
this
>>    process.
>>       - Definitions of timestamp data types have been clarified
>>       - Appendices A and B have been removed
>=20
> BTW, the indentation of that section makes it difficult to read. Can =
you get all the text - including the wrapped lines - to the right of the =
bullets?
>=20
>=20
>>=20
>> 1.2. IPFIX Documents Overview
>>=20
>>    The IPFIX protocol provides network administrators with access to =
IP
>>    flow information.  The architecture for the export of measured IP
>>    flow information out of an IPFIX Exporting Process to a Collecting
>=20
> I'd be tempted to drop "IP" from "IP flow information" (x2).
>=20
>=20
>>    Process is defined in [RFC5470], per the requirements defined in
>>    [RFC3917].  The IPFIX specifications [RFC5101bis] document =
specifies
>=20
> "The IPFIX specifications document [RFC5101bis]" ?
>=20
>=20
>>    how IPFIX data records and templates are carried via a number of
>>    transport protocols from IPFIX Exporting Processes to IPFIX
>>    Collecting Processes.
>>=20
>> =20
>>=20
>> Claise, Trammell            Standards Track                     [Page =
4]
>>=20
>> Internet-Draft          IPFIX Information Model          October 3, =
2012
>>=20
>>=20
>>    Four IPFIX optimizations/extensions are currently specified: a
>>    bandwidth saving method for the IPFIX protocol in [RFC5473], an
>>    efficient method for exporting bidirectional flow in [RFC5103], a
>=20
> "bidirectional flows", plural.
>=20
>=20
>>    method for the definition and export of complex data structures in
>>    [RFC6313], and the specification of the Protocol for IPFIX =
Mediations
>>    [IPFIX-MED-PROTO] based on the IPIFX Mediation Framework =
[RFC6183].
>=20
> s/IPIFX/IPFIX/
>=20
>=20
>>=20
>>    IPFIX has a formal description of IPFIX Information Elements, =
their
>>    name, type and additional semantic information, as specified in =
this
>>    document, with the export of the Information Element types =
specified
>>    in [RFC5610].
>>=20
>>    [IPFIX-CONF] specifies a data model for configuring and monitoring
>>    IPFIX and PSAMP compliant devices using the NETCONF protocol, =
while
>>    the [RFC5815bis] specifies a MIB module for monitoring.
>=20
> - "the"
>=20
>=20
>>=20
>>    In terms of development, [RFC5153] provides guidelines for the
>>    implementation and use of the IPFIX protocol, while [RFC5471]
>>    provides guidelines for testing.
>>=20
>>    Finally, [RFC5472] describes what type of applications can use the
>>    IPFIX protocol and how they can use the information provided.  It
>>    furthermore shows how the IPFIX framework relates to other
>>    architectures and frameworks.
>>=20
>> 2.  Properties of IPFIX Protocol Information Elements
>>=20
>> 2.1.  Information Element Specification Template
>>=20
>>    Information in messages of the IPFIX protocol is modeled in terms =
of
>>    Information Elements of the IPFIX information model. The IPFIX
>>    Information Elements mentioned in Section 5 are specified in =
[IPFIX-
>>    IANA]. For specifying these Information Elements, a template is =
used
>>    that is described below.
>=20
> At first I misunderstood "template" here. It's all in the context.
>=20
>=20
>>=20
>>    All Information Elements specified for the IPFIX protocol MUST =
have
>>    the following properties defined:
>>=20
>>    name - A unique and meaningful name for the Information Element.
>>=20
>>    elementId - A numeric identifier of the Information Element.  If =
this
>>       identifier is used without an enterprise identifier (see
>>       [RFC5101bis] and enterpriseId below), then it is globally =
unique
>>       and the list of allowed values is administered by IANA.  It is
>>       used for compact identification of an Information Element when
>>       encoding Templates in the protocol.
>>=20
>>    description - The semantics of this Information Element. Describes
>> =20
>>=20
>> Claise, Trammell            Standards Track                     [Page =
5]
>>=20
>> Internet-Draft          IPFIX Information Model          October 3, =
2012
>>=20
>>=20
>>       how this Information Element is derived from the Flow or other
>>       information available to the observer. Information Elements of
>>       dataType string or octetArray which have a length constraints
>=20
> - "a"
>=20
>=20
>>       (fixed length, minimum and/or maximum length) MUST note these
>>       constraints in their description.
>>=20
>>    dataType - One of the types listed in Section 3.1 of this document =
or
>>       registered in the IANA IPFIX Information Element Data Types
>>       registry. The type space for attributes is constrained to
>>       facilitate implementation. The existing type space does however
>>       encompass most basic types used in modern programming =
languages,
>>       as well as some derived types (such as ipv4Address) that are
>>       common to this domain and useful to distinguish.
>=20
> At first there seemed to be missing text at the end of this line.
>=20
>=20
>>=20
>>    status - The status of the specification of this Information =
Element.
>>       Allowed values are 'current' and 'deprecated'. All =
newly-defined
>>       Information Elements have 'current' status. The process for =
moving
>>       Information Elements to the 'deprecated' status is defined in
>>       Section 5.2 of [IPFIX-IE-DOCTORS].
>>=20
>>    Enterprise-specific Information Elements MUST have the following
>>    property defined:
>>=20
>>    enterpriseId - Enterprises may wish to define Information Elements
>>       without registering them with IANA, for example, for
>>       enterprise-internal purposes.  For such Information Elements, =
the
>>       Information Element identifier described above is not =
sufficient
>>       when the Information Element is used outside the enterprise.  =
If
>>       specifications of enterprise-specific Information Elements are
>>       made public and/or if enterprise-specific identifiers are used =
by
>>       the IPFIX protocol outside the enterprise, then the
>>       enterprise-specific identifier MUST be made globally unique by
>>       combining it with an enterprise identifier.  Valid values for =
the
>>       enterpriseId are defined by IANA as Structure of Management
>>       Information (SMI) network management private enterprise codes.
>>       They are defined at http://www.iana.org/assignments/enterprise-
>>       numbers.
>=20
> Move the URL to an I-ref? It's mentioned again below.
>=20
>=20
>>=20
>>    All Information Elements specified for the IPFIX protocol either =
in
>>    this document or by any future extension MAY have the following
>>    properties defined:
>>=20
>>    dataTypeSemantics - The integral types may be qualified by =
additional
>>       semantic details.  Valid values for the data type semantics are
>>       specified in Section 3.2 of this document or in a future =
extension
>>       of the information model.
>=20
> Section 3.2.1 specifies that "quantity" is the default semantic, so =
this isn't really a MAY.
> If semantics is an optional property, why does it have a default?
>=20
>=20
>>=20
>>    units - If the Information Element is a measure of some kind, the
>> =20
>>=20
>> Claise, Trammell            Standards Track                     [Page =
6]
>>=20
>> Internet-Draft          IPFIX Information Model          October 3, =
2012
>>=20
>>=20
>>       units identify what the measure is.
>>=20
>>    range - Some Information Elements may only be able to take on a
>>       restricted set of values that can be expressed as a range =
(e.g., 0
>>       through 511 inclusive).  If this is the case, the valid =
inclusive
>>       range should be specified.
>=20
> Should we make any comment on the invalidity of values outside the =
range?
>=20
>=20
>>=20
>>    reference - Identifies additional specifications that more =
precisely
>>       define this item or provide additional context for its use.
>>=20
>>=20
>>    The following two Information Element properties are defined to =
allow
>>    the management of an Information Element registry with Information
>>    Element definitions that may be updated over time, per the process
>>    defined in Section 5.2 of [IPFIX-IE-DOCTORS].
>>=20
>>    revision - The revision number of an Information Element, starting =
at
>>       0 for Information Elements at time of definition, and =
incremented
>>       by one for each revision.
>>=20
>>    date - The date of the entry of this revision of the Information
>>       Element into the registry.
>>=20
>>    For Information Elements of the string or octetArray data types =
which
>>    have size limits (minimum and/or maximum size, or fixed length), =
the
>>    limits MUST be defined within the description of the Information
>>    Element.
>=20
> This repeats earlier text in the "description" section.
>=20
>=20
>>=20
>> 2.2.  Scope of Information Elements
>>=20
>>    By default, most Information Elements have a scope specified in =
their
>>    definitions.
>>=20
>>    o  The Information Elements listed in Sections 5.2 and 5.3, and
>>       similar Information Elements in [IPFIX-IANA], have a default of =
"a
>>       specific Metering Process" or of "a specific Exporting =
Process",
>>       respectively.
>>=20
>>    o  The Information Elements listed in Sections 5.4-5.11, and =
similar
>>       Information Elements in [IPFIX-IANA], have a scope of "a =
specific
>>       Flow".
>>=20
>>    Within Data Records defined by Option Templates, the IPFIX =
protocol
>>    allows further limiting of the Information Element scope.  The new
>>    scope is specified by one or more scope fields and defined as the
>>    combination of all specified scope values; see Section 3.4.2.1 on
>>    IPFIX scopes in [RFC5101bis].
>>=20
>> =20
>>=20
>> Claise, Trammell            Standards Track                     [Page =
7]
>>=20
>> Internet-Draft          IPFIX Information Model          October 3, =
2012
>>=20
>>=20
>> 2.3.  Naming Conventions for Information Elements
>>=20
>>    The following naming conventions were used for naming Information
>>    Elements in this document.  It is recommended that extensions of =
the
>>    model use the same conventions.
>>=20
>>    o  Names of Information Elements SHOULD be descriptive.
>>=20
>>    o  Names of Information Elements MUST be unique within the IANA
>>       registry.   Enterprise-specific Information Elements SHOULD be
>>       prefixed with a vendor name.
>=20
> Unique within IANA's *IPFIX* registry.
> Add xref to the registry.
>=20
>=20
>>=20
>>    o  Names of Information Elements MUST start with non-capitalized
>>       letters.
>>=20
>>    o  Composed names MUST use capital letters for the first letter of
>>       each component (except for the first one).  All other letters =
are
>>       non-capitalized, even for acronyms.  Exceptions are made for
>>       acronyms containing non-capitalized letters, such as 'IPv4' and
>>       'IPv6'.  Examples are sourceMacAddress and =
destinationIPv4Address.
>=20
> Combination of the above rules means that IANA will name an IE "foo", =
while the ES equivalent is named "enterpriseFoo".
> It's unfortunate that "foo" !=3D "Foo".
>=20
>=20
>>=20
>>    o  Middleboxes [RFC3234] may change Flow properties, such as the
>>       Differentiated Service Code Point (DSCP) value or the source IP
>>       address.  If an IPFIX Observation Point is located in the path =
of
>>       a Flow before one or more middleboxes that potentially modify
>>       packets of the Flow, then it may be desirable to also report =
Flow
>>       properties after the modification performed by the middleboxes.
>>       An example is an Observation Point before a packet marker =
changing
>>       a packet's IPv4 Type of Service (TOS) field that is encoded in
>>       Information Element ipClassOfService.  Then the value observed =
and
>>       reported by Information Element ipClassOfService is valid at =
the
>>       Observation Point, but not after the packet passed the packet
>>       marker.  For reporting the change value of the TOS field, the
>>       IPFIX information model uses Information Elements that have a =
name
>>       prefix "post", for example, "postIpClassOfService".  =
Information
>>       Elements with prefix "post" report on Flow properties that are =
not
>>       necessarily observed at the Observation Point, but which are
>>       obtained within the Flow's Observation Domain by other means
>>       considered to be sufficiently reliable, for example, by =
analyzing
>>       the packet marker's marking tables.
>>=20
>> 3.  Type Space
>>=20
>>    This section describes the abstract data types that can be used =
for
>>    the specification of IPFIX Information Elements in Section 4.
>>    Section 3.1 describes the set of abstract data types.
>>=20
>>    Abstract data types unsigned8, unsigned16, unsigned32, unsigned64,
>> =20
>>=20
>> Claise, Trammell            Standards Track                     [Page =
8]
>>=20
>> Internet-Draft          IPFIX Information Model          October 3, =
2012
>>=20
>>=20
>>    signed8, signed16, signed32, and signed64 are integral data types.
>=20
> They're just different sizes of the same types.
>=20
>=20
>>    As described in Section 3.2, their data type semantics can be =
further
>>    specified, for example, by 'totalCounter', 'deltaCounter',
>>    'identifier', or 'flags'.
>>=20
>> 3.1.  Abstract Data Types
>>=20
>>    This section describes the set of valid abstract data types of the
>>    IPFIX information model.  Note that further abstract data types =
may
>>    be specified by future extensions of the IPFIX information model.
>>=20
>> 3.1.1.  unsigned8
>>=20
>>    The type "unsigned8" represents a non-negative integer value in =
the
>>    range of 0 to 255.
>>=20
>> 3.1.2.  unsigned16
>>=20
>>    The type "unsigned16" represents a non-negative integer value in =
the
>>    range of 0 to 65535.
>>=20
>> 3.1.3.  unsigned32
>>=20
>>    The type "unsigned32" represents a non-negative integer value in =
the
>>    range of 0 to 4294967295.
>>=20
>> 3.1.4.  unsigned64
>>=20
>>    The type "unsigned64" represents a non-negative integer value in =
the
>>    range of 0 to 18446744073709551615.
>>=20
>> 3.1.5.  signed8
>>=20
>>    The type "signed8" represents an integer value in the range of =
-128
>>    to 127.
>>=20
>> 3.1.6.  signed16
>>=20
>>    The type "signed16" represents an integer value in the range of
>>    -32768 to 32767.
>>=20
>> 3.1.7.  signed32
>>=20
>>    The type "signed32" represents an integer value in the range of
>>    -2147483648 to 2147483647.
>>=20
>> 3.1.8.  signed64
>>=20
>> =20
>>=20
>> Claise, Trammell            Standards Track                     [Page =
9]
>>=20
>> Internet-Draft          IPFIX Information Model          October 3, =
2012
>>=20
>>=20
>>    The type "signed64" represents an integer value in the range of
>>    -9223372036854775808 to 9223372036854775807.
>>=20
>> 3.1.9.  float32
>>=20
>>    The type "float32" corresponds to an IEEE single-precision 32-bit
>>    floating point type as defined in [IEEE.754.1985].
>>=20
>> 3.1.10.  float64
>>=20
>>    The type "float64" corresponds to an IEEE double-precision 64-bit
>>    floating point type as defined in [IEEE.754.1985].
>>=20
>> 3.1.11.  boolean
>>=20
>>    The type "boolean" represents a binary value.  The only allowed
>>    values are "true" and "false".
>>=20
>> 3.1.12.  macAddress
>>=20
>>    The type "macAddress" represents a string of 6 octets.
>>=20
>> 3.1.13.  octetArray
>>=20
>>    The type "octetArray" represents a finite-length string of octets.
>>=20
>> 3.1.14.  string
>>=20
>>    The type "string" represents a finite-length string of valid
>>    characters from the Unicode character encoding set
>>    [ISO.10646-1.1993].  Unicode allows for ASCII [ISO.646.1991] and =
many
>>    other international character sets to be used.
>>=20
>> 3.1.15.  dateTimeSeconds
>>=20
>>    The data type dateTimeSeconds is an unsigned 32-bit integer
>>    representing the number of seconds since the UNIX epoch, 1 January
>>    1970 at 00:00 UTC, as defined in [POSIX.1].
>>=20
>> 3.1.16.  dateTimeMilliseconds
>>=20
>>    The data type dateTimeMilliseconds is an unsigned 64-bit integer
>>    containing the number of milliseconds since the UNIX epoch, 1 =
January
>>    1970 at 00:00 UTC, as defined in [POSIX.1].
>>=20
>> 3.1.17.  dateTimeMicroseconds
>>=20
>>    The type "dateTimeMicroseconds" represents a time value with
>> =20
>>=20
>> Claise, Trammell            Standards Track                    [Page =
10]
>>=20
>> Internet-Draft          IPFIX Information Model          October 3, =
2012
>>=20
>>=20
>>    microsecond precision according to the NTP Timestamp format as
>>    defined in section 6 of [RFC5905].
>>=20
>> 3.1.18.  dateTimeNanoseconds
>>=20
>>    The type "dateTimeNanoseconds" represents a time value with
>>    nanosecond precision according to the NTP Timestamp format as =
defined
>>    in section 6 of [RFC5905].
>>=20
>> 3.1.19.  ipv4Address
>>=20
>>    The type "ipv4Address" represents a value of an IPv4 address.
>=20
> "a value", as if it has many values? What exactly is the "value" of an =
address?
>=20
> Consider "The type "ipv4Address" represents an IPv4 address." ?
>=20
>=20
>>=20
>> 3.1.20.  ipv6Address
>>=20
>>    The type "ipv6Address" represents a value of an IPv6 address.
>=20
> Similarly.
>=20
>=20
>>=20
>> 3.2.  Data Type Semantics
>>=20
>>    This section describes the set of valid data type semantics of the
>>    IPFIX information model. A registry of data type semantics is
>>    established in [RFC5610]; the restrictions on the use of semantics
>=20
> Surely IANA is the reference point, rather than 5610?
> eg, if new semantics are added, they'll be listed in IANA without =
raising errata against 5610.
>=20
>=20
>>    below are compatible with those specified in section 3.10 of that
>>    document. These semantics apply only to numeric types, as noted in
>>    the description of each semantic below.
>>=20
>>    Further data type semantics may be specified by future extensions =
of
>>    the IPFIX information model.
>=20
> State the required 5226 action / process for that, eg expert review.
> Or, xref the section where that's stated.
>=20
>=20
>>=20
>> 3.2.1.  quantity
>>=20
>>    A numeric (integral or floating point) value representing a =
measured
>>    value pertaining to the record. This is distinguished from =
counters
>>    that represent an ongoing measured value whose "odometer" reading =
is
>>    captured as part of a given record. This is the default semantic =
type
>>    of all numeric data types.
>>=20
>> 3.2.2.  totalCounter
>>=20
>>    An numeric value reporting the value of a counter. Counters are
>>    unsigned and wrap back to zero after reaching the limit of the =
type.
>>    For example, an unsigned64 with counter semantics will continue to
>>    increment until reaching the value of 2**64 - 1. At this point, =
the
>>    next increment will wrap its value to zero and continue counting =
from
>>    zero. The semantics of a total counter is similar to the semantics =
of
>>    counters used in SNMP, such as Counter32 defined in [RFC2578]. The
>>    only difference between total counters and counters used in SNMP =
is
>>    that the total counters have an initial value of 0. A total =
counter
>> =20
>>=20
>> Claise, Trammell            Standards Track                    [Page =
11]
>>=20
>> Internet-Draft          IPFIX Information Model          October 3, =
2012
>>=20
>>=20
>>    counts independently of the export of its value.
>>=20
>> 3.2.3.  deltaCounter
>>=20
>>    An numeric value reporting the value of a counter. Counters are
>>    unsigned and wrap back to zero after reaching the limit of the =
type.
>>    For example, an unsigned64 with counter semantics will continue to
>>    increment until reaching the value of 2**64 - 1. At this point, =
the
>>    next increment will wrap its value to zero and continue counting =
from
>>    zero. The semantics of a delta counter is similar to the semantics =
of
>>    counters used in SNMP, such as Counter32 defined in RFC 2578
>>    [RFC2578]. The only difference between delta counters and counters
>>    used in SNMP is that the delta counters have an initial value of =
0. A
>>    delta counter is reset to 0 each time its value is exported.
>=20
> What if the cache entry is removed but not exported (eg, an export =
filter blocks the export) ?
> Then the counter was not exported, so it should not be reset to 0?
>=20
> ie, the reset action is more to do with the cache entry expiring than =
whatever happens to it next.
>=20
>=20
>>=20
>> 3.2.4.  identifier
>>=20
>>    An integral value that serves as an identifier. Specifically,
>>    mathematical operations on two identifiers (aside from the =
equality
>>    operation) are meaningless. For example, Autonomous System ID 1 *
>>    Autonomous System ID 2 is meaningless. Identifiers MUST be one of =
the
>>    signed or unsigned data types.
>=20
> We could also have non-numeric identifiers, eg wlanSSID is a string =
identifier.
>=20
>=20
>>=20
>> 3.2.5.  flags
>>=20
>>    An integral value that represents a set of bit fields. Logical
>>    operations are appropriate on such values, but not other =
mathematical
>>    operations. Flags MUST always be of an unsigned data type.
>>=20
>> 4.  Information Element Identifiers
>>=20
>>    All Information Elements defined in the IANA IPFIX Information
>>    Element registry [IPFIX-IANA] have their identifiers assigned by
>>    IANA.
>>=20
>>    The value of these identifiers is in the range of 1-32767. Within
>>    this range, Information Element identifier values in the sub-range =
of
>>    1-127 are compatible with field types used by NetFlow version 9
>>    [RFC3954]; Information Element identifiers in this range MUST NOT =
be
>>    assigned unless the Information Element is compatible with the
>>    NetFlow version 9 protocol. Such Information Elements may ONLY be
>>    requested by a NetFlow v9 expert, to be designated by the IESG.
>>=20
>>    In general, IANA will add newly registered Information Elements to
>>    the registry, assigning the lowest available Information Element
>>    identifier in the range 128-32767.
>>=20
>>    Enterprise-specific Information Element identifiers have the same
>> =20
>>=20
>> Claise, Trammell            Standards Track                    [Page =
12]
>>=20
>> Internet-Draft          IPFIX Information Model          October 3, =
2012
>>=20
>>=20
>>    range of 1-32767, but they are coupled with an additional =
enterprise
>>    identifier. For enterprise-specific Information Elements, =
Information
>>    Element identifier 0 is also reserved. Enterprise-specific
>>    Information Element identifiers can be chosen by an enterprise
>>    arbitrarily within the range of 1-32767. The same identifier may =
be
>>    assigned by other enterprises for different purposes; these
>>    Information Elements are distinct because the Information Element
>>    identifier is coupled with an enterprise identifier.
>>=20
>>    Enterprise identifiers MUST be registered as SMI network =
management
>>    private enterprise code numbers with IANA.  The registry can be =
found
>>    at http://www.iana.org/assignments/enterprise-numbers.
>=20
> Add the URL as an I-ref per earlier comment.
>=20
>=20
>>=20
>> 4.1.  NetFlow version 9 compatible Information Element Identifiers
>>=20
>>    Information Elements with identifiers from 1-127 are reserved for
>>    compatibility with corresponding fields in NetFlow version 9
>>    [RFC3954].
>=20
> This simply repeats the second paragraph of section 4 above.
>=20
>=20
>>=20
>>=20
>> 5.  Information Element Categories
>>=20
>>    This section describes the Information Element category for the =
IPFIX
>>    information model at the time that [RFC5102] was published. Since
>>    this category field is not part of the IANA process for assigning =
new
>>    Information Element (even though it has been reused, for example, =
in
>=20
> s/Element/Elements/
>=20
>=20
>>    [RFC5103]), the newest Information Elements in IANA [IPFIX-IANA]
>>    don't have this classification. The elements are grouped into 12
>>    groups according to their semantics and their applicability:
>=20
> TBD: are categories useful? If not, let's say they're deprecated and =
not discuss them further.
>=20
>=20
>>=20
>>    1.   Identifiers
>>    2.   Metering and Exporting Process Configuration
>>    3.   Metering and Exporting Process Statistics
>>    4.   IP Header Fields
>>    5.   Transport Header Fields
>>    6.   Sub-IP Header Fields
>>    7.   Derived Packet Properties
>>    8.   Min/Max Flow Properties
>>    9.   Flow Timestamps
>>    10.  Per-Flow Counters
>>    11.  Miscellaneous Flow Properties
>>    12.  Padding
>>=20
>>    The Information Elements that are derived from fields of packets =
or
>=20
> s/fields of packets/packet fields/
>=20
>=20
>>    from packet treatment, such as the Information Elements in groups
>>    4-7, can typically serve as Flow Keys used for mapping packets to
>>    Flows.
>>=20
>> =20
>>=20
>> Claise, Trammell            Standards Track                    [Page =
13]
>>=20
>> Internet-Draft          IPFIX Information Model          October 3, =
2012
>>=20
>>=20
>>    If they do not serve as Flow Keys, their value may change from =
packet
>>    to packet within a single Flow.  For Information Elements with =
values
>>    that are derived from fields of packets or from packet treatment =
and
>=20
> s/fields of packets/packet fields/
>=20
>=20
>>    for which the value may change from packet to packet within a =
single
>>    Flow, the IPFIX information model defines that their value is
>>    determined by the first packet observed for the corresponding =
Flow,
>>    unless the description of the Information Element explicitly
>>    specifies a different semantics.  This simple rule allows writing =
all
>=20
> I don't think it's appropriate for the infomodel to define this.
>=20
> In some cases, the same IE may be observed in different ways according =
to the implementation.
> By the above definition, we'd need multiple IEs.
>=20
>=20
>>    Information Elements related to header fields once when the first
>>    packet of the Flow is observed.  For further observed packets of =
the
>>    same Flow, only Flow properties that depend on more than one =
packet,
>>    such as the Information Elements in groups 8-11, need to be =
updated.
>=20
> This model is based on an historic and simplistic understanding of the =
MP.
>=20
> Today we may not be able to determine all the key fields until some =
variable number of packets have been observed.
> eg, consider if a key field is in fragment N > 1.
>=20
>=20
>>=20
>>    Information Elements with a name having the "post" prefix, for
>>    example, "postIpClassOfService", do not report properties that =
were
>>    actually observed at the Observation Point, but retrieved by other
>>    means within the Observation Domain.  These Information Elements =
can
>>    be used if there are middlebox functions within the Observation
>>    Domain changing Flow properties after packets passed the =
Observation
>>    Point.
>=20
> s/changing/which change/
>=20
>=20
>>=20
>>=20
>> 5.1.  Identifiers
>>=20
>>    Information Elements grouped in the table below are identifying
>>    components of the IPFIX architecture, of an IPFIX Device, or of =
the
>>    IPFIX protocol.  All of them have an integral abstract data type =
and
>>    data type semantics "identifier" as described in Section 3.2.4.
>>=20
>>    Typically, some of them are used for limiting scopes of other
>>    Information Elements.  However, other Information Elements MAY be
>>    used for limiting scopes.  Note also that all Information Elements
>>    listed below MAY be used for other purposes than limiting scopes.
>>=20
>>    =
+-----+---------------------------+-----+---------------------------+
>>    |  ID | Name                      |  ID | Name                     =
 |
>>    =
+-----+---------------------------+-----+---------------------------+
>>    | 141 | lineCardId                | 148 | flowId                   =
 |
>>    | 142 | portId                    | 145 | templateId               =
 |
>>    |  10 | ingressInterface          | 149 | observationDomainId      =
 |
>>    |  14 | egressInterface           | 138 | observationPointId       =
 |
>>    | 143 | meteringProcessId         | 137 | commonPropertiesId       =
 |
>>    | 144 | exportingProcessId        |     |                          =
 |
>>    =
+-----+---------------------------+-----+---------------------------+
>>=20
>>    See [IPFIX-IANA] for the definitions of these Information =
Elements.
>=20
> Instead of repeating this over and over, just say it once in section =
5. ?
>=20
>=20
>>=20
>>=20
>> =20
>>=20
>> Claise, Trammell            Standards Track                    [Page =
14]
>>=20
>> Internet-Draft          IPFIX Information Model          October 3, =
2012
>>=20
>>=20
>>    5.2.  Metering and Exporting Process Configuration
>>=20
>>    Information Elements in this section describe the configuration of
>>    the Metering Process or the Exporting Process.  The set of these
>>    Information Elements is listed in the table below.
>>=20
>>    =
+-----+--------------------------+-----+----------------------------+
>>    |  ID | Name                     |  ID | Name                      =
 |
>>    =
+-----+--------------------------+-----+----------------------------+
>>    | 130 | exporterIPv4Address      | 213 | exportInterface           =
 |
>>    | 131 | exporterIPv6Address      | 214 | exportProtocolVersion     =
 |
>>    | 217 | exporterTransportPort    | 215 | exportTransportProtocol   =
 |
>>    | 211 | collectorIPv4Address     | 216 | collectorTransportPort    =
 |
>>    | 212 | collectorIPv6Address     | 173 | flowKeyIndicator          =
 |
>>    =
+-----+--------------------------+-----+----------------------------+
>>=20
>>    See [IPFIX-IANA] for the definitions of these Information =
Elements.
>>=20
>>=20
>>=20
>> 5.3.  Metering and Exporting Process Statistics
>>=20
>>    Information Elements in this section describe statistics of the
>>    Metering Process and/or the Exporting Process.  The set of these
>>    Information Elements is listed in the table below.
>>=20
>>    =
+-----+-----------------------------+-----+-------------------------+
>>    |  ID | Name                        |  ID | Name                   =
 |
>>    =
+-----+-----------------------------+-----+-------------------------+
>>    |  41 | exportedMessageTotalCount   | 165 | ignoredOctetTotalCount =
 |
>>    |  40 | exportedOctetTotalCount     | 166 | notSentFlowTotalCount  =
 |
>>    |  42 | exportedFlowRecordTotalCount| 167 | =
notSentPacketTotalCount |
>>    | 163 | observedFlowTotalCount      | 168 | notSentOctetTotalCount =
 |
>>    | 164 | ignoredPacketTotalCount     |     |                        =
 |
>>    =
+-----+-----------------------------+-----+-------------------------+
>>=20
>>    See [IPFIX-IANA] for the definitions of these Information =
Elements.
>>=20
>>=20
>>=20
>> 5.4.  IP Header Fields
>>=20
>>    Information Elements in this section indicate values of IP header
>>    fields or are derived from IP header field values in combination =
with
>>    further information.
>>=20
>>    =
+-----+----------------------------+-----+--------------------------+
>>    |  ID | Name                       |  ID | Name                    =
 |
>> =20
>>=20
>> Claise, Trammell            Standards Track                    [Page =
15]
>>=20
>> Internet-Draft          IPFIX Information Model          October 3, =
2012
>>=20
>>=20
>>    =
+-----+----------------------------+-----+--------------------------+
>>    |  60 | ipVersion                  | 193 | nextHeaderIPv6          =
 |
>>    |   8 | sourceIPv4Address          | 195 | ipDiffServCodePoint     =
 |
>>    |  27 | sourceIPv6Address          | 196 | ipPrecedence            =
 |
>>    |   9 | sourceIPv4PrefixLength     |   5 | ipClassOfService        =
 |
>>    |  29 | sourceIPv6PrefixLength     |  55 | postIpClassOfService    =
 |
>>    |  44 | sourceIPv4Prefix           |  31 | flowLabelIPv6           =
 |
>>    | 170 | sourceIPv6Prefix           | 206 | isMulticast             =
 |
>>    |  12 | destinationIPv4Address     |  54 | fragmentIdentification  =
 |
>>    |  28 | destinationIPv6Address     |  88 | fragmentOffset          =
 |
>>    |  13 | destinationIPv4PrefixLength| 197 | fragmentFlags           =
 |
>>    |  30 | destinationIPv6PrefixLength| 189 | ipHeaderLength          =
 |
>>    |  45 | destinationIPv4Prefix      | 207 | ipv4IHL                 =
 |
>>    | 169 | destinationIPv6Prefix      | 190 | totalLengthIPv4         =
 |
>>    | 192 | ipTTL                      | 224 | ipTotalLength           =
 |
>>    |   4 | protocolIdentifier         | 191 | payloadLengthIPv6       =
 |
>>    =
+-----+----------------------------+-----+--------------------------+
>>=20
>>    See [IPFIX-IANA] for the definitions of these Information =
Elements.
>>=20
>>=20
>>=20
>> 5.5.  Transport Header Fields
>>=20
>>    The set of Information Elements related to transport header fields
>>    and length includes the Information Elements listed in the table
>>    below.
>>=20
>>    =
+-----+---------------------------+-----+---------------------------+
>>    |  ID | Name                      |  ID | Name                     =
 |
>>    =
+-----+---------------------------+-----+---------------------------+
>>    |   7 | sourceTransportPort       | 238 | tcpWindowScale           =
 |
>>    |  11 | destinationTransportPort  | 187 | tcpUrgentPointer         =
 |
>>    | 180 | udpSourcePort             | 188 | tcpHeaderLength          =
 |
>>    | 181 | udpDestinationPort        |  32 | icmpTypeCodeIPv4         =
 |
>>    | 205 | udpMessageLength          | 176 | icmpTypeIPv4             =
 |
>>    | 182 | tcpSourcePort             | 177 | icmpCodeIPv4             =
 |
>>    | 183 | tcpDestinationPort        | 139 | icmpTypeCodeIPv6         =
 |
>>    | 184 | tcpSequenceNumber         | 178 | icmpTypeIPv6             =
 |
>>    | 185 | tcpAcknowledgementNumber  | 179 | icmpCodeIPv6             =
 |
>>    | 186 | tcpWindowSize             |  33 | igmpType                 =
 |
>>    =
+-----+---------------------------+-----+---------------------------+
>>=20
>>    See [IPFIX-IANA] for the definitions of these Information =
Elements.
>>=20
>>=20
>>=20
>>=20
>> =20
>>=20
>> Claise, Trammell            Standards Track                    [Page =
16]
>>=20
>> Internet-Draft          IPFIX Information Model          October 3, =
2012
>>=20
>>=20
>> 5.6.  Sub-IP Header Fields
>>=20
>>    The set of Information Elements related to Sub-IP header fields
>>    includes the Information Elements listed in the table below.
>>=20
>>    =
+-----+---------------------------+-----+---------------------------+
>>    |  ID | Name                      |  ID | Name                     =
 |
>>    =
+-----+---------------------------+-----+---------------------------+
>>    |  56 | sourceMacAddress          | 201 | mplsLabelStackLength     =
 |
>>    |  81 | postSourceMacAddress      | 194 | mplsPayloadLength        =
 |
>>    |  58 | vlanId                    |  70 | mplsTopLabelStackSection =
 |
>>    |  59 | postVlanId                |  71 | mplsLabelStackSection2   =
 |
>>    |  80 | destinationMacAddress     |  72 | mplsLabelStackSection3   =
 |
>>    |  57 | postDestinationMacAddress |  73 | mplsLabelStackSection4   =
 |
>>    | 146 | wlanChannelId             |  74 | mplsLabelStackSection5   =
 |
>>    | 147 | wlanSSID                  |  75 | mplsLabelStackSection6   =
 |
>>    | 200 | mplsTopLabelTTL           |  76 | mplsLabelStackSection7   =
 |
>>    | 203 | mplsTopLabelExp           |  77 | mplsLabelStackSection8   =
 |
>>    | 237 | postMplsTopLabelExp       |  78 | mplsLabelStackSection9   =
 |
>>    | 202 | mplsLabelStackDepth       |  79 | mplsLabelStackSection10  =
 |
>>    =
+-----+---------------------------+-----+---------------------------+
>>=20
>>    See [IPFIX-IANA] for the definitions of these Information =
Elements.
>>=20
>>=20
>> 5.7.  Derived Packet Properties
>>=20
>>    The set of Information Elements derived from packet properties =
(for
>>    example, values of header fields) includes the Information =
Elements
>>    listed in the table below.
>>=20
>>    =
+-----+---------------------------+-----+---------------------------+
>>    |  ID | Name                      |  ID | Name                     =
 |
>>    =
+-----+---------------------------+-----+---------------------------+
>>    | 204 | ipPayloadLength           |  18 | bgpNextHopIPv4Address    =
 |
>>    |  15 | ipNextHopIPv4Address      |  63 | bgpNextHopIPv6Address    =
 |
>>    |  62 | ipNextHopIPv6Address      |  46 | mplsTopLabelType         =
 |
>>    |  16 | bgpSourceAsNumber         |  47 | mplsTopLabelIPv4Address  =
 |
>>    |  17 | bgpDestinationAsNumber    | 140 | mplsTopLabelIPv6Address  =
 |
>>    | 128 | bgpNextAdjacentAsNumber   |  90 | =
mplsVpnRouteDistinguisher |
>>    | 129 | bgpPrevAdjacentAsNumber   |     |                          =
 |
>>    =
+-----+---------------------------+-----+---------------------------+
>>=20
>>    See [IPFIX-IANA] for the definitions of these Information =
Elements.
>>=20
>>=20
>>=20
>>=20
>> =20
>>=20
>> Claise, Trammell            Standards Track                    [Page =
17]
>>=20
>> Internet-Draft          IPFIX Information Model          October 3, =
2012
>>=20
>>=20
>> 5.9.  Flow Timestamps
>>=20
>>    Information Elements in this section are timestamps of events.
>>=20
>>    Timestamps flowStartSeconds, flowEndSeconds, =
flowStartMilliseconds,
>>    flowEndMilliseconds, flowStartMicroseconds, flowEndMicroseconds,
>>    flowStartNanoseconds, flowEndNanoseconds, and
>>    systemInitTimeMilliseconds are absolute and have a well-defined =
fixed
>>    time base, such as, for example, the number of seconds since 0000 =
UTC
>>    Jan 1st 1970.
>=20
> It's a bit dangerous to give this example, since it could be misread =
as being the actual definition.
> xref sections 3.1.15 - 3.1.18 where the time bases are stated.
>=20
>=20
>>=20
>>    Timestamps flowStartDeltaMicroseconds and flowEndDeltaMicroseconds
>>    are relative timestamps only valid within the scope of a single
>>    IPFIX Message.  They contain the negative time offsets relative to
>>    the export time specified in the IPFIX Message Header.  The =
maximum
>=20
> In order for the EP to populate *DeltaMicroseconds in a flow record, =
it must first know what Export Time it's going to stamp into the IPFIX =
header, and the flow record must be exported with that given second... =
unless we allow that data may be exported somewhat asynchronously to the =
header timestamping (eg, if there's a queue of outgoing packets at a =
level below the EP, eg in the IP stack). If a flow's export is delayed =
such that the Export Time changes, then these deltas must be =
recalculated. Practically, that may not be possible.
>=20
> Anyway, why do we have microsecond offsets from a "seconds" time?
>=20
> In short, these two IEs seem flawed and should be deprecated.
>=20
>=20
>>    time offset that can be encoded by these delta counters is 1 hour, =
11
>>    minutes, and 34.967295 seconds.
>>=20
>>    Timestamps flowStartSysUpTime and flowEndSysUpTime are relative
>>    timestamps indicating the time relative to the last
>>    (re-)initialization of the IPFIX Device.  For reporting the time
>>    of the last (re-)initialization, systemInitTimeMilliseconds can
>>    be reported, for example, in Data Records defined by Option
>>    Templates.
>>=20
>>    =
+-----+---------------------------+-----+---------------------------+
>>    |  ID | Name                      |  ID | Name                     =
 |
>>    =
+-----+---------------------------+-----+---------------------------+
>>    | 150 | flowStartSeconds          | 156 | flowStartNanoseconds     =
 |
>>    | 151 | flowEndSeconds            | 157 | flowEndNanoseconds       =
 |
>>    | 152 | flowStartMilliseconds     | 158 | =
flowStartDeltaMicroseconds|
>>    | 153 | flowEndMilliseconds       | 159 | flowEndDeltaMicroseconds =
 |
>>    | 154 | flowStartMicroseconds     | 160 | =
systemInitTimeMilliseconds|
>>    | 155 | flowEndMicroseconds       |  22 | flowStartSysUpTime       =
 |
>>    |     |                           |  21 | flowEndSysUpTime         =
 |
>>    =
+-----+---------------------------+-----+---------------------------+
>>=20
>>    See [IPFIX-IANA] for the definitions of these Information =
Elements.
>>=20
>> 5.10.  Per-Flow Counters
>>=20
>>    Information Elements in this section are counters all having =
integer
>>    values.  Their values may change for every report they are used =
in.
>>    They cannot serve as part of a Flow Key used for mapping packets =
to
>>    Flows.  However, potentially they can be used for selecting =
exported
>=20
> Well, octetDeltaCount could be used to make all packets of the same =
size hash to the same bucket.
>=20
> More realistically, these could be used when aggregating flows into =
other flows. eg, all the flows with the same number of packets, or all =
the flows with the same TCP SYN count.
>=20
>=20
>>    Flows, for example, by only exporting Flows with more than a
>>    threshold number of observed octets.
>>=20
>> =20
>>=20
>> Claise, Trammell            Standards Track                    [Page =
18]
>>=20
>> Internet-Draft          IPFIX Information Model          October 3, =
2012
>>=20
>>=20
>>    There are running counters and delta counters.  Delta counters are
>>    reset to zero each time their values are exported.  Running =
counters
>>    continue counting independently of the Exporting Process.
>>=20
>>    There are per-Flow counters and counters related to the Metering
>>    Process and/or the Exporting Process.  Per-Flow counters are Flow
>>    properties that potentially change each time a packet belonging to
>>    the Flow is observed.  The set of per-Flow counters includes the
>>    Information Elements listed in the table below.  Counters related =
to
>>    the Metering Process and/or the Exporting Process are described in
>>    Section 5.3.
>>=20
>>    =
+-----+---------------------------+-----+---------------------------+
>>    |  ID | Name                      |  ID | Name                     =
 |
>>    =
+-----+---------------------------+-----+---------------------------+
>>    |   1 | octetDeltaCount           | 134 | droppedOctetTotalCount   =
 |
>>    |  23 | postOctetDeltaCount       | 135 | droppedPacketTotalCount  =
 |
>>    | 198 | octetDeltaSumOfSquares    |  19 | =
postMCastPacketDeltaCount |
>>    |  85 | octetTotalCount           |  20 | postMCastOctetDeltaCount =
 |
>>    | 171 | postOctetTotalCount       | 174 | =
postMCastPacketTotalCount |
>>    | 199 | octetTotalSumOfSquares    | 175 | postMCastOctetTotalCount =
 |
>>    |   2 | packetDeltaCount          | 218 | tcpSynTotalCount         =
 |
>>    |  24 | postPacketDeltaCount      | 219 | tcpFinTotalCount         =
 |
>>    |  86 | packetTotalCount          | 220 | tcpRstTotalCount         =
 |
>>    | 172 | postPacketTotalCount      | 221 | tcpPshTotalCount         =
 |
>>    | 132 | droppedOctetDeltaCount    | 222 | tcpAckTotalCount         =
 |
>>    | 133 | droppedPacketDeltaCount   | 223 | tcpUrgTotalCount         =
 |
>>    =
+-----+---------------------------+-----+---------------------------+
>>=20
>>=20
>> See [IPFIX-IANA] for the definitions of these Information Elements.
>>=20
>>=20
>> 5.11.  Miscellaneous Flow Properties
>>=20
>>    Information Elements in this section describe properties of Flows
>>    that are related to Flow start, Flow duration, and Flow =
termination,
>>    but they are not timestamps as the Information Elements in Section
>>    5.9 are.
>>=20
>>    =
+-----+---------------------------+-----+---------------------------+
>>    |  ID | Name                      |  ID | Name                     =
 |
>>    =
+-----+---------------------------+-----+---------------------------+
>>    |  36 | flowActiveTimeout         | 161 | flowDurationMilliseconds =
 |
>>    |  37 | flowIdleTimeout           | 162 | flowDurationMicroseconds =
 |
>>    | 136 | flowEndReason             |  61 | flowDirection            =
 |
>>    =
+-----+---------------------------+-----+---------------------------+
>>=20
>> =20
>>=20
>> Claise, Trammell            Standards Track                    [Page =
19]
>>=20
>> Internet-Draft          IPFIX Information Model          October 3, =
2012
>>=20
>>=20
>> See [IPFIX-IANA] for the definitions of these Information Elements.
>>=20
>>=20
>>=20
>> 5.12.  Padding
>>=20
>>    This section contains a single Information Element that can be =
used
>>    for padding of Flow Records.
>>=20
>>    IPFIX implementations may wish to align Information Elements =
within
>>    Data Records or to align entire Data Records to 4-octet or 8-octet
>>    boundaries.  This can be achieved by including one or more
>>    paddingOctets Information Elements in a Data Record.
>>=20
>>    =
+-----+---------------------------+-----+---------------------------+
>>    |  ID | Name                      |  ID | Name                     =
 |
>>    =
+-----+---------------------------+-----+---------------------------+
>>    | 210 | paddingOctets             |     |                          =
 |
>>    =
+-----+---------------------------+-----+---------------------------+
>>=20
>> See [IPFIX-IANA] for the definitions of these Information Elements.
>>=20
>>=20
>>=20
>> 6.  Extending the Information Model
>>=20
>>    A key requirement for IPFIX is to allow for extension of the
>>    Information Model maintained by IANA. The process for extending =
the
>>    Information Model is described in [IPFIX-IE-DOCTORS], which also
>>    provides guidelines for authors and reviewers of new Information
>>    Element definitions.
>>=20
>>    For new Information Elements, the type space defined in Section 3 =
can
>>    be used. If required, new abstract data types can be added to the
>>    subregistry defined in [RFC5610]. New abstract data types MUST be
>>    defined in IETF Standards Track documents.
>=20
> Isn't IANA the master reference for that registry?
>=20
> What's the policy for adding new IEs? By which I mean, cite one of the =
definitions from section 4.1 of RFC5226.
>=20
>=20
>>=20
>>    Enterprises may wish to define Information Elements without
>>    registering them with IANA. IPFIX explicitly supports
>>    enterprise-specific Information Elements. Enterprise-specific
>>    Information Elements are described in Sections 2.1 and 4; =
guidelines
>>    for using them appear in [IPFIX-IE-DOCTORS].
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>> =20
>>=20
>> Claise, Trammell            Standards Track                    [Page =
20]
>>=20
>> Internet-Draft          IPFIX Information Model          October 3, =
2012
>>=20
>>=20
>> 7.  IANA Considerations
>>=20
>> 7.1.  IPFIX Information Elements
>>=20
>> This document refers to Information Elements, for which the Internet
>> Assigned Numbers Authority (IANA) has created the IPFIX Information
>> Element Registry [IPFIX-IANA]. The columns of this registry must at
>> minimum be able to store the information defined in the template in
>> Section 2.1; it may contain other information as necessary for the
>> management of the registry.
>>=20
>> New assignments for IPFIX Information Elements will be administered =
by
>=20
> s/will be/are/
>=20
>=20
>> IANA through Expert Review [RFC5226], i.e., review by one of a group =
of
>> experts designated by the IESG. Further considerations for this =
review
>> are specified in [IPFIX-IE-DOCTORS].
>>=20
>> Future assignments added to the IPFIX Information Element Registry =
which
>> require subregistries for enumerated values (e.g. section 7.2, below)
>> must have those subregistries added simultaneously with the new
>> assignment; additions to these subregistries must be subject to =
Expert
>> Review [RFC5226]. Unless specified at assignment time, the experts =
for
>> the subregistry will be the same as for the Information Element =
registry
>> as a whole.
>>=20
>> Changes may also be made to the entries in this registry from time to
>> time; the process for these changes are specified in =
[IPFIX-IE-DOCTORS].
>>=20
>> [NOTE to IANA: please update the Reference for the IPFIX Information
>> Element Registry to refer to this document.]
>>=20
>> [NOTE to IANA: on publication of this document, please set the =
Revision
>> of all existing Information Elements to 0.]
>>=20
>> [NOTE to IANA: on publication of this document, please set the Date =
of
>> all existing Information Elements to the publication date of this
>> document.]
>>=20
>> [NOTE to IANA: on publication of this document, please set the Name =
of
>> all existing Reserved Information Elements to "Assigned for NetFlow =
v9
>> compatibility", and the reference to [RFC3954].]
>=20
> NB this works now that 312 and 315 have been assigned as requested in =
data-link-layer-monitoring, else those would have been incorrectly =
attributed.
>=20
>=20
>>=20
>> 7.2.  MPLS Label Type Identifier
>>=20
>> Information Element #46, named mplsTopLabelType, carries MPLS label
>> types.  Values for 5 different types have initially been defined.  =
For
>> ensuring extensibility of this information, IANA has created a new
>> subregistry for MPLS label types and filled it with the initial list
>> from the description Information Element #46, mplsTopLabelType.
>> =20
>>=20
>> Claise, Trammell            Standards Track                    [Page =
21]
>>=20
>> Internet-Draft          IPFIX Information Model          October 3, =
2012
>>=20
>>=20
>> New assignments for MPLS label types will be administered by IANA
>=20
> s/will be/are/
>=20
>=20
>> through Expert Review [RFC5226], i.e., review by one of a group of
>> experts designated by an IETF Area Director.  The group of experts =
must
>> double check the label type definitions with already defined label =
types
>> for completeness, accuracy, and redundancy.  The specification of new
>> MPLS label types MUST be published using a well-established and
>> persistent publication medium.
>>=20
>> [NOTE to IANA: please update the Reference for the IPFIX MPLS Label =
Type
>> subregistry to refer to this document.]
>>=20
>> 7.3.  XML Namespace and Schema
>>=20
>> [IPFIX-XML-SCHEMA] defines an XML schema for IPFIX Information =
Element
>> definitions.  All Information Elements specified in [IPFIX-IANA] are
>> defined by this schema.  This schema may also be used for specifying
>> further Information Elements in future extensions of the IPFIX
>> information model in a machine-readable way.
>>=20
>> [IPFIX-XML-SCHEMA] uses URNs to describe an XML namespace and an XML
>> schema for IPFIX Information Elements conforming to a registry =
mechanism
>> described in [RFC3688].  Two URI assignments have been made.
>>=20
>> 1.  Registration for the IPFIX information model namespace
>>     *  URI: urn:ietf:params:xml:ns:ipfix-info
>>     *  Registrant Contact: IETF IPFIX Working Group ,
>>        as designated by the IESG .
>>     *  XML: None.  Namespace URIs do not represent an XML.
>>=20
>> 2.  Registration for the IPFIX information model schema
>>     *  URI: urn:ietf:params:xml:schema:ipfix-info
>>     *  Registrant Contact: IETF IPFIX Working Group ,
>>        as designated by the IESG .
>>=20
>> Using a machine-readable syntax for the information model enables the
>> creation of IPFIX-aware tools that can automatically adapt to
>> extensions to the information model, by simply reading updated
>> information model specifications.
>>=20
>> The wide availability of XML-aware tools and libraries for client
>> devices is a primary consideration for this choice.  In particular,
>> libraries for parsing XML documents are readily available.  Also,
>> mechanisms such as the Extensible Stylesheet Language (XSL) allow for
>> transforming a source XML document into other documents.  This
>> document was authored in XML and transformed according to [RFC2629].
>>=20
>> It should be noted that the use of XML in Exporters, Collectors, or
>> other tools is not mandatory for the deployment of IPFIX.  In
>> =20
>>=20
>> Claise, Trammell            Standards Track                    [Page =
22]
>>=20
>> Internet-Draft          IPFIX Information Model          October 3, =
2012
>>=20
>>=20
>> particular, Exporting Processes do not produce or consume XML as part
>> of their operation.  It is expected that IPFIX Collectors MAY take
>> advantage of the machine readability of the information model vs.
>> hard coding their behavior or inventing proprietary means for
>> accommodating extensions.
>>=20
>> [NOTE to IANA: please update the Reference for the the IPFIX
>> information model namespace and schema to refer to this document.]
>>=20
>> 7.4.  Addition, Revision, and Deprecation
>>=20
>> As stated in Section 6, addition, revision, and deletion of =
Information
>> Elements in the IPFIX Information Element registry is subject to a
>> process described in [IPFIX-IE-DOCTORS]. The IE-DOCTORS experts =
mentions
>=20
> s/mentions/mentioned/
>=20
>=20
>> in this process are to be appointed by the IESG.
>=20
> When was/will that be done? Where are/will they be listed? How will =
IANA know who they are?
>=20
>=20
>>=20
>> When IANA receives a request to add, revise, or deprecate an =
Information
>> Element in the IPFIX Information Elements Registr, it forwards the
>=20
> s/Registr/Register/
>=20
>=20
>> request to the IE-DOCTORS experts for review.
>>=20
>> When IANA receives an approval for a request to add an Information
>> Element definition from the IE-DOCTORS experts, it adds that =
Information
>> Element to the registry. The approved request may include changes =
from
>> the original request.
>=20
> Changes made by the requester, the experts, or IANA?
>=20
>=20
> P.
>=20
>=20
>>=20
>> When IANA receives an approval for a request to revise an Information
>> Element definition from the IE-DOCTORS experts, it changes that
>> Information Element's definition in the registry, and updates the
>> Revision and Date columns as appropriate. The approved request may
>> include changes from the original request. If the original =
Information
>> Element was added to the registry with IETF consensus (i.e., was =
defined
>> by an RFC), the revision will require IETF consensus as well.
>>=20
>> When IANA receives an approval for a request to deprecate an =
Information
>> Element definition from the IE-DOCTORS experts, it changes that
>> Information Element's definition in the registry, and updates the
>> Revision and Date columns as appropriate. The approved request may
>> include changes from the original request. If the original =
Information
>> Element was added to the registry with IETF consensus (i.e., was =
defined
>> by an RFC), the deprecation will require IETF consensus as well.
>>=20
>>=20
>> 8.  Security Considerations
>>=20
>> The IPFIX information model itself does not directly introduce =
security
>> issues.  Rather, it defines a set of attributes that may for privacy =
or
>> business issues be considered sensitive information.
>>=20
>> =20
>>=20
>> Claise, Trammell            Standards Track                    [Page =
23]
>>=20
>> Internet-Draft          IPFIX Information Model          October 3, =
2012
>>=20
>>=20
>> For example, exporting values of header fields may make attacks =
possible
>> for the receiver of this information, which would otherwise only be
>> possible for direct observers of the reported Flows along the data =
path.
>>=20
>> The underlying protocol used to exchange the information described =
here
>> must therefore apply appropriate procedures to guarantee the =
integrity
>> and confidentiality of the exported information.  Such protocols are
>> defined in separate documents, specifically the IPFIX protocol =
document
>> [RFC5101bis].
>>=20
>> This document does not specify any Information Element carrying =
keying
>> material.  If future extensions will do so, then appropriate =
precautions
>> need to be taken for properly protecting such sensitive information.
>>=20
>> 9.  Acknowledgements
>>=20
>> The editors would like to thanks the authors of the RFC5102 =
[RFC5102],
>> as this document is directly based upon this original RFC: Juergen
>> Quittek, Stewart Bryant, Paul Aitken, and Jeff Meyer.
>>=20
>> 10.  References
>>=20
>> 10.1.  Normative References
>>=20
>>    [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
>>               Requirement Levels", BCP 14, RFC 2119, March 1997.
>>=20
>>    [RFC5905]  Mills, D., Delaware, U., Martin, J., Burbank, J. and W.
>>               Kasch, "Network Time Protocol Version 4: Protocol and
>>               Algorithms Specification", RFC 5905, June 2010
>>=20
>>    [RFC5101bis]
>>               Claise, B., and B. Trammell, Editors, "Specification of
>>               the IP Flow Information eXport (IPFIX) Protocol for the
>>               Exchange of IP Traffic Flow Information", draft-ietf-
>>               ipfix-protocol-rfc5101bis-00, Work in Progress, =
November
>>               2011.
>>=20
>>    [IPFIX-IE-DOCTORS]
>>               Trammell, B., and B. Claise, "Guidelines for Authors =
and
>>               Reviewers of IPFIX Information Elements", draft-ietf-
>>               ipfix-ie-doctors-00, Work in Progress, November 2011.
>>=20
>> 10.2.  Informative References
>>=20
>>    [IEEE.754.1985]
>>               Institute of Electrical and Electronics Engineers,
>>               "Standard for Binary Floating-Point Arithmetic", IEEE
>> =20
>>=20
>> Claise, Trammell            Standards Track                    [Page =
24]
>>=20
>> Internet-Draft          IPFIX Information Model          October 3, =
2012
>>=20
>>=20
>>               Standard 754, August 1985.
>>=20
>>    [ISO.10646-1.1993]
>>               International Organization for Standardization,
>>               "Information Technology - Universal Multiple-octet =
coded
>>               Character Set (UCS) - Part 1: Architecture and Basic
>>               Multilingual Plane", ISO Standard 10646-1, May 1993.
>>=20
>>    [ISO.646.1991]
>>               International Organization for Standardization,
>>               "Information technology - ISO 7-bit coded character set
>>               for information interchange", ISO Standard 646, 1991.
>>              =20
>>    [POSIX.1]  IEEE 1003.1-2008 - IEEE Standard for Information
>>               Technology - Portable Operating System Interface, IEEE,
>>               2008.
>>=20
>>    [RFC2578]  McCloghrie, K., Perkins, D., and J. Schoenwaelder,
>>               "Structure of Management Information Version 2 =
(SMIv2)",
>>               STD 58, RFC 2578, April 1999.
>>=20
>>    [RFC2629]  Rose, M., "Writing I-Ds and RFCs using XML", RFC 2629,
>>               June 1999.
>>=20
>>    [RFC3234]  Carpenter, B. and S. Brim, "Middleboxes: Taxonomy and
>>               Issues", RFC 3234, February 2002.
>>=20
>>    [RFC3444]  Pras, A. and J. Schoenwaelder, "On the Difference =
between
>>               Information Models and Data Models", RFC 3444, January
>>               2003.
>>=20
>>    [RFC3688]  Mealling, M., "The IETF XML Registry", BCP 81, RFC =
3688,
>>               January 2004.
>>=20
>>    [RFC3917]  Quittek, J., Zseby, T., Claise, B., and S. Zander,
>>               "Requirements for IP Flow Information Export (IPFIX)", =
RFC
>>               3917, October 2004.
>>=20
>>    [RFC3954]  Claise, B., Ed., "Cisco Systems NetFlow Services Export
>>               Version 9", RFC 3954, October 2004.
>>=20
>>    [RFC5102]  Trammell, B., and E. Boschi, "Bidirectional Flow Export
>>               Using IP Flow Information Export (IPFIX)", RFC 5103,
>>               January 2008.
>>=20
>>    [RFC5103]  Quittek, J., Bryant, S. Claise, B., Aitken, P., and J.
>>               Meyer, "Information Model for IP Flow Information =
Export",
>> =20
>>=20
>> Claise, Trammell            Standards Track                    [Page =
25]
>>=20
>> Internet-Draft          IPFIX Information Model          October 3, =
2012
>>=20
>>=20
>>               RFC 5102, January 2008.
>>=20
>>    [RFC5153]  Boschi, E., Mark, L., Quittek J., and P. Aitken, "IP =
Flow
>>               Information Export (IPFIX) Implementation Guidelines",
>>               RFC5153, April 2008.
>>=20
>>    [RFC5226]  Narten, T. and H. Alvestrand, "Guidelines for Writing =
an
>>               IANA Considerations Section in RFCs", BCP 26, RFC 5226,
>>               May 2008.
>>=20
>>    [RFC5470]  Sadasivan, G., Brownlee, N., Claise, B., and J. =
Quittek,
>>               "Architecture for IP Flow Information Export", RFC5470,
>>               March 2009.
>>=20
>>    [RFC5471]  Schmoll, C., Aitken, P., and B. Claise, "Guidelines for =
IP
>>               Flow Information Export (IPFIX) Testing", RFC5471, =
March
>>               2009.
>>=20
>>    [RFC5472]  Zseby, T., Boschi, E., Brownlee, N., and B. Claise, "IP
>>               Flow Information Export (IPFIX) Applicability", =
RFC5472,
>>               March 2009.
>>=20
>>    [RFC5473]  Boschi, E., Mark, L., and B. Claise, "Reducing =
Redundancy
>>               in IP Flow Information Export (IPFIX) and Packet =
Sampling
>>               (PSAMP) Reports", RFC5473, March 2009.
>>=20
>>    [RFC5610]  Boschi, E., Trammell, B., Mark, L., and T. Zseby,
>>               "Exporting Type Information for IP Flow Information =
Export
>>               (IPFIX) Information Elements", July 2009.
>>=20
>>    [RFC6313]  Claise, B., Dhandapani, G., Aitken, P, and S. Yates,
>>               "Export of Structured Data in IP Flow Information =
Export
>>               (IPFIX)", RFC6313, July 2011.
>>=20
>>    [RFC6183]  Kobayashi, A., Claise, B., Muenz, G, and K. Ishibashi, =
"IP
>>               Flow Information Export (IPFIX) Mediation: Framework",
>>               RFC6183, April 2011.
>>=20
>>    [IPFIX-CONF]
>>               Muenz, G., Claise, B., and P. Aitken, "Configuration =
Data
>>               Model for IPFIX and PSAMP", draft-ietf-ipfix-
>>               configuration-model-10, Work in Progress, July 2011.
>>=20
>>    [IPFIX-MED-PROTO]
>>               Claise, B., Kobayashi, A., and B. Trammell, =
"Specification
>>               of the Protocol for IPFIX Mediations", =
draft-ietf-ipfix-
>>               mediation-protocol-00, Work in Progress, December 2011.
>>=20
>> =20
>>=20
>> Claise, Trammell            Standards Track                    [Page =
26]
>>=20
>> Internet-Draft          IPFIX Information Model          October 3, =
2012
>>=20
>>=20
>>    [RFC5815bis]
>>               Dietz, T., Kobayashi, A., Claise, B., and G. Muenz,
>>               "Definitions of Managed Objects for IP Flow Information
>>               Export", draft-ietf-ipfix-rfc5815bis-01.txt, Work in
>>               Progress, January 2012.
>>=20
>>    [IPFIX-IANA]
>>               http://www.iana.org/assignments/ipfix/ipfix.xml
>>=20
>>    [IPFIX-XML-SCHEMA]
>>               http://www.iana.org/assignments/xml-
>>               registry/schema/ipfix.xsd
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>> =20
>>=20
>> Claise, Trammell            Standards Track                    [Page =
27]
>>=20
>> Internet-Draft          IPFIX Information Model          October 3, =
2012
>>=20
>>=20
>> Authors' Addresses
>>=20
>>    Benoit Claise
>>    Cisco Systems, Inc.
>>    De Kleetlaan 6a b1
>>    1831 Diegem
>>    Belgium
>>=20
>>    Phone: +32 2 704 5622
>>    EMail: bclaise@cisco.com
>>=20
>>=20
>>    Brian Trammell
>>    Swiss Federal Institute of Technology Zurich
>>    Gloriastrasse 35
>>    8092 Zurich
>>    Switzerland
>>=20
>>    Phone: +41 44 632 70 13
>>    EMail: trammell@tik.ee.ethz.ch
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>> Claise, Trammell            Standards Track                    [Page =
28]


From trammell@tik.ee.ethz.ch  Wed Oct 31 02:43:04 2012
Return-Path: 
X-Original-To: ipfix@ietfa.amsl.com
Delivered-To: ipfix@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 01E5721F845F for ; Wed, 31 Oct 2012 02:43:04 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -6.671
X-Spam-Level: 
X-Spam-Status: No, score=-6.671 tagged_above=-999 required=5 tests=[AWL=0.728,  BAYES_00=-2.599, GB_I_LETTER=-2, J_CHICKENPOX_33=0.6, J_CHICKENPOX_37=0.6, RCVD_IN_DNSWL_MED=-4]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jZzAqibQ1D0u for ; Wed, 31 Oct 2012 02:43:02 -0700 (PDT)
Received: from smtp.ee.ethz.ch (smtp.ee.ethz.ch [129.132.2.219]) by ietfa.amsl.com (Postfix) with ESMTP id 8AEF021F845C for ; Wed, 31 Oct 2012 02:43:01 -0700 (PDT)
Received: from localhost (localhost [127.0.0.1]) by smtp.ee.ethz.ch (Postfix) with ESMTP id 56393D930C; Wed, 31 Oct 2012 10:43:00 +0100 (MET)
X-Virus-Scanned: by amavisd-new on smtp.ee.ethz.ch
Received: from smtp.ee.ethz.ch ([127.0.0.1]) by localhost (.ee.ethz.ch [127.0.0.1]) (amavisd-new, port 10024) with LMTP id vDfk4Px6JcVz; Wed, 31 Oct 2012 10:43:00 +0100 (MET)
Received: from [10.0.27.100] (cust-integra-121-161.antanet.ch [80.75.121.161]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: briant) by smtp.ee.ethz.ch (Postfix) with ESMTPSA id 9E789D9309; Wed, 31 Oct 2012 10:42:59 +0100 (MET)
Mime-Version: 1.0 (Apple Message framework v1283)
Content-Type: text/plain; charset=iso-8859-1
From: Brian Trammell 
In-Reply-To: <5090547C.5020803@cisco.com>
Date: Wed, 31 Oct 2012 10:42:58 +0100
Content-Transfer-Encoding: quoted-printable
Message-Id: 
References: <506CBFE3.10607@auckland.ac.nz> <5090547C.5020803@cisco.com>
To: Paul Aitken 
X-Mailer: Apple Mail (2.1283)
Cc: Nevil Brownlee , IPFIX Working Group 
Subject: Re: [IPFIX] WG Last Call for draft-ietf-ipfix-information-model-rfc5102bis-05.txt
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: IPFIX WG discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Wed, 31 Oct 2012 09:43:04 -0000

Hi, Paul,

Many, many thanks for your review. We hadn't actually done a new =
revision (as we got no commentary during the LC), so let's call these =
WGLC comments and roll them into a -07 revision, which I'll submit once =
the Atlanta window is open again (or once this thread comes to a =
conclusion, whichever comes first).

Specific comments inline; omitted sections accepted without comment, as =
usual. Comments on Section 5 deferred until we have a consensus on =
whether we keep it (see other thread).

On Oct 30, 2012, at 11:28 PM, Paul Aitken wrote:

> Some IEs are missing from this document, although they are defined in =
IANA's IPFIX registry:
>=20
>    All the IE's from section 5.8. "Min/Max Flow Properties" of [5102] =
are missing:
>=20
>        6    tcpControlBits
>        25    minimumIpTotalLength
>        26    maximumIpTotalLength
>        52    minimumTTL
>        53    maximumTTL
>        64    ipv6ExtensionHeaders
>        208    ipv4Options
>        209    tcpOptions
>=20
>=20
>    This 5103 IE is missing:
>=20
>        239    biflowDirection
>=20
>=20
>    The following IEs defined by cisco are listed by IANA, but not in =
this text:
>=20
>        82    interfaceName
>        83    interfaceDescription
>        91    mplsTopLabelPrefixLength
>        98    postIpDiffServCodePoint
>        99    multicastReplicationFactor
>        105-127
>        225-236
>        240+
>=20
>=20
>    The following IEs are not mentioned here, although they are =
detailed in draft-yourtchenko-cisco-ies :
>=20
>        3, 34, 35, 38, 39, 43, 48-51, 65-69, 84, 87, 89, 92-93, 100, =
101, 102, 103, 104, 94-97.

The IEs in 5102bis' categories were chosen because they appeared in =
5102; we decided to keep the categorization but explicitly not to update =
the document to point to IEs added since 5102's publication. (Section =
5.8 appears to be an editing oversight; I'm not sure when it fell out of =
the document, but is not in any working copy of the doc I have; I'll add =
it back in).

However, if we replace Section 5 as I propose in the other thread, this =
discussion becomes moot.

> Please find specific feedback inline:
>=20
>>=20
>> 1.1. Changes since RFC 5102
>>=20
>>    This document obsoletes the Proposed Standard revision of the =
IPFIX
>>    Protocol Specification [RFC5102].  The following changes have been
>>    made to this document with respect to the previous document:
>>=20
>>       - All outstanding technical and editorial errata filed on the
>>    [RFC5102] as of publication time have been corrected
>>       - All references into [RFC5101] have been updated to =
[RFC5101bis],
>>    reflecting changes in that document as necessary
>>       - Information element definitions have been removed, as the
>>    reference for these is now [IPFIX-IANA]; categorizations of
>>    information elements as defines in [RFC5102] have been retained in
>>    section 5.
>>       - The process for modifying [IPFIX-IANA] has been improved, and =
is
>>    now described in [IPFIX-IE-DOCTORS]; Section 6 has been updated
>>    accordingly, and a new section 7.3 gives IANA considerations for =
this
>>    process.
>>       - Definitions of timestamp data types have been clarified
>>       - Appendices A and B have been removed
>=20
> BTW, the indentation of that section makes it difficult to read. Can =
you get all the text - including the wrapped lines - to the right of the =
bullets?

I'm not an nroff hacker. Any pointers?

It's also not clear that this section should survive IESG processing / =
RFC editor processing.

>> 2.3.  Naming Conventions for Information Elements
>>=20
>>    The following naming conventions were used for naming Information
>>    Elements in this document.  It is recommended that extensions of =
the
>>    model use the same conventions.
>>=20
>>    o  Names of Information Elements SHOULD be descriptive.
>>=20
>>    o  Names of Information Elements MUST be unique within the IANA
>>       registry.   Enterprise-specific Information Elements SHOULD be
>>       prefixed with a vendor name.
>> =08
>>    o  Names of Information Elements MUST start with non-capitalized
>>       letters.
>>=20
>>    o  Composed names MUST use capital letters for the first letter of
>>       each component (except for the first one).  All other letters =
are
>>       non-capitalized, even for acronyms.  Exceptions are made for
>>       acronyms containing non-capitalized letters, such as 'IPv4' and
>>       'IPv6'.  Examples are sourceMacAddress and =
destinationIPv4Address.
>=20
> Combination of the above rules means that IANA will name an IE "foo", =
while the ES equivalent is named "enterpriseFoo".
> It's unfortunate that "foo" !=3D "Foo".

We have the same issue with RFC5103. Fortunately most platforms have an =
equivalent of tolower()/toupper(), but yeah, this is a little bit of a =
pain.

>> 3.  Type Space
>>=20
>>    This section describes the abstract data types that can be used =
for
>>    the specification of IPFIX Information Elements in Section 4.
>>    Section 3.1 describes the set of abstract data types.
>>=20
>>    Abstract data types unsigned8, unsigned16, unsigned32, unsigned64,
>>    signed8, signed16, signed32, and signed64 are integral data types.
>=20
> They're just different sizes of the same types.

Yep. But we defined them this way, and there are strong hints to =
implementors contained therein as to how big the backing store for an IE =
must be even if it's reduced-length.

>>=20
>> 3.2.  Data Type Semantics
>>=20
>>    This section describes the set of valid data type semantics of the
>>    IPFIX information model. A registry of data type semantics is
>>    established in [RFC5610]; the restrictions on the use of semantics
>=20
> Surely IANA is the reference point, rather than 5610?
> eg, if new semantics are added, they'll be listed in IANA without =
raising errata against 5610.

This is the pointer to the establishment of the registry; the registry =
is a subregistry of the IANA IPFIX registry so there is no direct =
reference to it; will add an [IPFIX-IANA] xref.

>=20
>>    below are compatible with those specified in section 3.10 of that
>>    document. These semantics apply only to numeric types, as noted in
>>    the description of each semantic below.
>>=20
>>    Further data type semantics may be specified by future extensions =
of
>>    the IPFIX information model.
>=20
> State the required 5226 action / process for that, eg expert review.
> Or, xref the section where that's stated.

The intention here is if you want new semantics, you have to update =
5102bis, since these aren't just a matter of encoding, but of rather =
deep internal implementation questions on EPs and CPs alike. So: =
Standards Action. (Will also make a parallel clarification in Section =
3.1 above)

>> 3.2.3.  deltaCounter
>>=20
>>    An numeric value reporting the value of a counter. Counters are
>>    unsigned and wrap back to zero after reaching the limit of the =
type.
>>    For example, an unsigned64 with counter semantics will continue to
>>    increment until reaching the value of 2**64 - 1. At this point, =
the
>>    next increment will wrap its value to zero and continue counting =
from
>>    zero. The semantics of a delta counter is similar to the semantics =
of
>>    counters used in SNMP, such as Counter32 defined in RFC 2578
>>    [RFC2578]. The only difference between delta counters and counters
>>    used in SNMP is that the delta counters have an initial value of =
0. A
>>    delta counter is reset to 0 each time its value is exported.
>=20
> What if the cache entry is removed but not exported (eg, an export =
filter blocks the export) ?
> Then the counter was not exported, so it should not be reset to 0?
>=20
> ie, the reset action is more to do with the cache entry expiring than =
whatever happens to it next.

As I read it, this language was chosen in an attempt to be =
implementation-independent; nowhere in 5102 does the word "cache" =
appear, since that makes assumptions about how flow records are created =
in an MP. (The terminology and implicit design choices associated =
therewith crept into IPFIX in 6728, but I suppose there was nothing to =
be done there; you have to make some assumptions about MP internals if =
you're going to describe how to configure an MP.)

How about the more neutral:

A delta counter is reset to 0 each time it is exported and/or expires =
without export.

>> 3.2.4.  identifier
>>=20
>>    An integral value that serves as an identifier. Specifically,
>>    mathematical operations on two identifiers (aside from the =
equality
>>    operation) are meaningless. For example, Autonomous System ID 1 *
>>    Autonomous System ID 2 is meaningless. Identifiers MUST be one of =
the
>>    signed or unsigned data types.
>=20
> We could also have non-numeric identifiers, eg wlanSSID is a string =
identifier.

Do we need to mark string identifiers as identifiers? The definition of =
identifier is you can't do math on it, and if you're doing math on =
strings or representing things on which one could do math with strings, =
you're just doing it wrong.=20

>> 5.  Information Element Categories
>>=20
>>    This section describes the Information Element category for the =
IPFIX
>>    information model at the time that [RFC5102] was published. Since
>>    this category field is not part of the IANA process for assigning =
new
>>    Information Element (even though it has been reused, for example, =
in
>=20
> s/Element/Elements/
>=20
>=20
>>    [RFC5103]), the newest Information Elements in IANA [IPFIX-IANA]
>>    don't have this classification. The elements are grouped into 12
>>    groups according to their semantics and their applicability:
>=20
> TBD: are categories useful? If not, let's say they're deprecated and =
not discuss them further.

If they're deprecated, then we should remove Section 5 from this =
document in its entirety.

Actually, I don't see anything in your comments below on this section =
which would point to it being a useful thing to try and keep; see other =
thread I just started. I'll hold of on further changes to Section 5 (and =
keep your comments below intact)=20

>=20
>>=20
>>    1.   Identifiers
>>    2.   Metering and Exporting Process Configuration
>>    3.   Metering and Exporting Process Statistics
>>    4.   IP Header Fields
>>    5.   Transport Header Fields
>>    6.   Sub-IP Header Fields
>>    7.   Derived Packet Properties
>>    8.   Min/Max Flow Properties
>>    9.   Flow Timestamps
>>    10.  Per-Flow Counters
>>    11.  Miscellaneous Flow Properties
>>    12.  Padding
>>=20
>>    The Information Elements that are derived from fields of packets =
or
>=20
> s/fields of packets/packet fields/
>=20
>=20
>>    from packet treatment, such as the Information Elements in groups
>>    4-7, can typically serve as Flow Keys used for mapping packets to
>>    Flows.
>>=20
>> =20
>>=20
>> Claise, Trammell            Standards Track                    [Page =
13]
>>=20
>> Internet-Draft          IPFIX Information Model          October 3, =
2012
>>=20
>>=20
>>    If they do not serve as Flow Keys, their value may change from =
packet
>>    to packet within a single Flow.  For Information Elements with =
values
>>    that are derived from fields of packets or from packet treatment =
and
>=20
> s/fields of packets/packet fields/
>=20
>=20
>>    for which the value may change from packet to packet within a =
single
>>    Flow, the IPFIX information model defines that their value is
>>    determined by the first packet observed for the corresponding =
Flow,
>>    unless the description of the Information Element explicitly
>>    specifies a different semantics.  This simple rule allows writing =
all
>=20
> I don't think it's appropriate for the infomodel to define this.
>=20
> In some cases, the same IE may be observed in different ways according =
to the implementation.
> By the above definition, we'd need multiple IEs.
>=20
>=20
>>    Information Elements related to header fields once when the first
>>    packet of the Flow is observed.  For further observed packets of =
the
>>    same Flow, only Flow properties that depend on more than one =
packet,
>>    such as the Information Elements in groups 8-11, need to be =
updated.
>=20
> This model is based on an historic and simplistic understanding of the =
MP.
>=20
> Today we may not be able to determine all the key fields until some =
variable number of packets have been observed.
> eg, consider if a key field is in fragment N > 1.
>=20
>=20
>>=20
>>    Information Elements with a name having the "post" prefix, for
>>    example, "postIpClassOfService", do not report properties that =
were
>>    actually observed at the Observation Point, but retrieved by other
>>    means within the Observation Domain.  These Information Elements =
can
>>    be used if there are middlebox functions within the Observation
>>    Domain changing Flow properties after packets passed the =
Observation
>>    Point.
>=20
> s/changing/which change/
>=20
>=20
>>=20
>>=20
>> 5.1.  Identifiers
>>=20
>>    Information Elements grouped in the table below are identifying
>>    components of the IPFIX architecture, of an IPFIX Device, or of =
the
>>    IPFIX protocol.  All of them have an integral abstract data type =
and
>>    data type semantics "identifier" as described in Section 3.2.4.
>>=20
>>    Typically, some of them are used for limiting scopes of other
>>    Information Elements.  However, other Information Elements MAY be
>>    used for limiting scopes.  Note also that all Information Elements
>>    listed below MAY be used for other purposes than limiting scopes.
>>=20
>>    =
+-----+---------------------------+-----+---------------------------+
>>    |  ID | Name                      |  ID | Name                     =
 |
>>    =
+-----+---------------------------+-----+---------------------------+
>>    | 141 | lineCardId                | 148 | flowId                   =
 |
>>    | 142 | portId                    | 145 | templateId               =
 |
>>    |  10 | ingressInterface          | 149 | observationDomainId      =
 |
>>    |  14 | egressInterface           | 138 | observationPointId       =
 |
>>    | 143 | meteringProcessId         | 137 | commonPropertiesId       =
 |
>>    | 144 | exportingProcessId        |     |                          =
 |
>>    =
+-----+---------------------------+-----+---------------------------+
>>=20
>>    See [IPFIX-IANA] for the definitions of these Information =
Elements.
>=20
> Instead of repeating this over and over, just say it once in section =
5. ?
>=20
>=20
>>=20
>>=20
>> =20
>>=20
>> Claise, Trammell            Standards Track                    [Page =
14]
>>=20
>> Internet-Draft          IPFIX Information Model          October 3, =
2012
>>=20
>>=20
>>    5.2.  Metering and Exporting Process Configuration
>>=20
>>    Information Elements in this section describe the configuration of
>>    the Metering Process or the Exporting Process.  The set of these
>>    Information Elements is listed in the table below.
>>=20
>>    =
+-----+--------------------------+-----+----------------------------+
>>    |  ID | Name                     |  ID | Name                      =
 |
>>    =
+-----+--------------------------+-----+----------------------------+
>>    | 130 | exporterIPv4Address      | 213 | exportInterface           =
 |
>>    | 131 | exporterIPv6Address      | 214 | exportProtocolVersion     =
 |
>>    | 217 | exporterTransportPort    | 215 | exportTransportProtocol   =
 |
>>    | 211 | collectorIPv4Address     | 216 | collectorTransportPort    =
 |
>>    | 212 | collectorIPv6Address     | 173 | flowKeyIndicator          =
 |
>>    =
+-----+--------------------------+-----+----------------------------+
>>=20
>>    See [IPFIX-IANA] for the definitions of these Information =
Elements.
>>=20
>>=20
>>=20
>> 5.3.  Metering and Exporting Process Statistics
>>=20
>>    Information Elements in this section describe statistics of the
>>    Metering Process and/or the Exporting Process.  The set of these
>>    Information Elements is listed in the table below.
>>=20
>>    =
+-----+-----------------------------+-----+-------------------------+
>>    |  ID | Name                        |  ID | Name                   =
 |
>>    =
+-----+-----------------------------+-----+-------------------------+
>>    |  41 | exportedMessageTotalCount   | 165 | ignoredOctetTotalCount =
 |
>>    |  40 | exportedOctetTotalCount     | 166 | notSentFlowTotalCount  =
 |
>>    |  42 | exportedFlowRecordTotalCount| 167 | =
notSentPacketTotalCount |
>>    | 163 | observedFlowTotalCount      | 168 | notSentOctetTotalCount =
 |
>>    | 164 | ignoredPacketTotalCount     |     |                        =
 |
>>    =
+-----+-----------------------------+-----+-------------------------+
>>=20
>>    See [IPFIX-IANA] for the definitions of these Information =
Elements.
>>=20
>>=20
>>=20
>> 5.4.  IP Header Fields
>>=20
>>    Information Elements in this section indicate values of IP header
>>    fields or are derived from IP header field values in combination =
with
>>    further information.
>>=20
>>    =
+-----+----------------------------+-----+--------------------------+
>>    |  ID | Name                       |  ID | Name                    =
 |
>> =20
>>=20
>> Claise, Trammell            Standards Track                    [Page =
15]
>>=20
>> Internet-Draft          IPFIX Information Model          October 3, =
2012
>>=20
>>=20
>>    =
+-----+----------------------------+-----+--------------------------+
>>    |  60 | ipVersion                  | 193 | nextHeaderIPv6          =
 |
>>    |   8 | sourceIPv4Address          | 195 | ipDiffServCodePoint     =
 |
>>    |  27 | sourceIPv6Address          | 196 | ipPrecedence            =
 |
>>    |   9 | sourceIPv4PrefixLength     |   5 | ipClassOfService        =
 |
>>    |  29 | sourceIPv6PrefixLength     |  55 | postIpClassOfService    =
 |
>>    |  44 | sourceIPv4Prefix           |  31 | flowLabelIPv6           =
 |
>>    | 170 | sourceIPv6Prefix           | 206 | isMulticast             =
 |
>>    |  12 | destinationIPv4Address     |  54 | fragmentIdentification  =
 |
>>    |  28 | destinationIPv6Address     |  88 | fragmentOffset          =
 |
>>    |  13 | destinationIPv4PrefixLength| 197 | fragmentFlags           =
 |
>>    |  30 | destinationIPv6PrefixLength| 189 | ipHeaderLength          =
 |
>>    |  45 | destinationIPv4Prefix      | 207 | ipv4IHL                 =
 |
>>    | 169 | destinationIPv6Prefix      | 190 | totalLengthIPv4         =
 |
>>    | 192 | ipTTL                      | 224 | ipTotalLength           =
 |
>>    |   4 | protocolIdentifier         | 191 | payloadLengthIPv6       =
 |
>>    =
+-----+----------------------------+-----+--------------------------+
>>=20
>>    See [IPFIX-IANA] for the definitions of these Information =
Elements.
>>=20
>>=20
>>=20
>> 5.5.  Transport Header Fields
>>=20
>>    The set of Information Elements related to transport header fields
>>    and length includes the Information Elements listed in the table
>>    below.
>>=20
>>    =
+-----+---------------------------+-----+---------------------------+
>>    |  ID | Name                      |  ID | Name                     =
 |
>>    =
+-----+---------------------------+-----+---------------------------+
>>    |   7 | sourceTransportPort       | 238 | tcpWindowScale           =
 |
>>    |  11 | destinationTransportPort  | 187 | tcpUrgentPointer         =
 |
>>    | 180 | udpSourcePort             | 188 | tcpHeaderLength          =
 |
>>    | 181 | udpDestinationPort        |  32 | icmpTypeCodeIPv4         =
 |
>>    | 205 | udpMessageLength          | 176 | icmpTypeIPv4             =
 |
>>    | 182 | tcpSourcePort             | 177 | icmpCodeIPv4             =
 |
>>    | 183 | tcpDestinationPort        | 139 | icmpTypeCodeIPv6         =
 |
>>    | 184 | tcpSequenceNumber         | 178 | icmpTypeIPv6             =
 |
>>    | 185 | tcpAcknowledgementNumber  | 179 | icmpCodeIPv6             =
 |
>>    | 186 | tcpWindowSize             |  33 | igmpType                 =
 |
>>    =
+-----+---------------------------+-----+---------------------------+
>>=20
>>    See [IPFIX-IANA] for the definitions of these Information =
Elements.
>>=20
>>=20
>>=20
>>=20
>> =20
>>=20
>> Claise, Trammell            Standards Track                    [Page =
16]
>>=20
>> Internet-Draft          IPFIX Information Model          October 3, =
2012
>>=20
>>=20
>> 5.6.  Sub-IP Header Fields
>>=20
>>    The set of Information Elements related to Sub-IP header fields
>>    includes the Information Elements listed in the table below.
>>=20
>>    =
+-----+---------------------------+-----+---------------------------+
>>    |  ID | Name                      |  ID | Name                     =
 |
>>    =
+-----+---------------------------+-----+---------------------------+
>>    |  56 | sourceMacAddress          | 201 | mplsLabelStackLength     =
 |
>>    |  81 | postSourceMacAddress      | 194 | mplsPayloadLength        =
 |
>>    |  58 | vlanId                    |  70 | mplsTopLabelStackSection =
 |
>>    |  59 | postVlanId                |  71 | mplsLabelStackSection2   =
 |
>>    |  80 | destinationMacAddress     |  72 | mplsLabelStackSection3   =
 |
>>    |  57 | postDestinationMacAddress |  73 | mplsLabelStackSection4   =
 |
>>    | 146 | wlanChannelId             |  74 | mplsLabelStackSection5   =
 |
>>    | 147 | wlanSSID                  |  75 | mplsLabelStackSection6   =
 |
>>    | 200 | mplsTopLabelTTL           |  76 | mplsLabelStackSection7   =
 |
>>    | 203 | mplsTopLabelExp           |  77 | mplsLabelStackSection8   =
 |
>>    | 237 | postMplsTopLabelExp       |  78 | mplsLabelStackSection9   =
 |
>>    | 202 | mplsLabelStackDepth       |  79 | mplsLabelStackSection10  =
 |
>>    =
+-----+---------------------------+-----+---------------------------+
>>=20
>>    See [IPFIX-IANA] for the definitions of these Information =
Elements.
>>=20
>>=20
>> 5.7.  Derived Packet Properties
>>=20
>>    The set of Information Elements derived from packet properties =
(for
>>    example, values of header fields) includes the Information =
Elements
>>    listed in the table below.
>>=20
>>    =
+-----+---------------------------+-----+---------------------------+
>>    |  ID | Name                      |  ID | Name                     =
 |
>>    =
+-----+---------------------------+-----+---------------------------+
>>    | 204 | ipPayloadLength           |  18 | bgpNextHopIPv4Address    =
 |
>>    |  15 | ipNextHopIPv4Address      |  63 | bgpNextHopIPv6Address    =
 |
>>    |  62 | ipNextHopIPv6Address      |  46 | mplsTopLabelType         =
 |
>>    |  16 | bgpSourceAsNumber         |  47 | mplsTopLabelIPv4Address  =
 |
>>    |  17 | bgpDestinationAsNumber    | 140 | mplsTopLabelIPv6Address  =
 |
>>    | 128 | bgpNextAdjacentAsNumber   |  90 | =
mplsVpnRouteDistinguisher |
>>    | 129 | bgpPrevAdjacentAsNumber   |     |                          =
 |
>>    =
+-----+---------------------------+-----+---------------------------+
>>=20
>>    See [IPFIX-IANA] for the definitions of these Information =
Elements.
>>=20
>>=20
>>=20
>>=20
>> =20
>>=20
>> Claise, Trammell            Standards Track                    [Page =
17]
>>=20
>> Internet-Draft          IPFIX Information Model          October 3, =
2012
>>=20
>>=20
>> 5.9.  Flow Timestamps
>>=20
>>    Information Elements in this section are timestamps of events.
>>=20
>>    Timestamps flowStartSeconds, flowEndSeconds, =
flowStartMilliseconds,
>>    flowEndMilliseconds, flowStartMicroseconds, flowEndMicroseconds,
>>    flowStartNanoseconds, flowEndNanoseconds, and
>>    systemInitTimeMilliseconds are absolute and have a well-defined =
fixed
>>    time base, such as, for example, the number of seconds since 0000 =
UTC
>>    Jan 1st 1970.
>=20
> It's a bit dangerous to give this example, since it could be misread =
as being the actual definition.
> xref sections 3.1.15 - 3.1.18 where the time bases are stated.
>=20
>=20
>>=20
>>    Timestamps flowStartDeltaMicroseconds and flowEndDeltaMicroseconds
>>    are relative timestamps only valid within the scope of a single
>>    IPFIX Message.  They contain the negative time offsets relative to
>>    the export time specified in the IPFIX Message Header.  The =
maximum
>=20
> In order for the EP to populate *DeltaMicroseconds in a flow record, =
it must first know what Export Time it's going to stamp into the IPFIX =
header, and the flow record must be exported with that given second... =
unless we allow that data may be exported somewhat asynchronously to the =
header timestamping (eg, if there's a queue of outgoing packets at a =
level below the EP, eg in the IP stack). If a flow's export is delayed =
such that the Export Time changes, then these deltas must be =
recalculated. Practically, that may not be possible.
>=20
> Anyway, why do we have microsecond offsets from a "seconds" time?
>=20
> In short, these two IEs seem flawed and should be deprecated.
>=20
>=20
>>    time offset that can be encoded by these delta counters is 1 hour, =
11
>>    minutes, and 34.967295 seconds.
>>=20
>>    Timestamps flowStartSysUpTime and flowEndSysUpTime are relative
>>    timestamps indicating the time relative to the last
>>    (re-)initialization of the IPFIX Device.  For reporting the time
>>    of the last (re-)initialization, systemInitTimeMilliseconds can
>>    be reported, for example, in Data Records defined by Option
>>    Templates.
>>=20
>>    =
+-----+---------------------------+-----+---------------------------+
>>    |  ID | Name                      |  ID | Name                     =
 |
>>    =
+-----+---------------------------+-----+---------------------------+
>>    | 150 | flowStartSeconds          | 156 | flowStartNanoseconds     =
 |
>>    | 151 | flowEndSeconds            | 157 | flowEndNanoseconds       =
 |
>>    | 152 | flowStartMilliseconds     | 158 | =
flowStartDeltaMicroseconds|
>>    | 153 | flowEndMilliseconds       | 159 | flowEndDeltaMicroseconds =
 |
>>    | 154 | flowStartMicroseconds     | 160 | =
systemInitTimeMilliseconds|
>>    | 155 | flowEndMicroseconds       |  22 | flowStartSysUpTime       =
 |
>>    |     |                           |  21 | flowEndSysUpTime         =
 |
>>    =
+-----+---------------------------+-----+---------------------------+
>>=20
>>    See [IPFIX-IANA] for the definitions of these Information =
Elements.
>>=20
>> 5.10.  Per-Flow Counters
>>=20
>>    Information Elements in this section are counters all having =
integer
>>    values.  Their values may change for every report they are used =
in.
>>    They cannot serve as part of a Flow Key used for mapping packets =
to
>>    Flows.  However, potentially they can be used for selecting =
exported
>=20
> Well, octetDeltaCount could be used to make all packets of the same =
size hash to the same bucket.
>=20
> More realistically, these could be used when aggregating flows into =
other flows. eg, all the flows with the same number of packets, or all =
the flows with the same TCP SYN count.
>=20
>=20
>>    Flows, for example, by only exporting Flows with more than a
>>    threshold number of observed octets.
>>=20
>> =20
>>=20
>> Claise, Trammell            Standards Track                    [Page =
18]
>>=20
>> Internet-Draft          IPFIX Information Model          October 3, =
2012
>>=20
>>=20
>>    There are running counters and delta counters.  Delta counters are
>>    reset to zero each time their values are exported.  Running =
counters
>>    continue counting independently of the Exporting Process.
>>=20
>>    There are per-Flow counters and counters related to the Metering
>>    Process and/or the Exporting Process.  Per-Flow counters are Flow
>>    properties that potentially change each time a packet belonging to
>>    the Flow is observed.  The set of per-Flow counters includes the
>>    Information Elements listed in the table below.  Counters related =
to
>>    the Metering Process and/or the Exporting Process are described in
>>    Section 5.3.
>>=20
>>    =
+-----+---------------------------+-----+---------------------------+
>>    |  ID | Name                      |  ID | Name                     =
 |
>>    =
+-----+---------------------------+-----+---------------------------+
>>    |   1 | octetDeltaCount           | 134 | droppedOctetTotalCount   =
 |
>>    |  23 | postOctetDeltaCount       | 135 | droppedPacketTotalCount  =
 |
>>    | 198 | octetDeltaSumOfSquares    |  19 | =
postMCastPacketDeltaCount |
>>    |  85 | octetTotalCount           |  20 | postMCastOctetDeltaCount =
 |
>>    | 171 | postOctetTotalCount       | 174 | =
postMCastPacketTotalCount |
>>    | 199 | octetTotalSumOfSquares    | 175 | postMCastOctetTotalCount =
 |
>>    |   2 | packetDeltaCount          | 218 | tcpSynTotalCount         =
 |
>>    |  24 | postPacketDeltaCount      | 219 | tcpFinTotalCount         =
 |
>>    |  86 | packetTotalCount          | 220 | tcpRstTotalCount         =
 |
>>    | 172 | postPacketTotalCount      | 221 | tcpPshTotalCount         =
 |
>>    | 132 | droppedOctetDeltaCount    | 222 | tcpAckTotalCount         =
 |
>>    | 133 | droppedPacketDeltaCount   | 223 | tcpUrgTotalCount         =
 |
>>    =
+-----+---------------------------+-----+---------------------------+
>>=20
>>=20
>> See [IPFIX-IANA] for the definitions of these Information Elements.
>>=20
>>=20
>> 5.11.  Miscellaneous Flow Properties
>>=20
>>    Information Elements in this section describe properties of Flows
>>    that are related to Flow start, Flow duration, and Flow =
termination,
>>    but they are not timestamps as the Information Elements in Section
>>    5.9 are.
>>=20
>>    =
+-----+---------------------------+-----+---------------------------+
>>    |  ID | Name                      |  ID | Name                     =
 |
>>    =
+-----+---------------------------+-----+---------------------------+
>>    |  36 | flowActiveTimeout         | 161 | flowDurationMilliseconds =
 |
>>    |  37 | flowIdleTimeout           | 162 | flowDurationMicroseconds =
 |
>>    | 136 | flowEndReason             |  61 | flowDirection            =
 |
>>    =
+-----+---------------------------+-----+---------------------------+
>>=20
>> =20
>>=20
>> Claise, Trammell            Standards Track                    [Page =
19]
>>=20
>> Internet-Draft          IPFIX Information Model          October 3, =
2012
>>=20
>>=20
>> See [IPFIX-IANA] for the definitions of these Information Elements.
>>=20
>>=20
>>=20
>> 5.12.  Padding
>>=20
>>    This section contains a single Information Element that can be =
used
>>    for padding of Flow Records.
>>=20
>>    IPFIX implementations may wish to align Information Elements =
within
>>    Data Records or to align entire Data Records to 4-octet or 8-octet
>>    boundaries.  This can be achieved by including one or more
>>    paddingOctets Information Elements in a Data Record.
>>=20
>>    =
+-----+---------------------------+-----+---------------------------+
>>    |  ID | Name                      |  ID | Name                     =
 |
>>    =
+-----+---------------------------+-----+---------------------------+
>>    | 210 | paddingOctets             |     |                          =
 |
>>    =
+-----+---------------------------+-----+---------------------------+
>>=20
>> See [IPFIX-IANA] for the definitions of these Information Elements.
>>=20
>>=20
>> 7.4.  Addition, Revision, and Deprecation
>>=20
>> As stated in Section 6, addition, revision, and deletion of =
Information
>> Elements in the IPFIX Information Element registry is subject to a
>> process described in [IPFIX-IE-DOCTORS]. The IE-DOCTORS experts =
mentions
>> in this process are to be appointed by the IESG.
>=20
> When was/will that be done? Where are/will they be listed? How will =
IANA know who they are?

Feedback from IANA indicates that this process is IANA-managed and =
internal; i.e. this document should leave it up to them.

Thanks, cheers,

Brian=

From paitken@cisco.com  Wed Oct 31 06:26:57 2012
Return-Path: 
X-Original-To: ipfix@ietfa.amsl.com
Delivered-To: ipfix@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8781921F87A4 for ; Wed, 31 Oct 2012 06:26:57 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -10.878
X-Spam-Level: 
X-Spam-Status: No, score=-10.878 tagged_above=-999 required=5 tests=[AWL=1.121, BAYES_00=-2.599, GB_I_LETTER=-2, J_CHICKENPOX_37=0.6, RCVD_IN_DNSWL_HI=-8]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FZ+DD2H0a9Xv for ; Wed, 31 Oct 2012 06:26:56 -0700 (PDT)
Received: from ams-iport-1.cisco.com (ams-iport-1.cisco.com [144.254.224.140]) by ietfa.amsl.com (Postfix) with ESMTP id 0786321F87A7 for ; Wed, 31 Oct 2012 06:26:55 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=7728; q=dns/txt; s=iport; t=1351690017; x=1352899617; h=message-id:date:from:mime-version:to:cc:subject: references:in-reply-to:content-transfer-encoding; bh=W37yapdEIIUxAhBHarJOf01HhFcJobCJMiJi7xGM9+w=; b=e0G120Yf98dUu/2NWe+zgJrS9Y+f5HvXkM6yE1f97K3t4Tt1Q7diPw+2 D95QspfoeBceYXyXnt7xIM7W6ZIwjo35wi19ZJEunOG51yFZy+nxJZHQq 4X9y3vO+8p3CSfZBmrPXOOHHhworhCZHMZNoGZQLIlmAsSqgwmSkaQdoH w=;
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: AhUFAHkmkVCQ/khM/2dsb2JhbABEwBSDWYEIgh4BAQEDARIBChtAAQULCxgJFg8JAwIBAgFFBg0BBwEBHodeBpwKoBCLeBWGJgOVdoVpiG6Ba4JvgVw
X-IronPort-AV: E=Sophos;i="4.80,687,1344211200"; d="scan'208";a="146075185"
Received: from ams-core-3.cisco.com ([144.254.72.76]) by ams-iport-1.cisco.com with ESMTP; 31 Oct 2012 13:26:55 +0000
Received: from [144.254.153.40] (dhcp-144-254-153-40.cisco.com [144.254.153.40]) by ams-core-3.cisco.com (8.14.5/8.14.5) with ESMTP id q9VDQsol025905; Wed, 31 Oct 2012 13:26:54 GMT
Message-ID: <5091271E.3050206@cisco.com>
Date: Wed, 31 Oct 2012 13:26:54 +0000
From: Paul Aitken 
User-Agent: Mozilla/5.0 (X11; Linux i686; rv:15.0) Gecko/20120912 Thunderbird/15.0.1
MIME-Version: 1.0
To: Brian Trammell 
References: <506CBFE3.10607@auckland.ac.nz> <5090547C.5020803@cisco.com> 
In-Reply-To: 
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Cc: Nevil Brownlee , IPFIX Working Group 
Subject: Re: [IPFIX] WG Last Call for draft-ietf-ipfix-information-model-rfc5102bis-05.txt
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: IPFIX WG discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Wed, 31 Oct 2012 13:26:57 -0000

Brian,

A few replies inline; other stuff cut for brevity.


> Hi, Paul,
>
> Many, many thanks for your review. We hadn't actually done a new revision (as we got no commentary during the LC), so let's call these WGLC comments and roll them into a -07 revision, which I'll submit once the Atlanta window is open again (or once this thread comes to a conclusion, whichever comes first).

To be clear, Nevil started WGLC on -05 (see the subject line above) just 
hours after -06 was announced. I reviewed -06.


> Specific comments inline; omitted sections accepted without comment, as usual. Comments on Section 5 deferred until we have a consensus on whether we keep it (see other thread).
>
> On Oct 30, 2012, at 11:28 PM, Paul Aitken wrote:
>
>> Some IEs are missing from this document, although they are defined in IANA's IPFIX registry:
>>
>>     All the IE's from section 5.8. "Min/Max Flow Properties" of [5102] are missing:
>>
>>         6    tcpControlBits
>>         25    minimumIpTotalLength
>>         26    maximumIpTotalLength
>>         52    minimumTTL
>>         53    maximumTTL
>>         64    ipv6ExtensionHeaders
>>         208    ipv4Options
>>         209    tcpOptions
>>
>>
>>     This 5103 IE is missing:
>>
>>         239    biflowDirection
>>
>>
>>     The following IEs defined by cisco are listed by IANA, but not in this text:
>>
>>         82    interfaceName
>>         83    interfaceDescription
>>         91    mplsTopLabelPrefixLength
>>         98    postIpDiffServCodePoint
>>         99    multicastReplicationFactor
>>         105-127
>>         225-236
>>         240+
>>
>>
>>     The following IEs are not mentioned here, although they are detailed in draft-yourtchenko-cisco-ies :
>>
>>         3, 34, 35, 38, 39, 43, 48-51, 65-69, 84, 87, 89, 92-93, 100, 101, 102, 103, 104, 94-97.
> The IEs in 5102bis' categories were chosen because they appeared in 5102; we decided to keep the categorization but explicitly not to update the document to point to IEs added since 5102's publication. (Section 5.8 appears to be an editing oversight; I'm not sure when it fell out of the document, but is not in any working copy of the doc I have; I'll add it back in).
>
> However, if we replace Section 5 as I propose in the other thread, this discussion becomes moot.

~somewhat. However proto-IE-doctors should check that these IE's all 
conform to the 5102bis specifications.


>> Please find specific feedback inline:
>>
>>> 1.1. Changes since RFC 5102
>>>
>>>     This document obsoletes the Proposed Standard revision of the IPFIX
>>>     Protocol Specification [RFC5102].  The following changes have been
>>>     made to this document with respect to the previous document:
>>>
>>>        - All outstanding technical and editorial errata filed on the
>>>     [RFC5102] as of publication time have been corrected
>>>        - All references into [RFC5101] have been updated to [RFC5101bis],
>>>     reflecting changes in that document as necessary
>>>        - Information element definitions have been removed, as the
>>>     reference for these is now [IPFIX-IANA]; categorizations of
>>>     information elements as defines in [RFC5102] have been retained in
>>>     section 5.
>>>        - The process for modifying [IPFIX-IANA] has been improved, and is
>>>     now described in [IPFIX-IE-DOCTORS]; Section 6 has been updated
>>>     accordingly, and a new section 7.3 gives IANA considerations for this
>>>     process.
>>>        - Definitions of timestamp data types have been clarified
>>>        - Appendices A and B have been removed
>> BTW, the indentation of that section makes it difficult to read. Can you get all the text - including the wrapped lines - to the right of the bullets?
> I'm not an nroff hacker. Any pointers?

Personally, I had to hack rfc2xml to make it do what I wanted.


> It's also not clear that this section should survive IESG processing / RFC editor processing.

Since this doc obsoletes 5102, I find it useful to get a high level 
overview of what was changed, and for that overview to be given early in 
the document.

Else, add an editorial note to remove the section.


>>> 2.3.  Naming Conventions for Information Elements
>>>
>>>     The following naming conventions were used for naming Information
>>>     Elements in this document.  It is recommended that extensions of the
>>>     model use the same conventions.
>>>
>>>     o  Names of Information Elements SHOULD be descriptive.
>>>
>>>     o  Names of Information Elements MUST be unique within the IANA
>>>        registry.   Enterprise-specific Information Elements SHOULD be
>>>        prefixed with a vendor name.
>>> 
>>>     o  Names of Information Elements MUST start with non-capitalized
>>>        letters.
>>>
>>>     o  Composed names MUST use capital letters for the first letter of
>>>        each component (except for the first one).  All other letters are
>>>        non-capitalized, even for acronyms.  Exceptions are made for
>>>        acronyms containing non-capitalized letters, such as 'IPv4' and
>>>        'IPv6'.  Examples are sourceMacAddress and destinationIPv4Address.
>> Combination of the above rules means that IANA will name an IE "foo", while the ES equivalent is named "enterpriseFoo".
>> It's unfortunate that "foo" != "Foo".
> We have the same issue with RFC5103. Fortunately most platforms have an equivalent of tolower()/toupper(), but yeah, this is a little bit of a pain.

So does the "names must be unique" rule include or exclude case?

eg, are "fondantHem" and "fondAnthem" the same name? Are "coveredRaft" 
and "coveRedraft" the same?


>>> 3.2.3.  deltaCounter
>>>
>>>     An numeric value reporting the value of a counter. Counters are
>>>     unsigned and wrap back to zero after reaching the limit of the type.
>>>     For example, an unsigned64 with counter semantics will continue to
>>>     increment until reaching the value of 2**64 - 1. At this point, the
>>>     next increment will wrap its value to zero and continue counting from
>>>     zero. The semantics of a delta counter is similar to the semantics of
>>>     counters used in SNMP, such as Counter32 defined in RFC 2578
>>>     [RFC2578]. The only difference between delta counters and counters
>>>     used in SNMP is that the delta counters have an initial value of 0. A
>>>     delta counter is reset to 0 each time its value is exported.
>> What if the cache entry is removed but not exported (eg, an export filter blocks the export) ?
>> Then the counter was not exported, so it should not be reset to 0?
>>
>> ie, the reset action is more to do with the cache entry expiring than whatever happens to it next.
> As I read it, this language was chosen in an attempt to be implementation-independent; nowhere in 5102 does the word "cache" appear, since that makes assumptions about how flow records are created in an MP. (The terminology and implicit design choices associated therewith crept into IPFIX in 6728, but I suppose there was nothing to be done there; you have to make some assumptions about MP internals if you're going to describe how to configure an MP.)
>
> How about the more neutral:
>
> A delta counter is reset to 0 each time it is exported and/or expires without export.

By their nature, delta counters don't exist after they expire or are 
exported - so they're not really reset.

In fact the total and delta counter mechanisms are identical; only their 
observation periods set them apart. ie the difference between them is 
due to time. So I think the definition should capture that.

P.

From trammell@tik.ee.ethz.ch  Wed Oct 31 07:33:24 2012
Return-Path: 
X-Original-To: ipfix@ietfa.amsl.com
Delivered-To: ipfix@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 03C6821F871D for ; Wed, 31 Oct 2012 07:33:24 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -7.299
X-Spam-Level: 
X-Spam-Status: No, score=-7.299 tagged_above=-999 required=5 tests=[AWL=0.700,  BAYES_00=-2.599, GB_I_LETTER=-2, J_CHICKENPOX_37=0.6, RCVD_IN_DNSWL_MED=-4]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Oo2Jhwa0Jjro for ; Wed, 31 Oct 2012 07:33:22 -0700 (PDT)
Received: from smtp.ee.ethz.ch (smtp.ee.ethz.ch [129.132.2.219]) by ietfa.amsl.com (Postfix) with ESMTP id 76E0621F871F for ; Wed, 31 Oct 2012 07:33:22 -0700 (PDT)
Received: from localhost (localhost [127.0.0.1]) by smtp.ee.ethz.ch (Postfix) with ESMTP id 62706D930B; Wed, 31 Oct 2012 15:33:21 +0100 (MET)
X-Virus-Scanned: by amavisd-new on smtp.ee.ethz.ch
Received: from smtp.ee.ethz.ch ([127.0.0.1]) by localhost (.ee.ethz.ch [127.0.0.1]) (amavisd-new, port 10024) with LMTP id Cx4UkmndOwok; Wed, 31 Oct 2012 15:33:21 +0100 (MET)
Received: from pb-10243.ethz.ch (pb-10243.ethz.ch [82.130.102.152]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: briant) by smtp.ee.ethz.ch (Postfix) with ESMTPSA id 0D0CDD9309; Wed, 31 Oct 2012 15:33:21 +0100 (MET)
Mime-Version: 1.0 (Apple Message framework v1283)
Content-Type: text/plain; charset=iso-8859-1
From: Brian Trammell 
In-Reply-To: <5091271E.3050206@cisco.com>
Date: Wed, 31 Oct 2012 15:33:20 +0100
Content-Transfer-Encoding: quoted-printable
Message-Id: <68907E1B-1D38-4F3A-B0E0-F47628F989F0@tik.ee.ethz.ch>
References: <506CBFE3.10607@auckland.ac.nz> <5090547C.5020803@cisco.com>  <5091271E.3050206@cisco.com>
To: Paul Aitken 
X-Mailer: Apple Mail (2.1283)
Cc: Nevil Brownlee , IPFIX Working Group 
Subject: Re: [IPFIX] WG Last Call for draft-ietf-ipfix-information-model-rfc5102bis-05.txt
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: IPFIX WG discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Wed, 31 Oct 2012 14:33:24 -0000

hi Paul,

(inline)^n.

On 31 Oct 2012, at 14:26 , Paul Aitken wrote:

> Brian,
>=20
> A few replies inline; other stuff cut for brevity.
>=20
>=20
>> Hi, Paul,
>>=20
>> Many, many thanks for your review. We hadn't actually done a new =
revision (as we got no commentary during the LC), so let's call these =
WGLC comments and roll them into a -07 revision, which I'll submit once =
the Atlanta window is open again (or once this thread comes to a =
conclusion, whichever comes first).
>=20
> To be clear, Nevil started WGLC on -05 (see the subject line above) =
just hours after -06 was announced. I reviewed -06.

Ah, oops. I presumed that was an oversight; I thought -06 got WGLC'd.

>> Specific comments inline; omitted sections accepted without comment, =
as usual. Comments on Section 5 deferred until we have a consensus on =
whether we keep it (see other thread).
>>=20
>> On Oct 30, 2012, at 11:28 PM, Paul Aitken wrote:
>>=20
>>> Some IEs are missing from this document, although they are defined =
in IANA's IPFIX registry:
>>>=20
>>>    All the IE's from section 5.8. "Min/Max Flow Properties" of =
[5102] are missing:
>>>=20
>>>        6    tcpControlBits
>>>        25    minimumIpTotalLength
>>>        26    maximumIpTotalLength
>>>        52    minimumTTL
>>>        53    maximumTTL
>>>        64    ipv6ExtensionHeaders
>>>        208    ipv4Options
>>>        209    tcpOptions
>>>=20
>>>=20
>>>    This 5103 IE is missing:
>>>=20
>>>        239    biflowDirection
>>>=20
>>>=20
>>>    The following IEs defined by cisco are listed by IANA, but not in =
this text:
>>>=20
>>>        82    interfaceName
>>>        83    interfaceDescription
>>>        91    mplsTopLabelPrefixLength
>>>        98    postIpDiffServCodePoint
>>>        99    multicastReplicationFactor
>>>        105-127
>>>        225-236
>>>        240+
>>>=20
>>>=20
>>>    The following IEs are not mentioned here, although they are =
detailed in draft-yourtchenko-cisco-ies :
>>>=20
>>>        3, 34, 35, 38, 39, 43, 48-51, 65-69, 84, 87, 89, 92-93, 100, =
101, 102, 103, 104, 94-97.
>> The IEs in 5102bis' categories were chosen because they appeared in =
5102; we decided to keep the categorization but explicitly not to update =
the document to point to IEs added since 5102's publication. (Section =
5.8 appears to be an editing oversight; I'm not sure when it fell out of =
the document, but is not in any working copy of the doc I have; I'll add =
it back in).
>>=20
>> However, if we replace Section 5 as I propose in the other thread, =
this discussion becomes moot.
>=20
> ~somewhat. However proto-IE-doctors should check that these IE's all =
conform to the 5102bis specifications.

Point; will make a note of this list somewhere.

>=20
>>> Please find specific feedback inline:
>>>=20
>>>> 1.1. Changes since RFC 5102
>>>>=20
>>>>    This document obsoletes the Proposed Standard revision of the =
IPFIX
>>>>    Protocol Specification [RFC5102].  The following changes have =
been
>>>>    made to this document with respect to the previous document:
>>>>=20
>>>>       - All outstanding technical and editorial errata filed on the
>>>>    [RFC5102] as of publication time have been corrected
>>>>       - All references into [RFC5101] have been updated to =
[RFC5101bis],
>>>>    reflecting changes in that document as necessary
>>>>       - Information element definitions have been removed, as the
>>>>    reference for these is now [IPFIX-IANA]; categorizations of
>>>>    information elements as defines in [RFC5102] have been retained =
in
>>>>    section 5.
>>>>       - The process for modifying [IPFIX-IANA] has been improved, =
and is
>>>>    now described in [IPFIX-IE-DOCTORS]; Section 6 has been updated
>>>>    accordingly, and a new section 7.3 gives IANA considerations for =
this
>>>>    process.
>>>>       - Definitions of timestamp data types have been clarified
>>>>       - Appendices A and B have been removed
>>> BTW, the indentation of that section makes it difficult to read. Can =
you get all the text - including the wrapped lines - to the right of the =
bullets?
>> I'm not an nroff hacker. Any pointers?
>=20
> Personally, I had to hack rfc2xml to make it do what I wanted.

Bah; okay, I'll look somewhere for some nroff that does things correctly =
and copy from that then.

>> It's also not clear that this section should survive IESG processing =
/ RFC editor processing.
>=20
> Since this doc obsoletes 5102, I find it useful to get a high level =
overview of what was changed, and for that overview to be given early in =
the document.

Okay, that's one vote, so we'll keep it.

>>>> 2.3.  Naming Conventions for Information Elements
>>>>=20
>>>>    The following naming conventions were used for naming =
Information
>>>>    Elements in this document.  It is recommended that extensions of =
the
>>>>    model use the same conventions.
>>>>=20
>>>>    o  Names of Information Elements SHOULD be descriptive.
>>>>=20
>>>>    o  Names of Information Elements MUST be unique within the IANA
>>>>       registry.   Enterprise-specific Information Elements SHOULD =
be
>>>>       prefixed with a vendor name.
>>>> =08
>>>>    o  Names of Information Elements MUST start with non-capitalized
>>>>       letters.
>>>>=20
>>>>    o  Composed names MUST use capital letters for the first letter =
of
>>>>       each component (except for the first one).  All other letters =
are
>>>>       non-capitalized, even for acronyms.  Exceptions are made for
>>>>       acronyms containing non-capitalized letters, such as 'IPv4' =
and
>>>>       'IPv6'.  Examples are sourceMacAddress and =
destinationIPv4Address.
>>> Combination of the above rules means that IANA will name an IE =
"foo", while the ES equivalent is named "enterpriseFoo".
>>> It's unfortunate that "foo" !=3D "Foo".
>> We have the same issue with RFC5103. Fortunately most platforms have =
an equivalent of tolower()/toupper(), but yeah, this is a little bit of =
a pain.
>=20
> So does the "names must be unique" rule include or exclude case?
>=20
> eg, are "fondantHem" and "fondAnthem" the same name? Are "coveredRaft" =
and "coveRedraft" the same?

Ugh. Point. My inclination is we should define naming match as =
case-insensitive, since the intention is not to have these names used on =
the wire, and the only examples I can think of of collision are =
typographical in nature.

>>>> 3.2.3.  deltaCounter
>>>>=20
>>>>    An numeric value reporting the value of a counter. Counters are
>>>>    unsigned and wrap back to zero after reaching the limit of the =
type.
>>>>    For example, an unsigned64 with counter semantics will continue =
to
>>>>    increment until reaching the value of 2**64 - 1. At this point, =
the
>>>>    next increment will wrap its value to zero and continue counting =
from
>>>>    zero. The semantics of a delta counter is similar to the =
semantics of
>>>>    counters used in SNMP, such as Counter32 defined in RFC 2578
>>>>    [RFC2578]. The only difference between delta counters and =
counters
>>>>    used in SNMP is that the delta counters have an initial value of =
0. A
>>>>    delta counter is reset to 0 each time its value is exported.
>>> What if the cache entry is removed but not exported (eg, an export =
filter blocks the export) ?
>>> Then the counter was not exported, so it should not be reset to 0?
>>>=20
>>> ie, the reset action is more to do with the cache entry expiring =
than whatever happens to it next.
>> As I read it, this language was chosen in an attempt to be =
implementation-independent; nowhere in 5102 does the word "cache" =
appear, since that makes assumptions about how flow records are created =
in an MP. (The terminology and implicit design choices associated =
therewith crept into IPFIX in 6728, but I suppose there was nothing to =
be done there; you have to make some assumptions about MP internals if =
you're going to describe how to configure an MP.)
>>=20
>> How about the more neutral:
>>=20
>> A delta counter is reset to 0 each time it is exported and/or expires =
without export.
>=20
> By their nature, delta counters don't exist after they expire or are =
exported - so they're not really reset.

Implementation-dependent. You cannot at the protocol level force the MP =
implementation to forget things it doesn't want to.

> In fact the total and delta counter mechanisms are identical; only =
their observation periods set them apart. ie the difference between them =
is due to time. So I think the definition should capture that.

again, _in the exemplar design you use to reason about the protocol_ =
yes, but the exact implementation of these counters is (rightly) left =
out of the protocol.

Which, indeed, the more neutral language I proposed above fails to do =
(it captures the concepts of "reset" and "expiration"). How about:

A delta counter counts only observations made since the last export of a =
Flow Record for a given Flow.

Cheers,

Brian


From paitken@cisco.com  Wed Oct 31 07:55:51 2012
Return-Path: 
X-Original-To: ipfix@ietfa.amsl.com
Delivered-To: ipfix@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 736E221F8818 for ; Wed, 31 Oct 2012 07:55:51 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -10.402
X-Spam-Level: 
X-Spam-Status: No, score=-10.402 tagged_above=-999 required=5 tests=[AWL=0.197, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id DiP-0wLRk4oa for ; Wed, 31 Oct 2012 07:55:51 -0700 (PDT)
Received: from ams-iport-1.cisco.com (ams-iport-1.cisco.com [144.254.224.140]) by ietfa.amsl.com (Postfix) with ESMTP id BA1F021F8817 for ; Wed, 31 Oct 2012 07:55:50 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=744; q=dns/txt; s=iport; t=1351695350; x=1352904950; h=message-id:date:from:mime-version:to:cc:subject: references:in-reply-to:content-transfer-encoding; bh=gnIjV/ETGYrTm0G1jYaoRkYyw1K6/japvYJC+/YG/9s=; b=amjQ4P4D1T5Sph2zDIk8DhkROHizkNa8fpOY39ytJMBAjYiSdCWB9mhl //wyan4m0ZU06WeoObC7bQqNRpm2B1xRWXUNbdBhqS+XnqzQ9zB7dV2mO ikH4eB4Xjmxivp4UToFr0aL/tH7DRoILoE0XewbxRsUhzstrzm/kmJbwM w=;
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: AlIJALw6kVCQ/khN/2dsb2JhbABEwmIEBIECgQiCHgEBAQMBEgEKG0ABBQsLIRYPCQMCAQIBRQYNAQcBAR6HXgacIKAXkjMDlXaFaYhugWuCbw
X-IronPort-AV: E=Sophos;i="4.80,687,1344211200"; d="scan'208";a="146079295"
Received: from ams-core-4.cisco.com ([144.254.72.77]) by ams-iport-1.cisco.com with ESMTP; 31 Oct 2012 14:55:48 +0000
Received: from [144.254.153.40] (dhcp-144-254-153-40.cisco.com [144.254.153.40]) by ams-core-4.cisco.com (8.14.5/8.14.5) with ESMTP id q9VEtl4v009415; Wed, 31 Oct 2012 14:55:47 GMT
Message-ID: <50913BF3.2080408@cisco.com>
Date: Wed, 31 Oct 2012 14:55:47 +0000
From: Paul Aitken 
User-Agent: Mozilla/5.0 (X11; Linux i686; rv:15.0) Gecko/20120912 Thunderbird/15.0.1
MIME-Version: 1.0
To: Brian Trammell 
References: <506CBFE3.10607@auckland.ac.nz> <5090547C.5020803@cisco.com>  <5091271E.3050206@cisco.com> <68907E1B-1D38-4F3A-B0E0-F47628F989F0@tik.ee.ethz.ch>
In-Reply-To: <68907E1B-1D38-4F3A-B0E0-F47628F989F0@tik.ee.ethz.ch>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Cc: Nevil Brownlee , IPFIX Working Group 
Subject: Re: [IPFIX] WG Last Call for draft-ietf-ipfix-information-model-rfc5102bis-05.txt
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: IPFIX WG discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Wed, 31 Oct 2012 14:55:51 -0000

Brian,

> My inclination is we should define naming match as case-insensitive, since the intention is not to have these names used on the wire, and the only examples I can think of of collision are typographical in nature.

+1.


> How about:
>
> A delta counter counts only observations made since the last export of a Flow Record for a given Flow.

That's back to my point that the observation might not be exported.

More to the point, delta/total counting is a function of the MP. Whether 
the MP hands data to the EP, and what the EP does or does not do with 
that data, should have no bearing on how the MP meters subsequent data. 
So the definition should be purely in MP terms ("last export" being an 
EP term).

P.



From trammell@tik.ee.ethz.ch  Wed Oct 31 07:58:46 2012
Return-Path: 
X-Original-To: ipfix@ietfa.amsl.com
Delivered-To: ipfix@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D7F8C21F859A for ; Wed, 31 Oct 2012 07:58:46 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -6.949
X-Spam-Level: 
X-Spam-Status: No, score=-6.949 tagged_above=-999 required=5 tests=[AWL=-0.350, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Gsg+q-y6qshc for ; Wed, 31 Oct 2012 07:58:46 -0700 (PDT)
Received: from smtp.ee.ethz.ch (smtp.ee.ethz.ch [129.132.2.219]) by ietfa.amsl.com (Postfix) with ESMTP id 4BE5321F852B for ; Wed, 31 Oct 2012 07:58:46 -0700 (PDT)
Received: from localhost (localhost [127.0.0.1]) by smtp.ee.ethz.ch (Postfix) with ESMTP id 9A413D930B; Wed, 31 Oct 2012 15:58:45 +0100 (MET)
X-Virus-Scanned: by amavisd-new on smtp.ee.ethz.ch
Received: from smtp.ee.ethz.ch ([127.0.0.1]) by localhost (.ee.ethz.ch [127.0.0.1]) (amavisd-new, port 10024) with LMTP id G71tRLvuDjrH; Wed, 31 Oct 2012 15:58:45 +0100 (MET)
Received: from pb-10243.ethz.ch (pb-10243.ethz.ch [82.130.102.152]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: briant) by smtp.ee.ethz.ch (Postfix) with ESMTPSA id 41A20D9309; Wed, 31 Oct 2012 15:58:45 +0100 (MET)
Mime-Version: 1.0 (Apple Message framework v1283)
Content-Type: text/plain; charset=iso-8859-1
From: Brian Trammell 
In-Reply-To: <50913BF3.2080408@cisco.com>
Date: Wed, 31 Oct 2012 15:58:44 +0100
Content-Transfer-Encoding: quoted-printable
Message-Id: <8ED0D683-536C-46B6-8E5A-3CC3B7CB678F@tik.ee.ethz.ch>
References: <506CBFE3.10607@auckland.ac.nz> <5090547C.5020803@cisco.com>  <5091271E.3050206@cisco.com> <68907E1B-1D38-4F3A-B0E0-F47628F989F0@tik.ee.ethz.ch> <50913BF3.2080408@cisco.com>
To: Paul Aitken 
X-Mailer: Apple Mail (2.1283)
Cc: Nevil Brownlee , IPFIX Working Group 
Subject: Re: [IPFIX] WG Last Call for draft-ietf-ipfix-information-model-rfc5102bis-05.txt
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: IPFIX WG discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Wed, 31 Oct 2012 14:58:47 -0000

Hi, Paul,

On 31 Oct 2012, at 15:55 , Paul Aitken wrote:

> How about:
>>=20
>>=20
>> A delta counter counts only observations made since the last export =
of a Flow Record for a given Flow.
>=20
> That's back to my point that the observation might not be exported.
>=20
> More to the point, delta/total counting is a function of the MP. =
Whether the MP hands data to the EP, and what the EP does or does not do =
with that data, should have no bearing on how the MP meters subsequent =
data. So the definition should be purely in MP terms ("last export" =
being an EP term).

What verb does the MP apply to a unit of information when it gives it to =
the EP?

A delta counter counts only observations made since the last Flow Record =
for a given Flow was measured.

Or maybe we can sidestep the action completely:

A delta counter counts only observations made since the previous Flow =
Record for a given Flow.

Cheers,

Brian=

From paitken@cisco.com  Wed Oct 31 08:22:21 2012
Return-Path: 
X-Original-To: ipfix@ietfa.amsl.com
Delivered-To: ipfix@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7726B21F8872 for ; Wed, 31 Oct 2012 08:22:21 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -10.435
X-Spam-Level: 
X-Spam-Status: No, score=-10.435 tagged_above=-999 required=5 tests=[AWL=0.164, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id LpxIvvNlOImn for ; Wed, 31 Oct 2012 08:22:21 -0700 (PDT)
Received: from ams-iport-4.cisco.com (ams-iport-4.cisco.com [144.254.224.147]) by ietfa.amsl.com (Postfix) with ESMTP id A1C8821F874E for ; Wed, 31 Oct 2012 08:22:20 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=1005; q=dns/txt; s=iport; t=1351696940; x=1352906540; h=message-id:date:from:mime-version:to:cc:subject: references:in-reply-to:content-transfer-encoding; bh=vm7gcFVF5/PR2vrQbDF0ShTHJjKYoNhaoExtP6fvNLQ=; b=f+vCKohPwAZsIR18fGBkASFqZ2g2p+ZAo5EsEcoYB0bkj68d3WxALoPc ToV1E3333DapOzMqBkyIi0Cspql0pYLXXS2nDODVB8tV+pCI/PgyB73XO DK7ltw/Ryk0E1Lr45Yad8GivK8ktGhxT30w+zd4V9uBRKipBEL6nkV34m s=;
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: Av8EAKxBkVCQ/khR/2dsb2JhbABEw2+BCIIeAQEBAwESAQobQAEFCwshFg8JAwIBAgFFBg0BBwEBHodeBpwzoBqSMwOVdoVpiG6Ba4Jv
X-IronPort-AV: E=Sophos;i="4.80,687,1344211200";  d="scan'208";a="9243370"
Received: from ams-core-1.cisco.com ([144.254.72.81]) by ams-iport-4.cisco.com with ESMTP; 31 Oct 2012 15:22:19 +0000
Received: from [144.254.153.40] (dhcp-144-254-153-40.cisco.com [144.254.153.40]) by ams-core-1.cisco.com (8.14.5/8.14.5) with ESMTP id q9VFMJ8h001595; Wed, 31 Oct 2012 15:22:19 GMT
Message-ID: <5091422B.9070607@cisco.com>
Date: Wed, 31 Oct 2012 15:22:19 +0000
From: Paul Aitken 
User-Agent: Mozilla/5.0 (X11; Linux i686; rv:15.0) Gecko/20120912 Thunderbird/15.0.1
MIME-Version: 1.0
To: Brian Trammell 
References: <506CBFE3.10607@auckland.ac.nz> <5090547C.5020803@cisco.com>  <5091271E.3050206@cisco.com> <68907E1B-1D38-4F3A-B0E0-F47628F989F0@tik.ee.ethz.ch> <50913BF3.2080408@cisco.com> <8ED0D683-536C-46B6-8E5A-3CC3B7CB678F@tik.ee.ethz.ch>
In-Reply-To: <8ED0D683-536C-46B6-8E5A-3CC3B7CB678F@tik.ee.ethz.ch>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Cc: Nevil Brownlee , IPFIX Working Group 
Subject: Re: [IPFIX] WG Last Call for draft-ietf-ipfix-information-model-rfc5102bis-05.txt
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: IPFIX WG discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Wed, 31 Oct 2012 15:22:21 -0000

Brian,

> What verb does the MP apply to a unit of information when it gives it to the EP?
>
> A delta counter counts only observations made since the last Flow Record for a given Flow was measured.
>
> Or maybe we can sidestep the action completely:
>
> A delta counter counts only observations made since the previous Flow Record for a given Flow.

No, that's still involving export.

What should the MP do if the flow ends *but isn't exported* ? Just the 
same as when the flow *is* exported. So the definition of deltaCount is 
independent of export, flow records, etc.

So the definitions have to be about the metering time, and particularly 
that we've started metering again. However, we can't write that, because 
some implementations may not hold state that tells them this. So all we 
know is that totalCounters meter from the start of the MP, while delta 
counters meter a potentially shorter interval, reporting the value 
metered since the start of that interval.

P.

From johnwcrt@au1.ibm.com  Wed Oct 31 14:38:16 2012
Return-Path: 
X-Original-To: ipfix@ietfa.amsl.com
Delivered-To: ipfix@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3A8BE21F85A8 for ; Wed, 31 Oct 2012 14:38:16 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -9.998
X-Spam-Level: 
X-Spam-Status: No, score=-9.998 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, J_CHICKENPOX_54=0.6, RCVD_IN_DNSWL_HI=-8]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id wk3AqoeFzbSW for ; Wed, 31 Oct 2012 14:38:15 -0700 (PDT)
Received: from e23smtp06.au.ibm.com (e23smtp06.au.ibm.com [202.81.31.148]) by ietfa.amsl.com (Postfix) with ESMTP id E948121F8566 for ; Wed, 31 Oct 2012 14:38:13 -0700 (PDT)
Received: from /spool/local by e23smtp06.au.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for  from ; Thu, 1 Nov 2012 07:35:51 +1000
Received: from d23relay03.au.ibm.com (202.81.31.245) by e23smtp06.au.ibm.com (202.81.31.212) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted;  Thu, 1 Nov 2012 07:35:50 +1000
Received: from d23av03.au.ibm.com (d23av03.au.ibm.com [9.190.234.97]) by d23relay03.au.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id q9VLc2Ox16449594 for ; Thu, 1 Nov 2012 08:38:03 +1100
Received: from d23av03.au.ibm.com (loopback [127.0.0.1]) by d23av03.au.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id q9VLc1Md027509 for ; Thu, 1 Nov 2012 08:38:02 +1100
Received: from d23mlc03.au.ibm.com (d23mlc03.au.ibm.com [9.190.26.210]) by d23av03.au.ibm.com (8.14.4/8.13.1/NCO v10.0 AVin) with ESMTP id q9VLc1E2027506; Thu, 1 Nov 2012 08:38:01 +1100
In-Reply-To: <29FB5FA9-B84E-4B5F-97FF-5F81C826F6D0@tik.ee.ethz.ch>
References:   <5087B96B.7020500@cisco.com>  <508850F7.2080801@net.in.tum.de> <50885B49.6050603@cisco.com>  <5088666F.1090106@cisco.com>  <50898454.2000706@net.in.tum.de> <508AB8FB.3060807@plixer.com> <508AE290.3020902@cisco.com> <5b06f4caa55ce88df0f606fb59e19785@net.in.tum.de> <508FC64D.3000006@cisco.com> <03BF5948-51D3-417B-AD8A-5F6B678A9F46@tik.ee.ethz.ch>  <29FB5FA9-B84E-4B5F-97FF-5F81C826F6D0@tik.ee.ethz.ch>
To: Brian Trammell 
MIME-Version: 1.0
X-KeepSent: 4D322E16:60F8D4B2-CA257AA8:0076B4D3; type=4; name=$KeepSent
X-Mailer: Lotus Notes Release 8.5.3 September 15, 2011
Message-ID: 
From: John Court 
Date: Thu, 1 Nov 2012 07:37:17 +1000
X-MIMETrack: Serialize by Router on d23mlc03/23/M/IBM(Release 8.5.3FP2HF29 | July 24, 2012) at 01/11/2012 08:37:17, Serialize complete at 01/11/2012 08:37:17
Content-Type: multipart/alternative; boundary="=_alternative 0076D6164A257AA8_="
x-cbid: 12103121-7014-0000-0000-0000021C1202
Cc: ipfix@ietf.org
Subject: Re: [IPFIX] Export of long lived flow information
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: IPFIX WG discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Wed, 31 Oct 2012 21:38:16 -0000

This is a multipart message in MIME format.
--=_alternative 0076D6164A257AA8_=
Content-Type: text/plain; charset="US-ASCII"

Personally I always find negative logic requires a double read to make 
sure I understand it.  Whats wrong with the positive logic equivalent ?

>>If this Information Element is present in a Flow Record, Flow properties 
in this Flow Record do not refer to packets observed before/after this 
>>timestamp.

Becomes :

If this Information Element is present in a Flow Record, Flow properties 
in this Flow Record ONLY refer to packets observed after/before this 
timestamp.

Thanks


John Court





From:   Brian Trammell 
To:     Gerhard Muenz , 
Cc:     Paul Aitken , John Court/Australia/IBM@IBMAU, 

Date:   31/10/2012 18:17
Subject:        Re: [IPFIX] Export of long lived flow information



Hi, Gerhard, all,

Thought about this a bit too, and I can't come up anything better than the 
negative clarification. I might consider changing "preceding" and 
"succeeding" to "before" and "after" to simplify the language for 
non-native speakers, and stick an "observed" in there to make clear we're 
talking about observation time:

If this Information Element is present in a Flow Record, Flow properties 
in this Flow Record do not refer to packets observed before/after this 
timestamp.

It either case, it does make things a lot more explicitly clear, so I'd 
definitely add it to the timestamp/duration IEs.

Thanks, cheers,

Brian

On Oct 31, 2012, at 8:52 AM, Gerhard Muenz wrote:

> 
> Hi all,
> 
> I scanned through the IANA registry and briefly checked occurrences of 
"this Flow". I do not think that we need change them in general, except 
for those IEs which define a measurement interval.
> 
> So, we have:
> - flowStart* (including flowStartDeltaMicroseconds)
> - flowEnd* (including flowEndDeltaMicroseconds)
> - flowDuration*
> 
> My first shot was the following change for absolute flowStart*/flowEnd* 
timestamps:
> 
> OLD:
> The absolute timestamp of the first|last packet of this Flow.
> NEW:
> The absolute timestamp of the first|last packet accounted in this Flow 
Record.
> 
> We could add a sentence clarifying that the Flow properties reported in 
this Flow Record refer to the given measurement interval.
> At the moment, I only find a good "negative" way to describe this:
> "If this Information Element is present in a Flow Record, Flow 
properties in this Flow Record do not refer to packets 
preceding|succeeding this timestamp."
> 
> Do you have better suggestions?
> Or do you think that this clarification is not necessary?
> 
> Regards,
> Gerhard
> 
> 
> On 30.10.2012 13:38, Brian Trammell wrote:
>> Hi, Paul, all,
>> 
>> If someone throws together a quick summary of the position for me I'd
>> be happy make a couple of slides and do the presentation in person in
>> Atlanta.
>> 
>> From where I sit, it looks like we just go ask IANA to make the
>> registry change. Wearing my IE-Doctors-author hat (I can't wear an IE
>> Doctor hat, there aren't any yet. :) ), I'd say this revision would be
>> covered (and permissible) under point 2 in section 5.2 of ie-doctors.
>> 
>> Best regards,
>> 
>> Brian
>> 
>> 
>> On 30 Oct 2012, at 13:21 , Paul Aitken wrote:
>> 
>>> Gerhard,
>>> 
>>> I agree with your definitions. Thanks for clarifying.
>>> 
>>> So what's the next step?
>>> 
>>>   * Update the IANA definitions?
>>>   * Add clarifications to the WG documents?
>>> 
>>> 
>>> Do we need a short presentation at the upcoming WG meeting?
>>> 
>>> P.
>>> 
>>> 
>>> On 30/10/12 11:51, Gerhard Muenz wrote:
>>>> Paul,
>>>> 
>>>> On 26.10.2012 20:20, Paul Aitken wrote:
>>>>> Andrew, Gerhard,
>>>>> 
>>>>>>> My understanding is that both, deltaCounts and totalCounts contain 
the number of packets or octets observed in the indicated time interval. 
So, for identical flowStart* and flowEnd* timestamps, the values are the 
same.
>>>>>> This is my understanding as well.
>>>>> 
>>>>> There has to be a difference between delta and total counts, else we
>>>>> wouldn't have both of them!
>>>>> 
>>>>> Suppose we have a permanent cache, so the cache entries never 
expire.
>>>>> 
>>>>> For a new flow starting at t0 with a first export at t1, the
>>>>> timestamps, delta, and total counts are the same.
>>>>> 
>>>>> However with the second export at t2, the total and delta counts are
>>>>> different although their timestamps match (they'll both say, "t0 to
>>>>> t2").
>>>> 
>>>> No, this would contradict the new definition of flowStart* we are 
just discussing.
>>>> If delta counts are exported for the interval (t1,t2), then 
flowStart* is t1.
>>>> If delta counts are exported for the interval (t0,t2), then 
flowStart* is t0.
>>>> If total counts are exported, flowStart is always t0.
>>>> These statements hold regardless of which type of cache is used by 
the Metering Process. In general, the information model does not care 
about how the cache is implemented. The exported information just must 
follow the IE definition.
>>>> 
>>>>> 
>>>>> With the traditional (non-permanent) cache, the entry would probably
>>>>> have been removed at t1 and re-created on a subsequent packet, so at
>>>>> t2 the delta and total counts would both be equal. However it'd be
>>>>> incorrect to report the total count, because that's defined as the
>>>>> total number of packets or bytes ..."since the Metering Process
>>>>> (re-)initialization for this Observation Point".
>>>> 
>>>> You must not export total counters in this case because you reset 
counters before re-initialization of the Metering Process.
>>>> 
>>>> Thanks,
>>>> Gerhard
>>>> 
>>>>> 
>>>>> 
>>>>>>> However, the description of totalCounts says that you report the 
number of packets or octets observed for this Flow since 
re-initialization. So, you must never reset the counter for this Flow, 
even after observing a FIN or RST.
>>>>>>> If you reset flow counters, or if you remove Flows from your 
Cache, you cannot use totalCounts any more unless you re-initialize the 
Metering Process (e.g. after flushing the entire permanent Cache).
>>>>>> 
>>>>>> I can try some tests later, but from what I have seen (and been 
told) many totals being exported are in fact just a delta sent once at the 
end of the flow.  If a later flow had the same IPs, protocol, and ports as 
an earlier flow I'm pretty sure a new start time will be sent rather than 
the the first time that flow was seen since reinitializing the metering 
process.
>>>>> 
>>>>> So the MP uses a traditional (non-permanent) cache. In RFC 6728
>>>>> terms, a TimeoutCache or NaturalCache rather than a PermanentCache.
>>>>> 
>>>>> 
>>>>>> Or to put it an other way I think deltas are being sent, but called 
totals by the implementation because it seemed like the right thing to do 
for a value being sent once at the end of the flow.
>>>>> 
>>>>> The collector could be aggregating deltas to keep running totals.
>>>>> 
>>>>> 
>>>>>> I suspect that totals reporting on the export process (eg 
exportedOctetTotalCount, exportedMessageTotalCount) are, however, reported 
with a start time that is only reset on reinitialization.
>>>>> 
>>>>> Definitely, because these are defined as "The total number of X that
>>>>> the Exporting Process has sent since the Exporting Process
>>>>> (re-)initialization ...".
>>>>> 
>>>>> P.
>>> 
>>> _______________________________________________
>>> IPFIX mailing list
>>> IPFIX@ietf.org
>>> https://www.ietf.org/mailman/listinfo/ipfix



--=_alternative 0076D6164A257AA8_=
Content-Type: text/html; charset="US-ASCII"

Personally I always find negative logic
requires a double read to make sure I understand it.  Whats wrong
with the positive logic equivalent ?



>>If this Information Element is present in
a Flow Record, Flow properties in this Flow Record do not refer to packets
observed before/after this >>timestamp.



Becomes :



If this Information Element is present in a Flow Record,
Flow properties in this Flow Record ONLY refer to packets observed after/before
this timestamp.



Thanks





John Court











From:      
 Brian Trammell <trammell@tik.ee.ethz.ch>

To:      
 Gerhard Muenz <muenz@net.in.tum.de>,


Cc:      
 Paul Aitken <paitken@cisco.com>,
John Court/Australia/IBM@IBMAU, <ipfix@ietf.org>

Date:      
 31/10/2012 18:17

Subject:    
   Re: [IPFIX]
Export of long lived flow information









Hi, Gerhard, all,



Thought about this a bit too, and I can't come up anything better than
the negative clarification. I might consider changing "preceding"
and "succeeding" to "before" and "after"
to simplify the language for non-native speakers, and stick an "observed"
in there to make clear we're talking about observation time:



If this Information Element is present in a Flow Record, Flow properties
in this Flow Record do not refer to packets observed before/after this
timestamp.



It either case, it does make things a lot more explicitly clear, so I'd
definitely add it to the timestamp/duration IEs.



Thanks, cheers,



Brian



On Oct 31, 2012, at 8:52 AM, Gerhard Muenz wrote:



> 

> Hi all,

> 

> I scanned through the IANA registry and briefly checked occurrences
of "this Flow". I do not think that we need change them in general,
except for those IEs which define a measurement interval.

> 

> So, we have:

> - flowStart* (including flowStartDeltaMicroseconds)

> - flowEnd* (including flowEndDeltaMicroseconds)

> - flowDuration*

> 

> My first shot was the following change for absolute flowStart*/flowEnd*
timestamps:

> 

> OLD:

> The absolute timestamp of the first|last packet of this Flow.

> NEW:

> The absolute timestamp of the first|last packet accounted in this
Flow Record.

> 

> We could add a sentence clarifying that the Flow properties reported
in this Flow Record refer to the given measurement interval.

> At the moment, I only find a good "negative" way to describe
this:

> "If this Information Element is present in a Flow Record, Flow
properties in this Flow Record do not refer to packets preceding|succeeding
this timestamp."

> 

> Do you have better suggestions?

> Or do you think that this clarification is not necessary?

> 

> Regards,

> Gerhard

> 

> 

> On 30.10.2012 13:38, Brian Trammell wrote:

>> Hi, Paul, all,

>> 

>> If someone throws together a quick summary of the position for
me I'd

>> be happy make a couple of slides and do the presentation in person
in

>> Atlanta.

>> 

>> From where I sit, it looks like we just go ask IANA to make the

>> registry change. Wearing my IE-Doctors-author hat (I can't wear
an IE

>> Doctor hat, there aren't any yet. :) ), I'd say this revision
would be

>> covered (and permissible) under point 2 in section 5.2 of ie-doctors.

>> 

>> Best regards,

>> 

>> Brian

>> 

>> 

>> On 30 Oct 2012, at 13:21 , Paul Aitken wrote:

>> 

>>> Gerhard,

>>> 

>>> I agree with your definitions. Thanks for clarifying.

>>> 

>>> So what's the next step?

>>> 

>>>   * Update the IANA definitions?

>>>   * Add clarifications to the WG documents?

>>> 

>>> 

>>> Do we need a short presentation at the upcoming WG meeting?

>>> 

>>> P.

>>> 

>>> 

>>> On 30/10/12 11:51, Gerhard Muenz wrote:

>>>> Paul,

>>>> 

>>>> On 26.10.2012 20:20, Paul Aitken wrote:

>>>>> Andrew, Gerhard,

>>>>> 

>>>>>>> My understanding is that both, deltaCounts
and totalCounts contain the number of packets or octets observed in the
indicated time interval. So, for identical flowStart* and flowEnd* timestamps,
the values are the same.

>>>>>> This is my understanding as well.

>>>>> 

>>>>> There has to be a difference between delta and total
counts, else we

>>>>> wouldn't have both of them!

>>>>> 

>>>>> Suppose we have a permanent cache, so the cache entries
never expire.

>>>>> 

>>>>> For a new flow starting at t0 with a first export
at t1, the

>>>>> timestamps, delta, and total counts are the same.

>>>>> 

>>>>> However with the second export at t2, the total and
delta counts are

>>>>> different although their timestamps match (they'll
both say, "t0 to

>>>>> t2").

>>>> 

>>>> No, this would contradict the new definition of flowStart*
we are just discussing.

>>>> If delta counts are exported for the interval (t1,t2),
then flowStart* is t1.

>>>> If delta counts are exported for the interval (t0,t2),
then flowStart* is t0.

>>>> If total counts are exported, flowStart is always t0.

>>>> These statements hold regardless of which type of cache
is used by the Metering Process. In general, the information model does
not care about how the cache is implemented. The exported information just
must follow the IE definition.

>>>> 

>>>>> 

>>>>> With the traditional (non-permanent) cache, the entry
would probably

>>>>> have been removed at t1 and re-created on a subsequent
packet, so at

>>>>> t2 the delta and total counts would both be equal.
However it'd be

>>>>> incorrect to report the total count, because that's
defined as the

>>>>> total number of packets or bytes ..."since the
Metering Process

>>>>> (re-)initialization for this Observation Point".

>>>> 

>>>> You must not export total counters in this case because
you reset counters before re-initialization of the Metering Process.

>>>> 

>>>> Thanks,

>>>> Gerhard

>>>> 

>>>>> 

>>>>> 

>>>>>>> However, the description of totalCounts says
that you report the number of packets or octets observed for this Flow
since re-initialization. So, you must never reset the counter for this
Flow, even after observing a FIN or RST.

>>>>>>> If you reset flow counters, or if you remove
Flows from your Cache, you cannot use totalCounts any more unless you re-initialize
the Metering Process (e.g. after flushing the entire permanent Cache).

>>>>>> 

>>>>>> I can try some tests later, but from what I have
seen (and been told) many totals being exported are in fact just a delta
sent once at the end of the flow.  If a later flow had the same IPs,
protocol, and ports as an earlier flow I'm pretty sure a new start time
will be sent rather than the the first time that flow was seen since reinitializing
the metering process.

>>>>> 

>>>>> So the MP uses a traditional (non-permanent) cache.
In RFC 6728

>>>>> terms, a TimeoutCache or NaturalCache rather than
a PermanentCache.

>>>>> 

>>>>> 

>>>>>> Or to put it an other way I think deltas are being
sent, but called totals by the implementation because it seemed like the
right thing to do for a value being sent once at the end of the flow.

>>>>> 

>>>>> The collector could be aggregating deltas to keep
running totals.

>>>>> 

>>>>> 

>>>>>> I suspect that totals reporting on the export
process (eg exportedOctetTotalCount, exportedMessageTotalCount) are, however,
reported with a start time that is only reset on reinitialization.

>>>>> 

>>>>> Definitely, because these are defined as "The
total number of X that

>>>>> the Exporting Process has sent since the Exporting
Process

>>>>> (re-)initialization ...".

>>>>> 

>>>>> P.

>>> 

>>> _______________________________________________

>>> IPFIX mailing list

>>> IPFIX@ietf.org

>>> https://www.ietf.org/mailman/listinfo/ipfix






--=_alternative 0076D6164A257AA8_=--


From johnwcrt@au1.ibm.com  Wed Oct 31 15:02:19 2012
Return-Path: 
X-Original-To: ipfix@ietfa.amsl.com
Delivered-To: ipfix@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9C02B21F87B8 for ; Wed, 31 Oct 2012 15:02:19 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -7.998
X-Spam-Level: 
X-Spam-Status: No, score=-7.998 tagged_above=-999 required=5 tests=[AWL=-2.000, BAYES_00=-2.599, HTML_MESSAGE=0.001, J_CHICKENPOX_54=0.6, RCVD_IN_DNSWL_MED=-4]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 48hKpXEjvbNJ for ; Wed, 31 Oct 2012 15:02:18 -0700 (PDT)
Received: from e23smtp04.au.ibm.com (e23smtp04.au.ibm.com [202.81.31.146]) by ietfa.amsl.com (Postfix) with ESMTP id 5A1D621F879A for ; Wed, 31 Oct 2012 15:02:16 -0700 (PDT)
Received: from /spool/local by e23smtp04.au.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for  from ; Thu, 1 Nov 2012 07:57:43 +1000
Received: from d23relay03.au.ibm.com (202.81.31.245) by e23smtp04.au.ibm.com (202.81.31.210) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted;  Thu, 1 Nov 2012 07:57:41 +1000
Received: from d23av03.au.ibm.com (d23av03.au.ibm.com [9.190.234.97]) by d23relay03.au.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id q9VM25DD64946398 for ; Thu, 1 Nov 2012 09:02:06 +1100
Received: from d23av03.au.ibm.com (loopback [127.0.0.1]) by d23av03.au.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id q9VM24KS023337 for ; Thu, 1 Nov 2012 09:02:04 +1100
Received: from d23mlc03.au.ibm.com (d23mlc03.au.ibm.com [9.190.26.210]) by d23av03.au.ibm.com (8.14.4/8.13.1/NCO v10.0 AVin) with ESMTP id q9VM24Z9023334; Thu, 1 Nov 2012 09:02:04 +1100
To: Brian Trammell 
MIME-Version: 1.0
X-KeepSent: 55562F5E:2B7AD32B-CA257AA8:00789598; type=4; name=$KeepSent
X-Mailer: Lotus Notes Release 8.5.3 September 15, 2011
Message-ID: 
From: John Court 
Date: Thu, 1 Nov 2012 08:01:20 +1000
X-MIMETrack: Serialize by Router on d23mlc03/23/M/IBM(Release 8.5.3FP2HF29 | July 24, 2012) at 01/11/2012 09:01:20, Serialize complete at 01/11/2012 09:01:20
Content-Type: multipart/alternative; boundary="=_alternative 007909AB4A257AA8_="
x-cbid: 12103121-9264-0000-0000-0000029680FA
Cc: ipfix@ietf.org
Subject: [IPFIX] Fw:  Export of long lived flow information
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: IPFIX WG discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Wed, 31 Oct 2012 22:02:19 -0000

This is a multipart message in MIME format.
--=_alternative 007909AB4A257AA8_=
Content-Type: text/plain; charset="US-ASCII"

Damn,

Sorry about the previous message, when I re-read it the third time I 
realised that the positive allows the confusion of failing to clearly 
indicate what "isn't" included.

I am not sure if this is better but what about :

If this Information Element is present in a Flow Record, Flow properties 
in this Flow Record MUST account for ALL packets observed after/before 
this timestamp.


John Court


----- Forwarded by John Court/Australia/IBM on 01/11/2012 07:56 -----

From:   John Court/Australia/IBM
To:     Brian Trammell , 
Cc:     ipfix@ietf.org, Gerhard Muenz , Paul Aitken 

Date:   01/11/2012 07:37
Subject:        Re: [IPFIX] Export of long lived flow information


Personally I always find negative logic requires a double read to make 
sure I understand it.  Whats wrong with the positive logic equivalent ?

>>If this Information Element is present in a Flow Record, Flow properties 
in this Flow Record do not refer to packets observed before/after this 
>>timestamp.

Becomes :

If this Information Element is present in a Flow Record, Flow properties 
in this Flow Record ONLY refer to packets observed after/before this 
timestamp.

Thanks


John Court





From:   Brian Trammell 
To:     Gerhard Muenz , 
Cc:     Paul Aitken , John Court/Australia/IBM@IBMAU, 

Date:   31/10/2012 18:17
Subject:        Re: [IPFIX] Export of long lived flow information



Hi, Gerhard, all,

Thought about this a bit too, and I can't come up anything better than the 
negative clarification. I might consider changing "preceding" and 
"succeeding" to "before" and "after" to simplify the language for 
non-native speakers, and stick an "observed" in there to make clear we're 
talking about observation time:

If this Information Element is present in a Flow Record, Flow properties 
in this Flow Record do not refer to packets observed before/after this 
timestamp.

It either case, it does make things a lot more explicitly clear, so I'd 
definitely add it to the timestamp/duration IEs.

Thanks, cheers,

Brian

On Oct 31, 2012, at 8:52 AM, Gerhard Muenz wrote:

> 
> Hi all,
> 
> I scanned through the IANA registry and briefly checked occurrences of 
"this Flow". I do not think that we need change them in general, except 
for those IEs which define a measurement interval.
> 
> So, we have:
> - flowStart* (including flowStartDeltaMicroseconds)
> - flowEnd* (including flowEndDeltaMicroseconds)
> - flowDuration*
> 
> My first shot was the following change for absolute flowStart*/flowEnd* 
timestamps:
> 
> OLD:
> The absolute timestamp of the first|last packet of this Flow.
> NEW:
> The absolute timestamp of the first|last packet accounted in this Flow 
Record.
> 
> We could add a sentence clarifying that the Flow properties reported in 
this Flow Record refer to the given measurement interval.
> At the moment, I only find a good "negative" way to describe this:
> "If this Information Element is present in a Flow Record, Flow 
properties in this Flow Record do not refer to packets 
preceding|succeeding this timestamp."
> 
> Do you have better suggestions?
> Or do you think that this clarification is not necessary?
> 
> Regards,
> Gerhard
> 
> 
> On 30.10.2012 13:38, Brian Trammell wrote:
>> Hi, Paul, all,
>> 
>> If someone throws together a quick summary of the position for me I'd
>> be happy make a couple of slides and do the presentation in person in
>> Atlanta.
>> 
>> From where I sit, it looks like we just go ask IANA to make the
>> registry change. Wearing my IE-Doctors-author hat (I can't wear an IE
>> Doctor hat, there aren't any yet. :) ), I'd say this revision would be
>> covered (and permissible) under point 2 in section 5.2 of ie-doctors.
>> 
>> Best regards,
>> 
>> Brian
>> 
>> 
>> On 30 Oct 2012, at 13:21 , Paul Aitken wrote:
>> 
>>> Gerhard,
>>> 
>>> I agree with your definitions. Thanks for clarifying.
>>> 
>>> So what's the next step?
>>> 
>>>   * Update the IANA definitions?
>>>   * Add clarifications to the WG documents?
>>> 
>>> 
>>> Do we need a short presentation at the upcoming WG meeting?
>>> 
>>> P.
>>> 
>>> 
>>> On 30/10/12 11:51, Gerhard Muenz wrote:
>>>> Paul,
>>>> 
>>>> On 26.10.2012 20:20, Paul Aitken wrote:
>>>>> Andrew, Gerhard,
>>>>> 
>>>>>>> My understanding is that both, deltaCounts and totalCounts contain 
the number of packets or octets observed in the indicated time interval. 
So, for identical flowStart* and flowEnd* timestamps, the values are the 
same.
>>>>>> This is my understanding as well.
>>>>> 
>>>>> There has to be a difference between delta and total counts, else we
>>>>> wouldn't have both of them!
>>>>> 
>>>>> Suppose we have a permanent cache, so the cache entries never 
expire.
>>>>> 
>>>>> For a new flow starting at t0 with a first export at t1, the
>>>>> timestamps, delta, and total counts are the same.
>>>>> 
>>>>> However with the second export at t2, the total and delta counts are
>>>>> different although their timestamps match (they'll both say, "t0 to
>>>>> t2").
>>>> 
>>>> No, this would contradict the new definition of flowStart* we are 
just discussing.
>>>> If delta counts are exported for the interval (t1,t2), then 
flowStart* is t1.
>>>> If delta counts are exported for the interval (t0,t2), then 
flowStart* is t0.
>>>> If total counts are exported, flowStart is always t0.
>>>> These statements hold regardless of which type of cache is used by 
the Metering Process. In general, the information model does not care 
about how the cache is implemented. The exported information just must 
follow the IE definition.
>>>> 
>>>>> 
>>>>> With the traditional (non-permanent) cache, the entry would probably
>>>>> have been removed at t1 and re-created on a subsequent packet, so at
>>>>> t2 the delta and total counts would both be equal. However it'd be
>>>>> incorrect to report the total count, because that's defined as the
>>>>> total number of packets or bytes ..."since the Metering Process
>>>>> (re-)initialization for this Observation Point".
>>>> 
>>>> You must not export total counters in this case because you reset 
counters before re-initialization of the Metering Process.
>>>> 
>>>> Thanks,
>>>> Gerhard
>>>> 
>>>>> 
>>>>> 
>>>>>>> However, the description of totalCounts says that you report the 
number of packets or octets observed for this Flow since 
re-initialization. So, you must never reset the counter for this Flow, 
even after observing a FIN or RST.
>>>>>>> If you reset flow counters, or if you remove Flows from your 
Cache, you cannot use totalCounts any more unless you re-initialize the 
Metering Process (e.g. after flushing the entire permanent Cache).
>>>>>> 
>>>>>> I can try some tests later, but from what I have seen (and been 
told) many totals being exported are in fact just a delta sent once at the 
end of the flow.  If a later flow had the same IPs, protocol, and ports as 
an earlier flow I'm pretty sure a new start time will be sent rather than 
the the first time that flow was seen since reinitializing the metering 
process.
>>>>> 
>>>>> So the MP uses a traditional (non-permanent) cache. In RFC 6728
>>>>> terms, a TimeoutCache or NaturalCache rather than a PermanentCache.
>>>>> 
>>>>> 
>>>>>> Or to put it an other way I think deltas are being sent, but called 
totals by the implementation because it seemed like the right thing to do 
for a value being sent once at the end of the flow.
>>>>> 
>>>>> The collector could be aggregating deltas to keep running totals.
>>>>> 
>>>>> 
>>>>>> I suspect that totals reporting on the export process (eg 
exportedOctetTotalCount, exportedMessageTotalCount) are, however, reported 
with a start time that is only reset on reinitialization.
>>>>> 
>>>>> Definitely, because these are defined as "The total number of X that
>>>>> the Exporting Process has sent since the Exporting Process
>>>>> (re-)initialization ...".
>>>>> 
>>>>> P.
>>> 
>>> _______________________________________________
>>> IPFIX mailing list
>>> IPFIX@ietf.org
>>> https://www.ietf.org/mailman/listinfo/ipfix



--=_alternative 007909AB4A257AA8_=
Content-Type: text/html; charset="US-ASCII"

Damn,



Sorry about the previous message, when
I re-read it the third time I realised that the positive allows the confusion
of failing to clearly indicate what "isn't" included.



I am not sure if this is better but
what about :



If this Information Element is present in a Flow Record,
Flow properties in this Flow Record MUST account for ALL packets observed
after/before this timestamp.





John Court





----- Forwarded by John
Court/Australia/IBM on 01/11/2012 07:56 -----



From:      
 John Court/Australia/IBM

To:      
 Brian Trammell <trammell@tik.ee.ethz.ch>,


Cc:      
 ipfix@ietf.org, Gerhard
Muenz <muenz@net.in.tum.de>, Paul Aitken <paitken@cisco.com>

Date:      
 01/11/2012 07:37

Subject:    
   Re: [IPFIX]
Export of long lived flow information







Personally I always find negative logic
requires a double read to make sure I understand it.  Whats wrong
with the positive logic equivalent ?



>>If this Information Element is present in
a Flow Record, Flow properties in this Flow Record do not refer to packets
observed before/after this >>timestamp.



Becomes :



If this Information Element is present in a Flow Record,
Flow properties in this Flow Record ONLY refer to packets observed after/before
this timestamp.



Thanks





John Court











From:      
 Brian Trammell <trammell@tik.ee.ethz.ch>

To:      
 Gerhard Muenz <muenz@net.in.tum.de>,


Cc:      
 Paul Aitken <paitken@cisco.com>,
John Court/Australia/IBM@IBMAU, <ipfix@ietf.org>

Date:      
 31/10/2012 18:17

Subject:    
   Re: [IPFIX]
Export of long lived flow information









Hi, Gerhard, all,



Thought about this a bit too, and I can't come up anything better than
the negative clarification. I might consider changing "preceding"
and "succeeding" to "before" and "after"
to simplify the language for non-native speakers, and stick an "observed"
in there to make clear we're talking about observation time:



If this Information Element is present in a Flow Record, Flow properties
in this Flow Record do not refer to packets observed before/after this
timestamp.



It either case, it does make things a lot more explicitly clear, so I'd
definitely add it to the timestamp/duration IEs.



Thanks, cheers,



Brian



On Oct 31, 2012, at 8:52 AM, Gerhard Muenz wrote:



> 

> Hi all,

> 

> I scanned through the IANA registry and briefly checked occurrences
of "this Flow". I do not think that we need change them in general,
except for those IEs which define a measurement interval.

> 

> So, we have:

> - flowStart* (including flowStartDeltaMicroseconds)

> - flowEnd* (including flowEndDeltaMicroseconds)

> - flowDuration*

> 

> My first shot was the following change for absolute flowStart*/flowEnd*
timestamps:

> 

> OLD:

> The absolute timestamp of the first|last packet of this Flow.

> NEW:

> The absolute timestamp of the first|last packet accounted in this
Flow Record.

> 

> We could add a sentence clarifying that the Flow properties reported
in this Flow Record refer to the given measurement interval.

> At the moment, I only find a good "negative" way to describe
this:

> "If this Information Element is present in a Flow Record, Flow
properties in this Flow Record do not refer to packets preceding|succeeding
this timestamp."

> 

> Do you have better suggestions?

> Or do you think that this clarification is not necessary?

> 

> Regards,

> Gerhard

> 

> 

> On 30.10.2012 13:38, Brian Trammell wrote:

>> Hi, Paul, all,

>> 

>> If someone throws together a quick summary of the position for
me I'd

>> be happy make a couple of slides and do the presentation in person
in

>> Atlanta.

>> 

>> From where I sit, it looks like we just go ask IANA to make the

>> registry change. Wearing my IE-Doctors-author hat (I can't wear
an IE

>> Doctor hat, there aren't any yet. :) ), I'd say this revision
would be

>> covered (and permissible) under point 2 in section 5.2 of ie-doctors.

>> 

>> Best regards,

>> 

>> Brian

>> 

>> 

>> On 30 Oct 2012, at 13:21 , Paul Aitken wrote:

>> 

>>> Gerhard,

>>> 

>>> I agree with your definitions. Thanks for clarifying.

>>> 

>>> So what's the next step?

>>> 

>>>   * Update the IANA definitions?

>>>   * Add clarifications to the WG documents?

>>> 

>>> 

>>> Do we need a short presentation at the upcoming WG meeting?

>>> 

>>> P.

>>> 

>>> 

>>> On 30/10/12 11:51, Gerhard Muenz wrote:

>>>> Paul,

>>>> 

>>>> On 26.10.2012 20:20, Paul Aitken wrote:

>>>>> Andrew, Gerhard,

>>>>> 

>>>>>>> My understanding is that both, deltaCounts
and totalCounts contain the number of packets or octets observed in the
indicated time interval. So, for identical flowStart* and flowEnd* timestamps,
the values are the same.

>>>>>> This is my understanding as well.

>>>>> 

>>>>> There has to be a difference between delta and total
counts, else we

>>>>> wouldn't have both of them!

>>>>> 

>>>>> Suppose we have a permanent cache, so the cache entries
never expire.

>>>>> 

>>>>> For a new flow starting at t0 with a first export
at t1, the

>>>>> timestamps, delta, and total counts are the same.

>>>>> 

>>>>> However with the second export at t2, the total and
delta counts are

>>>>> different although their timestamps match (they'll
both say, "t0 to

>>>>> t2").

>>>> 

>>>> No, this would contradict the new definition of flowStart*
we are just discussing.

>>>> If delta counts are exported for the interval (t1,t2),
then flowStart* is t1.

>>>> If delta counts are exported for the interval (t0,t2),
then flowStart* is t0.

>>>> If total counts are exported, flowStart is always t0.

>>>> These statements hold regardless of which type of cache
is used by the Metering Process. In general, the information model does
not care about how the cache is implemented. The exported information just
must follow the IE definition.

>>>> 

>>>>> 

>>>>> With the traditional (non-permanent) cache, the entry
would probably

>>>>> have been removed at t1 and re-created on a subsequent
packet, so at

>>>>> t2 the delta and total counts would both be equal.
However it'd be

>>>>> incorrect to report the total count, because that's
defined as the

>>>>> total number of packets or bytes ..."since the
Metering Process

>>>>> (re-)initialization for this Observation Point".

>>>> 

>>>> You must not export total counters in this case because
you reset counters before re-initialization of the Metering Process.

>>>> 

>>>> Thanks,

>>>> Gerhard

>>>> 

>>>>> 

>>>>> 

>>>>>>> However, the description of totalCounts says
that you report the number of packets or octets observed for this Flow
since re-initialization. So, you must never reset the counter for this
Flow, even after observing a FIN or RST.

>>>>>>> If you reset flow counters, or if you remove
Flows from your Cache, you cannot use totalCounts any more unless you re-initialize
the Metering Process (e.g. after flushing the entire permanent Cache).

>>>>>> 

>>>>>> I can try some tests later, but from what I have
seen (and been told) many totals being exported are in fact just a delta
sent once at the end of the flow.  If a later flow had the same IPs,
protocol, and ports as an earlier flow I'm pretty sure a new start time
will be sent rather than the the first time that flow was seen since reinitializing
the metering process.

>>>>> 

>>>>> So the MP uses a traditional (non-permanent) cache.
In RFC 6728

>>>>> terms, a TimeoutCache or NaturalCache rather than
a PermanentCache.

>>>>> 

>>>>> 

>>>>>> Or to put it an other way I think deltas are being
sent, but called totals by the implementation because it seemed like the
right thing to do for a value being sent once at the end of the flow.

>>>>> 

>>>>> The collector could be aggregating deltas to keep
running totals.

>>>>> 

>>>>> 

>>>>>> I suspect that totals reporting on the export
process (eg exportedOctetTotalCount, exportedMessageTotalCount) are, however,
reported with a start time that is only reset on reinitialization.

>>>>> 

>>>>> Definitely, because these are defined as "The
total number of X that

>>>>> the Exporting Process has sent since the Exporting
Process

>>>>> (re-)initialization ...".

>>>>> 

>>>>> P.

>>> 

>>> _______________________________________________

>>> IPFIX mailing list

>>> IPFIX@ietf.org

>>> https://www.ietf.org/mailman/listinfo/ipfix






--=_alternative 007909AB4A257AA8_=--


From trammell@tik.ee.ethz.ch  Wed Oct 31 23:22:42 2012
Return-Path: 
X-Original-To: ipfix@ietfa.amsl.com
Delivered-To: ipfix@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9E39021F8448 for ; Wed, 31 Oct 2012 23:22:42 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -6.153
X-Spam-Level: 
X-Spam-Status: No, score=-6.153 tagged_above=-999 required=5 tests=[AWL=-0.154, BAYES_00=-2.599, J_CHICKENPOX_54=0.6, RCVD_IN_DNSWL_MED=-4]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id QZSykebs7Orm for ; Wed, 31 Oct 2012 23:22:41 -0700 (PDT)
Received: from smtp.ee.ethz.ch (smtp.ee.ethz.ch [129.132.2.219]) by ietfa.amsl.com (Postfix) with ESMTP id 5451C21F848B for ; Wed, 31 Oct 2012 23:22:40 -0700 (PDT)
Received: from localhost (localhost [127.0.0.1]) by smtp.ee.ethz.ch (Postfix) with ESMTP id 878BFD930A; Thu,  1 Nov 2012 07:22:38 +0100 (MET)
X-Virus-Scanned: by amavisd-new on smtp.ee.ethz.ch
Received: from smtp.ee.ethz.ch ([127.0.0.1]) by localhost (.ee.ethz.ch [127.0.0.1]) (amavisd-new, port 10024) with LMTP id CE1csphyq5Nv; Thu,  1 Nov 2012 07:22:38 +0100 (MET)
Received: from [10.0.27.100] (cust-integra-122-165.antanet.ch [80.75.122.165]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: briant) by smtp.ee.ethz.ch (Postfix) with ESMTPSA id 29037D9305; Thu,  1 Nov 2012 07:22:38 +0100 (MET)
Mime-Version: 1.0 (Apple Message framework v1283)
Content-Type: text/plain; charset=us-ascii
From: Brian Trammell 
In-Reply-To: 
Date: Thu, 1 Nov 2012 07:22:37 +0100
Content-Transfer-Encoding: quoted-printable
Message-Id: <2B2D0366-B036-4830-A2D9-47940CDC6E79@tik.ee.ethz.ch>
References: 
To: John Court 
X-Mailer: Apple Mail (2.1283)
Cc: ipfix@ietf.org
Subject: Re: [IPFIX] Export of long lived flow information
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: IPFIX WG discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Thu, 01 Nov 2012 06:22:42 -0000

Hi, John,

Thanks, but neither of these work, really. The former must make =
reference to both timestamps, and the latter doesn't account for the =
possibility that an MP may either fail to observe a packet, decide not =
to include a packet for its own reasons, or be dealing with observations =
that aren't really packets at all.

Cheers,

Brian


On Oct 31, 2012, at 11:01 PM, John Court wrote:

> Damn,=20
>=20
> Sorry about the previous message, when I re-read it the third time I =
realised that the positive allows the confusion of failing to clearly =
indicate what "isn't" included.=20
>=20
> I am not sure if this is better but what about :=20
>=20
> If this Information Element is present in a Flow Record, Flow =
properties in this Flow Record MUST account for ALL packets observed =
after/before this timestamp.=20
>=20
>=20
> John Court
>=20
>=20
> ----- Forwarded by John Court/Australia/IBM on 01/11/2012 07:56 -----=20=

>=20
> From:        John Court/Australia/IBM=20
> To:        Brian Trammell ,=20
> Cc:        ipfix@ietf.org, Gerhard Muenz , Paul =
Aitken =20
> Date:        01/11/2012 07:37=20
> Subject:        Re: [IPFIX] Export of long lived flow information=20
>=20
>=20
> Personally I always find negative logic requires a double read to make =
sure I understand it.  Whats wrong with the positive logic equivalent ?=20=

>=20
> >>If this Information Element is present in a Flow Record, Flow =
properties in this Flow Record do not refer to packets observed =
before/after this >>timestamp.=20
>=20
> Becomes :=20
>=20
> If this Information Element is present in a Flow Record, Flow =
properties in this Flow Record ONLY refer to packets observed =
after/before this timestamp.=20
>=20
> Thanks=20
>=20
>=20
> John Court
>=20
>=20
>=20
>=20
>=20
> From:        Brian Trammell =20
> To:        Gerhard Muenz ,=20
> Cc:        Paul Aitken , John =
Court/Australia/IBM@IBMAU, =20
> Date:        31/10/2012 18:17=20
> Subject:        Re: [IPFIX] Export of long lived flow information=20
>=20
>=20
>=20
> Hi, Gerhard, all,
>=20
> Thought about this a bit too, and I can't come up anything better than =
the negative clarification. I might consider changing "preceding" and =
"succeeding" to "before" and "after" to simplify the language for =
non-native speakers, and stick an "observed" in there to make clear =
we're talking about observation time:
>=20
> If this Information Element is present in a Flow Record, Flow =
properties in this Flow Record do not refer to packets observed =
before/after this timestamp.
>=20
> It either case, it does make things a lot more explicitly clear, so =
I'd definitely add it to the timestamp/duration IEs.
>=20
> Thanks, cheers,
>=20
> Brian
>=20
> On Oct 31, 2012, at 8:52 AM, Gerhard Muenz wrote:
>=20
> >=20
> > Hi all,
> >=20
> > I scanned through the IANA registry and briefly checked occurrences =
of "this Flow". I do not think that we need change them in general, =
except for those IEs which define a measurement interval.
> >=20
> > So, we have:
> > - flowStart* (including flowStartDeltaMicroseconds)
> > - flowEnd* (including flowEndDeltaMicroseconds)
> > - flowDuration*
> >=20
> > My first shot was the following change for absolute =
flowStart*/flowEnd* timestamps:
> >=20
> > OLD:
> > The absolute timestamp of the first|last packet of this Flow.
> > NEW:
> > The absolute timestamp of the first|last packet accounted in this =
Flow Record.
> >=20
> > We could add a sentence clarifying that the Flow properties reported =
in this Flow Record refer to the given measurement interval.
> > At the moment, I only find a good "negative" way to describe this:
> > "If this Information Element is present in a Flow Record, Flow =
properties in this Flow Record do not refer to packets =
preceding|succeeding this timestamp."
> >=20
> > Do you have better suggestions?
> > Or do you think that this clarification is not necessary?
> >=20
> > Regards,
> > Gerhard
> >=20
> >=20
> > On 30.10.2012 13:38, Brian Trammell wrote:
> >> Hi, Paul, all,
> >>=20
> >> If someone throws together a quick summary of the position for me =
I'd
> >> be happy make a couple of slides and do the presentation in person =
in
> >> Atlanta.
> >>=20
> >> =46rom where I sit, it looks like we just go ask IANA to make the
> >> registry change. Wearing my IE-Doctors-author hat (I can't wear an =
IE
> >> Doctor hat, there aren't any yet. :) ), I'd say this revision would =
be
> >> covered (and permissible) under point 2 in section 5.2 of =
ie-doctors.
> >>=20
> >> Best regards,
> >>=20
> >> Brian
> >>=20
> >>=20
> >> On 30 Oct 2012, at 13:21 , Paul Aitken wrote:
> >>=20
> >>> Gerhard,
> >>>=20
> >>> I agree with your definitions. Thanks for clarifying.
> >>>=20
> >>> So what's the next step?
> >>>=20
> >>>   * Update the IANA definitions?
> >>>   * Add clarifications to the WG documents?
> >>>=20
> >>>=20
> >>> Do we need a short presentation at the upcoming WG meeting?
> >>>=20
> >>> P.
> >>>=20
> >>>=20
> >>> On 30/10/12 11:51, Gerhard Muenz wrote:
> >>>> Paul,
> >>>>=20
> >>>> On 26.10.2012 20:20, Paul Aitken wrote:
> >>>>> Andrew, Gerhard,
> >>>>>=20
> >>>>>>> My understanding is that both, deltaCounts and totalCounts =
contain the number of packets or octets observed in the indicated time =
interval. So, for identical flowStart* and flowEnd* timestamps, the =
values are the same.
> >>>>>> This is my understanding as well.
> >>>>>=20
> >>>>> There has to be a difference between delta and total counts, =
else we
> >>>>> wouldn't have both of them!
> >>>>>=20
> >>>>> Suppose we have a permanent cache, so the cache entries never =
expire.
> >>>>>=20
> >>>>> For a new flow starting at t0 with a first export at t1, the
> >>>>> timestamps, delta, and total counts are the same.
> >>>>>=20
> >>>>> However with the second export at t2, the total and delta counts =
are
> >>>>> different although their timestamps match (they'll both say, "t0 =
to
> >>>>> t2").
> >>>>=20
> >>>> No, this would contradict the new definition of flowStart* we are =
just discussing.
> >>>> If delta counts are exported for the interval (t1,t2), then =
flowStart* is t1.
> >>>> If delta counts are exported for the interval (t0,t2), then =
flowStart* is t0.
> >>>> If total counts are exported, flowStart is always t0.
> >>>> These statements hold regardless of which type of cache is used =
by the Metering Process. In general, the information model does not care =
about how the cache is implemented. The exported information just must =
follow the IE definition.
> >>>>=20
> >>>>>=20
> >>>>> With the traditional (non-permanent) cache, the entry would =
probably
> >>>>> have been removed at t1 and re-created on a subsequent packet, =
so at
> >>>>> t2 the delta and total counts would both be equal. However it'd =
be
> >>>>> incorrect to report the total count, because that's defined as =
the
> >>>>> total number of packets or bytes ..."since the Metering Process
> >>>>> (re-)initialization for this Observation Point".
> >>>>=20
> >>>> You must not export total counters in this case because you reset =
counters before re-initialization of the Metering Process.
> >>>>=20
> >>>> Thanks,
> >>>> Gerhard
> >>>>=20
> >>>>>=20
> >>>>>=20
> >>>>>>> However, the description of totalCounts says that you report =
the number of packets or octets observed for this Flow since =
re-initialization. So, you must never reset the counter for this Flow, =
even after observing a FIN or RST.
> >>>>>>> If you reset flow counters, or if you remove Flows from your =
Cache, you cannot use totalCounts any more unless you re-initialize the =
Metering Process (e.g. after flushing the entire permanent Cache).
> >>>>>>=20
> >>>>>> I can try some tests later, but from what I have seen (and been =
told) many totals being exported are in fact just a delta sent once at =
the end of the flow.  If a later flow had the same IPs, protocol, and =
ports as an earlier flow I'm pretty sure a new start time will be sent =
rather than the the first time that flow was seen since reinitializing =
the metering process.
> >>>>>=20
> >>>>> So the MP uses a traditional (non-permanent) cache. In RFC 6728
> >>>>> terms, a TimeoutCache or NaturalCache rather than a =
PermanentCache.
> >>>>>=20
> >>>>>=20
> >>>>>> Or to put it an other way I think deltas are being sent, but =
called totals by the implementation because it seemed like the right =
thing to do for a value being sent once at the end of the flow.
> >>>>>=20
> >>>>> The collector could be aggregating deltas to keep running =
totals.
> >>>>>=20
> >>>>>=20
> >>>>>> I suspect that totals reporting on the export process (eg =
exportedOctetTotalCount, exportedMessageTotalCount) are, however, =
reported with a start time that is only reset on reinitialization.
> >>>>>=20
> >>>>> Definitely, because these are defined as "The total number of X =
that
> >>>>> the Exporting Process has sent since the Exporting Process
> >>>>> (re-)initialization ...".
> >>>>>=20
> >>>>> P.
> >>>=20
> >>> _______________________________________________
> >>> IPFIX mailing list
> >>> IPFIX@ietf.org
> >>> https://www.ietf.org/mailman/listinfo/ipfix
>=20
>=20


From trammell@tik.ee.ethz.ch  Wed Oct 31 23:35:09 2012
Return-Path: 
X-Original-To: ipfix@ietfa.amsl.com
Delivered-To: ipfix@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4D28E21F851B for ; Wed, 31 Oct 2012 23:35:09 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -6.422
X-Spam-Level: 
X-Spam-Status: No, score=-6.422 tagged_above=-999 required=5 tests=[AWL=0.177,  BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ja5H5+Ul-WYq for ; Wed, 31 Oct 2012 23:35:08 -0700 (PDT)
Received: from smtp.ee.ethz.ch (smtp.ee.ethz.ch [129.132.2.219]) by ietfa.amsl.com (Postfix) with ESMTP id A554E21F84AE for ; Wed, 31 Oct 2012 23:35:08 -0700 (PDT)
Received: from localhost (localhost [127.0.0.1]) by smtp.ee.ethz.ch (Postfix) with ESMTP id 14A1CD930A; Thu,  1 Nov 2012 07:35:07 +0100 (MET)
X-Virus-Scanned: by amavisd-new on smtp.ee.ethz.ch
Received: from smtp.ee.ethz.ch ([127.0.0.1]) by localhost (.ee.ethz.ch [127.0.0.1]) (amavisd-new, port 10024) with LMTP id hijp+zD38rQf; Thu,  1 Nov 2012 07:35:06 +0100 (MET)
Received: from [10.0.27.100] (cust-integra-122-165.antanet.ch [80.75.122.165]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: briant) by smtp.ee.ethz.ch (Postfix) with ESMTPSA id B8140D9305; Thu,  1 Nov 2012 07:35:06 +0100 (MET)
Mime-Version: 1.0 (Apple Message framework v1283)
Content-Type: text/plain; charset=iso-8859-1
From: Brian Trammell 
In-Reply-To: <5091422B.9070607@cisco.com>
Date: Thu, 1 Nov 2012 07:35:05 +0100
Content-Transfer-Encoding: quoted-printable
Message-Id: <48C58B9E-EE52-415D-896F-AA15F8B7A6C4@tik.ee.ethz.ch>
References: <506CBFE3.10607@auckland.ac.nz> <5090547C.5020803@cisco.com>  <5091271E.3050206@cisco.com> <68907E1B-1D38-4F3A-B0E0-F47628F989F0@tik.ee.ethz.ch> <50913BF3.2080408@cisco.com> <8ED0D683-536C-46B6-8E5A-3CC3B7CB678F@tik.ee.ethz.ch> <5091422B.9070607@cisco.com>
To: Paul Aitken 
X-Mailer: Apple Mail (2.1283)
Cc: Nevil Brownlee , IPFIX Working Group 
Subject: Re: [IPFIX] WG Last Call for draft-ietf-ipfix-information-model-rfc5102bis-05.txt
X-BeenThere: ipfix@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: IPFIX WG discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Thu, 01 Nov 2012 06:35:09 -0000

On Oct 31, 2012, at 4:22 PM, Paul Aitken wrote:

> Brian,
>=20
>> What verb does the MP apply to a unit of information when it gives it =
to the EP?
>>=20
>> A delta counter counts only observations made since the last Flow =
Record for a given Flow was measured.
>>=20
>> Or maybe we can sidestep the action completely:
>>=20
>> A delta counter counts only observations made since the previous Flow =
Record for a given Flow.
>=20
> No, that's still involving export.
>=20
> What should the MP do if the flow ends *but isn't exported* ?

In the idealized architecture, the MP can't know, so it just keeps =
exporting, clearly. If your implementation allows loss between the MP =
and the EP, there is nothing conceptually different between this =
situation and loss between the EP and the CP, except you can't use =
wireshark to debug it. :)

> Just the same as when the flow *is* exported. So the definition of =
deltaCount is independent of export, flow records, etc.
>=20
> So the definitions have to be about the metering time, and =
particularly that we've started metering again. However, we can't write =
that, because some implementations may not hold state that tells them =
this. So all we know is that totalCounters meter from the start of the =
MP, while delta counters meter a potentially shorter interval, reporting =
the value metered since the start of that interval.

I've stared at this for a while and I can't come up with a way to =
express it that's unconvoluted enough for my taste. I still don't see =
why my last attempt at a definition above necessarily invokes export -- =
it's the MP sending on the information in a proto-Flow Record to the EP =
and deciding to start the counters over. So I suppose for the corner =
case that the MP sends something up to the EP and the EP drops it we =
could complicate the language a bit:

A delta counter counts only observations made since the previous Flow =
Record for a given Flow, as seen from the point of view of the Metering =
Process (i.e., discounting any failure or refusal to export the Flow =
Record on the part of the Exporting Process or failure to receive the =
Flow Record on the part of the Collecting Process).

although truth be told I think this overly complicated and unreadable =
for the magnitude of the corner case it addresses. Maybe without the =
i.e. phrase?

Cheers,

Brian=