From majordomo@mil.doit.wisc.edu Mon Sep 8 16:18:01 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA18516 for ; Mon, 8 Sep 2003 16:18:01 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 19wS3c-0006Fl-00 for ipfix-list@mil.doit.wisc.edu; Mon, 08 Sep 2003 14:52:16 -0500 Received: from mailhost2.auckland.ac.nz ([130.216.1.4]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 19wS3a-0006Fc-00 for ipfix@net.doit.wisc.edu; Mon, 08 Sep 2003 14:52:14 -0500 Received: from mailhost.auckland.ac.nz (mailhost [130.216.191.61]) by mailhost2.auckland.ac.nz (8.12.9/8.12.9/8.12.9-ua) with ESMTP id h88JqDs0003058 for ; Tue, 9 Sep 2003 07:52:13 +1200 (NZST) Received: from localhost (mailhost.auckland.ac.nz [127.0.0.1]) by mailhost.auckland.ac.nz (Postfix) with ESMTP id 844D033F68 for ; Tue, 9 Sep 2003 07:50:54 +1200 (NZST) Received: from mailhost.auckland.ac.nz ([127.0.0.1]) by localhost (smtp2.ec.auckland.ac.nz [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 30440-09 for ; Tue, 9 Sep 2003 07:50:54 +1200 (NZST) Received: from hotlava.auckland.ac.nz (hotlava.auckland.ac.nz [130.216.191.123]) by mailhost.auckland.ac.nz (Postfix) with ESMTP id 6B03F33F66 for ; Tue, 9 Sep 2003 07:50:54 +1200 (NZST) Received: (from apache@localhost) by hotlava.auckland.ac.nz (8.11.6/8.11.6) id h88JqCJ02443 for ipfix@net.doit.wisc.edu; Tue, 9 Sep 2003 07:52:12 +1200 X-Authentication-Warning: hotlava.auckland.ac.nz: apache set sender to n.brownlee@auckland.ac.nz using -f Received: from dyn54.caida.org (dyn54.caida.org [192.172.226.54]) by hotlava.auckland.ac.nz (Horde) with HTTP for ; Tue, 9 Sep 2003 07:52:12 +1200 Message-ID: <1063050732.e897d5f106ae4@hotlava.auckland.ac.nz> Date: Tue, 9 Sep 2003 07:52:12 +1200 From: Nevil Brownlee To: ipfix@net.doit.wisc.edu Subject: [ipfix] Default IPFIX transport: time for discussion! MIME-Version: 1.0 Content-Type: text/plain; charset="ISO-8859-1" Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Internet Messaging Program (IMP) 4.0-cvs X-Originating-IP: 192.172.226.54 X-Virus-Scanned: by amavisd-new at mailhost.auckland.ac.nz Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Hello all: About a month ago I posted a list of issues I want to see some IPFIX consensus on. There were one or two responses (thanks to the people who posted those), but no real discussion. Now that everyone is back from summer holidays, let's get back to work on IPFIX - I'd like to see revisions of all the drafts this month, so that we can produce near-complete versions by the late-October deadline for IETF 58. So ... I think we need to decide on a default (MUST implement) transport protocol for IPFIX, so as to guarantee interoperability. The question is, which one? The candidates available now are SCTP and TCP, others could emerge in the next few years. The Protocol draft has sections spelling out how IPFIX could be implemented using SCTP, with technical arguments for that. First, would someone please send in some text spelling out how IPFIX could be implemented using TCP, along with the technical arguments for that. We need that to fill in the blanks in the draft! Second, I realise that it will be hard to choose between them, but lets at least start be trying to set out the technology reasons for each one. Notice that having a default protocol doesn't mean that implementors would only use that one. Any vendor who felt that the non-default protocol provided performance advantages to its users would of course be free to implement both and encourage users to pick the one which best suited their user network environment. Last, if you think that we need only specify a list of IPFIX protocols, lets hear your argument supporting that view. Cheers, Nevil ----------------------------------------------------------------------- Nevil Brownlee Director, Technology Development Phone: +64 9 373 7599 x88941 ITSS, The University of Auckland FAX: +64 9 373 7021 Private Bag 92019, Auckland, New Zealand ------------------------------------------------- This mail sent through University of Auckland http://www.auckland.ac.nz/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Sep 23 13:00:08 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA24365 for ; Tue, 23 Sep 2003 13:00:08 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 1A1q99-0004Yy-00 for ipfix-list@mil.doit.wisc.edu; Tue, 23 Sep 2003 11:36:15 -0500 Received: from psg.com ([147.28.0.62]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 1A1q98-0004Yq-00 for ipfix@net.doit.wisc.edu; Tue, 23 Sep 2003 11:36:14 -0500 Received: from [127.0.0.1] (helo=roam.psg.com) by psg.com with esmtp (Exim 4.22) id 1A1q96-000M9S-Nm; Tue, 23 Sep 2003 16:36:12 +0000 Received: from localhost ([127.0.0.1] helo=roam.psg.com) by roam.psg.com with esmtp (Exim 4.22) id 1A1q95-0006De-9c; Tue, 23 Sep 2003 09:36:11 -0700 From: Randy Bush MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Date: Tue, 23 Sep 2003 09:36:10 -0700 To: grow@lists.uoregon.edu, ipfix@net.doit.wisc.edu, mboned@network-services.uoregon.edu, dnsop@cafax.se Cc: Bert Wijnen Subject: [ipfix] FW: "Last Call for Input to the 57th IETF Meeting Proceedings" Message-Id: Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit [ thanks for the nudge, bert ] GROW, IPFIX, MBONED and DNSOP have not yet submitted minutes. randy --- From: Rebecca Bunch [mailto:rbunch@foretec.com] Sent: dinsdag 23 september 2003 16:45 To: wgchairs@ietf.org; bofchairs@ietf.org Cc: harald@alvestrand.no; Barbara B. Fuller; iesg@ietf.org Subject: "Last Call for Input to the 57th IETF Meeting Proceedings" ***This message is for all Working Groups and BoFs that met at IETF 57 in Vienna.*** Dear Working Group and BoF Chairs: The *final* deadline for submission of minutes and presentation slides for inclusion in the proceedings of the 57th IETF is Thursday, September 25, 2003 at 15:00 ET. If you wish your materials to be included in the proceedings, then please send them to minutes@ietf.org by that time. Any materials received after that time will not be included. As of today, 18 groups have not yet submitted any materials, 10 have submitted minutes but no slides, and 6 have submitted slides but no minutes. The current status of the proceedings, including the list of groups in each category, is available at: www.ietf.org/proceedings/03jul/index.html If you have already submitted materials, we encourage you to review them on line and send any corrections to minutes@ietf.org. Thank you in advance for your cooperation. The IETF Secretariat Rebecca F. Bunch Foretec Seminars, Inc. 1895 Preston White Dr. Suite 100 Reston, VA 20191 703-620-9053, x5365 703-620-9071 -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Mon Sep 29 21:23:51 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA20440 for ; Mon, 29 Sep 2003 21:23:50 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 1A48rv-0001rP-00 for ipfix-list@mil.doit.wisc.edu; Mon, 29 Sep 2003 19:59:59 -0500 Received: from sj-iport-2-in.cisco.com ([171.71.176.71] helo=sj-iport-2.cisco.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 1A48rv-0001rK-00 for ipfix@net.doit.wisc.edu; Mon, 29 Sep 2003 19:59:59 -0500 Received: from cisco.com (171.71.177.237) by sj-iport-2.cisco.com with ESMTP; 29 Sep 2003 17:59:18 -0700 Received: from ios-xdm4.cisco.com (ios-xdm4.cisco.com [171.70.69.142]) by sj-core-1.cisco.com (8.12.9/8.12.6) with ESMTP id h8U0xuwn009165 for ; Mon, 29 Sep 2003 17:59:57 -0700 (PDT) Received: from cisco.com (dhcp-171-71-204-142.cisco.com [171.71.204.142]) by ios-xdm4.cisco.com (8.8.8-Cisco List Logging/CISCO.WS.1.2) with ESMTP id RAA09816 for ; Mon, 29 Sep 2003 17:59:56 -0700 (PDT) Message-ID: <3F78D58C.7050708@cisco.com> Date: Mon, 29 Sep 2003 17:59:56 -0700 From: Ganesh Sadasivan Organization: Cisco Systems Inc User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.1) Gecko/20020823 X-Accept-Language: en-us, en MIME-Version: 1.0 To: ipfix@net.doit.wisc.edu Subject: [ipfix] Variant Field Types. Content-Type: multipart/alternative; boundary="------------010800020807040500090306" Precedence: bulk Sender: majordomo listserver --------------010800020807040500090306 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Hi, The information model only has types for fixed length objects and variable length string like objects. There are a number of information element types (listed below) that would be better defined in terms of fixed-size run-time specified objects. The type variation comes due to difference in the size of the objects used because of an implementation, as demanded by the network or otherwise. The way to make this work is to use the template length field to determine the exact sub-type: The definitions of a variant field type can be stated as: type varUnsignedInt if (element.length == 1) type = unsignedByte else if (element.length <= 2) type = unsignedShort else if (element.length <= 4) type = unsignedInt else if (element.length <= 8) type = unsignedLong else type = undefined type varInt if (element.length == 1) type = byte else if (element.length <= 2) type = short else if (element.length <= 4) type = int else if (element.length <= 8) type = long else type = undefined The candidates that fit into the description above are (I may have missed some also): - ingressPort - egressPort - packetCount - byteCount - sourceAS - destinationAS - nextHopAS - droppedPacketCount - samplingInterval - droppedByteCount Please comment. Thanks Ganesh --------------010800020807040500090306 Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7bit 
Hi,

The information model <draft-ietf-ipfix-info-01.txt> only
has types for fixed length objects and variable length 
string like objects.

There are a number of information element types (listed below)
that would be better defined in terms of fixed-size run-time
specified objects. The type variation comes due to difference
in the size of the objects used because of an implementation,
as demanded by the network or otherwise.

The way to make this work is to use the template
length field to determine the exact sub-type:

The definitions of a variant field type can be stated as:

type varUnsignedInt
    if (element.length == 1)
         type = unsignedByte
    else if (element.length <= 2)
         type = unsignedShort
    else if (element.length <= 4)
         type = unsignedInt
    else if (element.length <= 8)
         type = unsignedLong
   else type = undefined

type varInt
    if (element.length == 1)
         type = byte
    else if (element.length <= 2)
         type = short
    else if (element.length <= 4)
         type = int
    else if (element.length <= 8)
         type = long
   else type = undefined

The candidates that fit into the description above are (I may
have missed some also):
- ingressPort 
- egressPort 
- packetCount 
- byteCount 
- sourceAS 
- destinationAS
- nextHopAS 
- droppedPacketCount
- samplingInterval
- droppedByteCount

Please comment.

Thanks
Ganesh
--------------010800020807040500090306-- -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Sep 30 13:00:40 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA02004 for ; Tue, 30 Sep 2003 13:00:40 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 1A4NVm-0003Tx-00 for ipfix-list@mil.doit.wisc.edu; Tue, 30 Sep 2003 11:38:06 -0500 Received: from atlrel8.hp.com ([156.153.255.206]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 1A4NVk-0003Tn-00 for ipfix@net.doit.wisc.edu; Tue, 30 Sep 2003 11:38:05 -0500 Received: from xatlrelay1.atl.hp.com (xatlrelay1.atl.hp.com [15.45.89.190]) by atlrel8.hp.com (Postfix) with ESMTP id 4670C1C0229F for ; Tue, 30 Sep 2003 12:38:04 -0400 (EDT) Received: from xatlbh2.atl.hp.com (xatlbh2.atl.hp.com [15.45.89.187]) by xatlrelay1.atl.hp.com (Postfix) with ESMTP id 3F8541C000A3 for ; Tue, 30 Sep 2003 12:38:04 -0400 (EDT) Received: by xatlbh2.atl.hp.com with Internet Mail Service (5.5.2655.55) id ; Tue, 30 Sep 2003 12:38:03 -0400 Message-ID: <1D3D2C371FCBD947A7897FABBD3533A502960330@xsun01.ptp.hp.com> From: "MEYER,JEFFREY D (HP-Cupertino,ex1)" To: ipfix@net.doit.wisc.edu Subject: RE: [ipfix] Variant Field Types. Date: Tue, 30 Sep 2003 12:37:52 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2655.55) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C38771.314D4E70" Precedence: bulk Sender: majordomo listserver This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C38771.314D4E70 Content-Type: text/plain; charset="iso-8859-1" Hi, Although the use of variant types may seem convenient from the perspective of an exporter, I think the various consumers of the data would benefit from sticking with an explicit type system. Consider operations which involve summing fields, storing them to a DB, etc. Explicit typing ensures that these consuming applications can handle these values appropriately and also makes the information modeller think about how these types are to be used. The examples you cite: - ingressPort - egressPort - packetCount - byteCount - sourceAS - destinationAS - nextHopAS - droppedPacketCount - samplingInterval - droppedByteCount fall into a few categories, identifiers (such as ports and AS's), counts and intervals. For the identifiers, I believe the size is fixed by the relevant RFC's (i.e. why would I have a 1 byte Port?). For counts, the use of 32 vs. 64 bits is a valid question, however these can be handled by making them explicit (i.e. the information model says if I get this value it is a 32-bit quantity or a 64-bit quantity and the code will address it accordingly). For intervals, the resolution would determine which size is most appropriate. The option always exist to define variations on these data items explicitly. E.g. byteCount32 and byteCount64. If there is a need to vary the sizes for various fields I'd much rather see the rationale for each discrete sizing made explicit vs. forcing all the downstream consumers of this \ information have to deal with special cases and ambiguity. Regards, Jeff Meyer -----Original Message----- From: Ganesh Sadasivan [mailto:gsadasiv@cisco.com] Sent: Monday, September 29, 2003 6:00 PM To: ipfix@net.doit.wisc.edu Subject: [ipfix] Variant Field Types. Hi, The information model only has types for fixed length objects and variable length string like objects. There are a number of information element types (listed below) that would be better defined in terms of fixed-size run-time specified objects. The type variation comes due to difference in the size of the objects used because of an implementation, as demanded by the network or otherwise. The way to make this work is to use the template length field to determine the exact sub-type: The definitions of a variant field type can be stated as: type varUnsignedInt if (element.length == 1) type = unsignedByte else if (element.length <= 2) type = unsignedShort else if (element.length <= 4) type = unsignedInt else if (element.length <= 8) type = unsignedLong else type = undefined type varInt if (element.length == 1) type = byte else if (element.length <= 2) type = short else if (element.length <= 4) type = int else if (element.length <= 8) type = long else type = undefined The candidates that fit into the description above are (I may have missed some also): - ingressPort - egressPort - packetCount - byteCount - sourceAS - destinationAS - nextHopAS - droppedPacketCount - samplingInterval - droppedByteCount Please comment. Thanks Ganesh ------_=_NextPart_001_01C38771.314D4E70 Content-Type: text/html; charset="iso-8859-1"
Hi,
 
  Although the use of variant types may seem convenient from the perspective of an
exporter, I think the various consumers of the data would benefit from sticking with
an explicit type system.
 
  Consider operations which involve summing fields, storing them to a DB, etc.
 
  Explicit typing ensures that these consuming applications can handle these values
appropriately and also makes the information modeller think about how these
types are to be used.
 
  The examples you cite:
 
- ingressPort
- egressPort
- packetCount
- byteCount
- sourceAS
- destinationAS
- nextHopAS
- droppedPacketCount
- samplingInterval
- droppedByteCount
 
  fall into a few categories, identifiers (such as ports and AS's), counts and intervals.
 
  For the identifiers, I believe the size is fixed by the relevant RFC's (i.e. why would I
have a 1 byte Port?).  For counts, the use of 32 vs. 64 bits is a valid question, however
these can be handled by making them explicit (i.e. the information model says if
I get this value it is a 32-bit quantity or a 64-bit quantity and the code will address
it accordingly).  For intervals, the resolution would determine which size is most
appropriate.
 
  The option always exist to define variations on these data items explicitly.  E.g.
byteCount32 and byteCount64.
 
  If there is a need to vary the sizes for various fields I'd much rather see the rationale
for each discrete sizing made explicit vs. forcing all the downstream consumers of this \
information have to deal with special cases and ambiguity.
 
 
Regards,
 
  Jeff Meyer
 
-----Original Message-----
From: Ganesh Sadasivan [mailto:gsadasiv@cisco.com]
Sent: Monday, September 29, 2003 6:00 PM
To: ipfix@net.doit.wisc.edu
Subject: [ipfix] Variant Field Types.

Hi,

The information model <draft-ietf-ipfix-info-01.txt> only
has types for fixed length objects and variable length 
string like objects.

There are a number of information element types (listed below)
that would be better defined in terms of fixed-size run-time
specified objects. The type variation comes due to difference
in the size of the objects used because of an implementation,
as demanded by the network or otherwise.

The way to make this work is to use the template
length field to determine the exact sub-type:

The definitions of a variant field type can be stated as:

type varUnsignedInt
    if (element.length == 1)
         type = unsignedByte
    else if (element.length <= 2)
         type = unsignedShort
    else if (element.length <= 4)
         type = unsignedInt
    else if (element.length <= 8)
         type = unsignedLong
   else type = undefined

type varInt
    if (element.length == 1)
         type = byte
    else if (element.length <= 2)
         type = short
    else if (element.length <= 4)
         type = int
    else if (element.length <= 8)
         type = long
   else type = undefined

The candidates that fit into the description above are (I may
have missed some also):
- ingressPort 
- egressPort 
- packetCount 
- byteCount 
- sourceAS 
- destinationAS
- nextHopAS 
- droppedPacketCount
- samplingInterval
- droppedByteCount

Please comment.

Thanks
Ganesh
------_=_NextPart_001_01C38771.314D4E70-- -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Sep 30 13:28:52 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA03365 for ; Tue, 30 Sep 2003 13:28:52 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 1A4Nx6-0004TN-00 for ipfix-list@mil.doit.wisc.edu; Tue, 30 Sep 2003 12:06:20 -0500 Received: from ams-iport-1.cisco.com ([144.254.74.5]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 1A4Nx5-0004TF-00 for ipfix@net.doit.wisc.edu; Tue, 30 Sep 2003 12:06:19 -0500 Received: from cisco.com (144.254.74.60) by ams-iport-1.cisco.com with ESMTP; 30 Sep 2003 19:04:46 +0200 Received: from cisco.com (localhost [127.0.0.1]) by ams-msg-core-1.cisco.com (8.12.2/8.12.6) with ESMTP id h8UH6EGT018165; Tue, 30 Sep 2003 19:06:14 +0200 (MET DST) Received: from cisco.com (dhcp-rea-gp250-64-103-65-151.cisco.com [64.103.65.151]) by cisco.com (8.8.8/2.6/Cisco List Logging/8.8.8) with ESMTP id SAA06104; Tue, 30 Sep 2003 18:06:17 +0100 (BST) Message-ID: <3F79B808.3050804@cisco.com> Date: Tue, 30 Sep 2003 18:06:16 +0100 From: Stewart Bryant Reply-To: stbryant@cisco.com Organization: Cisco Systems, Inc. User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 Netscape/7.1 (ax) X-Accept-Language: en-us, en MIME-Version: 1.0 To: "MEYER,JEFFREY D (HP-Cupertino,ex1)" CC: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Variant Field Types. References: <1D3D2C371FCBD947A7897FABBD3533A502960330@xsun01.ptp.hp.com> In-Reply-To: <1D3D2C371FCBD947A7897FABBD3533A502960330@xsun01.ptp.hp.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit MEYER,JEFFREY D (HP-Cupertino,ex1) wrote: > Hi, > > Although the use of variant types may seem convenient from the > perspective of an > exporter, I think the various consumers of the data would benefit from > sticking with > an explicit type system. > Variant types allow better use of resources and the ability to upgrade fields without the need to firstly get IETF approval, and secondly synchronise deployment of the exporter and collector upgrade. Anything that counts is a good example. To cope with the high end systems we really need to make them all 64bits. That means that tiny system that are memory and export limited and which may never wrap a 16 or 32 bit counter need to devote resources to 64 bits. If we were designing this say 5 years ago we would probably not have considered 32 bit AS numbers, now we need to. There are bound to be other IE that need to be upgraded, and for each and every one we need to synchronise both exporter and collector. By introducing this flexibility we get greater scope for how we handle these upgrades. > Consider operations which involve summing fields, storing them to a > DB, etc. > The collector storage is less likely to be the long pole in the tent than the exporter or the transmission bandwidth to the collector. The collector can therefore more readily be implemented to accept the largest reasonable value than can the exporter. > Explicit typing ensures that these consuming applications can handle > these values > appropriately and also makes the information modeller think about how these > types are to be used. But they have to think worst case during deployment, and my contention is that this places undue stress on the exporter and on the network bandwidth. > > The examples you cite: > > - ingressPort > - egressPort > - packetCount > - byteCount > - sourceAS > - destinationAS > - nextHopAS > - droppedPacketCount > - samplingInterval > - droppedByteCount > > fall into a few categories, identifiers (such as ports and AS's), > counts and intervals. > > For the identifiers, I believe the size is fixed by the relevant RFC's > (i.e. why would I > have a 1 byte Port?). Clearly but not long ago you would have said "why would I have a 4 byte AS number" Also you are thinking in terms of the "examples" we have in the current RFCs. We need to design this both for future RFCs and for vendor specific IEs. > For counts, the use of 32 vs. 64 bits is a valid > question, however > these can be handled by making them explicit (i.e. the information model > says if > I get this value it is a 32-bit quantity or a 64-bit quantity and the > code will address > it accordingly). For intervals, the resolution would determine which > size is most > appropriate. But this then makes it harder for tiny systems or cramps large systems. > > The option always exist to define variations on these data items > explicitly. E.g. > byteCount32 and byteCount64. Thay leads to two (or more) varients that the collector has to support. The alternative is for the collector to support the larger, and the exporter to support what ever size it wants. An alternative way to think of this is as a data compression technology. > > If there is a need to vary the sizes for various fields I'd much > rather see the rationale > for each discrete sizing made explicit vs. forcing all the downstream > consumers of this \ > information have to deal with special cases and ambiguity. > That requires too much (unnecessary) synchronisation between exporter and collector. Regards Stewart > > Regards, > > Jeff Meyer > > > > -----Original Message----- > From: Ganesh Sadasivan [mailto:gsadasiv@cisco.com] > Sent: Monday, September 29, 2003 6:00 PM > To: ipfix@net.doit.wisc.edu > Subject: [ipfix] Variant Field Types. > > Hi, > > The information model only > has types for fixed length objects and variable length > string like objects. > > There are a number of information element types (listed below) > that would be better defined in terms of fixed-size run-time > specified objects. The type variation comes due to difference > in the size of the objects used because of an implementation, > as demanded by the network or otherwise. > > The way to make this work is to use the template > length field to determine the exact sub-type: > > The definitions of a variant field type can be stated as: > > type varUnsignedInt > if (element.length == 1) > type = unsignedByte > else if (element.length <= 2) > type = unsignedShort > else if (element.length <= 4) > type = unsignedInt > else if (element.length <= 8) > type = unsignedLong > else type = undefined > > type varInt > if (element.length == 1) > type = byte > else if (element.length <= 2) > type = short > else if (element.length <= 4) > type = int > else if (element.length <= 8) > type = long > else type = undefined > > The candidates that fit into the description above are (I may > have missed some also): > - ingressPort > - egressPort > - packetCount > - byteCount > - sourceAS > - destinationAS > - nextHopAS > - droppedPacketCount > - samplingInterval > - droppedByteCount > > Please comment. > > Thanks > Ganesh -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Sep 30 13:44:40 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA04034 for ; Tue, 30 Sep 2003 13:44:39 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 1A4OBe-0004xz-00 for ipfix-list@mil.doit.wisc.edu; Tue, 30 Sep 2003 12:21:22 -0500 Received: from usmail.xacct.com ([204.253.100.12]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 1A4OBc-0004xq-00 for ipfix@net.doit.wisc.edu; Tue, 30 Sep 2003 12:21:20 -0500 Received: from Givoly (inside.us.xacct.com [204.253.100.102]) by usmail.xacct.com (8.11.2/8.11.2) with SMTP id h8UHUSC22262 for ; Tue, 30 Sep 2003 10:30:30 -0700 From: "Tal Givoly" To: Subject: RE: [ipfix] Variant Field Types. Date: Tue, 30 Sep 2003 10:21:12 -0700 Message-ID: MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_038A_01C3873C.926620E0" X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 In-Reply-To: <1D3D2C371FCBD947A7897FABBD3533A502960330@xsun01.ptp.hp.com> Precedence: bulk Sender: majordomo listserver This is a multi-part message in MIME format. ------=_NextPart_000_038A_01C3873C.926620E0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Ganesh, Jeff makes very good points IMHO. Having the complexity of arbitrary length should have strong justification. In particular, for all numeric fields that have fixed length (e.g. identifiers), those should be preferred rather than having variable length. Tal -----Original Message----- From: majordomo listserver [mailto:majordomo@mil.doit.wisc.edu]On Behalf Of MEYER,JEFFREY D (HP-Cupertino,ex1) Sent: Tuesday, September 30, 2003 9:38 AM To: ipfix@net.doit.wisc.edu Subject: RE: [ipfix] Variant Field Types. Hi, Although the use of variant types may seem convenient from the perspective of an exporter, I think the various consumers of the data would benefit from sticking with an explicit type system. Consider operations which involve summing fields, storing them to a DB, etc. Explicit typing ensures that these consuming applications can handle these values appropriately and also makes the information modeller think about how these types are to be used. The examples you cite: - ingressPort - egressPort - packetCount - byteCount - sourceAS - destinationAS - nextHopAS - droppedPacketCount - samplingInterval - droppedByteCount fall into a few categories, identifiers (such as ports and AS's), counts and intervals. For the identifiers, I believe the size is fixed by the relevant RFC's (i.e. why would I have a 1 byte Port?). For counts, the use of 32 vs. 64 bits is a valid question, however these can be handled by making them explicit (i.e. the information model says if I get this value it is a 32-bit quantity or a 64-bit quantity and the code will address it accordingly). For intervals, the resolution would determine which size is most appropriate. The option always exist to define variations on these data items explicitly. E.g. byteCount32 and byteCount64. If there is a need to vary the sizes for various fields I'd much rather see the rationale for each discrete sizing made explicit vs. forcing all the downstream consumers of this \ information have to deal with special cases and ambiguity. Regards, Jeff Meyer -----Original Message----- From: Ganesh Sadasivan [mailto:gsadasiv@cisco.com] Sent: Monday, September 29, 2003 6:00 PM To: ipfix@net.doit.wisc.edu Subject: [ipfix] Variant Field Types. Hi, The information model only has types for fixed length objects and variable length string like objects. There are a number of information element types (listed below) that would be better defined in terms of fixed-size run-time specified objects. The type variation comes due to difference in the size of the objects used because of an implementation, as demanded by the network or otherwise. The way to make this work is to use the template length field to determine the exact sub-type: The definitions of a variant field type can be stated as: type varUnsignedInt if (element.length == 1) type = unsignedByte else if (element.length <= 2) type = unsignedShort else if (element.length <= 4) type = unsignedInt else if (element.length <= 8) type = unsignedLong else type = undefined type varInt if (element.length == 1) type = byte else if (element.length <= 2) type = short else if (element.length <= 4) type = int else if (element.length <= 8) type = long else type = undefined The candidates that fit into the description above are (I may have missed some also): - ingressPort - egressPort - packetCount - byteCount - sourceAS - destinationAS - nextHopAS - droppedPacketCount - samplingInterval - droppedByteCount Please comment. Thanks Ganesh ------=_NextPart_000_038A_01C3873C.926620E0 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
Ganesh,
 
Jeff=20 makes very good points IMHO. Having the complexity of arbitrary length = should=20 have strong justification. In particular, for all numeric fields that = have fixed=20 length (e.g. identifiers), those should be preferred rather than having = variable=20 length.
 
Tal
-----Original Message-----
From: majordomo listserver = [mailto:majordomo@mil.doit.wisc.edu]On Behalf Of MEYER,JEFFREY D=20 (HP-Cupertino,ex1)
Sent: Tuesday, September 30, 2003 9:38=20 AM
To: ipfix@net.doit.wisc.edu
Subject: RE: [ipfix] = Variant=20 Field Types.

Hi,
 
  Although the use of variant types may seem convenient = from the=20 perspective of an
exporter, I think the various consumers of the data would = benefit from=20 sticking with
an=20 explicit type system.
 
  Consider operations which involve summing fields, = storing them=20 to a DB, etc.
 
  Explicit typing ensures that these consuming = applications can=20 handle these values
appropriately and also makes the information modeller think = about how=20 these
types are to be used.
 
  The examples you cite:
 
- ingressPort =
-=20 egressPort
- packetCount
- byteCount
- sourceAS
-=20 destinationAS
- nextHopAS
- droppedPacketCount
-=20 samplingInterval
- droppedByteCount
 
  fall into a few categories, identifiers (such as ports = and=20 AS's), counts and intervals.
 
  For the identifiers, I believe the size is fixed by = the relevant=20 RFC's (i.e. why would I
have a 1 byte Port?).  For counts, the use of 32 vs. 64 = bits is a=20 valid question, however
these can be handled by making them explicit (i.e. the = information=20 model says if
I=20 get this value it is a 32-bit quantity or a 64-bit quantity and the = code will=20 address
it=20 accordingly).  For intervals, the resolution would determine = which size=20 is most
appropriate.
 
  The option always exist to define variations on these = data items=20 explicitly.  E.g.
byteCount32 and byteCount64.
 
  If there is a need to vary the sizes for various = fields I'd much=20 rather see the rationale
for=20 each discrete sizing made explicit vs. forcing all the downstream = consumers of=20 this \
information have to deal with special cases = and=20 ambiguity.
 
 
Regards,
 
  Jeff Meyer

=
-----Original Message-----
From: Ganesh Sadasivan = [mailto:gsadasiv@cisco.com]
Sent: Monday, September 29, = 2003 6:00=20 PM
To: ipfix@net.doit.wisc.edu
Subject: [ipfix] = Variant=20 Field Types.

Hi,

The information model <draft-ietf-ipfix-info-01.txt> only
has types for fixed length objects =
and variable length=20
string like objects.

There are a number of information =
element types (listed below)
that would be better defined in terms of fixed-size run-time
specified objects. The type =
variation comes due to difference
in the size of the objects used because of an implementation,
as demanded by the network or otherwise.

The way to make this work is to =
use the template
length field to determine the =
exact sub-type:

The definitions of a variant field type can be stated as:

type varUnsignedInt
    if (element.length =3D=3D 1)
         type =3D unsignedByte
    else if (element.length <=3D 2)
         type =3D unsignedShort
    else if (element.length <=3D 4)
         type =3D unsignedInt
    else if (element.length <=3D 8)
         type =3D unsignedLong
   else type =3D undefined

type varInt
    if (element.length =3D=3D 1)
         type =3D byte
    else if (element.length <=3D 2)
         type =3D short
    else if (element.length <=3D 4)
         type =3D int
    else if (element.length <=3D 8)
         type =3D long
   else type =3D undefined

The candidates that fit into the description above are (I may
have missed some also):
- ingressPort=20
- egressPort=20
- packetCount=20
- byteCount=20
- sourceAS=20
- destinationAS
- nextHopAS=20
- droppedPacketCount
- samplingInterval
- droppedByteCount

Please comment.

Thanks
Ganesh
------=_NextPart_000_038A_01C3873C.926620E0-- -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Sep 30 13:57:33 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA04708 for ; Tue, 30 Sep 2003 13:57:32 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 1A4ONd-0005OI-00 for ipfix-list@mil.doit.wisc.edu; Tue, 30 Sep 2003 12:33:45 -0500 Received: from atlrel9.hp.com ([156.153.255.214]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 1A4ONc-0005OC-00 for ipfix@net.doit.wisc.edu; Tue, 30 Sep 2003 12:33:44 -0500 Received: from xatlrelay2.atl.hp.com (xatlrelay2.atl.hp.com [15.45.89.191]) by atlrel9.hp.com (Postfix) with ESMTP id 9D7091C01C93; Tue, 30 Sep 2003 13:33:43 -0400 (EDT) Received: from xatlbh2.atl.hp.com (xatlbh2.atl.hp.com [15.45.89.187]) by xatlrelay2.atl.hp.com (Postfix) with ESMTP id 7897D1C0009F; Tue, 30 Sep 2003 13:33:43 -0400 (EDT) Received: by xatlbh2.atl.hp.com with Internet Mail Service (5.5.2655.55) id ; Tue, 30 Sep 2003 13:33:43 -0400 Message-ID: <1D3D2C371FCBD947A7897FABBD3533A502960333@xsun01.ptp.hp.com> From: "MEYER,JEFFREY D (HP-Cupertino,ex1)" To: "'stbryant@cisco.com'" , "MEYER,JEFFREY D (HP-Cupertino,ex1)" Cc: ipfix@net.doit.wisc.edu Subject: RE: [ipfix] Variant Field Types. Date: Tue, 30 Sep 2003 13:33:35 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2655.55) Content-Type: text/plain; charset="iso-8859-1" Precedence: bulk Sender: majordomo listserver Hi, One possible way to address Stewart's concern, which seems primarily around exporters not wanting to fill in 64-bit fields, is to separate the concern of the information model and the encoding. Specifically, since the current protocol specification explicitly states lengths used in templates, rather than the type, an exporter could chose to send quantities in 4 bytes or 2 bytes or whatever. The consumer would know that the quantity was actually targeted at some specified size and process it accordingly. Obviously this only works for "downsizing" data items, not upsizing them. But since the arguments seem around "constrained" systems, then downsizing would appear to be the use case. Changes in things like AS number from 16 to 32-bytes are changes in the information model, I'd prefer to see these reflected with explicit updates. If explicit types are good enough for C, Java, and DB's I imagine they're good enough for IPFIX. -- Jeff > -----Original Message----- > From: Stewart Bryant [mailto:stbryant@cisco.com] > Sent: Tuesday, September 30, 2003 10:06 AM > To: MEYER,JEFFREY D (HP-Cupertino,ex1) > Cc: ipfix@net.doit.wisc.edu > Subject: Re: [ipfix] Variant Field Types. > > > > > MEYER,JEFFREY D (HP-Cupertino,ex1) wrote: > > Hi, > > > > Although the use of variant types may seem convenient from the > > perspective of an > > exporter, I think the various consumers of the data would > benefit from > > sticking with > > an explicit type system. > > > > Variant types allow better use of resources and the ability to upgrade > fields without the need to firstly get IETF approval, and secondly > synchronise deployment of the exporter and collector upgrade. > > Anything that counts is a good example. To cope with the high end > systems we really need to make them all 64bits. That means that > tiny system that are memory and export limited and which may never > wrap a 16 or 32 bit counter need to devote resources to 64 bits. > > If we were designing this say 5 years ago we would probably not have > considered 32 bit AS numbers, now we need to. There are bound to be > other IE that need to be upgraded, and for each and every one we need > to synchronise both exporter and collector. > > By introducing this flexibility we get greater scope for how we handle > these upgrades. > > > Consider operations which involve summing fields, storing > them to a > > DB, etc. > > > > The collector storage is less likely to be the long pole in the tent > than the exporter or the transmission bandwidth to the collector. The > collector can therefore more readily be implemented to accept the > largest reasonable value than can the exporter. > > > Explicit typing ensures that these consuming applications > can handle > > these values > > appropriately and also makes the information modeller think > about how these > > types are to be used. > > But they have to think worst case during deployment, and my contention > is that this places undue stress on the exporter and on the network > bandwidth. > > > > > The examples you cite: > > > > - ingressPort > > - egressPort > > - packetCount > > - byteCount > > - sourceAS > > - destinationAS > > - nextHopAS > > - droppedPacketCount > > - samplingInterval > > - droppedByteCount > > > > fall into a few categories, identifiers (such as ports and AS's), > > counts and intervals. > > > > For the identifiers, I believe the size is fixed by the > relevant RFC's > > (i.e. why would I > > have a 1 byte Port?). > > Clearly but not long ago you would have said "why would I > have a 4 byte > AS number" > > Also you are thinking in terms of the "examples" we have in > the current > RFCs. We need to design this both for future RFCs and for > vendor specific > IEs. > > > For counts, the use of 32 vs. 64 bits is a valid > > question, however > > these can be handled by making them explicit (i.e. the > information model > > says if > > I get this value it is a 32-bit quantity or a 64-bit > quantity and the > > code will address > > it accordingly). For intervals, the resolution would > determine which > > size is most > > appropriate. > > But this then makes it harder for tiny systems or cramps > large systems. > > > > > The option always exist to define variations on these data items > > explicitly. E.g. > > byteCount32 and byteCount64. > > Thay leads to two (or more) varients that the collector has > to support. > The alternative is for the collector to support the larger, > and the exporter > to support what ever size it wants. An alternative way to > think of this > is as a data compression technology. > > > > > If there is a need to vary the sizes for various fields I'd much > > rather see the rationale > > for each discrete sizing made explicit vs. forcing all the > downstream > > consumers of this \ > > information have to deal with special cases and ambiguity. > > > > That requires too much (unnecessary) synchronisation between > exporter and collector. > > Regards > > Stewart > > > > > Regards, > > > > Jeff Meyer > > > > > > > > -----Original Message----- > > From: Ganesh Sadasivan [mailto:gsadasiv@cisco.com] > > Sent: Monday, September 29, 2003 6:00 PM > > To: ipfix@net.doit.wisc.edu > > Subject: [ipfix] Variant Field Types. > > > > Hi, > > > > The information model only > > has types for fixed length objects and variable length > > string like objects. > > > > There are a number of information element types (listed below) > > that would be better defined in terms of fixed-size run-time > > specified objects. The type variation comes due to difference > > in the size of the objects used because of an implementation, > > as demanded by the network or otherwise. > > > > The way to make this work is to use the template > > length field to determine the exact sub-type: > > > > The definitions of a variant field type can be stated as: > > > > type varUnsignedInt > > if (element.length == 1) > > type = unsignedByte > > else if (element.length <= 2) > > type = unsignedShort > > else if (element.length <= 4) > > type = unsignedInt > > else if (element.length <= 8) > > type = unsignedLong > > else type = undefined > > > > type varInt > > if (element.length == 1) > > type = byte > > else if (element.length <= 2) > > type = short > > else if (element.length <= 4) > > type = int > > else if (element.length <= 8) > > type = long > > else type = undefined > > > > The candidates that fit into the description above are (I may > > have missed some also): > > - ingressPort > > - egressPort > > - packetCount > > - byteCount > > - sourceAS > > - destinationAS > > - nextHopAS > > - droppedPacketCount > > - samplingInterval > > - droppedByteCount > > > > Please comment. > > > > Thanks > > Ganesh > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Sep 30 14:01:22 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA04842 for ; Tue, 30 Sep 2003 14:01:22 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 1A4OY7-0005p7-00 for ipfix-list@mil.doit.wisc.edu; Tue, 30 Sep 2003 12:44:35 -0500 Received: from sj-iport-2-in.cisco.com ([171.71.176.71] helo=sj-iport-2.cisco.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 1A4OY5-0005p1-00 for ipfix@net.doit.wisc.edu; Tue, 30 Sep 2003 12:44:33 -0500 Received: from cisco.com (171.71.177.238) by sj-iport-2.cisco.com with ESMTP; 30 Sep 2003 10:44:05 -0700 Received: from ios-xdm4.cisco.com (ios-xdm4.cisco.com [171.70.69.142]) by sj-core-5.cisco.com (8.12.9/8.12.6) with ESMTP id h8UHiVXg011733; Tue, 30 Sep 2003 10:44:31 -0700 (PDT) Received: from cisco.com (dhcp-171-71-204-184.cisco.com [171.71.204.184]) by ios-xdm4.cisco.com (8.8.8-Cisco List Logging/CISCO.WS.1.2) with ESMTP id KAA11123; Tue, 30 Sep 2003 10:44:31 -0700 (PDT) Message-ID: <3F79C0FE.7020407@cisco.com> Date: Tue, 30 Sep 2003 10:44:30 -0700 From: Ganesh Sadasivan Organization: Cisco Systems Inc User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.1) Gecko/20020823 X-Accept-Language: en-us, en MIME-Version: 1.0 To: "MEYER,JEFFREY D (HP-Cupertino,ex1)" CC: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Variant Field Types. References: <1D3D2C371FCBD947A7897FABBD3533A502960330@xsun01.ptp.hp.com> Content-Type: multipart/alternative; boundary="------------070907050102000509070509" Precedence: bulk Sender: majordomo listserver --------------070907050102000509070509 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit MEYER,JEFFREY D (HP-Cupertino,ex1) wrote: > Hi, > > Although the use of variant types may seem convenient from the > perspective of an > exporter, I think the various consumers of the data would benefit from > sticking with > an explicit type system. It would benefit the exporter, the n/w, flexibility of interpretation of fields in the information model and the collector. > > Consider operations which involve summing fields, storing them to a > DB, etc. One way to solve this problem is if the collector assumes the largest size for computations? > > Explicit typing ensures that these consuming applications can handle > these values > appropriately and also makes the information modeller think about how > these > types are to be used. > > The examples you cite: > > - ingressPort > - egressPort > - packetCount > - byteCount > - sourceAS > - destinationAS > - nextHopAS > - droppedPacketCount > - samplingInterval > - droppedByteCount > > fall into a few categories, identifiers (such as ports and AS's), > counts and intervals. > > For the identifiers, I believe the size is fixed by the relevant > RFC's (i.e. why would I > have a 1 byte Port?). For counts, the use of 32 vs. 64 bits is a > valid question, however > these can be handled by making them explicit (i.e. the information > model says if > I get this value it is a 32-bit quantity or a 64-bit quantity and the > code will address > it accordingly). For intervals, the resolution would determine which > size is most > appropriate. > For AS the size could be 2 or 4 bytes. You may prefer to use a 1 byte port if you are a small system as against a 4 byte for a gigantic system. Both have their own implementation reasons. > The option always exist to define variations on these data items > explicitly. E.g. > byteCount32 and byteCount64. > Yes. But is'nt this not more of an overhead to introduce new field type as the requirements come in and then make sure that it gets deployed in exporter and collector? > If there is a need to vary the sizes for various fields I'd much > rather see the rationale > for each discrete sizing made explicit vs. forcing all the downstream > consumers of this \ > information have to deal with special cases and ambiguity. > > This is not a special case handling as you can see that the list above covers about 40% of the fields defined today in the information model. It is way to handle ambiguity arising out of implementation differences. Thanks Ganesh > Regards, > > Jeff Meyer > > > > -----Original Message----- > *From:* Ganesh Sadasivan [mailto:gsadasiv@cisco.com] > *Sent:* Monday, September 29, 2003 6:00 PM > *To:* ipfix@net.doit.wisc.edu > *Subject:* [ipfix] Variant Field Types. > >Hi, > >The information model only >has types for fixed length objects and variable length >string like objects. > >There are a number of information element types (listed below) >that would be better defined in terms of fixed-size run-time >specified objects. The type variation comes due to difference >in the size of the objects used because of an implementation, >as demanded by the network or otherwise. > >The way to make this work is to use the template >length field to determine the exact sub-type: > >The definitions of a variant field type can be stated as: > >type varUnsignedInt > if (element.length == 1) > type = unsignedByte > else if (element.length <= 2) > type = unsignedShort > else if (element.length <= 4) > type = unsignedInt > else if (element.length <= 8) > type = unsignedLong > else type = undefined > >type varInt > if (element.length == 1) > type = byte > else if (element.length <= 2) > type = short > else if (element.length <= 4) > type = int > else if (element.length <= 8) > type = long > else type = undefined > >The candidates that fit into the description above are (I may >have missed some also): >- ingressPort >- egressPort >- packetCount >- byteCount >- sourceAS >- destinationAS >- nextHopAS >- droppedPacketCount >- samplingInterval >- droppedByteCount > >Please comment. > >Thanks >Ganesh > > --------------070907050102000509070509 Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7bit

MEYER,JEFFREY D (HP-Cupertino,ex1) wrote:
Hi,
 
  Although the use of variant types may seem convenient from the perspective of an
exporter, I think the various consumers of the data would benefit from sticking with
an explicit type system.
It would benefit the exporter, the n/w, flexibility of interpretation of fields in the
information model  and the collector.
 
  Consider operations which involve summing fields, storing them to a DB, etc.
One way to solve this problem is if the collector assumes the largest size for
computations?
 
  Explicit typing ensures that these consuming applications can handle these values
appropriately and also makes the information modeller think about how these
types are to be used.
 
  The examples you cite:
 
- ingressPort
- egressPort
- packetCount
- byteCount
- sourceAS
- destinationAS
- nextHopAS
- droppedPacketCount
- samplingInterval
- droppedByteCount
 
  fall into a few categories, identifiers (such as ports and AS's), counts and intervals.
 
  For the identifiers, I believe the size is fixed by the relevant RFC's (i.e. why would I
have a 1 byte Port?).  For counts, the use of 32 vs. 64 bits is a valid question, however
these can be handled by making them explicit (i.e. the information model says if
I get this value it is a 32-bit quantity or a 64-bit quantity and the code will address
it accordingly).  For intervals, the resolution would determine which size is most
appropriate.
 
For AS the size could be 2 or 4 bytes.
You may prefer to use a 1 byte port if you are a small system as against a 4 byte for
a gigantic system. Both have their own implementation reasons.
  The option always exist to define variations on these data items explicitly.  E.g.
byteCount32 and byteCount64.
 
Yes. But is'nt this not more of an overhead to introduce new field type as the requirements
come in and then make sure that it gets deployed in exporter and collector?

  If there is a need to vary the sizes for various fields I'd much rather see the rationale
for each discrete sizing made explicit vs. forcing all the downstream consumers of this \
information have to deal with special cases and ambiguity.
 
 
This is not a special case handling as you can see that the list above covers about
40% of the fields defined today in the information model. It is way to handle
ambiguity arising out of implementation differences.

Thanks
Ganesh
Regards,
 
  Jeff Meyer

 
-----Original Message-----
From: Ganesh Sadasivan [mailto:gsadasiv@cisco.com]
Sent: Monday, September 29, 2003 6:00 PM
To: ipfix@net.doit.wisc.edu
Subject: [ipfix] Variant Field Types.

Hi,

The information model <draft-ietf-ipfix-info-01.txt> only
has types for fixed length objects and variable length 
string like objects.

There are a number of information element types (listed below)
that would be better defined in terms of fixed-size run-time
specified objects. The type variation comes due to difference
in the size of the objects used because of an implementation,
as demanded by the network or otherwise.

The way to make this work is to use the template
length field to determine the exact sub-type:

The definitions of a variant field type can be stated as:

type varUnsignedInt
    if (element.length == 1)
         type = unsignedByte
    else if (element.length <= 2)
         type = unsignedShort
    else if (element.length <= 4)
         type = unsignedInt
    else if (element.length <= 8)
         type = unsignedLong
   else type = undefined

type varInt
    if (element.length == 1)
         type = byte
    else if (element.length <= 2)
         type = short
    else if (element.length <= 4)
         type = int
    else if (element.length <= 8)
         type = long
   else type = undefined

The candidates that fit into the description above are (I may
have missed some also):
- ingressPort 
- egressPort 
- packetCount 
- byteCount 
- sourceAS 
- destinationAS
- nextHopAS 
- droppedPacketCount
- samplingInterval
- droppedByteCount

Please comment.

Thanks
Ganesh

--------------070907050102000509070509-- -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Sep 30 16:28:11 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA14575 for ; Tue, 30 Sep 2003 16:28:10 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 1A4Qyp-0003D6-00 for ipfix-list@mil.doit.wisc.edu; Tue, 30 Sep 2003 15:20:19 -0500 Received: from atlrel8.hp.com ([156.153.255.206]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 1A4Qyo-0003Cx-00 for ipfix@net.doit.wisc.edu; Tue, 30 Sep 2003 15:20:18 -0500 Received: from xatlrelay2.atl.hp.com (xatlrelay2.atl.hp.com [15.45.89.191]) by atlrel8.hp.com (Postfix) with ESMTP id B4D251C02434 for ; Tue, 30 Sep 2003 16:20:17 -0400 (EDT) Received: from xatlbh1.atl.hp.com (xatlbh1.atl.hp.com [15.45.89.186]) by xatlrelay2.atl.hp.com (Postfix) with ESMTP id A8C951C000AF for ; Tue, 30 Sep 2003 16:20:17 -0400 (EDT) Received: by xatlbh1.atl.hp.com with Internet Mail Service (5.5.2655.55) id ; Tue, 30 Sep 2003 16:20:17 -0400 Message-ID: <1D3D2C371FCBD947A7897FABBD3533A502960336@xsun01.ptp.hp.com> From: "MEYER,JEFFREY D (HP-Cupertino,ex1)" To: "MEYER,JEFFREY D (HP-Cupertino,ex1)" Cc: ipfix@net.doit.wisc.edu Subject: RE: [ipfix] Variant Field Types. Date: Tue, 30 Sep 2003 16:20:07 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2655.55) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C38790.3DB3CDA0" Precedence: bulk Sender: majordomo listserver This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01C38790.3DB3CDA0 Content-Type: text/plain; charset="iso-8859-1" Ganesh, I'm not clear on how this benefits the collector. Tal and myself produce different implementations of carrier deployed products which collect from 100's of devices and dozens of protocols, perhaps our experience here is worth something, and it tells us variant is undesireable on the collector... Regarding your other point: One way to solve this problem is if the collector assumes the largest size for computations? This was my point in the e-mail to Stewart. Have the information model be EXPLICIT about what is the largest size which is being modeled for a data item and declare it. Allow the exporter to declare a smaller size in the template, if the exporter knows that an individual record will always be accomodated (based on its implementation) in the smaller size. So for instance this might say that byteCount should be declared in the information model as a 64-bit int, while AS is probably sufficient at 32-bits. This accomodates the: exporter - can use smaller size if known n/w - can reduce BW if know a smaller size is used info model - explicit is better than "vague" collector - addresses persistence and aggregation issues Can we call this a compromise? Regards, Jeff Meyer -----Original Message----- From: Ganesh Sadasivan [mailto:gsadasiv@cisco.com] Sent: Tuesday, September 30, 2003 10:45 AM To: MEYER,JEFFREY D (HP-Cupertino,ex1) Cc: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Variant Field Types. MEYER,JEFFREY D (HP-Cupertino,ex1) wrote: Hi, Although the use of variant types may seem convenient from the perspective of an exporter, I think the various consumers of the data would benefit from sticking with an explicit type system. It would benefit the exporter, the n/w, flexibility of interpretation of fields in the information model and the collector. Consider operations which involve summing fields, storing them to a DB, etc. One way to solve this problem is if the collector assumes the largest size for computations? Explicit typing ensures that these consuming applications can handle these values appropriately and also makes the information modeller think about how these types are to be used. The examples you cite: - ingressPort - egressPort - packetCount - byteCount - sourceAS - destinationAS - nextHopAS - droppedPacketCount - samplingInterval - droppedByteCount fall into a few categories, identifiers (such as ports and AS's), counts and intervals. For the identifiers, I believe the size is fixed by the relevant RFC's (i.e. why would I have a 1 byte Port?). For counts, the use of 32 vs. 64 bits is a valid question, however these can be handled by making them explicit (i.e. the information model says if I get this value it is a 32-bit quantity or a 64-bit quantity and the code will address it accordingly). For intervals, the resolution would determine which size is most appropriate. For AS the size could be 2 or 4 bytes. You may prefer to use a 1 byte port if you are a small system as against a 4 byte for a gigantic system. Both have their own implementation reasons. The option always exist to define variations on these data items explicitly. E.g. byteCount32 and byteCount64. Yes. But is'nt this not more of an overhead to introduce new field type as the requirements come in and then make sure that it gets deployed in exporter and collector? If there is a need to vary the sizes for various fields I'd much rather see the rationale for each discrete sizing made explicit vs. forcing all the downstream consumers of this \ information have to deal with special cases and ambiguity. This is not a special case handling as you can see that the list above covers about 40% of the fields defined today in the information model. It is way to handle ambiguity arising out of implementation differences. Thanks Ganesh Regards, Jeff Meyer -----Original Message----- From: Ganesh Sadasivan [ mailto:gsadasiv@cisco.com ] Sent: Monday, September 29, 2003 6:00 PM To: ipfix@net.doit.wisc.edu Subject: [ipfix] Variant Field Types. Hi, The information model only has types for fixed length objects and variable length string like objects. There are a number of information element types (listed below) that would be better defined in terms of fixed-size run-time specified objects. The type variation comes due to difference in the size of the objects used because of an implementation, as demanded by the network or otherwise. The way to make this work is to use the template length field to determine the exact sub-type: The definitions of a variant field type can be stated as: type varUnsignedInt if (element.length == 1) type = unsignedByte else if (element.length <= 2) type = unsignedShort else if (element.length <= 4) type = unsignedInt else if (element.length <= 8) type = unsignedLong else type = undefined type varInt if (element.length == 1) type = byte else if (element.length <= 2) type = short else if (element.length <= 4) type = int else if (element.length <= 8) type = long else type = undefined The candidates that fit into the description above are (I may have missed some also): - ingressPort - egressPort - packetCount - byteCount - sourceAS - destinationAS - nextHopAS - droppedPacketCount - samplingInterval - droppedByteCount Please comment. Thanks Ganesh ------_=_NextPart_001_01C38790.3DB3CDA0 Content-Type: text/html; charset="iso-8859-1"
Ganesh,
 
  I'm not clear on how this benefits the collector.  Tal and myself produce different implementations of carrier deployed products which collect from 100's of devices and dozens of protocols, perhaps our experience here is worth something, and it tells us variant is undesireable on the collector...
 
  Regarding your other point:
 
One way to solve this problem is if the collector assumes the largest size for
computations?
 
  This was my point in the e-mail to Stewart.  Have the information model be EXPLICIT about what is the largest size which is being modeled for a data item and declare it.  Allow the exporter to declare a smaller size in the template, if the exporter knows that an individual record will always be accomodated (based on its implementation) in the smaller size.
 
  So for instance this might say that byteCount should be declared in the information model as a 64-bit int, while AS is probably sufficient at 32-bits.
 
  This accomodates the:
      exporter - can use smaller size if known
      n/w - can reduce BW if know a smaller size is used
      info model - explicit is better than "vague"
      collector - addresses persistence and aggregation issues
 
  Can we call this a compromise?
 
Regards,
 
  Jeff Meyer
-----Original Message-----
From: Ganesh Sadasivan [mailto:gsadasiv@cisco.com]
Sent: Tuesday, September 30, 2003 10:45 AM
To: MEYER,JEFFREY D (HP-Cupertino,ex1)
Cc: ipfix@net.doit.wisc.edu
Subject: Re: [ipfix] Variant Field Types.



MEYER,JEFFREY D (HP-Cupertino,ex1) wrote:
Hi,
 
  Although the use of variant types may seem convenient from the perspective of an
exporter, I think the various consumers of the data would benefit from sticking with
an explicit type system.
It would benefit the exporter, the n/w, flexibility of interpretation of fields in the
information model  and the collector.
 
  Consider operations which involve summing fields, storing them to a DB, etc.
One way to solve this problem is if the collector assumes the largest size for
computations?
 
  Explicit typing ensures that these consuming applications can handle these values
appropriately and also makes the information modeller think about how these
types are to be used.
 
  The examples you cite:
 
- ingressPort
- egressPort
- packetCount
- byteCount
- sourceAS
- destinationAS
- nextHopAS
- droppedPacketCount
- samplingInterval
- droppedByteCount
 
  fall into a few categories, identifiers (such as ports and AS's), counts and intervals.
 
  For the identifiers, I believe the size is fixed by the relevant RFC's (i.e. why would I
have a 1 byte Port?).  For counts, the use of 32 vs. 64 bits is a valid question, however
these can be handled by making them explicit (i.e. the information model says if
I get this value it is a 32-bit quantity or a 64-bit quantity and the code will address
it accordingly).  For intervals, the resolution would determine which size is most
appropriate.
 
For AS the size could be 2 or 4 bytes.
You may prefer to use a 1 byte port if you are a small system as against a 4 byte for
a gigantic system. Both have their own implementation reasons.
  The option always exist to define variations on these data items explicitly.  E.g.
byteCount32 and byteCount64.
 
Yes. But is'nt this not more of an overhead to introduce new field type as the requirements
come in and then make sure that it gets deployed in exporter and collector?

  If there is a need to vary the sizes for various fields I'd much rather see the rationale
for each discrete sizing made explicit vs. forcing all the downstream consumers of this \
information have to deal with special cases and ambiguity.
 
 
This is not a special case handling as you can see that the list above covers about
40% of the fields defined today in the information model. It is way to handle
ambiguity arising out of implementation differences.

Thanks
Ganesh
Regards,
 
  Jeff Meyer

 
-----Original Message-----
From: Ganesh Sadasivan [mailto:gsadasiv@cisco.com]
Sent: Monday, September 29, 2003 6:00 PM
To: ipfix@net.doit.wisc.edu
Subject: [ipfix] Variant Field Types.

Hi,

The information model <draft-ietf-ipfix-info-01.txt> only
has types for fixed length objects and variable length 
string like objects.

There are a number of information element types (listed below)
that would be better defined in terms of fixed-size run-time
specified objects. The type variation comes due to difference
in the size of the objects used because of an implementation,
as demanded by the network or otherwise.

The way to make this work is to use the template
length field to determine the exact sub-type:

The definitions of a variant field type can be stated as:

type varUnsignedInt
    if (element.length == 1)
         type = unsignedByte
    else if (element.length <= 2)
         type = unsignedShort
    else if (element.length <= 4)
         type = unsignedInt
    else if (element.length <= 8)
         type = unsignedLong
   else type = undefined

type varInt
    if (element.length == 1)
         type = byte
    else if (element.length <= 2)
         type = short
    else if (element.length <= 4)
         type = int
    else if (element.length <= 8)
         type = long
   else type = undefined

The candidates that fit into the description above are (I may
have missed some also):
- ingressPort 
- egressPort 
- packetCount 
- byteCount 
- sourceAS 
- destinationAS
- nextHopAS 
- droppedPacketCount
- samplingInterval
- droppedByteCount

Please comment.

Thanks
Ganesh

------_=_NextPart_001_01C38790.3DB3CDA0-- -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Sep 30 18:30:04 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA19431 for ; Tue, 30 Sep 2003 18:30:03 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 1A4SrN-0007DX-00 for ipfix-list@mil.doit.wisc.edu; Tue, 30 Sep 2003 17:20:45 -0500 Received: from mailhost2.auckland.ac.nz ([130.216.191.4]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 1A4SrJ-0007DN-00; Tue, 30 Sep 2003 17:20:41 -0500 Received: from mailhost.auckland.ac.nz (mailhost [130.216.191.61]) by mailhost2.auckland.ac.nz (8.12.9/8.12.9/8.12.9-ua) with ESMTP id h8UMKcEE005089; Wed, 1 Oct 2003 10:20:39 +1200 (NZST) Received: from localhost (mailhost.auckland.ac.nz [127.0.0.1]) by mailhost.auckland.ac.nz (Postfix) with ESMTP id 1E54333EB1; Wed, 1 Oct 2003 10:20:02 +1200 (NZST) Received: from mailhost.auckland.ac.nz ([127.0.0.1]) by localhost (mailhost.auckland.ac.nz [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 13379-12; Wed, 1 Oct 2003 10:20:02 +1200 (NZST) Received: from hotlava.auckland.ac.nz (hotlava.auckland.ac.nz [130.216.191.123]) by mailhost.auckland.ac.nz (Postfix) with ESMTP id 068B533EB0; Wed, 1 Oct 2003 10:20:02 +1200 (NZST) Received: (from apache@localhost) by hotlava.auckland.ac.nz (8.11.6/8.11.6) id h8UMKcU12514; Wed, 1 Oct 2003 10:20:38 +1200 X-Authentication-Warning: hotlava.auckland.ac.nz: apache set sender to n.brownlee@auckland.ac.nz using -f Received: from nebbiolo.itss.auckland.ac.nz (nebbiolo.itss.auckland.ac.nz [130.216.4.167]) by hotlava.auckland.ac.nz (Horde) with HTTP for ; Wed, 1 Oct 2003 10:20:38 +1200 Message-ID: <1064960438.7d6afcd706513@hotlava.auckland.ac.nz> Date: Wed, 1 Oct 2003 10:20:38 +1200 From: Nevil Brownlee To: ipfix-reqs@net.doit.wisc.edu Cc: ipfix-chairs@net.doit.wisc.edu Subject: [ipfix-reqs] Fwd: Re: IPFIX Requirements issues (resend) MIME-Version: 1.0 Content-Type: text/plain; charset="ISO-8859-1" Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Internet Messaging Program (IMP) 4.0-cvs X-Originating-IP: 130.216.4.167 X-Virus-Scanned: by amavisd-new at mailhost.auckland.ac.nz Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: 7bit Hi Juergen et al: Herewith email from Randy, he wants to try to move the IPFIX requirements draft forward without waiting for Allison. Can you respond to his request below asap, please? Cheers, Nevil ----- Forwarded message from randy@psg.com ----- Date: Tue, 30 Sep 2003 08:04:11 -0700 From: Randy Bush Reply-To: Randy Bush Subject: Re: IPFIX Requirements issues (resend) To: Nevil Brownlee > About a month ago you said: > >> allison is on vacation and almost completely off of email > > Any idea of when she'll be back? I'd like to get the IPFIX Requirements > draft finished sometime soon. in the absense of allison's response, i will now move the process. how do you feel the wg has addressed her comments? To: randy@psg.com Subject: draft-ietf-ipfix-reqs-09.txt Cc: iesg@ietf.org Date: Thu, 03 Apr 2003 08:55:15 -0800 From: Allison Mankin This is a good document overall. The applications include usage-based accounting. They should cite RFC 2975, and indicate that the particular niche for usage-based accounting would not be met if reliability was not very high. Section 5.1 reliability requirements do not meet the needs, nor do the timestamping of first and last packet of a flow, because the flow may have been mischaracterized and that is first and last of different flows. The way to satisfy this issue is to caveat the discussion of usage-based accounting with strong words on reliability that counter the comments on lesser reliability and more probabilistic techniques for other applications of ipfix. Requirements on the ipfix implementation - the document is long and I wonder if the working group really meant the protocol's confidentiality and anonymization features to be so optional - SHOULD confidentiality, MAY anonymization. Just for implementation. randy ----- End forwarded message ----- ----------------------------------------------------------------------- Nevil Brownlee Director, Technology Development Phone: +64 9 373 7599 x88941 ITSS, The University of Auckland FAX: +64 9 373 7021 Private Bag 92019, Auckland, New Zealand ------------------------------------------------- This mail sent through University of Auckland http://www.auckland.ac.nz/ -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Sep 30 18:58:05 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA20050 for ; Tue, 30 Sep 2003 18:58:04 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 1A4TJj-0000Pi-00 for ipfix-list@mil.doit.wisc.edu; Tue, 30 Sep 2003 17:50:03 -0500 Received: from sj-iport-3-in.cisco.com ([171.71.176.72] helo=sj-iport-3.cisco.com) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 1A4TJi-0000Pc-00 for ipfix@net.doit.wisc.edu; Tue, 30 Sep 2003 17:50:02 -0500 Received: from cisco.com (171.71.177.238) by sj-iport-3.cisco.com with ESMTP; 30 Sep 2003 15:54:57 -0700 Received: from ios-xdm4.cisco.com (ios-xdm4.cisco.com [171.70.69.142]) by sj-core-5.cisco.com (8.12.9/8.12.6) with ESMTP id h8UMnxXg002716; Tue, 30 Sep 2003 15:49:59 -0700 (PDT) Received: from cisco.com (dhcp-171-71-204-184.cisco.com [171.71.204.184]) by ios-xdm4.cisco.com (8.8.8-Cisco List Logging/CISCO.WS.1.2) with ESMTP id PAA11334; Tue, 30 Sep 2003 15:49:59 -0700 (PDT) Message-ID: <3F7A0896.1040405@cisco.com> Date: Tue, 30 Sep 2003 15:49:58 -0700 From: Ganesh Sadasivan Organization: Cisco Systems Inc User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.1) Gecko/20020823 X-Accept-Language: en-us, en MIME-Version: 1.0 To: "MEYER,JEFFREY D (HP-Cupertino,ex1)" CC: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Variant Field Types. References: <1D3D2C371FCBD947A7897FABBD3533A502960336@xsun01.ptp.hp.com> Content-Type: multipart/alternative; boundary="------------040807080302090302080003" Precedence: bulk Sender: majordomo listserver --------------040807080302090302080003 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Jeff, MEYER,JEFFREY D (HP-Cupertino,ex1) wrote: > Ganesh, > > I'm not clear on how this benefits the collector. Tal and myself > produce different implementations of carrier deployed products which > collect from 100's of devices and dozens of protocols, perhaps our > experience here is worth something, and it tells us variant is > undesireable on the collector... > New collector versions need not be deployed if the field size is changed from the exporter. This is true as far the portion which tries to decode the packet containing the flow records. Maybe I do not understand all the collector implementation. But what I infer from your arguments is that having an explicit type helps the collector in making computations on the the received data easier. But are'nt we dealing with encoding->exporting->decoding data defined by the information model here? > Regarding your other point: > > One way to solve this problem is if the collector assumes the largest > size for > computations? > > This was my point in the e-mail to Stewart. Have the information > model be EXPLICIT about what is the largest size which is being > modeled for a data item and declare it. Allow the exporter to declare > a smaller size in the template, if the exporter knows that an > individual record will always be accomodated (based on its > implementation) in the smaller size. > > So for instance this might say that byteCount should be declared in > the information model as a 64-bit int, while AS is probably sufficient > at 32-bits. > > This accomodates the: > exporter - can use smaller size if known > n/w - can reduce BW if know a smaller size is used > info model - explicit is better than "vague" How about defining the information model of the relevant fields with varsize and mentioning that the collector should use explicit type 'X' for a given varsize field. Otherwise vendors who implement exporters would be left in dark. > collector - addresses persistence and aggregation issues > > Can we call this a compromise? Let's wait to hear from some more people. Thanks Ganesh > > Regards, > > Jeff Meyer > > -----Original Message----- > *From:* Ganesh Sadasivan [mailto:gsadasiv@cisco.com] > *Sent:* Tuesday, September 30, 2003 10:45 AM > *To:* MEYER,JEFFREY D (HP-Cupertino,ex1) > *Cc:* ipfix@net.doit.wisc.edu > *Subject:* Re: [ipfix] Variant Field Types. > > > > MEYER,JEFFREY D (HP-Cupertino,ex1) wrote: > >> Hi, >> >> Although the use of variant types may seem convenient from the >> perspective of an >> exporter, I think the various consumers of the data would benefit >> from sticking with >> an explicit type system. > > It would benefit the exporter, the n/w, flexibility of > interpretation of fields in the > information model and the collector. > >> >> Consider operations which involve summing fields, storing them >> to a DB, etc. > > One way to solve this problem is if the collector assumes the > largest size for > computations? > >> >> Explicit typing ensures that these consuming applications can >> handle these values >> appropriately and also makes the information modeller think about >> how these >> types are to be used. >> >> The examples you cite: >> >> - ingressPort >> - egressPort >> - packetCount >> - byteCount >> - sourceAS >> - destinationAS >> - nextHopAS >> - droppedPacketCount >> - samplingInterval >> - droppedByteCount >> >> fall into a few categories, identifiers (such as ports and >> AS's), counts and intervals. >> >> For the identifiers, I believe the size is fixed by the >> relevant RFC's (i.e. why would I >> have a 1 byte Port?). For counts, the use of 32 vs. 64 bits is a >> valid question, however >> these can be handled by making them explicit (i.e. the >> information model says if >> I get this value it is a 32-bit quantity or a 64-bit quantity and >> the code will address >> it accordingly). For intervals, the resolution would determine >> which size is most >> appropriate. >> > > For AS the size could be 2 or 4 bytes. > You may prefer to use a 1 byte port if you are a small system as > against a 4 byte for > a gigantic system. Both have their own implementation reasons. > >> The option always exist to define variations on these data >> items explicitly. E.g. >> byteCount32 and byteCount64. >> > > Yes. But is'nt this not more of an overhead to introduce new field > type as the requirements > come in and then make sure that it gets deployed in exporter and > collector? > >> If there is a need to vary the sizes for various fields I'd >> much rather see the rationale >> for each discrete sizing made explicit vs. forcing all the >> downstream consumers of this \ >> information have to deal with special cases and ambiguity. >> >> > > This is not a special case handling as you can see that the list > above covers about > 40% of the fields defined today in the information model. It is > way to handle > ambiguity arising out of implementation differences. > > Thanks > Ganesh > >> Regards, >> >> Jeff Meyer >> >> >> >> -----Original Message----- >> *From:* Ganesh Sadasivan [mailto:gsadasiv@cisco.com] >> *Sent:* Monday, September 29, 2003 6:00 PM >> *To:* ipfix@net.doit.wisc.edu >> *Subject:* [ipfix] Variant Field Types. >> >>Hi, >> >>The information model only >>has types for fixed length objects and variable length >>string like objects. >> >>There are a number of information element types (listed below) >>that would be better defined in terms of fixed-size run-time >>specified objects. The type variation comes due to difference >>in the size of the objects used because of an implementation, >>as demanded by the network or otherwise. >> >>The way to make this work is to use the template >>length field to determine the exact sub-type: >> >>The definitions of a variant field type can be stated as: >> >>type varUnsignedInt >> if (element.length == 1) >> type = unsignedByte >> else if (element.length <= 2) >> type = unsignedShort >> else if (element.length <= 4) >> type = unsignedInt >> else if (element.length <= 8) >> type = unsignedLong >> else type = undefined >> >>type varInt >> if (element.length == 1) >> type = byte >> else if (element.length <= 2) >> type = short >> else if (element.length <= 4) >> type = int >> else if (element.length <= 8) >> type = long >> else type = undefined >> >>The candidates that fit into the description above are (I may >>have missed some also): >>- ingressPort >>- egressPort >>- packetCount >>- byteCount >>- sourceAS >>- destinationAS >>- nextHopAS >>- droppedPacketCount >>- samplingInterval >>- droppedByteCount >> >>Please comment. >> >>Thanks >>Ganesh >> >> > --------------040807080302090302080003 Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7bit Jeff,

MEYER,JEFFREY D (HP-Cupertino,ex1) wrote:
Ganesh,
 
  I'm not clear on how this benefits the collector.  Tal and myself produce different implementations of carrier deployed products which collect from 100's of devices and dozens of protocols, perhaps our experience here is worth something, and it tells us variant is undesireable on the collector...
 
New collector versions need not be deployed if the field size is changed from the exporter.
This is true as far the portion which tries to decode the packet containing the flow records.
Maybe I do not understand all the collector implementation. But what I infer from your
arguments is that having an explicit type helps the collector in making computations on the
the received data easier.
But are'nt we dealing with encoding->exporting->decoding data defined by the information
model here?

  Regarding your other point:
 
One way to solve this problem is if the collector assumes the largest size for
computations?
 
  This was my point in the e-mail to Stewart.  Have the information model be EXPLICIT about what is the largest size which is being modeled for a data item and declare it.  Allow the exporter to declare a smaller size in the template, if the exporter knows that an individual record will always be accomodated (based on its implementation) in the smaller size.
 
  So for instance this might say that byteCount should be declared in the information model as a 64-bit int, while AS is probably sufficient at 32-bits.
 

  This accomodates the:
      exporter - can use smaller size if known
      n/w - can reduce BW if know a smaller size is used
      info model - explicit is better than "vague"
How about  defining the information model of the relevant fields with varsize
and mentioning that the collector should use explicit type 'X' for a given
varsize field. Otherwise vendors who implement exporters would be left in
dark.
      collector - addresses persistence and aggregation issues
 
  Can we call this a compromise?
Let's wait to hear from some more people.

Thanks
Ganesh

 
Regards,
 
  Jeff Meyer
-----Original Message-----
From: Ganesh Sadasivan [mailto:gsadasiv@cisco.com]
Sent: Tuesday, September 30, 2003 10:45 AM
To: MEYER,JEFFREY D (HP-Cupertino,ex1)
Cc: ipfix@net.doit.wisc.edu
Subject: Re: [ipfix] Variant Field Types.



MEYER,JEFFREY D (HP-Cupertino,ex1) wrote:
Hi,
 
  Although the use of variant types may seem convenient from the perspective of an
exporter, I think the various consumers of the data would benefit from sticking with
an explicit type system.
It would benefit the exporter, the n/w, flexibility of interpretation of fields in the
information model  and the collector.
 
  Consider operations which involve summing fields, storing them to a DB, etc.
One way to solve this problem is if the collector assumes the largest size for
computations?
 
  Explicit typing ensures that these consuming applications can handle these values
appropriately and also makes the information modeller think about how these
types are to be used.
 
  The examples you cite:
 
- ingressPort
- egressPort
- packetCount
- byteCount
- sourceAS
- destinationAS
- nextHopAS
- droppedPacketCount
- samplingInterval
- droppedByteCount
 
  fall into a few categories, identifiers (such as ports and AS's), counts and intervals.
 
  For the identifiers, I believe the size is fixed by the relevant RFC's (i.e. why would I
have a 1 byte Port?).  For counts, the use of 32 vs. 64 bits is a valid question, however
these can be handled by making them explicit (i.e. the information model says if
I get this value it is a 32-bit quantity or a 64-bit quantity and the code will address
it accordingly).  For intervals, the resolution would determine which size is most
appropriate.
 
For AS the size could be 2 or 4 bytes.
You may prefer to use a 1 byte port if you are a small system as against a 4 byte for
a gigantic system. Both have their own implementation reasons.
  The option always exist to define variations on these data items explicitly.  E.g.
byteCount32 and byteCount64.
 
Yes. But is'nt this not more of an overhead to introduce new field type as the requirements
come in and then make sure that it gets deployed in exporter and collector?

  If there is a need to vary the sizes for various fields I'd much rather see the rationale
for each discrete sizing made explicit vs. forcing all the downstream consumers of this \
information have to deal with special cases and ambiguity.
 
 
This is not a special case handling as you can see that the list above covers about
40% of the fields defined today in the information model. It is way to handle
ambiguity arising out of implementation differences.

Thanks
Ganesh
Regards,
 
  Jeff Meyer

 
-----Original Message-----
From: Ganesh Sadasivan [mailto:gsadasiv@cisco.com]
Sent: Monday, September 29, 2003 6:00 PM
To: ipfix@net.doit.wisc.edu
Subject: [ipfix] Variant Field Types.

Hi,

The information model <draft-ietf-ipfix-info-01.txt> only
has types for fixed length objects and variable length 
string like objects.

There are a number of information element types (listed below)
that would be better defined in terms of fixed-size run-time
specified objects. The type variation comes due to difference
in the size of the objects used because of an implementation,
as demanded by the network or otherwise.

The way to make this work is to use the template
length field to determine the exact sub-type:

The definitions of a variant field type can be stated as:

type varUnsignedInt
    if (element.length == 1)
         type = unsignedByte
    else if (element.length <= 2)
         type = unsignedShort
    else if (element.length <= 4)
         type = unsignedInt
    else if (element.length <= 8)
         type = unsignedLong
   else type = undefined

type varInt
    if (element.length == 1)
         type = byte
    else if (element.length <= 2)
         type = short
    else if (element.length <= 4)
         type = int
    else if (element.length <= 8)
         type = long
   else type = undefined

The candidates that fit into the description above are (I may
have missed some also):
- ingressPort 
- egressPort 
- packetCount 
- byteCount 
- sourceAS 
- destinationAS
- nextHopAS 
- droppedPacketCount
- samplingInterval
- droppedByteCount

Please comment.

Thanks
Ganesh


--------------040807080302090302080003-- -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Sep 30 20:23:30 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA22297 for ; Tue, 30 Sep 2003 20:23:30 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 1A4UZV-00039s-00 for ipfix-list@mil.doit.wisc.edu; Tue, 30 Sep 2003 19:10:25 -0500 Received: from relay.pair.com ([209.68.1.20]) by mil.doit.wisc.edu with smtp (Exim 3.13 #1) id 1A4UZU-00039j-00 for ipfix@net.doit.wisc.edu; Tue, 30 Sep 2003 19:10:24 -0500 Received: (qmail 69726 invoked from network); 1 Oct 2003 00:10:22 -0000 Received: from 207-237-37-137.c3-0.nyr-ubr3.nyr.ny.cable.rcn.com (HELO set) (207.237.37.137) by relay.pair.com with SMTP; 1 Oct 2003 00:10:22 -0000 X-pair-Authenticated: 207.237.37.137 Reply-To: From: "Carter Bullard" To: "'Ganesh Sadasivan'" , "'MEYER,JEFFREY D \(HP-Cupertino,ex1\)'" Cc: Subject: RE: [ipfix] Variant Field Types. Date: Tue, 30 Sep 2003 20:10:06 -0400 Organization: QoSient,LLC Message-ID: <000101c387b0$6168d540$0200a8c0@newyork.qosient.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.4510 Importance: Normal In-Reply-To: <3F7A0896.1040405@cisco.com> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 Precedence: bulk Sender: majordomo listserver Content-Transfer-Encoding: quoted-printable Gentle people, It is terribly important that IPFIX support the transport of vendor specific data. Because of this, I expect IPFIX to support variant field types. =20 One could argue that variant field types are simply the implementation of a very large number of explicit types (4 bit int, 8 bit int, 16 bit int, 4-bit src address, 4 bit protocol field, etc....). I support the notion of variant field types because it is the only mechanism for improving transport efficiency, which I believe should be the primary design goal for IPFIX. I'm not convinced that collector complexity should have any impact on IPFIX's data model. Let IPFIX be efficient, expect the collectors do what is needed to support it. Carter -----Original Message----- From: majordomo listserver [mailto:majordomo@mil.doit.wisc.edu] On = Behalf Of Ganesh Sadasivan Sent: Tuesday, September 30, 2003 6:50 PM To: MEYER,JEFFREY D (HP-Cupertino,ex1) Cc: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Variant Field Types. Jeff, MEYER,JEFFREY D (HP-Cupertino,ex1) wrote: Ganesh, I'm not clear on how this benefits the collector. Tal and myself = produce different implementations of carrier deployed products which collect = from 100's of devices and dozens of protocols, perhaps our experience here is worth something, and it tells us variant is undesireable on the = collector... New collector versions need not be deployed if the field size is changed from the exporter. This is true as far the portion which tries to decode the packet = containing the flow records. Maybe I do not understand all the collector implementation. But what I = infer from your=20 arguments is that having an explicit type helps the collector in making computations on the=20 the received data easier. But are'nt we dealing with encoding->exporting->decoding data defined by = the information model here?=20 Regarding your other point: One way to solve this problem is if the collector assumes the largest = size for=20 computations? This was my point in the e-mail to Stewart. Have the information = model be EXPLICIT about what is the largest size which is being modeled for a = data item and declare it. Allow the exporter to declare a smaller size in = the template, if the exporter knows that an individual record will always be accomodated (based on its implementation) in the smaller size. So for instance this might say that byteCount should be declared in = the information model as a 64-bit int, while AS is probably sufficient at 32-bits. This accomodates the: exporter - can use smaller size if known n/w - can reduce BW if know a smaller size is used info model - explicit is better than "vague" How about defining the information model of the relevant fields with varsize and mentioning that the collector should use explicit type 'X' for a = given=20 varsize field. Otherwise vendors who implement exporters would be left = in dark. collector - addresses persistence and aggregation issues Can we call this a compromise? Let's wait to hear from some more people. Thanks Ganesh Regards, Jeff Meyer -----Original Message----- From: Ganesh Sadasivan [mailto:gsadasiv@cisco.com] Sent: Tuesday, September 30, 2003 10:45 AM To: MEYER,JEFFREY D (HP-Cupertino,ex1) Cc: ipfix@net.doit.wisc.edu Subject: Re: [ipfix] Variant Field Types. MEYER,JEFFREY D (HP-Cupertino,ex1) wrote: Hi, Although the use of variant types may seem convenient from the = perspective of an exporter, I think the various consumers of the data would benefit from sticking with an explicit type system. It would benefit the exporter, the n/w, flexibility of interpretation of fields in the=20 information model and the collector. Consider operations which involve summing fields, storing them to a = DB, etc. One way to solve this problem is if the collector assumes the largest = size for=20 computations? Explicit typing ensures that these consuming applications can handle = these values appropriately and also makes the information modeller think about how = these=20 types are to be used. The examples you cite: - ingressPort=20 - egressPort=20 - packetCount=20 - byteCount=20 - sourceAS=20 - destinationAS - nextHopAS=20 - droppedPacketCount - samplingInterval - droppedByteCount fall into a few categories, identifiers (such as ports and AS's), = counts and intervals. For the identifiers, I believe the size is fixed by the relevant RFC's (i.e. why would I=20 have a 1 byte Port?). For counts, the use of 32 vs. 64 bits is a valid question, however these can be handled by making them explicit (i.e. the information model says if I get this value it is a 32-bit quantity or a 64-bit quantity and the = code will address it accordingly). For intervals, the resolution would determine which = size is most appropriate. For AS the size could be 2 or 4 bytes. You may prefer to use a 1 byte port if you are a small system as against = a 4 byte for=20 a gigantic system. Both have their own implementation reasons. The option always exist to define variations on these data items explicitly. E.g. byteCount32 and byteCount64. Yes. But is'nt this not more of an overhead to introduce new field type = as the requirements come in and then make sure that it gets deployed in exporter and = collector? If there is a need to vary the sizes for various fields I'd much = rather see the rationale for each discrete sizing made explicit vs. forcing all the downstream consumers of this \ information have to deal with special cases and ambiguity. This is not a special case handling as you can see that the list above covers about=20 40% of the fields defined today in the information model. It is way to handle=20 ambiguity arising out of implementation differences. Thanks Ganesh Regards, Jeff Meyer =20 -----Original Message----- From: Ganesh Sadasivan [mailto:gsadasiv@cisco.com] Sent: Monday, September 29, 2003 6:00 PM To: ipfix@net.doit.wisc.edu Subject: [ipfix] Variant Field Types. Hi, The information model only has types for fixed length objects and variable length=20 string like objects. There are a number of information element types (listed below) that would be better defined in terms of fixed-size run-time specified objects. The type variation comes due to difference in the size of the objects used because of an implementation, as demanded by the network or otherwise. The way to make this work is to use the template length field to determine the exact sub-type: The definitions of a variant field type can be stated as: type varUnsignedInt if (element.length =3D=3D 1) type =3D unsignedByte else if (element.length <=3D 2) type =3D unsignedShort else if (element.length <=3D 4) type =3D unsignedInt else if (element.length <=3D 8) type =3D unsignedLong else type =3D undefined type varInt if (element.length =3D=3D 1) type =3D byte else if (element.length <=3D 2) type =3D short else if (element.length <=3D 4) type =3D int else if (element.length <=3D 8) type =3D long else type =3D undefined The candidates that fit into the description above are (I may have missed some also): - ingressPort=20 - egressPort=20 - packetCount=20 - byteCount=20 - sourceAS=20 - destinationAS - nextHopAS=20 - droppedPacketCount - samplingInterval - droppedByteCount Please comment. Thanks Ganesh =20 -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/ From majordomo@mil.doit.wisc.edu Tue Sep 30 20:45:30 2003 Received: from mil.doit.wisc.edu (mil.doit.wisc.edu [128.104.31.31]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA22962 for ; Tue, 30 Sep 2003 20:45:30 -0400 (EDT) Received: from majordomo by mil.doit.wisc.edu with local (Exim 3.13 #1) id 1A4Uzw-00041t-00 for ipfix-list@mil.doit.wisc.edu; Tue, 30 Sep 2003 19:37:44 -0500 Received: from palrel13.hp.com ([156.153.255.238]) by mil.doit.wisc.edu with esmtp (Exim 3.13 #1) id 1A4Uzv-00041j-00 for ipfix@net.doit.wisc.edu; Tue, 30 Sep 2003 19:37:43 -0500 Received: from xparelay2.ptp.hp.com (xparelay2.ptp.hp.com [15.1.28.65]) by palrel13.hp.com (Postfix) with ESMTP id D27E51C00BE1 for ; Tue, 30 Sep 2003 17:37:42 -0700 (PDT) Received: from xpabh2.ptp.hp.com (xpabh2.ptp.hp.com [15.1.28.61]) by xparelay2.ptp.hp.com (Postfix) with ESMTP id CCBD11C000AD for ; Tue, 30 Sep 2003 17:37:42 -0700 (PDT) Received: by xpabh2.ptp.hp.com with Internet Mail Service (5.5.2655.55) id ; Tue, 30 Sep 2003 17:37:42 -0700 Message-ID: <1D3D2C371FCBD947A7897FABBD3533A502960338@xsun01.ptp.hp.com> From: "MEYER,JEFFREY D (HP-Cupertino,ex1)" To: ipfix@net.doit.wisc.edu Subject: RE: [ipfix] Variant Field Types. Date: Tue, 30 Sep 2003 17:37:38 -0700 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2655.55) Content-Type: text/plain; charset="ISO-8859-1" Precedence: bulk Sender: majordomo listserver Carter, IPFIX supports vendor specific data. The information model is extensible. So you're covered on that point. Regarding transport efficiency, the proposed compromise was that one explicitly indicate the maximal size of an information item in the model, but allow the template to indicate that fewer bytes are used in the record. Keep in mind that the protocol enables two entities to communicate, an exporter and a collector. Implementation feasability and practicality on both ends make for something that will be adopted. From an implementation perspective, the exporters are likely to be more purpose built in their behavior, while the collectors will likely be more general purpose to support a variety of exporters. As such the exporters will likely have a fixed information model which they simple declare, whereas the collectors will likely require configuration for given devices. So from a collector perspective, the machine readable and unambiguous information model is important, and as proposed does not constrain the exporter. Regards, Jeff Meyer > -----Original Message----- > From: Carter Bullard [mailto:carter@qosient.com] > Sent: Tuesday, September 30, 2003 5:10 PM > To: 'Ganesh Sadasivan'; 'MEYER,JEFFREY D (HP-Cupertino,ex1)' > Cc: ipfix@net.doit.wisc.edu > Subject: RE: [ipfix] Variant Field Types. > > > Gentle people, > It is terribly important that IPFIX support the > transport of vendor specific data. Because of this, > I expect IPFIX to support variant field types. > One could argue that variant field types are simply the > implementation of a very large number of explicit > types (4 bit int, 8 bit int, 16 bit int, 4-bit src > address, 4 bit protocol field, etc....). > > I support the notion of variant field types because > it is the only mechanism for improving transport efficiency, > which I believe should be the primary design goal for IPFIX. > I'm not convinced that collector complexity should have > any impact on IPFIX's data model. Let IPFIX be efficient, > expect the collectors do what is needed to support it. > > Carter > > > > > > -----Original Message----- > From: majordomo listserver > [mailto:majordomo@mil.doit.wisc.edu] On Behalf Of > Ganesh Sadasivan > Sent: Tuesday, September 30, 2003 6:50 PM > To: MEYER,JEFFREY D (HP-Cupertino,ex1) > Cc: ipfix@net.doit.wisc.edu > Subject: Re: [ipfix] Variant Field Types. > > > Jeff, > > MEYER,JEFFREY D (HP-Cupertino,ex1) wrote: > > Ganesh, > > I'm not clear on how this benefits the collector. Tal and > myself produce > different implementations of carrier deployed products which > collect from > 100's of devices and dozens of protocols, perhaps our > experience here is > worth something, and it tells us variant is undesireable on > the collector... > > New collector versions need not be deployed if the field size > is changed > from the exporter. > This is true as far the portion which tries to decode the > packet containing > the flow records. > Maybe I do not understand all the collector implementation. > But what I infer > from your > arguments is that having an explicit type helps the collector > in making > computations on the > the received data easier. > But are'nt we dealing with encoding->exporting->decoding data > defined by the > information > model here? > > > Regarding your other point: > > One way to solve this problem is if the collector assumes the > largest size > for > computations? > > This was my point in the e-mail to Stewart. Have the > information model be > EXPLICIT about what is the largest size which is being > modeled for a data > item and declare it. Allow the exporter to declare a smaller > size in the > template, if the exporter knows that an individual record > will always be > accomodated (based on its implementation) in the smaller size. > > So for instance this might say that byteCount should be > declared in the > information model as a 64-bit int, while AS is probably sufficient at > 32-bits. > > > > This accomodates the: > exporter - can use smaller size if known > n/w - can reduce BW if know a smaller size is used > info model - explicit is better than "vague" > How about defining the information model of the relevant fields with > varsize > and mentioning that the collector should use explicit type > 'X' for a given > varsize field. Otherwise vendors who implement exporters > would be left in > dark. > > collector - addresses persistence and aggregation issues > > Can we call this a compromise? > Let's wait to hear from some more people. > > Thanks > Ganesh > > > > Regards, > > Jeff Meyer > -----Original Message----- > From: Ganesh Sadasivan [mailto:gsadasiv@cisco.com] > Sent: Tuesday, September 30, 2003 10:45 AM > To: MEYER,JEFFREY D (HP-Cupertino,ex1) > Cc: ipfix@net.doit.wisc.edu > Subject: Re: [ipfix] Variant Field Types. > > > > > MEYER,JEFFREY D (HP-Cupertino,ex1) wrote: > > Hi, > > Although the use of variant types may seem convenient from > the perspective > of an > exporter, I think the various consumers of the data would benefit from > sticking with > an explicit type system. > It would benefit the exporter, the n/w, flexibility of > interpretation of > fields in the > information model and the collector. > > > Consider operations which involve summing fields, storing > them to a DB, > etc. > One way to solve this problem is if the collector assumes the > largest size > for > computations? > > > Explicit typing ensures that these consuming applications > can handle these > values > appropriately and also makes the information modeller think > about how these > types are to be used. > > The examples you cite: > > - ingressPort > - egressPort > - packetCount > - byteCount > - sourceAS > - destinationAS > - nextHopAS > - droppedPacketCount > - samplingInterval > - droppedByteCount > > fall into a few categories, identifiers (such as ports and > AS's), counts > and intervals. > > For the identifiers, I believe the size is fixed by the > relevant RFC's > (i.e. why would I > have a 1 byte Port?). For counts, the use of 32 vs. 64 bits > is a valid > question, however > these can be handled by making them explicit (i.e. the > information model > says if > I get this value it is a 32-bit quantity or a 64-bit quantity > and the code > will address > it accordingly). For intervals, the resolution would > determine which size > is most > appropriate. > > For AS the size could be 2 or 4 bytes. > You may prefer to use a 1 byte port if you are a small system > as against a 4 > byte for > a gigantic system. Both have their own implementation reasons. > > The option always exist to define variations on these data items > explicitly. E.g. > byteCount32 and byteCount64. > > Yes. But is'nt this not more of an overhead to introduce new > field type as > the requirements > come in and then make sure that it gets deployed in exporter > and collector? > > > If there is a need to vary the sizes for various fields I'd > much rather > see the rationale > for each discrete sizing made explicit vs. forcing all the downstream > consumers of this \ > information have to deal with special cases and ambiguity. > > > This is not a special case handling as you can see that the list above > covers about > 40% of the fields defined today in the information model. It is way to > handle > ambiguity arising out of implementation differences. > > Thanks > Ganesh > > Regards, > > Jeff Meyer > > > > -----Original Message----- > From: Ganesh Sadasivan [mailto:gsadasiv@cisco.com] > Sent: Monday, September 29, 2003 6:00 PM > To: ipfix@net.doit.wisc.edu > Subject: [ipfix] Variant Field Types. > > > Hi, > > The information model only > has types for fixed length objects and variable length > string like objects. > > There are a number of information element types (listed below) > that would be better defined in terms of fixed-size run-time > specified objects. The type variation comes due to difference > in the size of the objects used because of an implementation, > as demanded by the network or otherwise. > > The way to make this work is to use the template > length field to determine the exact sub-type: > > The definitions of a variant field type can be stated as: > > type varUnsignedInt > if (element.length == 1) > type = unsignedByte > else if (element.length <= 2) > type = unsignedShort > else if (element.length <= 4) > type = unsignedInt > else if (element.length <= 8) > type = unsignedLong > else type = undefined > > type varInt > if (element.length == 1) > type = byte > else if (element.length <= 2) > type = short > else if (element.length <= 4) > type = int > else if (element.length <= 8) > type = long > else type = undefined > > The candidates that fit into the description above are (I may > have missed some also): > - ingressPort > - egressPort > - packetCount > - byteCount > - sourceAS > - destinationAS > - nextHopAS > - droppedPacketCount > - samplingInterval > - droppedByteCount > > Please comment. > > Thanks > Ganesh > > > -- Help mailto:majordomo@net.doit.wisc.edu and say "help" in message body Unsubscribe mailto:majordomo@net.doit.wisc.edu and say "unsubscribe ipfix" in message body Archive http://ipfix.doit.wisc.edu/archive/