Subject: [ipfix-eval] IPFix Summary

From: "Reinaldo Penno" <[email-masked]>
Date: Sun, 27 Oct 2002 14:30:54 -0800

I burned a good chunk of my weekend trying to summarize our discussions. If
you think I didn't capture what was said adequately, please speak up. This
email does not substitute Juergen's on open issues, but is actually trying
to close some of those open issues. (Should I copy the main ipfix list?)

I revised most of the emails starting from Jeff Meyer's - "Discussion of
Candidate Protocols" on - 9/30.

a)Regarding metering requirements and protocol candidates.

There is consensus that the metering requirements should be N/A when
choosing the IPfix Export Protocol.

A good summary was provided by Peter.

* The protocol should cover only the issues of delivering data from the
exporter to the collector. It must be expressive 
enough to contain all the data defined in the data model. 

* The data model should cover the identification of the metrics and
attributes, their meanings, and how they are grouped 
together. 

* The architecture should cover issues such as where data are observed, what
data are required and what are optional, when 
reliability is needed, how the data can be manipulated (e.g., calculating
bandwidth), etc., etc.

b) Regarding reliability

   - Congestion awareness. 

It is covered on the requirements draft

   - Message ordering (critical when the data spans several packets, e.g.
variable length fields) 

It seems there is consensus that message ordering is important.

   - Transport reliability - 

There is some confusion around this problem since transport reliability is
*not* explicit mentioned in the reqs draft, but 
since congestion awareness is, some people take it for granted. 

   - require application-level ACKs in the protocol (SHOULD be after
     data are persisted)

There is consensus that application layer ACKs is important

   - require enable de-duplication of received records through a unique key

There is consensus that de-duplication is important.

   - Exporter should keep track of number of flow records + some important
totals
     and and periodically communicate this information to the
     collector.

There is consensus that this or some other solution to capture the 
amount of lost *flow records for each key/field* (not packets) is important

c) Regarding High-Availability

we should have the ability to send to multiple collectors and for the 
exporter to be able to switch from one collector to another.

d) Regarding Timestamps

New text for section 5.4 (Disclaimer before somebody goes for my neck: this
affects the 
architecture/requirements, not the protocol evaluation.)

   The metering process MUST be able to generate wire-line timestamps 
   (rather then flow cache timestamps) for the first and the last 
   observed packet of a flow. The timestamp resolution MUST be at least 
   the one of the sysUpTime [RFC1213], which is one centisecond.

e) Regarding the Views on the Problem

Some people advocate that exporting IP flows has nothing unique to it, it's
well known problem 
only with a different data model. I hope I captured this right. Anyway, here
it goes a quotation.

"However, the statement that IP flow is somehow unique
in its data requirements and as such a more generalized
"data mover" is somehow problematic, is just plain wrong."

f) Regarding the Field Experience of IPDR

Some people wanted more information on this, so I'm pasting an email 
from Aron Heintz [[email-masked]]. Any more in-depth information please
direct your 
questions to Jeff Meyer [[email-masked]] or Aron Heitz.

"
-----Original Message-----
From: Aron Heintz [mailto:[email-masked]]
Sent: Thursday, October 10, 2002 3:47 PM
To: [email-masked]; Harrington, David; Jeff Meyer
Subject: RE: [ipfix-eval] Discussion of Candidate Protocols

 It seems to me that NDM-U wins the "free, open, and widely deployed" crown
hands down.  NDM-U 3.1 is completely free and publicly available (see
below).  By the end of this year, more than 70% of the mediation and billing
packages (by market share) will have proven NDM-U 3.1 *real-world*
interoperability.

 What could be more important than this?  It appears that some of you are
debating minor technical points when you might approach the question "Who is
going to receive the data we are sending?  How do they want to see it?"
Technologies do not win by virtue of their theoretical perfection.  The OSI
model is close to theoretical perfection.  TCP/IP is not.
"

---

From: [email-masked]
Date: Mon, 28 Oct 2002 09:43:00 -0500

> Reinaldo Penno wrote:
> 
> I burned a good chunk of my weekend trying to summarize our
> discussions. If you think I didn't capture what was said adequately,
> please speak up. This email does not substitute Juergen's on open
> issues, but is actually trying to close some of those open issues.
> (Should I copy the main ipfix list?)
> 
> I revised most of the emails starting from Jeff Meyer's - "Discussion
> of Candidate Protocols" on - 9/30.
> 
> * The protocol should cover only the issues of delivering data from
> the exporter to the collector. It must be expressive
> 
> * The architecture should cover issues such as where data are
> observed, what data are required and what are optional, when
> 
> reliability is needed, how the data can be manipulated (e.g.,
> calculating bandwidth), etc., etc.
> 
> b) Regarding reliability
> 
>    - Congestion awareness.
> 
>    - Message ordering (critical when the data spans several packets,
> e.g. variable length fields)
> 
> There is some confusion around this problem since transport
> reliability is *not* explicit mentioned in the reqs draft, but
> 

	I'm concerned that we are going towards a one size fits
	all in terms of reliability. However, with added reliability 
	comes added overhead.  So what about those applications that
	do not need such reliability and would rather take the added
	capacity?

>    - Exporter should keep track of number of flow records + some
> important totals

	I'm not sure what you mean here. Send each record to multiple 
	collectors or failover to another server when the current
	collector goes down?

> d) Regarding Timestamps
> 
> New text for section 5.4 (Disclaimer before somebody goes for my neck:
> this affects the

	Making things like this a MUST does not help. Vendors
	wont or maybe can't change their hardware to comply. 

	I would advocate having 2 different timestamps. One with 
	a meaning of wire-line timestamps and the other to mean
	cache timestamp.

	Paul
> Some people advocate that exporting IP flows has nothing unique to it,
> it's well known problem
> only with a different data model. I hope I captured this right.
> Anyway, here it goes a quotation.
> 
> from Aron Heintz [[email-masked]]. Any more in-depth information
> please direct your

--
Help        mailto:[email-masked] and say "help" in message body
Unsubscribe mailto:[email-masked] and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/

---

From: [email-masked]
Date: Mon, 28 Oct 2002 09:43:00 -0500

Paul

---

From: "Juergen Quittek" <[email-masked]>
Date: Tue, 29 Oct 2002 22:07:58 +0100

Reinaldo,

Many thanks for this digest!

I added comments mainly having the requirements draft in mind.

    Juergen

> choosing the IPfix Export Protocol.

There are a few exceptions: 

  - Some of the metering requirements depend on the export protocol
      - Absence of reliability of the metering protocol need to be 
        indicated to the collector (section 5.1).
      - Switch to and from overload behavoir need to be indicated 
        to the collector (section 5.3).

  - Some of the candidates, for example NetFlow, also define
    properties of the metering process. If they do so, these
    properties should be checked against the requirements.

> *not* explicit mentioned in the reqs draft, but 

How do you see the difference between "transport reliability" (you are
right, it is not mentioned in the draft) and "reliability of the 
data transfer" (mentioned in the draft in Section 6.3.2.)?

> There is consensus that application layer ACKs is important

It looks like I really missed something here. 
What is an application level ACK compared to an IPFIX protocol level ACK?
Wouldn't transport level ACKs also serve quite well?

> There is consensus that de-duplication is important.

That's great! 
Is there also consensus that its coverage in Section 8.3. is sufficient?

> amount of lost *flow records for each key/field* (not packets) is important

I think there is also consensus that this is a solution (a good one), 
and not a requirement.

> we should have the ability to send to multiple collectors and for the 

Does your phrase "we should" imply that, you suggest to replace the 
first "MAY" in Section 8.3 by "SHOULD"?

> exporter to be able to switch from one collector to another.

Yes, we should append some text to Section 6.3.2 covering failover.
What about:

  "If the exporting process is capable of detecting loss of connection 
   to a collecting process, it SHOULD be able to perform failover to 
   a specified backup collecting process."

>    the one of the sysUpTime [RFC1213], which is one centisecond.

Here is a slightly different proposal. It uses pure IPFIX terminology
and does not require us to define the terms "wire-line timestamp" and
"flow cache timestamp". We haven't even defined the term "flow cache":

  "The metering process MUST be able to generate timestamps for the
   first and the last observation of a packet of a flow at the 
   observation point. The timestamp resolution MUST be at least the 
   one of the sysUpTime [RFC1213], which is one centisecond."

> "data mover" is somehow problematic, is just plain wrong."

Still I believe you can use many generic data movers for IPFIX.
This has advantages (re-use of definition and stack implementation, 
already stable protocol, ...) as well as disadvantages (efficiency, ...)
Otherwise, we could immediatedly rule out Diameter.

-- 
 ‹Z@

---

From: "Juergen Quittek" <[email-masked]>
Date: Tue, 29 Oct 2002 22:07:58 +0100

Reinaldo,

    Juergen

That's great! 

-- 
 ‹Z@

---

From: [email-masked]
Date: Tue, 29 Oct 2002 18:12:07 -0500

Juergen Quittek wrote:

[snip]

> data transfer" (mentioned in the draft in Section 6.3.2.)?

	This is an important and complex topic and I don't think
	we have discussed it adequately. I would like to see the
	list take another pass at it. 

> I think there is also consensus that this is a solution (a good one),

	The requirement consensus, I believe, was this...

	From: http://ipfix.doit.wisc.edu/archive/1241.html

		Benoit Claise wrote:
		[snip]
		> 
		> Unless you want to say:
		>         If flow information is lost, a means to gauge the amount of
		>         loss MUST be available from the Exporter _or the_ Collector
or both.
		>         Possible reasons for flow data loss include loss of packets
on
		>         the network path, resource shortage at the exporter,
Collector
		>         crash, etc...
		
		        This is fine with me.
			
        		Paul

>    one of the sysUpTime [RFC1213], which is one centisecond."

	I have trouble with these words "first and the last observation of a
packet"

	It makes the assumption that all hardware vendors timestamp each
	and every packet and many do not.

	I would prefer to have the percise definition in the data model
	and have the requirement looser, something like this...

        "The metering process MUST be able to generate timestamps for
the
         start and end of a flow. The timestamp resolution MUST be at
least 
	 of the sysUpTime [RFC1213], which is one centisecond."

	The data model can then define the precise meaning of the timestamp
	used (e.g. wire-timestamp of the last packet observed or in-activity
	timer).

	Paul

>  <Z@

---

From: [email-masked]
Date: Tue, 29 Oct 2002 18:12:07 -0500

Juergen Quittek wrote:

[snip]

		Benoit Claise wrote:
		[snip]
or both.
on
Collector
		
		        This is fine with me.
			
        		Paul

	Paul

---

From: "Peter Ludemann" <[email-masked]>
Date: Thu, 31 Oct 2002 11:50:41 -0800

Juergen Quittek wrote Tuesday, October 29, 2002 1:08 PM:

[snip]

> data transfer" (mentioned in the draft in Section 6.3.2.)?

Answered below.

> Wouldn't transport level ACKs also serve quite well?

No, transport level ACK is merely a necessary condition, not a sufficient
condition. [OK, I exaggerate: it is possible to combine the transport and
application ACKs; but that would be similar to re-implementing TCP or SCTP
over UDP, something which the NFS folks have concluded is silly.]

1. Transportation level ACKs exist for:
  - network-level congestion awareness;
  - ensuring that all the data arrive at the other *machine*
    (not necessarily at the application: see #3) in order
    and without duplicates.

2. TCP can have quite long timeouts before it decides that
   a connection is down (in some cases, this can be hours).
   Application level ACKs allow more timely detection that
   there is a problem, and quicker fail-over.

3. TCP level ACKs only mean that the data reached the other
   machine. They do *not* mean that the data have been properly
   processed or persisted. (In theory, the collector could
   have a TCP stack that doesn't send an ACK until the
   application has handled the data; but I'm not aware of
   any such TCP implementations; they certainly would require
   a different API than the Berkeley socket API.)

4. Application level ACKs give a more accurate picture of
   how fast the data are being processed by the collector.
   With large buffers and tuning for "long fat pipes", TCP's
   ACKs (e.g., RFC 1323) may not be very indicative of how
   well the collector is able to keep up. (If the receiver
   can't keep up, eventually the buffers will fill up and
   transmission will block. But with application level ACKs,
   this can be determined more quickly and a switch over to
   an alternate receiver done sooner.)

5. The application-level ACK strategy is not prescribed in
   any of the candidate protocols, as far as I know. However,
   the usual method is to use a combination of timeout and
   volume (e.g., send an ACK at least every 1000 records or
   2 seconds, whichever comes first). Although this can be
   recommended for a protocol, it should not be mandated; for
   example, our implementation ties into a "commit/rollback"
   model of data persistency and sends an ACK whenever a "commit"
   has been done successfully (this commit is usually tied to
   volume and time, but can be triggered by other events).

(Remark #2 does not apply to SCTP but the other remarks do.)

- peter

---

From: "Peter Ludemann" <[email-masked]>
Date: Thu, 31 Oct 2002 11:50:41 -0800

[snip]

Answered below.

- peter

---

From: "Aron Heintz" <[email-masked]>
Date: Thu, 31 Oct 2002 16:32:23 -0500

To quote David Harrington, cochair of the SNMPv3 IETF group:  "Given two
technically comparable candidates, we should consider the likelihood that
vendors will ship the protocol in their devices and that vendors' customers
will deploy the functionality in their networks.  The market is the most
important decider of industry standards."

    In response to Reinaldo's query regarding Field Experience of IPDR, I'd
like to submit the list of companies that contributed to the development of
the NDM-U specification.  Asterisks denote companies that are known to be
currently *using* the specification.

	Industrial users today comprise almost 30 companies, with a combined market
capitalization over $50 Billion.  IPDR offers immediate market acceptance
for IPFIX.

CHARTER MEMBERS
* Ace*Comm
* Amdocs
* Convergys
* CSG Systems, Inc.
* ECTel
* Hewlett-Packard
* Intrado
* Narus
Nortel
OneWave Technologies, Inc.
Rogers AT&T Wireless
* Sprint PCS
TeleStrategies
* Telus

SUPPORTING MEMBERS
* ADC
* American Management Systems
* Billing Concepts
Cisco
* Comptel
Cotton Management
DST Innovis
HNC Software
Illuminet
Infosys
* Marconi
* NEC America
NetTest
* Openet Telecom
Siemens
Sofrecom
Telcordia Technologies
* TSI Telecommunication Services

CURRENT ASSOCIATE AND PAST CONTRIBUTING MEMBERS
AM-Beo
Broadband Communications Solutions Corp.
Centrisoft
CPqD
* Digital Route
Enition
Formula Telecom Solutions
KORNETS
MetraTech Corp.
Portal Software
TeleGea, Inc.
Trendium
* Xacct Technologies
Abiliti Solutions
Accenture
Agilent Technologies
Alcatel
Alltel Information Services
AP Engines
Apogee Networks
Aptis
ASC Billing Solutions
Asiainfo Holdings, Inc.
AT&T
BayPackets, Inc.
Bridgewater Systems Corp
Brix Networks, Inc.
BusinessEdge Solutions
CACI
CCMI
Clarent
* Daleen
Digiquant
Digital Fuel
DMR Consulting Group
EDB4tel
EHPT USA Inc
Elron Technologies
EmpowerTel
Ericsson Telecom B.V.
Evergent Technologies
Extent Technologies
FileTek
* H.O. Systems
Hatteras Networks, Inc.
Huawei Technologies
Hughes Network Systems
Infocomm
* Info Directions
InformationView Solutions Corporation (Now Vibrant Solutions)
InfoVista Corporation
* Intec
Intel
IP23
IPCell Technologies, Inc
JAMDAT Mobile Inc.
LongBoard, Inc.
Lucent Technologies
Magardi, Inc.
Metapath Software International Ltd.
Mind CTI, Inc.
mSafe
NCR
Netcom Consultants
Netconn Solutions
NetEye
Nexus Telecom AG
Opencon Systems
P-Cube Ltd.
Pacific Bell
Packeteer, Inc.
Portal Software
Primal Systems Inc.
Proxima Systems Limited
Qubiz GmbH
Quintrex Data Systems Group
Radguard
* RateIntegration
Savera Systems
* Sepro
* Sema Telecom
Shenzhen Zhongxing-suntek Data Communication
     Co. Ltd.
Sigma Systems Group
Suntec Business Systems
TCSI Corporation
Telchemy
Teleglobe Communications Corporation
TeleKnowledge, Inc.
Telesens KCSL
Telia Mobile AB
Telution LLC
Tertio Limited
TMNG, Inc.
TopLayer Networks, Inc.
USHA Communications Technology (UBEST India)
* Virtual Summit, inc.
Vpacket.com

-----Original Message-----
From: majordomo listserver [mailto:[email-masked]]On Behalf Of
Reinaldo Penno
Sent: Sunday, October 27, 2002 5:31
To: [email-masked]
Cc: [email-masked]
Subject: [ipfix-req] IPFix Summary

a) ...
from Aron Heintz...

---

From: "Aron Heintz" <[email-masked]>
Date: Thu, 31 Oct 2002 16:32:23 -0500

CHARTER MEMBERS
* Ace*Comm
* Amdocs
* Convergys
* CSG Systems, Inc.
* ECTel
* Hewlett-Packard
* Intrado
* Narus
Nortel

SUPPORTING MEMBERS
* ADC

-----Original Message-----

a) ...