Subject: [ipfix-req] IPFix Summary

From: "Reinaldo Penno" <[email-masked]>
Date: Sun, 27 Oct 2002 14:30:54 -0800

I burned a good chunk of my weekend trying to summarize our discussions. If
you think I didn't capture what was said adequately, please speak up. This
email does not substitute Juergen's on open issues, but is actually trying
to close some of those open issues. (Should I copy the main ipfix list?)

I revised most of the emails starting from Jeff Meyer's - "Discussion of
Candidate Protocols" on - 9/30.

a)Regarding metering requirements and protocol candidates.

There is consensus that the metering requirements should be N/A when
choosing the IPfix Export Protocol.

A good summary was provided by Peter.

* The protocol should cover only the issues of delivering data from the
exporter to the collector. It must be expressive 
enough to contain all the data defined in the data model. 

* The data model should cover the identification of the metrics and
attributes, their meanings, and how they are grouped 
together. 

* The architecture should cover issues such as where data are observed, what
data are required and what are optional, when 
reliability is needed, how the data can be manipulated (e.g., calculating
bandwidth), etc., etc.

b) Regarding reliability

   - Congestion awareness. 

It is covered on the requirements draft

   - Message ordering (critical when the data spans several packets, e.g.
variable length fields) 

It seems there is consensus that message ordering is important.

   - Transport reliability - 

There is some confusion around this problem since transport reliability is
*not* explicit mentioned in the reqs draft, but 
since congestion awareness is, some people take it for granted. 

   - require application-level ACKs in the protocol (SHOULD be after
     data are persisted)

There is consensus that application layer ACKs is important

   - require enable de-duplication of received records through a unique key

There is consensus that de-duplication is important.

   - Exporter should keep track of number of flow records + some important
totals
     and and periodically communicate this information to the
     collector.

There is consensus that this or some other solution to capture the 
amount of lost *flow records for each key/field* (not packets) is important

c) Regarding High-Availability

we should have the ability to send to multiple collectors and for the 
exporter to be able to switch from one collector to another.

d) Regarding Timestamps

New text for section 5.4 (Disclaimer before somebody goes for my neck: this
affects the 
architecture/requirements, not the protocol evaluation.)

   The metering process MUST be able to generate wire-line timestamps 
   (rather then flow cache timestamps) for the first and the last 
   observed packet of a flow. The timestamp resolution MUST be at least 
   the one of the sysUpTime [RFC1213], which is one centisecond.

e) Regarding the Views on the Problem

Some people advocate that exporting IP flows has nothing unique to it, it's
well known problem 
only with a different data model. I hope I captured this right. Anyway, here
it goes a quotation.

"However, the statement that IP flow is somehow unique
in its data requirements and as such a more generalized
"data mover" is somehow problematic, is just plain wrong."

f) Regarding the Field Experience of IPDR

Some people wanted more information on this, so I'm pasting an email 
from Aron Heintz [[email-masked]]. Any more in-depth information please
direct your 
questions to Jeff Meyer [[email-masked]] or Aron Heitz.

"
-----Original Message-----
From: Aron Heintz [mailto:[email-masked]]
Sent: Thursday, October 10, 2002 3:47 PM
To: [email-masked]; Harrington, David; Jeff Meyer
Subject: RE: [ipfix-eval] Discussion of Candidate Protocols

 It seems to me that NDM-U wins the "free, open, and widely deployed" crown
hands down.  NDM-U 3.1 is completely free and publicly available (see
below).  By the end of this year, more than 70% of the mediation and billing
packages (by market share) will have proven NDM-U 3.1 *real-world*
interoperability.

 What could be more important than this?  It appears that some of you are
debating minor technical points when you might approach the question "Who is
going to receive the data we are sending?  How do they want to see it?"
Technologies do not win by virtue of their theoretical perfection.  The OSI
model is close to theoretical perfection.  TCP/IP is not.
"

---

From: "Reinaldo Penno" <[email-masked]>
Date: Tue, 29 Oct 2002 02:21:33 -0800

Hello Paul,

comments inline...

> From: [email-masked] [mailto:[email-masked]]
> Sent: Monday, October 28, 2002 6:43 AM
> To: Penno, Reinaldo [BL60:0430:EXCH]
> Cc: [email-masked]; [email-masked]
> Subject: Re: [ipfix-eval] IPFix Summary
> 
> 
> > Reinaldo Penno wrote:
> > 
> > I burned a good chunk of my weekend trying to summarize our
> > discussions. If you think I didn't capture what was said adequately,
> > please speak up. This email does not substitute Juergen's on open
> > issues, but is actually trying to close some of those open issues.
> > (Should I copy the main ipfix list?)
> > 
> > I revised most of the emails starting from Jeff Meyer's - 
> "Discussion
> > of Candidate Protocols" on - 9/30.
> > 
> > * The protocol should cover only the issues of delivering data from
> > the exporter to the collector. It must be expressive
> > 
> > * The architecture should cover issues such as where data are
> > observed, what data are required and what are optional, when
> > 
> > reliability is needed, how the data can be manipulated (e.g.,
> > calculating bandwidth), etc., etc.
> > 
> > b) Regarding reliability
> > 
> >    - Congestion awareness.
> > 
> >    - Message ordering (critical when the data spans several packets,
> > e.g. variable length fields)
> > 
> > There is some confusion around this problem since transport
> > reliability is *not* explicit mentioned in the reqs draft, but
> > 
> 	I'm concerned that we are going towards a one size fits
> 	all in terms of reliability. However, with added reliability 
> 	comes added overhead.  So what about those applications that
> 	do not need such reliability and would rather take the added
> 	capacity?
> 

I noticed some discussions around this topic (billing, which requires
reliability vs traffic profiling, which does not). That's a good point.
Would you take a stab and phrase it for the requirements draft so that the
result is not ambigous and convey the necessary message? 

thanks,

Reinaldo

> >    - Exporter should keep track of number of flow records + some
> > important totals
> 	I'm not sure what you mean here. Send each record to multiple 
> 	collectors or failover to another server when the current
> 	collector goes down?

failover.

> 
> 
> > d) Regarding Timestamps
> > 
> > New text for section 5.4 (Disclaimer before somebody goes 
> for my neck:
> > this affects the
> 	Making things like this a MUST does not help. Vendors
> 	wont or maybe can't change their hardware to comply. 
> 
> 	I would advocate having 2 different timestamps. One with 
> 	a meaning of wire-line timestamps and the other to mean
> 	cache timestamp.
> 
> 	Paul
> > 
> > Some people advocate that exporting IP flows has nothing 
> unique to it,
> > it's well known problem
> > only with a different data model. I hope I captured this right.
> > Anyway, here it goes a quotation.
> > 
> > from Aron Heintz [[email-masked]]. Any more in-depth information
> > please direct your
> >  It seems to me that NDM-U wins the "free, open, and widely 
> deployed"
> > crown
> >  What could be more important than this?  It appears that 
> some of you
> > are

---

From: "Reinaldo Penno" <[email-masked]>
Date: Tue, 29 Oct 2002 02:21:33 -0800

Hello Paul,

comments inline...

thanks,

Reinaldo

failover.