Subject: RE: [ipfix-req] Re: IPFIX Requirements Questions

From: "Carter Bullard" <[email-masked]>
Date: Thu, 26 Sep 2002 16:05:10 -0400

I apologize, and no ones fault, but e-mails with 5 levels of
indention are impossible to read.  Could you guys
make your points so we don't have to dig for 5 minutes
to find it.

Carter

Carter Bullard
QoSient, LLC
300 E. 56th Street, Suite 18K
New York, New York  10022

[email-masked]
Phone +1 212 588-9133
Fax   +1 212 588-9134
http://qosient.com

-----Original Message-----
From: majordomo listserver [mailto:[email-masked]] On
Behalf Of [email-masked]
Sent: Thursday, September 26, 2002 3:57 PM
To: Ganesh Sadasivan
Cc: req
Subject: Re: [ipfix-req] Re: IPFIX Requirements Questions

Ganesh Sadasivan wrote:
> 
> [email-masked] wrote:
> 
> > Ganesh Sadasivan wrote:
> > >
> > > [email-masked] wrote:
> > >
> > > > Ganesh Sadasivan wrote:
> > > > >
> > > > > [email-masked] wrote:
> > > > >
> > > > > > I wanted to summarize where we are with the issues I raised 
> > > > > > a while back. A few still need to be resolved.
> > > > > >
> > > > > > [email-masked] wrote:
> > > > > > >
> > > > > > > As I work on the advocates document some questions have 
> > > > > > > come to mind on the requirements document.
> > > > > > >
> > > > > > > 4.3 Distinguishing flows using Transport Header Fields
> > > > > > >
> > > > > > >         In the case of fragmented packets there are
> > > > > > >         no port numbers. Are we saying flow state
information
> > > > > > >         MUST be maintained? In general, it is not clear
from
> > > > > > >         the document what the requirements are for
fragmented
> > > > > > >         packets.
> > > > > >
> > > > > >         RESOLVED: State information MAY be kept for
> > > > > >                   proper flow designation.
> > > > > >
> > > > > > >
> > > > > > > 4.4 MPLS Label
> > > > > > >
> > > > > > >         In the case of dynamic LSP's the label is not
useful.
> > > > > > >         Why require distinguishing flows based on label.
> > > > > > >         What can be done with that information?
> > > > > >
> > > > > >         RESOLVED: We agreed to distinguish by label and to 
> > > > > > report label.
> > > > > >
> > > > > > >
> > > > > > > 5.3 Overload Behavior
> > > > > > >
> > > > > > >         This section needs to be re-worked a little. Some
> > > > > > >         of the MUST's impose undue burden on the device.
> > > > > > >
> > > > > > >         1. Must distinguish flow records before and after
the
> > > > > > >            behavior change.
> > > > > > >
> > > > > > >                 In the case where the behavior is to drop
> > > > > > >                 overflow flows, why do we need to
distinguish
> > > > > > >                 flows. Perhaps I misunderstood something.
> > > > > > >
> > > > > > >         2. All flows from previous behavior MUST be 
> > > > > > > terminated.
> > > > > > >
> > > > > > >
> > > > > > >                 Again, if I simply dropped excess flows
why terminate
> > > > > > >                 all existing ones. This will likely cause
more overflow
> > > > > > >                 as the get reestablished. This also seems
to go to
> > > > > > >                 far towards implementation.
> > > > > > >
> > > > > > >         3. Meeting process MUST NOT merge previous 
> > > > > > > records.
> > > > > > >
> > > > > > >         I think what we are trying to say is flows with a
different
> > > > > > >         definition MUST be distinguishable. How that is
done is not
> > > > > > >         part of the requirements doc.
> > > > > >
> > > > > >         OPEN
> > > > > >
> > > > > >         We agreed that not all behaviors require terminating

> > > > > > flows.
> > > > > >
> > > > > >         We still need more dialogue on the following
> > > > > >
> > > > > >         1. Do we need to define a term for the set of
properties
> > > > > >            that is used to distinguish flows.
> > > > >
> > > > > Is it enough if we give an exporter state along with the 
> > > > > export packet.
> > > >
> > > >         What do you mean by "exporter state"?
> > >
> > > Whether it is in "overload condition" , "normal" etc. Sorry for 
> > > being unclear. What I meant was, use the exporter state along with

> > > the other information  defined in section 4.0 to distinguish 
> > > flows. The methods that I mentioned below are just examples of 
> > > implementation.
> >
> >         I think this points to an area of weakness for us (IPFIX).
We are
> >         having a hard time having a discussion.
> >
> >         Lets say that DP = "Distinguising Properties". Where DP is
> >         the set of fields and functions, etc... that are used
> >         to define a flow. Lets assume for now that the set of
possible
> >         fields and functions, etc... is fully defined somewhere
> >         (it's not, but assume it is for now.).
> >
> >         If we require that each flow can be mapped to its DP
> >         then the discussion moves to how that happens.
> >
> >         There are many ways. Here are a few which come to mind...
> >
> 
> Does it matter how the DP is identified & send (whether with each flow

> or  split across flow & control or otherwise) is sent, from a req. doc

> perspective? As long as we define DP (which is what you are suggesting

> to do) and tell that DP must give enough information so that the full 
> flow context can be  derived at the collector, will it not solve this 
> issue?
> 

	Yes it will. I gave example to make sure we were on the
	same page. They would not go in the req doc.

> >
> >         1) send the DP with each flow.
> >         2) send the DP with an ID. Then each flow contains
> >            a DP-ID field.
> >         3) Send the DP for noraml operation and the DP for
> >            overload situation. Then only state changes need
> >            to be delivered.
> >
> >         #3 above seems to be what you are talking about.
> >
> >         However, I think we need to define DP as a term and the set
of
> >         possible fields and functions first. Add the requirement of
mapping
> >         a flow to a DP.
> 
> As I see DP is a function of  {1.metering process(s) employed,  2. 
> exporter state, 3. fields collected}.

	I don't see a direct relationship #1 and #2/#3. For example
	I could have a DP of source and dest IP but collect (i.e.
report)
	Src/Dst AS number. Also, the state may change from normal
	to overload with no change in DP but rather a dropping of
	additional flows.

	I think DP stands pretty much on its own. In other words
	fields collected and the state are not part of the DP
	definition. Those things may influence which DP is used
	but they are not part of the DP definition itself.

	But I think we are getting a little ahead of the game.
	We need to define DP and its fields and functions.

	Once we have consensus that we need to define DP and its
	fields and functions and we agree we need to map flow
	to DP, then we can kick off the discussion.

> There is a furthur inter dependency between
> * 2&1 (with the exporter state change metering could change),
> * 2 & 3 (with the exporter state change fields collected could 
> change),
> * 1 & 3 (fields are input tometering functions).
> 
> Am I missing anything?
> 
> > Then the discussion can move to how we
> >         can best do the mapping. Then the discussion of what happens
> >         under overload conditions will be clearer.
> >
> > >
> > > >
> > > >
> > > > > This assumes that a control infromation which explaining the 
> > > > > action taken corresponding to the exporter state has been send

> > > > > apriori.
> > > >
> > > >         Statement #1 above doesn't say anything about what is
sent
> > > >         when or where.
> > > >
> > > >         I'm asking more of a spec question. We don't define a
term for the
> > > >         set of properties that are used to uniquely identify a 
> > > > flow.
> > > >
> > > > > This way
> > > > > the flows itself need not be changed.
> > > > > The other way is to send a different flow.
> > > >
> > > >         This is the same as terminating exiting flows, which I'm
> > > >         trying to avoid.
> > >
> > > Why so?
> >
> >         Because you are going to send a new flow for all existing
flows
> >         during overload sitiation. It will probably not be 1 for 1
but
> >         rather many flows now being collapsed into one during 
> > overload.
> >
> >         Isn't that what you meant by "send a different flow".
> 
> Yes. But why do we not leave room for such a case?

	The current document says all flows must be terminated. I
	was pushing to allow for other alternatives, including
	terminating all flows.
> 
> >
> > >
> > > >
> > > >
> > > > > Even though the export record
> > > > > may look  the same, it is identified by a different id and the

> > > > > id->config mapping is assumed to have been send when the 
> > > > > overload condition started.
> > > > >
> > > > > >
> > > > > >
> > > > > >         2. Do we need to require that flows can be mapped to
the
> > > > > >            term defined in #1
> > > > >
> > > > > It depends on the apporach taken.
> > > >
> > > >         I don't think it does. Statement #2 say we need to ba
able to
> > > >         tell which flow mapped to which definition. It doesn't
> > > >         say anything about how that mapping is done, only that
> > > >         it can be done somehow.
> > > >
> > > > >
> > > > > >
> > > > > >
> > > > > > >
> > > > > > > 5.4 Timestamps
> > > > > > >
> > > > > > >         Time stamps to not always mapped to the first
packet and last
> > > > > > >         packet. In many cases, the end timestamp is merely
when
> > > > > > >         the flow timed out and has nothing to do with when
the
> > > > > > >         last packet was seen. Allowing both semantics may
be useful.
> > > > > > >         A re-wording like this...
> > > > > > >
> > > > > > >                 The metering process MUST be able to
generate timestamps for the
> > > > > > >                 start and end of a flow. The metering
process MAY also provide
> > > > > > >                 timestamps for the first and the last
observed packet
> > > > > > >                 of a flow. The timestamp resolution MUST
be at least the one of
> > > > > > >                 the sysUpTime [RFC1213], which is one 
> > > > > > > centisecond.
> > > > > > >
> > > > > > >         Note - this does not mandate 2 timestamp fields
for a flow. You
> > > > > > >         could have one element for each meaning (e.g.
Flow-start-time,
> > > > > > >         first-packet-time,
flow-start-and-first-packet-time) and use
> > > > > > >         the one with the desired meaning.
> > > > > >
> > > > > >         OPEN
> > > > > >
> > > > > >         This issue still needs more dialogue to gain 
> > > > > > clarity.
> > > > > >
> > > > > >         Ganesh believes it is possible to detect last packet
time within
> > > > > >         msec accuracy.
> > > > >
> > > > > Well, as you mentioned a while ago if some one implements an 
> > > > > ager which ages out a million cache entries in s/w, the answer

> > > > > is no. It seems to me it is not something a vendor would 
> > > > > venture into doing. It is a good time for some doing something

> > > > > in the similar lines to speak up;) Thanks Ganesh
> > > >
> > > >         Some vendors already do this. It is not a theoretical
problem
> > > >         but a real one. And you don't need a million flows for
it
> > > >         to be a problem. You could have just one flow. If the
> > > >         ager checks say every 30 seconds, then your end time has
> > > >         a maximum of a 30 second delay.
> > >
> > > Is that for some reason or is it a limitation in the 
> > > implementation? -ganesh
> >
> >         It is merely software based aging. Probably just an initial
> >         system design choice.
> 
> Are you indicating that there could be products in which the design 
> cannot be changed to suit the msec granularity timing requirements of 
> IPFIX?

	Yes, because if there is no current hardware support and there
	can be lots of flows then you don't have much choice but to
increase
	the aging time. That type of platform exists today.

	Paul

> -ganesh
> 
> >
> >
> >         Paul
> >
> > >
> > > >
> > > >
> > > >         Paul
> > > >
> > > > >
> > > > > >
> > > > > >
> > > > > >         I believe it is not possible without hardware assist
and not
> > > > > >         all platform will have that capability.
> > > > > >
> > > > > > >
> > > > > > > 6.1 information Model
> > > > > > >
> > > > > > >         9  Packet Counter
> > > > > > >         10 Byte counter
> > > > > > >
> > > > > > >                 As mentioned earlier, for fragments this
> > > > > > >                 requires state information? If there is no
> > > > > > >                 state info, then what flow are the counted
> > > > > > >                 towards?
> > > > > >
> > > > > >         RESOLVED(I think): no mention needs to me made in
the packet or byte
> > > > > >                            counter section since fragment
counting is a matter
> > > > > >                            for flow designation.
> > > > > >
> > > > > > >
> > > > > > >         13 BGP AS number
> > > > > > >
> > > > > > >                 Is this the AS number for the observation
point, the
> > > > > > >                 source/destination address in the packet
or some
> > > > > > >                 combination?
> > > > > >
> > > > > >         RESOLVED: These MAY be reported.
> > > > > >
> > > > > >                 1. Source IP AS
> > > > > >                 2. Destination IP AS
> > > > > >                 3. Observation Point AS (which may be a a
list if it
> > > > > >                                          participates on 
> > > > > > more than one)
> > > > > >
> > > > > >         I would like to propose 2 more
> > > > > >
> > > > > >                 Next hop AS
> > > > > >                 previous hop AS
> > > > >
> > > > > Why not the AS PATH?
> > > > >
> > > > > >
> > > > > >
> > > > > > >
> > > > > > >         14. MPLS Label
> > > > > > >
> > > > > > >                 As stated earlier, in the case of dynamic
LSP's the
> > > > > > >                 label has no meaning.
> > > > > > >
> > > > > > >         21 multicast replication number
> > > > > > >
> > > > > > >                 The document stated this can change over
the life
> > > > > > >                 of the flow. But no mention of what
happens when
> > > > > > >                 it changes. Is that a new flow? If not,
what meaning
> > > > > > >                 does it have when reported? What can I do 
> > > > > > > with it?
> > > > > >
> > > > > >         RESOLVED: If a new flow is desired, it becomes part
of the
> > > > > >                 "Distinguishing Properties". If not, then it
can still
> > > > > >                 be reported and ignored when it changes.
> > > > > > >
> > > > > > >
> > > > > > > 6.2 Data Model
> > > > > > >
> > > > > > >         What about allowing Vendors to add information
independently?
> > > > > > >         Some data to be transported may be vendor specific
and never
> > > > > > >         make it into the IPFIX spec.
> > > > > >
> > > > > >         RESOLVED: It is allowed now and we do not need to
mention
> > > > > >                 it specifically.
> > > > > >
> > > > > > >
> > > > > > > 8.3 Several Collecting Processes
> > > > > > >
> > > > > > >         No mention is made when reporting to several
devices
> > > > > > >         of ensuring the duplicate data does not cause a
double
> > > > > > >         counting problem.
> > > > > >
> > > > > >         RESOLVED: This requirement needs to be added.
> > > > > >
> > > > > > >
> > > > > > > 9. Special Device Considerations
> > > > > > >
> > > > > > >         I'm not sure of the meaning for the following...
> > > > > > >
> > > > > > >         ...
> > > > > > >
> > > > > > >    Please note that here, the observation
> > > > > > >    point of a single flow cannot exceed the set of most
fine-granular
> > > > > > >    observation points linked to a single metering process,
because only
> > > > > > >    the metering process can merge packets observed at
different fine-
> > > > > > >    granular observation points to a joint flow.
> > > > > > >
> > > > > > >         ...
> > > > > > >
> > > > > > >    Also the
> > > > > > >    locations of metering processes are not of any
relevance for this
> > > > > > >    document (in contrast to the locations of observation
points and the
> > > > > > >    exporting processes).
> > > > > > >
> > > > > >
> > > > > >         These were just for my understanding.
> > > > > >
> > > > > > > Paul
> > > > > >
> > > > > > --
> > > > > > Help        mailto:[email-masked] and say
"help" in message body
> > > > > > Unsubscribe mailto:[email-masked] and say 
> > > > > > "unsubscribe ipfix" in message body
> > > > > > Archive     http://ipfix.doit.wisc.edu/archive/
> >
> > --
> > Help        mailto:[email-masked] and say "help" in
message body
> > Unsubscribe mailto:[email-masked] and say "unsubscribe 
> > ipfix" in message body
> > Archive     http://ipfix.doit.wisc.edu/archive/

--
Help        mailto:[email-masked] and say "help" in message
body
Unsubscribe mailto:[email-masked] and say "unsubscribe
ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/

--
Help        mailto:[email-masked] and say "help" in message body
Unsubscribe mailto:[email-masked] and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/