Subject: RE: [ipfix] Multiple levels of Reliability and Directionality

From: "Paul Calato" <[email-masked]>
Date: Mon, 18 Nov 2002 13:35:04 -0800

-----Original Message----- 
	From: Ganesh Sadasivan [mailto:[email-masked]] 
	Sent: Fri 11/15/2002 11:58 AM 
	To: Paul Calato 
	Cc: Benoit Claise; Tal Givoly; Jeff Meyer; [email-masked]; 'ipfixx' 
	Subject: Re: [ipfix] Multiple levels of Reliability and Directionality

	[email-masked] wrote:
	
	> Benoit Claise wrote:
	> >
	> > Paul,
	> >
	> > First of all, let me take back your initial email, for clarity
	> > purposes
	> >
	> >         High   - Little to no data loss
	> >         Medium - Minimal data loss is accepted
	> >         Low    - Data sent is best effort
	> >
	> > The six combinations are shown below...
	> >
	> >                | Uni  |  Bi
	> >         -------+-----+--------
	> >         High   |  1   |    4
	> >         -------+-----+-------
	> >         Medium |  2   |    5
	> >         -------+-----+-------
	> >         Low    |  3   |    6
	> >
	> > I think (IMHO) the most likely combinations are...
	> >
	> >         3. Low reliability and uni-directional. This will be used
	> >            where the device and collector are in close proximity,
	> >            there is high volume and some data loss is acceptable for
	> >            the given application. Applications such as
	> > Attack/Intrusion
	> >            Detection Traffic Profiling, etc...
	> >
	> >         5. Bi-directional medium reliability. This will be used
	> >            where high volume and increased reliability are needed.
	> >            Applications
	> >            such as Usage-based Accounting where 95th percentile
	> > billing
	> >            model is used. Or even Traffic Engineering where network
	> >            policy decisions are based on the data collected.
	> >
	> >         4. Bi-directional high reliability would be used where the
	> >            Usage-based Accounting application has strict requirements
	> >            on data loss.
	> >
	> > I agree on 3, this is the minimum.
	> >
	> > I agree on 5 but I fear that the application level ACK will not always
	> > be a possible solution from a router point of view. What if the router
	> > exports so much flow records that it can't cache them while waiting
	> > for the ACK.
	> > So basically, resending the flow records is not always possible.
	> > So, if you conclude that the requirement for 5 is the bidirectionality
	> > with application level ACK, then this requirement is not always
	> > possible!
	>
	>         Agreed. In very high volume deployments I would
	>         use option #3.
	>
	> > IMHO, 5 could alos be
	> >         2 with best-effort
	> >         or 2 with a carefully designed network
	>
	>         If the customer can design their accounting network
	>         in such away. But I'm not sure we want to require that.
	>
	>         Lets assume that aggregation is happening at
	>         a reasonably high level (which drops the flow rate
	>         to a manageable level). But the consequence is that
	>         one lost record can be a huge chunk of missing data.
	>
	>         In this case reliability is the issue not the data
	>         rate. I'm advocating a reliability option (note I said option)
	>         as a way to solve this.
	>
	>
	> >
	> > Regarding 4, Randy replied already
	> >
	> >      > Somehow, the requirements discussion suffers from a split:
	> >      > There are requirements resulting from the intention of
	> >      > standardzing a highly reliable metering standard for
	> >      precise
	> >      > accounting (and billing). Differing from these are
	> >      requirements
	> >      > for a simpler metering standard more oriented at what I
	> >      would
	> >      > call "typical Internet reliability".
	> >
	> >      <ad> the original goal, which the iesg thinks was chartered,
	> >      is the
	> >      latter.
	> >
	>
	>         There was a lot of push back by the group on that statement.
	>         I asked for clarification but have not received any.
	
	I feel the base line IPFIX protocol does require just 3 or 6. This would
	provide the simplest and the most efficient means for all applications.
	For those applications like billing with more stringent requirements,
	why can't extensions be built on top of the base protocol as a higher
	level application? By doing this we can cover the entire matrix.
	There is a vast majority of apps that need just the 3.
	and building a multi-level reliable scheme into the basre IPFIX protocol
	would result in a less efficient use of the protocol for apps that really
	don't care for this.

	[paul] If we have a simple, uni-directional unreliable base (#3) then adding reliability

	on top of that base can still be part of the IPFIX working group. It would seem there

	are important applications which need that feature, there is support on the list for specifying

	it and it seems there are valid deployment scenarios where it would be used (e.g. aggregated

	records).

	Would you support an approach with #3 as a base and  #4 and #5 as additions? I think the

	key here is really keeping #3 as the base and #4 and #5 as enhancements. For example,

	if #3 solves passing the template info #4 and #5 cannot change how that is done. Otherwise

	it would lead to too much divergence.

	[Paul-end]
	
	Thanks
	Ganesh

	>
	>
	>         Paul
	>
	> > > Let me see if we all agree on a couple things...
	> > >
	> > >         1. There is no such thing as partially uni-directional.
	> > >         2. IPFIX ought to support a uni-directional mode of
	> > > operation.
	> > >
	> > Agreed.
	> >
	> > Regards, Benoit.
	> >
	> > >
	> > > Paul
	> > >
	> > > Tal Givoly wrote:
	> > >
	> > >
	> > >> Jeff,
	> > >>
	> > >> I agree with you completely here - the protocol should focus on a
	> > >> primarily
	> > >> uni-directional feed from network/service elements
	> > >> (metering笚梮���
	> > >> process) to collection process. Adding capabilities (such as
	> > >> parameter
	> > >> configuration, provisioning of other sorts, etc.) here is a
	> > >> slippery slope.
	> > >>
	> > >> I would add, however, that the bi-directionality is needed both
	> > >> for
	> > >> assurance that messages are delivered as well as, IMHO, version,
	> > >> capability
	> > >> and template negotiation (which can be simplified as needed by
	> > >> device).
	> > >> Version and capability negotiation allows for forward/backward
	> > >> interoperability and compatibility. Template negotiation allows
	> > >> for more
	> > >> efficient application-aware communication. As Peter explained
	> > >> earlier
	> > >> (http://ipfix.doit.wisc.edu/archive/1381.html - scenario #1 or
	> > >> #2), it is
	> > >> possible to have all these capabilities and have no impact on the
	> > >> metering/export processes complexity - the complexity can be,
	> > >> optionally,
	> > >> added to the collection process (off the device) in order to allow
	> > >> for
	> > >> interoperability with multiple concurrent
	> > >> versions/capabilities/templates in
	> > >> the devices.
	> > >>
	> > >> Essentially, I believe we achieved consensus regarding the need
	> > >> for template
	> > >> capabilities to allow separation of the data model from the
	> > >> protocol.
	> > >> Version/capability negotiation are very useful for any protocol in
	> > >> order to
	> > >> increase its applicability and useful lifespan.
	> > >>
	> > >> Tal
	> > >>
	> > >> -----Original Message-----
	> > >> From: Jeff Meyer [mailto:[email-masked]]
	> > >> Sent: Monday, November 11, 2002 9:39 AM
	> > >> To: [email-masked]
	> > >> Cc: Tal Givoly; [email-masked]; 'ipfixx'
	> > >> Subject: Re: [ipfix] Multiple levels of Reliability and
	> > >> Directionality
	> > >>
	> > >> OK, then based on this minimal definition of
	> > >> bi-directional, I'm in complete agreement.
	> > >>
	> > >> It's the sliperry slope of other things one might
	> > >> want to exchange once this two way channel is open,
	> > >> which concerns me.  There are lots of things one
	> > >> might want to exchange, but many of these are probably
	> > >> better addressed by reusing existing protocols,
	> > >> vs. stuffing into the IPFIX protocol.
	> > >>
	> > >> For example, controlling metering process is something
	> > >> which can be separated out from IPFIX.  For example
	> > >> by using SNMP and a MeterMIB.
	> > >>
	> > >> Just trying to apply the KISS (Keep it simple stupid)
	> > >> principle.
	> > >>
	> > >> Regards,
	> > >>
	> > >>    Jeff Meyer
	> > >>    hp
	> > >>
	> > >> [email-masked] wrote:
	> > >>
	> > >>
	> > >>
	> > >> > Jeff Meyer wrote:
	> > >> >
	> > >> >
	> > >> >
	> > >> >>  Hi,
	> > >> >>
	> > >> >>    I agree with the aspect of varied reliability.
	> > >> >>
	> > >> >>    I am less clear on the need for bi-directional
	> > >> >>  behavior.  The three scenarios you describe I
	> > >> >>  agree with, but I'm not sure how the 2nd and
	> > >> >>  3rd scenario as described have any dependence
	> > >> >>  on bi-directional aside from the some level of
	> > >> >>  application layer ack.
	> > >> >>
	> > >> >>
	> > >> >>
	> > >> >>
	> > >> >       An application layer ACK all by itself makes the
	> > >> >       protocol bi-directional.
	> > >> >
	> > >> >       Scenario #1 does not require the sender to also be a
	> > >> > receiver.
	> > >> >
	> > >> >       Scenarios #2 & #3 do require the sender to also be a
	> > >> >       receiver (even if only for ACK's).
	> > >> >
	> > >> >       Whether or not we decide to add other features in
	> > >> >       scenarios #2 & #3 (such as field negotiation, capability
	> > >> >       negotiation, etc...) is up to the WG. But they could
	> > >> >       not be added to scenario #1.
	> > >> >
	> > >> >       Paul
	> > >> >
	> > >> >
	> > >> >
	> > >> >
	> > >> >
	> > >> >>  Regards,
	> > >> >>
	> > >> >>    Jeff Meyer
	> > >> >>
	> > >> >>  Tal Givoly wrote:
	> > >> >>
	> > >> >>
	> > >> >>
	> > >> >>
	> > >> >> > Paul,
	> > >> >> >
	> > >> >> > I agree completely - that captures the essense of the
	> > >> >> > requirements. It
	> > >> >> > allows balancing different application's need for different
	> > >> >> > levels of
	> > >> >> > reliability.
	> > >> >> >
	> > >> >> > Tal
	> > >> >> >
	> > >> >> > -----Original Message-----
	> > >> >> > From: majordomo listserver [
	> > >> >> > mailto:[email-masked]]On Behalf
	> > >> >> > Of Carter Bullard
	> > >> >> > Sent: Thursday, November 07, 2002 8:40 AM
	> > >> >> > To: [email-masked]; 'ipfixx'
	> > >> >> > Subject: RE: [ipfix] Multiple levels of Reliability and
	> > >> >> > Directionality
	> > >> >> >
	> > >> >> >
	> > >> >> > Hey Paul,
	> > >> >> >  My 2 cents worth sez, absolutely yes!
	> > >> >> > Carter
	> > >> >> >
	> > >> >> >
	> > >> >> >
	> > >> >> >
	> > >> >> >
	> > >> >> >>  -----Original Message-----
	> > >> >> >>  From: majordomo listserver
	> > >> >> >>  [mailto:[email-masked]] On Behalf Of
	> > >> >> >>  [email-masked]
	> > >> >> >>  Sent: Thursday, November 07, 2002 9:46 AM
	> > >> >> >>  To: ipfixx
	> > >> >> >>  Subject: [ipfix] Multiple levels of Reliability and
	> > >> >> >>  Directionality
	> > >> >> >>
	> > >> >> >>
	> > >> >> >>
	> > >> >> >>  Does IPFIX need to support multiple levels of reliability
	> > >> >> >>  and directionality?
	> > >> >> >>
	> > >> >> >>  The applications IPFIX needs to support are quite varied
	> > >> >> >>  and have differing needs in terms of reliability. Devices
	> > >> >> >>  also have varying needs in terms of directionality.
	> > >> >> >>
	> > >> >> >>  I would like the WG to consider requirements on the protocol
	> > >> >> >>  to support a tiered approach meet these needs.
	> > >> >> >>
	> > >> >> >>  Direction is either bi-directional or uni-directional. I've
	> > >> >> >>  split
	> > >> >> >>  reliability into 3 categories...
	> > >> >> >>
	> > >> >> >>        High   - Little to no data loss
	> > >> >> >>        Medium - Minimal data loss is accepted
	> > >> >> >>        Low    - Data sent is best effort
	> > >> >> >>
	> > >> >> >>
	> > >> >> >>  The six combinations are shown below...
	> > >> >> >>
	> > >> >> >>
	> > >> >> >>               | Uni  |  Bi
	> > >> >> >>        -------+-----+--------
	> > >> >> >>        High   |  1   |    4
	> > >> >> >>        -------+-----+-------
	> > >> >> >>        Medium |  2   |    5
	> > >> >> >>        -------+-----+-------
	> > >> >> >>        Low    |  3   |    6
	> > >> >> >>
	> > >> >> >>
	> > >> >> >>
	> > >> >> >>  I think (IMHO) the most likely combinations are...
	> > >> >> >>
	> > >> >> >>        3. Low reliability and uni-directional. This will be
	> > >> >> >>  used
	> > >> >> >>           where the device and collector are in close
	> > >> >> >>  proximity,
	> > >> >> >>           there is high volume and some data loss is
	> > >> >> >>  acceptable for
	> > >> >> >>           the given application. Applications such as
	> > >> >> >>  Attack/Intrusion Detection
	> > >> >> >>           Traffic Profiling, etc...
	> > >> >> >>
	> > >> >> >>        5. Bi-directional medium reliability. This will be
	> > >> >> >>  used
	> > >> >> >>           where high volume and increased reliability are
	> > >> >> >>  needed. Applications
	> > >> >> >>           such as Usage-based Accounting where 95th
	> > >> >> >>  percentile billing
	> > >> >> >>           model is used. Or even Traffic Engineering where
	> > >> >> >>  network policy
	> > >> >> >>           decisions are based on the data collected.
	> > >> >> >>
	> > >> >> >>        4. Bi-directional high reliability would be used where
	> > >> >> >>  the
	> > >> >> >>           Usage-based Accounting application has strict
	> > >> >> >>  requirements
	> > >> >> >>           on data loss.
	> > >> >> >>
	> > >> >> >>  Does IPFIX need to support multiple levels of reliability
	> > >> >> >>  and directionality?
	> > >> >> >>
	> > >> >> >>  Paul
	> > >> >> >>
	> > >> >> >>  --
	> > >> >> >>  Help        mailto:[email-masked] and say
	> > >> >> >>  "help"
	> > >> >> >>  in message body
	> > >> >> >>  Unsubscribe mailto:[email-masked] and say
	> > >> >> >>  "unsubscribe ipfix" in message body
	> > >> >> >>  Archive     http://ipfix.doit.wisc.edu/archive/
	> > >> >> >>
	> > >> >> >>
	> > >> >> >>
	> > >> >> >>
	> > >> >> >>
	> > >> >> > --
	> > >> >> > Help        mailto:[email-masked] and say "help"
	> > >> >> > in message
	> > >> >> > body
	> > >> >> > Unsubscribe mailto:[email-masked] and say
	> > >> >> > "unsubscribe ipfix" in message body
	> > >> >> > Archive     http://ipfix.doit.wisc.edu/archive/
	> > >> >> >
	> > >> >> >
	> > >> >> > --
	> > >> >> > Help        mailto:[email-masked] and say "help"
	> > >> >> > in message
	> > >> >> >
	> > >> >> >
	> > >> body
	> > >>
	> > >>
	> > >> >> > Unsubscribe mailto:[email-masked] and say
	> > >> >> > "unsubscribe ipfix" in message body
	> > >> >> > Archive     http://ipfix.doit.wisc.edu/archive/
	> > >> >> >
	> > >> >> >
	> > >> >> >
	> > >> >> >
	> > > --
	> > > Help        mailto:[email-masked] and say "help" in
	> > > message body
	> > > Unsubscribe mailto:[email-masked] and say
	> > > "unsubscribe ipfix" in message body
	> > > Archive     http://ipfix.doit.wisc.edu/archive/
	> > >
	> > >
	>
	> --
	> Help        mailto:[email-masked] and say "help" in message body
	> Unsubscribe mailto:[email-masked] and say
	> "unsubscribe ipfix" in message body
	> Archive     http://ipfix.doit.wisc.edu/archive/

��ޖ���[h�f���h���z�ݢ+��+��njwlk/�zZ��y��Ơy��I칻�&ޙ�����j:+v���w���"��"���v�vƲ��칻�&ފ�������,j��܀��b��m����*_�ݢ+��+��n��܆+�

---

From: Ganesh Sadasivan <[email-masked]>
Date: Tue, 19 Nov 2002 15:46:24 -0800

>         it and it seems there are valid deployment scenarios where it would be used (e.g. aggregated

What I meant here is that a system implementing #3 alone should be sufficient enough
to be IPFIX compliant. Depending on validity and extensiveness of deployment one could
put in additional features on top of #3  . This list I guess would  be a growing one, as
different apps have different sets of constraints.
-ganesh

Agreed.

Thanks
Ganesh