Subject: RE: [ipfix] Multiple levels of Reliability and Directionality

From: "Reinaldo Penno" <[email-masked]>
Date: Thu, 7 Nov 2002 10:12:31 -0800

yep, good idea. 

> -----Original Message-----
> From: Tal Givoly [mailto:[email-masked]]
> Sent: Thursday, November 07, 2002 12:29 PM
> To: [email-masked]; [email-masked]; 'ipfixx'
> Subject: RE: [ipfix] Multiple levels of Reliability and Directionality
> 
> 
> Paul,
> 
> I agree completely - that captures the essense of the requirements. It
> allows balancing different application's need for different levels of
> reliability.
> 
> Tal
> 
> -----Original Message-----
> From: majordomo listserver 
> [mailto:[email-masked]]On Behalf
> Of Carter Bullard
> Sent: Thursday, November 07, 2002 8:40 AM
> To: [email-masked]; 'ipfixx'
> Subject: RE: [ipfix] Multiple levels of Reliability and Directionality
> 
> 
> Hey Paul,
>   My 2 cents worth sez, absolutely yes!
> Carter
> 
> > -----Original Message-----
> > From: majordomo listserver
> > [mailto:[email-masked]] On Behalf Of
> > [email-masked]
> > Sent: Thursday, November 07, 2002 9:46 AM
> > To: ipfixx
> > Subject: [ipfix] Multiple levels of Reliability and Directionality
> >
> >
> >
> > Does IPFIX need to support multiple levels of reliability
> > and directionality?
> >
> > The applications IPFIX needs to support are quite varied
> > and have differing needs in terms of reliability. Devices
> > also have varying needs in terms of directionality.
> >
> > I would like the WG to consider requirements on the protocol
> > to support a tiered approach meet these needs.
> >
> > Direction is either bi-directional or uni-directional. I've split
> > reliability into 3 categories...
> >
> >         High   - Little to no data loss
> >         Medium - Minimal data loss is accepted
> >         Low    - Data sent is best effort
> >
> >
> > The six combinations are shown below...
> >
> >
> >                | Uni  |  Bi
> >         -------+------+---------
> >         High   |  1   |    4
> >         -------+------+--------
> >         Medium |  2   |    5
> >         -------+------+--------
> >         Low    |  3   |    6
> >
> >
> >
> > I think (IMHO) the most likely combinations are...
> >
> >         3. Low reliability and uni-directional. This will be used
> >            where the device and collector are in close proximity,
> >            there is high volume and some data loss is acceptable for
> >            the given application. Applications such as
> > Attack/Intrusion Detection
> >            Traffic Profiling, etc...
> >
> >         5. Bi-directional medium reliability. This will be used
> >            where high volume and increased reliability are
> > needed. Applications
> >            such as Usage-based Accounting where 95th
> > percentile billing
> >            model is used. Or even Traffic Engineering where
> > network policy
> >            decisions are based on the data collected.
> >
> >         4. Bi-directional high reliability would be used where the
> >            Usage-based Accounting application has strict 
> requirements
> >            on data loss.
> >
> > Does IPFIX need to support multiple levels of reliability
> > and directionality?
> >
> > Paul
> >
> > --
> > Help        mailto:[email-masked] and say "help"
> > in message body
> > Unsubscribe mailto:[email-masked] and say
> > "unsubscribe ipfix" in message body
> > Archive     http://ipfix.doit.wisc.edu/archive/
> >
> 
> 
> 
> --
> Help        mailto:[email-masked] and say "help" 
> in message
> body
> Unsubscribe mailto:[email-masked] and say
> "unsubscribe ipfix" in message body
> Archive     http://ipfix.doit.wisc.edu/archive/
> 
> 
> --
> Help        mailto:[email-masked] and say "help" 
> in message body
> Unsubscribe mailto:[email-masked] and say
> "unsubscribe ipfix" in message body
> Archive     http://ipfix.doit.wisc.edu/archive/
>

---

From: "Reinaldo Penno" <[email-masked]>
Date: Fri, 15 Nov 2002 08:12:42 -0800

comments inline...

> From: [email-masked] [mailto:[email-masked]]
> Sent: Friday, November 15, 2002 8:10 AM
> To: Benoit Claise
> Cc: Tal Givoly; Jeff Meyer; [email-masked]; 'ipfixx'
> Subject: Re: [ipfix] Multiple levels of Reliability and Directionality
> 
> 
> Benoit Claise wrote:
> > 
> > Paul,
> > 
> > First of all, let me take back your initial email, for clarity
> > purposes
> > 
> > Attack/Intrusion
> >            Detection Traffic Profiling, etc...
> > 
> >            where high volume and increased reliability are needed.
> >            Applications
> >            such as Usage-based Accounting where 95th percentile
> > billing
> >            model is used. Or even Traffic Engineering where network
> >            policy decisions are based on the data collected.
> > 
> > I agree on 3, this is the minimum.
> > 
> > I agree on 5 but I fear that the application level ACK will 
> not always
> > be a possible solution from a router point of view. What if 
> the router
> > exports so much flow records that it can't cache them while waiting
> > for the ACK.
> > So basically, resending the flow records is not always possible.
> > So, if you conclude that the requirement for 5 is the 
> bidirectionality
> > with application level ACK, then this requirement is not always
> > possible!
> 
> 	Agreed. In very high volume deployments I would 
> 	use option #3.
> 	
> > IMHO, 5 could alos be
> >         2 with best-effort
> >         or 2 with a carefully designed network
> 
> 	If the customer can design their accounting network
> 	in such away. But I'm not sure we want to require that.
> 	
> 	Lets assume that aggregation is happening at
> 	a reasonably high level (which drops the flow rate
> 	to a manageable level). But the consequence is that
> 	one lost record can be a huge chunk of missing data.
> 
> 	In this case reliability is the issue not the data
> 	rate. I'm advocating a reliability option (note I said option)
> 	as a way to solve this.	
> 	
> 
> > 
> > Regarding 4, Randy replied already
> > 
> >      > Somehow, the requirements discussion suffers from a split:
> >      > There are requirements resulting from the intention of
> >      > standardzing a highly reliable metering standard for
> >      precise
> >      > accounting (and billing). Differing from these are
> >      requirements
> >      > for a simpler metering standard more oriented at what I
> >      would
> >      > call "typical Internet reliability".
> > 
> >      <ad> the original goal, which the iesg thinks was chartered,
> >      is the
> >      latter.
> > 
> 
> 	There was a lot of push back by the group on that statement.
> 	I asked for clarification but have not received any.

IMO the statement included generalizations such as  "in my world, we use
octet counting for that kind of application" and "typical Internet
reliability" that several people questioned.

and then in a following email

"some _current_ inter-vendor incompatibilities,
fix congestion control, etc"

and this might go more into the core of problem. We have several "current"
vendors with somewhat distinct protocols. I should also say different
providers (based on some emails to the list) with different needs. What kind
of compromise can be achieved knowing that there are people that want
relible billing accounting (per flows, applications, etc) and some people
that just want traffic profiling and we still end up with standard widely
adopted? 

-RP

> 
> 	Paul
> 
> > > Let me see if we all agree on a couple things...
> > >
> > >         1. There is no such thing as partially uni-directional.
> > >         2. IPFIX ought to support a uni-directional mode of
> > > operation.
> > >
> > Agreed.
> > 
> > Regards, Benoit.
> > 
> > >
> > > Paul
> > >
> > > Tal Givoly wrote:
> > >
> > >
> > >> Jeff,
> > >>
> > >> I agree with you completely here - the protocol should focus on a
> > >> primarily
> > >> uni-directional feed from network/service elements
> > >> (metering+exporting
> > >> process) to collection process. Adding capabilities (such as
> > >> parameter
> > >> configuration, provisioning of other sorts, etc.) here is a
> > >> slippery slope.
> > >>
> > >> I would add, however, that the bi-directionality is needed both
> > >> for
> > >> assurance that messages are delivered as well as, IMHO, version,
> > >> capability
> > >> and template negotiation (which can be simplified as needed by
> > >> device).
> > >> Version and capability negotiation allows for forward/backward
> > >> interoperability and compatibility. Template negotiation allows
> > >> for more
> > >> efficient application-aware communication. As Peter explained
> > >> earlier
> > >> (http://ipfix.doit.wisc.edu/archive/1381.html - scenario #1 or
> > >> #2), it is
> > >> possible to have all these capabilities and have no impact on the
> > >> metering/export processes complexity - the complexity can be,
> > >> optionally,
> > >> added to the collection process (off the device) in 
> order to allow
> > >> for
> > >> interoperability with multiple concurrent
> > >> versions/capabilities/templates in
> > >> the devices.
> > >>
> > >> Essentially, I believe we achieved consensus regarding the need
> > >> for template
> > >> capabilities to allow separation of the data model from the
> > >> protocol.
> > >> Version/capability negotiation are very useful for any 
> protocol in
> > >> order to
> > >> increase its applicability and useful lifespan.
> > >>
> > >> Tal
> > >>
> > >> From: Jeff Meyer [mailto:[email-masked]]
> > >> Sent: Monday, November 11, 2002 9:39 AM
> > >> To: [email-masked]
> > >> Cc: Tal Givoly; [email-masked]; 'ipfixx'
> > >> Subject: Re: [ipfix] Multiple levels of Reliability and
> > >> Directionality
> > >>
> > >> OK, then based on this minimal definition of
> > >> bi-directional, I'm in complete agreement.
> > >>
> > >> It's the sliperry slope of other things one might
> > >> want to exchange once this two way channel is open,
> > >> which concerns me.  There are lots of things one
> > >> might want to exchange, but many of these are probably
> > >> better addressed by reusing existing protocols,
> > >> vs. stuffing into the IPFIX protocol.
> > >>
> > >> For example, controlling metering process is something
> > >> which can be separated out from IPFIX.  For example
> > >> by using SNMP and a MeterMIB.
> > >>
> > >> Just trying to apply the KISS (Keep it simple stupid)
> > >> principle.
> > >>
> > >> Regards,
> > >>
> > >>    Jeff Meyer
> > >>    hp
> > >>
> > >> [email-masked] wrote:
> > >>
> > >>
> > >>
> > >> > Jeff Meyer wrote:
> > >> >
> > >> >
> > >> >
> > >> >>  Hi,
> > >> >>
> > >> >>    I agree with the aspect of varied reliability.
> > >> >>
> > >> >>    I am less clear on the need for bi-directional
> > >> >>  behavior.  The three scenarios you describe I
> > >> >>  agree with, but I'm not sure how the 2nd and
> > >> >>  3rd scenario as described have any dependence
> > >> >>  on bi-directional aside from the some level of
> > >> >>  application layer ack.
> > >> >>
> > >> >>
> > >> >>
> > >> >>
> > >> >       An application layer ACK all by itself makes the
> > >> >       protocol bi-directional.
> > >> >
> > >> >       Scenario #1 does not require the sender to also be a
> > >> > receiver.
> > >> >
> > >> >       Scenarios #2 & #3 do require the sender to also be a
> > >> >       receiver (even if only for ACK's).
> > >> >
> > >> >       Whether or not we decide to add other features in
> > >> >       scenarios #2 & #3 (such as field negotiation, capability
> > >> >       negotiation, etc...) is up to the WG. But they could
> > >> >       not be added to scenario #1.
> > >> >
> > >> >       Paul
> > >> >
> > >> >
> > >> >
> > >> >
> > >> >
> > >> >>  Regards,
> > >> >>
> > >> >>    Jeff Meyer
> > >> >>
> > >> >>  Tal Givoly wrote:
> > >> >>
> > >> >>
> > >> >>
> > >> >>
> > >> >> > Paul,
> > >> >> >
> > >> >> > From: majordomo listserver [
> > >> >> > mailto:[email-masked]]On Behalf
> > >> >> > Of Carter Bullard
> > >> >> >>  Direction is either bi-directional or 
> uni-directional. I've
> > >> >> >>  split
> > >> >> >>        4. Bi-directional high reliability would be 
> used where
> > >> >> >>  the
> > > Help        mailto:[email-masked] and say "help" in
> > > message body

---

From: "Carter Bullard" <[email-masked]>
Date: Mon, 18 Nov 2002 17:59:01 -0500

Hey Dave,
Hope all is well!  Yes, I apologize for my humor, but
I thought I was being funny, as I never thought capability
negotiation was a requirement.   Of course I should know
better, so I apologize to the list.  I'll try to remember
to put the ole ;0) at the end next time!!!!

Sorry I won't make it to Hotlanta for the meeting,
as I've been glued to the couch since my foot surgery
in Sept.  Atlanta is my home town, mom and siblings are
all there, and so I'll be missing more than just an IETF
event.  Hopefully I'll make the next one, hopefully without
the cast and crutches.

I believe that you guys should make good progress
in coming to some decisions on the protocol.  My vote
is to keep it simple, avoid specifying an IPFIX probe,
and not getting in the way of vendors using IPFIX to
transport new types of IP flow metrics.  It would be a
shame to go to all this effort and not be able to support
the next wave of flow technology.  If there is anyway to
make IPFIX reliable, that would be great, but not at the
expense of getting a usable protocol out the door.  There
is always V2 of everything.

Have fun, and be sure and try the Varsity for fast hamburgers
and hot dogs, its about 10 blocks from the hotel, just across
the Interstate from Georgia Tech.   Mostly a cultural
experience, and the hot dogs are very good.

Best Regards!!

Carter

> [mailto:[email-masked]] On Behalf Of Dave Plonka
> Sent: Monday, November 18, 2002 2:49 PM
> To: [email-masked]
> Cc: Carter Bullard
> Hi Carter,
> On Mon, Nov 18, 2002 at 02:26:52PM -0500, Carter Bullard wrote:
> >    It may be of interest for the group to look at
> > RFC-3340 which deals with the issues of generic application
> > transport and capability negotiation, as well as reliability.
> > http://www.ietf.org/rfc/rfc3340.txt?number=3340.
> > 
> > Doesn't seem that hard.  When did IPFIX drop capability
> > negotiation as a requirement?
> 
> I don't believe IPFIX ever had capability negotiation as a 
> requirement.
> 
> Quoting from "http://ipfix.doit.wisc.edu/IETF53/minutes.txt":
> 
>    Regarding capability negotiation, consensus amongst attendees (by a
>    hum) was that neither exporter configuration nor capabilities
>    negotiation between collectors and exporters should be addressed by
>    IPFIX.  The chairs will follow-up with a call on the 
> mailing list on
>    this topic.
> 
> I was remiss in not following up beyond posting the minutes 
> to the list
> for review, but AFAIK the issue has not been contentious given our
> "standardize existing practice" mantra and that negotiation 
> hasn't been
> substantively mentioned since before IETF53, when Juergen posted his
> concerns about negotiation feature creep:
> 
   http://ipfix.doit.wisc.edu/archive/0461.html

to which there were no follow-ups.

Dave

-- 
[email-masked]  http://net.doit.wisc.edu/~plonka  ARS:N9HZF
Madison, WI

--
Help        mailto:[email-masked] and say "help" in message
body
Unsubscribe mailto:[email-masked] and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/

--
Help        mailto:[email-masked] and say "help" in message body
Unsubscribe mailto:[email-masked] and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/