Subject: Re: [ipfix] Mailing list digest, Friday 8 March

From: Benoit Claise <[email-masked]>
Date: Tue, 12 Mar 2002 15:15:46 +0100 (CET)

> > 
> > On 11 Mar, [email-masked] wrote:
> > >
> > >>    - Nevil, 4 Mar: Note setting out IESG / WH chair position on
> > >>      this issue.  In brief, if IPFIX doesn't mandate congestion-
> > >>      aware transport, the WG RFCs won't get approved.
> > >
> > >       I believe there are a large number of real world deployment
> > >       cases where IPFIX is not going over the Internet. In fact,
> > >       it is likely to be the majority of deployment cases. Is there
> > >       disagreement on this point?
> > >
> > >       If not, we ought to decide the best way to solve the problem
> > >       first, then decide what that means for us as an IETF group;
> > >       not the other way around.
> > 
> > As an IETF WG we're honour-bound to come up with an architecture/
> > protocol/whatever that's good for the Internet overall, that's why
> > we have to use congestion-aware transport.
> > 'Solving the problem,'
> > in a non-Internet setting, is not
> 
> 	You are missing my point. If the problem space is 95%
> 	in the non-internet setting and we solve it for the
> 	internet setting, we solve the wrong problem. What's
> 	the point of that? 
> 
> 	We should not ignore our most prevalent deployment
> 	scenario because it doesn't fit nicely within IETF 
> 	borders. We should say this is how to "best" solve
> 	our problem space and then decide what to do next.

Paul, I agree 100% with you!

The direction the IPFIX WG is currently force to take is:
- congestion aware.
  What for? for the 5 % customers which are going to export over the internet
- reliable transport because we have almost no other option
  What for? reliability is mainly needed for billing purposes but not for traffic profiling, traffic engineering, ddos detection, user profiling, capacicity planning, etc..
  so again for a subset of the applications using the IPFIX data

So just to be "honorable", we're fixing the problem of a probe (no memory problem/not exporting a lot of data) exporting billing information to a remote end of the internet. And we're forcing everybody else to apply the same protocol or ... follow the do-what-you-want-in-your-intranet procedure.

I'm just a little dissapointed... I thought that the IETF was based on consensus and running code.

Regards, benoit.

> 
> 	It would certainly be good if we can come up with
> 	something that works in both settings. I don't
> 	see why we can't accomplish that.
> 
> > 
> > -----------------------------------------------------------------------
> >    Nevil Brownlee                   Director, Technology Development
> >    Phone: +64 9 373 7599 x8941      ITSS, The University of Auckland
> >    FAX: +64 9 373 7021      Private Bag 92019, Auckland, New Zealand
> > 
> > --
> > Help        mailto:[email-masked] and say "help" in message body
> > Unsubscribe mailto:[email-masked] and say
> > "unsubscribe ipfix" in message body
> > Archive     http://ipfix.doit.wisc.edu/archive/
> 
> --
> Help        mailto:[email-masked] and say "help" in message body
> Unsubscribe mailto:[email-masked] and say
> "unsubscribe ipfix" in message body
> Archive     http://ipfix.doit.wisc.edu/archive/
> 

--
Help        mailto:[email-masked] and say "help" in message body
Unsubscribe mailto:[email-masked] and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/

---

From: "Kevin Zhang" <[email-masked]>
Date: Tue, 12 Mar 2002 12:37:10 -0500

> -----Original Message-----
> From: majordomo listserver [mailto:[email-masked]]On Behalf
> Of Benoit Claise
> Sent: Tuesday, March 12, 2002 9:16 AM
> To: [email-masked]; [email-masked]
> Cc: [email-masked]
> Subject: Re: [ipfix] Mailing list digest, Friday 8 March
> 
> 
> > > 
>   What for? reliability is mainly needed for billing purposes but 
> not for traffic profiling, traffic engineering, ddos detection, 
> user profiling, capacicity planning, etc..
> So just to be "honorable", we're fixing the problem of a probe 
> (no memory problem/not exporting a lot of data) exporting billing 
> information to a remote end of the internet. And we're forcing 
> everybody else to apply the same protocol or ... follow the 
> do-what-you-want-in-your-intranet procedure.
> 

I believe the core of the problem is whether a router is able and willing to buffer one rto+delta worth of flow records in order to allow for any reliability, regardless the transport is UDP or TCP. The design or limitation of a router should not dominate the direction of IPFIX.  

In terms of current deployment, from our experience, reliability is the most important requirement from carriers.  The reason why exporting over WAN is relatively few is exactly due to the fact that there's no standard approach, which the IPFIX WG is addressing.

The choices in front of us is whether to address real customer needs that makes IPFIX highly relevant and benefit the industry as a whole, or standardizing some vendor's existing solution that has a very limited audience.  

In terms of consensus, my observation is that there is at least as much support for a reliable transport as TCP. Currently, CRANE and LFAF are running over TCP (running code).

Thanks,

Kevin 

> ÿñÞ–™šŠ[hþf£¢·hšçzßÝ¢+ÿÂ+ýçnjwlk/ázZÿŠyž²Æ yºÉIì¹»®&Þ™¨¥¶æj:+v‰¨þwýÚ"·ü"±ÏÞvæ§vÆ²þéì¹»®&ÞŠ—âÇø§™ë,j›¡Ü€Èb½èm¶Ÿÿþ*_‹Ý¢+ÿÂ+ýçnýªÜ†+Þ

---

From: Michael MacFaden <[email-masked]>
Date: Tue, 12 Mar 2002 10:57:47 -0800

On Tue, Mar 12, 2002 at 03:15:46PM +0100, Benoit Claise wrote:
>  What for? reliability is mainly needed for billing purposes but not for 
> traffic profiling, traffic engineering, ddos detection, user profiling, capacicity planning, etc..

When I last spoke with Daniel McRobb who wrote 'cflowd' when he was a CAIDA, he wanted 
NetFlow over TCP not UDP.....And he certainly wasn't interested in doing 'billing apps' at the time.

So I think this issue can't be divided this neatly. 

Mike MacFaden

---

From: "Peter Phaal" <[email-masked]>
Date: Tue, 12 Mar 2002 11:53:34 -0800

> From: majordomo listserver
> [mailto:[email-masked]]On Behalf
> Of Michael MacFaden
> Sent: Tuesday, March 12, 2002 10:58 AM
> To: [email-masked]
> > traffic profiling, traffic engineering, ddos detection,
> NetFlow over TCP not UDP.....And he certainly wasn't
> interested in doing 'billing apps' at the time.
>

Another way to look at the need for reliable transport is to consider how
heavily the data is aggregated. If you have fine grain flow data (e.g.
NetFlow 5) then the loss of a record isn't as serious and can more easily be
compensated for than the loss of a highly aggregated flow record (e.g.
NetFlow 8).

A reliable transport is desireable for highly aggregated flow data. The cost
of the extra protocol overhead and buffering in this case is minimal since
there isn't very much data being transported.

In the case of fine grain flows the data rates can be very high and the
overhead of using a reliable transport is not as easy to justify, especially
since the loss of some data can be relatively easily compensated for.

There seems to be a loose correspondance between highly aggregated data and
billing and fine grain data and traffic engineering, DoS detection, etc. It
seems that there are clear enough differences that one should be able to
select a transport that suits the aggregation scheme and application, rather
than picking a single transport and limiting IPFIX to one or other
application area.

Why not support both TCP and UDP and include a requirement that if IPFIX is
being used across the Internet backbone then TCP MUST/SHOULD be used?

Peter
----------------------
Peter Phaal
InMon Corp.

[email-masked]

---

From: Peram Marimuthu <[email-masked]>
Date: Tue, 12 Mar 2002 12:56:02 -0800

> being used across the Internet backbone then TCP MUST/SHOULD be used?

Well said. However some people are out of touch with reality and are mandating things
that will never get implemented. There will be so many non-compliant implementations, but
who cares

Peram

---

From: [email-masked]
Date: Tue, 12 Mar 2002 14:25:51 -0800 (PST)

Hello all:

Bickering about UDP is clearly a rathole, lets stop spending list
time on it.

IPFIX is concerned about producing a clear, understandable and easily
implementable standard for exporting flow information.  The data model,
and how that data is mapped into whatever application protocol we select
is what we should be focussing our design efforts on.

More to the point, there's clearly strong support for reliable
transport, and the ability to recover from collector or exporter
failures.  Seems to me that SCTP could provide both of these features,
wheras if we use TCP we have to habdle failover within the IPFIX
application protocol.  Anyone got opinions on this notion?

Nevil

---

From: "Peter Ludemann" <[email-masked]>
Date: Tue, 12 Mar 2002 14:54:46 -0800

I agree that SCTP is the preferred transport; unfortunately it isn't as
universally available.

So, I would suggest a protocol that could use *either* TCP or SCTP as a
transport. Interestingly enough, the protocol doesn't need to specify
windows, time-outs,  fail-overs, multi-homing, etc.; that can be left to the
implementation. The only thing [I think] that the protocol needs for TCP and
wouldn't need for SCTP is record-level acknowledgment. For this, the
protocol specification merely needs to state that if SCTP (or similar)
transport is used, the receiver does not need to send acknowledgments. (The
protocol might also allow messages such as "heartbeat".)

We could also state that TCP is an interim transport solution and is
intended to eventually be replaced by SCTP everywhere.

BTW, for those who really want non-reliable transfer, see
http://www.ietf.org/internet-drafts/draft-ietf-tsvwg-usctp-00.txt, which
extends SCTP to allow unreliable delivery.

- peter

> Of [email-masked]
> Sent: Tuesday, March 12, 2002 2:26 PM
> Cc: Peram Marimuthu
> Subject: [ipfix] Re: congestion awareness, reliability
>
>
>
> Hello all:
>

---

From: [email-masked]
Date: Wed, 13 Mar 2002 09:56:11 -0500

> Bickering about UDP is clearly a rathole, lets stop spending list

	I don't see it as bickering about UDP, I see it as
	trying to get to an understanding of what we mean
	by reliability, when do we need it and when don't
	we need it. Peter Phaal made some excellent points
	on this in his mail

		http://ipfx.doit.wisc.edu/list/ipfx/archive/0831.html

	We are also getting to an understanding of when and
	where we need congestion control. Seems like healthy
	discussion to me.

> transport, 

	I have not seen that on this list. I have seen reliability 
	asked for but the defintion of that is not clear to me. It 
	could mean a reliable transport or it could mean no loss of 
	data transported, they are not the same.

	I have also seen support for congestion aware transport 
	in some situations. 

> and the ability to recover from collector or exporter failures.  
> Seems to me that SCTP could provide both of these features,

---

From: Peram Marimuthu <[email-masked]>
Date: Wed, 13 Mar 2002 08:27:38 -0800

I agree with Paul. Ganesh presented some arguments at the last IETF on
why TCP will be troublesome for high-end platforms. I have seen *no* arguments
or analysis to say why this is not a problem. All I hear is "TCP or close your
working group".

TCP or SCTP will work for some scenarios and is required for some problems.
But to mandate it for all scenarios and all problems just shows that people with
power are not willing to listen.

Peram

---

From: [email-masked]
Date: Wed, 13 Mar 2002 11:45:37 -0500

Peram Marimuthu wrote:
> why TCP will be troublesome for high-end platforms. 

	I have to disagree with that statement. Ganesh presented
	some calculations. From what I recall a lot of the
	assumptions that went into those calculations were
	called into question.

> I have seen *no* arguments

---

From: Peram Marimuthu <[email-masked]>
Date: Wed, 13 Mar 2002 09:06:57 -0800

[email-masked] wrote:

> Peram Marimuthu wrote:
> >
> > Ganesh presented some arguments at the last IETF on

Ok. If I am right, Ganesh's presentation said that we need 6.8Gbytes
of memory and 30% CPU utilization to support Netflow (with TCP) on a 40G router.
Let us assume that he is completely wrong and the requirement is just 1/5th.
Even then we need a buffer of about 1.4Gbytes.

Now, what I am looking for is a presentation with the right calculations
and acceptable assumptions that shows buffer requirements for a 40G box.
Assume 100 ms RTT, 50Kbps as bandwidth per flow, 40 bytes as flow record
size,  path mtu is 1500 bytes and you will have to support 1 retransmission.
Convince that buffering requirements are not very high and TCP will work here.

Thanks
Peram

---

From: Michael MacFaden <[email-masked]>
Date: Wed, 13 Mar 2002 12:09:51 -0800

>application protocol.  Anyone got opinions on this notion?

If we want to be able to use any arbitrary transport, then
detection of failed connections or applications must be done in
the protocol. I remember some bizarre behaviors in 
Microsoft platforms where the tcp session was still up but the 
application had long since died...

I suggest the time to detect failure must be deterministic so a complaint
implementation can be measured. 

Finally, determining what to do on failure could take some time to
develop but is well worth it. I've seen the following in
existing protocols:

1) Just detect you've lost data, if running unicast, you are SOL. if
   running multicast, then make collectors figure out who has the 
   most complete set of data. 

2) Detect you've lost data and then keep simple aggregate byte/pdu
   counters available via MIB or CLI to track how much has been lost.

3) Keep a queue of records (telco's demand such data spooled to disk for later file xfer) 
   and wait till connection is established,
   and perform tail drops when queue is full. On reconnection, schedule xfer
   of the existing queued elements and any new data. Very nice to have
   congestion aware transport here when you have two streams going..

4) Attempt to switch to an aggregate method to keep tx queue
   from filling up too fast hoping another connection will
   be available.

Mike MacFaden

---

From: Ganesh Sadasivan <[email-masked]>
Date: Wed, 13 Mar 2002 13:39:30 -0800

>         assumptions that went into those calculations were

The main question I heard was the # of memory copies and I guess the
numbers would be similar for UDP. But in  terms of protocol processing
TCP is much more intensive when compared to UDP (checksum calculation,
ack processing, RTT adjustment).  Agreed that there
are many schemes to make TCP efficient - zero-copy, checksum
offloading, interrupt coalesing etc. many of which require special h/w  if the
CPU utilization needs to remain low. But this cannot be assumed if IPFIX
were to go into existing deployments.