Subject: RE: [ipfix-arch] Clarification on Arch document, section 7.4

From: "Reinaldo Penno"<[email-masked]>
Date: Thu, 21 Feb 2002 12:59:34 -0800

-----Original Message-----
From: Norseth, KC [mailto:[email-masked]]
Sent: Thursday, February 21, 2002 6:41 AM
To: Penno, Reinaldo [SC9:T327:EXCH]; [email-masked]
Subject: RE: [ipfix-arch] Clarification on Arch document, section 7.4

-----Original Message-----
From: Reinaldo Penno [mailto:[email-masked]] 
Sent: Tuesday, February 19, 2002 12:10 PM
To: [email-masked]
Subject: [ipfix-arch] Clarification on Arch document, section 7.4

a) "3. For long aging flows, the exporter SHOULD export the flow records on
regular basis, in order to:" 

I think it should say "...the exporter SHOULD be able to export the flow..."

b) then.. 

   "a.  report the flow records periodic accounting information to the
collector 
    b.  avoid counter wrapping This activity timeout SHOULD be configurable"

It should be 

"a. provide periodic accounting information to the collector 
b. avoid counter wrapping." 

This activity SHOULD be configurable. Btw, can someone elaborate on the
counter wrapping stuff?  

It is very simple to send so much data though the line on a flow that the
byte counter rolls over.  This is very typical in 1 gig and 10g  ports.
counter 32 is not large enough field size for this. 

yes, that's what I tought. So the solution is just to augment th efield to a
64 counter. It seems that the peirodic reporting is completely orthogonal to
this specific issue.

c) then.. 

"4. If the exporter experiences resources constraints, a flow MAY be
prematurely expired (example: memory)" 

Is there any other choice? This seems too implementation specific. 

regards, 

Reinaldo

---

From: [email-masked]
Date: Thu, 21 Feb 2002 13:44:02 -0800 (PST)

> 64 counter. It seems that the peirodic reporting is completely orthogonal to

You've covered the 'counter wrapping' problem - a flow information meter
(whatever that turns out to be) needs counters wide enough not to wrap
too often, and that becomes more of a problem at high link speeds.
In RTFM, all counts are kept in 64-bit unsigned counters so that they
don't wrap too often.  

Of more interest, I don't remember seeing any IPFIX discussion
concerning just how information about long-running flows is to be
exported.  In particular, I believe that counters should never be reset.
That would mean that successive exports of the same flow would have
increasing counter values, a collector would need to remember the
previous value and subtract it.  Furthermore, provided unsigned
arithmetic is used, the difference is correct even if the counter has
wrapped.  Provided, of course that the counter has only wrapped once!

Cheers, Nevil

-----------------------------------------------------------------------
   Nevil Brownlee                   Director, Technology Development
   Phone: +64 9 373 7599 x8941      ITSS, The University of Auckland
   FAX: +64 9 373 7021      Private Bag 92019, Auckland, New Zealand

--
Help        mailto:[email-masked] and say "help" in message body
Unsubscribe mailto:[email-masked] and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/

---

From: [email-masked]
Date: Thu, 21 Feb 2002 16:59:30 -0500

> > 64 counter. It seems that the peirodic reporting is completely orthogonal to

	We need to be careful here. Some of us need to get counters
	directly from the hardware so just changing the size is
	not an option. Doing it in software can be an issue too
	because that means extra memory is needed for each flow.

> too often, and that becomes more of a problem at high link speeds.

	Not only the link speed but the aggregation level too.
	The higher level aggregations tend to get more bytes/packets
	per second. 

> exported.  In particular, I believe that counters should never be reset.

	One more thing to keep in mind here is that most hardware
	does not provide a way to do an atomic get and set operation.
	So if you do reset it, you'll likely loose some information.

> increasing counter values, 

	The other way is the box keeps track of the last reported
	in counter in software and still provide deltas. The problem with
	cumulative counts is on failover the collector will not
	have the last count. I'd like to keep collector to collector
	requirements to a minimum or none if possible.

> a collector would need to remember the

---

From: "Peter Ludemann" <[email-masked]>
Date: Thu, 21 Feb 2002 14:17:35 -0800

[email-masked] wrote Thursday, February 21, 2002 1:44 PM:
To: [email-masked]

> wrapped.  Provided, of course that the counter has only wrapped once!

In our experience, it's better to output deltas. Absolute values are needed
only if you have an unreliable data exporter. (In fact, we have some modules
which turn absolute values (e.g., from SNMP) into deltas [taking into
account wrapped counters of course].)

The reason for delta values is that they make a much cleaner data model
[each delta value has start/end timestamps, of course]. If I want to produce
reports that show values with various intervals (5-minutes, hourly, daily,
weekly), it's easy to write queries that sum things up this way. With
absolute values, things are somewhat uglier. (SQL doesn't have unsigned
arithmetic)

But, as I said, deltas only work if you have a reliable flow exporter.
Otherwise, you have no choice but to use absolute values and deal with the
overflow issues, which can happen even with 64-bit counters.

- peter

----
Peter Ludemann           [email-masked]
Chief Architect          +1.408.330.5732
XACCT Technologies, Inc.
2900 Lakeside Drive      Santa Clara, CA 95054 USA

---

From: Ganesh Sadasivan <[email-masked]>
Date: Thu, 21 Feb 2002 18:40:33 -0800

As a policy:
Export flow records when counters go past a threshold.

Keep IEs for
* Absolute counts
* Delta counts
whichever  would suit the appropriate implementation.

my $.02
Ganesh

---

From: Benoit Claise <[email-masked]>
Date: Fri, 22 Feb 2002 12:26:53 +0100

Nevil,

> exported.  In particular, I believe that counters should never be reset.

Why?
For long run flows, you can just export the flow records with the acconting
information seen during that interval.
If you must export the flow records because:
 - you faced the long aging time timeout (for example: 30 minutes)
 - the counters is about to wrap.
then you consider the flow to be ended (expired) and export the accounting data
for that period.
Now a new flow with the same characterics will be directly created. Not a problem.
And you start the accounting from 0 for that "new" flow.
The collector won't be confused at all and don't need to remember any values.

Regards, Benoit

---

From: [email-masked]
Date: Fri, 22 Feb 2002 15:18:57 -0500

> And you start the accounting from 0 for that "new" flow.

	Most hardware can't do a get and set in one
	operation. So in between reading the count
	and setting it to zero you may loose some
	of the count.

---

From: Benoit Claise <[email-masked]>
Date: Sat, 23 Feb 2002 00:23:40 +0100

Paul,

>         and setting it to zero you may loose some

I'm not too sure about the details of most hardware, but this is the way it works with
netflow.

Regards, Benoit

---

From: [email-masked]
Date: Mon, 25 Feb 2002 10:07:27 -0500

Thought I would reply direct.

I would check on that. Since the hardware
is the one that increments the counter as
packets are going through, the only way to 
read the count and then set it to zero as
one operation would be to stop the hardware
from updating the counter while the read-set-to-zero
operation is happening. 

So either the netflow hardware really supports that
operation or a tradeoff was made to leave the issue
as is.

It would be good to know as it affects how to best
accomplish intermediate updates should we go that way.

Paul

---

From: Benoit Claise <[email-masked]>
Date: Wed, 27 Feb 2002 11:32:05 +0100

Paul,

> operation or a tradeoff was made to leave the issue as is.

I called our specialist. Without going into the implementation details, his answer is:
Our h/w does not loose any data when it happens. I'd think it's a bug if some h/w does it.

Here is my view on the absolute/delta counters, as I wrote in a previous email. But I would
like to expand on absolute/delta counters.

If you think of keeping an absolute counter value, this is a counter you must maintain for
all active flows. What should be the size? You have no idea initially ... a 32 or 64 bit
counter? So you must have a 64 bit counter for every single flow. This implies a lot of
memory (if not done in hardware) and it implies an extra 32 bits sent to the collector for
every flow records!

If you think of sending delta values, it's delta compared to what? To the flow aging time,
right?
Side question: Does it mean that you must have an absolute counter as well?

So why don't we forget about absolute or delta counters but speak about the counters of the
flow  live time.

For long run flows, you can just export the flow records with the accounting information
seen during the flow live time. Knowing a long run flow is actually a serie of shorter flows
with the same characteristics.

You must export the flow records because:
 - you faced the long aging time timeout (for example: 30 minutes), then the live time of
the flow is 30 minutes. We export the counters for that specific period of 30 minutes.
 - the 32 bits counter is about to wrap. Then you consider the flow to be ended (expired)
sooner and export the accounting data for that period, whatever the period is.

Now, because packets with the same characterics continue to arrive, a new flow with the same
characterics will be directly created. Obviously with a counter starting at 0.
That way, the collector won't be confused at all and you have an easier implementation.

The explanation above was this idea when Ganesh and I wrote:
http://www.ietf.org/internet-drafts/draft-ietf-ipfix-data-00.txt

5.4. Flow Expiration

      ...

       3. For long aging flows, the exporter SHOULD export the flow
         records on regular basis, in order to:

         a.  report the flow records periodic accounting information
             to the collector
         b.  avoid counter wrapping

          THERE IS A SMALL FORMATTING PROBLEM. THERE SHOULD BE A NEW LINE
          This activity timeout SHOULD be configurable

      4. If the exporter experiences resources constraints, a flow
         MAY be prematurely expired (example: memory)

Regards, Benoit

---

From: [email-masked]
Date: Wed, 27 Feb 2002 15:46:35 -0500

> So why don't we forget about absolute or delta counters but speak about the counters of the

	I think we need to offer both in order to support
	existing devices. Some hardware only have cumulative
	counters with no atomic get/set operation. 

> with the same characteristics.

	This depends on how flow updates get reported. Since 
	most of the flow information is static attributes then
	they can be reported once with updates only reporting
	counts and a "flowID" (aka LFAP style). This can
	save a lot of bits on the wire. But there are drawbacks
	to this approach as well. I assume we'll get into those
	nasty details at some point.