Subject: [ipfix-eval] Gopal's evaluation draft contribution

From: <[email-masked]>
Date: Wed, 16 Oct 2002 18:15:12 -0400

Greetings,

Please find the individual evaluation of IPFIX protocol candidates. I used the templates prepared 
by J. Quittek and thank for his work it saves lot of time.

Feel free to comment on this material. 

Thanks and Regards
Ram Gopal. L

---

From: "Peter Ludemann" <[email-masked]>
Date: Sun, 20 Oct 2002 23:09:38 -0700

[email-masked] wrote Wednesday, October 16, 2002 3:15 PM in an
attachment:

>   [3.1.5. Time Synchronization (5.5)]
>
>   It my impression that by correlating server boot
>   time and client boot time each flow record may be
>   analyzed. To be clarified with the protocol authors?

The "boot time" in CRANE is intended mainly to allow deduplication when the
exporter restarts. There is no connection between any of the client or
server boot times. The exported data can, of course, contain time stamps -
how these relate to the boot time is up to the data model.

The initial connection to a data exporter has a server (collector) boot
time, but it's not clear to me how the exporter could use that to set its
own clock. Any clock information sent from the collector would either be a
rather crude mechanism or would require re-inventing NTP. [There is also the
separate issue of synchronization amongst the servers (collectors) -- again,
NTP is our solution.]

We have used standard NTP for synchronizing redundant probes with sufficient
precision to allow de-duplication (for billing purposes, the customer wanted
redundant probes). So, from our experience, synchronization is outside the
flow export protocol's scope and is best done with NTP (for the really
precision-minded, there are NTP servers that use GPS time signals; but we
haven't needed to use these).

(You'll notice a theme here: we use TCP or SCTP for transport-level
reliability, IPsec for security, NTP for time synchronization, decouple the
data model from the protocol, etc. Why re-invent wheels or have unnecessary
dependencies?)

- peter

--
Help        mailto:[email-masked] and say "help" in message body
Unsubscribe mailto:[email-masked] and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/

---

From: [email-masked]
Date: Mon, 21 Oct 2002 09:51:35 -0400

I expected to see some conclusions.

Paul

[email-masked] wrote:
>   ------------------------------------------------------------------------
>                                        Name: draft-gopal-ipfix-eval-00.txt
>    draft-gopal-ipfix-eval-00.txt       Type: Plain Text (text/plain)
>                                    Encoding: base64
>                                 Description: draft-gopal-ipfix-eval-00.txt

---

From: "Tal Givoly" <[email-masked]>
Date: Wed, 23 Oct 2002 00:22:51 -0700

Ram,

As I really liked the fact that Juergen layed out his evaluation
in a matrix. Therefore, for the group's convenience, I took the
liberty and tabularized your evaluation according to the same type
of matrix (with the content you provided, however excluding the
comments).

If you feel this is useful to maintain, I encourage you to
double-check, in case I copied something incorrectly. I had
to make some interpretation of items that had numbering mismatch
with labels - perhaps you would like to clarify them as well.

Ram's evaluation scoresheet:

   LFAP   CRANE  IDPR   Net-   Dia-
                        Flow   meter
    P      T      E      P      E     Reliability (5.1)
    F      F      F      ?      E     Sampling (5.2)
    T      F      E      T      F     Overload Behavior (5.3)
    T      T      T      T      T     Timestamps (5.4)
    T      T      E      T      T     Time Synchronization (5.5)
    T      T      E      T      T     Flow Expiration (5.6)
    E      ?      ?      ?      ?     Multicast Flows (5.7)
    E      E      F      E      F     Ignore Port Copy (5.8)
    T      T      T      T      T     Information Model (6.1)
    T      T      T      T      T     Data Model (6.2)
    T      T      T      E      T     Congestion Awareness (6.3.1)
    T      T      T      T      T     Reliability (6.3.2)
    T      T      T      E      T     Security (6.3.3)
    E      E      E      E      T     Push/Pull/Regular Reporting (6.4/5)
    T      T      T      T      T     Notif. on Specif. Events (6.6)
    E      E      E      E      E     Anonymization (6.7)
    T      T      E      T      T     Configuration of Metering (7.1)
    E      T      E      T      T     Configuration of Exporting (7.2)
    E      ?      ?      ?      T     Openness (8.1)
    T      T      E      T      T     Several Collecting Proc (8.3)
    T      T      F      ?      T     Special Device Consideration (9)
    T      T      T      E      T     Security Considerations (10)
-------------------------------------------------------------------
   14     15      8     11     16     number of Ts
    6      3      9      6      3     number of Es
    1      0      0      1      0     number of Ps
    1      2      3      0      2     number of Fs
    0      2      2      4      1     number of ?s

I believe it is also very important to note whether the requirement
stipulated MAY/SHOULD/MUST, etc. By numerically comparing these
protocols against the overall list of requirements as equals, it is
difficult to capture the DEGREE of compliance with the requirements.

Perhaps a column should be added to highlight this?

From: majordomo listserver [mailto:[email-masked]]On Behalf
Of [email-masked]
Sent: Wednesday, October 16, 2002 3:15 PM
To: [email-masked]
Subject: [ipfix-eval] Gopal's evaluation draft contribution

Greetings,

Please find the individual evaluation of IPFIX protocol candidates. I used
the templates prepared
by J. Quittek and thank for his work it saves lot of time.

Thanks and Regards
Ram Gopal. L

---

From: "Tal Givoly" <[email-masked]>
Date: Thu, 24 Oct 2002 19:22:13 -0700

Ram,

Below are my comments to the evaluation you provided
(http://ipfix.doit.wisc.edu/archive/att-1277/01-draft-gopal-ipfix-eval-00.tx
t). Rather than quote entire sections or portions of them, I merely refer to
section number:

2.4. NetFlow

NetFlow, as proposed, is UDP. Merely extending it to use any other protocol
(e.g. TCP) doesn't add any reliability to it. Making it reliable would
require making it bi-directional, thus adding the added features and some of
the complexity available in CRANE/Diameter/LFAP. Application-level
acknowledgement is minimally required to indicate the collection process has
actually received what has been sent reliably.

One could argue regarding what the word "reliably" means above. This could
be interpreted by some as "best effort" (send ack once data is received
in-memory), but based on the requirement draft, I believe "reliably" implies
that it must be possible to ensure the level of reliability described  here:
persisted or voted with redundant collecting processes - ensuring durability
in face of up to N failures, where N is defined by operator, usually N=1.

Making it work over TCP would, at least, ensure in-order arrival, and the
fact that template defintions arrive prior to the flow records themselves,
however this is not assured without this extension.

The dynamic nature of the templates seems to actually make them
indistinguishable from the collector's perspective (all the template defines
is what are the fields within - the identifier itself is totally dynamic and
non-negotiable). I've asked the NetFlow protocol advocate, in a separate
submission, to clarify how multiple templates are maintained concurrently -
i.e. how does the collector distinguish between them to identify their
semantics (not only record content) as they are temporary in nature. In the
response, it becomes clear that identification of different templates is
totally implicit by their attribute contents. Although this does allow for
distinction among some templates, it is an expensive way to create such a
distinction.

CRANE also allows for modification of Templates. It is, in fact, designed
for continuous operation under changed templates (see Simon's evaluation
that determines this as well).

3.1.2. Sampling (5.2)

As I explained earlier in my response to both Simon's and Juergen's
evaluation (in http://ipfix.doit.wisc.edu/archive/1316.html):

CRANE clearly allows distinguishing between sampled data and non-sampled
data. As CRANE, similarly to IPDR and Diameter, doesn't impose any
restrictions on the Metering Process, the latter can easily submit sampled
data in a dedicated Template. Even if you consider that there is no
intrinsic support specifically for Sampling Behavior within the protocol
itself, it is
definitely Extendable to that degree, would you not agree?

I fully agree with Jeff's following statement :

<snip (from http://ipfix.doit.wisc.edu/archive/1330.html) >
    CRANE, Diameter and IPDR are all examples of protocols whose focus is
solely on the exporting process, putting no restrictions on the behavior of
the "Metering Process".
<snip>

As explained by Cisco recently, NetFlow distinguishes between Sampled data
and unsampled data by the existence of attributes within flow record itself.
It too doesn't define exactly how sampling is described besides that implied
distinction.

3.1.3. Overload Behavior (5.3)

CRANE defines the interaction between the exporting process (E) and the
collecting process (C). This interaction provides specific flow control and
interfaces that allow the metering process (M) to begin performing sampling,
for instance to react to overload. The exporting process is made aware of
overload in the collecting process and redirects flows to alternative
collecting processes. The metering process, once faced with overload
conditions, can easily use templates dedicated to reporting at times of
overload, to export flow information.

Should the aforementioned behavior be reconsidered in your evaluation
concluding that CRANE fails the overload behavior?

I further believe that NetFlow v9, in fact, cannot be considered fully
compliant with overload behavior - as it doesn't react to flow control, it
cannot be aware of overload in the collecting process that may require
introducing of overload behavior. Let me expand on this: Since NetFlow v9 is
not bi-directional one cannot tell from (E) if data is lost. One would have
to check with both (C) and (E) to figure this out. This is very painful
resulting in excessive deployment costs.

It is fully implementation dependent how memory footprint is handled.
Current CRANE implementations and deployments allow the host metering
process full control of the footprint of the application completely. In
fact, the implementation allows the network element to replace all resource
allocation functions to fit the environment. Therefore, priority of the
process, its resource utilization, etc. are all subject to implementation.

3.1.7.Multicast Flows  (5.7)

Shouldn't this requirement be considered a requirement on the metering
process rather than the exporter process? As there is no restrictions
imposed by CRANE on the data export by the metering process (by virtue of
support for multiple concurrent templates), CRANE is clearly extendable to
support this requirement.

3.1.14.Reliability  (6.3.2)

As far as I understand, NetFlow doesn't indicate lack of reliability in the
protocol itself (for all cases where reliability is incomplete). It leaves
this primarily to the collecting process to resolve. Existence of sequence
numbers within the protocol implies the collector must compare these upon
receipt. When multiple collecting processes are involved (and each may see a
different subset of the UDP packets), it is a very complex function of
multiple receivers to figure out the lack of reliability.

Should that be considered totally compliant?

While it has been mentioned by several evaluators that adding reliability to
NetFlow is an "easy" thing, I believe this has yet to be satisfactorily
demonstrated do the degree that the final proposal properties (including
reliability and the complexity it may introduce), be evaluated.

3.1.23.Openness (8.1)

I agree with other evaluations that did find CRANE totally compliant with
this requirement. Furthermore, it provides full version and capability
negitiation to allow for backward and forward compatibility (a la Fax/Modem
negotiations on capabilities). I believe this is a very-nice-to-have
attribute in any protocol, especially one implemented in context of flow
information export that may have many different devices to interface with
(each of potentially different versions). Although this has not been raised
as an explicit requirement, it may be interpreted to be part of the
extensibility requirement.

Can you help me understand the question you have regarding CRANE's
compliance so that we can eliminate any doubt regarding this aspect?

Tal

-----Original Message-----

Greetings,

Thanks and Regards
Ram Gopal. L

---

From: Benoit Claise <[email-masked]>
Date: Mon, 28 Oct 2002 16:38:16 +0100

Tal and all,

>however this is not assured without this extension.
I've been looking for the "reliability" keywords in the requirement 
draft. 2 sections appear:

5.1.  Reliability

   The metering process MUST either be reliable or missing reliability
   MUST be known and indicated. The metering process is reliable if each
   packet passing the observation point is metered according to the
   configuration of the metering process. If, e.g. due to some overload,
   not all passing packets can be included into the metering process,
   then the metering process MUST be able to detect this failure and to
   report about it.

6.3.2.  Reliability

   Absence of reliability of the data transfer, for example caused by
   loss or reordering of packets containing flow information, MUST be
   indicated.

   Please note that if an unreliable transport protocol is used,
   reliability can be provided by higher layers. In such a case only
   lack of overall reliability MUST be indicated. For example reordering
   could be dealt with by adding a sequence number to each packet.

It seems that in the recent days discussions, we are discussing of 
reliable transport (that I just took this as an example above), 
bidirectional communications, application level ACK, load balancing, 
failover, etc... And some of them are even becoming MUST in some people 
emails.
I would simply like to stress to look back at the requirement draft, 
which is derived from the WG charter. And to use this as a basis for 
discussion.
And if there is something wrong with the requirement draft (but I think 
it's getting pretty complete after 6 iterations), let's change first the 
requirement draft.

Regards, Benoit.