Subject: [ipfix-req] Terminology again

From: Juergen Quittek <[email-masked]>
Date: Wed, 13 Feb 2002 20:48:34 +0100

Hi all,

We had some terminology discussion and it has not
converged yet. Therfore I like to raise one issue again:

For the terinology section of the requirements draft
(I belive now it will be a subset of the architecture's
terminology) I would like to have two identifiers for
the following:

 1. the function block containing all metering functions
    This block gets as input observed (and potentially
    sampled) packets from the observation point. It classifies
    packets maps them to flows and creates/updates the
    according flow record.

 2. the function block sending flow records to the collector

I am fine with splitting blocks more or modifying them
slightly, but I prefer to have this separation, because
also the requirements are split in a very similar way.

My initial naive suggestion was calling block 1. "meter"
and calling block 2. "exporter". However this might (no it
did already) cause confusion. Therefore, I am looking for
better terms. Any suggestions?

    Juergen
-- 
Juergen Quittek     [email-masked]     Tel: +49 6221 90511-15
NEC Europe Ltd.,    Network Laboratories     Fax: +49 6221 90511-55
Adenauerplatz 6, 69115 Heidelberg, Germany   http://www.ccrle.nec.de

--
Help        mailto:[email-masked] and say "help" in message body
Unsubscribe mailto:[email-masked] and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/

---

From: Sebastian Zander <[email-masked]>
Date: Thu, 14 Feb 2002 11:07:28 +0100

Hi Juergen,

> also the requirements are split in a very similar way.

I also would like to split those for at least 2 reasons.
First we create a more general model because on a router linecard
both are be combined but on a dedicated hardware/software based
meter this will probably not the case. I don't think we should limit
ipfix to routers only. Second the separation is nice to clarify
the scope. The scope of the ipfix wg is to standardize the
exporter (at least the interface to the outside). It's out
of scope to standardize the meter (although we put some req.
on it).

I am not sure whether sampling is a function of the observation
point or the meter.
 
> better terms. Any suggestions?

I have no better terms. I think what's maybe confusing is that we don't
have a term for the overall process (meter+exporter+observation).
What could be appropriate? flow monitor? 

Another possibility: Name the overall process metering and rename
the inner function block 1 which does classification etc.

Sebastian

-- 
Sebastian Zander                         E-mail: [email-masked]
FhI FOKUS / Global Networking (GloNe)    Tel: +49-30-3463-7287
Kaiserin-Augusta-Allee 31                Fax: +49-30-3463-8287 
D-10589 Berlin, Germany                  www.fokus.fhg.de/usr/sebastian.zander

---

From: Juergen Quittek <[email-masked]>
Date: Thu, 14 Feb 2002 18:09:56 +0100

Hi Sabastian,

> I am not sure whether sampling is a function of the observation

Neither am I, but we should agree on one or the other soon.
What about the following (short version of more detailed
suggestions already discussed for the architecture)?

   +-------+   +-----------+   +-------+   +--------+   +--------+
   | obs.  |   | header    |   |       |   | flow   |   | ex-    |
   | point |-->| capturing |-->| meter |-->| record |-->| porter |
   +-------+   +-----------+   +-------+   +--------+   +--------+

   |                                                             |
    \____________________________  _____________________________/
                                 \/
                  IPFIX traffic flow measurement module

If can tell me a good reason I am fine with merging header capturing
and meter, but in general I see them as different function blocks.

    Juergen

---

From: Ganesh Sadasivan <[email-masked]>
Date: Thu, 14 Feb 2002 11:07:19 -0800

Hi Jurgene,
   I am of the opinion that "header capturing" is a metering
   function.I see restrictions with your model. Escpecially
   if some kind of metering is done on collected flows within
   the router. It was with this in mind that I send out in one of
   my previous mails the high level picture. Did not get any
   response. So I am resending with some more modification. I am
   slightly inclined to think that the metering process within
   the observation domain could be hierarchical but have not put
   it here.
   What are your inputs?

+---------------------------------------------------------------+
|                   Observation Domain                          |
|                                                               |
|          +-------+                      +-------+ Packet level|
|          |Meter1 |         ....         |MeterM | ({Fi}, {Si})|
|          |Process|                      |Process|             |
|          +-------+                      +-------+             |
|              ^                              ^                 |
|              |                              |                 |
|      +-------+-------+              +-------+--------+        |
|      |               |              |                |        |
|      v               v              v                v        |
|+------------+  +------------+  +------------+   +------------+|
||Obsv Point11|..|Obsv Point1k|  |Obsv PointM1|.. |Obsv PointMn||
|+------------+  +------------+  +------------+   +------------+|
+---------------------------------------------------------------+
              ^                     ^
              |                     |
              v                     v
         +----------+          +----------+
         |Meter (O1)|   ....   |Meter (OP)| Flow Level
         +----------+          +----------+ ({Fi}, {Si})
                 |                |
                 +-------+--------+
                         |
                         v (uniqued ID for an observation domain)
                 +---------------+
                 |  exporter     |
                 +---------------+

Thanks
Ganesh

---

From: Juergen Quittek <[email-masked]>
Date: Thu, 14 Feb 2002 23:22:09 +0100

Hi Ganesh,

>    the observation domain could be hierarchical but have not put
Please excuse that I did not reply so far on your drawing.
I did not understand it when I saw it the first time and sent
it to the printer to study it later. Unfortunately, that's where
it still is. I should better have sent you my questions on this:

1. What is the difference between a meter process and a meter?
2. How and from where do meters get input? from meter processes?
3. Is there only one exporter per observation domain?
4. What would be the difference between an observation domain
   ID and an exporter ID?

Cheers,

    Juergen

---

From: Ganesh Sadasivan <[email-masked]>
Date: Thu, 14 Feb 2002 14:45:17 -0800

Hi Jurgene,
   The picture does not appear garbled in the archives. Probably you
   can look at it there.See inline for my answers.

> 1. What is the difference between a meter process and a meter?

Sorry that is a mistake, I meant "meter process" for both cases
(within and outside the observation domain).

> 2. How and from where do meters get input? from meter processes?

Since there is no separate meters and meter processes, you can think of
it as meter process within the observation domain, feeding the the meter
process outside the observation domain.

> 3. Is there only one exporter per observation domain?

No. The exporter need not be tied to an observation domain.
Example: Take the case of 2 linecards in a router., linecard 1 and
linecard 2. Each of the LC forms an observation domain. The
meters associated directly with observation points perform flow
creation/updation based on the packets. There could be meters at
a higher level (i.e per observation domain) that do furthur filtering
or aggregation.However, the flows collected at the observation
domain LC1& LC2 could be
exported via LC2. Thus the exporter which is common to LC1
and LC2 resides in LC2. The flow export packet from
each of the observation domain is identified by a unique ID.

> 4. What would be the difference between an observation domain

The export flow packet is identified by the observation domain ID .
I am not sure we need the Exporter ID (or do we?)

Thanks
Ganesh

---

From: Juergen Quittek <[email-masked]>
Date: Fri, 15 Feb 2002 00:29:57 +0100

Hi Ganesh,

>    can look at it there.See inline for my answers.

I could read the picture well (even in your reply).

> (within and outside the observation domain).

I think "meter" is a very well suited name for a box
hosting a metering process!

> process outside the observation domain.

Meter processes can be cascaded?
  What is the input to a meter process?
  It must have the same structure than the output?
  Otherwise I do not see how you could cascade them.

How is the total metering process shared by the packet meter
processes and the flow meter processes?
In other words: what are the differences between these
(different?) kinds of meters inside and outside the observation
domain?

    Juergen

---

From: Ganesh Sadasivan <[email-masked]>
Date: Thu, 14 Feb 2002 16:06:32 -0800

> hosting a metering process!

Ok.

> Meter processes can be cascaded?

yes.

>   What is the input to a meter process?

The input to the meter at the lowest level  is packets.
Flows are cretaed as a result of the lowest level
meter process. At subsequent level,the input to meter would
be flows. As I mentioned before the first level meter is a must
but not the subsequent ones.

>   Otherwise I do not see how you could cascade them.

What does this mean?

> processes and the flow meter processes?

They are independent entities.

> (different?) kinds of meters inside and outside the observation

The one outside observation domain operates on a bigger scope.
For example: Say there are 2  lowest level meters:
M11 extracts the 5 tuple IPV4  flows from interface1 at a
sampling rate of 1 in 10.
M12 extracts the 5 tuple IPV4  flows from interface2 at a
sampling rate of 1 in 20.
A meter at the level of observation domain sees the flows
created by M11 and M12 and does one more level of
sampling of all the flows created by both meters at a
rate of 1 in 2 before exporting it.

Ganesh

---

From: Juergen Quittek <[email-masked]>
Date: Fri, 15 Feb 2002 08:13:43 +0100

Hi Ganesh,

Your ideas are getting clearer to me. Still I have some
questions. Please find them inline.

> but not the subsequent ones.

What are 'flows' here? Do you mean already flow records
containing flow statistics?
Or are these packet headers tagged with some flow ID?

>>   Otherwise I do not see how you could cascade them.

If you have two function blocks with identical names
I assume their functions are identical. However the
functions seem to be applied to different data for inside
and outside meter processes. And I still do not understand
whether they produce similar output. For sure the last
stage must produce flow records. What is produces by the
earlier stages I asked already above.

> rate of 1 in 2 before exporting it.

I am not sure if you can do sampling at the second stage
after you already did packet counting at the first level.
What vwould the first level (inside) meter processes do
other than sampling? Do they classify packets, determine
the flow ID and forward some data on a per packet base to
the second stage (outside) meter processes?

Cheers,

    Juergen

---

From: Sebastian Zander <[email-masked]>
Date: Fri, 15 Feb 2002 08:59:53 +0100

Hi Juergen, Ganesh

I try to answer to you both in one mail.

Juergen:

I think header capturing should be named packet capturing for generality because
otherwise we have to define what header means (which layer) and impose limits
here?

I though header capturing is the functionality of the observation point!? Since
you make it a separate block what's the functionality left for observation point?

I agree to your blocks but do we need that high level of decomposition? Why not 
put packet capturing in observation point and flow record in meter? My intuitive 3
function blocks are: (1) Get packets from wire, (2) process packets & generate flow
records and (3) export flow records. If we really do need further decomposition than
it's fine with me but otherwise stay on the highest suitable level.

I think on a high level sampling can be done as part of packet capturing or as part 
of metering. With further decomposition I think a sampling block would be between 
capturing & meter. But do we really need to define this?  

BTW I like your overall name but maybe the "traffic" could be omitted to make it 
shorter?

Ganesh:

I think cascading meters is a nice idea but that implies that the input interface
of a meter is the same as the output interface as Juergen said. Why then limit the
model to only 2 steps?

I do not understand why in your picture the observation point is behind the meter.
I think it's the other way around. If then the observation point definition
is extended to cover an observation domain you have a fully recursive definition
and can cascade meters as you want (or not if you don't). This would read e.g.:
More than 1 meter could be grouped forming a (virtual) observation point for a
following meter. In contrast to a real observation point which is a lincard or
NIC. I think your example fits into this model.

This approach would converge the 2 models.

Additionally there was a comment from Peter Phaal who proposed to cascade exporters. 
I think this could be done with the model, if a collector can be seen as an observation
point/domain as well. 

My guess is that that we need an exporter ID in addition to obs. point ID(s) to 
differentiate between different exporters at the collector level. Would be
nice to have 2 ID types: (1) Where has the flow been observed, 
(2) who has told me about it. But im not 100% sure about this. Omitting an exporter 
ID would at least require some assumptions such that an observation point has only 
1 exporter...

Sebastian

---

From: Ganesh Sadasivan <[email-masked]>
Date: Fri, 15 Feb 2002 09:01:52 -0800

Hi Jurgene,
    See inline..

> Or are these packet headers tagged with some flow ID?

Yes flow records.
Do we need packet headers tagged
with flow ID at all ? Even if they appear in thet form of packet
header, it would be better to call them flow records even
though they may not the final structures to be exported.

> earlier stages I asked already above.

I agree with the fact  that input to the first level meter
is packets and to the rest of the levels it is flow records.
The outputs produced by meters at any level should be
flow records of some form which may not be what would
be finally exported.We can distinguish between packet level
and flow level meters.

> the second stage (outside) meter processes?

Packet counting is not the targeted functionality in the
example above.It is more of a traffic engineering application.
The first level would sample packets and install the flow records
into a flow cache (like how it is done in netflow today). The
second level takes the flows from flow cache as input and
does furthur sampling. The example is not a very clear one. It
would have been better if the sampling at second level is replaced
by say 2 filters filtering on some subsets of fields from the
flow records.

Thanks
Ganesh

---

From: Ganesh Sadasivan <[email-masked]>
Date: Fri, 15 Feb 2002 09:27:59 -0800

Hi Sebastian,
   The observation point is the place from where the packets from the wire are seen.
   The metering functionality acts on these packets to create flows. So even though
   the drawing shows  meter behind the observation point, the metering is happening
   at the same place as the observation point in most of the cases. In the case of
   centralised proicessing where all the packets reach a shared memory, the metering
   is done after the packet is seen at the observation point.
   See more inline..

> you make it a separate block what's the functionality left for observation point?

Your case of  capturing the packet header is a part of flow creation. This is a part of
    metering process itself. In some other case this functionality may not be necessary.
    Say if one is looking for packet length, capturing header is not a must.

> of a meter is the same as the output interface as Juergen said. Why then limit the

Yes the first level meter which happens at the observation point, sees packets
instead of flows. The rest of the levels should be seeing  flow records of
some form which need not be the final exported records.

> NIC. I think your example fits into this model.

Yes one can cascade meters at multiple levels. But why we did not mention it is because
from a practical standpoint, this would suffice. Today in most of the cases
one level metering is what is done and  recently only we heard about more requirements
for  a 2nd level meter .

> ID would at least require some assumptions such that an observation point has only

Agreed on the last sentence. But why does the collector need to know
"(2) who has told me about it"? To me the observation domain ID should
identify the flow packets within and across exporters.

Thanks
Ganesh

---

From: Juergen Quittek <[email-masked]>
Date: Sat, 16 Feb 2002 16:42:42 +0100

Hi Ganesh,

Thank you for your explanations. Now I got your points
and agree, that this is a reasonable architecture for
an IPFIX implementation on a router.

Now, regarding terminology in the requirements document,
do you think we need to to mention the observation domain
in there?

I do not think so, but would suggest to cover this only in the
architecture. Not because it does not make sense, but because
so far we do not have specific requirements for observation
domains. If we have some I am fine with adding (and explaining)
it.

Until then I suggest the following list of terms
omitting the term 'meter':

- obs. point
- metering process
- flow record
- exporter (or export process)
- collector
- export process

    Juergen

---

From: Sebastian Zander <[email-masked]>
Date: Mon, 18 Feb 2002 09:05:36 +0100

Hi Ganesh,

comments inline

>    is done after the packet is seen at the observation point.

I agree. What has confused me in your picture is that in the upper part the meter is
above observation points and then there are meter and export below. I would suggest
putting the 2nd level meter and export above (with the export as the top) to clearly show 
the way of the data: observation->1st meter->2nd meter->export
 
>     Say if one is looking for packet length, capturing header is not a must.

That's true for a router. A passive probe must always do packet capturing to get packet
information.

Again I propose to avoid that kind of discussion by not getting into too much details 
in the model. Do we really need a separate block for packet capturing in the model? 

> identify the flow packets within and across exporters.

I was thinking about the following: You get data from more than one IPFIX enabled box and 
you have a collector which performs aggregation on the data and re-exports it via IPFIX.
Would be nice to be able to do this for acalability. But the aggregator could also be
identified by an observation point id if the definition is flexible enough. The
aggregator should also include the list of original observation points. This would
be a similar approach as the SSRC, CSRC in Real-Time Transport Protocol (RTP).

In case of in-band configuration you may probably need an exporter id for identification 
of an ipfix exporter. 

In case of having an authentication mechanism between exporter and collector you also need 
some kind of exporter id.

But if you consider the last 2 things out of scope a flexible observation point id
seems sufficient.
 
Sebastian