Subject: [PSAMP] Metering Process and/or Selection Process

From: Benoit Claise <[email-masked]>
Date: Fri, 13 Oct 2006 18:31:41 +0200

Tanja, all,

I want to start a new email thread, discussing one of the point raised 
by Tanja in 
http://www1.ietf.org/mail-archive/web/psamp/current/msg00325.html

    2: Section 3.3.1: As far as I know we agreed that we use the same
    metering process definition for IPFIX and PSAMP. So the only
    difference I see between PSAMP and IPFIX processes are the
    Information elements that are reported. So I found section 3.3.1. a
    bit confusing.

Note: in Dallas, we agreed to replace the "measurement process" by the 
"metering process"
While re-reading this section 3.3.1, I agree this is confusing. So we 
have to change this text/picture     

   The figure B indicates the sequence of the processes (selection and 
   exporting) within the PSAMP Device. 
          
                  +----------+      +-----------+ 
        Observed  | Metering |      | Exporting | 
        Packet--->| Process  |----->| Process   |--->Collector 
        Stream    +----------+      +-----------+ 
    
       Figure B: PSAMP Processes 
                          
   The Selection Process, which takes an Observed Packet Stream as its 
   input and produces Packet Reports as its output, is an integral part 
   of the Metering Process, which by its definition produces Flow 
   Records as its output. 

I was wondering how to change this: basically, we've got 2 choices!
_Drawing 1:_
 
           +------------------+
           | Metering Process |
           | +-----------+    |     +-----------+
 Observed  | | Selection |    |     | Exporting |
 Packet--->| | Process   |--------->| Process   |--->Collector
 Stream    | +-----------+    |     +-----------+
           +------------------+

This would be consistent with the text in section 3.3.1:

    The Selection Process, which takes an Observed Packet Stream as its
    input and produces Packet Reports as its output, is an integral part
    of the Metering Process, which by its definition produces Flow
    Records as its output.

_Drawing 2:_

           +-----------+   +----------+    +-----------+
 Observed  | Selection |   | Metering |    | Exporting |
 Packet--->| Process   |-->| Process  |--->| Process   |--->Collector
 Stream    +-----------+   +----------+    +-----------+
 
    Figure B: PSAMP Processes

This would be consistent with the text in section 3.2.2:

   Selection Process 
         
   A Selection Process takes the Observed Packet Stream as its input and 
   selects a subset of that stream as its output. 

However, we quickly realize that we've an inconsistency in the following 
definitions;

   Selection Process 
         
   A Selection Process takes the Observed Packet Stream as its input and 
   selects a subset of that stream as its output. 

   Report Stream 
         
   The Report Stream is the output of a Selection Process, comprising 
   two distinguished types of information: Packet Reports, and Report 
   Interpretation.

One definition says that the selection process doesn't generate the 
report stream while another says the reverse.
To solve this issue we've got 2 solutions:

_Solution 1:_
We divide the metering process into a Selection Process (Packet Stream 
in, Selected Packets out) and a Reporting Process (Selected Packets In, 
Packet Reports Out). This is the way it's done in [PSAMP-FRAMEWORK]:

      * Report Stream: 
            
           The report stream is the output of a reporting process, 
           comprising two distinguished types of information: packet 
           reports, and report interpretation. 

As far as I recall, we decided to remove the Reporting Process from 
[PSAMP-PROTO]... This rules out solution 1.

_Solution 2:_

           +-----------+   +----------+    +-----------+
 Observed  | Selection |   | Metering |    | Exporting |
 Packet--->| Process   |-->| Process  |--->| Process   |--->Collector
 Stream    +-----------+   +----------+    +-----------+
 
    Figure B: PSAMP Processes

So it means that PSAMP = IPFIX + an extra Selection Process.
We must update

   Report Stream 
         
   The Report Stream is the output of a Metering (AND NOT SELECTION) Process, 
   comprising two distinguished types of information: Packet Reports, and Report 
   Interpretation

_Solution 3: _

           +------------------+
           | Metering Process |
           | +-----------+    |     +-----------+
 Observed  | | Selection |    |     | Exporting |
 Packet--->| | Process   |--------->| Process   |--->Collector
 Stream    | +-----------+    |     +-----------+
           +------------------+

So the metering process: Packet Stream in, Packet Reports out.
We must update

   Report Stream 
         
   The Report Stream is the output of a Metering (AND NOT SELECTION) Process, 
   comprising two distinguished types of information: Packet Reports, and Report 
   Interpretation

What's your preferred solution?

Regards, Benoit.

_______________________________________________
PSAMP mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/psamp

---

From: Gerhard Muenz <[email-masked]>
Date: Fri, 13 Oct 2006 19:54:33 +0200

Benoit,

In draft-muenz-ipfix-configuration-00, I tried to combine the different
aspects of an IPFIX/PSAMP Metering Process:

4.3.  Metering Process

   Metering Processes perform the data processing within the IPFIX
   Device.  In case that the input of a Metering Process is raw packets
   from Observation Points, the data are processed and transformed into
   exportable records.  If the input is records received from Collecting
   Processes or other Metering Processes, the records are processed and
   transformed into a new stream of records.

   In principle, packet-based and flow-based metering can be
   distinguished.  Various packet-based processing techniques are
   specified in [I-D.ietf-psamp-sample-tech], and their configurable
   parameters are defined in [I-D.ietf-psamp-mib].  [I-D.ietf-psamp-
   framework] divides the packet-based Metering Process into a packet
   selection process and a reporting process.  This is mapped on
   corresponding structures in the configuration data model.

   Flow-based processing techniques are described in [I-D.ietf-ipfix-
   architecture] and [I-D.dressler-ipfix-aggregation].  [RFC3917] gives
   an overview on the configurable parameters.  [I-D.dressler-ipfix-
   aggregation] proposes a description language for flow metering rules
   (therein called aggregation rules) that define the Flow Keys as well
   as the metered and exported flow attributes.  This rule-based
   description of flow metering can also be applied to devices that do
   not support multiple metering rules.  For example, if a device
   performs flow metering with a single set of Flow Keys only, this can
   be mapped to exactly one metering rule.

   All in all, the configurable parameters of a Metering Process can be
   summarized as follows:

   Packet Selection:
      Incoming raw packets can be processed in a sequence of filters and
      sampling algorithms.  The possible filtering and sampling
      parameters are defined in [I-D.ietf-psamp-mib].

   Packet Reporting:
      If the output of the Metering Process are records with packet-
      based monitoring data (or PSAMP data), the packet reporting
      parameters define the Information Elements that are present in the
      records.

   Flow Metering:
      Flow-based metering can be described by one or more metering
      rules.  Each metering rule defines the Information Elements that
      are present in the resulting records.  Two different types of
      Information Elements are distinguished: Information Elements that
      are used as Flow Keys and Information Elements specifying the
      additional information reported for each flow.  The application of
      a metering rule can be restricted to incoming data matching given
      patterns.  Apart from metering rules, the flow-based data
      processing depends on active and inactive flow timeout values that
      control the flow expiration.

   Next Pointer:
      The output of a Metering Process can be passed to Exporting
      Processes and/or to other Metering Processes.

This allows mapping different realizations of the Metering Processes:

PSAMP probe: Packet Selection (optional) + Packet Reporting
IPFIX probe: Packet Selecton (optional) + Flow Metering
Concentrator: Flow Metering (alone)

Note that my considerations were based on the assumption that we combine
the former PSAMP Measurement Process and the IPFIX Metering Process into
a single process. Of course, there are other possibilities, e.g.
defining new types of Processes as you propose.

Regards,
Gerhard

Benoit Claise wrote:
> Drawing 1:
> Drawing 2:
> Solution 1:
> Solution 2:
> Solution 3:
> ------------------------------------------------------------------------
>

---

From: Gerhard Muenz <[email-masked]>
Date: Mon, 16 Oct 2006 12:44:03 +0200

Hi Benoit,

Yes, solution 3 is a good start. I see the following possible enhancements:

1) In order to avoid confusion, we should not define processes within
processes but find another word for "Selection Process". For example, we
could call it "Packet Selection Function". (If you do not like
"function", you can replace it by something else.)

2) Sampled packets can be either used for PSAMP or IPFIX export. In the
first case, we need a "Packet Reporting Function" to extract the
exported IEs and to generate the report stream. In the second case, we
have a "Flow Metering Function" that generates the flow records.

So the whole picture could look like this:

PSAMP Device:

           +--------------------------+
           | Metering Process         |
           | +---------+  +---------+ |   +-----------+
 Observed  | |Packet   |  |Packet   | |   | Exporting |
 Packet--->| |Selection|->|Reporting|---->| Process   |--->Collector
 Stream    | +---------+  +---------+ |   +-----------+
           +--------------------------+

IPFIX Device:

           +--------------------------+
           | Metering Process         |
           | +---------+  +---------+ |   +-----------+
 Observed  | |Packet   |  |Flow     | |   | Exporting |
 Packet--->| |Selection|->|Metering |---->| Process   |--->Collector
 Stream    | +---------+  +---------+ |   +-----------+
           +--------------------------+

Regards,
Gerhard

Benoit Claise wrote:
>  Hi Gerhard,
> 
> Do I understand correctly that you favor solution 3 in my email.
> This is also my preferred solution.
> 
> Regards, Benoit.
>> Benoit,
>>
>>> [email-masked] <mailto:[email-masked]>
>> [email-masked] <mailto:[email-masked]>

---

From: Benoit Claise <[email-masked]>
Date: Mon, 16 Oct 2006 12:51:55 +0200

Gerhard,
> processes 
I don't see why!
> but find another word for "Selection Process". For example, we
A couple of remarks:
- this is too late to add a "Flow Metering" process into the IPFIX device.
- your "packet reporting" is the "reporting process" as defined 
initially by [PSAMP-ARCH]. We decided to remove this term in one of the 
last IETF meeting.

So, this is the reason why I think the best/easier solution is:

    Solution 3:

               +------------------+
               | Metering Process |

       Report Stream 

Regards, Benoit.

---

From: Gerhard Muenz <[email-masked]>
Date: Mon, 16 Oct 2006 14:28:16 +0200

Benoit,

> - this is too late to add a "Flow Metering" process into the IPFIX device.

It is not a process, but a function(ality) of the Metering Process which
is implicitly described in the IPFIX-ARCH. Hence, it is not necessary to
change anything in the IPFIX documents. However, I think it is useful to
clarify the relationship between sampling and flow metering in the
PSAMP-ARCH.

> initially by [PSAMP-ARCH]. We decided to remove this term in one of the

That is fine, I also would not call it a process. But the
function(ality) is still needed, so why not clarifying in PSAMP-ARCH
where it is and what it does.

Regards,
Gerhard