Subject: [IPFIX] Re: [PSAMP] Metering Process and/or Selection Process

From: Gerhard Muenz <[email-masked]>
Date: Mon, 16 Oct 2006 12:40:02 +0200

Hi Benoit,

Yes, solution 3 is a good start. I see the following possible enhancements:

1) In order to avoid confusion, we should not define processes within
processes but find another word for "Selection Process". For example, we
could call it "Packet Selection Function". (If you do not like
"function", you can replace it by something else.)

2) Sampled packets can be either used for PSAMP or IPFIX export. In the
first case, we need a "Packet Reporting Function" to extract the
exported IEs and to generate the report stream. In the second case, we
have a "Flow Metering Function" that generates the flow records.

So the whole picture could look like this:

PSAMP Device:

           +--------------------------+
           | Metering Process         |
           | +---------+  +---------+ |   +-----------+
 Observed  | |Packet   |  |Packet   | |   | Exporting |
 Packet--->| |Selection|->|Reporting|---->| Process   |--->Collector
 Stream    | +---------+  +---------+ |   +-----------+
           +--------------------------+

IPFIX Device:

           +--------------------------+
           | Metering Process         |
           | +---------+  +---------+ |   +-----------+
 Observed  | |Packet   |  |Flow     | |   | Exporting |
 Packet--->| |Selection|->|Metering |---->| Process   |--->Collector
 Stream    | +---------+  +---------+ |   +-----------+
           +--------------------------+

Regards,
Gerhard

Benoit Claise wrote:
>  Hi Gerhard,
> 
> Do I understand correctly that you favor solution 3 in my email.
> This is also my preferred solution.
> 
> Regards, Benoit.
>> Benoit,
>>
>> In draft-muenz-ipfix-configuration-00, I tried to combine the different
>> aspects of an IPFIX/PSAMP Metering Process:
>>
>>
>> 4.3.  Metering Process
>>
>>    Metering Processes perform the data processing within the IPFIX
>>    Device.  In case that the input of a Metering Process is raw packets
>>    from Observation Points, the data are processed and transformed into
>>    exportable records.  If the input is records received from Collecting
>>    Processes or other Metering Processes, the records are processed and
>>    transformed into a new stream of records.
>>
>>    In principle, packet-based and flow-based metering can be
>>    distinguished.  Various packet-based processing techniques are
>>    specified in [I-D.ietf-psamp-sample-tech], and their configurable
>>    parameters are defined in [I-D.ietf-psamp-mib].  [I-D.ietf-psamp-
>>    framework] divides the packet-based Metering Process into a packet
>>    selection process and a reporting process.  This is mapped on
>>    corresponding structures in the configuration data model.
>>
>>    Flow-based processing techniques are described in [I-D.ietf-ipfix-
>>    architecture] and [I-D.dressler-ipfix-aggregation].  [RFC3917] gives
>>    an overview on the configurable parameters.  [I-D.dressler-ipfix-
>>    aggregation] proposes a description language for flow metering rules
>>    (therein called aggregation rules) that define the Flow Keys as well
>>    as the metered and exported flow attributes.  This rule-based
>>    description of flow metering can also be applied to devices that do
>>    not support multiple metering rules.  For example, if a device
>>    performs flow metering with a single set of Flow Keys only, this can
>>    be mapped to exactly one metering rule.
>>
>>    All in all, the configurable parameters of a Metering Process can be
>>    summarized as follows:
>>
>>    Packet Selection:
>>       Incoming raw packets can be processed in a sequence of filters and
>>       sampling algorithms.  The possible filtering and sampling
>>       parameters are defined in [I-D.ietf-psamp-mib].
>>
>>    Packet Reporting:
>>       If the output of the Metering Process are records with packet-
>>       based monitoring data (or PSAMP data), the packet reporting
>>       parameters define the Information Elements that are present in the
>>       records.
>>
>>    Flow Metering:
>>       Flow-based metering can be described by one or more metering
>>       rules.  Each metering rule defines the Information Elements that
>>       are present in the resulting records.  Two different types of
>>       Information Elements are distinguished: Information Elements that
>>       are used as Flow Keys and Information Elements specifying the
>>       additional information reported for each flow.  The application of
>>       a metering rule can be restricted to incoming data matching given
>>       patterns.  Apart from metering rules, the flow-based data
>>       processing depends on active and inactive flow timeout values that
>>       control the flow expiration.
>>
>>    Next Pointer:
>>       The output of a Metering Process can be passed to Exporting
>>       Processes and/or to other Metering Processes.
>>
>>
>> This allows mapping different realizations of the Metering Processes:
>>
>> PSAMP probe: Packet Selection (optional) + Packet Reporting
>> IPFIX probe: Packet Selecton (optional) + Flow Metering
>> Concentrator: Flow Metering (alone)
>>
>> Note that my considerations were based on the assumption that we combine
>> the former PSAMP Measurement Process and the IPFIX Metering Process into
>> a single process. Of course, there are other possibilities, e.g.
>> defining new types of Processes as you propose.
>>
>> Regards,
>> Gerhard
>>
>>
>> Benoit Claise wrote:
>>   
>>>   Tanja, all,
>>>
>>> I want to start a new email thread, discussing one of the point raised
>>> by Tanja in
>>> http://www1.ietf.org/mail-archive/web/psamp/current/msg00325.html
>>>
>>>     2: Section 3.3.1: As far as I know we agreed that we use the same
>>>     metering process definition for IPFIX and PSAMP. So the only
>>>     difference I see between PSAMP and IPFIX processes are the
>>>     Information elements that are reported. So I found section 3.3.1. a
>>>     bit confusing.
>>>
>>> Note: in Dallas, we agreed to replace the "measurement process" by the
>>> "metering process"
>>> While re-reading this section 3.3.1, I agree this is confusing. So we
>>> have to change this text/picture     
>>>
>>>    The figure B indicates the sequence of the processes (selection and 
>>>    exporting) within the PSAMP Device. 
>>>           
>>>                   +----------+      +-----------+ 
>>>         Observed  | Metering |      | Exporting | 
>>>         Packet--->| Process  |----->| Process   |--->Collector 
>>>         Stream    +----------+      +-----------+ 
>>>     
>>>        Figure B: PSAMP Processes 
>>>                           
>>>    The Selection Process, which takes an Observed Packet Stream as its 
>>>    input and produces Packet Reports as its output, is an integral part 
>>>    of the Metering Process, which by its definition produces Flow 
>>>    Records as its output. 
>>>
>>>
>>>
>>> I was wondering how to change this: basically, we've got 2 choices!
>>> Drawing 1:
>>>  
>>>            +------------------+
>>>            | Metering Process |
>>>            | +-----------+    |     +-----------+
>>>  Observed  | | Selection |    |     | Exporting |
>>>  Packet--->| | Process   |--------->| Process   |--->Collector
>>>  Stream    | +-----------+    |     +-----------+
>>>            +------------------+
>>>
>>> This would be consistent with the text in section 3.3.1:
>>>
>>>     The Selection Process, which takes an Observed Packet Stream as its
>>>     input and produces Packet Reports as its output, is an integral part
>>>     of the Metering Process, which by its definition produces Flow
>>>     Records as its output.
>>>
>>> Drawing 2:
>>>
>>>            +-----------+   +----------+    +-----------+
>>>  Observed  | Selection |   | Metering |    | Exporting |
>>>  Packet--->| Process   |-->| Process  |--->| Process   |--->Collector
>>>  Stream    +-----------+   +----------+    +-----------+
>>>  
>>>     Figure B: PSAMP Processes
>>>
>>> This would be consistent with the text in section 3.2.2:
>>>
>>>    Selection Process 
>>>          
>>>    A Selection Process takes the Observed Packet Stream as its input and 
>>>    selects a subset of that stream as its output. 
>>>
>>>
>>>
>>>
>>>
>>> However, we quickly realize that we've an inconsistency in the following
>>> definitions;
>>>
>>>    Selection Process 
>>>          
>>>    A Selection Process takes the Observed Packet Stream as its input and 
>>>    selects a subset of that stream as its output. 
>>>
>>>    Report Stream 
>>>          
>>>    The Report Stream is the output of a Selection Process, comprising 
>>>    two distinguished types of information: Packet Reports, and Report 
>>>    Interpretation.
>>>
>>>
>>> One definition says that the selection process doesn't generate the
>>> report stream while another says the reverse.
>>> To solve this issue we've got 2 solutions:
>>>
>>>
>>> Solution 1:
>>> We divide the metering process into a Selection Process (Packet Stream
>>> in, Selected Packets out) and a Reporting Process (Selected Packets In,
>>> Packet Reports Out). This is the way it's done in [PSAMP-FRAMEWORK]:
>>>
>>>       * Report Stream: 
>>>             
>>>            The report stream is the output of a reporting process, 
>>>            comprising two distinguished types of information: packet 
>>>            reports, and report interpretation. 
>>>
>>> As far as I recall, we decided to remove the Reporting Process from
>>> [PSAMP-PROTO]... This rules out solution 1.
>>>
>>> Solution 2:
>>>
>>>            +-----------+   +----------+    +-----------+
>>>  Observed  | Selection |   | Metering |    | Exporting |
>>>  Packet--->| Process   |-->| Process  |--->| Process   |--->Collector
>>>  Stream    +-----------+   +----------+    +-----------+
>>>  
>>>     Figure B: PSAMP Processes
>>>
>>> So it means that PSAMP = IPFIX + an extra Selection Process.
>>> We must update
>>>
>>>    Report Stream 
>>>          
>>>    The Report Stream is the output of a Metering (AND NOT SELECTION) Process, 
>>>    comprising two distinguished types of information: Packet Reports, and Report 
>>>    Interpretation
>>>
>>>
>>> Solution 3:
>>>
>>>  
>>>            +------------------+
>>>            | Metering Process |
>>>            | +-----------+    |     +-----------+
>>>  Observed  | | Selection |    |     | Exporting |
>>>  Packet--->| | Process   |--------->| Process   |--->Collector
>>>  Stream    | +-----------+    |     +-----------+
>>>            +------------------+
>>>
>>> So the metering process: Packet Stream in, Packet Reports out.
>>> We must update
>>>
>>>    Report Stream 
>>>          
>>>    The Report Stream is the output of a Metering (AND NOT SELECTION) Process, 
>>>    comprising two distinguished types of information: Packet Reports, and Report 
>>>    Interpretation
>>>
>>>
>>> What's your preferred solution?
>>>
>>> Regards, Benoit.
>>>
>>>
>>>
>>>  
>>>
>>>
>>> ------------------------------------------------------------------------
>>>
>>> _______________________________________________
>>> PSAMP mailing list
>>> [email-masked] <mailto:[email-masked]>
>>> https://www1.ietf.org/mailman/listinfo/psamp
>>>     
>> _______________________________________________
>> PSAMP mailing list
>> [email-masked] <mailto:[email-masked]>
>> https://www1.ietf.org/mailman/listinfo/psamp
>>   
> 

_______________________________________________
IPFIX mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ipfix

---

From: Gerhard Muenz <[email-masked]>
Date: Mon, 16 Oct 2006 12:40:02 +0200

Hi Benoit,

PSAMP Device:

IPFIX Device:

Regards,
Gerhard