Subject: [IPFIX] IPFIX Mediation terminologies again

From: Atsushi Kobayashi <[email-masked]>
Date: Mon, 27 Jul 2009 13:37:32 +0900

Dear Gerhard, Brian, Benoit, and all

After my much thought and Gerhard suggestion, I think that current
terminologies need some minor changes. I will appreciate your reviewing
them in prior to the meeting. Sorry for long mail.

OLD (-04)

   IPFIX Mediation

      IPFIX Mediation is the manipulation and conversion of records for
      subsequent export using IPFIX, by applying mediation functions to
      a stream of records.

- Manipulation of records seem not include selection function. According
to PSAMP Selection Process, it say "a stream of records" instead of
"records."
- I think first reader maybe wonder what mediation functions indicate.
We can remove the last sentense.

NEW

   IPFIX Mediation

      IPFIX Mediation is the manipulation and conversion of a stream of
      records for subsequent export using IPFIX.

OLD (-04)

   Intermediate Process

      An Intermediate Process takes a sequence of records from a
      Collecting Process, Metering Process, IPFIX File Reader, or
      another Intermediate Process; performs some transformation on
      these records based upon the content of the records themselves,
      keeps state across multiple records, configuration parameters, or
      other data; and passes a sequence of transformed records on to an
      Exporting Process, IPFIX File Writer, or another Intermediate
      Process.  Typically, an Intermediate Process is hosted by an IPFIX
      Mediator.  Alternatively, an Intermediate Process may be hosted by
      an Original Exporter.

- There is no description regarding IPFIX Mediation.
- Configuration parameters are covered in other data. Maybe we can
remove this.
- I found typo "keeps state", it will be replaced with "kept state".

NEW

   Intermediate Process

      An Intermediate Process implements functions of IPFIX Mediation.
      It takes a stream of records as its input from a Collecting
      Process, Metering Process, IPFIX File Reader, or another
      Intermediate Process; performs some transformation on these
      records based upon the content of the records themselves, kept
      state across multiple records, or other data; and passes a stream
      of transformed records as its output on to an Exporting Process,
      IPFIX File Writer, or another Intermediate Process.
      Typically, an Intermediate Process is hosted by an IPFIX Mediator.
      Alternatively, an Intermediate Process may be hosted by
      an Original Exporter.

OLD (-04)

   IPFIX Mediator

      An IPFIX Mediator is an IPFIX Device that provides mediation
      capabilities by receiving records from some data source, hosting
      zero or more Intermediate Processes to transform those records,
      and exporting those records in IPFIX Messages via an Exporting
      Process.  In the common case, an IPFIX Mediator receives records
      from a Collecting Process but could also receive records from data
      sources not encoded using IPFIX, e.g., in the case of NetFlow V9
      protocol translation.

- Which case is considered in zero Intermediate Process?
Regarding transport protocol mediation, we concluded simple relaying
UDP-to-UDP is not IPFIX Mediation. However, relaying UDP-to-SCTP needs
some function to manipulate IPFIX Messages. For example, Exporting
Process needs to send Template Withdrawal Messages, when a Collecting
Process detects refresh time out of Template.  I think that an
Intermediate Process can give the trigger to Exporting Process.
In my conclusion, zero Intermediate Process means not IPFIX Mediation.

NEW

   IPFIX Mediator

      An IPFIX Mediator is an IPFIX Device that provides mediation
      capabilities by receiving records from some data source, hosting
      one or more Intermediate Processes to transform those records,
      and exporting those records in IPFIX Messages via an Exporting
      Process.  In the common case, an IPFIX Mediator receives records
      from a Collecting Process but could also receive records from data
      sources not encoded using IPFIX, e.g., in the case of NetFlow V9
      protocol translation.

Regards,
Atsushi

-- 
Atsushi Kobayashi <[email-masked]>

---

From: Brian Trammell <[email-masked]>
Date: Mon, 27 Jul 2009 08:00:13 +0200

hi Atsushi, all:

replies inline.

>      records for subsequent export using IPFIX.

Looks good.

> - There is no description regarding IPFIX Mediation.

Why does the IntP definition need to say anything explicitly about  
mediation? It's obvious, to me anyway, from the fact that it's defined  
in the mediation document, and that it is hosted in a Mediator/OE...

> - Configuration parameters are covered in other data. Maybe we can

Good point.

> - I found typo "keeps state", it will be replaced with "kept state".

This is not an error; "keeps state" is in present tense in parallel  
with all other clauses in the list, and properly spelled. If the  
wording is confusing, suggest "maintains state" instead.

>      sources not encoded using IPFIX, e.g., in the case of NetFlow V9

This is still not accurate; it is entirely possible (i.e. distributor  
use case) that the records are not transformed, only rerouted. If we  
insist on requiring 1..n multiplicity on IntP in a Mediator, then  
suggest "hosting one or more Intermediate Processes to process those  
records"...

Regards,

Brian

---

From: Atsushi Kobayashi <[email-masked]>
Date: Mon, 27 Jul 2009 15:59:44 +0900

Hi Brian, and all,

On Mon, 27 Jul 2009 08:00:13 +0200

> in the mediation document, and that it is hosted in a Mediator/OE...

Sorry for lack of explanation. The current IntP takes a stream of record
as its input and transmits a stream of records as its output. In below
case, IntP gives the trigger to Exporting Process to send Withdrawal
Message rather than manipulating a stream of records. So, although we
can define another Process, I think it is better to avoid creating new
Process.

IntP can execute other function related to template or IPFIX Messages by
adding the first sentence. Does it make sense?

> wording is confusing, suggest "maintains state" instead.

Does it need a semicolon before "keeps state"?

An Intermediate Process takes a sequence of records from a Collecting
Process, Metering Process, IPFIX File Reader, or another Intermediate
Process; performs some transformation on these records based upon the
content of the records themselves; keeps state across multiple records,
                                 ^^^^^^^^^^^^^^^^^^^^^^^^

> suggest "hosting one or more Intermediate Processes to process those  

Good.

Regards,
Atsushi

---

From: Brian Trammell <[email-masked]>
Date: Mon, 27 Jul 2009 10:33:54 +0200

hi, Atsushi,

replies inline. :)

> adding the first sentence. Does it make sense?

Yes, now it does. Thanks. :)

> content of the records themselves; keeps state across multiple  

yes. but... ahhhhh... Okay, I see the issue here. It does keep state,  
but it also needs to make decisions based on the state that it keeps.  
Keeps/kept makes this read a bit confusingly, so let me see if I can  
fix this by rewording slightly:

NEW

  Intermediate Process

     An Intermediate Process implements the functions of IPFIX  
Mediation.
     It takes a stream of records as its input from a Collecting
     Process, Metering Process, IPFIX File Reader, or another
     Intermediate Process; performs some transformation on these
     records based upon the content of each record, state maintained  
across
     multiple records, or other data sources; and passes a stream
     of transformed records as its output on to an Exporting Process,
     IPFIX File Writer, or another Intermediate Process.
     Typically, an Intermediate Process is hosted by an IPFIX Mediator.
     Alternatively, an Intermediate Process may be hosted by
     an Original Exporter.

This still needs to handle the case of record routing -- where the  
output of the IntP is some configuration input to a downstream EP. So  
how about:

NEWER

  Intermediate Process

     An Intermediate Process implements the functions of IPFIX  
Mediation.
     It takes a stream of records as its input from a Collecting
     Process, Metering Process, IPFIX File Reader, or another
     Intermediate Process; performs some transformation on these
     records, or makes some decision about their subsequent processing,
     based upon the content of each record, state maintained across
     multiple records, or other data sources; and passes a stream
     of transformed records as its output on to an Exporting Process,
     IPFIX File Writer, or another Intermediate Process.
     Typically, an Intermediate Process is hosted by an IPFIX Mediator.
     Alternatively, an Intermediate Process may be hosted by
     an Original Exporter.

Regards,

Brian

---

From: Gerhard Muenz <[email-masked]>
Date: Mon, 27 Jul 2009 10:45:13 +0200

Hi Brian,

>>> in the mediation document, and that it is hosted in a Mediator/OE...

There does not have to be a reference to IPFIX Mediation in the
definition of IP. Though, I thought it would make sense.

> Keeps/kept makes this read a bit confusingly, so let me see if I can fix
> this by rewording slightly:
> 
> NEW
> 
>  Intermediate Process
> 
>     An Intermediate Process implements the functions of IPFIX Mediation.

Why did you add "the"?

We explicitly wrote "implements functions" and not "implements the
functions" because an Intermediate Process can implement any number and
types of functions of IPFIX Mediation, but probably not all functions
that exist.

>     records based upon the content of each record, state maintained across
>     multiple records, or other data sources; 

fine.

>     and passes a stream
>     An Intermediate Process implements the functions of IPFIX Mediation.

same as above.

>     multiple records, or other data sources; 

also fine.

>     and passes a stream

Gerhard

-- 
Dipl.-Ing. Gerhard Münz
Chair for Network Architectures and Services (I8)
Department of Informatics
Technische Universität München
Boltzmannstr. 3, 85748 Garching bei München, Germany
Phone:  +49 89 289-18008       Fax: +49 89 289-18033
E-mail: [email-masked]    WWW: http://www.net.in.tum.de/~muenz

---

From: Brian Trammell <[email-masked]>
Date: Mon, 27 Jul 2009 10:48:35 +0200

hi, Gerhard,

inline. :)

Regards,
Brian

> types of functions of IPFIX Mediation, but probably not all functions

Point. Oops. Hadn't actually noticed that I'd added it.

>>    An Intermediate Process implements the functions of IPFIX  

Suggest, then, "An Intermediate Process implements IPFIX Mediation  
functions" (prepositional form here is a little awkward).

---

From: Benoit Claise <[email-masked]>
Date: Mon, 27 Jul 2009 11:08:17 +0200

Dear all,

I have also some (minor) comments on the terminology.
Why don't we simply meet after the IPFIX meeting today to sort out the 
remaining issues?
Most of the time, a 15 minute discussion is more efficient than a long 
email thread...

Regards, Benoit.

---

From: Brian Trammell <[email-masked]>
Date: Mon, 27 Jul 2009 11:16:50 +0200

hi, Benoit,

Indeed. Sounds good.

Regards,

Brian

On Jul 27, 2009, at 11:08 AM, Benoit Claise wrote:

> Why don't we simply meet after the IPFIX meeting today to sort out  
> the remaining issues?
> Most of the time, a 15 minute discussion is more efficient than a  
> long email thread...
>
> Regards, Benoit.

---

From: Atsushi Kobayashi <[email-masked]>
Date: Mon, 27 Jul 2009 18:18:55 +0900

Dear Benoit, Brian and Gerhard,

Benoit, Ok.
The following terms include Brian and Gerhard feedbacks.

IPFIX Mediation

Intermediate Process

    An Intermediate Process implements IPFIX Mediation functions. It
    takes a stream of records as its input from a Collecting Process,
    Metering Process, IPFIX File Reader, or another Intermediate
    Process; performs some transformation on these records, or makes
    some decision about their subsequent processing, based upon the
    content of each record, state maintained across multiple records, or
    other data sources; and passes a stream of transformed records as
    its output on to an Exporting Process, IPFIX File Writer, or another
    Intermediate Process. Typically, an Intermediate Process is hosted
    by an IPFIX Mediator. Alternatively, an Intermediate Process may be
    hosted by an Original Exporter.

IPFIX Mediator

    An IPFIX Mediator is an IPFIX Device that provides mediation
    capabilities by receiving records from some data source, hosting one
    or more Intermediate Processes to process those records, and
    exporting those records in IPFIX Messages via an Exporting Process.
    In the common case, an IPFIX Mediator receives records from a
    Collecting Process but could also receive records from data sources
    not encoded using IPFIX, e.g., in the case of NetFlow V9 protocol
    translation.

Regards,
Atsushi

On Mon, 27 Jul 2009 11:08:17 +0200
Benoit Claise <[email-masked]> wrote:

> _______________________________________________
> IPFIX mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ipfix