Subject: Re: [IPFIX] WG last call on draft-ietf-ipfix-mediators-problem-statement-06

From: Atsushi Kobayashi <[email-masked]>
Date: Thu, 03 Dec 2009 01:13:04 +0900

Hi Gerhard, and all,

Sorry for late reply. I have missed this mail till now.

On Sat, 14 Nov 2009 23:31:44 +0100
Gerhard Muenz <[email-masked]> wrote:

> Hi Atsushi,
> comments inline.
> Regards,
> Gerhard
> Atsushi Kobayashi wrote:
> > Hi Gerhard,
snip

> >>
> >> BTW, I would be careful with terms like "optimize" and "optimization".
> >>
> > 
> > I will change it as follows. Could you please rewrite it by your word,
> > if not fine.
> 
> As I do not know what you want to say, it is difficult for me to rewrite
> it in my own words.
> 
> > 
> > 1.  Introduction
> > 
> >    One advantage of Flow-based measurement results from easily offering
> >    the traffic monitoring of a huge amount of traffic.  While the usage
> >    is applied to any networks and to multiple measurement applications,
> 
> You could start like this:
> 
> An advantage of Flow-based measurement is that it allows monitoring hugh
> amounts of traffic observed at distributed observation points.
> Flow-based measurement may serve various purposes and applications with
> very different requirements.
> 
> >    network administrators need to adjust some parameters of metering
> >    devices and of multiple measurement applications to these resources.
> 
> I do not understand this semi-phrase. What do you mean by "to these
> resources"?
> 

They point to CPU powers or memories on Exporter/Collecotor.

> >    IP traffic growth and a wide variety of measurement application make
> >    the adjustment further difficult.  To make a measurement system
> 
> Why is it more difficult to adjust parameters in the presence of IP
> traffic growth?
> In the first sentence, you write that Flow-based measurement can monitor
> huge amounts of traffic, so traffic growth should not be a problem.

So, I mean that it is difficult to catch up the performance extension
for Collector/Exporter side according to the increase of IP traffic
growth.
> 
> >    flexible and scalable, an intermediate device can generally be
> >    applied to the system platform.
> 
> Maybe you can discuss and clarify this paragraph with your co-authors.
> I can hardly guess what the reasoning is supposed to be.
> 
Could you please check the following paragraph, once more?

1.  Introduction

   An advantage of flow-based measurement is that it allows monitoring
   large amount of traffic observed at distributed observation points.
   While flow-based measurement can be applied to one of various
   purposes and applications, it is difficult for flow-based measurement
   to apply to multiple applications with very different requirements in
   parallel.  Network administrators need to adjust some parameters of
   metering devices according to the most severe one of requirements of
   multiple measurement applications within measurement system
   performance and capacity.  IP traffic growth consuming furthermore
   measurement system capacity make it further difficult to keep the
   adjustment.  To make a measurement system flexible and scalable, an
   intermediate device can generally be applied to the system.

> > 
> >>>    The IPFIX requirements defined in [RFC3917] mention examples of
> >>>    intermediate devices, such as IPFIX Proxies or Concentrators, there
> >> missing conjunction, such as "but", "yet", or something like that
> >>
> >> The term "intermediate" or "intermediate device" does not appear in 
> >> RFC3917. So, explain why you call these devices like this.
> >> (For example, because they are located between Exporters and Collectors.)
> > 
> > Is it OK?
> > 
> >    The IPFIX requirements defined in [RFC3917] mention examples of
> >    intermediate devices located between Exporters and Collectors, such
> >    as IPFIX proxies or concentrators.  But, there are no documents
> >    defining a generalized concept for such intermediate devices.  This
> 
> ok.
> 
> > 
> >>>    are no documents defining a generalized concept for such intermediate
> >>>    devices.  This document addresses that issue by defining IPFIX
> >>>    Mediation, a generalized intermediate device concept for IPFIX, and
> >>>    examining in detail the motivations behind its application.
> >>>
> >>>    This document is structured as follows: section 2 describes the
> >>>    terminology used in this document, section 3 gives an IPFIX/PSAMP
> >>>    document overview, section 4 introduces general problems related to
> >>>    flow-based measurement, section 5 describes some applicability
> >>>    examples where IPFIX Mediations would be beneficial, and, finally,
> >>>    section 6 describes some problems an IPFIX Mediation implementation
> >>>    might face.
> >>
> >>> 2.  Terminology and Definitions
> >>>
> >>>    The IPFIX-specific and PSAMP-specific terminology used in this
> >>>    document is defined in [RFC5101] and [RFC5476], respectively.  In
> >>>    this document, as in [RFC5101] and [RFC5476], the first letter of
> >>>    each IPFIX-specific and PSAMP-specific term is capitalized along with
> >>>    the IPFIX Mediation-specific term defined here.
> >>>
> >>>    In this document, we use the generic term "record stream" to denote a
> >> I would not call "record stream" a "term" unless it appears in the list 
> >> below with capitalized first letter.
> >>
> >>>    set of flow- or packet-based data records with their additional
> >> I would not say that the records are flow-based or packet-based. They 
> >> contain flow or packet information.
> >>
> > 
> > Actually, I am not sure how to say that.
> > It seems hard to include "record stream" in capitalized terms, because
> > it covers everything, such as IPFIX Flow Record, PSAMP Packet Record, NF
> > flow record, and something else. Could you put your suggestion?
> > 
> > Is the following paragraph fine?
> > 
> >    In this document, we use "record stream" to denote a set of flow- or
> >    packet-based information with their additional information that flows
> >    from data sources, whether encoded in IPFIX protocol as IPFIX Data
> >    Records, or non-IPFIX protocols.  In IPFIX protocol, we use the
> >    generic term Data Records for IPFIX Flow Records, PSAMP Packet
> >    Reports, and Data Records defined by Options Templates, unless an
> >    explicit distinction is required.
> 
> What about:
> 
> In this document, we call "record stream" a stream of records carrying
> flow- or packet-based information. The records may be encoded as IPFIX
> Data Records in any other format.
> 

Ok, Thanks.

> >>>    information that flows from data sources, whether encoded in IPFIX
> >>>    protocol as IPFIX Data Records, or non-IPFIX protocols.  In IPFIX
> >>>    protocol, we use the generic term Data Records for IPFIX Flow
> >>>    Records, PSAMP Packet Reports, and Data Records defined by Options
> >>>    Templates, unless an explicit distinction is required.
> >> Do we need the last sentence?
> > 
> > I think Yes. It indicates the usage of term Data Records to prevent
> > readers from confusing it with IPFIX Flow Record and PSAMP packet report.
> 
> I think the contrary. This sentence may confuse the reader.
> 
> "In IPFIX protocol" is wrong because the "PSAMP Packet Report" does not
> appear in RFC5101.
> Therefore, it seems that you redefine the term Data Record, also this is
> not what you want to do.
> 
> If you really think that the sentence is necessary, maybe you like this:
> "Note that the term Data Records comprises IPFIX Flow Records, PSAMP
> Packet Reports, and Data Records defined by Options Templates."
> 
> BTW: In IPFIX-CONFIG, I also use this term in the same way without
> providing any further explanation.
> 
Ok, I follows your draft way.

> 
> >>>    Original Exporter
> >>>
> >>>       An Original Exporter is an IPFIX Device that hosts the Observation
> >>>       Points where the metered IP packets are observed.
> >>>

snip

> >>>
> >>> 4.4.  Summary
> >>>
> >>>    In optimizing the resources of a measurement system, it is important
> >> I still do not understand what "optimize the resources" is supposed to mean.
> >>
> > 
> > I will change it as follows.
> > 
> > 4.4.  Summary
> > 
> >    In adjusting some parameters to the resources of a measurement
> >    system, it is important to use traffic data reduction techniques as
> >    early as possible, e.g., at the Exporter.  However, this
> 
> I do not understand. Maybe you want to say:
> 
> Due to resource limitations of the measurement system, it is important to...
> 
> >    implementation is made difficult by heterogeneous environment of
> >    exporting devices to meet the requirements of different monitoring
> >    applications.
> > 
> > 
> >>>    to use traffic data reduction techniques as early as possible, e.g.,
> >>>    at the Exporter.  However, this implementation is made difficult by
> >>>    heterogeneous environment of exporting devices.
> >> Please revise the entire paragraph above. It only talks about data 
> >> reduction. I do not think that this is the core of mediation.
> >>
> > No. It imply the distribution of traffic data in heterogeneous
> > enviroment. Please suggest your idea.
> 
> The summary covers 4.1 and 4.3. I miss the problem mentioned in 4.2.
> 

Coukd you please see it.

4.4.  Summary

   Due to resource limitations of the measurement system, it is
   important to use traffic data reduction techniques as early as
   possible, e.g., at the Exporter. (4.1) 
   However, this implementation is made difficult by heterogeneous
   environment of exporting devices.(4.2)  On the other hand, keeping
   data accuracy and flow granularity to meet the requirements of
   different monitoring applications requires a scalable and flexible
   collecting infrastructure.(4.3)

Is it OK?

> > 
> >>>    This implies that a new Mediation function is required in typical
> >>>    Exporter-Collector architectures.  Based on some applicability
> >>>    examples, the next section shows the limitation of the typical
> >>>    Exporter-Collector architecture model and the IPFIX Mediation
> >>>    benefits.
> >>
> >>> 5.  Mediation Applicability Examples
> >>>
> >>> 5.1.  Adjusting Flow Granularity
> >>>
> >>>    A set of common properties of simplest Flow type is a fixed 5-tuple
> >>>    of protocol, source and destination IP addresses, and source and
> >> As you talk about "Flow Keys", you should use this term and not invent a 
> >> new one!
> > 
> > Actually, when I started to write this subsection, it implied covering
> > IPFIX and NetFlow.
> > 
> > I will change subsection as follows.
> > 
> > 5.1.  Adjusting Flow Granularity
> > 
> >    A simplest set Flow Keys is a fixed 5-tuple of protocol, source and
> >    destination IP addresses, and source and destination port numbers.  A
> >    shorter set of Flow Keys, such as a triple, a double, or a single
> >    property, (for example network prefix, peering autonomous system
> >    number, or BGP Next-Hop fields), creates more aggregated Flow
> >    Records.  This is especially useful for measuring router-level
> >    traffic matrices in a core network domain and for easily adjusting
> >    the performance of Exporters and Collectors.
> > 
> >    Implementation analysis:
> > 
> >       Implementations for this case depend on where Flow granularity is
> >       adjusted.  More suitable implementations use configurable Metering
> >       Processes in Original Exporters.  The cache in the Metering
> >       Process can specify its own set of Flow Keys and extra fields.
> >       The Original Exporter thus generates Flow Records of the desired
> >       Flow granularity.
> > 
> >       In the case where a Metering Process hosting no ability to change
> >       the Flow Keys in Original Exporter creates Flow Records, or PSAMP
> >       Packet Reports, an IPFIX Mediator can aggregate Data Records based
> >       on a new set of Flow Keys.  Even in the case where the Metering
> 
> Even in the case of a Metering Process...
> 
Ok. Thanks.

> >       Process hosting this ability, an IPFIX Mediator can further
> >       aggregate the Flow Records.
> 
> fine.
> 

snip

> >>>
> >>> 5.2.  Hierarchical Collecting Infrastructure
> >>>
> >>>    The increase of IPFIX Exporters, the increase of the traffic, and the
> 
> increasing number of exporters?
> 
Yes.

Increasing number of IPFIX Exporters, amount of IP traffic, and variety
of treatments expected to be performed on the Data Records is more and
more difficult to handle within a single Collector.

> >>>    variety of treatments expected to be performed over the Data Records
> >> over => on
> >>
> > Ok.
> > 
> >>>    is more and more difficult to handle within a single Collector.
> >>>    Hence to increase the collecting (e.g., the bandwidth capacity) and
> >>>    processing capacity, distributed Collectors must be deployed.  As a
> >>>    possible approach, a hierarchical structure is useful for increasing
> >> I don't understand how the collecting and processing capacity increases 
> >> thanks to a hierarchical structure.
> >>
> >> The capacity increases because I implement more resources in the 
> >> network, e.g. more Collectors.
> > 
> > Although increasing number of Collectors operated independently makes it
> > more capacity, we cannot measure the all of amount of traffic data.
> 
> Still not sure what you mean.
> The original exporter measures traffic, not the collector.
> If you cannot measure all the traffic, you need more powerful routers.

Actually, if there is the capacity limitation in Exporter, as you say,
powerful exporter would be necessary. It is so apparent to go without
saying.

> 
> IMO, the hierarchy is useful to gather data which has been collected at
> different places in the network. But the hierarchy is not needed to
> deploy distributed collectors.
> 

Ok, I will change the section title as "Collecting Infrastructure".
And, I will change whole subsection as follows.

5.2.  Collecting Infrastructure

   Increasing number of IPFIX Exporters, amount of IP traffic, and the
   variety of treatments expected to be performed on the Data Records is
   more and more difficult to handle within a single Collector.

   Implementation analysis:

      To increase the collecting (e.g., the bandwidth capacity) and
      processing capacity, distributed Collectors close to Exporters
      need to be deployed.  In such a case, those Collectors would
      become IPFIX Mediators, re-exporting Data Records on demand to
      centralized applications.  To cope with the variety of treatments,
      one possible implementation uses an Intermediate Process deciding
      to which Collector(s) to export each record.  More specific cases
      are described in section 5.9.

> >>>    the measurement systems capacity, both in export bandwidth capacity
> >>>    and in collecting capacity.
> >>>
> >>>    Implementation analysis:
> >>>
> >>>       To cope with the increase of IPFIX Exporters and traffic, one
> >> "number of IPFIX Exporters" (only IPFIX?)
> > Ok.
> > 
> >>>       possible implementation uses IPFIX Concentrators to build a
> >>>       hierarchical collection system.  To cope with the variety of
> >>>       treatments, one possible implementation uses IPFIX Distributors to
> >>>       build a distributed collection system.  More specific cases are
> >>>       described in section 5.9.
> >>
snip

> >>>
> >>> 5.6.  Data Record Anonymization
> >>>
> >>>    IPFIX exports across administrative domains can be used to measure
> >>>    traffic for wide-area traffic engineering or to analyze Internet
> >>>    traffic trends, as described in the spatial composition across
> >>>    administrative domains in the previous subsection.
> >>>    In such a case, administrators need to adhere to privacy protection
> >>>    policies and prevent access to confidential traffic measurements by
> >>>    other people.  Typically, anonymization techniques enables the
> >> enable
> >>
> > Yes.
> > 
> >>>    provision of traffic data to other people without violating these
> >>>    policies.
> >>>
> >>>    Generally, anonymization modifies a data set to protect the identity
> >>>    of the people or entities described by the data set from being
> >>>    disclosed.  It also attempts to preserve sets of network traffic
> >>>    properties useful for a given analysis while ensuring the data cannot
> >>>    be traced back to the specific networks, hosts, or users generating
> >>>    the traffic.  For example, IP address anonymization is particularly
> >>>    important for avoiding the identification of the users, hosts, and
> >> remove second "the"
> >>
> > Yes.
> > 
> >>>    routers.  As another example, when ISP provides a traffic monitoring
> >> an ISP
> >>
> > Yes.
> > 
> >>>    service to end customers by their own Exporters, even in case of
> >>>    exporting interface index fields, network administrators take care of
> >>>    anonymizing its fields to avoid disclosing the vulnerability.
> >> Why does an interface represent a vulnerability?
> >>
> > 
> > Interface indexes assignment policies in several vendors are different.
> > Some vender indexes show upper bytes indicating line card, lower byte
> > indicating interfaces.
> > By recognizing its policy, we can presume the vendors of Exporter.
> 
> Ok. But I would not say that disclosing an interface index automatically
> discloses a vulnerability.
> 
> "vulnerability" => "information"

Ok. I will change as follows.

As another example, when an ISP provides a traffic monitoring service to
end customers by their own Exporters, even in case of exporting
interface index fields, network administrators take care of anonymizing
its fields to avoid from identifying vendor or software version by
disclosing the information.

> 
> > 
> >>>    Implementation analysis:
> >>>
> >>>       One possible implementation for this case uses an anonymization
> >>>       function at the Original Exporter.  However, this increases the
> >>>       load on the Original Exporter.  A more flexible implementation
> >>>       uses a separate IPFIX Masquerading Proxy between the Original
> >>>       Exporter and Collector.
> >>
> >>
> >>

snip

> >>>
> >>> 7.  Summary and Conclusion
> >>>
> >>>    This document described the problems that network administrators have
> >>>    been facing, the applicability of IPFIX Mediation to these problems,
> >>>    and the problems related to the implementation of IPFIX Mediators.
> >>>    To assist the operations of the Exporters and Collectors, there are
> >>>    various IPFIX Mediations from which the administrators may select.
> >>>    Examples of the applicability of IPFIX Mediation are as follows.
> >>>
> >>>    o  Regarding large-scale measurement system, IPFIX Concentrators or
> >>>       IPFIX Distributors help to achieve traffic analysis with high data
> >>>       accuracy and fine flow granularity even as IP traffic grows.  As
> >>>       IPFIX Mediation capabilities, Flow sampling, aggregation, and
> >>>       composition are effective.
> >> Sampling and aggregation reduce the accuracy or granularity.
> >> Correlation seems to be appropriate.
> >>
> > 
> > I means as follows.
> > 
> >    o  Regarding large-scale measurement system, IPFIX Mediator with
> >       IPFIX File Writer helps to achieve traffic analysis with high data
> >       accuracy and fine flow granularity even as IP traffic grows.  As
> >       IPFIX Mediation capabilities, Flow sampling, aggregation, and
> >       composition are effective.
> 
> Flow sampling => Flow selection ?
> 
> > 
> >>>    o  Regarding data retention, IPFIX Mediators enhance the export
> >>>       reliability, and the storage of the measurement system.
> >>>
> >>>    o  Regarding the distribution of Data Records, IPFIX Distributors
> >>>       help to achieve multipurpose traffic analysis for different
> >>>       organizations, or help to achieve respective traffic analysis
> >> remove "respective"?
> >>
> > Ok.
> > 
> >>>       based on Data Record types(IPv4, IPv6, MPLS, and VPN).
> >>>
> >>>    o  Regarding the IPFIX export across domains, IPFIX Masquerading
> >>>       Proxies help administrators to anonymize or filter Data Records,
> >>>       preventing privacy violations.
> >>>
> >>>    o  Regarding interoperability, IPFIX Proxies provide interoperability
> >>>       between legacy protocols and IPFIX, even during the migration
> >> even => for example
> > Ok.
> > 
> >>>       period to IPFIX.
> >>>
> >>>    As a result, the IPFIX Mediation benefits become apparent.  However,
> >>>    there are still some open issues with the use of IPFIX Mediators.
> >>>
> >>>    o  Both Observation Point and IPFIX Message header information, such
> >>>       as the Exporter IP address, Observation Domain ID, and Export Time
> >>>       field, might be lost.  This data should therefore be communicated
> >>>       between the Original Exporter and Collector via the IPFIX
> >>>       Mediator.
> >>>
> >>>    o  IPFIX Mediators are required to manage Transport Sessions,
> >>>       Template IDs, and Observation Domain IDs.  Otherwise, anomalous
> >>>       IPFIX Messages could be created.
> >>>
> >>>    o  Data Records defined by Options Templates, such as those reporting
> >>>       the Sampling rate and Sampling algorithm used, might be lost
> >>>       during IPFIX Mediation.  If a Collector is not informed of current
> >>>       Sampling rates, traffic information might become worthless.
> >>>
> >>>    These problems stem from the fact that no standards regarding IPFIX
> >>>    Mediation have been set.  In particular, the minimum set of
> >>>    information that should be communicated between Original Exporters
> >>>    and Collectors, the management between different IPFIX Transport
> >>>    Sessions, and the internal components of IPFIX Mediators should be
> >>>    standardized.
> >>
> >> There is a lot of repetition in this section.
> >>
> > 
> > You means this section should be removed.
> 
> If it just repeats what has already been set, then probably yes.
> 

I will furthermore summarise the section as follows.

7.  Summary and Conclusion

   This document described the problems that network administrators have
   been facing, the applicability of IPFIX Mediation to these problems,
   and the problems related to the implementation of IPFIX Mediators.
   To assist the operations of the Exporters and Collectors, this
   document demonstrates there are various IPFIX Mediations from which
   the administrators may select.

   However, there are still some open issues with the use of IPFIX
   Mediators.  These issues stem from the fact that no standards
   regarding IPFIX Mediation have been set.  In particular, the minimum
   set of information that should be communicated between Original
   Exporters and Collectors, the management between different IPFIX
   Transport Sessions, and the internal components of IPFIX Mediators
   should be standardized.

Regards,
Atsushi

---

From: Gerhard Muenz <[email-masked]>
Date: Thu, 03 Dec 2009 10:09:38 +0100

Hi Atsushi,

comments and improvements inline.

Regards,
Gerhard

 >    intermediate device can generally be applied to the system.

- What about the following:

    Network administrators need to adjust the parameters of the metering
    devices to fulfill the requirements of every single measurement
    application.  Such configurations are often not supported by the
    metering devices, either because of functional restrictions or
    because of limited computational and memory resources, which inhibit
    the metering of large amounts of traffic with the desired setup.
    IPFIX Mediation fills the gap between restricted metering
    capabilities and the requirements of measurement applications by
    introducing an intermediate device called IPFIX Mediator.

>    collecting infrastructure.(4.3)

- Have you switched 4.2 and 4.3?
- I would remove (4.*) in the text above.

>    more and more difficult to handle within a single Collector.

Increasing numbers of IPFIX Exporters, IP traffic growth, and the
variety of treatments expected to be performed on the Data Records make 
it more and more difficult to implement all measurement applications 
within a single Collector.

>       centralized applications.  To cope with the variety of treatments,

measurement applications

>       to which Collector(s) to export each record.  More specific cases

...deciding to which Collectors each record is exported.

> disclosing the information.

As another example, when an ISP provides traffic monitoring service to
end customers, network administrators take care of anonymizing
interface index fields which could disclose any information about the 
vendor or software version of the Exporters.

>    document demonstrates there are various IPFIX Mediations from which

...demonstrates that there exist various IPFIX Mediation functions from 
which the administrators may select.

-- 
Dipl.-Ing. Gerhard Münz
Chair for Network Architectures and Services (I8)
Technische Universität München - Department of Informatics
Boltzmannstr. 3, 85748 Garching bei München, Germany
Phone:  +49 89 289-18008       Fax: +49 89 289-18033
E-mail: [email-masked]    WWW: http://www.net.in.tum.de/~muenz

---

From: Atsushi Kobayashi <[email-masked]>
Date: Thu, 03 Dec 2009 19:38:23 +0900

Hi Gerhard,

Thank you for your contribution.
Please see inline.

On Thu, 03 Dec 2009 10:09:38 +0100

snip

>     introducing an intermediate device called IPFIX Mediator.

Fine with me. 

snip

> - Have you switched 4.2 and 4.3?

yes. It's my mistake.

> - I would remove (4.*) in the text above.
Yes.  I would like to remove them before submitting it.

snip

> within a single Collector.

Fine with me.

Yes. 

snip

> vendor or software version of the Exporters.
Fine.

> which the administrators may select.

Thanks.

Regards,
Atsushi

--- 
Atsushi KOBAYASHI  <[email-masked]>
NTT Information Sharing Platform Lab.
tel:+81-(0)422-59-3978 fax:+81-(0)422-59-5637