Subject: Re: [IPFIX] WG last call on draft-ietf-ipfix-mediators-problem-statement-03

From: Benoit Claise <[email-masked]>
Date: Tue, 28 Jul 2009 10:37:02 +0200

Gerhard,

Thinking some more, I would like to come back to that point below.

>
>>
>> 6.8.  Consideration for Aggregation
>>
>>    In case of Flow Key aggregation, Time Composition, and Spatial
>>    Composition, there are the following considerations:
>>
>>    o  Aggregation rule for non Flow Keys
>>     
>
> "non Flow Keys" => "non-key fields"
>
>   
>>       There are no obvious rules of non Flow Keys.  For example, if an
>>     
>
> "non Flow Keys" => "non-key fields"
>
>   
>>       IPFIX Mediation receives two Flow Records with different DSCP
>>       values, and this DSCP field is not a Flow Key, those two Flow
>>       Records can be aggregated based on the Flow Keys value.  However,
>>       there is no rules for what the DSCP value should be for the
>>       aggregated Data Record.  Potential solutions are: the value of
>>     
>
> Actually, there is a rule!
> Unless specified differently, the value of the first packet must be
> included in the record. Hence, as long as timestamps are present and
> allow to determine which one of the Flow Records includes the earliest
> packet, the decision is evident!
>   
I agree with this rule on Exporter.
However, for a Mediator, it's not obvious that we should follow it.
Anyway, it should be specified in the mediation framework or protocol draft.
In the mean time, I would like to keep this issue in the problem statement.

Regards, Benoit.

>   
>>       single of the two DSCP, the value 0 (in this case, the value 0 is
>>       a valid DSCP value), or removing a DSCP field in its Data Record.
>>
>>    o  Configured Selection Fraction on aggregation
>>
>>       There is no obvious rules of how to compute Configured Selection
>>       Fraction, and whether a Mediator should report Configured
>>     
>
> remove ", and whether....,"
> It is obvious, that Configured or Attained Selection Fraction is
> necessary to interpret the Flow Records.
>
>
>   
>>       Selection Fraction, when aggregation resulting from Sampling.  For
>>     
>
> "...when aggregating IPFIX Flow Records of sampled packets."
>
>   
>>       example, special care must be taken in the following: aggregation
>>       resulting from the different Configured Selection Fraction,
>>       aggregation resulting from different Sampling techniques, such as
>>       Systematic Count-Based Sampling and Random n-out-of-N Sampling
>>       etc.
>>     
>
>
>   
>> 7.  Summary and Conclusion
>>
>>    This document described the problems that network administrators have
>>    been facing, the applicability of IPFIX Mediation to these problems,
>>    and the problems related to the implementation of IPFIX Mediators.
>>    To assist the operations of the Exporters and Collectors, there are
>>    various IPFIX Mediations from which the administrators may select.
>>    Examples of the applicability of IPFIX Mediation are as follows.
>>
>>    o  Regarding large-scale measurement system, IPFIX Concentrators or
>>       IPFIX Distributors help to achieve traffic analysis with high data
>>       accuracy and fine flow granularity even as IP traffic grows.  As
>>       IPFIX Mediation capabilities, Flow selection Sampling,
>>     
>
> flow sampling?
>
>   
>>       aggregation, and composition are effective.
>>
>>    o  Regarding data retention, IPFIX Mediators enhance the export
>>       reliability, and the storage of the measurement system.
>>
>>    o  Regarding the distribution of Data Records, IPFIX Distributors
>>       help to achieve multipurpose traffic analysis for different
>>       organizations, or help to achieve respective traffic analysis
>>       based on Data Record types(IPv4, IPv6, MPLS, and VPN).
>>
>>    o  Regarding IPFIX Exporting across domains, IPFIX Masquerading
>>     
>
> exporting
>
>   
>>       Proxies help administrators to anonymize or filter Flow Records/
>>     
>
> Data Records
>
>   
>>       Packet Reports, preventing privacy violations.
>>
>>    o  Regarding interoperability, IPFIX Proxies provide interoperability
>>       between legacy protocols and IPFIX, even during the migration
>>       period to IPFIX.
>>
>>    As a result, the IPFIX Mediation benefits become apparent.  However,
>>    there are still some open issues with the use of IPFIX Mediators.
>>
>>    o  Both Observation Point and IPFIX Message header information, such
>>       as the Exporter IP address, Observation Domain ID, and Export Time
>>       field, might be lost.  This data should therefore be communicated
>>       between the Original Exporter and Collector via the IPFIX
>>       Mediator.
>>
>>    o  IPFIX Mediators are required to manage Transport Sessions,
>>       Template IDs, and Observation Domain IDs.  Otherwise, anomalous
>>       IPFIX messages could be created.
>>
>>    o  Data advertised by Option Templates from the Original Exporter,
>>     
>
> _Options_ Templates
>
> better:
> "Options Data Records exported by the Original Exporter, such as those
> reporting the Sampling rate...."
>
> Maybe the term "Options Data Record" should be introduced to simplify
> things. BTW, we had the same problem in per-sctp-stream, and always
> wrote "Data Record defined by an Options Template Record", which is not
> very easy to read.
>
>   
>>       such as the Sampling rate and Sampling algorithm used, might be
>>       lost.  If a Collector is not informed of current Sampling rates,
>>       traffic information might become worthless.
>>
>>    These problems stem from the fact that no standards regarding IPFIX
>>    Mediation have been set.  In particular, the minimum set of
>>    information that should be communicated between Original Exporters
>>    and Collectors, the management between different IPFIX Transport
>>    Sessions, and the internal components of IPFIX Mediators should be
>>    standardized.
>>     
>
>
>   
>> 8.  Security Considerations
>>
>>    A flow-based measurement system must prevent potential security
>>    threats: the disclosure of confidential traffic data, injection of
>>    incorrect data, and unauthorized access to traffic data.  These
>>    security threats of the IPFIX protocol are covered by the security
>>    considerations section in [RFC5101] and are still valid for IPFIX
>>    Mediators.
>>
>>    And a measurement system must also prevent following security threats
>>     
>
> "...the following..."
>
>   
>>    related to IPFIX Mediation:
>>
>>    o  Attacks against IPFIX Mediator
>>
>>       IPFIX Mediators can be considered as a prime target for attacks,
>>       as an alternative to IPFIX Exporters and Collectors.  IPFIX
>>       Proxies or Masquerading Proxies need to prevent unauthorized
>>       access or denial-of-service (DoS) attacks from untrusted public
>>       networks.
>>
>>    o  Man-in-the-middle attack by untrusted IPFIX Mediator
>>
>>       The Collector-Mediator-Exporter structure model would increase the
>>       risk of the man-in-the-middle attack.
>>
>>    o  Configuration on IPFIX Mediation
>>
>>       In the case of IPFIX Distributors and IPFIX Masquerading Proxies,
>>       an accidental misconfiguration and unauthorized access to
>>       configuration data could lead to the crucial problem of disclosure
>>       of confidential traffic data.
>>     
>
>
>   
> ------------------------------------------------------------------------
>
> _______________________________________________
> IPFIX mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ipfix
>

---

From: Gerhard Muenz <[email-masked]>
Date: Tue, 28 Jul 2009 11:22:13 +0200

Benoit,

> In the mean time, I would like to keep this issue in the problem statement.

RFC5102:

   If they do not serve as Flow Keys, their value may change from packet
   to packet within a single Flow.  For Information Elements with values
   that are derived from fields of packets or from packet treatment and
   for which the value may change from packet to packet within a single
   Flow, the IPFIX information model defines that their value is
   determined *by the first packet* observed for the corresponding Flow,
   unless the description of the Information Element explicitly
   specifies a different semantics.

So, if a Collector receives a Flow Record with non-key field
ipDiffServCodePoint, it can assume that this is the DSCP value of the
first packet of all packets in the Flow.

Are you suggesting that RFC5102 is not valid for Mediator?
I'm not sure if this is useful.

If you want to keep this issue in the problem statement draft, the text
should be changed. It should be clarified that RFC5102 specifies that a
non-key field value is taken from the first packet. The issue I see is
that a Mediator might have problems to determine which one of the
original Flow Records contains the first packet. For example, the flow
start timestamp might be missing or have equal values for multiple
original Flow Records. If the router clocks are not very well
synchronized, the timestamps may be biased.

Regards,
Gerhard

-- 
Dipl.-Ing. Gerhard Münz
Chair for Network Architectures and Services (I8)
Department of Informatics
Technische Universität München
Boltzmannstr. 3, 85748 Garching bei München, Germany
Phone:  +49 89 289-18008       Fax: +49 89 289-18033
E-mail: [email-masked]    WWW: http://www.net.in.tum.de/~muenz

---

From: Benoit Claise <[email-masked]>
Date: Tue, 28 Jul 2009 11:29:24 +0200

Gerhard,
> Are you suggesting that RFC5102 is not valid for Mediator?
No. This rule is still fine.
However, in case of aggregation (this is the title for section 6.8 
Consideration for Aggregation), the question is: what should be Mediator do?
If the Mediator receives two Flow Records from different Exporters:
    one with DSCP=1 (non-key field)
    one with DCSP=2 (non-key field)
What to put in the aggregated Flow Record is the question? And it's not 
obvious to me that it should be the one of the Flow Record that started 
the first...
That's all I want to say.
> synchronized, the timestamps may be biased.
Two good reasons why the rule you mentioned can't be applied as it is, 
and why it's a real problem to be described in the problem statement.

Regards, Benoit.

---

From: Benoit Claise <[email-masked]>
Date: Thu, 30 Jul 2009 01:45:00 +0200

Gerhard,

I proposed this text.

Whether the aggregation is based on time or spatial composition, caution 
should be taken on how to aggregate non Flow Keys in a IPFIX Mediation.

The IPFIX Information Model [RFC5102] is very clear about the content of 
a non Flow Key Information Element on the Original Exporter: its value 
is determined *by the first packet* observed for the corresponding Flow, 
unless the description of the Information Element explicitly specifies a 
different semantics.

However, this rule might not apply in a Mediation Function. For example, 
if an IPFIX Mediator receives two Flow Records with identical Flow Keys 
values but with different DSCP values (the DSCP being a non Flow Key), 
what should be the DSCP value of the aggregated Flow Record? Potential 
solutions are: assign the value of the first received Flow Record within 
the aggregation time, assign a special value such as 0 (However, in this 
case, the value 0 is a valid DSCP value), remove the DSCP Information 
Element from the Data Record, change the DSCP Information Element from a 
non Flow Key to a Flow Key when re-exporting the aggregated Flow Record, 
etc...

Furthermore, rules must be specify on how to aggregate the new 
Configured Selection Fraction an IPFIX Mediator should report when 
aggregating IPFIX Flow Records with different sampling rates. Finally, 
special care must be taken when aggregating Flow Records resulting from 
different Sampling techniques such as Systematic Count-Based Sampling 
and Random n-out-of-N Sampling for example.

Feedback?

Regards, Benoit.

> What to put in the aggregated Flow Record is the question? And it's 
> not obvious to me that it should be the one of the Flow Record that 
> started the first...

---

From: Gerhard Muenz <[email-masked]>
Date: Thu, 30 Jul 2009 10:13:41 +0200

> should be taken on how to aggregate non Flow Keys in a IPFIX Mediation.

non Flow Keys => non-key fields
I was told to do so in ipfix configuration (by Paul, I think).

either
in a IPFIX Mediation => in IPFIX Mediation
or
in a IPFIX Mediation => in an IPFIX Mediator

> The IPFIX Information Model [RFC5102] is very clear about the content of

not sure if "information model" should be capitalized

> unless the description of the Information Element explicitly specifies a

RFC5102 does not talk about Original Exporter yet.

What about:
"The IPFIX information model [RFC5102] specifies that the value of
non-key fields, which are derived from fields of packets or from packet
treatment and for which the value may change from packet to packet
within a single Flow, is determined by the first packet observed for the
corresponding Flow, unless the description of the Information Element
explicitly specifies a different semantics."

> non Flow Key to a Flow Key when re-exporting the aggregated Flow Record,

What about:
"However, this simple rule might not be appropriate when aggregating
Flow Records which have different values in a non-key field. For
example, if two Flows with identical Flow Key values are measured at
different Observation Points, they may contain identical packets
observed at different locations in the network and at different points
in time. On their way from the first to the second Observation Point,
some of the packet fields, such as the DSCP, may have changed. Hence, if
the Information Element ipDiffServCodePoint is included as a non-key
field, it can be useful to include the DSCP value observed at either the
first or the second Observation Point in the resulting Flow Record,
depending on the application."

I'm sure that there are more examples. This is just one where it seems
to be obvious to me that just looking at "the first packet" with respect
to time does not make sense.

Regards,
Gerhard

Technische Universität München - Department of Informatics

---

From: Benoit Claise <[email-masked]>
Date: Thu, 30 Jul 2009 11:45:31 +0200

Gerhard,

Thanks. Kobayashi-san and I came with the following proposal, strongly 
based on your proposal.

Whether the aggregation is based on time or spatial composition, caution 
should be taken on how to aggregate non-key fields in IPFIX Mediation. 
The IPFIX information model [RFC5102] specifies that the value of 
non-key fields, which are derived from fields of packets or from packet 
treatment and for which the value may change from packet to packet 
within a single Flow, is determined by the first packet observed for the 
corresponding Flow, unless the description of the Information Element 
explicitly specifies a different semantics.

However, this simple rule might not be appropriate when aggregating Flow 
Records which have different values in a non-key field. For
example, if two Flows with identical Flow Key values are measured at 
different Observation Points, they may contain identical packets 
observed at different locations in the network and at different points 
in time. On their way from the first to the second Observation Point, 
some of the packet fields, such as the DSCP, may have changed. Hence, if 
the Information Element ipDiffServCodePoint is included as a non-key 
field, it can be useful to include the DSCP value observed at either the 
first or the second Observation Point in the resulting Flow Record, 
depending on the application.

Other potential solutions include: removing the Information Element 
ipDiffServCodePoint from the Data Record
when re-exporting the aggregate Flow Record, changing the Information 
Element ipDiffServCodePoint from a non key-field to a Flow Key when 
re-exporting the aggregated Flow Record, or assigning a non valid value 
for the Information Element to express to the Collector that this 
Information Element is meaningless.

Fine?

Regards, Benoit.

---

From: Gerhard Muenz <[email-masked]>
Date: Thu, 30 Jul 2009 12:32:01 +0200

Benoit,

I do not like "assigning a non valid value for the Information Element
to express to the Collector that this Information Element is
meaningless" - this should only be considered as a work-around for
devices which are not able to handle Templates appropriately.
But if you want to keep this option, leave it in.

Gerhard

---

From: Benoit Claise <[email-masked]>
Date: Thu, 30 Jul 2009 14:43:06 +0200

Gerhard,
> But if you want to keep this option, leave it in.
I don't like this solution, and I don't propose it.
However, the problem statement document is present:
    1. to state the problem
    2. to justify why we need to work on a framework/protocol...
        which will have to chose the right solution

Regards, Benoit.