Subject: [ipfix-info] Proposal for IFPIX observations at middleboxes

From: Sven Anderson <[email-masked]>
Date: Thu, 04 Aug 2005 19:05:42 +0200

Hi all,

I will try to explain again my idea for reporting flows from middleboxes
as clear and short(?) as possible:

When IP packets travel through a middlebox, their properties may change.
Examples would be TTL, IP adresses (NAT), DSCP or even fragmentation or
replication (multicasting). That means, that if an Observation Domain
includes several Observation Points, and the same packet passes more than
one of these Observation Points, some properties can be ambiguous in this
Observation Domain.

You could solve this by saying, that every propery of a flow-report
has to derive from the same observation point. But this would be
restrictive. Sometimes it is desirable, that you can classify a single
Flow by properties, that derive from _different_ Observation Points. For
example a flow-definition that includes the source IP address before and
after a NAT, or the DSCP at three different Observation Points (before
ingress linecard, after ingress linecard, after egress linecard).

To realise this, you need a way to use certain properties more than one
time in a flow-template, to correspond to the different states at the
different Observation Points. One way to do this, is the introduction of
additional post- I.E.s, which then correspond to the first and the last
Observation Point in the Observation Domain. But this is a restriction to
two special Observation Points, and the second example from above is not
solvable with this approach.

A more general solution would be to allow the use of the same I.E. more
than one time in the same template. The order of the multiple I.E.s
corresponds to the different observations as the packet travels through
the middlebox. The problem here is, that you cannot tell then, to which of
the different observations the _singular_ I.E.s belong to.

To solve this, you need a possibility to build goups of I.E.s in the
Template: The I.E. values in one "Observation Group" always derive from
the same Observation Point (for each packet!). The Observation Groups are
ordered accordingly to the sequence of Observation Points the packet is
passing. Of course there are also fields which don't depend on the
Observation Point and can be reported in any Observation Group, like
ingressInterface (not egress!), as it is always the same, no matter where
in the middlebox the packet is observed. (You may say, that these fields
_have_ to be reported in the first group then, if this is an advantage.)

Important to note is, that packets of a Flow defined by (for example) two
Observation Groups don't necessarily pass the same sequence of two
Observation Points. They just share the same Flow poperties at the their
first and second Observation Points respectively. To make sure, that all
packets passed the same sequence of Observation Points you have to include
an (yet to be defined) I.E. "observationPointID" as a Flow Key in each
Observation Group.

BTW.: It's possible that a Flow has the same observationPointID in
different Observation Groups. For example if your Flow contains two
Observation Groups, corresponding to Observation Points at the ingress and
egress interface, you could have Flows where ingress and egress interface
and therefore the observationPointID is the same (e.g. after some NAT).

The next question is: what is a good way to define these groups? Here are
just two ideas:

- my first idea, which I descibed in an former mail, was to define
"Combined Templates", which are a ordered list of other Templates. Each
Template in the Combined Template represents an Observation Group. The
reports for Combined Templates are just normal reports with all the Fields
of all the listed Templates concatenated. The disadvantage is, that a
change of IPFIX-PROTO is necessary.

- another idea is to define a special separator I.E. with length 0, like
"newObservationGroup". This pseudo-field separates the Fields in the
Template into different Observation Groups. In the reports they don't
appear, but the collector has the template and knows where to split. Here
no change in IPFIX-PROTO is necessary. (Jeroen Massar had a very similar idea)

I think this concept is a superset of the proposals of Jürgen and Benoit.
Instead of using post-I.E.s Benoit could use Templates like this (using
the second grouping-mechanism):

<sourceIPv4Address>
[...different Fields...]
<octetDeltaCount>
<DSCP>
*<newObservationGroup>*
<DSCP>

The DSCP field in the second ObservationGroup is the same as his postDSCP
field.

Jürgen would do it maybe like this:

<destinationIPv4Address>
*<newObservationGroup>*
<sourceIPv4Address>
<destinationIPv4Address>
[...different Fields...]
<octetDeltaCount>

instead of using an preDestinationIPv4Address Field.

Why do I like this concept:

- it includes all the possibilities of the other solutions.

- you can have as many Observation Groups as you want. (I have an
VPN-Relay here, where I will need 3 Observation Groups, which is not
possible with the pre-/post- solutions.)

- it is always clear, which fields derive from the same Observation Points
(same packet state). This is especially true for the counter Fields! You
can have even the same counter in different Observation Groups, if you
expect the packet size to be changed.

- you don't need any post-/pre- variants of the I.E.s You just need the
newObservationGroup I.E.. The observationPointID is a good idea anyway, I
think.

- you don't need complicated semantics, you just report an ordered
sequence of packet states. (And that's all you know, in fact.)

I'm not really sure, but I think, that the in- and out- counters also get
obsolete then. Wouldn't it be the same as having a counter in the first
and last Observation Group?

A side note: I think a packet-altering instance - like a NAT process -
which is reporting information to the metering process, should always be
seen as _two_ observation points, one before and one after the change.

Anyway, this is my "work in progress" idea. I hope I could present this
quite complex object in an understandable manner. Now tell me, why it is a
bad idea. :-)

Cheers,

Sven

-- 
Sven Anderson
Institute for Informatics - http://www.ifi.informatik.uni-goettingen.de
Georg-August-Universitaet Goettingen
Lotzestr. 16-18, 37083 Goettingen, Germany

--
Help        mailto:[email-masked] and say "help" in message body
Unsubscribe mailto:[email-masked] and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/

---

From: Juergen Quittek <[email-masked]>
Date: Wed, 31 Aug 2005 09:26:46 +0200

Hi Sven,

As said before, your proposal of introducing 'observation groups' into the
IPFIX protocol came at a bad time just after we agreed on a solution on the
middlebox problem at the IPFIX session at IETF 63.  At the session we also
discussed the proposal you made before the session.

Anyway, in the past we have been discussing means for structuring IPFIX
records several times already.

The general feeling was that we should use such means carefully only if really
necessary, because it complicates the protocol and defining a clear semantics
that is unambiguous in all complex cases is not that easy.

Also we only added structuring, such a scopes and sequential ordering, only
when strong use cases were present that could not be covered with more simple
solutions.  since we agreed a rather simple solution for the middlebox problem
at the IETF meeting, I do not think we should re-enter this discussion with a
more complex (though mightier) solution.

Thanks,

    Juergen
-- 
Juergen Quittek        [email-masked]       Tel: +49 6221 90511-15
NEC Europe Ltd.,       Network Laboratories        Fax: +49 6221 90511-55
Kurfuersten-Anlage 36, 69115 Heidelberg, Germany   http://www.netlab.nec.de