Subject: Re: [ipfix-protocol] Observation Domain ID uniqueness

From: Benoit Claise <[email-masked]>
Date: Thu, 01 Jun 2006 16:23:45 +0200

Hi,

Sorry to reply to myself, but I think this important issue should be 
corrected in the protocol draft.
So a simple acknowledgment from the mailing list would be sufficient.
Note: I've been asked by our AD to produce a new version of the protocol 
draft these days so that it could go on the agenda of the 6/8 IESG call.

Regards, Benoit.
> Dear all,
>
> I know it was addressed a couple of times on the mailing list. 
> However, while discussing with Lutz Mark and Paul Aitken, I think we 
> discovered the issue
>
> the definition of the observation domain id differs between
> ipfix-arch and ipfix-proto.
>
> -in ipfix-arch the id is unique within the ipfix device
> -in ipfix-proto the id is unique to the exporting process
>
> ------------------------------------------------------------
>
> [ARCH]
> 5.4.  Observation Domain
>
>   The Observation Domain is a logical block that presents a single
>   identity for a group of Observation Points within an IPFIX Device.
>   Each {Observation Point, Metering Process} pair belongs to a single
>   Observation Domain.  An IPFIX Device could have multiple Observation
>   Domains, each of which has a subset of the total set of Observation
>   Points in it.  Each Observation Domain must carry a unique ID within
>   the context of an IPFIX Device.  Note that in case of multiple
>   Observation Domains, a unique ID per Observation Domain must be
>   transmitted as a parameter to the Exporting Function.  That unique ID
>   is referred to as the IPFIX Observation Domain ID.
>
>
> [PROTO]
> 3.3
>   Observation Domain ID
>           A 32-bit identifier of the Observation Domain that is
>           locally unique to the Exporting Process.  The Exporting
>           Process uses the Observation Domain ID to uniquely identify
>           to the Collecting Process the Observation Domain that
>           metered the Flows.  Collecting Processes SHOULD use the
>           combination of the Exporter (exporterIPv4Address,
>           exporterIPv6Address, or exportingProcessId) and the
>           Observation Domain ID field to separate different export
>           streams originating from the same Exporting Process.  The
>           Observation Domain ID SHOULD be 0 when no specific
>           Observation Domain ID is relevant for the entire IPFIX
>           Message.  For example, when exporting the Exporting Process
>           Statistics, or in case of hierarchy of Collector when
>           aggregated data records are exported.
>
>
> We think that the Observation Domain Id should be unique per IPFIX 
> Device, and not per Exporting Process.
> Otherwise, we could end up in the following situation: one sysadmin 
> configures a set of observation points with observation domain 1, 
> while a second sysadmin configures another set of observation points 
> (on a different line card for example) with the same observation 
> domain Id.
> If each observation domain exports using its own exporting process 
> with the source IP address, how would the collector differentiate the 
> template IDs?
>
> Also, the terminology sections of IFPIX-PROTO and IPFIX-ARCH are not 
> in sync.
> Multiple small changes (including in the terminology sections) should 
> be inserted. For example: IPFIX-PROTO observation domain definition 
> says "In the IPFIX Message it generates, the Observation Domain 
> includes its Observation Domain ID, which is unique per Exporting 
> Process. "
>
> Feedback?
>
> Regards, Benoit.
>
> -- 
> Help        mailto:[email-masked] and say "help" in 
> message body
> Unsubscribe mailto:[email-masked] and say
> "unsubscribe ipfix" in message body
> Archive     http://ipfix.doit.wisc.edu/archive/

--
Help        mailto:[email-masked] and say "help" in message body
Unsubscribe mailto:[email-masked] and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/

---

From: Gerhard Muenz <[email-masked]>
Date: Thu, 01 Jun 2006 17:34:44 +0200

Dear Benoit,

I think your proposal is reasonable. ObservationID being unique with the
 IPFIX Device implies that it is also unique to the Exporting Process
(unless somebody constructs a situation where one Exporting Process
exports data from two devices ;) ).

Regards,
Gerhard

-- 
Dipl.-Ing. Gerhard Münz
Computer Networks and Internet
Wilhelm Schickard Institute for Computer Science
University of Tuebingen
Auf der Morgenstelle 10C 9P16, D-72076 Tuebingen, Germany
Phone: +49 7071 29-70534 / Fax: +49 7071 29-5220
EMail: [email-masked]
WWW:   http://net.informatik.uni-tuebingen.de/~muenz

---

From: Brian Trammell <[email-masked]>
Date: Thu, 1 Jun 2006 13:25:20 -0400

Benoit,

Redefining the Observation Domain to be unique per (exporting) IPFIX  
Device partially fixes the problem, but it does not address the  
problem of Exporting Process/exporting device identification at the  
Collecting Process, of which the Exporting Process IP address  
collision issue is a subset. If an IPFIX device is using multihomed  
SCTP associations, for example, a Collecting Process may not treat  
two Observation Domains that are really identical as such.

Therefore, I would propose again that instead of defining Observation  
Domains to be unique per exporting IPFIX Device, that Observation  
Domains are unique per Session, where a Session is a TCP connection,  
an SCTP association, or a (time-limited) UDP four-tuple. This grants  
the most flexibility to implementors while being, in my opinion, more  
logically sound... If an exporting device implementor wishes instead  
to enforce domain uniqueness per Device, that would be consistent  
with this definition.

Regards,

Brian

On May 30, 2006, at 11:08 AM, Benoit Claise wrote:

> However, while discussing with Lutz Mark and Paul Aitken, I think  
> we discovered the issue
>
> while a second sysadmin configures another set of observation  
> points (on a different line card for example) with the same  
> observation domain Id.
> with the source IP address, how would the collector differentiate  
> the template IDs?
>
> Also, the terminology sections of IFPIX-PROTO and IPFIX-ARCH are  
> not in sync.
> Multiple small changes (including in the terminology sections)  
> should be inserted. For example: IPFIX-PROTO observation domain  
> definition says "In the IPFIX Message it generates, the Observation  
> Domain includes its Observation Domain ID, which is unique per  
> Exporting Process. "
>
> Feedback?
>
> Regards, Benoit.
>
> --

---

From: Juergen Quittek <[email-masked]>
Date: Fri, 02 Jun 2006 08:54:04 +0200

Brian and Benoit,

I would favor the Observation Domain identifier to be unique per
Exporting Process.  I consider 'unique per session' as also feasible.
But I do not think it is feasible to have the Observation Domain unique
per IPFIX device.  I see the following problems with this approach:

  - We would exclude cases where the Observation Domain is not on
    the same box as the exporter.  There are four such cases discussed
    on RFC 3917, page 21 (protocol converter, remote observation point,
    concentrator, and proxy).  According to the current definition of
    an IPFIX device in the protocol draft

>  IPFIX Device
>
>    An IPFIX Device hosts at least one Observation Point, a Metering
>    Process and an Exporting Process.

    none of them are IPFIX devices.  There is no point in asking an ID
    unique per IPFIX device if there is no IPFIX device present in the
    scenario.

Let's assume we fix this by re-defining the term IPFIX device, for
example, by stating that an IPFIX device hosts at least one of the
following: an Observation Point, a Metering Process, or an Exporting
Process.  Then I still see problems with having the Observation Domain
unique per IPFIX device:

  - How would two implementations (of potentially different vendors)
    in the same box coordinate their Observation Domain naming schemes?
    The IPFIX standard is not proposing one.

  - What would an exporter do that reports for several boxes?
    This could be in one of the cases mentioned above (protocol
    converter, remote observation point, concentrator, or proxy).
    In case of the concentrator, all input to the concentrator
    already uses Observation Domains that are unique to their
    respective IPFIX device.  The concentrator would still not
    be able to use them without risking that two Observation
    Domains on different boxes have the same ID.

Thanks,

    Juergen

---

From: Lutz Mark <[email-masked]>
Date: Fri, 02 Jun 2006 13:55:36 +0200

Hi Juergen,

>    unique per IPFIX device if there is no IPFIX device present in the

good point!

Presently boxes like concentrator, proxy, etc. are not mentioned
in the architecture draft.

> Process. 

or all boxes, which generate IPFIX data are IPFIX devices.

> Then I still see problems with having the Observation Domain
> unique per IPFIX device:
> 

right, this is a problem.

>    Domains on different boxes have the same ID.

The protocol draft proposes to use ID zero in that cases. Alternatively
it could generate new Observation Domain IDs for these data because the
ID has only to be unique per IPFIX device.

I tend to change the definition of the IPFIX device to
cover proxies etc.

Best regards,
Lutz

---

From: Lutz Mark <[email-masked]>
Date: Fri, 02 Jun 2006 14:57:58 +0200

Hi Brian,

> Device partially fixes the problem, but it does not address the problem 
> of Exporting Process/exporting device identification at the Collecting 
> Process, of which the Exporting Process IP address collision issue is a 
> subset. If an IPFIX device is using multihomed SCTP associations, for 
> example, a Collecting Process may not treat two Observation Domains that 
> are really identical as such.
> 
> Domains are unique per Session, where a Session is a TCP connection, an 
> SCTP association, or a (time-limited) UDP four-tuple. This grants the 
> most flexibility to implementors while being, in my opinion, more 
> logically sound... If an exporting device implementor wishes instead to 
> enforce domain uniqueness per Device, that would be consistent with this 
> definition.

Having the Observation Domain ID unique per session will ease the
implementation of the exporter and collector part of the protocol.
As you have mentioned an Observation Domain ID that is unique per
IPFIX device implies that the ID is unique per session. Having
the ID unique only per session has the drawback that on the
collector side the ID is quite useless without the session details
and one need additional means to group data from different sessions.
If the ID is unique per IPFIX device, the collector can store
the Observation Domain ID together with the flow data (for each
IPFIX device). And an application can request flow data per
observation domain.

Best regards,
Lutz

---

From: Brian Trammell <[email-masked]>
Date: Fri, 2 Jun 2006 10:10:02 -0400

Lutz and Juergen,

On Jun 2, 2006, at 8:57 AM, Lutz Mark wrote:

>
> Hi Brian,
>
>> Redefining the Observation Domain to be unique per (exporting)  
>> IPFIX Device partially fixes the problem, but it does not address  
>> the problem of Exporting Process/exporting device identification  
>> at the Collecting Process, of which the Exporting Process IP  
>> address collision issue is a subset. If an IPFIX device is using  
>> multihomed SCTP associations, for example, a Collecting Process  
>> may not treat two Observation Domains that are really identical as  
>> such.
>> Therefore, I would propose again that instead of defining  
>> Observation Domains to be unique per exporting IPFIX Device, that  
>> Observation Domains are unique per Session, where a Session is a  
>> TCP connection, an SCTP association, or a (time-limited) UDP four- 
>> tuple. This grants the most flexibility to implementors while  
>> being, in my opinion, more logically sound... If an exporting  
>> device implementor wishes instead to enforce domain uniqueness per  
>> Device, that would be consistent with this definition.
>
> IPFIX device implies that the ID is unique per session.

> Having

Indeed... I hadn't thought of that. We would need some sort of  
session identifier, whether a simple IE or compound, to store session  
details.

> IPFIX device). And an application can request flow data per

Could we address Juergen's concerns with respect to the imprecision  
of the present IPFIX Device definition (an architectural term that  
I'm not personally convinced should have any real meaning at the  
protocol level) by instead defining uniqueness per {Metering Process,  
Exporting Process} tuple?

Argh... actually, neither of these will work as root scopes. The  
{Meter, Exporter} tuple requires stability over time - it requires  
that no Observation Domain ID ever be reused, which is clearly  
ridiculous. A Session (which is really an {Exporter, Collector,  
Source Port, Start Time, End Time} tuple with multiple addresses  
possible for Exporter and Collector in the case of SCTP multihoming)  
is not naturally storable at the collector side.

The "right" thing to do in this case would seem to be to scope  
Observation Domain IDs to the superset of both, a {Meter, Exporter,  
Collector, Start Time, End Time} tuple, but this just seems to be a  
bit too heavy. With what frequency to we expect realistic Exporting  
Processes to assign new Observation Domain IDs to logically identical  
Domains?

As a thought exercise in whether we're overthinking this: what would  
happen if we went to the other extreme, stepped back completely,  
simply defined the Observation Domain as we do now without  
guaranteeing its uniqueness anywhere, and required the implementation  
or deployment to avoid observation domain ID collision out of band?

Regards,

Brian

---

From: Gerhard Muenz <[email-masked]>
Date: Sat, 03 Jun 2006 17:26:22 +0200

Dear all,

We have discussed the Observation Domain ID issue a lot. Something came
into my mind and maybe it is useful to solve the problem.

There are two different things that we want to do:

1) Identify the Exporting Process because we need it for the template
management.

2) Identify the Observation Domain because we want to know where the
traffic as been observed.

The problem is that the Observation Domain ID (formerly known as Source
ID) is in the header of the IPFIX message. This is not logic because the
Obervation Domain ID is linked to a record while the IPFIX header is
something that the Exporting Process is responsible for.

Wouldn't it be better to replace the Observation Domain ID in the IPFIX
header by the Exporter ID?
Then, the pair exporter IP address (that we get from the IP header)
together with the Exporter ID in the IPFIX header could identify the
Exporting Process.

If ever needed, the Observation Domain ID can be exported with the
individual records or within option data.

Regards,
Gerhard

Brian Trammell wrote:
> Indeed... I hadn't thought of that. We would need some sort of session
> identifier, whether a simple IE or compound, to store session details.
> 
> Could we address Juergen's concerns with respect to the imprecision of
> the present IPFIX Device definition (an architectural term that I'm not
> personally convinced should have any real meaning at the protocol level)
> by instead defining uniqueness per {Metering Process, Exporting Process}
> tuple?
> 
> Argh... actually, neither of these will work as root scopes. The {Meter,
> Exporter} tuple requires stability over time - it requires that no
> Observation Domain ID ever be reused, which is clearly ridiculous. A
> Session (which is really an {Exporter, Collector, Source Port, Start
> Time, End Time} tuple with multiple addresses possible for Exporter and
> Collector in the case of SCTP multihoming) is not naturally storable at
> the collector side.
> 
> Collector, Start Time, End Time} tuple, but this just seems to be a bit
> too heavy. With what frequency to we expect realistic Exporting
> happen if we went to the other extreme, stepped back completely, simply
> defined the Observation Domain as we do now without guaranteeing its
> uniqueness anywhere, and required the implementation or deployment to
> avoid observation domain ID collision out of band?
> 
> Regards,
> 
> Brian
>

---

From: Brian Trammell <[email-masked]>
Date: Mon, 5 Jun 2006 08:58:20 -0400

Gerhard,

The issue with this is that Observation Domain ID also holds a  
special role within the protocol; it is the "root" of all scope  
Information Elements (as well as Template IDs, which are themselves a  
special case of scope IEs): the uniqueness and comparability of  
templateID, flowID and commonPropertiesID are only guaranteed within  
a unique Observation Domain ID. The issue with identifying Exporting  
Processes, etc. is that there needs to be _some_ way to guarantee the  
uniqueness and comparability of Observation Domain IDs themselves -  
as we probably don't want to either do the Win32 thing, size it up to  
a GUID, and cross our fingers; or introduce any additional complexity  
either to the protocol or to implementations thereof by specifying  
some sort of Observation Domain ID registration process or other  
mechanism for avoiding collisions.

Regards,

Brian

On Jun 3, 2006, at 11:26 AM, Gerhard Muenz wrote:

>>>> IPFIX
>>>> Observation
>> session
>> {Meter,
>> simply

---

From: Brian Trammell <[email-masked]>
Date: Tue, 6 Jun 2006 10:41:40 -0400

On Jun 6, 2006, at 7:06 AM, Gerhard Muenz wrote:

>
> Brian,
>
> I understand your point. But as we see, using a single identifier for
> both transport-related (who is the Exporter) and content-related  
> (who is
> the Observation Point) information seems to be an unsolvable  
> problem. I
> think that there is no problem to have two different "roots" of  
> scopes,
> one for transport-related information needed for template  
> management and
> the like and another one for content-related information.

Hmmm... I'd not thought of this approach. I'm not yet convinced it  
works (especially because templateId can appear as a scope in a  
scoped data record, though the main use for this seems to go away  
with draft-boschi-ipfix-reducing-redundancy)... but I'd always  
assumed a single root, I guess because as an implementor it seemed  
simpler.

Certainly, treating templates as having special scoping rules seems  
like a good idea, especially if you disallow abuse of the templateId  
as a general-purpose scope (which, again, we can do because we have  
commonPropertiesId).

> Let's have a look at the case of a concentrator. It receives IPFIX
> messages from other IPFIX devices, merges the records in its Metering
> Process, and exports them again. The received records can be from
> various Observation Domains (at least I do not see any reason why  
> not).
> So, which value do you take as Observation Domain ID for the exported
> IPFIX packets? The protocol draft says that you can put a zero, but is
> this satisfactory? Or do you define a virtual Observation Domain as a
> superset of the original ones? It becomes more tricky if there are two
> Exporting Processes running on the same concentrator.

Yes - according to the protocol draft at present, the exported  
messages have observationDomain 0, with observationDomain set to  
something non-zero in each data record (which becomes a _requirement_  
if you care about counting distinct flows in the case that the  
observation domains covered by the concentrator may make multiple  
observations of the same flow). The templateIds in the message then  
become scoped to odId 0, and other scoped records may be explicitly  
scoped to other observation domains.

Actually, when you consider templateIds as having special scoping  
rules, you can remove the requirement that they be rooted at an  
observationDomain, and require only that templateIds be unique per  
_session_. Then you can remove templateId as an information element  
(because you don't need to scope anything to it anymore, and so you  
won't have any issues with finite template lifetimes), and the  
problem seems to go away.

Regards,

Brian

>> special
>> role within the protocol; it is the "root" of all scope Information
>> Elements (as well as Template IDs, which are themselves a special  
>> case
>> of scope IEs): the uniqueness and comparability of templateID, flowID
>> and commonPropertiesID are only guaranteed within a unique  
>> Observation
>> Domain ID. The issue with identifying Exporting Processes, etc. is  
>> that
>> there needs to be _some_ way to guarantee the uniqueness and
>> comparability of Observation Domain IDs themselves - as we probably
>> don't want to either do the Win32 thing, size it up to a GUID, and  
>> cross
>> our fingers; or introduce any additional complexity either to the
>> protocol or to implementations thereof by specifying some sort of
>> Observation Domain ID registration process or other mechanism for
>> avoiding collisions.
>>
>> Regards,
>>
>> Brian
>>
>>> --Dipl.-Ing. Gerhard Münz

---

From: Gerhard Muenz <[email-masked]>
Date: Thu, 08 Jun 2006 09:20:10 +0200

Apparently my mail from two days ago did not make it to the mailing
list, so I send it again. Sorry if you get this message twice.

-------- Original Message --------
Subject: Re: [ipfix-protocol] Observation Domain ID uniqueness
Date: Tue, 06 Jun 2006 13:06:57 +0200
From: Gerhard Muenz <[email-masked]>
To: Brian Trammell <[email-masked]>
CC: [email-masked]

Brian,

I understand your point. But as we see, using a single identifier for
both transport-related (who is the Exporter) and content-related (who is
the Observation Point) information seems to be an unsolvable problem. I
think that there is no problem to have two different "roots" of scopes,
one for transport-related information needed for template management and
the like and another one for content-related information.

various Observation Domains (at least I do not see any reason why not).

Gerhard

Brian Trammell wrote:
> Gerhard,
> 
> The issue with this is that Observation Domain ID also holds a special
> Elements (as well as Template IDs, which are themselves a special case
> and commonPropertiesID are only guaranteed within a unique Observation
> Domain ID. The issue with identifying Exporting Processes, etc. is that
> don't want to either do the Win32 thing, size it up to a GUID, and cross

---

From: Brian Trammell <[email-masked]>
Date: Thu, 8 Jun 2006 10:52:58 -0400

Benoit,

The discussion on this has wandered a bit - to try to summarize and  
bring this to a decision (with apologies if this is either 1.  
something you've already decided or 2. too late for tomorrow's call):

1. The present IPFIX-PROTO solution of declaring observationDomainIDs  
(herein: odId) as unique per Exporting Process requires coordination  
of odId assignment among all the Exporting Processes in a given  
installation, and has a circular definition problem: Exporting  
Process identification if present in a message stream must be  
exported in a scoped data record; scoped data records require a  
template to interpret; templates are identified by a templateID;  
templateIDs are unique per odId; odIds are unique per Exporting  
Process ID.

2. The present IPFIX-ARCH solution (and the proposed solution in the  
original message) of declaring odId as unique per IPFIX Device  
requires extension of the Device definition to cover processes such  
as proxies, some method of coordinating odId assignment among  
multiple disparate processes on the same Device, and (possibly) the  
creation of a Device identifier so that odId uniqueness can be  
maintained in storage at the Collecting Process. This may also have  
the circular definition problem as above.

3. Declaring odId as unique per Session requires an explicit  
definition of Session (which is troublesome in the UDP case), and  
requires the storage of session details at the Collecting Process in  
order to keep persistent odId uniqueness, which is a little ugly.  
However, it avoids the circular uniqueness of templates.

Note the template circular definition problem goes away completely if  
templateIDs are defined to be unique per session, not per odId. This  
makes sense as templateIDs are less likely to be stored by the  
Collecting Process than odIds. The problem of templateID exhaustion  
(which is what I presume defining them to be unique per odId was  
intended to solve) goes away if you forbid templateIDs to be used as  
a general purpose scope, and use the commonPropertiesID (which is a  
64-bit space as opposed to a 16-bit one) from reducing-redundancy  
instead.

Hope this is useful, and clarifies the threads on the subject a bit...

Regards,

Brian

---

From: Carter Bullard <[email-masked]>
Date: Thu, 8 Jun 2006 11:20:38 -0400

Hey Gerhard,
    Argus uses the same ID and IE for both functions, but we
distinguish when the ID is used as a source identifier (your
observation domain) and when its used as a transport
session identiifier (your exporter domain?).   The fact that the
ID is the same is more coincidental and convenient than
anything else, leveraging on the property that a globally
unique identifier is also locally unique (its the same as
DECNET using an ethernet address as both a layer 2
and layer 3 address.

    The difference between Argus and IPFIX in this area, is that
the Source Identifier is mandatory for every record, and readers
don't modify it.   Argus's Transport Session Identifier, has "hop-
to-hop" significance, and can be tossed, modified, whatever, by
any Argus data reader/processor.   So, some records will
have multiple copies of the same IE, one for hop-to-hop transport,
one for end-to-end identification.

Carter

>>> WWW:   http://net.informatik.uni-tuebingen.de/~muenz

Carter Bullard
CEO/President
QoSient, LLC
150 E. 57th Street Suite 12D
New York, New York 10022

+1 212 588-9133 Phone
+1 212 588-9134 Fax

---

From: Andrew Johnson <[email-masked]>
Date: Thu, 08 Jun 2006 17:01:13 +0100

Brian Trammell wrote:
[SNIP]
> intended to solve) goes away if you forbid templateIDs to be used as a 
> general purpose scope, and use the commonPropertiesID (which is a 64-bit 
> space as opposed to a 16-bit one) from reducing-redundancy instead.

I don't think the templateId should be used for scope.  I don't think
this is really a special rule, just something that falls out of how
scope and options should be used.

When a collector receives a record, that record does not have a templateID
in it.  A template was used to decode the record, but after that stage
the templateId can be discarded as it is not relevant to the data in the
record.  Using templateId as scope means that a special rule will have to
be made so that the templateId will be stored in the decoded record.

The same applies to any other inherited field, such as odId, exporterID,
etc.  If these fields are in the record sent by the exporter then they
can be decoded with any options that exporter sent.

Andrew

---

From: "Thomas Dietz" <[email-masked]>
Date: Fri, 9 Jun 2006 11:04:50 +0200

Jürgen,

> -----Original Message-----
> From: majordomo listserver 
> [mailto:[email-masked]] On Behalf Of Juergen Quittek
> Sent: Friday, June 02, 2006 8:54 AM
> To: Brian Trammell; Benoit Claise
> Cc: [email-masked]

Remote Observation Point:

How do we identify the original device where the metering took place in the
case of a remote observation point? What if the remote device has serveral
observation domains? How does the collector know that the exporter is
exporting for multiple devices?

Concentrator:

Is an observation domain really helpful here? If we are concentrating data
from the same source observation domain and exporter it does not change the
original observation domain. If we are concentrating data from different
observation domains and exporters what can we tell about the original
observation domain then?

Similar questions arise with protocol translaters and proxies. I guess we
have to define the term observation domain for all of those special devices
seperately because a common definition would always reduce the information
carried.

>     Domains on different boxes have the same ID.

Exactly what I argued above... How can we fix this without a) loosing
information and b)have a single definition for all devices?

Since the base standard defines the IPFIX device as you cited above the
Observation Domain should - in my opinion - be unique per device. Since
devices like concentraters are covered by seperate drafts/RFCs those
documents should redefine the Observation Domain for those devices.

Regards,

Thomas

---

From: "Thomas Dietz" <[email-masked]>
Date: Fri, 9 Jun 2006 10:50:26 +0200

> [mailto:[email-masked]] On Behalf Of Andrew Johnson
> Sent: Thursday, June 08, 2006 6:01 PM
> To: Brian Trammell
> Cc: Benoit Claise; [email-masked]; Lutz 
> Mark; Gerhard Muenz; Juergen Quittek

The templateId definitely will be used as scope. How do you tell that the
collector which flowkey is used for a template. And even if there is no need
in IPFIX, PSAMP will have the templateId as scope since it will report the
filtering and sampling methods used to get the information for a template.

Regards,

Thomas

---

From: Andrew Johnson <[email-masked]>
Date: Fri, 09 Jun 2006 12:39:40 +0100

Thomas Dietz wrote:
[SNIP]
> collector which flowkey is used for a template.

To interpret the flowkey field you will have to know something about the
template.  The flowkey field is a special case where a field is used to
try and patch up a shortcoming of the protocol.

> And even if there is no need

Every PSAMP record contains a selectorSequenceId which maps to a set of
selectorIds.  These IDs are used to identify the set of filtering and
sampling methods used to get the information in this record.  There is
no use of the templateId in the PSAMP protocol as far as I remember.

regards

Andrew

---

From: Benoit Claise <[email-masked]>
Date: Fri, 09 Jun 2006 16:10:15 +0200

Brian,

Thanks for this summary, and sorry for the delay.
> something you've already decided or 
Only WG consensus will :)
> 2. too late for tomorrow's call):
I consider this issue more important that respecting the deadline and 
having some issues with the published RFC afterwards.
> installation, and has a circular definition problem: Exporting Process 
> identification if present in a message stream must be exported in a 
> scoped data record; scoped data records require a template to 
> interpret; templates are identified by a templateID; templateIDs are 
> unique per odId; odIds are unique per Exporting Process ID.
We agree that this is the problem to be fixed, on the top of the 
following discrepancy:
-in [IPFIX-ARCH] the id is unique within the ipfix device
-in [IPFIX-PROTO] the id is unique to the exporting process
> requires extension of the Device definition to cover processes such as 
> proxies, 
Well, let's say that the problem is slightly different. The IPFIX device 
definition doesn't match RFC3917
> some method of coordinating odId assignment among multiple disparate 
> processes on the same Device, and (possibly) the creation of a Device 
> identifier so that odId uniqueness can be maintained in storage at the 
> Collecting Process. This may also have the circular definition problem 
> as above.
>
We agree this is heavy, not to say ugly :)
> general purpose scope, and use the commonPropertiesID (which is a 
Here is two proposals.

*_Proposal 1: _*
This is what I proposed initially, and summarized by Thomas:

    "Since the base standard defines the IPFIX device as you cited above the
    Observation Domain should - in my opinion - be unique per device. Since
    devices like concentrators are covered by separate drafts/RFCs those
    documents should redefine the Observation Domain for those devices."

So
- We keep the IPFIX device definition as defined right now, and it 
doesn't cover the special devices from RFC3917.
  Anyway, those special devices are not covered in [IPFIX-ARCH].
- Then we can say: Observation Domain Id unique per IPFIX device

Advantages: simple change
Disadvantages: doesn't take into account the special device from RFC3917

*_Proposal 2:_*
We add the Metering Process IP address in the IPFIX header. Note: this 
idea came up when reading Gerhard's email.
Since this is a major change, feedback is more than welcome since I may 
have forgotten something.
- We would augment the ipfix device to special devices from RFC 3917
- We would keep "the observation domain id is unique within the ipfix 
device" as in [IPFIX-ARCH], as it was favor by some people on the 
mailing list (Simon, Thomas, Paul, Lutz, Gerhard, myself).
- This would work for special observation points, assuming that the 
metering process has got an IP address.

       +---+     +---+     +---+
       | E-+->   | E-+->   | E-+------------->---+
       | | |     | | |     | | | +---+         +-+-----+
       +-+-+     | M |     | M | | E-+------->-+-C-M-E-+->
         |       | | |     | | | | | | +---+   +-+-----+
       +-+-+     +-+-+     | O | | M | | E-+->---+
       | | |       |       +---+ | | | | | |
       | M |     +-+-+           | O | | M |
       | | |     | | |           +---+ | | |           +-----+
       | O |     | O |                 | O |        ->-+-C-E-+->
       +---+     +---+                 +---+           +-----+

      Protocol   Remote             Concentrator        Proxy
      Converter  Observation

  In case of a concentrator that would aggregate some data from multiple 
observation point/metering process, the IP address would be one of the 
new metering process, so the one of the concentrator. If the 
concentrator doesn't aggregate (simply re-export), then the initial 
metering process IP address is kept.
- As a bonus, this would serve as the unique device identifier required 
by the multi-homed SCTP association (Brian's issues) when the Metering 
Process and Exporting Process are on the same physical box. So when both 
IP addresses (Metering Process IP address and source IP address of the 
export packet) are on the same box.... Not difficult to check via SNMP.
- Note: IPv4 versus IPv6. We would need a bit somewhere in the header to 
differentiate the two types in the header.

Advantage: this proposal takes care of the special devices in RFC3917
Disadvantage: this is a major change. Another round of reviews and 
last-call are certainly needed.
Disadvantage: [IPFIX-ARCH] should also be changed to speak about the 
RFC3917 special devices.

Considering that the session ID proposal is ugly (to use Brian's words), 
I only see these two proposals/solutions.
Personally, I favor the proposal 1. Please let me know what you think.

Regards, Benoit.

---

From: Brian Trammell <[email-masked]>
Date: Fri, 9 Jun 2006 10:57:06 -0400

On Jun 9, 2006, at 10:10 AM, Benoit Claise wrote:

>> (with apologies if this is either 1. something you've already  
>> decided or
> I consider this issue more important that respecting the deadline  
> and having some issues with the published RFC afterwards.

Benoit,

I was sort of hoping you'd say both these things. :)

>> 1. The present IPFIX-PROTO solution of declaring  
>> observationDomainIDs (herein: odId) as unique per Exporting  
>> Process requires coordination of odId assignment among all the  
>> Exporting Processes in a given installation, and has a circular  
>> definition problem: Exporting Process identification if present in  
>> a message stream must be exported in a scoped data record; scoped  
>> data records require a template to interpret; templates are  
>> identified by a templateID; templateIDs are unique per odId; odIds  
>> are unique per Exporting Process ID.

I think we can separate the templateId circularity problem and solve  
it separately from odId scoping merely by defining them as unique per  
session and not recommended for any sort of persistent storage. But  
that would seem now to be a different thread...

>> 2. The present IPFIX-ARCH solution (and the proposed solution in  
>> the original message) of declaring odId as unique per IPFIX Device  
>> requires extension of the Device definition to cover processes  
>> such as proxies,
> Well, let's say that the problem is slightly different. The IPFIX  
> device definition doesn't match RFC3917
>> some method of coordinating odId assignment among multiple  
>> disparate processes on the same Device, and (possibly) the  
>> maintained in storage at the Collecting Process. This may also  
>> have the circular definition problem as above.

>> requires the storage of session details at the Collecting Process  
>> in order to keep persistent odId uniqueness, which is a little  
>> ugly. However, it avoids the circular uniqueness of templates.

I still think using a Session as a special scope is a good idea for  
non-persistent information; i.e., templates.

>>
>> Note the template circular definition problem goes away completely  
>> if templateIDs are defined to be unique per session, not per odId.  
>> This makes sense as templateIDs are less likely to be stored by  
>> the Collecting Process than odIds. The problem of templateID  
>> exhaustion (which is what I presume defining them to be unique per  
>> odId was intended to solve) goes away if you forbid templateIDs to  
>> be used as a general purpose scope, and use the commonPropertiesID  
>> (which is a 64-bit space as opposed to a 16-bit one) from reducing- 
>> redundancy instead.
> Here is two proposals.
>
> Proposal 1:
> "Since the base standard defines the IPFIX device as you cited  
> above the Observation Domain should - in my opinion - be unique per  
> device. Since devices like concentrators are covered by separate  
> drafts/RFCs those documents should redefine the Observation Domain  
> for those devices." So

Also, this doesn't fix problems with the persistence of  
observationDomainId at the collector-side -- let's say I have odId  
scoped data (because I'm using commonPropertiesId which is unique per  
Observation Domain). Now I need to store some sort of Device  
identifier for each record. How do I generate this device ID at the  
collector side? If it's by layer3 remote address, we have all the  
same problems with multihomed SCTP. This would seem to point to  
proposal 2. But I'm not happy with proposal 2, for the reasons you've  
pointed out...

Maybe I'm over-thinking this - maybe we _can_ realistically not  
address Collecting Process derivation of Device identifier for  
storage purposes, and simply say that it is up to each Collecting  
Process to allow user configuration to ensure it does not compare  
scopes or records across incomparable observationDomainIds. Then, we  
can use a few years of implementation experience to move toward a  
solution more like the second proposal in IPFIXv2.

> Proposal 2:
> We add the Metering Process IP address in the IPFIX header. Note:  
> this idea came up when reading Gerhard's email.

If we're going to do this I'd suggest a 128-bit "Metering Process  
Identifier" with a requirement that Metering Processes MUST support  
the use of one of the Metering Process' IP addresses and SHOULD allow  
the user to configure the device with a different Metering Process  
Identifier, e.g. if a single machine with a single management  
interface is running multiple metering processes across multiple  
observation domains.

Of course, now we're using 160 bits per message (128 mpId, 32 odId)  
as "root" scope. This seems excessive.

> Since this is a major change, feedback is more than welcome since I  
> may have forgotten something.
> - We would keep "the observation domain id is unique within the  
> ipfix device" as in [IPFIX-ARCH], as it was favor by some people on  
> the mailing list (Simon, Thomas, Paul, Lutz, Gerhard, myself).

(see above on metering process IP address differentiation)

(I really like this diagram, by the way... it clarifies well what we  
mean by all the various special devices under consideration)

> In case of a concentrator that would aggregate some data from  
> multiple observation point/metering process, the IP address would  
> be one of the new metering process, so the one of the concentrator.  
> If the concentrator doesn't aggregate (simply re-export), then the  
> initial metering process IP address is kept.
> - As a bonus, this would serve as the unique device identifier  
> required by the multi-homed SCTP association (Brian's issues) when  
> the Metering Process and Exporting Process are on the same physical  
> box. So when both IP addresses (Metering Process IP address and  
> source IP address of the export packet) are on the same box.... Not  
> difficult to check via SNMP.
> - Note: IPv4 versus IPv6. We would need a bit somewhere in the  
> header to differentiate the two types in the header.

Or you could simply use a v6 address always and use v4 address  
mapping (see 2.5.5.2. IPv4-Mapped IPv6 Address, RFC 4291).

> last-call are certainly needed.

If leaving device identification up to the Collecting Process'  
discretion is _at all_ acceptable, this disadvantage alone is more  
than enough to recommend Proposal 1.

Regards,

Brian

---

From: Benoit Claise <[email-masked]>
Date: Fri, 09 Jun 2006 18:01:24 +0200

Brian,
>>> observationDomainIDs (herein: odId) as unique per Exporting Process 
>>> requires coordination of odId assignment among all the Exporting 
>>> Processes in a given installation, and has a circular definition 
>>> problem: Exporting Process identification if present in a message 
>>> stream must be exported in a scoped data record; scoped data records 
>>> require a template to interpret; templates are identified by a 
>>> templateID; templateIDs are unique per odId; odIds are unique per 
>>> Exporting Process ID.
>>> as proxies,
>>> processes on the same Device, and (possibly) the creation of a 
>>> Device identifier so that odId uniqueness can be maintained in 
>>> storage at the Collecting Process. This may also have the circular 
>>> definition problem as above.
>
>>> This makes sense as templateIDs are less likely to be stored by the 
>> "Since the base standard defines the IPFIX device as you cited above 
>> the Observation Domain should - in my opinion - be unique per device. 
>> Since devices like concentrators are covered by separate drafts/RFCs 
>> those documents should redefine the Observation Domain for those 
>> devices." So
> observationDomainId at the collector-side 
However in practice, the common sense imposes that for a router, the 
slot number is used (or a line card for a probe):
    observationDomainId 1 is for line card 1
    observationDomainId 2 is for line card 2
    etc...
So, I'm wondering if this persistence would happen a lot in practice. 
This is why I'm in favor of this proposal 1: common sense would solve it.
> -- let's say I have odId scoped data (because I'm using 
> commonPropertiesId which is unique per Observation Domain). Now I need 
> to store some sort of Device identifier for each record. How do I 
> generate this device ID at the collector side? 
[IPFIX-PROTO]:
                        Collecting Processes SHOULD use the
                        combination of the Exporter (exporterIPv4Address,
                        exporterIPv6Address, or exportingProcessId) and the
                        Observation Domain ID field to separate 
different export
                        streams originating from the same Exporting Process.

In the worst case, i.e. SCTP and source IP address change because of the 
multi-homed feature (which takes by default the IP address of the 
outgoing interface), the collector would have to check whether both IP 
addresses are coming from the same exporter. As simple snmpwalk would 
tell us that. Optionally, a Option Template could help.

Note: there is a unique SCTP AssociationId (32 bit integer, unique for 
the association life time) which could be used, but the new issue is 
that we would have a different behaviour per transport protocol.
> If it's by layer3 remote address, we have all the same problems with 
> multihomed SCTP. This would seem to point to proposal 2. But I'm not 
> happy with proposal 2, for the reasons you've pointed out...
>
> address Collecting Process derivation of Device identifier for storage 
> purposes, and simply say that it is up to each Collecting Process to 
> allow user configuration to ensure it does not compare scopes or 
> records across incomparable observationDomainIds. Then, we can use a 
> few years of implementation experience to move toward a solution more 
> like the second proposal in IPFIXv2.
This is also why I prefer proposal 1.
We concluded a few times already that a IPFIX version 2 might have a 
flexible header....
> interface is running multiple metering processes across multiple 
The management of it doesn't seem obvious at first glance.

Regards, Benoit.
>
> Of course, now we're using 160 bits per message (128 mpId, 32 odId) as 
> "root" scope. This seems excessive.
>
>> multiple observation point/metering process, the IP address would be 
>> one of the new metering process, so the one of the concentrator. If 
>> the concentrator doesn't aggregate (simply re-export), then the 
>> - Note: IPv4 versus IPv6. We would need a bit somewhere in the header 
>> to differentiate the two types in the header.
>
> Or you could simply use a v6 address always and use v4 address mapping 
> (see 2.5.5.2. IPv4-Mapped IPv6 Address, RFC 4291).
>

---

From: Paul Aitken <[email-masked]>
Date: Sat, 10 Jun 2006 13:11:28 +0100

Benoit,

> have forgotten something.

Then all the flows in one export packet must come from the same Metering 
Process IP address.

This makes it difficult for multiple Metering Processes with different 
IP addresses to be associated with a single Exporting Process, since the 
EP would have to maintain seperate export packets for each unique 
Metering Process IP address that it's currently dealing with.

eg, consider multiple line cards, each with its own MP and own MP 
address, but exporting through a single export process on an RP.

The RP export process must maintain individual exports for each MP
address (ie, each LC), so it effectively becomes a set of virtual 
exporters, one for each LC.

But without the Metering Process IP address in the IPFIX header, the EP 
can mix flows from many different MP into one export packet.

If the Metering Process IP address is required then it should be in the 
flow data (or common properties) and not in the IPFIX header.

-- 
Paul Aitken
Cisco Systems Ltd, Edinburgh, Scotland.

---

From: Gerhard Muenz <[email-masked]>
Date: Tue, 13 Jun 2006 01:05:46 +0200

Dear all,

I'll try to make a final statement on this topic, after rereading all
the mails of the last weeks.

I could live with both odId unique per Exporting Process or unique per
IPFIX Device. I think there is no big difference and t is more a
philosophic question. E.g. I have no problem considering two instances
of a monitoring probe software that are running on the same machine and
capture packets on the same interface, as two different devices or two
different exporters on a single device. It does not really matter to me.

I'm not in favor of uniqueness per session. It makes things more
complicated. Consider a monitor that crashes and restarts with the same
configuration, and you are not allowed to treat old and new monitoring
data the same way... Usually the administrator knows about his
monitoring probes and when bigger configuration changes occur that
result in different monitoring data than before.

My opinion concerning the discussion of persistence of scopes, Template
IDs etc. is that we should restrict our responsability to the path from
the observation point to the collector and not any further. If done with
care, Template IDs can serve as scopes on this path. If the
administrator wants to store the monitoring data and requires long-term
persistence of this information, it's up to him to store additional
context information.

I would like to consider concentrators, proxies etc. as IPFIX Devices.
They come with their own Exporting Process(es), so uniqueness per
Exporting Process or uniqueness per Device makes no big difference. They
can either set odId to zero or - if it makes sense - they can use odId
to specify some kind of virtual Observation Domains. The administrator
will know that such identifiers do not correspond to physical interfaces.

Regards,
Gerhard

---

From: Lutz Mark <[email-masked]>
Date: Tue, 13 Jun 2006 12:11:27 +0200

Dear all,

> metering process has got an IP address.

Proposal 3:

o update IPFIX Device Definition and IPFIX-ARCH
   to cover special devices.
o OberservationDomainID unique per IPFIX Device
   (the initial idea was to have a source id, which could
    be e.g. the IPv4 address of the device)
o TemplateIDs locally unique on the path between
   exporter and collector.

advantage: simple change. in line with RFC3917
disadvantage: ObservationDomainID collisions have to be
               avoided out of band.

Best regards,
Lutz

---

From: Juergen Quittek <[email-masked]>
Date: Tue, 13 Jun 2006 17:04:55 +0200

Lutz,

Just a quick comment:

>    to cover special devices.

I support this.

>     be e.g. the IPv4 address of the device)

This is at least not "simple" as you claim below.
There several places in the text where we request
the observation domain ID to be unique per exporting
process.  A quick scan showed at least
  - 4 in the protocol draft
  - 1 in the info model
  - and even one in the architecture draft !
All these changes need to be made carefully.

In contrast, there is only one place in all documents
that requests it to be unique per IPFIX device.
this is in section 5.4 of the architecture document.

> o TemplateIDs locally unique on the path between

This is not clear to me. Do you mean per session?
Or do you mean per {exporting process, collecting process} pair?

Thanks,

    Juergen

---

From: Juergen Quittek <[email-masked]>
Date: Tue, 13 Jun 2006 18:03:21 +0200

Hi Gerhard,

--On 13.06.2006 1:05 Uhr +0200 Gerhard Muenz wrote:

> philosophic question.

For me it is not at all a philosophic problem.
It's an administrative problem.

An observation domain ID does not have any format. If its
value is 0x0003 then this may mean line card #3, interface
with ifIndex #3 or routing processor #3.  We do not have any means
at all to find out what it means.  So what is the value of making
it unique per device?

Let's assume I have two different exporters at a device.
Then who assigns observation domain IDs?
Is there a lookup service where an implementation can find out
what is used?

Let's assume I install a tool like nProbe (assuming it supported IPFIX).
and configure it to measure at eth1 using libpcap.  How do I call
the Observation domain? 0x0001, because it is eth1?
Then I start another tool, lets say a WLAN Scanner, such as kismet,
on interface wlt1 How do I call this observation domain?

If we ask the ID to be unique per device, we should discuss how they
are assigned or retrieved.  If we do not find a good solution, I would
strongly propose to drop uniqueness per devices and stick with
uniqueness per exporter.

Thanks,

    Juergen

> E.g. I have no problem considering two instances

I am also not convinced that uniqueness per session is a good concept.

> I would like to consider concentrators, proxies etc. as IPFIX Devices.

Can we agree on defining that an IPFIX device contains at least one
exporting process but may host any number of additional exporting processes,
metering processes and observation points?

> will know that such identifiers do not correspond to physical interfaces.

This looks like something to be discussed in the implementation guidelines.

Thanks,

    Juergen

---

From: Lutz Mark <[email-masked]>
Date: Tue, 13 Jun 2006 18:05:08 +0200

Juergen,

> Or do you mean per {exporting process, collecting process} pair?

I didn't say per session because in case of a proxy
there are multiple sessions. One between exporter and
proxy and one between proxy and collector. And there
is no need for the proxy to do the template management.

Hm, however I think it makes it easier to have
TemplateIDs valid only within a session.

Best regards,
Lutz

---

From: Gerhard Muenz <[email-masked]>
Date: Tue, 13 Jun 2006 18:34:52 +0200

Hi Jürgen,

Juergen Quittek wrote:
[...]
> It's an administrative problem.

Of course, I just imagine that both solution are somehow feasible.

> Is there a lookup service where an implementation can find out

It can be done by the administrator through configuration.

> the Observation domain? 0x0001, because it is eth1?

It is configurable, you can choose 0x0001 or any other number.

> on interface wlt1 How do I call this observation domain?

The same.

> strongly propose to drop uniqueness per devices and stick with

As I said, it is up to the administrator.

[...]

> metering processes and observation points?

Fine with me.

> This looks like something to be discussed in the implementation guidelines.

Or in documents describing specific issues of these kinds of devices.

Regards,
Gerhard

---

From: Juergen Quittek <[email-masked]>
Date: Tue, 13 Jun 2006 19:19:17 +0200

Hi Gerhard,

>> on interface wlt1 How do I call this observation domain?

If I also call it 0x0001, then this ID is not unique anymore per device,
just per exporting process.

    Juergen

---

From: Gerhard Muenz <[email-masked]>
Date: Tue, 13 Jun 2006 19:24:25 +0200

Hi Jürgen,

Juergen Quittek wrote:
[...]
> just per exporting process.

I meant it is also configurable, i.e. up to the administrator to assign
unique IDs.

Gerhard

---

From: Paul Aitken <[email-masked]>
Date: Tue, 13 Jun 2006 23:08:45 +0100

Juergen,

> at all to find out what it means.  So what is the value of making

An observation domain simply identifies a set of observation points. We 
don't necessarily have to know exactly what those points are - though 
that could perhaps be expressed in a message using the "Identifiers" IE 
group.

The important thing is to know that the set of observation points being 
reported on is somehow different from another set of observation points.

eg one message is about eth1 while another is about eth2. Or one message 
is about line card 3 while another is about line card 4.

So observation domain N is simply the Nth set of observation points 
being reported upon.

> Is there a lookup service where an implementation can find out

That's one possibility, but it's up to the implimentation. Perhaps the 
assignments are made administratively, perhaps they're allocated 
centrally, perhaps there are specially trained elves in the box. Who 
cares? The numbers are allocated out of band as far as IPFIX is 
concerned. All we can do is provide guidelines or set requirements for 
that numbering.

> the Observation domain? 0x0001, because it is eth1?

That would make sense, though any number would suffice. eg, if you 
administratively configured 0x000n for eth-n or line card n or 
workstation n or LAN n or office N. Similarly, the device could use 
self-configured values - in which case the manufacturer should make 
clear what those values mean.

> on interface wlt1 How do I call this observation domain?

Again, any value would suffice. But the key thing is whether it should 
be the same value as for eth1, or a different value?

If you're exporting both data streams to the same collector - or if you 
might reconcile them on one collector in future - then you absolutely 
must use a different observation domain IDs to indicate that different 
sets of observation points were being observed.

Else your collector should see that the two sets of data that were 
collected pertain to the same set of observation points at the same time 
and incorrectly try to reconcile the wired data from eth1 with the 
wireless data from wlt1.

(If you were exporting data to different collectors and there was no way 
that the two sets of data would ever be reconciled, then it wouldn't 
actually matter whether or not they had the same observation domain IDs. 
But best practice would be to use different IDs anyway.)

> are assigned or retrieved. 

Why? The values are simply allocated out of band. Whether 
administratively or automatically depending on what is being observed - 
as long as different IDs are used to represent each set of observation 
points.

> If we do not find a good solution, I would

Uniqueness per exporter is simply wrong! Imagine that I configure an 
exporter reporting on each interface such that eth-n is observation 
domain 0x000n. All is good.

Later I start a second exporter, but by some quirk (administrative typo, 
lightning strike, whatever) it's reporting eth-n as observation domain 
0x000n+1.

The data from each exporter is quite self-consistent, and it's not much 
of an issue if these exporters report to different collectors.

However the data from the two exporters cannot be correctly reconciled 
when the data from the two collectors is eventually cross-referenced or 
when both exporters are reconfigured to export to just one collector.

Clearly the observation domains must be consistent across all the 
exporters on the device, and not unique per exporer.

This is because observation domains are a property of the system as a 
whole, related to the metering processes - and certainly not a property 
of each individual exporting process!

If multiple metering processes are observing the same set of observation 
points - ie, the same observation domain - then they should report the 
same observation domain ID so that the collector(s) may correctly 
reconcile corresponding data.

Cheers.

---

From: Paul Aitken <[email-masked]>
Date: Wed, 14 Jun 2006 00:01:26 +0100

Gerhard,

> complicated. Consider a monitor that crashes and restarts with the 
> same configuration, and you are not allowed to treat old and new 
> monitoring data the same way.

I completely agree.

> I would like to consider concentrators, proxies etc. as IPFIX 

Fine by me.

And Juergen wrote:

> exporting process but may host any number of additional exporting 
> processes, metering processes and observation points?

Agreed - because if a device does not export, then IP Flow Information
*eXport* can hardly claim to apply, can it?

> Exporting Process or uniqueness per Device makes no big difference.

It does if you have multiple exporters reporting information about the
same set of observations points but using different observation IDs for
that. See my earlier mail.

> They can either set odId to zero or - if it makes sense - they can 
> use odId to specify some kind of virtual Observation Domains.

That's exactly what such a device should do, because it's no longer
observing data from the same set of observation points - ie, the same
observation domain - as the original device.

Instead, this is a new observation domain that's defined in terms of
the new device.

Arguably a collector observes data arriving in incoming export packets,
rather than on physical interfaces.

---

From: Andrew Johnson <[email-masked]>
Date: Wed, 14 Jun 2006 01:46:09 +0100

Hello all

> It's an administrative problem.

I agree, I have more questions than answers inline :)

> at all to find out what it means.  So what is the value of making

Indeed, and how would a collector identify that different exporting
processes within the same device?  One suggestion might be that
the source IP address identifies the device and the transport
source port identifies the exporting process.  Multi-homing and NAT
would have large consequences for these assumptions.

> Is there a lookup service where an implementation can find out

If my two metering processes are metering the same interfaces will
I get the same ID?  I would expect so, but that implies my central
management system is going to have to take care of all those IDs
we have which are unique per observation domain.

> strongly propose to drop uniqueness per devices and stick with

I agree, although mandating that the ID be configurable per metering
process would be an option it is not one that I am in favour of.

Andrew

---

From: Andrew Johnson <[email-masked]>
Date: Wed, 14 Jun 2006 02:39:02 +0100

> reported on is somehow different from another set of observation points.

Why is this important to know?  If you have no further information about
an observation domain other than an ID what can it be used for?

> is about line card 3 while another is about line card 4.

While it is useful to know the interface that a flow record is reporting
on, that is what the interface IEs are for.  They give very explicit
information that is a lot more useful that some arbitrary ID.

> concerned. All we can do is provide guidelines or set requirements for 

I feel that where the responsibility lies in enforcing the appropriate
uniqueness of identifiers is an important question that we must answer.

> clear what those values mean.

The definition of the IE should contain sufficient meaning.  In this
case it is an abstract ID with no real meaning in itself but it
provides scope for a number of our other IDs and potentially could
be used as scope in an option to identify properties of that
observation domain.

> as long as different IDs are used to represent each set of observation 

The only advantage I can see of having the observation domain unique
per device rather than per exporting process is when you have more
than one exporting process reporting on the same observation domain.

> Uniqueness per exporter is simply wrong! 

That's quite a strong statement and I disagree.  The management of
this ID has several options and we are simply trying to find the best
one.

> Imagine that I configure an 

Observation domains are not a property of an IPFIX device.  Traditionally
we are used to the set of observation points to be within a single device,
but there is no need to this to be the case.  If I have a set of devices
communicating to a concentrator device, the concentrator will have an
ID which represents the superset of all the observation points it is
reporting on.  If my metering devices are sending this information to
a second concentrator device, the seconds device will pick its own
observation domain ID.

The observation domain is a property of the entire monitoring system
(possibly multi-device), but managing a set of identifiers to represent
each observation domain belongs to the exporting process.

> reconcile corresponding data.

I'm still not sure what you mean by "reconcile" the data.  Aggregation
or finding duplicates would be better achieved by putting more relevant
data in the flow record, such as device IP address and interface SNMP
index.

regards

Andrew

---

From: Andrew Johnson <[email-masked]>
Date: Wed, 14 Jun 2006 02:38:33 +0100

> result in different monitoring data than before.

What do you mean be old and new monitoring data?  I would expect that
during a restart the templates being used could be lost and will have
to be redefined.  Other IDs may have problems too, for example the
common properties ID.

If an exporting process crashes and restarts, is it the same process?

> persistence of this information, it's up to him to store additional

I think that our responsibility in the definition of the protocol would
include specifying all the information that an IPFIX protocol stack
should pass to an application using it.  If we want any contextual
information to be passed to the application with each record then
we must specify it.

I think template IDs should only be used by the IPFIX stack but
observation domain IDs must be available in order to manage the
uniqueness of other IDs.  For example, an option with the scope
of flowId has an implicit scope of observationDomainId.

> will know that such identifiers do not correspond to physical interfaces.

Also, note that these intermediate devices could add an observationDomainId
element to the records they export to identify the original domain that the
record was reported for.

regards

Andrew

---

From: Juergen Quittek <[email-masked]>
Date: Wed, 14 Jun 2006 05:07:19 +0200

Hi Paul,

--On 13.06.2006 23:08 Uhr +0100 Paul Aitken wrote:

> An observation domain simply identifies a set of observation points.
> We don't necessarily have to know exactly what those points are - though
> that could perhaps be expressed in a message using the "Identifiers" IE group.

I discussed this option with Thomas yesterday.  Still we do not see a clean
way of doing so.  do you have an example?

> reported on is somehow different from another set of observation points.

Do you want to ensure that every packet is reported only once?
I do not think this is feasible for general IPFIX devices.

Just consider one observation domain for incoming traffic at a line card
and another one for outgoing traffic in a scheduler of another line card.
Or measurement at a central routing engine or multicast daemon.

> So observation domain N is simply the Nth set of observation points being reported upon.
>
> That's one possibility, but it's up to the implimentation. Perhaps the assignments are made administratively, perhaps they're allocated centrally, perhaps there are specially trained elves in the box. Who cares? The numbers are allocated out of band as
> far as IPFIX is concerned. All we can do is provide guidelines or set requirements for that numbering.
>
>
> That would make sense, though any number would suffice. eg, if you administratively configured 0x000n for eth-n or line card n or workstation n or LAN n or office N. Similarly, the device could use self-configured values - in which case the
> manufacturer should make clear what those values mean.
>
>

If I run both in parallel, using the same value would not be possible,
if we want to be unique per device.

> If you're exporting both data streams to the same collector - or if you might
> reconcile them on one collector in future - then you absolutely must use a
> different observation domain IDs to indicate that different sets of observation
> points were being observed.
 If we require "unique per device", I cannot use the same ID twice.

> Else your collector should see that the two sets of data that were collected
> pertain to the same set of observation points at the same time and incorrectly
> try to reconcile the wired data from eth1 with the wireless data from wlt1.
>
> (If you were exporting data to different collectors and there was no way that
> the two sets of data would ever be reconciled, then it wouldn't actually matter
> whether or not they had the same observation domain IDs. But best practice would be to use
> different IDs anyway.)

Yes. of course.

> Why? The values are simply allocated out of band. Whether administratively
> or automatically depending on what is being observed - as long as different
> IDs are used to represent each set of observation points.

This does not help much without knowing how the sets are defined.
I one set is on a line card at directly a the lines (observing arriving
packets) and another set on the same line card at the backplane (observing
packets leaving the card), then what does different IDs help you more than
that packets were measured a somehow different observation points?

> Uniqueness per exporter is simply wrong! Imagine that I configure an exporter
> reporting on each interface such that eth-n is observation domain 0x000n.
> All is good.
>
> lightning strike, whatever) it's reporting eth-n as observation domain 0x000n+1.
>
> The data from each exporter is quite self-consistent, and it's not much of an
> issue if these exporters report to different collectors.
>
> However the data from the two exporters cannot be correctly reconciled when
> the data from the two collectors is eventually cross-referenced or when both
> exporters are reconfigured to export to just one collector.
>
> Clearly the observation domains must be consistent across all the exporters on
> the device, and not unique per exporer.

I fully understand the need for it.  Still I have doubts that we are asking
for something that can only be guaranteed if all components come from the
same (coordinating) source.

> This is because observation domains are a property of the system as a whole,
> related to the metering processes - and certainly not a property of each
> individual exporting process!

Here I disagree.  Why shall we disallow dynamic configuration of observation
domains.  What is the problem with configuring my meter to treat interfaces
1, 7, and 13 as one observation domain?

Cheers,

    Juergen

> points - ie, the same observation domain - then they should report the same
> observation domain ID so that the collector(s) may correctly reconcile
> corresponding data.
>
> Cheers.
> --
> Paul Aitken

---

From: Paul Aitken <[email-masked]>
Date: Wed, 14 Jun 2006 12:36:29 +0100

Juergen,

> way of doing so.  do you have an example?

I was thinking to use an option message using only scope fields (ie, 
with Field Count = Scope Field Count). But scope really applies to 
specific templates, so it's not a clean solution.

> I do not think this is feasible for general IPFIX devices.

No; there's no requirement for that. In fact, it may be desireable to 
report some packets more than once - eg, to make additional reports to 
backup collectors.

In that case it's crucial that all the collectors understand that all 
the reports are for the same observation, so that the packet is not 
later mistakenly counted N times.

That's why I think per-exporter observation domain IDs are wrong.

>> cares? The numbers are allocated out of band as
>> self-configured values - in which case the

Yes, I know that. But it's possible if the requirement is only "unique 
per exporter", and I wanted to explore the repercussions of that.

> If we require "unique per device", I cannot use the same ID twice.

Correct.

>> collected
>> administratively

Clearly it's desireable to have a mechanism to describe what an 
observation domain actually means. It might be desireable to report the 
location as eg "ingressInterface 2", or to report it in terms of several 
observation points which then need their own locations specified.

But to answer your question: knowing whether the same set of points were 
observed or whether it was a different set of points is quite crucial - 
even if we don't know quite what those particular points are - because 
without that knowledge we might incorrectly suppose to aggregate all the 
data together, or fail to aggregate data that does actually pertain to 
the same observations.

Suppose my primary exporter fails for a while, and my secondary exporter 
kicks in until the primary returns. I need to be able to reconcile the 
data sent through the secondary exporter with the data from the primary.

If each exporter uses its own observation domain IDs then I won't be 
able to do that very sucessfully.

Rather, I would want to match the OD's reported by the secondary 1:1 
with the OD's already known on the primary.

>> exporter
>> when
>> whole,

I don't see a problem with that.

My point is that "observation domain" tells you about where data were 
observed, and since exporting processes do not do any observing, the OD 
is clearly not a property of the EP.

Cheers.

---

From: Paul Aitken <[email-masked]>
Date: Wed, 14 Jun 2006 13:01:52 +0100

Andrew,

> processes within the same device?

Does it even need to? Probably the collector is more interested in the 
observed data than in details of the process that delivered it.

> One suggestion might be that
> source port identifies the exporting process.

The source port could change if the export process restarted. However, 
the data being reported by that EP remains consistent before and after 
the restart, and the collector must not treat the data received 
afterwards differently to the data received before.

Or if the data was temporarily exported by a different process (eg, 
during the primary export process's failure), the collector should not 
treat that data any differently just because it's coming from a 
different EP.

> I get the same ID?  I would expect so,

If you want to reconcile the data from the two metering processes then 
you really must use the same observation domain ID to indicate that the 
same set of observation points were being measured. Using different IDs 
indicates that different points are being measured, so the data aren't 
compatible and can't be reconciled.

> but that implies my central

Allocation of IDs could be done in many ways, even without a central 
system. eg you could allocate ID's in the form 0xnnNN to line card nn, 
obervation domain NN.

Cheers.

---

From: Paul Aitken <[email-masked]>
Date: Wed, 14 Jun 2006 13:01:58 +0100

Andrew,

> What do you mean be old and new monitoring data?

Clearly, data received at the collector before and after the restart.

> I would expect that

Even so, this should be transparent to the collector: it should be able 
to continue to append data received after the restart to data received 
before. (Unless for some reason the data being exported changed and is 
no longer compatible, eg fields were added or removed during the restart.)

> If an exporting process crashes and restarts, is it the same process?

I think not, but I also think it shouldn't matter at all.

> of flowId has an implicit scope of observationDomainId.

I very much agree.

> element to the records they export to identify the original domain that the

If an intermediate device can uniquely identify a single OD that 
originally reported the data (eg it's a proxy, or performing some 
filtering) then it would be as well reporting that data as if it were 
from the original OD (ie, spoofing).

But if it's doing some more complex operation (eg, aggregating records 
from different devices, or from different OD's on the same device) then 
reporting the original OD's would be more difficult.

ie, how would we report that we're aggregating data from OD 1 at 
10.0.0.1 and OD's 2 and 3 at 10.0.0.2 ?

Probably it's better to report that we are a new OD at the intermediate 
device, and use a yet to be determined method to report exactly what 
this OD represents. Clearly this is something that urgently needs addressed.

Cheers.

---

From: Paul Aitken <[email-masked]>
Date: Wed, 14 Jun 2006 13:41:48 +0100

Andrew,

>> The important thing is to know that the set of observation points 
>> being reported on is somehow different from another set of observation 
>> points.
> 
> 
> Why is this important to know?

Because the difference between { "some data" + "some more data" } and { 
"some data" + "some other data" } is that the first two pertain to a 
common set of observation points and can be reconciled, while the second 
two do not relate to the same observation points and cannot be reconciled.

To clarify: when you're watching the world cup, why is it important to 
know which players are in which team? The players (observation points) 
wear different shirts to show which team (observation domain) they 
belong to. And when a reserve comes on (some more data) it's immediately 
obvious which team (observation domain) he belongs to.

If we didn't know who was on which team (ie, which observation domain 
was which) then there'd simply be chaos!

> If you have no further information about

Putting together or seperating out observations made at different places.

eg, I have no clue who the players are in each team, but knowing the 
team name is quite sufficient to follow their progress.

>> eg one message is about eth1 while another is about eth2. Or one 
>> message is about line card 3 while another is about line card 4.
> 
> 

Sure; it was only a simple example.

Suppose we have two observation domains: OD1 = eth1, eth2, eth3 while 
OD2 = eth4, eth5, eth6. Even if it's not clear to us which interfaces 
are in each OD, it's clear that when we receive data about OD1 it's 
quite seperate from data received about OD2.

But when we receive some additional data about OD1 that clearly belongs 
with the previous data we received for OD1, and not with the previous 
data for OD2 nor in some seperate place again.

> be used as scope in an option to identify properties of that

Team names are quite abstract too, though they're quite sufficient for 
identifying the team's own properties. It's not necessary for the team 
name to convey any further information itself.

>> administratively or automatically depending on what is being observed 
>> - as long as different IDs are used to represent each set of 
>> observation points.
> 
> 

In which case it's quite crucial.

>> Imagine that I configure an exporter reporting on each interface such 
>> that eth-n is observation domain 0x000n. All is good.
>>
>> Later I start a second exporter, but by some quirk (administrative 
>> typo, lightning strike, whatever) it's reporting eth-n as observation 
>> domain 0x000n+1.
>>
>> The data from each exporter is quite self-consistent, and it's not 
>> much of an issue if these exporters report to different collectors.
>>
>> when the data from the two collectors is eventually cross-referenced 
>> or when both exporters are reconfigured to export to just one collector.
>>
>> whole, related to the metering processes - and certainly not a 
>> property of each individual exporting process!
> 
> 

The observation domain ID chosen at the second device should be chosen 
quite independantly of the first device, and may even be the same as an 
ID already in use by the first device.

[IPFIX-PROTO] says:

The same rule also allows separation of streams which happen to use the 
same observation domain ID but come from different addresses.

> each observation domain belongs to the exporting process.

I disagree on both points.

>> If multiple metering processes are observing the same set of 
>> observation points - ie, the same observation domain - then they 
>> should report the same observation domain ID so that the collector(s) 
>> may correctly reconcile corresponding data.
> 
> 
> I'm still not sure what you mean by "reconcile" the data.

I mean recognising that data received later in time or seperately (eg, 
on another collector) pertains to the same set of observation points on 
the same device, and dealing with it appropriately rather that storing 
it separately.

> Aggregation

Suppose eth1, 2 and 3 are redundant links to some department. They're in 
OD1 so I can measure that department's traffic. Similarly eth3, 4 and 5 
are links to another department, which are in OD2.

Using ODs to differentiate each department's traffic is clearly much 
simpler than trying to figure out which deparment "ingressInterface=4" is.

Cheers.

---

From: Paul Aitken <[email-masked]>
Date: Wed, 14 Jun 2006 13:49:50 +0100

Thomas,

> Here I would agree. But your example with the other exporter taking 
> over if the first is not available brings in a second component. How 
> does the collector now (because the second exporter most likely has a
> different IP address) that it receives data from the same IPFIX 
> device then?

Good question. Probably the second exporter must use (or at least spoof)
the first exporter's address, because [IPFIX-PROTO] says:

- and clearly we DON'T want to seperate these streams in this case.

>> filtering) then it would be as well reporting that data as if it 
>> were from the original OD (ie, spoofing).
> 
> 
> If it is proxying a single device then I very much agree again. But 
> if it proxies more than one device you need to find a way to map the 
> OD Ids.

I agree this is something we still need to address.

>> Probably it's better to report that we are a new OD at the 
>> intermediate device, and use a yet to be determined method to 
>> report exactly what this OD represents. Clearly this is something 
>> that urgently needs addressed.
> 
> 
> I agree with your proposal here, that a concentrator should use a new
> OD Id but I don't think we need to address the details here. Those 
> details (how to report orignial OD Ids etc.) should be handled by a 
> different RFC then.

Whether it's a new RFC or it simply holds up progress on an existing
one, clearly we need to define this mechanism!

Cheers.

---

From: "Thomas Dietz" <[email-masked]>
Date: Wed, 14 Jun 2006 14:40:28 +0200

Paul,

> [mailto:[email-masked]] On Behalf Of Paul Aitken
> Sent: Wednesday, June 14, 2006 2:02 PM
> To: Andrew Johnson
> Cc: Gerhard Muenz; [email-masked]
> Subject: Re: [ipfix-protocol] Observation Domain ID 
> uniqueness - summary and proposals
> 
> Andrew,
> 

Here I would agree. But your example with the other exporter taking over if
the first is not available brings in a second component. How does the
collector now (because the second exporter most likely has a different IP
address) that it receives data from the same IPFIX device then?

> from the original OD (ie, spoofing).

If it is proxying a single device then I very much agree again. But if it
proxies more than one device you need to find a way to map the OD Ids.

> intermediate 

I agree with your proposal here, that a concentrator should use a new OD Id
but I don't think we need to address the details here. Those details (how to
report orignial OD Ids etc.) should be handled by a different RFC then.

Best Regards,

Thomas

---

From: Andrew Johnson <[email-masked]>
Date: Wed, 14 Jun 2006 14:26:01 +0100

> observed data than in details of the process that delivered it.

If the odId is unique per device then a collector absolutely MUST be
able to identify that exporting processes are from the same or
different devices in order to identify the scope of the odId and
be able to understand when a matching odId from two different EPs
refers to the same or different observation domains.

If the odId is unique per EP then a collector MUST be able to
differentiate between EPs, potentially a much simpler problem.

Note: If the odId is unique per device and things like the
commonPropertiesId are unique per ObservationDomain then a collector
will have to group the options it receives from the same device but
different EPs.  This means that different EPs will be able to
override options sent by each other if the scope matches.

Furthermore, this means that whatever method is used to administer
the odIds would have to be extended for all the other IDs which
are unique per observation domain.

> afterwards differently to the data received before.

A template ID is unique per exporting process.  If an EP were to restart
and change source port then is it OK for the collector to decode records
with templates it received before the restart?  If not, then it should
be considered a new process.  Otherwise how does the collector identify
that it's the same process on a different port?

> compatible and can't be reconciled.

Why is the odId some magic identifier that lets you do this?  You still
sufficient information in the flow record in order to be able to reconcile
the information.

For example, imagine a simple router with 4 ports.  The observation domain
may be fixed at 1 to identify the largest domain of all 4 ports.  A Metering
Process may only monitor two of those ports.  A second process may monitor
a different pair, but there could be overlap.  To reconcile flow records
here you will probably need the ifIndex to be able to find duplicate flows
here.

To expand on that example, what if I have two concentrators aggregating
information from overlapping subsets of routers.  These concentrators
are separate IPFIX devices so have their odIds can't be compared.

In short, sufficient scope to reconcile data is not going to be solved
by making the odId unique per device.

> system. eg you could allocate ID's in the form 0xnnNN to line card nn, 

And how do you allocate all the other IDs that are unique per observation
domain?

Andrew

---

From: Benoit Claise <[email-masked]>
Date: Thu, 15 Jun 2006 17:03:04 +0200

Andrew,
>> However the data from the two exporters cannot be correctly 
>> reconciled when the data from the two collectors is eventually 
>> cross-referenced or when both exporters are reconfigured to export to 
>> just one collector.
>>
> reporting on.  
This is the key question, as summarized in my initial email. Which IPFIX 
definition do we choose?
Proposal 1:  We keep the IPFIX device definition as defined right now, 
and it doesn't cover the special devices from RFC3917. Protocol unchanged.
Proposal 2:  We include all special cases in the IPFIX device 
definition, and we have to change something in the protocol.

Regards, Benoit.

---

From: Juergen Quittek <[email-masked]>
Date: Fri, 16 Jun 2006 08:54:49 +0200

Hi Benoit,

--On 15.06.2006 17:03 Uhr +0200 Benoit Claise wrote:

> Proposal 2:  We include all special cases in the IPFIX device definition,
> and we have to change something in the protocol.

Which changes do you think we would need to apply?

Thanks,

    Juergen

---

From: Benoit Claise <[email-masked]>
Date: Fri, 16 Jun 2006 09:00:37 +0200

Hi Juergen,
>> definition,
I was reading/answering the emails in sequence.
Let's forget about this track (of changing the protocol) to focus on 
your last proposal http://ipfix.doit.wisc.edu/archive/3366.html, which I 
support.

Regards, Benoit.