Subject: [IPFIX] observation domain, template, and session: compromise

From: Benoit Claise <[email-masked]>
Date: Wed, 11 Oct 2006 15:08:18 +0200

Dear all,

Trying to address the concerns posted on the mailing recently regarding 
"observation domain, template, and session", Andrew, Paul, and I sat 
down and worked out a compromise. We're proposing to make template IDs 
unique per (Observation Domain ID, Session) pair. And NOT to scope the 
Observation Domain to the communication session between the Exporting 
Process and Collecting Process (as proposed by Andrew initially)
I think that this compromise will make everybody happy as this solution 
is mainly a clarification, as opposed to a protocol design change.

Problem Statement.
If I read the protocol draft now, one interpretation could be: as I see 
that the Template Ids are unique per Observation Domain, and that the 
Observation Domain Id are RECOMMENDED to be unique per IPFIX Device, 
this implies that the several Exporting Processes of the same IPFIX 
Device must synchronize the assignments of their Template Ids (to make 
sure that the Template Id are unique per Observation Domain). This might 
prove difficult/impossible in some cases.
Instead of imposing this, the other solution is to make the template IDs 
unique per (Observation Domain ID, Session) pair.
This makes sense as, logically, the template IDs have only a local 
significance between E.P. and C.P.
Note: I brought up the issue of load-balancing or backup where it would 
be nice to have the same template IDs between two connections. The 
solution in this case is to use a single E.P. when load-balancing or 
backup is required.

Please acknowledge that this compromise is fine with you.

Note: 3 minor changes are required in the information elements 
definitions in [IPFIX-INFO]

Regards, Andrew, Paul, and Benoit.

See below for the proposed changes.

_[IPFIX-PROTO]
__
_NEW:_
_

Transport Session
In SCTP, the transport session is known as the SCTP association, which 
is uniquely identified by the SCTP endpoints [RFC2960]; in TCP, the 
transport session is known as the TCP connection, which is uniquely 
identified by the combination of IP addresses and TCP ports used; In 
UDP, the transport session is known as the UDP session, which is 
uniquely identified by the combination of IP addresses and UDP ports used.

OLD
Template ID
      Each of the newly generated Template Records is given a unique
      Template ID.  This uniqueness is local to the Observation
      Domain that generated the Template ID.  Template IDs 0-255 are
      reserved for Template Sets, Options Template Sets, and other
      reserved Sets yet to be created.  Template IDs of Data Sets
      are numbered from 256 to 65535.  There are no constraints
      regarding the order of the Template ID allocation.

NEW:
Template ID
      Each of the newly generated Template Records is given a unique
      Template ID.  This uniqueness is local to the Transport Session and
     Observation Domain that generated the Template ID.  Template IDs 
0-255 are
      reserved for Template Sets, Options Template Sets, and other
      reserved Sets yet to be created.  Template IDs of Data Sets
      are numbered from 256 to 65535.  There are no constraints
      regarding the order of the Template ID allocation.

OLD (Section 4.4)
     templateId              An identifier of a Template that
                             locally unique to the Exporting
                             Process.  This Information
                             Element MUST be defined as a Scope
                             Field.
NEW
     templateId              An identifier of a Template. This Information
                                   Element MUST be defined as a Scope
                                   Field.

Section 3.1
OLD
Observation Domain ID
A 32-bit identifier of the Observation Domain that is locally unique to 
the Exporting Process.  The Exporting Process uses the Observation 
Domain ID to uniquely identify to the Collecting Process the Observation 
Domain that metered the Flows.  It is RECOMMENDED that this identifier 
is also unique per IPFIX Device.  Collecting Processes SHOULD use the 
combination of the Exporter (exporterIPv4Address, exporterIPv6Address, 
or exportingProcessId) and the Observation Domain ID field to separate 
different export streams originating from the same Exporting Process.  
The Observation Domain ID SHOULD be 0 when no specific Observation 
Domain ID is relevant for the entire IPFIX Message.  For example, when 
exporting the Exporting Process Statistics, or in case of hierarchy of 
Collector when aggregated data records are exported.

NEW
Observation Domain ID
A 32-bit identifier of the Observation Domain that is locally unique to 
the Exporting Process.  The Exporting Process uses the Observation 
Domain ID to uniquely identify to the Collecting Process the Observation 
Domain that metered the Flows.  It is RECOMMENDED that this identifier 
is also unique per IPFIX Device.  Collecting Processes SHOULD use the 
Transport Session and the Observation Domain ID field to separate 
different export streams originating from the same Exporting Process.  
The Observation Domain ID SHOULD be 0 when no specific Observation 
Domain ID is relevant for the entire IPFIX Message.  For example, when 
exporting the Exporting Process Statistics, or in case of hierarchy of 
Collector when aggregated data records are exported.

Section 8, Template Management
OLD
The Exporting Process assigns and maintains the Template IDs for the 
Exporter's Observation Domains. A newly created Template Record is 
assigned an unused Template ID by the Exporting Process.

NEW
The Exporting Process assigns and maintains the Template IDs per SCTP 
association for the Exporter's Observation Domains.  A newly created 
Template Record is assigned an unused Template ID by the Exporting Process.

OLD
A Template ID MUST be unique per Observation Domain. Different 
Observation Domains from the same Exporter may use the same Template ID 
value to refer to different Templates.

NEW
Different Observation Domains from the same SCTP association may use the 
same Template ID value to refer to different Templates.

OLD
When the Exporting Process restarts, due to the SCTP association 
shutdown, all Template assignments are lost and Template IDs MUST be 
re-assigned.

NEW
When the SCTP association shuts down or the Exporting Process restarts, 
all Template assignments are lost and Template IDs MUST be re-assigned.

Section 9, Collecting Process's side
OLD
Template IDs are unique per IPFIX Device and per Observation Domain.  If 
the Collecting Process receives a Template which has already been 
received but which has not previously been withdrawn (i.e. a Template 
Record from the same Exporter Observation Domain with the same Template 
ID), then the Collecting Process MUST shutdown the association.

NEW
Template IDs are unique per SCTP association and per Observation 
Domain.  If the Collecting Process receives a Template which has already 
been received but which has not previously been withdrawn (i.e. a 
Template Record from the same Exporter Observation Domain with the same 
Template ID received on the SCTP association), then the Collecting 
Process MUST shutdown the association.

OLD

If a Collecting Process receives a Template Withdraw Message, the 
Collecting Process MUST delete the corresponding Template Records 
associated with the specific Exporter and specific Observation Domain, 
and stop decoding IPFIX Messages that use the withdrawn Templates.
 
If the Collecting Process receives a Template Withdraw message for a 
Template Record it has not received before, it MUST reset the SCTP 
association, discard the IPFIX Message, and SHOULD log the error as it 
does for malformed IPFIX Messages. 
 
A Collecting Process that receives IPFIX Messages from several 
Observation Domains from the same Exporter MUST be aware that the 
uniqueness of the Template ID is not guaranteed across Observation Domains.

NEW
If a Collecting Process receives a Template Withdraw Message, the 
Collecting Process MUST delete the corresponding Template Records 
associated with the specific SCTP association and specific Observation 
Domain, and stop decoding IPFIX Messages that use the withdrawn Templates.
I
f the Collecting Process receives a Template Withdraw message for a 
Template Record it has not received before on this SCTP assocation, it 
MUST reset the SCTP association, discard the IPFIX Message, and SHOULD 
log the error as it does for malformed IPFIX Messages. 
 
A Collecting Process that receives IPFIX Messages from several 
Observation Domains on the same Transport Session MUST be aware that the 
uniqueness of the Template ID is not guaranteed across Observation Domains.

10.3.7 Collecting Process
OLD
At any given time the Collecting Process SHOULD maintain the following 
for all the current Template Records and Options Template Records: 
<Exporting Process, Observation Domain ID, Template ID, Template 
Definition, Last Received>.

NEW
Template IDs are unique per UDP session and per Observation Domain.  At 
any given time the Collecting Process SHOULD maintain the following for 
all the current Template Records and Options Template Records: <IPFIX 
Device, Exporter source UDP port, Observation Domain ID, Template ID, 
Template Definition, Last Received>.

10.4.2.2 Template Management

Template IDs are unique per TCP connection and per Observation Domain.  
A Collecting Process MUST record all Template and Options Template 
Records for the duration of the connection, as an Exporting Process is 
not required to re-export Template Records.

_[IPFIX-INFO]_

NEW

Transport Session
In SCTP, the transport session is known as the SCTP association, which 
is uniquely identified by the SCTP endpoints [RFC2960]; in TCP, the 
transport session is known as the TCP connection, which is uniquely 
identified by the combination of IP addresses and TCP ports used; In 
UDP, the transport session is known as the UDP session, which is 
uniquely identified by the combination of IP addresses and UDP ports used.

OLD
5.1.8.  templateId

   Description:
      An identifier of a Template that is locally unique within an
      Observation Domain.

NEW
5.1.8.  templateId

   Description:
      An identifier of a Template that is locally unique within an
      Observation Domain and the Transport Session.

OLD
5.1.7.  flowId

   Description:
      An identifier of a Flow that is unique within an Observation
      Domain.  This Information Element can be used to distinguish
      between different Flows if Flow Keys such as IP addresses and port
      numbers are not reported or are reported in separate records.

NEW
5.1.7.  flowId

   Description:
      An identifier of a Flow that is unique within an Observation
      Domain and the Transport Session.  This Information Element can be used to distinguish
      between different Flows if Flow Keys such as IP addresses and port
      numbers are not reported or are reported in separate records.

OLD
5.1.11.  commonPropertiesId

   Description:
      An identifier of a set of common properties that is unique per
      Observation Domain.  Typically, this Information Element is used
      to link to information reported in separate records.

NEW:
5.1.11.  commonPropertiesId

   Description:
      An identifier of a set of common properties that is unique per
      Observation Domain and Transport Session.  Typically, this Information Element is used
      to link to information reported in separate records.

_______________________________________________
IPFIX mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ipfix