Subject: [ipfix-protocol] draft-ietf-ipfix-protocol-09.txt available

From: Benoit Claise <[email-masked]>
Date: Tue, 08 Mar 2005 00:08:14 +0100

Dear all,

A new version of the IPFIX protocol draft has been compiled.
As we are in last-call, and as we might discuss some of these already 
resolved issues this week, allow me to share it with you.
If you haven't started yet reviewing draft-ietf-ipfix-protocol-08.txt, 
start with draft-ietf-ipfix-protocol-09.txt as this is much more complete
You can find the draft at 
ftp://ftp.netlab.nec.de/pub/internet-drafts/draft-ietf-ipfix-protocol-09.txt

Here is the list of improvement:
- there were inconsistencies in the packet formats of the previous draft.
  For example, the figure H was not exactly a generic format, and not exactly an example.
  Thomas Dietz rewrote the full sections 5 and 6. 
  It now contains generic format + separate examples.25 <#_Toc97709690>
- corrected a typo, reported by Mark Lutz "an Exporting Process MUST NOT 
  attempt to establish a connection more frequently than once per minute. "
- improved examples:
  * there were inconsistencies between hexa and non hexa value 
  * [IPFIX-INFO] Information Elements ID are now used 
  * the example now uses the lineCardId: PROTO-38 is closed: 
- new section 9.1 specifying the data types encoding. This was missing for time-related based types, for example.
  Thanks to Thomas Dietz for writing the text.
- consistence of field versus information elements
- improved abstract by Juergen Quittek
- Corrected the source ID definition: remove Process in "Exporter_ Process_ Observation Domain"
- removed ipfixOption in the specific Option Templates
  Integration of the changes proposed in my post "Review of the "7. Specific Reporting
  Requirements" and of the ipfixOption Information Element"
- the "time" Information Element in the specific Options Templates has been clarified. 
- Integration of the changes proposed in my post "some more info regarding the scope in [IPFIX-INFO]"
- Time related Information Element now in sync. with [IPFIX-INFO]
 [IPFIX-INFO] specifies: 

    5.6.15  flowStartDeltaUSeconds
       Description: The _positive _time offset of the first packet of
    this flow
          relative to a base time given in the IPFIX message header.

    5.6.16  flowEndDeltaUSeconds
       Description: The _positive _time offset of the last packet of
    this flow relative
          to a base time given in the IPFIX message header.

 => this implies this change in [IPFIX-] (positive keyword was added)
   For a Data Set with Flow Records requiring microsecond precision, 
   the IPFIX Message Header "Export Time" field SHOULD be calculated so 
   that each Flow Records flowStartDeltaUsec [IPFIX-INFO] and 
   flowEndDeltaUsec [IPFIX-INFO] would contain a 32 bit _positive_
   microsecond offset from the "Export Time" base timestamp. In which 
   case, if Flow Records with a different precision are needed 
   (millisecond or nanosecond), these MUST be sent in different Data 
   Sets. 
 
 => This implies that the method called the "one-pass approach" is not
    valid anymore because the offset could also be negative. 
    The following new text was introduced to cover the 2 phases approach

For a Data Set with Flow Records requiring microsecond precision, an 
example of the "Export Time" calculation is the following: before 
exporting the Flow Records, the Exporting Process takes as "Export Time" 
the lowest start time of all the Flow Records to be exported in this 
Data Set. The only constraint is that the highest end time of the Flow 
Records in that Data Set can not exceed the "Export Time" plus 71 
minutes. If a Flow Record end time exceeds this condition, this Flow 
Record must be exported in a subsequent IPFIX Message.

 => next definition also changed (second sentence added):
 Export Time 
           Time in seconds since 0000 UTC 1970, at which the IPFIX 
           Message Header leaves the Exporter. Alternatevily, the 
           Export Time can be a computed value: see the sectin 8
           "IPFIX Message Header "Export Time" and Flow Record Time"

We are very close to completion, only a few remaining issues:

  PROTO-100: Flow Data Records would become Data Records (consensus on 
   the mailing so far?)  
   +------------------+---------------------------------------------+  
   |                  |                    Contents                 |  
   |                  +--------------------+------------------------+  
   |       Set        |       Template     |        Record          |  
   +------------------+--------------------+------------------------+  
   |   Data Set       |          /         |     Data Record(s)     |  
   +------------------+--------------------+------------------------+  
   |   Template Set   | Template Record(s) |           /            |  
   +------------------+--------------------+------------------------+  
   | Options Template | Options Template   |           /            |  
   |       Set        |     Record(s)      |                        |  
   +------------------+--------------------+------------------------+  
 
     PROTO-101: Section 9.1.7 dateTimeMicroSeconds -> "Its encoding is 
     not defined yet." 
      
     PROTO-102: Duplicate chapter "the text from chapter 4 "Criteria for 
     Flow Expiration and Export" with [IPFIX-ARCH]. Erase this one? 

Regards, Benoit.

---

From: Lutz Mark <[email-masked]>
Date: Tue, 08 Mar 2005 14:00:42 +0100

>     The following new text was introduced to cover the 2 phases approach

I see that the proposed method saves some bytes on the wire,
but from an implementors point of view chapter 8 is quite ugly.

Keeping the 'one-pass approach' make the implementation a little
bit easier. I prefer not to limit the offset to positive values.

If there was already a decision not to allow negative offsets
we can omit negative offsets via sending these data in a separate
message:

8<----------------------------------------------

    By default, the IPFIX Message Header...

    For a Data Set with Data Records requiring microsecond or
    nanosecond precision, the IPFIX Message Header "Export Time"
    field SHOULD be calculated so that each time related Information
    Elements of the Data Records would contain a 32 bit positive
    offset from the "Export Time" base timestamp.

    For a Data Set with Data Records requiring microsecond or
    nanosecond precision, an example of the "Export Time" calculation
    is the following: The "Export Time" is set to the second
    portion of the first time related Information Element of the
    Data Set. Each subsequent Information Elements containing
    timestamps are translated into offsets. If an offset
    is negative or does not fit into 32 bit, the relevant Data
    Record must be exported in a subsequent IPFIX message.

    The Flow Records time precision will be implied from the Information
    Element definitions [IPFIX-INFO].

-------------------------------------------------->8

changes:
- Flow Record changed to Data Record
- allow export of microsecond and nanosecond precision IEs in
   the same ipfix message
- make section more generic via removing terms like
   flowStartDeltaUsec, flowEndDeltaUsec etc.

Best Regards,
Lutz

--
Help        mailto:[email-masked] and say "help" in message body
Unsubscribe mailto:[email-masked] and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/

---

From: Sebastian Zander <[email-masked]>
Date: Wed, 09 Mar 2005 13:03:19 +1100

Hi Benoit,

some comments on the draft up to and including section 10. i hope
i have time to read the rest tomorrow...

Section 3:
"Should there be any apparent discrepancy in definitions between these two
documents, the definitions defined in this document take precedence."

we must make sure that the same terms have the same defintion in both documents.
shouldn't be too difficult.

Defintion of Information Element
fields and scope fields are both attributes (terminology issue)? Flow Record
should be replaced by Data Record (even with the current defintion)

Section 3.1:
this section needs to be changed anyway. the text should be more concise
e.g. "A Template Set is composed of Template Record(s).  No Flow or Options
Data Record is included." should be replace by "A Template Set contains only
Template Record(s) [and no other records]."

Section 4:
4.1 seems off topic.
4.2 describes the behaviour of the exporting process and seems relevant for
implementers. however, it only uses should instead of SHOULD etc. so is it
supposed to be part of the spec or purely informative? in the later case it
could be removed, especially if it is in ipfix-arch.

Section 5 and 6:
why do we need 2 different sections 'Message Format' and 'IPFIX Message Format'?
integrate the 2 sections.

Section 5:
first paragraph below figure B: should be removed because the set ID is defined
in a later section.

"The format of the Data, Template and Options Template Sets will be
discussed later in this document."
i would remove that sentence, if you really wanna keep it add 'message
header' and the section numbers where the different things are actually
specified.

"The Exporter MUST code all binary ...
remove that sentence because this is explained in detail in section 9.

Section 6:

Message Header Field Descriptions, Length: 'message Header' -> 'Message Header'

Section 6.2, 1st paragraph: ", both the Template Set and the Option Template
Set." remove that part of the sentence

Section 6.2, 2nd paragraph: "The Field Ids used to identify Information Elements
are represented by the Field Type." i propose to replace Field Type by Field Id in
the protocol draft. having two terms for the same thing is confusing and field
type could be easily confused with the data type (ipfix-info) e.g.
"A numeric value that represents the type of the Information Element.  Refer to [IPFIX-INFO]."
Section 9 also uses type with the meaning of data type.

rename to 'Enterprise Bit'. Enterprise Field Type would be a Field Type with the
E bit set.

Field Length: replace 'bytes' by 'octets'. add: The field length may
be smaller than the defintion in ipfix-info if reduced size encoding is used (see section
9.2).

Section 6.3.2:

simplify and fix the length definition:

Total length of the Set in octets including the Set Header, all records and the
optional padding.  Because an individual Set MAY contain multiple records, the Length
value MUST be used to determine the position of the next Set.

Section 6.4:

heading must be changed to 'Record Formats'

"Templates greatly enhance the flexibility of the record
format because they allow the Collecting Process to process records
without necessarily knowing the interpretation of all the data in
the record."
'process' sounds misleading. the record could be received and stored but
it can't be really used without knowing the template defintion.

"A Template Record MAY exclusively contain IETF defined
Information Elements.  A Template Record MAY contain Enterprise
Specific Information Elements from one or more vendors.  A Template
Record MAY contain IETF and Enterprise Specific Information
Elements."
can we make it simpler? A Template Record can contain any combination
of IETF and Enterprise Specific Information Elements.

"There are no constraints regarding the order the Template ID allocation."
... the order _of_ the ...

again, why do we need a separate defintion for template records? an options
template with scope field count = 0 is basically a template record so the
defintion is redundant.

Section 6.4.2.1

can you please clarify if the scope fields are within the same field id
space as the information elements. if not where are the scope ids defined?

for some of the fields listed i cannot find a definition in ipfix-info e.g.
TemplateId. where are these fields defined?

Section 6.4.2.2

can we make the 1st paragraph simpler (see above)?

Section 8

"contain a 32 positive bit signed microsecond offset". what does that mean?

i don't understand why you want to change the meaning of Export Time? sounds
like a hack :) why can't Export Time still be the Export Time as defined in
section 6 and the micro second offsets are negative offsets (coded as positive
integers)?

Section 9.1

strings should use the length encoding as specified in section 10. is there an
octet/character array type?

Cheers,

Sebastian

Benoit Claise wrote:
> - Corrected the source ID definition: remove Process in "Exporter Process Observation Domain"
>        Description: The positive time offset of the first packet of this
>     flow
>        Description: The positive time offset of the last packet of this
>     flow relative
>    flowEndDeltaUsec [IPFIX-INFO] would contain a 32 bit positive

-- 
Sebastian Zander
http://caia.swin.edu.au

---

From: Brian Trammell <[email-masked]>
Date: Wed, 9 Mar 2005 10:39:10 -0600

On Mar 8, 2005, at 7:00 AM, Lutz Mark wrote:

>>     valid anymore because the offset could also be negative.     The 
>> following new text was introduced to cover the 2 phases approach
>

As I read ipfix-info, we can't do negative offsets because we don't 
have signed integers.

In addition, negative offsets complicate the implementation of analysis 
systems that index incoming ipfix streams by message export time 
(you're still doing two passes, you're just shifting the second pass 
over to the analysis application). However, the separate message 
solution seems like a good compromise...

Regards,

Brian

--
Brian H. Trammell / MTS, CERT/NetSA / <[email-masked]> / 412.268.9748

---

From: Simon Leinen <[email-masked]>
Date: Wed, 09 Mar 2005 21:40:23 +0100

Lutz Mark writes:
[Benoit had written:]
>> => This implies that the method called the "one-pass approach" is
>> not valid anymore because the offset could also be negative.

A one-pass approach is possible even if you mandate non-negative
relative timestamps, as long as you don't (as -09 unfortunately does)
mandate that the base time in the message header is EQUAL TO the
minimum of the (flow start) timestamps over all data records in the
message.

When the exporter starts a new IPFIX message (i.e. the first flow is
queued for export in a fresh message, it can set the base time by
estimating the lower bound for the oldest timestamp that can still be
in present in the flow cache.  In practice, it is easy to compute a
good and safe estimate for this based on flow expiry configuration.

Alternatively, you can derive the base timestamp if you can estimate
the HIGHEST timestamp that may occur in the new message, for example
by limiting the time between starting a new message and exporting it.
You can then simply use that maximum time minus the ~71 minutes (2*32
microseconds) as your base timestamp.

> bit easier. I prefer not to limit the offset to positive values.

As I said, I claim you can have both (one-pass implementation and only
non-negative relative timestamps).  So I don't think what you propose
is useful.  Note that I cannot relate to the application of relative
timestamps for nanosecond resolution, as you'd only have a window of
four seconds (2*32 nanoseconds) for all the timestamps in a message in
that case.

Regards,
-- 
Simon.

---

From: Lutz Mark <[email-masked]>
Date: Wed, 09 Mar 2005 22:15:20 +0100

Hi Simon,

> microseconds) as your base timestamp.

nice idea, in this case you are right. But I have an exporting process
in mind, which is separated from the metering process and which has no 
access to any data (like queues or flow timestamps) of the metering
process. Especially your method does not work if we want to allow the
export of non-flow data via IPFIX.

Best regards,
Lutz

---

From: Sebastian Zander <[email-masked]>
Date: Thu, 10 Mar 2005 12:20:28 +1100

Hi Benoit,

more comments:

Section 11:

"The Template Withdraw Message MUST not be sent until sufficient time
has elapsed to allow the Collecting Process to receive and process
the last data record using this Template information."

"The Template ID from a withdrawn Template MUST NOT be reused until
sufficient time has elapsed to allow for the Collecting Process to
receive and process the Template withdraw message."

how would an exporter be able to estimate this especially if different
exporters export to the same collector? any recommendations for
implementors?

Figure T:
from the text I understand that the message can contain multiple
<template ID, field count=0> lines but this is not shown in the figure.

this section combines the template management messages with some SCTP
transport issues. i'd prefer the move the transport specific stuff into
the SCTP transport section e.g. "If the Exporting Process restarts, the
SCTP association MUST be shutdown and restarted." and let this section
explain template management only. a few things in this section are not
understandable if one hasn't read the transport section first e.g."
"SCTP stream zero".

"More that one (Option) Template Set MAY be sent in an IPFIX Message."
that is already stated in section 5/6

Section 12:

can this be separated into generic and SCTP transport requirements
that go in the transport section?

..., and SHOULD log the error_._

"The Collecting Process MUST note the Field ID of any Information
Element that it does not understand and MAY discard that Information
Element from the Flow Record.

now Field ID has started to replace Field Type. good idea but it should
replace it everywhere.

i guess Flow Record should be Flow Data Record. having both terms is confusing
and a good argument against having Flow Data Record.

"The Collecting Process MUST note the size and position of any Vendor
Specified Information Element that ..."

maybe these are dumb questions: what is the position (byte offset in the message,
record,?)? why don't i have to record the same for unknown Field IDs/Types?

"More that one (Options) Template Set MAY be received in an IPFIX
Message."
"The Collector MUST accept padding in Flow Data Records, Options Data
Records and Template Records."

what does that mean? per definition ipfix messages can have more than one template
set and padding is well also defined. obviously a collector must be able to decode
properly formatted ipfix messages. a collector MUST be able to process more than one
(options) template set in a message (MAY???).

"... increases with the number of IPFIX data records in _the_ IPFIX Message."

Section 13

"The IPFIX Message Header 16-bit LENGTH field limits the length of a
IPFIX Message to 65536 octets including the header.  A Collecting
Process MUST be able to handle IPFIX Message lengths of up to 65536
octets."

65536 -> 65535, LENGTH -> Length

Section 13.2.4.1

"By default, the Collecting Process listens for connections on SCTP port XXXX (EDITOR NOTE: to be
assigned by IANA)."

and by default the exporting process tries to connect to this port.
(same comment for other transports)

Section 13.2.4.2

"When an Exporting Process has no more IPFIX Messages to send, it
SHOULD shutdown the SCTP association."

does it mean when the exporter is shutdown? or when the exporters doesn't get new flow data for a while?
if it includes the later it should mention a timeout...
(same comment for TCP)

Section 13.2.4.3

what is the purpose of this section? it repeats the defintion of an ipfix message and i can't see
any relation to SCTP. remove it.

Section 13.2.4.4

could the collector be configured to require a certain mode based on the application requirements?
because you usually would have more exporters than collectors that could make it easier to detect
configuration errors...

Section 13.2.4.5 and Section 13.2.5

i'd like to separate the generic and SCTP specific requirements

Section 13.3.6

"Note that following a configuration change a new Template ID should
be used and the old Template ID SHOULD NOT be reused until its
lifetime has expired."

the concept of template lifetimes hasn't been explained so far. isn't this a MUST NOT?

"Template Withdraw Messages SHOULD NOT be sent over UDP."

isn't this a MUST NOT?

Section 13.3.7

exporter and collector should be configured with the same refresh timeout / lifetime

Section 13.3.7

"At any given time the Collecting Process SHOULD maintain the
following for all the current Template Records and Options Template
Records: <Exporting Process, Observation Domain, Template ID,
Template Definition, Last Received>."

Observation Domain -> Source ID, Exporting Process means the IP address or DNS name or?

Section 13.4.2.1

LENGTH -> Length

"If an Exporting Process exports data from multiple Observation
Domains, it should be careful to choose IPFIX Message lengths
appropriately to avoid head-of-line blocking between different
Observation Domains."

TCP cannot avoid hol blocking on one connection only minimize it.
the recommendation is too use short lengths (MSS) or?

multiple tcp connection may be used to avoid hol blocking between
different observation points.

Section 14.4

"UDP is vulnerable to insertion attacks, however, randomization of the IPFIX
Sequence Number might mitigate this problem."

randomization of the _initial_ ipfix sequence number. otherwise we loose the
ability to detect lost records.

"Therefore a 64 bit cookie [L2TPv3] SHOULD be included
  as an element within all messages."

the cookie is not defined yet. it could be an extension of the header or
a one element data set in all messages (in that case it would have to be added
to the info model).

Cheers,

Sebastian

-- 
Sebastian Zander
http://caia.swin.edu.au

---

From: Benoit Claise <[email-masked]>
Date: Thu, 17 Mar 2005 16:55:34 +0100

Hi Sebastian,

Thanks for the review.
See inline.

> shouldn't be too difficult.

It's the case.

> should be replaced by Data Record (even with the current defintion)

done.

> Template Record(s) [and no other records]."

Done. The new section is now:

A Data Set is composed of Data Record(s).  No Template Record is 
included.  A Template Record or an Options Template Record defines the 
Data Record.

A Template Set contains only Template Record(s). 

An Options Template Set contains only Options Template Record(s). 

> could be removed, especially if it is in ipfix-arch.

Remove in this draft. Kept only in IPFIX-ARCH

> integrate the 2 sections.

Done.

> first paragraph below figure B: should be removed because the set ID 

Done.

> discussed later in this document."

Done.

> remove that sentence because this is explained in detail in section 9.

The section 9 is only related to Information Elements.

> Message Header Field Descriptions, Length: 'message Header' -> 

Done.

> Set." remove that part of the sentence

Done.

> the protocol draft.

This has been improved in version 10. Please have a look at the proposal.

> having two terms for the same thing is confusing and field
> Field Length: replace 'bytes' by 'octets'. 

done.

> add: The field length may

Done.

> value MUST be used to determine the position of the next Set.

Done.

> heading must be changed to 'Record Formats'

Well spot. Done.

> it can't be really used without knowing the template defintion.

What do you suggest?

> of IETF and Enterprise Specific Information Elements.

A Template Record contains any combination of IANA-assigned and/or 
enterprise-specific Information Elements identifiers.

> template with scope field count = 0 is basically a template record so the

Option Template has got the notion of scope. Personally I prefer to make 
a clear distinction.

> space as the information elements. if not where are the scope ids 

Isn't it clear from the text below?

   Note that other Information Elements such as 
   meteringProcessId, exportingProcessId, sourceId, etc. are also valid 
   scopes. The IPFIX protocol doesn't prevent the use of any 
   Information Elements for scope. However some Information Element 
   types don't make sense if specifed as scope. Example: the counters 
   Information Elements. 

> TemplateId. where are these fields defined?

This was done in the interim release of IPFIX-INFO that Juergen posted.

> can we make the 1st paragraph simpler (see above)?

Any suggestion?

> section 6 and the micro second offsets are negative offsets (coded as 

I agree with you:

Let me explain the history of this change.
In IPFIX-INFO, there was only unsigned32 and not signed32. As a 
constraint, the deltaTime for flow start and flow end had to be 
positive. As a consequence, the Export Time had to be lower than or 
equal to the lowest flow start time.
I made one mistake in the change: I added positive and forgot to remove 
signed :)

There is no opposition to http://ipfix.doit.wisc.edu/archive/2676.html 
so far...
Sebastian, do you agree with this?
Time to speak up for the rest of the WG... we are in last-call!

> octet/character array type?

Yes, added:

   string and octetArray

   The data type string represents a finite length string of valid 
   characters of the Unicode character encoding set. It is expected 
   that strings will be encoded in UTF-8 format. The string is send as 
   an array of octets. The length of the string and the octetArray is
   encoded as described in Section 9 (Variable Length Information
   Element). The length is followed by that many octets as encoded
   in the length.

Thanks for the valid feedback.

Regards, Benoit.

---

From: Lutz Mark <[email-masked]>
Date: Thu, 17 Mar 2005 17:46:33 +0100

Hi Benoit,

>> section 6 and the micro second offsets are negative offsets (coded as 

this does not work
- if the the offset does not fit into 32bit or
- if there are two timestamps to export in the same record, which
    differ more than 32Bit.

Like Simon already mentioned 32Bit means about 4.3 seconds for
nanosecond resolution and about 4300 seconds for microsecond
resolution.  So it is not unlikely.

Do we have a solution for that cases without using additional templates?

Best regards,
Lutz

---

From: Benoit Claise <[email-masked]>
Date: Fri, 18 Mar 2005 16:57:04 +0100

Sebastian,

Thanks again for the constructive feedback.
See inline.

> exporters export to the same collector? 

 A Template ID MUST be unique per Observation Domain. 

> any recommendations for

No real guideline what the sufficient time should be. A few seconds?

Done.

> SCTP association MUST be shutdown and restarted." 

In order to remove the spec. part of the sentence (MUST), I changed it to:
When the Exporting Process restarts, due the SCTP association shutdown, 
all Template assignments are lost and Template IDs MUST be re-assigned.

> and let this section

Removed.

> that go in the transport section?

The agreement was: the "template management" and the "collecting process 
side" sections will only discuss SCTP (the default protocol).
UDP and TCP, as the exceptions, will be restricted to their own sections.

Done.

> now Field ID has started to replace Field Type. good idea but it should

Information Element identifier is the right term now.

> record,?)? why don't i have to record the same for unknown Field 

Agreed.

I don't recall where the position comes from. Maybe someone else will!
Now, the paragraph says:

The Collecting Process MUST note the Information Element identifier of 
any Information Element that it does not understand and MAY discard that 
Information Element from the Flow Record. 

> "More that one (Options) Template Set MAY be received in an IPFIX

Removed.

> Records and Template Records."

Padding MAY be inserted by the exporter, but MUST be supported by the 
collector.

done.

Done.

> (same comment for other transports)

Done for the 3 transport protocols

> if it includes the later it should mention a timeout...

First one.
New proposal: When an Exporting Process is shutdown, it SHOULD shutdown 
the SCTP association. 

> any relation to SCTP. remove it.

Ok. I move the sentence "The Exporting Process uses the Source ID to 
uniquely identify to the Collecting Process the Observation Domain that 
metered the Flows. " to the source ID definition,  as  it helps 
understand the  concept.

> could make it easier to detect

That could be nice, but I think this is out of the scope of the export 
protocol itself.

> i'd like to separate the generic and SCTP specific requirements

As explained before, this was not the WG agreement.

> the concept of template lifetimes hasn't been explained so far. 

Forward reference added.

> isn't this a MUST NOT?

It depends on what is changed.
If you change the sampling rate, this is a MUST
If you change the flow expiration policy, it's not sure.
So I think a SHOULD is appropriate.

> "Template Withdraw Messages SHOULD NOT be sent over UDP."

What if you have only UDP, it would mean that you can use template 
withdraw messages?

> exporter and collector should be configured with the same refresh 

Couldn't you say:
I export my template every minute and my lifetime on the collector is 3 
minutes

> Observation Domain -> Source ID, Exporting Process means the IP 

Source ID.
New text: : <Exporting Process, Observation Domain Source ID, Template ID,
Template Definition, Last Received>."

Done.

"If an Exporting Process exports data from multiple Observation
Domains, it should be careful to choose IPFIX Message lengths
appropriately to _minmize _head-of-line blocking between different
Observation Domains."
Fine?

Done.

> a one element data set in all messages (in that case it would have to 

See the post from Stewart Bryant this March 18th.

Thanks for the constructive feedback.

Regards, Benoit.

>> If you haven't started yet reviewing 
>> draft-ietf-ipfix-protocol-08.txt, start with 
>> draft-ietf-ipfix-protocol-09.txt as this is much more complete
>>   Thomas Dietz rewrote the full sections 5 and 6.   It now contains 
>> generic format + separate examples.25 <#_Toc97709690>
>> - corrected a typo, reported by Mark Lutz "an Exporting Process MUST 
>> NOT   attempt to establish a connection more frequently than once per 
>> minute. "
>> - improved examples:
>>   * there were inconsistencies between hexa and non hexa value   * 
>> [IPFIX-INFO] Information Elements ID are now used   * the example now 
>> uses the lineCardId: PROTO-38 is closed: - new section 9.1 specifying 
>> the data types encoding. This was missing for time-related based 
>> types, for example.
>> - the "time" Information Element in the specific Options Templates 
>> has been clarified. - Integration of the changes proposed in my post 
>> "some more info regarding the scope in [IPFIX-INFO]"
>>    the IPFIX Message Header "Export Time" field SHOULD be calculated 
>> so    that each Flow Records flowStartDeltaUsec [IPFIX-INFO] and    
>> exporting the Flow Records, the Exporting Process takes as "Export 
>> Time" the lowest start time of all the Flow Records to be exported in 
>> this Data Set. The only constraint is that the highest end time of 
>> the Flow Records in that Data Set can not exceed the "Export Time" 
>> plus 71 minutes. If a Flow Record end time exceeds this condition, 
>> this Flow Record must be exported in a subsequent IPFIX Message.
>>
>>  Export Time            Time in seconds since 0000 UTC 1970, at which 
>> the IPFIX            Message Header leaves the Exporter. 
>> Alternatevily, the            Export Time can be a computed value: 
>> see the sectin 8
>>   PROTO-100: Flow Data Records would become Data Records (consensus 
>> on    the mailing so far?)     
>>    |                  |                    Contents                 
>> |     |                  
>> +--------------------+------------------------+     |       
>> Set        |       Template     |        Record          |     
>>    | Options Template | Options Template   |           /            
>> |     |       Set        |     Record(s)      
>> |                        |     
>>      PROTO-101: Section 9.1.7 dateTimeMicroSeconds -> "Its encoding 
>> is      not defined yet."            PROTO-102: Duplicate chapter 
>> "the text from chapter 4 "Criteria for      Flow Expiration and 
>> Export" with [IPFIX-ARCH]. Erase this one?
>> Regards, Benoit.
>>  
>
>

---

From: Sebastian Zander <[email-masked]>
Date: Wed, 23 Mar 2005 10:33:56 +1100

some comments back

>> shouldn't be too difficult.

is there a good reason to have different defintions? isn't this a bit
confusing?

> The section 9 is only related to Information Elements.

and it explains their encoding!

>> it can't be really used without knowing the template defintion.

how about "... the Collecting Process to process IPFIX messages without
necessarily knowing the interpretation of all data records."

> enterprise-specific Information Elements identifiers.

sounds good

> Option Template has got the notion of scope. Personally I prefer to make 

i think scope it a more general concept. currently it is closely tied to
"option data" but the notion of 'option' is very ipfix specific. if i'd want
to send data with ipfix that require the use of scope i'd have to use option
templates even if my data has nothing to do with 'options'.

>    types don't make sense if specifed as scope. Example: the counters 

is it somewhere explicitly stated that scope fields are information elements?

> Sebastian, do you agree with this?

i certainly agree to Nevil's proposal. 'm confused, is your proposal to
entirely remove delta timestamps?

> Time to speak up for the rest of the WG... we are in last-call!

cheers,

Sebastian

-- 
Sebastian Zander
http://caia.swin.edu.au

---

From: Sebastian Zander <[email-masked]>
Date: Wed, 23 Mar 2005 11:43:51 +1100

Hi Benoit,

some comments back

> No real guideline what the sufficient time should be. A few seconds?

well at least the network delay (could be a few hundred ms) plus some processing
delay). a few seconds probably.

> UDP and TCP, as the exceptions, will be restricted to their own sections.

i don't understand why these sections discuss transport issues when there
is a later section on SCTP transport. well, if this was wg consensus i can
live with that.

sounds good

> New proposal: When an Exporting Process is shutdown, it SHOULD shutdown 

ok

> metered the Flows. " to the source ID definition,  as  it helps 

ok

> That could be nice, but I think this is out of the scope of the export 

agreed

> So I think a SHOULD is appropriate.

i would make it a MUST then or discuss the different cases. because if you
make it a SHOULD only it means that e.g. in case the sample rate changes a
compliant application could reuse the id before the lifetime expires. but
this should not happen i guess.

> What if you have only UDP, it would mean that you can use template 

if i understand correctly UDP requires the lifetime mechanism and the benefit
of using template withdraw messages is rather low (implied by the SHOULD NOT).
i agree that you still can send them but the Exporter MUST NOT rely on them
because they may get lost.

> I export my template every minute and my lifetime on the collector is 3 

that works, but would it make sense to have a (much) smaller lifetime at the
exporter (reasons)? the opposite (smaller lifetime at collector) would
result in problems wouldn't it? i thought that the draft should be more
verbose about these issues.

> appropriately to minmize head-of-line blocking between different
> Observation Domains."
> Fine?

so what's the actual recommendation for choosing the message lengths?

>> different observation points.

i propose to add the above sentence (may -> MAY).

cheers,

Sebastian

-- 
Sebastian Zander
http://caia.swin.edu.au

---

From: Benoit Claise <[email-masked]>
Date: Wed, 23 Mar 2005 13:26:37 +0100

Hi Sebastian,

> is there a good reason to have different defintions? isn't this a bit

Nevil,
Regarding this sentence "Should there be any apparent discrepancy in 
definitions between these two documents, the definitions defined in this 
document take precedence.", I think that all the terms defined in the 
terminology section of [IPFIX-ARCH] are exactly the same as [IPFIX-PROTO].
Do you confirm?
If this is the case, we don't need this sentence.

> and it explains their encoding!

The sentence "The Exporter MUST code all binary integers of the Message 
Header and the different Sets in network byte order (also known as the 
big-endian byte ordering)." applies to everything.
The section "Linkage with the Information Model" contains the sentence 
"The Information Elements [IPFIX-INFO] MUST be sent in canonical format 
in network byte order." that is targetted specifically as Information 
Elements.
Is this a problem for you?

Done.

>> Option Template has got the notion of scope. Personally I prefer to 
>> make a clear distinction.
>
>

I thought initially that you wanted combine the Options Template Set and 
Template Set into a single Set, which have a scope length of 0 if 
appropriate.
Now I'm not sure anymore. Practically, what do you suggest?

>>    Note that other Information Elements such as    meteringProcessId, 
>> exportingProcessId, sourceId, etc. are also valid    scopes. The 
>> IPFIX protocol doesn't prevent the use of any    Information Elements 
>> for scope. However some Information Element    types don't make sense 
>> if specifed as scope. Example: the counters    Information Elements.
>
>

I now added the first sentence in this paragraph:

The scope is an Information Element specified in the IPFIX Information 
Model [IPFIX-INFO]. An IPFIX compliant implementation of the Collecting 
Process SHOULD support this minimum set of Information Elements as 
scope: LineCardId, TemplateId, exporterIPv4Address, exporterIPv6Address, 
and ingressInterface.  Note that other Information Elements such as 
meteringProcessId, exportingProcessId, sourceId, etc. are also valid 
scopes. The IPFIX protocol doesn't prevent the use of any Information 
Elements for scope. However some Information Element types don't make 
sense if specifed as scope. Example: the counters Information Elements.

Fine with you?

>> I made one mistake in the change: I added positive and forgot to 
>> remove signed :)
>>
>> There is no opposition to 
>> http://ipfix.doit.wisc.edu/archive/2676.html so far...

No. The IPFIX implementors will decide whether they need/want delta or 
absolute timestamps.

Regards, Benoit.

---

From: Benoit Claise <[email-masked]>
Date: Wed, 23 Mar 2005 15:42:00 +0100

Sebastian,

> delay). a few seconds probably.

We agree.
However, I think there is always a danger to specify arbitrary values 
like this in a specification.

> this should not happen i guess.

What about something such as:
"Following a configuration change that can modify the interpretation of 
the Data Records (for example, a sampling rate change) a new Template ID 
MUST be used and the old Template ID MUST NOT be reused until its 
lifetime has expired."

> of using template withdraw messages is rather low (implied by the 

Yes.

> because they may get lost.

What if we have UDP and a directly connected collector for which we are 
sure that we have enough bandwidth?
Do we want to close that possibility by saying:  "Template Withdraw 
Messages MUST NOT be sent over UDP." or "Template Withdraw Messages 
SHOULD NOT be sent over UDP. The Exporter MUST NOT rely on the Template 
Withdraw Messages sent over UDP."

The way I interpret the SHOULD NOT is: don't do it unless you have a 
good reason.
This is why I tend to prefer the initial wording: "Template Withdraw 
Messages SHOULD NOT be sent over UDP."

>> I export my template every minute and my lifetime on the collector is 
>> 3 minutes
>
>

I reorganized the section and added the sentence "The Template lifetime 
at the Collecting Process MUST be at least 3 times higher that the 
Template refresh timeout configured on the Exporting Process."

The Collecting Process MUST associate a lifetime with each Template 
received via UDP.  Templates not refreshed by the Exporting Process 
within the lifetime are expired at the Collecting Process.  If the 
template is not refreshed by the Exporting Process before that lifetime 
has expired, the Collecting Process MUST discard the Template, and any 
current and future associated Data Records.  In which case, an alarm 
MUST be logged.  The Collecting Process MUST NOT decode any further Data 
Records which are associated with that expired Template.  The Template 
lifetime at the Collecting Process MUST be at least 3 times higher that 
the Template refresh timeout configured on the Exporting Process. 

At any given time the Collecting Process SHOULD maintain the following 
for all the current Template Records and Options Template Records: 
<Exporting Process, Observation Domain Source ID, Template ID, Template 
Definition, Last Received>.

The Collecting Process SHOULD accept Data Records without the associated 
Template Record. If the Template Records have not been received at the 
time Data Records are received, the Collecting Process SHOULD store the 
Data Records for a short period of time and decode them after the 
Template Records are received.  The short period of time MUST be lower 
than the Template lifetime.

Feedback?

> so what's the actual recommendation for choosing the message lengths?

Simon, as author of this part of the draft, any feedback?

> i propose to add the above sentence (may -> MAY).

Do you mean: "Multiple TCP connections MAY be used to avoid head-of-line 
between different Observation _Domains_"?

Regards, Benoit.

---

From: Sebastian Zander <[email-masked]>
Date: Fri, 25 Mar 2005 10:11:08 +1100

Hi Benoit,

> Now I'm not sure anymore. Practically, what do you suggest?

yeah, my suggestion is to combine both definitions because de-facto the
template set is already a subset of the option template set (scope length 0).
the only advantage i see with the current 2 definitions is that you save 2
bytes in flow templates, which is not very much (especially with SCTP) for
effectively doubling the spec.

the notion of 'option templates' (the only templates that have scope) is not
very general.

is the flow keys option template an attempt to introduce scope to flow
templates? if yes (the current text in section 7.4 is a bit hard to understand
without having the field defintions) wouldn't it be much simpler to have scope
for all templates?

> sense if specifed as scope. Example: the counters Information Elements.
looks good

cheers,

Sebastian

-- 
Sebastian Zander
http://caia.swin.edu.au

---

From: Sebastian Zander <[email-masked]>
Date: Fri, 25 Mar 2005 10:28:27 +1100

Hi Benoit,

> like this in a specification.

on the other hand the danger of not providing any information may result
in problems, especially interoperability.

it might be good then to require such parameters to be configurable.

sounds good

> Messages SHOULD NOT be sent over UDP."

in my opinion you have 2 cases: either you assume 100% reliability over UDP
and than why do use the lifetime mechanism at all? or you assume possible
packet loss and then the exporter can never be sure if template withdrawal
messages arrive at the collector. so yes the exporter can send them and maybe
there are some benefits for the collector but the exporter must use the
lifetime mechanism.

> between different Observation Domains"?

yes

cheers,

Sebastian

-- 
Sebastian Zander
http://caia.swin.edu.au

---

From: Benoit Claise <[email-masked]>
Date: Fri, 25 Mar 2005 16:23:18 +0100

Dear all, :

>> I thought initially that you wanted combine the Options Template Set 
>> and Template Set into a single Set, which have a scope length of 0 if 
>> appropriate.

I would like to ping the rest of the working group regarding this issue.
Should we combine the Option Template Set and Template Set into a single 
Set? 

> without having the field defintions) wouldn't it be much simpler to 

The point is that you only need to send this flow keys option template 
if you want to know the flow keys associated with all the information 
elements in the data record.
Even if you would have a combined Option and Template Set, this would 
not be simplified what you would send:
    - you would still need a template for the data record
    - you would still need a template for the flow key
unless you want to send the flow key in every data record -> this is not 
a good idea!

Regards, Benoit.

---

From: Sebastian Zander <[email-masked]>
Date: Sun, 27 Mar 2005 13:08:29 +1000

> unless you want to send the flow key in every data record -> this is not 

well, my assumption was that the flow keys option template is always needed
because what can i do with the rest of the information without the flow key?
the flow key might not be needed if there are rules known by the exporter and
collector and something in the template would map to the rule(s) (is there
anything like a rule id in the info model?). the current draft does not really
explain these issues.

if you have to export the flow key information for each flow anyway you might
be better off to export it in the data record e.g. if there are lots of short-lived
flows that generate only one record. if you only have long flows and short interim
report intervals it might not be a good idea. depends on the application...

cheers,

Sebastian

-- 
Sebastian Zander
http://caia.swin.edu.au

---

From: Benoit Claise <[email-masked]>
Date: Wed, 30 Mar 2005 17:01:18 +0200

Sebastian,

"The Template Withdraw Message MUST not be sent until sufficient time
has elapsed to allow the Collecting Process to receive and process
the last data record using this Template information. This time MUST be 
configurable"

>> What if we have UDP and a directly connected collector for which we 
>> are sure that we have enough bandwidth?
>> SHOULD NOT be sent over UDP. The Exporter MUST NOT rely on the 
>> Template Withdraw Messages sent over UDP."
>>

Isn't it exactly what we are saying with the sections:
 "10.3.6 Template Management"

Following a configuration change that can modify the interpretation of 
the Data Records (for example, a sampling rate change) a new Template ID 
MUST be used and the old Template ID MUST NOT be reused until its 
lifetime (see section 10.3.7) has expired.               

Template Withdraw Messages SHOULD NOT be sent over UDP.           

 "10.3.  Collecting Process"

   The Collecting Process MUST associate a lifetime with each Template received via UDP.  

So If I translate your last sentence from  "so yes the exporter can send 
them and maybe there are some benefits for the collector but the 
exporter must use the lifetime mechanism." to:
" so yes the exporter SHOULD NOT send them (maybe there are some 
benefits for the collector, so that case you might send them) but the 
exporter/collector MUST anyway use the lifetime mechanism.

>> Do you mean: "Multiple TCP connections MAY be used to avoid 
>> head-of-line between different Observation Domains"?
>
>
> yes 

added

Regards, Benoit.

---

From: Benoit Claise <[email-masked]>
Date: Wed, 30 Mar 2005 17:41:03 +0200

Hi Sebastian,

>>>>>> Option Template has got the notion of scope. Personally I prefer 
>>>>>> to make a clear distinction.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>> I thought initially that you wanted combine the Options Template 
>>>> Set and Template Set into a single Set, which have a scope length 
>>>> of 0 if appropriate.
>> Should we combine the Option Template Set and Template Set into a 
>> single Set?
>>
>>>
>> The point is that you only need to send this flow keys option 
>> template if you want to know the flow keys associated with all the 
>> information elements in the data record.
>> unless you want to send the flow key in every data record -> this is 
>> not a good idea!
>
>
There is no rule ID in the information model.

Other reasons why you might not need to export the flow keys:
- The flow keys are obvious. If you send <input ifIndex, number of 
bytes, number of packets> the flow key is obvious. Even if you send 
<input ifIndex, dest prefix, number of bytes, number of packets>, it 
sounds reasonable to assume that flow keys are input ifIndex and 
destination prefix.
- There is only one set of flow keys defined in your entire network. For 
example <input ifIndex, src IP address, dest IP address, src L4 port, 
dest L4 port, DSCP, protocol> as specified in NetFlow.
My point is that all Template Records don't need to send their 
associated flow keys.

> report intervals it might not be a good idea. depends on the 

You lost me somewhere...
First of all, I encourage to send the flow keys in an Option Template, 
sent just once per Template definition.
However, the IPFIX protocol doesn't prevent you to have a Template that 
contains the flow-key information element. In this case, the flow-key 
information element will be sent part of every data records. A waste of 
bandwidth if you ask me.
Nothing to do with short or long flows, but with the flow 
classification: i.e. the flow keys that classified the metered packets, 
i.e. the Template definition

Regards,  Benoit.

---

From: Benoit Claise <[email-masked]>
Date: Wed, 30 Mar 2005 22:36:41 +0200

Hi Sebastian,

> definitions between these two documents, the definitions defined in 
> this document take precedence.", I think that all the terms defined in 
> the terminology section of [IPFIX-ARCH] are exactly the same as 
> [IPFIX-PROTO].
> Do you confirm?

As [IPFIX-ARCH] and [IPFIX-PROTO] have now the same terminology section, 
the sentence "Should there be any apparent discrepancy in definitions 
between these two documents, the definitions defined in this document 
take precedence." has been removed.

Regards, Benoit.

---

From: Robert Lowe <[email-masked]>
Date: Wed, 30 Mar 2005 15:12:03 -0600

> take precedence." has been removed.

Forgive the question... if they are both the same, then is it necessary
to have two of them?  Can't there just be a forward pointer to the other.
That way if changes are made, noone has to remember to make updates in
two places.  Is there a requirement I'm missing?  I know for NAT, a lot
of terminology was put into a separate RFC (RFC2663).

-Robert

---

From: Benoit Claise <[email-masked]>
Date: Thu, 31 Mar 2005 00:21:32 +0200

Hi Robert,

>>> this document take precedence.", I think that all the terms defined 
>>> in the terminology section of [IPFIX-ARCH] are exactly the same as 
>>> [IPFIX-PROTO].
>>> Do you confirm?
>> As [IPFIX-ARCH] and [IPFIX-PROTO] have now the same terminology 
>> section, the sentence "Should there be any apparent discrepancy in 
>> this document take precedence." has been removed.
>
>

Actually, I should have expressed myself at little bit differently.
The terminology sections are not quite similar as
- some terms are only defined in the [IPFIX-PROTO] terminology section: 
example, "Options Template Record"
- some terms are only defined in the [IPFIX-ARCH] terminology section: 
example, "Control Information, Data Stream"
However, when a term is defined in both drafts, you can be sure that the 
definitions are the same.

Regards, Benoit.

---

From: Nevil Brownlee <[email-masked]>
Date: Fri, 1 Apr 2005 04:20:47 +1200

> of terminology was put into a separate RFC (RFC2663).

The 'terminology' sections in 'protocol' and 'architecture' are similar,
but not identical.  The idea (mine, at least) is that 'architecture'
is a shorter list, with only enough of the terms defined for readers
to understand the 'architecture' draft.  The 'protocol' draft has the
full list, and the authoritative definitions (it's a standards-track
draft, 'architecture' is not).  Again, the 'architecture' versions
of the definitions have slightly-different workdings here and there,
aimed at making them (again, in my opinion) easier to read.  

As Benoit said, we've made sure there are no discrepancies in the
definitions between the two drafts, so maybe it's a bit pedantic to
keep the 'definitions in protocol document are definitive' statement.

Cheers, Nevil

-----------------------------------------------------------------------
   Nevil Brownlee                 Computer Science Department | ITSS
   Phone: +64 9 373 7599 x88941           The University of Auckland
   FAX: +64 9 373 7021      Private Bag 92019, Auckland, New Zealand

-------------------------------------------------
This mail sent through University of Auckland
http://www.auckland.ac.nz/