Subject: RE: [ipfix] 'Export Time' and deltaTime elements: proposal

From: "STEPHAN Emile RD-CORE-LAN" <[email-masked]>
Date: Tue, 22 Mar 2005 12:29:18 +0100

Hi Nevil and Benoit,

From my point of view "Export Time" is mandatory in the header; moreover templates and flow set definitions must not interfere with the meaning of the header of an IPFIX message.
    
Following is a more general proposal:
    
info.07-pre differs from info.06 mainly because it moves several abstract field types to field type (e.g. dateTimeMicroSeconds). 
    
The proposal merges this idea and the need of having the first record to carry absolute time while having the others to carry delta time.
    
It consists in defining sibling timestamp, and to describe in the IE 'Field Length' which type is used in the first data record and which one is used by the remaining records.

Details: (provided only as a support for verification and discussion)

 0                   1                   2                   3  
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1  
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 
|       Field Type              |  Rsv  | firstRecords|  others |  
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+  

firstRecords:
	hdr is 2 bits long;
	hType is 4 bits long;
	hLn is 3 bits long

'firstRecords' fields meaning: 
The field type of the 'hdr' first occurrences of this IE is the 'hType'th type of 'Field Type' definition. It is 'hLn'*4 bytes long.

'Others':
	rType is 4 bits long;
	rLn is 2 bits long;        

'Others' field meaning: 
The field type of the remaining occurrences of this IE is the 'rType'th type of 'Field Type' definition. It is 'rLn'*4 bytes long.

--------------------------------------------------------------------------

Example: A template and a data set using absolute and delta timestamp.

Info model: 

Field Type flowLastPacketTime: 
     	Field: ID 1022 
     	Types: flowStartMicroSeconds or flowStartDeltaUSeconds

Field Type flowFirstPacketTime:
     	Field: ID 1021 
     	Types: flowStartMicroSeconds or flowStartDeltaUSeconds    

Template:
       
 0                   1                   2                   3  
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1  
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+  
|         FlowSet ID = 0        |       Length = 22 bytes       |  
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+  
|               256             |       Field Length = 4        |  
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+  
|    1022 (flowFirstPacketTime) |   0   | 1 |  1  | 2 |  2  | 1 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+  
|   1021 (flowLastPacketTime)   |   0   | 1 |  1  | 2 |  2  | 1 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+  
|    8 (sourceIPv4Address)      |       Field Length = 4        |  
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 
|  12 (destinationIPv4Address)  |       Field Length = 4        |  
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+    

flowFirstPacketTime 'Field Length':

'firstRecords':  hdr '1', hType '1' and hLn '2':
'hdr' value sets to '1' means that only the first record has the field type flowEndMicroSeconds ('hType' value is 1) which is 2*4 bytes long ('hLn' value is '2').

'others': rType '2' and rLn '1':
'rType' value sets to '2' means that the remaining records type is flowEndDeltaUSeconds which is 1*4 bytes length ('rLn' value is '1').

The same applies for flowLastPacketTime.

Data set:

The first record one uses absolute timestamp, others use delta timestamp.
 
 0                   1                   2                   3  
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1  
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+  
|          Set ID = 256         |          Length = 32          |  
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+    
|                       4575688568345                           | absolute
+                                                               + micro
|                       7875224045765                           | seconds
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 
|                       192.181.17.7                            |  
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 
|                       192.181.17.18                           |  
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ delta
|                            6000                               | micro 
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ second
|                       192.181.17.17                           |  
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 
|                       192.181.17.19                           |  
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ delta 
|                              700                              | micro 
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ second
|                       192.181.17.25                           |  
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 
|                       192.181.17.28                           |  
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 

The example illustrates clearly the benefit of the proposal:
	 A data set combines absolute and delta times;
	 There is not side effect with the header of IPFIX message;
       The template format does not change;
       The protocol does not change;
       The encoding is as efficient as requested;

It applies for PSAMP too, which will carry so many timestamps!

Regards
Emile

-----Message d'origine-----
De : majordomo listserver [mailto:[email-masked]] De la part de Nevil Brownlee
Envoyé : samedi 19 mars 2005 18:07
À : Benoit Claise
Cc : [email-masked]
Objet : Re: [ipfix] 'Export Time' and deltaTime elements

Quoting Benoit Claise <[email-masked]>:

> What about this proposal?
> _IPFIX Message Header __"Export Time" and Flow Record Time _
> 
> The IPFIX Message Header "Export Time" field is the time in seconds
> since 0000 UTC 1970, at which the IPFIX Message Header leaves the
> Exporter.  The timing related Information Elements specified in
> [IPFIX-INFO] contain either the time since 0000 UTC 1970, or either a
> time offset compared to the IPFIX Message Header "Export Time".
> 
> For example, Data Records requiring a microsecond precision can export
> the flow start and end times with the absolute flowStartMicroSeconds and
> flowEndMicroSeconds Information Elements [IPFIX-INFO], containing the
> time since 0000 UTC 1970.  An alternate solution is to export the
> flowStartDeltaUSeconds and flowEndDeltaUSeconds Information Elements
> [IPFIX-INFO] in the Data Record, which respectively report the flow
> start and end time offsets compared to the IPFIX Message Header "Export
> Time".  The latter solution lowers the export bandwidth requirement
> while it increases the load on the Exporter as the Exporting Process
> must calculate the flowStartDeltaUSeconds and flowEndDeltaUSeconds of
> every single Data Records before exporting the IPFIX Message.
> 
> It must be noted that using time-related Information Elements with
> offset times compared to the IPFIX Message Header "Export Time" imposes
> some time constraints on the Data Records contained in the IPFIX
> Message.  In the example of flowStartDeltaUSeconds and
> flowEndDeltaUSeconds Information Elements [IPFIX-INFO], the Flow Record
> must be exported maximum 71 minutes after its creation. Otherwise, the
> 32 bits counters would not be sufficient to contains the flow start time
> offsets.

Yes, that's fine with me.

Cheers (and thanks), Nevil

-----------------------------------------------------------------------
   Nevil Brownlee                 Computer Science Department | ITSS
   Phone: +64 9 373 7599 x88941           The University of Auckland
   FAX: +64 9 373 7021      Private Bag 92019, Auckland, New Zealand

-------------------------------------------------
This mail sent through University of Auckland
http://www.auckland.ac.nz/

--
Help        mailto:[email-masked] and say "help" in message body
Unsubscribe mailto:[email-masked] and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/

--
Help        mailto:[email-masked] and say "help" in message body
Unsubscribe mailto:[email-masked] and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/