Subject: RE: [ipfix] Floating point numbers in IPFIX

From: "Thomas Dietz" <[email-masked]>
Date: Thu, 9 Feb 2006 15:06:07 +0100

Dear Benoit and all,
 
I agree to your proposal but would add a sentence like:
 
When reducing the size of a float type make sure that the precision is not
reduced due to the smaller mantissa.
 
You could make it more strong in writing "MUST make sure" but I think that's
not needed.
 
Best Regards,
 
Thomas
 
--
Thomas Dietz
Network Laboratories, NEC Europe Ltd., 69115 Heidelberg, Germany

  _____  

From: majordomo listserver [mailto:[email-masked]] On Behalf Of
Benoit Claise
Sent: Thursday, February 09, 2006 11:17 AM
To: Andrew Johnson
Cc: [email-masked]
Subject: Re: [ipfix] Floating point numbers in IPFIX

Dear all,

In an attempt to summarize all this thread conclusion, I have inserted the
following text.

Regards, Benoit.

6R6.2 Reduced Size Encoding of Integer and Float Types

Information Elements containing integer, string, float, and octetarray types
in the information model MAY be encoded using fewer octets than those
implied by their type in the information model definition [IPFIX-INFO],
based on the assumption that the smaller type is sufficient to carry any
value the Exporter may need to deliver.  This reduces the network bandwidth
requirement between the Exporter and the Collector.  Note that the
Information Element definitions [IPFIX-INFO] will always define the maximum
encoding size.

For instance the information model [IPFIX-INFO] defines byteCount as an
unsigned64 type, which would require 64-bits.  However if the Exporter will
never locally encounter the need to send a value larger than 4294967295, it
may chose to send the value instead as an unsigned32.  For example, a core
router would require an unsigned64 byteCount while an unsigned32 might be
sufficient for an access router.
<!--[if !supportLineBreakNewLine]-->
<!--[endif]-->

This behavior is indicated by the Exporter by specifying a type size with a
smaller length than that associated with the assigned type of the
Information Element.  In the example above the Exporter would place a length
of 4 versus 8 in the template.

If reduced sizing is used, it MUST only be applied to the following integer
types: unsigned64, signed64, unsigned32, signed32, unsigned16, signed16.
The same signed versus unsigned property MUST be preserved.  The reduction
in size can be to any number of octets smaller than the original type if the
data value still fits, i.e. so that only leading zeroes are dropped. For
example, an unsigned64 can be reduced in size to 7, 6, 5, 4, 3, 2, or 1
octet(s).

Reduced sizing can also be used on the float64 and float32.  The float32 not
only has a reduced number range, but due to the smaller mantissa is also
less precise.

The reduced size encoding MUST NOT be applied to dateTimeMicroSeconds or to
dateTimeNanoSeconds because these represent an inherent structure that would
be destroyed by using less than the original number of bytes.

Hi all 

In the IPFIX protocol draft, section 3.1.5, we have a definition 
of a floating point number.  This definition allows export of 
floating point numbers using the 32-bit format as specified in 
IEEE.754.  This same standard also allows floating point numbers 
to be defined in a 64-bit format. 

I would like to see a float64 type added and the abiltity to 
send float64 fields using float32 using a variation of the 
Reduced Size Encoding as used with the integers. 

I would like to propose that at the next editing phase we add 
the following type to the information model: 

3.1.6.  float64 

The type "float64" corresponds to an IEEE double-precision 64-bit 
floating point type as defined in [IEEE.754.1985]. 

In the protocol, section 6.2 could also be expanded to encompase 
the Reduced Size encoding for the floating point types. i.e. 

6.2      Reduced Size Encoding of Integer Types 
 Information Elements containing integer, float, string, and ... 
                                          ^^^^^^^ 
 octetarray types in the information model MAY be encoded using fewer 
 octets than those implied by their type in the information model 
 definition [IPFIX-INFO], ... 

 ... 
   If reduced sizing is used, it MUST only be applied to the following
integer types: unsigned64, signed64, unsigned32, signed32,  unsigned16,
signed16.  ... 

@@[NOTE: the signed types above are not in the data model]@@ 

 Reduced sizing can also be used on the float64 and float32.  The 
 float32 not only has a reduced number range, but due to the 
 smaller mantissa is also less precise. 

 The reduced size encoding MUST NOT be applied to ... 

Andrew 

-- 
Help        mailto:[email-masked] and say "help" in message
body 
Unsubscribe mailto:[email-masked] and say 
"unsubscribe ipfix" in message body 
Archive     http://ipfix.doit.wisc.edu/archive/

---

From: Stewart Bryant <[email-masked]>
Date: Thu, 09 Feb 2006 14:14:48 +0000

Thomas Dietz wrote:

> When reducing the size of a float type make sure that the precision is 
> not reduced due to the smaller mantissa.

It may be OK to reduce the precision - it's application dependent.

- Stewart

>  
> You could make it more strong in writing "MUST make sure" but I think 
> that's not needed.
>  
> Best Regards,
>  
> Thomas
>  
> --
> Thomas Dietz
>     ------------------------------------------------------------------------
>     From: majordomo listserver [mailto:[email-masked]] On
>     Behalf Of Benoit Claise
>     In an attempt to summarize all this thread conclusion, I have
>     inserted the following text.
>
>     Regards, Benoit.
>
>     Information Elements containing integer, string, float, and
>     octetarray types in the information model MAY be encoded using
>     fewer octets than those implied by their type in the information
>     model definition [IPFIX-INFO], based on the assumption that the
>     smaller type is sufficient to carry any value the Exporter may
>     need to deliver.  This reduces the network bandwidth requirement
>     between the Exporter and the Collector.  Note that the Information
>     Element definitions [IPFIX-INFO] will always define the maximum
>     encoding size.
>
>     For instance the information model [IPFIX-INFO] defines byteCount
>     as an unsigned64 type, which would require 64-bits.  However if
>     the Exporter will never locally encounter the need to send a value
>     larger than 4294967295, it may chose to send the value instead as
>     an unsigned32.  For example, a core router would require an
>     unsigned64 byteCount while an unsigned32 might be sufficient for
>     an access router.
>     This behavior is indicated by the Exporter by specifying a type
>     size with a smaller length than that associated with the assigned
>     type of the Information Element.  In the example above the
>     Exporter would place a length of 4 versus 8 in the template.
>
>     If reduced sizing is used, it MUST only be applied to the
>     following integer types: unsigned64, signed64, unsigned32,
>     signed32, unsigned16, signed16.  The same signed versus unsigned
>     property MUST be preserved.  The reduction in size can be to any
>     number of octets smaller than the original type if the data value
>     still fits, i.e. so that only leading zeroes are dropped. For
>     example, an unsigned64 can be reduced in size to 7, 6, 5, 4, 3, 2,
>     or 1 octet(s).
>
>     Reduced sizing can also be used on the float64 and float32. 
>     The float32 not only has a reduced number range, but due to the
>     The reduced size encoding MUST NOT be applied to
>     dateTimeMicroSeconds or to dateTimeNanoSeconds because these
>     represent an inherent structure that would be destroyed by using
>     less than the original number of bytes.
>
>
>>     Hi all
>>
>>        If reduced sizing is used, it MUST only be applied to the
>>     following  integer types: unsigned64, signed64, unsigned32,
>>     signed32,  unsigned16, signed16.  ...
>>
>>     Help        mailto:[email-masked] and say "help" in
>>     message body

---

From: Brian Trammell <[email-masked]>
Date: Thu, 9 Feb 2006 09:59:25 -0500

All,

There's something I'm missing here... Does IEEE 754 have anything to  
say on arbitrary reduced-precision/reduced-range floating point  
representations? Is there really a meaningful, say, six-byte  
"float64" that everyone can agree on? Do you reduce the exponent, or  
the mantissa, or both; in that case, each by how many bits? If my 6- 
byte float64 has a 10-bit exponent and a 36-bit fraction, and yours  
has an 8-bit exponent and a 34-bit fraction, how do we represent  
this? Do we open the float types to non-754, machine-specific  
representations?

Or are we really, practically, just talking about a way to represent  
a float64-specified IE as a float32?

- Brian

On Feb 9, 2006, at 9:14 AM, Stewart Bryant wrote:

>> When reducing the size of a float type make sure that the  
>> precision is not reduced due to the smaller mantissa.
>> You could make it more strong in writing "MUST make sure" but I  
>> think that's not needed.
>>
>> Best Regards,
>>
>> Thomas
>>
>> --
>> Thomas Dietz
>>> fewer

---

From: Paul Aitken <[email-masked]>
Date: Fri, 10 Feb 2006 00:50:08 +0000

Brian,

> "float64" that everyone can agree on? [...]

Noooo! This isn't what we want to do at all.

> a float64-specified IE as a float32?

Exactly.

Referring back to the email which started this thread:
http://ipfix.doit.wisc.edu/archive/3034.html

Andrew said:

> I would like to see a float64 type added and the abiltity to send
> float64 fields using float32 using a variation of the Reduced Size
> Encoding as used with the integers.

And Stewart's follow-up:

> I think that we should add float64 with the export compresion to
> float32.

So perhaps Benoit's text could be clearer, eg:

   Reduced sizing can also be used to reduce float64 to float32.
   The float32 not only has a reduced number range, but due to
   the smaller mantissa is also less precise.

Cheers.
-- 
Paul Aitken
Cisco Systems Ltd, Edinburgh, Scotland.

--
Help        mailto:[email-masked] and say "help" in message body
Unsubscribe mailto:[email-masked] and say
"unsubscribe ipfix" in message body
Archive     http://ipfix.doit.wisc.edu/archive/

---

From: Stewart Bryant <[email-masked]>
Date: Fri, 10 Feb 2006 09:54:09 +0000

>   the smaller mantissa is also less precise.

I agree

- Stewart

---

From: Benoit Claise <[email-masked]>
Date: Fri, 10 Feb 2006 11:07:38 +0100

Hi,
>>   the smaller mantissa is also less precise.
Agreed and done.

Regards, Benoit.