Subject: [netmod] float vs. decimal

From: Martin Bjorklund <[email-masked]>
Date: Wed, 11 Mar 2009 23:29:48 +0100 (CET)

Hi,

Here's a proposal for solving our float problem, which is one of the
few remaining open issues in the YANG document.

I suggest we remove the float types, and instead replace them with a
fix point decimal type, much like xs:decimal.  This is probably what
most people is looking for when they use floats in configuration data
models.

This could be done in many ways, but here's a simple (?) proposal.  We
would add one new type 'decimal64', which has a mandatory substatement
'fraction-digits'.  The value space would be the set of decimal
numbers which fit into a 64 bit (signed) integer, scaled with the
fraction part.  For example:

   leaf foo {
       type decimal64 {
           fraction-digits 3;
       }
   }

would allow all decimal values with three fraction digits between:

-9223372036854775.808 .. 9223372036854775.807

Ranges can be used:

   leaf bar {
       type decimal64 {
           fraction-digits 2;
           range "0..100";
       }
   }

this would allow values 0, 0.01, 0.02 .. 99.99, 100

In the canonical form, leading and trailing zeros are not printed,
except that 0.xx is printed with the leading 0.

In XPath evaluations, a decimal number will always be interpreted as a
64 bit floating point value (this is not the case with some textual
forms of floats).  This has the known rounding problems of course.

Drawback:  We loose the ability to model really large or really small
numbers, e.g. 1E234.

Comments?

/martin

---

From: Per Hedeland <[email-masked]>
Date: Thu, 12 Mar 2009 14:17:03 +0100

> most people is looking for when they use floats in configuration data

I guess I'm biased, but I think this is a very good proposal - in my
experience, the use of floats in data models for network devices ranges
from "don't care" to "serious problem" (the latter because HW and/or SW
may be unable to handle them) - but better-than-integer "resolution" is
often desirable. Minor issues:

> 'fraction-digits'.

Should probably require that fraction-digits > 0.

> except that 0.xx is printed with the leading 0.

I think the canonical form defined for xs:decimal may be more in line
with common expectations, i.e. decimal point and at least one digit
before and after is required => xx is printed as "xx.0".

--Per Hedeland

---

From: Phil Shafer <[email-masked]>
Date: Thu, 12 Mar 2009 09:51:31 -0400

>often desirable. Minor issues:

+1 here also.  I just did an inventory of our use of floats
and found:

6 percents (fractional-digits 2)
2 power ratings (POE) (fractional-digits 1)
3 routing metrics (fractional-digits 8)
2 billing rates (fractional-digits 2 (assumably))
1 ntp clock drift (unknown, but probably 4)

This proposal will satisfy all these.  Anyone want to check MIBs?

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Thu, 12 Mar 2009 09:18:39 -0700

> would allow all decimal values with three fraction digits between:

what are the lower and upper bounds for fraction-digits?
Is it 1..63? If so, then I approve of removing float32
and float64, and adding decimal64 instead.

> _

Andy

---

From: Balazs Lengyel <[email-masked]>
Date: Thu, 12 Mar 2009 17:24:52 +0100

+1 Balazs

> would add one new type 'decimal64', which has a mandatory substatement

BALAZS: I would not make it mandatory. Rather say that if it is absent it means
fraction-digits 2;

I think 2 is the most common, see Phil's examples.

(Not really important)

The value space would be the set of decimal
> _______________________________________________
> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod
> 

-- 
Balazs Lengyel                       Ericsson Hungary Ltd.
TSP System Manager
ECN: 831 7320                        Fax: +36 1 4377792
Tel: +36-1-437-7320     email: [email-masked]

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 12 Mar 2009 17:54:46 +0100

> This proposal will satisfy all these.  Anyone want to check MIBs?

There are no native floats in the SMIv2 so there are no native floats
in MIBs - if there are floats, then they are either shipped in Opaque
or hacked OCTET STRINGS or simply as character string. There are a few
IETF MIBs dealing with floats, e.g. the TeLinkBandwidth TC defined in
the TE-LINK-STD-MIB represents a float. The TC reprensents link
bandwidth in bps - I actually do not know why this should be float; a
64 bit scaled unsigned value should be pretty much sufficient.

That said, let me note that via the DISPLAY-HINT mechanism, the SMIv2
has kind of a "decimal32" type since you can insert a decimal point in
the rendering of integers, actually the SMIv2 has a signed and an
unsigned "decimal32".  (Is the proposed decimals64 always signed or
should we also have udecimal64, or short dec64 and udec64?) For
conversion of MIB modules, decimal64 will work very nicely.

/js

PS: If we do decimal64, we do get some obvious asymmetry since we have
    [u]int8|16|32|64 base types but then only [u?]decimal64. So if we
    were to open up the base types, we might as well do only int,
    uint, dec, and udec and all would be 64 bits. ietf-yang-types.yang
    can then define the [u]int8|16|32|64 variants for common usage and
    the spec actually gets shorter. (And smart implementations can in
    general pay attention to range restrictions when they generate
    storage allocations.)

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: Andy Bierman <[email-masked]>
Date: Thu, 12 Mar 2009 10:39:18 -0700

>     general pay attention to range restrictions when they generate

What about 32 and 64 bit, instead of just 64 bit?

I would prefer less numeric types,
with the extras moved to ietf-yang-types.yang.
IMO, we only need 6 built-in numeric types:

   int32:   32-bit signed integer
   uint32:  32-bit unsigned integer
   int64:   64-bit signed integer
   uint64:  64-bit unsigned integer
   dec64:   64-bit signed decimal
   udec64:  64-bit unsigned decimal

This matches SMIv2 and other common usage.
But it is not a big deal to have the other 4 types.
Either way, signed and unsigned decimal types
should be added, not just unsigned.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 12 Mar 2009 20:30:32 +0100 (CET)

> and float64, and adding decimal64 instead.

1..19.  Note that this is a *decimal* fix point, not a binary.  There
are 19 decimal digits in 9223372036854775807, thus the maximum is 19.

More details: if a derived type is constructed from a decmial64, the
fraction digits can be lowered, thus restricting the base type.  For
example:

  typedef foo {
      type decimal64 {
          fraction-digits 3;
      }
  }

and

  typedef bar {
      type foo {
          fraction-digits 1;
      }
  }

would restrict the range to

 -9223372036854775.9 .. 9223372036854775.8

/martin

---

From: Per Hedeland <[email-masked]>
Date: Thu, 12 Mar 2009 20:57:29 +0100

> are 19 decimal digits in 9223372036854775807, thus the maximum is 19.

Hm, is there an obvious problem with higher values? E.g. 21 would mean
that you can represent

  -0.009223372036854775808 .. 0.009223372036854775807

- that should be OK per se I think? Or at least just as OK as 19:

  -0.9223372036854775808 .. 0.9223372036854775807

If we want 'fraction-digits N' to imply that the full range of values
that can be expressed with N decimals should be covered - could make
sense I guess - the maximum would be 18.

--Per

---

From: Andy Bierman <[email-masked]>
Date: Thu, 12 Mar 2009 13:11:32 -0700

> Hm, is there an obvious problem with higher values? E.g. 21 would mean

I am concerned this simple change is getting more complicated
by the minute.  Maybe we should see a complete proposal first.

So there is no need for unsigned decimal? OK.
Let's not make the same mistake as SMIv2 in reverse.
(Counter64 only, then a hack for Unsigned64,
and still no Integer64.)

This is probably OK (signed but no unsigned).
At least a 'history table delta object' is still
possible (unlike SMIv2).

Andy

---

From: Balazs Lengyel <[email-masked]>
Date: Fri, 13 Mar 2009 11:52:06 +0100

> Hm, is there an obvious problem with higher values? E.g. 21 would mean

I think this rather complicates the original idea, and I don't see the use case for 23 digits 
precision. Do you ?

1..19 is fine or maybe 0..19

Balazs

---

From: Balazs Lengyel <[email-masked]>
Date: Fri, 13 Mar 2009 11:56:25 +0100

>  would allow all decimal values with three fraction digits between:

If you allow this change you will need some little rules to ensure that the value space is not 
increased. (Although you have bits available in the 64 bit int used you are not allowed to use 
them because that would increase the value space.

I propose, fraction-digits must remain unchanged.

Balazs

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Fri, 13 Mar 2009 12:22:11 +0100

On Wed, Mar 11, 2009 at 11:29:48PM +0100, Martin Bjorklund wrote:

> [...] For example:

So which of the following subtyping examples are legal and what is the
resulting value space?

typedef A {
  type dec64 { fraction-digits 2; range "0..100"; }
}

typedef B {
  type A { fraction-digits 1; }
}

typedef C {
  type A { fraction-digits 3; }
}

typedef D {
  type A { range "10.0..110.0"; }
}

typedef E {
  type A { fraction-digits 1; range "10.0..110.0"; }
}

/js

---

From: Per Hedeland <[email-masked]>
Date: Fri, 13 Mar 2009 12:59:31 +0100

On 2009-03-13 11:52, Balazs Lengyel wrote:
> I think this rather complicates the original idea, and I don't see the
> use case for 23 digits precision. Do you ?

No, I agree, it's just confusing - among other things since the
precision *wouldn't* be 23 digits.

> 1..19 is fine or maybe 0..19

Don't agree. What I tried to say above is that 19 is no more special
than 20 or 21 - the special number is 18, which is the max number where
you can represent all values. E.g. you can't represent

  0.9876543210987654321

even though that is 19 decimals. And I think 0 should be disallowed
because it's pointless (and possibly confusing) - don't use the decimal
type if you don't want any decimals, use an integer. I.e. my suggestion
is:  1..18

--Per

---

From: Phil Shafer <[email-masked]>
Date: Fri, 13 Mar 2009 07:56:34 -0400

> would allow all decimal values with three fraction digits between:

This example is fine, but I'd also want to be able to raise
the number of decimal digits for this scenario:

typedef percent {
    type decimal64 {
        fractional-digits 1;
    }
}

typedef really-accurate-percentage {
    type percent {
        fractional-digits 5;
    }
}

Not a huge sticking point, though....

More of a sticky issue is the 64/32/16/8.  I'm no type guru, but
would we be better off with "integer" and "decimal" as the builtin
types and then derive all the [u]int[64,32,16,8] in yang-types?  I
know we considered and discarded this earlier, but with decimal,
the issue becomes twice as ugly.

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Fri, 13 Mar 2009 08:03:51 -0400

>So which of the following subtyping examples are legal and what is the

And what does this do?

typedef zee {
    type decimal64 {
        fractional-digits 3;
        range 0.001..0.010;
    }
}

typedef why {
    type zee {
        fractional-digits 1;
    }
}

;^)

Thanks,
 Phil

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Fri, 13 Mar 2009 14:32:11 +0100

> the issue becomes twice as ugly.

An unlimited "integer" or "decimal" means that you better never use
this base type directly since implementations will likely not support
arbitrary precision integers and thus likely pick a random boundary.
This is bad if you ask me. So I am big fan of base types that have
well defined precision. Now, how many of those you need is a different
discussion. For smart implementations that understand range
restrictions, there is hardly a difference, except that you save some
imports and qualified type names (yang:uint8 vs. uint8) when types
with common precision are builtin.

/js

---

From: Per Hedeland <[email-masked]>
Date: Fri, 13 Mar 2009 15:29:32 +0100

> with common precision are builtin.

Also, if there should be variants, I don't see why they should follow the
pattern of the integer types. This is not another integer type, in fact
as the proposal is specifically to replace floats, those would be a more
obvious parallel. No-one asks for signed vs unsigned, 16-bit, or 8-bit
versions of floats, and even the case for a 32-bit float type is
probably pretty weak.

And FWIW, XML Schema defines a boatload of integer types (13 to be
precise, 5 more than yang), but there is only one xs:decimal - it is
signed, and requires 64 bits of storage in the unrestricted form.

It seems to me that having only a signed decimal64 is adequate, but
possibly an additional signed decimal32 could be motivated - the latter
alternative would be a pretty good match for the float types currently
defined in yang.

--Per

---

From: Balazs Lengyel <[email-masked]>
Date: Fri, 13 Mar 2009 17:02:30 +0100

> you can represent all values. E.g. you can't represent
Fully agree I vote for 1..18 fractional-digits and no change in derived types to fractional-digits.
Balazs

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 13 Mar 2009 17:05:34 +0100

> signed, and requires 64 bits of storage in the unrestricted form.

If I read the XSD spec correctly, this is only the minimum requirement
(actually defined as 18 decimal digits), but in general the decimal
numbers can be arbitrarily long.

> alternative would be a pretty good match for the float types currently

How about XPath expressions? Will floats be allowed there?

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Andy Bierman <[email-masked]>
Date: Fri, 13 Mar 2009 09:08:32 -0700

> types to fractional-digits.

+1

Andy

---

From: Andy Bierman <[email-masked]>
Date: Fri, 13 Mar 2009 09:13:39 -0700

> numbers can be arbitrarily long.

The 'arbitrarily long' part is why it is not implementable.

> How about XPath expressions? Will floats be allowed there?

All XPath numbers are IEEE 754 numbers.
There are no YANG numbers in XPath.
You have to convert the YANG number to 'double' to
use it in must/when/select.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 13 Mar 2009 17:24:25 +0100

> use it in must/when/select.

Also literals such as 1e3 must be allowed there.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Andy Bierman <[email-masked]>
Date: Fri, 13 Mar 2009 09:43:24 -0700

> Also literals such as 1e3 must be allowed there.

I think literals must be quoted in XPath 1.0, e.g., '1e3'.
1e3 is an invalid number.  Only the 11, or 11., or 11.111
forms are allowed, plus inv, -inv, and nan.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 13 Mar 2009 17:55:26 +0100

> forms are allowed, plus inv, -inv, and nan.

No, "foo:bar=1e3" works (just tried it).

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Andy Bierman <[email-masked]>
Date: Fri, 13 Mar 2009 10:01:24 -0700

> No, "foo:bar=1e3" works (just tried it).

 From the XPath 1.0 spec:

[29]   	Literal	   ::=   	'"' [^"]* '"'	
			| "'" [^']* "'"

[30]   	Number	   ::=   	Digits ('.' Digits?)?	
			| '.' Digits	

[31]   	Digits	   ::=   	[0-9]+

Where in the EBNF is this form of number supported?

Andy

---

From: Per Hedeland <[email-masked]>
Date: Fri, 13 Mar 2009 19:21:48 +0100

On 2009-03-13 17:05, Ladislav Lhotka wrote:
> Per Hedeland píae v Pá 13. 03. 2009 v 15:29 +0100:

Absolutely, that's what I meant by "requires". I.e. a minimum of 18
decimal digits, which translates to a minimum of 64 bits as long as your
storage is in integral numbers of octets - the 18 was obviously not
picked at random. The point was that they didn't see a need for a
decimal type with a lower storage requirement, like they did with all
the integer types.

--Per

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 13 Mar 2009 20:55:46 +0100 (CET)

Hi,

> So which of the following subtyping examples are legal and what is the

Well, Balazs and Andy think that fraction-digits cannot be changed,

>   type dec64 { fraction-digits 2; range "0..100"; }

0 0.01 0.02 .. 99.99 100

>   type A { fraction-digits 1; }

0 0.1 0.2 .. 99.9 100

>   type A { fraction-digits 3; }

not allowed - you cannot raise the number of fraction-digits.

>   type A { range "10.0..110.0"; }

not allowed - high range too big.

>   type A { fraction-digits 1; range "10.0..110.0"; }

not allowed - high range too big.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Fri, 13 Mar 2009 13:53:36 -0700

>>   type dec64 { fraction-digits 2; range "0..100"; }

I do not like this extra complexity.
What if there are a chain of typedefs
so various properties are derived from
different typedefs?

typedef A {
   type dec64 {
     fraction-digits 2;
     range "0..99.99";
   }
}

typedef B {
   type A {
     default 49.01;
   }
}

typedef C {
   type B {
      fraction-digits 1;
      range "1.1 .. max";
   }
}

typedef D {
   type C {
     default 19.1;
   }
}

See how changing the precision (fraction-digits)
breaks range and default?  It is much better
if decimal allows the same restrictions as
any other number.  No CLRs. No confusion.

Typedef doesn't have the fraction-digits you need?
Make a new typedef.
You can make all the changes you want ;-)

Andy

---

From: Andy Bierman <[email-masked]>
Date: Fri, 13 Mar 2009 14:01:41 -0700

> so various properties are derived from

I got the syntax wrong (!)

typedef A {
   type dec64 {
     fraction-digits 2;
     range "0..99.99";
   }
}

typedef B {
   type A;
   default 49.01;
}

typedef C {
   type B {
      fraction-digits 1;
      range "1.1 .. max";
   }
}

typedef D {
   type C;
   default 19.1;
}

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 13 Mar 2009 22:19:56 +0100 (CET)

> so various properties are derived from

[...]

> breaks range and default?

Ok.  But the same thing can happen with integers:

  typedef A {
      type int32;
      default 1;
  }

  typedef B {
      type A;
      range "2..max";
  }

However, the range issue convinced me.  I agree that it's best to not
allow fraction-digits to be changed.

/martin

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 20 Mar 2009 15:17:15 +0100 (CET)

Hi,

Here is proposed text for a decimal64 type.  This is supposed to
simply be a replacement for the float32 and float64 types.

** The decimal64 Built-in Type

The decimal64 type represents a subset of the real numbers, which can
be represented by decimal numerals.  The value space of decimal64 is
the set of numbers that can be obtained by multiplying a 64 bit signed
integer by a negative power of ten, i.e. expressible as i x 10^-n
where i is an integer64 and n is an integer between 1 and 18,
inclusively.

*** Lexicographic Representation

A decimal64 value is lexicographically represented as an optional sign
("+" or "-"), followed by a sequence of decimal digits, optionally
followed by a period ('.') as a decmial indicator and a sequence of
decimal digits.  If no sign is specified, "+" is assumed.

*** Canonical Form

The canonical form of a positive decimal64 does not include the sign
"+".  The decimal point is required.  Leading and trailing zeros are
prohibited, subject to the rule that there MUST be at least one digit
before and after the decimal point.  The value zero is represented as
"0.0".

*** Restrictions

A decmial64 type can be restricted with the "range" statement
(^range^).

*** The fraction-digits Statement @fraction-digits@

The "fraction-digits" statement, which is a substatement to the "type"
statement, MUST be present if the type is "decimal64".  It takes as an
argument an integer between 1 and 18, inclusively.  It controls the
size of the minimum difference between values of a decimal64 type, by
restricting the value space to numbers that are expressible as i x
10^-n where n is the fraction-digits argument.

*** Usage Example

  type decimal64 {
      fraction-digits 2;
      range "1 .. 3.14 | 10 | 20..max";
  }

/martin

---

From: Phil Shafer <[email-masked]>
Date: Fri, 20 Mar 2009 12:40:28 -0400

>before and after the decimal point.  The value zero is represented as

Can we make the decimal optional if the fractional part is zero,
so we get "0" not "0.0"?

Thanks,
 Phil

---

From: Per Hedeland <[email-masked]>
Date: Mon, 23 Mar 2009 09:10:00 +0100

> Can we make the decimal optional if the fractional part is zero,

That was in Martin's original proposal, I suggested that it would be
more "in line with expectations" or somesuch for decimal(s) to always be
present in the canonical form. I guess those expectations (i.e. mine)
may be due to the analogy with floats - you normally print decimal(s)
even if a float value happens to be integral (%g doesn't, though) - and
to the fact that the canonical form for xs:decimal requires this.

Not exactly strong technical arguments - do you have some for the
opposite viewpoint?

--Per