Subject: [netmod] Y59: what about decimal56 data type?

From: Andy Bierman <[email-masked]>
Date: Sat, 20 Dec 2014 09:55:25 -0800

Hi,

Should we be doing something in YANG 1.1 to make large number math safer?
IMO a guideline that warns "it won't work" it not doing very much.

Would a decimal56 data type be safe to convert to float64?
If so, should YANG have a new data type for "safe math"?

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 29 Dec 2014 16:59:20 +0100

> IMO a guideline that warns "it won't work" it not doing very much.

I agree, but I don't know how to make it safer, except for removing
decimal64, uint64 and int64 types, which is clearly unacceptable.

> If so, should YANG have a new data type for "safe math"?

The same problem affects int64 and uint64 as well: for the leaf definition

  leaf foo {
    type int64;
    must ". <= 1000000000000000000";
  }

the following value is accepted as valid:

  <foo>1000000000000000001</foo>

(both long numbers have 19 digits).

Lada

>
>
> Andy
>
> _______________________________________________
> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod

-- 
Ladislav Lhotka, CZ.NIC Labs
PGP Key ID: E74E8C0C

---

From: Andy Bierman <[email-masked]>
Date: Mon, 29 Dec 2014 08:17:04 -0800

> (both long numbers have 19 digits).

But this is int64.
If the designer picked decimal56 instead then this would
be an illegal value right?  The must-expr would not even matter, right?

Why would a real data model have a must-stmt that was not safe
for plain XPath (regardless of YANG usage)? What are some real use-cases?

The point of the data type is to provide the designer with a number that
is safe to use in XPath.  The designer would obviously have to choose this new
data type instead of one of the 64 bit data types (because an unsafe XPath
must/when expression was needed for this leaf).

I am not convinced anybody wants to write "large number XPath" in must/when
expressions. This may not be an operational problem.  I agree that the code
will do the wrong thing if this issue occurs. That much is clear.

Andy

---

From: David Spakes <[email-masked]>
Date: Mon, 29 Dec 2014 12:03:51 -0500 (EST)

Andy,

When I read your post about "safe math" this morning, it reminded me of 
the original discussion about decimal64 way back in 2009.  I had similar 
concerns at that time, so I propsed a derived type called "real".  My
last mention of it was in a post on October 26, 2009. I would like to 
resubmit it for the working group's consideration now.

   typedef real {
     type union {
       type decimal64 {
         fraction-digits 18;
         range "-0.999999999999999999 .. 0.999999999999999999";
       }
       type decimal64 {
         fraction-digits 17;
         range "-9.99999999999999999 .. 9.99999999999999999";
       }
       type decimal64 {
         fraction-digits 16;
         range "-99.9999999999999999 .. 99.9999999999999999";
       }
       type decimal64 {
         fraction-digits 15;
         range "-999.999999999999999 .. 999.999999999999999";
       }
       type decimal64 {
         fraction-digits 14;
         range "-9999.99999999999999 .. 9999.99999999999999";
       }
       type decimal64 {
         fraction-digits 13;
         range "-99999.9999999999999 .. 99999.9999999999999";
       }
       type decimal64 {
         fraction-digits 12;
         range "-999999.999999999999 .. 999999.999999999999";
       }
       type decimal64 {
         fraction-digits 11;
         range "-9999999.99999999999 .. 9999999.99999999999";
       }
       type decimal64 {
         fraction-digits 10;
         range "-99999999.9999999999 .. 99999999.9999999999";
       }
       type decimal64 {
         fraction-digits 9;
         range "-999999999.999999999 .. 999999999.999999999";
       }
       type decimal64 {
         fraction-digits 8;
         range "-9999999999.99999999 .. 9999999999.99999999";
       }
       type decimal64 {
         fraction-digits 7;
         range "-99999999999.9999999 .. 99999999999.9999999";
       }
       type decimal64 {
         fraction-digits 6;
         range "-999999999999.999999 .. 999999999999.999999";
       }
       type decimal64 {
         fraction-digits 5;
         range "-9999999999999.99999 .. 9999999999999.99999";
       }
       type decimal64 {
         fraction-digits 4;
         range "-99999999999999.9999 .. 99999999999999.9999";
       }
       type decimal64 {
         fraction-digits 3;
         range "-999999999999999.999 .. 999999999999999.999";
       }
       type decimal64 {
         fraction-digits 2;
         range "-9999999999999999.99 .. 9999999999999999.99";
       }
       type decimal64 {
         fraction-digits 1;
         range "-99999999999999999.9 .. 99999999999999999.9";
       }
       type enumeration {
         enum overflow;
         enum underflow;
       }
     }
     description
       "The real type defines a large, finite set of real numbers with
        varying degrees of magnitude and precision.  An object of type
        real is able to express configuration and state data as any of
        the real numbers in the set.  The real type is suitable for
        applications that deal with ratios in which the decimal point is
        not fixed to a single position.

        Positive numbers larger than 99999999999999999.9 in state data
        SHALL be represented as an overflow.  The enumerated value
        overflow is not valid for an <edit-config> operation.

        Negative numbers larger than -99999999999999999.9 in state data
        SHALL be represented as an underflow.  The enumerated value
        underflow is not valid for an <edit-config> operation.

        Real numbers between -99999999999999999.9 and 99999999999999999.9
        (inclusive) having a combination of magnitude and precision that
        falls outside of one of the range substatements in the union--
        including irrational numbers such as 'pi'--may only be
        represented by the real type through rounding or truncation.
        For example, pi could be rounded (up) to 3.14159265358979324 or
        truncated to 3.14159265358979323 to preserve as much of the
        precision information as possible.  Note that an application
        could choose to round (down) the value of pi to 3.14 or truncate
        the value of pi to 3.14.  This specification does not define
        rules for the rounding or truncating of such values, considering
        these decisions to be application-specific.";
   }

David Spakes

> (both long numbers have 19 digits).

-------------------------------------------------------------
  David Spakes                       email:   [email-masked]
  SNMP Research                      voice:   +1 865 573 1434
  3001 Kimberlin Heights Road          fax:   +1 865 573 9197
  Knoxville, TN  37920-9716  USA      http://www.snmp.com
-------------------------------------------------------------

---

From: Andy Bierman <[email-masked]>
Date: Mon, 29 Dec 2014 09:11:08 -0800

Hi,

I remember this thread.
I agree we should use your typedef for "float64".
I think another typedef for "float32" would also be needed.

Andy

On Mon, Dec 29, 2014 at 9:03 AM, David Spakes <[email-masked]> wrote:
> Andy,
>
> When I read your post about "safe math" this morning, it reminded me of the
> original discussion about decimal64 way back in 2009.  I had similar

---

From: David Spakes <[email-masked]>
Date: Mon, 29 Dec 2014 13:34:56 -0500 (EST)

Andy,

I changed typedef real to typedef float64, and I created a typedef 
float32, per your suggestions.  They are included below at the end
of the message.

David

>> resubmit it for the working group's consideration now.

    typedef float64 {
      type union {
        type decimal64 {
          fraction-digits 18;
        "The float64 type defines a large, finite set of real numbers with
         float64 is able to express configuration and state data as any of
         the real numbers in the set.  The float64 type is suitable for

    typedef float32 {
      type union {
        type decimal64 {
          fraction-digits 9;
          range "-0.999999999 .. 0.999999999";
        }
        type decimal64 {
          fraction-digits 8;
          range "-9.99999999 .. 9.99999999";
        }
        type decimal64 {
          fraction-digits 7;
          range "-99.9999999 .. 99.9999999";
        }
        type decimal64 {
          fraction-digits 6;
          range "-999.999999 .. 999.999999";
        }
        type decimal64 {
          fraction-digits 5;
          range "-9999.99999 .. 9999.99999";
        }
        type decimal64 {
          fraction-digits 4;
          range "-99999.9999 .. 99999.9999";
        }
        type decimal64 {
          fraction-digits 3;
          range "-999999.999 .. 999999.999";
        }
        type decimal64 {
          fraction-digits 2;
          range "-9999999.99 .. 9999999.99";
        }
        type decimal64 {
          fraction-digits 1;
          range "-99999999.9 .. 99999999.9";
        }
        type enumeration {
          enum overflow;
          enum underflow;
        }
      }
      description
        "The float32 type defines a large, finite set of real numbers with
         varying degrees of magnitude and precision.  An object of type
         float32 is able to express configuration and state data as any of
         the real numbers in the set.  The float32 type is suitable for
         applications that deal with ratios in which the decimal point is
         not fixed to a single position.

         Positive numbers larger than 99999999.9 in state data
         SHALL be represented as an overflow.  The enumerated value
         overflow is not valid for an <edit-config> operation.

         Negative numbers larger than -99999999.9 in state data
         SHALL be represented as an underflow.  The enumerated value
         underflow is not valid for an <edit-config> operation.

         Real numbers between -99999999.9 and 99999999.9 (inclusive)
         having a combination of magnitude and precision that
         falls outside of one of the range substatements in the union--
         including irrational numbers such as 'pi'--may only be
         represented by the real type through rounding or truncation.
         For example, pi could be rounded (down) to 3.14159265 or
         truncated to 3.14159265 to preserve as much of the
         precision information as possible.  Note that an application
         could choose to round (down) the value of pi to 3.14 or truncate
         the value of pi to 3.14.  This specification does not define
         rules for the rounding or truncating of such values, considering
         these decisions to be application-specific.";
    }

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 30 Dec 2014 10:50:18 +0100

> I think another typedef for "float32" would also be needed.

Two observations/comments:

- Sometimes people use floating point numbers when there really is no
  need to use them. There really is no need to represent a drop
  probability using IEEE floating point numbers. There is no need to
  represent bandwidth using IEEE floating point numbers. This is
  primarily an educational problem and perhaps it deserves adding text
  to RFC 6087bis.

- Introducing a typedef that sounds and smells like an IEEE
  float/double but in fact is different will cause confusion. If there
  really is a demonstrated need for floating point numbers, we better
  use something that maps easily to other programming languages and
  things like xpath.

It would help a lot seeing a use case that convinces me that the full
range of floating point numbers are needed for network management. The
argument 'protocol X uses IEEE floating point numbers => YANG has to
support IEEE floating point' with X in { RSVP, ... } is not by itself
convincing - I think it matters how IEEE floating point numbers are
used and whether the whole precision needs to be available at a
network management interface.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 30 Dec 2014 16:15:20 +0100

> be an illegal value right?  The must-expr would not even matter,

Well, yes, but int64 and uint64 need to be supported.

> for plain XPath (regardless of YANG usage)? What are some real

My example may not be realistic but there are certainly use cases where such
leafs could appear in equality comparisons, e.g. in some kind of uniqueness check.

> must/when expression was needed for this leaf).

I think it is important the designer is aware of the issue, and chooses
whatever is appropriate, e.g. uint32 or a range. IMO decimal56 looks weird.

> will do the wrong thing if this issue occurs. That much is clear.

It is a shortcoming of XPath 1.0 and as such it should be mentioned in
the YANG spec. I don't think we can do much more. XPath 2.0 would solve
this problem but it is not an option for YANG 1.1.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Tue, 30 Dec 2014 07:46:39 -0800

> Well, yes, but int64 and uint64 need to be supported.

But they cannot be supported.  That is the point of this issue.

> whatever is appropriate, e.g. uint32 or a range. IMO decimal56 looks weird.

Forget that type.
What about the float32 and float64 typedefs from David?

> this problem but it is not an option for YANG 1.1.

It should be mentioned, but a 1 line warning buried in 130 pages doesn't
help that much.  What do you say in the text?  "Make sure no 64 bit leaf
ever uses the most significant 8 bits".   "Make sure any loss of precision
results in an 'operation-failed' error".

Neither seems very implementable or enforceable.

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 30 Dec 2014 16:50:45 +0100

> Neither seems very implementable or enforceable.

A YANG compiler that is able to parse xpath expressions can generate
warnings if 64-bit leafs are used in expressions where the precision
may produce unexpected results.

/js

---

From: Andy Bierman <[email-masked]>
Date: Tue, 30 Dec 2014 08:06:54 -0800

>   use something that maps easily to other programming languages and

I agree that we are ignoring a very important fact -- NETCONF and RESTCONF
do not use binary message encoding, so it is impossible to send an IEEE
float/double.  It has to be re-encoded as something else.

The NETMOD WG (correctly) decided that requiring the server to parse
complex text expressions representing floating point numbers was a bad idea.
Decimal64 strings are much easier to parse.

> network management interface.

I agree, except this doesn't help legacy data models that have
float32 leafs.  Designers need to understand the difference between
the on-the-wire text encoding and internal binary representation.

If somebody wants to design a typedef for a textual representation of
a number that is really hard to parse, then they can do that.
It would be better to design a typedef like float32 or float64 that
is easy to parse and properly constrains the value set.

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 30 Dec 2014 19:04:07 +0100

> Decimal64 strings are much easier to parse.

I do not think parsing complexity is really an issue since strtod()
and friends solve that problem if you code is fine with using floating
point numbers.

/js