Subject: [netmod] 64-bit numbers in JSON encoding

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 18 Nov 2014 17:15:15 +0100

Hi,

one of the things that IMO requires further discussion is the way how
64-bit numbers (uint64, int64, decimal64) are encoded in JSON. Following
the recommendation of I-JSON, the encoding was changed from JSON number
to string.

However, as Carsten Bormann explained during the session in Honolulu,
the inability to deal with large numbers is an issue mainly for
JavaScript that uses IEEE double float as the only numeric type. Juergen
also mentioned via jabber that he knew about a JSON parser written for C
that had the same problem.

Anyway, I think it's worthwhile to consider pros and cons of introducing
an ugly CLR for 64-bit numbers and imposing more work on implementations
that could normally expect obtaining such numbers straight from a sound
JSON parser.

Lada

-- 
Ladislav Lhotka, CZ.NIC Labs
PGP Key ID: E74E8C0C

---

From: Robert Varga <[email-masked]>
Date: Tue, 18 Nov 2014 18:05:47 +0100

Hello,

> that could normally expect obtaining such numbers straight from a sound

I would like to express a strong preference for not performing 
special-casing on 64bit type encoding.

Thanks,
Robert

---

From: Kent Watsen <[email-masked]>
Date: Tue, 18 Nov 2014 18:09:57 +0000

>that could normally expect obtaining such numbers straight from a sound

Isn't JavaScript support a primary concern?   While apps using RESTCONF
would likely be programmed in other languages, let's not forget about
web-based interfaces...or what the "JS" in JSON stands for.

Thanks,
Kent

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 18 Nov 2014 20:35:26 +0100

> web-based interfaces...or what the "JS" in JSON stands for.

If we do not follow the I-JSON recommendations, we would have to
declare in a very explicit way that an implementation is required to
handle 64 bit signed and unsigned integers and that an implementation
that does not do so is not compliant.

You can, of course, do the right thing in almost any programming
language, including JavaScript:

  https://www.npmjs.org/package/json-bignum

In short, you need to do slow arithmetic using libraries that support
proper arithmetics, e.g.:

  https://github.com/peterolson/BigRational.js
  https://github.com/iriscouch/bigdecimal.js

To process 64-bit data precisely, you will need slow arithmetic
libraries anyway. This pain is part of the JavaScript experience,
regardless how we ship the numbers over the wire.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: Phil Shafer <[email-masked]>
Date: Tue, 18 Nov 2014 14:50:59 -0500

>> web-based interfaces...or what the "JS" in JSON stands for.

Lauguages like XSLT have the same issue.

http://www.w3.org/TR/xpath/#numbers

    3.5 Numbers

       A number represents a floating-point number. A number can
       have any double-precision 64-bit format IEEE 754 value [IEEE
       754].

>that does not do so is not compliant.

I think this is the root of the problem.  Our explicit rules would
not interact well with the normal encode/decode libraries for other
languages, which means one would be stuck reimplementing such logic,
or at best forking well-used libraries.  There are bignum libraries,
but plumbing the json parser in javascript to use them would not
be easy, and might have performance impacts since the parsing library
is likely native code.

As much as I dislike it, quoting 64bit integers seems unavoidable.

Thanks,
 Phil

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 19 Nov 2014 09:26:48 +0100

> web-based interfaces...or what the "JS" in JSON stands for.

Yes, that's what Carsten said. If we conclude that the role of
JavaScript in the NETCONF/RESTCONF ecosystem is important, then probably
there is currently no way other than encoding 64-bit numbers as strings.

By the way, JSON WG tries to avoid interpreting "JS" as JavaScript.

Lada

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 19 Nov 2014 09:45:29 +0100

> Lauguages like XSLT have the same issue.

Good that you mention it - it is in fact an issue for YANG as such. I
just tested this leaf definition:

  leaf foo {
    type decimal64 {
      fraction-digits 18;
    }
    must ". = 1";
  }

When validating an instance via Schematron (or any compliant XPath
implementation), this value will be found as valid:

<foo>1.0000000000000001</foo>

IMO this is a serious problem that could be misused as an attack
vector. I wonder - shouldn't we address it in YANG 1.1?

Lada

> _______________________________________________
> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 19 Nov 2014 09:53:18 +0100

> As much as I dislike it, quoting 64bit integers seems unavoidable.

I think there are two things to distinguish here.

- The first one is the choice of JSON libraries and whether they do
  support 64 bit signed/unsigned integers.

- The second is what you do once you have to deal with 64 bit
  signed/unsigned numbers.

Shipping 64bit integers as strings helps perhaps with the first
problem (if you are stuck with a JSON library that does not support 64
bit signed/unsigned integers). For the second issue, you most likely
need some support for 64 bit signed/unsigned integer arithmetic,
regardless whether the JSON encoding was a string or a number.

And it might be that for a number of applications, the price for the
second may be higher than for the first. Of course, if all your
application does is to show raw data to a human, then the second may
not be an issue. But then, showing a single counter reading is almost
always pretty useless.

/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 19 Nov 2014 10:08:06 +0100

> regardless whether the JSON encoding was a string or a number.

It would seem GMP (http://gmplib.org) could solve the second issue. These benchmarks are rather impressive:

http://www.wilfred.me.uk/blog/2014/10/20/the-fastest-bigint-in-the-west/

Lada

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 19 Nov 2014 10:30:55 +0100

> http://www.wilfred.me.uk/blog/2014/10/20/the-fastest-bigint-in-the-west/

Lada,

you seem to miss the point of my message. I did not claim that it
can't be done right with sufficient effort. That said, I am not sure
my web browser has a java interface for GMP - so doing it right in a
web browser likely limits your options to a "slow" implementation.

To be clear: I am afraid of applications that end up using doubles
without anybody noticing that this is problematic until the numbers
eventually hit the precision limit. A simple DDOS on a box to make
its management application fail would be a disaster.

My main concern is interoperability and robustness and I rate them
higher than 'beauty'. This is where we often seem to differ.

/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 19 Nov 2014 11:01:39 +0100

> web browser likely limits your options to a "slow" implementation.

Well, you said “number of applications”, not “my web browser”.

> its management application fail would be a disaster.

I agree, but it is the first of your issues that creates this problem. As soon as you can properly parse big numbers in the programming language of choice, the rest is IMO not that difficult.

> higher than 'beauty'. This is where we often seem to differ.

Note that I do agree on keeping 64-bit numbers as strings in JSON. The problem is bigger than that, though, because such numbers are also incompatible with XPath 1.0.

Lada