Subject: [netmod] octal numbers

From: Andy Bierman <[email-masked]>
Date: Fri, 12 Dec 2008 09:31:04 -0800

Hi,

I just noticed that sec. 9.2.1 says that octal numbers are encoded
as a zero followed by more octal digits.  But an integer value
is a sign (+/-) followed by a sequence of decimal digits.
These rules overlap. E.g.:

   052 octal is a legal representation of 52 decimal.

Do we really need to support octal numbers in YANG?
I mean, talk about legacy.  There are already 10 numeric
data types, and plenty of ways to specify defaults for them
with decimal, hexadecimal, and floating point formats.

If yes, then a deterministic prefix is needed instead of
what is there now. IMO, octal should be removed from YANG.

BTW, it might be good to mention in this section that
these number formats do not apply to XPath expressions.
Those are IEEE 754 FP numbers, not YANG numbers (no hex!)

Andy

_______________________________________________
netmod mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/netmod

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Fri, 12 Dec 2008 19:33:03 +0100

> what is there now. IMO, octal should be removed from YANG.

SMIng supported numbers following precisely the C language standard
and standard conversion library functions such as strtod() etc. I
believe this a reasonable choice. And in C, an integral number
starting with a leading zero is considered to be an octal number. In
other words, 052 is not a legal representation of 52 decimal and your
C compiler is going to reject 082 as a constant.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>
_______________________________________________
netmod mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/netmod

---

From: "Natale, Bob" <[email-masked]>
Date: Fri, 12 Dec 2008 13:34:33 -0500

+1

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Juergen Schoenwaelder
Sent: Friday, December 12, 2008 1:33 PM
To: Andy Bierman
Cc: NETMOD Working Group
Subject: Re: [netmod] octal numbers

/js

---

From: Andy Bierman <[email-masked]>
Date: Fri, 12 Dec 2008 11:08:03 -0800

> C compiler is going to reject 082 as a constant.

Well the YANG spec needs to be fixed then.
Doesn't seem like a great use-case to me though.
The ABNF does not mention octal numbers at all.
It does show that leading zeros are not allowed
on decimal numbers, but "-0" is valid. (I thought that was FP only).
The text in 9.2.1 should be fixed so it matches the ABNF.

Andy

---

From: David Partain <[email-masked]>
Date: Mon, 15 Dec 2008 16:06:15 +0100

> The text in 9.2.1 should be fixed so it matches the ABNF.

Looks like a good catch by Andy.  Let's make it consistent throughout the 
document, one way or another.  I personally see no reason to disallow octals, 
so I'd prefer that the doc be made consistent to include them.

David

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 15 Dec 2008 16:41:00 +0100 (CET)

> so I'd prefer that the doc be made consistent to include them.

Note that octal and hexadecimals can be used only for default values
in the YANG module, NOT in NETCONF PDUs.  The reason for not allowing
them on the wire is compatibility with XSD and RNG.  But this maybe
makes the use case for octals and hexadecimals pretty minor...

Anyway, the grammar today just says:

  default-stmt           = default-keyword sep string stmtend

There is no grammar specifying what the details of the default value
string (it will depend on the type).  The rule 'decimal-value' is used
in ranges only.

So I think the current text and grammar is consistent.

/martin

---

From: David Partain <[email-masked]>
Date: Mon, 15 Dec 2008 16:54:29 +0100

On Monday 15 December 2008 16.41.00 Martin Bjorklund wrote:
> > document, one way or another.  I personally see no reason to disallow
> > octals, so I'd prefer that the doc be made consistent to include them.
>

I'm happy if others are happy....  Octal is pretty ... um ... ancient (but so 
am I), so I don't think this is a big deal :-)

David

---

From: Andy Bierman <[email-masked]>
Date: Mon, 15 Dec 2008 08:01:44 -0800

> So I think the current text and grammar is consistent.

There is no ABNF for an octal or hexadecimal numbers.
Sec. 9.2.1 could be more clear that:

   - a leading zero in a number without a decimal point
     means it MUST be interpreted as an octal number

   - XML and XPath encoding of numbers are always IEEE 754 floating point.
     This section is only for YANG numbers in default-stmt strings.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 15 Dec 2008 23:41:57 +0100 (CET)

>      means it MUST be interpreted as an octal number

If we want to be compatible with XSD/RNG, we need to allow leading
zeros in the non-canonical form.  So we'll have to say that when
specifying default values, leading zeros are not allowed for decimal
numbers.

>    - XML and XPath encoding of numbers are always IEEE 754 floating point.

XPath 1.0 always interprets numbers as 64 bit floating point, but I
don't think we need to say anything about that here.  The XML encoding
is specified in the first paragraph of 9.2.1.

>      This section is only for YANG numbers in default-stmt strings.

The second paragraph of 9.2.1 is only for defaults.  The first is for
XML as well.  Maybe we should put the default stuff in its own
subsection, to make the distinction clear.

/martin

---

From: "Randy Presuhn" <[email-masked]>
Date: Tue, 16 Dec 2008 11:17:11 -0800

Hi -

> From: "Martin Bjorklund" <[email-masked]>
> To: <[email-masked]>
> Cc: <[email-masked]>
> Sent: Monday, December 15, 2008 2:41 PM
...
> is specified in the first paragraph of 9.2.1.
...

What happens when the bit pattern corresponding to the octal
(or hex) number would be a denormalized floating point value?

Would a comparison succeed when performed against another
value that happens to equal what the denormalized value would
become if subjected to floating point normalization?

Randy

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 16 Dec 2008 22:48:51 +0100 (CET)

> (or hex) number would be a denormalized floating point value?

I don't understand what you mean.  But note that the octal and hex
notation is merely syntactic sugar for default values specified in the
YANG module.  They are never used on-the-wire.

/martin

---

From: "Randy Presuhn" <[email-masked]>
Date: Tue, 16 Dec 2008 14:15:02 -0800

Hi -

> To: <[email-masked]>
> Cc: <[email-masked]>
> Sent: Tuesday, December 16, 2008 1:48 PM

> YANG module.  They are never used on-the-wire.

Tutorial picked at random:
http://www.randelshofer.ch/fhw/gri/float.html#sectionnormalizednumbers
http://www.randelshofer.ch/fhw/gri/float.html#chapterencodings

The point is that several different 64-bit patterns (whether specified in hex
or in octal) may correspond to exactly the same floating point number.
Still other 64-bit patters will correspond to things which are not numbers at all.
This will cause great difficulty in performing comparisons with things
that are integers that happen to have these bit patterns, much less actually
using the value as configuration data.

Randy

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 16 Dec 2008 23:20:14 +0100 (CET)

> or in octal) may correspond to exactly the same floating point

But the octal and hexadecimal form are only used to specify default
values for integer types.  They are not used for floating point
numbers.

/martin

---

From: "Randy Presuhn" <[email-masked]>
Date: Tue, 16 Dec 2008 14:32:19 -0800

Hi -

> Sent: Tuesday, December 16, 2008 2:20 PM

Ok.  I'll assume the earlier comments about Xpath using floating point
are a red herring.

Randy

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 16 Dec 2008 23:39:27 +0100 (CET)

Hi,

> Ok.  I'll assume the earlier comments about Xpath using floating point

I just wanted to understand Andy's comment:

I probably still don't understand what change to the text he was
looking for.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Tue, 16 Dec 2008 18:11:54 -0800

> I probably still don't understand what change to the text he was

I am trying to clarify the text.
If a smart guy like Randy can get confused by the YANG vs. XPath
expression number representations, then lots of people will get confused.

http://www.w3.org/TR/xpath#numbers

BTW, it is not just the default-stmt.
Numbers also show up in min-elements, max-elements, ranges,
and patterns.

Inside XPath strings like the must-stmt or when-stmt argument,
the YANG rules do not apply, just the XPath rules.

Andy

---

From: "Randy Presuhn" <[email-masked]>
Date: Tue, 16 Dec 2008 22:15:16 -0800

Hi -

> From: "Andy Bierman" <[email-masked]>
> To: "Martin Bjorklund" <[email-masked]>
> Cc: <[email-masked]>; <[email-masked]>
> Sent: Tuesday, December 16, 2008 6:11 PM
...
> the YANG rules do not apply, just the XPath rules.

I admit to being rather confused here.  If XPath uses IEE 754, then
there are plenty of 64-bit integers that cannot be represented, or,
worse still, will be treated as being equal to other 64-bit integers
that have different bit patterns.  This would seem to be a potential
trap for the unwary. Limiting the use of integer values to only those
values that will survive the IEEE 754  <-> integer round trip unscathed
seems a scary prospect.

Randy

---

From: "tom.petch" <[email-masked]>
Date: Thu, 18 Dec 2008 21:47:28 +0100

Having verified that XPath 1.0 does indeed specify that IEEE 754 shall be used
when comparing numbers (and warns that this may lose precision when converting
from decimal) I thought I would look up the precise text of IEEE 754.

Oh dear.

Google 'IEEE 754' brings up a sponsored link to the ANSI web site promising a
free download of IEEE 754.  Click on it and you do indeed get the ANSI web site
with a message that what you have selected is not available and suggests
modifying your search (whatever that was).

Go to IEEE and you find that IEEE 754 was reissued in August 2008 and so is not
yet out of purdah.  The 1985 version that XPath references is listed, .pdf and
size, but click on it and  .... 'not available'.  Try for the summary and ... ,
'not available'

Oh dear.  Perhaps we need a deviation from XPath 1.0  (I note that XPath 2.0 -
yes, noone implements it but as has been pointed out several times on this list,
we do not need an implementation - has dropped all reference to IEEE 754 and
adopted the common sense approach to comparisons that most other languages I
know of use) like the adoption of XPath 2.0

Tom Petch

---

From: Andy Bierman <[email-masked]>
Date: Thu, 18 Dec 2008 14:21:36 -0800

> Subject: Re: [netmod] octal numbers

Juergen pointed out the compelling factor in this area,
which is whether widely and freely available software
libraries exist or not.  If glibc FP functions don't
do the right thing, I am not writing my own,
so that better be good enough.  Nobody is going to
write custom FP code for YANG.  Get real (sic).

Andy

---

From: "Randy Presuhn" <[email-masked]>
Date: Thu, 18 Dec 2008 15:05:05 -0800

Hi -

> To: "tom.petch" <[email-masked]>
> Cc: "Randy Presuhn" <[email-masked]>; <[email-masked]>
> Sent: Thursday, December 18, 2008 2:21 PM
...
...

I heartily agree that requiring custom floating point code is
a non-starter.  The problem is that requiring comparisons
to be done in floating point may not give the results that
most humans would expect.  Your epsilon may vary.  :-)

Randy

---

From: "tom.petch" <[email-masked]>
Date: Sat, 20 Dec 2008 12:04:54 +0100

---- Original Message -----
From: "Randy Presuhn" <[email-masked]>
To: <[email-masked]>
Sent: Friday, December 19, 2008 12:05 AM
Subject: Re: [netmod] octal numbers

> most humans would expect.  Your epsilon may vary.  :-)

And I would at least like to understand what IEEE 754-1885 has to say about the
conversion, to understand what comparisons may produce, especially since we may
be losing precision in the decimal to FP conversion.  There may be no need to
write code but I think that the lack of ready availability of this document is
an issue.

Perhaps we should make glibc a normative reference:-)

Tom Petch

---

From: "Randy Presuhn" <[email-masked]>
Date: Sat, 20 Dec 2008 11:02:07 -0800

Hi -

> From: "tom.petch" <[email-masked]>
> To: "Randy Presuhn" <[email-masked]>; <[email-masked]>
> Sent: Saturday, December 20, 2008 3:04 AM
...
...

http://en.wikipedia.org/wiki/IEEE_754-1985 provides the important information.
It's fairly easy to describe some cases where information will *inherently* be
lost be the conversion, simply due to the limitations of the format, regardless
of the specific algorithm used to perform the conversion.  For example, for
positive integers, whenever there are more than 50 bits between the highest-
and lowest-order "1" bits, information will be lost in the conversion to floating point.

So, in that universe, 
   2**52 == (2**52) + 1;
   2**53 == (2**53) + 1 == (2**53) + 2 == (2**53) + 3;
   2**54 ==  (2**54) + 1 == (2**54) + 2 == (2**54) + 3 ==  (2**54) + 4
        == (2**54) + 5  == (2**54) + 6 == (2**54) + 7;

You see where this leads.  Of course, if you don't need to compare large
numbers precisely or manipulate thresholds for 64-bit counters using
this format, it'll be a non-issue.

Describing *all* the cases where information is lost is a bit trickier, since that
requires thinking about the specifics of the conversion algorithm and whether
normalization is enforced.  If you go beyond just talking about integer conversions,
and look at the general problem of decimal conversions / comparisons, then
life gets really interesting.  Recall that some useful numbers like decimal
0.1 cannot be represented *exactly* in this format.  Your epsilon may vary.  :-)

Randy

---

From: Andy Bierman <[email-masked]>
Date: Sat, 20 Dec 2008 11:42:00 -0800

> and lowest-order "1" bits, information will be lost in the conversion to floating point.

This is where the Product Manager would send all the engineers
off for psychiatric testing. ;-) We're building a CM system.
In all my years of MIB Doctoring, I have never even gotten
1 request for a floating point MIB object.  I heard of one in
the IETF once, to represent probabilities from 0 to 1.

I do not think vendors will go out of their way to
follow XPath 1.0 to the letter, just for YANG.

I can envision some router vendors who think their embedded
systems do not have enough code room to add a lot of FP library
support -- for must/when XPath evaluation that rarely gets
used at all, let alone used with numbers like 2**52, or 1.003501,
or real floating point math like (7.0303 * (1 div 9)).

If we want to make it really difficult for vendors to
adopt NETCONF and NETMOD then we should make a big deal
out of this.  We could just say "follow XPath 1.0"
and no more.  I would not want vendors to think:
"We cannot possibly conform to IEEE 754, so we cannot
conform to NETMOD/NETCONF.  Since the main reason
we are doing this work is to be standards-based,
there is no point in choosing NETMOD/NETCONF. Let's
go a different direction with XML."

Andy

---

From: "Randy Presuhn" <[email-masked]>
Date: Sat, 20 Dec 2008 12:20:18 -0800

Hi -

> To: "Randy Presuhn" <[email-masked]>
> Cc: <[email-masked]>
> Sent: Saturday, December 20, 2008 11:42 AM
...
> the IETF once, to represent probabilities from 0 to 1.

Yup.  Though there *are* cases where floating (or, more likely,
fixed) point numbers might be useful, to treat them as the
basis for comparison and other operations on things that
are really integers does not make sense.  Like it or not,
computers operate in the world of discrete mathemetics,
and floats are just a (very very useful and valuable) hack,
not some magical way of connecting to the world of
continuous mathematics.
 
> follow XPath 1.0 to the letter, just for YANG.

Yup.  A sanity check may be in order.

> or real floating point math like (7.0303 * (1 div 9)).

2**52 isn't so far-fetched if someone needs to be able to
specify thresholds on zero-based 64-bit counters.  But
I don't know how big an "if" that would be in the netconf/netmod world.

> go a different direction with XML."

I agree this may be a teapot tempest.
There are several possibilities to consider:

   - the cases where IEEE 754 comparisons will produce
     "surprising" answers don't matter because these sorts
     of values will never be used in models or data stores
     in places where XPath comparison with those values
     will happen

   - though the answers might be "surprising" to the naive,
     they're really not a problem

   - users of netmod applications *do* understand the implications
     of IEEE 754, and will design their models and XPath
     expressions accordingly

   - implementations don't really follow IEEE 754 rules when doing
     comparisons of things that are known to really be integers

I'm sure there are others, but someone needs to sort this out, to
avoid allowing it to turn into FUD.

Randy

---

From: Andy Bierman <[email-masked]>
Date: Sat, 20 Dec 2008 12:48:59 -0800

> avoid allowing it to turn into FUD.

Experience shows that the IESG would be obliged to assist us,
if we wanted to make a big deal out of this.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Sat, 20 Dec 2008 15:36:47 -0800

> I agree this may be a teapot tempest.

It needs some attention.
XPath 1.0 clearly requires floating point math support.

If NETCONF agents which support :xpath capability
actually do the FP math, then NETMOD has no issue.
I wonder if that is the case.

The YANG draft claims that an agent vendor
could choose to implement a must or when expression
with some 'hard-wired' internal code which emulates XPath
in every way.  I suppose the draft already covers
this problem with that text.  Given that this feature
will rarely get used, it is even reasonable.

(Can we choose as many answers as we want from the menu below? :-)

Andy

---

From: "tom.petch" <[email-masked]>
Date: Sun, 21 Dec 2008 11:51:45 +0100

----- Original Message -----
From: "Randy Presuhn" <[email-masked]>
Sent: Wednesday, December 17, 2008 7:15 AM

Randy

Not sure if you are now clear on this but my reading is  ..

Within expressions, XPath 1.0 allows integer and decimal

[30]    Number    ::=    Digits ('.' Digits?)?     | '.' Digits
[31]    Digits    ::=    [0-9]+

but insists that everything is converted to double within function or operation
so comparing two 100 digit numbers will first convert to double according to
IEEE 754-1985 (sigh) rules.

XPath 2.0 allows integer, decimal and double but only requires conversion if it
is required, so two 100 digit decimal numbers are compared as decimal numbers,
no conversion.

Neither version allows octal or hexadecimal.

Tom Petch

---

From: Andy Bierman <[email-masked]>
Date: Sun, 21 Dec 2008 09:15:43 -0800

> is required, so two 100 digit decimal numbers are compared as decimal numbers,

As Phil noted, XPath 2.0 is twice is big and at least
twice as complicated as XPath 1.0. IMO, a non-starter for YANG.

Our #1 priority is building a workable solution,
which can be used by vendors and their customers to
migrate from an ad-hoc proprietary CLI CM system
to a significantly automated standard XML CM system.

Let's try not to forget that as we work on Step Three out of twenty.
If we get this part wrong, there probably won't be any more steps.

I think the co-Chairs should figure out what to do from
here, because we have run the course on this thread.

Andy

---

From: "tom.petch" <[email-masked]>
Date: Sun, 21 Dec 2008 12:12:45 +0100

Randy

Ships in the night.

The other option I would add is to declare a deviation, now that we have
invented them, that comparisons should follow the rules of XPath 2.0 and not
XPath 1.0 (which remains the standard for all other aspects of XPath).

After all, XPath 2.0 might represent what XPath 1.0 would have been with ten
more years experience.

Tom Petch.