Subject: [core] YANG to CBOR mapping

From: peter van der Stok <[email-masked]>
Date: Thu, 19 Nov 2015 11:00:03 +0100

Hi CoOL authors.

I have looked at your section 5, and see an enormous overlap with the 
CoMI section 6. Actually the two proposals are almost completely 
interoperable, with a few exceptions. Much of the CoMI proposal is based 
on the work of Ladislav Lhotka, described in 
draft-ietf-netmod-yang-json. CoMI refers to this draft and uses it 
extensively. In the CoOL draft the yang-json draft is ignored. That is a 
pity because you are redoing much of the work already done in the 
yang-json draft. In the CoMI draft we used the results of yang-json 
draft, exchanged the YANG name by the hash value, and passed it through 
the diagnostic JSON to CBOR translator. Quite a satisfactory and elegant 
solution.
Below, I have summarized my comparison between CoMI YANG to CBOR and 
CoOL YANG to CBOR. Please check for omissions and mistakes.
Differences concern Binary byte string and Bits. The CoMI choice of CBOR 
type is derived from yang-json, and I should like to hear the opinion of 
Ladislav on this aspect.
Other differences concern decimal64, and int; but I expect that is an 
oversight in the CoOL draft.
A major difference is the encoding of lists and list instances; I 
discuss that in a separate e_mail.
Given the overlap of work and the need for the expertise of the netmod 
WG, I recommend that a YANG CBOR draft is submitted to the netmod wg and 
uses as much as possible the contents of yang-json draft. Alternatively, 
I can imagine that CBOR mapping is added to the yang-json draft if the 
author, Ladislav Lhotka, and the WGs agree with that.

Greetings,

Peter
______________________________________________________________________________
Comparison of draft veillette-core cool, denoted with CoOL
With draft vanderstok-core-comi, denoted with CoMI
And draft ietf-netmod-yang-json, denoted with yang-json
Simple YANG type can be :
Binary byte string:                   CoMI, major type 2;                
          CoOL,  major type 0
Bits:                                 CoMI, array of text;               
          CoOL, major type 0
Boolean:                              CoMI, major type 7 (20,21);        
          CoOL, major type 7 (20,21)
decimal64:                            CoMI, major type 0 (pos) and 
1(neg);        CoOL major type 0
empty:                                CoMI major type 7(22);             
          CoOL major type 7(22)
enumeration:                          CoMI, major type 0;                
          CoOL major type 0
  identityref:                         CoMI, major type 3;                
           CoOL major type 3
  int8, int16, int32, int64:           CoMI, major type 0 (pos) and 
1(neg);         CoOL major type 0
  leafref:                             CoMI, follows leaf type;           
           CoOL follows leaf type
  string:                              CoMI, major type 3,                
           CoOL major type 3
  uint8, uint16, uint32, uint64:       CoMI, major type 0;                
           CoOL major type 0

In netmod-yang-json draft  JSON objects are used:
JSON object := { name: JSON object}, where name is a string. For CoMI 
and CoOL the name can be an integer which is valid for diagnostic JSON 
used for CBOR, giving:
CBOR object := {integer: CBOR object}

Leaf:
Yang-json,  Name : value, where name is the string identifier of the 
leaf, and value of Simple YANG type
CoMI: major type 5 containing: uint64, value;      CoOL: not defined

Union:
Yang-json,  use corresponding media type for type of value
CoMI, use corresponding CBOR type; CoOL, use corresponding CBOR type

anyxml,
YANG-json, value can be of any type.
CoMI, not applicable; CoOL, can be any CBOR type

Anydate,
Yang-json, follows container
CoMI, not applicable,                           CoOL, not applicable

container,
yang-json,	 name:  JSON object
CoMI: major type 5;                             CoOL, major type 5

leaf-list
yang-json, name: [ value 1, value 2,……]
CoMI, major type 4,                             CoOL, major type 4

List
Yang-json, name:[ JSON object1, JSON object2, ….]
CoMI, major type 5 of major type 5,             CoOL, major type 4 of 
major type 5

-- 
Peter van der Stok
vanderstok consultancy
mailto: [email-masked]

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 19 Nov 2015 11:11:45 +0100

I am not CoOL but as contributor I think that the CBOR encoding of
YANG data should follow in style the JSON encoding document but it
should not normatively depend on the JSON encoding document. The
reason is modularity. There are systems that only want JSON and there
are systems that only want CBOR.

>From the NETMOD perspective, I am pretty sure CBOR is _not_ added to
the JSON encoding document. The reasons is modularity (see above) and
also the fact that the JSON document is pretty much done and several
other documents depend on it and we won't hold things off for another
N months.

Whether the CBOR encoding is done in CORE or NETMOD can be discussed.
I guess this is a decision whether this requires more CBOR expertise
or more YANG expertise. Given that this work started being discussed
in CORE, I personally would not mind to leave the work in CORE and to
ensure that NETMOD is kept updated when things become stable (e.g.,
the CBOR encoding document goes to WG last call).

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1 | 28759 Bremen | Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: Somaraju Abhinav <[email-masked]>
Date: Thu, 19 Nov 2015 10:37:15 +0000

Hi Juergen,

I agree with you about the modularity and like your proposal. However, there is one issue we need to watch out for. The CBOR spec. [RFC7049] already says something about converting from JSON to CBOR. So, we should probably be careful that we do not have two ways of getting to CBOR from YANG - via YANG to JSON to CBOR vs YANG to CBOR.

Abhinav
________________________________________
From: core [[email-masked]] on behalf of Juergen Schoenwaelder [[email-masked]]
Sent: Thursday, November 19, 2015 11:11 AM
To: [email-masked]
Cc: [email-masked]; Core
Subject: Re: [core] YANG to CBOR mapping

/js

_______________________________________________
core mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/core
________________________________________________________ The contents of this e-mail and any attachments are confidential to the intended recipient. They may not be disclosed to or used by or copied in any way by anyone other than the intended recipient. If this e-mail is received in error, please immediately notify the sender and delete the e-mail and attached documents. Please note that neither the sender nor the sender's company accept any responsibility for viruses and it is your responsibility to scan or otherwise check this e-mail and any attachments.

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 19 Nov 2015 13:30:24 +0100

Yes, good point. I did not track this. We need to check this carefully
and perhaps there is not even much left to do. But more likely we will
detect differences and then we need to figure out what to do about
them.

/js

---

From: Michel Veillette <[email-masked]>
Date: Thu, 19 Nov 2015 13:54:44 +0000

Hi Peter

Would you please explain your specific concerns about the YANG to CBOR mapping of decimal64?

YANG define decimal as follow:
   The decimal64 type represents a subset of the real numbers, which can
   be represented by decimal numerals. The value space of decimal64 is
   the set of numbers that can be obtained by multiplying a 64-bit
   signed integer by a negative power of ten, i.e., expressible as
   "i x 10^-n" where i is an integer64 and n is an integer between 1 and
   18, inclusively.

CoOL propose the use of CBOR unsigned integer

   Leafs of type decimal64 MUST be encoded using a CBOR unsigned integer
   data item (major type 0).
   Definition example [RFC7317]:

   leaf my-decimal {
     type decimal64 {
       fraction-digits 2;
       range "1 .. 3.14 | 10 | 20..max";
     }
   }

   Textual form: 257 (Represents decimal value 2.57)

   CBOR encoding: 19 0101

What is wrong in this mapping?

Regards,

Michel

-----Original Message-----
From: core [mailto:[email-masked]] On Behalf Of peter van der Stok
Sent: November-19-15 5:00 AM
To: Core <[email-masked]>
Cc: [email-masked]
Subject: [core] YANG to CBOR mapping

Hi CoOL authors.

I have looked at your section 5, and see an enormous overlap with the CoMI section 6. Actually the two proposals are almost completely interoperable, with a few exceptions. Much of the CoMI proposal is based on the work of Ladislav Lhotka, described in draft-ietf-netmod-yang-json. CoMI refers to this draft and uses it extensively. In the CoOL draft the yang-json draft is ignored. That is a pity because you are redoing much of the work already done in the yang-json draft. In the CoMI draft we used the results of yang-json draft, exchanged the YANG name by the hash value, and passed it through the diagnostic JSON to CBOR translator. Quite a satisfactory and elegant solution.
Below, I have summarized my comparison between CoMI YANG to CBOR and CoOL YANG to CBOR. Please check for omissions and mistakes.
Differences concern Binary byte string and Bits. The CoMI choice of CBOR type is derived from yang-json, and I should like to hear the opinion of Ladislav on this aspect.
Other differences concern decimal64, and int; but I expect that is an oversight in the CoOL draft.
A major difference is the encoding of lists and list instances; I discuss that in a separate e_mail.
Given the overlap of work and the need for the expertise of the netmod WG, I recommend that a YANG CBOR draft is submitted to the netmod wg and uses as much as possible the contents of yang-json draft. Alternatively, I can imagine that CBOR mapping is added to the yang-json draft if the author, Ladislav Lhotka, and the WGs agree with that.

Greetings,

Peter
Comparison of draft veillette-core cool, denoted with CoOL With draft vanderstok-core-comi, denoted with CoMI And draft ietf-netmod-yang-json, denoted with yang-json Simple YANG type can be :

In netmod-yang-json draft  JSON objects are used:
JSON object := { name: JSON object}, where name is a string. For CoMI and CoOL the name can be an integer which is valid for diagnostic JSON used for CBOR, giving:
CBOR object := {integer: CBOR object}

Leaf:
Yang-json,  Name : value, where name is the string identifier of the leaf, and value of Simple YANG type
CoMI: major type 5 containing: uint64, value;      CoOL: not defined

Union:
Yang-json,  use corresponding media type for type of value CoMI, use corresponding CBOR type; CoOL, use corresponding CBOR type

anyxml,

Anydate,

container,

leaf-list

List

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 19 Nov 2015 15:54:57 +0100

Hi Michel,

I'm not Peter, but:

{ "hat": "chair", "message": "
We get to do COMI in CoRE (we hope -- the charter is still in process)
because we have promised to work closely with the relevant network
management groups.  This means we'll need good reasons to deviate from
what NETMOD etc. already have agreed to do.
"}

So I think that Peter's reminder to stay close to
draft-ietf-netmod-yang-json -- unless we do need to deviate -- is quite
germane.

(If I were designing this from scratch, I would ask myself whether we
shouldn't stay closer to the schemaless roots of JSON, e.g. by exposing
the exponent in the representation data and not just in the YANG schema.
 But that is a discussion that maybe should be had in NETMOD, if at all.)

I think the specific observation about decimal64 is that it is based on
signed integers as the mantissae, so we need to allow negative numbers
in CBOR as well as unsigned ones.

To contrast this to a place where we may want to deviate, doing "bits"
as an (unsigned) integer sounds right (at least until we need more than
64 of them in one "bits" item).  This runs into the same numbering
issues we have with the intra-module names, though.

Grüße, Carsten

PS.: Typo:
   Leafs of type binary MUST be encoded using a CBOR byte string data
   item (major type 0).
(CBOR byte strings are major type 2.)

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 19 Nov 2015 16:27:34 +0100

> draft-ietf-netmod-yang-json -- unless we do need to deviate -- is quite

I do not understand what "stay close to draft-ietf-netmod-yang-json"
means. I think this requires an explanation.

To give you an example: draft-ietf-netmod-yang-json is following
I-JSON rules and one of them says that I-JSON numbers are limited in
precision. As a consequence, 64-bit numbers are I-JSON encoded as
strings for I-JSON compliance. I think CBOR does not require this
since it does not have this particular restriction. So do we now
continue to do this even though CBOR does not suffer from I-JSON
limitations?

I think my preference would be a clean mapping YANG to CBOR and not a
mapping YANG to I-JSON to CBOR that carries I-JSON restrictions over
to CBOR. The simple reason is that the longer the transformation
chain, the more arcane rules you accumulate.

But then the CBOR spec has already JSON to CBOR translation rules so
YANG to I-JSON to CBOR is already defined somehow. Someone needs to
compile a list of cases where YANG to CBOR would be different from
YANG to JSON to CBOR before we can take an informed decision.

/js

---

From: Michel Veillette <[email-masked]>
Date: Thu, 19 Nov 2015 15:35:40 +0000

Hi Carson

I already sent an email about the reasons why we need to define a direct mapping (YANG -> CBOR) instead of a two step mapping (YANG -> JSON -> CBOR). I hope the reasons provided are satisfactory to you.

It's important to note and realize that the actual CoMI draft already don't comply with draft-ietf-netmod-yang-json (e.g. list mapping)
The implement of bits using a CBOR byte string won’t also be aligned with draft-ietf-netmod-yang-json
Defining a two step mapping (YANG -> JSON -> CBOR) make things more complex and might not produce the most efficient encoding.

This doesn't means we design this from starch.
The draft-ietf-netmod-yang-json draft is used as a template to defined the (YANG -> CBOR) mapping.

Thanks for your comment about decimal64 and the typo.

Regrads

Michel

-----Original Message-----
From: Carsten Bormann [mailto:[email-masked]] 
Sent: November-19-15 9:55 AM
To: Michel Veillette <[email-masked]>
Cc: [email-masked]; Core <[email-masked]>; [email-masked]
Subject: Re: [core] YANG to CBOR mapping

Hi Michel,

I'm not Peter, but:

{ "hat": "chair", "message": "
We get to do COMI in CoRE (we hope -- the charter is still in process) because we have promised to work closely with the relevant network management groups.  This means we'll need good reasons to deviate from what NETMOD etc. already have agreed to do.
"}

So I think that Peter's reminder to stay close to draft-ietf-netmod-yang-json -- unless we do need to deviate -- is quite germane.

(If I were designing this from scratch, I would ask myself whether we shouldn't stay closer to the schemaless roots of JSON, e.g. by exposing the exponent in the representation data and not just in the YANG schema.
 But that is a discussion that maybe should be had in NETMOD, if at all.)

I think the specific observation about decimal64 is that it is based on signed integers as the mantissae, so we need to allow negative numbers in CBOR as well as unsigned ones.

To contrast this to a place where we may want to deviate, doing "bits"
as an (unsigned) integer sounds right (at least until we need more than
64 of them in one "bits" item).  This runs into the same numbering issues we have with the intra-module names, though.

Grüße, Carsten

PS.: Typo:

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 19 Nov 2015 11:37:31 +0100

> On 19 Nov 2015, at 11:00, peter van der Stok <[email-masked]> wrote:
> 
> Hi CoOL authors.
> 

Isn't it just an error? The CoOL draft says:

However, RFC 7049 defines major type 0 as unsigned integer.

> Given the overlap of work and the need for the expertise of the netmod WG, I recommend that a YANG CBOR draft is submitted to the netmod wg and uses as much as possible the contents of yang-json dr…

As Juergen wrote in his reply, there are good reasons not to do that.

Lada

> Binary byte string:                   CoMI, major type 2;                         CoOL,  major type 0
> Bits:                                 CoMI, array of text;                        CoOL, major type 0
> Boolean:                              CoMI, major type 7 (20,21);                 CoOL, major type 7 (20,21)
> decimal64:                            CoMI, major type 0 (pos) and 1(neg);        CoOL major type 0
> empty:                                CoMI major type 7(22);                      CoOL major type 7(22)
> enumeration:                          CoMI, major type 0;                         CoOL major type 0
> identityref:                         CoMI, major type 3;                          CoOL major type 3
> int8, int16, int32, int64:           CoMI, major type 0 (pos) and 1(neg);         CoOL major type 0
> leafref:                             CoMI, follows leaf type;                     CoOL follows leaf type
> string:                              CoMI, major type 3,                          CoOL major type 3
> uint8, uint16, uint32, uint64:       CoMI, major type 0;                          CoOL major type 0
> 
> CoMI, major type 5 of major type 5,             CoOL, major type 4 of major type 5
> 
> 
> 
> -- 
> Peter van der Stok
> vanderstok consultancy

--
Ladislav Lhotka, CZ.NIC Labs
PGP Key ID: E74E8C0C

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 20 Nov 2015 13:22:35 +0100

> On 19 Nov 2015, at 16:27, Juergen Schoenwaelder <[email-masked]> wrote:
> 

I agree, it isn't probably necessary to couple the CBOR mapping with JSON as defined in the yang-json draft. Some of the I-JSON restrictions are an ugly compromise, really, and you'd better avoid them. I believe a mapping between CBOR, JSON and XML encodings will be possible anyway.

Lada

---

From: Carsten Bormann <[email-masked]>
Date: Fri, 20 Nov 2015 14:13:34 +0100

>> means. I think this requires an explanation.

"-- unless we do need to deviate --"

>> continue to do this even though CBOR does not suffer from I-JSON

No.  Here we clearly need to deviate to reduce unnecessary complexity.

>> chain, the more arcane rules you accumulate.

I completely agree.  What I was trying to say was that the two mappings
should stay close, at least where the mappings can reasonably use the
common data model of JSON and CBOR.

The one place where we probably will have additional divergence is the
area of identifiers/selectors, where for COMI the CBOR mapping will need
to provide more flexibility.  Can we decouple this aspect from the other
elements of the YANG-CBOR mapping?  I hope so, but we haven't
demonstrated that yet.

>> YANG to I-JSON to CBOR is already defined somehow. 

... and the result of this two-step process might be useful as a
"default" mapping, which we use unless there is a reason to deviate.

>> Someone needs to

Indeed, and this is work that needs to be done very soon.

Do we need to build a formal design team for this work around the
extraction of the YANG-to-CBOR mapping from the COMI (and COOL) documents?

Grüße, Carsten