Subject: [core] yang cbor comments

From: peter van der Stok <[email-masked]>
Date: Thu, 17 Nov 2016 11:06:30 +0100

Hi ief-yang-cbor authors,

Reading this document I see that it is very necessary. Thanks for doing 
this work.

I did find some unclear parts, that I indicate below
Hope this helps,

peter
_____________________________________________________
Ietf-yang-cbor review.

Page 4: delta; not only difference with the parent but also difference 
with predecessor in a CBOR array.
Page 5 is this “#” a pound sign?
Page 6, line 8, s/key type/key types/
Page 6, line 12 s/RESFCONF/RESTCONF/
Page 6. Line 14 s/mapping avoid/mapping avoids/
Page 6, line 15 used solely for union?; there are 4 tags defined.
Pag 6 the last paragraph of section 3 can be removed, does not clarify 
anything for me.
Page 6 last line add: "according to the instance type."
Example page 8; the leaf hostname is missing in the diagnostic CBOR and 
CBOR (twice for names and SIDs)
Section 4.3, why is the search identifier missing in the example?
Page 10: s/encoding a/encoding of/
Page 10, line 9, s/enties/entries/
Page 10 “it is important…..encoded data” appears twice.
Page 12 and 13, what happened to the server identifier in the example?
Page 14-15; It is not clear to me how you can specify SIDs for the 
contents of anydata and anyxml. Anything may happen, including new 
identifiers, and they change dynamically. I can understand that the 
anyxml, anydata node has a SID, but I should like to see how the nodes 
inside the anyxml or Anydata instance are labelled with SIDs by the 
server.
Examples of section 5. Do the examples not need an identifier (CBOR 
key)? We now are faced with values which do not belong anywhere. Also 
the SID value needs to be specified.
For example section 5.1 can be written as follows:
The leaf mtu is assumed to have value 1280
The sid of mtu is 1350
The CBOR diagnostic notation of the pair is 1350: 1280, which results in
Page 15 /signed integer/negative integer/
Section 5.10.1; it is unclear where the value 1180 comes from.
Page 23 below -3: s/identify/identifies/
Section 5.13 is very unclear. There are many examples but it is not 
clear where all these values come from; what is the SID value, what was 
the leaf instance value, is one large puzzle. One example (instead of 
3), but with more explanations would help. In my opinion the text in 
belongs in the FETCH content format document that still needs to be 
written.

-- 
Peter van der Stok

---

From: Michel Veillette <[email-masked]>
Date: Thu, 17 Nov 2016 18:17:09 +0000

Hi Peter

My answers inline

New version available at: 
https://github.com/core-wg/yang-cbor/blob/master/draft-ietf-core-yang-cbor-latest.md

Diff available at:
https://github.com/core-wg/yang-cbor/commit/45226eda881e45ae6ca62c6dd4ee4b2509bc8820#diff-083fd5e71ffd63b12d8cd2854c3ad1bf 

Regards,
Michel

-----Original Message-----
From: core [mailto:[email-masked]] On Behalf Of peter van der Stok
Sent: Thursday, November 17, 2016 5:07 AM
To: Core <[email-masked]>
Subject: [core] yang cbor comments

Reading this document I see that it is very necessary. Thanks for doing this work.

I did find some unclear parts, that I indicate below Hope this helps,

peter

====> Page 4: delta; not only difference with the parent but also difference with predecessor in a CBOR array.

[I-D.core-yang-cbor] only make use of parent delta, sibling delta is used by the root collection of some CoMI methods.
Should we have a common definition of delta for both documents or should we tailor the definition for each one?
Which approach will be the less confusing?

====> Page 5 is this “#” a pound sign?

'#' is the comment delimiter currently used by [I-D.core-yang-cbor].
Is there a new standardized way to add comments to CBOR diagnostic notation we can use?

====> Page 6, line 8, s/key type/key types/

Fixed

====> Page 6, line 12 s/RESFCONF/RESTCONF/

Was already fixed in GIT

====> Page 6. Line 14 s/mapping avoid/mapping avoids/

Fixed

====> Page 6, line 15 used solely for union?; there are 4 tags defined.

Not sure what is the issue here.
The sentence start by "CBOR tags" plural to account for the fact we support multiple tags.
However, I'm adding anyxml which also require the use of tags.
"For instance, CBOR tags are used solely in the case of anyxml data nodes and the union datatype to distinguish ..."

====> Page 6 the last paragraph of section 3 can be removed, does not clarify anything for me.

Removed

====> Page 6 last line add: "according to the instance type."

Adding "according to the instance datatype "

====> Example page 8; the leaf hostname is missing in the diagnostic CBOR and CBOR (twice for names and SIDs) Section 4.3, why is the search identifier missing in the example?

Removing hostname form the definition.
The ietf-system, hostname is in a completely different part of the module.
See https://github.com/core-wg/yang-cbor/blob/master/registry/ietf-system%402014-08-06.tree 

====> Page 10: s/encoding a/encoding of/

Fixed

====> Page 10, line 9, s/enties/entries/

Fixed

====> Page 10 “it is important…..encoded data” appears twice.

Removed

====> Page 12 and 13, what happened to the server identifier in the example?

None of the YANG Data Node Instances defined in section 4 (leaf, container, leaf-list, list, anydata, anyxml) is encoded with its identifier.
Such identifiers are part of the instance-identifier carry in the request URI or within the payload.

====> Page 14-15; It is not clear to me how you can specify SIDs for the contents of anydata and anyxml. Anything may happen, including new identifiers, and they change dynamically. I can understand that the anyxml, anydata node has a SID, but I should like to see how the nodes inside the anyxml or Anydata instance are labelled with SIDs by the server.

anyxml as defined by https://tools.ietf.org/html/rfc7951#section-5.6 and by [I-D.core-yang-cbor] can't have inner data node.
An anyxml is very similar to a leaf/leaf-list with an associated identifier (SID) and an arbitrary content.

Adding clarification about tagging and an example.

    anyxml value may contain CBOR data items tagged with one of the tag listed in {{tag-registry}}, these tags shall be supported.

    The following example shows a valid CBOR encoded instance.

    Definition example from {{RFC7951}}:

    anyxml bar;

    CBOR diagnostic notation: [true, null, true]

    CBOR encoding: 83 f5 f6 f5

Anydata however can have inner data nodes.
The assignment of SIDs to these inner data nodes is outside the scope of [I-D.core-yang-cbor].

Possible examples:
These inner data nodes might be defined elsewhere in the data model, this will be the case of a event logger.
Each entry of an event logger need to be defined with an anydata and will contain notification data nodes which already have associated SIDs.

Alternatively, a range of registered SIDs can be used to tag these inner data nodes.
The meaning of these SIDs can be retrieved from the device or documented out of band.

Adding an example:

   The following example shows a possible use of anydata. In this example, the anydata is used to define
   data node containing a modification event, this data node can be defined within an YANG list to create an event logger.

   Definition example:

   anydata event;

   CBOR diagnostic notation:

   {
     2601 : "0/4/21",       # port-name
     2602 : "Open pin 2"    # port-fault
   }

   CBOR encoding:

   a2                         # map(2)
      19 0a29                 # unsigned(2601)
      66                      # text(6)
         302f342f3231         # "0/4/21"
      19 0a2a                 # unsigned(2602)
      6a                      # text(10)
         4f70656e2070696e2032 # "Open pin 2"

====> Examples of section 5. Do the examples not need an identifier (CBOR key)? We now are faced with values which do not belong anywhere. Also the SID value needs to be specified. For example section 5.1 can be written as follows:
The leaf mtu is assumed to have value 1280 The sid of mtu is 1350 The CBOR diagnostic notation of the pair is 1350: 1280, which results in Page 15 /signed integer/negative integer/

Same answer as above.
None of the YANG Data Node Instances defined in section 4 (leaf, container, leaf-list, list, anydata, anyxml) is encoded with its identifier.
Such identifiers are part of the instance-identifier carry in the request URI or within the payload.
This is the job of CoMI

 ====>Section 5.10.1; it is unclear where the value 1180 comes from.

Adding
as listed in [email-masked]'.

====> Page 23 below -3: s/dentify/identifies/

Fixed

====> Section 5.13 is very unclear. There are many examples but it is not clear where all these values come from; what is the SID value, what was the leaf instance value, is one large puzzle. One example (instead of 3), but with more explanations would help. In my opinion the text in belongs in the FETCH content format document that still needs to be written.

The multiple examples have been requested by different peoples including Carsten.
These examples show completely different use cases, single instance data node, list and list instance and are all useful.
For the question, "what is the SID value", this will be resolved by the implementation of the registrars sharing a common database (blockchain) of .sid files.

--
Peter van der Stok

_______________________________________________
core mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/core

---

From: Carsten Bormann <[email-masked]>
Date: Fri, 18 Nov 2016 05:53:52 +0900

> On 18 Nov 2016, at 03:17, Michel Veillette <[email-masked]> wrote:
> 

Standardized: no; RFC 7049 follows JSON and does not have comments.

But draft-greevenbosch-appsawg-cbor-cddl-09.txt Appendix G.5 proposes using “/“ as both start and end of comments (we have found that comments are often best used inline).  This is used in a number of documents already, e.g. the COSE specification draft-ietf-cose-msg that has almost completed IESG processing now.

Grüße, Carsten

---

From: Carsten Bormann <[email-masked]>
Date: Fri, 18 Nov 2016 05:56:47 +0900

On 18 Nov 2016, at 05:53, Carsten Bormann <[email-masked]> wrote:
> 
> draft-greevenbosch-appsawg-cbor-cddl-09.txt Appendix G.5 

(Forgot to add convenience link:)

https://tools.ietf.org/html/draft-greevenbosch-appsawg-cbor-cddl-09#appendix-G.5

Grüße, Carsten

---

From: Carsten Bormann <[email-masked]>
Date: Fri, 18 Nov 2016 06:22:17 +0900

In a hallway discussion yesterday, we discussed two invariants that I think we all had in our minds but probably should be made very explicit in the document.

1) SID delta processing when generating or consuming YANG-CBOR instances is pure arithmetic.  There is never a need to consult a YANG module or other table when performing the delta processing.

2) SID generation results in a SID file that is preserved for a version of a YANG module.  The generation of SIDs for the next version uses this SID file as an input.  So it never can happen that the SID file for the next version becomes incompatible with the previous version: All SIDs that were assigned previously still are valid in the next version (of course they may not mean that much anymore if the thing identified by the SID is being removed).

Now, the remaining question mark I have about 2 is what happens when one implementer starts generating SIDs for version 2 of a module, then for version 4; but another one already has SIDs for version 1 and then version 3.  That should be explained, too.

Grüße, Carsten

---

From: peter van der Stok <[email-masked]>
Date: Thu, 17 Nov 2016 23:33:58 +0100

Yes, but for the SID document.

Peter

Carsten Bormann schreef op 2016-11-17 22:22:
> In a hallway discussion yesterday, we discussed two invariants that I
> think we all had in our minds but probably should be made very
> explicit in the document.
> 
> 1) SID delta processing when generating or consuming YANG-CBOR
> instances is pure arithmetic.  There is never a need to consult a YANG
> module or other table when performing the delta processing.
> 
> 2) SID generation results in a SID file that is preserved for a
> version of a YANG module.  The generation of SIDs for the next version
> uses this SID file as an input.  So it never can happen that the SID
> file for the next version becomes incompatible with the previous
> version: All SIDs that were assigned previously still are valid in the
> next version (of course they may not mean that much anymore if the
> thing identified by the SID is being removed).
> 
> Now, the remaining question mark I have about 2 is what happens when
> one implementer starts generating SIDs for version 2 of a module, then
> for version 4; but another one already has SIDs for version 1 and then
> version 3.  That should be explained, too.
> 
> Grüße, Carsten
>

---

From: peter van der Stok <[email-masked]>
Date: Fri, 18 Nov 2016 01:15:22 +0100

See inline
> 
> ====> Page 4: delta; not only difference with the parent but also
> difference with predecessor in a CBOR array.
> 
> [I-D.core-yang-cbor] only make use of parent delta, sibling delta is
> used by the root collection of some CoMI methods.
> Should we have a common definition of delta for both documents or
> should we tailor the definition for each one?

I expect the CoMI document to use the definition of YANG-CBOR doc.
> Is there a new standardized way to add comments to CBOR diagnostic
"#" is OK, it is called differently: e.g. sharp but not pound.

> "For instance, CBOR tags are used solely in the case of anyxml data
> nodes and the union datatype to distinguish ..."

It is the word "solely"; I was afraid it remained from earlier versions.

> ====> Example page 8; the leaf hostname is missing in the diagnostic
> CBOR and CBOR (twice for names and SIDs) Section 4.3, why is the
> search identifier missing in the example?
> 
> See
> https://github.com/core-wg/yang-cbor/blob/master/registry/ietf-system%402014-08-06.tree

Can this be adapted, It is confusing to know what belongs to example and 
what not

> None of the YANG Data Node Instances defined in section 4 (leaf,
> container, leaf-list, list, anydata, anyxml) is encoded with its
> identifier.

Again, I find it confusing; Some text is needed to parse the examples. 
Say that only the value of the pair is shown.

> ====> Examples of section 5. Do the examples not need an identifier
> (CBOR key)? We now are faced with values which do not belong anywhere.
> Also the SID value needs to be specified. For example section 5.1 can
> be written as follows:
> The leaf mtu is assumed to have value 1280 The sid of mtu is 1350 The
> CBOR diagnostic notation of the pair is 1350: 1280, which results in
> None of the YANG Data Node Instances defined in section 4 (leaf,
> container, leaf-list, list, anydata, anyxml) is encoded with its
> identifier.

I seem to express myself badly.
It is only needed to say that mtu is assumed to have value 1280  and 
that the CBOR encoding of 1280 is:

At this moment the scope of the example CBOR code is unclear.

> ====> Section 5.13 is very unclear. There are many examples but it is
> not clear where all these values come from; what is the SID value,
> what was the leaf instance value, is one large puzzle. One example
> (instead of 3), but with more explanations would help. In my opinion
> the text in belongs in the FETCH content format document that still
> needs to be written.
> 
> These examples show completely different use cases, single instance
> data node, list and list instance and are all useful.
> For the question, "what is the SID value", this will be resolved by
> the implementation of the registrars sharing a common database
> (blockchain) of .sid files.

please see my responses above.
Not the examples are questioned but an explanation is needed that says 
that the CBOR code for the value is shown and not the whole leaf.

---

From: Michel Veillette <[email-masked]>
Date: Tue, 22 Nov 2016 17:59:12 +0000

Hi Peter

Answers [MV] inline
Updated draft is available on GIT.
http://core-wg.github.io/yang-cbor/draft-ietf-core-yang-cbor-latest.txt

Regards,
Michel

-----Original Message-----
From: peter van der Stok [mailto:[email-masked]] 
Sent: Thursday, November 17, 2016 7:15 PM
To: Michel Veillette <[email-masked]>
Cc: [email-masked]; Core <[email-masked]>
Subject: RE: [core] yang cbor comments

See inline

[MV] Definition updated with the one defined in:
[MV] https://tools.ietf.org/html/draft-ietf-core-sid-00#section-2 
 [MV] delta : Difference between the current SID and a reference SID. A reference SID is defined for each context for which deltas are used.

[MV] Comment delimiters have been updated to be aligned with 
[MV] https://tools.ietf.org/html/draft-greevenbosch-appsawg-cbor-cddl-09#appendix-G.5
[MV] Any text within and including a pair of slashes is considered a comment."
[MV] Examples can now be copy/pasted to http://cbor.me/ without errors.

[MV] Already fixed in git

> https://github.com/core-wg/yang-cbor/blob/master/registry/ietf-system%

Can this be adapted, It is confusing to know what belongs to example and what not

I seem to express myself badly.
It is only needed to say that mtu is assumed to have value 1280  and that the CBOR encoding of 1280 is:

[MV] in section 5.1, the example is introduced with the sentence:
[MV] "The following example shows the encoding of leaf 'mtu' set to 1280 bytes."
[MV] Is it the part that is unclear?
[MV] Can you propose an updated text to clarify this example?

please see my responses above.
Not the examples are questioned but an explanation is needed that says that the CBOR code for the value is shown and not the whole leaf.

[MV] This topic is discussed in section https://tools.ietf.org/html/draft-ietf-core-yang-cbor-03#section-3 
[MV]
[MV]   "Basic schema nodes such as leaf, leaf-list, list, anydata and anyxml can be encoded standalone.
[MV]    In this case, only the value of this schema node is encoded in CBOR. Identification of this value needs
[MV]    to be provided by some external means when required.
[MV]    A collection such as container, list instance, notification, RPC input, RPC output, action input and action
[MV]    output is serialized using a CBOR map in which each child schema node is encoded using a key and a value."
[MV]
[MV] Can you propose an updated text to clarify this topic?

---

From: Michel Veillette <[email-masked]>
Date: Tue, 22 Nov 2016 21:30:19 +0000

Hi Carsten

To clarify these items, I'm proposing to promote section 3 (YANG Schema Item iDentifier) as section 1 (Introduction). The following highlighted text have been added/updated to address these items.

Please review, comment, propose alternate text.

Regards,

Michel

_______________________________________________________________

1. Introduction

Some of the items defined in YANG [RFC7950] require the use of a unique identifier.  In both NETCONF and RESTCONF, these identifiers are implemented using names.  To allow the implementation of data models defined in YANG in constrained devices and constrained networks, a more compact method to identify YANG items is required. This compact identifier, called SID, is encoded using an unsigned integer. The following items are identified using SIDs:

o  identities

o  data nodes

o  RPCs and associated input(s) and output(s)

o  actions and associated input(s) and output(s)

o  notifications and associated information

o  YANG modules, submodules and features

To minimize its size, SIDs are often implemented using a delta from a reference SID and the current SID. Conversion from SIDs to deltas and back to SIDs are stateless processes solely based on the data serialized or deserialized.

To guaranty the uniqueness of each assigned SID, SID ranges MUST be registered. Section 7.1 provide more details about the registration process of SID range(s).

Assignment of SIDs can be automated, the recommended process to assign SIDs is as follows:

o  A tool extracts the different items defined for a specific YANG module.

o  The list of items is ordered by type and label.

o  SIDs are assigned sequentially for the entry point up to the size of the registered SID range.  This approach is recommended to minimize the serialization overhead, especially when delta encoding is implemented.

o  If the number of items exceeds the SID range(s) allocated to a YANG module, an extra range is added for subsequent assignments.

SIDs are assigned permanently, items introduced by a new revision of a YANG module are added to the list of SIDs already assigned.  This process can also be automated using the same method described above except that the assignment restart from the highest SID already assigned.

To avoid duplicate assignment of SIDs, the registration of the SIDs assigned to YANG module(s) is recommended.  Section 7.2 provide more details about the registration process of YANG modules and associated SIDs. In order to implement this registry, section 5 defines a standard file format (.sid) used to store and publish SIDs (.sid). This SID file (.sid) is the output of the SID assignment process and both the input and output of the update process.

_______________________________________________________________

-----Original Message-----
From: Carsten Bormann [mailto:[email-masked]]
Sent: Thursday, November 17, 2016 4:22 PM
To: Michel Veillette <[email-masked]>
Cc: Core <[email-masked]>
Subject: Re: [core] yang cbor comments

Grüße, Carsten

---

From: peter van der Stok <[email-masked]>
Date: Wed, 30 Nov 2016 09:28:49 +0100

Hi Michel,

Apologies for the late reaction, but I was occupied by other things, and 
almost forgot.

Let me give one example from section 5.9

The text says: The following example shows the encoding of leaf 
'interface-state-
    ref' set to the value "eth1".

However it is NOT the encoding of the leaf, but the encoding of the 
value of the leaf. (that's where my confusion comes from, sorry to seem 
to be nit-picking)

My proposal:

The following example shows the encoding of the value "eth1" assigned to 
the leaf 'interface-state-ref'.

I hope I made my point clearer.

You may start the sections 5, before section 5.1, with the following 
text:
In the folowing examples values of the discussed type are assigned to an 
example leaf of the discussed type.
Consectively, the CBOR encoding of the value is shown.
(Or similar but much better text.)

> [MV] This topic is discussed in section
> https://tools.ietf.org/html/draft-ietf-core-yang-cbor-03#section-3
> [MV]
> [MV]    A collection such as container, list instance, notification,
> RPC input, RPC output, action input and action
> [MV]    output is serialized using a CBOR map in which each child
> schema node is encoded using a key and a value."
> [MV]

---

From: Michel Veillette <[email-masked]>
Date: Wed, 30 Nov 2016 21:59:45 +0000

Hi Peter

=== About " My proposal: The following example shows the encoding of the value "eth1" assigned to the leaf 'interface-state-ref'."

All examples have been fixed with the following scheme:
   The following example shows the encoding of an 'interface-state-ref' leaf instance set to "eth1".
See diff at
   https://github.com/core-wg/yang-cbor/commit/6f514542d5d09ec5c992e095dd609c68303cdb89#diff-083fd5e71ffd63b12d8cd2854c3ad1bf

Your proposed solution works fine for this specific example but was hard to apply to all examples.
I'm also not sure about the use of "assigned" which seem to imply a previous PUT or POST.
I'm open the revisit this wording if needed.

=== About the into of section 5, I proposing the following text based on the intro present in https://tools.ietf.org/html/rfc7951#section-6 

The CBOR encoding of an instance of a leaf or leaf-list data node depends on the built-in type of that data node. The following sub-section defined the CBOR encoding of each built-in type supported by YANG as listed in [RFC7950] section 4.2.4. Each subsection shows an example value assigned to a data node instance of the discussed built-in type.

The update draft is available at http://core-wg.github.io/yang-cbor/draft-ietf-core-yang-cbor-latest.txt

Regards,
Michel

-----Original Message-----
From: peter van der Stok [mailto:[email-masked]] 
Sent: Wednesday, November 30, 2016 3:29 AM
To: Michel Veillette <[email-masked]>
Cc: [email-masked]; Core <[email-masked]>
Subject: RE: [core] yang cbor comments

Hi Michel,

Apologies for the late reaction, but I was occupied by other things, and almost forgot.

However it is NOT the encoding of the leaf, but the encoding of the value of the leaf. (that's where my confusion comes from, sorry to seem to be nit-picking)

My proposal:

The following example shows the encoding of the value "eth1" assigned to the leaf 'interface-state-ref'.

You may start the sections 5, before section 5.1, with the following
text:
In the folowing examples values of the discussed type are assigned to an example leaf of the discussed type.
Consectively, the CBOR encoding of the value is shown.
(Or similar but much better text.)