Subject: [netmod] yang next issue #46 binary encoding support

From: Juergen Schoenwaelder <[email-masked]>
Date: Fri, 29 Mar 2019 12:19:30 +0100

Hi,

this is issue is closed but I wonder whether this is correct. I have
several questions looking at the issue on github:

- Why is this not a YANG issue?
- Which workaround is better?
- Why is this tagged as a NETCONF issue?

If we want to support binary encodings, we need to allow modelers to
define which types map to a canonical binary representation in
addition to the canonical string representation. As stated in the
issue description, hard-wiring some types in the encoding
specifications is very limited.

In terms of backwards compatibility, this issue should IMHO be tagged
high (adding binary encoding for some types does not cause any
backwards compatibility problem since so far we have only strings).

While I do not have a solution proposal, I think this issue is worth
to look at and we should not close it right now.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1 | 28759 Bremen | Germany
Fax:   +49 421 200 3103         <https://www.jacobs-university.de/>

---

From: Andy Bierman <[email-masked]>
Date: Fri, 29 Mar 2019 09:07:18 -0700

> - Why is this tagged as a NETCONF issue?
Did you mean this should be NETCONF issue?
It is more of a protocol problem then a modeling problem.
The goal is to use the model unaltered.

> backwards compatibility problem since so far we have only strings).
Not so sure.
The base64 encoding could look like a valid string.
The receiver must know a binary type is being sent (XML and JSON both fail
here, but not CBOR).

> to look at and we should not close it right now.
I have a solution proposal, but I have not implemented it yet, so it it not
detailed...

Both sender and receiver need to agree on the binary encoding and how the
data is tagged as binary.

This expired draft should address that problem:
https://tools.ietf.org/html/draft-mahesh-netconf-binary-encoding-01

For every type T that they agree on, there are standard T.b2y() and T.y2b()
conversion functions.
There are also some extensions to define conversion templates so vendors
can add their own types.

The YANG modules do not need to actually be altered.  The peers will
negotiate the
set of types that will be sent as binary when the session starts.
The receiver knows T and the SID for each object, and will accept either
the YANG or binary encoding.

> /js
Andy

> _______________________________________________
> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod
>

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Fri, 29 Mar 2019 17:17:23 +0100

> The goal is to use the model unaltered.
 
I think it would be valuable if say the definition of ipv4-address
could state that a canonical binary representation is of type binary {
length 4; }. Doing this is only meaningful for some types but it would
allow to add more binary representations over time.
 
> The receiver must know a binary type is being sent (XML and JSON both fail

I am talking about CBOR, not about XML or JSON. I want to provide
hints to CBOR (or similar binary encodings) that values can be
represented in a different format. I do not expect these hints to be
used by XML or JSON. If you need binary encoding efficiency, use CBOR
instead of JSON.

> the YANG or binary encoding.

Sounds complex for me to negotiate this. I rather say once that a
binary encoding can ship an IPv6 address as type binary { length 16; }
and then CBOR will simply do the right thing.

/js

---

From: Andy Bierman <[email-masked]>
Date: Fri, 29 Mar 2019 09:30:19 -0700

> and then CBOR will simply do the right thing.
OK, but this would require new type names.
You cannot simply change some standard type to be a union with a binary
type.
This forces all implementations of that type to support the binary variant.
That breaks old clients that worked with the version before the binary
variant.

The ripple effect on the models changing types would be non-trivial.
Using this union-type approach forces every protocol to support the binary
encoding,
yet base64 in a union with strings is very error-prone.

/js

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Fri, 29 Mar 2019 19:46:24 +0100

> yet base64 in a union with strings is very error-prone.

I am not proposing do change the type definitions we have. My idea was
to have an optional additional definition for binary encodings. Here
is an ad-hoc example (I do not like the details of the syntax, but
perhaps this helps to understand the idea):

     typedef ipv4-address {
       type string {
         pattern
           '(([0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])\.){3}'
         +  '([0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])';
       }
       description
         "The ipv4-address type represents an IPv4 address in
          dotted-quad notation.";

       binary-representation {
         type binary {
	   length 4;
	 }
	 description
	   "The binary representation uses as 4-byte binary string
	    in network byte ordering.";
       }
     }

The CBOR encoder (or other binary encoders) would then encode the
value as a 4 byte binary value, the XML and JSON encoder would use the
canonical string representation.  If the binary-representation is not
specified, then the generic CBOR encoding rules apply. I assume that
additional binary representation definitions will only be needed for a
couple of types (and I might even be fine to restrict that to
typedefs). Anyway, details need work, but if we want to support more
efficient binary encodings, then I think we should keep the issue #46
open.

/js

---

From: Andy Bierman <[email-masked]>
Date: Fri, 29 Mar 2019 11:57:34 -0700

On Fri, Mar 29, 2019 at 11:46 AM Juergen Schoenwaelder <
[email-masked]> wrote:

> > > > I have a solution proposal, but I have not implemented it yet, so it
> it
> > > not
> > > > detailed...
> > > >

OK -- this is what I had in mind but off to the side, like a deviations
module.
If the client and server agree on the module containing the standard
extension usages
it will not be that complex in the protocol.

  ex:binary-representation ietf-inet-types:ipv4-address {
     ex:binary-length 4;
     ex:binary-pattern "b0.b1.b2.b3";
  }

I agree YANG 1.2 should have real statements instead of extensions.

Andy

---

From: Mahesh Jethanandani <[email-masked]>
Date: Fri, 29 Mar 2019 14:38:10 -0700

Based on this discussion, I think we should reopen and change the title of this issue as “binary encoding in YANG support”, while I open a new issue in netconf-next for “support for binary encoding in NETCONF”.

> On Mar 29, 2019, at 11:57 AM, Andy Bierman <[email-masked]> wrote:
> 
> 
> 
> On Fri, Mar 29, 2019 at 11:46 AM Juergen Schoenwaelder <[email-masked] <mailto:[email-masked]>> wrote:
> > [email-masked] <mailto:[email-masked]>> wrote:
> > 
> > > > [email-masked] <mailto:[email-masked]>> wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > https://tools.ietf..org/html/draft-mahesh-netconf-binary-encoding-01 <https://tools.ietf.org/html/draft-mahesh-netconf-binary-encoding-01>
> > > >
> OK -- this is what I had in mind but off to the side, like a deviations module.
> If the client and server agree on the module containing the standard extension usages
> Fax:   +49 421 200 3103         <https://www.jacobs-university.de/ <https://www.jacobs-university.de/>>
> [email-masked] <mailto:[email-masked]>
> https://www.ietf.org/mailman/listinfo/netmod <https://www.ietf.org/mailman/listinfo/netmod>
Mahesh Jethanandani
[email-masked]

---

From: "Rob Wilton (rwilton)" <[email-masked]>
Date: Sun, 31 Mar 2019 10:32:07 +0000

I also agree that we should reopen this issue to further discuss any language implications, and add it to the “Further Discuss” bucket.

I suggest that we just do this, unless someone objects.

Thanks,
Rob

From: netmod <[email-masked]> On Behalf Of Mahesh Jethanandani
Sent: 29 March 2019 21:38
To: Andy Bierman <[email-masked]>
Cc: NetMod WG <[email-masked]>
Subject: Re: [netmod] yang next issue #46 binary encoding support

On Mar 29, 2019, at 11:57 AM, Andy Bierman <[email-masked]<mailto:[email-masked]>> wrote:

On Fri, Mar 29, 2019 at 11:46 AM Juergen Schoenwaelder <[email-masked]<mailto:[email-masked]>> wrote:
On Fri, Mar 29, 2019 at 09:30:19AM -0700, Andy Bierman wrote:
> [email-masked]<mailto:[email-masked]>> wrote:
>
> > > [email-masked]<mailto:[email-masked]>> wrote:
> > >
> > > > Hi,
> > > >
> > > https://tools.ietf..org/html/draft-mahesh-netconf-binary-encoding-01<https://tools.ietf.org/html/draft-mahesh-netconf-binary-encoding-01>
> > >

     typedef ipv4-address {
       type string {
         pattern

       binary-representation {
         type binary {
           length 4;
         }
         description

/js

Andy

[email-masked]<mailto:[email-masked]>

Mahesh Jethanandani
[email-masked]<mailto:[email-masked]>