Subject: [netmod] hex-string as built-in type in future versions of YANG

From: Vladimir Vassilev <[email-masked]>
Date: Wed, 6 Nov 2019 13:29:41 +0000

Moving a netmod relevant topic from a thread on the bmwg mailing list 
https://mailarchive.ietf.org/arch/msg/bmwg/GwkVykKmOX7DFokCFynJVy_ZwZ8

On 29/10/2019 13.32, tom petch wrote:
> Picking up the point below about vlan, this is where the IETF is not
> doing the job it might.  'vlan' appears in many YANG modules with
> several flavours thereof; yes, we might get a better resolution on the
> netmod list but really, apart from our IEEE liaison, I do not know where
> to go for the best advice on this.  I saw this issue surface recently on
> I2RS which, thinking about it, makes sense and there, the outcome was
>       import ieee802-dot1q-types {...      reference
>          "IEEE Std 802.1Qcp-2018: Bridges and Bridged
>           Networks - Amendment: YANG Data Model.";
> i.e. anything vlan we import from the IEEE module.  I did query making
> an IETF module normatively dependent on an IEEE module and the ADs said
> that that was fine so that is the direction in which I think that the
> IETF should go.

I can think of at least 2 issues with importing ieee802-dot1q-types.yang 
in IETF modules. 1. the unclear license under which third party 
organizations can distribute it. 2. Its design

[1] The IETF has liaison with IEEE but the IETF modules are distributed 
under Simplified BSD License while the IEEE ones are not.  It is not 
clear to me what is the effective IEEE license for third party 
organizations that need to distribute the IEEE modules together with the 
importing IETF ones. If the cost of the 5 simple ethernet frame field 
types - ethertype, vlanid, tpid, pcp, cfi is embedding dependency on the 
conservative IEEE licensing policies I think the cost is too high.

[2] Some of the types are based on strings with complex lexical 
representation with canonical form specified in description statement 
which is neither scalable nor automation friendly. For example the 
"ethertype-type" type (just getting started):

... from [email-masked]

   typedef ethertype-type {
     type string {
       pattern "[0-9a-fA-F]{2}-[0-9a-fA-F]{2}";
     }
     description
       "The EtherType value represented in the canonical order defined
       by IEEE 802. The canonical representation uses uppercase
       characters.";
     reference
       "9.2 of IEEE Std 802-2014";
   }
...

IMO typedefs (not types) with constrains specified in a description 
statement especially if they are not part of ietf-yang-types (RFC6991) 
should be avoided. Those types are bad enough even when they are defined 
in ietf-yang-types e.g. "mac-address" AA:BB:CC:DD:EE:FF which is valid 
for rpc "input" data has to be treated as aa:bb:cc:dd:ee:ff (which is 
the canonical representation). The difference comes from the fact that 
"mac-address" is seamlessly converted to lowercase in many tools (not 
all). Those tools do not support the types defined in 
ieee802-dot1q-types in the same way. So there is no "to uppercase" 
seamless conversion for ethertype-type and "aB-cD" and "AA-CD" are 
treated as different values and you will have to figure out how your 
implementation can fix this on your own.

IMO ietf-yang-types:mac-address (and many others 
ieee802-dot1q-types:ethertype-type included) can be derived from 
ietf-yang-types:hex-string instead of string. It would be even better if 
"hex-string" is defined as a new built-in type in YANG next with an 
optional "width" sub-statement constraining the number of bits represented.

There are relevant discussions in 
https://github.com/netmod-wg/yang-next/issues/19  and 
https://github.com/netmod-wg/yang-next/issues/46 (posted the last 
paragraph of this proposal there).

/Vladimir

> Tom Petch

---

From: Schönwälder, Jürgen <[email-masked]>
Date: Wed, 6 Nov 2019 14:10:55 +0000

> implementation can fix this on your own.

     typedef mac-address {
       type string {
         pattern '[0-9a-fA-F]{2}(:[0-9a-fA-F]{2}){5}';
       }
       description
        "The mac-address type represents an IEEE 802 MAC address.
         The canonical representation uses lowercase characters.

         In the value set and its semantics, this type is equivalent
         to the MacAddress textual convention of the SMIv2.";
       reference
        "IEEE 802: IEEE Standard for Local and Metropolitan Area
                   Networks: Overview and Architecture
         RFC 2579: Textual Conventions for SMIv2";
     }

The problem here is that two organizations have rather different
preferences when it comes to uppercase and lowercase letters.
 
> paragraph of this proposal there).

Creating a hex-string built-in type in order to settle the debate
lowercase vs. uppercase? Note sure this is a strong technical reason
for creating a new built-in type. ;-)

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1 | 28759 Bremen | Germany
Fax:   +49 421 200 3103         <https://www.jacobs-university.de/>

---

From: Scott Mansfield <[email-masked]>
Date: Wed, 6 Nov 2019 16:29:22 +0000

The licensing information for IEEE YANG models can be found here...
https://github.com/YangModels/yang  (YANG Catalog's github repository)

If you scroll down there is the text of the README.md file, there you will 
find...

In the Models Directory Structure section:
yang/standard/ieee: standard modules (published or drafts) intended for IEEE 
submission [3]

is this information:
[3] IEEE License:

- All files contained within this sub-directory are considered to be intended 
as IEEE Contributions.

- All issues entered into the trouble ticket system for this directory are 
considered to be intended as IEEE Contributions.

- All pull requests submitted for this directory are considered to be intended 
as IEEE Contributions.

- All contributions to IEEE standards development (whether for an individual 
or entity standard) shall meet the requirements outlined in the IEEE-SA 
Copyright
Policy (https://standards.ieee.org/develop/policies/bylaws/sect6-7.html#7)

- Copyright release for YANG modules: Users may freely reproduce the YANG 
modules contained under /standard/ieee/ so that they can be used for their 
intended purpose.

If you really want the official YANG, you can extract the YANG out of the IEEE 
Standard document (yang is included as a txt attachment to pdf).  But the
copyright release is the same.  See 
https://ieeexplore.ieee.org/browse/standards/get-program/page/series?id=68 for 
a place to freely download 802 Standards.

Hope this helps.

Regards,
-scott.

-----Original Message-----
From: netmod <[email-masked]> On Behalf Of Vladimir Vassilev
Sent: Wednesday, November 6, 2019 8:30 AM
To: tom petch <[email-masked]>; [email-masked]
Subject: [netmod] hex-string as built-in type in future versions of YANG

On 29/10/2019 13.32, tom petch wrote:
> netmod list but really, apart from our IEEE liaison, I do not know
> where to go for the best advice on this.  I saw this issue surface
> recently on I2RS which, thinking about it, makes sense and there, the 
> outcome was
>           Networks - Amendment: YANG Data Model."; i.e. anything vlan
> we import from the IEEE module.  I did query making an IETF module
> normatively dependent on an IEEE module and the ADs said that that was
> fine so that is the direction in which I think that the IETF should
> go.

I can think of at least 2 issues with importing ieee802-dot1q-types.yang in 
IETF modules. 1. the unclear license under which third party organizations can 
distribute it. 2. Its design

[1] The IETF has liaison with IEEE but the IETF modules are distributed under 
Simplified BSD License while the IEEE ones are not.  It is not clear to me 
what is the effective IEEE license for third party organizations that need to 
distribute the IEEE modules together with the importing IETF ones. If the cost 
of the 5 simple ethernet frame field types - ethertype, vlanid, tpid, pcp, cfi 
is embedding dependency on the conservative IEEE licensing policies I think 
the cost is too high.

[2] Some of the types are based on strings with complex lexical representation 
with canonical form specified in description statement which is neither 
scalable nor automation friendly. For example the "ethertype-type" type (just 
getting started):

   typedef ethertype-type {
     type string {

IMO typedefs (not types) with constrains specified in a description statement 
especially if they are not part of ietf-yang-types (RFC6991) should be 
avoided. Those types are bad enough even when they are defined in 
ietf-yang-types e.g. "mac-address" AA:BB:CC:DD:EE:FF which is valid for rpc 
"input" data has to be treated as aa:bb:cc:dd:ee:ff (which is the canonical 
representation). The difference comes from the fact that "mac-address" is 
seamlessly converted to lowercase in many tools (not all). Those tools do not 
support the types defined in ieee802-dot1q-types in the same way. So there is 
no "to uppercase"
seamless conversion for ethertype-type and "aB-cD" and "AA-CD" are treated as 
different values and you will have to figure out how your implementation can 
fix this on your own.

IMO ietf-yang-types:mac-address (and many others 
ieee802-dot1q-types:ethertype-type included) can be derived from 
ietf-yang-types:hex-string instead of string. It would be even better if 
"hex-string" is defined as a new built-in type in YANG next with an optional 
"width" sub-statement constraining the number of bits represented.

There are relevant discussions in
https://protect2.fireeye.com/v1/url?k=f32414e9-afadbb20-f3245472-0cc47ad93e1c-3cac03145d53bba2&q=1&e=6bf134b3-1f82-4293-8625-2d5bf67d1c2d&u=https%3A%2F%2Fgithub.com%2Fnetmod-wg%2Fyang-next%2Fissues%2F19 
and
https://protect2.fireeye.com/v1/url?k=df09b925-838016ec-df09f9be-0cc47ad93e1c-3baced0791ac566c&q=1&e=6bf134b3-1f82-4293-8625-2d5bf67d1c2d&u=https%3A%2F%2Fgithub.com%2Fnetmod-wg%2Fyang-next%2Fissues%2F46 
(posted the last paragraph of this proposal there).

/Vladimir

> Tom Petch
_______________________________________________
netmod mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/netmod

---

From: Vladimir Vassilev <[email-masked]>
Date: Wed, 6 Nov 2019 18:15:25 +0000

> for creating a new built-in type. ;-)

There is no technical reason at all to have MAC addresses represented as 
pattern constrained strings and not "binary" of length 6 (as it was in 
SMIv2 MIBs) either. There is no technical reason for ethertype to be 
represented as pattern constrained string and not "binary" of length 2 
or "uint16". It is likely done because of the sane lexical 
representation requirement derived from NETCONF design requirement to be 
human readable on the wire and this is not the case when the data is 
base64 encoded or in the better case represented as decimal when humans 
are used to its hex-string representation.

This is what this proposed type offers - readable and writable 
representation (more readable and writable by humans then the base64 
representation of binary).

That said defining new uint* and binary built-in sub-statement 
{lexical-representation hex-string;} enabling alternative lexical 
representation might be the right solution:

typedef mac-address {
   type binary {
     length 6;
     lexical-representation hex-string;
   }
}

typedef ether-type {
   type uint16 {
     lexical-representation hex-string;
   }
}

/Vladimir

---

From: Schönwälder, Jürgen <[email-masked]>
Date: Wed, 6 Nov 2019 19:28:20 +0000

> are used to its hex-string representation.

The goal was to automate configuration and configuration data used to
be given to operators (and their automation scripts) in a textual
format. I think this is why we favored textual representations (the
data on the wire is encrypted anyway). In the MIB world, we had some
limited ways to define automated conversions between binary data and
their textual representation. This mechanism was good enough for MAC
addresses, but it already falls apart with representation of IPv6
addresses according to RFC 5952.

Yes, it might have been wise to not always tie YANG's 'binary' to
'base64' but this is what we did end up doing (after longer
discussions about this). One argument was that if you want a
hex-string, you can simply define it (as long as you do not care much
about supporting binary encodings - this we did not care much about
back than, likely a mistake if I look back).

Anyway, IETF preferring lowercase characters and IEEE preferring
uppercase characters for some shared types is a non-technological
conflict and I see no easy way to resolve this.

/js