Subject: [netmod] Import by revision-date or label vs semantic version

From: "Rob Wilton (rwilton)" <[email-masked]>
Date: Wed, 2 Sep 2020 10:51:38 +0000

Hi,

During the NETMOD 108 meeting I had made a comment that imports using revision-or-derived are not done using a semantic version number, but instead are done by revision label, which limits how they behave and what they are allowed to do.  Some participants were concerned that this might be confusing or even broken, and the outcome of that short discussion was that I should send an email to NETMOD with an example to help explain how they are proposed to work.

The main principle here is that the versioning drafts have a clear distinction between supporting an abstract version label vs a specific version label scheme (such as YANG Semver).

The new "revision-or-derived" extension is defined as part of base draft-ietf-netmod-yang-module-versioning.  The "revision-or-derived" extension takes a single argument that can either be a "revision date" or a "revision label".  It can be used regardless of the versioning scheme that is being used as a revision label, but therefore is also restricted to treating the revision label as an opaque textual label for a revision date.

So, making use of the examples in section 4.1 of https://tools.ietf.org/html/draft-ietf-netmod-yang-module-versioning-01

When a module has an import statement like this:

   import example-module {
     rev:revision-or-derived 2.0.0;
   }

Then the processing to find a suitable revision to import would be something like this (ignoring the issue of which revision is chosen from the set of suitable candidate revisions): 

1) Iterate suitable candidate "example-module" YANG files.
2) For each candidate file, parse the revision history, and check back through the revision history to see if a revision with label "2.0.0" exists.  If it does, then that module revision is a suitable candidate for import.  If no revision with label "2.0.0" exists then that module revision does not satisfy the import.  Note the tooling does not need to understand the format of the revision label at all, a textual comparison between labels is sufficient.

The algorithm works equivalently if the import was done using a revision date instead of a label (e.g., rev:revision-or-derived 2019-02-01), except that obviously the comparison in the revision history is done on the revision date rather than the revision labels.

-------

So, how does this interact with YANG Semver (or vanilla Semver 2.0.0)?

Well, this still works because each version of a YANG module contains the revision history back to the root of the version tree.

E.g., the YANG file defining version 2.2.0 would contain revisions for versions 2.2.0, 2.1.0, 2.0.0, 1.0.0 in its revision history, and hence would satisfy an import using label "2.0.0" or derived" solely because a revision with that label exists in its revision history.

However, if the revision history had entries pruned (i.e., perhaps 2.1.0 hadn't been included in the revision history so that it was just 2.2.0, 2.0.0, 1.0.0) then this particular YANG file for version 2.2.0 WOULD NOT satisfy an import for "revision-or-derived 2.1.0;" because the module's revision history does not contain revision 2.1.0.

So, the import revision-or-derived works fine for Semver version labels as long as the revision history is consistent and complete.

-------

Finally, there has been some discussion about whether it would be useful to have an import statement that restricts imports to only backwards compatible versions - I'll post a separate email on this.

If the WG decided that this is useful, then this could still be supported, and without needing to understand the revision label.  Instead, it can be done by checking the revision history for the "rev:nbc-changes" substatement that indicates where NBC changes have occurred in the revision history.  As long as the allocated YANG Semver revision labels are consistent with the use of the rev:nbc-changes" substatement in the revision history then it would still behave in the intuitive way. 

Regards,
Rob

[As an individual contributor]

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 2 Sep 2020 12:55:39 +0200

I continue to have a problem with changing YANG import semantics using
extension statements. Versioning people should understand that this is
an NBC change and hence they should request that the YANG version
number is changed.

/js

On Wed, Sep 02, 2020 at 10:51:38AM +0000, Rob Wilton (rwilton) wrote:
> _______________________________________________
> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1 | 28759 Bremen | Germany
Fax:   +49 421 200 3103         <https://www.jacobs-university.de/>

---

From: Andy Bierman <[email-masked]>
Date: Wed, 2 Sep 2020 07:52:10 -0700

> an NBC change and hence they should request that the YANG version
+1

IMO it is a huge mistake to think YANG will be easier to use in the long
run by
adding optional extensions to YANG 1.1 instead of introducing a new
language version.
YANG 1.1 will splinter into several dialects, all relying on different
subsets of an ad-hoc
set of language extensions.

/js

Andy

> > During the NETMOD 108 meeting I had made a comment that imports using
> revision-or-derived are not done using a semantic version number, but
> instead are done by revision label, which limits how they behave and what
> they are allowed to do.  Some participants were concerned that this might
> be confusing or even broken, and the outcome of that short discussion was
> that I should send an email to NETMOD with an example to help explain how
> they are proposed to work.
> >
> > The main principle here is that the versioning drafts have a clear
> distinction between supporting an abstract version label vs a specific
> version label scheme (such as YANG Semver).
> >
> > The new "revision-or-derived" extension is defined as part of base
> draft-ietf-netmod-yang-module-versioning.  The "revision-or-derived"
> extension takes a single argument that can either be a "revision date" or a
> "revision label".  It can be used regardless of the versioning scheme that
> is being used as a revision label, but therefore is also restricted to
> treating the revision label as an opaque textual label for a revision date.
> >
> > So, making use of the examples in section 4.1 of
> https://tools.ietf.org/html/draft-ietf-netmod-yang-module-versioning-01
> >
> > Then the processing to find a suitable revision to import would be
> something like this (ignoring the issue of which revision is chosen from
> the set of suitable candidate revisions):
> >
> > 2) For each candidate file, parse the revision history, and check back
> through the revision history to see if a revision with label "2.0.0"
> exists.  If it does, then that module revision is a suitable candidate for
> import.  If no revision with label "2.0.0" exists then that module revision
> does not satisfy the import.  Note the tooling does not need to understand
> the format of the revision label at all, a textual comparison between
> labels is sufficient.
> >
> > The algorithm works equivalently if the import was done using a revision
> date instead of a label (e.g., rev:revision-or-derived 2019-02-01), except
> that obviously the comparison in the revision history is done on the
> revision date rather than the revision labels.
> >
> >
> > -------
> >
> > Well, this still works because each version of a YANG module contains
> the revision history back to the root of the version tree.
> >
> > E.g., the YANG file defining version 2.2.0 would contain revisions for
> versions 2.2.0, 2.1.0, 2.0.0, 1.0.0 in its revision history, and hence
> would satisfy an import using label "2.0.0" or derived" solely because a
> revision with that label exists in its revision history.
> >
> > However, if the revision history had entries pruned (i.e., perhaps 2.1.0
> hadn't been included in the revision history so that it was just 2.2.0,
> 2.0.0, 1.0.0) then this particular YANG file for version 2.2.0 WOULD NOT
> satisfy an import for "revision-or-derived 2.1.0;" because the module's
> revision history does not contain revision 2.1.0.
> >
> > So, the import revision-or-derived works fine for Semver version labels
> as long as the revision history is consistent and complete.
> >
> > -------
> >
> > Finally, there has been some discussion about whether it would be useful
> to have an import statement that restricts imports to only backwards
> compatible versions - I'll post a separate email on this.
> >
> > If the WG decided that this is useful, then this could still be
> supported, and without needing to understand the revision label.  Instead,
> it can be done by checking the revision history for the "rev:nbc-changes"
> substatement that indicates where NBC changes have occurred in the revision
> history.  As long as the allocated YANG Semver revision labels are
> consistent with the use of the rev:nbc-changes" substatement in the
> revision history then it would still behave in the intuitive way.
> >
> >
> > Regards,
> > Rob
> >

---

From: "Sterne, Jason (Nokia - CA/Ottawa)" <[email-masked]>
Date: Wed, 16 Sep 2020 22:48:23 +0000

Hi all,

We have a practical problem though. YANG NEXT/2.0 is some indeterminate date in the future. Certainly measured in years (not months) and likely >2.

But we could really use a lot of this versioning work, including a more complete solution for importing by revision, now. Maybe we will roll this into YANG 2.0 but we can take advantage of it years before that will happen.

If we can add some improvements here, and the downside (i.e. clients/tools that don't understand the extensions) is no worse than today, then we should go ahead IMO.

The existing import behaviour is ambiguous in RFC 7950, so if multiple revisions of an imported module are available then two compilers could produce different schema. It is also quite likely that the compile would just break if the imported version was too old (i.e. missing dependency)

Consider this text from RFC8526:

An NMDA-compliant NETCONF server MUST implement the "ietf-netconf-nmda" module defined in this document, MUST support the operational state datastore, and MUST implement at least revision 2019-01-04 of the "ietf-yang-library" module defined in [RFC8525<https://tools.ietf.org/html/rfc8525>].

Notice the "at least revision 2019-01-04".

As another example: RFC8572 Secure Zero Touch Provisioning (SZTP) imports ietf-yang-types in order to use hex-string. But hex-string was only introduced in the second standard version of ietf-yang-types in rfc 6991. So the import should be something like "revision 2013-07-15 or derived;".

Rgds,
Jason

From: netmod <[email-masked]> On Behalf Of Andy Bierman
Sent: Wednesday, September 2, 2020 10:52 AM
To: Juergen Schoenwaelder <[email-masked]>; Rob Wilton (rwilton) <rwilton=[email-masked]>; [email-masked]
Subject: Re: [netmod] Import by revision-date or label vs semantic version

On Wed, Sep 2, 2020 at 3:55 AM Juergen Schoenwaelder <[email-masked]<mailto:[email-masked]>> wrote:
I continue to have a problem with changing YANG import semantics using
extension statements. Versioning people should understand that this is
an NBC change and hence they should request that the YANG version
number is changed.

+1

IMO it is a huge mistake to think YANG will be easier to use in the long run by
adding optional extensions to YANG 1.1 instead of introducing a new language version.
YANG 1.1 will splinter into several dialects, all relying on different subsets of an ad-hoc
set of language extensions.

/js

Andy

On Wed, Sep 02, 2020 at 10:51:38AM +0000, Rob Wilton (rwilton) wrote:
> [email-masked]<mailto:[email-masked]>

_______________________________________________
netmod mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/netmod

---

From: Andy Bierman <[email-masked]>
Date: Fri, 18 Sep 2020 05:52:32 -0700

On Wed, Sep 16, 2020 at 3:48 PM Sterne, Jason (Nokia - CA/Ottawa) <
[email-masked]> wrote:

> Hi all,
>
>
>
> We have a practical problem though. YANG NEXT/2.0 is some indeterminate
> date in the future. Certainly measured in years (not months) and likely >2.
>
>
>

Actually, the main problem is not that these import mechanisms are done
with external statements
instead of native statements.  That itself is a mistake because importing
other definitions
is critical to parsing a YANG module and the behavior should be
mandatory-to-implement.
External statements MAY be ignored.

The main problem is that import-by-revision was probably a mistake from the
start.
It's true that import-by-exact-revision should have never existed because
it is so fragile.  It should have been import-by-exact-or-later from the
start.
This covers the 95% case (BC changes to new revisions).

Most languages treat module search path requirements as a compiler
implementation detail
and not part of the language.  As Juergen has pointed out, a single SemVer
label for the entire
module is not fine-grained enough to be generally useful. Bumping the major
version will
appear to break lots of things that are in fact still working (so people
might not do it).

It should be intuitive that the question "what YANG module revisions work
together?"
is relative, and the answer is "It depends what you are trying to do".  The
problem is far
too complex for SemVer to solve.  That is useful to support multiple
release trains
and provide a familiar revision string format, but not this problem.

The answer to that complex question is "YANG Packages".
Real people need to maintain package definitions, just like software
packages for Linux distros.
Tools can use these files to identify all the required modules and
revisions based on the selected
packages.

Andy

But we could really use a lot of this versioning work, including a more
> complete solution for importing by revision, now. Maybe we will roll this
> into YANG 2.0 but we can take advantage of it years before that will happen.
>
>
>
> If we can add some improvements here, and the downside (i.e. clients/tools
> that don't understand the extensions) is no worse than today, then we
> should go ahead IMO.
>
>
>
> The existing import behaviour is ambiguous in RFC 7950, so if multiple
> revisions of an imported module are available then two compilers could
> produce different schema. It is also quite likely that the compile would
> just break if the imported version was too old (i.e. missing dependency)
>
>
>
> An NMDA-compliant NETCONF server MUST implement the "ietf-netconf-nmda"
> module defined in this document, MUST support the operational state
> datastore, and MUST implement at least revision 2019-01-04 of the
> "ietf-yang-library" module defined in [RFC8525
> <https://tools.ietf.org/html/rfc8525>].
>
>
>
> As another example: RFC8572 Secure Zero Touch Provisioning (SZTP) imports
> ietf-yang-types in order to use hex-string. But hex-string was only
> introduced in the second standard version of ietf-yang-types in rfc 6991.
> So the import should be something like "revision 2013-07-15 or derived;".
>
>
>
> Rgds,
>
> Jason
>
>
>
>
>
> *From:* netmod <[email-masked]> *On Behalf Of *Andy Bierman
> *Sent:* Wednesday, September 2, 2020 10:52 AM
> *To:* Juergen Schoenwaelder <[email-masked]>; Rob
> Wilton (rwilton) <rwilton=[email-masked]>; [email-masked]
> *Subject:* Re: [netmod] Import by revision-date or label vs semantic
> version
>
>
>
>
>
>
>

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Sun, 20 Sep 2020 19:02:55 +0200

If you change the YANG import semantics, then I think you make an NBC
change to YANG. Do the versioning people propose to release YANG 2,
which just changes the import semantics and nothing else? That I would
have understood from a semantic versioning perspective (where the
assumption is that bumping a major version number is not a big deal).
Or do the versioning people propose to make an NBC under the covers?

/js