Subject: [netmod] import-by-semver issue

From: Andy Bierman <[email-masked]>
Date: Sat, 23 Mar 2019 12:00:42 -0700

Hi,

I am wondering if there are implementations of this draft:

https://tools.ietf.org/html/draft-verdt-netmod-yang-semver-00

Specifically, implementation of the  'version' extension

https://tools.ietf.org/html/draft-verdt-netmod-yang-semver-00#section-3

IMO it is a really bad idea to put the semantics of how to import modules
in one of the modules that is imported.  Your example shows ietf-semver
imported first with no extension, but it could be last with a version
extension.

          // all other imports, then last....

    import ietf-semver {

      prefix "semver";

      semver:version 1.1.2;

    }

Translation unit parsing is something that needs to be built into the
compiler.
This should be part of YANG 1.2 if it is done.

  import example-module {

     prefix exmod;
     version 1.2.0+;
   }

To a compiler writer, the difference is huge. (ietf-semver extensions need
to
be built-in statements in YANG, at least 'version')

BTW, all the import examples are missing the mandatory prefix-stmt

Andy

---

From: "Rob Wilton (rwilton)" <[email-masked]>
Date: Sun, 24 Mar 2019 09:39:41 +0000

Hi Andy,

Yes we are currently implementing the module version field, although that may change depending on what different the final solution ends up being.

Support for import-by-version is less critical for us, and hence implementation would lag.

In terms of the issue that you raise:

-        I would expect that a compiler that understands semver to preload the semver extension, possibly allow with other common YANG type files, extensions).

-        For a compile that doesn’t understand semver then it would just ignore the extension, which should be fine.

I agree that it would be nice if this extension was part of the core YANG language, but I don’t think that is necessarily required.

Thanks,
Rob

From: netmod <[email-masked]> On Behalf Of Andy Bierman
Sent: 23 March 2019 19:01
To: NetMod WG <[email-masked]>
Subject: [netmod] import-by-semver issue

Hi,

IMO it is a really bad idea to put the semantics of how to import modules
in one of the modules that is imported.  Your example shows ietf-semver
imported first with no extension, but it could be last with a version extension.

    import ietf-semver {

      prefix "semver";

      semver:version 1.1.2;

    }

Translation unit parsing is something that needs to be built into the compiler.
This should be part of YANG 1.2 if it is done.

  import example-module {

     prefix exmod;

     version 1.2.0+;

   }

To a compiler writer, the difference is huge. (ietf-semver extensions need to
be built-in statements in YANG, at least 'version')

Andy

---

From: Andy Bierman <[email-masked]>
Date: Sun, 24 Mar 2019 04:06:54 -0700

On Sun, Mar 24, 2019 at 2:40 AM Rob Wilton (rwilton) <[email-masked]>
wrote:

> Hi Andy,
>
>
>
> Yes we are currently implementing the module version field, although that
> may change depending on what different the final solution ends up being.
>
>
>
> -        I would expect that a compiler that understands semver to
> preload the semver extension, possibly allow with other common YANG type
> files, extensions).
>
> -        For a compile that doesn’t understand semver then it would just
> ignore the extension, which should be fine.
>
>
>

Try to find even 1 compiler in the world that works this way.
Yes you can specially hack the extension implementation as if it were a
built-in keyword.

I agree that it would be nice if this extension was part of the core YANG
> language, but I don’t think that is necessarily required.
>
>
>

I don't think any of semver is required for anything.
IMO it provides a minor improvement to  people familiar with the
MAJOR.MINOR.PATCH numbering.
This improvement is lost as soon as extra letters are added and the string
is no longer familiar.

Import-by-semver seems like it is part of the base module (no if-feature on
the extension-stmt is even possible)
so skipping the implementation of it needs a deviation (oh wait, YANG 1.1
can't do that either).

Thanks,
Andy

>
>
>
>
> *From:* netmod <[email-masked]> *On Behalf Of *Andy Bierman
> *Sent:* 23 March 2019 19:01
> *To:* NetMod WG <[email-masked]>
> *Subject:* [netmod] import-by-semver issue
>
>
>
> Hi,
>
>
>

---

From: "Rob Wilton (rwilton)" <[email-masked]>
Date: Sun, 24 Mar 2019 14:07:15 +0000

Hi Andy,

There are many ways to write and design compilers.

Compilers that don’t understand import-by-semver just ignore the extension, no deviation is required.  They just import whichever arbitrary revision that they would have imported anyway, as specified by YANG 1.1.  The import-by-semver statement just reduces the set of valid modules revisions that can be used for import.

Thanks,
Rob

From: Andy Bierman <[email-masked]>
Sent: 24 March 2019 12:07
To: Rob Wilton (rwilton) <[email-masked]>
Cc: NetMod WG <[email-masked]>
Subject: Re: [netmod] import-by-semver issue

On Sun, Mar 24, 2019 at 2:40 AM Rob Wilton (rwilton) <[email-masked]<mailto:[email-masked]>> wrote:
Hi Andy,

Try to find even 1 compiler in the world that works this way.
Yes you can specially hack the extension implementation as if it were a built-in keyword.

I don't think any of semver is required for anything.
IMO it provides a minor improvement to  people familiar with the MAJOR.MINOR.PATCH numbering.
This improvement is lost as soon as extra letters are added and the string is no longer familiar.

Import-by-semver seems like it is part of the base module (no if-feature on the extension-stmt is even possible)
so skipping the implementation of it needs a deviation (oh wait, YANG 1.1 can't do that either).

Thanks,
Rob

Andy

From: netmod <[email-masked]<mailto:[email-masked]>> On Behalf Of Andy Bierman
Sent: 23 March 2019 19:01
To: NetMod WG <[email-masked]<mailto:[email-masked]>>
Subject: [netmod] import-by-semver issue

Hi,

    import ietf-semver {

      prefix "semver";

      semver:version 1.1.2;

    }

  import example-module {

     prefix exmod;

     version 1.2.0+;

   }

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Sun, 24 Mar 2019 15:31:16 +0100

> Compilers that don’t understand import-by-semver just ignore the extension, no deviation is required.  They just import whichever arbitrary revision that they would have imported anyway, as specifi…

If two compilers (one supporting semver, the other not) resolve
imports differently, then the design is in my view somewhat broken,
in particular if you allow NBC changes.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1 | 28759 Bremen | Germany
Fax:   +49 421 200 3103         <https://www.jacobs-university.de/>

---

From: "Rob Wilton (rwilton)" <[email-masked]>
Date: Sun, 24 Mar 2019 15:14:18 +0000

Hi Juergen,

> -----Original Message-----
> From: Juergen Schoenwaelder <[email-masked]>
> Sent: 24 March 2019 15:31
> Cc: Andy Bierman <[email-masked]>; NetMod WG <[email-masked]>
> > Compilers that don’t understand import-by-semver just ignore the extension,
> no deviation is required.  They just import whichever arbitrary revision that they
> would have imported anyway, as specified by YANG 1.1.  The import-by-semver
> statement just reduces the set of valid modules revisions that can be used for
> import.
> >
> 
> If two compilers (one supporting semver, the other not) resolve imports
> differently, then the design is in my view somewhat broken, in particular if you
> allow NBC changes.

But that is true of YANG compilers today.  If there are multiple revisions of a module that could be chosen, then each compiler is at liberty to decide which revision to choose (last paragraph of section 5.1.1 in RFC 7950).

So, I would argue that "import-by-version" doesn't make YANG compilers any less consistent that they are already today, since that inconsistency already exists.

I presume that the real solution here is to explicitly define the full set of implemented, import-only-modules to the compiler so that it always compiles a consistent schema.  Perhaps other compilers have different ways to solve this problem.

Note, I also think that YANG library has a similar inconsistency.  I.e. there is no guarantee that two different compilers will construct exactly the same datastore schema from the YANG library output.  I think that this is a design bug, but also one that we attempt to address in draft-verdt-netmod-yang-semver-00.

Thanks,
Rob

---

From: Martin Bjorklund <[email-masked]>
Date: Sun, 24 Mar 2019 16:21:30 +0100 (CET)

"Rob Wilton (rwilton)" <[email-masked]> wrote:
> > imports
> But that is true of YANG compilers today.  If there are multiple
> revisions of a module that could be chosen, then each compiler is at
> liberty to decide which revision to choose (last paragraph of section
> 5.1.1 in RFC 7950).

This is by design, of course.  With an "open" import (not import by
revision), the server implementor is free to implement any set of
modules that work together.

> So, I would argue that "import-by-version" doesn't make YANG compilers
> any less consistent that they are already today, since that
> inconsistency already exists.

I don't think I understand Andy's objection.   I also think that this
is just another implementation detail.

> I presume that the real solution here is to explicitly define the full
> set of implemented, import-only-modules to the compiler so that it
> always compiles a consistent schema.  Perhaps other compilers have
> different ways to solve this problem.
> 
> Note, I also think that YANG library has a similar inconsistency.
> I.e. there is no guarantee that two different compilers will construct
> exactly the same datastore schema from the YANG library output.

Can you elaborate?  Given an instance of YANG library, it should be
clear which set of modules are used.

> I
> think that this is a design bug, but also one that we attempt to
> address in draft-verdt-netmod-yang-semver-00.
> 
> Thanks,
> Rob
> 
> > 
> > /js
> > 
> > --
> _______________________________________________
> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod

---

From: "Rob Wilton (rwilton)" <[email-masked]>
Date: Sun, 24 Mar 2019 15:29:54 +0000

> From: Martin Bjorklund <[email-masked]>
> Sent: 24 March 2019 16:22
> Cc: [email-masked]; [email-masked]
> > > no deviation is required.  They just import whichever arbitrary
> > > revision that they would have imported anyway, as specified by YANG
> > > 1.1.  The import-by-semver statement just reduces the set of valid
> > > modules revisions that can be used for import.
> > > >
> > >
> This is by design, of course.  With an "open" import (not import by revision), the
> server implementor is free to implement any set of modules that work together.
> 
> Can you elaborate?  Given an instance of YANG library, it should be clear which
> set of modules are used.

For modules that are "import-only" rather than "implemented" then some modules might use import-by-revision to import a specific revision, and other modules might just use import (without revision).  In this case, the compiler can choose which revision it uses to satisfy that import (e.g. getting different grouping definitions).

In draft-verdt-netmod-yang-semver-00, we aim to resolve this ambiguity with the following text that incorporates semver, a similar solution or just using the latest revision dates could also be devised:

5.2.  Resolving ambiguous module imports

   A YANG datastore schema, defined in [RFC8525], can specify multiple
   revisions of a YANG module in the schema using the 'import-only'
   list, with the requirement from [RFC7950] that only a single revision
   of a YANG module may be implemented.

   If a YANG module import statement does not specify a specific version
   or revision within the datastore schema then it could be ambiguous as
   to which module revision the import statement should resolve to.
   Hence, a datastore schema constructed by a client using the
   information contained in YANG library may not exactly match the
   datastore schema actually used by the server.

   The following rules remove the ambiguity:

   If a module import statement could resolve to more than one module
   revision defined in the datastore schema, and one of those revisions
   is implemented (i.e., not an 'import-only' module), then the import
   statement MUST resolve to the revision of the module that is defined
   as being implemented by the datastore schema.

   If a module import statement could resolve to more than one module
   revision defined in the datastore schema, and none of those revisions
   are implemented, but one of more modules revisions specify a YANG
   semantic version, then the import MUST resolve to the module with the
   greatest version number, according to the version comparison rules in
   Section 3.

   If a module import statement could resolve to more than one module
   revision defined in the datastore schema, none of those revisions are
   implemented, and none of the modules revisions have a YANG semantic
   version number, then the import MUST resolve to the module that has
   the most recent revision date.

Thanks,
Rob

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Sun, 24 Mar 2019 17:14:51 +0100

> But that is true of YANG compilers today.  If there are multiple revisions of a module that could be chosen, then each compiler is at liberty to decide which revision to choose (last paragraph of s…

The difference is that NBC changes are not allowed. As long as you
find a certain symbol, it has fixed and predictable semantics.

Sorry, but changing the way the import statement works is an NBC
change of YANG and this can't be done with extensions.

/js

---

From: Andy Bierman <[email-masked]>
Date: Sun, 24 Mar 2019 10:49:30 -0700

On Sun, Mar 24, 2019 at 9:14 AM Juergen Schoenwaelder <
[email-masked]> wrote:

> liberty to decide which revision to choose (last paragraph of section 5.1.1
> in RFC 7950).
> >
>
I strongly agree.
The expected behavior for tools needs to be consistent, especially for
the construction of the schema tree, which depends on the import rules.

Implementation complexity should matter in the IETF, but it does not.
Just keep raising the complexity up to 10 and complain that the tools have
bugs,
as if the two are unrelated. (Simply looking for a hardwired string
"semver:version"
will not work since the prefix is not required to be "semver" in the
import-stmt.)

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Sun, 24 Mar 2019 21:54:33 +0100 (CET)

> will not work since the prefix is not required to be "semver" in the

Agreed, but it quite easy to first build a prefix map (prefix ->
module name), and then instead of "xxxx:semver" you see ("ietf-semver",
"semver"); and _this_ can be hardcoded in the compiler.  (pyang works
like this)

This said, it is a bit weird with:

    import ietf-semver {
      prefix "semver";
      semver:version 1.1.2;
    }

so in order to handle the "semver:version" statement, you need to
import the module that has the prefix "semver".  But we can't just
import it like a normal import, b/c it has the semver: statement that
modifies how we do imports!

/matrin