Subject: [netmod] comments on YANG versioning

From: Andy Bierman <[email-masked]>
Date: Thu, 8 Nov 2018 14:52:50 -0800

Hi,

A few comments on the netmod meeting yesterday

1) what is a bugfix?
It is not encouraging that the DT cannot agree on the scope of a bugfix.
But not sure it matters if NBC updates can occur for any reason.
IMO it is easy to define a bugfix in the IETF -- it is called an Errata.
If an Errata is approved for a YANG module in an RFC then it is a bugfix.

2) SEMVER to the rescue?
If every module release can be its own feature release train then the value
of
ascending numeric identifiers is greatly diminished. The (m) and (M) tags
do not really help.  I strongly agree with the comment that cherry-picking
new
features can (and should) be done with deviations.  Updates of old
revisions needs to be for bugfixes only.

I prefer the OpenConfig "SEMVER Classic" rather than introducing a new
incompatible complex numbering scheme to support something that
should not be done anyway.

3) Bundles and compatibility modules
I strongly agree this solution approach is far better than treating every
revision
as a separate feature release train.  I don't see how I am going to
track the major.minor.patch for 100 different modules.  SEMVER is not
very useful for telling if module A works with B, C, and D. Import by SEMVER
will probably be OK at first, but become too error-prone after awhile.

4) Automation tools
Ad-hoc WEB pages from IANA do not cut it anymore.
We need a way to get patch versions of modules published and usable
by automation tools (without an RFC) with just the Errata report as a patch.
SEMVER requires that a module be released with the change but this is not
that practical.
Think how yocto works, using a base source version of a package + patches.
(IMO we need YANG Packages, which would serve as recipes
for a set of modules, features, annotations, patches and deviations, that
have been
tested to work together.)

5) YANG 1.2 vs Extensions
IMO a new YANG version would be better than extensions, especially to
fix status-stmt, import-by-revision, deviations, and add annotation,
patch, and many other new mechanisms to help backward compatibility.

Andy

---

From: Mahesh Jethanandani <[email-masked]>
Date: Fri, 9 Nov 2018 09:37:35 +0700

I am wondering if we are looking at two sets of rules for the versioning system we come up with. One that vendors do with their "native" models to satisfy their customers, and the other what standard bodies follow for the models published by them.

For example, vendors can make feature and NBC fixes to models in releases that are not the latest, while standard bodies like IETF make changes in models that are always the latest, even as they use the same versioning system. Again, vendors use all the three numbers, but standard bodies use the first (major) and the last (patch) number.

Mahesh Jethanandani
[email-masked]

> On Nov 9, 2018, at 5:52 AM, Andy Bierman <[email-masked]> wrote:
> 
> Hi,
> 
> If every module release can be its own feature release train then the value of
> do not really help.  I strongly agree with the comment that cherry-picking new
> I strongly agree this solution approach is far better than treating every revision
> SEMVER requires that a module be released with the change but this is not that practical.
> for a set of modules, features, annotations, patches and deviations, that have been
> _______________________________________________
> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod

---

From: Joe Clarke <[email-masked]>
Date: Thu, 8 Nov 2018 22:46:47 -0500

> For example, vendors can make feature and NBC fixes to models in releases that are not the latest, while standard bodies like IETF make changes in models that are always the latest, even as they us…

That's what we initially discussed on the DT call.  It was then raised
that there may be standards bodies (even groups in the IETF) that would
want to fork a model and make changes to both branches.

Joe

---

From: Robert Wilton <[email-masked]>
Date: Wed, 14 Nov 2018 11:30:42 +0000

> If an Errata is approved for a YANG module in an RFC then it is a bugfix.

Ultimately we have customers that will say "this part of your YANG is 
broken" and we want you to fix it in that release train, either in the 
next release, or as a software patch.

Sometimes vendors will disagree with their customers as to whether it is 
really a bug fix or an enhancement.  Sometimes we will fix what we think 
is an obvious bug but that will unfortunately break another customer 
whom was relying on the existing behavior and then ask us to revert the fix.

I think that it should be down to the module author/owner to decide 
whether or not it is a bug fix or an enhancement, and I would just like 
a versioning scheme that allows these changes to be expressed.  None of 
this changes the fact that I think that we should be avoiding making 
these changes in the first place.  I.e. I think that there is a clear 
separation between what the versioning scheme should be able to express, 
and what is recommended practice.

> should not be done anyway.

SEMVER classic does not support making bug fixes (even bc ones) on older 
releases.

In an older release, SEMVER classic allows:
  - editorial changes, e.g. spelling corrections or clarifications in 
description statements that do not change the API semantics in anyway.
  - bug fixes to the *implementation*, but then we are not using SEMVER 
to version the implementation anyway, only the API.

If you want to allow bug fixes (even just bc ones) in an older release 
then you either need something like modified semver, or a different 
versioning scheme that allows them.  Or you do what Rob Shakir suggests 
and use deviations for this instead, which I think is a misuse of 
deviations.

> will probably be OK at first, but become too error-prone after awhile.

+1.  I think that the versioning solution needs to consider something 
like bundles or packages.

> SEMVER requires that a module be released with the change but this is 

I think that we need to have a place where all revisions of modules are 
published and easily accessible.

> patch, and many other new mechanisms to help backward compatibility.

Perhaps.  There is currently a long list of stuff on the YANG.next issue 
tracker.  My concern is that this effort will get bogged down.

Also, some of these changes we want to apply across YANG language 
versions.  E.g. I want the updated status behaviour to apply to all YANG 
modules on the server, regardless of whether of which version of YANG 
they were defined in.

Thanks,
Rob

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 14 Nov 2018 12:48:41 +0100 (CET)

Hi,

Robert Wilton <[email-masked]> wrote:
> Sometimes vendors will disagree with their customers as to whether it
> is really a bug fix or an enhancement.  Sometimes we will fix what we
> think is an obvious bug but that will unfortunately break another
> customer whom was relying on the existing behavior and then ask us to
> revert the fix.
> 
> whether or not it is a bug fix or an enhancement, and I would just
> like a versioning scheme that allows these changes to be expressed. 

So the requirement is that the versioning scheme must support
branching, and must support expressing NBC changes on any version?

This requirement isn't present in the current draft, AFAICT.

(not that I support it as a requirement)

> None of this changes the fact that I think that we should be avoiding
> making these changes in the first place.  I.e. I think that there is a
> clear separation between what the versioning scheme should be able to
> express, and what is recommended practice.
> 
> 
> >
> >
> > 2) SEMVER to the rescue?
> SEMVER classic does not support making bug fixes (even bc ones) on
> older releases.
> 
> versioning scheme that allows them.  Or you do what Rob Shakir
> suggests and use deviations for this instead, which I think is a
> misuse of deviations.

But, as you state in the solution draft, not even modified semver can
determine if a specific change is NBC or not.  It seems you would need
the entire history to determine this - which goes against the original
idea that a client should be able to easily determine if a new version
is NBC or not, compared to the version the client knows.

/martin

---

From: Robert Wilton <[email-masked]>
Date: Wed, 14 Nov 2018 15:47:58 +0000

Hi Martin,

> branching, and must support expressing NBC changes on any version?

I deem that 1.4 (without the sentence about versioning by software 
release) defines this:

        1.4  The solution MUST allow for backwards-compatible
             enhancements and bug fixes to be allowed in any non-latest
             release.

Although this text is ambiguous, perhaps stating it more clearly, I see 
the requirement as:

        1.4  The solution MUST allow for backwards-compatible
             enhancements, and backward-compatible or non-backwards compatible
             bug fixes to be allowed in non-latest releases.

> (not that I support it as a requirement)

But vendors *have* to do this today.  I don't think telling our 
customers that no, we can't fix that bug, because the versioning scheme 
doesn't allow it is really pragmatic.

Rob Shakir also indicated in the Netmod meeting that he does not support 
this requirement.  However, this conflicts with the fact that there are 
examples in OpenConfig when it has been necessary to apply fixes to 
older versions, which has been achieved using deviations.

> is NBC or not, compared to the version the client knows.

The (m|M) is intended to be a tool of last resort.  So provide a 
mechanism to make bug fixes to older versions, but don't encourage it.  
It provides a mechanism to provide bug fixes on an existing release, but 
at the cost that you lose some of the benefits of semver (which is 
unavoidable).

If the server is on a version of the form "A.B.X(m)" then the client 
knows that all changes between "A.B.0" and "A.B.X(m)" are backwards 
compatible.  If the version is "A.B.X(M)" then the client knows that 
there is at least one nbc change between "A.B.0" and "A.B.X(M)".  The 
client does not know whether going from "A.B.X(m|M)" to "A.B+1.0" is a 
backwards incompatible change or not.

Thanks,
Rob

> .

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 14 Nov 2018 17:04:26 +0100 (CET)

Robert Wilton <[email-masked]> wrote:
> Although this text is ambiguous, perhaps stating it more clearly, I
> see the requirement as:
> 
> customers that no, we can't fix that bug, because the versioning
> scheme doesn't allow it is really pragmatic.

The pragmatic thing to do is to violate the rule when you think your
customers can accept it, and introduce a NBC even though it doesn't
follow 7950 section 11 rules.

> Rob Shakir also indicated in the Netmod meeting that he does not
> support this requirement.  However, this conflicts with the fact that
> there are examples in OpenConfig when it has been necessary to apply
> fixes to older versions, which has been achieved using deviations.
> 
> 
> >
> >
> mechanism to make bug fixes to older versions, but don't encourage
> it.  It provides a mechanism to provide bug fixes on an existing
> release, but at the cost that you lose some of the benefits of semver
> (which is unavoidable).

Suppose I have version 1.2.3M.  Now I make an editorial fix to this
module, what is the next version?  1.2.4 or 1.2.4M?

/martin

---

From: Robert Wilton <[email-masked]>
Date: Wed, 14 Nov 2018 16:37:59 +0000

> follow 7950 section 11 rules.

Yes, but say we adopt vanilla semver, it cannot express this as an NBC 
change, or even as a BC change!

Say the released software on the server is using module version 1.2.0.  
Module versions 1.3.0 and 2.0.0 already exist with new added 
functionality (that we can't/won't backport)

Is it really right to label the fixed version as a patch update (e.g. 
going from 1.2.0 to 1.2.1, which semver states is an editorial change) 
even through it is actually an NBC change and could break some clients?

This is the reason that I introduced the modified semver scheme, so that 
it could be labelled as "1.2.1(M)".  I.e. the client can easily 
determine that it is an NBC update.

Of course, if there isn't a "2.0.0" version at the time that an NBC fix 
is made to 1.2.0 then it should follow the normal semver rules and use 
"2.0.0" instead.  The "(m|M)" notation is only used whether there isn't 
a semver number available to use because it has already been used for 
new development.  Similarly, if the code change that was present in 
"1.3.0" was only an BC bugfix, and if the fix being made was BC, then it 
would be recommended to use "1.4.0" for the bug fix version.  I.e. 
although I don't think that it pragmatic to require that new 
functionality be backported to an old release to pick up a bug fix, it 
does seem more pragmatic to think that a bug fix could be back ported to 
an older software release, if required. But ultimately I think that it 
is at the discretion of the module author to decide where to put the 
fix, with the suggestion that if it is created as  "1.2.1(M)" then the 
fix should probably also be made to head of the tree ("3.0.0") as well.

The real aim of the modified semver solution is to be able to describe 
these changes, rather than say that they are good or bad.  Hence 
guidance would be provided to recommend how they are used.

---

If the modified semver solution is really disliked, then an alternative 
is to just not use semver, and use a 3 digit version number, e.g. as I 
proposed to Juergen yesterday, when the "patch" field always means bug 
fix.  This has the downside that it is incompatible with what OpenConfig 
is doing though, and clients would have to use a schema compare tool to 
understand the changes between two modules.

> module, what is the next version?  1.2.4 or 1.2.4M?

1.2.4M.

Once you have put a "m" or "M" on an x.y._ branch, it is effectively 
poisoned to always keep the letter on that branch (although 'm' would be 
upgraded to 'M' in a NBC change is made). E.g. 3.4.5m could go to 3.4.6M.

My assumption here is that 'm' and 'M' should only be used 
infrequently.  If they end up being used heavily then the scheme does 
not work so well.  But in this case, this really points to the fact that 
the YANG modules need more review and to be more stable before they are 
released.  Again, the aim here is to make the m|M syntax to be somewhat 
limited to discourage and somewhat limit its use.

E.g. a more general solution here would be to allow a separate semver 
for each place where a bugfix is required.  E.g. if you but an nbc 
bugfix on branch "1.2.3" then that could be "1.2.3/2.0.0" ... but I see 
that this is adding way too much branch flexibility and complexity.

Thanks,
Rob

---

From: Andy Bierman <[email-masked]>
Date: Wed, 14 Nov 2018 20:42:56 -0800

On Wed, Nov 14, 2018 at 7:48 AM Robert Wilton <[email-masked]> wrote:

> Robert Wilton <[email-masked]> <[email-masked]> wrote:
>
This new 1.4 (2nd one) is much better. Thanks for rewriting it.

Andy

> But vendors *have* to do this today.  I don't think telling our customers
> that no, we can't fix that bug, because the versioning scheme doesn't allow
> it is really pragmatic.
>
> examples in OpenConfig when it has been necessary to apply fixes to older
> versions, which has been achieved using deviations.
>
>
> The (m|M) is intended to be a tool of last resort.  So provide a mechanism
> to make bug fixes to older versions, but don't encourage it.  It provides a
> mechanism to provide bug fixes on an existing release, but at the cost that
> you lose some of the benefits of semver (which is unavoidable).
>
> If the server is on a version of the form "A.B.X(m)" then the client knows
> that all changes between "A.B.0" and "A.B.X(m)" are backwards compatible.
> If the version is "A.B.X(M)" then the client knows that there is at least
> one nbc change between "A.B.0" and "A.B.X(M)".  The client does not know
> whether going from "A.B.X(m|M)" to "A.B+1.0" is a backwards incompatible
> change or not.
>
> Thanks,
> Rob
>
>
> /martin
> .
>
>
>