Subject: [netmod] Deviations and augmentations

From: Robert Wilton <[email-masked]>
Date: Mon, 12 Nov 2018 16:29:20 +0000

In the Thursday Netmod meeting, it was interesting to hear Rob Shakir 
describe how deviations and augmentations are used in OpenConfig to add 
functionality into an older YANG model where the semver rules prevent 
the version number from being incremented.

Further, I think that someone (Martin?) stated on the audio bridge that 
this was an intended/allowed behavior for deviations.

This surprised me, because I thought that RFC 7950 was quite explicit 
that this is not what deviations are intended for.  My reading of RFC 
7950 is that the deviation statement represents the case where the 
server *implementation* does not match the *specification*.  However, 
the versioning issue that we are discussing are bug fixes/changes in the 
specification rather than the bug fixes in the implementation.

Personally, I'm really not keen on using deviations to represent bug 
fixes to older YANG models for three reasons:

(i) It is changing the meaning of deviation.  It is much cleaner to keep 
the meaning of deviation statements as they are defined today, and not 
conflate their semantics.
(ii) A different mechanism is used to put a bug fix into an older branch 
rather than in the head of the development.
(iii) For clients to track the lifecycle of modules they would not only 
need to know the module version number but would also need to find and 
track all associated deviation modules.  This seems significantly more 
complex for clients than the modified semver that was proposed.

---

I think that has also been some suggestion that augmentations (or 
duplicate YANG modules with their major version number changed) can be 
used to make bug fixes in a completely backwards compatible way.  
However, I still don't understand a robust scheme of how this works.

---

Finally, there were some comments about using augmentation modules for 
enhancements.  This is fine, where appropriate (e.g. a non trivial 
number of data nodes are being added as an enhancement) then a separate 
module may be the right way to go. But here, I presume that the new 
functionality will always be tracked by that separate module.  If that 
functionality folds back into the original module at some point in the 
future, then obviously a non backwards compatible version change is 
being forced on to the client, along with additional work on the server 
as well.

I think that there are also many cases where the number of data nodes 
being added via an enhancement is small compared to the size of the 
module being updated.  In this case I believe that it better to add 
these data nodes into the module itself, perhaps predicated under 
if-feature if appropriate.

Thanks,
Rob

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 12 Nov 2018 17:33:51 +0100 (CET)

Robert Wilton <[email-masked]> wrote:
> describe how deviations and augmentations are used in OpenConfig to
> add functionality into an older YANG model where the semver rules
> prevent the version number from being incremented.
> 
> Further, I think that someone (Martin?) stated on the audio bridge
> that this was an intended/allowed behavior for deviations.

I said that using augmentations (not deviations) was one idea we
originally had for solving the "branching problem".

I think that this works for OC b/c they don't branch their modules.
Hence I think it is important that we decide if branching is a
requirement or not.

/martin

> the versioning issue that we are discussing are bug fixes/changes in
> the specification rather than the bug fixes in the implementation.
> 
> (i) It is changing the meaning of deviation.  It is much cleaner to
> keep the meaning of deviation statements as they are defined today,
> and not conflate their semantics.
> (ii) A different mechanism is used to put a bug fix into an older
> branch rather than in the head of the development.
> (iii) For clients to track the lifecycle of modules they would not
> only need to know the module version number but would also need to
> find and track all associated deviation modules.  This seems
> significantly more complex for clients than the modified semver that
> was proposed.
> 
> ---
> 
> number of data nodes are being added as an enhancement) then a
> separate module may be the right way to go. But here, I presume that
> the new functionality will always be tracked by that separate module. 
> If that functionality folds back into the original module at some
> point in the future, then obviously a non backwards compatible version
> change is being forced on to the client, along with additional work on
> the server as well.
> 
> _______________________________________________
> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod

---

From: Andy Bierman <[email-masked]>
Date: Mon, 12 Nov 2018 09:04:34 -0800

> Hence I think it is important that we decide if branching is a
Branching is a solution, not a requirement.
IMO deviate(not-supported) is intended to let the server describe what it
supports
without requiring branching. Clients already support deviations.

> /martin

Andy

---

From: Robert Wilton <[email-masked]>
Date: Mon, 12 Nov 2018 17:08:42 +0000

> originally had for solving the "branching problem".
Ah, OK. I agree that makes sense.

> Hence I think it is important that we decide if branching is a
So, I think that this probably works for adding enhancements, but not 
for the (arguably more important) bug fix case, unless there is a 
reasonable solution to having two config data nodes both modifying the 
same underlying property.  Perhaps under some reasonable constraints 
this could be made to work - but I don't know.

Of course, even for enhancements it is not necessarily a perfect 
solution.  E.g. backporting some subset of a module already 
coded/implemented in latest to an older release.  And yes, we really do 
get asked to do this sometimes, although it is relatively rare.

Thanks,
Rob

> .

---

From: Ebben Aries <[email-masked]>
Date: Mon, 12 Nov 2018 12:13:32 -0700

> Hence I think it is important that we decide if branching is a

Does this not already somewhat exist today by way of each vendors
augments and deviations to OC models?  While branching doesn't exist
within OC directly, each implementation today supports *a* version at a
given point in time and deviations and augments exist within these
"branches". Occasionally the need arises to introduce changes in the
vendor provided modules as a stopgap that are ultimately mapped to a sw
release.

This could be fixes or additional functionality not previously there to
satisfy not having a customer have to upgrade their network element
and/or move to more recent modules

> > https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_netmod&d=DwIFAw&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=GIehbDpQlo31lSi6WbnEkA&m=8si7cLdi3y5avgA6Nvi0V0TixjVoKFudWwp3mJNat5I&s=4OHPx7mvkQdY9-M_M5HEOcYS566LOUuNecztVjp_NFw&e=
> 
> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_netmod&d=DwIFAw&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=GIehbDpQlo31lSi6WbnEkA&m=8si7cLdi3y5avgA6Nvi0V0TixjVoKFudWwp3mJNat5I&s=4OHPx7mvkQdY9-M_M5HEOcYS566LOUuNecztVjp_NFw&e=

---

From: Balázs Lengyel <[email-masked]>
Date: Tue, 13 Nov 2018 13:46:59 +0000

Hello,

We also need a method for removing stuff. It does happen that
some functionality is deemed not important enough, outdated, too
expensive to maintain, so we want to remove it.

Augment is clearly not the tool for that.

Deviations are not intended for that  (from rfc 7950: "server
deviation: A failure of the server ...")

So we still need Semver(or something akin) and the possibility to
do NBC changes.

Balazs

On 2018. 11. 12. 18:08, Robert Wilton
wrote:

Robert Wilton <[email-masked]>
wrote:

In the Thursday Netmod meeting, it was
interesting to hear Rob Shakir

describe how deviations and augmentations are used in
OpenConfig to

add functionality into an older YANG model where the semver
rules

Further, I think that someone (Martin?) stated on the audio
bridge

I think that this works for OC b/c they don't branch their
modules.

requirement or not.

So, I think that this probably works for adding enhancements, but
not for the (arguably more important) bug fix case, unless there
is a reasonable solution to having two config data nodes both
modifying the same underlying property.  Perhaps under some
reasonable constraints this could be made to work - but I don't
know.

Of course, even for enhancements it is not necessarily a perfect
solution.  E.g. backporting some subset of a module already
coded/implemented in latest to an older release.  And yes, we
really do get asked to do this sometimes, although it is
relatively rare.

Thanks,

Rob

/martin

This surprised me, because I thought
that RFC 7950 was quite explicit

that this is not what deviations are intended for.  My reading
of RFC

7950 is that the deviation statement represents the case where
the

server *implementation* does not match the *specification*.
However,

the versioning issue that we are discussing are bug
fixes/changes in

the specification rather than the bug fixes in the
implementation.

Personally, I'm really not keen on using deviations to
represent bug

(i) It is changing the meaning of deviation.  It is much
cleaner to

keep the meaning of deviation statements as they are defined
today,

(ii) A different mechanism is used to put a bug fix into an
older

(iii) For clients to track the lifecycle of modules they would
not

only need to know the module version number but would also
need to

significantly more complex for clients than the modified
semver that

was proposed.

---

I think that has also been some suggestion that augmentations
(or

duplicate YANG modules with their major version number
changed) can be

used to make bug fixes in a completely backwards compatible
way.

However, I still don't understand a robust scheme of how this
works.

---

Finally, there were some comments about using augmentation
modules for

enhancements.  This is fine, where appropriate (e.g. a non
trivial

separate module may be the right way to go. But here, I
presume that

the new functionality will always be tracked by that separate
module.

If that functionality folds back into the original module at
some

point in the future, then obviously a non backwards compatible
version

change is being forced on to the client, along with additional
work on

the server as well.

I think that there are also many cases where the number of
data nodes

being added via an enhancement is small compared to the size
of the

module being updated.  In this case I believe that it better
to add

these data nodes into the module itself, perhaps predicated
under

Thanks,

Rob

netmod mailing list

[email-masked]

.

netmod mailing list

[email-masked]

--
Balazs Lengyel Ericsson Hungary Ltd.
Senior Specialist
Mobile: +36-70-330-7909 email: [email-masked]

---

From: Robert Wilton <[email-masked]>
Date: Tue, 13 Nov 2018 14:01:11 +0000

Hi Balazs,

On 13/11/2018 13:46, Balázs Lengyel wrote:
>
> Hello,
>
> We also need a method for removing stuff. It does happen that some 
> functionality is deemed not important enough, outdated, too expensive 
> to maintain, so we want to remove it.
>
>   * Augment is clearly not the tool for that.
>   * Deviations are not intended for that  (from rfc 7950: "server
> So we still need Semver(or something akin) and the possibility to do 
> NBC changes.
>
But one could perhaps argue that nodes should only be removed (i.e. 
obsoleted) at the development head, where semver is allowed.

Do you think that we also need to support obsoleting YANG nodes in 
branches that have already been released?

Thanks,
Rob

> Balazs
>
> On 2018. 11. 12. 18:08, Robert Wilton wrote:
>>
>>
>> reasonable solution to having two config data nodes both modifying 
>> the same underlying property.  Perhaps under some reasonable 
>> constraints this could be made to work - but I don't know.
>>
>> coded/implemented in latest to an older release.  And yes, we really 
>> do get asked to do this sometimes, although it is relatively rare.
>>
>> Thanks,
>> Rob
>>
>>>
>>>
>>> /martin
>>>
>>>
> Mobile: +36-70-330-7909              email:[email-masked]

---

From: Andy Bierman <[email-masked]>
Date: Tue, 13 Nov 2018 07:17:37 -0800

On Tue, Nov 13, 2018 at 5:46 AM, Balázs Lengyel <[email-masked]

> functionality is deemed not important enough, outdated, too expensive to
> maintain, so we want to remove it.
>
>    - Augment is clearly not the tool for that.
>    - Deviations are not intended for that  (from rfc 7950: "server
Removing nodes is easy with the status-stmt. Update the module and set the
status to deprecated or obsolete.

Andy

>
>
> So we still need Semver(or something akin) and the possibility to do NBC
> changes.
>
> Balazs
> Robert Wilton <[email-masked]> <[email-masked]> wrote:
>
> So, I think that this probably works for adding enhancements, but not for
> the (arguably more important) bug fix case, unless there is a reasonable
> solution to having two config data nodes both modifying the same underlying
> property.  Perhaps under some reasonable constraints this could be made to
> work - but I don't know.
>
> coded/implemented in latest to an older release.  And yes, we really do get
> asked to do this sometimes, although it is relatively rare.
>
> Thanks,
> Rob
>
>
>
> /martin
>
>

---

From: Robert Wilton <[email-masked]>
Date: Tue, 13 Nov 2018 15:28:36 +0000

On 13/11/2018 15:17, Andy Bierman wrote:
>
>
> On Tue, Nov 13, 2018 at 5:46 AM, Balázs Lengyel 
> <[email-masked] <mailto:[email-masked]>> wrote:
>
>     Hello,
>
>     functionality is deemed not important enough, outdated, too
> Removing nodes is easy with the status-stmt. Update the module and set 
> the status to deprecated or obsolete.

Yes, but obsoleting nodes should be regarded as a 
non-backwards-compatible change because it can break clients that were 
relying on those nodes.

Thanks,
Rob

>>>     Robert Wilton <[email-masked]> <mailto:[email-masked]> wrote:
>>>>     [email-masked] <mailto:[email-masked]>
>>>>     <https://www.ietf.org/mailman/listinfo/netmod>
>>>     .
>>>
>>
>>     [email-masked] <mailto:[email-masked]>
>>     <https://www.ietf.org/mailman/listinfo/netmod>
>
>     -- 
>     Mobile: +36-70-330-7909              email:[email-masked]  <mailto:[email-masked]>  
>
>
>     [email-masked] <mailto:[email-masked]>
>     <https://www.ietf.org/mailman/listinfo/netmod>
>
>

---

From: Andy Bierman <[email-masked]>
Date: Tue, 13 Nov 2018 07:32:14 -0800

On Tue, Nov 13, 2018 at 7:28 AM, Robert Wilton <[email-masked]> wrote:

> On Tue, Nov 13, 2018 at 5:46 AM, Balázs Lengyel <
> [email-masked]> wrote:
>
>> Hello,
>>
> Yes, but obsoleting nodes should be regarded as a non-backwards-compatible
> change because it can break clients that were relying on those nodes.
>

I don't think RFC 7950 says that.
Removing outdated functionality is exactly what the status-stmt is for.
IMO we should learn to use the YANG that is already there.

Andy

---

From: Robert Wilton <[email-masked]>
Date: Tue, 13 Nov 2018 15:54:11 +0000

On 13/11/2018 15:32, Andy Bierman wrote:
>
>
> On Tue, Nov 13, 2018 at 7:28 AM, Robert Wilton <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>
>>     <[email-masked]
>>     <mailto:[email-masked]>> wrote:
>>
>>         Hello,
>>
>>         some functionality is deemed not important enough, outdated,
>>         too expensive to maintain, so we want to remove it.
>>
>>           * Deviations are not intended for that (from rfc 7950:
>>             "server deviation: A failure of the server ...")
>>
>>
>>     Removing nodes is easy with the status-stmt. Update the module
>>     and set the status to deprecated or obsolete.
>
>     non-backwards-compatible change because it can break clients that
>     were relying on those nodes.
>
>

RFC 7950 states '"obsolete" means that the definition is obsolete and 
SHOULD NOT be implemented and/or can be removed from implementations.'

So if there was a client using those definitions it would break, hence 
it is a non-backwards-compatible change.

> Removing outdated functionality is exactly what the status-stmt is for.

Agreed.

> IMO we should learn to use the YANG that is already there.

We are trying to make what we have today better.

Specifically, so that a client can look at a semver numbers between two 
modules revisions and determine whether or not they might be broken by 
that change.

Today, just using the status element alone to mark removed nodes means 
that a client would have to check for all changes in the module between 
two revisions to determine whether or not the new module revision is 
backwards compatible with the old one.

Thanks,
Rob

>     Thanks,
>     Rob
>
>
>
> Andy
>
>>
>>
>>     Andy
>>
>>
>>         So we still need Semver(or something akin) and the
>>         possibility to do NBC changes.
>>
>>         Balazs
>>
>>>>         <mailto:[email-masked]> wrote:
>>>         So, I think that this probably works for adding
>>>         enhancements, but not for the (arguably more important) bug
>>>         fix case, unless there is a reasonable solution to having
>>>         two config data nodes both modifying the same underlying
>>>         property. Perhaps under some reasonable constraints this
>>>         could be made to work - but I don't know.
>>>
>>>         Of course, even for enhancements it is not necessarily a
>>>         perfect solution.  E.g. backporting some subset of a module
>>>         already coded/implemented in latest to an older release. 
>>>         And yes, we really do get asked to do this sometimes,
>>>         although it is relatively rare.
>>>
>>>         Thanks,
>>>         Rob
>>>
>>>>
>>>>
>>>>         /martin
>>>>
>>>>

---

From: Andy Bierman <[email-masked]>
Date: Tue, 13 Nov 2018 08:19:44 -0800

On Tue, Nov 13, 2018 at 7:54 AM, Robert Wilton <[email-masked]> wrote:

> So if there was a client using those definitions it would break, hence it
> is a non-backwards-compatible change.
>
>
>
>From RFC 7950, sec 11:

   o  A "status" statement may be added, or changed from "current" to
      "deprecated" or "obsolete", or changed from "deprecated" to
      "obsolete".

> modules revisions and determine whether or not they might be broken by that
> change.
>
> that a client would have to check for all changes in the module between two
> revisions to determine whether or not the new module revision is backwards
> compatible with the old one.
>

It is quite easy to check if the status-stmt is set to deprecated or
obsolete.
SEMVER is not very useful, especially if the major version is incremented
even though there were no NBC changes.

Andy

>>> reasonable solution to having two config data nodes both modifying the same
>>> underlying property.  Perhaps under some reasonable constraints this could
>>> be made to work - but I don't know.
>>>

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 13 Nov 2018 18:06:52 +0100

On Tue, Nov 13, 2018 at 03:54:11PM +0000, Robert Wilton wrote:

> Today, just using the status element alone to mark removed nodes means that
> a client would have to check for all changes in the module between two

A client needs to know whether it is _affected_ by changes. This
requires to match what is used by a client against what has been
changed (including any changed deviations). A three digit number
simply does not tell you that.

<soap>
Corner cases:

- An implementation claims to support the latest semver but then
  deviates things back to an earlier version.

- An implementation claims to support an earlier semver but then
  deviates things forward to a more recent version.

Of course, nobody would do that...
</soap>

My point is that semver is _at best_ a hint whether there is a version
mismatch potentially affecting a client. It is not sufficient to
determine whether there is indeed a problem.

In other words, robust automation requires to match what is used by a
client against what has been updated (taking deviations into account
during the comparison).

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1 | 28759 Bremen | Germany
Fax:   +49 421 200 3103         <https://www.jacobs-university.de/>

---

From: Robert Wilton <[email-masked]>
Date: Tue, 13 Nov 2018 17:54:27 +0000

> client against what has been updated (taking deviations into account

I don't disagree with any of this, I'm also a proponent of a schema 
comparison tool.

I think that my main point regarding version numbers is:

  - If we are going to use semver then we should define and follow the 
rules strictly.  I.e. assuming that the semver has been populated 
correctly then it should be possible to determine whether the change is 
a nbc, bc, or editorial change.  I think that a solution where semver is 
done in a half-hearted way is entirely useless (e.g. if clients cannot 
trust the version number).  I think that there is a pretty clear 
requirement to support NBC changes to older revisions (i.e. I really 
mean bug fixes but a looser interpretation than your definition), and 
hence I don't think that vanilla semver is sufficient.

- An alternative is to not use a semver version number at all, instead 
we define a non semantic 3 digit version number that is something like:

  major = significant changes to the module have been made
  minor = smaller changes/feature enhancements to the module have been made
  patch = a bug fix to the module has been made.

Clients, module readers, etc, and then required to use a schema 
comparison tool to determine what the actual changes between two 
revisions are and whether or not they will be broken by those changes.

Thanks,
Rob

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 13 Nov 2018 19:26:21 +0100

On Tue, Nov 13, 2018 at 05:54:27PM +0000, Robert Wilton wrote:
>  - If we are going to use semver then we should define and follow the rules
> strictly.  I.e. assuming that the semver has been populated correctly then
> it should be possible to determine whether the change is a nbc, bc, or
> editorial change.  I think that a solution where semver is done in a
> half-hearted way is entirely useless (e.g. if clients cannot trust the
> version number).  I think that there is a pretty clear requirement to
> support NBC changes to older revisions (i.e. I really mean bug fixes but a
> looser interpretation than your definition), and hence I don't think that
> vanilla semver is sufficient.

But even if you make the 'YANG world' use semvers correctly (not sure
how you want to achieve that but lets pretend you manage), then the
three digit number does not tell a client whether it will work or not.
What should a client do if semver indicate nbc? What should it do if
semver indicates bc? What should it do if semver indicates editorial
change? Where is the "protocol specification" for this?
 
> - An alternative is to not use a semver version number at all, instead we
> define a non semantic 3 digit version number that is something like:
> 
> Clients, module readers, etc, and then required to use a schema comparison
> tool to determine what the actual changes between two revisions are and
> whether or not they will be broken by those changes.

The only robust solution is schema comparison and we may think about
what kind of annotations make such schema comparisons easier or faster
or how we can help clients to adapt (e.g., providing pointers where
definitions have moved, providing machine readable information
allowing tools to track which modules replace others etc.).

/js

---

From: Andy Bierman <[email-masked]>
Date: Tue, 13 Nov 2018 13:27:37 -0800

> So we still need Semver(or something akin) and the possibility to do NBC

This aspect of deviations is unfortunate because it really is a good idea to
limit the number of ways to do the same thing in YANG.

It was a mistake to limit so many legitimate schema alterations and also a
mistake
to characterize all schema modifications as a server failure. Maybe if it
was called "patch"
instead of "deviation".

We should encourage servers to provide the most accurate schema possible.
Many vendors already maintain detailed deviations modules.

Balazs

Andy

---

From: Robert Wilton <[email-masked]>
Date: Wed, 14 Nov 2018 10:48:14 +0000

> change? Where is the "protocol specification" for this?

In theory, for editorial or bc changes an existing client, coded 
correctly SHOULD work unchanged (assuming that it doesn't barf on 
operational state that it doesn't understand or config that it isn't 
aware of)  I.e. I see that it is effectively making the same promise 
that 7950 module updates rules are trying to enforce today (putting 
status obsolete aside).

For an nbc change they would need to check, manually or using a schema 
compare tool, to see what has changed.

But I agree that a scheme level version number is really giving a 
suggestion rather than a robust promise that the client won't break.

Even the schema compare tool doesn't guarantee that a client won't 
break, since it does not guarantee that the server implementation 
doesn't have any bugs.

The next step is a solid set of tests that can be run against the 
updated schema and servers, and check that they behave as expected.  Rob 
Shakir indicated that OpenConfig is busy building such as test framework.

Then I think that the operator still rolls out the changes gradually, 
monitors for unexpected conditions, and is able to roll back if 
something goes wrong.

But none of these stages prove that the network will still be fine 
afterwards, they just help catch issues earlier, where they should be 
easier to handle, and give a bit more confidence that the previous step.

> allowing tools to track which modules replace others etc.).

Yes, I think that all of this is a good idea.  But I still think that 
having a human readable version number per module is also helpful, even 
if it is only to help humans label the module revision in an easy way.  
E.g. I think that a version number is easier to parse/remember than a 
revision date.

Thanks,
Rob