Subject: [netmod] Proposed consensus mail on yang-00000: module update rules

From: David Partain <[email-masked]>
Date: Tue, 28 Oct 2008 16:02:50 +0100

Greetings,

This is a call for consensus about issue 'yang-00000: module
update rules'.  See the wiki at
http://wiki.tools.ietf.org/wg/netmod/trac/wiki/Issues_yang#issue-00000

PLEASE speak up if you favor or object to the proposed rules,
including people who attended the interim.  Silence is impossible
to interpret.

Issue: Need to define the rules for how modules are updated (new
revisions published).

The first day of the interim meeting spent a great deal of time
talking about this issue.  This mail does not give a blow-by-blow
recounting of the discussion but rather tries to summarize what
was discussed and the conclusions.

The general principle is that when updating a module without name
changes, the module may become less constrained, but never more
constrained.

 - Why do we need to have these rules?

   * protect old client applications (old NETCONF clients
     talking to a new NETCONF server).  You're not allowed to
     change something that makes a client that was previously
     compliant suddenly no longer compliant.
   * protect those importing a module

 - We're talking about rules for:

   * New building blocks (e.g., typedefs)
   * New definitions
   * Organizational changes (e.g., inline to named types)
   * Bug fixes

 - Rules for adding things to an existing module:

A proposal from Balazs Lengyel:

  Balazs Lengyel provided a list of changes that Ericsson considers
  "compatible" changes:

    - adding new optional nodes
    - adding new RPCs
    - extending cardinality
    - extending the data range
    - change of default value of an optional attribute
    - relaxing rules on semantic change (e.g., config ->
      non-config, read-only -> read-write
    - adding data to notifications
    - updates to description clauses
    - updates to extensions

  where the policy is that if it's not on that list, it's
  considered "an incompatible change" and shouldn't be done without
  changing the namespace.

  Incompatible changes:

    - leaf order changes

  This fed into the subsequent suggested rules...

Changes affecting types:

  Martin Björklund asked:

    - Can you extend the value space when updating a document?
    - Can you restrict the value space when updating a document?

  For an answer, see the general principle above.

  In the end, changes that affect storage requirements for a type
  are not allowed.  (Editor: the consensus here is entirely
  unclear.)

Leaf change rules:

  (Thanks to Balazs for his notes here!)

  What are you allowed / not allowed to change about a leaf
  definition (given the general principle above):

  Changes that are permitted:

    - status with the current/deprecated/obsolete progression
    - new typedefs
    - new groupings
    - expand value range
    - remove musts
    - new optional non-mandatory nodes
    - allow new rpc
    - allow new notifications
    - allow new data in existing notifications
    - allow new optional parameters to rpc input(?) 
    - allow new parameters to rpc output(?) 
    - remove mandatory
    - reference (updated)
    - description (semantic equivalence)
    - length
    - add default (Editor: unsure of this!)
    - add new bits
    - add new enum values at the end of the list
    - config false ->true
    - (all kinds of "non-changes" as well)

  Changes that are not permitted:

    - delete nodes
    - leaf order changes (due to strict encoding order)
    - change basic type (e.g., uint8 -> int8)
    - change length restriction of string
    - make a type into union
    - no new mandatory nodes
    - change datatype of a leaf (underlying datatype at the
    - base of the type hierarchy)
    - add new membertypes to a union
    - change units
    - change default
    - add if-feature
    - change the value of enum or bits
    - add enums in the middle

Typedef change rules:

  The typedef rules discussion immediately morphed into a
  discussion of "import by revision", which will be reported on
  separately.  However, we reached rough consensus to proceed
  with "import by revision", and, by deciding this, we can use
  the same rules for typedefs as we have for leafs.  We have not
  talked about the rules for groupings yet.

  Martin Björklund is going to take the lead in writing a
  proposal for import by revision to cover import and include, as
  well as the rules for what can change between revisions
  (typedefs, groups, leafs, etc.).
_______________________________________________
netmod mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/netmod

---

From: Balazs Lengyel <[email-masked]>
Date: Tue, 28 Oct 2008 21:00:25 +0100

I support this. Balazs

-- 
Balazs Lengyel                       Ericsson Hungary Ltd.
TSP System Manager
ECN: 831 7320                        Fax: +36 1 4377792
Tel: +36-1-437-7320     email: [email-masked]
_______________________________________________
netmod mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/netmod

---

From: David Reid <[email-masked]>
Date: Tue, 28 Oct 2008 16:54:54 -0400

> including people who attended the interim.  Silence is impossible

I generally support the proposed rules. I'd like to see a more specific
proposal on change rules. Do I understand correctly that Martin is going
to write that proposal based on the summary in this thread? I think the
list Balazs provided is very good.

-David Reid

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 28 Oct 2008 22:27:10 +0100 (CET)

David Reid <[email-masked]> wrote:
> I'd like to see a more specific
> to write that proposal based on the summary in this thread?

Yes.  This is what I currently have written, which I planned to
include in the -02 draft, published next monday (10/11).  The text is
mostly copied from the SMIv2 text.

/martin

* Updating a Module

As experience is gained with a module, it may be desirable to revise
that module.  However, changes are not allowed if they have any
potential to cause interoperability problems between a client using an
original specification and a server using an updated specification.

For any published change, a new "revision" statement (^revision^)
SHOULD be included in front of the existing revision statements.
Furthermore, any necessary changes MUST be applied to any meta
statements, including the "organization" and "contact" statements
(^organization^, ^contact^).

Note that definitions contained in a module are available to be
imported by any other module, and are referenced in "import"
statements via the module name.  Thus, a module name MUST NOT be
changed.  Furthermore, the "namespace" statement MUST NOT be changed,
since all XML elements are encoded in the namespace.

Obsolete definitions MUST NOT be removed from modules since their
identifiers may still be referenced by other modules.

A definition may be revised in any of the following ways:

- An "enumeration" type may have new enums added, provided the old
  enums's values do not change.
- A "bits" type may have new bits added, provided the old bits's
  positions do not change. 
- A "range", "length", or "pattern" statement may expand the allowed
  value space.
- A "default" statement may be added.
- A "units" statement may be added.
- A "reference" statement may be added or updated.
- A "must" statement may be removed or its constraint relaxed.
- A "mandatory" statement may be removed or changed from "true" to
  "false".
- A "min-elements" statement may be removed, or changed to require less
  elements.
- A "max-elements" statement may be removed, or changed to allow more
  elements.
- A "description" statement may be added or clarified without changing
  the semantics of the definition.
- New typedefs, groupings, rpc, notifications, extensions, features,
  and identities may be added.
- New data definition statements may be added if they do not add
  mandatory nodes (^mandatory-nodes^) to existing nodes, or if they
  are conditionally dependent on a new feature (i.e. have a
  "if-feature" statement which refers to a new feature).
- A new "case" statement may be added.
- A node that was non-configuration may be changed to represent
  configuration, provided it is not mandatory (^mandatory-nodes^).
- An "if-feature" statement may be removed, provided its node is not
  mandatory (^mandatory-nodes^). 
- A "status" statement may be added, or changed from "current" to
  "deprecated" or "obsolete", or from "deprecated" to "obsolete".
- A "type" statement may be replaced with another "type" statement
  which does not change the syntax or semantics of the type.  For
  example, an inline type definition may be replaced with a typedef.
- Any set of data definition nodes may be replaced with another set of
  syntactically and semantically equivalent nodes.  For example, a set
  of leafs may be replaced by a uses of a grouping with the same
  leafs.
- A module may be split into a set of submodules, or submodule may be
  removed, provided the definitions in the module do not change in any
  other way than allowed here.
- The "prefix" statment may be changed, provided all local uses of the
  prefix also are changed.
 
Otherwise, if the semantics of any previous definition are changed
(i.e. if a non-editorial change is made to any definition other than
those specifically allowed above), then this MUST be achieved by a new
definition with a new identifier.

[Editor's Note: These rules work as long as we have import/include by
revision]
_______________________________________________
netmod mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/netmod

---

From: David Reid <[email-masked]>
Date: Wed, 29 Oct 2008 15:24:19 -0400

I like the text Martin wrote on change rules. In general, I like the idea
of listing what is allowed and saying everything else is not allowed since
we can't possible list everything that is not allowed. But there are some
cases that may be ambiguous if we don't explicitly say it is not allowed. For
example:

Can I change an int8 to an int16? The consensus at the interim was no, but
someone reading the spec may interpret that change as simply expanding the
range and thus be a legal change.

Are leaf order changes allowed? Again, the consensus at the interim was no,
but that may not be clear in the spec. It is clear that you cannot delete a
leaf, but it is not clear that you cannot change to ordering.

-David Reid

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 29 Oct 2008 22:24:52 +0100 (CET)

> range and thus be a legal change.

I'm guessing that it is this text that is not clear:

OLD:

  A "type" statement may be replaced with another "type" statement
  which does not change the syntax or semantics of the type.  For
  example, an inline type definition may be replaced with a typedef.

Is this text sufficient:

NEW:

  A "type" statement may be replaced with another "type" statement
  which does not change the syntax or semantics of the type.  For
  example, an inline type definition may be replaced with a typedef,
  but a int8 type cannot be replaced by a int16, since the syntax
  would change.

> leaf, but it is not clear that you cannot change to ordering.

Is this text ok:

  In statements which have any data definition statements as
  substatements, those data definition substatements MUST NOT be
  reordered.

Should I add an example?

  For example, the order among all leafs defined in a list must be
  maintained.

/martin

---

From: Phil Shafer <[email-masked]>
Date: Wed, 29 Oct 2008 17:50:12 -0400

>  but a int8 type cannot be replaced by a int16, since the syntax

If you think of text (XML) representation, the difference between
int8 and int16 is a matter of range.  If we want to make the "type
size guarantee" that we talked about at the interim, perhaps the
best way to say this (without talking about implementation-specific
issues like "sizeof" is to just say you can't change the base type
which underlies the chain of derived types (the ultimate base type).
This makes the rule concrete without talking about "syntax", "size",
or anything similar.  The "digress" should have a good explanation
of why we do/need this.

Thanks,
 Phil

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 29 Oct 2008 23:00:12 +0100 (CET)

Phil Shafer <[email-masked]> wrote:
> If we want to make the "type

Yes, I was thinking of using the term "base type", but as the term is
defined, it can't really be used.  So we might say "the base type
which underlies the chain of derived types", but if we do this we
really have to explain what this means, probably in a separate
subsection.  For example, how are unions treated?

/martin

---

From: Phil Shafer <[email-masked]>
Date: Wed, 29 Oct 2008 22:15:11 -0400

Martin Bjorklund writes:
>For example, how are unions treated?

The root base type of a union is "union", so you should be able to
do anything without triggering venom from size conscious programmers.

Of course, this means that anyone who cares about extensibility
will use unions of a single type, which defeats the whole purpose
of this type lockdown (which is quite fine with me).

IMHO you need to be able to handle strings, since that's XML's root
base type.  Any optimizations on top of that are merely optimizations,
and code should be able to revert to strings where needed (on the
client side).

Thanks,
 Phil

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 30 Oct 2008 10:15:31 +0100

> or anything similar.  The "digress" should have a good explanation

I agree with this approach, at least for simple types. For unions and
such things we might need additional text.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>
_______________________________________________
netmod mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/netmod

---

From: David Reid <[email-masked]>
Date: Thu, 30 Oct 2008 12:44:16 -0400

I like the proposed changes. On the reorder issue, I like the example. It is
not absolutely necessary, but I think it helps.

-David Reid

---

From: Andy Bierman <[email-masked]>
Date: Thu, 30 Oct 2008 10:25:13 -0700

David Reid wrote:
>....

Not the syntax, it is the semantics that will change.
The meaning of 'min' and 'max' in ranges is fixed,
based on the specific typedef chain.  Changing int8 to int16
changes the upper-bound, and the meaning of any 'max' keyword
upstream in the typedef chain.

>>   For example, the order among all leafs defined in a list must be

perhaps there should be an addition to the 'Terms' section
for the term 'schema order'?

Any data-def-stmt reordering (in groupings too) should be
an error.  Reordering enums or bits is an error.

I owe Martin some text on canonical ordering, and
hopefully it will be clear that the position of
any nodes ordered by YANG file definition (sibling order
within a parent construct) should remain stable for
the lifetime of the module.

>> /martin

Andy

---

From: "Randy Presuhn" <[email-masked]>
Date: Thu, 30 Oct 2008 10:57:09 -0700

Hi -

> From: "Andy Bierman" <[email-masked]>
> To: "David Reid" <[email-masked]>
> Cc: <[email-masked]>
> Sent: Thursday, October 30, 2008 10:25 AM
> Subject: Re: [netmod] Proposed consensus mail on yang-00000: module update rules
...
> upstream in the typedef chain.
...

This is actually an argument against adding values to a range in general.
Perhaps not surprisingly, this is consistent with the GDMO rules for
subclassing - a subclass can only *restrict*, not extend, the ranges of
inherited attributes in those kinds of models.  Though at first glance
this might seem backwards, it's actually the right thing to do.  (I'm
agreeing with Andy)  Even though netmod isn't O-O, it may be helpful
to think of it this way: a standardized superclass in that world typically
sets out the *architectural* constraints on whatever sort of thing is
being described, and vendor-specific subclasses derived from that
superclass  impose the additional limitations that characterize
particular implementations.  In that view, the only permissible refinement
of an attribute would be to further constrain its value set from the set of
values that are *architecturally* possible in the superclass.

This is *NOT* how things worked in SNMP SMI, which explicitly allows adding
values to enumerations, for example.

Consequently, I think this group needs to think about metamodels (in
the very broad sense) a bit, because a lot of the decisions about what
are and what are not "compatible" changes / extensions will flow from it.

Randy

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 30 Oct 2008 20:56:28 +0100 (CET)

> are and what are not "compatible" changes / extensions will flow from it.

All module upgrade rules we have are derived from two design goals:

  - Protect old clients
    We want a client that uses version x of a module to be able to
    function when talking to a server implementing version x+1.

  - Protect importers
    A new published module version must not break existing other
    modules that imports from the module.

So for example, the value space of a leaf must not be restricted in
the new version, since if e.g. an enumeration is removed, an old
client that tries to set the removed value will fail.

/martin

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 30 Oct 2008 20:57:33 +0100 (CET)

> upstream in the typedef chain.

The editor does not understand the edits.  Can you provide some text?

/martin

---

From: Balazs Lengyel <[email-masked]>
Date: Fri, 31 Oct 2008 18:10:03 +0100

Hello,
IMHO it would be nice to have these two principles in the text.
Balazs

---

From: Balazs Lengyel <[email-masked]>
Date: Fri, 31 Oct 2008 18:14:15 +0100

>   "if-feature" statement which refers to a new feature).
BALAZS: So a new mandatory leaf at the root level is allowed? I think it must not. That is a 
clear incompatible change will brake old managers.

Balazs