Subject: [netmod] Y45 again

From: Martin Bjorklund <[email-masked]>
Date: Sat, 28 Mar 2015 16:08:37 +0100 (CET)

Hi,

This is another attempt to solve the conformance problem, building on
the idea that Lada suggested.

The proposed solution does not require any changes to the upgrade
rules from 6020 (specifically, a typedef/grouping can expand its value
space).

The solution relies on the observation that import may or may not
indicate a run-time dependency between modules.  If module A imports
module B, and A just uses a typedef from B, there is no requirement
that a server that implements all protocol accessible nodes in A also
implements all protocol accessible nodes in B.

In the rest of the description, "implement a module" means "implements
all protocol accessible nodes from a module".

Some constructs imply a run-time dependency.  If module A augments
module B, there is a run-time dependency, meaning that a server that
implements A must also implement B (modulo features).  A leafref also
imply a run-time dependency.  (must and when expressions do not)

So, the idea is this:

  If a server implements a module A that imports B, and A augments B
  or has a leafref to a node in B, the server MUST implement *some
  version* of module B that has the nodes A references.  This is
  regardless of A imports B by revision or not.

  If a module A imports B by revision, any typedefs or a groupings
  from B are picked from the specified revision of B, regardless of
  which revision (if any!) of B the server implements.

  If a module A imports B without revision, and A uses typedefs and/or
  groupings from B, the server MUST pick and advertise one revision of
  B that has the used typedefs and groupings.  This requires a way to
  advertise a module with "conformance=no-protocol-accessible-nodes",
  so that the server can advertise the module w/o implementing its
  protocol accessible nodes.

All this taken together means that by looking at the advertised
modules and their import revisions, a client knows the exact types
and groupings used everywhere.

A module writer can choose to import by revision (thus locking down
the typedefs and groupings used), or import without revision (allowing
future updated typedefs and groupings to be incorporated).

Repeating the example in P1 from
draft-bjorklund-yang-conformance-problem-01:

  module mod-a {
    ...
    revision 2001-01-01;        // initial version
    typedef foo {
      type enumeration {
        enum q;
      }
    }
    grouping bar {
      leaf x { type string; }
    }
  }

  module mod-a {
    ...
    revision 2002-01-01;        // new version
    typedef foo {
      type enumeration {
        enum q;
        enum w;                 // enum added
      }
    }
    grouping bar {
      leaf x { type string; }
      leaf y { type string; }   // grouping extended
    }
  }

  module mod-b {
    ...
    revision 2001-01-01;
    import mod-a {
      prefix a;
      revision 2001-01-01;      // uses initial vsn of mod-a
    }
    leaf b {
      type a:foo;
    }
    container b2 {
      uses a:bar;
    }
  }

  module mod-c {
    ...
    revision 2002-01-01;
    import mod-a {
      prefix a;
      revision 2002-01-01;      // uses new vsn of mod-a
    }
    leaf c {
      type a:foo;
    }
    container c2 {
      uses a:bar;
    }
  }

  module mod-d {
    ...
    revision 2002-01-01;
    import mod-a                // uses unknown vsn of mod-a
      prefix a;
    }
    leaf d {
      type a:foo;
    }
    container d2 {
      uses a:bar;
    }
  }

A server might advertise:

  mod-a 2002-01-01
  mod-b 2001-01-01 // uses mod-a 2001-01-01, since import by revision
  mod-c 2002-01-01 // uses mod-a 2002-01-01, since import by revision
  mod-d 2002-01-01 // uses mod-a 2002-01-01, since it is advertised

Another server might advertise:

  mod-a 2001-01-01 conformance=no-protocol-accessible-nodes
  mod-c 2002-01-01 // uses mod-a 2002-01-01, since import by revision
  mod-d 2002-01-01 // uses mod-a 2001-01-01, since it is advertised

Note that it is also ok to advertise:

  mod-b 2001-01-01 // uses mod-a 2001-01-01, since import by revision
  mod-c 2002-01-01 // uses mod-a 2002-01-01, since import by revision

I.e., if import-by revision is used everywhere, a server may not
advertise the imported module.  Assume this was not the case.  Then
a server might advertise the following, which is unclear:

  mod-a 2001-01-01 conformance=no-protocol-accessible-nodes
  mod-a 2002-01-01 conformance=no-protocol-accessible-nodes
  mod-b 2001-01-01 // uses mod-a 2001-01-01, since import by revision
  mod-c 2002-01-01 // uses mod-a 2002-01-01, since import by revision
  mod-d 2002-01-01 // which version of a is used???

This solution solves the conformance drift problem (P1 in
draft-bjorklund-yang-conformance-problem-01) and conformance ambiguity
(P2) by having clear rules what import with and without revision
means.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Sat, 28 Mar 2015 11:21:14 -0700

Hi,

Some concerns:

1) the specific revisions of implemented protocol accessible objects
come from the protocol-specific advertisement mechanisms.
I would prefer that all YANG conformance requirements be
derivable from a single solution, based on the ietf-yang-library.
We are headed that way with YANG 1.1 for NETCONF.
RESTCONF and CoMI will both use the library exclusively to
announce conformance.

2) Off-line validation mechanisms that validate protocol messages
need to figure out the library contents somehow.  We should
not have to force all readers and writers to deal with import-by-revision
everywhere.  Instead, tool developers need to write smarter code.

3) This is complicated, but the simple solution of ad-hoc cut-and-pasting
of definitions was a disaster.  As Juergen pointed out to me yesterday,
this would lead to a situation where we had a nice clean ietf-yang-types,
but there would be copies of its various typedefs scattered over all the other
modules. Repeat for every module and the result is even much worse.

4) Now it will be OK to extend typedefs and groupings, but
the this leads to a scenario where essentially every single
definition has its own revision.  IMO it is very confusing
that modules base, A, and B are all using different revisions
of the "foo" typedef.

module T {
    ...
   revision 2013-01-01;
    typedef foo {
        type int32 { range "1..10"; }
     }
}

module T {
   ...
   revision 2014-01-01;
    typedef foo {
        type int32 { range "1..20"; }
     }
}

module T {
   ...
   revision 2015-01-01;
    typedef foo {
        type int32 { range "1..30"; }
     }
}

module base {
   ...
   import T {
        prefix T;
        revision-date 2015-01-01;
   }

   list some-list {
       key i;
       leaf i { type int32; }
       leaf foo-leaf { type T:foo; }
   }
}

module A {
   ...
   import T {
        prefix T;
        revision-date 2013-01-01;
   }
   import base { prefix b; }

    augment base:/some-list {
         leaf my-foo-leaf { type T:foo; }
    }
    list my-list {
       must "a >= b";
       key a;
       leaf a { type T:foo; }
       leaf b {
          type leafref { path /b:some-list/b:foo-leaf; }
       }
    }
}

module B {
   ...
   import T {
        prefix T;
        revision-date 2014-01-01;
   }
   import base { prefix b; }

    augment base:/some-list {
         leaf another-foo-leaf { type T:foo; }
    }
}

Andy

On Sat, Mar 28, 2015 at 8:08 AM, Martin Bjorklund <[email-masked]> wrote:
> _______________________________________________
> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod

---

From: Martin Bjorklund <[email-masked]>
Date: Sun, 29 Mar 2015 09:43:43 +0200 (CEST)

> RESTCONF and CoMI will both use the library exclusively to

Agree.  The ietf-yang-library text is already in 6020bis-04, so I
think this concern is addressed.

> everywhere.  Instead, tool developers need to write smarter code.

I am not sure this is a concern with the proposal?  Or did you mean
that such off-line validation would be simpler if we didn't have
import-by revision at all?

> modules. Repeat for every module and the result is even much worse.

Ok.

> that modules base, A, and B are all using different revisions

Agreed.  But as you noted in 3 the alternative is also pretty bad.

I think that data model designers should be very careful about using
import-by-revision, and at the other end, be very careful when
revising typedefs and groupings.

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Sun, 29 Mar 2015 13:08:59 +0200

Hi,

I like it, comments inline.

> On 28 Mar 2015, at 16:08, Martin Bjorklund <[email-masked]> wrote:
> 
> Hi,
> 
> all protocol accessible nodes from a module”.

I think it is more important to know what modules a server *advertises* (via hello or yang-library) because modules like ietf-inet-types define no protocol-accessible nodes, and yet they will have to be advertised.

Maybe we should instead introduce a term like “data model specification” because sometimes we deal with data models off line, outside the server-client context.

> imply a run-time dependency.  (must and when expressions do not)

How about identities? I think they should also belong to this category, i.e. the set of permitted values is determined from the modules (and revisions) a server advertises rather than from the imported revisions. 

>  which revision (if any!) of B the server implements.

… advertises.

>  protocol accessible nodes.

OK, I think this is both more flexible and more reliable than my previous proposal.

A related question is what to do with includes. Now that ietf-yang-library (as opposed to hello) allows for specifying submodules and their revisions, I think revision-date in includes should be always ignored.

Lada

--
Ladislav Lhotka, CZ.NIC Labs
PGP Key ID: E74E8C0C

---

From: Ladislav Lhotka <[email-masked]>
Date: Sun, 29 Mar 2015 14:50:10 +0200

> On 29 Mar 2015, at 09:43, Martin Bjorklund <[email-masked]> wrote:
> 

+1

> import-by revision at all?

I think it is about the “data model specification” as I wrote in my previous mail. For example, in pyang we would need to figure out how to pass the "conformance=no-protocol-accessible-nodes” flag on the command line. Actually, a better option might be to allow for passing this information in the ietf-yang-library format, perhaps replacing the --hello option.

>> modules. Repeat for every module and the result is even much worse.

On the other hand, Y45-02 would move all the mess to the modules that are designed for being imported. If module *readers* have the highest priority, we are actually obliged to keep modules like ietf-yang-types clean, lean and mean. 

>> definition has its own revision.  IMO it is very confusing

I don’t think so, it would be the case only if we introduced something like

uses foo {
    revision-date 2015-04-01;
}

> revising typedefs and groupings.

Typedefs and groupings defined and used in the same module have their revision effectively fixed. If one wants to put such definitions to a separate module, e.g. to increase readability, I think an import-by-revision might be fully appropriate. Open-ended schemas needn’t be everybody’s preference.

Also, for enumeration typedefs that are based on IANA or ISO registries, regular revisions are pretty much necessary.

So I don’t think import-by-revision and updated definitions have to be labelled as obnoxious.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Sun, 29 Mar 2015 06:38:02 -0700

> think this concern is addressed.

OK --  this needs to be followed as a general solution.
The library still needs the full inventory of YANG modules and
submodules so they can be pre-fetched (via the 'schema' URI
for each YANG file)

> import-by revision at all?

Just agreeing with you -- and making it clear that import-by-revision
SHOULD NOT be used just to make it easier for an off-line validation
tool to pick YANG revisions.

> Agreed.  But as you noted in 3 the alternative is also pretty bad.

Yes but the original solution of forbidding typedef and groupings
to be expanded (but not forbid corrections) is a much better solution.
This leads to maintenance being localized to the source of the definitions,
which is the correct design (same as yours).  But it also leads a
massive increase in the probability that programmer or operator errors
will be introduced because modules are just too hard to read.

> revising typedefs and groupings.

IMO the YANG guidelines MUST forbid the expansion of typedefs
and groupings in RFC published modules.  (Note I-D to I-D changes do
not count -- only RFC to RFC changes actually count).

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Sun, 29 Mar 2015 16:23:09 +0200 (CEST)

Ladislav Lhotka <[email-masked]> wrote:
> I think it is more important to know what modules a server
> *advertises* (via hello or yang-library) because modules like
> ietf-inet-types define no protocol-accessible nodes, and yet they will
> have to be advertised.

I think the two terms need to be separated.  If a server implements
the protocol-accessible nodes the module must be advertised
"normally".  If a server implements typedefs/groupings the module MAY
need to be advertised, and in that case it is advertised with
conformance=no-protocol-accessible-nodes.

> Maybe we should instead introduce a term like “data model
> specification” because sometimes we deal with data models off line,
> outside the server-client context.
> 
> > 
> How about identities? I think they should also belong to this
> category, i.e. the set of permitted values is determined from the
> modules (and revisions) a server advertises rather than from the
> imported revisions.

Well, you are right that the set of supported identities are
determined from the advertised modules, but this has nothing to do
with imports.

> >  which revision (if any!) of B the server implements.

No I meant implements (i.e., implements protocol-accessible nodes,
which means that it is also advertised).

> A related question is what to do with includes. Now that
> ietf-yang-library (as opposed to hello) allows for specifying
> submodules and their revisions, I think revision-date in includes
> should be always ignored.

I would be ok with *requiring* include-by revision.  If include
without revision is used, then ietf-yang-library is needed.

Anyway, I don't think the revision should be ignored, that would just
be confusing.  What problem does it solve?  Why should it be ok for a
server to include some other revision than the specified one?

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Sun, 29 Mar 2015 16:33:04 +0200

> On 29 Mar 2015, at 15:38, Andy Bierman <[email-masked]> wrote:
> 

I strongly disagree. Especially at this stage when many things are under development we need to remain flexible. Keeping modules in I-D stage for extended periods is not a solution either.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Sun, 29 Mar 2015 07:40:49 -0700

> conformance=no-protocol-accessible-nodes.

Yes -- there are two completely different uses of the YANG library:

1) reconstruction of the server YANG file set
    -- all modules, submodules and their revisions
        -- includes all YANG files, even meta-data like ietf-restconf or
            deviations modules
   -- <get-schema> or GET  /path/to/schema is required
   -- caching mechanisms are required
   -- support for shared YANG library servers is required for CoMI

2) determination of the server YANG conformance claims
    -- all modules, submodules and their revisions
    -- all enabled features
    -- all deviations

Andy

---

From: Andy Bierman <[email-masked]>
Date: Sun, 29 Mar 2015 07:45:19 -0700

....
> I strongly disagree. Especially at this stage when many things are under development we need to remain flexible. Keeping modules in I-D stage for extended periods is not a solution either.

I don't think we have any examples at all of any IETF RFC YANG modules that
are churning or are incapable of being updated in a responsible way.
The ietf-routing module does not count.  It is still an I-D.
The MIB Doctors have been able to oversee SMIv2 usage for a couple decades.
I think the YANG Doctors will be able to do the same for YANG.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Sun, 29 Mar 2015 17:01:54 +0200

> On 29 Mar 2015, at 16:45, Andy Bierman <[email-masked]> wrote:
> 
> ....

Because we don’t have many such modules in the first place, and those few that exist haven’t been heavily tested in many implementations so far. For example, I bet ietf-system will need to be changed before long. 

> I think the YANG Doctors will be able to do the same for YANG.

So why all the fuss with YANG, why aren’t we using SMIv2?

Lada

---

From: Andy Bierman <[email-masked]>
Date: Sun, 29 Mar 2015 08:36:16 -0700

> Because we don’t have many such modules in the first place, and those few that exist haven’t been heavily tested in many implementations so far. For example, I bet ietf-system will need to be chang…

Changing is fine. Adding new functionality in a way that does not break
the existing functionality is fine.  Those of us who have to deal with the
tech-support requests for a living understand why updating APIs in
a way that breaks existing functionality is a bad idea.

> So why all the fuss with YANG, why aren’t we using SMIv2?

Because YANG is way better, just like the stuff 20 years from now
will be way better than YANG.

It would be unwise to ignore the experience gained from SNMP evolution
and deployment.  NETCONF/YANG has leap-frogged SNMP/SMIv2
in many ways, but not all.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Sun, 29 Mar 2015 18:20:19 +0200

> On 29 Mar 2015, at 17:36, Andy Bierman <[email-masked]> wrote:
> 

Regarding typedefs and groupings, Y45-02 doesn’t leave much space for changes or additions. 

> a way that breaks existing functionality is a bad idea.

If you start a software project and fix all APIs in version 0.1, then you usually don’t get very far. And I’d argue the YANG module ecosystem is in version 0.001.

> will be way better than YANG.

I’d hope that people in 20 years will be using YANG 3.0 and modules with an ancestry line to the stuff we are doing now. But if YANG and its modules aren’t sufficiently adaptable, then at some moment it will become necessary to abandon it and start something completely new.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Sun, 29 Mar 2015 09:28:23 -0700

> Regarding typedefs and groupings, Y45-02 doesn’t leave much space for changes or additions.

Disagree...
First bullet of Y42-02
    Typedefs must not change their value space or semantics.
    If this is needed, a new typedef needs to be created.
    This does not affect the ability to make bug fixes.

A clarification is needed:

    Any type of sub-statement change, even extending the value range
    can be considered within a bug-fix.  E.g., a range can be extended if the
    original range was wrong. Enums can be added if they were left out by
    mistake, not intent.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Sun, 29 Mar 2015 18:43:17 +0200

> On 29 Mar 2015, at 18:28, Andy Bierman <[email-masked]> wrote:
> 

IMO registries that do not need *distributed* extensibility, such as country codes, should be implemented as enumerations that can change in subsequent revisions. Identities have their share of problems, too.

Lada

---

From: Martin Bjorklund <[email-masked]>
Date: Sun, 29 Mar 2015 19:53:51 +0200 (CEST)

> I think the YANG Doctors will be able to do the same for YANG.

Agreed.  And for this reason I don't think the IETF guidelines should
have MUST forbid extended value space for typedefs and groupings - in
some cases it makese sense.  Two such example are IANAifType and
IANAPhysClass.

/martin

---

From: Martin Bjorklund <[email-masked]>
Date: Sun, 29 Mar 2015 19:56:13 +0200 (CEST)

>     This does not affect the ability to make bug fixes.

NOTE: this new proposal is NOT Y45-02; the idea with the new proposal
is to allow value space expansion.

/martin

---

From: "Thomas D. Nadeau" <[email-masked]>
Date: Mon, 30 Mar 2015 10:10:43 -0400

> On Mar 29, 2015:10:45 AM, at 10:45 AM, Andy Bierman <[email-masked]> wrote:
> 
> ....

While perhaps true for the moment, I heard of a number that might soon. If you consider the open config draft and work others are doing, there will be cases.  Worse will be cases of existing RFCs that need to be updated because other modules depend on them.  This will make for quite a "house of cards" scenario unless we are careful.

	--Tom

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 30 Mar 2015 23:39:59 +0200

>     revision-date 2015-04-01;

So how is this any different than defining

    grouping foo-2015-04-01 {
        // ...
    }

and then later expand things using this:

    uses foo-2015-04-01;

The main simplification of versioning groupings and typedefs by giving
them different names is that I do not have to import multiple versions
of the same module in order to resolve all things used properly.

Note, I am not suggesting to tag definition with version numbers
because not all definitions change when a module is revised. Authors
of reusable stuff should be able to distinguish between editorial and
semantic changes. Users of reusable stuff should be safe from never
inheriting changes as a side effect of an updated import. Moving to a
new semantically different version must require explicit action. The
counter proposal that module maintainers need to check everything they
depend on whenever publishing a new model is a broken idea if you ask
me. I am not sure we really make progress.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1 | 28759 Bremen | Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 30 Mar 2015 23:46:46 +0200

Martin,

if I understand things correctly, your proposal still requires that I
update all definitions to the one I import from. If a grouping/typedef
got expanded, I can't use the original and the expanded version in the
same module, right? This may not be workable with bigger modules.

Here is a scenario. Lets take something like yang-types. In a future
revision, a typedef 'duration' might be added. At the same time, some
other definition gets expanded. If I now want to use 'duration', I
have to update all my usages of the expanded typedef (or worse copy
the old one into my module)?

/js

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 31 Mar 2015 00:20:57 +0200

> of the same module in order to resolve all things used properly.

In other words, if we want to explore this option, then we have
to treat

module m {
  revision 2015-04-01;

  typedef t {}
  grouping g {}
}

module m {
  revision 2015-04-02;

  typedef t {}
  grouping g {}
}

as defining t@2015-04-01 and g@2015-04-01. Modules simply resolve
the shorthand name to a versioned name:

module x {
  import m { revision 2015-04-01; }

  leaf l { type t; }   // resolves to t@2015-04-01
  uses g;              // resolves to g@2015-04-01
}

module y {
  import m;

  leaf l { type t; }   // resolves to t@2015-04-?? (depends on advertisements)
  uses g;	       // resolves to g@2015-04-?? (depends on advertisements)
}

module z {
  import m;

  // this requires to load two versions of m...
  leaf l { type t@2015-04-01; }  // use explicit version
  uses g@2015-04-02;             // use explicit version
}

Module z is perhaps ugly but completely future proof since the
definitions used are always clear. This requires to be able to load
multiple versions of m. Otherwise, this is functionally equivalent
with explicit version numbering.

Module x is reasonably future proof but requires that the module
maintainer is careful with import revision updates. It is necessary to
examine all imported symbols and to check that the new version is what
is really needed/wanted.

Module y is short but ambiguous without access to external information
once m gets updated. Until m gets updated in an incompatible way,
module y is however OK (and this is where we are with most of our
modules).

Note that this should then also be allowed:

module v {
  import m { revision 2015-04-01; }

  leaf l { type t; }   // resolves to t@2015-04-01
  uses g@2015-04-02;   // resolves to g@2015-04-02
}

This may look odd but the revision date in the import really just
sets a default.

Personally, I do not like to rely on external information to determine
what a module means. Anyway, this all boils down to the question
whether we want to require that tools can deal with multiple revisions
of the same module in order resolve definitions. This is the price to
pay for anything that does a bit more than the simple rule "whenever
you make a semantic change, give the definition a new name".

/js

---

From: Andy Bierman <[email-masked]>
Date: Mon, 30 Mar 2015 17:47:25 -0700

Hi,

This is getting way too complicated for real deployment.
All the solutions seem way more complicated than just not
changing the intended value set of a released typedef or grouping.

    typedef acme-type { ... }
    typedef acme-type.2 { ... }
    typedef acme-knob.3 { ... }

    grouping acme-grp { ... }

    grouping acme-grp.2 {
         uses acme-grp;
         // add new nodes here
    }
    grouping acme-grp.3 {
         uses acme-grp.2;
         // add new nodes here
    }

IMO it is not a burden to put the version in the name.
Just moving the version to a different encoding is not really solving
the problem.
It just makes it much worse.  The current solution allows each subscriber
to the typedef to decide when to change to a new version.  The publisher
cannot decide for everybody.  Cut-and-paste into N private versions
violates coupling and cohesion principles so it cannot be considered.

This is the least painful solution that works for everything.

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 31 Mar 2015 09:20:47 +0200

Hi,

even this type of solution is kind of broken since it does not provide
the very import information which versions and compatible and which are
semantically not compatible.

If I implement a tool that supports say ietf-inet-types:ip-address and
I ship this tool based on a certain revision of this definition, I
can't be sure this tool does the correct thing with a future revision
of the ietf-inet-tyeps module. In other words, I constantly need to
update tools to include information which revisions are semantically
equivalent. In other words, while I have lots of versioned definitions,
I do miss the most important bit that tells me which ones are
semantically equivalent and which ones are not.

Bottom line: Assigning a new name when a definition is updated in an
incompatible way remains the simplest and most predictable of all
solutions we came up with.

/js

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 31 Mar 2015 09:27:54 +0200

> changing the intended value set of a released typedef or grouping.

[...]

> This is the least painful solution that works for everything.

I tend to agree.

The only variation to consider is to be less strict for enums and
bits, that is allow additions since enums and bits are often used for
(controlled) extensions. In other words, to learn the actual value set
supported by an implementation, one has to ask the implementation
(similar to identities). (This is what SMIv2 allowed, you can add
named numbers to BITS and enumerated INTEGER types, but no other
changes to types are allowed.)

/js

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 31 Mar 2015 09:57:43 +0200 (CEST)

Juergen Schoenwaelder <[email-masked]> wrote:
> same module, right?

Correct.

> This may not be workable with bigger modules.
> 

Correct.

You have to be careful if you expand an existing typedef/grouping.  I
think this is ok.

/martin

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 31 Mar 2015 10:04:36 +0200 (CEST)

> changes to types are allowed.)

Why are enums and bits special in this regard?  Why are "controlled
extensions" easier to handle than an expanded string pattern?

If we can get this to work for enums and bits, the same mechanism
should imo work for the other types as well.  Either the exact type is
important or the client or it is not.

/martin

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 31 Mar 2015 10:29:04 +0200

> important or the client or it is not.

Yes, we make different promises:

- If my tool uses yang:date-and-time (e.g., it knows how parse it, how
  to store it efficiently internally, how to produce the canonical
  representation), I can rely on it to not change. This is reasonable
  and goodness.

- If my data model uses an identity, we make a different promise,
  namely that the set of identities is open ended and I will need
  runtime information to find out which set of identities can actually
  be used for a certain leaf in the data model. This is reasonable and
  goodness.

I argue that bits and enumerations are often used to model something
that is by design extensible in protocols. If you agree with that,
allowing this extensibility in data models is reasonable and goodness.

What are the alternatives? Translate all enums into identities for the
sake of being extensible? What do we do with bits then?

I understand the fundamental argument that either all types should be
strict or flexible. But I do not think either extreme leads to
something that is practically useful.

/js

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 31 Mar 2015 13:23:00 +0200 (CEST)

>   representation), I can rely on it to not change. This is reasonable

Agreed.  One would think that this typedef is not changed easily.

But OTOH, if we have this:

  typedef timeout {
    type uint32 {
      range "0 | 10..180";
    }
  }

and then in the next version we realize that this was too short so we
do:

  typedef timeout {
    type uint32 {
      range "0 | 10..3600";
    }
  }

this is probably easier to deal with in the code than the case where
additional enums are defined.  For example, how do you translate
between JSON and CoMI if you get an enum you don't know the definition
for?

> allowing this extensibility in data models is reasonable and goodness.

Right.  My new proposal allows this of course.

> something that is practically useful.

I agree that all strict has its problems, but I do not understand how
making enums/bit flexible is easier to deal with than making all types
flexible.

/martin

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 31 Mar 2015 13:54:55 +0200

> making enums/bit flexible is easier to deal with than making all types

So you are saying typedef drift is not an issue anymore? Or are you
saying typedef drift is an issue for all types except identityref and
you just do not like enums and bits be treated differently?

/js

---

From: Jan Lindblad <[email-masked]>
Date: Tue, 31 Mar 2015 14:16:00 +0200

> On 31 mar 2015, at 13:23, Martin Bjorklund <[email-masked]> wrote:
...
> additional enums are defined.

Another mechanism that’s been used quite often for “implementation constants”, like max values that vary between implementations or over time, is to have a section of the model containing such constants as config true values, but with access control rules effectively preventing any operator from ever changing them. Like this:

module some-application {
  container somethings {
    must “count(something) < /somethings/constants/max-somethings” {
      error-message “Too much of something”;
    }
    list something {
      key name;
      leaf name { type string; }
      leaf timeout {
        type uint32;
        must “current() < /somethings/constants/max-timeout” {
          error-message “Timeout too long”;
        }
      }
    }
    container constants {
      // Access control rules prevent operators from changing these
      leaf max-somethings {
        type uint32;
        mandatory true;
      }
      leaf max-timeout {
        type uint32;
        mandatory true;
      }
    }
  }
}

With this approach, IETF could standardize the model, yet allow implementations to pick reasonable constants for a variety of high- and low-end products, and change them over time.

/jan

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 31 Mar 2015 14:21:06 +0200

>     grouping acme-grp { ... }

In effect, this is really no different from allowing each "uses" and
"type" specify a particular revision, except that readability of modules
with typedefs/groupings will deteriorate in time.

Also, the relationship among different revisions of acme-type can be
inferred only from the similarity in names, which is brittle.  

Lada

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 31 Mar 2015 14:40:02 +0200

> same module, right? This may not be workable with bigger modules.

A workaround is to import different revisions in the main module and a
submodule, or two submodules.

BTW, does RFC 6020 forbid importing different revisions of the same
module using different prefixes? If not, it might solve this issue.

> the old one into my module)?

This is a fair argument, such situations should be avoided. In systems
offering cheap revisions, such as git, it is easily solved by using
fine-grained revisions, essentially one change per
revision. Unfortunately, the IETF process is not of this sort.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Tue, 31 Mar 2015 06:20:59 -0700

> with typedefs/groupings will deteriorate in time.

This does not introduce any new YANG statements that are confusing to read.
I totally disagree that this makes the module unreadable.
I think it makes the changes to typedefs and groupings very easy
to see and understand over time.  These new names will match the
protocol names that are introducing a new version
(eg. ssh-parms, ssh2-parms, http1.1-parms, http2.0-parms)

I find the new uses@214-01-01 to be very unreadable and very hard
to follow.  SMIv2 does not need this insane complexity, which tells
me the YANG crowd is really on the wrong track here.

> inferred only from the similarity in names, which is brittle.

I disagree -- having new names means that the old and new ranges
can still be used together in the same data model quite easilty
(e.g., SSH1 and SSH2 bother supported leafs of both types still exist).

Andy

---

From: Andy Bierman <[email-masked]>
Date: Tue, 31 Mar 2015 06:53:37 -0700

.....
> with explicit version numbering.

I am curious -- if import-by-revision is such a great way to manage
external dependencies, then why aren't there any programming languages
that work this way?

Can you name a single programming language in the entire world
that requires the programmer to remember the history of every
single function, class, or typedef, and allow multiple revisions
to be accessed at once?

IMO import-by-revision, especially uses-by-revision, is way too complicated,
and not proven over time to be the way dependencies are managed.
It is not proven at all, with good reason.

....

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 31 Mar 2015 16:06:36 +0200

> On 31 Mar 2015, at 15:53, Andy Bierman <[email-masked]> wrote:
> 
> .....
>> module z {
>>  import m;
>> 

Because you always use the locally-installed package version, and package managers take care of installing the dependencies - pip for Python, npm for JavaScript, cabal for Haskell, …

It is not that such systems are free of the problems we are facing, see

http://en.wikipedia.org/wiki/Dependency_hell

Lada

---

From: Andy Bierman <[email-masked]>
Date: Tue, 31 Mar 2015 07:14:07 -0700

> you just do not like enums and bits be treated differently?

If an existing client is hard coded to understand a certain version of
a leaf value-set
then extending it (for any type) will probably not be handled by the client.

I came to the conclusion (in the now expired YANG conformance draft)
that typedef and grouping drift was OK, as long as the server implemented
the new semantics where ever they were used.

This works because an old client thinking it is using the
old hard-coded typedef will still work, and a new client or
data-driven old client will try to use the new typedef.

The WG seems to want it both ways wrt/ extending typedefs and groupings.
If the change is so valuable that every usage from this date forward
needs to be the new version, then why allow a server to implement
the old version and the new version selectively (and concurrently)?

Extending typedefs and groupings actually works, and implementing
only 1 revision actually works, but only if the server implements
the advertised definitions consistently.

Andy

---

From: Phil Shafer <[email-masked]>
Date: Tue, 31 Mar 2015 15:31:11 -0400

>external dependencies, then why aren't there any programming languages

Most systems have to construct a versioning mechanism after the
fact, via installation tools or library systems.   There are
approximately one zillion means of doing this, mainly consed up
from the depths of the installation system programmers' minds.  To
my mind, the sheer volume of version-based installation tools (and
the number of times they fail) infers that no one's got it right
yet.

In YANG we're attempting to address this internally.  Doesn't mean
we're wrong, just that we know we can't depend on an external
technology to do this for us in a portable, interoperable way.
Lacking an "install" phase, we can't really count on apt-get, yum,
pip, gem, ninite, yast, pkg, brew, dpkg, port, fink, pacman, ipkg,
etc [1] to do the work for us.

Thanks,
 Phil

[1] Okay, that's off the top of my head.  I stopped before I reached
to wikipedia:
  http://en.wikipedia.org/wiki/List_of_software_package_management_systems

[2] Yes, there's really a "petget", the installer for "Puppy Linux".
Gag me with a chew toy!

[3] No, I really have no clue how to spell the past tense of "cons".

---

From: Phil Shafer <[email-masked]>
Date: Tue, 31 Mar 2015 15:41:04 -0400

>(eg. ssh-parms, ssh2-parms, http1.1-parms, http2.0-parms)

I don't think things are nearly that simple.  We're at version 17
of draft-ietf-netmod-routing-cfg and it's not even finished.  That's
up to 17 versions of each grouping/typedef threading together in
the style Lada gave, which means the "last call" readers don't get
the nice concise readable document we've been promising them.

It won't take long for authors to realise that non-global grouping
don't suffer this fate and to shift their BCP to not using our
global groupings/typedefs facilities.

That said, a module is publishing an API, a binding contract that
shouldn't randomly change.  If we accept that groupings and typedefs
are part of that API, then importers should able to depend on their
not changing.

On the other hand, if there groupings aren't flexible, what happens
when you have a bug?  Can you not change it because other users
might be relying on the broken behaviour?

It's really a whole rainbox of ugly, eh?

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Tue, 31 Mar 2015 15:45:40 -0400

>module using different prefixes? If not, it might solve this issue.

Sure but if that's the solution, we're in big trouble ;^)

One of the things I learned looking back at the JUNOS DDL files
while we were working on YANG was that whatever facilities you
provide, people will use them in ways you never intended; often
ways you never conceived of, and sometimes ways that turn your
stomach.  But the BCP for how to handle problems like this shouldn't
involve multiple imports.

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Tue, 31 Mar 2015 15:50:44 -0400

Jan Lindblad writes:
>Another mechanism that's been used quite often for 'implementation
>constants', like max values that vary between implementations or
>over time, is to have a section of the model containing such constants
>as config true values, but with access control rules effectively
>preventing any operator from ever changing them. Like this:

Relying on immutable configuration feels like a hack, not a BCP.

If we want box-specific constants, we should call them out as
a first-class construct.   I can certainly see their value,
but they aren't config and shouldn't masquerade as such.

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Tue, 31 Mar 2015 15:54:06 -0400

>changes to types are allowed.)

I'm confused; how does one ask the implementation for the list
of supported identities?

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Tue, 31 Mar 2015 12:56:23 -0700

> of draft-ietf-netmod-routing-cfg and it's not even finished.  That's

The change rules do not apply to Internet Drafts.  Never did.
They only apply to published RFCs.  So if you can find
17 versions of any MIB or YANG definition across 17 RFCs,
then let me know.

> global groupings/typedefs facilities.

The new names are not needed in Internet Drafts.
The change rules do not apply.  People break the change rules
in sec. 10 of RFC 6020 all the time during development
of a YANG module.  We only care about published modules.

> might be relying on the broken behaviour?

As I said before, the rigid change rules do not apply to bugs.
If the intent from the start was that the foo-range be 1..20,
but it got published as 1..10 by mistake, then changing it to 1..20
without a name change is OK because the intent was 1 .. 20
from the start.  We have done this several times with patterns
in IETF typedefs.

> It's really a whole rainbox of ugly, eh?

No, because you are wrong about how the change rules work.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 31 Mar 2015 22:52:17 +0200 (CEST)

> you just do not like enums and bits be treated differently?

The latter, but it is not so much that I don't like it, rather that I
do not understand why it can work for enums and bits, but not for
other types.

Also, I think that with the proposed solution, there is no ambiguity
anymore, even if import without revision is used.

/martin