Subject: [netmod] YANG conformance-stmt (corrected)

From: Andy Bierman <[email-masked]>
Date: Thu, 01 May 2008 22:02:35 -0700

Hi,

Here is a first stab at a conformance example,
even simpler than SMIv2 (which some people may or may not like).

   - only accessible objects can be listed in the conformance section.
     (This includes nodes, rpcs, and notifications)

   - only nodes in the module namespace can be listed in the conformance
     section.

   - there is no conformance info for a typedef or a grouping or augment
     (that I am suggesting).  Instead the expanded target nodes are
     specified in the module with the uses.

   - top-level augment from module A to a subtree in module B must be
     documented in the conformance section for module A

--------------------------

Phil's example:

module A {
     revision 2008-01-01 { ... }
     grouping one {
         leaf foo { ... }
         leaf goo { ... }
     }

     // no conformance section needed

}

// revision 2008-04-01 left out for brevity

module A {
     revision 2008-05-01 { ... }
     grouping one {
         leaf foo { ... }
         leaf goo { ... }
         leaf super-cool { ... }
         leaf way-super-cool { ... }
     }

     capability way-super-cool-stuff {
       description
         "Implements the Acme way super-cool features, if supported.";
     }

     // no conformance section needed
}

module B {
     import A {
         prefix A;
     }

     revision 2008-01-01 {
       description "Initial version.";
     }

     container zed {
         uses A:one;
     }

     conformance 2008-01-01 {
       requires "/zed/foo";
       requires "/zed/goo";
     }

}

module B {
     import A {
         prefix A;
     }

     revision 2008-05-01 {
       description "Added way-super-cool leaf to container zed.";
     }

     revision 2008-01-01 {
       description "Initial version.";
     }

     container zed {
         uses A:one;
     }

     conformance 2008-05-01 {
       requires "/zed/foo";
       requires "/zed/goo";
       requires "/zed/way-super-cool" {
         description "acme super cool stuff capability needed";
         when-capability "A:way-super-cool-stuff";
       }
     }

     conformance 2008-01-01 {
       requires "/zed/foo";
       requires "/zed/goo";
     }
}

module C {
     import A {
          prefix A;
     }
     import B { ... }   // do not see how this import is used

     revision 2008-04-01 {
       description "Initial version.";
     }

     container yancy {
         uses A:one;
     }

     conformance 2008-04-01 {
       requires "/yancy/foo";
       requires "/yancy/goo";
       requires "/yancy/super-cool";
     }
}

-------------------------------

In this example, module B wants the newest way-super-cool leaf
but not the super-cool leaf (could be deprecated by the time the
implementation for B is updated).

Module C was updated when super-cool was released, and has not
yet been updated to way-super-cool (may never be updated).

There should be a 1:1 correspondence between
conformance-stmt and revision-stmt.

Andy

_______________________________________________
netmod mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/netmod

---

From: Phil Shafer <[email-masked]>
Date: Fri, 02 May 2008 08:03:32 -0400

>         when-capability "A:way-super-cool-stuff";

Doesn't this turn into a requirement that each module explicitly
list the contents it's importing from another module?  It's worse
than SNMP, since I have to list complete hierarchies of stuff I
want to import;  if /zed/foo is a deep hiearchy, do I need to list
every container, list, and leaf that it contains?  if /zed/goo is
an enumeration, do I need to list all current values so when one
is added, I'm fine?

IMHO anything that requires me to repeat the contents of another
module is a non-starter.  If the other module is published, any
reader (human, device, application) should be able to know the
contents of it without me repeating it when I import it.   Revision-
specific imports are a much cleaner, simpler way to handle this.

Thanks,
 Phil

---

From: "David Harrington" <[email-masked]>
Date: Fri, 2 May 2008 08:50:41 -0400

Hi Phil,

I think the conformance statement allows the importing module to
specify that conformance to it requires version-specific conformance
to its dependencies, and all you need to list in the importing module
is the named conformance statement from the exporting module.

For example. using the snippet below, the conformance statement in the
importing module would need to specify
conformance 2008-09-01 {
	requires module A conformance 2008-05-01
}

It would not need to list the contents of the module A conformance. 

This seems similar to installing a Linux application that has
dependencies on, say, libxml v1.2. You don't need to list all the
variables and APIs supported by v1.2.

dbh

> -----Original Message-----
> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of Phil Shafer
> Sent: Friday, May 02, 2008 8:04 AM
> To: Andy Bierman
> Cc: [email-masked]
> Subject: Re: [netmod] YANG conformance-stmt (corrected)
> 
> Andy Bierman writes:
> >     conformance 2008-05-01 {
> >       requires "/zed/foo";
> >       requires "/zed/goo";

---

From: Andy Bierman <[email-masked]>
Date: Fri, 02 May 2008 07:07:00 -0700

> specific imports are a much cleaner, simpler way to handle this.

You are not repeating the contents of another module.
The uses-stmt causes a foreign grouping to be expanded
in your namespace, so it is in the current module.

IMO any solution that does not allow a module to import *exactly*
what it needs from another module is a non-starter.  Your proposal
requires that everything in the imported module at the time
of the specified version be imported, which may include stuff
added in the interim that is not desired.

An explicit list of nodes is not that painful, especially to
the readers, who count the most.  It is way easier to deal
with than figuring out by hand which exact nodes are part
of the module (after uses and augments are expanded, it
may look very different).  How do I know what was added
in rev 1, rev 2, rev 15?  I just keep 'diffing' files
by hand to figure out which objects are actually exported
in each version?  Or rely on the revision-stmt description,
which is not even required to be present, let alone be accurate.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Fri, 02 May 2008 07:17:25 -0700

> It would not need to list the contents of the module A conformance. 

What if this module needed only some of the objects added
to the 2008-05-01 version of module A?  What if module A
has 2 sets of 'features' (e.g, IPv4 or IPv6 version)?
Are SMIv2-like GROUPs needed instead?

conformance 2008-09-01 {
    requires "module A conformance 2008-05-01 group IPv6-objects";
}

(A more condensed syntax might be better.)

Andy

---

From: Andy Bierman <[email-masked]>
Date: Fri, 02 May 2008 07:50:18 -0700

> specific imports are a much cleaner, simpler way to handle this.

Perhaps we need a precise and agreed-upon definition of a grouping.
Does the module with the grouping or the module with the uses
decide the conformance requirements?  Don't the refinement statements
in the uses module have the potential of changing the conformance,
as well as the semantics of a node?

How does a compiler know what was in version N of a module,
if all it has is the latest version?  Will all implementations
need to maintain all versions of all modules somehow, and
be forced to support multiple versions of a module at runtime?

Andy

---

From: Phil Shafer <[email-masked]>
Date: Fri, 02 May 2008 10:46:55 -0400

>is the named conformance statement from the exporting module.

This sounds different than Andy's example, which listed required
paths from module A in module B.

>	requires module A conformance 2008-05-01

This approach seems similar to listing revisions in import
statements.  When I say:

    import A {
        revision 2008-05-01;
    }

I'm saying that I want the contents of that specific revision of
module A.  Modules are monolithic constructs (in that we don't have
a way of saying "I only implement x, y, and z from module A" (which
I think is goodness)), so by saying I want that revision of module
A, everyone is completely clear on what functionality I'm talking
about.

I guess there are two halves to the problem.  First is defining the
content which we are talking about (which revision-specific imports
solves), and second is defining for any specific device/implementation,
what portion of that content the device implements.  If we allow
SNMP-style implementation conformance statements, applications must
perform the "well it implements x but not y so how do I tell it to
do what I need it to do" logic.  I'm happy to just say that modules
are monoliths; implement it all.

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Fri, 02 May 2008 10:58:25 -0400

>be forced to support multiple versions of a module at runtime?

Your toolset need only know the contents of revisions of modules
that are imported by other modules.  If B imports 2008-01-01 of
module A and C imports 2008-04-01 of module A, your toolset would
need access to those two specific revisions of module A.  Other
revisions would not be needed.

This information would be needed at build time, not runtime.

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Fri, 02 May 2008 08:07:12 -0700

Phil Shafer wrote:
...

> are monoliths; implement it all.

This has never been good enough for SNMP,
and hasn't even been used in NETCONF.  We have capabilities,
which partition the functionality, sometimes down to separate
parameters in an RPC operation, or even separate enum values
within a single RPC paramater (e.g., rollback-on-error).

I would be happy with an architecture where the entire module
is the contract, except when conceptually partitioned with
capabilities.  Then a manager would use the module definition
and the <hello> reply to figure out exactly what an agent had.
This would align precisely with conformance definitions,
given in terms of [module-name, capability-URI] pairs,
without using any version date at all.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Fri, 02 May 2008 08:10:31 -0700

> revisions would not be needed.

So, all implementers and even module readers need to maintain
every version of every module ever used in the system.
This is not a feature.

> This information would be needed at build time, not runtime.

OK - static at build-time, used at runtime.

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Fri, 2 May 2008 17:11:45 +0200

> A, everyone is completely clear on what functionality I'm talking

In SMIv2 land, we do have reusable constructs that are often reduced
when they are used. A good example is the InetAddress version of a
union. And historically, we have produced several MIB modules where
you might implement only certain subsets. I am not sure it would have
been easy to break down all these MIBs in smaller functional units.
Perhaps with YANG's augments this is easier to do but we clearly lack
experience with that approach.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>
_______________________________________________
netmod mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/netmod

---

From: Phil Shafer <[email-masked]>
Date: Fri, 02 May 2008 11:17:59 -0400

>within a single RPC paramater (e.g., rollback-on-error).

With the current draft of YANG, the mapping between capabilities
and modules is 1:1.  We've deferred multiple capabilities to
later work.

>without using any version date at all.

That's what we've got now, so we can both be happy, eh?

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Fri, 02 May 2008 11:23:27 -0400

Andy Bierman writes:
>the readers, who count the most.

Try this for a large, deep grouping.  It's quite painful and
completely uninteresting to the reader, who isn't seeing enough of
the grouping to be useful and must go view the definition anyway.

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Fri, 02 May 2008 11:20:17 -0400

>every version of every module ever used in the system.

Nope, you only need the ones you import.  If your new module imports
an obscure old revision of another module, your readers will happily
complain that they can't find it and that you should be using
something more modern.

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Fri, 02 May 2008 08:49:13 -0700

> That's what we've got now, so we can both be happy, eh?

We need the when-capability statement somewhere.

BTW, this approach is how we extended the protocol with notifications.
We actually used a new namespace, but partitioned it with capabilities.
IMO, this isn't that different than just adding a new capability
to an existing namespace.

But it is not really good enough, because it could lead to
an explosion of capabilities, which would be just as hard
to manage as lots of module versions.  Also, the module
with the objects needs to define the 'when-capability'
partitioning (in the affected objects),
not the module importing the objects. This is backwards.
(Using when-capability in the conformance section of the
new module fixes that.)

Another problem with the "import FOO-MIB, exact version X" approach
is that every time I update FOO-MIB, such as fixing a typo or a bug
in some construct, I have to update and release (perhaps) every
version of FOO-MIB ever written.

I would like the import-by-revision feature a lot better if
the latest version of each module is all that is ever needed
to implement it.

Andy

---

From: "David Harrington" <[email-masked]>
Date: Fri, 2 May 2008 14:04:22 -0400

Hi Phil,

For some technologies, the monolithic approach doesn't work that well.

Andy uses an example in this thread of needing the IPv6 group but not
the IPv4 group. For an IPv6-only router, it doesn't make sense to
require that the IPv4 group be implemented. 

This could of course be handled by writing separate modules for IPv4
and IPv6, but if the functionality is identical except the group for
IP versions, you aren't making it easier for the reader. We would also
have to prescient to know how whether every vendor will implement
future products with support for all the functionality mentioned in a
module, so we knew whioch pieces neede to be broken out into separate
emodules for configuration.

Imagine RFC3805 (the Printer MIB) having to define a separate module
for each optional type of functionality, or requiring that the only
conformant implementations of the management data model support for
all of the optional functionality, even if that functionality is not
available in the device? 

(Juniper must be doing well if they can afford to pay engineers to
implement management for non-existent functionality. good job security
for management standards engineers if vendors agree to do this. ;-)

dbh

> From: Phil Shafer [mailto:[email-masked]] 
> Sent: Friday, May 02, 2008 10:47 AM
> To: David Harrington
> Cc: 'Andy Bierman'; [email-masked]

---

From: "David Harrington" <[email-masked]>
Date: Fri, 2 May 2008 14:10:41 -0400

Hi,
 comments inline. 

> and the <hello> reply to figure out exactly what an agent had.

s/had/claimed to have/
sadly, they are non necessarily the same.
managers still need to work with those agents/servers.

dbh

---

From: "David Harrington" <[email-masked]>
Date: Fri, 2 May 2008 14:20:04 -0400

> to manage as lots of module versions.  

I bet operators will love configuring/troubleshooting access controls
for this matrix of lots of module versions and lots of capabilities.
;-(

dbh

---

From: "David Harrington" <[email-masked]>
Date: Fri, 2 May 2008 14:36:40 -0400

> Nope, you only need the ones you import.  

I think agent implementers only need the ones they implement.

Manager applications and operators (and other module readers) will
need to keep a potentially-large database of versioned modules to
support multiple devices that support different versions.  

This is not a lot different than MIB modules. I believe there are
devices that support the RFC1643 Ether-MIB, and others that support
the RFC3635 MIB module. A device usually supports one or the other,
but an NMS needs to know about both versions if different devices
support different versions. 

SMIv2 set rules about how modules can be updated to prevent
incompatible differences, and we didn't have a whole tree of
dependencies, we only had one layer of IMPORTs. 

dbh

---

From: Andy Bierman <[email-masked]>
Date: Fri, 02 May 2008 11:46:03 -0700

> managers still need to work with those agents/servers.

not a standards problem.
You cannot write "an implementation MUST NOT contain any bugs
or misrepresent the actual NETCONF capabilities in any way"
in a standard.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Fri, 02 May 2008 11:51:48 -0700

> dependencies, we only had one layer of IMPORTs. 

This is a solved problem in SMIv2.
I don't mind the restriction that the importing module
must accept everything in the module for the revision specified.

But SMIv2 has a way to specify multiple conformance levels for
each version of a module, all contained within the latest version
of the module.  I think YANG needs to provide the same feature,
even if it is not implemented anything like SMIv2 MODULE-CONFORMANCE.

Andy

---

From: Phil Shafer <[email-masked]>
Date: Fri, 02 May 2008 15:26:04 -0400

>For some technologies, the monolithic approach doesn't work that well.

I completely agree.  I like the idea of using "when-capability" to
express conditional bits of modules.  I think this is fairly slick
and expressive.

But the last time around this pond, we tabled working on "when-capability"
as future work.  Are we un-tabling it?  Until we do, we have 1:1
and need to be happy about living with it.

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Fri, 02 May 2008 15:40:17 -0400

Andy Bierman writes:
>Also, the module

But "when-capability" _does_ put the conditionality on
the definer, not the importer:

module A {
    capability foo;

    grouping goo {
        leaf one { ... }

        when-capability foo {
            leaf two { ... }
        }
    }
}

module B {
    import A;

    container bee {
        use goo;  // gets bee/two iff :foo
    }
}

Or:

module D {
    container dee {
        leaf d1 { ... }
    }
}

module E {
    capability eee;

    when-capability eee {
        augment D:dee {
            when "d1 > 1500";
            leaf e2 { ... }
        }
    }
}

The point where the node is defined designates if it's conditional
on a capability by using the when-capability statement.

>version of FOO-MIB ever written.

Yes, there's no mechanism to explicitly say "this updates isn't
important enough to rev modules that import this one".  It would
be up to the module owners to make that call.  Or we could add
a mostly-never-used "functionally-equivalent-to" statement so
you can say:

    revision 2008-04-01.1 {
        description "Fixed mispelled 'Juniper'";
        functionally-equivalent-to 2008-04-01;
    }

which will bring it's own boatload of issues.

>the latest version of each module is all that is ever needed

If you always used the latest, you wouldn't need to specific the
version.

Thanks,
 Phil

---

From: "David Harrington" <[email-masked]>
Date: Fri, 2 May 2008 15:59:51 -0400

> or misrepresent the actual NETCONF capabilities in any way"

But we also don't want to make it so expensive to be compliant that
implementers will be motivated to lie about what they support, OR be
motivated to say they do not support something which actually they do
to a significant degree. 

Making compliance an all-or-nothing decision for the whole management
data model, when it is not an all-or-nothing logical compliance
decision for the technology being modeled is not really helpful. 

dbh

---

From: Andy Bierman <[email-masked]>
Date: Fri, 02 May 2008 13:16:45 -0700

> decision for the technology being modeled is not really helpful. 

Agreed -- worst case scenario is there is a capability URI for every
knob (maybe more than 1), and there are all kinds of undocumented
interactions in various permutations of capability-sets.

The conformance-to-group (in SMIv2) has proven both tractable
and politically necessary to get consensus on MIB objects.

In a way, 'capability-stmt' is a replacement for SMIv2 GROUP.
It is important to explicitly state what is in the capability
for a given module version, which capability-stmt does not provide.

We have no real experience with grouping/uses, multiple RFCs,
and updates to the grouping over time.  We need to really understand
how this is supposed to work now, not 5 years from now,
(when we will obviously know what to do ;-)

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Sat, 03 May 2008 23:37:50 +0200 (CEST)

Hi,

> the definer, not the importer:

I agree with Phil.  I think that introducing the two constructs
'capability' and 'when-capability' might solve the conformance
problem.  No need for an additional 'conformance' statement.

> on a capability by using the when-capability statement.

I like this, except that wrapping parts of the statements with a
'with-capability' statement isn't perfect from a readbility pow.  How
about:

   module E {
      capability eee;
    
      augment D:dee {
         when-capability eee;
         when "d1 > 1500";
         leaf e2 { ... }
      }
   }

Then you could also do:

   container foo {
       when-capability aaa;
       when-capability bbb;
       ...
   }

Instead of

  when-capability aaa {
      when-capability bbb {
        container foo { ... }
      }
  }

And also:
 
  capability "...:confirmed-commit" {
      when-capability "...:candidate";
      description "...";
  }

And maybe even:

  typedef ErrorOptionType {
      description "NETCONF 'error-option' Element Content";
      type enumeration { 
          enum stop-on-error;
          enum continue-on-error;
          enum rollback-on-error {
              when-capability "...:rollback-on-error";
          }
      }
      default "stop-on-error";
  }

/martin

---

From: Phil Shafer <[email-masked]>
Date: Sun, 04 May 2008 21:10:57 -0400

>         leaf e2 { ... }

This is fine, and keeps us from having to sweat the context
of the when-capability statement (and have differing sets of
contents).

Thanks,
 Phil

---

From: Wes Hardaker <[email-masked]>
Date: Mon, 05 May 2008 06:33:03 -0700

>>>>> On Fri, 02 May 2008 11:51:48 -0700, Andy Bierman <[email-masked]> said:

AB> I don't mind the restriction that the importing module
AB> must accept everything in the module for the revision specified.

I think the critical things (in my mind) are that:

1) Modules are backwards compatible.  If this isn't the case then either
   we'd allow the tree structure to change in semantics without
   re-wording the leaf names which, IMHO, is a recipe for problems or we
   stop publishing tree components that are "obsolete" (like the
   IF-FORWARD-MIB continuing to contain the ipForwardTable even though
   it's obsolete).  If modules are backwards compatible then you don't
   need to worry about specifying and exact version.

2) You probably do need a minimum conformance level though, especially
   if you want to blindly import a particular subtree within a module
   and not specify each leaf node.  (specifying each leaf node was
   required in SMIv2 and the imports clause became potentially large if
   you needed a lot of pieces from the imported MIB).

The solution to #2 is specifying a method to import portions from the
other module that "is at least greater than or equal to version X".

  import A { 
    at-least-revision 2008-05-01;
  }

Would be a safe way, if modules must be backwards compatible, to reduce
the number of loaded module As on this system to 1 (which had to be of a
revision greater than the maximum specified).

The downside, of course, to never allowing a subtree section to
disappear from a module is module growth.  You could solve that too by
allowing a double import though (A imports something from B but B has
outsourced it (now) to B-archive but the import from A to B will still work).

-- 
Wes Hardaker
Sparta, Inc.

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 05 May 2008 15:56:35 +0200 (CEST)

Hi,

Wes Hardaker <[email-masked]> wrote:
> 1) Modules are backwards compatible.  
[...]
>    If modules are backwards compatible then you don't

This has been our assumption as well.  The problem is 'groupings', and
what constitues a backwards compatible modification in a grouping.

  module a {
     revision 2008-05-01;
     grouping foo {
       leaf y { ... }
     }
  }

  module b {
    revision 2008-05-02;
    import a;
    
    container x {
      uses a:foo;
    }
  }

Now, b has the structure

   <x>
     <y>...</y>
   </x>

If a new leaf 'z' is added to the grouping foo in an updated version of
module a, the structure of b magically becomes:

   <x>
     <y>...</y>
     <z>...</z>
   </x>

even though b hasn't been updated.

So the conclusion is that adding structure to a (exported) grouping is
not allowed.  But this might be too restrictive, and would force
people to encode a version number in the grouping name:

  module a {
     revision 2008-05-08;
     grouping foo {
       leaf y { ... }
     }
     grouping foo2 {
       uses foo;
       leaf z { ... }
     }
  }

Although the versioning rules of YANG are not spelled out yet in the
draft, this is the current situation.  One solution is the versioned
import.

>     at-least-revision 2008-05-01;

Has this been a problem in SMI?  If I IMPORT ifAlias from IF-MIB, and
someone tries to use this togther with an old version of the IF-MIB it
won't work.

/martin

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 5 May 2008 16:40:53 +0200

> draft, this is the current situation.  One solution is the versioned

The alternative is to be explicit if you use a grouping what you use.
This is what we ended up with when we did SMIng in the NMRG (I am not
saying this is the solution but at least one solution to consider).

/js

---

From: Wes Hardaker <[email-masked]>
Date: Mon, 05 May 2008 08:17:32 -0700

>>>>> On Mon, 05 May 2008 15:56:35 +0200 (CEST), Martin Bjorklund <[email-masked]> said:

>> at-least-revision 2008-05-01;

MB> Has this been a problem in SMI?  If I IMPORT ifAlias from IF-MIB, and
MB> someone tries to use this togther with an old version of the IF-MIB it
MB> won't work.

No, but SMI doesn't let you import entire sections of a tree.  There is
no equivalent "import group".  If you want all the columns of a table to
be referenced in an SMI document you must manually import every single
column.

When importing a set (grouping) that may change in contents, it's
equally important that you can specify a minimum compatible version.

(If you're going to require exact version imports then it doesn't matter
anyway.  My point is you need either a >= operator or a = operator for
an import statement)
-- 
Wes Hardaker
Sparta, Inc.
_______________________________________________
netmod mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/netmod

---

From: Phil Shafer <[email-masked]>
Date: Mon, 05 May 2008 11:16:40 -0400

>saying this is the solution but at least one solution to consider).

This is painful with hiearchies of nodes, since you'd need to list
the complete hierarchy of nodes that you want to import, including
containers, leafs, lists, leaf-lists, enumerations, etc.  You'd end up
with something like:

    use foo {
        container system {
            container services {
                container ssh {
                    leaf version {
                        type enumeration {
                            enum v1;
                            enum v2;
                        }
                    }
                    leaf port;
                    leaf rate-limit;
                    leaf connection-limit;
                    leaf-list allow-address;
                    leaf-list deny-address;
                    container trace-options {
                        list flags {
                            leaf name {
                                type enumeration {
                                    enum listen;
                                    enum deny;
                                    enum auth;
                                    enum protocol-error;
                                    enum debug;
                                }
                            }
                        }
                    }
                }
            }
       }

At this point, what's the grouping doing for you?  It would be like
doing a #include of a file, but having to list the contents of that
file inline.  It's the belt and suspenders approach.  With big
suspenders.  ;^)

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Mon, 05 May 2008 08:25:40 -0700

>> I think the critical things (in my mind) are that:

We are in new uncharted territory wrt/ backward compatibility.
SMIv2 has strict rules (CLRs) for some important reasons.
One of them is the need to control the interactions between
modules.  We never want the 'protocol contents' of a module
to magically change because other modules have changed.

In SMIv2, only data types and external columns used in
INDEX clauses can be changed in this manner.  TCs can
only be changed in specific ways to maintain backward compatibility.

In YANG, the uses-stmt can cause new nodes to show up.
Augment doesn't matter.  The new nodes are really in the
original module's namespace, so this is not really a change
to the augmented module.

There MUST be some mechanism in YANG (either conformance or inline)
to 'lock down' the contents of such a uses-stmt, so it is unambiguous
which version of the grouping (from another module) is being used.

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 5 May 2008 17:46:04 +0200

[...]

Yes, this is verbose. But this is actually spelling out what happens.
Perhaps it is possible to improve the syntax but that is a secondary
issue.

Probably encoding a version number into the name of a grouping, that
is introducing versioned identifiers, is actually not such a bad
idea. This way, all versions of the grouping are easy to distinguish
and accessible from a single place and if I want to use a new version
of a grouping, I can be explicit. We just would need to figure out how
to legalize such version updates under the versioning rules.

I am concerned that having to deal with N version of the same module
and being able to fully understand the impact of changing an import
from one revision N to N+x is really tricky without having decent tool
support (because the tool would have to compute the verbose stuff to
allow comparisons).

/js

---

From: Andy Bierman <[email-masked]>
Date: Mon, 05 May 2008 08:54:15 -0700

>     at-least-revision 2008-05-01;

This doesn't really help the 'changing uses-stmt' problem.
If a grouping in module 'A' adds new columns each version,
it is still not clear to a manager which version of the grouping
is expected for a particular uses-stmt in the module importing 'A'.

The problem with "import exact version" and "import version range"
is that the upper bound on the version is artificial.  Module 'A'
will be updated for something that has nothing to do with the grouping,
and modules that import A will 'break' according to the rules.
Every one of these modules needs to be updated as well, reflecting
the new version of 'A'.  Worst of all, implementations will be forced
to maintain several versions of each module.  Implementers and
operators accustomed to only referring to the most recent documentation
might not appreciate this 'feature' of YANG.

IMO, none of the proposed solutions actually works, except my
conformance proposal (or any proposal which requires the leafs
in the module namespace to be explicitly listed for each module version).

I like the conformance approach better because it more precise.
I want to say "Subtree /foo/bar/A" is from version 2008-05-05 of
module A.  That is a much finer-grained tool than saying "I want to
import version 2008-0505 of module A".

Andy

---

From: Phil Shafer <[email-masked]>
Date: Mon, 05 May 2008 12:58:40 -0400

>Yes, this is verbose. But this is actually spelling out what happens.

Verbose, and a severe dilution of the functionality of groupings
as a shorthand for writing complex hierarchies.

>idea.

This requires the reader of "foo_v35" to thread their way back thru
the 34 earlier versions.  We trade benefits for "readers that want
to understand the history of all revisions of this module" against
"reader who want to understand the current revision of this module".
The former can look at previous revisions, resorting to tools if
needed to view the history.  And the latter can use a tool to display
the outcome of merging the various groupings together.  But which
reader do we want to optimize for?

Thanks,
 Phil

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 5 May 2008 19:48:22 +0200

On Mon, May 05, 2008 at 12:58:40PM -0400, Phil Shafer wrote:

> the 34 earlier versions.

I don't get this; please explain.

> We trade benefits for "readers that want to understand the history
> of all revisions of this module" against "reader who want to
> understand the current revision of this module".  The former can
> look at previous revisions, resorting to tools if needed to view the
> history.  And the latter can use a tool to display the outcome of
> merging the various groupings together.  But which reader do we want
> to optimize for?

The point is that "current revision" is pretty meaningless. If we do
versioned imports, the user will have to find the right version of the
module first to look at and only then the search becomes easier. The
question boils down to whether it is

a) easier to search for the right version of a definition in a
   single module or

b) easier to find the right version of the module to find the
   right definition.

Furthermore, if I cut'n'paste an example to an IETF mailing list, a
vendor bug tracker, a news forum etc., I have the feeling that

     uses foo:foo_v35;

or 

     grouping foo_v35 {
         // ...
     }

is actually easier for me to track since I do not need to maintain an
implicit version context established by the import clauses (and which
people likely leave out).

That said, I am sure we will have to deal with groupings that
implementors like to subset. I have some doubts that a standards body
will get all the groupings right such that all details will work for
every implementation equally well (which is a related subject).

/js

---

From: Phil Shafer <[email-masked]>
Date: Mon, 05 May 2008 14:17:03 -0400

>I don't get this; please explain.

grouping foo {
    leaf f1;
}

grouping foo_v2 {
    uses foo;
    leaf f2;
}

grouping foo_v3 {
    uses foo_v2;
    leaf f3;
}
...
grouping foo_v35 {
    uses foo_v34;
    leaf f35;
}

So to understand what's in foo_v35 takes some legwork.  You need
to follow the path of uses and grouping and ferret out what's in
the final foo_v35.

Even this assumes a linear parentage, which isn't possible, if, say,
foo_v20 wants to deprecate f2 from foo_v2.

But more importantly, this view is giving you the history of the
module, rather than what the current module contents are.  The
reviewer needs to see the current model, not the strung out
"foo"s of ancient versions.  They need to understand how the
pieces fit together, not neccessarily how they got there.

>module first to look at and only then the search becomes easier.

My assumption is that the reviewer will only be reviewing modern
stuff, which implies they are importing the modern revision of
their imports.

>every implementation equally well (which is a related subject).

Perhaps this should be done with refinements?  The refinement
can set the status to something that tells you I don't want
that particular node.  Imagine:

    grouping mine {
        uses some:standard-grouping {
            leaf yours {
                status unimplemented;  // cast-off?  uninteresting?  gone?
            }
        }
    }

For those the cast-iron stomachs, imagine using this in an augment
to let the app know you have to break the contract of the original
module.

    // I can't set the mtu on a logical unit on a sonet interface.
    // The hardware simply won't allow it.
    augment /if:interfaces/if:interface:/if:unit {
        when "if:type == 'sonet'";
        leaf mtu {
            status ignored;
        }
    }

Yeah, pretty gross....

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Mon, 05 May 2008 11:22:45 -0700

> vendor bug tracker, a news forum etc., I have the feeling that

Thanks for pointing out that you can just create a new grouping,
which has a different name.  This seems like the right thing to do
if any of the older versions are still in use.

A strict rule, but it works:

  * Once a grouping is published, it can never be changed, except
    to change the status from current to deprecated, then to obsolete.

    grouping foo_v35 {
      uses foo_v34;
      container yet-another-cool-feature { ... }
    }

I think that the module with the grouping needs to specify which
versions are still current, deprecated, or obsolete.

When a new module is created, it will normally use the latest version.
Deployed modules which use 'foo' can be updated as needed, but the
compiler must indicate if the version is not current.

It would be better if every 'uses foo' in the same module was
the exact same version of foo.  Is this important or just nice-to-have?

It would be better if there was a way to indicate the grouping
was version-sensitive, or if it is okay to always use the latest\
version.

It would be better if the YANG data constructs were
simple, and the conformance gunk at the end was complicated,
rather than making the data complicated.

Andy

---

From: "Randy Presuhn" <[email-masked]>
Date: Mon, 5 May 2008 11:32:27 -0600

Hi -

> From: "Andy Bierman" <[email-masked]>
> To: "Phil Shafer" <[email-masked]>; "Martin Bjorklund" <[email-masked]>; <[email-masked]>
> Sent: Monday, May 05, 2008 12:22 PM
...
...

It seems to me that the "current, deprecated, or obsolete" designation
really belongs in the conformance material, rather than attached to
the grouping. 

Randy

---

From: "David Harrington" <[email-masked]>
Date: Mon, 5 May 2008 14:36:17 -0400

Hi,

SNMP and SMI have lots of CLRs, but the change control rules in SMI
are not "crappy little" ones; they are critical to interoperability.

dbh

> [mailto:[email-masked]] On Behalf Of Andy Bierman
> Sent: Monday, May 05, 2008 11:26 AM
> To: Martin Bjorklund
> Cc: [email-masked]

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 5 May 2008 20:36:19 +0200

> pieces fit together, not neccessarily how they got there.

Yes, if I decide to do things this way. But I might also decide to
simply spell out foo_v35 in all its detail as a service to the
reader. I can do as I see fit. I guess one of the problems we are
facing here is that we do not really know (at least I don't) whether
at the end 90% of our definitions will be just groupings or whether
groupings will account for just some 10-20% of our definitions. And I
don't know what I should expect to be the average size of groupings.
So making trade off decisions is difficult; is there a way to phase
this so that we do not have to decide now and defer a decision to a
point where we are sure we better understand the trade-offs?
 
> stuff, which implies they are importing the modern revision of

Perhaps true for a reviewer. For someone having to deal with concrete
devices, you likely have to deal with all revisions that were ever
produced of a popular module. While this might be OK for IETF modules
since the IETF speed works as a natural barrier, it seems that having
to deal with many different versions of vendor modules can become an
interesting exercise for the reader.

>         when "if:type == 'sonet'";

But we should not overload "status" with this.

/js

---

From: Wes Hardaker <[email-masked]>
Date: Mon, 05 May 2008 11:35:22 -0700

>>>>> On Mon, 05 May 2008 14:17:03 -0400, Phil Shafer <[email-masked]> said:

PS> grouping foo {
PS>   leaf f1;
PS> }

PS> grouping foo_v2 {
PS>   uses foo;
PS>   leaf f2;
PS> }

If this type of thing is going to be common wouldn't it make more sense
to make the language account for the need to do versioned groupings to
make writing them (and reading them) easier?

grouping foo 1 {
  leaf f1;
  leaf f2;
}

grouping foo 2 {
  leaf f3;
  obsolete f2;
}
...

Obviously 1 and 2 could use the date specification form of versioning
that are already becoming standard for module revisions?  Consistency
in the spec would make sense.

You could also move the versioning to inside the grouping, though I'm
not sure I like it as much.  I haven't decided:

grouping foo {
  leaf 1:  f1;  # no obsolete point yet
  leaf 1:1 f2;  # obsoletes after 1
  leaf 2:  f3;  # created in version 2
 }

Or something like that.

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 5 May 2008 20:40:53 +0200

> really belongs in the conformance material, rather than attached to

Why? Status tells me something about the status of the definition,
whether I can still safely use it or not. So it is a property of
the definition itself.

/js

---

From: "David Harrington" <[email-masked]>
Date: Mon, 5 May 2008 14:52:44 -0400

I think one thing we need to consdier is what tools ar egoing to use
YAG modules. In large SNMP-based NMS systems like OpenView or
Spectrum, the parsing of MIB modules is a tiny piece of the
application. 

The MIB modules are parsed into application databases, and the
assumptions of backwards compatibility of MIB modules that are found
in SMI are embedded deep within the application code. 

I have very strong doubts that there will be a whole new generation of
YANG-specific tools. If we make th etransition a reasonably
cost-effective thing to do, then the existing tools will be updated to
support YANG. 

If we change the basic assumptions of backwards compatibility, then we
make it economically difficult to get vendors to update NMS
applications. SNMPv3 proved that changing some of the fundamental
concepts means NMS application vendors simply do not make the
transition.

We should try tio make sure we follow similar rules for backwards
compatibility. I think this means we need to explicitly list the leaf
nodes (etc.) that will be imported, either in IMPORT statements or in
compliance clauses; I am not sure which will be more read-friendly;
obviously only having to list them in one place makes it more
write-friendly.

dbh

> From: Andy Bierman [mailto:[email-masked]] 
> Sent: Monday, May 05, 2008 11:54 AM
> To: Wes Hardaker
> Cc: David Harrington; [email-masked]

---

From: "Randy Presuhn" <[email-masked]>
Date: Mon, 5 May 2008 11:53:22 -0600

Hi -

> From: "Juergen Schoenwaelder" <[email-masked]>
> To: "Randy Presuhn" <[email-masked]>
> Cc: <[email-masked]>
> Sent: Monday, May 05, 2008 12:40 PM
> Subject: Re: [netmod] current, deprecated & obsolete
>

Only if by "use" you mean "implement in new software".  For purposes of
communicating with deployed gear, it's meaningless.  If a box implements
a particular knob, and not its more recent replacement, then managing that
box means working with that knob, regardless of how obsolete it is.

That's why I think it makes more sense to think about it in terms of conformance
to a particular version of a specification, rather than as an intrinsic property of
that particular bit of the data model.

Randy

---

From: Phil Shafer <[email-masked]>
Date: Mon, 05 May 2008 14:51:48 -0400

>point where we are sure we better understand the trade-offs?

Sure, we go with what YANG does now.  The import statement imports
the current revision and you are left to your own wits to make
it work.

>While this might be OK for IETF modules

My guess is that vendors will choose between two strategies:
- one revision per release (1r/r)
- one module per release  (1m/r)

1r/r gives you the ability to define one "JUNOS" module that you
update each release.  Since devices only run one version of software,
the device announces "JUNOS?rev=2008-04-01", where the revision is
the release date of the software version.  This will work with one
release train, but vendors need to support multiple trains, one
for each release of software.

1m/r gives you a "JUNOS-9.1" module that you update only as "point"
releases of 9.1 are published ("9.1R1", "9.1R2").

For us, these modules will be generated from our existing DDL, until
we switch to using YANG internally.  Trying to auto-generate
"foo_v2"-style groupings will be a pain.  Even when using YANG-based
tools internally, I don't think I want developers too stare at this
history-based threading.  I just want them to see the current views
of the current module.  Adding a leaf to a grouping should be a
one-line fix, not something that requires adding a grouping, using
the old grouping, add the leaf, and then updating every user of
that grouping.

Also interesting is the ability of third-party on-device application
writers using our SDK to import the modules that are specific to
the release of JUNOS they are running.

Thanks,
 Phil

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 05 May 2008 20:55:47 +0200

> saying this is the solution but at least one solution to consider).

Right, I think for the purpose of reusing pre-baked groupings and
typedefs it should be sufficient to import an exact version of a foreign
module and I would suggest

import URI?revision;

I imagine a module author pretty much always wants to control what is
imported and therefore can choose an exact version of the foreign
module.

Changing the import statement as above would not work so well for the
other extension mechanism, namely augmenting the foreign module via
augment-stmt with absolute argument since this is probably supposed to
work also for future versions of the foreign module.

A solution could be to separate the two extension mechanisms:

1. Use import with an URI and revision really for importing very
specific foreign groupings and typedefs for reuse in the local module.

2. Devise a new statement for the function of augmenting the foreign
module, for example

    modify <module name> {
        append <abs-path> {
            ...
        }
        ...
    }

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 5 May 2008 21:09:32 +0200

On Mon, May 05, 2008 at 11:53:22AM -0600, Randy Presuhn wrote:
 
> Only if by "use" you mean "implement in new software".  

It is about "implement in new software" and "use in new modules".

> For purposes of communicating with deployed gear, it's meaningless.
> If a box implements a particular knob, and not its more recent
> replacement, then managing that box means working with that knob,
> regardless of how obsolete it is.

Sure, this is what we do anyway. And the market sometimes even forces
new software to implement deprecated or obsolete things.

> That's why I think it makes more sense to think about it in terms of
> conformance to a particular version of a specification, rather than
> as an intrinsic property of that particular bit of the data model.

I believe having a status property of all YANG definitions itself is
useful, in particular for the "use in new modules" scenario.

/js

---

From: "David Harrington" <[email-masked]>
Date: Mon, 5 May 2008 15:11:32 -0400

During the World Tour in 2002, operators complained that vendors kept
changing the interface in a way that broke their scripts, and they
were forced to update their scripts for (potentially) every release
from every vendor. 

Which approach to "uses" and imports minimizes the script churn
problem for operators?

dbh

> Sent: Monday, May 05, 2008 2:52 PM
> To: [email-masked]
> Cc: [email-masked]

---

From: "Randy Presuhn" <[email-masked]>
Date: Mon, 5 May 2008 12:22:43 -0600

Hi -

> From: "Phil Shafer" <[email-masked]>
> To: <[email-masked]>
> Cc: <[email-masked]>
> Sent: Monday, May 05, 2008 12:51 PM
...
...

Perhaps the authors of the "reuse" section in the presentation
https://www3.ietf.org/proceedings/08mar/slides/canmod-3/canmod-3.htm
could speak up.  Their approach is quite different from traditional
"inheritance" and the group "imports" that have been getting so
much discussion here.

My reading of their ideas:
 Key concepts:
   - Things that are reusable don't change in any substantive way.
   - A substantively changed version of any thing is a new thing.
   - The description of an implementation (version) is assembled
      largely from a bunch of reusable things, with the addition of
      whatever peculiar (non-reusable, vendor or implementation-
      specific) bits are needed

 Interesting properties:
  - the "inheritance" hierarchy is *very* shallow
  - the need for anything like "agent-capabilities" goes away
  - the need for machine-readable conformance material largely goes away

From a modeling / re-use perspective it's rather like going to an
"aspect-oriented" approach, allowing one to assemble a product's
management interfaces from the various capabilities that the product
needs to do its job.

Randy

---

From: Phil Shafer <[email-masked]>
Date: Mon, 05 May 2008 15:33:48 -0400

>were forced to update their scripts for (potentially) every release

The operators were talking about screen scraping.  We call it "Expect
Hell".  When the output of "show interfaces" changes, the regular
expression used in an expect script may no longer match the appropriate
information and you get more/less data than required.  NETCONF's
use of XML is meant to help build scripts that can survive over
releases, since well-written XPath expressions don't care about
unrelated content.  (Yes, there are folks that feel well-written
regular expressions can be created to survive over multiple releases,
but they are relatively few (scripts and folks)).

>Which approach to "uses" and imports minimizes the script churn

The content here is the same.  It's a matter of how it's written
down in the module.

Thanks,
 Phil