Subject: [netmod] staying in synch on data model discovery

From: Andy Bierman <[email-masked]>
Date: Sun, 09 Aug 2009 09:00:14 -0700

Hi,

I would like to see the NETCONF and NETMOD WGs act more
like they play for the same team regarding YANG and all
the YANG data models under development.

I mostly care about the ietf-netconf-state data model to
use the get-schema() operation with YANG.  If others want
XSD and RNG instead, then that's extra credit, but full
support for YANG is a must-have.

YANG modules, submodules, and deviation modules all need
to be listed in the <schemas> and retrievable with <get-schema>.

However, deviations have no revision that is advertised,
yet they clearly have a revision-date just like any other
YANG module.  This is somewhere between confusing and broken.

The <version> field in the <get-schema> operation is mandatory.
It should be optional, and if missing, then the
agent will pick whatever revision it wants (usually
there will just be 1 to pick from).

The <format> field in the <get-schema> operation is mandatory.
It should be optional, and the default should be 'YANG'.

IMO, the use of capabilities instead of YANG features needs
to stop.  Even more, ALL the data models under way in the
NETCONF WG should wait for YANG, and use YANG features,
not protocol capabilities to achieve.  That will eliminate
the possibility that there can be 16 valid variants of
the same leaf,and will help start out with a coherent set of
YANG modules using the same data types (e.g., ip-address issue).

I think the <schema> node should have another leaf:

   leaf moduleType {
     description
        "The type of data model file for this schema entry.";
     type enumeration {
        enum module {
          description
            "Indicates this entry is for a main module.";
       }
       enum submodule {
          description
            "Indicates this entry is for a sub-module.";
       }
       enum deviation {
          description
            "Indicates this entry is for a deviation module.";
       }
       enum other {
          description
            "Indicates this entry is for some other type of module.";
       }
    }
 }

This will help operators understand what all the 'extra' entries are for,
and allow better filtering of the <schemas> subtree.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 12 Aug 2009 09:00:57 +0200 (CEST)

Hi Andy,

> support for YANG is a must-have.

I agree.

> to be listed in the <schemas> and retrievable with <get-schema>.

Yes.  That is the intention of the current design.

> YANG module.  This is somewhere between confusing and broken.

What makes you think that modules with deviation statements are not
advertised with their revision?

> It should be optional, and the default should be 'YANG'.

I agree that these two parameters should be made optional, but I think
that an error should be returned if there are multiple revisions
and/or formats.  In most cases there will be just one revision and one
format so that's the one that will be returned.

> to stop.

Agreed.  For the current drafts, I think this applies to partial-lock
which we don't want to change at this point, and with-defaults which I
believe Juergen commented on in some other thread.  I think the
monitoring draft is fine.

>             "Indicates this entry is for some other type of module.";

I don't think this is necessary, and in any case, there is no such
thing as a 'deviation' module.  In theory, any module / submodule can
have deviation statements.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Wed, 12 Aug 2009 01:34:01 -0700

> advertised with their revision?

The module capability does not include a revision date
for the deviations that are attached to a particular module.
So if there is more than 1 version of a module advertised
as a deviation, it is not known by the manager which
version is being used as the deviation for 'foo'

  module=foo&revision=2009-01-01?deviations=bar,baz,baz2

So when I retrieve bar, baz, or baz2, I have no idea
what to put in the mandatory version field.  IMO -- kind of broken.

>>             "Indicates this entry is for some other type of module.";

Got to love that augment-stmt.
Who cares about standards when you got namespaces! ;-)

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 12 Aug 2009 10:52:41 +0200 (CEST)

> for the deviations that are attached to a particular module.

But you advertise your deviations module, and include the revision
there:

   urn:foo?module=foo&revision=2009-01-01?deviations=bar
   urn:bar?module=bar&revision=2009-04-01

So even if the bar module exists in multiple revisions (this would
happen if bar contains deviations AND reusable typedefs / groupings,
and some other modules import bar by different revision...), only one
is implemented by the device.  We do not support that you advertise
multiple versions of a module with data nodes and / or deviations.

/martin

---

From: "Ersue, Mehmet (NSN - DE/Munich)" <[email-masked]>
Date: Wed, 12 Aug 2009 12:17:07 +0200

Hi Andy, Martin,

I think we should bring the discussion related to NETCONF Monitoring
draft to the NETCONF maillist.

Since we believed the monitoring draft was nearly final I would
appreciate if you guys post clear requests possibly with a text
proposal. 
I think this will help the authors of the monitoring draft and the
NETCONF WG a lot.

One other question though as a contributor:

NETCONF WG developed "capabilities" and introduced several of them.
Is it your proposal they shouldn't be used?
I am wondering whether we need a WG decision or BCP-like statement
saying it shouldn't be used.
It would be interesting if you elaborate a bit why YANG features should
be used instead of NETCONF capabilities.
(I know some of the reasons but might be missing others. At the end we
need a record on the maillist if we want to push such a decision.)

Mehmet

> -----Original Message-----
> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of ext Martin Bjorklund
> Sent: Wednesday, August 12, 2009 9:01 AM
> To: [email-masked]
> Cc: [email-masked]
> Subject: Re: [netmod] staying in synch on data model discovery
> 
> Hi Andy,
> 
> _______________________________________________
> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod
>

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 12 Aug 2009 13:32:52 +0200

On Wed, Aug 12, 2009 at 12:17:07PM +0200, Ersue, Mehmet (NSN - DE/Munich) wrote:
 
> Is it your proposal they shouldn't be used?  I am wondering whether
> we need a WG decision or BCP-like statement saying it shouldn't be
> used.  It would be interesting if you elaborate a bit why YANG
> features should be used instead of NETCONF capabilities.  (I know
> some of the reasons but might be missing others. At the end we need
> a record on the maillist if we want to push such a decision.)

I believe we did not communicate this well...

NETCONF provides a capability exchange mechanism which uses URIs to
identify capabilities. The NETCONF protocol defines some core
capability URIs but otherwise leaves the format of capability URIs
pretty much open. YANG uses the NETCONF capability exchange and
imposes a format on the URIs to communicate among other things data
model defined features. The common format of the URIs allows to write
generic code to analyse these URIs. So everything is actually pretty
fine in the sense that NETCONF provides a mechanism and YANG uses and
further restricts the mechanism in order to enhance interoperability.

All we want to achieve is to do away with handcrafted URI formats
(often miscommunicated as "avoid capabilities") and instead use the
YANG feature mechanisms. The existing NETCONF URIs of course won't be
changed - but for new URIs, we like to use the YANG format.

Does this help?

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: Andy Bierman <[email-masked]>
Date: Wed, 12 Aug 2009 06:16:41 -0700

>    urn:bar?module=bar&revision=2009-04-01

but remember the thread on all this little fields?
You and Phil claimed we need them all so you do
not have to look at more than the "capability I care about".

Now when there is a real need to have the info at
once (so <get-schema> can be called), it is OK to fish through
all the module capabilities to find out.

> multiple versions of a module with data nodes and / or deviations.

There is no requirement in the YANG spec that 2 different versions
of the same module cannot be present.  Import-by-revision requires
the opposite -- the agent must be able to support multiple
versions of the same module (A imports B and C1, B imports C2).

To make matters even worse, revisions are a completely optional
concept in YANG, and a vendor may decide to never use them.
Then the mandatory 'version' parameter is really strange in this case.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 12 Aug 2009 15:42:30 +0200 (CEST)

> all the module capabilities to find out.

I assume you refer to this:
http://www.ietf.org/mail-archive/web/netmod/current/msg01764.html
and your mail before it.

The point was that the client can look at the capability list only -
it does not have to download and parse all modules in order to find
how a device deviates (or not).

> Then the mandatory 'version' parameter is really strange in this case.

Well, I already agreed to your idea that 'version' and 'format' should
be optional.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Wed, 12 Aug 2009 07:05:47 -0700

> how a device deviates (or not).

OK -- I don't want to re-open the module capability URI.
The normal use-cases are covered very well.

> Well, I already agreed to your idea that 'version' and 'format' should

I think this will make get-schema easier to use.
I implemented all of draft-07, and I allow an
empty string for version to mean "agent picks
the version, if there is one.  If not, then I'm
already asking for the correct version -- none".
In YANG terms, just leaving out the parameter would be cleaner.

Andy

---

From: "Randy Presuhn" <[email-masked]>
Date: Wed, 12 Aug 2009 10:43:39 -0700

Hi -

> From: "Martin Bjorklund" <[email-masked]>
> To: <[email-masked]>
> Cc: <[email-masked]>
> Sent: Wednesday, August 12, 2009 1:52 AM
...
> multiple versions of a module with data nodes and / or deviations.
...

Is this really realistic in a world of FRUs?  I can see how one might
shoehorn the models for different versions of a FRU into one, but
the deviations (which might be different for, e.g., different line cards)
seem less likely to work this way.

Randy

---

From: David Reid <[email-masked]>
Date: Fri, 14 Aug 2009 11:42:25 -0400

> multiple versions of a module with data nodes and / or deviations.

I want to make sure I understand this. Does this mean that it would be 
invalid for a netconf server to support 2 different revisions of the 
same yang module. For example:

urn:foo?module=foo&revision=2008-04-05
urn:foo?module=foo&revision=2009-01-01

-David Reid

---

From: Andy Bierman <[email-masked]>
Date: Fri, 14 Aug 2009 09:38:15 -0700

> urn:foo?module=foo&revision=2009-01-01

This would violate the entire purpose of import-by-revision.
YANG tools need to deal with multiple revisions, and that includes
the manager.

The manager is going to check its local module-store
for module=foo, revision=xxx, based on each individual capability URI.
If not found, or the namespace statement does not match
the module capability, then it is going to use <get-schema>
to retrieve any missing files.

Finally, it is going to attempt to compile and use the entire set of
modules and deviations and enable the features that the agent
advertised.  Through hardwired code, it is going to figure
out what it can from the particular combination of standard capabilities
advertised for the session.

Then the manager has the same exact view of the schema tree as the agent.

This used to sometimes take NMS developers many person-years
to get right with SNMP.  Now it takes about 100 milli-seconds with NETCONF.
That's why <get-schema> is so important and data silos are so evil.

The agent must advertise everything it has.
The manager must first figure out what the agent has,
rather than build its 'agent view' incrementally.
(tips from somebody who coded it wrong the first time...)

Andy