Subject: [Netconf] netconf yang module namespaces

From: Juergen Schoenwaelder <[email-masked]>
Date: Sun, 23 Oct 2011 21:53:00 +0200

Hi,

it is very good to see YANG data models to configure / monitor various
aspects of NETCONF. That said, I am concerned about the usage of
namespaces (and even at the risk that people say "go away with your
academic nit picking since we focus one thing at a time"), I felt that
I at least raise my concern. Right now, we have the following
namespaces

urn:ietf:params:xml:ns:yang:ietf-netconf-monitoring	RFC 6022
urn:ietf:params:xml:ns:netconf:base:1.0			RFC 6241
urn:ietf:params:xml:ns:netconf:notification:1.0		RFC 5277 (no YANG)
urn:ietf:params:xml:ns:yang:ietf-netconf-with-defaults	RFC 6243
urn:ietf:params:xml:ns:yang:ietf-netconf-acm		I-D.NACM
urn:ietf:params:xml:ns:yang:ietf-netconf-notifications	I-D.NSN

and there might be additional modules for say the TLS transport and
other NETCONF extensions coming to us soon. Technically, having all
these relatively small and modular data models live in their own
namespace is just fine for programs but for human readers of
configurations, the many namespace prefixes make reading and editing
configurations more complicated than it needs to be I think.

So the question is whether we should seriously consider using
submodules instead.  This would allow us to have a single namespace
and things like the acm configuration model or the TLS certificate
mapping configuration model just become submodules and part of this
common namespace. In a nutshell, we would have this little parent
module

module ietf-netconf-parent {

  namespace urn:ietf:params:xml:ns:yang:ietf-netconf-parent;
  prefix ncp;

  include ietf-netconf-acm {
    revision-date 2011-xx-xx;
  }

  include ietf-netconf-notifications {
    revision-date 2011-xx-yy;
  }

  // ...
}

and we would include an updated version of it (with additional
includes) whenever a new NETCONF data model gets published as RFC. I
think we need to answer this question now; RFC 6022 and RFC 6243 would
then be "early accidents" and RFC 6241 is special anyway since the
namespace had been fixed by the design of RFC 4741, which clearly
predates YANG (and the same would be true in case RFC 5277 ever gets
rewritten in YANG).

/js

PS: This idea apparently is not new, Martin kind of pointed to it
    already back on Tue, 11 Aug 2009 21:36:10 +0200.

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: Andy Bierman <[email-masked]>
Date: Sun, 23 Oct 2011 13:46:45 -0700

> common namespace. In a nutshell, we would have this little parent

I proposed a YANG module for a registry in draft-bierman-netmod-yang-usage-00.txt
in January, 2009.  It got taken out way before RFC 6087 was published.
I now realize our standards process is more robust when there are specific RFCs
to point at, so a registry would not really work for standard modules.

A 'main module' RFC that keeps changing all the time doesn't seem so good either.
Every RFC published would have a normative reference to this main module RFC,
except the sub-modules will reference an assortment of main module RFC numbers.
The RFC publishing process gets slowed down enough already due to normative references.
Waiting in line to get the main module updated doesn't work.

Separate namespaces are going to happen for augment-stmts anyway, so applications
and servers will not be able to ignore namespaces.

I think we could invent some YANG/NETCONF shorthand for the 'std' namespace, like in C++.
It may be too late now, but something that avoids a main module with include-stmts that
change constantly would work.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Sun, 23 Oct 2011 22:59:37 +0200 (CEST)

Andy Bierman <[email-masked]> wrote:
> > submodules instead.

I think we should (at least seriously consider it - now).

> > This would allow us to have a single namespace

One approach is to update the main module in the same document that
publishes the new submodule.  It's not like we publish new data models
so fast that we can't handle this.  One issue is how two handle if two
or more submodules gets published at the same time.  I guess one of
the new documents (or another new document) contains the module.

Another approach is to let IANA handle the main module.  It would then
be a normal IANA registry, but in YANG format.

> and servers will not be able to ignore namespaces.

That is not the goal here.  The goal is to reduce the number of
namespaces.

> > module ietf-netconf-parent {

We should have called the module in 6241
"ietf-netconf-protocol"... "parent" is not very descriptive... need to
think about a good name.

> > predates YANG (and the same would be true in case RFC 5277 ever gets

Our current charter allows an update to 6022, so maybe that one can be
fixed.  6243 only defines additional parameters to the operations
defined in 6241, but I guess they could have been done in the
"protocol" namespace if it was done as a submodule.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Sun, 23 Oct 2011 14:09:12 -0700

On 10/23/2011 01:59 PM, Martin Bjorklund wrote:
> Andy Bierman<[email-masked]>  wrote:

I do not think there are too many namespaces.
I do not agree there is a problem with each YANG module having
its own namespace, especially since the standard <capability> URI
does not support sub-modules.

I think it's OK if a WG want s to structure a new module so that
it uses sub-modules.  The modules that are already published are
done and should not be touched.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Sun, 23 Oct 2011 14:42:49 -0700

> I think we should (at least seriously consider it - now).

Since you brought it up, you should define the problem in detail,
so we can tell evaluate solution proposals.

A giant kitchen sink module seems worse than a URI for every little module.
The app probably needs to align with a specific version of each managed object,
so a list of submodule name/submodule revision dates would be needed if this
was removed from the the <hello>.

If there is some usable functional modularity, i.e., some module cohesion,
then sub-modules are appropriate.  I think the latest NETMOD work can be consolidated
because there are too many modules.

In hindsight, one ietf-netconf.yang for all the NETCONF protocol sub-modules might
be a little better than what we have (all modules), but it's not worth redoing it now.

> ...

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 24 Oct 2011 10:33:16 +0200

> done and should not be touched.

I agree with Andy. Moreover, I don't think NETCONF instance XML 
documents are intended to be read by humans, except for debugging. Also, 
if prefixes really were an issue, it is possible to use namespace 
declarations in a way that considerably reduces the use of prefixes.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Kent Watsen <[email-masked]>
Date: Mon, 24 Oct 2011 14:01:52 -0700

This thread appears the same as the one started a few months ago on the netmod list:  http://www.ietf.org/mail-archive/web/netmod/current/msg05931.html.  

That thread peters out, but I maintain that YANG unnecessarily imposes a technical constraint that could've been handled by a maintainer of multiple YANG modules (such as this group).

All we'd need to do is change section 5.3 from:

   All YANG definitions are specified within a module that is bound to a
   particular XML namespace [XML-NAMES], which is a globally unique URI
   [RFC3986].

To:

   All YANG definitions are specified within a module that is bound to
   a particular XML namespace [XML-NAMES].  Multiple YANG modules MAY
   share a namespace but, to guarantee no namespace collisions, each
   YANG module SHOULD have a unique XML namespace.

This would be a backwards-compatible change to the spec.

Thanks,
Kent

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 24 Oct 2011 23:24:21 +0200

> This would be a backwards-compatible change to the spec.

Submodules were designed into YANG to do this, allowing tools to check
that namespace collisions do not occur etc. What do we gain by not
using them?

Andy's argument that the namespaces matter in the <hello> exchange
assumes that namespaces are useful units to check for in server
capabilities. Given the way work is organized in bodies like the IETF,
this may or may not be true. I would rather go with less granualar
namespaces and encourage the definition of features. AFAIK, this
allows a client still to discover data model features supported by a
server without polluting the config itself with too many different
namespaces.

/js

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 24 Oct 2011 23:28:11 +0200 (CEST)

Hi,

Kent Watsen <[email-masked]> wrote:
> This thread appears the same as the one started a few months ago on the netmod
> list: http://www.ietf.org/mail-archive/web/netmod/current/msg05931.html.

It is related, but not the issue here.

Even with your proposed change to YANG, the question would be if this
new feature should be used in this particular case.  And for
IETF-based modules, your proposed solution would probably require new
IANA registries to keep track of the namespaces so no collisions
occur.

If we agree that we want to minimize the number of namespaces, I think
we could still use submodules as they are defined today.  The drawback
(compared with your proposal) is that each submodule's revision is not
directly advertised in the hello, but it is not clear to me that this
is a problem, and if it is a problem worth solving, we could design a
way to advertise submodules w/o changing 6020.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Mon, 24 Oct 2011 14:30:17 -0700

> This would be a backwards-compatible change to the spec.

I do not agree.
An application that expects a 1:1 mapping between YANG module and XML namespace
will not accept multiple YANG modules with different names but the same namespace URI.

IMO, this protocol is for applications, and it is not that difficult for
applications to deal with XML namespaces.  The YANG module reader doesn't
really care what is in the namespace-stmt string, so it does not hurt readability.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Mon, 24 Oct 2011 15:02:40 -0700

On 10/24/2011 02:28 PM, Martin Bjorklund wrote:
> Hi,
>
> Kent Watsen<[email-masked]>  wrote:

I think RFC 6020 is fine the way it is wrt XML namespaces.
I do not agree a giant kitchen-sink YANG module and/or XML namespace is a good idea.

I agree that on a case-by-case basis, YANG designers may choose to
divide a single module into a main module plus some submodules.

I don't really understand the problem.
Why is it a burden to encode NETCONF content with the XML namespace URI
specified in the YANG module namespace-stmt for that content?
The current protocol and YANG spec work fine for 1 module per XML namespace.
Augment depends on it.

Andy

---

From: Kent Watsen <[email-masked]>
Date: Mon, 24 Oct 2011 16:39:04 -0700

Every time this issue comes up, the first comment seems to be that this is what submodules can solve.  But this is simply not true, since in my case, each device chooses which revision of which "submodules" (if they were) they advertize  I cannot write a parent module that restricts the submodule revisions to a specific subset, at least not without writing a parent module for every possible combination of submodule revisions, which obviously would be wrong.

Submodules were designed primarily to support the notion of an operating system (Junos, IOS, etc.) being composed of set of components (routing, policy, auth, system, etc.), such that each release of the operating system (Junos 11.3R1) could have a single top-level module that includes the submodule from each of its components.  Submodules are perfect for this scenario.

But submodules don't work so well when the YANG modules are defined by a 3rd-party (IETF or, in my case, the vendor's in-house NMS group) where the devices have discretion as to which version of which modules they want to support - they want to cherry-pick what they support and incrementally support a newer version of a module without suddenly having to support a newer of all the modules.  Hence, we use top-level modules (not sub-modules), so that each device can granularly advertize what's supported.

Concrete example, we originally defined modules like this:

    module hardware {
        namespace "http://xml.juniper.net/dmi";   <--- same namespace
        prefix "dmi";
        revision "2008-11-05" {
            ...
        }
        ...
    }
    module software {
        namespace "http://xml.juniper.net/dmi";   <--- same namespace
        prefix "dmi";
        revision "2008-11-05" {
            ...
        }
        ...
    }
    module license {
        namespace "http://xml.juniper.net/dmi";   <--- same namespace
        prefix "dmi";
        revision "2008-11-05" {
            ...
        }
        ...
    }

But after the last thread not resolving the issue, we changed them to:

    module hardware {
        namespace "http://xml.juniper.net/dmi/hardware";   <--- different namespace
        prefix "dmi";
        revision "2008-11-05" {
            ...
        }
        ...
    }
    module software {
        namespace "http://xml.juniper.net/dmi/software";   <--- different namespace
        prefix "dmi";
        revision "2008-11-05" {
            ...
        }
        ...
    }
    module license {
        namespace "http://xml.juniper.net/dmi/license";   <--- different namespace
        prefix "dmi";
        revision "2008-11-05" {
            ...
        }
        ...
    }

The impact to the <hello> message is that it (unnecessarily) went from:

    http://xml.juniper.net/dmi?module=hardware&revision=2008-11-05
    http://xml.juniper.net/dmi?module=software&revision=2008-11-05
    http://xml.juniper.net/dmi?module=license&revision=2008-11-05

To:

    http://xml.juniper.net/dmi/hardware?module=hardware&revision=2008-11-05
    http://xml.juniper.net/dmi/software?module=software&revision=2008-11-05
    http://xml.juniper.net/dmi/license?module=license&revision=2008-11-05

[Of course, there's a similar impact to the "xmlns" attribute in the XML as well]

My biggest concern (reported last time) was/is that 6020 is a bit ambiguous.  At least my reading and those of a few others led us to believe what we had done originally was correct.  We felt justified in this conclusion in that YANG-forced solution seems to adds little value, since my group (or IETF, to this thread's OP's point) is the sole maintainer of our namespace(s) and hence can ourselves guarantee that no collision will ever occur.

Thanks,
Kent

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 25 Oct 2011 11:35:48 +0200

On Mon, Oct 24, 2011 at 04:39:04PM -0700, Kent Watsen wrote:
> 
> Every time this issue comes up, the first comment seems to be that
> this is what submodules can solve.  But this is simply not true,
> since in my case, each device chooses which revision of which
> "submodules" (if they were) they advertize I cannot write a parent
> module that restricts the submodule revisions to a specific subset,
> at least not without writing a parent module for every possible
> combination of submodule revisions, which obviously would be wrong.

[...]

So the issue seems to be that the revisions of submodules are not
announced in the hello exchange. Do we all agree with that?

And reading /netconf-state/schemas is also not an option to obtain
that information since people really like to have the details in the
<hello> exchange. Do we agree with that as well?

/js

---

From: Andy Bierman <[email-masked]>
Date: Tue, 25 Oct 2011 03:39:02 -0700

> announced in the hello exchange. Do we all agree with that?

yes -- I think I brought up this issue awhile back.
A new 'submodule' capability is needed that is simpler than the module capability.

   <static-URI-for-submodules>?submodule=foo&module=parent-of-foo&revision=2011-10-29

revision == submodule-revision-date
(no submodule features or deviations allowed!)
(may want parent-revision in the URL as well)

> <hello>  exchange. Do we agree with that as well?

yes -- implementing the <schema> data structure is optional and cannot be counted
on to be present, so <hello> is the only reliable option.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 25 Oct 2011 13:34:29 +0200

> This would be a backwards-compatible change to the spec.

So what would you do with prefixes? Do you want to allow each of the 
namespace-sharing modules to define a different prefix? I don't want to 
open this can of worms. Why is it a problem to choose a separate URI for 
each module if the module names are already different?

Lada

>
> Thanks,
> Kent
>
>
> _______________________________________________
> Netconf mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netconf

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C