Subject: [netmod] review of yang-09

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 05 Jan 2010 10:07:44 +0100

Hi,

here is my review of draft-ietf-netmod-yang-09:

Overall, despite its length and complexity, the draft remains quite
readable and comprehensible.

Fixed typos and suggestions for (hopefully uncontroversial) formulation
improvements are in this diff file:

http://staff.cesnet.cz/~lhotka/NETMOD/draft-ietf-netmod-yang-09.diff

Other comments follow: 

1. My main remaining concern is the possible impact of the "when"
   statement. This was discussed in a separate thread and some changes
   may already be pending, but I think it is necessary to make sure
   that the rules governing whether a given data node is mandatory or
   not are unaffected by "when" (or "must").

   Also, the first bullet in sec. 7.19.5 says:

   o  If the "when" statement is a child of an "augment" statement, then
      the context node is the augment's target node in the data tree, if
      the target node is a data node.  Otherwise, the context node is
      the closest ancestor node to the target node which is also a data
      node.

   and then later in the same section:

   The XPath expression MUST NOT include any references to the context
   node or any descendants of the context node.

   However, these rules are violated by the example in sec. 4.2.8: The
   context node is "user" (the target node of "augment") but the XPath
   expression refers to "class" which is a descendant of "user".

   The same problem will probably also happen for "when" under "uses"
   but this case is not mentioned in any of the bullets of
   sec. 7.19.5.

2. In several places, the text about (XML) namespaces and prefixes
   should be put more in line with the W3C recommendation (its 3rd
   edition became available recently). In particular:

   - sec. 5.3
     OLD 
     All YANG definitions are specified within a module that is bound
     to a particular XML Namespace [XML-NAMES], which is a globally
     unique URI [RFC3986].  A NETCONF client or server uses the
     namespace during XML encoding of data.
     NEW
     Names of most objects that are defined by a YANG module belong to
     an XML namespace [XML-NAMES] defined by the module's "namespace"
     statement (see Section 7.1.3).

   - sec. 5.3.1:
     OLD
     This namespace is "urn:ietf:params:xml:ns:yang:1".
     NEW
     The name of this namespace is "urn:ietf:params:xml:ns:yang:1".

   - sec. 7.1.4
     OLD
     The "prefix" statement is used to define the prefix associated
     with the module and its namespace.  The "prefix" statement's
     argument is the prefix string which is used as a prefix to access
     a module.  The prefix string MAY be used to refer to definitions
     contained in the module, e.g., "if:ifName".  A prefix follows the
     same rules as an identifier (see Section 6.2).

     When used inside the "module" statement, the "prefix" statement
     defines the prefix to be used when this module is imported.  To
     improve readability of the NETCONF XML, a NETCONF client or
     server which generates XML or XPath that use prefixes, the prefix
     defined by a module SHOULD be used, unless there is a conflict.

     NEW
     The "prefix" statement is used to define the namespace prefix
     [XML-NAMES] that is bound to the namespace URI of a module. A
     prefix follows the same rules as an identifier (see Section 6.2).

     When used inside the "module" statement, the "prefix" statement
     defines the prefix to be used when this module is imported.  To
     improve readability of the NETCONF XML, a NETCONF client or
     server which generates XML or XPath that use prefixes, SHOULD use
     the prefix defined by a module, unless there is a conflict.  The
     prefix MAY be used to refer to definitions contained in
     the module, e.g., "if:ifName".
     
   - sec. 7.5.7, and similarly in sec. 7.6.6, 7.7.6, 7.8.5 and 7.10.2
     OLD
     A container node is encoded as an XML element.  The element's
     name is the container's identifier, and its XML namespace is the
     module's XML namespace.
     NEW
     A container node is encoded as an XML element.  The element's
     local name is the container's identifier, and its namespace name
     is the module's XML namespace defined by the "namespace"
     statement.
     
   - sec. 7.2.2
     OLD
     The mandatory "prefix" substatement assigns a prefix for the
     module to which the submodule belongs.  All definitions in the
     local submodule and any included submodules can be accessed by
     using the prefix.
     NEW
     The mandatory "prefix" substatement binds a prefix to the
     namespace name of the module to which the submodule belongs.  The
     binding is scoped to the submodule where it appears. All
     names defined in the local submodule and any included submodules
     MAY be qualified by using the prefix.

3. The normalization of XML attributes may affect not only arguments
   in YIN but also the values of the "value" attribute in sec. 7.7.7.
   For example, a leaf-list item containing a tab character ("\t")
   cannot be addressed using the "value" attribute.

4. Sec. 7.5.3 says:

   Note that since all leaf values in the data tree are conceptually
   stored in their canonical form (see Section 7.6 and Section 7.7), any
   XPath comparisons are done on the canonical value.

   Does it mean that in the following model the "must" constraint can
   never be true?

   must "foo=bar";
   leaf foo {
       type uint8;
   }
   leaf bar {
       type decimal64 {
           fraction-digits 1;
       }
   }

   This would be a deviation from XPath which compares all numbers as
   floats.

5. The term "complex type" is used in sec. 4.1 and 4.2.6. I think this
   term should be explained in the Terminology section or avoided.

6. The term "list key" should be included in Terminology.

7. Sec. 5.1.2 should also say that the order of top-level data nodes
   is arbitrary (as it is in containers).

8. Sec. 4.1 should explicitly mention the deliberate limitations of
   YANG - inability to model XML attributes and recursive structures.

9. Sec. 5.1 says that the names of standard modules and submodules
   MUST be unique. It should also mention there is an IANA registry
   for these.

10. Maybe the server should also explicitly advertize that it uses
    YANG? For one thing, this would inform the client that the other
    capabilities are to be interpreted according to sec. 5.6.4.

11. Sec. 7.1.6 says: "When a submodule includes another submodule, the
    target submodule's definitions are made available to the current
    submodule." Does this mean the same rules as for "import" (the
    bullet list in sec. 7.1.5)?

12. Sec. 7.8.5: if list key(s) inside "rpc" are defined after a
    non-key sibling, which rule takes precedence? Do the keys still
    have to come before all other siblings?

13. Sec. 7.12:
   OLD
   The identifiers defined in the grouping are not bound to a namespace
   until the contents of the grouping are added to the schema tree via a
   "uses" statement that does not appear inside a "grouping" statement,
   at which point they are bound to the namespace of the current module.
   NEW
   The identifiers defined or unqualified names used in the grouping
   are not bound to a namespace until the contents of the grouping are
   added to the schema tree via a "uses" statement that does not
   appear inside a "grouping" statement, at which point they are bound
   to the namespace of the current module.

14. Sec 8.1:
   OLD 
   o  If the constraint is defined on state data, it MUST be true in a
      reply to the <get> command.
   NEW
   o  If the constraint is defined on state data, it MUST be true in a
      reply to the <get> command with no filter.

15. Sec. 4.2.9:
    OLD
    YANG allows the definition of NETCONF RPCs.
    NEW
    YANG allows to define the schema (signature) of NETCONF RPCs.

Happy New Year!

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 05 Jan 2010 16:48:25 +0100 (CET)

Hi Ladislav,

Thank you for the detailed review!

> http://staff.cesnet.cz/~lhotka/NETMOD/draft-ietf-netmod-yang-09.diff

All applied.

>    not are unaffected by "when" (or "must").

My interpretation of that discussion is that there is no consensus to
change the current behavior.

>    expression refers to "class" which is a descendant of "user".

I suggest:

OLD:

NEW:

  The XPath expression MUST NOT include any references to nodes
  defined in the "when" statement's parent statement.

>    but this case is not mentioned in any of the bullets of

I suggest:

OLD:

   o  If the "when" statement is a child of a "choice" or "case"
      statement, then the context node is the closest ancestor node to
      the "choice" or "case" node which is also a data node.

NEW:

   o  If the "when" statement is a child of a "uses", "choice", or "case"
      statement, then the context node is the closest ancestor node to
      the "uses", "choice" or "case" node which is also a data node.

>      statement (see Section 7.1.3).

I don't like the phrase "most objects" - it raises the question which
objects we mean.

>      The name of this namespace is "urn:ietf:params:xml:ns:yang:

Fixed.

>      the module, e.g., "if:ifName".

I don't think the suggested text is an improvement.  As the original
text says, the prefix maps to both the module
(e.g. "inet:ipv4-address") and XML namespace (e.g. in must XPath
expressions).

>      MAY be qualified by using the prefix.

Same comment here - the prefix is used to access the module, not only
namespace.

>      is the module's XML namespace defined by the "namespace"

Ok.  I suggest this (adding a ref):

  A container node is encoded as an XML element.  The element's local
  name is the container's identifier, and its namespace name is the
  module's XML namespace (see Section 7.1.3).

>    cannot be addressed using the "value" attribute.

You are right.  The only way to change such a leaf-list is to do a
replace of all entries.

>    This would be a deviation from XPath which compares all numbers as

No, YANG does not deviate from XPath.  This is mentioned in the
guidelines document as well (that you must be careful...)   We had a
long email discussion about this a while back.

>    term should be explained in the Terminology section or avoided.

In 4.1:

OLD:

   YANG permits the definition of complex types using reusable grouping
   of nodes. 

NEW:

   YANG permits the definition of reusable grouping of nodes. 

I think the usage of this term in 4.2.6 is ok:

   Groups of nodes can be assembled into the equivalent of complex types
   using the "grouping" statement. 

> 6. The term "list key" should be included in Terminology.

The document doesn't use the term "list key", so I don't think we need
to define it.

>    is arbitrary (as it is in containers).

OLD:

   NETCONF is capable of carrying any XML content as the payload in the
   <config> and <data> elements.  The top-level nodes of YANG modules
   are encoded as child elements within these elements.

NEW:

   NETCONF is capable of carrying any XML content as the payload in the
   <config> and <data> elements.  The top-level nodes of YANG modules
   are encoded as child elements, in any order, within these elements.

>    YANG - inability to model XML attributes and recursive structures.

It already says:

   limiting
   the problem space to allow expression of NETCONF data models, not
   arbitrary XML documents

which I think is sufficient.

>    MUST be unique. It should also mention there is an IANA registry

The registry is mentioned in 7.1 and 7.2.

>     capabilities are to be interpreted according to sec. 5.6.4.

I don't think this is necessary - if the server advertises a URI which
the client understands, the client can check if the URI represents a
YANG module or not.

>     bullet list in sec. 7.1.5)?

Yes.

>     have to come before all other siblings?

Yes.  I think the text is clear:

   The list's key nodes are encoded as subelements to the list's
   identifier element, in the same order as they are defined within the
   key statement.

   The rest of the list's child nodes are encoded as subelements to the
   list element, after the keys.  If the list defines RPC input or
   output parameters, the subelements are encoded in the same order as
   they are defined within the list statement.

Would s/the subelements/these subelements/ make it more clear?

>    to the namespace of the current module.

Fixed.

>       reply to the <get> command with no filter.

Fixed.

>     YANG allows to define the schema (signature) of NETCONF RPCs.

I think the current text is ok.

/martin

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 5 Jan 2010 17:48:23 +0100

> >      The name of this namespace is "urn:ietf:params:xml:ns:yang:

Are you sure? I also first thought this is a typo but later I believed
the :1 was a version number, similar to the :1.0 we have for the
NETCONF URI (somehow we don't have a minor version in mind).

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 05 Jan 2010 19:28:03 +0100 (CET)

Juergen Schoenwaelder <[email-masked]> wrote:
> Are you sure?

Actually, Lada's original text got truncated.  The correct text should
be:

  OLD

The URI itself is the same.

> I also first thought this is a typo but later I believed

I hope we don't have YANG version 1, 1.1, 2, 1.2, 2.0.1 etc...

/martin

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 05 Jan 2010 19:28:52 +0100 (CET)

Phil Shafer <[email-masked]> wrote:
> Martin Bjorklund writes:
> I'm lost trying to connect this with the laissez-faire attitude
> you (plural) wanted WRT typos in XPath expressions.  You allow
> references to nonsense, but want to restrict references to "when"
> nodes?  Makes no sense.
> 
> This isn't acceptable, since the application now has to notice
> this condition and invoke a completely different behavior.  See
> my proposal in other email.

Which other email?  Could you send a pointer?

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 05 Jan 2010 21:49:43 +0100

> change the current behavior.

But I actually meant to stick strictly to the current text. It says,
e.g., in sec. 7.6.5: "the leaf MUST exist", full stop, i.e., without
regard to any "when" or "must". Thus having the leaf definition

leaf foo {
    mandatory true;
    when "1=0";
}

the leaf MUST exist (if the corresponding ancestor exists), otherwise
the above condition is clearly violated.

>   defined in the "when" statement's parent statement.

But what if there are two sibling augment statements, both referring via
'when' to each other's nodes?

>       the "uses", "choice" or "case" node which is also a data node.

OK.

> I don't like the phrase "most objects" - it raises the question which

The old text seems to imply that all names within a module belong to the
module's namespace - which is not true.

...

> (e.g. "inet:ipv4-address") and XML namespace (e.g. in must XPath

I find this "overloading" slightly confusing.

...

>   module's XML namespace (see Section 7.1.3).

Fine.

...

> long email discussion about this a while back.

So the note in sec. 7.5.3 just says that all values are "canonized"
before they are used in XPath expressions, not just comparisons. For
numbers it's not relevant anyway.

>    using the "grouping" statement. 

If XSD complex types are meant here, I don't think groupings are really
equivalent to them.

> The document doesn't use the term "list key", so I don't think we need

OK, so how about "key leaf"?

>    are encoded as child elements, in any order, within these elements.

OK.

>    the problem space to allow expression of NETCONF data models, not

I guess NETCONF doesn't forbid attributes or recursive content models,
does it?

> The registry is mentioned in 7.1 and 7.2.

Sure, but I thought it could be mentioned here, too.

> the client understands, the client can check if the URI represents a

How can you check this? The URI needn't point to any resource.

> >     bullet list in sec. 7.1.5)?

Then the text should say so, because "submodule's definitions" may mean
different things.

>    they are defined within the list statement.

But we have two conflicting rules here:

rpc my-rpc {
    input {
        list foo {
            key clef;
            leaf bar { ... }
            leaf clef { ... }
        }
    }
    ...
}

The rule for list keys implies the order "clef bar" but the rule for RPC
implies "bar clef".

> Would s/the subelements/these subelements/ make it more clear?

...

> I think the current text is ok.

By "definition of NETCONF RPCs" I understand also the definition of
their semantics, which YANG doesn't address.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Andy Bierman <[email-masked]>
Date: Tue, 05 Jan 2010 12:56:33 -0800

> the above condition is clearly violated.

No -- we need to make it clear that there is a 'property
precedence' defined for YANG, just like an 'operator precedence'
in a programming language.

The when-stmt has precedence over the mandatory-stmt and
min-elements-stmt.  If it is false, then the schema node does not
conceptually exist.  How the validation code deals with
this is an implementation detail.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 05 Jan 2010 22:27:33 +0100

> this is an implementation detail.

But I strongly object exactly to stating this kind of precedence. One
then could not tell whether the schema node exists or not before
interpreting the data that the schema is supposed to describe.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 05 Jan 2010 22:48:49 +0100 (CET)

> references to nonsense, but want to restrict references to "when"

No you are right.  A better fix, for consistency, is to remove this
sentence.

/martin

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 05 Jan 2010 22:51:56 +0100 (CET)

Phil Shafer <[email-masked]> wrote:
> Attached.  My response was:

To the chairs:  it seems emails from Phil today don't make it to the
list.  This fact makes the conversation a bit difficult to follow...

I suspect they are classified as spam, and are held in the moderation
queue.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Tue, 05 Jan 2010 14:03:54 -0800

> interpreting the data that the schema is supposed to describe.

yes, so what's your point?
It may be hard to implement, but it is the intent of
the when-stmt.

If not, then uses-when and augment-when become very
restricted features, and they are very important.
It is very likely that nodes in a grouping
will have constraints related to members of the grouping.
We have also wanted a SPARSE-AUGMENTS in MIBs for 15 years,
so the use-case for augment-when seems quite string.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 05 Jan 2010 23:13:50 +0100 (CET)

> the above condition is clearly violated.

I understand that, but I think you are the only one objecting to the
current text, which says:

  "must", "mandatory", "min-elements", and "max-elements" constraints
  are not enforced if the parent node has a "when" or "if-feature"
  property that is not satisfied on the current device.

> The rule for list keys implies the order "clef bar" but the rule for RPC

No, the text says:

    The rest of the list's child nodes are encoded as subelements to
    the list element, after the keys.

There are no exceptions to this rule in the text.  All non-key child
nodes are encoded after the key nodes.

> their semantics, which YANG doesn't address.

The semantics is defined in the description clause.

/martin

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 05 Jan 2010 23:14:57 +0100 (CET)

> If XSD complex types are meant here, I don't think groupings are really

Ok.  Do you think we can say "structured type" instead?  Or should we
simply remove the sentence?

/martin

---

From: Phil Shafer <[email-masked]>
Date: Tue, 5 Jan 2010 18:51:47 -0500

>I suspect they are classified as spam, and are held in the moderation

My bad.  I upgraded, fixed /etc/mail/*.mc, but did "make" instead
of "make install".  Fixed now, and I'll resend the two emails
shortly.

Thanks,
 Phil

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 06 Jan 2010 08:41:22 +0100

> simply remove the sentence?

How about "reusable sets"? Only leafs have types in YANG.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 06 Jan 2010 08:56:00 +0100

>   property that is not satisfied on the current device.

Note that no parent node is involved in my example. On the other hand,
the above sentence also doesn't apply to

container outer {
    when "...";
    container inner {
        leaf foo {
            mandatory true;
            ...
        }
    }
}

> nodes are encoded after the key nodes.

Then I propose this:

OLD
   If the list defines RPC input or output parameters, the   
   subelements are encoded in the same order as they are defined within 
   the list statement.
NEW
   If the list defines RPC input or output parameters, the   
   subelements are encoded in the same order as they are defined within 
   the list statement, with the exception of key leafs that must 
   always come first.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 06 Jan 2010 09:30:33 +0100

> It may be hard to implement, but it is the intent of

So you are essentially saying that real men are able to parse XML
without having a schema. It may be true for simple examples, but with
all the flexibility that YANG offers, a strict schema is IMO a necessity
and first line of defense against attacks.

> so the use-case for augment-when seems quite string.

I think it is still possible to accomplish this even if "mandatory" is
immune to the effects of "when" (and "must", for that matter). The
complication in YANG, which is not present e.g. in RELAX NG, is that
"mandatory" in leafs permeates up to the ancestor NP containers.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Phil Shafer <[email-masked]>
Date: Wed, 6 Jan 2010 13:56:20 -0500

>How about "reusable sets"? Only leafs have types in YANG.

"reusable collections"?

Thanks,
 Phil

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 07 Jan 2010 07:50:57 +0100

> >How about "reusable sets"? Only leafs have types in YANG.

+1. Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 07 Jan 2010 10:36:56 +0100 (CET)

OLD:

NEW:

Groups of nodes can be assembled into resuable collections
using the "grouping" statement.

/martin