Subject: Re: [netmod] review of yang-09

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 05 Jan 2010 22:18:49 +0100

Phil Shafer píše v Út 05. 01. 2010 v 12:58 -0500:
> >
> >   - sec. 5.3.1:
> >     OLD
> >     This namespace is "urn:ietf:params:xml:ns:yang:1".
> >     NEW
> >     The name of this namespace is "urn:ietf:params:xml:ns:yang:1".
> 
> -1 to these two.  I do not see any additional clarity from the changes.

The URN is not the namespace. "Namespaces in XML 1.0 (Third Edition)"
has this definition in sec. 2.1: "For a name N in a namespace identified
by a URI I, the namespace name is I."

> 
> >   - sec. 7.1.4
> >     OLD
> >     The "prefix" statement is used to define the prefix associated
> >     with the module and its namespace.  The "prefix" statement's
> >     argument is the prefix string which is used as a prefix to access
> >     a module.  The prefix string MAY be used to refer to definitions
> >     contained in the module, e.g., "if:ifName".  A prefix follows the
> >     same rules as an identifier (see Section 6.2).
> >
> >     When used inside the "module" statement, the "prefix" statement
> >     defines the prefix to be used when this module is imported.  To
> >     improve readability of the NETCONF XML, a NETCONF client or
> >     server which generates XML or XPath that use prefixes, the prefix
> >     defined by a module SHOULD be used, unless there is a conflict.
> >
> >     NEW
> >     The "prefix" statement is used to define the namespace prefix
> >     [XML-NAMES] that is bound to the namespace URI of a module. A
> >     prefix follows the same rules as an identifier (see Section 6.2).
> >
> >     When used inside the "module" statement, the "prefix" statement
> >     defines the prefix to be used when this module is imported.  To
> >     improve readability of the NETCONF XML, a NETCONF client or
> >     server which generates XML or XPath that use prefixes, SHOULD use
> >     the prefix defined by a module, unless there is a conflict.  The
> >     prefix MAY be used to refer to definitions contained in
> >     the module, e.g., "if:ifName".
> 
> -1 also. "namespace prefix" is not a better term than "prefix" here.
> I think the intent is clear from the current text.  The term "namespace
> name" just sounds silly.

It is a terminus technicus used in the W3C recommendation cited above.
 
> 
> >3. The normalization of XML attributes may affect not only arguments
> >   in YIN but also the values of the "value" attribute in sec. 7.7.7.
> >   For example, a leaf-list item containing a tab character ("\t")
> >   cannot be addressed using the "value" attribute.
> 
> This is yet another case where one must ask "what were they thinking".
> The normalization of "value" and "key" means we need a new mechanism,
> since we cannot make tabs illegal in identifiers.

It's not only about identifiers, the "yang:value" attribute contains
leaf-list values.

> 
> The simplest change would be to have a "yang" namespace with "insert",
> "key", and "value" as elements of that namespace.  Then we'd require
> that these elements appear first under the parent element.  This would
> allow a simpler solution for multiple keys:
> 
>     <prefix>
>        <yang:insert>before</yang:insert>
>        <yang:keys>
>           <name>10.1.2.0/24</name>
>           <type>or-longer</type>
>        </yang:keys>
>        <name>10.1.1.0/24</name>
>        <type>exact</type>
>     </prefix>
> 
> (assuming "list prefix" is "key 'name type'").

Where would the edit-config "operation" attribute be?
 
> 
> Is this change acceptable?
> 

...

> >10. Maybe the server should also explicitly advertize that it uses
> >    YANG? For one thing, this would inform the client that the other
> >    capabilities are to be interpreted according to sec. 5.6.4.
> 
> Avertising a YANG based data model is sufficient, since there is
> nothing YANG-ish to support outside of a specific data model.  In
> fact, I could suppport 3 YANG models and 3 non-YANG models in the
> same box.  No YANG-specific operations can be done on the non-YANG
> models.
> 
> >12. Sec. 7.8.5: if list key(s) inside "rpc" are defined after a
> >    non-key sibling, which rule takes precedence? Do the keys still
> >    have to come before all other siblings?
> 
> I don't follow the question.  Please expand on it a bit.

I gave an example in my previous reply to Martin's message.

> 
> >13. Sec. 7.12:
> >   OLD
> >   The identifiers defined in the grouping are not bound to a namespace
> >   until the contents of the grouping are added to the schema tree via a
> >   "uses" statement that does not appear inside a "grouping" statement,
> >   at which point they are bound to the namespace of the current module.
> >   NEW
> >   The identifiers defined or unqualified names used in the grouping
> >   are not bound to a namespace until the contents of the grouping are
> >   added to the schema tree via a "uses" statement that does not
> >   appear inside a "grouping" statement, at which point they are bound
> >   to the namespace of the current module.
> 
> This is not true.  Unqualified names used in a grouping are resolved
> in the place where the grouping is defined, not where it is used.

grouping ggg {
     leaf foo {
        type uint8;
     }
     leaf bar {
        type uint8;
	must ". < ../foo";
     }

I think we agreed that both the definition and use of "foo" get the
namespace of the module where the grouping is used.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 05 Jan 2010 23:09:17 +0100 (CET)

> by a URI I, the namespace name is I."

I agree with Lada, we should follow the W3C terminology.

> It's not only about identifiers, the "yang:value" attribute contains

Right.

> Where would the edit-config "operation" attribute be?

Just as with the attribute solution - in the <prefix> element.

> namespace of the module where the grouping is used.

We did, and that's specified in 6.4.1 *for XPath expressions*.

However, unqualified type references (etc) are resolved where the
grouping is defined, as described in 7.11:

   Identifiers appearing inside the grouping are resolved relative to
   the scope in which the grouping is defined, not where it is used.

So the original text, which did not mention the usage of names, is
actually correct, and the text already specifies how unqualified
names are resolved.

/martin

---

From: Phil Shafer <[email-masked]>
Date: Tue, 5 Jan 2010 19:00:53 -0500

>I agree with Lada, we should follow the W3C terminology.

The new terminology is verbose and IMHO does more harm
than good.  Are we willing to s/prefix/namespace prefix/
and s/element name/element local name/ (or whatever the
new term is) throughout the entire YANG spec?

Thanks,
 Phil

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 06 Jan 2010 08:37:52 +0100

> Just as with the attribute solution - in the <prefix> element.

Hmm, this is not very nice, it would then be much better to have the
NETCONF operation as a subelement, too, and encapsulate the additional
info inside it.

>    the scope in which the grouping is defined, not where it is used.

Does this mean that "foo" inside the XPath expression in my example is
bound to the namespace of the module where "ggg" is defined? I hope not.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Andy Bierman <[email-masked]>
Date: Wed, 06 Jan 2010 01:09:48 -0800

> NETCONF operation as a subelement, too, and encapsulate the additional

I am strongly opposed to changing the insert operation
at this late date.  If YANG needs to redo the NETCONF protocol
this drastically, then the entire operation should be dropped,
and perhaps put into the NETCONF protocol some time in the future.

I don't really understand why the YANG data modeling language
needs to define new NETCONF protocol operations in the first place.
It seems like the NETCONF WG should be defining an 'insert'
operation if it was that important.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 06 Jan 2010 10:17:42 +0100 (CET)

> new term is) throughout the entire YANG spec?

I think the term "local name" is an improvement.  "namespace name" not
so much.

 - Is that Phil over there?
 - No no, it's the person with the name Phil.

So how about this (in 7 places):

OLD:

   A leaf node is encoded as an XML element.  The element's name is the
   leaf's identifier, and its XML namespace is the module's XML
   namespace.

NEW:

   A leaf node is encoded as an XML element.  The element's local name
   is the leaf's identifier, and its namespace is the module's XML
   namespace (see Section 7.1.3).

/martin

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 06 Jan 2010 10:29:39 +0100 (CET)

Andy Bierman <[email-masked]> wrote:
> at this late date.

If we have identified a problem, shouldn't we try to fix it?

One problem with the proposed solution is that it is awkward for
leaf-lists:

     <allow-user><yang:insert>before</yang:insert><yang:value>alice</yang:value>bob</allow-user>

Note that this is not the same:

     <allow-user>
       <yang:insert>before</yang:insert>
       <yang:value>alice</yang:value>
       bob
     </allow-user>

> If YANG needs to redo the NETCONF protocol

YANG needs to define the insert operation because it is tightly
coupled to the data model.  It is not a generic NETCONF operation.  It
can only be used for ordered-by user lists.  Other data modelling
languages may introduce their own special operations to handle their
special needs.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Wed, 06 Jan 2010 01:37:08 -0800

> If we have identified a problem, shouldn't we try to fix it?

We could remove the insert operation entirely,
and add it to the protocol in a proper form.

I do not agree that this is a significant problem.
The use of whitespace in identifiers is extremely rare.
Even if that happens, the entire list can be replaced,
and the insert operation not used at all.

> languages may introduce their own special operations to handle their

I do not buy this argument.
There is nothing YANG specific about
an insert operation.  The concept of multiple
elements, and the need to control the order of instances
of these elements, is not specific to YANG.

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 6 Jan 2010 10:46:50 +0100

> of these elements, is not specific to YANG.

I tend to agree with Andy that technically a better solution would
be a generic "improvement" of the edit-config operation. Procedurally,
NETCONF is not chartered to do NETCONF 1.1 and this makes it somewhat
problematic to implement this "improvement" in NETCONF soon...

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: Andy Bierman <[email-masked]>
Date: Wed, 06 Jan 2010 05:50:44 -0800

> problematic to implement this "improvement" in NETCONF soon...

I do not think the problem statement is very compelling,
and also the proposed solution (AFAIK w/o any real details)
is complicated and a massive change to the spec that may
have ripple effects -- it certainly would need another 2
months to write and review such a large change.

1) the insert operation is already redundant.
    One could use a replace operation.  It causes more
    bytes of XML to be transfered, but so what?

2) The 'problem' with attributes here is that if list keys
   or leaf-list nodes differ in value only in the
   consecutive whitespace characters, such that when the
   whitespace is normalized, the strings are the same.
   IMO this is not a real problem and even if it was,
   there is a workaround to the insert operation already.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Wed, 06 Jan 2010 06:18:22 -0800

> problematic to implement this "improvement" in NETCONF soon...

quick question -- why can't the client use character entities
in the key/value attributes to represent whitespace that needs
to exactly match the whitespace in the database?

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 06 Jan 2010 15:24:11 +0100

>     bytes of XML to be transfered, but so what?

The problem of "yang:value" normalization is the same for replace
operation.

>    there is a workaround to the insert operation already.

The normalization also affects character and entity references.

Lada

> 
> > /js
> > 
> 
> Andy
> _______________________________________________
> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 06 Jan 2010 15:35:01 +0100

> to exactly match the whitespace in the database?

Yes, this could actually work, if the client is careful enough - even
with the tab character. So maybe it's not a real issue after all.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Andy Bierman <[email-masked]>
Date: Wed, 06 Jan 2010 07:21:26 -0800

> with the tab character. So maybe it's not a real issue after all.

The client already has to make sure it replaces " with &quot;
when encoding an attribute, so why not replace space and tab also?

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 07 Jan 2010 10:35:25 +0100 (CET)

Ladislav Lhotka <[email-masked]> wrote:
> Yes, this could actually work, if the client is careful enough

Which, as it turns out, e.g. the function quoteattr in
xml.sax.saxutils in Python does.

> - even with the tab character. So maybe it's not a real issue after all.

Good!

So, maybe we should revert the pending changes to YIN as well -- there
is no reason to make elements for other arguments than the 'text'
arguments (description, reference, organization, contact,
error-message).

/martin

---

From: Andy Bierman <[email-masked]>
Date: Mon, 11 Jan 2010 10:21:18 -0800

> xml.sax.saxutils in Python does.

Let's nail down the exact text...

   >      &gt;
   <      &lt;
   "      &quot;
   &      &amp;
   \t     &#9;

   Does ' ' get converted to &nbsp; or &ensp; or &emsp; or &#32;

   Does \n get converted to &#10; or left alone?

   Do all other 'isspace(c)==true' characters get converted to &#num-c;

   Or do all other 'isprint(c)==false' characters get converted to &#num-c;

> arguments (description, reference, organization, contact,

put a stake in the ground and leave it there

Andy

---

From: Phil Shafer <[email-masked]>
Date: Mon, 11 Jan 2010 13:21:19 -0500

>   Does ' ' get converted to &nbsp; or &ensp; or &emsp; or &#32;

&nbsp; is not ' '.

>   Does \n get converted to &#10; or left alone?

Can't leave it alone.

>   Or do all other 'isprint(c)==false' characters get converted to &#num-c;

XML can't encode control characters, but we're DOA for those.  Hmmmm....

Perhaps we just need to define "well known" entities for these?  &c01;
&c02; &c03; &c04; etc.  Not being about to encode strings containing
control characters is a problem that I'd like to see solved.

I say this assuming that we don't need to explicitly declare these
entities.  32 lines of <!ENTITY...> would stink.

>put a stake in the ground and leave it there

We did.  It moved.  Let's move it back.  Please revert those changes
and let's limit ourselves to more serious issues.

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Mon, 11 Jan 2010 11:08:59 -0800

>>   Does ' ' get converted to &nbsp; or &ensp; or &emsp; or &#32;

OK -- so &#32; is the answer.

>>   Does \n get converted to &#10; or left alone?

OK

> control characters is a problem that I'd like to see solved.

Why not just encode all unprintable chars as &#num;

This seems to apply to element content as well.

> and let's limit ourselves to more serious issues.

the chart had a bug in it (which I found).
Arbitrary choices tend to produce more debate than normal.

1 more XML question about whitespace...

Is an XML parser expected to tidy up element whitespace
(beyond leading and trailing whitespace).
Let's say some leaf foo has some newlines in it:

Hello world
Hello YANG world
Hello

What if the XML is indented for easier reading:

  <rpc-reply>
     <data>
        <foo>
           Hello world
           Hello YANG world
           Hello
        </foo>
     </data>
  </rpc-reply>

Does this alter the content as follows:

Hello world
           Hello YANG world
           Hello

OR

           Hello world
           Hello YANG world
           Hello

Does the YANG text adjustment for double-quoted strings apply here?
If not, should it?

How important is it to preserve whitespace in YANG values?
Is this a server-specified behavior or standard?  Fixed or configurable?

Andy

---

From: Phil Shafer <[email-masked]>
Date: Mon, 11 Jan 2010 14:04:05 -0500

>Why not just encode all unprintable chars as &#num;

Discussed in an earlier thread, but the answer is because
the XML authors made control characters illegal.   The
encoding does not matter; the content is still illegal:

Idle % cat test.xml
<top>
 <char>&#1;&#2;&#3;&#4;</char>
</top>
Idle % xmllint test.xml
test.xml:2: parser error : xmlParseCharRef: invalid xmlChar value 1
 <char>&#1;&#2;&#3;&#4;</char>
           ^
test.xml:2: parser error : xmlParseCharRef: invalid xmlChar value 2
 <char>&#1;&#2;&#3;&#4;</char>
               ^
test.xml:2: parser error : xmlParseCharRef: invalid xmlChar value 3
 <char>&#1;&#2;&#3;&#4;</char>
                   ^
test.xml:2: parser error : xmlParseCharRef: invalid xmlChar value 4
 <char>&#1;&#2;&#3;&#4;</char>
                       ^
Idle % 

>How important is it to preserve whitespace in YANG values?

<filename>a  b</filename> is different than <filename>a b</filename>.

>Is this a server-specified behavior or standard?  Fixed or configurable?

Data model specific.

Thanks,
 Phil