Subject: Re: [netmod] Working Group Last Call: Mapping YANG to Document Schema Definition Languages and Validating NETCONF Content

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 09 Apr 2010 10:44:40 +0200

Hi,

this WGLC is over but I haven't received any comments (yet). I also
asked folks on the dsdl-comment mailing list for reviewing the document,
and specifically Jirka Kosek, who is a renowned XML guru and already
provided some input. I hope to get at least some comments from them, but
these will certainly be oriented on the XML & DSDL side of the work.

So what shall we do now? I have a few pending (non-critical) fixes that
should go into the next revision of the draft - does it make sense to do
it anytime soon?

Thanks, Lada

David Partain píše v Út 16. 03. 2010 v 11:08 +0100:
> Greetings,
> 
> This is a 21-day (note!  3 weeks) working group last call on the current 
> working group document:
> 
> - Mapping YANG to Document Schema Definition Languages and Validating NETCONF 
> Content
> - http://www.ietf.org/id/draft-ietf-netmod-dsdl-map-05.txt
> 
> This WGLC will end on 2010-04-06.  Please review the document and post 
> your comments on the mailing list.
> 
> Please use an appropriate Subject line so that the chairs and editor can 
> easily keep track of the issues that need to be dealt with.
> 
> Thanks.
> 
> David^2
> 
> (ps. all docs now in WGLC or IETF LC!)
> _______________________________________________
> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 13 Apr 2010 23:53:23 +0200 (CEST)

Ladislav Lhotka <[email-masked]> wrote:
> Hi,
> 
> this WGLC is over but I haven't received any comments (yet).

Sorry for the late reply, but here are my comments on this document.

------------------------------------------------------------

Section 1.

    accompanying rules for how to model configuration and status
 
s/status/state/
 
    information (in XML syntax) carried by NETCONF.  The IETF Operations
 
s/(in XML syntax)//
 
------------------------------------------------------------

Section 2.

I think you should list all the terms you import, with references.  No
need to copy & paste their meaning though.  For example, if I didn't
know that the term "information set" is defined in [XML] -
checking... - hmm, it wasn't defined there... I think you also need a
reference to xml-infoset.
 
------------------------------------------------------------

Section 2.

    "nmf"  NETMOD-specific XPath extension functions (see Section 12.6);
  
Bad reference to 12.6?   Should you have a separate section where you
define the NETMOD-specific XPath extensions?

------------------------------------------------------------

Section 2.1.

     o  ancestor datatype: Any datytype a given datatype is (transitively)
        derived from.
  
s/datytype/datatype/

------------------------------------------------------------

Section 2.1.

     o  conceptual data tree: A virtual XML document integrating all
        components of a YANG data model, i.e., configuration datastore,
       RPC methods (both requests and replies) and notifications.
 
s/RPC method/RPC operation/g  -- note the 'g'

and in some cases s/RPC/RPC operation/g
 
You use the terms "conceptual data tree", "conceptual tree schema" and
"conceptual tree" in many places in the document.  Are they the same?
If so, stick to one, if not, please defined the other meanings.

------------------------------------------------------------

Section 2.1.

       The conceptual tree is normally not instantiated, it only serves as a
       conceptual target for its schema.

I do not understand this.  The definition seems recursive.

------------------------------------------------------------

Section 3.

     While the mapping from YANG to DSDL described in this document is in
     principle invertible, the inverse mapping from DSDL to YANG is not in
     its scope.

AFAICT the mapping is not 100% invertible.  A simple (stupid) example
is

   description "See: foo"; 
and
   reference "foo";

which are both mapped to <a:documention>See foo</a:documentation>

So I wonder what the sentence about invertible mapping really says.
Is the intention that a YANG to DSDL mapping *could* in principle be
invertible, but this one isn't b/c it is out of scope?

------------------------------------------------------------

Section 3.

   To this end, the complexity of the mapping is
   distributed into two steps:

and then you list 3 steps 1-3...?

------------------------------------------------------------

Section 7.

   1.  The XML instance document can be immediately checked for
       grammatical and data type validity using the RELAX NG schema.

What does "immediately checked" mean?

What does "the RELAX NG schema" refer to?  Is it the output from the
first step or the second step?  I suggest you use two distinct terms
for these schemas.

------------------------------------------------------------

Section 8.1

     <nmt:top>
       ... configuration and status data ...

s/status/state/

------------------------------------------------------------

Section 8.1

     </nmt:top>
     <nmt:rpc-methods>
       <nmt:rpc-method>

Should these be nmt:rpcs and nmt:rpc, to align with YANG?

------------------------------------------------------------

Section 8.1.

   In general, these
   transformations should be relatively simple - they will typically

s/should be/are/  ?

------------------------------------------------------------

Section 9.

   The
   output RELAX NG schema will then contain only named pattern
   definitions translated from YANG groupings and typedefs.

This is somewhat confusing.  So far, I thought that the conceptual
tree also contained typedefs and groupings.  But this text tells me
that is not the case.  I think the handling of typedefs and groupings
should be mentioned in 8.1 where it says:

   The schema for the conceptual tree - a single RELAX NG schema with
   annotations - therefore has a quite similar logic as the input YANG
   module(s), the only major difference being the RELAX NG schema
   language.

------------------------------------------------------------

Section 9.2.

s/underline/underscore/

------------------------------------------------------------

Section 9.3.

   Translation rule 2 means for example that absolute XPath expressions
   appearing in the main configuration data tree always start with "nmt:
   netmod-tree/nmt:top/", those appearing in a notification always start
   with "nmt:netmod-tree/nmt:notifications/nmt:notification/", etc.

Should this be "/nmt:netmod-tree/..." ?

------------------------------------------------------------

Section 10.

I cannot see where the conceptual tree is actually defined.  There is
one example in 8.1 which shows the netmod-tree top-level container and
its children, but no specification.

Hmm, section 11 does this for substatements, but it doesn't define the
top-level structure.  Shouldn't chapter 11 go before this chapter?

------------------------------------------------------------

Section 10.

   multiple types) that is to be validated.  These document type can be,

s/type can/types can/  or s/These/the/

------------------------------------------------------------

Section 10.2.

   The value of the mandatory @context attribute of <sch:rule> (denoted
   as XELEM) MUST be set to the absolute path of the context element in
   the data tree.

Is this term "data tree" really "conceptual data tree", or did you
mean "data tree" as defined in YANG?  (same for other occurances of
this term).

------------------------------------------------------------

Section 10.2.1.

 For
   example, a candidate configuration referring to configuration
   parameters or state data of certain hardware will not pass full
   validation before the hardware is installed.

This is a somewhat controversial example.  config data cannot refer to
state data.  Pick a simpler example, e.g. all leafreaf referential
intergrity tests have to be true in a candidate configuration.

------------------------------------------------------------

Section 10.4.

   <dsrl:element-map>
     <dsrl:parent>XPARENT</dsrl:parent>
     <dsrl:name>ELEM</dsrl:name>
     <dsrl:default-content>DEFCONT</dsrl:default-content>
     </dsrl:element-map>

Picky: indentation.

------------------------------------------------------------

Section 10.4.

   The <rng:element>, <rng:group> or <rng:interleave>patterns appearing

Picky: s/>p/> p/

I do not know DSDL enough to have any comments on the contents of
chapter 11 and 12.

I have not reviewed the Appendices.

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 14 Apr 2010 11:04:04 +0200

Hi Martin,

thanks for the review, please see comments below.

Lada

Fixed.

>     information (in XML syntax) carried by NETCONF.  The IETF Operations

Removed.

I'll try to list all terms that are not generally known - you probably
don't mean terms like "XML element" here.

Actually, the reference to 12.6 is all right - that section describes
the only XPath extension used in the draft - nmf:evaluate(). I am not
sure about the usefulness of a separate section, if there is only one
function so far.

Fixed.

Changed all to "RPC operation". I assume that "RPC request" and "RPC
reply" is OK.

> If so, stick to one, if not, please defined the other meanings.

"conceptual data tree" == "conceptual tree"

I agree that this terminology is rather confusing, also because the
terms "data tree" and "conceptual" are used in the YANG draft. The idea
is as follows:

Conceptual tree is an XML instance document such as the one shown in
sec. 8.1., and conceptual tree schema is the annotated RELAX NG schema
for this instance document.

However, the conceptual tree (instance) is not interesting by itself. In
fact, the only role of the extra elements with "nmt" prefix is to
produce a RELAX NG pattern that demarcates the corresponding part in the
conceptual tree schema. For example, to extract the schema for a
datastore from the conceptual tree schema, one has to take the subtree
of <rng:element name="nmt:top">.

As an alternative, I was considering to use special markers directly in
the RELAX NG schema for combining the schemas for the datastore, RPCs
and notifications in one document. The advantage of the present approach
is that the conceptual tree schema as a whole is still a valid RELAX NG
schema.

I'd appreciate any ideas how to make this setup more straightforward and
the description less complex. 

See above.

Well, coming back to the original YANG module is of course impossible
(some groupings are expanded by the mapping, augments are applied etc.).
One could possibly think about producing a YANG module that is
equivalent in the sense that the data model is the same.

But I guess it's appropriate to remove this remark completely because
the inverse mapping is really out of scope and nobody is interested in
it (anymore).

I can only see two steps numbered 1 and 2, what do you mean?

I changed it to "The instance document is checked for ..."

> first step or the second step?  I suggest you use two distinct terms

Right, one RELAX NG schema is the "conceptual tree schema", which is the
result of the first mapping step, and the second step then produces a
RELAX NG schema for a concrete target instance document. I have to make
sure this distinction is clear.

Fixed.

OK, I can change it as you suggest.

The simplicity of course also depends on the tools used for the task. So
how about "The goal was to make these transformations relatively
straightforward - ..."?

No the conceptual tree (the instance) only contains elements coming from
groupings that are used in the input YANG modules. For modules like
ietf-inet-types, the conceptual tree is essentially empty - only
contains the top-level "nmt:*" elements.

>    module(s), the only major difference being the RELAX NG schema

The conceptual tree _schema_ does contain the definitions that are
really used.

Fixed.

Yes, but in fact the rule 2 also looks like an unnecessary complication.
In the second step, the top-level parts of the XPath expressions have to
be adjusted anyway to reflect the target instance document type - for
instance, they would become "/nc:rpc-reply/nc:data".

In the meantime, I also realized that for XPath expressions appearing
inside groupings the first part is actually variable as the same
grouping can be used both in the data and in an RPC. So I have to take
these cases into account.

The description is in the second paragraph of sec. 8.1, although
probably insufficient - see above.

> top-level structure.  Shouldn't chapter 11 go before this chapter?

Sections 6-8 try to explain the big picture whereas sec. 11 gives a
detailed specification of how the individual YANG statements are mapped.
I think this order is essentially correct.

"These document types can ..."

None of them, probably "target instance document" is appropriate here.
In the following example, the context value is
"/nc:rpc-reply/nc:data/dhcp:dhcp/dhcp:default-lease-time". How would you
call it?

OK. Actually, YANG defines validity only as full validity including the
referential integrity, so I am thinking about removing this phases stuff
from Schematron as well. Any opinions?

Fixed.

Fixed.

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 16 Apr 2010 10:37:30 +0200 (CEST)

I Lada,

I think your changes are fine.  See below for additional comments.

> for this instance document.

Ok, this is fine.  

> the description less complex. 

It is ok, just make sure you define both these terms in the
terminology section, and taht you use these terms
consistently. (I.e. remove the term conceptual data tree).

> I can only see two steps numbered 1 and 2, what do you mean?

Sorry, I messed up my notes.  This is in section 6:

   The mapping procedure is divided into two steps:

... and then you list 3 steps.

> how about "The goal was to make these transformations relatively

Ok.

> probably insufficient - see above.

I think it is insufficient.  The second paragraph describes in general
terms what it is, but the structure is not defined.  The structure
shows up in the example in paragraph three.  This is also related to
the next comment.  If section 8 is just for overview purposes, the
conceptual tree schema needs to be defined in 9-11.

> I think this order is essentially correct.

6-8 is fine.

The order of 9-11 does not seem natural:

   9. Mapping YANG Data Models to the Conceptual Tree Schema
  10. Mapping Conceptual Tree Schema to DSDL
  11. Mapping YANG Statements to Conceptual Tree Schema

It seems as 9 and 11 belong together; both detail how to map from YANG
to Conceptual Tree Schema.  The next step is ho to map to DSDL;
shouldn't that be described after 9 and 11?

> "/nc:rpc-reply/nc:data/dhcp:dhcp/dhcp:default-lease-time". How would you

This is a pretty important piece of the whole YANG->DSDL puzzle, so it
should have its own term.  "target instance document" is fine.   For
example, in section 6 you say:

   2.  In the second step, the conceptual tree schema is transformed in
       multiple ways to a coordinated set of DSDL schemas that can be
       used for validating a particular data object in a specific
       context.

> from Schematron as well. Any opinions?

There is one constraint which applies to all data trees / target
instance docuements, and that is "key".  All the others can be done in
one single phase.

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 16 Apr 2010 13:11:17 +0200

> consistently. (I.e. remove the term conceptual data tree).

Yup, I have to make up a replacement for this term, thinking about
"hybrid schema".

> Sorry, I messed up my notes.  This is in section 6:

OK, I pulled item #3 out of the list and made it a separate paragraph.

> conceptual tree schema needs to be defined in 9-11.

I'll try to improve it.

> shouldn't that be described after 9 and 11?

OK, I moved current chapter 10 after 11.

>        used for validating a particular data object in a specific

Right, all this has to use uniform term, so let's use "target instance
document".

> instance docuements, and that is "key".  All the others can be done in

Hmm, I am not sure we are talking about the same thing. In -05,
Schematron schemas define two phases - "full" and "noref" - one of them
has to be selected a priori for performing validation. The only
difference between them is that the latter doesn't check referential
integrity constraints related to leafref and instance-identifier types.
My proposal is to use only "full" as it is the only concept of validity
defined in YANG. Is "key" then anything special in this context? I don't
think so.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 16 Apr 2010 14:01:01 +0200 (CEST)

> defined in YANG. Is "key" then anything special in this context? I don't

If I understand this correctly, in order to check that keys are
unique, I must run Schematron on the target instance document.  If
there is only one phase, it will check not only keys, but all other
constraints as well.  But if the target is candidate, I may want to
check keys, but not the others.

/martin