Subject: [netmod] refinements and augments

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 09 May 2008 10:45:45 +0200

Hi,

since "augment" with a relative path seems to make sense only in
connection with "uses", I am thinking about unifying refinements and
augments, like this:

uses "some-grouping" {
    refine foo/bar {
        ...
    }
    augment baz/mek {
        ...
    }
}

Any opinions?

Lada
    
-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

_______________________________________________
netmod mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/netmod

---

From: Phil Shafer <[email-masked]>
Date: Fri, 09 May 2008 09:15:25 -0400

>connection with "uses", I am thinking about unifying refinements and

I like that this allows you to move deeper into the hierarchy
using a path instead of repeating nodes.

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Fri, 09 May 2008 07:53:32 -0700

> using a path instead of repeating nodes.

I dislike it for the same reason.
IMO, the weakest aspect of YANG is that it can be very difficult
for a human to visualize or determine the actual nodes that will
be present (in XML) for protocol operations.  This is mostly due to the
uses and augment statements, and making them more complex worsens
the problem.

IMO, the reader is #1 priority.
Massive flexibility for the writer to be 'clever' is not
nearly as important as the reader understanding
everything in the data model well enough to use it in
the protocol.

Since this is a very subjective topic, there is no way
to really pick the right balance, so we go by WG consensus instead.

>  Phil

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 09 May 2008 13:57:25 +0200

Martin Bjorklund píše v Pá 09. 05. 2008 v 12:50 +0200:
> Hi,
> 
> Ladislav Lhotka <[email-masked]> wrote:
> > Hi,
> > 
> > connection with "uses",
> I don't think this is a good idea.  Why make this restriction?  Also,
> see Phil's email in the "list keys" thread
> (http://www.ietf.org/mail-archive/web/netmod/current/msg00084.html)
> for an example of a use case where augment is used outside of a
> grouping.  (Ok in this particular case there is a 'uses' just before
> the augment, but the use case is equally valid w/o the this uses)

Is there any reason for restricting "when" only to "augment"? IMO "when"
could be allowed as a general method for specifying conditional content.
So instead

augment . {
    when <condition> ;
      ....
}

one could write

when <condition> {
    ....
}

I think in this case "augment" would be really needed only as a modifier
of "uses" since otherwise you can write the new nodes straight to the
place where they belong.

Lada

> /martin
-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Phil Shafer <[email-masked]>
Date: Fri, 09 May 2008 11:12:56 -0400

>uses and augment statements, and making them more complex worsens

True, it's subjective, and one man's complexity is another's
simplicity, but we're comparing:

    uses something {
        container eh {
            list be {
                leaf sea {
                    some refinement;
                }
            }
        }
    }

against:

    uses something {
        refine eh/be/sea {
            some refinement;
        }
    }

IMHO this does three things: makes is more obvious that this
is in fact a refinement, makes the point of the refinement
more clear, and removes some depth between the uses and the
actual refinement.

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Fri, 09 May 2008 08:34:32 -0700

> more clear, and removes some depth between the uses and the

I think the first version is easier to read, especially
if you are not an Xpath expert.  I also get to see the type
of node-tree (container, list, leaf), which is just as
interesting as the node names.  Descendant-schema-nodeid
does not provide this feature.

What if there are lots of refine statements, that are
allowed to be out of order, and maybe even duplicate target Xpaths?
What if there are refinements to be made at multiple levels?
Then the natural XML containment is lost and it gets way harder
to read.

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Fri, 9 May 2008 18:34:55 +0200

On Fri, May 09, 2008 at 08:34:32AM -0700, Andy Bierman wrote:
 
> if you are not an Xpath expert.

This somehow reminds to the discussion about the need for subtree
filtering because (a subset of) Xpath is really too complicated for a
non Xpath expert...

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>
_______________________________________________
netmod mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/netmod

---

From: Andy Bierman <[email-masked]>
Date: Fri, 09 May 2008 10:02:06 -0700

Phil Shafer wrote:
> Juergen Schoenwaelder writes:
> Sure, but this isn't an xpath, since it's got no axis, predicates,
> functions, etc.  It's a yang-path, which is pretty simple.
> 

I'll repeat the paragraph, since it was quoted out of context:

Note sentence 2: IMO, the type of node is very useful to see
Note sentence 3: I understand this is not full Xpath

Andy

---

From: Phil Shafer <[email-masked]>
Date: Fri, 09 May 2008 12:43:19 -0400

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Fri, 09 May 2008 15:35:40 -0400

>I'll repeat the paragraph, since it was quoted out of context:

I think Juergen was quoting the interesting part, which is that you
think someone who isn't an XPath expert will be somehow confused
by a sequence of tokens separated by slashes.  Given that most
operating systems have some directory-like mechanism, and that even
computer illliterates are exposed to URLs, I find this a shocking
claim.

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Fri, 09 May 2008 13:02:40 -0700

> computer illliterates are exposed to URLs, I find this a shocking

I don't think I said that.
I think I said the current YANG syntax is more readable than
the proposed changes.

When there are multiple clauses of this sort, mixed together,
at arbitrary points in the tree, pointing to arbitrary target nodes,
this seems way less readable than the current syntax.  In trivial examples,
there is not much difference at all.

Most importantly with the current syntax, the type of node is plain,
the refinements are grouped together, and the extra burden of 1 redundant
curly brace is not a concern to me.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 09 May 2008 23:51:03 +0200

> I dislike it for the same reason.

Well, another option would to follow the refinement method and write the
tree hierarchy explicitly.

What is IMO more important is that augment is now in fact used for three
different functions:

1. Conditional content:

   augment . {
     when condition;
     ....
   }

This could be replaced by a simpler and clearer statement

    when condition {
      ....
    }

with exactly the same semantics. It could be allowed in the same places
as augment is now.

2. Use of a grouping (internal or imported) with modifications. This
could be done this way:

   uses some-grouping {
     <modifiers>
   }

where <modifiers> include refinement of the existing nodes and/or adding
new nodes. For a reader it is IMO better to explicitly see what grouping
the modifiers apply to. Whether the modifiers are expressed using a path
or an explicit tree is secondary.

3. Export of own nodes to a foreign data model - now augment with an
absolute path. I think a new statement is appropriate here, perhaps

   export foreign-module {
     ....
   }

or something similar.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Sat, 10 May 2008 07:41:38 +0200

> absolute path. I think a new statement is appropriate here, perhaps

This really is the core of an augment and should be called augment;
export is a confusing name since we are not exporting anything.

/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Sat, 10 May 2008 11:32:08 +0200

> export is a confusing name since we are not exporting anything.

Sure, it could be called "augment", if this keyword is not used for
specifying the modifiers in "uses". One way for doing it (without "yang
path") could be this:

uses some-grouping {
  container foo {
    leaf bar {
      default 6378;
    }
  container baz {
    leaf mek {
      ....
    }
  }
}

where foo/bar branch exists in some-grouping while baz/mek doesn't. So,
augmenting the grouping is handled in the same way as refinements here -
nodes that don't exist in the grouping are just added.

I would still prefer to specify the modifiers using "yang path" in some
way though. IMO the distinction between containers and leafs isn't that
crucial - after all, in XML encoding they are all elements.

Lada

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Sat, 10 May 2008 11:54:33 +0200

On Sat, May 10, 2008 at 11:32:08AM +0200, Ladislav Lhotka wrote:
> Juergen Schoenwaelder p????e v So 10. 05. 2008 v 07:41 +0200:

So do I (just in case that was not clear).

/js

---

From: Andy Bierman <[email-masked]>
Date: Sat, 10 May 2008 08:30:32 -0700

> crucial - after all, in XML encoding they are all elements.

I prefer to have YANG be a high-level DML, which has building blocks
called 'container', 'list', leaf', etc.  In the YANG module, I prefer
not to reduce the DM to just more XML.  There are already plenty
of DMLs that do that, and we don't need another one.

When the node type is always the same (e.g., directories or
XML elements), then the 'foo/bar/baz' syntax is best.
But when the node type is variable, then this syntax is not explicit enough.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 09 May 2008 12:50:37 +0200 (CEST)

Hi,

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Sat, 10 May 2008 18:50:24 +0200

> But when the node type is variable, then this syntax is not explicit enough.

It is explicitly recorded in the grouping that is being modified and
this grouping must be checked in any case in order to make of the uses
statement. And for augments, all parts of the path must be containers.
As Phil pointed out, the analogy with file system paths, where the type
of the last part is also variable, should help the users understand the
meaning.

So I agree with you that it's better to distinguish refinements and
augments of groupings syntactically, but I would use the path notation
for both.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Andy Bierman <[email-masked]>
Date: Sat, 10 May 2008 08:12:36 -0700

> nodes that don't exist in the grouping are just added.

My #1 concern is that the DML be resistant to human-introduced
errors.  This approach will silently generate a new sub-tree
if the identifier is mis-spelled.  Instead of modifying
container 'naz', the user fat-fingers 'baz', and the
compiler doesn't complain at all.

I prefer to keep uses refinements and local augments separate.
They are not the same thing in concept or syntax.

I prefer to make the writer (and reader) verify
that they are modifying the correct node, and also prefer
to let the reader see the type of nodes that are being modified.
This is easier to read and easier to track down these nodes
elsewhere (in whatever module has the grouping).

It is not important that the clever DM writer knows what
   'refine foo/bar/baz/taz/pizazz' means.

The 1000s of readers may not be so familiar with the data model
is this clever writer.  For them, seeing the node type may offer
more context as to what the clever writer is doing, or at least
some more hints what keywords to use in the search.

   container foo {
     choice bar {
       case baz {
         container taz {
           leaf pizazz {  .... }
         }
       }
     }
   }

Of course this is more verbose than 'foo/bar/baz/taz/pizazz',
but it is more useful to the reader.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Sat, 10 May 2008 10:17:27 -0700

Ladislav Lhotka wrote:
> statement.

The grouping is not present.  Just the uses-stmt.

> And for augments, all parts of the path must be containers.

Not true -- this is where YANG gets really messy.

The containment tree can have 2 types of special top-level only nodes:
   - rpc
   - notification

The containment tree can have 2 types of real 'non-leaf' nodes:
   - container
   - list

The containment tree can have 3 types of real leafs:
   - leaf
   - leaf-list
   - anyxml

The containment tree can have 4 types of meta-nodes
that do not generate any XML:
   - choice
   - case
   - input
   - output

The containment tree can have 2 types of unnamed nodes:
   - uses
   - augment

Andy

---

From: Andy Bierman <[email-masked]>
Date: Sat, 10 May 2008 10:47:14 -0700

> augments of groupings syntactically, but I would use the path notation

Don't forget to leave out the prefixes for the special case nodes
called 'input' and 'output', since they are in the YANG meta-space,
not any XML namespace.

augment /nc:edit-config/input {
   leaf my-param { ... }
}

E.g:

   <nc:rpc>
     <nc:edit-config>
       <nc:all-the-standard-params... >
       <foo:my-param>17</foo:my-param>
     </nc:edit-config>
   <nc:rpc>

Of course, the other meta-nodes 'choice' and 'case' have names,
so they must have prefixes specified, to clarify which module
the name is from.  Case names and real node names can collide,
but it should be obvious if you see the same node name twice
that it's probably a single-node case-arm within a choice.

augment foo/bar/baz/taz/pizazz/name

Once you hunt down all the node definitions (nobody would ever make
you hunt down a generic node name like 'name' or 'type', right?),
it it clear what the model looks like after the augment.

I would rather make the writer hunt down the node types once,
than make every reader do this every time they read the module.

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Sun, 11 May 2008 23:24:03 +0200

> but it is more useful to the reader.

I guess this is subjective. For me, the noise kind of hides what is
being refinedÂat the end. Perhaps I belong to the other 1000s of
readers of this definition. ;-)

/js

---

From: Phil Shafer <[email-masked]>
Date: Sun, 11 May 2008 18:06:47 -0400

>nodes that don't exist in the grouping are just added.

This would mean the reader doesn't know what's a refinement and
what's an augmentation, so this would be very confusing.

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Sun, 11 May 2008 18:05:56 -0400

>readers of this definition. ;-)

I concur.  The first question the contents of the "uses" statement
must answer is "what's being refined?".  The noise words mask this.

Thanks,
 Phil

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 12 May 2008 10:12:26 +0200 (CEST)

Hi,

> what's an augmentation, so this would be very confusing.

Not with the new proposed syntax:

    uses something {
        refine eh/be/sea {
            <some refinement>
        }
        augment eh/be {
            leaf my-new-leaf {  ... }
        }
    }

I think this syntax makes it very clear what the intention is.

I was going to say that based on experience I didn't like the new
syntax.  In our proprietary DML we use this proposed syntax to add
implementation-specific annotations to data models - things like hints
to the code generator etc.  The idea is that the data model can be
clean and additional annotation files can be used to guide the code
generation.  This syntax is fine when there are just a few
modifications.  But if you have to list every node in the tree as a
separate path, it gets messy - more verbose and error-prone.  Compare:

   uses something {
     refine eh {
        <some refinement 1>
     }

     refine eh/be/sea {
        <some refinement 2>
     }

     refine eh/be/seb {
        <some refinement 3>
     }

     refine eh/be/seb {
        <some refinement 4>
     }
   }
 
vs.

   uses something {
     container eh {
       <some refinement 1>

       container be {
         leaf sea {
           <some refinement 2>
         }
         leaf seb {
           <some refinement 3>
         }
         leaf sec {
           <some refinement 4>
         }
       }
     }
   }

If we anticipate large, deep groupings where people often make many
refinements, then the original syntax is better.  If groupings are
smaller and/or people don't refine eveything, then the proposed syntax
is more clear.

/martin

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 12 May 2008 10:15:05 +0200 (CEST)

Hi,

> augment /nc:edit-config/input {

No, the spec currently says:

  The "input" statement defines an input node in the schema tree, with
  the name "input".

So the 'input' node *is* in the same schema tree just like the other
nodes.  Thus the correct syntax is:

  augment /nc:edit-config/nc:input {
      leaf my-param { ... }
  }

/martin

---

From: Andy Bierman <[email-masked]>
Date: Mon, 12 May 2008 04:26:02 -0700

>   augment /nc:edit-config/nc:input {

I don't see how the current text says one way or another
that there are N instances of the 'input' keyword,
each in its own namespace (nc:input, foo:input, bar:input).

IMO, 'input' is a YANG keyword, and YANG keywords MUST NOT have prefixes.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 12 May 2008 15:11:45 +0200

> The grouping is not present.  Just the uses-stmt.

If the grouping is not present, the person wanting to get sense of the
data model should find the module where it is defined - and it can be
easily seen there what the refinement and/or augment mean.

Lada 

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Phil Shafer <[email-masked]>
Date: Mon, 12 May 2008 09:29:37 -0400

>  augment /nc:edit-config/nc:input {

I agree that "input" is there, but don't see it having a namespace/prefix.

Thanks,
 Phil

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 12 May 2008 15:43:43 +0200 (CEST)

> IMO, 'input' is a YANG keyword, and YANG keywords MUST NOT have prefixes.

The text says that the 'input' keyword defines a node in the schema
tree.  Compare with the 'container' keyword - it also defines a node
in the schema tree.  The difference is that 'input' defines a node
with a static name ("input"), where 'container' takes the name as
argument.  So this construct:

  rpc foo {
    input {
      container bar { ... }
    }
  }

defines three nodes in the schema tree:

  foo
   |__ input
         |___ bar

By saying that "input" is just another node in the schema tree, the
explanation of 'augment' is easier and consistent.  If we say that the
nodes "input" and "output" are special and not really part of the
schema tree, we have to treat them different when we describe how
augment works.   I think the current solution is simpler.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Mon, 12 May 2008 09:01:19 -0700

> augment works.   I think the current solution is simpler.

Except that this is valid too:

augment nc:close-session {
   input {
     leaf close-log-message { type string; }
   }
}

This is where it gets confusing.
The node 'input' is a meta-node that does not produce any XML.
It is special because there can only be one of them.
There is no 'nc:input' and separate 'foo:input' nodes.
If the 'input' node already exists, then the augment cannot
add it again, or it is an error.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 13 May 2008 00:02:29 +0200 (CEST)

Hi,

> add it again, or it is an error.

You're right, but this makes the augment above broken.  Consider the
following:

  module a {
    ...  

and 

  module b {
    ...  
    augment nc:close-session {
      input {
        leaf close-log-reason { type string; }
      }
    }
  }

The problem is that once the input node exists, it cannot be created
in another augment.  But 'a' and 'b' might not know of each other.

There are a couple of solutions:

   1.  don't allow augment to add the input and output nodes

   2.  specify that when input/output is used in an augment, the node
       is added if it doesn't exist, and just merged if it does
       exist.  this makes it really different from normal
       augmentation... 

   3.  say that the input and output nodes _always_ exist when an rpc
       is created.  if input doesn't have any children then the rpc
       doesn't expect any parameters in the XML encoding, and similar
       for output.

Any other alternatives?

/martin

---

From: Andy Bierman <[email-masked]>
Date: Mon, 12 May 2008 16:16:22 -0700

>         leaf close-log-reason { type string; }

The 'input' and 'output' nodes are special this way ;-)
The different namespaces keep all other Descendant-schema-nodeid
components distinct.  This is good, because we have no way of
knowing if modules A and B will be used together, and therefore clash.
This design problem does not exist for any other nodes.

>        doesn't expect any parameters in the XML encoding, and similar

does (3) mean that the input/output forms of augment go away
completely?  I kind of like that approach.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 13 May 2008 09:15:28 +0200 (CEST)

Hi,

> completely?  I kind of like that approach.

Yes.  Something like:

    The "rpc" statement defines three nodes in the schema tree.  One
    top-level node with same name as the RPC method, and two child
    nodes "input" and "output", which are used to hold input
    parameters and response data to the RPC method respectively.

The text about "input" must be modified to say that the "input"
statement adds nodes to the "input" schema node.

Since the input schema node always exist, the follwoing would be legal:

    augment /nc:close-session/nc:input {
        leaf close-log-message { type string; }
    }

even though the input statement is not used in the defintion of
close-session.

/martin