Subject: [netmod] yang-data-ext issues

From: Martin Bjorklund <[email-masked]>
Date: Mon, 16 Apr 2018 14:56:17 +0200 (CEST)

Hi,

While preparing draft-ietf-netmod-yang-data-ext-02, it turned out that
it is not clear what, if any, restrictions should be enforced for
yang-data structures.  Even among the authors we have different ideas
for how this should work.

Background:

In 8040, the original yang-data extension had a restriction that said
that a yang-data structure MUST have exactly one container, since it
wouldn't be possible to have a yang-data structure in an XML instance
document otherwise.

Since people want to use yang-data structures in other places, this
restriction was lifted in the new draft:

   There is no longer an assumption that a yang data structure can
   only be used as a top-level abstraction, instead of nested within
   some other data structure.

With this in mind, here's a use case that I think we ought to support:

  rpc my-first-rpc {
    description
      "Bla bla...
       If an error occurs, <error-info> will contain an instance of
       the yang-data structure 'my-first-rpc-error-info'.";
    ...
  }

  yang-data my-first-rpc-error-info {
    leaf reason { ... }
    container user-info { ... }
  }

  rpc my-second-rpc {
    description
      "Bla bla...
       If an error occurs, <error-info> will contain an instance of
       the yang-data structure 'my-second-rpc-error-info'.";
    ...
  }

  yang-data my-second-rpc-error-info {
    leaf reason { ... }
    leaf important-url { ... }
  }

(maybe in the future we could even have a YANG extension statement to
formalize the description:

   rpc my-first-rpc {
     ...
     opx:error-info-structure my-first-rpc-error-info;
   }

but this is not point now.)

In the example above, note that the leaf "reason" is present in both
structures.  IMO this is not a problem, since these structures are
used in different contexts.

My point is that I think we should impose as few restrictions as
possible to the yang-data extension.  It should be up to the user of
yang-data to ensure that the structure is defined in such a way so
that it can be used properly.  For example, a structure that is
supposed to describe an XML instance document cannot define two leafs
at the top level.

If the WG agrees with what I wrote above, we need to change the
augment-yang-data extension so that you would write for example:

  yx:augment-yang-data /ex:my-first-rpc-error-info/ex:user-info {
    ...
  }

Comments?

/martin

---

From: Joe Clarke <[email-masked]>
Date: Mon, 16 Apr 2018 09:33:24 -0400

>   yx:augment-yang-data /ex:my-first-rpc-error-info/ex:user-info {

I found the "single container only" policy to be too restricting.  I was
modeling data that would [typically] be serialized to something other
than XML, and this just made me jump through more hoops than I wanted to
use rc:yang-data.

I agree with your laissez-faire proposal here.  I think there should be
some text to the same effect about how it is up to the author of the
yd:yang-data elements to make sure the modeled data can be encoded as
they require.

Joe

---

From: Andy Bierman <[email-masked]>
Date: Mon, 16 Apr 2018 07:36:16 -0700

Hi,

I am strongly opposed to this change because it breaks the rule in YANG 1.1
that there cannot be 2 sibling nodes defined in the same module namespace.

IMO since any yang-data nodes are ALLOWED to be used at the top-level,
then these top-level nodes cannot have conflicting names.

It is very important when parsing an instance document that the instance
data
can be associated with the correct schema.  This is not possible if the
same top-level node has multiple yang-data nodes defined.

If one needs to define data that is not top-level, (1) use augment-yang-data
or (2) use a different module.

Andy

On Mon, Apr 16, 2018 at 5:56 AM, Martin Bjorklund <[email-masked]> wrote:

> _______________________________________________
> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod
>

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 16 Apr 2018 16:52:52 +0200 (CEST)

Hi,

> then these top-level nodes cannot have conflicting names.

What do you mean with "used at the top-level"?  Do you mean that they
are allowed to be used e.g. in an XML instance document?  If so, we
should not allow more than one container either.

> same top-level node has multiple yang-data nodes defined.

Correct.  And a structure with more than one node is also not possible
to use in an instance document, but we still allow it.

There is no requirement that a yang-data structure MUST be used as a
single instance document.

My point is that it should be up to the designer that define a
yang-data structure to ensure the structure can be used properly.  In
some cases it means that the designer uses a single container, in some
cases it means that two different strcutures have unique nodes, and in
other cases it might mean that all lists have keys.  We should ensure
that all these use cases are supported.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Mon, 16 Apr 2018 07:55:03 -0700

> yd:yang-data elements to make sure the modeled data can be encoded as

How will your tool handle this?

module foo {
   x:yang-data fake-namespace1 {
      container top { ... }
   }

   x:yang-data fake-namespace2 {
      container top { ... }
   }

   container top { ... }

}

You parse an artifact file:

<top  xmlns="module-foo-namespace">
   <next> ... </next>
</top>

Current YANG says that the <toip> node can only be defined once.
This makes it possible for a parser to pick the correct schema.
If 2 or or more yang-data definitions have this name (foo:top)
then how does your tool pick the right one.?

This existing restriction in YANG serves a useful purpose.
Removing it would be unwise.

There is no restriction to limit the yang-data to a container.
This has already been removed.

My counter-proposal is to remove the argument for yang-data
since it servers no purpose:

  x:yang-data  {
     container top { ... }
  }

  x:yang-data {
      list bar { ... }
   }

Joe

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 16 Apr 2018 17:02:21 +0200 (CEST)

Hi,

> <top  xmlns="module-foo-namespace">

If the yang-data structures are supposed be used in instance documents
w/ no additional context information, then the designer should not use
two nodes with the same name in different strcuctures, just like he
wouldn't use two top-level containers.

But as my original email demonstrated, there are cases where the
context provides the information which structure to use.n

> since it servers no purpose:

I think this would be a mistake.  It wouldn't even be possible to talk
about the different structures in a module.  For example, the
subscribed notification draft has this:

   These are the following three yang-data
   structures for failed event stream subscriptions:

   1.  "establish-subscription-stream-error-info": This MUST be returned
        ...

And it would be impossible to add an extension like:

(see my original email for details)

/martin

---

From: Robert Wilton <[email-masked]>
Date: Mon, 16 Apr 2018 16:44:07 +0100

Don't groupings have a somewhat similar concern?

  E.g. if two groupings define the same data node name and are used at 
the same point then you would get a namespace clash, but YANG does not 
disallow the groupings:

      grouping foo_widget {
        leaf name {
          type string;
          description "Name of my foo widget";
        }
      }

      grouping bar_widget {
        leaf name {
          type string;
          description "Name of my bar widget";
        }
      }

      container all_widgets {
        uses foo_widget;
        uses bar_widget;
      }

The principal difference here, is that the compiler can easily check and 
reject the conflict at the uses statements.

Hence I think that it would be good if we could find a solution for 
yang-data-ext that doesn't not require all root yang-data nodes to be 
unique, since that feels somewhat clunky.  I.e. my preference is to keep 
them less restrictive, as Martin has proposed, if this is feasible.

Thanks,
Rob

On 16/04/2018 15:36, Andy Bierman wrote:
> It is very important when parsing an instance document that the 
> instance data
> On Mon, Apr 16, 2018 at 5:56 AM, Martin Bjorklund <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>     Hi,
>
>     [email-masked] <mailto:[email-masked]>
>     <https://www.ietf.org/mailman/listinfo/netmod>
>
>
>
>

---

From: Andy Bierman <[email-masked]>
Date: Mon, 16 Apr 2018 09:07:04 -0700

On Mon, Apr 16, 2018 at 8:44 AM, Robert Wilton <[email-masked]> wrote:

>  E.g. if two groupings define the same data node name and are used at the
> same point then you would get a namespace clash, but YANG does not disallow
> the groupings:
>
>      grouping foo_widget {
>        leaf name {
>          type string;

It is not clunky that 2 top-level YANG data nodes in the same module
have unique names. This is simple and deterministic.
This restriction has not been a problem so far.

The yang-data statement has to define the context or new abstract namespace,
or whatever this hack is called.  Every tool that implements yang-data has
to be able
to interpret a yang-data statement exactly the same way.

If you want to reinvent XSD substitutionGroup, then do it right.

Andy

>> but this is not point now.)

I see no reason to reinvent the grouping-stmt.
You could easily say opx:error-info-structure argument is a grouping name
as it is a yang-data name.

> netmod mailing [email-masked]://www.ietf.org/mailman/listinfo/netmod
>
>
>

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 16 Apr 2018 18:29:26 +0200 (CEST)

Andy Bierman <[email-masked]> wrote:
> have unique names.

If the yang-data structure is used nested within some other node, it
does not define a top-level node.  In this case it doesn't have to
have uniquely named nodes, and it doesn't have to define a single
container.

> This is simple and deterministic.

yang-data is a new construct.  A more restrictive version was defined
in RFC 8040, and these restrictions *were* problematic.  I don't want
to make the same mistake again.

> The yang-data statement has to define the context or new abstract namespace,

Yes, agreed.  Regardless of the outcome of this thread, the yang-data
statement has to define where the structure being defined is supposed
to be used.

> If you want to reinvent XSD substitutionGroup, then do it right.

This is not a new substitutionGroup.

/martin

---

From: Robert Wilton <[email-masked]>
Date: Mon, 16 Apr 2018 17:46:34 +0100

On 16/04/2018 17:07, Andy Bierman wrote:
>
>
> On Mon, Apr 16, 2018 at 8:44 AM, Robert Wilton <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>      E.g. if two groupings define the same data node name and are used
>     at the same point then you would get a namespace clash, but YANG
>     does not disallow the groupings:
>
>           grouping foo_widget {
>             leaf name {
>               type string;
>     The principal difference here, is that the compiler can easily
>     check and reject the conflict at the uses statements.
>
>     Hence I think that it would be good if we could find a solution
>     for yang-data-ext that doesn't not require all root yang-data
>     nodes to be unique, since that feels somewhat clunky.  I.e. my
>     preference is to keep them less restrictive, as Martin has
>     proposed, if this is feasible.
>
>
>
>
I agree with the statements above.

But it is not clear to me that yang-data-ext is really defining new top 
level data nodes that are part of the same tree/namespace as the 
configuration/state nodes.  In Martin's examples they were used within 
RPCs, and it the forcing the names to be unique in that context that I 
think would be clunky.  E.g. in Martin's example forcing different names 
for "reason" and "user-info" doesn't seem to be helpful.

> or whatever this hack is called.
Perhaps.  I think that this depends on how they are used.

Could a fix for this be something along the lines of:
  - yang-data names must be unique amongst other top level data nodes 
within the module.
  - if yang-data extensions are used at the top level then their name 
must be used as a single top level container.
  - if a yang-data extension is used within another structure then the 
yang-data name is excluded, and the top level nodes defined in the 
yang-data definition are used ....

>   Every tool that implements yang-data has to be able
I'm not familiar with them.  From a quick read, I don't see how they are 
related to the problem that we are trying to solve here.

Thanks,
Rob

---

From: Andy Bierman <[email-masked]>
Date: Mon, 16 Apr 2018 10:05:15 -0700

> Perhaps.  I think that this depends on how they are used.

The yang-data statement has to specify the expansion point, or
at least specify that it is or is not the top-level.

  yang-data top/name1 {
      container mydata;
  }

where context is something like "top" or "error-info", etc.

It is trivial to use groupings if the same set of nodes needs to be used in
different contexts:

  yang-data error-info/name1 {
      container mydata;
  }

Only the context named "top" is restricted to a resulting single-container
and cannot have duplicate names.

This is OK:

  x:yang-data error-info/my-error1 {
      leaf reason {}
  }

  x:yang-data error-info/my-error2 {
      leaf reason {}
  }

>  - if yang-data extensions are used at the top level then their name must
> be used as a single top level container.

A substitutionGroup allows a point int the schema to be identified by name.
Different elements can be defined that match this name, which then can be
used (like a YANG choice) at the specified schema point.
(e.g. error-info above is like a substitutionGroup)

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 17 Apr 2018 07:35:37 +0200

Hi,

this is a slippery slope. If we want to turn YANG into a general-purpose
schema language, it should IMO be done the other way around: design a
general-purpose schema language with a sound architecture, and then use
it for defining schemas of datastores, protocol messages or whatever.

YANG extensions make changes to the original YANG architecture way too
easy, but the result may be a bastard with no architecture at all.

Lada

-- 
Ladislav Lhotka
Head, CZ.NIC Labs
PGP Key ID: 0xB8F92B08A9F76C67

---

From: Kent Watsen <[email-masked]>
Date: Wed, 18 Apr 2018 17:26:53 +0000

I like Andy's proposal below, for the argument of the 'yang-data' statement to encode some meta-information regarding the context/namespace in which it's used, but I wonder how it really works.  For instance, would "top" and "error-info" be the only allowed base-path values for the argument? and what is the value of the remainder of the path?  are we expecting for there to be some kind us 'uses' statement that can refer to just the base-path component to implement substitution-group like behavior?

Kent // contributor

On 4/16/18, 1:05 PM, "netmod on behalf of Andy Bierman" <[email-masked]<mailto:[email-masked]> on behalf of [email-masked]<mailto:[email-masked]>> wrote:

On Mon, Apr 16, 2018 at 9:46 AM, Robert Wilton <[email-masked]<mailto:[email-masked]>> wrote:

On Mon, Apr 16, 2018 at 8:44 AM, Robert Wilton <[email-masked]<mailto:[email-masked]>> wrote:

 E.g. if two groupings define the same data node name and are used at the same point then you would get a namespace clash, but YANG does not disallow the groupings:

     grouping foo_widget {

       leaf name {

         type string;

       }

     }

     grouping bar_widget {

       leaf name {

         type string;

       }

     }

     container all_widgets {

       uses foo_widget;

       uses bar_widget;

     }

The principal difference here, is that the compiler can easily check and reject the conflict at the uses statements.

Hence I think that it would be good if we could find a solution for yang-data-ext that doesn't not require all root yang-data nodes to be unique, since that feels somewhat clunky.  I.e. my preference is to keep them less restrictive, as Martin has proposed, if this is feasible.

But it is not clear to me that yang-data-ext is really defining new top level data nodes that are part of the same tree/namespace as the configuration/state nodes.  In Martin's examples they were used within RPCs, and it the forcing the names to be unique in that context that I think would be clunky.  E.g. in Martin's example forcing different names for "reason" and "user-info" doesn't seem to be helpful.

  yang-data top/name1 {
      container mydata;
  }

It is trivial to use groupings if the same set of nodes needs to be used in different contexts:

This is OK:

Could a fix for this be something along the lines of:
 - yang-data names must be unique amongst other top level data nodes within the module.
 - if yang-data extensions are used at the top level then their name must be used as a single top level container.
 - if a yang-data extension is used within another structure then the yang-data name is excluded, and the top level nodes defined in the yang-data definition are used ....

If you want to reinvent XSD substitutionGroup, then do it right.
I'm not familiar with them.  From a quick read, I don't see how they are related to the problem that we are trying to solve here.

Thanks,
Rob

Andy

Thanks,
Rob

Andy

It is very important when parsing an instance document that the instance data
can be associated with the correct schema.  This is not possible if the
same top-level node has multiple yang-data nodes defined.

Andy

On Mon, Apr 16, 2018 at 5:56 AM, Martin Bjorklund <[email-masked]<mailto:[email-masked]>> wrote:
Hi,

Background:

  rpc my-first-rpc {
    description
      "Bla bla...

  rpc my-second-rpc {
    description
      "Bla bla...

   rpc my-first-rpc {
     ...

Comments?

/martin

_______________________________________________
netmod mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/netmod<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_netmod&d=DwMFaQ&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=9zkP0xnJUvZGJ9EPoOH7Yhqn2gsBYaGTvjISlaJdcZo&m=q6I_yKbXVoahv9h5I1wZiQMUeHLZ5XWuMohEYtypmzs&s=jECZMhypw9LtuxzuntkFNM-8lm7xpztYwDDLOxCM_8k&e=>

netmod mailing list

[email-masked]<mailto:[email-masked]>

https://www.ietf.org/mailman/listinfo/netmod<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_netmod&d=DwMFaQ&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=9zkP0xnJUvZGJ9EPoOH7Yhqn2gsBYaGTvjISlaJdcZo&m=q6I_yKbXVoahv9h5I1wZiQMUeHLZ5XWuMohEYtypmzs&s=jECZMhypw9LtuxzuntkFNM-8lm7xpztYwDDLOxCM_8k&e=>

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 18 Apr 2018 19:43:35 +0200 (CEST)

Hi,

[Kent, your email program has messed up the quoting in this thread.
It becomes quite difficult to follow.  And no, please don't invent a
new quoting style in every email thread...]

Kent Watsen <[email-masked]> wrote:
> I like Andy's proposal below, for the argument of the 'yang-data'
> statement to encode some meta-information regarding the
> context/namespace in which it's used, but I wonder how it really
> works.

I *think* that what Andy proposes is a way to formally specify the
context in which the yang-data structure will be used.

My proposal was to simply define the context in plain text.  A formal
way of defining the context would of course be useful -- if we can
find agreement on how that would work.  I don't think we should
hard-code a few contexts in this draft; the mechanism must be
extensible.

An alternative syntax could be to introduce a new substatement:

  yd:yang-data foo {
    yd:context ...;
    description "...";
  }

/martin

> For instance, would "top" and "error-info" be the only allowed
> base-path values for the argument? and what is the value of the
> remainder of the path?  are we expecting for there to be some kind us
> 'uses' statement that can refer to just the base-path component to
> implement substitution-group like behavior?
> 
> Kent // contributor
> 
> 
> On 4/16/18, 1:05 PM, "netmod on behalf of Andy Bierman"
> <[email-masked]<mailto:[email-masked]> on behalf of
> [email-masked]<mailto:[email-masked]>> wrote:
> 
> 
> 
> On Mon, Apr 16, 2018 at 9:46 AM, Robert Wilton
> <[email-masked]<mailto:[email-masked]>> wrote:
> 
> 
> 
> On Mon, Apr 16, 2018 at 8:44 AM, Robert Wilton
> <[email-masked]<mailto:[email-masked]>> wrote:
> 
> The principal difference here, is that the compiler can easily check
> and reject the conflict at the uses statements.
> 
> unique, since that feels somewhat clunky.  I.e. my preference is to
> keep them less restrictive, as Martin has proposed, if this is
> feasible.
> 
> 
> 
> But it is not clear to me that yang-data-ext is really defining new
> top level data nodes that are part of the same tree/namespace as the
> think would be clunky.  E.g. in Martin's example forcing different
> names for "reason" and "user-info" doesn't seem to be helpful.
> 
> 
> 
> 
>  - within the module.
>  - be used as a single top level container.
>  - yang-data name is excluded, and the top level nodes defined in the
>  - yang-data definition are used ....
> 
> 
> I'm not familiar with them.  From a quick read, I don't see how they
> are related to the problem that we are trying to solve here.
> 
> 
> On Mon, Apr 16, 2018 at 5:56 AM, Martin Bjorklund
> <[email-masked]<mailto:[email-masked]>> wrote:
> Hi,
>

---

From: Andy Bierman <[email-masked]>
Date: Wed, 18 Apr 2018 10:43:42 -0700

On Wed, Apr 18, 2018 at 10:26 AM, Kent Watsen <[email-masked]> wrote:

> statement to encode some meta-information regarding the context/namespace
> in which it's used, but I wonder how it really works.  For instance, would
> "top" and "error-info" be the only allowed base-path values for the
> argument? and what is the value of the remainder of the path?  are we
> expecting for there to be some kind us 'uses' statement that can refer to
> just the base-path component to implement substitution-group like behavior?
>
>
>

If we want to avoid defining these contexts, then we could just define root
vs. nonroot.

e,g:

x:yang-data /mydef1 {
  container foo;
}

x:yang-data mydef2 {
  leaf x;
  leaf y;
  container z;
}

Only an argument starting with '/' would be treated as a top-level data
node.

All other yang-data definitions are not allowed to appear as a root node.
The context where this yang-data is used is completely proprietary.
The mechanism used to expand this yang-data as if it was a grouping
is completely proprietary.

The augment-yang-data extension only applies to top-level yang-data
definitions.

However, my preference is to only standardize top-level yang-data.
I do not see any need for the other form since all functionality can be
achieved with a grouping and a proprietary YANG extension.

Kent // contributor

Andy

>
>
>
>
> On 4/16/18, 1:05 PM, "netmod on behalf of Andy Bierman" <
> [email-masked] on behalf of [email-masked]> wrote:
>
>
>
>
>
>
>
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_netmod&d=DwMFaQ&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=9zkP0xnJUvZGJ9EPoOH7Yhqn2gsBYaGTvjISlaJdcZo&m=q6I_yKbXVoahv9h5I1wZiQMUeHLZ5XWuMohEYtypmzs&s=jECZMhypw9LtuxzuntkFNM-8lm7xpztYwDDLOxCM_8k&e=>
>
>
>
>
>
> https://www.ietf.org/mailman/listinfo/netmod <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_netmod&d=DwMFaQ&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=9zkP0xnJUvZGJ9EPoOH7Yhqn2gsBYaGTvjISlaJdcZo&m=q6I_yKbXVoahv9h5I1wZiQMUeHLZ5XWuMohEYtypmzs&s=jECZMhypw9LtuxzuntkFNM-8lm7xpztYwDDLOxCM_8k&e=>
>
>
>
>
>
>
>
>
>

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 18 Apr 2018 19:47:44 +0200 (CEST)

Hi,

> However, my preference is to only standardize top-level yang-data.

What is "top-level" yang-data?

/martin

> > I am strongly opposed to this change because it breaks the rule in YANG 1..1
> >
> > that there cannot be 2 sibling nodes defined in the same module namespace..
> >
> >
> >

---

From: Andy Bierman <[email-masked]>
Date: Wed, 18 Apr 2018 10:51:59 -0700

On Wed, Apr 18, 2018 at 10:47 AM, Martin Bjorklund <[email-masked]> wrote:

> context/namespace
It is a data structure that represents an instance document.

Since this is the ONLY definition of yang-data we have in the standard,
what is a yang-data definition that represents some unspecified usage,
other than an instance document?

Why is this any different than a YANG grouping?

Andy

> the
> and
> keep
> names
> > > It is trivial to use groupings if the same set of nodes needs to be
> used
> > > in different contexts:
> > >
> > >
> > >
> > >
> are
> instance
> > > <https://urldefense.proofpoint.com/v2/url?u=https-
> 3A__www.ietf.org_mailman_listinfo_netmod&d=DwMFaQ&c=
> HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=
> 9zkP0xnJUvZGJ9EPoOH7Yhqn2gsBYaGTvjISlaJdcZo&m=q6I_
> yKbXVoahv9h5I1wZiQMUeHLZ5XWuMohEYtypmzs&s=jECZMhypw9LtuxzuntkFNM-
> 8lm7xpztYwDDLOxCM_8k&e=>
> > >
> > >
> > >
> > >
> > >
> > > https://www.ietf.org/mailman/listinfo/netmod <https://urldefense.
> proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_
> listinfo_netmod&d=DwMFaQ&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=
> 9zkP0xnJUvZGJ9EPoOH7Yhqn2gsBYaGTvjISlaJdcZo&m=q6I_
> yKbXVoahv9h5I1wZiQMUeHLZ5XWuMohEYtypmzs&s=jECZMhypw9LtuxzuntkFNM-
> 8lm7xpztYwDDLOxCM_8k&e=>
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
>

---

From: Kent Watsen <[email-masked]>
Date: Wed, 18 Apr 2018 19:02:15 +0000

Another and somewhat radical idea is to think of 'yang-data' as defining a data node, like a 'container', but not a config or opstate node.  Yes, this is different from rc:yang-data, which defines a transparent node, like 'choice', but maybe it's okay if we can get the substitution groups part working.  For instance:

From draft-ietf-anima-voucher:
    OLD:
       rc:yang-data voucher-artifact {
          container voucher {…}
        }
    NEW:
       yd:yang-data voucher {
          …
        }

From draft-ietf-netconf-zerotouch:
    OLD
         rc:yang-data "zerotouch-information" {
           choice information-type {
             container redirect-information {…}
             container onboarding-information {…}
           }
         }
    NEW
         yd:yang-data "redirect-information" {
             yd:substitution-group zerotouch-information;
             …
         }
         yd:yang-data "onboarding-information" {
             yd:substitution-group zerotouch-information;
             …
         }

And for the error-info use-case:

    yd:yang-data "error-info" {
        choice error-type {
            leaf my-error1 {…}
            leaf my-error2 {…}
        }
   }

Thoughts?

Kent

---

From: Robert Varga <[email-masked]>
Date: Sat, 21 Apr 2018 03:16:04 +0200

> easy, but the result may be a bastard with no architecture at all.

+1 in general, although I am not convinced we need to start from scratch.

The problem is that the metamodel behind YANG is not formally defined,
which means every implementer has to reverse-engineer it from
RFC6020/7960 (and 6241, and others). Since there is no metamodel spec,
it is very hard to reason about how an extension affects it, especially
in edge cases -- which can (and most probably will) lead to bastardization.

One example: YANG 1.1 was supposed to be a backwards-compatible, yet it
introduced multiple-inheritence to a language which was previously
strictly single-inheritence. That sort of change is a major revision of
the metamodel and certainly does not qualify as backwards-compatible at
that level of abstraction.

It may not be important for run-of-the-mill NETCONF operations, but
becomes pivotal when you are mapping YANG to other languages.

Regards,
Robert

---

From: Ladislav Lhotka <[email-masked]>
Date: Sun, 22 Apr 2018 14:56:51 +0200

> +1 in general, although I am not convinced we need to start from scratch.

I didn't mean to start from scratch, but the thing is that YANG spec contains
many references to the specific NM context that don't make sense in other
contexts, so these would need to be removed. To begin with, it is the config true/false dichotomy.

> in edge cases -- which can (and most probably will) lead to bastardization.

In fact, for the major XML schema languages the metamodel stems from a specific
formal validation algorithm. This article is quite instructive:

http://pike.psu.edu/publications/toit05.pdf

A useful side effect of the YANG-to-DSDL mapping was the evidence that YANG 1.0
was still pretty close to the mainstream of XML schema languages.
  
> that level of abstraction.

I'd defend YANG 1.1 here, I think it was a well managed and conservative update.
The meaning of backward compatibility was defined in the charter: YANG 1.0
modules should continue to work in 1.1. Apart from fixing evident bugs, I
believe this goal was achieved.

I am much more concerned with some of the post-1.1 features, also because YANG
is now being updated in several directions without a clear vision. And another
big problem is that YANG extensions are used for these changes, so we will
probably end up with several different versions of YANG, although formally
everything will be 1.1.  

Regarding your example: from my own experience, implementing validation of
identityref values with multiple inheritance is not terribly complicated in
comparison to single inheritance. I have other favorites for the most peculiar
feature in YANG.

> becomes pivotal when you are mapping YANG to other languages.

You are right, if such a mapping is what you need. I have myself given up
updating the DSDL mapping.

Lada

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 23 Apr 2018 08:52:22 +0200 (CEST)

> It is a data structure that represents an instance document.

So then I assume that you also want to change the current draft -01
behavior so that a yang-data structure MUST have a single container as
the child?  Just like in rc:yang-data.

> other than an instance document?

We want to create a flexible solution for creating structures that can
be used in other places than just self-contained instance document.
One example is the error-info data (see subscribed-notifications).

> Why is this any different than a YANG grouping?

?  We don't have these CLRs for groupings; the following is legal:

  grouping foo {
    leaf a { ... }
    leaf b { ... }
  }

  grouping bar {
    leaf b { ... }
  }

It is up to the user of the grouping to ensure it is used in a way so
that the result is legal.  I think yang-data should work the same
way.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Mon, 23 Apr 2018 04:10:41 -0700

On Sun, Apr 22, 2018 at 11:52 PM, Martin Bjorklund <[email-masked]> wrote:

> > > > > in which it's used, but I wonder how it really works.  For
> instance,
> > > would
> > > > > expecting for there to be some kind us 'uses' statement that can
> refer
> > > to
> > > > If we want to avoid defining these contexts, then we could just
> define
> > > root
> > > > vs. nonroot.
> > > >
> > > > e,g:
> > > >
> > > > x:yang-data /mydef1 {
> > > >   container foo;
> > > > }
> > > >
> > > > x:yang-data mydef2 {
> > > >   leaf x;
> > > >   leaf y;
> > > >   container z;
> > > > }
> > > >
> > > >

A yang-data structure representing an instance document needs to result in
a container.

> that the result is legal.  I think yang-data should work the same

We don't need yang-data to be another way to define a grouping, because
we already have grouping-stmt.

We don't need to re-invent another collection of data-def-stmts that are not
instantiated as data nodes automatically. Any place you could use yang-data
for this purpose, you could use a grouping instead.

Andy

> at
> check
> > > > > A substitutionGroup allows a point int the schema to be identified
> by
> > > name.
> > > > >
> > > > > Different elements can be defined that match this name, which then
> can
> > > be
> > > > >
> > > > > You could easily say opx:error-info-structure argument is a
> grouping
> > > name
> > > > >

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 23 Apr 2018 13:28:30 +0200 (CEST)

> A yang-data structure representing an instance document needs to result in

Right, and since you wrote:

    my preference is to only standardize top-level yang-data.

I assume that you want to re-add the CLR about a single container in
yang-ext?

> we already have grouping-stmt.

I agree.  At some point someone suggested "uses-yang-data", I think we
both agreed that this was a bad idea.

> for this purpose, you could use a grouping instead.

I'm not sure what you are trying to say.  Are you suggesting that we
don't need yang-data at all?

/martin

---

From: Andy Bierman <[email-masked]>
Date: Mon, 23 Apr 2018 08:59:38 -0700

On Mon, Apr 23, 2018 at 4:28 AM, Martin Bjorklund <[email-masked]> wrote:

> > > > > > On Wed, Apr 18, 2018 at 10:26 AM, Kent Watsen <
> [email-masked]>
> > > > > wrote:
> > > > > >
> > > > > > > argument? and what is the value of the remainder of the path?
> are
> > > we
> > > > > > > just the base-path component to implement substitution-group
> like
> > > > > behavior?
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > > Only an argument starting with '/' would be treated as a
> top-level
> > > data
> > > > > > node.
> > > > > >
> > > > > > All other yang-data definitions are not allowed to appear as a
> root
> > > node.

I do not understand the need for a yang-data structure that represents data
that can be instantiated anywhere and everywhere.  I do not want to break
existing tools that expect sibling data nodes in the same module namespace
to
be unique local-names.

I would rather stick with the yang-data in RFC 8040 than introduce a new
extension
with no restrictions.  Standard YANG extensions should be interoperable and
have
a clear purpose. If we do not need to define what a YANG extension does in
a way that can be observed somehow, then it does not need to be a standard.

Andy

>
>
> >
> >
> >
> > > /martin
> > >
> > >
> > >
> >
> > Andy
> >
> >
> >
> > >
> > > >
> > > >
> > > >
> > > > > /martin
> > > > >
> > > > >
> > > > >
> > > > Andy
> > > >
> > > >
> > > >
> > > > > > I do not see any need for the other form since all functionality
> can
> > > be
> > > > > > > On Mon, Apr 16, 2018 at 9:46 AM, Robert Wilton <
> [email-masked]>
> > > > > wrote:
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On Mon, Apr 16, 2018 at 8:44 AM, Robert Wilton <
> [email-masked]>
> > > > > wrote:
> > > > > > >
> > > > > > > same point then you would get a namespace clash, but YANG does
> not
> > > > > disallow
> > > > > > > the groupings:
> > > > > > >
> > > > > > >
> > > > > > >      grouping foo_widget {
> > > > > > >
> > > > > > >        leaf name {
> > > > > > >
> > > > > > >          type string;
> > > > > > >
> > > > > > > yang-data-ext that doesn't not require all root yang-data
> nodes to
> > > be
> > > > > > > The yang-data statement has to define the context or new
> abstract
> > > > > > > namespace,
> > > > > > >
> > > > > > >  - if a yang-data extension is used within another structure
> then
> > > the
> > > > > > > I am strongly opposed to this change because it breaks the
> rule in
> > > > > YANG 1..1
> > > > > > >
> > > > > > > can be associated with the correct schema.  This is not
> possible
> > > if the
> > > > > > >
> > > > > > > On Mon, Apr 16, 2018 at 5:56 AM, Martin Bjorklund <
> [email-masked]>
> > > > > wrote:
> > > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > While preparing draft-ietf-netmod-yang-data-ext-02, it turned
> out
> > > that
> > > > > > > In 8040, the original yang-data extension had a restriction
> that
> > > said
> > > > > > > that a yang-data structure MUST have exactly one container,
> since
> > > it
> > > > > > > (maybe in the future we could even have a YANG extension
> statement
> > > to
> > > > > > > possible to the yang-data extension.  It should be up to the
> user
> > > of

---

From: Robert Varga <[email-masked]>
Date: Mon, 23 Apr 2018 18:48:27 +0200

> believe this goal was achieved.

I agree and the upgrade process was very smooth, all things considered.

> probably end up with several different versions of YANG, although formally

Agree, and it's especially the interactions between extensions which are
critical to manage the outcome.

> comparison to single inheritance. I have other favorites for the most peculiar

It's not, but it forced us to incompatibly change our Java binding
specifications -- where we used to use abstract classes we have to use
interfaces. Luckily there is no state involved :)

Regards,
Robert

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 23 Apr 2018 18:51:04 +0200

On Sun, Apr 22, 2018 at 02:56:51PM +0200, Ladislav Lhotka wrote:

> I am much more concerned with some of the post-1.1 features, also
> because YANG is now being updated in several directions without a
> clear vision. And another big problem is that YANG extensions are
> used for these changes, so we will probably end up with several
> different versions of YANG, although formally everything will be
> 1.1.

I tend to agree. Ideally, we would carefully remove things from YANG
that did not meet the cost/benefit target (e.g., submodules),
reorganize definitions whenever possible (some NETCONF specific stuff
in the YANG specification should not be there, XML encoding may be
factored out) and incorporate new features (like yang-data) after we
have sufficient _experience_ to know that such new features will be
useful (which seems to be the case for yang-data).

Yes, such iterations likely take 2 years at IETF speed but this kind
of maintenance cost/effort is likely the price to be paied for
something that is being used at a larger scale.

Some people will say that the cost of a new language version is high.
(Well, when we did 1.1, some people said it will never be deployed.)
Anyway, not bumping the YANG version number but having instead several
(optional) language extensions is just hiding the version number
change under the carpet.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1 | 28759 Bremen | Germany
Fax:   +49 421 200 3103         <https://www.jacobs-university.de/>

---

From: Robert Varga <[email-masked]>
Date: Mon, 23 Apr 2018 19:18:38 +0200

> (optional) language extensions is just hiding the version number

Not quite, as extensions allow for modular/incremental evolution, as an
implementer I do not have to go through a long development cycle where I
need to rewire language aspects just to get the features my users need
for their models...

Regards,
Robert

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 23 Apr 2018 19:36:36 +0200

> need to rewire language aspects just to get the features my users need

Once we standardize extensions (and this is what I am talking about),
we better make sure that the whole stays consistent. Otherwise, we
will end up with patchwork where all the patches may work in isolation
but sooner or later they will fall apart when you look at all the
possible combinations.

I am in favour of managing a clean definition of the core of the YANG
language instead of creating patchwork. It is great if people create
and prototype new extensions but once such extensions are considered
to be ready for general use, we should roll them into the core (and
remove any cruft from the core at the same time).

/js

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 23 Apr 2018 21:49:23 +0200 (CEST)

Andy Bierman <[email-masked]> wrote:
> that can be instantiated anywhere and everywhere.

AFAIK noone is proposing that.

> I do not want to break
> a clear purpose.

Of course.

> If we do not need to define what a YANG extension does in

Agreed.

Not sure how any of this helps with the original issue though.

/martin

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 23 Apr 2018 21:51:10 +0200 (CEST)

Hi,

I am not sure what this statement tells us re. the issue in this email
thread.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Mon, 23 Apr 2018 12:58:03 -0700

> ....

You proposed that duplicate nodes were OK:

module X {
prefix x;

x:yang-data A {
   list foo { ... }
}

x:yang-data B {
  container foo { ... }
}

}

I do not want to allow any duplicates.
There are no encoding and parsing rules for instance data
that support this sort of duplicate.

yang-data definitions define conceptual data nodes (e.g, /x:foo)
Only one data-def-stmt (in yang-data or otherwise) can define a data node
/x:foo.
The descriptive names for the yang-data (A or B) do not define namespaces.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 23 Apr 2018 22:08:15 +0200 (CEST)

> I do not want to allow any duplicates.

Yes, I got that.

> that support this sort of duplicate.

This is not correct, as I have demonstrated earlier, and I think you
also accepted; if different structures are defined for different rpcs'
error-infos, then these structures can have the same child node names.

I think that we have to agree on the basics before disussing
solutions:

  1)  Should we do anything at all?

      (i.e., keep using yang-data in RFC 8040)

  2)  Should we define structures that only can be used in
      standalone instance documents?

      (i.e., *more* restrictive than yang-data in RFC 8040)

  3)  Should we define structures that can be used in standalone
      instance documents, error-info contents, and other places that
      we might not know right now?

      (i.e., *less* restrictive than yang-data in RFC 8040)

Since the current draft says:

   The "yang-data" extension statement from RFC
   8040 [RFC8040] is defined for this purpose, however it is limited in
   its functionality.

   The intended use of the "yang-data" extension is to model all or part
   of a protocol message, such as the "errors" definition in ietf-
   restconf.yang [RFC8040], or the contents of a file.  However,
   protocols are often layered such that the header or payload portions
   of the message can be extended by external documents.  The YANG
   statements that model a protocol need to support this extensibility
   that is already found in that protocol.

I thought we are doing (3).

/martin

---

From: Andy Bierman <[email-masked]>
Date: Mon, 23 Apr 2018 13:58:50 -0700

> remove any cruft from the core at the same time).
I agree with your concerns.
If the intent is that a YANG extension is optional, then its semantics have
to be well-scoped so that tools really can skip over the extension without
any problems.
If we create standard extensions that standard YANG 1.1 modules use, then
this can
become an unofficial add-on to YANG 1.1. (e.g. annotation could already be
considered as such)

/js
Andy

---

From: Robert Wilton <[email-masked]>
Date: Tue, 24 Apr 2018 09:30:17 +0100

[Renaming the thread because this doesn't seem to be directly applicable 
to the yang-data extension.]

I think that extensions are great in the sense that they allow YANG to 
evolve more quickly without requiring a fork lift version upgrade that 
may take a long time to produce.

I agree that all standard defined extensions should cleanly 
inter-operate with each other.

I also think that it is useful to periodically define new versions of 
YANG that fold back in all the extensions that have been successfully 
and are widely deployed.  And also to deprecate/obsolete stuff that 
hasn't worked well.

For me, the interesting question is about namespaces when those 
extensions are incorporated back into a new version of YANG:
- Does it just keep the extension prefix that the extension module was 
originally defined in?
- Or does the extension get folded into core of the YANG language and no 
extension prefix is used any more?
- Or does the extension get supported both with/without the extension 
namespace prefix?

Thanks,
Rob

On 23/04/2018 21:58, Andy Bierman wrote:
>
>
> On Mon, Apr 23, 2018 at 10:36 AM, Juergen Schoenwaelder 
> <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
> to be well-scoped so that tools really can skip over the extension 
> without any problems.
> If we create standard extensions that standard YANG 1.1 modules use, 
> then this can
> become an unofficial add-on to YANG 1.1. (e.g. annotation could 
> already be considered as such)
>
>
>
>     /js
>
>
> Andy
>
>     -- 
>     Fax:   +49 421 200 3103         <https://www.jacobs-university.de/
>     <https://www.jacobs-university.de/>>
>

---

From: Robert Wilton <[email-masked]>
Date: Tue, 24 Apr 2018 10:01:54 +0100

>        (i.e., keep using yang-data in RFC 8040)
There is also an option 1(b) which is to move the current yang-data 
definition on RFC 8040 into it's own document, just to fix the 
references issue.

>        (i.e., *more* restrictive than yang-data in RFC 8040)
I don't think that we should define something more restrictive that 
yang-data in RFC 8040.

>        (i.e., *less* restrictive than yang-data in RFC 8040)
I don't know about this one because I'm not sure that I understand the 
problem space well enough.

For some of the categories above would a choice statement + groupings 
works just as well as a yang-data extension?

A different thought, one that has probably been considered before:
  - Could all yang data definitions be defined using groupings instead.  
I.e. a grouping without any associated uses statements.
  - Perhaps an extra statement under the grouping could be used to 
indicate whether the grouping represents a yang data definition.

Thanks,
Rob

> .

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 24 Apr 2018 16:07:30 +0200

> may take a long time to produce.

This is of course a double-edged sword. In my view, extensions must not
influence the resulting data model, and I thought the last paragraph in
sec. 6.3.1 of RFC 6020 had been intended to mean exactly this.

I would prefer to indicate experimental versions of YANG in the version
string (e.g. as a topical branch), and use YANG extensions only for
stuff that's outside the realm of data modelling. For example, the
extension in NACM is perfectly OK because it is a hint for protocol
operation and doesn't affect the schema in any way.

> and are widely deployed.  And also to deprecate/obsolete stuff that 

Without a metamodel, it is difficult to avoid YANG becoming a kitchen
sink.

And what about extensions that won't make it into the new official
version?  For example, imagine that "openconfig:pattern" won't be
accepted - would it really matter? The forked versions will continue to
exist but they could still claim adherence to the IETF standard (which
is what marketing departments love).

Lada

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 24 Apr 2018 16:25:57 +0200

> need to rewire language aspects just to get the features my users need

But what prevents you from forking YANG, indicating it in the
"yang-version" string and implementing your new statements as built-ins?

It is tempting to think about extensions as a magic for adding modular
extensions but it breaks down as soon as such extensions interfere with
the YANG core (with and each other, or both).

Lada

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 24 Apr 2018 16:30:09 +0200

> I am not sure what this statement tells us re. the issue in this email

It tells us that, in my view, the approach taken in this document is a
bad idea.

Lada

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 24 Apr 2018 16:36:01 +0200 (CEST)

> It tells us that, in my view, the approach taken in this document is a

Do you mean that the WG shoud drop this document?  And people that
need yang-data should continue to use the version in 8040?  Or that
people that need yang-data do not have a valid use case and they
should do something else?

/martin

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 24 Apr 2018 16:43:23 +0200 (CEST)

> the YANG core (with and each other, or both).

Sure, nothing new here.  This is how the extension mechanism is
designed.  Do you feel that there are so many standard extensions that
interfere with each other that we actually have a problem?  I strongly
agree that we need to be careful with standard extensions, and I
strongly encourage vendors to not define extensions that interfere
with YANG core (been there, done that, bad idea).

In the case of yang-data, the fact is that it *is* used in standard
models (in ways not originally anticipated), and vendors have similar
proprietary extensions (we have a tailf:structure, and I think others
have reported similar things).

/martin

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 24 Apr 2018 16:51:04 +0200 (CEST)

Robert Wilton <[email-masked]> wrote:
> >>>> with no restrictions.  Standard YANG extensions should be
> >>>> interoperable
> >>> and
> >>>> have
> >>>> a clear purpose.
> >>> Of course.
> >>>

Sure, but I don't think it is worth the effort.

> I don't think that we should define something more restrictive that

I agree.

> works just as well as a yang-data extension?

Not sure what you mean, but before we had yang-data in 8040, people
used something like this:

  module my-module {
    ....
    container foo {
      description
        "This is not really a real data node that is supposed to be
         instantiated in any datastore.  Instead it is supposed to be
         the top-level node in an instance document that
         describes...";
    }
  }

By this is really a mis-use of YANG statements.  Wrapping it in an
extension solves that problem.
    
>  - Could all yang data definitions be defined using groupings
> instead.  I.e. a grouping without any associated uses statements.

Nodes in a grouping do not belong to any namespace.  Forcing such a
namespace binding with a substatement to "grouping" feels worse than
having a YANG extension with the nodes defined (which of course can be
done with "uses").

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 24 Apr 2018 17:23:03 +0200

> Do you mean that the WG shoud drop this document?  And people that

Yes. Doing it properly would amount to rewriting many parts of RFC 7950 for use
inside "yang-data".

> need yang-data should continue to use the version in 8040?  Or that

Preferably not.

> should do something else?

They may have a valid use case but YANG (in the current form) is not suitable
for it.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Tue, 24 Apr 2018 12:18:43 -0700

On Mon, Apr 23, 2018 at 1:08 PM, Martin Bjorklund <[email-masked]> wrote:

> > > > I do not understand the need for a yang-data structure that
> represents
> > > data
The use-case that has come up several times is (1).
People want to use YANG to define the schema for an XML or JSON
representation of a stand-alone document.

Item (3) needs to be machine-readable and deterministic for it to be even
remotely
feasible as a standard. There is no way a tool should have to match <x:foo>
to
the correct schema, based on the description-stmt inside some
yang-data-stmt.
The only data needed must be module + local-name.

The example you gave of different definitions of the <reason> leaf is a
really bad idea.
We should never try to define different schema for the same instance data,
where the module-name and local-name are the same, but the contents are
different.

Defining an extension that maps error-info data for a specific RPC might be
something worth standardizing.  It should not be done with yang-data,
but rather a different extension just for this purpose.

Andy

---

From: Kent Watsen <[email-masked]>
Date: Tue, 24 Apr 2018 21:13:12 +0000

> representation of a stand-alone document.

Agreed

> The only data needed must be module + local-name.

Or maybe: module + local-name + context, where context is one of:

  - data nodes
  - RPC/actions
  - notifications
  - standalone artifacts (files)

At least, it seems that these are different things.

[Note: RESTCONF places some of the context into the URL; two different data node resources can have identical module + local-name.]

It would be bad if two stand-alone artifacts had the same module+local-name (foo:bar).   However, it's okay to have a matching top-level data node called "foo:bar", since it is used in a different context.  Just like yang-data cannot be accessed as data via a protocol like NC or RC, so it is that traditionally-defined data nodes cannot be accessed as a stand-alone artifact (unless you’re a draft-author and need an instance document for an example).

> but rather a different extension just for this purpose.

Martin wrote before:

               rpc my-first-rpc {
                 ...

I imagined that he was hoping to limit what's needed to get draft-ietf-netconf-notification-messages out the door now, but I think another I-D should be proposed to augment the 'rpc' and 'action' statements with an anydata called "error-info-structure" so the YANG would be more like this:

               rpc my-first-rpc {
                 ...
                 opx:error-info-structure my-first-rpc-error-info {
                     …
                   }
               }

That is, to your point, with no reference to a yang-data data structure.

Kent // contributor

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 25 Apr 2018 08:57:51 +0200 (CEST)

>   - standalone artifacts (files)

I don't think we should use yang-data to model notifications.
Presumably you meant error-info from rpc or action.

And my point it is that I think that we should make yang-data
flexible enoough in itself, and not constrain it to the one or two use
cases we know now.

> [Note: RESTCONF places some of the context into the URL; two different
> data node resources can have identical module + local-name.]
> 
> It would be bad if two stand-alone artifacts had the same
> module+local-name (foo:bar).  However, it's okay to have a matching
> top-level data node called "foo:bar", since it is used in a different
> context.  Just like yang-data cannot be accessed as data via a
> protocol like NC or RC, so it is that traditionally-defined data nodes
> cannot be accessed as a stand-alone artifact (unless you’re a
> draft-author and need an instance document for an example).
> 
> 
> I imagined that he was hoping to limit what's needed to get
> draft-ietf-netconf-notification-messages out the door now, but I think
> another I-D should be proposed to augment the 'rpc' and 'action'
> statements with an anydata called "error-info-structure" so the YANG
> would be more like this:
> 
>                rpc my-first-rpc {
>                  ...

No I was thinking along the lines of:

  ydx:yang-data my-first-rpc-error-info {
    ...
  }

  rpc my-first-rpc {
    ...

I.e., use yang-data to define a structure, and use another statement
to tie them together.

If we define special statements with inline structures, we probably
also need special augment statements; with your example:

  rpc my-first-rpc {
    ...

  opx:augment-error-info-structure '/m:my-first-rpc'
                                 + '/m:my-first-rpc-error-info {
    ...
  }

/martin

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 25 Apr 2018 09:03:47 +0200 (CEST)

> where the module-name and local-name are the same, but the contents are

When the context is different this is not a problem.  Compare w/
augment:

    augment "/if:interfaces/if:interface" {
      container ipv4 { ... }
    }

    augment "/if:interfaces/if:interface-state" {
      container ipv4 { ... }
    }
     
These augements define the node "ipv4" in the the "ietf-ip" namespace,
but b/c the context is different, there is no risk for confusion.

> but rather a different extension just for this purpose.

See my email to Kent.  I don't understand why this can't be done with
yang-data.  This said, if we were to do a new version of YANG and
address this problem, we would define a new core keyword for this.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Wed, 25 Apr 2018 03:09:44 -0700

>                                  + '/m:my-first-rpc-error-info {

This could easily be defined to use groupings instead:

  grouping my-first-rpc-error-info {
    ...
  }

  rpc my-first-rpc {
    ...

 There is no need to reinvent the grouping.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Wed, 25 Apr 2018 03:13:33 -0700

On Wed, Apr 25, 2018 at 12:03 AM, Martin Bjorklund <[email-masked]> wrote:

> > > > > > I would rather stick with the yang-data in RFC 8040 than
> introduce a
> > > new
> > > > > > extension

But you are proposing that the target location not be specified,
or specified inside a description-stmt.

No matter what context you pick, DOUBLE augment is not allowed:

So yang-data cannot define the same node in multiple contexts.
Leaving the context unspecified is not an option.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 25 Apr 2018 14:38:41 +0200

> designed.  Do you feel that there are so many standard extensions that

My objections are also not new. Implementations are permitted to ignore
extensions, but if an extension affects the resulting schema, then
implementation A that supports the extension may end up working with
another schema than implementation B that doesn't support the
extension. This situation has to be avoided.

> interfere with each other that we actually have a problem?  I strongly

Perhaps not yet, but we do have extensions (or proposals thereof) that
interfere with core YANG. And my impression is that OpenConfig is going
to use extensions for creating their own YANG silo.

> have reported similar things).

I don't know how much effort those vendors invested into the
specification of such extensions, but the text of the current draft and
module descriptions leave quite a few questions unanswered. Here are
just a few that come to my mind:

- What is the accessible tree for XPath evaluations and leafrefs? In
  particular, error messages might want to refer to state data. Is it
  possible?

- This sentence makes no sense: "The XPath document root is the
  extension statement itself, ..." A YANG statement cannot be an XPath
  node.

- What is the meaning of leaf and leaf-list default values? In RFC 7950
  it is defined in terms of server behaviour but there is no server for
  yang-data.

- What does it mean if an action is defined inside yang-data?

- If a module is imported without revision and its groupings/typedefs
  are used inside yang-data, how is the exact revision of the imported
  module determined?

I suspect that users of such extensions accept the fact that some amount
of hand-waving is necessary. But then, why all this hassle? Why cannot
normal YANG be used for specifying a schema that is not intended for an
NM protocol? It requires just a little bit more hand-waving. The draft
doesn't explain why the extension is needed.

Also, I am quite sure that YANG experts can use it carefully so that it
makes sense. But as a Standard Track document it becomes an official
part of YANG that seems to give unsuspecting users enough rope for
hanging themselves.

Lada

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 25 Apr 2018 15:55:50 +0200

> should do something else?

One option is that people use yang-data as defined in RFC 8040 until
there is a version of YANG that has a proper and complete integrated
solution. (If for example yang-data is used to declare error content
for RPCs, then more extensions are needed or a proper integration into
YANG. Is it really good to introduce augment-yang-data (instead of
making augment work with say 'data' in YANG 1.2)? And then we do
uses-yang-data etc.

/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 25 Apr 2018 16:05:38 +0200

> One option is that people use yang-data as defined in RFC 8040 until

IMO, people should use plain YANG. With the new YANG library it will be possible
to confine such non-NM schemas in a special datastore so that the intention
should be clear and multi-module schemas with all the additional data (versions,
 features, deviations) can be used.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Wed, 25 Apr 2018 08:04:39 -0700

>  features, deviations) can be used.
I don't see how yang-data interferes with "plain YANG" at all.
It is for data that is not in scope for plain YANG.
A plain client can ignore yang-data and not affect and RPC, notification,
or data
definitions in plain YANG.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 25 Apr 2018 17:14:07 +0200

> It is for data that is not in scope for plain YANG.

My question is why this extension is needed in the first place.

> A plain client can ignore yang-data and not affect and RPC, notification, or
> data

A plain (NC/RC) client should never see such data even if it is not protected by
yang-data in YANG. On the other hand, tools will be able to process such schemas
(generate the ascii tree, convert it to something else, generate sample
instances etc.) without explicitly supporting yang-data.

Lada

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 26 Apr 2018 07:53:26 +0200

> My question is why this extension is needed in the first place.

For example, RFC 8040 could have used two modules instead of
"ietf-restconf", one with the contents of grouping "errors" and the
other with the contents of grouping "restconf". No extension.

What would be wrong with this solution? Instead, the reader is
overwhelmed with the complexity of the "yang-data" definition, and most
tools cannot process the module.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Thu, 26 Apr 2018 17:52:00 -0700

On Wed, Apr 25, 2018 at 10:53 PM, Ladislav Lhotka <[email-masked]> wrote:

> >> > > > > > I am not sure what this statement tells us re. the issue in
> this
> >> > email
> >> > > > > > thread.
> >> > > > >
> >> > IMO, people should use plain YANG. With the new YANG library it will
> be
> >> > possible
This is true. We used to do this before yang-data was available.

> tools cannot process the module.

There are tools that can use yang-data.
Not all use-cases involve a server to query for a yang-library.
Offline tools need to know about the special data somehow.
The yang-data extension prevents data-def-stmts from being treated
as if they were configuration or operational data.

I agree with you that unconstrained use of yang-data is questionable
for a standard extension. The bar should be that all tools which choose
to implement the extension should provide the user with the same behavior.
Declaring that behavior out-of-scope does not help interoperability at all.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 27 Apr 2018 11:41:03 +0200

> This is true. We used to do this before yang-data was available.

If I remember correctly, the stuff was inside groupings that were not used
anywhere.

> Not all use-cases involve a server to query for a yang-library.

Sure, but it is not necessary, I meant it just as an option. Such YANG modules
can be passed straight to tools.

> Offline tools need to know about the special data somehow.

Why? Let's say I want the ascii tree, and pyang will be able to generate it. All
right, there will be "rw" labels that don't apply but it is not a big deal.

> as if they were configuration or operational data.

This would be a problem if this yang-data is mixed with standard data in the
same module. IMO this can be avoided, and then for it is essentially irrelevant
for tools whether it is normal data or not.

> Declaring that behavior out-of-scope does not help interoperability at all.

Yes, and so my proposal here is to silently misuse YANG somewhat where necessary
rather than spend cycles on a Standard Track document that gives a false
impression of a general solution.

It would be great to remove NETCONF specifics from YANG and I'd be willing to
contribute to this work.

Lada

> > or

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 27 Apr 2018 12:03:25 +0200 (CEST)

Hi,

> If I remember correctly, the stuff was inside groupings that were not used

Which doesn't quite work, since no namespace is attached to the nodes.

> impression of a general solution.

I am strongly opposed to this.  IMO it is much better to put such
structures in an extension, which tools that don't understand it will
ignore, than relying on description statements in normal data nodes,
which no tool can understand without hard coding special cases.

> It would be great to remove NETCONF specifics from YANG and I'd be willing to

This is a different topic though.

/martin

---

From: Robert Wilton <[email-masked]>
Date: Fri, 27 Apr 2018 11:23:42 +0100

> Which doesn't quite work, since no namespace is attached to the nodes.
OK.  So, using plain groupings doesn't work.

In cases where groupings are being used within a YANG defined RPC, then 
presumably they do work OK?

Is the specific problem scenario where the data is external to YANG 
defined RPCs, but yet it is still desirable to use a YANG schema and one 
of the associated YANG encodings to describe/encode the data?

> which no tool can understand without hard coding special cases.
I'm also opposed to this.

Stuff that looks like configuration should be configuration, and stuff 
that looks like state should be state.  If this data is going to be 
described in YANG then I think that there must be a programmatic way to 
indicate that the resultant schema is not configuration or operational 
state, but something else instead.  An extension seems to achieve this.

Thanks,
Rob

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 27 Apr 2018 13:03:58 +0200

On Fri, 2018-04-27 at 11:23 +0100, Robert Wilton wrote:
> > > > > > > > > > > It tells us that, in my view, the approach taken in this
> > > > > > > > > > > document
> > > > > 
> > > > > is a
> > > > > > > > > > > bad idea.
> > > > > > > > > > 
> > > > > > > > IMO, people should use plain YANG. With the new YANG library it
> > > > > > > > will be
> > > > > > > > possible
> > > > > > > > to confine such non-NM schemas in a special datastore so that
> > > > > > > > the
> > > > > 
> > > > > intention
> > > > > > > > should be clear and multi-module schemas with all the additional
> > > > > > > > data
> > > > > > > > (versions,

Note that this is not what I was proposing. For RESTCONF errors, the module I
had in mind could be like this:

module ietf-restconf-errors {

  container errors { // same content as in RFC 8040 
    ...
  }

  ...

}

Please explain why this cannot serve the given purpose, apart from the fact that
it looks like configuration which it isn't - but this can be explained in the
module description.

With this module, one could validate error messages using generic YANG tools
etc.

(I am not proposing to update RFC 8040, just using it as an example.)

> state, but something else instead.  An extension seems to achieve this.

YANG spec deals exclusively with configuration and state data, and using its
statements inside an extension doesn't make this basic fact go away. Specifying
all necessary changes properly inside a description of an extension is simply
impossible.

Lada

> > > > > > A plain (NC/RC) client should never see such data even if it is not
> > > > > 
> > > > > protected by
> > > > > > yang-data in YANG. On the other hand, tools will be able to process
> > > > > > such
> > > > > 
> > > > > schemas
> > > > > > > > > for RPCs, then more extensions are needed or a proper
> > > > > > > > > integration
> > > > > 
> > > > > into

---

From: Robert Wilton <[email-masked]>
Date: Fri, 27 Apr 2018 12:19:41 +0100

> it looks like configuration which it isn't - but this can be explained in the
It is the "because it looks like configuration" that I don't like.

If the server supports and advertises this module then it is reasonable 
to expect that a client should be able to configure the errors 
container, since it is configuration ...

At least marking it as config false would be slightly better.

> all necessary changes properly inside a description of an extension is simply
If an implementation needs to support generating the error messages then 
they can support the yang-data extension if they want (or just hard code 
what they expect to receive).

Otherwise, devices can also ignore the yang-data extension and it 
doesn't seem to do any harm since its doesn't change the behaviour in 
any way.

Thanks,
Rob

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 27 Apr 2018 16:34:38 +0200

On Fri, 2018-04-27 at 12:19 +0100, Robert Wilton wrote:
> > > > > > > > > On Wed, Apr 25, 2018 at 7:05 AM, Ladislav Lhotka <[email-masked]
> > > > > > > > > z>
> > > > > > > > > wrote:
> > > > > > > > > > > > > > I am not sure what this statement tells us re. the
> > > > > > > > > > > > > > issue
> > > > > > > > > > > > > > in
> > > > > > > 
> > > > > > > this
> > > > > > > > > > email
> > > > > > > > > > > > > > thread.
> > > > > > > > > > > > > 
> > > > > > > > > > > > people that need yang-data do not have a valid use case
> > > > > > > > > > > > and
> > > > > > > > > > > > they
> > > > > > > > > > > One option is that people use yang-data as defined in RFC
> > > > > > > > > > > 8040
> > > > > > > > > > > until
> > > > > > > > > > 

IMO this won't change even if the same data is inside the "yang-data" extension
because RFC 7950 says in sec. 7.21.1:

   If the top node does not specify a "config" statement, the default is "true".

So even though the description of "yang-data" says that the "config" statement
is ignored if present, it doesn't mean that schema nodes lose the config
property.

This may look like nitpicking but it illustrates the general problem: YANG was
designed for a very specific context and using it elsewhere requires creative
interpretation of its spec, with "yang-data" extension or without.

> container, since it is configuration ...

There is no point for the server to advertise it as a part of the standard
datastores such as "intended". And in order to advertise it as a schema for
error messages, it is IMO possible to use the trick that I suggested: define a
special datastore for it, such as "error-messages". So it will be the datastore
that enforces the desired semantics, and clients that support it can use it in
the intended way.

> > > > > Why? Let's say I want the ascii tree, and pyang will be able to
> > > > > generate
> > > > > it. All
> > > > > This would be a problem if this yang-data is mixed with standard data
> > > > > in
> > > > > the
> > > > > > Declaring that behavior out-of-scope does not help interoperability
> > > > > > at
> > > > > > all.
> > > > > 

Or support the "error-messages" datastore, see above.

Lada

> > > > > It would be great to remove NETCONF specifics from YANG and I'd be
> > > > > willing
> > > > > to
> > > > > contribute to this work.
> > > > 
> > > > > > > > > > > solution. (If for example yang-data is used to declare
> > > > > > > > > > > error
> > > > > > > > > > > content
> > > > > > > > > > > making augment work with say 'data' in YANG 1.2)? And then
> > > > > > > > > > > we
> > > > > > > > > > > do
> > > > > > > > > > > uses-yang-data etc.
> > > > > > > > > > > 
> > > > > > > > > > > /js
> > > > > > > > > > > 
> > > > > > > > > > 
> > > > > > > > > > -- 
> > > > > > > > > > Ladislav Lhotka
> > > > > > > > > > Head, CZ.NIC Labs

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Fri, 27 Apr 2018 16:47:08 +0200

On Fri, Apr 27, 2018 at 04:34:38PM +0200, Ladislav Lhotka wrote:

> [...] define a special datastore for it, such as "error-messages".

This seems worse than using, well, RFC 8040 yang-data. The proper
clear solution for RPCs and actions would be to enable the definition
of error details right in the rpc/action definition (input, output,
error).

But something like yang-data seems to have uses in other contexts.

/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 27 Apr 2018 17:13:20 +0200

> This seems worse than using, well, RFC 8040 yang-data. The proper

Why it seems worse?

Agreed.

> But something like yang-data seems to have uses in other contexts.

So other datastores may be defined. I assume that YANG library data can be used
independently, not just on a NC/RC server, pretty much along the lines of draft-
lengyel-netmod-yang-instance-data.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Fri, 27 Apr 2018 08:58:32 -0700

> > This seems worse than using, well, RFC 8040 yang-data. The proper
Because this is not part of the NMDA.
IMO, the yang-data defined in RFC 8040 has a clear purpose, and it
is sufficient for that purpose, which is a YANG representation of
an instance document (such as a protocol message or file).

YANG is useful for defining data structures that can be represented in
different formats, yet it is independent of any 1 format.

I am in favor if keeping the yang-data in RFC 8040 and not
creating a new version of it that is unconstrained.
There does not seem to be consensus on how to do this,
or even consensus that it is a good idea.

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Fri, 27 Apr 2018 18:28:49 +0200

> > This seems worse than using, well, RFC 8040 yang-data. The proper

Unnecessary complexity.

/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 30 Apr 2018 16:09:43 +0200

> Because this is not part of the NMDA.

NMDA is extensible.

> an instance document (such as a protocol message or file).

The same is basically true even without the extension. For example, I
fail to see any benefit from using yang-data in a module like
ietf-zerotouch-information.

> different formats, yet it is independent of any 1 format.

Of course I see this potential. Unfortunately, YANG spec was explicitly
written for a very specific context. Using an extension for removing
this specificity is IMO an ugly hack that undermines the architecture. 

> or even consensus that it is a good idea.

If the extension is deemed necessary, I would also prefer this approach
to making the extension a Proposed Standard.

Lada

---

From: Kent Watsen <[email-masked]>
Date: Mon, 30 Apr 2018 17:34:23 +0000

>                                 + '/m:my-first-rpc-error-info {

Can the definition be inlined, like this?

  module my-module {
    ...
    rpc my-first-rpc {
      input {
         ...
      }
      output {
         ...
      }
      opx:error-info-structure {
        leaf error-type {
          type enumeration {
            enum error-a;
            enum error-b;
            enum error-c;
          }
        }
        choice error-info {
          case error-a {
            ...
          }
          case error-b {
            ...
          }
          case error-c {
            ...
          }
        }     
      }
    }
  }

Kent // contributor

---

From: Andy Bierman <[email-masked]>
Date: Mon, 30 Apr 2018 10:48:31 -0700

> > Because this is not part of the NMDA.

If your only tool is a hammer, then all your problems look like nails.
I fail to see how an "error-messages" datastore fits in with NMDA

> ietf-zerotouch-information.

I don't see the benefit of pretending all data-def-stmts represent
configuration or operational data.

Wrapping the data-def-stmts in an extension says "this is not configuration
or operational data".  This is useful for tools that can process YANG in
other contexts.

> this specificity is IMO an ugly hack that undermines the architecture.

I don't see the architecture as fragile or damaged if yang-data is used.

People are going to continue to push the boundaries of YANG capabilities.
This will happen with or without the IETF. Maybe this work should remain
proprietary or get moved to an opensource project.

Andy

> >> So other datastores may be defined. I assume that YANG library data can
> be
> >> used
> >> independently, not just on a NC/RC server, pretty much along the lines
> of
> >> draft-

---

From: Kent Watsen <[email-masked]>
Date: Mon, 30 Apr 2018 17:57:34 +0000

Lada writes:
> Andy writes:

I don't understand this, how else would you propose to define the
JSON-or-XML encoded payload of the CMS structure?

> If the extension is deemed necessary, I would also prefer this 
> approach to making the extension a Proposed Standard.

I don't understand talk about abandoning this draft.  There is no question
that it is needed (e.g., anima vouch, zerotouch, tail-f's "structure"),
and RFC 8040 is unsatisfactory because 1) it doesn't allow a top-level
'choice' between two containers and 2) it requires drafts to reference 
RFC 8040, even though the drafts may have nothing to do with RESTCONF.

Sure, maybe we have convinced ourselves that yang-data is not needed
to model error-info, but that doesn't negate the other use case.

We need a way to indicate that some data-model represents a stand-alone
data structure.  It is not configuration, operational state, an RPC, or
a notification.  It can only appear as a descendent of the 'module'
statement.  All 'action', 'notification', 'config', and 'if-feature'
descendent statements are ignored.  For the purpose of 'must' and '
when' statements, the yang-data structure is its own context root.  It
has a namespace and a unique local name (unique only to other yang-data
defined within the same module; it's okay if an RPC, notification, or
top-level data node has the same name).  Anything else?

Kent // contributor