Subject: Re: [netmod] yang-data-ext issues

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 1 May 2018 09:34:48 +0200

On Mon, Apr 30, 2018 at 05:57:34PM +0000, Kent Watsen wrote:
> 
> I don't understand talk about abandoning this draft.  There is no question
> that it is needed (e.g., anima vouch, zerotouch, tail-f's "structure"),
> and RFC 8040 is unsatisfactory because 1) it doesn't allow a top-level
> 'choice' between two containers and 2) it requires drafts to reference 
> RFC 8040, even though the drafts may have nothing to do with RESTCONF.
>

Re 1: RFC 8040 says: "It MUST contain data definition statements that
result in exactly one container data node definition." So a choice may
actually work as long as the result is exactly one container data
node. OK, the wording in the RFC 8040 statement is not clear since
'result' and 'definition' do not line up (does 'result' mean the
toplevel data node instances that are possible? In this case,
'definition' would be misleading).

Re 2: It does not really matter whether you import the extension from
RFC 8040 or some other module. Why is depending on A better than
depending on B? The definition in RFC 8040 is already know by tools.

I view the yang-data definition of RFC 8040 as a temporary solution, a
proper solution should in my view be part of YANG 1.x.

Since NMDA essentially binds all data tree definitions to datastores,
the yang-data construct allows us to define data structures that are
specifically not bound to any datastore, i.e., data structures that by
design can't be operated on directly with NMDA NETCONF/RESTCONF.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1 | 28759 Bremen | Germany
Fax:   +49 421 200 3103         <https://www.jacobs-university.de/>

---

From: Kent Watsen <[email-masked]>
Date: Tue, 1 May 2018 20:33:58 +0000

Juergen writes:
> Kent writes:

I agree, but Martin did not and he even modified `pyang` to throw an 
error on the zerotouch draft, thus forcing the current situation.  If
a new understanding regarding rc:yang-data can be reached, the zerotouch
draft can go back to using it.

> proper solution should in my view be part of YANG 1.x.

What is the "proper solution", and why does it need to wait for YANG 1.x?
In the discussion regarding moving faster, it's been suggested that YANG 
1.x could be a cherry-picking of some number of extension statements 
produced in other I-Ds.  I was hoping that this was the case here.  True,
some tools know about A, but if B is the LTS, then I'd hope to move to B
ASAP.

> design can't be operated on directly with NMDA NETCONF/RESTCONF.

Yes.

Kent // contributor

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 1 May 2018 22:43:27 +0200

> draft can go back to using it.

Well, the statement in RFC 8040 is not clear (to me).
 
> some tools know about A, but if B is the LTS, then I'd hope to move to B

You will end up with yang-data defined in ietf-restconf, yang-data
defined in ietf-yang-data-ext, and something long-term stable in YANG
1.x. Do people really believe that creating three variants will be
simpler and more effective? The experiment is yang-data in
ietf-restconf - why do we need another experiment?

/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 02 May 2018 08:54:18 +0200

Kent Watsen <[email-masked]> writes:

> Lada writes:
>> Andy writes:
>>>IMO, the yang-data defined in RFC 8040 has a clear purpose, and it
>>>is sufficient for that purpose, which is a YANG representation of
>>>an instance document (such as a protocol message or file).
>>
>> The same is basically true even without the extension. For example, I
>> fail to see any benefit from using yang-data in a module like
>> ietf-zerotouch-information.
>
> I don't understand this, how else would you propose to define the
> JSON-or-XML encoded payload of the CMS structure?

Do the same, just remove the extension. I don't see any terrible things
happening, and *all* tools shoud be able to process it easily.

>
>
>>> I am in favor if keeping the yang-data in RFC 8040 and not
>>> creating a new version of it that is unconstrained.
>>> There does not seem to be consensus on how to do this,
>>> or even consensus that it is a good idea.
>>
>> If the extension is deemed necessary, I would also prefer this 
>> approach to making the extension a Proposed Standard.
>

So let me pose this question - why is it actually needed in these cases?
I can see some need whenever such data has to be mixed with normal
configuration and state data in the same module, but it is (I guess) not
the case in the above applications.

> Sure, maybe we have convinced ourselves that yang-data is not needed
> to model error-info, but that doesn't negate the other use case.
>
> We need a way to indicate that some data-model represents a stand-alone
> data structure.  It is not configuration, operational state, an RPC, or
> a notification.  It can only appear as a descendent of the 'module'

That's fine, but YANG wasn't designed to be a general schema language,
and I am against making such fundamental changes via extensions.

Lada

> statement.  All 'action', 'notification', 'config', and 'if-feature'
> descendent statements are ignored.  For the purpose of 'must' and '
> when' statements, the yang-data structure is its own context root.  It
> has a namespace and a unique local name (unique only to other yang-data
> defined within the same module; it's okay if an RPC, notification, or
> top-level data node has the same name).  Anything else?
>
>
> Kent // contributor
>
>
>

-- 
Ladislav Lhotka
Head, CZ.NIC Labs
PGP Key ID: 0xB8F92B08A9F76C67

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 02 May 2018 09:16:53 +0200 (CEST)

Ladislav Lhotka <[email-masked]> wrote:
> On Fri, 2018-04-27 at 12:19 +0100, Robert Wilton wrote:
> > 
> > On 27/04/2018 12:03, Ladislav Lhotka wrote:
> > > On Fri, 2018-04-27 at 11:23 +0100, Robert Wilton wrote:
> > > > On 27/04/2018 11:03, Martin Bjorklund wrote:
> > > > > Hi,
> > > > > 
> > > > > Ladislav Lhotka <[email-masked]> wrote:
> > > > > > On Thu, 2018-04-26 at 17:52 -0700, Andy Bierman wrote:
> > > > > > > On Wed, Apr 25, 2018 at 10:53 PM, Ladislav Lhotka <[email-masked]>
> > > > > > > wrote:
> > > > > > > > Ladislav Lhotka <[email-masked]> writes:
> > > > > > > > 
> > > > > > > > > On Wed, 2018-04-25 at 08:04 -0700, Andy Bierman wrote:
> > > > > > > > > > On Wed, Apr 25, 2018 at 7:05 AM, Ladislav Lhotka <[email-masked]
> > > > > > > > > > z>
> > > > > > > > > > wrote:
> > > > > > > > > > > On Wed, 2018-04-25 at 15:55 +0200, Juergen Schoenwaelder
> > > > > > > > > > > wrote:
> > > > > > > > > > > > On Tue, Apr 24, 2018 at 04:36:01PM +0200, Martin Bjorklund
> > > > > > > > > > > > wrote:
> > > > > > > > > > > > > Ladislav Lhotka <[email-masked]> wrote:
> > > > > > > > > > > > > > Martin Bjorklund <[email-masked]> writes:
> > > > > > > > > > > > > > 
> > > > > > > > > > > > > > > Hi,
> > > > > > > > > > > > > > > 
> > > > > > > > > > > > > > > I am not sure what this statement tells us re. the
> > > > > > > > > > > > > > > issue
> > > > > > > > > > > > > > > in
> > > > > > > > 
> > > > > > > > this
> > > > > > > > > > > email
> > > > > > > > > > > > > > > thread.
> > > > > > > > > > > > > > 
> > > > > > > > > > > > > > It tells us that, in my view, the approach taken in
> > > > > > > > > > > > > > this
> > > > > > > > > > > > > > document
> > > > > > > > 
> > > > > > > > is a
> > > > > > > > > > > > > > bad idea.
> > > > > > > > > > > > > 
> > > > > > > > > > > > > Do you mean that the WG shoud drop this document?  And
> > > > > > > > > > > > > people that
> > > > > > > > > > > > > need yang-data should continue to use the version in
> > > > > > > > > > > > > 8040?  Or that
> > > > > > > > > > > > > people that need yang-data do not have a valid use case
> > > > > > > > > > > > > and
> > > > > > > > > > > > > they
> > > > > > > > > > > > > should do something else?
> > > > > > > > > > > > 
> > > > > > > > > > > > One option is that people use yang-data as defined in RFC
> > > > > > > > > > > > 8040
> > > > > > > > > > > > until
> > > > > > > > > > > 
> > > > > > > > > > > IMO, people should use plain YANG. With the new YANG library
> > > > > > > > > > > it
> > > > > > > > > > > will be
> > > > > > > > > > > possible
> > > > > > > > > > > to confine such non-NM schemas in a special datastore so
> > > > > > > > > > > that
> > > > > > > > > > > the
> > > > > > > > 
> > > > > > > > intention
> > > > > > > > > > > should be clear and multi-module schemas with all the
> > > > > > > > > > > additional
> > > > > > > > > > > data
> > > > > > > > > > > (versions,
> > > > > > > > > > >    features, deviations) can be used.
> > > > > > > > > > > 
> > > > > > > > > > 
> > > > > > > > > > I don't see how yang-data interferes with "plain YANG" at all.
> > > > > > > > > > It is for data that is not in scope for plain YANG.
> > > > > > > > > 
> > > > > > > > > My question is why this extension is needed in the first place.
> > > > > > > > 
> > > > > > > > For example, RFC 8040 could have used two modules instead of
> > > > > > > > "ietf-restconf", one with the contents of grouping "errors" and
> > > > > > > > the
> > > > > > > > other with the contents of grouping "restconf". No extension.
> > > > > > > > 
> > > > > > > 
> > > > > > > This is true. We used to do this before yang-data was available.
> > > > > > 
> > > > > > If I remember correctly, the stuff was inside groupings that were not
> > > > > > used
> > > > > > anywhere.
> > > > > 
> > > > > Which doesn't quite work, since no namespace is attached to the nodes.
> > > 
> > > Note that this is not what I was proposing. For RESTCONF errors, the module
> > > I
> > > had in mind could be like this:
> > > 
> > > module ietf-restconf-errors {
> > > 
> > >    container errors { // same content as in RFC 8040
> > >      ...
> > >    }
> > > 
> > >    ...
> > > 
> > > }
> > > 
> > > Please explain why this cannot serve the given purpose, apart from the fact
> > > that
> > > it looks like configuration which it isn't - but this can be explained in
> > > the
> > > module description.
> > 
> > It is the "because it looks like configuration" that I don't like.
> 
> IMO this won't change even if the same data is inside the "yang-data" extension
> because RFC 7950 says in sec. 7.21.1:
> 
>    If the top node does not specify a "config" statement, the default is "true".
> 
> So even though the description of "yang-data" says that the "config" statement
> is ignored if present, it doesn't mean that schema nodes lose the config
> property.

No, this is not correct.  7.21.1 says:

   If "config" is not specified, the default is the same as the parent
   schema node's "config" value.

Note how it says *schema node*.  Nodes in a grouping that don't have a
config statement don't have a config property.  Also, the same text
about ignoring config is present for input/output/notfication.

Nodes in a yang-data extension statement thus also do not have a
config value, since yang-data is not a data definition statement.

This is an argument *for* an extension statement.  If you just define
special nodes as normal nodes with a special description statement,
you really break the rules for YANG.

/martin

> This may look like nitpicking but it illustrates the general problem: YANG was
> designed for a very specific context and using it elsewhere requires creative
> interpretation of its spec, with "yang-data" extension or without.
> 
> > 
> > If the server supports and advertises this module then it is reasonable 
> > to expect that a client should be able to configure the errors 
> > container, since it is configuration ...
> 
> There is no point for the server to advertise it as a part of the standard
> datastores such as "intended". And in order to advertise it as a schema for
> error messages, it is IMO possible to use the trick that I suggested: define a
> special datastore for it, such as "error-messages". So it will be the datastore
> that enforces the desired semantics, and clients that support it can use it in
> the intended way.
> 
> > 
> > At least marking it as config false would be slightly better.
> > 
> > > 
> > > With this module, one could validate error messages using generic YANG tools
> > > etc.
> > > 
> > > (I am not proposing to update RFC 8040, just using it as an example.)
> > > 
> > > > OK.  So, using plain groupings doesn't work.
> > > > 
> > > > In cases where groupings are being used within a YANG defined RPC, then
> > > > presumably they do work OK?
> > > > 
> > > > Is the specific problem scenario where the data is external to YANG
> > > > defined RPCs, but yet it is still desirable to use a YANG schema and one
> > > > of the associated YANG encodings to describe/encode the data?
> > > > 
> > > > 
> > > > > > > > What would be wrong with this solution? Instead, the reader is
> > > > > > > > overwhelmed with the complexity of the "yang-data" definition, and
> > > > > > > > most
> > > > > > > > tools cannot process the module.
> > > > > > > 
> > > > > > > There are tools that can use yang-data.
> > > > > > > Not all use-cases involve a server to query for a yang-library.
> > > > > > 
> > > > > > Sure, but it is not necessary, I meant it just as an option. Such YANG
> > > > > > modules
> > > > > > can be passed straight to tools.
> > > > > > 
> > > > > > > Offline tools need to know about the special data somehow.
> > > > > > 
> > > > > > Why? Let's say I want the ascii tree, and pyang will be able to
> > > > > > generate
> > > > > > it. All
> > > > > > right, there will be "rw" labels that don't apply but it is not a big
> > > > > > deal.
> > > > > > 
> > > > > > > The yang-data extension prevents data-def-stmts from being treated
> > > > > > > as if they were configuration or operational data.
> > > > > > 
> > > > > > This would be a problem if this yang-data is mixed with standard data
> > > > > > in
> > > > > > the
> > > > > > same module. IMO this can be avoided, and then for it is essentially
> > > > > > irrelevant
> > > > > > for tools whether it is normal data or not.
> > > > > > 
> > > > > > > I agree with you that unconstrained use of yang-data is questionable
> > > > > > > for a standard extension. The bar should be that all tools which
> > > > > > > choose
> > > > > > > to implement the extension should provide the user with the same
> > > > > > > behavior.
> > > > > > > Declaring that behavior out-of-scope does not help interoperability
> > > > > > > at
> > > > > > > all.
> > > > > > 
> > > > > > Yes, and so my proposal here is to silently misuse YANG somewhat where
> > > > > > necessary
> > > > > > rather than spend cycles on a Standard Track document that gives a
> > > > > > false
> > > > > > impression of a general solution.
> > > > > 
> > > > > I am strongly opposed to this.  IMO it is much better to put such
> > > > > structures in an extension, which tools that don't understand it will
> > > > > ignore, than relying on description statements in normal data nodes,
> > > > > which no tool can understand without hard coding special cases.
> > > > 
> > > > I'm also opposed to this.
> > > > 
> > > > Stuff that looks like configuration should be configuration, and stuff
> > > > that looks like state should be state.  If this data is going to be
> > > > described in YANG then I think that there must be a programmatic way to
> > > > indicate that the resultant schema is not configuration or operational
> > > > state, but something else instead.  An extension seems to achieve this.
> > > 
> > > YANG spec deals exclusively with configuration and state data, and using its
> > > statements inside an extension doesn't make this basic fact go away.
> > > Specifying
> > > all necessary changes properly inside a description of an extension is
> > > simply
> > > impossible.
> > 
> > If an implementation needs to support generating the error messages then 
> > they can support the yang-data extension if they want (or just hard code 
> > what they expect to receive).
> 
> Or support the "error-messages" datastore, see above.
> 
> Lada
> 
> > 
> > Otherwise, devices can also ignore the yang-data extension and it 
> > doesn't seem to do any harm since its doesn't change the behaviour in 
> > any way.
> > 
> > Thanks,
> > Rob
> > 
> > 
> > > 
> > > Lada
> > > 
> > > > Thanks,
> > > > Rob
> > > > 
> > > > 
> > > > > 
> > > > > > It would be great to remove NETCONF specifics from YANG and I'd be
> > > > > > willing
> > > > > > to
> > > > > > contribute to this work.
> > > > > 
> > > > > This is a different topic though.
> > > > > 
> > > > > 
> > > > > /martin
> > > > > 
> > > > > 
> > > > > > Lada
> > > > > > 
> > > > > > > > Lada
> > > > > > > > 
> > > > > > > 
> > > > > > > Andy
> > > > > > >    
> > > > > > > > > > A plain client can ignore yang-data and not affect and RPC,
> > > > > > > > > > notification,
> > > > > > > > 
> > > > > > > > or
> > > > > > > > > > data
> > > > > > > > > > definitions in plain YANG.
> > > > > > > > > 
> > > > > > > > > A plain (NC/RC) client should never see such data even if it is
> > > > > > > > > not
> > > > > > > > 
> > > > > > > > protected by
> > > > > > > > > yang-data in YANG. On the other hand, tools will be able to
> > > > > > > > > process
> > > > > > > > > such
> > > > > > > > 
> > > > > > > > schemas
> > > > > > > > > (generate the ascii tree, convert it to something else, generate
> > > > > > > > > sample
> > > > > > > > > instances etc.) without explicitly supporting yang-data.
> > > > > > > > > 
> > > > > > > > > Lada
> > > > > > > > > 
> > > > > > > > > >    
> > > > > > > > > > > Lada
> > > > > > > > > > > 
> > > > > > > > > > 
> > > > > > > > > > Andy
> > > > > > > > > >    
> > > > > > > > > > > > there is a version of YANG that has a proper and complete
> > > > > > > > > > > > integrated
> > > > > > > > > > > > solution. (If for example yang-data is used to declare
> > > > > > > > > > > > error
> > > > > > > > > > > > content
> > > > > > > > > > > > for RPCs, then more extensions are needed or a proper
> > > > > > > > > > > > integration
> > > > > > > > 
> > > > > > > > into
> > > > > > > > > > > > YANG. Is it really good to introduce augment-yang-data
> > > > > > > > > > > > (instead of
> > > > > > > > > > > > making augment work with say 'data' in YANG 1.2)? And then
> > > > > > > > > > > > we
> > > > > > > > > > > > do
> > > > > > > > > > > > uses-yang-data etc.
> > > > > > > > > > > > 
> > > > > > > > > > > > /js
> > > > > > > > > > > > 
> > > > > > > > > > > 
> > > > > > > > > > > -- 
> > > > > > > > > > > Ladislav Lhotka
> > > > > > > > > > > Head, CZ.NIC Labs
> > > > > > > > > > > _______________________________________________
> > > > > > > > > > > netmod mailing list
> > > > > > > > > > > [email-masked]
> > > > > > > > > > > https://www.ietf.org/mailman/listinfo/netmod
> > > > > > > > > 
> > > > > > > > > -- 
> > > > > > > > > Ladislav Lhotka
> > > > > > > > > Head, CZ.NIC Labs
> > > > > > > > > _______________________________________________
> > > > > > > > > netmod mailing list
> > > > > > > > > [email-masked]
> > > > > > > > > https://www.ietf.org/mailman/listinfo/netmod
> > > > > > > > 
> > > > > > > > _______________________________________________
> > > > > > > > netmod mailing list
> > > > > > > > [email-masked]
> > > > > > > > https://www.ietf.org/mailman/listinfo/netmod
> > > > > > 
> > > > > > -- 
> > > > > > Ladislav Lhotka
> > > > > > Head, CZ.NIC Labs
> > > > > > _______________________________________________
> > > > > > netmod mailing list
> > > > > > [email-masked]
> > > > > > https://www.ietf.org/mailman/listinfo/netmod
> > > > > > 
> > > > > 
> > > > > _______________________________________________
> > > > > netmod mailing list
> > > > > [email-masked]
> > > > > https://www.ietf.org/mailman/listinfo/netmod
> > > > > .
> > > > > 
> > 
> > 
> -- 
> Ladislav Lhotka
> Head, CZ.NIC Labs

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 02 May 2018 09:25:27 +0200 (CEST)

Andy Bierman <[email-masked]> wrote:
> On Mon, Apr 30, 2018 at 7:09 AM, Ladislav Lhotka <[email-masked]> wrote:
> 
> > Andy Bierman <[email-masked]> writes:
> >
> > > On Fri, Apr 27, 2018 at 8:13 AM, Ladislav Lhotka <[email-masked]> wrote:
> > >
> > >> On Fri, 2018-04-27 at 16:47 +0200, Juergen Schoenwaelder wrote:
> > >> > On Fri, Apr 27, 2018 at 04:34:38PM +0200, Ladislav Lhotka wrote:
> > >> >
> > >> > > [...] define a special datastore for it, such as "error-messages".
> > >> >
> > >> > This seems worse than using, well, RFC 8040 yang-data. The proper
> > >>
> > >> Why it seems worse?
> > >>
> > >>
> > > Because this is not part of the NMDA.
> >
> > NMDA is extensible.
> >
> 
> 
> If your only tool is a hammer, then all your problems look like nails.
> I fail to see how an "error-messages" datastore fits in with NMDA
> 
> 
> 
> >
> I don't see the benefit of pretending all data-def-stmts represent
> configuration or operational data.
> 
> Wrapping the data-def-stmts in an extension says "this is not configuration
> or operational data".  This is useful for tools that can process YANG in
> other contexts.

I fully agree

> > > YANG is useful for defining data structures that can be represented in
> > > different formats, yet it is independent of any 1 format.
> >
> > Of course I see this potential. Unfortunately, YANG spec was explicitly
> > written for a very specific context. Using an extension for removing
> > this specificity is IMO an ugly hack that undermines the architecture.
> >
> >
> 
> I don't see the architecture as fragile or damaged if yang-data is used.
> 
> People are going to continue to push the boundaries of YANG capabilities.
> This will happen with or without the IETF.

Yes, and it already happens.

/martin

> Maybe this work should remain
> proprietary or get moved to an opensource project.
> 
> 
> 
> 
> > >
> > If the extension is deemed necessary, I would also prefer this approach
> > to making the extension a Proposed Standard.
> >
> > Lada
> >
> 
> 
> Andy
> 
> 
> >
> > >
> > >
> > >> > clear solution for RPCs and actions would be to enable the definition
> > >> > of error details right in the rpc/action definition (input, output,
> > >> > error).
> > >>
> > >> Agreed.
> > >>
> > >> >
> > >> > But something like yang-data seems to have uses in other contexts.
> > >>
> > >> So other datastores may be defined. I assume that YANG library data can
> > be
> > >> used
> > >> independently, not just on a NC/RC server, pretty much along the lines
> > of
> > >> draft-
> > >> lengyel-netmod-yang-instance-data.
> > >>
> > >> Lada
> > >>
> > >> >
> > >>
> > >
> > > Andy
> > >
> > >
> > >
> > >> > /js
> > >> >
> > >> --
> > >> Ladislav Lhotka
> > >> Head, CZ.NIC Labs

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 02 May 2018 09:27:51 +0200 (CEST)

> RFC 8040, even though the drafts may have nothing to do with RESTCONF.

and 3) it is not possible to augement 8040's yang-data.

> to model error-info

I don't think there have been any other proposal for how to model
error-info that have any sort of concensus behind it.

> , but that doesn't negate the other use case.
> 

/martin

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 02 May 2018 09:30:16 +0200 (CEST)

Kent Watsen <[email-masked]> wrote:
> error on the zerotouch draft

Originally pyang did not check this, but I was convinced by Andy and
the discussion on the ML that pyang was not correct.

The intention of the text in RFC 8040 was to allow for "uses" within a
yang-data statement.  It was not to allow for "choice".

>, thus forcing the current situation.  If
> produced in other I-Ds.  I was hoping that this was the case here.

+1

/martin

> True,

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 02 May 2018 09:52:25 +0200

On Wed, 2018-05-02 at 09:16 +0200, Martin Bjorklund wrote:
> > > > > > > > On Wed, Apr 25, 2018 at 10:53 PM, Ladislav Lhotka <[email-masked]
> >
> > > > > > > > wrote:

And also:

   If the top node does not specify a "config" statement, the default is
   "true".

> config value, since yang-data is not a data definition statement.

Are you saying that a container inside yang-data doesn't define a schema node?

> you really break the rules for YANG.

I wouldn't *define* the schema nodes as special, just the resulting schema may
be used differently. This shouldn't matter as long as this data has nothing to
do with NETCONF or RESTCONF.

My main objections against yang-data are:

- additional complexity

- bad example (Need - or don't like - something in YANG? Just introduce an
extension.)

Lada

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 02 May 2018 10:00:10 +0200 (CEST)

> Are you saying that a container inside yang-data doesn't define a schema node?

I mean that just like in groupings, nodes in yang-data don't have a
config property.

> - bad example (Need - or don't like - something in YANG? Just introduce an

extensions are *much* better than the alternative you propose (Need -
or don't like - something in YANG? Just document how you break the
rules in the description statement).

extensions have turned out to be really useful.   I agree that you
need to be careful with what you do though.

/martin

---

From: Robert Wilton <[email-masked]>
Date: Wed, 2 May 2018 10:02:42 +0100

>> or operational data".  This is useful for tools that can process YANG in

YANG already models RPCs, and 7950 makes it clear that the input/output 
parameters of those RPC statements are not configuration or state and 
are not modeled in datastores.  I.e.:

    Since the input tree is not part of any datastore, all "config"
    statements for nodes in the input tree are ignored.

    Since the output tree is not part of any datastore, all "config"
    statements for nodes in the output tree are ignored.

Isn't the YANG data extension just modelling fragments of YANG to be 
used in generic RPC messages?

This doesn't seem to be a fundamental change in YANG's scope, or 
architecture.

Thanks,
Rob

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 02 May 2018 11:25:06 +0200 (CEST)

Robert Wilton <[email-masked]> wrote:
> YANG already models RPCs, and 7950 makes it clear that the
> input/output parameters of those RPC statements are not configuration
> or state and are not modeled in datastores.  I.e.:
> 

The primary use case is not "generic RPC messages", but standalone
instance documents, error-info structures, etc.

> This doesn't seem to be a fundamental change in YANG's scope, or

I agree.

/martin

> >>> approach

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 2 May 2018 11:36:26 +0200

> > This doesn't seem to be a fundamental change in YANG's scope, or

The proper solution for rpcs and actions is to define error
information as part of the rpc/action. YANG 1.1 does not support
this but this is where it should be fixed.

Standalone instance documents (not tied to datastores) may have their
use cases as well but it feels odd to create support for standalone
instance documents as extensions and then to create even more
extensions to support augmentation of these instance documents and
whoever knows what comes next. For short-term needs, there is
yang-data defined in RFC 8040. The longer-term solution should IMHO be
a proper part of YANG and not an extension.

And if the current short-term solution requires an additional
container, then bam go for the additional container. If there is
serious pressure to use yang-data, then the additional container
should not stop people that need to use yang-data today.

/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 02 May 2018 12:35:12 +0200

> should not stop people that need to use yang-data today.

I agree with all of the above.

Lada

---

From: Robert Varga <[email-masked]>
Date: Fri, 4 May 2018 23:22:36 +0200

On 27/04/18 12:03, Martin Bjorklund wrote:
>> It would be great to remove NETCONF specifics from YANG and I'd be willing to
>> contribute to this work.

+1 and count me in, please.

Thanks,
Robert

---

From: Robert Varga <[email-masked]>
Date: Fri, 4 May 2018 23:32:13 +0200

On 27/04/18 12:03, Martin Bjorklund wrote:
>> If I remember correctly, the stuff was inside groupings that were not used
>> anywhere.

True, but that is can boil down to a simple data transplant:

1) treat the nodes declared in a grouping to be defined underneath their
containing grouping identifier
2) recognized when grouping instantiations are data-compatible (i.e.
disregarding namespace)

Unlike an extension, this behavior becomes default-on, not something you
opt into by using an extension -- which would necessitate a revision of
existing modules take advantage of.

Regards,
Robert

P.S.: OpenDaylight Java Bindinds do this to a certain, not perfect, extent.

---

From: Kent Watsen <[email-masked]>
Date: Tue, 29 May 2018 15:58:33 +0000

[resurrecting this thread]

Currently the zerotouch draft has a normative reference to this draft.
I will this week post an update to the zerotouch draft to resolve the
netconf list thread "a couple zerotouch-21 issues".   It would be easy
for me to also switch back to using rc:yang-data, but I won't do so if
this draft remains an active work-in-progress.

Please see below for more.

> this but this is where it should be fixed.

Agreed, but note that the subscribed-notifications draft (both the
published -12 and unpublished -13) are relying on being able to do
just this, and YANG-next is years away...

> whoever knows what comes next.

What feels "odd" about this?  Is it not using the extension statement
as it was intended?

> For short-term needs, there is yang-data defined in RFC 8040.

To be clear, the "short-term needs" are:

  a) zerotouch: to define a standalone instance document
  b) notification-messages: to define a new notification message
  c) subscribed-notifications: to define error-info structures

As I recall, this draft (not RFC 8040) is needed:

  - for (a), because rc:yang-data doesn't support a top-level
    "choice" statement spanning "container" statements.

  - for (b), in order to augment a base yang-data "message" 
    structure with additional nodes.

  - AFAIAA, RFC 8040 is sufficient for (c)

Has anything changed?   I don't think that we can un-adopt this
draft with said dependencies, right?

Kent

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 29 May 2018 19:09:00 +0200

> just this, and YANG-next is years away...

There is a description statement.

> What feels "odd" about this?  Is it not using the extension statement

For me, extensions that define new data definition statements are
borderline. RFC 7950 has this nice statement:

   o  extension: An extension attaches non-YANG semantics to statements.
      The "extension" statement defines new statements to express these
      semantics.

This does not help since we lack a definition for 'non-YANG semantics'
and yes I know that yang-data is today defined as an extension. But
for me, this is a hack and instead of creating a slightly more
generalized version of this hack, I prefer to stick to yang-data in
favor of a proper solution as part of YANG.
 
>     "choice" statement spanning "container" statements.

So create a container.
 
>     structure with additional nodes.

So you are creating another augmentation mechanism. I am concerned
about ending up with a zoo of different mechanisms if we go down this
path, we may end up with every project or vendor creating their own
variants.

With NMDA in place, YANG 1.1 is decribing schemas for datastores plus
operations and notifications. It is not a protocol message description
language or a standalone file format description language. If this is
needed, I prefer to create YANG X.Y - and if we manage the complexity
we have something that is ideally integrated and consistent.

> draft with said dependencies, right?

I am just voicing my opinion. It may very well be that the WG prefers
to go the route of not touching YANG 1.1 and instead patching around
its limitations with extensions.

My concern is simply driven that some want to patch in via extensions
support for describing protocol messages and standalone documents,
others want to patch via extensions and updates a different versioning
system, and who knows what comes next. In the long run, I am afraid
this will become a mess. And yes, it is always difficult to predict
the future - we need crystal balls. Perhaps as an extension. ;-)

/js

---

From: Andy Bierman <[email-masked]>
Date: Tue, 29 May 2018 12:21:23 -0700

> There is a description statement.

Automation tools generally use machine-readable statements, and avoid using
description-stmts.

The yang-data extensions do not solve this "rpc/action to error-info"
mapping
problem, so this problem is not a real factor here.

> favor of a proper solution as part of YANG.

The yang-data and augment-yang-data extensions do not re-define YANG data
nodes.
They define a new context for data nodes and prevent plain YANG tools from
confusing this new context with data for configuration or operational
datastores.

> >     "choice" statement spanning "container" statements.

I tend to agree that a top-level choice could be avoided.
This is not a really compelling use-case.

> we have something that is ideally integrated and consistent.

Yes it is a protocol message description language, but only
in an incomplete, hacky way.  Specific subtrees within protocol messages
are defined as special YANG data-defined portions and the rest is somehow
out of scope for YANG. The way nested roots (e.g., <config> or <data>)
are handled is the worst hack of all.  But IMO a new version of YANG is
needed
to really fix these problems.

> the future - we need crystal balls. Perhaps as an extension. ;-)

This is a separate (but important) topic -- YANG 1.1 + a vendor-selected
set of
optional extensions is not the same as a new version of YANG (with
mandatory-to-implement statements).

YANG designers cannot really rely on optional extensions across tool-chains.

Andy

---

From: Kent Watsen <[email-masked]>
Date: Tue, 29 May 2018 21:02:21 +0000

> Automation tools generally use machine-readable statements, and avoid
> using description-stmts.
>
> mapping problem, so this problem is not a real factor here.

If subscribed-notifications doesn't use "yang-data", then it would need
to use another extension statement, lest the 3 "error-info" statements
it defines are misconstrued as configuration or operational state.

> The yang-data and augment-yang-data extensions do not re-define YANG 
> data nodes. They define a new context for data nodes and prevent plain
> YANG tools from confusing this new context with data for configuration or 
> operational datastores.

Yes.

> This is not a really compelling use-case.

From https://mailarchive.ietf.org/arch/msg/netconf/T6OgCvbF30EOoU27b3T_-dOdj2E:

  The zerotouch draft has way too much text invested in the "zerotouch-
  information" artifact being polymorphic in nature to seriously consider
  having two unrelated artifacts, when the easier answer is to agree 
  that, in this case, the yang-data does in fact contain "data definition 
  statements that result in exactly one container data node definition".
  We don't even need to change any text, in this document [zerotouch] or
  in RFC 8040, in order to have this agreement.  We would only need 
  `pyang` to stop flagging it as an error.

So, yes, yang-data-ext isn't needed for zerotouch, there are options
on the table.  But, so long as this draft is a WIP, then it seems that 
zerotouch should use it.

> needed to really fix these problems.

Yes, we agree that a new version of YANG is ideal, but such is years away
and the NETCONF WG is not going to block these drafts waiting for it.  Do
we rather the notification-messages draft define a private version of the
"augment-yang-data" extension statement?

> set of optional extensions is not the same as a new version of YANG 
> (with mandatory-to-implement statements).
>
 
True, but it seems that this one might hit critical mass. Even if it doesn't,
scripts can auto-convert yang-data definitions into their descendent node 
definitions (container, or choice in zerotouch case) so existing tooling can
grok them.  I have scripts doing this today.

Kent