Subject: Re: [Netconf] RESTCONF #15: usage of groupings

From: "Ersue, Mehmet (NSN - DE/Munich)" <[email-masked]>
Date: Sun, 4 Jan 2015 19:01:19 +0000

Hi All,

we did not see any comments on Andy's proposed solution on Github since 2 weeks.
This issue is subject to close.

Mehmet 

> -----Original Message-----
> From: Netconf [mailto:[email-masked]] On Behalf Of ext Andy Bierman
> Sent: Monday, December 22, 2014 6:09 PM
> To: Ladislav Lhotka
> Cc: Netconf
> Subject: Re: [Netconf] RESTCONF #15: usage of groupings
> 
> Hi,
> 
> I updated the issue tracker with the concrete proposal for
> ietf-restconf groupings.
> 
> https://github.com/netconf-wg/restconf/issues/15
> 
> I forgot to mention that ietf-yang-patch.yang already uses the "errors" grouping
> within the rpc output parameters, so removing it will break YANG Patch.
> 
> Andy
> 
> 
> On Mon, Dec 15, 2014 at 10:49 AM, Ladislav Lhotka <[email-masked]> wrote:
> > Andy Bierman <[email-masked]> writes:
> >
> >> On Thu, Dec 11, 2014 at 4:48 AM, Martin Bjorklund <[email-masked]> wrote:
> >>> Ladislav Lhotka <[email-masked]> wrote:
> >>>>
> >>>> > On 11 Dec 2014, at 12:49, Martin Bjorklund <[email-masked]> wrote:
> >>>> >
> >>>> > Ladislav Lhotka <[email-masked]> wrote:
> >>>> >> Andy Bierman <[email-masked]> writes:
> >>>> >>
> >>>> >>> Hi,
> >>>> >>>
> >>>> >>> A new issue has been added to the tracker:
> >>>> >>>
> >>>> >>> https://github.com/netconf-wg/restconf/issues/15
> >>>> >>>
> >>>> >>> There are 3 solutions mentioned.
> >>>> >>> The default solution is S1, which will be implemented
> >>>> >>> unless there is WG consensus demonstrated to
> >>>> >>> do something else.
> >>>> >>
> >>>> >> The biggest problem with S1 is that it may mislead readers into
> >>>> >> thinking: "Hey, now I understand how namespaces work in groupings!"
> >>>> >>
> >>>> >> I also don't like the idea of defining an ad hoc extension.
> >>>> >
> >>>> > I disagree.  I think an extension is the best solution in this case.
> >>>> > However, we can use a different name than 'structure'.  'structure'
> >>>> > was originally my proposal for a built-in statement (YANG 1.1) for
> >>>> > this purpose.  We can call it 'restconf-structure' or 'restconf-data'
> >>>> > or something.
> >>>>
> >>>> So another document that wants to do the same should define its own ad
> >>>> hoc extension? Or import the ietf-restconf module?
> >>>
> >>> Suppose we make a new document that defines a new RESTCONF resource
> >>> type, and we need a structure for it, then that document would import
> >>> and use 'restconf-structure'.
> >>>
> >>>> >> If we want
> >>>> >> to turn YANG into a general schema language, it should be done
> >>>> >> properly.
> >>>> >>
> >>>> >> I think the simplest solution would be to remove the groupings and
> >>>> >> have
> >>>> >> "errors", "restconf", "collection" and "notification" as top-level
> >>>> >> containers. The namespace assignment then follows YANG rules.
> >>>> >
> >>>> > This is also _very_ misleading, b/c then it looks like a normal config
> >>>> > or state module, which one would expect a server to implement in the
> >>>> > normal way.
> >>>>
> >>>> What makes the solutions with groupings or extensions different?
> >>>
> >>> I agree that groupings are misleading.  But using an extension would
> >>> be different, b/c the extension defines the semantics - this is not
> >>> config or oper state, but something else.
> >>>
> >>
> >>
> >>
> >> I do not understand why these are special or misleading groupings.
> >> I cannot find any text in any RFC that says a YANG module with
> >> just groupings in it is invalid, improper, or even discouraged.
> >>
> >> I am already using 1 of these groupings in a logging model.
> >> Please explain why these are special groupings that cannot
> >> be used in a normal data model in another module.
> >>
> >> That is why I prefer solution S2-B
> >>
> >>   rc:structure errors {
> >>      uses errors;
> >>   }
> >>
> >> This assigns a module-name and a namespace and does not
> >> preclude re-usage of the data within a larger data structure.
> >
> > I don't want to block the resolution of this issue, so I propose that you
> > write a concrete definition of the extension, and I will then comment on
> > that.
> >
> > Lada
> >
> >>
> >>
> >>>> In
> >>>> any case, the module description has to state very clearly that it is
> >>>> a strawman module that’s not supposed to be used in the normal
> >>>> NETCONF/RESTCONF way. But if you above that deviate from YANG
> >>>> semantics, as the groupings do, you cannot even use standard YANG
> >>>> tools because they would produce wrong results.
> >>>>
> >>>> If the module is as I proposed, I can still use pyang for generating
> >>>> correct DSDL schemas, XML-JSON mapping etc.
> >>>>
> >>>> And if a server implementor likes the module and decides to implement
> >>>> it as if it was a normal module, so what?
> >>>>
> >>>> >
> >>>> >> S3 should IMO be fine, too, the structures aren't that complicated.
> >>>> >
> >>>> > I can see only drawbacks compared to S2.  Whatever we do in text it
> >>>> > will be less precise than an extension.
> >>>>
> >>>> I prefer less precise to precise but wrong.
> >>>
> >>> An extension is in no way wrong.
> >>>
> >>
> >> agreed
> >>
> >>>> Keep in mind that you are
> >>>> a YANG designer so you are not as likely to get confused by language
> >>>> anomalies as an average reader of the RESTCONF document.
> >>>
> >>>
> >>> /martin
> >>
> >> Andy
> >
> > --
> > Ladislav Lhotka, CZ.NIC Labs
> > PGP Key ID: E74E8C0C
> 
> _______________________________________________
> Netconf mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netconf

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 5 Jan 2015 10:23:07 +0100

> On 22 Dec 2014, at 18:09, Andy Bierman <[email-masked]> wrote:
> 
> Hi,
> 

I’d rather like to see the definition of the proposed extension, and I will comment on that.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Mon, 5 Jan 2015 05:20:38 -0800

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 5 Jan 2015 14:46:17 +0100

> On 05 Jan 2015, at 14:20, Andy Bierman <[email-masked]> wrote:
> 

The ietf-restconf module imports ietf-yang-types. How can an implementor figure out which revision of ietf-yang-types to use?

Lada

---

From: Andy Bierman <[email-masked]>
Date: Mon, 5 Jan 2015 05:59:07 -0800

> The ietf-restconf module imports ietf-yang-types. How can an implementor figure out which revision of ietf-yang-types to use?

Either the server advertises it or the ietf-netconf-monitoring
module contains that data.

How does this have anything to do with the restconf-resource extension?

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 5 Jan 2015 15:28:04 +0100

> On 05 Jan 2015, at 14:59, Andy Bierman <[email-masked]> wrote:
> 

IMO schemas outside the data and operation resources have to be fixed for the given RESTconf version.

> How does this have anything to do with the restconf-resource extension?

This has to do with using YANG as a general schema language, i.e. outside the normal NETCONF context.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Mon, 5 Jan 2015 07:12:41 -0800

> IMO schemas outside the data and operation resources have to be fixed for the given RESTconf version.

OK -- but the restconf-resource extension is on github
and the ietf-restconf module in draft-03 that does not
have an import-by-revision is not related to this issue.

However, I share concerns raised by Balazs that the increased complexity
and confusion that can result from trying to follow all the revision-specific
differences may be unacceptable.  (The solution is worse than the
original problem.)  It might work if the revision-date was the min-revision
instead of the exact revision, but we should debate Y45 on the NETMOD list.

> This has to do with using YANG as a general schema language, i.e. outside the normal NETCONF context.

Do you have any objections to the resource-identifier definition?

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 06 Jan 2015 10:50:33 +0100

> have an import-by-revision is not related to this issue.

It is related to that issue in the sense that it is IMO impossible to
pretend that YANG can be turned into a general-purpose schema language
just by wrapping a schema in an extension. RFC 6020 is too
NETCONF-centric and several things are undefined or ill-defined if used
outside the NETCONF context (or considerably more text is needed in the
definition of the restconf-resource extension):

- If a list is present, is it required to have keys or not?

- What would if-feature mean inside rc:restconf-resource?

- If "must" and "when" expressions are used, the accessible tree for
  their XPath expressions is undefined - sections 7.5.3 and 7.19.5 only
  deal with config and state data, notifications, and RPC input/output.

- What is the accessible tree for the "path" statement in leafref type?
  According to sec. 9.9.2 it could be "all state data on the device, and
  the <running/> datastore", but it doesn't make sense.

- If the identityref type is used, how can one determine the set of
  permitted values?

> instead of the exact revision, but we should debate Y45 on the NETMOD

It makes no sense to me to discuss conformance issues here because what
you want to do is to define certain fixed schemas as a part of the
RESTCONF spec.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Tue, 6 Jan 2015 06:54:31 -0800

> - If the identityref type is used, how can one determine the set of

This is determined by the identity base and not a problem

These are valid questions and the restconf-resource extension would have to
specify these details.  The short answer is that all the datastore
specific statements would be ignored and have no affect:

So what is your solution proposal instead?

What are the options?

  1) XSD
   Pros: it exists and can specify syntax
   Cons: Not widely liked or easy to learn. No JSON support
   so RESTCONF support for JSON would have to be removed,
   or wait for a JSON schema language to be standardized.

  2) Plain text
  Pros: plain text is always allowed in an RFC
  Cons: all tools must be hand-coded and plain text can be interpreted
  very loosely, leading to poor interoperability.

Do you have a solution proposal?

I completely agree that YANG is being used here in a manner different
than NETCONF.
YANG does not even completely work for RESTCONF, let alone resource
abstractions or a new datastore for I2RS.

We are using YANG as a low-level info model here.
We do not want to tightly couple the protocol messages to a specific syntax.
IMO this is a big weakness with NETCONF and YANG.
The message encoding could be XML, JSON, CBOR, EXI, who cares.
But YANG is tightly coupled to NETCONF and XML and that isn't
changing in YANG 1.1.

> you want to do is to define certain fixed schemas as a part of the

You brought up import-by-revision.
I agree it is irrelevant to this issue.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 6 Jan 2015 16:49:31 +0100

> On 06 Jan 2015, at 15:54, Andy Bierman <[email-masked]> wrote:
> 

Sec. 9.10.2: "On a particular server, the valid values are further restricted to the set of identities defined in the modules supported by the server.”

But there is no particular server, the schema has to be identical for all.

> specify these details.  The short answer is that all the datastore

Then one has to ask whether it is worth the hassle for this ad hoc purpose. And I don’t think we want to see other people following this example and defining extensions like this on their own.

>  very loosely, leading to poor interoperability.

I think the schemas are relatively trivial and plain text should do. 

> But YANG is tightly coupled to NETCONF and XML and that isn't

I have supported, actually proposed, to make YANG into a more general schema language by removing the dependence on NETCONF (and XML), but I believe this has to be done by rewriting the spec, and not by bending it via extension hacks. 

Lada

---

From: Andy Bierman <[email-masked]>
Date: Tue, 6 Jan 2015 08:31:36 -0800

> I think the schemas are relatively trivial and plain text should do.

Then please provide the replacement text.
IMO it is not simple to write the detailed XML and JSON encoding rules
for all the protocol messages. That's why we have schema languages instead.
The lack of machine-readable schema is a big step backwards.
IMO it would be better to drop JSON support completely, and use XSD.

This issue also applies to all the groupings in the YANG Patch document.
That document needs to be completely rewritten so the complex YANG
data structures are replaced with plain text or XSD as well.

So who else wants the ietf-restconf and ietf-yang-patch modules removed
and replaced with plain text?

Who wants to drop JSON support and use XSD?

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 06 Jan 2015 22:34:49 +0100 (CET)

> > On 06 Jan 2015, at 15:54, Andy Bierman <[email-masked]> wrote:

[...]

> I have supported, actually proposed, to make YANG into a more general
> schema language by removing the dependence on NETCONF (and XML)

I agree, to some extent.  I also think that a core statement
"structure" would be the best solution.  Meanwhile, in YANG 1.0, an
extension like the one proposed is the best way to solve this.

> but I
> believe this has to be done by rewriting the spec, and not by bending
> it via extension hacks.

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 07 Jan 2015 11:04:35 +0100

> extension like the one proposed is the best way to solve this.

Well, it's not proposed for YANG 1.1 either, and everything else sounds
like "when pigs fly" to me. Or could we perhaps consider it for
inclusion in YANG 1.1?

Actually, there is another possible solution, namely to use XSD or
RELAX NG for XML and draft-newton-json-content-rules-04 for JSON. It is
true that the latter is still in an early stage but its acceptance in the
JSON WG was positive and that WG seems to act fast.

Lada