Subject: Re: [netmod] Fwd: New Version Notification for draft-ietf-netmod-yang-json-05.txt

From: Randy Presuhn <[email-masked]>
Date: Tue, 15 Sep 2015 12:01:23 -0700 (GMT-07:00)

Hi -

>From: Juergen Schoenwaelder <[email-masked]>
>Sent: Sep 14, 2015 11:41 PM
>To: Ladislav Lhotka <[email-masked]>
>Cc: NETMOD Working Group <[email-masked]>
>Subject: Re: [netmod] Fwd: New Version Notification for draft-ietf-netmod-yang-json-05.txt
...
>My question is why the text is silent about the case where the data
>model is present. Should it not say that if the data model is present,
>the data encoded inside the anydata node must follow the rules of this
>document? Perhaps this is the implicit assumption but I think it will
>be useful to say this explicitly (if we agree on this).
>
>If the data model is not present, then I think an implementation is
>still expected to produce an encoding that follows the rules of this
>document as much as possible except that things that requires data
>model knowledge may be encoded differently (e.g., numbers appearing as
>strings or namespace names being different). I am thinking along the
>lines of this proposed new text:
>
>   An anydata data node can contain an unknown set of nodes that can
>   be modelled by YANG. A data model for anydata content may or may
>   not exist at run time.  If the data model for anydata content is
>   available, then the anydata content MUST be encoded according to
>   the rules of this specification. If the data model for anydata
>   content is not available, the encoding MUST follow the rules of
>   this specification except for rules that require data model
>   knowledge (and as a consequence, numbers may appear as strings or
>   namespace qualifiers may not match module names).

This leaves me wondering what it means for the data model for
anydata content to be "available".  In the case of ASN.1's
"ANY DEFINED BY" construct, there's an OBJECT IDENTIFIER to
unambiguously identify the grammar (and associated semantics)
to be used to understand the content, so tools can, if needed,
scurry off to obtain the parsing instructions for those
particular bits.  How does an implementation know in the case
of "anydata" which datamodel to use?

Randy

---

From: Andy Bierman <[email-masked]>
Date: Tue, 15 Sep 2015 12:21:44 -0700

> of "anydata" which datamodel to use?
Good questions....
The text "If the data model for anydata content is available" gives a hint
of just
what a hack anydata is in YANG.  The definition of anydata is that there is
no data model for the specified subtree.  The mere mention of an out-of-band
data mode is inappropriate and confusing.

I understand this is intended to support usage like in YANG Patch, where the
description-stmt of 'value' says that the child node must follow the schema
for the node in the target leaf.  More hacks to get YANG to work.

Once again we confuse conformance to YANG with conformance to
a module written in YANG.  The YANG Patch text can make additional
requirements for conformance to YANG Patch.  This is quite different
than generic conformance to RFC6020bis.

Randy

Andy

>
> _______________________________________________
> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod
>

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 16 Sep 2015 08:03:10 +0200

> of "anydata" which datamodel to use?

It can be stated in the description of the anydata statement. One can
then ask though why we need two constructs - anyxml and anydata -
because a data model can be specified in the description of an anyxml
statement as well.

Lada

-- 
Ladislav Lhotka, CZ.NIC Labs
PGP Key ID: E74E8C0C

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 16 Sep 2015 17:14:33 +0200

> of "anydata" which datamodel to use?

There are situations where we need ANY and not ANY DEFINED BY. Generic
RPCs sometimes work on arbitrary data models. I think ASN.1 allows
ANY as well.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1 | 28759 Bremen | Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 16 Sep 2015 17:17:59 +0200

> for the node in the target leaf.  More hacks to get YANG to work.

You are welcome to provide fixes.
 
> than generic conformance to RFC6020bis.

I do not think that we confuse conformance to YANG with conformance to
a module written in YANG.

/js

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 16 Sep 2015 17:19:42 +0200

> because a data model can be specified in the description of an anyxml

The data model used by anydata content is clearly identified. The
difference between anyxml and anydata has been discussed at length and
we should not go there again.

/js

---

From: Andy Bierman <[email-masked]>
Date: Wed, 16 Sep 2015 09:00:48 -0700

> You are welcome to provide fixes.

OLD:

NEW:

  be modelled by YANG.

 The YANG RFC itself should be silent about data-model specific
semantics that are added to an anydata subtree.  The text "if available"
is especially non-enforceable and therefore pointless.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 16 Sep 2015 19:50:24 +0200

> On 16 Sep 2015, at 17:19, Juergen Schoenwaelder <[email-masked]> wrote:
> 

Hmm, how is it clearly identified? There needn’t be any data model at all.

> we should not go there again.

Then the 6020bis doesn’t make this difference clear, to me at least.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Wed, 16 Sep 2015 11:25:25 -0700

On Wed, Sep 16, 2015 at 10:50 AM, Ladislav Lhotka <[email-masked]> wrote:

>
> > On 16 Sep 2015, at 17:19, Juergen Schoenwaelder <
This is an academic exercise, but the difference is very clear.

The "anyxml" node allows XML-specific details like processing instructions.
It is a blob of XML.  It is not JSON.  It is not YANG.  It is XML.
This is academic, because the vast majority of servers do not support anyxml
at all.  None. Zero.  Not allowed in the server.  The few that do treat
anyxml
as if it were anydata.

The "anydata" node is a blob of YANG data nodes.  It is not XML.
It is not JSON. This too will likely not be implemented in the vast majority
of servers.

Lada

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 16 Sep 2015 20:29:16 +0200

> On 16 Sep 2015, at 18:00, Andy Bierman <[email-masked]> wrote:
> 
> 
> 
> On Wed, Sep 16, 2015 at 8:17 AM, Juergen Schoenwaelder <[email-masked]> wrote:

This text is essentially in 6020bis, I see no need to repeat it. 

> is especially non-enforceable and therefore pointless.

I must admit I am getting lost in these discussions. It seems to me there is a lot of hand-waving and hidden assumptions that moreover differ from one person to another. As I already said in Prague, both anyxml and anydata are IMO constructs of marginal utility and it is frustrating we spend so much effort on them.

Lada

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 16 Sep 2015 20:55:42 +0200

> On 16 Sep 2015, at 20:25, Andy Bierman <[email-masked]> wrote:
> 
> 
> 
> at all.  None. Zero.  Not allowed in the server.  The few that do treat anyxml
> as if it were anydata.
> 

This is illusory unless a YANG data model is available, and, as you said, this is not guaranteed. For one, without a data model there is no way for mapping XML encoding to JSON and vice versa, so anydata received in one encoding cannot be sent back to clients supporting the other encoding.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Wed, 16 Sep 2015 12:29:17 -0700

On Wed, Sep 16, 2015 at 11:55 AM, Ladislav Lhotka <[email-masked]> wrote:

> This is illusory unless a YANG data model is available, and, as you said,
> this is not guaranteed. For one, without a data model there is no way for
> mapping XML encoding to JSON and vice versa, so anydata received in one
> encoding cannot be sent back to clients supporting the other encoding.
>
>

I don't like the idea of adding a requirement that the "hidden schema" be
used
so that JSON people will be happy to get a number instead of a string, as if
that is actually a data model.

Unless there is a hidden schema, then it just doesn't matter if the client
passed in a JSON number and another client reads back an XML string.
There is no data model at all.  There is not even any schema that
says 'foo' can have any child nodes or not, let alone the required
QName and syntax for that child node.

Users that want any sort of schema-based interoperability should not
use anydata instead of real data nodes.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 16 Sep 2015 22:11:26 +0200

> On 16 Sep 2015, at 21:29, Andy Bierman <[email-masked]> wrote:
> 
> 
> 
> I don't like the idea of adding a requirement that the "hidden schema" be used

Without a data model we also cannot map module names to XML namespaces.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Wed, 16 Sep 2015 13:51:53 -0700

On Wed, Sep 16, 2015 at 1:11 PM, Ladislav Lhotka <[email-masked]> wrote:

> > This is illusory unless a YANG data model is available, and, as you
> said, this is not guaranteed. For one, without a data model there is no way
> for mapping XML encoding to JSON and vice versa, so anydata received in one

Hence the value of using real data nodes instead of 'anydata'.
The 'anydata' statement says "this is a subtree without any schema".
So it doesn't really do much good to worry about encoding based on the
schema.

But if a YANG module declares an 'anydata' node then the description-stmt
for that node (or maybe extensions) MAY contain normative instructions for
encoding the descendant nodes within an anydata subtree.
This impacts conformance for that module, not general conformance
for 'anydata'.

Lada

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 17 Sep 2015 08:24:28 +0200

> The 'anydata' statement says "this is a subtree without any schema".

anyxml says the same.

> This impacts conformance for that module, not general conformance

Yes, but the same thing can be done with anyxml, right? It has been
demonstrated in RFC 6241, ietf-yang-patch and probably elsewhere, too,
and it does the job.

The use case of passing around literally "any XML" is really only
theoretical, I think some kind of schema is always assumed.

So I believe we don't need two data node types for this purpose. And an
advantage of anyxml is that its definition (from YANG point of view) is
clear and unambiguous, whereas for anydata the phrase "can be modelled
with YANG" may be interpreted in different ways. We are introducing a
dubious new concept in YANG 1.1 with no apparent gain.

Lada

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 21 Sep 2015 22:34:39 +0200

> I must admit I am getting lost in these discussions. It seems to me there is a lot of hand-waving and hidden assumptions that moreover differ from one person to another. As I already said in Prague…

I agree that anyxml is of marginal utility, anydata however is needed
for any rpc/action/notification or future language construct that can
work with generic YANG content and hence I think its behaviour and
encoding should be well defined.

/js

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 21 Sep 2015 22:40:29 +0200

> It is not JSON. This too will likely not be implemented in the vast majority

Even very basic things like <get-config> need something like anyxml or
anydata. Right now, we write anyxml but usually mean anydata. YANG 1.1
aims at fixing this.

/js

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 21 Sep 2015 22:44:05 +0200

> dubious new concept in YANG 1.1 with no apparent gain.

I do not think we are going to repeat that debate. I think we
concluded that anyxml does not mean any JSON.

/js

---

From: Andy Bierman <[email-masked]>
Date: Mon, 21 Sep 2015 13:53:01 -0700

Hi,

I agree that anydata is much better than anyxml because it represents
how this "schema replacement" is actually being used.

I don't worry about anyxml too much.
It would be better to just rename anyxml, or declare it really
means anydata, but that would not be backward-compatible.

If anyxml "XML semantics" were actually important, then YANG designers would
ask toolkit developers to support it correctly.  Since that has never
happened, it seems clear that anyxml never meant "any XML".

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 23 Sep 2015 10:14:52 +0200

> encoding should be well defined.

But then I believe we should have stricter rules for anydata than just
"an unknown set of nodes that can be modelled with YANG" - it should be
stated that the data model for an anydata instance MUST be known at
run-time. Otherwise I think anyxml can cover all use cases you mention
as well (as it has done in the past), and there is no need to introduce
a new data node type with a definition that allows for multiple
interpretations.

Lada

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 23 Sep 2015 15:11:20 +0200

> a new data node type with a definition that allows for multiple

Lada, we are not repeating the discussion. It was long and painful
enough and we finally accepted and verified Y34-05. If you have better
wording to propose, feel free to make a proposal. But I am not going
to open Y34 again just because you still do not like it.

That said, "MUST be known at run-time" may already fall apart with the
mount proposals discussed in NETCONF (or it is sufficiently unclear to
whom it MUST be known).

/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 23 Sep 2015 15:46:27 +0200

> On 23 Sep 2015, at 15:11, Juergen Schoenwaelder <[email-masked]> wrote:
> 

Experience has shown that the current formulation is understood differently by different people.

Earlier in this thread, you proposed this text to be added to the yang-json spec:

So let me repeat: shouldn’t this text really go to 6020bis? Or do you think that XML encoding needn’t abide by the data model if it is known?

> mount proposals discussed in NETCONF (or it is sufficiently unclear to

To the server and client so that they both work with the same data model. Of course, it is true that currently there are no means for passing this information (either way) but I guess your text above suffers from the same problem: how can you tell whether the data model for anydata content is available or not? 

Lada

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 28 Sep 2015 08:55:04 +0200

> So let me repeat: shouldn’t this text really go to 6020bis? Or do you think that XML encoding needn’t abide by the data model if it is known?

Are you suggesting there should be similar text for the XML encoding or
are you suggesting there should be abstract text for all encodings?
RFC6020bis says currently:

   An anydata node is encoded as an XML element.  The element's local
   name is the anydata's identifier, and its namespace is the module's
   XML namespace (see Section 7.1.3).  The value of the anydata node is
   a set of nodes, which are encoded as XML subelements to the anydata
   element.

Perhaps it means 'set of data nodes' where it says 'set of nodes'. The
main difference here is that encoding anydata into XML is somewhat
easier since you do not need to know whether 42 has to be encoded as a
string or a number etc.

> To the server and client so that they both work with the same data model. Of course, it is true that currently there are no means for passing this information (either way) but I guess your text abo…

The example is a RESTCONF server mounting data from a remove datastore
and using JSON encoding when talking to RESTCONF clients. Do we
require that such a server has to have knowledge about the data model
mounted in order to get the encoding right? Or do we expect that the
RESTCONF client can deal with situations where encodings may not 100%
match the data model (e.g. lack of type knowledge, lack of namespace
to module name mappings).

/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 29 Sep 2015 10:28:28 +0200

> are you suggesting there should be abstract text for all encodings?

I think it should be an encoding-independent statement - if the data
model is known, then anydata contents should be essentially no different
from a regular data (sub)tree.

>    a set of nodes, which are encoded as XML subelements to the anydata

Note, however, that this still includes contents that cannot be modelled
in YANG. The rules in the yang-json document (sec. 5.5) are similar but
they also try to exclude data that can never come from YANG.

>> To the server and client so that they both work with the same data
>> model. Of course, it is true that currently there are no means for
>> passing this information (either way) but I guess your text above
>> suffers from the same problem: how can you tell whether the data
>> model for anydata content is available or not?
>>
>

This is not clear to me. I think access to remote content and schema
availability are orthogonal problems.

Lada

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 29 Sep 2015 11:00:57 +0200

> from a regular data (sub)tree.

Can you make a concrete proposal how to change the text in section
7.10 of draft-ietf-netmod-rfc6020bis-07.txt? And then please post it
as a separate thread so we can track it as YANG 1.1 last call comment.

> availability are orthogonal problems.

I wanted to provide an example demonstrating that schema availability
is not always guaranteed and since (i) JSON and XML use different ways
to encode namespaces and (ii) JSON assumes some type information to be
available while XML does not encode any type information, the example
outlined above will either be expensive to implement (the mount server
essentially has to grab all necessary data models while mounting
something from a remote datastore) or it will produce encodings that
require the client to very lenient in order to interoperate.

Yes, in an ideal world, these would be orthogonal problems but due the
differences how the encodings work, they are not as orthogonal as one
might think.

/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 29 Sep 2015 16:31:18 +0200

> as a separate thread so we can track it as YANG 1.1 last call comment.

Done.

> require the client to very lenient in order to interoperate.

It is because JSON and XML are not just different encodings, as UCS-2
and UTF-8 are different encodings of Unicode that can be mapped
1-1. Information sets of XML and JSON differ considerably, so coercing
one into the other makes no sense.

It will be no different if, for example, CBOR is used as another
"encoding" for YANG data.

Maybe it would be better to use "serialization" rather than "encoding"
in 6020bis and draft-ietf-netmod-yang-json.

Lada

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 29 Sep 2015 18:11:03 +0200

> in 6020bis and draft-ietf-netmod-yang-json.

Changing the terms does not change the characteristics of the design:
The encodings (or serializations) are not transformable without having
the data models. This has consequences and the WG hopefully
understands them. The impact is likely low on a device hosting a
server and low on a true management system. But anything generic in
between those two (and I am sure there will be stuff in between plain
servers on devices and management systems - mount is just a first
example) will have to be able to grab data models as necessary in
order to pass information along.

/js