Subject: [netmod] JSON encoding of anyxml

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 5 Nov 2015 14:54:56 +0900

Hi,

at the NETMOD session Martin proposed that JSON encoding of anyxml instances be changed so that it is required to contain a JSON string with well-formed XML serialization inside. His argument was that "xml" is part on the data node name and also that RFC 6020 says it is "an unknown chunk of XML". I think, however, that the name and definition were influenced by the fact that XML was the only encoding under consideration when 6020 was being written. IMO, the definition could thus be interpreted as "an unknown chunk of data that follows the syntax of the current encoding".

Note that similar ambiguity of interpretations applies to 6020 statements about NETCONF, edit-config etc. - we saw it in the discussion about auto-delete behaviour.

A practical problem of Martin's proposal is that it doesn't provide a symmetric option to send arbitrary JSON chunks in JSON encoding (that may or may not be modelled with YANG). An option would be to introduce a new data node "anyjson" but I think this is really a bad approach - shall we then also add "anycbor" etc.? IMO YANG statements and rules should be encoding-independent.

I still think the best option would be to introduce "anydata" with my definition above so that in XML encoding it would be equivalent to "anyxml", and deprecate "anyxml". We can also introduce a substatement that could state that the instances of a particular anydata node are required to be modellable in YANG.

BTW, it seems that yang-patch needs to be able to transfer anyxml instances, so is it really possible to exclude anyxml from anydata content?

Lada 
 
--
Ladislav Lhotka, CZ.NIC Labs
PGP Key ID: E74E8C0C

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 5 Nov 2015 07:51:11 +0100

Lada,

this seems to be related to YANG 1.1 issue Y34 which we concluded with
consensus on Y34-05, which extends Y34-02. And Y34-02 says:

  'anyxml' would still be used to represent unrestricted XML, as is
  done in NETCONF.

Your repeated attempts to generalize anyxml does not give us
interoperability. In fact, Y34-05 says:

   - Document that implementations MAY have restrictions for anyxml and
     that anyxml is not necessarily interoperable; data model writers
     should use anydata whenever possible.
   - The guidelines document should say that YANG Doctors will review
     each use of anyxml in IETF modules when YANG 1.1 is adopted to
     avoid its use whenever possible.

Given the resolution of Y34 in YANG 1.1, Martin's proposal to encode
anyxml as a string that has XML inside makes sense.

/js

On Thu, Nov 05, 2015 at 02:54:56PM +0900, Ladislav Lhotka wrote:
> _______________________________________________
> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1 | 28759 Bremen | Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 5 Nov 2015 17:56:16 +0900

> On 05 Nov 2015, at 15:51, Juergen Schoenwaelder <[email-masked]> wrote:
> 
> Lada,
> 

The possibility of sending arbitrary (non-YANG) data in the native encoding can occassionally be useful, and even more so in JSON. For example, I have to work with a JSON-based format for specifying DNSSEC signing policies. While my plan is to eventually replace this format with YANG-modelled data, it would help me, as an interim solution, to simply send the data in the legacy format. That's why I want to retain the existing rules for JSON encoding of anyxml, unless something else is available. Sending XML documents as strings is still possible although IMO next to useless.

Lada

---

From: Robert Varga <[email-masked]>
Date: Mon, 9 Nov 2015 23:39:45 +0100

> The possibility of sending arbitrary (non-YANG) data in the native encoding can occassionally be useful, and even more so in JSON. For example, I have to work with a JSON-based format for specifyin…

I fear requiring JSON in anyxml will render RESTCONF-to-NETCONF proxies 
non-transparent in case the sender (a NETCONF NE) and receiver (an 
RESTCONF application) have out-of-band knowledge of the data being sent 
over anyxml. Given the proxy does not have this knowledge, it cannot 
reliably deal with lists, as they lack a container element in XML encoding.

Can we perhaps indicate the encoding of the anyxml data chunk in JSON as 
a separate well-known attribute?

Bye,
Robert

---

From: Andy Bierman <[email-masked]>
Date: Mon, 9 Nov 2015 18:19:54 -0800

Hi,

I am not in favor of anything XML or JSON specific in YANG.
In reality, nobody uses anyxml as a configuration data node,
so an improper roundtrip translation from JSON to XML
is not going to happen.

Encoding anyxml as a string is not going to happen either.
Not sure what the difference between 'anyxml' and 'type string'
is at that point.

Why does YANG even need a special type of leaf that is a blob of XML?
Can't a single-quoted string serve that purpose?

Andy

On Mon, Nov 9, 2015 at 2:39 PM, Robert Varga <[email-masked]> wrote:

>> The possibility of sending arbitrary (non-YANG) data in the native
>> encoding can occassionally be useful, and even more so in JSON. For
>> example, I have to work with a JSON-based format for specifying DNSSEC
>> signing policies. While my plan is to eventually replace this format with
>> YANG-modelled data, it would help me, as an interim solution, to simply
>> send the data in the legacy format. That's why I want to retain the
>> existing rules for JSON encoding of anyxml, unless something else is
>> available. Sending XML documents as strings is still possible although IMO
>> next to useless.
>>
>
> non-transparent in case the sender (a NETCONF NE) and receiver (an RESTCONF
> application) have out-of-band knowledge of the data being sent over anyxml.
> Given the proxy does not have this knowledge, it cannot reliably deal with
> lists, as they lack a container element in XML encoding.
>
> Can we perhaps indicate the encoding of the anyxml data chunk in JSON as a
> separate well-known attribute?
>
> Bye,
> Robert
>

---

From: Robert Varga <[email-masked]>
Date: Tue, 10 Nov 2015 15:16:15 +0100

Hello,

I am not favor of it, either, but RFC6020 is here and is being widely 
deployed. So is RESTCONF+JSON, which is favored by application 
developers in the field today, as is NETCONF devices producing anyxml. 
We do need a reasonable way of bridging the two -- no matter whether it 
is configuration or operational data.

At any rate, a simple string is not going to cut it, as it does not 
retain modeling context nor encoding details of the data being 
transported -- rendering reliable automated translation impossible.

Bye,
Robert

On 11/10/2015 03:19 AM, Andy Bierman wrote:
> On Mon, Nov 9, 2015 at 2:39 PM, Robert Varga <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>         The possibility of sending arbitrary (non-YANG) data in the
>         native encoding can occassionally be useful, and even more so
>         in JSON. For example, I have to work with a JSON-based format
>         for specifying DNSSEC signing policies. While my plan is to
>         eventually replace this format with YANG-modelled data, it
>         would help me, as an interim solution, to simply send the data
>         in the legacy format. That's why I want to retain the existing
>         rules for JSON encoding of anyxml, unless something else is
>         available. Sending XML documents as strings is still possible
>         although IMO next to useless.
>
>
>     I fear requiring JSON in anyxml will render RESTCONF-to-NETCONF
>     proxies non-transparent in case the sender (a NETCONF NE) and
>     receiver (an RESTCONF application) have out-of-band knowledge of
>     the data being sent over anyxml. Given the proxy does not have
>     this knowledge, it cannot reliably deal with lists, as they lack a
>     container element in XML encoding.
>
>     Can we perhaps indicate the encoding of the anyxml data chunk in
>     JSON as a separate well-known attribute?
>
>     Bye,
>     Robert
>
>     [email-masked] <mailto:[email-masked]>

---

From: Andy Bierman <[email-masked]>
Date: Tue, 10 Nov 2015 17:04:33 -0800

On Tue, Nov 10, 2015 at 6:16 AM, Robert Varga <[email-masked]> wrote:

> deployed. So is RESTCONF+JSON, which is favored by application developers
> in the field today, as is NETCONF devices producing anyxml. We do need a
> reasonable way of bridging the two -- no matter whether it is configuration
> or operational data.
>
>

I do not agree that the use of anyxml as arbitrary XML is deployed at all.
I do not agree that all the "extra XML bits" that are causing so much
concern
are ever saved in a datastore.  Martin says we need anyxml for RPC input
and output. Maybe. This seems proprietary, since no NETCONF/RESTCONF
operation depends on anything except elements and attributes.

If the JSON looks like a leaf, then no tool is going to analyse the string
to see if it a well-formed XML instance document, so it can be re-parsed.
Unless a JSON array or object is started, the tool will not
think it is dealing with any child nodes.

There are lots of things that NETCONF can do that are trimmed out of
RESTCONF.
JSON representation or raw XML seems really low priority to me.
Are there any use-cases? What processing instructions, character entities,
etc.
are being used in the wild already?  None? Don't we have enough real
problems to
solve without rat-holing over corner-cases all the time?

Andy

> At any rate, a simple string is not going to cut it, as it does not retain
> modeling context nor encoding details of the data being transported --
> rendering reliable automated translation impossible.
>
> Bye,
> Robert
>
> On Mon, Nov 9, 2015 at 2:39 PM, Robert Varga < <[email-masked]>[email-masked]>
> wrote:
>

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 11 Nov 2015 07:34:23 +0100 (CET)

> operation depends on anything except elements and attributes.

Specifically, we need anyxml for ietf-netconf, since NETCONF is
supposedly data modeling language agnostic.

One option could be to deprecte anyxml (or remove it) in YANG 1.1, and
if we ever do a new version of NETCONF, we use anydata instead.

/martin

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 11 Nov 2015 09:07:28 +0100

> if we ever do a new version of NETCONF, we use anydata instead.

We are effectively deprecating anyxml in YANG 1.1. We have reached
agreement on Y34-05 and I do not see any new arguments popping up.

The JSON encoding document has to specify the encoding of an
effectively deprecated construct that was designed to encode any XML.

/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 11 Nov 2015 11:46:04 +0100

> reliably deal with lists, as they lack a container element in XML

My view of anyxml is that it is a "controlled loophole" that allows
arbitrary non-YANG stuff to be sent. The only requirement is that it
won't break the syntax of the document in which it is embedded - and
this of course depends on the encoding used. It should be used scarcely,
and yes, it generally isn't interoperable and
encoding-transparent. Whenever it is needed, the implementations should
deal with these issues on an ad hoc basis.

So far it has mostly been used in specifications (NETCONF, yang-patch,
zerotouch), i.e. outside the network management context, where YANG
serves as an encoding-independent schema language.

I think it *may* occassionally be useful but it is also true that it can
always be substituted with a string or binary leaf. The only advantage of
sending such data in the "native" encoding is that it can be parsed in
one go.

> a separate well-known attribute?

I don't think this could be useful because I believe such schema-less
chunks really make sense only if their encoding is the same as that of
the outer document.

Lada

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 11 Nov 2015 11:53:56 +0100

> if we ever do a new version of NETCONF, we use anydata instead.

The simplest solution in this case would be to use a standard XML schema
language - XSD or RELAX NG.

Lada

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 11 Nov 2015 11:54:58 +0100

> On 11 Nov 2015, at 09:07, Juergen Schoenwaelder <[email-masked]> wrote:
> 

There is nothing in 6020bis indicating that "anyxml" is being deprecated.

Lada

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 11 Nov 2015 13:26:30 +0100

> There is nothing in 6020bis indicating that "anyxml" is being deprecated.

I wrote 'effectively deprecated' and here is the text in 6020bis.

   Since the use of anyxml limits the manipulation of the content, it is
   RECOMMENDED that the "anyxml" statement not be used to define
   configuration data.

   It should be noted that in YANG version 1, anyxml was the only
   statement that could model an unknown hierarchy of data.  In many
   cases this unknown hierarchy of data is actually modelled in YANG,
   but the exact YANG model is not known at design time.  In these
   situations, it is RECOMMENDED to use anydata (Section 7.10) instead
   of anyxml.

There is more text in Y34-05 that will go into the guidelines. I call
this "effectively deprecated", you can call it differently if you
want. Frankly, this thread is about how to encode anyxml in JSON not
about the YANG 1.1 issue Y34. You may not like Y34-05 but this is what
we ended up with.

/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 11 Nov 2015 14:24:15 +0100

> On 11 Nov 2015, at 13:26, Juergen Schoenwaelder <[email-masked]> wrote:
> 

The first paragraph is for config data and the second for data that can modelled with YANG. If we want to deprecate "anyxml" for use with data that are neither of these, 6020bis should say so. I'd be fine with that.

> about the YANG 1.1 issue Y34. You may not like Y34-05 but this is what

I am strongly against encoding serialized XML as JSON strings. It is IMO totally useless and the spec would have to deal with awkward complications because it is not true that arbitrary XML-encoded anyxml content can be used without changes in JSON encoding. 

Perhaps the best solution would be to state that JSON encoding is incompatible with data models that use "anyxml".

Lada

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 11 Nov 2015 14:44:12 +0100

> The first paragraph is for config data and the second for data that can modelled with YANG. If we want to deprecate "anyxml" for use with data that are neither of these, 6020bis should say so. I'd…

The guidelines will say that any usage of anyxml in the IETF will be
carefully checked by YANG doctors. See Y34 for all details.

> Perhaps the best solution would be to state that JSON encoding is incompatible with data models that use "anyxml".

Perhaps we can only settle this by doing an opinion poll since
debating this forever is not useful. So what are the options?

a) The JSON encoding does not define anyxml is not encoded at all. If
   you use anyxml in a data model, the content will not appear in JSON
   encodings.

b) The JSON encoding defines that anyxml data is encoded as a string
   containing XML.

c) The JSON encoding defines that anyxml is encoded in whatever way
   the implementor finds useful.

d) If the anyxml content is actually valid anydata content, encode it
   using anydata rules. Content that is not valid anydata content is
   not encoded at all.

Any options missing?

Observations:

 - Except b), none of the options is interoperable. Option d) is
   interoperable for the anydata subset of anyxml.
 - It seems a) is simplest to implement, b) might be expensive to
   implement and use.
 - Some implementations use internal data stores that can only deal
   with data that can be modelled with YANG and for those implementations
   anyxml is effectively the same as anydata and d) migth be close to
   a) in terms of implementation costs.

Any other observations missing?

/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 11 Nov 2015 14:53:49 +0100

> On 11 Nov 2015, at 14:44, Juergen Schoenwaelder <[email-masked]> wrote:
> 

Yes, the one that's in yang-json-06:

e) An anyxml instance is encoded as a JSON name/value pair which MUST
   satisfy I-JSON constraints.  Otherwise it is unrestricted, i.e., the
   value can be an object, array, number, string or one of the literals
   "true", "false" and "null".

My preference is e), and then a).

Lada

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 11 Nov 2015 14:59:19 +0100

> My preference is e), and then a).

Is e) not the same as c)? I assume that the JSON encoding will always
be valid JSON (or I-JSON to be precise). So it seems the only refinement
of e) is the toplevel.

/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 11 Nov 2015 15:14:13 +0100

> On 11 Nov 2015, at 14:59, Juergen Schoenwaelder <[email-masked]> wrote:
> 

c) sounds like the same data can be encoded in different ways depending on what the implementor finds useful, e.g. encode all numbers as strings.

Lada

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 11 Nov 2015 16:32:31 +0100

> c) sounds like the same data can be encoded in different ways depending on what the implementor finds useful, e.g. encode all numbers as strings.

But e) says the same, I fail to see the difference here.

/js

---

From: Andy Bierman <[email-masked]>
Date: Wed, 11 Nov 2015 08:31:14 -0800

On Wed, Nov 11, 2015 at 7:32 AM, Juergen Schoenwaelder <
[email-masked]> wrote:

> > > On 11 Nov 2015, at 14:59, Juergen Schoenwaelder <
> [email-masked]> wrote:
> > >
> > >>> On 11 Nov 2015, at 14:44, Juergen Schoenwaelder <
> [email-masked]> wrote:
> > >>>
> > >>>> The first paragraph is for config data and the second for data that
> can modelled with YANG. If we want to deprecate "anyxml" for use with data
> that are neither of these, 6020bis should say so. I'd be fine with that.
> > >>>>
> > >>>
> > >>>> I am strongly against encoding serialized XML as JSON strings. It
> is IMO totally useless and the spec would have to deal with awkward
> complications because it is not true that arbitrary XML-encoded anyxml
> content can be used without changes in JSON encoding.
> > >>>>
> > >>>> Perhaps the best solution would be to state that JSON encoding is
> incompatible with data models that use "anyxml".
> > >>>>
> > >>>
> > c) sounds like the same data can be encoded in different ways depending
> on what the implementor finds useful, e.g. encode all numbers as strings.
> >
>
IMO the only option is (f)

 f) If the anyxml content is actually valid anydata content, encode it
    using anydata rules. Content that is not valid anydata content is
    either not encoded at all, or encoded in an implementation-specific
manner

Nobody has answered my question about the "WEB banner" type of object.
The only reports we have ever heard on this sort of object indicate that
a leaf is used, NOT ANYXML!  Nobody seems to be using anyxml to store WEB
page snippets in their configuration.  Clearly a leaf is interoperable for
both XML and JSON, plus NETCONF or RESTCONF.  But we keep insisting
the YANG to JSON mapping needs to support this use of anyxml.

We use anyxml in RPC parameters in ietf-netconf, but the XML
is not expected to be mixed-mode XML.  Only elements and attributes
are used.

/js

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 11 Nov 2015 17:43:24 +0100

>     either not encoded at all, or encoded in an implementation-specific

This is essentially the same as d).

> is not expected to be mixed-mode XML.  Only elements and attributes

Well, there are subtle differences. For example, a leaf-list only
allows unique values. So by way of example, anydata is in fact more
restrictive than anyxml even if we only consider XML elements and
attributes.

/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 12 Nov 2015 08:10:51 +0100

> But e) says the same, I fail to see the difference here.

The way how c) is formulated is rather suggestive: beware, this is
non-interoperable by its very nature. I think it is as interoperable as
anyxml in XML encoding. Without knowing the context, translating anyxml
chunks from XML to JSON, and vice versa, is quite problematic even for
b).

BTW, if the ability to translate between XML and JSON encodings is so
critical for interoperability, then let me note that it's in general not
possible for "anydata" either.

Lada

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 12 Nov 2015 08:51:11 +0100

> chunks from XML to JSON, and vice versa, is quite problematic even for

e) is no better than c) in terms of interoperability
 
/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 12 Nov 2015 08:56:29 +0100

> On 12 Nov 2015, at 08:51, Juergen Schoenwaelder <[email-masked]> wrote:
> 

OK, I agree to add up votes for c) and e).

Lada

---

From: Andy Bierman <[email-masked]>
Date: Fri, 13 Nov 2015 10:19:31 -0800

On Wed, Nov 11, 2015 at 11:51 PM, Juergen Schoenwaelder <
[email-masked]> wrote:

> > >> >>>> The first paragraph is for config data and the second for data
> that can modelled with YANG. If we want to deprecate "anyxml" for use with
> data that are neither of these, 6020bis should say so. I'd be fine with
> that.
> > >> >>>>
> > >> >>>
> > >> >>>> I am strongly against encoding serialized XML as JSON strings.
> It is IMO totally useless and the spec would have to deal with awkward
> > >> >>>> Perhaps the best solution would be to state that JSON encoding
> is incompatible with data models that use "anyxml".
> > >> >>>>
> > >> >>>
> > >> c) sounds like the same data can be encoded in different ways
> depending on what the implementor finds useful, e.g. encode all numbers as
> strings.
> > >>
> > >

(e) does not even make sense. "Otherwise" is never paired with "MUST".

Interoperability expectations should be very low for anyxml.
We already established that over 1000 emails ago.

/js

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Sat, 14 Nov 2015 10:27:19 +0100

> On 13 Nov 2015, at 19:19, Andy Bierman <[email-masked]> wrote:
> 
> 
> 
> On Wed, Nov 11, 2015 at 11:51 PM, Juergen Schoenwaelder <[email-masked]> wrote:

That "otherwise it is unrestricted" was intended to mean "there are no restrictions other than this", so it looks like a gap in my understanding of English semantics. 

> We already established that over 1000 emails ago.

I agree, but on the other hand I think it can be useful in specific situations where both the client and server are prepared to understand and handle it properly. We cannot expect generic tools to do anything reasonable with it - it is only good to make sure they won't break.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Sat, 14 Nov 2015 09:05:00 -0800

On Sat, Nov 14, 2015 at 1:27 AM, Ladislav Lhotka <[email-masked]> wrote:

> That "otherwise it is unrestricted" was intended to mean "there are no
> restrictions other than this", so it looks like a gap in my understanding
> of English semantics.
>
> >
> I agree, but on the other hand I think it can be useful in specific
> situations where both the client and server are prepared to understand and
> handle it properly. We cannot expect generic tools to do anything
> reasonable with it - it is only good to make sure they won't break.
>

This is really vague.
The rules should not change for anyxml, so (c) is the best choice.
If somebody actually reported 2 real-world tools that are currently broken
because of anyxml misinterpretation, then this issue would be relevant.
Got any examples to share?  If not, then let's pick (c) and move on.

YANG 1.1 is going to take 2 more years if we slowly revisit every issue.
I thought the whole point of the issue tracker was to prevent this sort
of thing.   The rule should be "what new details have emerged that
should cause us to change the previous decision?"

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 16 Nov 2015 12:55:52 +0100

> should cause us to change the previous decision?"

Andy, please note that this is a discussion primarily around the JSON
document and not around the YANG 1.1 document.

/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 16 Nov 2015 13:09:20 +0100

> On 16 Nov 2015, at 12:55, Juergen Schoenwaelder <[email-masked]> wrote:
> 

Except that it might be useful to clarify in 6020bis whether "anyxml" really means

1. specifically "any XML", or

2. schema-less data in any supported encoding.

If it is #1, then arguably c/e are not acceptable options for the JSON encoding. I prefer #2.

Lada

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 16 Nov 2015 14:40:23 +0100

> If it is #1, then arguably c/e are not acceptable options for the JSON encoding. I prefer #2.

I believe the outcome of the discussion is that anyxml means any XML.
There also was clear agreement to not change the definition of anyxml.

/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 16 Nov 2015 15:04:31 +0100

> On 16 Nov 2015, at 14:40, Juergen Schoenwaelder <[email-masked]> wrote:
> 

I don't think there has been any clear consensus so far, I understand you and Martin support b) while Andy and I voted for c/e).

> There also was clear agreement to not change the definition of anyxml.

OK, using the same logic, sections that talk about NETCONF and edit-config (such as 7.9.6) don't apply to other protocols and their operations. If we can agree on this as a general principle, i.e. not to do any extrapolations of the 6020bis text, I am prepared to accept alternative b) for JSON ancoding of anyxml, but I'd like to ask those who support it to propose a concrete wording.

Thanks, Lada

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 16 Nov 2015 15:32:33 +0100

> OK, using the same logic, sections that talk about NETCONF and edit-config (such as 7.9.6) don't apply to other protocols and their operations. If we can agree on this as a general principle, i.e.…

To be clear: I made a statement about anyxml. Any generalization you
are trying to derive from my statement is not covered by my statement.

/js

---

From: Andy Bierman <[email-masked]>
Date: Mon, 16 Nov 2015 08:50:45 -0800

> document and not around the YANG 1.1 document.
That doesn't mean we haven't discussed this issue for a long time already.
I thought we decided anyxml is not that interoperable and so we have
anydata that is supposedly interoperable.

So why try to constrain the JSON encoding?
For anyone sending mixed mode XML encoded as JSON,
they can get creative and do whatever they want.

For everybody else, anydata is simple enough to encode and decode.

Andy

---

From: Robert Wilton <[email-masked]>
Date: Tue, 17 Nov 2015 10:19:15 +0000

As a possible compromise, what about something like:

The JSON encoding defines that anyxml may be encoded in whatever way the 
implementor finds useful, or even not at all.  If a preferred custom 
encoding is not being used, then it is suggested that anyxml data be 
encoded as a string containing XML to maximize legacy interoperability.

Rob

On 16/11/2015 16:50, Andy Bierman wrote:
>
>
> On Mon, Nov 16, 2015 at 3:55 AM, Juergen Schoenwaelder 
> <[email-masked] 
> <mailto:[email-masked]>> wrote:
>

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 17 Nov 2015 15:44:24 +0100

> On 17 Nov 2015, at 11:19, Robert Wilton <[email-masked]> wrote:
> 
> The JSON encoding defines that anyxml may be encoded in whatever way the implementor finds useful, or even not at all.  If a preferred     custom encoding is not being used, then it is suggested that anyxml data be encoded as a string containing XML to maximize legacy interoperability.

While this is quite complicated, I don't think it helps in any way. There are no anyxml objects that need to be encoded, so what an implementor finds useful is totally irrelevant. The yang-json spec should state what's permitted as the content of an anyxml instance in JSON encoding, and that's all. To this end, the current text is IMO sufficient, except that the "otherwise" part may be misleading - but it's not necessary.

Note that XML-in-JSON-string has its share of problems and may not be interoperable either: the content of an anyxml instance in XML encoding needn't be a well-formed XML document.

Lada

>> On Mon, Nov 16, 2015 at 3:55 AM, Juergen Schoenwaelder <[email-masked]> wrote:

---

From: Andy Bierman <[email-masked]>
Date: Tue, 17 Nov 2015 09:08:24 -0800

On Tue, Nov 17, 2015 at 6:44 AM, Ladislav Lhotka <[email-masked]> wrote:

> While this is quite complicated, I don't think it helps in any way. There
> are no anyxml objects that need to be encoded, so what an implementor finds
> useful is totally irrelevant. The yang-json spec should state what's
> permitted as the content of an anyxml instance in JSON encoding, and that's
> all. To this end, the current text is IMO sufficient, except that the
> "otherwise" part may be misleading - but it's not necessary.
>
> Note that XML-in-JSON-string has its share of problems and may not be
> interoperable either: the content of an anyxml instance in XML encoding
> needn't be a well-formed XML document.
>
>
So now anyxml includes XML that is not even well-formed?
This is news to me.  Maybe you mean it is a well-formed snippet
that may not be complete as an XML instance document.

You cannot say "MUST do X, otherwise do Y".  This is incorrect
use of RFC 2119 terminology.  If MUST is used, that is the only option.
You also need to explain exactly what interoperability is lost if the MUST
is not followed.  (e.g., mixed mode XML will not be translated properly to
JSON).

Andy

Lada

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 17 Nov 2015 21:03:53 +0100

> On 17 Nov 2015, at 18:08, Andy Bierman <[email-masked]> wrote:
> 
> 
> 

Sure. First, in the XML encoding the anyxml chunk may inherit namespace and entity declarations from the outer context. This could be corrected by adding these declarations to the JSON string. But then also, if we have

anyxml foo;

then this is a valid XML encoding

<foo>
  <bar/>
  <baz/>
</foo>

but the JSON string value

"foo": "<foo/><bar/>"   

is not a well-formed XML document because it doesn't have a single document element.

> use of RFC 2119 terminology.  If MUST is used, that is the only option.

OK, I will reformulate it.

> is not followed.  (e.g., mixed mode XML will not be translated properly to JSON).

If the MUST isn't followed, then the interoperability problems that motivated I-JSON restrictions come into play. It has nothing to do with mixed XML content - translation between JSON and XML (and vice versa) in general cannot be guaranteed.

Lada

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 17 Nov 2015 21:11:45 +0100

> On 17 Nov 2015, at 21:03, Ladislav Lhotka <[email-masked]> wrote:
> 
>> 

Sorry, this was supposed to be

"foo" : "<bar/><baz/>"

Lada

---

From: Andy Bierman <[email-masked]>
Date: Tue, 17 Nov 2015 12:18:32 -0800

On Tue, Nov 17, 2015 at 12:11 PM, Ladislav Lhotka <[email-masked]> wrote:

> >>> The JSON encoding defines that anyxml may be encoded in whatever way
> the implementor finds useful, or even not at all.  If a preferred
>  custom encoding is not being used, then it is suggested that anyxml data
> be encoded as a string containing XML to maximize legacy interoperability.
> >>
> >> While this is quite complicated, I don't think it helps in any way.
> There are no anyxml objects that need to be encoded, so what an implementor
> finds useful is totally irrelevant. The yang-json spec should state what's
> > Sure. First, in the XML encoding the anyxml chunk may inherit namespace
> and entity declarations from the outer context. This could be corrected by
> adding these declarations to the JSON string. But then also, if we have
> >
> > anyxml foo;
> >

(!) The internal representation of data is implementation-specific.

> Sorry, this was supposed to be

If the internal server code generates complex nodes they will
be rendered correctly.

   { "foo" : {
        "bar":[null],
        "baz":[null]
      }
   }

This is what our server will generate if the internal representation
would be rendered in XML as you describe. This encoding can be
easily converted to XML.

Andy

> > If the MUST isn't followed, then the interoperability problems that
> motivated I-JSON restrictions come into play. It has nothing to do with
> mixed XML content - translation between JSON and XML (and vice versa) in
> general cannot be guaranteed.
> >
> > Lada
> >
> >>
> >>
> >> Andy
> >>
> >>
> >>
> >> Lada
> >>
> >>>
> >>> Rob
> >>>
> >>>

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 18 Nov 2015 08:03:02 +0100

> On 17 Nov 2015, at 21:18, Andy Bierman <[email-masked]> wrote:
> 
> 
> 

Yes, but we are talking about encoding anyxml instances as JSON strings containing well-formed XML - this is what Martin proposed in Yokohama, if I understood it correctly.

And yes, I think we are wasting time. Seeking interoperability where there is none is useless. I propose this change to sec. 5.6 of the yang-json document:

OLD
   An anyxml instance is encoded as a JSON name/value pair which MUST
   satisfy I-JSON constraints.  Otherwise it is unrestricted, i.e., the
   value can be an object, array, number, string or one of the literals
   "true", "false" and "null".

NEW
   An anyxml instance is encoded as a JSON name/value pair which MUST
   satisfy I-JSON constraints.  The value can be an object, array, number,
   string or one of the literals "true", "false" and "null".

This BTW also allows for strings containing XML.

Lada

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 18 Nov 2015 09:36:03 +0100

Lada,

what is the purpose of the second sentence? I checked RFC 7159 and it
seems the proposed second sentence simply repeats the set of possible
JSON values listed in section 3 of RFC 7159. So why is this sentence
technically needed?

/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 18 Nov 2015 09:40:49 +0100

> On 18 Nov 2015, at 09:36, Juergen Schoenwaelder <[email-masked]> wrote:
> 

It's there to emphasize it can be any JSON value, because in one of the previous versions it was limited to an object. Maybe it's really not needed.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Wed, 18 Nov 2015 07:10:46 -0800

On Tue, Nov 17, 2015 at 11:03 PM, Ladislav Lhotka <[email-masked]> wrote:

> > > Sure. First, in the XML encoding the anyxml chunk may inherit
> namespace and entity declarations from the outer context. This could be
> corrected by adding these declarations to the JSON string. But then also,
> if we have
> > >
> > > anyxml foo;
> > >
> Yes, but we are talking about encoding anyxml instances as JSON strings
> containing well-formed XML - this is what Martin proposed in Yokohama, if I
> understood it correctly.
>
>

This string encoding is totally useless.
We will never implement it.  There is no way any client code is going to
double parse the string. (i.e. pass the string to an XML parser)

This is useful for WEB banners where the correct modeling tool is
(and always has been) a leaf.

And yes, I think we are wasting time. Seeking interoperability where there
> is none is useless. I propose this change to sec. 5.6 of the yang-json
> document:
>
> OLD

IMO this is a bit more clear:
s/Otherwise it is/It is otherwise/

It was not clear before that otherwise is in addition to MUST.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 18 Nov 2015 16:21:05 +0100

> On 18 Nov 2015, at 16:10, Andy Bierman <[email-masked]> wrote:
> 
> 
> 

Agreed. In fact, the main benefit of sending something as anyxml (in both XML and JSON encoding) IMO is that it can be parsed in one shot together with the rest of the data.

> It was not clear before that otherwise is in addition to MUST.

Yes, but perhaps it is not needed there, as Juergen suggested. This should suffice:

NEW
   An anyxml instance is encoded as a JSON name/value pair where the value
   MUST satisfy I-JSON constraints.

Lada