Subject: [netmod] RFC7952 JSON streaming decoding efficiency?

From: Robert Varga <[email-masked]>
Date: Wed, 27 Feb 2019 23:26:27 +0100

Hello,

reading RFC7952 sections 5.2.2-5.2.4, it would seem that metadata object
(5.2.2) can occur at any position in a JSON object, as well as metadata
attachments to leaf/leaf-list/anyxml (5.2.3, 5.2.4) can occur any
distance (before or after) from the data element they attach to.

The examples list the metadata object as the first item, and attachments
immediately after the data element, but I could find no text which would
make this required or recommended for document producers.

While this is completely workable when the parsing state is received
into a mutable structure, it requires full parser event buffering in
streaming setups.

One example would be JSON->XML transformation using token streaming
interfaces (like GSON's JsonReader and
javax.xml.stream.XMLStreamWriter): since metadata can occur at any place
in JSON, but is required to be emitted just after opening XML element,
we must completely parse the JSON document before we can start emitting
the XML document.

Is this intentional or just an omission?

Thanks,
Robert

---

From: "Rob Wilton (rwilton)" <[email-masked]>
Date: Thu, 28 Feb 2019 09:29:11 +0000

Hi Robert,

Isn't this just a limitation of JSON, in that the elements in an object are unordered (RFC 8259)?

Thanks,
Rob

-----Original Message-----
From: netmod <[email-masked]> On Behalf Of Robert Varga
Sent: 27 February 2019 22:26
To: [email-masked]
Subject: [netmod] RFC7952 JSON streaming decoding efficiency?

Hello,

reading RFC7952 sections 5.2.2-5.2.4, it would seem that metadata object
(5.2.2) can occur at any position in a JSON object, as well as metadata attachments to leaf/leaf-list/anyxml (5.2.3, 5.2.4) can occur any distance (before or after) from the data element they attach to.

The examples list the metadata object as the first item, and attachments immediately after the data element, but I could find no text which would make this required or recommended for document producers.

While this is completely workable when the parsing state is received into a mutable structure, it requires full parser event buffering in streaming setups.

One example would be JSON->XML transformation using token streaming interfaces (like GSON's JsonReader and
javax.xml.stream.XMLStreamWriter): since metadata can occur at any place in JSON, but is required to be emitted just after opening XML element, we must completely parse the JSON document before we can start emitting the XML document.

Thanks,
Robert

---

From: Robert Varga <[email-masked]>
Date: Thu, 28 Feb 2019 10:46:29 +0100

> Hi Robert,

Hey Rob,

> Isn't this just a limitation of JSON, in that the elements in an object are unordered (RFC 8259)?

Yes, but I believe this is object semantics leaking to on-wire format:
while a JSON object is inherently unordered, when emitting object to the
wire, implementations can choose to order the pairs to make
receiver-side processing more efficient.

I wonder if it would be of value to communicate such assumptions out of
band, like separate content-types. In case of RFC7952 the following
hints would be useful:
- the document does not contain metadata (i.e. plain RFC7951)
- "@" occurs as a first element in an object or not at all
- "@foo" occurs immediately after "foo" or not at all

This, of course, would be purely optional optimization...

Regards,
Robert

---

From: "Rob Wilton (rwilton)" <[email-masked]>
Date: Thu, 28 Feb 2019 09:54:57 +0000

Broadly sounds like a reasonable idea to me, but I suspect that others may argue that you are just creating a new JSON like language.

If we were to investigate anything like this, then it might also be a good idea to consider the ordering of list keys within a list entry (i.e. put them first).

Thanks,
Rob

-----Original Message-----
From: Robert Varga <[email-masked]> 
Sent: 28 February 2019 09:46
To: Rob Wilton (rwilton) <[email-masked]>; [email-masked]
Subject: Re: [netmod] RFC7952 JSON streaming decoding efficiency?

Hey Rob,

Yes, but I believe this is object semantics leaking to on-wire format:
while a JSON object is inherently unordered, when emitting object to the wire, implementations can choose to order the pairs to make receiver-side processing more efficient.

I wonder if it would be of value to communicate such assumptions out of band, like separate content-types. In case of RFC7952 the following hints would be useful:
- the document does not contain metadata (i.e. plain RFC7951)
- "@" occurs as a first element in an object or not at all
- "@foo" occurs immediately after "foo" or not at all

Regards,
Robert

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 28 Feb 2019 12:53:05 +0100

Hi Robert,

> receiver-side processing more efficient.

While this is possible in principle, it would require custom
serializers/deserializers, at least in some programming languages.

We tried to adhere to I-JSON [RFC 7494] that states: "The order of object
members in an I-JSON message does not change the meaning of an I-JSON message."

JSON has other issues when it comes to streaming, so perhaps it would be better
to use another representation.

Lada

> _______________________________________________
> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod
-- 
Ladislav Lhotka
Head, CZ.NIC Labs
PGP Key ID: 0xB8F92B08A9F76C67