Subject: [Netconf] Restconf: rendering lists

From: "Alberto Gonzalez Prieto (albertgo)" <[email-masked]>
Date: Tue, 20 May 2014 19:07:11 +0000

Hi,

I have some questions on how a server must render lists.
Looking at examples in the spec for getting data, I see that when rendering (in json) lists with only one entry , the responses use two approaches: rendering it as an array or a singleton.
(See snippets below).
The requests are different in that the second one requests a list and the first requests a container that contains a list. I am assuming this is not a factor when rendering a list. Is it?
Is the server free to choose between these two ways of encoding a list with a single entry?

GET /restconf/data/example-jukebox:jukebox?depth=3 HTTP/1.1
      Host: example.com
      Accept: application/yang.data+json

   The server might respond:

      HTTP/1.1 200 OK
      Date: Mon, 23 Apr 2012 17:11:30 GMT
      Server: example-server
      Cache-Control: no-cache
      Pragma: no-cache
      Content-Type: application/yang.data+json

      {
        "example-jukebox:jukebox" : {
          "library" : {
            "artist" : [ null ]
          },
          "playlist" : [ # 1 playlist encoded as an array
            {
              "name" : "Foo-One",
              "description" : "example playlist 1",
              "song" : [ null ]
            }
          ],
          "player" : {
            "gap" : 0.5
          }
        }
      }

      GET /restconf/data/example-jukebox:jukebox/
        library/artist/Foo%20Fighters/album  HTTP/1.1
      Host: example.com
      Accept: application/yang.data+json

   The server might respond:

      HTTP/1.1 200 OK
      Date: Mon, 23 Apr 2012 17:02:40 GMT
      Server: example-server
      Content-Type: application/yang.data+json
      Cache-Control: no-cache
      Pragma: no-cache
      ETag: a74eefc993a2b
      Last-Modified: Mon, 23 Apr 2012 11:02:14 GMT

      {
        "album" : {  # 1 album, no array encoding
          "name" : "Wasting Light",
          "genre" : "example-jukebox:alternative",
          "year" : 2011
        }
      }

Considering XML encoding now.

The XML equivalent of the example above could be something like:

      <album xmlns="http://example.com/ns/example-jukebox">

         <name>Wasting Light</name>
         <genre>example-jukebox:alternative</genre>
         <year>2011</year>
      </album>

Now, if there were more than one album, to have in the response a legal XML doc (a single root), I understand we would need to include album's parent.

Something like:

<artist>

    <album>

        (…)

    </album>

    <album>

        (…)

    </album>

</artist>

Is this the case?

If so, would it make sense encoding all lists (regardless the number of entries in it) using the same structure?

That is, rendering the example above as:

<artist>

    <album>

        (…)

    </album>

</artist>

Thanks,

Alberto

---

From: Andy Bierman <[email-masked]>
Date: Tue, 20 May 2014 12:41:25 -0700

On Tue, May 20, 2014 at 12:07 PM, Alberto Gonzalez Prieto (albertgo) <
[email-masked]> wrote:

>  Looking at examples in the spec for getting data, I see that when
> rendering (in json) lists with only one entry , the responses use two
> approaches: rendering it as an array or a singleton.
>  (See snippets below).
>  The requests are different in that the second one requests a list and the
> first requests a container that contains a list. I am assuming this is not
> a factor when rendering a list. Is it?
The 2nd example is a bug.
According to the JSON draft, all YANG lists are encoded as arrays.

Andy

>           *"playlist" : [ # 1 playlist encoded as an array
>             {
>               "name" : "Foo-One",
>           ],*
>         *"album" : {  # 1 album, no array encoding
>         }*
>         (...)
>         (...)
>         (...)
> _______________________________________________
> Netconf mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netconf
>
>

---

From: "Alberto Gonzalez Prieto (albertgo)" <[email-masked]>
Date: Tue, 20 May 2014 20:40:12 +0000

Thanks Andy,

I appreciate the prompt response.

Can you comment on the XML encoding?

Thanks,

Alberto

      <album xmlns="http://example.com/ns/example-jukebox"><http://example.com/ns/example-jukebox%22%3E>

Something like:

<artist>

    <album>

        (…)

    </album>

    <album>

        (…)

    </album>

</artist>

Is this the case?

<artist>

    <album>

        (…)

    </album>

</artist>

Thanks,

Alberto

_______________________________________________
Netconf mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/netconf

---

From: Andy Bierman <[email-masked]>
Date: Tue, 20 May 2014 14:07:01 -0700

>  Can you comment on the XML encoding?
It is currently an open issue for the next draft.
How should collections be handled?

If the target resource is a list, then a wrapper element
is needed in XML to return a valid XML instance document,
The element <data> or <collection> is being considered
for this purpose.

A side issue is whether the JSON encoding should also
have this wrapper, so client applications can use the
same code for both encodings, or should a JSON array should
be returned instead, without the wrapper (just the target resource list)?

The current proposal is to create a new "YANG collection" resource
and return a top-level <data> element if a list or leaf-list is the
target resource.  A wrapper for JSON will be used so the data
matches the resource type (application/yang.collection has N child nodes
that
are type application/yang.data).

Andy

---

From: "Alberto Gonzalez Prieto (albertgo)" <[email-masked]>
Date: Tue, 20 May 2014 21:32:27 +0000

Thanks Andy,

For the root of the xml document, is the parent of the list being considered?
A pro I see in this is that it would not require introducing nodes that are not present in yang modules.

On the "wrapper" for json. It would make sense to me aiming to keep contents and their representation orthogonal.
That is, if there is a wrapper for XML, also include it for JSON.

Thanks,

Alberto

From: Andy Bierman <[email-masked]<mailto:[email-masked]>>
Date: Tuesday, May 20, 2014 2:07 PM
To: Alberto Gonzalez Prieto <[email-masked]<mailto:[email-masked]>>
Cc: Netconf <[email-masked]<mailto:[email-masked]>>
Subject: Re: [Netconf] Restconf: rendering lists

On Tue, May 20, 2014 at 1:40 PM, Alberto Gonzalez Prieto (albertgo) <[email-masked]<mailto:[email-masked]>> wrote:
Thanks Andy,

matches the resource type (application/yang.collection has N child nodes that

Thanks,

Alberto

Andy

Something like:

<artist>

    <album>

        (…)

    </album>

    <album>

        (…)

    </album>

</artist>

Is this the case?

<artist>

    <album>

        (…)

    </album>

</artist>

Thanks,

Alberto

---

From: Andy Bierman <[email-masked]>
Date: Tue, 20 May 2014 14:50:06 -0700

> are not present in yang modules.
Except for top-level nodes -- then the datastore parent would be returned.
The co-authors agreed on a new resource type (collection)
to identify this special case:
   *  The 'select' parameter affects data returned in collections
   * The data returned may not represent a valid resource
      (because it is only a subset or depth altered)
   * The paging parameters being added (offset + limit) also only apply
     to collections.

Note that you can use RESTCONF to get the parent and all instances
of the list:

  container collection {
     list list { ... }
  }

  GET /restconf/datastore/collection?select=list

 On the "wrapper" for json. It would make sense to me aiming to keep
> contents and their representation orthogonal.
Agreed

Andy

>   From: Andy Bierman <[email-masked]>
> To: Alberto Gonzalez Prieto <[email-masked]>
> Cc: Netconf <[email-masked]>

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 21 May 2014 13:43:11 +0200

> be returned instead, without the wrapper (just the target resource list)?

In my view, this shouldn't be needed in JSON because it means unnecessary clutter there. Clients and servers should be able handle such differences in XML/JSON data encoding quite easily.

Lada

-- 
Ladislav Lhotka, CZ.NIC Labs
PGP Key ID: E74E8C0C

---

From: Andy Bierman <[email-masked]>
Date: Wed, 21 May 2014 08:01:42 -0700

On Wed, May 21, 2014 at 4:43 AM, Ladislav Lhotka <[email-masked]> wrote:

> In my view, this shouldn't be needed in JSON because it means unnecessary
> clutter there. Clients and servers should be able handle such differences
> in XML/JSON data encoding quite easily.
>
>
If there was a wrapper in JSON then the exact same client code
path expressions will work in both.  It is a hassle to alter the schema
for URI or message body parsing, based on encoding.

There are also operations/query parameters that only apply
to collections.  Returning the same node sometimes as
a data resource and other times as a collection resource
would be confusing.

Lada

Andy

> >>> Now, if there were more than one album, to have in the response a
> legal XML doc (a single root), I understand we would need to include
> album's parent.
> >>>
> >>> Something like:
> >>>
> >>>
> >>> <artist>
> >>>
> >>>     <album>
> >>>
> >>>         (...)
> >>>
> >>>     </album>
> >>>
> >>>     <album>
> >>>
> >>>         (...)
> >>>
> >>>     </album>
> >>>
> >>> </artist>
> >>>
> >>>
> >>> Is this the case?
> >>>
> >>> If so, would it make sense encoding all lists (regardless the number
> of entries in it) using the same structure?
> >>>

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 21 May 2014 18:11:47 +0200

> for URI or message body parsing, based on encoding.

Hmm… The dummy container should never appear in an URI, right? And as for message body parsing,
it is just a simple matter of adding (or removing) the extra container after the payload has been parsed.  

> a data resource and other times as a collection resource

The data model says whether it is a collection or not.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Wed, 21 May 2014 12:18:47 -0700

On Wed, May 21, 2014 at 9:11 AM, Ladislav Lhotka <[email-masked]> wrote:

> > In my view, this shouldn't be needed in JSON because it means
> unnecessary clutter there. Clients and servers should be able handle such
> differences in XML/JSON data encoding quite easily.
> >
> >
> Hmm... The dummy container should never appear in an URI, right? And as for
> message body parsing,
True -- it is just message body processing.
JSON and XML are consistent except XML
does not allow top-level arrays.

It is not just parsing. The conceptual model requires a single
instance of a top-level element.

I actually like the "parent node" solution better than returning
different structures for JSON and XML.

> The data model says whether it is a collection or not.
Only partly. Retrieving a collection is a protocol operation.

     list foo {
         key bar;
         leaf bar { type string; }
         container foo {
            leaf bar { type int32; }
         }
     }

  GET /restconf/data/foo?select=bar;foo/bar

What is returned?

  { "foo" : [
     { "bar": [ "fred", 42 ] },
     { "bar": [ "barney", 64 ] }
    ]
  }

What happens when 2 different "bar" nodes are combined?
They have the same module name and same local name.

Lada
Andy

---

From: Kent Watsen <[email-masked]>
Date: Thu, 22 May 2014 16:35:00 +0000

[KENT] From the client's perspective, I think it's OK to return two different structures, because I don't expect a given client to flip between XML and JSON within some block of code.  On the server side, sending two different structures  may or may not be trivial, depending if the json/xml encoders are separate or if its a single encoder with a parameter passed into it.   What's easier for the RESTCONF server?

---

From: Andy Bierman <[email-masked]>
Date: Thu, 22 May 2014 10:08:48 -0700

On Thu, May 22, 2014 at 9:35 AM, Kent Watsen <[email-masked]> wrote:

>  [KENT] From the client's perspective, I think it's OK to return two
> different structures, because I don't expect a given client to flip between
> XML and JSON within some block of code.  On the server side, sending two
> different structures  may or may not be trivial, depending if the json/xml
> encoders are separate or if its a single encoder with a parameter passed
> into it.   What's easier for the RESTCONF server?
>
>
>

The server knows which format it is going to send so always
using a wrapper for XML and never using it for JSON is trivial.

Streaming is not trivial.  I cannot over-state how much easier it
is to stream XML than JSON.  By streaming, I do not just mean
that the data is transmitted on the fly, and not rendered from
a static representation in memory.  That is hard enough in JSON.

On real servers, the instrumentation can be distributed.
The filtering implementation can be complicated.  The access control
can be implemented inline.  All of which make it impossible to
know when the headers are generated whether any data will actually
be returned for a given <get> or GET.

For NETCONF, the <data> wrapper is always sent, and there
is no problem for <get>.  But in RESTCONF, the server needs
to generate the status code header line before possibly knowing if
there will be any data to return, so it cannot always send 204 No Content
instead of 200 OK.

That's why we always return a <data> wrapper object (in XML or JSON)
if the query can possibly match some data.
The client must be prepared to get back an empty <data/> element
or in JSON:  { "data" : null }.  A message body has to be sent and it
is critical in XML to return a valid instance document if the search
matches multiple nodes.

Andy