Subject: [netconf] Question on restconf empty list reply

From: Olof Hagsand <[email-masked]>
Date: Sun, 23 Feb 2020 12:12:58 +0100

Hello,
In a restconf GET request of an empty YANG list using JSON encoding,
what is the expected reply? I.e. using a yang list "x" in module "m"
(pseudo http):
  GET /restconf/data/m:x
  Accept: application/yang-data+json

Is the reply (1):
  HTTP/1.1 200 OK
  {
    "m:x": []
  }

or should it be (2)
  "404 Not Found" status-line and error-tag value "invalid-value"?

Apologizes that this is a basic question, and I am sure this is resolved
properly, but we have some discussions with users on how to properly
interpret the description of GET in RFC 8040 Section 4.3 and RFC 7951.

Regards,
Olof Hagsand,
Clixon maintainer

---

From: Schönwälder, Jürgen <[email-masked]>
Date: Sun, 23 Feb 2020 11:28:30 +0000

My reading of RFC 7951 section 5.4 is that an empty list results in an
empty JSON array. A related question is likely whether the empty list
exists or not. RFC 8040 says a 404 response indicates that the
resource does not exist (or is not accessible). So both responses may
be reasonable, depending on the existence/accessibility of the resource.

/js

On Sun, Feb 23, 2020 at 12:12:58PM +0100, Olof Hagsand wrote:
> _______________________________________________
> netconf mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netconf

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1 | 28759 Bremen | Germany
Fax:   +49 421 200 3103         <https://www.jacobs-university.de/>

---

From: Olof Hagsand <[email-masked]>
Date: Sun, 23 Feb 2020 21:01:35 +0100

Thank you for the answer!
What does "the empty lists exists or not" mean, how is an existing empty
list different from a non-existent list wrt YANG? I would think that a
YANG list have instances (= it exists), or it does not have instances (=
is empty = does not exist).

Another related question is how the same request but for XML encoding
should be treated, ie a YANG list x with no elements (empty
list/non-existent)?

  GET /restconf/data/m:x
  Accept: application/yang-data+xml

Is the reply "404 Not Found" status-line and error-tag value
"invalid-value" (following the list resource doe snot exist)
or, following the JSON [] (existing empty list) reasoning, a "204 No
Content"?

Again apologizes for these basic questions,
Regards,
Olof Hagsand

---

From: Kent Watsen <[email-masked]>
Date: Mon, 24 Feb 2020 17:35:10 +0000

Hi Olof,

> is empty = does not exist).

Empty “list” (and “leaf-list”) handling seems to be related to the non-presence containers issue that has been tracked here: https://github.com/netmod-wg/yang-next/issues/88 (ps: just updated to reflect this thread).

Personally, I’m in the “explicit" camp.  That is, lists (and NP-containers) do not exist unless explicitly configured and they do not implicitly-disappear when empty.  My server distinguishes between missing (e.g., 404) and empty (204 for XML or 200 for JSON) lists.  Personally, I think it’s best to also return 204 (i.e., for JSON also) as its more RESTful and encoding-independent.   FWIW, RFC 8040 doesn’t mention the ability for GET to return 204, but it doesn’t disallow it either...

> should be treated, ie a YANG list x with no elements (empty

This issue is not encoding-specific, AFIACT.

The only list-related encoding issue I’m aware of is when  the query targets the list node directly, not a parent container, and the list contains more than one element.  In this case, it’s not possible to return a valid XML document, whereas there’s no issue with JSON.  

Off-topic, but a couple fixes have been proposed.  One creates an implicit wrapper node:

	==== start ====
	<collection xmlns=“foo”>
		<x>…</x>
		<x>…</x>
		<x>…</x>
	</collection>
	==== stop ====

Whereas another documents that, only in this case, is it possible to return an invalid XML document:

	==== start ====
	<x xmlns=“foo”>…</x>
	<x xmlns=“foo”>…</x>
	<x xmlns=“foo”>…</x>
	==== stop ====

> "invalid-value" (following the list resource does not exist)
> Content”?

Answered above, I think.

> Again apologizes for these basic questions,

No worries, sometimes it takes fresh eyes to see things  :)

Kent // contributor

---

From: Martin Björklund <[email-masked]>
Date: Mon, 24 Feb 2020 20:17:27 +0100 (CET)

Hi,

>   "404 Not Found" status-line and error-tag value "invalid-value"?

It should be 404, even if the list contains some entries.  The reason
for this is that there is no resource for the list itself, only for
list entries.  See section 3.5 of RFC 8040.

We used to have "collections" for this use case, but it was never finished.

/martin

---

From: Kent Watsen <[email-masked]>
Date: Mon, 24 Feb 2020 20:09:49 +0000

> list entries.  See section 3.5 of RFC 8040.

Assuming you mean the first paragraph not explicitly listing “list” and “leaf-list”, that conflicts with Section 4.3, Paragraph 5.

> We used to have "collections" for this use case, but it was never finished.

:sigh:   ;)

Kent // contributor

---

From: "Sterne, Jason (Nokia - CA/Ottawa)" <[email-masked]>
Date: Mon, 24 Feb 2020 20:36:38 +0000

Hi guys,

For NETCONF this should *not* return an error though right? It should just return an empty data set.

I don't know enough about RESTCONF but I'm surprised it would return an error in this case (too bad it would be inconsistent with NETCONF).

Jason

> -----Original Message-----
> From: netconf <[email-masked]> On Behalf Of Kent Watsen
> Sent: Monday, February 24, 2020 3:10 PM
> To: Martin Björklund <[email-masked]>
> Cc: [email-masked]
> Subject: Re: [netconf] Question on restconf empty list reply
> 
> Assuming you mean the first paragraph not explicitly listing “list” and “leaf-
> list”, that conflicts with Section 4.3, Paragraph 5.
> 
>

---

From: Martin Björklund <[email-masked]>
Date: Tue, 25 Feb 2020 18:38:23 +0100 (CET)

Kent Watsen <[email-masked]> wrote:
> Assuming you mean the first paragraph not explicitly listing “list”
> and “leaf-list”, that conflicts with Section 4.3, Paragraph 5.

Does it?  Actually, that paragraph is not quite clear.  What is a
"resource representing a YANG list object"?

/martin

---

From: Martin Björklund <[email-masked]>
Date: Tue, 25 Feb 2020 18:52:51 +0100 (CET)

Hi,

"Sterne, Jason (Nokia - CA/Ottawa)" <[email-masked]> wrote:
> I don't know enough about RESTCONF but I'm surprised it would return
> an error in this case (too bad it would be inconsistent with
> NETCONF).

The protocol operation is different in NETCONF compared to RESTCONF.
In NETCONF, you would do a get-config or get-data with a _filter_.  In
RESTCONF, you ask for a specific _resoource_, and there is no resource
for a complete list.

/martin

---

From: Olof Hagsand <[email-masked]>
Date: Tue, 25 Feb 2020 21:02:27 +0100

> We used to have "collections" for this use case, but it was never finished.

OK,

I find that explanation semantically sound, although somewhat
disappointing from a usability perspective.
But it makes sense and avoids the pitfalls of differences between
non-existent and empty lists, for example.
But as mentioned elsewhere, I then find paragraph 5 of section 3.5 of
RFC 8040 confusing at best, or even irrelevant, if this is the correct
interpretation.

Thanks.
--Olof

---

From: Kent Watsen <[email-masked]>
Date: Thu, 27 Feb 2020 21:39:43 +0000

Hi Martin,

> "resource representing a YANG list object”?

Right, I read that to mean the ‘list’ object itself, not any list-item.  I think that this view is supported by the continuation of that line:

   If a retrieval request for a data resource representing a YANG
   leaf-list or list object identifies more than one instance and XML
   encoding is used in the response, then an error response containing
   a "400 Bad Request" status-line MUST be returned by the server.  

How else could something else return “more than one instance” and, further, the text specially calls out the known XML encoding limitation (i.e., that there has to be a single document root), so what else could could it be?

Lastly, the paragraph ends:

   Note that a non-configuration list is not required to define any keys.
   In this case, the retrieval of a single list instance is not possible.

Which points to a case when keys are not (actually, cannot be) provided, so it seems that this paragraph as a whole regards targeting the ‘list’ as a resource when no keys are specified, regardless if config true or false.

Looking at our old restconf-collections draft, the examples in Appendix C suggest that we intended to enable the ‘list’ itself to be a resource.  For example (hint: ‘album’ is a ‘list’):

	GET /restconf/data/example-jukebox:jukebox/
	         library/artist=Foo%20Fighters/album/?limit=2

	[I feel that there’s a spurious ‘/‘ before the ‘?' above, and we really meant "...album?limit=2”]

But, regardless, no *keys* are being specified. Hence the inconsistency.  The immediate relief seems to be to either promote ‘list’ being a resource, or strike paragraph 5 in Section 4.3.

That said, I imagine the long-term plan *is* to promote ‘list’ to being a resource (via the restconf-collections draft or, perhaps, YANG-next), as that enables the collection-specific query params to be used.

Enabling the ‘list’ to be a resource  would also resolve the issue how to GET a top-level ‘list’ without filtering a datastore.  For example:

	module ietf-audit-logs {
	   …
	   list audit-log {
             config false;
	      …
	   }
	}

	GET /restconf//ds/ietf-datastores:operational/ietf-audit-logs:audit-log?limit=25

Versus the less-efficient:

	GET /restconf//ds/ietf-datastores:operational?fields=ietf-audit-logs:audit-log

	[this example also struggles with if it’s legal to specify the the collection-specific query params on a datastore resource]

Kent // contributor