Subject: Re: [Netconf] Representing URLs

From: Wojciech Dec <[email-masked]>
Date: Tue, 3 Dec 2013 16:39:40 +0100

Following up some of my earlier questions... Inline...

On 29 November 2013 16:59, Andy Bierman <[email-masked]> wrote:
>
>
>
> On Fri, Nov 29, 2013 at 6:01 AM, Wojciech Dec <[email-masked]> wrote:
>>
>> Hello Restconf authors,
>>
>> I would like to ask a few questions and seek your thoughts on the topic of
>> URL representation in the API
>> Currently Yang allows two forms by which one could seek to have URI data
>> be represented in a model:
>>
>> A.
>> leaf someUri {
>>     type instance-identifier;
>> //some Xpath expression to a node
>> }
>>
>> B.
>> leaf anotherUri {
>>     type yang:uri;
>>     default "/my_uri/is/here"
>> }
>>
>> Now, while the above is perhaps sufficient for some well known absolute
>> paths, there appear to be a couple of problems in terms of  a Restful API:
>>
>> 1. Based on the current Restconf spec, both A and B above when faced with
>> a GET would appear to expose a URI, which the client would have to do some
>> manipulation magic on it before use. What a Restful API would be more likely
>> to expose instead is a URL, eg in JSON:
>> {
>>     "url" : "http://example.com/files/v1/documents/abc123"
>> }
>
>
>
> I do not understand the concern.
> One leaf is //restconf/config/someUri and the other is
> /restconf/config/anotherUri.
> What is the manipulation magic?  Constructing /path/to/data/node based on
> YANG?
> That is the point of RESTCONF.  There are already plenty of solutions for
> using
> REST APIs for ad-hoc data.  I do not see any reason to develop RESTCONF for
> clients that want to ignore YANG.  There are already have plenty of choices
> for that.
>
>
>>
>>
>> It would appear to be sensible to add to the Restconf spec a URL
>> generation capability. I.e. have Restconf transform URIs into canonical
>> URLs. Thoughts?
>
>
> Can you describe the solution you have in mind?
>
>>
>>
>> 2. A URL to a data-model specific method
>> Suppose that the model was also defining an RPC, along the lines of the
>> "play" RPC in the Jukebox example. Now, as part of the song resource access
>> API, it would be natural to have such a method returned in a URL. That would
>> also be much more Resful than the currently implicit "/operations" resource
>> listing.
>> While it may be possible to use B. above to some degree, that is still
>> below par as it is not validated in the model.
>> Use of A. appears, to me at least, not possible since the RPC is not a
>> node.
>> Thus, is there a way to have Restconf return an RPC/services list for the
>> data? Eg:
>>
>> {
>>     "songs":
>>     [
>>         a list of songs, 1, 2, etc
>>     ],
>>     "rpc":
>>     {
>>         "play": [ "http://example.com/operations/example-jukebox:play"]
>>     }
>> }
>>
>
> The API already has /restconf/operations/<YANG-rpc-name>.
>
> YANG is not object-oriented, so /restconf/config/routing/<RPC-name>
> is not how the RPC is defined.  You are describing a proprietary
> extension.
>
>> 3. Use of current() function as predicate in URIs/URLs
>>
>> It would be useful to be able to use the "current()" function to construct
>> URIs/URLs returned in Restconf. The spec does not make it clear on whether
>> this would actually work in A or B above. Would it, or is there some other
>> way?
>>
>
>
> The URI is not an XPath expression. There are no predicates allowed,
> I don't think current() is allowed outside a predicate.

Ok, so what is the way in Yang to have a predicate (e.g. current())
based expression that ends up being represented as a URI in Restconf?
Use of the current() predicate in the instance-identifier appears not
to be supported (at least by pyang).

Thanks,
Wojciech.

>> Thanks,
>> Wojciech.
> Andy

---

From: Andy Bierman <[email-masked]>
Date: Tue, 3 Dec 2013 07:55:55 -0800

On Tue, Dec 3, 2013 at 7:39 AM, Wojciech Dec <[email-masked]> wrote:

> ...
There is none. An instance identifier is the absolution path from root
to the specific node being identified.  It cannot be used for another
purpose.  The ietf-yang-types module has an xpath1.0 typedef
that allows current() in a predicate.

Here is the ABNF from RFC 6020:

 node-identifier     = [prefix ":"] identifier

 ;; Instance Identifiers

   instance-identifier = 1*("/" (node-identifier *predicate))

   predicate           = "[" *WSP (predicate-expr / pos) *WSP "]"

   predicate-expr      = (node-identifier / ".") *WSP "=" *WSP
                         ((DQUOTE string DQUOTE) /
                          (SQUOTE string SQUOTE))

   pos                 = non-negative-integer-value

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 3 Dec 2013 16:58:42 +0100

> to be supported (at least by pyang).

Predicates in instance-identifiers can be used only for matching list keys against constant strings, see sec. 9.13 in RFC 6020.

Can you give an example of an effect you would like to achieve?

Lada

> 
> Thanks,
> Wojciech.
> 
>> 
>>> 
>>> Thanks,
>>> Wojciech.
>>> 
>> 
>> Andy
>> 
> _______________________________________________
> Netconf mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netconf

--
Ladislav Lhotka, CZ.NIC Labs
PGP Key ID: E74E8C0C

---

From: Wojciech Dec <[email-masked]>
Date: Tue, 3 Dec 2013 18:47:41 +0100

> Can you give an example of an effect you would like to achieve?

Starting with a basic example: In a data-model for interfaces/x/y, I
would like the ability to actually have a reference to another node in
the model, that in Restconf ends up shwoing up as a URI. Eg. getting
at the URI /interfaces/x/y, would return data which would also give me
a URI for "/line-cards/foo/serial-number".

A hypothetical Yang data-model for this could be:
list interfaces {
    key some;
    leaf some {
       type string;
    }
    list details;
      key id;
      leaf id {
        type string;
      }
     Other stuff
     leaf someUri {
         type instance-identifier;
     // Xpath expression to the line-cards/foo
     }
   }
}

In the instance-identifier, having a leafref like current()
restriction/replacement would appear to be useful in cases where wants
to construct such a URI by using as a piece the context of the current
node.

Open to your suggestions.

Thanks,
Wojciech.

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 3 Dec 2013 20:40:26 +0100

>     // Xpath expression to the line-cards/foo

Assuming that line-cards also appear somewhere in the data tree, a leafref would be a more natural way of representing the reference - and then you can use current(), too.

I have myself never used an instance-identifier in any data model yet, presumably they are mainly useful in notifications.

Lada

---

From: Wojciech Dec <[email-masked]>
Date: Wed, 4 Dec 2013 11:10:51 +0100

> I have myself never used an instance-identifier in any data model yet, presumably they are mainly useful in notifications.

So leafrefs are great, but if I interpret them correctly in rfc6020
(http://tools.ietf.org/html/rfc6020#page-124), their usage in the
context of Restconf would result not in a URI for the leaf being
passed to a client (say after a GET), but rather the value of that
leaf. It also does not appear to be suited to referencing a data node
(eg container).

Regards,
Wojciech.

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 4 Dec 2013 13:33:45 +0100

> leaf. It also does not appear to be suited to referencing a data node

In my view, the leafref value (such as a list key) can be passed in in a GET response and then it should be easy for the client-side application to construct the corresponding URI, XPath or whatever, because it has access to the data model. I know this goes against REST catechism but I am inclined to consider it a feature, not a bug.

Lada

---

From: Wojciech Dec <[email-masked]>
Date: Wed, 4 Dec 2013 15:57:35 +0100

> In my view, the leafref value (such as a list key) can be passed in in a GET response and then it should be easy for the client-side application to construct the corresponding URI, XPath or whateve…

It would be better to call it a missing feature.
Leafref is fine as is, and does pass the value. The recipient though
has no idea how to access the resource of that value unless the whole
(sub) tree is conveyed.
Using the example from 6020:

  leaf mgmt-interface {
         type leafref {
             path "../interface/name";
         }
     }

   An example of a corresponding XML snippet:

     <interface>
       <name>eth0</name>
     </interface>
     <interface>
       <name>lo</name>
     </interface>

     <mgmt-interface>eth0</mgmt-interface>

A client receiving eth0, has no idea about the URI of that resource.
Assuming that the client is coded to the data model, or is fully Yang
aware, and able to combine the "path ../interface/name" to make sense
of it all, is not just going against REST principles; it forces a
tight client-server coupling, even when one would not be required. I
expect a good number of client applications wishing to access he data
via Restconf, without necessarily having interest in the full
data-models, or even in altering configurations (eg reading a
topology).
As I wrote previously, having the option of Yang aware clients vs pure
REST would be a much stronger proposition.
What can we do to get the missing feature?

I would still see the "instance-identifier" either molded into a URI,
or a new "instance-identifier" like data-type that allow that.

Thoughts?

-Wojciech.

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 04 Dec 2013 16:12:50 +0100 (CET)

Wojciech Dec <[email-masked]> wrote:
> > In my view, the leafref value (such as a list key) can be passed in in
> > a GET response and then it should be easy for the client-side
> > application to construct the corresponding URI, XPath or whatever,
> > because it has access to the data model. I know this goes against REST
> > catechism but I am inclined to consider it a feature, not a bug.
> 

leafrefs are used when the "target" of the reference is defined in the
data model

instance-identifiers are used when the "target" of the reference can
be any node in the data store.

They are encoded in (NETCONF) XML as defined in 6020.

They *could* be encoded differently for RESTCONF.  But note that when
you write a value to a leafref, it is not very elegant (or useful
maybe) to write a complete URL.

I strongly oppose encoding-specific references - how would a data
model designer know which one to use?  restconf-leafref or leafref or
xxx-leafref?

/martin

---

From: Andy Bierman <[email-masked]>
Date: Wed, 4 Dec 2013 07:23:20 -0800

On Wed, Dec 4, 2013 at 6:57 AM, Wojciech Dec <[email-masked]> wrote:

> >>>>>>>>      "play": [ "
> http://example.com/operations/example-jukebox:play"]
> >>>>>>>>  }
> >>>>>>>> }
> >>>>>>>>
> >>>>>>>
> >>>>> Predicates in instance-identifiers can be used only for matching
> list keys against constant strings, see sec. 9.13 in RFC 6020.
> >>>>>
> >>> Assuming that line-cards also appear somewhere in the data tree, a
> leafref would be a more natural way of representing the reference - and
> then you can use current(), too.
> >>>
> >>> I have myself never used an instance-identifier in any data model yet,
> presumably they are mainly useful in notifications.
> >>
> > In my view, the leafref value (such as a list key) can be passed in in a
> GET response and then it should be easy for the client-side application to
> construct the corresponding URI, XPath or whatever, because it has access
> to the data model. I know this goes against REST catechism but I am
> inclined to consider it a feature, not a bug.
>

The client has the YANG model, otherwise it would not be
doing anything but attempt to render this leaf, since it doesn't
know the semantics or even the derived type (i-i or leafref).

If the client knows the YANG, then it knows the general
path expression to the referenced leaf.  These leafs are not
required to be unique (e.g., point at if-type leaf).  The leafref
object must be set to 1 of the values that exist on the device.
Your example points at a key leaf, but a keyref is just 1 form
of a leafref.

You seem to be assuming that the path-stmt nodeset must evaluate
to a single instance, and the client must be able to retrieve this instance.
However YANG leafref is not defined that way.  The value is not tied
to a specific referenced leaf instance, so the client cannot retrieve the
instance-identifier of that leaf (or all matching leafs).

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 04 Dec 2013 16:38:58 +0100 (CET)

> instance-identifier of that leaf (or all matching leafs).

Good point!

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 4 Dec 2013 16:43:15 +0100

> What can we do to get the missing feature?

RESTCONF is a protocol only by virtue of YANG data models that serve as a contract between the server and client. Other UIs, perhaps more RESTful or more suitable for human users, are certainly possible, but this is outside the scope of the WG - UI is a taboo word in the IETF in general.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Wed, 4 Dec 2013 07:57:59 -0800

On Wed, Dec 4, 2013 at 7:12 AM, Martin Bjorklund <[email-masked]> wrote:

> ....

This is actually a real problem in the YANG Patch draft.
There are some data-resource-identifier leafs, such as error-path,
that are imported from ietf-restconf.yang, but NETCONF-EX
needs to use the NETCONF error-path syntax (XPath) not a URI.

The refine-stmt does not allow a leaf type-stmt to be altered
so the grouping is unusable.  This is preventing YANG Patch
from being protocol independent.

There is a deterministic 1:1 mapping between data-resource-identifier
and instance-identifier for data nodes (by design).  Should a RESTCONF
client
or NETCONF client need to know about transforming XPath to/from URI?

I agree that it is messy to force the server to treat the same object
differently,
and render it according to the protocol used for the request.  This violates
every design principle for unified APIs.

   typedef both-identifiers {
      type union {
          type instance-identifier;
          type rc:data-resource-identifier;
      }
   }

A YANG model could be written to work with both data types.
This does not solve the problem of converting XPath to/from URI
depending on the protocol used.

We have to make sure YANG is protocol independent.
Any ideas on how to do that?

Andy

---

From: Andy Bierman <[email-masked]>
Date: Fri, 6 Dec 2013 11:43:02 -0800

Hi,

You have raised an important RESTCONF issue, which is the
client processing of server resource URIs.  I think the
client should be able to use the Location header URI for
all CRUD access to the data resource.

That's why the /restconf/config and /restconf/operational split
is broken. In order to retrieve operational data, the client
has to parse the URI and change it -- not too REST-ful ;-)

I think there should be one YANG data root: /restconf/data.
For RESTCONF, the ETag and Last-Modified headers are
only affected by configuration data resources.  The client
will have to add a query parameter to retrieve config=false
data, so it can also be aware that ETag and Last-Modified
are not affected by config=false data nodes.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 06 Dec 2013 22:27:20 +0100 (CET)

> has to parse the URI and change it -- not too REST-ful ;-)

RESTCONF's uri scheme is already not too REST-ful.  The uri's are
deterministic and known by both client and server.  The Location
header only works when the client has created a resource, but it
cannot be used to discover uris on the server.

The split into config and operational does not change this.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Fri, 6 Dec 2013 17:54:19 -0800

> The split into config and operational does not change this.
OK, but the split was supposed to get rid of the config query param
but it didn't.  Now config=true|false applies only to the /restconf/data
resource, not the /restconf/config resource.

I think we are better off without the /restconf/config resource and
just have /restconf/data + content=enum (config|operational|both)
[default:config]
The ETag and Last-Modified values are only returned when content=config.
They are only maintained for config=true data nodes.

Andy