Subject: Re: [core] REST, URIs, and constrained devices

From: Peter Bigot <[email-masked]>
Date: Wed, 29 Sep 2010 12:14:00 -0500

To raise the issue again: my concern is how to think about URIs where the
authority resolves to an IP multicast address, while retaining the core REST
concepts.  A couple months ago, I wrote this:

On Mon, Jul 26, 2010 at 10:19 AM, Peter Bigot <[email-masked]> wrote:

> 2) Groups of resources on distributed devices
>
> A second key feature of REST is that it describes a client-server model
> where addressable resources are transferred through a representation of
> their state.
>
> If CoRE wants to extend the concept of URI to denote a distributed resource
> which is an aggregation of independently managed state, it must define the
> processes by which (a) the state representation in a request is distributed,
> and (b) the state representations gathered from responses are combined to
> return to the REST client a meaningful representation of the state of the
> distributed resource.
>
> In particular, this is necessary for any access to constrained devices
> through an HTTP gateway, as implied by the "HTTP REST based API" requirement
> in the CoRE charter.  An HTTP client will not accept multiple responses to a
> single request.
>
> Peter
>

Subsequently, to explain the matter to co-workers, I wrote the following
extended example of how I would expect to use CoAP in a REST architecture
for home lighting management.  There are certainly restrictive assumptions
that could be made to make this work, like a client performing an analysis
on the authority to determine whether it's a unicast, multicast, (anycast?)
address.  In my opinion, imposing these restrictive assumptions make it
harder for CoAP to be inserted into a complex REST systems (with proxies,
intermediate servers, non-CoAP gateways, etc.) and still operate in the way
most people will expect.  It's this blind interoperability that makes REST
so powerful.  I really believe multicast has no place in a REST system, but
would like to hear details on how the two can be reconciled.

Peter

REST is an architectural style designed to access and manage distributed
hypermedia documents.  In the form of HTTP and CoAP, it uses a fixed set of
operations on resources.  A resource is a consistent mapping from a name
(its URI) to a view of server-side state (the resource representation).  The
operations of HTTP and CoAP include GET to obtain the state of a resource,
PUT to idempotently replace a resource state, POST for a non-idempotent
creation or change to the resource (may imply a side-effect), DELETE to
remove the resource entirely.  REST is actually much more than this (and is
not CRUD), but that is the relevant part for this discussion.

Assume I have a device, like the Insteon LampLink or InlineLinc, that
controls a 120V load.  This device has an IP network interface, and I want
to use CoAP to monitor and control it.  To simplify things, assume the
device presents a single resource, named "state".  This resource indicates
whether the device passes ("on") or blocks ("off") flow of electricity to
its load, and is represented by the ASCII text "on" or "off".  Assume that I
am permitted to update the state, and when doing so can cause the device to
change whether electricity flows to its load.

Let's say I install one of these devices in my living room, and connect it
to a lamp on the western end-table.  In the DHCP server for my home network,
I assign the name "westlr" and an IP address to the device.

Now assume I'm away from home, and forgot to turn off the lights before I
left.  My Android phone has a VPN connection that allows it to connect
directly to any host on my internal network, and I have a CoAP browser on my
phone (I wish....).  The browser allows me to specify a URI, has a box into
which I can type a resource representation to be sent as a message body,
buttons for the four REST operations, and a window in which the returned
state description is displayed.

If I I enter "coap://westlr/state" in the URI box and press GET, the result
window will contain the string "on" (without quotes), because I forgot to
turn off the lights.   If I type "off" into the message body window, then
press PUT, the lamp should turn off.  Pressing "GET" again should put the
string "off" in the result window.

Now let's say I put a similar device on the lamp on the other side of the
sofa, and registered it as "eastlr".  I can individually query and command
that lamp through the uri "coap://eastlr/state".

Now: Building automation folks naturally see this and say "But I want to
turn off all the lights simultaneously".  They think, "Hey, CoAP is based on
UDP, which supports multicast.  Why don't I select an IP multicast group
address, put it into DNS as "lamps", and have westlr and eastlr join that
group.  Then I can PUT "off" to "coap://lamps/state" and turn everything off
at once."

Granted that could work for PUT, probably.  But leave the PUT aside.  In
REST, the syntactic act of requesting a method on a resource is always OK;
whether that method has any meaning when applied to a resource is determined
by the server that processes the REST transaction.  I put
"coap://lamps/state" into my CoAP browser and press GET.  What shows up in
my result window?

Under the covers, my droid sends a CoAP GET message to the IP address that
"lamps" resolves to.  It has no idea that this is a multicast address (it
could be forced to examine it, but normally it wouldn't).  Fine, assume it
does, and it knows it could get responses from zero or more devices,
depending on whether the request packet reaches the west lamp, east lamp,
both, or neither.  Each of those will be a document containing "on" or
"off".  What's it supposed to display?  I suppose I'd want to see something
like "westlr: on; eastlr: off".  That's not something the browser can figure
out on its own.

Maybe eastlr is busy doing daily backups, and takes a few extra seconds to
reply.  How does the browser know when it's received "all" the responses?

Really, this describes the fundamental problem.  Putting "off" in the
message body window and pressing POST is the easy part.  Defining what
happens with all the other ways you can legally use CoAP or HTTP with this
URI "scheme://lamps/state" that happens to have an authority part that
resolves to a multicast group---that's the hard part.  Recall that neither
the client, nor any intermediate servers that happen to observe the
transaction as it passes between the client and the origin server(s) have
any knowledge about the operation except what can be determined from the
REST method, the URI, and the transaction message contents.  None of that
explains how to aggregate multiple representations into a single resource
state.  There *is* no single resource state, because the URI does not
identify  a resource.

One way to solve the building automation problem is to say that
"coap://lamps/state" is a distinct resource the value of which is the
aggregate view of the states of all devices that are members of group
"lamps", from the perspective of some authoritative view provided by a
single server.  That server can, under the covers, perform its updates in
whatever way it needs to: iterating through a list sending CoAP messages to
each lamp, or multicasting a non-REST "turn off" command.  What it can't do,
without opening a huge can of mess, is try to use REST over multicast to
manipulate a bunch of resources simultaneously.  That's not REST, at least
as I understand it.

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 29 Sep 2010 20:32:22 +0200

On Sep 29, 2010, at 19:14, Peter Bigot wrote:

> If CoRE wants to extend the concept of URI to denote a distributed resource which is an aggregation of independently managed state

I think that is a big IF.

I prefer to think about this as operating on a _set_ of resources.

Having a single identifier that identifies this set is relatively(*) easy.
As you note, defining the semantics of a PUT also is reasonably easy (assuming no errors); GET less so, because we suddenly need a way to represent the resulting table/dictionary/hash/mapping (mathematically, it's a function).

(*) The even more interesting part is defining who is part of the group (and thus, the domain of the above function).  Reducing the thought to IP multicast means that this group is a somewhat random subset of the set of nodes that have joined the multicast address; this assumes that the constrained network has a good way to manage the joins.  Taking out the randomness requires a reliable multicast (transport) protocol. RFC 5740 defines a good one; it just happens to be about half an order of magnitude more complex than TCP (which we are trying to avoid because it may be too complex for our constrained environments).
If you want to define another one, good luck: RFC 5740 was published in 2009 based on work that was started in the early to mid-1990s.

The other way to manage a group is through a proxy-like construction.  This then also can do any concept of aggregation that we care about.  Classical sensor network research has often attempted to integrate that proxy function with packet forwarding.

Gruesse, Carsten

---

From: Kerry Lynn <[email-masked]>
Date: Wed, 29 Sep 2010 18:35:47 -0400

Obviously this topic requires much thought and discussion, but I did want to
reiterate a few points that were made on today's call.  The plan is to gather
all the issues relating to Group Communications into a single working doc.

I agree with Carsten's point of view.  Conceptually, the result of a multicast
GET or PUT should be the same as if the client had unicast them serially
(that is, a set of {URI, representation} tuples).  Logically, this seems similar
to pub/sub.  Practically, there are major benefits to solving this problem:
- packet economy on constrained networks
- M2M resource discovery (solves the "chicken-and-egg" problem)
- apparent simultaneity of events (e.g. lighting applications)

For data links (or transports) that don't support multicast, a serial unicast
analog must be provided.  In either case it should be possible to
enumerate the members of a group.  For links that do support multicast,
I think there are a few implications:
- All multicast requests MUST be sent to the well-known port
- All multicast requests SHOULD operate on /.well-known/core URIs

One question is whether the application (or middleboxes) need to be
aware that a request is intended for a group.  Aside from the response
dithering already pointed out by Zach, here's where a separate scheme
(e.g. corem) as proposed by the old Microsoft I-D might be useful.

Lastly, in response to Carsten's comments about (non-)reliable multicast,
one strategy might be to send multiple requests, particularly for unconfirmed.
Of course then we'd need to avoid non-idempotent requests (POST).  In
some confirmed request scenarios (those requiring apparent simultaneity)
it may be preferable to repeat multicast requests even if it results in the
same amount of traffic as the equivalent reliable serial unicast.

Regards, -K-

On Wed, Sep 29, 2010 at 2:32 PM, Carsten Bormann <[email-masked]> wrote:
> _______________________________________________
> core mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/core
>

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 30 Sep 2010 01:08:53 +0200

Thanks, Kerry, for summarizing these important points.

One quick comment:

> one strategy might be to send multiple requests, particularly for unconfirmed.

This strategy is often called "saturation" in reliable multicast protocols, and it can indeed be used to increase the subset of the group membership from which a response actually arrives.  (To minimize redundant confirmations, a bloom filter can be sent with each repeated request).

> Of course then we'd need to avoid non-idempotent requests (POST).

We already have transaction IDs in CoAP, which solve this problem nicely.
The one thing that might need additional attention is the lifetime of a transaction; for unicast, this is pretty much limited by RESPONSE_TIMEOUT * (2 ^ MAX_RETRANSMIT - 1) (~ half a minute).  For multicast, it MAY be sensible  to extend that retention time, or we could decide that half a minute is enough.  In any case, the last paragraph of 4.2 may need to be extended.

Gruesse, Carsten