Subject: [core] OCF Observe on special interfaces, design question, RFC requirements?

From: Michael Koster <[email-masked]>
Date: Tue, 9 Jul 2019 13:38:43 -0700

Background:
OCF uses special URLs, selected by query parameters, to specialize views and functions on resources. Client applications use the CoRE defined "if" target attribute and use IANA registered values as OCF Core  Interface Types in the query to select these views.

We provide interfaces that report partial state representations, e.g. links list for just the links of a collection, batch for just the data items in a collection, etc., and one interface (called baseline) that does report the full state of the resource.

For example:
GET /machine/state/3?if=oic.if.s
selects the "sensor" interface which returns sensor values but maybe not things like units and range that may also be present in the resource.

The need:
In OCF we are in the process of adding more special URLs, as interface types, for creation and deletion of resources, and interface types for modifying link arrays.

We would like to use CoAP Observe on these new interface URLs to report only the updates to the resource, and not the entire state of the resource in each notification. You might say partial state from a temporal view; incremental state updates. 

The Problem:
When a client first Observes or tries to poll, for example using the create interface, there are no items to reply with (this issue is similar to the Pub/Sub empty topic issue). Later, when a resource is eventually created, then a notification is sent to all observers containing the representation that was used to create the resource.

Proposed Solution:
With OCF we have a default content format and can return an empty CBOR map or an empty CBOR array, as appropriate, for the initial GET + Observe response, and return an item in the payload on subsequent notifications whenever a new resource is created.

We would use 2.05 Content for the response code in all non-error notifications.

Question:
Does this violate any of the current requirements in RFC7252 or RFC7641, or other specifications? There is some text that suggests a full representation be transmitted on every response, but it seems to address the general case i.e. not special URLs, and also would not allow for over-the-wire protocol optimization.

Example:
client-1 ==> GET /some/collection?if=oic.if.create obs=0 // client-1 wants to be notified when items are created in the collection
server response ==> 2.05 Content obs {} 

(later)

client-2 ==> POST /some/collection?if=oic.if.create { item: foo } //item being created by client-2 in the collection
server response ==> 2.01 Created { item:foo, id:1234 }

(to client-1)
observe response ==> 2.05 Content { item:foo, id:1234 } // client-1 is notified of the new item

---

From: Klaus Hartke <[email-masked]>
Date: Wed, 10 Jul 2019 00:24:18 +0200

Michael Koster wrote:
> Does this violate any of the current requirements in RFC7252 or RFC7641, or other specifications?

I think RFC 7641 is pretty clear that a 2.05 (Content) notification
contains a resource state representation and not a representation of
changes since the last notification.

RFC 7641: "The goal of the protocol is to keep the resource state
observed by the client as closely in sync with the actual state at the
server as possible."

"Notifications are additional responses sent by the server in reply to
the single extended GET request that created the registration. [...]
The only difference between a notification and a normal   response is
the presence of the Observe Option."

RFC 7252: "The payload returned with the [2.05 (Content)] response is
a representation of the target resource."

RFC 7641: "As notifications are just additional responses sent by the
server in reply to a GET request, they are subject to caching as
defined in Section 5.6 of RFC 7252 [RFC7252]."

A good way to think of the Observe mechanism is to see it an
optimization of polling: Every time a client gets the response in
reply to a GET request without Observe option, it gets the current
representation if the resource. That representation only depends on
the resource state, never on the response to a previous request. Every
time a client gets a notification in reply to a GET request with
Observe option, it gets the current representation if the resource.
That representation only depends on the resource state, never on a
previous notification.

When observing a resource with a rather large representation, it might
often make more sense to observe a projection of that resource. That
is, to observe a resource X whose state depends on the state of
another resource Y.

Examples of possible resource state projections if the state of Y is a number:

* The state of X is that number rounded to a multiple of 5.
* The state of X is that number clipped to the range 0--100.
* The state of X is the moving average of that number over the last 60 seconds.
* The state of X is that number sampled every other second.
* The state of X is the difference between that number and 9000.

Examples of possible resource state projections if the state of Y is a
list of items:

* The state of X is the number of changes to that list since its creation.
* The state of X is the timestamp of the last change to that list.
* The state of X is a list of the last 10 additions/removals to that list.
* The state of X is a list of the additions/removals to that list
since 2019-07-10T00:21:00+0200.

Anything where the notifications depend on previous notifications
(e.g., time elapsed since the last notification, value has changed by
more than some value since the last notification, additions/removals
to some list since the last notification, etc.) is not a projection of
resource state and thus not pollable nor observable.

Klaus

---

From: Michael Koster <[email-masked]>
Date: Tue, 9 Jul 2019 16:34:31 -0700

Thank you, Klaus.

> On Jul 9, 2019, at 3:24 PM, Klaus Hartke <[email-masked]> wrote:
> 

This is already the intention with providing the alternate URI through the use of the query parameter. It is a different resource which is a projection of the state of the base resource.

klaus> * The state of X is a list of the last 10 additions/removals to that list.

Or perhaps the last one addition or removal...

What we are doing is not that different.

Best regards,

Michael

---

From: Michael Koster <[email-masked]>
Date: Tue, 9 Jul 2019 17:08:03 -0700

Hi Klaus,

What if we retained the last notified state such that it would also be returned by polling?

> On Jul 9, 2019, at 4:34 PM, Michael Koster <[email-masked]> wrote:
> 
> Thank you, Klaus.
> 
>> On Jul 9, 2019, at 3:24 PM, Klaus Hartke <[email-masked] <mailto:[email-masked]>> wrote:
>>

---

From: Michael Koster <[email-masked]>
Date: Tue, 9 Jul 2019 17:11:58 -0700

More formally, the observed (projected) resource, referenced through an alternate URL, would be a log of recent activity on the base resource, with a maximum depth of one. 

Is it a satisfactory definition?

> On Jul 9, 2019, at 5:08 PM, Michael Koster <[email-masked]> wrote:
> 
> Hi Klaus,
> 
>> On Jul 9, 2019, at 4:34 PM, Michael Koster <[email-masked] <mailto:[email-masked]>> wrote:
>> 
>> Thank you, Klaus.
>>

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 10 Jul 2019 07:52:08 +0200

There is, of course, nothing wrong with observing a resource that is the “last change” of another resource.  This just doesn’t give you a reliable sequence of all changes; that requires something like the “Series Transfer Pattern”, https://tools.ietf.org/html/draft-bormann-t2trg-stp

Grüße, Carsten

---

From: Michael Koster <[email-masked]>
Date: Tue, 9 Jul 2019 23:10:47 -0700

Hi Carsten,

Yes, as is, this is not guaranteed to deliver all of the updates. 

Best regards,

Michael

PS We were thinking along similar lines as STP, with the idea of subscriber queues in Pub/Sub, where each subscriber would have its own queue and cursor. Of course, one could combine the storage for elements that are common across more than one subscriber, and recover storage for items that have been consumed by all subscribers.

> On Jul 9, 2019, at 10:52 PM, Carsten Bormann <[email-masked]> wrote:
> 
> _______________________________________________
> T2TRG mailing list
> [email-masked]
> https://www.irtf.org/mailman/listinfo/t2trg