Subject: [core] Subscription and Tokens/EventId

From: Brian Frank <[email-masked]>
Date: Thu, 11 Nov 2010 07:25:26 -0500

I really just started digging into this and catching up on the emails.

First off, let me ask if I understand this correctly.  Based on the
observation I-D it sounds like the client can include a Token with its
subscribe.  Token size is being debated somewhere b/w 2-16 bytes.  However
if I read it correctly, the server is free to choose to send notifications
either with the URI *or* the Token (which maybe has been renamed EventId)?

If that is the proposed design, then I would suggest ditching the idea of
Tokens all together for matching up subscriptions with notifications.  In my
opinion requiring a server to store a token per subscription is way too much
overhead, even if the token is just a few bytes.

Here is how I am approaching the problem:  there are two required components
to a subscription:
  1. the client/subscriber's IP address and port
  2. an indication of which resource or resources are subscribed by a given
client

An efficient implementation is probably going to manage each unique client
with some table like this:

  - client 1: ip+port
  - client 2: ip+port

Side note I haven't seen mentioned: if a client fails to confirm a
notification message (has gone down), then likely the result should be
server should cancel all that clients subscriptions.

Then we are going to somehow mark which resources are subscribed by each
client:

  - resource 1: client 1
  - resource 2: client 1 + client 2
  - resource 3: no subscriptions

In Sedona we did this like this:
  a) assign each unique client an index in a fixed size table
  b) keep track of subscriptions using a bit in a bitmask on each resource

That model has worked quite well.  If you decide you want to support
16 simultaneous clients, then you add  2 byte overhead to each resource
object.  For a 100 resources, we add an extra 200 bytes and can support 1600
subscriptions in fixed sized tables (ignoring client ip+port table).  As
soon as we start having to keep track of something other than a boolean
state per client per resource things get ugly and complicated.

To me, storing 2 bytes per subscription seems like a big deal, let alone 4,
8, or even 16.  In Sedona's current design we can support 16 clients x 100
resources in 200 bytes (ignoring client ip+port table).  If the server is
forced to store 8 bytes for a token per subscription plus lets say at least
2 bytes to reference the client table and 2 bytes to reference the resource,
then that 200 bytes buys us a measly 16 subscriptions before we run out of
RAM in our tables.

So if I am reading the proposed spec right, and Token is optional I
personally would always choose to throw the Token away and just send my
notifications with URIs.  And I suspect that this is the design most limited
devices would choose since RAM tends to be the most limited resource.

Quite possible I am missing some fundamental in the proposed design, but
those our my thoughts on how I read thru the I-Ds and comments out there.

Brian

---

From: Klaus Hartke <[email-masked]>
Date: Thu, 11 Nov 2010 14:27:16 +0100

Brian Frank wrote:
> subscribe.  Token size is being debated somewhere b/w 2-16 bytes.

Right.

> However if I read it correctly, the server is free to choose to
> send notifications either with the URI *or* the Token

The current intent is that, if the client specifies a Token, the
server MUST include the Token in every notification; if the client
doesn't specify a Token, the server MUST include the URI.

> overhead, even if the token is just a few bytes.

What about

(a)
   - client 1a: 16(ip)+2(port)+16(token) = 34 bytes
   - client 1b: 16(ip)+2(port)+16(token) = 34 bytes
   - client 2a: 16(ip)+2(port)+16(token) = 34 bytes

   - resource 1: client 1a
   - resource 2: client 1b + client 2a
   - resource 3: no subscriptions

or

(b)
   - client 1: 16(ip)+2(port) = 18 bytes
   - client 2: 16(ip)+2(port) = 18 bytes

   - subscription 1: 2(pointer to client 1)+16(token) = 18 bytes
   - subscription 2: 2(pointer to client 1)+16(token) = 18 bytes
   - subscription 3: 2(pointer to client 2)+16(token) = 18 bytes

   - resource 1: subscription 1
   - resource 2: subscription 2 + subscription 3
   - resource 3: no subscriptions

That indeed requires more bytes than just

(c)
   - client 1: 16(ip)+2(port) = 18 bytes
   - client 2: 16(ip)+2(port) = 18 bytes

   - resource 1: client 1

but I don't think that brings the number of possible subscription down
from 1600 to 16...

Assuming 3 resources, 2 bytes per resource and (200-3*2=)194 bytes of
available space, then - in the worst case - the number of possible
subscriptions are

(a) 5.71 subscriptions
(b) 5.39 subscriptions
(c) 10.78 subscriptions

That looks like a factor of 2 to me.

In the average case, when there are multiple subscriptions from the
same IP+port, (b) should perform better than (a).

In best case, when all subscriptions are from the same IP+port, (b)
allows up to 9.78 subscriptions, which is already close to (c).

Just for fun:

(d)
   - prefix 1: 8 bytes

   - client 1: 2(pointer to prefix 1)+8(remaining ip address)+2(port) = 12 bytes
   - client 2: 2(pointer to prefix 1)+8(remaining ip address)+2(port) = 12 bytes

   - subscription 1: 2(pointer to client 1)+16(token) = 18 bytes
   - subscription 2: 2(pointer to client 1)+16(token) = 18 bytes
   - subscription 3: 2(pointer to client 2)+16(token) = 18 bytes

   - resource 1: subscription 1
   - resource 2: subscription 2 + subscription 3
   - resource 3: no subscriptions

Worst case: 5.11 subscriptions
Best case: 9.67 subscriptions

Klaus

---

From: Klaus Hartke <[email-masked]>
Date: Thu, 11 Nov 2010 15:33:02 +0100

Almost forgot: You also need to store the remaining lifetime
somewhere, which is per-subscription, not per-client. So (c) probably
doesn't even work (unless we remove the subscription lifetime
mechanism)...

Klaus

---

From: Zach Shelby <[email-masked]>
Date: Thu, 11 Nov 2010 23:10:29 +0800

> doesn't specify a Token, the server MUST include the URI.

And the ability to leave the token (and thus get the URI in each notification) out is a feature I really want to keep, sounds like Brian has the same in mind. The Token does have some advantages, especially in saving overhead for possibly long URIs (e.g. with a query component). The current philosophy is correct, with it is up to the client which should be used.

Zach

-- 
Zach Shelby, Chief Nerd, Sensinode Ltd.
http://zachshelby.org  - My blog "On the Internet of Things"
http://6lowpan.net - My book "6LoWPAN: The Wireless Embedded Internet"
Mobile: +358 40 7796297

---

From: Peter Bigot <[email-masked]>
Date: Thu, 11 Nov 2010 09:28:44 -0600

On Thu, Nov 11, 2010 at 9:10 AM, Zach Shelby <[email-masked]> wrote:

> And the ability to leave the token (and thus get the URI in each
> notification) out is a feature I really want to keep, sounds like Brian has
> the same in mind.

The rationale for this is that it enables client-independent response
messages, or clients that don't remember they subscribed?

> The Token does have some advantages, especially in saving overhead for
> possibly long URIs (e.g. with a query component). The current philosophy is
> correct, with it is up to the client which should be used.
>

I rather dislike CoAP having two mechanisms (compare Token; compare set of
URI options) to correlate asynchronous responses (one-time or observations)
with a request, especially since nobody's specified the rules to be used for
the URI-based approach.  I'll be interested to see how Uri-Authority is
handled in those rules.

Also not convinced requiring a server to support two client styles is in the
spirit of constrained devices.

Peter

---

From: Brian Frank <[email-masked]>
Date: Mon, 15 Nov 2010 09:33:48 -0500

I agree - I think we should just pick the one most limited devices will
probably support (which IMO is URI correlation).

Also I would really like to figure out how remove subscription lifetime as a
per subscription option.  As Klaus pointed out, that adds a pretty hefty
overhead per subscription.  Can we make lifetime a per client issue versus a
per subscription issue?

I think we could even simplify it even further to just say that as long as
the client is ack'ing the notifications we can assume it is still interested
in the subscription.  That requires no memory overhead, and solves the
primary issue which is how to release resources when communications to a
client is lost.

On Thu, Nov 11, 2010 at 10:28 AM, Peter Bigot <[email-masked]> wrote:

> Also not convinced requiring a server to support two client styles is in
> the spirit of constrained devices.
>
> Peter
>
> _______________________________________________
> core mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/core
>
>

---

From: Klaus Hartke <[email-masked]>
Date: Tue, 16 Nov 2010 17:58:57 +0100

> probably support (which IMO is URI correlation).

URI correlation doesn't suffice if a client has more than one
outstanding request to the same resource. A client implementation can
opt-out from having more than one outstanding request, but a server
implementation cannot.

> overhead per subscription.  Can we make lifetime a per client issue versus a

The Etag(s) known by a client are also per-subscription. A server may
opt-out from using Etags though.

> in the subscription. [...]

There are two issues that need to be addressed:

- a client needs to know if the server is still sending notifications, and
- a server needs to know if the client is still interested in
receiving notifications.

The current (Subscription-lifetime) mechanism addresses both: a server
knows that the client is still interested because the client
repeatedly sends subscription requests, and the client knows that the
server is sending notifications, because the server repeatedly
acknowledges subscription requests. If one of the two fails or they
disagree about the state of the subscription, the next subscription
request will fix it.

I like the idea of using ack'ing the notifications to signal that the
client is still interested in receiving notifications. To make sure
the server is still sending notifications, the client could refresh
the subscription after it didn't receive a notification for some time
(which it probably should be doing anyway).

I'm slightly concerned about 'lazy' implementations that simply ack
every response instead of checking if the response can be related to
an outstanding request. I heard people tend to send newsletters to
their spam folder rather than clicking the 'unsubscribe' link when
they want to get rid of them.

Klaus

---

From: [email-masked]
Date: Wed, 17 Nov 2010 12:12:40 +0100

>opt-out from having more than one outstanding request, but a server

What do you mean by the same resource ? The same URI-Path and URI-Query on
different servers ?

>(which it probably should be doing anyway).

Then "subscription lifetime" would be replaced by "maximum allowed time
between 2 notifications" ? This is an interesting idea because the client
will have to repeat the subscription request (with a potentially long URI)
only in case of failure or very infrequent events. Most of the time both
the server and the client know they can get in touch. And they reset their
time counter to the maximum allowed time on each notification. However in
case of very frequent notifications acking all notifications may not be the
optimal solution. The server should be able to send non confirmable
message. For example if the time since the last notification is short
compared to the maximum allowed time the server could decide to send the
next notification as a non confirmable message. Sadly this mechanism
requires more state because the server must store the maximum allowed time
plus the time counter. Maybe the state could be on a per client basis? If
we don't need a mechanism which is tolerant to transient failures the
server could also just release all observations from a client if a
notification is not acked as suggested by Brian. But if it is supposed to
be deployed in lossy networks I'm not sure this is the right solution.
Does that make sense ?

Matthieu

---

From: Klaus Hartke <[email-masked]>
Date: Wed, 17 Nov 2010 15:10:11 +0100

> What do you mean by the same resource ? The same URI-Path and URI-Query on

Example:

    C: GET coap://example.org:61616/resource
    C: POST coap://example.org:61616/resource "foo"
    S: 200 coap://example.org:61616/resource
    S: 405 coap://example.org:61616/resource

Which response belongs to which request?

Solution 1: "Then don't do it"

    C: GET coap://example.org:61616/resource
    S: 405 coap://example.org:61616/resource

    C: POST coap://example.org:61616/resource "foo"
    S: 200 coap://example.org:61616/resource

Solution 2: Tokens

    C: GET coap://example.org:61616/resource (token=42)
    C: POST coap://example.org:61616/resource (token=99) "foo"
    S: 200 (token=99)
    S: 405 (token=42)

> between 2 notifications" ?

Yes, the client could inform the server of that duration after which
it subscribes again. I could imagine it's just an implementation
detail how the server uses this information. The server could:

- send a confirmable notification with the current state just before
the time is up;
- end the subscription after it didn't send a notification for that
duration, so it doesn't have to keep the subscription around until the
next notification;
- simply ignore it and keep the subscription going as long as the
client is ack'ing notifications;
- simply ignore it and regularly kill all subscriptions.
- ...

> However in case of
> very frequent notifications acking all notifications may not be the optimal
> solution. The server should be able to send non confirmable message.

Right, the server should implement some strategy when to send
notifications confirmable or non-confirmable (ticket #67). In the best
case (which is very likely for very constrained nodes) the server
implementation knows about the application: For example, if the
resource represents a temperature sensor that changes its state every
second, the server implementation could be hard-coded to send the
notification for every n'th state change confirmable and all other
non-confirmable. This example would require only per-resource state.

Klaus

---

From: [email-masked]
Date: Wed, 17 Nov 2010 16:36:02 +0100

>Which response belongs to which request?

According to coap-3 the responses can't be asynchronous because the client
requests didn't include a Token option. So I would answer look at the
transaction ID.

C: CON TID 1 GET coap://example.org:61616/resource
C: CON TID 2 POST coap://example.org:61616/resource "foo"
S: ACK TID 1 200 coap://example.org:61616/resource
S: ACK TID 2 405 coap://example.org:61616/resource

But as I understand coap-observe all notifications (except the first one
transmitted with the ACK) are asynchronous responses so according to coap-3
a token is required.

Matthieu

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 17 Nov 2010 19:05:16 +0100

> Solution 2: Tokens

Solution 2a: require tokens for asynchronous responses to POST/PUT/DELETE, because the response is about the request, not about the resource.
(An asynchronous response to a GET is self-describing.)

Weird but elucidating example:

...
...

(Yes, that asynchronous response answers both requests!)

Gruesse, Carsten

---

From: [email-masked]
Date: Thu, 18 Nov 2010 11:10:11 +0100

>Solution 2a: require tokens for asynchronous responses to POST/PUT/DELETE,
because the response is about the request, not about the resource.

>(Yes, that asynchronous response answers both requests!)

I'm not sure a GET is completely self-describing because a client may want
ordered responses. Is it possible to have the following sequence?

   C: GET coap://example.org:61616/resource
   S: 200 coap://example.org:61616/resource "foo" [not received because of
network delay]
   S: 200 coap://example.org:61616/resource "foo" [retry OK]
   C: PUT coap://example.org:61616/resource "bar" (token=10)
   S: 200 (token=10)
   C: GET coap://example.org:61616/resource
      200 coap://example.org:61616/resource "foo" [previously delayed
response]
Here the client detects inconsistency and starts a failure procedure
   S: 200 coap://example.org:61616/resource "bar"

With Token option the same sequence should look like this:

   C: GET coap://example.org:61616/resource (token=5)
   S: 200 (token=5) "foo" [not received because of network delay]
   S: 200 (token=5) "foo" [retry OK]
   C: PUT coap://example.org:61616/resource "bar" (token=10)
   S: 200 (token=10)
   C: GET coap://example.org:61616/resource (token=15)
      200 (token=5) "foo" [previously delayed response]
Here the client has released the request with token=5 so it sends a RESET
but it continues with the current procedure
   C: RST
   S: 200 (token=15) "bar"

And do we really want to support different correlation methods? I would say
no just like Peter.

Matthieu