Subject: [core] Sketch of "Keep it simple" / "Never reified union types"

From: Christian Amsüss <[email-masked]>
Date: Fri, 8 Mar 2019 14:45:01 +0100

(interim follow-up)
Reply-To: 

Hello CoRE,

following up on the last interim, I've sketched up what I understood the
keep-it-simple solution to the pubsum problem of allowing empty topics
could look like.

Most of the text below deals with how proxies would handle it, the
actual client and server behaviors should be indeed simple and
straightforward.

Best regards
Christian

The Accept-Any-Of option
========================

A new option Accept-Any-Of is defined. It is critical (I don't fully see
why but follow Accept here), safe-to-foward and part of the cache key.
Its format is uint up to 2 long (indicating content types), it is Class
E in OSCORE, usable in requests only, and repeatable

(Repeatability is the only aspect in which it differs from Accept in
terms of option properties).

Its values indicate a list of acceptable representations in order of
decreasing preference. A server MUST answer with the first format it can
represent the requested state in, or 4.06 (Not Acceptable) if it could
answer successfully but the response would not match any of the option
values.

Proxy behavior
--------------

A proxy MAY serve a cached representation to a request with a different
sequence of Accept-Any-Of options, provided the second request has an
Accept value of the cached representation, or all the content formats
that precede the available content format in the second request's
Accept-Any-Of options also preceded the available representation in an
earlier (fresh) request's list.

When a request that carries Accept-Any-Of is answered 4.06 (or with any
but the first format requested by Accept-Any-Of), a proxy SHOULD [ we
can't have a MUST here w/o making it non-safe-to-forward, but I think
it's sufficient ] invalidate all known representations in any of the
requested formats (or the formats preceeding the returned one,
respectively).

Update to RFC7641
=================

The requirement that subsequent notifications carry the same
Content-Format option as the original response is lifted.

Impact
------

Changes to the returned media type can either happen when

* Accept-Any-Of was sent in the request -- in which case both server and
  client know the updated rules, or
* no Accept header was sent -- in which case the server whose
  representation changes to require a new content format has no clear
  way of indicating that under RFC7641 (ending with 4.06 Not Acceptable
  would be close but isn't the expected response to a repeated request);
  if the server changes the content format in a notification to an
  unaware client, the client would catch it as a bad response (probably
  similar to a response with a Content-Format not matching the sent
  Accept). The client might consider the observation over while the
  server does not, and will terminate the observation with a RST on the
  next notification (or close the connection in TCP).

Impact on proxies: A proxy that enforces the previous rule on
Content-Format staying constant would close observations (probably with
something like 5.02 Bad Gateway), and the client would need to
re-establish. No proxy implementations are known that implement that
behavior. [ But I know only one so this would definitely need to be
confirmed on the mailing list ].

[ I don't think that this has any actual interactions with the caching
model, as the caching considerations are independent of the content
format. ]

Usage in pubsub
===============

Topics that were created but do not have any publication are represented
in the application/nothing-here-to-see (or application/null? either TBD)
type.

The PUBLISH operation would probably not need to accept
nothing-here-to-see, at leat create-on-publish can't do that as there is
no type to set on the topic with such an initial publication.

Requests to the topic with an Accept option of the topic's type will
return 4.06 Not Acceptable both in observations (subscription) and plain
get requests (read). The subscribe (and possibly read) operation
therefore will not use the Accept header, but Accept-Any-Of with
nothing-here-to-see and the topic's designate type, in that sequence.
Thus, a proxy may serve a nothing-here-to-see as long as it is fresh (as
it always could), but it sees a notification carrying the actual type,
that evicts the nothing-here-to-see from its cache.  (Conversely, if a
topic goes to nothing-here-to-see, that will not clear out any
cached-but-still-fresh value from the cache. As long as publishers are
not allowed to explicitly send nothing-here-to-see, this will not
happen, because topics only enter nothing-here-to-see when their content
expires, at which time all its caches expire as well. The broker might
not even send the nothing-here-to-see in a notification [ ? ].)

-- 
This may seem a bit weird, but that's okay, because it is weird.
  -- perldata(1) about perl variables

---

From: Michael Koster <[email-masked]>
Date: Mon, 11 Mar 2019 08:23:23 -0700

Hi Christian (and CoRE),

In the spirit of handling "empty topic" case in the application layer, as we discussed, I propose to have the publisher be responsible to publish "nothing-here-to-see" as a topic's initial value if it has no initial value in the more meaningful content format. 

(Keeping in mind that, as defined, CoAP pub/sub only handles representations, and is not strictly a proxy for resource state, therefore should not be responsible for generating default representations in any particular content format, except the required link-format in the case of create-on-publish).

The pub-sub behavior would be to reply to READ/SUBSCRIBE only when something has been published. The topic creator is thus responsible to "promptly" publish nothing-here-to-see upon topic creation. Otherwise the broker simply acks any READ/SUBSCRIBE request and waits to send a response until something is published. 

The broker might be allowed to ack and queue subscribe (observe) requests that come in before something is published. 

When a topic's max-age expires, the broker returns 4.xx; there would be no separate data lifetime. Publish operations reset the topic lifetime and could change the topic's lifetime by including max-age in the PUBLISH request.

This way, the Accept-Any option and the nothing-here-to-see content formats may be defined in separate independent drafts, and the multipart format may be used with its potential accept-subformat options.

This is how I interpreted the result of the discussion on Thursday where we decided to keep the pub/sub draft simple. WIll this work?

Best regards,

MIchael

> On Mar 8, 2019, at 5:45 AM, Christian Amsüss <[email-masked]> wrote:
> 
> (interim follow-up)
> Reply-To: 
> 
> Hello CoRE,
> 
> _______________________________________________
> core mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/core

---

From: Jim Schaad <[email-masked]>
Date: Mon, 11 Mar 2019 08:51:46 -0700

Michael,

It is not clear to me below if a published data item is going to have an age value or not.  I would expect that they should but the minimum age would be 0.  At that point the age on the data item stays at 0, but the item is not removed from the subscription.

Jim

> -----Original Message-----
> From: core <[email-masked]> On Behalf Of Michael Koster
> Sent: Monday, March 11, 2019 8:23 AM
> To: Christian Amsüss <[email-masked]>
> Cc: core WG <[email-masked]>; Jaime Jiménez <[email-masked]>
> Subject: Re: [core] Sketch of "Keep it simple" / "Never reified union types"
> 
> Hi Christian (and CoRE),
> 
> In the spirit of handling "empty topic" case in the application layer, as we
> discussed, I propose to have the publisher be responsible to publish
> "nothing-here-to-see" as a topic's initial value if it has no initial value in the
> more meaningful content format.
> 
> (Keeping in mind that, as defined, CoAP pub/sub only handles
> representations, and is not strictly a proxy for resource state, therefore
> should not be responsible for generating default representations in any
> particular content format, except the required link-format in the case of
> create-on-publish).
> 
> The pub-sub behavior would be to reply to READ/SUBSCRIBE only when
> something has been published. The topic creator is thus responsible to
> "promptly" publish nothing-here-to-see upon topic creation. Otherwise the
> broker simply acks any READ/SUBSCRIBE request and waits to send a
> response until something is published.
> 
> The broker might be allowed to ack and queue subscribe (observe) requests
> that come in before something is published.
> 
> When a topic's max-age expires, the broker returns 4.xx; there would be no
> separate data lifetime. Publish operations reset the topic lifetime and could
> change the topic's lifetime by including max-age in the PUBLISH request.
> 
> This way, the Accept-Any option and the nothing-here-to-see content
> formats may be defined in separate independent drafts, and the multipart
> format may be used with its potential accept-subformat options.
> 
> This is how I interpreted the result of the discussion on Thursday where we
> decided to keep the pub/sub draft simple. WIll this work?
> 
> Best regards,
> 
> MIchael
> 
> > following up on the last interim, I've sketched up what I understood
> > the keep-it-simple solution to the pubsum problem of allowing empty
> > topics could look like.
> >
> > A new option Accept-Any-Of is defined. It is critical (I don't fully
> > see why but follow Accept here), safe-to-foward and part of the cache key.
> > Its format is uint up to 2 long (indicating content types), it is
> > Class E in OSCORE, usable in requests only, and repeatable
> >
> > decreasing preference. A server MUST answer with the first format it
> > can represent the requested state in, or 4.06 (Not Acceptable) if it
> > could answer successfully but the response would not match any of the
> > option values.
> >
> > Proxy behavior
> > --------------
> >
> > A proxy MAY serve a cached representation to a request with a
> > different sequence of Accept-Any-Of options, provided the second
> > request has an Accept value of the cached representation, or all the
> > content formats that precede the available content format in the
> > second request's Accept-Any-Of options also preceded the available
> > representation in an earlier (fresh) request's list.
> >
> > When a request that carries Accept-Any-Of is answered 4.06 (or with
> > any but the first format requested by Accept-Any-Of), a proxy SHOULD [
> > we can't have a MUST here w/o making it non-safe-to-forward, but I
> > think it's sufficient ] invalidate all known representations in any of
> > the requested formats (or the formats preceeding the returned one,
> > respectively).
> >
> >
> > Update to RFC7641
> > =================
> >
> > * Accept-Any-Of was sent in the request -- in which case both server
> > and  client know the updated rules, or
> > Content-Format staying constant would close observations (probably
> > with something like 5.02 Bad Gateway), and the client would need to
> > Topics that were created but do not have any publication are
> > represented in the application/nothing-here-to-see (or
> > application/null? either TBD) type.
> >
> > nothing-here-to-see, at leat create-on-publish can't do that as there
> > is no type to set on the topic with such an initial publication.
> >
> > return 4.06 Not Acceptable both in observations (subscription) and
> > plain get requests (read). The subscribe (and possibly read) operation
> > Thus, a proxy may serve a nothing-here-to-see as long as it is fresh
> > (as it always could), but it sees a notification carrying the actual
> > type, that evicts the nothing-here-to-see from its cache.
> > (Conversely, if a topic goes to nothing-here-to-see, that will not
> > clear out any cached-but-still-fresh value from the cache. As long as
> > publishers are not allowed to explicitly send nothing-here-to-see,
> > this will not happen, because topics only enter nothing-here-to-see
> > when their content expires, at which time all its caches expire as
> > well. The broker might not even send the nothing-here-to-see in a
> > notification [ ? ].)
> >
> >
> > --

---

From: Christian Amsüss <[email-masked]>
Date: Thu, 21 Mar 2019 08:47:10 +0100

Hello Michael and CoRE,

On Mon, Mar 11, 2019 at 08:23:23AM -0700, Michael Koster wrote:
> In the spirit of handling "empty topic" case in the application layer,
> as we discussed, I propose to have the publisher be responsible to
> publish "nothing-here-to-see" as a topic's initial value if it has no
> initial value in the more meaningful content format. 
> 
> representations, and is not strictly a proxy for resource state,
> therefore should not be responsible for generating default
> representations in any particular content format, except the required
> link-format in the case of create-on-publish).
> 
> "promptly" publish nothing-here-to-see upon topic creation. Otherwise
> the broker simply acks any READ/SUBSCRIBE request and waits to send a

That would mean that any faulty (or malicious) topic creator could force
the broker into (costly-to-maintain in terms of memory) long-poll
situations, I'd prefer not to do that. (An alternative would be that the
broker answers 5.03 "I'm not in a state to process this request, retry
later please" with some Max-Age).

My gut feeling is that the broker can be argued to be the authority on
the topic (at least formally it is) and would thus be justified to say
that "empty" is its current representation, but I'll need to read the
latest version of pubsub to argue that well (or be taught better).

> formats may be defined in separate independent drafts, and the
> multipart format may be used with its potential accept-subformat
> options.

I agree that Accept-Any can (and probably should) be separate.

Kind regards
Christian

-- 
To use raw power is to make yourself infinitely vulnerable to greater powers.
  -- Bene Gesserit axiom