Subject: [core] Observe updates, and call for proxy implementations (picking from 'Sketch of "Keep it simple"' thread)

From: Christian Amsüss <[email-masked]>
Date: Thu, 21 Mar 2019 09:47:08 +0100

Hello CoRE,

during IETF104, I'd like to discuss the possibility of small changes to
obsevation, which would mean an update to RFC7641.

Topics that have accumulated:

* Block1 and Observe.

  RFC8132 introduces the FETCH method which allows both observation and
  having a Block1'd request payload, but does not state those's
  interactions.

  AFAIR we've worked out how they'd work in practice on this list
  (roughly "as long as 2.31 is returned, the server won't accept the
  observation, but will on the last Block1"), but that should be written
  down at some point.

  That might be an opportunity to formally allow observation for all
  operations, provided I assume correctly that limiting Block1
  interaction was an important reason to not have POST+Observe.

  OSCORE worked around this by switching to FETCH for observations.

* Pinning of response content formats.

  The restriction that notifications need to carry the same
  Content-Format hinders the use of the Accept-Any variety of content
  negotiation in pubsub. (See proposal and impact ananlys^Wthoughts in
  quoted part below from the other thread).

  To me that restriction looks pretty arbitrary (though I see why it
  sounded good at the outset); I'm looking forward to discussion on its
  backgrounds.

* Pinning of all request options except ETag.

  This is a similar case as the pinning of response content formats.

  OSCORE worked its way around this by claiming that the inner options
  are the relevant ones, even though a proxy would have a hard time to
  know that. ("For OSCORE messages, this matching is to be done to the
  options in the decrypted message"). It was argued that a proxy can't
  really "be mad at" the client for doing it, for the client may simply
  have forgotten that the old observation was there, and is just
  requesting a new one with a most innocent look on its face.

My impression is that this can be a light update, especially if the
Block1/Observe part stays in there as "From the previous documents, it
(kind of implicitly but we better spell it out here) follows that".

That discussion would profit a lot from knowning which proxies are out
there and how they behave with respect to the open points; I'll start:

* aiocoap: The proxy implementation shipped with the library is lenient
  w/rt pinning. Block1+Observe is implemented as above (but not covered
  by the test suite yet).

Best regards
Christian

Proposal from [1]:
> Update to RFC7641
> =================
> 
> The requirement that subsequent notifications carry the same
> Content-Format option as the original response is lifted.
> 
> Impact
> ------
> 
> Changes to the returned media type can either happen when
> 
> * Accept-Any-Of was sent in the request -- in which case both server and
>   client know the updated rules, or
> * no Accept header was sent -- in which case the server whose
>   representation changes to require a new content format has no clear
>   way of indicating that under RFC7641 (ending with 4.06 Not Acceptable
>   would be close but isn't the expected response to a repeated request);
>   if the server changes the content format in a notification to an
>   unaware client, the client would catch it as a bad response (probably
>   similar to a response with a Content-Format not matching the sent
>   Accept). The client might consider the observation over while the
>   server does not, and will terminate the observation with a RST on the
>   next notification (or close the connection in TCP).
> 
> Impact on proxies: A proxy that enforces the previous rule on
> Content-Format staying constant would close observations (probably with
> something like 5.02 Bad Gateway), and the client would need to
> re-establish. No proxy implementations are known that implement that
> behavior. [ But I know only one so this would definitely need to be
> confirmed on the mailing list ].
> 
> [ I don't think that this has any actual interactions with the caching
> model, as the caching considerations are independent of the content
> format. ]

[1]: https://mailarchive.ietf.org/arch/msg/core/81SjKzfMJtSx6x8sc2P_RNq-M30

-- 
To use raw power is to make yourself infinitely vulnerable to greater powers.
  -- Bene Gesserit axiom

---

From: Christian Amsüss <[email-masked]>
Date: Sun, 24 Mar 2019 21:53:23 +0100

Hello WG,

brief update after some discussion primarily with Klaus:

> * Block1 and Observe.

Observe came first, so blockwise needs to be fixed. It's not this
thread's problem, issue closed.

More seriously: The behavior that probably follows from RFC8132 (fetch)
and observation seems simple, but the handling of observation resumption
is not trivial (in some cases sending the last Block1 request can be
sufficient to resume, sometimes the whole request operation needs to be
resumed).

Next step: Sketch it up for corr-clar.

> * Pinning of response content formats.

Seemed to be a straightforward idea at the time that wasn't throught
through in combination with what we know now (how could it).

Next step: Write update text, decide whether to spool it with any other
document or do a single-page update document that removes the restriction.
State that no proxies are known to enforce it.

(Can't just sit-and-wait in corr-clar as that is needed for any type of
pubsub that switch between empty-thing and value representations).

> * Pinning of all request options except ETag.

This one is not that simple, but in no hurry to solve.

Adding ETags has the (maybe unique) property that a response that the
meaning of responses is not modified; if any request reaffirming
messages get mixed up, worst to happen is a 2.01 rather than a 2.03.

It might be a good idea to steer this into deprecating ETag updates.
Might also be worth looking into in combination with nontraditional
responses.

Next steps: Further discussion, but no hurry either.

Best regards
Christian

---

From: Christian Amsüss <[email-masked]>
Date: Mon, 25 Mar 2019 14:37:01 +0100

Hello CoRE,

> State that no proxies are known to enforce it.

I've moved this together with the Accept-Any text from the pubsub
keep-it-simple/never-reified-unions thread[1], and written a short
draft[2] to specify it.

Abstract:

   This memo defines the Accept-Any option, which provides a more
   flexible content negotiation than the one originally specified for
   the Constrained Application Protocol (CoAP) in [RFC7252].  As this is
   particularly useful with but ruled out in CoAP observation
   ([RFC7641]), that is updated to allow it.

Kind regards
Christian

[1]: https://mailarchive.ietf.org/arch/msg/core/81SjKzfMJtSx6x8sc2P_RNq-M30
[2]: https://tools.ietf.org/html/draft-amsuess-core-accept-any-00

---

From: Christian Amsüss <[email-masked]>
Date: Thu, 28 Mar 2019 22:00:01 +0100

One more update here based on continued discussion and to clear out
misunderstandings.

> State that no proxies are known to enforce it.

Note that this proposal is not about changing what happens when an
Accept is sent (response is bound to be same format). It only opens up
observation to more elaborate negotiation patterns, and removes an
unanswered question about permissible error states.

The proposed Accept-Any option[1] (that builds on this but is not itself
the topic here) should not be used for "send full client supported list"
applications, the draft could be way more specific here.

It's also worth pointing out that this topic has only become pressing
due to the way pubsub handles observations. If nontraditional responses
become a part of pubsub, the issue may become a more theoretical one.

[1]: https://tools.ietf.org/html/draft-amsuess-core-accept-any-00

> Next steps: Further discussion, but no hurry either.

ETag updates are currently allowed and do have their use, but they also
come with considerations to be taken (like the client needing to be
prepared to still receive old messages, even for the updated request to
get lost if something switches round badly). We could write some
implementation guidance that describes the problems, and for some impls
it will be easier to just cancel / use a new token. It might be a good
idea to give implementation guidance with particular focus on OSCORE as
well (and think about OSCORE aware proxies in the course of that).

Using an active token (one that was not cancelled) and changing
something other than ETag is not conforming to the protocol, and the
server or proxy could do all kinds of things. A token is in use until a
response arrives on the token that has no Observe option.

Kind regards
Christian

-- 
Most people would leave. Not us. We're Vikings. We have stubbornness issues.
  -- Hiccup, son of Stoic