Subject: [core] Comments on draft-ietf-core-observe-10

From: "Peter A. Bigot" <[email-masked]>
Date: Thu, 26 Sep 2013 09:49:50 -0500

I've got some spare time and happened across an opportunity to play with 
some CoAP implementations.  To support that I've been trying to catch up 
on what's happened in CoRE since I was last here in 2011.

First issue:

In section 4.5 of draft-ietf-core-observe-10 there is the text:

    The server MUST limit the number of confirmable notifications for
    which an acknowledgement has not been received yet to NSTART (1 by
    default; see Section 4.7 of RFC XXXX [I-D.ietf-core-coap]).

As I read Section 4.7 of draft-ietf-core-18 it is a MUST that a Client 
(in this case "the server") limit exactly this type of message to NSTART 
instances for each end-point (observer).

What additional constraint is this sentence intended to provide?

Naively it might extend the limit to apply across all observers, but 
that would be counter-productive to the purpose of distributing 
observations.  It might relax the limit to refer to each single 
observable (rather than across all to which a given observer has 
subscribed), but I don't think draft-ietf-core-observe should be 
permitted to relax the underlying CoAP constraint that way.

Second issue:

In section 4.4 of draft-ietf-core-observe-10 there is the text:

    The sequence number selected for a notification MUST be greater than
    that of any preceding notification sent to the same client with the
    same token for the same resource.  The value of the Observe Option
    MUST be current at the time of transmission; if a notification is
    retransmitted, the server MUST update the value of the option to the
    sequence number that is current at that time before sending the
    message.

    The sequence numbers generated for a resource MUST provide an order
    among all notifications resulting from all requests from the same
    client endpoint.

This was revelatory, since it mandates that the transmission layer be 
able to validate and potentially update the (option) content of a 
message before (re-)transmitting it.  (A similar issue for Max-Age is 
consequent to Section 5.10.5 of draft-ietf-core-18, though it is a 
SHOULD there, and the requirement could be met without interacting with 
an upper layer.)  Since the value corresponding to the updated sequence 
number probably also changed, this implies the entire content of the 
un(re-)transmitted message would need to be replaced too.  Is this 
intentional?  Does it implicitly sanction the same sort of update for 
arbitrary messages in base CoAP?  I had thought this was agreed to be 
undesirable by the working group back in 2010 and the expectation was 
retransmissions at the CoAP layer could never change the message 
content, though I can't locate the reference.

The consequences if NSTART is greater than one are unclear but worrisome 
to me.

Wouldn't it be simpler to solve this by modifying the text in Section 
4.5 to cancel any unacknowledged Con or untransmitted Non-Con 
notification if the underlying resource changed?  Or, if that's too 
disruptive in situations of rapid change, doing so when a new 
notification for the same resource to the same endpoint is generated?

That may be what the steps in Section 4.5 are intended to require, 
though I don't understand step one:

    1.  Wait for the current transmission attempt to complete.

I don't know if "current transmission attempt" is just an active radio 
transmission that takes a few milliseconds, or a step of the 
backoff/retransmit/wait algorithm which might take a minute or more.  
The referenced to "completed transmission attempt timed out" in step 4 
suggests this step may take a relatively long time.  I think if there's 
any waiting going on the cancellation should occur immediately.

Regardless I think all this should be done relative to one notification, 
not NSTART, again because things get messy if NSTART is greater than 
one: there would legitimately be multiple active notifications to the 
same observer for the same resource with either conflicting or duplicate 
content.

Hope I'm not too confused about all this having been gone for so long.

Peter

---

From: Klaus Hartke <[email-masked]>
Date: Fri, 27 Sep 2013 04:39:47 +0300

Hi Peter,

welcome back and thanks for the feedback!

> As I read Section 4.7 of draft-ietf-core-18 it is a MUST that a Client (in
> this case "the server") limit exactly this type of message to NSTART

Section 4.7 of -coap-18 limits the client (the initiator of requests)
to NSTART, but actually states no requirements for the CoAP server
(the responder). This is iirc because in the normal request/response
case, the server only ever transmits something in reaction to the
client. In the observe case, though, the server does not only react to
the client but transmits notifications over a long period of time on
its own initiative. So this sentence is intended to put the same
requirements on the server for notifications as -coap-18 puts on the
client for requests.

> Naively it might extend the limit to apply across all observers, but that
> would be counter-productive to the purpose of distributing observations. It
> might relax the limit to refer to each single observable (rather than across
> all to which a given observer has subscribed), but I don't think
> draft-ietf-core-observe should be permitted to relax the underlying CoAP
> constraint that way.

The whole text is written in the scope of one client observing one
resource. But, of course, when a client observes two or more resources
on the same server, the server must perform congestion control over
the aggregate traffic.

> This was revelatory, since it mandates that the transmission layer be able
> to validate and potentially update the (option) content of a message before
> (re-)transmitting it.  (A similar issue for Max-Age is consequent to Section
> 5.10.5 of draft-ietf-core-18, though it is a SHOULD there, and the
> requirement could be met without interacting with an upper layer.)

The difference between Max-Age and the sequence number is that it's
only slightly less correct if Max-Age isn't updated on retransmission,
but things actually break if the sequence number isn't updated.

There are multiple possible implementations that fit the requirements.
If you use an implementation where the sequence number doesn't change
between retransmissions of the same notification, then you don't need
your transmission layer to update the option content.

> Since
> the value corresponding to the updated sequence number probably also
> changed, this implies the entire content of the un(re-)transmitted message
> would need to be replaced too.  Is this intentional?  Does it implicitly
> sanction the same sort of update for arbitrary messages in base CoAP?  I had
> thought this was agreed to be undesirable by the working group back in 2010
> and the expectation was retransmissions at the CoAP layer could never change
> the message content, though I can't locate the reference.

Section 4.4 of -observe-10 only says the the sequence number must be
up-to-date. When the value changes (i.e., the representation of the
current resource state), then that's a new transmission, as described
at the of Section 4.5.

> The consequences if NSTART is greater than one are unclear but worrisome to
> me.
>
> Wouldn't it be simpler to solve this by modifying the text in Section 4.5 to
> cancel any unacknowledged Con or untransmitted Non-Con notification if the
> underlying resource changed?  Or, if that's too disruptive in situations of
> rapid change, doing so when a new notification for the same resource to the
> same endpoint is generated?
>
> That may be what the steps in Section 4.5 are intended to require, though I
> don't understand step one:
>
> backoff/retransmit/wait algorithm which might take a minute or more.  The
> referenced to "completed transmission attempt timed out" in step 4 suggests
> this step may take a relatively long time.  I think if there's any waiting
> going on the cancellation should occur immediately.

I've clarified the text in SVN as follows:

   When the state of an observed resource changes while the number of
   outstanding acknowledgements is greater than or equal to NSTART, or
   while the waiting time for a non-confirmable notification has not
   elapsed yet, the server MUST proceed as follows:

   1.  Wait for the current transmission attempt to be acknowledged,
       rejected or to time out (confirmable transmission), or the
       waiting time to elapse (non-confirmable transmission).

It is necessary to wait for the transmission attempt to complete
first. Because otherwise, if the transmission is canceled and a new
transmission is started immediately, there are two problems when
notifications are generated more quickly than can be consumed (for
example, at a proxy located at the boundary between a fast network and
a constrained network):

* the client and/or network could be overloaded; and

* transmissions would never time out and therefore a dead client would
never be removed from the list of observers.

> Regardless I think all this should be done relative to one notification, not
> NSTART, again because things get messy if NSTART is greater than one: there
> would legitimately be multiple active notifications to the same observer for
> the same resource with either conflicting or duplicate content.

One of the goals of -observe is that

      Clients should see the new state after a state change as soon
      as possible, and they should see as many states as possible.
      (draft-ietf-core-observe-10#section-1.4)

So if it's possible to let a client see more intermediate state
changes by having more than one active notification at the same time,
that would help that goal. The notifications aren't conflicting --
they provide more detail; each notification has a sequence number that
lets the client put the messages back into order.

How and if NSTART may be allowed a value greater than one is an open
question at this time:

   Further congestion control optimizations and considerations are
   expected in the future, which may for example provide automatic
   initialization of the CoAP transmission parameters defined in
   Section 4.8, and thus may allow a value for NSTART greater than one.
   (draft-ietf-core-coap-18#section-4.7)

Best regards,
Klaus

PS: You can see the full changes I've made in SVN based on your
comments here: <http://trac.tools.ietf.org/wg/core/trac/changeset/1502>

---

From: "Peter A. Bigot" <[email-masked]>
Date: Thu, 26 Sep 2013 23:54:40 -0500

On 09/26/2013 08:39 PM, Klaus Hartke wrote:
> (the responder).

OK, I can see that now.

Aside: If there's another edit opportunity before draft-ietf-core-coap 
becomes an RFC, the first line of the second paragraph of 4.7:

    In order not to cause congestion, Clients (including proxies) MUST

should not capitalize Clients.  No similar use within a sentence is 
capitalized.

>   This is iirc because in the normal request/response

There is no "of course" if the specification isn't clear.  I can still 
read the text as disallowing more than NSTART notifications over all 
clients and/or resources, or I could read it as allowing more than 
NSTART notifications to a single client observing multiple resources.  I 
believe the intent is to allow the first, and disallow the second.  The 
wording should be clarified.

> your transmission layer to update the option content.

I believe I see how you're thinking about the relationship between 
sequence numbers, resource updates, and notifications, but I can't 
respond until the issue below is clarified.

>         waiting time to elapse (non-confirmable transmission).

I need to fix my understanding of what transmission and retransmission mean.

For comparison, Section 5.10.5 (Max-Age) of draft-ietf-core-coap-18 has:

    The value is intended to be current at the time of transmission.
    Servers that provide resources with strict tolerances on the value of
    Max-Age SHOULD update the value before each retransmission. (See
    also Section 5.7.1.)

Though this certainly applies to proxies, the wording suggests it also 
applies to origin servers.

With the default transmission parameters, a CON message with maximum 
initial timeout will be transmitted at most five times at offsets of 0, 
3, 6, 12, and 24 seconds, with a 48 second wait after the fourth 
retransmission, resulting in a MAX_TRANSMIT_WAIT of 93 seconds.

If the CON message has a Max-Age option with value 30, does the above 
text mean that the server SHOULD update the value in the Max-Age option 
per the following table while waiting for the reliable transmission to 
be completed?

    OffsetTime   Transmission#   Transmitted Max-Age
        0             1            30
        3             2            27
        9             3            21
       21             4             9
       45             5             0

In the context of Section 4.5 of observe-10: If this confirmable message 
were a notification (and did not have Max-Age), would a change in the 
value of the underlying resource at T=15 while this notification is 
outstanding force the server to execute the five step process?  At what 
time would step 1 of that process complete: 15 seconds, 21 seconds, 93 
seconds, or some other time?

>     (draft-ietf-core-coap-18#section-4.7)

Yes.  We need to resolve what Section 4.5 really requires of an 
implementation for NSTART=1 before assessing what it requires when 
NSTART>1 .

> comments here: <http://trac.tools.ietf.org/wg/core/trac/changeset/1502>

Thanks.

Peter

---

From: Klaus Hartke <[email-masked]>
Date: Fri, 27 Sep 2013 08:29:47 +0300

Peter A. Bigot wrote:
> There is no "of course" if the specification isn't clear.  I can still read
> the text as disallowing more than NSTART notifications over all clients
> and/or resources, or I could read it as allowing more than NSTART
> notifications to a single client observing multiple resources.  I believe
> the intent is to allow the first, and disallow the second.

That's indeed what is intended.

> The wording should be clarified.

Could you provide some text?

> applies to origin servers.

Yes.

> With the default transmission parameters, a CON message with maximum initial
> timeout will be transmitted at most five times at offsets of 0, 3, 6, 12,
> and 24 seconds, with a 48 second wait after the fourth retransmission,
> resulting in a MAX_TRANSMIT_WAIT of 93 seconds.

Yes.

> If the CON message has a Max-Age option with value 30, does the above text
> mean that the server SHOULD update the value in the Max-Age option per the
> following table while waiting for the reliable transmission to be completed?

Yes.

> were a notification (and did not have Max-Age), would a change in the value
> of the underlying resource at T=15 while this notification is outstanding
> force the server to execute the five step process?  At what time would step
> 1 of that process complete: 15 seconds, 21 seconds, 93 seconds, or some
> other time?

Each row in the table is one transmission attempt. So if the server
sends a confirmable notification for resource state X and the state
changes to Y at T=15, then the example would look as follows:

    OffsetTime   Transmission#   Transmitted State   Message ID
        0             1                  X              1230
        3             2                  X              1230
        9             3                  X              1230
       21             4                  Y              1231
       45             5                  Y              1231

That is, the server waits until the transmission attempt that is
current at T=15 (transmission #3) completes (here: times out; this is
step 1) and then starts a new transmission at T=21 (step 3) while it
keeps increasing the transmission counter and doubling the timer (step
4), so the overall transmission process ultimately times out at T=93.

Best regards,
Klaus

---

From: "Peter A. Bigot" <[email-masked]>
Date: Fri, 27 Sep 2013 08:28:43 -0500

So you understand my perspective, in my mental model of CoAP 
architectural layers an application at an endpoint creates a message and 
hands it off to a message layer for transmission.  With the exception of 
Max-Age, which can be handled within the message layer, nothing until 
now has *required* interaction with the higher layers (transaction or 
application/resource) during the process of reliable (re-)transmission 
by the message layer.  By coap-18 section 4.2 the higher layers may 
cancel the transmission sequence if something happens that allows the 
server to know that the receiver got the message, but calls in the other 
direction are not necessary.

> Could you provide some text?

I'll try, but this is taking way too much time for something I'm doing 
on the side.

>> following table while waiting for the reliable transmission to be completed?

Excellent.  In this case, the message layer is capable of doing this 
update without interacting with higher layers, since it can record when 
it received the message.  I expect that, when it does this, it does 
*not* update any other part of the message, including the message ID.  True?

> 4), so the overall transmission process ultimately times out at T=93.

Thanks.  That's clear, but I don't think it's the best solution.  (I do 
agree that, to satisfy congestion control, a new retransmission sequence 
cannot start until 93 seconds have elapsed or the sender can tell the 
receiver got the message, and we need to preserve that behavior.)

You're showing Message ID changing during the transmission of a 
confirmable message, and you don't have a column for the sequence 
number.  If you meant this as:

   OffsetTime   Transmission#   Transmitted State   Message ID Observe
       0             1                  X              1230 1000
       3             2                  X              1230 1000
       9             3                  X              1230 1000
      21             4                  Y              1231 1001
      45             5                  Y              1231 1001

then this is just weird because if the client got transmissions 3 and 4 
it has to send two confirmations, one for message ID 1230 and one for 
message ID 1231, to something that is essentially one confirmable 
message.  When the value of Max-Age changes in base CoAP nobody has said 
the Message ID also needs to change (unless you did in a response to 
that question above).

Is there any other case where CoAP has required the Message ID to change 
during a reliable transmission of a single message?

Or is this sequence to be conceived as a new message that starts 
part-way through the retransmission process, with transmission 
parameters not in their initial state?

Both of these interpretations seem to be creating new concepts 
unnecessarily and bother me.

You might have meant:

   OffsetTime   Transmission#   Transmitted State   Message ID Observe
       0             1                  X              1230 1000
       3             2                  X              1230 1000
       9             3                  X              1230 1000
      21             4                  Y              1230 1001
      45             5                  Y              1230 1001

which keep one message ID for the entire retransmission process. It is 
unusual, since it changes the content of the message during the 
retransmissions, but it still satisfies congestion limits and correct 
correlation of the resource state with a sequence number for ordering.  
It's also consistent with the pre-existing case of Max-Age.  Since the 
semantics of the notification is eventual consistency and the sequence 
number is paired with the state, it just works.

I have no problem with this version as a valid sequence.

Getting back to the context of the original text from section 4.4 of 
observe-10:

The question now is whether each of these retransmissions is a 
notification that, because of this clause, must be modified.  I propose 
the answer should be No because it is *not* necessary for correctness.

Changing the content of the notification because it is out of date is an 
optimization.  Is there anything in observe (other than this text) that 
*requires* the server to send the new state immediately?  The Freshness 
section allows the server to skip whole notifications to prevent congestion.

If NSTART>1, is there anything wrong with the following sequence?

     OffsetTime   Transmission#   Transmitted State   Message ID Observe
         0             1                  X              1230 1000
         3             2                  X              1230 1000
         9             3                  X              1230 1000
        15             1                  Y              1231 1001
        18             2                  Y              1231 1001
        21             4                  X              1230 1000
        24             3                  Y              1231 1001
        36             4                  Y              1231 1001
        45             5                  X              1230 1000
        60             5                  Y              1231 1001

Here we have interleaved outdated and current values.  The sequence 
number still allows the client to re-order the content, and as you noted 
there is value in allowing the client to see the older value.  Each 
notification remains essentially unchanged throughout its retransmission 
cycle (modulo Max-Age), and the CoAP layers remain decoupled.

I think the optimization can be permitted without requiring it with text 
something like:

    The sequence number selected for a notification MUST be greater than 
that
    of any preceding notification sent to the same client with the same 
token
    for the same resource.  If a confirmable notification must be
    retransmitted or a non-confirmable notification has been delayed, the
    server MAY update the option value and content of the message to 
reflect the
    sequence number and resource state that is current at the time of
    retransmission.  (This is not a new notification, and the Message ID 
does
    not change.)

Then the series of steps in 4.5 can remain with "MUST proceed" changed 
to "MAY", removing the text that suggests that the modifications produce 
a new notification, and adding the option of cancelling an older 
notification if its content has been updated to be redundant with a 
newer notification.

I think that if left as MUST it (a) couples the message layer with the 
service application to re-validate the content before transmission, and 
(b) introduces complexity and waste if NSTART>1 since it would mandate 
that multiple outstanding transmissions all update to have the same 
content except message ID.

Peter

---

From: Klaus Hartke <[email-masked]>
Date: Mon, 30 Sep 2013 09:07:39 +0300

Peter A. Bigot wrote:
> Or is this sequence to be conceived as a new message that starts part-way
> through the retransmission process, with transmission parameters not in
> their initial state?

This.

> Excellent.  In this case, the message layer is capable of doing this update
> without interacting with higher layers, since it can record when it received
> the message.  I expect that, when it does this, it does *not* update any
> other part of the message, including the message ID.  True?

When retransmitting a notification message, all parts of the message
stay the same, including the message ID, except that Max-Age SHOULD
and the Observe sequence number MUST be up-to-date at the time of each
transmission attempt, which may or may not require an update to the
message.

Section 4.4 of observe-10 describes two implementation techniques to
satisfy the 'MUST': one where, in your terms, the sequence numbers are
generated by the message layer. This can be done without communicating
with any higher layer, but requires a change to the message at each
transmission attempt. And one where the sequence numbers are generated
by a higher layer. This requires the higher layer to perform some
bookkeeping, but no changes to the message are required at the message
layer.

> With the exception of Max-Age, which can be
> handled within the message layer, nothing until now has *required*
> interaction with the higher layers (transaction or application/resource)
> during the process of reliable (re-)transmission by the message layer.

For supporting observe-10, the message layer should just be able to
inform the higher layer of a client that is gone, and to replace an
ongoing transmission process with a new transmission process that
inherits the current transmission parameters. Both can be implemented
without requiring the message layer to start an interaction with a
higher layer.

> *requires* the server to send the new state immediately?

Section 1.4 of observe-10 states that one of the goals of the protocol is that

      Clients should see the new state after a state change
      as soon as possible [...]

So it's not merely an optimization that a server tries to get a
representation of the the latest resource state to the client, but the
very purpose of the protocol.

Best regards,
Klaus

---

From: "Peter A. Bigot" <[email-masked]>
Date: Mon, 30 Sep 2013 04:10:24 -0500

>> through the retransmission process, with transmission parameters not in

You removed too much context: "This" was the one where the message ID 
changed for the third and fourth "retransmission", which to me says it's 
a new message.

> stay the same, including the message ID, except that Max-Age SHOULD

And here you say the message ID stays the same.

Which is it?

> transmission attempt, which may or may not require an update to the

If the sequence number is managed by the application and is a one-up 
counter incremented when the representation changes, then not only the 
sequence number but also the payload needs to change in order to satisfy 
this MUST.

> without requiring the message layer to start an interaction with a

Yes, but the second is a new expectation placed on layer interactions by 
observe that is not required by the underlying protocol (CoAP).

CoAP base permits (but does not require) an implementation where the 
upper layer can notify the message layer that it can stop retransmitting 
a confirmable message (because confirmation of receipt was detected 
through other means not visible to the message layer).

observe is now proposing that implementations allow higher layers to 
replace the content of an in-progress transmission.  You've not made 
clear the limits of this replacement.  Option values can change (this is 
not new).  Message ID may? may not? change (if it does, this is new). 
Can payload change, as it must if sequence numbers aren't simply clock 
readings?  Can the response code change, as it must if the resource was 
deleted?

Replacing message content isn't necessary if the message layer generates 
sequence numbers itself, just as it manages Max-Age, but this means the 
implementation where the sequence number changes in lock-step with 
resource representation change is no longer conformant and should be 
removed from section 4.4.

> very purpose of the protocol.

I think this is a matter of judgement.  observe disallows additional 
notifications when they would cause the server to violate congestion 
limitations already imposed on clients, and allows the server to 
postpone a notification if a new one is expected to occur soon.  So 
there are clearly limits to what "as soon as possible" permits and requires.

The consequence of your MUST violates either the association between a 
message-ID and the message content, or that between the message-ID and 
the process of transmitting a single confirmable message, or eliminates 
the second "valid implementation" in section 4.4.

The first two would be bad precedent for a protocol extension.  The 
third would add complexity to the message layer.  Better to simply 
accept that sometimes notifications will be out of date, and their 
information will be updated with a new notification "as soon as possible".

Peter

---

From: Klaus Hartke <[email-masked]>
Date: Mon, 30 Sep 2013 18:13:11 +0300

Peter,

draft-ietf-core-coap defines a protocol for request-response
interactions. draft-ietf-core-observe defines interactions where one
request can yield multiple notifications. This alone makes it clear
that -observe is not just an application of -coap, but an extension of
the core protocol.

Section 4.2 of coap-18 specifies the process for transmitting a
separate response. It involves one or more transmission attempts, a
retransmission counter and a retransmission timer. The transmitted
message does not change between transmission attempts, except for the
value of the Max-Age option.

Section 4.5 of observe-10 extends this slightly: if a server generates
a new notification while the transmission process for a previous
notification is still active, then -- to meet the goals of the
protocol, the demands of congestion control and the requirement to
detect clients that are gone -- the server aborts the transmission
process for the old notification (but not before the current
transmission attempt completes) and starts a new transmission process
for the new notification (but with the retransmission counter and
retransmission timer of the aborted process instead of the default
values). During each transmission process, the transmitted message
does not change between transmission attempts, except for the value of
the Max-Age option and potentially the sequence number in the Observe
option.

I'm not sure what's so difficult about this.

Best regards,
Klaus

On 30 September 2013 12:10, Peter A. Bigot <[email-masked]> wrote:
> changed for the third and fourth "retransmission", which to me says it's a
> new message.
>
>
>>
>>> update
> If the sequence number is managed by the application and is a one-up counter
> incremented when the representation changes, then not only the sequence
> number but also the payload needs to change in order to satisfy this MUST.
>
>
>>
> CoAP base permits (but does not require) an implementation where the upper
> layer can notify the message layer that it can stop retransmitting a
> confirmable message (because confirmation of receipt was detected through
> other means not visible to the message layer).
>
> observe is now proposing that implementations allow higher layers to replace
> the content of an in-progress transmission.  You've not made clear the
> limits of this replacement.  Option values can change (this is not new).
> Message ID may? may not? change (if it does, this is new). Can payload
> change, as it must if sequence numbers aren't simply clock readings?  Can
> the response code change, as it must if the resource was deleted?
>
> implementation where the sequence number changes in lock-step with resource
> representation change is no longer conformant and should be removed from
> section 4.4.
>
>
>>
> limitations already imposed on clients, and allows the server to postpone a
> notification if a new one is expected to occur soon.  So there are clearly
> limits to what "as soon as possible" permits and requires.
>
> message-ID and the message content, or that between the message-ID and the
> process of transmitting a single confirmable message, or eliminates the
> second "valid implementation" in section 4.4.
>
> The first two would be bad precedent for a protocol extension.  The third
> would add complexity to the message layer.  Better to simply accept that
> sometimes notifications will be out of date, and their information will be
> updated with a new notification "as soon as possible".
>
> Peter
> _______________________________________________
> core mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/core

---

From: "Peter A. Bigot" <[email-masked]>
Date: Mon, 30 Sep 2013 11:06:32 -0500

> value of the Max-Age option.

I certainly had this immutability expectation; but is there something in 
coap-18 that requires it, or was it simply assumed with the rather 
subtle SHOULD down in coap-18 section 5.10.5 permitting a single exception?

> I'm not sure what's so difficult about this.

Mostly that nothing in the documents told me as clearly as you just did 
that observe was intending to make such a significant change in an 
implementer's expectations regarding content changes within or process 
of a confirmable CoAP transmission.  In my opinion, this is not a 
"slight change" to these concepts.

My last comment: please make clear that when the steps in observe 4.5 
occur the message ID is (or is not) expected to be one of the things 
that changes when the new transmission starts with the retransmission 
state left over from the superseded transmission.  That clarity is 
absolutely lacking in the current draft for anybody who hasn't been 
actively involved in observe over the last three years.

Even after this long exchange I still don't know the answer.

Peter