Subject: [core] John Scudder's Discuss on draft-ietf-core-new-block-11: (with DISCUSS and COMMENT)

From: John Scudder via Datatracker <[email-masked]>
Date: Wed, 05 May 2021 17:41:32 -0700

John Scudder has entered the following ballot position for
draft-ietf-core-new-block-11: Discuss

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)

Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about DISCUSS and COMMENT positions.

The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-core-new-block/

----------------------------------------------------------------------
DISCUSS:
----------------------------------------------------------------------

For the most part I found this document relatively easy to follow, considering
my complete lack of background in CoAP. However, despite a concerted effort I
have not been able to nail down with confidence what the intended semantics of
several of your timeouts are, notably NON_RECEIVE_TIMEOUT. Some of the text
(for example, §4.4) implies that the timeout is an upper bound on how long an
implementation should wait before declaring a block to have been lost (“The
client SHOULD wait for up to NON_RECEIVE_TIMEOUT”). At the very least, this is
imprecise because the timeout increases exponentially with repeated timeouts —
but this is a relatively minor matter, discussed further in my comments.

Later, in §7.2, you say that expiry of the timeout is not the only trigger for
a 4.08 response:

   It is likely that the client will start transmitting the next set of
   MAX_PAYLOADS payloads before the server times out on waiting for the
   last of the previous MAX_PAYLOADS payloads.  On receipt of the first
   payload from the new set of MAX_PAYLOADS payloads, the server SHOULD
   send a 4.08 (Request Entity Incomplete) Response Code indicating any
   missing payloads from any previous MAX_PAYLOADS payloads.

It makes sense to me that you use this additional trigger. At this point in my
reading of the spec, my understanding of the retransmission algorithm was that
a 4.08 should be sent when either a payload is received from a new set of
MAX_PAYLOADS, or NON_RECEIVE_TIMEOUT expires. But then I got to the example in
10.2.3, which shows the client waiting for the expiration of
NON_RECEIVE_TIMEOUT even though it has received the first of a new set of
MAX_PAYLOADS, and I concluded that either I’ve missed something basic, or the
document is internally inconsistent.

As an aside, I’m also unclear as to why the only trigger you specify for
sending a 4.08 is the arrival of the first of a new MAX_PAYLOADS flight. Other
possible triggers I noticed include a gap in the sequence, and reception of a
payload with More=0.

Some of these issues are repeated in my comments, below — I’ve noted those in
the comment. Possibly in addressing this DISCUSS we’ll clear up some of those
comments too.

----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

Comments:

(draft-ietf-core-new-block-11)

1. Section 3.2

   This mechanism is not intended for general CoAP usage, and any use
   outside the intended use case should be carefully weighed against the
   loss of interoperability with generic CoAP applications.

I’m curious: is the only reason the mechanism isn’t intended for general usage,
the fact some implementations won’t support it? Or does it have other
deficiencies that also make it unsuitable?

2. Section 4.1

   Q-Block2 Option is useful with GET, POST, PUT, FETCH, PATCH, and
   iPATCH requests and their payload-bearing responses (2.01, 2.02,
   2.03, 2.04, and 2.05) (Section 5.5 of [RFC7252]).

I found the list of codes incomprehensible on first encountering it, since the
concept of response codes hadn’t been introduced yet. I do understand that the
document assumes familiarity with CoAP; nonetheless for basic clarity I think
this should say “(response codes 2.01, 2.02…”. Additionally, the reference to
RFC 7252 §5.5 doesn’t seem to be especially germane?

By the way, is 2.03 indeed a payload-bearing response? The only other place the
spec touches on it is in §4.4, which says “the server could respond with a 2.03
(Valid) response with no payload”.

3. Section 4.1

   To indicate support for Q-Block2 responses, the CoAP client MUST
   include the Q-Block2 Option in a GET or similar request (FETCH, for
   example), the Q-Block2 Option in a PUT or similar request, or the
   Q-Block1 Option in a PUT or similar request so that the server knows
   that the client supports this Q-Block functionality should it need to
   send back a body that spans multiple payloads.  Otherwise, the server
   would use the Block2 Option (if supported) to send back a message
   body that is too large to fit into a single IP packet [RFC7959].

Is this paragraph really supposed to mention both Q-Block2 and Q-Block1? In
particular, I’m confused by the mention of both of these in relation to PUT.

4. Section 4.1

   The Q-Block1 and Q-Block2 Options are unsafe to forward.  That is, a
   CoAP proxy that does not understand the Q-Block1 (or Q-Block2) Option
   MUST reject the request or response that uses either option.

Presumably (hopefully) this is simply describing the behavior of existing
spec-compliant proxies when processing the new messages. As such, is the MUST
appropriate? I would think not.

5. Section 4.3

      body.  Note that the last received payload may not be the one with
      the highest block number.

“Might not” would be less ambiguous than “may not”.

6. Section 4.4 (also two places in §4.3)

(This comment rehashes, in more detail, the difficulty explained in my DISCUSS.
You may want to skip over it until we’ve resolved the DISCUSS, after which this
may, or may not, be relevant.)

   The client SHOULD wait for up to NON_RECEIVE_TIMEOUT (Section 7.2)

I read this as meaning the client should wait for as little as zero, or as long
as NON_RECEIVE_TIMEOUT — that’s my understanding of “up to”. Is that the
intended meaning? If it is, I think it’s worth writing out as I’ve done, for
clarity. If it’s not, it definitely needs to be fixed.

There’s a similar issue with “up to NON_PARTIAL_TIMEOUT” later in the section.

Referring ahead to Section 7.2 muddies the waters further. Even though the text
quoted above says NON_RECEIVE_TIMEOUT is an upper limit on how long to wait,
§7.2 says it’s a lower limit instead... maybe? From §7.2:

   NON_RECEIVE_TIMEOUT is the initial maximum time to wait for a missing

“Maximum”, ok great, that means “upper bound” and so lines up with §4.4
although the “initial” is surprising since §4.4 doesn’t say anything about the
upper limit increasing. It continues:

   payload before requesting retransmission for the first time.  Every
   time the missing payload is re-requested, the time to wait value
   doubles.  The time to wait is calculated as:

      Time-to-Wait = NON_RECEIVE_TIMEOUT * (2 ** (Re-Request-Count - 1))

But this part says it’s (a) an exact time-to-wait, not a “maximum”, and (b) it
says it increases exponentially, so NON_RECEIVE_TIMEOUT isn’t a maximum at all,
but a minimum.

This later text in §7.2 implies that perhaps the problem in the above passages
is the word “maximum”, and it should simply be deleted:

   For the server receiving NON Q-Block1 requests, it SHOULD send back a
   2.31 (Continue) Response Code on receipt of all of the MAX_PAYLOADS
   payloads to prevent the client unnecessarily delaying.  If not all of
   the MAX_PAYLOADS payloads were received, the server SHOULD delay for
   NON_RECEIVE_TIMEOUT (exponentially scaled based on the repeat request
   count for a payload) before sending the 4.08 (Request Entity
   Incomplete) Response Code for the missing payload(s).

Similarly “up to” in the quote that began this comment should be “at least”.

Whether you adopt those suggestions or not,  it seems as though all this needs
to be rewritten with careful attention to conveying what the desired behavior
is.

But the plot thickens. Later in §7.2 we have

   It is likely that the client will start transmitting the next set of
   MAX_PAYLOADS payloads before the server times out on waiting for the
   last of the previous MAX_PAYLOADS payloads.  On receipt of the first
   payload from the new set of MAX_PAYLOADS payloads, the server SHOULD
   send a 4.08 (Request Entity Incomplete) Response Code indicating any
   missing payloads from any previous MAX_PAYLOADS payloads.

The point being that the retransmission request can be triggered by an event
other than timer expiration. So in that sense, “maximum” is right — it provides
an upper bound on how long to wait before requesting a retransmission — but in
another sense it’s wrong because the exponential increase is applied to it. I
think the word “maximum” is trying to do too much work, and more words are
probably required in order to make this clear. I also think the problem is
exacerbated by the fact both §4.4 and §7.2 are talking normatively about how to
use NON_RECEIVE_TIMEOUT. It seems as though the main description is found in
§7.2, and some confusion would be avoided by making §4.4 less specific, and
simply referring forward to §7.2.

And, as noted in my DISCUSS, example 10.2.3 muddies the waters still further
since it illustrates yet another behavior.

7. Section 4.4

   The client SHOULD wait for up to NON_RECEIVE_TIMEOUT (Section 7.2)
   after the last received payload for NON payloads before issuing a
   GET, POST, PUT, FETCH, PATCH, or iPATCH request that contains one or
   more Q-Block2 Options that define the missing blocks with the M bit
   unset.  The client MAY set the M bit to request this and later blocks
   from this MAX_PAYLOADS set.  Further considerations related to the
   transmission timing for missing requests are discussed in
   Section 7.2.

I find this whole paragraph pretty confusing with the dueling SHOULD and MAY,
where it appears the SHOULD might be doing two jobs at once. I *think* your
intent is something like the following?

“The client SHOULD wait as specified in Section 7.2 for NON payloads before
requesting retransmission of any missing blocks. Retransmission is requested by
issuing a GET, POST, PUT, FETCH, PATCH, or iPATCH request that contains one or
more Q-Block2 Options that define the missing block(s). Generally the M bit on
the Q-Block option(s) SHOULD be unset, although the M bit MAY be set to request
this and later blocks from this MAX_PAYLOADS set, see Section 10.2.4 for an
example of this in operation.”

8. Section 5

   If the size of the 4.08 (Request Entity Incomplete) response packet
   is larger than that defined by Section 4.6 [RFC7252], then the number
   of missing blocks MUST be limited so that the response can fit into a
   single packet.  If this is the case, then the server can send

Suggestion: “then the number of missing blocks reported MUST...” (The thing
being limited is not the actual number of missing blocks. You’re limiting the
number you report on.)

9. Section 7.1

   It is implementation specific as to whether there should be any
   further requests for missing data as there will have been significant
   transmission failure as individual payloads will have failed after
   MAX_TRANSMIT_SPAN.

This paragraph seems as though it’s a non-sequitur. It just doesn’t make sense
to me. :-(

10. Section 7.2

(This comment relates to the difficulty explained in my DISCUSS. You may want
to skip over it until we’ve resolved the DISCUSS, after which this may, or may
not, be relevant.)

   NON_TIMEOUT is the maximum period of delay between sending sets of
   MAX_PAYLOADS payloads for the same body.  By default, NON_TIMEOUT has
   the same value as ACK_TIMEOUT (Section 4.8 of [RFC7252]).

Presumably the use of “maximum” means it’s fine to delay zero seconds (or any
value lower than NON_TIMEOUT).

11. General

By the way, none of the timers specify jitter (and indeed, if read literally,
jitter would be forbidden). Is this intentional?

12. Section 7.2

   If the CoAP peer reports at least one payload has not arrived for
   each body for at least a 24 hour period and it is known that there
   are no other network issues over that period, then the value of
   MAX_PAYLOADS can be reduced by 1 at a time (to a minimum of 1) and
   the situation re-evaluated for another 24 hour period until there is
   no report of missing payloads under normal operating conditions.  The
   newly derived value for MAX_PAYLOADS should be used for both ends of
   this particular CoAP peer link.  Note that the CoAP peer will not
   know about the MAX_PAYLOADS change until it is reconfigured.  As a
   consequence of the two peers having different MAX_PAYLOADS values, a
   peer may continue indicate that there are some missing payloads as
   all of its MAX_PAYLOADS set may not have arrived.  How the two peer
   values for MAX_PAYLOADS are synchronized is out of the scope.

I take it this is just thrown in here as an operational suggestion? It’s not
specifying protocol, right? It seems a little misplaced, if so.

13. Section 10.1.3

(This comment relates to the aside in my DISCUSS. You may want to skip over it
until we’ve resolved the DISCUSS, after which this may, or may not, be
relevant.)

Why doesn’t the server request 1,9,10 in one go? Since its rxmt request is
triggered by rx of 11, one would think it could infer 10 had been lost.

14. Section 10.1.4 (also 10.3.3)

(This comment relates to the aside in my DISCUSS. You may want to skip over it
until we’ve resolved the DISCUSS, after which this may, or may not, be
relevant.)

Why doesn’t reception of a message with More=0 trigger the server to request
retransmission of the missing block? Why does it have to wait for timeout?

15. Section 10.2.3

(This comment relates to my DISCUSS. You may want to skip over it until we’ve
resolved the DISCUSS, after which this may, or may not, be relevant.)

Why doesn’t reception of QB2:10/0/1024 trigger the client to request
retransmission? Why does it have to wait for timeout? Similarly reception of
QB2:9/1/1024 later in the example.

16. Section 10.2.4

Since MAX_PAYLOADS is 10, why does the example say “MAX_PAYLOADS has been
reached” after payloads 2-9 have been retransmitted? That’s only 8 payloads.

---

From: <[email-masked]>
Date: Fri, 7 May 2021 05:02:28 +0000

Hi John,

Thank you for the detailed review. Much appreciated. 

Changes can be tracked at: https://tinyurl.com/new-block-latest 

Please see inline. 

Cheers,
Jon & Med

> -----Message d'origine-----
> De : John Scudder via Datatracker [mailto:[email-masked]]
> Envoyé : jeudi 6 mai 2021 02:42
> À : The IESG <[email-masked]>
> Cc : [email-masked]; [email-masked];
> [email-masked]; [email-masked]; [email-masked]
> Objet : John Scudder's Discuss on draft-ietf-core-new-block-11: (with
> DISCUSS and COMMENT)
> 
> email addresses included in the To and CC lines. (Feel free to cut
> this introductory paragraph, however.)
> 
> 
> For the most part I found this document relatively easy to follow,
> considering my complete lack of background in CoAP. However, despite
> a concerted effort I have not been able to nail down with confidence
> what the intended semantics of several of your timeouts are, notably
> NON_RECEIVE_TIMEOUT. Some of the text (for example, §4.4) implies
> that the timeout is an upper bound on how long an implementation
> should wait before declaring a block to have been lost (“The client
> SHOULD wait for up to NON_RECEIVE_TIMEOUT”).

The text around the timeout values have been made absolute (as per your COMMENTS), removing "up to", etc., and once the timeout has expired, then the next activity takes place. Does this help clarify things?

 At the very least, this
> is imprecise because the timeout increases exponentially with
> repeated timeouts — but this is a relatively minor matter, discussed
> further in my comments.
> 
> Later, in §7.2, you say that expiry of the timeout is not the only
> trigger for a 4.08 response:
> 
> It makes sense to me that you use this additional trigger. At this
> point in my reading of the spec, my understanding of the
> retransmission algorithm was that a 4.08 should be sent when either a
> payload is received from a new set of MAX_PAYLOADS, or
> NON_RECEIVE_TIMEOUT expires. But then I got to the example in 10.2.3,

The text you quoted is for Q-Block1, 10.2.3 is about Q-Block2. 

> which shows the client waiting for the expiration of
> NON_RECEIVE_TIMEOUT even though it has received the first of a new
> set of MAX_PAYLOADS, and I concluded that either I’ve missed
> something basic, or the document is internally inconsistent.

The client follows this behavior:

   from this MAX_PAYLOADS set.  

> 
> As an aside, I’m also unclear as to why the only trigger you specify
> for sending a 4.08 is the arrival of the first of a new MAX_PAYLOADS

"First" is not used in reference the block order in the next block, but the first one received. I see this is confusing. Made this change: s/the first/a

> flight. Other possible triggers I noticed include a gap in the
> sequence, and reception of a payload with More=0.
> 
> Some of these issues are repeated in my comments, below — I’ve noted
> those in the comment. Possibly in addressing this DISCUSS we’ll clear
> up some of those comments too.
> 
> 
> I’m curious: is the only reason the mechanism isn’t intended for
> general usage, the fact some implementations won’t support it? Or
> does it have other deficiencies that also make it unsuitable?

This is because the general application case may require touching other parts of the block-wise (interference with caching, mixing CON/NON, etc.). Also, the congestion control guards are good enough for the target case as we do have an indication at the DOTS level if the peers are not overloaded (heartbeats) and able to negotiate the probing-rate. Such features may not be available in the generic CoAP case. 

> I found the list of codes incomprehensible on first encountering it,
> since the concept of response codes hadn’t been introduced yet. I do
> understand that the document assumes familiarity with CoAP;
> nonetheless for basic clarity I think this should say “(response
> codes 2.01, 2.02…”. Additionally, the reference to RFC 7252 §5.5
> doesn’t seem to be especially germane?

The key element in the text you quoted is "payload-bearing responses". Hence the pointer to 5.5 of RFC7252.

> 
> By the way, is 2.03 indeed a payload-bearing response? The only other
> place the spec touches on it is in §4.4, which says “the server could
> respond with a 2.03

Noted. 

> Is this paragraph really supposed to mention both Q-Block2 and Q-
> Block1?

Yes, because: 

   A CoAP endpoint (or proxy) MUST support either both or neither of the
   Q-Block1 and Q-Block2 Options.

 In particular, I’m confused by the mention of both of these
> in relation to PUT.

The server will ignore the option if it supports it, otherwise it will send an error as this is critical option. The purpose is to notify/check Q-Block is supported. 

> Presumably (hopefully) this is simply describing the behavior of
> existing spec-compliant proxies when processing the new messages. As
> such, is the MUST appropriate? I would think not.

This is only for those that are tagged as unsafe to forward. The normative language is OK. 

> “Might not” would be less ambiguous than “may not”.

Fixed. Thanks.

> You may want to skip over it until we’ve resolved the DISCUSS, after
> which this may, or may not, be relevant.)
> 
> I read this as meaning the client should wait for as little as zero,
> or as long as NON_RECEIVE_TIMEOUT — that’s my understanding of “up
> to”. Is that the intended meaning? If it is, I think it’s worth
> writing out as I’ve done, for clarity. If it’s not, it definitely
> needs to be fixed.
> 
> Referring ahead to Section 7.2 muddies the waters further. Even
> though the text quoted above says NON_RECEIVE_TIMEOUT is an upper
> limit on how long to wait,
> “Maximum”, ok great, that means “upper bound” and so lines up with
> §4.4 although the “initial” is surprising since §4.4 doesn’t say
> anything about the upper limit increasing. It continues:
> 
> But this part says it’s (a) an exact time-to-wait, not a “maximum”,
> and (b) it says it increases exponentially, so NON_RECEIVE_TIMEOUT
> isn’t a maximum at all, but a minimum.

It is an exact time to wait when there is no packets. We made these changes: 

(1)
OLD:

NEW: 
  The client SHOULD wait for NON_RECEIVE_TIMEOUT (Section 7.2)

(2)

OLD: 
   payloads) for up to NON_PARTIAL_TIMEOUT (Section 7.2).

NEW: 
   payloads) for NON_PARTIAL_TIMEOUT (Section 7.2).

> 
> This later text in §7.2 implies that perhaps the problem in the above
> passages is the word “maximum”, and it should simply be deleted:

Fair comment. We made these changes: 

(1)
OLD:
  NON_TIMEOUT is the maximum period of delay

NEW:
  NON_TIMEOUT is the period of delay

(2)
OLD:
  NON_RECEIVE_TIMEOUT is the initial maximum time to wait

NEW:
  NON_RECEIVE_TIMEOUT is the initial time to wait

> Similarly “up to” in the quote that began this comment should be “at

Fixed above by complete removal of "up to".

> 
> Whether you adopt those suggestions or not,  it seems as though all
> this needs to be rewritten with careful attention to conveying what
> the desired behavior is.
> 
> The point being that the retransmission request can be triggered by
> an event other than timer expiration. So in that sense, “maximum” is
> right — it provides an upper bound on how long to wait before
> requesting a retransmission — but in another sense it’s wrong because
> the exponential increase is applied to it. I think the word “maximum”
> is trying to do too much work, and more words are probably required
> in order to make this clear. I also think the problem is exacerbated
> by the fact both §4.4 and §7.2 are talking normatively about how to
> use NON_RECEIVE_TIMEOUT. It seems as though the main description is
> found in §7.2, and some confusion would be avoided by making §4.4
> less specific, and simply referring forward to §7.2.
> 

We hope the changes above clarify this.

> And, as noted in my DISCUSS, example 10.2.3 muddies the waters still
> further since it illustrates yet another behavior.
> 
> 7. Section 4.4
> 
> I find this whole paragraph pretty confusing with the dueling SHOULD
> and MAY, where it appears the SHOULD might be doing two jobs at once.
> I *think* your intent is something like the following?
> 
> “The client SHOULD wait as specified in Section 7.2 for NON payloads
> before requesting retransmission of any missing blocks.
> Retransmission is requested by issuing a GET, POST, PUT, FETCH,
> PATCH, or iPATCH request that contains one or more Q-Block2 Options
> that define the missing block(s). Generally the M bit on the Q-Block
> option(s) SHOULD be unset, although the M bit MAY be set to request
> this and later blocks from this MAX_PAYLOADS set, see Section 10.2.4
> for an example of this in operation.”
> 

Thanks. We went with this change: 

   For NON payloads, the client SHOULD wait NON_RECEIVE_TIMEOUT
   (Section 7.2) after the last received payload before requesting
   retransmission of any missing blocks.  Retransmission is requested by
   issuing a GET, POST, PUT, FETCH, PATCH, or iPATCH request that
   contains one or more Q-Block2 Options that define the missing
   block(s).  Generally the M bit on the Q-Block2 Option(s) SHOULD be
   unset, although the M bit MAY be set to request this and later blocks
   from this MAX_PAYLOADS set, see Section 10.2.4 for an example of this
   in operation.

> Suggestion: “then the number of missing blocks reported MUST...” (The
> thing being limited is not the actual number of missing blocks.
> You’re limiting the number you report on.)

Agree. Fixed. 

> This paragraph seems as though it’s a non-sequitur. It just doesn’t
> make sense to me. :-(
> 

Changed to:

   Following the failure to transmit a packet due to packet loss after
   MAX_TRANSMIT_SPAN time (Section 4.8.2 of [RFC7252]), whether there
   should be any further requests for missing data is implementation
   specific.

> 10. Section 7.2
> 
> (This comment relates to the difficulty explained in my DISCUSS. You
> may want to skip over it until we’ve resolved the DISCUSS, after
> which this may, or may not, be relevant.)
> 
> Presumably the use of “maximum” means it’s fine to delay zero seconds
> (or any value lower than NON_TIMEOUT).

"maximum" has been deleted as per previous comments, thus implying an exact match.

> 
> 11. General
> 
> By the way, none of the timers specify jitter (and indeed, if read
> literally, jitter would be forbidden). Is this intentional?

No +/- tolerances have been defined. When a timer expires, then the next action takes place.

> I take it this is just thrown in here as an operational suggestion?
> It’s not specifying protocol, right? It seems a little misplaced, if
> so.

This is a guidance to adjust the max_payloads to avoid losses under normal conditions.  

> 
> 13. Section 10.1.3
> 
> (This comment relates to the aside in my DISCUSS. You may want to
> skip over it until we’ve resolved the DISCUSS, after which this may,
> or may not, be
> relevant.)
> 
> Why doesn’t the server request 1,9,10 in one go? Since its rxmt
> request is triggered by rx of 11, one would think it could infer 10
> had been lost.

Because only the missing blocks of the previous set are included and there may have been some packet arrival re-ordering for 10 and 11.

   On receipt of a payload
   from the new set of MAX_PAYLOADS payloads, the server SHOULD send a
   4.08 (Request Entity Incomplete) Response Code indicating any missing
   payloads from any previous MAX_PAYLOADS payloads.  Upon receipt of
   the 4.08 (Request Entity Incomplete) Response Code, the client SHOULD
   send the missing payloads before continuing to send the remainder of
   the MAX_PAYLOADS payloads and then go into another NON_TIMEOUT delay
   prior to sending the next set of payloads.

> (This comment relates to the aside in my DISCUSS. You may want to
> skip over it until we’ve resolved the DISCUSS, after which this may,
> or may not, be
> relevant.)
> 
> Why doesn’t reception of a message with More=0 trigger the server to
> request retransmission of the missing block? Why does it have to wait
> for timeout?

This is to cover on-path reordering issues. It was decided not to introduce another timeout to be sure all the re-ordered packets had arrived. 

> 
> 15. Section 10.2.3
> 
> (This comment relates to my DISCUSS. You may want to skip over it
> until we’ve resolved the DISCUSS, after which this may, or may not,
> be relevant.)
> 
> retransmission? Why does it have to wait for timeout?

This is fixed as per a similar comment from Ben:

OLD:
       [[NON_TIMEOUT (server) delay expires]]
          |     [[Server sends next set of payloads]]
          |<---------+ NON 2.05 M:0xab T:0xf0 O:1236 ET=23 QB2:10/0/1024
          |   ...    |
       [[NON_RECEIVE_TIMEOUT (client) delay expires]]
          |     [[Client realizes blocks are missing and asks for the
          |       missing ones in one go]]
          +--------->| NON GET /path M:0x04 T:0xf3 QB2:1/0/1024\
          |          |                             QB2:9/0/1024
          |     X<---+ NON 2.05 M:0xac T:0xf3 ET=23 QB2:1/1/1024 

NEW:
       [[NON_TIMEOUT (server) delay expires]]
          |     [[Server sends next set of payloads]]
          |<---------+ NON 2.05 M:0xab T:0xf0 O:1236 ET=23 QB2:10/0/1024
          |     [[On seeing a payload from the next set of payloads, 
          |      Client realizes blocks are missing and asks for the
          |       missing ones in one go]]
          +--------->| NON GET /path M:0x04 T:0xf3 QB2:1/0/1024\
          |          |                             QB2:9/0/1024
          |     X<---+ NON 2.05 M:0xac T:0xf3 ET=23 QB2:1/1/1024

 Similarly
> reception of

There has to be a delay here because of the potential of packet re-ordering arrival.

> 16. Section 10.2.4
> 
> Since MAX_PAYLOADS is 10, why does the example say “MAX_PAYLOADS has
> been reached” after payloads 2-9 have been retransmitted? That’s only
> 8 payloads.
> 

MAX_PAYLOADS is not a sliding window. It is a specific set of blocks, blocks #0 - #9 (payloads 1 to 10) in this case, then blocks #10 - #19 and so on. 

_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

---

From: John Scudder <[email-masked]>
Date: Wed, 19 May 2021 21:37:10 +0000

Hi Med, Jon,

My responses in-line.

> On May 7, 2021, at 1:02 AM, [email-masked] wrote:
> 
> [External Email. Be cautious of content]
> 
> 
> Hi John,
> 
> Changes can be tracked at: https://urldefense.com/v3/__https://tinyurl.com/new-block-latest__;!!NEt6yMaO-gk!URwuIUILv1-nibwOiR7nXAxQ0eWbNfKiJLAffIY9XW4eeoYdtLiQqTlOWTPvXA$
> 
> Please see inline.
> 
> Cheers,
> Jon & Med
> 

[…]

> The text around the timeout values have been made absolute (as per your COMMENTS), removing "up to", etc., and once the timeout has expired, then the next activity takes place. Does this help clari…

Yes, those changes, plus the other rewrites to §7.2, help a lot. This DISCUSS question is resolved as far as I’m concerned.

I would clear the DISCUSS, but I am concerned about my comment #11 (see below) and I’m going to raise that to a DISCUSS.

[…]

>> ——————————————————————————————————

[…]

> The key element in the text you quoted is "payload-bearing responses". Hence the pointer to 5.5 of RFC7252.

OK. I still think the text would be better if you inserted “response codes” within the parentheses, as in “(response codes 2.01, 2.01, …” instead of the current “(2.01, 2.02, …”.

[…]

> This is only for those that are tagged as unsafe to forward. The normative language is OK.

My point here is that the second sentence (the one that begins “that is”) is simply stating the behavior you would expect an existing proxy to exhibit if it were presented with the “unsafe to forward” options. Correct?

If so, then I think the MUST is inappropriate, since you are not specifying new behavior here.

[…]

> No +/- tolerances have been defined. When a timer expires, then the next action takes place.

I notice that RFC 7252 jitters its timers, for example:

   counter.  For a new Confirmable message, the initial timeout is set
   to a random duration (often not an integral number of seconds)
   between ACK_TIMEOUT and (ACK_TIMEOUT * ACK_RANDOM_FACTOR) (see
   Section 4.8)
…
   ACK_RANDOM_FACTOR MUST NOT be decreased below 1.0, and it SHOULD have
   a value that is sufficiently different from 1.0 to provide some
   protection from synchronization effects.

MAX_TRANSMIT_SPAN and MAX_TRANSMIT_WAIT are similarly jittered. A number of your introduced parameters 

   This document introduces new parameters MAX_PAYLOADS, NON_TIMEOUT,
   NON_RECEIVE_TIMEOUT, NON_MAX_RETRANSMIT, NON_PROBING_WAIT, and
   NON_PARTIAL_TIMEOUT primarily for use with NON (Table 3).

appear at least superficially similar to the timers the authors of RFC 7252 deemed important to jitter to prevent synchronization effects. Did you specifically consider jittering them, and decide that jitter was unnecessary? If so, can you explain what is different about your specification, compared to the base spec, that eliminates the concern?

> This is a guidance to adjust the max_payloads to avoid losses under normal conditions.

OK, but you haven’t said whether this is operational guidance (i.e., you expect this to be done by humans adjusting configuration parameters) or if it’s potentially to be done automatically the implementation. Given that "How the two peer values for MAX_PAYLOADS are synchronized is out of the scope” I would think it’s strictly operational guidance. If that’s correct and it’s operational guidance exclusively, I think it should not be part of §7.2. Section 7.2 specifies protocol; mixing in operational advice potentially confuses the reader. You could add an “Operational Advice for Tuning Parameters” major section, or even a subsection below 7.2, and the concern would be resolved.

By the way, nit:

   not have arrived.  How the two peer values for MAX_PAYLOADS are	
   synchronized is out of the scope.

“Out of the scope” -> “out of scope” or “out of the scope of this document"

> Because only the missing blocks of the previous set are included and there may have been some packet arrival re-ordering for 10 and 11.

I see. I guess I was thinking of this in terms of a sliding-window protocol, but your protocol is really still almost stop-and-wait, but with much larger blocks. Your rewrite helped a lot with this, thanks.

[…]

>          |     X<---+ NON 2.05 M:0xac T:0xf3 ET=23 QB2:1/1/1024

I presume you’re not concerned about re-ordering between payload 9 and 10 because NON_TIMEOUT is always guaranteed to be inserted between flights of payloads (unless the other side has sent a 2.31 Continue, which means nothing was lost), therefore there’s a delay inserted on the server side between 9 and 10.

One concern related to that: waiting NON_TIMEOUT isn’t actually required, it’s only RECOMMENDED, therefore this isn’t actually a guarantee. From §7.2:

   As the sending of many payloads of a single body may itself cause
   congestion, it is RECOMMENDED that after transmission of every
   MAX_PAYLOADS_SET of a single body, a delay is introduced of
   NON_TIMEOUT before sending the next MAX_PAYLOADS_SET to manage
   potential congestion issues.

I am curious why you made this a RECOMMENDED instead of a MUST. In a situation like this it would be preferable for you to explain to the implementor what situation they can ignore the RECOMMENDED in and what they should do instead, or of course to make it into a MUST.

Now that I’m looking at RECOMMENDED and SHOULD again, I notice that the specification for use of 2.31 in §4.3 has a similar potential issue:

      A response using this Response Code SHOULD NOT be generated for
      every received Q-Block1 Option request (Section 7.2).  It SHOULD
      only be generated when all the payload requests are Non-
      confirmable and a MAX_PAYLOADS_SET has been received by the
      server.  More details about the motivations for this optimization
      are discussed in Section 7.2.

I can imagine when an implementation might ignore the first SHOULD NOT: for example if it’s a very small implementation that doesn’t want to keep extra state. But how about the SHOULD? Under what circumstances is it OK for an implementation to use Response Code 2.31 when not all the payloads in a set have been received? If there are no such circumstances, this ought to be a MUST (or turn it around and say “MUST NOT be generated unless…”).

A similar concern exists for the specification of response 4.08.

I quickly skimmed the rest of the document for SHOULD/RECOMMENDED and didn’t find others that seemed equally problematic; however I’m of the school of thought that says you should ask yourself about every SHOULD, “why isn’t this a MUST?” and if you don’t have a good answer, change it.

[…]

> MAX_PAYLOADS is not a sliding window. It is a specific set of blocks, blocks #0 - #9 (payloads 1 to 10) in this case, then blocks #10 - #19 and so on.

More or less a meta-block, as I mention in my comment on #13, above. OK.

I have some new questions and comments related to your new revision:

17. Section 1:

   This document introduces the CoAP Q-Block1 and Q-Block2 Options which
   allow block-wise transfer to work with series of Non-confirmable
   messages, instead of lock-stepping using Confirmable messages
   (Section 3).  In other words, this document provides a missing piece
   of [RFC7959], namely the support of block-wise transfer using Non-
   confirmable where an entire body of data can be transmitted without
   the requirement for an acknowledgement (but recovery is available
   should it be needed).

As far as I can tell the spec does not really remove the requirement for acknowledgement, it just amortizes the acknowledgements by only sending them every MAX_PAYLOADS_SET. Response Code 2.31 is essentially an acknowledgement, and it gets sent that frequently, right? There’s also (if I recall correctly) some flavor of acknowledgement that is sent when the entire body has been transferred. So, I think the new paragraph isn’t accurate.

This observation also applies to this claimed benefit in §3:

   o  They support sending an entire body using NON messages without
      requiring an intermediate response from the peer.

Response Code 2.31 is exactly an intermediate response. I guess maybe your focus is that if the intermediate response isn’t received, transmission continues, albeit more slowly than it would otherwise, and unreliably too, so in that sense the responses aren’t “required”. I think this requires awfully close parsing of the word “required”, though.

18. Section 2:

   MAX_PAYLOADS_SET is the set of blocks identified by block numbers
   that, when divided by MAX_PAYLOADS, they have the same numeric

Remove “they” 

   result.  For example, if MAX_PAYLOADS is set to '10', a
   MAX_PAYLOADS_SET could be blocks #0 to #9, #10 to #19, etc.
   Depending on the data size, the MAX_PAYLOADS_SET may not comprise all
   the MAX_PAYLOADS blocks.

I don’t understand the last sentence ("Depending on the data size, the MAX_PAYLOADS_SET may not comprise all the MAX_PAYLOADS blocks.”) Are you trying to say that if the body size isn’t evenly divisible by MAX_PAYLOADS then the final MAX_PAYLOADS_SET will have fewer than MAX_PAYLOADS blocks in it?

(I do think this change, to introduce the term MAX_PAYLOADS_SET, is generally helpful; thanks.)

Thanks,

—John

---

From: <[email-masked]>
Date: Thu, 20 May 2021 13:17:31 +0000

Hi John, 

Please see inline. 

Cheers,
Jon & Med

> De : John Scudder [mailto:[email-masked]]
> Envoyé : mercredi 19 mai 2021 23:37
> À : BOUCADAIR Mohamed TGI/OLN <[email-masked]>
> Cc : The IESG <[email-masked]>; [email-masked];
> [email-masked]; [email-masked]; [email-masked]
> Objet : Re: John Scudder's Discuss on draft-ietf-core-new-block-11:
> (with DISCUSS and COMMENT)
> 
> Hi Med, Jon,
> 
> My responses in-line.
> 
[...]
> >> De : John Scudder via Datatracker [mailto:[email-masked]] Envoyé
> :
> >> jeudi 6 mai 2021 02:42 À : The IESG <[email-masked]> Cc :
> >> [email-masked]; [email-masked];
> >> [email-masked]; [email-masked]; [email-masked] Objet : John
> >> Scudder's Discuss on draft-ietf-core-new-block-11: (with DISCUSS
> and
> >> COMMENT)
> 
> […]
> 
> >> ------------------------------------------------------------------
> ---
> >> -
> >> DISCUSS:
> >> ------------------------------------------------------------------
> ---
> >> -
> >>
> > The text around the timeout values have been made absolute (as per
> your COMMENTS), removing "up to", etc., and once the timeout has
> expired, then the next activity takes place. Does this help clarify
> things?
> 
> Yes, those changes, plus the other rewrites to §7.2, help a lot. This
> DISCUSS question is resolved as far as I’m concerned.
> 
> I would clear the DISCUSS, but I am concerned about my comment #11
> (see below) and I’m going to raise that to a DISCUSS.
> 

Thanks.

> […]
> 
> >> ------------------------------------------------------------------
> ---
> >> -
> >> COMMENT:
> > The key element in the text you quoted is "payload-bearing
> responses". Hence the pointer to 5.5 of RFC7252.
> 
> OK. I still think the text would be better if you inserted “response
> codes” within the parentheses, as in “(response codes 2.01, 2.01, …”
> instead of the current “(2.01, 2.02, …”.
> 

OK. Fixed.

> […]
> 
> >> 4. Section 4.1
> >>
> >>   The Q-Block1 and Q-Block2 Options are unsafe to forward.  That
> is,
> >> a
> My point here is that the second sentence (the one that begins “that
> is”) is simply stating the behavior you would expect an existing
> proxy to exhibit if it were presented with the “unsafe to forward”
> options. Correct?
> 

Changed the text and added a pointer to Section 5.7.1 of RFC7252.

> appear at least superficially similar to the timers the authors of
> RFC 7252 deemed important to jitter to prevent synchronization
> effects. Did you specifically consider jittering them, and decide
> that jitter was unnecessary? If so, can you explain what is different
> about your specification, compared to the base spec, that eliminates
> the concern?

RFC7252 introduces ACK_RANDOM_FACTOR jitter and separately jitter for multicast responses (which is not relevant here).

The ACK_RANDOM_FACTOR is there for when re-transmitting a packet that has not been acknowledged for some reason by its peer. NON_TIMEOUT is for when the next MAX_PAYLOADS_SET can start transmission (not re-transmission) assuming a 'Continue' has not arrived in the interim, and so was not thought necessary to add in ACK_RANDOM_FACTOR style jitter here.

For NON_RECEIVE_TIMEOUT, what is important is that NON_RECEIVE_TIMEOUT is greater than NON_TIMEOUT (We say in the spec a minimum of one second) so that a peer does not fire off a re-transmission request before the local agent has a chance to start to transmit the next MAX_PAYLOADS_SET.  NON_RECEIVE_TIMEOUT is exponentially scaled for each retry to make sure that stability is preserved. So, again, ACK_RANDOM_FACTOR jitter was not thought to be necessary here.

NON_MAX_RETRANSMIT is a fixed count.

NON_PROBING_WAIT is used to put a limit on the potential delay that could incur when obeying PROBING_WAIT when there is no peer response. If the implementation goes with the default EXCHANGE_LIFETIME computation, then NON_PROBING_WAIT includes ACK_RANDOM_FACTOR in the math.

NON_PARTIAL_TIMEOUT if computed using the default EXCHANGE_LIFETIME includes ACK_RANDOM_FACTOR.

> >>   the situation re-evaluated for another 24 hour period until
> there
> >> is
> >>   newly derived value for MAX_PAYLOADS should be used for both
> ends
> >> of
> >>   consequence of the two peers having different MAX_PAYLOADS
> values,
> >> a
> OK, but you haven’t said whether this is operational guidance (i.e.,
> you expect this to be done by humans adjusting configuration
> parameters) or if it’s potentially to be done automatically the
> implementation. Given that "How the two peer values for MAX_PAYLOADS
> are synchronized is out of the scope” I would think it’s strictly
> operational guidance. If that’s correct and it’s operational guidance
> exclusively, I think it should not be part of §7.2. Section 7.2
> specifies protocol; mixing in operational advice potentially confuses
> the reader. You could add an “Operational Advice for Tuning
> Parameters” major section, or even a subsection below 7.2, and the
> concern would be resolved.

Please note that we removed this text as per the discussion with Ben.

> > Because only the missing blocks of the previous set are included
> and there may have been some packet arrival re-ordering for 10 and
> 11.
> 
> I see. I guess I was thinking of this in terms of a sliding-window
> protocol, but your protocol is really still almost stop-and-wait, but
> with much larger blocks. Your rewrite helped a lot with this, thanks.
> 
> […]
> 
> >> 15. Section 10.2.3
> >>
> I presume you’re not concerned about re-ordering between payload 9
> and 10 because NON_TIMEOUT is always guaranteed to be inserted
> between flights of payloads (unless the other side has sent a 2.31
> Continue, which means nothing was lost), therefore there’s a delay
> inserted on the server side between 9 and 10.
> 
> One concern related to that: waiting NON_TIMEOUT isn’t actually
> required, it’s only RECOMMENDED, therefore this isn’t actually a
> guarantee. From §7.2:
> 
> I am curious why you made this a RECOMMENDED instead of a MUST. In a
> situation like this it would be preferable for you to explain to the
> implementor what situation they can ignore the RECOMMENDED in and
> what they should do instead, or of course to make it into a MUST.

Because a continue signal may be received from the peer and then continue without waiting for the timeout to expire. 

This is to be linked with this text: 

      A response using this Response Code SHOULD NOT be generated for
      every received Q-Block1 Option request (Section 7.2).  It SHOULD
      only be generated when all the payload requests are Non-
      confirmable and a MAX_PAYLOADS_SET has been received by the
       server.  More details about the motivations for this optimization
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 
      are discussed in Section 7.2.
      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^

We could use **MUST unless a 'Continue' is received**, e.g.,

OLD:

NEW:
   As the sending of many payloads of a single body may itself cause
   congestion, after transmission of every MAX_PAYLOADS_SET of a single
   body, a delay MUST be introduced of NON_TIMEOUT before sending the
   next MAX_PAYLOADS_SET to manage potential congestion issues.
   However, if a 'Continue' is received from the peer for the current
   MAX_PAYLOADS_SET, then the next MAX_PAYLOADS_SET can start
   transmission immediately.

... but I know that many would argue this is a SHOULD. 

> 
> Now that I’m looking at RECOMMENDED and SHOULD again, I notice that
> the specification for use of 2.31 in §4.3 has a similar potential
> issue:
> 
> I can imagine when an implementation might ignore the first SHOULD
> NOT: for example if it’s a very small implementation that doesn’t
> want to keep extra state.

FYI, we changed the first "SHOULD NOT" to "MUST NOT" as we removed auto-scaling (down to 1).

 But how about the SHOULD? Under what
> circumstances is it OK for an implementation to use Response Code
> 2.31 when not all the payloads in a set have been received?

An example would be: If the peer is overloaded/can't process more blocks, it won't send the continue signal.  

 If there
> are no such circumstances, this ought to be a MUST (or turn it around
> and say “MUST NOT be generated unless…”).

The procedure does not mandate the continue signal to be received, but it allows for it as an optimization to shorten the delay between the sets. Hence the current wording.

> I quickly skimmed the rest of the document for SHOULD/RECOMMENDED and
> didn’t find others that seemed equally problematic; however I’m of
> the school of thought that says you should ask yourself about every
> SHOULD, “why isn’t this a MUST?” and if you don’t have a good answer,
> change it.
> 
> […]
> 
> >> 16. Section 10.2.4
> >>
> > MAX_PAYLOADS is not a sliding window. It is a specific set of
> blocks, blocks #0 - #9 (payloads 1 to 10) in this case, then blocks
> #10 - #19 and so on.
> 
> As far as I can tell the spec does not really remove the requirement
> for acknowledgement,

These are not required. They were added as an optimization to avoid the non-timeout if the peer decides to use it.

 it just amortizes the acknowledgements by only
> sending them every MAX_PAYLOADS_SET. Response Code 2.31 is
> essentially an acknowledgement, and it gets sent that frequently,
> right? There’s also (if I recall correctly) some flavor of
> acknowledgement that is sent when the entire body has been
> transferred. So, I think the new paragraph isn’t accurate.
> 
> Response Code 2.31 is exactly an intermediate response. I guess maybe
> your focus is that if the intermediate response isn’t received,
> transmission continues, albeit more slowly than it would otherwise,
> and unreliably too, so in that sense the responses aren’t “required”.
> I think this requires awfully close parsing of the word “required”,
> though.
> 
> 18. Section 2:
> 

Fixed. Thanks. 

> I don’t understand the last sentence ("Depending on the data size,
> the MAX_PAYLOADS_SET may not comprise all the MAX_PAYLOADS blocks.”)
> Are you trying to say that if the body size isn’t evenly divisible by
> MAX_PAYLOADS then the final MAX_PAYLOADS_SET will have fewer than
> MAX_PAYLOADS blocks in it?

We meant that the last set may include fewer blocks than MAX_PAYLOADS. Changed to:

 " Depending on the overall data
                    ^^^^^^^^ 
   size, the final MAX_PAYLOADS_SET may not comprise all the
             ^^^^^
   MAX_PAYLOADS blocks. "

Better?

---

From: John Scudder <[email-masked]>
Date: Thu, 20 May 2021 22:03:22 +0000

Hi Mohamed,

I think we are converging. My comments in line. I’ve snipped agreed points for brevity, indicated by […]. 

> On May 20, 2021, at 9:17 AM, [email-masked] wrote:

[…]

> NON_PARTIAL_TIMEOUT if computed using the default EXCHANGE_LIFETIME includes ACK_RANDOM_FACTOR.

Thanks for taking the time to explain. You don’t comment regarding whether NON_PROBING_WAIT and NON_PARTIAL_TIMEOUT should be jittered or not, you just explain that if they use the default they get jitter “for free”. The missing detail is that if they don’t use the default they don’t get jittered, so I think some consideration is still called for regarding whether having them not be jittered is OK.

[…]

[…]

>       server.  More details about the motivations for this optimization

I would be OK with either your proposed new text, or a SHOULD/MAY pair as in 

NEW:
  As the sending of many payloads of a single body may itself cause
  congestion, after transmission of every MAX_PAYLOADS_SET of a single
  body, a delay SHOULD be introduced of NON_TIMEOUT before sending the
  next MAX_PAYLOADS_SET to manage potential congestion issues.
  However, if a 'Continue' is received from the peer for the current
  MAX_PAYLOADS_SET, then the next MAX_PAYLOADS_SET MAY start
  transmission immediately.

If you want to stick with MUST I think you can clear up the pain with something like

NEW:
  As the sending of many payloads of a single body may itself cause
  congestion, after transmission of every MAX_PAYLOADS_SET of a single
  body, a delay MUST be introduced of NON_TIMEOUT before sending the
  next MAX_PAYLOADS_SET unless a 'Continue' is received from the peer 
  for the current MAX_PAYLOADS_SET, in which case the next 
  MAX_PAYLOADS_SET MAY start transmission immediately.

(To my eye presenting the option in this way makes it clear when the MUST does, and doesn’t, apply. This is my preferred form but I don’t insist.)

[…]

> These are not required. They were added as an optimization to avoid the non-timeout if the peer decides to use it.

As I mentioned below (“awfully close parsing”), I think that although you can find some justification for this reading, it’s debatable. Transmission of the acknowledgement (at least the final acknowledgement of the entire body, in the form of a Response Code) is required, is it not? Reception isn’t required though. Without the verb, I’m not sure whether I can say whether acknowledgement is, or isn’t, required.

I don’t insist that you change this, but I do think you could improve the clarity of the document, if you edited the above to read “… without the requirement that an acknowledgment be received from the peer"

>>      requiring an intermediate response from the peer.

And similarly, “… without requiring that an intermediate response be received from the peer.”

[…]

>   size, the final MAX_PAYLOADS_SET may not comprise all the

Improving. The word “comprise” is prone to misinterpretation in my experience, I would suggest something like “… there could be fewer than MAX_PAYLOADS blocks in the final MAX_PAYLOADS_SET.”

Thanks,

—John

---

From: <[email-masked]>
Date: Fri, 21 May 2021 12:41:57 +0000

Hi John,

As you can see in https://tinyurl.com/new-block-latest, we went with the following changes to better address your latest comment on the jitter:

(1) be explicit about the formula used for default values: 

OLD: 
   NON_PROBING_WAIT is used to limit the potential wait needed when
   using PROBING_RATE.  By default, NON_PROBING_WAIT has the same value
   as EXCHANGE_LIFETIME (Section 4.8.2 of [RFC7252]).

   NON_PARTIAL_TIMEOUT is used for expiring partially received bodies.
   By default, NON_PARTIAL_TIMEOUT has the same value as
   EXCHANGE_LIFETIME (Section 4.8.2 of [RFC7252]).

NEW:
   NON_PROBING_WAIT is used to limit the potential wait needed when
   using PROBING_RATE.  By default, NON_PROBING_WAIT is computed in the
   same way as EXCHANGE_LIFETIME (Section 4.8.2 of [RFC7252]) but with
   ACK_TIMEOUT and MAX_RETRANSMIT substituted with NON_TIMEOUT and
   NON_MAX_RETRANSMIT, respectively:

      NON_PROBING_WAIT = NON_TIMEOUT * ((2 ** NON_MAX_RETRANSMIT) - 1) *
      ACK_RANDOM_FACTOR + (2 * MAX_LATENCY) + NON_TIMEOUT

   NON_PARTIAL_TIMEOUT is used for expiring partially received bodies.
   By default, NON_PARTIAL_TIMEOUT is computed in the same way as
   EXCHANGE_LIFETIME (Section 4.8.2 of [RFC7252]).  This default value
   is calculated in the same way as NON_PROBING_WAIT.

(2) We don’t see a need to apply a jitter when a value is explicitly configured (these are expected to be large numbers, we don't care that much on the jitter). We added this text to be clear about the intent, and which BTW reflects the current implementation:  

NEW:
      When explicit values are
      configured for NON_PROBING_WAIT and NON_PARTIAL_TIMEOUT, these
      values are used without applying any jitter.

We also adopted your proposed edits in the message below. 

Thank you again for the careful review. This is highly appreciated. 

Cheers,
John & Med

> Envoyé : vendredi 21 mai 2021 00:03
> >>> No +/- tolerances have been defined. When a timer expires, then
> the
> >> next action takes place.
> >>
> > RFC7252 introduces ACK_RANDOM_FACTOR jitter and separately jitter
> for multicast responses (which is not relevant here).
> >
> > The ACK_RANDOM_FACTOR is there for when re-transmitting a packet
> that has not been acknowledged for some reason by its peer.
> NON_TIMEOUT is for when the next MAX_PAYLOADS_SET can start
> transmission (not re-transmission) assuming a 'Continue' has not
> arrived in the interim, and so was not thought necessary to add in
> ACK_RANDOM_FACTOR style jitter here.
> >
> > For NON_RECEIVE_TIMEOUT, what is important is that
> NON_RECEIVE_TIMEOUT is greater than NON_TIMEOUT (We say in the spec a
> minimum of one second) so that a peer does not fire off a re-
> transmission request before the local agent has a chance to start to
> transmit the next MAX_PAYLOADS_SET.  NON_RECEIVE_TIMEOUT is
> exponentially scaled for each retry to make sure that stability is
> preserved. So, again, ACK_RANDOM_FACTOR jitter was not thought to be
> necessary here.
> >
> > NON_PROBING_WAIT is used to put a limit on the potential delay that
> could incur when obeying PROBING_WAIT when there is no peer response.
> If the implementation goes with the default EXCHANGE_LIFETIME
> computation, then NON_PROBING_WAIT includes ACK_RANDOM_FACTOR in the
> math.
> >
> Thanks for taking the time to explain. You don’t comment regarding
> whether NON_PROBING_WAIT and NON_PARTIAL_TIMEOUT should be jittered
> or not, you just explain that if they use the default they get jitter
> “for free”. The missing detail is that if they don’t use the default
> they don’t get jittered, so I think some consideration is still
> called for regarding whether having them not be jittered is OK.
> 
> […]
> 
> >>>> 15. Section 10.2.3
> 
> […]
> 
> > Because a continue signal may be received from the peer and then
> continue without waiting for the timeout to expire.
> >
> (To my eye presenting the option in this way makes it clear when the
> MUST does, and doesn’t, apply. This is my preferred form but I don’t
> insist.)
> 
> […]
> 
> >> 17. Section 1:
> >>
> > These are not required. They were added as an optimization to avoid
> the non-timeout if the peer decides to use it.
> 
> As I mentioned below (“awfully close parsing”), I think that although
> you can find some justification for this reading, it’s debatable.
> Transmission of the acknowledgement (at least the final
> acknowledgement of the entire body, in the form of a Response Code)
> is required, is it not? Reception isn’t required though. Without the
> verb, I’m not sure whether I can say whether acknowledgement is, or
> isn’t, required.
> 
> I don’t insist that you change this, but I do think you could improve
> the clarity of the document, if you edited the above to read “…
> without the requirement that an acknowledgment be received from the
> peer"
> 
> >> it just amortizes the acknowledgements by only sending them every
> >> MAX_PAYLOADS_SET. Response Code 2.31 is essentially an
> >> acknowledgement, and it gets sent that frequently, right? There’s
> >> also (if I recall correctly) some flavor of acknowledgement that
> is
> >> sent when the entire body has been transferred. So, I think the
> new
> >> paragraph isn’t accurate.
> >>
> >>   Depending on the data size, the MAX_PAYLOADS_SET may not
> comprise
> >> all
> >>   the MAX_PAYLOADS blocks.
> >>
> Improving. The word “comprise” is prone to misinterpretation in my
> experience, I would suggest something like “… there could be fewer
> than MAX_PAYLOADS blocks in the final MAX_PAYLOADS_SET.”
> 
> Thanks,
> 
> —John

---

From: John Scudder <[email-masked]>
Date: Fri, 21 May 2021 16:16:47 +0000

Hi Med,

Your current working copy looks good. I’ve cleared my discuss.

—John

> On May 21, 2021, at 8:41 AM, [email-masked] wrote:
> 
> As you can see in https://urldefense.com/v3/__https://tinyurl.com/new-block-latest__;!!NEt6yMaO-gk!RlDk7GMcbxlFsT2QGl8ma04s1CggmQZQHcIP9aw2R2EP1rWkfQDAbIpzOgyR6g$ , we went with the following changes to better address your latest comment on the jitter:
>

---

From: Martin Duke <[email-masked]>
Date: Fri, 21 May 2021 09:28:22 -0700

So I'm not sure that "large numbers" are a sufficient reason to not worry
about jitter.

If two hosts simultaneously transmit on a quiet network, and cause losses
with each other. They both set the same retransmission timeout, and in
spite of no other traffic around cause the same collision, etc.

If an explicit configuration doesn't result in common round numbers, it's
an OK substitute, but I don't see any encouragement of that choice.

On Fri, May 21, 2021 at 9:17 AM John Scudder <jgs=
[email-masked]> wrote:

> > As you can see in
> https://urldefense.com/v3/__https://tinyurl.com/new-block-latest__;!!NEt6yMaO-gk!RlDk7GMcbxlFsT2QGl8ma04s1CggmQZQHcIP9aw2R2EP1rWkfQDAbIpzOgyR6g$
> , we went with the following changes to better address your latest comment
> on the jitter:
> >
> > (2) We don’t see a need to apply a jitter when a value is explicitly
> configured (these are expected to be large numbers, we don't care that much
> on the jitter). We added this text to be clear about the intent, and which
> BTW reflects the current implementation:
> >
> > NEW:
> >      When explicit values are
> > Ce message et ses pieces jointes peuvent contenir des informations
> confidentielles ou privilegiees et ne doivent donc
> > pas etre diffuses, exploites ou copies sans autorisation. Si vous avez
> recu ce message par erreur, veuillez le signaler
> > a l'expediteur et le detruire ainsi que les pieces jointes. Les messages
> electroniques etant susceptibles d'alteration,
> > This message and its attachments may contain confidential or privileged
> information that may be protected by law;

---

From: <[email-masked]>
Date: Fri, 21 May 2021 17:27:08 +0000

Hi Martin,

The comment about large numbers should be put in the context of the two timeouts discussed with John: NON_PARTIAL_TIMEOUT and NON_PROBING_WAIT. Consider the example of NON_PARTIAL_TIMEOUT, adding some jitter to it is unlikely to have an effect as that timer is about controlling when a received partial body will be discarded locally.

NON_PROBING_WAIT is about limiting the effect of PROBING_RATE when a peer does not reply. As a reminder, probing rate is defined as follows:

   The PROBING_RATE parameter in CoAP indicates the average data rate
   that must not be exceeded by a CoAP endpoint in sending to a peer
   endpoint that does not respond.

Cheers,
Med

De : Martin Duke [mailto:[email-masked]]
Envoyé : vendredi 21 mai 2021 18:28
À : John Scudder <jgs=[email-masked]>
Cc : BOUCADAIR Mohamed TGI/OLN <[email-masked]>; [email-masked]; [email-masked]; The IESG <[email-masked]>; [email-masked]; [email-masked]
Objet : Re: John Scudder's Discuss on draft-ietf-core-new-block-11: (with DISCUSS and COMMENT)

So I'm not sure that "large numbers" are a sufficient reason to not worry about jitter.

If two hosts simultaneously transmit on a quiet network, and cause losses with each other. They both set the same retransmission timeout, and in spite of no other traffic around cause the same collision, etc.

If an explicit configuration doesn't result in common round numbers, it's an OK substitute, but I don't see any encouragement of that choice.

On Fri, May 21, 2021 at 9:17 AM John Scudder <jgs=[email-masked]<mailto:[email-masked]>> wrote:
Hi Med,

—John

> On May 21, 2021, at 8:41 AM, [email-masked]<mailto:[email-masked]> wrote:
>
> As you can see in https://urldefense.com/v3/__https://tinyurl.com/new-block-latest__;!!NEt6yMaO-gk!RlDk7GMcbxlFsT2QGl8ma04s1CggmQZQHcIP9aw2R2EP1rWkfQDAbIpzOgyR6g$<https://urldefense.com/v3/__https:/tinyurl.com/new-block-latest__;!!NEt6yMaO-gk!RlDk7GMcbxlFsT2QGl8ma04s1CggmQZQHcIP9aw2R2EP1rWkfQDAbIpzOgyR6g$> , we went with the following changes to better address your latest comment on the jitter:
>
>> De : John Scudder [mailto:[email-masked]<mailto:[email-masked]>]
>> À : BOUCADAIR Mohamed TGI/OLN <[email-masked]<mailto:[email-masked]>>
>> Cc : The IESG <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>;
>> [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
>>> On May 20, 2021, at 9:17 AM, [email-masked]<mailto:[email-masked]> wrote:
>>
>> […]
>>
>>>>>> 11. General
>>>>>>

---

From: Martin Duke <[email-masked]>
Date: Fri, 21 May 2021 11:38:35 -0700

I agree that those two parameters have no need for jitter.

However, NON_TIMEOUT and NON_RECEIVE_TIMEOUT would appear to, but derive
from 7252 formulae that don't have the jitter included.

On Fri, May 21, 2021 at 10:27 AM <[email-masked]> wrote:

> Hi Martin,
>
>
>
> The comment about large numbers should be put in the context of the two
> timeouts discussed with John: NON_PARTIAL_TIMEOUT and NON_PROBING_WAIT.
> Consider the example of NON_PARTIAL_TIMEOUT, adding some jitter to it is
> unlikely to have an effect as that timer is about controlling when a
> received partial body will be discarded locally.
>
>
>
> NON_PROBING_WAIT is about limiting the effect of PROBING_RATE when a peer
> does not reply. As a reminder, probing rate is defined as follows:
>
>
>
> *De :* Martin Duke [mailto:[email-masked]]
> *Envoyé :* vendredi 21 mai 2021 18:28
> *À :* John Scudder <jgs=[email-masked]>
> *Cc :* BOUCADAIR Mohamed TGI/OLN <[email-masked]>;
> [email-masked]; [email-masked]; The IESG <
> [email-masked]>; [email-masked]; [email-masked]
> *Objet :* Re: John Scudder's Discuss on draft-ietf-core-new-block-11:
> <https://urldefense.com/v3/__https:/tinyurl.com/new-block-latest__;!!NEt6yMaO-gk!RlDk7GMcbxlFsT2QGl8ma04s1CggmQZQHcIP9aw2R2EP1rWkfQDAbIpzOgyR6g$>

---

From: <[email-masked]>
Date: Fri, 21 May 2021 20:16:10 +0100

Hi Martin,

NON_TIMEOUT has the same default value as ACK_TIMEOUT (Section 7.2 of the draft).  RFC7252 Section 4.2 “Messages Transmitted Reliably” has

   For a new Confirmable message, the initial timeout is set

   between ACK_TIMEOUT and (ACK_TIMEOUT * ACK_RANDOM_FACTOR)

As mentioned previously in this thread

“

“

Does that help?

Regards

Jon

From: Martin Duke [mailto: [email-masked]] 
Sent: 21 May 2021 19:39
To: [email-masked]
Cc: John Scudder; [email-masked]; [email-masked]; The IESG; [email-masked]; [email-masked]
Subject: Re: John Scudder's Discuss on draft-ietf-core-new-block-11: (with DISCUSS and COMMENT)

However, NON_TIMEOUT and NON_RECEIVE_TIMEOUT would appear to, but derive from 7252 formulae that don't have the jitter included.

Hi Martin, 

Cheers,

Med

On Fri, May 21, 2021 at 9:17 AM John Scudder <jgs=[email-masked]> wrote:

Hi Med,

—John

> As you can see in https://urldefense.com/v3/__https://tinyurl.com/new-block-latest__;!!NEt6yMaO-gk!RlDk7GMcbxlFsT2QGl8ma04s1CggmQZQHcIP9aw2R2EP1rWkfQDAbIpzOgyR6g$ <https://urldefense.com/v3/__https:/tinyurl.com/new-block-latest__;!!NEt6yMaO-gk!RlDk7GMcbxlFsT2QGl8ma04s1CggmQZQHcIP9aw2R2EP1rWkfQDAbIpzOgyR6g$>  , we went with the following changes to better address your latest comment on the jitter:
>

---

From: Martin Duke <[email-masked]>
Date: Fri, 21 May 2021 15:03:22 -0700

Thanks for humoring me as I try to verify that there isn't a problem here!

I think we've collapsed the potential problem to NON_TIMEOUT and
NON_RECEIVE_TIMEOUT.

Some scenarios:
(1) Let's say two senders simultaneously send MAX_PAYLOAD bursts over a
radio medium that causes them to interfere with each other. With no
response from the receiver at all, they both wait for NON_TIMEOUT and send
another burst, causing the same effect.

Can this not happen?

(2) Two receivers send bursts that arrive at a bottleneck at the same time,
causing congestion losses

The receivers will request missing blocks, wait NON_RECEIVE_TIMEOUT, and
trigger retransmission. For these to also collide, that would imply that
(1) the RTTs are similar, and (2) the timing of the first missing packet
signal is roughly simultaneous.

Is that correct?

Martin

On Fri, May 21, 2021 at 12:17 PM <[email-masked]> wrote:

> Hi Martin,
>
>
>
> NON_TIMEOUT has the same default value as ACK_TIMEOUT (Section 7.2 of the
> draft).  RFC7252 Section 4.2 “Messages Transmitted Reliably” has
>
>
>
> RFC7252 introduces ACK_RANDOM_FACTOR jitter and separately jitter for
> multicast responses (which is not relevant here).
>
>
>
> The ACK_RANDOM_FACTOR is there for when re-transmitting a packet that has
> not been acknowledged for some reason by its peer. NON_TIMEOUT is for when
> the next MAX_PAYLOADS_SET can start transmission (not re-transmission)
> assuming a 'Continue' has not arrived in the interim, and so was not
> thought necessary to add in ACK_RANDOM_FACTOR style jitter here.
>
>
>
> For NON_RECEIVE_TIMEOUT, what is important is that NON_RECEIVE_TIMEOUT is
> greater than NON_TIMEOUT (We say in the spec a minimum of one second) so
> that a peer does not fire off a re-transmission request before the local
> agent has a chance to start to transmit the next MAX_PAYLOADS_SET.
> NON_RECEIVE_TIMEOUT is exponentially scaled for each retry to make sure
> that stability is preserved. So, again, ACK_RANDOM_FACTOR jitter was not
> thought to be necessary here.
>
> “
>
>
>
> Does that help?
>
>
>
> Regards
>
>
>
> Jon
>
>
>
> *From:* Martin Duke [mailto: [email-masked]]
> *Sent:* 21 May 2021 19:39
> *To:* [email-masked]
> *Cc:* John Scudder; [email-masked];
> [email-masked]; The IESG; [email-masked]; [email-masked]
> *Subject:* Re: John Scudder's Discuss on draft-ietf-core-new-block-11:

---

From: <[email-masked]>
Date: Tue, 25 May 2021 12:15:11 +0000

Hi Martin,

Thank you for the follow-up.

Please see inline.

Cheers,
Jon & Med

De : Martin Duke [mailto:[email-masked]]
Envoyé : samedi 22 mai 2021 00:03
À : [email-masked]
Cc : BOUCADAIR Mohamed TGI/OLN <[email-masked]>; [email-masked]; John Scudder <jgs=[email-masked]>; [email-masked]; The IESG <[email-masked]>; [email-masked]; [email-masked]
Objet : Re: John Scudder's Discuss on draft-ietf-core-new-block-11: (with DISCUSS and COMMENT)

I think we've collapsed the potential problem to NON_TIMEOUT and NON_RECEIVE_TIMEOUT.

Some scenarios:
(1) Let's say two senders simultaneously send MAX_PAYLOAD bursts over a radio medium that causes them to interfere with each other. With no response from the receiver at all, they both wait for NON_TIMEOUT and send another burst, causing the same effect.

Can this not happen?

First, please note that some natural randomness will be observed by peers (including the effect of probing-rate locking/unlocking and traffic exchange profile of each peer). Please remember that PROBING_RATE is defined in RFC7252 as ** an average ** data rate let alone this assumption in the applicability scope:

  “Concretely, this mechanism primarily
   targets applications such as DDoS Open Threat Signaling (DOTS) that
   cannot use CON requests/responses because of potential packet loss
   and that support application-specific mechanisms to assess whether
   the remote peer is not overloaded and thus is able to process the
   messages sent by a CoAP endpoint (e.g., DOTS heartbeats in
   Section 4.7 of [RFC8782]).”

So, the full sync case to happen requires many conditions that might be difficult to satisfy.

Back to your question: If there is ** any response ** (which can be during or just after the second burst), such responses will come serially out of the slow link, so having different timestamps and hence the third and subsequent triggered bursts of the same body will now be out of phase. There is also a chance if jitter is added that there may be a timing equality on a subsequent burst if the initial burst is closely aligned.

However, if there is ** no response ** at all over multiple bursts of a body, then the scenario you raise is problematic.

We added this note:

      Note: Randomness may naturally be provided based on the traffic
      profile, how probing-rate is computed (as this is an average), and
      when the peer responds.  Randomness is explicitly added for some
      of the congestion control parameters to handle situations where
      everything is in sync when retrying.

and other changes that you can see at: https://tinyurl.com/new-block-latest.

(2) Two receivers send bursts that arrive at a bottleneck at the same time, causing congestion losses

The receivers will request missing blocks, wait NON_RECEIVE_TIMEOUT, and trigger retransmission. For these to also collide, that would imply that (1) the RTTs are similar, and (2) the timing of the first missing packet signal is roughly simultaneous.

Is that correct?

As the ‘request missing’ going back to the transmitters will come out of the slow link serially with different timestamps, the transmitters are unlikely to resend the missing packets at the same time.

We do have the following in the spec:

   It is likely that the client will start transmitting the next
   MAX_PAYLOADS_SET before the server times out on waiting for the last
   of the previous MAX_PAYLOADS_SET.  On receipt of a payload from the
   next MAX_PAYLOADS_SET, the server SHOULD send a 4.08 (Request Entity
   Incomplete) Response Code indicating any missing payloads from any
   previous MAX_PAYLOADS_SET.  Upon receipt of the 4.08 (Request Entity
   Incomplete) Response Code, the client SHOULD send the missing
   payloads before continuing to send the remainder of the
   MAX_PAYLOADS_SET and then go into another NON_TIMEOUT delay prior to
   sending the next MAX_PAYLOADS_SET.

or

   It is likely that the server will start transmitting the next
   MAX_PAYLOADS_SET before the client times out on waiting for the last
   of the previous MAX_PAYLOADS_SET.  Upon receipt of a payload from the
   new MAX_PAYLOADS_SET, the client SHOULD send a request indicating any
   missing payloads from any previous MAX_PAYLOADS_SET.  Upon receipt of
   such request, the server SHOULD send the missing payloads before
   continuing to send the remainder of the MAX_PAYLOADS_SET and then go
   into another NON_TIMEOUT delay prior to sending the next
   MAX_PAYLOADS_SET.
which will add in another degree of randomness depending which payload gets through to the receiver.

We don’t think that a change is needed here other than updating the quoted text to refer to NON_TIMEOUT_RANDOM instead of NON_TIMEOUT.

Martin

On Fri, May 21, 2021 at 12:17 PM <[email-masked]<mailto:[email-masked]>> wrote:
Hi Martin,