Subject: Re: [core] block1 size negotiation with 4.13 Request Entity Too Large

From: Achim Kraus <[email-masked]>
Date: Thu, 7 Jan 2021 21:44:00 +0100

Hi Carsten
Hi List,

sorry for that late follow up of that question from last summer.
Currently I try to cleanup that code in Californium.
Doing so, I'm not sure, what the intended behaviour in the following
scenario should be:
- the client starts with a request "GET block2(num=5, size=256)".
- the server is configured to send blocks only up to 64 bytes.

If that exchange would happen at num=0, it works fine to send just a
smaller response, but with "num=5", I'm not sure, what is intended.
So, does this result in:

- the server respond with "CONTENT block2(num=?, size=64)"
   the issue with that is in my opinion, that either the "num" must
   be changed, or the offset can not longer calculated by
   option.num * option.size. To change the "num" may cause issues,
   though that "num" seems to be used to ACK the blocks.

- the server respond with an error; but which one should be used?

best regards
Achim

Am 22.07.20 um 15:23 schrieb Carsten Bormann:
> On 2020-07-22, at 15:20, Achim Kraus <[email-masked]> wrote:
>>
>> Hi Carsten,
>>
>> thanks a lot!
>>
>>> So it is not forbidden to request a smaller block size later.
>>
>> My point is,
>> - was this intented or
>> - just not considered and therefore not explicitly forbidden?
>
> This was intended.  A Stateless server may simply not know what it did before.  It might of course simply give up if the block number is ≠ 0, but we gave it a little more freedom, at the cost of maybe causing a bit more coding for clients that want to cope with these highly stateless servers.
>
>> All examples and some text points more into the direction, that it is
>> intended to be negotiated only in the 1. exchange. I'm still not
>> convinced, I think to forbid this is the better approach.
>
> I don’t know why the server would keep its preferences secret up to a few blocks into the transfer unless it is of the above kind and faces a sudden resource shortage.
>
> Grüße, Carsten
>
>
>>
>> Sure, if no-one else has doubts, go for it.
>>
>> best regards
>> Achim Kraus
>>
>> Am 22.07.20 um 13:08 schrieb Carsten Bormann:
>>> Hi Achim,
>>>
>>> Good questions.
>>>
>>>> On 2020-07-22, at 09:56, Achim Kraus <[email-masked]> wrote:
>>>>
>>>> Dear list,
>>>>
>>>> I would really appreciate, if the detail questions about the 4.13
>>>> failover could be answered!
>>>>
>>>> Without answers, I'm still afraid, that implementations will became
>>>> "overcomplicated" and "less interoperable".
>>>>
>>>> 1. is it intended to change the block size only in the first exchange?
>>>> Or on other exchanges? (Additionally, if changing later is considered as
>>>> failure, how should that failure be handled/processed?)
>>>
>>> RFC 7959 says (Section 2):
>>>
>>>     In most cases, all blocks being transferred for a body (except for
>>>     the last one) will be of the same size.  (If the first request uses a
>>>     bigger block size than the receiver prefers, subsequent requests will
>>>     use the preferred block size.)  The block size is not fixed by the
>>>     protocol. […]
>>>
>>> Simon cited some more text, but this is about too many blocks, not about too big blocks.  So it is not forbidden to request a smaller block size later.  I would expect that this is a rather unusual case, so I would not be surprised if a client gives up if this happens in mid-transfer.  On the other hand, it is also not hard to enable a client to adapt.
>>>
>>>> 2. is it intended, that a 4.13 response without block1 option to a
>>>> request also without block1 option, should also use/try a failover with
>>>> a request with block 1 option?
>>>
>>> That’s what RFC 7959 says:
>>>
>>>     The error code 4.13 (Request Entity Too Large) can be returned at any
>>>     time by a server that does not currently have the resources to store
>>>     blocks for a block-wise request payload transfer that it would intend
>>>     to implement in an atomic fashion.  (Note that a 4.13 response to a
>>>     request that does not employ Block1 is a hint for the client to try
>>>     sending Block1, and a 4.13 response with a smaller SZX in its Block1
>>>     Option than requested is a hint to try a smaller SZX.)
>>>
>>> This is phrased as a note, because the response contains a hint; there is no requirement that Block1 is supported.
>>>
>>> Simon also asked:
>>>
>>>>    2) What exactly should send the server ? 4.13 or 4.13 + block1 with szx option ? is size1 should be used to guess the correct block size ?
>>>
>>> I would expect that the server should send the desired block size if it wants the client to change from the block size it used, so 4.13 + Block1.  Size1 is not about block size, but about the size of the whole body - if the server has a limitation there, it might want to error out on a 4.13 (without a Block1 option).
>>>
>>> Grüße, Carsten
>>>
>>
>

---

From: <[email-masked]>
Date: Thu, 7 Jan 2021 21:04:19 -0000

Hi Achim,

It is my understanding that

CONTENT block2(num=5, size=256)
CONTENT block2(num=20, size=64)

Are equivalent as they both define the same offset into the data, and so returning " CONTENT block2(num=20, size=64)" is valid for your example.

Regards

Jon

> -----Original Message-----
> From: Achim Kraus [mailto:[email-masked]]
> Sent: 07 January 2021 20:44
> To: Carsten Bormann
> Cc: [email-masked]
> Subject: Re: [core] block1 size negotiation with 4.13 Request Entity Too
> Large
> 
> Hi Carsten
> Hi List,
> 
> > This was intended.  A Stateless server may simply not know what it did
> before.  It might of course simply give up if the block number is ≠ 0, but we
> gave it a little more freedom, at the cost of maybe causing a bit more coding
> for clients that want to cope with these highly stateless servers.
> >
> > I don’t know why the server would keep its preferences secret up to a few
> blocks into the transfer unless it is of the above kind and faces a sudden
> resource shortage.
> >
> > Grüße, Carsten
> >
> >
> >>
> >>> Simon cited some more text, but this is about too many blocks, not
> about too big blocks.  So it is not forbidden to request a smaller block size
> later.  I would expect that this is a rather unusual case, so I would not be
> surprised if a client gives up if this happens in mid-transfer.  On the other
> hand, it is also not hard to enable a client to adapt.
> >>>
> >>>>    2) What exactly should send the server ? 4.13 or 4.13 + block1 with szx
> option ? is size1 should be used to guess the correct block size ?
> >>>
> >>> I would expect that the server should send the desired block size if it
> wants the client to change from the block size it used, so 4.13 + Block1.
> Size1 is not about block size, but about the size of the whole body - if the
> server has a limitation there, it might want to error out on a 4.13 (without a
> Block1 option).
> >>>
> >>> Grüße, Carsten
> >>>
> >>
> >
> 
> _______________________________________________
> core mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/core

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 7 Jan 2021 22:16:10 +0100

Hi Achim,

> - the client starts with a request "GET block2(num=5, size=256)".

The operative word here is “starts with”.
So this is the first time the server has a chance to react to the proposed block size.

Section 2 of RFC 7959 says:

   In most cases, all blocks being transferred for a body (except for
   the last one) will be of the same size.  (If the first request uses a
   bigger block size than the receiver prefers, subsequent requests will
   use the preferred block size.)  The block size is not fixed by the […]

This talks about the “first request”, which could be read as meaning “num=0”, but in this case clearly matches the premise.

>  though that "num" seems to be used to ACK the blocks.

I don’t know of any issues that the change in “num” would cause.
“Num” is just a terse encoding for the current byte position (*), based on “szx”, and when szx changes, num has to change in unison.
(Jon is right that, with the size divided by four, the num needs to be multiplied by four to start filling in the right byte position(**).)

> - the server respond with an error; but which one should be used?

Since the protocol has been designed to enable stateless servers, we didn’t see a need to provide an error that a stateless server would never know what to use for.

Grüße, Carsten

(*) Well, that is just given as an “Implementation note” in Section 2.2, but I would expect a sane implementation to follow this note.

(**) A prank implementation might try to fill in num=23 for size=64, as this is also part of the block 5 at size 256.  I don’t think that this behavior would be justifiable, as the case for num=0 clearly fills in the leftmost block.

---

From: Achim Kraus <[email-masked]>
Date: Thu, 7 Jan 2021 22:42:38 +0100

Hi Jon,
Hi carsten,

thanks for the fast answers!

So, changing the block-number to compensate a blocksize change, is the
intended behavior.

 > I don’t know of any issues that the change in “num” would cause.

That may depend on the interpretation of,

RFC 7959, 2.3. Block Options in Requests and Responses

Block 2 / Request
"The NUM field in the Block2 Option gives the block number of
the payload that is being requested to be returned in the
response."

The interpretation may be, either
- the requested "block number" is to be returned in the response, or
- the payload of the request block is to be returned.

best regards
Achim

---

From: Carsten Bormann <[email-masked]>
Date: Fri, 8 Jan 2021 00:06:05 +0100

> - the payload of the request block is to be returned.

It’s already Friday in Germany, so here goes:

http://doi.org/10.1353/lan.2003.0189:

Temperley, David. (2003). Ambiguity Avoidance in English Relative Clauses. Language. 79. 464-484. 10.1353/lan.2003.0189

The mouse the cat the dog chased chased ran
(The mouse {that the cat {that the dog chased} chased} ran)

I’m so glad I’m not a linguist.

So maybe a more practical reference can be found at stackexchange.

https://english.stackexchange.com/questions/140932/ambiguity-in-use-of-relative-pronouns says:
"You can use a relative pronoun without ambiguity if you make sure it immediately follows its antecedent.”
Well, that’s just one, unreferenced opinion, but I think this is indeed what is meant here (i.e., the “payload” is the referent [reference target] of the “that”.)

https://english.stackexchange.com/questions/169939/more-confusion-with-relative-pronoun-ambiguity can be used to muddy the water by distinguishing integrated relative clauses from supplementary relative clauses.
Since the above is an integrated relative clause (i.e., no comma), the “closest antecedent” rule still applies.

I’m fully convinced that the author had all this in mind (but then I’m not sure 🤔; I wrote that text on 2012-01-13(**) at 00:06:05, which was exactly nine years minus five days ago, for draft-ietf-core-block-05, published a few minutes later), and that the RFC editor later pondered the potential ambiguity carefully and decided that there wasn’t one.

Grüße, Carsten

(**) Which was a Friday, 13th.
The commit comment clarifies everything, though: 
it was the single word “Uff.” (the German word for “phew”).