Subject: [core] RFC 7967 (No-Response) together with block-wise transfers in CoAP

From: Fredrik Wegelid <[email-masked]>
Date: Wed, 26 Jun 2019 13:30:40 +0200

Hi,

I hope that I am using this email list in the correct way. I apologise if that is not the case.
I have some questions/thoughts about using the No-Response option (RFC 7967) together with block-wise transfers in COAP.

In my team we are using CoAP to send messages from devices to a server. The messages are of such a size that we need to make block-wise transfers. In an attempt to reduce the amount of traffic in the network we have looked at using the No-Response option. We are however not clear on how this would work if used together with block-wise transfers. We would like to use the No-Response option to make the server omit the 2.31 Continue responses from the server on each block. We would however still like to be able to get a response on the last block to indicate the status of the entire block sequence. 

Does this make sense? In RFC 7967 block-wise transfers are not mentioned. Are they meant to be used together?

Best regards,
Fredrik Wegelid

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 26 Jun 2019 13:43:48 +0200

Hi Fredrik,

I’m not sure I entirely understand the message exchanges you envision; maybe you can provide an example.

RFC 7967 has a way to suppress responses based on the response code.
Unfortunately, the granularity is on the class only at the moment.
The option that carries this information isn’t explicitly defined as an extension point.  Still, someone (the original author?  The WG?) might want to go ahead, e.g., by defining a bit in the option value that says “Not interested in 2.31 responses”.  (Maybe we want to understand what exactly a client would not be interested in a bit more before we go ahead allocating bits like this.)

Abhijan, what do you think?

Grüße, Carsten

> On Jun 26, 2019, at 13:30, Fredrik Wegelid <[email-masked]> wrote:
> 
> Hi,
> 
> _______________________________________________
> core mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/core
>

---

From: Fredrik Wegelid <[email-masked]>
Date: Wed, 26 Jun 2019 13:57:40 +0200

Hi Carsten,

This is how an example looks like without the No-Response option.

Client -> Server: 	NON POST Payload: “Block 1...”
Server -> Client:	NON 2.31 Continue
Client -> Server: 	NON POST Payload: “Block 2...”
Server -> Client:	NON 2.31 Continue

More blocks…

Client -> Server: 	NON POST Payload: “Final block...”
Server -> Client:	NON 2.01 Created (As an example)

With the No-Response option I wish the exchange to look something like this:

Client -> Server: 	NON POST (No-Response:26) Payload: “Block 1...”
Client -> Server: 	NON POST (No-Response:26) Payload: “Block 2...”

More blocks…

Client -> Server: 	NON POST Payload: “Final block...”
Server -> Client:	NON 2.01 Created (As an example)

This would mean that the server would not send the 2.31 responses for each block (as the No-Response option is set to 26) while it still would send the final response (since the final block does not have the No-Response option set).
This allows the client to ensure that all blocks were sent successfully. If they are not, the server would respond with another code. It would also mean that the amount of traffic on the network is reduced.

Best regards,
Fredrik

> On 26 Jun 2019, at 13:43, Carsten Bormann <[email-masked]> wrote:
> 
> Hi Fredrik,
>

---

From: Abhijan Bhattacharyya <[email-masked]>
Date: Thu, 27 Jun 2019 10:48:26 +0530

Hi Fredrick and Carsten,
Thanks for the discussion so far. Let me put my thoughts.

The proposal to reduce the traffic in case of block-wise transfer using no-response is obvious. However, there is a catch. As per my understanding, the use case considered for block-wise transfer relates to transfers demanding high integrity of the received data and very low complexity at the receiving end in re-assembling the segments. One example is firmware upgrade of remote sensors: you cannot miss a single bit. Also, the message reception is tightly sequential in order. In case you miss one segment or receive in out-of-order, the transfers seizes in between.

Now, Fredrick, coming to your solution: I do not know the QoS requirements in your perceived use case. You do not care about responses of intermediate blocks (essentially this means RESTfully fire-and-forget!) and you take the response of the last block to ensure the end of transfer. Now the question is, if you care for all the blocks at the receiving end (for which you did not care for a response) how do you ensure reception of all the blocks? Considering the resource constrains in IoT use cases it will be a wastage of bandwidth if you have sent all the segments from the transmit end and the receiver, after receiving the end block, realizes that one intermediate segment got missing. The whole transfer, and the resource consumption related to that, becomes useless. You might have rather stopped at the loss instance and take corrective measures to resume. That would save your resources. So, there something more to think and design carefully according to the exact QoS requirements of the perceived class of use-cases.

However, in case you are looking for real-time + low-latency + low-resource consuming transfer, you may please have a look at this proposal that we already submitted few months back: https://tools.ietf.org/html/draft-bhattacharyya-core-a-realist-02 . To relate this draft to your problem you may simply consider to model your block-wise traffic as a time-series traffic.

Having said that, I did once nurture some ideas to make block-wise transfer "statistically" more resource-efficient, yet reliable . But, must I be confirmed about the efficacy of my idea, both theoretically and experimentally, before I speak about that. I have kept that in back burner. In case there is enough interest I can renew the interest and discuss.

Thank you.     

With Best Regards
Abhijan Bhattacharyya
Consultant / Scientist,
{Internet Protocols | 5G | Standardization}, 
TCS Research,
Tata Consultancy Services
Building 1B,Ecospace
Plot -  IIF/12 ,New Town, Rajarhat,
Kolkata - 700160,West Bengal
India
Ph:- +91 33 66884691
Cell:- +919830468972 | +918583875003
Mailto: [email-masked]
Website: http://www.tcs.com
____________________________________________
Experience certainty.	IT Services
Business Solutions
Consulting
____________________________________________

-----"core" <[email-masked]> wrote: -----
To: Carsten Bormann <[email-masked]>
From: Fredrik Wegelid 
Sent by: "core" 
Date: 06/26/2019 05:28PM
Cc: core <[email-masked]>
Subject: Re: [core] RFC 7967 (No-Response) together with block-wise transfers in CoAP

"External email. Open with Caution"

Hi Carsten,

Client -> Server: NON POST Payload: &#8220;Block 1...&#8221;
Server -> Client:	NON 2.31 Continue
Client -> Server: NON POST Payload: &#8220;Block 2...&#8221;
Server -> Client:	NON 2.31 Continue

More blocks&#8230;

Client -> Server: NON POST Payload: &#8220;Final block...&#8221;
Server -> Client:	NON 2.01 Created (As an example)

Client -> Server: NON POST (No-Response:26) Payload: &#8220;Block 1...&#8221;
Client -> Server: NON POST (No-Response:26) Payload: &#8220;Block 2...&#8221;

More blocks&#8230;

Client -> Server: NON POST Payload: &#8220;Final block...&#8221;
Server -> Client:	NON 2.01 Created (As an example)

Best regards,
Fredrik

> I&#8217;m not sure I entirely understand the message exchanges you envision; maybe you can provide an example.
> 
> The option that carries this information isn&#8217;t explicitly defined as an extension point.  Still, someone (the original author?  The WG?) might want to go ahead, e.g., by defining a bit in the option value that says &#8220;Not interested in 2.31 responses&#8221;.  (Maybe we want to understand what exactly a client would not be interested in a bit more before we go ahead allocating bits like this.)
> 

_______________________________________________
core mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/core
=====-----=====-----=====
Notice: The information contained in this e-mail
message and/or attachments to it may contain 
confidential or privileged information. If you are 
not the intended recipient, any dissemination, use, 
review, distribution, printing or copying of the 
information contained in this e-mail message 
and/or attachments to it are strictly prohibited. If 
you have received this communication in error, 
please notify us by reply e-mail or telephone and 
immediately and permanently delete the message 
and any attachments. Thank you

---

From: Fredrik Wegelid <[email-masked]>
Date: Thu, 27 Jun 2019 15:22:55 +0200

Hi Abhijan,

Thank you for your response. It is great to see how engaged you guys are :)

The reason that we are talking about this solution are the requirements for the devices in our network. We have devices sending messages (that are big enough to require a blockwise transfer) that have very high requirements when it comes to battery consumption. In order to listen for responses the devices have to keep the radio turned on to listen for these responses. By not listening to the responses (except the last one) we can keep the radio turned off for a larger portion of the time and in such a way save battery. This is our reasoning for wanting to use No-Response together with blockwise.

I understand that from a network perspective this does not really make sense. To reduce the amount of traffic on the network I agree that it would be better to listen for the 2.31 Continue responses and restart the blockwise transfer in the case of a lost block. If we do not have to listen to these responses we could hopefully reduce the power consumption of the device.

BR,
Fredrik

> On 27 Jun 2019, at 07:18, Abhijan Bhattacharyya <[email-masked]> wrote:
> 
> Hi Fredrick and Carsten,
> However, in case you are looking for real-time + low-latency + low-resource consuming transfer, you may please have a look at this proposal that we already submitted few months back: https://tools.ietf.org/html/draft-bhattacharyya-core-a-realist-02 <https://tools.ietf.org/html/draft-bhattacharyya-core-a-realist-02> . To relate this draft to your problem you may simply consider to model your block-wise traffic as a time-series traffic.
> 
> Mailto: [email-masked] <mailto:[email-masked]>
> Website: http://www.tcs.com <http://www.tcs.com/>
> -----"core" <[email-masked] <mailto:[email-masked]>> wrote: -----
> To: Carsten Bormann <[email-masked] <mailto:[email-masked]>>
> From: Fredrik Wegelid 
> Sent by: "core" 
> Date: 06/26/2019 05:28PM
> Cc: core <[email-masked] <mailto:[email-masked]>>
> > On 26 Jun 2019, at 13:43, Carsten Bormann <[email-masked] <mailto:[email-masked]>> wrote:
> > 
> > Hi Fredrik,
> > 
> >> On Jun 26, 2019, at 13:30, Fredrik Wegelid <[email-masked] <mailto:[email-masked]>> wrote:
> >> 
> >> Hi,
> >> 
> >> [email-masked] <mailto:[email-masked]>
> >> https://www.ietf.org/mailman/listinfo/core <https://www.ietf.org/mailman/listinfo/core>
> >> 
> > 
> 
> [email-masked] <mailto:[email-masked]>
> https://www.ietf.org/mailman/listinfo/core <https://www.ietf.org/mailman/listinfo/core>

---

From: Simon Bernard <[email-masked]>
Date: Fri, 28 Jun 2019 12:18:46 +0200

Hi Fredrik,

Reading RFCs, I would assume that this is not really recommended.

The RFC7967 says that benefits are :
/"Reduction in server-side load by relieving the server from responding 
to requests for which responses are not necessary."/

But in case of block-wise transfer, the response seems clearly necessary 
or at least the acknowledgement which is the only mechanism available 
for UDP packet drop/reorder issue.
RFC7959 says :
/"using Non-confirmable requests within block-wise transfers outside the 
use case of Section 2.8 would escalate the overall non-delivery 
probability"/.
(translation CON is strongly recommended)

Generally block must be sent/received in right order else you could face 
issue like this :
/"If not all previous blocks are available at the server at the time of 
processing the final block, the transfer fails and error code 4.08 
(Request Entity Incomplete, Section 2.9.2) MUST be returned.  A server 
MAY also return a 4.08 error code for any (final or non-final) Block1 
transfer that is not in sequence; therefore, clients that do not have 
specific mechanisms to handle this case SHOULD always start with block 
zero and send the following blocks in order."/ (RFC7959)

As /"a sequence of block-wise transfers should be performed 'without 
undue delay'"/, piggybacked are really well-suited for block-wise 
transfers.  (RFC7959)

And using No-Response with CON would probably not help in your case.
/"Using this option with CON requests may not serve the desired purpose 
if piggybacked responses are triggered."/(RFC7967)

Simon

Le 27/06/2019 à 15:22, Fredrik Wegelid a écrit :
> The reason that we are talking about this solution are the 
> requirements for the devices in our network. We have devices sending 
> messages (that are big enough to require a blockwise transfer) that 
> have very high requirements when it comes to battery consumption. In 
> order to listen for responses the devices have to keep the radio 
> turned on to listen for these responses. By not listening to the 
> responses (except the last one) we can keep the radio turned off for a 
> larger portion of the time and in such a way save battery. This is our 
> reasoning for wanting to use No-Response together with blockwise.
>
> I understand that from a network perspective this does not really make 
> sense. To reduce the amount of traffic on the network I agree that it 
> would be better to listen for the 2.31 Continue responses and restart 
> the blockwise transfer in the case of a lost block. If we do not have 
> to listen to these responses we could hopefully reduce the power 
> consumption of the device.
>
> BR,
> Fredrik
>
>> On 27 Jun 2019, at 07:18, Abhijan Bhattacharyya 
>> <[email-masked] 
>> <mailto:[email-masked]>> wrote:
>>
>> Hi Fredrick and Carsten,
>> The proposal to reduce the traffic in case of block-wise transfer 
>> using no-response is obvious. However, there is a catch. As per my 
>> understanding, the use case considered for block-wise transfer 
>> relates to transfers demanding high integrity of the received data 
>> and very low complexity at the receiving end in re-assembling the 
>> segments. One example is firmware upgrade of remote sensors: you 
>> cannot miss a single bit. Also, the message reception is tightly 
>> sequential in order. In case you miss one segment or receive in 
>> out-of-order, the transfers seizes in between.
>>
>> Now, Fredrick, coming to your solution: I do not know the QoS 
>> requirements in your perceived use case. You do not care about 
>> responses of intermediate blocks (essentially this means RESTfully 
>> fire-and-forget!) and you take the response of the last block to 
>> ensure the end of transfer. Now the question is, if you care for all 
>> the blocks at the receiving end (for which you did not care for a 
>> response) how do you ensure reception of all the blocks? Considering 
>> the resource constrains in IoT use cases it will be a wastage of 
>> bandwidth if you have sent all the segments from the transmit end and 
>> the receiver, after receiving the end block, realizes that one 
>> intermediate segment got missing. The whole transfer, and the 
>> resource consumption related to that, becomes useless. You might have 
>> rather stopped at the loss instance and take corrective measures to 
>> resume. That would save your resources. So, there something more to 
>> think and design carefully according to the exact QoS requirements of 
>> the perceived class of use-cases.
>>
>> However, in case you are looking for real-time + low-latency + 
>> low-resource consuming transfer, you may please have a look at this 
>> proposal that we already submitted few months back: 
>> https://tools.ietf.org/html/draft-bhattacharyya-core-a-realist-02 . 
>> To relate this draft to your problem you may simply consider to model 
>> your block-wise traffic as a time-series traffic.
>>
>> Having said that, I did once nurture some ideas to make block-wise 
>> transfer "statistically" more resource-efficient, yet reliable . But, 
>> must I be confirmed about the efficacy of my idea, both theoretically 
>> and experimentally, before I speak about that. I have kept that in 
>> back burner. In case there is enough interest I can renew the 
>> interest and discuss.
>>
>> Thank you.
>>
>> With Best Regards
>> Abhijan Bhattacharyya
>> Consultant / Scientist,
>> <mailto:[email-masked]>
>> This would mean that the server would not send the 2.31 responses for 
>> each block (as the No-Response option is set to 26) while it still 
>> would send the final response (since the final block does not have 
>> the No-Response option set).
>> This allows the client to ensure that all blocks were sent 
>> successfully. If they are not, the server would respond with another 
>> code. It would also mean that the amount of traffic on the network is 
>> reduced.
>>
>> Best regards,
>> Fredrik
>>
>> > I’m not sure I entirely understand the message exchanges you 
>> envision; maybe you can provide an example.
>> >
>> > The option that carries this information isn’t explicitly defined 
>> as an extension point.  Still, someone (the original author?  The 
>> WG?) might want to go ahead, e.g., by defining a bit in the option 
>> value that says “Not interested in 2.31 responses”.  (Maybe we want 
>> to understand what exactly a client would not be interested in a bit 
>> more before we go ahead allocating bits like this.)
>> >
>> >> On Jun 26, 2019, at 13:30, Fredrik Wegelid 
>> <[email-masked] <mailto:[email-masked]>> wrote:
>> >>
>> >> Hi,
>> >>
>> >> I have some questions/thoughts about using the No-Response option 
>> (RFC 7967) together with block-wise transfers in COAP.
>> >>
>> >> In my team we are using CoAP to send messages from devices to a 
>> server. The messages are of such a size that we need to make 
>> block-wise transfers. In an attempt to reduce the amount of traffic 
>> in the network we have looked at using the No-Response option. We are 
>> however not clear on how this would work if used together with 
>> block-wise transfers. We would like to use the No-Response option to 
>> make the server omit the 2.31 Continue responses from the server on 
>> each block. We would however still like to be able to get a response 
>> on the last block to indicate the status of the entire block sequence.
>> >>
>> >> Does this make sense? In RFC 7967 block-wise transfers are not 
>> mentioned. Are they meant to be used together?
>> >>
>> >> Best regards,
>> >> Fredrik Wegelid

---

From: "Kraus Achim (INST/ECS4)" <[email-masked]>
Date: Fri, 28 Jun 2019 11:47:10 +0000

Hello Fredrik,

so the assumption, that RFC7967 generally applies also to RFC7959, seem to be negated.

> sending messages (that are big enough to require a blockwise transfer)

In my experience, RFC7959 is applied to two use-cases:
- a dynamic resource is too large to fit into one message and is therefore split in few blocks, either using block1 or block2
- a static giant resource must be transferred, potentially with pauses and resumptions, hopefully using block2

For the "giant resource", I think, it's not worth to try it, it will fail or is impossible because of block2.
For the "few blocks resource" you may generally have a chance, that it works good enough.
Your wording "big enough to require a blockwise transfer" points into that direction.
I'm still not clear, if your application requires, that ALL blocks are transferred, or if it may be acceptable, that the data of some blocks are missing.
Especially, if it's acceptable, that some blocks are missing, RFC7967 with a proprietary application layer may be a first step.  

Mit freundlichen Grüßen / Best regards 

Achim Kraus

Engineering Cloud Services 4 Bosch IoT Hub (INST/ECS4) 

Von: core <[email-masked]> Im Auftrag von Fredrik Wegelid
Gesendet: Donnerstag, 27. Juni 2019 15:23
An: Abhijan Bhattacharyya <[email-masked]>
Cc: core <[email-masked]>
Betreff: Re: [core] RFC 7967 (No-Response) together with block-wise transfers in CoAP

Hi Abhijan,

BR,
Fredrik

On 27 Jun 2019, at 07:18, Abhijan Bhattacharyya <mailto:[email-masked]> wrote:

Hi Fredrick and Carsten,

Thank you.     

With Best Regards
Abhijan Bhattacharyya
Consultant / Scientist,
{Internet Protocols | 5G | Standardization}, 
TCS Research,
Tata Consultancy Services
Building 1B,Ecospace
Plot -  IIF/12 ,New Town, Rajarhat,
Kolkata - 700160,West Bengal
India
Ph:- +91 33 66884691
Cell:- +919830468972 | +918583875003
Mailto: mailto:[email-masked]
Website: http://www.tcs.com/
____________________________________________
Experience certainty. IT Services
Business Solutions
Consulting
____________________________________________

-----"core" <mailto:[email-masked]> wrote: -----
To: Carsten Bormann <mailto:[email-masked]>
From: Fredrik Wegelid 
Sent by: "core" 
Date: 06/26/2019 05:28PM
Cc: core <mailto:[email-masked]>
Subject: Re: [core] RFC 7967 (No-Response) together with block-wise transfers in CoAP
"External email. Open with Caution"

Hi Carsten,

More blocks…

More blocks…

Best regards,
Fredrik

> On 26 Jun 2019, at 13:43, Carsten Bormann <mailto:[email-masked]> wrote:
> 
> Hi Fredrik,
> 
>> On Jun 26, 2019, at 13:30, Fredrik Wegelid <mailto:[email-masked]> wrote:
>> 
>> Hi,
>> 
>> mailto:[email-masked]

mailto:[email-masked]