Subject: [core] Large asynchronous notifications under DDoS: New BLOCK Option?

From: <[email-masked]>
Date: Tue, 7 Apr 2020 11:11:28 +0000

Hi all,

We are using Observe to receive notifications during attack events. These notifications are set as NON messages for reasons specific to DDoS conditions.

With DDoS telemetry information included (see draft-ietf-dots-telemetry), a notification may not fit one single message. The use of BLOCK2 is not convenient during attack times. A full description of the issue is described here: https://mailarchive.ietf.org/arch/msg/dots/Gbtf8bBWpxD9CWNBhS_TZtsWeP4/

We are considering some mechanisms to solve this issue. One of them is to define a new BLOCK option (similar to BLOCK2) that does not require the observer to send a GET to receive the next fragment. The server will send all the fragments. The observer will follow a SACK-like approach to request retransmission of missing fragments.

Please let us know whether you think this is a generic issue that should be solved at the CoAP or not. Suggestions are welcome.

Thank you.

Cheers,
Jon & Med

---

From: Christian Amsüss <[email-masked]>
Date: Tue, 7 Apr 2020 15:09:44 +0200

Hello Jon, hello Med,

I don't have full answers, but some data points (also pulling in the
cited mail):

> and a response (may need new code) that says status is too large to
> fit into a packet or has been truncated).

That an observation with blockwise would already do: If the observed
resource changes to have its representation larger than the block size
or MTU, it'd send the first block.

> The observer will follow a SACK-like approach to request
> retransmission of missing fragments.

There was a draft around on non-traditional responses[1] some time ago
that would provide building blocks from which a server could send
follow-up blocks in an unsolicited fashion: The second block would be a
message in the style of "This is a 2.05 Content response, which you
would have received as a response had you requested block2 of /path".

Those messages would not be ack'ed if they are NON, and the client can
selectively request blocks it thinks it missed (but, in such a setup,
would do that after a suitable timeout to not request something that is
already in flight).

That draft was not followed up on directly, but some of its ideas wound
up in [2] where observe notifications are sent to a multicast group. In
particular, the topic of which tokens would be usable for unsolicited
responses to unicast addresses has not received the discussion it'd
probably need.

Kind regards
Christian

[1]: https://tools.ietf.org/html/draft-bormann-core-responses-00
[2]: https://tools.ietf.org/html/draft-tiloca-core-observe-multicast-notifications-02

-- 
The detailed semantics of CoAP methods are "almost, but not entirely
unlike" [HHGTTG] those of HTTP methods.
[HHGTTG]: Adams, D., "The Hitchhiker's Guide to the Galaxy", October 1979.
  -- Shelby, et al., Internet-Draft Constrained Application Protocol (CoAP)

---

From: Jim Schaad <[email-masked]>
Date: Tue, 7 Apr 2020 08:18:36 -0700

I do not believe that there is anything today that says the observer is
required to do a GET to receive the next fragment in the event that it does
not care about what it says.   The question that sprints to mind is how
should the server/client decide that it does or does not want to retrieve
the entire content?

Jim

From: core <[email-masked]> On Behalf Of [email-masked]
Sent: Tuesday, April 7, 2020 4:11 AM
To: [email-masked]
Cc: Jon Shallow ([email-masked]) <[email-masked]>;
[email-masked]
Subject: [core] Large asynchronous notifications under DDoS: New BLOCK
Option?

Hi all,

We are using Observe to receive notifications during attack events. These
notifications are set as NON messages for reasons specific to DDoS
conditions. 

With DDoS telemetry information included (see draft-ietf-dots-telemetry), a
notification may not fit one single message. The use of BLOCK2 is not
convenient during attack times. A full description of the issue is described
here:
<https://mailarchive.ietf.org/arch/msg/dots/Gbtf8bBWpxD9CWNBhS_TZtsWeP4/>
https://mailarchive.ietf.org/arch/msg/dots/Gbtf8bBWpxD9CWNBhS_TZtsWeP4/ 

We are considering some mechanisms to solve this issue. One of them is to
define a new BLOCK option (similar to BLOCK2) that does not require the
observer to send a GET to receive the next fragment. The server will send
all the fragments. The observer will follow a SACK-like approach to request
retransmission of missing fragments. 

Please let us know whether you think this is a generic issue that should be
solved at the CoAP or not. Suggestions are welcome.

Thank you. 

Cheers,

Jon & Med

---

From: <[email-masked]>
Date: Tue, 7 Apr 2020 15:51:14 +0000

Hi Christian,

Thank you for sharing your thoughts. 

I don't see where in the two drafts an observer can request a particular missing fragment. 

Cheers,
Med

> -----Message d'origine-----
> De : Christian Amsüss [mailto:[email-masked]]
> Envoyé : mardi 7 avril 2020 15:10
> À : BOUCADAIR Mohamed TGI/OLN
> Cc : [email-masked]; Jon Shallow ([email-masked]);
> [email-masked]
> Objet : Re: [core] Large asynchronous notifications under DDoS: New
> BLOCK Option?
> 
> Hello Jon, hello Med,
> 
> follow-up blocks in an unsolicited fashion: The second block would be
> a message in the style of "This is a 2.05 Content response, which you
> would do that after a suitable timeout to not request something that
> is already in flight).
> 
> That draft was not followed up on directly, but some of its ideas
> wound up in [2] where observe notifications are sent to a multicast
> group. In particular, the topic of which tokens would be usable for
> unsolicited responses to unicast addresses has not received the
> discussion it'd probably need.
> 
> Kind regards
> Christian
>

---

From: <[email-masked]>
Date: Tue, 7 Apr 2020 16:48:48 +0000

Hi Jim,

That can be policy based at the client side + some signal to the server.

For our DOTS application, clients indicate to servers that they are (not) willing to receive all the content (below an excerpt of our spec):

   DOTS clients that are interested to receive pre- or ongoing mitigation
   telemetry (pre-or-ongoing-mitigation) information from a DOTS server
   (Section 8.2<https://tools.ietf.org/html/draft-ietf-dots-telemetry-06#section-8.2>) MUST set 'server-originated-telemetry' to 'true'.

And

   In order to signal telemetry data in a mitigation efficacy update, it

   is RECOMMENDED that the DOTS client has already established a DOTS

   telemetry setup session with the server in 'idle' time.

Clients can filter out data they are interested in (Uri-Query), e.g.,.

   Header: GET (Code=0.01)

   Uri-Path: ".well-known"

   Uri-Path: "dots"

   Uri-Path: "mitigate"

   Uri-Path: "cuid=dz6pHjaADkaFTbjr0JGBpw"

   Uri-Path: "mid=12332"

   Uri-Query: "target-alias=https1"

   Observe: 0

If not filter is included, all the content has to be returned to the client:

   Header: GET (Code=0.01)

   Uri-Path: ".well-known"

   Uri-Path: "dots"

   Uri-Path: "tm"

   Uri-Path: "cuid=dz6pHjaADkaFTbjr0JGBpw"

   Uri-Path: "tmid=123"

   Observe: 0

    Figure 34: GET to Subscribe to Telemetry Asynchronous Notifications

                           for a Specific 'tmid'

What is missing for us is a "BLOCK2"-Like option to send all fragments without waiting for a GET + retrieval of missing fragments (if any).

Cheers,
Med

De : Jim Schaad [mailto:[email-masked]]
Envoyé : mardi 7 avril 2020 17:19
À : BOUCADAIR Mohamed TGI/OLN; [email-masked]
Cc : 'Jon Shallow'; [email-masked]
Objet : RE: [core] Large asynchronous notifications under DDoS: New BLOCK Option?

I do not believe that there is anything today that says the observer is required to do a GET to receive the next fragment in the event that it does not care about what it says.   The question that sprints to mind is how should the server/client decide that it does or does not want to retrieve the entire content?

Jim

From: core <[email-masked]> On Behalf Of [email-masked]
Sent: Tuesday, April 7, 2020 4:11 AM
To: [email-masked]
Cc: Jon Shallow ([email-masked]) <[email-masked]>; [email-masked]
Subject: [core] Large asynchronous notifications under DDoS: New BLOCK Option?

Hi all,

Thank you.

Cheers,
Jon & Med

---

From: Achim Kraus <[email-masked]>
Date: Tue, 7 Apr 2020 18:50:02 +0200

Hi Med,

I think, not all assumption and requirements are clear to everyone.

I guess, your mission is to send a couple of "block" messages, but
without getting requested for the next-blocks, because the node is under
attack and already receiving too much. Therefore the SACK will be used a
little later as "cleanup"/"collect lefts".

My first idea about that was, that sending the "block-flush" just
creates the next attack. But, if you sure, that this doesn't happen,
then such an approach may help in that special situation.

I'm not sure, if you assume, that in the most cases all blocks will make
it to the destination when sending them. That may also depend on the
assumed number of blocks. If it's not assumed, that all block will be
received, your payload should be encoded in a way, that you could decode
the received blocks, even if some blocks are missing.

If that number is not too large, maybe a "blockwise with NSTART-X" will
also do it. Especially if DTLS is used, it may be possible to concat the
"ACK/next blocks" into one package (with multiple records). AFAIK, that
would require changes in implementations, which optimizes the
deduplication base on a strict sequence of blocks, but the difference
should be not too large.

best regards
Achim

Am 07.04.20 um 13:11 schrieb [email-masked]:
> Hi all,
>
> We are using Observe to receive notifications during attack events.
> These notifications are set as NON messages for reasons specific to DDoS
> conditions.
>
> With DDoS telemetry information included (see
> draft-ietf-dots-telemetry), a notification may not fit one single
> message. The use of BLOCK2 is not convenient during attack times. A full
> description of the issue is described here:
> We are considering some mechanisms to solve this issue. One of them is
> to define a new BLOCK option (similar to BLOCK2) that does not require
> the observer to send a GET to receive the next fragment. The server will
> send all the fragments. The observer will follow a SACK-like approach to
> request retransmission of missing fragments.
>
> Please let us know whether you think this is a generic issue that should
> be solved at the CoAP or not. Suggestions are welcome.
>
> Thank you.
>
> Cheers,
>
> Jon & Med
>
>
> _______________________________________________
> core mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/core
>

---

From: <[email-masked]>
Date: Tue, 7 Apr 2020 17:41:30 +0000

Hi Achim,

Please see inline. 

Cheers,
Med

> De : Achim Kraus [mailto:[email-masked]]
> Envoyé : mardi 7 avril 2020 18:50
> Cc : Jon Shallow ([email-masked]); [email-masked]

[Med] We are using CoAP for signaling DDoS attacks and for seeking help from a DDoS mitigator. The mitigation signal itself is compact as we do have a requirement that the signal channel survive under extreme DDoS conditions. We do have a mechanism to maintain the signal alive, to detect when it is lost and then to recover asap. 

We are now designing an extension to the base signaling that allows to share knowledge of attacks for better coordination and efficient mitigation. To that aim, we need to signal DDoS telemetry data between our DOTS agents (CoAP endpoints).  

> little later as "cleanup"/"collect lefts".

[Med] Yes.  

> then such an approach may help in that special situation.

[Med] Yes. 

> the received blocks, even if some blocks are missing.

[Med] This is will be nice to have. This might be managed at the application (DOTS) level but there are some challenges as well (make sure that the crafted data will fit a single DTLS packet).

---

From: "Jon Shallow" <[email-masked]>
Date: Tue, 7 Apr 2020 18:56:21 +0100

Hi Achim,

To add to Med's inline comments, the use case that we are trying to solve is
as follows.

We are trying to send telemetry information from a server to a unicast
client which has the potential to exceed the size of IP packet (possibly
multiple times too large).

The environment that this telemetry information is going to be transmitted
in will be where Distributed Denial of Service (DDoS) attacks are taking
place and so there is a high chance that network pipes will be running at or
near capacity and hence there is likely to be high packet loss.  It is
likely that the direction of the telemetry data is the same as that of the
packet loss.

This telemetry is an additional tool for passing information about how
mitigations of these DDoS attacks are being effective so that decisions can
be taken as to how better to mitigate the DDoS situation.  This is making
use of the DDoS Open Threat Signaling protocol (DOTS) described in RFC8612
with other drafts for components of this protocol in the final stages of
becoming RFCs ( https://tools.ietf.org/wg/dots/ ).

While not desirable, it is acceptable if the client is unable to receive the
telemetry data (out of band mechanisms such as phone calls can be used in
slow time if needed for tuning against attacks) but we would like to
maximize the possibility of the telemetry data getting through.

Currently with other data and small amounts of telemetry, the GET response
or Observe data can safely be sent off from the server and the client may or
may not get it as we are using NON-confirmable.  PUT (which has small
amounts of data) and DELETE are also sent as NON-confirmable - and the
client handles things if there is not seen a 2.XX etc. response to the
PUT/DELETE.

This issue is when the data is too large to fit into a single packet - we
currently use the CoAP BLOCK2 option which works fine under normal traffic
conditions, but relies on the client synchronously requesting the next block
of data.  We currently are doing this as NON-confirmable so a loss of a
block of data causes the transfer to stall with the need to do garbage
collection at a later point in time ion the server.  Using CONfirmable
causes everything to slow right down with the lossy network and prevents the
next CON transmission for a long time as NSTART is 1.

The data is CBOR based on YANG with the YANG parameters two-way mapped into
integers to give as much CBOR data reduction as possible, so cannot save any
space there.  It is theoretically possible to reduce the amount of telemetry
data by only asking for subsets of information as Med has referred to in
this email trail.

It is possible to solve the issue using IP fragmentation and let the client
re-assemble everything, but this is not ideal in a lossy network and there
is no way to ask for a particular IP fragment to be re-transmitted.

If there was a new CoAP option - let's call it NONBLOCK2, which the server
uses to send the all the data packets without waiting for the GET request of
the next block (only difference compared with BLOCK2).  If the client does
not see all the blocks after a timeout, it can then request the missing
blocks by using one or more of the NONBLOCK2 options in a new GET request -
or just wait until the next GET / Observe response.  This may be a way to
move forward if we cannot solve the issue in another way.

Regards

Jon

> -----Original Message-----
> From: Dots [mailto: [email-masked]] On Behalf Of
[email-masked]
> Sent: 07 April 2020 18:42
> To: Achim Kraus; [email-masked]
> Cc: Jon Shallow ([email-masked]); [email-masked]
> Subject: Re: [Dots] [core] Large asynchronous notifications under DDoS:
> New BLOCK Option?
> 
> Hi Achim,
> 
> Please see inline.
> 
> Cheers,
> Med
> 
> [Med] We are using CoAP for signaling DDoS attacks and for seeking help
> from a DDoS mitigator. The mitigation signal itself is compact as we do
have
> a requirement that the signal channel survive under extreme DDoS
> conditions. We do have a mechanism to maintain the signal alive, to detect
> when it is lost and then to recover asap.
> 
> We are now designing an extension to the base signaling that allows to
> share knowledge of attacks for better coordination and efficient
mitigation.
> To that aim, we need to signal DDoS telemetry data between our DOTS
> agents (CoAP endpoints).
> 
> [Med] This is will be nice to have. This might be managed at the
application
> (DOTS) level but there are some challenges as well (make sure that the
> crafted data will fit a single DTLS packet).
> 
> >
> https://mailarchive.ietf.org/arch/msg/dots/Gbtf8bBWpxD9CWNBhS_TZtsW
> eP4
> > /
> > >
> Dots mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/dots

---

From: Achim Kraus <[email-masked]>
Date: Tue, 7 Apr 2020 21:04:00 +0200

Hi Jon,

I'm still not sure, if such an approach would make sense:
- if multiple blocks are required without gaps
- the package loss is high
=> that ends up in not receiving any "large payload" at all.

FMPOV, the first thing to ensure is, that the "large payload" gets split
into "application blocks", which could be processed even when other
application blocks are missing.

I would also not assume, that a "in between proxies" will be able to
process blockwise resources with gaps. So I'm not sure, why
observer/notify is used. If it should be used, (and a proxy will not
work,) then you may consider to "misuse" the notifies and send your
"application blocks" just in multiple notifies (each application block
in one notify, without droping "old" notifies, because they are not old
:-) ). In my opinion that would require less changes to existing
implementations than introduce a "blockwise without next".

In the end, it would be interesting, if such a improvement really pays
off. Using some nodes with standard blockwise and others with the new
one should show the benefit. It would be great, if you then report your
numbers.

best regards
Achim

---

From: "Jon Shallow" <[email-masked]>
Date: Tue, 7 Apr 2020 21:01:08 +0100

Hi Achim,

It is good to get this feedback.  Please see inline Jon1>

Regards

Jon

> From: Dots [mailto: [email-masked]] On Behalf Of Achim Kraus
> Sent: 07 April 2020 20:04
> To: Jon Shallow; [email-masked]; [email-masked];
> [email-masked]

Jon> The packet loss should be transitory in that if the DDoS Mitigation is effective and reduces the rate of the DDoS Attack through the pipe normal (certainly DOTS) service is resumed.  DOTS needs to be able to continue with limiting working with high packet loss as "PUT new mitigation request" are still likely to get through.

> application blocks are missing.

Jon> We are also trying to do this as well, but it is not that straightforward.
> process blockwise resources with gaps.

Jon> In the DOTS case, we have the concept of DOTS gateways where the DOTS CoAP server and DOTS CoAP client stacks are fully terminated making things easier here.  This is a good point though and needs to be worked through.

> So I'm not sure, why
> work,) 

Jon> I don't quite follow why a proxy will not work unless you are referring to blockwise resources with gaps.

> then you may consider to "misuse" the notifies and send your

Jon> It may make for less changes, but I was expecting the NONBLOCK2 option to be handled within the application rather than at the CoAP layer.  In the DOTS case, server enhanced Telemetry information sending has to be explicitly enabled by the client so existing implementations will not get broken.

> one should show the benefit. It would be great, if you then report your

Jon> I will see what I can come up with here.  A good suggestion.  It may take a while though as I have only partially got the DOTS telemetry draft extensions coded.
~Jon

---

From: Achim Kraus <[email-masked]>
Date: Tue, 7 Apr 2020 22:26:31 +0200

Hi Jon,

Jon> I don't quite follow why a proxy will not work unless you are
referring to blockwise resources with gaps.

Yes, the point is the blockwise with gaps.

Jon> It may make for less changes, but I was expecting the NONBLOCK2
option to be handled within the application rather than at the CoAP
layer.  In the DOTS case, server enhanced Telemetry information sending
has to be explicitly enabled by the client so existing implementations
will not get broken.

If it's not a modification of the coap-layer, which kind of messages are
you plan to use?
Multiple NON-responses (wouldn't that be notifies)?
Or NON requests with "no-response"?

best regards
Achim

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 7 Apr 2020 23:18:58 +0200

Hi Achim,

> application blocks are missing.

Indeed, “application layer framing” comes to mind here; that is always a good idea if it cannot be ensured that all messages make it or if it is good to be able to process messages while still waiting for others.

                     .oOo.

I’m trying to understand the very specific network environment that we are targeting here.
Are we assuming packets are way more likely to make it from the server to the client than the other way around?
That indeed would call for additional capabilities that base CoAP does not have.
We also may not really care about congestion control much in a situation of massive (attacker-induced) congestion (although that might be misdiagnosed, so some care is still required); which would be another reason to maybe deviate from base CoAP.

I don’t have a design in mind at the moment.  It would need to cater for the fact that sending the other response messages for the request would be on the initiative of the server.
So observe (with non-confirmable responses) is maybe a good model indeed; what’s missing is some way to ask for gaps.

I just resubmitted a draft that we have been discussing for a while in T2TRG:
The Series Transfer Pattern (STP)
https://www.ietf.org/id/draft-bormann-t2trg-stp-02.html

This has some discussion that may be relevant here, although it does not address the specific DOTS problem.  I think it would be great if whatever we come up with to solve this problem would also be a step forward on the larger class of applications needing the series transfer pattern.

Grüße, Carsten

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 8 Apr 2020 00:28:05 +0200

> https://www.ietf.org/id/draft-bormann-t2trg-stp-02.html

https://www.ietf.org/id/draft-bormann-t2trg-stp-03.html

(forgot to process the figures.)

Grüße, Carsten

---

From: Christian Amsüss <[email-masked]>
Date: Wed, 8 Apr 2020 11:04:36 +0200

Hello Med,

> I don't see where in the two drafts an observer can request a particular missing fragment. 

Observation combined with block-wise transfer generally has the observer
request all the remaining blocks when an updates comes in, as
illustrated in [1]. Those do not even necessarily need to be requested
in sequence.

If a mechanism gets added that allows the server to send additional
blocks after a first request (and I'd appreciate that, maybe we can get
that resurfaced in the upcoming meeting's AOB section), the client may
miss some of them, but can still fall back to that original mechanism.

Kind regards
Christian

[1]: https://tools.ietf.org/html/rfc7959#section-3.4

-- 
To use raw power is to make yourself infinitely vulnerable to greater powers.
  -- Bene Gesserit axiom

---

From: <[email-masked]>
Date: Wed, 8 Apr 2020 09:56:13 +0000

Hi Carsten,

We are also considering how to solve the issue at the DOTS level. The good news is that we are not blindly overriding pieces of data that are received in distinct responses. We have some checks to update an active record. But as mentioned by Jon, there are some challenges as well. 

With regards to the network environment, the direction of the attack is the same as the one from servers to clients. That’s said, telemetry data can be exchanged in both directions:

* from client to servers: using a dedicated telemetry message or in an efficacy update shared with the server when a mitigation is active. This is done using PUT messages.
* from servers to clients: telemetry data can be sent in dedicated notification messages or as part of the mitigation status update. Both rely upon GET+Observe. Having a mechanism to ask for gaps would thus be helpful.

Cheers,
Med

> De : Carsten Bormann [mailto:[email-masked]]
> Envoyé : mardi 7 avril 2020 23:19
> À : Achim Kraus
> Cc : Jon Shallow; BOUCADAIR Mohamed TGI/OLN; core; [email-masked]
> Objet : Re: [core] [Dots] Large asynchronous notifications under DDoS:
> New BLOCK Option?
> 
> Hi Achim,
> 
> Indeed, “application layer framing” comes to mind here; that is always
> a good idea if it cannot be ensured that all messages make it or if it
> is good to be able to process messages while still waiting for others.
> 
>                      .oOo.
> 
> We also may not really care about congestion control much in a
> situation of massive (attacker-induced) congestion (although that
> might be misdiagnosed, so some care is still required); which would be
> another reason to maybe deviate from base CoAP.
> 
> I don’t have a design in mind at the moment.  It would need to cater
> for the fact that sending the other response messages for the request
> would be on the initiative of the server.
> So observe (with non-confirmable responses) is maybe a good model
> indeed; what’s missing is some way to ask for gaps.
> 
> This has some discussion that may be relevant here, although it does
> not address the specific DOTS problem.  I think it would be great if
> whatever we come up with to solve this problem would also be a step
> forward on the larger class of applications needing the series
> transfer pattern.
> 
> Grüße, Carsten

---

From: "Jon Shallow" <[email-masked]>
Date: Wed, 8 Apr 2020 11:41:08 +0100

Hi All,

Thanks for all your input so far.  It has helped my thinking, but I still do not have a clear answer yet as how to best do this for the DOTS specific situation as well as more general use cases.

Below is a more graphical way of trying to describe what is happening and how it may be possible to overcome some of the limitations of using BLOCK2 in a lossy traffic environment (caused by DDoS attacks).

Regards

Jon

Primary packet loss is from Server to Client
[Server is DDoS mitigating out in Internet, Client is on premise, DDoS flood against client]

GET followed by Observe response - all working - as we would do it today.

       CLIENT      SERVER
         |          |
         +--------->|   GET /path Token 0xf0 Observe 0
         |          |
         |<---------+   2.05 Token 0xf0 Observe 1234 Block2 0/1/1024
         |          |
         +--------->|   GET /path Token 0xf0 Block2 1/0/1024
         |          |
         |<---------+   2.05 Token 0xf0 Observe 1234 Block2 1/1/1024
         |          |
         +--------->|   GET /path Token 0xf0 Block2 2/0/1024
         |          |
         |<---------+   2.05 Token 0xf0 Observe 1234 Block2 2/1/1024
         |          |
         +--------->|   GET /path Token 0xf0 Block2 3/0/1024
         |          |
         |<---------+   2.05 Token 0xf0 Observe 1234 Block2 3/0/1024
Observe triggered
         |<---------+   2.05 Token 0xf0 Observe 1235 Block2 0/1/1024
         |          |
         +--------->|   GET /path Token 0xf1 Block2 1/0/1024
         |          |
         |<---------+   2.05 Token 0xf1 Observe 1235 Block2 1/1/1024
         |          |
         +--------->|   GET /path Token 0xf1 Block2 2/0/1024
         |          |
         |<---------+   2.05 Token 0xf1 Observe 1235 Block2 2/1/1024
         |          |
         +--------->|   GET /path Token 0xf1 Block2 3/0/1024
         |          |
         |<---------+   2.05 Token 0xf1 Observe 1235 Block2 3/0/1024

Confirmable ACK responses not displayed, nor Etag, Size2 or Maxage.

Now with some packet loss.
Observe triggered
         |<---------+   2.05 Token 0xf0 Observe 1236 Block2 0/1/1024
         |          |
         +--------->|   GET /path Token 0xf2 Block2 1/1/1024
         |          |
         |   X<-----+   2.05 Token 0xf2 Observe 1236 Block2 1/1/1024
         |          |
Timeout
Retry if Confirmable
         +--------->|   GET /path Token 0xf2 Block2 1/1/1024
         |          |
         |   X<-----+   2.05 Token 0xf2 Observe 1236 Block2 1/1/1024
         |          |
Retries continue - eventually timing out and possibly killing CoAP session.

Communications can be locked out for 90 seconds as CoAP NSTART is 1.

[DOTS uses NON-Confirmable for protocol reliability, not traffic reliability]

If not using Confirmable, then the Client needs to time out at the application layer and retry, but can only go forward 1 block at a time (does not have to be in sequential order).
Server may need to garbage collect on resource with Etag.

What may help the situation (but client can decide when current Etag/Maxage is no longer valid and stop continue requesting, and server may still need to garbage collect)

New CoAP Option NONBLOCK2 equivalent to BLOCK2, but does not rely on client doing GET to get the next block synchronously.

       CLIENT      SERVER
         |          |
         +--------->|   GET /path Token 0xf0 Observe 0 NonBlock2 0/0/1024
         |          |
         |<---------+   2.05 Token 0xf0 Observe 1234 NonBlock2 0/1/1024
         |          |
         |<---------+   2.05 Token 0xf0 Observe 1234 NonBlock2 1/1/1024
         |          |
         |<---------+   2.05 Token 0xf0 Observe 1234 NonBlock2 2/1/1024
         |          |
         |<---------+   2.05 Token 0xf0 Observe 1234 NonBlock2 3/0/1024
Observe triggered
         |<---------+   2.05 Token 0xf0 Observe 1235 NonBlock2 0/1/1024
         |          |
         |<---------+   2.05 Token 0xf0 Observe 1235 NonBlock2 1/1/1024
         |          |
         |<---------+   2.05 Token 0xf0 Observe 1235 NonBlock2 2/1/1024
         |          |
         |<---------+   2.05 Token 0xf0 Observe 1235 NonBlock2 3/0/1024
Observe triggered
         |<---------+   2.05 Token 0xf0 Observe 1236 NonBlock2 0/1/1024
         |          |
         |   X<-----+   2.05 Token 0xf0 Observe 1236 NonBlock2 1/1/1024
         |          |
         |   X<-----+   2.05 Token 0xf0 Observe 1236 NonBlock2 2/1/1024
         |          |
         |<---------+   2.05 Token 0xf0 Observe 1236 NonBlock2 3/0/1024
Client realises blocks are missing and asks for the missing ones in one go
         +--------->|   GET /path Token 0xf1 NonBlock2 1/0/1024 NonBlock2 2/0/1024
         |          |
         |   X<-----+   2.05 Token 0xf1 Observe 1236 NonBlock2 1/1/1024
         |          |
         |<---------+   2.05 Token 0xf1 Observe 1236 NonBlock2 2/1/1024
Get final missing block
         +--------->|   GET /path Token 0xf2 NonBlock2 1/0/1024
         |          |
         |<---------+   2.05 Token 0xf2 Observe 1236 NonBlock2 1/1/1024

Obviously the 3 second minimum for RTT calculations needs to be maintained so that the Attack situation is not made worse.  However, I think it acceptable that all the NonBlock2s for a particular data set are all sent as a stream.

> From: Dots [mailto: [email-masked]] On Behalf Of [email-masked]
> Sent: 08 April 2020 10:56
> To: Carsten Bormann
> Cc: Jon Shallow; [email-masked]; core
> We are also considering how to solve the issue at the DOTS level. The good
> news is that we are not blindly overriding pieces of data that are received in
> distinct responses. We have some checks to update an active record. But as
> mentioned by Jon, there are some challenges as well.
> 
> With regards to the network environment, the direction of the attack is the
> same as the one from servers to clients. That’s said, telemetry data can be
> exchanged in both directions:
> 
> * from client to servers: using a dedicated telemetry message or in an
> efficacy update shared with the server when a mitigation is active. This is
> done using PUT messages.
> * from servers to clients: telemetry data can be sent in dedicated
> notification messages or as part of the mitigation status update. Both rely
> upon GET+Observe. Having a mechanism to ask for gaps would thus be
> helpful.
> 
> Cheers,
> Med
>

---

From: Achim Kraus <[email-masked]>
Date: Wed, 8 Apr 2020 18:00:08 +0200

Hi Jon,

a "clear" answer will be hard to give, without actual experience of such
special communication situations. FMPOV it would require some
experiments under your assumed conditions to find answers.

 From your diagrams, I would conclude:
- best, if the transfer works without sending messages back
- to fill gaps, sending messages will be acceptable
- both together, should then provide a "good enough" solution.
Yes, experiments may show, that it works "good enough", or withdraw it
(because the drops are too high).

To some details of your sequences.
Observe/Notify is defined in
https://tools.ietf.org/html/rfc7959#section-3.4

- the follow up / next block transfers usually don't contain the observe
option, only the "head".
- the follow up / next block transfer don't need to share the token with
the head, they use the tokens defined by each requests.

In your scenario, there will be no "next block" request, and so you
reuse the observer "token". There maybe implementations, which fail
receiving follow up blocks, with observe options and the token of the
observer request.

Therefore your approach would require at least a implementation
according that special interpretation, but will not work with all "RFC
7252-7641-7959" compliant implementations, because for me it adds
special conventions.

best regards
Achim

---

From: Achim Kraus <[email-masked]>
Date: Wed, 8 Apr 2020 20:55:40 +0200

Hi Jon,

let me try to demonstrate, how “application layer framing” could look like:

> New CoAP Option NONBLOCK2 equivalent to BLOCK2, but does not rely on client doing GET to get the next block synchronously. >
>
>         CLIENT      SERVER
>           |          |
Do these messages in the application layer -> encode the messages => put
the encoded message as payload for ordinary notifies:

          CLIENT      SERVER
            |          |
            +--------->|   GET /path Token 0xf0 Observe 0 NonBlock2 0/0/1024
            |          |
            |<---------+   2.05 Token 0xf0 Observe 1234 { 2.05 Token
0xf0 Observe abc NonBlock2 0/1/1024 }
            |          |
            |<---------+   2.05 Token 0xf0 Observe 1235 { 2.05 Token
0xf0 Observe abc NonBlock2 1/1/1024 }
            |          |
            |<---------+   2.05 Token 0xf0 Observe 1236 { 2.05 Token
0xf0 Observe abc NonBlock2 2/1/1024 }
            |          |
            |<---------+   2.05 Token 0xf0 Observe 1237 { 2.05 Token
0xf0 Observe abc NonBlock2 3/0/1024 }

The outer notifies will be processed as usual, the application block
logic is yours.

Observe triggered
            |<---------+   2.05 Token 0xf0 Observe 1238 { 2.05 Token
0xf0 Observe abd NonBlock2 0/1/1024 }
            |          |
            |<---------+   2.05 Token 0xf0 Observe 1239 { 2.05 Token
0xf0 Observe abd NonBlock2 1/1/1024 }
            |          |
            |<---------+   2.05 Token 0xf0 Observe 1240 { 2.05 Token
0xf0 Observe abd NonBlock2 2/1/1024 }
            |          |
            |<---------+   2.05 Token 0xf0 Observe 1241 { 2.05 Token
0xf0 Observe abd NonBlock2 3/0/1024 }

Observe triggered
            |<---------+   2.05 Token 0xf0 Observe 1238 { 2.05 Token
0xf0 Observe abd NonBlock2 0/1/1024 }
            |          |
            |  X<------+   2.05 Token 0xf0 Observe 1239 { 2.05 Token
0xf0 Observe abd NonBlock2 1/1/1024 }
            |          |
            |  X<------+   2.05 Token 0xf0 Observe 1240 { 2.05 Token
0xf0 Observe abd NonBlock2 2/1/1024 }
            |          |
            |<---------+   2.05 Token 0xf0 Observe 1241 { 2.05 Token
0xf0 Observe abd NonBlock2 3/0/1024 }

            +--------->|   GET /path/ctrl Token 0xf1 { NonBlock2
1/0/1024 NonBlock2 2/0/1024 }
            |          |
            |  X<------+   2.05 Token 0xf0 Observe 1242 { 2.05 Token
0xf0 Observe abd NonBlock2 1/1/1024 }
            |          |
            |<...------+   2.05 Token 0xf0 Observe 1243 { 2.05 Token
0xf0 Observe abd NonBlock2 2/1/1024 }

Get final missing block
            +--------->|   GET /path/ctrl Token 0xf2 { / NonBlock2
1/0/1024 }
            |          |
            |<...------+   2.05 Token 0xf0 Observe 1244 { 2.05 Token
0xf0 Observe abd NonBlock2 1/1/1024 }

That should be not considered as "precise description of the application
layer framing". It should just give a first impression, how coap
standards could be used, and the specific stuff could be move to the
application layer (reusing coap again).

best regards
Achim

---

From: "Jon Shallow" <[email-masked]>
Date: Wed, 8 Apr 2020 20:25:25 +0100

Hi Achim

Thanks for this good feedback.  Please see inline Jon>
[I have just seen you sent an update which I will look at shortly]

Regards

Jon

> Sent: 08 April 2020 17:00
> To: Jon Shallow; [email-masked]; [email-masked];
> [email-masked]; [email-masked]

Jon> Yes
> - the follow up / next block transfers usually don't contain the observe

Jon> My mistake - only head should have the observe option.

> the head, they use the tokens defined by each requests.

Jon> Fair enough.  I was just using the 0xfc in Figure 13.

> receiving follow up blocks, with observe options and the token of the

Jon> Is this still true if Observe option is only in the head?  I would be using Etag.  The head observe triggered block is the same as with BLOCK2.

Jon> The GET for the missing responses would have Etag so we can be sure we are asking for the right thing.

Jon> The initial GET that sets up the observe subscription deliberately uses NonBlock2 so only client and servers that support nonBLock2 will use it.
> 7252-7641-7959" compliant implementations, because for me it adds

Jon> Removing the confusion about Observe option, I am struggling to think of other reasons why it may not work - but I am on a rapid learning curve here.
~Jon

> > Thanks for all your input so far.  It has helped my thinking, but I still do not
> have a clear answer yet as how to best do this for the DOTS specific situation
> as well as more general use cases.
> >
> > Below is a more graphical way of trying to describe what is happening and
> how it may be possible to overcome some of the limitations of using
> BLOCK2 in a lossy traffic environment (caused by DDoS attacks).
> >
> > Regards
> >
> > Jon
> >
> > If not using Confirmable, then the Client needs to time out at the
> application layer and retry, but can only go forward 1 block at a time (does
> not have to be in sequential order).
> > What may help the situation (but client can decide when current
> Etag/Maxage is no longer valid and stop continue requesting, and server
> may still need to garbage collect)
> >
> > New CoAP Option NONBLOCK2 equivalent to BLOCK2, but does not rely on
> client doing GET to get the next block synchronously.
> >
> >
> >         CLIENT      SERVER
> >           |          |
> > Obviously the 3 second minimum for RTT calculations needs to be
> maintained so that the Attack situation is not made worse.  However, I think
> it acceptable that all the NonBlock2s for a particular data set are all sent as a
> stream.
> >
> >
> >

---

From: "Jon Shallow" <[email-masked]>
Date: Wed, 8 Apr 2020 20:48:21 +0100

Hi Achim,

Having the DOTS CBOR data wrapped in CoAP and then wrapped in CoAP is interesting and could be a way forward here as it gives a lot of flexibility.

As mentioned in my previous response, looking at your suggestion, "Observe abc" or "Observe abd" are not really needed and could be replaced "Etag abc" and "Etag abd" respectively. 
Then with the GET to request the missing blocks could also include the appropriate Etag to make sure that the correct blocks for the resource are being asked for.

Then it may be possible to not have the double CoAP layer and just have the NonBlock2 option.

Regards

Jon

> Sent: 08 April 2020 19:56
> client doing GET to get the next block synchronously. >
> >
> >         CLIENT      SERVER
> >           |          |

---

From: <[email-masked]>
Date: Wed, 8 Apr 2020 21:04:18 +0000

Hi Achim, 

> 1/0/1024 NonBlock2 2/0/1024 }

Do existing CoAP implementations (libcoap, for example) support GETs with a payload?

Cheers,
Med

> Envoyé : mercredi 8 avril 2020 20:56
> À : Jon Shallow; BOUCADAIR Mohamed TGI/OLN; [email-masked];

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 8 Apr 2020 23:08:10 +0200

> Do existing CoAP implementations (libcoap, for example) support GETs with a payload?

That is what FETCH is for.

Grüße, Carsten

---

From: <[email-masked]>
Date: Wed, 8 Apr 2020 21:22:26 +0000

Re-,

Thanks, Carsten. 

This would mean that legacy CoAP implems do not support the coap-in-coap proposal without modification (given that rfc8132 support will be required). 

Actually, I'm struggling to understand what problem is solved by defining the NonBlock2 option but use it only with an extra coap layering. 

Cheers,
Med

> Envoyé : mercredi 8 avril 2020 23:08
> Cc : Achim Kraus; Jon Shallow; [email-masked]; [email-masked]

---

From: Achim Kraus <[email-masked]>
Date: Thu, 9 Apr 2020 07:59:32 +0200

Hi Med,

maybe, we have a different understanding of the CoAP principles.

RFC 7252:
single request-responses are matched by their tokens, chosen by the
client. The server must include that token in it's response and must not
make assumptions about it.

RFC 7641:
single request - multiple responses are matched by their tokens, chosen
by the client. The server must include that token in all responses and
must not make assumptions about it.

RFC 7959:
multiple request - multiple responses are used to transmit a large
resource using the principals of RFC 7252/7641.
RFC 7252 each single block transfer matches the single block request
with it's single block response by a token chosen by the client. The
server can not assume, that the tokens of several block-requests are
related.
RFC 7641 "single request - multiple responses" applies only to the head.
the download of the left blocks doesn't reuse the token, nor is the
server allowed, to make assumptions about the token.

=> you can't send the follow up blocks, because the token to do so is
not defined. It would have been defined by the (missing) client request.
If the token of the observe is used, the server makes a assumption and
relate request of RFC7959, which are not related by the token.

Therefore I asked also about the usage of observer/notify. If a PUT/POST
is used, then your transfer looks just pretty much as a blockwise with
NSTART-X. The drawback there will be the missing "blocksize" negotiation
at the head. If that is acceptable, it should have a chance comparable
to the proposal.

The NonBlock2 solution is not bad on it's own, but it disrupt the
principals for the tokens on the server side, at least in my understanding.

---------------------------------------------------------------------

 > Do existing CoAP implementations (libcoap, for example) support GETs
with a payload?

I know, it looks a little unfair to point on single items of other
example and then make buggy examples on my own :-). But I already admit
"That should be not considered as "precise description of the
application layer framing"".
In fact, there is no outer GET required. The outer request could also be
POST.

              +--------->|   POST /path/ctrl Token 0xf1 { GET /path
Token cde NonBlock2 1/0/1024 NonBlock2 2/0/1024 }

Just to mention, that this request triggers the notifies with the
missing block is "application layer convention". You may use whatever
definition you want to tell the server to (re-)send notifies with
blocks, there is no need of that inner GET request.

---------------------------------------------------------------------

best regards
Achim

---

From: <[email-masked]>
Date: Thu, 9 Apr 2020 07:01:55 +0000

Hi Achim,

First, I was not looking to "make your example buggy". I was trying to understand how this would work without requiring changes to the base specs. Your feedback is highly appreciated and very useful. Thank you.

So far we have three approaches:

(1) The use of nonblock2 option with a "relax" of the behavior at the server to send the fragments with the same token and without waiting for GETs. 

(2) The use of a shim application as you suggested, which requires an update on the server side but also on how clients on how to fill the gaps (use of PUT/FETCH). 

(3) A DOTS specific approach to build its own chunks and signal blocks as part of the CBOR. These blocks are handled as atomic notifications. If a block is missing, the client can use GET+Query to retrieve the gap. We don't require any modification at the base CoAP specs. The issue with this one is to decide what data to put in the blocks to avoid that when a block is passed to other layers, the overheads won't lead to fragmentation.
  
It seems to me that (2) is a little bit more complex vs (1). 

Cheers,
Med

> Envoyé : jeudi 9 avril 2020 08:00
> À : BOUCADAIR Mohamed TGI/OLN; Carsten Bormann
> Cc : Jon Shallow; [email-masked]; [email-masked]
>  > Do existing CoAP implementations (libcoap, for example) support
> GETs
> with a payload?
> 
> 
> > This would mean that legacy CoAP implems do not support the coap-in-
> coap proposal without modification (given that rfc8132 support will be
> required).
> >
> > Actually, I'm struggling to understand what problem is solved by
> defining the NonBlock2 option but use it only with an extra coap
> layering.
> >
> > Cheers,
> > Med
> >
> >>> Do existing CoAP implementations (libcoap, for example) support
> GETs
> >> with a payload?
> >>

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 9 Apr 2020 09:17:40 +0200

I apologize in advance for chiming in before having digested the whole thread.

I think that the idea of using observe with a modified block-wise protocol (1 below) can work.

I also think that application-layer framing (3 below) is useful.

I don’t see a contradiction between the two; they might be used together.

I would like to know how to handle two sub-problems here:

(a) using non-confirmable responses without additional requests brings us into PROBING_RATE territory.  What is actually the rate at which this exchange is supposed to happen?  Do we have other ways to manage capacity of the response path (a.k.a. congestion control)?  I.e., how does the server know which rate (e.g., for pacing) it should use for sending further messages?

(b) the semantics of observe is that a notification is the whole new state of the resource.  Proxies will implement it that way.  Of course block2 modifies this semantics a bit, so nonblock2 might do that too.  Still, I think we need to consider what proxies (or client caches) will make out of the mechanism we devise.

Grüße, Carsten

> On 2020-04-09, at 09:01, <[email-masked]> <[email-masked]> wrote:
> 
> Hi Achim,
>

---

From: <[email-masked]>
Date: Thu, 9 Apr 2020 07:56:16 +0000

Hi Carsten,

Please see inline. 

Cheers,
Med

> Envoyé : jeudi 9 avril 2020 09:18
> (a) using non-confirmable responses without additional requests brings
> us into PROBING_RATE territory.  What is actually the rate at which
> this exchange is supposed to happen?  Do we have other ways to manage
> capacity of the response path (a.k.a. congestion control)?  I.e., how
> does the server know which rate (e.g., for pacing) it should use for
> sending further messages?

[Med] 

(1)

Probing rate is negotiated during idle time between the client and server:

      probing-rate:  The average data rate that must not be exceeded by
         a DOTS agent in sending to a peer DOTS agent that does not
         respond (referred to as PROBING_RATE parameter in CoAP).

(2)

The probing-rate used in idle time may not be the same as the one using during at attack time. We do a value for each of them.

(3) in addition, we negotiate this additional "guard":

   DOTS agents MUST NOT send pre-or-ongoing-mitigation telemetry
   messages to the same peer more frequently than once every 'telemetry-
   notify-interval'

> 
> (b) the semantics of observe is that a notification is the whole new
> state of the resource.  Proxies will implement it that way.  Of course
> block2 modifies this semantics a bit, so nonblock2 might do that too.
> Still, I think we need to consider what proxies (or client caches)
> will make out of the mechanism we devise.

[Med] Agree for the generic CoAP case. 

For the particular case of DOTS, sessions are established hop-by-hp when a proxy (we called it, gateway) is involved. We have the full visibility on what happens.   

> > First, I was not looking to "make your example buggy". I was trying
> to understand how this would work without requiring changes to the
> base specs. Your feedback is highly appreciated and very useful. Thank
> you.
> >
> > (1) The use of nonblock2 option with a "relax" of the behavior at
> the server to send the fragments with the same token and without
> waiting for GETs.
> >
> > (2) The use of a shim application as you suggested, which requires
> an update on the server side but also on how clients on how to fill
> the gaps (use of PUT/FETCH).
> >
> > (3) A DOTS specific approach to build its own chunks and signal
> blocks as part of the CBOR. These blocks are handled as atomic
> notifications. If a block is missing, the client can use GET+Query to
> retrieve the gap. We don't require any modification at the base CoAP
> specs. The issue with this one is to decide what data to put in the
> blocks to avoid that when a block is passed to other layers, the
> overheads won't lead to fragmentation.
> >
> >> client. The server must include that token in it's response and
> must
> >> not
> >> In fact, there is no outer GET required. The outer request could
> also
> >> be
> >> POST.
> >>

---

From: <[email-masked]>
Date: Thu, 9 Apr 2020 08:08:12 +0000

Carsten, 

I forgot to mention for this additional "guard": it applies for the "full" notification (not it fragments).

Cheers,
Med

> 
> The probing-rate used in idle time may not be the same as the one
> using during at attack time. We do a value for each of them.
>

---

From: "Jon Shallow" <[email-masked]>
Date: Thu, 9 Apr 2020 10:20:06 +0100

Hi Achim,

Some clarifications on your excellent comments - I now understand the thinking behind the Token usage.

Please see inline Jon>

Regards

Jon

> Sent: 09 April 2020 08:02
> To: Achim Kraus; Carsten Bormann
> Cc: Jon Shallow; [email-masked]; [email-masked]
> First, I was not looking to "make your example buggy". I was trying to
> understand how this would work without requiring changes to the base
> specs. Your feedback is highly appreciated and very useful. Thank you.
> 
> (1) The use of nonblock2 option with a "relax" of the behavior at the server
> to send the fragments with the same token and without waiting for GETs.
> 
> (2) The use of a shim application as you suggested, which requires an
> update on the server side but also on how clients on how to fill the gaps
> (use of PUT/FETCH).
> 
> (3) A DOTS specific approach to build its own chunks and signal blocks as
> part of the CBOR. These blocks are handled as atomic notifications. If a block
> is missing, the client can use GET+Query to retrieve the gap. We don't
> require any modification at the base CoAP specs. The issue with this one is
> to decide what data to put in the blocks to avoid that when a block is passed
> to other layers, the overheads won't lead to fragmentation.
> 

Jon> With the new nonblock2 option, the non-head blocks can have no token, so no assumptions need to be made about the token.  Etag however will need to be used for association of the nonblock2 chunk with the correct data set.
> > relate request of RFC7959, which are not related by the token.

Jon> Again, by not having a token for the non-head blocks, this issue is resolved I believe.

> > at the head. If that is acceptable, it should have a chance comparable

Jon> Yes, certainly the server can initiate a PUT/POST back to the client (I like this outside of the box thinking).  

Jon> However, for me this breaks the spirit of Observe in that whenever a resource changes, all the subscribers are currently notified with a unsolicited response - here it is using a POST/PUT instead.

> > principals for the tokens on the server side, at least in my

Jon> So, if the principals of Tokens is solved (by not having them!) then this may work.

Jon> On further reflection, as this sending all the blocks stuff can also be done in the other direction, I think a better name for nonblock2 would be block4 (server to client aka block2) and block3 (client to server - aka block1).  Bloc4 and block3 can be NON-Confirmable or CONfirmable.

~jon

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 9 Apr 2020 11:22:31 +0200

On 2020-04-09, at 11:20, Jon Shallow <[email-masked]> wrote:
> 
> With the new nonblock2 option, the non-head blocks can have no token

CoAP messages always have a token.

Grüße, Carsten

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 9 Apr 2020 11:26:35 +0200

Hi Med,

thank you for updating me on this information!

> For the particular case of DOTS, sessions are established hop-by-hp when a proxy (we called it, gateway) is involved. We have the full visibility on what happens.   

So you have application-aware proxies (which may not even be CoAP proxies).

But that doesn’t mean other proxies aren’t involved; there is also the matter of client caches, which have similar properties (caching, but no multiplexing).  So far, we have tried to keep the proxy/client-caching concept valid with extensions on CoAP; this would be the first one where we would have to say “no CoAP proxies can be used”.  I was wondering whether we can avoid this situation (can = don’t make the solution too complex).

Grüße, Carsten

---

From: "Jon Shallow" <[email-masked]>
Date: Thu, 9 Apr 2020 10:27:34 +0100

Hi Carsten,

I stand corrected - a token with 0 length.

Regards

Jon

> From: Dots [mailto: [email-masked]] On Behalf Of Carsten Bormann
> Sent: 09 April 2020 10:23
> To: Jon Shallow
> Cc: [email-masked]; [email-masked]; [email-masked]; Achim
> Kraus

---

From: Achim Kraus <[email-masked]>
Date: Thu, 9 Apr 2020 13:02:24 +0200

Hi Jon,
Hi Med,

 > (1) The use of nonblock2 option with a "relax" of the behavior at the
server to send the fragments with the same token and without waiting for
GETs.

The point will be, how this relaxation actually looks like.
FMPOV, the hard thing will be:
The server MUST not make assumptions about the token, except that it is
copied from the clients request into the response.

Currently the followup blocks considered to have a empty token (Jon's
last note). That will hit any pending client requests with such a empty
token. The matching is considered to be done via the etag. But RFC 7252
defines the etag with:
"An entity-tag is intended for use as a resource-local identifier".
Though it's an resource local identifier, in my opinion, you can't use
that to relate your blocks. The right blocks may have that etag, but
others responses may have it as well (therefore "resource-local
identifier").
So, in my opinion, to know, to which resource the etags belongs to, the
related request is required. Usually matched by the token, but that
token is missing, because the client's request is missing.

best regards
Achim

---

From: <[email-masked]>
Date: Thu, 9 Apr 2020 11:26:07 +0000

Re,

(1)

We don't have an issue with other CoAP proxies as we use a distinct port number for DOTS.

(2)

For the pure CoAP case, the newBlock options will be marked as critical and unsafe to forward. There is no caching issue for a proxy that does not understand the option. GET with this option will be rejected by the proxy. The server will reject requests with the newBlock if it does not understand it. This is simple. 

For a proxy that supports block options, the blockwise spec does not impose to have all fragments. The spec leaves it open to implementations: if we don't have a loss, all fragments will be present. The proxy can be serve clients with individual blocks. 

We won't require all fragments to be reassembled at the proxy. We just need individual blocks. For example, if the client-proxy leg is lossy, the proxy can serve immediately the client with missing blocks. This will be part of the new behavior (which is fine given that these proxies are assumed to support the option).

Clients can determine the initial block size in initial newBlock. If the server does not like it and wants to send back a reduced size, that would be fine and the client would see one of the newBlock with smaller size and know to ask for the smaller sizes. 

I agree that we need to further assess this carefully but we are not sure yet there is an issue with regards to caching.

Cheers,
Med

> Envoyé : jeudi 9 avril 2020 11:27
> > For the particular case of DOTS, sessions are established hop-by-hp
> when a proxy (we called it, gateway) is involved. We have the full
> visibility on what happens.
> 
> But that doesn’t mean other proxies aren’t involved; there is also the
> matter of client caches, which have similar properties (caching, but
> no multiplexing).  So far, we have tried to keep the proxy/client-
> caching concept valid with extensions on CoAP; this would be the first
> one where we would have to say “no CoAP proxies can be used”.  I was
> wondering whether we can avoid this situation (can = don’t make the
> solution too complex).
> 
> Grüße, Carsten

---

From: "Jon Shallow" <[email-masked]>
Date: Thu, 9 Apr 2020 12:27:17 +0100

Hi Achim,

I hear you.

How about extending the NonBLock2 option to contain more information.  E.g. 

   o  the size of the block (SZX);

   o  whether more blocks are following (M);

   o  the relative number of the block (NUM) within a sequence of blocks
      with the given size.

   o  the block set id (like the IPv4 fragment id field) for re-assembly.

Thus making the nonblock2 option length 0 to, say, 7 bytes long. 

Regards

Jon 

> Sent: 09 April 2020 12:02
> To: Jon Shallow; 'Carsten Bormann'; [email-masked];