Subject: [core] Common request/response API

From: "Kovatsch Matthias" <[email-masked]>
Date: Thu, 6 Aug 2015 14:46:30 +0000

Dear list

In Prague, the problem of a common request/response API for all possible transports was raised. We now have something similar in the Californium project, where we want to improve the error handling for the CoapClient API as well as improve the information flow between Scandium (DTLS) and Californium (or rather the application code).

As a start, I collected possible exceptions that could signal what went wrong. While this might be a bit Java-centric, the intent is universal: have a common set of signals to tell the application above the request/response API what went wrong.

Network layer
- DestinationNotReachableException
- PacketTooBigException

Transport layer
- ConnectionRefusedException
- AuthenticationException
- TooManyOpenFilesException / InsufficientResourcesException (more OS-related, but happens at the socket level)

Application layer
- TimeoutException / NoReplyException
- InvalidURIException / InvalidTransportException (alternative transports API)
- RejectedException (not sure if the same as ConnectionRefusedException)

It would be great, if you could have a look, suggest more exceptions/signals/error codes, propose thinning/merging, or just comment on the direction. 

Ciao
Matthias

---

From: "Carey, Timothy (Timothy)" <[email-masked]>
Date: Thu, 6 Aug 2015 19:28:24 +0000

Matthias,

In your view the Application Layer that would receive these indications is the Request/Response Layer which then you return the appropriate Error code?

BR,
Tim

-----Original Message-----
From: Kovatsch Matthias [mailto:[email-masked]] 
Sent: Thursday, August 06, 2015 9:47 AM
To: core WG
Subject: [core] Common request/response API

Dear list

Network layer

Transport layer

Application layer

Ciao
Matthias

---

From: "Kovatsch Matthias" <[email-masked]>
Date: Fri, 7 Aug 2015 13:18:10 +0000

Hi Tim

> In your view the Application Layer that would receive these indications is the

> Request/Response Layer which then you return the appropriate Error code?

No, the Request/Response Layer is part of CoAP and defines the methods, response codes, options, etc. The user application uses an API ("Req/Res Layer API") to issue requests and receive responses and will receive these error codes. The codes need to be transport-agnostic. The transport itself can be chosen through the scheme used in the URI passed to the Req/Res API. In Californium, I will probably add additional API calls that only have effect on some transports (e.g., CON/NON). The user application can call them depending on what scheme/transport it chose. I tried to depict it:

+---------------------------------------------------+

|                                                   |

|        Application (custom / LWM2M / IPSO)        |

|                                                   |

+---------------[ Req/Res Layer API ]---------------+ <--

|                                                   |

|         Request/Response Layer (CoAP spec)        |

|                                                   |

+---------------------------------------------------+

| Message Layer (CoAP spec / alternative transport) |

+---------------------------------------------------+

|                         Transport Layer (UDP / DTLS / TCP / TLS / SMS)                               |

Ciao

Matthias

> In Prague, the problem of a common request/response API for all possible

> transports was raised. We now have something similar in the Californium

> project, where we want to improve the error handling for the CoapClient API as

> well as improve the information flow between Scandium (DTLS) and Californium

> (or rather the application code).

> As a start, I collected possible exceptions that could signal what went wrong.

> While this might be a bit Java-centric, the intent is universal: have a common set

> of signals to tell the application above the request/response API what went

> wrong.

> - TooManyOpenFilesException / InsufficientResourcesException (more OS-

> related, but happens at the socket level)

> It would be great, if you could have a look, suggest more

> exceptions/signals/error codes, propose thinning/merging, or just comment on

> the direction.

---

From: "Carey, Timothy (Timothy)" <[email-masked]>
Date: Fri, 7 Aug 2015 14:30:56 +0000

Matthias,

Then the exceptions you have listed should be reflected in Resp Codes?

BR,
Tim

From: Kovatsch Matthias [mailto:[email-masked]]
Sent: Friday, August 07, 2015 8:18 AM
To: Carey, Timothy (Timothy); core WG
Subject: RE: [core] Common request/response API

Hi Tim

|                                                   |

|                                                   |

|                                                   |

|                                                   |

Ciao

Matthias

---

From: "Kovatsch Matthias" <[email-masked]>
Date: Fri, 7 Aug 2015 15:11:45 +0000

No, response codes reflect success in this context (even 4.xx and 5.xx), since the request could be delivered and a response was received. Exceptions are for the things that could go wrong during transmission/reception.

From: Carey, Timothy (Timothy) [mailto:[email-masked]]
Sent: Friday, 7 August 2015 16:31
To: Kovatsch Matthias <[email-masked]>; core WG <[email-masked]>
Subject: RE: [core] Common request/response API

Matthias,

BR,
Tim

Hi Tim

|                                                   |

|                                                   |

|                                                   |

|                                                   |

Ciao

Matthias

---

From: "Carey, Timothy (Timothy)" <[email-masked]>
Date: Fri, 7 Aug 2015 16:27:12 +0000

Matthias,

So are you suggesting then that the API between the End Application and the Request Response Layer be:

     REQ    /\     /\      Application Layer (LWM2M,IPSO)
      |      |      |
------|------|------|------
      |      |
     \/     RESP  Abort    REQ/RESP Layer

I just wanted to be clear that this is an additional type of interaction.

BR,
Tim

From: Kovatsch Matthias [mailto:[email-masked]]
Sent: Friday, August 07, 2015 10:12 AM
To: Carey, Timothy (Timothy); core WG
Subject: RE: [core] Common request/response API

From: Carey, Timothy (Timothy) [mailto:[email-masked]]
Sent: Friday, 7 August 2015 16:31
To: Kovatsch Matthias <[email-masked]<mailto:[email-masked]>>; core WG <[email-masked]<mailto:[email-masked]>>
Subject: RE: [core] Common request/response API

Matthias,

BR,
Tim

Hi Tim

|                                                   |

|                                                   |

|                                                   |

|                                                   |

Ciao

Matthias

---

From: "Kovatsch Matthias" <[email-masked]>
Date: Tue, 11 Aug 2015 12:19:45 +0000

Yes, the application also needs to be informed of failures that are unrelated to CoAP such as failures at lower layers or the operating system (I wouldn't call it "Abort," though, but an exception or error).

How many "arrows" you have in the API between Req/Res Layer and Application Layer depends on your implementation. Note that they are equivalent:

-          Downward

o   1 down: attach all lower-layer-specific information to the request call (useful if the requirements change for each request)

o   2 down: have separate calls to configure the lower layers (e.g., choose the network operator for the cellular modem or relax the requirement for all requests to NON)

-          Upward

o   1 up: use a wrapper response struct/object that has either a CoAP response or an exception/error

o   2 up: split responses (return value or onSuccess(Response) handler) and exceptions (e.g., catch blocks or in an onError(Exception) handler).

The question for here is whether we can identify a common set of Exceptions that is agnostic of specific lower layers.

Regards
Matthias

From: Carey, Timothy (Timothy) [mailto:[email-masked]]
Sent: Freitag, 7. August 2015 18:27
To: Kovatsch Matthias <[email-masked]>; core WG <[email-masked]>
Subject: RE: [core] Common request/response API

Matthias,

BR,
Tim

Matthias,

BR,
Tim

Hi Tim

|                                                   |

|                                                   |

|                                                   |

|                                                   |

Ciao

Matthias

---

From: Robert Cragie <[email-masked]>
Date: Tue, 11 Aug 2015 18:31:04 +0100

I started to put together a strawman document on this topic and would
appreciate any comments.

https://docs.google.com/document/d/1a8BCVrKcGH0QmKDcV16SaJuD2ai_3v-ds3X7wf_32JE/edit?usp=sharing

Robert

On 11 August 2015 at 13:19, Kovatsch Matthias <[email-masked]> wrote:

> Yes, the application also needs to be informed of failures that are
> unrelated to CoAP such as failures at lower layers or the operating system
> (I wouldn’t call it “Abort,” though, but an exception or error).
>
>
>
> How many “arrows” you have in the API between Req/Res Layer and
> Application Layer depends on your implementation. Note that they are
> equivalent:
>
>
>
> -          Downward
>
> o   1 down: attach all lower-layer-specific information to the request
> call (useful if the requirements change for each request)
>
> o   2 down: have separate calls to configure the lower layers (e.g.,
> choose the network operator for the cellular modem or relax the requirement
> for all requests to NON)
>
> -          Upward
>
> o   2 up: split responses (return value or onSuccess(Response) handler)
> and exceptions (e.g., catch blocks or in an onError(Exception) handler).
>
>
>
> The question for here is whether we can identify a common set of
> Exceptions that is agnostic of specific lower layers.
>
>
>
> Regards
>
> Matthias
>
>
>
>
>
> *From:* Carey, Timothy (Timothy) [mailto:[email-masked]]
>
> *Sent:* Freitag, 7. August 2015 18:27
>
> *To:* Kovatsch Matthias <[email-masked]>; core WG <[email-masked]>
> *Subject:* RE: [core] Common request/response API
>
>
>
> Matthias,
>
>
>
> *From:* Kovatsch Matthias [mailto:[email-masked]
> <[email-masked]>]
> *Sent:* Friday, August 07, 2015 10:12 AM
> *To:* Carey, Timothy (Timothy); core WG
> *Subject:* RE: [core] Common request/response API
>
>
>
> No, response codes reflect success in this context (even 4.xx and 5.xx),
> since the request could be delivered and a response was received.
> Exceptions are for the things that could go wrong during
> transmission/reception.
>
>
>
>
>
> *From:* Carey, Timothy (Timothy) [mailto:[email-masked]
> <[email-masked]>]
> *Sent:* Friday, 7 August 2015 16:31
> *To:* Kovatsch Matthias <[email-masked]>; core WG <[email-masked]>
> *Subject:* RE: [core] Common request/response API
>
>
>
> Matthias,
>
>
>
> *From:* Kovatsch Matthias [mailto:[email-masked]
> <[email-masked]>]
> *Sent:* Friday, August 07, 2015 8:18 AM
> *To:* Carey, Timothy (Timothy); core WG
> *Subject:* RE: [core] Common request/response API
>
>
>
> Hi Tim
>
>
>
> No, the Request/Response Layer is part of CoAP and defines the methods,
> response codes, options, etc. The user application uses an API (“Req/Res
> Layer API”) to issue requests and receive responses and will receive these
> error codes. The codes need to be transport-agnostic. The transport itself
> can be chosen through the scheme used in the URI passed to the Req/Res API.
> In Californium, I will probably add additional API calls that only have
> effect on some transports (e.g., CON/NON). The user application can call
> them depending on what scheme/transport it chose. I tried to depict it:
>
>
>
> > From: Kovatsch Matthias [mailto:[email-masked]
> <[email-masked]>]
>
> _______________________________________________
> core mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/core
>
>

---

From: Floris Van den Abeele <[email-masked]>
Date: Wed, 12 Aug 2015 14:38:30 +0200

Would such signals also include sanity checks on CoAP requests that are
asked by the application? In some implementations the API itself can be
designed to prevent constructing invalid CoAP requests. But in our
implementation, the interface between the application and the CoAP
client (potentially running in a separate process on another machine) is
a socket and all the data about the request and the response (or
error/warning) is also transferred over this socket. Applications
describe their request via CSV strings, and the response is received as
a CSV string. This will sometimes lead to invalid requests if the
application is oblivious.

Our implementation includes sanity checks for some of the MUST (NOT)s in
the CoAP spec:
*  A Non-confirmable message always carries either a request or response
and MUST NOT be Empty -> Non messages must not be empty
* If a Method or Response Code is not defined to have a payload, then a
sender MUST NOT include one, and a recipient MUST ignore it. -> GET
requests must not include a payload.

Of course one could argue that in our case the logic generating the CSV
strings is actually part of the CoAP client library and not the
application itself, and that such signals should not be propagated to
the application and rather that the logic (i.e API) generating the CSV
strings should prevent applications from constructing empty
non-messages, GET requests without a payload, etc.

BR,
Floris

On 11-08-15 19:31, Robert Cragie wrote:
> On 11 August 2015 at 13:19, Kovatsch Matthias <[email-masked]
> <mailto:[email-masked]>> wrote:
>
>     Yes, the application also needs to be informed of failures that
>     are unrelated to CoAP such as failures at lower layers or the
>     operating system (I wouldn’t call it “Abort,” though, but an
>     exception or error).
>
>      
>
>     Application Layer depends on your implementation. Note that they
>     are equivalent:
>
>      
>
>     -          Downward
>
>     o   1 down: attach all lower-layer-specific information to the
>     request call (useful if the requirements change for each request)
>
>     o   2 down: have separate calls to configure the lower layers
>     (e.g., choose the network operator for the cellular modem or relax
>     the requirement for all requests to NON)
>
>     -          Upward
>
>     o   2 up: split responses (return value or onSuccess(Response)
>     handler) and exceptions (e.g., catch blocks or in an
>     onError(Exception) handler).
>
>      
>
>     *From:*Carey, Timothy (Timothy)
>     [mailto:[email-masked]
>     <mailto:[email-masked]>]
>     *To:* Kovatsch Matthias <[email-masked]
>     <mailto:[email-masked]>>; core WG <[email-masked]
>     <mailto:[email-masked]>>
>     So are you suggesting then that the API between the End
>     Application and the Request Response Layer be:
>
>      
>
>     *From:*Kovatsch Matthias [mailto:[email-masked]]
>     No, response codes reflect success in this context (even 4.xx and
>     5.xx), since the request could be delivered and a response was
>     received. Exceptions are for the things that could go wrong during
>     transmission/reception.
>
>      
>
>      
>
>     *From:*Carey, Timothy (Timothy)
>     [mailto:[email-masked]]
>     *To:* Kovatsch Matthias <[email-masked]
>     <mailto:[email-masked]>>; core WG <[email-masked]
>     <mailto:[email-masked]>>
>     *From:*Kovatsch Matthias [mailto:[email-masked]]
>     No, the Request/Response Layer is part of CoAP and defines the
>     methods, response codes, options, etc. The user application uses
>     an API (“Req/Res Layer API”) to issue requests and receive
>     responses and will receive these error codes. The codes need to be
>     transport-agnostic. The transport itself can be chosen through the
>     scheme used in the URI passed to the Req/Res API. In Californium,
>     I will probably add additional API calls that only have effect on
>     some transports (e.g., CON/NON). The user application can call
>     [email-masked] <mailto:[email-masked]>
> This body part will be downloaded on demand.

---

From: "Carey, Timothy (Timothy)" <[email-masked]>
Date: Thu, 13 Aug 2015 00:36:36 +0000

Robert,

Yes I think this might work. A generic set of primitives with unique meaning at the specific layer interaction.

Could this technique also be extended between the ML and Transport Layers?

BR,
Tim

From: Floris Van den Abeele [mailto:[email-masked]]
Sent: Wednesday, August 12, 2015 7:39 AM
To: [email-masked]
Cc: Carey, Timothy (Timothy); core WG; Kovatsch Matthias
Subject: Re: [core] Common request/response API

Would such signals also include sanity checks on CoAP requests that are asked by the application? In some implementations the API itself can be designed to prevent constructing invalid CoAP requests. But in our implementation, the interface between the application and the CoAP client (potentially running in a separate process on another machine) is a socket and all the data about the request and the response (or error/warning) is also transferred over this socket. Applications describe their request via CSV strings, and the response is received as a CSV string. This will sometimes lead to invalid requests if the application is oblivious.

Our implementation includes sanity checks for some of the MUST (NOT)s in the CoAP spec:
*  A Non-confirmable message always carries either a request or response and MUST NOT be Empty -> Non messages must not be empty
* If a Method or Response Code is not defined to have a payload, then a sender MUST NOT include one, and a recipient MUST ignore it. -> GET requests must not include a payload.

Of course one could argue that in our case the logic generating the CSV strings is actually part of the CoAP client library and not the application itself, and that such signals should not be propagated to the application and rather that the logic (i.e API) generating the CSV strings should prevent applications from constructing empty non-messages, GET requests without a payload, etc.

BR,
Floris
On 11-08-15 19:31, Robert Cragie wrote:
I started to put together a strawman document on this topic and would appreciate any comments.

Robert

On 11 August 2015 at 13:19, Kovatsch Matthias <[email-masked]<mailto:[email-masked]>> wrote:
Yes, the application also needs to be informed of failures that are unrelated to CoAP such as failures at lower layers or the operating system (I wouldn’t call it “Abort,” though, but an exception or error).

How many “arrows” you have in the API between Req/Res Layer and Application Layer depends on your implementation. Note that they are equivalent:

-          Downward

-          Upward

Regards
Matthias

From: Carey, Timothy (Timothy) [mailto:[email-masked]<mailto:[email-masked]>]
Sent: Freitag, 7. August 2015 18:27

Matthias,

BR,
Tim

Matthias,

BR,
Tim

Hi Tim

> Request/Response Layer which then you return the appropriate Error code?

No, the Request/Response Layer is part of CoAP and defines the methods, response codes, options, etc. The user application uses an API (“Req/Res Layer API”) to issue requests and receive responses and will receive these error codes. The codes need to be transport-agnostic. The transport itself can be chosen through the scheme used in the URI passed to the Req/Res API. In Californium, I will probably add additional API calls that only have effect on some transports (e.g., CON/NON). The user application can call them depending on what scheme/transport it chose. I tried to depict it:

|                                                   |

|                                                   |

|                                                   |

|                                                   |

Ciao

Matthias

_______________________________________________
core mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/core

---

From: Robert Cragie <[email-masked]>
Date: Fri, 14 Aug 2015 00:09:37 +0100

Hi Tim,

Yes, the intention was to extend it to transport layer as well, probably
based on the standard socket API, although this could also be generalised
in a primitive-based schema. I stopped short of doing that work as I
thought it was complete enough to put forward as a strawman to gauge
whether it is worth continuing. If people think it is worth it, I am happy
to add more content and open it up as a collaborative document on Google
docs.

Robert.

On 13 August 2015 at 01:36, Carey, Timothy (Timothy) <
[email-masked]> wrote:

> Robert,
>
>
>
> Yes I think this might work. A generic set of primitives with unique
> meaning at the specific layer interaction.
>
>
>
> *From:* Floris Van den Abeele [mailto:[email-masked]]
> *Sent:* Wednesday, August 12, 2015 7:39 AM
> *To:* [email-masked]
> *Cc:* Carey, Timothy (Timothy); core WG; Kovatsch Matthias
> *Subject:* Re: [core] Common request/response API
>
>
>
> implementation, the interface between the application and the CoAP client
> (potentially running in a separate process on another machine) is a socket
> and all the data about the request and the response (or error/warning) is
> also transferred over this socket. Applications describe their request via
> CSV strings, and the response is received as a CSV string. This will
> sometimes lead to invalid requests if the application is oblivious.
>
> strings is actually part of the CoAP client library and not the application
> itself, and that such signals should not be propagated to the application
> and rather that the logic (i.e API) generating the CSV strings should
> prevent applications from constructing empty non-messages, GET requests
> without a payload, etc.
>
> BR,
> Floris
>