Subject: [core] CoAP over TCP: Extending the shim Header Payload

From: Hannes Tschofenig <[email-masked]>
Date: Wed, 30 Mar 2016 19:01:39 +0200

Hi all,

Klaus and I had a brief call yesterday to discuss his review in more
detail. It became clear from his review comments that our current
approach of offering a simple shim header with length information is
insufficient. Various issues on the working group tracker reveal this
pattern, as discussed on the mailing list.

I would like to therefore suggest to change the shim header in the
following way*:

A command field that allows to differentiation the following types of
payloads:
  - KEEPALIVE
  - MSG
  - ERROR
  - CONFIG

The semantic is the following:

a) MSG: This is the current header structure (minus the version field).
The structure of this shim header is the same in the forwards and in the
backwards direction.

b) KEEPALIVE: Here we could exchange an empty payload or one that is
similar to the TLS heartbeat. A KEEPALIVE exchange can be initiated by
client as well as the server. The recipient must respond with a
KEEPALIVE message back to the initiator.

c) ERROR: This payload is sent in response to a MSG, KEEPALIVE or a
CONFIG message. It indicates at least the following errors:

- INVALID_LENGTH: Length information does not match the provided size.
- PROTOCOL_VERSION: The protocol version the client has attempted to
negotiate is recognized but not supported.
- DECODE_ERROR: Any other parsing failure of the provided shim header.
- UNRECOGNIZED_NAME: This error is sent in response to a SNI that isn't
understood by the server.
- UNEXPECTED_MESSAGE: An inappropriate message was received. This
happens when the client or the server provide a message that does not
match the expected message.

d) CONFIG: This message could be structured similar to the
ClientHello/ServerHello exchange offered by DTLS/TLS. The client
initiates a CONFIG exchange and the server responds with a CONFIG
message or returns an error. Subsequent to a CONFIG exchange MSG packets
are exchanged unless the configuration information can be obtained via
other means, such as via the use of TLS. The CONFIG message sent by the
client to the server indicates the supported version and server name
indication (and potentially other features) whereas the server only
selects the version (or other features) from the client-provided list or
returns an ERROR message instead. When TLS is used then this message is
likely not needed since the functionality is already provided by TLS.

How does this sound?

Ciao
Hannes

PS: Funny enough I encountered a similar situation with a somewhat
different protocol recently. I have been contributing to the FIDO
Alliance U2F BLE specification where we also noticed that a simple
length field as a shim is insufficient and then added protocol
functionality for error messaging and a keepalive mechanisms.

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 30 Mar 2016 21:06:19 +0200

I didn't write this up yet, but it seems I'll have to.

Any extension to the shim header will soon grow into a protocol that is
as complex as CoAP itself.

Instead, we should simply re-use a protocol that we already have: CoAP.

(If we really need to do all of this; I'm not convinced yet.)

See below for how (I haven't checked the details against your new
protocol, but it's the gist I'm after here).

Grüße, Carsten

# Signaling messages

Signaling messages are structured like any other CoAP message; they
have a code, a token, options, and optionally a payload.
The code for a signaling message comes from the 7.xx space.

Option numbers for signaling messages are specific to the message
code, i.e., they do not share the number space with CoAP options for
request/response messages or with signaling messages using other
codes.
Signaling options can be elective or critical; if a signaling message
option is critical and not understood by the receiver, it MUST abort
the connection.  (If the option is understood but somehow cannot be
carried out, the option defines how to handle the error.)

Payloads in signaling messages are diagnostic payloads, unless
otherwise determined by a signaling message option.

Five types of signaling messages are defined:

* Capability messages: indicate a capability to the peer.  Most
  capability options will be elective.  Capability indications are
  cumulative, i.e., a capability message without any option is a
  no-operation (and can be used as such).  (An option that is given
  might override a previous value for the same option; the option
  defines how to handle this, if needed.)  Most options are defined
  only for the first message in the connection.

* Ping, pong: A ping message is responded to by a pong message with
  the same token.  No options are defined for Ping messages in this
  specification, but options might be defined later.  As with all
  signaling messages, the receiver of a ping or pong message MUST
  ignore elective options it does not understand.

* Release.  A release message indicates that the sender does not want
  to continue maintaining the connection and opts for an orderly
  shutdown; the details are in the options.  A release message will
  normally be replied to by a release message as soon as possible by
  the receiver; only then is the connection closed.  (To do: define
  who actually closes the TCP connection.)

* Abort.  An abort message indicates that the sender is unable to
  continue maintaining the connection and cannot even wait for an
  orderly release; the sender shuts down the connection immediately
  after the abort (and may or may not wait for a release or abort
  message in the inverse direction).

Release and abort messages indicate one or more reasons using elective
options.

## Discussion

Capability messages remove the perceived need for versioning.
(Versioning is great for keeping incompatible implementations from
accidentally trying to talk to each other, but is not a good
evolution strategy.)

An effect similar to the ping/pong message can be achieved by creating
a resource that has appropriate no-operations semantics (e.g., returns
an empty payload on a GET or returns the payload given on a POST).
Handling the function as a signaling message keeps the resource name space
clean of such resources.

The need for orderly release only becomes apparent in the presence of
non-idempotent requests such as POST.
It does not solve the problem that a connection that breaks in an
uncontrolled way keeps the status of that request unclear.

---

From: Hannes Tschofenig <[email-masked]>
Date: Thu, 31 Mar 2016 14:14:06 +0200

Hi Carsten,

In my write-up I intentionally avoided describing what encoding to use
and, as you write below, maybe CoAP messages are the right encoding for
these types of message exchange. Whether that's indeed the case will
have to get analysed.

**IMHO it is important to answer the question whether WG participants
see the need to provide these features.**

Maybe there are other options. At the beginning we thought we could get
away with a very simplistic shim header and we are done with it. Now, it
appears that this is not really the case. I would like to have various
folks interested in the CoAP over TCP and TLS look at this to see
whether the additional complexity is worth the effort.

As alternatives I see the following:

 * Stick with the shim header and just not provide any of the advanced
features (with all the disadvantages that come along with it).

 * Switch to a different protocol altogether (such as HTTP/2) for those
cases where a TCP-alike behavior is needed.

 * Use CoAP over TLS over TCP only (without the ability to use CoAP over
TCP) since TLS already provides various for the features (such as
version negotiation, server naming, etc.) we are looking for.

Ciao
Hannes