Subject: [core] Preliminary Summary from CoAP#3 plug test in Las Vegas

From: Carsten Bormann <[email-masked]>
Date: Fri, 22 Nov 2013 12:13:01 -0800

While a couple of final tests are still ongoing, here is an impression from the CoAP#3 plugtest that is ending today:

Overall interoperability was 94.1 % at the time of writing, with 809 interoperable out of 860 tests completed.
Note that this figure includes a number of completely new implementations as well as a number of newer specifications, including OMA LWM2M.

The core-coap tests were interoperable at 95.9 %, link-format was even better at 98.4 %.
Remaining hotspots (~ 80 %) are ETags/Validation and separate responses in a lossy context.
Block and Observe are at 84.9 and 84.8 percent, respectively, reflecting that some of the implementations are rather new and some recent changes may not be fully implemented.

OMA LWM2M was at a stunning 97.4 % given that the 1.0 spec draft is not even a month old, but that result reflects a smaller sample size and an early test spec with limited coverage.
We found a number of issues where clarifications are desirable in the LWM2M spec draft.

Finally, we only completed very few DTLS tests, at 66.7 % interoperability.
The problems observed were generally with the retransmission mechanism.
We will certainly expand this segment in the next events.

Thanks go to ETSI for running and to OMA for hosting the event, as well as to all the participants for bringing lots of implementations to the event.

If you missed this one: The next CoAP/DTLS/LWM2M interop is not far away.
Discussions on the details are ongoing, but we might even have one in London colocated with IETF89.

Grüße, Carsten

---

From: Carsten Bormann <[email-masked]>
Date: Fri, 22 Nov 2013 14:07:02 -0800

For those who just can’t stop testing (or have missed the event):

I have deployed the updates for CoAP#3 to http://coap.me, so you can use coap://coap.me as a test CoAP server and use the web interface of coap.me as a test CoAP client against your server.
The test descriptions for CoAP and DTLS can be found at https://github.com/cabo/td-coap3 — these are not the official ETSI document (which will be available soon) but the “source code” for that.

If you want to arrange some unofficial remote testing before CoAP#4, please don’t hesitate to ask me.

Grüße, Carsten