Subject: Re: [core] CoRE v/s Compressed HTTP

From: Carsten Bormann <[email-masked]>
Date: Thu, 1 Jul 2010 09:42:00 +0200

Vipul,

the short answer is that we are chartered to do what we are doing.

A slightly longer answer is that there are two approaches: "shoehorn everything so it kind of works with HTTP" and "do what we actually need".  If you look at things like webhooks, these are dominated by considerations of the first kind, with little attempt at actually getting the architecture right.

Do we waste time by creating a simple protocol?  I know that we can waste a lot of time by shoehorning an existing protocol into a space that it wasn't designed for.  Compression means you still have all the complexity, the unexpected interactions, ...

The comparison to WAP is useful.  How did that fail?  It started out as this cute thing to do Web-like GUI apps over SMS.
Only gradually did it creep into its "Mobile web replacement" role.  And invented its own content format.
If we were doing another Web replacement here in CoRE, I'd indeed run away as fast as I can.  CoRE is about M2M.

Gruesse, Carsten

---

From: Zach Shelby <[email-masked]>
Date: Thu, 1 Jul 2010 14:25:48 +0300

Hi Vipul,

When you start compressing and hacking HTTP, it is not HTTP anymore. HTTP is one particular design of a web transfer protocol. As we saw from the Chopan work, the end result from "compressing" HTTP ends up looking a lot like CoAP. This is why we cooperated on designing CoAP. The conclusion we came to when chartering is that defining a clean subset of REST functionality specifically for M2M use is the right path, and rather than pretending it was called HTTP and getting stuck with all its complexity and pitfalls - we gave it its own name and limited the feature set. Really this is just about naming - we could have called it eHTTP or lightHTTP or M2M-HTTP  - but yuck... 

In the end both CoAP and HTTP are part of the web architecture, use the same REST primitives and CoAP supports a small subset of HTTP response codes along with content-types. So as Tom said, CoAP < HTTP. But at the same time CoAP = M2M while HTTP = Web. There is a big difference in the interaction models between a web browser and web server, and autonomous embedded devices (Smart Energy, Building Automation etc.). The asynchronous nature of those interactions, multicast, along with the use of UDP take a lot more tweaking than just sticking HTTP on top of UDP. SoAP-over-UDP is a good example...

That said, coap-01 is aiming at still cleaner integration with the rest of the web, while making the header and implementation still simpler. We are also working on a subscription technique in coap-observe which doesn't add any new methods (and doesn't require funky application layer stuff), and will allow easy integration with things like HTTP Long Poll, Webhooks, SIP etc. via a proxy. So hang in there!

Zach

On Jul 1, 2010, at 10:42 AM, Carsten Bormann wrote:

> _______________________________________________
> core mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/core

-- 
Zach Shelby, Chief Nerd, Sensinode Ltd.
http://zachshelby.org  - My blog "On the Internet of Things"
http://6lowpan.net - My book "6LoWPAN: The Wireless Embedded Internet"
Mobile: +358 40 7796297

---

From: "Simpson, Robby (GE Energy Services)" <[email-masked]>
Date: Thu, 1 Jul 2010 15:28:45 -0400

Hi Vipul,

You are not alone -- I too have been wondering why compressed HTTP was
taken off of the table from the very start.

A couple themes I continuously hear are:

- "HTTP is complex" -- I disagree with this one.  The vast majority of
what is possible with HTTP is not required and can be safely ignored.
Further, I have a feeling use cases will be found in certain scenarios
that will lead CoRE to needing that 'complexity' again for those use
cases.

- "CoRE is about M2M" -- HTTP is also used for M2M - quite a lot in
fact!  I doubt I need to list the plethora of examples that exist.

Specifically trying to create compressed HTTP instead of starting from
scratch will, IMO, end the concerns with HTTP mapping, lead to a
protocol that will likely be useful outside of extremely constrained
devices, and help prevent CoRE from creating something that is alienated
from the rest of the Web.

- Robby

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
Vipul Gupta
Sent: Thursday, July 01, 2010 2:54 AM
To: [email-masked]
Cc: Vipul Gupta
Subject: [core] CoRE v/s Compressed HTTP

I've been following the mailing list discussion for some time now and
even after reading through draft-ietf-core-coap-00, I'm unable to see 
the compelling justification for a brand new protocol over using 
compressed HTTP as a starting point which is what Brian's original 
draft (http://tools.ietf.org/html/draft-frank-6lowapp-chopan-00) did. 

I can't help but feel an uneasy sense a Deja Vu. I'm reminded of the 
early days of the WAP forum which initially took the stance that 
"standard" Internet protocols were too heavyweight for phones and set 
about defining its own set of incompatible protocols. I'm particularly 
aware of the security work where the forum defined WTLS as a "wireless" 
version of TLS (the resuse of the TLS acronym seemed like a marketing 
ploy because the two protocols were incompatible and some of the 
incompatibilities introduced security vulnerabilities in WTLS). Our 
work at Sun Labs on SSL/TLS for small devices (Palm PDAs and
subsequently
sensor nodes as small as the Berkeley mote[1][2]) showed otherwise. We 
demonstrated that one could profile SSL/TLS appropriately so it could 
still be used on the wireless devices of the day while maintaining 
compatibility with the large majority of existing deployments.
Eventually, 
the right thing to do turned out to be the incorporation of Elliptic 
Curve Cryptography (ECC) the public key cryptosystem used in WTLS, 
into TLS (RFC 4492). This is now implemented in OpenSSL/Apache and 
Firefox, IE and offers meaningful performance benefits even for 
regular/unconstrained hosts [3].

I see strong parallels between adding ECC key exchange to TLS and 
defining header compression for HTTP. I wonder whether there would 
still be a place for COAP as currently defined if compressed HTTP 
were to become a standard. To me, the following seems like a better 
approach:

- Define header compression for HTTP which has the potential of being 
 very useful even for regular/unconstrained browsers and servers 
 interacting over TCP (Note that HTTP header compression is an 
 important component of SPDY, http://en.wikipedia.org/wiki/SPDY).

- Define a profile for compressed HTTP to be used in COAP, e.g. 
 subset of headers to be supported, restrictions on the size of 
 compressed messages so they can be carried over UDP, nailing 
 down the content of specific messages like subscriptions/
 notifications and check-in (as in the chopan 
 draft). The Webhooks approach (http://webhooks.org, 
 http://wiki.webhooks.org/RESTful-WebHooks) seems clean 
 and simple but certain aspects of subscriptions/notifications are 
 currently underspecified.

- Define how to carry compressed HTTP over UDP, e.g. handling of 
 retransmissions, congestion control, transaction IDs, etc

This approach seems to have several advantages -- it makes HTTP mapping 
easier, frees up the WG to devote its time to other important issues 
(nailing down the content of subscription/notification messages, 
discovery etc), and once regular browsers and servers adopt HTTP header 
compression (Chrome already seems to have some support for this) it 
makes true end-to-end security between COAP devices and the existing 
Web a real possibility.

Am I alone in feeling this way? What am I missing?

vipul
---
http://labs.oracle.com/people/mybio.php?uid=32495

[1] http://www.cs.plu.edu/courses/netsec/arts/coms.pdf
[2] http://cents.cs.berkeley.edu/retreats/winter_2005/SSL_on_motes.pdf
[3] http://labs.oracle.com/projects/crypto/ecc-ssl-ndss2004.pdf
_______________________________________________
core mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/core

---

From: Richard Kelsey <[email-masked]>
Date: Thu, 01 Jul 2010 16:18:56 -0400

Hi Robby,

I am wondering what you see as the important differences
between CoRE and compressed HTTP.  Assuming that CoRE
continues down its current path and also delivers on having
a simple CoRE<->HTTP mapping, why wouldn't you consider the
result compressed HTTP?

It seems to me that most of the discussion on the CoRE lists
falls into two categories:
 - exactly what HTTP functionality is required
   by the use cases
 - how to do subscription and notification
The same discussions would be occuring, if in a slightly
different form, if CoRE had decided to call what they were
doing "compressed HTTP".
                                -Richard Kelsey

---

From: Zach Shelby <[email-masked]>
Date: Thu, 1 Jul 2010 23:13:02 +0300

Hi Robby,

We are actually agreeing. If you go and binary compress HTTP, hack away the features not needed for our requirements etc. - you end up with something that looks very much like CoAP (when we are done). 

This WG simply made the decision not to call this thing HTTP and to set a limited set of features and requirements to design for explicitly. The IETF ADs clearly told us that trying to hack HTTP would be a standardization nightmare and that production HTTP is much more complex than it seems. 

The charter goal is explicitly to make a RESTful protocol, which is part of the web architecture. By no means did CoAP start from scratch, we started with Chopan and the application requirements given to us. I think we've made this more controversial than needed? The biggest challenge (and difference from HTTP) we have to deal with are asynchronous transactions over UDP,  Chopan had to deal with the same thing. We also have some requirements that differ clearly from HTTP - we need to achieve those with REST.

If you want to call CoAP a kind of compressed HTTP then go for it!  As CoAP is going to be 100% REST compatible and supports a subset of application layer HTTP features this is very much part of the web (CoAP < HTTP). In coap-01 we have also been able to better separate the UDP transaction issues from REST, which will allow us to do e.g. subscription without any new methods. coap-01 will also have no problems with HTTP mapping. So we are getting there.

So if it is just the name in the end that is the problem - then why don't we change the name? Works for me.
 
- Embedded HTTP
- Constrained HTTP 
- Binary HTTP
- Compressed HTTP
- Constrained REST
- M2M HTTP
- M2M REST

Zach

---

From: "Don Sturek" <[email-masked]>
Date: Thu, 1 Jul 2010 13:32:42 -0700

Hi Richard,

I think the main difference with compressed HTTP and CoAP/HTTP proxy is that
the HTTP session is terminated at the proxy (along with security) and
re-generated in CoAP.

The HTTP proxy is one thing, however, mapping between TLS/DTLS sessions is
another.  You also need to be concerned about the type of device that is
doing this since the keys on both sessions will be in the clear in the
proxy.

Don

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
Richard Kelsey
Sent: Thursday, July 01, 2010 1:19 PM
To: Simpson, Robby (GE Energy Services)
Cc: [email-masked]
Subject: Re: [core] CoRE v/s Compressed HTTP

Hi Robby,

---

From: Paul Duffy <[email-masked]>
Date: Thu, 01 Jul 2010 16:35:34 -0400

Hi Zack,

I share Vipul's and Robby's concerns.  Recent conversation implied that 
CORE was headed off the HTTP semantic rails (addition of specific 
methods for subscription creation, etc.).  If that is no longer the 
case, and STATELESS translation between HTTP and COAP will be possible 
... that eases concerns.

I'm looking for interchangeability of HTTP/CORE and payload encodings as 
appropriate for the resource constraints of specific deployments.

Cheers

On 7/1/2010 4:13 PM, Zach Shelby wrote:
> If you want to call CoAP a kind of compressed HTTP then go for it!  As CoAP is going to be 100% REST compatible and supports a subset of application layer HTTP features this is very much part of the web (CoAP<  HTTP). In coap-01 we have also been able to better separate the UDP transaction issues from REST, which will allow us to do e.g. subscription without any new methods. coap-01 will also have no problems with HTTP mapping. So we are getting there.
>

---

From: Zach Shelby <[email-masked]>
Date: Fri, 2 Jul 2010 00:09:02 +0300

Paul,

On Jul 1, 2010, at 11:35 PM, Paul Duffy wrote:

> Hi Zack,
> 
> I share Vipul's and Robby's concerns.  Recent conversation implied that CORE was headed off the HTTP semantic rails (addition of specific methods for subscription creation, etc.).  If that is no longer the case, and STATELESS translation between HTTP and COAP will be possible ... that eases concerns.

Correct, coap-01 (and coap-observe) will not add new methods and a stateless translation is possible. 

But as Richard pointed out, the same conversations we would be having with compressed HTTP as well (the original EBHTML proposal had new methods via GENA e.g.). 

> I'm looking for interchangeability of HTTP/CORE and payload encodings as appropriate for the resource constraints of specific deployments.

So you mean applying the same web service interface design interchangeably over HTTP and CoRE? Sure, this is a requirement that makes sense, although of course HTTP has some limitations due to a lack of built-in discovery, multicast and subscription. These we can alleviate by making it easy for HTTP to borrow those paradigms from CoRE (or use CoRE in the case of multicast). I have already added a section about that WRT discovery in coap-01. The same should be possible for subscription.

Zach

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 1 Jul 2010 23:09:57 +0200

On Jul 01 2010, at 22:35, Paul Duffy wrote:

> interchangeability of HTTP/CORE

I think we can take from this discussion that there is a strong desire  
to be able to map HTTP to CoAP and back.
(At least those HTTP features that we do want to support.)

However, I'm not happy with a restriction that CoAP can not ever do  
anything that HTTP can't.
There are things that are extremely complex in HTTP that should be  
simple in a REST architecture.
In this case, it is the job of the HTTP gateway to manage the  
complexity, not the job of the CoAP implementation in every single  
light switch.
This is the main philosophical difference between the CoRE approach  
and "compressed HTTP".

Gruesse, Carsten

---

From: "Simpson, Robby (GE Energy Services)" <[email-masked]>
Date: Thu, 1 Jul 2010 17:13:54 -0400

Hi Richard,

I think largely it's a matter of goals and scoping, which may or may not
change the outcome.  For instance, there has been much debate as to
whether the same methods will be used between HTTP and the CoRE work.
As another example, there has been much debate regarding translation and
mapping with an intermediate device.  To me, a truly compressed HTTP
would be extremely similar to vanilla HTTP and may actually even be
preferred in many non-constrained devices (for preservation of
bandwidth, etc.).

In terms of goals, if we consider ourselves to be creating compressed
HTTP, we will look at HTTP and determine how to make it more compact and
efficient.  If we consider ourselves to be creating something from
scratch that can be mapped to HTTP, then we may or may not end up with
something that is similar to HTTP and may require a different set of
architectures.

If, in the end, whatever comes out has that drop-in look and feel, then
I don't care what we call it -- to me its not a matter of names.

- Robby

-----Original Message-----
From: Richard Kelsey [mailto:[email-masked]] 
Sent: Thursday, July 01, 2010 4:19 PM
To: Simpson, Robby (GE Energy Services)
Cc: [email-masked]
Subject: Re: [core] CoRE v/s Compressed HTTP

Hi Robby,

---

From: Gilman Tolle <[email-masked]>
Date: Thu, 1 Jul 2010 17:19:41 -0400

+1 on the concerns in this thread, especially as regards  
interchangeability between HTTP and CoAP.

Also, I'd like to be sure that whatever subscription mechanism we end  
up deciding is the "right" one for CoRE includes a concept similar to  
an explicit callback URL when making subscriptions. Simply using the  
source address when accepting a subscription request is rarely  
sufficient, because the source port is ephemeral, and the client may  
have chosen an arbitrary port on which to run CoAP and listen for  
notifications. In addition, explicitly specifying the receiver is  
necessary if the subscription request is being made through a proxy.  
I'd also suggest that the mechanism should allow the client to include  
some information that will be returned to it unchanged in the  
notification, to help it distinguish between different notifications.  
In an explicit callback URL, this would be the path component and the  
query string of the callback URL, but it may also be a simple opaque  
string.

Gil

On Jul 1, 2010, at 4:35 PM, Paul Duffy wrote:

> Hi Zack,
>
> I share Vipul's and Robby's concerns.  Recent conversation implied  
> that CORE was headed off the HTTP semantic rails (addition of  
> specific methods for subscription creation, etc.).  If that is no  
> longer the case, and STATELESS translation between HTTP and COAP  
> will be possible ... that eases concerns.
>
> I'm looking for interchangeability of HTTP/CORE and payload  
> encodings as appropriate for the resource constraints of specific  
> deployments.
>
> Cheers
>
>
>> We are actually agreeing. If you go and binary compress HTTP, hack  
>> away the features not needed for our requirements etc. - you end up  
>> with something that looks very much like CoAP (when we are done).
>>
>> This WG simply made the decision not to call this thing HTTP and to  
>> set a limited set of features and requirements to design for  
>> explicitly. The IETF ADs clearly told us that trying to hack HTTP  
>> would be a standardization nightmare and that production HTTP is  
>> much more complex than it seems.
>>
>> The charter goal is explicitly to make a RESTful protocol, which is  
>> part of the web architecture. By no means did CoAP start from  
>> scratch, we started with Chopan and the application requirements  
>> given to us. I think we've made this more controversial than  
>> needed? The biggest challenge (and difference from HTTP) we have to  
>> deal with are asynchronous transactions over UDP,  Chopan had to  
>> deal with the same thing. We also have some requirements that  
>> differ clearly from HTTP - we need to achieve those with REST.
>>
>> If you want to call CoAP a kind of compressed HTTP then go for it!   
>> As CoAP is going to be 100% REST compatible and supports a subset  
>> of application layer HTTP features this is very much part of the  
>> web (CoAP<  HTTP). In coap-01 we have also been able to better  
>> separate the UDP transaction issues from REST, which will allow us  
>> to do e.g. subscription without any new methods. coap-01 will also  
>> have no problems with HTTP mapping. So we are getting there.
>>

---

From: "Simpson, Robby (GE Energy Services)" <[email-masked]>
Date: Thu, 1 Jul 2010 17:25:15 -0400

> This is the main philosophical difference between the CoRE approach  

Carsten,

Good point.

Can you elaborate on these complexities?  I believe that will help some
of us understand the underlying issues.

Thanks,
Robby

---

From: Guido Moritz <[email-masked]>
Date: Fri, 2 Jul 2010 11:24:46 +0200

Vipul,

I must admit, while following the mailings in the list I had the same
thoughts. But it indeed turned out, that there are several differences in
design of CoAP and simple binary HTTP.

(1) Both rely on a RESTful style. This is a point that needs no further
discussion, but is the starting point for CoAP/HTTP mapping.

(2) CoAP want's to use UDP instead of TCP. First I also did not found an
obvious reason, until I read Saltzer et al. [1]. As long as we not need the
flow control mechanisms of TCP and we keep the packet sizes reasonable small
enough, the only task of the transport layer is the transport itself. Data
consistency is not the task of the transport layer. ACK must be done on the
application layer if you really want to be sure your message has been
processed. 
Using UDP now implies further problems if just using binary HTTP. HTTP is
server/client and thus request/response over the same connection (this is
why it is based on TCP and is also working behind a NAT). But UDP related
with the expected high delays in a multihop 6LoWPAN networks is what makes
it more a peer-to-peer network by design instead of the HTTP server/client
model. There might be scenarios where this statement is not fulfilled, but
CoAP should be universal applicable instead of tailored for short distance
single-hop scenarios. 

(3) HTTP has no eventing, Pub/Sub, call it whatever. It has already been
discussed that some concepts like long-living TCP connections are not
working due to relying on UDP. Using the concepts of WebHooks sounds
reasonable. It would ease up the mapping.

(4) HTTP has no device discovery concepts by default.

(5) HTTP is relying on TCP, but the charter and application scenarios say we
need multicast ... so we are back at UDP.

[1] http://web.mit.edu/Saltzer/www/publications/endtoend/endtoend.pdf

So when I go through all these arguments, I feel that whatever we designing
here, this is not longer simply binary HTTP. As Zach mentioned, we might
argue about the name. But CoAP is far beyond just making HTTP headers binary
and thus smaller. 

Just watching around and what I find first is WebDav. You may argue WebDav
is HTTP. But I would say it has extensions what makes it different from
native HTTP.

Regards,
Guido

> -----Ursprüngliche Nachricht-----
> Von: [email-masked] [mailto:[email-masked]] Im Auftrag von
Vipul
> Gupta
> Gesendet: Donnerstag, 1. Juli 2010 08:54
> An: [email-masked]
> Cc: Vipul Gupta
> Betreff: [core] CoRE v/s Compressed HTTP
> 
> work at Sun Labs on SSL/TLS for small devices (Palm PDAs and subsequently
> compatibility with the large majority of existing deployments. Eventually,

---

From: Guido Moritz <[email-masked]>
Date: Fri, 2 Jul 2010 11:30:10 +0200

OMG, it's me again pointing at the W3C SOAP Web Services stuff ;)

Maybe a short look in the WS-Eventing Spec might be helpful (only to see
it's concept): http://www.w3.org/Submission/WS-Eventing/ 

WS-Eventing defines several roles that are part of the process of
subscription for and event, receiving events and managing subscriptions. The
roles are client, event source, subscription manager and event sink. 
The client subscribes for an event source. This subscription is handled by
the subscription manager that must not be necessarily the same entity like
the event source. Because the event sink can be a different endpoint as the
client, a third party can create a subscription. So in this case the third
party can compose which event sink is connected to which event source.
Client, event source, subscription manager and event sink can be completely
different devices/endpoints. Might we adapt (parts of) this behavior
(independent of this 'which method' discussion). 

Best,
Guido

> Gilman Tolle
> Gesendet: Donnerstag, 1. Juli 2010 23:20
> An: [email-masked]
> Cc: [email-masked]
> Betreff: Re: [core] CoRE v/s Compressed HTTP
>

---

From: <[email-masked]>
Date: Fri, 2 Jul 2010 10:37:07 +0100

Yes, only a high-layer check can guarantee that what was sent is what was received, without corruption.
But to quote saltzer et al.:
"Clearly, some effort at the lower levels to improve network reliability can have a significant effect on
application performance."
So transport-layer data consistency can benefit implementations.

It's a series of tradeoffs and engineering decisions, not necessarily clear-cut.

L.

There is no end to the end-to-end argument.

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Guido Moritz
Sent: 02 July 2010 10:25

[..]
(2) CoAP want's to use UDP instead of TCP. First I also did not found an obvious reason, until I read Saltzer et al. [1]. As long as we not need the flow control mechanisms of TCP and we keep the packet sizes reasonable small enough, the only task of the transport layer is the transport itself. Data consistency is not the task of the transport layer. ACK must be done on the application layer if you really want to be sure your message has been processed.
[..]

---

From: Zach Shelby <[email-masked]>
Date: Fri, 2 Jul 2010 13:23:49 +0300

Guido,

This is definitely a good reference to be included in coap-observe which deals with the requirements and solution (next version) for subscription in CoAP.

Gil also had some good ideas for further requirements with regard to tokens and third-party subscriptions. Those should also be considered in coap-observe. 

Zach

---

From: Guido Moritz <[email-masked]>
Date: Fri, 2 Jul 2010 12:26:54 +0200

> application performance."
Jep, but from the CoAP point of view I would say this is what brings us
closer to UDP than to TCP. TCP uses ACKs and Retransmissions to get some
kind of reliability. But keep in mind that CoAP (and I assume it does) has
no huge packets on application layer.  Of course, if application layer
packets are much bigger than transport layer packets, the ACK/Retr mechanism
on transport layer is a huge advantage concerning performance issues. But I
think CoAP is more heading towards smaller packets. So packet size on
application and transport layer might have the same size and thus an
application packet fits in one transport packet. So there would be no need
for the TCP ACK/Retr of stuff TCP and you can do this on application layer
directly with the same efforts and higher degree of reliability ('Did it'
vs. 'Got it'). So the main question at this point: which message/packet
sizes do we assume for a single request/response/transmission/what ever.

The other side of the coin would the checksum mechanism of TCP. Do we want
to use this on our nodes or leave this up to other layers?

The last thing is the flow control/sliding window stuff. I think we will not
really need this in CoAP due to quite small receive/send buffers implied by
the resource constraints.

Guido

> Von: [email-masked] [mailto:[email-masked]]
> Gesendet: Freitag, 2. Juli 2010 11:37
> An: Guido Moritz
> Cc: [email-masked]
> Betreff: RE: [core] CoRE v/s Compressed HTTP
> 
> Guido Moritz
> flow control mechanisms of TCP and we keep the packet sizes reasonable
> small enough, the only task of the transport layer is the transport
itself.
> Data consistency is not the task of the transport layer. ACK must be done
> on the application layer if you really want to be sure your message has
> been processed.
> [..]

---

From: Zach Shelby <[email-masked]>
Date: Fri, 2 Jul 2010 13:49:48 +0300

Regarding UDP and congestion control, I really think everyone should read the contribution from Lars:

http://tools.ietf.org/id/draft-eggert-core-congestion-control-00.txt

One main point is that the whole congestion control model of TCP wouldn't really help us, the flow model is totally different here. The other issue we have in CoRE is that not all requests even need to be (or should be) reliable - e.g. multicast or streaming.

The current coap-01 limitation on CoAP message size is that the resulting datagram must fit into the MTU. For 95% of the cases this payload size is sufficient. For that 5% (firmware update?), which I am personally not sure we should optimize for, coap-misc-04 does have a proposed block option to enabled transferring blocks of a representation. This is something to experiment with. 

Zach

---

From: Robert Moskowitz <[email-masked]>
Date: Fri, 02 Jul 2010 07:40:17 -0400

> to use this on our nodes or leave this up to other layers?

I may be pointing out something well known, but ESP has a MUCH better 
'checksum' mechanism, but there is no retry tied to it.  If the packet 
authentication fails the packet is silently discarded.

>> obvious reason, until I read Saltzer et al. [1]. As long as we not need
>>      
> the
>    

-- 
Robert Moskowitz
Senior Technical Director
ICSA Labs, an Independent Division of Verizon Business Systems
W: 248-968-9809
F: 248-968-2824
VoIP: 248-291-0713
E: [email-masked]

There's no limit to what can be accomplished if it doesn't matter who 
gets the credit

---

From: "Don Sturek" <[email-masked]>
Date: Fri, 2 Jul 2010 05:50:46 -0700

Hi Zach,

I think what you wrote is not quite correct.

I think many solutions will need guaranteed delivery and I think many will
need fragmentation/reassembly.  Only a small subset of solutions will work
with CoAP/UDP and all application messaging fitting into a single packet.

The reason I mention this is that I fully expect to see a need in some of
the features that TCP offers though I agree that congestion control in a
mesh network is not one of them (at least as implemented in TCP).

Don

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Zach
Shelby
Sent: Friday, July 02, 2010 3:50 AM
To: Guido Moritz
Cc: [email-masked]; [email-masked]
Subject: Re: [core] CoRE v/s Compressed HTTP

Regarding UDP and congestion control, I really think everyone should read
the contribution from Lars:

One main point is that the whole congestion control model of TCP wouldn't
really help us, the flow model is totally different here. The other issue we
have in CoRE is that not all requests even need to be (or should be)
reliable - e.g. multicast or streaming.

The current coap-01 limitation on CoAP message size is that the resulting
datagram must fit into the MTU. For 95% of the cases this payload size is
sufficient. For that 5% (firmware update?), which I am personally not sure
we should optimize for, coap-misc-04 does have a proposed block option to
enabled transferring blocks of a representation. This is something to
experiment with. 

Zach

---

From: Zach Shelby <[email-masked]>
Date: Fri, 2 Jul 2010 15:57:37 +0300

Morning Don,

On Jul 2, 2010, at 3:50 PM, Don Sturek wrote:

> I think many solutions will need guaranteed delivery

Agreed, and we have that.

> and I think many will
> need fragmentation/reassembly.  

This is something we still need to experiment with and gain experience. Can applications that need that do it themselves? Or is the Block Option proposal in coap-misc-04 workable? If the WG concludes that we need it and we have a way to do it then we surely will.

> Only a small subset of solutions will work

How did we go from transferring payloads in the sizes of 10s of bytes, to several kBs in this WG? ;-) coap-01 requires this to fit in a single MTU, which would usually be between 1280-1500 bytes. At least for all the applications I have experience with in the 6LoWPAN world this is sufficient. For the few things like firmware transfer that need more, you can design a REST interface for that. What do you have in mind?

> mesh network is not one of them (at least as implemented in TCP).

Yep, Lars's document has a really good list of stuff we need to experiment with and integrate into the base coap document as they are verified. We hope to try some out already at the plugfest.

Zach

---

From: "Don Sturek" <[email-masked]>
Date: Fri, 2 Jul 2010 06:01:26 -0700

Hi Zach,

On the length, for IEEE 802.15.4 using 6LowPAN, I think we have around 50 or
so bytes to work with in the first block (application wise).  If that fits
all the data you want to send that is great.  However, many solutions need
more.  Even if the application architects its control packets to fit into
those 50 bytes, the application may need to do reporting/etc which will
cause at least a portion of those packets to be fragmented.

Don

-----Original Message-----
From: Zach Shelby [mailto:[email-masked]] 
Sent: Friday, July 02, 2010 5:58 AM
To: [email-masked]
Cc: 'Guido Moritz'; [email-masked]; [email-masked]
Subject: Re: [core] CoRE v/s Compressed HTTP

Morning Don,

> need fragmentation/reassembly.  

This is something we still need to experiment with and gain experience. Can
applications that need that do it themselves? Or is the Block Option
proposal in coap-misc-04 workable? If the WG concludes that we need it and
we have a way to do it then we surely will.

> with CoAP/UDP and all application messaging fitting into a single packet.

How did we go from transferring payloads in the sizes of 10s of bytes, to
several kBs in this WG? ;-) coap-01 requires this to fit in a single MTU,
which would usually be between 1280-1500 bytes. At least for all the
applications I have experience with in the 6LoWPAN world this is sufficient.
For the few things like firmware transfer that need more, you can design a
REST interface for that. What do you have in mind?

> mesh network is not one of them (at least as implemented in TCP).

Yep, Lars's document has a really good list of stuff we need to experiment
with and integrate into the base coap document as they are verified. We hope
to try some out already at the plugfest.

Zach

Zach
>>> Yes, only a high-layer check can guarantee that what was sent is what
was
>>> received, without corruption.

---

From: Zach Shelby <[email-masked]>
Date: Fri, 2 Jul 2010 16:04:48 +0300

> cause at least a portion of those packets to be fragmented.

Ahhh, now I see what you mean. As CoAP is at the application layer it doesn't limit the size of a CoAP message to a single link-layer frame, but instead to the IPv6 MTU. However, there is a recommendation in the draft that says a message SHOULD fit into an 802.15.4 frame if possible to avoid the penalty of fragmentation. 

We can adjust that text if you think that SHOULD is too strong? 

Zach

---

From: "Don Sturek" <[email-masked]>
Date: Fri, 2 Jul 2010 06:16:15 -0700

At least for our SE 2 application, we will clearly see fragmented packets.
We have several types of data that will drive this:
1)  Reports
2)  Lists of events (certainly you can control how many you ask for but at
least once you need to get the list of what you can ask for)

Don

-----Original Message-----
From: Zach Shelby [mailto:[email-masked]] 
Sent: Friday, July 02, 2010 6:05 AM
To: [email-masked]
Cc: 'Guido Moritz'; [email-masked]; [email-masked]
Subject: Re: [core] CoRE v/s Compressed HTTP

> cause at least a portion of those packets to be fragmented.

Ahhh, now I see what you mean. As CoAP is at the application layer it
doesn't limit the size of a CoAP message to a single link-layer frame, but
instead to the IPv6 MTU. However, there is a recommendation in the draft
that says a message SHOULD fit into an 802.15.4 frame if possible to avoid
the penalty of fragmentation. 

Zach

---

From: "Simpson, Robby (GE Energy Services)" <[email-masked]>
Date: Fri, 2 Jul 2010 09:45:36 -0400

> the resource constraints.

I think its useful to discuss flow control and congestion control
separately (they are, in fact, quite different).

I agree that many of the existing TCP congestion control algorithms have
less-than-ideal behavior on wireless meshes, for instance.  I further
agree that if we keep the number of packets low and avoid links that
have other flows, we may be able to get away with some less
sophisticated congestion control algorithms for CoAp.

Flow control, on the other hand, may be quite useful for constrained
devices.  As an example, my understanding is that the TCP flow control
logic was intended to prevent more constrained devices from being
overwhelmed by less constrained devices.  TCP's advertisement of
essentially free buffer space in the receiver is one useful way to
achieve flow control.  I would suggest we not dismiss such functionality
as even amongst constrained devices some devices may be able to process
2 packets whereas another device may be able to process 3.

- Robby

---

From: Ralph Droms <[email-masked]>
Date: Fri, 2 Jul 2010 10:25:59 -0400

Robby - you're correct in pointing out the flow control and congestion control are separate components of TCP.

In one special case, flow control can be used as a primitive form of congestion control.  If the receive window is set to one MSS, TCP becomes a send-and-wait protocol.

What is the concern with TCP congestion control:
* performance,
* code size,
* something else?

Do we have performance information on 6lowpan adaptation layer fragmentation?  Is It possible that TCP, with a very small MSS, would provide better performance for traffic that won't fit in a 6lowpan frame than either IP or 6lowpan fragmentation?

- Ralph

---

From: Ralph Droms <[email-masked]>
Date: Fri, 2 Jul 2010 10:30:17 -0400

Being more specific about "performance":

* network throughput
* computation in the endpoint nodes

- Ralph

---

From: Henning Schulzrinne <[email-masked]>
Date: Fri, 2 Jul 2010 10:35:25 -0400

A basic law of protocol design: all messages start out small, but they never get smaller. Corollary: even if you believe that you need only small messages, somebody else will think of a good application for larger ones - and won't be discouraged by the fact that you tell him that the protocol wasn't designed for that. 

(I can think of several high-visibility examples where this has been true - DNS, DHCP and SIP-over-UDP come to mind immediately.)

Henning

---

From: <[email-masked]>
Date: Fri, 2 Jul 2010 15:41:47 +0100

Messages expand to overfill the space allocated for them. 

-----Original Message-----
From: Henning Schulzrinne [mailto:[email-masked]] 
Sent: 02 July 2010 15:35
To: [email-masked]
Cc: 'Zach Shelby'; Wood L Dr (Electronic Eng); [email-masked]
Subject: Re: [core] CoRE v/s Compressed HTTP
Importance: High

Henning

On Jul 2, 2010, at 9:16 AM, Don Sturek wrote:

> 2)  Lists of events (certainly you can control how many you ask for 
> but at least once you need to get the list of what you can ask for)
> 
> Don
> 
>> On the length, for IEEE 802.15.4 using 6LowPAN, I think we have 
>> around 50
> or
>> so bytes to work with in the first block (application wise).  If that 
>> fits all the data you want to send that is great.  However, many 
>> solutions need more.  Even if the application architects its control 
>> packets to fit into those 50 bytes, the application may need to do 
>> reporting/etc which will cause at least a portion of those packets to be fragmented.
> 
> 
> doesn't limit the size of a CoAP message to a single link-layer frame, 
> but instead to the IPv6 MTU. However, there is a recommendation in the 
> draft that says a message SHOULD fit into an 802.15.4 frame if 
> possible to avoid the penalty of fragmentation.
>

---

From: "Don Sturek" <[email-masked]>
Date: Fri, 2 Jul 2010 07:53:42 -0700

Hi Robby,

Agree completely on flow control in TCP.

I guess my point is almost immediately after finishing CoAP over UDP, we
will begin to add back features of TCP.  I would list
fragmentation/reassembly, guaranteed delivery and flow control into that
list.

Don

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
Simpson, Robby (GE Energy Services)
Sent: Friday, July 02, 2010 6:46 AM
To: Guido Moritz; [email-masked]
Cc: [email-masked]
Subject: Re: [core] CoRE v/s Compressed HTTP

- Robby

---

From: Robert Moskowitz <[email-masked]>
Date: Fri, 02 Jul 2010 11:01:41 -0400

> Messages expand to overfill the space allocated for them.

Thus UDP jumbograms were born.

---

From: Richard Kelsey <[email-masked]>
Date: Fri, 02 Jul 2010 11:49:02 -0400

> From: Ralph Droms <[email-masked]>
> Date: Fri, 2 Jul 2010 10:25:59 -0400
> 
> Do we have performance information on 6lowpan adaptation layer
> fragmentation?  Is It possible that TCP, with a very small MSS, would
> provide better performance for traffic that won't fit in a 6lowpan
> frame than either IP or 6lowpan fragmentation?

Ralph,

One of the advantages of using 6lowpan fragmentation is that
the IPv6 header, TCP header, and TLS header and MAC (or
other security) do not get duplicated in each fragment.
This increases the available bandwidth and reduces the
processing overhead.  Sending a single TCP+TLS packet that
fills two 6lowpan fragments rather than two unfragmented
TCP+TLS packets switches 40+ bytes from being overhead to
being payload.  Given that those two unfragmented packets
have maybe 60 bytes of TCP payload each, that is a big
difference, as is having to process only half as many TLS
records.
                              -Richard Kelsey

---

From: Peter Bigot <[email-masked]>
Date: Fri, 2 Jul 2010 10:43:09 -0500

I had understood CoAP to be an application-layer protocol that assumes a
datagram-based transport.  This has a fundamental impact on its
architecture.

Extending CoAP to make it pretend to be a stream-oriented protocol seems
like a mistake.

An IEEE 802.15.4 MTU is clearly too small to be useful at the application
layer; the IPv6 MTU is better, but still arbitrary.  Why encode these
limitations into CoAP?  Instead, select some defensible message limit (64KB,
like UDP), so we know how many bits to reserve for length fields.  Make some
non-normative recommendations for common platforms and use cases like IEEE
802.15.4.

For the rest of it, delegate responsibility for meeting the length
requirements to the transport-layer and/or system designer.  IPv6 already
knows how to do fragment reassembly; other transport-layer protocols could
be used to solve the problem too.  I don't think replicating this sort of
feature should be added to CoAP's task set.  It'd be kinda like implementing
telnet over HTTP: technically possible, but for heaven's sake *why*?

Peter

On Fri, Jul 2, 2010 at 8:16 AM, Don Sturek <[email-masked]> wrote:

> fits

---

From: Ralph Droms <[email-masked]>
Date: Fri, 2 Jul 2010 11:44:04 -0400

Does 6lopwan fragmentation require reassembly at each intermediate node?

- Ralph

---

From: Zach Shelby <[email-masked]>
Date: Fri, 2 Jul 2010 18:44:41 +0300

Please read http://tools.ietf.org/id/draft-eggert-core-congestion-control-00.txt and then let's talk more about this, I found this a very useful draft. I think we all agree that features for rate control, congestion control and reliability that suit our charter requirements are needed. 

Regarding TCP we already had WG consensus to remove that from the scope of our work for now.

Zach

---

From: Robert Moskowitz <[email-masked]>
Date: Fri, 02 Jul 2010 12:05:09 -0400

On 07/02/2010 11:43 AM, Peter Bigot wrote:
> I had understood CoAP to be an application-layer protocol that assumes 
> a datagram-based transport.  This has a fundamental impact on its 
> architecture.

Is there anyone here that is conversant with IEEE 11073?  This is 
currently being used both in SmartGrid and in Medical BANs (why are not 
MBANs in the Req ID?) sensors and controllers.

Is core competing with 11073 or going beyond it or what?

BTW, upstream from the MBAN controllers in the Continua model is SOAP to 
HR7 systems.

>
> Extending CoAP to make it pretend to be a stream-oriented protocol 
> seems like a mistake.
>
> An IEEE 802.15.4 MTU is clearly too small to be useful at the 
> application layer; the IPv6 MTU is better, but still arbitrary.  Why 
> encode these limitations into CoAP?  Instead, select some defensible 
> message limit (64KB, like UDP), so we know how many bits to reserve 
> for length fields.  Make some non-normative recommendations for common 
> platforms and use cases like IEEE 802.15.4.
>
> requirements to the transport-layer and/or system designer.  IPv6 
> already knows how to do fragment reassembly; other transport-layer 
> protocols could be used to solve the problem too.  I don't think 
> replicating this sort of feature should be added to CoAP's task set.  
> It'd be kinda like implementing telnet over HTTP: technically 
> possible, but for heaven's sake *why*?
>
> Peter
>
> On Fri, Jul 2, 2010 at 8:16 AM, Don Sturek <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>     From: Zach Shelby [mailto:[email-masked]
>     <mailto:[email-masked]>]
>     To: [email-masked] <mailto:[email-masked]>
>     Cc: 'Guido Moritz'; [email-masked] <mailto:[email-masked]>;
>     [email-masked] <mailto:[email-masked]>
>     [email-masked] <mailto:[email-masked]>

---

From: Richard Kelsey <[email-masked]>
Date: Fri, 02 Jul 2010 12:47:33 -0400

> Date: Fri, 2 Jul 2010 11:44:04 -0400
> 

Yes, at least officially.  There are some games that can be
played to avoid intermediate reassembly, but to me they seem
fraught with potential difficulties.

For packets that fit into a few 6lowpan fragments I don't
think that the per-hop reassembly hurts performance all that
much.  Larger number of fragments are problematic, because
of the increased buffering needed on intermediate nodes and
the requirement that all fragments must traverse the same
route.  It would be good to have real data on this.

To get this back to UDP, which Zach just reminded us has
already been chosen by the group, the same fragment overhead
issues apply to UDP packets as well as TCP, if somewhat
less so.  The UDP header is smaller to begin with and can be
compressed by 6lowpan.  On the other had, the DTLS packet
overhead is a little greater that TLS, and the savings in
processing fewer (D)TLS records is still there.

                                   -Richard Kelsey

---

From: Geoff Mulligan <[email-masked]>
Date: Fri, 02 Jul 2010 13:00:49 -0600

Yes it does when using a route over design, though there are some tricks
that might be able to be used.

In a mesh under - no.

	geoff

---

From: "Oflynn, Colin" <[email-masked]>
Date: Fri, 2 Jul 2010 15:31:01 -0600

Hello All,

I think a key thing to remember about CoAP is that since it is using a RESTful approach to access resources, those same resources should be accessible by with normal HTTP over TCP. 

So for example CoAP can safely have a maximum size of say 1280 bytes. It prevents people from using CoAP for things it wasn't designed for by not even pretending to support them. To do firmware transfers for instance CoAP could at least point to a resource that can be accessed with something like FTP/TFTP etc. 

A node which is going to be routinely doing larger than 1280 byte transfers is probably not very constrained. Such a node will realistically be able to implement TCP/HTTP and can thus easily take advantage of all the features TCP supports. 

Keeping CoAP intensely focused on the constrained devices should make it possible to implement on those constrained devices. 

Regards,

  -Colin

-----Original Message-----
From: [email-masked] on behalf of Zach Shelby
Sent: Fri 02/07/2010 16:44
To: [email-masked]
Cc: [email-masked]; [email-masked]
Subject: Re: [core] CoRE v/s Compressed HTTP

Zach

---

From: Carsten Bormann <[email-masked]>
Date: Fri, 2 Jul 2010 23:51:51 +0200

> least once you need to get the list of what you can ask for)

coap-misc defines a block option to avoid the need for (too much) fragmentation in this case.
Since every block is acknowledged, this works well even in the presence of losses.
(Loss probabilities get much worse with many (n) fragments, as the per-packet loss probability 1-(1-p)^n is close to p*n for small p.
Some fragmentation is probably good to reduce header overhead; the implementation at coap://ns.tzi.org:61616/ therefore uses a default block size of 128 (which would typically need two or three 6lowpan packets per IP packet).
But that choice is up to the applications, nothing in CoAP stops us from using MTU-size packets (IPv6: 1280).
(I just wouldn't want to go beyond that without some path MTU discovery -- this is the reason I believe something like the block option should be a mandatory part of the protocol.)

As a general note, *discovering* information about adaptation layer fragmentation is nearly impossible, so either one of the sides knows something and feeds it to the application, or we end up with lots of adaptation layer fragmentation.  Fortunately, 6LoWPANs are stub networks, so if they are involved, at least one of the sides knows that.  That's why the the block option was designed so that both sides can influence the block size.

Gruesse, Carsten