Subject: [core] Reability of CoAP messages

From: "Adriano Pezzuto (apezzuto)" <[email-masked]>
Date: Tue, 6 Apr 2010 14:51:23 +0200

Hello Zach,
some comments about 2.8.1. UDP section of
http://www.ietf.org/id/draft-shelby-core-coap-00.txt.

"The Transaction ID in CoAP message headers is used to match
      responses to open requests and is generated by the client (common
      counter across all requests)."

I suppose, a receiver node will discard the message if a duplicated
Transaction ID is received.
If this is the case, I see two situations where this may have some sort
of issues:

1. A receiver node receives two (or more) different requests from two
different nodes and both requests have the same Transaction ID. The
receiver node will discard the last request.

2. An application manager running CoAP sends requests to all nodes of
the network (e.g. to node1, .... to node-i, .... to node-N). Not sure
but the application manager can send a new request to the node-i before
to receive the response to the last request and both requests have the
same transaction ID. The receiver node-i will discard the last request.

For case (1) a possible solution may be to use multiple separate counter
on the receiver node - one for each transmitter, instead of a common
counter. For case (2), a possible solution is to use a separate counter
on the application manager - one for each node, instead of a common
counter. May be a stop-and-wait (as stated in the same section of the
draft) would be sufficient to avoid the case (2) but not sure what you
mean exactly with stop-and-wait.

Thank you,
Adriano

---

From: "Angelo P. Castellani" <[email-masked]>
Date: Tue, 13 Apr 2010 15:12:34 +0200

(1) I suppose Transaction ID will be used for matching a session on
the server using the (clientIP, transactionID) pair.

(2) The same is valid on the client side, the session will be matched
using (serverIP, transactionID) pair.

However, as already pointed in a previous message titled "Some
comments on draft-shelby-core-coap-00" on this list, the use of
transaction ID seems really similar to the current use of client
source port in the Internet...

Bests,
Angelo

On Tue, Apr 6, 2010 at 14:51, Adriano Pezzuto (apezzuto)
<[email-masked]> wrote:
> _______________________________________________
> core mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/core
>

---

From: David Ryan <[email-masked]>
Date: Wed, 14 Apr 2010 11:36:50 +1000

Hi,

See comments inline.

> > receiver node will discard the last request.
On Tue, Apr 13, 2010 at 11:12 PM, Angelo P. Castellani
> the server using the (clientIP, transactionID) pair.

I'd say that a server node should always process a request and provide
a response and should not discard duplicate requests.  It is important
that the server does not need to retain state of client requests other
than current pending/processing requests.  Consider the scenario where
a client sends a request and the server sends a response, however, the
response is lost.  The client may attempt to retry the request using
the same transactionID.  The server will process the request again and
should send the same response.  This is where design of the REST based
server is important.  All transactions should be idempotent (allowing
safe retries).

> using (serverIP, transactionID) pair.

I think the transactionID counter across all nodes it is sending
requests should be the same. ie node1 (transactionid 1), node2
(transactionId 2)..  etc.
If there are multiple clients running on a single node then the source
port must be different as the transactionId may not be unique.

Regards,
David.

---

From: "Bob Dolin" <[email-masked]>
Date: Tue, 13 Apr 2010 21:52:03 -0700

I don't think defining away problems really makes them go away.... 

" All transactions should be idempotent (allowing safe retries)."

Let us consider the case of a household with a prepay meter. In this case, the household must pay for electricity in advance. Now let us consider that the household was notified via their in-home display that they are running low on their prepaid balance, so they decide to add to it. So, they use the internet, or whatever to transfer cash to the utility. Meanwhile, they are consuming energy at rates that vary with time (time of use pricing is coming or already here). Now, the utility adds some money to the meter, but it cannot know how much is left without querying first, and by the time it gets the response it is out of date due to the continuous consumption of electricity. How is the addition done in an idempotent way? Or is this class of applications not suitable for CoAPP? There are many examples of non-idempotent transactions in the world besides this one. Do we want to restrict CoAPP in all these unforeseen ways?

" I think the transactionID counter across all nodes it is sending
requests should be the same. ie node1 (transactionid 1), node2
(transactionId 2)..  etc.
If there are multiple clients running on a single node then the source
port must be different as the transactionId may not be unique."

So, are you suggesting that the server maintain a transaction ID counter/address/port for each destination and ensure that it is the same value for a single transaction across multiple clients? What will the storage requirements be for such an implementation?

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of David Ryan
Sent: Tuesday, April 13, 2010 6:37 PM
To: Angelo P. Castellani
Cc: [email-masked]
Subject: Re: [core] Reability of CoAP messages

Hi,

See comments inline.

Regards,
David.

---

From: David Ryan <[email-masked]>
Date: Wed, 14 Apr 2010 15:47:38 +1000

Hi Bob,

You raise some good points that need to be discussed.

> Let us consider the case of a household with a prepay meter. In this case, the household must pay for electricity in advance. Now let us consider that the household was notified via their in-home d…

Idempotent transactions will be a requirement for any protocol that is
based on a lossy network, especially one that is built on UDP.  There
can be no guarantee that a request or probably more importantly that a
response will be received.  I also don't think this issue will effect
just CoAPP; it will be true of any protocol that is built on
messaging.

The REST based answer to this problem is that transactions need to be
designed so that what may look non-idempotent can be made to be
idempotent at the transactional level.  For example, take the issue
you define of adding funds to a prepaid meter.  One method would be to
treat funds as a timestamped list.  Each time funds are added a
document is "put" on the server which has the additional funds.  From
a REST perspective you might have the following locations:

URI://prepaid_host/funds/2010_04_1_10:35:02
URI://prepaid_host/funds/2010_04_14_21:09:12

The contents of the document 2010_04_14_21:09:12 has the amount of
money added to the meter.  This REST structure doubles as both a
transaction list and adding funds to the prepaid meter.  When a client
adds funds it is performing a put or post to the location.  In the
event of a failure of a network failure the client can safely retry
the put or post.  If the server had previously received the document
then it will either replace it and calculate the difference between
documents or return an error code indicating that the document is
already there.  I'm sure there are numerous other ways to modify your
example so that it becomes idempotent.  Unless the Coap protocol is
going to be truly transactional with two phase commit then I think it
must be idempotent for all transactions.

If you have other transactions which might cause problems in regard to
being non-idempotent let me know.  Working through these issues is one
of the reasons I asked if anyone had built any models for a REST based
meter or home automation elements.  Without hitting some real world
problems we won't know if the Coap protocol will be suitable.

Obviously for your example the other requirement is that Coap support
a form of authentication for requests.  The current draft suggests
that DTLS might be a valid option. I'm yet to see if this is suitable
or fits the Coap model well.

> So, are you suggesting that the server maintain a transaction ID counter/address/port for each destination and ensure that it is the same value for a single transaction across multiple clients? Wha…

No, the server should only maintain the source details for requests
currently being processed.  It should not maintain any state between
requests.  A client should maintain a table of all requests that it is
expecting a response.  The client request table is simply indexed on
the transactionId.  In this way the client does not maintain a
different transaction id counter for each server it is sending
requests.

---

From: Zach Shelby <[email-masked]>
Date: Wed, 14 Apr 2010 08:50:11 +0300

Guys,

> Let us consider the case of a household with a prepay meter. In this case, the household must pay for electricity in advance. Now let us consider that the household was notified via their in-home d…

David maybe was too general on the idempotent thing, please read the coap draft's sections on methods (and the HTTP ones). Only really the READ method is idempotent, obviously other methods are not.

> So, are you suggesting that the server maintain a transaction ID counter/address/port for each destination and ensure that it is the same value for a single transaction across multiple clients? Wha…

It is the client which maintains the transaction ID. The server does not keep state about transaction IDs, but just responds to incoming requests.

Overall I think David explained this correctly, at least I am thinking along the same lines.

Zach

-- 
http://www.sensinode.com
http://zachshelby.org - My blog "On the Internet of Things"
http://6lowpan.net - New book - "6LoWPAN: The Wireless Embedded Internet"
Mobile: +358 40 7796297

Zach Shelby
Head of Research
Sensinode Ltd.
Kidekuja 2
88610 Vuokatti, FINLAND

This e-mail and all attached material are confidential and may contain legally privileged information. If you are not the intended recipient, please contact the sender and delete the e-mail from your system without producing, distributing or retaining copies thereof.

---

From: "Angelo P. Castellani" <[email-masked]>
Date: Wed, 14 Apr 2010 10:40:07 +0200

My comments were about matching response to request in (2), and about
matching equal (retransmitted?) requests in (1), which in my opinion,
must not generate another action (for methods non-idempotents) but
should however generate a response possibly the same response that the
original request has generated (for idempotents methods this is
straight-forward..).

I'm in no way suggesting that the server should keep state for
processed requests. However some state information is still required
to overcome TCP absence and still handle retransmission.

There are at least two different states where CoAP server needs to
keep state for incoming requests:
- HANDLING: when the request has been received and the response has
not been sent
>>> this is surely a duplicate request and should be ignored in my opinion
- HANDLED_WAIT: when the request has been received and the response
has been sent, similar to the CLOSE_WAIT state of TCP after the final
ACK over the connection.
Every handled request should be kept in this state until a timeout is
fired, and it is required response is not ACKed
>>> this should be handled as a duplicate, no action should be taken, however possibly the same response message should be generated by the node.

In my opinion this is required to handle response messages loss over
the channel, in that case the client will send again the request with
the same transaction ID and the server will respond with the same
response that is gone loss without action duplication.

Just a note, please consider removing transaction ID field, and using
the client source UDP port as transaction ID... When using 6lowpan
compression, UDP port field size automatically scales when that port
goes over a range and we've already 16bit there in the worst case.

If somebody isn't confident that this is good in every situation a
transaction ID option can be added, that can be paired with the client
source UDP port when needed.

Bests,
Angelo

---

From: David Ryan <[email-masked]>
Date: Wed, 14 Apr 2010 21:43:50 +1000

Hi Zach,

> David maybe was too general on the idempotent thing, please read the coap draft's sections on methods (and the HTTP ones). Only really the READ method is idempotent, obviously other methods are not.

I was just reading the definition of idempotent

"1. A function f : D -> D is idempotent if f (f x) = f x for all x in D.
I.e. repeated applications have the same effect as one. This can be
extended to functions of more than one argument, e.g. Boolean & has x
& x = x. Any value in the image of an idempotent function is a fixed
point of the function. "

Strictly speaking an idempotent request should always result in the
same response.  So I'm using the term idempotent a little incorrectly.
 I'm suggesting that applications should be designed so that any
request should result in the same outcome/state on the server even if
the request is resent.  So a CREATE request could be sent and
processed on a server, the response is lost and the client retries.
The result might be that the server response to the second request is
an error saying it is already created which different from the first
success response (ie not idempotent as the response is different),
however, the state on the server is idempotent.  Of course, it will be
possible that applications can be designed without this, but it will
be up to the application to deal with the results, not the Coap
protocol.

Hopefully that clear up my meaning. :)

Also, just to be a little pedantic, a READ method is not idempotent if
the READ results in a dynamic response (eg temperature sensor).  In
this case each response is different.  Obviously, a READ should always
be safe.  Although, once again there's nothing in the protocol to stop
applications from doing the wrong thing here too.

---

From: Clint Chaplin <[email-masked]>
Date: Sun, 18 Apr 2010 11:21:24 -0700

What is this:?

> This e-mail and all attached material are confidential and may contain legally privileged information. If you are not the intended recipient, please contact the sender and delete the e-mail from yo…

Do we allow such disclaimers on ietf email lists?
-- 
Clint (JOATMON) Chaplin
Principal Engineer
Corporate Standardization (US)
SISA

---

From: <[email-masked]>
Date: Sun, 18 Apr 2010 20:06:10 +0100

see  
http://www.ietf.org/mail-archive/web/ietf-announce/current/msg02914.html

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Clint Chaplin
Sent: 18 April 2010 19:21
To: Zach Shelby
Cc: [email-masked]
Subject: Re: [core] Reability of CoAP messages

What is this:?

---

From: Zach Shelby <[email-masked]>
Date: Sun, 18 Apr 2010 22:18:09 +0300

Clint,

Thanks for noticing, I actually fixed that last week when after realizing that annoying statement was hanging around by default. 

Zach