Subject: [core] Any experience in using CoAP responses to multicast requests?

From: "Dijk, Esko" <[email-masked]>
Date: Mon, 25 Mar 2013 09:16:19 +0000

Dear all,

one of the review comments to the GroupComm I-D concerns the lack of (implementation or simulation) experience with use of CoAP multicast, and the associated multitude of CoAP responses. The I-D provides guidelines in this area but not based on such experience.

Did anyone experiment with CoAP multicast? (Or knows someone, or a publication etc.)

regards,
Esko

________________________________
The information contained in this message may be confidential and legally protected under applicable law. The message is intended solely for the addressee(s). If you are not the intended recipient, you are hereby notified that any use, forwarding, dissemination, or reproduction of this message is strictly prohibited and may be unlawful. If you are not the intended recipient, please contact the sender by return e-mail and destroy all copies of the original message.

---

From: Isam Ishaq <[email-masked]>
Date: Tue, 26 Mar 2013 16:49:14 +0100

Dear Esko and all,

At iMinds we have some limited experience with multicast in wireless 
sensor networks. We have used multicast CoAP requests in order to 
discover sensors inside LLNs. At the networking layer of the LLN the 
multicast messages were routed as broadcasts. We have conducted our 
tests using our testbed with real sensor nodes and have experimented 
with network sizes up to 50 nodes in different network topologies with 
hop counts between 1 and 5 hops.

As expected, the use of multicast/broadcasts proved to be challenging.  
Even for small sensor networks (10 nodes or less), we had issues with 
broadcast storms in the network. One reason for these storms was related 
to routing (a basic AODV implementation in our case). Once the sensor 
nodes got the multicast and wanted to reply they all started asking for 
the route to the gateway at the same time, causing the network to become 
unresponsive. In order to combat this broadcast storms we let the sensor 
gateway broadcast a route reply packet in the network. This caused all 
sensors to establish a route to the gateway and eliminated the need for 
them to start asking for a route once they get the discovery request.

In our implementation the boarder router sending the broadcasts was also 
a constrained device (Tmote Sky). So it could not handle all the replies 
it got from the other sensors, because they arrived with small time gaps 
between them. This led to the receive buffer on the boarder router to 
start dropping packets. To help solving this issue we introduced random 
delays in the network -- before responding the CoAP requests (Leisure) 
and before forwarding broadcasts in the network.

By using these techniques we improved the delivery of the multicast and 
the resulting unicast replies very much. Still since multicasts are 
delivered in a non-confirmable way, there was no way to tell that the 
multicast did actually get to all nodes. In our case this was ok and 
just meant that the sensors that did not get the multicast the first 
time will get it the next time when a new periodic pull for discovery 
was done.

First results of this work have been published in the following paper:
http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6296943&isnumber=6296822

An extended version of this paper with more experimental results is 
currently under review. At this moment, we are also exploring the use of 
an intermediary to offer group communication functionality based on 
unicast (paper under review). We are also interested comparing this 
approach with a complete multicast solution for a LLN.

Best Regards,
Isam Ishaq

On 25-Mar-13 10:16 AM, Dijk, Esko wrote:
>
> Dear all,
>
> one of the review comments to the GroupComm I-D concerns the lack of 
> (implementation or simulation) experience with use of CoAP multicast, 
> and the associated multitude of CoAP responses. The I-D provides 
> guidelines in this area but not based on such experience.
>
> ------------------------------------------------------------------------
> The information contained in this message may be confidential and 
> legally protected under applicable law. The message is intended solely 
> for the addressee(s). If you are not the intended recipient, you are 
> hereby notified that any use, forwarding, dissemination, or 
> reproduction of this message is strictly prohibited and may be 
> unlawful. If you are not the intended recipient, please contact the 
> sender by return e-mail and destroy all copies of the original message.
>
>
> _______________________________________________
> core mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/core

-- 
Isam Ishaq
Department of Information Technology
Internet Based Communication Networks and Services (IBCN)
Ghent University - iMinds
Gaston Crommenlaan 8 (Bus 201), B-9050 Gent, Belgium
M: +32 (0)484 856 155
T: +32 (0)9 33 14946
T Secr: +32 (0)9 33 14900
F: +32 (0)9 33 14899
E: [email-masked]
W : www.ibcn.intec.UGent.be

---

From: "Dijk, Esko" <[email-masked]>
Date: Wed, 27 Mar 2013 14:04:17 +0000

Thanks Isam,

It's good to hear that the two measures (CoAP Leisure + random delay in forwarding) solved the issue. CoAP server implementations SHOULD use the Leisure, and the random forwarding delay is also applied in the Multicast Protocol for Low-power lossy networks (MPL) of the RoLL Working Group.

I would be interested to hear about the extended paper as well, when it is published.

regards,
Esko

From: [email-masked] [mailto:[email-masked]] On Behalf Of Isam Ishaq
Sent: Tuesday, March 26, 2013 16:49
To: [email-masked]
Subject: Re: [core] Any experience in using CoAP responses to multicast requests?

Dear Esko and all,

At iMinds we have some limited experience with multicast in wireless sensor networks. We have used multicast CoAP requests in order to discover sensors inside LLNs. At the networking layer of the LLN the multicast messages were routed as broadcasts. We have conducted our tests using our testbed with real sensor nodes and have experimented with network sizes up to 50 nodes in different network topologies with hop counts between 1 and 5 hops.

As expected, the use of multicast/broadcasts proved to be challenging.  Even for small sensor networks (10 nodes or less), we had issues with broadcast storms in the network. One reason for these storms was related to routing (a basic AODV implementation in our case). Once the sensor nodes got the multicast and wanted to reply they all started asking for the route to the gateway at the same time, causing the network to become unresponsive. In order to combat this broadcast storms we let the sensor gateway broadcast a route reply packet in the network. This caused all sensors to establish a route to the gateway and eliminated the need for them to start asking for a route once they get the discovery request.

In our implementation the boarder router sending the broadcasts was also a constrained device (Tmote Sky). So it could not handle all the replies it got from the other sensors, because they arrived with small time gaps between them. This led to the receive buffer on the boarder router to start dropping packets. To help solving this issue we introduced random delays in the network - before responding the CoAP requests (Leisure) and before forwarding broadcasts in the network.

By using these techniques we improved the delivery of the multicast and the resulting unicast replies very much. Still since multicasts are delivered in a non-confirmable way, there was no way to tell that the multicast did actually get to all nodes. In our case this was ok and just meant that the sensors that did not get the multicast the first time will get it the next time when a new periodic pull for discovery was done.

An extended version of this paper with more experimental results is currently under review. At this moment, we are also exploring the use of an intermediary to offer group communication functionality based on unicast (paper under review). We are also interested comparing this approach with a complete multicast solution for a LLN.

Best Regards,
Isam Ishaq

regards,
Esko

core mailing list

[email-masked]<mailto:[email-masked]>

--

Isam Ishaq

M: +32 (0)484 856 155

T: +32 (0)9 33 14946

F: +32 (0)9 33 14899

E: [email-masked]<mailto:[email-masked]>

W : www.ibcn.intec.UGent.be<http://www.ibcn.intec.UGent.be>

---

From: "Rahman, Akbar" <[email-masked]>
Date: Wed, 27 Mar 2013 16:03:51 -0400

Hi Isam,

Yes, it was indeed a very nice paper and your extra points below are
also informative.  I just had one question.  The paper describes your
Multicast GET scenarios.  Did you ever send any other Multicast methods
(e.g. PUT)?  And if so, did you observe any additional interesting
behavior?

Best Regards,

Akbar

From: [email-masked] [mailto:[email-masked]] On Behalf Of
Dijk, Esko
Sent: Wednesday, March 27, 2013 10:04 AM
To: Isam Ishaq; [email-masked]
Subject: Re: [core] Any experience in using CoAP responses tomulticast
requests?

Thanks Isam,

It's good to hear that the two measures (CoAP Leisure + random delay in
forwarding) solved the issue. CoAP server implementations SHOULD use the
Leisure, and the random forwarding delay is also applied in the
Multicast Protocol for Low-power lossy networks (MPL) of the RoLL
Working Group. 

I would be interested to hear about the extended paper as well, when it
is published.

regards,

Esko

From: [email-masked] [mailto:[email-masked]] On Behalf Of
Isam Ishaq
Sent: Tuesday, March 26, 2013 16:49
To: [email-masked]
Subject: Re: [core] Any experience in using CoAP responses to multicast
requests?

Dear Esko and all,

delays in the network - before responding the CoAP requests (Leisure)

First results of this work have been published in the following paper:
http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6296943&isnumber
=6296822

Best Regards,
Isam Ishaq

	Dear all,

	one of the review comments to the GroupComm I-D concerns the
lack of (implementation or simulation) experience with use of CoAP
multicast, and the associated multitude of CoAP responses. The I-D
provides guidelines in this area but not based on such experience.

	Did anyone experiment with CoAP multicast? (Or knows someone, or
a publication etc.)

	regards,

	Esko

	The information contained in this message may be confidential
and legally protected under applicable law. The message is intended
solely for the addressee(s). If you are not the intended recipient, you
are hereby notified that any use, forwarding, dissemination, or
reproduction of this message is strictly prohibited and may be unlawful.
If you are not the intended recipient, please contact the sender by
return e-mail and destroy all copies of the original message.

---

From: Isam Ishaq <[email-masked]>
Date: Thu, 28 Mar 2013 09:59:42 +0100

Hi Akbar,

We did not do any tests we other methods.
I do not expect that there will be a difference in the behavior as a 
result of changing the method alone.  What could cause different 
behaviors?

Best,
Isam

On Wednesday, March 27, 2013 9:03:51 PM, Rahman, Akbar wrote:
> Multicast GET scenarios.  Did you ever send any other Multicast
> methods (e.g. PUT)?  And if so, did you observe any additional
> interesting behavior?
>
> Best Regards,
>
> Akbar
>
> *From:[email-masked] [mailto:[email-masked]] *On Behalf
> Of *Dijk, Esko
> *Sent:* Wednesday, March 27, 2013 10:04 AM
> *To:* Isam Ishaq; [email-masked]
> *Subject:* Re: [core] Any experience in using CoAP responses
> tomulticast requests?
>
> Thanks Isam,
>
> It’s good to hear that the two measures (CoAP Leisure + random delay
> in forwarding) solved the issue. CoAP server implementations SHOULD
> use the Leisure, and the random forwarding delay is also applied in
> the Multicast Protocol for Low-power lossy networks (MPL) of the RoLL
> Working Group.
>
> *From:[email-masked] [mailto:[email-masked]] *On Behalf
> Of *Isam Ishaq
> *Sent:* Tuesday, March 26, 2013 16:49
> *To:* [email-masked]
> *Subject:* Re: [core] Any experience in using CoAP responses to
> multicast requests?
>
> Dear Esko and all,
>
> As expected, the use of multicast/broadcasts proved to be
> challenging.  Even for small sensor networks (10 nodes or less), we
> had issues with broadcast storms in the network. One reason for these
> storms was related to routing (a basic AODV implementation in our
> case). Once the sensor nodes got the multicast and wanted to reply
> they all started asking for the route to the gateway at the same time,
> causing the network to become unresponsive. In order to combat this
> broadcast storms we let the sensor gateway broadcast a route reply
> packet in the network. This caused all sensors to establish a route to
> the gateway and eliminated the need for them to start asking for a
> route once they get the discovery request.
>
> In our implementation the boarder router sending the broadcasts was
> also a constrained device (Tmote Sky). So it could not handle all the
> replies it got from the other sensors, because they arrived with small
> time gaps between them. This led to the receive buffer on the boarder
> router to start dropping packets. To help solving this issue we
> introduced random delays in the network – before responding the CoAP
> requests (Leisure) and before forwarding broadcasts in the network.
>
>
> By using these techniques we improved the delivery of the multicast
> and the resulting unicast replies very much. Still since multicasts
> are delivered in a non-confirmable way, there was no way to tell that
> the multicast did actually get to all nodes. In our case this was ok
> and just meant that the sensors that did not get the multicast the
> first time will get it the next time when a new periodic pull for
> discovery was done.
>
> currently under review. At this moment, we are also exploring the use
> of an intermediary to offer group communication functionality based on
>     one of the review comments to the GroupComm I-D concerns the lack
>     of (implementation or simulation) experience with use of CoAP
>     legally protected under applicable law. The message is intended
>     solely for the addressee(s). If you are not the intended
>     recipient, you are hereby notified that any use, forwarding,
>     dissemination, or reproduction of this message is strictly
>     prohibited and may be unlawful. If you are not the intended
>     recipient, please contact the sender by return e-mail and destroy
>     all copies of the original message.
>
>
>     [email-masked]  <mailto:[email-masked]>
>
> E:[email-masked]  <mailto:[email-masked]>
> W :www.ibcn.intec.UGent.be  <http://www.ibcn.intec.UGent.be>

---

From: Isam Ishaq <[email-masked]>
Date: Thu, 28 Mar 2013 10:09:41 +0100

Hello,

Is anybody aware of an MPL implementation that we can test?

Best,
Isam

On 27-Mar-13 3:04 PM, Dijk, Esko wrote:
>
> Thanks Isam,
>
> It's good to hear that the two measures (CoAP Leisure + random delay 
> introduced random delays in the network -- before responding the CoAP

---

From: "Rahman, Akbar" <[email-masked]>
Date: Sat, 30 Mar 2013 00:15:48 -0400

Hi Isam,

Basically, I was interested if your experiments included any behavior due to the unreliability of IP multicast such as the following example outlined in the Groupcomm I-D:

http://tools.ietf.org/html/draft-ietf-core-groupcomm-05#section-3.6

   o  Update group configuration data using multicast PUT: no
      suppression at all.  Use collected responses to identify which
      group members did not receive the new configuration; then attempt
      using CoAP CON unicast to update those group members.

Best Regards,

Akbar

-----Original Message-----
From: Isam Ishaq [mailto:[email-masked]] 
Sent: Thursday, March 28, 2013 5:00 AM
To: Rahman, Akbar
Cc: Dijk, Esko; [email-masked]
Subject: Re: [core] Any experience in using CoAP responses tomulticast requests?

Hi Akbar,

Best,
Isam