Subject: [Roll] connectivity traces to evaluate RPL [was: Determining DADR Contributions]

From: Thomas Watteyne <[email-masked]>
Date: Fri, 18 Sep 2009 11:36:44 -0700

Dear Sung, dear ROLLers,

I strongly support your efforts in building a testbed for ROLL's 
proposals. One complementary technique is to *replay the behavior of 
these routing protocols on connectivity traces* which are gathered in 
real-world deployments. Such traces indicate the states of the links 
(through their PDR) at different times,  and remove the need for - 
unrealistic? - propagation assumptions in simulation.

Jorge Ortiz and David Culler kindly lent us the connectivity traces they 
gathered earlier this year (thanks!), which we have used to populate a 
database, and to run RPL simulations on top. You can look at the result 
at http://wsn.eecs.berkeley.edu/connectivity/. In 
particular, http://wsn.eecs.berkeley.edu/connectivity/network_gradient.php?dataset=soda 
lets you display ETX-based DAG for the sink of your choice. While we 
have used this dataset to show that channel hopping makes sense in WSNs 
(see attached paper), you can obviously download the complete dataset 
(details on the website) to extract whatever information you need. As 
you can see, the system also let's you look at connectivity traces 
gathered by Dust Networks, and we are working on adding a third data set 
from a more urban-wide deployment I did last year.

As a sidenote, our aim is for this website to become a repository of 
connectivity traces in different scenarios (in/outdoors, with/without 
inteference etc). We are currently building firmware to be deployed on 
existing testbeds, but it would be great if we could *add the traces you 
may already have*. Please contact me if you have some, they could be 
very helpful to benchmark RPL against different usage scenarios.

Best regards from sunny Berkeley,
Thomas

> -------- Original Message --------
> Subject:     Re: [Roll] Determining DADR Contributions
> Date:     Thu, 17 Sep 2009 07:40:24 -0400
> From:     Sung Lee <[email-masked]>
> To:     Mischa Dohler <[email-masked]>
> CC:     [email-masked]
> References: 
> <[email-masked]> 
> <[email-masked]>
>
> Dear Mischa,
>
> As mentioned in my posting, we are currently working on a real test. 
> We completed an indoor experiment and are currently working on an 
> outdoor experiment where interference from radios such as 11.b/g and 
> others are present.  In these tests, we are checking to make sure that 
> our test set up reflects reality and checking many theoretical  
> measures to real measures (e.g.,the signal coverage). We are working 
> hard to get the test data ready for the interim meeting.
>
> Best regards,
> Sung
>
> Mischa Dohler wrote:
>> Jerry, thanks for the info. I know the theory and both standards fairly
>> well. As per your own words, the problem is exactly "if these channels
>> are used". Note that your WLAN equipment is not bound to use these 3
>> non-overlapping channels, so that these guardbands may very well be in
>> use. Furthermore, there is no guarantee that .15.4 will really find the
>> guardbands (for a whole list of reasons). In practice, some tests have
>> confirmed that connectivity breaks down because of this interference
>> issue. I see 16 WLAN stations at work and ZigBee doesn't even manage to
>> cover 10% of the theoretically promised distance. I guess this is one of
>> the reasons for the existence of .15.4c and .15.4d. Best regards, 
>> Mischa.
>>
>> [email-masked] wrote:
>>>
>>> Mischa,
>>>
>>> 802.15.4 was designed with 802.11 in mind.  802.15.4 is channelized 
>>> with
>>> 15 channels in the 2.4gHz band.  Following is an excerpt from the IEEE
>>> 802.15.4 specification.  Note how channels 15, 20, 25 and 26 run in the
>>> guardbands of 802.11 channels 1, 6 and 11.  Empirical testing has shown
>>> that the two wireless technologies can nicely coexist if these channels
>>> are used...
>>>
>>> *Mischa Dohler <[email-masked]>*
>>> Sent by: [email-masked]
>>>
>>> 09/16/2009 05:53 AM
>>>
>>>
>>> To
>>>     Sung Lee <[email-masked]>
>>> cc
>>>     [email-masked]
>>> Subject
>>>     Re: [Roll] Determining DADR Contributions
>>>
>>> Dear Sung,
>>>
>>> These connectivity graphs look convincing; however, I guess that one of
>>> your underlying assumptions is that you run the .15.4 network whilst 
>>> the
>>> .11 network is switched off. In reality, you will have loads of already
>>> deployed .11 networks heavily interfering with your .15.4 network which
>>> immediately impacts connectivity.
>>>
>>> Mischa.
>>>
>>> Sung Lee wrote:
>>>  > Dear ROLL WG members,
>>>  >
>>>  > This is a follow up to what we said we would post to the mailing
>>> list.
>>>  > Although this is the summary of simulation of 802.11b and 802.15.4
>>>  > comparison, we are currently working on the real test. We hope
>>> that we
>>>  > can provide this result before the ROLL interim meeting.
>>>  >
>>>  > We would appreciate any comments.
>>>  > Best regards,
>>>  > Sung
>>>  >
>>>  >
>>>  >> ------------------------------
>>>  >>
>>>  >> Message: 2
>>>  >> Date: Thu, 30 Jul 2009 17:21:21 -0400
>>>  >> From: "Ryusuke Masuoka" <[email-masked]>
>>>  >> Subject: [Roll] Determining DADR Contributions
>>>  >> To: "'ROLL WG'" <[email-masked]>
>>>  >> Message-ID: 
>>> <014601ca115b$b0629fb0$1127df10$@[email-masked]>
>>>  >> Content-Type: text/plain;    charset="us-ascii"
>>>  >>
>>>  >> Hi, ROLL WG members,
>>>  >>
>>>  >> In order to move ahead and for us to determine what we/DADR can
>>>  >> contribute, we (Fujitsu) would like to do the following.
>>>  >>
>>>  >> At the ROLL meeting, we realized that many people are interested in
>>>  >> 802.15.4 radio. Our current implementation is on 802.11b radio (1
>>>  >> Mbps) and two wireless characteristics are different. We thought
>>> that
>>>  >> many ROLL members could not determine how good DADR would be 
>>> when it
>>>  >> is applied to 802.15.4 radio. In that regard:
>>>  >>
>>>  >> (1) We will provide PER (packet error rate) and other wireless
>>>  >>   characteristics for both 802.11b (which we already have) and
>>> 802.15.4
>>>  >>   radios in a couple of weeks.
>>>  >>
>>>  >> (2) We will share our DADR 802.15.4 radio implementation experiment
>>>  >>   results by the end of August or in early September.
>>>  >>
>>>  >>   It would be a rather small (50 nodes or so) and preliminary with
>>>  >>   experiment assumptions, (average) hops, data reachability,
>>> etc.  (We
>>>  >>   plan to do a larger experiments (in the order of hundreds of
>>> nodes),
>>>  >>   but it will be somewhat later.)
>>>  >>
>>>  >>   As this is done as a part of system test for customer
>>> deployment, we
>>>  >>   are not sure we can accommodate them all, but please let us know
>>>  >>   what kinds of things/conditions/assumptions we should
>>>  >>   incorporate/consider/make in this experiment. We would appreciate
>>>  >>   your input very much.
>>>  >>
>>>  >> We also plan to see which LLN requirements DADR meets or not,
>>> according
>>>  >> to:
>>>  >>
>>>  >>   Overview of Existing Routing Protocols for Low Power and Lossy
>>> Networks
>>>  >>   draft-ietf-roll-protocols-survey-07
>>>  >>
>>>  >> so that we can better determine which parts of DADR are useful or
>>> not.
>>>  >>
>>>  >> We will try to be as fair as possible. However, if someone can
>>>  >> volunteer to do this, that would be great as we can get a
>>> third-party
>>>  >> evaluation, we would appreciate it very much and we will support 
>>> the
>>>  >> person/group with the information necessary. (... but I am afraid
>>> that
>>>  >> everyone other than us is too busy for this.)
>>>  >>
>>>  >> Regards,
>>>  >>
>>>  >> Ryu
>>>  >>
>>>  >>
>>>  >>
>>>  >>
>>>  >>
>>>  >
>>>  >
>>> ------------------------------------------------------------------------ 
>>>
>>>  >
>>>  > _______________________________________________
>>>  > Roll mailing list
>>>  > [email-masked]
>>>  > https://www.ietf.org/mailman/listinfo/roll
>>> _______________________________________________
>>> Roll mailing list
>>> [email-masked]
>>> https://www.ietf.org/mailman/listinfo/roll
>>> _______________________________________________
> Roll mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/roll

---

From: Philip Levis <[email-masked]>
Date: Tue, 22 Sep 2009 21:44:44 -0700

On Sep 18, 2009, at 11:36 AM, Thomas Watteyne wrote:

> connectivity traces in different scenarios (in/outdoors, with/ 
> without inteference etc). We are currently building firmware to be  
> deployed on existing testbeds, but it would be great if we could add  
> the traces you may already have. Please contact me if you have some,  
> they could be very helpful to benchmark RPL against different usage  
> scenarios.

Why not just use CRAWDAD?

One major issue with using this dataset is that it breaks spatial  
correlation. Each sender transmitted at a different point in real  
time, but in using the trace, you assume the sender periods overlap.

Phil

---

From: Thomas Watteyne <[email-masked]>
Date: Wed, 23 Sep 2009 10:05:52 -0700

Hi Phil, thanks for the pointer! We are looking for time-varying PDR 
connectivity traces, something which (unfortunately) we have been unable 
to find in CRAWDAD or similar projects. In an ideal world, those traces 
are dense in time, neighborhood and frequency, although trade-offs 
necessarily apply. You are absolutely right in pointing out that this 
dataset is not dense in time, which makes it unsuitable for simulations. 
We are working building a network that answers our needs, and the 
dataset we now have is, well, a first shot :)
Thomas

Philip Levis wrote:
>> inteference etc). We are currently building firmware to be deployed 
>> on existing testbeds, but it would be great if we could add the 
>> traces you may already have. Please contact me if you have some, they 
>> could be very helpful to benchmark RPL against different usage 
>> scenarios.
>