Subject: Re: [Roll] Working Group Last Call: draft-ietf-roll-protocols-survey-02

From: "Dearlove, Christopher (UK)" <[email-masked]>
Date: Fri, 5 Dec 2008 15:18:15 -0000

I think this document is correct in its assessment
that no existing routing protocol meets the five
rigid specifications given. However on the way to
that conclusion

- If the sole purpose is to answer the question as to
  whether any existing protocol meets the very tough
  standards given, then the purpose is met, the answer
  is no. But that doesn't actually help you take the
  next step. Are any of these protocols adaptable to
  meet theses requirements, or do you have to start
  from scratch? (And some of these protocols have more
  options than may be obvious. Just because as configured
  in existing specifications it fails, doesn't mean it
  always will. And some things that may appear fundamental
  may not be. For example I think it's quite easy to
  modify NHDP to bound its neighbourhood information in
  a dense network - and if it's not dense, it's bounded
  anyway. How well that modification works is however
  something that would need work to test.)

- And why don't these protocols meet these specifications?
  OK, maybe the designers could have done better (I don't
  know). But maybe first there are other major issues that
  have to be traded off (latency for example - you can really
  cut down on signalling overheads if you allow unbounded
  latency), and second, can those constraints actually all
  be met in a reasonable way? (If not, there's a danger of
  start from clean sheet, discover along the way you need to
  compromise, and then that someone's already been down that
  route.)

- Along the way, not everything is undisputed. For example
  in the traffic bounding considerations of OLSRv2 it
  is clear that neither the timing backoff nor "fisheye"
  variable flooding of OLSRv2 is taken into account.
  I'm not suggesting that these are enough to achieve
  the required O(N) bound (at least not without probably
  unacceptable delays) but the simple O(N^2) before MPR
  adjustment isn't necessarily quite that simple. (And how
  much you can use those features depends on density and
  variability, the latter of which is also a major issue.)

I'm not suggesting another round of modifying the document,
just putting down that the document has a narrow purpose,
and shouldn't be stretched beyond it. It's fine that someone
might say that this document shows that DYMO (for example)
out of the box isn't what you want. (I don't personally know
that to be true however.) But that shouldn't be taken as saying
DYMO doesn't work, we must start afresh. I don't know if
you could start with DYMO (or the other six protocols) and
get to where you want to go. I'm just saying this document
doesn't answer that question, but assuming that it does is a
future danger to be avoided.

As a minor nit however, in section 4.4, the use of
O(R log(L) + e) where e is a small constant is, while not
actually wrong, is confusing. O(R log(L) + e) is exactly
the same as O(R log(L)). It's only meaningful to include
e if using something other than big-O notation.

********************************************************************
This email and any attachments are confidential to the intended
recipient and may also be privileged. If you are not the intended
recipient please delete it from your system and notify the sender.
You should not copy it or use it for any purpose nor disclose or
distribute its contents to any other person.
********************************************************************

_______________________________________________
Roll mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/roll

---

From: "Ian Chakeres" <[email-masked]>
Date: Fri, 5 Dec 2008 11:49:50 -0500

I've reviewed draft-ietf-roll-protocols-survery. I have a few comments and nits.

In the Introduction, it would be great if LLN constraints and
requirements that other networks do not poses were listed/summarized
before reference to the routing-req docs. The existing text left me
guessing about what exactly was different.

I would suggest rephrasing the DYMO discussion related to hopcount in
Section 3. Perhaps stating that "distance" is the metric in DYMO, and
that hopcount is the minimum distance allowed.

In Section 4.2, the motivation provided is the possibly large network
size (250-millions) and deep network (20 hops). Network size and
routing information size do not have a direct connection when
hierarchical addressing is used. Perhaps this motivation could be
removed and simply the last sentence kept. Given this change maybe the
suitability metric should become routing table size, instead of table
scalability.

Based on the suitability table and description contained in the ROLL
survey document, I have updated the DYMO (v16) draft .  First, I added
text to indicate that distance may be influenced at multiple points in
the processing of RM. This changes should allow link and node metrics
to influence distance before and after RM processing. Secondly, I have
made forwarding of RERR optional (through strongly suggested). This
should allow a smart implementation (e.g. using an "active" precursor
list) to limit the propagation of RERR messages. Thank you for
pointing out these minor issues in DYMO.

If you have any other suggestions for the DYMO specification that you
feel would make it better fit to suit ROLL requirements, please let me
know.

Thanks.
Ian Chakeres

On Fri, Nov 21, 2008 at 1:36 AM, JP Vasseur <[email-masked]> wrote:
> Dear WG,
> draft-ietf-roll-protocols-survey-02 is now fully stable, the last revision
> has been published after the ROLL Interim meeting in October and all
> comments received so far had been addressed.
> This emails start a 2-week Working Group Last Call that will end on December
> 5 at noon ET.
> Please send your comments to the authors and on the mailing list.
> Thanks.
> JP.

---

From: JP Vasseur <[email-masked]>
Date: Fri, 5 Dec 2008 18:14:06 +0100

Thanks for your comments Christopher. We will also make sure to  
address the nits mentioned below.

JP.

---

From: Philip Levis <[email-masked]>
Date: Fri, 5 Dec 2008 10:37:24 -0800

>  compromise, and then that someone's already been down that

This is a great discussion to have. I think you're touching on an  
important point, that many of them were designed with different  
priorities than we encounter in LLNs. For example, if you want to run  
high throughput TCP, jitter and high latencies are an issue, while in  
LLNs this might not be as important as the energy efficiency of a low- 
throughput flow. These subtle differences lead to design choices which  
cause issues (unimagined in the general mesh case) in LLNs. That being  
said, there's a lot to be learned, as protocols meet some of the  
criteria.

>  variability, the latter of which is also a major issue.)

This is probably true. But I'm sure you can understand our trepidation  
in stating a strong bound based on some simple simulation studies from  
several years ago, when the actual mechanism is itself not clearly  
defined.

> future danger to be avoided.

I agree completely. The document has a very narrow scope because that  
is the current charter of ROLL. Also, it's important to resolve this  
question clearly, without conflating it with the much larger  
discussion of what steps the WG should take next.

> e if using something other than big-O notation.

True enough. The concern here is that *any* constant factor is elided  
from big-O notation. Given that many LLN devices have finite energy,  
one can then simply say that the cost of a protocol is bounded by a  
constant and... get nowhere. Maybe the thing to do is point out that  
this epsilon is variable dependent on the deployment and its  
considerations? So while it is a small constant in the context of a  
specific network, it is not constant across networks.

Many thanks for your comments,

Phil

---

From: "Omprakash Gnawali" <[email-masked]>
Date: Sun, 7 Dec 2008 18:37:46 -0800

On Fri, Dec 5, 2008 at 8:49 AM, Ian Chakeres <[email-masked]> wrote:
...
> suitability metric should become routing table size, instead of table

I would go a bit further and ask if scalability and routing table size
should be two different metrics. The former says, if I want to deploy
a network with 2 million nodes instead of 1 million nodes, how much
more memory would I need while the later should provide the absolute
bound on the memory requirement. I know these two metrics are somewhat
related but considering we know the size of the largest network, it
makes sense to talk about both.

Do people care more about scalability or bounds?

Does it make more sense to be more explicit about the bounds ? For
example, a node should not have to keep track of more than 2000
entries.

Does it make sense to talk about amount of RAM instead? Talking about
scalability is making this discussion about this particular metric
more abstract than it should be.

- om_p

---

From: Thomas Heide Clausen <[email-masked]>
Date: Mon, 8 Dec 2008 17:00:32 +0100

Dear All,

I've been reviewing the latest incarnation of this I-D, and I have a  
couple of general concerns with this I-D and, more specifically, with  
the approach it seems to be suggesting for the working group.

	o	The document seems to set the bar of acceptance at  "the holy  
grail of a routing protocol",
		i.e. defines "acceptable" to be "zero-cost [more on that in a  
following bullet],
		instant-convergence" -- thereby causing all routing protocols to  
almost by definition be
		judged "insufficient" -- which is reflected in the evaluation of  
the existing set of routing
		protocols.
	
	o	As I assume ROLL is designing "for the real world", compromises  
and trade-offs are
		paramount in protocol design. By elevating a set of criteria (sec.  
3 and 4 -- "table scalability",
		"loss response", "control cost", "link cost" and "node cost") above  
the rest, while pointing out
		that (i) an approach not meeting the bar on all of those is  
unacceptable and (ii) "applications
		introduce additional, more stringent, requirements", this I-D  
effectively limits the trade-offs
		possible for when designing protocols for these applications.

	o	From the previous point, it is not convincing that eliminating  
these trade-offs is justified.
		While I believe that indeed *any* application would welcome "the  
holy grail of a routing
		protocol", recognizing that such may not exist, it may for some  
application be right to
		trade off, say, "higher control cost" against  "low memory  
footprint". Something which is
		explicitly excluded (2nd paragraph section 4).

	o	Going by where the bar for acceptance is set in this document, I  
have thought and come
		up with indeed two solutions that have "zero-cost-instant- 
convergence" properties, and
		they both satisfy the IP delivery semantics. One is "drop all  
traffic, don't even try" and the
		other is "flood all traffic, don't maintain a routing table".

		Of course, those two proposals are in jest, as I am sure that the  
trade-off I am making
		-- sacrificing delivery ratio for the former, satisfying network  
traffic and energy (but not
		as a consequence of the routing protocol relaying control traffic)  
for the latter are not
		acceptable in an absolute sense -- but the routing protocol will  
itself require have
		zero-cost, zero-memory, instant-convergence.

	o	From the previous and first bullet, "control cost" is an  
insufficient measure. "Flooding all
		data" has zero "control cost" but a large overall cost (many  
retransmissions, lots of battery
		drain, lots of media use). A routing protocol incurring some  
"control cost" to gain lower
		"data  transmission cost" likely is a better tradeoff, especially  
if data traffic is a common
		occurrence in the network.

		IOW, looking at "control cost" as a separate metric is not terribly  
useful unless the
		consequences in terms of the "data cost" and "data delivery ratio"  
are considered
		as well.

Chris Dearlove summarize well in saying that the document does well  
in setting out a rigid set of "pass" criteria and observing that none  
of the six protocols analyzed satisfies those off-the-shel -- but  
that this should not be stretched to  assume that an entirely new  
design is required.

I also join Chris in his comment that, for example, OLSRv2 offers  
"timing backoff" and "fisheye variable flooding" without variations  
to the routing protocol [the specificities of how to set timers or  
vary flooding is a policy matter that is dependent on deployment, but  
can be done entirely conformant with its specification] -- and that  
this is not considered. I'll go further than Chris and suggest that  
not considering the use of such options, which are present in the  
protocols but where a policy specification dependent on deployment is  
required, casts doubt on the assessment of the protocols as presented  
- not just OLSRv2, but such options exist for other protocols as well.

I'll send another email with "nits" to the document, however I feel  
that the issues above need to be discussed and resolved.

Cheers,

Thomas

---

From: "Dearlove, Christopher (UK)" <[email-masked]>
Date: Mon, 8 Dec 2008 16:25:34 -0000

Philip Levis
> Christopher Dearlove

> throughput flow.

Yes, but. The "but" is that although latency may not be as much as an
issue in some LLNs as it may be in some other ad hoc networks, first
it may be (I can certainly conceive of an LLN with a low latency
requirement), but second even when it isn't, there are likely to be
some latency constraints, and that's an important part of the mix.
By ignoring it I may meet the three complexity constraints, but
imposing it I may not - even if it is fairly lax. (I only use latency
as an example.)

> These subtle differences lead to design choices which  
> cause issues (unimagined in the general mesh case) in LLNs.

I don't believe any requirement in ROLL is unimagined in the general
MANET case - in the decade or so the MANET WG has been around (quite
a bit of it before my time) many things have been imagined. What they
may have been is considered as part of a tradeoff and possibly given
a lower priority. Note that often the MANET WG has ended up with view
more like "X should be minimised, as far as possible, subject to other
constraints" rather than "X should be O(M), for M ...". This doesn't
mean that the MANET WG hasn't thought "wouldn't it be great if such
and such didn't increase as the network got larger" but rather
"unfortunately such and such has to increase with the size of the
network, given our other constraints, but let's try to keep it
manageable".

I use the MANET WG as an example. Not all the protocols considered are
MANET protocols of course. But I wouldn't take a bet that exactly the
same issues hadn't been considered by the designers of OSPF, even
though their tradeoffs are yet again different.

---

From: Thomas Heide Clausen <[email-masked]>
Date: Mon, 8 Dec 2008 17:57:02 +0100

Dear All,

I've been reviewing the latest incarnation of this I-D, and as  
promised in my previous email, I have a bunch of nits in addition to  
my more general concerns -- this is in addition to those general  
concerns, the items that made me stumble in my reviewing.

Terminology:

	o	Would it be possible to group "protocol acronyms for candidate
		routing protocols evaluated in this document"

	o	LLN is specified. L2N is used in section 4.4. Is that the same?

Introduction:

	o	I am vary when crediting "Moores Law" as having caused something

	As Moore's Law has reduced computer prices and form factors,

		Would it not be as correct to omit the reference hereto and just  
observe
		that prices and form factors have reduced?

	o	inter-networking -- why the hyphen?

	o	Paragraph 2: as we are in "introduction-space" and as consumer
		devices has been presented as an example device, I wonder...my (dated)
		consumer cellphone runs a stock MANET routing protocol for days  
over WiFi
		without recharge -- and as a consumer, this suffices for me.  
Clearly, the goal
		is also stuff that can't be recharged once every 2-3 days (embedded  
sensors,
		say) -- could a throwaway sentence be added to strengthen the low- 
power
		requirement somewhere?

		Specifically, the leap from consumer devices in paragraph 1 to this  
is brutat:

		Cost points and energy limitations cause these devices
	to have very limited resources: a few kB of RAM and
	a few MHz of CPU is typical.

		Which are "these devices" -- not the "laptops, palmtops and  
cellphones" cited in
		paragraph 1 ;)

	o	I would cut paragraph 2 into two paragraphs, just before

			In practice, wireless....

	o	Paragraph 3 starts:

			These low power and lossy networks introduce constraints and
    	requirements that other networks typically do not possess
    	([I-D.ietf-roll-home-routing-reqs] and
    	[I-D.ietf-roll-indus-routing-reqs]).  As they were not designed  
with
    	these requirements in mind, existing protocols may or may not work
    	well in LLNs.

			(i)	could we enumerate these constraints, or at least a subset  
hereof, to justify that
				"other networks" do not have those?

			(ii)	which are those "other networks", btw? ASes in the Internet?  
Bluetooth network surely does
				have some of the same constraints.

			(iii)	the last "As they were not designed" leaves me wondering  
what "they" refer to,
				could we not write "As such existing networks were not designed"
				-- Or something similar?

	o	Paragraph 3 ends:

			this document simply seeks to answer the question: do LLNs
	require a new protocol specification document at all?

			This document does, in my opinion, not actually answer this  
question. This document rather
			evaluates existing protocols and identifies areas where additional  
study is required to
			determine *if* these protocols can be parametrized and deployed OR  
if they can be adapted
			OR -- failing the two previous -- new protocol specifications are  
required.

			Also see Chris Dearlove's email.

Methodology:

	o	Paragraph 2:

			As per my previous email to the list, I think that this paragraph  
is incorrect and too limiting.
			It should rather read that this document only evaluates with  
respect to the very strict set of
			metrics listed, but that applications or deployments may make  
other trade-offs, including
			trading off the metrics in this document in favor of other benefits.

			IOW, the "necessary, but not sufficient" part, I think, is  
inappropriate and can limit the wg's
			ability to make appropriate trade-offs later.

	o	Paragraph 3:

			rouge space before the comma after olsrv2-citation

	o	Paragraph 4:

			"considering the best implementation possible"

		that's a very bold statement, and likely not correct. For example,  
OLSRv2 offers the ability
		to do timer-backoff and fisheye variable flooding -- which affects  
the average-case
		performance and where the protocol can be tuned such that the  
average-case is
		the actual deployment -- yet, this is not considered in the  
evaluation. The same applies
		for the other protocols.

Suitability:

	o	Paragraph 1:

			"generally applicable" is a very vague statement, what does that  
mean? The ID said
			earlier that the criteria are "necessary but not sufficient", so  
this I-D can not state
			based on the criteria any "general applicability", can it?

	o	Paragraph 2:

			I feel that two criteria are missing:

				o	path lengths (length as in "sum of costs")
				o	delivery ratio / success rate of data traffic

			Without those, the other metrics have little meaning.

	o	Section 4.2:

			o 	"of devices noted of the urban" -- should that not be
				"of devices as noted in the urban" ???

			o	2nd paragraph, it emerges that:

				low-power and lossy networks behave principally as data collection
	   networks and principally communicate through routers to data
	   collection points in the larger Internet

				I'm not disputing that, but if there are a set of such general  
characteristics, could
				it not be an idea listing those in a section up front  
("applicability statement" or the like)
				with due references to the individual requirements drafts?

				IOW, reading the I-D leaves me starting out thinking "Generally,  
wireless networks", and
				getting "drop by drop" narrowed down to specific traffic and  
other characteristics. I would
				much prefer seeing such "up front" to understand if the  
considerations in this I-D applies
				or not to a given problem that the reader might have?

	o	Section 4.3:

			o	It *seems* to me that among the requirements, the "energy  
constraint" issue is in direct
				conflict with this statement:

					The protocol should also "always be
	   	in the process of optimizing the system in response to changing  
link
    		statistics."				
			
				So conserving energy by dialing down the CPU speed is not an  
option considered?

	o	Section 4.4 and elsewhere:

			o	O(R log(n) +e), assuming R and e are constants, are not std. O 
(...) notation
				(edit, I see now that I get on-line that Dearlove made the same  
comment)

Routing Protocol Taxonomy

	o	2nd paragraph:

			o	"and routers synchronize their LSDBs"

				 I am not sure if it is intended, but I tend to think "OSPF  
database synchronization with  DRs"
				when reading the above -- and note that e.g. OLSR does not employ  
such an operation,
				nor does OSPF when deployed over the MANET interface type as  
described in some of the
				current OSPF I-Ds defining the protocol behavior over these  
interfaces.

				IOW, it's not clear exactly what is meant here.

	o	6th paragraph:

			o	a requirement of "4kb of RAM" appears out of the blue. Goes with  
the suggestion to
				have an "applicability statement" where such constraints are  
explicitly enumerated.
				I agree that running OSPF in 4KB of RAM is a challenge...as is IP ;)

	o	There's a loose "Protocols Today" on top of page 11. Should that  
be a subsection?

	o	3rd paragraph on page 11:

			Because wireless
	   links can suffer from significant temporal variation, link state
	   protocols can have higher traffic loads as topology changes must
	   propagate globally			

		Not necessarily true. LS protocols can run in a mode wherein the  
control traffic overhead is independent
		also of the topology changes. It's a trade-off, of course, against  
"something else".

		Also:
			"One major protocol, DSR, does not easily fit"

		Why "major"?

OLSR

	o	Should the heading not be "OLSR & OLSRv2" as the text applies to  
both?

	o	Neither OLSR nor OLSRv2 deploys "topology discovery packets", but  
propagates
		LSAs called TC messages. In general "discovery packets" are  
associated with the
		request-reply cycles of reactive protocols such as DYMO or AODV

	o	OLSRv2 incorporates metrics, in addition to hop-counts.

Security Considerations

	o	I am a little uncomfortable with this being left blank, as I  
believe that security considerations
		are important for also ROLL protocols, and as I believe that the  
different protocol options
		offer different challenges and benefits wrt. security.

Appendix A:

	o	OSPF -- does it even (as you describe it) run over the kinds of  
networks that are studied, without
		modifications? I.e. DR selection and such, would it work? I ask, as  
in the MANET-OSPF work in
		OSPF, a lot of effort has been made on these topics, and I wonder  
if you've been following that?

	o	OLSRv2
			
			-	1st line, "flooding this information" -- which information?

			-	I believe that ignoring the MPR, timer-backoff and FSR options,  
as present in
				the OLSRv2 specification, is not appropriate, and suggest that  
these be evaluated
				also in this section. Ignoring those is, at best, incorrect.

			-	changes in the link set does not REQUIRE immediately re-flooding  
of LS information.
				One MAY do so, but the protocol specifies considerations and  
constraints hereof.

Cheers,

Thomas

Cheers,

Thomas

---

From: Philip Levis <[email-masked]>
Date: Mon, 8 Dec 2008 09:02:27 -0800

> feel would make it better fit to suit ROLL requirements, please let me

Ian,

I'm glad our reading of the draft was helpful. We'll be sure to update  
the text to reflect your edits and points.

We've gone back and forth a lot on DYMO's relaxation of link cost  
computation, so it's great to hear DYMO's author's take. :)

Your point about hierarchical addressing is interesting, and I think  
it relates to comments others have made on fisheye routing and other  
state suppression approaches. Help me out if I'm mistaken here, but I  
don't recall any work on address allocation/distribution in wireless  
networks to enable good route aggregation. This is particularly  
difficult in the IPv6 case because of how addresses are specified. So  
I'm wary of assuming hierarchical addressing, when in some cases it  
might not be possible. It's not quite like the wired Internet where  
there is some aggregation due to the administrative domain nature of  
how addresses are allocated.

Phil

---

From: "Dearlove, Christopher (UK)" <[email-masked]>
Date: Mon, 8 Dec 2008 17:58:24 -0000

> state suppression approaches.

Not quite. Hierarchical allocation is indeed a state suppression
approach. However fisheye is a control message suppression approach.

> Help me out if I'm mistaken here, but I  
> networks to enable good route aggregation.

There's work in the literature, but the relevant IETF WG (autoconf)
hasn't got to solutions yet.

---

From: Thomas Heide Clausen <[email-masked]>
Date: Mon, 8 Dec 2008 19:50:34 +0100

> approach. However fisheye is a control message suppression approach.

And, it should be added that such are, of course complementing each  
other.

Thomas

---

From: "Emmanuel Baccelli" <[email-masked]>
Date: Tue, 9 Dec 2008 10:02:22 +0100

Hi all,

I think this draft is far from being ready to become an RFC, and such for
several reasons, some of which are, unfortunately, structural.

1 - The draft is evaluating moving targets such as OLSRv2, Dymo, NHDP, which
are potential solutions that are not finalized yet. These protocols could
meet more requirements in the end (maybe all of them?). For this reason, the
"pass/fail" table of section 5 cannot honestly list anything else but a
series of "question marks" (or "pass"), for now, for these protocols. This
makes this table rather useless, and thus this whole draft rather useless,
presently.

2 - The ROLL charter mentions that " Existing IGPs, MANET, NEMO, DTN routing
protocols will be part of evaluation." I did not see anything mentioned
about DTN protocols, or Nemo in the draft, not even a justification why they
are not mentioned.

3 - The draft is not constructive in its current shape. It is on the other
hand extremely controversial with its judgement, and does not reflect a
consensus within the community, not even a rough one.

Emmanuel

On Mon, Dec 8, 2008 at 7:50 PM, Thomas Heide Clausen <[email-masked]
> wrote:

> And, it should be added that such are, of course complementing each other.
>
> Thomas
>
>

---

From: JP Vasseur <[email-masked]>
Date: Tue, 9 Dec 2008 11:48:31 +0100

Hi Emmanuel,

I will let the authors of the draft answer the excellent comments that  
have been raised on the list during WG LC (and I'll add mine) but I  
would like to comment on your statement and hopefully clarify our  
objective.

On Dec 9, 2008, at 10:02 AM, Emmanuel Baccelli wrote:

> Hi all,
>
> I think this draft is far from being ready to become an RFC, and  
> such for several reasons, some of which are, unfortunately,  
> structural.
>
> 1 - The draft is evaluating moving targets such as OLSRv2, Dymo,  
> NHDP, which are potential solutions that are not finalized yet.  
> These protocols could meet more requirements in the end (maybe all  
> of them?). For this reason, the "pass/fail" table of section 5  
> cannot honestly list anything else but a series of "question  
> marks" (or "pass"), for now, for these protocols. This makes this  
> table rather useless, and thus this whole draft rather useless,  
> presently.

NO. Let me clarify something really important.

The WG cannot afford to wait until all protocols are finalized to get  
a routing solution for LLN. The industry is waiting for a routing  
solution for such networks and time is absolutely critical. No we have  
to analyze the state of the art as of today.

>
> 2 - The ROLL charter mentions that " Existing IGPs, MANET, NEMO, DTN  
> routing protocols will be part of evaluation." I did not see  
> anything mentioned about DTN protocols, or Nemo in the draft, not  
> even a justification why they are not mentioned.
>

Would you want to make a contribution ?

> 3 - The draft is not constructive in its current shape.

I do not think that your comment is fair. The draft has been discussed  
in the WG for quite some time and extensive discussion took place  
during the interim WG and on the ML.

> It is on the other hand extremely controversial with its judgement,  
> and does not reflect a consensus within the community, not even a  
> rough one.
>

Bear with us: we are not *judging* protocols: we are evaluating the  
existing routing protocols *in light* of our specific requirements.  
Yes one can argue for ever on a pass/fail criteria but we have to be  
driven by rough consensus to be able to move forward. Of course, if  
there is a mistake when evaluating a protocol it has to be fixed.

Thanks.

JP.

---

From: Thomas Heide Clausen <[email-masked]>
Date: Tue, 9 Dec 2008 12:51:51 +0100

Hi JP and all,

A quick comment/question below....

On Dec 9, 2008, at 11:48 AM, JP Vasseur wrote:

> Hi Emmanuel,
>
> I will let the authors of the draft answer the excellent comments  
> that have been raised on the list during WG LC (and I'll add mine)  
> but I would like to comment on your statement and hopefully clarify  
> our objective.
>
> The WG cannot afford to wait until all protocols are finalized to  
> get a routing solution for LLN. The industry is waiting for a  
> routing solution for such networks and time is absolutely critical.

Without being contentious, this is as such a fair point: "industry is  
waiting for a solution". An argument can be made that in that case  
"starting from scratch" would be counter-productive, and that it'd be  
beneficial to start from something already in existence, or already  
in development, if at all possible -- with the "already in  
development" being an asset as it gives a head start and,  
potentially, can accommodate the ROLL wg suggestions still.

I deduct, therefore, that the intent is to next analyze in detail the  
potential "tweaks and tunings and potential extensions" possible to  
the existing six or so protocols in the roll-protocols-survey-02?

Thomas

>  No we have to analyze the state of the art as of today.
>
>> 2 - The ROLL charter mentions that " Existing IGPs, MANET, NEMO,  
>> DTN routing protocols will be part of evaluation." I did not see  
> I do not think that your comment is fair. The draft has been  
> discussed in the WG for quite some time and extensive discussion  
> took place during the interim WG and on the ML.
>
>> It is on the other hand extremely controversial with its  
>> judgement, and does not reflect a consensus within the community,  
>> not even a rough one.
>>
>
> Yes one can argue for ever on a pass/fail criteria but we have to  
> be driven by rough consensus to be able to move forward. Of course,  
> if there is a mistake when evaluating a protocol it has to be fixed.
>
> Thanks.
>
> JP.
>
>> Emmanuel
>>
>>
>> On Mon, Dec 8, 2008 at 7:50 PM, Thomas Heide Clausen  
>> <[email-masked]> wrote:
>>

---

From: "Emmanuel Baccelli" <[email-masked]>
Date: Tue, 9 Dec 2008 12:54:05 +0100

On Tue, Dec 9, 2008 at 11:48 AM, JP Vasseur <[email-masked]> wrote:
>
>
> 1 - The draft is evaluating moving targets such as OLSRv2, Dymo, NHDP,
> which are potential solutions that are not finalized yet. These protocols
> could meet more requirements in the end (maybe all of them?). For this
> reason, the "pass/fail" table of section 5 cannot honestly list anything
> else but a series of "question marks" (or "pass"), for now, for these
> protocols. This makes this table rather useless, and thus this whole draft
> rather useless, presently.
>
>
> NO. *Let me clarify something really important.*
>
> The WG cannot afford to wait until all protocols are finalized to get a
> routing solution for LLN. The industry is waiting for a routing solution for
> such networks and time is absolutely critical. No we have to analyze the
> state of the art as of today.
>

This seems incoherent to me. On one hand you say "we need a solution right
now". And on the other hand, this document dismisses all existing
approaches, because "we want the dream protocol". To me this sounds like
we're heading towards another research phase, and you will thus not get a
solution before long.

> routing protocols will be part of evaluation." I did not see anything
> mentioned about DTN protocols, or Nemo in the draft, not even a
> justification why they are not mentioned.
>
>

I am suggesting we consider existing potential candidates before we go out
there and maybe reinvent the wheel.

> I do not think that your comment is fair. The draft has been discussed in
> the WG for quite some time and extensive discussion took place during the
> interim WG and on the ML.
>

Yes there have been some discussions, but there is no consensus on the
approach taken by this document. And there is no rough consensus either.

Emmanuel

---

From: JP Vasseur <[email-masked]>
Date: Tue, 9 Dec 2008 15:43:03 +0100

Hi Thomas,

On Dec 9, 2008, at 12:51 PM, Thomas Heide Clausen wrote:

> Without being contentious, this is as such a fair point: "industry  
> is waiting for a solution". An argument can be made that in that  
> case "starting from scratch" would be counter-productive, and that  
> it'd be beneficial to start from something already in existence, or  
> already in development, if at all possible -- with the "already in  
> I deduct, therefore, that the intent is to next analyze in detail  
> the potential "tweaks and tunings and potential extensions" possible  
> to the existing six or so protocols in the roll-protocols-survey-02?
>

Agreeing with your statement.

If we can find an existing solution, adapt it and make it work, that  
would be by far the preferred path to take.

Two comments though:
1) We all know that we can adapt lots of protocols to meet a set of  
requirements but we also need to make sure that we do not end up with  
a protocol with lots of features that we will not use and that will  
require extra memory, CPU, ... We are running on highly constrained  
environments with very limited resources.
2) Many of the active ROLL WG members participants have real world  
experience, thus we do not start "from scratch" per say, and we could  
leverage some of the experience learnt from deployed proprietary  
protocols. We will *no* embrace an existing proprietary protocol but  
we can learn lessons from deployed networks.

To summarize:
1) Extensive work has been done on the requirements front for 4  
identified use cases,
2) Before starting any protocol work, we had to answer to the  
following question: "Can we find an existing protocol that meets our  
requirements ?". The answer to this question is "no", which does not  
mean that we could not adapt an existing routing protocol (!). I think  
that we have a consensus here, otherwise let us know.
Note that we will not continue to work on this ID (hence the WG LC),  
the aim of which was to answer the question above.
3) The next step is to see whether (1) we can adapt an existing  
routing protocol with the help of the WG "owning" these protocols or  
(2) design a new routing protocol (in ROLL). For this we need to  
recharter. And we should explore both paths in parallel.

Does that make sense ?

Many Thanks.

JP.

---

From: JP Vasseur <[email-masked]>
Date: Tue, 9 Dec 2008 15:53:22 +0100

Hi Emmanuel,

On Dec 9, 2008, at 12:54 PM, Emmanuel Baccelli wrote:

> This seems incoherent to me. On one hand you say "we need a solution  
> right now". And on the other hand, this document dismisses all  
> existing approaches, because "we want the dream protocol". To me  
> this sounds like we're heading towards another research phase, and  
> you will thus not get a solution before long.

Emmanuel, as you might have noticed, there are many solutions  
*deployed* out there, unfortunately not IP-based and this is why we  
are here. This is not research at all (not even sure why we are saying  
this).
Yes the industry does need a solution *now*, which does not mean that  
we should rush out and select an existing routing protocol if it does  
not meet our requirements.

In other words,
1- There is a lot of expertise on routing for LLN in this WG
2- There is clearly a need for a solution *today*: look at all other  
(deployed) non IP solutions out there
3- Detailed analysis of the routing requirements has been conducted
4- We are not talking about a "dream protocol": again this is very  
much realistic since solutions do exist today (not IP unfortunately :- 
( ....)
5- We are not excluding to use an existing (extended) protocol at all  
provided that it meets our requirements and it does not carry extra  
overhead that will comprise their use on highly constrained device.

> I am suggesting we consider existing potential candidates before we  
> go out there and maybe reinvent the wheel.
>

Your help is greatly appreciated.

> Yes there have been some discussions, but there is no consensus on  
> the approach taken by this document.

Thanks for your feed-back here. What do you propose ?

> And there is no rough consensus either.
>

Please refer to the WG minutes + several emails on the ML, with a good  
consensus. That being said, I hear you and thanks for your comments. I  
do register your disagreement. Having read again others' email, I do  
not think that this is a general feeling.

Thanks.

JP.

---

From: Thomas Heide Clausen <[email-masked]>
Date: Tue, 9 Dec 2008 16:01:36 +0100

Hi JP,

On Dec 9, 2008, at 15:43 PM, JP Vasseur wrote:

>>> that have been raised on the list during WG LC (and I'll add  
>>> mine) but I would like to comment on your statement and hopefully  
>>> clarify our objective.
>>>
>>>> These protocols could meet more requirements in the end (maybe  
>>>> all of them?). For this reason, the "pass/fail" table of section  
>>>> 5 cannot honestly list anything else but a series of "question  
>>>> marks" (or "pass"), for now, for these protocols. This makes  
>>>> this table rather useless, and thus this whole draft rather  
>>>> useless, presently.
>>>
>> it'd be beneficial to start from something already in existence,  
>> or already in development, if at all possible -- with the "already  
>> in development" being an asset as it gives a head start and,  
>> the potential "tweaks and tunings and potential extensions"  
>> possible to the existing six or so protocols in the roll-protocols- 
>> survey-02?
>>
>
> If we can find an existing solution, adapt it and make it work,  
> that would be by far the preferred path to take.
>

So far so good -- it seems sensible to explore that.

> requirements but we also need to make sure that we do not end up  
> with a protocol with lots of features that we will not use and that  
> will require extra memory, CPU, ... We are running on highly  
> constrained environments with very limited resources.

No objections to this.

> experience, thus we do not start "from scratch" per say, and we  
> could leverage some of the experience learnt from deployed  
> proprietary protocols. We will *no* embrace an existing proprietary  
> protocol but we can learn lessons from deployed networks.

Just to clarify, I did not intend to dismiss the experience of the  
participants in the ROLL wg, and my references to "from scratch"  
referred only to "IETF heritage"  -- if you catch my drift.

> following question: "Can we find an existing protocol that meets  
> our requirements ?". The answer to this question is "no", which  
> does not mean that we could not adapt an existing routing protocol  
> (!). I think that we have a consensus here, otherwise let us know.
> Note that we will not continue to work on this ID (hence the WG  
> LC), the aim of which was to answer the question above.

This, I am not in agreement with at all.

As I indicated in my long email, the survey I-D is focusing narrowly  
on some (easily quantified) metrics, but ignoring others.

For example "control cost" is included, but "path cost" is not -- and  
one may in this case trade off one for another. The cited example  
(zero-control-cost, flood all data) was in jest, but there *are*  
protocols that trade off path lengths for lower control cost - and  
vice-versa.

Note that this is a single example only, I've identified others in  
other emails, and I shan't rehash it all here.

The point is, that making potentially constraining recommendations  
based on an incomplete set of non-independent evaluation metrics  
isn't a good starting point for the  wg -- rechartering (as you  
indicate below) or otherwise.

I'd like to suggest that the authors of the survey I-D take on board  
the comments produced thus far, and complemented by the evaluation  
metrics otherwise indicated.

As such, I respectfully join Emmanuel Baccelli in suggesting that the  
document is not ready for publication (yet).

Thomas

> routing protocol with the help of the WG "owning" these protocols  
> or (2) design a new routing protocol (in ROLL). For this we need to  
>>>> anything mentioned about DTN protocols, or Nemo in the draft,  
>>>> not even a justification why they are not mentioned.
>>>>
>>>
>>>> judgement, and does not reflect a consensus within the  
>>>> community, not even a rough one.
>>>>
>>>
>>> Bear with us: we are not *judging* protocols: we are evaluating  
>>> the existing routing protocols *in light* of our specific  
>>> requirements. Yes one can argue for ever on a pass/fail criteria  
>>> but we have to be driven by rough consensus to be able to move  
>>> forward. Of course, if there is a mistake when evaluating a  
>>> protocol it has to be fixed.
>>>
>>> Thanks.
>>>
>>> JP.
>>>
>>>> Emmanuel
>>>>
>>>>

---

From: "Dearlove, Christopher (UK)" <[email-masked]>
Date: Tue, 9 Dec 2008 15:03:54 -0000

> much realistic since solutions do exist today (not IP unfortunately
:-( ....)

Can you point me to an existing (doesn't have to be IP) protocol
that passes all five of the ID's tests, without some major drawback
in some other area. I for one would find it interesting and useful.
(Obviously it would have to have a specification, not just a claim
made about a proprietary protocol.)

If one or more such protocols exist, I don't see why they aren't
discussed at this stage.

---

From: Thomas Heide Clausen <[email-masked]>
Date: Tue, 9 Dec 2008 16:07:36 +0100

Hi JP,

Posting on a 'list, you run the risk that not just Baccelli respond.  
See below.

On Dec 9, 2008, at 15:53 PM, JP Vasseur wrote:

<SNIP>

> Please refer to the WG minutes + several emails on the ML, with a  
> good consensus. That being said, I hear you and thanks for your  
> comments. I do register your disagreement. Having read again  
> others' email, I do not think that this is a general feeling.
>

At this point, I think it fair to raise that in Minneapolis, ROLL and  
AUTOCONF were concurrently scheduled, and that for that reason  large  
part of the community (myself included) were unable to attend ROLL.

If scheduling would have permitted, I would have attended, and I  
would have made my concerns be known at the mike (and, hopefully,  
have had those reflected in the minutes as well).

Looking back over the mailing list postings, I also see a number of  
concerns raised by a number of people -- all of whom I know to have  
some experience designing/building/deploying protocols in a space  
similar or identical to ROLL.

So, I do not share the view expressed in your last paragraph above.

Thomas

---

From: JP Vasseur <[email-masked]>
Date: Tue, 9 Dec 2008 16:13:14 +0100

> made about a proprietary protocol.)

You missed my point ... I did not say that there were any of them  
satisfying these criterion.
But clearly there are non IP solutions being deployed, my point being  
that we need an
IP solution soon and we should make it right.

> If one or more such protocols exist, I don't see why they aren't

I do not think that it is in our charter to analyze *all* possible  
protocols.

---

From: Thomas Heide Clausen <[email-masked]>
Date: Tue, 9 Dec 2008 16:20:53 +0100

On Dec 9, 2008, at 16:13 PM, JP Vasseur wrote:

> But clearly there are non IP solutions being deployed, my point  
> being that we need an

I read Chris Dearlove as asking for something concrete, justifying  
that these five "criteria" that the survey I-D sets out are  
*attainable* (and without some major drawbacks on the other important  
metrics that the survey I-D is ignoring) and that we're not chasing a  
"pie in the sky".

> I do not think that it is in our charter to analyze *all* possible  

No, but if a good such protocol indeed existed, IP or not, having it  
before us might just help quell skepticism.
_______________________________________________
Roll mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/roll

---

From: "Dearlove, Christopher (UK)" <[email-masked]>
Date: Tue, 9 Dec 2008 15:20:48 -0000

>> in some other area.

> satisfying these criterion.

If you don't have a solution that meets your criteria, then imposing
your criteria (plus other essential criteria) might (I put it no
stronger than that) be requiring a "dream protocol".

---

From: JP Vasseur <[email-masked]>
Date: Tue, 9 Dec 2008 16:30:15 +0100

Hi,

> Posting on a 'list, you run the risk that not just Baccelli respond.

Which is fine, the IETF is open ;-)

In line,

> At this point, I think it fair to raise that in Minneapolis, ROLL  
> and AUTOCONF were concurrently scheduled, and that for that reason   
> large part of the community (myself included) were unable to attend  
> ROLL.
>

Well, the ID was discussed in Dublin, San Jose (interim WG meeting  
with dial-in) and in Minneapolis.

Extract of the minutes:

* San Francisco *
...

Subject: [Roll] Minutes of the Interim ROLL Working Group Meeting  
*and* WG consensus on the protocol survey conclusion

Dear WG,

Please find below the minutes of the ROLL Interim Working Group  
meeting that was help in San Jose on Oct 6.

This was an excellent very productive meeting where we managed to make  
good progress and reached a consensus on an important question related  
to a key Milestone.As usual such consensus must be confirmed on the  
mailing list.

When ROLL got chartered, one of our key milestones was to determined  
after some careful analysis of the routing requirements whether or not  
we could find an existing protocols that would meet our routing  
requirement with no change (with no protocol changes). All  
participants to the interim meeting agreed that there was no existing  
protocol that would meet the ROLL routing requirements *with no  
protocol work*. Do not hesitate to express your opinion on the mailing  
list before Oct 17 and then we will document the WG decision.

* Minneapolis *
...
3) Update of the Interim WG ROLL Meeting in San Jose (Chairs - 5mn) [15]

JP> Update on ROLL interim meeting held Oct. 2008.
The meeting was devoted to discussing protocol survey document and  
consensus reached subsequently on mailing list that no existing  
protocol would meet ROLL requirements
JP (to the room)> You can still comment if you disagree. Anybody  
disagreeing ?
No comment from the room.
...

*but* let's go back to a more constructive track ... what matters is  
to move forward and make sure that we move ahead.

> So, I do not share the view expressed in your last paragraph above.

Hopefully I clarified the WG's objectives (WG co-chair hat on).

Now you expressed concerns and we do have to address them.

May I ask to the authors of the ID to take one point at a time to  
answer them ?

Thanks.

JP.

---

From: Thomas Heide Clausen <[email-masked]>
Date: Tue, 9 Dec 2008 16:38:46 +0100

JP, All,

On Dec 9, 2008, at 16:30 PM, JP Vasseur wrote:

<SNIP>

> Hopefully I clarified the WG's objectives (WG co-chair hat on).

Yes indeed, thanks.

> May I ask to the authors of the ID to take one point at a time to  

Thanks, I appreciate that. I'll be happy to discuss/clarify my  
concerns further if/when the authors may deem it needed, and review  
updates as they occur.

Cheers,

Thomas

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Tue, 9 Dec 2008 17:45:39 +0100

Hi all:

Having attended the interim, I can concur that there was a clear consensus in the room and through the teleconf. The interim was not as crowded as the previous IETF meeting, yet there were more people there who raised their hands in favor than there are in a normal meeting considering all the non-voters in the room. To me that was a clear consensus, better than many I've seen. 

ROLL provides a niche where we can develop a simple and efficient mechanism that might not have all the power and goodies of the bigger brothers but would fit in a constrained device and do the minimum needed. The project might fail anyway if what we really want is a dream protocol, but it is certain to fail if we do not try. If enough people are willing to help, let's give ROLL a chance.

Pascal

>-----Original Message-----
>From: [email-masked] [mailto:[email-masked]] On Behalf Of Thomas Heide Clausen
>Sent: mardi 9 décembre 2008 16:08
>To: JP Vasseur (jvasseur)
>Cc: Emmanuel Baccelli; [email-masked]
>Subject: Re: [Roll] Working GroupLast Call:draft-ietf-roll-protocols-survey-02
>
>Hi JP,
>

---

From: "Childress, Steve" <[email-masked]>
Date: Tue, 9 Dec 2008 09:02:26 -0800

Have you folks previously considered SRI's TBRPF for mesh routing?
 
Steve Childress
[email-masked]

________________________________

From: [email-masked] [mailto:[email-masked]] On Behalf Of
Emmanuel Baccelli
Sent: Tuesday, December 09, 2008 1:02 AM
To: [email-masked]
Cc: [email-masked]; Thomas Heide Clausen;
[email-masked]
Subject: Re: [Roll] Working Group
LastCall:draft-ietf-roll-protocols-survey-02

Hi all,

I think this draft is far from being ready to become an RFC, and such
for several reasons, some of which are, unfortunately, structural.

1 - The draft is evaluating moving targets such as OLSRv2, Dymo, NHDP,
which are potential solutions that are not finalized yet. These
protocols could meet more requirements in the end (maybe all of them?).
For this reason, the "pass/fail" table of section 5 cannot honestly list
anything else but a series of "question marks" (or "pass"), for now, for
these protocols. This makes this table rather useless, and thus this
whole draft rather useless, presently.

3 - The draft is not constructive in its current shape. It is on the
other hand extremely controversial with its judgement, and does not
reflect a consensus within the community, not even a rough one.

Emmanuel

			Your point about hierarchical addressing is
interesting, and I think
			it relates to comments others have made on
fisheye routing and other
			state suppression approaches.

		Not quite. Hierarchical allocation is indeed a state
suppression
		approach. However fisheye is a control message
suppression approach.

	And, it should be added that such are, of course complementing
each other.
	
	Thomas

---

From: "Emmanuel Baccelli" <[email-masked]>
Date: Tue, 9 Dec 2008 18:16:33 +0100

Well I guess a lot of interested people could not attend the interim in
California, and a lot of people could not attend the last ROLL meeting in
Minneapolis either because Autoconf was scheduled simultaneously (which was
really unfortunate, I hope it does not happen again).Emmanuel

On Tue, Dec 9, 2008 at 5:45 PM, Pascal Thubert (pthubert) <
[email-masked]> wrote:

> Hi all:
>
> Having attended the interim, I can concur that there was a clear consensus
> in the room and through the teleconf. The interim was not as crowded as the
> previous IETF meeting, yet there were more people there who raised their
> hands in favor than there are in a normal meeting considering all the
> non-voters in the room. To me that was a clear consensus, better than many
> I've seen.
>
> ROLL provides a niche where we can develop a simple and efficient mechanism
> that might not have all the power and goodies of the bigger brothers but
> would fit in a constrained device and do the minimum needed. The project
> might fail anyway if what we really want is a dream protocol, but it is
> certain to fail if we do not try. If enough people are willing to help,
> let's give ROLL a chance.
>
> Pascal
>
> Thomas Heide Clausen

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Tue, 9 Dec 2008 18:32:06 +0100

Hi Thomas:

Every evaluation picks criteria and applies them. This doc has been around for awhile with its own clear list of criteria, which take their source in the requirements drafts. Someone could propose another doc with another set of criteria that would make perfect sense for a different set of requirements.

We could make a moving target of this doc and pursue the quest of the perfect evaluation of life, the universe and everything. But that's not the motivation nor the best interest of this group. In any case, do we expect a perfect one-fit-it-all protocol? I guess not. We'll have to trade off and prioritize, so while the evaluation draft has mostly served its purpose, we are far from being done with the requirement drafts themselves. 

For instance, in the case of industrial, there is an emphasys on reliability vs. path cost, which *is actually* achieved in the real (non-IP) world by building graphs that tend to enable multiple forwarding solutions at each hop, with the capability to revise the forwarding decision on a frame by frame basis after a few L2 retries. 

There are deployed IP protocols that enable similar non-equal-cost path computation, for instance in the DV world with EIGRP feasible successors. There is a huge amount of reusable art in traffic engineering. There is also interesting art in academia such as Tora that might fit the bill. What we need to do is build on this experience and start the work on real solutions to figure what optimum we can actually achieve for our requirements. 

Pascal

>Sent: mardi 9 décembre 2008 16:21
>Cc: Dearlove, Christopher (UK); Emmanuel Baccelli; [email-masked]
>Subject: Re: [Roll] WorkingGroup LastCall:draft-ietf-roll-protocols-survey-02
>
>

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Tue, 9 Dec 2008 18:43:55 +0100

Hi Emmanuel,

I do agree that was particularily infortunate, considering that in practice, autoconf can very well be a building block of the environment where ROLL is deployed to perform the routing. We need to alert the A-Ds to avoid such mishap in the future.

I do not necessarily reckon that this lot of people you are talking about would have necessarily voted against the consensus nor dramatically impacted the vote. I certainly trust that having to choose between the two sessions was a difficult decision for many, but we do need people willing to work in ROLL and for those people, ROLL can hardly be a second priority considering the milestones we need to achieve.

Pascal
________________________________________
From: [email-masked] [mailto:[email-masked]] On Behalf Of Emmanuel Baccelli
Sent: mardi 9 décembre 2008 18:17
To: [email-masked]
Subject: Re: [Roll] Working GroupLastCall:draft-ietf-roll-protocols-survey-02

Well I guess a lot of interested people could not attend the interim in California, and a lot of people could not attend the last ROLL meeting in Minneapolis either because Autoconf was scheduled simultaneously (which was really unfortunate, I hope it does not happen again).
Emmanuel
On Tue, Dec 9, 2008 at 5:45 PM, Pascal Thubert (pthubert) <[email-masked]> wrote:
Hi all:

Pascal

---

From: Philip Levis <[email-masked]>
Date: Tue, 9 Dec 2008 09:58:58 -0800

On Dec 8, 2008, at 8:00 AM, Thomas Heide Clausen wrote:

> couple of general concerns with this I-D and, more specifically,  
> with the approach it seems to be suggesting for the working group.

Well, the approach is the general consensus the working group reached  
after discussing the topic over the past year or so.

> the existing set of routing

I don't quite follow "zero cost" and "instant-convergence;" the cost  
metrics noted are clearly non-zero (and there are others not  
described), while convergence is not placed as a criterion.

> 		"loss response", "control cost", "link cost" and "node cost")  
> above the rest, while pointing out

I'm not sure I'd agree with your claim that the criteria set are not  
reasonable "for the real world." In particular, there are several  
members of the WG who have designed, implemented, and deployed  
protocols in products, scientific deployments, and other real-world  
uses of L2Ns. They seem to think the criteria are reasonable.

Tradeoffs are a critical part of any real system. Note, however, that  
the criteria specified should not be interpreted as MUST NOT ever do  
more than X; rather, MUST work with only X. As a trivial example,  
simply because a protocol's routing table needs to be able to scale  
with O(D) doesn't mean a protocol can't maintain a table larger than  
O(D). The former ensures that the design mechanisms are reasonable for  
the low-power devices L2Ns necessitate; the latter provides actual  
deployment and implementation flexibility. For these classes of  
networks, making the minimum resource requirements low is critical.

> 		zero-cost, zero-memory, instant-convergence.

I thought the document was pretty clear: there could be protocols  
which meet the criteria in the document yet are not sufficient. There  
are obviously other considerations. But as the question the document  
seeks to answer -- "Can ROLL use existing standards unchanged" -- is  
much narrower than "What should a protocol look like?" The purpose of  
this narrow scope is to allow the WG to answer the question without  
debating minutia on whether a particular criterion is more or less  
important. That is something for later WG discussion.

> 		IOW, looking at "control cost" as a separate metric is not  
> terribly useful unless the

I think you're reading this document in a way in which it was never  
intended, pretty explicitly so. This draft does not say "if a protocol  
meets these requirements, then it is OK." Otherwise, we end up in an  
adversarial situation, like the one you outline above, where we're  
trying to find holes in the criteria. Adding criteria such as  
"delivers data" would be a bit silly, adding complexity to the  
document when we all know that's of course important.

> in setting out a rigid set of "pass" criteria and observing that  
> none of the six protocols analyzed satisfies those off-the-shel --  
> but that this should not be stretched to  assume that an entirely  
> new design is required.

Please read through the mail archives. At no point has the document  
ever claimed so, and I've tried to be very very explicit in this. At  
the interim meeting, I was asked the question of "so what should we do  
next," and I demurred. The introduction is VERY explicit:

"Whether or not such a protocol involves modifications to an existing  
protocol or a new protocol entirely is outside the scope of this  
document: this document simply seeks to answer the question: do LLNs  
require a new protocol specification document at all?"

This concern with the document comes up again and again, and I cite  
the same points. For some reason readers come away with an impression  
of the document's claims which are completely contrary to its text. Do  
you have any suggestions on how we could make the document clearer?

> vary flooding is a policy matter that is dependent on deployment,  
> but can be done entirely conformant with its specification] -- and  
> that this is not considered. I'll go further than Chris and suggest  
> that not considering the use of such options, which are present in  
> the protocols but where a policy specification dependent on  
> deployment is required, casts doubt on the assessment of the  
> protocols as presented - not just OLSRv2, but such options exist for  
> other protocols as well.

Unfortunately, neither of these are specified in RFCs or mature  
drafts. Mario Gerla is wonderful and has great ideas, but it's tough  
to argue that analytical papers from almost a decade ago definitively  
establish the efficacy of a protocol mechanism well enough to  
determine IETF protocol specification.

Phil

---

From: Philip Levis <[email-masked]>
Date: Tue, 9 Dec 2008 11:47:42 -0800

On Dec 9, 2008, at 7:30 AM, JP Vasseur wrote:
> This was an excellent very productive meeting where we managed to  
> make good progress and reached a consensus on an important question  
> related to a key Milestone.As usual such consensus must be confirmed  
> on the mailing list.
>
> after some careful analysis of the routing requirements whether or  
> not we could find an existing protocols that would meet our routing  
> participants to the interim meeting agreed that there was no  
> existing protocol that would meet the ROLL routing requirements  
> *with no protocol work*. Do not hesitate to express your opinion on  
> the mailing list before Oct 17 and then we will document the WG  
> decision.

I think this is the key point.

The #1 question is whether the concerns on structure, criteria, etc.  
will change the conclusion of the document. If someone has strong  
evidence that existing protocols, as specified, can meet the  
requirements of ROLL, then that information would be of fantastic use  
and value to ROLL. But we seem to have consensus on this point:  
existing IETF specifications do not meet the requirements L2Ns  
introduce.

More specifically, if there's a disagreement with the conclusion of  
the document, then this means one of two things:

1) The analysis of an existing protocol is incorrect
2) A criterion is too harsh and should be weaker

The domain experts who have written the application drafts are the  
best commentators on 2); in fact, one of the criteria (control cost)  
did relax slightly based on feedback at the interim meeting.

So this leaves 1). The authors of TBRPF made some very helpful  
comments on the TBRPF analysis early in the document's lifetime, and  
we amended it appropriately. If there are other entries that are  
incorrect, then a statement of the correct analysis would be the best  
way to make the document more accurate.

Everything else besides this is really ancillary; we can debate at  
length whether there should be 4, 5, or 8 criteria, but if the  
conclusion is the same, then all this debate does is stifle ROLL from  
doing useful work.

Of course the document can be improved and comments are welcome. In  
the last revision, a few readers pointed out many things which were  
unclear, and which Stephen, Arsalan, and I tried to clarify. For  
example, it's critical that the document be very transparent on the  
methodology it uses and how it reaches its conclusions.

Phil

---

From: Kris Pister <[email-masked]>
Date: Tue, 09 Dec 2008 13:19:28 -0800

Christopher -
 the fact that existing protocols are proprietary makes it difficult to 
deliver on your request.
Proprietary protocols are generally not publicly documented, hence the 
desire to have an
IETF standard.
 I can point you to IEC PAS (Publicly Available Specification) 62591, 
which describes the
Wireless HART protocol, but 1) despite the name it's still a few months 
away from actually
being "public", and 2) it describes all of the mechanisms for gathering 
and disseminating
routing information but does not actually specify how the routing engine 
makes it's decisions.
That information is at this point truly proprietary.

The proof that it works has been in the field for almost two years now, 
and there are many
companies building and shipping it.  I believe that at least one 
implementation satisfies all of the
RoLL IDs tests, not just the five in the protocol survey document.
This protocol is not discussed in the survey document because 1) it's 
not an IETF protocol,
2) reasons 2 through N are irrelevant, reason #1 is enough.

Protocols exist that do what we want, they just haven't made it into 
RFCs yet.

ksjp

---

From: "Stephen Dawson-Haggerty" <[email-masked]>
Date: Tue, 9 Dec 2008 14:59:27 -0800

Thanks for all the constructive comments.  I think the general objections
are either that the document does too little, but well, or it does too much,
but poorly.  We have tried and will continue to try to address any technical
objections to the document; however as the chairs have pointed out, the
document is by nature insufficient to make a great number of statements
about this sort of networking.  It does seem that there is a rough consensus
(irrespective of this document) that no existing IP-based networking
specification is actually going to work in these networks, and so more work
is needed.

On Mon, Dec 8, 2008 at 8:00 AM, Thomas Heide Clausen <[email-masked]

> Dear All,
>
> I've been reviewing the latest incarnation of this I-D, and I have a couple
> of general concerns with this I-D and, more specifically, with the approach
> it seems to be suggesting for the working group.
>
>        o       The document seems to set the bar of acceptance at  "the
> holy grail of a routing protocol",
>                i.e. defines "acceptable" to be "zero-cost [more on that in
> a following bullet],
>                instant-convergence" -- thereby causing all routing
> protocols to almost by definition be
>                judged "insufficient" -- which is reflected in the
> evaluation of the existing set of routing
>                protocols.

Agreeing with Phil's comments, I'm also not sure I've heard other people
voice concerns that the criteria of this draft are impossible; this may have
actually been the case before the infirm meeting, but I don't think is the
case now.  I don't see any reason we can't build or modify (I'm not taking a
position on which one) a protocol to meet these goals, since several
existing protocols seem to come close, and techniques to meet each goal seem
to be relatively well understood.

>
>
>        o       As I assume ROLL is designing "for the real world",
> compromises and trade-offs are
>                paramount in protocol design. By elevating a set of criteria
> (sec. 3 and 4 -- "table scalability",
>                "loss response", "control cost", "link cost" and "node
> cost") above the rest, while pointing out
>                introduce additional, more stringent, requirements", this
> I-D effectively limits the trade-offs
>        o       From the previous point, it is not convincing that
> eliminating these trade-offs is justified.
>                While I believe that indeed *any* application would welcome
> "the holy grail of a routing
>                protocol", recognizing that such may not exist, it may for
> some application be right to
>        o       Going by where the bar for acceptance is set in this
> document, I have thought and come
>                up with indeed two solutions that have
> "zero-cost-instant-convergence" properties, and
>                they both satisfy the IP delivery semantics. One is "drop
> all traffic, don't even try" and the
>                Of course, those two proposals are in jest, as I am sure
> that the trade-off I am making
>                -- sacrificing delivery ratio for the former, satisfying
> network traffic and energy (but not
>                as a consequence of the routing protocol relaying control
> traffic) for the latter are not
>                acceptable in an absolute sense -- but the routing protocol
> will itself require have
>                "data  transmission cost" likely is a better tradeoff,
> especially if data traffic is a common
>                consequences in terms of the "data cost" and "data delivery
> ratio" are considered
>                as well.
>
> Chris Dearlove summarize well in saying that the document does well in
> setting out a rigid set of "pass" criteria and observing that none of the
> six protocols analyzed satisfies those off-the-shel -- but that this should
> not be stretched to  assume that an entirely new design is required.

No, it should not be stretched to say this.

>
>
> I also join Chris in his comment that, for example, OLSRv2 offers "timing
> backoff" and "fisheye variable flooding" without variations to the routing
> protocol [the specificities of how to set timers or vary flooding is a
> policy matter that is dependent on deployment, but can be done entirely
> conformant with its specification] -- and that this is not considered. I'll
> go further than Chris and suggest that not considering the use of such
> options, which are present in the protocols but where a policy specification
> dependent on deployment is required, casts doubt on the assessment of the
> protocols as presented - not just OLSRv2, but such options exist for other
> protocols as well.
>
> I'll send another email with "nits" to the document, however I feel that
> the issues above need to be discussed and resolved.
>
> Cheers,
>
> Thomas
>

---

From: "Arsalan Tavakoli" <[email-masked]>
Date: Tue, 9 Dec 2008 15:22:57 -0800

Just saw David's email after I'd written this, so mine definitely has some
overlap with that, but thought I'd still send it out.

---------------------------------------

Sorry to get into the fray so late.  I feel that some of the conversations
that we are having at this point seem to be like deja-vu, as we had debated
them (and I thought reached a consensus) previously.  I join Phil in
welcoming suggestions as to change in text in the document needed to more
explicitly state our purpose.
>From an evaluation criteria standpoint, as is the case with an survey
document, there are additional metrics that could have been considered.  But
we were very precise about this: this document lays out the requirements
that are necessary, but may not be sufficient for any protocol in the space.
 The derivation of these metrics came from a set of requirements documents,
written by authors with design/implementation/deployment experience in each
of the application domains.  While these application domains don't portend
to encompass all possible L2N deployments, they do compose a significant
majority, and hence serve as a valid derivation source for our metrics.

A lot of the comments have harped on the 'control cost' criteria, saying
that there is an inherent trade-off between delivery ratio/path latency and
the amount of control traffic.  This would be a valid complaint if we had
rejected all current protocols as untenable, and introduced a new protocol
from scratch that meets the control cost criteria and say this is the
protocol for ROLL.  However, first of all, the document specifically states
that the set of criteria is necessary, not sufficient.  Furthermore, when
the next step looks at selecting a protocol (either through extension/tweak
or from scratch), closer attention will be paid to other criteria,
potentially on a per-application-domain basis, at which point a given
existing protocol's trade-off decision can be revisited, tweaked, and
submitted as an applicable protocol that meets the criteria, if possible.
 That is NOT, and I repeat not, the purpose of this document.

In terms of the set of protocols chosen, this list was set out, pruned, and
agreed upon by members of the working group.  The specific requirements for
any protocol to be considered (as laid out by the charter), is that it has
to be an IETF protocol with an accompanying RFC (or in one or two cases, a
very mature internet draft).  We realize that there are actual
implementations that have been tweaked, extensions documented as conference
publications, and the like; however, as the ROLL charter points out, the
protocol that is adopted must be in the form of an RFC specification.  Now,
one of the comments made was that there might be some other protocols.  Even
given that we achieved consensus as a WG on this list or protocols, if there
is a protocol that meets all the criteria (and is specified in an RFC), then
obviously we would be glad to consider it and make this process simpler.
 However, if not, then that does not affect this I-D at all, and rather is
more pertinent to the subsequent step.

I feel that a great deal of debate has revolved around a misunderstanding of
what this document is intended to do.  This document simply seeks to
identify a set of quantifiable evaluation criteria, based on criteria
specifically laid out by application-domain experts, as necessary for any
ROLL routing protocol.  A set of existing IETF IP-based protocols must then
be evaluated against these criteria, using only strict interpretations of
RFC specifications (as required by the ROLL charter), to determine if any
existing protocol satisfies these requirements as-is.  That is it.  Any
other implications are being drawn mistakenly and are not part of this
document (only showing that the document is in fact serving its purpose of
providing a foundation to debate next steps).  The consensus reached by this
WG on the mailing list and in meetings, is that none of the protocols meet
all the requirements, as-is.  What this simply means, is that a new RFC
specification will need to be written for a ROLL protocol, and hence we need
to recharter.  However, we absolutely take NO position in the document in
terms of whether an existing protocol can be modified/extended or whether a
completely new protocol from scratch is needed.  That is a debate that
determines the future direction of the WG, but is not pertinent to this
document.  When a protocol 'fails', this does not imply that it is
untenable; rather, it implies that there are shortcomings in protocol (as
currently specified) in regards to the requirements of ROLL (which makes
sense, since these protocols were designed for different metrics in
different environments); addressing these shortcomings to allow the protocol
to meet the ROLL criteria are a completely viable, but future, option, that
should not hinder the progress of this ID.

Arsalan

On Tue, Dec 9, 2008 at 9:43 AM, Pascal Thubert (pthubert) <
[email-masked]> wrote:

> Hi Emmanuel,
>
> I do agree that was particularily infortunate, considering that in
> practice, autoconf can very well be a building block of the environment
> where ROLL is deployed to perform the routing. We need to alert the A-Ds to
> avoid such mishap in the future.
>
> I do not necessarily reckon that this lot of people you are talking about
> would have necessarily voted against the consensus nor dramatically impacted
> the vote. I certainly trust that having to choose between the two sessions
> was a difficult decision for many, but we do need people willing to work in
> ROLL and for those people, ROLL can hardly be a second priority considering
> the milestones we need to achieve.
>
> Pascal
> Subject: Re: [Roll] Working
> GroupLastCall:draft-ietf-roll-protocols-survey-02
>

---

From: "M. B. Anand" <[email-masked]>
Date: Tue, 09 Dec 2008 18:44:09 -0500

> approach taken by this document. And there is no rough consensus either.
I think rather that there was actually clear consensus on the mailing 
list. I think the document serves adequately its narrow purpose of 
evaluating the suitability of existing IETF protocols for their 
suitability for intersection of ROLL requirements. I think there may be 
extrapolations/impressions that are formed based on this document that 
lead us into discussions of dream protocols and such but the intent of 
the document is not to dictate what the next steps should be. I think on 
this basis there was consensus and I will re-iterate here my support for 
this document.

- Anand.

---

From: Jonathan Hui <[email-masked]>
Date: Tue, 9 Dec 2008 15:44:23 -0800

I'd also add that IP-based solutions that meet the criteria in the  
survey doc have also been shipping commercially for over two years.  
You can find a high-level description of one such solution as well as  
empirical results from a real application deployment in [1]. Open- 
source solutions exist as well [2]. They are not standardized nor are  
they specified in RFCs, and for that reason, they are not discussed in  
the survey doc.

[1] http://www.cs.berkeley.edu/~jwhui/pubs/jhui-sensys08-ipv6.pdf
[2] http://smote.cs.berkeley.edu:8000/tracenv/wiki/b6loWPAN

--
Jonathan Hui

On Dec 9, 2008, at 1:19 PM, Kris Pister wrote:

> Christopher -
>  the fact that existing protocols are proprietary makes it difficult  
> to deliver on your request.
> Proprietary protocols are generally not publicly documented, hence  
> the desire to have an
> IETF standard.
>  I can point you to IEC PAS (Publicly Available Specification)  
> 62591, which describes the
> Wireless HART protocol, but 1) despite the name it's still a few  
> months away from actually
> being "public", and 2) it describes all of the mechanisms for  
> gathering and disseminating
> routing information but does not actually specify how the routing  
> engine makes it's decisions.
> The proof that it works has been in the field for almost two years  
> now, and there are many
> This protocol is not discussed in the survey document because 1)  
> it's not an IETF protocol,

---

From: Tim Winter <[email-masked]>
Date: Wed, 10 Dec 2008 11:43:37 -0500

I think Phil has phrased it very concisely and I second his statements. 
Existing (as is) IETF protocols are the extent of the scope of the
survey.  Whether or not (unspecified) "tweaks and tunings and potential
extensions" can change the conclusion of with regard to an existing
protocol is beyond the scope and intent of the survey.

I am not aware of a fault in the analysis of existing protocols as such,
and I agree with the criteria that have been established.  Given this,
and the intended scope, I reiterate my support of
draft-ietf-roll-protocols-survey-02.

If you have some ideas on how to tweak/tune/extend to existing
protocols, or know of non-IETF protocols with desirable traits, please
keep around...  Such input may be needed in the future (to see if
recommendations can be made to other WG such that existing protocols can
be adapted, or if new protocol needs to be designed/incorporated into IETF).

I can attest to the fact that the industry needs a standard routing
solution.  Proprietary LLNs are being deployed *now*.  I would like to
see a viable, open, and standard IETF protocol emerge that is capable of
meeting industry needs in a timely manner.

-Tim

Philip Levis wrote:
>> protocol work*. Do not hesitate to express your opinion on the
>> mailing list before Oct 17 and then we will document the WG decision.
>
> existing IETF specifications do not meet the requirements L2Ns introduce.
>

---

From: "Omprakash Gnawali" <[email-masked]>
Date: Wed, 10 Dec 2008 10:49:22 -0800

I wish the analysis of the routing table size had gone beyond the
big-O analysis considering we already know the rough number of
destinations, depth of the network, and the total number of nodes. For
pass/fail analysis, something like "if a protocol requires tens of
thousands of entries, fail". Granted absolute bounds on many metrics
depend on density and what the network is trying to optimize, at least
this particular metric seems amenable to a more concrete analysis. I
don't think it is a requirement that we use big-O for all the metrics.

- om_p

---

From: Thomas Heide Clausen <[email-masked]>
Date: Tue, 16 Dec 2008 07:55:57 +0100

Hi Pascal,

Long time no see -- hope all is well in the south?

On Dec 9, 2008, at 6:32 PM, Pascal Thubert (pthubert) wrote:

> Hi Thomas:
>
> Every evaluation picks criteria and applies them. This doc has been  
> around for awhile with its own clear list of criteria, which take  
> their source in the requirements drafts. Someone could propose  
> another doc with another set of criteria that would make perfect  
> sense for a different set of requirements.
>

Right, I appreciate that any evaluation is the result of a choice of  
criteria -- and that's fine as such. However the choice has to be  
sensible such that the conclusions that can be drawn are useful.

In this case, I feel that a somewhat common, but none the less  
unfortunate, choice is made. I'll again emphasize the "control cost"  
metric. As it would be, we should not really care for the specific  
"control cost" but rather "the total cost of getting traffic from the  
source to the destination. This includes the control cost that  
incurs, of course, but not only: the cost of carrying data traffic  
should be considered as well. If the data traffic to be delivered is  
non-zero, the latter often is important -- which is one good reason  
for a routing protocol to trade of "control costs" for attaining  
"shorter/shortest paths" (according to whatever metric).

Hence, in an absolute sense, the metric "control cost" when taken  
alone does not state anything regarding a specific protocols'  
applicability to any domain: it does not indicate anything regarding  
the network load attained, nor on energy conservation (recognizing  
that transmission/receipt of *any* packet loads the network and  
requires energy from the relays).

I return on my caricature and propose a zero control-cost, zero state  
protocol, instant-convergence which goes "whenever you have data to  
deliver, flood it, and don't do any duplicate elimination". It'd  
score well according to the metrics -- but I assume that we agree  
that it'd be a particularly unsuitable protocol for general data  
delivery?

> We could make a moving target of this doc and pursue the quest of  
> the perfect evaluation of life, the universe and everything. But  
> that's not the motivation nor the best interest of this group.

I'm not encouraging that. I'm encouraging that a couple of missing  
pieces be added to the document, and that without those pieces the  
document isn't particularly helpful for the group.

> In any case, do we expect a perfect one-fit-it-all protocol? I  
> guess not.

I do not pretend to know that. And, my problem is that this I-D does  
not help clarifying that at all.

> We'll have to trade off and prioritize, so while the evaluation  
> draft has mostly served its purpose, we are far from being done  
> with the requirement drafts themselves.

It seems to me that this is not in line with what the tracker  
reveals. draft-ietf-roll-urban-routing-reqs is in IESG evaluation and  
draft-ietf-roll-home-routing-reqs in last-call, which indicates that  
the WG considers those to be done?

Thomas

> For instance, in the case of industrial, there is an emphasys on  
> reliability vs. path cost, which *is actually* achieved in the real  
> (non-IP) world by building graphs that tend to enable multiple  
> forwarding solutions at each hop, with the capability to revise the  
> forwarding decision on a frame by frame basis after a few L2 retries.
>
> There are deployed IP protocols that enable similar non-equal-cost  
> path computation, for instance in the DV world with EIGRP feasible  
> successors. There is a huge amount of reusable art in traffic  
> engineering. There is also interesting art in academia such as Tora  
> that might fit the bill. What we need to do is build on this  
> experience and start the work on real solutions to figure what  
> optimum we can actually achieve for our requirements.
>
> Pascal
>

---

From: Thomas Heide Clausen <[email-masked]>
Date: Tue, 16 Dec 2008 08:24:19 +0100

Dear Philip,

On Dec 9, 2008, at 6:58 PM, Philip Levis wrote:

>> I've been reviewing the latest incarnation of this I-D, and I have  
>> a couple of general concerns with this I-D and, more specifically,  
> Well, the approach is the general consensus the working group  
> reached after discussing the topic over the past year or so.

It may be so, and it may be entirely fair to decide to survey  
protocols based on a set of metrics, and to establish a set of  
metrics/criteria against which any ROLL protocol should be measured.

However, in that case, I still think that the approach is wrong. It  
would be a LOT cleaner to stipulate such metrics in an isolated I-D,  
rather than embed those in a "protocol survey".

That way, these evaluation criteria could be used to, objectively,  
evaluate both existing routing proposals, as well as whatever "new"  
proposals that were made on a level playing field. It is not clear as- 
is if the standard to which the protocols in this I-D is held is ALSO  
going to be that to which any "new" proposal is going to be held; the  
I-D does not answer this, the charter does not answer this, the other  
WG I-Ds do not answer this -- and I see postings talking about "trade- 
offs are necessary" on the list, but I do not know if that also  
implies trading-off on the criteria in the survey I-D.

If the answer is that the standard / set of criteria is to be the  
same, then I'd encourage that those be exposed and explored in  
isolation, and then applied to the different protocols later. If the  
answer is "no, there may be trade-offs also on those criteria, when  
we come to building protocols", then I am convinced that the approach  
is inappropriate.

Regardless of what approach is taken of the above, I do disagree with  
the specifics of the choice of metrics:

	*	the set of metrics chosen, as I believe those to be incomplete and  
to lead
		to the wrong conclusions being drawn;

	*	the fact that the I-D considers the selected metrics to be with  
bounds that are
		"set in stone", and not subject to trade-offs -- especially when  
the trade
		offs *may* reflect substantial gains on other metrics not considered

See  also my reply to Pascal for an example hereof.

> I don't quite follow "zero cost" and "instant-convergence;" the  
> cost metrics noted are clearly non-zero (and there are others not  
> I'm not sure I'd agree with your claim that the criteria set are  
> not reasonable "for the real world." In particular, there are  
> several members of the WG who have designed, implemented, and  
> deployed protocols in products, scientific deployments, and other  
> real-world uses of L2Ns. They seem to think the criteria are  
> reasonable.

See email to Pascal for example of why the metric "control cost" is  
not, in isolation, a useful metric unless one makes additional  
assumptions. Note, this is a single example, similar arguments could  
be made re. state, ....

> Tradeoffs are a critical part of any real system. Note, however,  
> that the criteria specified should not be interpreted as MUST NOT  
> ever do more than X; rather, MUST work with only X. As a trivial  
> example, simply because a protocol's routing table needs to be able  
> to scale with O(D) doesn't mean a protocol can't maintain a table  
> larger than O(D). The former ensures that the design mechanisms are  
> reasonable for the low-power devices L2Ns necessitate; the latter  
> provides actual deployment and implementation flexibility. For  
> these classes of networks, making the minimum resource requirements  
> low is critical.

>
>>
>> 	o	Going by where the bar for acceptance is set in this document,  
>> I have thought and come
>> 		Of course, those two proposals are in jest, as I am sure that  
>> the trade-off I am making
> which meet the criteria in the document yet are not sufficient.  
> There are obviously other considerations. But as the question the  
> document seeks to answer -- "Can ROLL use existing standards  
> unchanged" -- is much narrower than "What should a protocol look  
> like?" The purpose of this narrow scope is to allow the WG to  
> answer the question without debating minutia on whether a  
> particular criterion is more or less important. That is something  
> for later WG discussion.

I think that this is not the right conclusion to draw here. If you  
want to determine if an existing routing solution can be used  
unchanged, then you MUST necessarily know what you expect that  
routing solution to look like, at least in an abstract sense, and  
certainly to determine whether a particular criterion is more or less  
important.

I.e. to determine if an existing solution is applicable, you MUST  
determine which trade-offs you're ready to make. Otherwise, it could  
appear as if the goal was to simply go through the motions of  
dismissing "not-invented-here" proposals.

> intended, pretty explicitly so. This draft does not say "if a  
> protocol meets these requirements, then it is OK." Otherwise, we  
> end up in an adversarial situation, like the one you outline above,  
> where we're trying to find holes in the criteria. Adding criteria  
> such as "delivers data" would be a bit silly, adding complexity to  
> the document when we all know that's of course important.

Again, no. Looking at *a* criteria such as "control cost" in  
isolation has the inherent problem of not being useful. As I state in  
my email to Pascal, we do not CARE about "control cost", we care  
about "minimizing the total load on the network". Be as it may, one  
could trade off "a bit of control cost for a lot of path-shortening",  
which would be highly beneficial when you have actual data over the  
network.

>> Chris Dearlove summarize well in saying that the document does  
>> well in setting out a rigid set of "pass" criteria and observing  
>> that none of the six protocols analyzed satisfies those off-the- 
>> shel -- but that this should not be stretched to  assume that an  
>> entirely new design is required.
>
> ever claimed so, and I've tried to be very very explicit in this.  
> At the interim meeting, I was asked the question of "so what should  
> we do next," and I demurred. The introduction is VERY explicit:
>
> "Whether or not such a protocol involves modifications to an  
> existing protocol or a new protocol entirely is outside the scope  
> of this document: this document simply seeks to answer the  
> question: do LLNs require a new protocol specification document at  
> all?"
>
> the same points. For some reason readers come away with an  
> impression of the document's claims which are completely contrary  
> to its text. Do you have any suggestions on how we could make the  
> document clearer?

I am not sure that adding an explanatory text to the introduction  
would resolve the issue I have with this document.

>> "timing backoff" and "fisheye variable flooding" without  
>> variations to the routing protocol [the specificities of how to  
>> set timers or vary flooding is a policy matter that is dependent  
>> on deployment, but can be done entirely conformant with its  
>> specification] -- and that this is not considered. I'll go further  
>> than Chris and suggest that not considering the use of such  
>> options, which are present in the protocols but where a policy  
>> specification dependent on deployment is required, casts doubt on  
>> the assessment of the protocols as presented - not just OLSRv2,  
>> but such options exist for other protocols as well.
>
> drafts. Mario Gerla is wonderful and has great ideas, but it's  
> tough to argue that analytical papers from almost a decade ago  
> definitively establish the efficacy of a protocol mechanism well  
> enough to determine IETF protocol specification.

I find it curious that an analytical study by Gerla is being  
dismissed for a document emphasizing O(...) evaluations.

I also note that there're deployments of e.g. fisheye mechanisms "out  
there", I am aware of some in the context of OLSR(v2) -- and that the  
OLSRv2 specification explicitly contains what's needed for that. The  
parametrization for a given deployment is, of course, deployment- 
dependent.

Thomas

---

From: Thomas Heide Clausen <[email-masked]>
Date: Tue, 16 Dec 2008 08:28:02 +0100

On Dec 9, 2008, at 8:47 PM, Philip Levis wrote:

>> make good progress and reached a consensus on an important  
>> question related to a key Milestone.As usual such consensus must  
>> be confirmed on the mailing list.
>>
>> When ROLL got chartered, one of our key milestones was to  
>> determined after some careful analysis of the routing requirements  
>> whether or not we could find an existing protocols that would meet  
>> our routing requirement with no change (with no protocol changes).  
>> All participants to the interim meeting agreed that there was no  
>> *with no protocol work*. Do not hesitate to express your opinion  
>> on the mailing list before Oct 17 and then we will document the WG  
>> decision.
>
> The #1 question is whether the concerns on structure, criteria,  
> etc. will change the conclusion of the document. If someone has  
> strong evidence that existing protocols, as specified, can meet the  
> requirements of ROLL, then that information would be of fantastic  
> use and value to ROLL. But we seem to have consensus on this point:  

Be VERY careful, though: as I read the document, it sets a bar VERY  
high on some areas and VERY low in others. And, like it or not, this  
document does read "as if" it was a "ROLL routing solutions criteria  
specification". And, as such, it is incomplete.

> 2) A criterion is too harsh and should be weaker

No, wrong conclusion, there's a third option:

   3) The set of criteria is incomplete, and additional criteria  
should be added
        in order to complete the evaluation and guide the wg forward.

Thomas

> best commentators on 2); in fact, one of the criteria (control  
> cost) did relax slightly based on feedback at the interim meeting.
>
> comments on the TBRPF analysis early in the document's lifetime,  
> and we amended it appropriately. If there are other entries that  
> are incorrect, then a statement of the correct analysis would be  
> the best way to make the document more accurate.
>
> conclusion is the same, then all this debate does is stifle ROLL  
> from doing useful work.
>

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Wed, 17 Dec 2008 12:26:46 +0100

Hi Thomas:

What I mean is that the requirement drafts will be the source of the work to come, as opposed to the evaluation draft that merely indicates that there is work to be done. So we need to focus on fulfilling the requirements in a way that makes sense, and your comments below shed a clear light on what making sense means.

Sometime soon, I hope we'll see a number of proposed approaches and we'll need to evaluate them so as to select (at least) one and refine it to standard quality. Those new approaches should be a lot more specific to the ROLL problem and probably a lot more tricky to classify wrt the requirements. Thus more refined criteria will most certainly be needed and this discussion is certainly a good preparation.

Clearly the total cost of getting packets though is a great metric. It has to be balanced with expected latency, stealth/deep sleep capabilities and so on. Seems to me though that we should focus on where the proposals actually make a difference between one another and figure which exhibits the most crucial benefits for our requirements; but we can only do that once we have proposals on the table.

Pascal

>From: Thomas Heide Clausen [mailto:[email-masked]]
>Sent: mardi 16 décembre 2008 07:56
>To: Pascal Thubert (pthubert)
>Cc: JP Vasseur (jvasseur); Dearlove, Christopher (UK); Emmanuel Baccelli; [email-masked]

---

From: JP Vasseur <[email-masked]>
Date: Wed, 17 Dec 2008 18:04:52 +0100

Hi,

First of all, thanks for all the comments; even late comments are very  
much welcome to improve the quality of the document. I'm sure that  
there is only a communication issue, and we all want to have a  
constructive approach so as to quickly reach a consensus.

I would just want to stress a few facts before making a proposal:

1) We have limited time: non-IP and proprietary protocols/ 
architectures/solutions are being deployed today and the industry is  
asking for an IP-based routing protocol for LLN: we do need to do the  
right thing but we can't afford to argue for ever on issues that will  
not change the overall conclusion,
2) All the requirements documents have been written by authors with a  
real-world experience,
4) Designing a protocol that will fulfill those requirements is  
feasible (since some of them do exist today). See the "requirements"  
section below.

So far I think that we are on the same page.

Our charter

First, spell out a set of requirements for a few limited applications  
(almost complete).
Second, answer to this explicitly narrow question: "Can we use an  
*existing* IP protocols for LLNs ?". The protocol survey aims at  
answering this question. Not whether it should be a new protocol, an  
extension to an existing protocol, ...
Other WG items are not relevant to this discussion.

Requirements

As you know, we are not currently chartered to work on protocols. We  
first had to express our requirements and see whether an existing  
protocol would meet these requirements. One cannot ask us to  
demonstrate that we can design such a protocol without being chartered  
to do so. That being said, considering the number of existing and  
deployed protocols in the field designed by some of the people who  
wrote the requirements, we can have a good level of confidence that  
such objective is achievable. For the record, I've been working myself  
on such protocols for some time.

Just two important comments with regards to "requirements".

1- Requirements are "guidelines". When time will come to work on  
protocol (if re-chartered), anybody will be more than welcome to  
propose a protocol (protocol extensions or new protocol). We will have  
to make compromise indeed. I would like to remind that we do not have  
to support all the MUST in a single instantiation of the protocol.  
There are ways to design protocols with options (there are many  
protocols that have this property).
2- I would also like to remind that we live in a highly constrained  
environment with small footprint (few hundreds of bytes in RAM for  
routing is not unusual). We could not afford to use an existing  
adapted protocol that would meet our requirements if it requires lots  
of extra memory for features that are not needed in LLNs. Any  
additional memory has a cost, which matters a lot to LLNs.

Proposal

Now, let's try to be constructive and address your concerns.

1) Lack of background, context, ... in the document. Emmanuel is quite  
right that background is missing in the document as well as a clear  
conclusion: "how have we chosen the protocols in the survey, why  
protocols X, Y, Z, ... have not been included in the survey, ...". I  
saw that Phil agreed to add some text to address this issue.
Emmanuel, does that address your concern (to be confirmed once you see  
the text) ?
Phil, could you please propose some text ?

2) Missing protocols: we had to draw the line somewhere. *LOTS* of  
protocols have been proposed over the last decade. We had many  
suggestions to also look at protocol X, Y and Z. The decision was  
(according to our charter) to limit our survey to existing IETF  
protocols (RFC or very mature ID):  OSPF, OLSR, TBRPF, RIP, AODV,  
DYMO, DSR. The list is already fairly long. The charter mentions DTN.  
The WG decided not to include DTN and this needs to be documented.
Phil, could you please add some text ? Note that this does not mean  
that we may not borrow mechanism(s) from existing protocols by the way  
if we get re-chartered.

3) Is the goal to exclude existing routing protocols as potential  
candidates ? The answer is "no" but Thomas is right, this is currently  
not well explained in the document. The objective is to agree that  
none of the existing protocols meet the requirement but the solution  
might either be to develop a new one or extend an existing routing  
protocol (once we get re-chartered).
Thomas, would that address your concern ?

4) Evaluation metric should be in a separate document: the approached  
adopted by the WG was to derive metrics from the requirements  
documents so as to perform the protocol survey. Once we will start  
protocol design, we will go back to the requirement documents since  
this is *the* objective of Requirements IDs.

5) Missing Metrics and Incorrect Metrics: Let us know if you have a  
list of metrics that are in your opinion "missing" and which metrics  
are "incorrect/inappropriate"? Or are we now in agreement based on 4)  
above ?

Let's try to quickly reach a consensus and continue the excellent  
progress of the WG so far.

Many Thanks for your help.

JP.

---

From: "Dearlove, Christopher (UK)" <[email-masked]>
Date: Wed, 17 Dec 2008 18:09:11 -0000

I'm not listed as an action below, as my main point was (and remains)
the
danger that this document will be read as saying more than any, more
limited,
consensus agreement, rather than that the document is bad, although I
share
some concerns over whether the requirements used are "right". But I'm
not
actually opposed to the narrow conclusion that no existing protocol
surveyed
is ideal as is. It's important to make it clear what is, and is not,
being said.
 
Given all the real constraints on time etc., I think that the below is
on the right
track to improve the document. That leaves point 5 in the final list,
which ties
in with point 2 in the requirements list. I think the key issues are
overall cost vs.
"signalling" cost (it's the former that really matters) and latency.
(Crudely, I can see
at least three latency regimes - now, waiting for a route discovery
process, a long
term process such as various opportunistic store and forward approaches.
Note that
longer delays that may be necessitated by cutting down signalling
storage have
a potentially increased data queuing cost, back to the first point.) And
finally (for
this email) regarding various real solutions that put the onus on
gateways to do critical
work, have more storage and be more immediate and so on. The possible
congestion
and vulnerability issues of such key points may also be important.
 
Note that I'm not saying these points need to be addressed in detail
here. It's more
about putting down markers for the next stage, when these will matter.
 
(I'm not around much longer before the first full week in January
incidentally.)

-- 
Christopher Dearlove
Technology Leader, Communications Group
Networks, Security and Information Systems Department
BAE Systems Advanced Technology Centre
West Hanningfield Road, Great Baddow, Chelmsford, CM2 8HN, UK
Tel: +44 1245 242194  Fax: +44 1245 242124

BAE Systems (Operations) Limited
Registered Office: Warwick House, PO Box 87,
Farnborough Aerospace Centre, Farnborough, Hants, GU14 6YU, UK
Registered in England & Wales No: 1996687

From: JP Vasseur [mailto:[email-masked]] 
Sent: 17 December 2008 17:05
To: Thomas Heide Clausen; Emmanuel Baccelli; Dearlove, Christopher (UK)
Cc: Philip Levis; Arsalan Tavakoli; Stephen Dawson-Haggerty;
[email-masked]; David E. Culler
Subject: Proposal so as to reach a consensus on **
draft-ietf-roll-protocols-survey-02 **

*** WARNING ***

This mail has originated outside your organization,
either from an external partner or the Global Internet. 
Keep this in mind if you answer this message. 
	
Hi, 

First of all, thanks for all the comments; even late comments are very
much welcome to improve the quality of the document. I'm sure that there
is only a communication issue, and we all want to have a constructive
approach so as to quickly reach a consensus.

1) We have limited time: non-IP and proprietary
protocols/architectures/solutions are being deployed today and the
industry is asking for an IP-based routing protocol for LLN: we do need
to do the right thing but we can't afford to argue for ever on issues
that will not change the overall conclusion,
2) All the requirements documents have been written by authors with a
real-world experience,
4) Designing a protocol that will fulfill those requirements is feasible
(since some of them do exist today). See the "requirements" section
below.

Our charter

Requirements

As you know, we are not currently chartered to work on protocols. We
first had to express our requirements and see whether an existing
protocol would meet these requirements. One cannot ask us to demonstrate
that we can design such a protocol without being chartered to do so.
That being said, considering the number of existing and deployed
protocols in the field designed by some of the people who wrote the
requirements, we can have a good level of confidence that such objective
is achievable. For the record, I've been working myself on such
protocols for some time.

1- Requirements are "guidelines". When time will come to work on
protocol (if re-chartered), anybody will be more than welcome to propose
a protocol (protocol extensions or new protocol). We will have to make
compromise indeed. I would like to remind that we do not have to support
all the MUST in a single instantiation of the protocol. There are ways
to design protocols with options (there are many protocols that have
this property).
2- I would also like to remind that we live in a highly constrained
environment with small footprint (few hundreds of bytes in RAM for
routing is not unusual). We could not afford to use an existing adapted
protocol that would meet our requirements if it requires lots of extra
memory for features that are not needed in LLNs. Any additional memory
has a cost, which matters a lot to LLNs.

Proposal

1) Lack of background, context, ... in the document. Emmanuel is quite
right that background is missing in the document as well as a clear
conclusion: "how have we chosen the protocols in the survey, why
protocols X, Y, Z, ... have not been included in the survey, ...". I saw
that Phil agreed to add some text to address this issue.
Emmanuel, does that address your concern (to be confirmed once you see
the text) ?
Phil, could you please propose some text ?

2) Missing protocols: we had to draw the line somewhere. *LOTS* of
protocols have been proposed over the last decade. We had many
suggestions to also look at protocol X, Y and Z. The decision was
(according to our charter) to limit our survey to existing IETF
protocols (RFC or very mature ID):  OSPF, OLSR, TBRPF, RIP, AODV, DYMO,
DSR. The list is already fairly long. The charter mentions DTN. The WG
decided not to include DTN and this needs to be documented.
Phil, could you please add some text ? Note that this does not mean that
we may not borrow mechanism(s) from existing protocols by the way if we
get re-chartered.

3) Is the goal to exclude existing routing protocols as potential
candidates ? The answer is "no" but Thomas is right, this is currently
not well explained in the document. The objective is to agree that none
of the existing protocols meet the requirement but the solution might
either be to develop a new one or extend an existing routing protocol
(once we get re-chartered).
Thomas, would that address your concern ?

4) Evaluation metric should be in a separate document: the approached
adopted by the WG was to derive metrics from the requirements documents
so as to perform the protocol survey. Once we will start protocol
design, we will go back to the requirement documents since this is *the*
objective of Requirements IDs.

5) Missing Metrics and Incorrect Metrics: Let us know if you have a list
of metrics that are in your opinion "missing" and which metrics are
"incorrect/inappropriate"? Or are we now in agreement based on 4) above
?

JP.

---

From: JP Vasseur <[email-masked]>
Date: Wed, 17 Dec 2008 19:28:09 +0100

Hi Christopher,

On Dec 17, 2008, at 7:09 PM, Dearlove, Christopher (UK) wrote:

> I'm not listed as an action below, as my main point was (and  
> remains) the
> consensus agreement, rather than that the document is bad, although  
> I share
> some concerns over whether the requirements used are "right". But  
> I'm not
> Given all the real constraints on time etc., I think that the below  
> is on the right
> track to improve the document. That leaves point 5 in the final  
> list, which ties
> term process such as various opportunistic store and forward  
> approaches. Note that
> a potentially increased data queuing cost, back to the first point.)  
> And finally (for
> work, have more storage and be more immediate and so on. The  
> possible congestion

Thanks for your feed-back and input. Is it fair to say that you no  
longer have any "blocking" issue but you
want to make sure that we revisit your points when we will start the  
protocol work ?

Thanks.

JP.

> Cc: Philip Levis; Arsalan Tavakoli; Stephen Dawson-Haggerty; [email-masked] 
> ; David E. Culler
> Subject: Proposal so as to reach a consensus on ** draft-ietf-roll- 
> protocols-survey-02 **
>
> *** WARNING ***
>
> First of all, thanks for all the comments; even late comments are  
> very much welcome to improve the quality of the document. I'm sure  
> that there is only a communication issue, and we all want to have a  
> asking for an IP-based routing protocol for LLN: we do need to do  
> the right thing but we can't afford to argue for ever on issues that  
> will not change the overall conclusion,
> 2) All the requirements documents have been written by authors with  
> a real-world experience,
> First, spell out a set of requirements for a few limited  
> applications (almost complete).
> demonstrate that we can design such a protocol without being  
> chartered to do so. That being said, considering the number of  
> existing and deployed protocols in the field designed by some of the  
> people who wrote the requirements, we can have a good level of  
> confidence that such objective is achievable. For the record, I've  
> been working myself on such protocols for some time.
>
> propose a protocol (protocol extensions or new protocol). We will  
> have to make compromise indeed. I would like to remind that we do  
> not have to support all the MUST in a single instantiation of the  
> protocol. There are ways to design protocols with options (there are  
> many protocols that have this property).
> adapted protocol that would meet our requirements if it requires  
> lots of extra memory for features that are not needed in LLNs. Any  
> 1) Lack of background, context, ... in the document. Emmanuel is  
> quite right that background is missing in the document as well as a  
> clear conclusion: "how have we chosen the protocols in the survey,  
> why protocols X, Y, Z, ... have not been included in the  
> survey, ...". I saw that Phil agreed to add some text to address  
> this issue.
> Emmanuel, does that address your concern (to be confirmed once you  
> see the text) ?
> DYMO, DSR. The list is already fairly long. The charter mentions  
> DTN. The WG decided not to include DTN and this needs to be  
> documented.
> that we may not borrow mechanism(s) from existing protocols by the  
> way if we get re-chartered.
>
> candidates ? The answer is "no" but Thomas is right, this is  
> currently not well explained in the document. The objective is to  
> agree that none of the existing protocols meet the requirement but  
> the solution might either be to develop a new one or extend an  
> existing routing protocol (once we get re-chartered).
> 4) Evaluation metric should be in a separate document: the  
> approached adopted by the WG was to derive metrics from the  
> requirements documents so as to perform the protocol survey. Once we  
> will start protocol design, we will go back to the requirement  
> documents since this is *the* objective of Requirements IDs.
>
> are "incorrect/inappropriate"? Or are we now in agreement based on  
> 4) above ?
>

---

From: Philip Levis <[email-masked]>
Date: Wed, 17 Dec 2008 12:05:24 -0800

> Phil, could you please propose some text ?

The OSPF section will be renamed OSPF/IS-IS and I will add supporting  
text that notes the distinctions between the two and why their  
analysis is the same.

> way if we get re-chartered.

The current text reads:

"This document considers "existing routing protocols" to be protocols
that are specified in RFCs or, in the cases of DYMO
[I-D.ietf-manet-dymo] or OLSRv2 [I-D.ietf-manet-olsrv2] , a very
mature draft which will most likely become an RFC."

I propose

"This document considers "existing routing protocols" to be routing  
protocols
that are specified in RFCs or, in the cases of DYMO
[I-D.ietf-manet-dymo] or OLSRv2 [I-D.ietf-manet-olsrv2] , a very
mature draft which will most likely become an RFC. It does not
examine the Network Mobility Basic Support Protocol (NEMO)[cite],
DTN bundles[cite], or the DTN Licklider protocol[cite] because they
are not routing protocols."

> Thomas, would that address your concern ?

The current text reads:

"As they were not designed with
    these requirements in mind, existing protocols may or may not work
    well in LLNs.  The first step to reaching consensus on a routing
    protocol for LLNs is to decide which of these two is true.  If an
    existing protocol can meet LLN requirements without any changes,  
then
    barring extenuating circumstances, it behooves us to use an existing
    standard.  However, if no current protocol can meet LLN's
    requirements, then further work will be needed to define and
    standardize with a protocol that can.  Whether or not such a  
protocol
    involves modifications to an existing protocol or a new protocol
    entirely is outside the scope of this document: this document simply
    seeks to answer the question: do LLNs require a new protocol
    specification document at all?"

I'd appreciate advice on how to make this clearer.

Phil

---

From: Philip Levis <[email-masked]>
Date: Wed, 17 Dec 2008 12:06:41 -0800

Thomas,

(comments inline)

> "control cost" but rather "the total cost of getting traffic from  
> the source to the destination. This includes the control cost that  

I agree that "total cost" (control cost + data cost) is a better  
metric for some protocols; in particular, when considering throughput,  
the channel occupancy time of a protocol per bit delivered is a  
measure of its efficiency. But "control cost" does not address this  
case: rather, it addresses the common case in LLNs where there is a  
very very low data rate. In particular, there is a tradeoff between  
data reporting rate and lifetime: more packets use more battery.  
Furthermore, the data reporting interval may not be known a-priori  
(e.g., event-driven networks), so one cannot simply define a constant  
that matches the data rate. Otherwise you run into situations where  
there's a period of high data but the control interval is low so the  
topology cannot adapt quickly.

In short, simply sending periodic beacons is not a suitable approach  
for LLNs. It's a non-starter for a ROLL protocol. That's why this  
criterion is here: it articulates that control rates cannot surpass  
data rates. Of course you'd like them to be *much* lower, but this  
criterion sets an upper bound. While you might say that it's too weak,  
there are many protocols that don't meet it!

Phil

---

From: "Stephen Dawson-Haggerty" <[email-masked]>
Date: Wed, 17 Dec 2008 13:15:21 -0800

(comments below)

On Wed, Dec 17, 2008 at 12:06 PM, Philip Levis <[email-masked]> wrote:

>>> around for awhile with its own clear list of criteria, which take their
>>> source in the requirements drafts. Someone could propose another doc with
>>> another set of criteria that would make perfect sense for a different set of
>>> requirements.
>>>
>>>
>> criteria -- and that's fine as such. However the choice has to be sensible
>> such that the conclusions that can be drawn are useful.
>>
>> unfortunate, choice is made. I'll again emphasize the "control cost" metric.
>> As it would be, we should not really care for the specific "control cost"
>> but rather "the total cost of getting traffic from the source to the
>> destination. This includes the control cost that incurs, of course, but not
>> only: the cost of carrying data traffic should be considered as well. If the
>> data traffic to be delivered is non-zero, the latter often is important --
>> which is one good reason for a routing protocol to trade of "control costs"
>> for attaining "shorter/shortest paths" (according to whatever metric).
>>
>
> I agree that "total cost" (control cost + data cost) is a better metric for
> some protocols; in particular, when considering throughput, the channel
> occupancy time of a protocol per bit delivered is a measure of its
> efficiency. But "control cost" does not address this case: rather, it
> addresses the common case in LLNs where there is a very very low data rate.
> In particular, there is a tradeoff between data reporting rate and lifetime:
> more packets use more battery. Furthermore, the data reporting interval may
> not be known a-priori (e.g., event-driven networks), so one cannot simply
> define a constant that matches the data rate. Otherwise you run into
> situations where there's a period of high data but the control interval is
> low so the topology cannot adapt quickly.
>
> In short, simply sending periodic beacons is not a suitable approach for
> LLNs. It's a non-starter for a ROLL protocol. That's why this criterion is
> here: it articulates that control rates cannot surpass data rates. Of course
> you'd like them to be *much* lower, but this criterion sets an upper bound.
> While you might say that it's too weak, there are many protocols that don't
> meet it!

Yes, I think it's worth emphasizing that although the metric is called
"control cost," what that metric says is that the data rate must be bounded
by the data rate (except in the edge case); so in effect, we're treating the
data rate as an externally specified value, and then bounding the total
rate.  Now, bounding rates isn't exactly the same thing as bounding the
costs; for instance, if a protocol aggressively traded off stretch for
control traffic, you might pass this metric but still not be a very good
protocol.  Flooding every packet would be an example of this (I think this
was noted on the list at some point).

None of the protocols we examined do this; they all form routes which
attempt to optimize an additive link metric.  If they did do this, we would
have to come up with a different metric.  However, given "traditional"
protocols forward packets by building forwarding tables at each router, this
seems like a pretty good metric.

Would adding a note about this metric's limitations be a good idea?

---

From: "Ulrich Herberg" <[email-masked]>
Date: Wed, 17 Dec 2008 22:41:40 +0100

Comments inline...

> low so the topology cannot adapt quickly.

I don't understand your argument. Why is "control cost" a better criterion
than "total cost"? In terms of energy consumption, even a very low control
cost can inflict very high data traffic cost (e.g. the broadcast example).
It is the total amount of sent frames that is proportional to the energy
consumption, not just the control traffic. You say "more packets use more
battery". Correct, but "more packets" include more data packets as well as
more control packets.

> While you might say that it's too weak, there are many protocols that don't

I think you have to be careful when defining the criteria of a routing
protocol for LLNs not to have a particular routing solution in mind. Part of
the goal of this draft is to set objective criteria for selecting routing
protocols for LLNs. It is dangerous to dismiss any proactive routing
protocol in advance by saying "sending periodic beacons is not a suitable
approach for LLNs", and setting the selection criteria accordingly.

Regards,
Ulrich

---

From: "Stephen Dawson-Haggerty" <[email-masked]>
Date: Wed, 17 Dec 2008 14:24:12 -0800

> Every evaluation picks criteria and applies them. This doc has been around
>>>> for awhile with its own clear list of criteria, which take their source in
>>>> the requirements drafts. Someone could propose another doc with another set
>>>> of criteria that would make perfect sense for a different set of
>>>> requirements.
>>>>
>>>>
>> I agree that "total cost" (control cost + data cost) is a better metric
>> for some protocols; in particular, when considering throughput, the channel

We're really making a statement about the relationship between control and
data traffic.  In some sense this does put a bound on the total cost; it's
just parameterized by the data cost.  Our metric simply aims to make sure
the protocol won't do anything crazy, regardless of what data rate you
chose.  This means, for instance, if you want a very low duty cycle network
you can just send a small amount of data and not worry about the network
creating a lot of traffic on its own.

> approach for LLNs", and setting the selection criteria accordingly.

Agreed.  After the interm meeting we altered the draft based specifically on
this concern, and I do not believe that the document precludes proactive
routing protocols so long as they are tunable to prevent a lot of constant
control traffic in the absence of data.

Best,
Steve

---

From: "Dearlove, Christopher (UK)" <[email-masked]>
Date: Thu, 18 Dec 2008 09:56:53 -0000

protocol work ? 
 
Actually, if you look at the record, I have carefully never called for
this ID
to be blocked, although others have. I have expressed some reservations.
But my main point was, and is, to put a marker down to say that in the
future, there is a danger that the narrow point that this ID makes could
be over-read, and that it shouldn't.
 
That said, making the ID clearer on these points can only be a good
thing.
I regret I haven't had the time to contribute more.

---

From: "Dearlove, Christopher (UK)" <[email-masked]>
Date: Thu, 18 Dec 2008 10:03:14 -0000

> for LLNs.

If your LLN is mostly static then sending exponentially backed-off
beacons is not out of the question.

(I'm neither arguing for or against using such beacons, merely noting
they are not automatically disqualified.)

---

From: "Dearlove, Christopher (UK)" <[email-masked]>
Date: Thu, 18 Dec 2008 10:17:19 -0000

>  Would adding a note about this metric's limitations be a good idea?
  
That sounds like one. I guess if you split control and non-control costs
(which can be argued against, but I'll assume it for now) what is
important
is that as well as having a low control cost, you need to also specify
that
the control signalling does the job of producing efficient routing, i.e.
enabling low non-control costs. Inefficient routing can occur in at
least
three ways
 
- Using multiple routes. Of course some use of multiple routes might be
  a design feature for safety. But I think we're all agreed that
uncontrolled
  blind flooding (as an extreme case) is not a good solution.
 
- Creating inefficient routes. Ideally a minimum metric route would be
  found. A solution within a known factor of that might be OK. A
solution
  that always goes via a gateway may be acceptable (if the load on, and
  congestion at, the gateway is tolerable). The bad extreme case here is
  random routing (which in a finite sized network will eventally get you
to
  your destination, but ...).
 
- Latency. Apart from being an issue in its own right, latency (such as
  waiting for a route finding process to execute, or an extreme case in
  a dynamic network of storing and opportunistically forwarding) adds
  queuing memory costs (or increased packet losses if queuing is
  limited).
 
My point is that saying "control costs should be minimal" ideally needs
a "but control must do a good enough job" qualifier.

---

From: "Emmanuel Baccelli" <[email-masked]>
Date: Thu, 18 Dec 2008 12:02:48 +0100

On Wed, Dec 17, 2008 at 9:05 PM, Philip Levis <[email-masked]> wrote:

>> conclusion: "how have we chosen the protocols in the survey, why protocols
>> X, Y, Z, ... have not been included in the survey, ...". I saw that Phil
>> agreed to add some text to address this issue.
>> Emmanuel, does that address your concern (to be confirmed once you see the
>> text) ?
> The OSPF section will be renamed OSPF/IS-IS and I will add supporting text
> that notes the distinctions between the two and why their analysis is the
> same.

I guess that's not all, right? I suppose there will be additional text
justifying why the set of protocols (and not just OSPF/IS-IS) chosen to be
evaluated is indeed a good sample of what is done in the IETF. Somewhere in
this additional text, we also need some justification why DTN protocols are
ignored and why there is not a word about NEMO-RO. I would rather recommend
that one or two protocols such as PRoPHET (DTN protocol put forward
by Stephen Farrell) be evaluated too, but if there is justification in the
draft why not to do it, it would be OK I guess (at least with me). In any
case, the goal of these additions would be to make the document both more
self-contained and self-explanatory, while easier to relate with the content
of the current charter.

Emmanuel

>> protocols have been proposed over the last decade. We had many suggestions
>> to also look at protocol X, Y and Z. The decision was (according to our
>> charter) to limit our survey to existing IETF protocols (RFC or very mature
>> ID):  OSPF, OLSR, TBRPF, RIP, AODV, DYMO, DSR. The list is already fairly
>> long. The charter mentions DTN. The WG decided not to include DTN and this
>> needs to be documented.
>> we may not borrow mechanism(s) from existing protocols by the way if we get
>> re-chartered.
>>
>
> The current text reads:
>
>> candidates ? The answer is "no" but Thomas is right, this is currently not
>> well explained in the document. The objective is to agree that none of the
>> existing protocols meet the requirement but the solution might either be to
>> develop a new one or extend an existing routing protocol (once we get
>> re-chartered).
>   existing protocol can meet LLN requirements without any changes, then
>   standardize with a protocol that can.  Whether or not such a protocol
>> adopted by the WG was to derive metrics from the requirements documents so
>> as to perform the protocol survey. Once we will start protocol design, we
>> will go back to the requirement documents since this is *the* objective of
>> Requirements IDs.
>>
>> "incorrect/inappropriate"? Or are we now in agreement based on 4) above ?
>>
>>
> Phil
>

---

From: "David E. Culler" <[email-masked]>
Date: Thu, 18 Dec 2008 10:10:14 -0800

Indeed., such an approach would meet the criteria, although a fixed
beacon interval would not.  It is a separate question under what
conditions such a beacon strategy would provide adequate information.
(Yes, if things were truly static that is one case where it does.)  Of
course, there are various adaptive strategies that approach this
behavior while also dealing with changes.  That is the point.  Routing
protocols in this space need to attend to such issues, rather than
assume that beacons are cheap enough to be ignored.  We may find that
existing protocols become much more attractive once they pay more
careful attention to the beacon strategy.

---

From: "Stephen Dawson-Haggerty" <[email-masked]>
Date: Thu, 18 Dec 2008 11:54:32 -0800

I agree with everything you say, and will incorporate statements to this
effect into the new version.  I do see that you can argue against doing the
evaluation this way.  What I believe drove us to use this criteria is that
having picked a set of protocols which all form routes in a pretty classical
way, this is a fair way to compare them since they more or less achieve
similar results.  What we're getting at is that this assumption is implicit
in our metric...

If we were not writing an evaluation and instead were devising a set of
tests which any ROLL protocol would need to pass, this would be less useful
metric because it would not work for every possible protocol and might pass
things like the flooding counterexample.  However, given that these
protocols work as they do, it seems fair.

Steve

On Thu, Dec 18, 2008 at 2:17 AM, Dearlove, Christopher (UK) <
[email-masked]> wrote:

> is that as well as having a low control cost, you need to also specify that
> enabling low non-control costs. Inefficient routing can occur in at least
> three ways
>
>   found. A solution within a known factor of that might be OK. A solution
>   random routing (which in a finite sized network will eventally get you to

---

From: Philip Levis <[email-masked]>
Date: Thu, 18 Dec 2008 10:28:47 -0800

> they are not automatically disqualified.)

Absolutely. This discussion came up in the interim meeting. I should  
have been clearer. Of course one needs to be able to set up routes  
proactively, e.g., through beacon packets. The issue arises when  
correct operation of a protocol depends on nodes to send periodic  
(fixed period) beacons, and this period is uniform across the network.  
This imposes an energy load on nodes which may be energy starved. The  
common case of this behavior is inferring packet losses from beacon  
timers. In these situations, a node MUST send beacons at some pre- 
defined, static, fixed rate or it cannot participate in the topology;  
this mechanism is problematic when it comes to low-power operation.

The basic point of the control cost metric is to establish that, as  
the data rate approaches zero, the control rate should approach some  
very small and tunable epsilon.

Phil

---

From: Philip Levis <[email-masked]>
Date: Thu, 18 Dec 2008 10:30:09 -0800

Comment inline.

On Dec 18, 2008, at 3:02 AM, Emmanuel Baccelli wrote:

> The OSPF section will be renamed OSPF/IS-IS and I will add  
> supporting text that notes the distinctions between the two and why  
> their analysis is the same.
>
> I guess that's not all, right? I suppose there will be additional  
> text justifying why the set of protocols (and not just OSPF/IS-IS)  
> chosen to be evaluated is indeed a good sample of what is done in  
> the IETF. Somewhere in this additional text, we also need some  
> justification why DTN protocols are ignored and why there is not a  
> word about NEMO-RO. I would rather recommend that one or two  
> protocols such as PRoPHET (DTN protocol put forward by Stephen  
> Farrell) be evaluated too, but if there is justification in the  
> draft why not to do it, it would be OK I guess (at least with me).  
> In any case, the goal of these additions would be to make the  
> document both more self-contained and self-explanatory, while easier  
> to relate with the content of the current charter.
>

Emmanuel, did you read the rest of my message? I address NEMO/DTN on  
point 2. Please acknowledge.

Phil

---

From: Philip Levis <[email-masked]>
Date: Thu, 18 Dec 2008 11:08:32 -0800

On Dec 17, 2008, at 1:41 PM, Ulrich Herberg wrote:

>
>
> I don't understand your argument. Why is "control cost" a better  
> criterion than "total cost"? In terms of energy consumption, even a  
> very low control cost can inflict very high data traffic cost (e.g.  
> the broadcast example). It is the total amount of sent frames that  
> is proportional to the energy consumption, not just the control  
> traffic. You say "more packets use more battery". Correct, but "more  
> packets" include more data packets as well as more control packets.

Necessary, but not sufficient. That is, the control cost criterion is  
necessary for a ROLL protocol, but is not a sufficient determinant of  
its utility. The assumption is that the sufficient conditions,  
specified in the application requirements documents, preclude such an  
approach.

Such a broadcast protocol, for example, would satisfy neither the node  
cost nor link cost criteria.

A more technical answer is that any analysis of data cost requires a  
simplification of wireless such that the resulting analysis is neither  
helpful nor valid, because it must address the temporal properties of  
links. Don't forget, O() notation deals with the worst case. For most  
routing protocols, it is fairly simple to construct situations where  
they will have an unbounded total cost (e.g., cannot deliver packets).  
Evaluating data cost would therefore require specifying a simplified  
wireless communication model and assumptions of link behavior, to  
narrow the "expected" environments such that protocols can be  
analyzed, such as the independence of link losses, effects of  
interference, topology, etc. This kind of narrow investigation would  
itself be an issue of tremendous debate, of dubious worth, and  
furthermore would not change the conclusion of the document. So why  
add it?

> criterion sets an upper bound. While you might say that it's too  
> weak, there are many protocols that don't meet it!
>
> I think you have to be careful when defining the criteria of a  
> routing protocol for LLNs not to have a particular routing solution  
> in mind. Part of the goal of this draft is to set objective criteria  
> for selecting routing protocols for LLNs. It is dangerous to dismiss  
> any proactive routing protocol in advance by saying "sending  
> periodic beacons is not a suitable approach for LLNs", and setting  
> the selection criteria accordingly.

I apologize for writing a bit too tersely there; that's not what I  
meant. This came up in the interim meeting. The point that a protocol  
requires all nodes to send beacons at some fixed rate (above a very  
small epsilon) does not meet the control cost criterion does not  
preclude proactive routing. I 100% agree that we do not want to  
preclude such approaches. Please see my response to Christopher.

Phil

---

From: "Emmanuel Baccelli" <[email-masked]>
Date: Fri, 19 Dec 2008 09:58:06 +0100

On Thu, Dec 18, 2008 at 7:30 PM, Philip Levis <[email-masked]> wrote:

>> that one or two protocols such as PRoPHET (DTN protocol put forward by
>> Stephen Farrell) be evaluated too, but if there is justification in the
> Emmanuel, did you read the rest of my message? I address NEMO/DTN on point
> 2. Please acknowledge.
>

Here are some relevant pointers (non-exhaustive list):

http://tools.ietf.org/html/draft-irtf-dtnrg-prophet-01
http://tools.ietf.org/html/draft-thubert-tree-discovery-06

They are in scope (NEMO space and DTN space) and they are describing routing
protocols that may be of interest for ROLL.

If we just add the line you proposed, there will be no justification why
they are not evaluated. Do you agree?

So we have a choice to make: either
(i) justify why such routing approaches are not in scope, or
(ii) evaluate a couple of such protocols in the draft, so as to partially
cover DTN and NEMO spaces, as the charter indicates.

I suppose the latter wouldn't take so long? It's just a couple of additional
paragraphs and a couple of additional lines in the pass/fail table, like for
the protocols already evaluated in the draft... Do you agree?

Emmanuel

---

From: "Stephen Dawson-Haggerty" <[email-masked]>
Date: Fri, 19 Dec 2008 16:50:57 -0800

Hi All,
We are making good progress at integrating all this discussion into the next
revision; however, there has been a lot of traffic and I want to make sure
we hit on everyone's comments.  I've put below a preliminary changelog for
the document.  These are only the major changes so far; there are others
which are more typographical and thus not listed.  If you feel that I am
missing something which we discussed, please let me know either personally
or to the list.

Thanks,
Steve

  - OSPF changed to OSPF/IS-IS, explain the distinctions and why the
analysis is the same
  - Notes justifying why we don't look at NEMO/DTN (verbatim from JP, Phil)
since they're in the charter.
  - Update the "control cost" metric: note that this metric is "necessary
but not sufficient"
  - Be a more explicit about reqs and limitations in the intro.
  - Update DYMO to talk about distance, not hopcount
  - T. Clausen's long email (most should have been considered & addressed)
  - OLSRv2 changed to ? in the control cost metric, because it might be
possible to pass with the right sort of aging of updates. ("Fisheye State
Routing in Mobile Ad Hoc Networks").  It would almost certainly require a
new specification to say how to do this in a correct and inoperable way, but
might not violate the specification.  manet-fsr-00 seems like it has not
been updated in quite a while.  FSR does not alter the "table scalability ";
it's not claimed in the paper.

On Fri, Dec 19, 2008 at 12:58 AM, Emmanuel Baccelli <
[email-masked]> wrote:

>>> text that notes the distinctions between the two and why their analysis is
>>> the same.
>>>
> They are in scope (NEMO space and DTN space) and they are describing
> routing protocols that may be of interest for ROLL.
>
> I suppose the latter wouldn't take so long? It's just a couple of
> additional paragraphs and a couple of additional lines in the pass/fail
> table, like for the protocols already evaluated in the draft... Do you
> agree?
>
> Emmanuel
>
>
>
>
>
>
>>
>>
>>

---

From: Philip Levis <[email-masked]>
Date: Fri, 19 Dec 2008 17:03:34 -0800

> http://tools.ietf.org/html/draft-irtf-dtnrg-prophet-01

This one doesn't fit the requirement of "mature draft."

> http://tools.ietf.org/html/draft-thubert-tree-discovery-06

This is a multi-tunneling protocol across multiple in a nest NEMO  
environment. You're right, we should at least mention why this is not  
included.

>
> If we just add the line you proposed, there will be no justification  
> why they are not evaluated. Do you agree?
>
> (ii) evaluate a couple of such protocols in the draft, so as to  
> partially cover DTN and NEMO spaces, as the charter indicates.
>
> additional paragraphs and a couple of additional lines in the pass/ 
> fail table, like for the protocols already evaluated in the draft...  
> Do you agree?
>

I agree that we need to say something more in depth about the above  
NEMO draft. Do you agree that the DTN one does not satisfy the  
requirement?

Phil

---

From: JP Vasseur <[email-masked]>
Date: Sat, 20 Dec 2008 11:59:05 +0100

On Dec 20, 2008, at 2:03 AM, Philip Levis wrote:

>> Emmanuel, did you read the rest of my message? I address NEMO/DTN  
>> on point 2. Please acknowledge.
>>
>>
> environment. You're right, we should at least mention why this is  
> not included.
>
>

There are interesting mechanisms in this ID that we may end up using  
but again this is an individual submission.

>>
>> They are in scope (NEMO space and DTN space) and they are  
>> describing routing protocols that may be of interest for ROLL.
>
>
>
>
>>
>> If we just add the line you proposed, there will be no  
>> justification why they are not evaluated. Do you agree?
>>
>> fail table, like for the protocols already evaluated in the  
>> draft... Do you agree?
>>
>

---

From: "Emmanuel Baccelli" <[email-masked]>
Date: Mon, 22 Dec 2008 09:09:56 +0100

This sounds good to me.Emmanuel

On Sat, Dec 20, 2008 at 1:50 AM, Stephen Dawson-Haggerty <
[email-masked]> wrote:

> Hi All,
> We are making good progress at integrating all this discussion into the
> next revision; however, there has been a lot of traffic and I want to make
> sure we hit on everyone's comments.  I've put below a preliminary changelog
> for the document.  These are only the major changes so far; there are others
>> On Thu, Dec 18, 2008 at 7:30 PM, Philip Levis <[email-masked]>wrote:
>>
>>> Comment inline.
>>>

---

From: "Emmanuel Baccelli" <[email-masked]>
Date: Mon, 22 Dec 2008 09:17:26 +0100

>> This one doesn't fit the requirement of "mature draft."
On what do you base this judgement? Are there any such drafts in DTN that
are mature enough?  If indeed nothing was mature enough, then DTN would not
have been mentioned in the charter. Anyways, do you agree that saying "we
don't do DTN because there are no DTN routing protocols" is not quite
enough?

> There are interesting mechanisms in this ID that we may end up using but
> again this is an individual submission.
>

I do not see why this is an issue. I think that NEMO is mentioned in the
charter because of such RO drafts. Maybe what could be done is add a section
that would briefly address/evaluate this NEMO approach and one DTN approach,
as side matters that may in fact matter ;)

Emmanuel

>> I agree that we need to say something more in depth about the above NEMO
>> draft. Do you agree that the DTN one does not satisfy the requirement?
>>
>> Phil

---

From: Philip Levis <[email-masked]>
Date: Mon, 29 Dec 2008 14:10:10 -0800

On Dec 22, 2008, at 12:17 AM, Emmanuel Baccelli wrote:
>
> On what do you base this judgement? Are there any such drafts in DTN  
> that are mature enough?  If indeed nothing was mature enough, then  
> DTN would not have been mentioned in the charter. Anyways, do you  
> agree that saying "we don't do DTN because there are no DTN routing  
> protocols" is not quite enough?

Emmanuel,

The judgement basically was "RFCs except for these two MANET drafts  
that are evolutions of prior MANET protocols that have RFCs." OLSRv2  
and DYMO had both gone through many revisions, and so the call was  
that they were likely to be stable enough that the analysis based on a  
draft would not change significantly by the time the drafts become RFCs.

The charter doesn't say that all of these protocols will be evaluated  
in a table. Rather, it reads:

"Survey the applicability of existing protocols to LLNs. The aim of
this document will be to analyze the scaling and characteristics of
existing protocols and identify whether or not they meet the routing
requirements of the applications identified above. Existing IGPs, MANET,
NEMO, DTN routing protocols will be part of evaluation."

I think the key sentence there is

"analyze the scaling and characteristics of
existing protocols and identify whether or not they meet the routing
requirements of the applications identified above"

If a protocol is not a routing protocol, then it can't meet the  
routing requirements. At the time the charter was written, this close  
examination of NEMO and DTN had not yet occurred. In the process of  
writing the draft, we as a working group examined the relevant  
documents and found that their abstractions did not meet the needs of  
ROLL. We could try to force other kinds of protocols into a table  
designed for routing protocols, but that seems problematic and  
misleading. It was important that we look broadly when considering  
what might work for ROLL, but casting such a wide net meant there were  
some protocols that were really a bit out of scope.

I absolutely agree that the next revision needs to clearly state this  
reasoning, so that the document is transparent. We're working on  
fixing that. Do you agree with the reasoning?

Phil

---

From: "Emmanuel Baccelli" <[email-masked]>
Date: Wed, 31 Dec 2008 11:24:16 +0100

On Mon, Dec 29, 2008 at 11:10 PM, Philip Levis <[email-masked]> wrote:

> If a protocol is not a routing protocol, then it can't meet the routing
> requirements.

Let's look at one example I was pointing to, the beginning of the abstract
of http://tools.ietf.org/html/draft-thubert-tree-discovery-06 which reads:

"This paper describes a simple distance vector protocol that exposes

only a default route towards the infrastructure in a nested NEMO

configuration."

if this is not a routing protocol, what is it? If this is not in scope, then
I wonder what is.

By the way, I'm sure you'll agree that it makes more sense to evaluate this
than to evaluate OSPF, in the context of ROLL...

Maybe the next version of the draft will contain a convincing explanation
for these choices, but for now, I must say it still eludes me.

Emmanuel