Subject: Re: [Roll] AD review of draft-ietf-roll-routing-metric

From: JP Vasseur <[email-masked]>
Date: Tue, 7 Dec 2010 08:52:39 +0100

Hi Adrian,

On Nov 26, 2010, at 9:29 AM, Adrian Farrel wrote:

> Hi,
> 
> I have performed an AD review of your draft.
> 

Thanks !

> Don't panic!

Ah ok ;-)

> I review all drafts that I am responsible for before putting them
> forward for IETF last call. The main objective is to catch nits and
> minor issues that would show up during the last call or in IESG
> review. The intention is to help polish your document and make sure
> it is clean and shiny so that other reviewers will stick to the
> technical details.
> 
> Thanks for an important component of the RPL family.
> 
> Most of my comments are pretty trivial, but a couple have more meat
> on them and I'd like to see a quick respin of the document before I
> issue the IETF last call. As soon as I see a new revision posted,
> I'll set the ball in motion.
> 

Perfect - see below (skipping all issues that have been addressed in the new revision)

> Of course, all of my issues are up for discussion.
> 
> Thanks for the work,
> Adrian
> 
> ---
> 
> I am slightly worried about the complexity inherent in this spec.
> Although individual implementations and deployments might choose to use
> a very limited subset of the available (and flexible) options, it seems
> that an implementation must support a much wider subset of options,
> metrics, and constraints in order to be deployable into a wide range of
> networks.
> 
> Does this flexibility impact simplicity of implementation?
> 

JP> We do do not think so. Actually in most cases, implementations will implementation
a very small subset of the metrics/constraints. We have a number of choices and options
here, to address the requirements spelled out in the requirement RFCs but again it is neither
expected NOR required for an implementation to implement them all but certainly each of 
the metric/constraint/feature is justified by a specific requirement.That being said, I have 
been told that some implementations did implement them all.

> Section 1
> 
>   It must be noted that the use of dynamic metrics is not new and has
>   been experimented in ARPANET 2.  The use of dynamic metrics is not
>   trivial and great care must be given to the use of dynamic metrics
>   since it may lead to potential routing instabilities.  That being
>   said, lots of experience has been gained over the years on the use of
>   dynamic routing metrics, which have been deployed in a number of (non
>   IP) networks.
> 
> Can you add a couple of references? One for ARPANET 2 experimentation,

Added:

[Khanna1989J A. Zinky, A. Khanna, and G. Vichniac. "Performance of 
                the Revised Routing Metric for ARPANET and MILNET. 
                Submitted to MILCOM 89, March 1989

> and one for the "lots of experience gained".
> 

A bit more difficult ... people skills in the art would know but ... most are deployed proprietary
systems ...

> ---
> 
> Use of RFC 2119 language.
> 
> The first use (SHOULD) shows up late in Section 1 after a number of
> lower case "must". I think that the best way to smooth this out is to
> not use any 2119 language in the Introduction.
> 

Indeed, good catch, thanks.

> Figure 1
> 
> This is a copy of figure 22 in draft-ietf-roll-rpl.
> I strongly recommend that you do not duplicate protocol bit/byte
> definitions as this can lead to accidental discrepancies. Why not
> replace this figure with a reference?
> 
> ---
> 
> Bit numberings in fields. Please align the numbers so that we
> count bits in base 10.
> 

JP> IF ok with you, I'd rather leave it here, since the RPL document is fairly
large, this is to avoid too much work jumping between the two documents.

> Section 2.1
> 
> I think the A flag needs a little more constraint. Not only is it only
> valid when C=1 (as stated), but it is only valid when R=0.
> 
> Also
> OLD
>      and MUST be written to 0x00.
> NEW
>      and MUST be set to 0x00 and MUST be ignored on receipt.
> END
> 

JP> Yes, thanks.

> ---
> 
> Section 2.1
> 
>   Unrecognized TLVs MUST be ignored.
> 
> Do you want to clarify that "ignoring" includes leaving the TLVs in
> place in the object when it is forwarded along the DAG?
> 

JP> Clarified.

> ---
> 
> Section 2.2
> 
> Have you considered the case that the recording of metrics causes the
> Routing Metric/Constraint object to overflow?
> 

JP> Yes, this is clarified in the RPL spec, allowing for multiple DAG Metric container.
Still I added few words.

> --
> 
> Section 3.1
> 
>   The NSA object MAY be present in the DAG Metric Container.  There
>   MUST be no more than one NSA object as a constraint per DAG Metric
>   Container, and no more than one NSA object as a metric per DAG Metric
>   Container.
> 
> This is fine, but you will need to define somewhere what happens when
> a formatting error is detected. This can either be done on a case-by-
> case basis, or can be a general statement in Section 3.

JP> I added some text, thanks (Section 3): "The sections 3. and 4. specify several 
link and node metric/constraint objects. In some cases it is stated that there must not 
be more than one object of a specific type. In that case, if an RPL implementation 
receives more than one objet of that type, the second objet MUST silently be ignored. "

> 
> See also, Section 3.2
> 
>   The NE object MAY be present in the DAG Metric Container.  There MUST
>   be no more than one NE object as a constraint per DAG Metric
>   Container, and no more than one NE object as a metric per DAG Metric
>   Container.
> 
> etc., etc.
> 
> Incidentally, it might be more graceful to use "MUST NOT" instead of
> "MUST". For example:
> 
> OLD
>   The HP object MAY be present in the DAG Metric Container.  There MUST
>   be no more than one HP object as a constraint per DAG Metric
>   Container, and no more than one HP object as a metric per DAG Metric
>   Container.
> NEW
>   The HP object MAY be present in the DAG Metric Container.  There MUST
>   NOT be more one HP object as a constraint per DAG Metric Container,
>   and there MUST NOT be more than one HP object as a metric per DAG
>   Metric Container.
> END

All fixed.

> 
> Section 3.1
> 
> Is IANA expected to track the flags in the NSA object?

Yes this is in Section 6.3.

> 
> Are the Type values for the TLVs chosen from a common registry across
> all objects, or is there a separate registry for each object?
> 

Good catch ! Section added in IANA for a *common* registry for all TLVs.

> ---
> 
> Section 3.2
> 
>   Whenever possible, a node with low residual energy should not be
>   selected as a router
> 
> Do you have a reference for this?

JP> I reworded the text, since such decision will either be OF specific or driven by local policy. Thanks.

> 
> ---
> 
> Section 3.2
> 
>   Given the complexity of trying to address such a broad collection of
>   constraints, this document defines three levels of fidelity in the
>   solution.
> 
> I only find two levels defined in the section:
> - The simplest solution relies on a 2-bit field
> - The mid-complexity solution is a single parameter
> 
> Did I miss one, or did you?

We used to work on 3, and lead to two ... Forgot to update the text, done, thanks.

> 
> ---
> 
> Section 3.2
> 
>   For scavenging nodes, the 8 bit quantity is the power
>   provided by the scavenger divided by the power consumed by the
>   application, H=P_in/P_out, in units of percent.
> 
> and
> 
>   For
>   battery powered devices, H is the current expected lifetime divided
>   by the desired minimum lifetime.
> 
> Is the second case also expressed as a percentage?

Yes (added), thanks.

> 
> ---
> 
> Section 3.2
> 
> I'm slightly confused by the difference (or not) between the term 'H'
> and the term 'E-E'. Firstly, E-E looks like an arithmetic expression.
> But, more to the point, I *think* you put the value of H in the field
> E-E, so why have two names for the same thing?
> 

JP> Indeed, using one term: E-E

> ---
> 
> Objects, sub-objects, and TLVs
> 
> I think you have some inconsistency in naming.
> 
> At the top level we have "Routing Metric/Constraint object"
> The "object body" of this object is defined as:
>   The object body carries one or more sub-objects defined later in this
>   document.
> So I expect everything that follows in Sections 3 and 4 to be termed
> sub-objects.

Not always, for example, when there is one header followed by the main body, 
it is called an object. When there might be more than one (for example when 
recording several link ETX, we may have multiple sub-objects after the header).

> But you have also termed these things as "objects".
> 
> Some of the objects/sub-objects in Sections 3 and 4 can, themselves have
> sub-objects, while others launch straight into contained TLVs.
> 
> This is all a bit confusing (although there is nothing technically
> wrong) and it would be really nice to achieve some consistency of
> terminology.
> 
> ---
> 
> Section 3.2
> 
>   The format of the NE sub-object body is as follows:
> 
> And what Type value does this sub-object use?
> 

The NE object Type is to be assigned by IANA (recommended value=2)

> ---
> 
> Section 3.2
> 
>   o  T (node Type): 2-bit field indicating the node type.  E=0x00
>      designates a mains-powered node, E=0x01 a battery-powered node and
>      E=0x02 a node powered by an energy scavenger.
> 
> Do you mean T=0x00 etc.?
> 

JP> indeed ! Thanks.

> --
> 
>  
> ---
> 
> Section 3.3
> 
>   The HP object may be used as a constraint or a metric.  When used as
>   a constraint, the DAG root indicates the maximum number of hops that
>   a path may traverse.  When that number is reached, no other node can
>   join that path.  When used as a metric, each visited node simply
>   increments the Hop Count field.
> 
> Are you counting nodes or hops?
> Does the first node send the Hop Count as one or zero?
> 

Text added to include the first node.

> ---
> 
> Section 3.3
> 
> It looks to me that if the object is used as a constraint, it MUST also
> be present as a metric. This seems to be different from the burble at
> the top of the section. (and obviously, we can think of more optimal
> ways of signaling this - such as decrementing the hop count at each
> hop).

JP> Not sure why ... you could use it as a constraint (say to avoid too much
of delays if you know that the number of traversed hop may be an issue) and
still want to optimize against another metric such as the bandwidth ?

> 
> ---
> 
> Section 4.1
> 
>   For the deeply duty-cycled links
>   found in many LLNs
> 
> Will this term be clear to the reader?
> 

JP> Yes I do think so. 

> ---
> 
> Section 4.1
> 
> With special reference to my comment (above) about objects, sub-objects,
> and TLVs, it is really not helpful to have an object (which is actually
> a sub-object) and a sub-object of that object both called Throughput!
> 
> ---
> 
> Section 4.1
> 
>   Figure 9: Throughput sub-object format
> 
> And what Type value does this sub-object use?
> But, I might ask why you need sub-objects when the content is well-known
> to be 32-bit values, and the parent object has a length field.
> 
> ---
> 
> Section 4.1
> 
>   Throughput: 32 bits.  The Throughput is encoded in 32 bits in
>   unsigned integer format, expressed in bytes per second.
> 
> Are you sure this is enough? Yes, it seems like it today. How future-
> proof is this?

JP> 2^32 is huge ?

> 
> ---
> Section 4.2
> 
> Ditto object and sub-object names.
> Ditto sub-object Type value.
> 

JP> XXXX

> ---
> 
> Section 4.2
> 
> Again, it looks like the use of this object as a constraint is only
> possible if it is also used as a metric.

JP> Same comment as for the hop-count. It does not have to be used as a metric
when also used as a constraint.

> So (again) it looks like the
> rules for object presence are broken.
> 
> But that depends how it is used as a constraint! It might be used to
> mean that no link with unsatisfactory latency may be added to the DAG,
> or it might mean that the cumulative latency must not grow beyond the
> the constrained limit.
> 
> You need to clarify meaning and usage.
> 
> ---
> 
> Section 4.3
> 
>   Note that an RPL implementation MAY either use the LQL, the ETX or
>   both.
> 
> This immediately makes me worry about interoperability.

JP> But we have different object ... the reason being that depending on the deployment,
we may either use the LQL or the ETX ....

> 
> ---
> 
> Section 4.3.1
> 
>   By contrast, the LQL link metric may be
>   aggregated, in which case the sum of all LQLs may be reported
>   (additive metric)
> 
> How can this possibly be useful? Unless you know how many hops are
> traversed, the aggregate is meaningless!
> 

JP> This is correct. That being said, we wanted to leave it there, since
it could be useful in some specific scenario (for example, you may use
two values, 0 (good) 1 (bad), in which cases it provides you the number
of bad links traversed, a metric that you could use to select your path).

> ---
> 
> Section 4.3.1
> 
>     0               1               2               3
>     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4
>    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ...
>    |       Res     | LQL sub-object
>    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ...
> 
> The Res field does not appear to be described.
> 

JP> Added (elsewhere too).

> ---
> 
> Section 4.3.1
> 
> It seems to me that a type 0x02 aggregation is not very helpful because
> "unknown" will hide the otherwise 'best' link quality.
> 
> But maybe this doesn't matter because 'worst' link quality is more
> interesting.

JP> Correct this is one of the cases where this is useful indeed.

> ---
> Section 4.4.1
> Ditto Res field

JP> Added.

> ---
> 
> Section 4.4.1
> 
> How come the LC field in the two sub-objects have different sizes?
> 

JP> Oops ! Fixed. Thanks.

> ---
> 
> Section 4.4.1
> 
>   The use of the LC object is outside the scope of this document.
> 
> It seems to me you have gone off half cocked :-(
> 

JP> Poorly worded ... It should read "it is left to the implementer to define the meaning
of each bit of the 10-bit Link Color Flag field". The use itself is clearly defined. Thanks.
Text added.

> There is some description at the top of the section that implies a
> little about how LC is used. It seems closer to a color (a la ToS) than
> to a bit array (a la resource affinities).

JP> You're right and this is aimed at being used as a flag field, a la resource affinities.
I clarified the text.

> 
> If you are determined that this is out of scope for your document, I
> wonder why the object is defined, and I wonder why the section has
> preamble on the use of the meaning of LC. And I really wonder about the
> presence of section 4.4.2.
> 
> ---
> 
> Section 6
> 
> There doesn't seem to be any tracking of sub-object Type values.

As proposed earlier, let's have a common directory for all constraints/metrics object.
And cal them object if if the case where they are carried in other object. Same reasoning
applies to the TLV case (not that in the case of OSPF and ISIS we also have discrepancies,
some protocols called sub-TLV, a TLV carried within a TLV, other protocol using the same 
term regardless on whether or not the TLV is carried within another TLV or not.

This way, no need to add another registry, we can use the one defined in Section 6.1.
I added a common registry for the TLV in Section 6.2.

> 
> ---
> 
> Section 7
> 
> You really need a reference to draft-ietf-roll-security-framework.
> 
> I think that you made a key security comment in a previous section where
> you said that it is recommended that the algorithms used do not cause
> excess information flapping. A significant security attack might be
> mounted by periodically attacking the quality of a link resulting in the
> routing protocol flapping. This would cause wider damage than simply
> the disruption to the local link.

Excellent point. I did not explicitly point to the security framework since it does not
explicitly cover the routing metrics but I propose adding the following text: "Another 
attack may consist of making intermitment attacks on a link in an attempt to constantly 
modify the link quality and consequently the associated routing metric, thus leading to 
potential fluctuation in the DAG. It is thus RECOMMENDED for a RPL implementation 
to put in place mechanism so as to stop advertising routing metrics for highly unstable 
links that may be subject to attacks."

Let me know if that's OK.

Thanks again for the detailed review.

Cheers.

JP.

> 
> _______________________________________________
> Roll mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/roll

---

From: David Culler <[email-masked]>
Date: Tue, 7 Dec 2010 10:22:18 -0800

Adrian,
   Thanks very much for your efforts here.  It looks like the responses address the specific issues, but I'd like to know if you feel that they address the larger issue of unnecessary complexity.  This has been a pervasive concern with this particular piece of work.  We have managed to contain it a good bit and we have eliminated the fluff of earlier versions.

D.

On Dec 6, 2010, at 11:52 PM, JP Vasseur wrote:

> <Diff-metrics-12-13.webarchive>
> <draft-ietf-roll-routing-metrics-13.txt>
> 
> Cheers.
> 
> JP.
> 
>> 

David Culler
[email-masked]
Chair Computer Science
Associate Chair EECS

---

From: Adrian Farrel <[email-masked]>
Date: Wed, 08 Dec 2010 20:04:25 +0000

Hi JP,

Thanks for the excellent progress.

I have whittled down to just a couple of discussion points.

>> I am slightly worried about the complexity inherent in 
>> this spec. Although individual implementations and
>> deployments might choose to use a very limited subset
>> of the available (and flexible) options, it seems that an
>> implementation must support a much wider subset of
>> options, metrics, and constraints in order to be 
>> deployable into a wide range of networks.
>>
> JP> We do not think so. Actually in most cases,
> implementations will implementation a very small subset
> of the metrics/constraints. We have a number of choices
> and options here, to address the requirements spelled out
> in the requirement RFCs but again it is neither expected
> NOR required for an implementation to implement them
> all but certainly each of the metric/constraint/feature is
> justified by a specific requirement. That being said, I have 

So, my only concern in the complexity is what happens when two implementations
implement a different subset. Can those two implementations be deployed in the
same network. I think the answer is "yes", but that in this case only the
intersection of the two subsets can be used. I *think* the protocol can
dynamically handle these cases, but that the result may be that the desired
service cannot be provided.

The implication here is that each device must list the set of functions it
supports, and the operator must buy devices with a suitable function set.

This is OK if the intention is to document "profiles". For example, the "deploy
in an office" profile includes metrics x, y, and z, and constraints p and q.
Then the implementer can build to a profile and market accordingly.

Otherwise, the implementer is left wondering which subset to include and will
(as your correspondents inform you) end up implementing the lot. If implementing
the full set is the right solution, then why not make them all mandatory?

>> Can you add a couple of references?...
>> 
>> ...and one for the "lots of experience gained".
>
> A bit more difficult ... people skills in the art would know
> but ... most are deployed proprietary systems ...

Well, this text

   That being said, lots of experience has been gained
   over the years on the use of dynamic routing metrics, which have been
   deployed in a number of (non IP) networks.

sits as an unsubstantiated assertion. I don't like them.
If you really can't give a reference, can you consider whether the text is
necessary?

>> I strongly recommend that you do not duplicate protocol
>> bit/byte definitions as this can lead to accidental
>> discrepancies. Why not replace this figure with a
>> reference?
>
> JP> IF ok with you, I'd rather leave it here, since the RPL
> document is fairly large, this is to avoid too much work jumping
> between the two documents.

Well, you know my concern: if you have a duplicate definition then you risk
having contradicting definitions.
So, if you believe that this figure is helpful for the reader can you include...

Figure 1 is copied from Figure 22 in [I-D.ietf-roll-rpl] and is provided here
for information only. The definition of the DAG Metric Container format in
[I-D.ietf-roll-rpl] is normative.

>> Section 3.3
>>
>> It looks to me that if the object is used as a constraint, it MUST
>> also be present as a metric. This seems to be different from the
>>burble at the top of the section. 
>
> JP> Not sure why ... you could use it as a constraint (say to avoid
> too much of delays if you know that the number of traversed hop
> may be an issue) and still want to optimize against another metric 
> such as the bandwidth ?

Well, the text says...

   join that path. 

But, if the object states how many hops are allowed (i.e. is a constraint) then
how will you know how many hops have actually been traversed? For that you need
the metric.

Actually, this would apply to the use of all metric-based constraints. E.g., if
you set a constraint that the path cost must not be greater than x, you can
include this in a constraint variant of the object, but you also need to record
the metric as you go. See my comment on Section 4.2...

> JP> Same comment as for the hop-count. It does not have to be
> used as a metric when also used as a constraint.

>> Section 4.3
>>
>>   Note that an RPL implementation MAY either use the LQL, 
>>   the ETX or both.
>>
> JP> But we have different object ... the reason being that depending
> on the deployment, we may either use the LQL or the ETX ....

My point is that if this is an implementation choice, you may find some
implementations use only LQL, and some that only use ETX. That would meant that
you cannot deploy either mechanism because you don't have end-to-end support for
either.

Perhaps you do not mean that this is an implementation choice, but really a
deployment choice?

>> How can this possibly be useful? Unless you know how many
>> hops are traversed, the aggregate is meaningless!
>
> JP> This is correct. That being said, we wanted to leave it there, 
> since it could be useful in some specific scenario (for example,
> you may use two values, 0 (good) 1 (bad), in which cases it
> provides you the number of bad links traversed, a metric that
> you could use to select your path).

I'm not going to buy this because the values for quality levels have been
defined as 1 through 7 (not 0 or 1) with the highest value being the highest
quality (not with 0 being the good quality).

So, what is the purpose of an aggregate LQL?

>> Section 4.3.1
>> 
>> It seems to me that a type 0x02 aggregation is not very 
>> helpful because "unknown" will hide the otherwise 'best'
>> link quality.
>>
>> But maybe this doesn't matter because 'worst' link quality is
>> more interesting.
>

So you are agreeing that, while A=0x01 is useful, A=0x02 is probably not useful
(certainly in the presence of unknown LQL). I wonder if anyone even actually
wants to know the best LQL on a path: what would it be used for?

Thanks,
Adrian