Subject: [Roll] Review of draft-ietf-roll-enrollment-priority-04

From: Konrad Iwanicki <[email-masked]>
Date: Wed, 4 Aug 2021 22:31:15 +0200

Dear authors,

This is the first review I am doing for IETF's draft, so any suggestions 
on how I could improve the quality of my future reviews are welcome.

I have the following major issues with the draft. Perhaps some of them 
have been covered during some meetings (sorry for missing those); my 
comments are based just on what I got from the draft (and the related 
drafts).

1. I was confused by two seemingly contradictory statements.

Par. 2 of sect. 2 (starting with "With this specification") states that 
the introduced option "MUST be propagated down the DODAG without 
modification" (compared to what is issued by the DODAG root).

However, par. 2 of sect. 2.1 (starting with "6LRs that support this 
option") states that "nodes adjust this base value based upon their 
observed congestion, emitting their adjusted DIO value to their children."

What I understand is that the adjustment applies only in a particular 
case. More specifically, if a node receives the option, it copies it to 
its DIOs without any modification. In contrast, if the node does not 
receive the option but is capable of processing it, then it synthesizes 
the option for its DIOs by taking as the min priority the base value 
(0x40) adjusted with the node's local observations. This seems 
inconsistent, though. In particular, two children with the same parent 
can emit DIOs with different values of min priority. Why not simply omit 
the option in such a case?

2. The draft implicitly assumes a DIO processing policy that differs 
from RPL's original one.

More specifically, throughout the draft, it is implied or assumed that a 
node processes the option received only from its parent(s). For example:
- par. 8 of sect. 1 (starting with "A network operator might"): "... it 
would like to adjust the enrollment priority (the proxy priority field 
of [...]) among all nodes in the subtree of a congested link.";
- par. 1 of sect. 2.1 (starting with "A 6LR which did not support"): 
"Children and grandchildren nodes would therefore not receive any 
telemetry via that path and need to assume a default value.";
- par. 2 of sect. 2.1: "6LRs that support this option, but whose parent 
does not send it [...]" and the rest of the paragraph as well.

In contrast, RFC 6550, sect. 8.2.3 states that DIOs not only from 
parents but also other neighbors must be processed by a node. This 
guarantees correct route formation and maintenance.

Therefore, if the draft changes this policy with respect to the 
introduced option, I would expect it to state this change explicitly. 
Such a change would also require answering several questions, notably:
- From which parent do we process the option?
- If only from the preferred parent, what happens if a node has none?
- If from any, then what happens if a node gets different values of min 
priority or DODAG_Size from different parents?

However, I believe that a meta question worth answering first is: Do we 
want to have different min priority values in different subtrees or just 
one global value for the entire DODAG?

Either of the answers in my view requires explicit formulation and 
preferably also an explanation in the draft.

3. Even if the draft identified a single parent as the only source of 
min priority values for a node, there still would be a problem of value 
consistency.

Since the DODAG root can change its values of min priority and 
DODAG_Size and since a node can change its parents, there may still be a 
case when the node must decide whether to adopt the newly received 
values or not. Without a proper conflict resolution policy, it may be 
the case that when the root, for example, disables enrollment, then 
enables it again, then the DODAG may exhibit strange behaviors: the 
root's decision may not be propagated to all nodes, a decision may be 
revoked by nodes that have already accepted it, and the like.

There are many ways in which such conflict resolution can be provided. 
If I may suggest anything, I would recommend adding to the option a 
lollipop sequence counter (like those described in RFC 6550, sect. 7.2), 
let's call it osn. It would effectively order different versions of the 
option values produced by the DODAG root. In effect, whenever a node 
received an option (be it from a parent or another neighbor, depending 
on the way the previous issues are addressed), it would adopt the min 
priority and DODAG_Size values the option carries if and only if the osn 
value in the received option is greater than the node's locally stored 
osn. This would guarantee eventual consistency and avoid strange 
behaviors such as those mentioned above.

4. Why put DODAG_Size in the option?

I can imagine that other services/protocols would like to use it as 
well, and extracting this information from an option related solely to 
enrollment does not seem the best idea. I have been thinking for some 
time about an option for RPL that would provide some aggregate 
information about the DODAG to the nodes, such as its size, max depth, 
and the like. The option could also be used in the enrollment process. 
Do you think this makes sense?

5. It should be made clearer in the draft what is actually being configured.

The only place where field "proxy priority" of EB appears in the entire 
draft is par. 8 of sect. 1. Reading the draft for the first time I was 
confused what min priority for the option is used for and what is 
actually adjusted at various places.

Apart from these major issues, I have also a few editorial suggestions.

A. After the par. 1 of sect. 1, I would introduce a subsection heading 
titled, for instance, "Terminology Disambiguation" to prepare the 
readers for what they are going to see next.

B. Similarly, before par. 5 of sect. 1 (starting with "It has become 
clear that not every routing member"), I would introduce a subsection 
heading titled "Motivation and Overview", just "Overview", or something 
along these lines.

C. I would promote the "Terminology" subsection to a section.

D. I would move par. 1 of sect. 2 (starting with "The size of the DODAG 
is measured by the Root") toward the end of sect. 1 as it has little to 
do with the option definition.

E. I would fix the bullet list in sect. 2.1 so that it renders correctly 
in the text version of the draft.

Finally, I have numerous small editorial suggestions, but they seem 
immaterial at this point.

Hope the review helps.

Best,
-- 
- Konrad Iwanicki.

---

From: Rahul Jadhav <[email-masked]>
Date: Fri, 6 Aug 2021 12:17:43 +0530

Excellent review. Many thanks Konrad.

Please find my comments inline.

Thanks,
Rahul

> on how I could improve the quality of my future reviews are welcome.

[RJ] This is an excellent in-depth review. Appreciate.

> observed congestion, emitting their adjusted DIO value to their children."

[RJ] This is a good point. Will raise this as an issue on GH. The way I see
it, the value adjustment might have to be done going downstream and I would
like to see what others think of it. Nonetheless, the text will need
modifications.

> the option in such a case?

[RJ] Two children with the same parent are permitted to emit DIOs with
different values of min priority. Consider the case where child1 has fewer
resources than child2 and wants to limit downstream nodes joining through
it. This case requires different min-priority to be emitted. There are
several other reasons why the priorities could be different.

> introduced option, I would expect it to state this change explicitly.

[RJ] I don't think the draft changes this policy.
RFC 6550 says that the DIO will be received by a node from all its
neighbors and that is because of the broadcast nature of the DIOs. When
DIOs are received from the neighbors, these neighbors could be promoted to
the parent set. The draft does not change this processing behavior. Par 2
of sect 2.1 "6LRs that support this option, but whose parent does not send
it...." this statement does not change the 6550 behavior. All it says is
for some reason, if the selected parent does not support the option then
the draft dictates a particular behavior.

> priority or DODAG_Size from different parents?

[RJ] If we agree that we are not changing 6550 DIO processing behavior as
mentioned above, then these questions do not apply.

> one global value for the entire DODAG?

[RJ] This is the point to be discussed. IMO, the min priority is not a
constant global value. However, the draft text contradicts this and will
raise the issue in GH before we fix it to garner some discussion.

> revoked by nodes that have already accepted it, and the like.

[RJ] The exact conflict resolution policy is not within the scope of this
document. However, for the scenario you quoted, "What happens if the
enrollment is enabled/disabled by the root?" We do not expect the
enrollment enable/disable to be precisely known by all others i.e., there
could be nodes who do not have the latest status of the enrollment option.
This could cause some additional signaling i.e., a node might try to join
even though the (latest) priority did not allow it. This could lead to a
few more messaging but it should not result in any untoward behavior.

> behaviors such as those mentioned above.

[RJ] With LLNs there is no way to have guaranteed consistency. Consider the
case where a node goes off the network and comes back later and did not see
part of the traffic at all.
Also, note that there is other non-static information such as
metrics/constraints that are carried by the DIO. Worst case if the
information changes too quickly and the nodes might not have consistent
information there could be some additionally messaging but over time the
network should converge to the right configuration. I am not sure if
introducing something like OSN is the way to tackle this problem.

> 4. Why put DODAG_Size in the option?

[RJ] This was a recent proposition. Please check this discussion for
context, "[Roll] About measure DODAG size and priority
<https://mailarchive.ietf.org/arch/msg/roll/-MqqxvTW-3bM5F4k4qH7n5FaxYM/>"
https://mailarchive.ietf.org/arch/msg/roll/-MqqxvTW-3bM5F4k4qH7n5FaxYM/

> Do you think this makes sense?

[RJ] size is already there. For max-depth, what could be the use-case? Is
there anything else you have in mind? Would be nice to discuss it now.

> actually adjusted at various places.

[RJ] The aim of the draft was to be generic enough but also serve/fulfill
the use-case for the 6tisch scenario. Hence the EB point appears only in
the introduction.
In my previous experience where we used PANA
<https://en.wikipedia.org/wiki/Protocol_for_Carrying_Authentication_for_Network_Access>
for onboarding nodes in an AMI network, we needed a similar feature and we
ended up using a proprietary (but similar) signaling.

> readers for what they are going to see next.

[RJ] I think we should have defined the terms used in the document in the
Terminology section as done in all other drafts. Will do that? Do you see
any ambiguous terms used?

> heading titled "Motivation and Overview", just "Overview", or something

[RJ] Let me check if we could improve this.

> C. I would promote the "Terminology" subsection to a section.

[RJ] We'll add the terms used in the document in the Terminology section
(Section 1.1). The Terminology section defined today doesn't serve the
draft well. Needs update.

> do with the option definition.

[RJ] Makes sense to me.

> in the text version of the draft.

[RJ] The text rendering seems alright to me in the browser. Should we add a
colon ':' after the bullet tag?

> Hope the review helps.

[RJ] Certainly helps. Thank You.

>
> Best,
> --
> - Konrad Iwanicki.
>
> _______________________________________________
> Roll mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/roll
>

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Fri, 6 Aug 2021 11:02:41 +0000

Dear Konrad

Many fair points below. Many thanks for is a very useful review.

Please see below

> -----Original Message-----
> From: Roll <[email-masked]> On Behalf Of Konrad Iwanicki
> Subject: [Roll] Review of draft-ietf-roll-enrollment-priority-04
> 
> Dear authors,
> 

This is indeed very helpful, Konrad. With this quality of comments, the more the merrier. 

And as it goes, the ROLL interims are usually at very palatable times for people in central Europe. 
You're free and would be very welcome to join, no need to register to anything though we like to see the real names of attendees for the records.

For the rest, I'll react from Rahul's reply.

Keep safe!

Pascal

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Fri, 6 Aug 2021 12:45:05 +0000

The nodes then
   adjust this base value based upon their observed congestion, emitting
   their adjusted DIO value to their children.

From: Roll <[email-masked]> On Behalf Of Rahul Jadhav
Sent: vendredi 6 août 2021 8:48
To: Routing Over Low power and Lossy networks <[email-masked]>
Cc: roll-chairs <[email-masked]>
Subject: Re: [Roll] Review of draft-ietf-roll-enrollment-priority-04

[Pascal]: very much, yes!

Thanks,
Rahul

On Thu, 5 Aug 2021 at 02:02, Konrad Iwanicki <[email-masked]<mailto:[email-masked]>> wrote:
Dear authors,

[RJ] This is a good point. Will raise this as an issue on GH. The way I see it, the value adjustment might have to be done going downstream and I would like to see what others think of it. Nonetheless, the text will need modifications.

[Pascal] The field is the MIN priority and it is decided by the root. To my best knowledge/memory, the intent was to adjust the priority value that is reported outside of the DIO (in beacons) that cannot be lower than the minimum priority in the DIO, e.g., when the 6LR exposes its desire to be a join proxy, as discussed in this text:
“

   A 6LR which would otherwise be willing to act as a _Join Proxy_, will
   examine the minimum priority field, and to that number, add any
   additional local consideration (such as upstream congestion).

“
But the Min priority in the DIO is unchanged. Note that if it was, a child with multiple parents would get conflicting DIOs with both timely information and we would have to specify which value it reports. Right now the idea would be to report the most recent, though it is somewhat unclear how that is determined.

Still I agree that we need to improve the text, as Konrad says it is quite confusing.

If we want something that reports the load of the DODAG above, we’ll have to examine the value and if it is needed, then define the behavior.

[RJ] Two children with the same parent are permitted to emit DIOs with different values of min priority. Consider the case where child1 has fewer resources than child2 and wants to limit downstream nodes joining through it. This case requires different min-priority to be emitted. There are several other reasons why the priorities could be different.

[Pascal]

To Konrad’s point : I  agree with you that this creates a situation that is normally seen only when the root changes the value (which arguably may happen a lot and should be dampened with hysteresis and stuff). I believe we should retract the text that allows the 6LR to generate the option, reason is, if the root sets it and only the 6LR’s parent do not support, a child of the 6LR may see bith the Root’s value and the 6LR’s value. Dangerous. So the suggestion is to retract
“

The nodes then
   adjust this base value based upon their observed congestion, emitting
   their adjusted DIO value to their children                                                                                         “

“

To Rahul’s: This could be an important thing to report, and we can take it outside this thread. As of today we do not report the parent load/capability to take more children in DIOs. The reason we did not do that in the past was to limit the complexity of the RPL spec. Because you’d also need to explain what nodes do with the information, e.g., how to balance the graph so that everyone has at least one parent, which would involve some shoulder to shoulder pushback. So we said at the time, either all nodes have enough memory or you go for non-storing, but you never have to react to the parent’s load.

This is something we can reopen now, but probably under the RPLv2 umbrella.

[RJ] I don't think the draft changes this policy.
RFC 6550 says that the DIO will be received by a node from all its neighbors and that is because of the broadcast nature of the DIOs. When DIOs are received from the neighbors, these neighbors could be promoted to the parent set. The draft does not change this processing behavior. Par 2 of sect 2.1 "6LRs that support this option, but whose parent does not send it...." this statement does not change the 6550 behavior. All it says is for some reason, if the selected parent does not support the option then the draft dictates a particular behavior.

[Pascal] I believe that Konrad’s point is that
“

RPL limits the cases where a node may process DIO messages from

   deeper nodes to some form of local repair.

“  from RFC 6550, as explained in more details in  https://datatracker.ietf.org/doc/html/rfc6550#section-8.2.2.4

Which is IOW that a node normally ignores DIOs coming from deeper. But it needs to process DIOs for all nodes that may become potential parent.
On the one hand I’d say that if the option is expected to report parent load in the future, the node should parse the option in any DIO that is acceptable otherwise, and to Konrad’s point follow the exact same rules as RPL.
OTOH, this is more chances for getting different values along different asynchronous paths. Which makes the problem of finding the most recent value even harder.

Seems that the option should have its own sequence, which is a generic problem that was discussed in draft-thubert-roll-eliding-dio-information. Note that in this case, I do not believe that we should change the RCSS in that draft, so the best could be that the option has its own sequence. The text would change to reflect that we use any DIO that we normally use, and retain the value of the min priority from the most recent of all.

[RJ] If we agree that we are not changing 6550 DIO processing behavior as mentioned above, then these questions do not apply.

[Pascal] I think they do, per my replies above.

[RJ] This is the point to be discussed. IMO, the min priority is not a constant global value. However, the draft text contradicts this and will raise the issue in GH before we fix it to garner some discussion.

[Pascal] The min priority in the draft is effectively global, set by the root. If we want other info in the draft, it is still time to add it, but please let’s work on it now and then ship. There are external bodies waiting for this spec.

[Pascal] Indeed

[RJ] The exact conflict resolution policy is not within the scope of this document. However, for the scenario you quoted, "What happens if the enrollment is enabled/disabled by the root?" We do not expect the enrollment enable/disable to be precisely known by all others i.e., there could be nodes who do not have the latest status of the enrollment option. This could cause some additional signaling i.e., a node might try to join even though the (latest) priority did not allow it. This could lead to a few more messaging but it should not result in any untoward behavior.

 [Pascal] I believe we can get things a lot simpler if 1) the min comes only from the root as of now and 2) we add a sequence counter in the option that the Root increments to indicate the freshest.

[RJ] With LLNs there is no way to have guaranteed consistency. Consider the case where a node goes off the network and comes back later and did not see part of the traffic at all.
Also, note that there is other non-static information such as metrics/constraints that are carried by the DIO. Worst case if the information changes too quickly and the nodes might not have consistent information there could be some additionally messaging but over time the network should converge to the right configuration. I am not sure if introducing something like OSN is the way to tackle this problem.

[Pascal] Arf I should have read it through. I agree with Konrad again.

[RJ] This was a recent proposition. Please check this discussion for context, "[Roll] About measure DODAG size and priority<https://mailarchive.ietf.org/arch/msg/roll/-MqqxvTW-3bM5F4k4qH7n5FaxYM/>"
https://mailarchive.ietf.org/arch/msg/roll/-MqqxvTW-3bM5F4k4qH7n5FaxYM/

[Pascal] Indeed, we have the request from Wi-SUN which uses RPL. The goal is to balance the size of multiple large DODAGs in the same instance.

[RJ] size is already there. For max-depth, what could be the use-case? Is there anything else you have in mind? Would be nice to discuss it now.

[Pascal] Indeed now is a good time. The max depth would be an indication for the 6LR NOT TO accept joins if it is too deep, so it is related to join. Now that’s also a very dangerous thing to do. It would happensin the field that some networks get very deep, e.g., when one of the Roots reboot and large DODAGs merge to absorb the nodes left orphan.

[RJ] The aim of the draft was to be generic enough but also serve/fulfill the use-case for the 6tisch scenario. Hence the EB point appears only in the introduction.
In my previous experience where we used PANA<https://en.wikipedia.org/wiki/Protocol_for_Carrying_Authentication_for_Network_Access> for onboarding nodes in an AMI network, we needed a similar feature and we ended up using a proprietary (but similar) signaling.

 [Pascal] All true, and yet we have an opportunity to clarify that text that we need to leverage before IESG, because that point would come back with a revenge. Let’s clarify.

For the editorial discussions, I agree with Konrad’s point and  Rahul’s answers and leave it your echange.

Keep safe!

Pascal

---

From: Michael Richardson <[email-masked]>
Date: Mon, 09 Aug 2021 14:48:40 -0400

Konrad Iwanicki <[email-masked]> wrote:
    > I have the following major issues with the draft. Perhaps some of them have
    > been covered during some meetings (sorry for missing those); my comments are
    > based just on what I got from the draft (and the related drafts).

    > Par. 2 of sect. 2 (starting with "With this specification") states that the
    > introduced option "MUST be propagated down the DODAG without modification"
    > (compared to what is issued by the DODAG root).

Good catch.  I will remove the words "without modification"

    > case. More specifically, if a node receives the option, it copies it to its
    > DIOs without any modification. In contrast, if the node does not receive the
    > option but is capable of processing it, then it synthesizes the option for
    > its DIOs by taking as the min priority the base value (0x40) adjusted with
    > the node's local observations.

Agreed.

    > This seems inconsistent, though.
    > In particular, two children with the same parent can emit DIOs with different
    > values of min priority. Why not simply omit the option in such a case?

So you are assuming the DAG:

         A
        / \
       B   C
      / \   \
     D  E   F

And "A" does not know to send this option, so both B and C assume a value of 0x40.
Then, each one adjusts this based upon their local conditions.
D+E might see a value of 0x50 (if B added 16), while F might see a value of
0x60 (if C added 32).
This would make new nodes prefer to enroll with D or E, rather than F.

The reason is that perhaps B has more resources than C for new children.

    > 2. The draft implicitly assumes a DIO processing policy that differs from
    > RPL's original one.

    > More specifically, throughout the draft, it is implied or assumed that a node
    > processes the option received only from its parent(s).
    > For example:
    > - par. 8 of sect. 1 (starting with "A network operator might"): "... it would
    > like to adjust the enrollment priority (the proxy priority field of [...])
    > among all nodes in the subtree of a congested link.";

    > "Children and grandchildren nodes would therefore not receive any telemetry
    > via that path and need to assume a default value.";

    > - par. 2 of sect. 2.1: "6LRs that support this option, but whose parent does
    > not send it [...]" and the rest of the paragraph as well.

    > In contrast, RFC 6550, sect. 8.2.3 states that DIOs not only from parents but
    > also other neighbors must be processed by a node. This guarantees correct
    > route formation and maintenance.

Yes, it's true.  A node should evaluate DIOs from other sources, and if those
sources are better, then it should switch parents.  But at any point, a node
really has only one parent.
The other nodes are potential parents, but not parents.
Perhaps RFC6550 has some other terms which I've neglected to use.

    > 3. Even if the draft identified a single parent as the only source of min
    > priority values for a node, there still would be a problem of value
    > consistency.

    > Since the DODAG root can change its values of min priority and DODAG_Size and
    > since a node can change its parents, there may still be a case when the node
    > must decide whether to adopt the newly received values or not. Without a
    > proper conflict resolution policy, it may be the case that when the root, for
    > example, disables enrollment, then enables it again, then the DODAG may
    > exhibit strange behaviors: the root's decision may not be propagated to all
    > nodes, a decision may be revoked by nodes that have already accepted it, and
    > the like.

A root that did this, would have to increment the DTSN right each time it
changed it's mind.

    > There are many ways in which such conflict resolution can be provided. If I
    > may suggest anything, I would recommend adding to the option a lollipop
    > sequence counter (like those described in RFC 6550, sect. 7.2), let's call it
    > osn. It would effectively order different versions of the option values
    > produced by the DODAG root. In effect, whenever a node received an option (be
    > it from a parent or another neighbor, depending on the way the previous
    > issues are addressed), it would adopt the min priority and DODAG_Size values
    > the option carries if and only if the osn value in the received option is
    > greater than the node's locally stored osn. This would guarantee eventual
    > consistency and avoid strange behaviors such as those mentioned above.

I'm not convinced we need another lollipop counter.
I think that we already have that.

    > I can imagine that other services/protocols would like to use it as well, and
    > extracting this information from an option related solely to enrollment does
    > not seem the best idea. I have been thinking for some time about an option
    > for RPL that would provide some aggregate information about the DODAG to the
    > nodes, such as its size, max depth, and the like. The option could also be
    > used in the enrollment process. Do you think this makes sense?

We agreed to merge two drafts.  The DODAG_Size came from the other draft.
We think that the two values are very much related and it made sense to send
them together.

    > titled, for instance, "Terminology Disambiguation" to prepare the readers for
    > what they are going to see next.

okay, I have re-organized some text.

    > B. Similarly, before par. 5 of sect. 1 (starting with "It has become clear
    > that not every routing member"), I would introduce a subsection heading
    > titled "Motivation and Overview", just "Overview", or something along these
    > lines.

    > D. I would move par. 1 of sect. 2 (starting with "The size of the DODAG is
    > measured by the Root") toward the end of sect. 1 as it has little to do with
    > the option definition.

okay.

    > E. I would fix the bullet list in sect. 2.1 so that it renders correctly in
    > the text version of the draft.

markdown issue.

--
Michael Richardson <[email-masked]>   . o O ( IPv6 IøT consulting )
           Sandelman Software Works Inc, Ottawa and Worldwide

---

From: Konrad Iwanicki <[email-masked]>
Date: Mon, 9 Aug 2021 23:09:37 +0200

Dear Michael and all,

First of all, thank you for your kind words regarding the review. I am 
actually planning to attend the August interim, because I will be 
presenting the rnfd draft, so we can discuss the enrollment priority 
draft in depth as well.

Below I just quickly reply inline to Michael's comments as they led to a 
new version of the draft. I was actually replying to Pascal's and 
Rahul's comments while the new draft version appeared, and the new 
version still has some issues, which I wanted to point out in those replies.

So what do I do then, write another review for the new version where I 
can explain those issues?

> Perhaps RFC6550 has some other terms which I've neglected to use.

I believe RFC 6550 uses term "parent" for any neighbor with a rank lower 
than the node itself and "preferred parent" for the parent (conceptually 
a single one) that the node uses as the next hop on its upward routes. 
This may be one of the sources of my confusion. After your explanation, 
I understand your point of view on the protocol. Thanks.

> A root that did this, would have to increment the DTSN right each time it

OK but:

1. This is not mentioned in the new version of the draft.

2. In non-storing mode, this would generate upward traffic from every 
member of a DODAG, which may be problematic. To explain, suppose that 
the MOP is indeed non-storing and that the root sets its min priority to 
0x7f to disable enrollments because it is unable to handle any more 
traffic. According to your suggestion, doing this, the root increments 
its DTSN. Per point 1. of the ordered list in Sect. 9.6 or RFC 6550, 
every node seeing the incremented DTSN from its parent issues a DAO 
message upwards. Per point 2. of the same list, the node also increments 
its own DTSN. In effect, every node in the DODAG sends a DAO to the 
root, which may swamp the root---a situation that the root wanted to 
avoid in the first place by disabling enrollments. This situation may be 
even worse if DAO-Acks are to be sent in reply to the DAOs.

3. Even if the this approach were covered in the draft, it still has 
some issues, which I was about to explain but the new version of the 
draft appeared.

> I think that we already have that.

I would say this need not be the best choice per:

a) my reply above

b) the fact that DTSN is conceptually for something else.

>> We think that the two values are very much related and it made sense to send

OK, now I understand. However, this also generates some further issues I 
wanted to point in my reply to Rahul's and Pascal's comments.

Best,
-- 
- Konrad Iwanicki.

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Tue, 10 Aug 2021 07:24:40 +0000

> Le 9 août 2021 à 23:10, Konrad Iwanicki <[email-masked]> a écrit :
> 
> Dear Michael and all,
> 
> First of all, thank you for your kind words regarding the review. I am actually planning to attend the August interim, because I will be presenting the rnfd draft, so we can discuss the enrollment priority draft in depth as well.
> 
> Below I just quickly reply inline to Michael's comments as they led to a new version of the draft. I was actually replying to Pascal's and Rahul's comments while the new draft version appeared, and the new version still has some issues, which I wanted to point out in those replies.
> 
> So what do I do then, write another review for the new version where I can explain those issues?
> 
> I believe RFC 6550 uses term "parent" for any neighbor with a rank lower than the node itself and "preferred parent" for the parent (conceptually a single one) that the node uses as the next hop on its upward routes. This may be one of the sources of my confusion. After your explanation, I understand your point of view on the protocol. Thanks.
> 

Actually not OK 

The DTSN triggers DAO messages; this seems unrelated and undesirable. 

Maybe the suggestion was really to resync the DAG to a new version that would operate with the new minimum?

That’s actually not a convincing idea either, since the minimum value of priority is not configured but a Dynamic computation that may change a lot mostly during the DODAG formation.

> 2. In non-storing mode, this would generate upward traffic from every member of a DODAG, which may be problematic. To explain, suppose that the MOP is indeed non-storing and that the root sets its min priority to 0x7f to disable enrollments because it is unable to handle any more traffic. According to your suggestion, doing this, the root increments its DTSN. Per point 1. of the ordered list in Sect. 9.6 or RFC 6550, every node seeing the incremented DTSN from its parent issues a DAO message upwards. Per point 2. of the same list, the node also increments its own DTSN. In effect, every node in the DODAG sends a DAO to the root, which may swamp the root---a situation that the root wanted to avoid in the first place by disabling enrollments. This situation may be even worse if DAO-Acks are to be sent in reply to the DAOs.
> 
True 
It’s not  just non storing it’s all modes

> 3. Even if the this approach were covered in the draft, it still has some issues, which I was about to explain but the new version of the draft appeared.
> 
Yes we need to sync. I think Michael still has in mind that the minimum is incremented down. That was the initial idea. 

But that’s creating more problems that help mostly due to the dodag structure and cut the capability for the root to give a direct feedback to the nodes. I now think this information should be set by the root and remain throughout the dodag.

To be discussed at the next interim…

> b) the fact that DTSN is conceptually for something else.

And my draft on versioning the configuration is also something else. We need a new sequence either in this option or global to all future status options from the root. That sequence would not alter the RPL operation as version and DTSN do.

> OK, now I understand. However, this also generates some further issues I wanted to point in my reply to Rahul's and Pascal's comments.

Looking forward. Thanks a bunch Konrad for opening the discussion on the draft. This is incredibly helpful!

You all keep safe,

Pascal

---

From: Konrad Iwanicki <[email-masked]>
Date: Tue, 10 Aug 2021 11:01:29 +0200

Hi Pascal and all,

Thanks for the comments. Now I have a full picture of how each of you 
envision the protocol. Before the interim, I will try up to write up all 
my remarks regarding these ideas and the issues I mentioned throughout 
my last reply, so that hopefully we have some starting point for discussion.

I will probably close the thread at this point and post an e-mail as a 
review of version 05.

Best,
-- 
- Konrad Iwanicki.

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Tue, 10 Aug 2021 09:23:45 +0000

Great! 

Please ask the chairs for 20-30 minutes at the interim. Since we want to ship the draft, this is probably urgent enough and interesting enough to justify aa large slot with enough discussion time to reach consensus.

Keep safe;

Pascal

> From: Konrad Iwanicki <[email-masked]>
> Sent: mardi 10 août 2021 11:01
> Cc: [email-masked]; Michael Richardson <[email-masked]>;
> Pascal Thubert (pthubert) <[email-masked]>
> my remarks regarding these ideas and the issues I mentioned throughout my
> last reply, so that hopefully we have some starting point for discussion.
> 
> presenting the rnfd draft, so we can discuss the enrollment priority draft
> in depth as well.
> >>
> >> Below I just quickly reply inline to Michael's comments as they led to
> a new version of the draft. I was actually replying to Pascal's and
> Rahul's comments while the new draft version appeared, and the new version
> still has some issues, which I wanted to point out in those replies.
> >>
> >>> Yes, it's true.  A node should evaluate DIOs from other sources, and
> >>> if those sources are better, then it should switch parents.  But at
> >>> any point, a node really has only one parent.
> >> I believe RFC 6550 uses term "parent" for any neighbor with a rank
> lower than the node itself and "preferred parent" for the parent
> (conceptually a single one) that the node uses as the next hop on its
> upward routes. This may be one of the sources of my confusion. After your
> explanation, I understand your point of view on the protocol. Thanks.
> >>
> DODAG_Size and
> >>> A root that did this, would have to
> >>> increment the DTSN right each time it changed it's mind.
> >>
> >> OK but:
> >>
> >
> > Actually not OK
> >
> > That’s actually not a convincing idea either, since the minimum value of
> priority is not configured but a Dynamic computation that may change a lot
> mostly during the DODAG formation.
> >
> >
> member of a DODAG, which may be problematic. To explain, suppose that the
> MOP is indeed non-storing and that the root sets its min priority to 0x7f
> to disable enrollments because it is unable to handle any more traffic.
> According to your suggestion, doing this, the root increments its DTSN.
> Per point 1. of the ordered list in Sect. 9.6 or RFC 6550, every node
> seeing the incremented DTSN from its parent issues a DAO message upwards.
> Per point 2. of the same list, the node also increments its own DTSN. In
> effect, every node in the DODAG sends a DAO to the root, which may swamp
> the root---a situation that the root wanted to avoid in the first place by
> disabling enrollments. This situation may be even worse if DAO-Acks are to
> be sent in reply to the DAOs.
> >>
> > True
> some issues, which I was about to explain but the new version of the draft
> appeared.
> >>
> > Yes we need to sync. I think Michael still has in mind that the minimum
> is incremented down. That was the initial idea.
> >
> > But that’s creating more problems that help mostly due to the dodag
> structure and cut the capability for the root to give a direct feedback to
> the nodes. I now think this information should be set by the root and
> remain throughout the dodag.
> >
> > And my draft on versioning the configuration is also something else. We
> need a new sequence either in this option or global to all future status
> options from the root. That sequence would not alter the RPL operation as
> version and DTSN do.
> >
> >>
> >>>> We think that the two values are very much related and it made
> >>>> sense to send them together.
> >>
> >> OK, now I understand. However, this also generates some further issues
> I wanted to point in my reply to Rahul's and Pascal's comments.
> >

---

From: Michael Richardson <[email-masked]>
Date: Tue, 10 Aug 2021 13:30:08 -0400

{I'm getting emails in funny orders because roll-chairs gets to be faster, as
I'm the secretary}

    > can explain those issues?

You can do that, or open issues at
https://github.com/roll-wg/draft-ietf-roll-enrollment-priority
(or I can do that).

(Then we can iterate on the issue, but in general I prefer using the ML)

I have opened an issue:
   https://github.com/roll-wg/draft-ietf-roll-enrollment-priority/issues/5

It might be that we need a create a ConfigurationNumber lollilop counter.
I could live with that.   I note that DIO Configuration essentially has the
same problem, right?

    > upward routes. This may be one of the sources of my confusion. After
    > your explanation, I understand your point of view on the
    > protocol. Thanks.

If you think that we should use the word "preferred parent", I'm good with
that.

The nice thing about having new eyes (like yours) on things is that after 5-6
reads of a document, one tends to think we know what it says, and often we
are subtly wrong.

    >> A root that did this, would have to increment the DTSN right each time
    >> it changed it's mind.

    > 1. This is not mentioned in the new version of the draft.

I agree that it needs to be explained.
I resist adding a new lollipop counter, because that's something new that
needs to be written to flash.

I think that I am wrong to suggest DTSN increment...  I claim heat stroke. :-)

I think that incrementing DODAGVersionNumber is the right answer.
I think that all of the Configuration work we've been discussing would also
depend upon DODAGVersionNumber, and this is just among that group of things.

    > some issues, which I was about to explain but the new version of the

Every version has a diff, so most of us deal with the diff to know if the
things we want fixed are actually fixed.

    >>> We agreed to merge two drafts.  The DODAG_Size came from the other
    >>> draft.  We think that the two values are very much related and it
    >>> made sense to send them together.

    > I wanted to point in my reply to Rahul's and Pascal's comments.

if it turns out that the two things change at significantly different rates,
then combining them may have been a poor choice.

--
]               Never tell me the odds!                 | ipv6 mesh networks [
]   Michael Richardson, Sandelman Software Works        |    IoT architect   [
]     [email-masked]  http://www.sandelman.ca/        |   ruby on rails    [

---

From: Michael Richardson <[email-masked]>
Date: Tue, 10 Aug 2021 13:37:59 -0400

> That’s actually not a convincing idea either, since the minimum value
    > of priority is not configured but a Dynamic computation that may change
    > a lot mostly during the DODAG formation.

I guess that this is more dynamic than the capabilities and the like.

First idea: it does not belong in DIO, but should be some new dynamic
configuration message.   The Capabilities do this.
But, this requires another (broadcast/multicast) TXop, and so really we
really want it as part of the DIO, I think.

One thought is that yes, it needs to be in the DIO, but that it should be a
compound object, containing the needed new lollipop counter, and then be
extensible with additional sub-TLVs.  Or maybe just fixed size TVs.

---

From: Michael Richardson <[email-masked]>
Date: Tue, 10 Aug 2021 13:42:11 -0400

> is incremented down. That was the initial idea.

So there are three possible things, I think:
1) root expresses constant to entire DODAG
   each 6LN expresses it's local increment

2) root expresses base value, 6LNs increment it

3) root expresses base value, 6LNs increment or decrement it

I can't see a situation where there is a need to decrement it.
Is there someone who has a use case that needs that?

I thought that (2) was simplest in terms of code.
A change to the minimum priority should not reset trickle or cause new DIOs
in my opinion.

    > need a new sequence either in this option or global to all future
    > status options from the root. That sequence would not alter the RPL
    > operation as version and DTSN do.

In the end, I agree.

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Tue, 10 Aug 2021 20:35:45 +0000

> Le 10 août 2021 à 19:42, Michael Richardson <[email-masked]> a écrit :
> 
> 

We’re getting there; this  is akin to but not a configuration option;  an informational option providing data that may help the network self manage…

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Tue, 10 Aug 2021 20:45:42 +0000

> I thought that (2) was simplest in terms of code.

It’s not that simple since a node cannot infer what the value really means.

With 1) plus a sequence, the value has a global meaning and the most recent value is the truth. Nodes can take a same logical decision regardless of their location in the network, eg jump onto another disaggregation to rebalance. 

With 2) different parents may give different values for the same sequence. Which should our node pick? The one from the preferred parent? Or should the node reparent to the best offer? There’s complexity and consequences to that decision of picking one value. And then no way to know the abstract desirability of the next dodag since the value from the root is lost.

---

From: Konrad Iwanicki <[email-masked]>
Date: Tue, 10 Aug 2021 23:00:58 +0200

Hi Pascal,

> With 2) different parents may give different values for the same sequence. Which should our node pick? The one from the preferred parent? Or should the node reparent to the best offer? There’s comp…

Let me reply with the issues and a possible way of addressing some of 
them in the e-mail I have already been writing for a while.

Best,
-- 
- Konrad Iwanicki.

---

From: Michael Richardson <[email-masked]>
Date: Tue, 10 Aug 2021 20:25:42 -0400

Pascal Thubert \(pthubert\) <pthubert=[email-masked]> wrote:
    > With 2) different parents may give different values for the same
    > sequence. Which should our node pick? The one from the preferred
    > parent? Or should the node reparent to the best offer? There’s
    > complexity and consequences to that decision of picking one value. And
    > then no way to know the abstract desirability of the next dodag since
    > the value from the root is lost.

A node should only care about the value from the preferred parent.
These values should never affect parent selection, but just be about other
nodes either enrolling, or joining this PANID.

So I don't understand what you write.

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Wed, 11 Aug 2021 00:51:42 +0000

Hello Michael 

I guess that’s because we have different usages in mind.

For one thing we are building a DAG bot a tree and the intention is effectively to forward via multiple parents at the same time, and receive from multiple parents as well. 

The preferred parent is useful in the Rank computation and event horizon, but it is not the only next hop.

Second is that the in non storing the child accepting a join does not create state in the parent. So even if the parent is saturated the child might be ok. So why should the parent affect the child ? And then nodes often reparent. Too late the node has children should he abandon them now that the new preferred parent does not want any?

More subtle there’s the point that the new status option should not affect routing. If the value changes the capability of this node to have children isn’t that an incentive to move?

The real bottle neck for traffic is  usually the usage of the radio to the root. If that is saturated we’ll have more complex issues than birth control. Maybe rate control. Maybe incentive to emigrate. All that is interesting but beyond this draft.

On the other hand the global knowledge of the dag status is useful not just to join a dig but also to leave it for a least loaded one.

I believe that the root perspective without a change is what keeps the draft simple… 

Take care,

Pascal

> Le 11 août 2021 à 02:26, Michael Richardson <[email-masked]> a écrit :
> 
>

---

From: Konrad Iwanicki <[email-masked]>
Date: Wed, 11 Aug 2021 23:24:36 +0200

Dear Rahul, Pascal, Michael, and all,

As promised, I am sending my remarks about the enrollment priority 
draft, version 05, so as to point out the issues I have with the draft, 
thereby providing some starting point for discussion at the August 
interim. Sorry for the text being rather lengthy, but I wanted to 
highlight all major issues that came to my mind. I am not providing any 
editorial changes, because those below are more important at this point.

PROTOCOL OPERATION WRT. FIELD MIN PRIORITY

Based on your comments on my previous review, I gather that there are 
two different ideas on how the protocol should work, clarifying which 
was also the reason for the questions I put in my previous review (of 
the draft version 04).

Unless noted otherwise, the text below refers only to the management of 
field min priority. In other words, let us completely ignore field 
DODAG_Size for a while. We will return to it later.

IDEA 1 (GLOBAL)

The first idea on how the min priority field should be managed is 
supported by Pascal, and I will refer to it as GLOBAL. Essentially, it 
works as follows:

1. The min priority values are generated solely by the DODAG root node; 
other nodes are not allowed to change these values.

2. A node extracts the value from a received option, stores it locally, 
and uses it as a base when computing the proxy priority for EBs, which 
may also account for the node's local constraints (e.g., load); however, 
per 1., only the originally received (and stored) value of min priority 
is used in the options in DIOs sent by the node.

3. An option received from ANY node can be used as the source of the min 
priority value but the receiving node has to be able to judge whether 
the received value is fresher than the one it already stores locally.

4. To this end, a lollipop sequence counter that I proposed, let's call 
it OSN, can be used. I suggested adding it to the option, but others 
suggested using some existing counters. This is immaterial for 
now---more on that later; instead, let us just assume that we have OSN 
mapped to some counter.

5. In any case, the root node bumps its OSN when it produces a new value 
of min priority. (This condition can also be relaxed but I'd rather not 
make this text longer. We can discuss this during the interim.)

6. Whenever a node receives an option with a value of OSN that is 
greater than its own local value of OSN, it adopts the received value of 
OSN and the received value of min priority as its local ones. Those 
values will then be sent in the options in DIOs transmitted by the node.

This is a really simple and robust solution. Its main advantage is that 
there are normally many paths that allow a node to learn the value of 
min priority, which alleviates the problem of nodes that do not support 
the option or nodes that experience (temporal) down times, overload, and 
the like. The problem of the different paths being asynchronous, and 
hence offering different propagation times for the values of min 
priority, is in turn solved by OSN: the counter unambiguously 
discriminates between fresher and older values. The effect is that as 
long as each node has at least one neighbor (not necessarily a parent) 
that supports the option, the min priory values will be eventually 
consistent globally.

In contrast, the main drawback of this solution is that the value of min 
priority is global. In particular, it does not take into account any 
heterogeneity in the load, available resources, etc. in different 
regions of the DODAG.

IDEA 2 (FLEXIBLE)

Michael and Rahul thus advocate an alternative idea, let's call it 
FLEXIBLE. In this approach, it is possible to adjust the value of min 
priority per node by incrementing the base value the node has originally 
received. Although the new version (05) of the draft does not describe 
this in sufficient detail, from Michael's comments I gather that the 
envisioned solution is something as follows:

1. The root sets a value of min priority in the option in its DIOs.

2. A non-root node obtains the value of its min priority only from an 
option received from its preferred parent and stores this value locally. 
It can then only increase the local value, for instance, accounting for 
its local constraints, before putting it into EBs and the options 
transmitted in its own DIOs to its children. In other words, the value 
of min priority can only grow at each hop, reflecting the local 
constraints of the nodes on the path upward to the root.

3. Since for each node, there is only one source of min priority values 
(its preferred parent), there are no conflicts. Consequently, in theory, 
we could omit OSN, as is the case in the current version of the draft. 
(This approach is wrong, as we will see shortly.)

4. To be compatible with the DIO processing rules in RFC 6550, a node 
receiving an option with min priority from any neighbor performs the 
processing according to the following rule (currently missing in the 
draft): "When processing the option, a node ignores the value of min 
priority in the option if the option does not come from the node's 
preferred parent." (This rule is again wrong, as we will see shortly.)

5. If the node's preferred parent does not support the option, the node 
assumes a default value (0x40) for its local min priority. This value, 
potentially increased to account for the node's local constraints, is 
put in the options in the DIOs transmitted by the node to its children.

The main advantage of this approach is that the min priority value can 
be adjusted in a DODAG subtree. The approach thus offers much more 
flexibility in configuring the values: an entire subtree that is 
overloaded or experiences other problems can adopt a higher min priority 
values so as to deflect enrollment traffic to other parts of the DODAG. 
A disadvantage is in turn that the approach precisely defines the path 
along which a node has to learn its value.

PROBLEMS WITH IDEA 2 (AND A POSSIBLE SOLUTION)

However, because of the existence of just a single path for propagating 
min priority value to each node, FLEXIBLE, as presented hitherto, is wrong.

To explain, an important use-case, mentioned also in the present version 
of that draft, is disabling enrollment in an entire DODAG. This is done 
by the DODAG root setting its min priority value to 0x7f. Now suppose 
the DODAG root has done this and that there exists a node, let's call it 
X, in the DODAG that does not support the option. The children of node X 
would thus adopt values 0x40+delta (depending on their local 
constraints) as their min priority values. Since 0x40+delta need not be 
equal to 0x7f, an entire subtree of node X need not disable enrollment, 
which is invalid.

In general, in FLEXIBLE, nodes that do not support the option are a 
major problem, as their ancestors disregard any decisions of upstream 
nodes, including the root. The problem can be solved as follows.

- We use multiple paths for propagating min priority information but one 
of them is preferred by each node (the one through the node's preferred 
parent).

- To ensure consistency, we do employ the aforementioned OSN. Again, for 
simplicity let's assume that it is incremented whenever the DODAG root 
produces a new value of min priority but, like in GLOBAL, this condition 
can be relaxed.

- There is no default value (0x40) a node ever adopts for its min 
priority. (Apart perhaps from the value accompanying the initial OSN value.)

- Each node locally stores:
   * the last value of OSN
   * the accompanying value of min priority
   * a bit, let's call it PB, indicating whether the above values come 
from the node's preferred parent (1) or not (0)

- In contrast, to the present version of the draft, we maintain the 
following invariant:
     "For a given OSN value, the value of min priority at any node is 
greater than or equal to the value of min priority at the root node."
   This is to ensure that no node, including the children of a node that 
does not support the option, is able to override the root's setting of 
min priority down for a given value of OSN. In particular, the invariant 
ensures that if the root disables enrollment globally, then so will all 
nodes (under the same assumptions on connectivity as in GLOBAL).

Given these points, the specific rules for processing the option are as 
follows:

1. If a node receives an option with a greater value of OSN than its 
local one, then eventually it adopts the received values of OSN and min 
priority as local ones, and sets PB appropriately (i.e., to 1 if the 
option is received from its preferred parent or 0 otherwise). It does 
not have to do this immediately. In particular, it may wait for a while 
to receive the option with the greater OSN from its preferred parent. 
Nevertheless, eventually it MUST adopt the greater OSN (and an 
accompanying min priority).

2. If a node receives an option with the same value of OSN as its local 
one then:
   a. if the option is received from its preferred parent, then it 
adopts the value of min priority from the option and sets PB to 1.
   b. otherwise,
     i. if its local PB is 1, then it ignores the received option;
     ii. if its local PB is 0, then it sets its local min priority to 
the minimum of its local min priority and the value of min priority from 
the option

3. If a node receives an option with a smaller value of OSN than its 
local one, then it ignores the received option.

4. If a node changes its preferred parent, then it sets its PB to 0.

5. If a node sends the option in its own DIO, then it puts in the option 
its local value of min priority or a greater value (e.g., taking into 
account the node's local constraints). It can never sent the option with 
a lower value of min priority than its local one.

This should do what I would expect from FLEXIBLE. However, I wrote this 
up in a haste, so it would be great if somebody verified the above 
algorithm.

As a side note, the algorithm suggests that FLEXIBLE is indeed a bit 
more involved than GLOBAL, so a decision should be made, which of them 
to put in the draft.

FURTHER ISSUES UNCOVERED IN THE DRAFT OR TO BE DISCUSSED AT THE INTERIM

Irrespective of the decision whether to settle down on GLOBAL or 
FLEXIBLE, I believe that the following issues also have to be covered in 
the draft (and discussed during the interim).

1. Should DODAG_Size be part of the option?

I mentioned this in my previous review and there has been some 
discussion already. I would just like to add one observation to that 
discussion: whereas min priority can follow either GLOBAL or FLEXIBLE, I 
believe there is an agreement that DODAG_Size should follow GLOBAL when 
it comes to propagating its value in the network. This information may 
influence our decision on which of the approaches to adopt for min 
priority and whether to put DODAG_Size in the option at all or move it 
to another one.

2. Which counter to use as OSN?

I proposed a dedicated counter, which I believe is a clean solution. 
However, there are other possibilities. I also believe that we agreed 
that DTSN from DIO base is not the best choice. In my view, nor is 
DODAGVersion but we can discuss this.

3. How do changes to min priority (and OSN) affect DIO Trickle timers?

This is currently not covered in the draft. However, I would imagine 
that some decisions regarding min priority should be propagated fast in 
a DODAG, and hence require Trickle timer resets. In particular, I would 
expect that disabling enrollment globally should be disseminated as soon 
as possible. Therefore, if I were to suggest anything, I would say that 
a change by a node of min priority to the maximal value (0x7f) from a 
lower one MUST entail a Trickle timer reset. In turn, a change from the 
maximal value to a smaller one MAY/SHOULD entail a Trickle timer reset. 
Other changes are, at least in my view, not that important, but this can 
and should be discussed.

4. How does a (temporal) lack of a preferred parent affect the proxy 
priority of the node in EBs (and in general, the node's behavior)?

This is also ignored in the draft but it may happen that a node does not 
have any preferred parent for a while. In this case, it is unable to 
handle any enrollments. What should it do? I would expect that it would 
advertise the maximal proxy priority in EBs. However, should it also 
modify its local min priority? I would probably be against.

Plus there are smaller issues I mentioned earlier, and I probably forgot 
something. In any case, sorry again for such long a text but I hope it 
may help during the interim.

Best,
-- 
- Konrad Iwanicki.

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Thu, 12 Aug 2021 09:47:16 +0000

Hello Konrad, 

This is great explanations of both options and how they could be made to work.

 What we need to add is, what do we want them for, which use case are we serving?

GLOBAL serves the need for a node to compare the load of the current DODAG with that of other DODAGs it could jump to. The use case is balancing DODAGs. 

FLEXIBLE gives an indication of load of a preferred path. What would be the use of that?

- in storing mode it could indicate the capability of creating DAO state. But you'll find looking at it that you'd need a lot more than good or bad. If a node moves to another parent and has 100 children it needs to know that the new parent and its path up to the root have 100 rooms left. OK, it would split the DAOs between 2 parents, but if the bottleneck is up there where the paths from the parents cross, it fails. Bottom line, hard problem that we avoided so far.

- in any mode, it could indicate the throughput load. But then: 
  * usually (P2MP or MP2P with homogeneous radios) that overload affects primarily the root's radio space, so there is no benefit in increasing the minimum, the root's one is the worst already. 
  * there's the need to normalize the increment, IOW to explain by what the min value should be increased by the 6LRs and when so the result down there is comparable between preferred parent chains. Like the Rank computation which imposes a single OF, you need to impose a single operation throughout. From there, all the problems of updating OFs, using multiple OFs, etc...
  * the real need would be to rebalance the tree, but rebalancing a tree in a distributed protocol is another hard problem, subject to loops

So what is FLEXIBLE for? Convince me we need it and I'll support doing both.

On the side, I completely agree with you sequence counter in any case. The counter should follow section 7 of RFC 6550, including freshness assertion, etc... in which case there's little additional to describe.

Keep safe,

Pascal

> Sent: mercredi 11 août 2021 23:25
> Subject: [Roll] Review of draft-ietf-roll-enrollment-priority-05
> 
> As promised, I am sending my remarks about the enrollment priority draft,
> version 05, so as to point out the issues I have with the draft, thereby
> providing some starting point for discussion at the August interim. Sorry
> for the text being rather lengthy, but I wanted to highlight all major
> issues that came to my mind. I am not providing any editorial changes,
> because those below are more important at this point.
> 
> 
> Based on your comments on my previous review, I gather that there are two
> different ideas on how the protocol should work, clarifying which was also
> the reason for the questions I put in my previous review (of the draft
> version 04).
> 
> and uses it as a base when computing the proxy priority for EBs, which may
> also account for the node's local constraints (e.g., load); however, per
> 1., only the originally received (and stored) value of min priority is
> used in the options in DIOs sent by the node.
> 
> priority value but the receiving node has to be able to judge whether the
> received value is fresher than the one it already stores locally.
> 
> 4. To this end, a lollipop sequence counter that I proposed, let's call it
> OSN, can be used. I suggested adding it to the option, but others
> suggested using some existing counters. This is immaterial for now---more
> on that later; instead, let us just assume that we have OSN mapped to some
> counter.
> 
> 6. Whenever a node receives an option with a value of OSN that is greater
> than its own local value of OSN, it adopts the received value of OSN and
> the received value of min priority as its local ones. Those values will
> then be sent in the options in DIOs transmitted by the node.
> 
> there are normally many paths that allow a node to learn the value of min
> priority, which alleviates the problem of nodes that do not support the
> option or nodes that experience (temporal) down times, overload, and the
> like. The problem of the different paths being asynchronous, and hence
> offering different propagation times for the values of min priority, is in
> turn solved by OSN: the counter unambiguously discriminates between
> fresher and older values. The effect is that as long as each node has at
> least one neighbor (not necessarily a parent) that supports the option,
> the min priory values will be eventually consistent globally.
> 
> heterogeneity in the load, available resources, etc. in different regions
> of the DODAG.
> 
> 
> IDEA 2 (FLEXIBLE)
> 
> transmitted in its own DIOs to its children. In other words, the value of
> min priority can only grow at each hop, reflecting the local constraints
> of the nodes on the path upward to the root.
> 
> potentially increased to account for the node's local constraints, is put
> in the options in the DIOs transmitted by the node to its children.
> 
> The main advantage of this approach is that the min priority value can be
> adjusted in a DODAG subtree. The approach thus offers much more
> of that draft, is disabling enrollment in an entire DODAG. This is done by
> the DODAG root setting its min priority value to 0x7f. Now suppose the
> DODAG root has done this and that there exists a node, let's call it X, in
> the DODAG that does not support the option. The children of node X would
> thus adopt values 0x40+delta (depending on their local
> In general, in FLEXIBLE, nodes that do not support the option are a major
> problem, as their ancestors disregard any decisions of upstream nodes,
> including the root. The problem can be solved as follows.
> 
> does not support the option, is able to override the root's setting of min
> priority down for a given value of OSN. In particular, the invariant
> 1. If a node receives an option with a greater value of OSN than its local
> one, then eventually it adopts the received values of OSN and min priority
> as local ones, and sets PB appropriately (i.e., to 1 if the option is
> received from its preferred parent or 0 otherwise). It does not have to do
> this immediately. In particular, it may wait for a while to receive the
> option with the greater OSN from its preferred parent.
>    a. if the option is received from its preferred parent, then it adopts
> the value of min priority from the option and sets PB to 1.
>    b. otherwise,
>      ii. if its local PB is 0, then it sets its local min priority to the
> minimum of its local min priority and the value of min priority from the
> option
> 
> 3. If a node receives an option with a smaller value of OSN than its local
> one, then it ignores the received option.
> 
> account the node's local constraints). It can never sent the option with a
> lower value of min priority than its local one.
> 
> This should do what I would expect from FLEXIBLE. However, I wrote this up
> in a haste, so it would be great if somebody verified the above algorithm.
> 
> As a side note, the algorithm suggests that FLEXIBLE is indeed a bit more
> involved than GLOBAL, so a decision should be made, which of them to put
> in the draft.
> 
> 
> Irrespective of the decision whether to settle down on GLOBAL or FLEXIBLE,
> I believe that the following issues also have to be covered in the draft
> (and discussed during the interim).
> 
> I mentioned this in my previous review and there has been some discussion
> already. I would just like to add one observation to that
> believe there is an agreement that DODAG_Size should follow GLOBAL when it
> comes to propagating its value in the network. This information may
> priority and whether to put DODAG_Size in the option at all or move it to
> another one.
> 
> However, there are other possibilities. I also believe that we agreed that
> DTSN from DIO base is not the best choice. In my view, nor is DODAGVersion
> but we can discuss this.
> 
> This is currently not covered in the draft. However, I would imagine that
> some decisions regarding min priority should be propagated fast in a
> DODAG, and hence require Trickle timer resets. In particular, I would
> as possible. Therefore, if I were to suggest anything, I would say that a
> change by a node of min priority to the maximal value (0x7f) from a lower
> one MUST entail a Trickle timer reset. In turn, a change from the maximal
> value to a smaller one MAY/SHOULD entail a Trickle timer reset.
> something. In any case, sorry again for such long a text but I hope it may
> help during the interim.
> 
> Best,
> --
> - Konrad Iwanicki.
>