Subject: [RTG-DIR] draft-ietf-trill-rbridge-protocol-15.txt

From: Acee Lindem <[email-masked]>
Date: Tue, 16 Feb 2010 19:50:59 -0500

Hello,

I have been selected as the Routing Directorate reviewer for this draft. The Routing Directorate seeks to review all routing or routing-related drafts as they pass through IETF last call and IESG review. The purpose of the review is to provide assistance to the Routing ADs. For more information about the Routing Directorate, please see http://www.ietf.org/iesg/directorate/routing.html

Although these comments are primarily for the use of the Routing ADs, it would be helpful if you could consider them along with any other IETF Last Call comments that you receive, and strive to resolve them through discussion or by updating the draft.

Document: draft-ietf-trill-rbridge-protocol-15.txt
Reviewer: Acee Lindem
Review Date: 2010-02-16
IETF LC End Date:  TBD
Intended Status: Proposed Standard

Summary:

Given the complexity of the protocol, I believe interoperable implementations are the imperative.  Also, there are some areas which I believe should be addressed in the document.

Major Issues:

   I think this is a very complex protocol when you consider all the
   pieces and the interaction with IEEE 802.1 bridging. Hence, I think
   interoperable implementations are necessary for document advancement.
   Without them, I would think the intended status should be Experimental.

   There is a lengthy discussion on determining the inter-RBridge MTU for
   the purposes of TRILL control packets. However, what happens an end-station
   sends a max-MTU packet and the the 40 bytes of ethernet outer header
   and TRILL header cause the packet to be too large to be transported between
   RBridges?

   In a TRILL domain with multiple distribution trees, how does an ingress
   RBridge determine which distribution tree to use? I read section 4.5
   several times and it is not clear to me how the ingress RBridge selects
   a distribution tree (other than everyone using the highest priority).
   Also, nickname terminology is confusing. Is the nickname1 tree the
   distribution tree rooted at RBridge 1 or is the nickname space for
   the distribution trees separate?

   The document should explain the philosophy and usage of confidence
   level with respect to the {port, VLAN, MAC address} tuples. This
   discussion should also tie in the confidence configuration in section
   5.1.

Minor Issues:

   The RBridge configuration in section 5 should include the default values
   and ranges of values.

   The use of ESADI is inconsistent. In some cases, it is used as one would
   expect in that it refers to a piece of End Station Address Distribution
   Information. However, in other cases it is used to refer to the protocol
   exchange of ESADI (e.g., see the second paragraph of page 14). This is
   very confusing.

   Also inconsistent is the usage of "RBridge" and "Rbridge".  Given the
   former is the title of the protocol, I'm assuming all instances should
   be "RBridge".

Nits:

   Page 18, Section 2.5.1, 1st paragraph, replace "In that case," with
   "In this case,".

   Page 23, Section 3.4, replace "a known unicast TRILL data frame" with
   "a known unicast MAC address".

   Page 26, Section 3.7.3, 1st bullent, shouldn't this contain a reference
   to "[layer2]"?

   Page 28, Section 3.7.3, second bullet, restructure the sentence
   beginning "Because of the ... " for readability.

   Page 44, Section 4.2.6, first full paragraph, replace "RBridges ," with
   "RBridges," and "next  hop" with "next hop".

   Page 46, Section 4.4.1, third paragraph, replace "completely independently"
   with "completely independent".

   Page 48, Section 4.4.2, first full paragraph, capitalize or somehow denote
   the "Appointed-Forwarders TLV" and possibly reference "[layer]".

   Page 52, Section 4.4.5, first paragraph, replace "permitted to a" with
   "permitted on a".

   Page 59, Section 4.5.5, first bullet, replace "nickanme1" with "nickname1".

   Page 67, Section 4.8, first sentence, can this be restructured to not be
   one continuous stream of consciousness?

   Page 70, Section 4.8.3, first bullet, replace "who's" with "whose" consistent
   with this usage.

   Page 72, Section 4.9.1, first full paragraph, replace
   "to the left in the this list" with "to the left of the other bit in this list".

   Page 75. Section 4.9.3.2, second paragraph, replace "in the first set" with
   "in the lower priority set".

   Page 77, Section 5.1, replace "number of distribution tree" with "number of
   distribution trees".

   Page 78, Section 5.2, fifth bullet, replace "it disable learning" with "it
   disables learning".

Thanks,
Acee

---

From: Donald Eastlake <[email-masked]>
Date: Thu, 18 Feb 2010 00:07:34 -0500

Hi Acee,

Thanks for your detailed comments.

On Tue, Feb 16, 2010 at 7:50 PM, Acee Lindem <[email-masked]> wrote:
> Hello,
>
> I have been selected as the Routing Directorate reviewer for this
> draft. The Routing Directorate seeks to review all routing or
> routing-related drafts as they pass through IETF last call and IESG
> review. The purpose of the review is to provide assistance to the
> Routing ADs. For more information about the Routing Directorate,
> please see http://www.ietf.org/iesg/directorate/routing.html

> Although these comments are primarily for the use of the Routing
> ADs, it would be helpful if you could consider them along with any
> other IETF Last Call comments that you receive, and strive to
> resolve them through discussion or by updating the draft.

> Intended Status: Proposed Standard

The IETF Last Call End Date was January 11th, 2010.

> Given the complexity of the protocol, I believe interoperable
> implementations are the imperative.  Also, there are some areas
> which I believe should be addressed in the document.

See responses below.

>   Without them, I would think the intended status should be

I see no justification for this. As far as I am aware, there is no
four step standardization system (Experimental -> Proposed -> Draft ->
Full) in the IETF for "complex" protocols, as opposed to a three step
standardization system for "simple" protocols.

I am not claiming the specification is perfect. But I believe it
clearly fits the criterion for Proposed Standard from RFC 2026:

   A Proposed Standard specification is generally stable, has resolved
   known design choices, is believed to be well-understood, has received
   significant community review, and appears to enjoy enough community
   interest to be considered valuable.  However, further experience
   might result in a change or even retraction of the specification
   before it advances.

It is also true that RFC 2026 contains the following:

   The IESG may require implementation and/or operational experience
   prior to granting Proposed Standard status to a specification that
   materially affects the core Internet protocols or that specifies
   behavior that may have significant operational impact on the
   Internet.

Based on the above paragraph, there was a policy requiring "One or
more implementations" of their major features for core Internet
routing protocols prior to Proposed Standard [RFC1264]. However, (1)
that policy is no longer generally in effect [RFC4794] and (2) TRILL
is not a core Internet routing protocol. It is a Local Area Networking
protocol based on a well understood and deployed routing protocol,
level 1 IS-IS.

>   There is a lengthy discussion on determining the inter-RBridge MTU
>   for the purposes of TRILL control packets. However, what happens
>   if an end-station sends a max-MTU packet and the 40 bytes of
>   ethernet outer header and TRILL header cause the packet to be too
>   large to be transported between RBridges?

Obviously, when a frame exceeds MTU on an Ethernet link, it is
generally discarded. (There did not appear to be any interest in the
TRILL WG in adding a link fragmentation protocol for such cases.)

The consensus of the TRILL WG is that this is not a problem as modern
switches have adequate headroom for headers and encapsulation and the
like. However, to assure safe operation of the TRILL Protocol in case
of limited MTU and to be able to take advantage of a high campus wide
inter-Rbridge link MTU, the TRILL MTU feature was added.

I thought the draft was clear on the applicability of this feature. It
is careful, both in the Section heading and in the text, to say that
the TRILL MTU feature is for "Inter-RBridge Links". It also includes
the following paragraph (where "Sz" has already been defined as the
campus wide minimum inter-Rbridge link MTU):

   Sz has no direct effect on end-stations and is not directly related
   to any end-station to end-station "path MTU". Methods of using Sz or
   any link MTU information gathered by TRILL IS-IS in the traffic
   engineering of routes or the determination of any path MTU is beyond
   the scope of this document.

A sentence could be add to the above paragraph such as "Native frames
that, after TRILL encapsulation, exceed the MTU of a link on which
they would be sent will generally be discarded."

By the way, I'm not sure where you get "40 bytes" for TRILL
encapsulation. It adds either 20 bytes or 24 bytes, depending on
whether an outer VLAN tag is sent on the wire (unless there are
options included in the TRILL Header): 12 for the outer destination
and source MAC addresses, possibly 4 for the outer VLAN tag, and 8 for
the TRILL header. In a fully RBridged point-to-point network there
would be no reason to include such an outer VLAN tag on TRILL data or
control frames sent between RBridges, so only 20 bytes would normally
be added.

>   In a TRILL domain with multiple distribution trees, how does an
>   ingress RBridge determine which distribution tree to use? I read
>   section 4.5 several times and it is not clear to me how the
>   ingress RBridge selects a distribution tree (other than everyone
>   using the highest priority).  Also, nickname terminology is
>   confusing. Is the nickname1 tree the distribution tree rooted at
>   RBridge 1 or is the nickname space for the distribution trees
>   separate?

On tree selection: Other than the TRILL requirement that the ingress
RBridge announce the set of distribution trees that it might use, what
does it matter? The RBridge campus will operate "correctly" regardless
of what tree(s) are used by the ingress as long as it puts all the
frames that are part of the same order-dependent flow on the same
tree. In the absence of other considerations, it is probably best to
use the tree whose root RBridge is least cost from the ingress
RBridge. But, other than that, such questions as which sub-set of
distribution trees to use if you are multipathing multi-destination
frames seems like an excellent area for implementer creativity and
beyond the scope of this document.

On nickname terminology: In early drafts, there was only one nickname
per RBridge, so distribution trees could equally well be considered to
be named by a nickname or uniquely determined by their root
RBridge. More recently the option of multiple nicknames per RBridge
was added and now distribution trees are uniquely labeled only by
their nickname. Nicknames are also permitted for pseudo-nodes. There
is only one nickname allocation space. Every nickname in use in a
campus designates either an RBridge or a pseudo-node and one or more
nicknames also each designate a distribution tree.

With this explanation, do you think that the draft text still needs
clarification and if so do you have any suggestions?
	
>   discussion should also tie in the confidence configuration in
>   section 5.1.

Well, how about something like the following new text:

   The confidence level mechanism allows an RBridge campus manager to
   cause certain address learning sources to prevail over others. In a
   default configuration, without the optional ESADI protocol,
   addresses are only learned from observing local native frames and
   the decapsulation of received TRILL data frames. Both of these
   sources default to confidence level 0x20 so, since learning at an
   equal or high confidence overrides previous learning, the learning
   in such a default case mimics default 802.1 bridge learning.

   While RBridge campus management policies are beyond the scope of
   this document, here are some example types of policies that can be
   implemented with the confidence mechanism and the rational for
   each:

   o  Locally received native frames might be considered more reliable
      than decapsulated frames received from remote parts of the
      campus. To stop MAC addresses learned from such local frames
      from being usurped by remotely received forged frames, the
      confidence in locally learned addresses could be increased or
      that in addresses learned from remotely sourced decapsulated
      frames decreased.

   o  MAC address information learned through a cryptographically
      authenticated Layer 2 registration protocol, such as 802.1X with
      a cryptographically based EAP method, might be considered more
      reliable than information learned through the mere observation
      of data frames. When such authenticated learned address
      information is transmitted via the ESADI protocol, the use of
      authentication in the TRILL EASDI LSP frames could make
      tampering with it in transit very difficult. As a result, it
      might be reasonable to announce such authenticated information
      via the ESADI protocol with a high confidence, so it would
      override any alternative learning from data observation.

   Manually configured address information is generally considered
   static and so defaults to a confidence of 0xFF while no other
   source of such information can be configured to a confidence any
   higher than 0xFE. However, for other cases, such as where the
   manual configuration is just a starting point which the Rbridge
   campus manager wishes to be dynamically overrideable, the
   confidence of such manually configured information may be
   configured to a lower value.

>   The RBridge configuration in section 5 should include the default
>   values and ranges of values.

That can be done for the configuration items added by TRILL. But where
whole blocks of parameters are incorporated by reference (such as all
802.1Q-2005 port VLAN and priority parameters), I don't see a need to
list in the TRILL specification all the default values and ranges
copied from the referenced document.

>   The use of ESADI is inconsistent. In some cases, it is used as one
>   would expect in that it refers to a piece of End Station Address
>   Distribution Information. However, in other cases it is used to
>   refer to the protocol exchange of ESADI (e.g., see the second
>   paragraph of page 14). This is very confusing.

Is this all that unusual? "LSP" is sometimes used to mean a protocol
message and sometimes to mean the data in an LSP. But it would be
reasonable to do a review and replace "ESDAI" with "ESADI data" or
"ESADI protocol" or the like, where appropriate for clarity.

>   Also inconsistent is the usage of "RBridge" and "Rbridge".  Given
>   the former is the title of the protocol, I'm assuming all
>   instances should be "RBridge".

Well, the draft states the following in Section 1.3: "The second
letter in Rbridge is case insensitive. Both Rbridge and RBridge are
correct."  :-)

Interestingly enough, I don't recall any comment like yours being
posted to the TRILL WG mailing list. Nor was this mentioned by any of
the previous Independent Expert, Directorate, or other reviews,
including not being mentioned in the GenART review. Nor has it been
mentioned by any AD. Under the circumstances I'd be inclined to leave
it as is.

> Nits:

Thanks for all the corrections and suggestions for improvement
below. I'd be happy to make changes based on them.

>   Page 18, Section 2.5.1, 1st paragraph, replace "In that case,"
>   with "In this case,".

I actually think "that" reads better than "this". But how about
replacing "In that case" with "For such configurations".

>   Page 23, Section 3.4, replace "a known unicast TRILL data frame"
>   with "a known unicast MAC address".

OK.

>   Page 26, Section 3.7.3, 1st bullet, shouldn't this contain a
>   reference to "[layer2]"?

OK.

>   beginning "Because of the ... " for readability.

OK. Can split into two sentences.

>   Page 44, Section 4.2.6, first full paragraph, replace "RBridges ,"
>   with "RBridges," and "next  hop" with "next hop".

OK.

>   with "completely independent".

I think it would be better to replaced "completely independently of
whether they are padded or not" with "even though they are not
padded".

>   Page 48, Section 4.4.2, first full paragraph, capitalize or
>   somehow denote the "Appointed-Forwarders TLV" and possibly
>   reference "[layer2]".

OK to both changes.

>   Page 52, Section 4.4.5, first paragraph, replace "permitted to a"
>   with "permitted on a".

OK.

>   Page 59, Section 4.5.5, first bullet, replace "nickanme1" with

OK.

>   Page 67, Section 4.8, first sentence, can this be restructured to
>   not be one continuous stream of consciousness?

Sure. How about changing it to start as follows:

   RBridges have to learn the MAC addresses and VLANs of their locally
   attached end stations for link/VLAN pairs for which they are the
   appointed forwarders. Learning this enables them to do the
   following:

And then convert the two main bullet points into separate sentences.

>   Page 70, Section 4.8.3, first bullet, replace "who's" with "whose"
>   consistent with this usage.

OK.

>   Page 72, Section 4.9.1, first full paragraph, replace "to the left
>   in the this list" with "to the left of the other bit in this
>   list".

OK.

>   Page 75. Section 4.9.3.2, second paragraph, replace "in the first
>   set" with "in the lower priority set".

OK, that change does make it clearer.

>   Page 77, Section 5.1, replace "number of distribution tree" with
>   "number of distribution trees".

OK.

>   Page 78, Section 5.2, fifth bullet, replace "it disable learning"
>   with "it disables learning".

OK.

> Acee

Thanks,
Donald [document Editor]
=============================
 Donald E. Eastlake 3rd   +1-508-333-2270 (cell)
 155 Beaver Street
 Milford, MA 01757 USA
 [email-masked]

---

From: Acee Lindem <[email-masked]>
Date: Wed, 24 Feb 2010 14:52:10 -0500

Hi Donald,
See inline.
On Feb 18, 2010, at 12:07 AM, Donald Eastlake wrote:

> is not a core Internet routing protocol.

The ADs informed me that there is no requirement in the INT area for interoperable implementations or protocol experience. Hence, I guess it is a judgement call.  If not two implementations, is there one righteous implementation that interoperates with itself?

> It is a Local Area Networking

Well, not exactly. It is ISIS but the hello protocol is modified, it is being used for multicast distribution tree with new rules for tie breaking, and the application supported by the L2 topology information is brand new.

> they would be sent will generally be discarded."

Maybe this should be stated explicitly. It seems to me that any draft that adds an additional level of encapsulation (outer ethernet + TRILL header) should have some discussion.

> beyond the scope of this document.

I may have missed exactly how it works. So, at the ingress, the frame is unicast to the root of the distribution tree selected by the ingress. The root of the distribution tree will then re-encapsulate it and send  it on the distribution tree as a multicast. Correct? If so, then I agree with your statement.

> clarification and if so do you have any suggestions?

Maybe section 3.7 could explicitly state that the distribution tree nicknames and the RBridge nicknames are independent of one another. Or if they are not, precisely explain the correlation.

>   configured to a lower value.

This would be good. In the last sentence, do you mean the MAC addresses are dynamically learned or changing the configuration of the confidence level? I suspect the former.

> copied from the referenced document.

I agree (these reference documents could simply be cited, if they are not already).

> "ESADI protocol" or the like, where appropriate for clarity.

I support this change.

> mentioned by any AD. Under the circumstances I'd be inclined to leave

As a point of reference, I believe IPsec standard started out allowing multiple permutations of uppercase and lowercase. However, in RFC 4301, they settled on "IPsec". Here is an excerpt:

   The spelling "IPsec" is preferred and used throughout this and all
   related IPsec standards.  All other capitalizations of IPsec (e.g.,
   IPSEC, IPSec, ipsec) are deprecated.  However, any capitalization of
   the sequence of letters "IPsec" should be understood to refer to the
   IPsec protocols.

Thanks,
Acee

---

From: Donald Eastlake <[email-masked]>
Date: Fri, 26 Feb 2010 01:13:37 -0500

Hi Acee,

On Wed, Feb 24, 2010 at 2:52 PM, Acee Lindem <[email-masked]> wrote:
>>> ...
>> ...
>> ...

> The ADs informed me that there is no requirement in the INT area for
> interoperable implementations or protocol experience. Hence, I guess
> it is a judgement call.  If not two implementations, is there one
> righteous implementation that interoperates with itself?

There is an open source implementation for Solaris.

(By the way, according to UNH IOL, there are nine companies talking
with them about interoperability testing. And, although there are no
hardware TRILL products publicly announced, I was at NANOG 48 earlier
this week and am at the Ethernet Summit right now. At both there are
switch company booth personnel who seem quite happy to boast verbally
about how strongly their companies support TRILL and how they are
working on implementations.)

>> It is a Local Area Networking protocol based on a well understood
>> and deployed routing protocol, level 1 IS-IS.

> Well, not exactly. It is ISIS but the hello protocol is modified, it
> is being used for multicast distribution tree with new rules for tie
> breaking, and the application supported by the L2 topology
> information is brand new.

It is my impression that the IESG came to a general conclusion that
the application of any extra implementation/interoperability
requirement to the TRILL base protocol specification for Proposed
Standard would be at the discretion of the Internet Area Directors for
two reasons:
    Such additional requirements are best applied to protocols or
protocol changes that would be expected to have pervasive effects on
the Internet, such as changes to TCP or BGP, not a local area
networking protocol. (I'm also adding an explict statement that it is
not a goal of TRILL to scale beyond the size of exisitng bridged LANs.)
     There was no hint that such a requirement would be imposed in the
TRILL Charter or history so this would be an unexpected last-minute
requirement.

>> ... [The draft]

> Maybe this should be stated explicitly. It seems to me that any
> draft that adds an additional level of encapsulation (outer ethernet
> + TRILL header) should have some discussion.

I will add the above sentence in draft version -16 to make this explicit.

> I may have missed exactly how it works. So, at the ingress, the
> frame is unicast to the root of the distribution tree selected by
> the ingress. The root of the distribution tree will then
> re-encapsulate it and send it on the distribution tree as a
> multicast. Correct? If so, then I agree with your statement.

No. That's not how it works. With a bidirectional tree, an
encapsulated packet can be injected anywhere, and travels from that
point. So the ingress encapsulates it, and it just travels from
there. The packet is not first unicast to the root and then
re-encapsulated. The reasons it is good to use a tree whose root is
least cost from the ingress RBridge are discussed in the first
paragraph of Section 4.5.

By the way, it turns out that in Section 4.6.1.2 of the current -15
draft it does explicitly state that, by default, an RBridge using one
distribution tree chooses the one whose root is least cost from that
RBridge. (While not pseudo code, section 4.6 is the most detailed and
painstaking description in the draft of how to handle every type of
frame it is possible for an RBridge to receive.) I will also add this
to Section 4.5 where you were, quite reasonably, looking for it.

> Maybe section 3.7 could explicitly state that the distribution tree
> nicknames and the RBridge nicknames are independent of one
> another. Or if they are not, precisely explain the correlation.

OK, how about adding the following at the end of 3.7.3:

   Every nickname in use in a campus identifies an RBridge (or
   pseudonode) and every nickname designates a distribution tree
   rooted at the RBridge (or pseudonode) it identifies. However, only
   a limited number of these potential distribution trees are actually
   computed by all the RBridge in a campus and they all compute the
   same set of distribution trees, as discussed in Section 4.5.

> This would be good. In the last sentence, do you mean the MAC
> addresses are dynamically learned or changing the configuration of
> the confidence level? I suspect the former.

Yes. I will replace "the manual configuration is" with "the manually
configred MAC addresses are".

Thanks,
Donald

>>>...

>...

---

From: Stewart Bryant <[email-masked]>
Date: Fri, 26 Feb 2010 10:16:05 +0000

Donald
> (I'm also adding an explict statement that it is
When you say add a statement, are you thinking about a few words buried 
deep  in the fine print, or a headline statement in capitals at the 
front  of the form "THIS VERSION OF THE TRILL SPECIFICATION IS NOT 
CERTIFIED FOR DEPLOYMENT IN LARGE SCALE WIDE AREA NETWORKS!"

The concern is that operators may use Trill  for hollow core, in which 
case a protocol bug could bring down an ISP in a way that requires 
rebooting of their entire net. The intent of the warning is to make sure 
operators do adequate whitebox  testing before they attempt any such 
deployment, and as such it needs to be suitably visible.

- Stewart