Subject: Re: [RTG-DIR] draft-ietf-trill-rbridge-protocol-15.txt

From: Donald Eastlake <[email-masked]>
Date: Tue, 2 Mar 2010 15:40:18 -0500

Hi Acee,

On Wed, Feb 24, 2010 at 2:52 PM, Acee Lindem <[email-masked]> wrote:
> Hi Donald,
> See inline.
> On Feb 18, 2010, at 12:07 AM, Donald Eastlake wrote:
>
>> Hi Acee,
>>
>> Thanks for your detailed comments.
>>...
>>> Minor Issues:
>>
>>>  The RBridge configuration in section 5 should include the default
>>>  values and ranges of values.
>>
>> That can be done for the configuration items added by TRILL. But where
>> whole blocks of parameters are incorporated by reference (such as all
>> 802.1Q-2005 port VLAN and priority parameters), I don't see a need to
>> list in the TRILL specification all the default values and ranges
>> copied from the referenced document.

> I agree (these reference documents could simply be cited, if they
> are not already).

The changes to Section 5 are moderately extensive. A candidate Section
5 draft is attached.

>>...

> Thanks,
> Acee

Thanks,
Donald

---

From: Acee Lindem <[email-masked]>
Date: Tue, 2 Mar 2010 16:35:52 -0500

Hi Donald, 
Ok - I think you've addressed my issues. I did reread section 4.5 and I think I better understand the multi-destination frame forwarding. Rather than defining a reference protocol state, as is done in RFC 4601, this document defines the requirements and rules for forwarding frames from a given RBridge on a chosen distribution tree (both of which are carried in the TRILL header). 
Thanks,
Acee 
On Feb 26, 2010, at 1:13 AM, Donald Eastlake wrote:

>>> On Tue, Feb 16, 2010 at 7:50 PM, Acee Lindem <[email-masked]> wrote:
>>>> ...
>>> 
>>>> I think this is a very complex protocol when you consider all the
>>>> pieces and the interaction with IEEE 802.1 bridging. Hence, I think
>>>> interoperable implementations are necessary for document advancement.
>>>> Without them, I would think the intended status should be
>>>> Experimental.
>>> 
>>> ...
>>> 
>>> I am not claiming the specification is perfect. But I believe it
>>> clearly fits the criterion for Proposed Standard from RFC 2026:
>>> 
>>>  A Proposed Standard specification is generally stable, has resolved
>>>  known design choices, is believed to be well-understood, has received
>>>  significant community review, and appears to enjoy enough community
>>>  interest to be considered valuable.  However, further experience
>>>  might result in a change or even retraction of the specification
>>>  before it advances.
>>> 
>>> ...
> 
>> The ADs informed me that there is no requirement in the INT area for
>> interoperable implementations or protocol experience. Hence, I guess
>> it is a judgement call.  If not two implementations, is there one
>> righteous implementation that interoperates with itself?
> 
> There is an open source implementation for Solaris.
> 
> (By the way, according to UNH IOL, there are nine companies talking
> with them about interoperability testing. And, although there are no
> hardware TRILL products publicly announced, I was at NANOG 48 earlier
> this week and am at the Ethernet Summit right now. At both there are
> switch company booth personnel who seem quite happy to boast verbally
> about how strongly their companies support TRILL and how they are
> working on implementations.)
> 
>>> It is a Local Area Networking protocol based on a well understood
>>> and deployed routing protocol, level 1 IS-IS.
> 
>> Well, not exactly. It is ISIS but the hello protocol is modified, it
>> is being used for multicast distribution tree with new rules for tie
>> breaking, and the application supported by the L2 topology
>> information is brand new.
> 
> It is my impression that the IESG came to a general conclusion that
> the application of any extra implementation/interoperability
> requirement to the TRILL base protocol specification for Proposed
> Standard would be at the discretion of the Internet Area Directors for
> two reasons:
>    Such additional requirements are best applied to protocols or
> protocol changes that would be expected to have pervasive effects on
> the Internet, such as changes to TCP or BGP, not a local area
> networking protocol. (I'm also adding an explict statement that it is
> not a goal of TRILL to scale beyond the size of exisitng bridged LANs.)
>     There was no hint that such a requirement would be imposed in the
> TRILL Charter or history so this would be an unexpected last-minute
> requirement.
> 
>>>> There is a lengthy discussion on determining the inter-RBridge MTU
>>>> for the purposes of TRILL control packets. However, what happens
>>>> if an end-station sends a max-MTU packet and the 40 bytes of
>>>> ethernet outer header and TRILL header cause the packet to be too
>>>> large to be transported between RBridges?
>>> 
>>> Obviously, when a frame exceeds MTU on an Ethernet link, it is
>>> generally discarded. (There did not appear to be any interest in the
>>> TRILL WG in adding a link fragmentation protocol for such cases.)
>>> 
>>> The consensus of the TRILL WG is that this is not a problem as modern
>>> switches have adequate headroom for headers and encapsulation and the
>>> like. However, to assure safe operation of the TRILL Protocol in case
>>> of limited MTU and to be able to take advantage of a high campus wide
>>> inter-Rbridge link MTU, the TRILL MTU feature was added.
>>> 
>>> ... [The draft]
>>> is careful, both in the Section heading and in the text, to say that
>>> the TRILL MTU feature is for "Inter-RBridge Links". It also includes
>>> the following paragraph (where "Sz" has already been defined as the
>>> campus wide minimum inter-Rbridge link MTU):
>>> 
>>>  Sz has no direct effect on end-stations and is not directly related
>>>  to any end-station to end-station "path MTU". Methods of using Sz or
>>>  any link MTU information gathered by TRILL IS-IS in the traffic
>>>  engineering of routes or the determination of any path MTU is beyond
>>>  the scope of this document.
>>> 
>>> A sentence could be add to the above paragraph such as "Native frames
>>> that, after TRILL encapsulation, exceed the MTU of a link on which
>>> they would be sent will generally be discarded."
> 
>> Maybe this should be stated explicitly. It seems to me that any
>> draft that adds an additional level of encapsulation (outer ethernet
>> + TRILL header) should have some discussion.
> 
> I will add the above sentence in draft version -16 to make this explicit.
> 
>>> By the way, I'm not sure where you get "40 bytes" for TRILL
>>> encapsulation. It adds either 20 bytes or 24 bytes, depending on
>>> whether an outer VLAN tag is sent on the wire (unless there are
>>> options included in the TRILL Header): 12 for the outer destination
>>> and source MAC addresses, possibly 4 for the outer VLAN tag, and 8 for
>>> the TRILL header. In a fully RBridged point-to-point network there
>>> would be no reason to include such an outer VLAN tag on TRILL data or
>>> control frames sent between RBridges, so only 20 bytes would normally
>>> be added.
>>> 
>>>> In a TRILL domain with multiple distribution trees, how does an
>>>> ingress RBridge determine which distribution tree to use? I read
>>>> section 4.5 several times and it is not clear to me how the
>>>> ingress RBridge selects a distribution tree (other than everyone
>>>> using the highest priority).  Also, nickname terminology is
>>>> confusing. Is the nickname1 tree the distribution tree rooted at
>>>> RBridge 1 or is the nickname space for the distribution trees
>>>> separate?
>>> 
>>> On tree selection: Other than the TRILL requirement that the ingress
>>> RBridge announce the set of distribution trees that it might use, what
>>> does it matter? The RBridge campus will operate "correctly" regardless
>>> of what tree(s) are used by the ingress as long as it puts all the
>>> frames that are part of the same order-dependent flow on the same
>>> tree. In the absence of other considerations, it is probably best to
>>> use the tree whose root RBridge is least cost from the ingress
>>> RBridge. But, other than that, such questions as which sub-set of
>>> distribution trees to use if you are multipathing multi-destination
>>> frames seems like an excellent area for implementer creativity and
>>> beyond the scope of this document.
> 
>> I may have missed exactly how it works. So, at the ingress, the
>> frame is unicast to the root of the distribution tree selected by
>> the ingress. The root of the distribution tree will then
>> re-encapsulate it and send it on the distribution tree as a
>> multicast. Correct? If so, then I agree with your statement.
> 
> No. That's not how it works. With a bidirectional tree, an
> encapsulated packet can be injected anywhere, and travels from that
> point. So the ingress encapsulates it, and it just travels from
> there. The packet is not first unicast to the root and then
> re-encapsulated. The reasons it is good to use a tree whose root is
> least cost from the ingress RBridge are discussed in the first
> paragraph of Section 4.5.
> 
> By the way, it turns out that in Section 4.6.1.2 of the current -15
> draft it does explicitly state that, by default, an RBridge using one
> distribution tree chooses the one whose root is least cost from that
> RBridge. (While not pseudo code, section 4.6 is the most detailed and
> painstaking description in the draft of how to handle every type of
> frame it is possible for an RBridge to receive.) I will also add this
> to Section 4.5 where you were, quite reasonably, looking for it.
> 
>>> On nickname terminology: In early drafts, there was only one nickname
>>> per RBridge, so distribution trees could equally well be considered to
>>> be named by a nickname or uniquely determined by their root
>>> RBridge. More recently the option of multiple nicknames per RBridge
>>> was added and now distribution trees are uniquely labeled only by
>>> their nickname. Nicknames are also permitted for pseudo-nodes. There
>>> is only one nickname allocation space. Every nickname in use in a
>>> campus designates either an RBridge or a pseudo-node and one or more
>>> nicknames also each designate a distribution tree.
>>> 
>>> With this explanation, do you think that the draft text still needs
>>> clarification and if so do you have any suggestions?
> 
>> Maybe section 3.7 could explicitly state that the distribution tree
>> nicknames and the RBridge nicknames are independent of one
>> another. Or if they are not, precisely explain the correlation.
> 
> OK, how about adding the following at the end of 3.7.3:
> 
>   Every nickname in use in a campus identifies an RBridge (or
>   pseudonode) and every nickname designates a distribution tree
>   rooted at the RBridge (or pseudonode) it identifies. However, only
>   a limited number of these potential distribution trees are actually
>   computed by all the RBridge in a campus and they all compute the
>   same set of distribution trees, as discussed in Section 4.5.
> 
>>>> The document should explain the philosophy and usage of confidence
>>>> level with respect to the {port, VLAN, MAC address} tuples. This
>>>> discussion should also tie in the confidence configuration in
>>>> section 5.1.
>>> 
>>> Well, how about something like the following new text:
>>> 
>>>  The confidence level mechanism allows an RBridge campus manager to
>>>  cause certain address learning sources to prevail over others. In a
>>>  default configuration, without the optional ESADI protocol,
>>>  addresses are only learned from observing local native frames and
>>>  the decapsulation of received TRILL data frames. Both of these
>>>  sources default to confidence level 0x20 so, since learning at an
>>>  equal or high confidence overrides previous learning, the learning
>>>  in such a default case mimics default 802.1 bridge learning.
>>> 
>>>  While RBridge campus management policies are beyond the scope of
>>>  this document, here are some example types of policies that can be
>>>  implemented with the confidence mechanism and the rational for
>>>  each:
>>> 
>>>  o  Locally received native frames might be considered more reliable
>>>     than decapsulated frames received from remote parts of the
>>>     campus. To stop MAC addresses learned from such local frames
>>>     from being usurped by remotely received forged frames, the
>>>     confidence in locally learned addresses could be increased or
>>>     that in addresses learned from remotely sourced decapsulated
>>>     frames decreased.
>>> 
>>>  o  MAC address information learned through a cryptographically
>>>     authenticated Layer 2 registration protocol, such as 802.1X with
>>>     a cryptographically based EAP method, might be considered more
>>>     reliable than information learned through the mere observation
>>>     of data frames. When such authenticated learned address
>>>     information is transmitted via the ESADI protocol, the use of
>>>     authentication in the TRILL EASDI LSP frames could make
>>>     tampering with it in transit very difficult. As a result, it
>>>     might be reasonable to announce such authenticated information
>>>     via the ESADI protocol with a high confidence, so it would
>>>     override any alternative learning from data observation.
>>> 
>>>  Manually configured address information is generally considered
>>>  static and so defaults to a confidence of 0xFF while no other
>>>  source of such information can be configured to a confidence any
>>>  higher than 0xFE. However, for other cases, such as where the
>>>  manual configuration is just a starting point which the Rbridge
>>>  campus manager wishes to be dynamically overrideable, the
>>>  confidence of such manually configured information may be
>>>  configured to a lower value.
> 
>> This would be good. In the last sentence, do you mean the MAC
>> addresses are dynamically learned or changing the configuration of
>> the confidence level? I suspect the former.
> 
> Yes. I will replace "the manual configuration is" with "the manually
> configred MAC addresses are".
> 
> Thanks,
> Donald
> 
>>>> ...
> 
>> Thanks,
>> Acee
>> 
>> ...
>>> 
>>> Thanks,
>>> Donald [document Editor]
>>> =============================
>>> Donald E. Eastlake 3rd   +1-508-333-2270 (cell)
>>> 155 Beaver Street
>>> Milford, MA 01757 USA
>>> [email-masked]

---

From: Donald Eastlake <[email-masked]>
Date: Tue, 2 Mar 2010 16:50:10 -0500

> Ok - I think you've addressed my issues. I did reread section 4.5 and I think I better understand the multi-destination frame forwarding. Rather than defining a reference protocol state, as is done…

Yes. Thanks,
Donald
=============================
 Donald E. Eastlake 3rd   +1-508-333-2270 (cell)
 155 Beaver Street   +1-508-634-2066 (home)
 Milford, MA 01757 USA
 [email-masked]

---

From: Donald Eastlake <[email-masked]>
Date: Wed, 3 Mar 2010 12:53:09 -0500

Hi Stewart,

>> (I'm also adding an explict statement that it is not a goal of
>> TRILL to scale beyond the size of exisitng bridged LANs.)

> When you say add a statement, are you thinking about a few words
> buried deep in the fine print, or a headline statement in capitals
> at the front of the form "THIS VERSION OF THE TRILL SPECIFICATION IS
> NOT CERTIFIED FOR DEPLOYMENT IN LARGE SCALE WIDE AREA NETWORKS!"

On "a few words" I was thinking of a sentence, such as the following:

"The TRILL protocol, as specified herein, is designed to be a Local
Area Network protocol and not designed with the goal of scaling beyond
the size of existing local bridged LANs."

On the location of this sentence, a logical place seems to be in the
first part of Section 1 just before the sentence that references RFC
5556 ("Transparent Interconnection of Lots of Links (TRILL): Problem
and Applicability Statement") but I'm open to other suggestions.

On the suggested wording re "certified", I am not a lawyer but it is
my understand that stating something is certified for some purpose
creates legal liability for failure to perform properly for that
purpose. And that for this reason the IETF does not "certify" any of
its protocols for any purpose whatsoever.

> The concern is that operators may use Trill for hollow core, in
> which case a protocol bug could bring down an ISP in a way that
> requires rebooting of their entire net. The intent of the warning is
> to make sure operators do adequate whitebox testing before they
> attempt any such deployment, and as such it needs to be suitably
> visible.

To the extent that such a possible problem can be mitigated by
testing, this seems like a misplaced concern. While users should be
encouraged to do testing, for the most part they depend upon the
testing done and the assurances given by equipement vendors. I do not
think that, at this stage in the development of computer networks,
significant vendors implement and ship new protocols without testing
and that users such as ISPs only purchase from vendors which do such
testing. I believe this is the subtext of RFC 4794. Presumably this is
among the reasons that nine vendors have contacted the University of
New Hamshire Interoperability Laboratory with a view to arranging and
participating in TRILL testing. Such testing is, of course, not an
IETF activity.

> - Stewart

Thanks,
Donald (document editor)

--