Subject: Re: [bess] shepherd review of draft-ietf-bess-evpn-etree

From: Thomas Morin <[email-masked]>
Date: Thu, 5 Jan 2017 10:54:14 +0100

Hi Ali,

2016-12-19, Ali Sajassi (sajassi):
> I have modified section 2.2 (copied below) to elaborate why coloring 
> approach for Leaf/Root MAC addresses is used in this draft. Also, the 
> use of single RT for this scenario is mentioned just as “MAY”. Please 
> review the text below and let me know if you still have 
> questions/comments:

Thanks for providing text that goes in the right direction.
I still have a few comments below.

-Thomas

>
> 2.2 Scenario 2: Leaf OR Root site(s) per AC
>
>    In this scenario, a PE receives traffic from either Root OR Leaf
>    sites (but not both) on a given Attachment Circuit (AC) of an EVI. In
>    other words, an AC (ES or ES/VLAN) is either a Root AC or a Leaf AC
>    (but not both).
>
>
>                      +---------+            +---------+
>                      |   PE1   |            |   PE2   |
>     +---+            |  +---+  |  +------+  |  +---+  |            +---+
>     |CE1+-----ES1----+--+   |  |  |      |  |  | +--+---ES2/AC1--+CE2|
>     +---+    (Leaf)  |  |MAC|  |  | MPLS |  |  |MAC|  |   (Leaf)   +---+
>                      |  |VRF|  |  |  /IP |  |  |VRF|  |
>                      |  |   |  |  |      |  |  |   |  |            +---+
>                      |  |   |  |  |      |  |  | +--+---ES2/AC2--+CE3|
>                      |  +---+  |  +------+  |  +---+  |   (Root)   +---+
>                      +---------+            +---------+
>
>    Figure 2: Scenario 2
>
>    In this scenario, just like the previous scenario (in section 2.1),
>    two Route Targets (one for Root and another for Leaf) can be used.
>    However, the difference is that on a PE with both Root and Leaf ACs,
>    all remote MAC routes are imported and thus there needs to be a way
>    to differentiate remote MAC routes associated with Leaf ACs versus
>    the ones associated with Root ACs in order to apply the proper
>    ingress filtering.
>
>    In order to support such ingress filtering on the ingress PE with
>    both Leaf and Root ACs, one the following two approaches can be used:

reverting A and B would be more natural since solution B corresponds to 
the starting point "what we had before this spec"

>
>    A) Color MAC addresses with Leaf (or Root) color before distributing
>    them in BGP to other PEs depending on whether it is learned on a Leaf

s/it is/they are/

>    AC (or a Root AC)

I think removing the parenthesis is needed for the 'whether' statement 
to parse.

>
>    B) Use two MAC-VRFs (two bridge tables per VLANs) - one for Root ACs
>    and another for Leaf ACs.

I think "(two bridge tables per VLANs)" is inexact:  "two bridge tables 
per VLAN if a given VLAN exists on the PE for both Leaf and Root ACs of 
a given EVI" ?

Similarly, in the following paragraph, I think "per VLAN" should be 
replaced by "per E-TREE EVI having both Root and Leaf ACs".

>
>    Maintaining two bridge tables per VLAN requires either two lookups be
>    performed per MAC address in either direction in case of a miss, or
>    duplicating many MAC addresses between the two bridge tables
>    belonging to the same VLAN (same E-TREE instance). The duplication of
>    MAC addresses are need for both locally learned and remotely learned
>    MAC addresses.

Since it is said above "Maintaining two bridge tables per VLAN requires 
either two lookups [...] or duplicating many MAC addresses [...]", 
saying "The duplication of MAC addresses is needed for [...]"  is 
surprising, so I guess the intent is rather "Unless two lookups are 
made, duplication of MAC addresses would be needed for [...]".

>    Locally learned MAC addresses from Leaf ACs need to be
>    duplicated onto Root bridge table and locally learned MAC addresses
>    from Root ACs need to be duplicated onto Leaf bridge table. Remotely
>    learned MAC addresses from Root ACs need to be copied onto both Root
>    and Leaf bridge tables.
>    Neither double lookups nor MAC duplications
>    are considered viable options; therefore, this draft recommends the
>    use of MAC address coloring for this scenario as detailed in section
>    3.1.

I think that this explanation is too elliptic compared to the strong 
("not viable") conclusion. As soon as we talk about implementation 
details, a more detailed discussion is required on why, and under which 
assumptions, some things are impossible  -- there can be many different 
way to implement a dataplane. Without explaining what "two lookups" 
exactly means in this context, it's hard to follow why it would be 
required if duplicating MAC addresses is not done, and why it is latter 
concluded as "not viable":
- doing multiple lookups is something that is far from being uncommon on 
router platforms
- on software platforms the impact of doing multiple lookups can be 
reduced to mostly zero (e.g. with OpenVSwitch that would only impact the 
first packets of a flow)
- if the dataplane can leverage the colouring information to avoid doing 
two lookups, then perhaps this hardware ability can be leveraged to 
support the two MAC-VRFs approach with only one lookup (building one 
table, marking MAC entries as Leaf entries if they were learned with 
routes carrying only the Leaf RT?)  --- don't misunderstand me: I'm not 
claiming that this works (I haven't looked closely enough), but simply 
that the text provided is not sufficient to exclude this kind of solution

The "duplicating MAC addresses" alternative is explained better, but 
still, nothing is explained on why this is "not viable". It seems to be 
as something rather belonging to the realm of "having a scalability 
impact", but even looking in this respect we are not talking about a 
change of order of magnitude.

>
>    For this scenario, if for a given EVI, the vast majority of PEs will
>    have both Leaf and Root sites attached, even though they may start as
>    Root-only or Leaf-only PEs, then a single RT per EVI MAY be used in
>    order to alleviate  additional configuration overhead associated with

"to alleviate  additional configuration overhead associated with ..." -> 
"to alleviate the configuration overhead associated with ..." ?

>    using two RTs per EVI at the expense of having unwanted MAC addresses
>    on the Leaf-only PEs.
>
>

>
> From: Thomas Morin <[email-masked] 
> <mailto:[email-masked]>>
> Organization: Orange
> Date: Thursday, December 15, 2016 at 4:12 AM
> To: Cisco Employee <[email-masked] <mailto:[email-masked]>>, Loa 
> Andersson <[email-masked] <mailto:[email-masked]>>, "George Swallow -T (swallow 
> - MBO PARTNERS INC at Cisco)" <[email-masked] 
> <mailto:[email-masked]>>, Eric Rosen <[email-masked] 
> <mailto:[email-masked]>>, BESS <[email-masked] <mailto:[email-masked]>>
> Cc: Martin Vigoureux <[email-masked] 
> <mailto:[email-masked]>>
> Subject: Re: shepherd review of draft-ietf-bess-evpn-etree
>
> Hi Ali,
>
> 2016-12-13, Ali Sajassi (sajassi):
>>
>> 2016-12-10, Ali Sajassi (sajassi):
>>> Your suggestion regarding multiple MAC-VRFs per EVI for E-TREE, 
>>> impacts lot more sections than just section 2.2 for which you 
>>> suggested some texts. It drastically  impacts section 3.1 (known 
>>> unicast traffic), and it also impacts section 3.2 (BUM traffic) and 
>>> section 5.1.
>>
>> Can you detail why ?
>> The understanding that leads me to this suggestion is that the 
>> 2-RT+split-horizon scenario in 2.1, then applied to Root/Leaf PE in a 
>> 2.2.1 would not require new procotol procedures nor changes in the 
>> text that as I understand provides procedures for 2.2(.2) and 2.3.
>> 2nd try. As my 1st response got truncated for some reason.
>>
>> The reason that impacts more sections than just sec. 2, is that the 
>> proposed 2.2.1 would be an alternative option for section 3.1. In 
>> section 3.1, the root/leaf indication for MAC addresses are done via 
>> flag-bit defined in section 5.1 and it only uses a single MAC-VRF 
>> (single bridge table per VLAN) per RFC 7432. If we go with two 
>> MAC-VRFs (e.g., two bridge tables) per VLAN, then that is an 
>> alternative way of doing the same thing described in section 3.1. 
>> This alternative way has big ramifications on the platform as it 
>> requires duplicating MACs and managing multiple bridge tables per VLAN.
>
> Since 2.1 and the proposed 2.2.1 do not require new protocol 
> procedures (they only require split-horizon locally in Leaf MAC-VRFs), 
> if you state clearly that the procedures in the document are here to 
> address 2.2.2 and 2.3, then you don't need to modify the content of 
> the document after section 2 (more exactly, you will need minor update 
> like changing the current "This scheme applies to all scenarios 
> described in section 2." in section 3 into "This scheme applies to 
> scenarios described in 2.2.2 and 2.3".
>
> The "big ramifications" above are then not about section 3, but just 
> the (platform specific-drawbacks) of 2.2.2 that we have already 
> discussed and that can be covered in 2.2.2.
>
>> Maybe what you really want is to allow for scenario 2.2 to operate 
>> with two RTs which has the benefits of both 2.2.1 and 2.2.2 and non 
>> of the drawbacks. So, maybe we can clarify the current text to make 
>> sure that this comes out clearly – ie, a PE can have single MAC–VRF 
>> can have multiple RTs.
>
> You could mention that, but for me the key things is:
> - documenting the motivation for the new procedures
> - not arbitrarily /restrict/ 2.2.2 to one RT (but why not document 
> identified drawbacks)
>
>>
>>> Furthermore, it creates a new paradigm for EVPN that was never 
>>> intended for because of creating two MAC-VRFs (and two bridge 
>>> tables) for the same VLAN.
>>
>> The "<new thing> created a new paradigm that <RFX xyz> was never 
>> intended for" is a not generally valid, or sufficiently detailed, 
>> argument: if it was, then you might go as far as challenging the 
>> whole E-Tree spec on the same kind grounds (and many other new things).
>>
>> So here is where it seems we have a gap to bridge: I still don't 
>> understand what in RFC7432 describes an intention of "not supporting 
>> two MAC-VRFs for the same VLAN".
>>
>> I tried to explain the relationship between EVI, MAC-VRF, bridge 
>> table, and VLAN in my previous email per RFC 7432. However, lets park 
>> this discussion for time being as I think it is secondary.
>
> Ok, feel free to revisit if you think that RFC7432 would preclude 
> procedures that end up being described in this draft
>
>> I think you agree that if we have a single solution that has all the 
>> benefits of your proposed 2.2.1 and 2.2.2 and none of the drawbacks, 
>> it is much more preferable with having two solutions each with its 
>> own advantages and draw backs, right? If so, then existing text in 
>> 2.2 was intended to convey that. However, we can clarify it 
>> further – e.g, make it clear that for PE with root & leaf in the same 
>> EVI, we can use a single MAC-VRF with two RTs (one for leaf and 
>> another for root).
>
> As said above my key concern is having the document clearly spell out 
> the motivation for new specs.
> If this implies documenting the fact that already existing procedure 
> can be used, but have drawbacks, then so be it ; there would be no 
> point in hiding that, right ?
>
>
>>
>>> The WG LC was completed on 3/29/16 and I am sure it is not your 
>>> intention to have major changes to the doc at this stage where 
>>> multiple vendors have already implemented the draft.
>>
>> As you know, there are different stages at which people do reviews on 
>> a doc after WGLC, an which may lead doc editors to introduce 
>> significant --editorial or technical-- changes in a document. 
>> Sometimes that leads to documents going back to the working group.
>>
>> However my root intention as doc shepherd, of course, is not to 
>> propose a major change, but merely to able to answer the standard 
>> question of the shepherd review -- on the reviews done, on document 
>> readiness, and on the document quality -- in a way as positive and 
>> sincere as possible. In particular questions (3) (4) and (6).
>>
>> So, hopefully the answers to these three questions are now 
>> clear. I believe your main concern is to ensure that we can apply 
>> two-RT approach of sec. 2.1  to sec. 2.2 (and we can still do and 
>> still have a single MAC-VRF)
>
> See above.
>
>>
>>
>>> This draft talks about two kinds of traffic filtering: a) ingress 
>>> filtering for known unicast and b) egress filtering for BUM traffic. 
>>> What you are suggesting is an alternate mechanism for ingress filtering.
>>
>> (well I'm not suggesting the mechanism itself --which section 2.1 
>> already does-- but simply to document that it can still apply without 
>> the constraint of avoiding the presence of a Root MAC-VRF and a Leaf 
>> MAC-VRF on a same PE)
>>
>>> Although having multiple VRFs (and forwarding tables) are fine for 
>>> IP-VPNs because the unknown traffic is always dropped, multiple VRFs 
>>> for the same VLAN is not OK for L2 traffic because of flooding of 
>>> unknown traffic. That’s why in section 6 of RFC 7432, for all 
>>> service interface types, the draft talks about a single MAC-VRF per 
>>> EVI per PE and in case of VLAN-aware mode,  multiple VLANs per 
>>> MAC-VRF but only a single bridge table per VLAN. In other words, the 
>>> bottom line is that there can only be a single bridge table per VLAN 
>>> in order to avoid unnecessary flooding.
>>
>>
>>
>>> When you have two MAC-VRFs per VLAN (one for root ACs and another 
>>> for Leaf ACs), then you either need to duplicate lots of MAC 
>>> addresses between these two VRFs, or do lookup on both of these 
>>> VRFs. Either ways this is not a good option relative to keeping a 
>>> single VRF table for both root and leaf sites and just have a 
>>> single-bit indication on whether a MAC is associated with root or 
>>> leaf (as currently described approach in the draft).  I
>>
>>
>> In the above, it seems you agree that it can work, and you are able 
>> to offer reasons why it is not the preferred option, then why not 
>> just document that it can work and provides these reasons as the 
>> motivations that lead to proposing a new specs ?
>>
>> Sure, I can do that. [...]
>
> Ok.
> I'll be happy to review a new revision and hopefully post the shepherd 
> review.
>
> Thanks,
>
> -Thomas
>
>
>>
>> (it seems you have an unfinished last sentence: "I [...]" )
>>
>>
>>>
>>>>>
>>>>> (assuming the previous point is resolved:)
>>>>>
>>>>> With this mechanism above, isn't it possible to have on a given 
>>>>> PE, for a single E-TREE EVI, both Leaves and Roots, as long as 
>>>>> distinct MAC-VRFs are used (one for Leaves and one for Roots) ?   
>>>>> (it seems to me that the assymetric import/export RT would do what 
>>>>> is needed to build an E-TREE, we would just have a particular case 
>>>>> where a Leaf MAC-VRF and a Root MAC-VRF for a given E-TREE end up 
>>>>> on a single PE)
>>>>>
>>>>>
>>>>> That’s not possible because per definition of an EVI, there is 
>>>>> only a single MAC-VRF per EVI for a PE.
>>>>
>>>> Where can I read such a definition ? (the Terminology section in 
>>>> RFC7432 does not say that, unless I'm missing something).
>>>> And that seems a completely arbitrary restriction.
>>>> (just thinking that a given PE device can be split in two logical 
>>>> devices show that it can work)
>>>>
>>>> Section 6 of RFC7432 where it gives definitions for different 
>>>> service interface types, it specifies the relationship between 
>>>> MAC-VRF and VLAN (bridge table) and how many MAC-VRF (and bridge 
>>>> tables) can be per EVI.
>>>
>>> This section of RFC7434 discusses many different things for the 
>>> different variants.
>>> Can you provide a specific pointer about "how many MAC-VRFs can be 
>>> per EVI" ?
>>>
>>> Ali> Section 6 of RFC7432 spells out the relationship between EVI, 
>>> MAC-VRF, and bridge tables for all service interfaces very clearly.
>>> In all service interfaces, the RFC says there is one MAC-VRF per EVI 
>>> on a given PE.
>>> Now, if the service interface is “vlan-aware”, then there are 
>>> several bridge tables for that single MAC-VRF – ie, one bridge table 
>>> per VLAN. In all service interfaces, you can ONLY have one bridge 
>>> table per VLAN.
>>
>> This answer is everything but a specific pointer.
>> If Section 6 of RFC7432 says all this very clearly, I guess it should 
>> be possible to extract quotes about "there is one MAC-VRF per EVI on 
>> a given PE", right ?
>>
>>
>>>
>>>> In bridging world, there can only be a single bridge table per VLAN 
>>>> in a device.
>>>
>>> I still don't find here anything that would preclude having, on a 
>>> given PE, for a given E-TREE EVI, one Leaves MAC-VRF and one Roots 
>>> MAC-VRF: can't these two MAC-VRFs use different internal VLANs (with 
>>> translation if the external VLANs are constrained).
>>>
>>> Ali>  Lets assume we are using vlan-based service and thus there is 
>>> only a single bridge table per MAC-VRF, then what you are suggesting 
>>> is two use two MAC-VRFs (two bridge tables) for the same EVI (same 
>>> VLAN). This results in some duplications of MAC addresses and would 
>>> only work if flooding is disabled (more on this later).
>>
>> "results in some duplications of MAC" is perhaps a drawback, but 
>> nothing like "just does not work" ?
>>
>> "would only work if flooding is disabled": why ?  (you wrote "(more 
>> on this later)" but I couldn't identify anything recent from you in 
>> the rest of the email below)
>>
>>
>> From an helicopter view, I can't see what fundamentally would become 
>> problematic between "two MAC-VRFs on two distinct PEs" and the same 
>> "two MAC-VRFs on a same PEs", at worse it is as efficient or as 
>> inefficient as having them on separate PEs (think logical router 
>> without anykind of dataplane optimisation), and we can't exclude that 
>> the PE could have local implementation details to do better than that.
>>
>>
>>>>
>>>>> Besides, I don’t understand what good does it do to have two 
>>>>> MAC-VRFs on the same PE (one for Leafs and another for Roots)
>>>>
>>>> Well, the "what is good for" is pretty simple: it means you can 
>>>> have, just by tailoring the import/export policies like in 2.1, 
>>>> something as useful as the scenario in 2.2.
>>>>
>>>> There can only be a single bridge table per VLAN. Now even if you 
>>>> add some kind of logic to form two logical PEs in single physical 
>>>> PE, you end up replicating all the MAC addresses associated with 
>>>> the root sites in two bridge tables.
>>>
>>> Your point above certainly does not sound to me as "it can't be 
>>> done": some may think that the above is an acceptable cost, some 
>>> others may find ways to make this "replication" with a low overhead, 
>>> on some platforms the cost may be negligible, etc.
>>>
>>>
>>>>
>>>>
>>>>> because Leafs and Roots need to talk to each other and thus we 
>>>>> want them to be in the same MAC-VRF.
>>>>
>>>> The fact that Leafs and Roots need to talk to each other does not 
>>>> mean that they *have* to be in the same MAC-VRF, you can rely on 
>>>> the local MPLS dataplane inside the PE to carry the traffic between 
>>>> Roots and Leaves can be passed between a Leaf MAC-VRF and a Root 
>>>> MAC-VRF (and you can possibly implement a shortcut not involving 
>>>> MPLS encap/decap).
>>>>
>>>> Anything is possible but at what cost.
>>>
>>> You know, for cost it is not always obvious to reach conclusions 
>>> that are true for all implementations and all targets.
>>>
>>>> The current proposal is very efficient in terms of forwarding path 
>>>> as well as control plane.
>>>
>>> Sure, but what I question is not the new solution but the lack of 
>>> discussion on why using the existing specs was not considered good 
>>> enough.
>>>
>>>
>>> I think that my concern of clearly explaining the scenarios and 
>>> motivations for this new spec could be addressed by splitting 
>>> section 2.2 into a 2.2.1 describing the approach from 2.1 and its 
>>> possible drawbacks, and a 2.2.2 having essentially the content of 
>>> current section 2.2.
>>>
>>> Here is a proposal:
>>>
>>> 2.2 Scenario 2: Leaf of Root site(s) per AC
>>>
>>> In these scenarii, a PE receives traffic from either Root OR Leaf
>>> sites (but not both) on a given Attachment Circuit (AC) of an EVI. In
>>> other words, an AC (ES or ES/VLAN) is either associated with Root(s)
>>> or Leaf(s) (but not both).
>>>
>>> 2.2.1 Scenario 2a: Leaf OR Root site(s) per AC, separate Leaf/Root 
>>> MAC-VRFs
>>>
>>> +---------+            +---------+
>>> |   PE1   |            |   PE2   |
>>> +---+            |  +---+  | +------+  |  +---+  | +---+
>>> |CE1+-----ES1----+--+   |  |  | |  |  |MAC+--+---ES2/AC1--+CE2|
>>> +---+    (Leaf)  |  |MAC|  |  | MPLS |  |  |VRF|  |   (Leaf)   +---+
>>> |  |VRF|  |  |  /IP |  |  '---'  |
>>> |  |   |  |  |      |  |  .---.  |
>>> |  |   |  |  |      |  |  |MAC| |            +---+
>>> |  |   |  |  |      |  | |VRF+--+---ES2/AC2--+CE3|
>>> |  +---+  |  +------+  |  +---+  | (Root)   +---+
>>> +---------+            +---------+
>>>
>>> Figure 2: Scenario 2a
>>>
>>> In this scenario, the RT constraint procedures described in section 
>>> 2.1 could
>>> also be used. The feasibility and efficiency of this approach depends on
>>> platforms specifics.
>>>
>>> This approach will lead toduplication of a large proportion of MAC 
>>> addresseson
>>>    PEs having both Leaf and Root sites, and is hence considered less 
>>> suitable for
>>>    deployment contexts where the vast majority of PEs are likely to 
>>> ultimately
>>> have both Leaf and Root sites attached to them.
>>>
>>> 2.2.2 Scenario 2b: Leaf OR Root site(s) per AC, single MAC-VRF
>>>
>>> +---------+            +---------+
>>> |   PE1   |            |   PE2   |
>>> +---+            |  +---+  | +------+  |  +---+  | +---+
>>> |CE1+-----ES1----+--+   |  |  | |  |  |   +--+---ES2/AC1--+CE2|
>>> +---+    (Leaf)  |  |MAC|  |  | MPLS |  |  |MAC|  |   (Leaf)   +---+
>>> |  |VRF|  |  |  /IP |  |  |VRF|  |
>>> |  |   |  |  |      |  |  |   | |            +---+
>>> |  |   |  |  |      |  |  | +--+---ES2/AC2--+CE3|
>>> |  +---+  |  +------+  |  +---+  | (Root)   +---+
>>> +---------+            +---------+
>>>
>>> Figure 2: Scenario 2b
>>>
>>> This scenario will alleviate keys drawbacks from Scenario 2a, in 
>>> particular
>>> by avoiding duplication of MAC addresses on Leaf/Root PEs and 
>>> avoiding the
>>>    operational overhead of managing more than one RT.
>>>    This approach comes at the expense of having routes for unneeded 
>>> MAC addresses on Leaf-only PEs, and is hence considered less 
>>> suitable for deployment contexts where the vast majority of PEs 
>>> would remain Leaf-only.    Unlike Scenario 1 and Scenario 2a, this scenario requires additional procedures
>>>     provided in this document.
>>>
>>>
>>> (And this last sentence should be added to section 2.3 as well)
>>>
>>>>
>>>>>> For this scenario, if for a given
>>>>>>    EVI, the majority of PEs will eventually have both Leaf and Root
>>>>>>    sites attached, even though they may start as Root-only or 
>>>>>> Leaf-only
>>>>>>    PEs, then it is recommended to use a single RT per EVI and avoid
>>>>>>    additional configuration and operational overhead.
>>>>>
>>>>> Why this recommendation ?
>>>>> Even with a majority of PEs having both Leaves and Roots, there 
>>>>> can remain (up to 49% of) PEs having only Leaves, which will 
>>>>> uselessly have all routes to other Leaves.
>>>>>
>>>>> So "it is recommended" above, deserves to be explained more, I think.
>>>>>
>>>>> OK, I changed “majority” to “vast majority” :-)
>>>>
>>>> My point was not to nit pick on "majority", but was that you should 
>>>> explain why you recommend that.
>>>> As the text currently reads, the cost of the recommendation can be 
>>>> identified: having useless routes on the fraction of PEs having 
>>>> only Leaves.
>>>> But the gain brought by the recommendation is not even mentioned, 
>>>> not to say explained.
>>>> Hence: why ?
>>>> (Why is it a useful tradeoff to have useless routes on some, even 
>>>> if only one, PE ?)
>>>>
>>>> Changed the last sentence from:
>>>> "then it is recommended to use a single RT per EVI and avoid 
>>>> additional configuration and operational overhead.”
>>>> To
>>>> "then it is recommended to use a single RT per EVI and avoid 
>>>> additional configuration and operational overhead
>>>> at the expense of having unwanted MAC addresses on the Leaf PEs."
>>>
>>> Ok. I adapted and incorporated this addition into my proposed text 
>>> splitting 2.2 into a 2.2.1 and a 2.2.2.
>>>
>>> Best,
>>>
>>> -Thomas
>>>
>>>
>>
>

---

From: "Ali Sajassi (sajassi)" <[email-masked]>
Date: Thu, 12 Jan 2017 23:16:02 +0000

Hi Thomas,

I have incorporated your new comments and generated the new text attached here. Please review section 2.2 and let me know if there are any more comments. Please refer below for the comment resolution explanation of some of the comments.

Regards,
Ali

From: Thomas Morin <[email-masked]<mailto:[email-masked]>>
Organization: Orange
Date: Thursday, January 5, 2017 at 1:54 AM
To: Cisco Employee <[email-masked]<mailto:[email-masked]>>, Loa Andersson <[email-masked]<mailto:[email-masked]>>, "George Swallow -T (swallow - MBO PARTNERS INC at Cisco)" <[email-masked]<mailto:[email-masked]>>, Eric Rosen <[email-masked]<mailto:[email-masked]>>, BESS <[email-masked]<mailto:[email-masked]>>
Cc: Martin Vigoureux <[email-masked]<mailto:[email-masked]>>
Subject: Re: shepherd review of draft-ietf-bess-evpn-etree

Hi Ali,

2016-12-19, Ali Sajassi (sajassi):
I have modified section 2.2 (copied below) to elaborate why coloring approach for Leaf/Root MAC addresses is used in this draft. Also, the use of single RT for this scenario is mentioned just as “MAY”. Please review the text below and let me know if you still have questions/comments:

-Thomas

                     +---------+            +---------+
                     |   PE1   |            |   PE2   |

   Figure 2: Scenario 2

reverting A and B would be more natural since solution B corresponds to the starting point "what we had before this spec"

Done.

s/it is/they are/

Done.

   AC (or a Root AC)

I think removing the parenthesis is needed for the 'whether' statement to parse.

Done.

I think "(two bridge tables per VLANs)" is inexact:  "two bridge tables per VLAN if a given VLAN exists on the PE for both Leaf and Root ACs of a given EVI" ?

That’s fine. Done!

Similarly, in the following paragraph, I think "per VLAN" should be replaced by "per E-TREE EVI having both Root and Leaf ACs".

A single EVI can consist of many VLANs (in case of VLAN-aware bundle service), so, “per VLAN” is right. However, to make it more exact as above, I’ll change it to “per VLAN (when both Root and Leafs ACs exist for that VLAN) requires …”.

Since it is said above "Maintaining two bridge tables per VLAN requires either two lookups [...] or duplicating many MAC addresses [...]", saying "The duplication of MAC addresses is needed for [...]"  is surprising, so I guess the intent is rather "Unless two lookups are made, duplication of MAC addresses would be needed for [...]".

That is correct. I’ll change it to the sentence that you suggested: “Unless two lookups are made, …"

I think that this explanation is too elliptic compared to the strong ("not viable") conclusion. As soon as we talk about implementation details, a more detailed discussion is required on why, and under which assumptions, some things are impossible  -- there can be many different way to implement a dataplane. Without explaining what "two lookups" exactly means in this context, it's hard to follow why it would be required if duplicating MAC addresses is not done, and why it is latter concluded as "not viable":
- doing multiple lookups is something that is far from being uncommon on router platforms
- on software platforms the impact of doing multiple lookups can be reduced to mostly zero (e.g. with OpenVSwitch that would only impact the first packets of a flow)
- if the dataplane can leverage the colouring information to avoid doing two lookups, then perhaps this hardware ability can be leveraged to support the two MAC-VRFs approach with only one lookup  (building one table, marking MAC entries as Leaf entries if they were learned with routes carrying only the Leaf RT?)  --- don't misunderstand me: I'm not claiming that this works (I haven't looked closely enough), but simply that the text provided is not sufficient to exclude this kind of solution

The "duplicating MAC addresses" alternative is explained better, but still, nothing is explained on why this is "not viable". It seems to be as something rather belonging to the realm of "having a scalability impact", but even looking in this respect we are not talking about a change of order of magnitude.

The non-viable conclusion was based on investigation that I did for micro-code and ASIC based platforms; however, I see your point and I am para-phrasing the sentence as follow to leave room for future investigation.
"In order to avoid two MAC-VRFs, this draft introduces the coloring option (B) as detailed in section 3.1"

"to alleviate  additional configuration overhead associated with ..." -> "to alleviate the configuration overhead associated with ..." ?

Done.

From: Thomas Morin <[email-masked]<mailto:[email-masked]>>
Organization: Orange
Date: Thursday, December 15, 2016 at 4:12 AM
To: Cisco Employee <[email-masked]<mailto:[email-masked]>>, Loa Andersson <[email-masked]<mailto:[email-masked]>>, "George Swallow -T (swallow - MBO PARTNERS INC at Cisco)" <[email-masked]<mailto:[email-masked]>>, Eric Rosen <[email-masked]<mailto:[email-masked]>>, BESS <[email-masked]<mailto:[email-masked]>>
Cc: Martin Vigoureux <[email-masked]<mailto:[email-masked]>>
Subject: Re: shepherd review of draft-ietf-bess-evpn-etree

Hi Ali,

2016-12-10, Ali Sajassi (sajassi):
Your suggestion regarding multiple MAC-VRFs per EVI for E-TREE, impacts lot more sections than just section 2.2 for which you suggested some texts. It drastically  impacts section 3.1 (known unicast traffic), and it also impacts section 3.2 (BUM traffic) and section 5.1.

Can you detail why ?
The understanding that leads me to this suggestion is that the 2-RT+split-horizon scenario in 2.1, then applied to Root/Leaf PE in a 2.2.1 would not require new procotol procedures nor changes in the text that as I understand provides procedures for 2.2(.2) and 2.3.
2nd try. As my 1st response got truncated for some reason.

The reason that impacts more sections than just sec. 2, is that the proposed 2.2.1 would be an alternative option for section 3.1. In section 3.1, the root/leaf indication for MAC addresses are done via flag-bit defined in section 5.1 and it only uses a single MAC-VRF (single bridge table per VLAN) per RFC 7432. If we go with two MAC-VRFs (e.g., two bridge tables) per VLAN, then that is an alternative way of doing the same thing described in section 3.1. This alternative way has big ramifications on the platform as it requires duplicating MACs and managing multiple bridge tables per VLAN.

Since 2.1 and the proposed 2.2.1 do not require new protocol procedures (they only require split-horizon locally in Leaf MAC-VRFs), if you state clearly that the procedures in the document are here to address 2.2.2 and 2.3, then you don't need to modify the content of the document after section 2  (more exactly, you will need minor update like changing the current "This scheme applies to all scenarios described in section 2." in section 3 into "This scheme applies to scenarios described in 2.2.2 and 2.3".

The "big ramifications" above are then not about section 3, but just the (platform specific-drawbacks) of 2.2.2 that we have already discussed and that can be covered in 2.2.2.

Maybe what you really want is to allow for scenario 2.2 to operate with two RTs which has the benefits of both 2.2.1 and 2.2.2 and non of the drawbacks. So, maybe we can clarify the current text to make sure that this comes out clearly – ie, a PE can have single MAC–VRF can have multiple RTs.

You could mention that, but for me the key things is:
- documenting the motivation for the new procedures
- not arbitrarily /restrict/ 2.2.2 to one RT (but why not document identified drawbacks)

Furthermore, it creates a new paradigm for EVPN that was never intended for because of creating two MAC-VRFs (and two bridge tables) for the same VLAN.

The "<new thing> created a new paradigm that <RFX xyz> was never intended for" is a not generally valid, or sufficiently detailed, argument: if it was, then you might go as far as challenging the whole E-Tree spec on the same kind grounds (and many other new things).

So here is where it seems we have a gap to bridge: I still don't understand what in RFC7432 describes an intention of "not supporting two MAC-VRFs for the same VLAN".

I tried to explain the relationship between EVI, MAC-VRF, bridge table, and VLAN in my previous email per RFC 7432. However, lets park this discussion for time being as I think it is secondary.

Ok, feel free to revisit if you think that RFC7432 would preclude procedures that end up being described in this draft

I think you agree that if we have a single solution that has all the benefits of your proposed 2.2.1 and 2.2.2 and none of the drawbacks, it is much more preferable with having two solutions each with its own advantages and draw backs, right? If so, then existing text in 2.2 was intended to convey that. However, we can clarify it further – e.g, make it clear that for PE with root & leaf in the same EVI, we can use a single MAC-VRF with two RTs (one for leaf and another for root).

As said above my key concern is having the document clearly spell out the motivation for new specs.
If this implies documenting the fact that already existing procedure can be used, but have drawbacks, then so be it ; there would be no point in hiding that, right ?

The WG LC was completed on 3/29/16 and I am sure it is not your intention to have major changes to the doc at this stage where multiple vendors have already implemented the draft.

As you know, there are different stages at which people do reviews on a doc after WGLC, an which may lead doc editors to introduce significant --editorial or technical-- changes in a document. Sometimes that leads to documents going back to the working group.

However my root intention as doc shepherd, of course, is not to propose a major change, but merely to able to answer the standard question of the shepherd review -- on the reviews done, on document readiness, and on the document quality -- in a way as positive and sincere as possible. In particular questions (3) (4) and (6).

So, hopefully the answers to these three questions are now clear. I believe your main concern is to ensure that we can apply two-RT approach of sec. 2.1  to sec. 2.2 (and we can still do and still have a single MAC-VRF)

See above.

This draft talks about two kinds of traffic filtering: a) ingress filtering for known unicast and b) egress filtering for BUM traffic. What you are suggesting is an alternate mechanism for ingress filtering.

(well I'm not suggesting the mechanism itself --which section 2.1 already does-- but simply to document that it can still apply without the constraint of avoiding the presence of a Root MAC-VRF and a Leaf MAC-VRF on a same PE)

Although having multiple VRFs (and forwarding tables) are fine for IP-VPNs because the unknown traffic is always dropped, multiple VRFs for the same VLAN is not OK for L2 traffic because of flooding of unknown traffic. That’s why in section 6 of RFC 7432, for all service interface types, the draft talks about a single MAC-VRF per EVI per PE and in case of VLAN-aware mode,  multiple VLANs per MAC-VRF but only a single bridge table per VLAN. In other words, the bottom line is that there can only be a single bridge table per VLAN in order to avoid unnecessary flooding.

When you have two MAC-VRFs per VLAN (one for root ACs and another for Leaf ACs), then you either need to duplicate lots of MAC addresses between these two VRFs, or do lookup on both of these VRFs. Either ways this is not a good option relative to keeping a single VRF table for both root and leaf sites and just have a single-bit indication on whether a MAC is associated with root or leaf (as currently described approach in the draft).  I

In the above, it seems you agree that it can work, and you are able to offer reasons why it is not the preferred option, then why not just document that it can work and provides these reasons as the motivations that lead to proposing a new specs ?

Ok.
I'll be happy to review a new revision and hopefully post the shepherd review.

Thanks,

-Thomas

With this mechanism above, isn't it possible to have on a given PE, for a single E-TREE EVI, both Leaves and Roots, as long as distinct MAC-VRFs are used (one for Leaves and one for Roots) ?   (it seems to me that the assymetric import/export RT would do what is needed to build an E-TREE, we would just have a particular case where a Leaf MAC-VRF and a Root MAC-VRF for a given E-TREE end up on a single PE)

That’s not possible because per definition of an EVI, there is only a single MAC-VRF per EVI for a PE.

Where can I read such a definition ? (the Terminology section in RFC7432 does not say that, unless I'm missing something).
And that seems a completely arbitrary restriction.
(just thinking that a given PE device can be split in two logical devices show that it can work)

Section 6 of RFC7432 where it gives definitions for different service interface types, it specifies the relationship between MAC-VRF and VLAN (bridge table) and how many MAC-VRF (and bridge tables) can be per EVI.

This section of RFC7434 discusses many different things for the different variants.
Can you provide a specific pointer about "how many MAC-VRFs can be per EVI" ?

Ali> Section 6 of RFC7432 spells out the relationship between EVI, MAC-VRF, and bridge tables for all service interfaces very clearly.
In all service interfaces, the RFC says there is one MAC-VRF per EVI on a given PE.
Now, if the service interface is “vlan-aware”, then there are several bridge tables for that single MAC-VRF – ie, one bridge table per VLAN. In all service interfaces, you can ONLY have one bridge table per VLAN.

This answer is everything but a specific pointer.
If Section 6 of RFC7432 says all this very clearly, I guess it should be possible to extract quotes about "there is one MAC-VRF per EVI on a given PE", right ?

In bridging world, there can only be a single bridge table per VLAN in a device.

I still don't find here anything that would preclude having, on a given PE, for a given E-TREE EVI, one Leaves MAC-VRF and one Roots MAC-VRF: can't these two MAC-VRFs use different internal VLANs (with translation if the external VLANs are constrained).

Ali>  Lets assume we are using vlan-based service and thus there is only a single bridge table per MAC-VRF, then what you are suggesting is two use two MAC-VRFs (two bridge tables) for the same EVI (same VLAN). This results in some duplications of MAC addresses and would only work if flooding is disabled (more on this later).

"results in some duplications of MAC" is perhaps a drawback, but nothing like "just does not work" ?

"would only work if flooding is disabled": why ?  (you wrote "(more on this later)" but I couldn't identify anything recent from you in the rest of the email below)

>From an helicopter view, I can't see what fundamentally would become problematic between "two MAC-VRFs on two distinct PEs" and the same "two MAC-VRFs on a same PEs", at worse it is as efficient or as inefficient as having them on separate PEs (think logical router without anykind of dataplane optimisation), and we can't exclude that the PE could have local implementation details to do better than that.

Besides, I don’t understand what good does it do to have two MAC-VRFs on the same PE (one for Leafs and another for Roots)

Well, the "what is good for" is pretty simple: it means you can have, just by tailoring the import/export policies like in 2.1, something as useful as the scenario in 2.2.

There can only be a single bridge table per VLAN. Now even if you add some kind of logic to form two logical PEs in single physical PE, you end up replicating all the MAC addresses associated with the root sites in two bridge tables.

Your point above certainly does not sound to me as "it can't be done": some may think that the above is an acceptable cost, some others may find ways to make this "replication" with a low overhead, on some platforms the cost may be negligible, etc.

because Leafs and Roots need to talk to each other and thus we want them to be in the same MAC-VRF.

The fact that Leafs and Roots need to talk to each other does not mean that they *have* to be in the same MAC-VRF, you can rely on the local MPLS dataplane inside the PE to carry the traffic between Roots and Leaves can be passed between a Leaf MAC-VRF and a Root MAC-VRF (and you can possibly implement a shortcut not involving MPLS encap/decap).

You know, for cost it is not always obvious to reach conclusions that are true for all implementations and all targets.

The current proposal is very efficient in terms of forwarding path as well as control plane.

Sure, but what I question is not the new solution but the lack of discussion on why using the existing specs was not considered good enough.

I think that my concern of clearly explaining the scenarios and motivations for this new spec could be addressed by splitting section 2.2 into a 2.2.1 describing the approach from 2.1 and its possible drawbacks, and a 2.2.2 having essentially the content of current section 2.2.

Here is a proposal:

2.2.1 Scenario 2a: Leaf OR Root site(s) per AC, separate Leaf/Root MAC-VRFs

                     +---------+            +---------+
                     |   PE1   |            |   PE2   |

   Figure 2: Scenario 2a

   In this scenario, the RT constraint procedures described in section 2.1 could
   also be used. The feasibility and efficiency of this approach depends on
   platforms specifics.

   This approach will lead toduplication of a large proportion of MAC addresses on
   PEs having both Leaf and Root sites, and is hence considered less suitable for
   deployment contexts where the vast majority of PEs are likely to ultimately
   have both Leaf and Root sites attached to them.

                     +---------+            +---------+
                     |   PE1   |            |   PE2   |

   Figure 2: Scenario 2b

   This scenario will alleviate keys drawbacks from Scenario 2a, in particular
   by avoiding duplication of MAC addresses on Leaf/Root PEs and avoiding the
   operational overhead of managing more than one RT.

   This approach comes at the expense of having routes for unneeded MAC addresses
   on Leaf-only PEs, and is hence considered less suitable for deployment contexts
   where the vast majority of PEs would remain Leaf-only.   Unlike Scenario 1 and Scenario 2a, this scenario requires additional procedures
   provided in this document.

   sites attached, even though they may start as Root-only or Leaf-only

Why this recommendation ?
Even with a majority of PEs having both Leaves and Roots, there can remain (up to 49% of) PEs having only Leaves, which will uselessly have all routes to other Leaves.

My point was not to nit pick on "majority", but was that you should explain why you recommend that.
As the text currently reads, the cost of the recommendation can be identified: having useless routes on the fraction of PEs having only Leaves.
But the gain brought by the recommendation is not even mentioned, not to say explained.
Hence: why ?
(Why is it a useful tradeoff to have useless routes on some, even if only one, PE ?)

Changed the last sentence from:
"then it is recommended to use a single RT per EVI and avoid additional configuration and operational overhead.”
To
"then it is recommended to use a single RT per EVI and avoid additional configuration and operational overhead
at the expense of having unwanted MAC addresses on the Leaf PEs."

Ok. I adapted and incorporated this addition into my proposed text splitting 2.2 into a 2.2.1 and a 2.2.2.

Best,

-Thomas

---

From: Thomas Morin <[email-masked]>
Date: Fri, 13 Jan 2017 10:22:33 +0100

Hi Ali,

One comment on what seems to me the last pending issue.

Ali:
 >>Thomas:
> details, a more detailed discussion is required on why, and under 
> which assumptions, some things are impossible  -- there can be many 
> different way to implement a dataplane. Without explaining what "two 
> lookups" exactly means in this context, it's hard to follow why it 
> would be required if duplicating MAC addresses is not done, and why it 
> is latter concluded as "not viable":
> reduced to mostly zero (e.g. with OpenVSwitch that would only impact 
> the first packets of a flow)
> - if the dataplane can leverage the colouring information to avoid 
> doing two lookups, then perhaps this hardware ability can be leveraged 
> to support the two MAC-VRFs approach with only one lookup  (building 
> one table, marking MAC entries as Leaf entries if they were learned 
> with routes carrying only the Leaf RT?)  --- don't misunderstand me: 
> I'm not claiming that this works (I haven't looked closely enough), 
> but simply that the text provided is not sufficient to exclude this 
> kind of solution
>
> still, nothing is explained on why this is "not viable". It seems to 
> be as something rather belonging to the realm of "having a scalability 
> The non-viable conclusion was based on investigation that I did for 
> micro-code and ASIC based platforms; however, I see your point and I 
> am para-phrasing the sentence as follow to leave room for 
> future investigation.

This is better, but:
- the text still makes a statement that either two lookups or 
duplicating MACs is required, without much explanation
- the logic is not consolidated: if either X or Y is required, then its 
illogical to say "In order to avoid X, this drafts introduces Z...."  (Z 
being different than Y)
- the text still misses an explanation on why avoiding two MAC-VRFs was 
desired

If we can't (or don't want to spend time on) explaining the details on 
why something else than "two MAC RVFs" was deemed useful, then perhaps 
we could simplify the whole paragraph into something like:

       Because option (A) is not believed to be implementable without 
dataplane performance inefficiencies some platforms, this drafts 
introduces the coloring option (B) in section 3.1.

Best,

-Thomas

2017-01-13, Ali Sajassi (sajassi):
> reverting A and B would be more natural since solution B corresponds 
> to the starting point "what we had before this spec"
>
> Done.
>
>>
> I think "(two bridge tables per VLANs)" is inexact:  "two bridge 
> tables per VLAN if a given VLAN exists on the PE for both Leaf and 
> Root ACs of a given EVI" ?
>
> That’s fine. Done!
>
>
> A single EVI can consist of many VLANs (in case of VLAN-aware bundle 
> service), so, “per VLAN” is right. However, to make it more exact as 
> above, I’ll change it to “per VLAN (when both Root and Leafs ACs exist 
> for that VLAN) requires …”.
>
>
>
>>
> Since it is said above "Maintaining two bridge tables per VLAN 
> requires either two lookups [...] or duplicating many MAC addresses 
> [...]", saying "The duplication of MAC addresses is needed for [...]"  
> is surprising, so I guess the intent is rather "Unless two lookups are 
> That is correct. I’ll change it to the sentence that you 
> suggested: “Unless two lookups are made, …"
>
>
> details, a more detailed discussion is required on why, and under 
> which assumptions, some things are impossible  -- there can be many 
> different way to implement a dataplane. Without explaining what "two 
> lookups" exactly means in this context, it's hard to follow why it 
> would be required if duplicating MAC addresses is not done, and why it 
> is latter concluded as "not viable":
> reduced to mostly zero (e.g. with OpenVSwitch that would only impact 
> the first packets of a flow)
> - if the dataplane can leverage the colouring information to avoid 
> doing two lookups, then perhaps this hardware ability can be leveraged 
> to support the two MAC-VRFs approach with only one lookup  (building 
> one table, marking MAC entries as Leaf entries if they were learned 
> with routes carrying only the Leaf RT?)  --- don't misunderstand me: 
> I'm not claiming that this works (I haven't looked closely enough), 
> but simply that the text provided is not sufficient to exclude this 
> kind of solution
>
> still, nothing is explained on why this is "not viable". It seems to 
> be as something rather belonging to the realm of "having a scalability 
> The non-viable conclusion was based on investigation that I did for 
> micro-code and ASIC based platforms; however, I see your point and I 
> am para-phrasing the sentence as follow to leave room for 
> future investigation.
> "to alleviate  additional configuration overhead associated with ..." 
> -> "to alleviate the configuration overhead associated with ..." ?
>
> Done.
>
>> To: Cisco Employee <[email-masked] <mailto:[email-masked]>>, 
>> Loa Andersson <[email-masked] <mailto:[email-masked]>>, "George Swallow -T 
>> (swallow - MBO PARTNERS INC at Cisco)" <[email-masked] 
>>> 2-RT+split-horizon scenario in 2.1, then applied to Root/Leaf PE in 
>>> a 2.2.1 would not require new procotol procedures nor changes in the 
>> procedures (they only require split-horizon locally in Leaf 
>> MAC-VRFs), if you state clearly that the procedures in the document 
>> are here to address 2.2.2 and 2.3, then you don't need to modify the 
>> content of the document after section 2  (more exactly, you will need 
>> minor update like changing the current "This scheme applies to all 
>> scenarios described in section 2." in section 3 into "This scheme 
>> applies to scenarios described in 2.2.2 and 2.3".
>>
>>> table, and VLAN in my previous email per RFC 7432. However, lets 
>>> park this discussion for time being as I think it is secondary.
>>
>>> further – e.g, make it clear that for PE with root & leaf in 
>>> the same EVI, we can use a single MAC-VRF with two RTs (one for leaf 
>>> and another for root).
>>
>>> As you know, there are different stages at which people do reviews 
>>> on a doc after WGLC, an which may lead doc editors to introduce 
>>>> filtering for known unicast and b) egress filtering for BUM 
>>>> traffic. What you are suggesting is an alternate mechanism for 
>>>> ingress filtering.
>>>
>>> already does-- but simply to document that it can still apply 
>>> without the constraint of avoiding the presence of a Root MAC-VRF 
>>> and a Leaf MAC-VRF on a same PE)
>>>
>>>> IP-VPNs because the unknown traffic is always dropped, multiple 
>>>> VRFs for the same VLAN is not OK for L2 traffic because of flooding 
>>>> of unknown traffic. That’s why in section 6 of RFC 7432, for all 
>>>> MAC-VRF but only a single bridge table per VLAN. In other words, 
>>>> the bottom line is that there can only be a single bridge table per 
>>>> VLAN in order to avoid unnecessary flooding.
>>>
>>>
>>>
>>>>>> (it seems to me that the assymetric import/export RT would do 
>>>>>> what is needed to build an E-TREE, we would just have a 
>>>>>> particular case where a Leaf MAC-VRF and a Root MAC-VRF for a 
>>>>>> given E-TREE end up on a single PE)
>>>>>>
>>>>>>
>>>> several bridge tables for that single MAC-VRF – ie, one bridge 
>>>> table per VLAN. In all service interfaces, you can ONLY have one 
>>>> bridge table per VLAN.
>>>
>>> If Section 6 of RFC7432 says all this very clearly, I guess it 
>>> should be possible to extract quotes about "there is one MAC-VRF per 
>>> EVI on a given PE", right ?
>>>
>>>
>>>>
>>>> MAC-VRF: can't these two MAC-VRFs use different internal VLANs 
>>>> (with translation if the external VLANs are constrained).
>>>>
>>>> only a single bridge table per MAC-VRF, then what you are 
>>>> suggesting is two use two MAC-VRFs (two bridge tables) for the same 
>>>> EVI (same VLAN). This results in some duplications of MAC addresses 
>>>> and would only work if flooding is disabled (more on this later).
>>>
>>> without anykind of dataplane optimisation), and we can't exclude 
>>> that the PE could have local implementation details to do better 
>>> than that.
>>>
>>>
>>>>>
>>>> others may find ways to make this "replication" with a low 
>>>> overhead, on some platforms the cost may be negligible, etc.
>>>>
>>>>
>>>>>
>>>>>
>>>>> the local MPLS dataplane inside the PE to carry the traffic 
>>>>> between Roots and Leaves can be passed between a Leaf MAC-VRF and 
>>>>> a Root MAC-VRF (and you can possibly implement a shortcut not 
>>>>> involving MPLS encap/decap).
>>>>>
>>>>    This approach will lead toduplication of a large proportion of 
>>>> MAC addresseson

---

From: "Ali Sajassi (sajassi)" <[email-masked]>
Date: Fri, 13 Jan 2017 18:41:24 +0000

Hi Thomas,

Please refer to the comment resolution below:

Date: Friday, January 13, 2017 at 1:22 AM

Hi Ali,

This is better, but:
- the text still makes a statement that either two lookups or duplicating MACs is required, without much explanation
- the logic is not consolidated: if either X or Y is required, then its illogical to say "In order to avoid X, this drafts introduces Z...."  (Z being different than Y)
- the text still misses an explanation on why avoiding two MAC-VRFs was desired

If we can't (or don't want to spend time on) explaining the details on why something else than "two MAC RVFs" was deemed useful, then perhaps we could simplify the whole paragraph into something like:

      Because option (A) is not believed to be implementable without dataplane performance inefficiencies some platforms, this drafts introduces the coloring option (B) in section 3.1.

I modified the last sentence as below:
"Because of potential inefficiencies associated with data-plane implementation of additional MAC lookup or duplication of MAC entries, option (A) is not believed to be implementable without dataplane performance inefficiencies in some platforms and thus this draft introduces the coloring option (B) as detailed in section 3.1.”

The logic is as follow: Option A requires either X (two MAC lookup) or Y (duplication of MAC entries). Because of potential data-plane inefficiencies for X or Y (which is needed for option A), we are suggesting option B.

The whole paragraph is as below:
"Maintaining two MAC-VRFs (two bridge tables) per VLAN (when both Leaf and Root ACs exists for that VLAN) would either require two lookups be performed per MAC address in each direction in case of a miss, or duplicating many MAC addresses between the two bridge tables belonging to the same VLAN (same E-TREE instance). Unless two lookups are made, duplication of MAC addresses would be needed for both locally learned and remotely learned MAC addresses. Locally learned MAC addresses from Leaf ACs need to be duplicated onto Root bridge table and locally learned MAC addresses from Root ACs need to be duplicated onto Leaf bridge table. Remotely learned MAC addresses from Root ACs need to be copied onto both Root and Leaf bridge tables. Because of potential inefficiencies associated with data-plane implementation of additional MAC lookup or duplication of MAC entries, option (A) is not believed to be implementable without dataplane performance inefficiencies in some platforms and thus this draft introduces the coloring option (B) as detailed in section 3.1."

Cheers,
Ali

Best,

-Thomas

Hi Ali,

-Thomas

                     +---------+            +---------+
                     |   PE1   |            |   PE2   |

   Figure 2: Scenario 2

Done.

s/it is/they are/

Done.

   AC (or a Root AC)

Done.

That’s fine. Done!

Done.

Hi Ali,

Can you detail why ?

See above.

Ok.

Thanks,

-Thomas

Here is a proposal:

                     +---------+            +---------+
                     |   PE1   |            |   PE2   |

   Figure 2: Scenario 2a

                     +---------+            +---------+
                     |   PE1   |            |   PE2   |

   Figure 2: Scenario 2b

Best,

-Thomas

---

From: Thomas Morin <[email-masked]>
Date: Fri, 13 Jan 2017 21:33:09 +0100

Hi Ali,

I'm fine with the new text, thanks for the work on addressing my comments.
You can post a revision and I will wrap-up the shepherd review in parallel.

Best,

-Thomas

2017-01-13, Ali Sajassi (sajassi):
>> would be required if duplicating MAC addresses is not done, and why 
>> it is latter concluded as "not viable":
>> doing two lookups, then perhaps this hardware ability can be 
>> leveraged to support the two MAC-VRFs approach with only one lookup  
>> (building one table, marking MAC entries as Leaf entries if they were 
>> learned with routes carrying only the Leaf RT?)  --- don't 
>> misunderstand me: I'm not claiming that this works (I haven't looked 
>> closely enough), but simply that the text provided is not sufficient 
>> to exclude this kind of solution
>>
>> be as something rather belonging to the realm of "having a 
>> scalability impact", but even looking in this respect we are not 
>> talking about a change of order of magnitude.
>>
> - the logic is not consolidated: if either X or Y is required, then 
> its illogical to say "In order to avoid X, this drafts introduces 
> Z...."  (Z being different than Y)
> "Because of potential inefficiencies associated with data-plane 
> implementation of additional MAC lookup or duplication of MAC entries, 
> option (A) is not believed to be implementable without dataplane 
> performance inefficiencies in some platforms and thus this draft 
> introduces the coloring option (B) as detailed in section 3.1.”
>
> The logic is as follow: Option A requires either X (two MAC lookup) or 
> Y (duplication of MAC entries). Because of potential data-plane 
> inefficiencies for X or Y (which is needed for option A), we are 
> suggesting option B.
>
> "Maintaining two MAC-VRFs (two bridge tables) per VLAN (when both Leaf 
> and Root ACs exists for that VLAN) would either require two lookups be 
> performed per MAC address in each direction in case of a miss, or 
> duplicating many MAC addresses between the two bridge tables belonging 
> to the same VLAN (same E-TREE instance). Unless two lookups are made, 
> duplication of MAC addresses would be needed for both locally learned 
> and remotely learned MAC addresses. Locally learned MAC addresses from 
> Leaf ACs need to be duplicated onto Root bridge table and locally 
> learned MAC addresses from Root ACs need to be duplicated onto Leaf 
> bridge table. Remotely learned MAC addresses from Root ACs need to be 
> copied onto both Root and Leaf bridge tables. Because of potential 
> inefficiencies associated with data-plane implementation of additional 
> MAC lookup or duplication of MAC entries, option (A) is not believed 
> to be implementable without dataplane performance inefficiencies in 
> some platforms and thus this draft introduces the coloring option (B) 
> as detailed in section 3.1."
>
> Cheers,
> Ali
>
>
> Best,
>
> -Thomas
>
>
>
>>> approach for Leaf/Root MAC addresses is used in this draft. Also, 
>>> the use of single RT for this scenario is mentioned just as “MAY”. 
>>> Please review the text below and let me know if you still have 
>>> questions/comments:
>>
>> above, I’ll change it to “per VLAN (when both Root and Leafs ACs 
>> exist for that VLAN) requires …”.
>>
>>
>>
>>>
>> is surprising, so I guess the intent is rather "Unless two lookups 
>> are made, duplication of MAC addresses would be needed for [...]".
>>
>> would be required if duplicating MAC addresses is not done, and why 
>> it is latter concluded as "not viable":
>> doing two lookups, then perhaps this hardware ability can be 
>> leveraged to support the two MAC-VRFs approach with only one lookup 
>> (building one table, marking MAC entries as Leaf entries if they were 
>> learned with routes carrying only the Leaf RT?)  --- don't 
>> misunderstand me: I'm not claiming that this works (I haven't looked 
>> closely enough), but simply that the text provided is not sufficient 
>> to exclude this kind of solution
>>
>> be as something rather belonging to the realm of "having a 
>> scalability impact", but even looking in this respect we are not 
>> talking about a change of order of magnitude.
>>
>>>>> unicast traffic), and it also impacts section 3.2 (BUM traffic) 
>>>>> and section 5.1.
>>>>
>>>> Can you detail why ?
>>>> a 2.2.1 would not require new procotol procedures nor changes in 
>>>> the text that as I understand provides procedures for 2.2(.2) and 2.3.
>>>> section 3.1, the root/leaf indication for MAC addresses are done 
>>>> via flag-bit defined in section 5.1 and it only uses a single 
>>>> MAC-VRF (single bridge table per VLAN) per RFC 7432. If we go with 
>>>> two MAC-VRFs (e.g., two bridge tables) per VLAN, then that is an 
>>> content of the document after section 2  (more exactly, you will 
>>> need minor update like changing the current "This scheme applies to 
>>> all scenarios described in section 2." in section 3 into "This 
>>> scheme applies to scenarios described in 2.2.2 and 2.3".
>>>
>>>> understand what in RFC7432 describes an intention of "not 
>>>> supporting two MAC-VRFs for the same VLAN".
>>>>
>>>> I think you agree that if we have a single solution that has all 
>>>> the benefits of your proposed 2.2.1 and 2.2.2 and none of the 
>>>> drawbacks, it is much more preferable with having two solutions 
>>>> each with its own advantages and draw backs, right? If so, then 
>>>> existing text in 2.2 was intended to convey that. However, we can 
>>>> clarify it further – e.g, make it clear that for PE with root & 
>>>> leaf in the same EVI, we can use a single MAC-VRF with two RTs (one 
>>>> for leaf and another for root).
>>>
>>>>> VRFs for the same VLAN is not OK for L2 traffic because of 
>>>>> flooding of unknown traffic. That’s why in section 6 of RFC 7432, 
>>>>> for all service interface types, the draft talks about a single 
>>>>> MAC-VRF per EVI per PE and in case of VLAN-aware mode,  multiple 
>>>>> VLANs per MAC-VRF but only a single bridge table per VLAN. In 
>>>>> other words, the bottom line is that there can only be a single 
>>>>> bridge table per VLAN in order to avoid unnecessary flooding.
>>>>
>>>>
>>>>
>>>> should be possible to extract quotes about "there is one MAC-VRF 
>>>> per EVI on a given PE", right ?
>>>>
>>>>
>>>>>
>>>>> Ali>  Lets assume we are using vlan-based service and thus there 
>>>>> is only a single bridge table per MAC-VRF, then what you are 
>>>>> suggesting is two use two MAC-VRFs (two bridge tables) for the 
>>>>> same EVI (same VLAN). This results in some duplications of MAC 
>>>>> addresses and would only work if flooding is disabled (more on 
>>>>> this later).
>>>>
>>>> From an helicopter view, I can't see what fundamentally would 
>>>> become problematic between "two MAC-VRFs on two distinct PEs" and 
>>>> the same "two MAC-VRFs on a same PEs", at worse it is as efficient 
>>>> or as inefficient as having them on separate PEs (think logical 
>>>> router without anykind of dataplane optimisation), and we can't 
>>>> exclude that the PE could have local implementation details to do 
>>>> better than that.
>>>>
>>>>
>>>>>>
>>>>>    This approach comes at the expense of having routes for 
>>>>> unneeded MAC addresses on Leaf-only PEs, and is hence considered 
>>>>> less suitable for deployment contexts where the vast majority of 
>>>>> PEs would remain Leaf-only.    Unlike Scenario 1 and Scenario 2a, this scenario requires additional procedures
>>>>>> As the text currently reads, the cost of the recommendation can 
>>>>>> be identified: having useless routes on the fraction of PEs 
>>>>>> having only Leaves.

---

From: "Ali Sajassi (sajassi)" <[email-masked]>
Date: Fri, 13 Jan 2017 23:45:53 +0000

Hi Thomas,

Thanks very much for shepherding this draft and providing valuable comments. I have just posted the new rev with all the edits incorproated.

We can now move to the next draft in the queue - draft-ietf-bess-evpn-overlay-07<https://datatracker.ietf.org/doc/draft-ietf-bess-evpn-overlay/>.

Regards,
Ali

Date: Friday, January 13, 2017 at 12:33 PM

Hi Ali,

Best,

-Thomas

Hi Ali,

This is better, but:

Cheers,
Ali

Best,

-Thomas

Hi Ali,

-Thomas

                     +---------+            +---------+
                     |   PE1   |            |   PE2   |

   Figure 2: Scenario 2

Done.

s/it is/they are/

Done.

   AC (or a Root AC)

Done.

That’s fine. Done!

Done.

Hi Ali,

Can you detail why ?

See above.

Ok.

Thanks,

-Thomas

Here is a proposal:

                     +---------+            +---------+
                     |   PE1   |            |   PE2   |

   Figure 2: Scenario 2a

                     +---------+            +---------+
                     |   PE1   |            |   PE2   |

   Figure 2: Scenario 2b

Best,

-Thomas