Subject: RE: Discussion on forwarding performance

From: "Sam Cao" <[email-masked]>
Date: Tue, 12 Jun 2012 14:19:36 +0800

Dave and all,

Sorry for later reply and thank you very much for your comments.

Based on your comments I assume you believe that the scope of the S-tag is
only the PW section, so it is imposed by the ingress PE and removed by the
egress PE?
[Sam] Yes. My understanding on this is, yes. Or just as Yuanlong said, VLAN
mapping (S-tag is imposed by ingress PE) may be processed on a logical port.

And you would prefer recreating an S-tag if an egress PE had a subtending
PBN and stripping it if it in is an ingress from a PBN?
[Sam] Dave, no, my understanding is, VLAN ID (Or S-tag you said) is useful
and negotiated between ingress PE and egress PE. If it is striped out,
egress PE will forward the frame to the corresponding ACs. So don't need to
recreate new S-tag.

Sam

-----Original Message-----
From: David Allan I [mailto:[email-masked]] 
Sent: Thursday, May 17, 2012 12:21 AM
To: Sam Cao; 'Lucy yong'; 'Balus, Florin Stelian (Florin)'; 'Jiangyuanlong';
'Daniel Cohn'; 'Alexander Vainshtein'
Cc: [email-masked]
Subject: RE: Discussion on forwarding performance

Hi Sam:

Thanks for explaining your reasoning, this is useful.

Based on your comments I assume you believe that the scope of the S-tag is
only the PW section, so it is imposed by the ingress PE and removed by the
egress PE?

And you would prefer recreating an S-tag if an egress PE had a subtending
PBN and stripping it if it in is an ingress from a PBN?

So this has little to do with ETREE itself, but tag interworking with a PBN
or not?

Thanks
Dave

-----Original Message-----
From: Sam Cao [mailto:[email-masked]]
Sent: Tuesday, May 15, 2012 7:09 PM
To: 'Lucy yong'; David Allan I; 'Balus, Florin Stelian (Florin)';
'Jiangyuanlong'; 'Daniel Cohn'; 'Alexander Vainshtein'
Cc: [email-masked]
Subject: RE: Discussion on forwarding performance

Lucy/David/Florin,

Thank you very much for your comments.

I can not give the accurate data to support my conclusion, and just conclude
that if we implement it in NP module. Except for common operations of 2
approaches, Tunnel Label push/pop-out, MAC-based forwarding, and etc.,
Dual-VLAN needs one more VLAN push/pop-out operation while forwarding every
frame. So compared with Multi-PW, it needs more resource and time for NP
module if there are VSIs or PWs up limit to the capacity on one PE, and
standing on software architect's side, the performance will be decreased by
at least 20%, I guess. I implemented one prototype which has similar
solution as current Multi-PW, and the performance will be decreased by 5%,
compared with traditional VPLS. Daniel has experience on 2-PW deployment,
maybe he can give accurate data on Multi-PW.

Florin, do you mean that you have implemented E-Tree in MPLS network? If
chipset can support this and we don't care NP or other solutions, yes, we
can ignore any side effect on forwarding performance. Yes, it can support
E-tree in IP (Ethernet) network, but I read some CHIP datasheets, after
strip PW label out, chip can not handle 2 VLAN IDs at same time. Could you
please give me some hints, say, which chip you used?

Maybe if we raised only one issue in one mail, we can get more and insight
comments :). I will raise another after we agree with this in the main :).

Thanks,

Sam

-----Original Message-----
From: Lucy yong [mailto:[email-masked]]
Sent: Wednesday, May 16, 2012 4:15 AM
To: David Allan I; Balus, Florin Stelian (Florin); Sam Cao; Jiangyuanlong;
'Daniel Cohn'; 'Alexander Vainshtein'
Cc: [email-masked]
Subject: RE: Discussion on forwarding performance

+1

We can't just make a conclusion from nowhere.

Lucy

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
David Allan I
Sent: Tuesday, May 15, 2012 2:55 PM
To: Balus, Florin Stelian (Florin); Sam Cao; Jiangyuanlong; 'Daniel Cohn';
'Alexander Vainshtein'
Cc: [email-masked]
Subject: RE: Discussion on forwarding performance

Florin:

I agree. The statement, at least to me, is a non-sequitor...

Cheers
Dave

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
Balus, Florin Stelian (Florin)
Sent: Tuesday, May 15, 2012 12:53 PM
To: Sam Cao; 'Jiangyuanlong'; 'Daniel Cohn'; 'Alexander Vainshtein'
Cc: [email-masked]
Subject: RE: Discussion on forwarding performance

Sam,

> Obviously the forwarding performance of Dual-VLAN will be much lower
> than
what of Multi-PW.

Sorry I could not keep up with some of the threads on this subject but the
above statement caught my eyes. We have been doing all kind of egress
operations on VLANs in the egress PE for the last 8 years and I am not aware
of any "much lower" forwarding performance. As far as I know the chipsets
used in the PEs can do this processing no problem. Do you want to clarify
which kind of hardware may have problems?

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf
> Of Sam Cao
> Sent: Tuesday, May 15, 2012 6:32 AM
> To: 'Jiangyuanlong'; 'Daniel Cohn'; 'Alexander Vainshtein'
> Cc: [email-masked]
> Subject: RE: Discussion on forwarding performance
>
> Hi Yuanlong,
>
> 3rd mapping will make data plane complex, I think. Anyway, we reached
> a consensus on Multi-PW implementation.
>
> If we use 3rd mapping, I agree with your opinion, maybe dual-VLAN
> seems simple, but 2 mapping is enough. Then we can focus on data plane
> performance. While stripe PW label out on egress PE, data-plane knows
> how to forward the frames from PW, to Root AC or Leaf AC or all. But
> if we use Dual-VLAN, after strip PW label out, data plane should strip
> VLAN-ID out and then does same forwarding work as Multi-PW does. The
> most important is, do one more operation while forwarding E-Tree
> frames. Obviously the forwarding performance of Dual-VLAN will be much
> lower than what of Multi-PW.
>
> Regards,
>
> Yuqun (Sam) Cao
> E-mail: [email-masked]
>
> -----Original Message-----
> From: Jiangyuanlong [mailto:[email-masked]]
> Sent: Tuesday, May 15, 2012 7:03 PM
> To: Sam Cao; 'Daniel Cohn'; 'Alexander Vainshtein'
> Cc: [email-masked]
> Subject: RE: Discussion on E-Tree and H-VPLS
>
> See my further comments in line.
>
> -----Original Message-----
> From: Sam Cao [mailto:[email-masked]]
> Sent: Tuesday, May 15, 2012 6:14 PM
> To: Jiangyuanlong; 'Daniel Cohn'; 'Alexander Vainshtein'
> Cc: [email-masked]
> Subject: RE: Discussion on E-Tree and H-VPLS
>
> Hi Yuanlong,
>
> Thank you very much for your comments. I draw the topology you gave,
> and 7 PWs will be established if we follow 01.
>
> Root AC ---- PE 1 ----- PW 1 -------- PE 2 ----- Leaf _AC
>                |  \                 /  ||
>                |   \  /===PW 6,7===    ||
>                 PW 2  -----PW 5------\  ||
>                |  /----Root PW 3 --- \ ||
> Root _AC ---- PE 3                    PE 4 ------ Root AC
>                |   \                  /  |
>                |    ----Leaf PW 4-----   |
>             Leaf AC                    Leaf AC
> Yes, on PE 3 there are several PWs, but if you want to clarify it,
> there are only 3 types, one can carry frames originated from Root and
> Leaf AC(one endpoint of this PW should be Root-only, otherwise this is
> invalid), one only can carry frames from Root AC, and the last can
> carry frames from Leaf ACs. On second thoughts, we still can classify
> the PWs into 2 sets, one is Leaf set, and another is Root set, and the
> union of the sets Leaf and Root is the PW we called it as compatible
> PW (Maybe this is not correct, we can find one good term on this).
>
> So this optimization still follows the original design of Dual-PW
> approach, Leaf PW only carries frames from Leaf AC; Root-PW only
> carries frames from Root AC. Is it right?
>
> I don't know whether chip can support this forwarding behavior for
> compatible PW or not, but if we implement it in NP, it is nearly same
> as VPLS. The only difference is, setup 2 mappings, one between
> Leaf-ACs and Leaf-PWs, one between Root-ACs and Root-PWs. For
> traditional VPLS, there is only one mapping.
>
> [JY] It seems you may need a 3rd mapping: from both root & leaf AC to
> a compatible PW.
> BTW, for the forwarding and reverse direction, the mapping may be
> asymmetric for you compatible PW.
>
> Based on my understanding, all drafts should have similar mapping.
>
> [JY] Dual-VLAN seems simpler here.
>
> Thanks,
>
> Sam
>
>
> -----Original Message-----
> From: Jiangyuanlong [mailto:[email-masked]]
> Sent: Tuesday, May 15, 2012 3:32 PM
> To: Sam Cao; 'Daniel Cohn'; 'Alexander Vainshtein'
> Cc: [email-masked]
> Subject: RE: Discussion on E-Tree and H-VPLS
>
> Sam,
>
> Thanks, please see my further comments with [JY].
>
> Ok, we go on the case in your mail, where PE1 has one Root-only AC,
> AC1, and
> PE2 has one Leaf-only AC, AC2. In general we can setup 2
> "unidirectional"
> PWs (bidirectional PW, but just carry unidirectional frames), Root-PW
> which carries frames originated from AC1 and Leaf-PW which carries
> frames originated from AC2. But only one PW also can work, just as
> some members commented. Ok, we setup one PW between PE1 and PE2, we
> can call this PW as compatible PW or something else. Frames originated
> from Root or leaf ACs will be carried via this PW. Its forwarding
> behavior is fully same as what traditional VPLS did, MAC learning on
> AC or PW in one E-Tree. Or say, if one PE has Root-only ACs, it will
> setup one PW with another PE, Root-only, Leaf-only or Root-Leaf-Mixed.
> If 2 PEs are root-Leaf-Mixed or other cases, then 2 PWs will be
> established. As you know, this can be done on control plane.
>
> [JY] Assume there are 2 other nodes PE3 & PE4 in this network
> scenario, and both PE3 and PE4 have both root and leaf ACs, then there
> are compatible PWs from PE3 which may carry both root and leaf traffic
> (to PE1), which may carry only root traffic (to PE2); and further
> there are root PW and leaf PW to PE4. Thus, the VSI on PE3 has at
> least 4 different types of PW transmitting behaviours with regard to
> the E-Tree traffic. In the reverse direction, the VSI on PE3 further
> has at least
> 4 different types of PW receiving behaviours. Not sure how you will
> accommodate for these PWs in both the data plane and control plane.
>
> Regards,
> Yuanlong
>
> Is this clear for you? Is this necessary to optimize PW setup in this
> case?
>
> Maybe we need to add one paragraph on forwarding behavior.
>
> Again thank you very much for your comments,
>
> Sam
>
> -----Original Message-----
> From: Jiangyuanlong [mailto:[email-masked]]
> Sent: Tuesday, May 15, 2012 10:27 AM
> To: Sam Cao; 'Daniel Cohn'; 'Alexander Vainshtein'
> Cc: [email-masked]
> Subject: RE: Discussion on E-Tree and H-VPLS
>
> Please see my comments in line.
>
> -----Original Message-----
> From: Sam Cao [mailto:[email-masked]]
> Sent: Monday, May 14, 2012 8:31 PM
> To: Jiangyuanlong; 'Daniel Cohn'; 'Alexander Vainshtein'
> Cc: [email-masked]
> Subject: RE: Discussion on E-Tree and H-VPLS
>
> Hi Yuanlong,
>
> We have discussed this in another thread. In fact, Daniel or I have
> given the answers to the questions you raised here for several time :).
> If we do in this way, we will reach deadlock and can not move forward
> :). I try to explain it again.
>
> [JY] I will apologize if you had ever provided such answers before,
> and you may just refer to the links rather than repeat the explanation.
> As you can see, I just try to understand the mechanism of 2PW and its
> implications, but the I-D does not include enough information.
>
> As you know, initial design we will setup 2 PWs all the time (if do in
> this way, I think that you have no question),
>
> [JY] I will be concerned with the operational complexity of this
> approach.
>
> but in most cases 2 PWs are not
> necessary. So in 01 version, we optimize it.
>
> "Root-only VSI <-> any VSI: only root PW required"
> "Leaf-only VSI <-> leaf-only VSI: no PWs required"
>
> In other cases 2 PWs are needed. If so, "Leaf-only -> root-only = one
> leaf PW" is not correct: On Leaf-only side, yes, one Leaf-only PW is
> created between Local Leaf type with remote Root type; on root-only
> side, one root-only PW is created. Maybe it is better to call this as
> compatible or mixed PW. So the left items you list are not correct for
> multi_PW solution.
>
> [JY] This is something new, right? To be honest, I could not
> understand what is your point: doesn't the mixed PW as you called also
> consist of one Leaf-only PW in one direction and one root-only PW in
> the other direction?
> Furthermore, the case you took is also one case in your guideline
> "Root-only VSI <-> any VSI: only root PW required", don't you think
> that only one root PW is required following this guideline?
> Actually, I don't care much about what is the bidirectional PW or
> unidirectional PW called, but how can a VSI support such a mixed
> scenarios:
> traditional VSI assumes only one PW is required for each peer VSI, and
> its MAC leaning is based on bidirectional PW. But for 2PW, there are
> bidirectional root PW, bidirectional leaf PW, mixed PW, etc..., how to
> support these kinds of PWs in the same VSI is my top concern.
>
> BTW, I guess you also care this case we also have discussed before:
> for example, if we configure one Leaf AC on root-only PE (then it will
> be root-leaf-mixed PE), we will teardown the compatible PW and
> re-setup PWs again with 01.
> [JY] This was discussed in the emails indeed.
>
> Regards,
>
> Yuqun (Sam) Cao
> E-mail: [email-masked]
>
> -----Original Message-----
> From: Jiangyuanlong [mailto:[email-masked]]
> Sent: Monday, May 14, 2012 7:10 PM
> To: Daniel Cohn; Alexander Vainshtein; Sam Cao
> Cc: [email-masked]
> Subject: RE: Discussion on E-Tree and H-VPLS
>
> Hi Daniel,
>
> Fine, now it seems more in line with what we had proposed in 2VLAN: a
> spoke PW behaves like root/leaf AC for a PE-r.
>
> But the same problem may also apply to the root/spoke "core PW" for
> Multi-PW:
> According to Multi-PW, only one PW is required between two PEs except
> when both PEs are mixed with root and leaf, so these PWs may be formed
> by combinations of the following unidirectional PW cases (extracted
> and adapted from Josh's email):
> Root-only -> any VSI = one root PW
> Leaf-only -> root-only = one leaf PW
> Leaf-only -> mixed = one leaf PW
> Mixed -> root-only = one (root+leaf) PW Mixed -> leaf-only = one
> (root+leaf) PW
>
> I have a concern that the forwarding plane of PE to implement this
> will be very different from the traditional VPLS.
>
> Regards,
> Yuanlong
>
> -----Original Message-----
> From: Daniel Cohn [mailto:[email-masked]]
> Sent: Monday, May 14, 2012 5:00 PM
> To: Jiangyuanlong; Alexander Vainshtein; Sam Cao
> Cc: [email-masked]
> Subject: RE: Discussion on E-Tree and H-VPLS
>
> Hi Yuanlong,
>
> Like I wrote below, root/leaf spoke PW behave like root/leaf AC, not
> like root/spoke "core PW". So no changes to forwarding plane once this
> is understood.
>
> DC
>
> -----Original Message-----
> From: Jiangyuanlong [mailto:[email-masked]]
> Sent: Monday, May 14, 2012 11:36 AM
> To: Daniel Cohn; Alexander Vainshtein; Sam Cao
> Cc: [email-masked]
> Subject: RE: Discussion on E-Tree and H-VPLS
>
> Daniel, please see my comments in line.
>
> -----Original Message-----
> From: Daniel Cohn [mailto:[email-masked]]
> Sent: Monday, May 14, 2012 3:37 PM
> To: Jiangyuanlong; Alexander Vainshtein; Sam Cao
> Cc: [email-masked]
> Subject: RE: Discussion on E-Tree and H-VPLS
>
> Hi Yuanlong,
>
> As I see it, for the PE-r spoke (figure 4 in RFC 4762) we establish a
> single PW per AC - root PW for root AC and leaf PW for leaf AC. With
> local configuration at the PE-rs as part of the spoke PW provisioning.
> And the PE-rs will treat root/leaf spoke PWs exactly as it treat
> root/leaf ACs. So when leaf-originated BUM traffic arrives from the
> core (over a leaf PW), it will be forwarded over all root spoke PWs
> but not on the leaf spoke PWs.
>
> [JY] So in the reverse direction, you need to transport both root and
> leaf traffic from PE-rs over a root PW to PE-r, and transport root
> traffic over a leaf PW. It seems against the definition of root PW and
> leaf PW in the multi-PW draft. Don't this make the forwarding plane of
> PE-rs more complex?
>
> The forwarding plane is exactly the same as described in the multi-PW
> draft, where spoke PWs are treated exactly like ACs.
>
> Regards,
>
> Daniel
>
>
> -----Original Message-----
> From: Jiangyuanlong [mailto:[email-masked]]
> Sent: Friday, May 11, 2012 5:03 AM
> To: Daniel Cohn; Alexander Vainshtein; Sam Cao
> Cc: [email-masked]
> Subject: RE: Discussion on E-Tree and H-VPLS
>
> Hi Daniel and all,
>
> When you set up two PWs from the PE-rs to a PE-r for each of its AC,
> do you mean that one PW is bidirectional root PW and the other is
> bidirectional leaf PW?
> My concern is: where should the leaf traffic be filtered, on the PE-rs
> or on the PE-r?
> If filtered on the PE-r, then lots of bandwidth will be wasted (for
> example, if the PE-r is attached with 9 leafs, then the BUM traffic
> from one of its leafs will be multiplied by 8 times, and be forwarded
> by the PE-rs to the same PE-r).
> If filtered on the PE-rs, not sure how you will design its forwarding
> plane, can you give a hint?
>
> Thanks,
> Yuanlong
>
> ------------------------------
> Date: Wed, 9 May 2012 09:16:36 +0300
> From: "Daniel Cohn" <[email-masked]>
> To: "Alexander Vainshtein" <[email-masked]>,        "Sam
>       Cao" <[email-masked]>, <[email-masked]>
> Cc: [email-masked]
> Subject: RE: Discussion on E-Tree and H-VPLS
> Message-ID: <44F4E579A764584EA9BDFD07D0CA08130780CBA6@tlvmail1>
> Content-Type: text/plain;     charset="ISO-2022-JP"
>
> Hi Sasha,
>
> It's actually very simple. A frame that originated in a root AC is
> always transmitted only in root PW, no matter what the frame type
> (known/unknown unicast or broadcast). This is how the frame source
> information is propagated across the VPLS. So in the H-VPLS example,
> the PE-r will never forward a frame received over a root PW on any
> leaf PW, only on root PWs (toward the core or toward other spokes).
>
> Hope this clarifies it, regards,
>
> Daniel
>
> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf
> Of Alexander Vainshtein
> Sent: Wednesday, May 09, 2012 7:59 AM
> To: Sam Cao; [email-masked]
> Cc: [email-masked]
> Subject: RE: Discussion on E-Tree and H-VPLS
>
> Sam, Lizhong and all,
> You've written that in the two-PW solution combined with H-VPLS the
> PE- r must set up two PWs with each MTU-s.
>
> If this is the case, what kind of forwarding logic should be used to
> prevent sending a BUM frame received from a root-PW to a given MTU-S:
> - back to the same MTU-S on the corresponding leaf-PW?
> - twice (to both root-PW and Leaf-PW) to another MTU-s? And, BTW, how
> is the PW selected in this case?
>
> I am aware of a technique of multiple split horizon groups in PE-r
> which could be possibly used for this purpose. However, since these
> groups have to be represented explicitly in the data plane, their
> potential number is limited by the forwarding HW. Other methods could
> be probably used for the same purpose, but they would probably subject
> to similar HW-based limitations.
>
> I suspect that with the dual-PW approach, the HW would pose an
> implicit limit, say, on a number of MTU-s that can be connected to the
> same PE-r and that this limit could be quite low in most cases.
>
> Based on this I suspect that the H-VPLS issue as a critical drawback
> of the dual-PW approach to E-Tree.
>
> My 2c,
>      Sasha
>
>
> ________________________________________
> From: [email-masked] [[email-masked]] on behalf of Sam
> Cao [[email-masked]]
> Sent: Wednesday, May 09, 2012 3:40 AM
> To: [email-masked]
> Cc: [email-masked]
> Subject: RE: Discussion on E-Tree and H-VPLS
>
> Hi Lizhong?
>
> Thank you very much for your comments. I updated the result on this
> question.
>
> [Lizhong] agree with the above analysis. And we did not say it is a
> technical problem, but it is an operational problem again.
> [Sam] I agree. Dual-VLAN does not make sense. I also discussed this
> with Giles and Yuanlong in another mail, and we reached agreement on
> this:
> MTU should know the access mode, VPWS or VPLS, VPWS mode should
> configure VLAN ID on MTU but VPLS mode can not. I thought this is NOT
> reasonable :), but we can figure this out in draft. It seems ok.
>
> [Lizhong] not fully understand. Do you mean,  when VPWS accessing for
> H-VPLS, the PE-r is also necessary to configure two PWs (root and leaf
> PW) for each AC access?
> [Sam] Yes.
>
> Thanks,
>
> Sam
>
>
>
>

---

From: "Rogers, Josh" <[email-masked]>
Date: Tue, 12 Jun 2012 08:52:38 -0400

I was under the impression that multi-pw method supported both ethernet
tagged mode (0x0004), and ethernet raw mode (0x0005), where 2vlan
supported only ethernet tagged mode (0x0004) ?  Is that correct?

>> plane, can you give a hint?

This E-mail and any of its attachments may contain Time Warner Cable proprietary information, which is privileged, confidential, or subject to copyright belonging to Time Warner Cable. This E-mail is intended solely for the use of the individual or entity to which it is addressed. If you are not the intended recipient of this E-mail, you are hereby notified that any dissemination, distribution, copying, or action taken in relation to the contents of and attachments to this E-mail is strictly prohibited and may be unlawful. If you have received this E-mail in error, please notify the sender immediately and permanently delete the original and any copy of this E-mail and any printout.

---

From: Jiangyuanlong <[email-masked]>
Date: Wed, 13 Jun 2012 01:42:40 +0000

For some special scenarios, Dual-VLAN may use a PW in raw mode. For example, when the PE is working in Compatible mode.