]> VMware, Inc.

sboutros@vmware.com

Cisco

sajassi@cisco.com

Cisco

ssalam@cisco.com

Juniper Networks

jdrake@juniper.net

Nokia

jorge.rabadan@nokia.com

This document describes how Ethernet VPN (EVPN) can be used to support Virtual Private Wire Service (VPWS) in MPLS/IP networks. EVPN enables the following characteristics for VPWS: single-active as well as all-active multi-homing with flow-based load-balancing, eliminates the need for Pseudowire (PW) signaling, and provides fast protection convergence upon node or link failure.

This document describes how EVPN can be used to support VPWS in MPLS/IP networks. The use of EVPN mechanisms for VPWS (EVPN-VPWS) brings the benefits of EVPN to Point to Point (P2P) services. These benefits include single-active redundancy as well as all-active redundancy with flow-based load-balancing. Furthermore, the use of EVPN for VPWS eliminates the need for traditional way of PW signaling for P2P Ethernet services, as described in section 4. provides the ability to forward customer traffic to/from a given customer Attachment Circuit (AC), without any Media Access Control (MAC) lookup. This capability is ideal in providing P2P services (aka VPWS services). defines Ethernet Virtual Private Line (EVPL) service as P2P service between a pair of ACs (designated by VLANs) and Ethernet Private Line (EPL) service, in which all traffic flows are between a single pair of ports, that in EVPN terminology would mean a single pair of Ethernet Segments ES(es). EVPL can be considered as a VPWS with only two ACs. In delivering an EVPL service, the traffic forwarding capability of EVPN is based on the exchange of a pair of Ethernet Auto-discovery (A-D) routes; whereas, for more general VPWS as per , traffic forwarding capability of EVPN is based on the exchange of a group of Ethernet AD routes (one Ethernet AD route per AC/ES). In a VPWS service, the traffic from an originating Ethernet Segment can be forwarded only to a single destination Ethernet Segment; hence, no MAC lookup is needed and the MPLS label associated with the per EVPN instance (EVI) Ethernet A-D route can be used in forwarding user traffic to the destination AC. For both EPL and EVPL services, a specific VPWS service instance is identified by a pair of per-EVI Ethernet A-D routes which together identify the VPWS service instance endpoints and the VPWS service instance. In the control plane the VPWS service instance is identified using the VPWS service instance identifiers advertised by each Provider Edge node (PE). In the data plane the value of the MPLS label advertised by one PE is used by the other PE to send traffic for that VPWS service instance. As with the Ethernet Tag in standard EVPN, the VPWS service instance identifier has uniqueness within an EVPN instance. For EVPN routes, the Ethernet Tag IDs are set to zero for Port-based, VLAN-based, and VLAN-bundle interface mode and set to non-zero Ethernet Tag IDs for VLAN-aware bundle mode. Conversely, for EVPN- VPWS, the Ethernet Tag ID in the Ethernet A-D route MUST be set to a non-zero value for all four service interface types. In terms of route advertisement and MPLS label lookup behavior, EVPN- VPWS resembles the VLAN-aware bundle mode of such that when a PE advertises per-EVI Ethernet A-D route, the VPWS service instance serves as a 32-bit normalized Ethernet Tag ID. The value of the MPLS label in this route represents both the EVI and the VPWS service instance, so that upon receiving an MPLS encapsulated packet, the disposition PE can identify the egress AC from the MPLS label and subsequently perform any required tag translation. For EVPL service, the Ethernet frames transported over an MPLS/IP network SHOULD remain tagged with the originating VLAN-ID (VID) and any VID translation MUST be performed at the disposition PE. For EPL service, the Ethernet frames are transported as is and the tags are not altered. The MPLS label value in the Ethernet A-D route can be set to the Virtual Extensible LAN (VXLAN) Network Identifier (VNI) for VXLAN encapsulation as per , and this VNI will have a local scope per PE and may also be equal to the VPWS service instance identifier set in the Ethernet A-D route. When using VXLAN encap, the BGP Encapsulation extended community is included in the Ethernet A-D route as described in [I-D.ietf-evpn-overlay]. The VXLAN VNI like the MPLS label that will be set in the tunnel header used to tunnel Ethernet packets from all the service interface types defined in section 2. The EVPN-VPWS techniques defined in this document has no dependency on the tunneling technology. The Ethernet Segment identifier encoded in the Ethernet A-D per-EVI route is not used to identify the service. However it can be used for flow-based load-balancing and mass withdraw functions as per the baseline. As with standard EVPN, the Ethernet A-D per-ES route is used for fast convergence upon link or node failure. The Ethernet Segment route is used for auto-discovery of the PEs attached to a given multi-homed Customer Edge node (CE) and to synchronize state between them.

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 . EVPN: Ethernet VPN MAC: Media Access Control MPLS: Multi Protocol Label Switching. OAM: Operations, Administration and Maintenance. PE: Provide Edge Node. ASBR: Autonomous System Border Router CE: Customer Edge device e.g., host or router or switch. EVPL: Ethernet Virtual Private Line. EPL: Ethernet Private Line. EP-LAN: Ethernet Private LAN. EVP-LAN: Ethernet Virtual Private LAN. S-VLAN: Service VLAN identifier. C-VLAN: Customer VLAN identifier. VID: VLAN-ID. VPWS: Virtual Private Wire Service. EVI: EVPN Instance. P2P: Point to Point. VXLAN: Virtual Extensible LAN. DF: Designated Forwarder. L2: Layer 2. MTU: Maximum Transmission Unit. eBGP: Exterior Border Gateway Protocol. iBGP: Internal Border Gateway Protocol. ES: Ethernet Segment on a PE refers to the link attached to it, this link can be part of a set of links attached to different PEs in multi homed cases, or could be a single link in single homed cases. ESI: Ethernet Segment Identifier. Single-Active Mode: When a device or a network is multi-homed to two or more PEs and when only a single PE in such redundancy group can forward traffic to/from the multi-homed device or network for a given VLAN, then such multi-homing or redundancy is referred to as "Single-Active". All-Active: When a device is multi-homed to two or more PEs and when all PEs in such redundancy group can forward traffic to/from the multi-homed device for a given VLAN, then such multi-homing or redundancy is referred to as "All-Active". VPWS Service Instance: It is represented by a pair of EVPN service labels associated with a pair of endpoints. Each label is downstream assigned and advertised by the disposition PE through an Ethernet A-D per-EVI route. The downstream label identifies the endpoint on the disposition PE. A VPWS service instance can be associated with only one VPWS service identifier.

With this service interface, a VPWS instance identifier corresponds to only a single VLAN on a specific interface. Therefore, there is a one-to-one mapping between a VID on this interface and the VPWS service instance identifier. The PE provides the cross-connect functionality between an MPLS LSP identified by the VPWS service instance identifier and a specific <port,VLAN>. If the VLAN is represented by different VIDs on different PEs and different ES(es), (e.g., a different VID per Ethernet segment per PE), then each PE needs to perform VID translation for frames destined to its Ethernet segment. In such scenarios, the Ethernet frames transported over an MPLS/IP network SHOULD remain tagged with the originating VID, and a VID translation MUST be supported in the data path and MUST be performed on the disposition PE.

With this service interface, a VPWS service instance identifier corresponds to multiple VLANs on a specific interface. The PE provides the cross-connect functionality between the MPLS label identified by the VPWS service instance identifier and a group of VLANs on a specific interface. For this service interface, each VLAN is presented by a single VID which means no VLAN translation is allowed. The receiving PE, can direct the traffic based on EVPN label alone to a specific port. The transmitting PE can cross-connect traffic from a group of VLANs on a specific port to the MPLS label. The MPLS-encapsulated frames MUST remain tagged with the originating VID.

This service interface is a special case of the VLAN bundle service interface, where all of the VLANs on the port are mapped to the same VPWS service instance identifier. The procedures are identical to those described in .

Contrary to EVPN, in EVPN-VPWS this service interface maps to a VLAN- based service interface (defined in section 2.1) and thus this service interface is not used in EVPN-VPWS. In other words, if one tries to define data plane and control plane behavior for this service interface, one would realize that it is the same as that of VLAN-based service.

This document specifies the use of the per-EVI Ethernet A-D route to signal VPWS services. The Ethernet Segment Identifier field is set to the customer ES and the Ethernet Tag ID 32-bit field MUST be set to the VPWS service instance identifier value. The VPWS service instance identifier value MAY be set to a 24-bit value and when a 24-bit value is used, it MUST be right aligned. For both EPL and EVPL services using a given VPWS service instance, the pair of PEs instantiating that VPWS service instance will each advertise a per-EVI Ethernet A-D route with its VPWS service instance identifier and will each be configured with the other PE's VPWS service instance identifier. When each PE has received the other PE's per-EVI Ethernet A-D route, the VPWS service instance is instantiated. It should be noted that the same VPWS service instance identifier may be configured on both PEs. The Route-Target (RT) extended community with which the per-EVI Ethernet A-D route is tagged identifies the EVPN instance in which the VPWS service instance is configured. It is the operator's choice as to how many and which VPWS service instances are configured in a given EVPN instance. However, a given EVPN instance MUST NOT be configured with both VPWS service instances and standard EVPN multi- point services.

This document defines a new extended community , to be included with per-EVI Ethernet A-D routes. This attribute is mandatory if multihoming is enabled. Type(0x06)/Sub-type(0x04)(2 octet) Control Flags (2 octets) L2 MTU (2 octets) Reserved (2 octets)

The following bits in the Control Flags are defined; the remaining bits MUST be set to zero when sending and MUST be ignored when receiving this community. Meaning P If set to 1 in multihoming single-active scenarios, it indicates that the advertising PE is the Primary PE. MUST be set to 1 for multihoming all-active scenarios by all active PE(s). B If set to 1 in multihoming single-active scenarios, it indicates that the advertising PE is the Backup PE. C If set to 1, a Control word MUST be present when sending EVPN packets to this PE. It is recommended to include the control word in the absence of Entropy Label. L2 MTU (Maximum Transmission Unit) is a 2-octet value indicating the MTU in bytes. A received L2 MTU of zero means no MTU checking against local MTU is needed. A received non-zero MTU MUST be checked against local MTU and if there is a mismatch, the local PE MUST NOT add the remote PE as the EVPN destination for the corresponding VPWS service instance. The usage of the Per ES Ethernet A-D route is unchanged from its usage in , i.e., the "Single-Active" bit in the flags of the ESI Label extended community will indicate if single-active or all- active redundancy is used for this ES. In multihoming scenarios, the B and P flags MUST be cleared. A PE that receives an update with both B and P flags set MUST treat the route as a withdrawal. If the PE receives a route with both B and P clear, it MUST treat the route as a withdrawal from the sender PE. In a multihoming all-active scenario, there is no Designated Forwarder (DF) election, and all the PEs in the ES that are active and ready to forward traffic to/from the CE will set the P Flag. A remote PE will do per-flow load-balancing to the PEs that set the P Flag for the same Ethernet Tag and ESI. The B Flag in control flags SHOULD NOT be set in the multihoming all-active scenario and MUST be ignored by receiving PE(s) if set. In multihoming single-active scenario for a given VPWS service instance, the DF election should result in the Primary-elected PE for the VPWS service instance advertising the P Flag set and the B Flag clear, the Backup elected PE should advertise the P Flag clear and the B Flag set, and the rest of the PEs in the same ES should signal both P and B Flags clear. When the primary PE/ES fails, the primary PE will withdraw the associated Ethernet A-D routes for the VPWS service instance from the remote PE and the remote PEs should then send traffic associated with the VPWS instance to the backup PE. DF re-election will happen between the PE(s) in the same ES, and there will be a newly elected primary PE and newly elected backup PE that will signal the P and B Flags as described. A remote PE SHOULD receive the P Flag set from only one Primary PE and the B Flag set from only one Backup PE. However during transient situations, a remote PE receiving a P Flag set from more than one PE will select the last advertising PE as the primary PE when forwarding traffic. A remote PE receiving a B Flag set from more than one PE will select the last advertising PE as the backup PE. A remote PE MUST receive P Flag set from at least one PE before forwarding traffic. If a network uses entropy labels per then the C Flag MUST NOT be set and control word MUST NOT be used when sending EVPN- encapsulated packets over a P2P LSP.

| Native Service | | Service (AC) | |<-PSN1->| |<-PSN2->| | (AC) | V V V V V V | | +-----+ +-----+ +-----+ +-----+ | +----+ | | PE1 |======|ASBR1|==|ASBR2|===| PE3 | | +----+ | |-------+-----+ +-----+ +-----+ +-----+-------| | | CE1| | | |CE2 | | |-------+-----+ +-----+ +-----+ +-----+-------| | +----+ | | PE2 |======|ASBR3|==|ASBR4|===| PE4 | | +----+ ^ +-----+ +-----+ +-----+ +-----+ ^ | Provider Edge 1 ^ Provider Edge 2 | | | | | | | | EVPN Inter-provider point | | | |<---------------- Emulated Service -------------------->| ]]>

iBGP sessions are established between PE1, PE2, ASBR1 and ASBR3, possibly via a BGP route-reflector. Similarly, iBGP sessions are established between PE3, PE4, ASBR2 and ASBR4. eBGP sessions are established among ASBR1, ASBR2, ASBR3, and ASBR4. All PEs and ASBRs are enabled for the EVPN SAFI and exchange per-EVI Ethernet A-D routes, one route per VPWS service instance. For inter- AS option B, the ASBRs re-advertise these routes with the NEXT_HOP attribute set to their IP addresses as per . The link between the CE and the PE is either a C-tagged or S-tagged interface, as described in [802.1Q], that can carry a single VLAN tag or two nested VLAN tags and it is configured as a trunk with multiple VLANs, one per VPWS service instance. It should be noted that the VLAN ID used by the customer at either end of a VPWS service instance to identify that service instance may be different and EVPN doesn't perform that translation between the two values. Rather, the MPLS label will identify the VPWS service instance and if translation is needed, it should be done by the Ethernet interface for each service. For single-homed CE, in an advertised per-EVI Ethernet A-D route the ESI field is set to 0 and the Ethernet Tag ID is set to the VPWS service instance identifier that identifies the EVPL or EPL service. For a multi-homed CE, in an advertised per-EVI Ethernet A-D route the ESI field is set to the CE's ESI and the Ethernet Tag ID is set to the VPWS service instance identifier, which MUST have the same value on all PEs attached to that ES. This allows an ingress PE in a multihoming all-active scenario to perform flow-based load-balancing of traffic flows to all of the PEs attached to that ES. In all cases traffic follows the transport paths, which may be asymmetric. The VPWS service instance identifier encoded in the Ethernet Tag ID in an advertised per-EVI Ethernet A-D route MUST either be unique across all ASs, or an ASBR needs to perform a translation when the per-EVI Ethernet A-D route is re-advertised by the ASBR from one AS to the other AS. A per-ES Ethernet A-D route can be used for mass withdraw to withdraw all per-EVI Ethernet A-D routes associated with the multi-home site on a given PE.

In EVPN, service endpoint discovery and label signaling are done concurrently using BGP. Whereas, with VPWS based on , label signaling is done via LDP and service endpoint discovery is either through manual provisioning or through BGP. In existing implementations of VPWS using pseudowires(PWs), redundancy is limited to single-active mode, while with EVPN implementation of VPWS both single-active and all-active redundancy modes can be supported. In existing implementations with PWs, backup PWs are not used to carry traffic, while with EVPN, traffic can be load-balanced among different PEs multi-homed to a single CE. Upon link or node failure, EVPN can trigger failover with the withdrawal of a single BGP route per EVPL service or multiple EVPL services, whereas with VPWS PW redundancy, the failover sequence requires exchange of two control plane messages: one message to deactivate the group of primary PWs and a second message to activate the group of backup PWs associated with the access link. Finally, EVPN may employ data plane egress link protection mechanisms not available in VPWS. This can be done by the primary PE (on local AC down) using the label advertised in the per-EVI Ethernet A-D route by the backup PE to encapsulate the traffic and direct it to the backup PE.

On a link or port failure between the CE and the PE for both single and multi-homed CEs, unlike the PE MUST withdraw all the associated Ethernet A-D routes for the VPWS service instances on the failed port or link.

EVPN-VPWS uses Ethernet A-D route advertisements for single-homed Ethernet Segments. Therefore, upon a link/port failure of this single-homed Ethernet Segment, the PE MUST withdraw the associated per-EVI Ethernet A-D routes.

For a faster convergence in multi-homed scenarios with either Single- Active Redundancy or All-active redundancy, a mass withdraw technique is used. A PE previously advertising a per-ES Ethernet A-D route, can withdraw this route by signaling to the remote PEs to switch all the VPWS service instances associated with this multi-homed ES to the backup PE.

The authors would like to acknowledge Jeffrey Zhang, Wen Lin, Nitin Singh, Senthil Sathappan, Vinod Prabhu, Himanshu Shah, Iftekhar Hussain, Alvaro Retana and Acee Lindem for their feedback and contributions to this document.

The mechanisms in this document use EVPN control plane as defined in . Security considerations described in are equally applicable. This document uses MPLS and IP-based tunnel technologies to support data plane transport. Security considerations described in and in [I-D.ietf-evpn-overlay] are equally applicable.

This document creates a registry called "EVPN Layer 2 Attributes Control Flags". New registrations will be made through the "RFC Required" procedure defined in .

&RFC2119; &RFC7432; &RFC4448; &RFC4271; &RFC4360; &RFC5226; &RFC7348; Metro Ethernet Forum &RFC4664;

In addition to the authors listed on the front page, the following co-authors have also contributed to this document: Daniel Voyer Bell Canada