]> Juniper Networks Inc.

Exora Business Park Bangalore KA 560103 India shraddha@juniper.net

Juniper Networks Inc.

msri@juniper.net

Individual Contributor

kapil.it@gmail.com

Ciena

samson.cse@gmail.com

China Mobile

Beijing China xuxiaohu_ietf@hotmail.com

Routing Routing area OAM EPE BGP-LS BGP SPRING SDN SID Egress Peer Engineering (EPE) is an application of Segment Routing to solve the problem of egress peer selection. The Segment Routing based BGP-EPE solution allows a centralized controller, e.g. a Software Defined Network (SDN) controller to program any egress peer. The EPE solution requires the node or the SDN controller to program the PeerNode Segment Identifier(SID) describing a session between two nodes, the PeerAdj SID describing the link (one or more) that is used by sessions between peer nodes, and the PeerSet SID describing any connected interface to any peer in the related group. This document provides new sub-TLVs for EPE Segment Identifiers (SID) that would be used in the MPLS Target stack TLV (Type 1), in MPLS Ping and Traceroute procedures.

Egress Peer Engineering (EPE) as defined in is an effective mechanism to select the egress peer link based on different criteria. In this scenario, egress peers may belong to a completely different ownership. The EPE-SIDs provide means to represent egress peer nodes, links, sets of links and sets of nodes. Many network deployments have built their networks consisting of multiple Autonomous Systems, either for the ease of operations or as a result of network mergers and acquisitions. The inter-AS links connecting any two Autonomous Systems could be traffic-engineered using EPE-SIDs in this case, where there is single ownership but different AS numbers. It is important to validate the control plane to forwarding plane synchronization for these SIDs so that any anomaly can be detected easily by the network operator. EPE-SIDs may also be used in ingress SR policy to choose exit points where the remote AS belongs to completely different ownership. This scenario is out of scope of this document.

+---------+ +------+ | | | | | H B------D G | | +---/| AS 2 |\ +------+ | |/ +------+ \ | |---L/8 A AS1 C---+ \| | | |\\ \ +------+ /| AS 4 |---M/8 | | \\ +-E |/ +------+ | X | \\ | K | | +===F AS 3 | +---------+ +------+

In this reference diagram, EPE-SIDs are configured on AS1 towards AS2 and AS3 and advertised in BGP-LS . In certain cases the EPE-SIDs advertised by the control plane may not be in synchronization with the label programmed in the data plane. For example, on C a PeerAdj SID could be advertised to indicate it is for the link C->D. Due to some software anomaly, the actual data forwarding on this PeerAdj SID could be happening over the C->E link. If E had relevant data paths for further forwarding the packet, this kind of anomaly will go unnoticed by the network operator. A detailed example of a correctly programmed state and an incorrectly programmed state along with a description of how the incorrect state can be detected is described in . A FEC definition for the EPE-SIDs will define the details of the control plane association of the SID. The data plane validation of the SID will be done during the MPLS traceroute procedure. When there is a multi-hop EBGP session between the ASBRs, PeerNode SID is advertised, and the traffic MAY be load-balanced between the interfaces connecting the two nodes. In the reference diagram, C and F could have a PeerNode-SID advertised. When the OAM packet is received on F, it needs to be validated that the packet came from one of the two interfaces connected to C. This document provides Target Forwarding Equivalence Class (FEC) stack TLV definitions for EPE-SIDs. This solution requires that the node constructing the target FEC stack can determine the type of the SIDs along the path of the LSP. Other procedures for MPLS Ping and Traceroute as defined in section 7 and clarified by are applicable for EPE-SIDs as well.

provides mechanisms to advertise the EPE-SIDs in BGP-LS. These EPE-SIDs may be used to build Segment Routing paths as described in or using Path Computation Element Protocol (PCEP) extensions as defined in . Data plane monitoring for such paths which consist of EPE-SIDs will use extensions defined in this document to build the Target FEC stack TLV. The MPLS Ping and Traceroute procedures MAY be initiated by the head-end of the Segment Routing path or a centralized topology-aware data plane monitoring system as described in . The extensions in and do not define how to carry the details of the SID that can be used to construct the FEC. Such extensions are out of the scope for this document. The node initiating the data plane monitoring may acquire the details of EPE-SIDs through BGP-LS advertisements as described in . There may be other possible mechanisms to learn the definition of the SID from controller. Details of such mechanisms are out of scope for this document. The EPE-SIDs are advertised for inter-AS links which run EBGP sessions. does not define the detailed procedures to operate EBGP sessions in a scenario with unnumbered interfaces. Therefore, these scenarios are out of scope for this document. Anycast and multicast addresses are not in the scope of this document. During AS migration scenario procedures described in may be in force. In these scenarios, if the local and remote AS fields in the FEC as described in carries the globally configured ASN and not the "local AS" as defined in , the FEC validation procedures may fail. As described in , this document defines FEC stack TLVs for EPE-SIDs, that can be used in detecting MPLS data plane failures . This mechanism applies to paths created across across ASes of co-operating administrations. If the ping or traceroute packet enters a non co-operating AS domain, it might be dropped by the routers in the non co-operating domain. Although complete path validation cannot be done across, non co-operating domains, it still provides useful information that the ping/traceroute packet entered a non co-operating domain.

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14, , when, and only when, they appear in all capitals, as shown here.

Three new sub-TLVs are defined for the Target FEC Stack TLV (Type 1), the Reverse-Path Target FEC Stack TLV (Type 16), and the Reply Path TLV (Type 21).

Sub-Type Sub-TLV Name -------- --------------- TBD1 PeerAdj SID Sub-TLV TBD2 PeerNode SID Sub-TLV TBD3 PeerSet SID Sub-TLV

0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |Type = TBD2 | Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Local AS Number (4 octets) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Remote AS Number (4 octets) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Local BGP router ID (4 octets) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Remote BGP Router ID (4 octets) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Type : 2 octets Value:TBD2 Length : 2 octets Value: 16 Local AS Number : 4 octets The unsigned integer representing the AS number of the AS to which the PeerNode SID advertising node belongs. If Confederations are in use, and if the remote node is a member of a different Member-AS within the local Confederation, this is the Member-AS Number inside the Confederation and not the Confederation Identifier. Remote AS Number : 4 octets The unsigned integer representing the AS number of the AS of the remote node for which the PeerNode SID is advertised. If Confederations are in use, and if the remote node is a member of a different Member-AS within the local Confederation, this is the Member-AS Number inside the Confederation and not the Confederation Identifier. Local BGP Router ID : 4 octets unsigned integer representing the BGP Identifier of the PeerNode SID advertising node as defined in and . Remote BGP Router ID : 4 octets unsigned integer representing the BGP Identifier of the remote node as defined in and . When there is a multi-hop EBGP session between two ASBRs, PeerNode SID is advertised for this session and traffic can be load balanced across these interfaces. An EPE controller that does bandwidth management for these links should be aware of the links on which the traffic will be load-balanced. As per , the node advertising the EPE SIDs will send Downstream Detailed Mapping TLV (DDMAP TLV) specifying the details of nexthop interfaces, the OAM packet will be sent out. Based on this information controller MAY choose to verify the actual forwarding state with the topology information controller has. On the router, the validation procedures will include, received DDMAP validation as specified in to verify the control and forwarding state synchronization on the two routers. Any discrepancies between controller's state and forwarding state will not be detected by the procedures described in the document.

0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |Type = TBD1 | Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Adj-Type | RESERVED | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Local AS Number (4 octets) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Remote AS Number (4 octets) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Local BGP router ID (4 octets) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Remote BGP Router ID (4 octets) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Local Interface address (4/16 octets) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Remote Interface address (4/16 octets) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Type : 2 octets Value: TBD1 Length : 2 octets Value: variable based on IPv4/IPv6 interface address. Length excludes the length of Type and Length fields.For IPv4 interface addresses length will be 28 octets. In case of IPv6 address length will be 52 octets. Adj-Type : 1 octet Value: Set to 1 when the Adjacency Segment is IPv4 Set to 2 when the Adjacency Segment is IPv6 RESERVED : 3 octets. MUST be zero when sending, and ignored on receiving. Local AS Number : 4 octets The unsigned integer representing the AS number of the AS to which the PeerAdj SID advertising node belongs. If Confederations are in use, and if the remote node is a member of a different Member-AS within the local Confederation, this is the Member-AS Number inside the Confederation and not the Confederation Identifier. Remote AS Number : 4 octets The unsigned integer representing the AS number of the AS of the remote node for which the PeerAdj SID is advertised. If Confederations are in use, and if the remote node is a member of a different Member-AS within the local Confederation, this is the Member-AS Number inside the Confederation and not the Confederation Identifier. Local BGP Router ID : 4 octets unsigned integer representing the BGP Identifier of the PeerAdj SID advertising node as defined in and . Remote BGP Router ID : 4 octets unsigned integer representing the BGP Identifier of the remote node as defined in and . Local Interface Address :4 octets/16 octets In case of PeerAdj SID, Local interface address corresponding to the PeerAdj SID should be specified in this field. For IPv4,this field is 4 octets; for IPv6, this field is 16 octets. Link-local IPv6 addresses are not in the scope of this document. Remote Interface Address :4 octets/16 octets In case of PeerAdj SID Remote interface address corresponding to the PeerAdj SID should be apecified in this field. For IPv4, this field is 4 octets; for IPv6, this field is 16 octets. Link-local IPv6 addresses are not in the scope of this document.. mandates sending local interface ID and remote interface ID in the Link Descriptors and allows a value of 0 in the remote descriptors. It is useful to validate the incoming interface for an OAM packet and if the remote descriptor is 0 this validation is not possible. allows optional link descriptors of local and remote interface addresses as described in section 4.2. This document RECOMMENDs sending these optional descriptors and using them to validate incoming interface. When these local and remote interface addresses are not available, an ingress node can send 0 in the local and/or remote interface address field. The receiver SHOULD skip the validation for the incoming interface if the address field contains 0.

0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |Type = TBD3 | Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Local AS Number (4 octets) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Local BGP router ID (4 octets) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | No.of elements in set | Reserved | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Remote AS Number (4 octets) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Remote BGP Router ID (4 octets) | ++-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-++ One element in set consists of below details +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Remote AS Number (4 octets) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Remote BGP Router ID (4 octets) | ++-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-++

Type : 2 octets Value: TBD3 Length : 2 octets Value: Expressed in octets and variable based on the number of elements in the set. The length field does not include the length of Type and Length fields. Local AS Number :4 octets The unsigned integer representing the AS number of the AS to which the PeerSet SID advertising node belongs. If Confederations are in use, and if the remote node is a member of a different Member-AS within the local Confederation, this is the Member-AS Number inside the Confederation and not the Confederation Identifier. Local BGP Router ID : 4 octets unsigned integer representing the BGP Identifier of the PeerSet SID advertising node as defined in and . No.of elements in set: 2 octets The number of remote ASes over which the set SID performs load balancing. Reserved : 2 octets. MUST be zero when sent and ignored when received. Remote AS Number : 4 octets The unsigned integer representing the AS number of the AS of the remote node for which the PeerSet SID is advertised. If Confederations are in use, and if the remote node is a member of a different Member-AS within the local Confederation, this is the Member-AS Number inside the Confederation and not the Confederation Identifier. Remote BGP Router ID : 4 octets unsigned integer representing the BGP Identifier of the remote node as defined in and . PeerSet SID may be associated with a number of PeerNode SIDs and PeerAdj SIDs. The remote AS number and the Router ID of each of these PeerNode SIDs PeerAdj SIDs MUST be included in the FEC.

When a remote ASBR of the EPE-SID advertisement receives the MPLS OAM packet with top FEC being the EPE-SID, it MUST perform validity checks on the content of the EPE-SID FEC sub-TLV. The basic length check should be performed on the received FEC.

PeerAdj SID ----------- if Adj type = 1 Length should be 28 octets If Adj type =2 Length should be 52 octets PeerNode SID ------------- Length = ( 20 + No.of IPv4 interface pairs * 8 + No.of IPv6 interface pairs * 32 ) octets PeerSet SID ----------- Length = (9 + No.of elements in the set * (8 + No.of IPv4 interface pairs * 8 + No.of IPv6 interface pairs * 32)) octets

If a malformed FEC sub-TLV is received, then a return code of 1, "Malformed echo request received" as defined in MUST be sent. The below section is appended to the procedure given in Section 7.4 point 4a of .

Segment Routing IGP-Prefix, IGP-Adjacency SID and EPE-SID Validation : Receiving node term used in this section implies the node that receives OAM message with the FEC stack TLV. Else, if the Label-stack-depth is 0 and the Target FEC Stack sub-TLV at FEC-stack-depth is TBD1 (PeerAdj SID sub-TLV), { Set the Best-return-code to 10, "Mapping for this FEC is not the given label at stack-depth if any below conditions fail: - Validate that the receiving node's BGP Local AS matches with the remote AS field in the received PeerAdj SID FEC sub-TLV. - Validate that the receiving node's BGP Router-ID matches with the Remote Router ID field in the received PeerAdj SID FEC. - Validate that there is a EBGP session with a peer having local AS number and BGP Router-ID as specified in the Local AS number and Local Router-ID field in the received PeerAdj SID FEC sub-TLV. If the Remote interface address is not zero, validate the incoming interface. Set the Best-return-code to 35 "Mapping for this FEC is not associated with the incoming interface" [RFC8287] if any below conditions fail: - Validate the incoming interface on which the OAM packet was receieved, matches with the remote interface specified in the PeerAdj SID FEC sub-TLV If all above validations have passed, set the return code to 3 "Replying router is an egress for the FEC at stack-depth" } Else, if the Target FEC sub-TLV at FEC-stack-depth is TBD2 (PeerNode SID sub-TLV), { Set the Best-return-code to 10, "Mapping for this FEC is not the given label at stack-depth if any below conditions fail: - Validate that the receiving node's BGP Local AS matches with the remote AS field in the received PeerNode SID FEC sub-TLV. - Validate that the receiving node's BGP Router-ID matches with the Remote Router ID field in the received PeerNode SID FEC. - Validate that there is a EBGP session with a peer having local AS number and BGP Router-ID as specified in the Local AS number and Local Router-ID field in the received PeerNode SID FEC sub-TLV. If all above validations have passed, set the return code to 3 "Replying router is an egress for the FEC at stack-depth". } Else, if the Target FEC sub-TLV at FEC-stack-depth is TBD3 (PeerSet SID sub-TLV), { Set the Best-return-code to 10, "Mapping for this FEC is not the given label at stack-depth" if any below conditions fail: - Validate that the Receiving Node BGP Local AS matches with one of the remote AS field in the received PeerSet SID FEC sub-TLV. - Validate that the Receiving Node BGP Router-ID matches with one of the Remote Router ID field in the received PeerSet SID FEC sub-TLV. - Validate that there is a EBGP session with a peer having local AS number and BGP Router-ID as specified in the Local AS number and Local Router-ID field in the received PeerSet SID FEC sub-TLV. If all above validations have passed, set the return code to 3 "Replying router is an egress for the FEC at stack-depth" }

IANA is requested to allocate three new Target FEC stack sub-TLVs from the "Sub-TLVs for TLV types 1,16 and 21" subregistry in the "TLVs" registry of the "Multi-Protocol Label switching (MPLS) Label Switched Paths (LSPs) Ping parameters" namespace. PeerAdj SID Sub-TLV : TBD1 PeerNode SID Sub-TLV: TBD2 PeerSet SID Sub-TLV : TBD3 The three lowest free values from the Standard Tracks range should be allocated if possible.

The EPE-SIDs are advertised for egress links for Egress Peer Engineering purposes or for inter-AS links between co-operating ASes. When co-operating domains are involved, they can allow the packets arriving on trusted interfaces to reach the control plane and get processed. When EPE-SIDs are created for egress TE links where the neighbor AS is an independent entity, it may not allow packets arriving from external world to reach the control plane. In such deployments MPLS OAM packets will be dropped by the neighboring AS that receives the MPLS OAM packet. In MPLS traceroute applications, when the AS boundary is crossed with the EPE-SIDs, the FEC stack is changed. does not mandate that the initiator upon receiving an MPLS Echo Reply message that includes the FEC Stack Change TLV with one or more of the original segments being popped remove a corresponding FEC(s) from the Target FEC Stack TLV in the next (TTL+1) traceroute request. If an initiator does not remove the FECs belonging to the previous AS that has traversed, it may expose the internal AS information to the following AS being traversed in traceroute.

This section is to be removed before publishing as an RFC. RFC-Editor: Please clean up the references cited by this section before publication. This section records the status of known implementations of the protocol defined by this specification at the time of posting of this Internet-Draft, and is based on a proposal described in . The description of implementations in this section is intended to assist the IETF in its decision processes in progressing drafts to RFCs. Please note that the listing of any individual implementation here does not imply endorsement by the IETF. Furthermore, no effort has been spent to verify the information presented here that was supplied by IETF contributors. This is not intended as, and must not be construed to be, a catalog of available implementations or their features. Readers are advised to note that other implementations may exist.

Juniper networks reported a prototype implementation of this draft.

Thanks to Loa Andersson, Dhruv Dhody, Ketan Talaulikar, Italo Busi and Alexander Vainshtein, Deepti Rathi for careful review and comments. Thanks to Tarek Saad for providing the example described in Appendix section.

&RFC8287; &RFC8029; &RFC9086; &RFC2119; &RFC8174; &RFC8690; &RFC6793; &RFC9087; &RFC7705; &RFC8403; &RFC8664; &RFC4271; &RFC5065; &RFC6286; &RFC7942; &RFC9256;

This section describes an example of correctly programmed state and incorrectly programmed state and provides details on how the new sub-TLVs described in this document can be used to validate the correctness. Consider the diagram from , Correctly programed state: • C assigns label 16001 and binds it to adjacency C->E • C signals label 16001 is bound to adjacency C->E (e.g. via BGP-LS) • Controller/Ingress programs an SR path that has SID/label 16001 to steer packet on the exit point from C onto adjacency C->E • Using MPLS trace procedures defined in this document, the PeerAdj SID Sub-TLV is populates with entities to be validated by C when OAM packet reaches it. • C receives the OAM packet, it validates the top label (16001) is indeed corresponding to the entities populated in the PeerAdj SID Sub-TLV Incorrectly programed state: • C assigns label 16001 and binds it to adjacency C->D • The controller learns of PeerAdj SID label 16001 is bound to adjacency C->E (e.g. via BGP-LS) – this could be a software bug on C or on controller • Controller/Ingress programs an SR path that has SID/label 16001 to steer packet on the exit point from C onto adjacency C->E • Using MPLS trace procedures defined in this document, the PeerAdj SID Sub-TLV is populates with entities to be validated by C (including local/remote interface address of C->E) when OAM packet reaches it. • C receives the OAM packet, it validates the top label (16001) is NOT bound to C->E as populated in the PeerAdj SID Sub-TLV and can respond with the respective error code