Subject: [spring] Alvaro Retana's Discuss on draft-ietf-spring-srv6-network-programming-20: (with DISCUSS and COMMENT)

From: Alvaro Retana via Datatracker <[email-masked]>
Date: Wed, 23 Sep 2020 13:57:57 -0700

Alvaro Retana has entered the following ballot position for
draft-ietf-spring-srv6-network-programming-20: Discuss

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)

Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.

The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-spring-srv6-network-programming/

----------------------------------------------------------------------
DISCUSS:
----------------------------------------------------------------------

I am balloting DISCUSS because I find the document unclear and lacking proper
technical details around significant functionality, as reflected in my first 3
points.  The fourth point is related to the registration policy (which doesn't
match the definition in rfc8126), and my last point is for the IESG to consider.

(1) Pseudocode and normative behavior

The use of pseudocode was chosen as the mechanism to specify behavior, as
explained in §4:

   An implementation of the pseudocode is compliant as long as the externally
   observable wire protocol is as described by the pseudocode.

Clarity in the pseudocode is essential because it is used to determine
compliance.  Several places need improvement:

(1a) In §4.1/§4.13/§4.15, the pseudocode is missing an ELSE after S04, to
include the error conditions if SL != 0.  A check for an error condition when
SL is decremented is also needed.  As written, the pseudocode could process the
packet (SL == 0) *and* send an ICMP time exceeded message... :-(

I'm using as a reference the pseudocode in §4.3.1.1/rfc8754, which includes the
same initial statement.

(1b) It would be nice if the behavior in §4.1.1 were also specified using
pseudocode.  As written, I am not sure if the intent is to process any
upper-layer header or only specific ones.  Is the objective for this operation
to be similar to the one in §4.3.1.2/rfc8754?  Please be specific on what is
meant by "allowed by local configuration".

[Note: this point by itself is not DISCUSS-worthy, but §4.1.1 is used, for
different reasons, in some of the other items I point to below.  That is why I
include it here.]

(1c) §4.4/§4.6: S01 of the second piece of pseudocode is an instruction for
processing a non-IPv6 upper header.  However, earlier in that section, it is
specified that the SID "is associated with one or more L3 IPv6 adjacencies/an
IPv6 FIB table".  How can the upper header not be IPv6 if the specification
explicitly says it has to be?

(1d) §4.5/§4.7 have the same issue but related to IPv4.

(1e) §4.9 also has the same issue when it specifies that "End.DX2 SID...is
associated with one outgoing interface I", but allows for the processing of
non-ethernet payloads which could then be forwarded through a different
outgoing interface.

(1f) §4.11/§4.12 allows the processing of non-ethernet payloads, which will not
be "associated with an L2 Table T" as described.

(2) §4.12 describes the only behavior that can carry an ARG.  I don't
understand how it works:

      Arg.FE2 is encoded in the SID as an (k*x)-bit value.  These bits
      represent a list of up to k OIFs, each identified with an x-bit
      value.  Values k and x are defined on a per End.DT2M SID basis.  The
      interface identifier 0 indicates an empty entry in the interface
      list.

Let's assume a router has 10 possible OIFs, and the operator uses 4-bit values
to identify them; then, the ARG would take 40 bits of the SID.  Is that how the
math works?

Assuming my interpretation is correct, for 20 OIFs and 5-bit values we would
need 100 bits.  Considering the examples in §3.2, where a /64 is allocated to a
router, this behavior wouldn't have enough bits!  I realize that maybe a better
encoding would be to use a 20-bit field, each representing an interface. 
However, there would still be a limit of < 64 OIFs.  Am I missing something?

I'm trying to ultimately get to the fact that there are limits to this
behavior, but they are not described in the document.  Please clearly explain
any limitations and any possible workaround.

(3) The description of the flavors in §4.16 is also unclear.

The section starts with this introduction:

   The PSP, USP and USD flavors are variants of the End, End.X and End.T
   behaviors.  For each of these behaviors these flavors MAY be
   supported for a SID either individually or in combinations.

By being "variants", I interpret that the behavior is different than what is
specified in §4.1.

(3a) Some of the behaviors, as listed in Table 4, include an indication of the
flavors.  How are the values interpreted?  For example, the Table lists 8
different behaviors related to End:

| 1           | 0x0001 |   End (no PSP, no USP)  |    [This.ID]     |
| 2           | 0x0002 |       End with PSP      |    [This.ID]     |
| 3           | 0x0003 |       End with USP      |    [This.ID]     |
| 4           | 0x0004 |     End with PSP&USP    |    [This.ID]     |
...
| 28          | 0x001C |       End with USD      |    [This.ID]     |
| 29          | 0x001D |     End with PSP&USD    |    [This.ID]     |
| 30          | 0x001E |     End with USP&USD    |    [This.ID]     |
| 31          | 0x001F | End with PSP, USP & USD |    [This.ID]     |

Is value 1 what is specified in §4.1?  Or does it include USD, which is not
explicitly excluded)?

(3b) If a behavior with more than one flavor is signaled, how should the
receiving node determine which one to apply?  I guess that the application of
behaviors 4 or 29 depends on the number of SLs -- the expected behavior should
be clearly specified.

(3c) Is it assumed that all nodes support all behaviors?  Are there mandatory
to implement behaviors?  Should the behavior be advertised before it is used?

(3d) §4.16.1.2:

   When a SID of PSP-flavor is processed at a non-penultimate SR Segment
   Endpoint Node, the PSP behavior is not performed as described in the
   pseudocode below since Segments Left would not be zero.

For example, for the End behavior, I'm assuming that behavior 1 is performed
instead of 2 (or 4, or 29, or 31) if SL != 0.  Should this be done even if the
node did not advertise the non-PSP flavor?  If the node is not known to support
the PSP flavor, should it be an error to receive a packet requesting that
behavior?

If only the PSP flavor is advertised, can the Source assume that the node also
supports the non-PSP flavor?

  [BTW, I'm asking about advertisement because §4.16.1.1 makes the statement
  that the nodes "advertise the SIDs instantiated on them via control plane
  protocols as described in Section 9".  Even though §9 talks about control
  plane protocols are "not necessary for an SDN control plane" because "one
  expects the controller to explicitly provision the SIDs".]

(3e) §4.16.2 describes the USP flavor, which is one where the endpoint consumes
the packet by processing the next header.  I don't understand how the outcome
due to the extended process is different from the original one in §4.1.  Can
you please explain?  It seems to me that the externally observable result is
the same.

I have the same question about the USD flavor and the externally observable
behavior related to §4.1.

In general, the observable behavior of §4.1, USP, and USD seem the same to me. 
The next two points are related.

(3f) §4.16.3 describes the USD flavor, which assumes that the decapsulation
results in a packet that can be forwarded.  Can the FIB lookup result in a
local destination?

(3g) Does the USD flavor mean that, for the End behavior (as described in
§4.1), the action of "process the next header in the packet" cannot result in a
forwarded packet?  Same question for the USP behavior?

(3h) The last paragraph in §4.16.3:

   An implementation that supports the USD flavor in conjunction with
   the USP flavor MAY optimize the packet processing by first looking
   whether the conditions for the USD flavor are met, in which case it
   can proceed with USD processing else do USP processing.

What are the "conditions for the USD flavor"?  As far as I can tell from the
document, the only condition is for the specific behavior to be signaled.  What
else?

Going back to the questions above...  When is the option to optimize possible? 
Does a specific behavior have to be used?  Behavior 30 (End with USP&USD)?  Or
can it also optimize if behavior 3 (End with USP) is signaled?

(4) §10.2 creates a new registry with an "FCFS" registration procedure.  I am
assuming that this is the same as the "First Come First Served" (no
abbreviation!) policy from rfc8126; please add a reference if that is the case.
 The description used is not the same as what rfc8126 specifies:

- "Requests for allocation...must include a...preferably also a brief
  description of how the value will be used."   Using "preferably" indicates
  that a description is optional.  However, it is not optional in rfc8126.

- "...brief description...may be provided with a reference to an Internet
  Draft or an RFC or in some other documentation that is permanently and
  readily available."  There is no such requirement in rfc8126.  For example,
  the "Specification Required" policy requires "a permanent and readily
  available public specification".  Is that what you want  instead?

(5) This point is for the IESG to discuss.

§4.16.1.2:

      The End, End.X and End.T behaviors with PSP do not contravene
      Section 4 of [RFC8200] because the destination address of the
      incoming packet is the address of the node executing the behavior.

The spring WG's interpretation of rfc8200 was a central point in the appeal
presented against the WG consensus on this document.  The text above, I
believe, reflects that consensus.

However, given that the document relies on the spring WG's interpretation of
rfc8200, I think it would be better if the text is explicit.

Suggestion: to add at the end of the paragraph>

   This conclusion represents the consensus interpretation of the spring WG.

----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

(1) §3.1:

   An SRv6 endpoint behavior MAY require additional information for its
   processing (e.g. related to the flow or service).  This information
   may be encoded in the ARG bits of the SID.

The sentence is simply stating a fact, not a normative behavior.   s/MAY/may

(2) All the examples in §3.2 have a /48 prefix allocated to the SRv6 deployment
(and then /64s per node).  Is it possible to start with a different SRv6
infrastructure allocation, a /64, or /96 maybe?  If so, please include an
example.  If not, please explain any limitations.

(3) §4 starts by saying that "Each FIB entry indicates the behavior associated
with a SID instance and its parameters."  But the previous section (§3.3. SID
Reachability) says that "node N would advertise IPv6 prefix(es) matching the
LOC parts covering its SIDs or shorter-mask prefix" (not the behavior).  IOW,
§3.3 sets the expectation of an advertisement covering just the LOC, but §4
seems to expect entries that cover the LOC+FUNC.  Which one is correct?

In the end, it may not matter which entry is in the FIB, as long as the SID is
reachable.  However, the specification of the behavior feels sloppy.

(4) §4.9/§4.10: For the S04 step, perhaps decompose it into individual actions
(similar to S04-S06 in §4.7).

(5) §4.11/§4.12  "S05. Learn the exposed MAC Source Address..."

The note related to this step says that in "EVPN, the learning...is done via
the control plane"...but here it is done via the data plane.  What, if any, is
the effect on EVPN operation?  Are there issues with learning conflicting
information from different sources?  It seems to me that it could be relatively
easy to spoof the source and create unexpected entries in the L2 table.  Please
point to the EVPN documents where this type of operation is considered.

(6) §4.10/§4.11/§4.12 don't have references to the example applications
mentioned.  Please add Informative references.

(7) §4.13/§4.15 instantiate a Binding SID, but only in the case where SL != 0. 
What about the case where a Binding SID wouldn't require an extra encapsulation
(SL == 0)?  Is there a reason that it is not supported in this document?

(8) §5.1: I'm assuming that the last line in this section (the one starting
with S03) should be proceeded by "Note:".

(9) §5.1: "The H.Encaps behavior is valid for any kind of Layer-3 traffic." 
While it may be used for any kind of traffic, I'm assuming that there will be a
policy that determines which traffic is encapsulated using a specific SRv6
policy, right?  Please be specific about that.

(10) §5.3: "Ethernet [IEEE.802.3_2012]"  Please use the reference when Ethernet
is first used in the document.  [I have the same question as Rob related to the
version of the 802.3 spec.]

(11) §5.3: "...MUST remove the preamble or frame check sequence (FCS) from the
Ethernet frame upon encapsulation and the decapsulating node MUST regenerate
the preamble or FCS before forwarding Ethernet frame."   Which one?  The
preamble can be easily recreated by the receiver, while removing the FCS may be
more problematic -- even if the FCS is not checked in transit, it seems that it
would be important to carry it.  In any case, the real question here is: why
use "or"?  Is it left at the discretion of the encapsulating node?  Are there
any considerations when selecting?

(12) All the headend behaviors (§5) include this text:

   The push of the SRH MAY be omitted when the SRv6 Policy only contains
   one segment and there is no need to use any flag, tag or TLV.

If the endpoint behavior indicates the PSP or USP flavors, what should the
receiver do?  Clearly there is no SRH to pop.  Is this an error or should the
receiver simply ignore the flavor?

(13) §6: "counter...for traffic that matched that SID and was processed
correctly"  Does "processed correctly" include when the result being an ICMP
error message?  Or should those be counted separately?

(14) §7: I'm guessing that "flow-based hash" and "load-balancing hash" are the
same thing, is that correct?  It would be nice to use consistent terminology.

(15) §8: A rogue node inside the SR domain may (on purpose) signal the wrong
behavior for a flow, which may result in the delivery of the traffic to the
wrong destination (potentially including destinations outside the domain),
among other things.  Note that this action is possible even if the rogue node
is authenticated and authorized to generate an SRH.  I didn't find this threat
mentioned in rfc8402/rfc8754.

(16) §9.4: I'm not sure what the purpose of §9 is, as a whole.  But the summary
in §9.4 puzzles me more; what is the intent?  Does Table 1 indicate that, for
example, an IGP implementation should not advertise the End.B6.Encaps behavior?
 Does Table 2 indicate that only BGP-LS should signal the ability to
H.Encaps.L2?  I am confused about the value/intent because the text clearly
says that the control plane is outside the scope of this document.

(17) [nits]

s/an network operator/a network operator

s/one billionth and one millionth of the assigned address space/one billionth
and one millionth of the available address space

s/packet's header Section 7/packet's header (Section 7)/g

s/bundle(LAG)/bundle (LAG)
Please expand LAG.

---

From: Erik Kline <[email-masked]>
Date: Wed, 23 Sep 2020 17:22:32 -0700

> (5) This point is for the IESG to discuss.

This seems like an important -- and entirely reasonable -- thing to note.

---

From: "[email-masked]" <[email-masked]>
Date: Thu, 24 Sep 2020 10:28:01 +0800

Why not get the interpretation from 6man WG since RFC8200 is one of the outcomes of 6man WG?

Best Regards,
Zhenqiang Li

[email-masked]
 
From: Erik Kline
Date: 2020-09-24 08:22
To: Alvaro Retana
CC: SPRING WG; spring-chairs; Bruno Decraene; The IESG; Joel Halpern; draft-ietf-spring-srv6-network-programming
Subject: Re: [spring] Alvaro Retana's Discuss on draft-ietf-spring-srv6-network-programming-20: (with DISCUSS and COMMENT)
(5) This point is for the IESG to discuss.

§4.16.1.2:

---

From: "Pablo Camarillo (pcamaril)" <[email-masked]>
Date: Fri, 25 Sep 2020 18:28:48 +0000

Hi Alvaro,

Thanks for your review. See inline with [PC].
(Note that we've just posted rev21 of the draft.)

Thanks,
Pablo.

-----Original Message-----
From: Alvaro Retana via Datatracker <[email-masked]> 
Sent: miércoles, 23 de septiembre de 2020 22:58
To: The IESG <[email-masked]>
Cc: [email-masked]; [email-masked]; [email-masked]; Bruno Decraene <[email-masked]>; Joel Halpern <[email-masked]>
Subject: Alvaro Retana's Discuss on draft-ietf-spring-srv6-network-programming-20: (with DISCUSS and COMMENT)

When responding, please keep the subject line intact and reply to all email addresses included in the To and CC lines. (Feel free to cut this introductory paragraph, however.)

I am balloting DISCUSS because I find the document unclear and lacking proper technical details around significant functionality, as reflected in my first 3 points.  The fourth point is related to the registration policy (which doesn't match the definition in rfc8126), and my last point is for the IESG to consider.

The use of pseudocode was chosen as the mechanism to specify behavior, as explained in §4:

Clarity in the pseudocode is essential because it is used to determine compliance.  Several places need improvement:

(1a) In §4.1/§4.13/§4.15, the pseudocode is missing an ELSE after S04, to include the error conditions if SL != 0.  A check for an error condition when SL is decremented is also needed.  As written, the pseudocode could process the packet (SL == 0) *and* send an ICMP time exceeded message... :-(
[PC] Please check the pseudocode in rev 21. We have included the missing break statement in S03. I believe the current pseudocode is good.

I'm using as a reference the pseudocode in §4.3.1.1/rfc8754, which includes the same initial statement.

(1b) It would be nice if the behavior in §4.1.1 were also specified using pseudocode.  As written, I am not sure if the intent is to process any upper-layer header or only specific ones.  Is the objective for this operation to be similar to the one in §4.3.1.2/rfc8754?  Please be specific on what is meant by "allowed by local configuration".
[PC] Yes, we can structure the text in 4.1.1 in pseudocode form. The processing is not the same as RFC8754/Section 4.3.1.2. The “allowed by local configuration” is to enable the processing of only specific types of Upper-layer Headers for packets addressed to an SRv6 SID of the specific behaviors. E.g. An operator may not wish to have BGP sessions (or in general any TCP traffic) destined to a local SID, but may want to enable ICMPv6 packet processing for OAM purposes. 
<OLD>
4.1.1.  Upper-Layer Header

   When processing the Upper-layer Header of a packet matching a FIB
   entry locally instantiated as an SRv6 End SID, if Upper-layer Header
   processing is allowed by local configuration (e.g.  ICMPv6), then
   process the upper-layer header.  Otherwise, send an ICMP parameter
   problem message to the Source Address and discard the packet.  Error
   code 4 (SR Upper-layer Header Error) and Pointer set to the offset of
   the upper-layer header.
</OLD>
<NEW>
4.1.1.  Upper-Layer Header

When processing the Upper-layer Header of a packet matching a FIB entry locally instantiated as an SRv6 End SID do the following:

S01. If (Upper-Layer Header type is allowed by local configuration) {
S02.   Process the Upper-layer Header
S03. } Else {
S04.   Send an ICMP Parameter Problem to the Source Address,
          Code 4 (SR Upper-layer Header Error),
          Pointer set to the offset of the Upper-layer Header,
          Interrupt packet processing and discard the packet.
S05  }

Notes.
S01. An operator may not wish to have any TCP traffic destined to a local SID, but may want to enable ICMPv6 packet processing for OAM purposes.
</NEW>

[Note: this point by itself is not DISCUSS-worthy, but §4.1.1 is used, for different reasons, in some of the other items I point to below.  That is why I include it here.]

(1c) §4.4/§4.6: S01 of the second piece of pseudocode is an instruction for processing a non-IPv6 upper header.  However, earlier in that section, it is specified that the SID "is associated with one or more L3 IPv6 adjacencies/an
IPv6 FIB table".  How can the upper header not be IPv6 if the specification explicitly says it has to be?
[PC] The pseudocode is convoluted. I propose to turn it around for 4.4, 4.5, 4.6 and 4.7. As an example with 4.4:
<OLD>
   When processing the Upper-layer header of a packet matching a FIB
   entry locally instantiated as an SRv6 End.DX6 SID, the following is
   done:

   S01. If (Upper-Layer Header type != 41(IPv6) ) {
   S02.    Process as per Section 4.1.1
   S03. }
   S04. Remove the outer IPv6 Header with all its extension headers
   S05. Forward the exposed IPv6 packet to the L3 adjacency J
</OLD>
<NEW>
   When processing the Upper-layer header of a packet matching a FIB
   entry locally instantiated as an SRv6 End.DX6 SID, the following is
   done:

   S01. If (Upper-Layer Header type == 41(IPv6) ) {
   S02.    Remove the outer IPv6 Header with all its extension headers
   S03.    Forward the exposed IPv6 packet to the L3 adjacency J
   S04. }
   S05. Else {
   S06.    Process as per Section 4.1.1
   S07. }
</NEW>

(1d) §4.5/§4.7 have the same issue but related to IPv4.
[PC] We’ve clarified the pseudocode on the same lines as 1c.

(1e) §4.9 also has the same issue when it specifies that "End.DX2 SID...is associated with one outgoing interface I", but allows for the processing of non-ethernet payloads which could then be forwarded through a different outgoing interface.
[PC] We’ve clarified the pseudocode on the same lines as 1c.

(1f) §4.11/§4.12 allows the processing of non-ethernet payloads, which will not be "associated with an L2 Table T" as described.
[PC] We’ve clarified the pseudocode on the same lines as 1c.

(2) §4.12 describes the only behavior that can carry an ARG.  I don't understand how it works:

Let's assume a router has 10 possible OIFs, and the operator uses 4-bit values to identify them; then, the ARG would take 40 bits of the SID.  Is that how the math works?

Assuming my interpretation is correct, for 20 OIFs and 5-bit values we would need 100 bits.  Considering the examples in §3.2, where a /64 is allocated to a router, this behavior wouldn't have enough bits!  I realize that maybe a better encoding would be to use a 20-bit field, each representing an interface. 
However, there would still be a limit of < 64 OIFs.  Am I missing something?
[PC] For the End.DT2M behavior, Arg.FE2 is a locally allocated Ethernet Segment Identifier that is used for split-horizon filtering as described in RFC7432. The text that you have quoted above needs to be removed since it is trying to describe one way of allocation (which obviously has its limitations). Instead, we will update this text to clarify the semantics and purpose of the Arg.FE2 and its allocation method would be left as implementation specific (just similar to an ESI label).  
[PC]Please see the updated text of rev21.

I'm trying to ultimately get to the fact that there are limits to this behavior, but they are not described in the document.  Please clearly explain any limitations and any possible workaround.

By being "variants", I interpret that the behavior is different than what is specified in §4.1.

(3a) Some of the behaviors, as listed in Table 4, include an indication of the flavors.  How are the values interpreted?  For example, the Table lists 8 different behaviors related to End:

Is value 1 what is specified in §4.1?  Or does it include USD, which is not explicitly excluded)?
[PC] This has been corrected in revision 20. Can you please recheck and let me know? E.g. 0x0001 does not include any of the flavors.

[PC] For the remainder of the comments in this section, lets recall that a single Segment Endpoint Behavior codepoint is bound to a SID at a segment endpoint node.   A node computing a segment list with a particular SID knows its associated behavior.  A segment endpoint node receiving a packet destined to a locally instantiated SID performs only the processing associated with the behavior bound to that SID.  A behavior is only bound to a SID, never to a node.
(3b) If a behavior with more than one flavor is signaled, how should the receiving node determine which one to apply?  I guess that the application of behaviors 4 or 29 depends on the number of SLs -- the expected behavior should be clearly specified.
[PC] The segment endpoint node receiving a packet destined to a SID with behavior 4 applies only the processing associated with the SID (I.e. behavior 4).

(3c) Is it assumed that all nodes support all behaviors?  Are there mandatory to implement behaviors?  Should the behavior be advertised before it is used?
[PC] The answer to first two questions is no. For the third, if a node computing a segment list does not know of a SID (and its behavior) it will not be able to 'use' the SID in a segment list.

(3d) §4.16.1.2:

For example, for the End behavior, I'm assuming that behavior 1 is performed instead of 2 (or 4, or 29, or 31) if SL != 0.  Should this be done even if the node did not advertise the non-PSP flavor?  
[PC] If a SID of END behavior (1) is instantiated at a segment endpoint node, a packet destined to that SID will only ever be processed with behavior 1.

If the node is not known to support the PSP flavor, should it be an error to receive a packet requesting that behavior?
[PC] If a node does not support PSP, then it has not instantiated any SID with a segment endpoint behavior including PSP, and it is not possible for it to receive a packet destined to a local SID it has not instantiated.

If only the PSP flavor is advertised, can the Source assume that the node also supports the non-PSP flavor?
[PC] If a SID with PSP flavor is advertised (I.e. segment endpoint behavior codepoint 2) by a segment endpoint node, a SR source node can only expect that SID has the behavior bound to it.

(3e) §4.16.2 describes the USP flavor, which is one where the endpoint consumes the packet by processing the next header.  I don't understand how the outcome due to the extended process is different from the original one in §4.1.  Can you please explain?  It seems to me that the externally observable result is the same.
[PC] We have use-cases where the packets with SRH may be destined to applications or host implementations running in containers. The USP flavor is useful to remove the consumed SRH from the extension header chain before sending over to the application stack – we’ve seen this with smartNICs. As such the perspective on externally observability differs and hence we believe it is needed to specify this.

I have the same question about the USD flavor and the externally observable behavior related to §4.1.
[PC] The USD flavor specifically enables the de-encapsulation of inner IP packet and its further forwarding (consider use-case like TI-LFA where encapsulation is done on the PLR and de-encapsulation has to be done on the last node of the repair list). In this case the PLR node that is crafting the SID list wants to ensure that the last segment in the repair list is able to perform decapsulation.

(3f) §4.16.3 describes the USD flavor, which assumes that the decapsulation results in a packet that can be forwarded.  Can the FIB lookup result in a local destination?
[PC] Please refer the previous comment about the use-case and so yes, we normally expect the decapsulation results in a packet that is forwarded out. However, the inner packet may also be destined to a local address.

(3g) Does the USD flavor mean that, for the End behavior (as described in §4.1), the action of "process the next header in the packet" cannot result in a forwarded packet?  Same question for the USP behavior?
[PC] Please refer to the previous comments. There is no such assumptions on neither the base End behavior nor End with USP.

What are the "conditions for the USD flavor"?  As far as I can tell from the document, the only condition is for the specific behavior to be signaled.  What else?
[PC] I've removed this paragraph. This is an implementation optimization and provides no value to the pseudocode preceding it.

Going back to the questions above...  When is the option to optimize possible? 
Does a specific behavior have to be used?  Behavior 30 (End with USP&USD)?  Or can it also optimize if behavior 3 (End with USP) is signaled?

(4) §10.2 creates a new registry with an "FCFS" registration procedure.  I am assuming that this is the same as the "First Come First Served" (no
abbreviation!) policy from rfc8126; please add a reference if that is the case.
[PC] Ack to change FCFS for “First Come First Served [RFC8126]”.

[PC] Indeed, the current text is wrong. My bad. I've updated the text with this diff below. Also, I’m not sure whether that paragraph is really needed. Maybe just putting in the table “First Come First Served [RFC8126]” is sufficient as RFC8126 already describes what is written in the text below. If it can be removed please let me know.
<OLD>
   Requests for allocation from within the FCFS range must include a
   point of contact and preferably also a brief description of how the
   value will be used.  This information may be provided with a
   reference to an Internet Draft or an RFC or in some other
   documentation that is permanently and readily available.
</OLD>
<NEW>
   Requests for allocation from within the First Come First Serve range must include a
   point of contact and a brief description of how the
   value will be used.
</NEW>

§4.16.1.2:

The spring WG's interpretation of rfc8200 was a central point in the appeal presented against the WG consensus on this document.  The text above, I believe, reflects that consensus.

However, given that the document relies on the spring WG's interpretation of rfc8200, I think it would be better if the text is explicit.

(1) §3.1:

The sentence is simply stating a fact, not a normative behavior.   s/MAY/may
[PC] This is fixed in rev20.

(2) All the examples in §3.2 have a /48 prefix allocated to the SRv6 deployment (and then /64s per node).  Is it possible to start with a different SRv6 infrastructure allocation, a /64, or /96 maybe?  If so, please include an example.  If not, please explain any limitations.
[PC] The examples are based on real deployments and as such reflect the practical aspects of those operators SRv6 infrastructure allocation designs. It would be counter-productive and misleading to provide artificially manufactured examples (and then why just /96 and not something else?). The document does not pose any limitations.

(3) §4 starts by saying that "Each FIB entry indicates the behavior associated with a SID instance and its parameters."  But the previous section (§3.3. SID
Reachability) says that "node N would advertise IPv6 prefix(es) matching the LOC parts covering its SIDs or shorter-mask prefix" (not the behavior).  IOW,
§3.3 sets the expectation of an advertisement covering just the LOC, but §4 seems to expect entries that cover the LOC+FUNC.  Which one is correct?
[PC] Section 3.3 talks about the Prefix Reachability advertisement for the SRv6 Locator. Section 4, is talking about FIB entries instantiated on the SR Segment Endpoint Node that has allocated those local SIDs – please note that they are local entries. There is no discussion on their advertisement in Section 4. Can you please check if the update text in Section 4 in v21 clarifies?

In the end, it may not matter which entry is in the FIB, as long as the SID is reachable.  However, the specification of the behavior feels sloppy.

(4) §4.9/§4.10: For the S04 step, perhaps decompose it into individual actions (similar to S04-S06 in §4.7).
[PC] Fixed in rev21.

The note related to this step says that in "EVPN, the learning...is done via the control plane"...but here it is done via the data plane.  What, if any, is the effect on EVPN operation?  Are there issues with learning conflicting information from different sources?  It seems to me that it could be relatively easy to spoof the source and create unexpected entries in the L2 table.  Please point to the EVPN documents where this type of operation is considered.
[PC] Indeed, text is inaccurate. I've updated the note to the following:
<OLD>
   S05.  In EVPN, the learning of the exposed MAC Source Address is done
   via the control plane.
</OLD>
<NEW>
S03. In EVPN RFC7432, the learning of the exposed MAC Source Address is done via control plane. In L2VPN VPLS RFC4761 RFC4762 reachability is obtained by standard learning bridge functions in the data plane.
</NEW>

(6) §4.10/§4.11/§4.12 don't have references to the example applications mentioned.  Please add Informative references.
[PC] Ack. I’ve add the references to RFC7432, RFC4761, RFC4762, RFC8317, RFC4761.

(7) §4.13/§4.15 instantiate a Binding SID, but only in the case where SL != 0. 
What about the case where a Binding SID wouldn't require an extra encapsulation (SL == 0)?  Is there a reason that it is not supported in this document?
[PC] Such a requirement has not yet arisen. If need arises in the future, a new behavior may be defined by a future document. This document provides the framework and the extensibility to do so.

(8) §5.1: I'm assuming that the last line in this section (the one starting with S03) should be proceeded by "Note:".
[PC] Fixed.

(9) §5.1: "The H.Encaps behavior is valid for any kind of Layer-3 traffic." 
While it may be used for any kind of traffic, I'm assuming that there will be a policy that determines which traffic is encapsulated using a specific SRv6 policy, right?  Please be specific about that.
[PC] This document does not describe how traffic is steered into an SR Policy, it may be steered by a route installed by BGP, a static route, some application specific selection, etc.. The steering of a packet into an SR Policy is out of scope of this document..

(10) §5.3: "Ethernet [IEEE.802.3_2012]"  Please use the reference when Ethernet is first used in the document.  [I have the same question as Rob related to the version of the 802.3 spec.]
[PC] Fixed in rev20.

(11) §5.3: "...MUST remove the preamble or frame check sequence (FCS) from the Ethernet frame upon encapsulation and the decapsulating node MUST regenerate
the preamble or FCS before forwarding Ethernet frame."   Which one?  The
preamble can be easily recreated by the receiver, while removing the FCS may be more problematic -- even if the FCS is not checked in transit, it seems that it would be important to carry it.  In any case, the real question here is: why use "or"?  Is it left at the discretion of the encapsulating node?  Are there any considerations when selecting?
[PC] You are correct. Corrected to the following:
<OLD>
The encapsulating node MUST remove the preamble or frame check sequence (FCS) from the Ethernet frame upon encapsulation and the decapsulating node MUST regenerate the preamble or FCS before forwarding Ethernet frame.
</OLD>
<NEW>
The encapsulating node MUST remove the preamble (if any) and frame check sequence (FCS) from the Ethernet frame upon encapsulation and the decapsulating node MUST regenerate, as required, the preamble and FCS before forwarding Ethernet frame.
</NEW>

If the endpoint behavior indicates the PSP or USP flavors, what should the receiver do?  Clearly there is no SRH to pop.  Is this an error or should the receiver simply ignore the flavor?
[PC] If there is no SRH, then the SRH processing is not executed. 
The PSP and USP flavors only make changes in the SRH processing pseudocode, hence it is not executed.

(13) §6: "counter...for traffic that matched that SID and was processed correctly"  Does "processed correctly" include when the result being an ICMP error message?  Or should those be counted separately?
[PC] Packets that result in an ICMP error message or those that are dropped are not counted as correctly processed. I've updated the text.
<OLD>
   A node supporting this document SHOULD implement a pair of traffic
   counter (one for packets and one for bytes) per local SID entry, for
   traffic that matched that SID and was processed correctly.
</OLD>
<NEW>
   A node supporting this document SHOULD implement a pair of traffic
   counters (one for packets and one for bytes) per local SID entry, for
   traffic that matched that SID and was processed successfully (i.e. packets which generate ICMP Error Messages or are dropped are not counted). 
</NEW>

(14) §7: I'm guessing that "flow-based hash" and "load-balancing hash" are the same thing, is that correct?  It would be nice to use consistent terminology.
[PC] Fixed.

(15) §8: A rogue node inside the SR domain may (on purpose) signal the wrong behavior for a flow, which may result in the delivery of the traffic to the wrong destination (potentially including destinations outside the domain), among other things.  Note that this action is possible even if the rogue node is authenticated and authorized to generate an SRH.  I didn't find this threat mentioned in rfc8402/rfc8754.
[PC] The control plane protocol specifics are outside the scope of this document. I am not able to parse this comment and what is it that needs to be addressed in this document.

(16) §9.4: I'm not sure what the purpose of §9 is, as a whole.  But the summary in §9.4 puzzles me more; what is the intent?  Does Table 1 indicate that, for example, an IGP implementation should not advertise the End.B6.Encaps behavior?
 Does Table 2 indicate that only BGP-LS should signal the ability to H.Encaps.L2?  I am confused about the value/intent because the text clearly says that the control plane is outside the scope of this document.
[PC] The section provides an overview of the role of control plane routing protocols in the advertisements of the SRv6 Locator and the SIDs along with their behaviors – all new aspects that have been introduced in this document. Based on the SRv6 solutions developed around the behaviors introduced in this document, it indicates what information is expected to be advertised via which protocol. It does not describe “how” since that is clearly outside the scope of this document and part of the individual routing protocol extensions.

(17) [nits]

s/an network operator/a network operator
s/one billionth and one millionth of the assigned address space/one billionth and one millionth of the available address space
s/packet's header Section 7/packet's header (Section 7)/g
[PC] Those three have been fixed.

s/bundle(LAG)/bundle (LAG)
Please expand LAG.
[PC] "(LAG)" has been removed as per the comments of another AD.

Thank you for your time Alvaro!

---

From: Martin Vigoureux <[email-masked]>
Date: Mon, 28 Sep 2020 10:03:10 +0200

Hello Pablo,

thank you. In relation to item 5, here is a suggestion discussed by the 
IESG during the Telechat:

"In the context of this specification, the End, End.X and End.T 
behaviors with PSP do not contravene Section 4 of [RFC8200] because the 
destination address of the incoming packet is the address of the node 
executing the behavior."

-m

---

From: Fernando Gont <[email-masked]>
Date: Mon, 28 Sep 2020 05:16:06 -0300

Hello, Martin, Alvaro, & other members of the IESG,

This document is applying a very interesting interpretation of RFC8200, 
and I believe such interpretation steps out of the charter of Spring 
(or, for instance, the RTG Area).

IPv6 has always forbidden en-route insertion and removal of IPv6 EHs. 
RFC2460 contains this text (very similar to what's in its predecessor 
RFC1883):

    With one exception, extension headers are not examined or processed
    by any node along a packet's delivery path, until the packet reaches
    the node (or each of the set of nodes, in the case of multicast)
    identified in the Destination Address field of the IPv6 header.
    There, normal demultiplexing on the Next Header field of the IPv6
    header invokes the module to process the first extension header, or
    the upper-layer header if no extension header is present.  The
    contents and semantics of each extension header determine whether or
    not to proceed to the next header.
  [....]
    The exception referred to in the preceding paragraph is the Hop-by-
    Hop Options header, which carries information that must be examined
    and processed by every node along a packet's delivery path, including
    the source and destination nodes.

The text highlights that HBH options are processed by every node on the 
path, and that other EHs are only processed when the packet reaches the 
address in the "Destination Address" field. Indeed, RHs are processed 
when the packets get to such destination address, and other EHs and 
upper layer protocols are processed when the packet reaches the 
Destination Address. For packets that employ a RH, these later EHs and 
upper layer protocols will get processed by the node(s) identified in 
the Destination Address when Segments Left==0 -- that is, by the final 
destination.

In the same timeframe that the rfc2460bis effort (that eventually lead 
to RFC8200) was being pursued, an individual internet-draft 
(draft-previdi-6man-segment-routing-header) was proposing to perform 
en-route header insertion/removal. When this later draft appeared in the 
radar of 6man participants, there were many objections to such proposal, 
as it was violating existing standards.

The proponents of draft-previdi-6man-segment-routing-header argued that 
RFC2460 didn't explicitly forbid en-route EH-insertion/removal, and 
hence their proposal was not violating any specifications. Some of them 
claimed that the use of the term "processing" was ambiguous.

While it was clear to the vast majority of the 6man wg that the IPv6 
architecture and the ipv6 specification didn't allow for EH 
insertion/removal, participants felt that, since the rfc2460bis effort 
was ongoing, rfc2460bis should take the chance to be crystal clear on 
this point, to avoid possible future claims that such behavior was allowed.

While the document was still in the 6man wg, the proponents of 
draft-previdi-6man-segment-routing-header resisted any clarifications in 
this respect (please see 
<https://mailarchive.ietf.org/arch/msg/ietf/j1O11x4ICMUWGJmzlJfCx0y0-c8/>), 
and hence 6man shippoed for publication a version of the document that 
didn't spell out the prohibition of en-route EH insertion/removal.

During the IETF LC for the document, there was a strong push to have 
rfc2460bis be explicit about the prohibition of en-route 
insertion/removal of extension headers, and some subset of the 
participants crafted the text that eventually made it into RFC8200. 
Essentially, they spelled out all the prohibited actions, as explicitly 
sa possible, and added them along the existing text. The resulting text was:

    Extension headers (except for the Hop-by-Hop Options header) are not
    processed, inserted, or deleted by any node along a packet's delivery
    path, until the packet reaches the node (or each of the set of nodes,
    in the case of multicast) identified in the Destination Address field
    of the IPv6 header.

While the intent was to be more explicit about the things that were 
prohibited, the addition of these extra terms alongside the existing 
text ended up introducing a bug in the specification since, af crafted, 
the text suggests that nodes that get to process EHs also get to e.g. 
insert and remove EHs.  This is why I consider this a bug, and why I've 
submitted Erratum 6003 on RFC8200.

RFC8200 contains, in Appendix B, a summary of the changes from RFC2460, 
and notes that:

    o  Clarified that extension headers (except for the Hop-by-Hop
       Options header) are not processed, inserted, or deleted by any
       node along a packet's delivery path.

This spells out the intent of the corresponding text. 
draft-ietf-6man-rfc2460bis-13 (the last internet-draft version of 
RFC8200), it spells out the intent of the specification with even more 
detail:

       01)  Added text that Extension headers must never be inserted by
            any node other than the source of the packet.

Finally, draft-ietf-6man-rfc2460bis-08, the document that was shipped by 
the 6man wg to the IESG for publication, was indeed very explicit about 
the serious problems with en-route insertion/removal of EHs:

    The insertion of Extension Headers by any node other than the source
    of the packet causes serious problems.  Two examples include breaking
    the integrity checks provided by the Authentication Header Integrity
    [RFC4302], and breaking Path MTU Discovery which can result in ICMP
    error messages being sent to the source of the packet that did not
    insert the header, rather than the node that inserted the header.

This very same problems are caused when the node inserting or removing 
EHs is a node listed in an RH.

It is also worth noting that RFC8200 elevated the status of IPv6 to 
"Internet Standard". Since RFC2460 by no means allows for en-route 
header insertion/removal/deletion, the incorporation of this "feature" 
in RFC8200 would have prevented the elevation of IPv6 to Internet 
Standard. This is yet another indication that Erratum 6003 refers to a 
bug in the specification, as opposed to an intended protocol behavior 
that would be a change from the preceding revision of the standard, 
RFC2460.

As per the above, it would seem to me that before a Spring documents 
makes any arbitrary interpretation of RFC8200 that happens to represent 
a change to the IPv6 Architecture, 6man should clarify the topic, and if 
the IPv6 architecture is to be changed, the IETF as a whole should be 
involved (since that would probably be even out of the scope of 6man).

Thanks,
Fernando

On 28/9/20 05:03, Martin Vigoureux wrote:
>> Cc: [email-masked]; 
>> [email-masked]; [email-masked]; Bruno Decraene 
>> <[email-masked]>; Joel Halpern <[email-masked]>
>> Subject: Alvaro Retana's Discuss on 
>> draft-ietf-spring-srv6-network-programming-20: (with DISCUSS and COMMENT)
>>
>> email addresses included in the To and CC lines. (Feel free to cut 
>> this introductory paragraph, however.)
>>
>>
>> I am balloting DISCUSS because I find the document unclear and lacking 
>> proper technical details around significant functionality, as 
>> reflected in my first 3 points.  The fourth point is related to the 
>> registration policy (which doesn't match the definition in rfc8126), 
>> and my last point is for the IESG to consider.
>>
>> (1a) In §4.1/§4.13/§4.15, the pseudocode is missing an ELSE after S04, 
>> to include the error conditions if SL != 0.  A check for an error 
>> condition when SL is decremented is also needed.  As written, the 
>> pseudocode could process the packet (SL == 0) *and* send an ICMP time 
>> exceeded message... :-(
>> [PC] Please check the pseudocode in rev 21. We have included the 
>> missing break statement in S03. I believe the current pseudocode is good.
>>
>> (1b) It would be nice if the behavior in §4.1.1 were also specified 
>> using pseudocode.  As written, I am not sure if the intent is to 
>> process any upper-layer header or only specific ones.  Is the 
>> objective for this operation to be similar to the one in 
>> §4.3.1.2/rfc8754?  Please be specific on what is meant by "allowed by 
>> local configuration".
>> [PC] Yes, we can structure the text in 4.1.1 in pseudocode form. The 
>> processing is not the same as RFC8754/Section 4.3.1.2. The “allowed by 
>> local configuration” is to enable the processing of only specific 
>> types of Upper-layer Headers for packets addressed to an SRv6 SID of 
>> the specific behaviors. E.g. An operator may not wish to have BGP 
>> sessions (or in general any TCP traffic) destined to a local SID, but 
>> may want to enable ICMPv6 packet processing for OAM purposes.
>> <OLD>
>> entry locally instantiated as an SRv6 End SID do the following:
>>
>> S01. An operator may not wish to have any TCP traffic destined to a 
>> local SID, but may want to enable ICMPv6 packet processing for OAM 
>> purposes.
>> </NEW>
>>
>> [Note: this point by itself is not DISCUSS-worthy, but §4.1.1 is used, 
>> for different reasons, in some of the other items I point to below.  
>> That is why I include it here.]
>>
>> (1c) §4.4/§4.6: S01 of the second piece of pseudocode is an 
>> instruction for processing a non-IPv6 upper header.  However, earlier 
>> in that section, it is specified that the SID "is associated with one 
>> or more L3 IPv6 adjacencies/an
>> IPv6 FIB table".  How can the upper header not be IPv6 if the 
>> specification explicitly says it has to be?
>> [PC] The pseudocode is convoluted. I propose to turn it around for 
>> 4.4, 4.5, 4.6 and 4.7. As an example with 4.4:
>> <OLD>
>> (1e) §4.9 also has the same issue when it specifies that "End.DX2 
>> SID...is associated with one outgoing interface I", but allows for the 
>> processing of non-ethernet payloads which could then be forwarded 
>> through a different outgoing interface.
>> (1f) §4.11/§4.12 allows the processing of non-ethernet payloads, which 
>> will not be "associated with an L2 Table T" as described.
>> Let's assume a router has 10 possible OIFs, and the operator uses 
>> 4-bit values to identify them; then, the ARG would take 40 bits of the 
>> SID.  Is that how the math works?
>>
>> Assuming my interpretation is correct, for 20 OIFs and 5-bit values we 
>> would need 100 bits.  Considering the examples in §3.2, where a /64 is 
>> allocated to a router, this behavior wouldn't have enough bits!  I 
>> realize that maybe a better encoding would be to use a 20-bit field, 
>> each representing an interface.
>> [PC] For the End.DT2M behavior, Arg.FE2 is a locally allocated 
>> Ethernet Segment Identifier that is used for split-horizon filtering 
>> as described in RFC7432. The text that you have quoted above needs to 
>> be removed since it is trying to describe one way of allocation (which 
>> obviously has its limitations). Instead, we will update this text to 
>> clarify the semantics and purpose of the Arg.FE2 and its allocation 
>> method would be left as implementation specific (just similar to an 
>> ESI label).
>> behavior, but they are not described in the document.  Please clearly 
>> explain any limitations and any possible workaround.
>>
>> (3a) Some of the behaviors, as listed in Table 4, include an 
>> indication of the flavors.  How are the values interpreted?  For 
>> example, the Table lists 8 different behaviors related to End:
>>
>> [PC] This has been corrected in revision 20. Can you please recheck 
>> and let me know? E.g. 0x0001 does not include any of the flavors.
>>
>>
>> [PC] For the remainder of the comments in this section, lets recall 
>> that a single Segment Endpoint Behavior codepoint is bound to a SID at 
>> a segment endpoint node.   A node computing a segment list with a 
>> particular SID knows its associated behavior.  A segment endpoint node 
>> receiving a packet destined to a locally instantiated SID performs 
>> only the processing associated with the behavior bound to that SID.  A 
>> behavior is only bound to a SID, never to a node.
>> (3b) If a behavior with more than one flavor is signaled, how should 
>> the receiving node determine which one to apply?  I guess that the 
>> application of behaviors 4 or 29 depends on the number of SLs -- the 
>> expected behavior should be clearly specified.
>> [PC] The segment endpoint node receiving a packet destined to a SID 
>> with behavior 4 applies only the processing associated with the SID 
>> (I.e. behavior 4).
>>
>> (3c) Is it assumed that all nodes support all behaviors?  Are there 
>> mandatory to implement behaviors?  Should the behavior be advertised 
>> before it is used?
>> [PC] The answer to first two questions is no. For the third, if a node 
>> computing a segment list does not know of a SID (and its behavior) it 
>> will not be able to 'use' the SID in a segment list.
>>
>> (3d) §4.16.1.2:
>>
>> For example, for the End behavior, I'm assuming that behavior 1 is 
>> performed instead of 2 (or 4, or 29, or 31) if SL != 0.  Should this 
>> be done even if the node did not advertise the non-PSP flavor?
>> [PC] If a SID of END behavior (1) is instantiated at a segment 
>> endpoint node, a packet destined to that SID will only ever be 
>> processed with behavior 1.
>>
>> If the node is not known to support the PSP flavor, should it be an 
>> error to receive a packet requesting that behavior?
>> [PC] If a node does not support PSP, then it has not instantiated any 
>> SID with a segment endpoint behavior including PSP, and it is not 
>> possible for it to receive a packet destined to a local SID it has not 
>> instantiated.
>>
>> [PC] If a SID with PSP flavor is advertised (I.e. segment endpoint 
>> behavior codepoint 2) by a segment endpoint node, a SR source node can 
>> only expect that SID has the behavior bound to it.
>>
>> (3e) §4.16.2 describes the USP flavor, which is one where the endpoint 
>> consumes the packet by processing the next header.  I don't understand 
>> how the outcome due to the extended process is different from the 
>> original one in §4.1.  Can you please explain?  It seems to me that 
>> the externally observable result is the same.
>> [PC] We have use-cases where the packets with SRH may be destined to 
>> applications or host implementations running in containers. The USP 
>> flavor is useful to remove the consumed SRH from the extension header 
>> chain before sending over to the application stack – we’ve seen this 
>> with smartNICs. As such the perspective on externally observability 
>> differs and hence we believe it is needed to specify this.
>>
>> [PC] The USD flavor specifically enables the de-encapsulation of inner 
>> IP packet and its further forwarding (consider use-case like TI-LFA 
>> where encapsulation is done on the PLR and de-encapsulation has to be 
>> done on the last node of the repair list). In this case the PLR node 
>> that is crafting the SID list wants to ensure that the last segment in 
>> the repair list is able to perform decapsulation.
>>
>> (3f) §4.16.3 describes the USD flavor, which assumes that the 
>> decapsulation results in a packet that can be forwarded.  Can the FIB 
>> lookup result in a local destination?
>> [PC] Please refer the previous comment about the use-case and so yes, 
>> we normally expect the decapsulation results in a packet that is 
>> forwarded out. However, the inner packet may also be destined to a 
>> local address.
>>
>> (3g) Does the USD flavor mean that, for the End behavior (as described 
>> in §4.1), the action of "process the next header in the packet" cannot 
>> result in a forwarded packet?  Same question for the USP behavior?
>> [PC] Please refer to the previous comments. There is no such 
>> assumptions on neither the base End behavior nor End with USP.
>>
>> What are the "conditions for the USD flavor"?  As far as I can tell 
>> from the document, the only condition is for the specific behavior to 
>> be signaled.  What else?
>> [PC] I've removed this paragraph. This is an implementation 
>> optimization and provides no value to the pseudocode preceding it.
>>
>> Does a specific behavior have to be used?  Behavior 30 (End with 
>> USP&USD)?  Or can it also optimize if behavior 3 (End with USP) is 
>> signaled?
>>
>> (4) §10.2 creates a new registry with an "FCFS" registration 
>> procedure.  I am assuming that this is the same as the "First Come 
>> First Served" (no
>> [PC] Indeed, the current text is wrong. My bad. I've updated the text 
>> with this diff below. Also, I’m not sure whether that paragraph is 
>> really needed. Maybe just putting in the table “First Come First 
>> Served [RFC8126]” is sufficient as RFC8126 already describes what is 
>> written in the text below. If it can be removed please let me know.
>> <OLD>
>> The spring WG's interpretation of rfc8200 was a central point in the 
>> appeal presented against the WG consensus on this document.  The text 
>> above, I believe, reflects that consensus.
>>
>> However, given that the document relies on the spring WG's 
>> interpretation of rfc8200, I think it would be better if the text is 
>> explicit.
>>
>> (2) All the examples in §3.2 have a /48 prefix allocated to the SRv6 
>> deployment (and then /64s per node).  Is it possible to start with a 
>> different SRv6 infrastructure allocation, a /64, or /96 maybe?  If so, 
>> please include an example.  If not, please explain any limitations.
>> [PC] The examples are based on real deployments and as such reflect 
>> the practical aspects of those operators SRv6 infrastructure 
>> allocation designs. It would be counter-productive and misleading to 
>> provide artificially manufactured examples (and then why just /96 and 
>> not something else?). The document does not pose any limitations.
>>
>> (3) §4 starts by saying that "Each FIB entry indicates the behavior 
>> associated with a SID instance and its parameters."  But the previous 
>> section (§3.3. SID
>> Reachability) says that "node N would advertise IPv6 prefix(es) 
>> matching the LOC parts covering its SIDs or shorter-mask prefix" (not 
>> the behavior).  IOW,
>> §3.3 sets the expectation of an advertisement covering just the LOC, 
>> but §4 seems to expect entries that cover the LOC+FUNC.  Which one is 
>> correct?
>> [PC] Section 3.3 talks about the Prefix Reachability advertisement for 
>> the SRv6 Locator. Section 4, is talking about FIB entries instantiated 
>> on the SR Segment Endpoint Node that has allocated those local SIDs – 
>> please note that they are local entries. There is no discussion on 
>> their advertisement in Section 4. Can you please check if the update 
>> text in Section 4 in v21 clarifies?
>>
>> In the end, it may not matter which entry is in the FIB, as long as 
>> the SID is reachable.  However, the specification of the behavior 
>> feels sloppy.
>>
>> The note related to this step says that in "EVPN, the learning...is 
>> done via the control plane"...but here it is done via the data plane.  
>> What, if any, is the effect on EVPN operation?  Are there issues with 
>> learning conflicting information from different sources?  It seems to 
>> me that it could be relatively easy to spoof the source and create 
>> unexpected entries in the L2 table.  Please point to the EVPN 
>> documents where this type of operation is considered.
>> S03. In EVPN RFC7432, the learning of the exposed MAC Source Address 
>> is done via control plane. In L2VPN VPLS RFC4761 RFC4762 reachability 
>> is obtained by standard learning bridge functions in the data plane.
>> </NEW>
>>
>> (6) §4.10/§4.11/§4.12 don't have references to the example 
>> applications mentioned.  Please add Informative references.
>> What about the case where a Binding SID wouldn't require an extra 
>> encapsulation (SL == 0)?  Is there a reason that it is not supported 
>> in this document?
>> [PC] Such a requirement has not yet arisen. If need arises in the 
>> future, a new behavior may be defined by a future document. This 
>> document provides the framework and the extensibility to do so.
>>
>> (8) §5.1: I'm assuming that the last line in this section (the one 
>> starting with S03) should be proceeded by "Note:".
>> [PC] Fixed.
>>
>> While it may be used for any kind of traffic, I'm assuming that there 
>> will be a policy that determines which traffic is encapsulated using a 
>> specific SRv6 policy, right?  Please be specific about that.
>> [PC] This document does not describe how traffic is steered into an SR 
>> Policy, it may be steered by a route installed by BGP, a static route, 
>> some application specific selection, etc.. The steering of a packet 
>> into an SR Policy is out of scope of this document..
>>
>> (10) §5.3: "Ethernet [IEEE.802.3_2012]"  Please use the reference when 
>> Ethernet is first used in the document.  [I have the same question as 
>> Rob related to the version of the 802.3 spec.]
>> [PC] Fixed in rev20.
>>
>> (11) §5.3: "...MUST remove the preamble or frame check sequence (FCS) 
>> from the Ethernet frame upon encapsulation and the decapsulating node 
>> MUST regenerate
>> preamble can be easily recreated by the receiver, while removing the 
>> FCS may be more problematic -- even if the FCS is not checked in 
>> transit, it seems that it would be important to carry it.  In any 
>> case, the real question here is: why use "or"?  Is it left at the 
>> discretion of the encapsulating node?  Are there any considerations 
>> when selecting?
>> The encapsulating node MUST remove the preamble or frame check 
>> sequence (FCS) from the Ethernet frame upon encapsulation and the 
>> decapsulating node MUST regenerate the preamble or FCS before 
>> forwarding Ethernet frame.
>> </OLD>
>> <NEW>
>> The encapsulating node MUST remove the preamble (if any) and frame 
>> check sequence (FCS) from the Ethernet frame upon encapsulation and 
>> the decapsulating node MUST regenerate, as required, the preamble and 
>> FCS before forwarding Ethernet frame.
>> </NEW>
>>
>> If the endpoint behavior indicates the PSP or USP flavors, what should 
>> the receiver do?  Clearly there is no SRH to pop.  Is this an error or 
>> should the receiver simply ignore the flavor?
>> (13) §6: "counter...for traffic that matched that SID and was 
>> processed correctly"  Does "processed correctly" include when the 
>> result being an ICMP error message?  Or should those be counted 
>> separately?
>> [PC] Packets that result in an ICMP error message or those that are 
>> dropped are not counted as correctly processed. I've updated the text.
>> <OLD>
>>     traffic that matched that SID and was processed successfully (i.e. 
>> packets which generate ICMP Error Messages or are dropped are not 
>> counted).
>> </NEW>
>>
>> (14) §7: I'm guessing that "flow-based hash" and "load-balancing hash" 
>> are the same thing, is that correct?  It would be nice to use 
>> consistent terminology.
>> [PC] Fixed.
>>
>> (15) §8: A rogue node inside the SR domain may (on purpose) signal the 
>> wrong behavior for a flow, which may result in the delivery of the 
>> traffic to the wrong destination (potentially including destinations 
>> outside the domain), among other things.  Note that this action is 
>> possible even if the rogue node is authenticated and authorized to 
>> generate an SRH.  I didn't find this threat mentioned in rfc8402/rfc8754.
>> [PC] The control plane protocol specifics are outside the scope of 
>> this document. I am not able to parse this comment and what is it that 
>> needs to be addressed in this document.
>>
>> (16) §9.4: I'm not sure what the purpose of §9 is, as a whole.  But 
>> the summary in §9.4 puzzles me more; what is the intent?  Does Table 1 
>> indicate that, for example, an IGP implementation should not advertise 
>> the End.B6.Encaps behavior?
>> H.Encaps.L2?  I am confused about the value/intent because the text 
>> clearly says that the control plane is outside the scope of this 
>> document.
>> [PC] The section provides an overview of the role of control plane 
>> routing protocols in the advertisements of the SRv6 Locator and the 
>> SIDs along with their behaviors – all new aspects that have been 
>> introduced in this document. Based on the SRv6 solutions developed 
>> around the behaviors introduced in this document, it indicates what 
>> information is expected to be advertised via which protocol. It does 
>> not describe “how” since that is clearly outside the scope of this 
>> document and part of the individual routing protocol extensions.
>>
>> (17) [nits]
>>
>> s/one billionth and one millionth of the assigned address space/one 
>> billionth and one millionth of the available address space
> _______________________________________________
> spring mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/spring

-- 
Fernando Gont
e-mail: [email-masked] || [email-masked]
PGP Fingerprint: 7809 84F5 322E 45C7 F1C9 3945 96EE A9EF D076 FFF1

---

From: "Pablo Camarillo (pcamaril)" <[email-masked]>
Date: Mon, 28 Sep 2020 16:13:10 +0000

Martin, Alvaro,

That sentence seems good. I've just posted revision 22 of the draft including it (together with a couple of nit-fixes as pointed by Jingrong).

Thanks,
Pablo.

https://tools.ietf.org/html/draft-ietf-spring-srv6-network-programming-22

-----Original Message-----
From: Martin Vigoureux <[email-masked]> 
Sent: lunes, 28 de septiembre de 2020 10:03
To: Pablo Camarillo (pcamaril) <[email-masked]>; Alvaro Retana <[email-masked]>; The IESG <[email-masked]>
Cc: Bruno Decraene <[email-masked]>; [email-masked]; Joel Halpern <[email-masked]>; [email-masked]; [email-masked]
Subject: Re: Alvaro Retana's Discuss on draft-ietf-spring-srv6-network-programming-20: (with DISCUSS and COMMENT)

Hello Pablo,

thank you. In relation to item 5, here is a suggestion discussed by the IESG during the Telechat:

"In the context of this specification, the End, End.X and End.T behaviors with PSP do not contravene Section 4 of [RFC8200] because the destination address of the incoming packet is the address of the node executing the behavior."

-m

---

From: Alvaro Retana <[email-masked]>
Date: Mon, 28 Sep 2020 15:50:04 -0700

On September 25, 2020 at 2:28:53 PM, Pablo Camarillo wrote:

Pablo:

Hi!

I still have a couple of comments related to the DISCUSS portion.  And
some non-blocking comments later on.

I looked at -22.

Thanks!

Alvaro.

I think we need some more discussion on 1c, 3b, 3d.

3e-3g can be resolved with a more text on the expected types of use
cases.  See more below.

...
> upper-layer header or only specific ones. Is the objective for this
> operation to be similar to the one in §4.3.1.2/rfc8754? Please be specific
> on what is meant by "allowed by local configuration".
>
[PC] Yes, we can structure the text in 4.1.1 in pseudocode form. The
[PC] processing is not the same as RFC8754/Section 4.3.1.2. The “allowed by
[PC] local configuration” is to enable the processing of only specific types
[PC] of Upper-layer Headers for packets addressed to an SRv6 SID of the
[PC] specific behaviors. E.g. An operator may not wish to have BGP sessions
[PC] (or in general any TCP traffic) destined to a local SID, but may want to
[PC] enable ICMPv6 packet processing for OAM purposes.
...

§4.1.1 is called from different places, while processing different
behaviors.  Is it expected that the "local configuration" will cover
each behavior individually, or will the operator be able to configure
a single policy for all?  In either case, it would be good to mention
it.

...
> specified that the SID "is associated with one or more L3 IPv6
> adjacencies/an IPv6 FIB table". How can the upper header not be IPv6 if the
[PC] The pseudocode is convoluted. I propose to turn it around for 4.4, 4.5,
[PC] 4.6 and 4.7. As an example with 4.4:
...

I still have the same question.  For example, in §4.4/End.DX6, I don't
understand under which conditions the upper-layer won't be IPv6.

The behavior in §4.1.1 now starts by saying: "Upper-Layer Header type
is allowed by local configuration".  Same question: For example, for
End.DX6, when would something other than IPv6 be allowed by local
configuration?

It seems to me that if the behavior "is associated with one or more L3
IPv6 adjacencies", then the contents should either be IPv6 or be
discarded. Otherwise the sender may include something else (IPv4, for
example) and it may be forwarded...

I may be missing something obvious, I just don't know what.

[Similar concerns for other sections where the payload is sent to
§4.1.1 if the contents don't match what the behavior expects.]

...
...
> behaviors 4 or 29 depends on the number of SLs -- the expected behavior
> should be clearly specified.
>
[PC] The segment endpoint node receiving a packet destined to a SID with
[PC] behavior 4 applies only the processing associated with the SID (I.e.
[PC] behavior 4).

Sorry, I mixed up some of the terminology.  Let me try this one again.

For an endpoint behavior that indicates more than one flavor, which
one should be applied?

For some of the behaviors, 29 (End with PSP&USD) for example, the
flavor used seems to depend on the number of SLs: if received with SL
== 0, then the flavor is USD, but if received with SL == 1 then use
PSP.  But for other behaviors, 30 (End with USP&USD) for example,
which flavor should be applied if both are supported?

...
> instead of 2 (or 4, or 29, or 31) if SL != 0. Should this be done even if
> the node did not advertise the non-PSP flavor?
>
[PC] If a SID of END behavior (1) is instantiated at a segment endpoint node,
[PC] a packet destined to that SID will only ever be processed with behavior
[PC] 1.

Redo:  If the SID used indicates behavior 2 (End with PSP), but the
node is the last one (not the penultimate one), then what should it
do?  This result is probably an error from the sender.  Should the
receiver drop the packet, or process as behavior 1?   Assuming that
the node instantiated SIDs for both behaviors.

...
> consumes the packet by processing the next header. I don't understand how
> the outcome due to the extended process is different from the original one
> in §4.1. Can you please explain? It seems to me that the externally
> observable result is the same.
>
[PC] We have use-cases where the packets with SRH may be destined to
[PC] applications or host implementations running in containers. The USP
[PC] flavor is useful to remove the consumed SRH from the extension header
[PC] chain before sending over to the application stack – we’ve seen this
[PC] with smartNICs. As such the perspective on externally observability
[PC] differs and hence we believe it is needed to specify this.

Ok.  Please include the use case so that it is clear to others that
the external behavior is different.

> behavior related to §4.1.
[PC] The USD flavor specifically enables the de-encapsulation of inner IP
[PC] packet and its further forwarding (consider use-case like TI-LFA where
[PC] encapsulation is done on the PLR and de-encapsulation has to be done on
[PC] the last node of the repair list). In this case the PLR node that is
[PC] crafting the SID list wants to ensure that the last segment in the
[PC] repair list is able to perform decapsulation.

Ok.  Please include the use case...

> results in a packet that can be forwarded. Can the FIB lookup result in a
[PC] Please refer the previous comment about the use-case and so yes, we
[PC] normally expect the decapsulation results in a packet that is forwarded
[PC] out. However, the inner packet may also be destined to a local address.

Just to make sure I understand, if it is ok for the destination to be
a local address then the external observable behavior is equivalent to
End, right?

> §4.1), the action of "process the next header in the packet" cannot result
> in a forwarded packet? Same question for the USP behavior?
>
[PC] Please refer to the previous comments. There is no such assumptions on
[PC] neither the base End behavior nor End with USP.

Right...here again, the external behaviors may then be the same.

I see how there may be differences, in line with the use cases, and
how some of the behaviors may end up being the same...sometimes.
Let's then add sample use case information and move on.

...
> (4) §10.2 creates a new registry with an "FCFS" registration procedure.
...
[PC] Indeed, the current text is wrong. My bad. I've updated the text with
[PC] this diff below. Also, I’m not sure whether that paragraph is really
[PC] needed. Maybe just putting in the table “First Come First Served
[PC] [RFC8126]” is sufficient as RFC8126 already describes what is written in
[PC] the text below. If it can be removed please let me know.

Yes, I think you don't need the additional paragraph.

...

We may need to talk a little bit more about 5.

...
> different SRv6 infrastructure allocation, a /64, or /96 maybe? If so, please
> include an example. If not, please explain any limitations.
>
[PC] The examples are based on real deployments and as such reflect the
[PC] practical aspects of those operators SRv6 infrastructure allocation
[PC] designs. It would be counter-productive and misleading to provide
[PC] artificially manufactured examples (and then why just /96 and not
[PC] something else?). The document does not pose any limitations.

I don't see how it can be misleading to provide a different example.
The fact that all the examples have similar assumptions might give the
impression that a limitation exists.  I defer to you/AD.

...
> the control plane"...but here it is done via the data plane. What, if any,
> is the effect on EVPN operation? Are there issues with learning conflicting
> information from different sources? It seems to me that it could be
> relatively easy to spoof the source and create unexpected entries in the L2
> table. Please point to the EVPN documents where this type of operation is
> considered.
>
[PC] Indeed, text is inaccurate. I've updated the note to the following:
[PC]
[PC] S03. In EVPN RFC7432, the learning of the exposed MAC Source Address is
[PC] done via control plane. In L2VPN VPLS RFC4761 RFC4762 reachability is
[PC] obtained by standard learning bridge functions in the data plane.

I'm not sure if the updated note means that S03 doesn't apply to EVPN,
or if it just confirms that EVPN expects to learn from the control
plane.  ??

...
> is authenticated and authorized to generate an SRH. I didn't find this
> threat mentioned in rfc8402/rfc8754.
>
[PC] The control plane protocol specifics are outside the scope of this
[PC] document. I am not able to parse this comment and what is it that needs
[PC] to be addressed in this document.

Sorry, let me try again:

In the data plane...  A rogue SR headend may, on purpose, use the
wrong endpoint SID.

For example, an endpoint may support End.DT6 for several IPv6 tables.
If the wrong SID is used, then the wrong table may be used to forward
the packet.  If multiple tenants have overlapping addresses, the
packet may then be forwarded to the wrong destination.

I believe that this is a new vulnerability introduced my this document
that cannot be mitigated using the HMAC TLV, for example.  IOW, the
draft assumes that the SRH will be populated with the correct SIDs,
but that may not always be the case if a node has been overtaken.
There's not much that can be done to mitigate this issue, but I think
it is important to mention.

> (16) §9.4: I'm not sure what the purpose of §9 is, as a whole. But the
> summary in §9.4 puzzles me more; what is the intent? Does Table 1 indicate
> that, for example, an IGP implementation should not advertise the
> End.B6.Encaps behavior?
[PC] The section provides an overview of the role of control plane routing
[PC] protocols in the advertisements of the SRv6 Locator and the SIDs along
[PC] with their behaviors – all new aspects that have been introduced in this
[PC] document. Based on the SRv6 solutions developed around the behaviors
[PC] introduced in this document, it indicates what information is expected
[PC] to be advertised via which protocol. It does not describe “how” since
[PC] that is clearly outside the scope of this document and part of the
[PC] individual routing protocol extensions.

"it indicates what information is expected to be advertised via which protocol"

So...does that mean that an IGP is not expected to advertise
H.Encaps.L2, or that it cannot advertise it?   I'm trying to figure
out if there is any normative expectation -- I'm assuming not because
it's been said several times that the control plane if out of scope.
If the control plane is not bound by whatever this section says, then
why do we even have it?

I noticed that you wrote that the text is "Based on the SRv6 solutions
developed around the behaviors introduced in this document", which I
guess means that this is a reflection of the existing control plane
drafts (??).  If that is the case, then, at best, this seems like an
attempt to put the carriage in front of the horses, knowing what the
outcome may be, but without making reference to them.  I still don't
see the value.

This is a non-blocking point -- I think this section can be removed
and the value of the document wouldn't be reduced.  I will defer to
Martin.

---

From: "Pablo Camarillo (pcamaril)" <[email-masked]>
Date: Wed, 30 Sep 2020 13:18:33 +0000

Hi Alvaro,

Many thanks for the very prompt review and feedback. Inline with [PC2].
(Note that I’ve posted rev23 with some changes as per comments below)

Cheers,
Pablo.

https://tools.ietf.org/html/draft-ietf-spring-srv6-network-programming-23

-----Original Message-----
From: Alvaro Retana <[email-masked]> 
Sent: martes, 29 de septiembre de 2020 0:50
To: The IESG <[email-masked]>; Pablo Camarillo (pcamaril) <[email-masked]>
Cc: Joel Halpern <[email-masked]>; [email-masked]; Bruno Decraene <[email-masked]>; [email-masked]; [email-masked]
Subject: RE: Alvaro Retana's Discuss on draft-ietf-spring-srv6-network-programming-20: (with DISCUSS and COMMENT)

Pablo:

Hi!

I still have a couple of comments related to the DISCUSS portion.  And some non-blocking comments later on.

I looked at -22.

Thanks!

Alvaro.

3e-3g can be resolved with a more text on the expected types of use cases.  See more below.

...
> process any upper-layer header or only specific ones. Is the objective 
> for this operation to be similar to the one in §4.3.1.2/rfc8754? 
> Please be specific on what is meant by "allowed by local configuration".
>
[PC] Yes, we can structure the text in 4.1.1 in pseudocode form. The [PC] processing is not the same as RFC8754/Section 4.3.1.2. The “allowed by [PC] local configuration” is to enable the processing of only specific types [PC] of Upper-layer Headers for packets addressed to an SRv6 SID of the [PC] specific behaviors. E.g. An operator may not wish to have BGP sessions [PC] (or in general any TCP traffic) destined to a local SID, but may want to [PC] enable ICMPv6 packet processing for OAM purposes.
...

§4.1.1 is called from different places, while processing different behaviors.  Is it expected that the "local configuration" will cover each behavior individually, or will the operator be able to configure a single policy for all?  In either case, it would be good to mention it.

[PC2] In the document we've left 'local configuration' up to an implementation.  Whether an implementation implements the local configuration on an interface as an ACL or globally for all SIDs or per SID via some API is not for this document to decide, and has no impact on interoperability. 

...
> or more L3 IPv6 adjacencies/an IPv6 FIB table". How can the upper 
> header not be IPv6 if the specification explicitly says it has to be?
>
[PC] The pseudocode is convoluted. I propose to turn it around for 4.4, 4.5, [PC] 4.6 and 4.7. As an example with 4.4:
...

I still have the same question.  For example, in §4.4/End.DX6, I don't understand under which conditions the upper-layer won't be IPv6.

The behavior in §4.1.1 now starts by saying: "Upper-Layer Header type is allowed by local configuration".  Same question: For example, for End.DX6, when would something other than IPv6 be allowed by local configuration?

It seems to me that if the behavior "is associated with one or more L3
IPv6 adjacencies", then the contents should either be IPv6 or be discarded. Otherwise the sender may include something else (IPv4, for
example) and it may be forwarded...

[PC2] One common example is to allow ICMPv6 processing for packets destined to an End.DX6 SID (or any of the other behaviors) for OAM operations (e.g. ping to that SID). 

...
...
> expected behavior should be clearly specified.
[PC] The segment endpoint node receiving a packet destined to a SID with [PC] behavior 4 applies only the processing associated with the SID (I.e.
[PC] behavior 4).

For an endpoint behavior that indicates more than one flavor, which one should be applied?

For some of the behaviors, 29 (End with PSP&USD) for example, the flavor used seems to depend on the number of SLs: if received with SL == 0, then the flavor is USD, but if received with SL == 1 then use PSP.  But for other behaviors, 30 (End with USP&USD) for example, which flavor should be applied if both are supported?

[PC2] When a behavior (e.g. End) is combined with one or more flavors (e.g. USP & USD), their combined pseudocode is what determines the packet processing. In the specific example of USP&USD (when SL=0), the pseudocode would end up first removing the processed SRH and then, depending on the next upper-layer header, also removing the outer IPv6 encapsulation header if/when there is an inner IP packet.

...
> performed instead of 2 (or 4, or 29, or 31) if SL != 0. Should this be 
> done even if the node did not advertise the non-PSP flavor?
>
[PC] If a SID of END behavior (1) is instantiated at a segment endpoint node, [PC] a packet destined to that SID will only ever be processed with behavior [PC] 1.

Redo:  If the SID used indicates behavior 2 (End with PSP), but the node is the last one (not the penultimate one), then what should it do?  This result is probably an error from the sender.  Should the receiver drop the packet, or process as behavior 1?   Assuming that the node instantiated SIDs for both behaviors.

[PC2] The result is not an error from the sender. Since the node is the 'last one', I'll take that to mean that SL=0.  
When processing the SRH, lines S02/S03 of the END pseudocode says to "stop processing the SRH and proceed to process the next header".  So the PSP processing defined starting at S14.1 never occurs.

...
> original one in §4.1. Can you please explain? It seems to me that the 
> externally observable result is the same.
>
[PC] We have use-cases where the packets with SRH may be destined to [PC] applications or host implementations running in containers. The USP [PC] flavor is useful to remove the consumed SRH from the extension header [PC] chain before sending over to the application stack – we’ve seen this [PC] with smartNICs. As such the perspective on externally observability [PC] differs and hence we believe it is needed to specify this.

Ok.  Please include the use case so that it is clear to others that the external behavior is different.

[PC2] Sure. I’ve included the following text in rev23.
<NEW>
One of the applications of the USP flavor is when a packet with an SRH is destined to an application on hosts with smartNICs implementing SRv6. The USP flavor is used to remove the consumed SRH from the extension header chain before sending the packet to the host.
</NEW> 

> observable behavior related to §4.1.
[PC] The USD flavor specifically enables the de-encapsulation of inner IP [PC] packet and its further forwarding (consider use-case like TI-LFA where [PC] encapsulation is done on the PLR and de-encapsulation has to be done on [PC] the last node of the repair list). In this case the PLR node that is [PC] crafting the SID list wants to ensure that the last segment in the [PC] repair list is able to perform decapsulation.

[PC2] Sure. I've included the following text in rev23.
<NEW>
One of the applications of the USD flavor is the case of TI-LFA in P routers with encapsulation. The USD flavor allows the last Segment Endpoint Node in the repair path list to decapsulate the IPv6 header added at the TI-LFA Point of Local Repair and forward the inner packet.
</NEW> 

> lookup result in a local destination?
[PC] Please refer the previous comment about the use-case and so yes, we [PC] normally expect the decapsulation results in a packet that is forwarded [PC] out. However, the inner packet may also be destined to a local address.

Just to make sure I understand, if it is ok for the destination to be a local address then the external observable behavior is equivalent to End, right?

[PC2] Not quite. In order to get the same externally observable behavior you would need to configure the END Upper-layer header processing defined in 4.1.1 to allow decapsulation and processing of inner IP packets. Only in that case you would get the same externally observable behavior. This seems to be orthogonal to whether the destination of the inner packet is a local address or not.  

> result in a forwarded packet? Same question for the USP behavior?
[PC] Please refer to the previous comments. There is no such assumptions on [PC] neither the base End behavior nor End with USP.

I see how there may be differences, in line with the use cases, and how some of the behaviors may end up being the same...sometimes.
Let's then add sample use case information and move on.

[PC2] Indeed, I see your point. Added the information on the applications as per the replies above.

...
> (4) §10.2 creates a new registry with an "FCFS" registration procedure.
...
[PC] Indeed, the current text is wrong. My bad. I've updated the text with [PC] this diff below. Also, I’m not sure whether that paragraph is really [PC] needed. Maybe just putting in the table “First Come First Served [PC] [RFC8126]” is sufficient as RFC8126 already describes what is written in [PC] the text below. If it can be removed please let me know.

[PC2] Thanks for the confirmation. Removed in rev23.

...

...
> please include an example. If not, please explain any limitations.
[PC] The examples are based on real deployments and as such reflect the [PC] practical aspects of those operators SRv6 infrastructure allocation [PC] designs. It would be counter-productive and misleading to provide [PC] artificially manufactured examples (and then why just /96 and not [PC] something else?). The document does not pose any limitations.

I don't see how it can be misleading to provide a different example.
The fact that all the examples have similar assumptions might give the impression that a limitation exists.  I defer to you/AD.

[PC2] Sure. We have also added text to indicate that these examples do not preclude other addressing and allocation schemes. 
<NEW>These examples do not preclude any other IPv6 addressing allocation scheme.</NEW>

...
> unexpected entries in the L2 table. Please point to the EVPN documents 
> where this type of operation is considered.
>
[PC] Indeed, text is inaccurate. I've updated the note to the following:
[PC]
[PC] S03. In EVPN RFC7432, the learning of the exposed MAC Source Address is [PC] done via control plane. In L2VPN VPLS RFC4761 RFC4762 reachability is [PC] obtained by standard learning bridge functions in the data plane.

I'm not sure if the updated note means that S03 doesn't apply to EVPN, or if it just confirms that EVPN expects to learn from the control plane.  ??

[PC2] Both. In EVPN learning occurs via control plane, hence the EVPN process does not leverage line S03. In L2VPN VPLS the reachability is learned through learning bridge function in the dataplane (hence leveraging line S03).  

...
> generate an SRH. I didn't find this threat mentioned in rfc8402/rfc8754.
[PC] The control plane protocol specifics are outside the scope of this [PC] document. I am not able to parse this comment and what is it that needs [PC] to be addressed in this document.

Sorry, let me try again:

In the data plane...  A rogue SR headend may, on purpose, use the wrong endpoint SID.

For example, an endpoint may support End.DT6 for several IPv6 tables.
If the wrong SID is used, then the wrong table may be used to forward the packet.  If multiple tenants have overlapping addresses, the packet may then be forwarded to the wrong destination.

I believe that this is a new vulnerability introduced my this document that cannot be mitigated using the HMAC TLV, for example.  IOW, the draft assumes that the SRH will be populated with the correct SIDs, but that may not always be the case if a node has been overtaken.
There's not much that can be done to mitigate this issue, but I think it is important to mention.

[PC2] I am trying to follow the point of this discussion. This is exactly the same in MPLS today  -or in VxLAN, …-, no? If an Ingress PE sets the wrong Service label on the packet, then the packet will be sent to the wrong VPN instance, no?

> summary in §9.4 puzzles me more; what is the intent? Does Table 1 
> clearly says that the control plane is outside the scope of this document.
>
[PC] The section provides an overview of the role of control plane routing [PC] protocols in the advertisements of the SRv6 Locator and the SIDs along [PC] with their behaviors – all new aspects that have been introduced in this [PC] document. Based on the SRv6 solutions developed around the behaviors [PC] introduced in this document, it indicates what information is expected [PC] to be advertised via which protocol. It does not describe “how” since [PC] that is clearly outside the scope of this document and part of the [PC] individual routing protocol extensions.

So...does that mean that an IGP is not expected to advertise H.Encaps.L2, or that it cannot advertise it?   I'm trying to figure out if there is any normative expectation -- I'm assuming not because it's been said several times that the control plane if out of scope.
If the control plane is not bound by whatever this section says, then why do we even have it?

[PC2] This document provides a high-level view of how control plane protocols may interact with this document. This document provides what would be a logical control-plane advertising. That said, this document is not introducing any normative requirement/limitation for control planes.
To your question of why having it: if I would be reading this from scratch, I would first read the SRH and then NET-PGM. After reading NET-PGM I would go into the specific routing protocol documents. To me it would be nice to already in NET-PGM provide a bit of an overview of what to expect in each one of those…even if this is not normative text.
 
I noticed that you wrote that the text is "Based on the SRv6 solutions developed around the behaviors introduced in this document", which I guess means that this is a reflection of the existing control plane drafts (??).  If that is the case, then, at best, this seems like an attempt to put the carriage in front of the horses, knowing what the outcome may be, but without making reference to them.  I still don't see the value.

This is a non-blocking point -- I think this section can be removed and the value of the document wouldn't be reduced.  I will defer to Martin.

[PC2] I believe this section eases the reading of this document and the associated ones; but I agree that this section is not giving any normative definition. I understand your point, but I do think it has value. I’ll check with Martin.

Thank you for your time.