Subject: Re: [sfc] OK. Enough with the looping (spiralling :-) [Was: decrementing service function path index]

From: "Adrian Farrel" <[email-masked]>
Date: Sun, 6 Nov 2016 10:08:08 -0000

I'm wondering whether debating the dictionary definition of "decrement" is going to be profitable (especially as you'll find it hard to discover this verbized noun in a dictionary). In common usage it appears to mean "decrease", and meaning of the noun "decrement" appears to imply a general reduction.

However, clearly in computing, when applied to a counter (such as a loop counter), decrement is meant to mean "reduce by one".

Now we *could* go into a discussion about whether the SI is a hop counter and has as its main purpose being a TTL for the packet. Or we c*could* argue about whether the SI's primary purpose is as a pointer into the SFP.

I don't believe any of that discussion is helpful. Some people read "MUST decrement the SI" to mean reduce by one, and some thought it meant apply a reduction. I don't even think it will help us to count up who thought what.

It is certain, I think, that introducing clarity to the spec is needed. All we have to agree is what it should say.

I want to enable and support your implementation, Dave. 
It must be allowed for your implementation to automatically decrement SI by 1. And it must be allowed for your implementation to not have to support a configuration interface to vary that behavior.  Hence the text I proposed makes it clear that this is the default behavior.

However, I cannot see why it is harmful to allow an SF to make a different reduction in the value of SI. This will not break the SFF because it provides generic processing based on the SPI/SI of the received packet, and it doesn't break the SI itself. The only other moving part is the Classifier and it, too, operates on the packet in hand and has no expectations of the SI value. 

If someone can show me how an SF that modifies the SI by other than -1 would break the system or architecture, I will gladly shut up.

BTW. Consider a firewall SF. It receives packets and either filters them out or passes them on. Suppose that, instead of dropping the packets it filters out it wants to send them to a different SF that performs various security analysis tasks. To forward the packets it just needs to decrement the SI and the packet will continue down the SPF. To pass the filtered packets to the analysis SF it needs to change the SFP/SI - are we saying that the firewall SF is not allowed to do this, but a Classifier (co-resident or not) must be passed instructions (presumably in metadata) to do this?

Adrian

> -----Original Message-----
> From: Dave Dolson [mailto:[email-masked]]
> Sent: 03 November 2016 17:54
> To: Jim Guichard (jguichar); [email-masked]; [email-masked]
> Subject: RE: [sfc] OK. Enough with the looping (spiralling :-) [Was: decrementing
> service function path index]
> 
> I have never understood "increment" or "decrement" used alone to mean
> anything other than "by 1".
> One writes, "decrement by X" if X is not 1.
> 
> This issue matters to me because Sandvine has a service function that performs
> service index decrementing (by one, of course) without requiring control-plane
> configuration.
> To me the functional separation of Classifier and Service Function is not academic;
> it's real, and it is an elegant separation of responsibility.
> 
> Referring also to draft-ietf-sfc-control-plane, the C3 interface to the SF does not
> include forwarding information (with the exception of how to send a reverse
> packet--but there are alternatives).
> 
> So in terms of "seeing it done right", the functional separation and simplicity is
> "right".
> 
> 
> -----Original Message-----
> From: sfc [mailto:[email-masked]] On Behalf Of Jim Guichard (jguichar)
> Sent: Thursday, November 03, 2016 12:21 PM
> To: [email-masked]; [email-masked]
> Subject: Re: [sfc] OK. Enough with the looping (spiralling :-) [Was: decrementing
> service function path index]
> 
> Hi Adrian,
> 
> What I intended has no bearing on the discussion and I am simply pointing out
> that the WG has never to my knowledge exhibited confusion on this point
> although I accept that the assumption of decrement by 1 should be clarified. I
> might also point out that numerous implementations are already doing the
> decrement by 1 so I guess the confusion was not widespread. So in all fairness
> why is it unreasonable to ask why a behavioral change is necessary when the
> existing architecture already supports what is being asked for?
> 
> Jim
> 
> 
> On 11/3/16, 12:04 PM, "sfc on behalf of Adrian Farrel" <[email-masked] on
> behalf of [email-masked]> wrote:
> 
> >Gee shucks, Jim,
> >
> >Do you think changing "decrement" to "decrement by 1" is not a behavioral
> change just because it is what you intended when (with your chair hat off) you
> reviewed the document?
> >
> >Or are you one of the people you are asking (with your chair hat on) to bring
> forward new information?
> >
> >I, too, would like to see the NSH spec ship, but I would prefer to see it right.
> >
> >Adrian
> >
> >PS If both you and Joel simultaneously remove your chair hats, will we hit an
> iceberg? :-)
> >
> >PPS I find the discussion about what an SF does compared to what an SF with a
> co-resident re-classifier does, to be dancing on the head of a pin labelled
> "Functional Architecture". You are all welcome to implement a functional
> architecture (and bless you for it), but that is not how products are marked when
> they ship. You will not be selling "Firewall with Added Reclassifier". Furthermore,
> the SFF will be entirely unaware of whether reclassification took place. So (at the
> risk of repeating myself), why is "SF MUST decrement by 1" a requirement in this
> spec, and why is my proposed compromise text not acceptable?
> >
> >> -----Original Message-----
> >> From: Jim Guichard (jguichar) [mailto:[email-masked]]
> >> Sent: 03 November 2016 14:53
> >> To: Joel M. Halpern; Eric C Rosen; Dave Dolson; Adrian Farrel; Surendra Kumar
> >> (smkumar); [email-masked]
> >> Subject: Re: [sfc] OK. Enough with the looping (spiralling :-) [Was:
> decrementing
> >> service function path index]
> >>
> >> [Chair hat off…]
> >> Given that the currently defined architecture allows for the requested
> behavior
> >> through reclassification, I personally see no reason for a change to the way
> >> decrement of service index is currently defined, other than making it explicit
> that
> >> decrement means “by 1”.
> >>
> >> [Chair hat on…]
> >> I would note that we already have working group agreement on the current
> NSH
> >> specification. Also, the issue of the ease of modification of SFPs was part of
> >> earlier working group discussion.  Is there new information that those
> requesting
> >> a behavioral change wish to bring forward?
> >>
> >> Jim
> >>
> >>
> >>
> >> On 11/3/16, 9:55 AM, "sfc on behalf of Joel M. Halpern" <sfc-
> [email-masked]
> >> on behalf of [email-masked]> wrote:
> >>
> >> >You ask whether using non-consecuitive integers for the SI in the path
> >> >breaks something.
> >> >It is not forbidden.  You can, after all, reclassify at every hop.  (And
> >> >many current solutions do so.)
> >> >
> >> >However, it is misleading as to the intent and expected usage of SFC.
> >> >While one can manage to make it work with monotonically decreasing but
> >> >non-consecutive SI, it is significantly more complex.
> >> >
> >> >As far as I can tell, the place where this is an issue for the BGP SFC
> >> >control draft is not in the protocol definition.  Rather, it is in the
> >> >examples which all treat non-consecutive as the normal case.  This is
> >> >confusing to readers and likely to drive folks towards assumptions about
> >> >SFC behavior that will not work without additional mechanisms.
> >> >
> >> >Yours,
> >> >Joel
> >> >
> >> >On 11/3/16 9:20 AM, Eric C Rosen wrote:
> >> >> On 11/2/2016 4:50 AM, Dave Dolson wrote:
> >> >>> For those reasons, I would prefer to see the SF action limited to
> >> >>> decrement-by-one. If you require otherwise, co-locate a classifier
> >> >>> with the SF.
> >> >>
> >> >> From the perspective of draft-mackie-bess-nsh-bgp-control-plane, it
> >> >> doesn't matter whether the SPI/SI gets modified by the SF or by a
> >> >> co-resident classifier.  The only thing that matters is what SPI/SI
> >> >> values the SFF will need to use as input to its forwarding engine.
> >> >>
> >> >> A slightly different issue is whether the successive elements of an SPI
> >> >> have to have successive integers as their respective SI's.  That doesn't
> >> >> seem to be necessary for the proper functioning of the system, so we
> >> >> didn't want to assume it.  By not using successive integers, it becomes
> >> >> easier to insert and/or delete entries.  And if the SF knows the SI
> >> >> (call it "x") of the current element in the chain, it doesn't really
> >> >> have to know the SI of the next element in the chain, since the SFF can
> >> >> interpret "x-1" to mean "next after x", and can then modify the NSH to
> >> >> change the SI to the actual SI of the next element in the chain.
> >> >>
> >> >> Does this create a technical problem of some sort?
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> >> >
> >> >_______________________________________________
> >> >sfc mailing list
> >> >[email-masked]
> >> >https://www.ietf.org/mailman/listinfo/sfc
> >
> >_______________________________________________
> >sfc mailing list
> >[email-masked]
> >https://www.ietf.org/mailman/listinfo/sfc
> _______________________________________________
> sfc mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/sfc

---

From: Loa Andersson <[email-masked]>
Date: Sun, 6 Nov 2016 18:56:08 +0800

Adrian,

On 2016-11-06 18:08, Adrian Farrel wrote:
> If someone can show me how an SF that modifies the SI by
> other than -1 would break the system or architecture, I will
> gladly shut up.

I can't think of a case where it is harmful to decrement by another
value than 1.

OTOH hand have shown that it is useful?

/Loa

-- 

Loa Andersson                        email: [email-masked]
Senior MPLS Expert                          [email-masked]
Huawei Technologies (consultant)     phone: +46 739 81 21 64

---

From: "Joel M. Halpern" <[email-masked]>
Date: Sun, 6 Nov 2016 11:17:17 -0500

With regard to your last question, the approach for security functions 
(which I agree is one of the common kinds of service functions to 
trigger branching), what we said in WG discussion was that conceptually, 
the SF adds metadeata indicating that this is a security-violating 
packet, and that a classifer marks the packet with the correct new SFP. 
That classifer can be colocated, or it can be adjacent.  Separating this 
means that the needed control information for translating the metadata 
to the new SFP ID and Index is associated with the classification 
functionality.  And by modeling it that way we allow the range of 
implementations without requiring control to know which implementation 
is being done.

At least that was what I understand during the discussions (when I was a 
participant, not WG co-chair.)

Yours,
Joel

On 11/6/16 5:08 AM, Adrian Farrel wrote:
...
> BTW. Consider a firewall SF. It receives packets and either filters
> them out or passes them on. Suppose that, instead of dropping the
> packets it filters out it wants to send them to a different SF that
> performs various security analysis tasks. To forward the packets it
> just needs to decrement the SI and the packet will continue down the
> SPF. To pass the filtered packets to the analysis SF it needs to
> change the SFP/SI - are we saying that the firewall SF is not allowed
> to do this, but a Classifier (co-resident or not) must be passed
> instructions (presumably in metadata) to do this?
>
> Adrian

---

From: "Adrian Farrel" <[email-masked]>
Date: Sun, 6 Nov 2016 23:43:39 -0000

> packet, and that a classifer marks the packet with the correct new SFP.

Right, I see that in 4.8 of 7665.
What we said in draft-mackie-bess-nsh-bgp-control-plane-01.txt was...
   An SFI may influence this choice process by passing additional
   information back along with the packet and NSH.

I think this is aligned with "conceptual addition of metadata", so I believe we are aligned. Is that OK?

Adrian

---

From: <[email-masked]>
Date: Mon, 7 Nov 2016 13:36:54 +0000

Hi Joel, all,

Another way to proceed, is what is documented in the CP draft:

==
   SFs may need to output some processing results of packets to the SFC
   control plane.  This information can be used by the SFC control plane
   to update the SFC classification rules and the SFP Forwarding Policy
   Table entries.

...

   o  SF execution status: Some SFs may need to send information to the
      control plane to fine tune SFPs.  For example, a threat-detecting
      SF can periodically send the threat characteristics via this
      interface, such as high probability of threat with packet of a
      given size.  The control plane can then add an appropriate
      matching criteria to SFF to steer traffic to a scrubbing center.
===

Cheers,
Med

> -----Message d'origine-----
> De : sfc [mailto:[email-masked]] De la part de Joel M. Halpern
> Envoyé : dimanche 6 novembre 2016 17:17
> À : [email-masked]; 'Dave Dolson'; [email-masked]
> Objet : Re: [sfc] OK. Enough with the looping (spiralling :-) [Was:
> decrementing service function path index]
>

---

From: "Joel M. Halpern" <[email-masked]>
Date: Mon, 7 Nov 2016 08:41:15 -0500

Yes, notifying control about classification of future packets in a flow 
is an allowed and useful security SF behavior.  It is however not 
sufficient.  We also need to allow (and do) local notification from the 
SF to the classifier using metadata in the packet.

Yours,
Joel

---

From: <[email-masked]>
Date: Mon, 7 Nov 2016 14:08:19 +0000

Joel,

Can you please clarify what you meant by "We also need to allow (and do) local notification from the
SF to the classifier using metadata in the packet."

I'm not sure to parse that part correctly. 

Do you mean that a forward classifier will consume metadata that will be inserted by an SF or that a classifier will instruct an SF by means of some metadata to send notifications to the classifier?

Thank you.

Cheers,
Med

> De : Joel M. Halpern [mailto:[email-masked]]
> Envoyé : lundi 7 novembre 2016 14:41
> À : BOUCADAIR Mohamed IMT/OLN; [email-masked]; 'Dave Dolson';
> [email-masked]

---

From: "Joel M. Halpern" <[email-masked]>
Date: Mon, 7 Nov 2016 09:22:24 -0500

In a common case, the SF determines that the packet it is currently 
holding needs to be moved to a different chain.
Changing the chain is done by a classifier.
The conceptual model for this case is that the SF adds information to 
the metadata, and triggers reclassification.  The classifier uses the 
metadata to determine what chain the packet belongs on.

As an implementation, the classifier may be co-located with the SF, and 
the metadata may not actually appear anywhere.  (Conceptually, the 
classifier chose to remove it, implementation-wise the interface was 
internal and could do whatever it needed.

Another valid implementation is that the classifier is at the SFF, and 
the metadata is visible on the interface to the SFF / Classifier.

Whatever the implementation, this behavior is usually (but not always) 
used in conjunction with the control notification that the packets of 
the flow belong on a different flow.

Yours,
Joel

---

From: "Joel M. Halpern" <[email-masked]>
Date: Mon, 7 Nov 2016 09:24:42 -0500

(For context, the note I just sent, retained below, is a recapitulation 
of earlier WG discussions. -- Joel)

On 11/7/16 9:22 AM, Joel M. Halpern wrote:
>> Do you mean that a forward classifier will consume metadata that will
>> be inserted by an SF or that a classifier will instruct an SF by means
>> of some metadata to send notifications to the classifier?
>>
>> Thank you.
>>
>> Cheers,
>> Med
>>

---

From: Eric C Rosen <[email-masked]>
Date: Thu, 10 Nov 2016 13:35:59 -0500

> OTOH hand have shown that it is useful? 

I think there is a real issue here, but I don't think it has been 
captured well in this discussion.

IMHO the real issue is whether the successive elements (SFIs) in a 
service path (as identified by a specific SPI) are required to have 
successive integers as their SIs.

I don't think this has much to do with the behavior of the SF.   I 
understand that the SF needs a way to mark the NSH to indicate "I'm done 
with this packet".   Apparently the only way for the SF to do this is to 
decrement the SI by something.  And we'd like the SF to be able to 
indicate "I'm done" without knowing anything about the SPI.  This 
implies that the decrement needs to be a fixed value. And if it's going 
to be a fixed value, it might as well be 1.

It wouldn't make sense to have an option to configure an SF to 
"decrement by 5", say, because if we don't want to require the 
successive elements of the SPI to have successive integers as their 
respective SIs, we also don't want to require that they have successive 
multiples of 5 as their SIs.  If we wanted the SF to do anything other 
than decrement by 1, we'd really need to pass it enough information 
about the SPI to enable it to identify the next hop.  That seems 
undesirable.

So suppose the SPI has two elements, one with SI=3 and one with SI=5.  
If an SF gets a packet with SI=5 in the NSH, it processes the packet, 
sets the SI to 4, and sends the packet to the SFF.

Initially I thought that the SFF could interpret "SI=4" as meaning "the 
next hop after SI=5".  The SFF knows the SPI, can see that the next hop 
after 5 is 3, and thus could change the SI in the NSH to 3, and then 
forward the packet.   However, I've been told that the SFF isn't 
supposed to do stuff like this, because only a classifier can modify the 
SPI/SI at this point.  So  instead we'd have to say that the SFF has a 
co-resident classifier that modifies the SI as described.   Of course, 
if you deploy SFFs that don't have the co-resident classifier then you 
wouldn't be able to deploy SPIs with non-successive integers as the 
SIs.  Well, if you want to deploy an optional feature, you need to 
deploy software that supports that feature.

I've only seen one really interesting argument against creating SPIs 
with non-successive SIs, the argument that says that that would make it 
more difficult to figure out the "reverse" of an SPI.  This will require 
some further thought.

Now to Loa's question (or my reinterpretation of it) of whether there is 
an advantage to having non-successive SIs in an SPI, I'd say yes.  If 
you think that SPIs may get modified (by insertion or deletion of 
elements) dynamically, and that such modifications need to be 
distributed dynamically, it seems like a good idea to be able to make 
these modifications without changing the SIs of the elements that were 
present before the modification.   After all, one can think of an SPI/SI 
as an address of an SFI.  You want to be able to add/remove SFIs from an 
SPI without changing the addresses of all the SFIs.  This will get you 
much more consistent/predictable behavior during the convergence period 
where not everyone knows about the change yet.  (Imagine if you couldn't 
add or remove a system from the network without changing the addresses 
of all the other systems.)

Of course, someone will say that the WG has already determined that the 
SPIs never need to be modified dynamically.  But that's really
a deployment decision, not a decision that should be forced by the 
architecture or the protocol design.

---

From: Lucy yong <[email-masked]>
Date: Fri, 11 Nov 2016 01:41:09 +0000

Hi Eric,

"You want to be able to add/remove SFIs from an SPI without changing the addresses of all the SFIs."

Long time ago, we argued that using SPI/SI to indicate SI's location on a SFC path has the nature where adding/removing a SI to an SFC results the change of some SIs locations (those after the SI), thus SFFs have to be updated accordingly. We thought that is bad design. The counter argument then was that, adding/removing a SI to an existing SFC is considered as a new SFC that will be given a new SPI, configuring SFFs for a new SPC is a normal operation.

Not sure what convergence concern you have. Do you want that, if an SFF can't reach an SF temporarily due to a failure, allow the SFF to resets SI and forward to next SI?   

Regards,
Lucy 

-----Original Message-----
From: sfc [mailto:[email-masked]] On Behalf Of Eric C Rosen
Sent: Thursday, November 10, 2016 12:36 PM
To: Loa Andersson; [email-masked]; 'Dave Dolson'; [email-masked]
Subject: Re: [sfc] OK. Enough with the looping (spiralling :-) [Was: decrementing service function path index]

> OTOH hand have shown that it is useful? 

I think there is a real issue here, but I don't think it has been captured well in this discussion.

IMHO the real issue is whether the successive elements (SFIs) in a service path (as identified by a specific SPI) are required to have successive integers as their SIs.

I don't think this has much to do with the behavior of the SF.   I 
understand that the SF needs a way to mark the NSH to indicate "I'm done 
with this packet".   Apparently the only way for the SF to do this is to 
decrement the SI by something.  And we'd like the SF to be able to indicate "I'm done" without knowing anything about the SPI.  This implies that the decrement needs to be a fixed value. And if it's going to be a fixed value, it might as well be 1.

It wouldn't make sense to have an option to configure an SF to "decrement by 5", say, because if we don't want to require the successive elements of the SPI to have successive integers as their respective SIs, we also don't want to require that they have successive multiples of 5 as their SIs.  If we wanted the SF to do anything other than decrement by 1, we'd really need to pass it enough information about the SPI to enable it to identify the next hop.  That seems undesirable.

So suppose the SPI has two elements, one with SI=3 and one with SI=5.  
If an SF gets a packet with SI=5 in the NSH, it processes the packet, sets the SI to 4, and sends the packet to the SFF.

Initially I thought that the SFF could interpret "SI=4" as meaning "the next hop after SI=5".  The SFF knows the SPI, can see that the next hop after 5 is 3, and thus could change the SI in the NSH to 3, and then 
forward the packet.   However, I've been told that the SFF isn't 
supposed to do stuff like this, because only a classifier can modify the SPI/SI at this point.  So  instead we'd have to say that the SFF has a 
co-resident classifier that modifies the SI as described.   Of course, 
if you deploy SFFs that don't have the co-resident classifier then you wouldn't be able to deploy SPIs with non-successive integers as the SIs.  Well, if you want to deploy an optional feature, you need to deploy software that supports that feature.

I've only seen one really interesting argument against creating SPIs with non-successive SIs, the argument that says that that would make it more difficult to figure out the "reverse" of an SPI.  This will require some further thought.

Now to Loa's question (or my reinterpretation of it) of whether there is an advantage to having non-successive SIs in an SPI, I'd say yes.  If you think that SPIs may get modified (by insertion or deletion of
elements) dynamically, and that such modifications need to be distributed dynamically, it seems like a good idea to be able to make these modifications without changing the SIs of the elements that were 
present before the modification.   After all, one can think of an SPI/SI 
as an address of an SFI.  You want to be able to add/remove SFIs from an SPI without changing the addresses of all the SFIs.  This will get you much more consistent/predictable behavior during the convergence period where not everyone knows about the change yet.  (Imagine if you couldn't add or remove a system from the network without changing the addresses of all the other systems.)

Of course, someone will say that the WG has already determined that the SPIs never need to be modified dynamically.  But that's really a deployment decision, not a decision that should be forced by the architecture or the protocol design.

---

From: Eric C Rosen <[email-masked]>
Date: Mon, 14 Nov 2016 16:06:39 -0500

On 11/10/2016 8:41 PM, Lucy yong wrote:
> Long time ago, we argued that using SPI/SI to indicate SI's location on a SFC path has the nature where adding/removing a SI to an SFC results the change of some SIs locations (those after the SI), thus SFFs have to be updated accordingly. We thought that is bad design.

I would tend to agree that that is not a good design.

> The counter argument then was that, adding/removing a SI to an existing SFC is considered as a new SFC that will be given a new SPI, configuring SFFs for a new SPC is a normal operation.

I don't find this to be credible.   Management of the SPI name space is 
an issue for the operators, it's not up to the WG to decide that the SPI 
value must change whenever an SF is added or removed.

> Not sure what convergence concern you have.
Suppose SPI 17 has two elements: SF1, SF2, SF3.  And suppose that the 
respective SPI/SI values for these three SFs are17/255, 17/254, and 
17/253 respectively.

Now suppose we want to remove SF2, so that SPI 17 now has two elements: 
SF1, SF3.  We have to change SF3's SPI/SI to 17/254.

During the convergence period, some nodes will think that 17/254 means 
"SF2" (old value), and some will think that it means "SF3" (new value).  
Further, some nodes will think that 17/253 means "SF3", and some will 
think that 17/253 is an invalid value.

So consider an NSH packet, with SPI/SI 17/254, at SFF1.  If SFF1 knows 
only the old value of SPI 17, it will send the packet to SF2. SFF1 will 
get the packet back with an SPI/SI of 17/253.  To SFF1, this means "next 
hop is SF3".  Let's suppose SFF1 forwards the packet to SFF2, because it 
knows that there is an instance of SF3 attached to SFF2.  However, SFF2 
knows the new value of SPI 17, and hence thinks that 17/253 is invalid.  
So SFF2 drops the packet.  A packet has now been lost during the 
convergence period.

This packet loss would not have happened if we had not changed the 
SPI/SI of SF3 from 17/253 to 17/254.  In that case, SFF2 would have 
known that 17/253 means SF3.  Also, 17/253 would still be the last hop 
of SPI 17, so there'd be no confusion about when the chain was finished.

> Do you want that, if an SFF can't reach an SF temporarily due to a failure, allow the SFF to resets SI and forward to next SI?
>
>
That's an interesting question.  I'm not sure what the desired behavior 
is in that case.

---

From: "Paul Quinn (paulq)" <[email-masked]>
Date: Thu, 17 Nov 2016 13:46:56 +0000

Eric,

> On Nov 14, 2016, at 4:06 PM, Eric C Rosen <[email-masked]> wrote:
> 

<trimmed>

> Suppose SPI 17 has two elements: SF1, SF2, SF3.  And suppose that the respective SPI/SI values for these three SFs are17/255, 17/254, and 17/253 respectively.
> 
> Now suppose we want to remove SF2, so that SPI 17 now has two elements: SF1, SF3.  We have to change SF3's SPI/SI to 17/254.
> 
> During the convergence period, some nodes will think that 17/254 means "SF2" (old value), and some will think that it means "SF3" (new value).  Further, some nodes will think that 17/253 means "SF3", and some will think that 17/253 is an invalid value.
> 
> So consider an NSH packet, with SPI/SI 17/254, at SFF1.  If SFF1 knows only the old value of SPI 17, it will send the packet to SF2. SFF1 will get the packet back with an SPI/SI of 17/253.  To SFF1, this means "next hop is SF3".  Let's suppose SFF1 forwards the packet to SFF2, because it knows that there is an instance of SF3 attached to SFF2.  However, SFF2 knows the new value of SPI 17, and hence thinks that 17/253 is invalid.  So SFF2 drops the packet.  A packet has now been lost during the convergence period.
> 
> This packet loss would not have happened if we had not changed the SPI/SI of SF3 from 17/253 to 17/254.  In that case, SFF2 would have known that 17/253 means SF3.  Also, 17/253 would still be the last hop of SPI 17, so there'd be no confusion about when the chain was finished.

This convergence issue isn't really germance to NSH.  You need to have a correct view of the state of the network, for a given plane, when moving packets.

---

From: "Paul Quinn (paulq)" <[email-masked]>
Date: Thu, 17 Nov 2016 13:47:23 +0000

> On Nov 10, 2016, at 1:35 PM, Eric C Rosen <[email-masked]> wrote:
> 
> I don't think this has much to do with the behavior of the SF.   I understand that the SF needs a way to mark the NSH to indicate "I'm done with this packet".   Apparently the only way for the SF to do this is to decrement the SI by something.  And we'd like the SF to be able to indicate "I'm done" without knowing anything about the SPI.  This implies that the decrement needs to be a fixed value. And if it's going to be a fixed value, it might as well be 1.
> 
> So suppose the SPI has two elements, one with SI=3 and one with SI=5.  If an SF gets a packet with SI=5 in the NSH, it processes the packet, sets the SI to 4, and sends the packet to the SFF.
> 
> Initially I thought that the SFF could interpret "SI=4" as meaning "the next hop after SI=5".  The SFF knows the SPI, can see that the next hop after 5 is 3, and thus could change the SI in the NSH to 3, and then forward the packet.   However, I've been told that the SFF isn't supposed to do stuff like this, because only a classifier can modify the SPI/SI at this point.  So  instead we'd have to say that the SFF has a co-resident classifier that modifies the SI as described.   Of course, if you deploy SFFs that don't have the co-resident classifier then you wouldn't be able to deploy SPIs with non-successive integers as the SIs.  Well, if you want to deploy an optional feature, you need to deploy software that supports that feature.
> 
> Now to Loa's question (or my reinterpretation of it) of whether there is an advantage to having non-successive SIs in an SPI, I'd say yes.  If you think that SPIs may get modified (by insertion or deletion of elements) dynamically, and that such modifications need to be distributed dynamically, it seems like a good idea to be able to make these modifications without changing the SIs of the elements that were present before the modification.   After all, one can think of an SPI/SI as an address of an SFI.  You want to be able to add/remove SFIs from an SPI without changing the addresses of all the SFIs.  This will get you much more consistent/predictable behavior during the convergence period where not everyone knows about the change yet.  (Imagine if you couldn't add or remove a system from the network without changing the addresses of all the other systems.)
> 
> Of course, someone will say that the WG has already determined that the SPIs never need to be modified dynamically.  But that's really
> a deployment decision, not a decision that should be forced by the architecture or the protocol design.
> 
> 

On the contrary: dynamically updating SPI is supported and the need for that functionality is clear.

---

From: "Paul Quinn (paulq)" <[email-masked]>
Date: Thu, 17 Nov 2016 13:47:19 +0000

Eric,

> I don't find this to be credible.   Management of the SPI name space is an issue for the operators, it's not up to the WG to decide that the SPI value must change whenever an SF is added or removed.
> 

This convergence issue isn't germance to NSH.  Devices need to have a consistent view of the state of the network, for a given plane, when moving packets.  You are implying some things about the control plane, that might not be true, if for, example a "pull" model is used.

> That's an interesting question.  I'm not sure what the desired behavior is in that case.

In general it comes down to simple policy.  For the most part services, particularly, when security services are involved, the default is to stop sending packets down the chain.

---

From: Eric C Rosen <[email-masked]>
Date: Thu, 17 Nov 2016 11:15:33 -0500

Paul> This convergence issue isn't germane to NSH.

I don't see how such a statement could possibly be true.  The behavior 
of the routing system during the convergence period after a change 
always has to be understood.

Paul> Devices need to have a consistent view of the state of the 
network, for a given plane, when moving packets.

Routing in general would be a lot simpler if convergence were 
instantaneous.  In reality, devices do not always have a consistent view 
of the state of the network.

Paul> You are implying some things about the control plane, that might 
not be true, if for, example a "pull" model is used.

It is true that I was assuming a particular control plane model, but I 
don't think a "pull" model eliminates the convergence issues.

Lucy> Do you want that, if an SFF can't reach an SF temporarily due to a 
failure, allow the SFF to resets SI and forward to next SI?

Eric> That's an interesting question. I'm not sure what the desired 
behavior is in that case.

Paul> In general it comes down to simple policy. For the most part 
services, particularly, when security services are involved, the default 
is to stop sending packets down the chain.

Can the policy be different for different SF types?  If so, is the SFF 
expected to know the type of the next hop SF, so that it can determine 
the proper policy to apply when the next hop is unreachable?

Eric> Of course, someone will say that the WG has already determined 
that the SPIs never need to be modified dynamically. But that's really a 
deployment decision, not a decision that should be forced by the 
architecture or the protocol design.

Paul> On the contrary: dynamically updating SPI is supported and the 
need for that functionality is clear.

Glad to hear it.  But some of the long-time WG participants have 
suggested that if you modify a service path dynamically, you have to 
change the SPI value that you use to denote that path in the NSH.

Eric> each SFI knows the SFF to which it is directly connected

Paul> SFIs don't need to know that: they simply return the packet from 
whence it came.

It's pretty hard to return a packet from whence it came if you don't 
know from whence it came.   Perhaps you are saying that each SFI has a 
co-resident SFF, so it always knows how to return the packet to it.  But 
that's not any different than what I said, except for the fact that 
you'll have the looping problem I described.

Paul> You appear to be thinking of SFF as a PE-like device. That is 
decidedly not what SFFs are. They are simply logical front-ends for 
service functions.

I was just thinking of an SFF as an abstract entity of some sort that 
looks up an SPI/SI value, and determines from that lookup what the next 
SFI hop is, and how to get the packet there.

Paul> As I mentioned above, the SFI, SF2-2 in this case, simply returns 
the packet to SFF1. Nothing more complex than that.

That wouldn't make much sense in my example, even assuming that SF2-2 
knew how to return the packet to SFF1, because SFF1 and SF2-2 are not 
co-resident, and you wouldn't want to send the packet across the network 
just to figure out where its next hop is.

Paul> "remote" and "direct" are distinctions without a difference.

I don't see how that could possibly be the case.  Are you saying that 
SFC load-balancing decisions are not allowed to take factors like 
latency or load into account?

Paul> The current (implemented) model works, is simple to understand and 
provides the requisite behavior for chaining.

What a convincing argument!

---

From: "Paul Quinn (paulq)" <[email-masked]>
Date: Fri, 18 Nov 2016 18:39:31 +0000

> On Nov 17, 2016, at 11:15 AM, Eric C Rosen <[email-masked]> wrote:
> 
> 
> I don't see how such a statement could possibly be true.  The behavior of the routing system during the convergence period after a change always has to be understood.
> 

Eric, you are correct, rather, I should have said: unique to NSH, nor does NSH introduce any unique requirements.  In fact, given the limited scope of NSH forwarding (i.e. not global routing), NSH lookup merely results in mapping to the network topology, where routing occurs.

---

From: Eric C Rosen <[email-masked]>
Date: Mon, 21 Nov 2016 13:22:21 -0500

On 11/18/2016 1:39 PM, Paul Quinn (paulq) wrote:
> given the limited scope of NSH forwarding (i.e. not global routing), NSH lookup merely results in mapping to the network topology, where routing occurs.
Each SFF looks up a single SPI/SI value to determine the next service 
hop, i.e., the next SFI, to which a given packet needs to be sent.  If 
the SFI denoted by a particular SPI/SI value changes while the packet is 
in flight, the SFFs may make conflicting decisions. If it takes time for 
all the SFFs to see that the SPI/SI value's denotation has changed, then 
there is an NSH-specific convergence issue.

This can be avoided if the SFI(s) denoted by a particular SPI/SI value 
do not change while a packet is in flight.

If we want to allow the path denoted by a particular SPI value to change 
(via the addition or deletion of SFIs), but we don't want SFI(s) denoted 
by a particular SPI/SI to change, then the SIs in a given path cannot be 
required to be sequential integers.