Subject: [sfc] decrementing service function path index

From: "Joel M. Halpern" <[email-masked]>
Date: Tue, 1 Nov 2016 13:26:32 -0400

I was having a discussion (see IDR list) about SFC, and it was observed 
that in the NSH draft we never specify a meaning for "decrement".

While I have assumed a meaning, assumptions are not good for 
interoperable implementations.

Do people understand the current document to require decrementing by 1, 
or to allow a larger decrement?

More importantly, what do we want to say in the spec about this?  If we 
want to allow decrement by more than 1, since the SF is doing the 
decrement, we need to talk about the SF being configured to know how 
much to decrement.  In my personal opinion, that seems overly complex 
and fragile.

Yours,
Joel

---

From: Sumandra Majee <[email-masked]>
Date: Tue, 1 Nov 2016 17:29:32 +0000

I have always assumed (implicitly) that decrement of SI means S1-1 by the service function once the service is rendered.

On 11/1/16, 10:26 AM, "sfc on behalf of Joel M. Halpern" <[email-masked] on behalf of [email-masked]> wrote:

    

    Yours,
    Joel
    
    _______________________________________________
    sfc mailing list
    [email-masked]
    https://www.ietf.org/mailman/listinfo/sfc

---

From: "Adrian Farrel" <[email-masked]>
Date: Tue, 1 Nov 2016 18:02:42 -0000

Well, of course, the discussion was on the BESS list, not the IDR list.

I have always assumed that "decrement" meant "reduce" but not necessarily
"reduce by one."

But that may be because I am puzzled about why there is a mix of functional
architecture and protocol architecture in the same document. 
When a re-classifier is "co-resident" with an SF, can anyone *outside* the SF
tell who it was that decremented the SI by more than one?

Adrian

> -----Original Message-----
> From: sfc [mailto:[email-masked]] On Behalf Of Joel M. Halpern
> Sent: 01 November 2016 17:27
> To: [email-masked]
> Subject: [sfc] decrementing service function path index
>

---

From: "Joel M. Halpern" <[email-masked]>
Date: Tue, 1 Nov 2016 14:08:21 -0400

Sorry for crossing IDR and BES.

With regard to the architectural components, the NSH draft does not 
define those.  It is using the architectural, logical, components and 
architecture defined by the SFC Architecture documents.

Yours,
Joel

---

From: John E Drake <[email-masked]>
Date: Tue, 1 Nov 2016 18:34:38 +0000

Hi,

'Decrement' has a very well defined meaning and it is different than 'decrement by 1'.  Given that service function paths need to be defined and distributed to the nodes in an SFC network I don't see any reason why that definition can't include the SI values.

This allows a service function path to be modified by either adding or deleting elements without having to renumber its unaffected portions.

Yours Irrespectively,

John

> Sent: Tuesday, November 1, 2016 1:27 PM
> To: [email-masked]
> I was having a discussion (see IDR list) about SFC, and it was observed that in the
> NSH draft we never specify a meaning for "decrement".
> 
> While I have assumed a meaning, assumptions are not good for interoperable
> implementations.
> 
> Do people understand the current document to require decrementing by 1, or to
> allow a larger decrement?
> 
> More importantly, what do we want to say in the spec about this?  If we want to
> allow decrement by more than 1, since the SF is doing the decrement, we need
> to talk about the SF being configured to know how much to decrement.  In my
> personal opinion, that seems overly complex and fragile.
> 
> Yours,
> Joel
>

---

From: Dave Dolson <[email-masked]>
Date: Tue, 1 Nov 2016 18:44:31 +0000

I had understood the service function must decrement the SI value by exactly one before returning to the SFF.
(This doesn't require any configuration.)

Given John's different interpretation, let's make it explicit.

-Dave

-----Original Message-----
From: sfc [mailto:[email-masked]] On Behalf Of John E Drake
Sent: Tuesday, November 01, 2016 2:35 PM
To: Joel M. Halpern; [email-masked]
Cc: Adrian Farrel; Eric Rosen
Subject: Re: [sfc] decrementing service function path index

Hi,

Yours Irrespectively,

John

---

From: "Adrian Farrel" <[email-masked]>
Date: Tue, 1 Nov 2016 18:47:40 -0000

> architecture defined by the SFC Architecture documents.

I'd like to understand where in 7665 it says that the SF is responsible for
moving the packet on to the next SF in the path, rather than the SFF.

I'd also like to understand why the NSH document (that tells us how to
encapsulate, and what to do at each hop in the path) needs to divide the
responsibility for bits of protocol work between different logical functional
entities. In short, why does the SF decrement the SI, why isn't this an SFF
function? What benefit comes from doing it this way or did a choice simply have
to be made?

Thanks,
Adrian

---

From: "Adrian Farrel" <[email-masked]>
Date: Tue, 1 Nov 2016 18:54:57 -0000

Hi Dave,

So the thing at question here is "can you support a non-configured chain?"
That is, can the "thing" know by how much to decrement the SI without being
configured?

Well, certainly making a rule that it MUST decrement by 1 avoids that. But note
that configuration is needed so that the SFF knows how to forward to the next
hop, so the logical node (i.e. the SF+SFF) has to be configured.

We could get further into why the SF does the decrement (not the SFF), or we
could say that:

"The SF is responsible for setting the SI value for the next hop in the SFP and
absent any other information it MUST decrement the SI by 1."

If we do that, I can do what I want, and you can do what you want, and
everything works fine. And the protocol is undisturbed!

Thanks,
Adrian

> From: sfc [mailto:[email-masked]] On Behalf Of Dave Dolson
> Sent: 01 November 2016 18:45
> To: John E Drake; Joel M. Halpern; [email-masked]
> 'Decrement' has a very well defined meaning and it is different than
'decrement
> by 1'.  Given that service function paths need to be defined and distributed
to the
> nodes in an SFC network I don't see any reason why that definition can't
include
> the SI values.
> 
> This allows a service function path to be modified by either adding or
deleting
> elements without having to renumber its unaffected portions.
> 
> Yours Irrespectively,
> 
> John
> 
> 
> > I was having a discussion (see IDR list) about SFC, and it was observed that
in
> the

---

From: "Surendra Kumar (smkumar)" <[email-masked]>
Date: Tue, 1 Nov 2016 19:03:44 +0000

There are two aspects to this. 1) Who decrements and 2) By how much

For #1) We specified doing this entirely outside the SF, in the SFF here to not have non-classifying SFs concern with SPI manipulation: https://datatracker.ietf.org/doc/draft-kumar-sfc-nsh-forwarding/

For #2) I expect a default behavior - to decrement by 1, but allow the control plane to configure it to a different number, as suggested below.

Thanks,
Surendra.

-----Original Message-----
From: sfc [mailto:[email-masked]] On Behalf Of Adrian Farrel
Sent: Tuesday, November 01, 2016 11:55 AM
To: 'Dave Dolson' <[email-masked]>; 'John E Drake' <[email-masked]>; 'Joel M. Halpern' <[email-masked]>; [email-masked]
Cc: 'Adrian Farrel' <[email-masked]>; 'Eric Rosen' <[email-masked]>
Subject: Re: [sfc] decrementing service function path index

Hi Dave,

So the thing at question here is "can you support a non-configured chain?"
That is, can the "thing" know by how much to decrement the SI without being configured?

Well, certainly making a rule that it MUST decrement by 1 avoids that. But note that configuration is needed so that the SFF knows how to forward to the next hop, so the logical node (i.e. the SF+SFF) has to be configured.

We could get further into why the SF does the decrement (not the SFF), or we could say that:

"The SF is responsible for setting the SI value for the next hop in the SFP and absent any other information it MUST decrement the SI by 1."

If we do that, I can do what I want, and you can do what you want, and everything works fine. And the protocol is undisturbed!

Thanks,
Adrian

'decrement
to the
include
deleting
in
> > Do people understand the current document to require decrementing by 
> > 1, or
> to
> > More importantly, what do we want to say in the spec about this?  If 
> > we want
> to
> > allow decrement by more than 1, since the SF is doing the decrement, 
> > we need to talk about the SF being configured to know how much to 
> > decrement.  In my personal opinion, that seems overly complex and fragile.
> >
> > Yours,
> > Joel
> >

---

From: Dave Dolson <[email-masked]>
Date: Tue, 1 Nov 2016 19:34:41 +0000

The SF decrements the index so that the SFF doesn't have to look at the packet origin in order to determine whether or not to decrement it.
We don't want the SFF to have logic like:
If NSH packet is from one of addresses x, y z 
    Decrement SI

As currently defined, the SFF simply routes packets by SPI/SI without having to do consider source address.

-----Original Message-----
From: sfc [mailto:[email-masked]] On Behalf Of Adrian Farrel
Sent: Tuesday, November 01, 2016 2:48 PM
To: 'Joel M. Halpern'; [email-masked]
Subject: Re: [sfc] decrementing service function path index

Thanks,
Adrian

---

From: "Adrian Farrel" <[email-masked]>
Date: Tue, 1 Nov 2016 19:59:55 -0000

Just clarifying my understanding of what you said (not necessarily agreeing ;-)

You are saying that an SFF is simply a forwarder and cannot tell the difference
between a packet received on a transport tunnel from another SFF and a packet
received (back) from an SF that it serves.

Or, more precisely, you are saying that the SFF is simply a forwarder that
cannot tell the difference between passing a packet to a local SF and passing a
packet on to a remote SFF.

And the SFF sees a packet n+1 times for an SFP with n local SFs.

Is that your point?

Cheers,
Adrian

--
Support an author and your imagination.
Tales from the Wood - Eighteen new fairy tales.
More Tales from the Wood - Eighteen MORE new fairy tales.
https://www.feedaread.com/profiles/8604/
http://www.amazon.co.uk/Tales-Wood-Adrian-Farrel/dp/1786100924
Or buy from me direct.

> From: Dave Dolson [mailto:[email-masked]]
> Sent: 01 November 2016 19:35
> To: [email-masked]; 'Joel M. Halpern'; [email-masked]
> Subject: RE: [sfc] decrementing service function path index
> 
> The SF decrements the index so that the SFF doesn't have to look at the packet
> origin in order to determine whether or not to decrement it.
> As currently defined, the SFF simply routes packets by SPI/SI without having
to
> do consider source address.
> 
> 
>

---

From: Dave Dolson <[email-masked]>
Date: Tue, 1 Nov 2016 20:26:37 +0000

Adrian,
Thank you for paraphrasing in a clear manner.

Yes, I'm saying that the current (as I understand it) design permits the SFF to be a simple forwarder,
not needing to discriminate between packets from another SFF or returning from an SF.
A single forwarding table handles both cases.
E.g., at SFF1, when SI=x, send to SF1; when SI=x+1, send to SFF2

So it could tell the difference, but doesn't need to.
In some cases, an SFF can be a single-interface device, so "from SFF" or "from SF" is not as simple 
as checking ingress interface, for example.

-----Original Message-----
From: Adrian Farrel [mailto:[email-masked]] 
Sent: Tuesday, November 01, 2016 4:00 PM
To: Dave Dolson; 'Joel M. Halpern'; [email-masked]
Subject: RE: [sfc] decrementing service function path index

Is that your point?

Cheers,
Adrian

to

---

From: "Surendra Kumar (smkumar)" <[email-masked]>
Date: Tue, 1 Nov 2016 23:03:40 +0000

Since the forwarders already exists, they are logical components to support and perform NSH based forwarding - forwarding on SFPs, rather than treat them as dumb forwarders. There is benefit to be had, offloads being one of them, on top of end of chain forwarding, etc. needed.

I would rather have SFs focus on servicing the traffic than be burdened with SPI management (whether it is tens or thousands of SPIs) and the forwarding thereof. IOW, consume metadata and service traffic and leave the SFP management to the external forwarders. This not only simplifies the SFs, it also reduces the control plane touch point as it concerns to the SPI management.

And you don't need complex logic to differentiate between traffic received from one SFF or a SF, this is what we addressed in the forwarding draft - trivial to implement even in hardware.

Thanks, 
Surendra.

-----Original Message-----
From: sfc [mailto:[email-masked]] On Behalf Of Dave Dolson
Sent: Tuesday, November 01, 2016 1:27 PM
To: [email-masked]; 'Joel M. Halpern' <[email-masked]>; [email-masked]
Subject: Re: [sfc] decrementing service function path index

Adrian,

Yes, I'm saying that the current (as I understand it) design permits the SFF to be a simple forwarder, not needing to discriminate between packets from another SFF or returning from an SF.
A single forwarding table handles both cases.
E.g., at SFF1, when SI=x, send to SF1; when SI=x+1, send to SFF2

So it could tell the difference, but doesn't need to.
In some cases, an SFF can be a single-interface device, so "from SFF" or "from SF" is not as simple as checking ingress interface, for example.

-----Original Message-----
From: Adrian Farrel [mailto:[email-masked]] 
Sent: Tuesday, November 01, 2016 4:00 PM
To: Dave Dolson; 'Joel M. Halpern'; [email-masked]
Subject: RE: [sfc] decrementing service function path index

Is that your point?

Cheers,
Adrian

to

---

From: "Adrian Farrel" <[email-masked]>
Date: Tue, 1 Nov 2016 23:09:05 -0000

Thanks Surendra,

You said it better than me.

I don't mind if an SF also munges with the NSH, but I think it is wrong to say:
- the SFF MUST NOT
- the SF MUST munge in a specific way.

A

> From: Surendra Kumar (smkumar) [mailto:[email-masked]]
> Sent: 01 November 2016 23:04
> To: Dave Dolson; [email-masked]; 'Joel M. Halpern'; [email-masked]
> Since the forwarders already exists, they are logical components to support
and
> perform NSH based forwarding - forwarding on SFPs, rather than treat them as
> dumb forwarders. There is benefit to be had, offloads being one of them, on
top
> of end of chain forwarding, etc. needed.
> 
> I would rather have SFs focus on servicing the traffic than be burdened with
SPI
> management (whether it is tens or thousands of SPIs) and the forwarding
> thereof. IOW, consume metadata and service traffic and leave the SFP
> management to the external forwarders. This not only simplifies the SFs, it
also
> reduces the control plane touch point as it concerns to the SPI management.
> 
> And you don't need complex logic to differentiate between traffic received
from
> one SFF or a SF, this is what we addressed in the forwarding draft - trivial
to
> implement even in hardware.
> 
> Thanks,
> Surendra.
> 
> Yes, I'm saying that the current (as I understand it) design permits the SFF
to be a
> simple forwarder, not needing to discriminate between packets from another
> SFF or returning from an SF.
> In some cases, an SFF can be a single-interface device, so "from SFF" or "from
SF"
> is not as simple as checking ingress interface, for example.
> 
>

---

From: Dave Dolson <[email-masked]>
Date: Tue, 1 Nov 2016 23:21:53 +0000

I agree that the SF should be simple.
But don't confuse decrementing the SI with complexity.
There is absolutely no configuration required to have the SF decrement the SI of every packet.
And there is a lot of benefit to keeping the SFF from having rules about packet source.

As far as I'm concerned, the SF decrements the SI, and it has been described this way for a long time. Furthermore, the forwarding is described as a lookup table based on SPI/SI. There is no mention of the source in the forwarding table.

  Original Message
From: Surendra Kumar (smkumar)
Sent: Tuesday, November 1, 2016 7:03 PM
To: Dave Dolson; [email-masked]; 'Joel M. Halpern'; [email-masked]
Subject: RE: [sfc] decrementing service function path index

Thanks,
Surendra.

-----Original Message-----
From: sfc [mailto:[email-masked]] On Behalf Of Dave Dolson
Sent: Tuesday, November 01, 2016 1:27 PM
To: [email-masked]; 'Joel M. Halpern' <[email-masked]>; [email-masked]
Subject: Re: [sfc] decrementing service function path index

Adrian,

-----Original Message-----
From: Adrian Farrel [mailto:[email-masked]]
Sent: Tuesday, November 01, 2016 4:00 PM
To: Dave Dolson; 'Joel M. Halpern'; [email-masked]
Subject: RE: [sfc] decrementing service function path index

Is that your point?

Cheers,
Adrian

to

---

From: Eric C Rosen <[email-masked]>
Date: Fri, 11 Nov 2016 11:58:21 -0500

> E.g., at SFF1, when SI=x, send to SF1; when SI=x+1, send to SFF2
I wonder what the WG has decided about the following case.

Suppose Node 1 contains SFF1 and SF1-1, while Node 2 contains SFF2 and 
SF1-2, where SF1-1 and SF1-2 are two instances of of the same service 
function (SF1).

So SFF1 is attached locally to SF1-1, and SFF2 is attached locally to SF1-2,

Now suppose that the SPI/SI value "x/y" means "SF1".

When SFF1 sees "x/y" in the NSH, it may choose to load balance between 
SF1-1 and SF1-2.  This means it may forward either locally to SF1-1 or 
over the network to SF1-2.

Suppose SFF1 chooses SF1-2.  It then needs to tunnel the packet through 
the underlay to Node 2.

What happens when the packet arrives at Node 2?  If the packet is seen 
by SFF2, SFF2 will see "x/y" in the NSH, and may choose to forward 
either locally to SF1-2 or over the network to SF1-1. Since SFF2 doesn't 
know that the packet came from another SFF, nothing stops it from 
choosing SF1-1.

That is,  SFF1 chooses to use SF1-2, and SFF2 chooses to use SF1-1, thus 
forming a loop.  Since this loop is between the two SFFs, the SI never 
gets decremented.

Presumably this type of load-balancing is allowed, but the loop must be 
prevented.

One way to prevent the loop is for SFF2 to know whether a packet with 
"x/y" in the NSH has already been seen by another forwarder or not.

> as checking ingress interface, for example.

Presumably there's a tunnel of some sort, through the underlay, that 
leads from SFF1 to SFF2.   This tunnel could be regarded as a virtula 
ingress interface.

Alternatively, SFF1 could tunnel a packet directly to SF1-2, bypassing 
SFF2 entirely.  SFF2 then wouldn't actually process the packet until 
SF1-2 has finished with it.  The load balancing decision would be made 
only once (by SFF1).  in this case, the SFFs don't need to know about a 
packet's ingress "interface" because SFFs don't get packets from each 
other directly.

This requires only that the tunnel through the underlay contain 
demultiplexing information that will cause the packet, once it arrives 
at Node 2, to be delivered directly to SF1-2, bypassing SFF2.

This latter model seems to be what is required by section 5.5 of 
RFC7665, but the RFC is not very explicit about it.

---

From: Ron Parker <[email-masked]>
Date: Fri, 11 Nov 2016 17:33:17 +0000

Eric,

I agree with your analysis.   Especially around:

   Presumably this type of load-balancing is allowed, but the loop must be prevented.

   One way to prevent the loop is for SFF2 to know whether a packet with "x/y" in the NSH has already been seen by another forwarder or not.

Do you think that an SFF could/should discern this by examining the source IP of the packet it receives, or do you think we need some sort of "source id" in the NSH base header, itself?

   Ron

-----Original Message-----
From: sfc [mailto:[email-masked]] On Behalf Of Eric C Rosen
Sent: Friday, November 11, 2016 11:58 AM
To: Dave Dolson <[email-masked]>; [email-masked]; 'Joel M. Halpern' <[email-masked]>; [email-masked]
Subject: [GRAYMAIL] Re: [sfc] decrementing service function path index

On 11/1/2016 4:26 PM, Dave Dolson wrote:
> Yes, I'm saying that the current (as I understand it) design permits 
> the SFF to be a simple forwarder, not needing to discriminate between packets from another SFF or returning from an SF.

Suppose Node 1 contains SFF1 and SF1-1, while Node 2 contains SFF2 and SF1-2, where SF1-1 and SF1-2 are two instances of of the same service function (SF1).

When SFF1 sees "x/y" in the NSH, it may choose to load balance between
SF1-1 and SF1-2.  This means it may forward either locally to SF1-1 or over the network to SF1-2.

Suppose SFF1 chooses SF1-2.  It then needs to tunnel the packet through the underlay to Node 2.

What happens when the packet arrives at Node 2?  If the packet is seen by SFF2, SFF2 will see "x/y" in the NSH, and may choose to forward either locally to SF1-2 or over the network to SF1-1. Since SFF2 doesn't know that the packet came from another SFF, nothing stops it from choosing SF1-1.

That is,  SFF1 chooses to use SF1-2, and SFF2 chooses to use SF1-1, thus forming a loop.  Since this loop is between the two SFFs, the SI never gets decremented.

Presumably this type of load-balancing is allowed, but the loop must be prevented.

One way to prevent the loop is for SFF2 to know whether a packet with "x/y" in the NSH has already been seen by another forwarder or not.

> In some cases, an SFF can be a single-interface device, so "from SFF" 
> or "from SF" is not as simple as checking ingress interface, for example.

Alternatively, SFF1 could tunnel a packet directly to SF1-2, bypassing
SFF2 entirely.  SFF2 then wouldn't actually process the packet until
SF1-2 has finished with it.  The load balancing decision would be made only once (by SFF1).  in this case, the SFFs don't need to know about a packet's ingress "interface" because SFFs don't get packets from each other directly.

This requires only that the tunnel through the underlay contain demultiplexing information that will cause the packet, once it arrives at Node 2, to be delivered directly to SF1-2, bypassing SFF2.

This latter model seems to be what is required by section 5.5 of RFC7665, but the RFC is not very explicit about it.

---

From: "Jim Guichard (jguichar)" <[email-masked]>
Date: Fri, 11 Nov 2016 17:39:32 +0000

Well unless I am missing something SFF2 in this example will never see the NSH as SFF1 will tunnel the packet directly to SF1-2 using whatever transport is in play. Remember that the SFF will only forward based on NSH if it sees it (i.e. the outer transport terminates at the SFF) but in this case it will simply see an IP packet on its merry way to SF1-2. 

Jim

---

From: Eric C Rosen <[email-masked]>
Date: Fri, 11 Nov 2016 15:25:00 -0500

Jim> Well unless I am missing something SFF2 in this example will never 
see the NSH as SFF1 will tunnel the packet directly to SF1-2 using 
whatever transport is in play.

That is exactly my point; for this sort of load balancing to work with 
NSH as it is, one needs to have a transport tunnel from each SFF to each 
remote SFI that the SFF may select as the next hop in a service path.  
This means that for every type of transport that might be "in play", you 
have to figure out what to put in the transport header in order to 
identify a particular SFI.

The fact that the transport tunnels connect SFFs to SFIs rather than 
connecting SFFs to each other is something that I don't think is 
apparent from the documents or from some of the recent threads on the 
mailing list.

(Luckily, draft-mackie-bess-nsh-bgp-control-plane handles this well, 
because it provides encapsulation information on a per-SFI basis.)

Ron>  Do you think that an SFF could/should discern this by examining 
the source IP of the packet it receives, or do you think we need some 
sort of "source id" in the NSH base header, itself

If we wanted to require that transport tunnels only go between SFFs, I 
think it would be sufficient just to have a flag in the NSH that means 
"the load balancing decision has already been made".  I don't think a 
source id is needed.

But I'm not proposing changes to NSH, just exploring the ramifications 
of some of the existing design decisions.

---

From: Dave Dolson <[email-masked]>
Date: Fri, 11 Nov 2016 21:13:59 +0000

I think there is some layering confusion here.
As I see it, once you put NSH inside another transport (e.g., to deliver it from an SFF to an SF), the intermediate nodes aren't doing NSH anymore.
So it might make sense to draw diagrams of logical connectivity.
Eric, I think in your examples all of the SFFs are "connected" to all of the SFs at the logical level.

-----Original Message-----
From: Eric C Rosen [mailto:[email-masked]] 
Sent: Saturday, November 12, 2016 5:25 AM
To: Jim Guichard (jguichar); Ron Parker; Dave Dolson; [email-masked]; 'Joel M. Halpern'; [email-masked]
Subject: Re: [sfc] Re: decrementing service function path index

Jim> Well unless I am missing something SFF2 in this example will never
see the NSH as SFF1 will tunnel the packet directly to SF1-2 using whatever transport is in play.

That is exactly my point; for this sort of load balancing to work with NSH as it is, one needs to have a transport tunnel from each SFF to each remote SFI that the SFF may select as the next hop in a service path.  
This means that for every type of transport that might be "in play", you have to figure out what to put in the transport header in order to identify a particular SFI.

The fact that the transport tunnels connect SFFs to SFIs rather than connecting SFFs to each other is something that I don't think is apparent from the documents or from some of the recent threads on the mailing list.

(Luckily, draft-mackie-bess-nsh-bgp-control-plane handles this well, because it provides encapsulation information on a per-SFI basis.)

Ron>  Do you think that an SFF could/should discern this by examining
the source IP of the packet it receives, or do you think we need some sort of "source id" in the NSH base header, itself

If we wanted to require that transport tunnels only go between SFFs, I think it would be sufficient just to have a flag in the NSH that means "the load balancing decision has already been made".  I don't think a source id is needed.

But I'm not proposing changes to NSH, just exploring the ramifications of some of the existing design decisions.

---

From: Ron Parker <[email-masked]>
Date: Fri, 11 Nov 2016 21:18:48 +0000

Eric, you raise an excellent point about transport tunnels.     Here is my apparently incomplete or incorrect mental model:

* SFF's have direct or tunneled connectivity to the SFI's that they "front-end"
* SFF's have direct or tunneled connectivity to other SFF's
* SFI's MAY have direct or tunneled connectivity to multiple SFF's
   * in this case, in my mental model, each of those SFF's simultaneously "front-ends" the shared SFIs
* When an SFF needs to forward to an SFI that it doesn't front-end, it progresses the packet to the SFF that does front-end that SFI

I think the biggest discrepancy to reality is apparently the last bullet.   In my model, SFF to SFI associations are explicit (i.e., configured or communicated via c-plane).   I would be obliged if you could enumerate how far off I am so I can consider the implications and consequences.

Thanks.

   Ron

---

From: "Adrian Farrel" <[email-masked]>
Date: Sat, 12 Nov 2016 22:08:07 -0000

Ron, I find myself agreeing with most of your "incomplete or incorrect" model.

But I wonder....

If an SFI is connected (by tunnels or otherwise) to more than one SFF (i.e., the SFI is front-ended by more than one SFF), how does it decide through which connection to return a packet that it has processed? The decision could be as simple as to assume bidirectional tunnels and send each packet back through the same tunnel on which it was received maintaining a little per packet state in the SF, or could be as complex as making a forwarding decision based on some knowledge of the SFP. It could require inspection of the NSH or could be the product of load balancing.

Now, before the howling grows to severe :-)  this suggests to me that connecting an SFI to multiple SFFs might be unwise.

Adrian

> From: Ron Parker [mailto:[email-masked]]
> Sent: 11 November 2016 21:19
> To: Eric C Rosen; Jim Guichard (jguichar); Dave Dolson; [email-masked]; 'Joel
> M. Halpern'; [email-masked]
> Subject: RE: [sfc] Re: decrementing service function path index
> 
> * When an SFF needs to forward to an SFI that it doesn't front-end, it progresses
> the packet to the SFF that does front-end that SFI
> 
> I think the biggest discrepancy to reality is apparently the last bullet.   In my
> model, SFF to SFI associations are explicit (i.e., configured or communicated via c-
> plane).   I would be obliged if you could enumerate how far off I am so I can
> consider the implications and consequences.
> 
> Thanks.
> 
>    Ron
> 
> 
> To: Jim Guichard (jguichar) <[email-masked]>; Ron Parker
> <[email-masked]>; Dave Dolson <[email-masked]>;
> [email-masked]; 'Joel M. Halpern' <[email-masked]>; [email-masked]
> That is exactly my point; for this sort of load balancing to work with NSH as it is,
> one needs to have a transport tunnel from each SFF to each remote SFI that the
> SFF may select as the next hop in a service path.
> This means that for every type of transport that might be "in play", you have to
> figure out what to put in the transport header in order to identify a particular SFI.
> 
> The fact that the transport tunnels connect SFFs to SFIs rather than connecting
> SFFs to each other is something that I don't think is apparent from the documents
> or from some of the recent threads on the mailing list.
> 
> (Luckily, draft-mackie-bess-nsh-bgp-control-plane handles this well, because it
> provides encapsulation information on a per-SFI basis.)
> 
> the source IP of the packet it receives, or do you think we need some sort of
> "source id" in the NSH base header, itself
> 
> If we wanted to require that transport tunnels only go between SFFs, I think it
> would be sufficient just to have a flag in the NSH that means "the load balancing
> decision has already been made".  I don't think a source id is needed.
>

---

From: "Surendra Kumar (smkumar)" <[email-masked]>
Date: Sun, 13 Nov 2016 02:47:25 +0000

Isn't that programmed by the control plane as part of laying out the SFP ?
IOW, for a given SFP, there is one (and only one) anchor SFF for each SFI, to keep the implementation simpler. Otherwise, I tend to agree with the assessment on complexities: asymmetric forwarding and problematic to state-full SFFs, etc.

Surendra. 

-----Original Message-----
From: sfc [mailto:[email-masked]] On Behalf Of Adrian Farrel
Sent: Saturday, November 12, 2016 2:08 PM
To: 'Ron Parker' <[email-masked]>; 'Eric C Rosen' <[email-masked]>; Jim Guichard (jguichar) <[email-masked]>; 'Dave Dolson' <[email-masked]>; 'Joel M. Halpern' <[email-masked]>; [email-masked]
Subject: Re: [sfc] decrementing service function path index

But I wonder....

Adrian

> To: Eric C Rosen; Jim Guichard (jguichar); Dave Dolson; 
> [email-masked]; 'Joel M. Halpern'; [email-masked]
>    * in this case, in my mental model, each of those SFF's 
> simultaneously "front- ends" the shared SFIs
> * When an SFF needs to forward to an SFI that it doesn't front-end, it 
> progresses the packet to the SFF that does front-end that SFI
> 
> Jim> Well unless I am missing something SFF2 in this example will 
> Jim> never
> NSH as it is, one needs to have a transport tunnel from each SFF to 
> each remote SFI that the SFF may select as the next hop in a service path.
> This means that for every type of transport that might be "in play", 
> you have to figure out what to put in the transport header in order to identify a particular SFI.
> 
> apparent from the documents or from some of the recent threads on the mailing list.
> 
> "the load balancing decision has already been made".  I don't think a source id is needed.
>

---

From: "Adrian Farrel" <[email-masked]>
Date: Sun, 13 Nov 2016 02:57:15 -0000

That makes it simpler, but is it simple enough?

If an SFI is front-ended by different SFFs on different SFPs, does the SFI have
to understand the SPI and "route" back to the correct SFF?

I thought that was function we were trying to keep out of the SF.

Adrian

> Sent: 13 November 2016 02:47
> To: [email-masked]; 'Ron Parker'; 'Eric C Rosen'; Jim Guichard
(jguichar);
> 'Dave Dolson'; 'Joel M. Halpern'; [email-masked]
> IOW, for a given SFP, there is one (and only one) anchor SFF for each SFI, to
keep
> the implementation simpler. Otherwise, I tend to agree with the assessment on
> complexities: asymmetric forwarding and problematic to state-full SFFs, etc.
> 
> Surendra.
> 
> To: 'Ron Parker' <[email-masked]>; 'Eric C Rosen'
> <[email-masked]>; Jim Guichard (jguichar) <[email-masked]>; 'Dave
> Dolson' <[email-masked]>; 'Joel M. Halpern' <[email-masked]>;
> [email-masked]
> If an SFI is connected (by tunnels or otherwise) to more than one SFF (i.e.,
the SFI
> is front-ended by more than one SFF), how does it decide through which
> connection to return a packet that it has processed? The decision could be as
> simple as to assume bidirectional tunnels and send each packet back through
the
> same tunnel on which it was received maintaining a little per packet state in
the
> SF, or could be as complex as making a forwarding decision based on some
> knowledge of the SFP. It could require inspection of the NSH or could be the
> product of load balancing.
> 
> Now, before the howling grows to severe :-)  this suggests to me that
connecting
> an SFI to multiple SFFs might be unwise.
> 
> Adrian
> 
> > Eric, you raise an excellent point about transport tunnels.     Here is my
> apparently
> > incomplete or incorrect mental model:
> >
> > model, SFF to SFI associations are explicit (i.e., configured or
communicated via
> c-
> > you have to figure out what to put in the transport header in order to
identify a
> particular SFI.
> > "the load balancing decision has already been made".  I don't think a source
id is
> needed.

---

From: "Surendra Kumar (smkumar)" <[email-masked]>
Date: Sun, 13 Nov 2016 04:11:51 +0000

Ah, I was speaking of CP choosing SFFs for an <SFP, SFI (SI)>.

Obviously, yes, CP and SF touch points are a problem as has been discussed many times. Add this to the list of those problems, in programming the SF.

There are two approaches to forwarding:
1. CF -> SF1 -> SF2 -> SF3: with only SF managed <SPI,SI>
2. CF -> SFFx(SF1) -> SFFy(SF2) -> SFFz (SF3) : with only SFF managed <SPI,SI>

#1 is the "dumb" forwarders, over the top approach - inefficiencies are obvious.
What is mandated is something in between, which is unnecessary and adds absolutely no value.

Surendra.

-----Original Message-----
From: sfc [mailto:[email-masked]] On Behalf Of Adrian Farrel
Sent: Saturday, November 12, 2016 6:57 PM
To: Surendra Kumar (smkumar) <[email-masked]>; 'Ron Parker' <[email-masked]>; 'Eric C Rosen' <[email-masked]>; Jim Guichard (jguichar) <[email-masked]>; 'Dave Dolson' <[email-masked]>; 'Joel M. Halpern' <[email-masked]>; [email-masked]
Subject: Re: [sfc] decrementing service function path index

If an SFI is front-ended by different SFFs on different SFPs, does the SFI have to understand the SPI and "route" back to the correct SFF?

Adrian

(jguichar);
keep
> <[email-masked]>; Jim Guichard (jguichar) <[email-masked]>; 
> 'Dave Dolson' <[email-masked]>; 'Joel M. Halpern' 
> <[email-masked]>; [email-masked]
the SFI
> connection to return a packet that it has processed? The decision 
> could be as simple as to assume bidirectional tunnels and send each 
> packet back through
the
the
> SF, or could be as complex as making a forwarding decision based on 
> some knowledge of the SFP. It could require inspection of the NSH or 
> could be the product of load balancing.
> 
connecting
> > * When an SFF needs to forward to an SFI that it doesn't front-end, 
> > it progresses the packet to the SFF that does front-end that SFI
> >
communicated via
> > That is exactly my point; for this sort of load balancing to work 
> > with NSH as it is, one needs to have a transport tunnel from each 
> > SFF to each remote SFI that the SFF may select as the next hop in a service path.
identify a
> > Ron>  Do you think that an SFF could/should discern this by 
> > Ron> examining
> > the source IP of the packet it receives, or do you think we need 
> > some sort of "source id" in the NSH base header, itself
> >
> > If we wanted to require that transport tunnels only go between SFFs, 
> > I think it would be sufficient just to have a flag in the NSH that 
> > means "the load balancing decision has already been made".  I don't 
> > think a source
id is
> needed.
> >
> > But I'm not proposing changes to NSH, just exploring the 
> > ramifications of some of the existing design decisions.
> >
> 
>

---

From: Eric C Rosen <[email-masked]>
Date: Mon, 14 Nov 2016 14:54:39 -0500

> Eric, I think in your examples all of the SFFs are "connected" to all of the SFs at the logical level.

Suppose SFF1 connects directly to SF1-1 and SF2-1, whereas SFF2 connects 
directly to SF1-2 and SF2-2.  And suppose that an SPI/SI of 17/255 means 
"SF1", while an SPI/SI of 17/254 means "SF2".

I was thinking that:

- each SFI knows the SFF to which it is directly connected;

- there is a unidirectional transport connection from an SFF to those 
SFIs to which it is not locally connected.

Then the processing might be something like the following:

SFF1 sees an NSH packet whose SPI/SI is 17/255.

SFF1 selects SF1-1 and delivers the packet to it.

SF1-1 processes the packet (based on its NSH), decrements the SI, and 
delivers the packet with an SPI/SI of 17/254 to SFF1.

SFF1 processes the packet (based on its NSH), and selects SF2-2. SFF1 
sends the packet to SF2-2 over a unidirectional transport connection.

SF2-2 processes the packet (based on its NSH), decrements the SI, and 
delivers the packet with SPI/253 to SFF2.

SFF2 recognizes (based on the NSH) that the packet is at the end of its 
service path.

Note that when SF2-2 is done with the packet, it does not send it to 
SFF1, it sends it to SFF2.   The "direct" connection and the "remote" 
connection are not treated the same.  The "direct" connection could of 
course be implemented as a transport session. But in this model, SF1-2 
and SF2-2 do not know about SFF1, they only know about SFF2.  So I think 
it would be a bit misleading to say that the direct connection and the 
remote connection are both logical connections.  The remote connection 
is really unidirectional (SFF-->SFI) and the SFI doesn't know who sent 
the packet.  The local connection is bidirectional (SFF<-->SFI).

It might have made more sense to say that SFF1 should send an SPI/254 
packet through a transport connection to SFF2, and that SFF2 should 
dispatch the packet to an SF2 instance without first sending it to 
another SFF.  But this would require one of the following two solutions:

- either a flag in the NSH that means "already seen by an SFF", or

- the SFF would have to know that packets received on certain transport 
connections are from an SFF.

Both of these solutions are simple enough, but both appear to have been 
ruled out by the WG for some reason.

---

From: Eric C Rosen <[email-masked]>
Date: Mon, 14 Nov 2016 15:03:43 -0500

On 11/11/2016 4:18 PM, Ron Parker wrote:
> Here is my apparently incomplete or incorrect mental model:
>

I think this is a very sensible model.  It requires that an SFI know 
which SFF front-ends it, and it requires that that an SFF can tell 
whether a packet it is processing just came from another SFF or from an 
SFI.  But what we are being told is that the WG has already rejected 
that model ;-(

On 11/12/2016 5:08 PM, Adrian Farrel wrote:
> If an SFI is connected (by tunnels or otherwise) to more than one SFF (i.e., the SFI is front-ended by more than one SFF), how does it decide through which connection to return a packet that it has processed? The decision could be as simple as to assume bidirectional tunnels and send each packet back through the same tunnel on which it was received maintaining a little per packet state in the SF, or could be as complex as making a forwarding decision based on some knowledge of the SFP.

Or the SFI could know which SFF is it's "front end", and that SFF may or 
may not be the one from which the packet was received.

---

From: "Paul Quinn (paulq)" <[email-masked]>
Date: Thu, 17 Nov 2016 13:47:59 +0000

> On Nov 14, 2016, at 2:54 PM, Eric C Rosen <[email-masked]> wrote:
> 
> Suppose SFF1 connects directly to SF1-1 and SF2-1, whereas SFF2 connects directly to SF1-2 and SF2-2.  And suppose that an SPI/SI of 17/255 means "SF1", while an SPI/SI of 17/254 means "SF2".
> 
> I was thinking that:
> 

SFIs don't need to know that: they simply return the packet from whence it came.

> - there is a unidirectional transport connection from an SFF to those SFIs to which it is not locally connected.
> 
> SF1-1 processes the packet (based on its NSH), decrements the SI, and delivers the packet with an SPI/SI of 17/254 to SFF1.
> 
> SFF1 processes the packet (based on its NSH), and selects SF2-2. SFF1 sends the packet to SF2-2 over a unidirectional transport connection.
> 
> SF2-2 processes the packet (based on its NSH), decrements the SI, and delivers the packet with SPI/253 to SFF2.

You appear to be thinking of SFF as a PE-like device.  That is decidedly not what SFFs are.  They are simply logical front-ends for service functions.  As I mentioned above, the SFI, SF2-2 in this case, simply returns the packet to SFF1.  Nothing more complex than that.

If you need multiple instances of SFF1 (let's say for redundancy), then it's an implementation decision how to ensure SPI/SI tables are consistent between them.

> 
> SFF2 recognizes (based on the NSH) that the packet is at the end of its service path.
> 
> Note that when SF2-2 is done with the packet, it does not send it to SFF1, it sends it to SFF2.   The "direct" connection and the "remote" connection are not treated the same.  The "direct" connection could of course be implemented as a transport session. But in this model, SF1-2 and SF2-2 do not know about SFF1, they only know about SFF2.  So I think it would be a bit misleading to say that the direct connection and the remote connection are both logical connections.  The remote connection is really unidirectional (SFF-->SFI) and the SFI doesn't know who sent the packet.  The local connection is bidirectional (SFF<-->SFI).
> 

"remote" and "direct" are distinctions without a difference.  SFF1 is "responsible" for the services that are connected to it, that's it. 

> It might have made more sense to say that SFF1 should send an SPI/254 packet through a transport connection to SFF2, and that SFF2 should dispatch the packet to an SF2 instance without first sending it to another SFF.  But this would require one of the following two solutions:
> 
> - the SFF would have to know that packets received on certain transport connections are from an SFF.
> 
> Both of these solutions are simple enough, but both appear to have been ruled out by the WG for some reason.
> 
> 

Because, frankly, they aren't needed.  The current (implemented) model works, is simple to understand and provides the requisite behavior for chaining.