Subject: Re: [forces] NH review WAS(Re: BasicMetadataDispatch WAS(Re: LFB LPM issue

From: "Chuanhuang Li" <[email-masked]>
Date: Fri, 22 Apr 2011 12:55:22 +0800

Hi, Jamal,
    Thanks! I have some comments. 
> 1)I dont think NexthopID (componentID="1") should be part of the table.
> The passed NexthopID is an _index_ into the NextHopTable and does not
> need to be part of the table.
Chuanhuang: whether need we differentiate between the truly table index and 
the "index" which reflect the pointing relationship between two table? This maybe  
a common question. The so-called "truly table index" is just the internal order 
for building the table in an LFB.  For example, there are two implementing methods for  
NH table in two different FE, they can place the same entry in different place(different 
truly table index), but the NexthopID is the same.  This kind of definition can reduce some 
"hard" coordination between LFBs.

> 2)Renaming OutputLogicalPortID to OutputLogicalPortIndex for readability.
Chuanhuang: From previous discussion, the "OutputLogicalPortID" is exactly an L3 port ID,
It isn't an index. It have physic meaning. This should be continue to have in the NH table.  
Maybe, do you want to add an element named "NHEncapInfoIndex" to the definition of NH table?

> 3)Renaming EncapOutputIndex to LFBOutputSelectIndex to indicate
> which LFB port we are selecting to go to the downstream LFB instance.
Chuanhuang: I agree.

>              <component componentID="?">
>                 <name>LFBOutputSelectIndex</name>
>                 <synopsis>LFB Group output port index to select
>                           downstream LFB port. Some possibilities
>                           of downstream LFB instances are:
> 			  a) EtherEncapsulator
> 			  b) A metadata Dispatcher
> 			  Typically, there will be only one LFB port
>                 </synopsis>
>                 <typeRef>uint32</typeRef>
>              </component>
Chuanhuang:  There are not only two cases you have showed. The downstream 
LFB instance also can be redirectOut or other media encapsulator. 

>  a)MaxOutputPorts - what is the semantics of that given that we
>  only have one "SuccessOut" LFB port?
Chuanhuang: Capability "MaxOutputPorts" has been removed, please see the email:
            subject: port group
            time: 2010-12-22

Yours,
Chuanhuang

---

From: "Joel M. Halpern" <[email-masked]>
Date: Fri, 22 Apr 2011 01:03:53 -0400

Whether the FE gets to pi9ck the entry index for entries in a table 
depends upon how the CE creates the entries, and whether there is 
provision for the FE to autonomously create entries.
For most tables, there is no autonomous creation, and the CE will select 
the index to use.  Using the explicit index, rather than using a content 
value as a content-index, will generally be significantly more efficient.
The tables we are discussing are ones where the CE will be creating entries.

Also, note, these indices are used between LFBs in the same FE.  We are 
not passing metadata between the FEs.  (although it would still work then.)

Yes, we need the L3 Logical Port ID metadata, and we need the 
mediaencaptable index.  these are separate pieces of metadata.

Yours,
Joel

---

From: "Chuanhuang Li" <[email-masked]>
Date: Fri, 22 Apr 2011 14:38:35 +0800

Hi,Joel

Yes, it will be more efficient if using the explicit index between LFBs in the same 
FE. But, CE need manage the unified routing tables for all FEs. If CE create an entry,
all FEs with differnet implementing ways need to create the same table index. This 
hard coupling may constrain user's implementation. From CE point, loose coupling 
between LFBs may be helpful to his manager. From FE point, FE can implement the 
table more flexible.

---

From: "Joel M. Halpern" <[email-masked]>
Date: Fri, 22 Apr 2011 11:01:09 -0400

I do not see any need for all the FEs in the NE to use the same index.
The index is stored in one LFB of an FE, and used in another LFB of that 
same FE.
Since the view of things and the capabilities from each FE is different, 
an implementation of a CE which attempted to use the same data 
representation to represent all the FEs is simply not going to work. 
The CE needs to keep a per-FE representation.  No, this is not specified 
by the protocol.  It is not a protocol issue.  However, it does follow 
from looking at what the CE needs to do.
The simplest example is that given multiple FEs, for any given prefix 
some FEs will need to have next-hops pointing to other FEs, while some 
FEs will have external next-hops for that prefix.
Even the prefix granularity may be different on different FEs.

Yours,
Joel

---

From: Jamal Hadi Salim <[email-masked]>
Date: Sat, 23 Apr 2011 11:46:01 -0400

Chuanhuang,

On Fri, Apr 22, 2011 at 12:55 AM, Chuanhuang Li
<[email-masked]> wrote:
[..]
> "hard" coordination between LFBs.

As Joel has argued - there is only one index, so i dont see any change needed
from what i said.

Having said that since the issue of "NextHop FE" or "NextHop Module ID"
has come up many times before.  I have been guilty of bringing up that
discussion many times (I dont do it intentionaly, it just shows up in my brain
cells given my past experience).
People implement with NH tables pointing to either other chips or linecards
in addition to the NH ID. My opinion is we need to have a discussion in the
draft somewhere about it. The model should be able to handle
it without assuming inter-FE LFBs.
Joel, what do you think?

> Maybe, do you want to add an element named "NHEncapInfoIndex" to the definition of

Ok. When i update, I will have both MediaIndex or EncapIndex (please pick one
name) as well as L3PortID.

> LFB instance also can be redirectOut or other media encapsulator.

I will add those two as well as "etc" to mean there could be others.

> Chuanhuang: Capability "MaxOutputPorts" has been removed, please see the email:

I am still confused because my question is still not answered:
We only have "successout" or "exception". What is the point of
LFBOutputSelectIndex then? The CE knows what is to be sent on "successout".
There is no need to set it anywhere.
Who decides what the values of LFBOutputSelectIndex? Are we going to
standardize?

cheers,
jamal

---

From: "Joel M. Halpern" <[email-masked]>
Date: Sat, 23 Apr 2011 12:02:06 -0400

Sorry, I am missing something.

Is the question the existence of intermediate LFBs between the NH LFB 
and the media encapsulator?  The metadata carries the information across 
that.

Is the concern that there may be other, next-hop-specific, LFBs that 
need to know which next-hop is intended?  It seems to me, since the CE 
creates all the entries, that the CE can, in hte normal case, just 
create entries at the correct index.
If necessary, an LFB could have an auxiliary table, indexed by the 
logical output port and the next hop encapsulation index, which provides 
the local processing index.  Or the unusual LFB could use the next hop 
encapsulation index as a content index, stroing the value in the base 
table.  Lots of ways for it tow work, depending upon what problem the 
LFB is dealing with.

I do not see why any change from what was suggested at the meeting is 
needed:  Next Hop LFB produces an output logical port ID and a next hop 
ecnapsulation index.  It also produces, where known, the next hop IP 
address, in case someone needs that.

Yours,
Joel

---

From: Jamal Hadi Salim <[email-masked]>
Date: Sat, 23 Apr 2011 12:37:20 -0400

Joel,

My main concern is that this discussion will come up again (100.5%). Or there
may be confusion from someone implementing as to the semantics.
It is useful for the reader and pragmatic for the authors to provide clarity.

For example, we have clearly stated that the burden is on the FE to do
handle the massaging that maps its LPM/NH tables
(given some implementations have the LPM encapsulating the NH info vs
others with separate NH LFB/table). As far as the CE is concerned, they are
separate.
My thinking is that the idea of a NH implementation with a reference to a
moduleID applies  similarly. i.e we may need to pick a side, but we need
to state and reason it clearly.

cheers,
jamal

On Sat, Apr 23, 2011 at 12:02 PM, Joel M. Halpern <[email-masked]> wrote:
> Is the question the existence of intermediate LFBs between the NH LFB and
> the media encapsulator?  The metadata carries the information across that.
>
> Is the concern that there may be other, next-hop-specific, LFBs that need to
> know which next-hop is intended?  It seems to me, since the CE creates all
> the entries, that the CE can, in hte normal case, just create entries at the
> correct index.
> If necessary, an LFB could have an auxiliary table, indexed by the logical
> output port and the next hop encapsulation index, which provides the local
> processing index.  Or the unusual LFB could use the next hop encapsulation
> index as a content index, stroing the value in the base table.  Lots of ways
> for it tow work, depending upon what problem the LFB is dealing with.
>

---

From: "Joel M. Halpern" <[email-masked]>
Date: Sat, 23 Apr 2011 13:33:56 -0400

If what you are asking for is better explanation, I would agree with 
that.  The amount of discussion we have needed makes it clear we need 
good explanatory text.  Otherwise, not only will the discussion come up 
again, but folks will misunderstand what we have put together.

I thought you were asking for a different structuring of the LFBs, 
tables, and metadata.

Yours,
Joel

On 4/23/2011 12:37 PM, Jamal Hadi Salim wrote:
> On Sat, Apr 23, 2011 at 12:02 PM, Joel M. Halpern<[email-masked]>  wrote:

---

From: Jamal Hadi Salim <[email-masked]>
Date: Sat, 23 Apr 2011 14:22:06 -0400

Indeed I am asking for better explanation - and maybe even an example
i.e I am happy for it not to be part of the model.

cheers,
jamal

On Sat, Apr 23, 2011 at 1:33 PM, Joel M. Halpern <[email-masked]> wrote:
> If what you are asking for is better explanation, I would agree with that.
>  The amount of discussion we have needed makes it clear we need good
> explanatory text.  Otherwise, not only will the discussion come up again,
> but folks will misunderstand what we have put together.
>
> I thought you were asking for a different structuring of the LFBs, tables,
> and metadata.
>
> Yours,
> Joel
>
>>> logical

---

From: "Chuanhuang Li" <[email-masked]>
Date: Mon, 25 Apr 2011 09:20:27 +0800

>Even the prefix granularity may be different on different FEs.

OK, I accept this point. I have no problem about this.

Yours,
Chuanhuang

---

From: "Chuanhuang Li" <[email-masked]>
Date: Mon, 25 Apr 2011 09:21:59 +0800

Hi,Jamal 

>Who decides what the values of LFBOutputSelectIndex? Are we going to

"successout" is an output port group. "LFBOutputSelectIndex" is used to select an output 
from this group.

Yours,
Chuanhuang

---

From: Jamal Hadi Salim <[email-masked]>
Date: Mon, 25 Apr 2011 08:17:26 -0400

Chuanhuang,

> "successout" is an output port group. "LFBOutputSelectIndex" is used to select an output

Sorry I missed it - it was right there in the text ;->
I will fix the text and re-post again.

my brain is slow:  how does the CE knows what LFBOutputSelectIndex 32
bit value is?
I know that the LFB Topology will eventually have an LFB selector
indication for the
port - but how do we go there to the value in LFBOutputSelectIndex of
this table?

cheers,
jamal

---

From: "Joel M. Halpern" <[email-masked]>
Date: Mon, 25 Apr 2011 10:29:19 -0400

The CE normally connected the various elements of the group output port 
to the successor LFBs.  Therefore it know what it connected at which index.
In unusual cases, those connections are fixed, or established by an 
earlier CE.  In that case, the CE reads the same structure it would 
write to set them up, and learns what the connections are.

Yours,
Joel

---

From: Jamal Hadi Salim <[email-masked]>
Date: Mon, 25 Apr 2011 10:35:50 -0400

So the LFBOutputSelectIndex is the table index for table as defined in:
FEObject/LFBTopology ?
It may need some mention in the text or xml synopsis.

cheers,
jamal

On Mon, Apr 25, 2011 at 10:29 AM, Joel M. Halpern <[email-masked]> wrote:
> The CE normally connected the various elements of the group output port to
> the successor LFBs.  Therefore it know what it connected at which index.
> In unusual cases, those connections are fixed, or established by an earlier
> CE.  In that case, the CE reads the same structure it would write to set
> them up, and learns what the connections are.
>
> Yours,
> Joel
>

---

From: Joel <[email-masked]>
Date: Mon, 25 Apr 2011 10:47:12 -0400

Yes.  And this is the way we work with all output groups.   So it seems 
strange to make a big point of saying so in this document.
Yours,
Joel

---

From: Jamal Hadi Salim <[email-masked]>
Date: Mon, 25 Apr 2011 10:55:53 -0400

> strange to make a big point of saying so in this document.

Well, I couldnt remember - thats why ;->
The only additional text i added was the "Note" at the end of the
synopsis:

===
             <component componentID="?">
                <name>LFBOutputSelectIndex</name>
                <synopsis>LFB Group output port index to select
                          downstream LFB port. Some possibilities
                          of downstream LFB instances are:
                            a) EtherEncapsulator
                            b) Other type of media LFB
                            c) A metadata Dispatcher
                            d) A redirect LFB
                            e) etc
                         Typically, there will be only one LFB port where
                         one of the above is connected
                         Note: LFBOutputSelectIndex is the table index
                         for table FEObject/LFBTopology for this port
                </synopsis>
                <typeRef>uint32</typeRef>
             </component>

====

cheers,
jamal

---

From: "Chuanhuang Li" <[email-masked]>
Date: Mon, 25 Apr 2011 23:47:44 +0800

Hi, Jamal
>                <typeRef>uint32</typeRef>

1)Typically, there may be more LFBs connected to the group output ports at the same time. For 
example, RedirectOut LFB and EtherEncapsulator(or metadata Dispatcher) will connect after NH, 
if the media is Ethernet.

2) I don't think "LFBOutputSelectIndex"  is the table index for LFBTopology table. 
LFBOutputSelectIndex is just the NH group output index.
 For the static LFB topology, as Joel said, CE can knows what indexes NH are using to connect with 
some LFBs. Please see the LFBLinkType definition in FEObject LFB:
             <dataTypeDef> 
               <name>LFBLinkType</name> 
               <synopsis> 
                 Link between two LFB instances of topology 
               </synopsis> 
               <struct> 
                 <element elementID="1"> 
                   <name>FromLFBID</name> 
                   <synopsis>LFB src</synopsis> 
                   <typeRef>LFBSelectorType</typeRef> 
                 </element> 
                 <element elementID="2"> 
                   <name>FromPortGroup</name> 
                   <synopsis>src port group</synopsis> 
                   <typeRef>string</typeRef> 
                 </element> 
                 <element elementID="3"> 
                   <name>FromPortIndex</name> 
                   <synopsis>src port index</synopsis> 
                   <typeRef>uint32</typeRef> 
                 </element> 
                 ...........   
 
     If FE supports dynamic topology, CE will reconfig the link relationship by itself. 
That is, what index will be set is charged by CE . (CE knows all the possible indexes which was  
limited in the "SupportedLFBs" capability in FEObject LFB.) 

Yours,
Chuanhuang

---

From: Jamal Hadi Salim <[email-masked]>
Date: Mon, 25 Apr 2011 12:13:38 -0400

Hi Chuanhuang,

> if the media is Ethernet.

I can remove the sentence that starts with "Typically..."

>  For the static LFB topology, as Joel said, CE can knows what indexes NH are
> using to connect with some LFBs. Please see the LFBLinkType definition in
> FEObject LFB:

I think you are right - still we need to make that clear.
So i am assuming this is the FromPortIndex 32-bit value that is of
interest here.

cheers,
jamal

---

From: Jamal Hadi Salim <[email-masked]>
Date: Mon, 25 Apr 2011 12:19:11 -0400

And here are my notes so far for NH. Please take a look at the XXX
parts as well as the other parts carefully.

[I have a few other things i would like to move on once i can hand this
and the Etherencap over to you.]

cheers,
jamal

---

From: "Joel M. Halpern" <[email-masked]>
Date: Mon, 25 Apr 2011 12:34:02 -0400

Commenting on your general items:

1) I agree, using downstream and upstream relative to the flow of 
packets makes sense.

2) <outputPort group="true"> is the way, as defined by the model, on e 
declares that a given output port from instances of a given LFB class is 
a group output port.

Yours,
Joel

---

From: Jamal Hadi Salim <[email-masked]>
Date: Mon, 25 Apr 2011 12:46:46 -0400

On Mon, Apr 25, 2011 at 12:34 PM, Joel M. Halpern <[email-masked]> wrote:
> 1) I agree, using downstream and upstream relative to the flow of packets
> makes sense.
>
> declares that a given output port from instances of a given LFB class is a
> group output port.

Indeed. RFC 5812 is very clear - i was being lazy.

cheers,
jamal