Subject: Re: Link Bundling

From: Debanjan Saha <[email-masked]>
Date: Wed, 05 Jul 2000 09:18:28 -0500

Yakov,

Yakov Rekhter wrote:
 > Kireeti and myself would like to ask the MPLS WG to
> accept draft-kompella-mpls-bundle-01.txt as an
> MPLS WG document.

I requested some clarification (see attached) on the draft. 
Have n't heard anything from the authors to date. Any chance
that they will be addressed?

Regards,
Debanjan

-------------------------------------------------------------

Quoted from section 3.1 of the draft..

"Component links may either be unnumbered, or all components links
   must be numbered identically.  In the former case, the bundled link
   may be either unnumbered or numbered with IP addresses assigned to
   some "virtual" interfaces on an LSR (it is assumed that an LSR may
   have multiple virtual interfaces).  In the latter case, the bundled
   link is numbered the same as the component links."

In the first sentence, you say "all component links must be numbered 
identically". What does "identically" really mean?

In the last sentence, you say "the bundled link is numbered the same as 
the component links". Does it mean that the bundle can be numbered with
the IP address of any one of the component links?

Also, have you considered taking SRLG information into account for
bundling?

What about including resource information of the kind?
        4 OC-48 
        2 OC-192 etc...

Current mode of bandwidth accounting does not provide sufficient 
information on bandwidth granularity which is very important for
optical networks.

Debanjan
        
-- 
Debanjan Saha                         Phone: 732-923-4264
Senior Network Architect              Fax:   732-923-9804
Tellium Optical Systems               http://www.tellium.com

---

From: Yakov Rekhter <[email-masked]>
Date: Wed, 05 Jul 2000 05:57:32 -0700

Folks,

Yakov.

---

From: George Swallow <[email-masked]>
Date: Wed, 05 Jul 2000 17:42:08 -0400

> accept draft-kompella-mpls-bundle-01.txt as an

Consensus on this will be assessed on 7/12.  Opinions should be aired
by then.

...George

==================================================================
George Swallow       Cisco Systems                   (978) 244-8143
                     250 Apollo Drive
                     Chelmsford, Ma 01824

---

From: Bora Akyol <[email-masked]>
Date: Fri, 07 Jul 2000 11:38:51 -0700

George, Yakov, et al.,

I do not think that this internet draft should be adopted as an MPLS
working group document. My primary objection to this document is the fact
that bundling of L1/L2 interfaces has nothing specific to do with MPLS.
In fact, this is an L2 abstraction that should be completely transparent
to MPLS, IP or ISIS. I feel that this particular draft is very focused on
MPLS-specific concerns and does not provide a good abstraction of the L2
bundling aspects and also places significant restrictions on what can be
done with a bundled link and what kind of members it can have.

I personally know of at least one implementation (ours) of bundled links
that forms such an abstraction to all upper layer protocols. In this
implementation, a heterogeneous group of L1/L2 interfaces are bonded
together at L2 NCP layer such that IP NCP, MPLS NCP, ISIS NCP treats the
bundled (bonded) interface as just another type of interface. When MPLS
RSVP messages are passed over the bundled interface, the labels that are
assigned are programmed into all members of the bundled links even though
in reality a hash is used so that only one of the members are used (to
prevent reordering). Bandwidth management is handled by means of the
apriori knowledge of the hash. Also note that this particular
implementation is not MLPPP since no additional headers are added to the
packets.
For MPLS specific concerns, the methods that are used in this approach
can be expanded. For IP, this particular approach works very well.

If there is interest from the MPLS WG, we (myself and another colleague)
would be perfectly willing to author an Internet Draft on this particular
bond implementation. The question that I have is what is the proper forum
to submit a generalized layer 2 bundling draft in the IETF. It is
certainly relevant to MPLS, but it is just as relevant to IP, ISIS, etc.

Regards

Bora Akyol

---

From: Lou Berger <[email-masked]>
Date: Mon, 10 Jul 2000 17:26:20 -0400

Bora,

There is a difference specific to MPLS, i.e., both sides of a link need to 
agree on the label.  You only briefly mentioned this key point in your 
mail.  I believe the choices open to synchronize labels over bundled/bonded 
links are:
   1. Allow the receipt of each label over any/all links in the bundle.
      This is essentially the equivalent of what's done for IP.
      This approach just doesn't work for non-packet links (MPLamdaS).

   2. The link is selected algorithmically (per your message)
      Some algorithm would need to be standardized.  An algorithmic
      approach is likely to either be artificially limiting (by not
      taking into account allocations or loading) or so complex that
      it's prone to error.

   3. Have the underlying layer signals the link.
      This can be done using some or some lower level signaling protocol
      (such as LMP.)

   4. Have MPLS signaling just provide the link information.
      This is what's documented in the draft.  It does mean that signaling
      needs to convey link information, but it allows greater flexibility
      than an algorithmic approach and doesn't require the use of yet
      another signaling protocol to setup each LSP.

      Note: In this case, the underlying layer can still pick the link,
      it's just transported by signaling, much like RSVP opaquely
      carries int-serv information.  There's no problem if you want to
      use your hash, or if someone else wants to use more dynamic
      information.

      Also, the draft doesn't require that routing and IP understand the
      component links (they can operate on aggregated information).
      We can certainly alter the text to make all this more explicit.

It seems to me that much of your objection is really based on that link 
selection is dynamic rather than algorithmic and that the draft isn't 
explicit about routing, IP and even signaling not needing to understand the 
specifics of the component links.  Is this correct?

Lou

---

From: Greg Waters <[email-masked]>
Date: Mon, 10 Jul 2000 18:20:45 -0400

Lou, you're getting warm.  You suggest that option (1) is not good for 
MPLamdaS, perhaps because you assume that a given LSP's packets appear on a 
single link during bundled operation?  I read you as thinking that a label 
goes to waste on all but one link in the bundle, in option (1).

Bora's model, which I find valuable, is to allow one or more suitable 
IPv4-bearing and IPv6-bearing LSPs to appear simultaneously, load-balanced, 
on all links within a bundle.  The traffic volume within each LSP is 
irrelevant to the load balancing, unlike Kireeti's model.  As with some 
link aggregation methods, the load-balancing uses IPv4/v6 address hashing, 
or another technique that maintains order within each microflow.  Given a 
rich hash in relation to the bundle size, it does not matter what hashing 
algorithm each hop uses.

The benefit: A "best-effort" or "premium" Internet LSP can be configured 
with a capacity ten times larger than one's fastest physical link.  I won't 
suggest that this major benefit extends to "CBR LSPs" of a kind which don't 
tolerate burstiness and which defy good load-balancing.

Recent demos showed that Bora's model works over optical switches in which 
all traffic on the fibers in one bundle are switched to as many fibers in 
another bundle.  I suppose that falls short of the MPLamdaS requirements, 
of which I confess ignorance.  --Greg W., Avici Systems

At 05:26 PM 7/10/00 -0400, Lou Berger wrote:
>mail.  I believe the choices open to synchronize labels over 
>bundled/bonded links are:
>explicit about routing, IP and even signaling not needing to understand 
>the specifics of the component links.  Is this correct?
>Lou
>
>> > ...George Swallow

---

From: Lou Berger <[email-masked]>
Date: Mon, 10 Jul 2000 19:48:01 -0400

Greg,
         You are correct, spreading a single LSP over multiple component 
links is not supported in the bundle draft.  I agree this could have some 
value for best-effort LSPs.  (The draft was concerned with TE LSPs.)  We 
could support such "spreading" by adding a special link value or bit to 
indicated that the label should be made available on all PSC (packet) links 
of a bundle/bond.  (The spreading algorithm need not be standardized for 
this case.)

Lou

At 06:20 PM 7/10/00 -0400, Greg Waters wrote:
>MPLamdaS, perhaps because you assume that a given LSP's packets appear on 
>a single link during bundled operation?  I read you as thinking that a 
>label goes to waste on all but one link in the bundle, in option (1).
>
>IPv4-bearing and IPv6-bearing LSPs to appear simultaneously, 
>load-balanced, on all links within a bundle.  The traffic volume within 
>each LSP is irrelevant to the load balancing, unlike Kireeti's model.  As 
>with some link aggregation methods, the load-balancing uses IPv4/v6 
>address hashing, or another technique that maintains order within each 
>microflow.  Given a rich hash in relation to the bundle size, it does not 
>matter what hashing algorithm each hop uses.
>
>with a capacity ten times larger than one's fastest physical link.  I 
>won't suggest that this major benefit extends to "CBR LSPs" of a kind 
>which don't tolerate burstiness and which defy good load-balancing.
>
>>There is a difference specific to MPLS, i.e., both sides of a link need 
>>to agree on the label.  You only briefly mentioned this key point in your

---

From: Bora Akyol <[email-masked]>
Date: Mon, 10 Jul 2000 17:37:58 -0700

See my comments within:

>       This approach just doesn't work for non-packet links (MPLamdaS).

OK.

>       taking into account allocations or loading) or so complex that

I think this is a presumption. OSPF uses a common algorithm and we have seen
interoperable, distributed routing computation happen. Also, what I gave was an
example, it can easily be extended to support loading of individual link
members.

>       This can be done using some or some lower level signaling protocol

Sure.

> specifics of the component links.  Is this correct?

No

My objection is based on the fact that this draft does not address the
abstraction of physical links into a logical entity that can be used by L3 and
above.

---

From: Lou Berger <[email-masked]>
Date: Mon, 10 Jul 2000 21:00:52 -0400

Bora,
         see comments in-line below.

At 05:37 PM 7/10/00 -0700, Bora Akyol wrote:

>[...]

Agreed, it is a presumption.  But having two algorithms running in parallel 
that are each sampling traffic levels to select the link used by an LSP 
just scares me.  Standardizing such an algorithm doesn't make a whole lot 
of sense to me either.  This isn't the same as h-b-h routing, which mostly 
works when information is out of sync.

>[...]

You're right.  The draft doesn't do this, but nor should it.  I think it's 
neutral on the existence of such an abstraction.  It does define how to use 
such an abstraction (which could be implementation dependent) and provides 
a vehicle to carry information (link id) that can be used by such an 
abstraction when setting up an LSP.  It also works fine if no abstraction 
is present.

Lou

>[...]

---

From: Greg Waters <[email-masked]>
Date: Tue, 11 Jul 2000 08:35:47 -0400

At 07:48 PM 7/10/00 -0400, Lou Berger wrote:
> indicated that the label should be made available on all PSC (packet) 
> links of a bundle/bond.  (The spreading algorithm need not be 
> standardized for this case.)
>Lou

I was talking about TE LSPs--in particular, TE LSPs whose capacity is 
greater than the fastest physical link.  Having that kind of LSP greatly 
simplifies the scaling and resiliency of TE.  You can widen the bundle at 
any hop without having to thread new LSPs to use it.  You can, for example, 
increase the capacity of an existing LSP without regard to the elements of 
bundles that it traverses.  It seems possible to simplify not only the IGP 
topology, but also to collapse the strands of TE LSPs, with Bora's bundling 
model.
   --Greg W., Avici Systems

>>Recent demos showed that Bora's model works over optical switches in 
>>which all traffic on the fibers in one bundle are switched to as many 
>>fibers in another bundle.  I suppose that falls short of the MPLamdaS 
>>requirements, of which I confess ignorance.  --Greg W., Avici Systems
>>
>>>to agree on the label.  You only briefly mentioned this key point in 
>>>your mail.  I believe the choices open to synchronize labels over

---

From: Lou Berger <[email-masked]>
Date: Tue, 11 Jul 2000 14:22:21 -0400

At 08:35 AM 7/11/00 -0400, Greg Waters wrote:
>> links is not supported in the bundle draft.  I agree this could have 
>> some value for best-effort LSPs.  (The draft was concerned with TE 
>> LSPs.)  We could support such "spreading" by adding a special link value 
>> or bit to indicated that the label should be made available on all PSC 
>> (packet) links of a bundle/bond.  (The spreading algorithm need not be 
>any hop without having to thread new LSPs to use it.  You can, for 
>example, increase the capacity of an existing LSP without regard to the 
>elements of bundles that it traverses.

I may be wrong (since I of course can't speak for anyone else) but, I don't 
think any of us would argue that allowing LSPs to span multiple links is 
worth investigating.  That said, I do feel that we should get the simple, 1 
link/LSP, solution out there without having it wait for the harder to 
standardize, n-links/LSP, solution.

>It seems possible to simplify not only the IGP topology, but also to 
>collapse the strands of TE LSPs, with Bora's bundling model.

This statement confuses me.  I believe the necessary topology information 
is the same in either approach.

Lou

---

From: Yakov Rekhter <[email-masked]>
Date: Tue, 11 Jul 2000 11:29:01 -0700

Lou,

> standardize, n-links/LSP, solution.

For some types of links (e.g., links between OXCs) n-links/LSP solution
is just not feasible (due to the nature of the links). And we certainly
need to have a solution for such links.

Yakov.

---

From: Alex Zinin <[email-masked]>
Date: Tue, 11 Jul 2000 11:50:13 -0700

> need to have a solution for such links.

As well as the whole thing about LSPs capacity loses its meaning
in the optical context...

The idea of spanning an LSP among multiple links is elegant, I think.
However, the last thing we need to achieve is packet reordering
and this means we need a flow-aware mechanism to be implemented
on top of MPLS switching engine, which is not a problem per se,
but will indeed require some more work.

Alex.