Subject: [lisp] #63: On the use of anycast addresses for RLOCs (from Y. Rekhter's review)

From: "lisp issue tracker" <[email-masked]>
Date: Wed, 21 Apr 2010 09:19:38 -0000

#63: On the use of anycast addresses for RLOCs (from Y. Rekhter's review)
------------------------------------------+---------------------------------
Reporter:  luigi@…                        |       Owner:     
    Type:  technical                      |      Status:  new
Priority:  major                          |   Component:  alt
Severity:  -                              |    Keywords:     
------------------------------------------+---------------------------------
 Section 8.2

  Note that more than one CE router at a site can be configured with
  the same IP address. In this case an RLOC is an anycast address. This
  allows resilience between the CE routers. That is, if a CE router
  fails, traffic is automatically routed to the other routers using the
  same anycast address. However, this comes with the disadvantage where
  the site cannot control the entrance point when the anycast route is
  advertised out from all border routers.

 The above means that at the minimum the price of resilience between
 the CE routers is the inability to support traffic engineering?
 Specifically, the ISP loses any influence over the choice of which ETR
 should be used to reach the multi-homed enterprise.

 Moreover, if CEs are xTRs and use anycast addresses, then RLOCs are
 anycast addresses as well, and thus can not be topologically
 significant. Thus if an enterprise is multi-homed to two (or more)
 ISPs, then use of anycast addresses for CEs would require to route
 such addresses as /32 throughout the whole Internet.

 Also, if anycast addresses are used as RLOCs, then how would one
 deal with a situation where initially both ETR1 and ETR2 advertise
 10.1.1/24 and 10.1.2/24, ITR1 routes traffic for 10.1.1.1 to ETR1, but
 then ETR1, while still being up, loses connectivity to 10.1.1/24?

-- 
Ticket URL: <http://trac.tools.ietf.org/wg/lisp/trac/ticket/63>
lisp <http://tools.ietf.org/lisp/>
LISP WG document issues

---

From: "lisp issue tracker" <[email-masked]>
Date: Wed, 21 Apr 2010 09:20:21 -0000

Priority:  major                          |   Component:  draft-ietf-lisp
Changes (by luigi@…):

  * component:  alt => draft-ietf-lisp

-- 
Ticket URL: <http://zinfandel.levkowetz.com/wg/lisp/trac/ticket/63#comment:1>
lisp <http://tools.ietf.org/lisp/>
LISP WG document issues

---

From: "lisp issue tracker" <[email-masked]>
Date: Thu, 22 Apr 2010 12:36:19 -0000

Description changed by luigi@…:

> then ETR1, while still being up, loses connectivity to 10.1.1/24?

New description:

 '''This is Comment 38'''

 Section 8.2

--

-- 
Ticket URL: <http://trac.tools.ietf.org/wg/lisp/trac/ticket/63#comment:2>
lisp <http://tools.ietf.org/lisp/>
LISP WG document issues

---

From: Eliot Lear <[email-masked]>
Date: Thu, 22 Apr 2010 15:06:00 +0200

>   should be used to reach the multi-homed enterprise.

First of all, this is one of several means to provide resiliency, and so 
to say this is a minimum price is overstating it.  It's one of the tools 
in the toolbox.

SPs have great say over their traffic because they continue to control 
any and all BGP announcements they make, and they continue to control 
their own IGP, as well as whatever load splitting they wish to do based 
on a destination address (LISP or otherwise).

>   such addresses as /32 throughout the whole Internet.

Wow.  No.  Anycast has a scope based on who routes it.  If it's routed 
within an ISP it has a scope within an ISP.  If it is only routed within 
the enterprise it has an even narrower scope.  It could be routed 
between two ISPs.  Mix and match.

>   then ETR1, while still being up, loses connectivity to 10.1.1/24?

We call this a black hole.  That's not a LISP issue, by the way.  The 
same issue has to be addressed with L3 routing.  A smart box might 
notice that it has lost internal connectivity, and act accordingly.

Eliot

---

From: Yakov Rekhter <[email-masked]>
Date: Thu, 22 Apr 2010 12:35:49 -0700

Eliot,

> on a destination address (LISP or otherwise).
 
Sure, but these are the mechanisms provided by the current routing
system, not by LISP.

To reflect this in the spec please add the following to section 8.2:
 
   Use of anycast addresses for resiliency in LISP results in
   inability to use LISP for traffic engineering, and specifically
   in inability to use LISP to influence the choice of whith ETR
   should be used to reach a multi-homed enterprise. In other
   words, when anycast addresses are used as RLOCs, LISP by itself
   does not provide any traffic engineering capabilities.

> between two ISPs.  Mix and match.
 
The observation specifically cites the case of an enterprise
"multi-homed to two (or more) ISPs". With this in mind please
consider the case of site A attached to ISPs X, Y, site B attached
to ISPs Y and Z, and site C attached to ISPs X, Y, and Z.  Both
site A, site B, and site C use anycast addresses as their RLOCs.
What are the procedures for allocating these anycast addresses ?
What are the rules for advertisting routes to these anycast addresses ?

> notice that it has lost internal connectivity, and act accordingly.
 
This is a LISP issue, as LISP proposes to use anycast addresses
as RLOCs. Thus LISP specs have to document how xTRs should deal
with that issue. "A smart box might notice that it has lost
internal connectivity, and act accordingly" is *not* an adequate
documentation.

Yakov.

---

From: Eliot Lear <[email-masked]>
Date: Thu, 22 Apr 2010 21:56:31 +0200

Hi Yakov,

>     does not provide any traffic engineering capabilities.

I guess I think the text adequately addresses the issue, and see below.

> What are the procedures for allocating these anycast addresses ?

First, why is this limited to LISP?  Wouldn't this be an issue for any 
anycast service?  Second, there is an implicit assumption in what you 
write that this would be a globally scoped anycast- that is, people 
would be announcing anycast willy nilly.  I think that unlikely since it 
would in effect replace one routing entry of an end system for another.

On the other hand, a more localized use of anycast seems to carry very 
limited impact on the routing system.

> What are the rules for advertisting routes to these anycast addresses ?

Again, why is this a LISP issue?  What are the rules for announcing any 
anycast route?

> internal connectivity, and act accordingly" is *not* an adequate

Yakov- how do you handle a case where you have a router originating a 
BGP announcement for a network that it is no longer connected to?  Why 
would this situation be any different?

---

From: Yakov Rekhter <[email-masked]>
Date: Thu, 22 Apr 2010 13:16:28 -0700

Eliot,

> First, why is this limited to LISP?  

Because we are discussing LISP.

> Wouldn't this be an issue for any 

Please answer the questions I asked you.
  
> Again, why is this a LISP issue?  

Because we are discussing LISP.

> What are the rules for announcing any 
> anycast route?

That does not answer the questions I asked above.

> would this situation be any different?

That is not an acceptable answer.

Yakov.

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 22 Apr 2010 22:56:04 +0200

Yakov,

>    does not provide any traffic engineering capabilities.

That is not correct. No one says that LISP site may not have 4 ETRs with 
each pair using anycast address as RLOCs yet provide for traffic 
engineering between such pairs to a LISP site.

> What are the rules for advertisting routes to these anycast addresses ?

Again no one says from what I know that a site is attached with single 
ETR to a given ISP. So with this in mind if site uses redundancy with 
it's attachments each anycast address may come from each ISP pool of PI 
addresses which would be fully able to be aggregated in the Internat 
without _any_ need to advertise /32s beyond area spanning given set of ETRs.

R.

---

From: Eliot Lear <[email-masked]>
Date: Thu, 22 Apr 2010 23:04:38 +0200

> Because we are discussing LISP.

The general principles of anycast (both benefits and limitations) are 
well understood.
>> anycast service?

Please answer this question.  Because absent an answer we do not lead to 
changes to a document.

>> Second, there is an implicit assumption in what you

Absent a desire to implement a global anycast, existing allocation 
mechanisms within a service provider will suffice, and when two SPs are 
involved, a bilateral (or multilateral) agreement allows for an 
announcement of a specific address or block.

> Because we are discussing LISP.

No, we're not.  We're discussing basic L3 routing at this point.

> That does not answer the questions I asked above.

Yes it does because you can make use of those very same rules.

> That is not an acceptable answer.

Thank you for your opinion.  A cornerstone of LISP and engineering in 
general is to build on what has come before.  This principle applies 
here as well.  Sad you seem not to understand such things.

Eliot

---

From: Dino Farinacci <[email-masked]>
Date: Thu, 22 Apr 2010 14:05:39 -0700

>   does not provide any traffic engineering capabilities.

I don't think it's all or nothing here. You could have this situation:

o A multi-national company with locators in the US and Europe.
o There are 4 connections to the Internet to reach the site.
o Two connections are in the US. Two connections are in Europe.
o The locator-set has 2 RLOCs, each with priority 1.
o RLOC 1 is an anycast address assigned to both US connections.
o RLOC 2 is an anycast address assigned to both European connections.

This is an active/active policy for the site where they wouldn't have  
to run any locator reachability algorithm. The underlying network  
would route packets to 1 of the connection points within a RLOC.

So this model is active/active at a hierarchical level:

	    RLOC1        RLOC2       active/active
            US1 US2     Euro1 Euro2   active/active/active/active

The LISP ITR selects either RLOC1 or RLOC2 based on the 5-tuple hash  
of the inner header. And underlying routing selects the physical path  
to the ITR selected RLOC.

Dino

---

From: Yakov Rekhter <[email-masked]>
Date: Thu, 22 Apr 2010 14:06:14 -0700

Robert,

> engineering between such pairs to a LISP site.

Ok. Let me modify the text I proposed to reflect this, as follows:

    Use of anycast addresses for resiliency in LISP results in
    inability to use LISP for traffic engineering among the xTRs
    that use the same anycast address, and specifically in inability
    to use LISP to influence the choice of which ETR should be used
    to reach a multi-homed enterprise if these ETRs use the same
    anycast address. In other words, when anycast addresses are
    used as RLOCs, LISP by itself does not provide any traffic
    engineering capabilities among the xTRs that use the same anycast
    address.

> without _any_ need to advertise /32s beyond area spanning given set of ETRs.

Please describe how this would be applied in the scenario I presented
above. For the purpose of the example, assume that ISP X has 10.1/16
pool, ISP Y has 20.2/16 pool, and ISP Z has 30.3/16 pool.

Yakov.

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 22 Apr 2010 23:28:27 +0200

Hi Yakov,

>      engineering capabilities among the xTRs that use the same anycast

I think I could come up with much shorter paragraph still reflecting all 
from the above:

"If packets are encapsulated to a given destination address (in 
particular anycast address) one should not expect that they would end up 
in a different destination then the one listed in the destination 
address of the packet's header."

> pool, ISP Y has 20.2/16 pool, and ISP Z has 30.3/16 pool.

This is similar example as just provided by Dino, but let me explain 
based on your exact data points provided:

Site A would be attached to ISP X with ETR1 and ETR1' - 10.1.1.1
Site A would be attached to ISP Y with ETR2 and ETR2' - 20.2.1.1

Site B would be attached to ISP Y with ETR3 and ETR3' - 20.2.1.2
Site B would be attached to ISP Z with ETR4 and ETR4' - 30.3.1.1

Site C would be attached to ISP X with ETR5 and ETR5' - 10.1.1.2
Site C would be attached to ISP Y with ETR6 and ETR6' - 20.2.1.3
Site C would be attached to ISP Z with ETR7 and ETR7' - 30.3.1.2

It is clearly an example of an enterprise multi-homed to two (or more) ISPs.

R.

---

From: Yakov Rekhter <[email-masked]>
Date: Thu, 22 Apr 2010 14:39:03 -0700

Robert,

> address of the packet's header."

That does not explicitly spell out the implications of using anycast 
addresses on the LISP TE capabilities.

> It is clearly an example of an enterprise multi-homed to two (or more) ISPs.

In your example sites A and B have 4 ETRs, while site C has 6 ETRs.

My scenario is different - site A has two ETRs, one connected to ISP X,
another to ISP Y. Ditto for site B. Site C has three ETRs, one
connected to ISP X, another to ISP Y, and the last one to ISP Z.

What are the addresses used by these ETRs ?

Yakov.

---

From: Dino Farinacci <[email-masked]>
Date: Thu, 22 Apr 2010 14:45:42 -0700

> n your example sites A and B have 4 ETRs, while site C has 6 ETRs.
>

In your case, and will probably be the typical case, each ETR address  
will be the PA address from the provider the ETR is connected to.

If you want to throw anycast into the mix, then create a site D with 2  
ETRs, each connected to the *same* ISP. In that case, an anycast RLOC  
could be used and assigned to each ETR.

Dino

---

From: Olivier Bonaventure <[email-masked]>
Date: Thu, 22 Apr 2010 23:47:32 +0200

Robert,
> address of the packet's header."

I'm not convinced that this reflects all of the above.

Anycast addresses are one of the possible solutions to provide
resiliency, but they are not the only ones. Other techniques can be used
to provide resiliency in case of failures and they do not necessarily
suffer from the same drawbacks as anycast addresses.
Given the scalability limitations of anycast address, we should probably
not recommend their utilisations as RLOCs except in environments where
scalability is not a concern.

Olivier

-- 
http://inl.info.ucl.ac.be , UCLouvain, Belgium

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 22 Apr 2010 23:54:19 +0200

Hi Olivier,

 > Given the scalability limitations of anycast address, we should
 > probably not recommend their utilisations as RLOCs except in
 > environments where scalability is not a concern.

I think you are 100% right. And I do not think anyone is recommending 
use of anycast addresses on ETRs across ISPs or in any other network 
location where scalability of the current system would be of any concern.

Cheers,
R.

---

From: Yakov Rekhter <[email-masked]>
Date: Mon, 26 Apr 2010 07:45:01 -0700

Dino,

> could be used and assigned to each ETR.

In this case please add the following to secion 8.2:

  When more than one CE at a site is configured with the same IP
  address (in which case an RLOC is an anycast address), then in
  order to avoid negative implications on routing scalability (in
  order to prevent carrying RLOCs as individual /32 (or /128 for
  IPv6) routes throughout the whole Internet), for a given site only
  CEs connected to the same ISP may be configured with the same IP
  address.

Yakov.

---

From: Dino Farinacci <[email-masked]>
Date: Mon, 26 Apr 2010 07:58:00 -0700

>  CEs connected to the same ISP may be configured with the same IP

Noted. Thanks.

Dino