Subject: RE: PI/metro/geo [Re: The state of IPv6 multihoming development]

From: "Tony Li" <[email-masked]>
Date: Tue, 29 Oct 2002 15:27:34 -0800

|   However, if we can build a naming 
|   hierarchy in which
|   most multihomed networks connect to two places that are 
|   very close in
|   the hierarchy, this buys us aggregatability. Obviously, 
|   this is never
|   going to be a long term solution, but it makes for a very good short
|   term solution as it can work pretty much immediately by 
|   just adopting a
|   new address allocation scheme and some creative BGP filtering.

Unfortunately, that's not going to be so simple.  What happens
when some organization that spans geographic areas decides
that they want to multihome through providers on either sides
of their particular continent?  Or even different continents?

|   It's hardly the same thing, as perpetual motion is something that
|   doesn't work in theory or practice, no matter which way you 
|   slice it.
|   (Obviously I'm not saying something can't move "forever", but rather
|   that you can't derive energy from something forever.) You can make
|   routing work in any way you want, it's just that some ways 
|   are cheaper
|   or faster than others. That's one of the problems with 
|   networking: even
|   if done wrong it can work so people don't even see they're doing it
|   wrong. But then sometimes this is an advantage as you can still get
|   things to work even if there is a reason why it can't be done right.

Exactly.  What we want to build is a routing architecture that is 
never going to have to be replaced.  It's too difficult to reach in and
change it, so you'd like it to be done right the first time.  Since
scalability is key and the way to scale is to keep the overhead low,
a solution that has higher overhead is sub-optimal and to be avoided.

Geographic aggregation is fine if the topology does indeed provide
a convenient aggregation boundary.  But in many cases, there will be no 
such convenient boundary.  What do you do?  You cannot make an exception
and flat route any enterprise that has such links, because then their
overhead dominates the cost of all routing and you're not sufficiently
scalable.

Now I admit that you can warp the virtual topology by tunneling in such
a way that any two points on the graph are adjacent.  But by doing this,
you are abstracting routing away from the physical topology and you're
insuring that routing never makes a correct decision with respect to 
that physical topology.

|   Simple = using little information. Flat routing gives you all the
|   information you need. Depening on aggregates leads to less optimal
|   forwarding decisions.

Routing Theorem #1.  ;-)

|   > The K/K paper does show that, given certain reasonable 
|   assumptions about
|   > topology, that the increase in path length due to their 
|   information hiding
|   > scheme grows very small as the network gets larger, 
|   although of course their
|   > whole paper is about mono-metric systems.
|   
|   It seems to me you have some very clear opinions on how this should
|   work. Can we expect a draft?

You might surf around for "Nimrod" and "Chiappa".  There's about an
entire book's worth of material around about this.

As has been remarked elsewhere, we've been around and around and
around on this topic for the last decade.  We have no new answers,
only the bitter pill of reality that we cannot all seem to swallow 
at the same time.

Tony

---

From: Masataka Ohta <[email-masked]>
Date: Wed, 30 Oct 2002 09:20:58 +0859 ()

Tony;

> scalability is key

Exactly.

However, routing architecture intelligent enough to support
multihoming does not scale.

The basic problem is that intelligent network does not scale.

The basic principle against the problem is end to end.

> and the way to scale is to keep the overhead low,

Low overhead is nothing more than a constant factor of improvement
and is useless for real large network.

							Masataka Ohta

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Wed, 30 Oct 2002 12:57:42 +0100 (CET)

On Tue, 29 Oct 2002, Tony Li wrote:

> |   However, if we can build a naming  hierarchy in which
> |   very close in the hierarchy, this buys us aggregatability.

> of their particular continent?  Or even different continents?

Very few non-ISP organizations do this, as it requires them to move
traffic over their internal network which costs money, while receiving
the traffic where it eventually needs to be usually doesn't cost more
than receiving it at any other location.

In fact, large organizations with a large address block of their own
often announce more specifics in different locations. There is usually
no technical reason for having a large block rather than several smaller
ones.

If we can solve the problem for all other multihomers except the ones
that connect to the net in locations very far apart and _need_ to do
"internal transit", I think that would be good enough. Don't forget that
at this time, NOBODY other than ISPs gets to multihome in IPv6.

> |   That's one of the problems with networking: even

> never going to have to be replaced.

I don't think that's in the charter for this wg, but I agree this is a
worthy goal. However, we should also recognize that _some_ form of
multihoming in IPv6 is necessary _soon_ (at a recent AMS-IX technical
meeting the fact that the exchange couldn't get provider independent
IPv6 address space was debated at length, RIPE has similar problems and
that's just what I know first hand). That's the reason I came up with my
provider-internal draft, which I originally named "geo for now" because
it only provides a short-term solution. Unfortunately, it looks like the
RIRs aren't going to implement a new address allocation mechanism as
long as this doesn't get the IETF stamp of approval, despite the fact
that the draft only proposes new operational practices and not a new
protocol.

> It's too difficult to reach in and
> change it, so you'd like it to be done right the first time.

I don't think "this" is the first time.  :-)

> a convenient aggregation boundary.

Geography in itself is useless, as it isn't visible to the network.
However, aggregation makes the logical path longer. If we can use
geography to map long logical paths to short physical ones, that isn't
an issue and we can aggregate as aggressively as possible. In other
words: if I have to route a packet to a place where more specific
information is known, it helps if this place is in the direction the
packet has to travel anyway.

> But in many cases, there will be no

If we require a solution (even a short-term one) to address this fully,
then yes, this is a problem. However, I think we can leave this problem
unsolved for now as there doesn't seem to be a good reason for an
organization to do this other than cost saving and we're not in the
business of making the internet a more expensive place for all to
accomodate savings for some.

> only the bitter pill of reality that we cannot all seem to swallow

I think we're getting closer as we've been mostly discussing
multi-address solutions the past week and these are compatible with the
reality pill. There are many concerns with this type of solution, but I
haven't seen any show stoppers yet.

The only question is whether we can agree on a temporary
fix to give us more time to develop one or more of these solutions.

Iljitsch

---

From: RJ Atkinson <[email-masked]>
Date: Wed, 30 Oct 2002 14:19:07 -0500

On Wednesday, Oct 30, 2002, at 06:57 America/Montreal, Iljitsch van 
> than receiving it at any other location.

Our samples spaces must be different.  I think that many non-ISP 
organisations
multi-home via different providers on either sides of North America.
The examples tend to be larger organisations rather than a mom-and-pop,
but there seem to be very many examples of such organisations that do
this today.

Ran

---

From: "Tony Hain" <[email-masked]>
Date: Wed, 30 Oct 2002 12:57:18 -0800

Tony Li wrote:
> ...
> never going to have to be replaced.  It's too difficult to 
> reach in and change it, so you'd like it to be done right the 
> first time.  

Unfortunately 'never' is a very long time, and we will continue to do
nothing while we wait for the answer. 

> Since scalability is key and the way to scale is 
> to keep the overhead low, a solution that has higher overhead 
> is sub-optimal and to be avoided.
> 
> Geographic aggregation is fine if the topology does indeed 
> provide a convenient aggregation boundary.  But in many 
> cases, there will be no 
> such convenient boundary.  What do you do?  You cannot make 
> an exception and flat route any enterprise that has such 
> links, because then their overhead dominates the cost of all 
> routing and you're not sufficiently scalable.

This point is arguable, because it is making a blanket statement about a
process with multiple causes. For the truly global enterprise with
multiple exposure points and global internal network that could handle a
route shift, why not? There are fewer than 11,000 toady as evidenced by
the number of AS's injecting prefixes into BGP. (I suspect the real
number is less than 2,000 that could deal with taking a full traffic
load at multiple points) The real problem is that there is no clear way
to say one organization can while another can't. For the case where an
enterprise is simply looking for better/cheaper service in a remote
region, we are into economic rather than technical arguments. These
organizations aren't concerned about their impact on routing, so the
only way to deal with them is to give them the level of service they
need while making it cheaper to constrain the topology than it is to
blow out the tables. 

Tony

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Wed, 30 Oct 2002 22:52:38 +0100 (CET)

> multi-home via different providers on either sides of North America.

It is certainly possible we're seeing different things from the places
we're sitting. But the real question isn't whether they _connect_ on
opposite ends of a continent or ocean, but whether they need to announce
a globally visible route in both locations. This is only absolutely
necessary if they prefer traffic to flow over their internal network
rather than over an ISP network. It is also desireable for backup
purposes but I feel this isn't an important enough reason to allow it.

---

From: RJ Atkinson <[email-masked]>
Date: Wed, 30 Oct 2002 17:38:26 -0500

On Wednesday, Oct 30, 2002, at 16:52 America/Montreal, Iljitsch van 
Beijnum wrote:
>  It is also desireable for backup

Backup is a big driver for current multihomed customers.
Disallowing that really reduces the applicability of an approach.

Ran

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Wed, 30 Oct 2002 23:43:01 +0100 (CET)

> Disallowing that really reduces the applicability of an approach.

I don't mean that these people shouldn't have backups. What I mean is
that injecting a globally visible route into the infrastructure 100% of
the time to avoid sub-optimal, but otherwise functional routing 1% of
the time isn't a good use of scarce resources.