Subject: RE: Multihoming by IP Layer Address Rewriting (MILAR)

From: "Michel Py" <[email-masked]>
Date: Mon, 3 Sep 2001 18:02:37 -0700

Iljitsch,

I completely agree with the following:

Iljitsch van Beijnum wrote:
>>
>> - the transport layer is not the best place to implement
>> multihoming: the natural place is layer 3 and implementing
>> it there also avoids large amounts of API problems and
>> higher layer changes
>>
>>- multihoming should work in all stages of a connection,
>> including setup
>>
>> - tunnels are a Bad Thing: they waste bandwidth, hide
>> the network topology and current PMTU discovery standards
>> and implementations easily break in the presence of careless
>> tunnel deployment
>>
>> - a router implementation is good: no need to change large
>> numbers of hosts

I would tend to say "visible over TWO or more transit networks"
for the following but agree on the rest.

>> In the presence of a good multiple-addresses multihoming
>> scheme, the requirement for getting an address block could
>> be being globally visible over three or more transit networks.
>> I would be very surprised if the number of networks that
>> qualify under these conditions is more than a few
>> thousand world wide.

I am a little more reserved about this:

>> - using multiple address ranges is the only way disconnect
>> the growth in multihoming from the growth of the global
>> routing table

Until someone finds a mechanism that can do it. Besides, the
growth of the multihoming table, if it contain multihoming
prefixes only and not the result of poor summarizations could
be controllable as you mentionned above.

>> Before I write up a draft, let me run it by the list.
Please take the time to have a look at:

http://arneill-py.sacramento.ca.us/draft-py-multi6-mhtp-01.txt
<http://arneill-py.sacramento.ca.us/draft-py-multi6-mhtp-01.txt> 

(I'd rather have the list look at this unpublished version
opposed to the published -00).

Thanks
Michel

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Tue, 4 Sep 2001 11:05:31 +0200 (CEST)

> for the following but agree on the rest.

That way, every current style multihomer qualifies. I can see some
dual-homed networks having a block of their own, but I certainly wouldn't
want all of them to have an address block of their own. Also, using
addresses from two blocks is not unreasonable, but a customer dual-homing
to two triple-homing ISPs would have six address ranges, that is too much.

Also note "globally visible". Here in The Netherlands many ISPs have one
"real" uplink that is globally visible, but also connect to the Amsterdam
Internet Exchange. I think it is reasonable in such a setup that they
would use address space from their transit ISP and announce that space
over the AMS-IX to local/regional peers. That way, the DFZ stays small but
local/regional traffic can still find the shortest path.

I know internet exchanges are out of vogue in the US, but they are a
reality in Europe. The AMS-IX carries 3 Gbit/s of traffic, compared to 300
Mbit for the New York Internet Exchange, even though bandwidth is a lot
cheaper in the US than in NL. The number of people that live in The
Netherlands is comparable to that in the NYC metro area. (But then the
AMS-IX is one of the four to six main exchange points in Europe.)

> Until someone finds a mechanism that can do it.

Ok, you're right, other ways, such as on-demand created routes, are
possible, but I don't see it happening.

> Besides, the

I don't think it can be controlled, unless draconian measures are taken.
I'm not as pessimistic as some others: I think the current multihoming
practices could continue to work for the foreseeable future if BGP
implementations can be optimized.

Still, if we can create a good host-based multihoming solution, this will
drain away much of the growth of current multihoming practices, giving us
all more breathing room. Also, it makes multihoming reachable for
everyone. With IPv6 we're in the unique position that the implementations
are still very fluid, so we should take advantage of that.

However, your proposal depends on changing the Internet core routing,
which is very hard to change. Note that there is hardly any IPv6 routing
in the core, while the changes in BGP necessary for this are minimal. Your
approach is much more radical and I have a hard time seeing it happen. On
the other hand, since there is so little IPv6 routing going on, there is a
window of opportunity there as well.

The main advantage of your proposal is that you solve the "alternative
address discovery" problem.

> http://arneill-py.sacramento.ca.us/draft-py-multi6-mhtp-01.txt

Note that a network that is part of the DFZ does not have "an ISP". You
pay an ISP to route traffic to all destinations connected to the Net, in
other words: you pay for a default. In the default-free zone there are no
defaults (but the reverse is not always true: a network running without a
default is not necessarily part of the DFZ) and you don't pay others to
handle your traffic: you peer with them. (Although I'm sure there are some
caveats there.)

Iljitsch

---

From: RJ Atkinson <[email-masked]>
Date: Tue, 04 Sep 2001 10:26:06 -0400

>AMS-IX is one of the four to six main exchange points in Europe.)

        The above is not a good comparison.  Virtually no 
international links terminate in NYC metro (most US-side termination 
of trans-Atlantic links is in/near southern NJ instead).  Lots of 
international links terminate in AMS metro area.

        AMS-IX, LINX, MAE-E, MAE-W, and Pennsauken NAP are all viable
exchanges (and will be long into the future) because they facilitate
*regional* and *international* interconnects, not merely metro 
interconnects.

Ran
[email-masked]

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Tue, 4 Sep 2001 17:12:51 +0200 (CEST)

>         The above is not a good comparison.

Granted, it shows only part of the picture.

> Virtually no

Chicken/egg: why would all those lines that come in in cities like
Noordwijk, Domburg and Alkmaar go to Amsterdam? Because all the ISPs are
there. And why are all the ISPs there? Because the AMS-IX is there. (Well,
ok, the fact that some people fail to notice that there is more to NL than
Amsterdam probably has something to do with it too.)

I seem to remember many lines going directly into NYC in the past, though.

> *regional* and *international* interconnects, not merely metro

Apart from being a regional/international exchange, the AMS-IX also
functions as a national exchange. In the US, it makes sense to transport
traffic a few hundred kilometers to a large exchange point. In Europe,
that often means you are in another country, which makes lines more
expensive and harder to get. So every country has at least one national
exchange, and not connecting to that exchange is very bad for business
for ISPs.

Look at the Brussels IX: an exchange 200 km from two of the largest
echanges in the continent wouldn't have a chance in the US, while in a
small country like Belgium it still carries 400 Mbps and has 45 members.

Real metro interconnects haven't surfaced here either.

Iljitsch

---

From: RJ Atkinson <[email-masked]>
Date: Tue, 04 Sep 2001 12:06:04 -0400

At 11:12 04/09/01, Iljitsch van Beijnum wrote:
>there. 

Actually, no.  They go there because that is how the physical
fibre runs -- and those aren't the driving cities in any
event.  It is the fibre runs from Amsterdam to Paris, Frankfurt, 
Scandanavia, under the ocean, and other longer distance places 
that make AMS-IX interesting to ISPs as an interconnect.

>I seem to remember many lines going directly into NYC in the past, though.

Not as the trans-Atlantic fibre runs it doesn't -- and hasn't ever.
In NYC there is too much chance of a ship anchor or fishing boat
or dredge accidentally pulling up the cable when it gets near 
to the shoreline.

>functions as a national exchange. 

        ISPs are indifferent to national exchanges.  It is the regional/
international exchanges that make economic sense to ISPs today.

>So every country has at least one national exchange, and 
>not connecting to that exchange is very bad for business for ISPs.

        Not when I've seen the full economic analysis. For example 
in the EU our analysis was that LINX and AMS-IX were the only two
exchange points that made economic sense for the particular 
multi-continent ISP at the time (which is reasonably recently).

        However, lets take this offline unless it gets tied firmly
back to multi6 topics in the interest of good list S/N ratio. :-)

Ran
[email-masked]

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Tue, 4 Sep 2001 22:01:24 +0200 (CEST)

> international exchanges that make economic sense to ISPs today.

A bit of economic sense. Many years ago, I did some traffic analysis. I
concluded that for an average dial-up ISP 45% of the traffic was
transatlantic, 10% to the rest of Europe and 45% to destinations within
The Netherlands. I haven't repeated the full analysis, but as far as I've
seen, this pattern has stayed very much the same.

Now, considering that:

- there are no other exchanges to speak of (yet) in NL

- relatively few other Dutch networks connect to other European exchanges,
  and those that do to relatively few of them

would you agree that it makes economic sense for a Dutch network to
connect to the AMS-IX?

I've seen it time and time again (and I've had a hand in it a few times
too): as soon as a Dutch network that provides transit services, be it
dial-up, leased lines or colo, starts to use more than one or two E1s it
connects to the AMS-IX.

> back to multi6 topics in the interest of good list S/N ratio. :-)

I've made all my points on the subject, I think.

Iljitsch van Beijnum

---

From: RJ Atkinson <[email-masked]>
Date: Tue, 04 Sep 2001 16:21:24 -0400

>would you agree that it makes economic sense for a Dutch network to

I've already agreed that any European network ought to be at AMS-IX, 
regardless of that network's location, simply because its at a place 
where lots of long-haul fibres terminate. 

Where we disagree is the relative utility of connecting at each
national/metropolitan exchange.  I think most lack a good business
case (from an ISP perspective), so we ought not use metro-oriented
addressing as the basis for multi6 solutions in the near-term.

Ran
[email-masked]