Subject: Provider Independent addressing usage

From: Pekka Savola <[email-masked]>
Date: Tue, 10 Jul 2001 22:24:21 +0300 (EEST)

Hi,

I just read the recent PI addressing usage I-D.

Based on the quotes at the end of the I-D, it would appear that at least
some of my concerns are shared by others, but nonetheless, let's hope this
brings up something new.

Problems with the PI approach:

 1) people don't seem to want too specific routes in the DFZ.  With
current policies, I'd estimate 100-200 routes is the maximum. (though the
technical constraints are a couple of orders of magnitude larger).
Specifying these with prefix length may give the wrong impression, as the
space would be rather densely populated (minus the oceans, sahara etc.)

 2) because of non-specific advertisements, operators would be forced to
carry a lot of traffic not their own, or throw it out.  Neither is
acceptable, economically or from user's point-of-view.

 3) the only ones that could really gain from advertisements are
regional/area/etc. IX's that have a very high level of regional ISP
penetration; e.g. if address belongs to the region, it'd be reachable
through the members of an IX with over 99.9% probability.

 4) ISP's operating at the IX would have to advertise, among themselves,
full /48 routes.  In some regions, where there are dozens of millions of
Internet users, this is probably not an acceptable solution either.  So
the problem goes back to 2), but in smaller scale where there are
often _no_ IX's to exchange the traffic.

==>

So, it's often the case, e.g. in Europe, that a country has about one IX.
The PI solution would work if the prefix(es) of the region belonging to
that country were advertised to by that IX, and _everyone's_ (in that
country) /48 PI prefix be advertised within that IX (whether it's directly
associated, or a smaller client ISP of the peering bigger ISP).

Additionally there would have to be some peering with neighbouring IX's,
so the locations near the borders of a country could be sent to the right
country.

If this was not done, some regional ISP's would have to capture and carry
some traffic they have no idea if they can deliver or dumped.  Or, the
traffic could be sent to some smaller-than-regional IX's which by
definition didn't exist.

Take the netcraft web server count by domain for example
[http://www.netcraft.com/Survey/Reports/0106/bydomain/]; I think it's fair
to assume that each /48 site would have at least one if not more web
servers.  United Kingdom appears to have about 2.8 million, so say 3
million /48's.  Even if there were 10 IX's in the UK (distributed nicely
by geographical areas, no less), this would still be a whole lot of
specific /48 routes between participating ISP's.

These appear to be rather unscalable restrictions.

==>

So some portions of this resemble the optimal 6to4 relay router finding
issue (using anycast, for example); only, this is a about 2^28 times
bigger problem....

-- 
Pekka Savola                 "Tell me of difficulties surmounted,
Netcore Oy                   not those you stumble over and fall"
Systems. Networks. Security.  -- Robert Jordan: A Crown of Swords

---

From: "Tony Hain" <[email-masked]>
Date: Fri, 13 Jul 2001 09:50:07 -0700

Pekka Savola wrote:

> Based on the quotes at the end of the I-D, it would appear that
> at least some of my concerns are shared by others, but
> nonetheless, let's hope this brings up something new.

Good, the reason I sent it out was to generate additional comments so the
document can cover as many concerns as possible.

>  1) people don't seem to want too specific routes in the DFZ.
> With current policies, I'd estimate 100-200 routes is the
> maximum. (though the technical constraints are a couple of
> orders of magnitude larger). Specifying these with prefix length
> may give the wrong impression, as the space would be rather
> densely populated (minus the oceans, sahara etc.)

I agree that removing specifics from the DFZ is the goal, but punching holes
in provider aggregates is the current reality and that mechanism is directly
contrary to the goal. I don't understand your point about prefix length.
What I was trying to say in the draft was that each of the regions outside
of the current scope would be represented by single prefix. The population
density of these remote regions has no impact on routing in the local
region.

>  2) because of non-specific advertisements, operators would be
> forced to carry a lot of traffic not their own, or throw it
> out.  Neither is acceptable, economically or from user's
> point-of-view.

Simply FUD. There may not be a customer relationship between an intermediate
operator and the destination, but the traffic was originated via a customer
relationship with the source, and there is some operator with a relationship
to the destination. Inter-provider agreements are what they are, and they
have little relationship to end customers. When a site is connected to
multiple operators each of them become identifiable entities to the
intermediate operator, so there is a potential for customer relationships
between providers. If end-sites are directly connecting to exchanges, the
exchange becomes the logical customer of the intermediate operators and
passes through the costs to the connecting customer.

> 3) the only ones that could really gain from advertisements
> are regional/area/etc. IX's that have a very high level of
> regional ISP penetration; e.g. if address belongs to the
> region, it'd be reachable through the members of an IX with
> over 99.9% probability.

The PI addresses that actually have an attached site will always be
reachable through some set of the ISP's in the region. It appears that you
are concerned that some of the ISP's in the region are not participants in
an exchange. This is not required as long as one of the providers that are
participating is willing to tie it all together. The only mechanism to
enforce this is for the inter-region providers to refuse more specifics, or
charge exorbitant fees.

> 4) ISP's operating at the IX would have to advertise, among
> themselves, full /48 routes.  In some regions, where there
> are dozens of millions of Internet users, this is probably
> not an acceptable solution either.  So the problem goes back
> to 2), but in smaller scale where there are often _no_ IX's
> to exchange the traffic.

Lack of an exchange in the current topology is not relevant to what might be
necessary to scale millions of multi-homed sites in a specific scope. Look
back a few years and you would find only a few exchanges globally.
Additional exchanges were built as the engineering trade-offs showed value
in establishing an exchange at a particular place. The design of this PI
mechanism allows exchanges to be created at the scopes that make sense for
local engineering reasons, without impact about on any other scopes.

> ==>
>
> So, it's often the case, e.g. in Europe, that a country has
> about one IX. The PI solution would work if the prefix(es)
> of the region belonging to that country were advertised to
> by that IX, and _everyone's_ (in that country) /48 PI prefix
> be advertised within that IX (whether it's directly associated,
> or a smaller client ISP of the peering bigger ISP).

Forget country boundaries, and see the previous note about when and where
exchanges get built. Certainly the scopes relevant to an existing IX should
have their prefixes exchanged by the participants in the IX, or a route
server at the IX coordinated the actual prefix/provider relationships.

> Additionally there would have to be some peering with
> neighbouring IX's, so the locations near the borders of a
> country could be sent to the right country.

Depending on the provider/scope/IX relationships there may need to be some
fine-tuning of scope sizes exchanged at specific IX's. As noted in the
document anytime there are overlapping scopes, some coordination is
required, but that does not necessarily mean peering.

> If this was not done, some regional ISP's would have to capture
> and carry some traffic they have no idea if they can deliver or > dumped.
Or, the traffic could be sent to some smaller-than-
> regional IX's which by definition didn't exist.

I don't understand. If there is a site at the defined prefix there will be
an explicit entry known to the providers for that scope. If there is no site
there may be connection attempts, but that is no different from unassigned
provider-based prefix addresses.

> [http://www.netcraft.com/Survey/Reports/0106/bydomain/];
> I think it's fair to assume that each /48 site would have at
> least one if not more web servers.  United Kingdom appears to
> have about 2.8 million, so say 3 million /48's.  Even if there
> were 10 IX's in the UK (distributed nicely by geographical
> areas, no less), this would still be a whole lot of specific
> /48 routes between participating ISP's.

Yes; if all sites were multi-homed and the providers chose to implement it
that way. When they become concerned about scale, history shows they will
implement more IX's to break up the space.

> These appear to be rather unscalable restrictions.

They will appear to be unscalable if one insists on maintaining current
interconnect topology while at the same time watching the edge topology move
to massive multi-homing. I will not claim that the PI mechanism is
necessarily the best choice, but it is much better at global scaling than
punching holes in provider aggregates.

> ==>
>
> So some portions of this resemble the optimal 6to4 relay router
> finding issue (using anycast, for example); only, this is a
> about 2^28 times bigger problem....

The real problem is that the definition of 'optimal' depends on perspective.
The ISP perspective will be very different from the end customer
perspective. The goal of PI is to balance those perspectives with a slight
weighting toward the end customer site.

Tony

---

From: Pekka Savola <[email-masked]>
Date: Tue, 17 Jul 2001 10:28:46 +0300 (EEST)

> density of these remote regions has no impact on routing in the local

Punching holes is the current reality with _IPv4_.  In IPv6 world (at
least 6bone), too specific routes are filtered by almost everybody; so
this would not work.

> passes through the costs to the connecting customer.

FUD how?  I don't see flaw in my point _if_ there are no significant
exchanges in the regions in question.  If there are a lot of exchanges,
ie. "neutral ground", this is not a problem.

Consider:

Assume source is in e.g. Europe and the destination in someone from a
different provider in a backwater city in the U.S.  Assume there is no
regional exchange close by.

There are a few issues here:
1) which out-of-the-country provider is selected
2) which intercontinental carrier is selected
3) where the intercontinental carrier drops the traffic
4) where someone else picks it up

1) no problem and no matter; the customer has default routes and direct
relation with his ISP's.

2) all EU-US carriers probably want to advertise reachability to all US
destinations, I'd suspect.  So, one of these is picked.

3) The carrier does not know what to actually do about the packet yet, but
let's assume it just pushes it toward a PoP nearest to the region.

If there is no IX in the region, the carrier can either dump the packet
(if it doesn't know a more specific route), or throw it at some operator
operating in the region.

4) Some operator operating in the region _might_ get a packet, but it
can't know yet whether the destination in the area is reachable or not.
So it could take the packet, muck around with it, perhaps route it closer
to the destination (geographically) and "throw the hot potato to the next
guy".

As said, these problems are "fixed" by:
 1) lots and lots of IX's (up to the level where having full /48's is
acceptable), or
 2) full /48 routes in all ISP's, carriers etc. which makes the entire
routing table size consideration moot

> enforce this is for the inter-region providers to refuse more specifics, or

What I'm worried about is situation where SiteA is customer of ISPA, and
ISPA does not participate in exchange in SiteA's region, but does
participate in some other exchanges.  Assume SiteB is in the same region
as SiteA, using ISPB which does participate in the regional exchange.  Now
the exchange advertises the reachability of regional prefix, but SiteA
will not be reachable.  ISPB and ISPA might not be willing to "tie it
together" if there is no customer relationship between them (ie, not ISP -
Carrier).

I think one of the goals is that inter-region providers, or "upper layers"
in general, don't have to know all the specific routes.

> local engineering reasons, without impact about on any other scopes.

Perhaps the practical requirement for IX's should be noted more clearly in
the draft.

> server at the IX coordinated the actual prefix/provider relationships.

Countries often have an IX, and as international connections often require
paying for carrier service or getting possibly rather expensive
connections, it's often that regional boundaries are actually
de-facto country boundaries.

Bigger IX's (probably most of the ones we regard as real exchanges now)
are most often "international".

> provider-based prefix addresses.

See the concern above on ISPA offering service in an area, but not
necessarily peering, or participating in an exchange (if one exists) in
that particular area (but perhaps knowing ISPB from some other areas).

> implement more IX's to break up the space.

If PI addressing were to be adopted at large, there would have to be more
IX's.  Someone has to administer, and pay for them.  This might become an
important point to consider.

---

From: "Tony Hain" <[email-masked]>
Date: Tue, 17 Jul 2001 18:09:39 -0700

Pekka Savola wrote:

> Punching holes is the current reality with _IPv4_.  In IPv6 world 
> (at least 6bone), too specific routes are filtered by almost 
> everybody; so this would not work.

Filtering more-specifics was an IPv4 activity 5 years ago, but the reality of providing the service the end customer is paying for trumps the 'network-centric' viewpoint every time. If a site uses provider-based addresses and expects those to be routable through another provider then the upstream table of those two providers has to accept the explicit entry. No amount of wishing will make that untrue.

> FUD how?  I don't see flaw in my point _if_ there are no 
> Significant exchanges in the regions in question.  If there are 
> a lot of exchanges, ie. "neutral ground", this is not a problem.
>
> Consider:
>
> Assume source is in e.g. Europe and the destination in someone 
> from a different provider in a backwater city in the U.S.  
> Assume there is no regional exchange close by.
>
> 1) no problem and no matter; the customer has default routes 
> and direct relation with his ISP's.
> 
> 2) all EU-US carriers probably want to advertise reachability 
> to all US destinations, I'd suspect.  So, one of these is 
> picked.
> 
> 3) The carrier does not know what to actually do about the 
> packet yet, but let's assume it just pushes it toward a PoP 
> nearest to the region.
> 
> If there is no IX in the region, the carrier can either dump 
> the packet (if it doesn't know a more specific route), or throw 
> it at some operator operating in the region.
> 
> 4) Some operator operating in the region _might_ get a packet, 
> but it can't know yet whether the destination in the area is 
> reachable or not.
> So it could take the packet, muck around with it, perhaps route 
> it closer to the destination (geographically) and "throw the hot 
> potato to the next guy".
> 
>  1) lots and lots of IX's (up to the level where having full 
> /48's is acceptable), or
>  2) full /48 routes in all ISP's, carriers etc. which makes 
> the entire routing table size consideration moot

The reason I say FUD is the argument appears to be ignoring the origin of the routing entry. If an operator is forwarding a packet toward a pop in the destination region, it had to get that route from somewhere. The only way your scenario works is if the operator is not providing service in the destination region and decided to make up a route to that prefix for internal use. If a provider does this, yes the traffic will die, but this is no different than making up any other prefix.

The only way a provider would know which provider to hand a packet off to is not a hot-potato guess, but hearing a routing advertisement. If the origin provider hands off the packet based on the longest-match prefix for the destination, the expectation is that the packet will get delivered. If there are providers not participating in the exchange covering that region, the provider holding the packet would need to know the specifics from them. This is no different from provider-based prefixes; there is no magic.

There may be additional exchanges built, but this will be a clear engineering driven decision based on the number of multi-homed sites and the acceptable size of the local routing table. This is in contrast to the current set of exchanges that were built for purely political reasons, or simply avoiding the politically motivated international tariffs. Yes this mechanism will shift the balance back from private peering toward exchange-based peering, but is that a bad thing? It certainly opens up the local service area in a way that locking sites to a provider-based address will never do. The cost of connecting to multiple exchanges will act as a natural counter balance to prevent an absurd number of them from being created.

> What I'm worried about is situation where SiteA is customer 
> of ISPA, and ISPA does not participate in exchange in SiteA's 
> region, but does participate in some other exchanges.  Assume 
> SiteB is in the same region as SiteA, using ISPB which does 
> participate in the regional exchange.  Now the exchange 
> advertises the reachability of regional prefix, but SiteA
> will not be reachable.  ISPB and ISPA might not be willing 
> to "tie it together" if there is no customer relationship 
> between them (ie, not ISP - Carrier).

We are talking about multi-homed sites, right? So I assume you mean SiteA is a customer of ISPA & ISPC, while SiteB is a customer of ISPB & ISPD. If ISPA or ISPC neither one participate in the regional exchange, and their upstream filters the specific for SiteA (as it should), SiteA has a reason to switch to ISPB or ISPD. Since the addresses are not derived from ISPA or ISPC, there is no deterrent causing SiteA to stick with its current lame providers. SiteA is paying for a service; insulation from problems any individual provider may have; and in this scenario ISPA & ISPC both have a problem that SiteA wants to avoid.

> I think one of the goals is that inter-region providers, or 
> "upper layers" in general, don't have to know all the specific 
> routes.

I agree completely, but they can't push back and refuse to know specifics unless they have a replacement mechanism that gets the bits delivered to the paying customer. Provider-based specifics are all we have currently defined, and they are demonstrably broken for this purpose. 

> Perhaps the practical requirement for IX's should be noted 
> more clearly in the draft.

Please send text.

> Countries often have an IX, and as international connections 
> often require paying for carrier service or getting possibly 
> rather expensive connections, it's often that regional 
> boundaries are actually de-facto country boundaries.
> 
> Bigger IX's (probably most of the ones we regard as real 
> exchanges now) are most often "international".

I will give you the point that costs change as national boundaries are crossed, but the size and location of an exchange have no mandatory relationship to the prefix lengths exchanged there. For example, assume there is a massive exchange in London with 100's of providers participating covering all of the UK, and nearby another exchange, say Moscow, where there may only be 5 providers, but they cover all of Russia. The fact that one has more participants, or covers a region approximately 10 degrees square, while the other covers a region 20 x 150 degrees has nothing to do with the number of multi-homed sites supported. They may both be inadequate, or may both be overkill for the required service. 

The requirement is that all participants agree on the set of prefixes to be exchanged, and that set will almost assuredly contain multiple lengths to avoid overlapping with a neighboring exchange. 

> If PI addressing were to be adopted at large, there would have 
> to be more IX's.  Someone has to administer, and pay for them.  
> This might become an important point to consider.

And the multi-homed sites, which created the need, would be the logical place to look for the money. Currently there is no way to associate the site that creates the problem with where the pain is felt; but something like this where the providers could point at a specific local cost to support the site's goal, and the site could in turn see explicit value for the additional cost, might help that problem. 

Tony

---

From: "Jon (Taz) Mischo" <[email-masked]>
Date: Tue, 17 Jul 2001 20:30:00 -0500 (CDT)

> in general, don't have to know all the specific routes.

This is exactly what the NA PSTN had to move away from.  Next thing you
know, your DSL router or cable modem or POTS modem is going to have a
coinbox attached to it.

-Taz

-- 
        "Be liberal in what you accept,
      and conservative in what you send."
--Jon Postel (1943-1998) RFC 1122, October 1989

---

From: Pekka Savola <[email-masked]>
Date: Wed, 18 Jul 2001 13:42:08 +0300 (EEST)

On Tue, 17 Jul 2001, Tony Hain wrote:
> Filtering more-specifics was an IPv4 activity 5 years ago, but the
> reality of providing the service the end customer is paying for trumps
> the 'network-centric' viewpoint every time. If a site uses
> provider-based addresses and expects those to be routable through
> another provider then the upstream table of those two providers has to
> accept the explicit entry. No amount of wishing will make that untrue.

That's not enough; every client's ISP in DFZ also has to accept the
explicit entry, at least in the case that:
 1) you want to protect from failure of your original PB ISP, or
 2) your PB ISP will not co-operate with secondary ISP, and give your
traffic to them when needed your PB-ISP link is dead.

Further, this protects against link loss toward PB-ISP, but not against
bigger failures _unless_ every router in DFZ has the explicit entry (so
return packets don't get blackholed, or sent to PB-ISP not Secondary-ISP).

When this level (really, at least 95%!) of filter-free 'ness is required,
multihomers might not see breaking the aggregates as a _working_ solution.

I don't know how IPv4 filtering was done 5 years ago, but currently I
believe many organizations in DFZ filter e.g. from /25 to up; I see this
as equivalent to filtering /48 in IPv6 world.

> The reason I say FUD is the argument appears to be ignoring the origin
> of the routing entry. If an operator is forwarding a packet toward a pop
> in the destination region, it had to get that route from somewhere. The
> only way your scenario works is if the operator is not providing service
> in the destination region and decided to make up a route to that prefix
> for internal use. If a provider does this, yes the traffic will die, but
> this is no different than making up any other prefix.

The problem here is, that unless the carrier or big ISP makes up a route
(e.g. based on assumption it has a PoP with a few customers in the area,
but no exchange nearby), e.g. with adding a static Null0 route and
redistributing that into BGP, and only uses specifics, it has to exchange
specifics with other carriers or huge ISP's and the number of these might
not be small.

But anyway.. this is perhaps kinda theoretical discussion as I doubt that
many sites would jump to use PI immediately, and the problem would be
rather unlikely to "explode" that soon.

> > more clearly in the draft.

Perhaps, something like, to be included in General Considerations:

It is believed that the amount of overly-specific route advertisements can
be minimized using exchange-based aggregation in different scopes.  This
way there would be no need for non-aggregated advertisements in the
networks interconnecting different scopes.  However, setting up exchanges
is not always technically, economically or otherwise feasible; it is a
balance between the amount of full /48 prefixes ISP's operating in the
scope are willing to accept, and between forming an exchange, or some
other "neutral ground", and using provider independent aggregates.  It is
believed that when the list of overly-specific route advertisements grows
too long, some sort of traffic exchanging will be established between the
vast majority of ISP's operating in that scope.

---

From: Pekka Savola <[email-masked]>
Date: Wed, 18 Jul 2001 15:33:51 +0300 (EEST)

On Wed, 18 Jul 2001, Brian E Carpenter wrote:
> Er, that is the goal of all route aggregation techniques and is the thing that
> multihoming makes very hard. It's the governing clause for every single
> thing that multi6 discusses. PI is one of the oldest ideas around to satisfy this
> goal and the question is still the same as always: is the deployment of all the
> necessary IX's a realistic expectation?

No; it's unrealistic to believe people start forming IX's just to appease
one multihoming solution..

> I think you'll find this goes back at least to a draft by Bill Simpson about 5
> years ago, and I still haven't seen a convincing argument why all those IX's will
> spring into existence.

.. But luckily all the IX's aren't required.  As long as operators in a
region are satisfied with carrying full /48 routes to every site in each
region, there are not that many problems.

When you get to the 10,000th PI site in a region, I'm sure more exchanging
will be considered if it hasn't been by then..

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 18 Jul 2001 07:14:48 -0500

Brian

---

From: "Tony Hain" <[email-masked]>
Date: Wed, 18 Jul 2001 10:42:05 -0700

Brian E Carpenter wrote:

> I think you'll find this goes back at least to a draft by Bill 
> Simpson about 5 years ago, and I still haven't seen a convincing 
> argument why all those IX's will spring into existence.

I never claimed it was novel. :) I think the difference between now and then is that at that time the discussion centered around having to make a hard choice between PI or provider-aggregates. The insurmountable problem was that exchanges had to come into existence immediately, or PI would sink the routing system. I believe that we are at a point were the host and routing implementations can realistically support both approaches simultaneously, and there are enough existing exchanges to bootstrap the process. Taking all of the multi-homed site problems out of PA space will allow meeting the IPv6 routing goal of reducing the table by 10x or more, as long as we can agree on a scaleable PI approach for those problems. It would be nice if someone with access to current topology and routing information would provide feedback about the real impact of each proposal from the perspective of each continent. I still contend from a theoretical perspective that if you write-off NA/Europe as hopelessly intertwined for the moment, my approach reduces the tables in the other 75% of the world to a handful of entries. I also believe that over time with a scaleable PI mechanism, even the NA/Europe problem will be cut in half. 

Tony

---

From: Jim Bound <[email-masked]>
Date: Wed, 18 Jul 2001 14:28:47 -0400 (EDT)

The IXs are in process now first in Europe and now in Asia.
Do not view this as U.S. focus that is just one part of the Internet.

/jim

---

From: Tony Li <[email-masked]>
Date: Wed, 18 Jul 2001 14:42:43 -0700

| The IXs are in process now first in Europe and now in Asia.  Do not view
 | this as U.S. focus that is just one part of the Internet.
 |
 | > The insurmountable problem was that exchanges had to come into
 | > existence immediately, or PI would sink the routing system. 

I think a very important question is about whether or not exchanges are
part of the natural economically incented growth of the Internet.  What
we've seen in the US is that they made some sense for awhile, but that
private interconnect is drawing the traffic away.  One then has to ask how
this architecture will play out given an underlying private interconnect
architecture.  

If one can get better (i.e., more reliable, higher bandwidth) service by
being in PA space and relying on that provider's private interconnect, why
then would anyone want to be in PI space?  For those places where exchanges
do not currently exist, it seems unlikely that the creation of a routing
architecture would cause those exchanges to come into being.  And for those
places where exchanges already exist, wouldn't most customer trend away
from PI space?

The meta point here is that whatever architecture is selected, it needs to
operate with the natural economic order.  Otherwise carriers and customers
will both be incented to circumvent the architecture to optimize their own
benefit.

Tony

---

From: "Tony Hain" <[email-masked]>
Date: Wed, 18 Jul 2001 17:36:11 -0700

Tony Li wrote:

> I think a very important question is about whether or not 
> exchanges are part of the natural economically incented 
> growth of the Internet.  What we've seen in the US is that 
> they made some sense for awhile, but that private interconnect 
> is drawing the traffic away.  One then has to ask how this 
> architecture will play out given an underlying private 
> interconnect architecture. 

Actually what you have seen in the US is the natural tension between captivity and openness play out. With the possible exception of the CIX, the exchanges in the US have always been about lowering interconnect costs to open the playing field to the small operator, and that is economically contrary to the large operators interests. Yes the big players want to lower costs, but their mechanism is to make the little guys become customers and pay for the private interconnects.

> If one can get better (i.e., more reliable, higher bandwidth)
> service by being in PA space and relying on that provider's
> private interconnect, why then would anyone want to be in PI 
> space?  For those places where exchanges do not currently 
> exist, it seems unlikely that the creation of a routing 
> architecture would cause those exchanges to come into being.  
> And for those places where exchanges already exist, wouldn't 
> most customer trend away from PI space?

If one trusted that single provider to be reliable and fast, they should in fact be in PA space. The counter question is; why would one want any association with a provider's address space when that trust does not exist? Customers are trending away from the traditional defined PI space in IPv4 simply because it is hard to get. At the same time the routing table growth shows they are in fact trending toward PI as an architecture, by requiring the PA space to be hacked up.

A routing architecture by itself will not cause exchanges to be built. What will cause them to be built is a demonstrable business model that shows it is cheaper to go that route than to fight with the instability of the current bloat. Something I sent in a related private thread:

The only way I see anything new being deployed is for someone to sit down with real data and show a sustainable business model that actually lowers overall costs. That exercise in itself is substantial enough that nobody will take it on unless there is general consensus that there is sufficient potential in the approach, and that its scope is bounded and understood. 

The reason I have been persisting on this is that I think we can bound this approach, and it is simple to understand and deploy. I am explicitly trading off efficiency in the address consumption for simplicity in the derivation and routing, as well as trading off maximal routing efficiency for end-to-end system level efficiency. 

> The meta point here is that whatever architecture is selected, 
> it needs to operate with the natural economic order.  Otherwise
> carriers and customers will both be incented to circumvent the
> architecture to optimize their own benefit.

Since those perspectives on 'optimal benefit' are in direct conflict, circumvention will happen in any case. The real question is can we define an architecture that limits the need to circumvent it to accomplish each party's goals? Or when it is circumvented, contain the damage. 

Tony

---

From: [email-masked]
Date: Sun, 22 Jul 2001 17:41:25 -0400

>>>>> "Tony" == Tony Li <[email-masked]> writes:
    Tony> I think a very important question is about whether or not exchanges are
    Tony> part of the natural economically incented growth of the Internet.  What
    Tony> we've seen in the US is that they made some sense for awhile, but that
    Tony> private interconnect is drawing the traffic away.  One then has to

  Is this because the exchanges are unprofitable, they fail to scale, or is
it due to concentration of ownership?

  It seems that exchanges are of greater benefit when there are many ISPs,
while bilateral agreements are easier when there are fewer players.

    Tony> The meta point here is that whatever architecture is selected, it needs to
    Tony> operate with the natural economic order.  Otherwise carriers and customers

  I'm not sure the the US marketplace is a very good example of "national
economic order" due to legacy of monopoly telco. At this point, we get into
politics.
 
]       ON HUMILITY: to err is human. To moo, bovine.           |  firewalls  [
]   Michael Richardson, Sandelman Software Works, Ottawa, ON    |net architect[
] [email-masked] http://www.sandelman.ottawa.on.ca/ |device driver[
] panic("Just another NetBSD/notebook using, kernel hacking, security guy");  [

---

From: Randy Bush <[email-masked]>
Date: Sun, 22 Jul 2001 15:17:24 -0700

>> private interconnect is drawing the traffic away.  One then has to

the exchanges are still there, the number of isps meeting at them has not
(yet?) declined significantly.  the *traffic* *proportion* has declined,
likely because the cost for providers to use private interconnect has
fallen with the price of circuits, and private interconnects are more
easily managed.  i guess this might be your "fail to scale" point.

> while bilateral agreements are easier when there are fewer players.

uh, bi-lats are what are used at the public exchanges.  no one went for
mlpas.

>> The meta point here is that whatever architecture is selected, it needs
>> to operate with the natural economic order.  Otherwise carriers and
>> customers
> economic order" due to legacy of monopoly telco.

then please remind us what country does not have the history (or present)
of monopoly telco?  this leaves us without lessons to follow, not a good
space.

randy

---

From: [email-masked] (Michael Richardson)
Date: Mon, 23 Jul 2001 13:11:18 -0400

-----BEGIN PGP SIGNED MESSAGE-----

>>>>> "Randy" == Randy Bush <[email-masked]> writes:
    >> So there is no "natural economic order" to learn from, is my point.

    Randy> i wish all my competitors were similarly hamstrung by having to start from
    Randy> a tabula rasa every morning.

  If the market has been distorted, then the market has been distorted. It
means that we can not claim that our technical discussion will be immune to
political concerns. 

  It might be, in a theoretically "natural" market that there is a business
case for IXs, but that given where we are now, there is no reason for them,
and little likelyhood that their need will return. 

  I genuinely hope that this isn't the case!

  My impression of the discussion is: if we assume IXs will grow in number, 
then multihoming becomes easier/cheaper.

---

From: Randy Bush <[email-masked]>
Date: Mon, 23 Jul 2001 08:50:26 -0700

> If two ISPs, who happen to have equipement located at an exchange decided
> to pull a x-over cable between their two boxes, is this a public or
> private circuit?

it is generally considered to be private peering.

> So there is no "natural economic order" to learn from, is my point.

i wish all my competitors were similarly hamstrung by having to start from
a tabula rasa every morning.

randy

---

From: [email-masked] (Michael Richardson)
Date: Mon, 23 Jul 2001 11:21:30 -0400

>>>>> "Randy" == Randy Bush <[email-masked]> writes:
    mcr> It seems that exchanges are of greater benefit when there are many ISPs,
    mcr> while bilateral agreements are easier when there are fewer players.

    Randy> uh, bi-lats are what are used at the public exchanges.  no one went for
    Randy> mlpas.

  I realize that.

  I meant that the private circuits take actual work to arrange vs using the exchange.

  If two ISPs, who happen to have equipement located at an exchange decided
to pull a x-over cable between their two boxes, is this a public or private circuit?
  
    >>> to operate with the natural economic order.  Otherwise carriers and

    mcr> I'm not sure the the US marketplace is a very good example of "national
  (my typo should be "natural")
    mcr> economic order" due to legacy of monopoly telco.

    Randy> then please remind us what country does not have the history (or present)
    Randy> of monopoly telco?  this leaves us without lessons to follow, not a good

  Practically none.

  This may be a situation where we will have to write a document aimed at the 
regulators before we can properly solve a technical problem.

---

From: Brian E Carpenter <[email-masked]>
Date: Mon, 23 Jul 2001 17:16:34 -0500

Michael Richardson wrote:
...
> then multihoming becomes easier/cheaper.

And we therefore need to find a technical solution that funnels the ISPs
in this direction, i.e. makes it in their selfish self-interest to connect
via IXes.

I've been waiting for that since March 1995 at least (that is the date
on draft-simpson-exchanges-00.txt, and no, I don't have soft copy or a URL).

  Brian

---

From: "Paul Francis" <[email-masked]>
Date: Tue, 24 Jul 2001 10:11:23 -0700

I've been pecking at an internet draft for GxSE, but I'm not sure I'll be
done with it before London.

What I can do, though, is put together some slides with the basic mechanism
and use that to lead a *small* bar bof if there is interest.  The goal of
the BOF would be to flesh out the design a bit, figure out the pros and
cons, and decide if it is something worth pursuing.

Any interest?

PF