Subject: A new spin on multihoming: multihoming classes.

From: Michel Py <[email-masked]>
Date: Wed, 5 Sep 2001 09:53:37 -0700 (PDT)

multi6 folks,

I think that one of the problems we are having is that
we are trying to design a universal solution for a too
broad topic.

Let's divide multihoming into two classes:
- Entreprise class.
- user class.

- Entreprise class is for companies that receive a BGP
feed from two different TLAs and have been allocated a
centrally managed multihomed block of IPv6 addresses.
These entreprises run mission-critical web servers
with large pipes to the Net and want 99.999%
availability and global load sharing among their
different connections.
The multihoming solution for entreprise class must be
contained at the network or internet layer. My
personal submission to entreprise class is:
http://arneill-py.sacramento.ca.us/draft-py-multi6-mhtp-01.txt

- User class is for home/soho setups that dual home
both to DSL and cable (example). These users might be
given dynamically assigned IPv6 addresses or prefixes
and are in potentially very large numbers. These users
mailnly use their connections for surfing and
eventually run minimal/personal servers over a slow
uplink. The potentially very large number of user
class multihomed sites and high volatility of IP
addresses makes it very difficult to centrally
administer.
The multihoming solution for user class could be based
on transport or higher levels. Survivability and
availability are nice to have but second to the ease
of a large scale deployment.

Any comments?
Michel.

__________________________________________________
Do You Yahoo!?
Get email alerts & NEW webcam video instant messaging with Yahoo! Messenger
http://im.yahoo.com

---

From: Michael Richardson <[email-masked]>
Date: Wed, 05 Sep 2001 14:32:37 -0400

-----BEGIN PGP SIGNED MESSAGE-----

>>>>> "Michel" == Michel Py <[email-masked]> writes:
    Michel> Let's divide multihoming into two classes:
    Michel> - Entreprise class.
    Michel> - user class.

  Let's be more precise here:
	- Ruling lords			(aka "servers")
	- serfs	 (no, not "surfs")	(aka "clients")

  This is not peer to peer/end-to-end.

  It totally fails to provide for the very large number of organizations
that in the IPv4 world do not do BGP with anyone because the barrier to entry 
is too high for IPv4. They would like not to be held hostage by the ISPs.
  
    Michel> - User class is for home/soho setups that dual home
    Michel> both to DSL and cable (example). These users might be
    Michel> given dynamically assigned IPv6 addresses or prefixes
    Michel> and are in potentially very large numbers. These users

  These users are right now given two IPv4 addresses (one from each).
  Typical usage is now to have two web proxies talking ICP to each other,
each running on a gateway that has its default route pointing at the
appropriate network. NAT for the real network, and screw IPsec, VoIP, and
anything else that is really peer-peer.

  Given a bit of competition on the DSL market, one can usually find at
least one static IP, often two if you realize that cable operators just use
DHCP for configuration (until very recently). Thus, 2x IPv6 /48s via 6to4 are 
immediately available to these organizations (including my "soho"). 

  We are not yet at 1 computer per household member, but in middle class
families, we are getting close. Add in PDAs and gameboys via household
bluetooth or 802.11, and I do not see how the "client/server" scenario you
propose makes any sense.
  
]       ON HUMILITY: to err is human. To moo, bovine.           |  firewalls  [
]   Michael Richardson, Sandelman Software Works, Ottawa, ON    |net architect[
] [email-masked] http://www.sandelman.ottawa.on.ca/ |device driver[
] panic("Just another NetBSD/notebook using, kernel hacking, security guy");  [

---

From: Marc Blanchet <[email-masked]>
Date: Wed, 05 Sep 2001 13:41:17 -0400

>Let's divide multihoming into two classes:

good classification

>uplink.

However, I would include in this class the "fiber-to-the home" and wireless 
802.11 community networks.
Still a user class, but the "slow uplink" might not be right for those users.

Marc.

>The potentially very large number of user

---

From: Shane Kerr <[email-masked]>
Date: Thu, 6 Sep 2001 10:34:07 +0200

On 2001-09-05 14:32:37 -0400, Michael Richardson wrote:
> that in the IPv4 world do not do BGP with anyone because the barrier
> to entry is too high for IPv4. They would like not to be held hostage
> by the ISPs.

While I'm not sure I agree with the politically-charged nature of this
post, I agree that there shouldn't be any difference between "big
honkin' server" end-to-end connectivity and "mom's recipe box server".
End-to-end is end-to-end, and I think that the multiple-IP solution is
the right tactic.

However, there is a need for network multihoming, which will probably
need to be solved as well.  But given that most of the growth of the
routing table seems to be end users desiring reliability and speed, I
think the network multihoming problem can wait.

-- 
Shane
Carpe Diem

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Thu, 6 Sep 2001 11:53:54 +0200 (CEST)

> think the network multihoming problem can wait.

It never hurts to give the subject some thought.

Many of the large networks would really like an 8k global routing table
in IPv6. That means: no more than 8192 ISPs with a PA block of their own.
(I'll be using IPv4 terminology.)

I think this is a reasonable number if we can enforce strict requirements
on holding PA blocks. Avoiding renumbering can't be a reason for having a
PA block any more, even if this requires millions of hosts to renumber
when there is some change in connectivity. People will feel they're held
hostage by their existing ISP unless renumbering becomes completely
painless. I think the tools for the actual renumbering are there, but
changing DNS entries for all these hosts might be a problem.

Then there are the enterprises that want to multihome at the network
level. Obviously, giving those PI space would be impossible. So they would
have to announce a /48 out of one ISP's PA block to other ISPs. The other
ISPs will allow this, because it brings in cash. Small ISPs will be happy
to receive the /48s at exchange points, because that saves them cash.
However, we can expect some of the larger ISPs to filter the /48s because
they have no incentive for carrying the more specific routes.

Still, as long as all of its other ISPs peer with the "primary" ISP (the
one the enterprise got its addresses from), the enterprise will be
reachable when the link to its primary ISP fails or that ISP's POP fails.
Only when the entire primary ISP is wiped off the network, they'll be
unreachable from networks that filter the /48s.

But in that case, they can still fall back to multi-address / host level
multihoming.

Iljitsch van Beijnum

---

From: Peter Tattam <[email-masked]>
Date: Thu, 6 Sep 2001 20:53:21 +1000 (EST)

> changing DNS entries for all these hosts might be a problem.

A6 is supposed to deal with this quickly and painlessly.

> But in that case, they can still fall back to multi-address / host level

Peter

--
Peter R. Tattam                            [email-masked]
Managing Director,    Trumpet Software International Pty Ltd
Hobart, Australia,  Ph. +61-3-6245-0220,  Fax +61-3-62450210

---

From: Randy Bush <[email-masked]>
Date: Thu, 06 Sep 2001 07:05:35 -0700

> in IPv6. That means: no more than 8192 ISPs with a PA block of their own.

really?  as it forms the base of your argument, i must ask if there is a
basis for this assertion?

my impression is that large (and small) providers would like fairly stable
routing which converges predictably in reasonable time.  if that can be done
with ten, 100k, or 5,000,000 routes is not so important.  but, as i said,
that is merely my impression.

randy

---

From: RJ Atkinson <[email-masked]>
Date: Thu, 06 Sep 2001 11:02:31 -0400

>basis for this assertion?

        I'd be comfortable with more than 8192 provided 
the routing system still met the criterion Randy outlines below.

>routing which converges predictably in reasonable time.  

        Yep.

>if that can be done with ten, 100k, or 5,000,000 routes is 
>not so important.  but, as i said, that is merely my impression.

        Some few folk think that somewhere between 100K and 300K prefixes
it isn't stable to have "fairly stable routing which converges
predictably in reasonable time" when using current BGP and current
BGP algorithms.  Those folk are credible, but I haven't seen any
equations yet myself.

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Thu, 6 Sep 2001 20:47:27 +0200 (CEST)

> basis for this assertion?

I've read words to that effect on this list, and I may have read the same
thing in a document (draft, RFC, RIR document), but I'm not absolutely
sure on the latter.

> that is merely my impression.

What are you saying? That there is no problem? That BGP implementations
need to get better and stop muliplying flaps? Or something else?

If we want an 8k routing table, we can't give ISPs more than a hand full
of PA blocks and we have to make sure only real ISPs get them. Not
limiting the number of PA blocks per ISP probably won't hurt much: with
the larger blocks they get with IPv6 they won't burn through them as fast
as in IPv4. If we relax the "real" ISP rule, there will be some more
growth, since many enterprises will become "ISPs" to be able to multihome
without problems. But this is pretty expensive, so the growth of the
global routing table should still be fairly reasonable. if we allow PI
and/or /48s out of PA space in the DFZ without any restrictions, the
routing table will grow as we've never seen before, and it is very likely
that many of the new networks will have less stable connections than
current networks and the new, large routing table will also be less
stable.

---

From: Randy Bush <[email-masked]>
Date: Thu, 06 Sep 2001 13:31:28 -0700

>> my impression is that large (and small) providers would like fairly
>> stable routing which converges predictably in reasonable time.  if that
>> can be done with ten, 100k, or 5,000,000 routes is not so important.
>> but, as i said, that is merely my impression.

> What are you saying? That there is no problem?

i don't see how my words could be interpreted that way.

> That BGP implementations need to get better and stop muliplying flaps?

if indeed bgp does that, and it is detrimental to stability, convergence,
etc., then i would think that fixing this would be one (possibly small)
tactical action to progress toward the goals i thought i was outlining.

> If we want an 8k routing table

maybe i need to be more explicit.  it is not clear to me that this is a
useful criterion.

but, to indulge in your tactical view

> if we allow PI and/or /48s out of PA space in the DFZ without any
> restrictions

i am not aware of this being seriously proposed.  the discussions i am
hearing imply /[23..36] in the inter-provider dfz.  /48s are for sites,
and i for one, am not inclined to recommend listening to /48s from peers.

randy

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Thu, 6 Sep 2001 23:06:55 +0200 (CEST)

> tactical action to progress toward the goals i thought i was outlining.

Have a look at RIPE document 210, section 3.1:
http://www.ripe.net/ripe/docs/ripe-210.html

> maybe i need to be more explicit.  it is not clear to me that this is a

Hence the "if". Just looking at the implications.

> and i for one, am not inclined to recommend listening to /48s from peers.

I think the large networks won't (although they will announce them to
peers if customers pay them for that), but I think the small networks will
listen to them at exchange points. It's cheaper for them to buy some extra
memory for their router and route traffic to as many /48s as possible over
the exchange rather than to the networks the /48s belong to, since that
connection will very likely be transit and cost them money.

Iljitsch van Beijnum

---

From: Michel Py <[email-masked]>
Date: Thu, 6 Sep 2001 14:48:30 -0700 (PDT)

> 	- serfs	 (no, not "surfs")	(aka "clients")

First, serfs are not limited to be clients, second
I don't have a problem with being a serf at home. I
run
a personal web site out of a mighty Pentium 133 server
and a residential Pacific Bell aDSL (128kbps
upstream).
If I can multihome it for $30 a month more, I will
give
it a thought.

And yes, I am also a ruling lord at the office, we
have
multiple OC48s, a robot to change backup tapes, a
multi-
terabyte SAN array, and a 24/7 staff and NOC.

> It totally fails to provide for the very large
> number of organizations that in the IPv4 world do
> not do BGP with anyone because the barrier to entry

I don't agree with that, at least not in the US. If
you
can't afford a BGP feed is because you don't really
need it. Yes, it would be nice to have my home network
connected with two DS3s to two ISPs, I could actually
configure it myself, I just don't make enough money to
justify it. A company that connects to the Net with a
T1
does not need BGP

> We are not yet at 1 computer per household member,

I actually have 7 IP connected computers in my two-
person household not to mention those that are too old
to be connected with IP.

> Add in PDAs and  gameboys via household bluetooth
> or 802.11, and I do not see how the "client/server"
> scenario you propose makes any sense.

I do not see where the client/server part is in what I
propose, could you devellop that part?

Michel.

---

From: Michel Py <[email-masked]>
Date: Thu, 6 Sep 2001 15:34:46 -0700 (PDT)

> be right for those users.
I have rarely seen these 802.11 networks reaching even
half of their rated
11 mbps speed. Besides, lots of them connect to a
T1...

Lets define "slow uplink":

- Anything less than a full T1 ?

(other proposals welcome)

---

From: "Michael H. Lambert" <[email-masked]>
Date: Thu, 06 Sep 2001 20:43:15 -0400

> http://arneill-py.sacramento.ca.us/draft-py-multi6-mhtp-01.txt

There is a class of sites for which all addresses are likely not created 
equal:  those with connectivity to both the commodity Internet and 
high-performance research and education networks.  For these sites, 
DNS-based solutions might not prove adequate because applications or even 
pairs of hosts expect to talk only over the high-performance networks; 
falling back to the commodity network isn't an option; it indicates failure 
of the connection.

Michael

+-----------------------------------------------------------------------+
| Michael H. Lambert, Network Engineer           Phone: +1 412 268-4960 |
| Pittsburgh Supercomputing Center               FAX:   +1 412 268-8200 |
| 4400 Fifth Avenue, Pittsburgh, PA  15213       [email-masked]        |
+-----------------------------------------------------------------------+

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Fri, 7 Sep 2001 10:17:26 +0200 (CEST)

On Thu, 6 Sep 2001, Michael H. Lambert wrote:

> DNS-based solutions might not prove adequate

Do these networks use two address ranges, one for the high performance
network and one for the regular network? If so, how is this handled in the
DNS today?

---

From: Greg Maxwell <[email-masked]>
Date: Fri, 7 Sep 2001 07:59:48 -0400 (EDT)

> maybe i need to be more explicit.  it is not clear to me that this is a

I think a more useful criterion is that we want a networks routing table
to grow O(n) with the number of peers and customers that they have (i.e.
ports) rather then O(x^2) or O(x log x) in the case of a meshed Internet
which is not highly aggregated, where x is the number of user networks on
the Internet not just the ones connected to you.

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Fri, 7 Sep 2001 15:08:16 +0200 (CEST)

> the Internet not just the ones connected to you.

None of these represent the current size of a BGP table. I don't think
it's possible to remove the x completely from the equation, but O(log x)
(which I think more or less represents a table with just the TLAs) is
pretty good too.

However, the current size of the routing table is not nearly as bad as you
say. It's something like O((pz + 1) * x) where p is the number of eBGP
peers for a router, x the total global number of routes and z the fraction
of the global routing table received from each peer. By decreasing the
number of routers, it is possible keep the number of entries in a router's
BGP table fairly close to 2x. Unfortunately, larger numbers of routers
mean increased processing.

A small routing table has disadvantages too: the routing process has to do
its job based on less information, so paths will be longer. We shouldn't
try to limit the actual size of routing tables in networks, but only make
sure the _minimum_ routing table necessary for connectivity is kept as
small as possible, however scenic the routes may get using this table.

It would also help if we could find a way to keep regional traffic
regional without announcing every single /48 individually. Here in Europe
we have some experience in transatlantic routing for regional traffic
(this was quite common five or six years ago) and I don't think many
people want it back.

---

From: "Michael H. Lambert" <[email-masked]>
Date: Fri, 07 Sep 2001 15:46:37 -0400

> network and one for the regular network? If so, how is this handled in the

Historically (in the era of isolated, expensive supercomputers), it was 
common to put the high-performance interface (called foo-hippi, for 
example) on a separate /24 in a site /16.  This subnet was often routed 
statically within the site and announced separately (for topological 
reasons) to the research network.  At the time that mechanism worked fine.

The current trend is toward highly distributed systems (eg, Grids) and a 
user community which probably has less of a grasp of the system as a whole 
than did the previous generation of users.  I think that under v6 it will 
be necessary to keep the Grid machines within the aggregates, although one 
could probably make a case for high-performance providers to send to (or 
accept from) peer providers aggregate-breaking announcements.

Michael

---

From: Geoff Huston <[email-masked]>
Date: Fri, 07 Sep 2001 19:52:23 +1000

>BGP algorithms.  Those folk are credible, but I haven't seen any

Again, we appear to be wandering off topic, but right now its not obvious 
where the scaling issues drive us into instability. While the precise 
number of prefixes is in an of itself not the major issue with scaling the 
routing system, its the 'density' of the topology over which BGP is 
attempting to operate, and the rate of dynamic change of the prefix set and 
associated AS path vector and related prefix attributes which are a 
concern. The suspicion is that these concerns are in some way related to 
the number of prefixes, and that a greater number of prefixes in the 
routing table may infer a greater density of interconnectivity, and an 
increase in the rate of dynamic change that in turn may cause BGP. as 
currently used to take longer to propagate changes and explore more 
'infeasible' routes on its path to convergence.

But as to the assertion that 8,192 is some magical preferred number of 
'root prefixes', then I'm not sure that I can agree.

The leverage which appears to offer some promise still appears to be the 
delineation in the V6 address space of 'identity' and 'routing goop' 
(GSE-styled approach) so that the routing goop can reflect, to the extent 
possible, implicit hierarchies in the inter-provider connectivity space.

(However it is still unclear to me, at any rate, that address hierarchies 
in the routing goop are going to provide 'natural' hierarchical aggregation 
of routing identifiers, but such a discussion is probably still off in the 
weeds with respect to the charter of this working group.)

    Geoff

---

From: Peter Tattam <[email-masked]>
Date: Sat, 8 Sep 2001 14:03:07 +1000 (EST)

> 'root prefixes', then I'm not sure that I can agree.

Geoff,

I am concerned that the whole multihoming issue hinges on the answer to whether
BGP can be made to work with larger DFZ than we anticipated.

I'm not sure where 8K comes from.  8K = 2^13. 

Maybe it relates to the size of a practical switching table in very fast
routers built using ASICs. 

Maybe it's just a figure which was plucked out of the air in relation to the
BGP table size at the time of the IPv6 standards being formed.

Maybe it's a figure designed to stabilize the core given the high bandwidths
we're likely to experience in the future.  We have probably gotten to the point
where we finally have enough capacity on inter site links and now RTT delays
will be governed

My impression however is that it has been generally agreed (amongst Ipv6 people
at least) that BGP in its current form isn't going to scale.

If I asked you the question "how big is too big" what would you answer, and how
would base your measure?

My understanding wasn't that it was hardware size limitations that we critical,
but more that the amount of information grows faster than O(n) resulting in
exchange of that information causing instability of the DFZ.  

In my opinion some of the measures that prevent route flapping may actually
cause more damage than good because they introduce stepping functions into the
differential equation that would model the DFZ.  They work now because the
paramters have been determined empirically - I'm uncertain that they will scale
indefinitely.

Peter

---

From: Michael Richardson <[email-masked]>
Date: Sat, 08 Sep 2001 13:48:28 -0400

-----BEGIN PGP SIGNED MESSAGE-----

>>>>> "Peter" == Peter Tattam <[email-masked]> writes:
    >> 'root prefixes', then I'm not sure that I can agree.

    Peter> I am concerned that the whole multihoming issue hinges on the
    Peter> answer to whether 
    Peter> BGP can be made to work with larger DFZ than we anticipated.

    Peter> I'm not sure where 8K comes from.  8K = 2^13. 

    Peter> Maybe it relates to the size of a practical switching table in very fast
    Peter> routers built using ASICs. 

  No.
  Currently shipping OC-48 capable ASICs do 100K+ easily (for IPv4). 
  Announced OC-192 ASICs do the same or more, and most designs have roadmaps
to OC-768, assuming that the OIF/NPF people finish defining appropriate
electrical interfaces to get packets between ASICs at that rate :-)
 
  Some solutions scale as the number of bits that matter. I.e. a single /128
entry simply takes twice as much space as two /64s, while other solutions
must store as many bits as the worst case length. The pathological situation
(highest price, highest power dissipation) is ternary CAMs which seem to do
72 bits or 144 bits only as options. 

  I too would like to know about this 8K DFZ.
  I can see that there are BGP convergence reasons why one might want to
attempt to keep the size small, but this number scares me.

    Peter> In my opinion some of the measures that prevent route flapping may actually
    Peter> cause more damage than good because they introduce stepping functions into the
    Peter> differential equation that would model the DFZ.  They work now because the
    Peter> paramters have been determined empirically - I'm uncertain that
    Peter> they will scale 
    Peter> indefinitely.

  That's an interesting observation.

---

From: Ramakrishna Gummadi <[email-masked]>
Date: Sat, 8 Sep 2001 12:31:47 -0700 (PDT)

> attempt to keep the size small, but this number scares me.

I may be wrong, but isn't this number from rfc2374? To quote from section
4.0:

   The size of the Top-Level Aggregation Identifier is 13 bits.  This
   allows for 8,192 TLA ID's.  This size was chosen to insure that the
   default-free routing table in top level routers in the Internet is
   kept within the limits, with a reasonable margin, of the current
   routing technology.  The margin is important because default-free
   routers will also carry a significant number of longer (i.e., more-
   specific) prefixes for optimizing paths internal to a TLA and between
   TLAs.

-ramki

---

From: Michel Py <[email-masked]>
Date: Sat, 8 Sep 2001 16:42:28 -0700 (PDT)

Ramki-

--- Ramakrishna Gummadi <[email-masked]> wrote:
>> I may be wrong, but isn't this number from rfc2374?
>> To quote from section 4.0:
>> The size of the Top-Level Aggregation Identifier
>> is 13 bits.  This allows for 8,192 TLA ID's. This
>> size was chosen to insure that the default-free
>> routing table in top level routers in the Internet
>> is kept within the limits, with a reasonable margin
>> of the current routing technology.  The margin is
>> important because default-free routers will also
>> carry a significant number of longer (i.e., more-
>> specific) prefixes for optimizing paths internal
>> to a TLA and between TLAs.

This is pretty much what I say in section 5.5.2 of
the -01 draft of MHTP:
(
http://arneill-py.sacramento.ca.us/draft-py-multi6-mhtp-01.txt
)

5.5.2. Simplified routing on high-bandwidth routers: 
Since backbone routers would no longer need to handle 
multihomed traffic, the IPv6 DFZ could be summarized
in the 
spirit that has guided its inception. To take the 
summarization to an unrealistically absurd level, the
IPv6 
backbone could be summarized at the /16 boundary, and
the 
6bone could be summarized at the /28 boundary without 
compromising the multihoming capabilities of MHTP.

I other words: Although the IPv6 DFZ could
THEORATICALLY
be summarized to the /16 boundary (which effectively
be
a 8k routig table when all 8192 TLAs have been
assigned)
the reality is that more specific prefixes are more
than
likely to be seen there.

Michel.

---

From: Peter Tattam <[email-masked]>
Date: Sun, 9 Sep 2001 14:04:00 +1000 (EST)

> 72 bits or 144 bits only as options.

Ok.  Point taken.  Flies in the face of the RFC recommendations mentioned in
other docs.

So we have the technology to make large routing crossbars.  I guess the issue
is what's feeding that crossbars.  Am I right in guessing that a lot of the BGP
stuff happens as a background process in the routers, and it's managing that
BGP mesh in a stable manner that's at issues.  Can one assume that technology
will keep track of the expansion of the internet necessary?

At issue is a big question as to whether BGP will scale *across the whole
network* . I haven't seen a clear informed answer to this yet.   Many of the
studies I've seen refer only to empirical studies and analysis of the current
v4 network and predictions of what is likely in the next year or so.

One problem that I see (correct me if I'm wrong) with the current v4 BGP system
is that *IF* there is a BGP storm it affects *everyone* whether they are
multihomed or not.  In other words, those sites which aren't multihomed are
penalized.   Stability of the core is then threatened merely by the act of
allowing multihoming via long prefixes.

It would be a very worthy post grad thesis to do a realistic simulation of the
BGP system to see exactly what happens when it is scaled up in size - there
must be one around I'm sure, but it might need a supercomputer to do it. 

I guess it comes down to fundamentals.  For a router based solution, you need
to fill the DFZ with all the possible MH information for every MH site in the
world and intuitively we think this can't possibly scale.  To date the ratio of
MH sites to non-MH sites has been low - kept that way because of availability &
cost. However with the advent of widespread always on connectivity and
bandwidth (e.g. Cable, DSL, 802.11) the potential availablility of MH is likely
to increase that ratio faster than would have been initially predicted.  The
concern is that even if a core router could switch fast enough & with what
seems a reasoanble number of routing table entries, the overengineering
required for a given router lifetime is difficult to predict with the current
BGP based MH solution.

The suggested IPv6 solution is to limit the choices that the DFZ routers need
to make and push the MH decision making to the edges.  By doing so, you throw
away the MH information visible at the core and you have to regain it another
way (which I've said before).

Once thing we do know scales reasonably well is that of DNS.  It has the job of
providing name to address mapping for almost every address on the planet and it
does it by virtue of localization of information & caching.  It has a few
bottle necks I suspect where there are heavily loaded domain names (e.g. .com) 
but it survives because it can be managed with technology which is cheaper han
routers and where memory cost integral is reducing at a much faster rate I
suspect than router memory and ASICs. 

So I can only think that to achieve the scalability we want, we still need to
work on decentralizing the MH decision making no matter how confident we think
that core routers are going to cope.

Finally I might add that I have to concede there is one fundamental flaw in the
host based or DNS based solutions to MH and that was eloquently pointed out by
someone else.  DNS needs routing so routing can't depend on DNS which means we
can't use it as it is currently implemented, primarily because DNS is agnostic
towards routing conditions.

This implies that you need a globally reachable sub system that is independent
of MH conditions which will then supply the MH information for any site based
or transport based MH solutions to function.   Very likely it could borrow
heavily from DSN, but it would need to be redesigned to be resilient to MH
conditions which I believe DNS is not.

My final conclusion is that we need a reachability caching system somewhat like
DNS but which is reachability aware.  Traditionally BGP has met this target but
this places the burden on routers to manage it.

An idea springs to mind - why not have an alternative system based on BGP
*independent* of the routing system from which reachability information can be
gathered.  Such a BGP system would need to be extended so that whole DFZ would
not need to be kept in the BGP server, but rather the localized routing needs
of the site instead, much in the same way that a DNS server only keeps the
localized name mapping that it requires for the sites current connections.

Such a system would meet the needs of keeping the *DFZ of core routers* small,
provide information for load balancing, retain a strong aggregation structure,
reduce instability of the core by forbidding advertising long prefixes.  I
believe it could be executed on hardware comparable to that which DNS uses.
Such a service could be referenced by node based multihoming solutions to
provide an accurate and timely source of information for such solutions.

If this has been thought of before I completely apologize to the original
proposer.

Peter

-- 
Peter R. Tattam [email-masked] Managing Director, Trumpet Software
International Pty Ltd Hobart, Australia, Ph. +61-3-6245-0220, Fax
+61-3-62450210

---

From: Ramakrishna Gummadi <[email-masked]>
Date: Sat, 8 Sep 2001 22:31:39 -0700 (PDT)

Hi Peter,

> localized name mapping that it requires for the sites current connections.

My draft
(http://www.ietf.org/internet-drafts/draft-ramki-multi6-nlmp-00.txt),
together with rfc2660, takes a somewhat similar approach by marking unusable
prefixes with a special BGP attribute and both localizing and
aggregating this unreachability information in BGP. It would be great if
you can tell me what you think of it.

thanks,
ramki

---

From: Peter Tattam <[email-masked]>
Date: Sun, 9 Sep 2001 16:37:09 +1000 (EST)

I have no desire to change existing BGP system - it will be needed to keep the
aggregated direct routing framework in place.  The key point I was trying to
make was decoupling of an alternative BGP cache from the routing
infrastructure.

From my reading of 

  http://www.telstra.net/gih/papers/ietf50-bgp.pdf

it becomes clear that multihoming is already a major degradation issue for the
Ipv4 network based on current BGP models.  I especially note the growth in the
longer prefixes (/24).

---

From: Ramakrishna Gummadi <[email-masked]>
Date: Sun, 9 Sep 2001 02:19:09 -0700 (PDT)

> Ipv4 network based on current BGP models.

My claim is that current BGP can be modified to both eliminate
separate multihoming entries and include the same
information that would be provided by a separate reachability cache.

-ramki

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Sun, 9 Sep 2001 13:28:44 +0200 (CEST)

> v4 network and predictions of what is likely in the next year or so.

Even if we assume the stability of new multihomers is the same of that of
current networks, the load on the BGP route processing system will
increase faster than the number of routes because each update needs to be
done on a larger table. This means order O(x^2) if doing an update scales
in a linear way or O(x log x) if it scales logarithmically.

> towards routing conditions.

Eloquent, but wrong. In the multi-address multihoming (MAMH) proposals
we've seen lately, there are never changes to single homed routing. So as
long as the domain name system only depends on single homed routing, there
is no problem. Also, no actual forwarding decisions depend on DNS
information, it is only used to find out addresses associated with a
hostname (or possibly with another address). This is hardly a new thing
for the DNS.

If we're going to do MAMH, we need three things:

1. initial address discovery: DNS already does this
2. address negotiation: mobile IP already does this
3. failover: this we don't have

Not a strict requirement, but it would be very helpful to have a choice
whether the end hosts or a proxy/router/NAT-like box does the necessary
multi-address multihoming processing.

> conditions which I believe DNS is not.

In the DNS, the information for each zone is replicated over two or more
servers. Rather than making sure you can always get at a certain server,
the DNS makes sure you can always get at the information by trying servers
until a live one is found. If all TCP/IP applications (well, the DNS is
not really an "application") did this, there would be no need for
multihoming support.

> localized name mapping that it requires for the sites current connections.

In BGP, all the routers in the path need to know a route. Since that can't
be a requirement here (back to BGP scaling) we're limited to multi-address
multihoming or tunneling/rewriting schemes.

So I would send a query along the lines of "what are all of the addresses
for a:b:c:d::1/128 and how reachable are they" to a root reachability
server. That one would probably return "a:b::1:2:3 is authorative for
a:b::/32" and assuming no more recursion a:b::1:2:3 would answer:

"alternative prefixes for a:b:c:d::/64 are:
a:b:c:d:: min RTT: 1700 us, bandwidth 1544 kbps, reliability 100%
d:e:f:9:: min RTT: 520 ms, bandwidth 44736 kbps, reliability 99%
5:6:7:8:: min RTT: 20 ms, bandwidth 128 kbps, reliability 93%
additional information:
d:e:f:9::1/128 reliability 0%"

meaning the site is connected over a T1, a satellite DS3 and ISDN dial-up,
and the host in question not being able to use the ISDN line at the
moment.

Iljitsch van Beijnum

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Sun, 9 Sep 2001 13:37:33 +0200 (CEST)

> information that would be provided by a separate reachability cache.

BGP can of course tell you that a certain prefix is _not_ reachable, but
it can not tell you that a specific address within that prefix _is_
reachable. Better BGP multihoming doesn't mean there is no place for
multi-address multihoming.

---

From: Daniel Senie <[email-masked]>
Date: Sun, 09 Sep 2001 08:05:48 -0400

>3. failover: this we don't have

4. a separate IP address prefix for each pipe from a site to the Internet, 
all of equal size.

This point seems to be missing from the discussion. While with IPv6 we like 
to think of sizing as infinite, we are busy dividing that space by 2 or 3 
to account for a large percentage of sites multihoming. That might be OK, 
or it might not.

This issue also raises some concerns in terms of whether the RIRs are 
willing to play along.

Clearly in an IPv4 world, assigning additional prefixes of equal size to a 
site is not going to go over well, and yet it's already an option, see below.

>multi-address multihoming processing.

There are multi-ported NAT Boxes on the market today which provide a flavor 
of multihoming with IPv4. This approach, with one-to-one NAT, can provide 
some level of service today to make a site "multihomed" in an IPv4 world. 
Note, however, the consumption of address space.

So let's keep in mind that when we make the routing table in the DFZ small 
using host-based multihoming, we're having a significant effect on the 
amount of IP address space we're causing to be allocated.

This may be a tradeoff we're willing to make. I have not seen anyone 
mention it in the arguments made here, which concerns me. As a community 
and standards body, we have not always done a good job of considering 
future impacts of standards promoted.

>not really an "application") did this, there would be no need for

And it would potentially take a LONG time to get connected to a web server, 
mail server, etc., if there were one or more dead machines on a list. Since 
it takes a while to get DNS changes propagated, that could continue for 
some time.

>and the host in question not being able to use the ISDN line at the

Are the DNS servers magically in a place where THEY are better connected? 
Arguably with this multi-connected site, the DNS servers could appear to be 
on separate networks, thus well protected. However, the time to find a 
responsive DNS server would have to be added to the other access times. 
Further the TTLs on the DNS data would need to be short, to reflect the 
changing nature of the data that you want to present. This can and is done 
today for load balancing, but the downside is excess DNS traffic and 
latency. Again, this may be a tradeoff we're willing to make. Let's just be 
sure we understand its effects on the whole.

>Iljitsch van Beijnum

-----------------------------------------------------------------
Daniel Senie                                        [email-masked]
Amaranth Networks Inc.                    http://www.amaranth.com

---

From: Peter Tattam <[email-masked]>
Date: Sun, 9 Sep 2001 22:19:44 +1000 (EST)

> hostname (or possibly with another address). This is hardly a new thing

Umm..  There is a pathological situation where a DNS server within the site may
not be accessible if that DNS server were only visible from a single address
prefix.  A secondary DNS might be visible at another address range but may not
be for similar reasons as the primary netowrk fault. You could multi home the
SOA & NS entries but it might complicate lookups - however you are implying
this is not done. The site could be black holed from the DNS system if it were
relying on a single homing their DNS servers, even if it were visible via other
prefixes.  It can be avoided by strategic placement of secondary servers, but
in practice it might prove difficult.

What I'm saying is that the DNS needs multihoming to work for it to be
fully reliable.  Do you still disagree?  Show me what I'm missing here.

I'll respond to the rest of your message when I've digested it.

Peter

---

From: Peter Tattam <[email-masked]>
Date: Sun, 9 Sep 2001 22:32:12 +1000 (EST)

> not really an "application") did this, there would be no need for

My other message addresses this. I'd like to add that I'm not sure I agree with
you that this is adequate.  Also, because I believe that DNS is agnostic
towards routing, in practice if I assume you would MH SOA & NS entries, you
might find DNS lookups slow down if random NS entries were used to locate DNS
information if one MH path was less optimal than another.  You might not get
black holed, but DNS resolution for the site would end up being degraded in the
presence of multihoming.

Can you see my reasoning?  For DNS to be efficient or even work at all, it has
to be MH aware to do its job.  It's circular and hence I believe there is a
problem.

There is a circular issue in the current DNS system, but we resolve it through
the use of geographically distinct secondary servers and gluing in the
delegation via a fixed IP address. This works because the routing system always
provides a path to the backup server in the instance that the primary DNS is
down.  I'm not so sure it will be so simple in a multi prefix environment.

Peter

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Sun, 9 Sep 2001 16:03:54 +0200 (CEST)

On Sun, 9 Sep 2001, Peter Tattam wrote:

> be for similar reasons as the primary netowrk fault.

This happens without multihoming too. The primary and secondary servers
should not both go down when there is a network problem.

> You could multi home the
> this is not done.

I would just give the name server in a MAMH network two addresses, and
list both addresses in the delegating zone. That way, name service for a
dual-homed network with two name servers would be available from four
addresses. When an address is unreachable queries to that address will
time out and have to be tried again at another address. This adds a delay
to the process, which is not good but not so big a problem that we have
to attach any consequences to it, IMO.

> The site could be black holed from the DNS system if it were

Today, it is possible to multihome a server without any changes to any
protocol. This means giving the server addresses from ISP A and from ISP
B. This works. The problem is: what happens when one of them is down? This
problem can be fixed by trying different addresses until you find one that
works. Telnet does this, BIND does this. Another problem is: when
something goes down in the middle of a session, what do you do? This is
not a problem for the DNS, since it mostly uses single request/reply
transactions over UDP without keeping sessions active and it retries
everything so even with TCP it works out in the end.

What we're trying to do is not make multiple-address multihoming as such
possible: it already is. What we want is mechanisms to hide IP and lower
layer failures from the transport layer.

Iljitsch

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Sun, 9 Sep 2001 16:16:01 +0200 (CEST)

On Sun, 9 Sep 2001, Peter Tattam wrote:

> Also, because I believe that DNS is agnostic

For a single query this might be a problem. But unless I'm mistaken, BIND
takes note of the round trip times it sees and (mostly) prefers the faster
address over the slower one on subsequent queries.

---

From: Daniel Senie <[email-masked]>
Date: Sun, 09 Sep 2001 10:37:28 -0400

>address over the slower one on subsequent queries.

Be a bit careful here, or you fall into a trap that some of the DNS load 
balancer vendors fall into. You care about the distance/latency between the 
RESOLVER and the target machine, NOT between the RESOLVER's Name Server and 
the target's name server. Several load balancer products ensure the latter 
case is not a problem, however, they do not address the issue of proximity 
of the name server doing the lookup on behalf of the user/service/whatever. 
The recursive name servers used when making requests are NOT required to be 
topologically close to the machine requesting the lookup.

I bring this up just so we don't go down any paths that could lead to bad 
assumptions about DNS resolution as a cure-all.

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Sun, 9 Sep 2001 16:41:33 +0200 (CEST)

> >3. failover: this we don't have

Hm, I might have another one: sending traffic with a source address from
ISP A over a link to ISP B can be a problem.

> to account for a large percentage of sites multihoming. That might be OK,

That's one or two bits out of 128. Considering that 16 bits are wasted in
the IEEE -> EUI-64 conversion, I don't see the problem in IPv6.

In IPv4, things are different, of course. Still, a single host could have
several addresses if this serves an important function, I think.

> This issue also raises some concerns in terms of whether the RIRs are

They might not like all of this, but for the opposite reason: when just a
hand full of ISPs need address space and AS numbers, that means less work
for them.

> site is not going to go over well,

Agree.

> Note, however, the consumption of address space.

Aren't current sessions still lost with current NAT?

> future impacts of standards promoted.

You are right, we should pay attention to this. Michel Py talks about this
in section 6.2.4 of his draft.

> responsive DNS server would have to be added to the other access times.

That would be a good reason to incorporate this in the current DNS, since
we already have to wait for the name to address translation. If the other
information is then returned as well, this would save time.

> changing nature of the data that you want to present.

I do not propose to store data that needs to be changed often in the DNS.

Iljitsch van Beijnum

---

From: Daniel Senie <[email-masked]>
Date: Sun, 09 Sep 2001 10:57:10 -0400

>ISP A over a link to ISP B can be a problem.

Especially if folks are doing ingress filtering. Proper address selection 
(and figuring out how to decide) is the hardest part of this to do in 
hosts, unless the hosts have a lot of information pushed to them.

>the IEEE -> EUI-64 conversion, I don't see the problem in IPv6.

Though the use of MAC address is now an optional thing, due to privacy 
concerns. Yes, there's a large block of addresses. The concern is that we 
not give away too many bits for one thing or another. We also have the 
issue that all providers to a site will need to supply prefixes that 
essentially match (i.e. same sizes from each, so that things overlay 
cleanly). The RIRs would need to adjust their policies to permit this.

>several addresses if this serves an important function, I think.

It does, but would 2 or 3 different upstreams each provide a block of 
addresses large enough for a web server farm? At what point is the 
consumption of addresses considered more expensive than handing the site an 
AS number, and allowing them to advertise their blocks?

>Aren't current sessions still lost with current NAT?

With a Multi-homed NAT solution, it's possible to have sessions load 
balanced, as the box doing the NAT is a router connected to all possible 
exits. The box can decide which link to connect out on, and thus outbound 
connections are cleanly load balanced (and the returning packets for those 
links will follow the same path back in). For inbound traffic to a site, 
load balancers (e.g. DNS tricks and such) permit balancing between the 
various links. The two functions can be handled by the same equipment, so 
that line utilization can be managed for each link.

One of my concerns is that these boxes (available today) will look more 
attractive to many people than a migration to IPv6. Indeed, if a site can 
get multiple providers to supply enough address space, then for many cases 
this is an efficient and cost-effective way to get into multihoming. 
Certainly such boxes could be made to play in the IPv6 space as well, or do 
6to4 in the course of their work.

>information is then returned as well, this would save time.

Other than packet size limitations with DNS over UDP, or overhead if 
everything spills over to TCP.

>I do not propose to store data that needs to be changed often in the DNS.

Unfortunately, one of the things this type of multihoming lends itself to 
is load balancing manipulation. By altering the order of the addresses in 
responses (i.e. doing something other than round robin) is a pretty 
effective way of altering balancing. Setting TTLs very low makes this 
possible. If we go down this path, people WILL be setting their TTLs very 
low. Some will argue for ignoring the TTLs and caching longer, but that'll 
create significant problems for many applications, and likely increase 
support costs in the long run.

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Sun, 9 Sep 2001 19:11:04 +0200 (CEST)

> cleanly). The RIRs would need to adjust their policies to permit this.

I haven't followed the latest developments, but I think the plan is to
give each organization its own /48 and each network, however small (even a
dial-up user) its own /64. So the blocks are always the same size, that
helps. Don't underestimate the huge address space of IPv6: even if you're
multihoming within each TLA (costing 16 bits) with a /48 there is room for
4 billion multihomers.

> AS number, and allowing them to advertise their blocks?

As it is now, you can only be sure that you won't be filtered with a /20.
That means 1300 triple-homed web servers...

[Load balancing with NAT]

> 6to4 in the course of their work.

Essentially you're just making renumbering easier with these boxes: they
don't help much when a line goes down. I wouldn't call this multihoming.
I'm certainly not a big fan of NAT, but it does help preserve address
space in IPv4. Load balancing using NAT is an even bigger hack that
regular NAT and has little to do with multihoming: you're trying to
balance the load over the servers, not over the lines.

> low.

If we depend on the DNS to only deliver the "best" address. What I propose
is that we use the DNS to deliver all addresses and find out which ones
work and which don't separately. This way, there is no harm in keeping a
broken address in the DNS and the TTL doesn't have to be especially low.

But even if people use low TTLs for things like load balancing, that's not
necessarily evil. Only the A records for the server in question need to
have a low TTL, the rest of DNS tree can have a regular TTL. This means
the resolving name server can directly contact the destination name
server, so only the endpoints experience the higher load. Presumably, the
resources needed for a single DNS query/reply will be dwarfed by the
subsequent "real" communication that takes place.

> Some will argue for ignoring the TTLs and caching longer, but that'll

I don't see the problem... Why would a low TTL for "leaf" records be such
a big problem that people will want to break protocols? Obviously we want
high TTLs for NS records and such.

From another message:

> balancer vendors fall into. You care about the distance/latency between
> the RESOLVER and the target machine, NOT between the RESOLVER's Name Server
> and the target's name server. Several load balancer products ensure the
> latter case is not a problem, however, they do not address the issue of
> proximity of the name server doing the lookup on behalf of the
> user/service/whatever.

By resolver you mean the resolver library on the host that wants to set up
a session?

The latency between a host and the caching name server it uses must be as
small as possible, since applications such as WWW need to get at the
caching name server a lot, and usually the user is waiting during that
time. Also, in many cases this information is not cached locally, so the
same information may be requested many times.

> The recursive name servers used when making requests are NOT required to
> be topologically close to the machine requesting the lookup.

They are in my book.  :-)

> I bring this up just so we don't go down any paths that could lead to
> bad assumptions about DNS resolution as a cure-all.

Yes, we should be careful with the domain name system. On the other hand,
we don't want to prematurely close off paths that could lead to the
multihoming walhalla...

Iljitsch

---

From: Daniel Senie <[email-masked]>
Date: Sun, 09 Sep 2001 14:50:21 -0400

>That means 1300 triple-homed web servers...

No, you misunderstand.

With a multi-port NAT setup, you use ISP-provided space from each provider. 
There's no BGP, no ASN, no advertisements, and no filtering, 'cause you're 
within someone's large block on each of the links.

>don't help much when a line goes down. I wouldn't call this multihoming.

Wrong. These boxes work QUITE WELL in the event of a line outage. Outbound 
connections from the site which were in progress over the dead link 
obviously die, but new connections will go only over the functioning links. 
With reasonable DNS-based load balancing and short TTLs, the inbound 
connectivity will be relatively unaffected as well.

>balance the load over the servers, not over the lines.

I'm not a big fan of NAT either, despite having implemented it and done a 
fair bit of writing about it, but it CAN be useful, and this "multihoming" 
approach using NAT is fairly workable. Products are on the market which do 
this already.

>broken address in the DNS and the TTL doesn't have to be especially low.

Ah, but if you give out more than one address, but specifically manipulate 
the order of the entries given out, and keep the TTLs low, it's possible to 
get a degree of load balancing at the same time.

>subsequent "real" communication that takes place.

Assuming the DNS and the actual traffic are going in the same neighborhood, 
which admittedly is the most common case.

>high TTLs for NS records and such.

Agree, people shouldn't do these things, but some appear to. Some 
applications cache DNS data, for example, which is a concern. Such caching 
should be limited to the timeouts specified, but will applications bother? 
A better resolver API would probably be a good thing.

>By resolver you mean the resolver library on the host that wants to set up

Yes.

>same information may be requested many times.

Web browsers cache DNS info for at least short periods. If they didn't 
browsing over a dialup would probably be painfully slow. There are 
instances where they'll be far away. While having the name server close 
speeds things up for the user or server making the request, having that 
topological closeness ALSO be used to try to determine best site for load 
balancing and other such games is a concern.

> > be topologically close to the machine requesting the lookup.

Hope you don't have to use VPNs much. That's one of several cases where the 
name servers can wind up being VERY far away.

>we don't want to prematurely close off paths that could lead to the

My concern is that we don't choose paths prematurely that may result in 
other troubles. DNS has been our kitchen sink for a long time, and while 
it's done well, some of the ways it's being used are problematic, and the 
situation could get worse. Let's consider these approaches but be careful.

---

From: Michel Py <[email-masked]>
Date: Sun, 9 Sep 2001 16:57:37 -0700 (PDT)

Daniel -

>> Hm, I might have another one: sending traffic with
>> a source address from ISP A over a link to ISP B
>> can be a problem.

There is a MAMH version of MHTP that is exactly what
you and Ilsitch are talking about (NAT based, initial
resolution based on DNS and it does have failover,
too). I discarded it for two reasons: it uses DNS
and what you mention just above. The idea is
technically workable as long as you accept that
the system is far from being perfect, but it has not
a single chance of political acceptance.

>> There are multi-ported NAT Boxes on the market
>> today which provide a flavor of multihoming with
>> IPv4. This approach, with one-to-one NAT, can
>> provide some level of service today to make a site
>> "multihomed" in an IPv4 world.

You can configure it with a Cisco router using
reflexive access-lists, but the load balancing implies
a hack on the DNS server.

>> One of my concerns is that these boxes (available
>> today) will look more attractive to many people
>> than a migration to IPv6.

This is not the place to discuss it, and it remains
to be seen. I can tell you quite frankly that I am
familiar with that configuration and that it would
easily integrate with MHTP and I have decided not
to make a fool of myself by waiting to see if it
pumps up speed in the IPv4 world first. If it does,
I'll publish it.

Michel

---

From: Michael Richardson <[email-masked]>
Date: Mon, 10 Sep 2001 16:03:35 -0400

-----BEGIN PGP SIGNED MESSAGE-----

>>>>> "Peter" == Peter Tattam <[email-masked]> writes:
    Peter> The suggested IPv6 solution is to limit the choices that the DFZ routers need
    Peter> to make and push the MH decision making to the edges.  By doing so, you throw
    Peter> away the MH information visible at the core and you have to regain it another
    Peter> way (which I've said before).

  Let me reword this:
      push the MH decision away from the DFZ

  The thing about IX based solutions with geographically significant
addressing is that it pushes the problem down to the IX and municipal area
network. 

  My preferred solution is to use geographically significant addresses as the
PI address format, but to then use some kind of address exchange mechanism
inband (a la what you have described).

  I am not clear why this "inband" system is any different than MobileIP,
nor why it should be secure when MobileIP is not. My preference is therefore
to use either Binding Updates or a minor variation of them to do this. 

  Geographically significant addresses may never get optimal routing. 
  It may even be necessary to configure tunnels to geographical address
concentrators to get the first packet through. 

  (This is easily done with IKE and IPsec and scales rather well as the
relationship with the address concentrator must be pre-existing. I would
further suggest that it is entirely appropriate that these system be a
municipally governed service, but that is a local jurisdiction issue)

  If I wanted to put a web server up in this scenario, I'd give
www.example.com a geographically significant address in DNS. I'd could then
have references to the rest of the site be www-ispA.example.com or some such
with a PA address that leads to the appropriate place, or just depend upon
each host having some kind of PI->PA cache.

  [This can wreck havoc with long term bookmarks, but many of these web sites
are entirely generated anyway, and bookmarks fail here anyway. Putting a URL
in that is appropriate for "bookmark"ing is probably a todo for W3C anyway]

    Peter> Finally I might add that I have to concede there is one
    Peter> fundamental flaw in the 
    Peter> host based or DNS based solutions to MH and that was eloquently pointed out by
    Peter> someone else.  DNS needs routing so routing can't depend on DNS which means we
    Peter> can't use it as it is currently implemented, primarily because DNS is agnostic
    Peter> towards routing conditions.

  While I would agree that we would like to have the root, and TLD servers
multihomed (and in particular, we'd like to have a stable set of addresses to 
put into named.ca), DNS has some rather nice features for discovering the
entire list of addresses of nameservers already. The first thing that one
usually does when a DNS server starts is to ask the root servers for a list
of root servers. They could easily return all of their PA.

  So, while I would agree that having a *routing* system that depends upon
DNS is a deadlock, having a *multihoming* system that depends upon DNS may
not be a loss.
  I would prefer that we did not rely on DNS however.

    Peter> An idea springs to mind - why not have an alternative system based on BGP
    Peter> *independent* of the routing system from which reachability information can be
    Peter> gathered.  Such a BGP system would need to be extended so that whole DFZ would
    Peter> not need to be kept in the BGP server, but rather the localized routing needs
    Peter> of the site instead, much in the same way that a DNS server only keeps the
    Peter> localized name mapping that it requires for the sites current connections.

  Like a geographical IX system?

    Peter> If this has been thought of before I completely apologize to the original
    Peter> proposer.

  I think it is in one draft.
  Hmm. Now I don't know which one! We need a better list of drafts under
consideration.