Subject: Re: On the use of multiple PA prefixes or a single PI prefix for IPv6 multihoming

From: [email-masked] (Noel Chiappa)
Date: Fri, 22 Oct 2004 10:09:41 -0400 (EDT)

> From: Iljitsch van Beijnum <[email-masked]>

    > I wasn't aware that I'm so set in my ways that people can predict
    > whether I'll like something in advance...

It's not just you, almost *nobody* likes it! So I'm merely going with the
odds! :-) Although I admit you seem to not be afraid of offbeat ideas, so you
may well like it at a purely technical level. Although I predict you will be
daunted by the magnitude of the change... :-)

    > So let's hear it, I'm interested to know if you're right. (-:

Well, basically, I think we need to move the "path selection" function
out of the "network" (i.e. the routers), and let the hosts (or, if they
are lazy/stupid, an agent acting on their behalf) do it. To do this, they
obviously need more information, which I claim should be maps of network
topology.

I have two standards points to make to people who roll their eyes at this
hopeless technological naivete, and say that it's "obviously" way too
complex a job to give to the hosts:

- Back in the 1970's, the "network" took care of flow/congestion control and
also reliability, and now hosts do all this - using some fairly complex
algorithms that took us decades to get working well. And having made that
fundamental architectural reorganization, it's now "obvious" to everyone that
that's the way it ought to be.

- This is the way we do cars and highways, and it seems to work just fine.

Anyway, making this (admittedly major) architectural change kills about
17 dozen birds, of which the "how do I pick the best address for the
destination" is one.

And yes, I know this is a really, really, *really*, ***REALLY*** big
change. But, surprisingly, when you start working through the details,
there are ways to incrementally deploy it, ways which don't mandate
changes to all hosts before it can begin to be useful.

	Noel

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Fri, 22 Oct 2004 22:07:57 +0200

> daunted by the magnitude of the change... :-)

Hm...

> obviously need more information, which I claim should be maps of 

Source routing, you mean?

As Marcelo explained, we are already on the path towards some very 
loose source routing (only the exit ISP) if we let hosts select 
source/destination address combinations and in the internal network 
there is source address based routing.

> - This is the way we do cars and highways, and it seems to work just 

Well, based on these two arguments (especially the second) the natural 
progression here would be "source routering" where every packet 
contains the code to forward itself and routers are just big fat java 
virtual machines.  :-)

I'm not really sold on the car analogy, but I agree that fear of 
complexity in hosts in unfounded. Typical hosts have tons more complex 
algorithms on board than typical routers. However, we do want hosts to 
be simple with regard to the amount of configuration information they 
must receive before they can go about their business: it's not a good 
idea to have to load a full BGP table or similar topology map before a 
host can communicate, or to have to upgrade all hosts in order to fix 
interdomain routing problems (of which we have plenty).

> 17 dozen birds, of which the "how do I pick the best address for the

I doubt it. There is just too much information to be able to make 
optimal internet-wide routing decisions before the first packet is 
exchanged.

But that's not a big deal. If you have multiple paths, you can just add 
some multipath logic with per-path windows to TCP and the like and 
suddenly you're using all paths to capacity.

(I was involved with implementing something where real-time traffic was 
"anycast" over several TCP streams towards different destinations. 
(Don't ask why we used TCP.) We were supposed to spread traffic 
randomly over the different TCP sessions. This was a disaster as we 
were now limited by the slowest. But simply skipping the sessions that 
were blocked because of a full window allowed us to use the combined 
bandwidth of all sessions.)

> changes to all hosts before it can begin to be useful.

Well, we're incrementing things now for multihoming support so this 
might be a good time to get some of this stuff in...

---

From: Pekka Nikander <[email-masked]>
Date: Sun, 24 Oct 2004 11:26:40 +0300

Noel,

This starts to be off-topic for the multi6 list, but...

> ...

... I'd like to hear what are some of these 17 dozen birds.

I'm admittedly a layman in routing, but thinking about the amount
of routing information and its stability, it appears to me that it
will anyway be necessary to provide aggregated versions of this
information at various levels.  Hence, it sounds reasonable to
assume that a host could have some better knowledge about the network
topology (and perhaps congestion state) "close" to it, but less
information, or more aggregated information, about what happens
further away.  Hence, any path selection performed by the end hosts
will be, at best, an approximation.

I could imagine that two (or a group of) communicating hosts could
jointly be able to make a fairly good approximation of the routing
state close to them, and thereby make a good selection about which
local paths to use among the available ones.  Combining this information
with upper layer RTT and bandwidth estimates, the hosts might be
able to create some kind of a local model about the traffic
characteristics of the paths, but only over time.  In the case of
site multi-homing, sharing this information with other hosts on the
site may make sense, just as a number of people have pointed out.

Based on this straw man analysis, I get the feeling that moving
path selection to the end hosts might be good because

   - the end hosts may use information from their peers to make
     better selections, and

   - the end hosts may more easily use dynamic upper layer
     congestion information to revise their choices on need

On the other hand, either network assistance or letting the network
to do the selection looks good because

   - the network may learn more from a number of end-hosts,
     thereby being able to create a more accurate model of
     the network condition

   - the network needs to have this topology information anyway,
     as we cannot expect the hosts to learn the topology without
     some help from the routers

The situation seems to be become more interesting if we assume that
there is a full blown id/loc split in place.  That would allow
the network to have more dynamic IP addresses, slowly changing the
addresses to better match the actual network topology.  Under such
conditions, it becomes an interesting exercise to figure out how
to represent the actual topology, as IP addresses cannot be directly
used to represent nodes, and identifiers are unlikely to be
directly aggregatable.

Anyway, I fail to clearly see the consequences of such a change,
i.e. moving "path selection" to end hosts, and would like to hear
more about your insights.

--Pekka Nikander

---

From: Brian E Carpenter <[email-masked]>
Date: Mon, 25 Oct 2004 11:04:53 +0200

> This starts to be off-topic for the multi6 list, but...

It is a bit, but "choose the new source/dest address pair" is
one of those separable components in any multi6 solution. As in
my previous message, I think we should concentrate on the interfaces
between those components, since one size probably doesn't fit all,
and topology maps won't appear overnight.

     Brian