Subject: RE: PI/metro/geo [Re: The state of IPv6 multihoming development]

From: "Tony Li" <[email-masked]>
Date: Fri, 1 Nov 2002 00:37:34 -0800

Tony,

This is blatantly unfair.  As an "arms merchant", I fully want
to sell gear to all interested parties, both ISPs and enterprise
alike.  And, as a certain AD is fond of reminding me, the enterprise
is a very big market.  Why would I want to torque the enterprise?

Instead, I want to help all parties by growing the net in a 
reasonable and scalable way.  That should be what we're all trying
to do here.

The operation issues that I know of NEVER favor restricting topology.
In fact, restricting topology is the result of geographic aggregation
as it forces providers to interconnect within the aggregation boundary.
Operational issues instead favor a haphazard, need driven, cost
optimized topology.  Yes, ok it's chaotic, but it's the result of
capitalism at its finest.

Provider independence isn't necessarily helped only by PI space.
There are other ways.  The real point is to avoid the pain and
anguish of renumbering.  If we were to go down the GSE path, for
example, changing providers simply means changing the "routing gorp".
The host identifier remains the same.  You reconfigure your border
router(s) and you're done.  Infinitely less painful than dealing with
the boring things like acutally bringing up a new tail circuit.  ;-)

You are also assuming that aggregation and multihoming are somehow 
at odds.  They are not.  All you have to do is to disconnect the
locator and the identifier as GSE does and aggregate only the 
locators.  Each host in a multihomed domain has a single identifier,
but multiple locators.  No deaggregation happens because the locators
are bound to the provider and thus we have good topological
'addressing'.  The enterprise doesn't lose because we tweak the
transport to base the pseudoheader on the host identifier and then
let the locator be free to be replaced by border routers.  Connections
flip back and forth between locators easily.

How does this not solve the (non-ideological) issues at hand?

Tony

|   -----Original Message-----
|   From: Tony Hain [mailto:[email-masked]]
|   Sent: Thursday, October 31, 2002 11:07 PM
|   To: 'Randy Bush'
|   Cc: Tony Li; [email-masked]
|   Subject: RE: PI/metro/geo [Re: The state of IPv6 multihoming
|   development]
|   
|   
|   Randy Bush wrote:
|   > </ad hat>
|   > >> We need to decide which is more important: the scalability and 
|   > >> durability of the routing subsystem or the convenience of 
|   > >> non-connection based addressing.  When we have 
|   consensus on this 
|   > >> point, then all else will follow.
|   > > The answer to the question depends on which side of the 
|   > table you are 
|   > > sitting on.
|   > 
|   > there is little doubt in my mind which side i sit on.  
|   > interesting that tli seems to be on the same side.  but then 
|   > he always kinda groked the operational issues.
|   
|   Operational issues in the ISP space have always favored restricting
|   topology or the knowledge about unaggregated parts thereof. 
|   At the same
|   time, operational issues in the edge enterprise space favor provider
|   independence for maximum flexibility. We have 
|   representative voices for
|   the ISP issues, where are the comparable voices for the enterprise
|   issues? The ISP focus carried round 1, so the allocation policy only
|   allows for strict aggregation via provider blocks. The 
|   enterprise demand
|   for multi-homing is the fundamental issue this WG is supposed to
|   address, but the primary voices are those insisting on maximal
|   aggregation for the ISP routing system. It is hard to 
|   believe this will
|   end up with a well balanced result that considers all the 
|   requirements. 
|   
|   Speaking of, we have a requirements doc that needs to get 
|   published. 
|   
|   Tony
|   
|

---

From: Masataka Ohta <[email-masked]>
Date: Fri, 1 Nov 2002 18:32:08 +0859 ()

Tony Li;

> is a very big market.  Why would I want to torque the enterprise?

As a person who is involved in design and operation of a nation
wide 10Gbps backbone of a commercial ISP, I can say that TE is
to have more bandwidth.

Others may have different diffinition. But, those with high cost
definition such as MPLS will, or has already, bankrupt.

It's called fair competition and is applicable to non-ISP
enterprises.

Make routers dumb and cheap.

> If we were to go down the GSE path, for
> example,

:  Note that 8+8 proposal must not be confused by latter proposal of
:  routing goof by Mike O'dell, which is a proposal to use intelligent
:  routers to rewrite source addresses to prevent source address
:  spoofing and to tunnel between intelligent routers for pseudo
:  multihoming, both of which are against the end to end principle and,
:  thus, lacks robustness and/or scalability.

						Masataka Ohta

---

From: "Tony Hain" <[email-masked]>
Date: Fri, 1 Nov 2002 10:08:23 -0800

Tony Li wrote:
> Tony,
> 
> This is blatantly unfair.  As an "arms merchant", I fully 
> want to sell gear to all interested parties, both ISPs and 
> enterprise alike.  And, as a certain AD is fond of reminding 
> me, the enterprise is a very big market.  Why would I want to 
> torque the enterprise?

I think you read more into that note than was there. It was not an
attack on anyone, I was simply observing that the recent discussion was
focused on core routing issues.

> reasonable and scalable way.  That should be what we're all 
> trying to do here.

I believe it is, I was just asking if we have the appropriate
representation to make a balanced choice when the hard trade-offs start
being made.

> The operation issues that I know of NEVER favor restricting 

Favoring is not the same as enforcing. 

> In fact, restricting topology is the result of 
> geographic aggregation as it forces providers to interconnect 
> within the aggregation boundary. 

I am not trying to promote any particular solution, just make sure the
requirements of the edge are represented.

> Operational issues instead 
> favor a haphazard, need driven, cost optimized topology.  
> Yes, ok it's chaotic, but it's the result of capitalism at its finest.

It is the 'cost optimized' part that makes the difference. Which pockets
are we talking about protecting? I am not saying we should favor any
particular pocket, just asking if we are doing it unintentionally by the
representatives around the table.

> 
> Provider independence isn't necessarily helped only by PI 
> space. There are other ways.  The real point is to avoid the 
> pain and anguish of renumbering. 

That is one reason; simply being in control of your own destiny is
another. 

>  If we were to go down the 
> GSE path, for example, changing providers simply means 
> changing the "routing gorp". The host identifier remains the 
> same.  You reconfigure your border
> router(s) and you're done.  Infinitely less painful than 
> dealing with the boring things like acutally bringing up a 
> new tail circuit.  ;-)

This argument is continually shot at using the multi-prefix model as not
workable because there are too many static tables inside enterprises
that 'need' to carry the real address for access control. I am not
arguing it is valid or correct, just that the viewpoint exists.

> at odds.  They are not.  All you have to do is to disconnect 
> the locator and the identifier as GSE does and aggregate only the 
> locators.  Each host in a multihomed domain has a single 
> identifier, but multiple locators.  No deaggregation happens 
> because the locators are bound to the provider and thus we 
> have good topological 'addressing'.  The enterprise doesn't 
> lose because we tweak the transport to base the pseudoheader 
> on the host identifier and then let the locator be free to be 
> replaced by border routers.  Connections flip back and forth 
> between locators easily.

This model was probably possible 4 years ago, but at this point I doubt
tweaking the pseudoheader can even be on the table. Also, 'freely
replaced' usually sets off the alarm bells of the spoof-sensitive
security types. That does not mean we GSE is hopeless, just that we will
probably have to use something like MIPv6 to mask the label swapping. In
the abstract, there is not much difference between GSE and a Care-of
Address. So for routing & packet forwarding, if the CoA had finer
grained pattern replacement happening the end systems would not be aware
of it. The missing link would be letting the host know what the current
publicly visible address is so it could pass that to the CN.

Tony

---

From: RJ Atkinson <[email-masked]>
Date: Sat, 2 Nov 2002 07:28:38 -0500

> representatives around the table.

All the pockets.  Trying to shift costs onto ISPs just means they
are forced to raise rates and shift them back to customers.  Terribly
sorry, but that's how the economics work in this case.  And the
part I strongly dislike is that those raised rates propogate much more
widely than the subset of sites that cause the ISP costs -- and harm
the general growth and health of the global Internet.

> tweaking the pseudoheader can even be on the table.

If router vendors had a knob that enabled separation of 
location/identity
and the IDR infrastructure was compatible with that, I bet a whole lot
of enterprise users would enable that knob eagerly in order to obtain
multi-homing and backup and other things they desire.

It might be interesting to run that experiment, though obviously the
implementation specifics would need to be worked out first.

> Also, 'freely
> security types.

I've been accused of Internet paranoia in the past.  Please note that
I disagree with the assertion that such paranoia is reasonable in the
case of separating location from identity.  Unreasonable paranoia
exists no matter what one does, so ought not be a variable in the
decision process.

> That does not mean we GSE is hopeless, just that we will
> probably have to use something like MIPv6 to mask the label swapping.
> In the abstract, there is not much difference between GSE and a Care-of

I'm happy to discuss whichever implementation details.  I just think 
that
from an architectural perspective, we need to separate location and 
identity
to make any significant headway here.

Ran

---

From: Masataka Ohta <[email-masked]>
Date: Sun, 3 Nov 2002 08:49:47 +0859 ()

Ran;

> If router vendors had a knob that enabled separation of 

Separation of locatioin/identity itself does not require any
modification on routers.

Poor GSE does.

							Masataka Ohta

---

From: RJ Atkinson <[email-masked]>
Date: Mon, 4 Nov 2002 11:22:02 -0500

On Saturday, Nov 2, 2002, at 18:50 America/Montreal, Masataka Ohta 
> Separation of locatioin/identity itself does not require any

Please point us towards your online I-D that describes the
above approach.

Ran

---

From: Masataka Ohta <[email-masked]>
Date: Wed, 6 Nov 2002 06:39:35 +0859 ()

Ran;

> Please point us towards your online I-D that describes the

As I revised my ID just before the deadline, it is attached.

The revision is on source address selection as follows:

   Note that policy based selection of a source address is incompatible
   to the end to end principle, because it must be selected as the
   destination address by the policy of the other end of the
   communication using information local to the other end of the
   communication.  With so much asymmetric routing of the Internet
   today, proper destination address to reply can not be guessed by the
   querier.

   Thus, DNS query should be modified to carry all the addresses of
   clients and servers should try from the most favorable to the least
   ones, based on locally available information such as metric in intra
   domain full routing table.

   Any source address may be selected.

   However, to enable source address filtering to discard packets with
   source addresses not belonging to an ISP, it is useful to enable a
   host, not some intelligent intermediate router, select a source
   address compatible with an outgoing ISP.  For that purpose, intra
   domain routing protocols should maintain routing table entries with
   not only preference values of an external routes, but also proper
   prefixes to be selected for source addresses, if the entries are
   chosen by a host.

						Masataka Ohta
--

INTERNET DRAFT                                                   M. Ohta
draft-ohta-e2e-multihoming-03.txt          Tokyo Institute of Technology
                                                           November 2002

               The Architecture of End to End Multihoming

Status of this Memo

   This document is an Internet-Draft and is subject to all provisions
   of Section 10 of RFC2026.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as Internet-
   Drafts.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet- Drafts as reference
   material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/1id-abstracts.html

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html

Abstract

   This memo describes the architecture of end to end multihoming.  End
   to end multihoming does not burden routing system for multihoming.
   That is, even extensive use of end to end multihoming does not
   increase the number of entries in a global routing table.

   Traditionally with IPv4, multihoming capability is offered by an
   intelligent routing system, which, as is always the case with
   violating the end to end principle, lacks scalability on a global
   routing table size and robustness against link failures.

   On the other hand, with end to end multihoming, multihoming is
   supported by transport (TCP) or application layer (UDP etc.) of end
   systems and does not introduce any problem in the network and works
   as long as there is some connectivity between the end systems.

   Because end to end multihoming is performed in end systems, the
   architecture needs no routing protocol changes Instead, APIs and
   applications must be modified to some extent.

M. Ohta                  Expires on May 1, 2003                 [Page 1]

INTERNET DRAFT           End to End Multihoming            November 2002

1. Introduction

   Multihoming is a way for hosts have robust connectivity to the
   Internet.  Traditionally with IPv4, multihoming has been offered
   through intelligence of routing system, that is, the end to end
   principle has been ignored.

   However, as discussed in section 2, with the explosive deployment of
   the Internet, as is always the case with intelligent networking, IPv4
   style multihoming was revealed its lack of scalability and
   robustness.

   Instead, multihoming can be supported based on the end to end
   principle by assigning multiple addresses to an interface of a host
   and let end systems choose an appropriate address at the transport or
   the application layer.

   To support the end to end multihoming, no change is necessary on
   routing protocols. Instead, APIs and applications must be modified to
   detect and react against the loss of connection.  In case of TCP
   where there is a network wide agreement on the semantics of the loss
   of connectivity, most of the work can be done by the kernel code at
   the transport layer, though some timing may be adjusted for some
   application. However, in general, the condition of "loss of
   connectivity" varies application by application that the multihoming
   must directly be controlled by application programs.

   With IPV6, there is a partial effort of end to end multihoming that
   it is of course that an interface has multiple addresses.

   However, because the principle of the end to end multihoming is
   recognized merely subconsciously and because the current routing
   architecture violates the end to end principle in various ways as a
   result of partial attempt to avoid the lack of scalability of routing
   table size, there are a lot of attempts:

      to keep the APIs and application programs as is

      to modify routing system intelligent to let it automatically offer
      end to end multihoming

   which makes IPv6 multihoming as bad as that of IPv4.

   This memo describes why multihoming by intelligent routing system is
   harmful, how the current routing architecture is damaged and how the
   APIs and the applications should be modified to implement the end to
   end multihoming.

M. Ohta                  Expires on May 1, 2003                 [Page 2]

INTERNET DRAFT           End to End Multihoming            November 2002

2. Multihoming by Intelligent Routing System is Harmful

2.1 Routing Table Size

   See IAB Network Layer Workshop.

2.2 Lack of Robustness

   With route aggregation, routing information can be carried only for
   an aggregated area that a loss of connectivity for a part of the area
   can not be detected.

   For example, if a multihomed site with multiple subnets has a single
   global routing table entry, and if a site is partitioned, only a part
   of the partitioned can be reached with the global routing table
   entry.

   A solution is to let all the subnets of the site (or, ultimately,
   hosts) individually have global routing table entry, scalability of
   which is so absurd that no one bothers to try.

3. The Problems of Current Routing Architecture

   End systems (hosts) are end systems. To make the end to end principle
   effectively work, the end systems must have all the available
   knowledge to make decisions by the end systems themselves.

   With regard to multihoming, when an end system want to communicate
   with a multihomed end system, the end system must be able to select
   most appropriate (based on the local information) destination address
   of the multihomed end system.

   However, some think it of course to separate routers and nodes and
   let hosts not have routing information (which means the current IPv6
   architecture is broken) and, worse, let most routers use default
   routes.

   With detailed route information, end systems can use the information
   as a hint to select the best destination address.  However, with
   default route, end systems have no idea on what is the best address
   and must blindly try all the possibilities at random.

   It was partly because full routing table was large and was not be
   able to be held in a chip on end systems and partly because hosts
   should not be affected by routing protocol changes.

   In the past, IPv4 address was not assigned with hierarchy and scale
   of integration was small.

M. Ohta                  Expires on May 1, 2003                 [Page 3]

INTERNET DRAFT           End to End Multihoming            November 2002

   But, with smaller full routing table and larger scale of integration,
   there is no reason not to have full routing table on every end
   system.

   As there are a lot of routers used in LAN or even in home, it is not
   so meaningful that we don't have to upgrade software on hosts, if we
   have to upgrade software on routers.

   The situation is worse with multicasting.  For example, IGMP, which
   separates routers and nodes, is a total nonsense as IGMP has been
   revised several times upon multicast routing protocol changes.
   Moreover, all the legacy multicast routing protocols use intelligent
   routing system to deliver group specific information and does not
   scale. Multicast architecture must be redone with the end to end
   principle in mind. SM (Static Multicast) proposed by the Authors is
   such a proposal but latter proposals (such as "simple multicast" or
   "AS based static allocation") modified it without understanding the
   underlying end to end principle and are useless.

   Once a full routing table is available on all the end systems, it is
   easy for the end systems try all the destination addresses, from the
   most and to the least favorable ones, based on the routing metric.

   Note that end to end multihoming works with the separation between
   inter domain BGP and intra domain routing protocols, if BGP routers,
   based on domain policy, assign external routes preference values
   (metric) of intra domain routing protocols.

   One may still be allowed, though discouraged, to have local
   configuration with dumb end systems and an intelligent proxy. But,
   such configuration should be implemented with a protocol for purely
   local use without damaging the global protocol.

4. Modifications on APIs and Applications

   With TCP, applications must be able to pass multiple addresses to
   transport layer (e.g. BSD socket). All the other processing can be
   performed by transport layer (typically in kernel) using default or
   application specific timing of TCP.

   TCP itself must be modified that all the possible addresses of a host
   is transmitted to its peer through a TCP option. TCP connections are
   identified with all the addresses constitute an identical connection.

   Without TCP, applications must be able to detect loss of connectivity
   in application dependent way and try other addresses by themselves or
   tell transport layer to do so.  Applications must still be able to
   pass multiple addresses of the destination to transport layer (e.g.

M. Ohta                  Expires on May 1, 2003                 [Page 4]

INTERNET DRAFT           End to End Multihoming            November 2002

   BSD socket) to receive a packet to alternative addresses sent from
   the other end of the communication.

   The easiest way for applications know all the addresses of the
   destination is to use DNS. With DNS reverse, followed by forward,
   lookup, applications can get a list of all the addresses of the
   destination from an address of the destination.

   Note that policy based selection of a source address is incompatible
   to the end to end principle, because it must be selected as the
   destination address by the policy of the other end of the
   communication using information local to the other end of the
   communication.  With so much asymmetric routing of the Internet
   today, proper destination address to reply can not be guessed by the
   querier.

   Thus, DNS query should be modified to carry all the addresses of
   clients and servers should try from the most favorable to the least
   ones, based on locally available information such as metric in intra
   domain full routing table.

   Any source address may be selected.

   However, to enable source address filtering to discard packets with
   source addresses not belonging to an ISP, it is useful to enable a
   host, not some intelligent intermediate router, select a source
   address compatible with an outgoing ISP.  For that purpose, intra
   domain routing protocols should maintain routing table entries with
   not only preference values of an external routes, but also proper
   prefixes to be selected for source addresses, if the entries are
   chosen by a host.

   With DNS, it is also required that DNS reverse lookup works properly.
   But, as the reverse lookup is the mechanism to delegate IP addresses,
   the requirement is no more demanding than assigning valid IP
   addresses.

   A problem is that locally scoped address (IPv6 link and site local
   addresses) can not be used for reverse look up. Use of 8+8 addressing
   proposed by one of an Author with globally unique IID (Internet ID)
   and ILOC (Internet Locator) is strongly encouraged. With 8+8
   addressing, DNS reverse lookup can be performed with IID part only.

   Note that 8+8 proposal must not be confused by latter proposal of
   routing goof by Mike O'dell, which is a proposal to use intelligent
   routers to rewrite source addresses to prevent source address
   spoofing and to tunnel between intelligent routers for pseudo
   multihoming, both of which are against the end to end principle and,

M. Ohta                  Expires on May 1, 2003                 [Page 5]

INTERNET DRAFT           End to End Multihoming            November 2002

   thus, lacks robustness and/or scalability.

5. Conclusions

   For robust and scalable multihoming, IPv6 separation of nodes and
   routers must be removed and the transition to the end to end
   multihoming should occur with the transition to IPv6.

   The modification is not so difficult, because most of the
   applications, which must be modified for IPV6 anyway, use TCP only
   and most of the UDP applications are DNS, which already tries all the
   addresses, or multicast capable ones, which are hopeless.

   One may argue that we can't further delay the transition to IPv6
   merely because of a random proposal on multihoming.

   Then, it is a good idea to start another transition, separated from
   ones with legacy IPv6, by allocating a new address prefix of IPv6
   address space for the end to end multihoming with 8+8 addressing.

   It is important to make the end to end principle work by keeping the
   number of top level routing prefix under the new address prefix small

   Politics of address space allocation may be avoided if those who are
   allocated IPv6 address space with the current prefix are
   automatically allocated corresponding address space with the new
   prefix.

   It should be noted that, because of the end to end nature, the
   architecture can be implemented purely on end systems without
   modifying routing functionality of existing IPv4 or IPv6 routers.

6. Security Considerations

   The author believes there is no special security concern.

7. Author's Address

   Masataka Ohta
   Graduate School of Information Science and Engineering
   Tokyo Institute of Technology
   2-12-1, O-okayama, Meguro-ku, Tokyo 152-8552, JAPAN

   Phone: +81-3-5734-3299
   Fax: +81-3-5734-3299
   EMail: [email-masked]

M. Ohta                  Expires on May 1, 2003                 [Page 6]

---

From: Pekka Nikander <[email-masked]>
Date: Wed, 06 Nov 2002 10:38:06 +0200

Ohta-san,

>    The author believes there is no special security concern.

You don't discuss the potential flooding DoS attack that
has been discussed at this list a few times during the last
days, and that was fairly well documented in
draft-aura-mipv6-bu-attacks-01.txt (expired, but available at
http://research.microsoft.com/users/tuomaura/Mobile%20IPv6/draft-aura-mipv6-bu-attacks-01.txt )

draft-aura considers the flooding only in the context of
MIPv6, but it applies almost directly to the kind of
end-host multi-homing you are suggesting.

You also don't discuss potential problems due to DNS
spoofing.  Even though the situation in your proposal
does not seem to be any worse than today, IMHO the
small differences and their implications should be
briefly discussed.

--Pekka Nikander

---

From: Masataka Ohta <[email-masked]>
Date: Wed, 6 Nov 2002 18:47:50 +0859 ()

Pekka;

> days,

DoS attacks similar to SYN flooding is possible and similary
protected against that it is not of special concern.

> http://research.microsoft.com/users/tuomaura/Mobile%20IPv6/draft-aura-mipv6-bu-attacks-01.txt )

It is not available from microsoft that I found it in google cache.

> spoofing.

DNS, just as the rest of the Internet, is weakly secure.

> Even though the situation in your proposal

Brief consideration is that there is no special concern.

Note that it is not a "security discussion" section.

In no event, I won't waste 27 pages of document only to "briefly
discuss" security implications.

						Masataka Ohta