Subject: RE: Next question...

From: "Tony Li" <[email-masked]>
Date: Fri, 6 Dec 2002 10:38:59 -0800

|   > I suspect most of the complaints about host based locator 
|   selection is
|   > really an indirect fear of loss of control.
|   
|   i suspect it is folk with a knowledge of just how difficult 
|   and complex
|   global routing is when isolated to a site border router and 
|   our minds
|   boggle when we think of all the hosts having to do it too.

I have to disagree with both.  First, there is no way that
the site border router OR the host is going to have full information
about global routing.  They *might* have some limited policy
information and a mechanism to retrieve locators.

My concern is not so much about loss of control as it is with
the administrative burden of implementing the policies.  Most
policies are going to be site-wide, not host specific.  Having
to distribute this policy and keep it consistent across an
enterprise can be done by our standard system administration tools,
but the effort involved in doing so is non-trivial and the
number of exception hosts is likely to be small, so this just
seems like a poor ROI for the site administrator(s).

Our job, as architects, is not to create an architecture with
infinite flexibility.  Rather, it is to reject those options
which lead to unnecessary complications down the line.  Control
at the host seems like an infrequently used feature.  Note that
policy for a specific host can be implemented at either the 
site border router (SBR) or at the host, so removing the policy
decision from the host isn't actually eliminating architectural
flexibility.

Tony

---

From: "Tony Hain" <[email-masked]>
Date: Fri, 6 Dec 2002 15:14:26 -0800

Tony Li wrote:
> the site border router OR the host is going to have full 
> information about global routing.  They *might* have some 
> limited policy information and a mechanism to retrieve locators.
> 
> the administrative burden of implementing the policies.  

Isn't perception of burden really a function of direct vs. indirect
control? 

> Most policies are going to be site-wide, not host specific.  
> Having to distribute this policy and keep it consistent 
> across an enterprise can be done by our standard system 
> administration tools, but the effort involved in doing so is 
> non-trivial and the number of exception hosts is likely to be 
> small, so this just seems like a poor ROI for the site 
> administrator(s).

I agree that most policies are site wide, but ROI depends on how
valuable the exception list is. If you are in the exception list, your
perspective is different than if you are not. As far as triviality, that
depends on the mechansim. If it were something like a static policy on
all host to prefer the prefix order in the RA, it seems pretty trivial
to me.

> which lead to unnecessary complications down the line.  

And our disagreement really comes down to definition of 'necessary'. 

> Control at the host seems like an infrequently used feature.  

Frequency is not an indication of value. Besides it is done on every
connection today, so that seems pretty frequent to me. 

> Note that policy for a specific host can be implemented at either the 
> site border router (SBR) or at the host, so removing the 
> policy decision from the host isn't actually eliminating 
> architectural flexibility.

I absolutely disagree. Hosts actually make those policy decisions today,
and the system works. The machine I am typing this on has 4 IPv4
addresses, yet it has no trouble opening a connection to CNN.com by
picking from that list of possible sources, and the 8 available IPv4
destination addresses. There seems to be a fear that providing more
choices to the host will magically make the world more complex. 

The only thing that doesn't happen when the host selects the locator is
that the network administrator doesn't get to enforce fine-grained TE
for the return traffic. If this is what we are defining as 'unnecessary
complications', I have to question the necessity of changing the entire
infrastructure so that it will support the arbitrary replacement of
locators without affecting applications. That sounds a lot more complex
to me than simply adding one more policy to the set of things the host
administrators have to do already.

Tony

---

From: "Craig A. Huegen" <[email-masked]>
Date: Mon, 9 Dec 2002 17:52:13 -0600 (Central Standard Time)

> choices to the host will magically make the world more complex.

Start tracing to all those addresses you get with www.cnn.com.  That is
*not* multihoming, that's local infrastructure resiliency via DNS
load-balancing.  Those prefixes are all located within Atlanta.  Take a
look in BGP route tables and see that the real path selection happens in
the routing infrastructure via BGP paths.

The fact that your host randomly picked a destination address from DNS and
used the outbound interface's primary IP address for a new connection
isn't true multihoming.

Now, take a scenario:  Your host has source addresses from aaaa::/16,
bbbb::/16, cccc::/16, and dddd::/16.  Destination host has source
addresses from aaab::/16 and dddf::/16.  You pay $100k per megabyte from
the provider allocated aaa0::/12, and $2 per megabyte from the other
three.  You also need every host in your organization to be able to use
the aaa0::/12 provider to reach a very critical service with address
aaaf::/16, which you're willing to pay the $100k per megabyte for.  This
critical service also has connectivity via providers B and D in bbbe::/16
and ddde::/16, but with sub-par connectivity and you want to avoid that.

Using address selection or even RA ordering, you cannot specify this
policy.  But these types of policies are perfectly realistic today in the
IPv4 world.

In order to emulate this policy, I would have to specify a policy which
forced a destination address change through NAT in order to accomplish my
objective.

IMO, an absolute requirement here is that we have a policy capability
that's far greater than longest bit match or preferred order for RA
prefixes.  Policy in path selection for multihomed enterprises, at a
minimum, needs to be able to prefer or avoid certain intersections of
source and destination addresses for a given destination *host* (not
address!) that would normally be avoided or preferred.  A nice-to-have is
a policy engine that is also aware of AS-path data so that policies can be
programmed for AS preference when it involves more than one AS between two
communicating organizations.

> administrators have to do already.

Take a look at the scenario above.  These are examples of the types of
policy that enterprises need.

/cah

---
Craig A. Huegen, Chief Network Architect      C i s c o  S y s t e m s
IT Transport, Network Technology & Design           ||        ||
Cisco Systems, Inc., 400 East Tasman Drive          ||        ||
San Jose, CA  95134, (408) 526-8104                ||||      ||||
email: [email-masked]       CCIE #2100      ..:||||||:..:||||||:..

---

From: "Tony Hain" <[email-masked]>
Date: Mon, 9 Dec 2002 18:14:17 -0800

Craig A. Huegen wrote:
> *not* multihoming, that's local infrastructure resiliency via 
> DNS load-balancing.  Those prefixes are all located within 
> Atlanta.  Take a look in BGP route tables and see that the 
> real path selection happens in the routing infrastructure via 
> BGP paths.
> 
> The fact that your host randomly picked a destination address 
> from DNS and used the outbound interface's primary IP address 
> for a new connection isn't true multihoming.

All the host is capable of doing is selecting source and destination
locators. What is so hard to understand about; the routing system does
path selection, because the host doesn't know anything more than
locators. 

There is no difference between the host selecting the source locator
based on an RA ordering or doing that based on a defined primary. Those
are functionally the same capability. This leaves the question of
selecting destination locator, which could be random, or based on some
deterministic process. Either way the host is the one selecting the
locator. The fact that it is local load balancing vs. multi-homing is
only known in the routing system.

> 
> Now, take a scenario:  Your host has source addresses from 
> aaaa::/16, bbbb::/16, cccc::/16, and dddd::/16.  Destination 
> host has source addresses from aaab::/16 and dddf::/16.  You 
> pay $100k per megabyte from the provider allocated aaa0::/12, 
> and $2 per megabyte from the other three.  You also need 
> every host in your organization to be able to use the 
> aaa0::/12 provider to reach a very critical service with 
> address aaaf::/16, which you're willing to pay the $100k per 
> megabyte for.  This critical service also has connectivity 
> via providers B and D in bbbe::/16 and ddde::/16, but with 
> sub-par connectivity and you want to avoid that.
> 
> Using address selection or even RA ordering, you cannot 
> specify this policy.  But these types of policies are 
> perfectly realistic today in the IPv4 world.
> 
> In order to emulate this policy, I would have to specify a 
> policy which forced a destination address change through NAT 
> in order to accomplish my objective.

So show how that would be done without nat in IPv4. I presume this will
start by limiting the source address list to 1 choice, but there is no
clear way to avoid the bbbe & ddde services if that is what DNS returns
to the host. Yes it is easy to route the outbound to the preferred
provider despite the destination locator, but that is also true for
IPv6. The real rub comes on the return path. If the connection is opened
to the wrong address, the return traffic will not take the preferred
ingress path. Even if it is, there is no guarantee that the cost
structure at the remote site will match, so the traffic might end up on
the least preferred path anyway. 

> 
> IMO, an absolute requirement here is that we have a policy 
> capability that's far greater than longest bit match or 
> preferred order for RA prefixes.  Policy in path selection 
> for multihomed enterprises, at a minimum, needs to be able to 
> prefer or avoid certain intersections of source and 
> destination addresses for a given destination *host* (not
> address!) that would normally be avoided or preferred.  

Path selection is a routing function. To implement the policy outlined
above you would have to intercept and possibly rewrite all dns messages
as well.

Tony

> A 
> nice-to-have is a policy engine that is also aware of AS-path 
> data so that policies can be programmed for AS preference 
> when it involves more than one AS between two communicating 
> organizations.
> 
> > The only thing that doesn't happen when the host selects 
> the locator 
> > is that the network administrator doesn't get to enforce 
> fine-grained 
> > TE for the return traffic. If this is what we are defining as 
> > 'unnecessary complications', I have to question the necessity of 
> > changing the entire infrastructure so that it will support the 
> > arbitrary replacement of locators without affecting 
> applications. That 
> > sounds a lot more complex to me than simply adding one more 
> policy to 
> > the set of things the host administrators have to do already.
> 
> Take a look at the scenario above.  These are examples of the 
> types of policy that enterprises need.
> 
> /cah
> 
> ---