Subject: Host-based may be the way to go, but network controls are neccessary

From: "Aldrin Isaac" <[email-masked]>
Date: Tue, 19 Nov 2002 10:57:30 -0500

More opinions from the enterprise camp:

I've come to the conclusion that a host-based host-only multihoming
solution may be the only solution required for true end-to-end
multihoming since a *good solution* should meet all requirements, bar
simplicity, and should include mobility as well.

However there are currently no routing knobs for a multi-homed site to
control via which site-exit a packet should be forwarded based on the
packet's source prefix (there are a hundred reasons why NAT is used
today, and this is one of them).  This will result in unnecessary
"misses" (which means delay, dead packets, IDS alarms, etc) during the
connection setup of any host-based solution within a multi-homed site
because of the current source address selection process.

Network controls are therefore required to allow for the forwarding of
packets over paths that will obviously *not* reject them, and provide
an early warning to a host when it selects a source address that will
result in failed transmission.

This form of network control is certainly missing even in today’s IPv4
routing toolkit, but the difference is that with IPv4 it is acceptable
to “fix” this using NAT, DFZ aggregation hole punching and other
methods.  I think if this routing issue can be solved multi6 can focus
on a true host-based end-to-end multihoming solution.

-- aldrin

---

From: "Craig A. Huegen" <[email-masked]>
Date: Tue, 19 Nov 2002 13:22:29 -0600 (Central Standard Time)

> simplicity, and should include mobility as well.

A host-based, host-only multihoming solution does not give the network the
visibility into alternate paths to a destination, and therefore cannot
apply policy to it.

The only point of control a network operator would have in that
environment is some kind of policy mechanism on hosts.  I don't see it
scaling to hundreds of thousands of nodes unless the central policy point
is a function of the network.

/cah

---
Craig A. Huegen, Chief Network Architect      C i s c o  S y s t e m s
IT Transport, Network Technology & Design           ||        ||
Cisco Systems, Inc., 400 East Tasman Drive          ||        ||
San Jose, CA  95134, (408) 526-8104                ||||      ||||
email: [email-masked]       CCIE #2100      ..:||||||:..:||||||:..

---

From: Shane Kerr <[email-masked]>
Date: Tue, 19 Nov 2002 20:57:32 +0100

On 2002-11-19 13:22:29 -0600, Craig A. Huegen wrote:
> > I've come to the conclusion that a host-based host-only
> > multihoming solution may be the only solution required for true
> > end-to-end multihoming since a *good solution* should meet all
> > requirements, bar simplicity, and should include mobility as well.
> 
> A host-based, host-only multihoming solution does not give the
> network the visibility into alternate paths to a destination, and
> therefore cannot apply policy to it.
> 
> environment is some kind of policy mechanism on hosts.  I don't see
> it scaling to hundreds of thousands of nodes unless the central
> policy point is a function of the network.

Perhaps it is best to identify three categories of operator:

1. Large network operators
These people can get IP space and AS from their RIR, and advertise
into the DFZ, as in IPv4 today.

2. Small end-users
This means home users, and especially small businesses.  Right now,
they have no way to multihome at all.  I don't see any feasible way to
allow these users to multihome except for host-based multihoming.

3. Somewhere in-between
I hope that this is the area that you are always talking about when
you declare that multihoming requires policy control.

Which category is this group trying to help?  If it is #1, then we're
done.  If it is #2, then I think we have a good start.  If it is #3,
then there is basically MHAP.  I think for the long-term support of
category #3 we need real routing changes, perhaps BGP won't do it.

If we can agree on this breakdown perhaps we can move forward by
identify the needs of each.

-- 
Shane Kerr
RIPE NCC

---

From: "Craig A. Huegen" <[email-masked]>
Date: Tue, 19 Nov 2002 15:57:42 -0600 (Central Standard Time)

> into the DFZ, as in IPv4 today.

ARIN's IPv6 policy (which is almost globally equivalent to the rest of
the RIR's, right?), states:

---
5.1.1. Initial allocation criteria

To qualify for an initial allocation of IPv6 address space, an
organization must:

a) be an LIR;

b) not be an end site;

c) plan to provide IPv6 connectivity to organizations to which it will
assign /48s, by advertising that connectivity through its single
aggregated address allocation; and

d) have a plan for making at least 200 /48 assignments to other
organizations within two years.
---

This policy disqualifies large enterprises based on b).

The bootstrap criteria allowed some end-users who deploy IPv6 to get
address space.  I'm hoping that we can come up with a multihoming method
that would cover the large enterprise networks as well as the larger of
the "in-betweens" you mentioned, such that the allocated space to
end-users could be given back.

I concur with you that host-based multihoming may be sufficient for a
class of smaller networks, like homes and small businesses.

Also, remember that large enterprise networks generally do not announce
their entire address space from every Internet attachment point.  This
essentially turns each region served by a particular Internet attachment
point into a "site".  If we went into particulars, I know this one large
enterprise network that has 5 "major" Internet attachment points (for
general access and services) globally, and a handful of "minor" Internet
attachment points (for VPN only).  Should that enterprise then get 5 /32's
for the major points, and use host-based multihoming for the minor points?
Or, should the enterprise get a single /32, but announce (and have
accepted through a recommendation from this WG supporting it) /36's into
the DFZ?  Either way, that's 5 routes in the DFZ for this network.

> identify the needs of each.

I hope the group is trying to address all three sizes you mention.  Size
isn't the only factor, though -- I can envision a small site that needs
greater policy control than allowing the end-host to pick the closest
matching source/dest addresses.

/cah

---

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 19 Nov 2002 23:15:59 +0100

Craig,

If the IETF invents a viable multihoming solution that requires
the RIR policies to be modified, but does not damage aggregation,
we have well established channels for negotiating a change of
policy with our friends, the RIRs. So we shouldn't take current 
RIR policy as a constraint on the solution space.

    Brian

---

From: "Craig A. Huegen" <[email-masked]>
Date: Tue, 19 Nov 2002 16:31:28 -0600 (Central Standard Time)

Brian,

Sorry, that's not an argument I meant to make.

The argument I meant to make is that I have not observed a consensus (or
anywhere near it) that we should make IP space available to the large
end-users from the RIR's.  To be honest, if we did come up with that
conclusion for the large networks (and were able to classify "large
networks" and address my "multi-site" comments below with it as well),
then most of the problems of the large enterprise go away.

I floated the idea of giving PI space to the large organizations earlier
this year and vaguely remember it not getting a warm reception. :)

(Part of the miscommunication was a mis-read of Shane's reply -- I
originally read it to mean that he suggested large enterprises can do this
today.)

/cah

---

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Wed, 20 Nov 2002 00:12:43 +0100 (CET)

> this year and vaguely remember it not getting a warm reception. :)

This can be a viable solution if you can figure out a way to limit the
number of globally visible PI blocks that are going to be assigned by
the RIRs. There doesn't seem to be a reasonable mechanism to accomplish
this.

Note that today the RIRs seem to have firm policies, but in reality you
can pretty much get anything you want (save a /8) from them if you try
hard enough.

---

From: "Michael H. Lambert" <[email-masked]>
Date: Tue, 19 Nov 2002 20:00:26 -0500

> is a function of the network.

I don't think there is any requirement that a host make its decisions about 
address selection _in vacuo_.  It makes sense for it to be able to obtain 
assistance from a source which is aware of network policy.

Michael

+-----------------------------------------------------------------------+
| Michael H. Lambert, Network Engineer           Phone: +1 412 268-4960 |
| Pittsburgh Supercomputing Center               FAX:   +1 412 268-8200 |
| 4400 Fifth Avenue, Pittsburgh, PA  15213       [email-masked]        |
+-----------------------------------------------------------------------+

---

From: Erik Nordmark <[email-masked]>
Date: Wed, 20 Nov 2002 03:26:06 +0100 (CET)

> because of the current source address selection process.

I would be useful (at least for me) to understand the input and output
parameters to the policy control.

The most flexible one is to provide the list of possible destination addresses
and source addresses as input, and get as a result the desired combinations
(presumably ranked so that there is an indication of the fallback order 
when the 1st one doesn't work or fails).

But building solutions for that is likely to cause significant performance 
overhead (at least for a host based solution).
Thus I wonder if there are policy functions that could be good enough even
though they can only choose the source and the destination is given by the
host.

  Erik

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Wed, 20 Nov 2002 14:44:46 +0100 (CET)

> overhead (at least for a host based solution).

That doesn't have to be the case. Today, hosts already perform a
host/domain name to IP address mapping. You wouldn't take a significant
performance hit by looking up the addresses learnt from the DNS in the
routing table and tag them with some kind of routing metric. And if you
have access to routing at this stage, you can see which exit each
addresss takes so which source prefix should be appropriate.

Alternatively, you could cache this information in the host, which will
either buy you a lot (host with many sessions) or next to nothing (host
with few sessions). The next step would be to share this information
between hosts on the same subnet or within the same site.

> though they can only choose the source and the destination is given by the

I think if we do this the "smart but wrong" way we'll live to regret it
when we need to do it the right way later. For instance, if applications
try to be smart about selecting one address from several given by the
DNS, this makes it impossible for the resolver to do this right and
simply list the addresses in the preferred order.

---

From: Erik Nordmark <[email-masked]>
Date: Wed, 20 Nov 2002 19:44:16 +0100 (CET)

> addresss takes so which source prefix should be appropriate.

The host most likely just has one (or two) default routes, so it can't
make this determination locally. Even the first hop router might not
have much information about external destinations. The routing metrics,
in whichever router you find them, might not capture and might not be capable
of capturing the policy folks want - routing is defined to route packets
and not to describe which source addresses are preferred for which
destinations.

One could naively make the host query a site-wide server for policy decisions
e.g. each time the host needs to choose source and destination addresses
for a connection. That would have performance implications.

Something that would have less of a performance implication is that
the host chooses a destination address from the ones returned by the DNS,
and e.g. the site exit router can detect that the source address is
not the best one and send back a "please use different source address".
In this case the border router would not know whether there are alternative
destination addresses that the host could have chosen, thus
I suspect the set of polices that can be handled is less then in the more 
general case.

Not that 8+8/GSE didn't have a way for the network to influence the initial 
destination address to use; the host could learn a new destination
RG from the return traffic.

Hence my (so far) futile attempts to understand what policy control are needed
or desired.

  Erik

---

From: "Craig A. Huegen" <[email-masked]>
Date: Wed, 20 Nov 2002 12:59:06 -0600 (Central Standard Time)

On Wed, 20 Nov 2002, Erik Nordmark wrote:

> In this case the border router would not know whether there are
> alternative destination addresses that the host could have chosen, thus
> I suspect the set of polices that can be handled is less then in the
> more general case.

For a larger enterprise network, this becomes a showstopper.  If the host
picks a destination address on the least preferred network, the network
infrastructure has no way to redirect him.

An alternative is to push policy to the hosts that's a bit smarter than
longest-bit-match, except in a large enterprise, programming policy to
100,000+ hosts is extremely hard to do unless it's standardized and
centralized in the network.  Now, I have no problems with the host
learning policy (through RA, or through a DNS mechanism, or whatever) but
it needs to be a required, standard part of IPv6.

/cah

---

---

From: "Michael H. Lambert" <[email-masked]>
Date: Wed, 20 Nov 2002 14:16:00 -0500 (EST)

> for a connection. That would have performance implications.

Is this approach made more attractive with 1) cacheing and 2) a hierarchy
of such servers at the site?  There is still overhead, but is it any more
than that associated with, say, joining a multicast group or setting up an
SVC?

Michael

+----------------------------------------------------------------------------+
| Michael H. Lambert, Network Engineer                Phone: +1 412 268-4960 |
| Pittsburgh Supercomputing Center                    FAX:   +1 412 268-8200 |
| 4400 Fifth Avenue, Pittsburgh, PA  15213            [email-masked]        |
+----------------------------------------------------------------------------+

---

From: Kurt Erik Lindqvist <[email-masked]>
Date: Wed, 20 Nov 2002 20:58:32 +0100

> addresss takes so which source prefix should be appropriate.

How would you propagate routing or metric changes to host after it have 
done the lookup?

- kurtis -

---

From: Francis Dupont <[email-masked]>
Date: Wed, 20 Nov 2002 23:04:24 +0100

In your previous mail you wrote:

=> this is the "source redirect", IMHO something we really need.

=> as Christian showed in his draft, there are cases where a "wrong" source
doesn't work. It is important to handle such cases, and optimization should
come after (so do the source redirect now and continue to think about
better/more complete solutions).

=> this is known too to be a major security hole...

   Hence my (so far) futile attempts to understand what policy control
   are needed or desired.
   
=> a minimal control is needed (cf Christian's draft, the argument is
ingress filtering). More is surely desired but complex enough that
no proposal is already a clear win.

Thanks

[email-masked]
   
PS: draft-huitema-multi6-hosts-01.txt

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Thu, 21 Nov 2002 00:39:54 +0100 (CET)

> have much information about external destinations.

True. But the address info comes from the DNS, and it shouldn't be a
problem to hack the DNS to be aware of routing or do this the clean way
with a new protocol. (In my book, DNS is up for an overhaul anyway since
it still requires the host to know DNS addresses.)

> The routing metrics,

I think the two are similar enough to be useful. We could also think
about an end-to-end address preference protocol (as long as we're
retiring the DNS anyway...) but that will probably take a bit longer to
do.

> for a connection. That would have performance implications.

1. The host can cache this information when there are many connections
   to the same host in quick succession (HTTP...)
2. The host does DNS queries today anyway

> not the best one and send back a "please use different source address".

That is also a useful approach although I think Christian doesn't like
it.

> Hence my (so far) futile attempts to understand what policy control are needed

Simple. If you have two or more connections to the internet, you want to
use them to your advantage. This means balancing the total traffic (not
individual sessions) over all links and being able to prefer one link
over another for individual destinations.

People with stateful firewalls also really like their incoming and
outgoing traffic to take the same path, but they don't get that today
either.

Iljitsch

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Thu, 21 Nov 2002 00:43:52 +0100 (CET)

> How would you propagate routing or metric changes to host after it have

Today: you don't. The host can't change its behavior when the TCP
session has been established anyway.

Future: when it becomes possible for a host to jump from one set of
addresses to another (as in SCTP) it certainly becomes useful to do
this. The crude way: drop packets, the host will get the hint. The nice
way could be an ICMP message.

---

From: Kurt Erik Lindqvist <[email-masked]>
Date: Thu, 21 Nov 2002 00:52:57 +0100

> way could be an ICMP message.

That means that the router would have to keep even more state than 
today so it would know when to send an update to the host.

State is among the most costly thing for network operators, you want to 
avoid it. To make this scale you would have to send periodic updates 
and the host would know only when the next update comes.

So with packet-loss or the above, there will first be a service impact 
- then a effect on the routing path. Now, if we assume that the primary 
reason for anyone to go to multihoming (I know there are others) is for 
network resilience (this will most likely be the first one at least), 
they are actually not getting that with the above model.

Why do I get that MPLS feeling here...

I realize that I am jumping in and out of discussions and not giving a 
very clear statement myself. I have promised both Sean and Michel to 
write up my impressions (FWIW) from Sundays meetings and post it to the 
list. I will try and get it done during the plenary..:)

- kurtis -

---

From: Peter Tattam <[email-masked]>
Date: Thu, 21 Nov 2002 11:02:31 +1100 (EST)

> over another for individual destinations.

As an ISP, I often want the reverse.  Some links are backup and cost a lot more
to run and should be deprecated.  Other links might be flaky or have bizarre
characteristics (e.g, radio or satellite) so that they should be deprecated for
some reason (or only be used in one direction).

My suppliers have been rather unfriendly in honouring BGP priorities, and
sometimes my only level of control has been to shut the links down which kind
of defeats the purpose of multihoming.

> outgoing traffic to take the same path, but they don't get that today

Peter

--
Peter R. Tattam                            [email-masked]
Managing Director,    Trumpet Software International Pty Ltd
Hobart, Australia,  Ph. +61-3-6245-0220,  Fax +
61-3-62450210

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Thu, 21 Nov 2002 01:06:28 +0100 (CET)

> today so it would know when to send an update to the host.

I'm not sure what you mean by "know when to send an update". When the
router (or other box designated for this function) sees too much traffic
over one link it just starts sending back ICMPs to redirect some of it.

> and the host would know only when the next update comes.

We're not talking about network operators here, but about enterprises.
If they want this, they'll have to pay for it. Just like traffic
engineering in BGP today costs money in the form of man hours.

> So with packet-loss or the above, there will first be a service impact

Unreliable datagram service. Sometimes you lose one.

> Why do I get that MPLS feeling here...

I don't know. When the only tool you have is a hammer, everything looks
like a nail?

Iljitsch

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Thu, 21 Nov 2002 01:08:44 +0100 (CET)

> some reason (or only be used in one direction).

How is this the reverse, other than that the list of "individual
destinations" is very long (the whole internet)?

---

From: Erik Nordmark <[email-masked]>
Date: Thu, 21 Nov 2002 02:19:24 +0100 (CET)

> it needs to be a required, standard part of IPv6.

Thanks Craig, this was the type of info I was looking for.

Do you have ideas on how complex the policies typically are?
For instance, would it make sense to push all of the rules into the hosts
or is the set of rules so large that a host-based solution would need to
cache subsets of the rules on demand?

  Erik