Subject: Re: draft-ietf-multi6-multihoming-requirements-03

From: Joe Abley <[email-masked]>
Date: Mon, 1 Jul 2002 12:45:08 -0400

On Wednesday, June 26, 2002, at 04:00 , Joe Abley wrote:

> On Wednesday, June 26, 2002, at 03:29 , Tony Hain wrote:
>
>> While I believe this document still assumes too much in terms of
>> affecting policy outside the domain of direct administrative control, I
>> have to agree it is time to ship it. There is one comment about the 
>> IESG
>> in 3.2.7 that simply doesn't belong in any I-D, so I wouldn't be
>> surprised when the IESG asks to have it changed or removed. If the 
>> words
>> 'within the IESG' were removed, the sentence would stand and makes much
>> more sense than worrying about IETF process in a mechanism requirements
>> doc.
>
> I have made that change in my working copy.

Does anybody have any other proposed changes to this document, or should 
I roll a -04 with that change only, ready for wg last call?

Joe

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Mon, 1 Jul 2002 20:04:03 +0200 (CEST)

On Mon, 1 Jul 2002, Joe Abley wrote:

> >> If the words

> I roll a -04 with that change only, ready for wg last call?

There seems to be a discrepancy between the requirements listed in the
draft and the fact that everyone assumes there won't be a solution that
can meet all of them. I'm afraid this opens up the possibility of problems
down the road when people can easily dismiss otherwise promising
multihoming solutions by pointing out it doesn't meet one or more
requirements.

Some text indicating which requirements are absolutely essential and which
are also important, but can be dropped if there aren't any multihoming
solutions that satisfy them would be good, I think.

On the other hand: I certainly don't want to slow down the whole process.

Iljitsch van Beijnum

---

From: Joe Abley <[email-masked]>
Date: Mon, 1 Jul 2002 14:05:19 -0400

> solutions that satisfy them would be good, I think.

Do you have such text to incorporate?

Joe

---

From: Masataka Ohta <[email-masked]>
Date: Tue, 2 Jul 2002 02:53:23 +0859 ()

Joe Abley;

> I roll a -04 with that change only, ready for wg last call?

Can't we simply admit the fact that none of us have any operational
experience to be able to discuss multi6 requirement document
to be used later to evaluate proposals and move on without it?

						Masataka Ohta

---

From: Joe Abley <[email-masked]>
Date: Mon, 1 Jul 2002 14:06:58 -0400

> to be used later to evaluate proposals and move on without it?

Are you suggesting there are no network operators here?

---

From: Masataka Ohta <[email-masked]>
Date: Tue, 2 Jul 2002 03:07:45 +0859 ()

Joe Abley;

> Are you suggesting there are no network operators here?

No, I am not, as I am involved in operation of mobile IPv6 network
with more than 150 routers.

					Masataka Ohta

---

From: Joe Abley <[email-masked]>
Date: Mon, 1 Jul 2002 14:25:33 -0400

> with more than 150 routers.

Ah, I understand the point you were making, then. Sorry for my confusion.

Joe

---

From: Eric Gauthier <[email-masked]>
Date: Mon, 1 Jul 2002 14:33:08 -0400

> Are you suggesting there are no network operators here?

Ok, I've been lurking for a while, but here are my $0.02.  I'm a 
network architect at a large university.  We're testing IPv6 on our
campus and intend to multihome with our pTLA.  We currently multihome 
with our IPv4 address space and had issues that we've solved regarding 
multihoming, especially in cases of Internet 1 versus Internet 2 traffic.  
So, just in case anyone is worried, there are people lurking here that 
do ops and not just people who are curious about the theoretical workings 
of IPv6.  

Secondly, IMHO, we've laid out what we think are requirements and there
doesn't appear to be anything in there that's unreasonable to hope for 
and expect.  Its now up to the brainiac's in round two to come up with a 
solution.  Just because I can't think of one doesn't mean that someone 
else won't.  But, if it turns out that someone does a proof that our 
requirements are mutually exclusive or no one is able to come up with a 
solution, we'll then have the experience of those who tried to work it 
out to determine what, specifically, is the largest subset of our draft 
that can be implemented or where we went wrong.  Until we get that 
concrete feedback, we're only guessing at what can and cannot be done.

I think we've written our exam question as best we can.  Its time to ship 
it to the students for them to solve and hope/pray that they can...

Eric :)

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Mon, 1 Jul 2002 23:40:46 +0200 (CEST)

> Do you have such text to incorporate?

3.1 Capabilities of IPv4 Multihoming

   The following capabilities of current IPv4 multihoming practices MUST
   be supported by an IPv6 multihoming architecture.  IPv4 multihoming
   is discussed in more detail in [1].

Would be replaced by:

3.1 Capabilities of IPv4 Multihoming

   The following capabilities of current IPv4 multihoming practices MUST
   be supported by an IPv6 multihoming architecture to a similar level as
   current IPv4 multihoming does. None of them is optional, but they can
   be omitted or only partially supported if an architecture supporting
   the capability to at least current IPv4 levels is unfeasible or
   doesn't meet the additional requirements listed in section 3.2.

   IPv4 multihoming is discussed in more detail in [1].

Then there is:

3.1.4 Policy

   A customer may choose to multihome for a variety of policy reasons
   beyond technical scope (e.g.  cost, acceptable use conditions, etc.)
   For example, customer C homed to ISP A may wish to shift traffic of a
   certain class or application, NNTP, for example, to ISP B as matter
   of policy.  A new IPv6 multihoming proposal MUST provide support for
   site-multihoming for external policy reasons.

I don't think this is the intention, but it _can_ be read as if IPv4
multihoming has the capability to direct traffic for certain applications
over one link and traffic for other applications over another link. Since
this is certainly not the case, at least the example should go, possibly
the whole thing. (Does it require anything useful anyway?)

Iljitsch van Beijnum

---

From: Joe Abley <[email-masked]>
Date: Mon, 1 Jul 2002 23:27:55 -0400

> the whole thing. (Does it require anything useful anyway?)

The example I have used for this before concerns people who operate 
networks in places where satellite communications are substantially 
cheaper than terrestrial communications (this is most of the planet, by 
area).

It is common practice to number news, mail, and HTTP proxy servers 
(tuned for retrieval of objects over satellite-like latencies) within 
subnets which can be advertised such that inbound traffic from the rest 
of the world arrives on those servers via a satellite link, whereas all 
other traffic (including traffic directly aimed at subscriber addresses) 
arrives via terrestrial fibre.

This crude approach takes the traffic load of asynchronous, 
latency-tolerant services off the expensive terrestrial fibre, and 
leaves it (the fibre) for use by more interactive, latency-intolerant 
traffic. Costs are reduced, performance for customers remains acceptable.

Often, satellite bandwidth is bundled by satellite operators with 
transit. ISPs who buy such satellite transit are very unlikely to use 
the satellite provider for their terrestrial transit. Being multi-homed 
in this fashion is common practice today in many parts of the world.

I agree that existing IPv4 multihoming practices/CIDR abuse do not 
inherently provide mechanisms for making routing decisions based on 
criteria encapsulated *within* IP. However, changing the requirements to 
specify something else (like segregation of different parts of an 
address space over different transit circuits) seems to me a bit too 
much like presupposing a solution.

I think the basic requirement should stay. I am not married to the words 
currently in 3.1.4, however, so if you have better ones I'd be glad to 
hear them.

Joe

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 02 Jul 2002 10:31:16 +0200

>    IPv4 multihoming is discussed in more detail in [1].

I wonder why we are using MUST (upper case) terminology in any case.
It isn't the IETF tradition to have strict requirements documents and
require mathematical conformance; this whole thing amounts to a strong
recommendation, not a standards-track MUST. Also "None of them is optional,
but they can be omitted..." makes my head hurt. Why not just change the
language in the whole document to must/should (lower case)?

(This also answers Masataka's point, I think.)

> the whole thing. (Does it require anything useful anyway?)

Yes. Today this sort of policy is generally implemented by fiddling with
DNS entries and addresses. Maybe we can do better for IPv6 if we think about
it.  The example is a valid *requirement* but it should be cited
in the IPv6 context.

    Brian

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Tue, 2 Jul 2002 15:08:51 +0200 (CEST)

> arrives via terrestrial fibre.

Ok, nothing wrong with that. But BGP doesn't provide any functionality to
really help with this. Since it can be done now without help from BGP,
there is no reason an IPv6 solution should specifically cater to this
need. Just not getting in the way of more specific routes or policy
routing should be enough.

> currently in 3.1.4, however, so if you have better ones I'd be glad to

"Existing IPv4 multihoming practices can coexist with policy-based routing
and forwarding mechanisms. An IPv6 multihoming architecture should
retain this capability."

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Tue, 2 Jul 2002 15:18:13 +0200 (CEST)

> I wonder why we are using MUST (upper case) terminology in any case.

Don't ask me, I think there are too many caps in the world anyway.

> Also "None of them is optional,
> but they can be omitted..." makes my head hurt.

It was the best I could come up with. It's your turn now.  :-)

> Why not just change the

I'm all for that. But that doesn't mean the sentence above can be omitted.

> it.  The example is a valid *requirement* but it should be cited

I am currently multihomed in IPv6 (you have to practive what you preach)
by having two tunnels, each with address space associated with them. I am
only allowed to send out packets with source addresses A over tunnel A,
and source addresses B over tunnel B. This makes some sense, but it breaks
the hop-by-hop forwarding paradigm. It would be good if there were
something better to say about this than "if you do, I'll just filter them,
ha ha". And if we require routers to look at source addresses, why not
protocols, ports and diffserv code points as well? But I'm pretty sure
this isn't something that should be solved in this document at this time.

---

From: Joe Abley <[email-masked]>
Date: Tue, 2 Jul 2002 11:27:16 -0400

On Tuesday, July 2, 2002, at 09:08 , Iljitsch van Beijnum wrote:

> really help with this.

Well, BGP plus CIDR abuse does. If I operate 203.97.0.0/17, and I obtain 
transit from two providers as vaguely described, I can hang all my 
satellite-tolerant gear in (say) 203.97.2.0/24 and advertise just 
203.97.2.0/24 over the satellite circuit. The fact that I am able to 
advertise covered routes to make things work is central to the way that 
multi-homing is made to work with v4, and is one example of things we 
are trying to provide core-safe alternatives for in v6.

>  Since it can be done now without help from BGP,

It seems feasible to me that there will be solutions to the requirement 
as currently stated on the draft that do not need to rely on policy 
routing or the advertisement of covered routes. I do not have such a 
solution in mind, but I don't see why we should discount the possiblity 
that one might exist.

> and forwarding mechanisms. An IPv6 multihoming architecture should

As I said, I think that presupposes a solution.

Joe

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Wed, 3 Jul 2002 00:22:29 +0200 (CEST)

On Tue, 2 Jul 2002, Joe Abley wrote:

> > Ok, nothing wrong with that. But BGP doesn't provide any functionality
> > to really help with this.

> 203.97.2.0/24 over the satellite circuit.

And you must also make sure the servers you communicate with are routed
over the satellite link. If you don't, you'll have to use policy routing.

> The fact that I am able to

Basically you are using two different address ranges here, one multihomed
and one single homed. (The fact that they overlap in this particular
instance is not important.) This is such a basic capability that I can't
even imagine a multihoming solution that conflicts with it.

> solution in mind, but I don't see why we should discount the possiblity

Making scalable multihoming work is difficult enough without overloading
it with extra nice to have features. But since I think this functionality
can be provided regardless of the multihoming solution, and there is
little harm in looking to optimize things, I'm not going to object very
strongly.

> As I said, I think that presupposes a solution.

I didn't have one in mind when I wrote it.

---

From: Joe Abley <[email-masked]>
Date: Tue, 2 Jul 2002 18:31:01 -0700

> I didn't have one in mind when I wrote it.

I was talking about the solution which involves "policy-based
routing and forwarding mechanisms". Seems to me that the requirement
in the document right now could be met without using those.

For example, suppose the pure aggregation/no hole-punching addressing
and route advertisement scheme is strictly adhered to, and the effect
of multi-homing is to number each interface with an address from each
transit provider. Further suppose that the TCP endpoint address
selection algorithm can be influenced by some hierarchical policy
signalling mechanism. In that case, inbound traffic towards a TCP
endpoint could be moved between transit providers without resorting
to CIDR abuse.

I'm not saying that such a scheme would feature in a multihoming
architecture v6, or that it is not without other substantial problems,
but it does sound to me like a solution to the specific requirement
in question which does not rely on advertising covering supernets
(or other disparate routes) to different transit providers in order
to achieve the desired goal.

More generally, you seem to be saying that the requirement as stated
reduces to something trivial, which of course should be supported by
any adequate multihoming architecture. If I have got that right, then
what's the harm in letting the requirement stand in the document?

Joe

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Wed, 3 Jul 2002 13:49:26 +0200 (CEST)

> in the document right now could be met without using those.

Ah, I see. I was thinking about:

1. Both ends have the special applications use different addresses that
   are routed over just one connection (probably better to use PA space
   for this rather than a more specific in BGP)

2. Policy routing based on application characteristics such as IP address
   or protocol/port number

3. Diffserv code point that has the effect a specific connection is
   selected

Are there additional ways to accomplish this?

> endpoint could be moved between transit providers without resorting

You still need either:

- BOTH ends to have their IP address for this application in a separate
  range, or

- use policy routing based on address or protocol/port to select the right
  exit connection

The former is workable for things like NNTP, where the traffic flow is
easily predicted and some optimization in IP addressing is well worth it.
The latter is a dirty hack IMO, because it breaks hop-by-hop forwarding.

When we get basic multihoming working for IPv6, there is probably a lot we
can do to optimize load balancing while providing differentiated services
(and/or the other way around).

> what's the harm in letting the requirement stand in the document?

There is not much harm. Still, my original point remains: the wording
could lead to confusion about current IPv4 capabilities.

Iljitsch van Beijnum

---

From: Joe Abley <[email-masked]>
Date: Wed, 3 Jul 2002 11:02:09 -0400

> - BOTH ends to have their IP address for this application in a separate

This happens by virtue of the fact that each transit provider delegates 
a block of space to the multi-homed site. If the site is single-homed, 
it only has one transit provider, and hence only one IP address. The 
fictional address selection criteria might apply to both ends of the 
session. Yes, there are big holes in this approach :)

> The latter is a dirty hack IMO, because it breaks hop-by-hop forwarding.

I have usually seen this deployed without policy routing to manipulate 
the path of outbound traffic. Frequently the satellite channels used are 
uni-directional in order to save money on (or avoid regulatory hassles 
with) an uplink from the multi-homed site.

> could lead to confusion about current IPv4 capabilities.

I will look at the words. The requirement I was trying to describe is 
most certainly met by the current v4 regime; it's a description of an 
approach which is already in widespread use.

There is actually a separate (probably now expired) v4 capabilities 
draft which requires content. It needs more work, but I've had zero 
contributions to it; I will try to review it and propose a path forward 
with it in the next week or so.

Joe