Subject: [RAM] Traffic Engineering

From: RJ Atkinson <[email-masked]>
Date: Fri, 29 Dec 2006 09:53:46 -0500

I think that some folks on this list are not familiar
with what issues/problems are currently being addressed with
traffic engineering.  There was concern expressed at NANOG
about potential loss of ability to use "traffic engineering"
if SHIM6 were widely deployed, but the details of those concerns
haven't been widely explained.

	I also think that different parts of the Internet are
using traffic engineering to address different sets of issues/problems.

	It would be helpful if operators [1] could enumerate the
myriad ways that traffic engineering is currently being used,
including what solution folks are trying to achieve.  That is,
the kind of detail that I'm looking for something more like "force
traffic off of link A until that link can be upgraded, because that
link is heavily congested just now" not "re-route traffic".

	Then maybe someone, ideally some operator person, could
maintain a list of these uses and problem/solution information.
This would help in evaluating whether potential solutions are
sufficient to meet existing known operational needs.

Thanks,

Ran
[email-masked]

[1] "operator" in MY parlance means any person/organisation
that operates a chunk of network.  It is in no way limited
to ISPs or NSPs, at least in my own usage of the word.

_______________________________________________
RAM mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ram

---

From: Roland Dobbins <[email-masked]>
Date: Fri, 29 Dec 2006 08:12:12 -0800

On Dec 29, 2006, at 6:53 AM, RJ Atkinson wrote:

> ms are currently being addressed with

SHIM6 simply isn't practical for a number of reasons, difficulties in  
traffic engineering with an architecture of this type being least  
among them.

Just handing off path-selection functionality to the endstation and  
the concept of multiple addresses on the endstation creates a  
troubleshooting and support nightmare which neither enterprises nor  
individuals can be expected to accept.  The problem SHIM6 purported  
to solve is a) much more fundamental and larger in scope than the  
SHIM6 proponents realized (read all of RFC4192 for a sense of this)  
and b) is rendered moot by the fact that registries are now providing  
PI space to organizations which need it, anyways.

-----------------------------------------------------------------------
Roland Dobbins <[email-masked]> // 408.527.6376 voice

		All battles are perpetual.

     		   -- Milton Friedman

---

From: David Meyer <[email-masked]>
Date: Fri, 29 Dec 2006 08:29:54 -0800

> link is heavily congested just now" not "re-route traffic".

	Jason Schiller has a couple of nice presentations on this
	topic. See e.g.,

	 http://www.nanog.org/mtg-0510/schiller.html
	 http://www.nanog.org/mtg-0505/schiller.html

	he also had a nice presentation at the R&A workshop. I'll
	see if I can get that publicly available somewhere.

> sufficient to meet existing known operational needs.

	Good idea, though I'm not sure why it would need to be an
	ops type (they're already under quite a bit of stress in
	the day job, etc).

	--dmm

---

From: Jari Arkko <[email-masked]>
Date: Fri, 29 Dec 2006 19:04:29 +0200

Roland, Ran
> individuals can be expected to accept.  The problem SHIM6 purported to
> solve is a) much more fundamental and larger in scope than the SHIM6
> proponents realized (read all of RFC4192 for a sense of this) and b)
> is rendered moot by the fact that registries are now providing PI
> space to organizations which need it, anyways.
Let me take a slightly different viewpoint into this. There
are legitimate needs for providers to perform traffic
engineering, including TE related to multihomed
sites. Most of the pain that NANOG people and others are
feeling comes from the fact that we really have no
good, scaleable solution for this. We are starting to
hand off PI space, which we know works but (a) may
increase the overall Internet routing difficulties and
(b) is unlikely to be given to organizations of all sizes.

But it does not necessarily follow that Shim6 is a
bad solution. The question you have to ask is who
are these solutions targeted for. I fully agree that
Shim6 is not -- at least as defined now -- a solution
that makes allows centralized TE and control of
multihoming, and it does not look appealing for
very big sites where PI space looks a lot more
attractive. But at the same time, a host based
solution is very attractive in SOHO and when
you either are or want to be independent of
provider decisions in what you do. I don't
think we want the multihoming decisions of
my multi-interface laptop to be handled by
the network; that wouldn't scale.

My personal  belief at the moment is that
we will be needing both network/provider
based and host based solutions. It might
even be that we can discover a mechanism
that allows us to do both. Yes, provider TE
is a requirement but that's not the same
thing as provider-based solutions being the
only one that we need.

In getting to those solutions, it is indeed
important that we understand the different
TE requirements. Its also good to keep in mind
that many of the problems we are seeing
are not only multihoming issues, but a
combination of issues related to provider
independence, renumbering, TE, peering
complexity growth, limitations of the protocols
we have, and even errors and malicious acts.
So lets solve a bigger problem than just
multihoming TE.

(Also, we've already got multiple addresses
on endstations, quite independent of Shim6
or other host-based multihoming solutions.
And we are starting to see devices that
make multihoming decisions on the host,
regardless of any specific support in the
IP layer for that. As an example, I have
a cell phone that has seven radio interfaces,
most of which can run IP and some of which
will appear as separate interfaces from an
IP perspective.)

--Jari

---

From: Roland Dobbins <[email-masked]>
Date: Fri, 29 Dec 2006 09:15:53 -0800

On Dec 29, 2006, at 9:04 AM, Jari Arkko wrote:

>  But at the same time, a host based
> provider decisions in what you do.

I don't believe this is attractive to SOHO users in the least; the  
technical complexity of troubleshooting has shot up significantly.

> I don't

I'm not sure I agree with this statement.

> will appear as separate interfaces from an

Yes, and merely utilizing such a system as an end-user is difficult  
for even fairly technical folks, and this paradigm in general is  
quite problematic for enterprises who wish to integrate more-or-less- 
universally mobile network end-points into their networks.

It's somewhat easier for folks to understand, operate, and administer  
endpoints with multiple logical network addresses which correspond  
with multiple media (including varying flavors of wireless), or even  
on the same media with logical partitioning a la VLANs, than a single  
interface on the same physical network which carries multiple logical  
network overlays.  Doing things like maintaining ACLs, firewall  
rules, and the various traffic engineering tweaks which sparked this  
topic rapidly becomes difficult, if not impossible.

     		   -- Milton Friedman

---

From: Per Heldal <[email-masked]>
Date: Fri, 29 Dec 2006 18:36:52 +0100

> technical complexity of troubleshooting has shot up significantly.

How do you know what type of tools will be available at deployment time
for a technology for which the specification is not yet complete, and
currently lacks implementations to even play with?

[snip]

> 
> Yes, and merely utilizing such a system as an end-user is difficult for
> even fairly technical folks, and this paradigm in general is quite
> problematic for enterprises who wish to integrate
> more-or-less-universally mobile network end-points into their networks.
> 
> endpoints with multiple logical network addresses which correspond with
> multiple media (including varying flavors of wireless), or even on the
> same media with logical partitioning a la VLANs, than a single interface
> on the same physical network which carries multiple logical network
> overlays.  Doing things like maintaining ACLs, firewall rules, and the
> various traffic engineering tweaks which sparked this topic rapidly
> becomes difficult, if not impossible.

Isn't it premature to assume that there will be no tools and
developments happening on top on shim6?  Would you similarly dismiss the
original specifications for e.g. TCP on the assumption that there will
be no such things as stateful firewalls and load-balancers?

I'm not convinced that shim6 is the final solution. Still it's not
impossible to imagine solutions to coordinate path-selection among large
numbers of hosts in an end-network to achieve site-wide policies for
traffic-management.

//per

---

From: Roland Dobbins <[email-masked]>
Date: Fri, 29 Dec 2006 10:41:18 -0800

> currently lacks implementations to even play with?

I believe the basic concept to be so fatally flawed that there is no  
way a set of tools would make this technology feasible.

> be no such things as stateful firewalls and load-balancers?

Absent a fundamental change to the loosely-coupled nature of the  
protocol stack, the deconflation of the host and locator IDs, and a  
complete revolution in network-management tools/systems (and a vendor  
renaissance to implement all of this) inspired by SHIM (in other  
words, this sort of technology applied to something which most  
definitively is not IPv6 or even a closely-related derivative  
thereof), I can forecast no tools or developments which would make  
this technology less problematic and more supportable on any kind of  
scale beyond networking enthusiasts and tinkerers.

> Still it's not

Although the desirability and operational complexity of such a system  
are deserving of serious discussion and investigation, it is a  
reasonable extrapolation to at least posit something of this sort,  
hich is why I stated that the possibility of traffic engineering  
issues raised by SHIM6 were the least of its problems.

     		   -- Milton Friedman

---

From: Bruce Curtis <[email-masked]>
Date: Fri, 29 Dec 2006 15:09:34 -0600

On Dec 29, 2006, at 10:12 AM, Roland Dobbins wrote:

> SHIM6 simply isn't practical for a number of reasons, difficulties  
> in traffic engineering with an architecture of this type being  
> least among them.

   While SHIM6 may make traffic engineering more difficult there may  
still be potential architectures with hierarchical routing and  
multiple addresses on the endstation that still allow traffic  
engineering.  For example there was a discussion on architecture- 
[email-masked] of a model where all of the multiple addresses of  
both endstations are included in each packet.

> individuals can be expected to accept.

   While having a single global address for the end station is  
certainly simpler I'm not sure that having multiple addresses for an  
endstation is totally unacceptable.  Endstations already deal with a  
simple form of multiple addresses since they have a localhost address  
and an IPv4 address on the network and with dual stack an  IPv6  link  
local address plus an IPv6 global address.

   And essentially enterprises have already accepted the concept of  
multiple addresses for and endstation when they implement NAT, the  
endstation has a local/private address and a global address.

   On a historical note I'm glad to see them go but it wasn't  
considered unacceptable for an endstation to have an IPv4 address, an  
IPX address plus an Appletalk/Ethertalk address.

   Also I'm not a fan of VPN clients but that is another example of  
an endstation with multiple addresses that has largely been accepted.

> The problem SHIM6 purported to solve is a) much more fundamental  
> and larger in scope than the SHIM6 proponents realized (read all of  
> RFC4192 for a sense of this) and b) is rendered moot by the fact  
> that registries are now providing PI space to organizations which  
> need it, anyways.
>

---
Bruce Curtis                         [email-masked]
Certified NetAnalyst II                701-231-8527
North Dakota State University

---

From: marcelo bagnulo braun <[email-masked]>
Date: Fri, 29 Dec 2006 22:25:12 +0100

Hi Roland,

> individuals can be expected to accept.

I think this is an important point.
As i read your input, a solution that results in that the end points 
need to manage multiple IP addresses is simply a non starter, right?

So, do people agree that a solution MUST expose a SINGLE IP address to 
the hosts within the multihomed site?

In your opinion, is there any other design choice made by the shim6 
protocol that is flawed that we can also learn from and try to avoid 
repeating?

Regards, marcelo

>   The problem SHIM6 purported to solve is a) much more fundamental and 
> larger in scope than the SHIM6 proponents realized (read all of 
> RFC4192 for a sense of this) and b) is rendered moot by the fact that 
> registries are now providing PI space to organizations which need it, 
> anyways.
>

---

From: Roland Dobbins <[email-masked]>
Date: Fri, 29 Dec 2006 13:33:35 -0800

On Dec 29, 2006, at 1:09 PM, Bruce Curtis wrote:

> certainly simpler I'm not sure that having multiple addresses for  
> an endstation is totally unacceptable.  Endstations already deal  
> with a simple form of multiple addresses since they have a  
> localhost address and an IPv4 address on the network and with dual  
> stack an  IPv6  link local address plus an IPv6 global address.

The percentage of hosts with IPv4/IPv6 dual-stack enabled which is  
not the result of an OS vendor default and which is consciously used  
for anything is nil.

Multiple addresses are fine in terms of domain/functional  
separation.  When multiple addresses are utilized to access a single  
logical domain on a workstation owned/used by a human being, things  
get out of hand very quickly in terms of supportability.

> endstation has a local/private address and a global address.

See above - where NAT is implemented, there is generally an 'inside'  
address used for the domain of intranet communications and an  
external address for the domain of everything else (i.e., the  
Internet).  Even in extranet-types of deployments with dual-NATting  
taking place on both sides, with all the headaches and  
troubleshooting complexities a setup of that type entails, it is  
still orders of magnitude easier to support a special-purpose  
installation of this type within the bounded domain of site-to-site/ 
org-to-org connectivity than attempting to do so with a general user  
population within an enterprise, much less a population of home/SOHO  
users.

> considered unacceptable for an endstation to have an IPv4 address,  
> an IPX address plus an Appletalk/Ethertalk address.

These worked pretty well when they were necessary; they gave way to  
translational gateways and then proxies, and then have for the most  
part withered away entirely.  Again, there was no 'casual' use of  
such systems at the whim of the user population, and therefore the  
supportability problems of such architectures, ugly and problematic  
as they were, pales in comparison to what will be encountered on a  
general system intended for casual use by everyone in the world to do  
anything and everything he wants, anytime he wishes to do so.

> an endstation with multiple addresses that has largely been accepted.

Again, see above - firstly, VPN support for enterprises is a bounded  
logical domain (and even supporting VPN users is no fun, if you've  
ever done it; so, the argument undermines itself completely even  
apart from the logical domain issue).

     		   -- Milton Friedman

---

From: "Joel M. Halpern" <[email-masked]>
Date: Fri, 29 Dec 2006 17:14:22 -0500

At 04:25 PM 12/29/2006, marcelo bagnulo braun wrote:
>So, do people agree that a solution MUST expose a SINGLE IP address 
>to the hosts within the multihomed site?

No.
While I would like to see a solution where the host does not have to 
be aware of multiple addresses,
a) it may be that even in such a solution, some hosts may benefit by 
such awareness
b) if we can spell out how the hosts make sensible choices, and why 
they need to make those choices, then it may be beneficial to require 
the host to have visibility to the multiple addresses.

Further, it may be possible that hosts need / want to be aware of 
multiple remote addresses, without necessarily having awareness of 
their own multiplicity.

There are many possibilities.
While I have preferences and hopes as to what can work, and what is 
"simple", the guideline has to be "simple enough to do the job", not 
"simple enough to meet an arbitrary standard."

Yours,
Joel M. Halpern

PS: The biggest problem with most of the multiple address work on 
hosts in the past (including some of the deprecated IPv6 address 
forms) was that hosts had no sensible way to understand how to use 
the addresses.

---

From: Bruce Curtis <[email-masked]>
Date: Fri, 29 Dec 2006 16:31:28 -0600

On Dec 29, 2006, at 3:33 PM, Roland Dobbins wrote:

> not the result of an OS vendor default and which is consciously  
> used for anything is nil.

   But dual stack was well discussed and accepted as the best method  
to migrate from IPv4 to IPv6.

> separation.  When multiple addresses are utilized to access a  
> single logical domain on a workstation owned/used by a human being,  
> things get out of hand very quickly in terms of supportability.

   What if an endstation had a single address that contained the  
information about prefixes from it's multiple ISPs?  If the  
endstation just considered it a single address but the address  
contained information about multiple paths that could be used by  
devices in the core of the Internet for Traffic Engineering would you  
consider that acceptable?

> See above - where NAT is implemented, there is generally an  
> 'inside' address used for the domain of intranet communications and  
> an external address for the domain of everything else (i.e., the  
> org-to-org connectivity than attempting to do so with a general  
> user population within an enterprise, much less a population of  
> home/SOHO users.

     We will have to agree to disagree on this point.  I don't agree  
that multiple addresses would be orders of magnitude more difficult  
to support than NAT.  In fact given the choice I would rather support  
a network with multiple addresses on endstations than one with NAT.

> general system intended for casual use by everyone in the world to  
> do anything and everything he wants, anytime he wishes to do so.

   But wouldn't the user be limited to using one of the multiple  
addresses for the endstation?  They couldn't just casually chose any  
address.

> Again, see above - firstly, VPN support for enterprises is a  
> bounded logical domain (and even supporting VPN users is no fun, if  
> you've ever done it; so, the argument undermines itself completely  
> even apart from the logical domain issue).

   Well, I agree that supporting VPN users is no fun.  So we do agree  
on at least one thing.  :-)

   However the problems I've encountered didn't seem to stem from the  
endstations essentially having multiple addresses.

---

---

From: Igor Gashinsky <[email-masked]>
Date: Fri, 29 Dec 2006 17:52:26 -0500 (EST)

some comments in-line...

On Fri, 29 Dec 2006, Bruce Curtis wrote:
:: 
:: On Dec 29, 2006, at 10:12 AM, Roland Dobbins wrote:
:: 
:: >
:: > On Dec 29, 2006, at 6:53 AM, RJ Atkinson wrote:
:: >
:: > > ms are currently being addressed with
:: > > traffic engineering.  There was concern expressed at NANOG
:: > > about potential loss of ability to use "traffic engineering"
:: > > if SHIM6 were widely deployed, but the details of those concerns
:: > > haven't been widely explained.
:: 
:: While SHIM6 may make traffic engineering more difficult there may still 
:: be potential architectures with hierarchical routing and multiple 
:: addresses on the endstation that still allow traffic engineering.  For 
:: example there was a discussion on [email-masked] of a 
:: model where all of the multiple addresses of both endstations are
:: included in each packet.
::
:: > Just handing off path-selection functionality to the endstation and the
:: > concept of multiple addresses on the endstation creates a troubleshooting
:: > and support nightmare which neither enterprises nor individuals can be
:: > expected to accept.
:: 
::   While having a single global address for the end station is certainly
:: simpler I'm not sure that having multiple addresses for an endstation is
:: totally unacceptable.  Endstations already deal with a simple form of
:: multiple addresses since they have a localhost address and an IPv4 address on
:: the network and with dual stack an  IPv6  link local address plus an IPv6
:: global address.

So, let's take a look at the scalability of this solution -- lets say that 
I'm a very large global content provider, with a very complex TE 
requirement for my outbound traffic (business reasons, performance driven, 
politics, etc). I also have 100k-200k servers which serve end-users. 
Currently, all of my TE decisions happen on relatively few (100-200) 
routers that have full BGP feeds, several million paths in the routing 
table, and understand that if I have pathX and pathY to an end user, they 
should prefer pathX (ie basic BGP-based TE). 

With Shim6, even if every end-station will provide me with all of their 
endpoint ip's during the initial handshake (ie establish shim state 
immediately), I now have 200k *servers* that each must deal with the 
overhead (albeit very small) of maintaining that state, as well as be 
loaded with full routing tables (and all of those paths) to make the Te 
decisions (ipX is via pathX, and ipY is via pathY, and since we prefer 
pathX vs pathY, chose ipX), which is a rather huge overhead for every 
server and every connection for that server. This is absolutely a 
non-starter for any decent size content provider based on that lack of 
scalability alone.

And, yes, I know we can come up with some sort of a centralized TE server 
which will hold the full tables, and when queried will reply with which 
IP to use, but now you are essentially delaying every new connection, 
inserting another element and a point of failure/complexity into the 
equation, and that is something that most of us are not willing to live 
with (well, we might be willing to live with the complexity, but our 
users are not willing to live with the connection delays -- u should see 
the number of complaints we get if we drop the first SYN of a connection, 
and this would be a very similar delay).

-igor

----------------------+----------------------+------------------
Igor Gashinsky, CISSP | Network Architecture | Yahoo! Inc.
 [email-masked]   |  cell 917.807.2213   | Do You... Yahoo?
----------------------+----------------------+------------------

RAM mailing list

[email-masked]

---

From: "Spencer Dawkins" <[email-masked]>
Date: Fri, 29 Dec 2006 18:52:07 -0600

Hi, Marcelo,

Thank you for asking. One topic that came up repeatedly at NANOG was concern 
about SHIM6 reliance on hosts that are massively vulnerable to viruses and 
botnetting (visualize a botnet controller switching all owned hosts 
selecting transit paths from TWC to SBCGLOBAL, within a minute or two).

Spencer

From: "marcelo bagnulo braun" <[email-masked]>
To: "Roland Dobbins" <[email-masked]>
Cc: <[email-masked]>
Sent: Friday, December 29, 2006 3:25 PM
Subject: other requirement? (was Re: [RAM] Traffic Engineering

Hi Roland,

El 29/12/2006, a las 17:12, Roland Dobbins escribió:

> traffic engineering with an architecture of this type being least among 
> them.
>
> Just handing off path-selection functionality to the endstation and the 
> concept of multiple addresses on the endstation creates a troubleshooting 
> and support nightmare which neither enterprises nor individuals can be 
> expected to accept.

Regards, marcelo

> larger in scope than the SHIM6 proponents realized (read all of RFC4192 
> for a sense of this) and b) is rendered moot by the fact that registries 
> are now providing PI space to organizations which need it, anyways.
>

---

From: "Chris L. Morrow" <[email-masked]>
Date: Sat, 30 Dec 2006 04:14:23 +0000 (GMT)

would it be possible to not have html email on this list? :) it's just
ugly and unnecessary.

---

From: Andrew McGregor <[email-masked]>
Date: Sat, 30 Dec 2006 18:28:08 +1300

On 30/12/2006, at 7:41 AM, Roland Dobbins wrote:

> I believe the basic concept to be so fatally flawed that there is  
> no way a set of tools would make this technology feasible.

Inasmuch as there are HIP implementations available and their  
characteristics are similar to those a SHIM6 implementation would  
have, I disagree.  They're very easy to use, in fact can be largely  
ignored once in operation.  Now, it is true that in a multi-homed  
situation path selection may be a problem, but I believe simply  
paying attention to ECN notification would be sufficient to allow a  
good balance between administration and ease of use.

> complete revolution in network-management tools/systems (and a  
> vendor renaissance to implement all of this) inspired by SHIM (in  
> other words, this sort of technology applied to something which  
> most definitively is not IPv6 or even a closely-related derivative  
> this technology less problematic and more supportable on any kind  
> of scale beyond networking enthusiasts and tinkerers.

That's an assertion, I'd like to see some discussion as to why you  
believe this.

Andrew McGregor

---

From: Jari Arkko <[email-masked]>
Date: Sat, 30 Dec 2006 10:25:09 +0200

Hi Spencer,
>
> Thank you for asking. One topic that came up repeatedly at NANOG was
> concern about SHIM6 reliance on hosts that are massively vulnerable to
> viruses and botnetting (visualize a botnet controller switching all
> owned hosts selecting transit paths from TWC to SBCGLOBAL, within a
> minute or two).
Botnet effects on any networking technology is always
an important concern. But in this case I think you also
have to ask whether its host-based multihoming designs
(SHIM6, MOBIKE, HIP) that create this problem or something
else. It would seem that a botnet can send using the
source and destination prefixes it wants to, so it appears that
it has the same denial of service capability irrespective
of any functionality in the IP layer. Or did I miss something?

--Jari

---

From: Marshall Eubanks <[email-masked]>
Date: Sat, 30 Dec 2006 07:46:56 -0500

Dear Jari;

> of any functionality in the IP layer. Or did I miss something?

I too may be missing something, but isn't the idea behind SHIM6 is  
that the host
is multi-homed ? It has locators in two or more domains, so it can  
select paths.
So, in principle, a botnet operator could know what paths were possible.

If you are using announced PI or even PA space, the multi-homing is
hidden from the host in most cases, so I don't see how you could make  
that switch.

What's not clear to me is what, exactly, the feared attack is. How is  
switching an outbound DOS
from provider A to provider B and back worse than just blasting  
packets out of provider A, which can
be done now ?

> --Jari

Regards
Marshall

---

From: "Bound, Jim" <[email-masked]>
Date: Sat, 30 Dec 2006 10:00:42 -0500

My view is any TE future architecture uses that does not assume all but
the IP Headers and possibly some new options before the data payload
(e.g. TCP, SCTP, future-transport-updates) is encrypted is a
non-starter. Use of Identifiers, ports, et al should not be assumed,
thus TE needs an overhaul too.
 
/jim

________________________________

	From: Bruce Curtis [mailto:[email-masked]] 
	Sent: Friday, December 29, 2006 4:10 PM
	To: [email-masked]
	Subject: Re: [RAM] Traffic Engineering

			ms are currently being addressed with
			traffic engineering.  There was concern
expressed at NANOG
			about potential loss of ability to use "traffic
engineering"
			if SHIM6 were widely deployed, but the details
of those concerns
			haven't been widely explained.

		SHIM6 simply isn't practical for a number of reasons,
difficulties in traffic engineering with an architecture of this type
being least among them.

	  While SHIM6 may make traffic engineering more difficult there
may still be potential architectures with hierarchical routing and
multiple addresses on the endstation that still allow traffic
engineering.  For example there was a discussion on
[email-masked] of a model where all of the multiple
addresses of both endstations are included in each packet.

		Just handing off path-selection functionality to the
endstation and the concept of multiple addresses on the endstation
creates a troubleshooting and support nightmare which neither
enterprises nor individuals can be expected to accept. 

	  And essentially enterprises have already accepted the concept
of multiple addresses for and endstation when they implement NAT, the
endstation has a local/private address and a global address.

	  Also I'm not a fan of VPN clients but that is another example
of an endstation with multiple addresses that has largely been accepted.

		The problem SHIM6 purported to solve is a) much more
fundamental and larger in scope than the SHIM6 proponents realized (read
all of RFC4192 for a sense of this) and b) is rendered moot by the fact
that registries are now providing PI space to organizations which need
it, anyways.

-----------------------------------------------------------------------
		Roland Dobbins <[email-masked]> // 408.527.6376
voice

		       -- Milton Friedman

	

	---

---

From: "Bound, Jim" <[email-masked]>
Date: Sat, 30 Dec 2006 10:03:36 -0500

Not to be me-too on the mail list but I would appreciate that too.
Greatly in fact.
/jim 

> -----Original Message-----
> From: Chris L. Morrow [mailto:[email-masked]] 
> Sent: Friday, December 29, 2006 11:14 PM
> To: Bruce Curtis
> Cc: [email-masked]
> would it be possible to not have html email on this list? :) 
> it's just ugly and unnecessary.
>

---

From: Roland Dobbins <[email-masked]>
Date: Sat, 30 Dec 2006 10:15:53 -0800

On Dec 29, 2006, at 1:25 PM, marcelo bagnulo braun wrote:

> As i read your input, a solution that results in that the end  
> points need to manage multiple IP addresses is simply a non  
> starter, right?

Yes, because of the administrative and resource overhead (think  
iACLs, firewall rules, etc., the burden of maintain twice or more the  
number as are currently maintained, and consuming twice or more the  
TCAM space in ASICs to hold those iACLs and other filtering/policy  
enforcement mechanisms, MAC/CAM table sizes, along with things like  
uRPF, and so forth) and because of the support burden (think help- 
desk) and because of the potential for abuse (think compromised  
endpoints).

> to the hosts within the multihomed site?

I'm not sure what you mean by this.  My preference (due to the  
abovementioned reasons) is that any posited solution work with each  
endpoint a) having a minimum of 1 IP address and b) not being  
required to participate in routing decisions itself, but rather  
leaving path selection to the networking infrastructure unless the  
endpoint is explicitly configured to participate in routing via a  
routing daemon or what-have-you (which today is mainly done on  
servers, and not on many of those, in the scheme of things).

     		   -- Milton Friedman

---

From: Roland Dobbins <[email-masked]>
Date: Sat, 30 Dec 2006 10:20:28 -0800

On Dec 29, 2006, at 2:14 PM, Joel M. Halpern wrote:

> they need to make those choices, then it may be beneficial to  
> require the host to have visibility to the multiple addresses.
>

 From an end-user standpoint (enterprise or SOHO/consumer), from a  
help-desk standpoint (enterprise or SP), from an administrative  
standpoint (enterprise or SP), from a security standpoint (i.e.,  
mandating that compromised endpoints must have the capability of  
participating in routing decisions), and from a hardware resource  
standpoint (TCAMs for iACLs, MAC/CAM table info, etc.), mandating  
this type of behavior is a recipe for disaster, IMHO.

If you can make it optional, fine - I think it's all wasted effort  
because folks won't make use of it for the above reasons, anyways  
(again, how many endpoints run routing daemons today?).

     		   -- Milton Friedman

---

From: Roland Dobbins <[email-masked]>
Date: Sat, 30 Dec 2006 10:32:17 -0800

> to migrate from IPv4 to IPv6.

Yes, people posit this.  I believe it remains to be seen whether this  
is the best method.

> devices in the core of the Internet for Traffic Engineering would  
> you consider that acceptable?

I believe that the help-desk/troubleshooting issues surrounding this  
sort of thing, not to mention the security considerations, make this  
highly undesirable as mandatory behavior.

Again, how many endpoints run routing daemons today?

> to support than NAT.  In fact given the choice I would rather  
> support a network with multiple addresses on endstations than one  
> with NAT.

That's because you've never been tasked with a) maintaining and  
updating large numbers of iACLs/firewall rules/SNMP ACLs/vty ACL/etc.  
(these things are more fluid than you think, especially in the Cloud  
Cuckoo-Land world in which some portion of an organization's internal  
addressing would change according to the whims of business  
relationships), and you've never had to deal with the effects of  
overflowing finite ASIC resources due to the doubled/trebled/ 
quadrupled/quintupled number of ACEs/firewall rules/etc. such a  
scheme would require.

Multihomed <> dual-homed, keep in mind.

> addresses for the endstation?  They couldn't just casually chose  
> any address.

Most users don't know or care what IP addresses -are- in the first  
place.  I recently came to the surprising realization that  
apparently, most of the casual users of the Internet today seem to  
not type in URLs or make much use of bookmarks at all; rather, they  
make use of Google or Yahoo or some intranet search engine as their  
homepages, and simply type in what it is they believe they're looking  
for and then noodle through the proffered links until they find  
whatever seems to meet their needs.  I'm still gobsmacked by this.

Users use applications.  The applications then make use of APIs to  
talk down the networking stack in order to, ultimately, generate  
packets.  With a mandatory multiple address scenario, the  
troubleshooting process a tech-savvy individual would engage in to  
resolve an issue has been made much more complex, much less the  
process followed by a help-desk working with non-technical users.

>> bounded logical domain (and even supporting VPN users is no fun,  
>> if you've ever done it; so, the argument undermines itself  
>> completely even apart from the logical domain issue).
>
>   Well, I agree that supporting VPN users is no fun.  So we do  
> agree on at least one thing.  :-)
>
>   However the problems I've encountered didn't seem to stem from  
> the endstations essentially having multiple addresses.

I agree with you; that's not the main problem with VPNs at all.  It's  
one of the problems, but it's a bit down-list.

     		   -- Milton Friedman

---

From: Roland Dobbins <[email-masked]>
Date: Sat, 30 Dec 2006 10:55:00 -0800

> good balance between administration and ease of use.

I was unclear; I'm talking about all overhead required for things  
like iACLs, SNMP ACLs, vty ACLs, firewall rules, QoS policies, etc.,  
and what would be required to both support those things in hardware  
and simply maintain/update them from an administrative standpoint,  
and the ever-present help-desk burden.

> That's an assertion, I'd like to see some discussion as to why you  

I believe that the loosely-coupled nature of IPv4, which was a big  
asset in terms of getting something implemented and deployed, was a  
huge asset which helped TCP/IP win out over other competing  
protocols.  I also believe that this loosely-coupled paradigm, while  
being an asset during the adoption phase, has now become a millstone  
round our necks, because it has led us to rely on Rube Goldbergian  
mechanisms such as ACLs, firewall rules, etc. due to the lack of  
coupling and feedback mechanism for establishing identity/entitlement/ 
authentication/authorization up and down the stack.  The current  
conflation of host and locator information in TCP/IP (both IPv4 and  
IPv6) contributes to this brittleness.

As long as we are stuck with what are essentially approximated out-of- 
band mechanisms for trying to implement policy, the administrative  
burden and potential for outages related to mistakes during the  
administrative process, and as long as we must have help-desks for  
enterprise users and SP customers, any proposal which mandates  
multiple IP addresses and endpoint routing decisions is DOA, IMHO.

     		   -- Milton Friedman

---

From: Roland Dobbins <[email-masked]>
Date: Sat, 30 Dec 2006 11:13:53 -0800

> of any functionality in the IP layer. Or did I miss something?

A simple thought-experiment would be to consider what the current  
situation with regards to outbound DDoS from botted hosts would be  
like today if a routing daemon were mandated for each and every host  
in the network.

Ensuring that each and every compromised host provides attacker with  
a mechanism to make path selection for his DoS packets is an  
unwelcome innovation, IMHO.  This would allow the attacker to select/ 
vary his path based upon performance, impact to the infrastructure,  
capacity, and to complicate traceback.

It's pretty obviosu that there would be considerable benefits in  
terms of reachability and resiliency in *today's IPv4 world* if each  
and every host ran a routing daemon and participated in the IGP(s) of  
the network(s) on which they are deployed.  The fact that the  
overwhelming majority of hosts today do *not* run routing daemons,  
and that mechanisms such as HSRP/GLBP/VRRP/CARP have proven popular  
mechanisms for maximizing said reachability/resiliency, strongly  
implies that any and all proposals which aim to mandate host-based  
path-selection have significant drawbacks which make them impractical  
and undesirable - else we'd already be running routing daemons on the  
majority of IPv4 hosts today.

     		   -- Milton Friedman

---

From: marcelo bagnulo braun <[email-masked]>
Date: Sat, 30 Dec 2006 20:35:37 +0100

El 30/12/2006, a las 19:15, Roland Dobbins escribió:

> Yes, because of the administrative and resource overhead (think iACLs, 
> firewall rules, etc., the burden of maintain twice or more the number 
> as are currently maintained, and consuming twice or more the TCAM 
> space in ASICs to hold those iACLs and other filtering/policy 
> uRPF, and so forth) and because of the support burden (think 
> help-desk) and because of the potential for abuse (think compromised 
> endpoints).

i can see these are valid concerns for medium/big sites.
but do you think these are issues for soho sites? i mean, my home 
network doesn't have ACL, no TCAM for storing filters no MAC tables and 
a very simple firewall.

otoh, from perspective of each ISP the site is singlehomed, and it 
should only care about the prefix it has delegated to the site, so i 
fail to see why this would be a problem

> endpoint a) having a minimum of 1 IP address

i guess you mean a maximum of 1 IP address... i mean, you sure need a 
minimum of 1 ip address if you want to be part of an IP network :-)

in other words, hosts must have only one IP (global) address and not 
more (not talking about link local)

> and b) not being required to participate in routing decisions itself, 
> but rather leaving path selection to the networking infrastructure 
> unless the endpoint is explicitly configured to participate in routing 
> via a routing daemon or what-have-you (which today is mainly done on 

understand this point.

Regards, marcelo

---

From: "Chris L. Morrow" <[email-masked]>
Date: Sat, 30 Dec 2006 19:51:27 +0000 (GMT)

> network doesn't have ACL, no TCAM for storing filters no MAC tables and

I think that trying to put limits on the number if ip addresses a host has
is folly... There will always be corner cases where more/less are
required/mandated/in-use. I think that the core issue is, which I think
roland is getting at, that maintaining a limited number of addresses is
desirable. Probably someone will jump on 'addresses' and ask: "locator or
id or both?" I'm not sure that right now it matters... 'limited number of
globally known things' is the point, less state for each end point in the
larger world view is desirable.

Perhaps one way to view shim6 and it's breathren is as parts of the
puzzle, maybe they aren't THE multihoming solution, but maybe they work
for some portion of the multihoming crowd? Perhaps your SOHO solution
provided by your helpdesk relies on SHIM6 (and they accept the burdens it
brings, or doesn't tools-dependent).

I think what's been said loudly and often by larger network operators and
content providers is that SHIM6 isn't solving the problem(s) they see,
it's not even attempting to address those problems. So, they require
something else to do multihoming, they also realize that that 'something
else' may require radical changes, and will have to scale to very large
numbers over the next 10-15 years.

-chris

---

From: Roland Dobbins <[email-masked]>
Date: Sat, 30 Dec 2006 12:27:51 -0800

On Dec 30, 2006, at 11:51 AM, Chris L. Morrow wrote:

> I think that the core issue is, which I think

Very aptly put, Chris - that is the crux of the matter, and I believe  
it to be a generalizable principle.

> brings, or doesn't tools-dependent).

Yes, people do all sorts of outwardly-seemingly crazy/inefficient  
things on their own networks (often for very good situationally- 
specific reasons), and will continue to do so.  What we must not do  
is attempt to mandate such behaviors for the global Internet (I say  
'attempt' because such proposals will fail under their own weight and  
limitations; best not to waste any more time trying to breathe life  
into the stillborn, IMHO).

     		   -- Milton Friedman

---

From: Roland Dobbins <[email-masked]>
Date: Sat, 30 Dec 2006 12:29:54 -0800

On Dec 30, 2006, at 11:35 AM, marcelo bagnulo braun wrote:

> network doesn't have ACL, no TCAM for storing filters no MAC tables  
> and a very simple firewall.

So, are we positing that SOHO users should run with what is  
essentially one protocol set/paradigm, and larger sites another, with  
all that entails in terms of fragmenting the global Internet,  
creating interesting issues for application developers, etc.?  The  
majority of home users/SOHO sites are not multihomed, nor will they  
be for quite some time unless/until MANET-type networking becomes the  
norm, and probably not even then in the sense of multiple  
simultaneous links, but rather in a failover type of scenario.  It's  
a bit more blurry in terms of some mobile devices, but the direction  
seems to be devices which can opportunistically leverage whatever the  
'best' network is at any given moment (based upon performance, cost,  
administrative policies, content/service availability, etc.).

I believe that the help-desk/troubleshooting problems discussed  
previously at length precludes *mandating* multiple IP addresses, it  
is a nonstarter.

> i guess you mean a maximum of 1 IP address... i mean, you sure need  
> a minimum of 1 ip address if you want to be part of an IP network :-)

No, I meant minimum - in other words, whatever solution is decided  
upon, it must work with a minimum of one IP address per host, and  
with a maximum of whatever someone chooses to configure on his host.

> in other words, hosts must have only one IP (global) address and  
> not more (not talking about link local)

No, see above - one or more, but the minimum requirement = 1.

     		   -- Milton Friedman

---

From: marcelo bagnulo braun <[email-masked]>
Date: Sat, 30 Dec 2006 23:55:21 +0100

El 30/12/2006, a las 21:29, Roland Dobbins escribió:

> all that entails in terms of fragmenting the global Internet, creating 
> interesting issues for application developers, etc.?

i guess so.

but let me put in other words.

As i see it multihoming means very different things for different users

Suppose a big site, which is geographically distributed, with different 
ISPs in different countries and has thousands of nodes and network 
admin team very qualified. This type of site has strong TE requirements 
and really needs centralized management. OTOH, these type of sites are 
likely to be a reduced number, so from the global routing system 
perspective, they do not have high scalability requirements (e.g. we 
could give them PI blocks and they could do BGP)

Suppose a home that has CATV and dsl line. This type of sites are 
likely to have very limited management and their TE requirements seems 
to be much more reduced (they don't have links in different countries 
to choose from). OTOH, the number of potential multihomed sites of this 
type for the medium term future may be really high, so the scalabilty 
requirements of a solution for this type of sites will be really 
important (we cannot give them PI and BGP, first because the 
contribution to the global routing table would be very high and second, 
they are no likely to be  able to configure BGP)

So, multihoming solution for big sites and for small sites have very 
different requirements both from the capabilities they need to provide 
and from the global routing system scalability p.o.v. (I am not even 
considering ISP multihoming, which would result in another different 
set of requirements i guess)
So, trying to provide a single solution that fits all the types of 
multihoming sites imho is a much more complex problem than having 
different solutions for different type of multihomed sites.

Perhaps it is a more accessible approach to try to identify the 
requirements for the different type of multihomed sites (see Fred's 
Baker draft on multihoming analysis for example) and then try to define 
solutions for each of them

---

From: Roland Dobbins <[email-masked]>
Date: Sat, 30 Dec 2006 15:04:08 -0800

On Dec 30, 2006, at 2:55 PM, marcelo bagnulo braun wrote:

> Baker draft on multihoming analysis for example) and then try to  
> define solutions for each of them

Sure, this is what folks do, anyways.  But it's important that any  
general solution does not impose unwarranted and burdensome  
requirements on the global Interent.  I believe that SHIM6 and  
similar proposals do this, and so I don't want to see this posited as  
'the multihoming solution'; neither do I care if some particular  
multihomed organization wishes to implement SHIM6 on its own network,  
just don't force SHIM6 or anything like it on me as a base  
requirement for multihoming.

     		   -- Milton Friedman

---

From: Andrew McGregor <[email-masked]>
Date: Sun, 31 Dec 2006 20:06:43 +1300

On 31/12/2006, at 7:55 AM, Roland Dobbins wrote:

>> have, I disagree.  They're very easy to use, in fact can be  
>> largely ignored once in operation.  Now, it is true that in a  
>> multi-homed situation path selection may be a problem, but I  
>> believe simply paying attention to ECN notification would be  
>> sufficient to allow a good balance between administration and ease  
>> of use.
>
> like iACLs, SNMP ACLs, vty ACLs, firewall rules, QoS policies,  
> etc., and what would be required to both support those things in  
> hardware and simply maintain/update them from an administrative  
> standpoint, and the ever-present help-desk burden.

Right, I see.  Well, here's one area where HIP has a significant  
advantage over SHIM6.  The concept of an LSI helps a great deal in  
tidying this up from the point of view of the endpoints, and many of  
the things we do now (DHCP binding to MAC addresses, for instance)  
still work for the infrastructure.  I don't regard the issues as  
particularly difficult; yes, things do get a bit different, but the  
overhead doesn't look nearly as bad as you seem to fear.

> protocols.  I also believe that this loosely-coupled paradigm,  
> while being an asset during the adoption phase, has now become a  
> millstone round our necks, because it has led us to rely on Rube  
> Goldbergian mechanisms such as ACLs, firewall rules, etc. due to  
> the lack of coupling and feedback mechanism for establishing  
> identity/entitlement/authentication/authorization up and down the  
> stack.  The current conflation of host and locator information in  
> TCP/IP (both IPv4 and IPv6) contributes to this brittleness.

Yes, exactly.  There's been a lot of discussion about this around the  
HIP space.  There is far too much use of IP address based  
authentication.  IMO a network that relies on that is badly designed  
anyway, in that doing this creates administrative work.

>
> As long as we are stuck with what are essentially approximated out- 
> of-band mechanisms for trying to implement policy, the  
> administrative burden and potential for outages related to mistakes  
> during the administrative process, and as long as we must have help- 
> desks for enterprise users and SP customers, any proposal which  
> mandates multiple IP addresses and endpoint routing decisions is  
> DOA, IMHO.

Certainly those issues are problems... however, AFAICT there are  
solutions to the problems of securing networks that do not require  
fixed IP addresses and that have already been specified by the IETF  
and other organisations.  There's nothing in TLS or SASL that needs  
constant IP addresses, for example.  That network administrators  
choose other ad-hockery to deal with those issues is really their own  
problem.  In my mind it has been clear for more than a decade that it  
is best to configure systems such that you don't need to have fixed  
IP addresses in configurations...

Andrew

---

From: Andrew McGregor <[email-masked]>
Date: Sun, 31 Dec 2006 20:10:42 +1300

On 31/12/2006, at 9:29 AM, Roland Dobbins wrote:

>> network doesn't have ACL, no TCAM for storing filters no MAC  
>> tables and a very simple firewall.
>
> essentially one protocol set/paradigm, and larger sites another,  
> with all that entails in terms of fragmenting the global Internet,  
> be for quite some time unless/until MANET-type networking becomes  
> the norm, and probably not even then in the sense of multiple  
> simultaneous links, but rather in a failover type of scenario.   
> It's a bit more blurry in terms of some mobile devices, but the  
> direction seems to be devices which can opportunistically leverage  
> whatever the 'best' network is at any given moment (based upon  
> performance, cost, administrative policies, content/service  
> availability, etc.).

In other words, mobile hosts end up multihomed, by default.  Since  
this is the majority of workstations by now (check sales of laptops)  
and will likely be cellphones as well, the solutions that work for  
those nodes will become ubiquitous pretty quickly.

> previously at length precludes *mandating* multiple IP addresses,  
> it is a nonstarter.

I believe that due to the preponderance of highly mobile devices,  
that argument will very quickly become irrelevant.

Andrew

---

From: marcelo bagnulo braun <[email-masked]>
Date: Sun, 31 Dec 2006 13:21:32 +0100

El 31/12/2006, a las 0:04, Roland Dobbins escribió:

> Sure, this is what folks do, anyways.

good, so, how many different multihoming solutions do we need? i.e. 
what type of multihoming sites with different requirements do you 
identify?

>   But it's important that any general solution does not impose 
> unwarranted and burdensome requirements on the global Interent.

good point
actually, so far, the main emphasis was that the solution does not 
impose additional burden on the routing system, but i agree with you 
that this should be extended to other domains
Regards, marcelo

>  I believe that SHIM6 and similar proposals do this, and so I don't 
> want to see this posited as 'the multihoming solution'; neither do I 
> care if some particular multihomed organization wishes to implement 
> SHIM6 on its own network, just don't force SHIM6 or anything like it 
> on me as a base requirement for multihoming.
>

---

From: "Spencer Dawkins" <[email-masked]>
Date: Sun, 31 Dec 2006 06:19:40 -0600

And, of course, I may not understand every part of the concern, but ...

I remember two general areas in discussion - a specific concern (the one 
Marshall is trying to understand) that botnet attacks might be more effect 
on transit networks if path selection is under host control, and a more 
general concern that SHIM6 gives botnetters another tool for attacks from 
hosts with security that generally reeks, so no telling how that tool might 
be exploited in the future.

I know I'm not the only person who was listening at NANOGs recently. Does 
this sound familiar to anyone else?

Thanks,

Spencer

From: "Marshall Eubanks" <[email-masked]>
To: "Jari Arkko" <[email-masked]>
Cc: "Spencer Dawkins" <[email-masked]>; <[email-masked]>
Sent: Saturday, December 30, 2006 6:46 AM
Subject: Re: other requirement? (was Re: [RAM] Traffic Engineering

> I too may be missing something, but isn't the idea behind SHIM6 is  that 
> the host
> is multi-homed ? It has locators in two or more domains, so it can  select 
> paths.
> from provider A to provider B and back worse than just blasting  packets 
> out of provider A, which can
> be done now ?
>
>> --Jari
>>
>
> Regards
> Marshall
>
>>