Subject: Re: one size doesn' fit all (was Re: other requirement? (was Re: [RAM] Traffic Engineering

From: Roland Dobbins <[email-masked]>
Date: Wed, 3 Jan 2007 07:05:25 -0800

On Jan 3, 2007, at 6:41 AM, Pekka Savola wrote:

> Maybe you should elaborate in which ways exactly the pain will get  
> significantly (more than double) worse than it currently is.

1.	Multihoming doesn't necessarily mean just two connections - it  
means => 2.

2.	Hardware capacity isn't unlimited (isn't that one of the reasons  
why we're all here in the first place, heh?)

3.	Besides the multiple-IP issue and having to maintain 2 or more  
ACEs or firewall rules or whatever for each host
	(and QoS mappings, etc.), there's the issue of being forced to  
renumber and churn all those things based upon
	the vicissitudes of transient business relationships with various  
SPs.  And of course the TE issue internal
	to the enterprise itself.

4.	Even if the pain were only to double, that's not within our  
remit.  Our goal must be to fix this problem in our own
	backyards, without unduly burdening the multihoming customer base.

SHIM6 is a classic example of cost-shifting in terms of OPEX, CAPEX,  
complexity, and supportability.  It's simply not practical from  
either a technical or an economic standpoint, and enterprises (i.e.,  
the multihoming customer base) won't hold still for it.

-----------------------------------------------------------------------
Roland Dobbins <[email-masked]> // 408.527.6376 voice

		All battles are perpetual.

     		   -- Milton Friedman

_______________________________________________
RAM mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ram

---

From: Pekka Savola <[email-masked]>
Date: Wed, 3 Jan 2007 16:41:53 +0200 (EET)

On Sun, 31 Dec 2006, Roland Dobbins wrote:
> I mean 'mandating multiple identifiers of the type which go into ACLs, 
> firewall rules, QoS rules, etc. for filtering/matching packets' is out.

Being a network administrator myself, I have to deal with IP addresses 
and update them regularly a lot -- I agree this is a significant pain.

But I have difficulty seeing why a model with multiple addresses would 
be _signficantly_ worse than the current one.  Aren't we just talking 
about (roughly) two times the number of ACL entries, firewall rules, 
QoS entries, etc..  Doubling (at most: in many cases both would be 
updated at the same time, reducing the burden) an effort is still in 
the same order of magnitude.  In the end, this might even encourage 
better automation of places where IP addresses are managed so you 
might even save some time ;-)

In my book, if the pain is currently 'huge', double the pain is still 
just 'huge' pain.

-- 
Pekka Savola                 "You each name yourselves king, yet the
Netcore Oy                    kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings

---

From: "Bound, Jim" <[email-masked]>
Date: Wed, 3 Jan 2007 10:17:24 -0500

Roland,

Your not arguing that hosts should only have one set of locators are
you?  Just checking?

thanks
/jim 

> -----Original Message-----
> From: Roland Dobbins [mailto:[email-masked]] 
> Sent: Wednesday, January 03, 2007 10:05 AM
> To: [email-masked]
> Subject: Re: one size doesn' fit all (was Re: other 
> requirement? (was Re:[RAM] Traffic Engineering
> 
> 
> 	(and QoS mappings, etc.), there's the issue of being 
> forced to renumber and churn all those things based upon
> 	the vicissitudes of transient business relationships 
> with various SPs.  And of course the TE issue internal
> SHIM6 is a classic example of cost-shifting in terms of OPEX, 
> CAPEX, complexity, and supportability.  It's simply not 
> practical from either a technical or an economic standpoint, 
> and enterprises (i.e., the multihoming customer base) won't 
> hold still for it.
>

---

From: Roland Dobbins <[email-masked]>
Date: Wed, 3 Jan 2007 07:19:02 -0800

On Jan 3, 2007, at 7:17 AM, Bound, Jim wrote:

> you?

No - just one set of 'things which must go into ACLs, firewall rules,  
QoS policies', etc.

     		   -- Milton Friedman

---

From: Ralph Droms <[email-masked]>
Date: Wed, 03 Jan 2007 10:20:31 -0500

Might be a problem if the multiplier is > 2 (suppose the organization uses
ULAs, too; throw some VPNs in the mix?).  Hm, I wonder if the problem might
be n^2 counting src/dst pairs?

Roland - are you supposing that multiplying the number of addresses managed
in network elements by n (where n is a small integer) might start to exceed
some internal limits in the network elements.  Are customers going to have
to come back to us (Cisco) for more memory?  Or is it an administrative
headache that could be managed with better NM tools?

- Ralph

---

From: Pekka Savola <[email-masked]>
Date: Wed, 3 Jan 2007 17:28:24 +0200 (EET)

On Wed, 3 Jan 2007, Roland Dobbins wrote:
> 1.	Multihoming doesn't necessarily mean just two connections - it means 
> => 2.

In most cases, I suspect two covers 95%+ of multihomers.  The rest are 
certain to get PI based solution in any case so for them the number is 
likely 1.

> 2.	Hardware capacity isn't unlimited (isn't that one of the reasons why 
> we're all here in the first place, heh?)

When you push the limits of your hardware, you'll either justify 
better vendor or hardware or change the design (e.g., to use jumphosts 
for infrastructure ACLs etc) to accommodate the hardware restrictions.

Again, if we are talking about merely doubling the size, this is not a 
concern.  This mailing list would not have been set up if folks were 
pissing their pants by the prospect of 400K routes (and that's it) in 
their routers.  Churn and 1M, 10M or 100M is more of a corcern.

> 3.	Besides the multiple-IP issue and having to maintain 2 or more ACEs 
> or firewall rules or whatever for each host

"Don't push if there's resistance".  You make it sound like frequent 
renumbering, just to save 100$ a month or year by switching SPs once a 
year (or even more frequently) is a good idea.  Guess what -- that's 
the tradeoff.  Some folks in fact do calculate that switching SPs (and 
getting savings) is not worth the cost of renumbering and hence don't 
switch SPs (or do so in a manner that doesn't require renumbering).

In any case, renumbering already applies in those SOHO like contexts 
today with IPv4 and I see no difference in this context.  Big sites 
use PI space or maybe ULAs.  Some SME-like enterprises are likely 
happy enough with their PA space and don't renumber frequently.

> 4.	Even if the pain were only to double, that's not within our remit. 
> Our goal must be to fix this problem in our own

As already discussed, there's already pain somewhere else.  Currently 
some folks are pushing their pain to the routing system.  Shim6 is 
trying to help a subset of multihomers by offering another choice. 
Whichever approach is chosen by the site is mostly up to the site.

Maybe you missed the news that SHIM6 doesn't solve -- and isn't 
intended to solve -- every site's multihoming problems.

---

From: Roland Dobbins <[email-masked]>
Date: Wed, 3 Jan 2007 07:29:00 -0800

On Jan 3, 2007, at 7:20 AM, Ralph Droms wrote:

> to come back to us (Cisco) for more memory?

Yes, for all vendors.  Memory, TCAM capacity for processing of ACLs  
in hardware, whatever.  In many cases, things which aren't field- 
upgradable except via forklift.

> Or is it an administrative

Yes, but since there aren't adequate NM tools for the numbers of ACLs/ 
firewall rules/QoS policies/etc. we have today, it's hard to  
extrapolate that those tools will sprint into existence tomorrow  
which will help solve this problem for even larger numbers of those  
things, either, heh.

With SHIM6, CAPEX, OPEX, and complexity costs (along with loss of TE  
capabilities) are simply shifted from SPs to their multihomed  
enterprise customers.  It is not a real solution, and enterprises  
won't go for it because of these reasons.

     		   -- Milton Friedman

---

From: "Bound, Jim" <[email-masked]>
Date: Wed, 3 Jan 2007 10:32:53 -0500

Thanks
/jim 

> Sent: Wednesday, January 03, 2007 10:19 AM
> To: [email-masked]
> requirement? (was Re:[RAM]Traffic Engineering
> 
> 
> No - just one set of 'things which must go into ACLs, 
> firewall rules, QoS policies', etc.
>

---

From: Roland Dobbins <[email-masked]>
Date: Wed, 3 Jan 2007 07:42:26 -0800

On Jan 3, 2007, at 7:28 AM, Pekka Savola wrote:

> Again, if we are talking about merely doubling the size, this is  
> not a concern.  This mailing list would not have been set up if  
> folks were pissing their pants by the prospect of 400K routes (and  
> that's it) in their routers.  Churn and 1M, 10M or 100M is more of  
> a corcern.

Maybe it's not a concern for -you-.  But we can't blithely say that  
it isn't a concern for others - quite the opposite, in fact.  And as  
Ralph points out, this can end up being an n^2 problem quite easily.   
It's also important to remember that there's an OPEX/admin/helpdesk  
problem here, too.

[ Some folks on this list have already gone through outages and  
hardware churn triggered when the global routing table exceeded 128K;  
they're not going to be pleased when they hit other limitations in  
other hardware from various vendors, either, at thresholds which are  
still lower than even 1M. ]

As tli already pointed out, it's highly desirable to have a solution  
which works for all classes of sites, if size and internal complexity  
are the only differentiators we can identify.  So far, no one's put  
forth any objective criteria upon which to predicate a split.

The sudden tolerance for PI space gratifies/puzzles me, though - I  
thought it was anathema, and that folks here aren't't in the main  
amused that PI space is being allocated.  If PI space is now  
tolerable, then we can all go home and call it a day, and wait on  
Moore's Law (along with open chequebooks) to save us.  I doubt that's  
the case, though.

     		   -- Milton Friedman

---

From: Eliot Lear <[email-masked]>
Date: Wed, 03 Jan 2007 17:12:31 +0100

Roland Dobbins wrote:
> Yes, but since there aren't adequate NM tools for the numbers of 
> ACLs/firewall rules/QoS policies/etc. we have today, it's hard to 
> extrapolate that those tools will sprint into existence tomorrow which 
> will help solve this problem for even larger numbers of those things, 
> either, heh.
>

Let me say this again: SHIM6 is NOT what will cause your TCAMs to 
explode in size.  It is the act of multihoming itself and propagating a 
new prefix that does that.  In fact WITHOUT multihoming multiple 
prefixes are likely to be propagated.  In fact the problems that you 
mention exist in any circumstance in which a network will be 
renumbered.  No one can reasonably expect to renumber anything beyond a 
SOHO network all at once.

Eliot

---

From: Roland Dobbins <[email-masked]>
Date: Wed, 3 Jan 2007 08:20:50 -0800

On Jan 3, 2007, at 8:12 AM, Eliot Lear wrote:

> explode in size.

If we force enterprises to double/treble/n^2 the number of ACLs/ 
firewall rules/QoS policies they have, it will cause -their- TCAMs to  
explode, -their- memory to be exhausted, etc.

If you don't believe me, we can sit down in a lab and push the  
maximum number of ACL entries into any platform/linecard you like,  
and then we can shove enough additional ACEs into it that it either  
a) starts software-switching or b) somewhat non-deterministically  
decides which entries to shove out in order to make room for the new  
ones.  I've experienced both in production networks, and wasn't  
amused - and that's without SHIM6.

This isn't a Cisco-specific issue; all hardware has limitations of  
one type or another, at one scaling-factor or another, no one's  
hardware has infinite capacity for -anything-, including ACEs/ 
firewall rules/QoS policies/etc.  The focus on IPv6 multihoming has  
been pretty much exclusively on the hardware costs an exploding  
routing table would impose upon SPs; my point is that SHIM6 simply  
shifts those costs, with a bit of transmutation, onto the bank  
accounts of enterprises.  And everyone's support-desk costs will go  
up due to the additional complexity, in addition to all the other  
issues.

     		   -- Milton Friedman

---

From: Peter Sherbin <[email-masked]>
Date: Wed, 3 Jan 2007 08:24:11 -0800 (PST)

> The sudden tolerance for PI space gratifies/puzzles me

Is there anything fundamentally wrong with AS hierarchy edged by
PoPs/APs(wired/wireless) where ASN is added to a source PI address for global
routing?

Thanks,

Peter

> Moore's Law (along with open chequebooks) to save us.  I doubt that's  

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com

---

From: Bruce Curtis <[email-masked]>
Date: Wed, 3 Jan 2007 13:12:10 -0600

On Jan 3, 2007, at 8:41 AM, Pekka Savola wrote:

>
>
> But I have difficulty seeing why a model with multiple addresses  
> would be _signficantly_ worse than the current one.  Aren't we just  
> talking about (roughly) two times the number of ACL entries,  
> firewall rules, QoS entries, etc..  Doubling (at most: in many  
> cases both would be updated at the same time, reducing the burden)  
> an effort is still in the same order of magnitude.  In the end,  
> this might even encourage better automation of places where IP  
> addresses are managed so you might even save some time ;-)

   And might it be possible for a vendor to implement a macro or  
variable that is used in rules or ACLs to represent the multiple  
prefixes?  If so the number of rules that humans must work with would  
not necessarily double.  (But the equipment may still need to deal  
with double the memory etc.)

---
Bruce Curtis                         [email-masked]
Certified NetAnalyst II                701-231-8527
North Dakota State University

---

From: Roland Dobbins <[email-masked]>
Date: Wed, 3 Jan 2007 11:19:02 -0800

On Jan 3, 2007, at 11:12 AM, Bruce Curtis wrote:

> If so the number of rules that humans must work with would not  
> necessarily double.  (But the equipment may still need to deal with  
> double the memory etc.)

The number of representational objects or attributes of said objects  
humans must account for would certainly double or treble or n^2 or  
what-have-you, and the equipment must have sufficient resources to  
handle the additional entries/representations, as you indicate.

But this is an implementation issue which will vary from vendor to  
vendor, platform to platform.  Not all vendors/platforms will have  
such capabilities, and the resource issue remains the same, so we  
really can't push a solution which imposes this kind of burden.

That's OK, though, because it sounds as if some of the models folks  
are beginning to think about wouldn't impose this sort of burden.

     		   -- Milton Friedman

---

From: David Meyer <[email-masked]>
Date: Thu, 4 Jan 2007 09:43:56 -0800

> thought it was anathema, and that folks here aren't in the main  

	I don't know who has "tolerance" for the current RIR
	policies; there is just little choice, as we haven't
	built a technology that avoids teh "tyranny of CIDR"
	(Vix's term, referring to CIDR provider lock). BTW, its
	far from clear that Moore's Law will save anything;
	that's different than saying that we can build <X> (for
	some X; well, than and the fact that Moore's Law just an
	empirical observation). 

	In any event, the Moore's Law "line of reasoning" turns
	out to be such a massive rathole that I'd prefer we stay
	away from that one (again, to the extent its possible).

	--dmm

---

From: Marshall Eubanks <[email-masked]>
Date: Thu, 4 Jan 2007 13:03:51 -0500

>> Moore's Law (along with open chequebooks) to save us.  I doubt that's

As one of the instigators of this "tolerance" (ARIN 2002-3 and  
2005-1), I think that it is widely misunderstood.

The contentions were that
1.) there are some small entities that needed to multi-home, and thus  
needed (small amounts of)
PI space,
2.) that they were doing it anyway, and actually injecting at least  
as much and probably more into the routing tables than they would  
under its adoption, and also having to deal with being black-holed  
and other problems on an
ad hoc basis and
3.) _that this was a relatively small usage that would have a small  
to negligible effect on the actual routing table expansion_.

To date, I think that all three contentions have been proven by  
practice and this has worked rather well. This is discussed from time  
to time on the ARIN PPML, and AFAIK no data has been introduced to  
the contrary. This was (in my opinion) not an attempt to subvert the  
system, but rather to make it work better for a wider class of users.

If you want to claim that this is a general threat, please produce  
data supporting your assertions.

Regards
Marshall

---

From: Roland Dobbins <[email-masked]>
Date: Thu, 4 Jan 2007 10:07:59 -0800

> data supporting your assertions.

To be clear, I think this policy hack is a Good Thing, especially  
whilst we work on this solution-set - I was referring to apparent  
tolerance on the part of the poster to whom I was responding, in a  
TIC sort of manner.

		          Technology is legislation.

                     -- Karl Schroeder

---

From: David Meyer <[email-masked]>
Date: Thu, 4 Jan 2007 10:09:08 -0800

Marshall,

> 2005-1), I think that it is widely misunderstood.

	I understand. However, the fact remains that PI space is
	more or less readily available. What is looming then, is
	a swamp that has the potential to be many orders of
	magnitude larger than what we faced with v4 (something
	about learning the lessons of history comes to mind).

	In any event, I'm not "blaming/critizing/whatever"
	anyone; there just wasn't another solution available, and
	that (IMO) is on us. 
	
	--dmm

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 05 Jan 2007 15:30:43 +0100

On 2007-01-03 16:42, Roland Dobbins wrote:
...
> thought it was anathema, and that folks here aren't't in the main amused 
> that PI space is being allocated.  If PI space is now tolerable, then we 
> can all go home and call it a day, 

I think we should be clear that today we appear to have to choose between
(1) PI prefixes, which break aggregation, and (2) locator rewriting, which
allows aggregation. Locator rewriting comes in many forms*, some of which
are still theoretical, but they are all isomorphic at some level.

If anybody knows a third choice, that would be of great interest.

*e.g. NAT, map and encap, GSE, shim6

     Brian

---

From: Bruce Curtis <[email-masked]>
Date: Fri, 5 Jan 2007 12:02:59 -0600

On Jan 5, 2007, at 8:30 AM, Brian E Carpenter wrote:

>> tolerable, then we can all go home and call it a day,
>

   Well including all of the destination locators in a packet could  
work without locator rewriting in theory but I think most such  
proposals envision the possibility of rewriting the primary locator  
in at least the case of when the route to the primary locator is down  
for compatibility with existing equipment.

   Maybe (2) should be "Multiple global locators, which allows  
aggregation."  And locator rewriting is a subset of (2) which has  
been most discussed.  Today NAT is a sort of locator rewriting and it  
doesn't allow aggregation.

---

---

From: Peter Sherbin <[email-masked]>
Date: Fri, 5 Jan 2007 10:55:38 -0800 (PST)

> (1) PI prefixes,... (2) locator rewriting,... a third choice:

All things have globally unique id/names. Only routers have aggregated addresses.

A thing discloses its name to a router it is attached to. The router maintains a
name database of things attached. The router advertises upstream its address only.

A router on a private network presents itself to an upstream router as a thing with
a name.

An open contract between providers and subscribers allows billing for traffic from a
given name. Subscribers change providers at will.

Thanks,
Peter

---

From: Tony Li <[email-masked]>
Date: Fri, 5 Jan 2007 13:04:09 -0800

On Jan 5, 2007, at 10:02 AM, Bruce Curtis wrote:

> Today NAT is a sort of locator rewriting and it doesn't allow  
> aggregation.

NAT permits most excellent aggregation as long as you're willing to  
deal with PA addresses.  What really becomes cumbersome tho, is when  
multihoming is involved.  Then, hosts interacting with the hosts  
behind the NAT aren't aware of the multiple locators allocated to  
their correspondent and havoc ensues.

So, I'd rather say that NAT does rewrite locators, but in an  
uncoordinated way that defeats most of the benefits of multihoming.   
As such, it's pretty much useless as a long-term architectural strategy.

Tony

---

From: "Chris L. Morrow" <[email-masked]>
Date: Sat, 06 Jan 2007 02:22:37 +0000 (GMT)

> As such, it's pretty much useless as a long-term architectural strategy.

useless without something that allows the locators switching to not affect
the upper layer protocol state? or useless entirely? (perhaps if the upper
layer protocol state were unaffected it wouldn't be called NAT?)

---

From: Tony Li <[email-masked]>
Date: Fri, 5 Jan 2007 18:52:28 -0800

> layer protocol state were unaffected it wouldn't be called NAT?)

Useless without a distributed database that coordinates locator/ 
identifier bindings and manages available locators, at which point it  
looks a heck of a lot like something much more powerful than just  
NAT.  Shim7, anyone?  ;-)

Tony

---

From: "Chris L. Morrow" <[email-masked]>
Date: Sat, 06 Jan 2007 03:02:21 +0000 (GMT)

> looks a heck of a lot like something much more powerful than just

I think I might miss why it's 'useless', i think it'd be quite nice (say
for mobile things or even laptops) if you could easily migrate from
location to location and maintain connectivity without restarting all the
vpn/http/ssh/blah sessions I have to restart everytime my laptop moves.
Something that let my locator's change 'at will' (provide some assurance
that I'm really me during/after move, sure) would be nice. With that
doesn't NAT at my 2 ISP link points (NAT to PA space in each ISP) just
look like that sort of mobility? (essentially I move from isp to isp in
some form of rapid sucession).

I admit that I oversimplified things, but 'useless' it didn't seem.

---

From: Russ White <[email-masked]>
Date: Fri, 05 Jan 2007 23:33:45 -0500

> Useless without a distributed database that coordinates
> locator/identifier bindings and manages available locators, at which
> point it looks a heck of a lot like something much more powerful than
> just NAT.  Shim7, anyone?  ;-)

Just call it mobile IP, and be done with it! :-)

I'll be the heretic for the day, and say that I really think the main
problem with NAT isn't that it's NAT, but rather that is hits us just at
the locater/identifier divide. We'd safely hidden the dual nature of IP
addresses for years, pretending that DNS records were ID's, and IP
addresses locaters, when NAT came along and showed us the reality of the
situation.

Of course, it also doesn't help that we call Port Address Translators
(PATs) Network Address Translators (NATs), when they really aren't the
same thing at all. PATs are used to get around address space
limitations. Assuming IPv6 solves the address space issue, IPv6 + NAT,
disallowing PATs, might not be a bad thing all in all.

We should, perhaps, be a little more kind to our NATs. Just thinking out
loud....

:-)

Russ

- --
[email-masked] CCIE <>< Grace Alone

---

From: Tony Li <[email-masked]>
Date: Fri, 5 Jan 2007 21:11:57 -0800

> I admit that I oversimplified things, but 'useless' it didn't seem.

Chris,

Please first understand that we're talking about 'useless' for  
solving the multihoming problem and implementing a true loc/id  
split.  NAT is wonderful for helping to preserve address space.   
Horrible in that it violates a clean e2e architecture.

That said, even if you exist behind a NAT where there's non-trivial  
topology and you have to roam and get a new DHCP address for your  
laptop, the NAT box is going to have NO idea that it's the same  
laptop and your TCP sessions will necessarily close.

Regards,
Tony

---

From: "Chris L. Morrow" <[email-masked]>
Date: Sat, 06 Jan 2007 06:46:53 +0000 (GMT)

> laptop and your TCP sessions will necessarily close.

today it does because it needs to maintain state and because the upper
layer protocol(s) also track the locator/id as a single entity, yes :( It
seemed to me that GSE/8+8 gave an opportunity to get away from that set of
problems. (locator re-writing)

I think in the context of the larger network there is a place for both PI
and locator re-write. I'm not sure that locator re-write should be done
'anywhere' but at provider/customer edge it seems simpler to accomplish.
There will always be sites with enough complexity/money/reasons to
want/need PI space I think that has to be accepted to some extent.

---

From: marcelo bagnulo braun <[email-masked]>
Date: Sat, 6 Jan 2007 14:07:55 +0100

> vpn/http/ssh/blah sessions I have to restart everytime my laptop moves.

but NAT by itself doesn't give you that, i think, you need an 
additional protocol

> some form of rapid sucession).

right, so in order to preserve you established vpn/http/ssh connections 
you need a mobility (or a multihoming) protocol and you also need a 
mechanism to discover the public address that the nat is using from the 
PA block...

> I admit that I oversimplified things, but 'useless' it didn't seem.

i don't think that NAT are useless, especially in v6 when you can do 
one to one address mapping, but for me the key feature they provide is 
address protability avoiding provider lock in.

They do not seem to be useful by themselves to preserve established 
connections through outages in multihomed environments though (even 
they could provide some more limited fault tolerance, like allowing new 
communications after an outage)

Regards, marcelo

---

From: [email-masked]
Date: Mon, 8 Jan 2007 16:12:26 +0100

Tony,

> looks a heck of a lot like something much more powerful than just

Why don't we use the BGP table to coordinate loc/id bindings?.
This way the routing system would not depend on an external
mechanism like the DNS.

Regards,
Juanjo

---

From: Brian E Carpenter <[email-masked]>
Date: Mon, 08 Jan 2007 17:21:56 +0100

> This way the routing system would not depend on an external

Could we maybe focus on the (apparent) fact that we need a
loc/id mapping service, and figure out what it needs to do?
After that, we can consider whether it should be implemented
using DNS, BGP, or avian carriers.

    Brian