Subject: Thousands of large PI customers [Re: [RAM] Incremental Deployment of LISP]

From: Brian E Carpenter <[email-masked]>
Date: Mon, 02 Apr 2007 11:14:13 +0200

On 2007-03-30 19:49, Fleischman, Eric wrote:
...
> ... For example, my coworkers
> and I have been telling various IETF mail lists for the past decade that
> it is non-negotiable that we will solely accept PI addresses whether or
> not the IETF permits a PA-PI address distinction to exist or not. The
> world's other largest companies and governments are almost certainly
> going to make similar demands. 

I believe this simply doesn't matter from a DFZ scaling point of
view. A few thousand large PI customers is surely not an issue.

What matters is the potential for 10 million PI customers, unless
we provide an alternative approach for them. (That was the starting
point for shim6 - it isn't aimed at Boeing or IBM.)

LISP seems to lead to a world with 10M PI customers, and therefore
a PI to PA map with 10M entries.

On 2007-04-01 05:14, Ross Callon wrote:
...
> Perhaps we need a terminology distinction between "a PI address
> which is in the global routing table -- just like now", and "a PI
> address that is not in the global routing table, but needs to be
> mapped to a PA address". Perhaps "Routable PI" (RPI) and
> "Nonroutable PI" (NPI)? 

But we have that. ULAs are non-routable PI. That's exactly what they
were designed for. RFC1918 addresses are really just defective ULAs,
given the shortage of IPv4 space.

Puzzle me this. What is the difference between (a) using LISP between
two PI sites over some piece of ISP locator space and (b) a PPVPN between
those two sites?

     Brian

_______________________________________________
RAM mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ram

---

From: "Fleischman, Eric" <[email-masked]>
Date: Mon, 2 Apr 2007 08:29:49 -0700

>From: Brian E Carpenter [mailto:[email-masked]] 
>LISP seems to lead to a world with 10M PI customers, and therefore a 
>PI to PA map with 10M entries.

I understand your concern, but LISP need not be deployed in this
fashion. If the tunnel boundaries are deep within the ISP, perhaps at
their borders with other ISPs, then they will aggregate the end users
they support, much like the current system.

Unlike the current system, both the ISP and the end user are provided
flexibility that don't currently exist. With LISP, ISPs are presented
huge opportunities for market differentiation and end users are
presented an opportunity to address their infrastructures as they see
fit.

Issues that are currently the concern of the entire Internet because
everyone impacts everyone else in the current model now become business
issues negotiated between the local ISP and the end user because the ISP
(and their confederations) now become mini-Internets, modularizing the
true Internet. 

LISP needn't be an all-or-nothing solution. I envision strong market
pressures preserving L2VPN and L3VPN, in addition to supporting LISP as
the generic service for most end users.

---

From: Brian E Carpenter <[email-masked]>
Date: Mon, 02 Apr 2007 17:37:14 +0200

> they support, much like the current system.

But I think we need to design the locator mapping service to deal
with the worst case; if that is really 10M entries I am not too
frightened. It isn't the same problem as 10M DFZ routes.

> the generic service for most end users.

And my question was really about whether these are two models, or
simply two tunings of the same model.

     Brian

---

From: Roland Dobbins <[email-masked]>
Date: Mon, 2 Apr 2007 08:40:24 -0700

> frightened. It isn't the same problem as 10M DFZ routes.

Here's one extrapolation of the possible future in this regard:

<http://www.nanog.org/mtg-0606/fuller.html>

-----------------------------------------------------------------------
Roland Dobbins <[email-masked]> // 408.527.6376 voice

         Words that come from a machine have no soul.

                       -- Duong Van Ngo

---

From: "Fleischman, Eric" <[email-masked]>
Date: Mon, 2 Apr 2007 09:31:13 -0700

>From: Roland Dobbins [mailto:[email-masked]] 
>Does a LISP-like solution lend itself to MANET?  

Since I've been working with MANET for many years now, this is a point
that I have thought about quite a bit. In my personal opinion (i.e., I
don't expect you to agree), the historic IP protocol is inherently
designed for stable environments and becomes severely challenged by
MANET attributes (e.g., mobility, signal intermittence, transitory
relationships). The two (i.e., traditional IP and MANET) can (and do)
coexist, but their interface point(s) needs to satisfy the requirements
of both communities. 

It is my personal opinion (i.e., I again don't expect you to agree) that
because BGP is particularly poorly suited for supporting MANETs (e.g.,
pairwise relationships, no discovery), that a natural place to relate
MANETs with traditional IP is at the AS boundary, because to extend
MANETs beyond the AS boundary needs to overcome the vast problems that
BGP has in highly mobile, highly signal intermittence environments. 

However, regardless of where the boundary is placed, the MANET entity
needs to be represented to the larger stable network environment from a
node that appears to be stable to the stable environment, and it is
desirable that the MANET reachability information reported from that
node to the stable environment is dampened (e.g., abstracted) to reduce
the detrimental affects of the MANET environment upon the stable
environment (e.g., route churning). ISPs are currently a key part of the
"stable environment" and to interface with them, the MANET BGP interface
node and the MANET community it represents needs to be reported to
appear to be as stable as possible. This is true for all Internet models
using BGP interfaces, including LISP.

>Is MANET at some point 
>in the foreseeable future going to become the norm, with complete 
>disintermediation between what we now think of as SPs and customers?

MANET is already the norm in some communities, but practitioners of IP
are highly incentivized to restrict MANET deployments to boundaries that
are as clearly defined as possible, because of the challenges that OSPF,
IS-IS, BGP, PIM-DM, PIM-SM, etc. have in supporting MANET, particularly
as a MANET community itself scales to hundreds or thousands of nodes.

---

From: "Fleischman, Eric" <[email-masked]>
Date: Mon, 2 Apr 2007 09:40:44 -0700

>Does a LISP-like solution retain the concept of a single global 
>Internet?  If not, have we 'outgrown' that concept due to various 
>technical, business, and political reasons?

My concern is that I have been unable to articulate what our current
Internet Architecture is. I spent the entire Vancouver IETF privately
asking our leadership for their vision, and I became very discouraged
from my queries. 

In many ways, LISP is nothing other than a formalization of the current
IPv6-IPv4 migration techniques. The difference, of course, is that the
current migration techniques postulate an eventual IPv6 end state. 

If you believe that a pure IPv6 future will occur, then LISP could be a
possible transition state for you, but not an appropriate final state. 

However, if you believe as I do that current IPv6 design is inherently
self-defeating, then LISP represents a possibly achievable replacement
for our current mythology and a possible realistic basis to become our
current Internet Architecture.

---

From: Dino Farinacci <[email-masked]>
Date: Mon, 2 Apr 2007 09:56:44 -0700

> frightened. It isn't the same problem as 10M DFZ routes.

I am actually not worried about the aggregateability of the EID- 
prefix space. Since it will be based on an allocation hierarchy and  
not a topology heirarchy.

What I do worry about is the locator-set placement which can cause  
deaggregateability of the allocated EID-prefix block.

Dino

---

From: Roger Jorgensen <[email-masked]>
Date: Mon, 2 Apr 2007 19:02:09 +0200 (CEST)

sorry if this post is a bit off topic but one question popped into my head 
when I´ve been reading through the latest posts made on this list..

On Mon, 2 Apr 2007, Fleischman, Eric wrote:
<snip>
> current Internet Architecture.

What has to be changed with IPv6 to avoid it being self-defeating?

-- 

------------------------------
Roger Jorgensen              | - ROJO9-RIPE  - RJ85P-NORID
[email-masked]           | - IPv6 is The Key!
-------------------------------------------------------

---

From: "Fleischman, Eric" <[email-masked]>
Date: Mon, 2 Apr 2007 10:03:29 -0700

>> pressures preserving L2VPN and L3VPN, in addition to supporting LISP 
>> as the generic service for most end users.

>simply two tunings of the same model.

This is an excellent question, Brian. In my mind, they are two very
different models, and I wonder if different answers to this question
doesn't underlie some of this discussion? 

I believe that they are two models because we currently use L3VPN (or
L2VPN) to connect geographically distributed parts of our own company
together. For example, we arrange with our ISPs to support MLPS paths
across their infrastructures so that a part of our company in Region A
can talk to a part of our company in Region B.

I imagine that L3VPN can also be used to link together privately defined
confederations of end users from different ASes into a private network. 

Regardless, the L2VPN and L3VPN service is what I refer to as "range
extension" because it appears to us to be a logical link from our point
of view. This logical link can be located at any level of the IP
Topology hierarchy.

However, I believe that LISP is fundamentally different. LISP is what I
call "reach back" because it provides a mechanism for ASes to
communicate together via BGP. That is, LISP represents an implementation
of the network of networks and provides a logical mesh supporting formal
BGP connectivity between ASes.

---

From: "Fleischman, Eric" <[email-masked]>
Date: Mon, 2 Apr 2007 10:16:19 -0700

From: Roger Jorgensen [mailto:[email-masked]] 
>What has to be changed with IPv6 to avoid it being self-defeating?

The architecture must not require widespread end user deployment in
order to work, as it currently does -- either that, or there must be a
truly compelling reason from the end user perspective for the end user
to want to support IPv6. 

IPv6 was originally designed to solve ISP problems to which the end user
has no visibility.

IPv6 represents an expense for the end user but does not offer any
functionality to justify that expense -- there is no killer app for
IPv6. Until a killer app emerges, the end user is naturally motivated to
avoid introducing non-value-added variability to their infrastructure
and thereby significantly increase their own operations and support
costs.

Worse yet, until IPv6 permitted PI addresses, it represented a mechanism
that required large end users to be held captive to their ISPs. This is
untenable to any end user and violates a fundamental law of economics
that businesses support the needs of their customers. Rather, the model
originally used by IPv6 was that customers support the needs of their
vendors. Not!!!

---

From: David Conrad <[email-masked]>
Date: Mon, 2 Apr 2007 10:23:15 -0700

Hi,

On Apr 2, 2007, at 9:40 AM, Fleischman, Eric wrote:
>> Internet?

We gave "a single global Internet" up when IPv6 was standardized  
since it created a parallel ("dual stack") Internet that uses the  
same domain name tree.

> current Internet Architecture.

IPv6, as defined, is little more than IPv4 with more bits.  As lots  
of people pointed out lots of times, the size of the address wasn't  
really the problem, rather the scalability of the routing system  
was.  We're now trying to come up with a solution to the scalability  
problem that doesn't involve "use more thrust".  LISP and its ilk are  
the first realistic attempts to actually deal with scaling the  
Internet up.

Rgds,
-drc

---

From: "Templin, Fred L" <[email-masked]>
Date: Mon, 2 Apr 2007 10:37:01 -0700

I agree with Eric, but I would add that it doesn't have to
be particularly mobile or even ad-hoc to be considered a
MANET. Here is the way I am defining MANET in my I-Ds:

  "(A MANET is) a connected network region that comprises
   MANET routers that maintain a routing structure among
   themselves over links with asymmetric reachability
   characteristics (see: [RFC2461], Section 2.2).  A MANET
   may be as large as an Autonomous System (AS) or as small
   as an individual site, and may also be a subnetwork of a
   larger site.  A MANET router (and its downstream-attached
   links) is a "site" unto itself, and a MANET is therefore
   a "site-of-sites"."

So, the types of MANETs that Eric is describing are covered
under this definition, but so are home networks,  vehicular
platforms, "sites" within large enterprises, large enterprises
themselves, etc. etc. It just happens that these more static
and/or managed examples are really just special cases of MANETs.

Fred
[email-masked]

> -----Original Message-----
> From: Fleischman, Eric 
> Sent: Monday, April 02, 2007 9:31 AM
> To: [email-masked]; [email-masked]
> Subject: [RAM] MANETs and LISP
>

---

From: Tony Li <[email-masked]>
Date: Mon, 2 Apr 2007 10:39:42 -0700

> deaggregateability of the allocated EID-prefix block.

Aggregate-ability of locators is wholly irrelevant in a push model,  
since the entire database must be distributed anyway.

Have we converged on only using a pull model for the mapping?

Tony

---

From: Roland Dobbins <[email-masked]>
Date: Mon, 2 Apr 2007 10:40:09 -0700

On Apr 2, 2007, at 10:16 AM, Fleischman, Eric wrote:

> IPv6.

The only caveat I would add is spimes/supply-chain/mobility/DCN  
address exhaustion.

                       -- Duong Van Ngo

---

From: Roland Dobbins <[email-masked]>
Date: Mon, 2 Apr 2007 10:42:51 -0700

On Apr 2, 2007, at 10:23 AM, David Conrad wrote:

> As lots of people pointed out lots of times, the size of the  
> address wasn't really the problem, rather the scalability of the  
> routing system was.

The address-space issue is more pertinent today than it was when CIDR  
was posited.

                       -- Duong Van Ngo

---

From: David Conrad <[email-masked]>
Date: Mon, 2 Apr 2007 10:48:23 -0700

Eric,

> truly compelling reason from the end user perspective for the end user

Or perhaps, the architecture (and changes to that architecture) must  
be essentially transparent to the end user and their supporting  
organizations.

> IPv6 was originally designed to solve ISP problems to which the end  

Not really.  IPv6 was designed to solve a political problem (namely,  
OSI and the ITU's assumption that it would control the future of data  
communications).  Governments and industry were convinced the amount  
of address space represented by 32 bits was a problem and the IETF  
community needed a response to NSAPs.

As far as I can tell, ISPs didn't really care all that much about the  
amount of address space.  They still don't.  What they appear to care  
about is minimizing cost of infrastructure and the big concern  
driving RAM is that the cost of providing that infrastructure may  
(there is still some argument) grow super-linearly.

> and thereby significantly increase their own operations and support

To clarify, I'd argue that the vast majority of end users couldn't  
care less whether they had IPv4, IPv6, NSAPs, or lima beans for  
addresses.  What end users care about is continued accessibility to  
network services of various flavors and kinds.  Until this can be  
provided over IPv6, with no need for any significant end user (or end  
user supporting organization) configuration changes, IPv6 will remain  
dead in the water.

> originally used by IPv6 was that customers support the needs of their

Yep.

Rgds,
-drc

---

From: "Fleischman, Eric" <[email-masked]>
Date: Mon, 2 Apr 2007 10:48:49 -0700

>The only caveat I would add is spimes/supply-chain/mobility/DCN address

>exhaustion.

That's why we have NAT. It's a sad state of affairs that NAT is cheaper
and more practical for the end user today than IPv6 is. 

Concerning IPv6 mobility, QoS, security, etc. (i.e., all of the claimed
IPv6 technical advantages):  Well, maybe if you are using Mobile IP
there are some advantages -- but enough to justify a deployment???
Probably not. If you aren't using Mobile IP, then these claims are all
fairly empty promises today.

---

From: "Fleischman, Eric" <[email-masked]>
Date: Mon, 2 Apr 2007 10:59:18 -0700

>From: David Conrad [mailto:[email-masked]] 

organizations.

Yes!!! I fully agree. LISP offers this promise. Whatever the IETF
finally chooses also needs to do this, if we are going to finally have a
realistic (non-mythical) architecture.

>provided over IPv6, with no need for any significant end user (or 
>end user supporting organization) configuration changes, IPv6 will 
>remain dead in the water.

I agree except for those situations where a powerful political force
decrees that IPv6 will be deployed. In those situations, the OSI
historical precedent showed that OSI was indeed deployed -- in the most
minimal fashion permitted by the powerful decrees. OSI was primarily
deployed in real life in niche environments (e.g., Aeronautical
Telecommunications Networks, electrical industry, ...).

---

From: Roland Dobbins <[email-masked]>
Date: Mon, 2 Apr 2007 11:00:28 -0700

> and more practical for the end user today than IPv6 is.

NAT doesn't scale for spimes, supply-chain, mobility, nor for DCNs,  
and it is too rigid for these applications, as well.  Some SPs have  
deployed IPv6 on their DCNs because of this set of issues.

                       -- Duong Van Ngo

---

From: Roland Dobbins <[email-masked]>
Date: Mon, 2 Apr 2007 11:02:53 -0700

On Apr 2, 2007, at 10:48 AM, David Conrad wrote:

> Not really.  IPv6 was designed to solve a political problem  
> (namely, OSI and the ITU's assumption that it would control the  
> future of data communications).  Governments and industry were  
> convinced the amount of address space represented by 32 bits was a  
> problem and the IETF community needed a response to NSAPs.

It has been said in some quarters that IPv6 is can be viewed as  
'GOSIP Strikes Back'.

;>

                       -- Duong Van Ngo

---

From: "Fleischman, Eric" <[email-masked]>
Date: Mon, 2 Apr 2007 11:05:13 -0700

>> That's why we have NAT. It's a sad state of affairs that NAT is 
>> cheaper and more practical for the end user today than IPv6 is.

>deployed IPv6 on their DCNs because of this set of issues.

Supply chain -- lots of examples of NATs
Mobility -- depends on the architecture. MONETs have been deployed using
NATs.
Rigid? If you say so, though the number of deployments suggests that
rigidity is being somehow overcome.

---

From: "Templin, Fred L" <[email-masked]>
Date: Mon, 2 Apr 2007 11:06:50 -0700

Tony,
 
> Have we converged on only using a pull model for the mapping?

By "pull", do you mean the LISP 2/3 model instead of LISP1/1.5?
If so, I agree.

Fred

---

From: Roland Dobbins <[email-masked]>
Date: Mon, 2 Apr 2007 11:10:06 -0700

On Apr 2, 2007, at 11:05 AM, Fleischman, Eric wrote:

> If you say so, though the number of deployments suggests that

NATs are by definition topologically static.  When the soda can  
leaves the factory, is trucked to the grocery store, is put on  
another truck, and then delivered into my refrigerator, it seems to  
me that I can't reasonably expect all those various supply-chain  
networks (including one in my flat called the Internet) to be  
appropriately NATted.

                       -- Duong Van Ngo

---

From: Tony Li <[email-masked]>
Date: Mon, 2 Apr 2007 12:28:55 -0700

> frightened. It isn't the same problem as 10M DFZ routes.

It can be made to be close.  ;-)

If we settle on a push model for mapping, and allow encapsulations to  
happen at arbitrary points in the network, then this looks a great  
deal like a core FIB.

I'd prefer that we change either of the preconditions.

Tony

---

From: Roger Jorgensen <[email-masked]>
Date: Mon, 2 Apr 2007 21:51:29 +0200 (CEST)

> originally used by IPv6 was that customers support the needs of their

In short, IPv6 are just more IP addresses, nothing else. Now we´re all 
trying to solve the rest of the issues Internet have/will get.

I´m just new to this side of the IPv6 discussion and are now trying to 
figure out what the issue is people are trying to solve. Think I become a 
bit wiser now.

--

---

From: Roger Jorgensen <[email-masked]>
Date: Mon, 2 Apr 2007 22:15:19 +0200 (CEST)

On Mon, 2 Apr 2007, David Conrad wrote:
<snip>
> To clarify, I'd argue that the vast majority of end users couldn't care less 
> whether they had IPv4, IPv6, NSAPs, or lima beans for addresses.  What end 
> users care about is continued accessibility to network services of various 
> flavors and kinds.  Until this can be provided over IPv6, with no need for 
> any significant end user (or end user supporting organization) configuration 
> changes, IPv6 will remain dead in the water.

That´s all so true. I work for an organization/company owned 
indirect by the state and our only goal is to provide electronic 
infrastructure so there can be widespread use of what the world know as 
Telemedicine. We have so much trouble with IPv4 due to every possible 
reason and how they would react to IPv6...no idea.

Bottom line, our end-users, doctors working with patients everyday only 
care about one singel thing, to be able to communicate with the hospitals 
and other they need in their daily work.
IPv6 are faaaar from ready for those users even tho most of our problems 
with be solved with IPv6. Problems like NAT, double NAT and tripple NAT, 
conflicting IP addresses due to every hospital/county have their own 
network using RFC1918 addresses...

--

---

From: Russ White <[email-masked]>
Date: Mon, 02 Apr 2007 20:06:05 -0400

> BGP has in highly mobile, highly signal intermittence environments. 

One thing that disturbs me about this entire discussion is the provider
centric view of the world. Yes, providers exist in a certain form today.
They might not exist in that form, or any form, in the next ten years.
If we design the Internet around current provider modes of operation,
we'll have frozen a business model in place with no specific reason to
have done so, other than that's the business model that existed when the
music stopped.

As for MANET, if you look at the MANET as some sort of cloud that
connects to some provider, and LISP as a provider edge only solution,
then your analysis is right. In fact, as long as you see MANET as a
cloud attached to a provider, just about anything will work with MANET,
since it really doesn't run with a MANET, but on the stable network the
MANET actually attaches to.

If you try to actually figure out how LISP of any type would work _in_ a
MANET, rather than with a MANET attached to it, you'll face much larger
challenges, though. As long as the network locators remain stable, we
are being told LISP will reduce the churn in the network. I'm not at all
convinced of this, but let's run with it a second, and see where it leads.

Now, when the locators and the IDs are in constant flux, then the
mapping system between the two has to deal with churn on both sides, not
just one, and things don't line up so nicely. My guess is this would be
worse than "normal" IP, but without spending some time on a whiteboard,
it would be hard to know, for certain.

:-)

Russ

- --
[email-masked] CCIE <>< Grace Alone

---

From: Russ White <[email-masked]>
Date: Mon, 02 Apr 2007 20:21:24 -0400

> realistic (non-mythical) architecture.

What gain would an Enterprise be able to point to if they deployed LISP
at their internal edge routers?

1. They made their internal routing tables smaller. They weren't big to
begin with.

2. They reduced the churn in their routing tables, and, most likely,
increased the churn in some other table.

3. They added more software and, quite possibly, boxes to support it.

4. They made their ISP's tables smaller. Big deal.

If I'm an Enterprise customer, why am I buying this? I'm certainly
cementing the current business model of the Internet in place, but I'm
not certain that buys me much of anything, either, does it?

:-)

Russ

- --

---

From: Roland Dobbins <[email-masked]>
Date: Mon, 2 Apr 2007 17:28:59 -0700

> have done so, other than that's the business model that existed  

This is what I was trying to get at indirectly by asking about  
disintermedation and MANET.

;>

The current NSP business model has been largely shaped by the  
conflation of host/locator, and with the notion of upstream/ 
downstream/peering as distinct modes of connectivity as an indirect  
result, IMHO.  Given the impetus behind the LISP proposal in no small  
part is a reaction to various economic pressures, incentives for  
selfish behavior, etc. which surround that business model, it seems  
that spending some time thinking about whether what we're doing is  
'futureproofed' in this regard might be worthwhile.

                       -- Duong Van Ngo

---

From: Roland Dobbins <[email-masked]>
Date: Mon, 2 Apr 2007 17:32:14 -0700

> not certain that buys me much of anything, either, does it?

It may be that the primary advantage for some enterprises would be to  
say that they're Doing Something in terms of deploying/enabling IPv6,  
which they may be under various non-technical pressures to deploy,  
irrespective of need and/or direct applicability to a problem-set.

                       -- Duong Van Ngo

---

From: David Conrad <[email-masked]>
Date: Mon, 2 Apr 2007 17:47:13 -0700

Russ,

> at their internal edge routers?

- The ability to change service providers without affecting internal  
hosts.
- Not having to convince/pay the enterprise's service provider(s) to  
route an additional PI prefix (the ISP-size of the edge would, of  
course, be provider aggregatable).

The former benefit is, of course, gained by using PI (unless the RIRs  
reverse current trends and made PI (for use in routing, that is  
locators) more difficult to obtain).  However, as the routing system  
bloats, I suspect the latter benefit will come more and more into play.

Note that since end point identifiers do not need to be aggregatable  
for routing purposes, there is no reason they need to be allocated  
hierarchically.  As such, alternative allocation models could be  
used.  Not sure this is a benefit, but thought I'd mention it.

Rgds,
-drc

---

From: "Darrel Lewis \(darlewis\)" <[email-masked]>
Date: Mon, 2 Apr 2007 17:48:45 -0700

Russ White wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> 
>> finally chooses also needs to do this, if we are going to finally
>> have a realistic (non-mythical) architecture.
> 
> What gain would an Enterprise be able to point to if they
> deployed LISP at their internal edge routers?
> 
> 1. They made their internal routing tables smaller. They
> weren't big to begin with.
> 
> 2. They reduced the churn in their routing tables, and, most
> likely, increased the churn in some other table.
> 
> If I'm an Enterprise customer, why am I buying this? I'm
> certainly cementing the current business model of the
> Internet in place, but I'm not certain that buys me much of anything,
> either, does it? 
> 

As an enterprise, having PI space instead of PA space is a huge win.
Why do you think that there is so much pressure on the regestries to
provide PI space right now?  LISP with real PI ID space allows me to
handle two huge issues: 1) merging of enterprise networks, and 2)
provider independence.  These advantages seem pretty obvious to me, am I
missing something?

-D

---

From: Russ White <[email-masked]>
Date: Mon, 02 Apr 2007 20:54:14 -0400

> - The ability to change service providers without affecting internal hosts.

They already have this.

> course, be provider aggregatable).

Which SP is, today, refusing to accept business by way of refusing to
advertise PI space? I don't know of any, myself. For the most part,
providers are in business to make money, and paying customers are hard
to turn away.

To the contrary, I know of several large customers who have gone to PI
space specifically because they could not get non-PI space from their
providers.

> Note that since end point identifiers do not need to be aggregatable for
> routing purposes, there is no reason they need to be allocated
> hierarchically.  As such, alternative allocation models could be used. 
> Not sure this is a benefit, but thought I'd mention it.

This just transfers the problem from the routing table to some other
table--it doesn't solve the problem, in any way.

:-)

Russ

- --

---

From: Russ White <[email-masked]>
Date: Mon, 02 Apr 2007 20:57:30 -0400

> provider independence.  These advantages seem pretty obvious to me, am I

What do I get over just deploying PI space?

:-)

Russ

- --

---

From: Roland Dobbins <[email-masked]>
Date: Mon, 2 Apr 2007 18:08:37 -0700

On Apr 2, 2007, at 5:48 PM, Darrel Lewis ((darlewis)) wrote:

> 1) merging of enterprise networks

This is certainly a potential benefit of a LISP-like architecture - a  
way to avoid address-space collision, dual NATs, etc.

It's also a useful potentially useful concept for a transitional  
mechanism.

                       -- Duong Van Ngo

---

From: David Conrad <[email-masked]>
Date: Mon, 2 Apr 2007 18:21:39 -0700

Russ,

On Apr 2, 2007, at 5:54 PM, Russ White wrote:
> advertise PI space?

You removed the part where I tied the two together, specifically:

>> However, as the routing system bloats, I suspect the latter  
>> benefit will come more and more into play.

The whole RAM discussion was initiated in part because some large  
ISPs looked into their crystal ball and got nervous about how quickly  
routing information was being generated.  When they also looked at  
their favorite routing vendors product roadmaps, they got more  
nervous.  And then some smart people looked at trends in routing  
hardware manufacturing, power consumption, heat dissipation, etc. and  
suggested that "Houston, we have a problem."

Maybe both the folks at large ISPs and the various smart people are  
wrong.  If you make the assumption that "every edge gets PI" can and  
will scale until we deploy IPv7 (as some on the IAB have done), then  
the problem magically goes away (since it never really existed).

However, if you are not comfortable in making that sort of  
assumption, then you have to start looking for operationally  
deployable solutions.

You a betting man?  :-)

Rgds,
-drc

---

From: Tony Li <[email-masked]>
Date: Mon, 2 Apr 2007 18:22:36 -0700

On Apr 2, 2007, at 5:54 PM, Russ White wrote:

>> used.

This is simply incorrect.  Moving the problem from the RIB/FIB to a  
partially populated cache at a much lower access rate does solve the  
problem: the costs for such equipment will be borne directly and  
solely by the people on the data path.

Tony

---

From: Robin Whittle <[email-masked]>
Date: Tue, 03 Apr 2007 12:29:42 +1000

David Conrad wrote (Re: [LISP and the Global Internet Architecture):

> IPv6, as defined, is little more than IPv4 with more bits.  As lots  of
> people pointed out lots of times, the size of the address wasn't really
> the problem, rather the scalability of the routing system was.  We're
> now trying to come up with a solution to the scalability problem that
> doesn't involve "use more thrust".  LISP and its ilk are the first
> realistic attempts to actually deal with scaling the Internet up.

This may be so.  In that case perhaps the SRAM-mapped FIB approach:

http://www.ietf.org/internet-drafts/draft-whittle-sram-ip-forwarding-01

is the second.

There have been well over a hundred messages on this list since I
wrote about my SRAM-based forwarding draft - all of them concerning
LISP.  I know it takes a while to read my draft, but the idea is
simple.  I hope that there will be some discussion of it in the next
few weeks.

I am concerned that people in this field have spent so long - a
decade or more - assuming that route aggregation must always be
maximised, that they will find it hard to imagine all the benefits
of tens or hundreds of thousands of AS end-users and ISPs being able
to advertise PI addresses.  The SRAM approach handles millions of
separately advertised /24s - without any regard to topology and
without burdening anyone, other than with the burden of BGP traffic
and router CPU load for each change.

A single Static RAM chip can be installed in each interface of
future DFZ routers, taking 4 nsec to map every one of the 14,680,064
/24s to its own Forwarding Equivalent Class (which interface to send
the packet to). As long as the /24 BGP limit remains, this
completely solves the hardware scaling issues of IPv4 routing in
terms of actually handling the packets, which is the most difficult
task.  The same chip can handle IPv6, within some limits which would
need to be carefully decided.

A single 72 Mbit chip in each interface, (or two for larger routers)
provides single-cycle packet forwarding for:

IPv4: The entire space of 3.7 billion IP addresses in 14 million
      prefixes.

IPv6: (for instance) 17 billion /48 user networks advertised
      as 2 million /35 prefixes - each with 8192 /48 user networks.

without any extra delay, complication or power consumption even if
every prefix was advertised separately to create worst-case route
disaggregation.  This means that there is no problem (for the FIB
function of these routers) in giving large numbers of AS end-users
Provider Independent addresss for both IPv4 and IPv6.

This is not a case of "using more thrust".  It is a simpler and more
elegant technique than TCAM and tree-structured memory lookups,
which has not been possible in the past because the SRAM chips were
not big enough.  Now these chips are in production, and since the
routers in the DFZ are all going to need replacement in 5 to 7 years
due to the growth in the BGP routing table anyway, I think it will
be perfectly feasible (say in 2014) to have a DFZ with 14 million
IPv4 routes - *if* BGP and the router's CPUs can handle the required
communications.

As long as AS end-users are not changing the way they advertise
prefixes on an hourly, or even daily basis - meaning they only
change the global BGP routing table when they change their ISP
(including in rare instances of failure) - then I imagine the BGP
routing system will cope, as long as routers have somewhat faster
CPUs and more memory.  My understanding of instability in BGP is
that it only affects the prefixes which are being changed.  So if
one AS end-user changes the router which one or more of their
prefixes are advertised on, and if it takes the global system some
time to propagate this, then it is a cost they alone bear.  I
understand that flap damping is intended to reduce the propagation
of rapid changes, so if this is extended in some agreed manner to
discourage recurrent, frequent - and therefore presumably deliberate
- changes, then the load on the BGP system should not be too high.

Most of the load at present is with long prefixes and probably
"badly managed" systems - at least according to the interests of
those who run DFZ routers.

  http://bgpupdates.potaroo.net/instability/bgpupd.html

"Adds and Wdls per Prefix Length" at:

  http://www.cidr-report.org

shows the great majority of changes are for /22, /23 and /24 prefixes.

    /16   +20   -21
    /17   +23   -13
    /18   +40   -23
    /19   +82   -80
    /20  +115   -82
    /21  +159  -124
    /22  +226  -131
    /23  +297  -177
    /24 +1302 -1085
    /25    +0    -0

Yet these cover only 0.85% of the advertised address space, and
(according to my ping survey, which only gives some indication) they
don't appear to have particularly high rates of address utilisation.
 There is a graphic at:

  http://www.firstpr.com.au/ip/

One problem with assigning PI space to many more smaller AS
end-users is that many more smaller prefixes will probably be poorly
managed.  With the direct mapped SRAM-FIB, there is no problem with
where they advertise the prefixes, or how many they advertise.
However the router CPU and RAM of all the DFZ routers need to cope
with the number, and with the rate of changes.

The SRAM-based approach does not provide seamless connectivity for
TCP etc. when changing the prefix to another router.  Some
disruption would occur - but it would not disrupt other prefixes. It
would fully support basic multihoming for redundancy when an ISP
fails.  It could support rapid changes for time-of-day sensitive
traffic management, as long as the BGP system could handle it, which
I suspect it can't or shouldn't have to.

There would be moderately "more thrust" in the CPU and memory area,
but the single SRAM chip is vastly lest thrustful than the current
TCAM etc. approaches to hardware forwarding, especially if those
approaches were scaled to handle millions of routes.  Those TCAM
etc. systems would remain, to handle routes longer than /24 (as
flagged with a special value of FEC in the SRAM), and to handle
other types of packet.

All IPv4 unicast packets (and IPv6 unicast packets within the
defined address range) would be handled by the SRAM system, which
would correctly classify all the user traffic.  TCAM etc. would also
be used for the complex needs of border and internal routers, but
there would not need to be much TCAM in DFZ transit routers -
probably only enough to handle the prefixes used for
router-to-router traffic.  Maybe that traffic would be low enough to
handle with the CPU, so pure transit routers might not need TCAM at all.

Whereas TCAM etc. can require lengthy rewrites (during which it
can't classify packets), to rearrange many of the rules when a
single route is added, the SRAM-FIB approach can be updated quickly
with one or more writes, since there is no need to rearrange the
data which implements the rules for other prefixes.

  - Robin

---

From: Dino Farinacci <[email-masked]>
Date: Mon, 2 Apr 2007 21:26:36 -0700

> since the entire database must be distributed anyway.

But is it one entry per site. It can be only if there are one set of  
locators per site.

> Have we converged on only using a pull model for the mapping?

I don't think so.

Dino

---

From: Dino Farinacci <[email-masked]>
Date: Mon, 2 Apr 2007 23:00:46 -0700

> partially populated cache at a much lower access rate does solve  
> the problem: the costs for such equipment will be borne directly  
> and solely by the people on the data path.

This is correct and the motivation for LISP.

Dino

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 03 Apr 2007 13:10:04 +0200

>> Have we converged on only using a pull model for the mapping?

Nor me. I think we should consider this as a whole separate topic
from the LISP model as such. It seems to me that every solution we've
talked about that is even vaguely in the id/loc split space calls
for an id/loc mapping service, so we should consider it as an
orthogonal matter.

     Brian

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 03 Apr 2007 13:55:44 +0200

> way to avoid address-space collision, dual NATs, etc.

Except that with IPv6, you need neither LISP nor internal NATs to merge
address spaces. As has been observed, the main win in IPv6 is having
enough address space, so that collision isn't an issue and NAT isn't a
requirement.

The ability to change ISPs without excessive cost is harder, since
nobody likes renumbering. Again, it isn't an issue for IPv6 sites
big enough to procure PI space. And I wonder how many SMBs would
really see renumbering a few printers as a major cost, if they had
to use PA space instead of Net 10?

I still see multihoming and TE as the main economic drivers here.

> 
> It's also a useful potentially useful concept for a transitional mechanism.

 From what to what?

     Brian

---

From: Russ White <[email-masked]>
Date: Tue, 03 Apr 2007 08:36:36 -0400

> This is correct and the motivation for LISP.

1. Do we have evidence this cache will actually work? We have lots of
experience showing caches don't work in this sort of environment. If we
have such evidence, can you point to it?

2. Do we have evidence showing the first x packets being dropped while
the cache is populated through a complex ICMP based process? If so,
could you point to it?

3. If the LISP cache can "solve" the growing table size, why not just go
back to a cache between the RIB and the FIB? Wouldn't this provide the
same benefit, without all the tunneling work? Can you explain the
difference between simply going back to an on demand filled cache based
on the RIB, verses LISP?

4. How does LISP solve the problem of having more packet processing on
the switching engines, and the power drain associated with the ever
increasing gate counts on line cards?

5. How long will it be before someone asks for a line card that can look
into the "inner" LISP header and filter/etc based on that? Or the third,
or the fourth?

:-)

Russ

- --

---

From: Russ White <[email-masked]>
Date: Tue, 03 Apr 2007 08:43:54 -0400

> But is it one entry per site. It can be only if there are one set of

At the cost of having a larger loc/id mapping table.

>> Have we converged on only using a pull model for the mapping?

IMHO, if we're going to do anything, we need a push model for the loc/id
mapping. A pull model would likely destroy any path into true mobility.
The final result of a pull model would likely be a loc/id mapping system
designed for occasional draws of information being overwhelmed with
constant draws of information to support mobility--you can already
imagine "pre-pull" systems that speculatively cache loc/id mappings to
support mobility.

:-)

Russ

- --

---

From: Eliot Lear <[email-masked]>
Date: Tue, 03 Apr 2007 16:23:13 +0200

Brian,
> for an id/loc mapping service, so we should consider it as an
The characteristics of various proposals vary widely from no new mapping 
needed (SHIM6), use of DNS for the mapping (HIP), to LISP (???).  In the 
case of SHIM6 and HIP it is at least possible to query a mapping service 
either prior to a communication (HIP) or during setup (SHIM6), but the 
number of participants is meant to scale to the number of hosts using 
the service.  Because LISP does not touch the end host, in the first 
place you have a lower scaling requirement to the number of provider 
edge routers (worst case).  Furthermore, there is the ability to 
aggregate PI address space.  You cannot aggregate HITs and there is no 
similar concept for SHIM6.  To me that means that the number of entries 
is *AT MOST* O(PI addresses in the system).

And so I believe you need to consider the ID/Loc mapping service for 
each mechanism individually.

Eliot

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 03 Apr 2007 16:51:19 +0200

> each mechanism individually.

I need to write something quite long to explain why I think you're
wrong. It may take a while.

     Brian

---

From: "Templin, Fred L" <[email-masked]>
Date: Tue, 3 Apr 2007 07:57:25 -0700

> Because LISP does not touch the end host,

A small point of clarification; since LISP is deployed on
routers, it follows that LISP does not touch the end *host*.
But, it does not follow that LISP does not touch the end
*platform*, where an end platform consists of a router and
one or more hosts.

Fred

---

From: David Meyer <[email-masked]>
Date: Tue, 3 Apr 2007 08:03:36 -0700

> for an id/loc mapping service, so we should consider it as an

	It seems pretty clear that the encap part of the
	map/encap schemes is relatively easy. As I think you are
	pointing out, it is the map functionality that is
	difficult (hard). And in particular, if we move the
	scaling problem (such as I see it, understanding there is
	no consensus on that issue; let's not open that here) to
	the map functionality, then we will have to deal with
	that problem there, and it is unclear that that is an
	easier domain in which to do this work. Now, maybe it is,
	but I haven't seen anything that would lead one (or me in
	particular) to that conclusion. In fact, I can see it
	being much harder for a multitude of reasons, many of
	which will not be technical (cf. infrastructure ENUM).
	
	That being said, any map/encap proposal that does not at
	least specify the required properties of the map scheme
	will be very weak indeed (since such a proposal will be
	skirting the hard part of all of this). So while I admire
	the desire to decouple the map functionality, it seems
	very little will be gained by doing so, since many of the
	hard problems will remain unsolved.

	--dmm

---

From: "Templin, Fred L" <[email-masked]>
Date: Tue, 3 Apr 2007 08:10:26 -0700

> 	hard problems will remain unsolved.

Well said, David. For better or worse, IPvLX (due to be updated
based on exchanges on this list) at least takes a stab at what
the mapping function might look like. For everyone who thinks
they have a better proposal, let's hear 'em...

Fred

---

From: David Meyer <[email-masked]>
Date: Tue, 3 Apr 2007 08:18:16 -0700

> edge routers (worst case).  

	Even though LISP is intended to be a "network side"
	solution, I really don't see any reason that it (ITR or
	even ETR) couldn't be implemented in a host.

> Furthermore, there is the ability to aggregate PI address space.
                     ^^

	Is, or is not? 
	
> You cannot aggregate HITs and there is no similar concept for
> SHIM6.  To me that means that the number of entries is *AT
> MOST* O(PI addresses in the system). 

	Right, and that's really a large part of the problem
	(O(number_of(PI prefixes)), in the event you are one of
	the those who believe there is a scaling problem).

> each mechanism individually.

	Considering each separately is a fine idea, but it would
	truely be a shame if this became another place to
	proliforate the ways to do thing <X> (in this case X is a
	mapping service of some sort). 

	--dmm

---

From: "Templin, Fred L" <[email-masked]>
Date: Tue, 3 Apr 2007 08:24:26 -0700

Just to reserve the acronym, I am contemplating a future
proposal: "IP with eXtreme eXtension of Links (IPXXL)".
It would be very much like IPvLX, only bigger.

Fred

---

From: David Conrad <[email-masked]>
Date: Tue, 3 Apr 2007 09:47:00 -0700

Russ,

On Apr 3, 2007, at 5:43 AM, Russ White wrote:
> mapping.

A push model moves the routing information thrash from a relatively  
small number of very well connected, extremely powerful "core"  
routers to a very large number of relatively poorly connected (in  
comparison to the core), relatively weak edge(-ish) routers.  This  
will almost certainly imply changes to the routing system will take  
much longer to propagate and convergence of the routing system will  
be a wistful dream.

> A pull model would likely destroy any path into true mobility.

Actually...

Assume you have a node with identifier X moving between service  
provider with locator "A" and service provider with locator "B".   
While in the "A" service provider realm, the mapping is X->A.  At the  
A/B boundary, X notifies B of its entrance, changing its mapping to X- 
 >B.  However, in a pull model, you'll have a cache and traffic will  
continue to go to A until the cache entry expires.  To address this,  
X notifies A of the transition, inserting a forwarding mapping (that  
will last until the cache entries can be assumed to expire) at A to  
forward X traffic to B.  If X moves from B to C, it can either update  
its forward mapping at A to point to C or it can insert a forward  
mapping at B that points to C.

> imagine "pre-pull" systems that speculatively cache loc/id mappings to

Given caching and forwarding as described above, I'm not sure why  
you'd think a pull system would be overwhelmed.

How would a push model work in the mobility case?

Rgds,
-drc

---

From: Ross Callon <[email-masked]>
Date: Tue, 03 Apr 2007 13:12:04 -0400

>But is it one entry per site. It can be only if there are one set of

I understand that for those sites that are multi-homed and are
using a PI address, this might be turned into one entry per site
in the mapping table. In this case one entry in the routing table
turns into one entry in the mapping table (modulo the possibility
that there might be more sites that choose to be multi-homed
and use a PI address, than there would have been with the
current routing and addressing scheme).

What about sites that are currently single-homed and use a
PA address? Will they need to be listed in the mapping table?

Suppose that there is a host Hd in a dual homed site, Sd, that
has a packet to send to a host Hs in a single-homed site Ss. A
router (called Rd) in the dual-homed site receives the packet with
a source address for Hd (which is not globally routable) and a
destination address Hs (which is globally routable). I am thinking
that we might want to still use encapsulation in this case since
the packet would otherwise be going out on the global Internet
with a source address that is not routable. Thus Rd might
encapsulate the packet in another IP header with its own address
(Rd) as the source address, but it needs to know what address to
use for the destination address. If it uses the address of the
destination host in the encapsulating header, then either the
destination host will need to know to decapsulate the extra header,
or some router close to the destination will need to know to snoop
packets that are not to it, and strip off the extra header. If nothing
else this snooping and stripping seems to break other uses of IP-in-IP
encapsulation, unless we get a unique value of the protocol field in
the IP header reserved for LISP IP-in-IP encapsulation (which should
be simple to do if LISP is widely deployed). If on the other hand the
encapsulating router uses the address of a router close to the
destination (the one that is going to do the  decapsulation) as the
destination address in the encapsulating header, then the host Hs
or its site Ss needs to be in the mapping table, which would make
the table much larger than the IP routing table. (if the table is in
something that resembles DNS with a pull model then the size
doesn't seem to be an issue -- if we use a push model then having
each single-homed site in the mapping table would seem to be a
significant issue).

>>Have we converged on only using a pull model for the mapping?

I think that I agree that I haven't noticed convergence on this point.

Ross

---

From: Tony Li <[email-masked]>
Date: Tue, 3 Apr 2007 11:36:52 -0700

> 	hard problems will remain unsolved.

I agree that the mapping mechanism is the keystone in any map-and- 
encap solution.

It seems to me that if we could elevate the discussion to a more  
conceptual level about how to create scalable mappings that we would  
accelerate the overall process.

I think we do know that there are mapping mechanisms that are not  
going to scale.  We can pretty quickly characterize those by looking  
at the rate of mapping operations times the size of the state that  
needs to be stored.  This "rate * state" product is key because when  
you have both high rate and large state, it implies that the  
implementor cannot reasonably make good space/time tradeoffs.

There are obvious ways of attacking the rate * state product.  You  
can decrease the rate by distribution.   You can decrease the state  
by going to a pull model.  I suspect that there are other approaches...

Tony

---

From: Tony Li <[email-masked]>
Date: Tue, 3 Apr 2007 14:07:37 -0700

> A push model moves the routing information thrash

Let's be very careful here.  We were talking about using the push  
model for the id->locator mapping.  That's no more part of 'routing'  
than DNS is today.

Tony

---

From: Bruce Curtis <[email-masked]>
Date: Tue, 3 Apr 2007 18:58:47 -0500

On Apr 3, 2007, at 4:07 PM, Tony Li wrote:

> model for the id->locator mapping.  That's no more part of  
> 'routing' than DNS is today.
>
> Tony

   If I'm following correctly don't LISP and other similar solutions  
move the path up/down information out of the core "routing"  
information and closer to the edge and place it into the "id- 
 >locator" mapping?

   So the "information thrash" about paths being up/down is moved  
whether the model is push or pull?

---
Bruce Curtis                         [email-masked]
Certified NetAnalyst II                701-231-8527
North Dakota State University

---

From: David Conrad <[email-masked]>
Date: Tue, 3 Apr 2007 19:01:29 -0700

Robin,

I haven't had time to read your draft and I am not qualified to  
comment on hardware architecture, but...

On Apr 2, 2007, at 7:29 PM, Robin Whittle wrote:
> to advertise PI addresses.

The benefits to flat routing are obvious.  The concern is that at  
some point flat routing won't scale.  Where that point is is open to  
debate.  What you are proposing is to move that point much higher  
based on new technology (or perhaps new advances that make old  
technology more cost effective).  There is still a point (as you  
acknowledge) where flat routing breaks down.  Whether it is far  
enough in the future to not matter is one question.

> This is not a case of "using more thrust".

Sure it is.  By analogy, folks are noticing propeller-based  
technology can't break the sound barrier and you're proposing this  
new "jet-based" technology.  It might work or it might rip the wings  
off the airplane.

> *if* BGP and the router's CPUs can handle the required communications.

That's part of the problem.  People often say the problem is that the  
"global routing table is too big".  This is poor terminology for many  
reasons, but mostly because the problem isn't the _size_ of the  
table, but rather the flux and propagation of routing information in  
conjunction with the need to lookup bits of that information really  
really quickly.  Your proposal (if I understand it) addresses the  
latter bit.  There is still the former bit.

> My understanding of instability in BGP is
> that it only affects the prefixes which are being changed.

I believe this depends on the type of instability, but there are  
others on this list who can explore this in much more detail than I.

> managed.

Right.  One advantage of a loc/id separation with mapping approach is  
that it permits end "sites" (for some value of the variable "site")  
to play in the multi-homing pond without having to also play with  
BGP.  As you note, the folks most likely to screw up BGP are also  
most likely the folks who would have the long prefixes being inserted  
into the routing system.  BGP is sort of the soft underbelly of the  
Internet...

Rgds,
-drc

---

From: David Conrad <[email-masked]>
Date: Tue, 3 Apr 2007 21:24:20 -0700

Tony,

> 'routing' than DNS is today.

OK.  Poor use of terminology.  Today, we have only "routing thrash"  
where reachability information as defined by prefixes announced by  
BGP bounces around.  In a loc/id model, we still have routing thrash  
but we'd be introducing a new type of thrash, call it "mapping  
thrash" which would be addition/changes/deletions of identifier to  
locator(s) mapping information.  For end to end connectivity, you  
have to contend with both thrashings.

In both pull and push, routing thrash is obviously the same.   
However, in a pull model, since mappings are fetched on demand,  
mapping thrash is minimized as only active "connections" are fetched.  
If you're not talking to a site, you don't care that its mapping has  
changed.  In a push model, any change to any mapping needs to be  
pushed out to all mapping points, regardless of whether you need that  
information or not.

Rgds,
-drc

---

From: Robin Whittle <[email-masked]>
Date: Wed, 04 Apr 2007 15:21:53 +1000

Hi David,

I agree with all your concerns.

I think the /24 limit on IPv4 is sufficient for the foreseeable
future, since it could allow a much more flexible and intensive use
of the address space than is currently the case.  In the future,
when bigger RAM chips are cheaper, this could be extended to /25 or /26.

Some careful decisions would need to be made about IPv6, because the
current global unicast range involves too many changing bits to map
into SRAM.  I think that a restricted range of global unicast
addresses could still provide more than enough addresses for the
next decade or so, after which a new standard for more SRAM and an
extension of the range could be made.

> > This is not a case of "using more thrust".

It is "more thrust", in the sense that it requires new routers in
the DFZ, with a new design to a particular future standard.  But in
the timeframe I am suggesting, all those routers will need to be
replaced anyway.  I suggest a standard in 2007, new routers in 2009
or 2010 and ubiquitous deployment of the new routers (or old ones
with massive TCAM etc.) by the end of 2014.

I believe my proposal is not simply "more thrust", since the easiest
way to achieve the proposed flat routing capability will involve low
power consumption and moderate redesign with a standard SRAM chip,
rather than complex new functions in ASICs or microcode or more and
more TCAM.  I think the result will be simpler, cheaper and less
power-hungry than continuing to make larger versions of the current
architecture for the next 10 to 20 years.  I also think router
manufacturers would implement SRAM in the way I describe anyway,
irrespective of this proposal, because it is a cheaper alternative
to massive TCAM for the task of likely future IPv4 DFZ traffic.  I
think they could be confident the /24 BGP limit will be extended to
/25 or longer in the lifetime of the next generation of routers.

A standard - or agreement - such as I am proposing would encourage
this adoption of SRAM flat routing, especially if there was a
standardised way of handling IPv6, perhaps in the 1/8 of the chip
not needed for IPv4.

More CPU and RAM thrust is required for the RIB and BPG traffic, as
I acknowledged - but CPUs and SRAM/DRAM are commodity devices and
are continually getting cheaper and faster - probably outpacing the
current growth in the routing table.  However, I expect SRAM-based
flat routing would lead to greater rates of growth than in recent years.

Compared to the task of turning RIB into TCAM (and its attached
SRAM) data, it is trivially easy to convert the RIB into the FIB
data in the SRAM.  Updates to the SRAM FIB data are also simple and
do not upset any other prefixes or delay the classification of
packets.  So in this respect, I think of the proposal as using more
"elegant design" thrust and less "brute force hardware and
complexity" thrust.

You wrote:

> That's part of the problem.  People often say the problem is that
> the "global routing table is too big".  This is poor terminology
> for many reasons, but mostly because the problem isn't the _size_
> of the  table, but rather the flux and propagation of routing
> information in conjunction with the need to lookup bits of that
> information really really quickly.  Your proposal (if I understand
> it) addresses the latter bit.  There is still the former bit.

Yes.

> others on this list who can explore this in much more detail than

I hope someone with BGP expertise will confirm my guess that if a
prefix is advertised at a different router to the one it was
previously advertised at, that it may take time for the changes to
propagate through the whole DFZ, as various routers reconfigure
their RIB for this prefix - but that this does not affect the BGP
traffic or forwarding of packets for other prefixes.

> > end-users is that many more smaller prefixes will probably be
> > poorly managed.
>
> Right.  One advantage of a loc/id separation with mapping approach
> is  that it permits end "sites" (for some value of the variable
> "site") to play in the multi-homing pond without having to also
> play with BGP.  As you note, the folks most likely to screw up BGP
> are also most likely the folks who would have the long prefixes
> being inserted into the routing system.  BGP is sort of the soft
> underbelly of the Internet...

I agree.

The SRAM-based flat routing system would enable very efficient,
piecemeal, assignment of address space in small chunks.  I intend
that this encourage the direct connection of hundreds of thousands
of AS networks, all with PI space for both IPv4 and IPv6.  If this
could be done, there would be tremendous benefits.

However, it is likely that the global BGP system would need to be
made more robust against badly managed border routers, against
attacks and to discourage AS operators from changing their
advertisements more rapidly, in general, than is deemed desirable by
everyone else.

  - Robin

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 04 Apr 2007 14:02:04 +0200

On 2007-04-03 18:47, David Conrad wrote:
...
> How would a push model work in the mobility case?

It seems to me that in a foreign-agent c/o-address
type of mobility there is no issue here: the id
to be mapped is that of the FA, not that of the
mobile host.

Or another way to look at it is that MobileIP already
solves the id/loc mapping problem using a rendezvous
mechanism, where the rendezvous point is the home agent.

     Brian

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 04 Apr 2007 14:34:30 +0200

On 2007-04-04 07:21, Robin Whittle wrote:
...
> into SRAM.  

There's a reason for that. Because of the address shortage, we have been
forced over the last 15 years to use IPv4 space extremely conservatively,
avoiding sparse allocation at all costs. One of the drivers for the choice
of 128 bits for IPv6 (rather than say 64) was to avoid this artificial
pressure for compact allocation. Sparse allocation is much less annoying
for sites, since it's much less likely to lead to churn in an addressing
plan (which is an even bigger pain point than simple prefix renumbering).

Or in other words, the IPv6 equivalent of a /24 is a /48 or even a /56.

     Brian

---

From: Jari Arkko <[email-masked]>
Date: Wed, 04 Apr 2007 15:33:53 +0300

> just one, and things don't line up so nicely.

First, I think Eric had a point in keeping some amount of separation between
MANET techniques and the techniques needed in the global Internet. If
the global network topology stays relatively static, it makes sense to
tailor
the global routing mechanisms for that situation. But it is a feature, not
a bug that independent networks on the edges can choose to run mechanisms
that suit their specific needs, ranging from manual configuration to MANETs.

However, there is a general issue with churn that probably deserves
some discussion.

Clearly, we want the core of the network to be relatively stable, and
if it employs only PA addressing that is a big step towards stability
in the core BGP table. But I'd like to understand
to what extent this change solves the churn issues that we are
seeing today -- presumably there are other reasons for this
beyond, say, TE related to PI space. Do we understand whether those
other reasons will still create enough churn to worry about?

Another question is the stability of the new mapping table that
proposals such as EFIT or LISP need. Since this table has (ID, LOC)
entries, it depends on the stability of the identifiers and locators.
Lets talk about the locator part first:

- Change or addition of a provider serving this network results in
  a mapping change. This is expected to be a relatively slow process.

- "TE" needs might require updating attributes associated with the
  mapping (such as the order of locators). If there is such a need,
  it is likely to result in much more frequent changes than the
  above one.

  (I'm just trying to establish how fast the data changes, not
  say anything about the implementation. Clearly you could store
  information either in a database or communicate between the
  peers, for instance.)

- If the goal is to support network mobility as well, then the
  requirements become significantly more demanding. To
  support this at a rate where you would drop no voice packets
  or at most few packets upon movement is hard. Basically,
  neither push or pull would suffice, though pull + reactive
  updates might.

Identifier part:

- Network re-design, organizational mergers/splits, etc. likely
  results in some changes to the identifiers, use of multiple
  identifiers, splitting identifier spaces, etc. This is again
  expected to be a relatively slow process.

- If the identifier database holds "identifier prefixes", then
  it changes only as the entire network changes. But if it
  points to specific identifiers it becomes bigger, and
  requires changes as hosts are added, removed, or
  move around from one network to another.

- But movement of the hosts within the network
  does not impact the global mapping table at all --
  irrespective of whether the network employs OSPF
  or MANET routing protocols.

Jari

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 04 Apr 2007 14:34:30 +0200

Brian

---

From: Jari Arkko <[email-masked]>
Date: Wed, 04 Apr 2007 15:33:53 +0300

Identifier part:

Jari

---

From: "Templin, Fred L" <[email-masked]>
Date: Wed, 4 Apr 2007 08:20:19 -0700

>   neither push or pull would suffice, though pull + reactive

One alternative for pull + reactive would be DNS + MOBIKE.

Fred

---

From: JFC Morfin <[email-masked]>
Date: Thu, 05 Apr 2007 05:30:12 +0200

>enough address space, so that collision isn't an issue and NAT isn't a

Unfortunately I must disagree.

As it is documented today IPv6:
- does not permit (except if my proposition is supported by a 
separate /3 blok) to orthogonally support world location, addressing 
and local identification numbering plans (only IPv6 under LISP 
permits a world identification scheme)
- does not document the user space organisation, removing most of the 
application developper/user interest into its use at this stage.

Does not permit alone (chained IPv6 under LISP permits it) to address 
very wide address plans as most probably required in coming quantic 
and space related processes and communications.

Cheers.
jfc

---

From: Robin Whittle <[email-masked]>
Date: Mon, 09 Apr 2007 22:38:06 +1000

Brian Carpenter wrote:

RW > Some careful decisions would need to be made about IPv6,
RW > because the current global unicast range involves too many
RW > changing bits to map into SRAM.

> There's a reason for that. Because of the address shortage,
> we have been forced over the last 15 years to use IPv4 space
> extremely conservatively, avoiding sparse allocation at all
> costs.

My impression is that this pressure has been in the last 10 years or
so.  Whole /8s were allocated to some companies and organisations in
the early 1990s, and 12 of them are not even advertised at present.
 This is visible as a purple '0' in column 4 of the table at the end of:

  http://www.firstpr.com.au/ip/host-density-per-prefix/

Most of the remainder are lightly used.  For instance, 19 of them
are advertised as a single /8.  I think it is unlikely that these
prefixes are heavily populated with actively used IP addresses, if
each one is advertised from the one router.  Random pings to these
prefixes generally result in none or few acknowledgments.

> One of the drivers for the choice of 128 bits for
> IPv6 (rather than say 64) was to avoid this artificial
> pressure for compact allocation. Sparse allocation is much
> less annoying for sites, since it's much less likely to
> lead to churn in an addressing plan (which is an even bigger
> pain point than simple prefix renumbering).

But if the flat routing system I am proposing (as David Conrad
correctly described it) was in place, then most or all of the
long-held assumptions about addressing which underlie what you
mention, would no longer apply.

With a flat routing system, an organisation can get a prefix of
addresses, effectively permanently (PI for AS end-users including in
IPv6), to last its needs for new addresses for a year or so.  When
the end-user or ISP needs more, they can get another prefix.  In
terms of the DFZ, it doesn't matter that the one large organisation
has a few dozen or a thousand or so /24s, even if some or all of
them are non-contiguous.  I am not suggesting addresses only be
assigned in /24s - bigger organisations need more addresses per
assignment, but with flat routing they don't need a single large
block of space which will last them for a decade or two.

Within each AS end-user's network, they only need SRAM-equipped
routers for border routers with two or more links to upstream ISPs.
The rest of the routing can be handled well by conventional amounts
of TCAM etc., unless the local prefixes are so numerous as to
stretch the limits of TCAM.  This seems unlikely except for all but
the biggest AS end-users.

Probably our biggest problem is that the global BGP routing table is
requiring more and more expensive TCAM, or an equally expensive and
complex alternative, for every DFZ router.  The SRAM approach solves
this problem with a modest design addition for the next generation
of DFZ routers.

An AS end-user might prefer to receive a single large slab of
addresses and to do whatever they like with that over the next
decade or two.  But having a bunch of smaller prefixes, which are
effectively for keeps and which are all routable globally, no matter
how they are split (down to a /24 granularity) should be about as
easy to manage, although it is not as aesthetically pleasing.

Even if an AS end-user had some big slab, such as a /8 - as some do
- then if they are going to support multiple sites, each multihomed,
this generates a complex set of routing rules in their internal
networks anyway.  I don't see much practical difference between
building a network incrementally, with separate, effectively
permanent, smaller pieces of address space - which are all well
utilised - than starting with one huge slab and populating it piecemeal.

Without a flat routing system, the principle is something like this:
 Each AS end-user (or ISP) needs to constrain its address
utilisation for the next decade or so to either one prefix, or as
few prefixes as possible, in order to maximise route aggregation.

Because there is no financial cost in getting IP addresses, because
the RIRs are so keen to maximise route aggregation, and since an AS
end-user is happier with more IP addresses than with less, the
pattern has been to hand out slabs of IP address space which are
larger rather than smaller.  Consequently, most of this space is
sparsely used - and with only ~150 million to 300 million (I guess
at a maximum) IP addresses on the Net currently in use (~110 million
responding to pings), we are widely believed to be close to running
out of address space.  We are, as long as the current requirement
for route aggregation remains.

LISP in theory routes around this, but by the time it is implemented
widely, most IP addresses will have been assigned and advertised as
part of the BGP routing system.  LISP 1.0 doesn't achieve anything
for reducing the number of BGP routes.  LISP 1.5 and above could -
but that will require some mechanism for removing large numbers of
prefixes from the BGP routing system, relegating them to a
second-class service which depends on the LISP infrastructure.  I
can't see why anyone would agree to their prefixes being downgraded
in this way, unless perhaps LISP provided some traffic engineering
benefits which are not available on globally routable BGP prefixes.

The idea of route aggregation and the feeling of vast space which is
easily provided by having a bunch more bits in the addressing
scheme, gives IPv6 a heady feeling of endless horizons, for free.
In some ways, this is true - for instance each /48 provides 64k user
networks, each of which have essentially unlimited numbers of public
IP addresses.

Assuming each small to medium non-AS end-user gets a /48 - say homes

> Or in other words, the IPv6 equivalent of a /24 is a /48 or

A /48 has bits 79 to 0 in flux - giving each such user 64K LANs of
up to billions of hosts.

What my SRAM proposal needs is a more compact handling of the bits
124 to, for instance 92 (a /35 prefix).  This will feel really bad
to anyone who feels good about vast expanses of address space, for
instance on the basis that a single prefix will last an organisation
for decades, no matter how much it expands.   But that feeling isn't
necessary in a flat routing model, because each AS end-user or ISP
can easily get more space, non-contiguously, as they need it.

One approach is to have SRAM mapping for 2 million /35s.  Each /35
provides 8192 /48s.  This system would provide for assignment,
effectively permanently, of globally routable space, down to /35
prefixes - which is capable of supporting 16 billion /48 user
networks.  This is Provider Independent space for both ISPs and AS
end-users.  This space greatly exceeds IPv4s even if we think of a
/48 as equivalent to a single IPv4 address.  It will also be much
more efficiently used than IPv4 space, (thinking again of an IPv6 as
equivalent to a single IPv4 IP address) because it can be split into
2 million globally routable PI /35 prefixes, each of which can be
used more flexibly than we are used to with the current focus on
maximising route aggregation.

In this example, the SRAM system would span a /14, with for
instance, RIRs typically handing out space in /32s, each of which
could be split into 8 separately advertisable /35s.  Because there
are a quarter million of these /32s to go around, and because each
AS end-user or ISP would use one reasonably fully before asking for
another, this address space would go a long way.

At a later date, a subsequent generation of routers with more memory
could cover this /14 and the next one or three /14s, making the
total SRAM-handled flat routing space into a /12 or so.

These figures all probably sound frighteningly restrictive compared
to the /3 which is currently being allocated and assigned.  But that
space can't be flat routed.  It would require vast amounts of TCAM
in all DFZ routers if IPv6 is to be widely adopted.

If people can forget two decades of thinking about the need for
route aggregation, and can imagine that the BGP system - or
something to replace or supplement it - can handle the larger number
of routes, then I think they will be able to imagine that the IPv6
example I gave is still a vast address space, and that flat routing
down to /24 in IPv4 will facilitate a much more effective use of the
3.6 billion IPv4 addresses.

  - Robin

  http://tools.ietf.org/html/draft-whittle-sram-ip-forwarding-01
  http://www.firstpr.com.au/ip/sram-ip-forwarding/

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 10 Apr 2007 14:10:29 +0200

Robin,

> can easily get more space, non-contiguously, as they need it.

You missed my point. Enterprise network managers design addressing
plans that covers perhaps hundreds of sites with tens of LANs each.
They are not interested in acquiring scraps of address space each
time they need some more. This is nothing to do with flat routing;
even the largest enterprise networks can easily be flat routed.
It's to do with administrative simplicity.

     Brian

---

From: Robin Whittle <[email-masked]>
Date: Wed, 11 Apr 2007 15:15:55 +1000

Hi Brian,

I accept that the most administratively simple approach to address
assignment is for an ISP or AS end-user organisation to get a single
block of address space which is big enough to suit its greatest
possible needs for the next decade or two.

The two reasons for providing such a large, single, long-lasting
block seem to be this administrative simplicity and the desire for
route aggregation.

Most users of an address block such as this will want to split it
geographically for its various branch offices, and for each office
(or even for a small organisation with a single site) to split it
and advertise it on topologically diverse parts of the Net.  The
more topologically diverse, the better, in terms of providing robust
multihoming.  So I think the goal of assigning larger blocks less
frequently for the purpose of preserving route aggregation is
illusory.

This leaves the other goal of administrative simplicity for each
medium to large organisation.  I assume that small organisations
with a single site could get by with a single assignment for quite a
number of years - for instance a /22 to /24which they can advertise
as one or more separate/24s for multihoming.  There is also the goal
of administrative simplicity for RIRs handing out a large block at
one time rather than a lot of smaller blocks more frequently.

The trouble is with the traditional approach to address assignment
is that really large slabs of address space are assigned, with
generally minimal utilisation - with the result that we are running
out of fresh address space.  Then everyone has to adopt LISP or
IPv6, both of which have a bunch of costs and problems.

I don't think that the immense global, shared, cost of:

1 - Being forced into IPv6 just due to shortage of fresh address
    space in IPv4.

or

2 - Having to pay for massive TCAM in all DFZ routers.

or

3 - Having to change IPv4 usage such as with LISP, including new
    routers all over the edge of the network, and a large
    subset of address space being excluded from being the
    BGP routing system.

is justified by the administrative simplicity of medium and larger
users having a single slab of addresses, compared to them getting
multiple smaller blocks as they need them and the deployment of a
new generation of SRAM-based DFZ routers.

I think we may be able to avoid 2 and 3 above, and not be forced
into IPv6 for another decade or more, if the IPv4 address space is
used much more efficiently.  To do this, it needs to be sliced and
diced with freedom.  In the past, routers couldn't be built to
handle this.  Now I believe they can.

For my suggestion to be practical, there needs to be more CPU power
and RAM in the DFZ routers and the changes in BGP advertised
prefixes probably need to be restricted to long-term changes
reflecting basic network topology and to short-term changes due to
genuine outages.  I do not propose the global BGP system be loaded
down by some prefixes being changed for short-term, recurring
purposes, such as adapting to daily changes in traffic patterns.

Since the DFZ routers will all be replaced in 5 years or less, I
suggest the new ones be equipped with SRAM so they easily route (or
perhaps switch) packets to 14.5 million freely assignable IPv4 /24
prefixes.  This should require minimal incremental expense in the
DFZ, which all Net users share.  It requires no expenditure on LISP
routers in provider and AS end-user networks.  It requires no
inefficient headers, new protocols or centralised mapping databases.

I think these aspects of LISP, including - as best I can understand
- the need (with LISP 1.5+) to remove a substantial proportion of
the address space from the global BGP system and handle it with
LISP, would be more administratively complex than medium to large
organisations having to get address space in smaller,
non-contiguous, chunks.

 - Robin

> Robin,
>
>> What my SRAM proposal needs is a more compact handling of the
>> bits 124 to, for instance 92 (a /35 prefix).  This will feel
>> really bad to anyone who feels good about vast expanses of
>> address space, for instance on the basis that a single prefix
>> will last an organisation for decades, no matter how much it
>> expands.   But that feeling isn't necessary in a flat routing
>> model, because each AS end-user or ISP can easily get more space,
>> non-contiguously, as they need it.
>
> You missed my point. Enterprise network managers design
> addressing plans that covers perhaps hundreds of sites with tens
> of LANs each.  They are not interested in acquiring scraps of
> address space each time they need some more. This is nothing to do
> with flat routing; even the largest enterprise networks can easily
> be flat routed.

---

From: Dino Farinacci <[email-masked]>
Date: Wed, 11 Apr 2007 01:29:37 -0700

> organisations having to get address space in smaller,

Allow me to clarify one point. LISP does not remove the topologically  
significant locator prefixes from the core. They will use traditional  
CIDR aggregation as they have been since CIDR has been introduced. It  
is the PI prefixes which are not aggregatable that need to removed at  
the same time as allowing site-based and provider-based ingress  
traffic engineering without advertising more specific routes to do so.

Dino

---

From: "Templin, Fred L" <[email-masked]>
Date: Wed, 11 Apr 2007 09:46:25 -0700

Robin,

I may be speaking outside my area of expertise, but why
must LISP-like solutions and more robust core routers be
seen as mutually exclusive? Why not have it such that we
leave the core as IPv4 and build IPv6 out at the edges.

That way, the core is still going to grow and probably
need the more robust routers, but there will be a hard
upper limit to the growth. Meanwhile, the number of
interconnected IPv6 devices at the edges could scale to
virtually limitless proportions while the core remains
manageable.

Am I making any sense here?

Fred

> From: Robin Whittle [mailto:[email-masked]] 
> Sent: Tuesday, April 10, 2007 10:16 PM
> To: [email-masked]
> Subject: Re: [RAM] LISP etc. is not the only path to scalable routing
> 
> Hi Brian,
>

---

From: Robin Whittle <[email-masked]>
Date: Thu, 12 Apr 2007 15:21:50 +1000

Hi Fred,

I am working on a longer message contemplating how LISP and
SRAM-based forwarding might complement each other.

  - Robin