Subject: [RAM] Ivip (was ViP ...)

From: Robin Whittle <[email-masked]>
Date: Sat, 16 Jun 2007 21:54:31 +1000

Sorry this is so long.  I figure one long message is better than
many short ones.

Below I respond to questions raised by Dino and Christian in "ViP:
Anycast ITRs in the DFZ & mobile tunnels".  I will reply to Ved
about NEMO in a separate message.

I am now calling this proposal Ivip, for reasons given below.

I am developing this proposal by thinking out loud on this list,
because my first proposal (SRAM mapping of the FIB) was a lot of
work in isolation and turned out not to be directed at the most
important problem.

The easiest way to get the feel of what I intend Ivip or LISP to do
is to imagine this as a plumbing problem.

Despite whatever improvements might be made to BGP, it seems we have
to accept that the Internet can only have about X many "pipes"
(divisions of the address space) by which the BGP routing system can
physically move packets to border routers.  The current value of X
is above 200,000 and we are hoping it won't grow to much more than
double this for a decade or two.  But natural growth would see it
double in a few years.

I think the task is to provide millions of end-users - ideally tens
or hundreds of millions of end-users, including companies, schools
and SOHO people - with the same experience as if they had their own
portable "pipe" they could take to various ISPs.  For multihoming,
they need two links to two ISPs, and they need to switch the
Internet's pipe for their address, or subnet of addresses, from one
ISP to another.

So we need a system which makes a 300,000 (relatively fixed) pipe
system behave much as if it had millions or tens of millions of more
flexible pipes.

Both LISP and Ivip do this by putting multi-pipe receptors (ETRs) in
the edge networks, either at border routers or inside the edge networks.

Both LISP and Ivip have a central system determining how the pipe
endpoints are pointed to the ETRs.  The difference is that the
pipe's inputs with LISP must be inside or at the border of the edge
networks of the sending hosts, while for Ivip, they can be either at
the border routers of those edge networks or anywhere else in the
Net.  Ideally, Ivip ITR functions are performed at border routers or
at transit routers close to the border routers of the sending hosts.
 But maybe it doesn't matter where the ITR is, as long as the total
path length is not much longer than would be the case without having
to pass through an ITR.  (It would be technically possible to put
Ivip's ITRs inside edge networks, or even do the encapsulation in
the sending host - but I don't see much point in either of these
approaches.  Encapsulation in the sending host would make a mess if
the host was behind NAT.)

I think part of the difficulty in this discussion is that my idea of
what LISP and Ivip (or any similar solution) can and should do is
rather different from some other people's ideas - or at least from
Dino's.  Maybe Dino and others see LISP or whatever as being used
primarily by existing AS-end users to handle whole subnets such as
/20, /24 etc. with each such subnet being tunnelled to one ETR.  I
think of this as a "Bring Your Own PI address space" plan.

I see LISP or Ivip as providing "effectively PI" addresses in much
smaller chunks than BGP can handle, including especially individual
IP addresses, to millions of customers who need multihoming and
perhaps portability.  The vast majority of those customers would
never get an ASN or want to know about BGP.  I see this as a way of
making much better use of scarce IPv4 space, which hopefully will
leave more space for new ISPs and for the larger AS-end-users who
would be best served with their own PI space and current BGP
approaches to multihoming and portability.

I discuss this in answer to Dino's questions below.

   - Robin

I have renamed this proposal Ivip (pr: eye-vip), though maybe "IVip"
or "IViP" would be better:

   Internet Vastly Improved Plumbing
   Internet Versatile redIrection of Packets
   Internet Versatile indirection Protocol
   Initiative to Virtualise IP addresses
     . . . etc.

This is a short acronym which is not particularly widely used, and a
quick search didn't reveal any use with this capitalisation.  "iViP"
won't do, because it looks too cute and commercial, like "iPod" -
and the Internet richly deserves its capital "I".

"Vastly improved Plumbing" is probably easiest way to understand
what I am trying to achieve.

I think the same thing can be achieved with LISP, but LISP requires
that the ITRs be either internal or border routers.  Ivip's ITRs can
be internal routers, border routers or transit routers.
(Theoretically, the Ivip ITR function could be performed in an
isolated router just for this purpose with a single connection to
any transit router.)

The primary benefit of Ivip over LISP (not counting Ivip's mobile
tunnel idea) is that it should be easy to incrementally deploy.
With LISP, any recipient host which uses a LISP EID as its address
will only be reachable from sending hosts located in edge networks
which have upgraded their border routers and/or internal routers to
perform the ITR function.  This is because LISP EIDs are IP
addresses which do not match a BGP-advertised prefix.

Ivip is a lot like LISP, but when a sending host sends a packet to
what I call an "Ivip-mapped IP address", the packet will be
delivered even if the sending host's edge network has not altered
its routers in any way.  The encapsulation will be performed by an
Ivip ITR (Ingress Tunnel Router) somewhere in the "core".  This is
most likely to be a transit router which also performs Ivip ITR
functions.

I think that there are different conceptions of what LISP can or
should do - for instance I think my understanding of LISP's power
and scope is more expansive than Dino's.  Not counting the mobile
tunnel idea, my conception of LISP's power and scope to provide
portable IP addresses, multihoming and maybe traffic engineering is
just as high as for Ivip.  I think Ivip's major advantage is that it
could be happily incrementally deployed - even as a paid for service
- whereas I think there is a major chicken-and-egg problem with
incrementally deploying LISP.

Below, I try to explain my conception of how LISP and Ivip would
best be used to achieve the major goals (multihoming, portability
and Traffic Engineering - TE) of those who use IP addresses, while
minimising the growth of the "global BGP routing table".

Let's say there was what I will call a "master subnet" which was
"LISP-mapped" or "Ivip-mapped": 22.22.0.0/16 .

The entire LISP or Ivip system would probably handle multiple such
"master subnets".  Most of these "master subnets" would hopefully be
bigger (shorter prefixes) than this for Ivip, because I intend Ivip
to minimise the number of new BGP advertisements while having as
much address space as possible to split up and assign to different
customers.

In both LISP and Ivip, any packet sent to an address in this range:

  22.22.0.0 to 22.22.255.255

will only be delivered to its proper destination if it is
encapsulated by an ITR, sent to the correct ETR and then
decapsulated and locally routed to the destination host.

For both LISP and Ivip, the ETR must:

1 - Have an IP address which can be reached directly via
    the current BGP routing system.

2 - Must be located in the edge network of the receiving host -
    either as a border router or an internal router.

3 - Must connect to a part of the internal network which is able
    to forward the decapsulated packet to the receiving host.

(It is also possible for the receiving host to do its own
decapsulation, although this probably achieves nothing useful other
than potential multihoming and TE, since the host needs a normal BPG
address to be able to do this.)

Now, for either LISP or Ivip I will consider different ways the
central database could direct packets addressed to this "master
subnet" to be handled by the ITR.

Mode 1 - Tunnel packets addressed to the entire subnet to a single
         ETR.

Mode 2 - Split the subnet into two or more smaller subnets, with
         packets addressed to each smaller subnet being tunnelled to
         different ETRs in the same edge network.

Mode 3 - Split the subnet into 2 to 10 smaller subnets, with packets
         addressed to each smaller subnet being tunnelled to ETRs in
         different edge networks.

Mode 4 - As for (3) but split the subnet into hundred or thousands
         of much smaller subnets, including individual IP addresses,
         and tunnel packets addressed to these subnets to ETRs in
         hundreds or thousands of different edge networks.

As I understand it, Modes 1 and 2 provide no substantial benefit in
terms of reducing the growth of the global BGP routing table.  There
may be some benefits in multihoming or portability of numbers
(making these mapped numbers usable no matter which ISP the ETR and
receiving hosts are located in).

With LISP, each initial "master subnet" does not produce any burden
on the global BGP routing table, whereas with Ivip, each one does
present a burden, since it is advertised in BGP.

With Ivip - and the proposal's viability rests on this being
possible - there are (ideally) thousands of Ivip ITRs in the "core"
of the Net, each advertising this "master subnet" or as many "master
subnets" as there are in the entire Ivip system.  No matter where a
sending host sends the packet from, if the packet is addressed to
one of these "master subnets" it will soon be forwarded to one of
these Ivip ITRs and tunnelled to the proper ETR.

I intend that the "master subnets" which are "Ivip-mapped" be
generally large, such as tens of thousands of IP addresses.  An
extreme case would to dedicate one or more /8s to Ivip.  (If this
was done, there could still be load balancing between multiple ITRs
in the same location, by each advertising a portion of this.)

I intend that each such "master subnet" be split into *many* smaller
sections, including single IP addresses, as for Mode 4 above.

Let's say the Internet has the following characteristics:

10,000 ISPs, with a total of 100,000 border routers, most of them
multihomed, advertising a total of 200,000 prefixes.  Most of these
prefixes are shorter than /24, because ISPs are generally interested
in large slabs of address space so they can connect lots of customers.

10,000 "AS-end-users", meaning companies, non-profit organisations,
universities, government bodies and a few keen individuals - not
ISPs.  These have 50,000 border routers, most of them multihomed,
and they advertise 100,000 prefixes.  Many of them advertise a
single /24 on one or the other of their links, even though they only
need one or a few IP addresses.

For simplicity I am discussing only IPv4 addresses and am ignoring
3G mobile carriers who probably want huge slabs of IP addresses for
their hundreds of millions of handsets.  (They may want IPv6 to fit
in with the Mobile Multimedia Subsystem.)

Let's say these ISPs and AS-end-users are quite happy with their
current situation, although their needs generally grow, which places
upwards pressure on the size (300,000) of the global BGP routing
table.  That growth is part of the problem, but the major problems
in IPv4 routing and addressing are:

1 - The shortage of fresh IPv4 addresses due to the need to
    assign them in largish chunks, including in the forlorn hope
    of maximising route aggregation.  Also due to the /24 limit
    on BGP advertisements which means 256 long subnets are
    assigned to many AS-end-users when they would be quite happy
    with a handful of addresses, if there was some other way
    of achieving multihoming, portability and maybe TE.

2 - The desire and need for new ISPs to connect their border routers
    and to gain new, large, slabs of IPv4 space which they will
    split up into still-largish subnets and advertise on BGP.  These
    new ISPs, over the next decade or two, number in the thousands
    or tens of thousands.

3 - The hundreds of thousands, millions and potentially tens of
    millions of large, medium and especially small end-users who
    want and need multihoming, ideally would like IP address
    portability, and who may want some TE capabilities.

    Without a new routing and addressing architecture, these will
    be either forced to become AS-end-users and further burden the
    global BGP system with routers and advertised prefixes, or will
    have to put up with single-point of failure links to an ISP,
    using non-portable IP addresses from that ISP.

I see LISP and Ivip as equally applicable to solving these problems
- with the small caveat that Ivip does add a little further to size
of the global BGP routing table for each "master subnet" the system
handles.

I intend that Ivip work with a relatively small number of "master
subnets" - a few to at most several hundred.  However, it is
possible that after 2011, it will make sense to grab sizable chunks
of IPv4 address space from whoever it is currently assigned to, when
that space is either not being used or is being used so lightly that
this cannot be justified while so many other ISPs or end-users need
the space and will be able (via LISP or Ivip) to use it very
efficiently.

"Efficiently", to me, means that the majority of addresses are used
by a host or a router.

I foresee the primary benefit of LISP or Ivip being the ability to
give an end-user a single IP address, or a small subnet such as four
to 16 addresses, which closely matches their immediate and
near-future needs, where those addresses will be:

a - Portable and usable via any ISP connection, as long as the ISP
    has an ETR (or as many ETRs are as needed to support such
    customers).

b - Multihomed to a certain extent with a single upstream link to
    the ISP, assuming the ISP is multihomed.  However this is not
    full multihoming.

c - Capable of being truly multihomed, if the end-user has two
    or more links to ISPs which have ETRs.  Then, the end-user
    (or some automated fault sensing centralised control system)
    will control the central LISP/Ivip database to tunnel packets
    to whichever ISP's ETRs it desires.

d - Capable of some degree of TE, such load balancing by having
    some of the end-user's IP addresses using one ISP's ETR and
    the others using the ETR of one or more other ISPs.

With a successful LISP system a single, unified Ivip system or even
multiple, independent Ivip systems (assuming the Ivip systems all
use relatively large "master subnets"), then provided all these
approaches divide their spaces down very finely, including to
individual addresses, this enables the Internet to cope with problem
3 above, very nicely, without any of those new end-users needing to
become Autonomous Systems, have BGP border routers, clutter the
global routing table with their /24s or waste precious IPv4 space
with those 256 long subnets when they probably only want one or a
handful of IP addresses.

(I am assuming widespread or ubiquitous NAT adoption for IPv4.
Maybe if IPv4 had always been flat-routed to individual IP address
granularity we could have used the space efficiently and avoided the
address shortage and therefore the evils of NAT, but I can't see how
the IPv4 Internet will ever be substantially free of it.)

Problem (2) is not altered directly by LISP or Ivip.  However, it is
made easier to solve if LISP or Ivip largely or completely handles
the vast number of end-users who need multihoming, portability etc.

As long as LISP or Ivip divide their subnets up very finely, then
this is the best possible contribution which can be made to solving
the address shortage problem (1).  Over time, I foresee Ivip or
something similar being used to efficiently use more and more of the
currently assigned address space.  All this reduces pressure on the
supply of fresh space.

In principle, if LISP or Ivip had been implemented from the
inception of the Internet, then the BGP routing system might only
cover a small subset of the available space.  The whole system could
have been engineered with a maximum number of advertised prefixes, a
maximum number of border routers - and therefore a maximum (and
hopefully high) number of ISPs, or at least ISP sites.  Then, most
of the rest of the address space could be handled with LISP or Ivip.

This raises difficult practical questions of how the ITR, for either
LISP or Ivip, efficiently handles hundreds of millions or even
billions of divisions of the address space for different tunnelling
treatment.

I am thinking of ASIC and RAM-based techniques optimised for such
lookups - which is one reason I am attracted to the idea of the ITR
function being performed by a smaller number of the most advanced
routers, such as transit routers (Ivip), rather than a larger number
of less advanced routers in edge networks (LISP).

If LISP was successful at individually mapping the IP addresses and
small subnets of millions of multihoming, portable IP address
desiring end-users, the requirements on each ITR's FIB would be
really onerous - and performing the functions in the router's CPU
would lead to really poor performance.

I hope the above description conveys my vision of Ivip (or LISP)
vastly improving the plumbing of the Internet.  Either should be
able to achieve, for end-users, the appearance of millions of
flexible "pipes", when in fact the BGP routing system only provides
a few hundred thousand physical pipes.

Christian Vogt wrote in message 01524 (with ViP changed to Ivip):

> is it correct to say that ViP anchors an access (or edge)
> network's prefix at the V-router somewhere within the DFZ?

If Ivip was used in modes (1) or (2) above, where an entire isolated
prefix of Ivip-mapped address space (for instance bounded by BGP
advertised prefixes above and below) had packets addressed to this
range tunnelled always to the one edge network, then the answer may
 be "yes".  However, like LISP, there are many such ITRs and the
destination address (EID for LISP) would be accessible by tunnelling
from any of those ITRs.

A LISP ITR which was also a multihomed border router would be in the
DFZ.

I will try to avoid the distracting term "V-router".  There would be
many Ivip ITRs and as I wrote above.  I think a single largish
"master subnet" like 22.22.0.0 to 22.22.255.255 should be split into
hundreds or thousands of separate mappings, to serve hundreds or
thousands of users.  Generally those users would connect by hundreds
or perhaps thousands of ISPs, but it would be technically possible
to have all those end-users accessing the Net via a single ISP.  At
least each one could move to another ISP which also had one or more
ETRs and take their IP address with them.

So generally I see each "master subnet" in LISP or Ivip being used
to direct packets to hundreds or thousands of separate edge networks.

Generally I think the best place for Ivip ITRs is at border routers
(single and multihomed) or in transit routers, so my initial mention
of ITRs in the DFZ was too loose, since single-homed border routers
are not in the DFZ.

Ivip ITRs could also be internal routers in the sending-host's edge
network.  They would grab the raw packets before they got to the
border router or out past the border router.  This could be a way of
removing some or all of the load of ITR work from the border
routers, which already have a huge number of prefixes in their FIBs.
(Assuming they are multihomed, with the full BGP set, plus all the
prefixes in the local network of telco carriers, ISPs and large
corporations etc.)

Whether a LISP or Ivip ITR uses push (has the complete database at
all times) or pull (has to query, learning not to query about
prefixes for which it recently got a "not mapped" response), the FIB
function has a lot of work to do and a large amount of data to
store.  My current feeling is that this can and should be done by a
smaller number of really advanced routers, such as the CRS-1, with
lots of memory and flexible dedicated CPUs (188 32 bit CPUs on a
single ASIC, the world's largest) - rather than on a larger number
of smaller ones.

> In this case, I would see a difference between Ivip and LISP
> in terms of responsibilities:
>
> - In Ivip, it is the responsibility of the destination access
>   network to set up (or subscribe to) the anchor router for its
>   prefix.
>
> - LISP relies on a router in (or at the border of) the source
>   access network to perform this functionality.

Ivip, in principle, could work with a single ITR (which I think is
what you mean by "anchor router") which is accessible via BGP.
However all traffic would have to be encapsulated by that single
router, so I suggest there be thousands of ITRs all over the world,
at border routers or close to border routers.

The way I am explaining Ivip is as a single, global, system with a
single centralised database which all ITRs have an up-to-date
version of.  This requires a push system, I guess, or some fancy
continuous "pull" system.  Ivip or LISP could work with a partial
database, with a "pull" system querying a central or distributed
system whenever a packet arrives with an unfamiliar IP address, but
that is very messy, has inherent delays, and still requires a fancy
FIB function and a lot of data storage.

There could be multiple Ivip systems all operating independently.

For instance, if I had an ASN and had a prefix assigned to me, I
could set up a bunch of ITRs linked to a central database.  Then, as
long as at least some ISPs had a suitable ETR (which only has to
strip the IP-in-IP header off a packet which is addressed to one if
its IP addresses) then I could rent out individual IP addresses or
subnets from my assignment, to any customer who was happy to use one
of the ETR capable ISPs.  (The ISP would need to configure their
routers to handle the IP addresses I assign to those end-users.)

This would be the RW Ivip network - so any customer of mine (someone
I rented one or more IP addresses to) moved to another ISP, they
would need to tell my system the IP address of the ETR in their new
ISP.  I think this means roughly what you wrote: "set up (or
subscribe to) the anchor router for its prefix.".

> This ViP concept reminds me of Mobile IP (v4 or v6), where
> a mobile host anchors its home network prefix with a home
> agent, the location of which is independent of the peer that
> the mobile host communicates with.  Both ViP and Mobile IP
> use tunneling between the anchor and the destination.

Yes, but Mobile IP, as I understand it and with the Ivip-mobile
(ViP-mobile) as I described in:

  http://www1.ietf.org/mail-archive/web/ram/current/msg01518.html

there is a two-way encrypted tunnel (or maybe two or more such
tunnels at the same time using different IP addresses, different
links via different ISPs etc.) between the mobile device and a
single "anchor router".

My Ivip-mobile idea is like this, assuming there are multiple hosts
H1 to H4 with ordinary BGP-routable IP addresses sending packets to
the one mobile device.

There are various ordinary Border Routers (BR) Transit Routers (TR)
shown for realism.

The ITRs are either border routers or transit routers.

     ITR1
H1---(BR)---------(TR)-\
     /                  \
H2--/                   (TR)-\
                        /     \
                       /       \   Translating
                      /         \  Tunnel Router
	     ITR2    /		 \					H3---(BR)----(TR)---/
(TTR)~~~~~(TR)~~~~(BR)~~~(NAT)~~(MH)
                 \               /
H4---(RR)-------(TR)------------/

H1 and H2 send raw packets to ITR1, which is the border router of
their edge network.  ITR1 tunnels the packets to the TTR using
IP-in-IP encapsulation.  LISP could do this too.

H3's and H4's packets leave their edge networks.  (This could not
happen with LISP, because they are not addressed to a prefix which
is advertised in BGP.)  These packets are forwarded by the BGP
routing system to the nearest router which advertises the prefix
they match.  For H3 and H4, this is the transit router ITR2.  ITR2
does the same as ITR1 - tunnels the packets to the TTR.

If this was Ivip-basic, both ITRs would tunnel the packets to the
ETR inside the edge network where the receiving host resides.

In principle, I guess the TTR could be a border or perhaps an
internal router, but it makes most sense if it is either a transit
router, or at least in the core of the Internet.  In practice, there
would be many TTRs, all over the world.  The Mobile Host (MH) would
somehow create an encrypted two-way tunnel to a TTR which was close
or closest to its current location.

That is a totally separate tunnel, shown as ~~~~~.  It is initiated
by the MH, as a VPN tunnel.  As such, it doesn't matter if the MH is
behind one or more NATs.  It would also be able to create this
tunnel to the chosen TTR if its current connection to the Net was
via Ivip-basic, but lets not try to think about that now!

With Ivip-basic, as per the diagram in my first message 01518, the
system only handles packets from left-to right.  Packets in the
return direction use ordinary BGP routing, assuming the host on the
left has an ordinary BGP-advertised IP address.

With this Ivip-mobile scenario, packets going back from the MH to
any of the H1 to H4 hosts, first travel in the encrypted VPN tunnel
to the TTR.  The TTR then extracts those packets and throws them
into the ordinary BGP routing system.  Since all hosts H1 to H4 have
  ordinary IP addresses, the packets travel by normal means to them.

The TTR needs to check the packets it extracts and forwards, for
instance to check the source address is as expected, to prevent
spoofing.

If there was an H5, which used an Ivip-basic-mapped IP address,
there are two ways the TTR could send packets to it.  Firstly, it
could place the packets into the ordinary BGP routing system, in
which case they would soon find themselves being encapsulated at an
ITR and tunnelled to H5's ETR.  Secondly, the TTR could perform the
ITR function itself: encapsulating the packet with IP-in-IP, and
then letting the main BGP routing system transport it to H5's ETR.

If there was an H6, which used Ivip-mobile, the same two approaches
could be used.  H6's ETR is actually a TTR, presumably different
from the TTR in the diagram above.  A TTR would also have to handle
packets between two mobile hosts which have tunnels to it.

I guess these mobile services are going to be a commercial service
since I can't imagine anyone running routers in the middle of the
Net like this for free.

I think a mobile host should be able to set up a second tunnel to a
different TTR, or perhaps a second tunnel via a different edge
network to the same TTR.  In the latter case, the TTR can decide
which tunnel to use, depending on some handshake or whatever to see
which one works best, or at all.

Ideally, if the one mobile host had two tunnels via two separate
ISP's networks (such as a 3G link and a WiFi link) to two TTRs, it
could switch to using the second TTR.  This would require that the
central database be changed so all the ITRs would tunnel (IP-in-IP)
packets intended for this mobile host to the second TTR rather than
the first.  This is just like taking a host from one ISP to another,
and changing the database to point to the new ISP's ETR.

>From the point of view of an ITR, it makes no difference whether the
ETR is an ordinary Ivip-basic ETR or a TTR.

> In fact, ViP is a proposal for network mobility.  It enables an
> access network to change ISPs (which is nothing else but a
> topological "movement") without losing reachability via the
> original prefix.

The explanations I have given were for single IP addresses - but the
same principles apply to an entire prefix.

Considering the whole "master subnet" or "master prefix"
22.22.0.0/16 which is one of the isolated prefixes handled by this
example Ivip system, I assume that prefixes above and below this are
not handled by Ivip.  So to use this prefix for Ivip, it has to be
advertised in BGP.  If the whole prefix was not split (Mode 1 above)
then it could be used as you describe, providing portability amongst
ISPs for the whole prefix.  This would save the end-user from having
to renumber when moving to another ISP.  However, it doesn't really
help reduce the size of the BGP routing table, since without LISP or
Ivip, this user would have got an ASN, been assigned the prefix as
PI space, and advertised it in BGP.

Also, with LISP or Ivip, if the end-user has two links to two ISPs
and can alter the mapping database rapidly when the currently used
ISP's link fails, then they have achieved multihoming.

LISP generally helps reduce the size of the global BGP routing
table, because no matter how small or large its "master prefixes",
and no matter whether the "master prefix" is split to be mapped to
multiple customers at multiple ETRs in multiple edge networks,
whatever it provides is without additional BGP advertisements.
(However this is on the assumption that devoting a prefix to LISP
doesn't punch a hole in a shorter prefix - longer subnet - which
causes it to be advertised as two prefixes rather than one.)

> Interesting...

Thanks!

Dino wrote, in message 01519:

>>      An Ivip-mapped address will be universally reachable as long
>>      as there is a single reachable Ivip ITR which can receive
>>      packets from the sending host and send encapsulated
>>      packets to the ETR for this address.
>
> When you say LISP-mapped or Ivip-mapped, it is not clear to me if
> you mean "map to" or "map from". Can you please clarify?

By "Ivip-mapped" I mean an IP address (or subnet/prefix) which is
not handled fully by the current BGP system, but causes packets
destined for this address or subnet/prefix to be tunnelled to an ETR
close to the receiving host.  So I mean the original address, as
used by the sending host (EID for LISP) is "mapped to" the IP
address of an ETR.  I haven't considered multiple ETRs, load
balancing etc. as you do in LISP, but that should work as for LISP.

I am trying to avoid "identifier", "RLOC" etc. terminology.

I would describe an IP address (or subnet/prefix) which is an EID
under LISP as "LISP-mapped".  Packets sent to this address or subnet
or prefix are "mapped" to be tunnelled to another IP address - the
RLOC - which is the address of the ETR.

An "Ivip-mapped" address is not directly reachable via the ordinary
BGP routing system.  However, a packet addressed to an "Ivip-mapped"
address will be forwarded by conventional BPG routers to the nearest
Ivip ITR.  If all is well, the ITR will tunnel it to the correct
ETR, just as with LISP.

>> 4 -  Point 3 has a profound effect on how the two proposals may
>>      be incrementally deployed.  With LISP, there is a serious
>>      and perhaps insurmountable barrier to anyone deciding to
>>      connect a host to a LISP-mapped address.  The first people
>>      to do this will find their addresses generally unreachable
>>      since virtually no edge networks will have upgraded any
>>      of their internal or border routers to perform LISP ITR
>>      functions.  Why should they?  It is a huge investment to
>>      enable communications with a handful of rebels who are
>>      voluntarily opting out of the BGP system (albeit for the
>>      benefit of humanity).
>
> A LISP site will have to use both PI and PA addresses for some
> time.  However, when it uses PI addresses that *will not* be in
> the core routing tables, other sites can reach them only if they
> are LISP enabled. Otherwise, the non-LISP enabled sites will reach
> the LISP site with PA addresses.
>
> So we can still reduce the size of the routing table and arguably
> more-specific injection from other PA blocks.

I understand from this that your transition mechanism involves each
receiving host (each host with a "LISP-mapped" IP address) having
two IP addresses at once: one conventional and one "LISP-mapped".  I
am not sure how this would be practical, unless host and operating
system software in the sending host could somehow choose between two
IP addresses returned by DNS, and use whichever one worked first.
If that could be done, the transition mechanisms would double the
number of IP addresses in use, at a time when they are in short supply.

Also, as long as the receiving host was reachable without LISP, I
don't see why the edge network of the sending host should invest in
one or more LISP ITRs.

I assume users cannot be expected to be involved in any of this.
For instance I assume it is not expected that people would be asked
to use wwwl.example.com if their ISP had LISP ITRs or otherwise to
use www.example.com.

>>      I can't imagine how LISP would ever get off the ground,
>>      since early adopters would have extremely patchy
>>      connectivity.  No-one would put a crucial service on a
>>      LISP-mapped address.  Since all crucial services would
>>      be available on ordinary BGP-routed addresses, why should
>>      edge networks spend money on a new or upgraded router?
>
> Because they want better control over their ingress traffic flows.
> They need the level of indirection to achieve this.

I think TE is a minor concern compared to the demand for multihoming
and portability - at least among the millions of smaller end-users I
am primarily thinking of.

I can't imagine anyone wanting to switch their server, site or
desktop computer to an IP address which had only marginal or
imperfect two-way communications with the rest of the Net in order
to achieve TE, portability, multihoming or even to be paid thousands
of dollars!

>>      ViP requires no changes in the edge network of the sender.
>>      It can be implemented with a single ITR on some BGP router
>>      somewhere in the world, and a single ETR in the edge
>>      network of the destination host.
>
> Can you explain how Ivip works with PI addresses?

I think your plans for using LISP - and no-doubt the plans of others
- differ considerably from mine.  I will try to state my guesses
about what you have in mind.  Please correct me if I am wrong.

I think you envisage LISP primarily serving the needs of
AS-end-users and perhaps ISPs.  I think this means current and
future AS-end-users, most of whom primarily want multihoming and
portability for their network.

I think you are primarily thinking of AS-end-users who want or need
thousands of IP addresses.

I foresee the most important users of LISP or Ivip as being partly
large end-users who may now have an AS or might otherwise be tempted
to get one in the future.  I figure virtually none of these people
or organisations have any enthusiasm for BGP.  They just want a
number of IP addresses connected to the Net, with multihoming via
two different links to two ISPs and/or with the ability to keep
their numbers so they don't have to renumber their networks when
they select another ISP.

The other users, probably the most important and numerous set, are
those who only need one or a few IP addresses.  I guess these
end-users are probably not too fussed about portability.  Their
networks are either small or have so few exposed IP addresses that
its not too much trouble to renumber a handful of routers or hosts.
 I figure they primarily want multihoming via two separate links, so
their businesses and organisations (their lives, with Second Life .
. . ) are not literally dangling from a single piece of fibre or copper.

I wonder about users who really want and need 256 or more IP
addresses.  Unless they are a little hosting company, I can't
imagine that many end-users really need 256 IP addresses at any one
site.  The most likely reason I guess end-users want to get an ASN
and 1024 IP addresses is so they can split them into four /24s and
advertise them at each of their branch offices.

It would be best if someone could provide real examples based on
experience rather than my guesses about what users want.  I am
thinking of people running small companies, schools, hospitals,
seriously Net-connected home offices etc.  Most of these end-users
want to run a mail server and perhaps reasonably high-volume web
servers from their physical sites.

I don't think LISP or Ivip helps much with new ISPs or the expansion
of current ISPs.  All ISPs, unless they are to become part of some
second-rate sub-caste of BGP-less ISPs, need an ASN, full BGP
expertise, multihomed border routers and their own slabs of address
space so they can do traditional BGP-based multihoming, offer
LISP/Ivip host connections etc.

I am hoping that LISP / Ivip or whatever will enable large areas of
address space to be very finely divided and used to meet the needs
of millions of companies, schools, SOHO people etc.  This will make
it unnecessary for most of them to get ASN numbers and to apply for
genuinely PI address space.

I think you envisage a "BYO PI space" model for LISP.

For both LISP and Ivip I foresee some larger entity, perhaps RIRs or
perhaps ISPs or commercial operators, providing stable, large,
ranges of address space for the system, and then having the system
divide it finely for millions, tens or even hundreds of millions of
end-users.

If there were plenty of ISPs with ETRs in place, there is nothing to
stop multiple ISPs or whoever else setting up their own Ivip systems
now.  All they need to do is have their own PI address space, set up
a database and a bunch of ITRs around the net - for instance at
various Internet peering exchanges or at the border routers of ISPs
around the world - and they can start renting out IP addresses and
subnets of addresses to anyone they like, provided those users
connect to the Net with an ISP who runs an ETR and is prepared to
route these packets to these customers.

>> 6 -  Because LISP needs to be a single system, it needs to be
>
> LISP needs to be a single system only when the entire network
> runs on PI addresses. That won't happen for a very long time.

What I meant to say is that I don't think that there could be
multiple LISP systems operating at once, because each system would
have to convince a huge number of ISPs all over the world to invest
in, or allow the installation of, an ITR which would handle the EID
to RLOC mapping of the IP addresses each system used as EIDs.

Maybe there could be a tech standard for ITRs and an edge network
could somehow let multiple LISP system operators control its
operations - but that sounds unlikely.

I could imagine multiple Ivip systems, because there is no need to
install ITRs in edge networks.  What would be required, as with
multiple LISP system, is a tech standard for the ETRs so all systems
could land their encapsulated packets in edge networks, without each
edge network needing to dedicate an ETR to each specific system.

If there were multiple Ivip systems, each system's ITRs would only
handle packets addressed to BGP advertised prefixes which were the
PI address space of that system's operators.

Since these ITRs will cost money, and do work for specific
end-users, I think there are arguments for multiple competing Ivip
systems to be created, charging the recipient host users for their
IP address space and according to traffic volumes.  Such a network
of ITRs could also be extended to do Ivip-mobile, which would
involve a lot more resources and definitely need to be a paid-for
service.

Multiple Ivip systems wouldn't need to know about each other, but it
would be best if they all operated on similar technical principles
and could all use the same ETRs in edge networks.

>> 7 -  One of the primary benefit of Ivip, together with providing
>>      reachability from the outset without requiring sending
>>      edge networks to do anything, is that there would be a
>>      smaller number of ITRs in the system.  These will tend to
>>      be bigger than with LISP and will have more intensive work
>>      to do.
>>
>>      I am thinking of direct RAM-based lookup approaches,
>>      involving two or so memory cycles, to speed this operation
>>      in hardware.
>>      Perhaps one or more models of current high-end router,
>>      such as the CRS-1, could perform these functions nicely,
>>      with an extensive firmware upgrade.
>>
>>
>> 8 -  With Ivip, packets to be encapsulated reach the edge of the
>>      BGP system or are forwarded within it, either to a single
>>      ITR (if the Ivip system only has one V-router) or to one
>>      of multiple ITR functions in separate V-routers all of
>>      which have the same IP address.
>
> Do you actually think a few ITRs is going to handle the load of
> the Internet?

To start with, strictly speaking, only one is needed to make the
system work.  To minimise the extra travel of packets, and for the
system to scale indefinitely, there would need to be hundred,
thousands and then tens of thousands of ITRs.

I figure this would be cheaper, more stable, and more scalable than
replacing all DFZ routers with souped-up versions to handle the
alternative: an ever growing BGP routing table.

I figure if there is a way that existing hardware, such as the 188
CPUs per SPP chip and massive high-speed RAM of the CRS-1's ingress
FIB could do the ITR functions, with little or no main CPU
involvement, it would be best to have a smaller number of these
around the Net, each directly "push" updated with the mapping
database second-by-second, than to get the same work done by much
slower software-intensive hacks in smaller routers in edge networks.
 Those more numerous smaller ITRs would also be more of a hassle to
keep fully updated.

>>      Ivip uses Anycast to provide multiple redundant paths
>>      within the BGP system for packets to find their way to
>>      an ITR.  Anycast is not normally used for TCP
>>      communications, but I am hoping it will be appropriate
>>      here, since the individual ITRs only encapsulate the
>>      packet and tunnel it to the ETR.
>
> I have actually tested LISP using anycast addresses and it has the
> same property. If ITRs have the following mapping cached:
>
> {0.0.0.0/0, > 1.1.1.1, 2.2.2.2} where 1.1.1.1 and 2.2.2.2 are
> anycast addresses, you can have various ITRs use the closest
> ETRs based on the above locator addresses.

My understanding of what you are doing here is a completely
different use of anycast than what I had in mind.  I was thinking of
anycast as a way of making multiple BGP routers accept (or rather
attract) the packets which were addressed to a prefix which needed
to be handled by an ITR.  So I was thinking of all those ITRs having
the same IP address.

Now I don't think this anycast has any purpose.  All that is
required is to make multiple ITRs advertise the same prefix.

Is there any precedent for two or more BGP routers advertising the
same prefix?  From what little I know of BGP, it should work fine -
but I would really appreciate a BGP expert's view of this.

I understand the purpose of your use of anycast is to deliver the
packets to a receiving host closest to the ITR.  Since the sending
host is presumably close to the ITR and the receiving host is surely
close to its ETR, this provides faster responses, less traffic
across the Net etc.  I hadn't thought of this.  I think it is a good
idea.

Since the ITR function of Ivip would be similar or identical to that
of LISP, the same thing could be done.

> In the spec we call these boxes "reencapsulating routers", where
> the anycast box decapsulates and then does a lookup on the inner
> DA where it has a different mapping (a list of locators where the
> site actually is) to reach the site's ETR.

Hmmm - this is a totally different purpose from what I guessed when
writing my response above.  I don't understood the need for multiple
encapsulations in LISP or Ivip.  Can you give an example of when
this would be helpful?

>> 9 -  LISP and what I will call "Ivip-basic" both have their ETRs
>>      in edge networks, or at the border router.  These ETRs must
>>      have an interface with an IP address which is reachable via
>>      the global BGP system.  These only handle packets one way -
>>      from the sender HA to the LISP/Ivip-mapped destination
>>      address of HB.
>>
>>      Packets in the return direction HA <- HB are sent normally,
>>      without relying on tunnelling, if the destination HA's
>>      address is not LISP/Ivip-mapped.  Otherwise, a similar
>>      process occurs with the packet being intercepted by an ITR
>>      and tunnelled to the ETR which handles HA's address.
>
> This can only work if HA is using a PA address.

I don't think it matters whether HA is using PI or PA addresses.  As
long as HA's address is part of a BGP advertised prefix which is not
handled by Ivip ITRs, then the the packets to HA will flow to it as
they do today, without using LISP or Ivip.

I wrote this point 9 as part of a sequence of differences between
LISP and Ivip, but this point only describes similarities and then a
separate mobile tunnelling idea.

>> Example of ViP-basic
>> --------------------
>>
>> ASCII Art only works with fixed width fonts:
>>
>>
>>   Edge Network A
>>                           [ITR]
>> HA---(IR)----(BR1NA)------(VR1)-----(TR2)--(TR4)   Edge Network B
>>                     \         \                \
>>                      \         \                \        [ETR]
>>                       TR1)----(VR2)---(TR3)---(BR1NB)---(IR)---HB
>>                          \             /
>>                           \           /
>>                          (TR5)----(TR6)
>>                                       \
>>                                        \           Edge Network C
>>                                         \ [ETR]
>>                                         (BR1NC)----HC
>>                                                \
>>                                                 ---HD
>
> I don't understand the benefit of reducing the number of ITRs but
> still maintaining an ETR in each site. You still have to manage
> the ETRs. So the gain is not that great.

I think the ETRs have a light workload and don't need any fancy
database.  Maybe existing routers can be programmed to look out for
an IP-in-IP packet addressed to one of their IP addresses and then
to simply pop off the header and treat the remaining packet as if it
had just arrived.  I recognise there may be more to do, perhaps
regarding ITRs trying to see if the ETR considers a destination host
reachable at present, and maybe things to do with transferring
hop-counts to the decapsulated packet.

I think ETRs are easy and could probably be generic.  I don't think
they need central management.

ETRs only need be installed at ISPs where customers are paying for
access and have chosen to use LISP/Ivip-mapped addresses.

Ivip requires a suitable number of high-performance ITRs out in the
Net in general, and LISP requires at least one ITR in every edge
network, ISP or AS-end-user, where there is a host which wants to
communicate with the host which is on a "LISP-mapped" IP address.

>> HA's address is 11.11.11.11, which is part of a prefix which is
>> advertised in BGP and which is not covered by Ivip mapping.
>>
>> HB's address is 22.22.0.1, which is part of 22.22.0.0/16, which
>> is also advertised in BGP and which is Ivip-mapped.  The other
>> hosts' addresses are:
>>
>>   HC = 22.22.4.1
>>   HD = 22.22.4.2
>
> If this is the example, why do you need ITRs or ETRs at all? If
> you have routeable addresses just move the packets as we do today.
>
> Dino

Because my intention is to split up the 65,536 addresses to be used
by customers far and wide.  In principle, each single IP address
could be mapped to a host in a different edge network.

I think your idea of how this stuff can best be used is rather
different from mine.

_______________________________________________
RAM mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ram

---

From: Robin Whittle <[email-masked]>
Date: Sat, 16 Jun 2007 22:03:12 +1000

The diagram in my previous email was crook.  Here it is again:

     ITR1
H1---(BR)-------(TR)-\
     /                \
H2--/                 (TR)-\
                      /     \
                     /       \   Translating
                    /         \  Tunnel Router
           ITR2    /           \			
H3---(BR)--(TR)---/            (TTR)~~(TR)~~(BR)~~(NAT)~~(MH)
               \               /
H4---(RR)-----(TR)------------/

---

From: Robin Whittle <[email-masked]>
Date: Sun, 17 Jun 2007 21:49:22 +1000

I am replying to Ved's and Christian's comments in "Re: ViP:
Anycast ITRs in the DFZ & mobile tunnels", with "ViP"
changed to "Ivip" (eye-vip).

I apologise for the length again.  It would probably not be
so hard to describe Ivip to one person, face-to-face, with
conversation and hand-drawn diagrams.  Trying to do it for
a bunch of people via email takes more words.

VK> To me, the Ivip approach looks similar to the network mobility
VK> (nemo) support approach being discussed in the IETF nemo WG.
VK> In nemo, the mobile router's home agent advertises the mobile
VK> network prefix such that the packets sent by a correspondent node
VK> to an address belonging to the nemo prefix go to the home network
VK> of the mobile network, where the home agent tunnels them to the
VK> current location of the mobile router. The mobile router
VK> decapsulates and forwards the packets to the addressed node.

I threw this proposal together quickly without checking other
protocols - and I don't claim it is highly original.

There are two aspects to the proposal at present: "Ivip-basic"
and "Ivip-mobile".

Ivip-basic
----------

  H1---(ITR1)---\
      /          \
  H2-/            \
                  (ETR1)----RH
                  /
  H3---(ITR2)----/

       (ITR3)     (ETR2)

Sending hosts H1, H2 and H3 are assumed to have ordinary IP
addresses, reachable via BGP.  The ITR function is generally
performed in border routers or transit routers, but can be
performed in interior routers of the edge network.

Packets from the Receiving Host RH flow by ordinary BGP
routers back to the sending hosts.  If a sending host is also
using an Ivip-mapped IP address, then the same process occurs
in reverse, with the packet sent by RH finding their way to
the first ITR, probably at or near the border router of its
edge network, and being encapsulated and sent to the ETR for
the host on the left side.  None of this is shown in the
above diagram.

ETRs are provided by ISPs, or the AS-end-users who run the
edge networks.  They only serve receiving hosts in those
edge networks.

How does the edge network know that the decapsulated
packets emerging from the ETR must be sent to a particular
host in its network?  The ETR and the routers nearby
need to be configured to look out for this IP address,
or the Ivip-mapped subnet, and route it internally to the
receiving host.  With Ivip, these packets are addressed to a
prefix (a large subnet I call the "master-subnet") which is
advertised in BGP.  With LISP, they are not.

To support either LISP-mapped or Ivip-mapped IP addresses,
the edge network needs not only an ETR to decapsulate the
packets, but it needs a routing rule specific for that
IP address or subnet which directs packets addressed to that
address or subnet to wherever the particular host is located.

This is probably a non-trivial task for an edge network,
but the are being paid by their customer with the host which
uses LISP-mapped or Ivip-mapped addresses.

Assuming a single global Ivip system, the ITRs may be provided
by various operators, but work as a single system, without any
constraints on whose packets they will encapsulate.

What may be unique about Ivip is that these ITRs are numerous
and located (usually) at border or transit routers where any
sender's packets will eventually be sent to one.  What may
also be unique is the idea that all these ITRs advertise
the same set of prefixes: whatever "master prefixes" the Ivip
system handles.

Generally, these "master prefixes" are split into many smaller
sections, each with a different mapping to an ETR.  This works
fine down to individual IP addresses.  There needs to be a
single central system for controlling the mapping - and every
ITR has an up-to-date copy of this database.

Ivip-basic is a one-way, public, system.  Ideally it would be
provided for free, but there probably needs to be some fees
for running the ITRs and administering the database.  These
could be paid for by the Ivip "customers" - those individuals
and organisations etc. who pay for one or more IP addresses
in the system.

As shown above, the ETRs are in the edge network in which
RHs are located or connected to.

Each RH in this depiction has a single link to its ETR, but
an RH could easily have links to two ETRs, especially in two
edge networks, to achieve multihoming.

Ivip-mobile
-----------

Ivip-mobile is an extension of the above, but would probably
always be a commercial service.  There could be any number of
Ivip-mobile operators.  Each would have a set of special ETRs
called TTRs (Translating Tunnel Routers).  These TTRs are
in general either at border routers or more likely amongst
the transit routers.  TTRs could also be in edge networks.

  H1---(ITR1)---\
      /          \
  H2-/            \
                  (TTR1)~~~~~~~~MH
                  /            /
  H3---(ITR2)----/            /
                             /
       (ITR3)     (TTR2)~~~~/

The Mobile Host (MH) could be behind NAT and always
establishes an encrypted 2-way tunnel to at least one TTR.
(I am assuming an encrypted and compressed 2-way VPN-like
tunnel, but I guess it could also be done by simple two-way
IP-in-IP tunnelling.)

The Mobile Host may establish tunnels to multiple TTRs,
including using completely different links, such as ADSL and
GPRS radio.

The Ivip system maps the MH's IP address (or prefix) to one of
the TTRs, so all the encapsulated packets from the world's
ITRs arrive at that TTR.  Some central monitoring system
could detect that one TTR is dead, or that its link to the
MH is dead, and then the mapping could be changed to the
other active TTR.

To the Ivip system, which is really the ITRs and their
controlling database, there is probably no way of telling
whether the currently mapped-to IP address points to an ETR
which is for Ivip-basic (meaning the RH is in an edge network
and the ETR is a border or internal router of that edge
network, with packets flowing back independently of these
arrangements) or whether the mapped-to IP address points to
a TTR.

A TTR is simply an ETR which the MH tunnels to, and which
therefore also acts like the MH because this TTR is where
the MH's outgoing packets are first let loose on the Net.

With mobile-IPv6 (as I understand it), there is exactly one
router which the MH tunnels to - it is the router which runs
the LAN in which the MH's home address is located.

Mobile-IPv6 can also cause corresponding hosts (H1 to H3 etc.)
to send packets to and from the MH directly, rather than
using the home router.  This is more efficient, but requires
the corresponding host to have special mobile-IPv6
capabilities.

Ivip-mobile is very different from mobile-IPv6 in that the MH
can choose one or more TTRs to tunnel to.  Ideally it will
choose the ones which are closest to its current location.
Also, there is no "home IP address" on a LAN, as there must
be with mobile-IPv6.

As far as I know, mobile-IPv4:

  http://www.mip4.org

relies on a "home-router" and home IP address too.

Likewise, as far as I know, NEMO does as well:

  http://tools.ietf.org/html/rfc3963
  http://tools.ietf.org/wg/nemo/

Also, like LISP, Ivip works with single IP addresses or any
number of IP addresses.  I am not sure whether mobile-IPv6,
mobile-IPv4 support subnets, but I guess they could do it as
individual arrangements for multiple IP addresses.  NEMO
supports subnets.

The reason a figment of our imagination like Ivip can boldly
invoke a worldwide network of ITRs while mobile-IPv6/4
cannot is that now we are desperate.

If we don't do something on a grand scale such as this, we
will all be doomed to paying for endless upgrades to all the
Internet's transit and multihomed border routers as the BGP
routing table grows to 500,000, 1,000,000, 2,000,000 etc.

LISP boldly invokes at least one ITR in every edge network
which has a host which needs to communicate with a host or
network whose IP address or subnet is handled by LISP.  In
practice, for LISP-mapped IP addresses to be attractive to
anyone, this means most edge networks must have one or more
ITRs.

Ivip less boldly invokes ITR functions broadly distributed in
the border routers and transit routers.  I can get away
with this, as can the LISP authors, because we are desperate.

Both LISP and Ivip require the relatively modest ETRs to be
installed in the edge network of any host which has its
address or subnet handled by LISP or Ivip.

Ivip could also exist as multiple independent networks of
ITRs, each with its own "master-subnets" and its own central
database.

Since Ivip involves a bunch of souped-up routers at border
router and transit router locations, it is not much more of
a step to have some of these, or perhaps separate networks
of routers, performing TTR functions.  Each TTR system
could be entirely independent of other TTR systems, and of
the one or more ITR systems (provided the TTR did not also
implement ITR functions).  The multiple TTR systems could
compete and provide commercial services to the customers
who they provide Ivip-mapped address space for.

Please see my previous long message where I write about
my conception of Ivip or LISP, which I think is different of
Dino's "BYO address space" vision for LISP.

If there was a single Ivip system of ITRs, presumably the
system would not be allowed to fail, so any address space
it hands out via its mapping functions would be essentially
"Provider Independent".

If there were competing Ivip systems, then all of them,
apart from perhaps one "universal public" one, would be
dependent on some company, ISP etc. and be for-profit - so
the IP address still depends on that company.

However, in both cases, where the customer whose address is
mapped by these systems chooses to attach to the Net (their
choice of ISPs) is completely unrelated to the Ivip system(s).

Any TTR systems could be independent of the Ivip systems.

A customer could choose to use any number of competing TTR
systems.  They have nothing to do with IP addresses, since
their TTRs are only of value to the customer if the ITRs
tunnel packets to the TTRs, and since the customer pays for,
or has control of, the mapping of their IP address, the
customer only depends on the ITR system for the security of
"tenure" of this address - and not at all on any TTR systems
it uses.

Each TTR could also act as an ITR for that system's
"master-subnets".  If there was a single, global,
system of ITRs, it would be good if the privately run
TTRs also acted as ITRs within the global Ivip system.

Otherwise, packets would have to first find an ITR and be
tunnelled to the TTR when they are going to an Ivip-mobile
host.

For simplicity, the following assumes there is a single
set of "master-subnets" (AKA "master-prefixes") which are
handled by a single Ivip system which includes an Ivip-mobile
function as well.

It is likely that a router which performs TTR functions will
also act as an ITR.

  H1------(TR)------------(ETR1)------RH1
            \
  MH2~~~~~~(TTR1)~~~~~~~~MH1
            /  \
  H2----(ITR1)  \
         /      (TR)
  H3----/         \
                   \
  MH3~~~~~~~~~~~(TTR2)

H1 and H2 have ordinary BGP routed IP addresses.  Packets sent
from H1 to MH1 are received by TTR1 which acts like an ITR.
Ordinarily, an ITR encapsulates the packet and tunnels it to
the ETR to which the destination (MH1) IP address is mapped
to.  In this case, TTR1 is the destination ETR, so there is
no need to encapsulate the packet.  It is simply placed on
the VPN 2-way tunnel ~~~~~~~~ and sent to MH1.

Packets from MH1 to H1 follow the reverse path - they come
out of the VPN tunnel at TTR1 and are forwarded via normal
BGP routing back to H1.

The same would be true between H1 and MH2.

Packets from H2 to MH1 (or MH2) would be encapsulated by ITR1
and tunnelled (one way, IP-in-IP) to MH1's ETR, which is TTR1.
TTR1 decapsulates them and sends them on the VPN tunnel to
MH1.

Packets from MH1 to H2 pop out of the VPN tunnel at TTR1 and
use ordinary BGP forwarding to get to H2.

RH1 has an Ivip-mapped IP address.  It is not using
Ivip-mobile - just Ivip-basic.

In this diagram, the nearest ITR to H1 is TTR1.  When H1 sends
a packet to RH1, it is forwarded by the TR transit router to
TTR1, and encapsulated.  TTR1 sends the packet in a 1-way
IP-in-IP tunnel to ETR1 (at the border of, or within an edge
network), which forwards it within the edge network to RH1.

Packets from RH1 must be accepted by the border router of its
edge network (which could be ETR1 or some other router not
shown) and are sent via ordinary BGP forwarding to H1.

A packet from MH3 to MH1 goes via 2-way VPN tunnel to TTR2
where it is encapsulated and sent on a 1-way IP-in-IP tunnel
to MH1's ETR, which is TTR1.  This decapsulates it and sends
the packet along its 2-way VPN tunnel to MH1.  Packets going
from MH1 to MH2 follow exactly the opposite path, with a
1-way encapsulated IP-in-IP tunnel from TTR1 to TTR2.

Mobile-IPv6 doesn't assume anything about routers, except its
home router.  It doesn't assume anything about corresponding
hosts, since ordinary hosts can simply communicate with
the mobile host as if it was at its home location, with the
home router tunnelling to and from the mobile host, wherever
it is.

Ivip doesn't assume anything about corresponding hosts, or
the host whose IP address or subnet is mapped by Ivip.

Ivip does assume there is a system of ITRs - ideally all
around the Net at border routers and transit routers.

Ivip requires the receiving host have at least one ETR.  For
Ivip-basic, there will be one ETR in the edge network where
the receiving host is currently connected.  This mapping can
be switched to another ETR, for instance for multihoming.
There could also be load balancing between multiple ETRs, as
LISP proposes.

VK> Analogy:
VK> (1) Ivip routers are similar to the mobile network home agents,
VK>     except that the home agents are confined to home networks
VK>     where as Ivip routers may be distributed in the Internet.

Yes, but the TTRs are similar to the home-agent.  With
Ivip-mobile the mobile host's IP address doesn't have a home
anywhere.  Packets from ordinary IP addresses are tunnelled by
ITRs to the currently chosen ETR, and that ETR (in the case of
a mobile host) is a TTR which the mobile host has a tunnel to.

This means that wherever the mobile host is and wherever the
corresponding hosts are, assuming there are plenty of ITRs and
TTRs, the total path length will be the same as, or not much
longer, than without Ivip.

VK>     If we suppose that home agents can be distributed like Ivip
VK>     routers, then these two approaches are exactly the same.

In my perhaps faulty understanding of mobile-IPv4, mobile-IPv6
and NEMO, this could not occur, because the home-agent router
needs to be connected to an edge network whose border router
advertises the address range the router handles.  It can't
be moved somewhere else, assuming that router only handles a
part of the subnet its edge network advertises, because to do
so would create two or three BGP advertisements instead of
one.

VK> (2) Ivip's ETRs are similar to nemo mobile routers, which update
VK>     Ivip routers (home agent) with the tunnel-end addresses.

I read the first ten pages of NEMO's RFC3963.  NEMO extends
mobile-IPv6 with a "mobile router".  This is somewhat like a
mobile-IPv6 mobile node, except it is a router which has links
to the multiple mobile hosts.  Also, there is no "route
optimization" as in mobile-IPv6 where capable correspondent
nodes can send packets directly to and from whatever IP
address the mobile host (or NEMO mobile router) is currently
located at.  So with NEMO, all traffic to the multiple mobile
nodes goes through the fixed location (I assume) home agent
router, and then via a two-way tunnel to the mobile router
- which can flit from one IP address to another, taking its
flock of mobile nodes with it.

I don't think there is much resemblance between any of these
pre-existing mobile systems and either Ivip-basic or its
Ivip-mobile extension.

Ivip's notion of large numbers of public ITRs and TTRs (by
paid-for access I guess) with the mobile host choosing
which TTR to use at any time, does not seem to have a
parallel in these pre-existing mobile systems, which
assume a fixed home-agent router.

Ivip-mobile will only be possible because there will be
a global system of ITRs which tunnel packets to any
BGP-reachable IP address is currently selected to act as the
ETR for the mobile-host's Ivip-mapped IP address.

Without that global ITR system, steering packets anywhere on
the NET, the pre-existing technologies mobile-IPv4/6-NEMO
must rely on all packets going to a home-agent router, except
for mobile-IPv6 which can use "route-optimization" and set
up smart-enough correspondent nodes to send packets directly
to and from the mobile node.

LISP could just as easily be extended as I suggest with
Ivip-mobile.  Ivip is similar to or identical to LISP, or
at least some type of LISP, except that it should be easy to
incrementally deploy and except that it does require all its
"master-subnets" (hopefully few and large) to be advertised
in BGP.

VK> (3) In nemo, a bidirectional tunnel is set up to avoid packet
VK>     filtering at border routers. Whereas in Ivip, it is assumed
VK>     that the BRs forward packets with Ivip-mapped addresses.

In Ivip-basic, with the receiving host semi-permanently in an
edge network, I assume that the edge network's routers are
programmed to allow packets from its IP address to be sent to
the global BGP system.  If not, then the receiving host would
need to use Ivip-mobile with a TTR outside this edge network.

I tend to think of the TTR function being implemented within
routers which are already transit routers, or in routers
which are located at peering points and are primarily
connected to transit and border routers.  However a TTR could
be a border router or inside an edge network, as long as that
border router or edge network allowed egress of the outgoing
packets (which could have almost any source address) to the
BGP routing system.

VK> There are some well known problems associated with such anchor
VK> point-based approaches, such as increased packet delivery
VK> delay, non-optimal routing, tunnelling overhead, possibilities
VK> of single point of failure, etc. To avoid these problems,
VK> route optimization mechanisms are being investigated in the
VK> nemo WG. Shouldn't we look for the ID-loc split solutions that
VK> avoid such problems as much as possible?

Ivip-basic isn't at all like these pre-existing mobile
protocols.  The Ivip-mobile extension only loosely resembles
them, because there is no one fixed home agent.

Altogether, Ivip-basic and Ivip-mobile should enable packets
to flow without much extra distance or hops.  There is still
a tunnelling overhead, and perhaps problems with MTU limits.

Christian Vogt quoted Ved Kafle and responded:

VK> To me, the Ivip approach looks similar to the network mobility
VK> (nemo) support approach being discussed in the IETF nemo WG. ...
CV>
CV> Absolutely.  This is what I was saying.

I think Ivip is very different from NEMO etc.

VK> Analogy:
VK> (1) Ivip routers are similar to the mobile network home agents,
VK>     except that the home agents are confined to home networks
VK>     where as Ivip routers may be distributed in the Internet.
CV>
CV> Actually, Ivip routers are confined in the same way as home
CV> agents in that both can only support "home" prefixes that
CV> correspond to their topological location in the Internet.

As I explained above, and in a long message which started this
thread, this is not the case.  There are lots of TTRs to
choose from, so ideally the mobile node will choose one close
to it.  This is only possible because there is a global
network of ITRs, acting like a coordinated bunch of mirrors,
tunnelling all the mobile node's traffic to this TTR, another
TTR or wherever the mobile node (or some other control system)
tells the ITRs to tunnel the packets to.

CV> After all, the idea of Ivip routers is to have them advertise
CV> sub-prefixes from their ISP's global routing prefix so that
CV> the sub-prefixes can be aggregated with the global routing
CV> prefix and do not require a separate slot in the DFZ routing
CV> table.

An Ivip ITR is one of many which advertises one or more
"master prefixes".  Each ITR attracts packets addressed to
these prefixes and tunnels them off to potentially millions of
different ETRs.  A TTR acts like an ETR as far as the ITR
system is concerned.  Its just that a TTR is probably not in
an edge network - and it connects to the host which has the
Ivip-mapped IP address via a two-way tunnel.

VK> There are some well known problems associated with such anchor
VK> point-based approaches, such as increased packet delivery
VK> delay, non-optimal routing, tunnelling overhead, possibilities
VK> of single point of failure, etc. ...
CV>
CV> Right.  And things don't get better if we do the indirection
CV> for all hosts, not just mobile hosts or hosts moving with
CV> mobile networks.

Other than the central database, I don't think there is a
central point of failure for Ivip-basic or its Ivip-mobile
extension.  Assuming the mobile host has a BGP reachable
IP address, or is behind a NAT firewall which has such an
address, it can set up a two-way tunnel to one or more TTRs.
If one goes down, it can use another.  Ideally it would use
nearby ones.

Ivip-basic, like LISP, relies on the receiving host having a
connection to a particular ETR.  LISP, Ivip-basic and
Ivip-mobile all enable the receiving host to set up links
to multiple ETRs and then (directly, or by some automatic
system, since the receiving node could be out of contact
after the failure of its active link) the ITRs can all be
told to tunnel packets to whichever ETR is currently used.

This is genuine multihoming, including for Ivip-mobile - if
the two links to the two TTRs are by physically different
networks.

VK>                                    To avoid these problems,
VK> route optimization mechanisms are being investigated in the
VK> nemo WG. Shouldn't we look for the ID-loc split solutions that
VK> avoid such problems as much as possible?

CV> Without an intention to prefer either mechanism, I just want
CV> to note that the advantage of Ivip is a better transition path,
CV> whereas LISP performs better from the perspective of packet
CV> propagation latencies and fault tolerance.

Thanks for agreeing with what I think is the most important
difference - that Ivip looks like it would be easier to
introduce.

CV> As a matter of fact, LISP is nothing else but route
CV> optimization (in Mobile IPv6 terms) for Ivip.

LISP has the ITR function inside or at the border of the
edge network.  Ivip typically has the ITR function at the
border router or amongst the transit routers.

With LISP, if the ITR function is at the border router
or in an internal router which is on the shortest path
between the sending host and the border router, then the
total path likely to be the same as for a direct packet
(other than a potentially longer path in the destination
edge network if the ETR isn't exactly on the shortest
path to the receiving host).

With Ivip, the same is true if either of the following
typical conditions is true:

1 - The ITR function is at the border router of the
    sending edge network.

2 - The ITR function is in a transit router which is on
    the shortest path towards the destination edge
    network's border router.

So Ivip's total path length would only be longer than LISP's
if the ITR function was not in the sending edge network's
border router but in a transit router (or perhaps another
border router of another edge network) which is off the
shortest path to the destination network's border router.

If Ivip or some similar system is adopted, before long, any
ISP or AS-end-user network with any substance would want to
install the ITR function at its border routers and/or at
internal routers - to reduce the processing load on the
border routers.  They would want to do this so their outgoing
Ivip-mapped packets do not depend on anyone else's external
ITRs.

A fully hardware optimised ITR, with its big database and
need for constant updates from the central Ivip database,
could be quite expensive.  But it has to be provided either
by the edge network, or by some ISP or transit provider
the edge-network uses for its upstream links.

I am not sure how the economics would work best.  Maybe
for smaller ISPs and AS-end-user edge networks it would
be better to pay their upstream provider to handle the
ITR function.

Over time, as ITR-capable hardware costs reduced, more and
more smaller ISPs and AS-end-user networks would probably
find it better to run their own ITR rather than pay to use
their upstream provider's ITR.

I think only those smaller edge-networks would not run
ITRs in the long-run.  The most important thing is that Ivip
could start running with just a handful of ITRs somewhere in
the Net.  Ideally there would be multiple ITRs for each
country.

The only significant path-length increases with Ivip would
occur when the sending edge network is nowhere near an ITR.

This would occur in the early days of introduction.

But once Ivip became established and a significant amount
of traffic was reliant on ITRs, competition between
upstream providers would cause them to ensure there were
ITRs in upstream paths, rather than off to one side somewhere.

There are two other situations of practical interest, in
which Ivip would also produce no increase in total path
length.

Firstly, it is perfectly possible to install Ivip ITRs inside
edge networks.  Then, the situation would be identical to
LISP: assuming the ITR was on the path from the sending host
to the border router, there would be no extra path length.

Edge networks could install internal ITRs to take the load
off their border routers.  However, this would slightly add
to the traffic load inside the network, due to the IP-in-IP
overhead.

The second scenario is most likely to occur when the
destination edge networks are very close or their routers
are peers.

If the sending edge network has no ITR, the packets are
forwarded to the closest ITR, which is the border router of
the destination edge network.  This would work fine.  That
border router could encapsulate the packet and forward it
to the ETR inside the edge network.  There are a few
variations on this . . .

Firstly, maybe the border router is an ITR and the ETR is
a separate router inside the network - but the border router
has a route for the destination host's IP address already
programmed into it, as part of the whole destination edge
network being set up to send packets from the ETR to the
destination host.  Then, there would be no actual involvement
of Ivip, other than this border router was advertising the
whole "master-subnet" of which this destination IP address
was a part.  The fact that this border router received the
packet means that it forwards it towards the destination host
in its network.  The ETR wouldn't be involved and there would
be no encapsulation.

This would be the case if the border router's FIB applied the
local routing rules before the ITR function's rules - which
it should.

Secondly, if the border router's FIB applied the ITR
functions first, then it would tunnel the packet to the ETR
and all would be well.  (Unless the ETR was itself, in which
case it should just forward the packet on the internal
routing system.)

In either case, there would be no increase in path length -
above any detour inside the destination network to reach
the ETR.

If the sending host and receiving host are in the same edge
network, and if the whole edge network has its routers
programmed to handle the IP address of the receiving host,
then Ivip is not involved at all.

If the sending host was in some other part of the edge network
where the routers didn't know about how to route to the
receiving host, then the packet would be forwarded towards the
border router.  En-route, if it encountered an ITR, all would
be well.  If the edge network had no ITR, then the packet
would be forwarded out the border router and would find its
way to the nearest ITR.  Then it would be tunnelled back to
the same edge network's ETR.  This would definitely add to
the path length!

Here is a potential gotcha.

Lets say a host has two links to ETRs in two edge networks
A and B, and it is currently using the link to A.  This means
the global Ivip system's ITRs are tunnelling packets to A's
ETR.

Both edge networks must have their routing systems set up
to forward packets for the recipient host's Ivip-mapped
address (or subnet) to the link which leads to the receiving
host.  (For instance a router or a host computer with two
fibre links to two ISPs.)

Let's say there is a sending host in edge-network B.  That
sending host's packets will flow directly along the link from
B to the recipient host, even though the recipient host is
currently using the link to ISP for its connection to the Net.

For absolutely full connectivity, either:

1 - The receiving host needs to accept packets from both
    links at once.

or

2 - The network B needs to not forward the packets to the
    link to the recipient host until it somehow knows that
    the recipient host has switched to using network B's
    ETR.

Option 1 sounds best.

I apologise for the length of my messages.  I hope that
if you read them fully, you will understand Ivip and my
idea of what LISP can be used for in a way which is
independent of NEMO, mobile-IPv4/6 etc.   I think the
differences are more important than any similarities.

  - Robin

---

From: Markus Stenberg <[email-masked]>
Date: Mon, 18 Jun 2007 13:48:34 +0900

> single ASIC, the world's largest) - rather than on a larger number

I think that as an employee of a hardware company, selling big hardware is
a good; however, in practise, I think that it would be more reasonable to
provide support for both pull and push, especially if you plan to add churn
to the system. The churn numbers caused by more mobile (and/or TE-using?) 
users on centralized routers (and the communication channel between them
used for 'push' traffic) would be insane, and non-core router could not
handle that sort of non-scoped 'push' traffic anyway. The pure-push model
with per-host information granularity sounds likely to make the scalability
problem bigger, not smaller..

The typical arguments against pull solution are as follows:

- there is connection delay

- routers cannot store packets due to NGbps interface(s) it is saturating

 - => there is packet loss if the router in middle needs to do lookup

However, the connection delay is there already in significant number of
cases (DNS), and it isn't making the world a much worse place.

About the packet storage, or percentage of meaningful packets that need to
be stored during pull lookup - in typical case, 

- frequently more than one connection per destination, (HTTP, FTP, some P2P
  protocols),

- more than one source per destination, (some destinations being more equal
  than others), and

- traffic would most likely stop around first packet until lookup was
  completed anyway due to very few protocols blasting (meaningful) traffic
  without replies from the other end. (in case of HTTP, the client wouldn't
  even open more connections before the first request completes.)

I know for a fact that the ~1 packet caching pull has been used in
on-demand L2TP and VPN solutions. I'm curious about how the numbers would
work in core or large customer networks though. Anyone have any concrete
data? 

For anything smaller scale, it is clearly doable given modern hardware. 

Cheers,

-Markus

(*) I tried to look up some on the intarweb, but unfortunately all netflow
repositories I could find seemed to be of 'if you are researcher, snailmail
us with application in triplicate for processing' nature..

---

From: Robin Whittle <[email-masked]>
Date: Mon, 18 Jun 2007 16:25:11 +1000

Hi Markus,

You wrote:

> I think that it would be more reasonable to provide support
> for both pull and push, especially if you plan to add churn
> to the system. The churn numbers caused by more mobile
> (and/or TE-using?) users on centralized routers (and the
> communication channel between them used for 'push' traffic)
> would be insane, and non-core router could not handle that
> sort of non-scoped 'push' traffic anyway. The pure-push model
> with per-host information granularity sounds likely to make
> the scalability problem bigger, not smaller..

I agree there needs to be an option for smaller ITRs to operate
with less state and to use a "pull" query-based approach.

This would also enable a host to do its own ITR function.  If
this was implemented widely in operating systems, the system
(LISP or Ivip) would be highly scalable, since each host doesn't
have to store much state, unless perhaps it is a big server.

However, the system would still work fine for hosts without
built-in encapsulation.

Peer-to-peer file-sharing networks may send packets to lots of
hosts all over the Net.  This could be quite a burden on
LISP/Ivip compared to traditional two-way communication.

On one hand I am attracted by the idea of fewer really fast
ITRs, each with the whole mapping database in their FIB.  But
when I think about millions or hundreds of millions of separate
mappings, for individual IP addresses and for small and large
subnets, it becomes very daunting to receive these updates, and
to change the FIB accordingly, not to mention having an FIB
which could actually store them.

I think the FIB always needs to know what ranges of addresses
are being mapped via Ivip (the one or probably hundreds or
thousands of "master-subnets", each of which may be split into
different mappings).  Otherwise, whenever a packet arrives which
is not in one of these subnets and not matched by a BGP prefix,
the ITR has to query the central database.  The central database
would need to tell all the ITRs what subnets of the whole
address space are currently covered by the LISP/Ivip mapping
system.  I figure this would change only slowly.

When a "pull" ITR receives a packet whose destination is an IP
address within a "master-subnet" and the ITR doesn't have
up-to-date mapping information for that address, it queries the
central database (or a distributed database or whatever).  The
replies need to come with a caching time and a description of
the subnet they refer to.

For instance, if there is a subnet all mapped to the IP address
of one ETR and the ITR queries the database about a specific IP
address within this range, then the database needs to reply not
just with the IP address of the ETR, but with:

  This address to reach the ETR should be used for the range of
  IP addresses X to Y.
  Cache this result for 300 seconds.

If the database has no mapping for the IP address which the
query concerns, it needs to tell the ITR something like:

  There is no mapping for this IP address or for the surrounding
  range of addresses X to Y.
  Cache this result for 3600 seconds.

> I think that as an employee of a hardware company, selling big
> hardware is a good; . . .

I have some idea of the heroics of the CRS-1's 188-CPU ASIC and
its banks of reduced latency DRAM, running the Tree-Bitmap
algorithm:

http://www.firstpr.com.au/ip/sram-ip-forwarding/router-fib/#Cisco_CRS_1

The FIB of a busy ITR is a daunting prospect - and it must be
done with one or more CPUs and multiple fast RAM lookups using
something like Tree-Bitmap, because TCAM can't do it.  TCAM
can't scale to the number of Ivip/LISP mappings.  These will be
in the millions to hundreds of millions over time, and it has
some really nasty update gotchas when there is a need to move a
lot of rules to different locations in the TCAM.

I guess the CRS-1's SPP chip is highly flexible and would be as
good a basis for doing the fancy ITR FIB as any other piece of
hardware which currently exists.  Before deciding that LISP,
Ivip or whatever was the way forward, it would be best to have a
reliable estimate of how many mappings and packets per second
the CRS-1 MSC's Ingress FIB SPP system could be programmed to
handle.  I guess that new firmware would be required for the
best results.

At least the FIB, central CPU and communications heroics of a
LISP or Ivip ITR only need to be implemented on a fraction of
the routers in the Net.  Without something like this, we would
have to go to about this much trouble for all the DFZ routers
before too long.

As far as I know, the ETR function of decapsulating an IP-in-IP
packet is pretty easy.  Can routers with current firmware be
configured to do this?

Maybe the overall LISP/Ivip system would require some extra
functionality in the ETR, perhaps to confirm that the
destination host was reachable - if ping or some alternative
techniques were blocked for security reasons.

 - Robin        http://www.firstpr.com.au/ip/ivip/

---

From: "Ved Kafle" <[email-masked]>
Date: Mon, 18 Jun 2007 16:14:45 +0900

Robin:

Thanks for your detail clarification. I just add a few points here.

In my opion, the difference between the mobile IP and Ivip is based on
the following points:
(1) types of addresses used to identify end hosts; (home address vs. PI
addresses)
(2) where the mapping information is maintained in; (home agent vs. ITR,
ETR, TTR)
(3) where tunnels start and where they end (home agents and mobile host
vs. ITR and ETR or TTR)

Home agents are confined to home networks whereas ITRs can be
distributed in the Internet. Home agents advertise the reachability to
mobile host's home addresses whereas ITRs advertise reachability to
Ivip-mapped addresses. Ivip assumes that many ITRs may advertise the
same prefix. 

Mobile IP research is quite matured by now. From this experience, we can
explore if Mobile IP can be extended to cover the Ivip type distributed
mapping system by forming a federation of home agents and advertising
many network prefixes by each member of the federation. Ivips
distributed ETRs share some common features with the MAPs (mobility
anchor points) of hierarchical mobile IP. (How about pushing MAPs
further deep into the network, so that they function as TTR?)

Let's think if Ivip can get benefits from Mobile IP experience, or
Mobile IP can get benefits from the Ivip idea.

Ved

> -----Original Message-----
> From: Robin Whittle [mailto:[email-masked]] 
> Sent: Sunday, June 17, 2007 8:49 PM
> To: [email-masked]
> Cc: Christian Vogt; Ved Kafle
> Subject: Re: [RAM] Ivip (was ViP ...)
> 
> 
> Anycast ITRs in the DFZ & mobile tunnels", with "ViP" changed 
> to "Ivip" (eye-vip).
> 
> conversation and hand-drawn diagrams.  Trying to do it for a 
> bunch of people via email takes more words.
> 
> 
> VK> (nemo) support approach being discussed in the IETF nemo WG. In 
> VK> nemo, the mobile router's home agent advertises the 
> mobile network 
> VK> prefix such that the packets sent by a correspondent node to an 
> VK> address belonging to the nemo prefix go to the home 
> network of the 
> VK> mobile network, where the home agent tunnels them to the current 
> VK> location of the mobile router. The mobile router decapsulates and 
> VK> forwards the packets to the addressed node.
> 
> There are two aspects to the proposal at present: 
> "Ivip-basic" and "Ivip-mobile".
> 
> Ivip-basic
> ----------
> 
>   H1---(ITR1)---\
>       /          \
>   H2-/            \
>                   (ETR1)----RH
>                   /
>   H3---(ITR2)----/
> 
>        (ITR3)     (ETR2)
> 
> 
> packets, but it needs a routing rule specific for that IP 
> address or subnet which directs packets addressed to that 
> Assuming a single global Ivip system, the ITRs may be 
> provided by various operators, but work as a single system, 
> without any constraints on whose packets they will encapsulate.
> 
> also be unique is the idea that all these ITRs advertise the 
> same set of prefixes: whatever "master prefixes" the Ivip 
> system handles.
> 
> Generally, these "master prefixes" are split into many 
> smaller sections, each with a different mapping to an ETR.  
> This works fine down to individual IP addresses.  There needs 
> to be a single central system for controlling the mapping - 
> and every ITR has an up-to-date copy of this database.
> 
> called TTRs (Translating Tunnel Routers).  These TTRs are in 
> general either at border routers or more likely amongst the 
> transit routers.  TTRs could also be in edge networks.
> 
>   H1---(ITR1)---\
>       /          \
>   H2-/            \
>                   (TTR1)~~~~~~~~MH
>                   /            /
>   H3---(ITR2)----/            /
>                              /
>        (ITR3)     (TTR2)~~~~/
> 
> 
> The Ivip system maps the MH's IP address (or prefix) to one 
> of the TTRs, so all the encapsulated packets from the world's 
> could detect that one TTR is dead, or that its link to the MH 
> is dead, and then the mapping could be changed to the other 
> active TTR.
> 
> therefore also acts like the MH because this TTR is where the 
> MH's outgoing packets are first let loose on the Net.
> 
> Mobile-IPv6 can also cause corresponding hosts (H1 to H3 
> etc.) to send packets to and from the MH directly, rather 
> than using the home router.  This is more efficient, but 
> requires the corresponding host to have special mobile-IPv6 
> capabilities.
> 
> invoke a worldwide network of ITRs while mobile-IPv6/4 cannot 
> is that now we are desperate.
> 
> anyone, this means most edge networks must have one or more ITRs.
> 
> the border routers and transit routers.  I can get away with 
> this, as can the LISP authors, because we are desperate.
> 
> ITRs, each with its own "master-subnets" and its own central database.
> 
> router and transit router locations, it is not much more of a 
> step to have some of these, or perhaps separate networks of 
> routers, performing TTR functions.  Each TTR system could be 
> entirely independent of other TTR systems, and of the one or 
> more ITR systems (provided the TTR did not also implement ITR 
> functions).  The multiple TTR systems could compete and 
> provide commercial services to the customers who they provide 
> Ivip-mapped address space for.
> 
> system would not be allowed to fail, so any address space it 
> hands out via its mapping functions would be essentially 
> "Provider Independent".
> 
> "tenure" of this address - and not at all on any TTR systems it uses.
> 
> 
> "master-subnets".  If there was a single, global, system of 
> ITRs, it would be good if the privately run TTRs also acted 
> as ITRs within the global Ivip system.
> 
> tunnelled to the TTR when they are going to an Ivip-mobile host.
> 
> H1 and H2 have ordinary BGP routed IP addresses.  Packets 
> sent from H1 to MH1 are received by TTR1 which acts like an 
> ITR. Ordinarily, an ITR encapsulates the packet and tunnels 
> it to the ETR to which the destination (MH1) IP address is 
> mapped to.  In this case, TTR1 is the destination ETR, so 
> there is no need to encapsulate the packet.  It is simply 
> placed on the VPN 2-way tunnel ~~~~~~~~ and sent to MH1.
> 
> and tunnelled (one way, IP-in-IP) to MH1's ETR, which is 
> TTR1. TTR1 decapsulates them and sends them on the VPN tunnel to MH1.
> 
> In this diagram, the nearest ITR to H1 is TTR1.  When H1 
> sends a packet to RH1, it is forwarded by the TR transit 
> router to TTR1, and encapsulated.  TTR1 sends the packet in a 
> 1-way IP-in-IP tunnel to ETR1 (at the border of, or within an 
> edge network), which forwards it within the edge network to RH1.
> 
> hosts, since ordinary hosts can simply communicate with the 
> mobile host as if it was at its home location, with the home 
> router tunnelling to and from the mobile host, wherever it is.
> 
> anywhere.  Packets from ordinary IP addresses are tunnelled 
> by ITRs to the currently chosen ETR, and that ETR (in the 
> case of a mobile host) is a TTR which the mobile host has a tunnel to.
> 
> corresponding hosts are, assuming there are plenty of ITRs 
> and TTRs, the total path length will be the same as, or not 
> much longer, than without Ivip.
> 
> In my perhaps faulty understanding of mobile-IPv4, 
> mobile-IPv6 and NEMO, this could not occur, because the 
> home-agent router needs to be connected to an edge network 
> whose border router advertises the address range the router 
> handles.  It can't be moved somewhere else, assuming that 
> router only handles a part of the subnet its edge network 
> advertises, because to do so would create two or three BGP 
> advertisements instead of one.
> 
> 
> mobile-IPv6 mobile node, except it is a router which has 
> links to the multiple mobile hosts.  Also, there is no "route 
> paid-for access I guess) with the mobile host choosing which 
> TTR to use at any time, does not seem to have a parallel in 
> these pre-existing mobile systems, which assume a fixed 
> home-agent router.
> 
> for mobile-IPv6 which can use "route-optimization" and set up 
> smart-enough correspondent nodes to send packets directly to 
> and from the mobile node.
> 
> Ivip-mobile.  Ivip is similar to or identical to LISP, or at 
> least some type of LISP, except that it should be easy to 
> "master-subnets" (hopefully few and large) to be advertised in BGP.
> 
> 
> VK> point-based approaches, such as increased packet delivery delay, 
> VK> non-optimal routing, tunnelling overhead, possibilities of single 
> VK> point of failure, etc. To avoid these problems, route 
> optimization 
> VK> mechanisms are being investigated in the nemo WG. 
> Shouldn't we look 
> VK> for the ID-loc split solutions that avoid such problems 
> as much as 
> VK> possible?
> 
> CV> Actually, Ivip routers are confined in the same way as 
> home agents 
> CV> in that both can only support "home" prefixes that correspond to 
> CV> their topological location in the Internet.
> 
> As I explained above, and in a long message which started 
> this thread, this is not the case.  There are lots of TTRs to 
> TTR or wherever the mobile node (or some other control 
> system) tells the ITRs to tunnel the packets to.
> 
> 
> CV> sub-prefixes from their ISP's global routing prefix so that the 
> CV> sub-prefixes can be aggregated with the global routing 
> prefix and do 
> CV> not require a separate slot in the DFZ routing table.
> 
> these prefixes and tunnels them off to potentially millions 
> of different ETRs.  A TTR acts like an ETR as far as the ITR 
> VK> point-based approaches, such as increased packet delivery delay, 
> VK> non-optimal routing, tunnelling overhead, possibilities of single 
> VK> point of failure, etc. ...
> CV>
> CV> Right.  And things don't get better if we do the 
> indirection for all 
> CV> hosts, not just mobile hosts or hosts moving with mobile networks.
> 
> extension.  Assuming the mobile host has a BGP reachable IP 
> address, or is behind a NAT firewall which has such an 
> Ivip-mobile all enable the receiving host to set up links to 
> multiple ETRs and then (directly, or by some automatic 
> the two links to the two TTRs are by physically different networks.
> 
> 
> VK>                                    To avoid these problems, route 
> VK> optimization mechanisms are being investigated in the nemo WG. 
> VK> Shouldn't we look for the ID-loc split solutions that avoid such 
> VK> problems as much as possible?
> 
> 
> CV> Without an intention to prefer either mechanism, I just 
> want to note 
> CV> that the advantage of Ivip is a better transition path, 
> whereas LISP 
> CV> performs better from the perspective of packet 
> propagation latencies 
> CV> and fault tolerance.
> 
> difference - that Ivip looks like it would be easier to introduce.
> 
> 
> optimization (in 
> CV> Mobile IPv6 terms) for Ivip.
> 
> Ivip-mapped packets do not depend on anyone else's external ITRs.
> 
> the Net.  Ideally there would be multiple ITRs for each country.
> 
> to the traffic load inside the network, due to the IP-in-IP overhead.
> 
> destination edge networks are very close or their routers are peers.
> 
> border router could encapsulate the packet and forward it to 
> the ETR inside the edge network.  There are a few variations 
> on this . . .
> 
> local routing rules before the ITR function's rules - which it should.
> 
> above any detour inside the destination network to reach the ETR.
> 
> If the sending host was in some other part of the edge 
> network where the routers didn't know about how to route to 
> the receiving host, then the packet would be forwarded 
> towards the border router.  En-route, if it encountered an 
> ITR, all would be well.  If the edge network had no ITR, then 
> the packet would be forwarded out the border router and would 
> find its way to the nearest ITR.  Then it would be tunnelled 
> back to the same edge network's ETR.  This would definitely 
> add to the path length!
> 
> 
> the global Ivip system's ITRs are tunnelling packets to A's ETR.
>

---

From: Robin Whittle <[email-masked]>
Date: Mon, 18 Jun 2007 20:05:42 +1000

Hi Ved,

I agree with your summary of differences, except that with:

> (1) types of addresses used to identify end hosts; (home
>     address vs. PI addresses)

One way I am envisaging this is that the PI address space is
really assigned (in a typically large subnet) to some central
system which runs the Ivip system and that system splits it up
into smaller subnets and individual IP addresses, which then
become effectively PI for whoever the central system assigns
them to.

Alternatively, maybe an ISP or AS-end-user with PI space
dedicates the space "Y" to the Ivip system.  Then, it alone
controls how that is mapped.  Maybe it maps the whole subnet to
a single ETR, which gives it portability (which it already has)
or perhaps some other benefit.  This doesn't really reduce the
size of the BGP routing table, unless that subnet adjoins two
other subnets which are already mapped via Ivip.  Then, what was
previously three entries in the global BGP routing table:

X  >> BGP mapped to all the ITRs in the world.
Y  >> BGP mapped to some border router on an edge network.
Z  >> BGP mapped to all the ITRs in the world.

becomes a single entry:

X }
Y }>> BGP mapped to all the ITRs in the world.
Z }

This improves things for all the world's DFZ routers, and the
owner of this PI space can do what they like with their Y space,
changing the mapping, splitting it up etc. without causing any
fuss whatsoever for all the world's non-ITR BGP routers.

Most likely, the owner of the PI space will split up the mapping
as it likes down to single IP addresses, and then use them as it
wishes.  Some it may use for its own purposes.  Others it may
assign to customers.  In that case, the customer does not really
have eternally portable PI space, because they still depend on
the existence, management expertise, billing system etc. of the
ISP or AS-end-user who owns the "master-subnet" Y, or the Y part
of the larger "master-subnet" XYZ.  But as long as they maintain
good relations, the customer can use that IP address or subnet
via whichever edge network they like, because the customer
controls the mapping of their IP addresses or subnet.

Also, with:

> (2) where the mapping information is maintained in; (home
>     agent vs. ITR, ETR, TTR)

The mapping information for Ivip or LISP is contained in a
central, or perhaps distributed, database.  For each IP address
or subnet it is under the control of whoever the system has
assigned that IP address or subnet to.  In a mobile situation,
or a multihomed situation, there may need to be rapidly executed
changes of mapping, when one link fails for instance.  There
could be a number of systems which detect the changed situation
and use automatic, cryptographically secured, access to the
central database to change the mapping.

The central database information is either pushed to ITRs or the
ITRs have to pull the mapping, using a query and time-cached
response system.

Either way, the ITRs all over the world which are tunnelling
packets have the same mapping information for this particular IP
address or subnet.

The ETR is not involved in the mapping at all.  It just receives
any IP-in-IP packet sent to it, and pops off the outer header to
reveal the original packet.  (Maybe it does something with
hopcount.)

If the ETR is a TTR for a 2-way tunnel to the mobile host, this
TTR is still not directly involved in the mapping.  Perhaps this
TTR would have functions to control the central database mapping
for this mobile host's IP address or subnet.  However, I think
it would be better for some completely different, system to do
this.  Then, if this particular TTR dies, that separate system
can detect this and change the database's mapping (and
therefore, within a few seconds ideally, the behavior of all the
world's ITRs) to an alternative TTR with which the mobile device
also has established a 2-way tunnel.

Also:

> (3) where tunnels start and where they end (home agents and
>     mobile host vs. ITR and ETR or TTR)

being pernickety, I think someone reading this could think there
is a 2-way tunnel tunnel between an ITR and an ETR or TTR.

Mobile IP, as I understand it, apart from "route optimization"
involves a 2-way tunnel between the mobile host and the home
agent, or Mobile Anchor Point (MAP).  I guess this may be
encrypted and compressed, but not necessarily.

The same kind of 2-way tunnel exists between an Ivip-mobile
mobile host and its TTR.  The mobile host may also have tunnels
to other TTRs, but at any one time only one of them will be the
TTR which all the ITRs are tunnelling (1-way) packets to.

The Ivip mobile host must have its own IP address of some kind
and it must establish this tunnel to the TTR.  It somehow should
choose one or more "nearby" TTRs, where "nearby" could mean
totally different TTRs in geographically and topologically
different parts of the Net, if the two links the mobile host are
via different edge networks, such as one GSM-GPRS edge network
and the other ADSL via NAT.  In that case, the mobile host would
have two separate edge-network-provided IP addresses it uses for
tunnelling to its two TTRs.

The tunnelling from ITR to TTR is purely 1-directional, unless
the ITR also operates as the ETR in the reverse direction, which
would be likely if this ITR/ETR is a border router.

Assuming the correspondent host has an ordinary IP address,
packets from it to the mobile host enter a 1-way tunnel from the
ITR to the TTR.  There, they pop out of the tunnel and are put
into a separate 2-way tunnel to the mobile host.  There is
typically only one 2-way tunnel from the TTR to the mobile host.
 However, the mobile host can establish multiple tunnels via
different edge networks (or even the same one, I guess) to that
TTR, or more likely to other TTRs.

There could be hundred or even thousand of ITRs at any one point
in time tunnelling (1-way) packets to the currently active TTR
for this mobile host.  When the mapping in the database changes,
all the ITRs will switch their 1-way tunnels to the new TTR.
There will be a transition time, depending on how the ITRs do
their updates, so ideally the mobile host will receive packets
on all the tunnels it has to TTRs.

Outgoing packets from the mobile host to the correspondent hosts
will flow back along the 2-way tunnel to the TTR.  Then, if they
are addressed to ordinary (meaning direct BGP-reachable) IP
addresses, the TTR will let them be carried by the ordinary BGP
system to the correspondent hosts.  If the destination address
of a correspondent host is Ivip-mapped, the TTR may act as an
ITR and encapsulate it, then let it be carried by normal means
to the ETR which is currently the ETR for that correspondent
host.  (Except for when that ETR is the same router as this TTR.)

So the flow of packets from correspondent hosts, left to right,
to the mobile host MH1 is like this:

[CH1]-----(ITR1)-------------(TTR1)~~~~~~~~[MH1]
                             / | \
[CH2-ITR]--------(BR1)------/  |  \~~~~~~~~[MH2]
                  ETR         /
                             /
[CMH3]~~~~~~(TTR2)----------/

CH1's packets are raw to ITR1, 1-way tunnelled to ITR1 and then
2-way ~~~~ tunnelled to MH1.

CH2 has its own ITR encapsulation function, so there is a 1-way
tunnel from it to TTR1 where it joins the 2-way tunnel to MH1.

CHM3 has a 2-way tunnel to its TTR2, which sends on a 1-way
tunnel to TTR1 where it joins the 2-way tunnel to MH1.  Traffic
back from MH1 to CMH3 goes on a 2-way tunnel to TTR1 and then on
a 1-way tunnel in the opposite direction (so maybe it is a 2-way
IP-in-IP tunnel) from TTR1 to TTR2.  Then it goes on the 2-way
tunnel to CMH3.

Traffic between MH1 and MH2 is simply via two-way tunnels
through TTR1.

Traffic coming out of MH1 to CH1 or CH2 goes via a 2-way tunnel
to TTR1.

Let's say CH1 has an ordinary IP address.  TTR1 simply pops the
packets addressed to CH1 out and forwards them by the ordinary
BGP system.

In this example, CH2 has an Ivip-mapped address.  It has its
mapping set so all the world's ITRs' 1-way tunnel packets go to
the ETR which is implemented at the Border Router BR1 of its
edge network.

In this example, I assume TTR1 also acts as an ITR, so it sends
packets to CH2 via a 1-way IP-in-IP tunnel to BR1.  BR1's ETR
function pops off the IP-in-IP header and the local routing
system is configured to forward the packet to CH2.

This is a long-winded way of saying that I think your phrase:

  "where tunnels start and where they end"

doesn't capture the fact that these are one, two or three
different tunnels of different types in series, and some of them
are 1-way tunnels converging on a single TTR.  I figure you
understand this, but I just wanted to point out that there isn't
really a single tunnel, for instance, between CH1 and MH1.

> Mobile IP can get benefits from the Ivip idea.

I am sure Ivip or similar can benefit from people who work with
Mobile IP.  I only have a cursory understanding of Mobile IP.

With either LISP or Ivip, the ITRs are like a massive,
steerable, set of reflectors which can beam packets to whichever
 ETR/MAP/TTR the destination host specifies, or which some other
system decides should be used to maintain contact with the
destination host.  All that is needed there is an ETR function,
and the ITR/Ivip system doesn't really know or care whether that
is an ordinary ETR or a MAP/TTR.

Mobile IP people probably dreamed of such a thing in the past,
but didn't have the impetus to make it happen.  The crisis in
routing and addressing means that one or more global systems of
ITRs are probably going to be built.  The ITR system would
enable Mobile IP to be Vastly(tm) more mobile and efficient
because the mobile host can pick and choose dynamically between
whatever TTRs suit it best as it moves.

In my last message I wrote it would be good if host operating
systems could perform the ITR functions of looking up mapping
and encapsulating the host's outgoing packets.

That won't work for most home or office desktop computers or
internal servers, because they will be behind NAT.  The outgoing
packets will be to an ETR in the destination host's network and
the packets in the return direction will be from the IP address
of the destination host - so the NAT won't know how to rewrite
the packet and forward it to whichever of its local hosts
initiated the communication.

Generally, I think it would be friendly if operating systems had
an option to do these ITR functions, but first they need a
reliable way of determining whether they are behind a NAT.

I understand that ICE is the framework to do this within:

  http://tools.ietf.org/html/draft-ietf-mmusic-ice

There are three circumstances where I think a host could and
maybe should encapsulate its own outgoing packets:

1 - If it has an ordinary BGP-reachable IP address.

2 - If it has an Ivip-mapped address and is using Ivip-basic.

3?- If it has an Ivip-mapped address and is using a 2-way
    tunnel to a TTR (Ivip-mobile).

However mobile devices may have long link delays, so it could
take quite a while to get the mapping information, delaying the
application programs packets from being sent, adding to storage
requirements etc.  Maybe, for any host using a TTR, it would be
best to expect the TTR to act also as an ITR, since it probably
has more memory and CPU grunt and has a much faster connection
to the Net to make queries.  (Ideally, the TTR's ITR function
would have a full copy of the database.)

Maybe I am thinking of the term "mobile" too literally.
Generally speaking any host on a long-delay final link, such as
a GPRS or worse, GSM data call (~1 second ping times...),
especially a link which is congested or flaky, shouldn't be
busying itself with looking up mapping information and
encapsulating packets, which makes them longer.

Also, the NAT device's outgoing packets might well be
encapsulated by the NAT device - but only if that NAT device
could be sure it was the outside layer of NAT.

I don't think it matters whether the NAT device has its own
normal BGP IP address, or whether it has an Ivip-mapped address
- just as long as it is not behind a NAT itself.

I hope Ivip/LISP encapsulation in ITRs, in hosts or in NAT
egress ports themselves wouldn't upset STUN, TURN or ICE - all
of which are having enough trouble dealing with the unknown and
likely unpredictable behavior of one or more layers of suspected
NAT.

I guess that BEHAVE, RFC4787, which seeks to standardise NAT
behavior, also needs to be considered.

There are many scenarios to consider when trying ensure that a
major architectural change such as we are considering won't be
another can of worms like NAT.

---

From: "Joel M. Halpern" <[email-masked]>
Date: Mon, 18 Jun 2007 11:06:51 -0400

The note on the list comparing lvip with Mobile IP suggested a 
question in my mind.

In LISP. the TR (the device provides I and E functionality) is 
providing service directly to a specific set of customers.  Its 
capacity is tied to those customers, and it is operated either by 
those customers or by the ISP associated with those customers.  If we 
assume that there is value in having this, one can envision how the 
capability gets paid for.

If there is a rendezvous point somewhere off in the net, advertising 
a short prefix, and then disagregating that prefix into tunnels, that 
device is operated by someone.  But that someone is not the customer 
whose blocks are being advertised, nor is it the ISP serving those 
customers.  This leads to both authentication questions (how do you 
know this rendezvous box is allowed to issue that advertisement) and 
business model questions (how does it get paid for?

While conceptually decomposing the devices and separating the 
deployment is attractive, I fear that there are issues raised by such 
a separation that require attention.

Yours,
Joel

---

From: Robin Whittle <[email-masked]>
Date: Tue, 19 Jun 2007 15:42:37 +1000

This is another long "thinking-aloud" brainstorming message,
concerning where ITRs would be, who would run them and pay for
them, how Ivip might be introduced as a single system, whether
RIRs should allow ASNs to use their assigned space for Ivip
systems which are separate from the main one etc.

Hi Joel,

You wrote:

> The note on the list comparing lvip with Mobile IP suggested a question
> in my mind.
> 
> In LISP. the TR (the device provides I and E functionality) is providing
> service directly to a specific set of customers.  Its capacity is tied
> to those customers, and it is operated either by those customers or by
> the ISP associated with those customers.  If we assume that there is
> value in having this, one can envision how the capability gets paid for.

By "TR" I think you mean "Tunnel Router" - I guess a router
which also provides LISP ITR and ETR functions.  These functions
don't have to be on the one router.

These LISP ITR/ETR routers could be internal routers and/or
border routers.  Theoretically the whole edge network could be
served by the ITR/ETR functions being implemented solely a the
border routers.  For scaling purposes, it might be better to
have ITR functions in routers carrying packets towards the
border routers, and to do the decapsulation ETR work in those or
in other internal routers.

In all cases, these functions can only be used by hosts which
are part of, or which connect to the net through, that edge
network, so this generally solves the question of who pays for
it: the users of the edge network.

> If there is a rendezvous point somewhere off in the net, advertising a
> short prefix, and then disagregating that prefix into tunnels, that
> customers.  This leads to both authentication questions (how do you know
> this rendezvous box is allowed to issue that advertisement) and business
> model questions (how does it get paid for?
> 
> While conceptually decomposing the devices and separating the deployment
> is attractive, I fear that there are issues raised by such a separation
> that require attention.

Regarding authorisation, I will first try to correct my earlier
confusion about anycasting.  I am now sure that what I am
suggesting for Ivip ITRs really is "anycasting", although a
little different from the usual approach.

I understand that ordinary anycasting in the BGP system involves
multiple BGP routers all over the world advertising a particular
subnet, let's say 11.0.0.0/24.  Then, each such router sends the
packets it receives which are addressed to this subnet to local
hosts which have these IP addresses.  Those hosts accept each
packet and generate a response packet, which the BGP router
handles normally and forwards via the global BGP system to the
correspondent host which sent the initial packet.

This works fine for UDP single query-response systems, such as
DNS queries.  It is not normally used for TCP sessions, since
the correspondent host might at various times have its packets
forwarded to multiple such anycast routers.

Ivip uses the same anycast BGP advertisements.  So whatever
current authentication or policing arrangements work for BGP
anycast - I don't know if there are any - might work for Ivip's
ITRs.

The difference between DNS anycast and Ivip's anycast approach
is that packets received by ITRs all over the world are always
tunneled to the same ETR and so arrive at the one physical host
which has that IP address.  This means it will work fine for
TCP, as far as I know.  Maybe there could be flaky situations in
which the path taken by packets from the correspondent host
fluctuates to direct them to two or more ITRs, with one ITR
having a much longer path to the ETR.  Then, there could be some
out-of-order packets.  But the Internet doesn't guarantee
in-order packet delivery, and in most situations I guess that
the packets would flow to a single ITR in any one period of time
such as 0.2 to 2 seconds.

The business question of who will pay for public-access ITRs
outside edge networks only applies to Ivip's ITRs.  As with
LISP, ordinary (Ivip-basic) ETRs are only located at the border
routers or internal routers of edge networks and can only
successfully decapsulate packets and have them forwarded to the
hosts with Ivip-mapped IP addresses as long as the host is in
that network (and so pays the owner of the network), as long as
the edge network routing system knows how to route the packets
to the host, and as long as the assignee of that Ivip-mapped IP
address or subnet has configured the central database to map
packets to this ETR.

I will leave the question of TTRs which act as EARs for
Ivip-mobile out of this discussion, except to say that I
envisage there being multiple companies operating such systems
of TTRs, with mobile hosts needing to authenticate themselves
before being able to use their services, which the operators of
the mobile hosts, who are the assignees of the Ivip-mapped
addresses, would need to pay for.  ETRs probably don't need to
be coordinated in any way - but maybe there could be some
protocol for asking them to report on connectivity with the host
for a particular IP address.   At a minimum, a TTR must act as
an ETR and as an ordinary BGP router for egress of packets.  As
such, it needs one or a few normal IP addresses.  It doesn't
take up large amounts of address space, as the Ivip ITR system
does.  A TTR may well integrate an ITR function, as I mentioned
in my previous message.  I figure a single global Ivip system
would allow anyone to run an ITR function, provided they
complied with a set of rules, such as always correctly
tunnelling the packets it receives, using the most up-to-date
mapping information etc.

Before discussing the question of who pays to run ITRs, I want
to think more about terminology and situations with the end-user
whose IP addresses are mapped by the Ivip network.

I will try using the term "assignee of an Ivip-mapped address"
to refer to whoever has the effective authority to control the
mapping of a single IP address or a subnet (prefix) within the
Ivip system.

Here are some examples:

1 - One of the many "master-subnets" handled by the Ivip system
    is 11.11.0.0/16.  This was assigned by an RIR as PI address
    to company Z.

    Company Z is the assignee of this subnet (AKA prefix) in
    RIR terms.

    Company Z has registered with the central "Ivip Authority"
    - whoever controls the database which controls the ITRs -
    for simplicity I am assuming a single Ivip system.  There
    could be multiple systems, in which case company Z may have
    registered with the VVV Ivip network.

    In either case, company Z has told the Ivip system that all
    that system's ITRs should advertise 11.11.0.0/16 on all its
    with those ITRs tunnelling the packets however company Z
    chooses.

    Company Z has authenticated access to the control interface
    for the database, so one or more people or hosts under
    company Z's control are physically able to alter the
    contents of the database, and so have complete control
    over how the ITRs tunnel packets sent to 11.11.0.0/16.

    Company Z could have these 65,536 IP addresses mapped in
    various ways:

    a - The whole subnet could be tunneled to a single ETR.

    b - The subnet could be split and tunneled to multiple
        ETRs as somewhat smaller subnets.

    c - The subnet could be split, potentially into individual
        IP addresses, each to be tunneled to an independently
        chosen ITR.

    In all cases, company Z could do this for its own internal
    purposes and/or it could use this address space partially
    or wholly for its customers.

    If some or all of the subnet is assigned to customers, then
    presumably those customers are paying for this service.
    If the Ivip system involves fees, then Company Z needs to
    pay those fees and presumably collect fees from any
    customers.

    In order for this to be useful for the customers each
    customer needs some way of controlling which ETR their
    IP address(es) or subnet(s) are mapped to.

    This could be by company Z having its own user interface
    system which its customers use, via various interfaces
    including a web page for manual configuration, or some
    protocol interface for automated operation.  Company Z's
    system would convey all changes immediately to the central
    Ivip database.

    Company Z may also run a system which checks connectivity of
    its customers who want multihoming, and switches the
    mapping of their address(es) to another ETR if their
    host or router can't be reached via the ETR which
    packets are currently being tunnelled to.

    An alternative would be that company Z tells the
    "Ivip-authority" that until further notice, its customer AA
    should be authorised to directly control the mapping of one
    or more IP addresses, subnets etc. within 11.11.0.0/16.
    This enables customer AA to have faster, more direct,
    control of the mapping of the IP addresses they have been
    "assigned" by company Z.

    In this case customer AA would be the "assignee".

    Customer AA probably doesn't have an ASN or know what BGP
    is.  Customer AA has no relationship with an RIR.  It
    depends entirely on the stability and continued good
    relations with company Z for its continued use of this
    IP address space.

    If company Z has physical control of the mapping of
    customer's AA addresses, but is doing so entirely as
    customer AA desires, then I would still call customer AA
    the "assignee".  This is all sounding rather legalistic...

2 - The Ivip system itself may have been assigned PI space by
    an RIR.  Then, it can assign control (presumably for some
    kind of fee) to end-users for any one, two or up to millions
    of IP addresses.  These do not necessarily need to be
    ordinary prefixes or on binary boundaries.  There is no
    concept of route-aggregation in this Ivip-mapped space.

    The Ivip system could advertise hundreds or even thousands
    of "master-prefixes".  Some could be directly assigned by
    RIRs or others could be assigned by AS-end-users or ISPs
    which had this space assigned to them by RIRs.

    If the Ivip-authority assigns 33.33.27.64 - 33.33.29.16 to
    company N, company N is the "assignee" of these addresses.
    If company N assigns 33.33.28.80 to 33.33.28.87 to company
    M, then company M is the assignee of these 8 addresses.

So there are various ways by which the end-user assignee could
have gained their addresses.  They may be assigned directly from
an RIR and then the end-user authorises the Ivip system to
advertise and map them.  The Ivip system may be assigned PI
space and then rents it out, assigns it or whatever to other
companies or end-users.  An end-user may get their space via a
chain of RIR > Ivip-authority > wholesaler 1 > retailer 2 >
final end-user.

Let's say the RIRs collectively create, or act as a single
Ivip-authority.  While there may also be commercial or
independent Ivip systems, those systems are only going to be of
value if they have lots of well-located ITRs.  For the moment,
let's assume that the single global Ivip-authority system is so
successful that no-one tries to set up their own.

I will try to imagine a stable state of widespread Ivip usage,
then try to imagine how a single system could develop into that
state incrementally.

Let's say the Ivip-authority had enough funding to run a
centralised database server which was capable of sending updates
to ITRs, either by push or by pull methods, and a basic user
interface of manual web-based configuration and direct protocol
interfaces for automated systems to control the mapping of one
or more IP addresses or subnets.   I know virtually nothing
about the finances and business arrangements of RIRs, domain
name authorities, commercial domain name registers and the root,
TLD and major second-level name servers.  I figure the whole
system is paid for largely or wholly by the fees we pay to
register domain names.

Assuming a single Ivip system without any competitors, I guess
there would need to be some similar commercial reseller system
for IP addresses, similar in some ways to that for domain name
registrations, and probably involving the same companies.
However, I hope that the demand for Ivip-mapped addresses would
be for a smaller subset than the mass-market, whole of business
and organisational and personal-domain-name humanity, which the
domain-name business serves.

Hopefully, most people wouldn't need to worry about the nature
of the IP addresses they use for home and small office access.

Hopefully, Ivip-mapped addresses will be managed by end-users
(or their network managers or ISPs) with genuine needs for
portability, multihoming and TE etc.  Also, these are just
numbers, not names, so hopefully the whole thing would be more
technical, less marketing driven etc. than the domain-name business.

I feel sure there will soon emerge some kind of market-based
system for deciding who has IP addresses, because when fresh
supplies run out, the RIRs won't want to handle the endless
disputes and requests about who should have space, when some
ISPs or AS-end-users have large slabs of space which they appear
not to be using efficiently, or perhaps at all.  With a market,
that space would either be returned to the RIRs or sold or
rented out or whatever.

There may well be a commercial market for addresses within the
growing proportion of addresses which are mapped via Ivip or
whatever.  This may be quite separate for whatever commercial
arrangements develop for raw directly BGP-routed space, in 256
IP address increments, as currently assigned by RIRs to ISPs and
AS-end-users.

Competing systems of ITRs, each operated by a company and
renting out the mapping of its IP addresses, could only persist
as long as at least one RIR considered this to be a legitimate
use of the space it had assigned.  Normally, I understand, RIRs
assign space according to projected need of an ISP or
AS-end-users - need for space in their own physical networks.

A company or organisation running an Ivip system is not running
a physical network.  They would be engaging in the currently
unprecedented practice (I think) of having a bunch of anycast
routers all over the Net, for the purpose of tunnelling packets
one-way to one or probably numerous edge networks, usually not
their own.

So I think it is probably a matter of IANA/RIR policy whether
there will ever be multiple Ivip systems.

I imagine this stable, very widely used Ivip-authority single
Ivip system of ITRs being made up of ITRs and various other
things which are not operated by the Ivip-authority itself.

The Ivip-authority would be running something like the first
levels of the DNS.  There would be millions or billions of IP
addresses under its control.  Many of them would have stable
mapping which would only change every few years.  Some might
have mappings which change every few minutes, as a mobile device
takes its packets through different TTRs.

Maybe the Ivip-authority charges for its services partly by:

1 - Total number of IP addresses.

2 - Total number of changes, where a single change can apply to
    a large number of IP addresses, according to whatever
    physical protocol is used to send the changes to the ITRs.
    So changing the mapping of 32 randomly distributed IP
    addresses would be charged as 32 changes, but only as
    one change if it was a contiguous set of addresses which
    could be specified in a single change message.

3 - Also, perhaps, by the number of queries it receives for
    these addresses via any "pull" system it operates - but
    this is open to abuse by hackers who want to bankrupt
    an assignee of an Ivip-mapped address.

There are some significant scaling problems and questions of
commercial cost in any such system.

Let's say that most edge-networks perform their own ITR
functions.  They do this to remain competitive, since the
alternative is relying on ITRs out in the DFZ.  These may
involve longer paths or dropped packets due to being overloaded.

This would be ironic:

   LISP ITR's must be in internal routers or border routers.

   LISP is never attempted because no-one wants to be the first
   to use LISP-mapped addresses while most edge-networks have
   no ITRs.

   Ivip is launched instead, with an initial seeding of ITRs in
   the DFZ.

   After a significant amount, say 20% of traffic, flows to
   Ivip-mapped addresses, pretty much every edge network has
   its own ITRs.

So Ivip winds up resembling LISP, in many respects!

Ivip's only two real differences from LISP are:

1 - Mapped addresses are advertised in BGP.

2 - ITRs can be in the DFZ, using anycast.

(Someone would have thought of these sooner or later.  My idea
of using Ivip ITRs to beam packets to freely chosen TTRs for
Ivip-mobile is equally applicable to LISP and would have been
seized upon by anyone thinking about LISP and mobile IP at the
same time.  My idea of how LISP or Ivip should be used may
differ from how others think about it, and I have written more
expansively/ramblingly and given some diagrams, maybe more
concrete examples etc., without a large number of variants
numbered to two decimal places.)

With ubiquitous adoption, there wouldn't be a lot of difference
between LISP and Ivip.  But the fact (I think) that Ivip could
be adopted, and that it could continue to work with a handful of
non-upgraded edge networks, is a vital distinction.

Very roughly, here is how a single global Ivip system might be
developed.  Please excuse my probable ignorance of how the
various organisations work together.

1 - IETF develops protocols, standards etc.

2 - IANA and RIRs decide that a single Ivip system needs to be
    established, and that anyone using address space for their
    own Ivip system is not using it in accordance with
    prevailing policies.

3 - ICANN, IANA, RIRs and maybe peak organisations of domain
    name resellers set up some new authority to run the
    database.

4 - A handful of transit providers promise to run ITRs around
    the world to get the system going and to support, in the
    long term, edge networks which don't provide their own
    ITRs, although that service would probably be slow.

    (Maybe there is a peak body of transit providers and major
    telcos/ISPs who could commit their members to providing some
    level of ITR support.  This would cost money, but by this
    time, Ivip or whatever would be seen as the best or only
    hope of containing the growth in the BGP routing table
    which is threatening to make all their routers obsolete.)

5 - In the light of the Ivip alternative, maybe the RIRs
    develop some policies about the addresses they assign
    for normal BGP use, how assignees are expected to use
    that space without excessive numbers of changes etc.

    Also, there may need to be some new policies about RIRs
    assigning large slabs of address space to the Ivip system.
    The more space, and the fewer the divisions between its
    contiguous divisions, the fewer BGP advertisements will be
    needed.

The ISPs and AS-end-users could install ETRs and organise their
internal routing systems to cope with the various Ivip-mapped IP
addresses or subnets in their edge networks.  This shouldn't
cost much, and only needs to be done by edge networks where
customers or internal users will want to locate hosts with
Ivip-mapped IP addresses.

ITR capabilities in routers could be developed.  As a low-cost
alternative, it might be possible to develop open-source
software for a dedicated ITR box, such as a 64 bit Linux box
with a 1G or 10G interface.  This can do nothing but be a BGP
advertiser for the Ivip "master-prefixes" and send the
encapsulated packets back on the same Ethernet cable.  This
"stub-ITR" could be built for a few thousand dollars and plug
into an interface of any router, to make that router effectively
an ITR.  Load balancing could be done by having multiple boxes,
each advertising a subset of the Ivip system's "master-prefixes".

If existing routers couldn't do the ETR function, there could be
a similar Linux box with open-source software for that too.
This would greatly ease the adoption costs for many ISPs and
edge networks.

After a few years, router vendors would implement ITR and ETR
functions as standard in routers - so the ITR function would
migrate into real routers as part of the standard upgrade cycle.

Ivip's ETR functions (not counting TTRs, which may be outside or
inside edge networks) must be implemented either at border
routers and/or internal routers.  In either case, as with LISP,
it is clear that the users of those ETRs will pay their ISP in
some way for traffic which flows through those ETRs.  I think
the ETR cost could be low, since they don't require a database,
don't require authenticated access or maybe any management at
all.  It may be as simple as having a bunch of routers in the
network, including the border routers, all providing ETR
functions, and giving advice to users inside the edge network
about which ETR they should best map their Ivip-mapped IP
address or subnet to.

If Ivip became ubiquitous, then as for LISP, I think most of the
edge networks would implement their own ITR functions, at
internal and/or border routers.  This would provide their users
with the best performance, because the ISP could guarantee that
all packets generated in the edge network for Ivip-mapped IP
addresses would be locally encapsulated, removing any dependence
on external ITRs.

Lets say that Ivip ITRs are widely deployed in most edge
networks and many users depend on Ivip-mapped IP addresses.

Any hosting company with a proportion of their servers using
Ivip-mapped addresses (I am not sure that this is a justifiable
use of Ivip - there's nothing wrong with web servers using PA
addresses from whatever hosting company they are sitting is and
having the customer's nameserver point to that address) will
want to ensure those servers are reachable via any edge network,
including those with no internal ITRs.

The hosting company could do this by making their border routers
act as ITRs.  They could also plant an ITR just outside their
network.

The ITR, in both cases, I assume would have to advertise the
complete set of "master-subnets" which the single global Ivip
scheme advertises.  (If there were multiple such schemes, the
hosting company's ITR would need to implement ITR functions for
the "master-subnets" of every such scheme for which some edge
networks did not themselves implement ITRs.)

An ITR should, I think, not just accept all incoming packets for
the "master-subnets" it advertises, but properly handle them by
tunneling them to whichever IP address of an ETR which they are
currently mapped to.

It would be unfriendly and disruptive to have an ETR advertising
the whole set of "master-prefixes" but dropping all packets
which were not to be tunneled to the hosting company's ETRs.  I
wonder how disruptive this would actually be, however.  The
hosting company could say: "If these packets found their way to
our border router, or our nearby outrigger ITR, then firstly
they came from edge networks which were not equipped with ITRs
and secondly those edge networks were deficient in these nearly
ubiquitous Ivip times by not ensuring there was an upstream ITR."

I am trying to find a long-term reason why an ISP, hosting
company, transit provider or AS-end-user would want to run an
ITR in the DFZ in the long term.

Someone needs to do this, but to provide a "second-best" service
so as to encourage edge networks to install their own ITRs.

Probably the transit providers are the best placed to run these
ITRs.  But why should they do so?

Initially, they might volunteer to do so in order to get Ivip
off the ground, since the rising tide of the BGP routing table
will be making them fearful.

Initially, the Ivip-authority could run a few ITRs to get the
system going.

A commercially run Ivip system would either run its own ITRs
and/or pay others, such as transit providers, and ISPs, to run
the ITR functions on their own routers.   If there were multiple
such commercial systems operating with compatible ITR <->
database pull or push protocols, then ISPs might have a single
router which runs the ITR functions of multiple commercial Ivip
systems on, charging each such system a fee for the traffic volume.

The commercial Ivip systems would generate revenue by charging
their customers according to IP addresses assigned, the number
of changes made to the mapping, and the volume of packets forwarded.

I am not sure if independent Ivip systems are a good idea.  My
sense is that they are not, as long as a single centrally run
Ivip system is well run.

Transition to a major new architecture requires a lot of thought.

---

From: Robin Whittle <[email-masked]>
Date: Fri, 22 Jun 2007 02:52:33 +1000

Here is some more discussion of the similarities and differences
between Ivip-mobile and various types of Mobile-IP.

This is from an off-list exchange with a list member who does
not at present have time to be fully involved with the
discussion, and who gave me permission to post this to the list.
 He provides links to I-Ds concerning new developments in Mobile-IP.

At the end, I also speculate that something like LISP/Ivip will
either be required in order for IPv6 to be widely adopted (if no
PI space is made available for AS-end-users) or if it does
become widely adopted (after PI space is made available to
AS-end-users).  In either case, I speculate that LISP/Ivip or
similar with its global network of ITRs beaming packets to
whichever ETR or TTR (or some other term for the tunnel router
which has a 2-way tunnel to mobile devices) would make mobile IP
very much easier and probably make SHIM6 less important.

RW> Ivip's notion of large numbers of public ITRs and TTRs (by
RW> paid-for access I guess) with the mobile host choosing
RW> which TTR to use at any time, does not seem to have a
RW> parallel in these pre-existing mobile systems, which
RW> assume a fixed home-agent router.
>
> <snip>
>
> That's not 100% correct: The Mobile IPv6 bootstrapping
> extensions (draft-ietf-mip6-bootstrapping-split-05.txt,
> draft-ietf-mip6-bootstrapping-integrated-dhc-04.txt) allow the
> mobile node to choose and change the home agent and obtain a
> corresponding home address dynamically. The home agent could
> be located in the mobile node's current access network (i.e.,
> a local home agent) or anywhere else (assuming the necessary
> trust relationships between Mobility Service Authorizer and
> Mobility Service Provider are in place; please see above draft
> for details).

Mobile IPv6 bootstrapping in split scenario
http://tools.ietf.org/html/draft-ietf-mip6-bootstrapping-split

  A Mobile IPv6 node requires a Home Agent address, a home
  address, and IPsec security associations with its Home Agent
  before it can start utilizing Mobile IPv6 service.  RFC 3775
  requires that some or all of these are statically configured.
  This document defines how a Mobile IPv6 node can bootstrap
  this information from non-topological information and security
  credentials preconfigured on the Mobile Node.  The solution
  defined in this document solves the Mobile IPv6 bootstrapping
  problem (RFC4640) when the Mobile Node's mobility service is
  authorized by a different service provider than basic network
  access, and is therefore generically applicable to any
  bootstrapping case.

MIP6-bootstrapping for the Integrated Scenario
http://tools.ietf.org/html/draft-ietf-mip6-bootstrapping-integrated-dhc

  Mobile IPv6 bootstrapping can be categorized into two primary
  scenarios, the split scenario and the integrated scenario.  In
  the split scenario, the mobile node's mobility service is
  authorized by a different service authorizer than the network
  access authorizer.  In the the integrated scenario, the mobile
  node's mobility service is authorized by the same service
  authorizer as the network access service authorizer.  This
  document defines a method for home agent information discovery
  for the integrated scenario.

> Furthermore, a mobile node may have multiple home addresses
> and may be registered with multiple home agents.
> draft-weniger-mobopts-mip6-cnlocpriv-02 describes a
> MIPv6-based scenario, where the mobile node bootstraps with
> home agents close to correspondent nodes. Such home agent
> would look even more similar to an ETR.

MIPv6 Correspondent Node-Targeted Location Privacy and Optimized
Routing
http://tools.ietf.org/html/draft-weniger-mobopts-mip6-cnlocpriv

  This draft discusses the problem of correspondent
  node-targeted location privacy in Mobile IPv6 and proposes a
  mechanism to achieve simultaneous optimized routing and strong
  correspondent node-targeted location privacy.  The mechanism
  utilizes the MIPv6 bootstrapping mechanisms and does neither
  require any new network entities nor changes to home agent or
  correspondent node implementations.

> Finally, a proxy entity in the network may send registration
> and act as tunnel endpoint on behalf of the mobile node. This
> is specified in the Proxy Mobile IPv6 draft
> (draft-ietf-netlmm-proxymip6-01.txt) and the network entity is
> called Mobile Access Gateway (MAG). The MAG would be similar
> to an ITR.
>
> With Proxy Mobile IPv6, the Mobile Node can be an unmodified
> IPv4 or IPv6 host (i.e., the host has no "second" address,
> since the care-of address is managed by the proxy entity).

Proxy Mobile IPv6
http://tools.ietf.org/html/draft-ietf-netlmm-proxymip6

  Host based IPv6 mobility is specified in Mobile IPv6 base
  specification [RFC3775].  In that model, the mobile node is
  responsible for doing the signalling to its home agent to
  enable session continuity as it moves between subnets.  The
  design principle in the case of host-based mobility relies on
  the mobile node being in control of the mobility management.
  Network based mobility allows IP session continuity for a
  mobile node without its involvement in mobility management.
  This specification describes a protocol solution for network
  based mobility management that relies on Mobile IPv6
  signalling and reuse of home agent functionality.  A proxy
  mobility agent in the network which manages the mobility for a
  mobile node is the reason for referring to this protocol as
  Proxy Mobile IPv6.

  It is possible to support mobility for IPv6 nodes by extending
  Mobile IPv6 [RFC-3775] signaling and reusing the home agent
  via a proxy mobility agent in the network.  This approach to
  supporting mobility does not require the mobile node to be
  involved in the signalling required for mobility management.
  The proxy mobility agent in the network performs the signalling
  and does the mobility management on behalf of the mobile node.

This is a fresh I-D, April 2007, and "Proxy Mobile IPv6" is
still a fairly novel term, with Google only finding 63 pages
mentioning it.  Previous Proxy Mobile IPv6 I-Ds are:

http://tools.ietf.org/html/draft-sgundave-mip6-proxymip6
http://tools.ietf.org/html/draft-devarapalli-netlmm-pmipv6-mipv6

Diag. 1

> So a possible MIPv6-based scenario could be the following:
>
>  MN1---(MAG1)---\
>       /          \
>  MN2-/            \
>                   (HA1)----CN
>                   /
>  MN3-------------/
>
>        (MAG2)     (HA2)
>
> MN3 would be Mobile IPv6 mobile node and MN1,MN2 would be
> Proxy Mobile IPv6 nodes. Tunnelling would happen between MAG1
> and HA1 and between MN3 and HA1. This looks quite similar to
> your ivip, no?

There are some similarities.  I will try to point out the
differences too.

Ivip-basic is just LISP (or rather some simple aspects of LISP,
without multiple encapsulation or messages going along with
packets regarding LISP-mapping of the source address) with the
ITRs automatically catching the raw packets from the
"correspondent nodes" (to use mobile IP terminology for an
ordinary host which is not part of the new system), no matter
where they originate from, because the Ivip-mapped address space
is part of one of many "master-subnets" which are advertised in
BGP.

With Ivip, if the raw packet, which is addressed to the
Ivip-mapped address, is not caught and tunnelled by the ITR
function of an internal or border router, it will soon be
forwarded to ITR outside the edge network.  There is a 1-way
tunnel from that ITR, and from many other ITRs handling packets
from other correspondent nodes, with all these tunnels
converging on an ETR, which terminates the tunnel and forwards
the raw packet to the destination host.

In general I think the similarities between this and mobile IP
are not very significant, however - I have only a cursory
understanding of Mobile-IP, and what you wrote in your second
email, which I have included in the quotes above, regarding
proxy Mobile IPv6 is something I had never heard of, which does
have some connection.

Firstly, the system doesn't require any special software on the
Correspondent Nodes (CN) or the host whose address is Ivip-mapped.

I understand no special software is required in Mobile-IPv6 for:

1 - A CN using ordinary Mobile-IPv6 communication to the home
    address of the Mobile Node (MN), where the home-agent
    router uses Tunnel Mode.  (Special software in the CN is
    required to use the more efficient alternative - route
    optimisation: direct communication with the MN at its
    care-of address.

2 - A MN using Proxy Mobile-IPv6.  I haven't read how it is
    done, but I understand that the MN has only its own IP
    address and it (and probably other MNs) are in a
    direct network connection with the proxy somehow does all
    the mobile-specific work for the one or more MNs it serves.

Secondly, LISP/Ivip is intended to be applicable for both IPv4
and IPv6 - although I have only been discussing IPv4 use of Ivip
so far.

Thirdly, except for Proxy Mobile IPv6, all mobile-IP systems
involve the mobile host having one or more physical IP addresses
(each is a "care-of" address) where it connects to the Net, and
having one stable IP address by which correspondent nodes send
it packets.  The simplest arrangement is Mobile-IPv6 in tunnel
mode where the mobile host always responds as if it has the
address it has on its home network, perhaps on an office LAN,
for instance.  This means the mobile host always has some fancy
software which implements all this, sending and receiving
packets to and from a tunnel, working from one or more physical
"care-of" IP addresses (each from a different wireless network
etc.) but allowing some or all of the operating system and
applications to work as if they had the mobile-host's home IP
address.

Ivip-basic and LISP do not involve the receiving host having any
IP address but its own.  I haven't stated this clearly before.

(By "receiving host" I mean the host whose IP address is
portable between ISPs, effectively mobile and able to be
multihomed by selecting different ETRs via links which use
different edge networks.  This "receiving host" can use this
same IP address when it is located in any ISP who supports it
with an ETR and correct internal routing configuration,
including letting the mobile host's packets go out to the Net,
as long as the ITRs all tunnel packets addresses to this IP
address to the correct ETR.)

In LISP/Ivip, the receiving host has no tunnel establishment or
end-point features.

In Ivip and I think in general LISP there is no 2-way tunnel.
Only 1-way tunnelling from potentially multiple ITRs to the one
ETR at any one time.  (I find the LISP I-D to cover so many
potential configurations that it is hard to understand clearly.)

I think this makes LISP or Ivip-basic very different from
Mobile-IP, except in some ways Proxy Mobile IPv6.

A MN with Proxy Mobile IPv6 is something like a "receiving host"
(sorry about this interchangeable "node" / "host" stuff) whose
IP address has been mapped by Ivip and is in an edge network,
perhaps with other receiving hosts and there is a single router
which performs its ETR and default gateway functions.

In both cases the MN / receiving host has only one IP address
and has no special software.  In both cases, the overall system
of Proxy Mobile IPv6 - or Ivip-basic and the local network's
ability to handle outgoing packets from the ETR/default gateway
- enable the MN / receiving host to be anywhere, but still
retain its one IP address.

With Proxy Mobile IPv6 router is called a "Mobile Access
Gateway" and I would redraw your diagram like this:

Diag. 2

  MN1---(MAG1)====
       /          =
  MN2-/            =
                   (HA1)----CN
                   =
  MN3==============

        (MAG2)     (HA2)

> MN3 would be Mobile IPv6 mobile node and MN1,MN2 would be
> Proxy Mobile IPv6 nodes. Tunnelling would happen between MAG1
> and HA1 and between MN3 and HA1. This looks quite similar to
> your ivip, no?

Where ----- means ordinary two way raw packets and ==== means
a 2-way tunnel between the Home Agent router and either the
Mobile Node (MN3) or the (MAG1) which I understand behaves like
one or more Mobile Nodes as far as the Home Agent is concerned.

I think it is challenging to understand all the variations of
mobile IP.  I find SHIM6 really hard to understand, though I
have not tried reading the whole of what I think is the most
up-to-date documents:

  RFC4177 and draft-ietf-shim6-proto.

I would like to fully understand all this.

Ivip-mobile is an extension to Ivip-basic.  The same principles
could be used with LISP (2.00+) With both LISP and Ivip-basic,
there is a large number of ITRs which 1-way tunnel packets
addressed to a particular IP address or subnet, wherever the
central system directs them to.  The tunnelled packets for any
given LISP/Ivip-mapped IP address are are all directed to a
particular BGP-mapped IP address, which is the ETR.

Ivip-mobile is simply:

1 - A mobile host (or some other system looking after its
    interests) being able to tell the database to have the
    ITRs tunnel the packets to any ETR, where the ETR is
    actually a TTR which has a two-way tunnel to the
    mobile host, previously established by the mobile host.

    As with Mobile IP's Home Agent, the TTR is the exit
    point for the mobile host's outgoing packets.  (The
    TTR may also perform an ITR function on those outgoing
    packets if they are addressed to an Ivip-mapped address,
    or simply tunnel the packets to a second mobile host it
    has a tunnel to, if the destination of the packets is
    the second mobile host's Ivip-mapped IP address.)

2 - The ability of the mobile host to build tunnels to
    multiple TTRs and by some means have the ITRs direct
    packets to one or another of the TTRs.

In your diagram:

>  MN1---(MAG1)---\
>  MN2-/            \
>                   (HA1)----CN
>  MN3-------------/
>        (MAG2)     (HA2)

the hosts/nodes whose IP address is handled by the mobility
system are on the left, where in my diagrams in previous
messages they were on the right.

Here is your diagram rewritten to match Ivip-basic:

Diag. 3

                             /---CN1
                            /
 RN1---(ETR1)~~(TR)~~~~~(ITR1)---CN2
      -          ~
 RN2--            ~
                   (ITR2)--------CN3
                   ~
 RN3--(ETR2)~~~~~~~

      (ETR3)           (ITR3)
       ....             ....
      (ETRn)           (ITRn)

      TR is an ordinary transit router.

      ITR1 might also be a border or transit router.

      ITR2 is also transit router.

----  Raw packets flowing right-to-left.

~~~~  1-way tunnelled (IP-in-IP) packets flowing
      generally right-to-left, except for the
      tunnel (ETR1)>(TR)>(ITR2)>(ETR2) which involves
      a little left-to-right in this diagram.  In
      this case, ITR2 is acting like an ordinary
      transit router for those encapsulated packets.

      The ETR functions are performed in internal or
      border routers.  Many other routers are not
      shown.

      If we add another ---- link from RN2 to ETR2 then
      RN2 will be multihomed.

RN1, RN2 and RN3 each have a single IP address, which is within
one of the "master-subnets" which all the ITRs advertise.  They
have no other IP address.

Raw packets addressed to RN1, RN2 or RN3 from CN1, CN2 or CN3
are all forwarded to the nearest ITR.  This ITR function may be
performed in a router inside their edge edge network, in a
border router, in a transit router or in a specialised ITR which
has a stub connection to a transit router.

Diag. 4
                   (ITR4)
                    ~  |
                    ~  |
   RN4---<-(ETR4)~~~(TR)--<--CN4

Here a packet addressed to an Ivip-mapped IP address travels
from CN4 to a transit router.  The transit router sends it to
the ITR4 which is the closest of potentially tens of thousands
of ITRs which advertise the complete set of "master-subnets" of
addresses which are handled by the Ivip system.

The ITR4 has only one connection - to the transit router.  It
encapsulates the packets in IP-in-IP headers addressed to
whichever ETR the original packet's address is mapped to,
according to the central database which the ETR has an
up-to-date copy of, or which it queries and maintains a partial
cache of.

The transit router then forwards it through various other
routers not shown to wherever in the world the ETR is.  The ETR
strips off the header and sends the raw packet on a local
network to the Receiving Node.

The ETR can be a relatively dumb, unmanaged, router which is not
at all part of the database-ITR system.  It simply decapsulates
the packet and the local routing system of the edge network must
be configured to forward the packet to the RNn.

The ETR function is always implemented in an internal or border
router.  It is never in the "core" - meaning implemented in a
transit router.  This is because the decapsulated packets have
the final IP address and need to be handled by a suitably
configured local routing system to get them to the RN.  Without
this special routing, they would be forwarded to the border
router, since they are part of one of the "master-subnets" which
all the ITRs advertise.  (With LISP, the mapped address is not
part of any BGP advertised prefix, but the local network still
needs to route the decapsulated packets to the RN, and allow
packets from the RN to be forwarded out of the border router and
then be forwarded by the normal BGP system.)

(It is also possible for the CN to perform the ITR function and
encapsulate its own outgoing packets, but that will make a mess
if it is behind a NAT.)

It is also possible for the RN to perform its own ETR functions.
In that case, it needs a second physical, local, IP address in
order that it can behave like its own ETR.  In mobile-IP that IP
address would be called a "care-of" address, which was reachable
via an ordinary BGP-advertised prefix.  In some ways, this would
resemble mobile-IP, because the RN itself is  taking part in
tunnelling.  However, for LISP or Ivip-basic, there is no 2-way
tunnel, and the edge network needs to be able to get the packets
sent by the RN out through the border router, for each RN to be
able to send packets back to the CN.  To the extent that this
resembles mobile-IP, the ITRs would be like the home-agent,
except that there are thousands of them, all potentially sending
packets to the RN, as if it was an ETR.  But the resemblance
with Mobile IP is slight, since the RN needs support from the
edge network to get its outgoing packets forwarded, which means
the edge network needs to know about and approve of its
LISP/Ivip-mapped IP address.

That is why I called it a Receiving Node - because LISP (as I
understand it) or Ivip-basic is a strictly one-way system.
Right-to-left in the above diagram.

It is up to the edge network and the normal BGP network to
support the carriage of packets from the RNs to the CNs.  If one
of the CNs also had a LISP/Ivip-mapped address, this would
involve ITRs and ETRs in the reverse direction,   But these
would generally be different ITRs, close to or within the edge
network depicted at the left of the diagram, and would certainly
be different ETRs than those depicted above, assuming the RNs
and the CNs were in different edge networks.

The diagram above does not depict packet flows left-to-right,
because they flow by ordinary BGP methods.

Now I will convert Diag. 3 to depict multiple Mobile
Nodes, using an Ivip-mobile extension: the ETRs are now TTRs
with a 2-way tunnel to the MN.

Diag. 5

                             /---CN1
                            /
 MN1===(TTR1)~~(TR)~~~~~(ITR1)---CN2
      =          ~
 MN2==            ~
                   (ITR2)--------CN3
                   ~
 MN3==(TTR2)~~~~~~~

      (TTR3)           (ITR3)
       ....             ....
      (TTRn)           (ITRn)

----  Raw packets flowing right-to-left.

~~~~  1-way tunnelled (IP-in-IP) packets as
      described in the previous diagram.

====  2-way tunnelled packets.

The TTR (ETR) functions could be located in transit routers,
border routers or internal routers.  With Ivip-basic, the ETR
function can't be in a transit router - it must be inside or at
the border router of an edge network.

A TTR function in a transit router can be tunnelled to by any
MN, no matter where it is, including behind a NAT.  Ideally,
each MN would pick a TTR nearby, perhaps in its own edge
network.  In that case, the TTR somewhat resembles a MAG of
Proxy Mobile IPv6.  However, with this idea of Ivip-mobile, the
TTRs always communicate with their MNs via two-way tunnels.

The main idea is that the TTRs can be outside edge networks, but
the MN will be able to find one far from whatever edge
network(s) it is currently connecting to the Net with.  This
way, the MN can make two tunnels, via two separate radio links
of two separate edge networks, to two TTRs which may be in those
 edge networks, at their border routers or outside the borders,
but not far away.  Then, it has true multihomed mobility, and it
(or some other central monitoring system) can direct the ITR
system to send its packets to whichever of the TTRs works best
at that moment.

If there was a link from MN2 to TTR2, then MN2 would be
multihomed.  It would have two "care-of" addresses, one from
each of the edge networks it is connected to. (I assume two edge
networks, such as two wireless networks, but it could still have
two tunnels to two TTRs from one "care-of" address in one edge
network.)

The only three changes from Diag. 3 to Diag. 5 are:

1 - I haven't shown the ordinary ETRs used for Ivip-basic -
    but they still exist.  By the way, the Ivip database
    and its ITRs don't need to know, or care, whether they are
    tunnelling packets to an ordinary ETR or to a TTR.  This
    part of the diagram - the ITR side - hasn't changed in Diag
    5 because it *is* "Ivip-basic".

2 - The MRs establish 2-way tunnels with their chosen TTRs.

3 - Packets flowing left to right from the MNs to the CNs
    first flow in the 2-way tunnel to a TTR.  Then the TTR
    uses the ordinary BGP routing system to get them to the
    CN destination.

We could make TTR2 into an ETR2 and have a link from MN2 to
 ETR2.

Diag. 6

                             /---CN1
                            /
 MN1===(TTR1)~~(TR)~~~~~(ITR1)---CN2
      =          ~
     =            ~
  MN2~             ~
      ~            (ITR2)--------CN3
       ~           ~
 MN3--(ETR2)~~~~~~~

Then MN3's link to the Net would be an ordinary 1-way LISP/Ivip
arrangement, via ETR2.

MN2 would have an edge-network-supplied "care-of" address, from
which it would establish a 2-way tunnel to TTR-1.  If the ITRs
tunnelled packets with MN2's Ivip-mapped address to TTR-1, MN2's
connection to the Net would be Ivip-basic to TTR1 and then
Ivip-mobile to itself.  TTR1 would probably receive its outgoing
packets and forward them via ordinary BPG.  Alternatively, since
MN2 is in an edge network with an ETR (ETR2), and has chosen to
use this as an optional link to the Net, presumably this edge
network has also allowed packets sent from MN2's
LISP-Ivip-mapped address to be forwarded to the border router
and handled by the normal BGP system.  In that case, MN2's
outgoing packets might flow via that edge network's routers,
which are not shown in the diagram.

If the ITRs tunnelled packet's addressed to MN2's packets to
TTR-1, its connection to the Net would be ordinary Ivip-basic
via ETR2.

There are some similarities between the Ivip-mobile link, and
mobile-IP.  However, it might be best to think of Ivip-basic and
its extension Ivip-mobile being like:

1 - The MN establishes one or more 2-way tunnel-mode links
    to TTRs, each of which is somewhat like a home-agent.
    However, the MN can choose between a bunch of these
    TTRs.  Some may be in the edge network, or be its
    border routers.  Some may be in the core.  TTRs are
    ideally all over the world.  They could be run by
    multiple companies with paid-for, authenticated, access.
    These one or more systems of TTRs, like the ETRs, do not
    need to have any relationship with the LISP/Ivip system
    (the database and the ITRs it controls).

2 - The MN's portable address has nothing to do with the
    addresses or physical/topological addresses of the TTRs.

3 - All packets sent by any number of correspondent nodes
    to the MN's Ivip-mapped (portable = mobile) address are
    funnelled by BGP anycast into any of the thousands or
    hundreds of thousands of ITRs all over the Net.

    (ITRs may be inside CN's edge networks, at the border or
    in the core.  Also, the CN can perform the ITR function if
    it is not behind NAT.)

4 - A central database controls which ETR or TTR receives
    the tunnelled packets which were addressed to the CN's
    Ivip-mapped (portable = mobile) address.

5 - There is no need for any special software in the CN or
    any changes in their edge networks. (LISP requires ITRs
    in CN's edge networks.)

6 - Total path lengths will be optimal or close to optimal
    assuming there are well-placed ITRs and the MN chooses
    a nearby TTR.  So there's no need for triangle routing,
    route optimisation, establishing trust between the
    CN and the MN etc. as is needed to optimise the path
    length taken by packets with mobile-IPv6.

7 - The TTR or ETR functions don't need to be coordinated
    by the Ivip system itself.  The Ivip system is the ITRs
    and the database which controls their behavior.

Because the explosion of the BGP routing table size is
threatening to cause excessive costs for everyone (and more
revenues for router manufacturers) to the tune of billions of
dollars, someone like me on the RAM or RRG list who is trying to
avert this can easily invoke a global network of ITRs and a
central database to control them.

If anyone had suggested such a thing be built just for the
benefit of mobile-IP users, the proposal would have been ignored.

Because we are desperate, this global network of ITRs seems like
the best (probably the only) way of averting either massive
router replacement and bloat, or major malfunction in the
Internet.  (I assume that it is impossible to institute a change
in Internet protocols, and therefore all routers, host operating
systems and probably application software.)

Ivip is just like LISP (or at least the more straightforward
aspects of LISP 2.00+ which I understand), except the ITRs can
be in the core, to serve end-users of networks who don't yet
have their own ITRs. This makes Ivip incrementally deployable,
while I believe LISP is not.

A fully functioning LISP or Ivip system is like a magic mirror
system, somewhat like the banks of steerable solar mirrors:

  http://www.nrel.gov/data/pix/Jpegs/00036.jpg

except that each IP address can be mapped to a different ETR,
and the mapping can be changed rapidly without upsetting the BPG
system at all.

This enables ordinary hosts to have their packets sent to
whichever MAG or Home-Agent a mobile-IP host desires.

The principles of LISP or Ivip apply equally to IPv4 and IPv6,
although there would probably be some significant differences in
implementation.  Since no-one thinks BGP can be replaced or
vastly improved, something like LISP or Ivip is needed in the
next few years.

Either LISP or Ivip makes it so much more simple and efficient
to do mobile-IP.

If IPv6 ever becomes widely adopted, perhaps something like LISP
or Ivip needs to be implemented for IPv6 too.  This could happen
if RIRs start handing out PI address space to AS-end-users.

Alternatively, if RIRs stick to current policy and don't hand
out PI space, someone will need to create a LISP or Ivip system
before ordinary networks (other than researchers, 3G mobile
networks and the Chinese government) adopt IPv6 widely.

Either way, I think IPv6 will have something like LISP or Ivip.
 This involves some special routers and a central control
system, but no direct involvement with host software and no
extra burden on most of the DFZ routers.

I guess this would make SHIM6 largely redundant.

SHIM6 is clearly not the portability and multihoming system most
end-users want, so on its own, it has not been enough to make
people want to adopt IPv6 without PI address space.

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 22 Jun 2007 09:37:47 +0200

...
> people want to adopt IPv6 without PI address space.

I suggest we don't have that conversation. I could write a long
rant in response - but it seems much more constructive to talk
about routing and addressing here; shim6 is an orthogonal
discussion, and I believe we should just let the WG finish
its job.

    Brian

---

From: Robin Whittle <[email-masked]>
Date: Sat, 23 Jun 2007 22:05:43 +1000

Here is a diagram which might help with understanding Ivip.

I am maintaining a list of the longer messages I write about
Ivip at: http://www.firstpr.com.au/ip/ivip/

At present, this is the rambling Ivip documentation. I plan to
write a much more readable Internet Draft in August.

Dino wrote over a week ago that Ivip was like LISP 1.5.  I have
tried to understand LISP 1.5 but have not been successful.  I
hope he or Vince can explain it better to me, or ideally to the
list.  I imagine many people have difficulty imagining concrete
examples of the LISP variants, because the descriptions in the
I-D are rather conceptual.

What I have written about LISP is just my understanding - and
doesn't include a bunch of LISP stuff such as ICMP messages,
multiple encapsulation etc.

I assume that both LISP and Ivip ITRs can have either a full
"push" copy of the central database or use "pull" and caching.

Here are four Edge Networks.

........                                 ..................
        .                               .
Edge-A   .                             .      Edge-B
         .                             .
    H01  .                             .    H04   IH01
         .                             .
    H02  .                             .    H05   IH02
        BR1                            .
    H03  .                            ETR1  H06   IH03
        .         TR1          TR2      .      ETR2
........                                 ..................

             ITR1      TR3

                 TR4             ITR2

.............                            ..................
             .                          .
  H10  IH07   ITR4                    .  ETR3  H07   IH04
              ETR4        TR5        BR2
  H11  IH08     .                     .  ITR3  H08   IH05
                .                     .
  H12  IH09     .                     .        H09   IH06
                .                     .
               .                      .
  Edge-D      .                        .     Edge-C
..............                          ...................

Key:

  TRx     Transit routers
  BRx     Border routers

  ITRx Ingress Tunnel Routers } In this example, these are also
  ETRx Egress Tunnel Routers  } function as internal or BGP
                              } routers.

          ETRs have at least one IP address which is a normal,
          BGP-reachable, IP address.

  Hxx     Hosts with ordinary BGP-routable IP addresses, either
          PI or PA.  For instance desktop machines and servers.

  IHxx    Hosts with one or more IP addresses which are mapped
          by the LISP/Ivip system.  These could be routers
          too.  See my earlier messages for how each such IP
          address is within one of the "master-subnets"
          which all the ITRs advertise.

With LISP 2+, hosts in Edge-A can't send packets to any of the
IHxx hosts with LISP-mapped addresses, because those addresses
are not part of prefixes advertised in BGP.  I think such
versions of LISP cannot be successfully deployed for this reason
- since no-one would want an IP address which some hosts
couldn't send packets to.

With Ivip, the mapped addresses are part of a BGP-advertised
prefix, so packets from Edge-A sent to such IP addresses are
forwarded to the nearest ITR, since all ITRs anycast advertise
the prefixes ("master-subnets") which Ivip maps.

There are five BGP advertised prefixes in this example.  One for
each edge network and one for the Ivip system - though in
reality the Ivip system would probably handle many prefixes.
The diagram shows 4 groups of normal hosts H0 to H12, each of
which uses PA addresses from its edge network.

The 9 hosts with Ivip-mapped addresses can each use their one or
more Ivip-mapped addresses provided they are in an edge network
with an ETR and that edge network's routing system firstly
forwards the decapsulated packets to the host and secondly
allows packets from that host's IP address to be forwarded
normally: to any internal host or out via the border router to
ordinary BGP routers.

Here are descriptions of the networks, with the BGP prefixes
each one advertises.

Edge-A   11.0.0.0/16
------

An ordinary ISP or AS-end-user network with no special router
functions to support LISP or Ivip.  The hosts in this network
are all ordinary hosts with conventional IP addresses which can
be reached from anywhere via the current BGP system.  For this
example, I assume these hosts have addresses provided by Edge-A
- PA (Provider Assigned) addresses. So there is no portability
between ISPs (different edge networks) for the IP addresses used
by these hosts.

It would also be possible for one or more hosts to be run by
some organisation with an ASN, in which case that AS-end-user
can allow Edge-A to advertise one of its PI (Provider
Independent) subnets at its border router and the hosts can use
that address space.  The primary purpose of LISP/Ivip is to
allow portability and multihoming to such end-users without them
needing to become Autonomous Systems or to have PI space
assigned to them.

Unless Ivip-mobile is used (which I have described in several
recent messages and which I will mention briefly below), the
hosts in this edge network cannot use LISP/Ivip-mapped
addresses.  They can however send packets to and from with all
the IHxx hosts in other edge networks - which have Ivip-mapped
addresses - by using one of the "core-ITRs", probably ITR1 which
is closest.

Edge-B   12.0.0.0/16
------

This network has been upgraded so its border router also
performs ETR functions.  The internal routing system has also
been configured to handle whatever the IP addresses are of three
hosts IH01, IH02 and IH03 which have LISP/Ivip-mapped IP addresses.

A second ETR2 function is implemented on an internal router.

Because there is no ITR in Edge-B, all its hosts rely on a
"core-ITR" when sending packets to any LISP/Ivip-mapped address
outside Edge-B.

Edge-C   14.0.0.0/16
------

Like Edge-B and Edge-D, this has a bunch of ordinary hosts and
IHxx hosts with LISP/Ivip-mapped addresses.

Any edge network with IHxx hosts needs at least one ETR.  This
can be at the border router or at an internal router.  Edge-C
has it in an internal router.

Edge-C has an ITR, also as in internal router.

These ITR and ETR functions are assumed to be additional to the
ordinary internal router functions.  However they could be
performed by a specialised router or server which only does this
function, and has a single link to an internal or border router.

Also, the ITR and ETR functions could be performed by the same
router or server.

Edge-D   15.0.0.0/16
------

This has IHxx hosts too, and its single border router performs
both ITR and ETR functions.   Hosts don't need to know where
ITRs are - Ivip-mapped packets are naturally forwarded to them.
 Whoever runs each Ivip-mapped host does need to know the IP
address of the one or more ETRs in the edge network which can
decapsulate packets for it.  However, the host itself does not
need any configuration.

There could be further ITRs and ETRs inside as well.

Note that all these IHxx hosts only have the IP address which is
LISP/Ivip-mapped to them.  This is not like most mobile-IP
scenarios (all scenarios except Proxy Mobile IPv6?) where the
host has a care-of address and some special software to create a
tunnel to a home agent or some other device, where that tunnel
carries packets to and from the IP address the host keeps when
it moves from one edge network to another.

I suppose a host could have one or more of both kinds of IP
address.  None of these hosts requires any special software.
All this applies to both IPv4 and IPv6, but I will use IPv4
example addresses.

I will assume a single Ivip system, but there could be multiple
such systems operating side-by-side, without any problems.  Each
would have its own database and set of ITRs, but they could all
tunnel packets to the same ETRs in edge networks all over the
world, which require no coordination.

A Master-Subnet and the mapping
-------------------------------

There are likely to be multiple "master-subnets" handled by the
Ivip - maybe hundreds or a few thousand.  Ideally they would be
larger subnets.  As more and more address space is Ivip-mapped,
the previously isolated subnets would tend to adjoin each other,
so less BGP advertisements would be required.

Let's say the Ford Motor Company, which (as far as I know) is
currently assigned 19.0.0.0/8 but which does not advertise it
(and therefore doesn't use it publicly), decides to give control
of this of the Ivip system.  That is 16,777,216 IP addresses.
Assuming the Ivip system doesn't also work with neighbouring
/8s, the Ivip system considers this as one of its
"master-subnets".  Every ITR advertises this subnet (anycast)
and all the other "master subnets".

Here I give some example mappings for two tiny sections of this
large master-subnet.

IP address    Host this    Current mapping: the
to be mapped  address is   IP address of an ETR
              mapped for

 19.0.0.0     IH05         13.0.3.1    ETR3
        1
        2     IH06         13.0.3.1    ETR3
        3     IH01         12.0.2.1    ETR1
        4     IH08         14.0.8.8    ETR4
        5     IH08         14.0.8.8    ETR4
        6     IH08         14.0.8.8    ETR4
        7     IH08         14.0.8.8    ETR4
        8     IH02         12.0.2.1    ETR1
 19.0.0.9     IH05         13.0.1.1    ETR3

. . . etc

19.75.0.0
        1     IH03         12.0.2.1    ETR1
        2
        3     IH07         14.0.8.8    ETR4
        4     IH04         13.0.1.1    ETR3
        5     IH09         14.0.8.8    ETR4
        6     IH02         12.0.7.7    ETR2

IH08 gets four contiguous IP addresses.  They are on binary
boundaries, but there's nothing about the Ivip system which
makes this necessary.   IH02 gets two IP addresses.

When the database changes the mapping for a large number of IP
addresses, say 1024 or so, it is generally faster and easier to
communicate this in a prefix format or as a range between two
addresses than to specify the change of mapping for each IP address.

If an ITR uses "pull" and enquires to the central database, or
some proxy server for it (perhaps a local "pull" database ITR?)
about 19.0.0.5, it would receive in the reply not just the
current mapping (14.0.8.8) but information indicating that the
same mapping applies for adjacent addresses 19.0.0.4 to 19.0.0.7
inclusive.

A multihomed host or router, not shown, could have two different
links such as via ADSL and a cable modem to Edge-B and Edge-C.
By changing the database entry for that host's or router's one
or more IP addresses from pointing to ETR2 to make them point to
ETR3, packets to the host would arrive on the Edge-C link rather
than from Edge-B.

Example packet flows
--------------------

----  Ordinary packets with their natural IP address.

~~~~  Packets encapsulated with an IP-in-IP header, so
      the outer header's destination address is that
      of whichever ETR the database currently specifies.

Normal <---> Ivip-mapped

 H01->-BR1----TR1----ITR1~~~~~TR3~~~~TR2~~~~ETR1-->-IH02

 H01-<-BR1----TR1--------------------TR2----ETR1--<-IH02

Left to right, the packet leaves Edge-A and is forwarded to the
nearest ITR.  All ITRs advertise 19.0.0.0/8.  ITR1 tunnels the
packet to 12.0.2.1, which is ETR1.  ETR1 decapsulates it and
Edge-B's routing system delivers the packet to IH02.

This is a longer path than would be the case without Ivip,
because the packet takes a detour to reach the nearest ITR.
Ideally, there would be ITRs scattered through the core of the
network.

When the two edge networks are far apart, it doesn't matter much
if there are only one or a few ITRs somewhere in the path.  All
that matters is that the packet does get to an ITR at some point.

If Edge-A and Edge-B were close to each other, but the nearest
ITR was a long way away, then the packets would travel on a
longer path than would otherwise be necessary.

With ITR functions located in transit routers, ideally they
would be one hop away from border routers, so there would be
little or no lengthening of the path.

A better result is if the ITR function is performed by the
border router, or an internal router.  Then there is no longer
path - only a slightly longer part of the path with the small
burden of the IP-in-IP header.

In the reverse direction, H01's address is an ordinary
non-Ivip-mapped address so it is forwarded normally.  ETR1 in
this case is acting as an ordinary border router.

Ivip-mapped <---> Ivip-mapped

In this case, I will illustrate packet flow between edge
networks which both have their own ITRs and ETRs, so there is no
increase in path length for using Ivip.

 IH08->-ITR4~~~~~TR5~~~~~BR2~~~ETR3->-IH06

 IH08-<-ETR4~~~~~TR5~~~~~BR2~~~ITR3-<-IH06

Ivip-mobile
-----------

All this means is that the ETR is a "TTR" (Translating Tunnel
Router).  The database and ITRs don't need to know the packets
are directed to a TTR.  I explained Ivip-mobile in greater
detail in previous messages.

The Mobile Host has some kind of Internet connection which gives
it a "care of" address.  This is different from normal Ivip,
where the host has only (or at least only needs) the Ivip-mapped
address.

The Mobile Host establishes a two-way tunnel of some kind, maybe
IP-in-IP or some compressed, encrypted tunnel, with the TTR of
its choice.  Ideally this would be inside the edge network the
host is currently connected to, or at its border router.
However the TTR could be at any location in the Net, including
inside other edge networks.  All it needs is a normal IP
address.  The host could be behind multiple layers of NAT - as
long as it can establish the tunnel.

There could be multiple independent networks of TTRs each with
TTRs in the core of the Internet, but ideally close to border
routers.  Then a host can find one which is pretty close so the
packet paths generally won't be much longer than necessary.

Having the TTR in the same edge network is good because there is
no difference in path length in the core of the Net, and minimal
increase in path length when communicating with other hosts in
the same edge network.

   - Robin