Subject: [RAM] LISP NERD/CONS, eFIT-APT & Ivip compared

From: Robin Whittle <[email-masked]>
Date: Tue, 17 Jul 2007 20:12:59 +1000

I started off wanting to discuss what was in common and different
between LISP 3.x and eFIT - however it grew into an attempt to
compare major aspects of the current proposals.

  - Robin      http://www.firstpr.com.au/ip/ivip/

LISP 1.0 and to a limited extent 1.5 is documented in the LISP-01
I-D:

  http://tools.ietf.org/html/draft-farinacci-lisp

There is nothing specific there or anywhere else about LISP 3.x -
however it has been stated on the RAM list that 3.x is the most
likely variant to be deployed.  1 and 1.5 are theoretical models
for trying out concepts - but the look very different to me from
3.x, so I don't know what lessons learnt with them would be
applicable to 3.x.  According to Dino's RAM list message 1703
(July 13), the three mapping distribution systems applicable to
LISP 3.x would be:

NERD   (No data driven Map Reply messages.)

  http://tools.ietf.org/html/draft-lear-lisp-nerd-01 (June 12)

CONS   (No data driven Map Reply messages.)

  http://tools.ietf.org/html/draft-meyer-lisp-cons-00 (July 3)

APT    (Perhaps some data driven Map Reply messages.)

  http://tools.ietf.org/html/draft-jen-apt-00 (July 2)

APT is designed not for LISP, but for eFIT:

  http://www.cs.ucla.edu/~lixia/papers/07SIG_IP6WS.pdf

    A Scalable Routing System Design for Future Internet
    Daniel Massey, Lan Wang, Beichuan Zhang and Lixia Zhang.
    July 2007

  http://tools.ietf.org/html/draft-wang-ietf-efit-00

    Of the efit-00 I-D, Lixia wrote on this list on 10 July:

    > That one is really outdated.  We are working on a new
    >  version.  Will send a pointer before Chicago.

So there are now five potential new architectures - all but one
with a push database arrangement:

LISP-NERD
LISP-CONS  (Pull.)
LISP-APT   (APT is only intended for eFIT.)
eFIT-APT
Ivip

All these are in an early stage of development. Ivip is
extensively documented, http://www.firstpr.com.au/ip/ivip/.

NERD is a push system in that all ITRs have a copy of each mapping
database for each prefix of LISP-mapped addresses.  It has only
modest speed goals.  "... updates to the mapping are intended to
be relatively rare".  The mapping database is not intended to
change when a link fails or is restored - so the ITRs must somehow
be smart enough to detect loss of reachability to an ETR, and
already have the alternative ETR's address in their database, to
switch to another ETR which is reachable.  Since there is no
ITR-to-ITR communication about this as far as I know, I guess each
ITR has to figure out reachability on its own when it tries to
send packets.  This is much more complex for the database contents
and for ITR functionality than Ivip - in which an external
monitoring system changes the mapping and the ITRs simply follow
the changes to the mapping, which need to be delivered very quickly.

While NERD is "push", in comparison to CONS - which is purely a
query and cache system - the mapping updates are not actually sent
to the ITRs.  Each ITR has to poll a server and retrieve the
update files.  However NERD-01 anticipates ITRs advertising the
update files they have, and downloading these files from
neighbouring ITRs, as an alternative to getting them from
centralised servers.  If each update file needs to be fully
downloaded and authenticated before it can be made available for
other ITRs to download, then this will lead to a slow propagation
of these files across the Net.

Alternative approaches using NNTP, and with ITRs only storing part
of the database with "push" and are also contemplated.  (I don't
understand how an ITR would have only part of the database, unless
it has another ITR with the full database to which it would tunnel
the packets it doesn't have mapping for, or a nearby full database
query server like in Ivip or eFIT-APT's Default Mappers.)

LISP-CONS is a "pull" system.  ITRs query and cache the mapping
data, via local CARs which also cache.  Requests and replies
traverse a mesh of CARs and (non-caching) CDR servers, all linked
via authenticated TCP sessions.

Depending on the caching time, LISP-CONS could work with more
frequent database updates than LISP-NERD, but there is no attempt
to speed things up as with Ivip's Notify (cache invalidation) or
the same concept in eFIT-APT: "updated mapping entry" - which are
real-time, local, push messages directed to the caching ITRs (and
for Ivip, the caching Query Servers) which they query.

eFIT-APT differs from LISP and Ivip in a number of respects,
including:

1 - ETR and ITR functions are always contained in the same device
    (TR) whereas these can be in separate devices for both LISP
    and Ivip.  These TRs will replace all current Provider Edge
    routers - or all PE routers will be upgraded to have these
    capabilities.

2 - The ITR function is a query and cache system, as with
    LISP-CONS and Ivip's ITRC (ITR Cache) and ITFH (Ingress Tunnel
    Function in Host), but unlike the full database ITR in
    LISP-NERD in in Ivip's ITRD.

3 - Each transit network is expected to run at least one "Default
    Mapper", which may be router (at least it can encapsulate
    packets, even if it only has one interface), and which has a
    real-time updated copy of the entire eFIT-APT mapping
    database.  (Except in the future for IPv6 - section 7.4.)
    There is nothing about implementation of Default Mappers in
    the I-D, but I think they would best be implemented in a
    server, rather than a conventional router with hardware FIB
    etc.)

    ITRs send queries to the Default Mapper in their network.
    Each query is packet they do not currently have mapping data
    for.  The Default Mapper encapsulates the packet and forwards
    it towards the ETR in the destination network, and then sends
    back an ICMP response to the ITR with the mapping information
    the ITR needs to handle such a packet in the future.  (If a
    future packet's DA was part of a prefix with the same mapping
    data, then the ITR would then know how to encapsulate it.)

    This aspect of the Default Mapper is analogous to Ivip's QSD
    (Query Server with Database) combined with Ivip's ITRD
    handling packets which can't immediately be tunneled by an
    ITRC or ITFH.

    This will lead to much faster and more reliable query
    responses than LISP-CONS - but as with Ivip, raises the
    challenging problem of distributing the database updates in
    real-time to all these Default Mappers.  However, eFIT-APT
    has very slow goals regarding the speed of distributing
    these updates.

    As with Ivip, there is no delay for packets which cannot be
    mapped by an ITR, although APT's Default Mapper or Ivip's
    "backup" (maybe I should call it a wicket-keeper . . .) ITR
    could be a bottleneck.  With Ivip, there could be a chain of
    ITRCs leading to a final ITRD.

4 - The Default Mappers can be reached by anycast within a
    particular provider network.  (I don't propose QSDs be
    accessed this way, but it is an interesting idea as an
    option, provided all queries, responses and Notifications can
    be done with UDP.  Since Notification involves the QSD
    ensuring that the message was received by whichever QSD or
    ITRC the QSD sent it to, TCP might be a better approach than
    UDP - so anycast QSDs may not be such a good idea.)

    There is no two-way communication when the Default Mapper
    sends the ICMP packet with the mapping information to the
    ITR.  Two-way communication would be tricky with anycast
    unless all the Default Mappers had very intimate links
    between them, which is to be avoided.  However, I guess
    non-guaranteed ICMP reply is fine, since if the ITR
    doesn't get it, the worst thing that can happen is that
    the ITR will send the next data packet to the Default Mapper
    as well.

    Default Mappers also have the equivalent of an Ivip "Notify":
    "M1 can respond with an updated mapping entry".  There's
    no mention of the ITR having to acknowledge this - and even
    if it did, if anycast is used, it would be difficult to ensure
    that the ack would go to the correct Default Mapper.  Both
    Ivip and APT should have a way of ensuring the updated mapping
    information really has been received by the ITR (APT) or QSC,
    ITRC or ITFH (Ivip) it was sent to.

5 - The ITR function of APT TRs does not make any decisions
    regarding which of multiple ETRs to tunnel to in a multihoming
    scenario.  Each ITR is told to tunnel packets which match a
    particular EID prefix to a single RLOC location: a single ETR.

    (However, see the next point about the ETR function performing
    various functions and sending messages back to an ITR.)

    This *greatly* simplifies the FIB work of the ITRs - and
    reduces the size and complexity of the mapping information
    they cache.  In this respect, the ITR function of an APT TR
    somewhat resembles an Ivip ITRC.  However, unlike an Ivip
    ITRC, and APT ITR is required to detect unreachability of
    the tunnel endpoint - the ETR's address.  It can do this via
    the ETR's address being unreachable, or via an ICMP
    Destination Unreachable message, or some other ICMP message
    for the third kind of failure (5.1.3).  (I am not convinced
    there would always be an ICMP Destination Unreachable message
    if there was no host at the address where the ITR (ITR
    function of a TR) is supposed to be.)  In any of these cases
    the ITR sends the next packet intended for this ETR to the
    Default Mapper instead, and it is the Default Mapper which
    makes the difficult decisions, including by tunneling
    the packet to an alternative ETR (assuming it is reachable
    via BGP) and waiting to see if there are any ICMP messages
    in response.

    I think the way the APT system responds to ICMP messages, at
    least the "Destination Unreachable" messages - which would not
    have any nonce from any APT data packet header (no header
    details are currently available) - makes it an easy target for
    spoofed ICMP messages for a DoS attack.

    Devolving the multihoming link selection problem, and the TE
    functions, to the Default Mapper is a good idea, I think, but
    involves more ICMP packets to ITRs with "updated mapping
    entry" information.  As noted above, I don't see any provision
    in APT for this to be delivered in a reliable manner.  There
    also needs to be some crypto or protection against an attacker
    spoofing packets with the Default Mapper's address, to ensure
    the ITR's cache is not corrupted by someone who wants to steer
    packets to their own address.

    The ITR function of TRs are also required to forward any ICMP
    messages they receive to their Default Mapper - which will use
    these to decide which of the various RLOC mappings are for
    ETRs which are currently reachable.  Again, all these
    communications need to be secured in some way against spoofed
    packets from an attacker.  (This is acknowledged in sect. 7.)

6 - 5.1.3 states that an ETR which can't reach the destination
    host or router will send the packet to its Default Mapper,
    (which has some complex rules about not sending it back to
    the same ETR) as well as sending an ICMP message back to
    the ITR in the sender's network which tunnelled the packet
    in the first place. (Actually, the packet could have been
    tunneled by the Default Mapper in the sender's network too.)

Here are some questions about each scheme, with "LISP" implying
LISP 3.x.  There is no concrete information about LISP 3.x other
than that implied by NERD and CONS that I won't consider how LISP
might work with APT.

Encapsulation of data
---------------------

LISP-NERD/CONS: Not documented yet, but presumably using UDP, with
the outer source address being that of the ITR which encapsulated
the packet. (Outer SA = ITR's address.)  See the recent messages
"ETRs checking src & dest addresses" about how I think this makes
it harder for the ETR's filtering system to protect against
spoofing of local source addresses.

eFIT-APT: Not documented, but APT mentions an ICMP Destination
Unreachable message being generated if the encapsulated packet
does not reach an ETR, so I guess this means UDP encapsulation
with outer SA = ITR (or Default Mapper) address, similar to that
described in LISP-01's description of LISP 1 / 1.5.

Ivip: Currently defined as IP-in-IP with outer SA = inner SA, to
help with the filtering.  However, it could be changed to UDP with
the ITR's address in the payload, with outer SA = inner SA, to
enable unwelcome packets to be traced back to the ITR which
encapsulated them.

Source of mapping data
----------------------

LISP-NERD: A hopefully small number of database authorities.  No
particular relationship to any provider network, as far as I know.
 Various other approaches are considered too.  This seems to be
different from what I call the "Bring-your-own address space"
model of LISP-CONS and maybe eFIT-APT.

LISP-CONS: CAR servers are authoritative.  These are distributed
around the Net and there are two or more in some redundant
arrangement - though it is not clear from the current I-D how a
CDR decides to forward a query to two or more redundant CARs, or
whether both replies are sent back through the CDR mesh to the CAR
which made the query.  I guess the CARs are related to provider
networks and the various LISP-mapped blocks of address space are
controlled by various providers.

eFIT-APT: Provider networks have one or more Default Mappers,
which in addition to many other functions, are the authoritative
source of mapping information for a set of prefixes which are
currently BGP advertised by that provider (but won't be at some
time in the future).  So as with LISP-CONS, it seems the system of
address management of mapped address space which is dependent on
existing provider networks.

If an end user is with provider X, using some of their PA
addresses, could they go to another provider Y for connectivity,
or X and Y or Y and Z for multihoming, and still have to rely on X
to to the mapping the way they want?

What if an AS-end-user with their own PI space is currently
connecting with provider X.  If they want to move to provider Y,
does this means that X's Default Mapper no longer is
authoritative, and that Y's is?  I am confused about exactly how
eFIT-APT works.

Ivip: Multiple independent Root Update Authorisation Systems
devolve responsibility to other systems and then to end-users.
Updates are funneled in real-time to a Replication system which
fans it out to ITRs and QSDs all over the Net.  So the address
space management, mapping changes etc. is not tied to any provider
organisation or to provider devices or networks.  I think
LISP-NERD's organisational aims may be similar.

Speed and method of update propagation
--------------------------------------

LISP-NERD: Slow.  ITRs are "full database" and download files from
each database authority.  The ITRs then poll for changes.
"Updates to the mapping will be relatively rare" and "are not
intended to change when a link either fails or is restored".  So
TE and multihoming service restoration are performed by the ITRs
and ETRs, rather than via changes to the mapping database.  I
guess times of tens of minutes to hours, depending on how busy the
ITR is about polling for changes.  In principle, it can be quite
fast, but that requires a huge burden of polling traffic.

LISP-CONS: Faster than LISP-NERD. All communications via a meshed
TCP-linked CDR network - queries go along it and responses come
back along it, rather than the responses going straight from the
authoritative CAR to the querying CAR.  (I understand this is
because the meshed network is secure and all the TCP sessions are
already running.)  ITRs are query and cache devices, via CARs
which also cache and which are also typically authoritative for
some part of the mapping database.  EID-to-RLOC mappings "are
assumed to change at low frequency".  As with LISP-NERD, TE and
multihoming service restoration are performed by the ITRs and
ETRs, rather than via changes to the mapping database.

eFIT-APT: Very slow indeed.  Default Mappers in each provider
network are authoritative, and push their data to other Default
Mappers via BGP.  Full database push and occasional update push
techniques are used.  "Transient failures do not cause mapping
announcements in APT."  "Permanent mapping changes are unlikely to
occur more than once per month per customer."

Ivip: As fast as possible - maybe a few seconds from the end-user
changing their mapping to all ITRs following the new mapping.
"Notify" messages from QSDs to ITRCs and ITFHs.  This requires an
ambitious Replication network, which will involve authentication.
 Full database downloads via HTTP are used by ITRs and QSDs when
they boot up, or to refresh their copy of each database if there
is corruption.

Method of performing short term MH and TE changes
-------------------------------------------------

LISP-NERD/CONS: The mapping database (really multiple databases)
gives instructions to ITRs, which detect failures and make
moment-to-moment decisions regarding TE and changes to which ETR
for multihoming.  This leads to a complex database, to complex
communications, complex ITR and ETR functions and to greater
difficulty securing these communications against attacks.

eFIT-APT: Similar in principle to LISP in that the mapping
database is not involved in the dynamic responses, and the ITRs
etc. have to work it out amongst themselves.  ITRs (or rather the
ITR functions of TRs, which also have an ETR function) are
involved in detecting failure, and send messages to their own
Default Mapper.  They may also send messages (as an ETR) to the
ITR which encapsulated the packet (and although it is not
mentioned, that encapsulation might have been done by a Default
Mapper in the sender's network, so there is a complex web of
potential communications, which will be difficult to make reliable
and difficult to secure against attack.

Ivip: The Ivip system (Root Update Authority Servers, Replicator
system and ITRs (really ITRs and ITRCs/ITFHs which rely on QSDs
and perhaps QSCs) are not involved at all in failure detection or
any explicit system of TE.  Therefore, the database is very simple
and more suitable for being fully stored in numerous ITRs and
QSDs.  The simpler database is also a lot easier to distribute
updates for.  The Replicator system is the most ambitious part of
the system, and aims to get updates to ITRs etc. very quickly.

TE and responses to mulithoming outages are done by some external
system - the end-user manually, or some automatic system they set
up, including perhaps their host or router or some multihoming or
mobility monitoring system.  So the Ivip system needs to be fast,
but it can be conceptually quite simple and involve very defined
and more easily secured communications than the other systems.
Also, since the control of the mapping is external and not at all
encoded into the Ivip system, it is endlessly flexible, simply by
the end-user giving their credentials to any software system which
controls the mapping the way they want it.

ITR function and placement
--------------------------

With LISP and Ivip, ETR and ITR functions may often be in the same
device.  ETR and ITRs always being in the one device is enforced
by the current definition of eFIT-APT - but I don't see why the
system needs to be so inflexible.  The ITR functions in LISP and
eFIT-APT must be on ordinary BGP reachable addresses.  Ivip ITRs
are typically on BGP reachable addresses, but can be on
Ivip-mapped addresses.  Those on Ivip-mapped addresses would
usually be ITRC and ITFH functions in hosts, rather than the full
database ITRD with its requirement for a constant feed of updates
from typically two Replicators.

LISP-NERD/CONS:  As noted above, ITRs have complex functions,
including making decisions on a packet-by-packet basis for MH
service restoration, TE and I think load balancing functions.  The
ITR may or may not communicate with the ETR, but I think the ITR
is always ready to accept ICMP messages as sign it is tunneling
the data packets to the wrong address.

Since LISP-mapped addresses are not in any prefix which is
advertised in BGP, ITRs must be inside or at the border of
provider or end-user networks.

eFIT-APT: The placement of TRs is specifically limited to the
border routers between providers (defined as any organisation such
as a provider which offers transit of packets through its network
which are not for its directly connected customers) and their
non-provider "customer" networks.  ITRs have complex communication
functions, including detecting failure.  However, they only tunnel
to one RLOC at any one time - the Default Mappers do the hard work
for TE and MH failure restoration - of deciding what the mapping
will be for each ITR.  Default Mappers also perform ITR functions
- usually for packets sent to them by ordinary ITRs which have no
mapping data for them.  Having the ITR and ETR functions fixed at
these routers - Provider Edge routers I think - seems likely to
produce a bottleneck and burden these routers with even greater
workloads.  This may also preclude doing ETR or ITR functions in
servers, which may be more cost effective than routers with all
the new functionality.

Ivip:  ITR functions can be in the sending host in routers in
customer and provider networks - and can be "anycast ITRs in the
core".  Core ITRs are typically full ITRDs.  ITRCs and ITFHs rely
on nearby query servers, of which there are caching and full
database types.  Packets not encapsulated by an ITRC or ITFH can
be left for another ITRC or ultimately an ITRD in the network or
in the core to encapsulate.

Unless I change Ivip so it uses UDP encapsulation to include the
ITR's address, the encapsulation is IP-in-IP, which has less
overhead in bytes and processing than UDP.  The ITR doesn't choose
different addresses to use as the tunnel endpoint - there is only
"RLOC" (LISP terminology) address for every "EID" address or prefix.

The ITR doesn't respond to ICMP messages, or send messages to
anything.  A conventional server could probably be programmed to
perform reasonable speed (500Mbps or more?) ITRD or ITRC
functions.   ITR functions can be performed on hosts and routers
which use Ivip-mapped addresses - but these can't be behind NAT.

ETR function and placement
--------------------------

ETRs must always be on BGP-reachable addresses.  So there are no
ETRs located in networks which themselves have all their addresses
mapped by the LISP/eFIT-APT/Ivip system.

LISP-NERD/CONS: ETRs are located in the provider network or at its
border router, or in the end-user's network or at its border (CE)
with the provider network - as long as the address is BGP
reachable.  I think ETRs have complex communication functions.

eFIT-APT: The placement of TRs is specifically limited to the
border routers between providers and their non-provider "customer"
networks.  ETRs have complex communication functions, including
detecting failure of the link to the end-user's host or router.
They send messages to their local Default Mapper and to the ITR
function of the TR which encapsulated the packet (which can
include the Default Mapper of the sender's network).

Ivip:  ETRs are typically located as per LISP, but there are some
exceptions.  Firstly, a TTR (Translating Tunnel Router) can be
outside the network the end-user's mobile node is connecting with,
with a 2-way tunnel from the mobile node.  Secondly, a host with a
BGP-reachable (care-of) address can perform its own ETR functions.

ETRs are not involved in any communication with ITRs or with the
database system.  They may be involved in failure detection, but
that is outside the scope of Ivip.  As with the other systems,
Ivip ETRs need to know how to get packets to the destination
host/router, based on the inner DA.  See the discussion on "ETRs
checking src & dest addresses" for some simple filtering they do
to drop packets where outer DA != inner DA - but this is part of
supporting the network's filtering of packets with spoofed local
addresses, which is a problem the other proposals have not yet
addressed.  That discussion also concerns why I think ETRs in any
of these schemes need to have an explicit connection (including a
tunnel) to the destination host/router, rather than relying on the
local routing system to know how to get decapsulated packets to
the destination.

Incremental deployability
-------------------------

LISP-NERD/CONS: According to current definitions and all my
off-list attempts to understand LISP, LISP-mapped addresses are
not part of BGP advertised prefixes.  So in order for a host to
send packets to a host on any LISP-mapped address, the sending
host must be in a network with an ITR.  I think this makes LISP
not at all incrementally deployable.  Why would an early adopter
want to use LISP-mapped addresses when it will make their hosts
unreachable for any communication initiated by a host on a
non-LISP-upgraded network?

eFIT-APT: I am still trying to figure out what the transition plan
would be, but maybe it is more incrementally deployable than LISP.

Prefixes which are to be subject to mapping are still advertised
in BGP and the PE routers (where the TR=ITR+ETR functions are
performed) can accept incoming packets both via encapsulation and
by the existing, normal, delivery of raw packets.  However, it
will only be feasible to no longer advertise the prefix in BGP
when all, or virtually all (depending on how many sending hosts
the end-user doesn't care about) provider networks fully implement
eFIT-APT at all their CE routers.  This would take years, since it
involves expensive new functionality in a lot of routers -
probably requiring new routers in many cases.

In the meantime, what benefits would there be for end-user
networks?  They can't gain any portability - unless they accept
that hosts in non-upgradeable networks won't be able to send them
packets.

So this a similar bleak situation to LISP.  LISP provides a
powerful disincentive to early adopters - unreachability from the
all non-upgraded networks.  However, to the extent that LISP is
introduced, it does incrementally decrease the number of prefixes
in the global BGP routing table.  With eFIT-APT, there would be no
benefit for early adopters (no portability without grave loss of
reachability - and TE only for packets from upgraded networks) and
no benefit for the whole network (removal of prefixes from the BGP
routing table) until virtually all networks had upgraded.

One of eFIT-APT's goals is increased security by making provider
addresses unreachable from end-user networks.  However, assuming
full reachability was to be maintained, this could only happen
once every provider had fenced off every single end-user network
with its eFIT-APT-capable CE routers.  Even then, this is a weak
form of security, since a single breach in this global fence will
enable an end-user network to send packets to any transit provider
address, including any ETR (including with spoofed source
addresses in the inner and outer headers) - so the target ETR is
unable to distinguish this from legitimate packets arriving from
other provider network's ITRs and Default Mappers.

Ivip: Every prefix assigned to the Ivip system remains advertised
in BGP.  The idea is that each such prefix be used for more
end-user networks, and that in general bigger (shorter) prefixes
should be assigned to Ivip and used to serve thousands of
end-users.  Even with smaller (longer) prefixes, as these are
assigned and become contiguous, their mapping should be merged
into one RUAS - whereas each smaller (longer) prefix might have
had a separate RUAS initially.  Hosts on Ivip-mapped addresses are
always reachable by hosts from non-upgraded networks, provided
there are sufficient "anycast ITRs in the core".

As with all proposals, there are major challenges in setting up
the infrastructure.  Ivip's Replicator and ITR system is
adventurous, but the ITRs are doing a simpler job and the overall
design is simpler than the designs of other proposals.  The ETRs
are free-wheeling, low-cost devices and require no coordination.
With sufficient provider networks with ETRs, one or more
independent Ivip or Ivip-like systems could be set up, and work
fine alongside each other (each with their own set of core ITRs,
or perhaps all driving data to a common set).  These could
probably run at a profit, because they provide portability,
multihoming (with basic TE) and mobility with full reachability
(and generally pretty good path lengths, if the core-ITRs are well
distributed) from all hosts.

_______________________________________________
RAM mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ram

---

From: "Templin, Fred L" <[email-masked]>
Date: Tue, 17 Jul 2007 08:00:14 -0700

> LISP-APT   (APT is only intended for eFIT.)

Please also keep IPvLX on your list (http://ipvlx.com).

Fred
[email-masked]

---

From: Eliot Lear <[email-masked]>
Date: Wed, 18 Jul 2007 09:22:08 +0200

Robin,

This is an excellent email that at least demonstrates where our drafts 
are not clear, if not our own thinking in some cases.  I think the basic 
issue that you are contending with is that many of the ideas contained 
in many of these documents are evolving.  However, you captured a few 
points very clearly that are worth reiterating now:

   1. LISP in its later stages assumes some method to retrieve a mapping
      from EID to RLOC that does not deal with operational state.  This
      is true for both LISP-CONS and LISP-NERD.  In these cases we do
      indeed assume that the ITRs and ETRs play an active role in
      determining operational state.  The way that happens is evolving,
      indeed.
   2. The difference between a LISP-CONS CAR and a NERD database
      authority is only in scope.  Each is authoritative for a group of
      mappings.  The real question is how many.  Taken to one extreme,
      there is only one CAR and it look a lot like a single NERD
      database authority for all of the world.  Taken to the other
      extreme, each ETR could play the function of the CAR.  Each has
      some benefits and limitations.
   3. You accurately assess that I really didn't care too much precisely
      HOW the ITR receives a NERD database.  In fact my point was that
      it doesn't really matter all that much because of the nature of
      the data.  As we are talking about a provisioned mapping, the data
      is relatively static and can be signed in bulk.  Once that has
      occurred, you could use IRC (!!) to distribute it because other
      sides could test that the information arrived intact.
   4. You raise a good point about the speed with which a database
      update makes it across the network in LISP-NERD.  There are two
      mitigating factors, however.  First of all, most updates will be
      small.  The intent is for even hosts that have been down for hours
      to be able to retrieve incremental updates.  Second, again because
      of the nature of the data, unlike routing data, a smart
      administrator will remove an advertisement some time before
      actually disabling an ETR.  Unless, there are ETRs which
      themselves use EIDs, we do not have to worry about one mapping
      being dependent on another.
   5. Finally, you comment about complexity of database in terms of
      separating operational and provisioned changes.  I envision an ITR
      running a process that retains state about mappings it is using. 
      I would expect a table to contain the RLOC of the ETR, /ifstate/
      (e.g., up or down), and a timer.  The /ifstate/ implies a next
      action, such as polling to see if the ETR has returned to service,
      or when to expect some sort of positive acknowledgement.  As I
      said above, I think this area is evolving.  But the key point is
      that this table is separate from the mapping itself, and only
      indicates reachability.  If an ETR is unreachable or reachable, it
      is unreachable or reachable for as many mappings as it has. 
      Furthermore, traffic is not redirected to other mappings based on
      in stream messages, lest we expose ourselves to that same-named
      class of attack.

The rest of your message requires a bit more digestion, but I do 
appreciate you taking the time to write it.  I think it helps quite a 
bit to advance our thinking.

Eliot

---

From: Robin Whittle <[email-masked]>
Date: Wed, 18 Jul 2007 20:29:37 +1000

Hi Fred and Eliot,

Fred, I didn't try to compare IPvLX (http://ipvlx.com) with the
others because I think it is very different from these.  Firstly,
I understand that IPvLX requires changes in host software.
Secondly, it involves IPv6, although as best I understand it I
think you want it to help with IPv4 communications, and doesn't
actually require IPv6 connectivity.  Thirdly, I don't understand
it anywhere near enough to evaluate it.

The first point is the most important to me.  As long as a
proposal to solve what I call the "crisis in routing and
addressing" involves changes to host software, I am convinced it
will never actually be deployed.  I am am wrong about any of this,
please correct me.

Thanks, Eliot, for your appreciative response and for clarifying
some points about the state maintained in a LISP-NERD ITR.  I
understand that "ifstate" and the timer will exist for every RLOC
address the ITR has recently been attempting to send packets to,
and that the database could have one or more likely many EID
addresses and prefixes mapped to this RLOC address.

I re-read the proposals and got enough of the material in my brain
at once to be able to compare them.  There may well be
inadequacies in what I wrote.  I have a version of the comparison at:

  http://www.firstpr.com.au/ip/ivip/comp/

with a note about how the proposals will no-doubt change rapidly.
 I won't try to keep this comparison up-to-date forever, but if
there are any things I should change I will do so.  I point to
this mailing list discussion from that page.

  - Robin

---

From: "Templin, Fred L" <[email-masked]>
Date: Wed, 18 Jul 2007 08:13:18 -0700

Hi Robin,

Thanks for looking. Just as clarification, IPvLX does not
require changes to host software; it requires changes to
router software. If the host and router occur on the same
physical platform (which is *not* a requirement), then it
may appear that host software changes are needed - but,
it is really only router software that is being changed.

About similarities to other proposals, I think if there
were to be a LISP 2 it would probably look a lot like
IPvLX. But, there is no reason IPvLX can't be extended
to support other mapping mechanisms than just DNS, too.

About IPv6, IPvLX accomplishes the ID/loc split by using
IPv6 addresses as EIDs and IPv4 addresses as RLOCs.

Thanks - Fred

> -----Original Message-----
> From: Robin Whittle [mailto:[email-masked]] 
> Sent: Wednesday, July 18, 2007 3:30 AM
> To: [email-masked]
> Cc: Eliot Lear; Templin, Fred L
> Subject: Re: [RAM] LISP NERD/CONS, eFIT-APT & Ivip compared
> 
> Hi Fred and Eliot,
>

---

From: Robin Whittle <[email-masked]>
Date: Thu, 19 Jul 2007 08:42:50 +1000

Hi Fred,

Thanks for clarifying my understanding of IPvLX to some extent.  I
still don't have anything like a clear enough understanding of it
to compare it with the other schemes.

Is IPvLX intended to help with IPv4 in respect of multihoming,
portability, TE, splitting up existing space into finer and more
flexibly used sections for more users and/or with  reduction in
BPG advertisements and/or churn in IPv4?

If not, then as with Six/One, I don't think IPvLX helps with the
most pressing problems of the foreseeable future - and so is not
comparable to LISP, eFIT-APT or Ivip.

If your proposal does help with IPv4 communications, with
non-upgraded hosts - for instance resulting in immediate or
long-term reduction in the size of the global BGP routing table,
and/or in improvements to multihoming, portability or TE then it
would be great if you could explain on the list exactly how this
works, with network diagrams, examples of deployment etc.

Can IPv4 hosts on non-upgraded networks send packets to IPv4 hosts
on upgraded networks?  If not, then I think IPvLX suffers from the
same problem as LISP-NERD or LISP-CONS - and so is not
incrementally deployable because early adopters (really any
adopters until adoption is universal) suffer loss of reachability
Early adopters suffer the worst costs or barriers, since hosts in
virtually no other networks will be able to reach their hosts.

Although there is no detailed documentation of how eFIT-APT could
be incrementally deployed (at least in terms of reachability and
what is and is not advertised in BGP), I imagined a way it could
be done while retaining reachability from non-upgraded networks.
However, this defers the reduction in the size of the BPG routing
table until either all networks are upgraded, or until enough of
them are that some upgraded networks don't mind about not being
reachable from those networks which are not - and so stop
advertising their prefixes in BPG.

How does IPvLX handle the requirement that a multihoming system
short-term changes to traffic flow?  I think service restoration
for a multihomed link is an example of what Eliot called
"operational state"?  This is an important aspect of architectural
goals in which Ivip differs completely from LISP or eFIT-APT.

LISP-NERD/CONS and eFIT-APT all have slow-changing databases which
carry instructions for the ITR-ETR system on how to cope with
short-term traffic flow requirements, such as loss of reachability
on one link, requiring packets to be tunneled to another ETR in
order to restore communications.  The short-term decisions are
made by the ITR-ETR and the mapping data does not change at all
during multihoming service restoration.  This means a more complex
database and *much* more complex ITR-ETR functionality, with much
more communication between ITRs, ETRs and in some cases recipients
of tunneled packets which are not ETRs which are ready to handle
the packets.  This introduces a much greater burden of complexity
in order to adequately protect against attacks and other security
problems.

In short, LISP-NERD/CONS and eFIT-APT all card-code short-term
behavior into the system itself, and make it configurable to some
extent by parameters in the mapping database.

Also, LISP-NERD/CONS and eFIT-APT do not attempt to assist with
mobile-IP - because their database distribution systems are too
slow with a mobile host's requirement that traffic flow to one ETR
or another in the different networks it has connections with.

Ivip is completely different.  The database is simple and intended
to be very rapidly changed.  Ivip does not attempt any
reachability testing or require its ITRs or ETRs to make any
decisions, such as for TE, multihoming service restoration or for
mobility.  This means Ivip can be used to support with mobile IP
just as it supports multihoming restoration, portability, finer
splitting of the address space among many more end-users etc. -
without actually containing any special database information or
ITR functions to support these separate applications.

Ivip more closely follows what I think is a common IETF
philosophy: single function, open-ended, building blocks which can
be used in combination with other building blocks to solve a wide
variety of problems, without having to create a monolithic system
for each particular problem.  (I don't know of a formal statement
of this, but it is evident from the whole nature of TCP/IP design,
DNS,  HTTP etc.)

Assuming IPvLX helps with ordinary IPv4 communications, it would
be great if you could explain on this list how IPvLX would be
deployed:

  Give a fully detailed example of the BGP or other benefits it
  brings to IPv4, and of how IPv4 packets flow in both
  directions between hosts in firstly two upgraded networks and
  secondly in one upgraded and one non-upgraded network.

  Describe the possible placement of upgraded or new routers.

  Describe the new features which are required in routers and
  how feasible these are to implement in the FIBs of current
  routers, with firmware upgrades.  The rest has to be done
  in the main CPU.  In both cases, it is important to discuss
  the communications, CPU and memory requirements of these new
  features for a reasonably widely deployed scenario.  Also,
  in the long-term future, can scaling be achieved by
  parallel routers, servers etc. rather than having to ramp
  up performance of routers which can't have their load split
  amongst others.  (eFIT-APT seems to suffer here, because all
  the ITR and ETR work has to be done in Customer Edge routers
  of provider networks.)

  Examples of packets which are used in IPv4 host communications
  and separate examples for IPv6.

  What other servers, databases, distribution systems etc.
  are required.

Then, ideally, you could write something about IPvLX for each of
the underlines headings in my comparison.

I would link to your message in the list or perhaps splice this
into my page:

    - Robin

---

From: "Templin, Fred L" <[email-masked]>
Date: Thu, 19 Jul 2007 09:58:26 -0700

> BPG advertisements and/or churn in IPv4?

IPvLX anticipates a future in which new growth within sites 
will be dominated by IPv6 applications and services, and that
the IPv4 growth curve will level off. This does not mean that
IPv4 applications and services would disappear, but rather
that pressures for multihoming, TE, etc. would be taken off
the IPv4 DFZ and instead be satisfied by a map-and-encaps
scheme that maps potentially many IPv4 RLOCs to the same IPv6
EID. Then, as more applications and services are transitioned
to IPv6, IPv4 address space could be reclaimed and redistributed
in finer-grained chunks such that more and more of the IPv4
address space is used as RLOCs for a site (and not EIDs).  
 
> works, with network diagrams, examples of deployment etc.

IPvLX leaves existing IPv4 communications alone and encourages
future growth in IPv6. It also encourages a longer-term trend
toward using IPv4 addresses more as RLOCs and less as EIDs.
 
> virtually no other networks will be able to reach their hosts.

No; there would be no loss of reachability. Non-upgraded
networks could communicate with upgraded networks using
IPv4 just as they do today. By proof of concept, Microsoft
Teredo is beginning to emerge in "upgraded" networks today
vis-a-vis Windows Vista deployment yet IPv4 remains fully
functional in those networks. IPvLX very closely parallels
the Microsoft Teredo model - both in terms of functional
behavior and anticipated deployment model.
 
> goals in which Ivip differs completely from LISP or eFIT-APT.

I didn't understand this.

> of this, but it is evident from the whole nature of TCP/IP design,

I don't see any questions regarding IPvLX in this.
 
>   brings to IPv4,

The benefits to IPv4 are that it encourages new growth in
IPv6 instead of IPv4.

>   and of how IPv4 packets flow in both

IPv4 works no differently between upgraded and non-upgraded
networks.

>   Describe the possible placement of upgraded or new routers.

Upgraded or new routers are deployed as site border routers.
 
>   the ITR and ETR work has to be done in Customer Edge routers

The neew feature for routers is the mapping engine.
 
>   and separate examples for IPv6.

IPv4 host communications is done exactly as it is today.
IPv6 host communications is done by tunneling IPv6 over IPv4.

>   What other servers, databases, distribution systems etc.

The current IPvLX proposal assumes a DNS mapping system;
future versions could use a different mapping mechanism.

Thanks - Fred

---

From: "Templin, Fred L" <[email-masked]>
Date: Thu, 19 Jul 2007 10:15:12 -0700

> and instead be satisfied by a map-and-encaps
> EID.

Sorry, this was poorly phrased; it should have said that
*many* IPv6 EIDs are mapped to *a few* IPv4 RLOCs - else,
what would be the benefit of transitioning to IPv6?

Fred

---

From: Robin Whittle <[email-masked]>
Date: Fri, 20 Jul 2007 11:13:19 +1000

Thanks Fred for clarifying my understanding of IPvLX.

I understand now that it is to support IPv6 user traffic over IPv4
networks, with the aim of encouraging IPv6 adoption and without (I
think) further burdening the global BGP routing tables for either
IPv4 or IPv6.

> I didn't understand this.

The primary goal of LISP, eFIT-APT and Ivip is to work with IPv4
traffic over the IPv4 routing system - or IPv6 traffic over the
IPv6 routing system - and in both cases to enable end-users to
achieve multihoming without having to advertise their address
space as a separate BGP prefix, and therefore without requiring
any change to BGP advertisements when their link from ISP A goes
down and they need their packets to come in via a link from ISP B.

Ivip enables the user to achieve multihoming service restoration
via fast database updates, driven by a user-provided mechanism,
whereas LISP and eFIT-APT have slower database distribution
systems and have the service restoration function built in to
their ITRs, ETRs etc.

I now realise that your proposal doesn't mention "multihoming".
This, and the fact that your proposal is only for IPv6 user
traffic, means that it is trying to solve different problems from
those which LISP, eFIT-APT and Ivip are trying to tackle.

> I don't see any questions regarding IPvLX in this.

OK - I was still discussing the differences between Ivip and the
other proposals' approach to multihoming service restoration.

> The benefits to IPv4 are that it encourages new growth in

I think any system which makes IPv6 easier to adopt, in a scalable
manner, is a good idea.  I don't know enough about IPv6 and the
various approaches such as 6to4 to be able to fully discern how
IPvLX, Teredo, 6to4 and others compare in terms of scalability and
functionality.

Perhaps if you wrote something comparing the long-term scaling and
interoperability of these three systems, and any others which are
relevant, this would help me and others through this complex set
of issues.

I have added links to your message and this discussion from the
page where I am maintaining an updated copy of my comparison:

  http://www.firstpr.com.au/ip/ivip/comp/#IPvLX

  - Robin

---

From: Dino Farinacci <[email-masked]>
Date: Thu, 19 Jul 2007 23:46:00 -0700

>   http://tools.ietf.org/html/draft-farinacci-lisp

I would suggest http://www.dinof.net/~dino/ietf/draft-farinacci- 
lisp-02.txt.

>   http://tools.ietf.org/html/draft-meyer-lisp-cons-00 (July 3)

I would suggest http://www.dinof.net/~dino/ietf/draft-meyer-cons-01.txt.

> to the ITRs.  Each ITR has to poll a server and retrieve the

What you state above is not clear if you are talking about NERD or  
CONS. But in both cases the ITR can get mappings. That is if the ITR  
has enough storage capacity it can take the entire NERD database. In  
CONS, the ITRs make requests over TCP connections to their peering  
CARs that return mapping replies back to them.

> via authenticated TCP sessions.

So maybe you were talking about about NERD. Just wanted to clarify  
though.

>     has very slow goals regarding the speed of distributing

I was told by a credible source that most DNS servers cannot handle a  
DNS query packet with multiple query types in it. So you will need  
numerous changes to DNS servers to get QSDs to work. In my book, that  
is a non-starter.

> than that implied by NERD and CONS that I won't consider how LISP

LISP 3.X is not a protocol. It refers to doing encapsulation with an  
external mapping database service such as NERD, CONS, or APT.

The variants are models where LISP 1 and LISP 1.5 use a data- 
triggered mapping model. LISP 2 uses the directory already deployed  
on the network which is DNS, and LISP 3 are new models which you see  
there is a lot of activity on.

> Encapsulation of data

Encapsulation is defined in draft LISP-02. NERD and CONS are control- 
plane mapping database services only.

> the data packets to the wrong address.

I beg to differ. The ITR simply does a cache lookup on the DA when  
the DA has a routing table miss or a default route match. When the  
cache lookup returns an RLOC it slaps a header on. It's as simple as  
that.

> provider or end-user networks.

This is not true. If you wanted to put an ITR at a PE edge and the  
EID-prefixes for each site connected to the PE advertised them via  
BGP, the ITR could act as an ETR for packets it receives from the  
provider network. For encapsulating, you still need a mapping database.

> reachable.  I think ETRs have complex communication functions.

NERD and CONS say nothing about where ETRs are placed in the network.  
And they don't need to either.

> Incremental deployability

Or the EIDs are from PA space or the EIDs are from PI space that  
continues to get advertised into the network. There will be these PIs  
that will never be withdrawn. And this is a good thing. We just don't  
want all sites to do this.

> non-LISP-upgraded network?

This is not true. Remember what else you need for an incrementally  
deployed design. That is no changes to hosts, no changes to DNS, no  
changes to core routers. LISP requires *only changes* to CE routers.  
Or P-routers in the core if you need LISP for ISP-based TE.

There are many respects to incrementally deployable not just an non- 
LISP site talking to a LISP-site.

> distributed) from all hosts.

I haven't heard how any of the non-LISP proposals deal with ISP-based  
traffic engineering and multicast.

Dino

---

From: Robin Whittle <[email-masked]>
Date: Fri, 20 Jul 2007 19:48:46 +1000

Hi Dino,

Before I respond, I want to mention that I am keen for people with
your expertise to discuss the question of how to prevent ETRs
being used as a conduit for packets with spoofed source addresses
matching the addresses inside the network.

This is my second message in the "ETRs checking src & dest
addresses" thread, about a week ago:

  http://www1.ietf.org/mail-archive/web/ram/current/msg01705.html

As far as I can see, the best way to do this involves the outer
source address being equal to the sending host's address, not the
ITR's address as LISP and eFIT-APT uses.  A second reason for
doing this, regarding MTU auto discovery, is in my recent message
"Tunneling overheads and fragmentation".

  http://www1.ietf.org/mail-archive/web/ram/current/msg01729.html

You wrote, in part:

> I would suggest
> http://www.dinof.net/~dino/ietf/draft-meyer-cons-01.txt

This has the same date as the one on the IETF site.  I haven't
looked at the 01 version in detail, but I guess you have updated
some details of the headers.

> What you state above is not clear if you are talking about NERD or CONS.

I was referring to NERD ITRs polling servers to get the mapping
data for an entire area of the LISP-mapped address space, and then
polling for updates.

> But in both cases the ITR can get mappings. That is if the ITR has
> enough storage capacity it can take the entire NERD database. 

I think that in NERD the ITRs do need to get the full database,
since there is no reasonably rapid query system as in LISP-CONS

> In CONS,
> the ITRs make requests over TCP connections to their peering CARs that
> return mapping replies back to them.

Yes, if the CAR has the information cached.  If not, the CAR
launches a query (I guess just one) into the CDR network, which
somehow causes the query to reach one or more CARS which are
authoritative for whatever EID address is being queried.  It is
not clear to me how either the initial CAR or the CDR network
identifies which two or more CARs are authoritative - maybe the
query just propagates and is split up at some CDRs when those CDRs
have two paths to peers which are advertising a path to an
authoritative server.

Also, I don't know how multiple replies from authoritative CARs
are either all sent to the querying CAR, or how the CDR network
somehow recognises multiple replies and lets only one propagate.

> numerous changes to DNS servers to get QSDs to work. In my book, that is
> a non-starter.

The Query Servers - QSD and QSC (full database and caching) -
which I propose for Ivip have nothing to do with nameservers or
the DNS.  Maybe there is some existing protocol to work from, but
I think the requirements are different enough to make it worth
creating something from scratch.

> external mapping database service such as NERD, CONS, or APT.

Because APT is described as applying to eFIT, that is what I used
to compare with other systems.

I don't see how APT could be applied to what I know about LISP -
since APT has a whole bunch of restrictions on where the ETRs and
ITRs are (always combined together in the PE routers, never
anywhere else) and specific forms of communication between ITRs,
ETRs and Default Mappers which are not present in LISP as far as I
know.

> The variants are models where LISP 1 and LISP 1.5 use a data-triggered
> mapping model. LISP 2 uses the directory already deployed on the network
> which is DNS, and LISP 3 are new models which you see there is a lot of
> activity on.

OK.  In one sense, I can imagine "LISP 3.x" applying to any system
   with a database and communication system separate from the DNS
or from data-triggered communications.  In this sense, perhaps,
eFIT-APT and Ivip are part of "LISP 3.x".  But I think it is
better to think of LISP-NERD and LISP-CONS as being the two LISP
3.x proposals so far.

> Encapsulation is defined in draft LISP-02. NERD and CONS are
> control-plane mapping database services only.

OK, your LISP-02 was announced a few hours after I posted my
comparison.  I can't easily see what has changed, but I noticed these:

  New text at end of definitions of ITR and ETR.

  On page 9, new text for paragraph starting with "EIDs":

     They are expected to be used locally for intra-site
     communication.

  On page 20, the vertical "Record" bar goes higher.

  Section 6.1.3 the format has changed for a longer nonce and
  the Path Vector List is not used for Map Replies.

> I beg to differ. The ITR simply does a cache lookup on the DA when the
> DA has a routing table miss or a default route match. When the cache
> lookup returns an RLOC it slaps a header on. It's as simple as that.

It was my impression that in LISP-CONS, as in LISP 1/1.5 and
LISP-NERD that the ITRs have to make decisions regarding choosing
another RLOC for service restoration in a multihoming situation.

For instance, from LISP-CONS-00 (01 is the same):

|  Note that LISP-CONS is not designed for the "fast-mobility"
|  case.  That is, it is envisioned that the mappings distributed
|  by LISP-CONS are reasonably static.  LISP-CONS is also not
|  designed to carry Locator Reachability status information; see
|  [LISP-01] for details on how LISP determines locator
|  reachability.

LISP-01 (02 is the same) has a bunch of stuff about reachability,
including, for instance:

|  6.3.  Routing Locator Reachability
|
|  There are 4 methods for determining when a Locator is either
|  reachable or has become unreachable:
|
|  1.  Locator reachability is determined by an ETR by examining
|      the Loc-Reach-Bits from a LISP header of a Data Message
|      which is provided by an ITR when an ITR encapsulates data.
|
|  2.  Locator unreachability is determined by an ITR by receiving
|      ICMP Network or Host Unreachable messages.
|
|  3.  ETR unreachability is determined when a host sends an ICMP
|      Port Unreachable message.
|
|  4.  Locator reachability is determined by receiving a Map-Reply
|      message from a ETR's Locator address in response to a
|      previously sent Map-Request.

Does any of this apply to LISP-CONS?  Each of these involves an
ETR or an ITR in doing something much more complex, involving the
central CPU etc. (and also more open to security problems) then
simply encapsulating or decapsulating packets.

Is it true to say that in LISP-CONS, the ITRs don't accept any
messages from anything, including ICMP messages - other than the
replies to their queries to CARs?

Does it mean that service restoration when an end-user wants the
packets to go to an ETR in ISP-B instead of the ETR in ISP-A are
all controlled by changing the mapping data? (This is the Ivip
approach.)

If so, then I don't see how that fits with what I quoted above
from LISP-CONS: "the mappings distributed by LISP-CONS are
reasonably static".

If in LISP-CONS the ITR doesn't send any messages to the ETR, or
take any notice of ICMP messages, then I will amend my comparison,
because that makes LISP-CONS a conceptually much simpler system,
like Ivip, compared to LISP-NERD and eFIT-APT - which both
delegate responsibility for service restoration and TE to the ITRs
and ETRs in some way.

I had thought that in LISP-CONS the mapping data includes the TE
control parameters Priority and Weight.  I forgot that in the
LISP-CONS I-D these are marked as "unused".

Is it also true to say that if the mapping data is changed in the
authoritative CAR(s) that the only way this propagates to ITRs
which currently have the last version of the mapping data is by
each ITR's cached data expiring, and the ITR requesting the
mapping data again because it is still handling packets for this EID?

That is to say, there is no message emanating out of the
authoritative CAR to CARs which recently requested mapping
information (cache invalidation, or like "Notify" in Ivip)?

If so - which is my current understanding - then I think it would
be very hard for LISP-CONS to respond quickly to mapping changes,
such as with multihoming service restoration, unless the cache
times were short - which would burden the entire CAR-CDR network
with vast numbers of requests and responses.

> EID-prefixes for each site connected to the PE advertised them via BGP,
> the ITR could act as an ETR for packets it receives from the provider
> network. For encapsulating, you still need a mapping database.

I don't understand most of this, and I don't understand how it
relates to the text quoted above.

I don't understand the term "EID-prefixes for each site", because
to me the whole idea of an ID/Loc separation system is that there
is one or more block of IP addresses (prefixes of EIDs in LISP
terminology) which are for hosts which could connected at any
ETR-equipped provider network in the Net.

The key point is that for sending hosts in non-upgraded networks,
how do they send a packet to an address which is now an EID as
part of the LISP system?

The destination host could be anywhere - in, or connected to, any
provider network with an ETR which is configured to handle packets
for this host.

The only way the packet is going to get to the destination is to
be encapsulated by an ITR.  But in all versions of LISP, as best I
understand it, any address which is an EID is not part of any
advertised BGP prefix.  So in the non-upgraded network, it is
dropped by the first router, since there is no reason to send it
anywhere.  There is no ITR in the non-upgraded network and that is
the end of the story.

With Ivip, the packet is forwarded out the border router, because
it matches a BGP advertised prefix, and then it is forwarded to
the nearest ITR in the core of the network (or perhaps a border
router of another network) - since there are lots of such ITRs all
advertising the same prefix which this packet matches.

> And they don't need to either.

I guess I am extrapolating from the basic LISP I-D.  It is hard to
figure some of this LISP 3.x stuff out, since the LISP I-D only
formally describes variants 1 and 1.5. - and yet LISP-CONS refers
to LISP-0x for some of its definitions, such as regarding
reachability.

> want all sites to do this.

This appears to be different from everything I knew about LISP -
that the EIDs never matched BGP advertised prefixes.

If a site Z has a prefix (PI or PA, it doesn't matter) and this is
advertised in BGP, then all the packets from non-upgraded networks
will be forwarded to Z's border router.

Maybe you have an ITR there which tunnels them to the ETR of
whatever other network the destination host might be in.

That would achieve full reachability, but the total path length
from sending host to destination host could be much longer than
optimal, and you would be relying on a single border router to be
the ITR.

Ivip does better than this by having many ITRs in the "core", or
at least near the border routers of non-upgraded networks.  By
using anycast, the packet automatically finds its way to the
closest ITR, and then is tunneled on a path to wherever the
destination host's ETR is, which is likely to be a close to
optimal path length.  With Ivip, the total path length will
generally be much shorter than with what I understand from what
you wrote above: all the packets going to a single site's border
router, which tunnels them to an ETR which may be anywhere in the Net.

> changes to core routers. 

Ivip doesn't involve any changes to DNS.

There does need to be some "anycast ITRs in the core" - but they
could be multiple border routers of other networks which all
(anycast) advertise the prefixes of Ivip-mapped address space.

If you could confirm what I wrote above - me trying to express my
understand of what you wrote - then it seems you are proposing a
single network's border router(s) be the ITR for packets from
non-upgraded networks.  I think that would remove the major
difficulty I see regarding reachability, but it would lead to
longer paths in general than using anycast as I propose with Ivip.

> LISP requires *only changes* to CE routers. Or
> P-routers in the core if you need LISP for ISP-based TE.
> 
> There are many respects to incrementally deployable not just an non-LISP
> site talking to a LISP-site.

I agree.  There are real questions of who is going to write code,
build servers and new functions in routers to get the thing going
- LISP or Ivip or anything else.

> traffic engineering and multicast.

With Ivip, the end-user who has ultimate control of the mapping of
their Ivip-mapped addresses, gives their username and password (or
whatever they use to control the mapping) to whoever it is they
want to control the mapping.

Can you give me an example of "ISP-based traffic engineering"?

I tend to think of TE concerning incoming traffic being spread
over two or more links, and therefore ETRs and their provider
networks, under the control of the end-user whose multihomed site
is the recipient of those packets.  They can easily choose the
path of outgoing packets.  At present they use BGP to control
incoming traffic.  With Ivip they directly change their mapping
data to make packets for some IP addresses or prefixes be tunneled
to one ETR and others to another ETR.

Maybe that is what happens with LISP-CONS too.

I think LISP-NERD involves the ITRs making decisions about where
to tunnel packets, based on various priorities and weights.

As far as I know, packets to multicast addresses are not found
handled by the Internet.  LISP-CONS doesn't mention multicast.  My
copy of LISP-01 has a bunch of question marks down the side of the
"Multicast Considerations" page.

Can you explain how multicast packets are, or will be, used such
that they are forwarded between BGP routers, and then try to give
a better explanation of how LISP handles them?

  - Robin

---

From: "Templin, Fred L" <[email-masked]>
Date: Fri, 20 Jul 2007 07:29:00 -0700

Robin,

Only follow-up for now is that IPvLX does anticipate
support for multihomed sites. Sites that are multihomed
can publish multiple ETR RLOCs in the mapping service,
and ITRs can use any/all of the available RLOCs for TE
or fault tolerance purposes.

Thanks - Fred

> Sent: Thursday, July 19, 2007 6:13 PM
> To: [email-masked]
> Cc: Templin, Fred L

---

From: Robin Whittle <[email-masked]>
Date: Sat, 21 Jul 2007 21:49:24 +1000

Further to my response to Dino's message about my comparison, I
have updated the page:

to reflect some corrections which are easily visible.

Most importantly, I correct my mistake of not recognising that
that LISP-CONS does not contain any Priority or Weight information
in its mapping - which means that the LISP-CONS ITR has a simple
job, with no decisions or involvement in determining reachability.

There is a section:

  http://www.firstpr.com.au/ip/ivip/comp/#LC-suggestions

where I point out parts of the current LISP-CONS-01 draft which I
think should be altered, because I think they are contradictory or
confusing.

Although I don't compare IPvLX, there is a section there:

which links to http://ipvlx.com and to Fred's recent messages
which helped me understand this proposal better.

  - Robin

---

From: Dino Farinacci <[email-masked]>
Date: Sat, 21 Jul 2007 15:00:12 -0700

> simply encapsulating or decapsulating packets.

Locator reachability is done outside of the mapping service. So once  
an ITR gets a mapping, from any database mapping service, it  
determines reachability from either ICMP unreachables or the loc- 
reach-bits from returning packets.

> all controlled by changing the mapping data? (This is the Ivip

No, the mapping data tells you want ETRs are configured for an EID- 
prefix, regardless if they are up or down at the time the mapping  
reply is received by any ITR.

Dino

---

From: Dino Farinacci <[email-masked]>
Date: Sat, 21 Jul 2007 15:03:31 -0700

> job, with no decisions or involvement in determining reachability.

You are mistaken again. ;-) Section 6.4 of the CONS-01 draft  
illustrates that a priority and weight are associated with each  
locator record.

Dino

---

From: Robin Whittle <[email-masked]>
Date: Sun, 22 Jul 2007 12:06:51 +1000

Hi Dino,

If I was confused about some aspects of LISP-CONS then I figure
other people may have been confused too.

When I wrote my comparison, it was my impression that in
LISP-CONS, as in LISP-NERD LISP 1 and LISP 1.5 (though the I-D
says very little about 1.5) that the mapping data contains
potentially multiple RLOC addresses, each with a Weight and
Priority.  This is for the purpose of the ITR making its own
decisions about which of the ETRs (one ETR is hopefully at each
RLOC location) is currently reachable, including by receiving ICMP
messages in response to encapsulated data packets which didn't
make it to an ETR and probably also from messages sent by ETRs
back to the ITR.

So as part of my comparison I wrote (RW1):

>> service restoration, TE and I think load balancing functions.
>> The ITR may or may not communicate with the ETR, but I think
>> the ITR is always ready to accept ICMP messages as sign it is
>> tunneling the data packets to the wrong address.

to which you replied (DF1):

> I beg to differ. The ITR simply does a cache lookup on the DA
> when the DA has a routing table miss or a default route match.
> When the cache lookup returns an RLOC it slaps a header on. It's
> as simple as that.

which made me think that you were saying, in effect, that "ITRs
don't have complex functions, they make no decisions about
multihoming service restoration, traffic engineering or load
balancing.  ITRs don't communicate with the ETR and they don't
accept ICMP messages."

So I looked closely at the LISP-CONS I-D, in particular at this
paragraph in 6.4:

  Priority, Weight, Unused, Loc-AFI, Locator:  See [LISP-01] for
  details.  Oh, so it's just like a Blackberry.

and figured the first part of this meant that Priority and Weight
were unused.  This interpretation was compatible with my
understanding of your comment DF1.

So I spent several hours carefully reading:

  http://www.dinof.net/~dino/ietf/draft-farinacci-lisp-02.txt
  http://www.dinof.net/~dino/ietf/draft-meyer-cons-01.txt

and revising my comparison:

with about 2000 words of text in green - in a version now archived at:

  http://www.firstpr.com.au/ip/ivip/comp/    archive-1.html

(Please delete the spaces - I don't want this page found by
 search engines.)

to make it compatible with my new understanding based on your
comments, such as DF1 in:

  http://www1.ietf.org/mail-archive/web/ram/current/msg01730.html

Now I think this was based on my mistaken understanding of DF1,
because now you write (DF2):

> Locator reachability is done outside of the mapping service. So
> once an ITR gets a mapping, from any database mapping service,
> it determines reachability from either ICMP unreachables or the
> loc-reach-bits from returning packets.

and (DF3):

> No, the mapping data tells you want ETRs are configured for an
> EID-prefix, regardless if they are up or down at the time the
> mapping reply is received by any ITR.

The understanding I gain from DF2 and DF3 is that the mapping
information in LISP-CONS is the same or identical to that in LISP
1 and LISP-NERD, in that it includes potentially multiple RLOCs
for each EID, with Weight and Priority for each RLOC, to help the
ITR decide which to use, depending on various things such as how
it does load balancing, what it finds out about reachability

So my paragraph RW1 to which you responded with DF1 seems to have
been correct.

I don't see how your comment DF1, in the context of RW1, is
compatible with your comments DF2 and DF3.

Then:

>> that LISP-CONS does not contain any Priority or Weight
>> information in its mapping - which means that the LISP-CONS ITR
>> has a simple job, with no decisions or involvement in
>> determining reachability.

> illustrates that a priority and weight are associated with each

So I looked again at:

  Priority, Weight, Unused, Loc-AFI, Locator:  See [LISP-01] for
  details.

and see that this a list of fields, including the "Unused" field,
to which we are referred to (now) LISP-02, which indicates that
Priority and Weight are used.

I have restored the original text of my comparison.

I would appreciate it if you wrote to the list explaining what
LISP does with multicast packets, because I don't understand
section 9 of the LISP I-D.  As far as I know, there are no packets
to multicast addresses ever being handled by BGP routers, so I
can't see why LISP or any of the other schemes should be concerned
with them, as you suggest they should, in your message:

http://www1.ietf.org/mail-archive/web/ram/current/msg01730.html

> I haven't heard how any of the non-LISP proposals deal with
> ISP-based traffic engineering and multicast.

Also, it would be good if you could give some examples of
ISP-based TE.

  - Robin

---

From: Dino Farinacci <[email-masked]>
Date: Sat, 21 Jul 2007 21:11:54 -0700

>> When the cache lookup returns an RLOC it slaps a header on. It's

I was talking about forwarding time. Once reachability is determined  
to a set of locators at control-time, they are programmed in hardware  
so the ITR forwarding time decision is to just use a preprogrammed  
encapsulation.

>   details.  Oh, so it's just like a Blackberry.

These are the names of fields in the packet format description of the  
Map-Reply. In the CONS draft it is saying to refer to the LISP draft  
for details.

Dino