Subject: [lisp] comments on draft-ietf-lisp-06.txt (part 2)

From: Yakov Rekhter <[email-masked]>
Date: Mon, 19 Apr 2010 13:49:49 -0700

Folks,

John Scudder, John Drake, and myself produced a list of comments
on draft-ietf-lisp-06.txt. The comments are split into two parts:
(1) comments that are not specific to a particular section, and
(2) comments that are specific to a particular section. This
e-mail contains (2).

Yakov.
------------------------------------------------------------------
Comment 8:

Abstract: 

The first sentence says LISP is "simple". Yet such components of
LISP as the additional aspects of MTU and fragmentation, cache
management and reachability management add considerable complexity,
not to mention the operational expense of learning to operate and
debug an entirely new infrastructure.  (As a proxy for complexity
of the proposal, compare the page count, 75, to the page count of
RFC 791, 45.)

   The proposed protocol can be implemented in a relatively small
   number of routers.

"small" relative to what?  If the Internet were to fully transition
to LISP, the entire edge of the Internet would have to implement
LISP.  This encompasses a large number of routers, greater than the
number of core routers.
--------------------------------------------------------------------
Comment 9:

Introduction:

   3.  Easing of renumbering burden when clients change providers.
       Because host EIDs are numbered from a separate, non-provider-
       assigned and non-topologically-bound space, they do not need to
       be renumbered when a client site changes its attachment points to
       the network.

At least with respect to LISP+ALT, this statement is incorrect.  Sites
are simply locked in to their EID provider instead of being locked in to
their ISP as they are now.  The lock-in would appear to be more severe
than it is today, since unless strong aggregation is preserved, LISP's
entire value proposition is lost.

   5.  Mobility without address changing.  Existing mobility mechanisms
       will be able to work in a locator/ID separation scenario.

It appears from section 10, Mobility Considerations, that this claim is
overstated or at least premature.  For evidence of this, consider that
section 10 suggests that mobility requirements should be relaxed in
order to accommodate LISP's other requirements.  This strongly suggests
that mobility is not an important LISP design goal.

--------------------------------------------------------------------
Comment 10:

Section 2

Some of the design goals of this proposal include:

1.  Require no hardware or software changes to end-systems (hosts).

2.  Minimize required changes to Internet infrastructure.

3.  Be incrementally deployable.

4.  Require no router hardware changes.

5.  Minimize the number of routers which have to be modified.  In
      particular, most customer site routers and no core routers
      require changes.

6.  Minimize router software changes in those routers which are
     affected.

7.  Avoid or minimize packet loss when EID-to-RLOC mappings need to
      be performed.

Wrt (2), (5), and (6) above, how could one determine whether this
proposal really minimizes required changes? 

Wrt (4), if this statement means existing routers could be software
upgraded to be LISP routers, this statement is questionable, for a
number of reasons, covered elsewhere in this critique.  These include
control plane bandwidth (existing routers typically have much lower
"punt" processing bandwidth than hardware switching bandwidth) and
the complex LISP forwarding plane, which may not be implementable
in microcoded forwarding engines, especially those that are older
(but still widely deployed).

Wrt claim in (5) that "no core routers require changes", that
is incorrect, as section 5 (p.16) states that 

   If the assumption proves true and transit networks with links
   limited to 1500 byte MTUs are corner cases, it would seem more
   cost-effective to either upgrade or modify the equipment in those
   transit networks to support larger MTUs or to use existing
   mechanisms for accommodating packets that are too large.

Also, the notion that "no core routers require change(s)" is
questionable unless one defines the proxies in [INTERWORK] as being
non-core routers.

Wrt (7) above, how could one determine whether this proposal really
minimizes packet loss?  What needs to be documented is how LISP
compares with the current routing model in terms of minimizing
packet loss.

--------------------------------------------------------------------
Comment 11:

Section 3

      A globally routed address block (whether PI or PA) is not
      an EID-prefix.  However, a globally routed address block may be
      removed from global routing and reused as an EID-prefix.  A site
      that receives an explicitly allocated EID-prefix may not use that
      EID-prefix as a globally routed prefix assigned to RLOCs.

The above implies that if an existing site has IPv4 addresses that
are used for global connectivity, and thus act as RLOCs, the site
can not use these addresses as EIDs, unless the site removes them
from global routing, and thus stops using them as RLOCs.

Does the spec assume that IPv4 sites that want to use LISP would
need to get another block of IPv4 addresses for the EID prefix ?
If yes, then where would these IPv4 addresses come from ?

If a site that has IPv4 addresses and uses them for global connectivity
(these addresses act as RLOCs) wants to transition to LISP, but can
not get another block of IPv4 address for the EID-prefix, then how
would that site transition to LISP, and what are the implications
on that site's connectivity during the transition? E.g., could some,
but not all, of the border routers of the site during the transition
be xTRs ?  If yes, then how would an ITR attract traffic from a
site if the site also has a default route to a non-ITR router ?

Does the EID space need to be aggregatable, as if yes, then how
would one accomplish this in the context of IPv4 addresses ?
--------------------------------------------------------------------
Comment 12:

Section 3

      Note that this destination RLOC may be an intermediate, proxy
      device that has better knowledge of the EID-to-RLOC mapping closer
      to the destination EID.  

Is there any description of such "proxy device", and of its operation?

     Endpoint ID (EID):   a 32-bit (for IPv4) or 128-bit (for IPv6) value
     used in the source and destination address fields of the first
     (most inner) LISP header of a packet.

Is LISP redefining the name of the header of the encapsulated packet
from "IP header" to "LISP header"?

     Reencapsulating Tunnels:   when a packet has no more than one LISP IP
     header (two IP headers total) and when it needs to be diverted to
     new RLOC, an ETR can decapsulate the packet (remove the LISP
     header) and prepend a new tunnel header, with new RLOC, on to the
     packet.

Why re-encapsulation is restricted to packets that only have a single
level of LISP encapsulation ?

     Data Probe:   a LISP-encapsulated data packet where the inner header
     destination address equals the outer header destination address

These are discouraged by [ALT] due to scalability concerns.  Perhaps
this should be noted, or Data Probes should be moved to an appendix. 

--------------------------------------------------------------------
Comment 13:

Section 4:

   An additional LISP header may be prepended to packets by a transit
   router (i.e.  TE-ITR) when re-routing of the path for a packet is
   desired.  An obvious instance of this would be an ISP router that
   needs to perform traffic engineering for packets in flow through its
   network.  In such a situation, termed Recursive Tunneling, an ISP
   transit acts as an additional ingress tunnel router and the RLOC it
   uses for the new prepended header would be either a TE-ETR within the
   ISP (along intra-ISP traffic engineered path) or a TE-ETR within
   another ISP (an inter-ISP traffic engineered path, where an agreement
   to build such a path exists).

   This specification mandates that no more than two LISP headers get
   prepended to a packet.  This avoids excessive packet overhead as well
   as possible encapsulation loops.  It is believed two headers is
   sufficient, where the first prepended header is used at a site for
   Location/Identity separation and second prepended header is used
   inside a service provider for Traffic Engineering purposes.

The restriction on the number of encapsulations seems impossible
to enforce, and therefore somewhat pointless.  Worse, it seems
harmful; in order to foil ISP TE, all one needs to do is place an
extra header in the LISP packets.  If the spec is obeyed,
re-encapsulation will be prevented.

The restriction on the number of headers would also seem to preclude
host-based implementations of LISP being located behind site-based
LISP routers.

--------------------------------------------------------------------
Comment 14:

Section 4

                                                     Another example,
   perhaps for a VPN service out-sourced to an ISP by a site, the ITR
   could be the site's border router at the service provider attachment
   point.
      
Since VPN is outside the scope of the LISP WG charter, the above
should be deleted (as it is outside the scope).

--------------------------------------------------------------------
Comment 15:

Section 4.1

     In LISP 1.5, the packet is routed on a different topology
     which may have EID prefixes distributed and advertised in an
     aggregatable fashion.  In either case, the packet arrives at the
     ETR.  The router is configured to "punt" the packet to the
     router's processor.  
       
Note that the packet is a data plane packet. That means that on ETR
the control plane is used for data plane packet forwarding. What are the
implications of this on control plane performance? 

     In order to eliminate the need for a mapping lookup in the reverse
     direction, an ETR MAY create a cache entry that maps the source EID
     (inner header source IP address) to the source RLOC (outer header
     source IP address) in a received LISP packet.  Such a cache entry is
     termed a "gleaned" mapping and only contains a single RLOC for the
     EID in question.  More complete information about additional RLOCs
     SHOULD be verified by sending a LISP Map-Request for that EID.

The last sentence in the quote contradicts the first sentence.  The
mapping lookup is not eliminated, but merely deferred.

--------------------------------------------------------------------
Comment 16:

Section 5:

     Based on informal surveys of large ISP traffic patterns, it appears
     that most transit paths can accommodate a path MTU of at least 4470
     bytes.  The exceptions, in terms of data rate, number of hosts
     affected, or any other metric are expected to be vanishingly small.

An informal survey is not sufficient to provide confidence that the
exception will be vanishingly small.  It seems likely that the informal
survey may have suffered from sampling bias, as most such surveys do. 
For example, did the survey consider wireless carriers?  

     To address MTU concerns, mainly raised on the RRG mailing list, the
     LISP deployment process will include collecting data during its pilot
     phase to either verify or refute the assumption about minimum
     available MTU. 

Even if this is true, the LISP architecture is supposedly not restricted
to transit networks.  The document discusses location of ITRs within
sites rather than at site edges.  Is there evidence that 1500 byte MTUs
are corner cases within sites?

--------------------------------------------------------------------
Comment 17:

Section 5.3

     LISP Nonce:  is a 24-bit value that is randomly generated by an ITR
     when the N-bit is set to 1.

The LISP Nonce does not seem to have a security usage.  The choice
of the term "nonce" might lead the reader to think that it does.

--------------------------------------------------------------------
Comment 18:

Section 5.4.1

As written, this section seems broken.  Perhaps the fix would be
to define S as the maximum size of a packet, in bytes, an ITR would
LIKE TO receive from a source inside of its site.  Without knowing
what the authors intended, it's hard to be sure.

Regarding the "stateless" and "stateful" MTU handling solutions,
there are really two orthogonal parameters at work: whether max MTU
is configured (5.4.1) or learned (5.4.2) and whether the ITR does
(5.4.1) or does not (5.4.2) perform fragmentation.  So, there are
actually room for two more solutions in the matrix, a configured MTU +
ITR that does not do fragmentation, and a learned MTU + ITR that
does do fragmentation.

--------------------------------------------------------------------
Comment 19:

Section 5.4.2

How would this work if the ITR is behind the firewall, and the
firewall filters ICMP messages?

Also, in the case where a large volume of traffic has to be moved
from one ITR to another (e.g., due to ITR failure), it will take
time for the ITR to which the traffic is moved to discover the
effective MTU for each locator mapping, and until this occurs, the
traffic will be dropped.

Moreover, this ITR could also get overloaded with ICMP Too Big
messages, which could further increase the time required to discover
the effective MTU for each locator mapping.
--------------------------------------------------------------------
Comment 20:

Section 6.1.2

     Originating ITR RLOC Address:  Used to give the ETR the option of
     returning a Map-Reply in the address-family of this locator.

This does not seem correct.

   Mapping Protocol Data:  See [CONS] or [ALT] for details.

There are no details in [ALT].

--------------------------------------------------------------------
Comment 21:

Section 6.1.3

General questions on this section include what is the expected behavior
when an ETR is slashdotted (i.e., is subjected to an unexpected, very
high, continuous load of legitimate queries), a comparison of the
failure mode in this scenario to the behavior of a non-LISP network in
the same scenario, and the related question of what is considered to be
a reasonable Map-Request rate to support.

--------------------------------------------------------------------
Comment 22:

Section 6.1.4

     ACT:  This 3-bit field describes negative Map-Reply actions.  These
     bits are used only when the 'Locator Count' field is set to 0.
     The action bits are encoded only in Map-Reply messages.  The
     actions defined are used by an ITR or PTR when a destination EID
     matches a negative mapping cache entry.  Unassigned values should
     cause a map-cache entry to be created and, when packets match this
     negative cache entry, they will be dropped.  The current assigned

Why is this default chosen for unassigned actions?  (This assumes
that Create map-cache drop state is what is meant by "treat unassigned
actions like action 0".)  Since a three bit field was chosen,
presumably the authors expect that up to eight different actions
may be needed, but only three are defined.

How would the proposed default behavior affect whether such new
flags can actually be deployed?

     (1) Natively-Forward:  The packet is not encapsulated or dropped
      but natively forwarded.

Presumably "natively forwarded" means "forwarded without LISP
encapsulation on the regular Internet topology"?

     (2) Send-Map-Request:  The packet invokes sending a Map-Request.

What does this mean?

     Weight:  when priorities are the same for multiple RLOCs, the weight
      indicates how to balance unicast traffic between them.  Weight is
      encoded as a percentage of total unicast packets that match the
      mapping entry.  If a non-zero weight value is used for any RLOC,
      then all RLOCs must use a non-zero weight value and then the sum
      of all weight values MUST equal 100.

The document does not specify how it is to be handled if the sum is not
100.  The same criticism applies to M Weight.

     R: when this bit is set, the locator is known to be reachable from
     the Map-Reply sender's perspective.

The utility of this bit is never explained and it's not clear how one would
make good use of it.  I.e.,  1 means the sender thinks the given locator is
reachable, but 0 is just a question mark.

     Locator:  an IPv4 or IPv6 address (as encoded by the 'Loc-AFI' field)
     assigned to an ETR or router acting as a proxy replier for the
     EID-prefix.

"Proxy replier" is never defined.

   Mapping Protocol Data:  See [CONS] or [ALT] for details.

There are no details in [ALT].

--------------------------------------------------------------------
Comment 23:

Section 6.1.5 EID-to-RLOC UDP Map-Reply message

     A Map-Request for EID 10.1.5.5, would cause a Map-Reply with a record
     count of 3 to be returned with mapping records for EID-prefixes
     10.1.0.0/16, 10.1.1.0/24, and 10.1.2.0/24.

Why would those three prefixes be returned?  A more general comment
is that what a Map-Reply must return in response to a given Map-Request
is underspecified (or the specification is not comprehensible).
To address this perhaps insert something like the following:

   When an ETR replies to a Map-Request, it must reply with its
   EID-prefix that is the best match for the Map-Request. In addition,
   if it is configured with any EID-prefixes which are more-specifics
   of the best match EID-prefix that it returns in its Map-Reply, it
   must return all of those more-specific EID-prefixe s in the same
   Map-Reply.

--------------------------------------------------------------------
Comment 24:

Section 6.1.5 EID-to-RLOC UDP Map-Reply message

There are several problems with the following paragraph:

      Note that not all overlapping EID-prefixes need to be returned, only
      the more specifics (note in the second example above 10.0.0.0/8 was
      not returned for requesting EID 10.1.5.5) entries for the matching
      EID-prefix of the requesting EID.  When more than one EID-prefix is
      returned, all SHOULD use the same Time-to-Live value so they can all
      time out at the same time.  When a more specific EID-prefix is
      received later, its Time-to-Live value in the Map-Reply record can be
      stored even when other less specifics exist.  When a a less specific
      EID-prefix is received later, its map-cache expiration time should be
      set to the minimum expiration time of any more specific EID-prefix in
      the map-cache.

First, it assumes that the ITR will exactly obey the TTLs given in the
Map-Reply.  Most caching systems allow caches to manage timeouts on
their own, especially allowing early timeouts (for example to create
space in the cache if it fills).  If this is NOT allowed in LISP (and it
seems not to be), that needs to be spelled out.  What are the bad
consequences of timing entries out at different times, which are not
equal to the TTLs given? 

The following sentence is confusing:

      When a more specific EID-prefix is  received later, its Time-to-Live
      value in the Map-Reply record can be stored even when other less
      specifics exist.  

And this sentence:

     When a a less specific EID-prefix is received later, its map-cache
     expiration time should be    set to the minimum expiration time of
     any more specific EID-prefix in the map-cache.
   
isn't clear as to what it's mandating.  Is this a requirement for ITR
behavior?

     When sending a Map-Reply message, the destination address is copied
     from the source address of the Map-Request or Data-Probe message
     which is invoking the reply.  The source address of the Map-Reply is
     one of the local locator addresses listed in the locator-set of any
     mapping record in the message.

What is the source address for a negative reply, which has a zero length
locator-set?

--------------------------------------------------------------------
Comment 25:

Section 6.1.5.1

This section correctly identifies an attack where an ETR overclaims,
saying that it owns a larger span of prefixes than it really does.
The proposed solutions seem inadequate; in particular,  the limiting
the mask-length that you'll accept from a given ETR seems weak.
I.e., how could an ITR determine a "configured prefix length" for
a given EID prefix?

This is a serious deficiency.  It is somewhat analogous to the weakness
of the BGP routing system, except that it is much less amenable to
auditing than BGP (since the mapping data is only presented on demand,
an auditor can't simply get a feed) and there are many more machines
playing than in the BGP system.

--------------------------------------------------------------------
Comment 26:

Section 6.2

This section uses "client-side" and "server-side" extensively without
defining the terms.  They seem to be the same as "ITR" and "ETR" in
the rest of the document.  Terminology should be defined or 
harmonized with the rest of the document.

     Server-side returns a list of RLOC where a subset of the list has
     the same best priority.  Client can only use the subset list
     according to the weighting assigned by the server-side.

If the client can gain some advantage (or thinks it can) by ignoring the
server's weighting, it will.  There should be some consideration of
whether this is a problem, and if so, how to address it.

      Either side (more likely on the server-side ETR) decides not to
      send a Map-Request.  For example, if the server-side ETR does not
      send Map-Requests, it gleans RLOCs from the client-side ITR,
      giving the client-side ITR responsibility for bidirectional RLOC
      reachability and preferability.

Would not this be of great use to native LISP hosts?  The document
doesn't consider native LISP hosts much, but for example, a
large server might speak LISP natively (consider a large search
company or CDN for instance).

      A "gleaned" map-cache entry, one learned from the source RLOC of a
      received encapsulated packet, is only stored and used for a few
      seconds, pending verification.  Verification is performed by
      sending a Map-Request to the source EID (the inner header IP
      source address) of the received encapsulated packet.

This means that "gleaning" does not reduce control traffic, it only
defers it (and allows packet exchanges to go a little faster).  That
would seem to be a mistake for certain classes of server for which no
transaction lasts longer than a few packets.

As an aside, LISP requires bidirectional connectivity beween ITR and ETR.
This is not a limitation in normal IP.

--------------------------------------------------------------------
Comment 27:

Section 6.3

Several mechanisms for determining RLOC reachability are currently
defined:

   1.  An ETR may examine the Loc-Status-Bits in the LISP header of an
       encapsulated data packet received from an ITR.  If the ETR is
       also acting as an ITR and has traffic to return to the original
       ITR site, it can use this status information to help select an
       RLOC.

   2.  An ITR may receive an ICMP Network or ICMP Host Unreachable
       message for an RLOC it is using.  This indicates that the RLOC is
       likely down.

   3.  An ITR which participates in the global routing system can
       determine that an RLOC is down if no BGP RIB route exists that
       matches the RLOC IP address.

   4.  An ITR may receive an ICMP Port Unreachable message from a
       destination host.  This occurs if an ITR attempts to use
       interworking [INTERWORK] and LISP-encapsulated data is sent to a
       non-LISP-capable site.

   5.  An ITR may receive a Map-Reply from a ETR in response to a
       previously sent Map-Request.  The RLOC source of the Map-Reply is
       likely up since the ETR was able to send the Map-Reply to the
       ITR.

   6.  When an ETR receives an encapsulated packet from an ITR, the
       source RLOC from the outer header of the packet is likely up.

   7.  An ITR/ETR pair can use the Locator Reachability Algorithms
       described in this section, namely Echo-Noncing or RLOC-Probing.

For the first mechanism to work it is not sufficient for the ETR
to have traffic "to return to the original ITR site" - the ETR has
to have traffic to return to the original ITR. And if the ETR has
no traffic to return to the original ITR (even if the ETR has traffic
to return to some other ITR of the site), then the first mechanism
is useless. 

Moreover, the first mechanism is predicated on the assumption that
the Loc-Status-Bits contain up to date information about reachability
of all ETRs at the site. The document describes how the ITR obtains
this information only for the case of CE-based or intra-site based
ITRs. Thus it does not cover the case where the ITR is the PE.
Moreover, for the case of CE-based ITR the scheme does not work if
the site runs RIP as its IGP. Furthermore it assumes that just
because the correspondent ITR can reach the given RLOC, the ETR
also will. Since IP connectivity is not always transitive in this
way, the fact that the corresponding ITR can reach the given RLOC
does not mean that the ETR also can. 

The second mechanism depends on ICMP Unreachable. As such it may
result in sustained traffic blackholing due to ICMP Unreachable
being dropped along the path. Also, how would that mechanism handle
DoS attacks caused by spoofed ICMP Unreachables ? Finally, if ITR
is within a site, and behind a firewall, would the firewall pass
ICMP Unreachables in the first place ?

The third mechanism is likely to be of very limited use, as an ETR
going down is unlikely to cause the route that covers the RLOC of
that ETR to be withdrawn (unless this is /32 route).

The fourth mechanism is applicable only for LISP interworking
between a LISP and a non-LISP site.

The fifth and the sixth mechanisms do provide the indication that
the ETR is up, but do not provide the information when the ETR is
down. As such they are not applicable for determining unreachability,
as unreachability requires the ability to determine that the ETR
is down.

That leaves us with the seventh mechanism. This mechanism offers
two options: Echo Nonce (section 6.3.1) and RLOC Probing (section
6.3.2). The Echo Nonce mechanism is applicable only when there is
a bidirectional flow between a pair of RLOCs. The RLOC Probing
mechanism has scaling limitations - quoting from 6.3.2:

     The major disadvantage of RLOC Probing is in the number of
     control messages required and the amount of bandwidth used to
     obtain those benefits, especially if the requirement for failure
     detection times are very small.

Given the above, what are the mechanisms that are available when
xTRs are PEs (as described in 8.3), and what are their scaling
characteristics ?

Consider an example of site A with two xTRs A1 and A2, and site B
with two xTRs, B1 and B2. Now imagine that outbound traffic from A
to B is using the ITR component of A1, and talking to the ETR
component B1 of B.  But the traffic from B back to A uses the ITR
component of B2 and is sending to the ETR component of A2.  So each
ITR->ETR flow is unidirectional, even though all four devices are
fully capable of bidirectional communication. What are the options
for the RLOC reachability in this scenario ? 

--------------------------------------------------------------------
Comment 28:

Section 6.3

     The ITR can test the reachability of the unreachable Locator by
     sending periodic Requests.  Both Requests and Replies MUST be rate-
     limited.  Locator reachability testing is never done with data
     packets since that increases the risk of packet loss for end-to-end
     sessions.

How would one use data packets for testing locator reachability ?

--------------------------------------------------------------------
Comment 29:

Section 6.3.1

     When there is bidirectional data flow between a pair of locators, a
     simple mechanism called "nonce echoing" can be used to determine
     reachability between an ITR and ETR.  When an ITR wants to solicit a
     nonce echo, it sets the N and E bits and places a 24-bit nonce in the
     LISP header of the next encapsulated data packet.

What would it mean if N were clear and E were set?

As a general comment about this section, it seems to require significant
new forwarding plane functionality and as such, should be considered
when thinking about whether LISP can be implemented in hardware.
--------------------------------------------------------------------
Comment 30:

Section 6.3.2

     cached by the ITR or PTR.  The ITR or PTR may include a mapping data
     record for its own database mapping information.

What exactly is "its own database mapping information" ?

--------------------------------------------------------------------
Comment 31:

Section 6.4

     Note that when a packet is LISP encapsulated, the source port number
     in the outer UDP header needs to be set.  Selecting a random value
     allows core routers which are attached to Link Aggregation Groups
     (LAGs) to load-split the encapsulated packets across member links of
     such LAGs.  Otherwise, core routers would see a single flow, since
     packets have a source address of the ITR, for packets which are
     originated by different EIDs at the source site.  A suggested setting
     for the source port number computed by an ITR is a 5-tuple hash
     function on the inner header, as described above.

"Random" in the second sentence is contradicted by the last sentence.

Also, it is not clear why the paragraph restricts itself to talking about
LAGs.  Load balancing is used plenty of other places.

--------------------------------------------------------------------
Comment 32:

Section 6.5

As a general comment, this section is in need of some editing to
make the prose more readable.

     When a locator record is added to the end of a locator-set, it is
     easy to update mappings.  We assume new mappings will maintain the
     same locator ordering as the old mapping but just have new locators
     appended to the end of the list.  So some ITRs can have a new mapping
     while other ITRs have only an old mapping that is used until they
     time out.  

This identifies a real problem;  i.e., the locator status bits are
contextually dependent on the locator set that was advertised, but
there is no explicit binding between them and the particular locator
set to which they relate.  Thus, any version skew between ITR and
ETR will result in misinterpretation of the locator status bits.

This seems like it could be a serious problem (though section 6.3
is so underspecified and hard to follow that this is difficult to
pin down) and needs to be understood better.  The fixes proposed
in this section seem complex and incomplete.

     When an ITR has only an old mapping but detects bits set
     in the loc-status-bits that correspond to locators beyond the list it
     has cached, it simply ignores them. However, this can only happen
     for locator addresses that are lexicographically greater than the
     locator addresses in the existing locator-set.

It is previously stated that locators in the locator-set must be
sorted; this would seem to indicate that you cannot add a locator
to the locator-set on demand, and that a process such as defined
in 6.5.1 must be used to try to ensure sync between ITR and ETR

There would seem to be a need to either get rid of locator-sets,
or come up with a better system for synchronizing them.  The
consequences of out-of-sync locator-sets should also be spelled
out.

--------------------------------------------------------------------
Comment 33:

Section 6.5.1

     The default setting for an EID-to-RLOC mapping TTL is 24 hours.

This is the only place in the document where the default is specified!

There is a missing step in the procedure, which is to restore the
TTL to 24 hours.

In general, this procedure would seem to lead to increased operational
complexity, and thus contradict the claims of reduced opex.

--------------------------------------------------------------------
Comment 34:

Section 6.5.2 Solicited-Map-Request

   Since the xTRs don't keep track of remote ITRs that have cached their
   mappings, they can not tell exactly who needs the new mapping
   entries.  So an xTR will solicit Map-Requests from sites it is
   currently sending encapsulated data to, and only from those sites.

"currently" needs to be quantified.  E.g., if the local xTR detected
change in the mapping at time T1, and sends the encapsulated data
to a particlar remote xTR at time T2, what is the max allowable
(T2-T1) ? Is it 1 sec, 1 min, 1 hour, 1 day ?

What is the meaning of "solicit Map-Requests from sites" ?  Does
it mean soliciting it from all the xTRs of a given site ? If yes,
then how would it determine all such xTRs ? And if no, and it means
just one xTR of a given site, then how would Solicited-Map-Request
work in the scenario where an xTR-A sends encapsulated data to xTR-B
of a given (remote) site, but that site uses some other xTR, xTR-C,
to send the data to xTR-A, and it is xTR-C that needs the new mapping ?

      The remote xTR retransmits the Map-Request slowly until it gets a

"slowly" needs to be quantified.

     The ETRs at the site with the changed mapping will reply to the
     Map-Request with a Map-Reply message provided the Map-Request
     nonce matches the nonce from the SMR.  The Map-Reply messages
     SHOULD be rate limited.  This is important to avoid Map-Reply
     implosion.

Is the implication that during this procedure, normal new Map-Requests
will not be replied to in the usual way, since they won't have the
nonce?

     with an EID destination to the mapping database system.  Since the
     mapping database system is more secure to reach an authoritative ETR,

What is the justification for this assumption about the mapping
database's security?

--------------------------------------------------------------------
Comment 35:

Section 7 Router Performance Considerations:

     LISP is designed to be very hardware-based forwarding friendly.  By
     doing tunnel header prepending [RFC1955] and stripping instead of re-
     writing addresses, existing hardware can support the forwarding model
     with little or no modification.  Where modifications are required,
     they should be limited to re-programming existing hardware rather
     than requiring expensive design changes to hard-coded algorithms in
     silicon.

This is simply an Assertion.

     When a tunnel encapsulated packet is received by an ETR, the outer
     destination address may not be the address of the router.  This
     makes it challenging for the control plane to get packets from the
     hardware.  This may be mitigated by creating special FIB entries
     for the EID-prefixes of EIDs served by the ETR (those for which
     the router provides an RLOC translation).  These FIB entries are
     marked with a flag indicating that control plane processing should
     be performed. The forwarding logic of testing for particular IP
     protocol number value is not necessary. No changes to existing,
     deployed hardware should be needed to support this.

The above explanation of why no changes to existing deployed
hardware should be needed is fairly confusing.

     On an ITR, prepending a new IP header is as simple as adding more
     bytes to a MAC rewrite string and prepending the string as part of
     the outgoing encapsulation procedure.  Many routers that support
     GRE tunneling [RFC2784] or 6to4 tunneling [RFC3056] can already
     support this action.

This looks like another Assertion.

     When a received packet's outer destination address contains an EID
     which is not intended to be forwarded on the routable topology
     (i.e.  LISP 1.5), the source address of a data packet or the
     router interface with which the source is associated (the
     interface from which it was received) can be associated with a VRF
     (Virtual Routing/Forwarding), in which a different (i.e. non-
     congruent) topology can be used to find EID-to-RLOC mappings.

What does the part about "When a received packet's ... to be forwarded
on the routable topology" have to do with the part about "the source
address of a data packet ...  can be associated with a VRF" ?

How does one determine whether the outer destination address
"contains an EID which is not intended to be forwarded on the
routable topology" ? Please answer this question both for IPv6
and for IPv4. 

--------------------------------------------------------------------
Comment 36:

Section 8

This section doesn't discuss host based LISP implementations, which
seems a possibility as it is discussed under mobility.  What are
the implications be of host based LISP on the control plane load
and does two headers restriction create a problem for host based
LISP?

In general, the more ITRs there are (with host based being the
limiting case),  the greater the load on distant ETRs, which have
no control of the number of ITRs.  This would seem to be a case of
costs and benefits not being aligned.

     ISP is doing the TE, then the site has no control.  Recursive
     tunneling generally will result in suboptimal paths but at the
     benefit of steering traffic to resource available parts of the
     network.

What exactly does "suboptimal" mean here?

--------------------------------------------------------------------
Comment 37:

Section  8.1

If ITR/ETR are first/last hop routers, then these routers need to
be routable within each site RLOC.  This means that if a site changes
providers, the RLOC of these routers would need to change.  Therefore,
any ACLs within each site would need to be modified to allow routing
on these RLOCs.

--------------------------------------------------------------------
Comment 38:

Section 8.2

     Note that more than one CE router at a site can be configured with
     the same IP address.  In this case an RLOC is an anycast address.
     This allows resilience between the CE routers.  That is, if a CE
     router fails, traffic is automatically routed to the other routers
     using the same anycast address.  However, this comes with the
     disadvantage where the site cannot control the entrance point when
     the anycast route is advertised out from all border routers.

The above means that at the minimum the price of resilience between
the CE routers is the inability to support traffic engineering?
Specifically, the ISP loses any influence over the choice of which
ETR should be used to reach the multi-homed enterprise.

Moreover, if CEs are xTRs and use anycast addresses, then RLOCs are
anycast addresses as well, and thus can not be topologically
significant. Thus if an enterprise is multi-homed to two (or more)
ISPs, then use of anycast addresses for CEs would require to route
such addresses as /32 throughout the whole Internet. 

Also, if anycast addresses are used as RLOCs, then how would one
deal with a situation where initially both ETR1 and ETR2 advertise
10.1.1/24 and 10.1.2/24, ITR1 routes traffic for 10.1.1.1 to ETR1,
but then ETR1, while still being up, loses connectivity to 10.1.1/24?
------------------------------------------------------------------
Comment 39:

Section 8.3

      Use of ISP PE routers as tunnel endpoint routers gives an ISP control
      over the location of the egress tunnel endpoints.  That is, the ISP
      can decide if the tunnel endpoints are in the destination site (in
      either CE routers or last-hop routers within a site) or at other PE
      edges.  
   
How is this done, as the CE router (probably) and last-hop within
a site (surely) is under control of the customer, not the ISP.

      The advantage of this case is that two or more tunnel headers
      can be avoided.  

Only if the extra header in question would be imposed by the first-hop
ISP.  Any other ISP would still need to impose their own header in order
to do TE.
   
     By having the PE be the first router on the path to encapsulate,
     it can choose a TE path first, and the ETR can decapsulate and
     re-encapsulate for a tunnel to the destination
     end site.

This is hard to understand.

--------------------------------------------------------------------
Comment 40:

Section 9

This section would benefit from being rewritten to increase clarity.

In the IPv4 case, more router complexity is introduced, analogous
to what a NAT has to do.  This feature appears to be intrinsically
unscalable, as an ETR serving a large enough set hosts doing
traceroute will eventually start breaking traceroute when its buffer
is exhausted.  But worse than just breaking it, it will return bogus
traceroutes.  This could happen if a large percentage of hosts were
running traceroute, or if the ETR were very large and serving many
hosts, only a small fraction of which was running traceroute.

Plus, how is the router supposed to identify a "traceroute packet"? 

--------------------------------------------------------------------
Comment 41:

Section 9.3

     segment 2 will make the tunnel look like one hop.  All the ITR has to
     do to make this work is to not copy the inner TTL to the outer,
     encapsulating header's TTL when a traceroute packet is encapsulated
     using an RLOC from a different address family.  This will cause no

Earlier it was claimed that copying the TTL is important for correctness
reasons.  Why is it not important in this case, but is important in all
other cases?

--------------------------------------------------------------------
Comment 42:

Section 10.3

     visited network using its original EID, then LISP will need to perform
    a route-returnability check on the new EID-RLOC mapping for
    that EID.

What is a "route-returnability check"?

     loss for the entire system.  Heuristics can be added to LISP to
     reduce the number of mapping changes required and to reduce the delay

This sounds like an assertion.

     per mapping change.  Also IP mobility can be modified to require
     fewer mapping changes.  In order to increase overall system
     performance, there may be a need to reduce the optimization of one
     area in order to place fewer demands on another.

This would appear to be a tacit admission that LISP has a problem with
IP mobility (and would therefore like IP mobility to go away).  More of
the same a few paragraphs down:

     Mobile IP packet exchange is designed for an environment in which all
     routing information is disseminated before packets can be forwarded.
     In order to allow the Internet to grow to support expected future
     use, we are moving to an environment where some information may have
     to be obtained after packets are in flight.  Modifications to IP
     mobility should be considered in order to optimize the behavior of
     the overall system.  Anything which decreases the number of new EID-
     RLOC mappings needed when a node moves, or maintains the validity of
     an EID-RLOC mapping for a longer time, is useful.

--------------------------------------------------------------------
Comment 43:

Section 12

There are many comments above that relate to security.  Grep for
"security" or "attack".  Other possible issues that come to mind that
should be explored here are whether LISP breaks RPF, and whether the
ubiquitous tunneling infrastructure could be reused as a botnet
anonymization service.

--------------------------------------------------------------------
Comment 44:

Section 12

     To deal with map-cache exhaustion attempts in an ITR/PTR, the
     implementation should consider putting a maximum cap on the number of
     entries stored with a reserve list for special or frequently accessed
     sites.  This should be a configuration policy control set by the
     network administrator who manages ITRs and PTRs.

Yet more complexity, both in implementation and operation (where is the
reduced opex?). All this contradicts the claim in the Abstract about
LISP being "simple".

--------------------------------------------------------------------
Comment 45:

14.1.  Normative References:

Why is HIP a normative reference?