Subject: Re: [lisp] ALT structure, robustness and the long-path problem

From: [email-masked] (Noel Chiappa)
Date: Tue, 8 Dec 2009 16:25:22 -0500 (EST)

> From: Robin Whittle <[email-masked]>

    > In a system such as LISP-NERD, there is no delay since the ITRs have
    > the full mapping database.

NERD has even less headroom for growth than ALT does. (E.g. it can't handle
mobile hosts.) It also has lots of useless overhead (e.g. keeping me totally
up to date about the mapping for some dentist's office in Japan that I am
never going to want to talk to).

    > It looks highly unscalable to concentrate about 1/224 of the world's
    > map requests through a single router.

Caching is your friend. The DNS works fine, even though 1/14 (or however many
root servers there are) of all the world's DNS requests go through a single
root server.

    > How is the above simple tree-like structure going to be elaborated upon
    > so the system continues to work when one or more ALT routers, or their
    > GRE tunnels, suffers transient or lasting failure?

Replication is your friend. (Q.v. DNS, again.)

    > The only advantages of ALT over NERD, APT or Ivip

You missed a very important one: it was easily coded and deployed since it
relies on existing protocols/code. If you think this is not significant,
implement and deploy one of the other alternatives.

	Noel

---

From: Robin Whittle <[email-masked]>
Date: Wed, 09 Dec 2009 12:07:34 +1100

Short version:   Responding to Noel's claim that ALT is suitable
                 for mobility while NERD is not, and to his
                 statement that ALT can be made to perform well
                 by the use of "replication".

                 Developing ALT as an experimental protocol is fine
                 - but the effort put into LISP is surely based on
                 the idea that there is or will be a mapping system
                 which makes LISP the optimal solution for the
                 routing scaling solution.  If ALT is not that
                 optimal system, then shouldn't we be developing
                 a better one ASAP, at least to an experimental
                 stage?

                 Is discussion of ALT's suitability for mobility
                 out of scope, since it is discussion of ALT?  If so
                 we could discuss it soon on the RRG, since on 15 Dec
                 we will be allowed to discuss *proposals* there.

Hi Noel,

You wrote:

>> I understand that LISP-ALT has been proposed as the best solution
>> for the routing scaling problem
>
> LISP, or ALT? They are not the same thing. ALT is one option for a
> _component_ of the overall LISP architecture.

At first there were no mapping systems for LISP.  Then there was NERD
and CONS.  Few people expected NERD to be the optimal solution for
the routing scaling problem, for reasons you mentioned.

CONS is a global query server system, with new protocols for the
network elements.  There was no single point which held all the
mapping.  A mapping query from an ITR in place X had to travel to an
ETR at place Y which was the authoritative query server for the EID
prefix in question, or perhaps to some other ETR at point Z which was
also authoritative.  (Maybe there were some caching functions in the
CONS network - I don't remember.)

ALT is also a global query server system, as described above, with no
caches in the ALT network itself.  Like CONS, the query has to travel
potentially across the world over the new network - but the response
comes back via the Internet.  (I recall that with CONS it came back
via the CONS network.)  The actual physical path over the CONS or ALT
network is not necessarily closely aligned with the shortest
available path through the DFZ, as determined by however the DFZ
routers choose their paths.

ALT, like CONS, suffers from delay and packet loss problems which
would cause ITRs to drop initial packets in some new communication flows.

It is reasonable to assume that so many people are pursuing LISP
because they believe that LISP in some form will be a better solution
to the routing scaling problem than anything else.  This includes the
other core-edge separation architectures (APT, Ivip and TRRP) and any
new architecture they might be able to think of themselves.

> mobile hosts.) 

Mobility along the lines of LISP-MN, with the MN being its own ETR,
won't be practical for IPv4 since the MN can't be behind NAT.  A
critique is [1].

For the TTR approach to mobility [2], NERD would have a problem in
that it takes a long time for all ITRs to know about a new ETR
address.  The same goes for any other core-edge separation scheme
apart from a scheme (currently only Ivip) which ensures that all ITRs
which need it get the updated mapping within a few seconds.   APT's
mapping distribution is not necessarily any faster than NERD's.

ALT would improve on NERD's long time delays, but only if either
ALT's caching time was shorter (which means more work for the ALT
network) and/or if the ALT system also enabled ITRs to get mapping
updates.

There may be a way with ALT of securely telling all the ITRs which
need to know about a new ETR address - telling them their cached
mapping is out of date and to use a new set of mapping information
instead.  This could be tricky since the authoritative source of the
mapping, the old ETR address, may not be usable once the change
occurs.  A stable Map Server (MS) would solve this problem.  There
are scaling problems with this - such as if thousands of ITRs need to
be informed of the new mapping.

To do this, something - such as the old ETR or the stable MS - needs
to know about every ITR which has requested mapping for the EID
prefix in question, within the current caching time specified in that
mapping.

It also needs to be able to securely contact each ITR and send it the
new mapping information.

In addition to sending a map update message to each ITR, there needs
to be acknowledgement of the receipt of the message.  If there is no
acknowledgement, the authoritative query server needs to resend.
Otherwise, loss of the update message will leave the ITR sending
packets to the wrong ETR address for the remainder of its caching time.

I don't understood how ALT works with two authoritative query servers
for the one EID prefix - an ETR or MS in one location and an ETR or
MS in another.  I guess it is like advertising a prefix from two
routers in the DFZ.  Anyway, to support the rapid changes to mapping
which are ideally needed for mobility, each such authoritative query
server needs to be able to securely, robustly and rapidly update the
mapping of every ITR which requested the mapping in the caching time.

If there is caching somewhere in the ALT system - for instance Dino
suggested caching at the Map Resolver (MR) a few weeks ago:

  http://www.ietf.org/mail-archive/web/rrg/current/msg05308.html

then the MRs would need to fan out the update to one or more ITRs
once they get a secure update of mapping information from the
authoritative ETR or MS.

If the TTR approach is used, then the only alternative seems to be
the MN being its own ETR, as with LISP-MN.  Then you have to change
the mapping in all the ITRs "instantly".  "Instantly" means as soon
as possible, because often when a MN gains a new address (a Care of
Address - CoA, though the LISP-MN ID doesn't use this term) it is
because it just lost its previous MN.

It would be fine if the MN had a CoA A on a WiFi link, found a new
CoA B on a 3G network and could use both while the ITRs were being
told to that the mapping had now changed to B.  However, when the MN
only has a single CoA, and this changes, then any delay in updating
the cache of all ITRs will result in loss of traffic packets.

A MN with a DHCP CoA address could have a new CoA at any time.  A MN
on a 3G network could get a new CoA when it physically connects to
base-stations which use a different IP server (I forget the proper
term).  This can happen without the MN moving at all, simply due to
the dynamic allocation of radio resources.  So for mobility with the
MN being its own ETR, any delay in updating ITR mapping means lost
connectivity.

This is not the case with the TTR approach, since the ETRs
(Translating Tunnel Routers) there are  stable and the MN makes
two-way tunnels to one or more of them.  As long as the MN has a CoA,
it can receive packets from as many ETRs (TTRs) as it likes.

With LISP-MN, you need instant updates of all ITRs' mapping - which
NERD can't do, and which will take some time (fractions of a second
to seconds) with ALT if it implements mapping updates.

With the TTR approach, it would be best to have rapid mapping
updates, so the MN can stop using the old ETR (TTR) sooner and so
traffic can use the new one, which is closer to the MN's new
topological location.  But you really don't *need* to update the
ITRs' mapping quickly.  If it took half an hour to update it, this
means that MNs need to maintain a tunnel to their old ETRs (TTRs) for
half an hour.

With "MN is its own ETR" mobility:

  NERD (assuming there is no ETR/MS to ITR mapping update message)
  would involve intolerably long losses of traffic packets with
  LISP-MN and ALT would result in disruptions of fractions of a
  second or seconds.

With the TTR approach:

  NERD and ALT would be OK - and any faster system would be better.

So I don't accept your statement that ALT can support mobility while
NERD can't.

> It also has lots of useless overhead (e.g. keeping me totally

Yes, NERD is not the optimal approach.

> root servers there are) of all the world's DNS requests go through a single

OK - so where are you going to put caches in the ALT network?  The MR
is the only logical place to do it.  Now you have ETRs or MSes
securely sending map updates to both ITRs and MRs, and the MRs
fanning these updates out securely to ITRs.  Each such update needs
to be completely secure against sustained attack, and it needs to be
securely acknowledged, with appropriate retry efforts bounded by some
kind of time-out.

You can do these things, but at great cost to the original attraction
of ALT - its simplicity.

> Replication is your friend. (Q.v. DNS, again.)

I don't see what "replication" means in the ALT network if it is
different from caching.  Can you give some examples?

> implement and deploy one of the other alternatives.

Ease of design is an important factor when testing and prototyping.
It is pretty much irrelevant when considering how best to modify the
Internet on a permanent basis to solve the scalable routing problem -
and ideally to support mobility and better use of IPv4 space.

Its fine that ALT is chosen now to build a test network.  But what is
the point of doing a test network with some architecture which is
different from that which is optimal for real large scale, permanent,
deployment?

Presumably all the enthusiasm for LISP is because people believe it
will be the optimal solution - which should mean they have evaluated
the alternatives (including APT and Ivip) and have decided that they
clearly can't be the optimal solution and that LISP in some form can be.

But if ALT's attraction is ease of design, not that it looks like it
will be the optimal solution, then what alternative architecture for
LISP is currently under development as that optimal solution?

I understand this WG is only chartered to develop LISP-ALT as an
experimental protocol.  So there is no formal expectation that ALT
will be the optimal solution to the routing scaling problem.

But why develop such an experimental mapping system for LISP if there
are no concrete plans for a mapping system which would be part of an
optimal solution?  The optimal mapping system would be very different
from ALT, so why spend years developing the ALT system, if it won't
tell us much about the optimal form of LISP?

  - Robin

[1]  http://www.ietf.org/mail-archive/web/lisp/current/msg00749.html
     http://www.ietf.org/mail-archive/web/lisp/current/msg00772.html

[2]  http://www.firstpr.com.au/ip/ivip/TTR-Mobility.pdf

---

From: Sam Hartman <[email-masked]>
Date: Tue, 08 Dec 2009 20:10:03 -0500

>>>>> "Noel" == Noel Chiappa <[email-masked]> writes:

    Noel> You missed a very important one: it was easily coded and
    Noel> deployed since it relies on existing protocols/code. If you
    Noel> think this is not significant, implement and deploy one of the
    Noel> other alternatives.

Seriously, it would be great to see someone working on testing a non-alt
mapping system.  I realize it's out of scope for this WG, but it's still
very interesting.

If you want to implement a mapping system but want to re-use a LISP
control plane, Margaret and I may be able to help, especially in a month
or two.  If you export the MR/MS interface, then you could work with any
lisp ITR/ETR.

---

From: "Darrel Lewis (darlewis)" <[email-masked]>
Date: Tue, 8 Dec 2009 17:14:57 -0800

>                  we will be allowed to discuss *proposals* there.

In my personal opinipon, LISP-NERD's problems in dealing with mobility
go way beyond the case LISP Mobile Node's use of mobility.  Slower
mobility, such as adding/deleting locators when a site adds/deletes
circuits are just as problematic if every ITR in the system has to know
about it before they can use it.

In short, /etc/hosts just doesn't scale.

-Darrel

---

From: Michael Menth <[email-masked]>
Date: Wed, 09 Dec 2009 15:39:30 +0100

Hi Sam,

> or two.  If you export the MR/MS interface, then you could work with any
We have already a basic version of FIRMS
http://www3.informatik.uni-wuerzburg.de/staff/menth/Publications/papers/Menth09-FIRMS.pdf
running in the G-Lab testbed and currently add all the security 
features. Of course it's a students' implementation in Java and 
certainly not carrier-grade, but it helps gaining some experience with 
alternative mapping systems, that's why we do it. Integrating the story 
in a LISP control plane certainly makes sense.

Kind regards,

    Michael

> _______________________________________________
> lisp mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/lisp
>   

-- 
Dr. Michael Menth, Assistant Professor
University of Wuerzburg, Institute of Computer Science
Am Hubland, D-97074 Wuerzburg, Germany, room B206
phone: (+49)-931/31-86644 (new), fax: (+49)-931/888-6632
mailto:[email-masked]
http://www3.informatik.uni-wuerzburg.de/research/ngn