Subject: Re: [lisp] Confirming consensus behind echo noncing

From: [email-masked] (Noel Chiappa)
Date: Fri, 4 Sep 2009 17:23:29 -0400 (EDT)

> From: Sam Hartman <[email-masked]>

    > All the locator reachability algorithms are up for experimentation. I
    > think it would be fair to re-open the question of whether any given
    > reachability algorithm is appropriate based on practical experience.

I honestly don't recall, at this point, how many problems have been seen with
reachability in testing over the last couple of years. (Maybe some of the
people involved in testing can add to this?)

I do know that a lot of the organizations which are interested in running
LISP have expressed concerns about reachability, and I do think that that has
to be a factor for the WG. (Clearly, if people aren't going to run a protocol
because of problems they perceive with it, then we'll never get to the point
where we can find out whether or not it works in practise...)

It certainly is the case that the design team has put a lot of time/energy
into efficient, robust and timely reachability testing algorithms over the
last year or so, and although I can't speak with 100% certainty as to the
reasons why, I don't think we would have done that without a good reason
(either user demand, or problems seen in service).

	Noel

---

From: [email-masked] (Noel Chiappa)
Date: Sat, 5 Sep 2009 10:34:53 -0400 (EDT)

> From: Sam Hartman <[email-masked]>

[A few points in addition to Dino's..]

    > echo nonces. ... they are not useful in any of the following situations
    >
    > * square routing
    > * triangle routing (a sends to b, c sends to a)

Note that nonces are xTR to xTR, not EID to EID. So even if a particular TCP
connection is square/triangle routed (e.g. your example above), there may
still be packets flowing back from B to A; perhaps for some other connection,
and/or pair of EIDs. E.g. perhaps there are two EID-ranges using B and C as
their xTRs, with different mappings, so there may be traffic going from A to
C for a different destination EID, and packets from the combination of the
two carry the nonces.

We can also write a configuration/usage document which points out to people
points such as that if they have a configuration which is 'strictly'
triangular (i.e. only one announced ETR, b, and only one ITR, c) they lose
the 'free' piggybacked reachability testing of nonces, and recommend against
such configurations.

    > cannot detect a full path failure: in order to conclude you cannot
    > reach someone you need to get packets from them

True, but if there is a full path-failure, the fact that the xTRs aren't
hearing nonces will cause them to do a probe (which is the robust last line
of defence), and that will fail, signalling the full failure. (And if there
are no packet flowing, there is _no_ mechanism other than a timeout which can
detect that anyway.)

Also, with appropriate (more complex, albeit) logic in the sender, nonces can
also be used to produce a 'down' signal in the forward-only direction (which
is the only direction nonces tell you about, actually), not just an 'up'
signal (and produce the 'down' signal more quickly than the simple 'we
haven't heard a nonce echoed in the N second timeout, time to do an explicit
probe' above). If an xTR has sent a nonce multiple times and not heard it
echoed, even though it has heard lots of packets from the other end since it
first sent the nonce, that is an indication that the forward direction is
down.

    > It seems likely that in any situation where you have multiple rlocs of
    > the same priority you'll likely run into that case if you have a small
    > number of flows.
    > Long term .. I think we'll end up with required control plane probing
    > of locators with possible optimizations through the data plane. 

Nonces are an optimization, one intended to produce 'pretty good'
reachability and liveness detection most of the time, at little cost. Yes,
there will be corner cases where it doesn't work - which is why it's not the
_only_ reachability and liveness detection mechanism.

The concept is to have a collection of mechanisms which _together_ produce
the desired characteristics of high robustness, low overhead, and fast
response; there doesn't seem to be a single mechanism that touches all of
these bases.

    > In that environment, I think echo nonces will serve no purpose.

Nonces are one leg of the {robustness, overhead, response} stool; take them
away, and you'll lose something on meeting these goals.

It's actually a four-way tradeoff, with the last axis being complexity -
having multiple mechanisms is more complex, but it does allow us to make
_each_ mechanism relatively simple, since none bear the full load of meeting
the 3-way goal.

    > the performance concerns that matter for probing on CPEs seem very
    > different than say XTRs at an Amazon data center.

But it takes two to tango. A customer CPE is probably talking to Amazon,
so we can't have a mechanism that makes sense _only_ at customer CPE's.

	Noel