Subject: [lisp] How blocked on versioning and PMTU are we?

From: Sam Hartman <[email-masked]>
Date: Mon, 01 Jun 2009 12:25:24 -0400

There has been a lot of discussion of the LISP header on the list.

If I'm reading things correctly,

* There appears to be consensus that we're marking part of the control
  packet header reserved.

* Dino has proposed a new header format.

A few people have agreed that Dino's format is a step forward.
However we have two related open issues:

1) PMTU handling--do we need reassembly/other PMTU information in the header

2) map versioning

Question to those who want to see changes related to PMTU or map
versioning.  Is it OK for Dino to go ahead and make the change he
proposed or do we need to block on resolution of these issues?

If we do change the header now rather than blocking on these two
issues, we cannot later use arguments about how recently we've changed
the header or about not wanting to introduce churn as arguments
against PMTU or map versioning.

As a warning, I do not yet understand the map versioning issue well
enough to make a consensus call.  One of the things I don't understand
about that issue is whether there has been enough discussion that it
would be reasonable to make a consensus call.

I do think I understand the PMTU issue well enough to believe that it
has not been sufficiently discussed in this forum, although I
understand that it has been discussed prior to this working group
being formed at length.

---

From: "Joel M. Halpern" <[email-masked]>
Date: Mon, 01 Jun 2009 12:41:52 -0400

Assuming, as you state below, that we agree taht we may change the 
header yet again, then I think adopting Dino's proposal at least for now 
is a good idea.
We need to get more discussion of at least the versioning (even after 
reviewing that draft, I am town as to whether it is a good idea or a bad 
idea.)

Yours,
Joel

Sam Hartman wrote:
> _______________________________________________
> lisp mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/lisp
>

---

From: Damien Saucez <[email-masked]>
Date: Mon, 01 Jun 2009 18:56:20 +0200

Maybe we can use Dino's proposal until we have a consensus and let the 
question about PMTU and versioning open for now.

Sam, could you start a new thread to start an "official" discussion on 
these two topics?

Cheers,

Damien Saucez
P.S.: we also have to discuss security, do we want security for the 
mapping only or also for the data packets? In the latter case, it means 
that we would probably change the header.

Joel M. Halpern wrote:
> header yet again, then I think adopting Dino's proposal at least for 
> now is a good idea.
> reviewing that draft, I am town as to whether it is a good idea or a 
> bad idea.)
>
> Yours,
> Joel
>
> Sam Hartman wrote:
>>

---

From: Sam Hartman <[email-masked]>
Date: Mon, 01 Jun 2009 13:17:37 -0400

y
    Damien> Maybe we can use Dino's proposal until we have a consensus
    Damien> and let the question about PMTU and versioning open for
    Damien> now.

    Damien> Sam, could you start a new thread to start an "official"
    Damien> discussion on these two topics?

I'm actually asking Fred to do that for PMTU.

Let me read the map versioning draft and start a thread.

---

From: Luigi Iannone <[email-masked]>
Date: Tue, 2 Jun 2009 09:14:30 +0200

Hi All,

I was replying to the mail of Dino:

http://www.ietf.org/mail-archive/web/lisp/current/msg00541.html

and I thought that may be is a good start for the discussion on map- 
versioning.

Attached are my comments to Dino's slide (format is pdf, but if you  
need any other format please ask), hope this will generate discussion.

Cheers

Luigi

---

From: Sam Hartman <[email-masked]>
Date: Tue, 02 Jun 2009 10:40:01 -0400

Your slides jump right into describing the solution with map
versioning, which really isn't such a great place to start a
discussion of it.

That discussion should start with the problem.

If I understand the mail between you and Dino correctly, there are two
concerns you have.

1) An attacker can manipulate the reachability bits and either create
a DOS or cause traffic to take a path that gives the attacker some
advantage.

2) Certain update situations could get the ITR and ETR out of sync
about what the reachability bits map to and this could cause a problem
for traffic.

Is that the problem you're trying to solve?

Dino's answer seems to be that:

1) If you care about these attacks you can verify the reachability
bits.  Presumably we (the WG) will need to consider what mandatory
guidance to give on this issue if we don't adopt map versioning.

2) There exist algorithms for adding/removing a locator in which the
changes to the reachability bits are non-ambiguous.  Presumably if we
don't adopt map versioning we'll need to make sure that the spec
mandates such an update procedure.

Have I got things right as far as I've gone?

---

From: Dino Farinacci <[email-masked]>
Date: Tue, 2 Jun 2009 10:31:06 -0700

Sam, let me phrase what we have been discussing another way. We have  
the following subject matters:

(1) How to update ITRs which have old map-caches when a database  
mapping for a LISP site changes.

(2) Should we put an 8-bit flags field in the data encapsulation header.

(3) Are there anymore MTU issues.

(4) Can the loc-reach-bits in the data encapsulation header can  
compromised.

I believe the status for each is:

(1) You can use SMRs for map-cache updates according to the existing  
spec or you can use draft-iannone-lisp-mapping-versioning-00.txt,  
referred to as "map-versioning". The later requires header format  
changes and Luigi gave 3 options in his IETF presentation.

(2) We have *some* consensus gain on this. However, the E-bit for  
doing nonce-echoing has not been fully flushed out. So we should wait  
for changes IMO.

(3) I looked through all of Fred Templin's emails and his suggested  
text and made clarifying changes to draft-ietf-lisp-01.txt. So I think  
this issue is closed. We have 2 mechanisms to deal with MTU issues  
with encapsulation. One is stateless and the other is stateful which  
was offered by Luigi and team. They are simple mechanisms that use  
existing standards (Fragmentation and PMTU Discovery).

(4) We have documented in the latest spec (went in many revisions ago)  
that the loc-reach-bits are a hint and used for a certain class of  
failures (close to the destination LISP site) and that the loc-reach- 
bits need verification by round-trip exchange between the receiver  
(that wants to take action on the loc-reach-bit change) and the sender  
of the loc-reach-bits.

Dino

---

From: Olivier Bonaventure <[email-masked]>
Date: Tue, 02 Jun 2009 21:58:39 +0200

Sam,

> That discussion should start with the problem.

I think that the problem can be described as follows :

How does an ETR responsible for an EID prefix informs an ITR about a
change in the mapping for this prefix ?

There are two different answers to this question.

The first answer, based on the current draft and Dino's presentation can
be summarised as follows :

- the most important change is a change in the reachability of one ETRs
that can be used to reach the prefix. The loc reach bits placed inside
the header of all encapsulated data packets can be used to inform the
ITR of any change in reachability of one of these ETRs

- the other changes (weigths, preference, ...) can be handled by first
decreasing the ttl of the mapping, ... see Dino's slides for more
information about this procedure.

The second answer, described in LISP Mapping Versioning, can be
summarised as follows :

- all changes (reachability, weight, preference, addition/removal of one
ETR) are equally important

- each mapping has a version number and any change in the mapping causes
a change of version number. The ITR places in the header of data
encapsulated packets the version number of its mapping and the version
number of the mapping of the destination prefix that it knows. By
looking at the version number found in the received data encapsulated
packets, it can check whether the mapping has changed and send a
Map-Request if a change has been detected to retrieve the new mapping

Olivier

-- 
http://inl.info.ucl.ac.be , UCLouvain, Belgium

---

From: "Templin, Fred L" <[email-masked]>
Date: Tue, 2 Jun 2009 13:29:22 -0700

Folks,

I'd like to try to summarize the MTU concerns as follows:

1. Insertion of an outer IP header at the ITR reduces the
   effective MTU on the path to the ETR.
2. The IPv4 IP_ID field is only 16bits (too small to safely
   support sustained fragmentation at high data rates).
3. Fragmentation in routers performed in slow path
4. Overhead for reassembly in ETRs is too onerous in some
   scenarios
5. Outer ICMP messages from routers on the path to the ETR may
   be lost and/or not contain enough information for translation
   into inner ICMP messages to send back to the original source
6. Outer ICMP messages from routers on the path may be
   insufficiently provisioned to allow for authentication
7. since the ITR is an in-the-network tunnel, hitting an MTU
   restriction in the path to the ETR results in a flood of
   ICMP Packet Too Big (PTB) messages coming from the network
   and bombarding the ITR

About proposed resolutions, the key idea is for the ITR to
only accept PTB feedback coming from ETRs and not accept it
from "anonymous" middleboxes. This feedback will get through
in instances where "vanilla" ICMP feedback might not, since
the feedback is sent in the control plane which may not
necessarily share the same channel as the data plane.

Comments?

Fred
[email-masked]

---

From: Sam Hartman <[email-masked]>
Date: Wed, 03 Jun 2009 11:29:51 -0400

>>>>> "Templin," == Templin, Fred L <[email-masked]> writes:

    Templin,> Comments?

Fred, thanks for your message.  I think that you've written down some
of the generic concerns with PMTU issues you might run into when you
introduce a tunnel into the network.  Now, you need to focus these
concerns on LISP.  First, Dino claims he made changes in 01 that
attempt to solve your PMTU issues.
I'd appreciate it if you could review lisp-01's discussions of MTU and then focus the discussion as it applies to LISP.

In particular, the draft presents arguments about why LISP packets are
unlikely to run into situations where the MTU is reduced.
How does that argument affect your concerns?

Also, the draft presents both a stateful and stateless mechanism for
handling PMTU concerns.  To what extent do the issues that you bring up apply in each of these cases?

--Sam

---

From: Dino Farinacci <[email-masked]>
Date: Wed, 3 Jun 2009 08:44:24 -0700

Let me add to Sam's reply.

>   effective MTU on the path to the ETR.

Correct. The case for all tunneling protocols.

>   support sustained fragmentation at high data rates).

You can use different source RLOCs if this would ever be a problem.

> 3. Fragmentation in routers performed in slow path

Not true.

> 4. Overhead for reassembly in ETRs is too onerous in some

Nothing in the LISP spec says the ETR reassembles fragments.

>   into inner ICMP messages to send back to the original source

See the spec how we solved this for IPv4 traceroute. The same problem  
exists there. Using the UDP source port you can find the originating  
source, if you wanted to propagate the ICMP Too Big message back to  
the source *at the time the ICMP Too Big was received by the ITR*. The  
stateful solution in the LISP spec says, you cache the effective MTU  
in the map-cache entry and the next packet sent by the source to this  
ITR will result in a ICMP Too Big message sent to the source. A much  
better solution IMO.

>   insufficiently provisioned to allow for authentication

This comment needs to be stated more specifically. Please restate. Any  
datagram protocol will have a difficult time being authenticated. You  
need lots of state and several packet exchanges. The Internet has  
survived quite well without it.

>   ICMP Packet Too Big (PTB) messages coming from the network

ICMP Too Big messages could bombard any system today (without LISP).  
It does not happen for these reasons:

(1) ICMP Too Big messages are rate-limited by all popular router  
implementations.
(2) ICMP Too Big messages are turned off in core routers.
(3) ICMP Too Big messages are filtered at site boundaries so an ITR  
may never see one.

> necessarily share the same channel as the data plane.

How could you expect an xTR at Google/Yahoo/MSN to support such  
control load? We have shown that locator reachability can't scale with  
probing. The same situation arises here too.

Dino

---

From: Eliot Lear <[email-masked]>
Date: Wed, 03 Jun 2009 17:51:38 +0200

>> 3. Fragmentation in routers performed in slow path

I'd like to go further and say that it shouldn't matter whether it (for 
many values of "it") is done in the slow path or fast path TODAY.  It 
CAN be done in the fast path if it is important to do there.

Eliot

---

From: Dino Farinacci <[email-masked]>
Date: Wed, 3 Jun 2009 09:08:42 -0700

> Map-Request if a change has been detected to retrieve the new mapping

I plan to reply to Luigi's email but have a quick reply for this point.

If the version numbers are used to simply identify a change and prompt  
a Map-Request, then why can't a single bit convey the same thing?

The fact that version numbers exist would convey that multiple mapping  
versions are allowed to exist. I am pretty sure that is not a good  
idea and creates other complexities. With a single bit, you indicate  
that the latest state is what is important to cache rather than  
dealing with intermediate states.

Dino

---

From: "Joel M. Halpern" <[email-masked]>
Date: Wed, 03 Jun 2009 12:24:35 -0400

With one bit of state, in the event of even a single packet loss there 
is ambiguity about what the bit means.  (If the sender sets the bit on 
only one packet, and that packet gets lost, the bit has gotten lost. 
Conversely, if the sender sets the bit on a sequence of packets, then 
the receiver has to figure out whether the successive indications are 
for the same change or different changes.

One could, of course, treat the single bit as a two-state wrapping 
counter.  That would address the problem above.  However, if there are 
two successive changes, that would look like no change at all.  This is 
the question the versioning document seems to be looking at in trying to 
decide how big the sequence number needs to be.  But I find the 
reasoning misleading.  Either there is active communication, or there 
isn't.  If there isn't, then timeouts on the mapping itself will prompt 
new queries.  And, the rate that matters is how often changes take 
place.  As long as the product of the rate-of-change and the supported 
idle-time for simplified detection, is less than the wrap, then we are 
fine. SO I suspect that if we want to go with a sequence number type 
behavior, we probably need more than 1 bit, but fewer than 16 (I end up 
at 8 myself.)

However, I think the question of how many bits to use is secondary to 
the question of which conceptual mechanism we want to have.

(Side note: A flapping xTR should be rapidly taken out of service.  O&M 
problem, not a base protocol issue.)

Yours,
Joel

Dino Farinacci wrote:
> If the version numbers are used to simply identify a change and prompt a 
> Map-Request, then why can't a single bit convey the same thing?
> 
> versions are allowed to exist. I am pretty sure that is not a good idea 
> and creates other complexities. With a single bit, you indicate that the 
> latest state is what is important to cache rather than dealing with 
> intermediate states.
> 
> Dino
>

---

From: Dino Farinacci <[email-masked]>
Date: Wed, 3 Jun 2009 09:44:12 -0700

> With one bit of state, in the event of even a single packet loss  
> there is ambiguity about what the bit means.  (If the sender sets  
> the bit on only one packet, and that packet gets lost, the bit has  
> gotten

I said 1 bit. And didn't not mean that 1 packet is sent. The bit can  
stay set in a small packet train of  packets. It works like this:

(1) Site A has a mapping database change.
(2) Site A is currently talking to Site B.
(3) Site A wants to update B's map-cache for itself.
(4) Site A sets the SMR bit in a sequence of packets to Site B. For  
other map-cache entries where the LRU is small, it *does not set the  
SMR-bit* for those sites.
(5) The packet with the SMR-bit set has nonce value <foobar>.
(6) Site B sees the SMR-bit when decapsulating data and tells control  
plane to send a Map-Request with the same nonce value <foobar>.
(7) Site A returns a Map-Reply with nonce value <foobar>.
(8) Site B is updated.

A 3 packet exchange where the first packet traversal piggybacks on  
data. The Site A ITR can control the number of places it receives Map- 
Requests by deciding which map-cache entries to SMR.

> lost. Conversely, if the sender sets the bit on a sequence of  
> packets, then the receiver has to figure out whether the successive  
> indications are for the same change or different changes.

Understand. But a new nonce value from Site A can tell Site B to  
rerequest. That is only going to happen if the changes are on the  
order of milliseconds. That is, within the trip-delay of the 3 packet  
exchange.

> counter.  That would address the problem above.  However, if there  
> are two successive changes, that would look like no change at all.

Interesting. So you say the bit is a counter with 2 values.

>  This is the question the versioning document seems to be looking at  
> in trying to decide how big the sequence number needs to be.  But I  
> find the reasoning misleading.  Either there is active  
> communication, or there isn't.  If there isn't, then timeouts on the  
> mapping itself will prompt new

Assuming time-outs are short.

The default we have in the implementation right now is 24 hours.

> queries.  And, the rate that matters is how often changes take  
> place.  As long as the product of the rate-of-change and the  
> supported idle-time for simplified detection, is less than the wrap,  
> then we are fine. SO I suspect that if we want to go with a sequence  
> number type behavior, we probably need more than 1 bit, but fewer  
> than 16 (I end up at 8 myself.)

Good assertion and conclusion Joel. Agree.

> However, I think the question of how many bits to use is secondary  
> to the question of which conceptual mechanism we want to have.

Right.

> (Side note: A flapping xTR should be rapidly taken out of service.   
> O&M problem, not a base protocol issue.)

Right, agree.

Dino

>> If the version numbers are used to simply identify a change and  
>> prompt a Map-Request, then why can't a single bit convey the same  
>> thing?
>> The fact that version numbers exist would convey that multiple  
>> mapping versions are allowed to exist. I am pretty sure that is not  
>> a good idea and creates other complexities. With a single bit, you  
>> indicate that the latest state is what is important to cache rather  
>> than dealing with intermediate states.
>> Dino

---

From: "Templin, Fred L" <[email-masked]>
Date: Wed, 3 Jun 2009 11:00:54 -0700

Dino,

I will take up Sam's suggestion to comment to the new LISP
text in a different message, but will address your comments
below first:

> -----Original Message-----
> From: Dino Farinacci [mailto:[email-masked]]
> Sent: Wednesday, June 03, 2009 8:44 AM
> To: Templin, Fred L
> Cc: [email-masked]
> Subject: Re: [lisp] Path MTU concerns and proposed resolutions
> 

Right.
 
> You can use different source RLOCs if this would ever be a problem.

The idea I am suggesting is to allow a little bit of
fragmentation initially, but tune it out as quickly as
possible. That way, there is no concern for undetected
reassembly errors (RFC4963) nor burning additional RLOCs.

> > 3. Fragmentation in routers performed in slow path

That's good to know, but its not really relevant to the
LISP discussion - it was merely intended as a general
comment that I will omit from future discussions.

> Nothing in the LISP spec says the ETR reassembles fragments.

I'm now coming to know that you are looking for LISP xTRs
in a variety of deployment scenarios and not just in the
case of global Internet DFZ routers. Whether an ETR would
be asked to reassemble or not has a great deal to do with
the specific deployment scenario. That is why I have
specified SEAL-lite, which asks only that the ETR report
fragmentation and not that it perform reassembly. 
 
> the source *at the time the ICMP Too Big was received by the ITR*. The

Presumably, traceroutes will not be flowing at line rate
and off-path attackers would have little to gain by trying
to disrupt a traceroute nor would they have a way of
guessing that a traceroute is in progress. So, the spec's
suggestion of having the ITR cache traceroute headers is
plausible *for that particular application only*.

For line rate data plane transmissions, however, having
the ITR cache the headers of all of the packets would be
intractable in terms of storage and processing overhead.
The ITR would also have only the 16 bits of the UDP source
port such that a relatively low data rate stream of spoofed
ICMPs from an off-path attacker could guess correctly and
expose a DOS vector. Finally, how could the ITR know when
it is OK to discard the cached headers? This is not a
problem for SEAL, as there is no header caching required. 

> ITR will result in a ICMP Too Big message sent to the source. A much

There is nothing novel nor surprising about asking the
ITR to cache per-ETR MTUs and drop packets that are too
big instead of allowing them into the tunnel - that's
just the way path MTU discovery is intended to work. I
have been saying this for years, and many others have
been saying it for many years before that. This is also
how SEAL works, and also how tunneling standards like
RFC4213 work.

> survived quite well without it.

Authentication is a loaded word and was not intended in
the sense of, e.g. setting up security associations,
inserting an AH header, etc.

What I was meaning to say was (sic) "outer ICMP messages
from routers on the path may be insufficiently provisioned
to provide an adequate nonce".

> (1) ICMP Too Big messages are rate-limited by all popular router

That may well be true, but it could lead to some hosts not
receiving timely MTU feedback. Having an xTR in front of
the hosts could lend to more timely MTU feedback. 

> (2) ICMP Too Big messages are turned off in core routers.

Then how is the ITR going to discover larger MTUs if large
packets are dropped silently by routers in the path from
the ITR to the ETR? Also, I don't find it acceptable for
the ITR to set a too-small upper bound on the tunnel MTU
if there are paths to some ETRs that could have accepted
a larger MTU.

Using SEAL, the ITR can set a large upper bound tunnel
MTU and allow the correct MTU for each ITR->ETR path to
be discovered.

> (3) ICMP Too Big messages are filtered at site boundaries so an ITR

That is one good reason why the MTU feedback should
be conveyed in tunnel protocol-specific error messages
and not ICMPs. 

> probing. The same situation arises here too.

I didn't get that - what control load are you referring to?

Fred

---

From: "Templin, Fred L" <[email-masked]>
Date: Wed, 3 Jun 2009 11:10:20 -0700

Eliot,

> From: Eliot Lear [mailto:[email-masked]]
> Sent: Wednesday, June 03, 2009 8:52 AM
> To: Dino Farinacci
> Cc: Templin, Fred L; [email-masked]

That is fine, and fast path would certainly be nice
in any case. I won't mention this from now on.

Fred

---

From: Olivier Bonaventure <[email-masked]>
Date: Wed, 03 Jun 2009 20:55:32 +0200

Joel,

> for the same change or different changes.

I agree

> behavior, we probably need more than 1 bit, but fewer than 16 (I end up

8 bits could be sufficient, but as you explained above I don't think
that one bit would be enough.

> the question of which conceptual mechanism we want to have.

Completely agree

> problem, not a base protocol issue.)

In an ideal network yes, but in practice in some parts of the network a
flapping link is better than no link at all. Measurements show that
flapping links are unfortunately frequent in IP networks, see e.g. :
http://www.ieee-infocom.org/2004/Papers/48_1.PDF
http://inl.info.ucl.ac.be/publications/achieving-sub-50-milliseconds-recover

Olivier

---

From: "Templin, Fred L" <[email-masked]>
Date: Wed, 3 Jun 2009 13:53:56 -0700

I thought I should mention that there is now a new version
of SEAL called "SEAL-lite". It differs from classical SEAL
in that the ETR never has to reassemble and the ITR never
has to fragment.

SEAL-lite is distinguished from classical SEAL by a
version number in the SEAL header. Both SEAL-lite and
classical SEAL are specified in the same document.
The document can be found here:

http://osprey67.com/seal/draft-templin-seal-24.txt

Fred

---

From: Sam Hartman <[email-masked]>
Date: Wed, 03 Jun 2009 17:32:15 -0400

>>>>> "Fred" == Templin, Fred L <[email-masked]> writes:

    Fred> I thought I should mention that there is now a new version
    Fred> of SEAL called "SEAL-lite". It differs from classical SEAL
    Fred> in that the ETR never has to reassemble and the ITR never
    Fred> has to fragment.

Fred, discussions of SEAL are off-topic for the LISP list.  We're
designing the LISP protocol here.

I appreciate that you believe that LISP has MTU handling issues that
SEAL does not.

You are welcome to contribute to the LISP effort.  The best way to do
that is to respond to my open message to you, showing how the generic
MTU handling concerns apply to LISP.  If we get consensus on that,
working on a solution would be great.

If elements of the solution you propose are drawn from SEAL, that
would be fine.  Obviously, if we adopt any such elements into LISP, we
would acknowledge both your contributions and the SEAL spec.

Meanwhile, please keep a tighter focus on how what you contribute
relates to LISP.

Sam Hartman
for the LISP chairs

---

From: "Templin, Fred L" <[email-masked]>
Date: Wed, 3 Jun 2009 14:37:59 -0700

Acknowledged.

Fred

> From: Sam Hartman [mailto:[email-masked]]
> Sent: Wednesday, June 03, 2009 2:32 PM
> To: Templin, Fred L
> Cc: [email-masked]
> Subject: Re: [lisp] SEAL-lite
>

---

From: "Templin, Fred L" <[email-masked]>
Date: Wed, 3 Jun 2009 15:38:16 -0700

Hi Sam,

Getting back on track now, I took a look at [LISP],
Section 5.4 and am now coming to a useful conclusion.
If we can somehow enlist *only* the ETR in reporting
packet size overflows (i.e., and not accept any ICMPs
from intermediate routers) then there need not be a
stateful mechanism since a stateless mechanism would
suffice.

Let's say that the ETR can be on the lookout for any
packets that arrive in fragments rather than whole
packets. As long as the ETR is diligent in reporting
enough of the packet fragment back to the ITR, the ITR
would have enough information to statelessly translate
it into an ICMP PTB to return to the original source.

So, [LISP] section 5.4.2 can be omitted, and Section
5.4.1 can be rewritten to allow for L to be as large
as the MTU of the interface over which the ITR->ETR
tunnel is configured. Section 5.4.1 would also specify
that the ETR must report fragmentation, and that the
ITR must translate the fragmentation reports into ICMP
PTB messages to return to the original source.

Below is a proposed rewrite of [LISP], Section 5.4.1;
please send comments.

Fred

--- begin ---
5.4.1.  A Stateless Solution to MTU Handling

   An ITR stateless solution to handle MTU issues is described as
   follows:

   1.  Define an architectural constant S for the maximum size of a
       packet, in bytes, an ITR would receive from a source inside of
       its site.

   2.  Define L to be the maximum size, in bytes, a packet of size S
       would be after the ITR prepends the LISP header, UDP header, and
       outer network layer header of size H.

   3.  Calculate: S + H = L.

   When an ITR receives a packet from a site-facing interface and adds H
   bytes worth of encapsulation to yield a packet size of L bytes, it
   resolves the MTU issue by first splitting the original packet into
   fragments, where the first fragment is (1500 - H) bytes and the
   other fragments are no larger than the first fragment.  A LISP header
   is then prepended to each fragment.  This will ensure that the new,
   encapsulated packets are no larger than 1500, which is highly
   unlikely to be above the effective tunnel MTU.

   When an ETR receives encapsulated inner IP fragments, it treats them
   as individually encapsulated packets.  It strips the LISP headers
then
   forwards each fragment to the destination host of the destination
   site.  The fragments are reassembled at the destination host into the
   single IP datagram that was originated by the source host.

   This behavior is performed by the ITR when the source host originates
   a packet with the DF field of the IP header is set to 0.  When the DF
   field of the IP header is set to 1, or the packet is an IPv6 packet
   originated by the source host, the ITR will drop the packet when the
   size is greater than L, and sends an ICMP Too Big message to the
   source with a value of S, where S is (L - H).

   When the outer header encapsulation uses an IPv4 header the ITR
   sets the DF bit to 0. After sending the packet, the ITR may receive
   a Fragmentation Report that contains enough of the first fragment of
   a LISP packet that experienced outer IPv4 fragmentation such that
   the LISP nonce is available and the text of the first-fragment can
   be used for translation into an ICMP Packet Too Big (PTB) message.
   The ITR then generates the ICMP PTB message and returns it to the
   original source.

   When an ETR receives a LISP packet that arrives as multiple outer
   IPv4 fragments, it discards the non-first-fragments and uses the
   first-fragment to generate a Fragmentation Report that it returns
   to the ITR. The ETR must include in the Fragmentation Report enough
   of the first-fragment such that the ITR can verify the nonce and
   translate the report into an ICMP PTB.

   This specification recommends that L be defined as the MTU of
   the interface over which the ITR->ETR tunnel is configured.
--- end ---

> Sent: Wednesday, June 03, 2009 8:30 AM
> To: Templin, Fred L
> Cc: [email-masked]
> I'd appreciate it if you could review lisp-01's discussions of MTU and
then focus the discussion as
> it applies to LISP.

---

From: "Templin, Fred L" <[email-masked]>
Date: Wed, 3 Jun 2009 15:46:16 -0700

> http://osprey67.com/seal/draft-templin-seal-24.txt

FYI, just so that no one is surprised I took the link
down until we can get discussion back on track.

Thanks - Fred

---

From: "Templin, Fred L" <[email-masked]>
Date: Wed, 3 Jun 2009 17:22:08 -0700

Sam,

Beg pardon, but the initial cut at this was not well
aligned with recursive tunneling. Here is another cut
that takes recursion into account:

Fred

--- begin ---

   3.  Calculate: S + H = L.

   When an ITR receives a packet from a site-facing interface and adds H
   bytes worth of encapsulation to yield a packet size of L bytes, it
   resolves the MTU issue by first splitting the original packet into
   fragments, where the first fragment is 1280 bytes and the other
   fragments are no larger than the first fragment.  A LISP header
   is then prepended to each fragment.  This will ensure that the new,
   encapsulated packets are no larger than (1280 + H), which is highly
   unlikely to be above the effective tunnel MTU.

   When an ETR receives encapsulated inner IP fragments, it treats
   them as individually encapsulated packets.  It strips the LISP
   headers then forwards each fragment to the destination host of the
   destination site.  The fragments are reassembled at the destination
   host into the single IP datagram that was originated by the source
   host.

   This behavior is performed by the ITR when the source host originates
   a packet with the DF field of the IP header set to 0. (If the source
   host included an inner LISP header in the packet, this behavior is
   only performed if the Report Fragmentation (RF) field of the inner
   LISP header is set to 0.)  When the DF field of the IP header is set
   to 1, the RF field of the inner LISP header (if present) is set to 1,
   or the packet is an IPv6 packet originated by the source host, the
   ITR will drop the packet when the size is greater than L, and send
   an ICMP Packet Too Big (PTB) message to the source with a value of
   S, where S is (L - H). If the original source was itself a LISP ITR,
   ITR instead sends a LISP Fragmentation Report message (see below).

   When the outer LISP encapsulation uses an IPv4 header the ITR sets
   the DF bit to 0 and sets the RF bit in the outer LISP header to the
   same value that was set in the inner LISP header (if present). If
   no inner LISP header is present, the ITR instead sets the RF bit
   in the outer LISP header to 1 if the DF bit of the inner IP packet
   is set to 1, or if the inner packet is an IPv6 packet.

   After sending the packet, the ITR may receive a LISP Fragmentation
   Report message that contains enough of the first fragment of a LISP
   packet that experienced outer IPv4 fragmentation such that the LISP
   nonce is available and the text of the first-fragment can be used
   for translation into an ICMP PTB message. If the original source was
   itself a LISP ITR, ITR instead translates the Fragmentation Report
   into a Fragmentation Report that the source ITR can interpret.

   When an ETR receives a LISP packet that arrives as multiple outer
   IPv4 fragments, it discards the non-first-fragments. If the RF bit
   is set in the outer LISP header, the ETR then uses the first
   fragment to generate a Fragmentation Report that it returns to
   the ITR. The ETR must include in the Fragmentation Report enough
   of the first-fragment such that the ITR can verify the nonce and
   translate the report into a PTB to send to the original source.

--- end ---

---

From: Luigi Iannone <[email-masked]>
Date: Thu, 4 Jun 2009 09:58:24 +0200

On Jun 3, 2009, at 18:44 , Dino Farinacci wrote:

> (4) Site A sets the SMR bit in a sequence of packets to Site B.

As far as I understand, at this point you need to keep track of the  
fact that B has asked for new mapping in order to reset the SMR bit.
With versioning you do not need this. You just put a new version  
number, B will understand that has to send a map-request and no  
further actions are needed from A.

Let me add something. What happens in the presence of Map-resolver  
that cache the entries? (as proposed in the lisp-ms-01 draft)

If you include the version number in the Map-Request, when the Map- 
resolver receive the packet it will easily understand that it needs as  
well to refresh its cache because a new mapping is available, thus it  
will forward the packet to the xTR.
How to do the same with the SMR bit?
I do not think that keeping trace of all the Map-resolvers that cached  
the mapping is a feasible way.

> For other map-cache entries where the LRU is small, it *does not set  
> the SMR-bit* for those sites.
> (6) Site B sees the SMR-bit when decapsulating data and tells  
> control plane to send a Map-Request with the same nonce value  
> <foobar>.
> data. The Site A ITR can control the number of places it receives  
> Map-Requests by deciding which map-cache entries to SMR.

If you really want you can do the same with versioning.

> rerequest.

What if we call this new nonce a version number? ;-)

> That is only going to happen if the changes are on the order of  
> milliseconds. That is, within the trip-delay of the 3 packet exchange.
>
>> communication, or there isn't.  If there isn't, then timeouts on  
>> the mapping itself will prompt new
>

Just for clarity. With versioning there is no change in the timeouts.

>> supported idle-time for simplified detection, is less than the  
>> wrap, then we are fine.

This is exactly what we were trying to argue in the draft.

>> SO I suspect that if we want to go with a sequence number type  
>> behavior, we probably need more than 1 bit, but fewer than 16 (I  
>> end up at 8 myself.)
>

Agree

Luigi

>>> mapping versions are allowed to exist. I am pretty sure that is  
>>> not a good idea and creates other complexities. With a single bit,  
>>> you indicate that the latest state is what is important to cache  
>>> rather than dealing with intermediate states.
>>> Dino

---

From: Luigi Iannone <[email-masked]>
Date: Thu, 4 Jun 2009 10:08:17 +0200

> prompt a Map-Request, then why can't a single bit convey the same  

Because a single bit doesn't tell you if the change is new compared to  
the actual mapping you have or is a duplicate of you last update.

Assume you receive a packet from X with the SMR bit set, you send a  
Map-Request, then you receive another packet from X with the SMR bit  
set, is this another change? or just the same like the first packet?

You can answer that you have rate limitation and you will ignore SMR  
bit from X for a certain time. Unless this time is very large (which  
is not a good in the case your Map-request is lost) you have no  
guarantee of not receiving a packet with the SMR bit set actually  
referring to the same mapping.

With versioning this ambiguity does not exist anymore.

Luigi

---

From: "Templin, Fred L" <[email-masked]>
Date: Thu, 4 Jun 2009 10:19:40 -0700

Here again is another update - this I think being close
to the final. This version is more adept at handling
recursion, and also accounts for pesky routers that
cause the first fragment to be the smallest when doing
IPv4 fragmentation (albeit with some state required).

Fred

--- begin ---
5.4.1.  MTU Handling

   An ITR solution to handle MTU issues is described as follows:

   1.  Define M to be the minimum of the MTU of the underlying
       interface over which the ITR->ETR tunnel is configured and
       the path MTU across the tunnel (if known).

   2.  Define an architectural constant S for the maximum size of a
       packet, in bytes, an ITR would receive from a source inside of
       its site and admit into the tunnel without first performing
       fragmentation (if the packet is allowed to be fragmented).

   3.  Define L to be the maximum size, in bytes, a packet of size S
       would be after the ITR prepends the LISP header, UDP header, and
       outer network layer header of size H.

   4.  Calculate: S + H = L.

   This specification recommends that L be defined as 1500.

   When an ITR receives a packet from a site-facing interface and adds H
   bytes worth of encapsulation to yield a packet size that is larger
   than L bytes, it resolves the MTU issue by first splitting the
   original packet into fragments. The first fragment must be the
   minimum of 1280 and (M - H) bytes, and the other fragments must be no
   larger than the first fragment. A LISP header is then prepended to
   each fragment.  This will ensure that the new, encapsulated packets
   are unlikely to be above the effective tunnel MTU.

   When an ETR receives encapsulated IP fragments, it treats them as
   individually encapsulated packets.  It strips the LISP headers then
   forwards each fragment to the destination host of the destination
   site.  The fragments are reassembled at the destination host into
   the single IP datagram that was originated by the source host.

   This behavior is performed by the ITR when the source host originates
   a non-LISP packet with the DF field of the IP header set to 0, or a
   LISP packet with the RF field of the LISP header set to 0. For all
   other packets, the ITR will encapsulate the packet and admit it into
   the tunnel if it is no larger than (M - H) bytes. Otherwise, it will
   drop the packet and return a Too Big message (i.e., an ICMP Packet
   Too Big message for non-LISP packets or a LISP Fragmentation Report
   for LISP packets). The message must contain as much of the invoking
   packet as possible without the entire message exceeding the minimum
   MTU (i.e., 576 bytes for IPv4 and 1280 bytes for IPv6).

   When the LISP encapsulation uses an IPv4 header the ITR sets the DF
   bit to 0. If the original packet was itself a LISP packet, the ITR
   sets the RF bit in the encapsulating LISP header to the same value
   that was set in the original LISP header. If the DF bit of the IP
   packet is set to 1, or if the packet is an IPv6 packet, the ITR
   sets the RF bit in the encapsulating LISP header to 1. In all other
   cases, the ITR sets the RF bit to 0.

   After sending the packet, the ITR may receive a LISP Fragmentation
   Report message. If the message contains enough of the packet-in-error
   and a reasonable MTU value (i.e., one no smaller than 576 bytes for
   IPv4 or 1280 bytes for IPv6), the ITR translates it into a Too Big
   message and returns it to the original source. Otherwise, the ITR
   uses a plateau table (e.g., as described in RFC1191) to cache a value
   for M which it iteratively decreases based on subsequent Too Bigs
   until no further Fragmentation Reports are received.

   When an ETR receives a fragmented LISP packet (i.e., one where the
   encapsulated packet experiences fragmentation on the path from the
   ITR), it either reassembles or discards the packet according to its
   configuration. If the RF bit is set to 1 in the outer LISP header,
   the ETR also generates a Fragmentation Report message that it returns
   to the ITR. The message must contain as much of the first-fragment as
   possible without the entire message exceeding the minimum MTU (i.e.,
   576 bytes for IPv4 and 1280 bytes for IPv6).
--- end ---

---

From: Michael Menth <[email-masked]>
Date: Fri, 05 Jun 2009 12:29:17 +0200

Hi,

I have a question regarding LISP+ALT. What information do ETRs respond 
to a map request?
* only one RLOC for the requested EID (the RLOC of the ETR that received 
the request)?
* all RLOCs for the requested EID?
If it's the latter, how does the ETR know about the other RLOCs?
By configuration through the network administrator of the user organization?
If so, does that imply that the ETR is under control of the user 
organization and part of its network?

Regards,

    Michael

-- 
Dr. Michael Menth, Assistant Professor
University of Wuerzburg, Institute of Computer Science
Am Hubland, D-97074 Wuerzburg, Germany, room B206
phone: (+49)-931/31-86644 (new), fax: (+49)-931/8888-6632
mailto:[email-masked]
http://www3.informatik.uni-wuerzburg.de/research/ngn

---

From: David Meyer <[email-masked]>
Date: Fri, 5 Jun 2009 05:53:51 -0700

> * all RLOCs for the requested EID?

	Typically, the Map-Reply will contain all of the RLOCs
	for the site. 

> organization and part of its network?

	Yes, in the case where LISP is fielded in the site's
	CPE. One could also imagine a managed service
	where the SP provides and manages the site's CPE(s). 

	In any event, deploying LISP in CPE-type boxes is
	expected to be a comon scenaro for LISP.

	Dave

---

From: Michael Menth <[email-masked]>
Date: Fri, 05 Jun 2009 15:47:48 +0200

Hi Dave,

is it important that the ETRs are the leaves of the ALT and respond to 
map requests? Or could also a different entitiy of the user organization 
do the same work provided it is reachable via an RLOC and has the 
knowledge about all the RLOCs for this edge network?

I'd like to understand the motivation for the design with the ETRs as 
leave nodes. Is ease of deployment (Every LISP edge network has an ETR 
which has an RLOC, so let the ETR answer the requests!) the only reason? 
Or is there also another reason that I do not see?

Regards,

    Michael

---

From: David Meyer <[email-masked]>
Date: Fri, 5 Jun 2009 07:15:30 -0700

Hey Michael,

> knowledge about all the RLOCs for this edge network?

	As long as there is a way for the Map-Request to find its
	way to whomever is authoritative for the mapping data, that
	entity *could* respond to the map-request. Note however
	that the ETR still needs to have a way of knowing that it
	should decap packets for the destined to EID space behind
	it, so you'd need to be able to provide that information
	to the ETRs in any event.

> Or is there also another reason that I do not see?

	There are a few reasons, including:

	(i).	The site is authoritative for its ingress policy

		We want the site to be authoritative for
		its ingress policy. The natural place to do 
		this is at the site's ETRs.

	(ii).	ETRs need to understand what their EID space is anyway

		Since the ETRs need to be configured to
		understand what their EID space is anyway (so
		they can decap packets and send them on to their
		destinations), they know the EID-to-RLOC mapping
		and so are the natural place to send
		Map-Requests. 

	(iii).	No caching in the ALT/simplicity

		We didn't see (haven't yet seen) the need to do
		caching in the ALT, so the original ALT design
		had the xTRs participating in the ALT directly
		(i.e., BGP peering with the ALT somewhere), so
		having ETRs respond to map-requests was the most
		straightforward approach.

	Hope this helps.

	Dave

---

From: Sam Hartman <[email-masked]>
Date: Tue, 09 Jun 2009 09:33:10 -0400

Fred has made a proposal for changes to the LISP MTU handling
procedure.  Presumably if the WG were to adopt these changes we'd need
to define an RF header bit and a fragmentation report message.

So far, no one besides Fred has indicated support for the proposal.
In order for the proposal to be adopted we'll need to reach a rough
consensus of the WG.  The chairs request that anyone wishing to
express support for this proposal do so by June 12, 2009.

---

From: "Templin, Fred L" <[email-masked]>
Date: Fri, 12 Jun 2009 15:38:06 -0700

Hi Sam,

I have come up with some slight adjustments wrt the LISP
path MTU procedures that I posted on 6/4/09. In particular,
I am no longer asking for a Report Fragmentation (RF) bit,
as each and every LISP packet will be used as an implicit
probe and the ETR will generate a Fragmentation Report if
any fragmentation occurred. Instead, I am asking for a 'D'
bit in the LISP header which is copied from the DF bit
in the inner IP packet.

I am also backing off on the claim that the ITR need not
maintain path MTU state in the general case. This needs
a bit more investigation, but it seems the most robust
approach might be to have the ITR cache the path MTU
and drop too-big packets instead of admitting them into
the tunnel.

Fred

> Sent: Tuesday, June 09, 2009 6:33 AM
> To: [email-masked]
[respond byJune 13]

---

From: Dino Farinacci <[email-masked]>
Date: Sat, 13 Jun 2009 11:00:10 -0700

Sorry for the long delay. See my responses inline.

>> I said 1 bit. And didn't not mean that 1 packet is sent. The bit  
>> can stay set in a small packet train of  packets. It works like this:
>>

You know that when B sends a Map-Request with the nonce from the data  
packet with the SMR-bit. That was our intent.

> further actions are needed from A.

Understand. But I don't want to carry version numbers is each and  
every packet. Since mappings down change at data packet delivery  
granularity, you don't need that information in the data packet.

I was thinking of overloading the nonce field as you suggested in your  
presentation. But if we are going to use the nonces for inline data- 
path liveness, then we can't update a mapping while in nonce echoing  
mode. But let me think about it.

> that cache the entries? (as proposed in the lisp-ms-01 draft)

It was suggested. We don't cache in the map-resolver in the code. If  
you cache anywhere on the ALT you have a even harder problem updating  
mappings. We already have challenges just updating the ITRs, let alone  
everywhere else. So I say, let's leave that alone for now and focus on  
updating the ITRs.

> resolver receive the packet it will easily understand that it needs  
> as well to refresh its cache because a new mapping is available,  
> thus it will forward the packet to the xTR.

I'm thinking that the version number and a nonce serve the same  
purpose. The only difference the version number can show sequence of  
updates. I'm not sure that is needed but all the ITRs care about is  
the latest version. And a changing nonce can tell you that. Because  
when it stops changing, the ETR has conveyed the latest information.

Think about that for a bit.

> How to do the same with the SMR bit?

The SMR-bit says use the nonce as a version number. We don't say that  
explicitly in the spec, but that's how it is used.

> I do not think that keeping trace of all the Map-resolvers that  
> cached the mapping is a feasible way.

Oh agree with you there.

>> For other map-cache entries where the LRU is small, it *does not  
>> set the SMR-bit* for those sites.
>>> packets, then the receiver has to figure out whether the  
>>> successive indications are for the same change or different changes.
>>

Ha ha, we think alike.  ;-)  But as you can see from above the nonce  
is already a version number. ;-)

>>> This is the question the versioning document seems to be looking  
>>> at in trying to decide how big the sequence number needs to be.   
>>> But I find the reasoning misleading.  Either there is active  

Same with the SMR mechanism. If you use SMRs you don't need clock- 
sweep. Clock-sweep is there as an operational technique and SMRs/map- 
versioning are protocol techniques.

We wanted to crawl before we walk and hence the reason for specing  
clock-seep. Plus, it works well for DNS updates when your expectations  
are that the updates will occur tomorrow and not today.

> This is exactly what we were trying to argue in the draft.

We think the rate of updates is going to be small. If in fact, the  
rate of change would be really high as in a mobile-node moving. Like  
maybe when 2 mobile-nodes are moving around quickly and talking to  
each other, when they move, they could clear their cache. That would  
cause things to get fixed (i.e. each would get the new locators).

So in the stationary cases, we think the rate wouldn't be nearly as  
high.

Dino

>>> (Side note: A flapping xTR should be rapidly taken out of  
>>> service.  O&M problem, not a base protocol issue.)
>>
>> Right, agree.
>>
>> Dino
>>
>>>
>>>
>>> Yours,
>>> Joel
>>>
>>> Dino Farinacci wrote:
>>>> not a good idea and creates other complexities. With a single  
>>>> bit, you indicate that the latest state is what is important to  
>>>> cache rather than dealing with intermediate states.
>>>> Dino

---

From: Dino Farinacci <[email-masked]>
Date: Sat, 13 Jun 2009 11:10:22 -0700

> Because a single bit doesn't tell you if the change is new compared  
> to the actual mapping you have or is a duplicate of you last update.

A single bit tells you the same thing as comparing version numbers.  
Plus the ITR doesn't have to store yet another piece of state to know  
what the last version number was the ETR told it.

The nonce will already be stored in the ITR so you can leverage that.

> set, is this another change? or just the same like the first packet?

It is another change if the nonce is different.

> With versioning this ambiguity does not exist anymore.

There is a 3-way handshake. Here is the scenario again. I am an ETR,  
you are in ITR. I want to update you. This is how it goes:

(1) I set SMR-bit with nonce value 17.
(2) You decap data-packet and send me a Map-Request with nonce 17.
(3) I send a Map-Reply with nonce 17.

If the Map-Request is not answered, you retransmit at the rate-limit  
interval. That covers the case when either the Map-Request or Map- 
Reply gets lost.

If during this time, I had *another* mapping change. I would send you  
data packets with nonce value 12 (notice a different value so it  
doesn't matter if the value is larger or smaller than the previous  
nonce) and the SMR-bit set. You can send me a Map-Request now with  
nonce value 12 and when I reply you know which one it is for (12 and  
not 17).

If I get a Map-Request for 17 and know that I want to update you with  
12, I still send you a Map-Reply with nonce value 12. You get it and  
see that you have been requesting 17 but the reply has 12 in it.

At this point you could be confused (not Luigi, he is never confused,  
but the ITR is confused), but since the lastest SMR packet had 12, you  
*know* the Map-Reply for 12 is the latest.

If packets are sent out of sequence, the Map-Reply is always dominate  
in all this. The last Map-Reply you get is always the latest.

Now we can keep the header at 8 bytes and still have versioning. What  
do you think?

Dino

---

From: Sam Hartman <[email-masked]>
Date: Mon, 15 Jun 2009 10:03:05 -0400

>>>>> "Dino" == Dino Farinacci <[email-masked]> writes:
    >> Assume you receive a packet from X with the SMR bit set, you
    >> send a Map-Request, then you receive another packet from X with
    >> the SMR bit set, is this another change? or just the same like
    >> the first packet?

    Dino> It is another change if the nonce is different.

Speaking as an individual, not the chair, can we get the spec to say this clearly?

---

From: Dino Farinacci <[email-masked]>
Date: Mon, 15 Jun 2009 08:13:21 -0700

> Speaking as an individual, not the chair, can we get the spec to say  

Yes, but end of week.

Dino

---

From: Luigi Iannone <[email-masked]>
Date: Sat, 20 Jun 2009 16:36:51 +0200

Hi All,

my comments are inline.

On Jun 13, 2009, at 20:00 , Dino Farinacci wrote:

>>>> the bit on only one packet, and that packet gets lost, the bit  
>>>> has gotten
>>>
> You know that when B sends a Map-Request with the nonce from the  
> data packet with the SMR-bit. That was our intent.
>

How things will go if the Map-Reply to B get dropped somewhere?

Compared to versioning, with your solution you have one additional  
interaction between the control plane and the data plane.

SMR bit is something that you set in the data plane in order to be  
part of the LISP header. Then, the control plane waits for the Map- 
Request from B, once it got that it has to reset the SMR bit in the  
data plane. Isn't it the case?

With versioning you just update the data plane in order to put the new  
version in the LISP header and that's all.

> I was thinking of overloading the nonce field as you suggested in  
> your presentation. But if we are going to use the nonces for inline  
> data-path liveness, then we can't update a mapping while in nonce  
> echoing mode. But let me think about it.

May be I was not clear, sorry. Our intention is not to enlarge the  
LISP header but rather to overload part of it. Actually, we were  
thinking about the reach-bits, not the nonce.

Let me try to explain why by describing how we implemented it in  
OpenLISP:

Assuming I am an OpenLISP ETR, my data plane receives a LISP packet  
that has to be decapsulated and carries a set a reach-bits that do not  
match what I have in the cache.

For security reasons, in OpenLISP, the data plane reach-bits are not  
updated, but through the mapping sockets the control plane is informed  
that for a specific mapping a new set of reach-bits have been received  
(the mapping and the new reach-bits are sent along with the message).

Now, as you suggested in previous discussions, the OpenLISP control  
plane sends a Map-Request in order to verify the information received  
from the data plane.

Hopefully, the OpenLISP ETR will receive a Map-Reply containing reach- 
bits, and you can have three cases:

a) Reach-bits from the Map-Reply match reach-bits received in the LISP  
packet (this is the good case).
b) Reach-bits from the Map-Reply are unchanged from what was in the  
cache but do not match the reach-bits received in the LISP packet.  
(you cannot exclude this case).
c) Reach-bits from the Map-Reply are different from the reach-bits  
that I have in the cache and are different from the reach-bits  
received in the LISP packet (again you cannot exclude this case).

In all of the three cases: a), b) and c) OpenLISP trusts the Mapping  
system because of some form of authentication used (e.g. like for the  
actual implementation of the mapping server), thus the cache is  
updated with the reach-bits of the Map-Reply (this looks like the most  
reasonable thing to do).

In all of the three cases OpenLISP does not care of which specific bit  
has changed, which means that reach-bits have a loose meaning. When an  
ETR receives a no matching reach-bit set it does not matter which bit  
has actually changed, the ETR queries the mapping system and trust  
what it gets.

Following this reasoning, instead of using reach-bits, you put version  
number which just tells the ETR "look, something has changed you need  
to query the mapping system". Whether the versioning has changed from  
3456 to 3457, or from 69 to 70, it does not matter for the ETR. It  
just needs to query the mapping system.

Having that said, in the draft it is explained the benefits introduced  
by putting in the LISP header two version numbers, the one of the  
mapping used to select the source RLOC and the one of the mapping used  
to select the destination RLOC.

Sorry, I know this is quite a long explanation, but I hope that it  
helps in clarifying the discussion.

> you cache anywhere on the ALT you have a even harder problem  
> updating mappings. We already have challenges just updating the  
> ITRs, let alone everywhere else. So I say, let's leave that alone  
> for now and focus on updating the ITRs.

Right.
But with versioning caching is possible (but only if the version  
number is included in the Map-Request).
Further, as I explained in the presentation in the last IETF,  
versioning helps in case of asymmetric traffic flows, where traffic  
from A to B uses a pair of xTRs that are totally different from the  
xTRs used for the traffic from B to A.

[snip]
>>>> One could, of course, treat the single bit as a two-state  
>>>> wrapping counter.  That would address the problem above.   
>>>> However, if there are two successive changes, that would look  
>>>> like no change at all.
>>>

You still need clock-sweep for reach-bit when you want to compact  
them. Right?

Anyway, I have nothing against the clock-sweep. It can be still useful.

[snip]

Luigi

---

From: Luigi Iannone <[email-masked]>
Date: Sat, 20 Jun 2009 17:03:28 +0200

> A single bit tells you the same thing as comparing version numbers.

Disagree. A version number can give you the information of when you  
are up to date. One single bit cannot convey the same amount of  
information unless you consider it as a 1-bit counter as suggested by  
Joel (even if then you can have wrap-around issues).
But this is not the case for the SMR bit.

> Plus the ITR doesn't have to store yet another piece of state to  
> know what the last version number was the ETR told it.
>

As I explained in a previous mail, our idea is to overload some part  
of the LISP header, not to extend it to more than 8 bytes, thus we are  
not adding any state here.

>> Map-Request, then you receive another packet from X with the SMR  
>> bit set, is this another change? or just the same like the first  
>> packet?
>
>> You can answer that you have rate limitation and you will ignore  
>> SMR bit from X for a certain time. Unless this time is very large  
>> (which is not a good in the case your Map-request is lost) you have  
>> no guarantee of not receiving a packet with the SMR bit set  
>> actually referring to the same mapping.
>>

Since I am decapsulating I assume I am the ETR and you  the ITR ;-)

> interval. That covers the case when either the Map-Request or Map- 

Correct me if I am wrong: when you (ITR) receive my (ETR) Map-Request,  
you reset the SMR-bit, while I will keep sending (rate-limited) Map- 
Request until I receive a Map-Reply. Is that right?

> If during this time, I had *another* mapping change. I would send  
> you data packets with nonce value 12 (notice a different value so it  
> If I get a Map-Request for 17 and know that I want to update you  
> with 12, I still send you a Map-Reply with nonce value 12. You get  
> it and see that you have been requesting 17 but the reply has 12 in  
> it.
>
> At this point you could be confused (not Luigi, he is never  
> confused, but the ITR is confused), but since the lastest SMR packet  
> had 12, you *know* the Map-Reply for 12 is the latest.
>
> If packets are sent out of sequence, the Map-Reply is always  
> dominate in all this. The last Map-Reply you get is always the latest.
>

What will the ETR (Luigi, which sometimes gets confused ;-)) do if it  
receives a packet with a different nonce but no SMR bit set?

> Now we can keep the header at 8 bytes and still have versioning.  
> What do you think?
>

Yes, but let me push things a bit .. ;-)
What about splitting the nonce_version number in two parts as  
suggested in our draft? This is very helpful in case of unidirectional  
traffic.

Cheers

Luigi

---

From: "Templin, Fred L" <[email-masked]>
Date: Mon, 22 Jun 2009 10:56:46 -0700

Luigi,

Speaking purely hypothetically (because I'm not yet convinced versioning

is needed), how many bits do you think you would need for this? Would

somewhere around 4 be sufficient?

Thanks - Fred

________________________________

From: Luigi Iannone [mailto:[email-masked]] 
Sent: Saturday, June 20, 2009 8:03 AM
To: Dino Farinacci
Cc: [email-masked]; [email-masked]
Subject: Re: [lisp] Map-Versioning

	Because a single bit doesn't tell you if the change is new
compared to the actual mapping you have or is a duplicate of you last
update.

Disagree. A version number can give you the information of when you are
up to date. One single bit cannot convey the same amount of information
unless you consider it as a 1-bit counter as suggested by Joel (even if
then you can have wrap-around issues). 

As I explained in a previous mail, our idea is to overload some part of
the LISP header, not to extend it to more than 8 bytes, thus we are not
adding any state here.

	Assume you receive a packet from X with the SMR bit set, you
send a Map-Request, then you receive another packet from X with the SMR
bit set, is this another change? or just the same like the first packet?

You can answer that you have rate limitation and you will ignore SMR bit
from X for a certain time. Unless this time is very large (which is not
a good in the case your Map-request is lost) you have no guarantee of
not receiving a packet with the SMR bit set actually referring to the
same mapping.

There is a 3-way handshake. Here is the scenario again. I am an ETR, you
are in ITR. I want to update you. This is how it goes:

	(1) I set SMR-bit with nonce value 17.
	(2) You decap data-packet and send me a Map-Request with nonce
17.

If the Map-Request is not answered, you retransmit at the rate-limit
interval. That covers the case when either the Map-Request or Map-Reply
gets lost.

Correct me if I am wrong: when you (ITR) receive my (ETR) Map-Request,
you reset the SMR-bit, while I will keep sending (rate-limited)
Map-Request until I receive a Map-Reply. Is that right?

If during this time, I had *another* mapping change. I would send you
data packets with nonce value 12 (notice a different value so it doesn't
matter if the value is larger or smaller than the previous nonce) and
the SMR-bit set. You can send me a Map-Request now with nonce value 12
and when I reply you know which one it is for (12 and not 17).

If I get a Map-Request for 17 and know that I want to update you with
12, I still send you a Map-Reply with nonce value 12. You get it and see
that you have been requesting 17 but the reply has 12 in it.

If packets are sent out of sequence, the Map-Reply is always dominate in
all this. The last Map-Reply you get is always the latest.

Now we can keep the header at 8 bytes and still have versioning. What do
you think?

What about splitting the nonce_version number in two parts as suggested
in our draft? This is very helpful in case of unidirectional traffic. 

Cheers

Luigi

Dino

---

From: Sam Hartman <[email-masked]>
Date: Mon, 22 Jun 2009 14:35:13 -0400

The chairs requested indications of support by June 12.  Besides
Fred's initial indication of support, no one else came forward
supporting this issue.  As such, the chairs do not see the necessary
support to adopt these changes and this issue is closed.

---

From: Dino Farinacci <[email-masked]>
Date: Mon, 22 Jun 2009 12:00:30 -0700

> suggested in our draft? This is very helpful in case of  
> unidirectional traffic.

Let's focus on this part for now. Which I think is an important issue.  
Let's frame the situation up as we have site A sending packets to site  
B. The traffic is unidirectional to home in on your point.

Let's call the ITRs at site A ITR "a" and ITR "a'" and "b" and "b'"  
are the ETRs at site B. Even if a or a' send to b or b', those LISP  
routers at site B don't have any map-cache state for site A.

When site B wants to update it's mapping database entries, neither b  
or b' will SMR because it thinks it's not talking to anyone. Plus, it  
has no offered data to send them, so it can't send an SMR-bit.

You like the idea of putting two version numbers in so Site A can tell  
Site B what version of B's mappings it is using.

Tell me if I am ac curatively describing your position?

... (pause and think before continuing below)

If the LISP routers at site B see that anyone is out of date, they  
would need to send something to the ITRs of site A. Well we don't want  
the LISP routers of site B to send a Map-Reply because that could be  
unsolicited. We would want the ITRs to send a Map-Request over the  
mapping database infrastructure to get updated. But we want them to do  
it at the rate that site B can handle the Map-Request load.

So one solution is for the site B LISP routers to send a Map-Request  
to site A with the SMR-bit set. Which in turn has the site A LISP  
routers send a Map-Request. This would fall into the current design.

... (pause and think before continuing below)

But while we have been experimenting with RLOC reachability liveness,  
you might think of an option where site A LISP routers might be  
sending Map-Requests to site B routers to make sure the locators the  
site A routers are using are up and operational. Well depending on how  
often this is done, the keepalive replies which are in the form of Map- 
Replies would have more up to date mapping data.

All I'm trying to say here is that maybe using the basic Map-Request  
and Map-Reply machinery we can solve liveness and fast mapping updates  
at the same time.

Just thinking out loud,
Dino