Subject: [lisp] WG Last Call draft-ietf-lisp-impact-01

From: Luigi Iannone <[email-masked]>
Date: Mon, 9 Mar 2015 15:22:12 +0100

Hi All,

the authors of the LISP Impact document  [https://tools.ietf.org/html/draft-ietf-lisp-impact-01 <https://tools.ietf.org/html/draft-ietf-lisp-impact-01>] requested the Work Group Last Call.

This email starts a WG Last Call, to end March 30th, 2015.
Because usually the pre-meeting period is already overloaded, the LC duration is set to three weeks.

Please review this updated WG document and let the WG know if you agree that it is ready for handing to the AD.
If you have objections, please state your reasons why, and explain what it would take to address your concerns.

Any raised issue will be discussed during the WG meeting in Dallas.

Thanks

Luigi & Joel

---

From: Ross Callon <[email-masked]>
Date: Mon, 30 Mar 2015 01:03:17 +0000

Generally I think that this document needs more work before it will be ready to submit for publication. Some comments on draft-ietf-lisp-impact-01:

First of all, I assume that the TBD at the end of section 3 will be fixed. This reads:

   TBD: add a paragraph to explain the operational difference while
   dealing with a pull model instead of a push.

Also in section 3, the third paragraph begins:

   In addition, Iannone and Bonaventure [IB07] show that the number of
   mapping entries that must be handled by an ITR of a campus network
   with 10,000 users is limited to few tens of thousands, and does not
   represent more than 3 to 4 Megabytes of memory.

Reference [IB07] is of course:

              Iannone, L. and O. Bonaventure, "On the cost of caching locator/id mappings", In
              Proc. ACM CoNEXT 2007, December 2007.

This paper states:

    In our analysis, we assume that the granularity of the
    EID-to-RLOC mapping is the prefix blocks assigned by
    RIRs. We call it /BGP granularity. In particular, we
    used the list of prefixes made available by the iPlane
    Project [15], containing around 240,000 entries. Using
    /BGP granularity means that each EID is first mapped
    on a /BGP prefix. The cache will thus contain /BGP
    to RLOC mappings.2 This is a natural choice, since
    routing locators are supposed to be border routers.

The authors should be aware that there is some aggregation / summarization being done in the operation of BGP routing, and that the granularity of routes which appear in the default-free BGP routing tables is fundamentally different from the granularity of enterprise network / ISP boundaries across which traffic is exchanged.

The same paragraph cites Kim et al.  [KIF11], which is Kim, J., Iannone, L., and A. Feldmann, "Deep dive into the lisp cache and what isps should know about it", In Proc.  IFIP Networking 2011, May 2011.  From this document:

    In addition, we use a local BGP prefixes
    database, fed with the list of BGP prefixes published by the iPlane Project [17].
    The database is used to group EID-to-RLOCs mappings with the granularity of
    existing BGP prefixes, because, as for today, there is no sufficient information
    to predict what will be the granularity of mappings in a LISP-enabled Internet.

I agree that "there is not sufficient information to predict what will be the granularity of mappings in a LISP-enabled Internet". However, not knowing what the mapping granularity will be does not justify using an extremely optimistic guess, and then acting as if the results are meaningful. These assumptions are clearly off by some number of orders of magnitude, but how many orders of magnitude is unknown. We will note that the current internet default-free routing table includes a few hundred thousand entries (roughly twice the 240,000 entries that existed when this study was done).

For example, we might assume that the intended global deployment model involves xTRs at the boundary between enterprise networks and service providers, and might note that there are several million companies in the USA alone (most of these are relatively small companies, of course). Thus there may be very roughly on the order of a hundred million or so companies worldwide. If each one had a separate entry in the mapping table, then the number of entries will be nearly 1,000 times larger than BGP-prefix granularity.

Section 4 mentions as one possible use of LISP: "enable mobility of subscriber end points". If individual end points are advertised into LISP, then the granularity of the mapping table may be on the order of individual systems. In this case the number of mapping table entries that could exist globally might be on the same order of magnitude as the number of people in the world, or very roughly 7 Billion entries. This would suggest that the mapping table might be roughly 30,000 times finer grained than was assumed in the referenced studies.

I don't see how we can accurately predict the control plane load of LISP without some sense for what the granularity of the mapping table will be. It should however be possible to bound the control plane load. The referenced studies give a lower bound on possible control plane load (it won't be any less), but give neither an accurate measurement nor an upper bound on the potential control plane load. I don't think that the document can claim to explain the impact of LISP without there being an attempt to measure an upper bound on the control plane load.

Finally, perhaps I missed it but I didn't see any discussion of the volume of overhead related to OAM traffic used for liveness detection (the need for ITR's to determine the reachability of ETR's).

Thanks, Ross

From: lisp [mailto:[email-masked]] On Behalf Of Luigi Iannone
Sent: Monday, March 09, 2015 10:22 AM
To: LISP mailing list list
Cc: [email-masked]
Subject: [lisp] WG Last Call draft-ietf-lisp-impact-01

Hi All,

the authors of the LISP Impact document  [https://tools.ietf.org/html/draft-ietf-lisp-impact-01] requested the Work Group Last Call.

Thanks

Luigi & Joel

---

From: "Joel M. Halpern" <[email-masked]>
Date: Sun, 29 Mar 2015 21:36:28 -0400

Putting aside the TBD (which of course needs to be fixed),  I have 
trouble figuring out what you want us to say about the main issue in 
your review.  On the one hand, this is the very issue that we have been 
asked to comment on, and on the other hand you say that we don't know. 
What do you think it is reasoanble to say?

Yours,
Joel

On 3/29/15 9:03 PM, Ross Callon wrote:
> Generally I think that this document needs more work before it will be
> ready to submit for publication. Some comments on draft-ietf-lisp-impact-01:
>
> The authors should be aware that there is some aggregation /
> summarization being done in the operation of BGP routing, and that the
> granularity of routes which appear in the default-free BGP routing
> tables is fundamentally different from the granularity of enterprise
> network / ISP boundaries across which traffic is exchanged.
>
> The same paragraph cites Kim et al.  [KIF11], which is Kim, J., Iannone,
> L., and A. Feldmann, "Deep dive into the lisp cache and what isps should
> know about it", In Proc.  IFIP Networking 2011, May 2011.  From this
> document:
>
> I agree that “there is not sufficient information to predict what will
> be the granularity of mappings in a LISP-enabled Internet”. However, not
> knowing what the mapping granularity will be does not justify using an
> extremely optimistic guess, and then acting as if the results are
> meaningful. These assumptions are clearly off by some number of orders
> of magnitude, but how many orders of magnitude is unknown. We will note
> that the current internet default-free routing table includes a few
> hundred thousand entries (roughly twice the 240,000 entries that existed
> when this study was done).
>
> For example, we might assume that the intended global deployment model
> involves xTRs at the boundary between enterprise networks and service
> providers, and might note that there are several million companies in
> the USA alone (most of these are relatively small companies, of course).
> Thus there may be very roughly on the order of a hundred million or so
> companies worldwide. If each one had a separate entry in the mapping
> table, then the number of entries will be nearly 1,000 times larger than
> BGP-prefix granularity.
>
> Section 4 mentions as one possible use of LISP: “enable mobility of
> subscriber end points”. If individual end points are advertised into
> LISP, then the granularity of the mapping table may be on the order of
> individual systems. In this case the number of mapping table entries
> that could exist globally might be on the same order of magnitude as the
> number of people in the world, or very roughly 7 Billion entries. This
> would suggest that the mapping table might be roughly 30,000 times finer
> grained than was assumed in the referenced studies.
>
> I don’t see how we can accurately predict the control plane load of LISP
> without some sense for what the granularity of the mapping table will
> be. It should however be possible to bound the control plane load. The
> referenced studies give a lower bound on possible control plane load (it
> won’t be any less), but give neither an accurate measurement nor an
> upper bound on the potential control plane load. I don’t think that the
> document can claim to explain the impact of LISP without there being an
> attempt to measure an upper bound on the control plane load.
>
> Finally, perhaps I missed it but I didn’t see any discussion of the
> volume of overhead related to OAM traffic used for liveness detection
> (the need for ITR’s to determine the reachability of ETR’s).
>
> Thanks, Ross
>
> *From:*lisp [mailto:[email-masked]] *On Behalf Of *Luigi Iannone
> *Sent:* Monday, March 09, 2015 10:22 AM
> *To:* LISP mailing list list
> *Cc:* [email-masked]
> *Subject:* [lisp] WG Last Call draft-ietf-lisp-impact-01
>
> Hi All,
>
> the authors of the LISP Impact document
>   [https://tools.ietf.org/html/draft-ietf-lisp-impact-01] requested the
> Work Group Last Call.
>
> _______________________________________________
> lisp mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/lisp
>

---

From: Ross Callon <[email-masked]>
Date: Mon, 30 Mar 2015 01:50:04 +0000

I suppose that there are two options:

1. Admit that we have no clue what the cache size or control overhead will be.

2. Repeat the cited studies, but with a pessimistic (worse case) rather than optimistic (better than best case) assumption about cache granularity. 

The second option would allow us to actually have some idea what would happen if LISP were deployed on an Internet-wide scale, but would of course take more time and more work. 

Ross

PS: I will be offline at meetings all of this week, so I might be a bit slower to respond for the next few days.  

-----Original Message-----
From: Joel M. Halpern [mailto:[email-masked]] 
Sent: Sunday, March 29, 2015 9:36 PM
To: Ross Callon; Luigi Iannone; LISP mailing list list
Cc: [email-masked]
Subject: Re: [lisp] WG Last Call draft-ietf-lisp-impact-01

Yours,
Joel

On 3/29/15 9:03 PM, Ross Callon wrote:
> I agree that "there is not sufficient information to predict what will
> be the granularity of mappings in a LISP-enabled Internet". However, not
> Section 4 mentions as one possible use of LISP: "enable mobility of
> subscriber end points". If individual end points are advertised into
> I don't see how we can accurately predict the control plane load of LISP
> won't be any less), but give neither an accurate measurement nor an
> upper bound on the potential control plane load. I don't think that the
> Finally, perhaps I missed it but I didn't see any discussion of the
> (the need for ITR's to determine the reachability of ETR's).
>
> Thanks, Ross
>

---

From: "Joel M. Halpern" <[email-masked]>
Date: Sun, 29 Mar 2015 21:55:19 -0400

Brian, Deborah, if we modified the section to say that we
do not know what the impact on cache size and control overhead will be, 
as the studies that have been made have been disputed as to their 
relevance to the core Internet case, would the document still meet the 
IESG requirements for evaluating the impact.  I don't want to ask the 
authors if they can live with a certain change if that change will cause 
other necessary reviewers to complain.

I do not see it as sensible to hold up the publication of this and the 
other LISP work while researchers go off to see if they can calculate 
better results.  Particularly not when publishing this document is 
required to do other work.

Yours,
Joel

On 3/29/15 9:50 PM, Ross Callon wrote:
> 2. Repeat the cited studies, but with a pessimistic (worse case)
> rather than optimistic (better than best case) assumption about cache
> granularity.
>
> The second option would allow us to actually have some idea what
> would happen if LISP were deployed on an Internet-wide scale, but
> would of course take more time and more work.
>
> Ross
>
> PS: I will be offline at meetings all of this week, so I might be a
> bit slower to respond for the next few days.
>
> -----Original Message----- From: Joel M. Halpern
> [mailto:[email-masked]] Sent: Sunday, March 29, 2015 9:36 PM To:
> Ross Callon; Luigi Iannone; LISP mailing list list Cc:
> [email-masked] Subject: Re: [lisp] WG Last
> Call draft-ietf-lisp-impact-01
>
> your review.  On the one hand, this is the very issue that we have
> been asked to comment on, and on the other hand you say that we don't
> know. What do you think it is reasoanble to say?
>
> Yours, Joel
>
>> Generally I think that this document needs more work before it will
>> be ready to submit for publication. Some comments on
>> draft-ietf-lisp-impact-01:
>>
>> summarization being done in the operation of BGP routing, and that
>> the granularity of routes which appear in the default-free BGP
>> routing tables is fundamentally different from the granularity of
>> enterprise network / ISP boundaries across which traffic is
>> exchanged.
>>
>> The same paragraph cites Kim et al.  [KIF11], which is Kim, J.,
>> Iannone, L., and A. Feldmann, "Deep dive into the lisp cache and
>> what isps should know about it", In Proc.  IFIP Networking 2011,
>> May 2011.  From this document:
>>
>> I agree that "there is not sufficient information to predict what
>> will be the granularity of mappings in a LISP-enabled Internet".
>> However, not knowing what the mapping granularity will be does not
>> justify using an extremely optimistic guess, and then acting as if
>> the results are meaningful. These assumptions are clearly off by
>> some number of orders of magnitude, but how many orders of
>> magnitude is unknown. We will note that the current internet
>> default-free routing table includes a few hundred thousand entries
>> (roughly twice the 240,000 entries that existed when this study was
>> done).
>>
>> For example, we might assume that the intended global deployment
>> model involves xTRs at the boundary between enterprise networks and
>> service providers, and might note that there are several million
>> companies in the USA alone (most of these are relatively small
>> companies, of course). Thus there may be very roughly on the order
>> of a hundred million or so companies worldwide. If each one had a
>> separate entry in the mapping table, then the number of entries
>> will be nearly 1,000 times larger than BGP-prefix granularity.
>>
>> Section 4 mentions as one possible use of LISP: "enable mobility
>> of subscriber end points". If individual end points are advertised
>> into LISP, then the granularity of the mapping table may be on the
>> order of individual systems. In this case the number of mapping
>> table entries that could exist globally might be on the same order
>> of magnitude as the number of people in the world, or very roughly
>> 7 Billion entries. This would suggest that the mapping table might
>> be roughly 30,000 times finer grained than was assumed in the
>> referenced studies.
>>
>> I don't see how we can accurately predict the control plane load of
>> LISP without some sense for what the granularity of the mapping
>> table will be. It should however be possible to bound the control
>> plane load. The referenced studies give a lower bound on possible
>> control plane load (it won't be any less), but give neither an
>> accurate measurement nor an upper bound on the potential control
>> plane load. I don't think that the document can claim to explain
>> the impact of LISP without there being an attempt to measure an
>> upper bound on the control plane load.
>>
>> Finally, perhaps I missed it but I didn't see any discussion of
>> the volume of overhead related to OAM traffic used for liveness
>> detection (the need for ITR's to determine the reachability of
>> ETR's).
>>
>> Thanks, Ross
>>
>> *From:*lisp [mailto:[email-masked]] *On Behalf Of *Luigi
>> Iannone *Sent:* Monday, March 09, 2015 10:22 AM *To:* LISP mailing
>> list list *Cc:* [email-masked] *Subject:*
>> [lisp] WG Last Call draft-ietf-lisp-impact-01
>>
>> Hi All,
>>
>> [https://tools.ietf.org/html/draft-ietf-lisp-impact-01] requested
>> the Work Group Last Call.
>>
>> Please review this updated WG document and let the WG know if you
>> agree that it is ready for handing to the AD.
>>
>> If you have objections, please state your reasons why, and explain
>> what it would take to address your concerns.
>>
>> _______________________________________________ lisp mailing list
>> [email-masked] https://www.ietf.org/mailman/listinfo/lisp
>>
>

---

From: Ronald Bonica <[email-masked]>
Date: Mon, 30 Mar 2015 15:17:58 +0000

Brian, Deborah,

I realize that the authors are frustrated. Maybe we can run with Joel's proposal if we make the disclaimer complete.

Shouldn't the disclaimer read something like, "We do not know the impact of cache size and control overhead on {x, y, z}", where x, y and z might be:

- LISP scaling
- LISP security
- LISP resiliency
- other ???

Given that, might we draw some conclusions regarding LISP's applicability?

                            Ron

> From: lisp [mailto:[email-masked]] On Behalf Of Joel M. Halpern
> Sent: Sunday, March 29, 2015 9:55 PM
> To: Brian Haberman; BRUNGARD, DEBORAH A (ATTLABS)
> Cc: LISP mailing list list
> Brian, Deborah, if we modified the section to say that we do not know what
> the impact on cache size and control overhead will be, as the studies that
> have been made have been disputed as to their relevance to the core
> Internet case, would the document still meet the IESG requirements for
> evaluating the impact.  I don't want to ask the authors if they can live with a
> certain change if that change will cause other necessary reviewers to
> complain.
> 
> I do not see it as sensible to hold up the publication of this and the other LISP
> work while researchers go off to see if they can calculate better results.
> Particularly not when publishing this document is required to do other work.
> 
> Yours,
> Joel
> 
> > The second option would allow us to actually have some idea what would
> > happen if LISP were deployed on an Internet-wide scale, but would of
> > course take more time and more work.
> >
> > Ross
> >
> > [email-masked] Subject: Re: [lisp] WG Last Call
> > draft-ietf-lisp-impact-01
> >
> >> Iannone, L., and A. Feldmann, "Deep dive into the lisp cache and what
> >> isps should know about it", In Proc.  IFIP Networking 2011, May 2011.
> >> From this document:
> >>
> >> the results are meaningful. These assumptions are clearly off by some
> >> number of orders of magnitude, but how many orders of magnitude is
> >> unknown. We will note that the current internet default-free routing
> >> table includes a few hundred thousand entries (roughly twice the
> >> 240,000 entries that existed when this study was done).
> >>
> >> companies, of course). Thus there may be very roughly on the order of
> >> a hundred million or so companies worldwide. If each one had a
> >> separate entry in the mapping table, then the number of entries will
> >> be nearly 1,000 times larger than BGP-prefix granularity.
> >>
> >> LISP, then the granularity of the mapping table may be on the order
> >> of individual systems. In this case the number of mapping table
> >> entries that could exist globally might be on the same order of
> >> magnitude as the number of people in the world, or very roughly
> >> 7 Billion entries. This would suggest that the mapping table might be
> >> roughly 30,000 times finer grained than was assumed in the referenced
> >> studies.
> >>
> >> LISP without some sense for what the granularity of the mapping table
> >> will be. It should however be possible to bound the control plane
> >> load. The referenced studies give a lower bound on possible control
> >> plane load (it won't be any less), but give neither an accurate
> >> measurement nor an upper bound on the potential control plane load. I
> >> don't think that the document can claim to explain the impact of LISP
> >> without there being an attempt to measure an upper bound on the
> >> control plane load.
> >>

---

From: Damien Saucez <[email-masked]>
Date: Tue, 31 Mar 2015 01:19:14 +0200

Ron,

As one of the authors I can say I am not frustrated :-)

We are looking on how to address in the best way the comments
of this thread.

Just that it is unrealistic to hope a new full study on the cache size from
research partners as such a study takes a lot of time and resources (to
produce the experimentation, then to get reviewed in a conference or
journal and finally get published) and would not change any conclusion
with regards to the analytical studies reference in the draft already and
that draw general conclusions.

The best solution could be to ask your customers to provide me the
necessary information so that I could evaluate the impact of LISP
for their needs. Without that, I am afraid that the only thing that we
can reasonably provide is theoretical/analytical studies.

The disclaimer you propose to add is just incorrect. It is not true that
we don’t know the impact of cache size. Actually, in the draft, we
already reference the analytical study [CCD12] that demonstrates
analytically that it is possible to link cache size, traffic pattern, and
miss rate.

Here is the paragraph present in the draft:

[CCD12]  generalizes the caching discussion and proposes an analytic
   model for the EID-to-RLOC cache size when prefix-level traffic has a
   stationary generating process.  The model shows that miss rate can be
   accurately predicted from the EID-to-RLOC cache size and a small set
   of easily measurable traffic parameters, meaning that operators can
   provision the EID-to-RLOC cache of their ITRs according to the miss
   rate they want to achieve for their given traffic.

That is clear enough for me and unless someone can prove the model
wrong I am perfectly fine with this observation that is supported by 
empirical data.

Cheers,

Damien Saucez

---

From: Luigi Iannone <[email-masked]>
Date: Tue, 31 Mar 2015 12:16:09 +0200

Hi All,

the WG LC period for the impact document is now come to and end. 
While there was clear consensus in the room during the last meeting, the latest reviews from 
Ross and Ron raise reasonable points to be clarified before going further.

Hence, the authors will work with Ross and Ron to make a fair description of 
LISP’s impact, and clarifying the assumptions used by the cited research work.

ciao

Luigi 

> On 09 Mar 2015, at 15:22, Luigi Iannone <[email-masked]> wrote:
> 
> Hi All,
>

---

From: Ronald Bonica <[email-masked]>
Date: Tue, 31 Mar 2015 17:48:44 +0000

Damien,

In his email below, Joel asks:

> >>if we modified the section to say that we do not know
> >> what the impact on cache size and control overhead will be, as the
> >> studies that have been made have been disputed as to their relevance
> >> to the core Internet case, would the document still meet the IESG
> >> requirements for evaluating the impact

I support Joel's request, so long as the disclaimer is complete. If cache size and control overhead have impact, they must have impact upon something. Do we know what they will impact? If so, tell us. If not, tell us that.

                                                                                                                           Ron

> From: Damien Saucez [mailto:[email-masked]]
> Sent: Monday, March 30, 2015 7:19 PM
> To: Ronald Bonica
> Cc: Joel Halpern; Brian Haberman; BRUNGARD, DEBORAH A (ATTLABS); LISP
> mailing list list
> We are looking on how to address in the best way the comments of this
> thread.
> 
> journal and finally get published) and would not change any conclusion with
> regards to the analytical studies reference in the draft already and that draw
> general conclusions.
> 
> necessary information so that I could evaluate the impact of LISP for their
> needs. Without that, I am afraid that the only thing that we can reasonably
> provide is theoretical/analytical studies.
> 
> The disclaimer you propose to add is just incorrect. It is not true that we
> don't know the impact of cache size. Actually, in the draft, we already
> reference the analytical study [CCD12] that demonstrates analytically that it is
> possible to link cache size, traffic pattern, and miss rate.
> 
> That is clear enough for me and unless someone can prove the model wrong
> I am perfectly fine with this observation that is supported by empirical data.
> 
> Cheers,
> 
> Damien Saucez
> 
> > I realize that the authors are frustrated. Maybe we can run with Joel's
> proposal if we make the disclaimer complete.
> >
> > Shouldn't the disclaimer read something like, "We do not know the impact
> of cache size and control overhead on {x, y, z}", where x, y and z might be:
> >
> > - LISP scaling
> > - LISP security
> > - LISP resiliency
> > - other ???
> >
> >> Brian, Deborah, if we modified the section to say that we do not know
> >> what the impact on cache size and control overhead will be, as the
> >> studies that have been made have been disputed as to their relevance
> >> to the core Internet case, would the document still meet the IESG
> >> requirements for evaluating the impact.  I don't want to ask the
> >> authors if they can live with a certain change if that change will
> >> cause other necessary reviewers to complain.
> >>
> >> I do not see it as sensible to hold up the publication of this and
> >> the other LISP work while researchers go off to see if they can calculate
> better results.
> >>> rather than optimistic (better than best case) assumption about
> >>> cache granularity.
> >>>
> >>> been asked to comment on, and on the other hand you say that we
> >>> don't know. What do you think it is reasoanble to say?
> >>>
> >>> Yours, Joel
> >>>
> >>>> what isps should know about it", In Proc.  IFIP Networking 2011, May
> 2011.
> >>>> From this document:
> >>>>
> >>>> (roughly twice the
> >>>> subscriber end points". If individual end points are advertised
> >>>> volume of overhead related to OAM traffic used for liveness
> >>>> detection (the need for ITR's to determine the reachability of ETR's).
> >>>>
> >>>> Thanks, Ross
> >>>>
> >>>> _______________________________________________ lisp
> mailing
> >> list