Subject: [tcpm] rto-consider updated

From: Mark Allman <[email-masked]>
Date: Wed, 21 Oct 2015 07:43:38 -0400

Folks-

I have been cleaning up some old items on my 'todo' list.  One of
them is an old draft I wrote on RTO considerations.  The idea is to
provide a set of high-level guidelines that estimators must follow,
but that leaves them free to set low-level details without being
standardized.  My view is this is mostly writing down what happens
now.

When I originally posted the draft several years ago it garnered
little traction.

My todo is to issue this as a tech report so that it lives somewhere
(and I can reference the idea and such).  But, I figured before I
did that I'd throw it up as an I-D again and make sure folks were
not interested.

So, I tried to to do that yesterday, but we are in the "no I-D
postings" period.  So, the (very short) I-D is here:

    http://www.icir.org/mallman/pubs/All15a/

I have a bit set to submit it when submissions are again accepted
(Nov 1).

Comments welcome, of course.  If the WG is interested, great.  If
not, that is fine and I won't say any more about it.

Thanks!

allman

--
http://www.icir.org/mallman/

---

From: Yuchung Cheng <[email-masked]>
Date: Wed, 21 Oct 2015 11:38:40 -0700

I am interested.

Following the spirit of reflecting the reality, I have some random
points after skimming your draft:

* (3) MUST exp. backoff: I agree on expback off is needed for safety.
But MUST "eventually" exp backoff reflects the reality better. AFAIK
Linux has the "thin-stream" optimization that optionally use linear
back for the first 6? retries. And I recall vaguely some stacks do so
based on some (old) research paper.

* Would be nice to talk about the well-known tricks for data-center
communications to lower the delayed ACK and RTO.

* Knowing the delayed ACK time of the remote matters. Some RFC (1122?)
suggests 500ms. I think everyone uses lower value 40ms - 200ms.

* I saw lots of (naive) attempts to lower the RTO for latency, it
would be nice to talk about the implication of spurious RTO.
  One best example is https://insouciant.org/#Google_Kitten_Search
that the "web-style" incast leads to massive spurious retransmission
and buffer bloat. This would not happen if the connection supports
Timestamp and the sender implements Eifel. but not all stacks support
TS (e.g., Windows).

On Wed, Oct 21, 2015 at 4:43 AM, Mark Allman <[email-masked]> wrote:
> _______________________________________________
> tcpm mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tcpm
>

---

From: Mark Allman <[email-masked]>
Date: Wed, 21 Oct 2015 22:55:47 -0400

> I am interested.

:)

> Following the spirit of reflecting the reality,

The reality I meant to reflect was that the details of RTO
implementations have diverged a bit from anything that is
standardized.  And, that seems to be OK.  So, by standard it seems
to me that perhaps we should just decide there are a few things that
we'd like to see hold and not pretend that all the details are
important enough to tightly specify.  Because the world has shown us
that this is simply not true.

> But MUST "eventually" exp backoff reflects the reality
> better. AFAIK Linux has the "thin-stream" optimization that
> optionally use linear back for the first 6? retries. And I recall
> vaguely some stacks do so based on some (old) research paper.

I dunno ... I guess I still think exponential backoff is the
ultimate bit that protects us and so I am a little wary of adding
too many wiggle words here (the draft does suggest that exponential
doesn't have to be 2x).  But, if the WG wants to take this up and
the WG consensus is for some sort of eventual exponential backoff
then I guess that is OK.

> communications to lower the delayed ACK and RTO.

This is what I don't really want to talk about.  I don't want to
codify tricks.  I want to give an umbrella under which various
estimators could flourish.

> suggests 500ms. I think everyone uses lower value 40ms - 200ms.

Well, ...

  - I am not sure what you're proposing.  Some option to let the
    receiver give the delayed ACK time?  Or, a "delayed" / "not
    delayed" bit?  Whatever it is I think it is beyond the scope of
    high level guidance.

  - This does bring to mind a pet peeve which I have sort of fallen
    into.  We talk about the RTO in terms of RTT, but it really
    should be in terms of expected feedback time.  I.e., the sender
    should not care about the delayed ACK timer.  But, the estimator
    needs to understand when to expect an ACK for a given piece of
    data.  That ACK might be delayed.  Or, it might not.  But,
    calling the time between data transmission and ACK reception the
    "RTT" is sort of mis-leading because it isn't just network
    latency we care about.

> would be nice to talk about the implication of spurious RTO.

I try to sketch this tradeoff.  I guess I could go back and see if
it could be improved.

Thanks!

allman

---

From: Yuchung Cheng <[email-masked]>
Date: Thu, 22 Oct 2015 11:52:58 -0700

> estimators could flourish.
OK. but worth articulate these non-goals in the draft.

>     "RTT" is sort of mis-leading because it isn't just network
agreed. that's also worth discussing in the draft. perhaps a fallacy
on designing "RTT" estimator.

Another point to consider: Linux is moving toward the model of
aggressive timeout to probe (TLP) then a conservative timeout to reset
cwnd when entire flight is surely lost. AFAIK QUIC also takes this
approach. https://tools.ietf.org/html/draft-tsvwg-quic-loss-recovery-00
I don't know where to fit in your draft, just FYI.

---

From: Mark Allman <[email-masked]>
Date: Mon, 26 Oct 2015 07:54:36 -0400

> OK. but worth articulate these non-goals in the draft.

I thought the goal was pretty clear, but I will certainly revisit
it. 

> on designing "RTT" estimator.

Yep.  It is on my list to tease this out.

(That said, these are great comments, but I am not seeing much
interest.  So, they may well just be reflected in my tech report.)

Thanks,
allman

---

From: "Scharf, Michael (Michael)" <[email-masked]>
Date: Mon, 26 Oct 2015 12:32:26 +0000

> -----Original Message-----
> From: tcpm [mailto:[email-masked]] On Behalf Of Mark Allman
> Sent: Monday, October 26, 2015 12:55 PM
> To: Yuchung Cheng
> Cc: Jana Iyengar; Ian Swett; [email-masked] Extensions
> Subject: Re: [tcpm] rto-consider updated
> 
> 

For what it is worth, if people (in particular in DCs) play with the RTO estimator, I think it would be useful to have a TCPM document that provides some guidelines on what is possible and what one should better avoid. This document seems like a good starting point for that.

I don't think we should codify tricks. But we may have to talk a bit more specifically about what people can or cannot do if they (believe to) know the RTT is small. To me, this could be an excellent value proposition for a TCPM document.

Michael (as random TCPM contributor)

---

From: Ted Faber <[email-masked]>
Date: Tue, 27 Oct 2015 08:25:27 -0700

On 10/26/2015 04:54 AM, Mark Allman wrote:
> 
>>>> * Would be nice to talk about the well-known tricks for
>>>> data-center communications to lower the delayed ACK and RTO.
>>> 
>>> This is what I don't really want to talk about.  I don't want
>>> to codify tricks.  I want to give an umbrella under which
>>> various estimators could flourish.
>> 

I did, too.

> 
>> agreed. that's also worth discussing in the draft. perhaps a
>> fallacy on designing "RTT" estimator.
> 

I'd like to see it published.  IMHO, it is ready for last call.

- -- 
Ted Faber, Computer Scientist, USC/ISI
http://www.isi.edu/~faber
http://www.isi.edu/~faber/contact.html