Subject: [tcpm] Fwd: I-D Action:draft-cheng-tcpm-fastopen-00.txt

From: Yuchung Cheng <[email-masked]>
Date: Fri, 11 Mar 2011 11:04:57 -0800

Hi folks,

We are proposing a new draft called TCP Fast Open (TFO) which allows data
exchange to begin in TCP handshake to improve performance.

We would like to gather some comments before we present it in the next tcpm
meeting in Prague. The presentation will also include some performance
numbers based on our beta implementation in Linux and the Chrome browser.

Thanks,

Yuchung

---------- Forwarded message ----------
From: <[email-masked]>
Date: Mon, Mar 7, 2011 at 4:45 PM
Subject: I-D Action:draft-cheng-tcpm-fastopen-00.txt
To: [email-masked]

A New Internet-Draft is available from the on-line Internet-Drafts
directories.

       Title           : TCP Fast Open
       Author(s)       : Y. Cheng, et al.
       Filename        : draft-cheng-tcpm-fastopen-00.txt
       Pages           : 17
       Date            : 2011-03-07

TCP Fast Open (TFO) allows data to be carried in the SYN or SYN-ACK
packets and consumed by the receiving end during the initial
connection handshake, thus providing a saving of up to one full round
trip time (RTT) compared to standard TCP requiring a three-way
handshake (3WHS) to complete before data can be exchanged.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-cheng-tcpm-fastopen-00.txt

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

_______________________________________________
I-D-Announce mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/i-d-announce
Internet-Draft directories: http://www.ietf.org/shadow.html
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

---

From: "Scheffenegger, Richard" <[email-masked]>
Date: Mon, 14 Mar 2011 11:15:19 -0000

Hi Yuchung,

your proposal will only address new tcp sessions preceeded by one "normal" 3WHS session setup. 

In the draft, you mention that Chrome keeps persistent connections for up to 4 min, while 35% of the requests are made using new sessions; Now I wonder, what fraction of these new sessions are set up towards a server, where the client already had an earlier normal session (or, what is the fraction of persistent connections, that get disconnected within that 4 min timeout due to external reasons such as a NAT gateways running out of resources, or into a timeout...)

Because that 35% value above appears to throw genuinely "new" sessions (new sites) and "lost" sessions (timed out, reset by intermediate device or server etc) to a known host into the same bucket - unless the wording around there is misleading me J

(Since I know how you value even small sub-1% improvements, I do see how this can help you, but I'm wondering if the complexity is worthwhile for a generic tcp stack that deals mostly with non-http'ish traffic).

Best regards, 

Richard Scheffenegger

From: Yuchung Cheng [mailto:[email-masked]] 
Sent: Freitag, 11. März 2011 20:05
To: [email-masked] Extensions
Cc: Mike Belshe; Sivasankar Radhakrishnan; Arvind Jain
Subject: [tcpm] Fwd: I-D Action:draft-cheng-tcpm-fastopen-00.txt

Hi folks,

We are proposing a new draft called TCP Fast Open (TFO) which allows data exchange to begin in TCP handshake to improve performance.

We would like to gather some comments before we present it in the next tcpm meeting in Prague. The presentation will also include some performance numbers based on our beta implementation in Linux and the Chrome browser.

Thanks,

Yuchung

---------- Forwarded message ----------

A New Internet-Draft is available from the on-line Internet-Drafts directories.

       Title           : TCP Fast Open

Internet-Draft <https://www.ietf.org/mailman/listinfo/i-d-announce%0d%0aInternet-Draft>  directories: http://www.ietf.org/shadow.html

---

From: Mark Allman <[email-masked]>
Date: Mon, 14 Mar 2011 12:20:33 -0400

> Now I wonder, what fraction of these new sessions are set up towards a
> server, where the client already had an earlier normal session

Well, I grabbed a connection log from yesterday for a monitoring point
that watches a fiber-to-the-home setup that serves 100 houses in
Cleveland, OH to test this out.  (Nothing magic here ... it just
happened to be handy data.)  For each outgoing TCP connection I looked
at whether there was a previous connection from the current source host
to the current destination host within the last [threshold] seconds.
The results for various thresholds:

            30sec      60sec      120sec      180sec      240sec      Inf
-----------------------------------------------------------------------------
Total       15.3M      15.3M      15.3M       15.3M       15.3M       15.3M
Prev         5.1M       6.1M       7.2M        7.8M        8.2M       15.1M
NoPrev      10.1M       9.2M       8.1M        7.4M        7.1M        145K

(modulo rounding errors; you get the point ...)

Where "Prev" means there was a previous connection in the time interval
and "NoPrev" means there was no previous connection.  The "Inf" column
removes the time threshold and just tests whether there was a previous
connection or not.  The analysis is not perfect.  E.g., the same source
could be used for a few machines behind a NAT in the given house.  But,
it gives a general idea.

> (Since I know how you value even small sub-1% improvements, I do see
> how this can help you, but I'm wondering if the complexity is
> worthwhile for a generic tcp stack that deals mostly with non-http'ish
> traffic). 

As you can see, this is not a (sub-)1% situation in the data I grabbed.
Maybe its a 1% situation somewhere.  And, while I agree that we should
obviously assess cost-v-benefit, setting up strawmen to knock down to
suggest this isn't useful is dubious.)

allman

---

From: William Allen Simpson <[email-masked]>
Date: Mon, 14 Mar 2011 13:21:41 -0400

On 3/14/11 7:15 AM, Scheffenegger, Richard wrote:
> your proposal will only address new tcp sessions preceeded by one “normal” 3WHS session setup.
>
Yes, that's a problem.  Most sessions are one-off.  Especially DNSsec TCP
queries, redirect sessions, etc.

I'd talked to various Google folks last year about their idea to treat the
second session differently, and left it as an open issue for TCPCT.  See
RFC-6013 Acknowledgments.  For idempotent operations, that's not a problem.

So, I'd view this as a follow-on draft to RFC-6013.  Certainly, the second
TCPCT session could send multiple data packets (the <SYN,ACK(SYN)>+data,
plus additional ACK+data packets).  SACK will acknowledge the SYN data, and
TCP itself handle any later holes completely transparently.

But I didn't want to add so much extra detail to TCPCT until we had a good
handle on how well it worked in the field.  We still don't have much data,
mostly because Google didn't follow-up after I'd included their requested
SYN+data features....

> In the draft, you mention that Chrome keeps persistent connections for up to 4 min, while 35% of the requests are made using new sessions; Now I wonder, what fraction of these new sessions are set up towards a server, where the client already had an
> earlier normal session (or, what is the fraction of persistent connections, that get disconnected within that 4 min timeout due to external reasons such as a NAT gateways running out of resources, or into a timeout…)
>
I'm also concerned that re-using the cookies for that long a time isn't
sufficiently secure.

In fact, it feels like they don't really understand the underlying security
issues that are handled by cookies.  We had this same problem over in the
IPsec group with ISAKMP.  It's not *only* a token, it's an anti-clogging
token for preventing DoS attacks.

Just because you stick cookies into a protocol, doesn't make it improve
security in and of itself.  Security first, performance second!

> Because that 35% value above appears to throw genuinely “new” sessions (new sites) and “lost” sessions (timed out, reset by intermediate device or server etc) to a known host into the same bucket – unless the wording around there is misleading me J
>
Indeed, this proposal worries me about other protocol and security issues.
  1) How does it handle path changes that affect options?
  2) How does it prevent use in DoS?

It's an OK rough draft, but needs a lot of work.

---

From: Yuchung Cheng <[email-masked]>
Date: Mon, 14 Mar 2011 11:26:30 -0700

The 35% number does include sessions visiting new sites. But Mark's
number suggests that apps frequently visit same IPs in short
intervals. We also have similar stat. We will update this part of text
to make it more clear. Thanks!

Yuchung

On Mon, Mar 14, 2011 at 4:15 AM, Scheffenegger, Richard <[email-masked]> wrote:
> In the draft, you mention that Chrome keeps persistent connections for up to
> 4 min, while 35% of the requests are made using new sessions; Now I wonder,
> what fraction of these new sessions are set up towards a server, where the
> client already had an earlier normal session (or, what is the fraction of
> persistent connections, that get disconnected within that 4 min timeout due
> to external reasons such as a NAT gateways running out of resources, or into
> a timeout…)
>
>
>
> Because that 35% value above appears to throw genuinely “new” sessions (new
> sites) and “lost” sessions (timed out, reset by intermediate device or
> server etc) to a known host into the same bucket – unless the wording around
> there is misleading me J
>
>
>
>
>
> (Since I know how you value even small sub-1% improvements, I do see how
> this can help you, but I’m wondering if the complexity is worthwhile for a
> generic tcp stack that deals mostly with non-http’ish traffic).
>
>
>
> Best regards,
>
>
>
> Richard Scheffenegger
>
>
>

---

From: William Allen Simpson <[email-masked]>
Date: Mon, 14 Mar 2011 14:36:55 -0400

> It's an OK rough draft, but needs a lot of work.

After a little bit of thought, there are some *VERY* serious problems.
This simply won't work in a heterogeneous environment.  All kinds of
failure modes.  Moreover, I told at least 1 author and 1 listed reviewer
about all of them as long ago as Feb 2010....

  3) The cookie isn't big enough, and there's no cookie pair.  The whole
scheme isn't crytologically sound!

Googlers should go back and look at my presentation, and review the slide:
"Size Matters".  I remember remarking to several folks how disappointing
that folks were more interested in the performance than the security. :-(

Security first, performance second!

  4) Sending a new option doesn't test the ability to send SYN data.  The
fact that you exchange an option doesn't tell you whether the 2nd session
will actually pass data.

TCPCT indicates that SYN data is successful by sending a SACK in the client
<ACK(SYN)>.  The server can cache that signal to indicate that SYN data is
acceptable on the next session.

Without a protocol signal, you end up losing the <SYN,ACK(SYN)> data -- or
worse, the whole packet might be filtered -- and sending extra data segments
that will be discarded.  Over and over again, as the server retransmits.

This leads to congestive collapse!  That's one reason (of many) that TCPCT
Responders never, Never, *NEVER* retransmit SYN segments!

  6) There's no checking for collapsed TCP segments.  There are many boxen
out there with the damn off-loading boards or chips.  Who even knows what
they might do with SYN data and a following segment?!?!

TCPCT checks for collapsed segments with second option, checking known data
fields at the beginning of the option segment.  It doesn't have any good
method of fixing the connection (the handshake simply doesn't complete).

But *not* checking these things could lead to data corruption!

---

From: "Scheffenegger, Richard" <[email-masked]>
Date: Mon, 14 Mar 2011 20:38:17 -0000

Thanks Mark,

I was genuinely wondering what would be the fraction of sessions that could benefit from this proposal; The data you provided and the comment by Arvind indicate, that for normal users performing one specific application (web browsing), a majority of session would benefit.

But it was my understanding that looking at only one single application and optimizing TCP just for that would only make sense, if that impact is large enough to warrant it. 

Best regards,

Richard Scheffenegger

> -----Original Message-----
> From: Mark Allman [mailto:[email-masked]]
> Sent: Montag, 14. März 2011 17:21
> To: Scheffenegger, Richard
> Cc: Yuchung Cheng; [email-masked]; Arvind Jain; Mike Belshe; Sivasankar
> Radhakrishnan
> Subject: Re: [tcpm] Fwd: I-D Action:draft-cheng-tcpm-fastopen-00.txt
> 
>

---

From: Mark Allman <[email-masked]>
Date: Mon, 14 Mar 2011 16:50:44 -0400

> I was genuinely wondering what would be the fraction of sessions that
> could benefit from this proposal; The data you provided and the
> comment by Arvind indicate, that for normal users performing one
> specific application (web browsing), a majority of session would
> benefit. 
> 
> But it was my understanding that looking at only one single
> application and optimizing TCP just for that would only make sense, if
> that impact is large enough to warrant it.  

I didn't say my data was from one application.  In fact, I said:

    For each outgoing TCP connection I looked at whether there was a
    previous connection from the current source host to the current
    destination host within the last [threshold] seconds.

I made no application-layer determination.

Arvind's data may be application-biased by being from Google.  I looked
at a population of users doing whatever it is they do with TCP.

allman

---

From: "Scheffenegger, Richard" <[email-masked]>
Date: Mon, 14 Mar 2011 21:06:10 -0000

Thank you!

Richard Scheffenegger

> Sent: Montag, 14. März 2011 19:27
> Cc: [email-masked]; Mike Belshe; Sivasankar Radhakrishnan; Arvind Jain
> > your proposal will only address new tcp sessions preceeded by one
> "normal"
> > 3WHS session setup.
> >
> >
> >
> > a timeout...)
> > Because that 35% value above appears to throw genuinely "new"
> sessions (new
> > sites) and "lost" sessions (timed out, reset by intermediate device
> or
> > server etc) to a known host into the same bucket - unless the wording
> around
> > there is misleading me J
> >
> >
> >
> >
> >
> how
> > this can help you, but I'm wondering if the complexity is worthwhile
> for a
> > generic tcp stack that deals mostly with non-http'ish traffic).
> >
> >
> >
> > Best regards,
> >
> >
> >
> > Richard Scheffenegger
> >
> >
> >

---

From: Mark Allman <[email-masked]>
Date: Mon, 14 Mar 2011 22:18:07 -0400

Bill-

> Security first, performance second!

This is a lot of "if you were there then you'd know".  But, I wasn't and
so I don't.  Can you frame this complaint in a way that us non-Googlers
might grok, please?

> Responders never, Never, *NEVER* retransmit SYN segments!

This is FUD.  Retransmitting SYNs is done using RTO backoff.  You'd be
hard to pressed to show me a scenario whereby this would cause
congestion collapse.  If this was *the only* traffic in the network,
well maybe, since it is unproductive is being filtered.  On the other
hand, if it is the only traffic in the network nobody cares about
congestion collapse because either way you're not getting anything done.
So, yeah, there would need to be some backoff-to-no-data-in-the-SYN sort
of approach.  But, congestion collapse is a pretty ridiculous assertion
here, IMHO.

> they might do with SYN data and a following segment?!?!

Again, I cannot parse.  I mean, yeah, who knows what these offload
engines do, but what do you mean "collapsing TCP segments"?

allman

---

From: William Allen Simpson <[email-masked]>
Date: Tue, 15 Mar 2011 09:48:41 -0400

> so I don't.  Can you frame this complaint in a way that us non-Googlers
Sorry.  Went through the history of protocols and thinking on amount of
client/initiator/requester unpredictable input to cryptographic hashes;
such as Port (16 bits), PPP's Magic Number (32 bits), Photuris (Port +
128-bit Cookie + other bits), etc.  Mentioned some notable failures,
going back to the Morris worm -- and IKE/ISAKMP.

Also presented "What's in a name?" -- about cookies themselves, the
requirements, etc.

This draft doesn't even use the most rudimentary of principles.  The token
shouldn't be called a "Cookie".  Perhaps "Twinkie" -- all fat and calories,
without nutritional value.

>> Without a protocol signal, you end up losing the<SYN,ACK(SYN)>  data -- or
The initial RTO on a SYN with no history is specified as 3 seconds in
RFC-1122.  If your server sends the second segment 3 seconds after SYN data,
and retransmits at 6 and 12 seconds, I'd be willing to suspend disbelief.  But
I'm pretty sure that's not what the draft has in mind.

Instead, think for a moment about the other end.  The Syn arrives with no
data, the client sends an ACK.  Then an out-of-sequence segment arrives, so
the client sends an ACK.  Then another out-of-sequence segment arrives, and
the client sends an ACK.  Etc.  Now think about ACK clocking and duplicate
ACK handling....

Wrapped your head around that?  Now think about a server that's sending
thousands of these packet trains into AT&T's iPhone network.  Plug *that*
into your network simulator....

Moreover, to entirely avoid use as a DoS source, you have to limit the
initial retransmissions to be triggered by the client/initiator only.
Otherwise, the server becomes a self-perpetuating source (for as many
retransmissions as it's willing to make).

> engines do, but what do you mean "collapsing TCP segments"?
Please read up on TCP Offload Engines and TCP Segmentation Offload.  There
are a bunch of Linux NICs that re-combine smaller segments into larger ones.
Although looking at the terrible code, I'm not sure it ever worked.  Don't
know of any other releases (everybody *I* respect is against it).

But we have to design for it.  We have to design for as many failure modes
as we can imagine, *and* what to do when the unimaginable happens!

---

From: Mark Allman <[email-masked]>
Date: Tue, 15 Mar 2011 13:01:28 -0400

> RFC-1122.

And in RFC2988.  RFC 2988bis uses 1sec.

> If your server sends the second segment 3 seconds after SYN data, and
> retransmits at 6 and 12 seconds, I'd be willing to suspend disbelief.
> But I'm pretty sure that's not what the draft has in mind.

That is precisely what RFC2988 would have you do.  My experience in
looking at packet traces says this is in fact what happens.  I read a
pre-cursor to the fast open draft and I don't recall it changing this,
but perhaps the fast open draft does and, if so, I think it'd be dubious
and at least worthy of a bunch of skepticism.  But, even without that I
seriously doubt you could show collapse.

> the client sends an ACK.  Etc.  Now think about ACK clocking and duplicate

I have thought about this stuff a tremendous amount of the years.  I
don't see what you're getting at here.  As best as I can tell you have
just described TCP's standard behavior.  Yes, an out of order packet
triggers an (duplicate) ACK transmission.  After a few of these the
sender retransmits and slows down.  What is your point?

> There are a bunch of Linux NICs that re-combine smaller segments into
> larger ones.  Although looking at the terrible code, I'm not sure it
> ever worked.  Don't know of any other releases (everybody *I* respect
> is against it).

There are lots of things that muck with packet contents.  I agree that
they need to be thought about / understood / dealt with.  On the other
hand, because there exists some broken 'thing' should not necessarily
prevent forward motion.

allman

---

From: Yuchung Cheng <[email-masked]>
Date: Tue, 15 Mar 2011 10:13:41 -0700

>> 1) How does it handle path changes that affect options?
Do you mean the network path change during a connection or in between
connections? please elaborate.

>> 2) How does it prevent use in DoS?
The entire security consideration is all about dealing with DoS
attacks. It may not be complete. Could you describe the specific
attack that we do not cover?

> scheme isn't crytologically sound!
I don't see how TFO can use cookie-pair to improve the cryptological part and
still achieve our goal. TFO does not share the same goal with TCPCT.

> that folks were more interested in the performance than the security. :-(
I did attend the talk years ago but I couldn't find the slide. Could
you send me
a copy? Thanks.

> fact that you exchange an option doesn't tell you whether the 2nd session
True

> acceptable on the next session.
Instead of sacking the data in SYN-ACK, why can't the client just acknowledge
it cumulatively?

> that will be discarded.  Over and over again, as the server retransmits.

Certainly any payload in SYN/SYN-ACK may got drop anytime. TFO deals with
by retransmitting without the data. When this happens, the data exchange starts
after handshake like regular TCP and TFO receive no latency benefit.

> Responders never, Never, *NEVER* retransmit SYN segments!

The SYN and SYN-ACK max retransmits count are still regulated by RFC1122.
TFO should send the same amount of data packets as regular TCP connection.
Maybe we aren't super clear in the draft but I couldn't see how this cause
congestion collapse?

> they might do with SYN data and a following segment?!?!
Could you explain what "collapsed TCP segments" are? Thanks.

Yuchung

---

From: William Allen Simpson <[email-masked]>
Date: Tue, 15 Mar 2011 14:40:37 -0400

> seriously doubt you could show collapse.
Ah, but it's *not* what this draft would have you do, although it's not
particularly clear about inter-packet timeouts:

    3. Send the SYN-ACK packet acknowledging the SYN and data sequence.
        The server MAY include data in the SYN-ACK packet.

    6. The server MAY send more data packets before the handshake
        completes. The maximum amount is ruled by the initial congestion
        window and the receiver window [RFC3390].

My reading sees *no* waiting before these packets.  Certainly not 3 seconds,
or even 1 second.  Because with waiting, there's no reason to bother with
this scheme; the client <ACK(SYN)> will likely arrive in 3 seconds.

> sender retransmits and slows down.  What is your point?
Please go back and read the draft. They don't only send <SYN,ACK(SYN)> on
retransmit.  They can re-send the whole packet train, with no waiting.
That's *not* TCP's standard behavior!

N.B: the draft uses inconsistent terminology on this point:

[Page 9]
    The network or servers may drop the SYN or SYN-ACK packets with the
    new cookie options which causes SYN or SYN-ACK timeouts. We RECOMMEND
    both the client and the server retransmit SYN and SYN-ACK without the
    cookie options on timeouts.

[Page 10]
    If the SYN-ACK timer fires, the server SHOULD retransmit a SYN-ACK
    packet without data and Fast Open Cookie options for compatibility
    reasons.

Instead, both places must be MUST!

> hand, because there exists some broken 'thing' should not necessarily
As long as it's designed not to cause data corruption, or to detect it
and fail the connection.  This draft doesn't handle it.

---

From: Mark Allman <[email-masked]>
Date: Tue, 15 Mar 2011 15:05:25 -0400

> or even 1 second.  

Yes, OK, but I still don't see your point.  These are *not retransmits*
so why would you apply the RTO?  Neither of the points above pertain to
the case of retransmits, so I completely don't follow your point.

The point of the document is to cut out the 3WHS time [*].  So, you send
the SYN+ACK and then the first window of data.  That window of data is
bounded (per 3390; probably should be by 5681, but same thing
window-size-wise).  And, at that point the host has not actually
detected loss or congestion in the network.  So, there is no reason to
RTO or take any sort of congestion control actions.  If loss is in fact
detected (likely via the RTO) then backoff will kick in.

[*] Note: I am not trying to strongly advocate the fast open scheme
    here.  My complaint is that you're claiming that somehow it is
    leading to congestion collapse without showing how.  And, in fact, I
    think its a dubious claim.

> That's *not* TCP's standard behavior!

If they re-send the entire window then that is not standard TCP and
sounds dubious at first glance.  I do have a bit set to read the
document. 

> >> There are a bunch of Linux NICs that re-combine smaller segments
> >> into larger ones.  Although looking at the terrible code, I'm not
> >> sure it ever worked.  Don't know of any other releases (everybody
> >> *I* respect is against it).
> >
> > There are lots of things that muck with packet contents.  I agree
> > that they need to be thought about / understood / dealt with.  On
> > the other hand, because there exists some broken 'thing' should not
> > necessarily prevent forward motion.
> 

So you're saying Fast Open is designed to cause data corruption?

TCP doesn't handle all data corruption either.  I could increment the
first byte of every TCP payload in a middlebox and adjust the checksum
accordingly and the TCP recipient would be none-the-wiser.  Networks do
stupid things.  Application-layer validation is a good thing (being
closer to the "end" and all).

allman

---

From: Jerry Chu <[email-masked]>
Date: Tue, 15 Mar 2011 23:00:17 -0700

On Mon, Mar 14, 2011 at 10:21 AM, William Allen Simpson <
[email-masked]> wrote:

> So, I'd view this as a follow-on draft to RFC-6013.

TFO and TCPCT have distinctly different goals so I don't see how one could
call
it a follow-on to RFC6013.

Yes we've had some discussion a year ago because after reading your then
still
unpublished and very sketchy draft I thought we could use TCPCT to get what
we
want. But after the meeting we concluded TCPCT is not suitable for us.

Certainly, the second
> mostly because Google didn't follow-up after I'd included their requested

I thought in our meeting we hit a constraint of TCPCT, which can only
accommodate
upto one MSS worth of response data in SYN-ACK, which was a show stopper for
us.
Besides that the design goals/requirement of TCPCT (defense against SYN
attack,
server stateless, security...) were just too different than TFO and hence
introducing
too much complexity that we felt necessary for TFO.

>
>
>  In the draft, you mention that Chrome keeps persistent connections for up
>> to 4 min, while 35% of the requests are made using new sessions; Now I
>> wonder, what fraction of these new sessions are set up towards a server,
>> where the client already had an
>> earlier normal session (or, what is the fraction of persistent
>> connections, that get disconnected within that 4 min timeout due to external
>> reasons such as a NAT gateways running out of resources, or into a timeout…)
>>

Unlike TCPCT, or designing a protocol for bank transaction, TFO is a
performance
project. Without performance there is no TFO so the design constraint for us
is really "getting the performance without compromising security". We spent
months pondering on potential security holes from TFO. (See section 6.)
Obviously we may have missed some, and can benefit greatly from folks
on this list. But please be specific on what we may have missed.

Jerry

>
>
>  Because that 35% value above appears to throw genuinely “new” sessions
>> (new sites) and “lost” sessions (timed out, reset by intermediate device or
> tcpm mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tcpm
>

---

From: Jerry Chu <[email-masked]>
Date: Wed, 16 Mar 2011 00:17:22 -0700

> I'm pretty sure that's not what the draft has in mind.

Like Mark I'm also very confused about your concern about "inter-packet
timeouts" (??)
These are first transmissions not REtransmissions so why would there need to
be
"inter-packet timeouts"?

Other than allowing data to be sent in SYN-SENT and SYN-RCVD states TFO does
NOT alter any TCP standard transmission/retransmission algorithms!

> That's *not* TCP's standard behavior!

Please go back and read the draft and point to us where we deviate from
(or even mention anything related to) the standard TCP retransmission
algorithm?

> Instead, both places must be MUST!

We can debate on whether MUST is really necessary but why would you
interpret
both paragraphs above as "the server, when justified, is allowed to
retransmit the
WHOLE UN-ACKed data"?

> and fail the connection.  This draft doesn't handle it.

I just don't follow your concern. Any buggy middle box or NIC offload
feature
can cause data corruption and any new TCP options/feature may risk some
more vulnerability from buggy h/w. Why would TFO be any worse?

Jerry

---

From: Yoshifumi Nishida <[email-masked]>
Date: Wed, 16 Mar 2011 03:50:50 -0700

Hello Yuchung,

A naive question..
why not setting small value for keep alive interval?
--
Yoshifumi Nishida
[email-masked]

---

From: William Allen Simpson <[email-masked]>
Date: Wed, 16 Mar 2011 09:04:50 -0400

On 3/16/11 2:00 AM, Jerry Chu wrote:
> TFO and TCPCT have distinctly different goals so I don't see how one could call
> Yes we've had some discussion a year ago because after reading your then still
> unpublished and very sketchy draft I thought we could use TCPCT to get what we
Thank you for clarifying.  I misunderstood our prior communications, and
your actions on joining the private TCPCT developers list.  I was unaware
that Google had decided to go its own way, and had been waiting for various
other parties to complete their work.  TCPCT as a specification has been
stable and fully specified for over a year!

Rather than continuing to wait, I'll post my own competing variant of this
proposal as a follow-on to TCPCT for comparison and discussion, as soon as
I've some spare hours.

However, I do not view flooding data on the second connection as important
as reducing all latency by including data on the first connection (and all
subsequent connections).  Much more bang for the buck!

The fact that your proposal does not test whether SYN data works *before*
flooding the data is a serious flaw, as it will in many cases actually
increase the number of RTT.  First do no harm....

I do not consider sacrificing reliability and security for performance a
good trade.

Moreover, this impedes ever adding any other TCP option ever again.  This
actively prevents inclusion of options approved by the IETF and this group
over the past few years.

---

From: "Scheffenegger, Richard" <[email-masked]>
Date: Wed, 16 Mar 2011 14:40:10 -0000

Richard Scheffenegger

> 
> However, I do not view flooding data on the second connection as
> important as reducing all latency by including data on the first
> connection (and all subsequent connections).  Much more bang for
> the buck!

Ever since browsers were allowed to use multiple connections, stepping
back from their use puts your session(s) at a disadvantage... this
behavior is questionable/discussionable, but never the less, fact.
 
> increase the number of RTT.  First do no harm....

Yes, you loose either the data contents of the SYN-flagged segments (1
RTT for recovery), of the full initial flight (also 1 RTT, but more
troublesome to the network).

> I do not consider sacrificing reliability and security for performance
> a good trade.

> Moreover, this impedes ever adding any other TCP option ever again.
> This actively prevents inclusion of options approved by the IETF and
this
> group over the past few years.

What did I miss here again? The Cookie is 4..16 bytes in size - the
option 6..18; What prevents me from using smaller cookies when I want to
use other options too? (MSS+WS+SACK+TS has the very same issue today
also).

Best regards,
  Richard

---

From: Arvind Jain <[email-masked]>
Date: Mon, 14 Mar 2011 09:28:22 -0700

Thanks Mark.

Re. 4 mins timeout on the browser for keeping persistent connections,
note that this proposal doesn't require you to have an established
connection to do the fast open. We can do fast open as long as the
client has a cookie cached from the server. The eviction time for that
could be much longer.

Arvind

---

From: Jerry Chu <[email-masked]>
Date: Wed, 16 Mar 2011 12:06:03 -0700

> stable and fully specified for over a year!

Sorry for any misunderstanding but I thought in our meeting we've made it
clear the response data for our target application is often > 1 MSS. How
would TCPCT, with server stateless achieve that? (See section 7.3 of our
draft for a comparison discussion.)

> subsequent connections).  Much more bang for the buck!

If you had a solution that allows data-in-syn on the first connection and
also
meets all the other requirements and is easy to implement, we would jump on
it! But how could you allow data to be accepted on the server side on the
first
connection while avoiding burning-your-CPU-cycle attack?

> increase the number of RTT.  First do no harm....

The draft already includes discussions in various places on deployment,
robustness, mitigation... This includes empirical data showing the vast
majority of the studied paths allow data-in-SYN, and suggestion to reduce
initRTO to one based on RTT measurement... Moreover, if this still turns
out to be an issue the endhost can always cache a path's data-in-SYN
friendliness. We could try to address some deployment issue in the
protocol itself by it would much complicate the protocol and probably open
up some other can of worms.

> actively prevents inclusion of options approved by the IETF and this group

May I ask how so? (And how is TCPCT with much larger cookies and the exotic
TCP header extension any better?)

Best,

Jerry

---

From: William Allen Simpson <[email-masked]>
Date: Wed, 16 Mar 2011 15:10:13 -0400

> behavior is questionable/discussionable, but never the less, fact.
Perhaps you could enlighten me?  I'm not able to parse "your".

What prevents TCPCT from operating over multiple connections?

> troublesome to the network).
With TCPCT, that doesn't happen.  And since the explicit design goal is
to reduce the RTT, this proposal doesn't meet that goal in that case.

BTW, are you sure that many/most stacks will accept the later segments
after the missing Syn+data segment?  It seems to me they are very likely
to be lost.

Also, don't forget that retransmission will reduce CWND, also defeating
the purpose.  If CWND is already less than maximum, only half or less of
the initial flight of segments will be sent.  They'll await the delayed
Ack, so total RTT at that point should be 3 or 4 extra RTT respectively.

In the worst case, CWND will be at 1 MSS, and it will take many more Acks
to expand.  That could be 8 or more RTT.

> use other options too? (MSS+WS+SACK+TS has the very same issue today
Surely you wouldn't use a 32-bit Twinkie (don't call it a cookie, it doesn't
meet the published requirements) as a security token for session recovery....
That's a birthday attack of only 2**16.

Indeed, with MSS+WS+SACK+TS+UTO, the best we can do now is 112-bit Twinkie.
And there's no room for both Twinkie and AO.

---

From: Yoshifumi Nishida <[email-masked]>
Date: Thu, 17 Mar 2011 00:08:06 -0700

Hi, please allow me to clarify my vague question.
I'm personally would like to know the reason why we cannot reuse TCP
connections more.
If middlebox timeout (NAT table expire) is major reason, I think a
simple solution
is to use small value for tcp's keep-alive interval.
Isn't this useful in the scenario of this draft?

Thanks,
--
Yoshifumi Nishida
[email-masked]

---

From: Yuchung Cheng <[email-masked]>
Date: Thu, 17 Mar 2011 10:31:03 -0700

Hi Yoshifumi,

I contacted someone who studied issues on TCP keep-alives and NAT
after your email but have not receive the reply yet.

I vaguely remembered his study showed
a) in general using TCP keep-alive helps
b) it may not be energy friendly for mobile devices
c) Some broken NAT boxes ignore keep-alive probes (since they don't
carry data?). I had one before :(

On the other hand, servers do close persistent connections (for
resource reasons).
E.g., apache pHTTP default timeout is 5s-15s only.

Yuchung