Subject: [tcpm] Feedback on TCP Fast Open?

From: "Scharf, Michael (Michael)" <[email-masked]>
Date: Fri, 2 Aug 2013 10:10:47 +0000

Hi all,

As mentioned today on the mic, the TCPM working group has a working group item that allows data to be carried in the SYN and SYN-ACK packets and that is consumed by the receiving end during the initial connection handshake, which is saves TCP handshakes. The document was discussed aleady extensively in TCPM and will be updated. TCPM's understanding is that it is not far away from WGLC.

The link is: http://tools.ietf.org/html/draft-ietf-tcpm-fastopen-04

In the spirit of today's session, feedback from the httpbis community would be very appreciated. Please note that this mechanism slightly changes the standard TCP semantics, as explained in the abstract:

   However TFO deviates from the standard TCP semantics in that the data in the
   SYN could be replayed to an application in some rare circumstances.
   Applications should not use TFO unless they can tolerate this issue,
   which is detailed in the Applicability section.

Further information can be found in Section 6 of the document.

Comments or reviews would be highly welcome. The discussion should take place on the TCPM mailing list.

Thanks!

Michael (TCPM co-chair)

---

From: William Chan (陈智昌) <[email-masked]>
Date: Fri, 2 Aug 2013 06:51:31 -0700

Thanks for the invitation for feedback.

I've been the primary contact on the Chromium / Google Chrome network stack
team for Yuchung and the other TCP Fast Open authors at Google, so while I
only lurk in tcpm, I've been reasonably involved in this particular
feature. I've thought a lot about how we would use this in Chromium if
available. For some detailed thoughts intended for a browser implementor
audience, you can read those details at my blog (
https://insouciant.org/tech/some-quick-thoughts-on-tcp-fast-open-and-chromium/
).

The short of it is, for vanilla HTTP, it's unclear how beneficial it would
be for us since we already have such gains for browser preconnect (our
browser feature that learns from past web browsing to speculatively
establish connections, typically just TCP connections but perhaps doing a
TLS or other handshakes too as needed). There's a fundamental tension
between our preconnect feature and TCP Fast Open for vanilla HTTP. Further
experimentation is necessary and we've been working on this. I'm hopeful we
can find a way to leverage this for our gain, but it requires more
investigation.

For TLS over TCP, there is a much more obvious win since we can squeeze the
TLS CLIENT_HELLO into the TCP SYN, and there is no concern about violating
idempotency. This combines with our preconnect feature very nicely, so I am
rather excited about the potential for this use case. While HTTPS adoption
is still low, I'm hopeful that it will increase over time (especially given
the context of recent revelations). So I think the importance of the HTTPS
use case will grow, and thus the potential importance of TCP Fast Open in
decreasing user perceived latency.

Cheers,
Will

On Fri, Aug 2, 2013 at 3:10 AM, Scharf, Michael (Michael) <
[email-masked]> wrote:

> Hi all,
>
> As mentioned today on the mic, the TCPM working group has a working group
> item that allows data to be carried in the SYN and SYN-ACK packets and that
> is consumed by the receiving end during the initial connection handshake,
> which is saves TCP handshakes. The document was discussed aleady
> extensively in TCPM and will be updated. TCPM's understanding is that it is
> not far away from WGLC.
>
> In the spirit of today's session, feedback from the httpbis community
> would be very appreciated. Please note that this mechanism slightly changes
> the standard TCP semantics, as explained in the abstract:
>

---

From: William Chan (陈智昌) <[email-masked]>
Date: Fri, 2 Aug 2013 08:47:05 -0700

Hey Willy!

Your description sounds very familiar. Please see this bug:
https://code.google.com/p/chromium/issues/detail?id=85229.

I am hesitant to open to the door to turning off preconnect in favor of TFO
(although it's definitely on the table). Preconnect has shown substantial
improvements in page load time. Indeed, it's one of our hugest improvements
ever. But I'm super sensitive to harming servers, and would love to get
details from any servers that are encountering issues. Please contact me if
you have details to share here.

And yes, the concern about the reliability of TFO is totally valid. And
that's another reason I'm hesitant to switch preconnect over to TFO because
preconnect Just Works from a client's perspective (yes, it may cost the
server).

Cheers.

On Fri, Aug 2, 2013 at 7:15 AM, Willy Tarreau <[email-masked]> wrote:

> Hi William,
>
> On Fri, Aug 02, 2013 at 06:51:31AM -0700, William Chan (?????????) wrote:
> > TLS or other handshakes too as needed).
>
> That's pretty interesting. Is this already enabled by default ? I'm asking
> because I've got several users of haproxy report me that their web site was
> regularly "attacked" by many connections in which no request is sent, and
> that because of this they had to increase the number of concurrent
> connections
> otherwise they can't stand the load. I asked if they thought it could be
> something like a bug in some JS application or something like this as I was
> no aware of the preconnect feature. It's been a bit hard to analyse, since
> they see no request, they can't get any information on the user agent for
> example. The thing is that it does not look like a regular attack since the
> load is more or less constant, and not very high. So till now it was always
> possible to work around this by increasing the connection limits 2-10
> times.
>
> But now I'm thinking that *if it was a preconnect behaviour*, there could
> possibly be some harm there. I have no idea how many connections a browser
> can send to recently visited sites, but for sites which use a short
> keep-alive
> timeout to limit the concurrency, having a significant increase on the
> number
> of concurrent connections can be a problem.
>
> Note that I'm talking using a conditional form, as I can't provide evidence
> for this to be related to a preconnect feature, but your description really
> matches what I observed, and I am really wondering about the risks and
> possibile impacts based on something that could appear related. If the
> increase in connection count may be significant for small sites, then maybe
> TFO could be a decent alternative (though it will clearly not pass through
> every firewall).
>
> Best regards,
> Willy
>
>

---

From: Willy Tarreau <[email-masked]>
Date: Fri, 2 Aug 2013 16:15:29 +0200

Hi William,

that because of this they had to increase the number of concurrent connections
possible to work around this by increasing the connection limits 2-10 times.

But now I'm thinking that *if it was a preconnect behaviour*, there could
possibly be some harm there. I have no idea how many connections a browser
can send to recently visited sites, but for sites which use a short keep-alive
timeout to limit the concurrency, having a significant increase on the number
of concurrent connections can be a problem.

Best regards,
Willy

---

From: Willy Tarreau <[email-masked]>
Date: Fri, 2 Aug 2013 17:59:09 +0200

> https://code.google.com/p/chromium/issues/detail?id=85229.

Indeed, thanks for the link. Next time I get such a report, I'll pass it.

> you have details to share here.

No problem. Till now that was only speculation by me but as you can guess
it's quite hard to get anything relevant in this situation. Also, users
who experience this tend to enable logging for the first time while debugging
the issue, so you cannot even compare the IP address with the last that was
logged to get info about the UA.

> preconnect Just Works from a client's perspective (yes, it may cost the

I understand. I still find that TFO has some uses on the local network, to
save round trips and packets, in fact to make it as cheap as a keep-alive
connection, but with less permanent states. I intend to experiment with
it between haproxy and some static servers for example. It can be nice
between clients and proxies probably.

I'm having some ideas that could be interesting to investigate with preconnect.
Indeed, instead of keeping a connection in idle for a long time, maybe sending
an "OPTIONS * HTTP/1.1" request would be useful :

  1) it would allow the server to know the UA
  2) it would make the server switch to the keep-alive timeout instead of the
     request timeout, meaning that the connections would probably be maintained
     less time (at least the server would regain the control over the connection
     time).
  3) a loaded server could still disable keep-alive or block these requests and
     save vast amounts of resources
  4) if combined with TFO, you'd get the information in advance about TFO 
     support along the path :-)

If "OPTIONS * HTTP/1.1" doesn't work often enough, then maybe sending a
"GET /favicon.ico" would be more appropriate (and the response could be
cached BTW, saving one round trip later).

Just a few ideas :-)

Willy

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Fri, 02 Aug 2013 20:56:01 +0000

In message <[email-masked]

>is consumed by the receiving end during the initial connection handshake,

Uhm, didn't we try that once with TTCP only to find out that it
opened a major DoS hole ?

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
[email-masked]         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.

---

From: Nico Williams <[email-masked]>
Date: Fri, 2 Aug 2013 16:15:58 -0500

> opened a major DoS hole ?

This one is different.  You must have done one normal exchange and
exchange "key material" (cached on both ends, free to fall off the
cache at any time) to be used for fast opens.

Nico
--

---

From: Peter Lepeska <[email-masked]>
Date: Fri, 2 Aug 2013 19:19:43 -0400

"it's unclear how beneficial it would
> TLS or other handshakes too as needed)"

It would benefit any time you encounter a host that preconnect has not
learned about yet. Surely this provides a benefit. It also has lower cost
since it will result in fewer (zero) connection mistakes since it's not
doing anything speculatively. Don't get me wrong I'm a big fan of the
benefits of speculative prefetching in general, but only in the case where
the underlying protocol can't solve the problem without mistakes.

TCP Fast Open seems great as long as it doesn't introduce any other
problems (such as increased DoS vulnerability).

Peter

Peter

On Fri, Aug 2, 2013 at 5:15 PM, Nico Williams <[email-masked]> wrote:

> > In message
> <[email-masked]

---

From: "Adrien de Croy" <[email-masked]>
Date: Sun, 04 Aug 2013 00:00:40 +0000

we get tech support queries about this as well...

"what are all these connections without a URL showing in activity".

Nice little tech support ticket generator feature.

We figured it was pre-emtive connecting, since eventually a request may 
be made on the connection.

------ Original Message ------
From: "Willy Tarreau" <[email-masked]>
To: "William Chan (?????????)" <[email-masked]>
Cc: "Scharf, Michael (Michael)" <[email-masked]>; 
"[email-masked]" <[email-masked]>; "[email-masked]" 
<[email-masked]>
Sent: 3/08/2013 2:15:29 a.m.
Subject: Re: Feedback on TCP Fast Open?

---

From: "Scharf, Michael (Michael)" <[email-masked]>
Date: Mon, 5 Aug 2013 11:50:59 +0000

Hi Will (and the others in this thread),

Thanks a lot for this excellent feedback and the insights! This is very much appreciated.

@Authors: Actually, I wonder if the specific advantages of TFO for HTTPS could be better described in the document, i.e., in Section 6.3? For instance, by separate applicability sections for HTTP and HTTPS? If idempotency is not an issue for HTTPS, it would be worthwile to explicitly stress that.

Just as a thought...

Michael

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On 
> Behalf Of William Chan (???)
> Sent: Friday, August 02, 2013 3:52 PM
> To: Scharf, Michael (Michael)
> Cc: [email-masked]; [email-masked]
> I've been the primary contact on the Chromium / Google Chrome 
> network stack team for Yuchung and the other TCP Fast Open 
> authors at Google, so while I only lurk in tcpm, I've been 
> reasonably involved in this particular feature. I've thought 
> a lot about how we would use this in Chromium if available. 
> For some detailed thoughts intended for a browser implementor 
> audience, you can read those details at my blog 
> (https://insouciant.org/tech/some-quick-thoughts-on-tcp-fast-o
> pen-and-chromium/).
> 
> The short of it is, for vanilla HTTP, it's unclear how 
> beneficial it would be for us since we already have such 
> gains for browser preconnect (our browser feature that learns 
> from past web browsing to speculatively establish 
> connections, typically just TCP connections but perhaps doing 
> a TLS or other handshakes too as needed). There's a 
> fundamental tension between our preconnect feature and TCP 
> Fast Open for vanilla HTTP. Further experimentation is 
> necessary and we've been working on this. I'm hopeful we can 
> find a way to leverage this for our gain, but it requires 
> more investigation.
> 
> For TLS over TCP, there is a much more obvious win since we 
> can squeeze the TLS CLIENT_HELLO into the TCP SYN, and there 
> is no concern about violating idempotency. This combines with 
> our preconnect feature very nicely, so I am rather excited 
> about the potential for this use case. While HTTPS adoption 
> is still low, I'm hopeful that it will increase over time 
> (especially given the context of recent revelations). So I 
> think the importance of the HTTPS use case will grow, and 
> thus the potential importance of TCP Fast Open in decreasing 
> user perceived latency.
> 
> Cheers,
> Will
> 
> 
> On Fri, Aug 2, 2013 at 3:10 AM, Scharf, Michael (Michael) 
> <[email-masked]> wrote:
> 
> 
> 	Hi all,
> 	
> 	As mentioned today on the mic, the TCPM working group 
> has a working group item that allows data to be carried in 
> the SYN and SYN-ACK packets and that is consumed by the 
> receiving end during the initial connection handshake, which 
> is saves TCP handshakes. The document was discussed aleady 
> extensively in TCPM and will be updated. TCPM's understanding 
> is that it is not far away from WGLC.
> 	
> 	The link is: 
> http://tools.ietf.org/html/draft-ietf-tcpm-fastopen-04
> 	
> 	In the spirit of today's session, feedback from the 
> httpbis community would be very appreciated. Please note that 
> this mechanism slightly changes the standard TCP semantics, 
> as explained in the abstract:
> 	
> 	Comments or reviews would be highly welcome. The 
> discussion should take place on the TCPM mailing list.
> 	
> 	Thanks!
> 	
> 	Michael (TCPM co-chair)
> 	
> 	
> 
> 
>

---

From: Yuchung Cheng <[email-masked]>
Date: Fri, 9 Aug 2013 16:37:56 -0700

> @Authors: Actually, I wonder if the specific advantages of TFO for HTTPS could be better described in the document, i.e., in Section 6.3? For instance, by separate applicability sections for HTTP a…
It's already in -04

http://tools.ietf.org/html/draft-ietf-tcpm-fastopen-04#section-6.3
> _______________________________________________
> tcpm mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tcpm

---

From: "Scharf, Michael (Michael)" <[email-masked]>
Date: Fri, 23 Aug 2013 18:18:19 +0000

Hi Yuchung, inline...

> From: Yuchung Cheng [mailto:[email-masked]] 
> Sent: Saturday, August 10, 2013 1:38 AM
> Cc: William Chan (陈智昌); [email-masked]; [email-masked]
> Subject: Re: [tcpm] Feedback on TCP Fast Open?
> 
> > @Authors: Actually, I wonder if the specific advantages of 
> TFO for HTTPS could be better described in the document, 
> i.e., in Section 6.3? For instance, by separate applicability 
> sections for HTTP and HTTPS? If idempotency is not an issue 
> for HTTPS, it would be worthwile to explicitly stress that.
> It's already in -04
> 

Yes, of course I've seen this. My question is whether advantages of TFO for HTTPS could be ***better*** described in the document, for instance, by ***separate applicability sections*** for HTTP and TLS/HTTPS.

For instance, I was thinking of two separate sections for the content that is now in section 6.3.1: 

6.3.1 Applicability for HTTP

6.3.2 Applicability for TLS/HTTPS

To me, Will's explanation on the applicability to TLS seems useful information, and currently this is somehow hidden at the very end of section 6.3.1. If possible, a reference why idempotency does not matter for TLS would be a great plus - but I am not a TLS expert myself and cannot provide that reference out of my head.

Michael (as individual)

> >> I've been the primary contact on the Chromium / Google 
> Chrome network 
> >> stack team for Yuchung and the other TCP Fast Open authors 
> at Google, 
> >> so while I only lurk in tcpm, I've been reasonably 
> involved in this 
> >> particular feature. I've thought a lot about how we would 
> use this in 
> >> Chromium if available.
> beneficial it 
> >> would be for us since we already have such gains for browser 
> >> preconnect (our browser feature that learns from past web 
> browsing to 
> >> speculatively establish connections, typically just TCP 
> connections 
> >> but perhaps doing a TLS or other handshakes too as 
> needed). There's a 
> Fast Open 
> >> for vanilla HTTP. Further experimentation is necessary and 
> we've been 
> >> working on this. I'm hopeful we can find a way to leverage 
> this for 
> >> our gain, but it requires more investigation.
> >>
> >> For TLS over TCP, there is a much more obvious win since we can 
> >> squeeze the TLS CLIENT_HELLO into the TCP SYN, and there is no 
> >> concern about violating idempotency. This combines with our 
> >> preconnect feature very nicely, so I am rather excited about the 
> >> potential for this use case. While HTTPS adoption is still 
> low, I'm 
> >> hopeful that it will increase over time (especially given 
> the context 
> >> of recent revelations). So I think the importance of the HTTPS use 
> >> case will grow, and thus the potential importance of TCP 
> Fast Open in 
> >>       As mentioned today on the mic, the TCPM working group has a 
> >> working group item that allows data to be carried in the SYN and 
> >> SYN-ACK packets and that is consumed by the receiving end 
> during the 
> >> initial connection handshake, which is saves TCP handshakes. The 
> >> document was discussed aleady extensively in TCPM and will be 
> >> updated. TCPM's understanding is that it is not far away from WGLC.
> >>
> >>       The link is:
> >>       In the spirit of today's session, feedback from the httpbis 
> >> community would be very appreciated. Please note that this 
> mechanism 
> >> slightly changes the standard TCP semantics, as explained in the 
> >> abstract:
> >>
> >>          However TFO deviates from the standard TCP 
> semantics in that 
> >> the data in the
> >>       Comments or reviews would be highly welcome. The discussion 
> >> should take place on the TCPM mailing list.
> >>
> >>       Thanks!
> >>
> >>       Michael (TCPM co-chair)
> >>
> >>
> >>
> >>
> >>