Subject: [tcpm] Question on TCP Fast Open

From: Sara Dickinson <[email-masked]>
Date: Fri, 22 Jan 2016 14:49:10 +0000

Hi, 

I would like to ask for some clarification on the expected behaviour of clients when using TCP Fast Open (RFC7413). 

I’ve observed in inter-op testing of various implementations that after a successful TFO cookie exchange one client (Linux - Kernel 4.4) does not acknowledge data sent by the server in the SYN-ACK in subsequent TFO connections. As a result the server must re-transmit the data, which introduces a delay. RFC7413 clearly states that the server ‘MAY’ send data in the SYN-ACK (Section 4.2.2). My question is for clarification on what clients should do when receiving the SYN-ACK (also Section 4.2.2) :

“ Client: Receiving SYN-ACK

   The client SHOULD perform the following steps upon receiving the SYN-
   ACK:

   1. If the SYN-ACK has a Fast Open option, an MSS option, or both,
      update the corresponding cookie and MSS information in the cookie
      cache.

   2. Send an ACK packet.  Set acknowledgment number to RCV.NXT and
      include the data after SND.UNA if data is available." 

I read this as clients SHOULD acknowledge data in the SYN-ACK if it is present - is that correct? Since this is only a SHOULD what are the reasons a client may choose not to do that?

Regards

Sara.

---

From: Yuchung Cheng <[email-masked]>
Date: Fri, 22 Jan 2016 08:35:58 -0800

> I read this as clients SHOULD acknowledge data in the SYN-ACK if it is present - is that correct? Since this is only a SHOULD what are the reasons a client may choose not to do that?
Linux AFAICT does not implement that (yet). It's that simple :-) but
it's certainly do-able.

The reason Jerry and I didn't implement that was because in our use
cases the (Web) server does not have data available to send when
receiving SYN. But your use case can be different that requires this.
For now, a simple work-around for the server is to send a pure SYN-ACK
followed immediately by a data packet.

I am curious what other implementations do (e.g., IOS).

HTH

>
>
>
> Regards
>
> Sara.
>
>
>
> _______________________________________________
> tcpm mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tcpm
>

---

From: Sara Dickinson <[email-masked]>
Date: Fri, 22 Jan 2016 16:51:51 +0000

> On 22 Jan 2016, at 16:35, Yuchung Cheng <[email-masked]> wrote:
> 

Hi, 

Thanks for the response.

> Linux AFAICT does not implement that (yet). It's that simple :-) but

Right, I thought that might be the case, just wanted to double check my understanding of the spec first :-)

> receiving SYN. But your use case can be different that requires this.

Yes, the application I am testing with is a recursive DNS server which is able to deliver data quickly when the answer to a query is in the cache. 

> I am curious what other implementations do (e.g., IOS).

From my observations the OS X server TFO implementation also waits until after the handshake is complete to send data (but the client does support receiving data in the SYN-ACK). But server side TFO has very recently been implemented in FreeBSD and in that case the server does send the data in the SYN-ACK, which is where I came across the problem. 

Regards

Sara.

---

From: "Scharf, Michael (Nokia - DE)" <[email-masked]>
Date: Fri, 22 Jan 2016 17:05:14 +0000

Just out of curiosity a very unrelated question on TCP Fast Open: Have the different TFO implementations indeed moved from the experimental ID to the option kind 34 registered by IANA?

I am asking since we had discussions on that transition a long time ago, and I wonder if we now have lessons learnt...

Thanks

Michael

---

From: Yuchung Cheng <[email-masked]>
Date: Fri, 22 Jan 2016 09:28:09 -0800

Linux by default uses 34 but also support exp. The client would start with
34. if it didn't work the client retries exp and remembers that in the
cache, and takes 2 connections to get a cookie which is ok. But I am not
sure this is generally applicable for other features using the exp-id opt.

The plan is to retire exp-support after another couple years.

On Fri, Jan 22, 2016 at 9:05 AM, Scharf, Michael (Nokia - DE) <
[email-masked]> wrote:

> Just out of curiosity a very unrelated question on TCP Fast Open: Have the
> different TFO implementations indeed moved from the experimental ID to the
> option kind 34 registered by IANA?
>
> I am asking since we had discussions on that transition a long time ago,
> and I wonder if we now have lessons learnt...
>
> Thanks
>
> Michael
>
>

---

From: Sara Dickinson <[email-masked]>
Date: Fri, 22 Jan 2016 17:29:11 +0000

> On 22 Jan 2016, at 17:05, Scharf, Michael (Nokia - DE) <[email-masked]> wrote:
> 

To my knowledge:

- the OS X and FreeBSD implementations both use only 34

- Linux used only the experimental code up to kernel 4.1 when the default for the client was switched to 34 (with fallback to the experimental option available if that failed) and support on the server for receiving either option.  I don’t believe this was back-ported, so for example Ubunutu 14.04 (LTS release) cannot inter-op with the FreeBSD/OS X implementations. 

Sara.

---

From: Christoph Paasch <[email-masked]>
Date: Sun, 24 Jan 2016 11:53:37 -0800

Hello Sara,

> From my observations the OS X server TFO implementation also waits until after the handshake is complete to send data (but the client does support receiving data in the SYN-ACK). But server side TF…

OSX does send data when it receives a SYN+valid TFO-cookie right away (after
having sent an "empty" SYN/ACK), without waiting for the third ack.
Exactly the same way Linux does it.

Have you seen in your tests that OSX server does not send data right after
the SYN/ACK? Do you have a packet-trace of this observation?

Cheers,
Christoph

---

From: Sara Dickinson <[email-masked]>
Date: Mon, 25 Jan 2016 13:23:55 +0000

> On 24 Jan 2016, at 19:53, Christoph Paasch <[email-masked]> wrote:
>> 

Many thanks for clarifying this - I was going to double check the OS X code but you saved me a job.

> the SYN/ACK? Do you have a packet-trace of this observation?

In my (limited) testing I have only ever seen response data being returned after the 3-way handshake for both Linux and OS X, whereas the same application will send data in the SYN/ACK on FreeBSD. I’ll send you a packet-trace off list. 

Regards

Sara.

---

From: Eric Dumazet <[email-masked]>
Date: Tue, 26 Jan 2016 06:15:16 -0800

On Fri, 2016-01-22 at 14:49 +0000, Sara Dickinson wrote:
> 
> Hi, 
> 
> 
> I would like to ask for some clarification on the expected behaviour
> of clients when using TCP Fast Open (RFC7413). 
> 
> 
> I’ve observed in inter-op testing of various implementations that
> after a successful TFO cookie exchange one client (Linux - Kernel 4.4)
> does not acknowledge data sent by the server in the SYN-ACK in
> subsequent TFO connections. As a result the server must re-transmit
> the data, which introduces a delay. RFC7413 clearly states that the
> server ‘MAY’ send data in the SYN-ACK (Section 4.2.2). My question is
> for clarification on what clients should do when receiving the SYN-ACK
> (also Section 4.2.2) :
> 
> 
> I read this as clients SHOULD acknowledge data in the SYN-ACK if it is
> present - is that correct? Since this is only a SHOULD what are the
> reasons a client may choose not to do that?
> 
> 
Hi Sara.

I implemented the linux patch to accept the payload included in SYNACK
message.

I would advise a server adds payload in SYNACK only if it has 4+ MSS
worth of data to send, so that fast retransmit could repair the damage
caused by lack of client support.

---

From: Eric Dumazet <[email-masked]>
Date: Tue, 26 Jan 2016 10:26:24 -0800

> caused by lack of client support.

Or, when server receives the ACK packet coming from client, it can
immediately tell if the SYN-ACK payload was acked or not.

If not -> immediately send the data that was in the SYN-ACK, without
waiting for a timeout.