Subject: Re: [tcpm] [tcpinc] TCP's treatment of data in SYN packets

From: Yuchung Cheng <[email-masked]>
Date: Tue, 26 Jul 2016 10:18:00 -0700

On Sun, Jul 24, 2016 at 4:45 PM, Kyle Rose <[email-masked]> wrote:
>
> In the absence of a TFO cookie, what is the behavior of TCP in the presence of data in the SYN packet? The ability of ENO to send early data on session resumption to servers that potentially don't understand ENO depends on the servers throwing this data away so it can be replaced in the following ACK.
>
> RFC 4987 section 3.5 (https://tools.ietf.org/html/rfc4987#section-3.5) states that:
>
> q( If data accompanies the SYN segment, then this data is not
>    acknowledged or stored by the receiver, and will require
>    retransmission. )
>
> But I have read elsewhere that the server might queue it up and wait for the 3WHS to complete (though if it is conformant, it will ACK only a single byte for the SYN and have the blob retransmitted by the client anyway).

We did some tests with this when during TFO development. AFAIK Linux,
*BSD/iOS, Windows all return a SYN that acks only the ISN but not the
data. None of them queues the data hence the active side needs to
retransmit the data. But my knowledge is 5 years old.

FWIW I and other TFO developers in other stacks (cc'd) have all found
that some middle-boxes will crash or hang on receiving SYN-data
(regardless of TFO cookie option or not). For instance, I found a bug
in Linux netfliter that the conntrack module assumes SYN never carries
data, so the internal sequence was only incremented by one on SYN-data
packe, and led to rejecting every packet afterward. Others have found
FW blocking the IP completely (see last meetings material by Praveen).
We were talking about putting a TFO-bis on these issues.

>
> I guess I'm interested in both normative and observed behavior. I'm happy to inquire elsewhere if someone has a suggestion. (tcpm?)
>
> Kyle
>
>
> _______________________________________________
> Tcpinc mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tcpinc

---

From: Kyle Rose <[email-masked]>
Date: Tue, 26 Jul 2016 18:10:43 -0400

> > But I have read elsewhere that the server might queue it up and wait for
> the 3WHS to complete (though if it is conformant, it will ACK only a single
> byte for the SYN and have the blob retransmitted by the client anyway).
>

Which naturally makes me curious if this behavior has changed since TFO was
implemented. What sort of tools do/did you have to do this kind of probing?

> We were talking about putting a TFO-bis on these issues.

This is really good information: thanks! I think if tcpinc decides to
pursue early data, we'll want to learn as much as possible from your
experience.

Kyle

---

From: David Mazieres <[email-masked]>
Date: Wed, 27 Jul 2016 10:35:09 -0700

> pursue early data, we'll want to learn as much as possible from your

So just to scope what we had in mind, "pursue early data" sounds a bit
aggressive.  What I think we had in mind is just saying something about
what receivers MUST do when receiving a SYN segment with data, not
encouraging people to do that.

Since TFO will eventually cause people to address these middlebox
issues, it would be nice to avoid creating a second round of problems
where TCP-ENO implementations also crash or do weird stuff.  And it
would also be nice if TCP-ENO didn't disable this possible optimization.

What I had in mind was something along the lines of saying that the last
TEP suboption in host A's SYN segment is the SYN TEP, and that host B
MUST ignore and MUST NOT ACK any data in SYN segments unless the
negotiated TEP is the SYN TEP and that TEP explicitly defines the use of
data in SYN segments.  Furthermore, SYN data MUST be discarded in the
event that TCP-ENO is disabled, even when such data has already been
ACKed.  TEPs MAY define the use of data in SYN segments for when they
are the SYN TEP, but SHOULD limit such usage to cases in which it is
known that the passive opener and network path both support ENO.

David

---

From: Kyle Rose <[email-masked]>
Date: Wed, 27 Jul 2016 14:11:28 -0400

> known that the passive opener and network path both support ENO.

It's interoperability with older stacks that drove my question: if there
are multiple machines behind an IP, some of which support ENO and some of
which don't, do we break the charter by sending *any* data in the SYN
(outside of TFO, which as we've established is unusable for tcpinc)?

The charter says: q( The protocol must gracefully fall-back to TCP if the
remote peer does not support the proposed extensions. )

If TCP's behavior in the absence of a TFO cookie is to discard data in the
SYN, normatively and (especially) observationally, then your second
sentence ("Furthermore, SYN data MUST be dscarded...") is true when a host
doesn't support ENO, and so we can safely put whatever we like in the SYN.

If not, then unexpected data in the SYN *might* be forwarded to the
application as garbage by some TCP stacks, resulting in mostly-undefined
behavior but almost certainly connection breakage; in that case, do we need
some explicit signaling from the server on a prior connection for when SYN
data on resumption attempts is okay, or is "...but SHOULD limit such
usage..." good enough?

Kyle

---

From: Jeremy Harris <[email-masked]>
Date: Wed, 27 Jul 2016 19:18:34 +0100

On 27/07/16 18:35, David Mazieres wrote:
>  Furthermore, SYN data MUST be discarded in the
> ACKed.

                                   ^^^^^^^^^
This sounds like a recipe for implementation errors and confusion.
-- 
Cheers,
  Jeremy

---

From: Joe Touch <[email-masked]>
Date: Wed, 27 Jul 2016 11:43:57 -0700

> This sounds like a recipe for implementation errors and confusion.

Sounds like a semantics error. TCP should never ACK data it discards. It
is possible to ACK the segment without ACKing the data -- I presume
that's what is intended here.

Joe

---

From: David Mazieres <[email-masked]>
Date: Wed, 27 Jul 2016 14:45:11 -0700

Kyle Rose <[email-masked]> writes:

> do we need some explicit signaling from the server on a prior
> connection for when SYN data on resumption attempts is okay, or is
> "...but SHOULD limit such usage..." good enough?

This is precisely the question on which some input from more experienced
members of the working group could be helpful.

Basically I'd like to ensure that ENO conflicts as little as possible
with future use of data in SYN segments.  So rather than just flat-out
prohibiting it (and having people not even test the corner case), I
think it would be good to specify that receivers explicitly ignore the
data in places where it couldn't possibly be useful.  And then once you
are there, why not say that TEPs MAY define data but probably shouldn't
send it for the time being in the remaining cases.

David

---

From: Derek Fawcus <[email-masked]>
Date: Thu, 28 Jul 2016 00:24:19 +0100

On Wed, Jul 27, 2016 at 02:45:11PM -0700, David Mazieres wrote:
> with future use of data in SYN segments.

My understanding of what TCP _allows_ (absent FO) is that one can send
data in the SYN,  it can be ACK'ed in the SYN-ACK,  but there may be
some text indicating it is not to be delivered to the receiving
application unless and until the 3whs completes.

Implying that a passive opener which chose to ack such data bytes would
have to buffer such data until the handshake completes;  but I'm not
aware of any stack which actually works that way. (Due to DOS attack).

So I'd expect it would probably be safe to start defining some new
expectations / constraints for use of such data in SYN segments.

However,  one could envisage say a user space TCP stack (for a SERVER)
which supported the following sequence:

        CLIENT                                         SERVER

   1)   SYN-SENT     --> <SYN, data1> -->             SYN-RECEIVED

   2)   ESTABLISHED  <-- <SYN, ACK(data1), data2> <-- ESTABLISHED

   3)   FIN-WAIT-1   --> <FIN, ACK(data2)> -->        CLOSE-WAIT

followed by the rest of the normal close sequence.

i.e. something slightly similar to T/TCP but without trying to optimise
the close sequence,  while still allowing a request-response exchange
to be done in 1-RTT if all goes well.

DF

---

From: Derek Fawcus <[email-masked]>
Date: Thu, 28 Jul 2016 07:37:54 +0100

On Thu, Jul 28, 2016 at 12:24:19am +0100, Derek Fawcus wrote:
> aware of any stack which actually works that way.

Except the old KA9Q NOS,  which was tickling my memory...

It did have the ability to send such SYN data.

DF