Subject: [tcpm] Other places for extended TCP options in SYN-only packets

From: David Borman <[email-masked]>
Date: Tue, 11 Nov 2014 13:07:36 -0600

I’ve been trying to figure out some other place to put extended TCP options in a SYN-only packet.  For a standard IPv4/TCP packet, because there is only one length field, there is only one place to put additional TCP option space that will be ignored by legacy systems: encode it into a new IPv4 option.  But I’ll be the first to admit this isn’t a great solution:
  1) IPv4 is also limited to just 40 bytes of option space
  2) IPv4 options get looked at by every hop on the path
  3) The presence of IPv4 options would probably cause slow path processing in routers.
The slow path processing is mitigated by the fact that this only applies to SYN-only packets.  But providing less than 40 additional bytes makes this very uninteresting.

If only TCP had an explicit length field, we could put extended option space after the TCP data.  And there is a way to do that: encapsulate the IPv4/TCP packet, such that the outer IPv4 header had a length that included both the inner IPv4/TCP packet, plus the extended options, and the inner IPv4/TCP packet length excluded the extended options.  Systems that don’t understand the extended options would just see the inner IPv4/TCP header, and ignore the trailing data from the outer IPv4 packet.  The obvious problem with this is that I wouldn’t expect it to work with legacy systems, because it depends on systems already having IPv4-in-IPv4 support, which I doubt, since IPv4-in-IPv4 is mainly used to create tunnels.

But then I started thinking about IPv6/TCP.  Additional TCP option space could be encoded into an IPv6 Destination Options header.  The advantages of this over IPv4 are:
  1) There is almost 2K of space; though each option is limited to 255 bytes, multiple options could be allowed if that became necessary.
  2) Destination Options headers are not processed at intermediate hops.
  3) With the top 2 bits of the option type set to zero, it will be ignored by hosts that don’t understand it.
  4) Hosts that don’t understand it will just see it as a normal TCP SYN-only packet.

So for IPv6, it is possible to transmit the expanded TCP option space in the same packet in a manner that won't confuse legacy IPv6/TCP implementations.

A solution for extended TCP option space in SYN-only packets doesn’t need to be the same for IPv4/TCP and IPv6/TCP.  If we can have a clean solution for IPv6, we should do that rather than force a less-than desireable IPv4 solution on IPv6 connections.

			-David Borman

---

From: Florian Westphal <[email-masked]>
Date: Tue, 11 Nov 2014 20:41:52 +0100

David Borman <[email-masked]> wrote:
> But then I started thinking about IPv6/TCP.  Additional TCP option space could be encoded into an IPv6 Destination Options header.

Yes, but unfortunately ipv6 extension headers do not work.

See e.g.
https://tools.ietf.org/html/draft-gont-v6ops-ipv6-ehs-in-real-world-01

Tables on page 6.

---

From: David Borman <[email-masked]>
Date: Tue, 11 Nov 2014 14:39:28 -0600

> On Nov 11, 2014, at 1:41 PM, Florian Westphal <[email-masked]> wrote:
> 

Which doesn’t say that extension headers don’t work, it says that for their test, when using a Destination Header with a PadN option, roughly 12%-21% of the connections didn’t succeed.

I’m getting tired of the “it won’t pass through 100% of todays middle boxes” being the first or only thing that gets considered.

As I said before:  We need to first decide what we want TCP to look like, and then find a way to transition from current implementations to the new design, which might include middle boxes having to change, rather than letting current middle box implementations dictate how we can or can’t change TCP.

		-David Borman

---

From: Joe Touch <[email-masked]>
Date: Tue, 11 Nov 2014 16:41:24 -0800

On 11/11/2014 11:07 AM, David Borman wrote:
> I’ve been trying to figure out some other place to put extended TCP
> options in a SYN-only packet.  For a standard IPv4/TCP packet,
> because there is only one length field, there is only one place to
> put additional TCP option space that will be ignored by legacy
> systems: encode it into a new IPv4 option.  But I’ll be the first to
> admit this isn’t a great solution:

We've pushed ourselves into a corner by not pushing back on these
optimizations. All our option spaces are getting killed in the name of
profit.

These are the times I wish we had both of the following:
	- an IAB worthy of the name
	- a compliance process

> The slow path processing is mitigated by the fact that this only
> applies to SYN-only packets.  But providing less than 40 additional
> bytes makes this very uninteresting.

+1

AFAICT, the key motivation for larger option space isn't "a little
more", but rather "an order of magnitude more" - e.g., I'd love to have
that so I can use a decent-sized Diffie-Hellman exchange in TCP-AO-encrypt.

> If only TCP had an explicit length field, we could put extended 
> option space after the TCP data.

Agreed. However, the NCP->TCP/IP split design wasn't really a complete
split. TCP isn't completely separate from IP, and the reuse of the
pseudoheader fields from IP in TCP causes us many problems:

	- no length field for TCP
	- forwarding ID == transport connection ID

> And there is a way to do that: encapsulate the IPv4/TCP packet, such
> that the outer IPv4 header had a length that included both the inner
> IPv4/TCP packet, plus the extended options, and the inner IPv4/TCP
> packet length excluded the extended options. 

This is a simpler way to achieve the same result as "Innerspace" - i.e.,
it's going directly towards encapsulation. It's basically TCP-in-TCP as
I suggested.

However, at that point, why do anything so complex? Just run the
simplest, plainest existing features on the outer TCP and use the inner
one to support options that ONLY the endpoint is supposed to know about.

That said, until we're willing to say that some middlebox behavior is
incorrect, we're just pushing off the inevitable - a middlebox that
parses the packets two levels deep, and then messes things up again.

...
> But then I started thinking about IPv6/TCP.  Additional TCP option
> space could be encoded into an IPv6 Destination Options header. 

Same problem. Slow path, and you also have to deal with OPS and SEC folk
who are (IMO) trying to deprecate every "feature" of these protocols
without going through INTAREA. (I've already noted that on both of their
lists).

IMO, a feature that's expensive to deploy is a responsibility - not a
security risk or operational problem. Again, IMO, the companies that
make money on designing Internet products should not be trying to
dumb-down our protocols by this slippery-slope approach; they should
either swim or sink due on the capability of the devices they sell.

However, on that point:
> 1) There is almost 2K of space; though each option is limited to 255
> bytes, multiple options could be allowed if that became necessary.

Maybe, but that was before OPS and SEC got involved. They've been trying
to say that the chain of nested headers should be limited (less work ==
more $$).

>   2) Destination Options headers are not processed at intermediate hops.

TCP isn't supposed to be either, and look where we are with that. So
you'll just be shouting at router vendors who think they can sell
"advanced capabilities" by having their routers process those options
anyway.

> 3) With the top 2 bits of the option type set to zero, it will be
> ignored by hosts that don’t understand it.

As with the "dest opts" values, these too are ignored by router vendors
who want to sell operators "features" they think are useful.

> 4) Hosts that don’t understand it will just see it as a normal TCP

We like to think that these things are true, but in general, even when a
spec says to ignore such things we already find endpoints that don't. So
we need to validate that first.

> So for IPv6, it is possible to transmit the expanded TCP option
> space in the same packet in a manner that won't confuse legacy IPv6/TCP
> implementations.
> 
> A solution for extended TCP option space in SYN-only packets doesn’t
> need to be the same for IPv4/TCP and IPv6/TCP. If we can have a clean
> solution for IPv6, we should do that rather than force a less-than
> desireable IPv4 solution on IPv6 connections.

In principle, I agree, but I don't think you've found anything but a
different way to have middleboxes mess things up.

Joe

---

From: Olivier Bonaventure <[email-masked]>
Date: Thu, 13 Nov 2014 20:25:18 +0100

David,

> But then I started thinking about IPv6/TCP.  Additional TCP option space
> could be encoded into an IPv6 Destination Options header.  The advantages
> of this over IPv4 are:
> So for IPv6, it is possible to transmit the expanded TCP option space in
> the same packet in a manner that won't confuse legacy IPv6/TCP
> implementations
>

I think that this is the cleanest possible solution with regular TCP. For
MPTCP, we adopt a similar approach as the control stream draft to encode
options in the payload
https://tools.ietf.org/html/draft-paasch-mptcp-control-stream-00

> A solution for extended TCP option space in SYN-only packets doesn’t need
> to be the same for IPv4/TCP and IPv6/TCP.  If we can have a clean solution
> for IPv6, we should do that rather than force a less-than desireable IPv4
> solution on IPv6 connections.
>

I completely agree. If we can solve the issue for IPv6, we are future
proof. We cannot live forever with all the IPv4 problems and IPv6 is being
widely deployed

Olivier

---

From: "Scheffenegger, Richard" <[email-masked]>
Date: Thu, 13 Nov 2014 19:57:08 +0000

I believe Michio Honda’s work pointed out, that IPv6 DestOpt currently does not work on 16-17% IPv6 paths in the internet…

Finally, it seems there is  a large number of proposals making use of DestOpt for various purposes currently.

I wonder how much actually useful data can eventually be delivered per segment if all these DestOpt uses would be deployed…

Richard Scheffenegger

From: tcpm [mailto:[email-masked]] On Behalf Of Olivier Bonaventure
Sent: Donnerstag, 13. November 2014 09:25
To: David Borman
Cc: [email-masked] ([email-masked])
Subject: Re: [tcpm] Other places for extended TCP options in SYN-only packets

David,

So for IPv6, it is possible to transmit the expanded TCP option space in the same packet in a manner that won't confuse legacy IPv6/TCP implementations

I think that this is the cleanest possible solution with regular TCP. For MPTCP, we adopt a similar approach as the control stream draft to encode options in the payload
https://tools.ietf.org/html/draft-paasch-mptcp-control-stream-00

I completely agree. If we can solve the issue for IPv6, we are future proof. We cannot live forever with all the IPv4 problems and IPv6 is being widely deployed

Olivier