Subject: Re: [tcpm] TCP Loopback Connections with the Same Src/Dest Port

From: David Borman <[email-masked]>
Date: Mon, 5 Aug 2013 11:00:12 -0500

On Jul 30, 2013, at 11:50 AM, Joe Touch <[email-masked]> wrote:

> 
> 
> On 7/30/2013 1:16 AM, Fernando Gont wrote:
>> On 07/30/2013 04:55 AM, Joe Touch wrote:
>>>> 
>>>> Part of TCP is support for simultaneous opens. If you don't support
>>>> them, ether that's not TCP, or that's a buggy implementation.
>>> 
>>> Again simultaneous < reflexive.
>> 
>> There is no difference in the TCP code that is required for
>> mirrored-endpoint-connections than the code that is required for
>> simultaneous-{open, close} to work.
> 
> First, there is code in TCP to deal with similar special cases (e.g., not negotiating the window size when on the local subnet). So there's already similar special-case code deployed.

That's an example about *implementation* details, not something that is part of the TCP protocol.  There're lots of optimizations in implementations that can be characterized as special-case code, but that's an implementation detail, not a protocol special case, e.g. TCP header prediction.

> 
> However, as I noted, the simultaneous open/close is only a subset of what needs to be vetted to ensure reflexive connections work.

There is no reason to treat a tcp connection connecting to itself any different than a TCP connection connecting to another host.  Every situation that can happen on a TCP self-connection can be constructed on a TCP connection between hosts.  If you can point out anything in the protocol that breaks in a TCP self-connection, then that is something that needs to be fixed in the protocol.  A bug is a bug, and if it shows up in the TCP self-connection case, then it is just waiting to show up in then non-self connected case.

From a protocol standpoint, self-connected TCP sessions are not disallowed and work today, it is implementations that have bugs that keep them from working, and those bugs can show up in the non-self-connected case.  To argue that we have to do a bunch of work to vet them to support them assumes that they are currently not allowed, which is false.  There is nothing in the TCP protocol which prohibits them.

Really, this is a non-issue; If you think otherwise, then write up your own draft to change the TCP protocol to explicitly disallow self-connected TCP sessions.

			-David

> 
>> In any case, my take is that the high-order-bit of this I-D is not the
>> mirrered-endpoint connections, but rather all the other stuff: the
>> update that is needed for simultaneous opens, simultaneous closes, and
>> simultaneous probes to work as expected -- and it looks like everyone
>> (both of us, and other folks that have viced their opinion, plus the
>> implementations out there) agrees that that must be addressed.
> 
> I agree with that.
> 
>> The mirrored-endpoints corner case is something on which I have a
>> position, but, as noted, is a secondary issue (at the end of the day, I
>> could myself live with not supporting them).
> 
> I agree this is a separate issue.
> 
> Joe
> _______________________________________________
> tcpm mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tcpm

---

From: Joe Touch <[email-masked]>
Date: Mon, 05 Aug 2013 10:35:24 -0700

On 8/5/2013 9:00 AM, David Borman wrote:
>> First, there is code in TCP to deal with similar special cases
>> (e.g., not negotiating the window size when on the local subnet).
>> So there's already similar special-case code deployed.
>
> That's an example about *implementation* details, not something that
> is part of the TCP protocol. There're lots of optimizations in
> implementations that can be characterized as special-case code, but
> that's an implementation detail, not a protocol special case, e.g. TCP
> header prediction.

"Implementation details" shouldn't affect either what we see on the wire
or state at the endpoints, but short-circuiting TCP's windowing on the
local LAN does *exactly* that. Are you arguing that this short-cut is
not also a protocol violation?

>> However, as I noted, the simultaneous open/close is only a subset
>> ofwhat needs to be vetted to ensure reflexive connections work.
>
> There is no reason to treat a tcp connection connecting to itself
> any different than a TCP connection connecting to another host.

There's also no reason to treat a TCP connection connecting to another 
host on the same subnet as different from a TCP connection connecting 
elsewhere - BUT WE DO.

Joe

---

From: David Borman <[email-masked]>
Date: Mon, 5 Aug 2013 13:22:51 -0500

> not also a protocol violation?

Huh?  I assumed you are talking about the MSS, if not, please be more specific.  If the negotiated MSS is being ignored, then that's a protocol violation.  If a larger MSS is being negotiated because it knows it's on the local subnet, then that is not a protocol violation.

But that's beside the point.  My point is that special code in a TCP implementation doesn't matter, what matters is whether or not there is special case processing in the TCP protocol, and in the TCP protocol there isn't any special case processing for, or prohibition against, a self-connected TCP connection.

> There's also no reason to treat a TCP connection connecting to another host on the same subnet as different from a TCP connection connecting elsewhere - BUT WE DO.

You are confusing implementation optimizations with the TCP protocol specification.  Stop doing that, it just muddies the water.  Knowledge that a TCP connection is on the same subnet allows for optimizations to improve performance, and there is nothing wrong with that, e.g., negotiating a larger MSS for "local" traffic, and a smaller MSS for "remote" traffic.

At the end of the day, a self-connected TCP connection is perfectly valid and not prohibited by the TCP specification, whether or not they are useful to any application is a moot point.

			-David

---

From: Joe Touch <[email-masked]>
Date: Mon, 05 Aug 2013 11:34:02 -0700

On 8/5/2013 11:22 AM, David Borman wrote:
> Huh?  I assumed you are talking about the MSS, if not, please be more specific.

CWND discovery on the local subnet.

> If the negotiated MSS is being ignored, then that's a protocol
> violation.  If a larger MSS is being negotiated because it knows it's
> on the local subnet, then that is not a protocol violation.

But neither MSS nor CWND is being "negotiated". The negotiation protocol 
- both on the wire and at the endpoints - is being circumvented.

Joe

---

From: David Borman <[email-masked]>
Date: Mon, 5 Aug 2013 13:58:41 -0500

On Aug 5, 2013, at 1:34 PM, Joe Touch <[email-masked]> wrote:

> But neither MSS nor CWND is being "negotiated". The negotiation protocol - both on the wire and at the endpoints - is being circumvented.

If the implementation is ignoring these things, then that's a protocol violation. There is always an MSS value; if MSS isn't received, then 536 must be used, as per RFC-1122.  RFC-1122 also requires that congestion avoidance be used.

But this is a rat-hole that has nothing to do self-connected TCP connections.

			-David

---

From: David Borman <[email-masked]>
Date: Mon, 5 Aug 2013 14:22:39 -0500

On Jul 29, 2013, at 8:18 PM, Joe Touch <[email-masked]> wrote:

> 
> 
> On 7/29/2013 5:58 PM, Fernando Gont wrote:
>> On 07/30/2013 01:55 AM, Joe Touch wrote:
>>> On 7/29/2013 4:47 PM, Fernando Gont wrote:
>>>> at the end of the day, reflected endpoints are just a
>>>> specific case of "simultaneous open".
>>> 
>>> They're more than that.
>>> 
>>> They're simultaneous everything, with the same ISN.
>> 
>> Does it really matter that the ISNs are the same? (packets are not
>> actaully traveling over a network)
> 
> It doesn't to me, but again this is a corner case that some systems don't expect.

The sequence space, and hence the ISN, is independent in each direction.  TCP is explicitly designed so that either direction can start at any value.  Any implementation that has problems with them starting at the same place is broken.

>>> FWIW, an app needs special code to deal with the self-reflected case
>>> anyway - it needs to know that only one incarnation should close the
>>> connection, vs. giving each incarnation a unique socket pair.
>> 
>> Well, if the connection is closed, you get a simultaneous close.
> 
> So from an app-writer's viewpoint, why would all connections be simultaneous open and simultaneous close? What if that's a case that either the protocol or the app doesn't want - maybe it wants to try to de-sync by executing a timer and retrying.
> 
> It won't work on a reflexive socket. It will work everywhere else.

If an app decides to create and use a TCP connection connected to itself, then of course it'll need special code to deal with it.  It has to have special code to create it in the first place.

> 
>> [Yes, if anything, I would only expect applications that "really know
>> what they're doing" to use this "mirrored-endpoints connections"...
>> However, when ti comes to kernel code, it takes more work to not support
>> them , than to do it (assuming that you support simultaneous opens,
>> simultaneous closes, etc.)]
> 
> I disagree; the work to not do it is very simple (it's just a test). The work to ensure it always works could be a lot more.
> 
> The key question isn't whether this is a 'fun case' that the protocol ought to handle; it's whether we should bother to ensure it is always supported. IMO, the special case is a lot simpler than testing every possible alternative.

No, there is nothing in the TCP protocol that explicitly disallows them.  The TCP protocol works fine, it's just implementations with bugs that have problems with them.

> 
>>> I recall that there have also been extensions that don't handle the
>>> simultaneous case - that basically just punt, and say "try again" when
>>> it happens. When these are two different sockets, that's a reasonable
>>> approach; when it's a single reflexive socket, it would continually fail.
>> 
>> Me, I'd argue that if e.g. simultaneous-opens are part of TCP, your
>> extension should work with them rather than rely on "let's retry and
>> hope that the SYNs don't cross on the network".
> 
> "handling simultaneous opens" can be done two ways:
> 	- support them
> 	- don't support them, and retry to de-sync
> 
> The latter will never work with reflexive connections, but it's just as valid an approach.

Absolutely not.  There is no way to "de-sync."  The crossing SYN case happens when both sides are doing an active connect; both sides will be in SYN-SENT state, from there you either transition to SYN-RCVD state if you get a SYN, or to ESTABLISHED if you get a SYN/ACK.  The only way to do your "de-sync" is to close the connection, and then have one side switch to being a passive connect, which is then no longer both sides doing an active connect.

			-David

---

From: Fernando Gont <[email-masked]>
Date: Mon, 05 Aug 2013 16:31:01 -0300

> host on the same subnet as different from a TCP connection connecting

Let's try to stay on focus. The document that Dave and me co-authored
describes a concrete bug in the TCP sequence number validation rules.

1) Do you disagree with the problem and/or with the proposed mitigation?

2) Is there anything in the standard that says that mirrored connections
should not be supported? Any reason for which these connections should
be a special case?

Thanks,
-- 
Fernando Gont
SI6 Networks
e-mail: [email-masked]
PGP Fingerprint: 6666 31C6 D484 63B2 8FB1 E3C4 AE25 0D55 1D4E 7492

---

From: Joe Touch <[email-masked]>
Date: Mon, 05 Aug 2013 13:19:59 -0700

On 8/5/2013 12:31 PM, Fernando Gont wrote:
> 1) Do you disagree with the problem

Only if the problem can be shown as an example that does not require 
self-connects. Otherwise, it's specific to that corner case, and if 
self-connect works it ought not require such special treatment.

> and/or with the proposed mitigation?

It's hard to tell; it would be useful to have a subsection highlight 
exactly what is different and why.

> should not be supported? Any reason for which these connections should

If, as David claims, this is already the case, then there is no need for 
any of the text in Section 4.2.

IMO, the following text in RFC 793 is what prohibits them:

     Each connection is uniquely specified by a pair of sockets
     identifying its two sides.

There are no such "two sides", nor any "pair" in a mirrored connection. 
The ability to connect to do loopbacks is already established by the 
127.0.0.0/8 subnet and the numerous ports available on each of those 
addresses.

Joe

---

From: Fernando Gont <[email-masked]>
Date: Mon, 05 Aug 2013 17:44:09 -0300

On 08/05/2013 05:19 PM, Joe Touch wrote:
> self-connects. 

Of course. I mentioned a few times that the bug affects simultaneous
opens, simultaneous closes, and simultaneous windows probes.

The self-connect is just a specific case of the simultaneous open --
i.e., a scenario that deterministically leads to simultaneous opens --
just that.

> exactly what is different and why.

That's why we wrote the I-D. What's the part that is not clear, and why?

> any of the text in Section 4.2.

It is needed because some implementations have such bug. -- It shouldn't
be a special case... but some implementations (unfortunately) treat it
as such.

>     identifying its two sides.

Where does RFC793 says that the two endpoints must be different?

Cheers,

---

From: Joe Touch <[email-masked]>
Date: Mon, 05 Aug 2013 15:05:03 -0700

> i.e., a scenario that deterministically leads to simultaneous opens --

That's a better motivation for discussing the self-connect case, but 
then that seems at best an aside to the issue.

> That's why we wrote the I-D. What's the part that is not clear, and why?

The ID talks about the total of what's different, but it seems like 
you've changed a few equations of the edge conditions. I see them, but 
had to flip pages to see what they were, and I didn't see each line 
changed explained in a simple sentence.

> be a special case... but some implementations (unfortunately) treat it

It is not related to the issue here, as you note above, except as one 
way that always causes the issue. As a result, calling it out as a 
specific expectation is neither needed nor appropriate.

> Where does RFC793 says that the two endpoints must be different?

Two sides.

Where are the two sides of a self-connect? There's only one connect 
call, and only one socket. It's not a pair of anything.

Joe

---

From: Fernando Gont <[email-masked]>
Date: Mon, 05 Aug 2013 19:58:38 -0300

On 08/05/2013 07:05 PM, Joe Touch wrote:
> you've changed a few equations of the edge conditions. 

Simple allow for one byte to the left of the window, to handle all the
problematic cases. That will cause the control data to be acted upon.

> I see them, but

I thought the equation was self-explaining, and none had argued
otherwise. Me, I have no problem to add a note about that if people
think that would help.

> specific expectation is neither needed nor appropriate.

It *is* related. Some implementations banned self connects because they
thought of them as a vulnerability. But the vulnerability really had to
do with improper handling of simultaneous opens, rather than self-connects.

This I-D fixes the simultaneous-* problem, and notes "hey, you shouldn't
have banned self-connects... the problem was elsewhere".

> call, and only one socket. It's not a pair of anything.

I don't follow. When you implement or specify a protocol you care about
local interface calls, and packets you receive. You don't care about
remote interface calls, but only abut the resulting packets.

Cheers,

---

From: David Borman <[email-masked]>
Date: Mon, 5 Aug 2013 18:32:55 -0500

>> On 08/05/2013 05:19 PM, Joe Touch wrote:
...
> Where are the two sides of a self-connect? There's only one connect call, and only one socket. It's not a pair of anything.

Be careful with terminology.  You are confusing the BSD socket interface with the "socket" as defined in RFC 793, and they are *not* the same thing.  In RFC 793, a "socket" only refers to the addressing on one side of a connection, i.e. a single IP address and a single port, an no state information.  From RFC 793:

    To allow for many processes within a single Host to use TCP
    communication facilities simultaneously, the TCP provides a set of
    addresses or ports within each host.  Concatenated with the network
    and host addresses from the internet communication layer, this forms
    a socket.  A pair of sockets uniquely identifies each connection.
    That is, a socket may be simultaneously used in multiple
    connections.

When RFC 793 talks about a "pair of sockets", it's saying a connection is identified by the combination of a local IP-address/TCP-port and a foreign IP-address/TCP-port, and hence multiple *connections* can have the same "local socket" by having different "foreign sockets".

And the sentence you quoted comes from:

    The reliability and flow control mechanisms described above require
    that TCPs initialize and maintain certain status information for
    each data stream.  The combination of this information, including
    sockets, sequence numbers, and window sizes, is called a connection.
    Each connection is uniquely specified by a pair of sockets
    identifying its two sides.

Again, the RFC 793 "socket" is just a single IP address and TCP port, it is not a BSD socket, and the "pair of sockets" just means you have both local and remote IP addresses and TCP ports.

For a self-connected BSD socket, the bind() specifies the RFC 793 "local socket", and the connect() specifies the RFC 793 "foreign socket", thus you have an RFC 793 "pair of sockets", and a unique connection held within a single BSD socket.

		-David

---

From: Joe Touch <[email-masked]>
Date: Tue, 6 Aug 2013 09:17:36 -0700

> otherwise. Me, I have no problem to add a note about that if people

You give a before and after as a long section, but IMO it would be useful to highlight exactly what has changed without expecting readers to do a "diff".

> have banned self-connects... the problem was elsewhere".

You shouldn't have banned self-connects for this reason. There are other good reasons to avoid self-connects, which I've discussed.

> remote interface calls, but only abut the resulting packets.

You also care about a few other things that could interfere with reflexive connections, e.g., how the state machines handle API calls or events that send packets vs. those that receive them. The sentence I cite, IMO, avoids that potential problem by referring to two sides. 

Joe

---

From: Joe Touch <[email-masked]>
Date: Tue, 6 Aug 2013 09:26:59 -0700

On Aug 5, 2013, at 4:32 PM, David Borman <[email-masked]> wrote:

> Be careful with terminology. You are confusing the BSD socket interface with the "socket" as defined in RFC 793, and they are *not* the same thing.

I have been consistently referring to a socket as an IP address and a port. Although the warning is useful to others on this list, it does not apply to my posts in this thread.

In a reflexive connection, one socket is used for both sides of a connection. That's one side with one socket. And that's not what RFC 793 says.

Speaking of Unix:

> For a self-connected BSD socket, the bind() specifies the RFC 793 "local socket", and the connect() specifies the RFC 793 "foreign socket",

So you're using the same 793-socket for both sides. That's one socket on a connection with one side (where, IMO, "side" is where things like local connection state exist). 

----

FWIW, my primary concerns with this thread are:

	a) focusing on the hang issue as primarily for reflexive connections
		it's equally important for non-reflexive

	b) promoting the use of reflexive connections
		there are a lot of good reasons to avoid them; the hack to get
		around creating a /tmp file is a very bad use case (/tmp can't hang,
		but self-connections can easily block and their storage depends 
		heavily on the amount of Unix socket buffer)

Joe