Subject: [tcpm] TCP-Rehash draft

From: Bruno Mongazon-Cazavet <[email-masked]>
Date: Mon, 21 Feb 2011 17:05:39 +0100

Dear working group members,

I wonder if you have any chance to look at the TCP-Rehash draft:
http://tools.ietf.org/html/draft-mongazon-tcpm-tcp-rehash-00

i posted a few months ago.

Did you read it and would consider any interest and follow-up in the 
working group context ?
On my side, i have a pretty serious running implementation of the draft 
in Linux 2.6.27 and Linux 2.6.34.
I am ready to discuss and present both the draft and implementation in 
next IETF meeting.
Would you be interested in this ?

Thanks for your time and consideration.
Bruno Mongazon.

---

From: "Smith, Donald" <[email-masked]>
Date: Mon, 21 Feb 2011 14:11:55 -0700

Sharing: Author's permission required.
[email-masked]

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Bruno Mongazon-Cazavet
> Sent: Monday, February 21, 2011 9:06 AM
> To: [email-masked] Extensions
> Subject: [tcpm] TCP-Rehash draft
>
I did and yes.
> _______________________________________________
> tcpm mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tcpm

This communication is the property of Qwest and may contain confidential or
privileged information. Unauthorized use of this communication is strictly
prohibited and may be unlawful.  If you have received this communication
in error, please immediately notify the sender by reply e-mail and destroy
all copies of the communication and any attachments.

---

From: Joe Touch <[email-masked]>
Date: Tue, 22 Feb 2011 18:25:46 -0800

Hi, all,

FWIW, there was a discussion of whether this proposal was a subset of 
the features of multipathTCP on the multipathTCP list back in Nov.

Some other points:
- rehash is a bad term to use; it implies a security hash, rather than 
altering the addresses of a TCP connection

- the option includes stateful exchanges of options
	TCP options are stateless w.r.t. the TCP state machine,
	and must be able to be processed out of order and
	assume losses (they aren't running inside the TCP
	data stream); that appears to challenge the design
	of this mechanism (it seems to be addressed in part,
	but corner cases aren't considered)

- the option seems to be twice as long as necessary
	it seems like you already have the current or old IP
	address in the TCP pseudoheader (IP header); why
	do you need to send both?

	you clearly need room for flags somewhere, but can
	accomplish that with 1 byte (or 2 if padded), rather
	than 4

- the option appears to deal only with the address changing
	however, the ports are also specific to the endpoints
	as well; it should not be assumed that a connection
	can be moved with its ports intact

Finally, I don't see how requiring IPsec to use this mechanism makes it 
simpler than MPTCP (which doesn't require IPsec).

There are many other issues which are not addressed, notably how 
applications assume that successive or concurrent connections to the 
same IP address go to the same endpoint (e.g., for FTP, or web 
transactions); similar concerns are raised in the Intarea "address 
sharing" doc, and should be discussed here.

Joe

---

From: Bruno Mongazon-Cazavet <[email-masked]>
Date: Wed, 23 Feb 2011 10:34:03 +0100

> the features of multipathTCP on the multipathTCP list back in Nov.

Yes i saw this.
Having also some knowledge and experience with MPTCP, i would argue that 
features are somewhat different.
TCP-Rehash and MPTCP looks more complementary than competing.

> altering the addresses of a TCP connection

"rehash" term is used in reference to the "hash" mechanism performed by 
TCP/IP Linux stack to find a TCP control block giving a 5-Tuple.
When choosing this name, no security matter was considered. Just the 
capability to recompute the hash value of a TCP-cb for a given 5-tuple 
(old address) with a new 5-tuple (new address). That's why i called it 
"rehash".

I must agree that the term might not be the best but i wanted to avoid 
terms like "migrate" or alike.

> 	but corner cases aren't considered)

There are couples of RFC that use TCP options in a stateful way (i.e. 
RFC1323 with timestamp echo).
Basically MPTCP options also work the same.
I don't see any restriction in IETF in using TCP options to maintain 
state information.

I agree about "out of order" and "losses" issues and it is true it 
challenged a bit the design.
I came up with the following invariants:

(a) the first packet out with the new address must carry the REHASH-ADDR 
option if the connection is "rehash" capable
(b) if REHASH-ADDR has been sent, and some packets are still received 
with the old address, the receiver did not yet "rehash" and the sender 
drops old packets and provides again the REHASH-ADDR option until a 
maximum number of retry is reached.

Could you provide some details on the corner cases that are not covered ?

> 	do you need to send both?

I need both because if there is a NAT in between, the IP header source 
address will differ.
Although TCP-Rehash is currently described as not-NAT capable, provision 
in the TCP option would allow to do this later.

> 	accomplish that with 1 byte (or 2 if padded), rather

Provision for flags is in bytes 0-1.

> 	can be moved with its ports intact

Right, the ports are currently missing in the option and could be added 
in further revision of the draft.
Please consider that this is draft-0 and thus claims it might lack of 
some features.

> simpler than MPTCP (which doesn't require IPsec).

Just saying that if your network is insecure and because TCP-Rehash 
security is low, you can use IPsec.

> sharing" doc, and should be discussed here.

Fine, can you provide pointer to this document, please.

Best regards,
Bruno.

---

From: Lars Eggert <[email-masked]>
Date: Wed, 23 Feb 2011 11:50:49 +0200

Hi,

> TCP-Rehash and MPTCP looks more complementary than competing.

right. MPTCP is clearly a superset of what your proposal wants to achieve. However, since the IETF has committed to standardize MPTCP, it seems kind of superfluous to take on a second, incompatible proposal with an overlapping but reduced feature set.

That said, I wonder if someone interested (you?) could take MPTCP and strip it down, so that more or less the functionality that you described for "TCP-Rehash" remained. In other words, produce a document that describes a "lightweight" implementation of only some of the features of MPTCP, and in a way that remained compatible with MPTCP (= used the MPTCP options and some of the MPTCP state machine.)

TCP-Rehash is an interesting proposal, don't get me wrong. But it simply came to the IETF too late, with MPTCP being pretty far along the standardization process and Linux code for it readily available (https://scm.info.ucl.ac.be/trac/mptcp/).

Lars

---

From: Bruno Mongazon-Cazavet <[email-masked]>
Date: Wed, 23 Feb 2011 11:37:46 +0100

> TCP-Rehash is an interesting proposal, don't get me wrong. But it simply came to the IETF too late, with MPTCP being pretty far along the standardization process and Linux code for it readily avail…

Hi Lars,

I understand these concerns and commit to the most constructive approach 
from IETF standards perspective.

Would it be possible to arrange a 15 minutes slot for me at IETF-80 in 
Prague so i can present the draft, provide a demonstration of the Linux 
implementation and have a working group agreement on next steps ?

You might choose TCPM or MPTCP (or both) working group(s). Please choose 
what is the most convenient.

Regards,
Bruno.

---

From: Joe Touch <[email-masked]>
Date: Wed, 23 Feb 2011 11:12:20 -0800

Some final suggestions; I'm guessing the rest can wait for your 
presentation...

On 2/23/2011 1:34 AM, Bruno Mongazon-Cazavet wrote:
...
> RFC1323 with timestamp echo).

That's not quite state in the way I was assuming it. It's OK to base 
your response on what you see (that's immediate context), but there 
can't be a "state machine" of transitions based on the options, AFAICT - 
i.e., your response can't/shouldn't depend on previously received options.

The only place where such a transition can occur readily is during the 
SYN exchange.

> I don't see any restriction in IETF in using TCP options to maintain

There's no formal restriction, but TCP likes to resend segments as 
needed, and those segments aren't often recomputed from scratch 
(including options) when resent. So the options sent must, AFAICT, be 
stable w.r.t. retransmission, reordering, and loss. It's implicit in 
that part of the TCP mechanism.

> maximum number of retry is reached.

That's OK, but the question is what happens when a segment is 
retransmitted. If you have to check the options for every segment just 
before it is retransmitted, you have a problem IMO.

> Could you provide some details on the corner cases that are not covered ?

Loss, retransmission, user requesting a second relocation while one is 
pending, etc.

> in the TCP option would allow to do this later.

This is confusing. If you need it for NATs, then you need to show that 
mechanism now.

If that mechanism is pending and you're OK waiting to develop it, then 
develop this version without that assumption, IMO.

> Provision for flags is in bytes 0-1.

For TCP options, byte 0 = KIND, and shouldn't vary (you shouldn't expect 
to use more than one KIND for a single option), and byte 1 = length.

The format should be shown as a sequence of words (see RFC793, e.g.), 
but here's what you have:

	KIND (1 byte; default for TCP options)
	len (1 byte; default for TCP options) - you set this to 16
	control (2 bytes)
	nonce (4 bytes)
	addr1 (4 bytes)
	addr2 (4 bytes)

FWIW, control is really bytes 2-3, not 0-1.

If you want flags in the control bytes, you should reserve flag fields, 
e.g., declare the first byte as type, and the second as flags.

For the Init exchange, you can easily just send the nonce, and in that 
case the length gives you the type, e.g.:

	KIND
	len = 6
	nonce (4 bytes)

so when len=6 AND this appears in a SYN (IMO, that's the only time/place 
this ought to be exchanged), then you can record/exchange the nonces and 
conserve some space.

Later messages can just be:
	KIND
	len = 10
	nonce (4)
	addr (4)

This saves a lot of space, which is needed for options like SACK.

Again, no need to send both old and new addrs; the current addr is in 
the IP header. You need send only the one you're going to, AFAICT.

> Please consider that this is draft-0 and thus claims it might lack of

AOK; again, length can deal with that issue.

> security is low, you can use IPsec.

The rules you have require IPsec in the general case; only in limited, 
known-secure environments can you avoid that. If that's your plan, 
you're making a system that's much lighter than MPTCP, but then relying 
on a system which is arguably more complex ;-)
> Fine, can you provide pointer to this document, please.

https://datatracker.ietf.org/doc/draft-ietf-intarea-shared-addressing-issues/

There have been discussions about this doc recently on the Intarea 
mailing list too that are worth checking out.

Joe

---

From: Bruno Mongazon-Cazavet <[email-masked]>
Date: Fri, 25 Feb 2011 09:47:20 +0100

> i.e., your response can't/shouldn't depend on previously received options.

[BM] Why can't there be a state machine based on the options ?

> that part of the TCP mechanism.

[BM] I think it is already the case. If you append the TCP options at 
the very bottom level of the TCP stack, there is no problem.
You don't even know whether it is a retransmit. It does not change the 
TCP regular behavior. You just look in TCP block and check whether you 
need to add a TCP option or not.

> before it is retransmitted, you have a problem IMO.

[BM] Please see previous remark. Basically:

-TCP-Rehash option processing happens at the very low level of TCP stack 
(in the solely parts where every packet goes out or in)
-at that place, i check in the (little) state-machine for TCP-Rehash if 
there is a TCP-option to add or process

In the solely part of TCP in and out, you don't even know if the packet 
is retransmitted or not.

> Loss, retransmission, user requesting a second relocation while one is

[bm] loss and retransmission is handled. The case of second relocation 
while one is pending is interesting and honestly i don't yet have a 
satisfying solution. Imagine that your stack changes addresses faster 
than the rate of packets. I can write a word for this in the draft.

> develop this version without that assumption, IMO.

[BM] Right. NAT-traversal should be in the draft, but a day is only 24h !!

> the IP header. You need send only the one you're going to, AFAICT.

[BM] Thanks Joe, your proposals looks reasonable.

> AOK; again, length can deal with that issue.

[BM] ok got it.

> https://datatracker.ietf.org/doc/draft-ietf-intarea-shared-addressing-issues/

[BM] thanks.

---

From: Bruno Mongazon-Cazavet <[email-masked]>
Date: Fri, 25 Feb 2011 09:52:18 +0100

Le 24/02/2011 11:21, Scheffenegger, Richard a écrit :
> Bruno,
> Last time I checked, MPTCP negotiates all options that require state changes; and RFC1323 was designed specifically to aid stateless tracking of RTT (instead of what appears to have become endemic, keeping state and tracking each individual sent segment, and matching the ACKs to the proper segment, for "timestamp-less", fine-granularity RTT tracking purposes...

[BM] This is a good point of MPTCP.

> Perhaps using Linux as basis is not the base departure point to argue about IETF compliant behavior (Linux deviates in so many ways from a IETF-adhering stack, that it's hard to delineate all the interactions; this is not intended as a  judgmental statement).

[BM] I use what i have ! I test specifications through implementation as 
IETF requires. I could also have done it on FreeBSD, just lack of time.

Regards,
Bruno.

>     Richard
> From: Bruno Mongazon-Cazavet [mailto:[email-masked]]
> Sent: Mittwoch, 23. Februar 2011 10:34
> To: Joe Touch
> Cc: [email-masked] Extensions
> Subject: Re: [tcpm] TCP-Rehash draft
>
>

---

From: Joe Touch <[email-masked]>
Date: Fri, 25 Feb 2011 09:04:32 -0800

> [BM] Why can't there be a state machine based on the options ?

A few reasons:

1) options come out of order and unreliably

2) there is no notion of how options would change if a segment is 
retransmitted

i.e., #2 means that NOTHING in the packet can be used as context for the 
option (e.g., sequence number, etc.), and changing the options for 
different retransmissions of a given segment is *required* for 
state-machine based options.

> need to add a TCP option or not.

See above. The problem is that it's difficult to provide enough context 
for an option so that it can truly be evaluated independently of the 
packet in which it occurs, and unless you do so, it won't work with 
retransmissions. Further, most TCPs don't set options for each 
retransmission differently - that's a major change, and needs to be 
specified very clearly as a requirement.

> there is a TCP-option to add or process

First, you've described how *you* happen to implement it, not how it is 
specified. You need to change the spec to *require* that order of 
processing, and that's quite outside the typical protocol specification 
(it creeps into implementation).

> than the rate of packets. I can write a word for this in the draft.

You need to prohibit that in the state machine.

Joe

---

From: Bruno Mongazon-Cazavet <[email-masked]>
Date: Mon, 28 Feb 2011 09:39:41 +0100

> state-machine based options.

[BM] The only requirement for TCP Options is expressed in page 4 of the 
draft:

    Whenever TCP packets carrying a TCP Rehash option
    needs retransmission, the TCP Rehash option shall go again together
    with the retransmitted packet without breaking the TCP ordering
    logic.

So there is nothing related to the packet content in that sentence, and 
this is the general case for TCP-Rehash.

> retransmissions.

[BM] I am trying to do so, being independent of the packet content. I 
don't think the draft says so. The only thing that is considered is the 
SYN bit during initial 3way handshake. Apart from that, the packet 
content is never considered when adding a TCP-Rehash option.

>   Further, most TCPs don't set options for each

[BM] the option is not set differently, that's the point, it is just 
eventually added.

> (it creeps into implementation).

[BM] Page 6 the draft says:

   Prior to
    issue the REHASH-ADDR option, the initiator of connection movement
    must perform locally the connection rehash and ensure that the
    REHASH-ADDR option is mandatory carried in the first outgoing packet
    flowing from the new IP address.

You might consider this is an implementation constraint, but this is the 
basic principle of the rehash.
If you can't add the rehash option on the first outgoing packet carrying 
the new address, you can't do rehash.
I think it is not implementation but functional. This is how it is drawn 
in the draft. To me it is part of protocol specification.

> You need to prohibit that in the state machine.

[BM] Right i can add it to "Usage restriction" section 4.

Regards,
Bruno.