Subject: [tcpm] draft-touch-tcpm-2140bis: wording improvement suggestions?

From: Michael Welzl <[email-masked]>
Date: Thu, 13 Oct 2016 09:12:46 +0200

Hi Michael,

At the last IETF, you said (from the minutes):

"There is value in documenting the implementations, there may be opportunities to modernise the wording. You can make significant improvements as you edit the document.”

I remember saying that we’d appreciate concrete suggestions, but we didn’t get any - so, with the cutoff deadline approaching, I’m repeating my request… anything specific you’d suggest to change, so we can consider it?

Cheers,
Michael

---

From: "Scharf, Michael (Nokia - DE)" <[email-masked]>
Date: Thu, 13 Oct 2016 09:27:46 +0000

Hi Michael,

My comment was e.g. regarding "Section 10. Implementation Observations", which discusses T/TCP in three long paragraphs and then has one sentence and a simple table on Linux kernel version 4.6 and FreeBSD 10 (which I like).

I personally think T/TCP could be by and large be moved to an appendix and the document should focus on what has been widely deployed in the last two decades.

Best regards

Michael

-----Original Message-----
From: Michael Welzl [mailto:[email-masked]] 
Sent: Thursday, October 13, 2016 9:13 AM
To: Scharf, Michael (Nokia - DE) <[email-masked]>
Cc: [email-masked]
Subject: draft-touch-tcpm-2140bis: wording improvement suggestions?

Hi Michael,

Cheers,
Michael

---

From: Michael Welzl <[email-masked]>
Date: Thu, 13 Oct 2016 12:10:43 +0200

That was useful, thanks!

Michael

> On 13. okt. 2016, at 11.27, Scharf, Michael (Nokia - DE) <[email-masked]> wrote:
> 
> Hi Michael,
>

---

From: Yuchung Cheng <[email-masked]>
Date: Thu, 13 Oct 2016 06:21:14 -0700

I remained my objection that WG spends time on this. If this is BCP,
please show that it _actually_ improves performance on busy servers
(!= running netperf over same ip pair 1000 times).

The draft claims it potentially improves performance and lists a
variety of reason it may not (thanks). But do we really know it helps?
I only have negative data points (based on Linux's impl) and would
appreciate positive data points before we move forward.

On Thu, Oct 13, 2016 at 3:10 AM, Michael Welzl <[email-masked]> wrote:
> _______________________________________________
> tcpm mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tcpm

---

From: Michael Welzl <[email-masked]>
Date: Thu, 13 Oct 2016 16:18:23 +0200

Hi Yuchung,

First, we had a clear “no” for the BCP suggestion at the last meeting, so we don’t need to discuss this context. It’s Informational.

Second, can you be a bit more specific about “it” in your statement below?
(I suspect you mean ssthresh?)

Note that we intend to make the language more neutral based on your comment at the last meeting, but we have not *yet* done so.
Since you say “thanks” to us giving reasons “it" may *not* improve performance, I wonder: maybe the only thing that has changed since you commented on it in the last meeting is that you now actually read it?

Cheers,
Michael

> On 13. okt. 2016, at 15.21, Yuchung Cheng <[email-masked]> wrote:
>

---

From: Yuchung Cheng <[email-masked]>
Date: Sat, 15 Oct 2016 16:29:15 -0700

On Thu, Oct 13, 2016 at 7:18 AM, Michael Welzl <[email-masked]> wrote:
>
> Hi Yuchung,
>
> First, we had a clear “no” for the BCP suggestion at the last meeting, so
we don’t need to discuss this context. It’s Informational.
ack
> (I suspect you mean ssthresh?)
It means the TCB sharing done by real implementation (e.g. Linux metrics
cache and FreeBSD).

>
> Note that we intend to make the language more neutral based on your
comment at the last meeting, but we have not *yet* done so.
> Since you say “thanks” to us giving reasons “it" may *not* improve
performance, I wonder: maybe the only thing that has changed since you
commented on it in the last meeting is that you now actually read it?
I have read the drafts more than once since its initial version. To double
check I didn't miss something important I re-read again before replying
this email.

Since we want to update RFC2140, I assume we believe metrics cache are
useful (so it is informational to IETF people). I am asking the (positive)
data points, not simply the fact it's been implemented.

Now let's look into the details of the actual info suggested for sharing,
and lets skip ssthresh since we've discussed.

RTT/RTTVAR: RTT is readily available after SYN, so the most useful place is
SYN timeout. But the draft says we should not set anything more aggressive
than the minimum requirement by standard, which is 1sec by RFC6298. How can
fast connection retry SYN faster to lower handshake time? or I mis-read the
draft. Recently a detailed investigation on Linux RTO also question the
form of RTT caching (paper
<https://drive.google.com/file/d/0B6yJM3bPQR5Da3l5Q1ZaRC1TZUU/view?usp=sharing>).
Also another report by Anna:

MSS,PMTU: I know MSS cache is useful for Fast Open b/c we need to decide
how much to put in SYN. Other than that I really don't know how this helps
performance for something we communicated at handshake. Would that reduce
potential middlebox issues? not clear.

cwnd/ssthresh: we've covered this.

The bis draft lists newly identified issues with TCB (e.g. ECMP) but it's
unclear whether the implementor should do something or just ignore. To
conclude, my one question is after ~20 years of RFC2140 published, is the
current implementation of TCB sharing useful (by data)?

If the draft does not address this question, what's new in this
informational bis?

[to be specific, I am not asking to incorporate my comments into the next
revision. I am asking data]

> >>> My comment was e.g. regarding "Section 10. Implementation
Observations", which discusses T/TCP in three long paragraphs and then has
one sentence and a simple table on Linux kernel version 4.6 and FreeBSD 10
(which I like).
> >>>
> >>> I personally think T/TCP could be by and large be moved to an
appendix and the document should focus on what has been widely deployed in
the last two decades.
> >>> "There is value in documenting the implementations, there may be
opportunities to modernise the wording. You can make significant
improvements as you edit the document.”
> >>>
> >>> I remember saying that we’d appreciate concrete suggestions, but we
didn’t get any - so, with the cutoff deadline approaching, I’m repeating my
request… anything specific you’d suggest to change, so we can consider it?

---

From: Joe Touch <[email-masked]>
Date: Mon, 17 Oct 2016 14:36:03 -0700

Hi, Yuchung,

2140 sharing benefits depend on the length of the new connection and the
amount to which its parameters vary from TCP's defaults. All connections
start with defaults; the difference is whether they depend on
previous/concurrent connections or are fixed in advance.The extent to
which the metrics cache is useful thus depends on the actual connections
at the time.

Regarding some of those metrics:

On 10/15/2016 4:29 PM, Yuchung Cheng wrote:
> ...
>
> RTT/RTTVAR: RTT is readily available after SYN, so the most useful
> place is SYN timeout. But the draft says we should not set anything
> more aggressive than the minimum requirement by standard, which is
> 1sec by RFC6298. How can fast connection retry SYN faster to lower
> handshake time? or I mis-read the draft. Recently a detailed
> investigation on Linux RTO also question the form of RTT caching
> (paper

RTTVAR won't be meaningful for several RTTs and it is important in how
RTT is interpreted. RTTs shouldn't be spun down too low with 2140, but
they could be much higher. I.e., the opposite condition of the paper
you're citing.

The extent to which these values affects new connects decreases over
time, so they are more important for the first few round trips (as is
true for most shared parameters).

> MSS,PMTU: I know MSS cache is useful for Fast Open b/c we need to
> decide how much to put in SYN. Other than that I really don't know how
> this helps performance for something we communicated at handshake.
> Would that reduce potential middlebox issues? not clear.

SYNs can contain data even without FO. Having past or concurrent context
can be also be useful to detect when multipath routing is occuring (when
it changes with a new connection) or when routes are not stable (when
the values vary over time).

> The bis draft lists newly identified issues with TCB (e.g. ECMP) but
> it's unclear whether the implementor should do something or just
> ignore. To conclude, my one question is after ~20 years of RFC2140
> published, is the current implementation of TCB sharing useful (by data)?

The answer depends on the context of the connections. What we can know
is whether this is consistent with TCP (it is) and whether it has the
potential for providing more information to the new connection (it
does). What a new connection does with the information is up to the
implementer.

> If the draft does not address this question, what's new in this

A more detailed discussion of what has been implemented and the issues
involved.

Joe

---

From: Yuchung Cheng <[email-masked]>
Date: Wed, 26 Oct 2016 22:10:52 -0700

On Mon, Oct 17, 2016 at 2:36 PM, Joe Touch <[email-masked]> wrote:
> previous/concurrent connections or are fixed in advance.The extent to which
> the metrics cache is useful thus depends on the actual connections at the
> time.
>
> draft. Recently a detailed investigation on Linux RTO also question the form
> of RTT caching (paper). Also another report by Anna:
>
>
> RTTVAR won't be meaningful for several RTTs and it is important in how RTT
> is interpreted. RTTs shouldn't be spun down too low with 2140, but they
> could be much higher. I.e., the opposite condition of the paper you're
> citing.
>
> The extent to which these values affects new connects decreases over time,
> so they are more important for the first few round trips (as is true for
> most shared parameters).
>
> MSS,PMTU: I know MSS cache is useful for Fast Open b/c we need to decide how
> much to put in SYN. Other than that I really don't know how this helps
> SYNs can contain data even without FO. Having past or concurrent context can
> be also be useful to detect when multipath routing is occuring (when it
> changes with a new connection) or when routes are not stable (when the
> values vary over time).
>
> The answer depends on the context of the connections. What we can know is
> whether this is consistent with TCP (it is) and whether it has the potential
> for providing more information to the new connection (it does). What a new
> connection does with the information is up to the implementer.
>
OK I am convinced if the purpose is updated new issues then the -bis
makes sense.

IMO the most crucial feature of connection sharing
is caching negative info, not positive info: pathological behaviors of
the path and servers that can resort in total dis-connectivity. This
is especially true for new features that uses new options and
'uncommon' protocol
behaviors  (e.g, TFO, tcp-inc watch out). Worst case I've seen is middle-box
completely blocks all further packets from an IP after seeing a
SYN-data packet. Some middle-box would wipe every option in SYN
once if the SYN has an experimental option. The connection would work
but no wscale and sack warrant poor performance, and the user is left
wondering why his expensive new phone is so slow.

While using IP-based indexing can still be too coarse for negative
caching it works fine here by being conservative of bad paths (i.e.
when in doubt don't try the fancy new feature).

---

From: Joe Touch <[email-masked]>
Date: Thu, 27 Oct 2016 10:04:04 -0700

On 10/26/2016 10:10 PM, Yuchung Cheng wrote:
> the path and servers that can resort in total dis-connectivity. 

The goal of 2140 sharing is to reduce transients. The sort of negative
information you're considering isn't a transient. Thus I'm worried about
the impact of caching it for the following reasons:

- many of the mechanisms you mentioned aren't dual-stack (e.g., tcp-inc,
or consider TCP-AO). There's not necessarily a "if secure doesn't work,
retry without it".

- the probe test for each connection could be intended to yield a
different result, e.g., security often is configured differently for
different ports within a host.

- negative cached results could cause a host to fail to recover from
errors, e.g., until the cache is cleared

E.g., TCP-AO failures MUST NOT be cached for different SYN destination
ports to the same host or to any port to a different host

We'd have to look into each of the other variants more closely to
determine whether negative caching makes sense or not. IMO, this could
be mentioned with a strong warning to avoid negative caching unless
based on specific analysis of a given feature.

Joe

---

From: Yuchung Cheng <[email-masked]>
Date: Thu, 27 Oct 2016 18:03:34 -0700

> the impact of caching it for the following reasons:
I agree with your assessment of bad reaction to negative cache could
result in worse result overall, and many of them need to be carefully
done case by case.

but my high level point is that I found the usefulness of negative
caching is larger, in fact much larger than caching RTT and other TCP
vars.