Subject: Re: [tcpm] New Version - draft-ietf-tcpm-newcwv-07.txt

From: Raffaello Secchi <[email-masked]>
Date: Thu, 16 Oct 2014 13:21:19 +0100

Dear Yuchung, Mirja, Alex et al,

So we've now had time to revisit the changes to Linux and the
differences between the ID and the Linux code.

A patch was submitted and followed the ID. This was available up to Linux 3.15.

In Linux, there was an update submitted that conflicted with the
new-cwv patch, so new-cwv is not now supported - we think this is what
was recently discussed on the list. From what we understand, the
conflicting update centred around fixed needs to support TSO. This was
explained in the email from Yuchung on 22/09/2014 describes this work
from Neal Cardwell.

To us, it seems this does not implement a window validation procedure,
so this isn't the same as new-cwv. We'll highlight what we see are the
key differences below, and others may comment - especially if we get
this wrong.

---

new-cwv as described in the ID proposes is a list of things that are
needed to validate the path capacity and manage the cwnd.

+ FS v. pipeACK
An important part of the draft (section 4.2) requires a path to be
validated before using the path capacity information - in contrast,
traditional TCP used the flight size as an estimate of capacity,
assuming that in the last RTT the sender sent at least FS and this
must have been ACK'ed.

new-cwv in contrast, does not make this assumption (this would be
"bold" for a varying rate app - which may have sent little of nothing
in the last RTT or so). This lead us to introduce pipeACK. This adds
some complexity but is robust - specifically it does not base
decisions on data sent in the last RTT (FS), but builds a history of
the peak capacity available in recent history.

The recent Linux patches preserve cwnd (similar to new-cwv 4.3) but
uses the (non-validated) FS instead of pipeACK.

This has three side effects:
1) It doesn't validate the capacity so a sender that resumes sending,
uses FS for calculation, and resumes sending faster. The new-cwv ID
recommends cwnd is increased in data-limited conditions only if
segment transmissions can be validated. This prevents for example the
cwnd from increasing for at least one RTT when resuming after an idle
period and limits the damage to cwnd in case of congestion.

2) A bursty app can now result in a cwnd that is far less than the
recent capacity, and is very very conservative for a busty sender. If
a loss is met during a data-limited period of low-rate transmission,
cwnd goes back to RW. We propose to use pipeack instead which consider
the transmission history in the last second.

3) FS can be more variable than pipeack. In the Linux code, the
variable max_packets_out is set to the maximum of FS for the previous
RTT. Considering that the FS can be very variable for a bursty sender,
we think that taking the maximum is the right thing to obtain a stable
reference. We recommended extending the period over which the maximum
is calculated to at least one second.

+ By default Linux does not keep the cwnd open for long idle-periods.
It uses CWV to decay cwnd exponentially instead.

Closing cwnd exponentially degrades performance of some bursty apps
and creates a dependency between TCP performance and RTT. Linux
currently offers the 'tcp_slow_start_after_idle' switch to disable
this behaviour. If tcp_slow_start_after_idle is disabled, TCP keeps
cwnd indefinitely.

new-CWV recommends keeping cwnd for at most for five minutes, after
which cwnd is reduced to align the congestion state to the new
data-limited regime. Not closing the cwnd may be dangerous if the
connection produces larger-than-usual bursts and anyway is
incompatible with the idea that the cwnd has been (recently)
validated.

+ Current Linux does not take into account the number of
retransmissions in resetting cwnd after a loss episode - this can be a
significant issue when the previous cwnd proves invalid (e.g. due to a
path change).

For this reason, new-CWV section 4.4.1. encourages a (D-R)/2 reduction
after the FR. In this case, the number of retransmissions 'R' may be
significant compared to the FS 'D'. We expect (D-R)/2 to converge the
cwnd faster to the new sustainable rate. There could be a chance that
R is over conservative - however this is not likely to often be
significant, since the clause only applies when the sender was
non-validated.

A patch was also submitted to FreeBSD which is compliant with the new-cwv ID.

We hope this helps explain where we have arrived, and if there is
interest from the community, we would be glad to resubmit an updated
patch to implement new-cwv.

Raffaello & Gorry

---

From: "Scharf, Michael (Michael)" <[email-masked]>
Date: Fri, 17 Oct 2014 16:01:40 +0000

> key differences below, and others may comment - especially if we get

Thanks! This analysis is very useful.

Here is a random thought: I wonder if one could describe new-cwv as a set of modular components/algorithms, instead of the current "all-in-one" algorithm. Each of these components could perhaps be implemented in more than one way, obviously resulting in different trade-offs.

Actually, I doubt that there has every been an agreement among TCP stacks how to exactly perform congestion window validation, and I don't know if even inside TCPM we will ever have such a common understanding.

Thus, as a remedy to the current situation, would it be possible to put both new-cwv and the Linux approach (and possible other algorithms) under a common umbrella? Such a new-cwv I-D could first describe the basic functions that have to be solved (i.e., a framework), plus some example algorithms (i.e., the well-known variants we are aware of).

Any thoughts?

Michael (as individual contributor)