Subject: [tcpm] On Sender Control of Delayed Acks in TCP

From: "Scheffenegger, Richard" <[email-masked]>
Date: Wed, 29 Apr 2020 19:02:03 +0200

Hi,

as many here know, I'm very much interested in transactional data
exchange across TCP sessions, rather than the more common architectural
approach of assuming unlimited data to transmit unidirectional.

One of the issues in that space is, that these application limited
sessions can very frequently run out of data, and expirience idle periods.

Eliciting an ACK under certain circumstances, for a timely continuation
of the data transmission / growth of the congestion window is a known
method to reduce network delay.

E.g. Linux has been using the CWR flag for the purpose of sending out an
immediate ACK by the receiver, since there is an increased chance of a
very small cwnd when the sender had just reduced it's congestion window.

https://lore.kernel.org/patchwork/patch/970486/

and we also found latency improvements doing this when running
ECN-enabled sessions

https://reviews.freebsd.org/D22670

There are also numerous advisories, suggesting to disable delayed ACK
completely when latency and/or timely bandwidth recovery under
adversarial conditions is a major driver (e.g.
https://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/techpaper/performance/esxi7-nfs-read-perf.pdf
for one that was just published very recently).

To summarize: While I agree that under steady load conditions, there are
probably some benefits to be had by TCP itself thinning out ACKs, and
address certain environment (e.g. asymetric link bandwidth), there are
also conditions, where a timely ACK is crucial in maintaining low
latency and timely bandwidth recovery. Short of having a mechansim do
explicitly do this, at least for the latter issue, TLP (or TLP-like)
methods are the only ones to currently address that - but the duplicate
unconditional transmission of a (small) data packet does have additional
cost associated with it.

Best regards,
    Richard

---

From: Neal Cardwell <[email-masked]>
Date: Wed, 29 Apr 2020 17:07:51 -0400

> https://reviews.freebsd.org/D22670

Yes, agreed.

IMHO an explicit ACK-pull mechanism would be very nice for the cwnd<=1 case.

In datacenter networks the BDPs tend to be so small, and the numbers
of flows can be so large, that it's not unusual for a flow's fair
share of the BDP to be one packet or less. If the flows want to
maintain low queuing delays, then ideally they would be able to reduce
their cwnd to 1 packet (or effectively even lower, with pacing). But
the current behavior of TCP delayed ACKs often causes terrible delays
with a cwnd of 1.

The Linux TCP approach of sending an immediate ACK upon receiving a
packet with CWR set (see Richard's link above) helps a lot with this
case. But that's a non-standard heuristic, and doesn't help for
senders that have a congestion control that does not use ECN CWR
signals (e.g. Accurate ECN). Unless I'm missing something, it seems
like endpoints using Accurate ECN are going to run into this same
latency issue. To avoid this, I guess they may want to either use a
cwnd >= 2, or make use of some new explicit ACK-pull mechanism.

best,
neal

---

From: Jonathan Morton <[email-masked]>
Date: Thu, 30 Apr 2020 00:33:35 +0300

> On 30 Apr, 2020, at 12:07 am, Neal Cardwell <ncardwell=[email-masked]> wrote:
> 

Just to note, this is a problem that SCE wouldn't be affected by, as it doesn't change the ECE/CWR handling.

 - Jonathan Morton