Subject: [tcpm] question regarding draft-gomez-tcpm-delack-suppr-reqs

From: Yuchung Cheng <[email-masked]>
Date: Wed, 29 Apr 2020 10:02:54 -0700

thanks for the talk!

https://tools.ietf.org/html/draft-gomez-tcpm-delack-suppr-reqs-01#section-3.1
"Note that, while Appropriate Byte Counting (ABC) [RFC3465] might be used
   to address this problem, it remains as an experimental mechanism, not
   fully included in RFC 5681, which specifies standard TCP congestion
   control.  "

It's not clear what "fully included" means here. AFAICT, RFC5681
includes the essence of ABC explicitly.

https://tools.ietf.org/html/rfc5681#section-3.1

" During slow start, a TCP increments cwnd by at most SMSS bytes for
   each ACK received that cumulatively acknowledges new data.  Slow
   start ends when cwnd exceeds ssthresh (or, optionally, when it
   reaches it, as noted above) or when congestion is observed.  While
   traditionally TCP implementations have increased cwnd by precisely
   SMSS bytes upon receipt of an ACK covering new data, we RECOMMEND
   that TCP implementations increase cwnd, per:

      cwnd += min (N, SMSS)                      (2)

   where N is the number of previously unacknowledged bytes acknowledged
   in the incoming ACK.  This adjustment is part of Appropriate Byte
   Counting [RFC3465] and provides robustness against misbehaving
   receivers that may attempt to induce a sender to artificially inflate
   cwnd using a mechanism known as "ACK Division" [SCWA99].  ACK
   Division consists of a receiver sending multiple ACKs for a single
   TCP data segment, each acknowledging only a portion of its data.  A
   TCP that increments cwnd by SMSS for each such ACK will
   inappropriately inflate the amount of data injected into the network."

As a side note, Linux stack has implemented the principle of abc in
its congestion control (including slow start)

regarding delayed ACK harming slow-start or latency, I am not too sure either:

1) The ACK is delayed but not the data

2) The delay of the ACK can indeed delay cwnd growth during slowstart.
But the ACK is delayed because the packet is likely a sub-MSS data,
which is again likely caused by application limit (i.e. the
application has sent all its data like a HTTP response). In this case
the delay of the cwnd growth does not hurt the performance since the
application has no data left to send.

   The only penalty occurs when the application has more data to send
but cwnd is still gated by the arrival of the delayed ACK. However, in
many request - response protocol, either the (non-delayed) ACKs
triggered by prior full-MSS data inflight or the delayed ACKs would
have returned to grow the cwnd to sending immediately.

I can construct scenarios where delayed ACK significantly hurts
application. But I don't think the scenario is too common. I will
suggest more tests to quantify the effect of delayed ACK effect on
slow-start.

---

From: Gorry Fairhurst <[email-masked]>
Date: Wed, 29 Apr 2020 18:54:56 +0100

First of all thank you for bringing this topic to the WG. I do think it 
is important that we discuss what ACKs are used for, how we expect them 
to be used and what the practical implications are for using ACKs across 
Internet paths.

About the slides, I didn't have much time at the Mic. so maybe I sounded 
rather harsh, but I would suggest we do need to set a relatively high 
bar to adopting a piece of work in this space ... and we do need data 
and real understanding of what we are trying to fix and how we can avoid 
making it worse by accident in other cases.

We went through a rather lengthy discussion in tsvwg on a similar (but 
different) topic regarding immediate negative acknowledgment for SCTP 
(RFC 7053- SACK-IMMEDIATELY Extension for SCTP) and found there were a 
few key places where for that protocol this did make sense.

However, I actually don't believe your list of issues (i). Here is 
why... (I am interested if you think differently or know more).

I've tried to roughly follow your points:

* Slow start
- I agree Delayed ACKs increase slow start, the more you delay the 
bigger the impact. However, the impact is for cases where ACKs are 
delayed on intervals comparable with or more than the Path RTT.
- DAASS is a simple fix, that appears to help a lot when there is data 
waiting for the arrival of the delayed ACK. ABC is important also (well 
what I think people understand by ABC - rather than the RFC on this).
* High bit rate environments and short data segments

If you really don’t want delayed ACKs, the protocol using TCP ought to 
disable nagle. Or you would perhaps use DAASS... I’m not sure which 
problem you speak about?

* Beyond classic ACK transmission behavior
I do agree that Delayed ACKs preclude using sender behaviors intended to 
quickly and non-intrusively probe for available capacity during slow 
start. This is important to me, but I am very wary about the idea of 
sending more ACKs to do this, and we need to be careful to gain 
sufficient information about what the receiver has seen - not just to 
get a bunch of separate (each cumulative) ACKs. This really doesn’t give 
much fidelity to the sender. I’d be happy to talk more about what might 
be better!
* IoT scenarios
I can see the argument of an ACK energy cost for the IoT device. At 
least, I would assume IoT devices can be tuned, and the apps they talk 
to can be tuned. Appropriate guidance helps!

* Bursty Apps
I think you can add varying workloads as something important.  By which 
I mean apps that do transactional stuff, or are controlled by 
applications where the data transmission need varies.When we looked at 
CWV and the ideas that the applications control the traffic patterns in 
one group of applications, we also noted that these applications change 
the way they use the network. That’s important for a timely restart to 
growing the cwnd (etc). Such applications can also be very sensitive to  
network delay.  These are probably also of interest!

So....  the reason for hesitancy overall is I see this as tricky space 
to get correct.

There are many places where fewer ACKs are good - if you have per ACK 
interrupt costs - if the capacity consumed by ACKs is important - the 
cost of sending in the link technology is high - etc. This is 
complicated for TCP (and much less so for QUIC - because QUIC has a 
notion of pacing; QUIC’s loss recovery is different; and QUIC’s ACKs are 
not easily thinned, at least at present). Any TCP method has to live 
with networks that experience pain from more ACKs, but also may deploy 
(various) mitigations.

There are also cases where you’d like more information than a delayed 
ACK provides (e.g. chirping and similar probing methods) and there are 
application pathologies that would love an ACK at the end of a packet 
burst (be that 1 or 100s of packets). Any method has to live with a wide 
variety of paths and applications.

Happy to discuss more. And I suspect other people may also respond on 
list. We have lots of data for QUIC and TCP - but most of our data 
focusses on testbeds or on actual broadband satellite services.

Gorry

---

From: Bob Briscoe <[email-masked]>
Date: Wed, 29 Apr 2020 22:58:42 +0100

Gorry,

On 29/04/2020 18:54, Gorry Fairhurst wrote:
>
>
> First of all thank you for bringing this topic to the WG. I do think 
> it is important that we discuss what ACKs are used for, how we expect 
> them to be used and what the practical implications are for using ACKs 
> across Internet paths.
>
> About the slides, I didn't have much time at the Mic. so maybe I 
> sounded rather harsh, but I would suggest we do need to set a 
> relatively high bar to adopting a piece of work in this space ... and 
> we do need data and real understanding of what we are trying to fix 
> and how we can avoid making it worse by accident in other cases.
>
I appreciate you taking time to qualify your initial harsh words. I 
would ask you to still take even more time over this review. People 
listen to you on this subject, and I think you are glossing over large 
areas of the issue too quickly.

> waiting for the arrival of the delayed ACK. 

Any heuristic at the receiver to detect the end of a pattern of start-up 
such as slow-start, ossifies that pattern into the Internet. Sender 
control of the DelACK ratio removes that ossification.

> ABC is important also (well what I think people understand by ABC - 
> rather than the RFC on this).

Yes.

> I do agree that Delayed ACKs preclude using sender behaviors intended 
> to quickly and non-intrusively probe for available capacity during 
> slow start. This is important to me, but I am very wary about the idea 
> of sending more ACKs to do this, and we need to be careful to gain 
> get a bunch of separate (each cumulative) ACKs. This really doesn’t 
> give much fidelity to the sender. I’d be happy to talk more about what 
> might be better!

Indeed, ACK frames in QUIC that give the arrival times of data packets 
are an alternative - as long as they are not stretched beyond the time 
at which they would have been useful.

But we are more limited for space in TCP. As long as a TCP server is 
serving bulk and short flows, it can put more effort into receiving more 
ACKs during flow startup, and save its processing power by using less 
frequent ACKs for the majority of its data - transmitted in congestion 
avoidance.

> to can be tuned. Appropriate guidance helps!

Tuning is for a scenario, not for widely differing scenarios. The 
message of this draft is that a connection sometimes needs to be able to 
adapt its ACK ratio at run-time. Not every IoT device is always deployed 
for just one scenario. Neither is its peer. To me this says adaptation 
mustn't be complex (and doesn't have to be).

>
> * Bursty Apps
> I think you can add varying workloads as something important.  By 
> which I mean apps that do transactional stuff, or are controlled by 
> CWV and the ideas that the applications control the traffic patterns 
> in one group of applications, we also noted that these applications 
> change the way they use the network. That’s important for a timely 
> restart to growing the cwnd (etc). Such applications can also be very 
> sensitive to  network delay.  These are probably also of interest!

Yes.

> So....  the reason for hesitancy overall is I see this as tricky space 

But trickiness is the point of starting this requirements document, 
isn't it?
Please can you articulate better why you seem negative about this exercise?

> notion of pacing; QUIC’s loss recovery is different; and QUIC’s ACKs 
> are not easily thinned, at least at present). Any TCP method has to 
> live with networks that experience pain from more ACKs, but also may 
> deploy (various) mitigations.

Yes. But reading between the lines I hear you say "Don't start this 
exercise".
Pls explicitly articulate your implicit message.

> burst (be that 1 or 100s of packets). Any method has to live with a 
> wide variety of paths and applications.
>

Thanks for elaborating.

Bob

>
> Gorry
>
>
>
> _______________________________________________
> tcpm mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tcpm

-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/

---

From: Gorry Fairhurst <[email-masked]>
Date: Thu, 30 Apr 2020 09:36:47 +0100

See some more explanation in line.

On 29/04/2020 22:58, Bob Briscoe wrote:
>> them to be used and what the practical implications are for using 
>> ACKs across Internet paths.
>>
>> We went through a rather lengthy discussion in tsvwg on a similar 
>> (but different) topic regarding immediate negative acknowledgment for 
>> SCTP (RFC 7053- SACK-IMMEDIATELY Extension for SCTP) and found there 
>> were a few key places where for that protocol this did make sense.
>>
>> - DAASS is a simple fix, that appears to help a lot when there is 
>> data waiting for the arrival of the delayed ACK. 
>
> Any heuristic at the receiver to detect the end of a pattern of 
> start-up such as slow-start, ossifies that pattern into the Internet. 
> Sender control of the DelACK ratio removes that ossification.
>
I initially thought the same, DAASS tried to fix multiple problems. 
However, as regards DAASS  reducing the waiti for the ACK to a rounds of 
initial cwnd growth, I think the key understanding is the sender use of 
byte accounting (for stretch-ACKs) not individual ACKs, (that might 
imply a need for some burst-mitigation for stretch ACKs, because you 
reduce ACK clocking).

I like what I understand Chrome's QUIC does:  In the first 100 received 
packets, send an ACK for each packet (could be each pair of packets).  
This rapidly grows a cwnd of about 140 packets (assuming exponetial 
growth). Once the cwnd is >100 then an ACK covering 10 packets/segments 
becomes a small proportion of the cwnd (the additional delay from 
per-packet ACKs is small). The receiver can moves to a larger default 
ACK ratio.

>> If you really don’t want delayed ACKs, the protocol using TCP ought 
>> to disable nagle. Or you would perhaps use DAASS... I’m not sure 
>> which problem you speak about?
>>
>>
>> slow start. This is important to me, but I am very wary about the 
>> idea of sending more ACKs to do this, and we need to be careful to 
>> gain sufficient information about what the receiver has seen - not 
>> just to get a bunch of separate (each cumulative) ACKs. This really 
>> doesn’t give much fidelity to the sender. I’d be happy to talk more 
>> about what might be better!
>
Interesting, I think.
> serving bulk and short flows, it can put more effort into receiving 
> more ACKs during flow startup, and save its processing power by using 
> less frequent ACKs for the majority of its data - transmitted in 
> congestion avoidance.
>
>
Also seems true.
>> least, I would assume IoT devices can be tuned, and the apps they 
>> talk to can be tuned. Appropriate guidance helps!
>
> message of this draft is that a connection sometimes needs to be able 
> to adapt its ACK ratio at run-time. Not every IoT device is always 
> deployed for just one scenario. Neither is its peer. To me this says 
> adaptation mustn't be complex (and doesn't have to be).
>
Heterogeniety of the network segments is tricky. Some optimisations 
could be handled by link-specific methods - header compression perhaps 
is an example, other things need to be adapted end-to-end. However, 
end-to-end transport has to work for all types of path (links) - and 
often does not know the path.
>> applications where the data transmission need varies.When we looked 
>> at CWV and the ideas that the applications control the traffic 
>> patterns in one group of applications, we also noted that these 
>> applications change the way they use the network. That’s important 
>> for a timely restart to growing the cwnd (etc). Such applications can 
>> also be very sensitive to network delay.  These are probably also of 
>> interest!
>
> Yes.
>
>>
>> So....  the reason for hesitancy overall is I see this as tricky 
>> space to get correct.
>
The slide asked about adopting this work - my words were against doing 
that at that time with that revision of this document.

I miss seeing presentations with results and showing the thinking. I  
wonder if we need a specific agenda slot to focus on such topics. I 
expect we'll learn a lot from some data from a variety of 
problem-spaces/use-cases, especially if the timeslot is sufficient to 
include thinking and experience from people outside of TCPM. (That's 
another thought - and probably should be another thread).

>> ACK provides (e.g. chirping and similar probing methods) and there 
>> are application pathologies that would love an ACK at the end of a 
>> packet burst (be that 1 or 100s of packets). Any method has to live 
>> with a wide variety of paths and applications.
>>
Gorry