Subject: [tcpm] New Version Notification for draft-grimes-tcpm-tcpsce-00.txt

From: "Rodney W. Grimes" <[email-masked]>
Date: Mon, 8 Jul 2019 17:07:19 -0700 (PDT)

I have posted a new version with updated proper tcpm working group
name of draft-grimes-tcpm-tcpsce-00
(formerly draft-grimes-tcpmwg-tcpsce-00.txt)

This is first cut at how SCE works in TCP

Cross posting this to both tsvwg and tcpm due to overlap

> A new version of I-D, draft-grimes-tcpm-tcpsce-00.txt
> has been successfully submitted by Rodney W. Grimes and posted to the
> IETF repository.
> 
> Name:		draft-grimes-tcpm-tcpsce
> Revision:	00
> Title:		Some Congestion Experienced in TCP
> Document date:	2019-07-08
> Group:		Individual Submission
> Pages:		5
> URL:            https://www.ietf.org/internet-drafts/draft-grimes-tcpm-tcpsce-00.txt
> Status:         https://datatracker.ietf.org/doc/draft-grimes-tcpm-tcpsce/
> Htmlized:       https://tools.ietf.org/html/draft-grimes-tcpm-tcpsce-00
> Htmlized:       https://datatracker.ietf.org/doc/html/draft-grimes-tcpm-tcpsce
> 
> 
> Abstract:
>    This memo classifies a TCP code point ESCE ("Echo Some Congestion
>    Experienced") for use in feedback of IP code point SCE ("Some
>    Congestion Experienced").

-- 
Rod Grimes                                                 [email-masked]

---

From: "Black, David" <[email-masked]>
Date: Fri, 12 Jul 2019 17:18:59 +0000

Rod,

This draft appears to be focused on obtaining a TCP header bit for SCE.

As an alternative to this single-bit proposal, please take a look at draft-ietf-tcpm-accurate-ecn (https://datatracker.ietf.org/doc/draft-ietf-tcpm-accurate-ecn/) and consider how that functionality might (or might not) be usable with SCE. 

Thanks, --David

> -----Original Message-----
> From: tsvwg <[email-masked]> On Behalf Of Rodney W. Grimes
> Sent: Monday, July 8, 2019 8:07 PM
> To: [email-masked]; [email-masked]
> Subject: [tsvwg] New Version Notification for draft-grimes-tcpm-tcpsce-
> 00.txt
> 
> 
> [EXTERNAL EMAIL]
>

---

From: Jonathan Morton <[email-masked]>
Date: Fri, 12 Jul 2019 21:10:12 +0300

> On 12 Jul, 2019, at 8:18 pm, Black, David <[email-masked]> wrote:
> 

We did look into this, but found some shortcomings that simple reuse of the NS bit (or assigning one of the other spare TCP header bits adjacent to NS) neatly overcomes.

First is that much of AccECN is focused on providing accurate feedback of CE marks, which is not very relevant to SCE since the existing ECE/CWR mechanism from RFC-3168 works well.  For this purpose the NS bit is also consumed, forming a 3-bit field that straddles a byte boundary and is thus awkward to parse.

A simple transformation of this field into one that counts SCE marks doesn't work, because feedback of CE marks is then lost.  Retaining the original response to purely CE-based congestion feedback is a crucial part of SCE's design for incremental deployment.

To get feedback of ECT(1) which carries the SCE information, the full length of the AccECN TCP Option must be used, inflating the volume of the ack stream - and concerns have been raised about TCP Options in discussions here.  A similar concern made one of our other ideas (replacing TCP Timestamps with a slight larger option that would absorb the NOP padding currently used) less desirable, though that also avoided the ack-inflation problem.

We therefore tried to explore ideas that posed minimum risk of confusing firewalls or increasing software complexity.  We considered that reusing a bit that had previously been assigned to an ECT(1) related mechanism would be safest, and subsequently found that it could convey all the information needed with sufficient reliability.

I agree that this draft could be improved to explain the above choice and give more detail on the expected semantics.  I'll work with Rod to sort that out.

 - Jonathan Morton

---

From: "Rodney W. Grimes" <[email-masked]>
Date: Fri, 12 Jul 2019 12:39:47 -0700 (PDT)

> This draft appears to be focused on obtaining a TCP header bit for SCE.

Yes, that is correct.

> As an alternative to this single-bit proposal, please take a look at draft-ietf-tcpm-accurate-ecn (https://datatracker.ietf.org/doc/draft-ietf-tcpm-accurate-ecn/) and consider how that functionalit…

I specifically did not do what Accurate Ecn attempts as that requires
a TCP option negotiation at connection establishment to negotiate the
overloading of other TCP header bits.  SCE only needs 1 new bit, and
does not alter the behavior of any current bits.

This proposal (TCPSCE) does not in any way effect the current use of
any of the other bits, and use of the NS bit should be fully backwards
compatible and ignored by pre SCE implementations and does not require
a TCP option negotiation.

I have to get additional data but have been lead to understand that
adding a tcp option and doing that negotiation is going to cause a
great deal of pain in the ability to deploy accurate ecn as it is
currently designed due to middle box inspection and handling.

Regards,
Rod

---

From: Bob Briscoe <[email-masked]>
Date: Thu, 25 Jul 2019 14:20:02 -0400

Rod,

> does not alter the behavior of any current bits.
Just to correct this slightly - in AccECN, there is no explicit TCP 
option negotiation. Negotiating use of the 3 header bits also implicitly 
states that the endpoints might have implemented the TCP option. 
However, it's not mandatory to send the option (it is recommended), and 
it's not mandatory to even implement the logic to read in the option 
(that is highly recommended). Reason: the protocol has to be able to 
work even if the option is stripped on the path.

> currently designed due to middle box inspection and handling.
There's already been concern about burning TCP header codepoints, which 
might use up space that prevents something in future. The idea of AccECN 
is to enable generic more accurate feedback. It was based on a 
protracted requirements-gathering exercise [RFC7560], and lots of 
compromising along the way.

The idea was to have a generic wire protocol with a dumb receiver, so 
that the same feedback protocol could support multiple needs for 
feedback by different TCP congestion control algorithms.

So a fairly inefficient re-use of the 'NS' TCP header flag for one 
particular experiment is very unlikely to fly, particularly when the 
experiment it supports doesn't satisfy all the requirements in 7560.

For instance, I think the reason the tcpsce draft discusses multiple 
ways of doing the feedback is that, in the presence of pure ACK loss 
(which is often due to deliberate ACK thinning), none of the three 
solutions preserve reliable delivery of the ACK signal.

In the case of Single ACK (4.1), the additional ACK load would be 
unacceptable.

In the case of immediate ACKing of an IP-SCE (not delaying the ACK - 
S.4.2), it introduces bias (i.e. a higher proportion of ESCE ACKs to 
non-ESCE), so it becomes hard/impossible for the sender to reverse 
engineer what congestion feedback level was intended.

In the case of dithering (4.3), the congestion signal frequently becomes 
deferred (and there may not be a next packet for some time).

The requirements [RFC7560] recognise that this is a multi-dimensional 
compromise so any solution had to explain the rationale for its 
particular choices.

However, I don't think this solution is anywhere near as efficient in 
its use of the limited bit-space as many of the other rejected AccECN 
proposals have been, let alone the currently chosen one.

Bob

-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/

---

From: Jonathan Morton <[email-masked]>
Date: Thu, 25 Jul 2019 14:53:45 -0400

> On 25 Jul, 2019, at 2:20 pm, Bob Briscoe <[email-masked]> wrote:
> 
> The idea was to have a generic wire protocol with a dumb receiver, so that the same feedback protocol could support multiple needs for feedback by different TCP congestion control algorithms.
> 
> So a fairly inefficient re-use of the 'NS' TCP header flag for one particular experiment is very unlikely to fly, particularly when the experiment it supports doesn't satisfy all the requirements in 7560.

AccECN burns the same bit to provide higher fidelity feedback of CE, without addressing our need to feed back the distinction between ECT(1) and ECT(0) at all (unless the TCP Option is used).  Since higher fidelity feedback of CE is not useful for SCE, using NS in this way is actually more efficient for us.  Happily, AccECN and SCE can coexist on different flows, thanks to the fact that AccECN does have a negotiation phase which SCE can naturally reject.

> For instance, I think the reason the tcpsce draft discusses multiple ways of doing the feedback is that, in the presence of pure ACK loss (which is often due to deliberate ACK thinning), none of the three solutions preserve reliable delivery of the ACK signal.

In SCE, *reliable* feedback of SCE signals is not actually required, both because the control loop is naturally stable, and because RFC-3168's CE feedback *is* reliable and thus offers a safe fallback.  Of course we understand that the design constraints for L4S' feedback mechanism were different.

Ack thinning is also something we have explicitly considered, given that Cake includes an optional ack-filter which does exactly that.  (We have, for example, added consideration of the NS bit to Cake's ack-filter, which was a trivial patch.)  Mathematically, the most extreme errors possible in either direction, due to ack thinning, are easily corrected during subsequent RTTs.

 - Jonathan Morton

---

From: Bob Briscoe <[email-masked]>
Date: Thu, 25 Jul 2019 16:27:19 -0400

Jonathan,

On 25/07/2019 14:53, Jonathan Morton wrote:
> AccECN burns the same bit to provide higher fidelity feedback of CE, without addressing our need to feed back the distinction between ECT(1) and ECT(0) at all (unless the TCP Option is used).  Since higher fidelity feedback of CE is not useful for SCE, using NS in this way is actually more efficient for us.
As you say, AccECN does support SCE - with the option. That was because 
it was generally agreed that accuracy of the CE signal was paramount, 
rather than trying to do two things with 3 bits and under-achieving for 
both.

We did have another scheme with 5 & 3 codepoints in the 3 bits for two 
counters - look back over the draft history. But the WG decided 
simplicity was also important in a CC protocol, cos lack of bugs is also 
important.
>   Happily, AccECN and SCE can coexist on different flows, thanks to the fact that AccECN does have a negotiation phase which SCE can naturally reject.
You will find there is resistance to starting to use the NS flag after 
having negotiated RFC3168 ECN feedback, but without any explicit 
negotiation. You will probably be asked how a future experiment would be 
able to use the NS flag if your experiment fails (assuming it is adopted 
as IETF work in the first place). You are walking into a world of bit 
scarcity.

> In SCE, *reliable* feedback of SCE signals is not actually required, both because the control loop is naturally stable, and because RFC-3168's CE feedback *is* reliable and thus offers a safe fallb…
My point wasn't so much about safety, it was about about accuracy and 
particularly biased error. The essence of low latency congestion control 
is keeping the queue low without losing utilization, which requires 
accuracy. If your feedback has a biased but unknown error, it's really 
hard to accurately converging towards a target.

> Ack thinning is also something we have explicitly considered, given that Cake includes an optional ack-filter which does exactly that.  (We have, for example, added consideration of the NS bit to C…
Exactly - if your feedback has a biased error, you will usually miss the 
target and try again in the next round. No point continuing this 
discussion - I'm just saying you'll need to see what happens in reality 
when the feedback is thinned.

Bob

---

From: "Rodney W. Grimes" <[email-masked]>
Date: Fri, 26 Jul 2019 01:54:57 -0700 (PDT)

Bob,

Responding to specifics of draft-grimes-tcpm-tcpsce-00.txt
conversation in this thread as its author.

> rather than trying to do two things with 3 bits and under-achieving for 

It supports TCPSCE only in the sense that once AccECN TWH fails
to find a L4S endpoint it does fall back to ECN as defined in
RFC3168 which TCPSCE builds upon by reuse of the NS bit which
was freed up by RFC8311 (to provide a feedback path for the
higher fadelity SCE forward mark.)

Further more by using this fall back position it means that
none of the remaining AccENC TWH code points that exist today
would need to be consumed in an attempt to negotiate this use.

Given that many of the 8 code points created by doing the L4S
TWH modifications are already consumed by L4S and that if you
do the modified L4S handshake you gain access to all of the
ECN TCP layer bits I feel it would be wasteful to consume that
also limited resource for a single bit access to NS vs a possible
future 3 bit access to (NS, ECE, CWR)

> simplicity was also important in a CC protocol, cos lack of bugs is also 

Intresting end phrase "lack of bugs is also important", I shall simply
state that SCE and SCETCP as running code is a significantly small amount
of code and the fact that a prototype working implementation was completed
in ~24 man hours speaks to its simplicty and elagance of design.

Though I would not qualify that as Internet deployable code, it was
adaquate such that we could conduct experiments and debug the IP
layer SCE marking code being implemented in a Linux versioned middle box.

> as IETF work in the first place). You are walking into a world of bit 

See above, we are leaving the remaining AccECN TCP TWH code points
open for future override of NS/ACE, ECE, and CWR rather than consume
the code points left in that TWH to consome 1 bit in what is effectely
an unused code point in the TWH (fall back to RFC3168 ECN).

> hard to accurately converging towards a target.

We have found no issues in our testing thus far that would indicated
that either of the (small) error amount or bias direction has a significant
impact on the convergence rate or target of the closed loop system.
Including doing a random drop of ack packet test.  I can agree that the
higher the accuracy of this feedback the better, but the solution is robust
as designed.  Much as you do not need super high accuracy input to
a steering wheel to keep a car centered in its curving lane.

I would not qualify this as "really hard to accurately converge",
if it was most closed loop control loop systems would fail.  If
on the other hand it was open loop then the accuracy would need
to be near perfect, is that the issue your trying to raise?

> when the feedback is thinned.

We are aware of the ack thinning concerns, and do plan to fully evaluate
that as a potential problem, though to date our experiments indicate it
is minimally impacting.  There is also all the discussion going on about
maybe ack thinning is not such a grand idea on its own merits.