Subject: [tcpm] CWD flag use in DCTCP

From: "Scheffenegger, Richard" <[email-masked]>
Date: Tue, 11 Mar 2014 13:37:18 +0000

Hi,

Section 2.3 misses out a small, but crucial detail.

When I discussed DCTCP deployments and interoperability with Murari in Prague (IETF80), he indicated that a DCTCP sender will still send (at least?) one CWD flag in each window where cwnd was reduced.

However, none of the DCTCP papers mention the handling of the CWD flag, leaving open three potential variants:

a)      CWD is not set when DCTCP is in use
     There is no need towards a DCTCP client, as ECE doesn't need to be reset, ever. However, this badly breaks comparibility with any existing classic ECN TCP stack. I haven't investigated, but I think this is most unlike to be in the code.
b)      CWD is set once per window
     This is similar to the behavior of classic ECN TCP, and provides conservative interoperability. I suspect that this is what DCTCP code currently does - but again, haven't measured it yet.
c)      CWD is set whenever cwnd is reduced (which could be nearly every packet for a few windows time). Working towards a classic ECN TCP receiver, this could provide slightly more CE signals, when the RTT increases significantly during a congestion episode. However, the exact behavior of the interaction between DCTCP sender and classic receiver would be much more complex, with potentially unexpected outcomes. Nevertheless, this also may have gone into the code.

Furthermore, the draft is a bit vague how and when cwnd is updated. From the formula used, this appears to be the initial DCTCP update algorithm, reacting  once per window only; According to [1], dynamically updating cwnd on every receipt of ECE:

cwnd = cwnd + (1/cwnd - alpha/2)*MSS

would also address the RTT unfairness problem of the initial DCTCP algorithm. However, I understand that this draft is to document the deployed mechanism, thus that latter point would be something for a different document.

I think it would not only be me and Midori who would appreciate filling out these details in the draft.

Richard Scheffenegger

[1] Alizadeh, Mohammad, Adel Javanmard, and Balaji Prabhakar. "Analysis of DCTCP: stability, convergence, and fairness." Proceedings of the ACM SIGMETRICS joint international conference on Measurement and modeling of computer systems. ACM, 2011.

---

From: Stephen Bensley <[email-masked]>
Date: Wed, 12 Mar 2014 21:13:03 +0000

Hi Richard,

My name is Stephen Bensley. I'm one of the authors of the draft. I'm also the development lead for the Data Path and Transports team in the Operating Systems Group at Microsoft. I was not involved in the original design and implementation of DCTCP, but my team currently owns and maintains this code.

I'm guessing by CWD, you really mean the CWR flag.

The CWR handling on the sender is exactly as per RFC 3168. The receiver ignores the CWR flag.

In reviewing this, I realized that the draft is incorrect about the cwnd update behavior. In particular, cwnd is not updated whenever the TCP congestion estimate is updated. It is updated as per RFC 3168; the only difference is that cwnd is scaled by (1 - DCTCP.Alpha / 2) instead of always being reduced by half.

Windows does not implement the RTT fairness tweak suggested in the paper you referenced below.

I'll fix all of this in the next version of the draft.

Thanks for the feedback.

From: tcpm [mailto:[email-masked]] On Behalf Of Scheffenegger, Richard
Sent: Tuesday, March 11, 2014 6:37 AM
To: [email-masked]
Cc: tcpm ([email-masked])
Subject: [tcpm] CWD flag use in DCTCP

Hi,

a.       CWD is not set when DCTCP is in use
There is no need towards a DCTCP client, as ECE doesn't need to be reset, ever. However, this badly breaks comparibility with any existing classic ECN TCP stack. I haven't investigated, but I think this is most unlike to be in the code.
b.      CWD is set once per window
This is similar to the behavior of classic ECN TCP, and provides conservative interoperability. I suspect that this is what DCTCP code currently does - but again, haven't measured it yet.
c.       CWD is set whenever cwnd is reduced (which could be nearly every packet for a few windows time). Working towards a classic ECN TCP receiver, this could provide slightly more CE signals, when the RTT increases significantly during a congestion episode. However, the exact behavior of the interaction between DCTCP sender and classic receiver would be much more complex, with potentially unexpected outcomes. Nevertheless, this also may have gone into the code.

Richard Scheffenegger

---

From: "Scheffenegger, Richard" <[email-masked]>
Date: Thu, 13 Mar 2014 11:22:56 +0000

Hi Stephen,

You are correct, I meant the CWR flag, of course. And I am relieved that what I understood from Murari a few years ago, about the interaction of a DCTCP sender with a classic ECN (3168) receiver is in fact correct. You may want to dedicate a short section on that behavior, as sometimes it's not evident that such a combination works mostly similar to classic ECN, provided the marking rate becomes high enough. (I think, the EWMA term in calculating alpha will delay the reaction of DCTCP though, thus result in preferential use of bandwidth by DCTCP over 3168 classic ECN TCP; but at least classic ECN shouldn't be locked out completely)

Thank you for considering the feedback, I appreciate it.

Richard Scheffenegger

NetApp
[email-masked]<mailto:[email-masked]>
+43 1 3676811 3146 Office (2143 3146 - internal)
+43 676 654 3146 Mobile
www.netapp.com<http://www.netapp.com>

EURO PLAZA
Gebäude G, Stiege 7, 3.OG
Am Euro Platz 2
A-1120 Wien

From: Stephen Bensley [mailto:[email-masked]]
Sent: Mittwoch, 12. März 2014 22:13
To: Scheffenegger, Richard; [email-masked]
Cc: tcpm ([email-masked])
Subject: RE: CWD flag use in DCTCP

Hi Richard,

Thanks for the feedback.

From: tcpm [mailto:[email-masked]] On Behalf Of Scheffenegger, Richard
Sent: Tuesday, March 11, 2014 6:37 AM
To: [email-masked]<mailto:[email-masked]>
Cc: tcpm ([email-masked]<mailto:[email-masked]>)
Subject: [tcpm] CWD flag use in DCTCP

Hi,

Richard Scheffenegger