From nobody Mon May 4 06:16:08 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2FE8E1A009C for ; Mon, 4 May 2015 06:16:07 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.511 X-Spam-Level: X-Spam-Status: No, score=-1.511 tagged_above=-999 required=5 tests=[BAYES_50=0.8, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id EZP_y7Vqdrfo for ; Mon, 4 May 2015 06:16:02 -0700 (PDT) Received: from pegasus.erg.abdn.ac.uk (pegasus.erg.abdn.ac.uk [139.133.204.173]) by ietfa.amsl.com (Postfix) with ESMTP id 15EE71A0099 for ; Mon, 4 May 2015 06:16:01 -0700 (PDT) Received: from erg.abdn.ac.uk (galactica.erg.abdn.ac.uk [139.133.210.32]) by pegasus.erg.abdn.ac.uk (Postfix) with ESMTPA id 8CDBB1B001DA for ; Mon, 4 May 2015 14:16:11 +0100 (BST) Received: from 212.159.18.54 (SquirrelMail authenticated user gorry) by erg.abdn.ac.uk with HTTP; Mon, 4 May 2015 14:15:13 +0100 Message-ID: <358f1d40d2b8d38605f322d4b70e8b46.squirrel@erg.abdn.ac.uk> In-Reply-To: References: <655C07320163294895BBADA28372AF5D16C939DA@FR712WXCHMBA15.zeu.alcatel-lucent.com> Date: Mon, 4 May 2015 14:15:13 +0100 From: gorry@erg.abdn.ac.uk To: "tcpm@ietf.org" User-Agent: SquirrelMail/1.4.23 [SVN] MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal Archived-At: Subject: Re: [tcpm] Feedback on WG acceptance of draft-zimmermann-tcpm-cubic-01 X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 May 2015 13:16:07 -0000 I support work on this topic. I think that such a document should be maintained within TCPM. Here are my thoughts: - WG informational draft It changes TCP, but there is quite a lot of understanding of how cubic operates. It could document what is currently implemented, and I think would be useful exactly in this form. My take is that I do not think it really competes with any planned IETF work - but would introduce an alternate CC. A focus on correcting the text to reflect what is implemented could make this a fast procedure. That would be a major selling point to me. - EXP? I think it could be published as EXP. The classification at first seems odd, in that there may be more operational deployment experience with CUBIC than many IETF specs. However, at least it is more "aggressive" in CC than some IETF-defined specs. If the intent of EXP status was to flag this as not yet finally perfected by IETF experience and to show that the IETF could in future improve this further, then that would seem a reasonable position. (I'm not even saying I know of a list of improvements to make). - PS? If the WG evaluation concludes that the evidence is good, then it could be a PS, that would need a consensus call. That call is I suggest impossible to make until we open the document up to detailed scrutiny and debate. I can see merits in publishing as INFO asap. Gorry (as an individual) > wrote: >> Dear all, >> >> During IETF 91 [1] and on the list there has been strong support for WG >> adoption of draft-zimmermann-tcpm-cubic [2]. It seems consensus in TCPM >> to adopt this document. >> >> In order to move forward, the chairs seek for additional community >> guidance regarding the intended status: >> >> a) Proposed standard >> >> b) Experimental >> >> c) Informational >> > From nobody Thu May 7 07:59:41 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DD9421A9115 for ; Thu, 7 May 2015 07:59:39 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 1.149 X-Spam-Level: * X-Spam-Status: No, score=1.149 tagged_above=-999 required=5 tests=[BAYES_50=0.8, HELO_EQ_FR=0.35, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GmoYyXUHHa9Q for ; Thu, 7 May 2015 07:59:38 -0700 (PDT) Received: from zproxy220.enst-bretagne.fr (zproxy220.enst-bretagne.fr [192.108.117.9]) by ietfa.amsl.com (Postfix) with ESMTP id 0BA5D1A912D for ; Thu, 7 May 2015 07:59:34 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by zproxy220.enst-bretagne.fr (Postfix) with ESMTP id ADEA130270 for ; Thu, 7 May 2015 16:59:32 +0200 (CEST) Received: from zproxy220.enst-bretagne.fr ([127.0.0.1]) by localhost (zproxy220.enst-bretagne.fr [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id GJd88-LGsU8m for ; Thu, 7 May 2015 16:59:32 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by zproxy220.enst-bretagne.fr (Postfix) with ESMTP id 427743026E for ; Thu, 7 May 2015 16:59:32 +0200 (CEST) X-Virus-Scanned: amavisd-new at zproxy220.enst-bretagne.fr Received: from zproxy220.enst-bretagne.fr ([127.0.0.1]) by localhost (zproxy220.enst-bretagne.fr [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id ZzMc3ADTrqKN for ; Thu, 7 May 2015 16:59:32 +0200 (CEST) Received: from [IPv6:2001:660:7301:3728:7029:3b3:d619:ec9b] (passerelle-interne.enst-bretagne.fr [192.108.117.210]) by zproxy220.enst-bretagne.fr (Postfix) with ESMTPSA id 1F23E30270 for ; Thu, 7 May 2015 16:59:32 +0200 (CEST) From: Nicolas Kuhn Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Message-Id: <882B8BE8-D02D-4E46-90B0-FE9CB2E88BF9@telecom-bretagne.eu> Date: Thu, 7 May 2015 16:59:32 +0200 To: tcpm@ietf.org Mime-Version: 1.0 (Mac OS X Mail 8.0 \(1990.1\)) X-Mailer: Apple Mail (2.1990.1) Archived-At: Subject: [tcpm] draft-ietf-tcpm-rtorestart-07 X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 May 2015 14:59:40 -0000 Hi all,=20 I had a look at this draft-ietf-tcpm-rtorestart-07 draft.=20 I have some minor comments on the document.=20 ############################################################# - "The RTO Restart (RTOR) mechanism described in this document makes the RTO slightly more aggressive when the number of outstanding segments is too small for fast retransmit to work, in an attempt to enable faster loss recovery for all segments while being robust to reordering." The RTO is not aggressive ; but the retransmission scheme ruled by RTO = expirations is.=20 I would recommend to rephrase that.=20 ############################################################# - =E2=80=9C The RTO Restart (RTOR) mechanism described in this = document makes the RTO slightly [=E2=80=A6] highly varying RTTs, e.g. mobile networks." This whole paragraph comes to early in the document, as the reader has = no clear=20 idea about what RTOR is actually doing. I propose to add that in a = dedicated sub- section in Section 5. ############################################################# - =E2=80=9C 3 RTO Restart Overview=E2=80=9D -> should it not be =E2=80=9C3= - RTO Overview and rationale of RTOR =E2=80=9C or something equivalent ? It is confusing, as you do not actually present RTOR.=20 Also, in this section 3: The first paragraph finishes with =E2=80=9CHence, in most situations, = the time before a retransmission is triggered is equal to "RTO + = RTT=E2=80=9D.=E2=80=9D=20 But the example to illustrate it comes after - I think things should = moved around to: 1- introduce the classic RTO=20 2- explain the rationale of classic RTO 3- example to show that RTO+RTT is common 4- more parameters could affect that (2 outstanding segments; other = factors) 5- conclusion on =E2=80=9Chence, RTO+RTT is common for application = limited traffic" I would propose to introduce a ToC for this section 3: 3- RTO Overview and rationale of RTOR 3-1. Experienced RTO is RTO+RTT 3-2. RTO safety margin 3-3. Rationale of RTOR: when fast retransmit is not possible ############################################################# Section 4 : should focus on the description of the algorithm only.=20 I would keep=20 =E2=80=9C=20 This approach makes the RTO more aggressive than the standardized approach in [RFC6298] but still conforms to the requirement in [RFC6298] that segments must not be retransmitted earlier than RTO seconds after their original transmission. " for the discussion section (at this stage of the reading, the reader = still does not know exactly what RTOR is actually doing).=20 ############################################################# Section 4: "it will prevent RTOR from being more aggressive and potentially causing RTOs instead of fast retransmits.=E2=80=9D More aggressive than what ? Has the "four" being experimentally obtained = ? I think more discussion is required on this parameter.=20 Regards,=20 Nicolas= From nobody Fri May 8 02:33:13 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 789E91A8945; Fri, 8 May 2015 02:33:10 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -101.9 X-Spam-Level: X-Spam-Status: No, score=-101.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, USER_IN_WHITELIST=-100] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jA-xdfCHiy4K; Fri, 8 May 2015 02:33:07 -0700 (PDT) Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 727061A000B; Fri, 8 May 2015 02:33:07 -0700 (PDT) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: The IESG To: "IETF-Announce" X-Test-IDTracker: no X-IETF-IDTracker: 6.0.2.p2 Auto-Submitted: auto-generated Precedence: bulk Sender: Message-ID: <20150508093307.15938.17355.idtracker@ietfa.amsl.com> Date: Fri, 08 May 2015 02:33:07 -0700 Archived-At: Cc: tcpm@ietf.org Subject: [tcpm] Last Call: (Updating TCP to support Rate-Limited Traffic) to Experimental RFC X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Reply-To: ietf@ietf.org List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 May 2015 09:33:10 -0000 The IESG has received a request from the TCP Maintenance and Minor Extensions WG (tcpm) to consider the following document: - 'Updating TCP to support Rate-Limited Traffic' as Experimental RFC The IESG plans to make a decision in the next few weeks, and solicits final comments on this action. Please send substantive comments to the ietf@ietf.org mailing lists by 2015-05-22. Exceptionally, comments may be sent to iesg@ietf.org instead. In either case, please retain the beginning of the Subject line to allow automated sorting. Abstract This document provides a mechanism to address issues that arise when TCP is used to support traffic that exhibits periods where the sending rate is limited by the application rather than the congestion window. It provides an experimental update to TCP that allows a TCP sender to restart quickly following a rate-limited interval. This method is expected to benefit applications that send rate-limited traffic using TCP, while also providing an appropriate response if congestion is experienced. It also evaluates the Experimental specification of TCP Congestion Window Validation, CWV, defined in RFC 2861, and concludes that RFC 2861 sought to address important issues, but failed to deliver a widely used solution. This document therefore recommends that the status of RFC 2861 is moved from Experimental to Historic, and that it is replaced by the current specification. The file can be obtained via https://datatracker.ietf.org/doc/draft-ietf-tcpm-newcwv/ IESG discussion can be tracked via https://datatracker.ietf.org/doc/draft-ietf-tcpm-newcwv/ballot/ No IPR declarations have been submitted directly on this I-D. From nobody Tue May 12 14:21:49 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B8FF91A9035 for ; Tue, 12 May 2015 14:21:48 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.91 X-Spam-Level: X-Spam-Status: No, score=-6.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id L3aHukARyaEs for ; Tue, 12 May 2015 14:21:46 -0700 (PDT) Received: from boreas.isi.edu (boreas.isi.edu [128.9.160.161]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id C110D1A902C for ; Tue, 12 May 2015 14:21:46 -0700 (PDT) Received: from [128.9.160.252] (pen.isi.edu [128.9.160.252]) (authenticated bits=0) by boreas.isi.edu (8.13.8/8.13.8) with ESMTP id t4CLDA3H009635 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Tue, 12 May 2015 14:13:10 -0700 (PDT) Message-ID: <55526CE4.70906@isi.edu> Date: Tue, 12 May 2015 14:13:08 -0700 From: Joe Touch User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.6.0 MIME-Version: 1.0 To: RFC Errata System , mankin@psg.com, rbonica@juniper.net, spencerdawkins.ietf@gmail.com, mls.ietf@gmail.com, michael.scharf@alcatel-lucent.com, nishida@sfc.wide.ad.jp, pasi.sarolahti@iki.fi References: <20150512204619.0B5CE180092@rfc-editor.org> In-Reply-To: <20150512204619.0B5CE180092@rfc-editor.org> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Archived-At: Cc: tcpm@ietf.org Subject: Re: [tcpm] [Technical Errata Reported] RFC5925 (4365) X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 May 2015 21:21:48 -0000 Hi, all, Thanks to Michael Sharf for pointing this out. This can be flagged as "wait for update" because it's not critical to the operation of the option. Joe On 5/12/2015 1:46 PM, RFC Errata System wrote: > The following errata report has been submitted for RFC5925, > "The TCP Authentication Option". > > -------------------------------------- > You may review the report below and at: > http://www.rfc-editor.org/errata_search.php?rfc=5925&eid=4365 > > -------------------------------------- > Type: Technical > Reported by: Joe Touch > > Section: 7.6 > > Original Text > ------------- > TCP's 4-bit data offset requires that the options end 60 bytes (15 > 32-bit words) after the header begins, including the 20-byte header. > This leaves 40 bytes for options, of which 15 are expected in current > implementations (listed below), leaving at most 25 for other uses. > TCP-AO consumes 16 bytes, leaving 9 bytes for additional SYN options > (depending on implementation dependant alignment padding, which could > consume another 2 bytes at most). > > o SACK permitted (2 bytes) [RFC2018][RFC3517] > > o Timestamps (10 bytes) [RFC1323] > > o Window scale (3 bytes) [RFC1323] > > > Corrected Text > -------------- > TCP's 4-bit data offset requires that the options end 60 bytes (15 > 32-bit words) after the header begins, including the 20-byte header. > This leaves 40 bytes for options, of which 19 are expected in current > implementations (listed below), leaving at most 21 for other uses. > TCP-AO consumes 16 bytes, leaving 5 bytes for additional SYN options > (depending on implementation dependent alignment padding, which could > consume another 2 bytes at most). > > o SACK permitted (2 bytes) [RFC2018][RFC3517] > > o Timestamps (10 bytes) [RFC1323] > > o Window scale (3 bytes) [RFC1323] > > o Maximum Segment Size (4 bytes) [RFC793] > > > Notes > ----- > MSS was missing in the original text. New text includes MSS and updates numbers accordingly. > > Also corrects a spelling error (dependant -> dependent), which is non-technical but included in the revised text. > > Instructions: > ------------- > This erratum is currently posted as "Reported". If necessary, please > use "Reply All" to discuss whether it should be verified or > rejected. When a decision is reached, the verifying party (IESG) > can log in to change the status and edit the report, if necessary. > > -------------------------------------- > RFC5925 (draft-ietf-tcpm-tcp-auth-opt-11) > -------------------------------------- > Title : The TCP Authentication Option > Publication Date : June 2010 > Author(s) : J. Touch, A. Mankin, R. Bonica > Category : PROPOSED STANDARD > Source : TCP Maintenance and Minor Extensions > Area : Transport > Stream : IETF > Verifying Party : IESG > From nobody Wed May 13 04:53:04 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 16C7B1A1B66; Wed, 13 May 2015 04:53:01 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.91 X-Spam-Level: X-Spam-Status: No, score=-3.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, MIME_8BIT_HEADER=0.3, RCVD_IN_DNSWL_MED=-2.3, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6FYIbg2toW-0; Wed, 13 May 2015 04:52:58 -0700 (PDT) Received: from smtp.ee.ethz.ch (smtp.ee.ethz.ch [129.132.2.219]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id B6E641A1B65; Wed, 13 May 2015 04:52:57 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by smtp.ee.ethz.ch (Postfix) with ESMTP id 4A42AD930B; Wed, 13 May 2015 13:52:56 +0200 (MEST) X-Virus-Scanned: by amavisd-new on smtp.ee.ethz.ch Received: from smtp.ee.ethz.ch ([127.0.0.1]) by localhost (.ee.ethz.ch [127.0.0.1]) (amavisd-new, port 10024) with LMTP id SZRGWCCH6Dye; Wed, 13 May 2015 13:52:56 +0200 (MEST) Received: from [82.130.103.143] (nb-10510.ethz.ch [82.130.103.143]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: mirjak) by smtp.ee.ethz.ch (Postfix) with ESMTPSA id E09B9D930A; Wed, 13 May 2015 13:52:55 +0200 (MEST) Message-ID: <55533B17.8060705@tik.ee.ethz.ch> Date: Wed, 13 May 2015 13:52:55 +0200 From: =?windows-1252?Q?Mirja_K=FChlewind?= User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.6.0 MIME-Version: 1.0 To: ietf@ietf.org, draft-ietf-tcpm-newcwv@ietf.org References: <20150508093307.15938.17355.idtracker@ietfa.amsl.com> In-Reply-To: <20150508093307.15938.17355.idtracker@ietfa.amsl.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit Archived-At: Cc: tcpm@ietf.org Subject: Re: [tcpm] Last Call: (Updating TCP to support Rate-Limited Traffic) to Experimental RFC X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 May 2015 11:53:01 -0000 Hi authors, I've review this document (missed the WG last call though) and support publication of this document. Find some comments below that I recommend to further improve readability and may or may not be addressed in the final version. Mirja -------------------- pipeACK (section 4.1 and 4.2 and 4.5.1): 1) Can you maybe add a sentence that explains why FlightSize is not sufficient. I understand the difference between the two variables but are there actually cases where it makes a big difference if one or the other is used? 2) This is just an idea and does not make a big difference: Instead of saying pipeACk is undefined, you could just set it to infinity or 'max' (what every the max in the implementation is) and this would always keep you in validated phase. 3) The implementation for pipeACK (based on HighACK) that you propose does not take into account that dupACK also acknowledge new data. Is this on purpose? If so please state explicitly. 4) Text says "When no meaurements are available, ..." Maybe say more explicitly "When no meaurements are available as the connection is idle" or "... as no data were sent during the last measurement period" or "... no ACK were received...". 5) I would appreciate a forward reference to section 4.5.1 here (as also done for other stuff as for cwnd-limited in the next section). 6) Could you also include pseudo code in 4.5.1? I far as I know you have an implementation, so it should not be extremely hard to come up with pseudo code and I personally think it would be really helpful... 7) nit in last paragraph: s/method defined on [RFC6675]is/ method defined in [RFC6675] is/ 8) Figure 1 (section 4.5.1): Maybe add a y-axis...? non-validated phase (section 4.3 and 4.4) 1) Maybe add one sentence saying "A connection is also in non-validated phase if pipeACK is zero which means the connection is idle". Saying this explicit would help me to understand things better. I just notice that this might actually not be true because in the section above you say that pipeACK is set to undefined if no measurements are available and that would mean the connection goes into validated phase... okay I think there is some clarification needed here. 2) In section 4.4 you first say "cwnd neither grows nor reduces while the sender remains in this phase" and then "A sender that is cwnd-limited MAY use the standard TCP method to increase cwnd". Please clarify! 3) In section 4.4.1: "A sender that detects a packet-drop MUST record the current FlightSize in the variable LossFlightSize ..." Is this "packet-drop" or "congestion notification"? I guess the difference is that if ECN is used R=0. So that's okay. Please check again that this also works correctly for ECN. 4) Is this cwnd = (Max(pipeACK,LossFlightSize) - R)/2 or cwnd = (Max(pipeACK,(LossFlightSize - R))/2 [only LossFlightSize - R] ? Because the text reads as if it should be the second... 5) "ssthresh is adjusted using the standard TCP method." This is not clear to me. Is ssthread set to cwnd/2 before the adjustment or to cwnd-1 after the adjustment? The first option would restart the connection in Slow Start... Please clarify! 6) "then this can result in the final cwnd after loss recovery being 1/4 of the cwnd" Shouldn't this be "being at max 1/4 of the cwnd"...? 7) "Subsequent updates to cwnd do not therefore reflect pipeACK history before any congestion event." Not sure I understand this sentence... which subsequent updates? 8) nit: s/updated during loss recoverySection 4.2/ updated during loss recovery as stated in Section 4.2/ Non-validated phase (section 4.4.3 and 5) 1) I would recommend to change the title to "4.4.3 Adjustment at the end of the Non-Validated Phase (NVP)" 2) "An application that remains in the non-validated phase for a period greater than the NVP is required to adjust its congestion control state. If the sender exits the non-validated phase after this period, it MUST update the ssthresh:" This is not fully clear to me. Do I have to adapt cwnd and ssthresh right after NVP or as soon as I leave the non-validated phase after being for at leats NVP time in this phase? If I'm not idle but only sending with a low rate, I should probably do this adjustment with the next ACK, no...? 3) Further, these adjustments mean that the connection will be in Slow Start. If I understand this correctly, this is not a problem as long as the connection stays in non-validated as the cwnd will not be further increased. But as soon as it leaves non-validated phase it will increase its cwnd as in Slow Start. I would like to have this spelled out explicitly. 4) This section does not tell/specify what value NVP should have. Please add a sentence that this is on purpose and add a reference to section 5. 5) Section 5 says "A period of five minutes was chosen for this NVP." This seeems to be a left over from an older version as nothing was specified so far. Please check! On 08.05.2015 11:33, The IESG wrote: > > The IESG has received a request from the TCP Maintenance and Minor > Extensions WG (tcpm) to consider the following document: > - 'Updating TCP to support Rate-Limited Traffic' > as Experimental RFC > > The IESG plans to make a decision in the next few weeks, and solicits > final comments on this action. Please send substantive comments to the > ietf@ietf.org mailing lists by 2015-05-22. Exceptionally, comments may be > sent to iesg@ietf.org instead. In either case, please retain the > beginning of the Subject line to allow automated sorting. > > Abstract > > > This document provides a mechanism to address issues that arise when > TCP is used to support traffic that exhibits periods where the > sending rate is limited by the application rather than the congestion > window. It provides an experimental update to TCP that allows a TCP > sender to restart quickly following a rate-limited interval. This > method is expected to benefit applications that send rate-limited > traffic using TCP, while also providing an appropriate response if > congestion is experienced. > > It also evaluates the Experimental specification of TCP Congestion > Window Validation, CWV, defined in RFC 2861, and concludes that RFC > 2861 sought to address important issues, but failed to deliver a > widely used solution. This document therefore recommends that the > status of RFC 2861 is moved from Experimental to Historic, and that > it is replaced by the current specification. > > > > > The file can be obtained via > https://datatracker.ietf.org/doc/draft-ietf-tcpm-newcwv/ > > IESG discussion can be tracked via > https://datatracker.ietf.org/doc/draft-ietf-tcpm-newcwv/ballot/ > > > No IPR declarations have been submitted directly on this I-D. > -- ------------------------------------------ Dipl.-Ing. Mirja Kühlewind Communication Systems Group Institute TIK, ETH Zürich Gloriastrasse 35, 8092 Zürich, Switzerland Room ETZ G93 phone: +41 44 63 26932 email: mirja.kuehlewind@tik.ee.ethz.ch ------------------------------------------ From nobody Wed May 13 08:05:45 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4922E1B2CFD for ; Wed, 13 May 2015 08:05:43 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.361 X-Spam-Level: X-Spam-Status: No, score=-0.361 tagged_above=-999 required=5 tests=[BAYES_40=-0.001, HELO_EQ_FR=0.35, RCVD_IN_DNSWL_LOW=-0.7, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Q4Q1-BhSpANF for ; Wed, 13 May 2015 08:05:38 -0700 (PDT) Received: from ironport01.unice.fr (ironport01.unice.fr [134.59.1.129]) by ietfa.amsl.com (Postfix) with ESMTP id 103B41A87A4 for ; Wed, 13 May 2015 08:05:36 -0700 (PDT) From: Dino.LOPEZ@unice.fr Received: from naxos1.unice.fr ([134.59.1.116]) by ironport13.unice.fr with ESMTP; 13 May 2015 17:05:36 +0200 Received: from Dinos-MacBook-Pro.local ([172.19.130.15]) (authenticated bits=0) by naxos1.unice.fr (8.13.8/8.13.8) with ESMTP id t4DF5ZGB008411 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Wed, 13 May 2015 17:05:35 +0200 Message-ID: <55536840.6050101@unice.fr> Date: Wed, 13 May 2015 17:05:36 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.6.0 MIME-Version: 1.0 To: tcpm@ietf.org Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Archived-At: Subject: [tcpm] about synchronization losses in Cubic and draft-zimmermann-tcpm-cubic-01 X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 May 2015 15:05:43 -0000 Dear all, FYI and in case it is of interest for the "Discussion" section in draft-zimmermann-tcpm-cubic-01, we have published an article where we explain how and under which scenarios Cubic suffers from higher loss synchronization than standard TCP. Our observations suggest that both large RTTs and Fast Convergence mechanism in Cubic catalyze the synchronization losses. We also suggest that synchronization is unavoidable in high speed versions of TCP and AQM shall help mitigating the problem Synchronization has been observed in simulations, testbeds and in the Internet. Some experimental results are available in the article. The paper can be downloaded at http://www.i3s.unice.fr/~sassatelli/articles/ITC2014.pdf Thank you! -- Dino L., Lucile S. & Guillaume UK University Nice Sophia-Antipolis France From nobody Wed May 13 14:07:01 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2CBFB1A9078 for ; Wed, 13 May 2015 14:07:00 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.91 X-Spam-Level: X-Spam-Status: No, score=-6.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XYuJDzeBFON1 for ; Wed, 13 May 2015 14:06:57 -0700 (PDT) Received: from smtp-fr.alcatel-lucent.com (fr-hpida-esg-02.alcatel-lucent.com [135.245.210.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 595E41A904F for ; Wed, 13 May 2015 14:06:56 -0700 (PDT) Received: from fr712usmtp2.zeu.alcatel-lucent.com (unknown [135.239.2.42]) by Websense Email Security Gateway with ESMTPS id BFCFFE97DA91A; Wed, 13 May 2015 21:06:50 +0000 (GMT) Received: from FR711WXCHHUB01.zeu.alcatel-lucent.com (fr711wxchhub01.zeu.alcatel-lucent.com [135.239.2.111]) by fr712usmtp2.zeu.alcatel-lucent.com (GMO) with ESMTP id t4DL6ofE019810 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Wed, 13 May 2015 23:06:51 +0200 Received: from FR712WXCHMBA15.zeu.alcatel-lucent.com ([169.254.7.91]) by FR711WXCHHUB01.zeu.alcatel-lucent.com ([135.239.2.111]) with mapi id 14.03.0195.001; Wed, 13 May 2015 23:06:51 +0200 From: "Scharf, Michael (Michael)" To: Joe Touch , RFC Errata System , "mankin@psg.com" , "rbonica@juniper.net" , "spencerdawkins.ietf@gmail.com" , "mls.ietf@gmail.com" , "nishida@sfc.wide.ad.jp" , "pasi.sarolahti@iki.fi" Thread-Topic: [Technical Errata Reported] RFC5925 (4365) Thread-Index: AQHQjPTz26oTPS9GGUaJT3EHlHrSpp14tXIAgAGwwzs= Date: Wed, 13 May 2015 21:06:50 +0000 Message-ID: <655C07320163294895BBADA28372AF5D483A99D6@FR712WXCHMBA15.zeu.alcatel-lucent.com> References: <20150512204619.0B5CE180092@rfc-editor.org>, <55526CE4.70906@isi.edu> In-Reply-To: <55526CE4.70906@isi.edu> Accept-Language: de-DE, en-US Content-Language: de-DE X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.202.192.13] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Archived-At: Cc: "tcpm@ietf.org" Subject: Re: [tcpm] [Technical Errata Reported] RFC5925 (4365) X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 May 2015 21:07:00 -0000 Indeed, "Hold for Document Update" could be appropriate.=0A= =0A= According to [1] "Only errors that could cause implementation or deployment= problems or significant confusion should be Verified." and "Things that ar= e clearly wrong but could not cause an implementation or deployment problem= should be Hold for Document Update." To me this errata is more in the latt= er camp.=0A= =0A= Thanks for fixing this!=0A= =0A= Michael=0A= =0A= [1] http://www.ietf.org/iesg/statement/errata-processing.html=0A= From nobody Thu May 14 06:39:41 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7639B1B29CE for ; Thu, 14 May 2015 06:39:39 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 1.569 X-Spam-Level: * X-Spam-Status: No, score=1.569 tagged_above=-999 required=5 tests=[BAYES_50=0.8, SPF_NEUTRAL=0.779, T_RP_MATCHES_RCVD=-0.01] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7OWibt_xt67D for ; Thu, 14 May 2015 06:39:35 -0700 (PDT) Received: from lauren.room52.net (lauren.room52.net [210.50.193.198]) by ietfa.amsl.com (Postfix) with ESMTP id D54CA1AD0C0 for ; Thu, 14 May 2015 06:38:49 -0700 (PDT) Received: from lgwl-lstewart2.corp.netflix.com (c110-22-60-167.eburwd6.vic.optusnet.com.au [110.22.60.167]) by lauren.room52.net (Postfix) with ESMTPSA id 055D37E81E for ; Thu, 14 May 2015 23:38:46 +1000 (EST) Message-ID: <5554A53E.6090608@room52.net> Date: Thu, 14 May 2015 23:38:06 +1000 From: Lawrence Stewart User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: tcpm@ietf.org Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Archived-At: Subject: [tcpm] Window update handling in the BSD TCP stack X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 May 2015 13:39:39 -0000 Hi all, I've been contemplating the window update code in FreeBSD's TCP input path (see [1] for full context): > 2796 /* > 2797 * Update window information. > 2798 * Don't look at window if no ACK: TAC's send garbage on first SYN. > 2799 */ > 2800 if ((thflags & TH_ACK) && > 2801 (SEQ_LT(tp->snd_wl1, th->th_seq) || > 2802 (tp->snd_wl1 == th->th_seq && (SEQ_LT(tp->snd_wl2, th->th_ack) || > 2803 (tp->snd_wl2 == th->th_ack && tiwin > tp->snd_wnd))))) { > 2804 /* keep track of pure window updates */ > 2805 if (tlen == 0 && > 2806 tp->snd_wl2 == th->th_ack && tiwin > tp->snd_wnd) > 2807 TCPSTAT_INC(tcps_rcvwinupd); > 2808 tp->snd_wnd = tiwin; > 2809 tp->snd_wl1 = th->th_seq; > 2810 tp->snd_wl2 = th->th_ack; > 2811 if (tp->snd_wnd > tp->max_sndwnd) > 2812 tp->max_sndwnd = tp->snd_wnd; > 2813 needoutput = 1; > 2814 } The code is documented essentially as above in Stevens Vol II and reflects RFC 793 pg 72 - it has been around a long time. I have some concerns that the logic is too restrictive in some situations, and would like to propose the following for discussion: > /* > * Update window information if it's an ACK (TAC's send garbage on > * first SYN) and: > * - The peer has ACKed (snd_wl2 < th_ack) or sent (snd_wl1 < th_seq) > * new data OR > * - A pure window update (snd_wl1 == th_ack && snd_wl2 == th_seq && > * tiwn != snd_wnd) with a new timestamp or if not doing timestamps, > * we special case zero window handling to ensure we'll enter persist. > */ > if ((thflags & TH_ACK) && > (SEQ_LT(tp->snd_wl1, th->th_seq) || > SEQ_LT(tp->snd_wl2, th->th_ack) || > (tp->snd_wl1 == th->th_seq && tp->snd_wl2 == th->th_ack && > tiwin != tp->snd_wnd && > (tsrecentupdated || (!tiwin && !(tp->t_flags & TF_RCVD_TSTMP)))))) { > /* keep track of pure window updates */ > if (tlen == 0 && tp->snd_wl2 == th->th_ack && > tiwin != tp->snd_wnd) > TCPSTAT_INC(tcps_rcvwinupd); > tp->snd_wnd = tiwin; > tp->snd_wl1 = th->th_seq; > tp->snd_wl2 = th->th_ack; > if (tp->snd_wnd > tp->max_sndwnd) > tp->max_sndwnd = tp->snd_wnd; > needoutput = 1; > } The tsrecentupdated variable is new, and I default it to 0 but update to 1 anywhere in tcp_do_segment() where tp->ts_recent is set to the incoming segment's tsval. The main motivation for the change is correct 0 window handling (the details of which I'll be vague about as this investigation is related to a DoS vector I'm working on a fix for). In the spirit of being liberal with what you accept, why shouldn't we accept pure window updates that shrink the window if we think they reflect the most up to date information from the peer (or in the case of no timestamps, reflect information we could act on to be safe)? Cheers, Lawrence [1] https://svnweb.freebsd.org/base/head/sys/netinet/tcp_input.c?view=markup#l2796 From nobody Fri May 15 05:33:40 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 05C511A1B21 for ; Fri, 15 May 2015 05:33:39 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.089 X-Spam-Level: X-Spam-Status: No, score=0.089 tagged_above=-999 required=5 tests=[BAYES_50=0.8, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lJb7TEgpHVBx for ; Fri, 15 May 2015 05:33:37 -0700 (PDT) Received: from hubrelay-by-04.bt.com (hubrelay-by-04.bt.com [62.7.242.140]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 13A641A1B00 for ; Fri, 15 May 2015 05:33:36 -0700 (PDT) Received: from EVMHR71-UKRD.domain1.systemhost.net (10.36.3.109) by EVMHR04-UKBR.bt.com (10.216.161.36) with Microsoft SMTP Server (TLS) id 8.3.348.2; Fri, 15 May 2015 13:33:31 +0100 Received: from EPHR01-UKIP.domain1.systemhost.net (147.149.196.177) by EVMHR71-UKRD.domain1.systemhost.net (10.36.3.109) with Microsoft SMTP Server (TLS) id 8.3.348.2; Fri, 15 May 2015 13:33:33 +0100 Received: from bagheera.jungle.bt.co.uk (132.146.168.158) by EPHR01-UKIP.domain1.systemhost.net (147.149.196.177) with Microsoft SMTP Server id 14.3.181.6; Fri, 15 May 2015 13:33:30 +0100 Received: from BTP075694.jungle.bt.co.uk ([10.109.52.84]) by bagheera.jungle.bt.co.uk (8.13.5/8.12.8) with ESMTP id t4FCXRxW022847 for ; Fri, 15 May 2015 13:33:27 +0100 Message-ID: <201505151233.t4FCXRxW022847@bagheera.jungle.bt.co.uk> X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Fri, 15 May 2015 13:33:28 +0100 To: tcpm IETF list From: Bob Briscoe MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-Scanned-By: MIMEDefang 2.56 on 132.146.168.158 Archived-At: Subject: [tcpm] Why is the minimum ssthresh 2*SMSS (rather than 1)? X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 May 2015 12:33:39 -0000 tcpm folks, Despite searching and asking around, I cannot find any definitive reasoning for the choice of 2*SMSS in eqn (4) in RFC5681. ssthresh = max (FlightSize / 2, 2*SMSS) (4) In testing a new AQM with multiple flows, now we've got the RTT really low, we've found that TCP's minimum window of 2 makes TCP ignore the AQM's signals ('cos it always rounds up to 2) so all the TCP flows drive up the queueing delay to give themselves a large-enough RTT - defeating the AQM. Bob ________________________________________________________________ Bob Briscoe, BT From nobody Fri May 15 10:12:41 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0977B1A8770 for ; Fri, 15 May 2015 10:12:40 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2 X-Spam-Level: X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id cN0I1V2YZIR4 for ; Fri, 15 May 2015 10:12:38 -0700 (PDT) Received: from mail-la0-x235.google.com (mail-la0-x235.google.com [IPv6:2a00:1450:4010:c03::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 87AE41A8768 for ; Fri, 15 May 2015 10:12:38 -0700 (PDT) Received: by lagr1 with SMTP id r1so47896740lag.0 for ; Fri, 15 May 2015 10:12:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=SHTYgby/XqwhyFGn2jKvefWNRV/zSmXslUP7w5Fgtk0=; b=KV/VH6pWs4BezQP0NaKGM9LPABVFDc/ez6viS0K5tM4EI132+ML0Vv9zWhW1W3EAzr f1JSU78+sbd9q1nqDdpy72iIbImQqD9o3NRRFy5et31lL2i7uKOmowcSOqZJ4hu4FRqu Pix2llouiaNSI5UZ4b2WMnCoxTWKJG1NUQMVb+J5WQJY0wtpyxXG4JnV8dA7vlzqNJbX ND+Do01HylEZDPaswnk4wEWm/nmWFam71+eYggd1cddBePllhaHhxMm24o2CskYcXNAP lzh9trMSVDAaAraW3Xl3iOLsMOXzRGSQQQg9UFiWWQyyz3rl/rGKbh2U0i+EjfW2odff 90KQ== MIME-Version: 1.0 X-Received: by 10.112.147.201 with SMTP id tm9mr7952478lbb.40.1431709956868; Fri, 15 May 2015 10:12:36 -0700 (PDT) Received: by 10.25.12.7 with HTTP; Fri, 15 May 2015 10:12:36 -0700 (PDT) In-Reply-To: <201505151233.t4FCXRxW022847@bagheera.jungle.bt.co.uk> References: <201505151233.t4FCXRxW022847@bagheera.jungle.bt.co.uk> Date: Fri, 15 May 2015 13:12:36 -0400 Message-ID: From: John Heffner To: Bob Briscoe Content-Type: text/plain; charset=UTF-8 Archived-At: Cc: tcpm IETF list Subject: Re: [tcpm] Why is the minimum ssthresh 2*SMSS (rather than 1)? X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 May 2015 17:12:40 -0000 On Fri, May 15, 2015 at 8:33 AM, Bob Briscoe wrote: > tcpm folks, > > Despite searching and asking around, I cannot find any definitive reasoning > for the choice of 2*SMSS in eqn (4) in RFC5681. > > ssthresh = max (FlightSize / 2, 2*SMSS) (4) > > In testing a new AQM with multiple flows, now we've got the RTT really low, > we've found that TCP's minimum window of 2 makes TCP ignore the AQM's > signals ('cos it always rounds up to 2) so all the TCP flows drive up the > queueing delay to give themselves a large-enough RTT - defeating the AQM. Delayed ack? From nobody Fri May 15 14:53:39 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 58A461A87BF for ; Fri, 15 May 2015 14:53:38 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.389 X-Spam-Level: X-Spam-Status: No, score=-1.389 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FM_FORGED_GMAIL=0.622, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XAISc3Lz5apP for ; Fri, 15 May 2015 14:53:37 -0700 (PDT) Received: from mail-oi0-x22e.google.com (mail-oi0-x22e.google.com [IPv6:2607:f8b0:4003:c06::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 251F71A878E for ; Fri, 15 May 2015 14:53:37 -0700 (PDT) Received: by oign205 with SMTP id n205so92676889oig.2 for ; Fri, 15 May 2015 14:53:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=tGeGTez7QsKliTDkgIb0XGsLXoHE/K7WclP7m4l+P4U=; b=eUrU0bQpGSvOdh2Zbwlkz+c7MMtuxF2wG0FeolQCsh2f7CdbPKxwFUEpdWQoULkaCJ YDC+vWm04959QPdSQhmDVpawdpSiABVhossu0qAosIKgzdKE0oKyWz4JXstG0Kedz4wJ 5o2NiNRmZth2zR4jDUn/kdBknjxjwNnjrYRt3zdtJ6QpqwDodUozgMY9u6eA70a53St9 6x6mhMI/JqUnTumsVc43bbA+I6Dbb9UuHJptbsYHVuF3QtWvRxEPhicrUOxpF2nZXfkP by3jvw75SHrEjSnbdBj0FGZQFOWRupSrdXZlC2qUsUKAMLbSfZA/r5VRgWO4rlIu/tDG KaeQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type; bh=tGeGTez7QsKliTDkgIb0XGsLXoHE/K7WclP7m4l+P4U=; b=Jw8VOxb+CB2a4iPpO9te1pWokYIeYYFY5aq2mYGw/yXd2905u6/7aSgkNIConrKURT lhfHvno7ZCVnb5P5VtrMYBeAXDYybeD0PbnE7WPXiB6eMJwJO+bv5r5xNIkY290FYNdo he1KmZW3rXqRKe+A9zvjTi+3gwEffqgb0MYJK80FbERKRjfa/tna7fO4nDmxFd22TKb7 O6FArFRlEcCDSPrHL+y19mmnMrwNeJoFX29i8nKnr/Ek9mqG/4cLlNcCXHza/y3EOSLh epuXadce81RVpXS/7hGeY31MY/ugl7v/F0Mw35MOlZl5EDEd3/6wwwVL/QXQNs3uD7u5 vABA== X-Gm-Message-State: ALoCoQl5Brvu/r3CHsZf+Hfoc3OVgAx+2wkaIpMAXM7lBCD4iz8mkIijSuXveqeEtAKE0G9MvuED X-Received: by 10.60.128.130 with SMTP id no2mr2969785oeb.70.1431726816535; Fri, 15 May 2015 14:53:36 -0700 (PDT) MIME-Version: 1.0 Received: by 10.182.230.230 with HTTP; Fri, 15 May 2015 14:52:56 -0700 (PDT) In-Reply-To: <201505151233.t4FCXRxW022847@bagheera.jungle.bt.co.uk> References: <201505151233.t4FCXRxW022847@bagheera.jungle.bt.co.uk> From: Yuchung Cheng Date: Fri, 15 May 2015 14:52:56 -0700 Message-ID: To: Bob Briscoe Content-Type: text/plain; charset=UTF-8 Archived-At: Cc: tcpm IETF list Subject: Re: [tcpm] Why is the minimum ssthresh 2*SMSS (rather than 1)? X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 May 2015 21:53:38 -0000 On Fri, May 15, 2015 at 5:33 AM, Bob Briscoe wrote: > tcpm folks, > > Despite searching and asking around, I cannot find any definitive reasoning > for the choice of 2*SMSS in eqn (4) in RFC5681. > > ssthresh = max (FlightSize / 2, 2*SMSS) (4) > > In testing a new AQM with multiple flows, now we've got the RTT really low, > we've found that TCP's minimum window of 2 makes TCP ignore the AQM's I am confused. min ssthresh != min-cwnd == 1? > signals ('cos it always rounds up to 2) so all the TCP flows drive up the > queueing delay to give themselves a large-enough RTT - defeating the AQM. > > > > Bob > > > > ________________________________________________________________ > Bob Briscoe, BT > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm From nobody Fri May 15 15:15:05 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CEC781A88A3 for ; Fri, 15 May 2015 15:15:04 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.013 X-Spam-Level: X-Spam-Status: No, score=-2.013 tagged_above=-999 required=5 tests=[BAYES_20=-0.001, MIME_8BIT_HEADER=0.3, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ZjkBXwoAJxHk for ; Fri, 15 May 2015 15:15:02 -0700 (PDT) Received: from script.cs.helsinki.fi (script.cs.helsinki.fi [128.214.11.1]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 307F51A8885 for ; Fri, 15 May 2015 15:15:01 -0700 (PDT) Received: from melkinpaasi.cs.helsinki.fi (melkinpaasi.cs.helsinki.fi [128.214.9.14]) (TLS: TLSv1/SSLv3,256bits,AES256-SHA) by mail.cs.helsinki.fi with ESMTPS; Sat, 16 May 2015 01:14:59 +0300 id 00000000005A003D.0000000055566FE3.000018BB Date: Sat, 16 May 2015 01:14:59 +0300 (EEST) From: =?ISO-8859-15?Q?Ilpo_J=E4rvinen?= X-X-Sender: ijjarvin@melkinpaasi.cs.helsinki.fi To: Bob Briscoe In-Reply-To: <201505151233.t4FCXRxW022847@bagheera.jungle.bt.co.uk> Message-ID: References: <201505151233.t4FCXRxW022847@bagheera.jungle.bt.co.uk> User-Agent: Alpine 2.02 (DEB 1266 2009-07-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-CS-Test-DKIM: none Archived-At: Cc: tcpm IETF list Subject: Re: [tcpm] Why is the minimum ssthresh 2*SMSS (rather than 1)? X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 May 2015 22:15:05 -0000 On Fri, 15 May 2015, Bob Briscoe wrote: > Despite searching and asking around, I cannot find any definitive reasoning > for the choice of 2*SMSS in eqn (4) in RFC5681. > > ssthresh = max (FlightSize / 2, 2*SMSS) (4) > > In testing a new AQM with multiple flows, now we've got the RTT really low, > we've found that TCP's minimum window of 2 makes TCP ignore the AQM's signals > ('cos it always rounds up to 2) so all the TCP flows drive up the queueing > delay to give themselves a large-enough RTT - defeating the AQM. Does this really matter that much whether ssthresh would be 1 or 2 SMSS? Both congestion avoidance and slow start are equally aggressive for the first RTT: when TCP start with cwnd=1 and the sender gets the ACK, the applied increase is cwnd = cwnd + 1 SMSS regardless of the operating mode. However, ssthresh == cwnd case is left open so there might be a difference depending on implementation: "When cwnd and ssthresh are equal, the sender may use either slow start or congestion avoidance." ...but I don't see how that would relate to "minimum window of 2" you're talking here (and ssthresh is not really "window" anyway). -- i. From nobody Sat May 16 04:52:24 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 493601B2BD7 for ; Sat, 16 May 2015 04:52:23 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.188 X-Spam-Level: X-Spam-Status: No, score=0.188 tagged_above=-999 required=5 tests=[BAYES_40=-0.001, J_CHICKENPOX_32=0.6, MIME_8BIT_HEADER=0.3, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XkvUFjNpyLgW for ; Sat, 16 May 2015 04:52:21 -0700 (PDT) Received: from hubrelay-rd.bt.com (hubrelay-rd.bt.com [62.239.224.98]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 053401A0194 for ; Sat, 16 May 2015 04:52:20 -0700 (PDT) Received: from EVMHR02-UKBR.domain1.systemhost.net (193.113.108.41) by EVMHR65-UKRD.bt.com (10.187.101.20) with Microsoft SMTP Server (TLS) id 14.3.181.6; Sat, 16 May 2015 12:52:19 +0100 Received: from EPHR01-UKIP.domain1.systemhost.net (147.149.196.177) by EVMHR02-UKBR.domain1.systemhost.net (193.113.108.41) with Microsoft SMTP Server (TLS) id 8.3.348.2; Sat, 16 May 2015 12:52:18 +0100 Received: from bagheera.jungle.bt.co.uk (132.146.168.158) by EPHR01-UKIP.domain1.systemhost.net (147.149.196.177) with Microsoft SMTP Server id 14.3.181.6; Sat, 16 May 2015 12:52:15 +0100 Received: from BTP075694.jungle.bt.co.uk ([10.111.112.211]) by bagheera.jungle.bt.co.uk (8.13.5/8.12.8) with ESMTP id t4GBqCmx001121; Sat, 16 May 2015 12:52:12 +0100 Message-ID: <201505161152.t4GBqCmx001121@bagheera.jungle.bt.co.uk> X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Sat, 16 May 2015 12:52:12 +0100 To: Ilpo =?iso-8859-1?Q?J=E4rvinen?= , "Yuchung Cheng" , John Heffner From: Bob Briscoe In-Reply-To: References: <201505151233.t4FCXRxW022847@bagheera.jungle.bt.co.uk> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1"; format=flowed Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.56 on 132.146.168.158 Archived-At: Cc: tcpm IETF list Subject: Re: [tcpm] Why is the minimum ssthresh 2*SMSS (rather than 1)? X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 16 May 2015 11:52:23 -0000 Ilpo & Yuchung, Sorry, the phrasing of my question was too curt. I'm not specifically interested in why ssthresh=20 has a min. I only phrased my question that way=20 because the way TCP fast recovery is specified=20 and implemented sets a min for cwnd based on the min for ssthresh. From RFC5681: 2. When the third duplicate ACK is received, a TCP MUST set ssthresh to no more than the value given in equation (4). ... 6. When the next ACK arrives that acknowledges previously unacknowledged data, a TCP MUST set cwnd to ssthresh (the value set in step 2). This is termed "deflating" the window. The only guess I've heard for why it says 2*SMSS=20 in eqn (4) is delayed ACKs (John Heffner, as well=20 as Michael Welzl when I was asking around before=20 I posted this to tcpm). Certainly delayed ACKs=20 will cause segments to bunch in twos even if the=20 window were less than two. However, if the RTT=20 was short enough (or the link congested enough)=20 to need a low window, at least segments could=20 bunch in twos at a lower rate than two per RTT. I believe TCP would not work against AQMs at low=20 RTTs if the min for ssthresh in Eqn (4) was zero,=20 not 2*SMSS. But there must have been a reason for=20 this minimum at some point in the past - that's what I'm trying to find out. If there is no strong reason to keep the 2*SMSS=20 minimum, the rule after a timeout would have to=20 be changed too. Specifically, the two changes needed to RFC5681 would be: CURRENT: ssthresh =3D max (FlightSize / 2, 2*SMSS) (4) PROPOSED: ssthresh =3D FlightSize / 2 (4) CURRENT upon a timeout [...] cwnd MUST be set to no more than the loss window, LW, which equals 1 full-sized segment. PROPOSED: upon a timeout [...] cwnd MUST be set to cwnd =3D min(LW, ssthresh) where the loss window, LW, which equals 1 full-sized segment. Bob At 23:14 15/05/2015, Ilpo J=E4rvinen wrote: >On Fri, 15 May 2015, Bob Briscoe wrote: > > > Despite searching and asking around, I cannot find any definitive= reasoning > > for the choice of 2*SMSS in eqn (4) in RFC5681. > > > > ssthresh =3D max (FlightSize / 2, 2*SMSS) (4) > > > > In testing a new AQM with multiple flows, now we've got the RTT really= low, > > we've found that TCP's minimum window of 2=20 > makes TCP ignore the AQM's signals > > ('cos it always rounds up to 2) so all the TCP flows drive up the= queueing > > delay to give themselves a large-enough RTT - defeating the AQM. > >Does this really matter that much whether ssthresh would be 1 or 2 SMSS? >Both congestion avoidance and slow start are equally aggressive for the >first RTT: when TCP start with cwnd=3D1 and the sender gets the ACK, the >applied increase is cwnd =3D cwnd + 1 SMSS regardless of the operating >mode. > >However, ssthresh =3D=3D cwnd case is left open so there might be a >difference depending on implementation: > "When cwnd and ssthresh are equal, the sender may use either > slow start or congestion avoidance." >...but I don't see how that would relate to "minimum window of 2" you're >talking here (and ssthresh is not really "window" anyway). > > >-- > i. At 22:52 15/05/2015, Yuchung Cheng wrote: >I am confused. min ssthresh !=3D min-cwnd =3D=3D 1? ________________________________________________________________ Bob Briscoe, BT=20 From nobody Sun May 17 08:30:33 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D3D8D1AD0CE for ; Tue, 12 May 2015 13:48:03 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -101.912 X-Spam-Level: X-Spam-Status: No, score=-101.912 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01, USER_IN_WHITELIST=-100] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id uan40gPrgkR4 for ; Tue, 12 May 2015 13:48:02 -0700 (PDT) Received: from rfc-editor.org (rfc-editor.org [IPv6:2001:1900:3001:11::31]) by ietfa.amsl.com (Postfix) with ESMTP id 58AA91AD0CA for ; Tue, 12 May 2015 13:48:02 -0700 (PDT) Received: by rfc-editor.org (Postfix, from userid 30) id 0B5CE180092; Tue, 12 May 2015 13:46:19 -0700 (PDT) To: touch@isi.edu, mankin@psg.com, rbonica@juniper.net, spencerdawkins.ietf@gmail.com, mls.ietf@gmail.com, michael.scharf@alcatel-lucent.com, nishida@sfc.wide.ad.jp, pasi.sarolahti@iki.fi X-PHP-Originating-Script: 6000:errata_mail_lib.php From: RFC Errata System Message-Id: <20150512204619.0B5CE180092@rfc-editor.org> Date: Tue, 12 May 2015 13:46:19 -0700 (PDT) Archived-At: X-Mailman-Approved-At: Sun, 17 May 2015 08:30:31 -0700 Cc: rfc-editor@rfc-editor.org, tcpm@ietf.org, touch@isi.edu Subject: [tcpm] [Technical Errata Reported] RFC5925 (4365) X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 May 2015 20:48:04 -0000 The following errata report has been submitted for RFC5925, "The TCP Authentication Option". -------------------------------------- You may review the report below and at: http://www.rfc-editor.org/errata_search.php?rfc=5925&eid=4365 -------------------------------------- Type: Technical Reported by: Joe Touch Section: 7.6 Original Text ------------- TCP's 4-bit data offset requires that the options end 60 bytes (15 32-bit words) after the header begins, including the 20-byte header. This leaves 40 bytes for options, of which 15 are expected in current implementations (listed below), leaving at most 25 for other uses. TCP-AO consumes 16 bytes, leaving 9 bytes for additional SYN options (depending on implementation dependant alignment padding, which could consume another 2 bytes at most). o SACK permitted (2 bytes) [RFC2018][RFC3517] o Timestamps (10 bytes) [RFC1323] o Window scale (3 bytes) [RFC1323] Corrected Text -------------- TCP's 4-bit data offset requires that the options end 60 bytes (15 32-bit words) after the header begins, including the 20-byte header. This leaves 40 bytes for options, of which 19 are expected in current implementations (listed below), leaving at most 21 for other uses. TCP-AO consumes 16 bytes, leaving 5 bytes for additional SYN options (depending on implementation dependent alignment padding, which could consume another 2 bytes at most). o SACK permitted (2 bytes) [RFC2018][RFC3517] o Timestamps (10 bytes) [RFC1323] o Window scale (3 bytes) [RFC1323] o Maximum Segment Size (4 bytes) [RFC793] Notes ----- MSS was missing in the original text. New text includes MSS and updates numbers accordingly. Also corrects a spelling error (dependant -> dependent), which is non-technical but included in the revised text. Instructions: ------------- This erratum is currently posted as "Reported". If necessary, please use "Reply All" to discuss whether it should be verified or rejected. When a decision is reached, the verifying party (IESG) can log in to change the status and edit the report, if necessary. -------------------------------------- RFC5925 (draft-ietf-tcpm-tcp-auth-opt-11) -------------------------------------- Title : The TCP Authentication Option Publication Date : June 2010 Author(s) : J. Touch, A. Mankin, R. Bonica Category : PROPOSED STANDARD Source : TCP Maintenance and Minor Extensions Area : Transport Stream : IETF Verifying Party : IESG From nobody Mon May 18 01:56:06 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3FB5A1A8830 for ; Mon, 18 May 2015 01:56:05 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.21 X-Spam-Level: X-Spam-Status: No, score=-4.21 tagged_above=-999 required=5 tests=[BAYES_50=0.8, RCVD_IN_DNSWL_HI=-5, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id I-uHPZZE1y2Q for ; Mon, 18 May 2015 01:56:03 -0700 (PDT) Received: from smtp-fr.alcatel-lucent.com (fr-hpgre-esg-01.alcatel-lucent.com [135.245.210.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id BE7311A007E for ; Mon, 18 May 2015 01:56:03 -0700 (PDT) Received: from fr711usmtp1.zeu.alcatel-lucent.com (unknown [135.239.2.122]) by Websense Email Security Gateway with ESMTPS id 9178868D60F58; Mon, 18 May 2015 08:56:00 +0000 (GMT) Received: from FR712WXCHHUB03.zeu.alcatel-lucent.com (fr712wxchhub03.zeu.alcatel-lucent.com [135.239.2.74]) by fr711usmtp1.zeu.alcatel-lucent.com (GMO) with ESMTP id t4I8u0SQ006865 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Mon, 18 May 2015 10:56:01 +0200 Received: from FR712WXCHMBA15.zeu.alcatel-lucent.com ([169.254.7.91]) by FR712WXCHHUB03.zeu.alcatel-lucent.com ([135.239.2.74]) with mapi id 14.03.0195.001; Mon, 18 May 2015 10:55:44 +0200 From: "Scharf, Michael (Michael)" To: "tcpm@ietf.org" Thread-Topic: WG acceptance of draft-zimmermann-tcpm-cubic Thread-Index: AdCRSGwSZZw2GKBJTrSKyf4Ew/gqfw== Date: Mon, 18 May 2015 08:55:43 +0000 Message-ID: <655C07320163294895BBADA28372AF5D483AEFB6@FR712WXCHMBA15.zeu.alcatel-lucent.com> Accept-Language: de-DE, en-US Content-Language: de-DE X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.239.27.41] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Archived-At: Cc: "draft-zimmermann-tcpm-cubic@tools.ietf.org" Subject: [tcpm] WG acceptance of draft-zimmermann-tcpm-cubic X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 May 2015 08:56:05 -0000 Hi all, This e-mail conforms the TCPM WG acceptance of draft-zimmermann-tcpm-cubic-= 01. Please resubmit as draft-ietf-tcpm-cubic-00. According to the list feedback, the TCPM consensus is that this document sh= ould head for a status as informational RFC. The chairs believe that it would be very valuable to have an RFC very soon,= in order to describe what is widely deployed in the Internet. We observe t= hat there is also support for a PS document in this space. However, a PS do= cument would possibly take longer to complete. Therefore, we think we can d= iscuss a PS document once draft-ietf-tcpm-cubic is mature. Thanks Michael, Pasi, Yoshifumi From nobody Mon May 18 02:12:43 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 379021A8845 for ; Mon, 18 May 2015 02:12:42 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.91 X-Spam-Level: X-Spam-Status: No, score=-6.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id OO49ADuIMWvu for ; Mon, 18 May 2015 02:12:40 -0700 (PDT) Received: from smtp-fr.alcatel-lucent.com (fr-hpida-esg-02.alcatel-lucent.com [135.245.210.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id BDA0A1A884E for ; Mon, 18 May 2015 02:12:36 -0700 (PDT) Received: from fr712usmtp2.zeu.alcatel-lucent.com (unknown [135.239.2.42]) by Websense Email Security Gateway with ESMTPS id 8933371C22CDA for ; Mon, 18 May 2015 09:12:33 +0000 (GMT) Received: from FR711WXCHHUB02.zeu.alcatel-lucent.com (fr711wxchhub02.zeu.alcatel-lucent.com [135.239.2.112]) by fr712usmtp2.zeu.alcatel-lucent.com (GMO) with ESMTP id t4I9CTNV013872 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL) for ; Mon, 18 May 2015 11:12:34 +0200 Received: from FR712WXCHMBA15.zeu.alcatel-lucent.com ([169.254.7.91]) by FR711WXCHHUB02.zeu.alcatel-lucent.com ([135.239.2.112]) with mapi id 14.03.0195.001; Mon, 18 May 2015 11:12:29 +0200 From: "Scharf, Michael (Michael)" To: "tcpm@ietf.org" Thread-Topic: Feedback on WG acceptance of draft-bensley-tcpm-dctcp-03 Thread-Index: AdB82dvas/DQQ7w2TuSWSxKiXj5XLgUbrOuQ Date: Mon, 18 May 2015 09:12:28 +0000 Message-ID: <655C07320163294895BBADA28372AF5D483AF007@FR712WXCHMBA15.zeu.alcatel-lucent.com> References: <655C07320163294895BBADA28372AF5D16C939E2@FR712WXCHMBA15.zeu.alcatel-lucent.com> In-Reply-To: <655C07320163294895BBADA28372AF5D16C939E2@FR712WXCHMBA15.zeu.alcatel-lucent.com> Accept-Language: de-DE, en-US Content-Language: de-DE X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.239.27.41] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Archived-At: Subject: Re: [tcpm] Feedback on WG acceptance of draft-bensley-tcpm-dctcp-03 X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 May 2015 09:12:42 -0000 Hi all, There have not been many responses to this e-mail. The chairs therefore con= clude that there is not enough support for adoption of draft-bensley-tcpm-d= ctcp-03 as TCPM WG item as of now. Since the TCPM community seems interested in further discussing this topic,= the chairs would offer to allocate time for corresponding discussions duri= ng upcoming TCPM meetings. Thanks Michael, Pasi, Yoshifumi PS: We recently found one possibly interesting study on operational experie= nce: https://www.usenix.org/system/files/conference/nsdi15/nsdi15-paper-jud= d.pdf > -----Original Message----- > From: tcpm [mailto:tcpm-bounces@ietf.org] On Behalf Of Scharf, Michael > (Michael) > Sent: Wednesday, April 22, 2015 10:54 AM > To: tcpm@ietf.org > Subject: [tcpm] Feedback on WG acceptance of draft-bensley-tcpm-dctcp- > 03 >=20 > Dear all, >=20 > During IETF 89 [1] and on the list there has been a discussion about WG > adoption of draft-bensley-tcpm-dctcp [2]. The understanding of the > chairs is that there is no clear consensus up to now. >=20 > In order to move forward, the chairs seek for community guidance > regarding the following options to handle this draft: >=20 > a) Adopt in TCPM as Informational >=20 > b) Adopt in TCPM as Experimental >=20 > c) Do not adopt in TCPM now >=20 > Option c) does not prevent publication outside TCPM, such as > publication by another working group (e.g., TSVWG), publication on > Independent Stream, etc. >=20 > The TCPM chairs believe that adoption of alternative congestion control > algorithms in TCPM should be backed by a strong consensus. Please also > review [3] and [4] regarding the difference between Experimental and > Informational. >=20 > Please let us know any feedback on a potential WG acceptance of draft- > bensley-tcpm-dctcp-03 until May 8. >=20 > Thanks! >=20 > Michael, Pasi, Yoshifumi >=20 >=20 > [1] http://www.ietf.org/proceedings/89/minutes/minutes-89-tcpm >=20 > [2] https://tools.ietf.org/html/draft-bensley-tcpm-dctcp-03 >=20 > [3] RFC 2026, Section 4.2.1, 4.2.2 >=20 > [4] https://www.ietf.org/iesg/informational-vs-experimental.html >=20 > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm From nobody Mon May 18 02:22:04 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DD3181A883D for ; Mon, 18 May 2015 02:22:01 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -5.51 X-Spam-Level: X-Spam-Status: No, score=-5.51 tagged_above=-999 required=5 tests=[BAYES_05=-0.5, RCVD_IN_DNSWL_HI=-5, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ruIBsmQiQPxx for ; Mon, 18 May 2015 02:21:59 -0700 (PDT) Received: from smtp-fr.alcatel-lucent.com (fr-hpgre-esg-01.alcatel-lucent.com [135.245.210.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id B4EC81A6F38 for ; Mon, 18 May 2015 02:21:59 -0700 (PDT) Received: from fr711usmtp1.zeu.alcatel-lucent.com (unknown [135.239.2.122]) by Websense Email Security Gateway with ESMTPS id 319ABB58B5A35; Mon, 18 May 2015 09:21:56 +0000 (GMT) Received: from FR711WXCHHUB02.zeu.alcatel-lucent.com (fr711wxchhub02.zeu.alcatel-lucent.com [135.239.2.112]) by fr711usmtp1.zeu.alcatel-lucent.com (GMO) with ESMTP id t4I9Lthq008016 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Mon, 18 May 2015 11:21:55 +0200 Received: from FR712WXCHMBA15.zeu.alcatel-lucent.com ([169.254.7.91]) by FR711WXCHHUB02.zeu.alcatel-lucent.com ([135.239.2.112]) with mapi id 14.03.0195.001; Mon, 18 May 2015 11:21:55 +0200 From: "Scharf, Michael (Michael)" To: "per.hurtig@kau.se" , "anna.brunstrom@kau.se" , "apetlund@simula.no" , "michawe@ifi.uio.no" Thread-Topic: [tcpm] draft-ietf-tcpm-rtorestart-07 Thread-Index: AQHQiNZ6HA7RERz4JUq8Z2geCdc8op2BhR3Q Date: Mon, 18 May 2015 09:21:54 +0000 Message-ID: <655C07320163294895BBADA28372AF5D483AF02D@FR712WXCHMBA15.zeu.alcatel-lucent.com> References: <882B8BE8-D02D-4E46-90B0-FE9CB2E88BF9@telecom-bretagne.eu> In-Reply-To: <882B8BE8-D02D-4E46-90B0-FE9CB2E88BF9@telecom-bretagne.eu> Accept-Language: de-DE, en-US Content-Language: de-DE X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.239.27.41] Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 MIME-Version: 1.0 Archived-At: Cc: "tcpm@ietf.org" Subject: Re: [tcpm] draft-ietf-tcpm-rtorestart-07 X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 May 2015 09:22:02 -0000 QXV0aG9ycywNCg0KVGhlc2UgKG1vc3RseSBlZGl0b3JpYWwpIGNvbW1lbnRzIGFyZSB0aGUgb25s eSBXR0xDIGNvbW1lbnRzIGZvciBkcmFmdC1pZXRmLXRjcG0tcnRvcmVzdGFydC0wNyB0aGF0IEkg YW0gYXdhcmUgb2YuIENvdWxkIHlvdSBwbGVhc2UgZGlzY3VzcyBob3cgdG8gYWRkcmVzcyB0aGVt Pw0KDQpUaGFua3MNCg0KTWljaGFlbA0KDQoNCg0KPiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0t LQ0KPiBGcm9tOiB0Y3BtIFttYWlsdG86dGNwbS1ib3VuY2VzQGlldGYub3JnXSBPbiBCZWhhbGYg T2YgTmljb2xhcyBLdWhuDQo+IFNlbnQ6IFRodXJzZGF5LCBNYXkgMDcsIDIwMTUgNTowMCBQTQ0K PiBUbzogdGNwbUBpZXRmLm9yZw0KPiBTdWJqZWN0OiBbdGNwbV0gZHJhZnQtaWV0Zi10Y3BtLXJ0 b3Jlc3RhcnQtMDcNCj4gDQo+IEhpIGFsbCwNCj4gDQo+IEkgaGFkIGEgbG9vayBhdCB0aGlzIGRy YWZ0LWlldGYtdGNwbS1ydG9yZXN0YXJ0LTA3IGRyYWZ0Lg0KPiBJIGhhdmUgc29tZSBtaW5vciBj b21tZW50cyBvbiB0aGUgZG9jdW1lbnQuDQo+IA0KPiAjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMj IyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjDQo+IA0KPiAtICJUaGUgUlRPIFJl c3RhcnQgKFJUT1IpIG1lY2hhbmlzbSBkZXNjcmliZWQgaW4gdGhpcyBkb2N1bWVudCBtYWtlcw0K PiB0aGUNCj4gICAgUlRPIHNsaWdodGx5IG1vcmUgYWdncmVzc2l2ZSB3aGVuIHRoZSBudW1iZXIg b2Ygb3V0c3RhbmRpbmcgc2VnbWVudHMNCj4gICAgaXMgdG9vIHNtYWxsIGZvciBmYXN0IHJldHJh bnNtaXQgdG8gd29yaywgaW4gYW4gYXR0ZW1wdCB0byBlbmFibGUNCj4gICAgZmFzdGVyIGxvc3Mg cmVjb3ZlcnkgZm9yIGFsbCBzZWdtZW50cyB3aGlsZSBiZWluZyByb2J1c3QgdG8NCj4gICAgcmVv cmRlcmluZy4iDQo+IA0KPiBUaGUgUlRPIGlzIG5vdCBhZ2dyZXNzaXZlIDsgYnV0IHRoZSByZXRy YW5zbWlzc2lvbiBzY2hlbWUgcnVsZWQgYnkgUlRPDQo+IGV4cGlyYXRpb25zIGlzLg0KPiBJIHdv dWxkIHJlY29tbWVuZCB0byByZXBocmFzZSB0aGF0Lg0KPiANCj4gIyMjIyMjIyMjIyMjIyMjIyMj IyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIw0KPiANCj4gLSDigJwg ICBUaGUgUlRPIFJlc3RhcnQgKFJUT1IpIG1lY2hhbmlzbSBkZXNjcmliZWQgaW4gdGhpcyBkb2N1 bWVudCBtYWtlcw0KPiB0aGUNCj4gICAgUlRPIHNsaWdodGx5IFvigKZdIGhpZ2hseSB2YXJ5aW5n IFJUVHMsIGUuZy4gbW9iaWxlIG5ldHdvcmtzLiINCj4gDQo+IFRoaXMgd2hvbGUgcGFyYWdyYXBo IGNvbWVzIHRvIGVhcmx5IGluIHRoZSBkb2N1bWVudCwgYXMgdGhlIHJlYWRlciBoYXMNCj4gbm8g Y2xlYXINCj4gaWRlYSBhYm91dCB3aGF0IFJUT1IgaXMgYWN0dWFsbHkgZG9pbmcuIEkgcHJvcG9z ZSB0byBhZGQgdGhhdCBpbiBhDQo+IGRlZGljYXRlZCBzdWItDQo+IHNlY3Rpb24gaW4gU2VjdGlv biA1Lg0KPiANCj4gIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMj IyMjIyMjIyMjIyMjIyMjIw0KPiANCj4gLSDigJwgMyBSVE8gUmVzdGFydCBPdmVydmlld+KAnSAt PiBzaG91bGQgaXQgbm90IGJlIOKAnDMtIFJUTyBPdmVydmlldyBhbmQNCj4gcmF0aW9uYWxlIG9m IFJUT1Ig4oCcIG9yIHNvbWV0aGluZyBlcXVpdmFsZW50ID8NCj4gSXQgaXMgY29uZnVzaW5nLCBh cyB5b3UgZG8gbm90IGFjdHVhbGx5IHByZXNlbnQgUlRPUi4NCj4gDQo+IEFsc28sIGluIHRoaXMg c2VjdGlvbiAzOg0KPiBUaGUgZmlyc3QgcGFyYWdyYXBoIGZpbmlzaGVzIHdpdGgg4oCcSGVuY2Us ICBpbiBtb3N0IHNpdHVhdGlvbnMsIHRoZSB0aW1lDQo+IGJlZm9yZSBhIHJldHJhbnNtaXNzaW9u IGlzIHRyaWdnZXJlZCBpcyBlcXVhbCB0byAiUlRPICsgUlRU4oCdLuKAnQ0KPiBCdXQgdGhlIGV4 YW1wbGUgdG8gaWxsdXN0cmF0ZSBpdCBjb21lcyBhZnRlciAtIEkgdGhpbmsgdGhpbmdzIHNob3Vs ZA0KPiBtb3ZlZCBhcm91bmQgdG86DQo+IDEtIGludHJvZHVjZSB0aGUgY2xhc3NpYyBSVE8NCj4g Mi0gZXhwbGFpbiB0aGUgcmF0aW9uYWxlIG9mIGNsYXNzaWMgUlRPDQo+IDMtIGV4YW1wbGUgdG8g c2hvdyB0aGF0IFJUTytSVFQgaXMgY29tbW9uDQo+IDQtIG1vcmUgcGFyYW1ldGVycyBjb3VsZCBh ZmZlY3QgdGhhdCAoMiBvdXRzdGFuZGluZyBzZWdtZW50czsgb3RoZXINCj4gZmFjdG9ycykNCj4g NS0gY29uY2x1c2lvbiBvbiDigJxoZW5jZSwgUlRPK1JUVCBpcyBjb21tb24gZm9yIGFwcGxpY2F0 aW9uIGxpbWl0ZWQNCj4gdHJhZmZpYyINCj4gDQo+IEkgd291bGQgcHJvcG9zZSB0byBpbnRyb2R1 Y2UgYSBUb0MgZm9yIHRoaXMgc2VjdGlvbiAzOg0KPiANCj4gMy0gUlRPIE92ZXJ2aWV3IGFuZCBy YXRpb25hbGUgb2YgUlRPUg0KPiAgIDMtMS4gRXhwZXJpZW5jZWQgUlRPIGlzIFJUTytSVFQNCj4g ICAzLTIuIFJUTyBzYWZldHkgbWFyZ2luDQo+ICAgMy0zLiBSYXRpb25hbGUgb2YgUlRPUjogd2hl biBmYXN0IHJldHJhbnNtaXQgaXMgbm90IHBvc3NpYmxlDQo+IA0KPiAjIyMjIyMjIyMjIyMjIyMj IyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjDQo+IA0KPiBTZWN0 aW9uIDQgOiBzaG91bGQgZm9jdXMgb24gdGhlIGRlc2NyaXB0aW9uIG9mIHRoZSBhbGdvcml0aG0g b25seS4NCj4gDQo+IEkgd291bGQga2VlcA0KPiDigJwNCj4gVGhpcyBhcHByb2FjaCBtYWtlcyB0 aGUgUlRPIG1vcmUNCj4gICAgYWdncmVzc2l2ZSB0aGFuIHRoZSBzdGFuZGFyZGl6ZWQgYXBwcm9h Y2ggaW4gW1JGQzYyOThdIGJ1dCBzdGlsbA0KPiAgICBjb25mb3JtcyB0byB0aGUgcmVxdWlyZW1l bnQgaW4gW1JGQzYyOThdIHRoYXQgc2VnbWVudHMgbXVzdCBub3QgYmUNCj4gICAgcmV0cmFuc21p dHRlZCBlYXJsaWVyIHRoYW4gUlRPIHNlY29uZHMgYWZ0ZXIgdGhlaXIgb3JpZ2luYWwNCj4gICAg dHJhbnNtaXNzaW9uLg0KPiAiDQo+IA0KPiBmb3IgdGhlIGRpc2N1c3Npb24gc2VjdGlvbiAoYXQg dGhpcyBzdGFnZSBvZiB0aGUgcmVhZGluZywgdGhlIHJlYWRlcg0KPiBzdGlsbCBkb2VzIG5vdCBr bm93IGV4YWN0bHkgd2hhdCBSVE9SIGlzIGFjdHVhbGx5IGRvaW5nKS4NCj4gDQo+ICMjIyMjIyMj IyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMNCj4g DQo+IFNlY3Rpb24gNDogIml0IHdpbGwgcHJldmVudCBSVE9SDQo+ICAgIGZyb20gYmVpbmcgbW9y ZSBhZ2dyZXNzaXZlIGFuZCBwb3RlbnRpYWxseSBjYXVzaW5nIFJUT3MgaW5zdGVhZCBvZg0KPiAg ICBmYXN0IHJldHJhbnNtaXRzLuKAnQ0KPiANCj4gTW9yZSBhZ2dyZXNzaXZlIHRoYW4gd2hhdCA/ IEhhcyB0aGUgImZvdXIiIGJlaW5nIGV4cGVyaW1lbnRhbGx5DQo+IG9idGFpbmVkID8NCj4gSSB0 aGluayBtb3JlIGRpc2N1c3Npb24gaXMgcmVxdWlyZWQgb24gdGhpcyBwYXJhbWV0ZXIuDQo+IA0K PiBSZWdhcmRzLA0KPiANCj4gTmljb2xhcw0KPiBfX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fXw0KPiB0Y3BtIG1haWxpbmcgbGlzdA0KPiB0Y3BtQGlldGYub3Jn DQo+IGh0dHBzOi8vd3d3LmlldGYub3JnL21haWxtYW4vbGlzdGluZm8vdGNwbQ0K From nobody Mon May 18 02:35:52 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 137D11A884D for ; Mon, 18 May 2015 02:35:51 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 4.521 X-Spam-Level: **** X-Spam-Status: No, score=4.521 tagged_above=-999 required=5 tests=[BAYES_50=0.8, FM_FORGED_GMAIL=0.622, HELO_EQ_JP=1.244, HOST_EQ_JP=1.265, HTML_MESSAGE=0.001, J_CHICKENPOX_32=0.6, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4A9u-k7XtmkB for ; Mon, 18 May 2015 02:35:48 -0700 (PDT) Received: from mail.sfc.wide.ad.jp (shonan.sfc.wide.ad.jp [IPv6:2001:200:0:8803::53]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 7CCF01A884C for ; Mon, 18 May 2015 02:35:48 -0700 (PDT) Received: from mail-wg0-f47.google.com (mail-wg0-f47.google.com [74.125.82.47]) by mail.sfc.wide.ad.jp (Postfix) with ESMTPSA id E964627813C for ; Mon, 18 May 2015 18:35:45 +0900 (JST) Received: by wgfl8 with SMTP id l8so28893826wgf.2 for ; Mon, 18 May 2015 02:35:43 -0700 (PDT) MIME-Version: 1.0 X-Received: by 10.180.78.135 with SMTP id b7mr20067826wix.65.1431941743491; Mon, 18 May 2015 02:35:43 -0700 (PDT) Received: by 10.194.27.193 with HTTP; Mon, 18 May 2015 02:35:43 -0700 (PDT) In-Reply-To: <201505161152.t4GBqCmx001121@bagheera.jungle.bt.co.uk> References: <201505151233.t4FCXRxW022847@bagheera.jungle.bt.co.uk> <201505161152.t4GBqCmx001121@bagheera.jungle.bt.co.uk> Date: Mon, 18 May 2015 02:35:43 -0700 Message-ID: From: Yoshifumi Nishida To: Bob Briscoe Content-Type: multipart/alternative; boundary=90e6ba475eebf5c441051657eaa8 Archived-At: Cc: tcpm IETF list Subject: Re: [tcpm] Why is the minimum ssthresh 2*SMSS (rather than 1)? X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 May 2015 09:35:51 -0000 --90e6ba475eebf5c441051657eaa8 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Hi Bob, I'm not very sure about this. With your proposal, if FlightSize is zero, cwnd will be zero. In this case, TCP doesn't have a chance to increase cwnd other than keep-alive? Thanks, -- Yoshi On Sat, May 16, 2015 at 4:52 AM, Bob Briscoe wrote: > Ilpo & Yuchung, > > Sorry, the phrasing of my question was too curt. > > I'm not specifically interested in why ssthresh has a min. I only phrased > my question that way because the way TCP fast recovery is specified and > implemented sets a min for cwnd based on the min for ssthresh. > > From RFC5681: > 2. When the third duplicate ACK is received, a TCP MUST set ssthresh > to no more than the value given in equation (4). > ... > 6. When the next ACK arrives that acknowledges previously > unacknowledged data, a TCP MUST set cwnd to ssthresh (the value > set in step 2). This is termed "deflating" the window. > > The only guess I've heard for why it says 2*SMSS in eqn (4) is delayed > ACKs (John Heffner, as well as Michael Welzl when I was asking around > before I posted this to tcpm). Certainly delayed ACKs will cause segments > to bunch in twos even if the window were less than two. However, if the R= TT > was short enough (or the link congested enough) to need a low window, at > least segments could bunch in twos at a lower rate than two per RTT. > > > I believe TCP would not work against AQMs at low RTTs if the min for > ssthresh in Eqn (4) was zero, not 2*SMSS. But there must have been a reas= on > for this minimum at some point in the past - that's what I'm trying to fi= nd > out. > > If there is no strong reason to keep the 2*SMSS minimum, the rule after a > timeout would have to be changed too. Specifically, the two changes neede= d > to RFC5681 would be: > > CURRENT: > ssthresh =3D max (FlightSize / 2, 2*SMSS) (4) > PROPOSED: > ssthresh =3D FlightSize / 2 (4) > > CURRENT > upon a timeout [...] cwnd MUST be set to > no more than the loss window, LW, which equals 1 full-sized segment. > PROPOSED: > upon a timeout [...] cwnd MUST be set to > cwnd =3D min(LW, ssthresh) > where the loss window, LW, which equals 1 full-sized segment. > > > Bob > > > At 23:14 15/05/2015, Ilpo J=C3=A4rvinen wrote: > >> On Fri, 15 May 2015, Bob Briscoe wrote: >> >> > Despite searching and asking around, I cannot find any definitive >> reasoning >> > for the choice of 2*SMSS in eqn (4) in RFC5681. >> > >> > ssthresh =3D max (FlightSize / 2, 2*SMSS) (4) >> > >> > In testing a new AQM with multiple flows, now we've got the RTT really >> low, >> > we've found that TCP's minimum window of 2 makes TCP ignore the AQM's >> signals >> > ('cos it always rounds up to 2) so all the TCP flows drive up the >> queueing >> > delay to give themselves a large-enough RTT - defeating the AQM. >> >> Does this really matter that much whether ssthresh would be 1 or 2 SMSS? >> Both congestion avoidance and slow start are equally aggressive for the >> first RTT: when TCP start with cwnd=3D1 and the sender gets the ACK, the >> applied increase is cwnd =3D cwnd + 1 SMSS regardless of the operating >> mode. >> >> However, ssthresh =3D=3D cwnd case is left open so there might be a >> difference depending on implementation: >> "When cwnd and ssthresh are equal, the sender may use either >> slow start or congestion avoidance." >> ...but I don't see how that would relate to "minimum window of 2" you're >> talking here (and ssthresh is not really "window" anyway). >> >> >> -- >> i. >> > > At 22:52 15/05/2015, Yuchung Cheng wrote: > >> I am confused. min ssthresh !=3D min-cwnd =3D=3D 1? >> > > > > ________________________________________________________________ > Bob Briscoe, BT > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm > --90e6ba475eebf5c441051657eaa8 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Hi Bob,

I'm not very sure about thi= s.
With your proposal, if FlightSize is zero, cwnd will be zero.<= /div>
In this case, TCP doesn't have a chance to increase cwnd othe= r than keep-alive?

Thanks,
--
= Yoshi

On Sat, = May 16, 2015 at 4:52 AM, Bob Briscoe <bob.briscoe@bt.com> w= rote:
Ilpo & Yuchung,

Sorry, the phrasing of my question was too curt.

I'm not specifically interested in why ssthresh has a min. I only phras= ed my question that way because the way TCP fast recovery is specified and = implemented sets a min for cwnd based on the min for ssthresh.

>From RFC5681:
2.=C2=A0 When the third duplicate ACK is received, a TCP MUST set ssthresh<= br> =C2=A0 =C2=A0 =C2=A0 =C2=A0to no more than the value given in equation (4).=
...
6.=C2=A0 When the next ACK arrives that acknowledges previously
=C2=A0 =C2=A0 =C2=A0 =C2=A0unacknowledged data, a TCP MUST set cwnd to ssth= resh (the value
=C2=A0 =C2=A0 =C2=A0 =C2=A0set in step 2).=C2=A0 This is termed "defla= ting" the window.

The only guess I've heard for why it says 2*SMSS in eqn (4) is delayed = ACKs (John Heffner, as well as Michael Welzl when I was asking around befor= e I posted this to tcpm). Certainly delayed ACKs will cause segments to bun= ch in twos even if the window were less than two. However, if the RTT was s= hort enough (or the link congested enough) to need a low window, at least s= egments could bunch in twos at a lower rate than two per RTT.


I believe TCP would not work against AQMs at low RTTs if the min for ssthre= sh in Eqn (4) was zero, not 2*SMSS. But there must have been a reason for t= his minimum at some point in the past - that's what I'm trying to f= ind out.

If there is no strong reason to keep the 2*SMSS minimum, the rule after a t= imeout would have to be changed too. Specifically, the two changes needed t= o RFC5681 would be:

CURRENT:
=C2=A0 =C2=A0 ssthresh =3D max (FlightSize / 2, 2*SMSS)=C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 (4)
 PROPOSED:
=C2=A0 =C2=A0 ssthresh =3D FlightSize / 2=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 (4)

CURRENT
=C2=A0 =C2=A0upon a timeout [...] cwnd MUST be set to
=C2=A0 =C2=A0no more than the loss window, LW, which equals 1 full-sized se= gment.
PROPOSED:
=C2=A0 =C2=A0upon a timeout [...] cwnd MUST be set to
=C2=A0 =C2=A0 =C2=A0cwnd =3D min(LW, ssthresh)
=C2=A0 =C2=A0where the loss window, LW, which equals 1 full-sized segment.<= br>

Bob


At 23:14 15/05/2015, Ilpo J=C3=A4rvinen wrote:
On Fri, 15 May 2015, Bob Briscoe wrote:

> Despite searching and asking around, I cannot find any definitive reas= oning
> for the choice of 2*SMSS in eqn (4) in RFC5681.
>
>=C2=A0 =C2=A0 ssthresh =3D max (FlightSize / 2, 2*SMSS)=C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 (4)
>
> In testing a new AQM with multiple flows, now we've got the RTT re= ally low,
> we've found that TCP's minimum window of 2 makes TCP ignore th= e AQM's signals
> ('cos it always rounds up to 2) so all the TCP flows drive up the = queueing
> delay to give themselves a large-enough RTT - defeating the AQM.

Does this really matter that much whether ssthresh would be 1 or 2 SMSS? Both congestion avoidance and slow start are equally aggressive for the
first RTT: when TCP start with cwnd=3D1 and the sender gets the ACK, the applied increase is cwnd =3D cwnd + 1 SMSS regardless of the operating
mode.

However, ssthresh =3D=3D cwnd case is left open so there might be a
difference depending on implementation:
=C2=A0 =C2=A0"When cwnd and ssthresh are equal, the sender may use eit= her
=C2=A0 =C2=A0 slow start or congestion avoidance."
...but I don't see how that would relate to "minimum window of 2&q= uot; you're
talking here (and ssthresh is not really "window" anyway).


--
=C2=A0i.

At 22:52 15/05/2015, Yuchung Cheng wrote:
I am confused. min ssthresh !=3D min-cwnd =3D=3D 1?



 ________________________________________________________________
Bob Briscoe,=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 BT
_______________________________________________
tcpm mailing list
tcpm@ietf.org
ht= tps://www.ietf.org/mailman/listinfo/tcpm

--90e6ba475eebf5c441051657eaa8-- From nobody Mon May 18 02:58:50 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 892C61A8855 for ; Mon, 18 May 2015 02:58:48 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.302 X-Spam-Level: X-Spam-Status: No, score=-2.302 tagged_above=-999 required=5 tests=[BAYES_20=-0.001, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3-kqNk49pdI0 for ; Mon, 18 May 2015 02:58:46 -0700 (PDT) Received: from sessmg22.ericsson.net (sessmg22.ericsson.net [193.180.251.58]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id C58BF1A883B for ; Mon, 18 May 2015 02:58:45 -0700 (PDT) X-AuditID: c1b4fb3a-f79ec6d000006dc0-75-5559b7d379df Received: from ESESSHC020.ericsson.se (Unknown_Domain [153.88.253.125]) by sessmg22.ericsson.net (Symantec Mail Security) with SMTP id 79.74.28096.3D7B9555; Mon, 18 May 2015 11:58:44 +0200 (CEST) Received: from haze (153.88.183.153) by smtp.internal.ericsson.com (153.88.183.80) with Microsoft SMTP Server id 14.3.210.2; Mon, 18 May 2015 11:58:43 +0200 Date: Mon, 18 May 2015 11:53:08 +0200 From: Erik Hugne To: Message-ID: <20150518095308.GA32297@haze> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFmpgluLIzCtJLcpLzFFi42KZGfG3VvfK9shQg3O3OC22nZzP5MDosWTJ T6YAxigum5TUnMyy1CJ9uwSujKUrbjIV7BSsuLq9roHxMl8XIyeHhICJxNXJ/SwQtpjEhXvr 2boYuTiEBI4ySjyYdJIVwlnAKLFv3U7GLkYODhYBVYnPU7JBGtgEtCQmPp/IBGKLCAhLnG3r YQexmQXUJN7evg8WFxZwkTi24jdYnFdAU2LjvNlMELagxMmZT1gg6nUkFuz+xAYynllAWmL5 Pw6QsKiAisSUk9vYQGwhIPv+y9nsExj5ZyHpnoWkexZC9wJG5lWMosWpxcW56UZGeqlFmcnF xfl5enmpJZsYgUF2cMtvqx2MB587HmIU4GBU4uFN4I8MFWJNLCuuzD3EKM3BoiTO69kVEiok kJ5YkpqdmlqQWhRfVJqTWnyIkYmDU6qBsb7x++915z0EPf2X87e484mIyk85ecNi/Yy96oJ9 S/8rrIjh7Zq+yi7hW+bCecv4Ji+ofuzFXfL46cX83ffZuEr2+XiePNYyt5sxz1y84YrrCokz wZ9anwvbVMzZXdyx+/vWPwwp7WuTPfpP3uEOn+9+pr+nWLrV66XZ8i+LBCRPfOdb8Pf4bCWW 4oxEQy3mouJEAH9Q0JETAgAA Archived-At: Cc: ingemar.s.johansson@ericsson.com Subject: [tcpm] ip_local_reserved_ports impact on inet_csk_get_port (Linux) X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 May 2015 09:58:48 -0000 I raised this on netdev a few weeks ago, but there was no recognition. Short story; Linux has a sysctl knob (net.ipv4.ip_local_reserved_ports) that can be used to exclude ports from the random local port assignment (net.ipv4.ip_local_port_range). If the initial random port is within the reserved ports subset, the port will be selected in an incremental/linear fashion. https://git.kernel.org/cgit/linux/kernel/git/davem/net-next.git/tree/net/ipv4/inet_connection_sock.c#n115 I believe this port reservation is a Linux specific thing, but i was hoping that i could get a tcpm comment if this is considered ok, or if the port selection algorithm needs to be smartened up in the case where there are reserved port ranges. //E ----- Forwarded message from Erik Hugne ----- Date: Tue, 5 May 2015 14:47:59 +0200 From: Erik Hugne To: davem@davemloft.net, netdev@vger.kernel.org Cc: Richard Alpe , Onar Olsen , ying.xue@windriver.com Subject: tcp: ip_local_reserved_ports impact on inet_csk_get_port() User-Agent: Mutt/1.5.21 (2010-09-15) Defining a port range in net.ipv4.ip_local_reserved_ports can cause a linear and predictable behavior of inet_csk_get_port(). This occurs when smallest_rover = rover = prandom_u32() % remaining + low; hits a value in the reserved range. The algorithm will then try the next consecutive port number until a free port is found. Example: net.ipv4.ip_local_port_range = 32768 61000 net.ipv4.ip_local_reserved_ports = 35000-61000 This will give ~92% chance that the initial random port will be in the reserved range, and that the port selection will be done linearly starting from 32768. Section 3.3 in RFC6056[1] describes several port selection algorithms, and Linux seems to follow #1. This does not seem to be the best alternative since e3826f1e946e ("net: reserve ports for applications using fixed port numbers") If the local port range is set not to overlap with the reserved ports, inet_csk_get_port will give a better randomness in the port selection. //E [1] https://tools.ietf.org/html/rfc6056 ----- End forwarded message ----- From nobody Mon May 18 04:15:08 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4D9051A891B for ; Mon, 18 May 2015 04:15:07 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.911 X-Spam-Level: X-Spam-Status: No, score=-6.911 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id BBI7J9FNFw14 for ; Mon, 18 May 2015 04:15:05 -0700 (PDT) Received: from mx142.netapp.com (mx142.netapp.com [216.240.21.19]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 106DE1A88F8 for ; Mon, 18 May 2015 04:15:05 -0700 (PDT) X-IronPort-AV: E=Sophos;i="5.13,453,1427785200"; d="scan'208";a="41951443" Received: from hioexcmbx01-prd.hq.netapp.com ([10.122.105.34]) by mx142-out.netapp.com with ESMTP; 18 May 2015 04:10:04 -0700 Received: from HIOEXCMBX07-PRD.hq.netapp.com (10.122.105.40) by hioexcmbx01-prd.hq.netapp.com (10.122.105.34) with Microsoft SMTP Server (TLS) id 15.0.1076.9; Mon, 18 May 2015 04:10:04 -0700 Received: from HIOEXCMBX07-PRD.hq.netapp.com ([fe80::7ce8:9818:3705:3318]) by hioexcmbx07-prd.hq.netapp.com ([fe80::7ce8:9818:3705:3318%21]) with mapi id 15.00.1076.000; Mon, 18 May 2015 04:10:04 -0700 From: "Eggert, Lars" To: "Scharf, Michael (Michael)" Thread-Topic: [tcpm] Feedback on WG acceptance of draft-bensley-tcpm-dctcp-03 Thread-Index: AdB82dvas/DQQ7w2TuSWSxKiXj5XLgUbrOuQABNTGwA= Date: Mon, 18 May 2015 11:10:03 +0000 Message-ID: References: <655C07320163294895BBADA28372AF5D16C939E2@FR712WXCHMBA15.zeu.alcatel-lucent.com> <655C07320163294895BBADA28372AF5D483AF007@FR712WXCHMBA15.zeu.alcatel-lucent.com> In-Reply-To: <655C07320163294895BBADA28372AF5D483AF007@FR712WXCHMBA15.zeu.alcatel-lucent.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: Apple Mail (2.2100) x-ms-exchange-transport-fromentityheader: Hosted x-originating-ip: [10.122.56.79] Content-Type: text/plain; charset="us-ascii" Content-ID: <792122F6919D4244B36B3C5017E53CBE@hq.netapp.com> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Archived-At: Cc: "tcpm@ietf.org" Subject: Re: [tcpm] Feedback on WG acceptance of draft-bensley-tcpm-dctcp-03 X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 May 2015 11:15:07 -0000 Hi, On 2015-5-18, at 11:12, Scharf, Michael (Michael) wrote: > There have not been many responses to this e-mail. The chairs therefore c= onclude that there is not enough support for adoption of draft-bensley-tcpm= -dctcp-03 as TCPM WG item as of now. thanks, we'll investigate the IRTF or Independent stream then. The document= is ready to go from our side. > Since the TCPM community seems interested in further discussing this topi= c, the chairs would offer to allocate time for corresponding discussions du= ring upcoming TCPM meetings. I don't want to sound snarky, but it seems at odds that the WG isn't intere= sted enough to publish the document, but still wants to discuss the topic. = Personally, my interest in discussing this further with a group that doesn'= t want to take on the document is now pretty low. Lars >=20 > Thanks >=20 > Michael, Pasi, Yoshifumi >=20 >=20 > PS: We recently found one possibly interesting study on operational exper= ience: https://www.usenix.org/system/files/conference/nsdi15/nsdi15-paper-j= udd.pdf >=20 >=20 >> -----Original Message----- >> From: tcpm [mailto:tcpm-bounces@ietf.org] On Behalf Of Scharf, Michael >> (Michael) >> Sent: Wednesday, April 22, 2015 10:54 AM >> To: tcpm@ietf.org >> Subject: [tcpm] Feedback on WG acceptance of draft-bensley-tcpm-dctcp- >> 03 >>=20 >> Dear all, >>=20 >> During IETF 89 [1] and on the list there has been a discussion about WG >> adoption of draft-bensley-tcpm-dctcp [2]. The understanding of the >> chairs is that there is no clear consensus up to now. >>=20 >> In order to move forward, the chairs seek for community guidance >> regarding the following options to handle this draft: >>=20 >> a) Adopt in TCPM as Informational >>=20 >> b) Adopt in TCPM as Experimental >>=20 >> c) Do not adopt in TCPM now >>=20 >> Option c) does not prevent publication outside TCPM, such as >> publication by another working group (e.g., TSVWG), publication on >> Independent Stream, etc. >>=20 >> The TCPM chairs believe that adoption of alternative congestion control >> algorithms in TCPM should be backed by a strong consensus. Please also >> review [3] and [4] regarding the difference between Experimental and >> Informational. >>=20 >> Please let us know any feedback on a potential WG acceptance of draft- >> bensley-tcpm-dctcp-03 until May 8. >>=20 >> Thanks! >>=20 >> Michael, Pasi, Yoshifumi >>=20 >>=20 >> [1] http://www.ietf.org/proceedings/89/minutes/minutes-89-tcpm >>=20 >> [2] https://tools.ietf.org/html/draft-bensley-tcpm-dctcp-03 >>=20 >> [3] RFC 2026, Section 4.2.1, 4.2.2 >>=20 >> [4] https://www.ietf.org/iesg/informational-vs-experimental.html >>=20 >> _______________________________________________ >> tcpm mailing list >> tcpm@ietf.org >> https://www.ietf.org/mailman/listinfo/tcpm >=20 > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm From nobody Mon May 18 06:11:14 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B3C7D1A00E6 for ; Mon, 18 May 2015 06:11:12 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.8 X-Spam-Level: X-Spam-Status: No, score=0.8 tagged_above=-999 required=5 tests=[BAYES_50=0.8] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id VK3zNVHA5Zgk for ; Mon, 18 May 2015 06:11:10 -0700 (PDT) Received: from rebar.astron.com (rebar.astron.com [IPv6:2620:106:3003:1f00:9e8e:99ff:fe15:cae4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id BACBB1A006B for ; Mon, 18 May 2015 06:11:10 -0700 (PDT) Received: by rebar.astron.com (Postfix, from userid 10080) id 8A54D17FDA8; Mon, 18 May 2015 13:11:08 +0000 (UTC) From: christos@zoulas.com (Christos Zoulas) Date: Mon, 18 May 2015 09:11:08 -0400 In-Reply-To: <20150518095308.GA32297@haze> from Erik Hugne (May 18, 11:53am) Organization: Astron Software X-Mailer: Mail User's Shell (7.2.6 beta(4.pl1)+dynamic 20000103) To: Erik Hugne , Message-Id: <20150518131108.8A54D17FDA8@rebar.astron.com> Archived-At: Cc: ingemar.s.johansson@ericsson.com Subject: Re: [tcpm] ip_local_reserved_ports impact on inet_csk_get_port (Linux) X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 May 2015 13:11:12 -0000 On May 18, 11:53am, erik.hugne@ericsson.com (Erik Hugne) wrote: -- Subject: [tcpm] ip_local_reserved_ports impact on inet_csk_get_port (Linux | I believe this port reservation is a Linux specific thing, but i was hoping | that i could get a tcpm comment if this is considered ok, or if the port | selection algorithm needs to be smartened up in the case where there are | reserved port ranges. As another data point, NetBSD also has the port reservation feature, but does not suffer from the linear scan: http://nxr.netbsd.org/xref/src/sys/netinet/portalgo.c christos From nobody Mon May 18 08:15:25 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 548E31A916E for ; Mon, 18 May 2015 08:15:23 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2 X-Spam-Level: X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Vl5WaEQIBILi for ; Mon, 18 May 2015 08:15:22 -0700 (PDT) Received: from mail-la0-x235.google.com (mail-la0-x235.google.com [IPv6:2a00:1450:4010:c03::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 3B6D71A9237 for ; Mon, 18 May 2015 08:15:12 -0700 (PDT) Received: by lagv1 with SMTP id v1so225128687lag.3 for ; Mon, 18 May 2015 08:15:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=QP699teuMhSny3lnaCnEqex5B/jBfL31jJW+JHVsigA=; b=Zetmto+FG6PgKM3wM8mTOBrT8CbVYmDBefH0buc4zRJ4R03OQGo/UC2FZQwwWrc0Wi NktnDq5zIx+cqkIMrV27NV5XPJ6e19oJspw4IyzfefxqvOOPqDjeHnplht+0Mia06FaG OTfV16Eb3an+5C4cf1HohBOU958+Bj7cBDa0UZzkSn7XUxqNT8k+q3BI31m2z2qHEBLA IcheUXqeyylzhwZcDlnMY0wYMBuhyBqqNFejZgnaiC+dRgb7V6W/E2WqacXXc0OA92Q7 J2OsRKhL9Kqd/kdaAjWf4/eVokoisQAcACt8CqGCTf8hXUzK9DmZ2LUqzZP/v38DxJTC a6zQ== MIME-Version: 1.0 X-Received: by 10.152.206.103 with SMTP id ln7mr17840144lac.40.1431962110709; Mon, 18 May 2015 08:15:10 -0700 (PDT) Received: by 10.25.12.7 with HTTP; Mon, 18 May 2015 08:15:10 -0700 (PDT) In-Reply-To: <201505161152.t4GBqCmx001121@bagheera.jungle.bt.co.uk> References: <201505151233.t4FCXRxW022847@bagheera.jungle.bt.co.uk> <201505161152.t4GBqCmx001121@bagheera.jungle.bt.co.uk> Date: Mon, 18 May 2015 11:15:10 -0400 Message-ID: From: John Heffner To: Bob Briscoe Content-Type: text/plain; charset=UTF-8 Archived-At: Cc: tcpm IETF list Subject: Re: [tcpm] Why is the minimum ssthresh 2*SMSS (rather than 1)? X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 May 2015 15:15:23 -0000 On Sat, May 16, 2015 at 7:52 AM, Bob Briscoe wrote: > Ilpo & Yuchung, > > Sorry, the phrasing of my question was too curt. > > I'm not specifically interested in why ssthresh has a min. I only phrased my > question that way because the way TCP fast recovery is specified and > implemented sets a min for cwnd based on the min for ssthresh. > > From RFC5681: > 2. When the third duplicate ACK is received, a TCP MUST set ssthresh > to no more than the value given in equation (4). > ... > 6. When the next ACK arrives that acknowledges previously > unacknowledged data, a TCP MUST set cwnd to ssthresh (the value > set in step 2). This is termed "deflating" the window. > > The only guess I've heard for why it says 2*SMSS in eqn (4) is delayed ACKs > (John Heffner, as well as Michael Welzl when I was asking around before I > posted this to tcpm). Certainly delayed ACKs will cause segments to bunch in > twos even if the window were less than two. However, if the RTT was short > enough (or the link congested enough) to need a low window, at least > segments could bunch in twos at a lower rate than two per RTT. With delayed ack, reducing cwnd to one breaks the ack clock. Transmission instead will be driven by the delayed ack timer. -John From nobody Mon May 18 08:19:09 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B625A1A913E for ; Mon, 18 May 2015 08:19:06 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -5.51 X-Spam-Level: X-Spam-Status: No, score=-5.51 tagged_above=-999 required=5 tests=[BAYES_05=-0.5, RCVD_IN_DNSWL_HI=-5, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id m6eCIq_s_1Zs for ; Mon, 18 May 2015 08:19:05 -0700 (PDT) Received: from boreas.isi.edu (boreas.isi.edu [128.9.160.161]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id CB24D1A916D for ; Mon, 18 May 2015 08:19:05 -0700 (PDT) Received: from [192.168.1.11] (pool-71-103-148-202.lsanca.dsl-w.verizon.net [71.103.148.202]) (authenticated bits=0) by boreas.isi.edu (8.13.8/8.13.8) with ESMTP id t4IFIVbA007581 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Mon, 18 May 2015 08:18:40 -0700 (PDT) Message-ID: <555A02C6.5050100@isi.edu> Date: Mon, 18 May 2015 08:18:30 -0700 From: Joe Touch User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.6.0 MIME-Version: 1.0 To: John Heffner , Bob Briscoe References: <201505151233.t4FCXRxW022847@bagheera.jungle.bt.co.uk> <201505161152.t4GBqCmx001121@bagheera.jungle.bt.co.uk> In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Archived-At: Cc: tcpm IETF list Subject: Re: [tcpm] Why is the minimum ssthresh 2*SMSS (rather than 1)? X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 May 2015 15:19:06 -0000 +1 This is also why the original initial window was 2*SMSS. Joe On 5/18/2015 8:15 AM, John Heffner wrote: > With delayed ack, reducing cwnd to one breaks the ack clock. > Transmission instead will be driven by the delayed ack timer. > > -John From nobody Mon May 18 09:40:37 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E48F01ACCD9 for ; Mon, 18 May 2015 09:40:36 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.2 X-Spam-Level: X-Spam-Status: No, score=-4.2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1JBJMPXKX8rA for ; Mon, 18 May 2015 09:40:33 -0700 (PDT) Received: from sessmg22.ericsson.net (sessmg22.ericsson.net [193.180.251.58]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id DE4961AD0A6 for ; Mon, 18 May 2015 09:40:27 -0700 (PDT) X-AuditID: c1b4fb3a-f79ec6d000006dc0-d6-555a15f9a3c5 Received: from ESESSHC010.ericsson.se (Unknown_Domain [153.88.253.125]) by sessmg22.ericsson.net (Symantec Mail Security) with SMTP id 57.9B.28096.9F51A555; Mon, 18 May 2015 18:40:25 +0200 (CEST) Received: from ESESSMB309.ericsson.se ([169.254.9.213]) by ESESSHC010.ericsson.se ([153.88.183.48]) with mapi id 14.03.0210.002; Mon, 18 May 2015 18:40:25 +0200 From: Erik Hugne To: Christos Zoulas Thread-Topic: [tcpm] ip_local_reserved_ports impact on inet_csk_get_port (Linux) Thread-Index: AQHQkWwa0UiIqNFhG029jyXYAHOQkp2B79WB Date: Mon, 18 May 2015 16:40:24 +0000 Message-ID: References: <20150518095308.GA32297@haze> from Erik Hugne (May 18, 11:53am),<20150518131108.8A54D17FDA8@rebar.astron.com> In-Reply-To: <20150518131108.8A54D17FDA8@rebar.astron.com> Accept-Language: en-GB, en-US Content-Language: en-GB X-MS-Has-Attach: X-MS-TNEF-Correlator: Content-Type: multipart/alternative; boundary="_000_BB045146AF8EED43BCE05F7F2DB3385311D0114BESESSMB309erics_" MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFvrBLMWRmVeSWpSXmKPExsUyM+Jvre5P0ahQgxfN1hZvNmZabDs5n8mB yWPJkp9MHk0T37AFMEVx2aSk5mSWpRbp2yVwZczZO5u9oEm04uLmS2wNjG8Euxg5OSQETCR2 PLvNAmGLSVy4t56ti5GLQ0jgKKNE44RFrBDOEkaJ71MeMoJUsQloSUx8PpEJxBYR0JT4eO4e WJxZIFZi5ak+oG4ODmGBQIlJL4QgSoIkNvdNZwcJiwgYSfzdkg8SZhFQleiY+Z4VxOYV8JW4 f+8j1N5GRomZrx+AHcQpYCXx4+xcNhCbUUBW4kvjamaIVeISTV9WskIcLSCxZM95ZghbVOLl 43+sEDX5Egs3XWeEWCAocXLmE5YJjCKzkLTPQlI2C0kZRFxP4sbUKWwQtrbEsoWvmSFsXYkZ /w6xIIsvYGRfxShanFpcnJtuZKSXWpSZXFycn6eXl1qyiREYVQe3/LbawXjwueMhRgEORiUe 3gT+yFAh1sSy4srcQ4zSHCxK4ryeXSGhQgLpiSWp2ampBalF8UWlOanFhxiZODilGhinG7x6 tPnxgWolT2uuE13VexeLlNyamG1iMLth4eRJ2mxm+oc5D2hs7/lnUGrWJvW0rWieueieNytW JJaq9lodeH9uzVFmn7W/HTU3nHh4ofelyh1RDmeBlihdze1FCTxGd2s+/257nSX3cM3KGGYz rclsc1N6p0SeezD55td1d9SPBGx8wRavxFKckWioxVxUnAgA9/jOMosCAAA= Archived-At: Cc: Ingemar Johansson S , "tcpm@ietf.org" Subject: Re: [tcpm] ip_local_reserved_ports impact on inet_csk_get_port (Linux) X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 May 2015 16:40:37 -0000 --_000_BB045146AF8EED43BCE05F7F2DB3385311D0114BESESSMB309erics_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable The default algo_bsd does seem to revert to a linear (pick next lower) if t= he initial port hits a reserved range.. //E On 18 May 2015 15:11, Christos Zoulas wrote: On May 18, 11:53am, erik.hugne@ericsson.com (Erik Hugne) wrote: -- Subject: [tcpm] ip_local_reserved_ports impact on inet_csk_get_port (Lin= ux | I believe this port reservation is a Linux specific thing, but i was hopi= ng | that i could get a tcpm comment if this is considered ok, or if the port | selection algorithm needs to be smartened up in the case where there are | reserved port ranges. As another data point, NetBSD also has the port reservation feature, but does not suffer from the linear scan: http://nxr.netbsd.org/xref/src/sys/netinet/portalgo.c christos --_000_BB045146AF8EED43BCE05F7F2DB3385311D0114BESESSMB309erics_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

The default algo_bsd does seem to revert to a linear (pick n= ext lower) if the initial port hits a reserved range..

//E

On 18 May 2015 15:11, Christos Zoulas <christos@z= oulas.com> wrote:
On May 18, 11:53am, erik.hugne@ericsson.com (Erik = Hugne) wrote:
-- Subject: [tcpm] ip_local_reserved_ports impact on inet_csk_get_port (Lin= ux

| I believe this port reservation is a Linux specific thing, but i was hopi= ng
| that i could get a tcpm comment if this is considered ok, or if the port<= br> | selection algorithm needs to be smartened up in the case where there are<= br> | reserved port ranges.

As another data point, NetBSD also has the port reservation feature, but does not suffer from the linear scan:

    http://nxr.netbsd.org/xref/src/sys/netinet/portalgo.c

christos
 --_000_BB045146AF8EED43BCE05F7F2DB3385311D0114BESESSMB309erics_-- From nobody Mon May 18 10:08:41 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3C2551A002F for ; Mon, 18 May 2015 10:08:40 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fDzkk_YNRfZO for ; Mon, 18 May 2015 10:08:38 -0700 (PDT) Received: from rebar.astron.com (rebar.astron.com [IPv6:2620:106:3003:1f00:9e8e:99ff:fe15:cae4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id BCCF41A007A for ; Mon, 18 May 2015 10:08:38 -0700 (PDT) Received: by rebar.astron.com (Postfix, from userid 10080) id 929C417FDA8; Mon, 18 May 2015 17:08:36 +0000 (UTC) From: christos@zoulas.com (Christos Zoulas) Date: Mon, 18 May 2015 13:08:36 -0400 In-Reply-To: from Erik Hugne (May 18, 4:40pm) Organization: Astron Software X-Mailer: Mail User's Shell (7.2.6 beta(4.pl1)+dynamic 20000103) To: Erik Hugne Message-Id: <20150518170836.929C417FDA8@rebar.astron.com> Archived-At: Cc: Ingemar Johansson S , "tcpm@ietf.org" Subject: Re: [tcpm] ip_local_reserved_ports impact on inet_csk_get_port (Linux) X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 May 2015 17:08:40 -0000 On May 18, 4:40pm, erik.hugne@ericsson.com (Erik Hugne) wrote: -- Subject: Re: [tcpm] ip_local_reserved_ports impact on inet_csk_get_port (L | The default algo_bsd does seem to revert to a linear (pick next lower) if | the initial port hits a reserved range.. algo_bsd is the "Traditional BSD Port Selection Algorithm" which is always linear as described in RFC 6056 section 2.2. christos From nobody Mon May 18 11:26:29 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1463E1B29FE for ; Mon, 18 May 2015 11:26:28 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.51 X-Spam-Level: X-Spam-Status: No, score=-0.51 tagged_above=-999 required=5 tests=[BAYES_05=-0.5, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2kiFDUWFzGfU for ; Mon, 18 May 2015 11:26:26 -0700 (PDT) Received: from webspace.isi.edu (webspace.isi.edu [128.9.64.65]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id CE1891AD34D for ; Mon, 18 May 2015 11:26:12 -0700 (PDT) Received: from [128.9.160.252] (pen.isi.edu [128.9.160.252]) (authenticated bits=0) by vapor.isi.edu (8.13.8/8.13.8) with ESMTP id t4IIPJPD027458 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Mon, 18 May 2015 11:25:19 -0700 (PDT) Message-ID: <555A2E8E.4070101@isi.edu> Date: Mon, 18 May 2015 11:25:18 -0700 From: Joe Touch User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.6.0 MIME-Version: 1.0 To: John Heffner , Bob Briscoe References: <201505151233.t4FCXRxW022847@bagheera.jungle.bt.co.uk> <201505161152.t4GBqCmx001121@bagheera.jungle.bt.co.uk> <555A02C6.5050100@isi.edu> In-Reply-To: <555A02C6.5050100@isi.edu> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Archived-At: Cc: tcpm IETF list Subject: Re: [tcpm] Why is the minimum ssthresh 2*SMSS (rather than 1)? X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 May 2015 18:26:28 -0000 On 5/18/2015 8:18 AM, Joe Touch wrote: > +1 > > This is also why the original initial window was 2*SMSS. To clarify: the initial CWND window spec'd in RFC2001 is 1 SMSS I recall reports when RFC2414 was being proposed that BSD never really implemented this; they started with 2*SMSS the interaction with 1*SMSS and delayed ACKs was known back in the late 90s (Mark Allman, Sally Floyd, et al.) which is why RFC2414 spec'd CWND in terms of an even number of SMSS bounded by 4K In general, CWND with an odd number of SMSS is a bad idea. Joe > > Joe > > On 5/18/2015 8:15 AM, John Heffner wrote: >> With delayed ack, reducing cwnd to one breaks the ack clock. >> Transmission instead will be driven by the delayed ack timer. >> >> -John From nobody Mon May 18 11:45:02 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A6B211B2A0D for ; Mon, 18 May 2015 11:44:59 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.012 X-Spam-Level: X-Spam-Status: No, score=0.012 tagged_above=-999 required=5 tests=[BAYES_05=-0.5, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FM_FORGED_GMAIL=0.622, HTML_MESSAGE=0.001, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id QTKL31IuHiep for ; Mon, 18 May 2015 11:44:58 -0700 (PDT) Received: from mail-ob0-x22a.google.com (mail-ob0-x22a.google.com [IPv6:2607:f8b0:4003:c01::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id F1CAB1B2A09 for ; Mon, 18 May 2015 11:44:57 -0700 (PDT) Received: by obcus9 with SMTP id us9so136743278obc.2 for ; Mon, 18 May 2015 11:44:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=GXZ5qmtsAXag9LQHZbsKUdKwEY5RECjlAsI47Sb34oM=; b=hSAi3rSBRPyYN/y3l7dmeyJJhteaQh5pMYgpspAJABUqJLeOcN3JcYBvKsy8HyQfbg DvOHc4QS4q0sol215nJ46R0Jav+pjPDaHMH+QtKhpHTx2WjBoWJsaE5TsF+cY2ZEYQlQ 5NvqpMOSdd6uV06mK7xmZDf/kk/LI0vu5n5RxtPssNOSR+VKrXX2fkL4J40PAyG2GTaB 6SZ5TdkBnHJ0DfOlt8bRn1rGMAIyEK1qzLQjBHo61Rnf47akept5jlkgYbilfK93oaC6 /V7fgOdYALz+R4lAICtNiAc62BxeGy1eb4SKR1egS+7hCFARCfRvixAxptWoj0ENHTJq cLrA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=GXZ5qmtsAXag9LQHZbsKUdKwEY5RECjlAsI47Sb34oM=; b=QWp9emc2v1O0/vK8Aee3yCHBQ97H7g7UJlLiuOx9AxvWw/izCeb0mE33Zd1hA+xLJX U9A6XVPrZXzJCEykq3yFXpWT+G2JD3taf+oFuMSalzOVaUl2vsQ0c9KrqD5SgwB5yaSN xnB5vSUxXioorRWWKnpXRtHN5BaKiaE3s64trAlJDCislFN2UDqj6HUs8q1JOalXDf7v O+fqmTQKCC1o9U0p5VxU5iaQn/B1hAxbuoCmFL1sLZaZ1pKGKl6+5Rqm53HmlQkdwDfs f7+0iwK9vybP1a2zCXLFehcUqyhdcVQo5qnYpvwPQCYIzydWBXOiwJ1I6cDrDuZmy7Hx KNvA== X-Gm-Message-State: ALoCoQmxCG9rmCpx0HsDhqyPR87/bjRC+iE83k6b/Qf/88YbwoG97v/7kMBGaYij1sWiUR0I12aI MIME-Version: 1.0 X-Received: by 10.202.217.196 with SMTP id q187mr20087921oig.64.1431974697358; Mon, 18 May 2015 11:44:57 -0700 (PDT) Received: by 10.182.31.84 with HTTP; Mon, 18 May 2015 11:44:57 -0700 (PDT) In-Reply-To: <555A2E8E.4070101@isi.edu> References: <201505151233.t4FCXRxW022847@bagheera.jungle.bt.co.uk> <201505161152.t4GBqCmx001121@bagheera.jungle.bt.co.uk> <555A02C6.5050100@isi.edu> <555A2E8E.4070101@isi.edu> Date: Mon, 18 May 2015 11:44:57 -0700 Message-ID: From: Matt Mathis To: Joe Touch Content-Type: multipart/alternative; boundary=001a113d579629fe4005165f97bb Archived-At: Cc: tcpm IETF list Subject: Re: [tcpm] Why is the minimum ssthresh 2*SMSS (rather than 1)? X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 May 2015 18:44:59 -0000 --001a113d579629fe4005165f97bb Content-Type: text/plain; charset=UTF-8 A smidge more detail: it was understood that for efficiency w/ DA we almost always want cwnd >= 2, however there are some corner cases (persistent timeouts due to high losses in particular) where this seems a bit unsafe. As a consequence you get to cwnd=1 in a few cases, but if that one packet is delivered, you immediately go to cwnd=2. This balance was carefully thought out at one point, but may be due to be revisited. Thanks, --MM-- The best way to predict the future is to create it. - Alan Kay Privacy matters! We know from recent events that people are using our services to speak in defiance of unjust governments. We treat privacy and security as matters of life and death, because for some users, they are. On Mon, May 18, 2015 at 11:25 AM, Joe Touch wrote: > > > On 5/18/2015 8:18 AM, Joe Touch wrote: > > +1 > > > > This is also why the original initial window was 2*SMSS. > > To clarify: > > the initial CWND window spec'd in RFC2001 is 1 SMSS > > I recall reports when RFC2414 was being proposed > that BSD never really implemented this; they started > with 2*SMSS > > the interaction with 1*SMSS and delayed ACKs was > known back in the late 90s (Mark Allman, Sally Floyd, et al.) > > which is why RFC2414 spec'd CWND in terms of an > even number of SMSS bounded by 4K > > In general, CWND with an odd number of SMSS is a bad idea. > > Joe > > > > > Joe > > > > On 5/18/2015 8:15 AM, John Heffner wrote: > >> With delayed ack, reducing cwnd to one breaks the ack clock. > >> Transmission instead will be driven by the delayed ack timer. > >> > >> -John > > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm > --001a113d579629fe4005165f97bb Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
A smidge more detail: it was understood that for efficienc= y w/ DA we almost always want cwnd >=3D 2, however there are some corner= cases (persistent timeouts due to high losses in particular) where this se= ems a bit unsafe.=C2=A0 As a consequence you get to cwnd=3D1 in a few cases= , but if that one packet is delivered, you immediately go to cwnd=3D2.=C2= =A0 This balance was carefully thought out at one point, but may be due to = be revisited.

Thanks,
--MM--
The best way t= o predict the future is to create it. =C2=A0- Alan Kay

Privacy matte= rs!=C2=A0 We know from recent events that people are using our services to = speak in defiance of unjust governments. =C2=A0 We treat privacy and securi= ty as matters of life and death, because for some users, they are.

On Mon, May 18, 2015 at 11:25 AM, Joe Touch = <to= uch@isi.edu> wrote:


On 5/18/2015 8:18 AM, Joe Touch wrote:
> +1
>
> This is also why the original initial window was 2*SMSS.

To clarify:

=C2=A0 =C2=A0 =C2=A0 =C2=A0 the initial CWND window spec'd in RFC2001= =C2=A0 is 1 SMSS

=C2=A0 =C2=A0 =C2=A0 =C2=A0 I recall reports when RFC2414 was being propose= d
=C2=A0 =C2=A0 =C2=A0 =C2=A0 that BSD never really implemented this; they st= arted
=C2=A0 =C2=A0 =C2=A0 =C2=A0 with 2*SMSS

=C2=A0 =C2=A0 =C2=A0 =C2=A0 the interaction with 1*SMSS and delayed ACKs wa= s
=C2=A0 =C2=A0 =C2=A0 =C2=A0 known back in the late 90s (Mark Allman, Sally = Floyd, et al.)

=C2=A0 =C2=A0 =C2=A0 =C2=A0 which is why RFC2414 spec'd CWND in terms o= f an
=C2=A0 =C2=A0 =C2=A0 =C2=A0 even number of SMSS bounded by 4K

In general, CWND with an odd number of SMSS is a bad idea.

Joe

>
> Joe
>
> On 5/18/2015 8:15 AM, John Heffner wrote:
>> With delayed ack, reducing cwnd to one breaks the ack clock.
>> Transmission instead will be driven by the delayed ack timer.
>>
>>=C2=A0 =C2=A0-John

_______________________________________________
tcpm mailing list
tcpm@ietf.org
ht= tps://www.ietf.org/mailman/listinfo/tcpm

--001a113d579629fe4005165f97bb-- From nobody Mon May 18 12:08:22 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 671321B2A67 for ; Mon, 18 May 2015 12:08:21 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.389 X-Spam-Level: X-Spam-Status: No, score=-1.389 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FM_FORGED_GMAIL=0.622, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id UwLOmE8QQ2Hm for ; Mon, 18 May 2015 12:08:19 -0700 (PDT) Received: from mail-ob0-x236.google.com (mail-ob0-x236.google.com [IPv6:2607:f8b0:4003:c01::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id BDBA01A001A for ; Mon, 18 May 2015 12:08:19 -0700 (PDT) Received: by obbkp3 with SMTP id kp3so137234430obb.3 for ; Mon, 18 May 2015 12:08:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=fVXnTrN+QwqDya5D9iogGv6hCkeYWD9MPKn06+CbH9c=; b=PG+dUaCz2il39G5wvrOApiAe30RYEPzxnxNGGytKGruUctSCtYb0iqC9BqIg57a5TM /TL623rxa5hBy646nYwu75au3IM7ObtY7npiOGaeEvktX9ApODkkaoQzzA+IbJqwcAbQ IWt848fvIuCMW+bje67jP3WV4qE95Rsdf9lDTh7rcc1mXWFsw7ZULdsy5UnyovhX9C4T gaVv8aL+xbM1Ta+wYq84FhTboi/jQMcYtkwVGPL+aG6nQ32KpgVMHhB3iKCFPB4hZ+Cy eA4BF/5q24RBvL8tvSg95cbk9x4dGFW19nFQY+6twDbRZh8vEQUFsWdQWSCw4R/2NVGs F95Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type; bh=fVXnTrN+QwqDya5D9iogGv6hCkeYWD9MPKn06+CbH9c=; b=mOMUGwTVdRB0eYWHPIh5lQJ+ok1S2q6ccIxJa/o36Eng+BclroKYu1JXtyHOmMwa2d +JkI3QdjGbym39w0YjQdiz4HIma2U2gpLPRT1t/VTu+1OX2kXtVoqfcScDSbOUsSzK5m vvSreXp7PC9HP9cRzAAVMn40JBs6oYwugHOoLFAW8MugM4EfxUUeoLEqxis0XPLsuYKM ykThQ7qXUCii+vhXPgG0UUZIoAv7hbzJA9ljn3id7hTlI8FwIv7VNhl9rg3UBBII38JT wd369Jxh1ENU5D92fFkewI14P+X8Fuz7Z8/Sahx0N9Znn/YZX7iYo0LUPh0MSCY3uZ/9 ZpWg== X-Gm-Message-State: ALoCoQnF6kGxVs69FCKubAgz7NoEf0xJMd3oRStF8B3W5nRN8eO0VHHn21goNsTzsKp8SIGEEZhw X-Received: by 10.202.201.3 with SMTP id z3mr20118116oif.130.1431976099194; Mon, 18 May 2015 12:08:19 -0700 (PDT) MIME-Version: 1.0 Received: by 10.182.230.230 with HTTP; Mon, 18 May 2015 12:07:38 -0700 (PDT) In-Reply-To: References: <201505151233.t4FCXRxW022847@bagheera.jungle.bt.co.uk> <201505161152.t4GBqCmx001121@bagheera.jungle.bt.co.uk> From: Yuchung Cheng Date: Mon, 18 May 2015 12:07:38 -0700 Message-ID: To: John Heffner Content-Type: text/plain; charset=UTF-8 Archived-At: Cc: tcpm IETF list Subject: Re: [tcpm] Why is the minimum ssthresh 2*SMSS (rather than 1)? X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 May 2015 19:08:21 -0000 On Mon, May 18, 2015 at 8:15 AM, John Heffner wrote: > > On Sat, May 16, 2015 at 7:52 AM, Bob Briscoe wrote: > > Ilpo & Yuchung, > > > > Sorry, the phrasing of my question was too curt. > > > > I'm not specifically interested in why ssthresh has a min. I only phrased my > > question that way because the way TCP fast recovery is specified and > > implemented sets a min for cwnd based on the min for ssthresh. > > > > From RFC5681: > > 2. When the third duplicate ACK is received, a TCP MUST set ssthresh > > to no more than the value given in equation (4). > > ... > > 6. When the next ACK arrives that acknowledges previously > > unacknowledged data, a TCP MUST set cwnd to ssthresh (the value > > set in step 2). This is termed "deflating" the window. > > > > The only guess I've heard for why it says 2*SMSS in eqn (4) is delayed ACKs > > (John Heffner, as well as Michael Welzl when I was asking around before I > > posted this to tcpm). Certainly delayed ACKs will cause segments to bunch in > > twos even if the window were less than two. However, if the RTT was short > > enough (or the link congested enough) to need a low window, at least > > segments could bunch in twos at a lower rate than two per RTT. > > With delayed ack, reducing cwnd to one breaks the ack clock. > Transmission instead will be driven by the delayed ack timer. Delayed-ack may not be a concern in fast recovery, because the receiver does not delay ACK when there is out-of-order data in the queue. Bob: does reducing ssthresh lower-bound to 1 help your aqm test case? > > -John From nobody Tue May 19 00:48:38 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 998241ACE43 for ; Tue, 19 May 2015 00:48:37 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hSyogeHtHfA1 for ; Tue, 19 May 2015 00:48:36 -0700 (PDT) Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id C44E81ACE45 for ; Tue, 19 May 2015 00:48:33 -0700 (PDT) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit To: X-Test-IDTracker: no X-IETF-IDTracker: 6.0.3 Auto-Submitted: auto-generated Precedence: bulk Message-ID: <20150519074833.7160.82332.idtracker@ietfa.amsl.com> Date: Tue, 19 May 2015 00:48:33 -0700 From: IETF Secretariat Archived-At: Subject: [tcpm] Milestones changed for tcpm WG X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 May 2015 07:48:37 -0000 Changed milestone "Submit document on CUBIC congestion control to the IESG for publication as Informational RFC", set state to active from review, accepting new milestone. URL: https://datatracker.ietf.org/wg/tcpm/charter/ From nobody Tue May 19 02:54:12 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0C4351A6FF2 for ; Tue, 19 May 2015 02:54:12 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.711 X-Spam-Level: X-Spam-Status: No, score=-0.711 tagged_above=-999 required=5 tests=[BAYES_40=-0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8ERzSoI98zPv for ; Tue, 19 May 2015 02:54:08 -0700 (PDT) Received: from hubrelay-rd.bt.com (hubrelay-rd.bt.com [62.239.224.99]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 557B31A6FFB for ; Tue, 19 May 2015 02:54:08 -0700 (PDT) Received: from EVMHR72-UKRD.domain1.systemhost.net (10.36.3.110) by EVMHR67-UKRD.bt.com (10.187.101.22) with Microsoft SMTP Server (TLS) id 8.3.348.2; Tue, 19 May 2015 10:54:06 +0100 Received: from EPHR02-UKIP.domain1.systemhost.net (147.149.100.81) by EVMHR72-UKRD.domain1.systemhost.net (10.36.3.110) with Microsoft SMTP Server (TLS) id 8.3.348.2; Tue, 19 May 2015 10:54:05 +0100 Received: from bagheera.jungle.bt.co.uk (132.146.168.158) by EPHR02-UKIP.domain1.systemhost.net (147.149.100.81) with Microsoft SMTP Server id 14.3.181.6; Tue, 19 May 2015 10:54:01 +0100 Received: from BTP075694.jungle.bt.co.uk ([10.215.131.145]) by bagheera.jungle.bt.co.uk (8.13.5/8.12.8) with ESMTP id t4J9s0ms012580; Tue, 19 May 2015 10:54:00 +0100 Message-ID: <201505190954.t4J9s0ms012580@bagheera.jungle.bt.co.uk> X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Tue, 19 May 2015 10:53:59 +0100 To: Matt Mathis , Joe Touch , "John Heffner" From: Bob Briscoe In-Reply-To: References: <201505151233.t4FCXRxW022847@bagheera.jungle.bt.co.uk> <201505161152.t4GBqCmx001121@bagheera.jungle.bt.co.uk> <555A02C6.5050100@isi.edu> <555A2E8E.4070101@isi.edu> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="=====================_1459739029==.ALT" X-Scanned-By: MIMEDefang 2.56 on 132.146.168.158 Archived-At: Cc: tcpm@ietf.org Subject: Re: [tcpm] Why is the minimum ssthresh 2*SMSS (rather than 1)? X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 May 2015 09:54:12 -0000 --=====================_1459739029==.ALT Content-Type: text/plain; charset="iso-8859-1"; format=flowed Content-Transfer-Encoding: quoted-printable Matt, Joe, John, Thx for explaining the history. I am saying what Matt just hinted at. "This decision may need to be= revisited." First, this affects well designed systems, even a=20 broadband network with a good set of local CDNs=20 and AQM installed everywhere, so most usage will=20 see a very low RTT. Not just overloaded, underprovisioned networks with no= AQM. For window, w < 2*SMSS, the sender would need to=20 pace segments at 1/w per RTT (as implemented in=20 Linux for TCP Nice). But only if it also implemented ABC. Then, at a low RTT, transmissions wouldn't be=20 driven by the delayed ACK timer, they would be=20 driven by the delayed ACK /factor/ (ie the number=20 of segments received before a delayed ACK is=20 emitted). With cwnd =3D 1, after 2 short RTTs, the=20 receiver would release an ACK, (before the DA=20 timer expires), which in turn would release two=20 segments. So it gets an average window of 1, but as 2 segments every 2 RTTs. If the min cwnd of 2*SMSS has been hit, sending=20 two segments per RTT but pushing up the queue to=20 do so, doesn't make throughput any higher. It=20 just increases the RTT for yourself and everyone else (even with an AQM). In the light of renewed interest in deploying=20 AQM, we have to review which is more important: * avoiding TCP being clocked by the delayed ACK=20 mechanism (does this make significant difference to performance?)? * or avoiding TCP artifically inflating the=20 bottleneck queue (which does make a real=20 difference - it reduces delay, for the user and everyone else)? Bob At 19:44 18/05/2015, Matt Mathis wrote: >A smidge more detail: it was understood that for=20 >efficiency w/ DA we almost always want cwnd >=3D=20 >2, however there are some corner cases=20 >(persistent timeouts due to high losses in=20 >particular) where this seems a bit unsafe.=C2 As=20 >a consequence you get to cwnd=3D1 in a few cases,=20 >but if that one packet is delivered, you=20 >immediately go to cwnd=3D2.=C2 This balance was=20 >carefully thought out at one point, but may be due to be revisited. > >Thanks, >--MM-- >The best way to predict the future is to create it. =C2 - Alan Kay > >Privacy matters!=C2 We know from recent events=20 >that people are using our services to speak in=20 >defiance of unjust governments. =C2 We treat=20 >privacy and security as matters of life and=20 >death, because for some users, they are. > >On Mon, May 18, 2015 at 11:25 AM, Joe Touch=20 ><touch@isi.edu> wrote: > > >On 5/18/2015 8:18 AM, Joe Touch wrote: > > +1 > > > > This is also why the original initial window was 2*SMSS. > >To clarify: > >=C2 =C2 =C2 =C2 the initial CWND window spec'd in RFC2001=C2 is 1 SMSS > >=C2 =C2 =C2 =C2 I recall reports when RFC2414 was being proposed >=C2 =C2 =C2 =C2 that BSD never really implemented this; they started >=C2 =C2 =C2 =C2 with 2*SMSS > >=C2 =C2 =C2 =C2 the interaction with 1*SMSS and delayed ACKs was >=C2 =C2 =C2 =C2 known back in the late 90s (Mark Allman, Sally Floyd,= et al.) > >=C2 =C2 =C2 =C2 which is why RFC2414 spec'd CWND in terms of an >=C2 =C2 =C2 =C2 even number of SMSS bounded by 4K > >In general, CWND with an odd number of SMSS is a bad idea. > >Joe > > > > > Joe > > > > On 5/18/2015 8:15 AM, John Heffner wrote: > >> With delayed ack, reducing cwnd to one breaks the ack clock. > >> Transmission instead will be driven by the delayed ack timer. > >> > >>=C2 =C2 -John > >_______________________________________________ >tcpm mailing list >tcpm@ietf.org >https://www.ietf.org/mailman/listinfo/tcpm > ________________________________________________________________ Bob Briscoe, BT=20 --=====================_1459739029==.ALT Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Matt, Joe, John,

Thx for explaining the history.

I am saying what Matt just hinted at. "This decision may need to be revisited."

First, this affects well designed systems, even a broadband network with a good set of local CDNs and AQM installed everywhere, so most usage will see a very low RTT. Not just overloaded, underprovisioned networks with no AQM.

For window, w < 2*SMSS, the sender would need to pace segments at 1/w per RTT (as implemented in Linux for TCP Nice). But only if it also implemented ABC.

Then, at a low RTT, transmissions wouldn't be driven by the delayed ACK timer, they would be driven by the delayed ACK /factor/ (ie the number of segments received before a delayed ACK is emitted). With cwnd =3D 1, after 2 short RTTs, the receiver would release an ACK, (before the DA timer expires), which in turn would release two segments. So it gets an average window of 1, but as 2 segments every 2 RTTs.

If the min cwnd of 2*SMSS has been hit, sending two segments per RTT but pushing up the queue to do so, doesn't make throughput any higher. It just increases the RTT for yourself and everyone else (even with an AQM).

In the light of renewed interest in deploying AQM, we have to review which is more important:
* avoiding TCP being clocked by the delayed ACK mechanism (does this make significant difference to performance?)?
* or avoiding TCP artifically inflating the bottleneck queue (which does make a real difference - it reduces delay, for the user and everyone else)?


Bob

At 19:44 18/05/2015, Matt Mathis wrote:
A smidge more detail: it was understood that for efficiency w/ DA we almost always want cwnd >=3D 2, however there are some corner cases (persistent timeouts due to high losses in particular) where this seems a bit unsafe.=C2  As a consequence you get to cwnd=3D1 in a few cases, but if that one packet is delivered, you immediately go to cwnd=3D2.=C2  This balance was carefully thought out at one point, but may be due to be revisited.

Thanks,
--MM--
The best way to predict the future is to create it. =C2 - Alan Kay

Privacy matters!=C2  We know from recent events that people are using our services to speak in defiance of unjust governments. =C2  We treat privacy and security as matters of life and death, because for some users, they are.

On Mon, May 18, 2015 at 11:25 AM, Joe Touch <touch@isi.edu> wrote:


On 5/18/2015 8:18 AM, Joe Touch wrote:
> +1
>
> This is also why the original initial window was 2*SMSS.

To clarify:

=C2  =C2  =C2  =C2  the initial CWND window spec'd i= n RFC2001=C2  is 1 SMSS

=C2  =C2  =C2  =C2  I recall reports when RFC2414 wa= s being proposed
=C2  =C2  =C2  =C2  that BSD never really implemente= d this; they started
=C2  =C2  =C2  =C2  with 2*SMSS

=C2  =C2  =C2  =C2  the interaction with 1*SMSS and delayed ACKs was
=C2  =C2  =C2  =C2  known back in the late 90s (Mark Allman, Sally Floyd, et al.)

=C2  =C2  =C2  =C2  which is why RFC2414 spec'd CWND= in terms of an
=C2  =C2  =C2  =C2  even number of SMSS bounded by 4K

In general, CWND with an odd number of SMSS is a bad idea.

Joe

>
> Joe
>
> On 5/18/2015 8:15 AM, John Heffner wrote:
>> With delayed ack, reducing cwnd to one breaks the ack clock.
>> Transmission instead will be driven by the delayed ack timer.
>>
>>=C2  =C2 -John

_______________________________________________
tcpm mailing list
tcpm@ietf.org
https://www.ietf.org/mailman/listinfo/tcpm

________________________________________________________________
Bob Briscoe,           &n= bsp;            =             &nbs= p;             BT --=====================_1459739029==.ALT-- From nobody Tue May 19 03:00:09 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A4D2B1A7011 for ; Tue, 19 May 2015 03:00:06 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.611 X-Spam-Level: X-Spam-Status: No, score=-2.611 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id M4zkfxusJrcp for ; Tue, 19 May 2015 03:00:05 -0700 (PDT) Received: from hubrelay-by-04.bt.com (hubrelay-by-04.bt.com [62.7.242.140]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 208441A1BC8 for ; Tue, 19 May 2015 03:00:04 -0700 (PDT) Received: from EVMHR01-UKBR.domain1.systemhost.net (193.113.108.40) by EVMHR04-UKBR.bt.com (10.216.161.36) with Microsoft SMTP Server (TLS) id 8.3.348.2; Tue, 19 May 2015 10:59:59 +0100 Received: from EPHR02-UKIP.domain1.systemhost.net (147.149.100.81) by EVMHR01-UKBR.domain1.systemhost.net (193.113.108.40) with Microsoft SMTP Server (TLS) id 8.3.348.2; Tue, 19 May 2015 11:00:02 +0100 Received: from bagheera.jungle.bt.co.uk (132.146.168.158) by EPHR02-UKIP.domain1.systemhost.net (147.149.100.81) with Microsoft SMTP Server id 14.3.181.6; Tue, 19 May 2015 10:59:59 +0100 Received: from BTP075694.jungle.bt.co.uk ([10.215.131.145]) by bagheera.jungle.bt.co.uk (8.13.5/8.12.8) with ESMTP id t4J9xwdK012669; Tue, 19 May 2015 10:59:58 +0100 Message-ID: <201505190959.t4J9xwdK012669@bagheera.jungle.bt.co.uk> X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Tue, 19 May 2015 10:59:57 +0100 To: Yuchung Cheng From: Bob Briscoe In-Reply-To: References: <201505151233.t4FCXRxW022847@bagheera.jungle.bt.co.uk> <201505161152.t4GBqCmx001121@bagheera.jungle.bt.co.uk> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-Scanned-By: MIMEDefang 2.56 on 132.146.168.158 Archived-At: Cc: tcpm IETF list Subject: Re: [tcpm] Why is the minimum ssthresh 2*SMSS (rather than 1)? X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 May 2015 10:00:06 -0000 Yuchung, At 20:07 18/05/2015, Yuchung Cheng wrote: >Bob: does reducing ssthresh lower-bound to 1 help your aqm test case? See email just sent to Joe, Matt, John. I knew we would need pacing for a sub-SMSS window, but I thought we could at least do an initial fix to reduce the min cwnd to 1*SMSS without pacing. However, people's responses have made me realise that we would need pacing in all cases when cwnd < 2*SMSS. You will know better than me whether pacing would be feasible to add. Bob ________________________________________________________________ Bob Briscoe, BT From nobody Tue May 19 05:02:10 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8292B1A8854 for ; Tue, 19 May 2015 05:02:08 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2 X-Spam-Level: X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id aegu6pY4fl1U for ; Tue, 19 May 2015 05:01:59 -0700 (PDT) Received: from mail-qk0-x231.google.com (mail-qk0-x231.google.com [IPv6:2607:f8b0:400d:c09::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 8A8B21A8852 for ; Tue, 19 May 2015 05:01:59 -0700 (PDT) Received: by qkgv12 with SMTP id v12so7492093qkg.0 for ; Tue, 19 May 2015 05:01:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=My/ykC3yvCQRUTbK+m0aVSlyr5NmxcTl7eaRqomkmcs=; b=WD5+AOUtPcOv5+FOEhtoe/7uNd31eFkhP1Ff/8bcpcjyt83tygmP8wpneAOvxI2UDy DSYZhwghnImFkgQhkLSS7mG7inuhAoq/GBD+YtirAHPJXznQrJagacB5V9XgX5w5w+H4 fLP9y+ydo17XOPsgcYFZq6FLTT/9PDKr1pNYQGXxmA2BvfKo5IhkgKtlPZ5ZAS+kM9rq /f8TiAq5BJ58g778iviiZtb02DGy4xeZusOZZh0Z5bhYgxSzCVSlkWndH7MZbpkVw/11 viOUxPSPvGcOhYXO2YapfU2DSWLXQ/Xc+HSd9yweFXHAVhQ9yo1Af0WZ11acbxW58WL8 RsEA== X-Received: by 10.140.148.76 with SMTP id 73mr2276319qhu.62.1432036918814; Tue, 19 May 2015 05:01:58 -0700 (PDT) Received: from [192.168.1.202] (c-98-236-186-123.hsd1.pa.comcast.net. [98.236.186.123]) by mx.google.com with ESMTPSA id d11sm8891281qgd.31.2015.05.19.05.01.57 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 19 May 2015 05:01:57 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (1.0) From: John Heffner X-Mailer: iPhone Mail (12F70) In-Reply-To: <201505190954.t4J9s0ms012580@bagheera.jungle.bt.co.uk> Date: Tue, 19 May 2015 08:01:56 -0400 Content-Transfer-Encoding: quoted-printable Message-Id: <2357C76B-A00C-465D-84E2-465E1AEC209D@gmail.com> References: <201505151233.t4FCXRxW022847@bagheera.jungle.bt.co.uk> <201505161152.t4GBqCmx001121@bagheera.jungle.bt.co.uk> <555A02C6.5050100@isi.edu> <555A2E8E.4070101@isi.edu> <201505190954.t4J9s0ms012580@bagheera.jungle.bt.co.uk> To: Bob Briscoe Archived-At: Cc: "" , Matt Mathis , Joe Touch Subject: Re: [tcpm] Why is the minimum ssthresh 2*SMSS (rather than 1)? X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 May 2015 12:02:08 -0000 > On May 19, 2015, at 5:53 AM, Bob Briscoe wrote: >=20 > Matt, Joe, John, >=20 > Thx for explaining the history. >=20 > I am saying what Matt just hinted at. "This decision may need to be revisi= ted." >=20 > First, this affects well designed systems, even a broadband network with a= good set of local CDNs and AQM installed everywhere, so most usage will see= a very low RTT. Not just overloaded, underprovisioned networks with no AQM.= >=20 > For window, w < 2*SMSS, the sender would need to pace segments at 1/w per R= TT (as implemented in Linux for TCP Nice). But only if it also implemented A= BC. >=20 > Then, at a low RTT, transmissions wouldn't be driven by the delayed ACK ti= mer, they would be driven by the delayed ACK /factor/ (ie the number of segm= ents received before a delayed ACK is emitted). With cwnd =3D 1, after 2 sho= rt RTTs, the receiver would release an ACK, (before the DA timer expires), w= hich in turn would release two segments. So it gets an average window of 1, b= ut as 2 segments every 2 RTTs. >=20 > If the min cwnd of 2*SMSS has been hit, sending two segments per RTT but p= ushing up the queue to do so, doesn't make throughput any higher. It just in= creases the RTT for yourself and everyone else (even with an AQM). >=20 > In the light of renewed interest in deploying AQM, we have to review which= is more important: > * avoiding TCP being clocked by the delayed ACK mechanism (does this make s= ignificant difference to performance?)? > * or avoiding TCP artifically inflating the bottleneck queue (which does m= ake a real difference - it reduces delay, for the user and everyone else)? Another degree of freedom is segment size. For instance, if the expected fa= ir share is less than 2 mtu a network operator could reduce mtu. Or tcp mig= ht try to be adaptive. -John From nobody Tue May 19 06:42:27 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CFB0A1B2EE8 for ; Tue, 19 May 2015 06:42:25 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.512 X-Spam-Level: X-Spam-Status: No, score=-2.512 tagged_above=-999 required=5 tests=[BAYES_05=-0.5, MIME_8BIT_HEADER=0.3, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id l6VwqhB6fMCP for ; Tue, 19 May 2015 06:42:21 -0700 (PDT) Received: from script.cs.helsinki.fi (script.cs.helsinki.fi [128.214.11.1]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 444751B2EE6 for ; Tue, 19 May 2015 06:40:55 -0700 (PDT) Received: from melkinpaasi.cs.helsinki.fi (melkinpaasi.cs.helsinki.fi [128.214.9.14]) (TLS: TLSv1/SSLv3,256bits,AES256-SHA) by mail.cs.helsinki.fi with ESMTPS; Tue, 19 May 2015 16:40:53 +0300 id 00000000005A1C9E.00000000555B3D65.000061E3 Date: Tue, 19 May 2015 16:40:45 +0300 (EEST) From: =?ISO-8859-15?Q?Ilpo_J=E4rvinen?= X-X-Sender: ijjarvin@melkinpaasi.cs.helsinki.fi To: John Heffner In-Reply-To: <2357C76B-A00C-465D-84E2-465E1AEC209D@gmail.com> Message-ID: References: <201505151233.t4FCXRxW022847@bagheera.jungle.bt.co.uk> <201505161152.t4GBqCmx001121@bagheera.jungle.bt.co.uk> <555A02C6.5050100@isi.edu> <555A2E8E.4070101@isi.edu> <201505190954.t4J9s0ms012580@bagheera.jungle.bt.co.uk> <2357C76B-A00C-465D-84E2-465E1AEC209D@gmail.com> User-Agent: Alpine 2.02 (DEB 1266 2009-07-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-CS-Test-DKIM: none Archived-At: Cc: Joe Touch , "" , Matt Mathis Subject: Re: [tcpm] Why is the minimum ssthresh 2*SMSS (rather than 1)? X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 May 2015 13:42:26 -0000 On Tue, 19 May 2015, John Heffner wrote: > > On May 19, 2015, at 5:53 AM, Bob Briscoe wrote: > > > > Matt, Joe, John, > > > > Thx for explaining the history. > > > > I am saying what Matt just hinted at. "This decision may need to be revisited." > > > > First, this affects well designed systems, even a broadband network with a good set of local CDNs and AQM installed everywhere, so most usage will see a very low RTT. Not just overloaded, underprovisioned networks with no AQM. > > > > For window, w < 2*SMSS, the sender would need to pace segments at 1/w per RTT (as implemented in Linux for TCP Nice). But only if it also implemented ABC. > > > > Then, at a low RTT, transmissions wouldn't be driven by the delayed ACK timer, they would be driven by the delayed ACK /factor/ (ie the number of segments received before a delayed ACK is emitted). With cwnd = 1, after 2 short RTTs, the receiver would release an ACK, (before the DA timer expires), which in turn would release two segments. So it gets an average window of 1, but as 2 segments every 2 RTTs. > > > > If the min cwnd of 2*SMSS has been hit, sending two segments per RTT but pushing up the queue to do so, doesn't make throughput any higher. It just increases the RTT for yourself and everyone else (even with an AQM). > > > > In the light of renewed interest in deploying AQM, we have to review which is more important: > > * avoiding TCP being clocked by the delayed ACK mechanism (does this make significant difference to performance?)? > > * or avoiding TCP artifically inflating the bottleneck queue (which does make a real difference - it reduces delay, for the user and everyone else)? > > Another degree of freedom is segment size. For instance, if the > expected fair share is less than 2 mtu a network operator could reduce > mtu. Or tcp might try to be adaptive. Linux actually already does segment size reduction for the cases where receiver advertized window is too small to fit enough packets. However, in case of congestion, reducing segment size works to some extent but in the end of that road lurks congestion collapse due to header overhead so it might not be wise to proceed towards that direction. Pacing, on the other hard, wouldn't impose similar utilization reductions due to overhead increase. -- i. From nobody Tue May 19 10:07:28 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9BC8B1B3138 for ; Tue, 19 May 2015 10:07:27 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.91 X-Spam-Level: X-Spam-Status: No, score=-1.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id WeEa2oTu6ULK for ; Tue, 19 May 2015 10:07:25 -0700 (PDT) Received: from nitro.isi.edu (nitro.isi.edu [128.9.208.207]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id C0BC51AD079 for ; Tue, 19 May 2015 10:07:25 -0700 (PDT) Received: from [128.9.160.252] (pen.isi.edu [128.9.160.252]) (authenticated bits=0) by nitro.isi.edu (8.13.8/8.13.8) with ESMTP id t4JH6erR011143 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Tue, 19 May 2015 10:06:40 -0700 (PDT) Message-ID: <555B6D9E.4090408@isi.edu> Date: Tue, 19 May 2015 10:06:38 -0700 From: Joe Touch User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.6.0 MIME-Version: 1.0 To: Bob Briscoe , Yuchung Cheng References: <201505151233.t4FCXRxW022847@bagheera.jungle.bt.co.uk> <201505161152.t4GBqCmx001121@bagheera.jungle.bt.co.uk> <201505190959.t4J9xwdK012669@bagheera.jungle.bt.co.uk> In-Reply-To: <201505190959.t4J9xwdK012669@bagheera.jungle.bt.co.uk> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-MailScanner-ID: t4JH6erR011143 X-ISI-4-69-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Archived-At: Cc: tcpm IETF list Subject: Re: [tcpm] Why is the minimum ssthresh 2*SMSS (rather than 1)? X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 May 2015 17:07:27 -0000 It might be useful to take a look at this too: http://www.computer.org/csdl/proceedings/icnp/1997/8061/00/80610205-abs.html On 5/19/2015 2:59 AM, Bob Briscoe wrote: > Yuchung, > > At 20:07 18/05/2015, Yuchung Cheng wrote: >> Bob: does reducing ssthresh lower-bound to 1 help your aqm test case? > > See email just sent to Joe, Matt, John. > > I knew we would need pacing for a sub-SMSS window, but I thought we > could at least do an initial fix to reduce the min cwnd to 1*SMSS > without pacing. However, people's responses have made me realise that we > would need pacing in all cases when cwnd < 2*SMSS. > > You will know better than me whether pacing would be feasible to add. > > > Bob > > > ________________________________________________________________ > Bob Briscoe, BT > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm From nobody Wed May 20 06:21:05 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 621E51A1A9D for ; Wed, 20 May 2015 06:21:04 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.701 X-Spam-Level: X-Spam-Status: No, score=0.701 tagged_above=-999 required=5 tests=[BAYES_50=0.8, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bOHSlXvVG3Fe for ; Wed, 20 May 2015 06:21:01 -0700 (PDT) Received: from mail-wg0-x22f.google.com (mail-wg0-x22f.google.com [IPv6:2a00:1450:400c:c00::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 2FAE91A1A0B for ; Wed, 20 May 2015 06:21:01 -0700 (PDT) Received: by wgfl8 with SMTP id l8so52705046wgf.2 for ; Wed, 20 May 2015 06:21:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=CGtg8CMJRSoXzR0x3pBIV9eRl8ZaSK5Is71zngDdkOU=; b=MbfxzmZNYd2dfJL4JZVtDUTNIL/yDU8tcuY+JSy2D79PE1sQzEnkMnI5r75pEK0kPw PC4lw1/Qtbnh7n5Aj2HfSNyxWj6p+My6NAzG5OGONhAwnE8cMHj0rRBzy2fnAbhiy4Gh nBh9+tShcapImBZtGyRYLcqyaxOkxGlaJ6WACnkb85sLlFqo6dY6PYsgQ+XcxnwRMkyF DU0DkeYEwwYwSf4Th35D7vkOTPz5QwEePu4HjQrUrkmZtUd/C3Q0uhnKUy5XSpfyHgpy VTL0v4EDB52+wjk2q/Huo1P0wVOfBOZ2jXmtPoIH/tJN4DVjijbF535wu5kdaI4beWHW /zXQ== MIME-Version: 1.0 X-Received: by 10.194.95.41 with SMTP id dh9mr65685915wjb.55.1432128060004; Wed, 20 May 2015 06:21:00 -0700 (PDT) Received: by 10.28.133.148 with HTTP; Wed, 20 May 2015 06:20:59 -0700 (PDT) Date: Wed, 20 May 2015 18:50:59 +0530 Message-ID: From: Tom Sanders To: tcpm@ietf.org Content-Type: multipart/alternative; boundary=047d7bea453c4a1c850516834c84 Archived-At: Subject: [tcpm] Why is Cubic said to be RTT-fair? X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 May 2015 13:21:04 -0000 --047d7bea453c4a1c850516834c84 Content-Type: text/plain; charset=UTF-8 Hi, As per my understanding, unlike other TCP variants, CUBIC claims to be RTT fair. This is because the window growth rate is independent of the RTT, and hence the performance is the same as on a low and a high RTT paths. Others are dependent upon RTT since their windows grow as they receive ACKs from their peers, while CUBIC doesnt (window growth is a function of time). This is theory. In practice, CUBIC (which is the default now on most linux distributions) also depends upon RTT. I connected to machines on a LAN and did a FTP transfer. I got a certain bandwidth. Next, i artificially inserted delay using "tc qdisc" in the path. I increased the delay on the ethernet interface connecting my linux machine to the LAN to be around 100ms. I immediately saw that the TCP performance went down. To bring it up to the same level as before i had to use multiple TCP streams. To me this means that CUBIC performance is dependent of the RTT. So why do we call it RTT-fair? Thanks, Toms --047d7bea453c4a1c850516834c84 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

Hi,

As per my understanding, unlike other TC= P variants, CUBIC claims to be RTT fair. This is because the window growth = rate is independent of the RTT, and hence the performance is the same as on= a low and a high RTT paths. Others are dependent upon RTT since their wind= ows grow as they receive ACKs from their peers, while CUBIC doesnt (window = growth is a function of time). This is theory.

In practice, CUBIC (w= hich is the default now on most linux distributions) also depends upon RTT.= I connected to machines on a LAN and did a FTP transfer. I got a certain b= andwidth. Next, i artificially inserted delay using "tc qdisc" in= the path. I increased the delay on the ethernet interface connecting my li= nux machine to the LAN to be around 100ms. I immediately saw that the TCP p= erformance went down.

To bring it up to the same level as before i h= ad to use multiple TCP streams. To me this means that CUBIC performance is = dependent of the RTT. So why do we call it RTT-fair?

Tha= nks, Toms
--047d7bea453c4a1c850516834c84-- From nobody Wed May 20 07:22:09 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C120A1A7018 for ; Wed, 20 May 2015 07:22:07 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 4.789 X-Spam-Level: **** X-Spam-Status: No, score=4.789 tagged_above=-999 required=5 tests=[BAYES_50=0.8, FREEMAIL_FROM=0.001, FRT_STOCK2=3.988, MALFORMED_FREEMAIL=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6IhyqzRfUd-B for ; Wed, 20 May 2015 07:22:07 -0700 (PDT) Received: from st11p05im-asmtp002.me.com (st11p05im-asmtp002.me.com [17.172.109.150]) (using TLSv1.2 with cipher DHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 9766B1A7014 for ; Wed, 20 May 2015 07:22:06 -0700 (PDT) Received: from [192.168.1.65] (76-220-59-128.lightspeed.sntcca.sbcglobal.net [76.220.59.128]) by st11p05im-asmtp002.me.com (Oracle Communications Messaging Server 7.0.5.35.0 64bit (built Dec 4 2014)) with ESMTPSA id <0NON01CZ7JWEBV10@st11p05im-asmtp002.me.com> for tcpm@ietf.org; Wed, 20 May 2015 14:21:55 +0000 (GMT) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.14.151,1.0.33,0.0.0000 definitions=2015-05-20_04:2015-05-19,2015-05-20,1970-01-01 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 suspectscore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=7.0.1-1412110000 definitions=main-1505200189 Content-type: text/plain; charset=windows-1252 MIME-version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) From: Rick Jones In-reply-to: Date: Wed, 20 May 2015 07:21:49 -0700 Content-transfer-encoding: quoted-printable Message-id: <7C415879-6BF7-403D-B904-CC2B4D32CA04@mac.com> References: To: Tom Sanders X-Mailer: Apple Mail (2.1878.6) Archived-At: Cc: tcpm@ietf.org Subject: Re: [tcpm] Why is Cubic said to be RTT-fair? X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 May 2015 14:22:07 -0000 On May 20, 2015, at 6:20 AM, Tom Sanders wrote: > Hi, >=20 > As per my understanding, unlike other TCP variants, CUBIC claims to be = RTT fair. This is because the window growth rate is independent of the = RTT, and hence the performance is the same as on a low and a high RTT = paths. Others are dependent upon RTT since their windows grow as they = receive ACKs from their peers, while CUBIC doesnt (window growth is a = function of time). This is theory. >=20 > In practice, CUBIC (which is the default now on most linux = distributions) also depends upon RTT. I connected to machines on a LAN = and did a FTP transfer. I got a certain bandwidth. Next, i artificially = inserted delay using "tc qdisc" in the path. I increased the delay on = the ethernet interface connecting my linux machine to the LAN to be = around 100ms. I immediately saw that the TCP performance went down. >=20 > To bring it up to the same level as before i had to use multiple TCP = streams. To me this means that CUBIC performance is dependent of the = RTT. So why do we call it RTT-fair? I cannot speak to CUBIC specifically, but I would think you would want = to make sure you were seeing effects of congestion window and not of the = classic TCP window. For example, if the FTP client/server you were = using makes an explicit setsockopt() to set the socket buffer sizes and = by extension the classic TCP window, your bumping of the RTT to around = 100ms may have taken the bandwidth down as a consequence of the classic: Throughput <=3D WindowSize/RTT Similarly, since you mention Linux, if the FTP client/server didn=92t = make explicit setsockopt() calls, the sysctl values for tcp_rmem and = tcp_wmem may have been such that the auto tuning of the window size = couldn=92t allow the classic TCP window to grow =93enough.=94 I suspect that the experiment needs to be setup to have the same classic = window size in both cases, sized per the above to be sufficient to = achieve the peak throughput in the high RTT case. Then you will have = eliminated classic window as a factor. Also, depending on the version of the Linux kernel you are using, there = may be some effects from tcp small queues or perhaps (a stretch?) even = byte queue limits which may not be playing well with netem. (Just = guessing there really) happy benchmarking, rick jones= From nobody Wed May 20 10:18:36 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D40A11A8996 for ; Wed, 20 May 2015 10:18:34 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 4.689 X-Spam-Level: **** X-Spam-Status: No, score=4.689 tagged_above=-999 required=5 tests=[BAYES_50=0.8, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, FRT_STOCK2=3.988, HTML_MESSAGE=0.001, SPF_PASS=-0.001] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id YT25u9kn1VX0 for ; Wed, 20 May 2015 10:18:33 -0700 (PDT) Received: from mail-wi0-x232.google.com (mail-wi0-x232.google.com [IPv6:2a00:1450:400c:c05::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 39B751A898D for ; Wed, 20 May 2015 10:18:33 -0700 (PDT) Received: by wicmc15 with SMTP id mc15so156530675wic.1 for ; Wed, 20 May 2015 10:18:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=lbBvLCod81jjCGJkHfxPQkafSoRiYiAOlP9gNRXxwyc=; b=S4SZfLLx2DttSJtzmsZDI3juZA7vf0T/pLzU07+niDqWupouzexoW7/aLQQUNTMnPr pgjIvvwOWe1aq8Nha/oaPY3HVVCpm1gFg4ofNKQbZOm6vsLe4WJB0soWHHBp1aG3H8c7 OoSWlujZnhuOAqUZUZYwN73MUqUEtwsQRPB2xvOsJVgyT141K3g0xv5Fx2mTf6syADvW j86q10RaSdK40mS9E+NuHjV1GiLBhdBNc3abWeG/O0fDO1rNGCDKLHuinbvl1wS3SeeL TkNJaND32Ik6CG2YFHbkMifB3wT6OjhDy671+rheOUvGCxpu90abzF3CENqkRzHXO38b 2Hfg== MIME-Version: 1.0 X-Received: by 10.180.188.13 with SMTP id fw13mr42739204wic.89.1432142311992; Wed, 20 May 2015 10:18:31 -0700 (PDT) Received: by 10.28.133.148 with HTTP; Wed, 20 May 2015 10:18:31 -0700 (PDT) In-Reply-To: <7C415879-6BF7-403D-B904-CC2B4D32CA04@mac.com> References: <7C415879-6BF7-403D-B904-CC2B4D32CA04@mac.com> Date: Wed, 20 May 2015 22:48:31 +0530 Message-ID: From: Tom Sanders To: Rick Jones Content-Type: multipart/alternative; boundary=001a11c26140c632fd0516869dad Archived-At: Cc: tcpm@ietf.org Subject: Re: [tcpm] Why is Cubic said to be RTT-fair? X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 May 2015 17:18:35 -0000 --001a11c26140c632fd0516869dad Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Thanks Rick, this was very helpful. So if there are no setsockopts, etc then CUBIC's performance, unlike other TCP variants, should not get impacted by RTTs, right? Is that a fair statement to make? Toms On 20 May 2015 at 19:51, Rick Jones wrote: > > On May 20, 2015, at 6:20 AM, Tom Sanders wrote: > > > Hi, > > > > As per my understanding, unlike other TCP variants, CUBIC claims to be > RTT fair. This is because the window growth rate is independent of the RT= T, > and hence the performance is the same as on a low and a high RTT paths. > Others are dependent upon RTT since their windows grow as they receive AC= Ks > from their peers, while CUBIC doesnt (window growth is a function of time= ). > This is theory. > > > > In practice, CUBIC (which is the default now on most linux > distributions) also depends upon RTT. I connected to machines on a LAN an= d > did a FTP transfer. I got a certain bandwidth. Next, i artificially > inserted delay using "tc qdisc" in the path. I increased the delay on the > ethernet interface connecting my linux machine to the LAN to be around > 100ms. I immediately saw that the TCP performance went down. > > > > To bring it up to the same level as before i had to use multiple TCP > streams. To me this means that CUBIC performance is dependent of the RTT. > So why do we call it RTT-fair? > > I cannot speak to CUBIC specifically, but I would think you would want to > make sure you were seeing effects of congestion window and not of the > classic TCP window. For example, if the FTP client/server you were using > makes an explicit setsockopt() to set the socket buffer sizes and by > extension the classic TCP window, your bumping of the RTT to around 100ms > may have taken the bandwidth down as a consequence of the classic: > > Throughput <=3D WindowSize/RTT > > Similarly, since you mention Linux, if the FTP client/server didn=E2=80= =99t make > explicit setsockopt() calls, the sysctl values for tcp_rmem and tcp_wmem > may have been such that the auto tuning of the window size couldn=E2=80= =99t allow > the classic TCP window to grow =E2=80=9Cenough.=E2=80=9D > > I suspect that the experiment needs to be setup to have the same classic > window size in both cases, sized per the above to be sufficient to achiev= e > the peak throughput in the high RTT case. Then you will have eliminated > classic window as a factor. > > Also, depending on the version of the Linux kernel you are using, there > may be some effects from tcp small queues or perhaps (a stretch?) even by= te > queue limits which may not be playing well with netem. (Just guessing > there really) > > happy benchmarking, > > rick jones --=20 Toms. --001a11c26140c632fd0516869dad Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Thanks Rick, this was very helpful.

So = if there are no setsockopts, etc then CUBIC's performance, unlike other= TCP variants, should not get impacted by RTTs, right? Is that a fair state= ment to make?

Toms
=
On 20 May 2015 at 19:51, Rick Jones <perfgee= k@mac.com> wrote:

On May 20, 2015, at 6:20 AM, Tom Sanders <toms.sanders@gmail.com> wrote:

> Hi,
>
> As per my understanding, unlike other TCP variants, CUBIC claims to be= RTT fair. This is because the window growth rate is independent of the RTT= , and hence the performance is the same as on a low and a high RTT paths. O= thers are dependent upon RTT since their windows grow as they receive ACKs = from their peers, while CUBIC doesnt (window growth is a function of time).= This is theory.
>
> In practice, CUBIC (which is the default now on most linux distributio= ns) also depends upon RTT. I connected to machines on a LAN and did a FTP t= ransfer. I got a certain bandwidth. Next, i artificially inserted delay usi= ng "tc qdisc" in the path. I increased the delay on the ethernet = interface connecting my linux machine to the LAN to be around 100ms. I imme= diately saw that the TCP performance went down.
>
> To bring it up to the same level as before i had to use multiple TCP s= treams. To me this means that CUBIC performance is dependent of the RTT. So= why do we call it RTT-fair?

I cannot speak to CUBIC specifically, but I would think you would want to m= ake sure you were seeing effects of congestion window and not of the classi= c TCP window.=C2=A0 For example, if the FTP client/server you were using ma= kes an explicit setsockopt() to set the socket buffer sizes and by extensio= n the classic TCP window, your bumping of the RTT to around 100ms may have = taken the bandwidth down as a consequence of the classic:

Throughput <=3D WindowSize/RTT

Similarly, since you mention Linux, if the FTP client/server didn=E2=80=99t= make explicit setsockopt() calls,=C2=A0 the sysctl values for tcp_rmem and= tcp_wmem may have been such that the auto tuning of the window size couldn= =E2=80=99t allow the classic TCP window to grow =E2=80=9Cenough.=E2=80=9D
I suspect that the experiment needs to be setup to have the same classic wi= ndow size in both cases, sized per the above to be sufficient to achieve th= e peak throughput in the high RTT case.=C2=A0 Then you will have eliminated= classic window as a factor.

Also, depending on the version of the Linux kernel you are using, there may= be some effects from tcp small queues or perhaps (a stretch?) even byte qu= eue limits which may not be playing well with netem.=C2=A0 (Just guessing t= here really)

happy benchmarking,

rick jones



--
Toms.
--001a11c26140c632fd0516869dad-- From nobody Wed May 20 10:39:30 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1B3391A89B9 for ; Wed, 20 May 2015 10:39:29 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 2.478 X-Spam-Level: ** X-Spam-Status: No, score=2.478 tagged_above=-999 required=5 tests=[BAYES_50=0.8, FRT_STOCK2=3.988, RCVD_IN_DNSWL_MED=-2.3, T_RP_MATCHES_RCVD=-0.01] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id exFkxnl3jGcf for ; Wed, 20 May 2015 10:39:27 -0700 (PDT) Received: from g4t3427.houston.hp.com (g4t3427.houston.hp.com [15.201.208.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 67FED1A89A8 for ; Wed, 20 May 2015 10:39:27 -0700 (PDT) Received: from g9t2301.houston.hp.com (g9t2301.houston.hp.com [16.216.185.78]) by g4t3427.houston.hp.com (Postfix) with ESMTP id A099EEC; Wed, 20 May 2015 17:39:26 +0000 (UTC) Received: from [16.103.148.51] (tardy.usa.hp.com [16.103.148.51]) by g9t2301.houston.hp.com (Postfix) with ESMTP id 70CD870; Wed, 20 May 2015 17:39:26 +0000 (UTC) Message-ID: <555CC6CC.4010109@hp.com> Date: Wed, 20 May 2015 10:39:24 -0700 From: Rick Jones User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.6.0 MIME-Version: 1.0 To: tcpm@ietf.org, toms.sanders@gmail.com References: <7C415879-6BF7-403D-B904-CC2B4D32CA04@mac.com> In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit Archived-At: Subject: Re: [tcpm] Why is Cubic said to be RTT-fair? X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 May 2015 17:39:29 -0000 On 05/20/2015 10:18 AM, Tom Sanders wrote: > Thanks Rick, this was very helpful. > > So if there are no setsockopts, etc then CUBIC's performance, unlike > other TCP variants, should not get impacted by RTTs, right? Is that a > fair statement to make? Like I said, I cannot speak to CUBIC specifically. But, since it is possible to alter the default congestion control under Linux, if you have a sufficiently stable test environment, you could run the different congestion controls through your added delay and see what you get. I would suggest setting the likes of net.core.[rw]mem_max to a value large enough to allow for "link rate" for your physical network even at 100ms of RTT, and then run the likes of either iperf or netperf (*) with them making the setsockopt()s to that value to eliminate concerns about sufficient classic TCP window and/or Linux's autotuning. Lather, rinse, repeat with the different congestion controls set to default via net.ipv4.tcp_congestion_control or if using netperf there is the test-specific -K option to specify the congestion control to use (assuming that functionality hasn't bitrotted) I believe that iperf can be told to emit interim results, which would allow you to see (perhaps) how things "ramp-up" - there is a similar, optional feature of netperf which will cause it to emit interim results at a specified cadence. Or you could just take packet traces and run them through the likes of tcptrace to get pretty pictures that way. rick (*) rather than FTP or some other file transfer because you want to eliminate filesystem effects > > Toms > > On 20 May 2015 at 19:51, Rick Jones > wrote: > > > On May 20, 2015, at 6:20 AM, Tom Sanders > wrote: > > > Hi, > > > > As per my understanding, unlike other TCP variants, CUBIC claims > to be RTT fair. This is because the window growth rate is > independent of the RTT, and hence the performance is the same as on > a low and a high RTT paths. Others are dependent upon RTT since > their windows grow as they receive ACKs from their peers, while > CUBIC doesnt (window growth is a function of time). This is theory. > > > > In practice, CUBIC (which is the default now on most linux > distributions) also depends upon RTT. I connected to machines on a > LAN and did a FTP transfer. I got a certain bandwidth. Next, i > artificially inserted delay using "tc qdisc" in the path. I > increased the delay on the ethernet interface connecting my linux > machine to the LAN to be around 100ms. I immediately saw that the > TCP performance went down. > > > > To bring it up to the same level as before i had to use multiple > TCP streams. To me this means that CUBIC performance is dependent of > the RTT. So why do we call it RTT-fair? > > I cannot speak to CUBIC specifically, but I would think you would > want to make sure you were seeing effects of congestion window and > not of the classic TCP window. For example, if the FTP > client/server you were using makes an explicit setsockopt() to set > the socket buffer sizes and by extension the classic TCP window, > your bumping of the RTT to around 100ms may have taken the bandwidth > down as a consequence of the classic: > > Throughput <= WindowSize/RTT > > Similarly, since you mention Linux, if the FTP client/server didn’t > make explicit setsockopt() calls, the sysctl values for tcp_rmem > and tcp_wmem may have been such that the auto tuning of the window > size couldn’t allow the classic TCP window to grow “enough.” > > I suspect that the experiment needs to be setup to have the same > classic window size in both cases, sized per the above to be > sufficient to achieve the peak throughput in the high RTT case. > Then you will have eliminated classic window as a factor. > > Also, depending on the version of the Linux kernel you are using, > there may be some effects from tcp small queues or perhaps (a > stretch?) even byte queue limits which may not be playing well with > netem. (Just guessing there really) > > happy benchmarking, > > rick jones > > > > > -- > Toms. > > > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm > From nobody Wed May 20 17:51:45 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 93F1A1ACDF5 for ; Wed, 20 May 2015 17:51:44 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 4.689 X-Spam-Level: **** X-Spam-Status: No, score=4.689 tagged_above=-999 required=5 tests=[BAYES_50=0.8, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, FRT_STOCK2=3.988, HTML_MESSAGE=0.001, SPF_PASS=-0.001] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qQTr7RRpBeVJ for ; Wed, 20 May 2015 17:51:42 -0700 (PDT) Received: from mail-wi0-x22e.google.com (mail-wi0-x22e.google.com [IPv6:2a00:1450:400c:c05::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 019EB1ACDE9 for ; Wed, 20 May 2015 17:51:42 -0700 (PDT) Received: by wicmx19 with SMTP id mx19so172755786wic.0 for ; Wed, 20 May 2015 17:51:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=NXydqlcPGwoQoEv3cxhTRg++sfpEaNqdhajZnu3vR+E=; b=skln/hqTQNHIbLvaVI4Ad4zbXWONL9LwKXtHjHdZlimlG86sPDdcvc1JeXcbbNPViQ 9aFUhpaHP96AgkC4B3CmTE65f5+BUm1j/jIQHHj7HlQ2diz68n/MHk8ElzzRQbbCrvKT xWINjcamxznjXBKIrscMd0PT620nbletGVrbGu3iNwjUw4jnIlU8RfTaoU37KO1ZccId De4B821eGqJUTKq18pIQ2MWjku17zwLzQ2jOSRP2OH8fOXQoyVrqsOHDroSl0wRWs4X8 IwGA5ztcXQ3Np4h7ou2ZNlNsDkE6ikGCw/c+r/MU41ZLs6Owi9dR4UCiVnK/Q/o72IDk ws1Q== MIME-Version: 1.0 X-Received: by 10.194.95.41 with SMTP id dh9mr168026wjb.55.1432169500749; Wed, 20 May 2015 17:51:40 -0700 (PDT) Received: by 10.28.133.148 with HTTP; Wed, 20 May 2015 17:51:40 -0700 (PDT) In-Reply-To: References: <7C415879-6BF7-403D-B904-CC2B4D32CA04@mac.com> Date: Thu, 21 May 2015 06:21:40 +0530 Message-ID: From: Tom Sanders To: Neal Cardwell Content-Type: multipart/alternative; boundary=047d7bea453c59b49a05168cf2fc Archived-At: Cc: "tcpm@ietf.org" Subject: Re: [tcpm] Why is Cubic said to be RTT-fair? X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 21 May 2015 00:51:44 -0000 --047d7bea453c59b49a05168cf2fc Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Hi Neal, This is odd. In the original Cubic paper ( http://www4.ncsu.edu/~rhee/export/bitcp/cubic-paper.pdf) by Rhee and Xu, they explicitly state that the Cubic is independent of RTT. "The main feature of CUBIC is that its window growth function is defined in real-time so that its growth will be independent of RTT. Our work was partially inspired by HTCP [5], whose window growth function is also based on real time. The congestion epoch period of CUBIC is determined by the packet loss rate alone. As TCP=E2=80=99s throughput is defined by the packe= t loss rate as well as RTT, the throughput of CUBIC is defined by only the packet loss rate. Thus, when the loss rate is high and/or RTT is short, CUBIC can operate in a TCP mode. Moreover, since the growth function is independent of RTT, its RTT fairness is guaranteed as different RTT flows will still grow their windows at the same rate." Am i missing something here? Thanks, Toms On 20 May 2015 at 23:13, Neal Cardwell wrote: > On Wed, May 20, 2015 at 1:18 PM, Tom Sanders > wrote: > > Thanks Rick, this was very helpful. > > > > So if there are no setsockopts, etc then CUBIC's performance, unlike > other > > TCP variants, should not get impacted by RTTs, right? Is that a fair > > statement to make? > > CUBIC, like most TCP congestion control variants, has significant > aspects that evolve on the scale of RTTs: for example, in slow start, > loss recovery, or when the CUBIC function is very steep, CUBIC's > behavior is heavily dependent on RTT (send some packets, wait for an > ACK, send some more packets,...). So CUBIC will not be > perfectly"RTT-fair. > > I think it might be fair to say something like: once a path is > saturated with CUBIC flows, as long as loss recovery or send/receive > buffers are not a bottleneck, then competition between CUBIC flows > should be more RTT-fair than Reno, in the long run, since for large > parts of the life cycle of the CUBIC flow the evolution of cwnd is > constrained by wall clock time rather than ACK arrival. > > neal > > > > Toms > > > > On 20 May 2015 at 19:51, Rick Jones wrote: > >> > >> > >> On May 20, 2015, at 6:20 AM, Tom Sanders > wrote: > >> > >> > Hi, > >> > > >> > As per my understanding, unlike other TCP variants, CUBIC claims to = be > >> > RTT fair. This is because the window growth rate is independent of > the RTT, > >> > and hence the performance is the same as on a low and a high RTT > paths. > >> > Others are dependent upon RTT since their windows grow as they > receive ACKs > >> > from their peers, while CUBIC doesnt (window growth is a function of > time). > >> > This is theory. > >> > > >> > In practice, CUBIC (which is the default now on most linux > >> > distributions) also depends upon RTT. I connected to machines on a > LAN and > >> > did a FTP transfer. I got a certain bandwidth. Next, i artificially > inserted > >> > delay using "tc qdisc" in the path. I increased the delay on the > ethernet > >> > interface connecting my linux machine to the LAN to be around 100ms.= I > >> > immediately saw that the TCP performance went down. > >> > > >> > To bring it up to the same level as before i had to use multiple TCP > >> > streams. To me this means that CUBIC performance is dependent of the > RTT. So > >> > why do we call it RTT-fair? > >> > >> I cannot speak to CUBIC specifically, but I would think you would want > to > >> make sure you were seeing effects of congestion window and not of the > >> classic TCP window. For example, if the FTP client/server you were > using > >> makes an explicit setsockopt() to set the socket buffer sizes and by > >> extension the classic TCP window, your bumping of the RTT to around > 100ms > >> may have taken the bandwidth down as a consequence of the classic: > >> > >> Throughput <=3D WindowSize/RTT > >> > >> Similarly, since you mention Linux, if the FTP client/server didn=E2= =80=99t make > >> explicit setsockopt() calls, the sysctl values for tcp_rmem and > tcp_wmem > >> may have been such that the auto tuning of the window size couldn=E2= =80=99t > allow > >> the classic TCP window to grow =E2=80=9Cenough.=E2=80=9D > >> > >> I suspect that the experiment needs to be setup to have the same class= ic > >> window size in both cases, sized per the above to be sufficient to > achieve > >> the peak throughput in the high RTT case. Then you will have eliminat= ed > >> classic window as a factor. > >> > >> Also, depending on the version of the Linux kernel you are using, ther= e > >> may be some effects from tcp small queues or perhaps (a stretch?) even > byte > >> queue limits which may not be playing well with netem. (Just guessing > there > >> really) > >> > >> happy benchmarking, > >> > >> rick jones > > > > > > > > > > -- > > Toms. > > > > _______________________________________________ > > tcpm mailing list > > tcpm@ietf.org > > https://www.ietf.org/mailman/listinfo/tcpm > > > --=20 Toms. --047d7bea453c59b49a05168cf2fc Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Hi Neal,

This is odd.

In the original Cubic paper (http://www4.ncsu.edu/~rhee/export/bitcp/cubic= -paper.pdf) by Rhee and Xu, they explicitly state that the Cubic is ind= ependent of RTT.

"The main feature of CUBIC is that its window growth function is defined in real-time so that its growth will be independent of RTT. Our work was partially inspired by HTCP [5], whose window growth function is also based on real time. The congestion epoch period of CUBIC is determined by the packet loss rate alone. As TCP=E2=80=99s throughput is defined by the packet loss rate as well as RTT, the throughput of CUBIC is defined by only the packet loss rate. Thus, when the loss rate is high and/or RTT is short, CUBIC can operate in a TCP mode. Moreover, since the growth function is independent of RTT, its RTT fairness is guaranteed as different RTT flows will still grow their windows at the same rate."

Am i missing something her= e?

Thanks, Toms

On 20 May 2015 at 23:13, Neal Cardwell <= span dir=3D"ltr"><ncardwell@google.com> wrote:
On Wed, May 20, 2015 at 1:18 PM, Tom Sanders <toms.sanders@gmail.com> wrote= :
> Thanks Rick, this was very helpful.
>
> So if there are no setsockopts, etc then CUBIC's performance, unli= ke other
> TCP variants, should not get impacted by RTTs, right? Is that a fair > statement to make?

CUBIC, like most TCP congestion control variants, has significant aspects that evolve on the scale of RTTs: for example, in slow start,
loss recovery, or when the CUBIC function is very steep, CUBIC's
behavior is heavily dependent on RTT (send some packets, wait for an
ACK, send some more packets,...). So CUBIC will not be
perfectly"RTT-fair.

I think it might be fair to say something like: once a path is
saturated with CUBIC flows, as long as loss recovery or send/receive
buffers are not a bottleneck, then competition between CUBIC flows
should be more RTT-fair than Reno, in the long run, since for large
parts of the life cycle of the CUBIC flow the evolution of cwnd is
constrained by wall clock time rather than ACK arrival.

neal


> Toms
>
> On 20 May 2015 at 19:51, Rick Jones <perfgeek@mac.com> wrote:
>>
>>
>> On May 20, 2015, at 6:20 AM, Tom Sanders <toms.sanders@gmail.com> wrote:
>>
>> > Hi,
>> >
>> > As per my understanding, unlike other TCP variants, CUBIC cla= ims to be
>> > RTT fair. This is because the window growth rate is independe= nt of the RTT,
>> > and hence the performance is the same as on a low and a high = RTT paths.
>> > Others are dependent upon RTT since their windows grow as the= y receive ACKs
>> > from their peers, while CUBIC doesnt (window growth is a func= tion of time).
>> > This is theory.
>> >
>> > In practice, CUBIC (which is the default now on most linux >> > distributions) also depends upon RTT. I connected to machines= on a LAN and
>> > did a FTP transfer. I got a certain bandwidth. Next, i artifi= cially inserted
>> > delay using "tc qdisc" in the path. I increased the= delay on the ethernet
>> > interface connecting my linux machine to the LAN to be around= 100ms. I
>> > immediately saw that the TCP performance went down.
>> >
>> > To bring it up to the same level as before i had to use multi= ple TCP
>> > streams. To me this means that CUBIC performance is dependent= of the RTT. So
>> > why do we call it RTT-fair?
>>
>> I cannot speak to CUBIC specifically, but I would think you would = want to
>> make sure you were seeing effects of congestion window and not of = the
>> classic TCP window.=C2=A0 For example, if the FTP client/server yo= u were using
>> makes an explicit setsockopt() to set the socket buffer sizes and = by
>> extension the classic TCP window, your bumping of the RTT to aroun= d 100ms
>> may have taken the bandwidth down as a consequence of the classic:=
>>
>> Throughput <=3D WindowSize/RTT
>>
>> Similarly, since you mention Linux, if the FTP client/server didn= =E2=80=99t make
>> explicit setsockopt() calls,=C2=A0 the sysctl values for tcp_rmem = and tcp_wmem
>> may have been such that the auto tuning of the window size couldn= =E2=80=99t allow
>> the classic TCP window to grow =E2=80=9Cenough.=E2=80=9D
>>
>> I suspect that the experiment needs to be setup to have the same c= lassic
>> window size in both cases, sized per the above to be sufficient to= achieve
>> the peak throughput in the high RTT case.=C2=A0 Then you will have= eliminated
>> classic window as a factor.
>>
>> Also, depending on the version of the Linux kernel you are using, = there
>> may be some effects from tcp small queues or perhaps (a stretch?) = even byte
>> queue limits which may not be playing well with netem.=C2=A0 (Just= guessing there
>> really)
>>
>> happy benchmarking,
>>
>> rick jones
>
>
>
>
> --
> Toms.
>
> __________________= _____________________________
> tcpm mailing list
> tcpm@ietf.org
> https://www.ietf.org/mailman/listinfo/tcpm
>



--
=
Toms.
--047d7bea453c59b49a05168cf2fc-- From nobody Thu May 21 02:43:43 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3793B1A92E3 for ; Thu, 21 May 2015 02:43:43 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.15 X-Spam-Level: X-Spam-Status: No, score=-1.15 tagged_above=-999 required=5 tests=[BAYES_50=0.8, HELO_EQ_SE=0.35, RCVD_IN_DNSWL_MED=-2.3] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id F0jQJot3WJYg for ; Thu, 21 May 2015 02:43:40 -0700 (PDT) Received: from tiger.dc.kau.se (smtp.kau.se [193.10.220.38]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 6EE9F1A8AE8 for ; Thu, 21 May 2015 02:43:40 -0700 (PDT) X-Spam-Processed: mail.kau.se, Thu, 21 May 2015 11:43:22 +0200 (not processed: spam filter heuristic analysis disabled) X-MDRemoteIP: 193.11.155.148 X-MDArrival-Date: Thu, 21 May 2015 11:43:22 +0200 X-Authenticated-Sender: anna.brunstrom@kau.se X-Return-Path: anna.brunstrom@kau.se X-Envelope-From: anna.brunstrom@kau.se Message-ID: <555DA8BE.7040908@kau.se> Date: Thu, 21 May 2015 11:43:26 +0200 From: Anna Brunstrom User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Nicolas KUHN References: <882B8BE8-D02D-4E46-90B0-FE9CB2E88BF9@telecom-bretagne.eu> In-Reply-To: <882B8BE8-D02D-4E46-90B0-FE9CB2E88BF9@telecom-bretagne.eu> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Archived-At: Cc: "tcpm@ietf.org" Subject: Re: [tcpm] draft-ietf-tcpm-rtorestart-07 X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 21 May 2015 09:43:43 -0000 Hi Nicolas, Thanks a lot for your comments! See inline for how we suggest to address them. On 2015-05-07 16:59, Nicolas Kuhn wrote: > Hi all, > > I had a look at this draft-ietf-tcpm-rtorestart-07 draft. > I have some minor comments on the document. > > ############################################################# > > - "The RTO Restart (RTOR) mechanism described in this document makes the > RTO slightly more aggressive when the number of outstanding segments > is too small for fast retransmit to work, in an attempt to enable > faster loss recovery for all segments while being robust to > reordering." > > The RTO is not aggressive ; but the retransmission scheme ruled by RTO expirations is. > I would recommend to rephrase that. We will change the wording to clarify that the effective RTO becomes more aggressive, not the actual RTO calculation. The retransmission scheme does not change. We will apply this clarification also in other parts of the document. > > ############################################################# > > - “ The RTO Restart (RTOR) mechanism described in this document makes the > RTO slightly [â€Ĥ] highly varying RTTs, e.g. mobile networks." > > This whole paragraph comes to early in the document, as the reader has no clear > idea about what RTOR is actually doing. I propose to add that in a dedicated sub- > section in Section 5. > > ############################################################# The aim of the paragraph is to introduce both possible gains and drawbacks with the scheme early in the document, and then address these in more detail later, but we agree that it contains too much detail at this point in the document. As part of the material in this paragraph has been inserted as a result of earlier reviews our suggested resolution is to not completely remove the paragraph, but to compress it and remove details that are hard to understand without knowing the details of the algorithm. > > - “ 3 RTO Restart Overview” -> should it not be “3- RTO Overview and rationale of RTOR “ or something equivalent ? > It is confusing, as you do not actually present RTOR. Yes, your proposed title is better and we will change the title accordingly. > > Also, in this section 3: > The first paragraph finishes with “Hence, in most situations, the time before a retransmission is triggered is equal to "RTO + RTT”.” > But the example to illustrate it comes after - I think things should moved around to: > 1- introduce the classic RTO > 2- explain the rationale of classic RTO > 3- example to show that RTO+RTT is common > 4- more parameters could affect that (2 outstanding segments; other factors) > 5- conclusion on “hence, RTO+RTT is common for application limited traffic” > > I would propose to introduce a ToC for this section 3: > > 3- RTO Overview and rationale of RTOR > 3-1. Experienced RTO is RTO+RTT > 3-2. RTO safety margin > 3-3. Rationale of RTOR: when fast retransmit is not possible > We will try to rearrange the material in line with your proposed outline. However as the section is not so long, we think that explicit sections may not be needed. > ############################################################# > > Section 4 : should focus on the description of the algorithm only. > > I would keep > “ > This approach makes the RTO more > aggressive than the standardized approach in [RFC6298] but still > conforms to the requirement in [RFC6298] that segments must not be > retransmitted earlier than RTO seconds after their original > transmission. > " > > for the discussion section (at this stage of the reading, the reader still does not know exactly what RTOR is actually doing). > Agreed, we will move the text. > ############################################################# > > Section 4: "it will prevent RTOR > from being more aggressive and potentially causing RTOs instead of > fast retransmits.” > > More aggressive than what ? Has the "four" being experimentally obtained ? > I think more discussion is required on this parameter. > This value was chosen to only enable RTOR when the use of fast retransmit for loss recovery isn’t possible. We will try and clarify this further in the text. Unless you (or anyone else) have further comments on the proposed changes, we will submit a new version with the updates discussed above next week. Thanks again for your comments. Anna > Regards, > > Nicolas > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm From nobody Fri May 22 02:40:55 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2E42B1AC43B for ; Fri, 22 May 2015 02:40:54 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.91 X-Spam-Level: X-Spam-Status: No, score=-3.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, MIME_8BIT_HEADER=0.3, RCVD_IN_DNSWL_MED=-2.3, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rPnKj9EzqF8Z for ; Fri, 22 May 2015 02:40:51 -0700 (PDT) Received: from smtp.ee.ethz.ch (smtp.ee.ethz.ch [129.132.2.219]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 57F1F1AC437 for ; Fri, 22 May 2015 02:40:50 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by smtp.ee.ethz.ch (Postfix) with ESMTP id 2EE37D9305; Fri, 22 May 2015 11:40:49 +0200 (MEST) X-Virus-Scanned: by amavisd-new on smtp.ee.ethz.ch Received: from smtp.ee.ethz.ch ([127.0.0.1]) by localhost (.ee.ethz.ch [127.0.0.1]) (amavisd-new, port 10024) with LMTP id xJRWhSlWECD3; Fri, 22 May 2015 11:40:48 +0200 (MEST) Received: from [192.168.178.33] (x5f702c99.dyn.telefonica.de [95.112.44.153]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: mirjak) by smtp.ee.ethz.ch (Postfix) with ESMTPSA id B4B2FD9303; Fri, 22 May 2015 11:40:48 +0200 (MEST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2098\)) From: =?utf-8?Q?Mirja_K=C3=BChlewind?= In-Reply-To: <655C07320163294895BBADA28372AF5D483AF007@FR712WXCHMBA15.zeu.alcatel-lucent.com> Date: Fri, 22 May 2015 11:40:50 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <655C07320163294895BBADA28372AF5D16C939E2@FR712WXCHMBA15.zeu.alcatel-lucent.com> <655C07320163294895BBADA28372AF5D483AF007@FR712WXCHMBA15.zeu.alcatel-lucent.com> To: "Scharf, Michael (Michael)" X-Mailer: Apple Mail (2.2098) Archived-At: Cc: "tcpm@ietf.org" Subject: Re: [tcpm] Feedback on WG acceptance of draft-bensley-tcpm-dctcp-03 X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 May 2015 09:40:54 -0000 Hi, sorry I=E2=80=99ve overlooked the initial email on this. Similar as = Gorry and Richard, I think this document should be published as = information (as the document currently states its indented status) and = there is the expertise in tcpm to review this document. Therefore I'm in = favor of adopting this as a woking document. As think as this document = =E2=80=9Aonly=E2=80=98 documents existing and deployed technology, there = is only some reviews needed to ensure that the document is well readable = and complete. Therefore there should be basically no technical = discussion needed and I believe that we could move forward quickly = within tcpm. If the document will be adopted in tcp, I=E2=80=99d also volunteer to = review. Btw., as a comment to the authors, in the last version I=E2=80=99v= e read, loss handling was not discussed. I think this should be added; = please see the linux kernel code for this. And that's actually another = point. I believe this document mainly documents what is implemented in = Windows. As there is an implementation as part of the Linux kernel in = the mean time, maybe the author should check this implementation as well = or make clear what exactly is document in this draft. Mirja =20 > Am 18.05.2015 um 11:12 schrieb Scharf, Michael (Michael) = : >=20 > Hi all, >=20 > There have not been many responses to this e-mail. The chairs = therefore conclude that there is not enough support for adoption of = draft-bensley-tcpm-dctcp-03 as TCPM WG item as of now. >=20 > Since the TCPM community seems interested in further discussing this = topic, the chairs would offer to allocate time for corresponding = discussions during upcoming TCPM meetings. >=20 > Thanks >=20 > Michael, Pasi, Yoshifumi >=20 >=20 > PS: We recently found one possibly interesting study on operational = experience: = https://www.usenix.org/system/files/conference/nsdi15/nsdi15-paper-judd.pd= f >=20 >=20 >> -----Original Message----- >> From: tcpm [mailto:tcpm-bounces@ietf.org] On Behalf Of Scharf, = Michael >> (Michael) >> Sent: Wednesday, April 22, 2015 10:54 AM >> To: tcpm@ietf.org >> Subject: [tcpm] Feedback on WG acceptance of = draft-bensley-tcpm-dctcp- >> 03 >>=20 >> Dear all, >>=20 >> During IETF 89 [1] and on the list there has been a discussion about = WG >> adoption of draft-bensley-tcpm-dctcp [2]. The understanding of the >> chairs is that there is no clear consensus up to now. >>=20 >> In order to move forward, the chairs seek for community guidance >> regarding the following options to handle this draft: >>=20 >> a) Adopt in TCPM as Informational >>=20 >> b) Adopt in TCPM as Experimental >>=20 >> c) Do not adopt in TCPM now >>=20 >> Option c) does not prevent publication outside TCPM, such as >> publication by another working group (e.g., TSVWG), publication on >> Independent Stream, etc. >>=20 >> The TCPM chairs believe that adoption of alternative congestion = control >> algorithms in TCPM should be backed by a strong consensus. Please = also >> review [3] and [4] regarding the difference between Experimental and >> Informational. >>=20 >> Please let us know any feedback on a potential WG acceptance of = draft- >> bensley-tcpm-dctcp-03 until May 8. >>=20 >> Thanks! >>=20 >> Michael, Pasi, Yoshifumi >>=20 >>=20 >> [1] http://www.ietf.org/proceedings/89/minutes/minutes-89-tcpm >>=20 >> [2] https://tools.ietf.org/html/draft-bensley-tcpm-dctcp-03 >>=20 >> [3] RFC 2026, Section 4.2.1, 4.2.2 >>=20 >> [4] https://www.ietf.org/iesg/informational-vs-experimental.html >>=20 >> _______________________________________________ >> tcpm mailing list >> tcpm@ietf.org >> https://www.ietf.org/mailman/listinfo/tcpm >=20 > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm From nobody Fri May 22 03:00:38 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 82D3B1B2AB3 for ; Fri, 22 May 2015 03:00:37 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 2.778 X-Spam-Level: ** X-Spam-Status: No, score=2.778 tagged_above=-999 required=5 tests=[BAYES_50=0.8, FRT_STOCK2=3.988, MIME_8BIT_HEADER=0.3, RCVD_IN_DNSWL_MED=-2.3, T_RP_MATCHES_RCVD=-0.01] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id M_7yyloQgaiR for ; Fri, 22 May 2015 03:00:35 -0700 (PDT) Received: from smtp.ee.ethz.ch (smtp.ee.ethz.ch [129.132.2.219]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id CAD651B2AB6 for ; Fri, 22 May 2015 03:00:33 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by smtp.ee.ethz.ch (Postfix) with ESMTP id 1B055D9303; Fri, 22 May 2015 12:00:32 +0200 (MEST) X-Virus-Scanned: by amavisd-new on smtp.ee.ethz.ch Received: from smtp.ee.ethz.ch ([127.0.0.1]) by localhost (.ee.ethz.ch [127.0.0.1]) (amavisd-new, port 10024) with LMTP id t9BYHGtIg6j4; Fri, 22 May 2015 12:00:31 +0200 (MEST) Received: from [192.168.178.33] (x5f702c99.dyn.telefonica.de [95.112.44.153]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: mirjak) by smtp.ee.ethz.ch (Postfix) with ESMTPSA id 89321D9302; Fri, 22 May 2015 12:00:31 +0200 (MEST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2098\)) From: =?utf-8?Q?Mirja_K=C3=BChlewind?= In-Reply-To: Date: Fri, 22 May 2015 12:00:32 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <7C415879-6BF7-403D-B904-CC2B4D32CA04@mac.com> To: Tom Sanders X-Mailer: Apple Mail (2.2098) Archived-At: Cc: "tcpm@ietf.org" Subject: Re: [tcpm] Why is Cubic said to be RTT-fair? X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 May 2015 10:00:37 -0000 Hi Tom, the text below only says that if you have two cubic flows with a = different base RTT competing at the same bottleneck they=E2=80=99ll = still converge to share the link equally. This does not mean that if you = change the RTT strongly during one connection that Cubic is able to = catch up fast.=20 In fact this is one weakness of Cubic that I have observed. If there are = change in base delay/rtt or strong increases in the available capacity, = Cubic needs a rather long time to catch up. This is because cubic has a = =E2=80=9Atarget value=E2=80=98 to which it increases it sending = rate/window quickly but then if the capacity cap is not somewhere close = to this value (as expected) it will probe very conservatively around = this value and it takes a rather long time until it gets back into a = phase where it increases its congestion window fast. So what do you mean by 'I immediately saw that the TCP performance went = down=E2=80=98. How long do you run your test? And over which period of = time this the performance go down? Mirja > Am 21.05.2015 um 02:51 schrieb Tom Sanders : >=20 > Hi Neal, >=20 > This is odd. >=20 > In the original Cubic paper = (http://www4.ncsu.edu/~rhee/export/bitcp/cubic-paper.pdf) by Rhee and = Xu, they explicitly state that the Cubic is independent of RTT. >=20 > "The main feature of CUBIC is that its window growth function is = defined in real-time so that its growth will be independent of RTT. Our = work was partially inspired by HTCP [5], whose window growth function is = also based on real time. The congestion epoch period of CUBIC is = determined by the packet loss rate alone. As TCP=E2=80=99s throughput is = defined by the packet loss rate as well as RTT, the throughput of CUBIC = is defined by only the packet loss rate. Thus, when the loss rate is = high and/or RTT is short, CUBIC can operate in a TCP mode. Moreover, = since the growth function is independent of RTT, its RTT fairness is = guaranteed as different RTT flows will still grow their windows at the = same rate." >=20 > Am i missing something here? >=20 > Thanks, Toms >=20 > On 20 May 2015 at 23:13, Neal Cardwell wrote: > On Wed, May 20, 2015 at 1:18 PM, Tom Sanders = wrote: > > Thanks Rick, this was very helpful. > > > > So if there are no setsockopts, etc then CUBIC's performance, unlike = other > > TCP variants, should not get impacted by RTTs, right? Is that a fair > > statement to make? >=20 > CUBIC, like most TCP congestion control variants, has significant > aspects that evolve on the scale of RTTs: for example, in slow start, > loss recovery, or when the CUBIC function is very steep, CUBIC's > behavior is heavily dependent on RTT (send some packets, wait for an > ACK, send some more packets,...). So CUBIC will not be > perfectly"RTT-fair. >=20 > I think it might be fair to say something like: once a path is > saturated with CUBIC flows, as long as loss recovery or send/receive > buffers are not a bottleneck, then competition between CUBIC flows > should be more RTT-fair than Reno, in the long run, since for large > parts of the life cycle of the CUBIC flow the evolution of cwnd is > constrained by wall clock time rather than ACK arrival. >=20 > neal >=20 >=20 > > Toms > > > > On 20 May 2015 at 19:51, Rick Jones wrote: > >> > >> > >> On May 20, 2015, at 6:20 AM, Tom Sanders = wrote: > >> > >> > Hi, > >> > > >> > As per my understanding, unlike other TCP variants, CUBIC claims = to be > >> > RTT fair. This is because the window growth rate is independent = of the RTT, > >> > and hence the performance is the same as on a low and a high RTT = paths. > >> > Others are dependent upon RTT since their windows grow as they = receive ACKs > >> > from their peers, while CUBIC doesnt (window growth is a function = of time). > >> > This is theory. > >> > > >> > In practice, CUBIC (which is the default now on most linux > >> > distributions) also depends upon RTT. I connected to machines on = a LAN and > >> > did a FTP transfer. I got a certain bandwidth. Next, i = artificially inserted > >> > delay using "tc qdisc" in the path. I increased the delay on the = ethernet > >> > interface connecting my linux machine to the LAN to be around = 100ms. I > >> > immediately saw that the TCP performance went down. > >> > > >> > To bring it up to the same level as before i had to use multiple = TCP > >> > streams. To me this means that CUBIC performance is dependent of = the RTT. So > >> > why do we call it RTT-fair? > >> > >> I cannot speak to CUBIC specifically, but I would think you would = want to > >> make sure you were seeing effects of congestion window and not of = the > >> classic TCP window. For example, if the FTP client/server you were = using > >> makes an explicit setsockopt() to set the socket buffer sizes and = by > >> extension the classic TCP window, your bumping of the RTT to around = 100ms > >> may have taken the bandwidth down as a consequence of the classic: > >> > >> Throughput <=3D WindowSize/RTT > >> > >> Similarly, since you mention Linux, if the FTP client/server = didn=E2=80=99t make > >> explicit setsockopt() calls, the sysctl values for tcp_rmem and = tcp_wmem > >> may have been such that the auto tuning of the window size = couldn=E2=80=99t allow > >> the classic TCP window to grow =E2=80=9Cenough.=E2=80=9D > >> > >> I suspect that the experiment needs to be setup to have the same = classic > >> window size in both cases, sized per the above to be sufficient to = achieve > >> the peak throughput in the high RTT case. Then you will have = eliminated > >> classic window as a factor. > >> > >> Also, depending on the version of the Linux kernel you are using, = there > >> may be some effects from tcp small queues or perhaps (a stretch?) = even byte > >> queue limits which may not be playing well with netem. (Just = guessing there > >> really) > >> > >> happy benchmarking, > >> > >> rick jones > > > > > > > > > > -- > > Toms. > > > > _______________________________________________ > > tcpm mailing list > > tcpm@ietf.org > > https://www.ietf.org/mailman/listinfo/tcpm > > >=20 >=20 >=20 > --=20 > Toms. > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm From nobody Fri May 22 10:33:41 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5910D1A8035; Fri, 22 May 2015 10:33:39 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6uXRRXFKK-qC; Fri, 22 May 2015 10:33:38 -0700 (PDT) Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 2729A1A7011; Fri, 22 May 2015 10:33:38 -0700 (PDT) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit From: internet-drafts@ietf.org To: X-Test-IDTracker: no X-IETF-IDTracker: 6.0.3 Auto-Submitted: auto-generated Precedence: bulk Message-ID: <20150522173338.31853.62521.idtracker@ietfa.amsl.com> Date: Fri, 22 May 2015 10:33:38 -0700 Archived-At: Cc: tcpm@ietf.org Subject: [tcpm] I-D Action: draft-ietf-tcpm-newcwv-11.txt X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 May 2015 17:33:39 -0000 A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the TCP Maintenance and Minor Extensions Working Group of the IETF. Title : Updating TCP to support Rate-Limited Traffic Authors : Godred Fairhurst Arjuna Sathiaseelan Raffaello Secchi Filename : draft-ietf-tcpm-newcwv-11.txt Pages : 25 Date : 2015-05-22 Abstract: This document provides a mechanism to address issues that arise when TCP is used to support traffic that exhibits periods where the sending rate is limited by the application rather than the congestion window. It provides an experimental update to TCP that allows a TCP sender to restart quickly following a rate-limited interval. This method is expected to benefit applications that send rate-limited traffic using TCP, while also providing an appropriate response if congestion is experienced. It also evaluates the Experimental specification of TCP Congestion Window Validation, CWV, defined in RFC 2861, and concludes that RFC 2861 sought to address important issues, but failed to deliver a widely used solution. This document therefore recommends that the status of RFC 2861 is moved from Experimental to Historic, and that it is replaced by the current specification. The IETF datatracker status page for this draft is: https://datatracker.ietf.org/doc/draft-ietf-tcpm-newcwv/ There's also a htmlized version available at: https://tools.ietf.org/html/draft-ietf-tcpm-newcwv-11 A diff from the previous version is available at: https://www.ietf.org/rfcdiff?url2=draft-ietf-tcpm-newcwv-11 Please note that it may take a couple of minutes from the time of submission until the htmlized version and diff are available at tools.ietf.org. Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ From nobody Fri May 22 10:38:39 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 710E61A86FC for ; Fri, 22 May 2015 10:38:38 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.511 X-Spam-Level: X-Spam-Status: No, score=-0.511 tagged_above=-999 required=5 tests=[BAYES_05=-0.5, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id h7BzkKLDIZ6V for ; Fri, 22 May 2015 10:38:35 -0700 (PDT) Received: from pegasus.erg.abdn.ac.uk (pegasus.erg.abdn.ac.uk [IPv6:2001:630:241:204::f0f0]) by ietfa.amsl.com (Postfix) with ESMTP id E02131A0013 for ; Fri, 22 May 2015 10:38:34 -0700 (PDT) Received: from erg.abdn.ac.uk (galactica.erg.abdn.ac.uk [139.133.210.32]) by pegasus.erg.abdn.ac.uk (Postfix) with ESMTPA id 807E31B001D2 for ; Fri, 22 May 2015 18:38:37 +0100 (BST) Received: from 212.159.18.54 (SquirrelMail authenticated user gorry) by erg.abdn.ac.uk with HTTP; Fri, 22 May 2015 18:37:40 +0100 Message-ID: In-Reply-To: <20150522173338.31853.44182.idtracker@ietfa.amsl.com> References: <20150522173338.31853.44182.idtracker@ietfa.amsl.com> Date: Fri, 22 May 2015 18:37:40 +0100 From: gorry@erg.abdn.ac.uk To: tcpm@ietf.org User-Agent: SquirrelMail/1.4.23 [SVN] MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal Archived-At: Subject: Re: [tcpm] New Version Notification for draft-ietf-tcpm-newcwv-11.txt X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 May 2015 17:38:38 -0000 The document editors have just uploaded a new revision of the CWV draft, intended to address this issues raised in the IETF LC. Specific comments are listed below, Best wishes, Gorry & Raffaello --- Mirja (TCPM) pipeACK (section 4.1 and 4.2 and 4.5.1): 1) Can you maybe add a sentence that explains why FlightSize is not sufficient. I understand the difference between the two variables but are there actually cases where it makes a big difference if one or the other is used? - Added to section 1. 2) This is just an idea and does not make a big difference: Instead of saying pipeACK is undefined, you could just set it to infinity or 'max' (what every the max in the implementation is) and this would always keep you in validated phase. - Not changed, this is an implementation choice, but setting to infinity requires care when considering other uses of the pipeACK, e.g., when calculating cwnd = (Max(pipeACK,LossFlightSize))/2. 3) The implementation for pipeACK (based on HighACK) that you propose does not take into account that dupACK also acknowledge new data. Is this on purpose? If so please state explicitly. - A sender MAY count TCP DupACKs that acknowledge new data when collecting the pipeACK sample. Other equivalent methods may be used. 4) Text says "When no measurements are available, ..." Maybe say more explicitly "When no measurements are available as the connection is idle" or "... as no data were sent during the last measurement period" or "... no ACK were received...". - (e.g., a sender that has been idle will have sent no data and received no ACKs) 5) I would appreciate a forward reference to section 4.5.1 here (as also done for other stuff as for cwnd-limited in the next section). - Done 6) Could you also include pseudo code in 4.5.1? I far as I know you have an implementation, so it should not be extremely hard to come up with pseudo code and I personally think it would be really helpful... We don’t plan to add this to 4.1, burt there is code available for Linux and BSD. 7) nit in last paragraph: s/method defined on [RFC6675]is/ method defined in [RFC6675] is/ - Done 8) Figure 1 (section 4.5.1): Maybe add a y-axis...? - pipeACK sample (Bytes) non-validated phase (section 4.3 and 4.4) 1) Maybe add one sentence saying "A connection is also in non-validated phase if pipeACK is zero which means the connection is idle". Saying this explicit would help me to understand things better. - Idle isn’t quite correct, what it means is no new data was ACK’ed - there could be data sent and pending ACK/ We changed 4.5.2 to clarify this. I just notice that this might actually not be true because in the section above you say that pipeACK is set to undefined if no measurements are available and that would mean the connection goes into validated phase... okay I think there is some clarification needed here. - see above 2) In section 4.4 you first say "cwnd neither grows nor reduces while the sender remains in this phase" and then "A sender that is cwnd-limited MAY use the standard TCP method to increase cwnd". Please clarify! - A TCP sender that enters the non-validated phase preserves the cwnd (i.e., the cwnd only increases after a sender fully uses the cwnd in this phase, otherwise the cwnd neither grows nor reduces). 3) In section 4.4.1: "A sender that detects a packet-drop MUST record the current FlightSize in the variable LossFlightSize ..." Is this "packet-drop" or "congestion notification"? I guess the difference is that if ECN is used R=0. So that's okay. Please check again that this also works correctly for ECN. - Agree, we removed explicit discussion of ECN - We agree R=0 when ECN is used and there is no loss, but we did not wish to further interpret the correct response for ECN. 4) Is this cwnd = (Max(pipeACK,LossFlightSize) - R)/2 - This is correct and more conservative. or cwnd = (Max(pipeACK,(LossFlightSize - R))/2 [only LossFlightSize - R] ? Because the text reads as if it should be the second... - Text has been fixed: The loss of segments indicates that the path capacity was exceeded by at least R, and hence the calculated cwnd is reduced by at least R before the window is halved. 5) "ssthresh is adjusted using the standard TCP method." This is not clear to me. Is ssthresh set to cwnd/2 before the adjustment or to cwnd-1 after the adjustment? The first option would restart the connection in Slow Start... Please clarify! - We use the standard method, with no change. 6) "then this can result in the final cwnd after loss recovery being 1/4 of the cwnd" Shouldn't this be "being at max 1/4 of the cwnd"...? - at most one quarter of the cwnd 7) "Subsequent updates to cwnd do not therefore reflect pipeACK history before any congestion event." Not sure I understand this sentence... which subsequent updates? - This reduction is conservative, and pipeACK is then reset to undefined, hence cwnd updates after a congestion event do not depend upon the pipeACK history before congestion was detected. 8) nit: s/updated during loss recoverySection 4.2/ updated during loss recovery as stated in Section 4.2/ - Done Non-validated phase (section 4.4.3 and 5) 1) I would recommend to change the title to "4.4.3 Adjustment at the end of the Non-Validated Phase (NVP)" - Done 2) "An application that remains in the non-validated phase for a period greater than the NVP is required to adjust its congestion control state. If the sender exits the non-validated phase after this period, it MUST update the ssthresh:" This is not fully clear to me. Do I have to adapt cwnd and ssthresh right after NVP or as soon as I leave the non-validated phase after being for at least NVP time in this phase? If I'm not idle, but only sending with a low rate, I should probably do this adjustment with the next ACK, no...? - Not sure of the difference, e hope the new intro helps with this. 3) Further, these adjustments mean that the connection will be in Slow Start. If I understand this correctly, this is not a problem as long as the connection stays in non-validated as the cwnd will not be further increased. But as soon as it leaves non-validated phase it will increase its cwnd as in Slow Start. I would like to have this spelled out explicitly. - ??? 4) This section does not tell/specify what value NVP period should have. Please add a sentence that this is on purpose and add a reference to section 5. - We see this problem= as a definition in the info section and added text to 4.4. 5) Section 5 says "A period of five minutes was chosen for this NVP." This seems to be a left over from an older version as nothing was specified so far. Please check! - Fixed ----------------------------------------------------------------------------------------- Martin (AD) The comment: - Section 5: " Routing algorithms may modify the network path, disrupting the RTT measurement and changing the capacity available to a TCP connection, however such changes do not usually occur within a time frame of a few minutes." That is an odd assumption, as a path change can happen at any time, though it is not very usually under normal conditions. Did actually mean to say that it is uncommon to happen? - Routing algorithms may change the the network path that is used by a transport. Although a change of path can in turn disrupt the RTT measurement and may result in a change of the capacity available to a TCP connection, we assume these path changes do not usually occur frequently (compared to a time frame of a few minutes). The two nits: - Section 4.4.1 " The pipeACK value is not updated during loss recoverySection 4.2. If there is a valid pipeACK value, the new cwnd is adjusted to reflect that a non-validated cwnd may be larger than the actual FlightSize, or recently used FlightSize (recorded in pipeACK). The updated cwnd therefore prevents overshoot by a sender significantly increasing its transmission rate during the recovery period." Typo in first sentence "recoverySection 4.2" - Done - Section 4.5.2. " A sender needs to implement a method that detects the cwnd-limited condition (see Section 4.4. This detects a condition where a sender" closing brackets after 4.4. missing - Done ----------------------------------------------------------------------------------------- Sec The security considerations defer to those of RFC 5681 with the claim that this document just describes an algorithm for one aspect of the congestion control procedures and thus that the generic security considerations apply; this claim seems true. The security considerations section of RFC 5681 feels a little bit short, but it does seem to cover the relevant risks, namely that an attacker could make the sender send more slowly or more quickly, with the latter possibly driving the network into congestion collapse. These are issues that protocol designers and implementors must be aware of (and avoiding congestion collapse is more important). The authors seem to have taken sufficient care to avoid this algorithm contributing to congestion collapse, and the concern about an attacker being able to slow down a sender is not really feasible to mitigate at this level, so the security considerations of RFC 5681, short as they are, seem adequate here. One would hope that implementors know that failing to follow the specification could lead to congestion collapse, so an explicit warning is not needed. The nits are basically all grammatical; I'll include them below and expect people other than the document authors/shepherd to stop reading here. -Ben My apologies in advance if I am applying too much of an American English perspective to this document written in British English. In the abstract, "TCP is used to support traffic that [...]", is "support" the best verb to use (as opposed to "transmit", "carry", "transport", etc.)? - Done The first paragraph of the Introduction might have a little more text to clarify how the cwnd differs from the FlightSize, for non-experts such as myself. (The cwnd is how much the sender is permitted to have outstanding/unack'd and the FlightSize is how much the sender actually has outstanding/unack'd?) Just another sentence after the second one, "The FlightSize is how many unacknowledged packets/bytes that are currently outstanding, and the cwnd is the maximum value the sender will allow FlightSize to take on" would be plenty. - Added Explanation, and reworked this text. Introduction, third paragraph, """[CWV] introduced the terminology of "application limited periods". This document describes [...]""" Usually, in an RFC, "this document" refers to the RFC itself, but in this case, "this document" seems to be being used to refer to the other document, RFC 2861. So, the references and text could be made more clear. - Updated to refer to RFC2861. A couple sentences later, "or by changing the rate the application sends", maybe put an "at which" in there somewhere? Absent context, I would expect "the rate the application sends" to mean that an application protocol was conveying a number which is interpreted as a rate in some fashion. - Changed Still in the Introduction, in the summary of the document, paragraph for section 4, maybe s/does this in a way/does so in a way/ ? Also, in the parenthetical at the end of that sentence, I think it reads better as "including both application-limited and idle senders". - Changed Still in the introduction, in the "goals of this update" list, first item, do bulk transfers consume the cwnd or fill it? - changed text I'm not sure that the fifth item is fully grammatical. Maybe add in an "to send extra traffic" somewhere? - changed text In the sixth item, there's a singular/plural mismatch between flows and network (unless it's supposed to be "the network"). - changed to benefiting both the flows and other flows sharing the network path Section 2, first paragraph, "the behaviour where a sender transmitted at a rate less than allowed by cwnd", is "where" or "when" better? - when Section 2, second paragraph, third sentence: I think it should start "While this" instead of just "This". The first comma in this sentence could also be removed, if desired. - While this In section 3 (Terminology), it might be nice to clarify that the listed terms are new to this document, or in addition to the RFC 5681 terms, or something like that. (I.e., "Additionally, the following terminology ...".) - additional Section 4.2, last paragraph, a space is missing after the [RFC6675] citation. - Done In section 4.4: if I understand correctly, the sliding window for pipeAck calculation is smaller than the NVP, so it is not the absolute amount of data which the sender transmits which can push pipeAck over (1/2)*cwnd, but rather the rate at which the data is transmitted. - pipeACK is a measure per RTT (like cwnd) Also in section 4.4, there is some superficial inconsistency between "a sender that enters the non-validated phase SHOULD preserve the cwnd" and the text telling the sender how to reduce cwnd if it encounters congestion and the (outer) bullet point wherein "a sender determines whether to increase the cwnd based on ...". The last paragraph of the section helps clarify the situation, but it might help readability to clarify the exceptions to the SHOULD (and the rationale for them) right after the claim is made. - Reworked. In section 4.4.1, there is a missing space after "recovery" and before the internal reference to Section 4.2. - Done Section 4.5.1, comma after "e.g.". - Done ----------------------------------------------------------------------------------------- Gen-ART Nits/editorial comments: 1. I am not sure that there is a need for the first sentence in section 1.2 - Moved to ACK section. 2. In section 4.4.1 MSS is used but not expanded Maximum Segment Size (MSS) - Done From nobody Fri May 22 18:16:38 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 79C841A874A for ; Fri, 22 May 2015 18:16:37 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 4.989 X-Spam-Level: **** X-Spam-Status: No, score=4.989 tagged_above=-999 required=5 tests=[BAYES_50=0.8, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, FRT_STOCK2=3.988, HTML_MESSAGE=0.001, MIME_8BIT_HEADER=0.3, SPF_PASS=-0.001] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id clAzQoeb9lJm for ; Fri, 22 May 2015 18:16:34 -0700 (PDT) Received: from mail-wi0-x22b.google.com (mail-wi0-x22b.google.com [IPv6:2a00:1450:400c:c05::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 31BB71A0119 for ; Fri, 22 May 2015 18:16:34 -0700 (PDT) Received: by wibt6 with SMTP id t6so3291199wib.0 for ; Fri, 22 May 2015 18:16:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Mne7GpqEMka8mS31HgMFRMhREWj0j2BzkmXsJbBOFhs=; b=iiJfPsJGervvIp3+9x5SmEANQ88juuw6HTBgRLbTMrX43709dfy3+Y6Z8Gqw2f9o0Z eS27RoeXDEDW/qRiZqs+5LatXTdl03UV/G9YWva2G79hvyuSJkS2WyEzwAF+s0UFLPUt HLLQSgHUC/qXVkKZl7uFPAsQdikysAjf+qbUGpdNpthwjQUEJEObTkfeMZW6wISWMStz /HwPsaT1hzkmpwJRMQl8Dlk2Z35kM9gOnbFM3zr2BJCZLWVomEgksZZh4H/TFEYk99HD x2Ni3nFqeNUHxyJ0R4rq68NlTXQzna38XGqVLSGibLVQnTKbRmLIpf9uqQtauzv9z2We 0Vlg== MIME-Version: 1.0 X-Received: by 10.194.184.79 with SMTP id es15mr1528605wjc.112.1432343792964; Fri, 22 May 2015 18:16:32 -0700 (PDT) Received: by 10.28.133.148 with HTTP; Fri, 22 May 2015 18:16:32 -0700 (PDT) In-Reply-To: References: <7C415879-6BF7-403D-B904-CC2B4D32CA04@mac.com> Date: Sat, 23 May 2015 06:46:32 +0530 Message-ID: From: Tom Sanders To: =?UTF-8?Q?Mirja_K=C3=BChlewind?= Content-Type: multipart/alternative; boundary=047d7b8738d0f9db890516b586a7 Archived-At: Cc: "tcpm@ietf.org" Subject: Re: [tcpm] Why is Cubic said to be RTT-fair? X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 23 May 2015 01:16:37 -0000 --047d7b8738d0f9db890516b586a7 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Hi Mirja, > > > the text below only says that if you have two cubic flows with a differen= t > base RTT competing at the same bottleneck they=E2=80=99ll still converge = to share > the link equally. This does not mean that if you change the RTT strongly > during one connection that Cubic is able to catch up fast. > > In fact this is one weakness of Cubic that I have observed. If there are > change in base delay/rtt or strong increases in the available capacity, > Cubic needs a rather long time to catch up. This is because cubic has a > =E2=80=9Atarget value=E2=80=98 to which it increases it sending rate/wind= ow quickly but > then if the capacity cap is not somewhere close to this value (as expecte= d) > it will probe very conservatively around this value and it takes a rather > long time until it gets back into a phase where it increases its congesti= on > window fast. > Then which congestion control algorithm fares better there? I thought Cubic was suited for large fat pipes -- bigger bandwidth-delay pipes. > > So what do you mean by 'I immediately saw that the TCP performance went > down=E2=80=98. How long do you run your test? And over which period of ti= me this > the performance go down? > I did a simple test. I connect two machines on my 1GB LAN.I do a tcp data transfer test and i get around ~900Mbps. I then added a delay using qdisc on my outgoing ethernet interface to simulate a longer RTT. So, if my ping earlier was around 0.255ms, it then became 120ms. I repeat the test and i see that i can only send about 200Mbps. I had to use multiple TCP flows to fill up my 1GB link with Cubic. I run this experiment for about 2 minutes, and Cubic is not able to catch up. I dont have long standing flows, and only need to send bursts of data, hence did not leave it running for very long. I was under the impression, after having read the original Cubic paper that Cubic's window growth rate is independent of RTT and hence wasnt expecting any difference in the throughput. Toms. > > Mirja > > > > Am 21.05.2015 um 02:51 schrieb Tom Sanders : > > > > Hi Neal, > > > > This is odd. > > > > In the original Cubic paper ( > http://www4.ncsu.edu/~rhee/export/bitcp/cubic-paper.pdf) by Rhee and Xu, > they explicitly state that the Cubic is independent of RTT. > > > > "The main feature of CUBIC is that its window growth function is define= d > in real-time so that its growth will be independent of RTT. Our work was > partially inspired by HTCP [5], whose window growth function is also base= d > on real time. The congestion epoch period of CUBIC is determined by the > packet loss rate alone. As TCP=E2=80=99s throughput is defined by the pac= ket loss > rate as well as RTT, the throughput of CUBIC is defined by only the packe= t > loss rate. Thus, when the loss rate is high and/or RTT is short, CUBIC ca= n > operate in a TCP mode. Moreover, since the growth function is independent > of RTT, its RTT fairness is guaranteed as different RTT flows will still > grow their windows at the same rate." > > > > Am i missing something here? > > > > Thanks, Toms > > > > On 20 May 2015 at 23:13, Neal Cardwell wrote: > > On Wed, May 20, 2015 at 1:18 PM, Tom Sanders > wrote: > > > Thanks Rick, this was very helpful. > > > > > > So if there are no setsockopts, etc then CUBIC's performance, unlike > other > > > TCP variants, should not get impacted by RTTs, right? Is that a fair > > > statement to make? > > > > CUBIC, like most TCP congestion control variants, has significant > > aspects that evolve on the scale of RTTs: for example, in slow start, > > loss recovery, or when the CUBIC function is very steep, CUBIC's > > behavior is heavily dependent on RTT (send some packets, wait for an > > ACK, send some more packets,...). So CUBIC will not be > > perfectly"RTT-fair. > > > > I think it might be fair to say something like: once a path is > > saturated with CUBIC flows, as long as loss recovery or send/receive > > buffers are not a bottleneck, then competition between CUBIC flows > > should be more RTT-fair than Reno, in the long run, since for large > > parts of the life cycle of the CUBIC flow the evolution of cwnd is > > constrained by wall clock time rather than ACK arrival. > > > > neal > > > > > > > Toms > > > > > > On 20 May 2015 at 19:51, Rick Jones wrote: > > >> > > >> > > >> On May 20, 2015, at 6:20 AM, Tom Sanders > wrote: > > >> > > >> > Hi, > > >> > > > >> > As per my understanding, unlike other TCP variants, CUBIC claims t= o > be > > >> > RTT fair. This is because the window growth rate is independent of > the RTT, > > >> > and hence the performance is the same as on a low and a high RTT > paths. > > >> > Others are dependent upon RTT since their windows grow as they > receive ACKs > > >> > from their peers, while CUBIC doesnt (window growth is a function > of time). > > >> > This is theory. > > >> > > > >> > In practice, CUBIC (which is the default now on most linux > > >> > distributions) also depends upon RTT. I connected to machines on a > LAN and > > >> > did a FTP transfer. I got a certain bandwidth. Next, i artificiall= y > inserted > > >> > delay using "tc qdisc" in the path. I increased the delay on the > ethernet > > >> > interface connecting my linux machine to the LAN to be around > 100ms. I > > >> > immediately saw that the TCP performance went down. > > >> > > > >> > To bring it up to the same level as before i had to use multiple T= CP > > >> > streams. To me this means that CUBIC performance is dependent of > the RTT. So > > >> > why do we call it RTT-fair? > > >> > > >> I cannot speak to CUBIC specifically, but I would think you would > want to > > >> make sure you were seeing effects of congestion window and not of th= e > > >> classic TCP window. For example, if the FTP client/server you were > using > > >> makes an explicit setsockopt() to set the socket buffer sizes and by > > >> extension the classic TCP window, your bumping of the RTT to around > 100ms > > >> may have taken the bandwidth down as a consequence of the classic: > > >> > > >> Throughput <=3D WindowSize/RTT > > >> > > >> Similarly, since you mention Linux, if the FTP client/server didn=E2= =80=99t > make > > >> explicit setsockopt() calls, the sysctl values for tcp_rmem and > tcp_wmem > > >> may have been such that the auto tuning of the window size couldn=E2= =80=99t > allow > > >> the classic TCP window to grow =E2=80=9Cenough.=E2=80=9D > > >> > > >> I suspect that the experiment needs to be setup to have the same > classic > > >> window size in both cases, sized per the above to be sufficient to > achieve > > >> the peak throughput in the high RTT case. Then you will have > eliminated > > >> classic window as a factor. > > >> > > >> Also, depending on the version of the Linux kernel you are using, > there > > >> may be some effects from tcp small queues or perhaps (a stretch?) > even byte > > >> queue limits which may not be playing well with netem. (Just > guessing there > > >> really) > > >> > > >> happy benchmarking, > > >> > > >> rick jones > > > > > > > > > > > > > > > -- > > > Toms. > > > > > > _______________________________________________ > > > tcpm mailing list > > > tcpm@ietf.org > > > https://www.ietf.org/mailman/listinfo/tcpm > > > > > > > > > > > -- > > Toms. > > _______________________________________________ > > tcpm mailing list > > tcpm@ietf.org > > https://www.ietf.org/mailman/listinfo/tcpm > > --=20 Toms. --047d7b8738d0f9db890516b586a7 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Hi Mirja,


the text below only says that if you have two cubic flows with a different = base RTT competing at the same bottleneck they=E2=80=99ll still converge to= share the link equally. This does not mean that if you change the RTT stro= ngly during one connection that Cubic is able to catch up fast.

In fact this is one weakness of Cubic that I have observed. If there are ch= ange in base delay/rtt or strong increases in the available capacity, Cubic= needs a rather long time to catch up. This is because cubic has a =E2=80= =9Atarget value=E2=80=98 to which it increases it sending rate/window quick= ly but then if the capacity cap is not somewhere close to this value (as ex= pected) it will probe very conservatively around this value and it takes a = rather long time until it gets back into a phase where it increases its con= gestion window fast.

Then which congest= ion control algorithm fares better there? I thought Cubic was suited for la= rge fat pipes -- bigger bandwidth-delay pipes.=C2=A0
=C2=A0
=

So what do you mean by 'I immediately saw that the TCP performance went= down=E2=80=98. How long do you run your test? And over which period of tim= e this the performance go down?

I did a= simple test.

I connect two machines on my 1GB LAN= .I do a tcp data transfer test and i get around ~900Mbps. I then added a de= lay using qdisc on my outgoing ethernet interface to simulate a longer RTT.= So, if my ping earlier was around 0.255ms, it then became 120ms. I repeat = the test and i see that i can only send about 200Mbps. I had to use multipl= e TCP flows to fill up my 1GB link with Cubic. I run this experiment for ab= out 2 minutes, and Cubic is not able to catch up. I dont have long standing= flows, and only need to send bursts of data, hence did not leave it runnin= g for very long.

I was under the impression, after= having read the original Cubic paper that Cubic's window growth rate i= s independent of RTT and hence wasnt expecting any difference in the throug= hput.

Toms.

Mirja


> Am 21.05.2015 um 02:51 schrieb Tom Sanders <toms.sanders@gmail.com>:
>
> Hi Neal,
>
> This is odd.
>
> In the original Cubic paper (http://www4.ncsu.edu/~rhee/exp= ort/bitcp/cubic-paper.pdf) by Rhee and Xu, they explicitly state that t= he Cubic is independent of RTT.
>
> "The main feature of CUBIC is that its window growth function is = defined in real-time so that its growth will be independent of RTT. Our wor= k was partially inspired by HTCP [5], whose window growth function is also = based on real time. The congestion epoch period of CUBIC is determined by t= he packet loss rate alone. As TCP=E2=80=99s throughput is defined by the pa= cket loss rate as well as RTT, the throughput of CUBIC is defined by only t= he packet loss rate. Thus, when the loss rate is high and/or RTT is short, = CUBIC can operate in a TCP mode. Moreover, since the growth function is ind= ependent of RTT, its RTT fairness is guaranteed as different RTT flows will= still grow their windows at the same rate."
>
> Am i missing something here?
>
> Thanks, Toms
>
> On 20 May 2015 at 23:13, Neal Cardwell <ncardwell@google.com> wrote:
> On Wed, May 20, 2015 at 1:18 PM, Tom Sanders <toms.sanders@gmail.com> wrote:
> > Thanks Rick, this was very helpful.
> >
> > So if there are no setsockopts, etc then CUBIC's performance,= unlike other
> > TCP variants, should not get impacted by RTTs, right? Is that a f= air
> > statement to make?
>
> CUBIC, like most TCP congestion control variants, has significant
> aspects that evolve on the scale of RTTs: for example, in slow start,<= br> > loss recovery, or when the CUBIC function is very steep, CUBIC's > behavior is heavily dependent on RTT (send some packets, wait for an > ACK, send some more packets,...). So CUBIC will not be
> perfectly"RTT-fair.
>
> I think it might be fair to say something like: once a path is
> saturated with CUBIC flows, as long as loss recovery or send/receive > buffers are not a bottleneck, then competition between CUBIC flows
> should be more RTT-fair than Reno, in the long run, since for large > parts of the life cycle of the CUBIC flow the evolution of cwnd is
> constrained by wall clock time rather than ACK arrival.
>
> neal
>
>
> > Toms
> >
> > On 20 May 2015 at 19:51, Rick Jones <perfgeek@mac.com> wrote:
> >>
> >>
> >> On May 20, 2015, at 6:20 AM, Tom Sanders <toms.sanders@gmail.com> wrote:
> >>
> >> > Hi,
> >> >
> >> > As per my understanding, unlike other TCP variants, CUBI= C claims to be
> >> > RTT fair. This is because the window growth rate is inde= pendent of the RTT,
> >> > and hence the performance is the same as on a low and a = high RTT paths.
> >> > Others are dependent upon RTT since their windows grow a= s they receive ACKs
> >> > from their peers, while CUBIC doesnt (window growth is a= function of time).
> >> > This is theory.
> >> >
> >> > In practice, CUBIC (which is the default now on most lin= ux
> >> > distributions) also depends upon RTT. I connected to mac= hines on a LAN and
> >> > did a FTP transfer. I got a certain bandwidth. Next, i a= rtificially inserted
> >> > delay using "tc qdisc" in the path. I increase= d the delay on the ethernet
> >> > interface connecting my linux machine to the LAN to be a= round 100ms. I
> >> > immediately saw that the TCP performance went down.
> >> >
> >> > To bring it up to the same level as before i had to use = multiple TCP
> >> > streams. To me this means that CUBIC performance is depe= ndent of the RTT. So
> >> > why do we call it RTT-fair?
> >>
> >> I cannot speak to CUBIC specifically, but I would think you w= ould want to
> >> make sure you were seeing effects of congestion window and no= t of the
> >> classic TCP window.=C2=A0 For example, if the FTP client/serv= er you were using
> >> makes an explicit setsockopt() to set the socket buffer sizes= and by
> >> extension the classic TCP window, your bumping of the RTT to = around 100ms
> >> may have taken the bandwidth down as a consequence of the cla= ssic:
> >>
> >> Throughput <=3D WindowSize/RTT
> >>
> >> Similarly, since you mention Linux, if the FTP client/server = didn=E2=80=99t make
> >> explicit setsockopt() calls,=C2=A0 the sysctl values for tcp_= rmem and tcp_wmem
> >> may have been such that the auto tuning of the window size co= uldn=E2=80=99t allow
> >> the classic TCP window to grow =E2=80=9Cenough.=E2=80=9D
> >>
> >> I suspect that the experiment needs to be setup to have the s= ame classic
> >> window size in both cases, sized per the above to be sufficie= nt to achieve
> >> the peak throughput in the high RTT case.=C2=A0 Then you will= have eliminated
> >> classic window as a factor.
> >>
> >> Also, depending on the version of the Linux kernel you are us= ing, there
> >> may be some effects from tcp small queues or perhaps (a stret= ch?) even byte
> >> queue limits which may not be playing well with netem.=C2=A0 = (Just guessing there
> >> really)
> >>
> >> happy benchmarking,
> >>
> >> rick jones
> >
> >
> >
> >
> > --
> > Toms.
> >
> > _______________________________________________
> > tcpm mailing list
> > tcpm@ietf.org
> > https://www.ietf.org/mailman/listinfo/tcpm
> >
>
>
>
> --
> Toms.
> _______________________________________________
> tcpm mailing list
> tcpm@ietf.org
> https://www.ietf.org/mailman/listinfo/tcpm




--
=
Toms.
--047d7b8738d0f9db890516b586a7-- From nobody Fri May 22 22:36:57 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 869F11A90BA for ; Fri, 22 May 2015 22:36:56 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 4.689 X-Spam-Level: **** X-Spam-Status: No, score=4.689 tagged_above=-999 required=5 tests=[BAYES_50=0.8, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, FRT_STOCK2=3.988, HTML_MESSAGE=0.001, SPF_PASS=-0.001] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 9qrEy-He_C_U for ; Fri, 22 May 2015 22:36:52 -0700 (PDT) Received: from mail-vn0-x22e.google.com (mail-vn0-x22e.google.com [IPv6:2607:f8b0:400c:c0f::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id F307B1A90B9 for ; Fri, 22 May 2015 22:36:51 -0700 (PDT) Received: by vnbg190 with SMTP id g190so2601360vnb.3 for ; Fri, 22 May 2015 22:36:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=cXZX4DiPxpi6r2IJZOx8ez0ySTkwov5Uwxc3atTrnAI=; b=KDvtKmQ6LcmOxw2fM7YfvIk4oqiEjDgPJHTDoxVa7o1rNgO+UaXsAXETIovdnvB2VO q8KVo/NUs2DDTekGGQ/DmEQLnNGUR5O4GtnTDsnelNXB0Ksk90JhOGUCAcOI/OHFKA3G nD8E7tJQ9KZRb7DwxNvqKerBADXFzWgr/CrNAwsimqikkQTxCgSSolFbOXLNVKhlntKb OgGi8uMo81a+BcnL/MCvQjIXI5+kdUJju3YaIibcyMv+dcxyJKSy7d5Xe1MXWTAo85o4 kgxc7doWT8EVwEF5xYtG5axJJJSj5NFX1nvR0SZ7qNs5CMvdlON8RGW4syEBp/PzqZEO zZCA== MIME-Version: 1.0 X-Received: by 10.52.228.228 with SMTP id sl4mr10405752vdc.15.1432359411255; Fri, 22 May 2015 22:36:51 -0700 (PDT) Received: by 10.52.99.67 with HTTP; Fri, 22 May 2015 22:36:51 -0700 (PDT) Received: by 10.52.99.67 with HTTP; Fri, 22 May 2015 22:36:51 -0700 (PDT) In-Reply-To: References: <7C415879-6BF7-403D-B904-CC2B4D32CA04@mac.com> Date: Sat, 23 May 2015 06:36:51 +0100 Message-ID: From: Gavin McCullagh To: Tom Sanders Content-Type: multipart/alternative; boundary=089e0111d802e60f0a0516b92955 Archived-At: Cc: "tcpm@ietf.org" Subject: Re: [tcpm] Why is Cubic said to be RTT-fair? X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 23 May 2015 05:36:56 -0000 --089e0111d802e60f0a0516b92955 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Hi Tom, What did you use to create the tcp flows? Was it iperf or similar? Gavin On 23 May 2015 02:16, "Tom Sanders" wrote: > Hi Mirja, > >> >> >> the text below only says that if you have two cubic flows with a >> different base RTT competing at the same bottleneck they=E2=80=99ll stil= l converge >> to share the link equally. This does not mean that if you change the RTT >> strongly during one connection that Cubic is able to catch up fast. >> >> In fact this is one weakness of Cubic that I have observed. If there are >> change in base delay/rtt or strong increases in the available capacity, >> Cubic needs a rather long time to catch up. This is because cubic has a >> =E2=80=9Atarget value=E2=80=98 to which it increases it sending rate/win= dow quickly but >> then if the capacity cap is not somewhere close to this value (as expect= ed) >> it will probe very conservatively around this value and it takes a rathe= r >> long time until it gets back into a phase where it increases its congest= ion >> window fast. >> > > Then which congestion control algorithm fares better there? I thought > Cubic was suited for large fat pipes -- bigger bandwidth-delay pipes. > > >> >> So what do you mean by 'I immediately saw that the TCP performance went >> down=E2=80=98. How long do you run your test? And over which period of t= ime this >> the performance go down? >> > > I did a simple test. > > I connect two machines on my 1GB LAN.I do a tcp data transfer test and i > get around ~900Mbps. I then added a delay using qdisc on my outgoing > ethernet interface to simulate a longer RTT. So, if my ping earlier was > around 0.255ms, it then became 120ms. I repeat the test and i see that i > can only send about 200Mbps. I had to use multiple TCP flows to fill up m= y > 1GB link with Cubic. I run this experiment for about 2 minutes, and Cubic > is not able to catch up. I dont have long standing flows, and only need t= o > send bursts of data, hence did not leave it running for very long. > > I was under the impression, after having read the original Cubic paper > that Cubic's window growth rate is independent of RTT and hence wasnt > expecting any difference in the throughput. > > Toms. > >> >> Mirja >> >> >> > Am 21.05.2015 um 02:51 schrieb Tom Sanders : >> > >> > Hi Neal, >> > >> > This is odd. >> > >> > In the original Cubic paper ( >> http://www4.ncsu.edu/~rhee/export/bitcp/cubic-paper.pdf) by Rhee and Xu, >> they explicitly state that the Cubic is independent of RTT. >> > >> > "The main feature of CUBIC is that its window growth function is >> defined in real-time so that its growth will be independent of RTT. Our >> work was partially inspired by HTCP [5], whose window growth function is >> also based on real time. The congestion epoch period of CUBIC is determi= ned >> by the packet loss rate alone. As TCP=E2=80=99s throughput is defined by= the packet >> loss rate as well as RTT, the throughput of CUBIC is defined by only the >> packet loss rate. Thus, when the loss rate is high and/or RTT is short, >> CUBIC can operate in a TCP mode. Moreover, since the growth function is >> independent of RTT, its RTT fairness is guaranteed as different RTT flow= s >> will still grow their windows at the same rate." >> > >> > Am i missing something here? >> > >> > Thanks, Toms >> > >> > On 20 May 2015 at 23:13, Neal Cardwell wrote: >> > On Wed, May 20, 2015 at 1:18 PM, Tom Sanders >> wrote: >> > > Thanks Rick, this was very helpful. >> > > >> > > So if there are no setsockopts, etc then CUBIC's performance, unlike >> other >> > > TCP variants, should not get impacted by RTTs, right? Is that a fair >> > > statement to make? >> > >> > CUBIC, like most TCP congestion control variants, has significant >> > aspects that evolve on the scale of RTTs: for example, in slow start, >> > loss recovery, or when the CUBIC function is very steep, CUBIC's >> > behavior is heavily dependent on RTT (send some packets, wait for an >> > ACK, send some more packets,...). So CUBIC will not be >> > perfectly"RTT-fair. >> > >> > I think it might be fair to say something like: once a path is >> > saturated with CUBIC flows, as long as loss recovery or send/receive >> > buffers are not a bottleneck, then competition between CUBIC flows >> > should be more RTT-fair than Reno, in the long run, since for large >> > parts of the life cycle of the CUBIC flow the evolution of cwnd is >> > constrained by wall clock time rather than ACK arrival. >> > >> > neal >> > >> > >> > > Toms >> > > >> > > On 20 May 2015 at 19:51, Rick Jones wrote: >> > >> >> > >> >> > >> On May 20, 2015, at 6:20 AM, Tom Sanders >> wrote: >> > >> >> > >> > Hi, >> > >> > >> > >> > As per my understanding, unlike other TCP variants, CUBIC claims >> to be >> > >> > RTT fair. This is because the window growth rate is independent o= f >> the RTT, >> > >> > and hence the performance is the same as on a low and a high RTT >> paths. >> > >> > Others are dependent upon RTT since their windows grow as they >> receive ACKs >> > >> > from their peers, while CUBIC doesnt (window growth is a function >> of time). >> > >> > This is theory. >> > >> > >> > >> > In practice, CUBIC (which is the default now on most linux >> > >> > distributions) also depends upon RTT. I connected to machines on = a >> LAN and >> > >> > did a FTP transfer. I got a certain bandwidth. Next, i >> artificially inserted >> > >> > delay using "tc qdisc" in the path. I increased the delay on the >> ethernet >> > >> > interface connecting my linux machine to the LAN to be around >> 100ms. I >> > >> > immediately saw that the TCP performance went down. >> > >> > >> > >> > To bring it up to the same level as before i had to use multiple >> TCP >> > >> > streams. To me this means that CUBIC performance is dependent of >> the RTT. So >> > >> > why do we call it RTT-fair? >> > >> >> > >> I cannot speak to CUBIC specifically, but I would think you would >> want to >> > >> make sure you were seeing effects of congestion window and not of t= he >> > >> classic TCP window. For example, if the FTP client/server you were >> using >> > >> makes an explicit setsockopt() to set the socket buffer sizes and b= y >> > >> extension the classic TCP window, your bumping of the RTT to around >> 100ms >> > >> may have taken the bandwidth down as a consequence of the classic: >> > >> >> > >> Throughput <=3D WindowSize/RTT >> > >> >> > >> Similarly, since you mention Linux, if the FTP client/server didn= =E2=80=99t >> make >> > >> explicit setsockopt() calls, the sysctl values for tcp_rmem and >> tcp_wmem >> > >> may have been such that the auto tuning of the window size couldn= =E2=80=99t >> allow >> > >> the classic TCP window to grow =E2=80=9Cenough.=E2=80=9D >> > >> >> > >> I suspect that the experiment needs to be setup to have the same >> classic >> > >> window size in both cases, sized per the above to be sufficient to >> achieve >> > >> the peak throughput in the high RTT case. Then you will have >> eliminated >> > >> classic window as a factor. >> > >> >> > >> Also, depending on the version of the Linux kernel you are using, >> there >> > >> may be some effects from tcp small queues or perhaps (a stretch?) >> even byte >> > >> queue limits which may not be playing well with netem. (Just >> guessing there >> > >> really) >> > >> >> > >> happy benchmarking, >> > >> >> > >> rick jones >> > > >> > > >> > > >> > > >> > > -- >> > > Toms. >> > > >> > > _______________________________________________ >> > > tcpm mailing list >> > > tcpm@ietf.org >> > > https://www.ietf.org/mailman/listinfo/tcpm >> > > >> > >> > >> > >> > -- >> > Toms. >> > _______________________________________________ >> > tcpm mailing list >> > tcpm@ietf.org >> > https://www.ietf.org/mailman/listinfo/tcpm >> >> > > > -- > Toms. > > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm > > --089e0111d802e60f0a0516b92955 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

Hi Tom,

What did you use to create the tcp flows?

Was it iperf or similar?

Gavin

On 23 May 2015 02:16, "Tom Sanders" &l= t;toms.sanders@gmail.com> = wrote:
Hi Mirja,


the text below only says that if you have two cubic flows with a different = base RTT competing at the same bottleneck they=E2=80=99ll still converge to= share the link equally. This does not mean that if you change the RTT stro= ngly during one connection that Cubic is able to catch up fast.

In fact this is one weakness of Cubic that I have observed. If there are ch= ange in base delay/rtt or strong increases in the available capacity, Cubic= needs a rather long time to catch up. This is because cubic has a =E2=80= =9Atarget value=E2=80=98 to which it increases it sending rate/window quick= ly but then if the capacity cap is not somewhere close to this value (as ex= pected) it will probe very conservatively around this value and it takes a = rather long time until it gets back into a phase where it increases its con= gestion window fast.

Then which congest= ion control algorithm fares better there? I thought Cubic was suited for la= rge fat pipes -- bigger bandwidth-delay pipes.=C2=A0
=C2=A0
=

So what do you mean by 'I immediately saw that the TCP performance went= down=E2=80=98. How long do you run your test? And over which period of tim= e this the performance go down?

I did a= simple test.

I connect two machines on my 1GB LAN= .I do a tcp data transfer test and i get around ~900Mbps. I then added a de= lay using qdisc on my outgoing ethernet interface to simulate a longer RTT.= So, if my ping earlier was around 0.255ms, it then became 120ms. I repeat = the test and i see that i can only send about 200Mbps. I had to use multipl= e TCP flows to fill up my 1GB link with Cubic. I run this experiment for ab= out 2 minutes, and Cubic is not able to catch up. I dont have long standing= flows, and only need to send bursts of data, hence did not leave it runnin= g for very long.

I was under the impression, after= having read the original Cubic paper that Cubic's window growth rate i= s independent of RTT and hence wasnt expecting any difference in the throug= hput.

Toms.

Mirja


> Am 21.05.2015 um 02:51 schrieb Tom Sanders <toms.sanders@gmail.com>:
>
> Hi Neal,
>
> This is odd.
>
> In the original Cubic paper (http://www4.ncsu.edu/~rhee/exp= ort/bitcp/cubic-paper.pdf) by Rhee and Xu, they explicitly state that t= he Cubic is independent of RTT.
>
> "The main feature of CUBIC is that its window growth function is = defined in real-time so that its growth will be independent of RTT. Our wor= k was partially inspired by HTCP [5], whose window growth function is also = based on real time. The congestion epoch period of CUBIC is determined by t= he packet loss rate alone. As TCP=E2=80=99s throughput is defined by the pa= cket loss rate as well as RTT, the throughput of CUBIC is defined by only t= he packet loss rate. Thus, when the loss rate is high and/or RTT is short, = CUBIC can operate in a TCP mode. Moreover, since the growth function is ind= ependent of RTT, its RTT fairness is guaranteed as different RTT flows will= still grow their windows at the same rate."
>
> Am i missing something here?
>
> Thanks, Toms
>
> On 20 May 2015 at 23:13, Neal Cardwell <ncardwell@google.com> wrote:
> On Wed, May 20, 2015 at 1:18 PM, Tom Sanders <toms.sanders@gmail.com> wrote= :
> > Thanks Rick, this was very helpful.
> >
> > So if there are no setsockopts, etc then CUBIC's performance,= unlike other
> > TCP variants, should not get impacted by RTTs, right? Is that a f= air
> > statement to make?
>
> CUBIC, like most TCP congestion control variants, has significant
> aspects that evolve on the scale of RTTs: for example, in slow start,<= br> > loss recovery, or when the CUBIC function is very steep, CUBIC's > behavior is heavily dependent on RTT (send some packets, wait for an > ACK, send some more packets,...). So CUBIC will not be
> perfectly"RTT-fair.
>
> I think it might be fair to say something like: once a path is
> saturated with CUBIC flows, as long as loss recovery or send/receive > buffers are not a bottleneck, then competition between CUBIC flows
> should be more RTT-fair than Reno, in the long run, since for large > parts of the life cycle of the CUBIC flow the evolution of cwnd is
> constrained by wall clock time rather than ACK arrival.
>
> neal
>
>
> > Toms
> >
> > On 20 May 2015 at 19:51, Rick Jones <perfgeek@mac.com> wrote:
> >>
> >>
> >> On May 20, 2015, at 6:20 AM, Tom Sanders <toms.sanders@gmail.com> = wrote:
> >>
> >> > Hi,
> >> >
> >> > As per my understanding, unlike other TCP variants, CUBI= C claims to be
> >> > RTT fair. This is because the window growth rate is inde= pendent of the RTT,
> >> > and hence the performance is the same as on a low and a = high RTT paths.
> >> > Others are dependent upon RTT since their windows grow a= s they receive ACKs
> >> > from their peers, while CUBIC doesnt (window growth is a= function of time).
> >> > This is theory.
> >> >
> >> > In practice, CUBIC (which is the default now on most lin= ux
> >> > distributions) also depends upon RTT. I connected to mac= hines on a LAN and
> >> > did a FTP transfer. I got a certain bandwidth. Next, i a= rtificially inserted
> >> > delay using "tc qdisc" in the path. I increase= d the delay on the ethernet
> >> > interface connecting my linux machine to the LAN to be a= round 100ms. I
> >> > immediately saw that the TCP performance went down.
> >> >
> >> > To bring it up to the same level as before i had to use = multiple TCP
> >> > streams. To me this means that CUBIC performance is depe= ndent of the RTT. So
> >> > why do we call it RTT-fair?
> >>
> >> I cannot speak to CUBIC specifically, but I would think you w= ould want to
> >> make sure you were seeing effects of congestion window and no= t of the
> >> classic TCP window.=C2=A0 For example, if the FTP client/serv= er you were using
> >> makes an explicit setsockopt() to set the socket buffer sizes= and by
> >> extension the classic TCP window, your bumping of the RTT to = around 100ms
> >> may have taken the bandwidth down as a consequence of the cla= ssic:
> >>
> >> Throughput <=3D WindowSize/RTT
> >>
> >> Similarly, since you mention Linux, if the FTP client/server = didn=E2=80=99t make
> >> explicit setsockopt() calls,=C2=A0 the sysctl values for tcp_= rmem and tcp_wmem
> >> may have been such that the auto tuning of the window size co= uldn=E2=80=99t allow
> >> the classic TCP window to grow =E2=80=9Cenough.=E2=80=9D
> >>
> >> I suspect that the experiment needs to be setup to have the s= ame classic
> >> window size in both cases, sized per the above to be sufficie= nt to achieve
> >> the peak throughput in the high RTT case.=C2=A0 Then you will= have eliminated
> >> classic window as a factor.
> >>
> >> Also, depending on the version of the Linux kernel you are us= ing, there
> >> may be some effects from tcp small queues or perhaps (a stret= ch?) even byte
> >> queue limits which may not be playing well with netem.=C2=A0 = (Just guessing there
> >> really)
> >>
> >> happy benchmarking,
> >>
> >> rick jones
> >
> >
> >
> >
> > --
> > Toms.
> >
> > _______________________________________________
> > tcpm mailing list
> > tcpm@ietf.org<= /a>
> > https://www.ietf.org/mailman/listinfo/tcpm
> >
>
>
>
> --
> Toms.
> _______________________________________________
> tcpm mailing list
> tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm




--
=
Toms.

_______________________________________________
tcpm mailing list
tcpm@ietf.org
ht= tps://www.ietf.org/mailman/listinfo/tcpm

--089e0111d802e60f0a0516b92955-- From nobody Sat May 23 01:53:13 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 138411AC529 for ; Sat, 23 May 2015 01:53:12 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 3.888 X-Spam-Level: *** X-Spam-Status: No, score=3.888 tagged_above=-999 required=5 tests=[BAYES_40=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, FRT_STOCK2=3.988, HTML_MESSAGE=0.001, SPF_PASS=-0.001] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id yNgSl2V4OVCK for ; Sat, 23 May 2015 01:53:08 -0700 (PDT) Received: from mail-wi0-x22e.google.com (mail-wi0-x22e.google.com [IPv6:2a00:1450:400c:c05::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 6A4301AC43D for ; Sat, 23 May 2015 01:53:07 -0700 (PDT) Received: by wichy4 with SMTP id hy4so8464217wic.1 for ; Sat, 23 May 2015 01:53:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=TUbjKcZU2I6XpE97ihBVeH3iiJQQizWXSdV6c4Ql/no=; b=BuJmCaV3JpBr6/Q92ygNa9k23F7sZKvQiFMV68DAdBwELDDPVjDr4qnbQlfov+x69n NGNzn+RFBLidHFJVTyzL16mwBQ/74N/V0rOOGXNx7Q/ph58VaDaoWLaIPvKvK/Km1u2O evumYNkloxhW4S3wgif3KRExjVKBAypOHYbC89H6l4EGMZqL77P1Ytduh9otgAl6Eiy/ el4nFI9wHoILhe/613NVuBKZ3EujsLbm7xOoosd4eWYPRRY2hcpPKmCGqWsLJR2IiKqP cBQ8hpuhBsKhlwH8ujF2UFZqkoYEmqi/kvRMYjaln4jgRi0veCZqfu7iOf9sxbkPGt/j x5QA== MIME-Version: 1.0 X-Received: by 10.180.74.68 with SMTP id r4mr14674447wiv.69.1432371186113; Sat, 23 May 2015 01:53:06 -0700 (PDT) Received: by 10.28.133.148 with HTTP; Sat, 23 May 2015 01:53:05 -0700 (PDT) In-Reply-To: References: <7C415879-6BF7-403D-B904-CC2B4D32CA04@mac.com> Date: Sat, 23 May 2015 14:23:05 +0530 Message-ID: From: Tom Sanders To: Gavin McCullagh Content-Type: multipart/alternative; boundary=f46d043d67e5bc23b10516bbe731 Archived-At: Cc: "tcpm@ietf.org" Subject: Re: [tcpm] Why is Cubic said to be RTT-fair? X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 23 May 2015 08:53:12 -0000 --f46d043d67e5bc23b10516bbe731 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable You can use iperf with parallel streams and the result is the same. In my case it was my own client/server pushing data over the TCP socket. I have a theory that the following happens: When i artificially induce the delay there are some packet losses since the outgoing ethernet interface has to buffer the packets (to simulate the link delay). Its probably of these packet losses that the throughput that i get with one TCP stream is much lower than the available bandwidth. When i increase the number of TCP streams, for reasons well known, i get better bandwidth. I think, and correct me if i am wrong here, if i were to induce latency WITHOUT packet losses then even one TCP stream would be able to fill up the network pipe (assuming that the kernel buffers are not the limiting issue). Thanks, Toms On 23 May 2015 at 11:06, Gavin McCullagh wrote: > Hi Tom, > > What did you use to create the tcp flows? > > Was it iperf or similar? > > Gavin > On 23 May 2015 02:16, "Tom Sanders" wrote: > >> Hi Mirja, >> >>> >>> >>> the text below only says that if you have two cubic flows with a >>> different base RTT competing at the same bottleneck they=E2=80=99ll sti= ll converge >>> to share the link equally. This does not mean that if you change the RT= T >>> strongly during one connection that Cubic is able to catch up fast. >>> >>> In fact this is one weakness of Cubic that I have observed. If there ar= e >>> change in base delay/rtt or strong increases in the available capacity, >>> Cubic needs a rather long time to catch up. This is because cubic has a >>> =E2=80=9Atarget value=E2=80=98 to which it increases it sending rate/wi= ndow quickly but >>> then if the capacity cap is not somewhere close to this value (as expec= ted) >>> it will probe very conservatively around this value and it takes a rath= er >>> long time until it gets back into a phase where it increases its conges= tion >>> window fast. >>> >> >> Then which congestion control algorithm fares better there? I thought >> Cubic was suited for large fat pipes -- bigger bandwidth-delay pipes. >> >> >>> >>> So what do you mean by 'I immediately saw that the TCP performance went >>> down=E2=80=98. How long do you run your test? And over which period of = time this >>> the performance go down? >>> >> >> I did a simple test. >> >> I connect two machines on my 1GB LAN.I do a tcp data transfer test and i >> get around ~900Mbps. I then added a delay using qdisc on my outgoing >> ethernet interface to simulate a longer RTT. So, if my ping earlier was >> around 0.255ms, it then became 120ms. I repeat the test and i see that i >> can only send about 200Mbps. I had to use multiple TCP flows to fill up = my >> 1GB link with Cubic. I run this experiment for about 2 minutes, and Cubi= c >> is not able to catch up. I dont have long standing flows, and only need = to >> send bursts of data, hence did not leave it running for very long. >> >> I was under the impression, after having read the original Cubic paper >> that Cubic's window growth rate is independent of RTT and hence wasnt >> expecting any difference in the throughput. >> >> Toms. >> >>> >>> Mirja >>> >>> >>> > Am 21.05.2015 um 02:51 schrieb Tom Sanders : >>> > >>> > Hi Neal, >>> > >>> > This is odd. >>> > >>> > In the original Cubic paper ( >>> http://www4.ncsu.edu/~rhee/export/bitcp/cubic-paper.pdf) by Rhee and >>> Xu, they explicitly state that the Cubic is independent of RTT. >>> > >>> > "The main feature of CUBIC is that its window growth function is >>> defined in real-time so that its growth will be independent of RTT. Our >>> work was partially inspired by HTCP [5], whose window growth function i= s >>> also based on real time. The congestion epoch period of CUBIC is determ= ined >>> by the packet loss rate alone. As TCP=E2=80=99s throughput is defined b= y the packet >>> loss rate as well as RTT, the throughput of CUBIC is defined by only th= e >>> packet loss rate. Thus, when the loss rate is high and/or RTT is short, >>> CUBIC can operate in a TCP mode. Moreover, since the growth function is >>> independent of RTT, its RTT fairness is guaranteed as different RTT flo= ws >>> will still grow their windows at the same rate." >>> > >>> > Am i missing something here? >>> > >>> > Thanks, Toms >>> > >>> > On 20 May 2015 at 23:13, Neal Cardwell wrote: >>> > On Wed, May 20, 2015 at 1:18 PM, Tom Sanders >>> wrote: >>> > > Thanks Rick, this was very helpful. >>> > > >>> > > So if there are no setsockopts, etc then CUBIC's performance, unlik= e >>> other >>> > > TCP variants, should not get impacted by RTTs, right? Is that a fai= r >>> > > statement to make? >>> > >>> > CUBIC, like most TCP congestion control variants, has significant >>> > aspects that evolve on the scale of RTTs: for example, in slow start, >>> > loss recovery, or when the CUBIC function is very steep, CUBIC's >>> > behavior is heavily dependent on RTT (send some packets, wait for an >>> > ACK, send some more packets,...). So CUBIC will not be >>> > perfectly"RTT-fair. >>> > >>> > I think it might be fair to say something like: once a path is >>> > saturated with CUBIC flows, as long as loss recovery or send/receive >>> > buffers are not a bottleneck, then competition between CUBIC flows >>> > should be more RTT-fair than Reno, in the long run, since for large >>> > parts of the life cycle of the CUBIC flow the evolution of cwnd is >>> > constrained by wall clock time rather than ACK arrival. >>> > >>> > neal >>> > >>> > >>> > > Toms >>> > > >>> > > On 20 May 2015 at 19:51, Rick Jones wrote: >>> > >> >>> > >> >>> > >> On May 20, 2015, at 6:20 AM, Tom Sanders >>> wrote: >>> > >> >>> > >> > Hi, >>> > >> > >>> > >> > As per my understanding, unlike other TCP variants, CUBIC claims >>> to be >>> > >> > RTT fair. This is because the window growth rate is independent >>> of the RTT, >>> > >> > and hence the performance is the same as on a low and a high RTT >>> paths. >>> > >> > Others are dependent upon RTT since their windows grow as they >>> receive ACKs >>> > >> > from their peers, while CUBIC doesnt (window growth is a functio= n >>> of time). >>> > >> > This is theory. >>> > >> > >>> > >> > In practice, CUBIC (which is the default now on most linux >>> > >> > distributions) also depends upon RTT. I connected to machines on >>> a LAN and >>> > >> > did a FTP transfer. I got a certain bandwidth. Next, i >>> artificially inserted >>> > >> > delay using "tc qdisc" in the path. I increased the delay on the >>> ethernet >>> > >> > interface connecting my linux machine to the LAN to be around >>> 100ms. I >>> > >> > immediately saw that the TCP performance went down. >>> > >> > >>> > >> > To bring it up to the same level as before i had to use multiple >>> TCP >>> > >> > streams. To me this means that CUBIC performance is dependent of >>> the RTT. So >>> > >> > why do we call it RTT-fair? >>> > >> >>> > >> I cannot speak to CUBIC specifically, but I would think you would >>> want to >>> > >> make sure you were seeing effects of congestion window and not of >>> the >>> > >> classic TCP window. For example, if the FTP client/server you wer= e >>> using >>> > >> makes an explicit setsockopt() to set the socket buffer sizes and = by >>> > >> extension the classic TCP window, your bumping of the RTT to aroun= d >>> 100ms >>> > >> may have taken the bandwidth down as a consequence of the classic: >>> > >> >>> > >> Throughput <=3D WindowSize/RTT >>> > >> >>> > >> Similarly, since you mention Linux, if the FTP client/server didn= =E2=80=99t >>> make >>> > >> explicit setsockopt() calls, the sysctl values for tcp_rmem and >>> tcp_wmem >>> > >> may have been such that the auto tuning of the window size couldn= =E2=80=99t >>> allow >>> > >> the classic TCP window to grow =E2=80=9Cenough.=E2=80=9D >>> > >> >>> > >> I suspect that the experiment needs to be setup to have the same >>> classic >>> > >> window size in both cases, sized per the above to be sufficient to >>> achieve >>> > >> the peak throughput in the high RTT case. Then you will have >>> eliminated >>> > >> classic window as a factor. >>> > >> >>> > >> Also, depending on the version of the Linux kernel you are using, >>> there >>> > >> may be some effects from tcp small queues or perhaps (a stretch?) >>> even byte >>> > >> queue limits which may not be playing well with netem. (Just >>> guessing there >>> > >> really) >>> > >> >>> > >> happy benchmarking, >>> > >> >>> > >> rick jones >>> > > >>> > > >>> > > >>> > > >>> > > -- >>> > > Toms. >>> > > >>> > > _______________________________________________ >>> > > tcpm mailing list >>> > > tcpm@ietf.org >>> > > https://www.ietf.org/mailman/listinfo/tcpm >>> > > >>> > >>> > >>> > >>> > -- >>> > Toms. >>> > _______________________________________________ >>> > tcpm mailing list >>> > tcpm@ietf.org >>> > https://www.ietf.org/mailman/listinfo/tcpm >>> >>> >> >> >> -- >> Toms. >> >> _______________________________________________ >> tcpm mailing list >> tcpm@ietf.org >> https://www.ietf.org/mailman/listinfo/tcpm >> >> --=20 Toms. --f46d043d67e5bc23b10516bbe731 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
You can use iperf with parallel streams and the result is = the same. In my case it was my own client/server pushing data over the TCP = socket.

I have a theory that the following happens:

When i artificially induce the delay there are some pa= cket losses since the outgoing ethernet interface has to buffer the packets= (to simulate the link delay). Its probably of these packet losses that the= throughput that i get with one TCP stream is much lower than the available= bandwidth. When i increase the number of TCP streams, for reasons well kno= wn, i get better bandwidth.

I think, and correct m= e if i am wrong here, if i were to induce latency WITHOUT packet losses the= n even one TCP stream would be able to fill up the network pipe (assuming t= hat the kernel buffers are not the limiting issue).

Thanks, Toms

On 23 May 2015 at 11:06, Gavin McCullagh <gmccullagh@gmail.com= > wrote:

Hi = Tom,

What did you use to create the tcp flows?

Was it iperf or similar?

Gavin

On 23 May 2015 02:16, "Tom Sanders" &l= t;toms.sanders@= gmail.com> wrote:
Hi Mirja,


the text below only says that if you have two cubic flows with a different = base RTT competing at the same bottleneck they=E2=80=99ll still converge to= share the link equally. This does not mean that if you change the RTT stro= ngly during one connection that Cubic is able to catch up fast.

In fact this is one weakness of Cubic that I have observed. If there are ch= ange in base delay/rtt or strong increases in the available capacity, Cubic= needs a rather long time to catch up. This is because cubic has a =E2=80= =9Atarget value=E2=80=98 to which it increases it sending rate/window quick= ly but then if the capacity cap is not somewhere close to this value (as ex= pected) it will probe very conservatively around this value and it takes a = rather long time until it gets back into a phase where it increases its con= gestion window fast.

Then which congest= ion control algorithm fares better there? I thought Cubic was suited for la= rge fat pipes -- bigger bandwidth-delay pipes.=C2=A0
=C2=A0
=

So what do you mean by 'I immediately saw that the TCP performance went= down=E2=80=98. How long do you run your test? And over which period of tim= e this the performance go down?

I did a= simple test.

I connect two machines on my 1GB LAN= .I do a tcp data transfer test and i get around ~900Mbps. I then added a de= lay using qdisc on my outgoing ethernet interface to simulate a longer RTT.= So, if my ping earlier was around 0.255ms, it then became 120ms. I repeat = the test and i see that i can only send about 200Mbps. I had to use multipl= e TCP flows to fill up my 1GB link with Cubic. I run this experiment for ab= out 2 minutes, and Cubic is not able to catch up. I dont have long standing= flows, and only need to send bursts of data, hence did not leave it runnin= g for very long.

I was under the impression, after= having read the original Cubic paper that Cubic's window growth rate i= s independent of RTT and hence wasnt expecting any difference in the throug= hput.

Toms.

Mirja


> Am 21.05.2015 um 02:51 schrieb Tom Sanders <toms.sanders@gmail.com>:
>
> Hi Neal,
>
> This is odd.
>
> In the original Cubic paper (http://www4.ncsu.edu/~rhee/exp= ort/bitcp/cubic-paper.pdf) by Rhee and Xu, they explicitly state that t= he Cubic is independent of RTT.
>
> "The main feature of CUBIC is that its window growth function is = defined in real-time so that its growth will be independent of RTT. Our wor= k was partially inspired by HTCP [5], whose window growth function is also = based on real time. The congestion epoch period of CUBIC is determined by t= he packet loss rate alone. As TCP=E2=80=99s throughput is defined by the pa= cket loss rate as well as RTT, the throughput of CUBIC is defined by only t= he packet loss rate. Thus, when the loss rate is high and/or RTT is short, = CUBIC can operate in a TCP mode. Moreover, since the growth function is ind= ependent of RTT, its RTT fairness is guaranteed as different RTT flows will= still grow their windows at the same rate."
>
> Am i missing something here?
>
> Thanks, Toms
>
> On 20 May 2015 at 23:13, Neal Cardwell <ncardwell@google.com> wrote:
> On Wed, May 20, 2015 at 1:18 PM, Tom Sanders <toms.sanders@gmail.com> wrote= :
> > Thanks Rick, this was very helpful.
> >
> > So if there are no setsockopts, etc then CUBIC's performance,= unlike other
> > TCP variants, should not get impacted by RTTs, right? Is that a f= air
> > statement to make?
>
> CUBIC, like most TCP congestion control variants, has significant
> aspects that evolve on the scale of RTTs: for example, in slow start,<= br> > loss recovery, or when the CUBIC function is very steep, CUBIC's > behavior is heavily dependent on RTT (send some packets, wait for an > ACK, send some more packets,...). So CUBIC will not be
> perfectly"RTT-fair.
>
> I think it might be fair to say something like: once a path is
> saturated with CUBIC flows, as long as loss recovery or send/receive > buffers are not a bottleneck, then competition between CUBIC flows
> should be more RTT-fair than Reno, in the long run, since for large > parts of the life cycle of the CUBIC flow the evolution of cwnd is
> constrained by wall clock time rather than ACK arrival.
>
> neal
>
>
> > Toms
> >
> > On 20 May 2015 at 19:51, Rick Jones <perfgeek@mac.com> wrote:
> >>
> >>
> >> On May 20, 2015, at 6:20 AM, Tom Sanders <toms.sanders@gmail.com> = wrote:
> >>
> >> > Hi,
> >> >
> >> > As per my understanding, unlike other TCP variants, CUBI= C claims to be
> >> > RTT fair. This is because the window growth rate is inde= pendent of the RTT,
> >> > and hence the performance is the same as on a low and a = high RTT paths.
> >> > Others are dependent upon RTT since their windows grow a= s they receive ACKs
> >> > from their peers, while CUBIC doesnt (window growth is a= function of time).
> >> > This is theory.
> >> >
> >> > In practice, CUBIC (which is the default now on most lin= ux
> >> > distributions) also depends upon RTT. I connected to mac= hines on a LAN and
> >> > did a FTP transfer. I got a certain bandwidth. Next, i a= rtificially inserted
> >> > delay using "tc qdisc" in the path. I increase= d the delay on the ethernet
> >> > interface connecting my linux machine to the LAN to be a= round 100ms. I
> >> > immediately saw that the TCP performance went down.
> >> >
> >> > To bring it up to the same level as before i had to use = multiple TCP
> >> > streams. To me this means that CUBIC performance is depe= ndent of the RTT. So
> >> > why do we call it RTT-fair?
> >>
> >> I cannot speak to CUBIC specifically, but I would think you w= ould want to
> >> make sure you were seeing effects of congestion window and no= t of the
> >> classic TCP window.=C2=A0 For example, if the FTP client/serv= er you were using
> >> makes an explicit setsockopt() to set the socket buffer sizes= and by
> >> extension the classic TCP window, your bumping of the RTT to = around 100ms
> >> may have taken the bandwidth down as a consequence of the cla= ssic:
> >>
> >> Throughput <=3D WindowSize/RTT
> >>
> >> Similarly, since you mention Linux, if the FTP client/server = didn=E2=80=99t make
> >> explicit setsockopt() calls,=C2=A0 the sysctl values for tcp_= rmem and tcp_wmem
> >> may have been such that the auto tuning of the window size co= uldn=E2=80=99t allow
> >> the classic TCP window to grow =E2=80=9Cenough.=E2=80=9D
> >>
> >> I suspect that the experiment needs to be setup to have the s= ame classic
> >> window size in both cases, sized per the above to be sufficie= nt to achieve
> >> the peak throughput in the high RTT case.=C2=A0 Then you will= have eliminated
> >> classic window as a factor.
> >>
> >> Also, depending on the version of the Linux kernel you are us= ing, there
> >> may be some effects from tcp small queues or perhaps (a stret= ch?) even byte
> >> queue limits which may not be playing well with netem.=C2=A0 = (Just guessing there
> >> really)
> >>
> >> happy benchmarking,
> >>
> >> rick jones
> >
> >
> >
> >
> > --
> > Toms.
> >
> > _______________________________________________
> > tcpm mailing list
> > tcpm@ietf.org<= /a>
> > https://www.ietf.org/mailman/listinfo/tcpm
> >
>
>
>
> --
> Toms.
> _______________________________________________
> tcpm mailing list
> tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm




--
=
Toms.

_______________________________________________
tcpm mailing list
tcpm@ietf.org
ht= tps://www.ietf.org/mailman/listinfo/tcpm




--
=
Toms.
--f46d043d67e5bc23b10516bbe731-- From nobody Sat May 23 08:36:22 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E562A1A0125 for ; Sat, 23 May 2015 08:36:20 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 4.789 X-Spam-Level: **** X-Spam-Status: No, score=4.789 tagged_above=-999 required=5 tests=[BAYES_50=0.8, FREEMAIL_FROM=0.001, FRT_STOCK2=3.988, MALFORMED_FREEMAIL=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id OIiWs4UZSTsv for ; Sat, 23 May 2015 08:36:16 -0700 (PDT) Received: from st11p05im-asmtp001.me.com (st11p05im-asmtp001.me.com [17.172.109.149]) (using TLSv1.2 with cipher DHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id CA6161A010C for ; Sat, 23 May 2015 08:36:16 -0700 (PDT) Received: from [192.168.1.65] (76-220-59-128.lightspeed.sntcca.sbcglobal.net [76.220.59.128]) by st11p05im-asmtp001.me.com (Oracle Communications Messaging Server 7.0.5.35.0 64bit (built Dec 4 2014)) with ESMTPSA id <0NOT00IJZ7CDVY20@st11p05im-asmtp001.me.com> for tcpm@ietf.org; Sat, 23 May 2015 15:36:16 +0000 (GMT) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.14.151,1.0.33,0.0.0000 definitions=2015-05-23_01:2015-05-22,2015-05-23,1970-01-01 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 suspectscore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=7.0.1-1412110000 definitions=main-1505230211 Content-type: text/plain; charset=windows-1252 MIME-version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) From: Rick Jones In-reply-to: Date: Sat, 23 May 2015 08:36:12 -0700 Content-transfer-encoding: quoted-printable Message-id: <10C985A4-9A5F-497A-B991-56E8EFDE9CBA@mac.com> References: <7C415879-6BF7-403D-B904-CC2B4D32CA04@mac.com> To: Tom Sanders X-Mailer: Apple Mail (2.1878.6) Archived-At: Cc: "tcpm@ietf.org" Subject: Re: [tcpm] Why is Cubic said to be RTT-fair? X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 23 May 2015 15:36:21 -0000 On May 23, 2015, at 1:53 AM, Tom Sanders wrote: > You can use iperf with parallel streams and the result is the same. In = my case it was my own client/server pushing data over the TCP socket. >=20 > I have a theory that the following happens: >=20 > When i artificially induce the delay there are some packet losses = since the outgoing ethernet interface has to buffer the packets (to = simulate the link delay). Its probably of these packet losses that the = throughput that i get with one TCP stream is much lower than the = available bandwidth. When i increase the number of TCP streams, for = reasons well known, i get better bandwidth. >=20 > I think, and correct me if i am wrong here, if i were to induce = latency WITHOUT packet losses then even one TCP stream would be able to = fill up the network pipe (assuming that the kernel buffers are not the = limiting issue). Not to be overly pedantic, but have you verified the packet losses - = either through a packet trace on (I would think) the receiver or by = looking at the netstat statistics on the sender before and after your = run(s)? Also, have you validated that indeed there is enough =93classic=94= TCP window to allow for 900 Mbit/s on a single stream at 120ms RTT? My = rough calculation based on: Throughput <=3D WindowSize/RTT 900 Mbit/s <=3D WindowSize/0.120s 108 Mbit <=3D WindowSize suggests a single stream with a 120 ms RTT would require a classic TCP = window of 13,500,000 bytes or more even without any packet losses. I = don=92t think I=92ve seen a net.ipv4.tcp_[rw]mem which defaulted to more = than 4-6MB (M =3D=3D 1048576) on the Linux systems I=92ve seen = regardless of how much memory they had in them. happy benchmarking, rick jones >=20 > Thanks, Toms >=20 > On 23 May 2015 at 11:06, Gavin McCullagh wrote: > Hi Tom, >=20 > What did you use to create the tcp flows? >=20 > Was it iperf or similar? >=20 > Gavin >=20 > On 23 May 2015 02:16, "Tom Sanders" wrote: > Hi Mirja, >=20 >=20 > the text below only says that if you have two cubic flows with a = different base RTT competing at the same bottleneck they=92ll still = converge to share the link equally. This does not mean that if you = change the RTT strongly during one connection that Cubic is able to = catch up fast. >=20 > In fact this is one weakness of Cubic that I have observed. If there = are change in base delay/rtt or strong increases in the available = capacity, Cubic needs a rather long time to catch up. This is because = cubic has a =82target value=91 to which it increases it sending = rate/window quickly but then if the capacity cap is not somewhere close = to this value (as expected) it will probe very conservatively around = this value and it takes a rather long time until it gets back into a = phase where it increases its congestion window fast. >=20 > Then which congestion control algorithm fares better there? I thought = Cubic was suited for large fat pipes -- bigger bandwidth-delay pipes.=20 > =20 >=20 > So what do you mean by 'I immediately saw that the TCP performance = went down=91. How long do you run your test? And over which period of = time this the performance go down? >=20 > I did a simple test. >=20 > I connect two machines on my 1GB LAN.I do a tcp data transfer test and = i get around ~900Mbps. I then added a delay using qdisc on my outgoing = ethernet interface to simulate a longer RTT. So, if my ping earlier was = around 0.255ms, it then became 120ms. I repeat the test and i see that i = can only send about 200Mbps. I had to use multiple TCP flows to fill up = my 1GB link with Cubic. I run this experiment for about 2 minutes, and = Cubic is not able to catch up. I dont have long standing flows, and only = need to send bursts of data, hence did not leave it running for very = long. >=20 > I was under the impression, after having read the original Cubic paper = that Cubic's window growth rate is independent of RTT and hence wasnt = expecting any difference in the throughput. >=20 > Toms. >=20 > Mirja >=20 >=20 > > Am 21.05.2015 um 02:51 schrieb Tom Sanders : > > > > Hi Neal, > > > > This is odd. > > > > In the original Cubic paper = (http://www4.ncsu.edu/~rhee/export/bitcp/cubic-paper.pdf) by Rhee and = Xu, they explicitly state that the Cubic is independent of RTT. > > > > "The main feature of CUBIC is that its window growth function is = defined in real-time so that its growth will be independent of RTT. Our = work was partially inspired by HTCP [5], whose window growth function is = also based on real time. The congestion epoch period of CUBIC is = determined by the packet loss rate alone. As TCP=92s throughput is = defined by the packet loss rate as well as RTT, the throughput of CUBIC = is defined by only the packet loss rate. Thus, when the loss rate is = high and/or RTT is short, CUBIC can operate in a TCP mode. Moreover, = since the growth function is independent of RTT, its RTT fairness is = guaranteed as different RTT flows will still grow their windows at the = same rate." > > > > Am i missing something here? > > > > Thanks, Toms > > > > On 20 May 2015 at 23:13, Neal Cardwell wrote: > > On Wed, May 20, 2015 at 1:18 PM, Tom Sanders = wrote: > > > Thanks Rick, this was very helpful. > > > > > > So if there are no setsockopts, etc then CUBIC's performance, = unlike other > > > TCP variants, should not get impacted by RTTs, right? Is that a = fair > > > statement to make? > > > > CUBIC, like most TCP congestion control variants, has significant > > aspects that evolve on the scale of RTTs: for example, in slow = start, > > loss recovery, or when the CUBIC function is very steep, CUBIC's > > behavior is heavily dependent on RTT (send some packets, wait for an > > ACK, send some more packets,...). So CUBIC will not be > > perfectly"RTT-fair. > > > > I think it might be fair to say something like: once a path is > > saturated with CUBIC flows, as long as loss recovery or send/receive > > buffers are not a bottleneck, then competition between CUBIC flows > > should be more RTT-fair than Reno, in the long run, since for large > > parts of the life cycle of the CUBIC flow the evolution of cwnd is > > constrained by wall clock time rather than ACK arrival. > > > > neal > > > > > > > Toms > > > > > > On 20 May 2015 at 19:51, Rick Jones wrote: > > >> > > >> > > >> On May 20, 2015, at 6:20 AM, Tom Sanders = wrote: > > >> > > >> > Hi, > > >> > > > >> > As per my understanding, unlike other TCP variants, CUBIC = claims to be > > >> > RTT fair. This is because the window growth rate is independent = of the RTT, > > >> > and hence the performance is the same as on a low and a high = RTT paths. > > >> > Others are dependent upon RTT since their windows grow as they = receive ACKs > > >> > from their peers, while CUBIC doesnt (window growth is a = function of time). > > >> > This is theory. > > >> > > > >> > In practice, CUBIC (which is the default now on most linux > > >> > distributions) also depends upon RTT. I connected to machines = on a LAN and > > >> > did a FTP transfer. I got a certain bandwidth. Next, i = artificially inserted > > >> > delay using "tc qdisc" in the path. I increased the delay on = the ethernet > > >> > interface connecting my linux machine to the LAN to be around = 100ms. I > > >> > immediately saw that the TCP performance went down. > > >> > > > >> > To bring it up to the same level as before i had to use = multiple TCP > > >> > streams. To me this means that CUBIC performance is dependent = of the RTT. So > > >> > why do we call it RTT-fair? > > >> > > >> I cannot speak to CUBIC specifically, but I would think you would = want to > > >> make sure you were seeing effects of congestion window and not of = the > > >> classic TCP window. For example, if the FTP client/server you = were using > > >> makes an explicit setsockopt() to set the socket buffer sizes and = by > > >> extension the classic TCP window, your bumping of the RTT to = around 100ms > > >> may have taken the bandwidth down as a consequence of the = classic: > > >> > > >> Throughput <=3D WindowSize/RTT > > >> > > >> Similarly, since you mention Linux, if the FTP client/server = didn=92t make > > >> explicit setsockopt() calls, the sysctl values for tcp_rmem and = tcp_wmem > > >> may have been such that the auto tuning of the window size = couldn=92t allow > > >> the classic TCP window to grow =93enough.=94 > > >> > > >> I suspect that the experiment needs to be setup to have the same = classic > > >> window size in both cases, sized per the above to be sufficient = to achieve > > >> the peak throughput in the high RTT case. Then you will have = eliminated > > >> classic window as a factor. > > >> > > >> Also, depending on the version of the Linux kernel you are using, = there > > >> may be some effects from tcp small queues or perhaps (a stretch?) = even byte > > >> queue limits which may not be playing well with netem. (Just = guessing there > > >> really) > > >> > > >> happy benchmarking, > > >> > > >> rick jones > > > > > > > > > > > > > > > -- > > > Toms. > > > > > > _______________________________________________ > > > tcpm mailing list > > > tcpm@ietf.org > > > https://www.ietf.org/mailman/listinfo/tcpm > > > > > > > > > > > -- > > Toms. > > _______________________________________________ > > tcpm mailing list > > tcpm@ietf.org > > https://www.ietf.org/mailman/listinfo/tcpm >=20 >=20 >=20 >=20 > --=20 > Toms. >=20 > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm >=20 >=20 >=20 >=20 > --=20 > Toms. > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm From nobody Tue May 26 04:38:15 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3BD8A1A8704 for ; Tue, 26 May 2015 04:38:14 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.5 X-Spam-Level: X-Spam-Status: No, score=-1.5 tagged_above=-999 required=5 tests=[BAYES_50=0.8, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JAJp3Z9Dz29B for ; Tue, 26 May 2015 04:38:12 -0700 (PDT) Received: from sessmg22.ericsson.net (sessmg22.ericsson.net [193.180.251.58]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 788421A86F1 for ; Tue, 26 May 2015 04:38:11 -0700 (PDT) X-AuditID: c1b4fb3a-f79ec6d000006dc0-13-55645b21b782 Received: from ESESSHC006.ericsson.se (Unknown_Domain [153.88.253.125]) by sessmg22.ericsson.net (Symantec Mail Security) with SMTP id 7B.A7.28096.12B54655; Tue, 26 May 2015 13:38:09 +0200 (CEST) Received: from ESESSMB205.ericsson.se ([169.254.5.148]) by ESESSHC006.ericsson.se ([153.88.183.36]) with mapi id 14.03.0210.002; Tue, 26 May 2015 13:38:08 +0200 From: Ingemar Johansson S To: "tcpm@ietf.org" Thread-Topic: TCP HyStart patch deployment Thread-Index: AdCXp+zNDAZ7zydkTra0wALqZYyJLQ== Date: Tue, 26 May 2015 11:38:08 +0000 Message-ID: <81564C0D7D4D2A4B9A86C8C7404A13DA32145DE4@ESESSMB205.ericsson.se> Accept-Language: sv-SE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [153.88.183.147] Content-Type: multipart/alternative; boundary="_000_81564C0D7D4D2A4B9A86C8C7404A13DA32145DE4ESESSMB205erics_" MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrDLMWRmVeSWpSXmKPExsUyM+Jvra5idEqowad+PYvd56ayWGw7OZ/J gcljyZKfTB4dxzcwBzBFcdmkpOZklqUW6dslcGWcXfaRueCoc8WLFp4Gxm12XYycHBICJhJL rz9gh7DFJC7cW8/WxcjFISRwlFHi/89ORghnCaNE+7U/jCBVbAI2EisPfQezRQSUJVbf/wBm MwvYS1xp6GYCsYUFVCVe/Z0NVaMlcXdGAyuErSdxfU4nWA0LUM2JjrcsIDavgK/Ey6//2EBs RgFZifvf77FAzBSXuPVkPhPEdQISS/acZ4awRSVePv4HNJMDyFaSmLY1DaI8X+LItxtMECMF JU7OfMIygVF4FpJJs5CUzUJSBhHXk7gxdQobhK0tsWzha2YIW1dixr9DLMjiCxjZVzGKFqcW F+emGxnppRZlJhcX5+fp5aWWbGIERs/BLb+tdjAefO54iFGAg1GJh3fB5eRQIdbEsuLK3EOM 0hwsSuK8nl0hoUIC6YklqdmpqQWpRfFFpTmpxYcYmTg4pRoYeR5NMXFddOH+3F+3eNauF1s5 411itGtB38XzXeqvWgJYXpifFunTrtW93BspNVfDtM1/0qlPiRWxvz0nXpLvf5o+z6r+vZQv x2uWbzL6WVypT9S8ZDTvnDm3+1q7fkMD99yCNO/N95uUTaPuMWT78k6oceJ5n5AccnPl8gXC 9sILT79fPmW2EktxRqKhFnNRcSIAeTquMH8CAAA= Archived-At: Cc: "end2end-interest@postel.org" Subject: [tcpm] TCP HyStart patch deployment X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 May 2015 11:38:14 -0000 --_000_81564C0D7D4D2A4B9A86C8C7404A13DA32145DE4ESESSMB205erics_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hi Does anybody know if the TCP Cubic HyStart patch described in the link belo= w is being used widely ? It does not seem to be implemented in the latest Linux code. http://patchwork.ozlabs.org/patch/85945/ Regards Ingemar =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D Ingemar Johansson M.Sc. Senior Researcher Ericsson AB Wireless Access Networks Labratoriegr=E4nd 11 971 28, Lule=E5, Sweden Phone +46-1071 43042 SMS/MMS +46-73 078 3289 ingemar.s.johansson@ericsson.com www.ericsson.com "No man has a good enough memory to be a successful liar" Abraham Lincoln =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D --_000_81564C0D7D4D2A4B9A86C8C7404A13DA32145DE4ESESSMB205erics_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

Hi

 

Does anybody know if the TCP Cu= bic HyStart patch described in the link below is being used widely ?

It does not seem to be implemen= ted in the latest Linux code.

http://patchwork.ozlabs.org/patch/85945/

 

Regards

Ingemar

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D

Ingemar Johansson  M.Sc.

Senior Researcher

 

Ericsson AB

Wireless Access Networks

Labratoriegr=E4nd 11

971 28, Lule=E5, Sweden

Phone +46-1071 43042

SMS/MMS +46-73 078 3289

ingemar.s.johansson@ericsson.com= =

www.ericsson.com

 

“No man has a good enough memory

  to be = a successful liar&= #8221;
    Abraham Lincoln
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D

 

--_000_81564C0D7D4D2A4B9A86C8C7404A13DA32145DE4ESESSMB205erics_-- From nobody Tue May 26 05:48:11 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E8AA31B2B36 for ; Tue, 26 May 2015 05:48:09 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.389 X-Spam-Level: X-Spam-Status: No, score=-1.389 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FM_FORGED_GMAIL=0.622, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id cokai_BSH0Qp for ; Tue, 26 May 2015 05:48:08 -0700 (PDT) Received: from mail-oi0-x22d.google.com (mail-oi0-x22d.google.com [IPv6:2607:f8b0:4003:c06::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id C68C11B2B33 for ; Tue, 26 May 2015 05:48:08 -0700 (PDT) Received: by oiww2 with SMTP id w2so76653210oiw.0 for ; Tue, 26 May 2015 05:48:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=V1kHFp8GkrESvVr5Vv2w+XNL8Vmtl973b1nN3epOUpM=; b=DAQMOtXVE6nAD/RCj4+1miVUSPxoG478ZwmT/wRmEEGr1hFPtFCbn3a/vDbg/kDsCA tPg0n9Mdl8sVzAxJOXlmrKnHYiYoLNtf+PWsZaytmGo79aQKIoSBe125lhppUUaDOnEB u97F7qFsQeKr9ESLz+aT4htFw7zfbPiXtDbKZOAlvhozgaB+LjSFpML8lI3RTH5PcZp7 X2BlyYf+EEbWui4bqvMM5xFcG8ZQrCLb23gpRhpGoStdzOl5fOdQ+GgaC0ypUwuM3SIc mC3F8yryPrEZaxyFKOw3aWO9H8iqDJmt/dciaJXaSNjByiHw0OgMmtD0YmzriPwCN6xw 21uA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=V1kHFp8GkrESvVr5Vv2w+XNL8Vmtl973b1nN3epOUpM=; b=dEZsema24/VlJiapAQIvcPU8bv5OWxhincUSCrPWY3oqCz4B1PDpl2yqSez1kNqKHA ZHUiARU9JzlgE/gIzdfga6EgZ7hy/qfuvOapEx6zUaIdiw4r04+3AvbKQGVSrqhQRd9k ti8YyPC2YqHt9aF5PMl2cow2c5/7TzwBnycVaBzTXC69TaTuZQBSvdM1X3aGdoSeOJ2f s8Hn+kHKHYusABn4nGEfztVcygxNoYFF9BxBx0zKaY7SChUYfragQDv4bJcWC+OukH/U MuzKtQ8AezHCHxvIm8g39dQhrh/PCdaHCi5dZCdh4hnENi2rsMYRo0VxfTKk1oVOeDiV 4zkA== X-Gm-Message-State: ALoCoQlYULQ+hpzMtSBmmVNgIjYA5r7sTHTKAe3Spv0jthloQHdWuDlzhL6+REFmnnBck5vS/55u MIME-Version: 1.0 X-Received: by 10.182.118.163 with SMTP id kn3mr21764700obb.80.1432644488279; Tue, 26 May 2015 05:48:08 -0700 (PDT) Received: by 10.202.88.214 with HTTP; Tue, 26 May 2015 05:48:08 -0700 (PDT) In-Reply-To: <81564C0D7D4D2A4B9A86C8C7404A13DA32145DE4@ESESSMB205.ericsson.se> References: <81564C0D7D4D2A4B9A86C8C7404A13DA32145DE4@ESESSMB205.ericsson.se> Date: Tue, 26 May 2015 08:48:08 -0400 Message-ID: From: Neal Cardwell To: Ingemar Johansson S Content-Type: text/plain; charset=UTF-8 Archived-At: Cc: Eric Dumazet , "tcpm@ietf.org" , "end2end-interest@postel.org" , Sangtae Ha Subject: Re: [tcpm] TCP HyStart patch deployment X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 May 2015 12:48:10 -0000 On Tue, May 26, 2015 at 7:38 AM, Ingemar Johansson S wrote: > Does anybody know if the TCP Cubic HyStart patch described in the link below > is being used widely ? > > It does not seem to be implemented in the latest Linux code. > > http://patchwork.ozlabs.org/patch/85945/ It appears that roughly a week after that thread started, the following patch was added to address that issue: http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=2b4636a5f8ca547000f6aba24ec1c58f31f4a91d Then later on the following patch was added to further refine the sensitivity of Hystart: http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=42eef7a0bb0989cd50d74e673422ff98a0ce4d7b Hope that helps, neal From nobody Tue May 26 07:09:58 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E5C691B2ED8 for ; Tue, 26 May 2015 07:09:55 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.201 X-Spam-Level: X-Spam-Status: No, score=-4.201 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7ZrJbWAIrAjX for ; Tue, 26 May 2015 07:09:54 -0700 (PDT) Received: from sessmg23.ericsson.net (sessmg23.ericsson.net [193.180.251.45]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 9C7381B2EE1 for ; Tue, 26 May 2015 07:09:48 -0700 (PDT) X-AuditID: c1b4fb2d-f794d6d000004501-c4-55647eaae72a Received: from ESESSHC005.ericsson.se (Unknown_Domain [153.88.253.125]) by sessmg23.ericsson.net (Symantec Mail Security) with SMTP id 1E.F5.17665.AAE74655; Tue, 26 May 2015 16:09:46 +0200 (CEST) Received: from ESESSMB205.ericsson.se ([169.254.5.148]) by ESESSHC005.ericsson.se ([153.88.183.33]) with mapi id 14.03.0210.002; Tue, 26 May 2015 16:09:45 +0200 From: Ingemar Johansson S To: Neal Cardwell Thread-Topic: [tcpm] TCP HyStart patch deployment Thread-Index: AdCXp+zNDAZ7zydkTra0wALqZYyJLf//8w0A///OCJA= Date: Tue, 26 May 2015 14:09:45 +0000 Message-ID: <81564C0D7D4D2A4B9A86C8C7404A13DA32147033@ESESSMB205.ericsson.se> References: <81564C0D7D4D2A4B9A86C8C7404A13DA32145DE4@ESESSMB205.ericsson.se> In-Reply-To: Accept-Language: sv-SE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [153.88.183.16] Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFupkkeLIzCtJLcpLzFFi42KZGfG3VndVXUqowbOFWhZPjz1it9h9biqL RcedvSwWR64cY7TYdnI+k8WXx1fZHNg8ds66y+6xYFOpx5IlP5k8Oo5vYA5gieKySUnNySxL LdK3S+DKaHhQUXBCpOLf4b1sDYwtIl2MnBwSAiYSM97sYYOwxSQu3FsPZHNxCAkcZZS4eXoz M4SzhFHi1MU5rCBVbAI2EisPfWcEsUUENCTuLnrACFLELNDDJLF28RKwImEBA4l1+0+zQRQZ SlxY0g9lW0k8P/aOGcRmEVCV6Lv9HqiZg4NXwFdiboszxLIpjBLPOhYzgdRwCgRKNHfvB6tn FJCVuP/9HguIzSwgLnHryXwmiLMFJJbsOc8MYYtKvHz8jxXCVpTYebadGWQ+s4CmxPpd+hCt ihJTuh+yg9i8AoISJ2c+YZnAKDYLydRZCB2zkHTMQtKxgJFlFaNocWpxcW66kbFealFmcnFx fp5eXmrJJkZg3B3c8lt3B+Pq146HGAU4GJV4eBdcTg4VYk0sK67MPcQozcGiJM7r1RUSKiSQ nliSmp2aWpBaFF9UmpNafIiRiYNTqoHR4LXkkUWPe1fuVz0/23rmqSCmv5VzrUz3NydFbAv6 tTU71K5bYs575RJmp6uVSw8UtrCe+8qwn/tZf+zXj8tUbZ8u0L/e9nFyTH7ChJ1Trv7lvef1 q1BP4NXxD++Fv138HR9x5V9PmPOdl05MBwyY9ywUnv2wnLdoxo7rrSwq/m/E/Z7c+P7lsBJL cUaioRZzUXEiAFJ8zgKcAgAA Archived-At: Cc: "tcpm@ietf.org" , Sangtae Ha , Ingemar Johansson S , Eric Dumazet , "end2end-interest@postel.org" Subject: Re: [tcpm] TCP HyStart patch deployment X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 May 2015 14:09:56 -0000 VGhhbmtzIGZvciB0aGUgcHJvbXB0IHJlc3BvbnNlLiANCg0KTm90ZWQgdGhvdWdoIHRoYXQgdGhl IGZpcnN0IHN1Z2dlc3RlZCBmaXJzdCBwYXRjaCB3YW50cyAoaHR0cDovL3BhdGNod29yay5vemxh YnMub3JnL3BhdGNoLzg1OTQ1LykgdG8gc3RyZXRjaCB0aGluZ3MgYSBiaXQgZnVydGhlciB0aGFu IHdoYXQgdGhlIGxpbnV4LmdpdCBwYXRjaGVzIHN1Z2dlc3QuDQoNCkkgaGF2ZSBpbXBsZW1lbnRl ZCBhIEh5U3RhcnQgKyBDdWJpYyAod2l0aCB0aGUgbGludXguZ2l0IHBhdGNoZXMpIGluIG91ciBM VEUgc3lzdGVtIHNpbXVsYXRvci4gSSBzZWVtIHRvIGdldCBpc3N1ZXMgdGhhdCBIeVN0YXJ0IHF1 aXRlIG9mdGVuIHNlZW0gdG8gZXhpdCB0b28gcXVpY2tseS4gSSBoYXZlIGRvdWJsZSBhbmQgdHJp cGxlIGNoZWNrZWQgdGhlIGNvZGUgYnV0IGRvbid0IGZpbmQgYW55IGRpZmZlcmVuY2VzIGFnYWlu c3QgdGhlIExpbnV4IGNvZGUuIE9uZSBwb3NzaWJsZSByZWFzb24gaXMgdGhhdCBMVEUgaW50cm9k dWNlcyBhbiBhbW91bnQgb2Ygc2NoZWR1bGluZyBqaXR0ZXIuIFF1ZXN0aW9uIGlzIGlmIHRoaXMg aXMgd2hhdCBpcyBjYXVzaW5nIHRoZSBwcm9ibGVtID8NCg0KSWYgSSBpbXBsZW1lbnQgdGhlIG9y aWdpbmFsIHBhdGNoICg4NTk0NSkgSSBzZWxkb20gZ2V0IGVhcmx5IGV4aXRzIGZyb20gSHlTdGFy dCwgb24gdGhlIG90aGVyIGhhbmQgSSBnZXQgbGFyZ2VyIG92ZXJzaG9vdHMsIGFuZCB0aGF0IGlz IG9mIGNvdXJzZSBub3Qgb3B0aW1hbC4NCg0KL0luZ2VtYXINCg0KDQoNCj4gLS0tLS1PcmlnaW5h bCBNZXNzYWdlLS0tLS0NCj4gRnJvbTogTmVhbCBDYXJkd2VsbCBbbWFpbHRvOm5jYXJkd2VsbEBn b29nbGUuY29tXQ0KPiBTZW50OiBkZW4gMjYgbWFqIDIwMTUgMTQ6NDgNCj4gVG86IEluZ2VtYXIg Sm9oYW5zc29uIFMNCj4gQ2M6IHRjcG1AaWV0Zi5vcmc7IGVuZDJlbmQtaW50ZXJlc3RAcG9zdGVs Lm9yZzsgRXJpYyBEdW1hemV0OyBZdWNodW5nDQo+IENoZW5nOyBTYW5ndGFlIEhhDQo+IFN1Ympl Y3Q6IFJlOiBbdGNwbV0gVENQIEh5U3RhcnQgcGF0Y2ggZGVwbG95bWVudA0KPiANCj4gT24gVHVl LCBNYXkgMjYsIDIwMTUgYXQgNzozOCBBTSwgSW5nZW1hciBKb2hhbnNzb24gUw0KPiA8aW5nZW1h ci5zLmpvaGFuc3NvbkBlcmljc3Nvbi5jb20+IHdyb3RlOg0KPiA+IERvZXMgYW55Ym9keSBrbm93 IGlmIHRoZSBUQ1AgQ3ViaWMgSHlTdGFydCBwYXRjaCBkZXNjcmliZWQgaW4gdGhlIGxpbmsNCj4g PiBiZWxvdyBpcyBiZWluZyB1c2VkIHdpZGVseSA/DQo+ID4NCj4gPiBJdCBkb2VzIG5vdCBzZWVt IHRvIGJlIGltcGxlbWVudGVkIGluIHRoZSBsYXRlc3QgTGludXggY29kZS4NCj4gPg0KPiA+IGh0 dHA6Ly9wYXRjaHdvcmsub3psYWJzLm9yZy9wYXRjaC84NTk0NS8NCj4gDQo+IEl0IGFwcGVhcnMg dGhhdCByb3VnaGx5IGEgd2VlayBhZnRlciB0aGF0IHRocmVhZCBzdGFydGVkLCB0aGUgZm9sbG93 aW5nIHBhdGNoDQo+IHdhcyBhZGRlZCB0byBhZGRyZXNzIHRoYXQgaXNzdWU6DQo+IA0KPiANCj4g aHR0cDovL2dpdC5rZXJuZWwub3JnL2NnaXQvbGludXgva2VybmVsL2dpdC90b3J2YWxkcy9saW51 eC5naXQvY29tbWl0Lz9pZD0yYjQNCj4gNjM2YTVmOGNhNTQ3MDAwZjZhYmEyNGVjMWM1OGYzMWY0 YTkxZA0KPiANCj4gVGhlbiBsYXRlciBvbiB0aGUgZm9sbG93aW5nIHBhdGNoIHdhcyBhZGRlZCB0 byBmdXJ0aGVyIHJlZmluZSB0aGUgc2Vuc2l0aXZpdHkNCj4gb2YgSHlzdGFydDoNCj4gDQo+IA0K PiBodHRwOi8vZ2l0Lmtlcm5lbC5vcmcvY2dpdC9saW51eC9rZXJuZWwvZ2l0L3RvcnZhbGRzL2xp bnV4LmdpdC9jb21taXQvP2lkPTQyZQ0KPiBlZjdhMGJiMDk4OWNkNTBkNzRlNjczNDIyZmY5OGEw Y2U0ZDdiDQo+IA0KPiBIb3BlIHRoYXQgaGVscHMsDQo+IG5lYWwNCg== From nobody Tue May 26 07:35:22 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0D0611A8AAA for ; Tue, 26 May 2015 07:35:22 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.389 X-Spam-Level: X-Spam-Status: No, score=-1.389 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FM_FORGED_GMAIL=0.622, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zg0AK2C2ymnv for ; Tue, 26 May 2015 07:35:20 -0700 (PDT) Received: from mail-ob0-x231.google.com (mail-ob0-x231.google.com [IPv6:2607:f8b0:4003:c01::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id CD3BB1A8AA8 for ; Tue, 26 May 2015 07:35:20 -0700 (PDT) Received: by obbea2 with SMTP id ea2so75707848obb.3 for ; Tue, 26 May 2015 07:35:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=MjXPlILQz16Dvz6Mr1PlEChmSsFe2gRX8S1Z3AChJyM=; b=j5D5rODMyAnwgzI8gPVg/TtPZ/ZX2a0aEZSZqnavIa0wxv5MjQhKKbS9dfXYNG1WYe A4vtMx8dL7OElk2AKyauwtxH95qrkkNHczg0APg3V4MNnwmgFoc41JUCV5n/w9CdNuKg M4Vok8jvpPLoVa4nwmT2w0hYHKdBM5GbaeZExExB+Qrb6g9+LnuctcIcL7QNHV2W3URK wajotRpI7siBPdrTOFzJXCmTgcwInB7aeof+YMRw71TQmNJRaP7fS86zBkAaLRIsAp44 x3zBqDYAtcZJaUYGGes1Byb7lqSnHo+6x1JqSQBEI6TV98VT1mpwCugLnFt68JXSZXAu P/9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=MjXPlILQz16Dvz6Mr1PlEChmSsFe2gRX8S1Z3AChJyM=; b=hx5OSp3JTSBeKi5N3mKYQYHEDlUzdZrUCWJTBxDXO0i9W749/qjJVHL8CqFnrPOluE WJPu2UFaVwPKsGMZbzXZiecLc5uByTJf3FHMkrR5jwbXGyrGO1HIChTh5re1vq23HYSQ yAY1mIPAuCh3Y0YcKGHs3MCwyrqdyy1roV4vRjEtHAcXx7DLroGiJ6TXVPIYF38jy1y+ s1pfiCVxHTKG8Xd2eN7cET/XpDixIl+bwnYCXrEiA+qqT6oyEcDAR16IYYlos1eNxyZY wzBb4xVb4WbrBLiVTXpF5PQ0W4Eu/QAga7WR4ktWzLUSLeN+BhEyYYLBIYTvqX9egy5P vGhw== X-Gm-Message-State: ALoCoQkxztnPtO9l1aXX+C7wvanh1Vqj2NxjmBXnbcDacX407phC3P2kPj+O+lAVUwDJlnP7sgQi MIME-Version: 1.0 X-Received: by 10.60.79.137 with SMTP id j9mr15920903oex.69.1432650920173; Tue, 26 May 2015 07:35:20 -0700 (PDT) Received: by 10.202.88.214 with HTTP; Tue, 26 May 2015 07:35:20 -0700 (PDT) In-Reply-To: <81564C0D7D4D2A4B9A86C8C7404A13DA32147033@ESESSMB205.ericsson.se> References: <81564C0D7D4D2A4B9A86C8C7404A13DA32145DE4@ESESSMB205.ericsson.se> <81564C0D7D4D2A4B9A86C8C7404A13DA32147033@ESESSMB205.ericsson.se> Date: Tue, 26 May 2015 10:35:20 -0400 Message-ID: From: Neal Cardwell To: Ingemar Johansson S Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Archived-At: Cc: Eric Dumazet , "tcpm@ietf.org" , "end2end-interest@postel.org" , Sangtae Ha Subject: Re: [tcpm] TCP HyStart patch deployment X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 May 2015 14:35:22 -0000 On Tue, May 26, 2015 at 10:09 AM, Ingemar Johansson S wrote: > Thanks for the prompt response. > > Noted though that the first suggested first patch wants (http://patchwork= .ozlabs.org/patch/85945/) to stretch things a bit further than what the lin= ux.git patches suggest. > > I have implemented a HyStart + Cubic (with the linux.git patches) in our = LTE system simulator. I seem to get issues that HyStart quite often seem to= exit too quickly. I have double and triple checked the code but don't find= any differences against the Linux code. One possible reason is that LTE in= troduces an amount of scheduling jitter. Question is if this is what is cau= sing the problem ? > > If I implement the original patch (85945) I seldom get early exits from H= yStart, on the other hand I get larger overshoots, and that is of course no= t optimal. In your tests without patch 85945, is it the HYSTART_ACK_TRAIN or HYSTART_DELAY mechanism that is triggering early exit of slow start? Eric Dumazet found HYSTART_ACK_TRAIN triggers early exit often, particularly if fq pacing is used. You may consider using just HYSTART_DELAY. What is the magnitude of the jitter and RTTs for which you are seeing probl= ems? The delay threshold of 2*min_rtt in the patch you are citing -- http://patchwork.ozlabs.org/patch/85945/ -- seems to allow a very big overshoot before exiting slow start. On the other hand, if you get good results for LTE and non-LTE simulations for something like: if (ca->curr_rtt > ca->delay_min + HYSTART_DELAY_THRESH(ca->delay_min >> 2)) { or if (ca->curr_rtt > ca->delay_min + HYSTART_DELAY_THRESH(ca->delay_min >> 1)) { then that starts to sound potentially applicable for general use. neal From nobody Thu May 28 05:08:09 2015 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DE7081A908F for ; Thu, 28 May 2015 05:08:07 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.201 X-Spam-Level: X-Spam-Status: No, score=-4.201 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gN3LDwRpD0WB for ; Thu, 28 May 2015 05:08:05 -0700 (PDT) Received: from sesbmg23.ericsson.net (sesbmg23.ericsson.net [193.180.251.37]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 4E0DC1A9050 for ; Thu, 28 May 2015 05:08:05 -0700 (PDT) X-AuditID: c1b4fb25-f79b66d000001131-0a-556705235431 Received: from ESESSHC007.ericsson.se (Unknown_Domain [153.88.253.125]) by sesbmg23.ericsson.net (Symantec Mail Security) with SMTP id 7B.97.04401.32507655; Thu, 28 May 2015 14:08:03 +0200 (CEST) Received: from ESESSMB208.ericsson.se ([169.254.8.216]) by ESESSHC007.ericsson.se ([153.88.183.39]) with mapi id 14.03.0210.002; Thu, 28 May 2015 14:08:03 +0200 From: Ingemar Johansson S To: Neal Cardwell Thread-Topic: [tcpm] TCP HyStart patch deployment Thread-Index: AdCXp+zNDAZ7zydkTra0wALqZYyJLf//8w0A///OCJCAAE/sAP/85ANw Date: Thu, 28 May 2015 12:08:02 +0000 Message-ID: <81564C0D7D4D2A4B9A86C8C7404A13DA34AE1E5E@ESESSMB208.ericsson.se> References: <81564C0D7D4D2A4B9A86C8C7404A13DA32145DE4@ESESSMB205.ericsson.se> <81564C0D7D4D2A4B9A86C8C7404A13DA32147033@ESESSMB205.ericsson.se> In-Reply-To: Accept-Language: sv-SE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [153.88.183.18] Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFupkkeLIzCtJLcpLzFFi42KZGfG3VleZNT3UYO8cBYunxx6xW+w+N5XF ouPOXhaLI1eOMVpsOzmfyeLL46tsDmweO2fdZfdYsKnUY8mSn0weHcc3MAewRHHZpKTmZJal FunbJXBlLL19l6Vgh1TFlFUvmRoYH0h2MXJySAiYSBzatJAZwhaTuHBvPVsXIxeHkMBRRomT fxrYIZwljBJvP+xnBaliE7CRWHnoOyOILSKgIXF30QNGkCJmgR4mibWLl4AVCQsYSKzbf5oN oshQ4sKSfijbTeLH+0/sIDaLgKrEwg2vWboYOTh4BXwljn2MBgkLCcxnkvjYaw9icwoESjxZ 8R+snFFAVuL+93ssIDazgLjErSfzmSCuFpBYsuc81AeiEi8f/2OFsBUlPr7axwgynllAU2L9 Ln2IVkWJKd0PwUbyCghKnJz5hGUCo9gsJFNnIXTMQtIxC0nHAkaWVYyixanFSbnpRsZ6qUWZ ycXF+Xl6eaklmxiBcXdwy2/VHYyX3zgeYhTgYFTi4X2wLi1UiDWxrLgy9xCjNAeLkjivZ1dI qJBAemJJanZqakFqUXxRaU5q8SFGJg5OqQZG0SIn2SSfOxN2Bi9Mely4/53S0e11+jsTIk60 742Rn8LWtd0u+kD0dMOayU56CZ5v/my7JJMcNS9u9rTzNQahqncVfu3euraQcdfknD3Prn5m fTqj3tvtlLoJI2fvjcN7P79mXGWl9db9E4Ot3SqGMPVnZfZGus1R0Y5fLt1gvecTczqubtVc JZbijERDLeai4kQAXT7bOJwCAAA= Archived-At: Cc: "tcpm@ietf.org" , Sangtae Ha , Ingemar Johansson S , Eric Dumazet , "end2end-interest@postel.org" Subject: Re: [tcpm] TCP HyStart patch deployment X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 May 2015 12:08:08 -0000 SGkNCkFuZCB0aGFua3MgZm9yIHRoZSB0aXBzLiBJIHdpbGwgbWFrZSBhIG1vcmUgdGhvcm91Z2gg aW52ZXN0aWdhdGlvbiB0byBzZWUgaG93IHRoZSBzdWdnZXN0ZWQgY2hhbmdlcyBiZWxvdyBhZmZl Y3RzIHBlcmZvcm1hbmNlLiBJdCBtYXkgdGFrZSBhIGNvdXBsZSBvZiB3ZWVrcyB0aG91Z2ggYmVm b3JlIEkgaGF2ZSBhbnl0aGluZyBtZWFuaW5nZnVsLg0KDQovSW5nZW1hcg0KDQo+IC0tLS0tT3Jp Z2luYWwgTWVzc2FnZS0tLS0tDQo+IEZyb206IE5lYWwgQ2FyZHdlbGwgW21haWx0bzpuY2FyZHdl bGxAZ29vZ2xlLmNvbV0NCj4gU2VudDogZGVuIDI2IG1haiAyMDE1IDE2OjM1DQo+IFRvOiBJbmdl bWFyIEpvaGFuc3NvbiBTDQo+IENjOiB0Y3BtQGlldGYub3JnOyBlbmQyZW5kLWludGVyZXN0QHBv c3RlbC5vcmc7IEVyaWMgRHVtYXpldDsgWXVjaHVuZw0KPiBDaGVuZzsgU2FuZ3RhZSBIYQ0KPiBT dWJqZWN0OiBSZTogW3RjcG1dIFRDUCBIeVN0YXJ0IHBhdGNoIGRlcGxveW1lbnQNCj4gDQo+IE9u IFR1ZSwgTWF5IDI2LCAyMDE1IGF0IDEwOjA5IEFNLCBJbmdlbWFyIEpvaGFuc3NvbiBTDQo+IDxp bmdlbWFyLnMuam9oYW5zc29uQGVyaWNzc29uLmNvbT4gd3JvdGU6DQo+ID4gVGhhbmtzIGZvciB0 aGUgcHJvbXB0IHJlc3BvbnNlLg0KPiA+DQo+ID4gTm90ZWQgdGhvdWdoIHRoYXQgdGhlIGZpcnN0 IHN1Z2dlc3RlZCBmaXJzdCBwYXRjaCB3YW50cw0KPiAoaHR0cDovL3BhdGNod29yay5vemxhYnMu b3JnL3BhdGNoLzg1OTQ1LykgdG8gc3RyZXRjaCB0aGluZ3MgYSBiaXQgZnVydGhlcg0KPiB0aGFu IHdoYXQgdGhlIGxpbnV4LmdpdCBwYXRjaGVzIHN1Z2dlc3QuDQo+ID4NCj4gPiBJIGhhdmUgaW1w bGVtZW50ZWQgYSBIeVN0YXJ0ICsgQ3ViaWMgKHdpdGggdGhlIGxpbnV4LmdpdCBwYXRjaGVzKSBp biBvdXIgTFRFDQo+IHN5c3RlbSBzaW11bGF0b3IuIEkgc2VlbSB0byBnZXQgaXNzdWVzIHRoYXQg SHlTdGFydCBxdWl0ZSBvZnRlbiBzZWVtIHRvIGV4aXQNCj4gdG9vIHF1aWNrbHkuIEkgaGF2ZSBk b3VibGUgYW5kIHRyaXBsZSBjaGVja2VkIHRoZSBjb2RlIGJ1dCBkb24ndCBmaW5kIGFueQ0KPiBk aWZmZXJlbmNlcyBhZ2FpbnN0IHRoZSBMaW51eCBjb2RlLiBPbmUgcG9zc2libGUgcmVhc29uIGlz IHRoYXQgTFRFIGludHJvZHVjZXMNCj4gYW4gYW1vdW50IG9mIHNjaGVkdWxpbmcgaml0dGVyLiBR dWVzdGlvbiBpcyBpZiB0aGlzIGlzIHdoYXQgaXMgY2F1c2luZyB0aGUNCj4gcHJvYmxlbSA/DQo+ ID4NCj4gPiBJZiBJIGltcGxlbWVudCB0aGUgb3JpZ2luYWwgcGF0Y2ggKDg1OTQ1KSBJIHNlbGRv bSBnZXQgZWFybHkgZXhpdHMgZnJvbQ0KPiBIeVN0YXJ0LCBvbiB0aGUgb3RoZXIgaGFuZCBJIGdl dCBsYXJnZXIgb3ZlcnNob290cywgYW5kIHRoYXQgaXMgb2YgY291cnNlIG5vdA0KPiBvcHRpbWFs Lg0KPiANCj4gSW4geW91ciB0ZXN0cyB3aXRob3V0IHBhdGNoIDg1OTQ1LCBpcyBpdCB0aGUgSFlT VEFSVF9BQ0tfVFJBSU4gb3INCj4gSFlTVEFSVF9ERUxBWSBtZWNoYW5pc20gdGhhdCBpcyB0cmln Z2VyaW5nIGVhcmx5IGV4aXQgb2Ygc2xvdyBzdGFydD8NCj4gDQo+IEVyaWMgRHVtYXpldCBmb3Vu ZCBIWVNUQVJUX0FDS19UUkFJTiB0cmlnZ2VycyBlYXJseSBleGl0IG9mdGVuLA0KPiBwYXJ0aWN1 bGFybHkgaWYgZnEgcGFjaW5nIGlzIHVzZWQuIFlvdSBtYXkgY29uc2lkZXIgdXNpbmcganVzdCBI WVNUQVJUX0RFTEFZLg0KPiANCj4gV2hhdCBpcyB0aGUgbWFnbml0dWRlIG9mIHRoZSBqaXR0ZXIg YW5kIFJUVHMgZm9yIHdoaWNoIHlvdSBhcmUgc2VlaW5nDQo+IHByb2JsZW1zPw0KPiANCj4gVGhl IGRlbGF5IHRocmVzaG9sZCBvZiAyKm1pbl9ydHQgaW4gdGhlIHBhdGNoIHlvdSBhcmUgY2l0aW5n IC0tDQo+IGh0dHA6Ly9wYXRjaHdvcmsub3psYWJzLm9yZy9wYXRjaC84NTk0NS8gLS0gc2VlbXMg dG8gYWxsb3cgYSB2ZXJ5IGJpZw0KPiBvdmVyc2hvb3QgYmVmb3JlIGV4aXRpbmcgc2xvdyBzdGFy dC4NCj4gDQo+IE9uIHRoZSBvdGhlciBoYW5kLCBpZiB5b3UgZ2V0IGdvb2QgcmVzdWx0cyBmb3Ig TFRFIGFuZCBub24tTFRFIHNpbXVsYXRpb25zIGZvcg0KPiBzb21ldGhpbmcgbGlrZToNCj4gDQo+ ICAgICAgICAgICAgICAgICAgICAgICAgIGlmIChjYS0+Y3Vycl9ydHQgPiBjYS0+ZGVsYXlfbWlu ICsNCj4gICAgICAgICAgICAgICAgICAgICAgICAgICAgIEhZU1RBUlRfREVMQVlfVEhSRVNIKGNh LT5kZWxheV9taW4gPj4gMikpIHsgb3INCj4gICAgICAgICAgICAgICAgICAgICAgICAgaWYgKGNh LT5jdXJyX3J0dCA+IGNhLT5kZWxheV9taW4gKw0KPiAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgSFlTVEFSVF9ERUxBWV9USFJFU0goY2EtPmRlbGF5X21pbiA+PiAxKSkgew0KPiANCj4gdGhl biB0aGF0IHN0YXJ0cyB0byBzb3VuZCBwb3RlbnRpYWxseSBhcHBsaWNhYmxlIGZvciBnZW5lcmFs IHVzZS4NCj4gDQo+IG5lYWwNCg==