From nobody Sun Apr 1 09:52:33 2018 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 68631126DFF for ; Sun, 1 Apr 2018 09:52:31 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.699 X-Spam-Level: X-Spam-Status: No, score=-2.699 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id QKETpGmATFqq for ; Sun, 1 Apr 2018 09:52:29 -0700 (PDT) Received: from mail-wm0-x235.google.com (mail-wm0-x235.google.com [IPv6:2a00:1450:400c:c09::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 593B7126DCA for ; Sun, 1 Apr 2018 09:52:29 -0700 (PDT) Received: by mail-wm0-x235.google.com with SMTP id w2so3167133wmw.1 for ; Sun, 01 Apr 2018 09:52:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=joSsCvzyf475LwPdILlBeVheRo4Mzt+Y8ntk8TUM8Z8=; b=b2JFCzJpb685b5AJ4le6X3WyV/T7m6ZeCrdJx0rAIUSkOQ9Lh7plUuwxCJU05311dl CTyPu48ekWh1fkMkCPZmhRJy0jgkLDClrjDm8X+l8i7giglc+ZZnPf9H30qrkjeyabkh 41L2NWwcEgAwfX3T9IkRuABBdLyO5mBD4vfkGznxzGl+3EYWvQto1yssEgVwfLBbMkqX im5d0lRlW9+C7Fqzt76PwUcskEuEvR+c84yFtcrlHhdTdARKuXtZHZYLIaSMeHrBIz92 xMY2v2DlVE9koSEjS1hZRNFuQHR002T80DL4XvRM4sGMfcOd1i3wV8ttPdz8Vt7tmr37 ZD1g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=joSsCvzyf475LwPdILlBeVheRo4Mzt+Y8ntk8TUM8Z8=; b=JRyrpVTuGI83wM+J5g82QkoaVX6imtdQLZFXif2Pv8dXUVdFL5Q6fK4Mmy2fZwwVoa BT3a+8OTFp/wwZmNXdbUy3an3074SHcuRkWXZQ4Xng/Yi5ARAPxwgWhaDh0DM9doDuNC 6BTlVTZ1nPthRDi9kSTt3TKTRSyAW0JVJ9qRhhw4Omz1/rJCoHaGFGeI82hzdoq3Rs13 76N6bJHSscfw3fdKtKKLutoIfJjhAQAeoJV3BBHiU7yTGgDgYC4sJmn5qXP4FKVjzuTk L0xb2Ar17AFnNJxnCDPcFhqBWQsfgAkpcwyTb5n5jzy5RmUgEeNgwj/845F3AV55JOgE X3uQ== X-Gm-Message-State: AElRT7EcraZcwzYNUMlyXXycGi59BN8c3FSy6SHemajh+wUEQL1rndfv XV4Ut0MyA4g4Xh5zNfulVR6jOQiWuiacCvImor8= X-Google-Smtp-Source: AIpwx49hF/QZO0l/8DDafDSibx6EPIT0phRcdExUy9BBPkCJ+TnX/6WnM+VYSM2cRn7l66324M04UGbLpjh48qD5z0c= X-Received: by 10.80.211.3 with SMTP id g3mr9992035edh.15.1522601547836; Sun, 01 Apr 2018 09:52:27 -0700 (PDT) MIME-Version: 1.0 Received: by 10.80.245.44 with HTTP; Sun, 1 Apr 2018 09:52:27 -0700 (PDT) In-Reply-To: References: <08fb7ec5406418e439391c66bc108d5d@strayalpha.com> <572210fa-74ae-0edf-ce10-d66153ae80a3@strayalpha.com> <5ABDE5ED.7060508@erg.abdn.ac.uk> <20180330141011.GI3590@faui40p.informatik.uni-erlangen.de> From: John Heffner Date: Sun, 1 Apr 2018 12:52:27 -0400 Message-ID: To: Mikael Abrahamsson Cc: Joe Touch , TCP Maintenance and Minor Extensions WG Content-Type: multipart/alternative; boundary="f40304388df864af1b0568cc4c30" Archived-At: Subject: Re: [tcpm] PLPMTUD for all protocols X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Apr 2018 16:52:31 -0000 --f40304388df864af1b0568cc4c30 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Fri, Mar 30, 2018 at 10:28 AM, Mikael Abrahamsson wrote: > On Fri, 30 Mar 2018, Joe Touch wrote: > > - use IPv6 and go home (we shouldn=E2=80=99t be optimizing PMTUD for IPv6= - we >> should just be sticking to IPv6 MTU requirements >> > > In the longer term, I want to be able to deploy jumbo frames and run 9k > access network including customer LAN. This means that a stupid MTU1280 > fallback isn't optimal, even though I'll gladly take that if the > alternative is no PMTU blackhole detect at all. > > > -- > Mikael Abrahamsson email: swmike@swm.pp.se > > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm > > I'd certainly be interested to look at concrete scenarios where performance is a concern, even better with numbers, if anyone has gathered them. Linux *will* probe upward after a black hole detection event has reduced the effective path mtu. One issue is that the currently-implemented probing heuristic results in sub-optimal effective path mtu that will remain in use indefinitely. It shouldn't take much to improve on what's there now if this is a real problem for people. I haven't looked at the code, but from the description above, it doesn't sound like FreeBSD implements what I would call RFC4821 PLPMTUD. FWIW, one of the objectives when writing RFC4821 was to make it easy to enable jumbo frames. At the time, I ran my own laptop and a handful of servers with a 9k or MTU for quite a while, and felt it generally worked well with the setting tcp_mtu_probing =3D 2, i.e., always starting with a small effective path mtu and probing upward. (I imagine more performance-sensitive applications, especially within data centers for instance, might make different trade-offs.) Thanks, -John --f40304388df864af1b0568cc4c30 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
On F= ri, Mar 30, 2018 at 10:28 AM, Mikael Abrahamsson <swmike@swm.pp.se><= /span> wrote:
On Fri, 30 Mar 2018, = Joe Touch wrote:

- use IPv6 and go home (we shouldn=E2=80=99t be optimizing PMTUD for IPv6 -= we should just be sticking to IPv6 MTU requirements

 In the longer term, I want to be able to deploy jumbo frames and run 9k acc= ess network including customer LAN. This means that a stupid MTU1280 fallba= ck isn't optimal, even though I'll gladly take that if the alternat= ive is no PMTU blackhole detect at all.


--
Mikael Abrahamsson=C2=A0 =C2=A0 email: swmike@swm.pp.se

_____________________= __________________________
tcpm mailing list
tcpm@ietf.org
https://www.ietf.org/mailman/listinfo/tcpm



I'd certainly be int= erested to look at concrete scenarios where performance is a concern, even = better with numbers, if anyone has gathered them.

Linux *will= * probe upward after a black hole detection event has reduced the effective= path mtu.=C2=A0 One issue is that the currently-implemented probing heuris= tic results in sub-optimal effective path mtu that will remain in use indef= initely.=C2=A0 It shouldn't take much to improve on what's there no= w if this is a real problem for people.

I haven'= ;t looked at the code, but from the description above, it doesn't sound= like FreeBSD implements what I would call RFC4821 PLPMTUD.

<= div style=3D"color:rgb(34,34,34);font-family:arial,sans-serif;font-size:sma= ll;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal= ;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;tex= t-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(2= 55,255,255);text-decoration-style:initial;text-decoration-color:initial">FW= IW, one of the objectives when writing RFC4821 was to make it easy to enabl= e jumbo frames.=C2=A0 At the time, I ran my own laptop and a handful of ser= vers with a 9k or MTU for quite a while, and felt it generally worked well = with the setting tcp_mtu_probing =3D 2, i.e., always starting with a small = effective path mtu and probing upward.=C2=A0 (I imagine more performance-se= nsitive applications, especially within data centers for instance, might ma= ke different trade-offs.)

Thanks,
=C2=A0 -John

= --f40304388df864af1b0568cc4c30-- From nobody Thu Apr 5 10:14:08 2018 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2C0D1120724 for ; Thu, 5 Apr 2018 10:14:07 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.909 X-Spam-Level: X-Spam-Status: No, score=-1.909 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, T_RP_MATCHES_RCVD=-0.01, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id aVam7vl3jkQd for ; Thu, 5 Apr 2018 10:14:04 -0700 (PDT) Received: from pegasus.erg.abdn.ac.uk (pegasus.erg.abdn.ac.uk [IPv6:2001:630:241:204::f0f0]) by ietfa.amsl.com (Postfix) with ESMTP id 969451201F8 for ; Thu, 5 Apr 2018 10:14:04 -0700 (PDT) Received: from dhcp-207-156.erg.abdn.ac.uk (unknown [IPv6:2001:630:241:207:b4be:8e72:cc28:8630]) by pegasus.erg.abdn.ac.uk (Postfix) with ESMTPSA id F14AD1B003A8; Thu, 5 Apr 2018 18:14:03 +0100 (BST) Reply-To: gorry@erg.abdn.ac.uk To: Yuchung Cheng , "tcpm@ietf.org Extensions" References: <152229608335.23885.1229382205419997341@ietfa.amsl.com> From: Gorry Fairhurst Organization: The University of Aberdeen is a charity registered in Scotland, No SC013683. Message-ID: <5a1446e0-c3c5-f82e-a56e-162fc39b6094@erg.abdn.ac.uk> Date: Thu, 5 Apr 2018 18:14:03 +0100 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:52.0) Gecko/20100101 Thunderbird/52.6.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Archived-At: Subject: Re: [tcpm] I-D Action: draft-ietf-tcpm-rfc793bis-08.txt X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Apr 2018 17:14:07 -0000 A couple of comments-on-comments in-line, Gorry On 29/03/2018 15:10, Yuchung Cheng wrote: > Thanks for the update. I enjoy reading the new diffserv text. > > comments: > > The IPv4 specification [1] includes a precedence value in the Type of > precedence will still have to check the precedence of incoming Service > field (TOS), that was also modified in [15], and then > segments and possibly raise the precedence level they use on the > obsoleted by the definition of Differentiated Services (DiffServ) > connection. [6]. I think this text kind of suggests ToS still exists. DiffServ is mature technology. I think we shouldn't even be hinting that ToS semantics are permitted - I'd suggest we firmly state that Diffserv has obsoleted the former TOS semantics. > In DiffServ the former precedence values are treated as Class > Selector codepoints, and methods for compatible treatment are > described in the DiffServ architecture. The RFC 793/1122 TCP > specification includes logic intending to have connections use the > highest precedence requested by either endpoint application, and to > keep the precedence consistent throughout a connection. > Right, RFC 7657 has slightly more refined text in the same vein on not change DSCPs too often. > There is an > assumption of bidirectional/symmetric precedence values, however, the > DiffServ architecture is asymmetric. Problems were described in [17] > and the solution described is to ignore IP precedence in TCP. Since > RFC 2873 is a Standards Track document (although not marked as > updating RFC 793), these checks are no longer a part of the TCP > standard defined in this document, though the DiffServ field value is > still is a part of the interface between TCP and the network layer, /is still is/ > and values can be indicated both ways between TCP and the > application. > > > I am not entirely I get the main points this paragraph tries to convey. Are they > 1. a TCP stack MUST implement RFC2873 > 2. precedence is obsoleted. > ? > I think something does have to be said about the way diffserv works, although RFC 2873 is all the more vague by avoiding all keywords. > > > Note that the DiffServ field is permitted to change during a > connection (section 4.2.4.2 of RFC 1122). However, the > application interface might not support this ability, > Well always possible, but do we really need to say that, since it kind of breaks diffserv. > and the > application does not have knowledge about individual TCP > segments, so this can only be done on a coarse granularity, at > best. This limitation is further discussed in RFC 7657 (sec > 5.1, 5.3, and 6) [37]. Generally, an application SHOULD NOT > change the DiffServ field value during the course of a > connection. > > > The underlying assumptions of this text is > 1. Application can not specify precisely the DiffServ at TCP segments level > 2. TCP does not tolerate reordering well today > > As API and TCP evolve and socket API isn't the only one, it would be > good to be explicit about these assumptions. e.g. > "Generally an application SHOULD NOT change DiffServ field value > during the course of a connection because it may cause excessive > reorderings and adversely hurts performance" > > nits: > p32: s/the DiffServ field value is still is a part/ the DiffServ field > value still is a part / > p68: s/ SND.NEXT/SND.NXT/ > p47: s/TCP may be buffer the data/TCP may buffer the data/ > > On Wed, Mar 28, 2018 at 9:01 PM, wrote: >> >> A New Internet-Draft is available from the on-line Internet-Drafts directories. >> This draft is a work item of the TCP Maintenance and Minor Extensions WG of the IETF. >> >> Title : Transmission Control Protocol Specification >> Author : Wesley M. Eddy >> Filename : draft-ietf-tcpm-rfc793bis-08.txt >> Pages : 102 >> Date : 2018-03-28 >> >> Abstract: >> This document specifies the Internet's Transmission Control Protocol >> (TCP). TCP is an important transport layer protocol in the Internet >> stack, and has continuously evolved over decades of use and growth of >> the Internet. Over this time, a number of changes have been made to >> TCP as it was specified in RFC 793, though these have only been >> documented in a piecemeal fashion. This document collects and brings >> those changes together with the protocol specification from RFC 793. >> This document obsoletes RFC 793, as well as 879, 2873, 6093, 6429, >> 6528, and 6691 that updated parts of RFC 793. It updates RFC 1122, >> and should be considered as a replacement for the portions of that >> document dealing with TCP requirements. It updates RFC 5961 due to a >> small clarification in reset handling while in the SYN-RECEIVED >> state. >> >> RFC EDITOR NOTE: If approved for publication as an RFC, this should >> be marked additionally as "STD: 7" and replace RFC 793 in that role. >> >> >> >> The IETF datatracker status page for this draft is: >> https://datatracker.ietf.org/doc/draft-ietf-tcpm-rfc793bis/ >> >> There are also htmlized versions available at: >> https://tools.ietf.org/html/draft-ietf-tcpm-rfc793bis-08 >> https://datatracker.ietf.org/doc/html/draft-ietf-tcpm-rfc793bis-08 >> >> A diff from the previous version is available at: >> https://www.ietf.org/rfcdiff?url2=draft-ietf-tcpm-rfc793bis-08 >> >> >> Please note that it may take a couple of minutes from the time of submission >> until the htmlized version and diff are available at tools.ietf.org. >> >> Internet-Drafts are also available by anonymous FTP at: >> ftp://ftp.ietf.org/internet-drafts/ >> >> _______________________________________________ >> tcpm mailing list >> tcpm@ietf.org >> https://www.ietf.org/mailman/listinfo/tcpm > > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm > From nobody Thu Apr 5 15:36:25 2018 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 450B41267BB for ; Thu, 5 Apr 2018 15:36:24 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.709 X-Spam-Level: X-Spam-Status: No, score=-2.709 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=google.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fepQljV4a9JP for ; Thu, 5 Apr 2018 15:36:21 -0700 (PDT) Received: from mail-it0-x236.google.com (mail-it0-x236.google.com [IPv6:2607:f8b0:4001:c0b::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id D366E120454 for ; Thu, 5 Apr 2018 15:36:20 -0700 (PDT) Received: by mail-it0-x236.google.com with SMTP id t192-v6so3333931itc.1 for ; Thu, 05 Apr 2018 15:36:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=f4mKuPny05u2hiRvLY07Xh1sPBbYhAYPwYF8ud/lYII=; b=mXAMQbtyScUne11VExt8ShfWlo7Ku1re7xit6zGWFECyWfvfbApmwFNVcbumuS2rjW RF0hXnFlzGUdgJYEBPuEh1sDgHuR/YLIAw81FJkm8cFWdWyIE6F9yX5BPW57wemIAItl gkfVoGZCoG+YoRoFze34a3nf9L3PncbKEYGqUHxJWtGJcYJ/Dv6nE2OrNYRjNN1Q+eom UrzRVfRydGGTJTASZX8VtWd8jWw+ti7RKJr5g9Y/XW9A/a/yuUQ9oXAe06DPnLj89FRy YTx7ZZBgGICGAVzzz8YS+MXENQpFSkKQix7FioXTX6mLjEfsTNSN2x3Q+RLoCrvaG+jL Ec2A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=f4mKuPny05u2hiRvLY07Xh1sPBbYhAYPwYF8ud/lYII=; b=kIexKp2eGmwXRxnXQcnN+f3NtLWgOFXDYRPxSlP8G/9ks7uH/j9zwu1p4RyVqxmjqL 99v+JLLo+QyIEuyGUUeVlcCzq6dKto4svVe/NEbikFdYxGoKQ4DwuGS+ZHWAfE1veN4+ 5+t1mnCU3zTeuricPxzUGN1sOq+2Z93JepXdSfhHBJLn9g3fMY5+Aep2RIXFsrvSAcsp qkaMYfC3vlR9UYgXuJ2Q7B6EZKYZ7emEvPJFLvhPMfsSTtjeaR14TugYLVXKM0xsBypc D2Xo9tEXwSYA/3AbDKFhsPIKjtiyslOw5juWQ5zE0M49+AkBz+753nNENWzPwEhdF+Hm wILA== X-Gm-Message-State: ALQs6tDm7pPt+rn2VoYbGkPc+Qf0zMmK+tkouyLzL0710rRknpQ138rk K5PyNqO4puJSmdS19uHC1PkNBOSX5DVbpUfdm4LqLnfp X-Google-Smtp-Source: AIpwx48yHVl7W1uaxmMxmEGx2zgqgYN8d8yMQ+bx2sJbXNnctdfeAKcvIvxWuunP6NHPQw98kvvqJt7nKKabDYBsc40= X-Received: by 2002:a24:7893:: with SMTP id p141-v6mr16253638itc.45.1522967779697; Thu, 05 Apr 2018 15:36:19 -0700 (PDT) MIME-Version: 1.0 Received: by 10.107.138.29 with HTTP; Thu, 5 Apr 2018 15:35:39 -0700 (PDT) In-Reply-To: <5a1446e0-c3c5-f82e-a56e-162fc39b6094@erg.abdn.ac.uk> References: <152229608335.23885.1229382205419997341@ietfa.amsl.com> <5a1446e0-c3c5-f82e-a56e-162fc39b6094@erg.abdn.ac.uk> From: Yuchung Cheng Date: Thu, 5 Apr 2018 15:35:39 -0700 Message-ID: To: Gorry Fairhurst Cc: "tcpm@ietf.org Extensions" Content-Type: multipart/alternative; boundary="0000000000008402dd05692191ca" Archived-At: Subject: Re: [tcpm] I-D Action: draft-ietf-tcpm-rfc793bis-08.txt X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Apr 2018 22:36:24 -0000 --0000000000008402dd05692191ca Content-Type: text/plain; charset="UTF-8" On Thu, Apr 5, 2018 at 10:14 AM, Gorry Fairhurst wrote: > A couple of comments-on-comments in-line, > > Gorry > > > On 29/03/2018 15:10, Yuchung Cheng wrote: > >> Thanks for the update. I enjoy reading the new diffserv text. >> >> comments: >> >> The IPv4 specification [1] includes a precedence value in the Type of >> precedence will still have to check the precedence of incoming Service >> field (TOS), that was also modified in [15], and then >> segments and possibly raise the precedence level they use on the >> obsoleted by the definition of Differentiated Services (DiffServ) >> connection. [6]. >> > > I think this text kind of suggests ToS still exists. DiffServ is mature > technology. I think we shouldn't even be hinting that ToS semantics are > permitted - I'd suggest we firmly state that Diffserv has obsoleted the > former TOS semantics. +1 > > > In DiffServ the former precedence values are treated as Class >> Selector codepoints, and methods for compatible treatment are >> described in the DiffServ architecture. The RFC 793/1122 TCP >> specification includes logic intending to have connections use the >> highest precedence requested by either endpoint application, and to >> keep the precedence consistent throughout a connection. >> > > > Right, RFC 7657 has slightly more refined text in the same vein on not > change DSCPs too often. > > There is an >> assumption of bidirectional/symmetric precedence values, however, the >> DiffServ architecture is asymmetric. Problems were described in [17] >> and the solution described is to ignore IP precedence in TCP. Since >> RFC 2873 is a Standards Track document (although not marked as >> updating RFC 793), these checks are no longer a part of the TCP >> standard defined in this document, though the DiffServ field value is >> still is a part of the interface between TCP and the network layer, >> > /is still is/ > >> and values can be indicated both ways between TCP and the >> application. >> >> >> I am not entirely I get the main points this paragraph tries to convey. >> Are they >> 1. a TCP stack MUST implement RFC2873 >> 2. precedence is obsoleted. >> ? >> >> I think something does have to be said about the way diffserv works, > although RFC 2873 is all the more vague by avoiding all keywords. > > >> >> Note that the DiffServ field is permitted to change during a >> connection (section 4.2.4.2 of RFC 1122). However, the >> application interface might not support this ability, >> > > > Well always possible, but do we really need to say that, since it kind of > breaks diffserv. > > and the >> application does not have knowledge about individual TCP >> segments, so this can only be done on a coarse granularity, at >> best. This limitation is further discussed in RFC 7657 (sec >> 5.1, 5.3, and 6) [37]. Generally, an application SHOULD NOT >> change the DiffServ field value during the course of a >> connection. >> >> >> > The underlying assumptions of this text is >> 1. Application can not specify precisely the DiffServ at TCP segments >> level >> 2. TCP does not tolerate reordering well today >> >> As API and TCP evolve and socket API isn't the only one, it would be >> good to be explicit about these assumptions. e.g. >> "Generally an application SHOULD NOT change DiffServ field value >> >> during the course of a connection because it may cause excessive >> reorderings and adversely hurts performance" >> >> nits: >> p32: s/the DiffServ field value is still is a part/ the DiffServ field >> value still is a part / >> p68: s/ SND.NEXT/SND.NXT/ >> p47: s/TCP may be buffer the data/TCP may buffer the data/ >> >> On Wed, Mar 28, 2018 at 9:01 PM, wrote: >> >>> >>> A New Internet-Draft is available from the on-line Internet-Drafts >>> directories. >>> This draft is a work item of the TCP Maintenance and Minor Extensions WG >>> of the IETF. >>> >>> Title : Transmission Control Protocol Specification >>> Author : Wesley M. Eddy >>> Filename : draft-ietf-tcpm-rfc793bis-08.txt >>> Pages : 102 >>> Date : 2018-03-28 >>> >>> Abstract: >>> This document specifies the Internet's Transmission Control Protocol >>> (TCP). TCP is an important transport layer protocol in the Internet >>> stack, and has continuously evolved over decades of use and growth of >>> the Internet. Over this time, a number of changes have been made to >>> TCP as it was specified in RFC 793, though these have only been >>> documented in a piecemeal fashion. This document collects and brings >>> those changes together with the protocol specification from RFC 793. >>> This document obsoletes RFC 793, as well as 879, 2873, 6093, 6429, >>> 6528, and 6691 that updated parts of RFC 793. It updates RFC 1122, >>> and should be considered as a replacement for the portions of that >>> document dealing with TCP requirements. It updates RFC 5961 due to a >>> small clarification in reset handling while in the SYN-RECEIVED >>> state. >>> >>> RFC EDITOR NOTE: If approved for publication as an RFC, this should >>> be marked additionally as "STD: 7" and replace RFC 793 in that role. >>> >>> >>> >>> The IETF datatracker status page for this draft is: >>> https://datatracker.ietf.org/doc/draft-ietf-tcpm-rfc793bis/ >>> >>> There are also htmlized versions available at: >>> https://tools.ietf.org/html/draft-ietf-tcpm-rfc793bis-08 >>> https://datatracker.ietf.org/doc/html/draft-ietf-tcpm-rfc793bis-08 >>> >>> A diff from the previous version is available at: >>> https://www.ietf.org/rfcdiff?url2=draft-ietf-tcpm-rfc793bis-08 >>> >>> >>> Please note that it may take a couple of minutes from the time of >>> submission >>> until the htmlized version and diff are available at tools.ietf.org. >>> >>> Internet-Drafts are also available by anonymous FTP at: >>> ftp://ftp.ietf.org/internet-drafts/ >>> >>> _______________________________________________ >>> tcpm mailing list >>> tcpm@ietf.org >>> https://www.ietf.org/mailman/listinfo/tcpm >>> >> >> _______________________________________________ >> tcpm mailing list >> tcpm@ietf.org >> https://www.ietf.org/mailman/listinfo/tcpm >> >> > --0000000000008402dd05692191ca Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


On Thu, Apr 5, 2018 at 10:14 AM, Gorry Fairhurst <= gorry@erg.abdn.ac= .uk> wrote:
A couple of com= ments-on-comments in-line,

Gorry


On 29/03/2018 15:10, Yuchung Cheng wrote:
Thanks for the update. I enjoy reading the new diffserv text.

comments:
<quote>
The IPv4 specification [1] includes a precedence value in the Type of
precedence will still have to check the precedence of incoming Service
field (TOS), that was also modified in [15], and then
segments and possibly raise the precedence level they use on the
obsoleted by the definition of Differentiated Services (DiffServ)
connection. [6].

I think this text kind of suggests ToS still exists. DiffServ is mature tec= hnology. I think we shouldn't even be hinting that ToS semantics are pe= rmitted - I'd suggest we firmly state that Diffserv has obsoleted the f= ormer TOS semantics.
+1

=C2=A0


In DiffServ the former precedence values are treated as Class
Selector codepoints, and methods for compatible treatment are
described in the DiffServ architecture. The RFC 793/1122 TCP
specification includes logic intending to have connections use the
highest precedence requested by either endpoint application, and to
keep the precedence consistent throughout a connection.
>
 Right, RFC 7657 has slightly more refined text in the same vein on not chan= ge DSCPs too often.

There is an
assumption of bidirectional/symmetric precedence values, however, the
DiffServ architecture is asymmetric. Problems were described in [17]
and the solution described is to ignore IP precedence in TCP. Since
RFC 2873 is a Standards Track document (although not marked as
updating RFC 793), these checks are no longer a part of the TCP
standard defined in this document, though the DiffServ field value is
still is a part of the interface between TCP and the network layer,
 /is still is/
and values can be indicated both ways between TCP and the
application.
</quote>

I am not entirely I get the main points this paragraph tries to convey. Are= they
1. a TCP stack MUST implement RFC2873
2. precedence is obsoleted.
?

 I think something does have to be said about the way diffserv works, althou= gh RFC 2873 is all the more vague by avoiding all keywords.


<quote>
Note that the DiffServ field is permitted to change during a
connection (section 4.2.4.2 of RFC 1122). However, the
application interface might not support this ability,
>
 Well always possible, but do we really need to say that, since it kind of b= reaks diffserv.

and the
application does not have knowledge about individual TCP
segments, so this can only be done on a coarse granularity, at
best. This limitation is further discussed in RFC 7657 (sec
5.1, 5.3, and 6) [37]. Generally, an application SHOULD NOT
change the DiffServ field value during the course of a
connection.
</quote>


The underlying assumptions of this text is
1. Application can not specify precisely the DiffServ at TCP segments level=
2. TCP does not tolerate reordering well today

As API and TCP evolve and socket API isn't the only one, it would be good to be explicit about these assumptions. e.g.
 "Generally an application SHOULD NOT change DiffServ field value
<= div class=3D"h5">
during the course of a connection because it may cause excessive
reorderings and adversely hurts performance"

nits:
p32: s/the DiffServ field value is still is a part/ the DiffServ field
value still is a part /
p68: s/ SND.NEXT/SND.NXT/
p47: s/TCP may be buffer the data/TCP may buffer the data/

On Wed, Mar 28, 2018 at 9:01 PM,=C2=A0 <internet-drafts@ietf.org> wrote:

A New Internet-Draft is available from the on-line Internet-Drafts director= ies.
This draft is a work item of the TCP Maintenance and Minor Extensions WG of= the IETF.

=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Title=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0: Transmission Control Protocol Specification
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Author=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = : Wesley M. Eddy
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Filename=C2=A0 =C2=A0 =C2=A0 =C2=A0 : dra= ft-ietf-tcpm-rfc793bis-08.txt
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Pages=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0: 102
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Date=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 : 2018-03-28

Abstract:
=C2=A0 =C2=A0 This document specifies the Internet's Transmission Contr= ol Protocol
=C2=A0 =C2=A0 (TCP).=C2=A0 TCP is an important transport layer protocol in = the Internet
=C2=A0 =C2=A0 stack, and has continuously evolved over decades of use and g= rowth of
=C2=A0 =C2=A0 the Internet.=C2=A0 Over this time, a number of changes have = been made to
=C2=A0 =C2=A0 TCP as it was specified in RFC 793, though these have only be= en
=C2=A0 =C2=A0 documented in a piecemeal fashion.=C2=A0 This document collec= ts and brings
=C2=A0 =C2=A0 those changes together with the protocol specification from R= FC 793.
=C2=A0 =C2=A0 This document obsoletes RFC 793, as well as 879, 2873, 6093, = 6429,
=C2=A0 =C2=A0 6528, and 6691 that updated parts of RFC 793.=C2=A0 It update= s RFC 1122,
=C2=A0 =C2=A0 and should be considered as a replacement for the portions of= that
=C2=A0 =C2=A0 document dealing with TCP requirements.=C2=A0 It updates RFC = 5961 due to a
=C2=A0 =C2=A0 small clarification in reset handling while in the SYN-RECEIV= ED
=C2=A0 =C2=A0 state.

=C2=A0 =C2=A0 RFC EDITOR NOTE: If approved for publication as an RFC, this = should
=C2=A0 =C2=A0 be marked additionally as "STD: 7" and replace RFC = 793 in that role.



The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/dra= ft-ietf-tcpm-rfc793bis/

There are also htmlized versions available at:
https://tools.ietf.org/html/draft-ietf-= tcpm-rfc793bis-08
https://datatracker.ietf.org/doc/html/draft-ietf-tcpm-rfc793bis-08

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?ur= l2=3Ddraft-ietf-tcpm-rfc793bis-08


Please note that it may take a couple of minutes from the time of submissio= n
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

_______________________________________________
tcpm mailing list
tcpm@ietf.org
https://www.ietf.org/mailman/listinfo/tcpm

_______________________________________________
tcpm mailing list
tcpm@ietf.org
https://www.ietf.org/mailman/listinfo/tcpm



--0000000000008402dd05692191ca-- From nobody Thu Apr 5 18:27:47 2018 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5B514126E01 for ; Thu, 5 Apr 2018 18:27:46 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.961 X-Spam-Level: X-Spam-Status: No, score=-3.961 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HEADER_FROM_DIFFERENT_DOMAINS=0.249, RCVD_IN_DNSWL_MED=-2.3, T_RP_MATCHES_RCVD=-0.01] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id dA4c3tEZ2_TC for ; Thu, 5 Apr 2018 18:27:41 -0700 (PDT) Received: from faui40.informatik.uni-erlangen.de (faui40.informatik.uni-erlangen.de [IPv6:2001:638:a000:4134::ffff:40]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id F055312E872 for ; Thu, 5 Apr 2018 18:27:40 -0700 (PDT) Received: from faui48f.informatik.uni-erlangen.de (faui48f.informatik.uni-erlangen.de [131.188.34.52]) by faui40.informatik.uni-erlangen.de (Postfix) with ESMTP id 51F0858C4B0 for ; Fri, 6 Apr 2018 03:27:36 +0200 (CEST) Received: by faui48f.informatik.uni-erlangen.de (Postfix, from userid 10463) id 4043E440214; Fri, 6 Apr 2018 03:27:36 +0200 (CEST) Date: Fri, 6 Apr 2018 03:27:36 +0200 From: Toerless Eckert To: tcpm@ietf.org Message-ID: <20180406012736.fpayup7perbhqn2p@faui48f.informatik.uni-erlangen.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: NeoMutt/20170113 (1.7.2) Archived-At: Subject: [tcpm] Better QoS for TCP ACK question X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Apr 2018 01:27:46 -0000 A couple of years ago i was told that a range of home routers would do some DPI based diffserv QoS, aka prioritizing TCP ACP messages in the queue before any other packets 9sonded like a strict priority queue for these packets). The goal was to improve performance when you had separate up and downstream TCP connections, especially when the upstream TCP connection delayed the downstream TCP connections ACKs. I never could find a lot of good details about this, e.g.: quantitiative measurements, so i was wondering if others here on the list are aware of it. Of course, if something like this would be significantly useful even in the face of current TCP congestion control mechanisms (and not only bad older ones). Cheers Toerless From nobody Fri Apr 6 13:56:02 2018 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 17C3A1242EA for ; Fri, 6 Apr 2018 13:56:00 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2 X-Spam-Level: X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=bobbriscoe.net Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id cut3NlRlcJzm for ; Fri, 6 Apr 2018 13:55:56 -0700 (PDT) Received: from server.dnsblock1.com (server.dnsblock1.com [85.13.236.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id A6C85126C83 for ; Fri, 6 Apr 2018 13:55:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=bobbriscoe.net; s=default; h=Content-Type:In-Reply-To:MIME-Version:Date: Message-ID:From:References:Cc:To:Subject:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=6pl97LbbKp0v4K2TlI67jgWmnby3MiqZYKfrG4XPCMA=; b=aJrjjQSs1NF3bgs35vg1jUPWp 2HIxlXNQl8ZsXVHKn0mybSTfgsTRYIPNXIlnqCU1N7gaSal6SKTdhtNUAUzDJVTPvpt4LG5XuxB/C IMnZWrYCsj9mTtgL67dwCjZcIAubErlxyo0UarY7LdOWQmaUPn6BbHoXPFJGnSoyYRO622Gl93xML JCgmJ7nORxkuPmXJxtaN7x1OPfM7Ui5mJau1zDQFWl4SHCPsGX5K8Nv2yAwmNpemMnGQc1UapfW8J sEVbFZ/ryMP0udnitIFWd/CwMcMXFYEJSA7LJglOb3PhAt6hhbFCkF7B2oPWmbdKTWL1fPUq1lFXG s7Q5M4WlQ==; Received: from [84.92.178.164] (port=35018 helo=[192.168.0.2]) by server.dnsblock1.com with esmtpsa (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim 4.89_1) (envelope-from ) id 1f4YOj-0008SY-Td; Fri, 06 Apr 2018 21:55:50 +0100 To: Michael Welzl Cc: Matt Mathis , tcpm IETF list , draft-ietf-tcpm-rack@tools.ietf.org References: <152029151070.12715.15497200993114582811@ietfa.amsl.com> <1c5bef94-eb4f-7aab-0b6b-cb2d02da7a9a@bobbriscoe.net> <6a451b96-af61-f5c8-87a9-9968e99027de@bobbriscoe.net> <1b5f83b1-d38c-d18c-b259-9817ae8f226c@bobbriscoe.net> <1D6E4D94-643D-497F-87E5-D4F8C72EA639@ifi.uio.no> From: Bob Briscoe Message-ID: Date: Fri, 6 Apr 2018 21:55:48 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 In-Reply-To: <1D6E4D94-643D-497F-87E5-D4F8C72EA639@ifi.uio.no> Content-Type: multipart/alternative; boundary="------------649DE44C8A37E74012268C41" Content-Language: en-GB X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - server.dnsblock1.com X-AntiAbuse: Original Domain - ietf.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - bobbriscoe.net X-Get-Message-Sender-Via: server.dnsblock1.com: authenticated_id: in@bobbriscoe.net X-Authenticated-Sender: server.dnsblock1.com: in@bobbriscoe.net Archived-At: Subject: Re: [tcpm] Vicious or Virtuous circle? Adapting reordering window to reordering degree X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Apr 2018 20:56:00 -0000 This is a multi-part message in MIME format. --------------649DE44C8A37E74012268C41 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Michael, Returning to this after a week tuned out (for which apologies)... On 29/03/18 12:42, Michael Welzl wrote: > Hi all, > > I was just about to propose a socket option for the upper limit on > reordering (e.g. as a function of the RTT) - to convey whether an > application is latency-critical or not. However, some of what Bob > wrote below throws me off… I keep only the relevant bits and > reordering them slightly for better context: > > >> On Mar 29, 2018, at 11:05 AM, Bob Briscoe > > wrote: >> >> Matt, >> >> 1. bandwidth inefficiency depends on spurious retransmissions, which >> *reduce* as reordering tolerance increases >> 2. loss recovery delay *increases* as reordering tolerance increases >> {Note 1} >> 3. delivery delay *reduces* as reordering tolerance increases {Note 2} >> > [snip] >> >> >> {Note 1}: There are other ways to remove losses, e.g. FEC, ECN. In an >> ideal world, if they were 100% successfully employed, there would be >> no trade off - increasing reordering tolerance would be wholly >> beneficial. > > The second sentence here confuses me: undoubtedly, for an application > that needs to get stuff in order, reordering can cause delay > (considering only one connection). That is, if it is produced e.g. by > in-network load balancing that sends some data over a longer-latency > path, that delay affects everything. E.g. ECN can’t help that?! [BB] I think you've overlooked the word "tolerance" in my Note 1. I was using the terminology Yuchung has been using: * Reordering degree = the amount of reordering introduced by the network * Reordering tolerance = the amount of reordering tolerated by the sender before retransmitting. If the sender increases its reordering tolerance, it waits longer before it retransmits, which increases the delay before losses get repaired. My Note 1 merely pointed out that, in an ideal world with ECN and FEC, retransmission would not be needed to repair losses, so reordering tolerance could be arbitrarily high. No need to sweat this point - it's theoretical, which is why I relegated it to a footnote (double irony: your reordering of my footnotes did not improve comprehension). > > >> {Note 2}: Shifting reordering to the end-system reduces head-of-line >> blocking of one stream when another has suffered a loss because it's >> the receiver's job to demultiplex between microflows, which naturally >> removes intra-flow HoL blocking. Also, if there are streams within an >> encrypted microflow (e.g. http2, quic), only the receiver can >> demultiplex the streams to resolve any HoL blocking. > > But we’re talking about TCP here, and generally I find this paragraph > hard to parse. You seem to have a multi-flow scenario in mind that > isn’t fully described. To begin with, without Minion’ish trickery, a > single TCP connection can only deliver data that arrives in order up > to the application, and hence reordering (when some data arrive later, > not earlier, because they use a longer-latency path) can cause delay > for that application. [BB] Actually, there's a bigger problem with my 3rd point - it's at a completely different timescale to the other two. * My first 2 trade offs occur immediately, at run-time, as soon as a particular sender alters its reordering tolerance. * Whereas the 3rd trade off occurs years after senders generally have increased their reordering tolerance (by deploying RACK) - once networks have evolved in response. Sorry I didn't make that clear. Actually, the first sentence of Note 2 was meant to apply to any transport protocol including TCP. Except I meant to write inter-flow, not intra-flow, so I don't blame you for being confused. You're right though about the second sentence - it does not apply to TCP, because it's about resolving HoL blocking between intra-flow streams, which TCP can't do. Sorry about both screw-ups. I tried to write that email quite precisely and carefully :( > > > A separate thing: > >> However, as I said in a previous posting, this doesn't address the >> *cost* trade off in processing and memory terms: >> >> 1. increasing reordering tolerance increases *receiver* memory & >> processing costs >> 2. reducing reordering tolerance increases *network* memory and >> processing costs (consequently capping network throughput) >> > About 1, isn’t this perhaps solved by your rwnd idea (your email to > Praveen)? [BB] Well, the rwnd idea allows the receiver to control its cost, otherwise control of reordering tolerance is purely with the sender. But this doesn't "solve" the bigger problem. It shifts the cost to the network (by making the sender reduce its reordering tolerance, which shifts to #2 scenario). But it doesn't make the host/network trade off go away. > If it is, then the answer to 2 is: fine, we can increase reordering > tolerance anyway. > > Generally, I agree with the pro-reordering musings in this thread, > e.g. from Matt - we have a silly situation in the network today, where > TCP acts as if it was a connection oriented network and in-network > devices jump through various hoops to maintain this illusion. In fact, > the Internet is a packet oriented network, and if we can tweak things > a little in a direction that allows people to tune the network in a > more packet oriented fashion (instead of TCP-connection-oriented), > then that’s a good thing. > > Looking at the draft, RACK asks for Experimental status. Couldn’t we > see "finding out how much reordering is acceptable” as part of the > experiment?  I mean, some version of RACK is in Linux already, and > things haven’t broken badly as far as I know.  This experiment would > at least let us uncover how severe the processing / memory costs above > really are. [BB] ...true, with the proviso that, over any reasonable experiment timescale, you wouldn't see the network evolve much, so you wouldn't know how much greater reordering might get once the rules for in-order forwarding are relaxed. > > Cheers, > ichMael   (Michael, reordered) > [BB] Cheers boB -- ________________________________________________________________ Bob Briscoe http://bobbriscoe.net/ --------------649DE44C8A37E74012268C41 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit Michael,

Returning to this after a week tuned out (for which apologies)...

On 29/03/18 12:42, Michael Welzl wrote:
Hi all,

I was just about to propose a socket option for the upper limit on reordering (e.g. as a function of the RTT) - to convey whether an application is latency-critical or not. However, some of what Bob wrote below throws me off… I keep only the relevant bits and reordering them slightly for better context:


On Mar 29, 2018, at 11:05 AM, Bob Briscoe <ietf@bobbriscoe.net> wrote:

Matt,
  1. bandwidth inefficiency depends on spurious retransmissions, which reduce as reordering tolerance increases
  2. loss recovery delay increases as reordering tolerance increases {Note 1}
  3. delivery delay reduces as reordering tolerance increases {Note 2}
[snip]

{Note 1}: There are other ways to remove losses, e.g. FEC, ECN. In an ideal world, if they were 100% successfully employed, there would be no trade off - increasing reordering tolerance would be wholly beneficial.

The second sentence here confuses me: undoubtedly, for an application that needs to get stuff in order, reordering can cause delay (considering only one connection). That is, if it is produced e.g. by in-network load balancing that sends some data over a longer-latency path, that delay affects everything. E.g. ECN can’t help that?!
[BB] I think you've overlooked the word "tolerance" in my Note 1.

I was using the terminology Yuchung has been using:
* Reordering degree = the amount of reordering introduced by the network
* Reordering tolerance = the amount of reordering tolerated by the sender before retransmitting.

If the sender increases its reordering tolerance, it waits longer before it retransmits, which increases the delay before losses get repaired.

My Note 1 merely pointed out that, in an ideal world with ECN and FEC, retransmission would not be needed to repair losses, so reordering tolerance could be arbitrarily high. No need to sweat this point - it's theoretical, which is why I relegated it to a footnote (double irony: your reordering of my footnotes did not improve comprehension).



{Note 2}: Shifting reordering to the end-system reduces head-of-line blocking of one stream when another has suffered a loss because it's the receiver's job to demultiplex between microflows, which naturally removes intra-flow HoL blocking. Also, if there are streams within an encrypted microflow (e.g. http2, quic), only the receiver can demultiplex the streams to resolve any HoL blocking.

But we’re talking about TCP here, and generally I find this paragraph hard to parse. You seem to have a multi-flow scenario in mind that isn’t fully described. To begin with, without Minion’ish trickery, a single TCP connection can only deliver data that arrives in order up to the application, and hence reordering (when some data arrive later, not earlier, because they use a longer-latency path) can cause delay for that application.
[BB] Actually, there's a bigger problem with my 3rd point - it's at a completely different timescale to the other two.
* My first 2 trade offs occur immediately, at run-time, as soon as a particular sender alters its reordering tolerance.
* Whereas the 3rd trade off occurs years after senders generally have increased their reordering tolerance (by deploying RACK) - once networks have evolved in response.
Sorry I didn't make that clear.

Actually, the first sentence of Note 2 was meant to apply to any transport protocol including TCP. Except I meant to write inter-flow, not intra-flow, so I don't blame you for being confused.

You're right though about the second sentence - it does not apply to TCP, because it's about resolving HoL blocking between intra-flow streams, which TCP can't do.

Sorry about both screw-ups. I tried to write that email quite precisely and carefully :(




A separate thing:

However, as I said in a previous posting, this doesn't address the cost trade off in processing and memory terms:
  1. increasing reordering tolerance increases receiver memory & processing costs
  2. reducing reordering tolerance increases network memory and processing costs (consequently capping network throughput)
About 1, isn’t this perhaps solved by your rwnd idea (your email to Praveen)?
[BB] Well, the rwnd idea allows the receiver to control its cost, otherwise control of reordering tolerance is purely with the sender.

But this doesn't "solve" the bigger problem. It shifts the cost to the network (by making the sender reduce its reordering tolerance, which shifts to #2 scenario). But it doesn't make the host/network trade off go away.

If it is, then the answer to 2 is: fine, we can increase reordering tolerance anyway.

Generally, I agree with the pro-reordering musings in this thread, e.g. from Matt - we have a silly situation in the network today, where TCP acts as if it was a connection oriented network and in-network devices jump through various hoops to maintain this illusion. In fact, the Internet is a packet oriented network, and if we can tweak things a little in a direction that allows people to tune the network in a more packet oriented fashion (instead of TCP-connection-oriented), then that’s a good thing.

Looking at the draft, RACK asks for Experimental status. Couldn’t we see "finding out how much reordering is acceptable” as part of the experiment?  I mean, some version of RACK is in Linux already, and things haven’t broken badly as far as I know.  This experiment would at least let us uncover how severe the processing / memory costs above really are.
[BB] ...true, with the proviso that, over any reasonable experiment timescale, you wouldn't see the network evolve much, so you wouldn't know how much greater reordering might get once the rules for in-order forwarding are relaxed.



Cheers,
ichMael   (Michael, reordered)

[BB] Cheers


boB





-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/
--------------649DE44C8A37E74012268C41-- From nobody Fri Apr 6 17:49:30 2018 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6C4A212706D; Fri, 6 Apr 2018 17:49:28 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 19drD0Ig8GN4; Fri, 6 Apr 2018 17:49:22 -0700 (PDT) Received: from mout.gmx.net (mout.gmx.net [212.227.17.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 152CD1267BB; Fri, 6 Apr 2018 17:49:21 -0700 (PDT) Received: from [10.249.68.49] ([217.70.210.5]) by mail.gmx.com (mrgmx102 [212.227.17.168]) with ESMTPSA (Nemesis) id 0MVayZ-1f01qN08xL-00Z2CE; Sat, 07 Apr 2018 02:48:28 +0200 To: Bob Briscoe , "Scharf, Michael (Nokia - DE/Stuttgart)" Cc: =?UTF-8?Q?Mirja_K=c3=bchlewind?= , "draft-ietf-tcpm-accurate-ecn@ietf.org" , "tcpm@ietf.org" References: <44FDECF5-A031-4343-BA1A-AE0D9C2C078C@tik.ee.ethz.ch> <7869808f-4e15-1180-da89-a0f2b043b46c@bobbriscoe.net> From: "Scheffenegger, Richard" Message-ID: <1764966d-b916-c7e7-3d96-370ecf110a11@gmx.at> Date: Sat, 7 Apr 2018 02:48:26 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 In-Reply-To: <7869808f-4e15-1180-da89-a0f2b043b46c@bobbriscoe.net> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-Provags-ID: V03:K1:c96fNvNErcCLECfEB9yfdKi4xaMKvCA2cnH/m6ZJ9KhwKP5C4Hb bCfj+5wH0NCeUnfFYVpB9feVjGJgf3OlJxo+eVdkWG9TXdVdnIzuLuNnKVop3RvCFhBPf// AmC/O1E9Uf7kOX4KqPzAmpzK+6jxq033I6/B6+DCMSMFQuwyPqnkhOWDmEVp/zWrT2zgpeI 00vbwNTZdGyTK3WknBtwA== X-UI-Out-Filterresults: notjunk:1;V01:K0:pts0RU9PXxc=:2b9ktlHLXgpKW6hq9sZV8o hzwldxmfnveVf5KwEJJ8O9+rDGFcHfaHe0xaFriJviQobVPiPePCBmxP/IS9JPk7qS/S1gpOF ziL+hIKavEuvUwuoaDpaPzjWMpwI/L0xqIxcYRLpVSjVJZkwfsHdDzZNyi8kZAfdUIszpg0AF MjNoZi+rQ8olChBA5cNGvYgQzBTxh/YfHMYtmO3d4KbWkWw65cNXvUxjvXnQiLqrAXxoD2LYq 6Dz3BKVv243g0TnaPcHrV2eHsLwvg07tJJActhl1uUGVauUUbW6b0Fh9ddXbsF2AdPD45UlOK SLLJoZJb3Ja26v3EwHINp6L+knHnqaYO+kTbS0rgwKHgxyz2LQd7LtoKwIsazuUJi1imhiR7S efpcrIJwWK7SRGKX1MlfDfsijhMnpzIkpS0vuLBoKj+2KhNiVnSqe9yvANxF61uj8OrXPtM8q MDsQoJr7CPt4FCSFY6fF436xGIydChL8ShyTRSpq1cr5PZMbgw6y4A0pz5pYIUHVttFNBE9r0 RlklmOGqm9BgAby8JujVVOybFEVN+umzPF1NfvceUCHLlT7Mi5xdolBAskhkL4KkVGyv0MP9Y UCiZYWAIfci58ELnfBTUD5SqGNyz8HXgcJygBI7knKiYiMM92S/k7wVhs2UsdEDam04l4Dxgj i8q9b7FSrKw4SXLqUutm0QMF+S8UF3RNrSj0F0AzvLrXRt9CPYUWyBl9YfMgc6JbVmABw0wpP GwZrkY4qmlAOMgmmf3A5RhXfih8qMT4X77Av3vDBNK6OVkU8HEbuPJN4vQTaAmD1DNPCG6jrK Tcx+9I6Bt0FCk1DZGW9I7qtV1yCk3TdlMyzn3KgYfqbgGv4pTo= Archived-At: Subject: Re: [tcpm] Comments on draft-ietf-tcpm-accurate-ecn X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Apr 2018 00:49:28 -0000 Here is the email referenced by Bob, which I missed to forward to the list: Hi Bob, > In fact, during capability negotiation we are now treating the 3 flags > as 8 codepoints, and we still avoid the codepoints that the ECN nonce > used, even tho we believe there is no nonce deployment. This proves we > are really not using any more header flags than have already been > burned. Actually, there are these two (marked ###) entries, which could be used as some extension of ECN signaling (different ACE encoding or something alike). There is the one for obsolete Nonce in the middle of the table, and the last entry, where all bits are reflected, is an extension of the original 3168 handshake when it was noted that some broken stacks would reflect unknown header bits back 1:1, instead of clearing them on the SYN/ACK. The entry marked “ooo” also falls into that category, I think I observed one or two examples of hosts with a stack – probably a broken ECN implementation attempt. But I think it’s safe to gloss over that 😊 +--------+--------+------------+-------------+----------------------+ | A | B | SYN A->B | SYN/ACK | Feedback Mode | | | | | B->A | | +--------+--------+------------+-------------+----------------------+ | | | AE CWR ECE | AE CWR ECE | | | AccECN | AccECN | 1 1 1 | 0 1 0 | AccECN (Not-ECT on | | | | | | SYN) | ooo| AccECN | AccECN | 1 1 1 | 0 1 1 | AccECN (ECT1 on SYN) | | AccECN | AccECN | 1 1 1 | 1 0 0 | AccECN (ECT0 on SYN) | | AccECN | AccECN | 1 1 1 | 1 1 0 | AccECN (CE on SYN) | | | | | | | ###| AccECN | Nonce | 1 1 1 | 1 0 1 | classic ECN | | AccECN | ECN | 1 1 1 | 0 0 1 | classic ECN | | AccECN | No ECN | 1 1 1 | 0 0 0 | Not ECN | | | | | | | | Nonce | AccECN | 0 1 1 | 0 0 1 | classic ECN | | ECN | AccECN | 0 1 1 | 0 0 1 | classic ECN | | No ECN | AccECN | 0 0 0 | 0 0 0 | Not ECN | | | | | | | ###| AccECN | Broken | 1 1 1 | 1 1 1 | Not ECN | +--------+--------+------------+-------------+----------------------+ We could state, that once another experiment has determined that there are no Nonce or Broken Stacks around any more (and there were extremely few broken ones, but no Nonce, around back in 2014 (?) when I checked this much to the amusement of the IETF plenary, since I also checked SMTP/POP3/IMAP ports 😊 ), that these two Codepoints in the negotiation phase allow further extention of the ECN feedback scheme in the future… Thinking about this a bit more – in the 2nd to last block of entries in this table, we have different Senders talking to AccECN; However, that table is not complete. We could add “A: future” with the following combinations 1 0 0 -> AccECN receiver 1 0 1 -> AccECN receiver 1 1 0 -> AccECN receiver To have a way that a backwards compatible, future handshake scheme can get the receiver into an AccECN compatible signaling mode, while a “future” receiver may react with one of the other codepoints to run in a different mode…. And “Broken” for completeness (paraphrasing 3168) 0 0 1 -> non ECN 0 1 0 -> non ECN I think if we make a statement about that potential future use for these codepoints, he might already be happy. One last point – shall we mention the ECT1 on SYN explicitly in the context of L4S-arch? What do you think? I think having a upwards compatible accecn feedback mode could be valuable (for theoretical transition from non-ECN -> 3168 -> AccECN -> future ECN) while retaining the capabilities of the previous generation of receiver feedback. Best regards, Richard Am 07.04.2018 um 01:16 schrieb Bob Briscoe: > Michael, > > I realize now that I misunderstood your question during the TCPM session > in London. (Sorry for the delay - I tuned out for a while). > > As well as the 2 codepoints that Richard has highlighted, the draft is > careful to require that implementations of the AccECN experiment will > silently accept future use of other unexpected codepoints ('forward > compatibility'). For instance, at the end of S.3.2,3 > : > > Note that the Data Sender MUST NOT test whether the arriving counter > in the initial ACE field has been initialized to a specific valid > value - the above check solely tests whether the ACE fields have been > incorrectly zeroed. This allows hosts to use different initial > values as an additional signalling channel in future. > > We have only used codepoints 6&7 here. Assuming we avoid zero, because > it might be due to bleaching, this leaves values 1-5 unused. > > The analysis I did of the remaining combinations of codepoints when > taken across the whole handshake is here: > http://bobbriscoe.net/projects/latency/ecn-modes.pdf > Page 1 covers the TCP ECN flags. > > The 1st row marked A-CU (short for Accurate ECN Currently Unassigned) > shows the 5 values that the above text keeps for future use. > The 2nd two rows marked A-CU are now used by AccECN (since draft-04) to > deal with the bleaching we discovered during testing. > All the combinations tagged with 'N' (for nonce) are likely to be the > most useful if needed for any additional negotiation space. > > The other one Richard tagged (labelled 'broken') is still polluted by a > non-negligible number of bugged servers that reflect the 6 most > significant TCP flags in the SYN back in the SYN/ACK. From a 2014 > measurement study that Richard and Mirja co-authored, there were still > about 0.3% of the Alexa 1M exhibiting the symptoms of this bug. I'm in > the process of checking our more recent data, where we specifically > checked if 111 used on the AccECN SYN was reflected. > > Taken as a whole, I admit this doesn't leave much flexibility for future > variation, but it's the best we could do with the available space > constraints. > Does this sufficiently address your concern? > > > Cheers > > > > Bob > > PS. Similarly, although we have not allowed any space in the AccECN TCP > Option for flag bits, we have allowed for different initial values of > the counters to be used to signal new versions of the protocol in > future. See the end of S.3.2.7.4 > : > > Note that the Data Sender MUST NOT test whether the arriving byte > counters in the initial AccECN Option have been initialized to > specific valid values - the above checks solely test whether these > fields have been incorrectly zeroed. This allows hosts to use > different initial values as an additional signalling channel in > future. > > > > On 19/03/18 16:04, Scharf, Michael (Nokia - DE/Stuttgart) wrote: >> For what it is worth, this is exactly the sort of discussion I believe we need to have. >> >> I would be much more comfortable if the document did foresee some way negotiate an "enhanced" version of Accurate ECN in a later phase - most notably a PS version of the protocol. If that would cost some functionality right now, as we may have to reserve some codepoints in the EXP version, that would be perfectly fine for me, as long as we could add that functionality later in a PS version (under our current understanding). >> >> I have not thought about the details. But we have only three reserved TCP header bits left after publication of this document and I am worried about that. >> >> Michael >> >> >>> -----Original Message----- >>> From: Richard Scheffenegger [mailto:rs.ietf@gmx.at] >>> Sent: Monday, March 19, 2018 4:27 PM >>> To: Scharf, Michael (Nokia - DE/Stuttgart); >>> Mirja Kühlewind >>> Cc:draft-ietf-tcpm-accurate-ecn@ietf.org;tcpm@ietf.org >>> Subject: Re: [tcpm] Comments on draft-ietf-tcpm-accurate-ecn >>> >>> Hi Michael, >>> >>> regarding your comment about the lack of future extensibility of the ECN / >>> AccECN header handshake without further bits... >>> >>> I wanted to note, that the logic table we have in the accecn draft currently >>> has two entries, that are reserved because of observed, but legacy, stacks >>> misbehaving when reflecting these TCP header bits. >>> >>> So there is a possibility to eventually use those entries - once the broken >>> legacy stacks have been confirmeed to have left the internet - as some >>> additional negotiation capability. >>> >>> Also, the feedback of the byte counters vs. CE packet counter are split into >>> using a tcp option and the tcp header bits. >>> An extention of AccECN could take the form of using a differen TCP option >>> together with the header counter negotiation... >>> >>> Earlier work by us determined that packing more information into the tcp >>> header bits alone (as an example) would likely need an additional bit (that >>> can also be used in a negotiation scheme)... >>> >>> >>> +--------+--------+------------+-------------+----------------------+ >>> | A | B | SYN A->B | SYN/ACK | Feedback Mode | >>> | | | | B->A | | >>> +--------+--------+------------+-------------+----------------------+ >>> | | | AE CWR ECE | AE CWR ECE | | >>> | AccECN | AccECN | 1 1 1 | 0 1 0 | AccECN (Not-ECT on | >>> | | | | | SYN) | >>> | AccECN | AccECN | 1 1 1 | 0 1 1 | AccECN (ECT1 on SYN) | >>> | AccECN | AccECN | 1 1 1 | 1 0 0 | AccECN (ECT0 on SYN) | >>> | AccECN | AccECN | 1 1 1 | 1 1 0 | AccECN (CE on SYN) | >>> | | | | | | >>> # | AccECN | Nonce | 1 1 1 | 1 0 1 | classic ECN | >>> | AccECN | ECN | 1 1 1 | 0 0 1 | classic ECN | >>> | AccECN | No ECN | 1 1 1 | 0 0 0 | Not ECN | >>> | | | | | | >>> | Nonce | AccECN | 0 1 1 | 0 0 1 | classic ECN | >>> | ECN | AccECN | 0 1 1 | 0 0 1 | classic ECN | >>> | No ECN | AccECN | 0 0 0 | 0 0 0 | Not ECN | >>> | | | | | | >>> # | AccECN | Broken | 1 1 1 | 1 1 1 | Not ECN | >>> +--------+--------+------------+-------------+----------------------+ >>> Best regards, >>> Richard >>> >>> >>> ----- Original Message ----- >>> From: "Scharf, Michael (Nokia - DE/Stuttgart)" >>> To: "Mirja Kühlewind" >>> Cc:; >>> Sent: Monday, March 12, 2018 1:59 AM >>> Subject: Re: [tcpm] Comments on draft-ietf-tcpm-accurate-ecn >>> >>> >>>> Hi Mirja, >>>> >>>> Thanks a lot for the explanation. I won't follow-up on some of the >>>> editorial suggestions. >>>> >>>> Yet, I continue to believe that some formal wording in the document needs >>>> to change, as explained below. >>>> >>>> Thanks >>>> >>>> Michael (with no hat on) >>>> >>>> >>>>> -----Original Message----- >>>>> From: Mirja Kühlewind [mailto:mirja.kuehlewind@tik.ee.ethz.ch] >>>>> Sent: Monday, March 05, 2018 1:54 PM >>>>> To: Scharf, Michael (Nokia - DE/Stuttgart) >>>>> Cc:draft-ietf-tcpm-accurate-ecn@ietf.org;tcpm@ietf.org >>>>> Subject: Re: Comments on draft-ietf-tcpm-accurate-ecn >>>>> >>>>> Hi Micheal, >>>>> >>>>> thanks for your feedback and sorry for my late reply. >>>>> >>>>> Please see inline. >>>>> >>>>>> Am 03.12.2017 um 20:17 schrieb Scharf, Michael (Nokia - DE/Stuttgart) >>>>> : >>>>>> Hi all, >>>>>> >>>>>> I have read draft-ietf-tcpm-accurate-ecn-05 (without the appendix). I >>>>> believe this document needs further work before moving forward. >>>>>> Please find below my comments marked as [ms]. I have read the >>>>> document independent of the review from Gorry. I apologize if there is >>>>> duplication. >>>>>> Thanks >>>>>> >>>>>> Michael (with no hat on) >>>>>> >>>>>> >>>>>> ****************************** >>>>>> >>>>>> * Abstract: >>>>>> >>>>>> Recently, new TCP mechanisms like Congestion Exposure (ConEx) or >>>>>> Data >>>>> Center TCP >>>>>> (DCTCP) need more accurate ECN feedback information whenever >>> more >>>>>> than one marking is received in one RTT. >>>>>> >>>>>> [ms] I don't think this statement is fully backed by RFC 8257. I >>>>>> suggest to >>>>> remove this, or replace it by a more generic statement that more accurate >>>>> information can be useful for several TCP extensions. >>>>> >>>>> I disagree. Both ConEx and DCTCP need more accurate information. They >>> do >>>>> not need the mechanism that is specified in this draft, however, this is >>>>> not >>>>> what the sentences is saying. >>>> In my understanding (as a non-native speaker), the use of the word "need" >>>> is not correct here. DCTCP as specified in RFC 8257 can be implemented >>>> without any such mechanism. >>>> >>>> What would work for me is something of the form "... Data Center TCP >>>> cannot get precise ECN feedback whenever more than one marking is >>> received >>>> in one RTT". >>>> >>>>>> This document specifies an >>>>>> experimental scheme to provide more than one feedback signal per >>> RTT >>>>>> in the TCP header. Given TCP header space is scarce, it overloads >>>>>> the three existing ECN-related flags in the TCP header and provides >>>>>> additional information in a new TCP option. >>>>>> >>>>>> [ms] This statement needs to be rewritten to correctly reflect what is >>>>> requested from IANA. My understanding is that this experimental >>> document >>>>> asks for allocation of a reserved TCP header flag. This needs to be >>>>> called out >>>>> prominently, IMHO. In addition, since this is not a standard, the >>>>> suggested >>>>> experimentation with the main TCP header must IMHO be explicitly >>>>> mentioned. I also suggest to have later in a document a section that >>>>> explicitly >>>>> explains why it is appropriate to modify the main TCP header in an >>>>> experiment. >>>>> >>>>> I don’t know if any requirement that IANA assignment need to be called >>>>> out >>>>> in the abstract but we can do that. However, I believe the question if >>>>> this >>>>> document should or should not assign the bit is still not completely >>>>> solved, or >>>>> is it? >>>> I believe this question will have to be reviewed during WGLC and, more >>>> importantly, IETF last call. For the moment, my concern is that the >>>> document correctly describes the IANA allocation. >>>> >>>> I would like to see here a statement such as : "Given TCP header space is >>>> scarce, this specification allocates a reserved header bit and overloads >>>> the two ECN flags in the TCP header ...". >>>> >>>>>> * 1. Introduction >>>>>> >>>>>> Recently, proposed mechanisms like Congestion Exposure (ConEx >>>>>> [RFC7713]), DCTCP [RFC8257] or L4S [I-D.ietf-tsvwg-l4s-arch] need >>>>>> more accurate ECN feedback information whenever more than one >>>>> marking >>>>>> is received in one RTT. >>>>>> >>>>>> [ms] At least for RFC 8257 seems to be implementable withoit this. >>>>>> Instead >>>>> of stating a "need", it would IMHO make more sense to discuss the >>>>> benefits >>>>> of the suggested mechanism in this document of its own, independent of >>>>> other proposals. To me, this document should be independent of other >>>>> documents and specifically other experiments. We have to think about >>>>> cases >>>>> where not all experiments are successful. Then independent documents >>> will >>>>> be more future-proof in future. >>>>> >>>>> This is a naming collision… The sentence was meant to say that these >>>>> mechanisms new more accurate ECN feedback than provided today by >>>>> RFC3168 but it was not meant to say that these mechanism have to use >>> the >>>>> scheme as specified in this document. >>>>> >>>>> I added the following part sentence: >>>>> >>>>> „Recently, proposed mechanisms like Congestion Exposure (ConEx >>>>> [RFC7713]), DCTCP [RFC8257] or L4S [I-D.ietf-tsvwg-l4s-arch] need more >>>>> accurate ECN feedback information than provided by the feedback >>> scheme >>>>> as specified in [RFC3168] whenever more than one marking is received in >>>>> one >>>>> RTT. This document specifies an alternative feedback scheme that >>> provides >>>>> more accurate information and could be used by these new TCP >>> extensions.“ >>>>> Does this help? >>>> See my proposal for the abstract. I continue to disagree with the term >>>> "need" but I think this can be sorted out by another term. >>>> >>>>>> If AccECN progresses from experimental to the standards >>>>>> track, it is intended to be a complete replacement for classic TCP/ >>>>>> ECN feedback, not a fork in the design of TCP. >>>>>> >>>>>> [ms] This sentence should be removed, as this is speculation. >>>>> Why? It states an intent… and that’s the intent that we have. >>>>> >>>>>> Until the AccECN experiment succeeds, [RFC3168] will remain as the >>>>>> standards track specification for adding ECN to TCP. >>>>>> >>>>>> [ms] This sentence should be removed (or reworded) >>>>> Why? Does it help to add an only here: >>>>> >>>>> "Until the AccECN experiment succeeds, [RFC3168] will remain as the only >>>>> standards track specification for adding ECN to TCP.“ >>>> This wording is better. >>>> >>>>>> AccECN feedback overloads flags and fields in the main TCP header >>>>>> with new definitions, so both ends have to support the new wire >>>>>> protocol before it can be used. >>>>>> >>>>>> [ms] In my reading this experimental document asks for *new* >>> allocation >>>>> of a reserved TCP header flag. >>>>> >>>>> Is this better? >>>>> >>>>> "AccECN feedback overloads the two existing ECN flags as well as the >>>>> currently reserved and previously called NS flag in the main TCP >>>>> header >>>>> with new definitions, so both ends have to support the new wire >>>>> protocol >>>>> before it can be used.“ >>>>> >>>>> I understand that you are not happy with the word „overload“ here but >>> the >>>>> point of this sentence really is that the flags can/could be used >>>>> differently >>>>> and therefore we need a new negotiation before we can use them. >>>> For me the following would work: "AccECN feedback overloads the two >>>> existing ECN flags and >>>> allocates the currently reserved and previously called NS flag in the main >>>> TCP header. >>>> Given the new definitions, both ends have to support the new wire >>> protocol >>>> before it can be used." >>>> >>>> I believe the wording has to be crystal clear on the reservation of bit 7 >>>> when it is discussed the first time in the text. In follow-up sections, >>>> maybe shorter terms could be used. >>>> >>>>> If you prefer, we can also remove the NS flag in this list, as ECN Nonce >>>>> was >>>>> anyway never deployed. >>>>> >>>>>> For that we refer to [RFC3168] or any RFC that >>>>>> specifies a different response to TCP ECN feedback, for example: >>>>>> [RFC8257]; or the ECN experiments referred to in >>>>>> [I-D.ietf-tsvwg-ecn-experimentation], namely: a TCP-based Low >>>>>> Latency >>>>>> Low Loss Scalable (L4S) congestion control >>>>>> [I-D.ietf-tsvwg-l4s-arch]; >>>>>> ECN-capable TCP control packets [I-D.ietf-tcpm-generalized-ecn], or >>>>>> Alternative Backoff with ECN (ABE) >>>>>> [I-D.ietf-tcpm-alternativebackoff-ecn]. >>>>>> >>>>>> [ms] At least ABE seems orthogonal. Anyway, I think this paragraph can >>>>>> just >>>>> be deleted. If other experiments need more accurate feedback, it is up to >>>>> them to explain how they would use this mechanism. This document >>> should >>>>> focus on how to signal the feedback, not how to use that. >>>>> >>>>> Yes, that is what the paragraph says. Isn’t it better to be explicit >>>>> about this? >>>>> >>>>>> It is likely (but not required) that the AccECN protocol will be >>>>>> implemented along with the following experimental additions to the >>>>>> TCP-ECN protocol: ECN-capable TCP control packets and >>>>>> retransmissions >>>>>> [I-D.ietf-tcpm-generalized-ecn], which includes the ECN-capable SYN/ >>>>>> ACK experiment [RFC5562]; and testing receiver non-compliance >>>>>> [I-D.moncaster-tcpm-rcv-cheat]. >>>>>> >>>>>> [ms] I am a big fan of simple, standalone documents. In my view, the >>>>>> TCPM >>>>> working group should publish draft-ietf-tcpm-accurate-ecn and draft-ietf- >>>>> tcpm-generalized-ecn independent documents, which probably implies >>> that >>>>> draft-ietf-tcpm-generalized-ecn does not use AccECN. If experimentation >>>>> with ECT in SYN requires a combination, this could be done in a new, >>>>> third >>>>> document. Apart from having simpler focused documents, this could >>>>> significantly help later with moving forward documents to standards >>>>> track. >>>>> >>>>> I disagree, however, this is a discussion to have on draft-ietf-tcpm- >>>>> generalized-ecn. I don’t see a problem in providing a reference here >>>>> that >>>>> says „it is likely…“ and nothing more. >>>>>> >>>>>> >>>>>> * 1.1. Document Roadmap >>>>>> >>>>>> [ms] A macroscopic comment is that this document has a lot of >>>>>> introduction >>>>> and tutorial text with lot's of redundancy towards other documents. I >>>>> think >>>>> the document can be made much easier to read by shorten it. In many >>> cases >>>>> this is just an editorial change as there is redundancy. As one such >>>>> example, >>>>> just remove this section. >>>>> >>>>> I guess this a matter of taste. As an AD, I’m a big fan of short and >>>>> concise >>>>> documents, however, some redundancy can also help understanding, >>>>> especially if you explain things multiple times but with a different >>>>> level of >>>>> detail. I personally would not need the roadmap but I know many people >>>>> who find these things helpful and to be honest I don’t see how removing >>>>> this >>>>> part makes the doc any better. If you don’t want it, don’t read it. >>>>> >>>>>> >>>>>> * 1.2. Goals >>>>>> >>>>>> [ms] I think this section can also just be removed. >>>>> I have to say I also don’t see the point of removing this part. Given we’ve >>>>> done the work on requirements, I think we should also link to this doc >>>>> somewhere. >>>>>> >>>>>> * 1.3. Experiment Goals >>>>>> >>>>>> TCP is critical to the robust functioning of the Internet, therefore >>>>>> any proposed modifications to TCP need to be thoroughly tested. The >>>>>> present specification describes an experimental protocol that adds >>>>>> more accurate ECN feedback to the TCP protocol. The intention is to >>>>>> specify the protocol sufficiently so that more than one >>>>>> implementation can be built in order to test its function, >>>>>> robustness >>>>>> and interoperability (with itself and with previous version of ECN >>>>>> and TCP). >>>>>> >>>>>> [ms] I think all what is written in this paragraph is obvious, no? >>>>>> Can't we just >>>>> delete this? >>>>> >>>>> Sure, however, I don’t think it hurts to spell it out. For me both is >>>>> fine, keep it >>>>> or remove it. >>>>> >>>>>> The experimental protocol will be considered successful if it is >>>>>> deployed and if it satisfies the requirements of [RFC7560] in the >>>>>> consensus opinion of the IETF tcpm working group. In short, this >>>>>> requires that it improves the accuracy and timeliness of TCP's ECN >>>>>> feedback, as claimed in Section 5, while striking a balance between >>>>>> the conflicting requirements of resilience, integrity and >>>>>> minimisation of overhead. It also requires that it is not unduly >>>>>> complex, and that it is compatible with prevalent equipment >>>>>> behaviours in the current Internet (e.g. hardware offloading and >>>>>> middleboxes), whether or not they comply with standards. >>>>>> >>>>>> Testing will mostly focus on fall-back strategies in case of >>>>>> middlebox interference. Current recommended strategies are >>>>>> specified >>>>>> in Sections 3.1.2, 3.2.3, 3.2.4 and 3.2.7. The effectiveness of >>>>>> these strategies depends on the actual deployment situation of >>>>>> middleboxes. Therefore experimental verification to confirm large- >>>>>> scale path traversal in the Internet is needed before finalizing >>>>>> this >>>>>> specification on the Standards Track. >>>>>> >>>>>> [ms] These two paragraphs must be entirely rewritten. As I have >>>>> mentioned before, I don't think an RFC should speculate about TCPM and >>>>> its >>>>> consensus opinion. I would suggest a wording along the lines of: >>>>>> >>>>>> The experimental protocol will be considered successful if >>>>>> testing confirms that the proposed mechanism can be deployed at >>>>>> large >>>>> scale. >>>>>> Testing will mostly focus on fall-back strategies in case of >>>>>> middlebox interference. Current recommended strategies are >>>>>> specified >>>>>> in Sections 3.1.2, 3.2.3, 3.2.4 and 3.2.7. The effectiveness of >>>>>> these strategies depends on the actual deployment situation of >>>>>> middleboxes. Therefore experimental verification to confirm large- >>>>>> scale path traversal in the Internet is needed, e.g., by support in >>>>>> major TCP stacks. >>>>>> >>>>>> >>>>> I don’t understand your point here. I don’t think that the paraphrase >>>>> speculates about the consensus of tcpm, in contrast it say tcpm has to >>>>> decided if the requirements previously specified by tcpm are sufficiently >>>>> fulfilled. I don’t see a reason to not mention the requirement draft as >>>>> this >>>>> draft as tcpm consensus and was written for this purpose. >>>> My suggested wording uses the expression "can be deployed at large >>> scale" >>>> and I believe this is relevant. >>>> >>>> The document already describes in Section 5 how the protocol satisfies the >>>> agreed requirements for a more accurate ECN feedback protocol >>> [RFC7560]. >>>> So, if the TCPM working group publishes this document with the content of >>>> Section 5, I believe the TCPM working group already has reached >>> consensus >>>> that the protocol meets requirements. In addition, it is possible that new >>>> requirements would be identified in future, e.g., as an outcome of the >>>> experiment, and that would obviously have to be considered by TCPM. In >>>> that case, for the success of the experiment not only RFC 7560 would >>>> matter, but also further requirements. My proposed wording does not >>> have >>>> all these problems. >>>> >>>> In a nutshell, I continue to believe that this section has to change. >>>> >>>>>> * 1.5. Recap of Existing ECN feedback in IP/TCP >>>>>> >>>>>> [ms] This section could probably be shortened as well. >>>>>> >>>>>> The last bit in byte 13 of the TCP header was defined as the Nonce >>>>>> Sum (NS) for the ECN Nonce [RFC3540]. RFC 3540 was never deployed >>>>>> so >>>>>> it is being reclassified as historic, making this TCP flag available >>>>>> for use by the AccECN experiment instead. >>>>>> >>>>>> [ms] This wording, as well as Figure 1, needs to take into account the >>>>>> IANA >>>>> status when draft-ietf-tsvwg-ecn-experimentation is published. >>>>> >>>>> Is does. However, I can explicitly say that is has be re-clssified as >>>>> reserved. >>>>> >>>>> "RFC 3540 was never deployed so it is being reclassified as historic >>>>> [I-D.ietf- >>>>> tsvwg-ecn-experimentation] and the respective flag has been marked as >>>>> „reserved“ in the IANA TCP Header Flags registry, making this TCP flag >>>>> available for use by the AccECN experiment instead.“ >>>>> >>>>> Better? >>>>> >>>>>> In my understanding, this experimental document asks for new >>> assignment >>>>> of a reserved TCP header flag. >>>>> >>>>> As I said I’m not sure if we have fully concluded this discussion yet. >>>>> However, >>>>> what we really would want to is mention somewhere that this >>> experiment >>>>> with this flags is running. I guess there are three options: >>>>> 1) keep it in the registry as reserved and conserve the knowledge in tcpm >>>>> that this experiment is running and no other experimental RFC such use >>>>> this >>>>> flags as long as this experiment is running. >>>>> 2) Keep is marked as reserved but add a note about this experiment in >>> the >>>>> IANA registry >>>>> 3) Or assign it right away with IESG approval. I guess in this case tcpm >>>>> could >>>>> also consider to change the registration policy to „IETF Review“. >>>> The current registration policy for the TCP header flags is "standards >>>> action". I understand that the IESG could approve exceptions. But given >>>> the policy, I believe the document has to be very precise on the request >>>> regarding bit 7. >>>> >>>>>> * 2. AccECN Protocol Overview and Rationale >>>>>> >>>>>> o an essential part that re-uses ECN TCP header bits to feed back >>>>>> the number of arriving CE marked packets. This provides more >>>>>> accuracy than classic ECN feedback, but limited resilience >>>>>> against >>>>>> ACK loss; >>>>>> >>>>>> [ms] The word "re-use" is IMHO not correct. >>>>> I think this is nit picking. Using a different phrasing here makes the >>>>> sentence >>>>> unnecessary complicated. We don’t try to some how get a round the fact >>>>> that we need to handle the flag registration correctly. However, here the >>>>> point really is to explain how the protocol word. The main point of using >>>>> the >>>>> work „re-use“ here is really that we say that these flags are or have >>>>> been >>>>> used different by other TCP extension (and we therefore need a proper >>>>> negotiation scheme). >>>> If the allocation of a reserved flag is correctly explained in the >>>> abstract and introduction, I think these sentences can use a bit relaxed >>>> terminology. >>>> >>>>>> The two part design was necessary, given limitations on the space >>>>>> available for TCP options and given the possibility that certain >>>>>> incorrectly designed middleboxes prevent TCP using any new options. >>>>>> >>>>>> [ms] IMHO it would make sense to more explicitly mention the >>> downsides >>>>> of only specifying an option and not allocating a TCP header flag, in >>>>> this >>>>> experimental document. >>>>> >>>>> We need to use the flags (all three of them) for the negotiation. >>>> I think that an explanation why negotiation by a TCP option would not >>>> solve all use cases (or requirements) would help here. >>>> >>>>>> The obvious alternative would be to postpone the header flag >>>>>> allocation to >>>>> a follow-up standards track document and just keep it reserved. >>>>> >>>>> We can still do that. We should discuss and make a final decision. >>>>> >>>>>> The essential part overloads the previous definition of the three >>>>>> flags in the TCP header that had been assigned for use by ECN. This >>>>>> design choice deliberately replaces the classic ECN feedback >>>>>> protocol, rather than leaving classic ECN feedback intact and adding >>>>>> more accurate feedback separately because: >>>>>> >>>>>> [ms] Similar like previous comments, in my reading there are only _two_ >>>>> ECN header flags. >>>>> >>>>> I think there are three flags that "had been assigned for use by ECN“ as >>>>> ECN >>>>> Nonce is also an ECN mechanism. The fact that one of the flags is now >>>>> marked as reserved instead, it not that important for me here. >>>>> >>>>>> And, in addition, I think care is needed with wording such "replaces >>>>>> the >>>>> classic ECN feedback". I don't think this experiment replaces the ECN >>>>> standards. That would be up to a follow-up PS. >>>>> >>>>> This sentence is not meant to say that RFC3168 is replaced. Actually we >>>>> don’t. >>>>> You can still use RFC3168 even if AccECN is implemented and deploy >>>>> (however, we do intent that AccECN will be used as the default scheme in >>>>> future and RFC3168 is hopefully simply not needed anymore at some >>> point, >>>>> even though you probably still need to have it implemented as the >>>>> negotiation specified in this draft covers that as well, anyway...). The >>>>> sentence says that if AccECN is negotiation, the header flags as used by >>>>> RFC3168 and previously ECN Nonce are used differently (aka re-used). >>> That’s >>>>> all. >>>>>> >>>>>> 2.1. Capability Negotiation >>>>>> >>>>>> AccECN is a change to the wire protocol of the main TCP header, >>>>>> therefore it can only be used if both endpoints have been upgraded >>>>>> to >>>>>> understand it. The TCP client signals support for AccECN on the >>>>>> initial SYN of a connection and the TCP server signals whether it >>>>>> supports AccECN on the SYN/ACK. The TCP flags on the SYN that the >>>>>> client uses to signal AccECN support have been carefully chosen so >>>>>> that a TCP server will interpret them as a request to support the >>>>>> most recent variant of ECN feedback that it supports. Then the >>>>>> client falls back to the same variant of ECN feedback. >>>>>> >>>>>> [ms] As this is an experimental specification, I would really like to >>>>>> see a >>>>> discussion how a future standards track version of more accurate ECN >>>>> could >>>>> be negotiated. >>>>> >>>>> As described in this draft. There will be no different. AccECN IS and >>>>> will >>>>> always be backward compatible with RFC3168. >>>> That is not the problem I think about. I wonder about a PS version of >>>> accurate ECN feedback that would possibly include changes as compared to >>>> this experiment, e.g., because the experiment may have some lessons >>>> learnt. What options would we have to negotiate the PS version, and how >>>> could a stack implementing the PS version figure out whether the remote >>>> end uses the experimental or the PS version of the protocol? >>>> >>>> The reason why I ask is because I don't see any easy solution to this but >>>> I may miss something. Maybe this would not be a concern if there were >>> some >>>> codepoints left. >>>> >>>>>> How could both endpoints detect whether the other one implements >>> the >>>>> future standards track version? >>>>> >>>>> If the initiator implements AccECN it will request it’s use. If the >>>>> receiver also >>>>> implement it, it will/can negotiate it, if not it will look like an >>>>> RFC3168 request >>>>> for the receiver (as the NS flags will be ignored in the SYN) and it will >>>>> negotiate RFC3168 ECN feedback if implemented. There is no additional >>>>> detection needed. >>>> My concern is the migration strategy for a future version, given that we >>>> only experiment right now. For instance, if the initiator implements the >>>> experimental version but the recipient implements the PS version of >>>> AccECN, how would that work? Under the assumption that there are >>> changes, >>>> would there be a way to know? Maybe the answer to this question is no, >>>> given the small number of bits we have. But not having any room for >>>> extensions is not necessarily good protocol design. >>>> >>>>>> For instance, would the only safe variant be that we allocate yet >>>>>> another >>>>> reserved TCP header flag in a proposed standard to negotiate the >>>>> standards >>>>> track version, thus investing another reserved bit in the TCP header? >>>>> >>>>> No, that’s exactly what we use the NS flags for in the handshake. >>>> If the PS version of AccECN was different to the experimental version of >>>> AccECN, and if there was a deployed base of both, I still believe we could >>>> end up in a situation in which we had to allocate yet another TCP header >>>> flag to distinguish the different versions of AccECN. The NS flag would >>>> not work for that if it is used by the experimental version. The risk of >>>> having to spend further TCP header flags somehow concerns me. Of >>> course, >>>> that problem would only matter if the AccECN experiment succeeds >>> somehow, >>>> but lessons learnt would require protocol changes, which is speculation. >>>> >>>>>> I may be wrong, but to me it is too early to speculate how the PS >>>>>> version >>>>> would look like, and whether it would have to be different to the >>>>> experimental version, due to lessons learnt. >>>>> >>>>> Of course you can always be wrong. However, the handshake negotiation >>> is >>>>> not the part we need experimentation for. That part is straight forward >>>>> and >>>>> works. If we really happen to detect a problem in that part, we would >>>>> need >>>>> to end the experiment declare failure and start over new. >>>> My concern is ending the experiment when the experiment got (partly) >>>> deployed in the Internet. In that case neither a new RFC nor a change of >>>> the IANA registry will solve the migration issue. >>>> >>>>>> I believe in the IETF we typically design protocols that allow future >>>>> extension, and it is not exactly clear to be how AccECN could be extended >>>>> later. >>>>> >>>>> This is an TCP extension. If we want future extension we use the usually >>>>> TCP >>>>> mechanism (by defining a new TCP option I guess). >>>> The root cause of my concern is that this proposal does *not* use the >>>> usual way to experiment with TCP by options. It experiments with a header >>>> flag, including in the SYN, and it seems to consume all codepoints. So, I >>>> see the risk of a protocol design "not ready for future improvements". >>>> >>>> I cannot easily propose text. Maybe "lack of extensibility" is just one of >>>> the short-comings of the protocol design that cannot be avoided but that >>>> short-coming would have to be noted. >>>> >>>>>> An AccECN TCP client does not send the new AccECN Option on the >>> SYN >>>>>> as SYN option space is limited and successful negotiation using the >>>>>> flags in the main header is taken as sufficient evidence that both >>>>>> ends also support the AccECN Option. The TCP server sends the >>>>>> AccECN >>>>>> Option on the SYN/ACK and the client sends it on the first ACK to >>>>>> test whether the network path forwards the option correctly. >>>>>> >>>>>> [ms] For what it is worth, I would personally be quite fine with >>>>>> allowing (or >>>>> even mandating) an option in the SYN in the experimental version of this >>>>> protocol. For instance, saving the SYN option space would then an >>>>> excellent >>>>> reason for moving towards the PS specification. I am also fine with being >>>>> in >>>>> the rough part of the consensus here. >>>>> >>>>> The point is that we really don’t need the option in the SYN as we don’t >>>>> use it >>>>> for negotiation purposes as we use the header bits instead. So why >>> should >>>>> we waste the space? >>>> For instance, mandating the option in the SYN would be away for the >>>> receiver to distinguish the experiment from a follow-up PS version of the >>>> spec, as the PS version may not mandate the option, to save header space. >>>> >>>> Maybe that proposal does not make any sense, and it may only have >>>> downsides. But the document already speculates about a PS-follow-up. So >>> it >>>> seems a valid question to ask if the EXP and the PS version of the spec >>>> have to be identical. This all comes down to the SYN negotiation. >>>> >>>>>> * 2.3. Delayed ACKs and Resilience Against ACK Loss >>>>>> >>>>>> If the AccECN Option is not available, e.g. it is being stripped by >>>>>> a >>>>>> middlebox, the AccECN protocol will only feed back information on CE >>>>>> markings (using the ACE field). Although not ideal, this will be >>>>>> sufficient, because it is envisaged that neither ECT(0) nor ECT(1) >>>>>> will ever indicate more severe congestion than CE, even though >>>>>> future >>>>>> uses for ECT(0) or ECT(1) are still unclear >>>>>> [I-D.ietf-tsvwg-ecn-experimentation]. >>>>>> >>>>>> [ms] This needs to be reworded >>>>> Why? >>>>> >>>>>> >>>>>> >>>>>> * 2.4. Feedback Metrics >>>>>> >>>>>> The CE packet counter in the ACE field and the CE byte counter in >>>>>> the >>>>>> AccECN Option both provide feedback on received CE-marks. The CE >>>>>> packet counter includes control packets that do not have payload >>>>>> data, while the CE byte counter solely includes marked payload >>>>>> bytes. >>>>>> If both are present, the byte counter in the option will provide the >>>>>> more accurate information needed for modern congestion control and >>>>>> policing schemes, such as DCTCP or ConEx. >>>>>> >>>>>> [ms] I suggest to write in the last sentence only "... the option will >>>>>> provide >>>>> the more accurate information needed for congestion control". In >>> general, >>>>> I >>>>> would prefer to have references to other mechanisms at only few (ideally >>>>> a >>>>> *single*) places in the document, instead of mixing them together. >>>>> >>>>> Sorry, I don’t see your point here. ConEx has been mentioned previously, >>>>> so >>>>> why not also mention it here. >>>> As written earlier, in my understanding DCTCP does not "need" this. I >>>> would suggest to have one place where to define exactly how this >>> mechanism >>>> can be used by other TCP extensions. Having said this, in this specific >>>> paragraph I am less concerned about that than elsewhere. >>>> >>>>>> Feedback in bytes is recommended in order to protect against the >>>>>> receiver using attacks similar to 'ACK-Division' to artificially >>>>>> inflate the congestion window, which is why [RFC5681] now >>> recommends >>>>>> that TCP counts acknowledged bytes not packets. >>>>>> >>>>>> [ms] At least the last part and the reference to RFC 5681 is IMHO not >>>>> needed here. >>>>> >>>>> Why? RFC5681 explains/refers the ACK division attack, so I think it is a >>>>> very >>>>> good reference to have here. >>>>>> >>>>>> >>>>>> * 2.5. Generic (Dumb) Reflector >>>>>> >>>>>> The ACE field provides information about CE markings on both data >>>>>> and >>>>>> control packets. According to [RFC3168] the Data Sender is meant to >>>>>> set control packets to Not-ECT. However, mechanisms in certain >>>>>> private networks (e.g. data centres) set control packets to be ECN >>>>>> capable because they are precisely the packets that performance >>>>>> depends on most. >>>>>> >>>>>> For this reason, AccECN is designed to be a generic reflector of >>>>>> whatever ECN markings it sees, whether or not they are compliant >>>>>> with >>>>>> a current standard. Then as standards evolve, Data Senders can >>>>>> upgrade unilaterally without any need for receivers to upgrade too. >>>>>> It is also useful to be able to rely on generic reflection behaviour >>>>>> when senders need to test for unexpected interference with markings >>>>>> (for instance [I-D.kuehlewind-tcpm-ecn-fallback] and >>>>>> [I-D.moncaster-tcpm-rcv-cheat]). >>>>>> >>>>>> The initial SYN is the most critical control packet, so AccECN >>>>>> provides feedback on whether it is CE marked. Although RFC 3168 >>>>>> prohibits an ECN-capable SYN, providing feedback of CE marking on >>>>>> the >>>>>> SYN supports future scenarios in which SYNs might be ECN-enabled >>>>>> (without prejudging whether they ought to be). For instance, >>>>>> [I-D.ietf-tsvwg-ecn-experimentation] updates this aspect of RFC 3168 >>>>>> to allow experimentation with ECN-capable TCP control packets. >>>>>> >>>>>> [ms] To me, the only thing that matters in this document that AccECN >>>>>> can >>>>> provide feedback on whether the SYN is CE marked. The discussion on >>> how >>>>> to experiment with ECT e.g. in SYNs IMHO does not belong into this >>>>> document. So it seems sufficient here to note that one of the benefits of >>>>> AccECN is that CE marks in SYNs can be fed back. >>>>> >>>>> I disagree. Explicitly saying that AccECN is only an feedback scheme and >>>>> DOES >>>>> NOT define how the information is used is VERY important because >>> people >>>>> come back to me over and over again and mix these things up. >>>>>> >>>>>> * 3.1.1. Negotiation during the TCP handshake >>>>>> >>>>>> Given the ECN Nonce [RFC3540] is being reclassified as historic, the >>>>>> present specification renames the TCP flag at bit 7 of the TCP >>>>>> header >>>>>> flags from NS (Nonce Sum) to AE (Accurate ECN) (see IANA >>>>>> Considerations in Section 6). >>>>>> >>>>>> [ms] As mentioned before, this needs to be rewritten to ask for new >>>>>> IANA >>>>> allocation of bit 7 in the TCP header flags. >>>>> >>>>> I really don’t understand this comment. That is what the IANA section >>>>> does >>>>> as referred here correctly. >>>> In my reading of >>>> https://www.iana.org/assignments/tcp-header-flags/tcp-header- >>> flags.xhtml, >>>> this flag currently has no name, i.e., it does not have the name NS. So >>>> this statement on "renaming" is formally incorrect IMHO. >>>> >>>> I'd suggest something like: "This specification assigns the name AE >>>> (Accurate ECN) to the TCP flag at bit 7; this flag has previously been >>>> known as NS (Nonce Sum) ...". >>>> >>>>> Again, yes, we can discuss if this document should do this, but if >>>>> published as >>>>> it is, section 6 says everything that is needs to say. (I does not put >>>>> any >>>>> request on IANA, instead it is written as it would show up >>>>> post-publication, >>>>> implicitly providing a request to IANA at approval time. That is fine.) >>>>> >>>>>> Table 2: ECN capability negotiation between Client (A) and Server >>>>>> (B) >>>>>> >>>>>> [ms] As far as I can see, in -05 this table allocates all existing >>>>>> codepoints, >>>>> while -03 had two currently unused codepoints. Not having any >>> codepoints >>>>> left seems to me not really future proof, e.g., regarding future proposed >>>>> standards in this space (and I personally believe that TCP header flags >>>>> must >>>>> be allocated in a PS). And I don't fully see a need of feeding back ECT0 >>>>> and >>>>> specifically ECT1 in the TCP header flags as part of the experiment. Do >>>>> we >>>>> know for sure that this is the only possible use case of these two >>>>> unallocated >>>>> header bits? And why can't e.g. this be done in a TCP option instead? Or >>>>> do I >>>>> miss something? >>>>> >>>>> The point is really, if we don’t assign them now and start deployment we >>>>> effetely we not be able to every assign them again because don’t have a >>>>> different negotiation mechanisms. Realizing this, it is just the right >>>>> think to >>>>> define the space completely that is negotiated as use for AccECN in the >>>>> handshake. >>>> I disagree that consuming all codepoints and not having room for future >>>> extensions is the "right thing" to do. It is in fact the wrong thing. But >>>> possibly there is no alternative with the few bits, so maybe we end up >>>> doing it. >>>> >>>> Yet, at minimum, using all codepoints needs to be reasoned in the >>>> document. Specifically since in -03 a different protocol design seemed >>>> possible, which looked to me more future-proof. >>>> >>>>>> * 3.1.2. Retransmission of the SYN >>>>>> >>>>>> However, >>>>>> current measurements imply that a drop is less likely to be due to >>>>>> middlebox interference than other intermittent causes of loss, e.g. >>>>>> congestion, wireless interference, etc. >>>>>> >>>>>> [ms] Such wording IMHO doesn't belong into normative text. This may >>>>> actually also apply to other heuristics discussed in this section, which >>>>> are not >>>>> really important for interoperability. >>>>> >>>>> I don’t really understand your point. This sentence is solely meant to >>>>> reason >>>>> the design decision to not say that a sender SHOULD attempt to re- >>>>> negotiation after a loss. >>>> One could split the reasoning from the normative design. The normative >>>> specification may still be relevant in 20 years from now. Middleboxes will >>>> almost likely be different in 20 years. Having said this, this is more of >>>> an editorial remark. >>>> >>>>>> 3.2.7. Path Traversal of the AccECN Option >>>>>> >>>>>> 3.2.7.1. Testing the AccECN Option during the Handshake >>>>>> >>>>>> The TCP client MUST NOT include the AccECN TCP Option on the SYN. >>>>>> >>>>>> [ms] I am not sure if I really understand the motivation for not >>>>>> allowing a >>>>> option in the SYN. If the sender has space in the SYN left, what is the >>>>> harm in >>>>> an experimental version of the protocol? And I may miss something, but >>>>> what would prevent the use of 2-byte option to negotiate the use of >>>>> AccECN, e.g., to avoid experimental allocation of bit 7 in the initial >>>>> SYN? >>>>> >>>>> I did have a draft on that as a proposal for an alternative design. >>>>> However, >>>>> the gourd was more supportive of this design as it is proof of middlebox >>>>> SYN >>>>> option mangling which is a know problem. >>>>> >>>>> Therefore we simply don’t need option in the SYN and there is no reason >>>>> to >>>>> waste the space. >>>>> >>>>>> While I think many tutorial text in this document could be shortened, I >>>>> believe the use of a reserved TCP header flag should be reasoned. >>>>> >>>>> I’m actually uncertain what you expect here. >>>> For instance, this reply with the explanation of SYN option mangling seems >>>> useful explanation. >>>> >>>>>> * 3.2.8. Usage of the AccECN TCP Option >>>>>> >>>>>> The following rules determine when a Data Receiver in AccECN mode >>>>>> sends the AccECN TCP Option, and which fields to include: >>>>>> >>>>>> Change-Triggered ACKs: If an arriving packet increments a different >>>>>> byte counter to that incremented by the previous packet, the Data >>>>>> Receiver MUST immediately send an ACK with an AccECN Option, >>>>>> without waiting for the next delayed ACK (this is in addition to >>>>>> the safety recommendation in Section 3.2.5 against ambiguity of >>>>>> the ACE field). >>>>>> >>>>>> This is stated as a "MUST" so that the data sender can rely on >>>>>> change-triggered ACKs to detect transitions right from the very >>>>>> start of a flow, without first having to detect whether the >>>>>> receiver complies. A concern has been raised that certain >>>>>> offload >>>>>> hardware needed for high performance might not be able to support >>>>>> change-triggered ACKs, although high performance protocols such >>>>>> as >>>>>> DCTCP successfully use change-triggered ACKs. >>>>>> >>>>>> [ms] To me this sounds like a perfect example for a SHOULD with >>>>>> additional >>>>> guidance why implementing this SHOULD is really important. >>>>> >>>>> This is one of the most discussed point from the author and we really >>>>> tried to >>>>> get additional guidance here of what to do also from outside the IETF but >>>>> did >>>>> no clear feedback. >>>>> >>>>> As explained this MUST enables additional functionality. However, this is >>>>> an >>>>> experiment document. If we detect that this MUST does actually hinder >>>>> implementation or has just never been implemented, we should >>> reconsider >>>>> this in the final PS RFC. >>>>> >>>>> I was on the SHOULD side of the discussion, but can say that the >>>>> implementation in Linux was way more simple then expected. Offloading >>>>> might be a different topic but that is where I could not get a clear >>>>> feedback >>>>> and offloading could probably be anyway optimized for use with AccECN >>> (if >>>>> it >>>>> gets deploy widely). >>>> If a MUST requires changes in hardware, I think there must be a clear >>>> reason. >>>> >>>> As individual contributor, with the current explanation in the text, I >>>> believe this has to be a SHOULD. >>>> >>>>> I though we added this as something to mention in the exp goals section, >>>>> but >>>>> obviously we didn’t. I added the following text now: >>>>> >>>>> "Another experimentation focus is the implementation feasibiliy of >>>>> change- >>>>> triggered ACKs as described in section 3.2.8. While on average this >>>>> should not >>>>> lead to a higher ACK rate, it changes the ACK patter which especially can >>>>> have >>>>> an impact on hardware offload. Further experimentation is needed to >>>>> advise >>>>> if this should a hard requirement or just prefer behavior.“ >>>> If it is unclear if a MUST can actually be implemented, having a MUST is >>>> in my opinion the wrong approach. >>>> >>>> One could equally state here that further experimentation is needed to >>>> determine whether the SHOULD can be upgraded to a MUST. >>>> >>>>>> For the avoidance of doubt, the change-triggered ACK mechanism is >>>>>> deliberately worded to ignore the arrival of a control packet with >>>>>> no >>>>>> payload, which therefore does not alter any byte counters, because >>>>>> it >>>>>> is important that TCP does not acknowledge pure ACKs. The change- >>>>>> triggered ACK approach will lead to some additional ACKs but it >>>>>> feeds >>>>>> back the timing and the order in which ECN marks are received with >>>>>> minimal additional complexity. >>>>>> >>>>>> [ms] The additional acks create network load. I think some wording is >>>>> needed on the tradeoff between information accuracy and network load. >>>>> There are network environments in which any additional packet is very >>>>> expensive (e.g., energy) and it is not clear to me how the protocol >>>>> design >>>>> takes into account the potential overhead of additional ACKs. Maybe this >>>>> could be another reason for a SHOULD. >>>>> >>>>> The above. However, this is not really an additional ACK because you do >>>>> delay the next one. Further experimentation needed. >>>> The document states "lead to some additional ACKs". If that does not >>>> increase network load, I think it has to be explicitly explained why the >>>> ACK load is at most equal to a current TCP stack, in all potential cases. >>>> If it can increases network load, it has to be reasoned why increasing >>>> load (and risk of reverse congestion and the like) is worth the effort. >>>> >>>> I agree that this may be an area of experimentation, but I believe then it >>>> has to be explained to implementers what the tradeoffs are. >>>> >>>>>> * 4.2. Compatibility with Other TCP Options and Experiments >>>>>> >>>>>> AccECN is compatible (at least on paper) with the most commonly >>> used >>>>>> TCP options: MSS, time-stamp, window scaling, SACK and TCP-AO. It >>>>>> is >>>>>> also compatible with the recent promising experimental TCP options >>>>>> TCP Fast Open (TFO [RFC7413]) and Multipath TCP (MPTCP [RFC6824]). >>>>>> >>>>>> [ms] I would suggest the wording "... compatible with the experimental >>>>> TCP options ..." or even "... compatible with the TCP options …". >>>>> >>>>> These option are to experimental..? >>>> "It is also compatible with the experimental TCP options TCP Fast Open >>>> (TFO [RFC7413]) and Multipath TCP (MPTCP [RFC6824])." would have same >>>> technical meaning. Having said this, this is editorial only. >>>> >>>>> The point of using „commonly used“ was to say that we checked on those >>> as >>>>> they seem important. Just because your favorite experiment option is not >>>>> listed here it doesn’t means it incompatible, we just didn’t check. I’m >>>>> okay to >>>>> removed „commonly used“ but I don’t think it makes anything better. >>>>> >>>>>> * 4.3. Compatibility with Feedback Integrity Mechanisms >>>>>> >>>>>> [ms] Quite a bit in this section is experimental work, which IMHO >>>>>> should be clearly emphasized. The one exception is… >>>>> I would really like to keep this section because integrity is usually the >>>>> fist >>>>> question that come up when I present AccECN. Effectively these are two >>>>> independent topics, however, I really think it help people to understand >>>>> the >>>>> whole picture if this is also discussed in this document. >>>>> >>>>>> However, TCP-AO is often too brittle to use on many end-to-end >>>>>> paths, where middleboxes can make verification fail in their >>>>>> attempts to improve performance or security, e.g. by >>>>>> resegmentation or shifting the sequence space. >>>>>> >>>>>> [ms] I am not sure if deployment challenges of other options need to be >>>>> discussed in this document. >>>>> >>>>> If we keep the discussion, I guess we should mention this as well. As the >>>>> doc >>>>> clearly stated section 4 is not meant to be normative. >>>> As far as I can see, there are use cases for TCP-AO where middleboxes are >>>> simply not a problem, but it is exactly this sort of discussion that may >>>> not be needed in this document. But I won't rat-hole on this comment >>> here. >>>>>> Originally the ECN Nonce [RFC3540] was proposed to ensure integrity >>>>>> of congestion feedback. With minor changes AccECN could be >>>>>> optimised >>>>>> for the possibility that the ECT(1) codepoint might be used as an >>>>>> ECN >>>>>> Nonce . However, given RFC 3540 is being reclassified as historic, >>>>>> the AccECN design has been generalised so that it ought to be able >>>>>> to >>>>>> support other possible uses of the ECT(1) codepoint, such as a lower >>>>>> severity or a more instant congestion signal than CE. >>>>>> >>>>>> [ms] The discussion of RFC 3540 can probably be removed to a large >>>>>> extent. >>>>> Unfortunately, please still think that ECN Nonce, even though is was >>>>> never >>>>> deployed and doesn’t really work, is the only was to provide integrity >>>>> protection and we need it as a prerequisite to deploy ECN at all… i would >>>>> really prefer to keep this in this non-normative part of the doc. >>>>> >>>>>> >>>>>> * 6. IANA Considerations >>>>>> >>>>>> [ms] I think this section needs to be rewritten to request a new >>>>>> allocation >>>>> of bit 7 of the TCP header flags. At least for the process I think it >>>>> would make >>>>> sense to have somewhere in the document a comprehensive explanation >>> of >>>>> why an experimental document requests a change of the main TCP >>> header, >>>>> and why this cannot be avoided (most notably in the initial SYN) by an >>>>> alternative protocol design. >>>>> >>>>> As I said previously this section is written as it would look like after >>>>> the >>>>> allocation has happened with publication approval of the IESG. This is >>>>> fine. >>>>> >>>>> Having a discussion about an experiment doc assigning a flag (or not) is >>>>> a >>>>> question for tcpm as a whole and not specifically this document. How do >>>>> we >>>>> envision to every use any further flags? We go to PS right away? Or >>>>> should >>>>> we change the registration policy? For me the latter makes actually more >>>>> sense. However, if we don’t want/can to decide this now, we also could >>> go >>>>> forward as it is with IESG approval. However, is this case it is also not >>>>> needed >>>>> to explain this in the document. The responsible AD has to explain this >>>>> to the >>>>> other IESG probably in the ballot or even better the shepherd could >>>>> provide >>>>> these information in the write-up. >>>>> >>>>>> >>>>>> * 9. Comments Solicited >>>>>> >>>>>> Comments and questions are encouraged and very welcome. They >>> can >>>>> be >>>>>> addressed to the IETF TCP maintenance and minor modifications >>>>>> working >>>>>> group mailing list, and/or to the authors. >>>>>> >>>>>> [ms] This section is not needed IMHO >>>>> Yes, it will be removed before publication. >>>>>> >>>>>> 10. References >>>>>> >>>>>> [I-D.ietf-tsvwg-ecn-experimentation] >>>>>> Black, D., "Relaxing Restrictions on Explicit Congestion >>>>>> Notification (ECN) Experimentation", >>>>>> draft-ietf-tsvwg-ecn- >>>>>> experimentation-07 (work in progress), October 2017. >>>>>> >>>>>> [ms] Normative reference? >>>>> Don’t see why. No need to read ietf-tsvwg-ecn-experimentation to >>>>> understand the spec in this doc. >>>>> >>>>> Thanks! >>>>> Mirja >>>>> >>>> _______________________________________________ >>>> tcpm mailing list >>>> tcpm@ietf.org >>>> https://www.ietf.org/mailman/listinfo/tcpm >>>> >>>> >>>> --- >>>> This email has been checked for viruses by AVG. >>>> http://www.avg.com >>>> >> _______________________________________________ >> tcpm mailing list >> tcpm@ietf.org >> https://www.ietf.org/mailman/listinfo/tcpm > > -- > ________________________________________________________________ > Bob Briscoehttp://bobbriscoe.net/ > From nobody Fri Apr 6 22:10:27 2018 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0D98F12D86D for ; Fri, 6 Apr 2018 22:10:22 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.31 X-Spam-Level: X-Spam-Status: No, score=-4.31 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=swm.pp.se Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XWw8VyeJ3dXN for ; Fri, 6 Apr 2018 22:10:19 -0700 (PDT) Received: from uplift.swm.pp.se (swm.pp.se [212.247.200.143]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 9719F129C6A for ; Fri, 6 Apr 2018 22:10:19 -0700 (PDT) Received: by uplift.swm.pp.se (Postfix, from userid 501) id 2B80AB1; Sat, 7 Apr 2018 07:10:17 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=swm.pp.se; s=mail; t=1523077817; bh=4r8PuoDf0HeMWtj92Zt8XgTcyyPmfBWfUBdQnbe+hco=; h=Date:From:To:cc:Subject:In-Reply-To:References:From; b=U2Xny47zPxHoQMe5HQqBEXDvS0QNAwK9omMBAZNS0RaWveDCD1f99ZgHyZpUwvVUk cywmqfFbKEh3hsNi4hrVZhFL6sR75hDtHF3lr2tCWef0HiQ+W3rYFLkDBJ28paTepT jQKtNk3NaOifBicd+BYatsOdJlF9vlenWdOi0S6Q= Received: from localhost (localhost [127.0.0.1]) by uplift.swm.pp.se (Postfix) with ESMTP id 26401B0; Sat, 7 Apr 2018 07:10:17 +0200 (CEST) Date: Sat, 7 Apr 2018 07:10:17 +0200 (CEST) From: Mikael Abrahamsson To: Toerless Eckert cc: tcpm@ietf.org In-Reply-To: <20180406012736.fpayup7perbhqn2p@faui48f.informatik.uni-erlangen.de> Message-ID: References: <20180406012736.fpayup7perbhqn2p@faui48f.informatik.uni-erlangen.de> User-Agent: Alpine 2.20 (DEB 67 2015-01-07) Organization: People's Front Against WWW MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Archived-At: Subject: Re: [tcpm] Better QoS for TCP ACK question X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Apr 2018 05:10:24 -0000 On Fri, 6 Apr 2018, Toerless Eckert wrote: > A couple of years ago i was told that a range of home routers > would do some DPI based diffserv QoS, aka prioritizing TCP ACP > messages in the queue before any other packets 9sonded like > a strict priority queue for these packets). The goal was to > improve performance when you had separate up and downstream TCP > connections, especially when the upstream TCP connection delayed > the downstream TCP connections ACKs. > > I never could find a lot of good details about this, e.g.: quantitiative > measurements, so i was wondering if others here on the list > are aware of it. Of course, if something like this would > be significantly useful even in the face of current TCP congestion > control mechanisms (and not only bad older ones). I know some people who have run two queues, one for packets less than 200 byte in size, and one for larger packets. They didn't even do any DPI, the setup was purely based on size of the packet. I am not aware of any deeper analysis on pros/cons of this setup, but these people were very happy with their setups for years and it solved their problems with a single large file transfer ruining their interactive performance, plus also "solved" the ACKs-get-stuck-behind-other-session-large-packets problem. This was used for platforms with otherwise quite limited QoS capabilities, so flow based queueing wasn't available. -- Mikael Abrahamsson email: swmike@swm.pp.se From nobody Sat Apr 7 09:52:29 2018 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4637C126C3D; Fri, 6 Apr 2018 16:16:42 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2 X-Spam-Level: X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=bobbriscoe.net Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id SlrCiP_dBXsg; Fri, 6 Apr 2018 16:16:33 -0700 (PDT) Received: from server.dnsblock1.com (server.dnsblock1.com [85.13.236.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 18B6A126BF0; Fri, 6 Apr 2018 16:16:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=bobbriscoe.net; s=default; h=Content-Type:In-Reply-To:MIME-Version:Date: Message-ID:From:References:Cc:To:Subject:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=K6smj3d5LBUkW8dtBygTqxdF+03/X5LN0J0t1Tbuwe8=; b=NojDJusKae0K9AR9pBMeNeUC7 AVF4JohmaYT6fj+UQfgCcWR2BQqO5TttM0lyr4GY6ik6wtQpxqjhd85Q+ZzZ6NvXhZIiNzeo4R6IF DsZ8ucBPHaoX1cGbGT9f4ixtSdKVRqw9ioewQ2St0+roVuDPzlbT4jhHih/7wnTSPeL1y1Sa34SYO qE53yJeF5c5y3cAkj7KlSh+adiPQigNaJK14+CMokP3i7u4vuPsaNBP/FZ/tIkLOxOu1An8lOH7ho ZL2kxFXF9iExn3fTzNDlIaLjYdPREN41pvWaBWdwanSe4yOyVljJqDr8cx+Uvr53+RgNQpW9Ebp1v iCPtFSwDw==; Received: from host-79-78-166-168.static.as9105.net ([79.78.166.168]:47664 helo=[192.168.2.6]) by server.dnsblock1.com with esmtpsa (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim 4.89_1) (envelope-from ) id 1f4aao-0003Ut-AG; Sat, 07 Apr 2018 00:16:28 +0100 To: "Scharf, Michael (Nokia - DE/Stuttgart)" Cc: Richard Scheffenegger , =?UTF-8?Q?Mirja_K=c3=bchlewind?= , "draft-ietf-tcpm-accurate-ecn@ietf.org" , "tcpm@ietf.org" References: <44FDECF5-A031-4343-BA1A-AE0D9C2C078C@tik.ee.ethz.ch> From: Bob Briscoe Message-ID: <7869808f-4e15-1180-da89-a0f2b043b46c@bobbriscoe.net> Date: Sat, 7 Apr 2018 00:16:25 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/alternative; boundary="------------29C23CFC30CF5D948F19EE0A" Content-Language: en-GB X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - server.dnsblock1.com X-AntiAbuse: Original Domain - ietf.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - bobbriscoe.net X-Get-Message-Sender-Via: server.dnsblock1.com: authenticated_id: in@bobbriscoe.net X-Authenticated-Sender: server.dnsblock1.com: in@bobbriscoe.net Archived-At: X-Mailman-Approved-At: Sat, 07 Apr 2018 09:52:27 -0700 Subject: Re: [tcpm] Comments on draft-ietf-tcpm-accurate-ecn X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Apr 2018 23:16:42 -0000 This is a multi-part message in MIME format. --------------29C23CFC30CF5D948F19EE0A Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Michael, I realize now that I misunderstood your question during the TCPM session in London. (Sorry for the delay - I tuned out for a while). As well as the 2 codepoints that Richard has highlighted, the draft is careful to require that implementations of the AccECN experiment will silently accept future use of other unexpected codepoints ('forward compatibility'). For instance, at the end of S.3.2,3 : Note that the Data Sender MUST NOT test whether the arriving counter in the initial ACE field has been initialized to a specific valid value - the above check solely tests whether the ACE fields have been incorrectly zeroed. This allows hosts to use different initial values as an additional signalling channel in future. We have only used codepoints 6&7 here. Assuming we avoid zero, because it might be due to bleaching, this leaves values 1-5 unused. The analysis I did of the remaining combinations of codepoints when taken across the whole handshake is here:     http://bobbriscoe.net/projects/latency/ecn-modes.pdf Page 1 covers the TCP ECN flags. The 1st row marked A-CU (short for Accurate ECN Currently Unassigned) shows the 5 values that the above text keeps for future use. The 2nd two rows marked A-CU are now used by AccECN (since draft-04) to deal with the bleaching we discovered during testing. All the combinations tagged with 'N' (for nonce) are likely to be the most useful if needed for any additional negotiation space. The other one Richard tagged (labelled 'broken') is still polluted by a non-negligible number of bugged servers that reflect the 6 most significant TCP flags in the SYN back in the SYN/ACK. From a 2014 measurement study that Richard and Mirja co-authored, there were still about 0.3% of the Alexa 1M exhibiting the symptoms of this bug. I'm in the process of checking our more recent data, where we specifically checked if 111 used on the AccECN SYN was reflected. Taken as a whole, I admit this doesn't leave much flexibility for future variation, but it's the best we could do with the available space constraints. Does this sufficiently address your concern? Cheers Bob PS. Similarly, although we have not allowed any space in the AccECN TCP Option for flag bits, we have allowed for different initial values of the counters to be used to signal new versions of the protocol in future. See the end of S.3.2.7.4 : Note that the Data Sender MUST NOT test whether the arriving byte counters in the initial AccECN Option have been initialized to specific valid values - the above checks solely test whether these fields have been incorrectly zeroed. This allows hosts to use different initial values as an additional signalling channel in future. On 19/03/18 16:04, Scharf, Michael (Nokia - DE/Stuttgart) wrote: > For what it is worth, this is exactly the sort of discussion I believe we need to have. > > I would be much more comfortable if the document did foresee some way negotiate an "enhanced" version of Accurate ECN in a later phase - most notably a PS version of the protocol. If that would cost some functionality right now, as we may have to reserve some codepoints in the EXP version, that would be perfectly fine for me, as long as we could add that functionality later in a PS version (under our current understanding). > > I have not thought about the details. But we have only three reserved TCP header bits left after publication of this document and I am worried about that. > > Michael > > >> -----Original Message----- >> From: Richard Scheffenegger [mailto:rs.ietf@gmx.at] >> Sent: Monday, March 19, 2018 4:27 PM >> To: Scharf, Michael (Nokia - DE/Stuttgart) ; >> Mirja Kühlewind >> Cc: draft-ietf-tcpm-accurate-ecn@ietf.org; tcpm@ietf.org >> Subject: Re: [tcpm] Comments on draft-ietf-tcpm-accurate-ecn >> >> Hi Michael, >> >> regarding your comment about the lack of future extensibility of the ECN / >> AccECN header handshake without further bits... >> >> I wanted to note, that the logic table we have in the accecn draft currently >> has two entries, that are reserved because of observed, but legacy, stacks >> misbehaving when reflecting these TCP header bits. >> >> So there is a possibility to eventually use those entries - once the broken >> legacy stacks have been confirmeed to have left the internet - as some >> additional negotiation capability. >> >> Also, the feedback of the byte counters vs. CE packet counter are split into >> using a tcp option and the tcp header bits. >> An extention of AccECN could take the form of using a differen TCP option >> together with the header counter negotiation... >> >> Earlier work by us determined that packing more information into the tcp >> header bits alone (as an example) would likely need an additional bit (that >> can also be used in a negotiation scheme)... >> >> >> +--------+--------+------------+-------------+----------------------+ >> | A | B | SYN A->B | SYN/ACK | Feedback Mode | >> | | | | B->A | | >> +--------+--------+------------+-------------+----------------------+ >> | | | AE CWR ECE | AE CWR ECE | | >> | AccECN | AccECN | 1 1 1 | 0 1 0 | AccECN (Not-ECT on | >> | | | | | SYN) | >> | AccECN | AccECN | 1 1 1 | 0 1 1 | AccECN (ECT1 on SYN) | >> | AccECN | AccECN | 1 1 1 | 1 0 0 | AccECN (ECT0 on SYN) | >> | AccECN | AccECN | 1 1 1 | 1 1 0 | AccECN (CE on SYN) | >> | | | | | | >> # | AccECN | Nonce | 1 1 1 | 1 0 1 | classic ECN | >> | AccECN | ECN | 1 1 1 | 0 0 1 | classic ECN | >> | AccECN | No ECN | 1 1 1 | 0 0 0 | Not ECN | >> | | | | | | >> | Nonce | AccECN | 0 1 1 | 0 0 1 | classic ECN | >> | ECN | AccECN | 0 1 1 | 0 0 1 | classic ECN | >> | No ECN | AccECN | 0 0 0 | 0 0 0 | Not ECN | >> | | | | | | >> # | AccECN | Broken | 1 1 1 | 1 1 1 | Not ECN | >> +--------+--------+------------+-------------+----------------------+ >> Best regards, >> Richard >> >> >> ----- Original Message ----- >> From: "Scharf, Michael (Nokia - DE/Stuttgart)" >> To: "Mirja Kühlewind" >> Cc: ; >> Sent: Monday, March 12, 2018 1:59 AM >> Subject: Re: [tcpm] Comments on draft-ietf-tcpm-accurate-ecn >> >> >>> Hi Mirja, >>> >>> Thanks a lot for the explanation. I won't follow-up on some of the >>> editorial suggestions. >>> >>> Yet, I continue to believe that some formal wording in the document needs >>> to change, as explained below. >>> >>> Thanks >>> >>> Michael (with no hat on) >>> >>> >>>> -----Original Message----- >>>> From: Mirja Kühlewind [mailto:mirja.kuehlewind@tik.ee.ethz.ch] >>>> Sent: Monday, March 05, 2018 1:54 PM >>>> To: Scharf, Michael (Nokia - DE/Stuttgart) >>>> Cc: draft-ietf-tcpm-accurate-ecn@ietf.org; tcpm@ietf.org >>>> Subject: Re: Comments on draft-ietf-tcpm-accurate-ecn >>>> >>>> Hi Micheal, >>>> >>>> thanks for your feedback and sorry for my late reply. >>>> >>>> Please see inline. >>>> >>>>> Am 03.12.2017 um 20:17 schrieb Scharf, Michael (Nokia - DE/Stuttgart) >>>> : >>>>> Hi all, >>>>> >>>>> I have read draft-ietf-tcpm-accurate-ecn-05 (without the appendix). I >>>> believe this document needs further work before moving forward. >>>>> Please find below my comments marked as [ms]. I have read the >>>> document independent of the review from Gorry. I apologize if there is >>>> duplication. >>>>> Thanks >>>>> >>>>> Michael (with no hat on) >>>>> >>>>> >>>>> ****************************** >>>>> >>>>> * Abstract: >>>>> >>>>> Recently, new TCP mechanisms like Congestion Exposure (ConEx) or >>>>> Data >>>> Center TCP >>>>> (DCTCP) need more accurate ECN feedback information whenever >> more >>>>> than one marking is received in one RTT. >>>>> >>>>> [ms] I don't think this statement is fully backed by RFC 8257. I >>>>> suggest to >>>> remove this, or replace it by a more generic statement that more accurate >>>> information can be useful for several TCP extensions. >>>> >>>> I disagree. Both ConEx and DCTCP need more accurate information. They >> do >>>> not need the mechanism that is specified in this draft, however, this is >>>> not >>>> what the sentences is saying. >>> In my understanding (as a non-native speaker), the use of the word "need" >>> is not correct here. DCTCP as specified in RFC 8257 can be implemented >>> without any such mechanism. >>> >>> What would work for me is something of the form "... Data Center TCP >>> cannot get precise ECN feedback whenever more than one marking is >> received >>> in one RTT". >>> >>>>> This document specifies an >>>>> experimental scheme to provide more than one feedback signal per >> RTT >>>>> in the TCP header. Given TCP header space is scarce, it overloads >>>>> the three existing ECN-related flags in the TCP header and provides >>>>> additional information in a new TCP option. >>>>> >>>>> [ms] This statement needs to be rewritten to correctly reflect what is >>>> requested from IANA. My understanding is that this experimental >> document >>>> asks for allocation of a reserved TCP header flag. This needs to be >>>> called out >>>> prominently, IMHO. In addition, since this is not a standard, the >>>> suggested >>>> experimentation with the main TCP header must IMHO be explicitly >>>> mentioned. I also suggest to have later in a document a section that >>>> explicitly >>>> explains why it is appropriate to modify the main TCP header in an >>>> experiment. >>>> >>>> I don’t know if any requirement that IANA assignment need to be called >>>> out >>>> in the abstract but we can do that. However, I believe the question if >>>> this >>>> document should or should not assign the bit is still not completely >>>> solved, or >>>> is it? >>> I believe this question will have to be reviewed during WGLC and, more >>> importantly, IETF last call. For the moment, my concern is that the >>> document correctly describes the IANA allocation. >>> >>> I would like to see here a statement such as : "Given TCP header space is >>> scarce, this specification allocates a reserved header bit and overloads >>> the two ECN flags in the TCP header ...". >>> >>>>> * 1. Introduction >>>>> >>>>> Recently, proposed mechanisms like Congestion Exposure (ConEx >>>>> [RFC7713]), DCTCP [RFC8257] or L4S [I-D.ietf-tsvwg-l4s-arch] need >>>>> more accurate ECN feedback information whenever more than one >>>> marking >>>>> is received in one RTT. >>>>> >>>>> [ms] At least for RFC 8257 seems to be implementable withoit this. >>>>> Instead >>>> of stating a "need", it would IMHO make more sense to discuss the >>>> benefits >>>> of the suggested mechanism in this document of its own, independent of >>>> other proposals. To me, this document should be independent of other >>>> documents and specifically other experiments. We have to think about >>>> cases >>>> where not all experiments are successful. Then independent documents >> will >>>> be more future-proof in future. >>>> >>>> This is a naming collision… The sentence was meant to say that these >>>> mechanisms new more accurate ECN feedback than provided today by >>>> RFC3168 but it was not meant to say that these mechanism have to use >> the >>>> scheme as specified in this document. >>>> >>>> I added the following part sentence: >>>> >>>> „Recently, proposed mechanisms like Congestion Exposure (ConEx >>>> [RFC7713]), DCTCP [RFC8257] or L4S [I-D.ietf-tsvwg-l4s-arch] need more >>>> accurate ECN feedback information than provided by the feedback >> scheme >>>> as specified in [RFC3168] whenever more than one marking is received in >>>> one >>>> RTT. This document specifies an alternative feedback scheme that >> provides >>>> more accurate information and could be used by these new TCP >> extensions.“ >>>> Does this help? >>> See my proposal for the abstract. I continue to disagree with the term >>> "need" but I think this can be sorted out by another term. >>> >>>>> If AccECN progresses from experimental to the standards >>>>> track, it is intended to be a complete replacement for classic TCP/ >>>>> ECN feedback, not a fork in the design of TCP. >>>>> >>>>> [ms] This sentence should be removed, as this is speculation. >>>> Why? It states an intent… and that’s the intent that we have. >>>> >>>>> Until the AccECN experiment succeeds, [RFC3168] will remain as the >>>>> standards track specification for adding ECN to TCP. >>>>> >>>>> [ms] This sentence should be removed (or reworded) >>>> Why? Does it help to add an only here: >>>> >>>> "Until the AccECN experiment succeeds, [RFC3168] will remain as the only >>>> standards track specification for adding ECN to TCP.“ >>> This wording is better. >>> >>>>> AccECN feedback overloads flags and fields in the main TCP header >>>>> with new definitions, so both ends have to support the new wire >>>>> protocol before it can be used. >>>>> >>>>> [ms] In my reading this experimental document asks for *new* >> allocation >>>> of a reserved TCP header flag. >>>> >>>> Is this better? >>>> >>>> "AccECN feedback overloads the two existing ECN flags as well as the >>>> currently reserved and previously called NS flag in the main TCP >>>> header >>>> with new definitions, so both ends have to support the new wire >>>> protocol >>>> before it can be used.“ >>>> >>>> I understand that you are not happy with the word „overload“ here but >> the >>>> point of this sentence really is that the flags can/could be used >>>> differently >>>> and therefore we need a new negotiation before we can use them. >>> For me the following would work: "AccECN feedback overloads the two >>> existing ECN flags and >>> allocates the currently reserved and previously called NS flag in the main >>> TCP header. >>> Given the new definitions, both ends have to support the new wire >> protocol >>> before it can be used." >>> >>> I believe the wording has to be crystal clear on the reservation of bit 7 >>> when it is discussed the first time in the text. In follow-up sections, >>> maybe shorter terms could be used. >>> >>>> If you prefer, we can also remove the NS flag in this list, as ECN Nonce >>>> was >>>> anyway never deployed. >>>> >>>>> For that we refer to [RFC3168] or any RFC that >>>>> specifies a different response to TCP ECN feedback, for example: >>>>> [RFC8257]; or the ECN experiments referred to in >>>>> [I-D.ietf-tsvwg-ecn-experimentation], namely: a TCP-based Low >>>>> Latency >>>>> Low Loss Scalable (L4S) congestion control >>>>> [I-D.ietf-tsvwg-l4s-arch]; >>>>> ECN-capable TCP control packets [I-D.ietf-tcpm-generalized-ecn], or >>>>> Alternative Backoff with ECN (ABE) >>>>> [I-D.ietf-tcpm-alternativebackoff-ecn]. >>>>> >>>>> [ms] At least ABE seems orthogonal. Anyway, I think this paragraph can >>>>> just >>>> be deleted. If other experiments need more accurate feedback, it is up to >>>> them to explain how they would use this mechanism. This document >> should >>>> focus on how to signal the feedback, not how to use that. >>>> >>>> Yes, that is what the paragraph says. Isn’t it better to be explicit >>>> about this? >>>> >>>>> It is likely (but not required) that the AccECN protocol will be >>>>> implemented along with the following experimental additions to the >>>>> TCP-ECN protocol: ECN-capable TCP control packets and >>>>> retransmissions >>>>> [I-D.ietf-tcpm-generalized-ecn], which includes the ECN-capable SYN/ >>>>> ACK experiment [RFC5562]; and testing receiver non-compliance >>>>> [I-D.moncaster-tcpm-rcv-cheat]. >>>>> >>>>> [ms] I am a big fan of simple, standalone documents. In my view, the >>>>> TCPM >>>> working group should publish draft-ietf-tcpm-accurate-ecn and draft-ietf- >>>> tcpm-generalized-ecn independent documents, which probably implies >> that >>>> draft-ietf-tcpm-generalized-ecn does not use AccECN. If experimentation >>>> with ECT in SYN requires a combination, this could be done in a new, >>>> third >>>> document. Apart from having simpler focused documents, this could >>>> significantly help later with moving forward documents to standards >>>> track. >>>> >>>> I disagree, however, this is a discussion to have on draft-ietf-tcpm- >>>> generalized-ecn. I don’t see a problem in providing a reference here >>>> that >>>> says „it is likely…“ and nothing more. >>>>> >>>>> >>>>> * 1.1. Document Roadmap >>>>> >>>>> [ms] A macroscopic comment is that this document has a lot of >>>>> introduction >>>> and tutorial text with lot's of redundancy towards other documents. I >>>> think >>>> the document can be made much easier to read by shorten it. In many >> cases >>>> this is just an editorial change as there is redundancy. As one such >>>> example, >>>> just remove this section. >>>> >>>> I guess this a matter of taste. As an AD, I’m a big fan of short and >>>> concise >>>> documents, however, some redundancy can also help understanding, >>>> especially if you explain things multiple times but with a different >>>> level of >>>> detail. I personally would not need the roadmap but I know many people >>>> who find these things helpful and to be honest I don’t see how removing >>>> this >>>> part makes the doc any better. If you don’t want it, don’t read it. >>>> >>>>> >>>>> * 1.2. Goals >>>>> >>>>> [ms] I think this section can also just be removed. >>>> I have to say I also don’t see the point of removing this part. Given we’ve >>>> done the work on requirements, I think we should also link to this doc >>>> somewhere. >>>>> >>>>> * 1.3. Experiment Goals >>>>> >>>>> TCP is critical to the robust functioning of the Internet, therefore >>>>> any proposed modifications to TCP need to be thoroughly tested. The >>>>> present specification describes an experimental protocol that adds >>>>> more accurate ECN feedback to the TCP protocol. The intention is to >>>>> specify the protocol sufficiently so that more than one >>>>> implementation can be built in order to test its function, >>>>> robustness >>>>> and interoperability (with itself and with previous version of ECN >>>>> and TCP). >>>>> >>>>> [ms] I think all what is written in this paragraph is obvious, no? >>>>> Can't we just >>>> delete this? >>>> >>>> Sure, however, I don’t think it hurts to spell it out. For me both is >>>> fine, keep it >>>> or remove it. >>>> >>>>> The experimental protocol will be considered successful if it is >>>>> deployed and if it satisfies the requirements of [RFC7560] in the >>>>> consensus opinion of the IETF tcpm working group. In short, this >>>>> requires that it improves the accuracy and timeliness of TCP's ECN >>>>> feedback, as claimed in Section 5, while striking a balance between >>>>> the conflicting requirements of resilience, integrity and >>>>> minimisation of overhead. It also requires that it is not unduly >>>>> complex, and that it is compatible with prevalent equipment >>>>> behaviours in the current Internet (e.g. hardware offloading and >>>>> middleboxes), whether or not they comply with standards. >>>>> >>>>> Testing will mostly focus on fall-back strategies in case of >>>>> middlebox interference. Current recommended strategies are >>>>> specified >>>>> in Sections 3.1.2, 3.2.3, 3.2.4 and 3.2.7. The effectiveness of >>>>> these strategies depends on the actual deployment situation of >>>>> middleboxes. Therefore experimental verification to confirm large- >>>>> scale path traversal in the Internet is needed before finalizing >>>>> this >>>>> specification on the Standards Track. >>>>> >>>>> [ms] These two paragraphs must be entirely rewritten. As I have >>>> mentioned before, I don't think an RFC should speculate about TCPM and >>>> its >>>> consensus opinion. I would suggest a wording along the lines of: >>>>> >>>>> The experimental protocol will be considered successful if >>>>> testing confirms that the proposed mechanism can be deployed at >>>>> large >>>> scale. >>>>> Testing will mostly focus on fall-back strategies in case of >>>>> middlebox interference. Current recommended strategies are >>>>> specified >>>>> in Sections 3.1.2, 3.2.3, 3.2.4 and 3.2.7. The effectiveness of >>>>> these strategies depends on the actual deployment situation of >>>>> middleboxes. Therefore experimental verification to confirm large- >>>>> scale path traversal in the Internet is needed, e.g., by support in >>>>> major TCP stacks. >>>>> >>>>> >>>> I don’t understand your point here. I don’t think that the paraphrase >>>> speculates about the consensus of tcpm, in contrast it say tcpm has to >>>> decided if the requirements previously specified by tcpm are sufficiently >>>> fulfilled. I don’t see a reason to not mention the requirement draft as >>>> this >>>> draft as tcpm consensus and was written for this purpose. >>> My suggested wording uses the expression "can be deployed at large >> scale" >>> and I believe this is relevant. >>> >>> The document already describes in Section 5 how the protocol satisfies the >>> agreed requirements for a more accurate ECN feedback protocol >> [RFC7560]. >>> So, if the TCPM working group publishes this document with the content of >>> Section 5, I believe the TCPM working group already has reached >> consensus >>> that the protocol meets requirements. In addition, it is possible that new >>> requirements would be identified in future, e.g., as an outcome of the >>> experiment, and that would obviously have to be considered by TCPM. In >>> that case, for the success of the experiment not only RFC 7560 would >>> matter, but also further requirements. My proposed wording does not >> have >>> all these problems. >>> >>> In a nutshell, I continue to believe that this section has to change. >>> >>>>> * 1.5. Recap of Existing ECN feedback in IP/TCP >>>>> >>>>> [ms] This section could probably be shortened as well. >>>>> >>>>> The last bit in byte 13 of the TCP header was defined as the Nonce >>>>> Sum (NS) for the ECN Nonce [RFC3540]. RFC 3540 was never deployed >>>>> so >>>>> it is being reclassified as historic, making this TCP flag available >>>>> for use by the AccECN experiment instead. >>>>> >>>>> [ms] This wording, as well as Figure 1, needs to take into account the >>>>> IANA >>>> status when draft-ietf-tsvwg-ecn-experimentation is published. >>>> >>>> Is does. However, I can explicitly say that is has be re-clssified as >>>> reserved. >>>> >>>> "RFC 3540 was never deployed so it is being reclassified as historic >>>> [I-D.ietf- >>>> tsvwg-ecn-experimentation] and the respective flag has been marked as >>>> „reserved“ in the IANA TCP Header Flags registry, making this TCP flag >>>> available for use by the AccECN experiment instead.“ >>>> >>>> Better? >>>> >>>>> In my understanding, this experimental document asks for new >> assignment >>>> of a reserved TCP header flag. >>>> >>>> As I said I’m not sure if we have fully concluded this discussion yet. >>>> However, >>>> what we really would want to is mention somewhere that this >> experiment >>>> with this flags is running. I guess there are three options: >>>> 1) keep it in the registry as reserved and conserve the knowledge in tcpm >>>> that this experiment is running and no other experimental RFC such use >>>> this >>>> flags as long as this experiment is running. >>>> 2) Keep is marked as reserved but add a note about this experiment in >> the >>>> IANA registry >>>> 3) Or assign it right away with IESG approval. I guess in this case tcpm >>>> could >>>> also consider to change the registration policy to „IETF Review“. >>> The current registration policy for the TCP header flags is "standards >>> action". I understand that the IESG could approve exceptions. But given >>> the policy, I believe the document has to be very precise on the request >>> regarding bit 7. >>> >>>>> * 2. AccECN Protocol Overview and Rationale >>>>> >>>>> o an essential part that re-uses ECN TCP header bits to feed back >>>>> the number of arriving CE marked packets. This provides more >>>>> accuracy than classic ECN feedback, but limited resilience >>>>> against >>>>> ACK loss; >>>>> >>>>> [ms] The word "re-use" is IMHO not correct. >>>> I think this is nit picking. Using a different phrasing here makes the >>>> sentence >>>> unnecessary complicated. We don’t try to some how get a round the fact >>>> that we need to handle the flag registration correctly. However, here the >>>> point really is to explain how the protocol word. The main point of using >>>> the >>>> work „re-use“ here is really that we say that these flags are or have >>>> been >>>> used different by other TCP extension (and we therefore need a proper >>>> negotiation scheme). >>> If the allocation of a reserved flag is correctly explained in the >>> abstract and introduction, I think these sentences can use a bit relaxed >>> terminology. >>> >>>>> The two part design was necessary, given limitations on the space >>>>> available for TCP options and given the possibility that certain >>>>> incorrectly designed middleboxes prevent TCP using any new options. >>>>> >>>>> [ms] IMHO it would make sense to more explicitly mention the >> downsides >>>> of only specifying an option and not allocating a TCP header flag, in >>>> this >>>> experimental document. >>>> >>>> We need to use the flags (all three of them) for the negotiation. >>> I think that an explanation why negotiation by a TCP option would not >>> solve all use cases (or requirements) would help here. >>> >>>>> The obvious alternative would be to postpone the header flag >>>>> allocation to >>>> a follow-up standards track document and just keep it reserved. >>>> >>>> We can still do that. We should discuss and make a final decision. >>>> >>>>> The essential part overloads the previous definition of the three >>>>> flags in the TCP header that had been assigned for use by ECN. This >>>>> design choice deliberately replaces the classic ECN feedback >>>>> protocol, rather than leaving classic ECN feedback intact and adding >>>>> more accurate feedback separately because: >>>>> >>>>> [ms] Similar like previous comments, in my reading there are only _two_ >>>> ECN header flags. >>>> >>>> I think there are three flags that "had been assigned for use by ECN“ as >>>> ECN >>>> Nonce is also an ECN mechanism. The fact that one of the flags is now >>>> marked as reserved instead, it not that important for me here. >>>> >>>>> And, in addition, I think care is needed with wording such "replaces >>>>> the >>>> classic ECN feedback". I don't think this experiment replaces the ECN >>>> standards. That would be up to a follow-up PS. >>>> >>>> This sentence is not meant to say that RFC3168 is replaced. Actually we >>>> don’t. >>>> You can still use RFC3168 even if AccECN is implemented and deploy >>>> (however, we do intent that AccECN will be used as the default scheme in >>>> future and RFC3168 is hopefully simply not needed anymore at some >> point, >>>> even though you probably still need to have it implemented as the >>>> negotiation specified in this draft covers that as well, anyway...). The >>>> sentence says that if AccECN is negotiation, the header flags as used by >>>> RFC3168 and previously ECN Nonce are used differently (aka re-used). >> That’s >>>> all. >>>>> >>>>> 2.1. Capability Negotiation >>>>> >>>>> AccECN is a change to the wire protocol of the main TCP header, >>>>> therefore it can only be used if both endpoints have been upgraded >>>>> to >>>>> understand it. The TCP client signals support for AccECN on the >>>>> initial SYN of a connection and the TCP server signals whether it >>>>> supports AccECN on the SYN/ACK. The TCP flags on the SYN that the >>>>> client uses to signal AccECN support have been carefully chosen so >>>>> that a TCP server will interpret them as a request to support the >>>>> most recent variant of ECN feedback that it supports. Then the >>>>> client falls back to the same variant of ECN feedback. >>>>> >>>>> [ms] As this is an experimental specification, I would really like to >>>>> see a >>>> discussion how a future standards track version of more accurate ECN >>>> could >>>> be negotiated. >>>> >>>> As described in this draft. There will be no different. AccECN IS and >>>> will >>>> always be backward compatible with RFC3168. >>> That is not the problem I think about. I wonder about a PS version of >>> accurate ECN feedback that would possibly include changes as compared to >>> this experiment, e.g., because the experiment may have some lessons >>> learnt. What options would we have to negotiate the PS version, and how >>> could a stack implementing the PS version figure out whether the remote >>> end uses the experimental or the PS version of the protocol? >>> >>> The reason why I ask is because I don't see any easy solution to this but >>> I may miss something. Maybe this would not be a concern if there were >> some >>> codepoints left. >>> >>>>> How could both endpoints detect whether the other one implements >> the >>>> future standards track version? >>>> >>>> If the initiator implements AccECN it will request it’s use. If the >>>> receiver also >>>> implement it, it will/can negotiate it, if not it will look like an >>>> RFC3168 request >>>> for the receiver (as the NS flags will be ignored in the SYN) and it will >>>> negotiate RFC3168 ECN feedback if implemented. There is no additional >>>> detection needed. >>> My concern is the migration strategy for a future version, given that we >>> only experiment right now. For instance, if the initiator implements the >>> experimental version but the recipient implements the PS version of >>> AccECN, how would that work? Under the assumption that there are >> changes, >>> would there be a way to know? Maybe the answer to this question is no, >>> given the small number of bits we have. But not having any room for >>> extensions is not necessarily good protocol design. >>> >>>>> For instance, would the only safe variant be that we allocate yet >>>>> another >>>> reserved TCP header flag in a proposed standard to negotiate the >>>> standards >>>> track version, thus investing another reserved bit in the TCP header? >>>> >>>> No, that’s exactly what we use the NS flags for in the handshake. >>> If the PS version of AccECN was different to the experimental version of >>> AccECN, and if there was a deployed base of both, I still believe we could >>> end up in a situation in which we had to allocate yet another TCP header >>> flag to distinguish the different versions of AccECN. The NS flag would >>> not work for that if it is used by the experimental version. The risk of >>> having to spend further TCP header flags somehow concerns me. Of >> course, >>> that problem would only matter if the AccECN experiment succeeds >> somehow, >>> but lessons learnt would require protocol changes, which is speculation. >>> >>>>> I may be wrong, but to me it is too early to speculate how the PS >>>>> version >>>> would look like, and whether it would have to be different to the >>>> experimental version, due to lessons learnt. >>>> >>>> Of course you can always be wrong. However, the handshake negotiation >> is >>>> not the part we need experimentation for. That part is straight forward >>>> and >>>> works. If we really happen to detect a problem in that part, we would >>>> need >>>> to end the experiment declare failure and start over new. >>> My concern is ending the experiment when the experiment got (partly) >>> deployed in the Internet. In that case neither a new RFC nor a change of >>> the IANA registry will solve the migration issue. >>> >>>>> I believe in the IETF we typically design protocols that allow future >>>> extension, and it is not exactly clear to be how AccECN could be extended >>>> later. >>>> >>>> This is an TCP extension. If we want future extension we use the usually >>>> TCP >>>> mechanism (by defining a new TCP option I guess). >>> The root cause of my concern is that this proposal does *not* use the >>> usual way to experiment with TCP by options. It experiments with a header >>> flag, including in the SYN, and it seems to consume all codepoints. So, I >>> see the risk of a protocol design "not ready for future improvements". >>> >>> I cannot easily propose text. Maybe "lack of extensibility" is just one of >>> the short-comings of the protocol design that cannot be avoided but that >>> short-coming would have to be noted. >>> >>>>> An AccECN TCP client does not send the new AccECN Option on the >> SYN >>>>> as SYN option space is limited and successful negotiation using the >>>>> flags in the main header is taken as sufficient evidence that both >>>>> ends also support the AccECN Option. The TCP server sends the >>>>> AccECN >>>>> Option on the SYN/ACK and the client sends it on the first ACK to >>>>> test whether the network path forwards the option correctly. >>>>> >>>>> [ms] For what it is worth, I would personally be quite fine with >>>>> allowing (or >>>> even mandating) an option in the SYN in the experimental version of this >>>> protocol. For instance, saving the SYN option space would then an >>>> excellent >>>> reason for moving towards the PS specification. I am also fine with being >>>> in >>>> the rough part of the consensus here. >>>> >>>> The point is that we really don’t need the option in the SYN as we don’t >>>> use it >>>> for negotiation purposes as we use the header bits instead. So why >> should >>>> we waste the space? >>> For instance, mandating the option in the SYN would be away for the >>> receiver to distinguish the experiment from a follow-up PS version of the >>> spec, as the PS version may not mandate the option, to save header space. >>> >>> Maybe that proposal does not make any sense, and it may only have >>> downsides. But the document already speculates about a PS-follow-up. So >> it >>> seems a valid question to ask if the EXP and the PS version of the spec >>> have to be identical. This all comes down to the SYN negotiation. >>> >>>>> * 2.3. Delayed ACKs and Resilience Against ACK Loss >>>>> >>>>> If the AccECN Option is not available, e.g. it is being stripped by >>>>> a >>>>> middlebox, the AccECN protocol will only feed back information on CE >>>>> markings (using the ACE field). Although not ideal, this will be >>>>> sufficient, because it is envisaged that neither ECT(0) nor ECT(1) >>>>> will ever indicate more severe congestion than CE, even though >>>>> future >>>>> uses for ECT(0) or ECT(1) are still unclear >>>>> [I-D.ietf-tsvwg-ecn-experimentation]. >>>>> >>>>> [ms] This needs to be reworded >>>> Why? >>>> >>>>> >>>>> >>>>> * 2.4. Feedback Metrics >>>>> >>>>> The CE packet counter in the ACE field and the CE byte counter in >>>>> the >>>>> AccECN Option both provide feedback on received CE-marks. The CE >>>>> packet counter includes control packets that do not have payload >>>>> data, while the CE byte counter solely includes marked payload >>>>> bytes. >>>>> If both are present, the byte counter in the option will provide the >>>>> more accurate information needed for modern congestion control and >>>>> policing schemes, such as DCTCP or ConEx. >>>>> >>>>> [ms] I suggest to write in the last sentence only "... the option will >>>>> provide >>>> the more accurate information needed for congestion control". In >> general, >>>> I >>>> would prefer to have references to other mechanisms at only few (ideally >>>> a >>>> *single*) places in the document, instead of mixing them together. >>>> >>>> Sorry, I don’t see your point here. ConEx has been mentioned previously, >>>> so >>>> why not also mention it here. >>> As written earlier, in my understanding DCTCP does not "need" this. I >>> would suggest to have one place where to define exactly how this >> mechanism >>> can be used by other TCP extensions. Having said this, in this specific >>> paragraph I am less concerned about that than elsewhere. >>> >>>>> Feedback in bytes is recommended in order to protect against the >>>>> receiver using attacks similar to 'ACK-Division' to artificially >>>>> inflate the congestion window, which is why [RFC5681] now >> recommends >>>>> that TCP counts acknowledged bytes not packets. >>>>> >>>>> [ms] At least the last part and the reference to RFC 5681 is IMHO not >>>> needed here. >>>> >>>> Why? RFC5681 explains/refers the ACK division attack, so I think it is a >>>> very >>>> good reference to have here. >>>>> >>>>> >>>>> * 2.5. Generic (Dumb) Reflector >>>>> >>>>> The ACE field provides information about CE markings on both data >>>>> and >>>>> control packets. According to [RFC3168] the Data Sender is meant to >>>>> set control packets to Not-ECT. However, mechanisms in certain >>>>> private networks (e.g. data centres) set control packets to be ECN >>>>> capable because they are precisely the packets that performance >>>>> depends on most. >>>>> >>>>> For this reason, AccECN is designed to be a generic reflector of >>>>> whatever ECN markings it sees, whether or not they are compliant >>>>> with >>>>> a current standard. Then as standards evolve, Data Senders can >>>>> upgrade unilaterally without any need for receivers to upgrade too. >>>>> It is also useful to be able to rely on generic reflection behaviour >>>>> when senders need to test for unexpected interference with markings >>>>> (for instance [I-D.kuehlewind-tcpm-ecn-fallback] and >>>>> [I-D.moncaster-tcpm-rcv-cheat]). >>>>> >>>>> The initial SYN is the most critical control packet, so AccECN >>>>> provides feedback on whether it is CE marked. Although RFC 3168 >>>>> prohibits an ECN-capable SYN, providing feedback of CE marking on >>>>> the >>>>> SYN supports future scenarios in which SYNs might be ECN-enabled >>>>> (without prejudging whether they ought to be). For instance, >>>>> [I-D.ietf-tsvwg-ecn-experimentation] updates this aspect of RFC 3168 >>>>> to allow experimentation with ECN-capable TCP control packets. >>>>> >>>>> [ms] To me, the only thing that matters in this document that AccECN >>>>> can >>>> provide feedback on whether the SYN is CE marked. The discussion on >> how >>>> to experiment with ECT e.g. in SYNs IMHO does not belong into this >>>> document. So it seems sufficient here to note that one of the benefits of >>>> AccECN is that CE marks in SYNs can be fed back. >>>> >>>> I disagree. Explicitly saying that AccECN is only an feedback scheme and >>>> DOES >>>> NOT define how the information is used is VERY important because >> people >>>> come back to me over and over again and mix these things up. >>>>> >>>>> * 3.1.1. Negotiation during the TCP handshake >>>>> >>>>> Given the ECN Nonce [RFC3540] is being reclassified as historic, the >>>>> present specification renames the TCP flag at bit 7 of the TCP >>>>> header >>>>> flags from NS (Nonce Sum) to AE (Accurate ECN) (see IANA >>>>> Considerations in Section 6). >>>>> >>>>> [ms] As mentioned before, this needs to be rewritten to ask for new >>>>> IANA >>>> allocation of bit 7 in the TCP header flags. >>>> >>>> I really don’t understand this comment. That is what the IANA section >>>> does >>>> as referred here correctly. >>> In my reading of >>> https://www.iana.org/assignments/tcp-header-flags/tcp-header- >> flags.xhtml, >>> this flag currently has no name, i.e., it does not have the name NS. So >>> this statement on "renaming" is formally incorrect IMHO. >>> >>> I'd suggest something like: "This specification assigns the name AE >>> (Accurate ECN) to the TCP flag at bit 7; this flag has previously been >>> known as NS (Nonce Sum) ...". >>> >>>> Again, yes, we can discuss if this document should do this, but if >>>> published as >>>> it is, section 6 says everything that is needs to say. (I does not put >>>> any >>>> request on IANA, instead it is written as it would show up >>>> post-publication, >>>> implicitly providing a request to IANA at approval time. That is fine.) >>>> >>>>> Table 2: ECN capability negotiation between Client (A) and Server >>>>> (B) >>>>> >>>>> [ms] As far as I can see, in -05 this table allocates all existing >>>>> codepoints, >>>> while -03 had two currently unused codepoints. Not having any >> codepoints >>>> left seems to me not really future proof, e.g., regarding future proposed >>>> standards in this space (and I personally believe that TCP header flags >>>> must >>>> be allocated in a PS). And I don't fully see a need of feeding back ECT0 >>>> and >>>> specifically ECT1 in the TCP header flags as part of the experiment. Do >>>> we >>>> know for sure that this is the only possible use case of these two >>>> unallocated >>>> header bits? And why can't e.g. this be done in a TCP option instead? Or >>>> do I >>>> miss something? >>>> >>>> The point is really, if we don’t assign them now and start deployment we >>>> effetely we not be able to every assign them again because don’t have a >>>> different negotiation mechanisms. Realizing this, it is just the right >>>> think to >>>> define the space completely that is negotiated as use for AccECN in the >>>> handshake. >>> I disagree that consuming all codepoints and not having room for future >>> extensions is the "right thing" to do. It is in fact the wrong thing. But >>> possibly there is no alternative with the few bits, so maybe we end up >>> doing it. >>> >>> Yet, at minimum, using all codepoints needs to be reasoned in the >>> document. Specifically since in -03 a different protocol design seemed >>> possible, which looked to me more future-proof. >>> >>>>> * 3.1.2. Retransmission of the SYN >>>>> >>>>> However, >>>>> current measurements imply that a drop is less likely to be due to >>>>> middlebox interference than other intermittent causes of loss, e.g. >>>>> congestion, wireless interference, etc. >>>>> >>>>> [ms] Such wording IMHO doesn't belong into normative text. This may >>>> actually also apply to other heuristics discussed in this section, which >>>> are not >>>> really important for interoperability. >>>> >>>> I don’t really understand your point. This sentence is solely meant to >>>> reason >>>> the design decision to not say that a sender SHOULD attempt to re- >>>> negotiation after a loss. >>> One could split the reasoning from the normative design. The normative >>> specification may still be relevant in 20 years from now. Middleboxes will >>> almost likely be different in 20 years. Having said this, this is more of >>> an editorial remark. >>> >>>>> 3.2.7. Path Traversal of the AccECN Option >>>>> >>>>> 3.2.7.1. Testing the AccECN Option during the Handshake >>>>> >>>>> The TCP client MUST NOT include the AccECN TCP Option on the SYN. >>>>> >>>>> [ms] I am not sure if I really understand the motivation for not >>>>> allowing a >>>> option in the SYN. If the sender has space in the SYN left, what is the >>>> harm in >>>> an experimental version of the protocol? And I may miss something, but >>>> what would prevent the use of 2-byte option to negotiate the use of >>>> AccECN, e.g., to avoid experimental allocation of bit 7 in the initial >>>> SYN? >>>> >>>> I did have a draft on that as a proposal for an alternative design. >>>> However, >>>> the gourd was more supportive of this design as it is proof of middlebox >>>> SYN >>>> option mangling which is a know problem. >>>> >>>> Therefore we simply don’t need option in the SYN and there is no reason >>>> to >>>> waste the space. >>>> >>>>> While I think many tutorial text in this document could be shortened, I >>>> believe the use of a reserved TCP header flag should be reasoned. >>>> >>>> I’m actually uncertain what you expect here. >>> For instance, this reply with the explanation of SYN option mangling seems >>> useful explanation. >>> >>>>> * 3.2.8. Usage of the AccECN TCP Option >>>>> >>>>> The following rules determine when a Data Receiver in AccECN mode >>>>> sends the AccECN TCP Option, and which fields to include: >>>>> >>>>> Change-Triggered ACKs: If an arriving packet increments a different >>>>> byte counter to that incremented by the previous packet, the Data >>>>> Receiver MUST immediately send an ACK with an AccECN Option, >>>>> without waiting for the next delayed ACK (this is in addition to >>>>> the safety recommendation in Section 3.2.5 against ambiguity of >>>>> the ACE field). >>>>> >>>>> This is stated as a "MUST" so that the data sender can rely on >>>>> change-triggered ACKs to detect transitions right from the very >>>>> start of a flow, without first having to detect whether the >>>>> receiver complies. A concern has been raised that certain >>>>> offload >>>>> hardware needed for high performance might not be able to support >>>>> change-triggered ACKs, although high performance protocols such >>>>> as >>>>> DCTCP successfully use change-triggered ACKs. >>>>> >>>>> [ms] To me this sounds like a perfect example for a SHOULD with >>>>> additional >>>> guidance why implementing this SHOULD is really important. >>>> >>>> This is one of the most discussed point from the author and we really >>>> tried to >>>> get additional guidance here of what to do also from outside the IETF but >>>> did >>>> no clear feedback. >>>> >>>> As explained this MUST enables additional functionality. However, this is >>>> an >>>> experiment document. If we detect that this MUST does actually hinder >>>> implementation or has just never been implemented, we should >> reconsider >>>> this in the final PS RFC. >>>> >>>> I was on the SHOULD side of the discussion, but can say that the >>>> implementation in Linux was way more simple then expected. Offloading >>>> might be a different topic but that is where I could not get a clear >>>> feedback >>>> and offloading could probably be anyway optimized for use with AccECN >> (if >>>> it >>>> gets deploy widely). >>> If a MUST requires changes in hardware, I think there must be a clear >>> reason. >>> >>> As individual contributor, with the current explanation in the text, I >>> believe this has to be a SHOULD. >>> >>>> I though we added this as something to mention in the exp goals section, >>>> but >>>> obviously we didn’t. I added the following text now: >>>> >>>> "Another experimentation focus is the implementation feasibiliy of >>>> change- >>>> triggered ACKs as described in section 3.2.8. While on average this >>>> should not >>>> lead to a higher ACK rate, it changes the ACK patter which especially can >>>> have >>>> an impact on hardware offload. Further experimentation is needed to >>>> advise >>>> if this should a hard requirement or just prefer behavior.“ >>> If it is unclear if a MUST can actually be implemented, having a MUST is >>> in my opinion the wrong approach. >>> >>> One could equally state here that further experimentation is needed to >>> determine whether the SHOULD can be upgraded to a MUST. >>> >>>>> For the avoidance of doubt, the change-triggered ACK mechanism is >>>>> deliberately worded to ignore the arrival of a control packet with >>>>> no >>>>> payload, which therefore does not alter any byte counters, because >>>>> it >>>>> is important that TCP does not acknowledge pure ACKs. The change- >>>>> triggered ACK approach will lead to some additional ACKs but it >>>>> feeds >>>>> back the timing and the order in which ECN marks are received with >>>>> minimal additional complexity. >>>>> >>>>> [ms] The additional acks create network load. I think some wording is >>>> needed on the tradeoff between information accuracy and network load. >>>> There are network environments in which any additional packet is very >>>> expensive (e.g., energy) and it is not clear to me how the protocol >>>> design >>>> takes into account the potential overhead of additional ACKs. Maybe this >>>> could be another reason for a SHOULD. >>>> >>>> The above. However, this is not really an additional ACK because you do >>>> delay the next one. Further experimentation needed. >>> The document states "lead to some additional ACKs". If that does not >>> increase network load, I think it has to be explicitly explained why the >>> ACK load is at most equal to a current TCP stack, in all potential cases. >>> If it can increases network load, it has to be reasoned why increasing >>> load (and risk of reverse congestion and the like) is worth the effort. >>> >>> I agree that this may be an area of experimentation, but I believe then it >>> has to be explained to implementers what the tradeoffs are. >>> >>>>> * 4.2. Compatibility with Other TCP Options and Experiments >>>>> >>>>> AccECN is compatible (at least on paper) with the most commonly >> used >>>>> TCP options: MSS, time-stamp, window scaling, SACK and TCP-AO. It >>>>> is >>>>> also compatible with the recent promising experimental TCP options >>>>> TCP Fast Open (TFO [RFC7413]) and Multipath TCP (MPTCP [RFC6824]). >>>>> >>>>> [ms] I would suggest the wording "... compatible with the experimental >>>> TCP options ..." or even "... compatible with the TCP options …". >>>> >>>> These option are to experimental..? >>> "It is also compatible with the experimental TCP options TCP Fast Open >>> (TFO [RFC7413]) and Multipath TCP (MPTCP [RFC6824])." would have same >>> technical meaning. Having said this, this is editorial only. >>> >>>> The point of using „commonly used“ was to say that we checked on those >> as >>>> they seem important. Just because your favorite experiment option is not >>>> listed here it doesn’t means it incompatible, we just didn’t check. I’m >>>> okay to >>>> removed „commonly used“ but I don’t think it makes anything better. >>>> >>>>> * 4.3. Compatibility with Feedback Integrity Mechanisms >>>>> >>>>> [ms] Quite a bit in this section is experimental work, which IMHO >>>>> should be clearly emphasized. The one exception is… >>>> I would really like to keep this section because integrity is usually the >>>> fist >>>> question that come up when I present AccECN. Effectively these are two >>>> independent topics, however, I really think it help people to understand >>>> the >>>> whole picture if this is also discussed in this document. >>>> >>>>> However, TCP-AO is often too brittle to use on many end-to-end >>>>> paths, where middleboxes can make verification fail in their >>>>> attempts to improve performance or security, e.g. by >>>>> resegmentation or shifting the sequence space. >>>>> >>>>> [ms] I am not sure if deployment challenges of other options need to be >>>> discussed in this document. >>>> >>>> If we keep the discussion, I guess we should mention this as well. As the >>>> doc >>>> clearly stated section 4 is not meant to be normative. >>> As far as I can see, there are use cases for TCP-AO where middleboxes are >>> simply not a problem, but it is exactly this sort of discussion that may >>> not be needed in this document. But I won't rat-hole on this comment >> here. >>>>> Originally the ECN Nonce [RFC3540] was proposed to ensure integrity >>>>> of congestion feedback. With minor changes AccECN could be >>>>> optimised >>>>> for the possibility that the ECT(1) codepoint might be used as an >>>>> ECN >>>>> Nonce . However, given RFC 3540 is being reclassified as historic, >>>>> the AccECN design has been generalised so that it ought to be able >>>>> to >>>>> support other possible uses of the ECT(1) codepoint, such as a lower >>>>> severity or a more instant congestion signal than CE. >>>>> >>>>> [ms] The discussion of RFC 3540 can probably be removed to a large >>>>> extent. >>>> Unfortunately, please still think that ECN Nonce, even though is was >>>> never >>>> deployed and doesn’t really work, is the only was to provide integrity >>>> protection and we need it as a prerequisite to deploy ECN at all… i would >>>> really prefer to keep this in this non-normative part of the doc. >>>> >>>>> >>>>> * 6. IANA Considerations >>>>> >>>>> [ms] I think this section needs to be rewritten to request a new >>>>> allocation >>>> of bit 7 of the TCP header flags. At least for the process I think it >>>> would make >>>> sense to have somewhere in the document a comprehensive explanation >> of >>>> why an experimental document requests a change of the main TCP >> header, >>>> and why this cannot be avoided (most notably in the initial SYN) by an >>>> alternative protocol design. >>>> >>>> As I said previously this section is written as it would look like after >>>> the >>>> allocation has happened with publication approval of the IESG. This is >>>> fine. >>>> >>>> Having a discussion about an experiment doc assigning a flag (or not) is >>>> a >>>> question for tcpm as a whole and not specifically this document. How do >>>> we >>>> envision to every use any further flags? We go to PS right away? Or >>>> should >>>> we change the registration policy? For me the latter makes actually more >>>> sense. However, if we don’t want/can to decide this now, we also could >> go >>>> forward as it is with IESG approval. However, is this case it is also not >>>> needed >>>> to explain this in the document. The responsible AD has to explain this >>>> to the >>>> other IESG probably in the ballot or even better the shepherd could >>>> provide >>>> these information in the write-up. >>>> >>>>> >>>>> * 9. Comments Solicited >>>>> >>>>> Comments and questions are encouraged and very welcome. They >> can >>>> be >>>>> addressed to the IETF TCP maintenance and minor modifications >>>>> working >>>>> group mailing list , and/or to the authors. >>>>> >>>>> [ms] This section is not needed IMHO >>>> Yes, it will be removed before publication. >>>>> >>>>> 10. References >>>>> >>>>> [I-D.ietf-tsvwg-ecn-experimentation] >>>>> Black, D., "Relaxing Restrictions on Explicit Congestion >>>>> Notification (ECN) Experimentation", >>>>> draft-ietf-tsvwg-ecn- >>>>> experimentation-07 (work in progress), October 2017. >>>>> >>>>> [ms] Normative reference? >>>> Don’t see why. No need to read ietf-tsvwg-ecn-experimentation to >>>> understand the spec in this doc. >>>> >>>> Thanks! >>>> Mirja >>>> >>> _______________________________________________ >>> tcpm mailing list >>> tcpm@ietf.org >>> https://www.ietf.org/mailman/listinfo/tcpm >>> >>> >>> --- >>> This email has been checked for viruses by AVG. >>> http://www.avg.com >>> > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm -- ________________________________________________________________ Bob Briscoe http://bobbriscoe.net/ --------------29C23CFC30CF5D948F19EE0A Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit Michael,

I realize now that I misunderstood your question during the TCPM session in London. (Sorry for the delay - I tuned out for a while).

As well as the 2 codepoints that Richard has highlighted, the draft is careful to require that implementations of the AccECN experiment will silently accept future use of other unexpected codepoints ('forward compatibility'). For instance, at the end of S.3.2,3:
   Note that the Data Sender MUST NOT test whether the arriving counter
   in the initial ACE field has been initialized to a specific valid
   value - the above check solely tests whether the ACE fields have been
   incorrectly zeroed.  This allows hosts to use different initial
   values as an additional signalling channel in future.
We have only used codepoints 6&7 here. Assuming we avoid zero, because it might be due to bleaching, this leaves values 1-5 unused.

The analysis I did of the remaining combinations of codepoints when taken across the whole handshake is here:
    http://bobbriscoe.net/projects/latency/ecn-modes.pdf
Page 1 covers the TCP ECN flags.

The 1st row marked A-CU (short for Accurate ECN Currently Unassigned) shows the 5 values that the above text keeps for future use.
The 2nd two rows marked A-CU are now used by AccECN (since draft-04) to deal with the bleaching we discovered during testing.
All the combinations tagged with 'N' (for nonce) are likely to be the most useful if needed for any additional negotiation space.

The other one Richard tagged (labelled 'broken') is still polluted by a non-negligible number of bugged servers that reflect the 6 most significant TCP flags in the SYN back in the SYN/ACK. From a 2014 measurement study that Richard and Mirja co-authored, there were still about 0.3% of the Alexa 1M exhibiting the symptoms of this bug. I'm in the process of checking our more recent data, where we specifically checked if 111 used on the AccECN SYN was reflected.

Taken as a whole, I admit this doesn't leave much flexibility for future variation, but it's the best we could do with the available space constraints.
Does this sufficiently address your concern?


Cheers



Bob

PS. Similarly, although we have not allowed any space in the AccECN TCP Option for flag bits, we have allowed for different initial values of the counters to be used to signal new versions of the protocol in future. See the end of S.3.2.7.4:

   Note that the Data Sender MUST NOT test whether the arriving byte
   counters in the initial AccECN Option have been initialized to
   specific valid values - the above checks solely test whether these
   fields have been incorrectly zeroed.  This allows hosts to use
   different initial values as an additional signalling channel in
   future.


On 19/03/18 16:04, Scharf, Michael (Nokia - DE/Stuttgart) wrote:
For what it is worth, this is exactly the sort of discussion I believe we need to have.

I would be much more comfortable if the document did foresee some way negotiate an "enhanced" version of Accurate ECN in a later phase - most notably a PS version of the protocol. If that would cost some functionality right now, as we may have to reserve some codepoints in the EXP version, that would be perfectly fine for me, as long as we could add that functionality later in a PS version (under our current understanding).

I have not thought about the details. But we have only three reserved TCP header bits left after publication of this document and I am worried about that.

Michael



-----Original Message-----
From: Richard Scheffenegger [mailto:rs.ietf@gmx.at]
Sent: Monday, March 19, 2018 4:27 PM
To: Scharf, Michael (Nokia - DE/Stuttgart) <michael.scharf@nokia.com>;
Mirja Kühlewind <mirja.kuehlewind@tik.ee.ethz.ch>
Cc: draft-ietf-tcpm-accurate-ecn@ietf.org; tcpm@ietf.org
Subject: Re: [tcpm] Comments on draft-ietf-tcpm-accurate-ecn

Hi Michael,

regarding your comment about the lack of future extensibility of the ECN /
AccECN header handshake without further bits...

I wanted to note, that the logic table we have in the accecn draft currently
has two entries, that are reserved because of observed, but legacy, stacks
misbehaving when reflecting these TCP header bits.

So there is a possibility to eventually use those entries - once the broken
legacy stacks have been confirmeed to have left the internet - as some
additional negotiation capability.

Also, the feedback of the byte counters vs. CE packet counter are split into
using a tcp option and the tcp header bits.
An extention of AccECN could take the form of using a differen TCP option
together with the header counter negotiation...

Earlier work by us determined that packing more information into the tcp
header bits alone (as an example) would likely need an additional bit (that
can also be used in a negotiation scheme)...


   +--------+--------+------------+-------------+----------------------+
   | A      | B      |  SYN A->B  |   SYN/ACK   | Feedback Mode        |
   |        |        |            |     B->A    |                      |
   +--------+--------+------------+-------------+----------------------+
   |        |        | AE CWR ECE |  AE CWR ECE |                      |
   | AccECN | AccECN | 1   1   1  |  0   1   0  | AccECN (Not-ECT on   |
   |        |        |            |             | SYN)                 |
   | AccECN | AccECN | 1   1   1  |  0   1   1  | AccECN (ECT1 on SYN) |
   | AccECN | AccECN | 1   1   1  |  1   0   0  | AccECN (ECT0 on SYN) |
   | AccECN | AccECN | 1   1   1  |  1   1   0  | AccECN (CE on SYN)   |
   |        |        |            |             |                      |
#  | AccECN | Nonce  | 1   1   1  |  1   0   1  | classic ECN          |
   | AccECN | ECN    | 1   1   1  |  0   0   1  | classic ECN          |
   | AccECN | No ECN | 1   1   1  |  0   0   0  | Not ECN              |
   |        |        |            |             |                      |
   | Nonce  | AccECN | 0   1   1  |  0   0   1  | classic ECN          |
   | ECN    | AccECN | 0   1   1  |  0   0   1  | classic ECN          |
   | No ECN | AccECN | 0   0   0  |  0   0   0  | Not ECN              |
   |        |        |            |             |                      |
#  | AccECN | Broken | 1   1   1  |  1   1   1  | Not ECN              |
   +--------+--------+------------+-------------+----------------------+
Best regards,
   Richard


----- Original Message -----
From: "Scharf, Michael (Nokia - DE/Stuttgart)" <michael.scharf@nokia.com>
To: "Mirja Kühlewind" <mirja.kuehlewind@tik.ee.ethz.ch>
Cc: <draft-ietf-tcpm-accurate-ecn@ietf.org>; <tcpm@ietf.org>
Sent: Monday, March 12, 2018 1:59 AM
Subject: Re: [tcpm] Comments on draft-ietf-tcpm-accurate-ecn



Hi Mirja,

Thanks a lot for the explanation. I won't follow-up on some of the
editorial suggestions.

Yet, I continue to believe that some formal wording in the document needs
to change, as explained below.

Thanks

Michael (with no hat on)



-----Original Message-----
From: Mirja Kühlewind [mailto:mirja.kuehlewind@tik.ee.ethz.ch]
Sent: Monday, March 05, 2018 1:54 PM
To: Scharf, Michael (Nokia - DE/Stuttgart) <michael.scharf@nokia.com>
Cc: draft-ietf-tcpm-accurate-ecn@ietf.org; tcpm@ietf.org
Subject: Re: Comments on draft-ietf-tcpm-accurate-ecn

Hi Micheal,

thanks for your feedback and sorry for my late reply.

Please see inline.


Am 03.12.2017 um 20:17 schrieb Scharf, Michael (Nokia - DE/Stuttgart)

<michael.scharf@nokia.com>:

Hi all,

I have read draft-ietf-tcpm-accurate-ecn-05 (without the appendix). I

believe this document needs further work before moving forward.

Please find below my comments marked as [ms]. I have read the

document independent of the review from Gorry. I apologize if there is
duplication.

Thanks

Michael (with no hat on)


******************************

* Abstract:

   Recently, new TCP mechanisms like Congestion Exposure (ConEx) or
Data

Center TCP

   (DCTCP) need more accurate ECN feedback information whenever

more

   than one marking is received in one RTT.

[ms] I don't think this statement is fully backed by RFC 8257. I
suggest to

remove this, or replace it by a more generic statement that more accurate
information can be useful for several TCP extensions.

I disagree. Both ConEx and DCTCP need more accurate information. They

do

not need the mechanism that is specified in this draft, however, this is
not
what the sentences is saying.

In my understanding (as a non-native speaker), the use of the word "need"
is not correct here. DCTCP as specified in RFC 8257 can be implemented
without any such mechanism.

What would work for me is something of the form "... Data Center TCP
cannot get precise ECN feedback whenever more than one marking is

received

in one RTT".


   This document specifies an
   experimental scheme to provide more than one feedback signal per

RTT

   in the TCP header.  Given TCP header space is scarce, it overloads
   the three existing ECN-related flags in the TCP header and provides
   additional information in a new TCP option.

[ms] This statement needs to be rewritten to correctly reflect what is

requested from IANA. My understanding is that this experimental

document

asks for allocation of a reserved TCP header flag. This needs to be
called out
prominently, IMHO. In addition, since this is not a standard, the
suggested
experimentation with the main TCP header must IMHO be explicitly
mentioned. I also suggest to have later in a document a section that
explicitly
explains why it is appropriate to modify the main TCP header in an
experiment.

I don’t know if any requirement that IANA assignment need to be called
out
in the abstract but we can do that. However, I believe the question if
this
document should or should not assign the bit is still not completely
solved, or
is it?

I believe this question will have to be reviewed during WGLC and, more
importantly, IETF last call. For the moment, my concern is that the
document correctly describes the IANA allocation.

I would like to see here a statement such as : "Given TCP header space is
scarce, this specification allocates a reserved header bit and overloads
the two ECN flags in the TCP header ...".


* 1.  Introduction

   Recently, proposed mechanisms like Congestion Exposure (ConEx
   [RFC7713]), DCTCP [RFC8257] or L4S [I-D.ietf-tsvwg-l4s-arch] need
   more accurate ECN feedback information whenever more than one

marking

   is received in one RTT.

[ms] At least for RFC 8257 seems to be implementable withoit this.
Instead

of stating a "need", it would IMHO make more sense to discuss the
benefits
of the suggested mechanism in this document of its own, independent of
other proposals. To me, this document should be independent of other
documents and specifically other experiments. We have to think about
cases
where not all experiments are successful. Then independent documents

will

be more future-proof in future.

This is a naming collision… The sentence was meant to say that these
mechanisms new more accurate ECN feedback than provided today by
RFC3168 but it was not meant to say that these mechanism have to use

the

scheme as specified in this document.

I added the following part sentence:

„Recently, proposed mechanisms like Congestion Exposure (ConEx
[RFC7713]), DCTCP [RFC8257] or L4S [I-D.ietf-tsvwg-l4s-arch] need more
accurate ECN feedback information than provided by the feedback

scheme

as specified in [RFC3168] whenever more than one marking is received in
one
RTT. This document specifies an alternative feedback scheme that

provides

more accurate information and could be used by these new TCP

extensions.“

Does this help?

See my proposal for the abstract. I continue to disagree with the term
"need" but I think this can be sorted out by another term.


   If AccECN progresses from experimental to the standards
   track, it is intended to be a complete replacement for classic TCP/
   ECN feedback, not a fork in the design of TCP.

[ms] This sentence should be removed, as this is speculation.

Why? It states an intent… and that’s the intent that we have.


   Until the AccECN experiment succeeds, [RFC3168] will remain as the
   standards track specification for adding ECN to TCP.

[ms] This sentence should be removed (or reworded)

Why? Does it help to add an only here:

"Until the AccECN experiment succeeds, [RFC3168] will remain as the only
standards track specification for adding ECN to TCP.“

This wording is better.


   AccECN feedback overloads flags and fields in the main TCP header
   with new definitions, so both ends have to support the new wire
   protocol before it can be used.

[ms] In my reading this experimental document asks for *new*

allocation

of a reserved TCP header flag.

Is this better?

"AccECN feedback overloads the two existing ECN flags as well as the
      currently reserved and previously called NS flag in the main TCP
header
      with new definitions, so both ends have to support the new wire
protocol
      before it can be used.“

I understand that you are not happy with the word „overload“ here but

the

point of this sentence really is that the flags can/could be used
differently
and therefore we need a new negotiation before we can use them.

For me the following would work: "AccECN feedback overloads the two
existing ECN flags and
allocates the currently reserved and previously called NS flag in the main
TCP header.
Given the new definitions, both ends have to support the new wire

protocol

before it can be used."

I believe the wording has to be crystal clear on the reservation of bit 7
when it is discussed the first time in the text. In follow-up sections,
maybe shorter terms could be used.


If you prefer, we can also remove the NS flag in this list, as ECN Nonce
was
anyway never deployed.


   For that we refer to [RFC3168] or any RFC that
   specifies a different response to TCP ECN feedback, for example:
   [RFC8257]; or the ECN experiments referred to in
   [I-D.ietf-tsvwg-ecn-experimentation], namely: a TCP-based Low
Latency
   Low Loss Scalable (L4S) congestion control
[I-D.ietf-tsvwg-l4s-arch];
   ECN-capable TCP control packets [I-D.ietf-tcpm-generalized-ecn], or
   Alternative Backoff with ECN (ABE)
   [I-D.ietf-tcpm-alternativebackoff-ecn].

[ms] At least ABE seems orthogonal. Anyway, I think this paragraph can
just

be deleted. If other experiments need more accurate feedback, it is up to
them to explain how they would use this mechanism. This document

should

focus on how to signal the feedback, not how to use that.

Yes, that is what the paragraph says. Isn’t it better to be explicit
about this?


   It is likely (but not required) that the AccECN protocol will be
   implemented along with the following experimental additions to the
   TCP-ECN protocol: ECN-capable TCP control packets and
retransmissions
   [I-D.ietf-tcpm-generalized-ecn], which includes the ECN-capable SYN/
   ACK experiment [RFC5562]; and testing receiver non-compliance
   [I-D.moncaster-tcpm-rcv-cheat].

[ms] I am a big fan of simple, standalone documents. In my view, the
TCPM

working group should publish draft-ietf-tcpm-accurate-ecn and draft-ietf-
tcpm-generalized-ecn independent documents, which probably implies

that

draft-ietf-tcpm-generalized-ecn does not use AccECN. If experimentation
with ECT in SYN requires a combination, this could be done in a new,
third
document. Apart from having simpler focused documents, this could
significantly help later with moving forward documents to standards
track.

I disagree, however, this is a discussion to have on draft-ietf-tcpm-
generalized-ecn. I don’t see a problem in  providing a reference here
that
says „it is likely…“ and nothing more.



* 1.1.  Document Roadmap

[ms] A macroscopic comment is that this document has a lot of
introduction

and tutorial text with lot's of redundancy towards other documents. I
think
the document can be made much easier to read by shorten it. In many

cases

this is just an editorial change as there is redundancy. As one such
example,
just remove this section.

I guess this a matter of taste. As an AD, I’m a big fan of short and
concise
documents, however, some redundancy can also help understanding,
especially if you explain things multiple times but with a different
level of
detail. I personally would not need the roadmap but I know many people
who find these things helpful and to be honest I don’t see how removing
this
part makes the doc any better. If you don’t want it, don’t read it.



* 1.2.  Goals

[ms] I think this section can also just be removed.

I have to say I also don’t see the point of removing this part. Given we’ve
done the work on requirements, I think we should also link to this doc
somewhere.


* 1.3.  Experiment Goals

   TCP is critical to the robust functioning of the Internet, therefore
   any proposed modifications to TCP need to be thoroughly tested. The
   present specification describes an experimental protocol that adds
   more accurate ECN feedback to the TCP protocol.  The intention is to
   specify the protocol sufficiently so that more than one
   implementation can be built in order to test its function,
robustness
   and interoperability (with itself and with previous version of ECN
   and TCP).

[ms] I think all what is written in this paragraph is obvious, no?
Can't we just

delete this?

Sure, however, I don’t think it hurts to spell it out. For me both is
fine, keep it
or remove it.


   The experimental protocol will be considered successful if it is
   deployed and if it satisfies the requirements of [RFC7560] in the
   consensus opinion of the IETF tcpm working group.  In short, this
   requires that it improves the accuracy and timeliness of TCP's ECN
   feedback, as claimed in Section 5, while striking a balance between
   the conflicting requirements of resilience, integrity and
   minimisation of overhead.  It also requires that it is not unduly
   complex, and that it is compatible with prevalent equipment
   behaviours in the current Internet (e.g. hardware offloading and
   middleboxes), whether or not they comply with standards.

   Testing will mostly focus on fall-back strategies in case of
   middlebox interference.  Current recommended strategies are
specified
   in Sections 3.1.2, 3.2.3, 3.2.4 and 3.2.7.  The effectiveness of
   these strategies depends on the actual deployment situation of
   middleboxes.  Therefore experimental verification to confirm large-
   scale path traversal in the Internet is needed before finalizing
this
   specification on the Standards Track.

[ms] These two paragraphs must be entirely rewritten. As I have

mentioned before, I don't think an RFC should speculate about TCPM and
its
consensus opinion. I would suggest a wording along the lines of:

<ms>
   The experimental protocol will be considered successful if
   testing confirms that the proposed mechanism can be deployed at
large

scale.

   Testing will mostly focus on fall-back strategies in case of
   middlebox interference.  Current recommended strategies are
specified
   in Sections 3.1.2, 3.2.3, 3.2.4 and 3.2.7.  The effectiveness of
   these strategies depends on the actual deployment situation of
   middleboxes.  Therefore experimental verification to confirm large-
   scale path traversal in the Internet is needed, e.g., by support in
   major TCP stacks.
</ms>


I don’t understand your point here. I don’t think that the paraphrase
speculates about the consensus of tcpm, in contrast it say tcpm has to
decided if the requirements previously specified by tcpm are sufficiently
fulfilled. I don’t see a reason to not mention the requirement draft as
this
draft as tcpm consensus and was written for this purpose.

My suggested wording uses the expression "can be deployed at large

scale"

and I believe this is relevant.

The document already describes in Section 5 how the protocol satisfies the
agreed requirements for a more accurate ECN feedback protocol

[RFC7560].

So, if the TCPM working group publishes this document with the content of
Section 5, I believe the TCPM working group already has reached

consensus

that the protocol meets requirements. In addition, it is possible that new
requirements would be identified in future, e.g., as an outcome of the
experiment, and that would obviously have to be considered by TCPM. In
that case, for the success of the experiment not only RFC 7560 would
matter, but also further requirements. My proposed wording does not

have

all these problems.

In a nutshell, I continue to believe that this section has to change.


* 1.5.  Recap of Existing ECN feedback in IP/TCP

[ms] This section could probably be shortened as well.

   The last bit in byte 13 of the TCP header was defined as the Nonce
   Sum (NS) for the ECN Nonce [RFC3540].  RFC 3540 was never deployed
so
   it is being reclassified as historic, making this TCP flag available
   for use by the AccECN experiment instead.

[ms] This wording, as well as Figure 1, needs to take into account the
IANA

status when draft-ietf-tsvwg-ecn-experimentation is published.

Is does. However, I can explicitly say that is has be re-clssified as
reserved.

"RFC 3540 was never deployed so it is being reclassified as historic
[I-D.ietf-
tsvwg-ecn-experimentation] and the respective flag has been marked as
„reserved“ in the IANA TCP Header Flags registry, making this TCP flag
available for use by the AccECN experiment instead.“

Better?


In my understanding, this experimental document asks for new

assignment

of a reserved TCP header flag.

As I said I’m not sure if we have fully concluded this discussion yet.
However,
what we really would want to is mention somewhere that this

experiment

with this flags is running. I guess there are three options:
1) keep it in the registry as reserved and conserve the knowledge in tcpm
that this experiment is running and no other experimental RFC such use
this
flags as long as this experiment is running.
2) Keep is marked as reserved but add a note about this experiment in

the

IANA registry
3) Or assign it right away with IESG approval. I guess in this case tcpm
could
also consider to change the registration policy to „IETF Review“.

The current registration policy for the TCP header flags is "standards
action". I understand that the IESG could approve exceptions. But given
the policy, I believe the document has to be very precise on the request
regarding bit 7.


* 2.  AccECN Protocol Overview and Rationale

   o  an essential part that re-uses ECN TCP header bits to feed back
      the number of arriving CE marked packets.  This provides more
      accuracy than classic ECN feedback, but limited resilience
against
      ACK loss;

[ms] The word "re-use" is IMHO not correct.

I think this is nit picking. Using a different phrasing here makes the
sentence
unnecessary complicated. We don’t try to some how get a round the fact
that we need to handle the flag registration correctly. However, here the
point really is to explain how the protocol word. The main point of using
the
work „re-use“ here is really that we say that these flags are or have
been
used different by other TCP extension (and we therefore need a proper
negotiation scheme).

If the allocation of a reserved flag is correctly explained in the
abstract and introduction, I think these sentences can use a bit relaxed
terminology.


   The two part design was necessary, given limitations on the space
   available for TCP options and given the possibility that certain
   incorrectly designed middleboxes prevent TCP using any new options.

[ms] IMHO it would make sense to more explicitly mention the

downsides

of only specifying an option and not allocating a TCP header flag, in
this
experimental document.

We need to use the flags (all three of them) for the negotiation.

I think that an explanation why negotiation by a TCP option would not
solve all use cases (or requirements) would help here.


The obvious  alternative would be to postpone the header flag
allocation to

a follow-up standards track document and just keep it reserved.

We can still do that. We should discuss and make a final decision.


   The essential part overloads the previous definition of the three
   flags in the TCP header that had been assigned for use by ECN.  This
   design choice deliberately replaces the classic ECN feedback
   protocol, rather than leaving classic ECN feedback intact and adding
   more accurate feedback separately because:

[ms] Similar like previous comments, in my reading there are only _two_

ECN header flags.

I think there are three flags that "had been assigned for use by ECN“ as
ECN
Nonce is also an ECN mechanism. The fact that one of the flags is now
marked as reserved instead, it not that important for me here.


And, in addition, I think care is needed with wording such "replaces
the

classic ECN feedback". I don't think this experiment replaces the ECN
standards. That would be up to a follow-up PS.

This sentence is not meant to say that RFC3168 is replaced. Actually we
don’t.
You can still use RFC3168 even if AccECN is implemented and deploy
(however, we do intent that AccECN will be used as the default scheme in
future and RFC3168 is hopefully simply not needed anymore at some

point,

even though you probably still need to have it implemented as the
negotiation specified in this draft covers that as well, anyway...). The
sentence says that if AccECN is negotiation, the header flags as used by
RFC3168 and previously ECN Nonce are used differently (aka re-used).

That’s

all.


2.1.  Capability Negotiation

   AccECN is a change to the wire protocol of the main TCP header,
   therefore it can only be used if both endpoints have been upgraded
to
   understand it.  The TCP client signals support for AccECN on the
   initial SYN of a connection and the TCP server signals whether it
   supports AccECN on the SYN/ACK.  The TCP flags on the SYN that the
   client uses to signal AccECN support have been carefully chosen so
   that a TCP server will interpret them as a request to support the
   most recent variant of ECN feedback that it supports.  Then the
   client falls back to the same variant of ECN feedback.

[ms] As this is an experimental specification, I would really like to
see a

discussion how a future standards track version of more accurate ECN
could
be negotiated.

As described in this draft. There will be no different. AccECN IS and
will
always be backward compatible with RFC3168.

That is not the problem I think about. I wonder about a PS version of
accurate ECN feedback that would possibly include changes as compared to
this experiment, e.g., because the experiment may have some lessons
learnt. What options would we have to negotiate the PS version, and how
could a stack implementing the PS version figure out whether the remote
end uses the experimental or the PS version of the protocol?

The reason why I ask is because I don't see any easy solution to this but
I may miss something. Maybe this would not be a concern if there were

some

codepoints left.


How could both endpoints detect whether the other one implements

the

future standards track version?

If the initiator implements AccECN it will request it’s use. If the
receiver also
implement it, it will/can negotiate it, if not it will look like an
RFC3168 request
for the receiver (as the NS flags will be ignored in the SYN) and it will
negotiate RFC3168 ECN feedback if implemented. There is no additional
detection needed.

My concern is the migration strategy for a future version, given that we
only experiment right now. For instance, if the initiator implements the
experimental version but the recipient implements the PS version of
AccECN, how would that work? Under the assumption that there are

changes,

would there be a way to know? Maybe the answer to this question is no,
given the small number of bits we have. But not having any room for
extensions is not necessarily good protocol design.


For instance, would the only safe variant be that we allocate yet
another

reserved TCP header flag in a proposed standard to negotiate the
standards
track version, thus investing another reserved bit in the TCP header?

No, that’s exactly what we use the NS flags for in the handshake.

If the PS version of AccECN was different to the experimental version of
AccECN, and if there was a deployed base of both, I still believe we could
end up in a situation in which we had to allocate yet another TCP header
flag to distinguish the different versions of AccECN. The NS flag would
not work for that if it is used by the experimental version. The risk of
having to spend further TCP header flags somehow concerns me. Of

course,

that problem would only matter if the AccECN experiment succeeds

somehow,

but lessons learnt would require protocol changes, which is speculation.


I may be wrong, but to me it is too early to speculate how the PS
version

would look like, and whether it would have to be different to the
experimental version, due to lessons learnt.

Of course you can always be wrong. However, the handshake negotiation

is

not the part we need experimentation for. That part is straight forward
and
works. If we really happen to detect a problem in that part, we would
need
to end the experiment declare failure and start over new.

My concern is ending the experiment when the experiment got (partly)
deployed in the Internet. In that case neither a new RFC nor a change of
the IANA registry will solve the migration issue.


I believe in the IETF we typically design protocols that allow future

extension, and it is not exactly clear to be how AccECN could be extended
later.

This is an TCP extension. If we want future extension we use the usually
TCP
mechanism (by defining a new TCP option I guess).

The root cause of my concern is that this proposal does *not* use the
usual way to experiment with TCP by options. It experiments with a header
flag, including in the SYN, and it seems to consume all codepoints. So, I
see the risk of a protocol design "not ready for future improvements".

I cannot easily propose text. Maybe "lack of extensibility" is just one of
the short-comings of the protocol design that cannot be avoided but that
short-coming would have to be noted.


   An AccECN TCP client does not send the new AccECN Option on the

SYN

   as SYN option space is limited and successful negotiation using the
   flags in the main header is taken as sufficient evidence that both
   ends also support the AccECN Option.  The TCP server sends the
AccECN
   Option on the SYN/ACK and the client sends it on the first ACK to
   test whether the network path forwards the option correctly.

[ms] For what it is worth, I would personally be quite fine with
allowing (or

even mandating) an option in the SYN in the experimental version of this
protocol. For instance, saving the SYN option space would then an
excellent
reason for moving towards the PS specification. I am also fine with being
in
the rough part of the consensus here.

The point is that we really don’t need the option in the SYN as we don’t
use it
for negotiation purposes as we use the header bits instead. So why

should

we waste the space?

For instance, mandating the option in the SYN would be away for the
receiver to distinguish the experiment from a follow-up PS version of the
spec, as the PS version may not mandate the option, to save header space.

Maybe that proposal does not make any sense, and it may only have
downsides. But the document already speculates about a PS-follow-up. So

it

seems a valid question to ask if the EXP and the PS version of the spec
have to be identical. This all comes down to the SYN negotiation.


* 2.3.  Delayed ACKs and Resilience Against ACK Loss

   If the AccECN Option is not available, e.g. it is being stripped by
a
   middlebox, the AccECN protocol will only feed back information on CE
   markings (using the ACE field).  Although not ideal, this will be
   sufficient, because it is envisaged that neither ECT(0) nor ECT(1)
   will ever indicate more severe congestion than CE, even though
future
   uses for ECT(0) or ECT(1) are still unclear
   [I-D.ietf-tsvwg-ecn-experimentation].

[ms] This needs to be reworded

Why?




* 2.4.  Feedback Metrics

   The CE packet counter in the ACE field and the CE byte counter in
the
   AccECN Option both provide feedback on received CE-marks.  The CE
   packet counter includes control packets that do not have payload
   data, while the CE byte counter solely includes marked payload
bytes.
   If both are present, the byte counter in the option will provide the
   more accurate information needed for modern congestion control and
   policing schemes, such as DCTCP or ConEx.

[ms] I suggest to write in the last sentence only "... the option will
provide

the more accurate information needed for congestion control". In

general,

I
would prefer to have references to other mechanisms at only few (ideally
a
*single*) places in the document, instead of mixing them together.

Sorry, I don’t see your point here. ConEx has been mentioned previously,
so
why not also mention it here.

As written earlier, in my understanding DCTCP does not "need" this. I
would suggest to have one place where to define exactly how this

mechanism

can be used by other TCP extensions. Having said this, in this specific
paragraph I am less concerned about that than elsewhere.


   Feedback in bytes is recommended in order to protect against the
   receiver using attacks similar to 'ACK-Division' to artificially
   inflate the congestion window, which is why [RFC5681] now

recommends

   that TCP counts acknowledged bytes not packets.

[ms] At least the last part and the reference to RFC 5681 is IMHO not

needed here.

Why? RFC5681 explains/refers the ACK division attack, so I think it is a
very
good reference to have here.



* 2.5.  Generic (Dumb) Reflector

   The ACE field provides information about CE markings on both data
and
   control packets.  According to [RFC3168] the Data Sender is meant to
   set control packets to Not-ECT.  However, mechanisms in certain
   private networks (e.g. data centres) set control packets to be ECN
   capable because they are precisely the packets that performance
   depends on most.

   For this reason, AccECN is designed to be a generic reflector of
   whatever ECN markings it sees, whether or not they are compliant
with
   a current standard.  Then as standards evolve, Data Senders can
   upgrade unilaterally without any need for receivers to upgrade too.
   It is also useful to be able to rely on generic reflection behaviour
   when senders need to test for unexpected interference with markings
   (for instance [I-D.kuehlewind-tcpm-ecn-fallback] and
   [I-D.moncaster-tcpm-rcv-cheat]).

   The initial SYN is the most critical control packet, so AccECN
   provides feedback on whether it is CE marked.  Although RFC 3168
   prohibits an ECN-capable SYN, providing feedback of CE marking on
the
   SYN supports future scenarios in which SYNs might be ECN-enabled
   (without prejudging whether they ought to be).  For instance,
   [I-D.ietf-tsvwg-ecn-experimentation] updates this aspect of RFC 3168
   to allow experimentation with ECN-capable TCP control packets.

[ms] To me, the only thing that matters in this document that AccECN
can

provide feedback on whether the SYN is CE marked. The discussion on

how

to experiment with ECT e.g. in SYNs IMHO does not belong into this
document. So it seems sufficient here to note that one of the benefits of
AccECN is that CE marks in SYNs can be fed back.

I disagree. Explicitly saying that AccECN is only an feedback scheme and
DOES
NOT define how the information is used is VERY important because

people

come back to me over and over again and mix these things up.


* 3.1.1.  Negotiation during the TCP handshake

   Given the ECN Nonce [RFC3540] is being reclassified as historic, the
   present specification renames the TCP flag at bit 7 of the TCP
header
   flags from NS (Nonce Sum) to AE (Accurate ECN) (see IANA
   Considerations in Section 6).

[ms] As mentioned before, this needs to be rewritten to ask for new
IANA

allocation of bit 7 in the TCP header flags.

I really don’t understand this comment. That is what the IANA section
does
as referred here correctly.

In my reading of
https://www.iana.org/assignments/tcp-header-flags/tcp-header-

flags.xhtml,

this flag currently has no name, i.e., it does not have the name NS. So
this statement on "renaming" is formally incorrect IMHO.

I'd suggest something like: "This specification assigns the name AE
(Accurate ECN) to the TCP flag at bit 7; this flag has previously been
known as NS (Nonce Sum) ...".


Again, yes, we can discuss if this document should do this, but if
published as
it is, section 6 says everything that is needs to say. (I does not put
any
request on IANA, instead it is written as it would show up
post-publication,
implicitly providing a request to IANA at approval time. That is fine.)


   Table 2: ECN capability negotiation between Client (A) and Server
(B)

[ms] As far as I can see, in -05 this table allocates all existing
codepoints,

while -03 had two currently unused codepoints. Not having any

codepoints

left seems to me not really future proof, e.g., regarding future proposed
standards in this space (and I personally believe that TCP header flags
must
be allocated in a PS). And I don't fully see a need of feeding back ECT0
and
specifically ECT1 in the TCP header flags as part of the experiment. Do
we
know for sure that this is the only possible use case of these two
unallocated
header bits? And why can't e.g. this be done in a TCP option instead? Or
do I
miss something?

The point is really, if we don’t assign them now and start deployment we
effetely we not be able to every assign them again because don’t have a
different negotiation mechanisms. Realizing this, it is just the right
think to
define the space completely that is negotiated as use for AccECN in the
handshake.

I disagree that consuming all codepoints and not having room for future
extensions is the "right thing" to do. It is in fact the wrong thing. But
possibly there is no alternative with the few bits, so maybe we end up
doing it.

Yet, at minimum, using all codepoints needs to be reasoned in the
document. Specifically since in -03 a different protocol design seemed
possible, which looked to me more future-proof.


* 3.1.2.  Retransmission of the SYN

   However,
   current measurements imply that a drop is less likely to be due to
   middlebox interference than other intermittent causes of loss, e.g.
   congestion, wireless interference, etc.

[ms] Such wording IMHO doesn't belong into normative text. This may

actually also apply to other heuristics discussed in this section, which
are not
really important for interoperability.

I don’t really understand your point. This sentence is solely meant to
reason
the design decision to not say that a sender SHOULD attempt to re-
negotiation after a loss.

One could split the reasoning from the normative design. The normative
specification may still be relevant in 20 years from now. Middleboxes will
almost likely be different in 20 years. Having said this, this is more of
an editorial remark.


3.2.7.  Path Traversal of the AccECN Option

3.2.7.1.  Testing the AccECN Option during the Handshake

   The TCP client MUST NOT include the AccECN TCP Option on the SYN.

[ms] I am not sure if I really understand the motivation for not
allowing a

option in the SYN. If the sender has space in the SYN left, what is the
harm in
an experimental version of the protocol? And I may miss something, but
what would prevent the use of 2-byte option to negotiate the use of
AccECN, e.g., to avoid experimental allocation of bit 7 in the initial
SYN?

I did have a draft on that as a proposal for an alternative design.
However,
the gourd was more supportive of this design as it is proof of middlebox
SYN
option mangling which is a know problem.

Therefore we simply don’t need option in the SYN and there is no reason
to
waste the space.


While I think many tutorial text in this document could be shortened, I

believe the use of a reserved TCP header flag should be reasoned.

I’m actually uncertain what you expect here.

For instance, this reply with the explanation of SYN option mangling seems
useful explanation.


* 3.2.8.  Usage of the AccECN TCP Option

   The following rules determine when a Data Receiver in AccECN mode
   sends the AccECN TCP Option, and which fields to include:

   Change-Triggered ACKs:  If an arriving packet increments a different
      byte counter to that incremented by the previous packet, the Data
      Receiver MUST immediately send an ACK with an AccECN Option,
      without waiting for the next delayed ACK (this is in addition to
      the safety recommendation in Section 3.2.5 against ambiguity of
      the ACE field).

      This is stated as a "MUST" so that the data sender can rely on
      change-triggered ACKs to detect transitions right from the very
      start of a flow, without first having to detect whether the
      receiver complies.  A concern has been raised that certain
offload
      hardware needed for high performance might not be able to support
      change-triggered ACKs, although high performance protocols such
as
      DCTCP successfully use change-triggered ACKs.

[ms] To me this sounds like a perfect example for a SHOULD with
additional

guidance why implementing this SHOULD is really important.

This is one of the most discussed point from the author and we really
tried to
get additional guidance here of what to do also from outside the IETF but
did
no clear feedback.

As explained this MUST enables additional functionality. However, this is
an
experiment document. If we detect that this MUST does actually hinder
implementation or has just never been implemented, we should

reconsider

this in the final PS RFC.

I was on the SHOULD side of the discussion, but can say that the
implementation in Linux was way more simple then expected. Offloading
might be a different topic but that is where I could not get a clear
feedback
and offloading could probably be anyway optimized for use with AccECN

(if

it
gets deploy widely).

If a MUST requires changes in hardware, I think there must be a clear
reason.

As individual contributor, with the current explanation in the text, I
believe this has to be a SHOULD.


I though we added this as something to mention in the exp goals section,
but
obviously we didn’t. I added the following text now:

"Another experimentation focus is the implementation feasibiliy of
change-
triggered ACKs as described in section 3.2.8. While on average this
should not
lead to a higher ACK rate, it changes the ACK patter which especially can
have
an impact on hardware offload. Further experimentation is needed to
advise
if this should a hard requirement or just prefer behavior.“

If it is unclear if a MUST can actually be implemented, having a MUST is
in my opinion the wrong approach.

One could equally state here that further experimentation is needed to
determine whether the SHOULD can be upgraded to a MUST.


   For the avoidance of doubt, the change-triggered ACK mechanism is
   deliberately worded to ignore the arrival of a control packet with
no
   payload, which therefore does not alter any byte counters, because
it
   is important that TCP does not acknowledge pure ACKs.  The change-
   triggered ACK approach will lead to some additional ACKs but it
feeds
   back the timing and the order in which ECN marks are received with
   minimal additional complexity.

[ms] The additional acks create network load. I think some wording is

needed on the tradeoff between information accuracy and network load.
There are network environments in which any additional packet is very
expensive (e.g., energy) and it is not clear to me how the protocol
design
takes into account the potential overhead of additional ACKs. Maybe this
could be another reason for a SHOULD.

The above. However, this is not really an additional ACK because you do
delay the next one. Further experimentation needed.

The document states "lead to some additional ACKs". If that does not
increase network load, I think it has to be explicitly explained why the
ACK load is at most equal to a current TCP stack, in all potential cases.
If it can increases network load, it has to be reasoned why increasing
load (and risk of reverse congestion and the like) is worth the effort.

I agree that this may be an area of experimentation, but I believe then it
has to be explained to implementers what the tradeoffs are.


* 4.2.  Compatibility with Other TCP Options and Experiments

   AccECN is compatible (at least on paper) with the most commonly

used

   TCP options: MSS, time-stamp, window scaling, SACK and TCP-AO.  It
is
   also compatible with the recent promising experimental TCP options
   TCP Fast Open (TFO [RFC7413]) and Multipath TCP (MPTCP [RFC6824]).

[ms] I would suggest the wording "... compatible with the experimental

TCP options ..." or even "... compatible with the TCP options …".

These option are to experimental..?

"It is also compatible with the experimental TCP options TCP Fast Open
(TFO [RFC7413]) and Multipath TCP (MPTCP [RFC6824])." would have same
technical meaning. Having said this, this is editorial only.


The point of using „commonly used“ was to say that we checked on those

as

they seem important. Just because your favorite experiment option is not
listed here it doesn’t means it incompatible, we just didn’t check. I’m
okay to
removed „commonly used“ but I don’t think it makes anything better.


* 4.3.  Compatibility with Feedback Integrity Mechanisms

[ms] Quite a bit in this section is experimental work, which IMHO
should be clearly emphasized. The one exception is…

I would really like to keep this section because integrity is usually the
fist
question that come up when I present AccECN. Effectively these are two
independent topics, however, I really think it help people to understand
the
whole picture if this is also discussed in this document.


      However, TCP-AO is often too brittle to use on many end-to-end
      paths, where middleboxes can make verification fail in their
      attempts to improve performance or security, e.g. by
      resegmentation or shifting the sequence space.

[ms] I am not sure if deployment challenges of other options need to be

discussed in this document.

If we keep the discussion, I guess we should mention this as well. As the
doc
clearly stated section 4 is not meant to be normative.

As far as I can see, there are use cases for TCP-AO where middleboxes are
simply not a problem, but it is exactly this sort of discussion that may
not be needed in this document. But I won't rat-hole on this comment

here.


          

            

              
   Originally the ECN Nonce [RFC3540] was proposed to ensure integrity
   of congestion feedback.  With minor changes AccECN could be
optimised
   for the possibility that the ECT(1) codepoint might be used as an
ECN
   Nonce . However, given RFC 3540 is being reclassified as historic,
   the AccECN design has been generalised so that it ought to be able
to
   support other possible uses of the ECT(1) codepoint, such as a lower
   severity or a more instant congestion signal than CE.

[ms] The discussion of RFC 3540 can probably be removed to a large
extent.


            

            
Unfortunately, please still think that ECN Nonce, even though is was
never
deployed and doesn’t really work, is the only was to provide integrity
protection and we need it as a prerequisite to deploy ECN at all… i would
really prefer to keep this in this non-normative part of the doc.



            

              

* 6.  IANA Considerations

[ms] I think this section needs to be rewritten to request a new
allocation


            

            
of bit 7 of the TCP header flags. At least for the process I think it
would make
sense to have somewhere in the document a comprehensive explanation


          

        

        
of


        

          

            
why an experimental document requests a change of the main TCP


          

        

        
header,


        

          

            
and why this cannot be avoided (most notably in the initial SYN) by an
alternative protocol design.

As I said previously this section is written as it would look like after
the
allocation has happened with publication approval of the IESG. This is
fine.

Having a discussion about an experiment doc assigning a flag (or not) is
a
question for tcpm as a whole and not specifically this document. How do
we
envision to every use any further flags? We go to PS right away? Or
should
we change the registration policy? For me the latter makes actually more
sense. However, if we don’t want/can to decide this now, we also could


          

        

        
go


        

          

            
forward as it is with IESG approval. However, is this case it is also not
needed
to explain this in the document. The responsible AD has to explain this
to the
other IESG probably in the ballot or even better the shepherd could
provide
these information in the write-up.



            

              

* 9.  Comments Solicited

   Comments and questions are encouraged and very welcome.  They


            

          

        

        
can


        

          

            
be


            

              
   addressed to the IETF TCP maintenance and minor modifications
working
   group mailing list <tcpm@ietf.org>, and/or to the authors.

[ms] This section is not needed IMHO


            

            
Yes, it will be removed before publication.


            

              

10.  References

   [I-D.ietf-tsvwg-ecn-experimentation]
              Black, D., "Relaxing Restrictions on Explicit Congestion
              Notification (ECN) Experimentation",
draft-ietf-tsvwg-ecn-
              experimentation-07 (work in progress), October 2017.

[ms] Normative reference?


            

            
Don’t see why. No need to read ietf-tsvwg-ecn-experimentation to
understand the spec in this doc.

Thanks!
Mirja



          

          
_______________________________________________
tcpm mailing list
tcpm@ietf.org
https://www.ietf.org/mailman/listinfo/tcpm


---
This email has been checked for viruses by AVG.
http://www.avg.com



        

      

      
_______________________________________________
tcpm mailing list
tcpm@ietf.org
https://www.ietf.org/mailman/listinfo/tcpm


    

    

    
-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/

  


--------------29C23CFC30CF5D948F19EE0A--


From nobody Sun Apr  8 12:56:05 2018
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 439AE126CBF for ; Sun,  8 Apr 2018 12:56:04 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.99
X-Spam-Level: 
X-Spam-Status: No, score=-1.99 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, T_SPF_PERMERROR=0.01] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=strayalpha.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id F9jHL8GtTlaK for ; Sun,  8 Apr 2018 12:56:01 -0700 (PDT)
Received: from server217-3.web-hosting.com (server217-3.web-hosting.com [198.54.115.226]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 894C5126BF7 for ; Sun,  8 Apr 2018 12:56:01 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=strayalpha.com; s=default; h=To:References:Message-Id: Content-Transfer-Encoding:Cc:Date:In-Reply-To:From:Subject:Mime-Version: Content-Type:Sender:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=62dhCixsPl8c5sCqgzC+4CrzbvFGToZLHgk/QxxP33k=; b=KPyEZqsoplmgne6fEw4PcOM7W y/gmfVv0KlEIXj9oz6bvflNpPx3E46oW2zZOwoiCtSv1nr9/CfnAoaAjaWrXdJtbklPs250Tt+ZC5 q/63zLkuRh1NdroU80scKoWXot4A11JrUWNqsIoU4JorRn3Yxk6pjghSOC4f+3Y9caJOfcWYNYX8/ JmCZ7+JckLVjbR952ohMpxBom7wOjZKPG5rgdHVPBE/VwwwSGyGeKQaTBO6XnbEJpNUkiLolDDLR2 st81imoLNuGf6GDbkFwmS0UULU3F+THOpyE0N1hEXRtYd1Jn3NH7ECcN+YKqz1yy0GLQ57MXK811W l09bmJJUA==;
Received: from [12.129.159.194] (port=3554 helo=[100.69.233.81]) by server217.web-hosting.com with esmtpsa (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.89_1) (envelope-from ) id 1f5GPv-0027Rw-3V; Sun, 08 Apr 2018 15:55:59 -0400
Content-Type: text/plain; charset=us-ascii
Mime-Version: 1.0 (1.0)
From: Joe Touch 
X-Mailer: iPad Mail (15E216)
In-Reply-To: 
Date: Sun, 8 Apr 2018 12:55:58 -0700
Cc: Michael Welzl , tcpm IETF list , Matt Mathis , draft-ietf-tcpm-rack@tools.ietf.org
Content-Transfer-Encoding: quoted-printable
Message-Id: 
References: <152029151070.12715.15497200993114582811@ietfa.amsl.com>   <1c5bef94-eb4f-7aab-0b6b-cb2d02da7a9a@bobbriscoe.net>  <6a451b96-af61-f5c8-87a9-9968e99027de@bobbriscoe.net>  <1b5f83b1-d38c-d18c-b259-9817ae8f226c@bobbriscoe.net> <1D6E4D94-643D-497F-87E5-D4F8C72EA639@ifi.uio.no> 
To: Bob Briscoe 
X-OutGoing-Spam-Status: No, score=-1.0
X-AntiAbuse: This header was added to track abuse, please include it with any abuse report
X-AntiAbuse: Primary Hostname - server217.web-hosting.com
X-AntiAbuse: Original Domain - ietf.org
X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12]
X-AntiAbuse: Sender Address Domain - strayalpha.com
X-Get-Message-Sender-Via: server217.web-hosting.com: authenticated_id: touch@strayalpha.com
X-Authenticated-Sender: server217.web-hosting.com: touch@strayalpha.com
X-Source: 
X-Source-Args: 
X-Source-Dir: 
X-From-Rewrite: unmodified, already matched
Archived-At: 
Subject: Re: [tcpm] Vicious or Virtuous circle? Adapting reordering window to reordering degree
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Sun, 08 Apr 2018 19:56:04 -0000

> On Apr 6, 2018, at 1:55 PM, Bob Briscoe  wrote:
>=20
> My Note 1 merely pointed out that, in an ideal world with ECN and FEC, ret=
ransmission would not be needed to repair losses,

ECN + FEC is not a guarantee of E2E transmission. Packets are lost for reaso=
ns other than congestion, and no amount of FEC overcomes losses that exceed w=
hat that FEC anticipates.

Joe=


From nobody Mon Apr 16 08:42:18 2018
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D799412E8A0 for ; Mon, 16 Apr 2018 08:42:17 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -4.2
X-Spam-Level: 
X-Spam-Status: No, score=-4.2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gdsISR-HAxA4 for ; Mon, 16 Apr 2018 08:42:14 -0700 (PDT)
Received: from pegasus.erg.abdn.ac.uk (pegasus.erg.abdn.ac.uk [139.133.204.173]) by ietfa.amsl.com (Postfix) with ESMTP id 9A70612E899 for ; Mon, 16 Apr 2018 08:42:14 -0700 (PDT)
Received: from dhcp-207-156.erg.abdn.ac.uk (unknown [IPv6:2001:630:241:207:1485:1e83:c2bf:3e66]) by pegasus.erg.abdn.ac.uk (Postfix) with ESMTPSA id 8518E1B001B9; Mon, 16 Apr 2018 16:42:12 +0100 (BST)
Reply-To: gorry@erg.abdn.ac.uk
To: Mikael Abrahamsson , Toerless Eckert 
Cc: tcpm@ietf.org
References: <20180406012736.fpayup7perbhqn2p@faui48f.informatik.uni-erlangen.de> 
From: Gorry Fairhurst 
Organization: The University of Aberdeen is a charity registered in Scotland,  No SC013683.
Message-ID: 
Date: Mon, 16 Apr 2018 16:42:12 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:52.0) Gecko/20100101 Thunderbird/52.6.0
MIME-Version: 1.0
In-Reply-To: 
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Archived-At: 
Subject: Re: [tcpm] Better QoS for TCP ACK question
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Mon, 16 Apr 2018 15:42:18 -0000

On 07/04/2018 06:10, Mikael Abrahamsson wrote:
> On Fri, 6 Apr 2018, Toerless Eckert wrote:
> 
>> A couple of years ago i was told that a range of home routers
>> would do some DPI based diffserv QoS, aka prioritizing TCP ACP
>> messages in the queue before any other packets 9sonded like
>> a strict priority queue for these packets). The goal was to
>> improve performance when you had separate up and downstream TCP
>> connections, especially when the upstream TCP connection delayed
>> the downstream TCP connections ACKs.
>>
>> I never could find a lot of good details about this, e.g.: quantitiative
>> measurements, so i was wondering if others here on the list
>> are aware of it. Of course, if something like this would
>> be significantly useful even in the face of current TCP congestion
>> control mechanisms (and not only bad older ones).
> 
> I know some people who have run two queues, one for packets less than 
> 200 byte in size, and one for larger packets. They didn't even do any 
> DPI, the setup was purely based on size of the packet. I am not aware of 
> any deeper analysis on pros/cons of this setup, but these people were 
> very happy with their setups for years and it solved their problems with 
> a single large file transfer ruining their interactive performance, plus 
> also "solved" the ACKs-get-stuck-behind-other-session-large-packets 
> problem.
> 
> This was used for platforms with otherwise quite limited QoS 
> capabilities, so flow based queueing wasn't available.
> 
Just to send something more to this list thread:

RFC 3449 from 2002 describes this in section 5.4.2 on ACKs-first 
Scheduling.

I aware this method was implemented in a variety of equipment around 
that time, although I suspect it can also cause some odd performance 
hits when using app-limited TCP sessions (where many data segments are 
also small and may also become reordered).

Gorry


From nobody Mon Apr 16 09:45:22 2018
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2084E124207 for ; Mon, 16 Apr 2018 09:45:20 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -3.95
X-Spam-Level: 
X-Spam-Status: No, score=-3.95 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HEADER_FROM_DIFFERENT_DOMAINS=0.25, RCVD_IN_DNSWL_MED=-2.3] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ry8_qsb13-Ql for ; Mon, 16 Apr 2018 09:45:17 -0700 (PDT)
Received: from faui40.informatik.uni-erlangen.de (faui40.informatik.uni-erlangen.de [IPv6:2001:638:a000:4134::ffff:40]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id C4F2B12DA13 for ; Mon, 16 Apr 2018 09:45:17 -0700 (PDT)
Received: from faui48f.informatik.uni-erlangen.de (faui48f.informatik.uni-erlangen.de [IPv6:2001:638:a000:4134::ffff:52]) by faui40.informatik.uni-erlangen.de (Postfix) with ESMTP id 8273658C4B1; Mon, 16 Apr 2018 18:45:13 +0200 (CEST)
Received: by faui48f.informatik.uni-erlangen.de (Postfix, from userid 10463) id 7584F440214; Mon, 16 Apr 2018 18:45:13 +0200 (CEST)
Date: Mon, 16 Apr 2018 18:45:13 +0200
From: Toerless Eckert 
To: Gorry Fairhurst 
Cc: Mikael Abrahamsson , tcpm@ietf.org
Message-ID: <20180416164513.wu4wwzuw5fgslrgt@faui48f.informatik.uni-erlangen.de>
References: <20180406012736.fpayup7perbhqn2p@faui48f.informatik.uni-erlangen.de>  
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: 
User-Agent: NeoMutt/20170113 (1.7.2)
Archived-At: 
Subject: Re: [tcpm] Better QoS for TCP ACK question
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Mon, 16 Apr 2018 16:45:20 -0000

On Mon, Apr 16, 2018 at 04:42:12PM +0100, Gorry Fairhurst wrote:
> Just to send something more to this list thread:
> 
> RFC 3449 from 2002 describes this in section 5.4.2 on ACKs-first Scheduling.

Ah, great. Thanks. Something to point people to. 

> I aware this method was implemented in a variety of equipment around that
> time, although I suspect it can also cause some odd performance hits when
> using app-limited TCP sessions (where many data segments are also small and
> may also become reordered).
> Gorry

I guess this would refer not to an actual TCP-ACK DPI based
scheduling that you could easily do on CPU processing home
gateways, but rather the packet-length based queuing Mikael
brought up ? Otherwise i don't understand.

Toerless


From nobody Mon Apr 16 09:53:33 2018
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A23EB126BF6 for ; Mon, 16 Apr 2018 09:53:32 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.9
X-Spam-Level: 
X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7VVOJJ38lIX4 for ; Mon, 16 Apr 2018 09:53:30 -0700 (PDT)
Received: from pegasus.erg.abdn.ac.uk (pegasus.erg.abdn.ac.uk [IPv6:2001:630:241:204::f0f0]) by ietfa.amsl.com (Postfix) with ESMTP id 2FA1B124207 for ; Mon, 16 Apr 2018 09:53:30 -0700 (PDT)
Received: from dhcp-207-156.erg.abdn.ac.uk (unknown [IPv6:2001:630:241:207:1485:1e83:c2bf:3e66]) by pegasus.erg.abdn.ac.uk (Postfix) with ESMTPSA id 2BDBC1B001B9; Mon, 16 Apr 2018 17:53:28 +0100 (BST)
Reply-To: gorry@erg.abdn.ac.uk
To: Toerless Eckert 
Cc: Mikael Abrahamsson , tcpm@ietf.org
References: <20180406012736.fpayup7perbhqn2p@faui48f.informatik.uni-erlangen.de>   <20180416164513.wu4wwzuw5fgslrgt@faui48f.informatik.uni-erlangen.de>
From: Gorry Fairhurst 
Organization: The University of Aberdeen is a charity registered in Scotland,  No SC013683.
Message-ID: 
Date: Mon, 16 Apr 2018 17:53:27 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:52.0) Gecko/20100101 Thunderbird/52.6.0
MIME-Version: 1.0
In-Reply-To: <20180416164513.wu4wwzuw5fgslrgt@faui48f.informatik.uni-erlangen.de>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Archived-At: 
Subject: Re: [tcpm] Better QoS for TCP ACK question
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Mon, 16 Apr 2018 16:53:32 -0000

On 16/04/2018 17:45, Toerless Eckert wrote:
> On Mon, Apr 16, 2018 at 04:42:12PM +0100, Gorry Fairhurst wrote:
>> Just to send something more to this list thread:
>>
>> RFC 3449 from 2002 describes this in section 5.4.2 on ACKs-first Scheduling.
> 
> Ah, great. Thanks. Something to point people to.
> 
>> I aware this method was implemented in a variety of equipment around that
>> time, although I suspect it can also cause some odd performance hits when
>> using app-limited TCP sessions (where many data segments are also small and
>> may also become reordered).
>> Gorry
> 
> I guess this would refer not to an actual TCP-ACK DPI based
> scheduling that you could easily do on CPU processing home
> gateways, but rather the packet-length based queuing Mikael
> brought up ? Otherwise i don't understand.
> 
> Toerless
> 
In those ancient days, I suspect most systems did do the "ACK-First" 
scheduling based only on estimating the size of the "ACK".

So I'm not saying TCP DPI-based could do special stuff when it sees data 
in the packet and try to smartly avoid reordering any segments, and that 
timestamps, SACK, and various other things could be taken into account. 
Whether this would be a great idea is another story.

Gorry



From nobody Tue Apr 17 11:56:08 2018
Return-Path: 
X-Original-To: tcpm@ietf.org
Delivered-To: tcpm@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id BD96C12D95F; Tue, 17 Apr 2018 11:56:05 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: IETF Meeting Session Request Tool 
To: 
Cc: michael.scharf@nokia.com, tcpm@ietf.org, ietf@kuehlewind.net, tcpm-chairs@ietf.org
X-Test-IDTracker: no
X-IETF-IDTracker: 6.78.0
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <152399136572.11523.8143327864621358279.idtracker@ietfa.amsl.com>
Date: Tue, 17 Apr 2018 11:56:05 -0700
Archived-At: 
Subject: [tcpm] tcpm - New Meeting Session Request for IETF 102
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.22
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Tue, 17 Apr 2018 18:56:06 -0000

A new meeting session request has just been submitted by Michael Scharf, a Chair of the tcpm working group.


---------------------------------------------------------
Working Group Name: TCP Maintenance and Minor Extensions
Area Name: Transport Area
Session Requester: Michael Scharf

Number of Sessions: 1
Length of Session(s):  2.5 Hours
Number of Attendees: 80
Conflicts to Avoid: 
 First Priority: iccrg tcpinc mptcp taps tsvarea tsvwg quic
 Second Priority: httpbis lwig rmcat teas
 Third Priority: rtcweb maprg panrg


People who must be present:
  Yoshifumi Nishida
  Michael Tuexen
  Michael Scharf
  Mirja Kuehlewind

Resources Requested:

Special Requests:
  
---------------------------------------------------------