From nobody Mon Aug 1 09:58:07 2016 Return-Path: X-Original-To: tcpm@ietf.org Delivered-To: tcpm@ietfa.amsl.com Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id E3DD912B036; Mon, 1 Aug 2016 09:58:05 -0700 (PDT) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit From: internet-drafts@ietf.org To: X-Test-IDTracker: no X-IETF-IDTracker: 6.29.0 Auto-Submitted: auto-generated Precedence: bulk Message-ID: <20160801165805.32516.53201.idtracker@ietfa.amsl.com> Date: Mon, 01 Aug 2016 09:58:05 -0700 Archived-At: Cc: tcpm@ietf.org Subject: [tcpm] I-D Action: draft-ietf-tcpm-rfc793bis-03.txt X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Aug 2016 16:58:06 -0000 A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the TCP Maintenance and Minor Extensions of the IETF. Title : Transmission Control Protocol Specification Author : Wesley M. Eddy Filename : draft-ietf-tcpm-rfc793bis-03.txt Pages : 90 Date : 2016-08-01 Abstract: This document specifies the Internet's Transmission Control Protocol (TCP). TCP is an important transport layer protocol in the Internet stack, and has continuously evolved over decades of use and growth of the Internet. Over this time, a number of changes have been made to TCP as it was specified in RFC 793, though these have only been documented in a piecemeal fashion. This document collects and brings those changes together with the protocol specification from RFC 793. This document obsoletes RFC 793 and several other RFCs (TODO: list all actual RFCs when finished). RFC EDITOR NOTE: If approved for publication as an RFC, this should be marked additionally as "STD: 7" and replace RFC 793 in that role. The IETF datatracker status page for this draft is: https://datatracker.ietf.org/doc/draft-ietf-tcpm-rfc793bis/ There's also a htmlized version available at: https://tools.ietf.org/html/draft-ietf-tcpm-rfc793bis-03 A diff from the previous version is available at: https://www.ietf.org/rfcdiff?url2=draft-ietf-tcpm-rfc793bis-03 Please note that it may take a couple of minutes from the time of submission until the htmlized version and diff are available at tools.ietf.org. Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ From nobody Mon Aug 1 10:03:25 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 17F9A12D1B7 for ; Mon, 1 Aug 2016 10:03:25 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.601 X-Spam-Level: X-Spam-Status: No, score=-2.601 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=mti-systems-com.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id aNeA-s3GSlyb for ; Mon, 1 Aug 2016 10:03:23 -0700 (PDT) Received: from mail-io0-x234.google.com (mail-io0-x234.google.com [IPv6:2607:f8b0:4001:c06::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 00F2312B046 for ; Mon, 1 Aug 2016 10:03:22 -0700 (PDT) Received: by mail-io0-x234.google.com with SMTP id b62so188722588iod.3 for ; Mon, 01 Aug 2016 10:03:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mti-systems-com.20150623.gappssmtp.com; s=20150623; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to:content-transfer-encoding; bh=GtWYXxPG8yoiExwJ4fQUt9RiQbnMw19Mm0UkUB39oJ0=; b=xtD0nSV3RW+S4Zalz8mn/07L1IquN37H41QzkqicyNvZX53dft9pmvw2ocjZuyG1gQ v536BQGjvx5TLWD0iUFYuRqwRlItNafP/nAP0SxiREoj/LSnA76dtnL8mk1ztmWmQXi2 RyGsqt4lRUvPGhKyT4pLsTwHX/0jiBpPZU/pFykgfGV41LwECMLkiHnVGKPIaTE41Oxe ZnhW6x/Za8D0pUV7FUIP8WMRtoHMU6CeOlIOaoEBj2Qe697qaG/Yv4gVTR3n2D8ReLBQ YpOZomZ/yQsU7pcH+vBQZZE0+UGiPBxTW2Cg1EILUvXFOlIEx99GEJLw+Vghy2zXwacF ZiMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=GtWYXxPG8yoiExwJ4fQUt9RiQbnMw19Mm0UkUB39oJ0=; b=B0Hgu2G9+179EYcXGYIUjJ3vtk/iN3fc8UfaeBqsodU4FKr9qXrDZBBpNs5pPBkLAm sIjlvBmbtGcd4tSmQX6QL7gTUeGOcIr7yvLBUW1zdb5iZjvbZ81MjBe+krmnsVIfFpeV O16D2zq17LSbbU4v8RBji32/MS16zLUL4hBnD+w+7/xXUAhOnLYFfzC4v6183+O3Jq/S 0kMacOpx1RNEzOgfaT7j/S7v7y71e1bTf2mLmn7n0J7n1rVaULldq7xHWIKCETPyH/Zj Y4VjEx3/XPv/nnbM9ScO1OOcLaFBzBS3j11FOJ3CuKlhsyTH/BoXqchT8iTj1r6rXqVp BR6A== X-Gm-Message-State: AEkoouswhtNQvVu3rvbC3KjlANimSBY77Mos9v51UGsIuDD5CPTBpPOOwvI6xChrdFfThw== X-Received: by 10.107.137.150 with SMTP id t22mr57087960ioi.20.1470071001704; Mon, 01 Aug 2016 10:03:21 -0700 (PDT) Received: from [192.168.1.104] (cpe-76-188-215-129.neo.res.rr.com. [76.188.215.129]) by smtp.gmail.com with ESMTPSA id j129sm8171568itj.5.2016.08.01.10.03.20 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 01 Aug 2016 10:03:21 -0700 (PDT) To: tcpm@ietf.org References: <20160801165805.32516.53201.idtracker@ietfa.amsl.com> From: Wesley Eddy Message-ID: <293cc8e7-8db4-0d27-5c4a-97f2812dddcf@mti-systems.com> Date: Mon, 1 Aug 2016 13:03:21 -0400 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: <20160801165805.32516.53201.idtracker@ietfa.amsl.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Archived-At: Subject: Re: [tcpm] I-D Action: draft-ietf-tcpm-rfc793bis-03.txt X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Aug 2016 17:03:25 -0000 There are some things in this draft that could use WG feedback. I'll start separate threads shortly rather than expect everyone to digest the diffs and figure it out! On 8/1/2016 12:58 PM, internet-drafts@ietf.org wrote: > A New Internet-Draft is available from the on-line Internet-Drafts directories. > This draft is a work item of the TCP Maintenance and Minor Extensions of the IETF. > > Title : Transmission Control Protocol Specification > Author : Wesley M. Eddy > Filename : draft-ietf-tcpm-rfc793bis-03.txt > Pages : 90 > Date : 2016-08-01 > > Abstract: > This document specifies the Internet's Transmission Control Protocol > (TCP). TCP is an important transport layer protocol in the Internet > stack, and has continuously evolved over decades of use and growth of > the Internet. Over this time, a number of changes have been made to > TCP as it was specified in RFC 793, though these have only been > documented in a piecemeal fashion. This document collects and brings > those changes together with the protocol specification from RFC 793. > This document obsoletes RFC 793 and several other RFCs (TODO: list > all actual RFCs when finished). > > RFC EDITOR NOTE: If approved for publication as an RFC, this should > be marked additionally as "STD: 7" and replace RFC 793 in that role. > > > > The IETF datatracker status page for this draft is: > https://datatracker.ietf.org/doc/draft-ietf-tcpm-rfc793bis/ > > There's also a htmlized version available at: > https://tools.ietf.org/html/draft-ietf-tcpm-rfc793bis-03 > > A diff from the previous version is available at: > https://www.ietf.org/rfcdiff?url2=draft-ietf-tcpm-rfc793bis-03 > > > Please note that it may take a couple of minutes from the time of submission > until the htmlized version and diff are available at tools.ietf.org. > > Internet-Drafts are also available by anonymous FTP at: > ftp://ftp.ietf.org/internet-drafts/ > > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm From nobody Mon Aug 1 20:52:54 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id BBA5B12D10D; Mon, 1 Aug 2016 20:52:47 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.699 X-Spam-Level: X-Spam-Status: No, score=-2.699 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id IL8yV3clPIK8; Mon, 1 Aug 2016 20:52:45 -0700 (PDT) Received: from mail-it0-x233.google.com (mail-it0-x233.google.com [IPv6:2607:f8b0:4001:c0b::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 3A99712B05E; Mon, 1 Aug 2016 20:52:45 -0700 (PDT) Received: by mail-it0-x233.google.com with SMTP id f6so189535835ith.0; Mon, 01 Aug 2016 20:52:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=l8diwhjPT6Vy60supX6vWSw8wbNCziXJt9ISImwHa8Q=; b=XBUNeC3HafUl/MRO5SpBGiAeChTV6rkNAd8XPSv3WigX6PRzuFMx3XElsadqIFPSxB wxWgng+PoOzg4IqjJj4vr8o+r/gW5VhuYL8VCwUmaCBIiYA0AIzKh9TK5vz5vsV8sVBM SUOav3LyUM0MFMHGL0KH7d/OBpWnaXoCpYMj0j1SIGuiqF3k01SKs75s8/yKCjVu/k5P DtfCF4xEnLY8Y14cP0ZKGMMGeC7ijfZ7J15c2Khjg95gVnS1fdxCiKDGEg3ax+04syax Ea8OZgLZ1DyQOrEJVchrP332jtlzjthdYpZU+55SOcZawLum5DNHa6iHgqZTrRzF5LUX kpZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=l8diwhjPT6Vy60supX6vWSw8wbNCziXJt9ISImwHa8Q=; b=YMK2SRqsIrdqBSo6RpcGMT0KD1anuvPnmUEuiGGTUCN8P+ySyt5v/AlUhVUWT8T6k0 AG5BkNjytFzyTJgbnN7Mph19WGrNjrYSYfg1988R84iTMhSYEH8KAu3FO9puXwLZg0E0 EOKKBBlB+qes7JnV96YVMVivQSV3uQyyelb0cCp9/ONQKd+M1XAcu+vt8sAjsskpUvX7 /WMyGly9dF1H2v0fsKiEOMQ0MIcMuIG7+9u6Kb+5JVpjLIK6rMVqpRkarO/UBW2fp1pM wAvT7w9MrDZL6HheD1GRb9YX/AhtmeVpg3pua8YvjPOeUwptQFhy2pg4ZTAY5MipI0JY WOMQ== X-Gm-Message-State: AEkoouvyH7VchZCx9DL+rWovJFYRo6beCLqiDp0adwSeGoxyx49RR8htMtaac89ZnPBYO5yDSOWM8F/o+NTwaQ== X-Received: by 10.36.155.213 with SMTP id o204mr45034067itd.96.1470109964454; Mon, 01 Aug 2016 20:52:44 -0700 (PDT) MIME-Version: 1.0 Received: by 10.107.14.16 with HTTP; Mon, 1 Aug 2016 20:52:43 -0700 (PDT) In-Reply-To: <20160728063754.GA24657@cowbell.employees.org> References: <87wpk7x9v6.fsf@ta.scs.stanford.edu> <877fc6ycuw.fsf@ta.scs.stanford.edu> <20160727232419.GA45841@cowbell.employees.org> <20160728063754.GA24657@cowbell.employees.org> From: Gavin McCullagh Date: Mon, 1 Aug 2016 20:52:43 -0700 Message-ID: To: Derek Fawcus Content-Type: multipart/alternative; boundary=94eb2c05fb2036513805390ea6d1 Archived-At: Cc: tcpinc , "tcpm@ietf.org Extensions" , "Mirja Kuehlewind \(IETF\)" , Kyle Rose , David Mazieres expires 2016-10-25 PDT Subject: Re: [tcpm] [tcpinc] TCP's treatment of data in SYN packets X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2016 03:52:48 -0000 --94eb2c05fb2036513805390ea6d1 Content-Type: text/plain; charset=UTF-8 Just to satisfy my curiosity, how did this work (or did it?) with TCP SYN Cookies? I can imagine a tcp listener which was implementing SYN Cookies only ACKing the initial SYN sequence number and not the data which it discarded, then the application retransmitting that data either on the subsequent ACK or the next data packet. Does that generally just work with most/all tcp implementations on the other side? I suppose it ought to? Gavin On Wed, Jul 27, 2016 at 11:37 PM, Derek Fawcus < dfawcus+lists-tcpcrypt@employees.org> wrote: > On Thu, Jul 28, 2016 at 12:24:19am +0100, Derek Fawcus wrote: > > > > Implying that a passive opener which chose to ack such data bytes would > > have to buffer such data until the handshake completes; but I'm not > > aware of any stack which actually works that way. > > Except the old KA9Q NOS, which was tickling my memory... > > It did have the ability to send such SYN data. > > DF > > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm > --94eb2c05fb2036513805390ea6d1 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Just to satisfy my curiosity, how did this work = (or did it?) with TCP SYN Cookies?

I can imagine a tcp listen= er which was implementing SYN Cookies only ACKing the initial SYN sequence = number and not the data which it discarded, then the application retransmit= ting that data either on the subsequent ACK or the next data packet.=C2=A0 = Does that generally just work with most/all tcp implementations on the othe= r side?=C2=A0 I suppose it ought to?

Gavin

On Wed, Jul 27, 2016 at 11:= 37 PM, Derek Fawcus <dfawcus+lists-tcpcrypt@employees.o= rg> wrote:
On Thu, Jul 28, 2016 at 12:24:19am +0100, Derek Fawcus wrote:
>
> Implying that a passive opener which chose to ack such data bytes woul= d
> have to buffer such data until the handshake completes;=C2=A0 but I= 9;m not
> aware of any stack which actually works that way.

Except the old KA9Q NOS,=C2=A0 which was tickling my memory...

It did have the ability to send such SYN data.

DF

_______________________________________________
tcpm mailing list
tcpm@ietf.org
https://www.ietf.org/mailman/listinfo/tcpm

--94eb2c05fb2036513805390ea6d1-- From nobody Tue Aug 2 01:24:30 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3632C12B035; Tue, 2 Aug 2016 01:24:29 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.566 X-Spam-Level: X-Spam-Status: No, score=-0.566 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_SORBS_WEB=0.77, SPF_HELO_PASS=-0.001, SPF_SOFTFAIL=0.665] autolearn=no autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=employees.org; domainkeys=pass (1024-bit key) header.from=dfawcus+lists-tcpcrypt@employees.org header.d=employees.org Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id eCSf2SxEx5qL; Tue, 2 Aug 2016 01:24:28 -0700 (PDT) Received: from incoming.kjsl.com (inbound02.kjsl.com [IPv6:2001:1868:2002::144]) (using TLSv1.2 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id E4D4612B004; Tue, 2 Aug 2016 01:24:27 -0700 (PDT) Received: from cowbell.employees.org ([65.50.211.142]) by ironport02.kjsl.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 02 Aug 2016 08:24:26 +0000 Received: from cowbell.employees.org (localhost [127.0.0.1]) by cowbell.employees.org (Postfix) with ESMTP id E50FE9CC82; Tue, 2 Aug 2016 01:24:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=employees.org; h=date:from :to:cc:subject:message-id:references:mime-version:content-type :in-reply-to; s=selector1; bh=hwRvknjKwqA+R1ITNIYhqzTRwj0=; b=Tf cuPE0mbb9BjiOQV+P36Fa8ZkLII/ZS2MVT7W+NYh4ZzVWZkiKlE4zyq4cj62TI4E FXGfYx1W/OcKdQU+Mx4OgMAAqCDMAKLwejtJBSQvW5qLPOgJmUk3kdadgaSc+YvA vljf3CYuJDTqG4ty5YkN2NGoRZ5287czyO1sxMX7E= DomainKey-Signature: a=rsa-sha1; c=nofws; d=employees.org; h=date:from :to:cc:subject:message-id:references:mime-version:content-type :in-reply-to; q=dns; s=selector1; b=cReV6VGYPTUkSnBH9/Mz5ZrlNS3J 68Wdi+vvnNb0RJ5f75tUP8tu8AJ66yk1wPPVxIjqk8Q1KCSzoWGsMIs9tJSGC30t 6rb5cfn5XBD0bVcgTwOVlHmksCc/HKWAkv2Swf9IR+/nyFHcDdw6f6/gBWaVCxp+ Oq2jTnybYs4LrMA= Received: by cowbell.employees.org (Postfix, from userid 1736) id D1DC79CC81; Tue, 2 Aug 2016 01:24:25 -0700 (PDT) Date: Tue, 2 Aug 2016 09:24:25 +0100 From: Derek Fawcus To: Gavin McCullagh Message-ID: <20160802082425.GA9117@cowbell.employees.org> References: <87wpk7x9v6.fsf@ta.scs.stanford.edu> <877fc6ycuw.fsf@ta.scs.stanford.edu> <20160727232419.GA45841@cowbell.employees.org> <20160728063754.GA24657@cowbell.employees.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.6.1 (2016-04-27) Archived-At: Cc: tcpinc , "tcpm@ietf.org Extensions" Subject: Re: [tcpm] [tcpinc] TCP's treatment of data in SYN packets X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2016 08:24:29 -0000 On Mon, Aug 01, 2016 at 08:52:43pm -0700, Gavin McCullagh wrote: > Just to satisfy my curiosity, how did this work (or did it?) with TCP SYN Cookies? The sender of the SYN would simply send the data again once the 3whs completed. This was before SYN cookies were invented. As I recall, I'd stopped using KA9Q before '96, which a search indicates is when SYN cookies started. The KA9Q configuration option was 'tcp syndata', if you do searches for 'ka9q nos' you'll find the code if you wish to play (or just read it). DF From nobody Tue Aug 2 11:05:07 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 46E2E12D84F; Tue, 2 Aug 2016 11:05:03 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -8.187 X-Spam-Level: X-Spam-Status: No, score=-8.187 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, RP_MATCHES_RCVD=-1.287] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id oZfEZBxVU3HQ; Tue, 2 Aug 2016 11:05:02 -0700 (PDT) Received: from boreas.isi.edu (boreas.isi.edu [128.9.160.161]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 653A912D84B; Tue, 2 Aug 2016 11:05:02 -0700 (PDT) Received: from [128.9.184.236] ([128.9.184.236]) (authenticated bits=0) by boreas.isi.edu (8.13.8/8.13.8) with ESMTP id u72I4Wsn001879 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Tue, 2 Aug 2016 11:04:33 -0700 (PDT) To: Derek Fawcus , Gavin McCullagh References: <87wpk7x9v6.fsf@ta.scs.stanford.edu> <877fc6ycuw.fsf@ta.scs.stanford.edu> <20160727232419.GA45841@cowbell.employees.org> <20160728063754.GA24657@cowbell.employees.org> <20160802082425.GA9117@cowbell.employees.org> From: Joe Touch Message-ID: <57A0E0B0.7010003@isi.edu> Date: Tue, 2 Aug 2016 11:04:32 -0700 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.7.2 MIME-Version: 1.0 In-Reply-To: <20160802082425.GA9117@cowbell.employees.org> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Archived-At: Cc: tcpinc , "tcpm@ietf.org Extensions" Subject: Re: [tcpm] [tcpinc] TCP's treatment of data in SYN packets X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2016 18:05:03 -0000 On 8/2/2016 1:24 AM, Derek Fawcus wrote: > On Mon, Aug 01, 2016 at 08:52:43pm -0700, Gavin McCullagh wrote: >> Just to satisfy my curiosity, how did this work (or did it?) with TCP SYN Cookies? > The sender of the SYN would simply send the data again once the 3whs completed. An important point - it would send exactly the same data again (or if it sent more data, the overlapping portions never changed). Joe From nobody Tue Aug 2 12:15:30 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A5EC112D8ED for ; Tue, 2 Aug 2016 12:15:28 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.601 X-Spam-Level: X-Spam-Status: No, score=-2.601 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=mti-systems-com.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id cCjqvcPS-SWz for ; Tue, 2 Aug 2016 12:15:27 -0700 (PDT) Received: from mail-io0-x233.google.com (mail-io0-x233.google.com [IPv6:2607:f8b0:4001:c06::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 546E012D8E8 for ; Tue, 2 Aug 2016 12:15:27 -0700 (PDT) Received: by mail-io0-x233.google.com with SMTP id q83so222576554iod.1 for ; Tue, 02 Aug 2016 12:15:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mti-systems-com.20150623.gappssmtp.com; s=20150623; h=to:from:subject:message-id:date:user-agent:mime-version :content-transfer-encoding; bh=8g2ptD+s/ZaJEsJtprd93lNRk/+cycfCRnVMj/4RfJc=; b=j2gdpmaX/VApEvZFNtjP6wzZSvFdoVT8PU9rU0T9014wq8hR8fhCjuczCLZH1yRgCn mf5M+Amn70JX+9+iMRXzhfATMe7UyK6DZd5uxzNRExsiX6Il04l5wpp88itU0rE4gwLy hwMOqy9zg52xS2andWN87bjZJJD6KL4Ws26Zb6vKOip3LCQiwuX9mHgg9I+kAleBWtVY nTuhWQ4hfe+8cP2Uy058rOdFAUwfoaanSU6jf85Gjd80/xWXw9kc0X5vRsa+9mSDDpUM 8jlcNPZZPlblpvgSIXwuAo4NoEURG6xjeLtkLqyXxbmcex1bwBWcN9S0FSLy61B6NEAl 104Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:to:from:subject:message-id:date:user-agent :mime-version:content-transfer-encoding; bh=8g2ptD+s/ZaJEsJtprd93lNRk/+cycfCRnVMj/4RfJc=; b=H1TDpMzcRqROgLHDg16ow19xtMY3plegR3KA74mtUVIn6u880eQz0UuIeiZ+GFKVx7 ptm0Sb5/u+zxLqiHOoTgzF8rVaUYSTpr1DkteLIUiMcTOWz0UpbBPrSB1HPBdxKhXM5R Wj9krI4mCR0U3odLSH5SzfTs3VPwmHP0LenofFn3E12tT/zLsl7ynkyPYxbmns9huU/6 rA0X5L26eqPxTkzQY3a38vWvlvOvv/y18CnhsVEC/f/NXaUPgT3jWzY1jt9cwBAqZJS2 b4POEzRn23k+pF/sMfqUhyvbXrhwIqbU14+tSC6B3t1eW+LYtj5MqgjY2R1OxQQUPEvy z3GQ== X-Gm-Message-State: AEkoouvb0h3IJq0vIrvXpBnEuSqtrrCloyFe9qzrpQUSNNKPbyRMyiZDpqL5C/FejVTOpg== X-Received: by 10.107.132.217 with SMTP id o86mr63729853ioi.18.1470165326318; Tue, 02 Aug 2016 12:15:26 -0700 (PDT) Received: from [192.168.1.104] (cpe-76-188-215-129.neo.res.rr.com. [76.188.215.129]) by smtp.gmail.com with ESMTPSA id w138sm1914926itc.8.2016.08.02.12.15.25 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 02 Aug 2016 12:15:25 -0700 (PDT) To: tcpm@ietf.org From: Wesley Eddy Message-ID: Date: Tue, 2 Aug 2016 15:15:25 -0400 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Archived-At: Subject: [tcpm] 793bis: removing "Window Management Suggestions" X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2016 19:15:28 -0000 In the latest revision of the 793bis draft, I added content from RFC 1122 on the SWS avoidance algorithms for sender-side and receiver-side. These are laid out in 1122, but were not present in 793. 793 includes a section called "Window Management Suggestions", that I believe was a precursor to the more-developed content in 1122 on: (1) SWS avoidance sender-side algorithm (2) SWS avoidance receiver-side algorithm (3) Nagle algorithm (4) delayed ACKs (5) zero-window probing With all of that 1122 content worked into 793bis, I did not think it made sense to retain this section of the document, but it will be good to confirm that with the WG. For convenience, the content from 793 that I believe is no longer needed is pasted below: Window Management Suggestions Allocating a very small window causes data to be transmitted in many small segments when better performance is achieved using fewer large segments. One suggestion for avoiding small windows is for the receiver to defer updating a window until the additional allocation is at least X percent of the maximum allocation possible for the connection (where X might be 20 to 40). Another suggestion is for the sender to avoid sending small segments by waiting until the window is large enough before sending data. If the the user signals a push function then the data must be sent even if it is a small segment. Note that the acknowledgments should not be delayed or unnecessary retransmissions will result. One strategy would be to send an acknowledgment when a small segment arrives (with out updating the window information), and then to send another acknowledgment with new window information when the window is larger. The segment sent to probe a zero window may also begin a break up of transmitted data into smaller and smaller segments. If a segment containing a single data octet sent to probe a zero window is accepted, it consumes one octet of the window now available. If the sending TCP simply sends as much as it can whenever the window is non zero, the transmitted data will be broken into alternating big and small segments. As time goes on, occasional pauses in the receiver making window allocation available will result in breaking the big segments into a small and not quite so big pair. And after a while the data transmission will be in mostly small segments. The suggestion here is that the TCP implementations need to actively attempt to combine small window allocations into larger windows, since the mechanisms for managing the window tend to lead to many small windows in the simplest minded implementations. Any thoughts or feedback on removing this from 793bis will be appreciated, thanks! From nobody Tue Aug 2 12:42:52 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C908712D90F for ; Tue, 2 Aug 2016 12:42:50 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.988 X-Spam-Level: X-Spam-Status: No, score=-3.988 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RP_MATCHES_RCVD=-1.287, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=google.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6Reb9_WStC-B for ; Tue, 2 Aug 2016 12:42:49 -0700 (PDT) Received: from mail-io0-x22f.google.com (mail-io0-x22f.google.com [IPv6:2607:f8b0:4001:c06::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 446F312D90D for ; Tue, 2 Aug 2016 12:42:49 -0700 (PDT) Received: by mail-io0-x22f.google.com with SMTP id b62so222675995iod.3 for ; Tue, 02 Aug 2016 12:42:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=oduNjxfHOtOSU1a2g9A6wLXIQlSC7yuGZx53XBmhZF4=; b=GfWak4tHLPfG2QHU73Xp7CovDyMOPhoTsAQqgsvJdoq83vThCRSbq6QH8RbxL0dpUU jpbpdv3RoK8+kFt6vMkRuPl1AV9WxkeFQ8vhl05dBCTrANSLWF55TefOPwGsBwuY/25d YN1Rz5e3YgdK7aK9sWuRTuAV6sZ78lkqO/gtRglxACcKDw/E26KgVAckhTVrsGeYWhZy adteSDVICy2nN4VT+T4G6IO70URSAUbzUT/cX8OSZ7nQzP2DRwg0uLIt+5VixpUUgdwn 4KvyxMCrLTzNg4P82Qfqn6dZ9P5siwdF+vISSNQCmzxnw6kkJ3/siuB/EltDqtvJb8t8 i3GQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=oduNjxfHOtOSU1a2g9A6wLXIQlSC7yuGZx53XBmhZF4=; b=bjm2WHQu8fKiLcM02J+DxItHOdzRASTg920rIoDT8nDy0ODX0HbLbDaFIFR9P1Jwzy 2CbJvjVKtYeg/ortOAYqbYEeHpDjfqe1DAL9uTFHIoPcEZ8qMWuEuiY0JQvWcVgMOa2b hYtN/kg17J3oniZG91UjOHZnzhSta2z2FZvI1Zzi+UAL4pZZ/szpBonl5cW4WrbGByE+ sTJvLwuLXvrL1uZsjsFFRvxMViST0DP4Pfs0Vt5dyXYPTdTtXjuAIb5gPv+ks2lqb0ea PgIhMO2tlNu04kl1y7wVgnvVV1WCMHbnVfltm+kOhdvkJsnY2s/CgnkA3XLsWRP99ebK I6ig== X-Gm-Message-State: AEkoouvZQeOWojP3yz2d2l2AJYknlebD0ZKQ0wmV4nBNuOzA+pjbOT/L9wAlLpBpUNJ5BKtLfJHWFbVfQlf1Xpm3 X-Received: by 10.107.59.7 with SMTP id i7mr66255903ioa.190.1470166968409; Tue, 02 Aug 2016 12:42:48 -0700 (PDT) MIME-Version: 1.0 Received: by 10.64.244.103 with HTTP; Tue, 2 Aug 2016 12:42:07 -0700 (PDT) In-Reply-To: References: From: Yuchung Cheng Date: Tue, 2 Aug 2016 12:42:07 -0700 Message-ID: To: Wesley Eddy Content-Type: text/plain; charset=UTF-8 Archived-At: Cc: "tcpm@ietf.org Extensions" Subject: Re: [tcpm] 793bis: removing "Window Management Suggestions" X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2016 19:42:51 -0000 On Tue, Aug 2, 2016 at 12:15 PM, Wesley Eddy wrote: > In the latest revision of the 793bis draft, I added content from RFC 1122 on > the SWS avoidance algorithms for sender-side and receiver-side. These are > laid out in 1122, but were not present in 793. > > 793 includes a section called "Window Management Suggestions", that I > believe was a precursor to the more-developed content in 1122 on: > > (1) SWS avoidance sender-side algorithm > > (2) SWS avoidance receiver-side algorithm > > (3) Nagle algorithm > > (4) delayed ACKs > > (5) zero-window probing > > > With all of that 1122 content worked into 793bis, I did not think it made > sense to retain this section of the document, but it will be good to confirm > that with the WG. > > For convenience, the content from 793 that I believe is no longer needed is > pasted below: +1 BTW on https://tools.ietf.org/html/draft-ietf-tcpm-rfc793bis-03#section-3.8.3.3 A TCP MAY keep its offered receive window closed indefinitely. As long as the receiving TCP continues to send acknowledgments in response to the probe segments, the sending TCP MUST allow the connection to stay open. The second unconditional MUST opens up a simple and effective DoS attack, which Linux no longer wants to be vulnerable of. https://patchwork.ozlabs.org/patch/392217/ > > Window Management Suggestions > > Allocating a very small window causes data to be transmitted in > many small segments when better performance is achieved using > fewer large segments. > > One suggestion for avoiding small windows is for the receiver to > defer updating a window until the additional allocation is at > least X percent of the maximum allocation possible for the > connection (where X might be 20 to 40). > > Another suggestion is for the sender to avoid sending small > segments by waiting until the window is large enough before > sending data. If the the user signals a push function then the > data must be sent even if it is a small segment. > > Note that the acknowledgments should not be delayed or unnecessary > retransmissions will result. One strategy would be to send an > acknowledgment when a small segment arrives (with out updating the > window information), and then to send another acknowledgment with > new window information when the window is larger. > > The segment sent to probe a zero window may also begin a break up > of transmitted data into smaller and smaller segments. If a > segment containing a single data octet sent to probe a zero window > is accepted, it consumes one octet of the window now available. > If the sending TCP simply sends as much as it can whenever the > window is non zero, the transmitted data will be broken into > alternating big and small segments. As time goes on, occasional > pauses in the receiver making window allocation available will > result in breaking the big segments into a small and not quite so > big pair. And after a while the data transmission will be in > mostly small segments. > > The suggestion here is that the TCP implementations need to > actively attempt to combine small window allocations into larger > windows, since the mechanisms for managing the window tend to lead > to many small windows in the simplest minded implementations. > > > Any thoughts or feedback on removing this from 793bis will be appreciated, > thanks! > > > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm From nobody Tue Aug 2 12:53:38 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B561A12D5C2 for ; Tue, 2 Aug 2016 12:53:37 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.187 X-Spam-Level: X-Spam-Status: No, score=-3.187 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RP_MATCHES_RCVD=-1.287] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id y02NBT7zAlPH for ; Tue, 2 Aug 2016 12:53:36 -0700 (PDT) Received: from nitro.isi.edu (nitro.isi.edu [128.9.208.207]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id DC94212D587 for ; Tue, 2 Aug 2016 12:53:35 -0700 (PDT) Received: from [128.9.184.236] ([128.9.184.236]) (authenticated bits=0) by nitro.isi.edu (8.13.8/8.13.8) with ESMTP id u72JqssK007173 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Tue, 2 Aug 2016 12:52:54 -0700 (PDT) To: Yuchung Cheng , Wesley Eddy References: From: Joe Touch Message-ID: <57A0FA15.6090507@isi.edu> Date: Tue, 2 Aug 2016 12:52:53 -0700 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.7.2 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-MailScanner-ID: u72JqssK007173 X-ISI-4-69-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Archived-At: Cc: "tcpm@ietf.org Extensions" Subject: Re: [tcpm] 793bis: removing "Window Management Suggestions" X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2016 19:53:38 -0000 On 8/2/2016 12:42 PM, Yuchung Cheng wrote: > On Tue, Aug 2, 2016 at 12:15 PM, Wesley Eddy wrote: >> In the latest revision of the 793bis draft, I added content from RFC 1122 on >> the SWS avoidance algorithms for sender-side and receiver-side. These are >> laid out in 1122, but were not present in 793. >> ... >> >> For convenience, the content from 793 that I believe is no longer needed is >> pasted below: > +1 +1 > BTW on https://tools.ietf.org/html/draft-ietf-tcpm-rfc793bis-03#section-3.8.3.3 > > A TCP MAY keep its offered receive window closed indefinitely. As > long as the receiving TCP continues to send acknowledgments in > response to the probe segments, the sending TCP MUST allow the > connection to stay open. > > > The second unconditional MUST opens up a simple and effective DoS > attack, which Linux no longer wants to be vulnerable of. > https://patchwork.ozlabs.org/patch/392217/ I strongly discourage attempts to try to bulletproof unsecured protocols. IMO, the ability of an endpoint to keep its window closed indefinitely is its prerogative, just as it is the prerogative of the other end to close such a connection. AFAICT, the Linux code is consistent with the second half of this statement, but I don't think it should be required or even mentioned. It's always obvious that an endpoint OS can and should protect its resources by whatever means it thinks necessary. These are not protocol decisions. Joe > > >> Window Management Suggestions >> >> Allocating a very small window causes data to be transmitted in >> many small segments when better performance is achieved using >> fewer large segments. >> >> One suggestion for avoiding small windows is for the receiver to >> defer updating a window until the additional allocation is at >> least X percent of the maximum allocation possible for the >> connection (where X might be 20 to 40). >> >> Another suggestion is for the sender to avoid sending small >> segments by waiting until the window is large enough before >> sending data. If the the user signals a push function then the >> data must be sent even if it is a small segment. >> >> Note that the acknowledgments should not be delayed or unnecessary >> retransmissions will result. One strategy would be to send an >> acknowledgment when a small segment arrives (with out updating the >> window information), and then to send another acknowledgment with >> new window information when the window is larger. >> >> The segment sent to probe a zero window may also begin a break up >> of transmitted data into smaller and smaller segments. If a >> segment containing a single data octet sent to probe a zero window >> is accepted, it consumes one octet of the window now available. >> If the sending TCP simply sends as much as it can whenever the >> window is non zero, the transmitted data will be broken into >> alternating big and small segments. As time goes on, occasional >> pauses in the receiver making window allocation available will >> result in breaking the big segments into a small and not quite so >> big pair. And after a while the data transmission will be in >> mostly small segments. >> >> The suggestion here is that the TCP implementations need to >> actively attempt to combine small window allocations into larger >> windows, since the mechanisms for managing the window tend to lead >> to many small windows in the simplest minded implementations. >> >> >> Any thoughts or feedback on removing this from 793bis will be appreciated, >> thanks! >> >> >> _______________________________________________ >> tcpm mailing list >> tcpm@ietf.org >> https://www.ietf.org/mailman/listinfo/tcpm > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm From nobody Tue Aug 2 13:14:55 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C0FBC12D946 for ; Tue, 2 Aug 2016 13:14:53 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.187 X-Spam-Level: X-Spam-Status: No, score=-3.187 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RP_MATCHES_RCVD=-1.287] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id LSHPijwm4hgz for ; Tue, 2 Aug 2016 13:14:51 -0700 (PDT) Received: from frantic.weston.borman.com (frantic.weston.borman.com [70.57.156.33]) (using TLSv1 with cipher EDH-RSA-DES-CBC3-SHA (112/168 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 4F9C812D944 for ; Tue, 2 Aug 2016 13:14:51 -0700 (PDT) Received: from [127.0.0.1] (frantic.weston.borman.com [70.57.156.33]) by frantic.weston.borman.com (8.14.7/8.14.7) with ESMTP id u72KEepQ025025; Tue, 2 Aug 2016 15:14:40 -0500 (CDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) From: David Borman In-Reply-To: <57A0FA15.6090507@isi.edu> Date: Tue, 2 Aug 2016 15:14:40 -0500 Content-Transfer-Encoding: quoted-printable Message-Id: References: <57A0FA15.6090507@isi.edu> To: Joe Touch X-Mailer: Apple Mail (2.3124) Archived-At: Cc: "tcpm@ietf.org Extensions" Subject: Re: [tcpm] 793bis: removing "Window Management Suggestions" X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2016 20:14:54 -0000 The point about keeping the connection open in the face of zero window = probes has the classic example of sending a job to the printer just = before heading off to lunch, and when you come back later discover that = the printer ran out of paper and stopped accepting new TCP data. The = printer keeps responding to the probes with a zero window, and when = paper is finally added to the printer, printing resumes, the TCP window = is opened and the job completes. Years ago there were examples of situations like this where TCP = connections were being timed out relatively quickly in the face of a = zero window, causing failures when there was no need for a failure. = It=E2=80=99d be nice if trying to address a potential DOS situation = didn=E2=80=99t cause normal situations to regress back to those days; it = then becomes a matter of balancing how long to wait before giving up on = a zero window connection. -David Borman > On Aug 2, 2016, at 2:52 PM, Joe Touch wrote: >=20 >=20 >=20 > On 8/2/2016 12:42 PM, Yuchung Cheng wrote: >> On Tue, Aug 2, 2016 at 12:15 PM, Wesley Eddy = wrote: >>> In the latest revision of the 793bis draft, I added content from RFC = 1122 on >>> the SWS avoidance algorithms for sender-side and receiver-side. = These are >>> laid out in 1122, but were not present in 793. >>> ... >>>=20 >>> For convenience, the content from 793 that I believe is no longer = needed is >>> pasted below: >> +1 > +1 >=20 >> BTW on = https://tools.ietf.org/html/draft-ietf-tcpm-rfc793bis-03#section-3.8.3.3 >> >> A TCP MAY keep its offered receive window closed indefinitely. As >> long as the receiving TCP continues to send acknowledgments in >> response to the probe segments, the sending TCP MUST allow the >> connection to stay open. >> >>=20 >> The second unconditional MUST opens up a simple and effective DoS >> attack, which Linux no longer wants to be vulnerable of. >> https://patchwork.ozlabs.org/patch/392217/ >=20 > I strongly discourage attempts to try to bulletproof unsecured = protocols. >=20 > IMO, the ability of an endpoint to keep its window closed indefinitely > is its prerogative, just as it is the prerogative of the other end to > close such a connection. AFAICT, the Linux code is consistent with the > second half of this statement, but I don't think it should be required > or even mentioned. >=20 > It's always obvious that an endpoint OS can and should protect its > resources by whatever means it thinks necessary. These are not = protocol > decisions. >=20 > Joe >=20 >=20 >=20 >>=20 >>=20 >>> Window Management Suggestions >>>=20 >>> Allocating a very small window causes data to be transmitted in >>> many small segments when better performance is achieved using >>> fewer large segments. >>>=20 >>> One suggestion for avoiding small windows is for the receiver = to >>> defer updating a window until the additional allocation is at >>> least X percent of the maximum allocation possible for the >>> connection (where X might be 20 to 40). >>>=20 >>> Another suggestion is for the sender to avoid sending small >>> segments by waiting until the window is large enough before >>> sending data. If the the user signals a push function then the >>> data must be sent even if it is a small segment. >>>=20 >>> Note that the acknowledgments should not be delayed or = unnecessary >>> retransmissions will result. One strategy would be to send an >>> acknowledgment when a small segment arrives (with out updating = the >>> window information), and then to send another acknowledgment = with >>> new window information when the window is larger. >>>=20 >>> The segment sent to probe a zero window may also begin a break = up >>> of transmitted data into smaller and smaller segments. If a >>> segment containing a single data octet sent to probe a zero = window >>> is accepted, it consumes one octet of the window now available. >>> If the sending TCP simply sends as much as it can whenever the >>> window is non zero, the transmitted data will be broken into >>> alternating big and small segments. As time goes on, = occasional >>> pauses in the receiver making window allocation available will >>> result in breaking the big segments into a small and not quite = so >>> big pair. And after a while the data transmission will be in >>> mostly small segments. >>>=20 >>> The suggestion here is that the TCP implementations need to >>> actively attempt to combine small window allocations into = larger >>> windows, since the mechanisms for managing the window tend to = lead >>> to many small windows in the simplest minded implementations. >>>=20 >>>=20 >>> Any thoughts or feedback on removing this from 793bis will be = appreciated, >>> thanks! >>>=20 >>>=20 >>> _______________________________________________ >>> tcpm mailing list >>> tcpm@ietf.org >>> https://www.ietf.org/mailman/listinfo/tcpm >> _______________________________________________ >> tcpm mailing list >> tcpm@ietf.org >> https://www.ietf.org/mailman/listinfo/tcpm >=20 > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm From nobody Tue Aug 2 13:18:37 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2CA9312D125 for ; Tue, 2 Aug 2016 13:18:36 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -8.187 X-Spam-Level: X-Spam-Status: No, score=-8.187 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, RP_MATCHES_RCVD=-1.287] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id uCOQsh1smHJz for ; Tue, 2 Aug 2016 13:18:34 -0700 (PDT) Received: from vapor.isi.edu (vapor.isi.edu [128.9.64.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1C6D712D0E9 for ; Tue, 2 Aug 2016 13:18:34 -0700 (PDT) Received: from [128.9.184.236] ([128.9.184.236]) (authenticated bits=0) by vapor.isi.edu (8.13.8/8.13.8) with ESMTP id u72KHa9I021747 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Tue, 2 Aug 2016 13:17:37 -0700 (PDT) To: David Borman References: <57A0FA15.6090507@isi.edu> From: Joe Touch Message-ID: <57A0FFE0.7020905@isi.edu> Date: Tue, 2 Aug 2016 13:17:36 -0700 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.7.2 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Archived-At: Cc: "tcpm@ietf.org Extensions" Subject: Re: [tcpm] 793bis: removing "Window Management Suggestions" X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2016 20:18:36 -0000 Hi, David, I agree. IMO, DOS situations should be handled by the OS by withdrawing resources *as needed*. That includes closing down zero-window situations as below, as well as closing idle connections when resources are limited. I don't see this as something the protocol needs to decide. Joe On 8/2/2016 1:14 PM, David Borman wrote: > The point about keeping the connection open in the face of zero window probes has the classic example of sending a job to the printer just before heading off to lunch, and when you come back later discover that the printer ran out of paper and stopped accepting new TCP data. The printer keeps responding to the probes with a zero window, and when paper is finally added to the printer, printing resumes, the TCP window is opened and the job completes. > > Years ago there were examples of situations like this where TCP connections were being timed out relatively quickly in the face of a zero window, causing failures when there was no need for a failure. It’d be nice if trying to address a potential DOS situation didn’t cause normal situations to regress back to those days; it then becomes a matter of balancing how long to wait before giving up on a zero window connection. > > -David Borman > >> On Aug 2, 2016, at 2:52 PM, Joe Touch wrote: >> >> >> >> On 8/2/2016 12:42 PM, Yuchung Cheng wrote: >>> On Tue, Aug 2, 2016 at 12:15 PM, Wesley Eddy wrote: >>>> In the latest revision of the 793bis draft, I added content from RFC 1122 on >>>> the SWS avoidance algorithms for sender-side and receiver-side. These are >>>> laid out in 1122, but were not present in 793. >>>> ... >>>> >>>> For convenience, the content from 793 that I believe is no longer needed is >>>> pasted below: >>> +1 >> +1 >> >>> BTW on https://tools.ietf.org/html/draft-ietf-tcpm-rfc793bis-03#section-3.8.3.3 >>> >>> A TCP MAY keep its offered receive window closed indefinitely. As >>> long as the receiving TCP continues to send acknowledgments in >>> response to the probe segments, the sending TCP MUST allow the >>> connection to stay open. >>> >>> >>> The second unconditional MUST opens up a simple and effective DoS >>> attack, which Linux no longer wants to be vulnerable of. >>> https://patchwork.ozlabs.org/patch/392217/ >> I strongly discourage attempts to try to bulletproof unsecured protocols. >> >> IMO, the ability of an endpoint to keep its window closed indefinitely >> is its prerogative, just as it is the prerogative of the other end to >> close such a connection. AFAICT, the Linux code is consistent with the >> second half of this statement, but I don't think it should be required >> or even mentioned. >> >> It's always obvious that an endpoint OS can and should protect its >> resources by whatever means it thinks necessary. These are not protocol >> decisions. >> >> Joe >> >> >> >>> >>>> Window Management Suggestions >>>> >>>> Allocating a very small window causes data to be transmitted in >>>> many small segments when better performance is achieved using >>>> fewer large segments. >>>> >>>> One suggestion for avoiding small windows is for the receiver to >>>> defer updating a window until the additional allocation is at >>>> least X percent of the maximum allocation possible for the >>>> connection (where X might be 20 to 40). >>>> >>>> Another suggestion is for the sender to avoid sending small >>>> segments by waiting until the window is large enough before >>>> sending data. If the the user signals a push function then the >>>> data must be sent even if it is a small segment. >>>> >>>> Note that the acknowledgments should not be delayed or unnecessary >>>> retransmissions will result. One strategy would be to send an >>>> acknowledgment when a small segment arrives (with out updating the >>>> window information), and then to send another acknowledgment with >>>> new window information when the window is larger. >>>> >>>> The segment sent to probe a zero window may also begin a break up >>>> of transmitted data into smaller and smaller segments. If a >>>> segment containing a single data octet sent to probe a zero window >>>> is accepted, it consumes one octet of the window now available. >>>> If the sending TCP simply sends as much as it can whenever the >>>> window is non zero, the transmitted data will be broken into >>>> alternating big and small segments. As time goes on, occasional >>>> pauses in the receiver making window allocation available will >>>> result in breaking the big segments into a small and not quite so >>>> big pair. And after a while the data transmission will be in >>>> mostly small segments. >>>> >>>> The suggestion here is that the TCP implementations need to >>>> actively attempt to combine small window allocations into larger >>>> windows, since the mechanisms for managing the window tend to lead >>>> to many small windows in the simplest minded implementations. >>>> >>>> >>>> Any thoughts or feedback on removing this from 793bis will be appreciated, >>>> thanks! >>>> >>>> >>>> _______________________________________________ >>>> tcpm mailing list >>>> tcpm@ietf.org >>>> https://www.ietf.org/mailman/listinfo/tcpm >>> _______________________________________________ >>> tcpm mailing list >>> tcpm@ietf.org >>> https://www.ietf.org/mailman/listinfo/tcpm >> _______________________________________________ >> tcpm mailing list >> tcpm@ietf.org >> https://www.ietf.org/mailman/listinfo/tcpm > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm From nobody Tue Aug 2 13:23:09 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 52E5B12D125 for ; Tue, 2 Aug 2016 13:23:08 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.988 X-Spam-Level: X-Spam-Status: No, score=-3.988 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RP_MATCHES_RCVD=-1.287, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=google.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hwGjP-p_wA4W for ; Tue, 2 Aug 2016 13:23:06 -0700 (PDT) Received: from mail-io0-x22d.google.com (mail-io0-x22d.google.com [IPv6:2607:f8b0:4001:c06::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 94ADE12D520 for ; Tue, 2 Aug 2016 13:23:06 -0700 (PDT) Received: by mail-io0-x22d.google.com with SMTP id m101so223513085ioi.2 for ; Tue, 02 Aug 2016 13:23:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=YzsNdGdVaiovyW7hXoYJlBoPKzmNlJPZsk7LMUAP8BI=; b=Knuvtzt1eINOoQ8v92QllpIvx4ktebRFUF1ugy6xguyW95XvgJ6c78FeSB7h7PVw/e HK4fJ0dw98q10uhOz2cTFJGmffHMbzfgnJO5RpWlL/qC9dZc1zQ9sBxA2ntNO0zbNwkr LjM4jz1wl+K/HRzrPWXz22YVvVqG52d+drSkX9bBB/iT836bbgYEr8R8DZKC142HpRsF rT5th7XBDd/gYVTOS0xt8clhTKAVZwmISE1I+oBk4zriY84fkI8ymZOrKZd4dMlEVBRY gDcCjvqhB4ICkdHSPgb+a8A6eboe4cppzcO9NYj4f2OY3+NHuE3495f8FkbVl9MPwjEA HLQg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=YzsNdGdVaiovyW7hXoYJlBoPKzmNlJPZsk7LMUAP8BI=; b=YwJ6g6NWqLFmzZXMYDiA7lRhzPd9s4eOeIUXVAqR3xeW8Mix4ycv1KHx0OpY/qX5Y0 jhHZCgwgPN9/abGbM/q86ReYnudfdN3dHBtr6mavj+mypSlAupyHn+FYzrrAIzOwxXI8 ob6vQGSoyry4RO2TmqYKqEBRki0PqoXW8YFZvvbhSWwkYW6/Xp7V6f34zfVH4iJ1ZYil nsNbbW8caUeldHJZInX8inpNUNAVYs4w+zaL2PAiDma9jXc3DOxIZF6NPpLjKv8zSzjK i07afhQJeNVmzhQ+szw+I5mPJ9BRv5f/qBd5NxZAuvZoSjMJjZFd9mK9O9FipVr4MAND +RAw== X-Gm-Message-State: AEkoouvR8UV5hfMN6t9dl07ZScO4rngN5rDnotupc216ewne4Szyu/ofXydsgi745pm2VeWp2RbWoZFDmBxLmsOl X-Received: by 10.107.201.135 with SMTP id z129mr72989197iof.114.1470169385637; Tue, 02 Aug 2016 13:23:05 -0700 (PDT) MIME-Version: 1.0 Received: by 10.64.244.103 with HTTP; Tue, 2 Aug 2016 13:22:25 -0700 (PDT) In-Reply-To: References: <57A0FA15.6090507@isi.edu> From: Yuchung Cheng Date: Tue, 2 Aug 2016 13:22:25 -0700 Message-ID: To: David Borman Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Archived-At: Cc: "tcpm@ietf.org Extensions" , Joe Touch Subject: Re: [tcpm] 793bis: removing "Window Management Suggestions" X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2016 20:23:08 -0000 On Tue, Aug 2, 2016 at 1:14 PM, David Borman wrote: > The point about keeping the connection open in the face of zero window pr= obes has the classic example of sending a job to the printer just before he= ading off to lunch, and when you come back later discover that the printer = ran out of paper and stopped accepting new TCP data. The printer keeps res= ponding to the probes with a zero window, and when paper is finally added t= o the printer, printing resumes, the TCP window is opened and the job compl= etes. > > Years ago there were examples of situations like this where TCP connectio= ns were being timed out relatively quickly in the face of a zero window, ca= using failures when there was no need for a failure. It=E2=80=99d be nice = if trying to address a potential DOS situation didn=E2=80=99t cause normal = situations to regress back to those days; it then becomes a matter of balan= cing how long to wait before giving up on a zero window connection. > > -David Borman I am well aware of the printer story and which part in my original email did I advocate to remove the MUST? Linux only chops the ropes when these connections have no real owners (i.e., orphaned socket in Linux) and the amount exceeds a configurable system limit. I am mentioning there is a real attack behind this simple MUST. > >> On Aug 2, 2016, at 2:52 PM, Joe Touch wrote: >> >> >> >> On 8/2/2016 12:42 PM, Yuchung Cheng wrote: >>> On Tue, Aug 2, 2016 at 12:15 PM, Wesley Eddy wrot= e: >>>> In the latest revision of the 793bis draft, I added content from RFC 1= 122 on >>>> the SWS avoidance algorithms for sender-side and receiver-side. These= are >>>> laid out in 1122, but were not present in 793. >>>> ... >>>> >>>> For convenience, the content from 793 that I believe is no longer need= ed is >>>> pasted below: >>> +1 >> +1 >> >>> BTW on https://tools.ietf.org/html/draft-ietf-tcpm-rfc793bis-03#section= -3.8.3.3 >>> >>> A TCP MAY keep its offered receive window closed indefinitely. As >>> long as the receiving TCP continues to send acknowledgments in >>> response to the probe segments, the sending TCP MUST allow the >>> connection to stay open. >>> >>> >>> The second unconditional MUST opens up a simple and effective DoS >>> attack, which Linux no longer wants to be vulnerable of. >>> https://patchwork.ozlabs.org/patch/392217/ >> >> I strongly discourage attempts to try to bulletproof unsecured protocols= . >> >> IMO, the ability of an endpoint to keep its window closed indefinitely >> is its prerogative, just as it is the prerogative of the other end to >> close such a connection. AFAICT, the Linux code is consistent with the >> second half of this statement, but I don't think it should be required >> or even mentioned. >> >> It's always obvious that an endpoint OS can and should protect its >> resources by whatever means it thinks necessary. These are not protocol >> decisions. >> >> Joe >> >> >> >>> >>> >>>> Window Management Suggestions >>>> >>>> Allocating a very small window causes data to be transmitted in >>>> many small segments when better performance is achieved using >>>> fewer large segments. >>>> >>>> One suggestion for avoiding small windows is for the receiver to >>>> defer updating a window until the additional allocation is at >>>> least X percent of the maximum allocation possible for the >>>> connection (where X might be 20 to 40). >>>> >>>> Another suggestion is for the sender to avoid sending small >>>> segments by waiting until the window is large enough before >>>> sending data. If the the user signals a push function then the >>>> data must be sent even if it is a small segment. >>>> >>>> Note that the acknowledgments should not be delayed or unnecessar= y >>>> retransmissions will result. One strategy would be to send an >>>> acknowledgment when a small segment arrives (with out updating th= e >>>> window information), and then to send another acknowledgment with >>>> new window information when the window is larger. >>>> >>>> The segment sent to probe a zero window may also begin a break up >>>> of transmitted data into smaller and smaller segments. If a >>>> segment containing a single data octet sent to probe a zero windo= w >>>> is accepted, it consumes one octet of the window now available. >>>> If the sending TCP simply sends as much as it can whenever the >>>> window is non zero, the transmitted data will be broken into >>>> alternating big and small segments. As time goes on, occasional >>>> pauses in the receiver making window allocation available will >>>> result in breaking the big segments into a small and not quite so >>>> big pair. And after a while the data transmission will be in >>>> mostly small segments. >>>> >>>> The suggestion here is that the TCP implementations need to >>>> actively attempt to combine small window allocations into larger >>>> windows, since the mechanisms for managing the window tend to lea= d >>>> to many small windows in the simplest minded implementations. >>>> >>>> >>>> Any thoughts or feedback on removing this from 793bis will be apprecia= ted, >>>> thanks! >>>> >>>> >>>> _______________________________________________ >>>> tcpm mailing list >>>> tcpm@ietf.org >>>> https://www.ietf.org/mailman/listinfo/tcpm >>> _______________________________________________ >>> tcpm mailing list >>> tcpm@ietf.org >>> https://www.ietf.org/mailman/listinfo/tcpm >> >> _______________________________________________ >> tcpm mailing list >> tcpm@ietf.org >> https://www.ietf.org/mailman/listinfo/tcpm > From nobody Tue Aug 2 13:25:31 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8586F12D127 for ; Tue, 2 Aug 2016 13:25:30 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.022 X-Spam-Level: X-Spam-Status: No, score=-2.022 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=microsoft.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id UNmUi81LWply for ; Tue, 2 Aug 2016 13:25:28 -0700 (PDT) Received: from NAM01-BY2-obe.outbound.protection.outlook.com (mail-by2nam01on0139.outbound.protection.outlook.com [104.47.34.139]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1617912D125 for ; Tue, 2 Aug 2016 13:25:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=Ej1ivqWtq6mrh+8OV1bjZzrcL+sVQVHUvfrpcJa+ax0=; b=RPLPAmmpeZ3dKeTWpR7dbnfjNamGlg4KiT84l4RlV0ZA3BASkMDXXdxPW0WV+4A/gDwMnCqNReWm+DkVAriS8RrWZerGtV5dlYuLa/xLypGoXyubuD2HNeN3/+4ASYd4duW4ZIi5RX1wfUJoe+y+xNfzK9yz1iL/MuZ3oThEahA= Received: from BN6PR03MB2675.namprd03.prod.outlook.com (10.173.143.150) by BN6PR03MB2674.namprd03.prod.outlook.com (10.173.143.149) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.549.15; Tue, 2 Aug 2016 20:25:25 +0000 Received: from BN6PR03MB2675.namprd03.prod.outlook.com ([10.173.143.150]) by BN6PR03MB2675.namprd03.prod.outlook.com ([10.173.143.150]) with mapi id 15.01.0549.022; Tue, 2 Aug 2016 20:25:25 +0000 From: Christian Huitema To: Joe Touch , David Borman Thread-Topic: [tcpm] 793bis: removing "Window Management Suggestions" Thread-Index: AQHR7PI/evOSruawdEeNHd9Le3qMZKA2EgKAgAADAoCAAAYXAIAAANEAgAABbgA= Date: Tue, 2 Aug 2016 20:25:25 +0000 Message-ID: References: <57A0FA15.6090507@isi.edu> <57A0FFE0.7020905@isi.edu> In-Reply-To: <57A0FFE0.7020905@isi.edu> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=huitema@microsoft.com; x-originating-ip: [2001:4898:80e8:c::621] x-ms-office365-filtering-correlation-id: ddc3a247-2dec-4691-fec3-08d3bb13230f x-microsoft-exchange-diagnostics: 1; BN6PR03MB2674; 6:DFfmd0DgaBRxy7N/keUXLxLHMiwFV6SjSh702tmb9M9ffHrmIjM7nqXsrAJFbYXHZ1YFq30R3gpixVSm3l/BWYl+RkRnRzj2cw8h0gwHKUhrMshNkzgU9zeZi8SeXO2TgBHMeIjU1rTyY8dP1+TvuyHTCDM9CLYEAdJVsqMRB5dVFAhaxlb5kCKR/DhHivG8HTzL3VFW1jyZWfHWEZaeJg/LcdtA2SEn7srLaHyfNki2NNylfgif7ufsmbSaYGO/OSVYpuI31mYGmhOFOkgfC6C0e4jfLHwsTlW5oTKe1DV8pC5qzjimAEERLRQJ7UNBhphLoGXbZa2iu1zg1ulncA==; 5:2NZgb+fTilFe4vHen1Q6x2vzZ5QRC0tCXYL/9L34uN81tuJu47ZOdtlRkjrK/0+XJkf3LisF/6B+Av5PCa0OI9y8YtZbnhOZ+WAmd3RvtK6BQEHTKGOphCRvdeufzc+mu3Tb16LvnUSNAP8vh8rSRA==; 24:z6++6AR2xQUb/ChJBjNHLzfUzcyWdkfli2lf7Ocev3Zrh/pl+ovR3+M+4HlRriPEByMnDgDgg9C0XOoe9MdnGOcH2XEjaS+w+5knjsq62NU=; 7:qsu4+X+yJkOPy1G4B+ndxbGCnh18Kw5TEwd7ANu0EvL7erk3do22vvXzKQ81RDOzArhLwgXvB5J/Yq10Vi4lxYcpH4P1X41C9fG1PVNovzrJ4gf/4I9PBhU5N9cAekT0CmO2vHeRdniRIhiJaNsWFMaTtMhF8AIY73Xa38kA0bSbTP1HjBH1g7duWpKrQOojDDk4VmrQyxTzBGQYXko4Q0V1Qk808Cl1Rja05rwgM5rhPr+HJxheBfPLS0lnGgkk x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:BN6PR03MB2674; x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(158342451672863); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(61425038)(601004)(2401047)(8121501046)(5005006)(10201501046)(3002001)(6055026)(61426038)(61427038); SRVR:BN6PR03MB2674; BCL:0; PCL:0; RULEID:; SRVR:BN6PR03MB2674; x-forefront-prvs: 0022134A87 x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(6009001)(7916002)(199003)(377454003)(189002)(24454002)(13464003)(3660700001)(86612001)(7736002)(7846002)(2950100001)(11100500001)(74316002)(3280700002)(7696003)(76576001)(9686002)(2900100001)(10400500002)(189998001)(99286002)(105586002)(97736004)(5005710100001)(86362001)(10290500002)(92566002)(87936001)(19580405001)(2171001)(68736007)(15975445007)(102836003)(305945005)(10090500001)(5001770100001)(586003)(54356999)(101416001)(106356001)(33656002)(19580395003)(2906002)(122556002)(81166006)(4326007)(81156014)(8990500004)(106116001)(8676002)(77096005)(76176999)(5002640100001)(8936002)(6116002)(93886004)(50986999)(3826002); DIR:OUT; SFP:1102; SCL:1; SRVR:BN6PR03MB2674; H:BN6PR03MB2675.namprd03.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; received-spf: None (protection.outlook.com: microsoft.com does not designate permitted sender hosts) spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 MIME-Version: 1.0 X-OriginatorOrg: microsoft.com X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Aug 2016 20:25:25.3913 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR03MB2674 Archived-At: Cc: "tcpm@ietf.org Extensions" Subject: Re: [tcpm] 793bis: removing "Window Management Suggestions" X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2016 20:25:30 -0000 QXQgYSBtaW5pbXVtLCB0aGF0IG1lYW5zIHN3aXRjaGluZyB0aGUgbGFuZ3VhZ2UgZnJvbSAiTVVT VCBhbGxvdyB0aGUgY29ubmVjdGlvbiB0byBzdGF5IG9wZW4iIHRvICJTSE9VTEQgYWxsb3cgdGhl IGNvbm5lY3Rpb24gdG8gc3RheSBvcGVuIiBtYXliZSB3aXRoIGEgY2F2ZWF0IGxpa2UgInVubGVz cyBzZXJ2ZXIgc2l6ZSByZXNvdXJjZSBtYW5hZ2VtZW50IHJlcXVpcmVzIHRoYXQgaXQgYmUgY2xv c2VkLiINCg0KQW5kIHRoZSBwcmludGVyLW91dC1vZi1wYXBlciBzY2VuYXJpbyByZWFsbHkgb3Vn aHQgdG8gYmUgc29sdmVkIGF0IHRoZSBhcHBsaWNhdGlvbiBsYXllci4gQWZ0ZXIgYWxsLCBpZiB5 b3UgYXJlIHdhaXRpbmcgZm9yIGEgcHJpbnRvdXQsIGtub3dpbmcgd2h5IHlvdSBhcmUgd2FpdGlu ZyBpcyBzb21ld2hhdCBoZWxwZnVsLi4uDQoNCi0tIENocmlzdGlhbiBIdWl0ZW1hDQoNCg0KDQo+ IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+IEZyb206IHRjcG0gW21haWx0bzp0Y3BtLWJv dW5jZXNAaWV0Zi5vcmddIE9uIEJlaGFsZiBPZiBKb2UgVG91Y2gNCj4gU2VudDogVHVlc2RheSwg QXVndXN0IDIsIDIwMTYgMToxOCBQTQ0KPiBUbzogRGF2aWQgQm9ybWFuIDxkYWJAd2VzdG9uLmJv cm1hbi5jb20+DQo+IENjOiB0Y3BtQGlldGYub3JnIEV4dGVuc2lvbnMgPHRjcG1AaWV0Zi5vcmc+ DQo+IFN1YmplY3Q6IFJlOiBbdGNwbV0gNzkzYmlzOiByZW1vdmluZyAiV2luZG93IE1hbmFnZW1l bnQgU3VnZ2VzdGlvbnMiDQo+IA0KPiBIaSwgRGF2aWQsDQo+IA0KPiBJIGFncmVlLiBJTU8sIERP UyBzaXR1YXRpb25zIHNob3VsZCBiZSBoYW5kbGVkIGJ5IHRoZSBPUyBieSB3aXRoZHJhd2luZw0K PiByZXNvdXJjZXMgKmFzIG5lZWRlZCouIFRoYXQgaW5jbHVkZXMgY2xvc2luZyBkb3duIHplcm8t d2luZG93IHNpdHVhdGlvbnMNCj4gYXMgYmVsb3csIGFzIHdlbGwgYXMgY2xvc2luZyBpZGxlIGNv bm5lY3Rpb25zIHdoZW4gcmVzb3VyY2VzIGFyZSBsaW1pdGVkLg0KPiANCj4gSSBkb24ndCBzZWUg dGhpcyBhcyBzb21ldGhpbmcgdGhlIHByb3RvY29sIG5lZWRzIHRvIGRlY2lkZS4NCj4gDQo+IEpv ZQ0KPiANCj4gT24gOC8yLzIwMTYgMToxNCBQTSwgRGF2aWQgQm9ybWFuIHdyb3RlOg0KPiA+IFRo ZSBwb2ludCBhYm91dCBrZWVwaW5nIHRoZSBjb25uZWN0aW9uIG9wZW4gaW4gdGhlIGZhY2Ugb2Yg emVybyB3aW5kb3cNCj4gcHJvYmVzIGhhcyB0aGUgY2xhc3NpYyBleGFtcGxlIG9mIHNlbmRpbmcg YSBqb2IgdG8gdGhlIHByaW50ZXIganVzdCBiZWZvcmUNCj4gaGVhZGluZyBvZmYgdG8gbHVuY2gs IGFuZCB3aGVuIHlvdSBjb21lIGJhY2sgbGF0ZXIgZGlzY292ZXIgdGhhdCB0aGUgcHJpbnRlcg0K PiByYW4gb3V0IG9mIHBhcGVyIGFuZCBzdG9wcGVkIGFjY2VwdGluZyBuZXcgVENQIGRhdGEuICBU aGUgcHJpbnRlciBrZWVwcw0KPiByZXNwb25kaW5nIHRvIHRoZSBwcm9iZXMgd2l0aCBhIHplcm8g d2luZG93LCBhbmQgd2hlbiBwYXBlciBpcyBmaW5hbGx5IGFkZGVkDQo+IHRvIHRoZSBwcmludGVy LCBwcmludGluZyByZXN1bWVzLCB0aGUgVENQIHdpbmRvdyBpcyBvcGVuZWQgYW5kIHRoZSBqb2IN Cj4gY29tcGxldGVzLg0KPiA+DQo+ID4gWWVhcnMgYWdvIHRoZXJlIHdlcmUgZXhhbXBsZXMgb2Yg c2l0dWF0aW9ucyBsaWtlIHRoaXMgd2hlcmUgVENQIGNvbm5lY3Rpb25zDQo+IHdlcmUgYmVpbmcg dGltZWQgb3V0IHJlbGF0aXZlbHkgcXVpY2tseSBpbiB0aGUgZmFjZSBvZiBhIHplcm8gd2luZG93 LCBjYXVzaW5nDQo+IGZhaWx1cmVzIHdoZW4gdGhlcmUgd2FzIG5vIG5lZWQgZm9yIGEgZmFpbHVy ZS4gIEl04oCZZCBiZSBuaWNlIGlmIHRyeWluZyB0byBhZGRyZXNzIGENCj4gcG90ZW50aWFsIERP UyBzaXR1YXRpb24gZGlkbuKAmXQgY2F1c2Ugbm9ybWFsIHNpdHVhdGlvbnMgdG8gcmVncmVzcyBi YWNrIHRvIHRob3NlDQo+IGRheXM7IGl0IHRoZW4gYmVjb21lcyBhIG1hdHRlciBvZiBiYWxhbmNp bmcgaG93IGxvbmcgdG8gd2FpdCBiZWZvcmUgZ2l2aW5nIHVwDQo+IG9uIGEgemVybyB3aW5kb3cg Y29ubmVjdGlvbi4NCj4gPg0KPiA+IAkJCS1EYXZpZCBCb3JtYW4NCj4gPg0KPiA+PiBPbiBBdWcg MiwgMjAxNiwgYXQgMjo1MiBQTSwgSm9lIFRvdWNoIDx0b3VjaEBJU0kuRURVPiB3cm90ZToNCj4g Pj4NCj4gPj4NCj4gPj4NCj4gPj4gT24gOC8yLzIwMTYgMTI6NDIgUE0sIFl1Y2h1bmcgQ2hlbmcg d3JvdGU6DQo+ID4+PiBPbiBUdWUsIEF1ZyAyLCAyMDE2IGF0IDEyOjE1IFBNLCBXZXNsZXkgRWRk eSA8d2VzQG10aS1zeXN0ZW1zLmNvbT4NCj4gd3JvdGU6DQo+ID4+Pj4gSW4gdGhlIGxhdGVzdCBy ZXZpc2lvbiBvZiB0aGUgNzkzYmlzIGRyYWZ0LCBJIGFkZGVkIGNvbnRlbnQgZnJvbSBSRkMgMTEy Mg0KPiBvbg0KPiA+Pj4+IHRoZSBTV1MgYXZvaWRhbmNlIGFsZ29yaXRobXMgZm9yIHNlbmRlci1z aWRlIGFuZCByZWNlaXZlci1zaWRlLiAgVGhlc2UNCj4gYXJlDQo+ID4+Pj4gbGFpZCBvdXQgaW4g MTEyMiwgYnV0IHdlcmUgbm90IHByZXNlbnQgaW4gNzkzLg0KPiA+Pj4+IC4uLg0KPiA+Pj4+DQo+ ID4+Pj4gRm9yIGNvbnZlbmllbmNlLCB0aGUgY29udGVudCBmcm9tIDc5MyB0aGF0IEkgYmVsaWV2 ZSBpcyBubyBsb25nZXIgbmVlZGVkDQo+IGlzDQo+ID4+Pj4gcGFzdGVkIGJlbG93Og0KPiA+Pj4g KzENCj4gPj4gKzENCj4gPj4NCj4gPj4+IEJUVyBvbiBodHRwczovL3Rvb2xzLmlldGYub3JnL2h0 bWwvZHJhZnQtaWV0Zi10Y3BtLXJmYzc5M2Jpcy0wMyNzZWN0aW9uLQ0KPiAzLjguMy4zDQo+ID4+ PiA8cXVvdGU+DQo+ID4+PiAgIEEgVENQIE1BWSBrZWVwIGl0cyBvZmZlcmVkIHJlY2VpdmUgd2lu ZG93IGNsb3NlZCBpbmRlZmluaXRlbHkuICBBcw0KPiA+Pj4gICBsb25nIGFzIHRoZSByZWNlaXZp bmcgVENQIGNvbnRpbnVlcyB0byBzZW5kIGFja25vd2xlZGdtZW50cyBpbg0KPiA+Pj4gICByZXNw b25zZSB0byB0aGUgcHJvYmUgc2VnbWVudHMsIHRoZSBzZW5kaW5nIFRDUCBNVVNUIGFsbG93IHRo ZQ0KPiA+Pj4gICBjb25uZWN0aW9uIHRvIHN0YXkgb3Blbi4NCj4gPj4+IDwvcXVvdGU+DQo+ID4+ Pg0KPiA+Pj4gVGhlIHNlY29uZCB1bmNvbmRpdGlvbmFsIE1VU1Qgb3BlbnMgdXAgYSBzaW1wbGUg YW5kIGVmZmVjdGl2ZSBEb1MNCj4gPj4+IGF0dGFjaywgd2hpY2ggTGludXggbm8gbG9uZ2VyIHdh bnRzIHRvIGJlIHZ1bG5lcmFibGUgb2YuDQo+ID4+PiBodHRwczovL3BhdGNod29yay5vemxhYnMu b3JnL3BhdGNoLzM5MjIxNy8NCj4gPj4gSSBzdHJvbmdseSBkaXNjb3VyYWdlIGF0dGVtcHRzIHRv IHRyeSB0byBidWxsZXRwcm9vZiB1bnNlY3VyZWQgcHJvdG9jb2xzLg0KPiA+Pg0KPiA+PiBJTU8s IHRoZSBhYmlsaXR5IG9mIGFuIGVuZHBvaW50IHRvIGtlZXAgaXRzIHdpbmRvdyBjbG9zZWQgaW5k ZWZpbml0ZWx5DQo+ID4+IGlzIGl0cyBwcmVyb2dhdGl2ZSwganVzdCBhcyBpdCBpcyB0aGUgcHJl cm9nYXRpdmUgb2YgdGhlIG90aGVyIGVuZCB0bw0KPiA+PiBjbG9zZSBzdWNoIGEgY29ubmVjdGlv bi4gQUZBSUNULCB0aGUgTGludXggY29kZSBpcyBjb25zaXN0ZW50IHdpdGggdGhlDQo+ID4+IHNl Y29uZCBoYWxmIG9mIHRoaXMgc3RhdGVtZW50LCBidXQgSSBkb24ndCB0aGluayBpdCBzaG91bGQg YmUgcmVxdWlyZWQNCj4gPj4gb3IgZXZlbiBtZW50aW9uZWQuDQo+ID4+DQo+ID4+IEl0J3MgYWx3 YXlzIG9idmlvdXMgdGhhdCBhbiBlbmRwb2ludCBPUyBjYW4gYW5kIHNob3VsZCBwcm90ZWN0IGl0 cw0KPiA+PiByZXNvdXJjZXMgYnkgd2hhdGV2ZXIgbWVhbnMgaXQgdGhpbmtzIG5lY2Vzc2FyeS4g VGhlc2UgYXJlIG5vdCBwcm90b2NvbA0KPiA+PiBkZWNpc2lvbnMuDQo+ID4+DQo+ID4+IEpvZQ0K PiA+Pg0KPiA+Pg0KPiA+Pg0KPiA+Pj4NCj4gPj4+PiAgICBXaW5kb3cgTWFuYWdlbWVudCBTdWdn ZXN0aW9ucw0KPiA+Pj4+DQo+ID4+Pj4gICAgICBBbGxvY2F0aW5nIGEgdmVyeSBzbWFsbCB3aW5k b3cgY2F1c2VzIGRhdGEgdG8gYmUgdHJhbnNtaXR0ZWQgaW4NCj4gPj4+PiAgICAgIG1hbnkgc21h bGwgc2VnbWVudHMgd2hlbiBiZXR0ZXIgcGVyZm9ybWFuY2UgaXMgYWNoaWV2ZWQgdXNpbmcNCj4g Pj4+PiAgICAgIGZld2VyIGxhcmdlIHNlZ21lbnRzLg0KPiA+Pj4+DQo+ID4+Pj4gICAgICBPbmUg c3VnZ2VzdGlvbiBmb3IgYXZvaWRpbmcgc21hbGwgd2luZG93cyBpcyBmb3IgdGhlIHJlY2VpdmVy IHRvDQo+ID4+Pj4gICAgICBkZWZlciB1cGRhdGluZyBhIHdpbmRvdyB1bnRpbCB0aGUgYWRkaXRp b25hbCBhbGxvY2F0aW9uIGlzIGF0DQo+ID4+Pj4gICAgICBsZWFzdCBYIHBlcmNlbnQgb2YgdGhl IG1heGltdW0gYWxsb2NhdGlvbiBwb3NzaWJsZSBmb3IgdGhlDQo+ID4+Pj4gICAgICBjb25uZWN0 aW9uICh3aGVyZSBYIG1pZ2h0IGJlIDIwIHRvIDQwKS4NCj4gPj4+Pg0KPiA+Pj4+ICAgICAgQW5v dGhlciBzdWdnZXN0aW9uIGlzIGZvciB0aGUgc2VuZGVyIHRvIGF2b2lkIHNlbmRpbmcgc21hbGwN Cj4gPj4+PiAgICAgIHNlZ21lbnRzIGJ5IHdhaXRpbmcgdW50aWwgdGhlIHdpbmRvdyBpcyBsYXJn ZSBlbm91Z2ggYmVmb3JlDQo+ID4+Pj4gICAgICBzZW5kaW5nIGRhdGEuICBJZiB0aGUgdGhlIHVz ZXIgc2lnbmFscyBhIHB1c2ggZnVuY3Rpb24gdGhlbiB0aGUNCj4gPj4+PiAgICAgIGRhdGEgbXVz dCBiZSBzZW50IGV2ZW4gaWYgaXQgaXMgYSBzbWFsbCBzZWdtZW50Lg0KPiA+Pj4+DQo+ID4+Pj4g ICAgICBOb3RlIHRoYXQgdGhlIGFja25vd2xlZGdtZW50cyBzaG91bGQgbm90IGJlIGRlbGF5ZWQg b3IgdW5uZWNlc3NhcnkNCj4gPj4+PiAgICAgIHJldHJhbnNtaXNzaW9ucyB3aWxsIHJlc3VsdC4g IE9uZSBzdHJhdGVneSB3b3VsZCBiZSB0byBzZW5kIGFuDQo+ID4+Pj4gICAgICBhY2tub3dsZWRn bWVudCB3aGVuIGEgc21hbGwgc2VnbWVudCBhcnJpdmVzICh3aXRoIG91dCB1cGRhdGluZyB0aGUN Cj4gPj4+PiAgICAgIHdpbmRvdyBpbmZvcm1hdGlvbiksIGFuZCB0aGVuIHRvIHNlbmQgYW5vdGhl ciBhY2tub3dsZWRnbWVudCB3aXRoDQo+ID4+Pj4gICAgICBuZXcgd2luZG93IGluZm9ybWF0aW9u IHdoZW4gdGhlIHdpbmRvdyBpcyBsYXJnZXIuDQo+ID4+Pj4NCj4gPj4+PiAgICAgIFRoZSBzZWdt ZW50IHNlbnQgdG8gcHJvYmUgYSB6ZXJvIHdpbmRvdyBtYXkgYWxzbyBiZWdpbiBhIGJyZWFrIHVw DQo+ID4+Pj4gICAgICBvZiB0cmFuc21pdHRlZCBkYXRhIGludG8gc21hbGxlciBhbmQgc21hbGxl ciBzZWdtZW50cy4gIElmIGENCj4gPj4+PiAgICAgIHNlZ21lbnQgY29udGFpbmluZyBhIHNpbmds ZSBkYXRhIG9jdGV0IHNlbnQgdG8gcHJvYmUgYSB6ZXJvIHdpbmRvdw0KPiA+Pj4+ICAgICAgaXMg YWNjZXB0ZWQsIGl0IGNvbnN1bWVzIG9uZSBvY3RldCBvZiB0aGUgd2luZG93IG5vdyBhdmFpbGFi bGUuDQo+ID4+Pj4gICAgICBJZiB0aGUgc2VuZGluZyBUQ1Agc2ltcGx5IHNlbmRzIGFzIG11Y2gg YXMgaXQgY2FuIHdoZW5ldmVyIHRoZQ0KPiA+Pj4+ICAgICAgd2luZG93IGlzIG5vbiB6ZXJvLCB0 aGUgdHJhbnNtaXR0ZWQgZGF0YSB3aWxsIGJlIGJyb2tlbiBpbnRvDQo+ID4+Pj4gICAgICBhbHRl cm5hdGluZyBiaWcgYW5kIHNtYWxsIHNlZ21lbnRzLiAgQXMgdGltZSBnb2VzIG9uLCBvY2Nhc2lv bmFsDQo+ID4+Pj4gICAgICBwYXVzZXMgaW4gdGhlIHJlY2VpdmVyIG1ha2luZyB3aW5kb3cgYWxs b2NhdGlvbiBhdmFpbGFibGUgd2lsbA0KPiA+Pj4+ICAgICAgcmVzdWx0IGluIGJyZWFraW5nIHRo ZSBiaWcgc2VnbWVudHMgaW50byBhIHNtYWxsIGFuZCBub3QgcXVpdGUgc28NCj4gPj4+PiAgICAg IGJpZyBwYWlyLiBBbmQgYWZ0ZXIgYSB3aGlsZSB0aGUgZGF0YSB0cmFuc21pc3Npb24gd2lsbCBi ZSBpbg0KPiA+Pj4+ICAgICAgbW9zdGx5IHNtYWxsIHNlZ21lbnRzLg0KPiA+Pj4+DQo+ID4+Pj4g ICAgICBUaGUgc3VnZ2VzdGlvbiBoZXJlIGlzIHRoYXQgdGhlIFRDUCBpbXBsZW1lbnRhdGlvbnMg bmVlZCB0bw0KPiA+Pj4+ICAgICAgYWN0aXZlbHkgYXR0ZW1wdCB0byBjb21iaW5lIHNtYWxsIHdp bmRvdyBhbGxvY2F0aW9ucyBpbnRvIGxhcmdlcg0KPiA+Pj4+ICAgICAgd2luZG93cywgc2luY2Ug dGhlIG1lY2hhbmlzbXMgZm9yIG1hbmFnaW5nIHRoZSB3aW5kb3cgdGVuZCB0byBsZWFkDQo+ID4+ Pj4gICAgICB0byBtYW55IHNtYWxsIHdpbmRvd3MgaW4gdGhlIHNpbXBsZXN0IG1pbmRlZCBpbXBs ZW1lbnRhdGlvbnMuDQo+ID4+Pj4NCj4gPj4+Pg0KPiA+Pj4+IEFueSB0aG91Z2h0cyBvciBmZWVk YmFjayBvbiByZW1vdmluZyB0aGlzIGZyb20gNzkzYmlzIHdpbGwgYmUNCj4gYXBwcmVjaWF0ZWQs DQo+ID4+Pj4gdGhhbmtzIQ0KPiA+Pj4+DQo+ID4+Pj4NCj4gPj4+PiBfX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXw0KPiA+Pj4+IHRjcG0gbWFpbGluZyBsaXN0 DQo+ID4+Pj4gdGNwbUBpZXRmLm9yZw0KPiA+Pj4+IGh0dHBzOi8vd3d3LmlldGYub3JnL21haWxt YW4vbGlzdGluZm8vdGNwbQ0KPiA+Pj4gX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX18NCj4gPj4+IHRjcG0gbWFpbGluZyBsaXN0DQo+ID4+PiB0Y3BtQGlldGYu b3JnDQo+ID4+PiBodHRwczovL3d3dy5pZXRmLm9yZy9tYWlsbWFuL2xpc3RpbmZvL3RjcG0NCj4g Pj4gX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18NCj4gPj4g dGNwbSBtYWlsaW5nIGxpc3QNCj4gPj4gdGNwbUBpZXRmLm9yZw0KPiA+PiBodHRwczovL3d3dy5p ZXRmLm9yZy9tYWlsbWFuL2xpc3RpbmZvL3RjcG0NCj4gPiBfX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fXw0KPiA+IHRjcG0gbWFpbGluZyBsaXN0DQo+ID4gdGNw bUBpZXRmLm9yZw0KPiA+IGh0dHBzOi8vd3d3LmlldGYub3JnL21haWxtYW4vbGlzdGluZm8vdGNw bQ0KPiANCj4gX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18N Cj4gdGNwbSBtYWlsaW5nIGxpc3QNCj4gdGNwbUBpZXRmLm9yZw0KPiBodHRwczovL3d3dy5pZXRm Lm9yZy9tYWlsbWFuL2xpc3RpbmZvL3RjcG0NCg== From nobody Tue Aug 2 13:32:50 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B67AA12D190 for ; Tue, 2 Aug 2016 13:32:48 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -8.187 X-Spam-Level: X-Spam-Status: No, score=-8.187 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, RP_MATCHES_RCVD=-1.287] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id pRs9xpZzeENC for ; Tue, 2 Aug 2016 13:32:47 -0700 (PDT) Received: from vapor.isi.edu (vapor.isi.edu [128.9.64.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1D7C012B039 for ; Tue, 2 Aug 2016 13:32:47 -0700 (PDT) Received: from [128.9.184.236] ([128.9.184.236]) (authenticated bits=0) by vapor.isi.edu (8.13.8/8.13.8) with ESMTP id u72KVcRR026072 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Tue, 2 Aug 2016 13:31:40 -0700 (PDT) To: Christian Huitema , David Borman References: <57A0FA15.6090507@isi.edu> <57A0FFE0.7020905@isi.edu> From: Joe Touch Message-ID: <57A1032A.7010300@isi.edu> Date: Tue, 2 Aug 2016 13:31:38 -0700 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.7.2 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Archived-At: Cc: "tcpm@ietf.org Extensions" Subject: Re: [tcpm] 793bis: removing "Window Management Suggestions" X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2016 20:32:49 -0000 On 8/2/2016 1:25 PM, Christian Huitema wrote: > At a minimum, that means switching the language from "MUST allow the connection to stay open" to "SHOULD allow the connection to stay open" maybe with a caveat like "unless server size resource management requires that it be closed." I disagree. The entirety of every protocol would turn into "SHOULD, unless resources become restricted". That's OS design, not protocol design. > And the printer-out-of-paper scenario really ought to be solved at the application layer. After all, if you are waiting for a printout, knowing why you are waiting is somewhat helpful... It should, but that's another reason not to build a solution into the protocol. Joe > > -- Christian Huitema > > > >> -----Original Message----- >> From: tcpm [mailto:tcpm-bounces@ietf.org] On Behalf Of Joe Touch >> Sent: Tuesday, August 2, 2016 1:18 PM >> To: David Borman >> Cc: tcpm@ietf.org Extensions >> Subject: Re: [tcpm] 793bis: removing "Window Management Suggestions" >> >> Hi, David, >> >> I agree. IMO, DOS situations should be handled by the OS by withdrawing >> resources *as needed*. That includes closing down zero-window situations >> as below, as well as closing idle connections when resources are limited. >> >> I don't see this as something the protocol needs to decide. >> >> Joe >> >> On 8/2/2016 1:14 PM, David Borman wrote: >>> The point about keeping the connection open in the face of zero window >> probes has the classic example of sending a job to the printer just before >> heading off to lunch, and when you come back later discover that the printer >> ran out of paper and stopped accepting new TCP data. The printer keeps >> responding to the probes with a zero window, and when paper is finally added >> to the printer, printing resumes, the TCP window is opened and the job >> completes. >>> Years ago there were examples of situations like this where TCP connections >> were being timed out relatively quickly in the face of a zero window, causing >> failures when there was no need for a failure. It’d be nice if trying to address a >> potential DOS situation didn’t cause normal situations to regress back to those >> days; it then becomes a matter of balancing how long to wait before giving up >> on a zero window connection. >>> -David Borman >>> >>>> On Aug 2, 2016, at 2:52 PM, Joe Touch wrote: >>>> >>>> >>>> >>>> On 8/2/2016 12:42 PM, Yuchung Cheng wrote: >>>>> On Tue, Aug 2, 2016 at 12:15 PM, Wesley Eddy >> wrote: >>>>>> In the latest revision of the 793bis draft, I added content from RFC 1122 >> on >>>>>> the SWS avoidance algorithms for sender-side and receiver-side. These >> are >>>>>> laid out in 1122, but were not present in 793. >>>>>> ... >>>>>> >>>>>> For convenience, the content from 793 that I believe is no longer needed >> is >>>>>> pasted below: >>>>> +1 >>>> +1 >>>> >>>>> BTW on https://tools.ietf.org/html/draft-ietf-tcpm-rfc793bis-03#section- >> 3.8.3.3 >>>>> >>>>> A TCP MAY keep its offered receive window closed indefinitely. As >>>>> long as the receiving TCP continues to send acknowledgments in >>>>> response to the probe segments, the sending TCP MUST allow the >>>>> connection to stay open. >>>>> >>>>> >>>>> The second unconditional MUST opens up a simple and effective DoS >>>>> attack, which Linux no longer wants to be vulnerable of. >>>>> https://patchwork.ozlabs.org/patch/392217/ >>>> I strongly discourage attempts to try to bulletproof unsecured protocols. >>>> >>>> IMO, the ability of an endpoint to keep its window closed indefinitely >>>> is its prerogative, just as it is the prerogative of the other end to >>>> close such a connection. AFAICT, the Linux code is consistent with the >>>> second half of this statement, but I don't think it should be required >>>> or even mentioned. >>>> >>>> It's always obvious that an endpoint OS can and should protect its >>>> resources by whatever means it thinks necessary. These are not protocol >>>> decisions. >>>> >>>> Joe >>>> >>>> >>>> >>>>>> Window Management Suggestions >>>>>> >>>>>> Allocating a very small window causes data to be transmitted in >>>>>> many small segments when better performance is achieved using >>>>>> fewer large segments. >>>>>> >>>>>> One suggestion for avoiding small windows is for the receiver to >>>>>> defer updating a window until the additional allocation is at >>>>>> least X percent of the maximum allocation possible for the >>>>>> connection (where X might be 20 to 40). >>>>>> >>>>>> Another suggestion is for the sender to avoid sending small >>>>>> segments by waiting until the window is large enough before >>>>>> sending data. If the the user signals a push function then the >>>>>> data must be sent even if it is a small segment. >>>>>> >>>>>> Note that the acknowledgments should not be delayed or unnecessary >>>>>> retransmissions will result. One strategy would be to send an >>>>>> acknowledgment when a small segment arrives (with out updating the >>>>>> window information), and then to send another acknowledgment with >>>>>> new window information when the window is larger. >>>>>> >>>>>> The segment sent to probe a zero window may also begin a break up >>>>>> of transmitted data into smaller and smaller segments. If a >>>>>> segment containing a single data octet sent to probe a zero window >>>>>> is accepted, it consumes one octet of the window now available. >>>>>> If the sending TCP simply sends as much as it can whenever the >>>>>> window is non zero, the transmitted data will be broken into >>>>>> alternating big and small segments. As time goes on, occasional >>>>>> pauses in the receiver making window allocation available will >>>>>> result in breaking the big segments into a small and not quite so >>>>>> big pair. And after a while the data transmission will be in >>>>>> mostly small segments. >>>>>> >>>>>> The suggestion here is that the TCP implementations need to >>>>>> actively attempt to combine small window allocations into larger >>>>>> windows, since the mechanisms for managing the window tend to lead >>>>>> to many small windows in the simplest minded implementations. >>>>>> >>>>>> >>>>>> Any thoughts or feedback on removing this from 793bis will be >> appreciated, >>>>>> thanks! >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> tcpm mailing list >>>>>> tcpm@ietf.org >>>>>> https://www.ietf.org/mailman/listinfo/tcpm >>>>> _______________________________________________ >>>>> tcpm mailing list >>>>> tcpm@ietf.org >>>>> https://www.ietf.org/mailman/listinfo/tcpm >>>> _______________________________________________ >>>> tcpm mailing list >>>> tcpm@ietf.org >>>> https://www.ietf.org/mailman/listinfo/tcpm >>> _______________________________________________ >>> tcpm mailing list >>> tcpm@ietf.org >>> https://www.ietf.org/mailman/listinfo/tcpm >> _______________________________________________ >> tcpm mailing list >> tcpm@ietf.org >> https://www.ietf.org/mailman/listinfo/tcpm From nobody Tue Aug 2 13:37:03 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5A22A12D76E for ; Tue, 2 Aug 2016 13:37:01 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -8.187 X-Spam-Level: X-Spam-Status: No, score=-8.187 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, RP_MATCHES_RCVD=-1.287] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8YaWxKyWo7lu for ; Tue, 2 Aug 2016 13:37:00 -0700 (PDT) Received: from vapor.isi.edu (vapor.isi.edu [128.9.64.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1C43012D186 for ; Tue, 2 Aug 2016 13:37:00 -0700 (PDT) Received: from [128.9.184.236] ([128.9.184.236]) (authenticated bits=0) by vapor.isi.edu (8.13.8/8.13.8) with ESMTP id u72Ka2cD027475 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Tue, 2 Aug 2016 13:36:03 -0700 (PDT) To: Yuchung Cheng , David Borman References: <57A0FA15.6090507@isi.edu> From: Joe Touch Message-ID: <57A10431.7040607@isi.edu> Date: Tue, 2 Aug 2016 13:36:01 -0700 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.7.2 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Archived-At: Cc: "tcpm@ietf.org Extensions" Subject: Re: [tcpm] 793bis: removing "Window Management Suggestions" X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2016 20:37:01 -0000 On 8/2/2016 1:22 PM, Yuchung Cheng wrote: > On Tue, Aug 2, 2016 at 1:14 PM, David Borman wrote: >> > The point about keeping the connection open in the face of zero window probes has the classic example of sending a job to the printer just before heading off to lunch, and when you come back later discover that the printer ran out of paper and stopped accepting new TCP data. The printer keeps responding to the probes with a zero window, and when paper is finally added to the printer, printing resumes, the TCP window is opened and the job completes. >> > >> > Years ago there were examples of situations like this where TCP connections were being timed out relatively quickly in the face of a zero window, causing failures when there was no need for a failure. It’d be nice if trying to address a potential DOS situation didn’t cause normal situations to regress back to those days; it then becomes a matter of balancing how long to wait before giving up on a zero window connection. >> > >> > -David Borman > I am well aware of the printer story and which part in my original > email did I advocate to remove the MUST? > > Linux only chops the ropes when these connections have no real owners > (i.e., orphaned socket in Linux) and the amount exceeds a configurable > system limit. I am mentioning there is a real attack behind this > simple MUST. There MIGHT BE an attack. There might not. Just because an endpoint does something you didn't expect, does not mean it is malicious. A variant of the Postel Principle applies here - IMO, anything a benign endpoint CAN do MUST NOT be interpreted as an attack. A node has the right to protect itself and its resources, but should not infer malice where it isn't known to exist. It might be worth noting somewhere the ways in which an *implementation* of TCP can help protect resources, but that's an implementation suggestion, not protocol guidance. Joe From nobody Tue Aug 2 13:56:52 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D4B3212D924 for ; Tue, 2 Aug 2016 13:56:50 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.187 X-Spam-Level: X-Spam-Status: No, score=-3.187 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RP_MATCHES_RCVD=-1.287] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GxhE6Y5g6vTl for ; Tue, 2 Aug 2016 13:56:49 -0700 (PDT) Received: from frantic.weston.borman.com (frantic.weston.borman.com [70.57.156.33]) (using TLSv1 with cipher EDH-RSA-DES-CBC3-SHA (112/168 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 693B412D5DE for ; Tue, 2 Aug 2016 13:56:48 -0700 (PDT) Received: from [127.0.0.1] (frantic.weston.borman.com [70.57.156.33]) by frantic.weston.borman.com (8.14.7/8.14.7) with ESMTP id u72Kuipu025292; Tue, 2 Aug 2016 15:56:45 -0500 (CDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) From: David Borman In-Reply-To: Date: Tue, 2 Aug 2016 15:56:44 -0500 Content-Transfer-Encoding: quoted-printable Message-Id: References: <57A0FA15.6090507@isi.edu> To: Yuchung Cheng X-Mailer: Apple Mail (2.3124) Archived-At: Cc: "tcpm@ietf.org Extensions" , Joe Touch Subject: Re: [tcpm] 793bis: removing "Window Management Suggestions" X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2016 20:56:51 -0000 Sorry, my intention was to provide context on the mailing list for why = that MUST is there, not necessarily aimed at you, but for anyone else = that is reading this list and might not be familiar with the background. I think it should remain a MUST, but implementations are always free to = do what they need to do when under attack or run out of resources. -David > On Aug 2, 2016, at 3:22 PM, Yuchung Cheng wrote: >=20 > On Tue, Aug 2, 2016 at 1:14 PM, David Borman = wrote: >> The point about keeping the connection open in the face of zero = window probes has the classic example of sending a job to the printer = just before heading off to lunch, and when you come back later discover = that the printer ran out of paper and stopped accepting new TCP data. = The printer keeps responding to the probes with a zero window, and when = paper is finally added to the printer, printing resumes, the TCP window = is opened and the job completes. >>=20 >> Years ago there were examples of situations like this where TCP = connections were being timed out relatively quickly in the face of a = zero window, causing failures when there was no need for a failure. = It=E2=80=99d be nice if trying to address a potential DOS situation = didn=E2=80=99t cause normal situations to regress back to those days; it = then becomes a matter of balancing how long to wait before giving up on = a zero window connection. >>=20 >> -David Borman > I am well aware of the printer story and which part in my original > email did I advocate to remove the MUST? >=20 > Linux only chops the ropes when these connections have no real owners > (i.e., orphaned socket in Linux) and the amount exceeds a configurable > system limit. I am mentioning there is a real attack behind this > simple MUST. >=20 >>=20 >>> On Aug 2, 2016, at 2:52 PM, Joe Touch wrote: >>>=20 >>>=20 >>>=20 >>> On 8/2/2016 12:42 PM, Yuchung Cheng wrote: >>>> On Tue, Aug 2, 2016 at 12:15 PM, Wesley Eddy = wrote: >>>>> In the latest revision of the 793bis draft, I added content from = RFC 1122 on >>>>> the SWS avoidance algorithms for sender-side and receiver-side. = These are >>>>> laid out in 1122, but were not present in 793. >>>>> ... >>>>>=20 >>>>> For convenience, the content from 793 that I believe is no longer = needed is >>>>> pasted below: >>>> +1 >>> +1 >>>=20 >>>> BTW on = https://tools.ietf.org/html/draft-ietf-tcpm-rfc793bis-03#section-3.8.3.3 >>>> >>>> A TCP MAY keep its offered receive window closed indefinitely. As >>>> long as the receiving TCP continues to send acknowledgments in >>>> response to the probe segments, the sending TCP MUST allow the >>>> connection to stay open. >>>> >>>>=20 >>>> The second unconditional MUST opens up a simple and effective DoS >>>> attack, which Linux no longer wants to be vulnerable of. >>>> https://patchwork.ozlabs.org/patch/392217/ >>>=20 >>> I strongly discourage attempts to try to bulletproof unsecured = protocols. >>>=20 >>> IMO, the ability of an endpoint to keep its window closed = indefinitely >>> is its prerogative, just as it is the prerogative of the other end = to >>> close such a connection. AFAICT, the Linux code is consistent with = the >>> second half of this statement, but I don't think it should be = required >>> or even mentioned. >>>=20 >>> It's always obvious that an endpoint OS can and should protect its >>> resources by whatever means it thinks necessary. These are not = protocol >>> decisions. >>>=20 >>> Joe >>>=20 >>>=20 >>>=20 >>>>=20 >>>>=20 >>>>> Window Management Suggestions >>>>>=20 >>>>> Allocating a very small window causes data to be transmitted = in >>>>> many small segments when better performance is achieved using >>>>> fewer large segments. >>>>>=20 >>>>> One suggestion for avoiding small windows is for the receiver = to >>>>> defer updating a window until the additional allocation is at >>>>> least X percent of the maximum allocation possible for the >>>>> connection (where X might be 20 to 40). >>>>>=20 >>>>> Another suggestion is for the sender to avoid sending small >>>>> segments by waiting until the window is large enough before >>>>> sending data. If the the user signals a push function then = the >>>>> data must be sent even if it is a small segment. >>>>>=20 >>>>> Note that the acknowledgments should not be delayed or = unnecessary >>>>> retransmissions will result. One strategy would be to send an >>>>> acknowledgment when a small segment arrives (with out updating = the >>>>> window information), and then to send another acknowledgment = with >>>>> new window information when the window is larger. >>>>>=20 >>>>> The segment sent to probe a zero window may also begin a break = up >>>>> of transmitted data into smaller and smaller segments. If a >>>>> segment containing a single data octet sent to probe a zero = window >>>>> is accepted, it consumes one octet of the window now = available. >>>>> If the sending TCP simply sends as much as it can whenever the >>>>> window is non zero, the transmitted data will be broken into >>>>> alternating big and small segments. As time goes on, = occasional >>>>> pauses in the receiver making window allocation available will >>>>> result in breaking the big segments into a small and not quite = so >>>>> big pair. And after a while the data transmission will be in >>>>> mostly small segments. >>>>>=20 >>>>> The suggestion here is that the TCP implementations need to >>>>> actively attempt to combine small window allocations into = larger >>>>> windows, since the mechanisms for managing the window tend to = lead >>>>> to many small windows in the simplest minded implementations. >>>>>=20 >>>>>=20 >>>>> Any thoughts or feedback on removing this from 793bis will be = appreciated, >>>>> thanks! >>>>>=20 >>>>>=20 >>>>> _______________________________________________ >>>>> tcpm mailing list >>>>> tcpm@ietf.org >>>>> https://www.ietf.org/mailman/listinfo/tcpm >>>> _______________________________________________ >>>> tcpm mailing list >>>> tcpm@ietf.org >>>> https://www.ietf.org/mailman/listinfo/tcpm >>>=20 >>> _______________________________________________ >>> tcpm mailing list >>> tcpm@ietf.org >>> https://www.ietf.org/mailman/listinfo/tcpm >>=20 >=20 > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm From nobody Tue Aug 2 14:15:00 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 93C1812D877 for ; Tue, 2 Aug 2016 14:14:58 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.999 X-Spam-Level: X-Spam-Status: No, score=-1.999 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Tqxpgbpei6kF for ; Tue, 2 Aug 2016 14:14:55 -0700 (PDT) Received: from mail-ua0-x22e.google.com (mail-ua0-x22e.google.com [IPv6:2607:f8b0:400c:c08::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 4334412D80E for ; Tue, 2 Aug 2016 14:14:52 -0700 (PDT) Received: by mail-ua0-x22e.google.com with SMTP id j59so138531796uaj.3 for ; Tue, 02 Aug 2016 14:14:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=1P1LSq2CQ7FL525xRzJVuE86PugvI2+y+btskXadUC8=; b=ygFvjVtGNtEKe+gqqezCMO8oEZ4wQM8dU47HfF/uTSfOJlJ4wpFcUYdqXoSRCY1Sws zsN/7YACIITpY5aaJUjl14ypmaYFexx3iu0rTYl3GVI4HzAENqEvEG1G3wt8aRmzUfxF gyAzX7M419qqHQzrd3DRu0fN0VPOJwvKMbu0MvEF3lyaE526IjiC5CIxfOGI7uo0vA2I EtEOTyrA1Qs1Be4nhAIaBpff2CdyOceOvlHBFMPlyQ7MCNt8kB36YmQLqFLGshem/M62 DTcT8qUuZy2QxT3J5sw2iLR2sa6w3gIpw7o00SDUD/FaSfAjo0tMCAgFI9uQPTF/WwXt BO8A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=1P1LSq2CQ7FL525xRzJVuE86PugvI2+y+btskXadUC8=; b=kn0iE9TX3j7qGRujV9BZvTKybyOG8BJl2y7lMDCDVoxkfNai1hkp36j1AWqbnWx91k iwiMo1KDHP00ci1DhQ4PNzS0EhBL9dXeIeurC4pcQ43NkvQC4iViAhFgZpHjm53Qx/bk B/3SqbrjgUZufSUaozwY9QnH2je8s7tcIYXSJU8nnwfMQw960FhS2VtwI+pgnp/D7AQq +uGLD4UoNTNGJiF0h8SWLvLwk2Y4+QdJBi9160vd9P92HcRHdXDGTx3D5xMuZPRzjjcY 9MwPrgz6EOzjsh2qUVayA70j1ODnP+tAn7PLrYngT7zr0T81ntg95SGd83TGg0huA9MK RoUQ== X-Gm-Message-State: AEkoous+AbUpa9Zee4tmTATlW59fCVLustrWzM7LOuf/NSWZEbchwlojto0YlACgtW39AX1nJpghTKK3rzpARQ== X-Received: by 10.159.53.2 with SMTP id o2mr26792301uao.18.1470172491274; Tue, 02 Aug 2016 14:14:51 -0700 (PDT) MIME-Version: 1.0 Received: by 10.176.80.42 with HTTP; Tue, 2 Aug 2016 14:14:50 -0700 (PDT) In-Reply-To: References: <57A0FA15.6090507@isi.edu> From: John Heffner Date: Tue, 2 Aug 2016 17:14:50 -0400 Message-ID: To: David Borman Content-Type: multipart/alternative; boundary=94eb2c04c6fe19dd5c05391d351c Archived-At: Cc: "tcpm@ietf.org Extensions" , Joe Touch Subject: Re: [tcpm] 793bis: removing "Window Management Suggestions" X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2016 21:14:59 -0000 --94eb2c04c6fe19dd5c05391d351c Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable This reminds me a lot of an old thread: https://www.ietf.org/mail-archive/web/tcpm/current/msg04040.html On Tue, Aug 2, 2016 at 4:56 PM, David Borman wrote: > Sorry, my intention was to provide context on the mailing list for why > that MUST is there, not necessarily aimed at you, but for anyone else tha= t > is reading this list and might not be familiar with the background. > > I think it should remain a MUST, but implementations are always free to d= o > what they need to do when under attack or run out of resources. > -David > > > On Aug 2, 2016, at 3:22 PM, Yuchung Cheng wrote: > > > > On Tue, Aug 2, 2016 at 1:14 PM, David Borman > wrote: > >> The point about keeping the connection open in the face of zero window > probes has the classic example of sending a job to the printer just befor= e > heading off to lunch, and when you come back later discover that the > printer ran out of paper and stopped accepting new TCP data. The printer > keeps responding to the probes with a zero window, and when paper is > finally added to the printer, printing resumes, the TCP window is opened > and the job completes. > >> > >> Years ago there were examples of situations like this where TCP > connections were being timed out relatively quickly in the face of a zero > window, causing failures when there was no need for a failure. It=E2=80= =99d be > nice if trying to address a potential DOS situation didn=E2=80=99t cause = normal > situations to regress back to those days; it then becomes a matter of > balancing how long to wait before giving up on a zero window connection. > >> > >> -David Borman > > I am well aware of the printer story and which part in my original > > email did I advocate to remove the MUST? > > > > Linux only chops the ropes when these connections have no real owners > > (i.e., orphaned socket in Linux) and the amount exceeds a configurable > > system limit. I am mentioning there is a real attack behind this > > simple MUST. > > > >> > >>> On Aug 2, 2016, at 2:52 PM, Joe Touch wrote: > >>> > >>> > >>> > >>> On 8/2/2016 12:42 PM, Yuchung Cheng wrote: > >>>> On Tue, Aug 2, 2016 at 12:15 PM, Wesley Eddy > wrote: > >>>>> In the latest revision of the 793bis draft, I added content from RF= C > 1122 on > >>>>> the SWS avoidance algorithms for sender-side and receiver-side. > These are > >>>>> laid out in 1122, but were not present in 793. > >>>>> ... > >>>>> > >>>>> For convenience, the content from 793 that I believe is no longer > needed is > >>>>> pasted below: > >>>> +1 > >>> +1 > >>> > >>>> BTW on > https://tools.ietf.org/html/draft-ietf-tcpm-rfc793bis-03#section-3.8.3.3 > >>>> > >>>> A TCP MAY keep its offered receive window closed indefinitely. As > >>>> long as the receiving TCP continues to send acknowledgments in > >>>> response to the probe segments, the sending TCP MUST allow the > >>>> connection to stay open. > >>>> > >>>> > >>>> The second unconditional MUST opens up a simple and effective DoS > >>>> attack, which Linux no longer wants to be vulnerable of. > >>>> https://patchwork.ozlabs.org/patch/392217/ > >>> > >>> I strongly discourage attempts to try to bulletproof unsecured > protocols. > >>> > >>> IMO, the ability of an endpoint to keep its window closed indefinitel= y > >>> is its prerogative, just as it is the prerogative of the other end to > >>> close such a connection. AFAICT, the Linux code is consistent with th= e > >>> second half of this statement, but I don't think it should be require= d > >>> or even mentioned. > >>> > >>> It's always obvious that an endpoint OS can and should protect its > >>> resources by whatever means it thinks necessary. These are not protoc= ol > >>> decisions. > >>> > >>> Joe > >>> > >>> > >>> > >>>> > >>>> > >>>>> Window Management Suggestions > >>>>> > >>>>> Allocating a very small window causes data to be transmitted in > >>>>> many small segments when better performance is achieved using > >>>>> fewer large segments. > >>>>> > >>>>> One suggestion for avoiding small windows is for the receiver t= o > >>>>> defer updating a window until the additional allocation is at > >>>>> least X percent of the maximum allocation possible for the > >>>>> connection (where X might be 20 to 40). > >>>>> > >>>>> Another suggestion is for the sender to avoid sending small > >>>>> segments by waiting until the window is large enough before > >>>>> sending data. If the the user signals a push function then the > >>>>> data must be sent even if it is a small segment. > >>>>> > >>>>> Note that the acknowledgments should not be delayed or > unnecessary > >>>>> retransmissions will result. One strategy would be to send an > >>>>> acknowledgment when a small segment arrives (with out updating > the > >>>>> window information), and then to send another acknowledgment wi= th > >>>>> new window information when the window is larger. > >>>>> > >>>>> The segment sent to probe a zero window may also begin a break = up > >>>>> of transmitted data into smaller and smaller segments. If a > >>>>> segment containing a single data octet sent to probe a zero > window > >>>>> is accepted, it consumes one octet of the window now available. > >>>>> If the sending TCP simply sends as much as it can whenever the > >>>>> window is non zero, the transmitted data will be broken into > >>>>> alternating big and small segments. As time goes on, occasiona= l > >>>>> pauses in the receiver making window allocation available will > >>>>> result in breaking the big segments into a small and not quite = so > >>>>> big pair. And after a while the data transmission will be in > >>>>> mostly small segments. > >>>>> > >>>>> The suggestion here is that the TCP implementations need to > >>>>> actively attempt to combine small window allocations into large= r > >>>>> windows, since the mechanisms for managing the window tend to > lead > >>>>> to many small windows in the simplest minded implementations. > >>>>> > >>>>> > >>>>> Any thoughts or feedback on removing this from 793bis will be > appreciated, > >>>>> thanks! > >>>>> > >>>>> > >>>>> _______________________________________________ > >>>>> tcpm mailing list > >>>>> tcpm@ietf.org > >>>>> https://www.ietf.org/mailman/listinfo/tcpm > >>>> _______________________________________________ > >>>> tcpm mailing list > >>>> tcpm@ietf.org > >>>> https://www.ietf.org/mailman/listinfo/tcpm > >>> > >>> _______________________________________________ > >>> tcpm mailing list > >>> tcpm@ietf.org > >>> https://www.ietf.org/mailman/listinfo/tcpm > >> > > > > _______________________________________________ > > tcpm mailing list > > tcpm@ietf.org > > https://www.ietf.org/mailman/listinfo/tcpm > > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm > --94eb2c04c6fe19dd5c05391d351c Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
This reminds me a lot of an old thread:=C2=A0https://www= .ietf.org/mail-archive/web/tcpm/current/msg04040.html

On Tue, Aug 2, 2016 at 4:56= PM, David Borman <dab@weston.borman.com> wrote:
Sorry, my intention was to provide context on th= e mailing list for why that MUST is there, not necessarily aimed at you, bu= t for anyone else that is reading this list and might not be familiar with = the background.

I think it should remain a MUST, but implementations are always free to do = what they need to do when under attack or run out of resources.
=C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 -David

> On Aug 2, 2016, at 3:22 PM, Yuchung Cheng <ycheng@google.com> wrote:
>
> On Tue, Aug 2, 2016 at 1:14 PM, David Borman <dab@weston.borman.com> wrote:
>> The point about keeping the connection open in the face of zero wi= ndow probes has the classic example of sending a job to the printer just be= fore heading off to lunch, and when you come back later discover that the p= rinter ran out of paper and stopped accepting new TCP data.=C2=A0 The print= er keeps responding to the probes with a zero window, and when paper is fin= ally added to the printer, printing resumes, the TCP window is opened and t= he job completes.
>>
>> Years ago there were examples of situations like this where TCP co= nnections were being timed out relatively quickly in the face of a zero win= dow, causing failures when there was no need for a failure.=C2=A0 It=E2=80= =99d be nice if trying to address a potential DOS situation didn=E2=80=99t = cause normal situations to regress back to those days; it then becomes a ma= tter of balancing how long to wait before giving up on a zero window connec= tion.
>>
>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 -David Borman
> I am well aware of the printer story and which part in my original
> email did I advocate to remove the MUST?
>
> Linux only chops the ropes when these connections have no real owners<= br> > (i.e., orphaned socket in Linux) and the amount exceeds a configurable=
> system limit.=C2=A0 I am mentioning there is a real attack behind this=
> simple MUST.
>
>>
>>> On Aug 2, 2016, at 2:52 PM, Joe Touch <touch@ISI.EDU> wrote:
>>>
>>>
>>>
>>> On 8/2/2016 12:42 PM, Yuchung Cheng wrote:
>>>> On Tue, Aug 2, 2016 at 12:15 PM, Wesley Eddy <wes@mti-systems.com> wrote:
>>>>> In the latest revision of the 793bis draft, I added co= ntent from RFC 1122 on
>>>>> the SWS avoidance algorithms for sender-side and recei= ver-side.=C2=A0 These are
>>>>> laid out in 1122, but were not present in 793.
>>>>> ...
>>>>>
>>>>> For convenience, the content from 793 that I believe i= s no longer needed is
>>>>> pasted below:
>>>> +1
>>> +1
>>>
>>>> BTW on http= s://tools.ietf.org/html/draft-ietf-tcpm-rfc793bis-03#section-3.8.3.3 >>>> <quote>
>>>>=C2=A0 A TCP MAY keep its offered receive window closed ind= efinitely.=C2=A0 As
>>>>=C2=A0 long as the receiving TCP continues to send acknowle= dgments in
>>>>=C2=A0 response to the probe segments, the sending TCP MUST= allow the
>>>>=C2=A0 connection to stay open.
>>>> </quote>
>>>>
>>>> The second unconditional MUST opens up a simple and effect= ive DoS
>>>> attack, which Linux no longer wants to be vulnerable of. >>>> https://patchwork.ozlabs.org/patch/392217= /
>>>
>>> I strongly discourage attempts to try to bulletproof unsecured= protocols.
>>>
>>> IMO, the ability of an endpoint to keep its window closed inde= finitely
>>> is its prerogative, just as it is the prerogative of the other= end to
>>> close such a connection. AFAICT, the Linux code is consistent = with the
>>> second half of this statement, but I don't think it should= be required
>>> or even mentioned.
>>>
>>> It's always obvious that an endpoint OS can and should pro= tect its
>>> resources by whatever means it thinks necessary. These are not= protocol
>>> decisions.
>>>
>>> Joe
>>>
>>>
>>>
>>>>
>>>>
>>>>>=C2=A0 =C2=A0Window Management Suggestions
>>>>>
>>>>>=C2=A0 =C2=A0 =C2=A0Allocating a very small window caus= es data to be transmitted in
>>>>>=C2=A0 =C2=A0 =C2=A0many small segments when better per= formance is achieved using
>>>>>=C2=A0 =C2=A0 =C2=A0fewer large segments.
>>>>>
>>>>>=C2=A0 =C2=A0 =C2=A0One suggestion for avoiding small w= indows is for the receiver to
>>>>>=C2=A0 =C2=A0 =C2=A0defer updating a window until the a= dditional allocation is at
>>>>>=C2=A0 =C2=A0 =C2=A0least X percent of the maximum allo= cation possible for the
>>>>>=C2=A0 =C2=A0 =C2=A0connection (where X might be 20 to = 40).
>>>>>
>>>>>=C2=A0 =C2=A0 =C2=A0Another suggestion is for the sende= r to avoid sending small
>>>>>=C2=A0 =C2=A0 =C2=A0segments by waiting until the windo= w is large enough before
>>>>>=C2=A0 =C2=A0 =C2=A0sending data.=C2=A0 If the the user= signals a push function then the
>>>>>=C2=A0 =C2=A0 =C2=A0data must be sent even if it is a s= mall segment.
>>>>>
>>>>>=C2=A0 =C2=A0 =C2=A0Note that the acknowledgments shoul= d not be delayed or unnecessary
>>>>>=C2=A0 =C2=A0 =C2=A0retransmissions will result.=C2=A0 = One strategy would be to send an
>>>>>=C2=A0 =C2=A0 =C2=A0acknowledgment when a small segment= arrives (with out updating the
>>>>>=C2=A0 =C2=A0 =C2=A0window information), and then to se= nd another acknowledgment with
>>>>>=C2=A0 =C2=A0 =C2=A0new window information when the win= dow is larger.
>>>>>
>>>>>=C2=A0 =C2=A0 =C2=A0The segment sent to probe a zero wi= ndow may also begin a break up
>>>>>=C2=A0 =C2=A0 =C2=A0of transmitted data into smaller an= d smaller segments.=C2=A0 If a
>>>>>=C2=A0 =C2=A0 =C2=A0segment containing a single data oc= tet sent to probe a zero window
>>>>>=C2=A0 =C2=A0 =C2=A0is accepted, it consumes one octet = of the window now available.
>>>>>=C2=A0 =C2=A0 =C2=A0If the sending TCP simply sends as = much as it can whenever the
>>>>>=C2=A0 =C2=A0 =C2=A0window is non zero, the transmitted= data will be broken into
>>>>>=C2=A0 =C2=A0 =C2=A0alternating big and small segments.= =C2=A0 As time goes on, occasional
>>>>>=C2=A0 =C2=A0 =C2=A0pauses in the receiver making windo= w allocation available will
>>>>>=C2=A0 =C2=A0 =C2=A0result in breaking the big segments= into a small and not quite so
>>>>>=C2=A0 =C2=A0 =C2=A0big pair. And after a while the dat= a transmission will be in
>>>>>=C2=A0 =C2=A0 =C2=A0mostly small segments.
>>>>>
>>>>>=C2=A0 =C2=A0 =C2=A0The suggestion here is that the TCP= implementations need to
>>>>>=C2=A0 =C2=A0 =C2=A0actively attempt to combine small w= indow allocations into larger
>>>>>=C2=A0 =C2=A0 =C2=A0windows, since the mechanisms for m= anaging the window tend to lead
>>>>>=C2=A0 =C2=A0 =C2=A0to many small windows in the simple= st minded implementations.
>>>>>
>>>>>
>>>>> Any thoughts or feedback on removing this from 793bis = will be appreciated,
>>>>> thanks!
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> tcpm mailing list
>>>>> tcpm@ietf.org
>>>>> https://www.ietf.org/mailman/listinfo= /tcpm
>>>> _______________________________________________
>>>> tcpm mailing list
>>>> tcpm@ietf.org
>>>> https://www.ietf.org/mailman/listinfo/tcp= m
>>>
>>> _______________________________________________
>>> tcpm mailing list
>>> tcpm@ietf.org
>>> https://www.ietf.org/mailman/listinfo/tcpm
>>
>
> _______________________________________________
> tcpm mailing list
> tcpm@ietf.org
> https://www.ietf.org/mailman/listinfo/tcpm

_______________________________________________
tcpm mailing list
tcpm@ietf.org
https://www.ietf.org/mailman/listinfo/tcpm

--94eb2c04c6fe19dd5c05391d351c-- From nobody Tue Aug 2 15:42:11 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0AD4C12D9EC for ; Tue, 2 Aug 2016 15:42:10 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.187 X-Spam-Level: X-Spam-Status: No, score=-3.187 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RP_MATCHES_RCVD=-1.287] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ibJfgsOks3rD for ; Tue, 2 Aug 2016 15:42:07 -0700 (PDT) Received: from nitro.isi.edu (nitro.isi.edu [128.9.208.207]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id C4F2112DA06 for ; Tue, 2 Aug 2016 15:41:50 -0700 (PDT) Received: from [128.9.160.211] (mul.isi.edu [128.9.160.211]) (authenticated bits=0) by nitro.isi.edu (8.13.8/8.13.8) with ESMTP id u72MfSNk006900 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Tue, 2 Aug 2016 15:41:28 -0700 (PDT) To: John Heffner , David Borman References: <57A0FA15.6090507@isi.edu> From: Joe Touch Message-ID: <20f38b36-2a8e-f1f9-beeb-4db94326a26e@isi.edu> Date: Tue, 2 Aug 2016 15:41:28 -0700 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-MailScanner-ID: u72MfSNk006900 X-ISI-4-69-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Archived-At: Cc: "tcpm@ietf.org Extensions" Subject: Re: [tcpm] 793bis: removing "Window Management Suggestions" X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2016 22:42:10 -0000 On 8/2/2016 2:14 PM, John Heffner wrote: > This reminds me a lot of an old > thread: https://www.ietf.org/mail-archive/web/tcpm/current/msg04040.html Thanks for pointing that out. RFC 6429 (the resulting doc) already says it's OK for the OS to terminate ZWP connections if it runs out of resources. It also very clearly states that this isn't a change to the standard and that it is an implementation issue. Thus I don't think this belongs in RFC793bis. Joe From nobody Wed Aug 3 12:50:42 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3C1D812D7AA for ; Wed, 3 Aug 2016 12:50:41 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.988 X-Spam-Level: X-Spam-Status: No, score=-3.988 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RP_MATCHES_RCVD=-1.287, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=google.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JEcA1aZxJzBR for ; Wed, 3 Aug 2016 12:50:38 -0700 (PDT) Received: from mail-it0-x22c.google.com (mail-it0-x22c.google.com [IPv6:2607:f8b0:4001:c0b::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 30C8112D13E for ; Wed, 3 Aug 2016 12:50:38 -0700 (PDT) Received: by mail-it0-x22c.google.com with SMTP id j8so47787225itb.1 for ; Wed, 03 Aug 2016 12:50:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=I0b0M/b5Xu071ctbqAkDnNZOgNJ+lSS3NxwnYDxhPvg=; b=YBJuJHo3WP5nvJnBKG1VId5vBRH3PSh/8Wl88IgLib0dT6N+nul9Jna7dxUOT3rlq4 XdmPagpdMqOw+Gipa1Kdlu4cEqyFkKzyqOlPZmpzO5PuxkGpJUIt+yTcjdIuASqs24Fl ca3IPLA6qyPPF3Rhz4klM0S+gPtFsom5zdtf1arR5Co4UoU4YCBCmBEx8khe3BMf2LT7 5nq2sbAWybk2WwPITMHm5mrtB+4qcTpYbV9tm82H4yIFxL/O+jh7/NsiV5iFJXpIC8eI FrDfoO78IDbNZbRjGW+DS/How8Wljxrwpk4jM7J9yQ07iGAuKuMNXm2zRa/Y4PK9J2s8 JKYg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=I0b0M/b5Xu071ctbqAkDnNZOgNJ+lSS3NxwnYDxhPvg=; b=cMAVR6ItI/kA7SXwni6cMN7WBwPHDRDhrqexCvt3B23i2D+q5qh3b9RQQos9WNnIc+ uTm6VrRdyPdu47s+kp4svUTialZqe5E9QezFkLmV2Cs4hoJ1zPMSR4sJ79wA1gzhZw9W vX9B4EiyUFH/JXb/fyvZyfXardcyRnfH6dkcPvisXSf6z32eyFSt/GKY07kHal0eFF1t AY7/zswwSS7cY6Z0fbGP4bowLUC+dq3cMk0x+mplMYCoZ58yQEW0mQBTYnuY3bSzGK1p rfdmgCl6u0E0aTHCFjGNYLfIrttsCF9y0LEs6uSFJa/Knz5RFwKrTJzoVP4B5IdVUARj MjQg== X-Gm-Message-State: AEkooutuf0YvHsjFI28yl26wL/OIOKC/+C3taBs7Q2GTSYQDgTX5NspFBX1H+BGt7VYDn6NkuZQB1L+CWw1j9NQx X-Received: by 10.36.41.205 with SMTP id p196mr28523823itp.76.1470253837262; Wed, 03 Aug 2016 12:50:37 -0700 (PDT) MIME-Version: 1.0 Received: by 10.64.244.103 with HTTP; Wed, 3 Aug 2016 12:49:56 -0700 (PDT) In-Reply-To: <20f38b36-2a8e-f1f9-beeb-4db94326a26e@isi.edu> References: <57A0FA15.6090507@isi.edu> <20f38b36-2a8e-f1f9-beeb-4db94326a26e@isi.edu> From: Yuchung Cheng Date: Wed, 3 Aug 2016 12:49:56 -0700 Message-ID: To: Joe Touch Content-Type: text/plain; charset=UTF-8 Archived-At: Cc: "tcpm@ietf.org Extensions" , David Borman Subject: Re: [tcpm] 793bis: removing "Window Management Suggestions" X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Aug 2016 19:50:41 -0000 On Tue, Aug 2, 2016 at 3:41 PM, Joe Touch wrote: > > > On 8/2/2016 2:14 PM, John Heffner wrote: >> This reminds me a lot of an old >> thread: https://www.ietf.org/mail-archive/web/tcpm/current/msg04040.html Thanks for the reference. I suggest a quick reference to RFC6429 at the end of the section 3.8.3.3 in 793bis. RFC 793 is THE reference to TCP, so making it informational on various caveats for new implementors is really useful. (and I expect Joe to disagree again, which he already did). > > Thanks for pointing that out. > > RFC 6429 (the resulting doc) already says it's OK for the OS to > terminate ZWP connections if it runs out of resources. > > It also very clearly states that this isn't a change to the standard and > that it is an implementation issue. > > Thus I don't think this belongs in RFC793bis. > > Joe > > > > From nobody Wed Aug 3 13:05:48 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 22B8212D813 for ; Wed, 3 Aug 2016 13:05:47 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.187 X-Spam-Level: X-Spam-Status: No, score=-3.187 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RP_MATCHES_RCVD=-1.287] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=neutral reason="invalid (public key: not available)" header.d=aero.org header.b=CX6T9xsq; dkim=pass (1024-bit key) header.d=aerospacecloud.onmicrosoft.com header.b=lujCxuR7 Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id oGN7sBj37vFK for ; Wed, 3 Aug 2016 13:05:45 -0700 (PDT) Received: from email5-west.aero.org (email5-west.aero.org [130.221.16.30]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 6F80A12B02A for ; Wed, 3 Aug 2016 13:05:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=aero.org; i=@aero.org; q=dns/txt; s=mailhub; t=1470254741; x=1501790741; h=from:to:cc:subject:date:message-id:references: content-transfer-encoding:mime-version; bh=ybg2tRTUi05i0NkHW4Nu4HdO0ZUR92GzWHrclG/NaMs=; b=CX6T9xsqL7g5R2U6l6mz1Bd+/5bUFKASajlblt//YxJvAlOEwflue1Ai dINxsADfdAEIY86w01HQaNUOKa5Y4JB81IeH3xTgnUJOJCkB1Cjzub7Gf 8NNDLyJxjGALsVP7SqdmsdtBHcV2kr3zZXlJYDKJ56e/pm7antdOQQLjG M=; x-SBRS: 3.5 x-SenderGroup: Inbound_Office365 X-IronPort-AV: E=McAfee;i="5700,7163,8246"; a="4022509" X-IronPort-AV: E=Sophos;i="5.28,467,1464678000"; d="scan'208";a="4022509" X-IPAS-Result: A2F6AADpTaJXhzHGZxdaAw4LAQEBAQGDfU4EKge5KoFXJiCFfQKCBRMBAQEBAQEBAw8BAQEKCwkJGS+CUzkQAQEBAQEBAQEBIwEBAQEBAQEBAQEBAQEBAQEBAQEWAggFJxwBARoBAQEDEgEnBgEBNwEPAgEIGB4QMiUCBAENBQgah3UDF6EeAYEoARxhBSgCim2FKgEBBYcPGIQNAQEBAQEBAQEBAQEBAQEBAQEBAQEBFAiKd4FKglIKGgIfJoJlgi+II4ctiWmGGHGJYY1VjDCDdyACgjcMEguBETtuAYcURgF+AQEB Received: from mail-dm2gcc01lp0049.outbound.protection.outlook.com (HELO gcc01-dm2-obe.outbound.protection.outlook.com) ([23.103.198.49]) by email5-west.aero.org with ESMTP/TLS/AES256-SHA; 03 Aug 2016 13:05:39 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=aerospacecloud.onmicrosoft.com; s=selector1-aero-org; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=+Ia+TYBwiFQH2uONSNbczXEpL3wGsV4rFWTSN7spVVw=; b=lujCxuR7cuV84IcNM0JadERCfsAUcwaT0+Gf7Po/1Ez+NhMBKhWHGRfX7jNhAJfBtTQ/KaJkFyXrDEU8oc1IhXyjTzVTaChZ84LtNdtALvJO1yZhjq2mNaCTeCYtB8bTmZHwFRpOgemFK+9MAG5kXBYaYVGoKWS5DY24Z/7oWt4= Received: from BN6PR09MB2130.namprd09.prod.outlook.com (10.173.160.146) by BN6PR09MB2132.namprd09.prod.outlook.com (10.173.160.148) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.549.15; Wed, 3 Aug 2016 20:05:35 +0000 Received: from BN6PR09MB2130.namprd09.prod.outlook.com ([10.173.160.146]) by BN6PR09MB2130.namprd09.prod.outlook.com ([10.173.160.146]) with mapi id 15.01.0549.023; Wed, 3 Aug 2016 20:05:35 +0000 From: Theodore V Faber To: Joe Touch , David Borman Thread-Topic: [tcpm] 793bis: removing "Window Management Suggestions" Thread-Index: AQHR7PI/6kwKea8XjEqRRmOBdBNOpw== Date: Wed, 3 Aug 2016 20:05:35 +0000 Message-ID: References: <57A0FA15.6090507@isi.edu> <57A0FFE0.7020905@isi.edu> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=theodore.v.faber@aero.org; x-originating-ip: [130.221.224.7] x-ms-office365-filtering-correlation-id: 18ac92cc-83cf-437c-b3b5-08d3bbd98858 x-microsoft-exchange-diagnostics: 1; BN6PR09MB2132; 6:gs6dg/1bg//q7yfc8F+PJO6Wl2kdYM9tStAnb1bNSYItElVIyzJpv5T+OkDxGlIR8np/YV4ECQ6lV8PwZR8VXEQmL+A1cuvOt/D2IMmR+Nz2yVX2yCsJbbNFz000t0/zm/YrjCLgmeAgyHB7Li3TwrYV1MM37HzNg/MeGL+pMQoEEr+2BlsTpyNexpTC5OCQtJlYnnhGIxUMD0B5DGsqytJ7wvCkxQKN7ey75YTPyzhqnzFAT5QxHPGOBTcUlc1HYp0ritmHvQKYvo0dQxGQpX3lHTOyvmF9suBa9FdGqJWVFvn9fghpjtGMv6lq9kmHhLkAl7c5R9a1U/AKtWxxPg==; 5:eihtbtRMjoWSH/2zUYHMantKpEIP7vAFNmNrSGRbD/6k8pWq65oHWOBkaRb4Bp3NynGixcSvYuuT9qf3JsACsBioxQc813ugsMuRcx9EFzxls0qbkx7tnl/cPz//71oR0kQWO0AlpZwrNbSRvjx5/g==; 24:sJLEgSva9+AQekFQep50hSrxUUrEkrzn9yOKIvLfmcoxXSzvr34Y+ih63JMkONWP4SWYyNb2D0sAS6n91KLCghviL/jq/GB5uUpvj2wDedo=; 7:7PAPL34UrP6mbh0ISnp/n+8LmAkuT99y77PKWBQ1nBuJ7oCvHpuOhMh07gOD7R2zJvtv1IfDwhHbAW13dKGMPFRdaPtLEbP/cK8xN/Hhz0K6241sgZz5gu9dghgm+5ZP1kZsSapB83g7p+cdPFja9scgvJQ7jVc61IyDn0Yll0IWSAY3ZyRa4NvOS1uf4I62FEGxyLqWdtglh1VwTJBLx+3A/ZsVJ7ItCVmLUqhgqHM1a0l97iBIDDg5fThYdOYk x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:BN6PR09MB2132; x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:; x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(601004)(2401047)(8121501046)(5005006)(10201501046)(3002001)(6055026); SRVR:BN6PR09MB2132; BCL:0; PCL:0; RULEID:; SRVR:BN6PR09MB2132; x-forefront-prvs: 00235A1EEF x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(6009001)(7916002)(24454002)(377454003)(54524002)(189002)(199003)(586003)(92566002)(4326007)(66066001)(101416001)(74316002)(110136002)(106116001)(76576001)(2906002)(93886004)(5002640100001)(50986999)(6116002)(3846002)(102836003)(106356001)(54356999)(76176999)(15975445007)(122556002)(2900100001)(97736004)(10400500002)(2171001)(87936001)(11100500001)(77096005)(33656002)(189998001)(68736007)(99286002)(105586002)(9686002)(5001770100001)(8936002)(8676002)(81166006)(7846002)(7736002)(7696003)(19580405001)(19580395003)(86362001)(3280700002)(3660700001)(575784001)(81156014)(305945005)(19627235001); DIR:OUT; SFP:1101; SCL:1; SRVR:BN6PR09MB2132; H:BN6PR09MB2130.namprd09.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: aero.org X-MS-Exchange-CrossTenant-originalarrivaltime: 03 Aug 2016 20:05:35.7332 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: c8294700-c5a4-4ca1-a876-1457d39899fd X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR09MB2132 Archived-At: Cc: "tcpm@ietf.org Extensions" Subject: Re: [tcpm] 793bis: removing "Window Management Suggestions" X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Aug 2016 20:05:47 -0000 -----BEGIN PGP SIGNED MESSAGE-----=0A= Hash: SHA512=0A= =0A= On 08/02/2016 01:18 PM, Joe Touch wrote:=0A= > Hi, David,=0A= > =0A= > I agree. IMO, DOS situations should be handled by the OS by=0A= > withdrawing resources *as needed*. That includes closing down=0A= > zero-window situations as below, as well as closing idle=0A= > connections when resources are limited.=0A= > =0A= > I don't see this as something the protocol needs to decide.=0A= =0A= Right. The relevant distinction is between OS and protocol. The same=0A= way the OS needn't inform open connections that the power has failed,=0A= the OS needn't tell open connections that it's responding to a DDoS.=0A= The possibility of catastrophic failure is assumed.=0A= =0A= (BTW, Joe, we're agreeing here. :-D)=0A= =0A= =0A= - -- =0A= Ted Faber =0A= Engineering Specialist=0A= Computer Systems Research Department=0A= 310-336-7373=0A= -----BEGIN PGP SIGNATURE-----=0A= Comment: GPGTools - https://gpgtools.org=0A= Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/=0A= =0A= iQIcBAEBCgAGBQJXok6QAAoJEFNjQnOBW8uOHM0QAJCLTxdrUziEbUlBHLoSGimy=0A= 8ALmh+fphYOw+M8mMJUa1Sdx6QeogNBLo/7kAU1fV/Fo+w+t6RTTJmLrwBypAuHg=0A= TZnYLY9ZPjfkuO+UgZRfzxRwG4KfkVVA56Zd+JT7kgPzqe0BFdSn2IRr7k/+7Azz=0A= vG/D0RhTW7wkXwPllt0h585eEnf0vKQxUXmuDMqI8gy8BNpB6Tq0vnIv1dK1lLxF=0A= zrGv3IFq6qyc5q4iLo3F1TXDGO/EFAFXorGBwk/uyg+nSiTAaR3DwH5brakCao0b=0A= Q5NAKbrs5+0gDXOJsVObSt6j7zWDF10rpHXmTIh8aV5LdOHHj9s1GaS59XjpJQ06=0A= DiiHmFxlNOE6TJXdk0WrDkRxzsJ/asYdv+pHaxAhxpT0q2t/l/KWpgGcxgZqtRSk=0A= 6wdnglvU34Yl7oEdvOhjvAiSds+rNZ34bJMU1ysKk50DC6JYAz6drQ+3lKPC/vHY=0A= B47TTK0Z5A2ig4IiTq/FqVJVLhsmFTP4IqF68zLdjkxZkHVUV27+RPsRFd3xwSMZ=0A= 7pwJhAzDt1N6I/E7RK8r5PTYDqY9sBWeJQGrHQx14/AXrXE7Ef2XWlhn+yZ68JMe=0A= SH96Iobg20SVWtvgyHJoCr94EQsCdzDRim+xnv9c4y6PVP83Li7pc4MdcwhXTcbA=0A= zGD7vGpbfx51c81L5Msa=0A= =3D+Yyn=0A= -----END PGP SIGNATURE-----=0A= From nobody Wed Aug 3 14:10:22 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B85D312D0FB for ; Wed, 3 Aug 2016 14:10:20 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.187 X-Spam-Level: X-Spam-Status: No, score=-3.187 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RP_MATCHES_RCVD=-1.287] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jr0GxQckt4EE for ; Wed, 3 Aug 2016 14:10:19 -0700 (PDT) Received: from nitro.isi.edu (nitro.isi.edu [128.9.208.207]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 3A0CC12B010 for ; Wed, 3 Aug 2016 14:10:19 -0700 (PDT) Received: from [128.9.184.236] ([128.9.184.236]) (authenticated bits=0) by nitro.isi.edu (8.13.8/8.13.8) with ESMTP id u73L9sxx008303 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Wed, 3 Aug 2016 14:09:54 -0700 (PDT) To: Yuchung Cheng References: <57A0FA15.6090507@isi.edu> <20f38b36-2a8e-f1f9-beeb-4db94326a26e@isi.edu> From: Joe Touch Message-ID: <57A25DA0.7090402@isi.edu> Date: Wed, 3 Aug 2016 14:09:52 -0700 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.7.2 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-MailScanner-ID: u73L9sxx008303 X-ISI-4-69-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Archived-At: Cc: "tcpm@ietf.org Extensions" , David Borman Subject: Re: [tcpm] 793bis: removing "Window Management Suggestions" X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Aug 2016 21:10:21 -0000 On 8/3/2016 12:49 PM, Yuchung Cheng wrote: > On Tue, Aug 2, 2016 at 3:41 PM, Joe Touch wrote: >> >> On 8/2/2016 2:14 PM, John Heffner wrote: >>> This reminds me a lot of an old >>> thread: https://www.ietf.org/mail-archive/web/tcpm/current/msg04040.html > Thanks for the reference. I suggest a quick reference to RFC6429 at > the end of the section 3.8.3.3 in 793bis. RFC 793 is THE reference to > TCP, so making it informational on various caveats for new > implementors is really useful. > > (and I expect Joe to disagree again, which he already did). You're right. To clarify, my view is that RFC6429 is a recommendation to the OS, not to TCP. That's the reason I don't think it belongs in a TCP doc. I think it would simply waste space to put in every way in which the OS needs to manage resources that TCP might either contribute to or be affected by. 793bis should not become a "junk drawer". The cleaner and simpler it is, the more likely it will be supported correctly. Joe From nobody Wed Aug 3 14:40:12 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 010DF12D0C1 for ; Wed, 3 Aug 2016 14:40:11 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.988 X-Spam-Level: X-Spam-Status: No, score=-3.988 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RP_MATCHES_RCVD=-1.287, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=google.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Gk3pUqcv6W9E for ; Wed, 3 Aug 2016 14:40:09 -0700 (PDT) Received: from mail-it0-x22f.google.com (mail-it0-x22f.google.com [IPv6:2607:f8b0:4001:c0b::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 0A71A12B015 for ; Wed, 3 Aug 2016 14:40:09 -0700 (PDT) Received: by mail-it0-x22f.google.com with SMTP id u186so309178199ita.0 for ; Wed, 03 Aug 2016 14:40:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=2SywEVDHsU79K9+YWXRKpUUTEdEOglPNRlFMmlrK2V4=; b=TyTfvwatnn6Rz73F52csiHhZ0CiiF/NArS2VD0zF6ajftXVSlFjis1oZ06oMYiYreG r2SwqoqRFBGFjELDWpUw0VvOxXAg8OnhBlJ5DH8/ts1T10UjfBl4Fwm1kv07vz6vAsGI 2h07Jyrk/qmLKMGEHfxO91dxnS2SRLOi9GigpXdoX8Sjg7WyhWa8Ym9YEzbwiAp+sRcX nB086eEvwpAgU6DG8VIX+9Q4f8JtKQhjgAcDVSdQrhYkQSPHdujHn6MjYMPHkzD87yqK E8l+PMV4+dHHeKoKfbIYfcBFql+kshpgrOKgR0YV37izQonv/xpk5WGXRfglRE3csYtv f5eg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=2SywEVDHsU79K9+YWXRKpUUTEdEOglPNRlFMmlrK2V4=; b=Ty48lZjCQ6CEUL4NzfX1TjLq9BDrs4f3sQ7MwLzzqhQa3Z4+Z1JkuA1U80bJUzfJQE MwAFhhmMeXdiBy3XHLxsKPxlRmKdenLLRtWbQmegbz64NOhI4VAtI0ulbKD2sfBqO+bF 3WYpTu1kmjGD81AwYtrYHIk9VrNdDWP2orS+64h1BN78U2ZXj3oMnK7xHs27VaYnblKw EgILGKj6CKjtWE2rhWNMBGmvLNHBowcIGxxa0hpFTc+mIMK1/MusOj/5UkJVWFCPRKct PWwsv29Jsx57qQXY5FojDoBW/ijqRiLI81WiqmIHthGysyUv3zcCqq/z9ExFCn4ISBUu NLIA== X-Gm-Message-State: AEkoouuFZ46NLljGWOwbPlglxLpDiJtDbFwC55TQ89p7Fa2Hup+Lxe5O2agQRj3hVYSzO8vH1MhPKzoRtuCVD3qU X-Received: by 10.36.150.70 with SMTP id z67mr25831630itd.80.1470260408246; Wed, 03 Aug 2016 14:40:08 -0700 (PDT) MIME-Version: 1.0 Received: by 10.64.244.103 with HTTP; Wed, 3 Aug 2016 14:39:27 -0700 (PDT) In-Reply-To: <57A25DA0.7090402@isi.edu> References: <57A0FA15.6090507@isi.edu> <20f38b36-2a8e-f1f9-beeb-4db94326a26e@isi.edu> <57A25DA0.7090402@isi.edu> From: Yuchung Cheng Date: Wed, 3 Aug 2016 14:39:27 -0700 Message-ID: To: Joe Touch Content-Type: text/plain; charset=UTF-8 Archived-At: Cc: "tcpm@ietf.org Extensions" , David Borman Subject: Re: [tcpm] 793bis: removing "Window Management Suggestions" X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Aug 2016 21:40:11 -0000 On Wed, Aug 3, 2016 at 2:09 PM, Joe Touch wrote: > > > On 8/3/2016 12:49 PM, Yuchung Cheng wrote: >> On Tue, Aug 2, 2016 at 3:41 PM, Joe Touch wrote: >>> >>> On 8/2/2016 2:14 PM, John Heffner wrote: >>>> This reminds me a lot of an old >>>> thread: https://www.ietf.org/mail-archive/web/tcpm/current/msg04040.html >> Thanks for the reference. I suggest a quick reference to RFC6429 at >> the end of the section 3.8.3.3 in 793bis. RFC 793 is THE reference to >> TCP, so making it informational on various caveats for new >> implementors is really useful. >> >> (and I expect Joe to disagree again, which he already did). > You're right. > > To clarify, my view is that RFC6429 is a recommendation to the OS, not > to TCP. That's the reason I don't think it belongs in a TCP doc. I think RFC6429 says Rather than making a change to the standard, this document clarifies what has been until now a misinterpretation of the tandard as specified in RFC 1122. RFC793-bis says 1. Purpose and Scope [snip] For several decades, RFC 793 plus a number of other documents have combined to serve as the specification for TCP [17]. Over time, a number of errata have been identified on RFC 793, as well as deficiencies in security, performance, and other aspects. A number of enhancements has grown and been documented separately. These were never accumulated together into an update to the base specification. > it would simply waste space to put in every way in which the OS needs to > manage resources that TCP might either contribute to or be affected by. > > 793bis should not become a "junk drawer". The cleaner and simpler it is, > the more likely it will be supported correctly. I will agree with you if RFC6429 is a piece of junk. but I don't think so. > > Joe From kobby.Carmona@qlogic.com Thu Aug 4 02:09:09 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7EC9212D692 for ; Thu, 4 Aug 2016 02:09:09 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.123 X-Spam-Level: X-Spam-Status: No, score=-1.123 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_NEUTRAL=0.779] autolearn=no autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=qlgc.onmicrosoft.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id yZgz0by5mJIj for ; Thu, 4 Aug 2016 02:09:07 -0700 (PDT) Received: from NAM03-BY2-obe.outbound.protection.outlook.com (mail-by2nam03on0094.outbound.protection.outlook.com [104.47.42.94]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 6226A12D1A2 for ; Thu, 4 Aug 2016 02:09:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=qlgc.onmicrosoft.com; s=selector1-qlogic-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=nmYzhdfteszXSUaN7AUR+Hqn9xNAigNny+YxeuJ8idM=; b=D18GH44S3Tonf5UZymHynW/unR1VxtBv8X9zmNUzSsc9avmdRTACOwNt6mFCByAe+1XshZImdhqaUtOPA0Ln1z5ik7tWEJmAJcRWPHIj63Sp7qCYAU899dCT9tJvZTskcfq7FQ7KF6+sjBfJScLCWXGyXP+5uZwepIby/vEmfHQ= Received: from MWHPR11MB1374.namprd11.prod.outlook.com (10.169.234.8) by MWHPR11MB1376.namprd11.prod.outlook.com (10.169.234.10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.549.15; Thu, 4 Aug 2016 09:08:58 +0000 Received: from MWHPR11MB1374.namprd11.prod.outlook.com ([10.169.234.8]) by MWHPR11MB1374.namprd11.prod.outlook.com ([10.169.234.8]) with mapi id 15.01.0549.023; Thu, 4 Aug 2016 09:08:58 +0000 From: Kobby Carmona To: "tcpm@ietf.org" Thread-Topic: Possible deadlock scenario with retransmission on both sides at the same time Thread-Index: AdHth2zqZmjYio0mRrKVRRoD8V9Fcg== Date: Thu, 4 Aug 2016 09:08:58 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=kobby.Carmona@qlogic.com; x-originating-ip: [31.168.140.228] x-ms-office365-filtering-correlation-id: 714f5f17-cba9-4340-a443-08d3bc46f834 x-microsoft-exchange-diagnostics: 1; MWHPR11MB1376; 20:+NmX+rzsOBrm8iRsk5gvsMPqohRqH7MaNljwdfQToI0L0+u/gF0p7DeM3J8SM/KxFZljVecgfWZvbuAKQOcATav8acgFKGv2Udl4O7J4sOiM+LoETl0lygg5t/mSThEt5q7ZZSU5Y3jYws7Amnmva9wsMKArjzMOORij6t1QgaE= x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:MWHPR11MB1376; x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:; x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(601004)(2401047)(8121501046)(5005006)(3002001)(10201501046); SRVR:MWHPR11MB1376; BCL:0; PCL:0; RULEID:; SRVR:MWHPR11MB1376; x-forefront-prvs: 00246AB517 x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(6009001)(7916002)(53754006)(189002)(199003)(2501003)(74316002)(5640700001)(3280700002)(3660700001)(86362001)(68736007)(6116002)(7846002)(66066001)(9686002)(92566002)(77096005)(81166006)(107886002)(81156014)(110136002)(2900100001)(97736004)(87936001)(122556002)(101416001)(8676002)(105586002)(54356999)(8936002)(50986999)(2906002)(305945005)(586003)(33656002)(106356001)(450100001)(99286002)(11100500001)(1730700003)(10400500002)(229853001)(5002640100001)(3846002)(102836003)(19580395003)(2351001)(76576001)(189998001)(7696003)(7736002); DIR:OUT; SFP:1102; SCL:1; SRVR:MWHPR11MB1376; H:MWHPR11MB1374.namprd11.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; received-spf: None (protection.outlook.com: qlogic.com does not designate permitted sender hosts) spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: qlogic.com X-MS-Exchange-CrossTenant-originalarrivaltime: 04 Aug 2016 09:08:58.4051 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 0d68a1f9-1490-4d0e-8767-a87dab3ef2ba X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR11MB1376 Archived-At: X-Mailman-Approved-At: Thu, 04 Aug 2016 10:43:38 -0700 Subject: [tcpm] Possible deadlock scenario with retransmission on both sides at the same time X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Aug 2016 09:10:16 -0000 Hi all, While running a bidirectional scenario with random drops in a network simul= ator of our (QLogic's NIC) TCP stack we found a case where it seems there i= s deadlock in the TCP protocol (the connection will keep sending pure acks = from both sides until RTO will expire multiple times and a RST will sent to= close the connection). The scenario is as follows (there is an example with numbers for each stage= assuming the MSS and each packet is 1000B): 1. Both sides are transmitting data and a single packet is dropped on eithe= r side and the next two packets are received properly Side A - SND.MAX=3D3000, SND.NXT=3D3000, SND.UNA=3D1000, RCV.NXT=3D11000, = out-of-order block 12000-13000 Side B - SND.MAX =3D13000, SND.NXT =3D13000, SND.UNA=3D11000, RCV.NXT=3D10= 00, out-of-order block 2000-3000 2. RTO timer expires on both sides Side A - SND.MAX=3D3000, SND.NXT=3D1000, SND.UNA=3D1000, RCV.NXT=3D11000, = out-of-order block 12000-13000 Side B - SND.MAX =3D13000, SND.NXT=3D11000, SND.UNA=3D11000, RCV.NXT=3D100= 0, out-of-order block 2000-3000 3. Both sides transmit a single packet to the peer: A->B - pkt.seq=3D1000, pkt.ack=3D11000, len=3D1000 B->A - pkt.seq=3D11000, pkt.ack=3D1000, len=3D1000 3. Both sides receive the packets and update the receive context: Side A - SND.MAX=3D3000, SND.NXT=3D2000, SND.UNA=3D1000, RCV.NXT=3D13000 Side B - SND.MAX=3D13000, SND.NXT=3D12000, SND.UNA=3D11000, RCV.NXT=3D3000 4. Both sides send another segment: A->B - pkt.seq=3D2000, pkt.ack=3D13000, len=3D1000 B->A - pkt.seq=3D12000, pkt.ack=3D3000, len=3D1000 5. Both sides don't accept the packet (and don't update SND.UNA) since the = sequence on the packet is less than RCV.NXT (sequence number check in page = 69 of RFC793) and send a pure ACK instead A->B - pkt.seq=3D2000, pkt.ack=3D13000, len=3D0 (pure ACK) B->A - pkt.seq=3D12000, pkt.ack=3D3000, len=3D0 (pure ACK) 6. This will continue forever (until the connection will be terminated by R= ST) since every packet that ends before RCV.NXT (even a retransmit from SND= .UNA) will be dropped. Did anyone encountered this issue before? Is the anything we missed on this= sequence? If this is indeed a real deadlock, there might be several solutions to this= which will require a modification in receive processing of RFC793. But I w= ould like to know if you think this is a real issue before dealing with sol= utions. Thanks, Kobby From nobody Thu Aug 4 12:36:43 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B2FCE12D7D2 for ; Thu, 4 Aug 2016 12:36:42 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.187 X-Spam-Level: X-Spam-Status: No, score=-3.187 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RP_MATCHES_RCVD=-1.287] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id en2d57jVF6Um for ; Thu, 4 Aug 2016 12:36:40 -0700 (PDT) Received: from frantic.weston.borman.com (frantic.weston.borman.com [70.57.156.33]) (using TLSv1 with cipher EDH-RSA-DES-CBC3-SHA (112/168 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id CE35D12D7CE for ; Thu, 4 Aug 2016 12:36:39 -0700 (PDT) Received: from [127.0.0.1] (frantic.weston.borman.com [70.57.156.33]) by frantic.weston.borman.com (8.14.7/8.14.7) with ESMTP id u74JabXF012660; Thu, 4 Aug 2016 14:36:37 -0500 (CDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) From: David Borman In-Reply-To: Date: Thu, 4 Aug 2016 14:36:37 -0500 Content-Transfer-Encoding: quoted-printable Message-Id: <7070553C-65D1-46EE-95F4-DAE82E1F5A5E@weston.borman.com> References: To: Kobby Carmona X-Mailer: Apple Mail (2.3124) Archived-At: Cc: "tcpm@ietf.org" Subject: Re: [tcpm] Possible deadlock scenario with retransmission on both sides at the same time X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Aug 2016 19:36:42 -0000 After you pull back SND.NXT and do the retransmission, you should then = restore SND.NXT back to where it was, not leave it at the backed off = value; then the ACKs wouldn=E2=80=99t be dropped, since they wouldn=E2=80=99= t have old seq values. For example, in the 4.4BSD fast retransmit code = it had: tcp_seq onxt =3D tp->snd_nxt; ... tp->snd_nxt =3D th->th_ack; ... (void) tcp_output(tp); ... if (SEQ_GT(onxt, tp->snd_nxt)) =20 tp->snd_nxt =3D onxt; -David Borman > On Aug 4, 2016, at 4:08 AM, Kobby Carmona = wrote: >=20 > Hi all, > While running a bidirectional scenario with random drops in a network = simulator of our (QLogic's NIC) TCP stack we found a case where it seems = there is deadlock in the TCP protocol (the connection will keep sending = pure acks from both sides until RTO will expire multiple times and a RST = will sent to close the connection). > The scenario is as follows (there is an example with numbers for each = stage assuming the MSS and each packet is 1000B): > 1. Both sides are transmitting data and a single packet is dropped on = either side and the next two packets are received properly > Side A - SND.MAX=3D3000, SND.NXT=3D3000, SND.UNA=3D1000, = RCV.NXT=3D11000, out-of-order block 12000-13000 > Side B - SND.MAX =3D13000, SND.NXT =3D13000, SND.UNA=3D11000, = RCV.NXT=3D1000, out-of-order block 2000-3000 > 2. RTO timer expires on both sides > Side A - SND.MAX=3D3000, SND.NXT=3D1000, SND.UNA=3D1000, = RCV.NXT=3D11000, out-of-order block 12000-13000 > Side B - SND.MAX =3D13000, SND.NXT=3D11000, SND.UNA=3D11000, = RCV.NXT=3D1000, out-of-order block 2000-3000 > 3. Both sides transmit a single packet to the peer: > A->B - pkt.seq=3D1000, pkt.ack=3D11000, len=3D1000 > B->A - pkt.seq=3D11000, pkt.ack=3D1000, len=3D1000 > 3. Both sides receive the packets and update the receive context: > Side A - SND.MAX=3D3000, SND.NXT=3D2000, SND.UNA=3D1000, = RCV.NXT=3D13000 > Side B - SND.MAX=3D13000, SND.NXT=3D12000, SND.UNA=3D11000, = RCV.NXT=3D3000 > 4. Both sides send another segment: > A->B - pkt.seq=3D2000, pkt.ack=3D13000, len=3D1000 > B->A - pkt.seq=3D12000, pkt.ack=3D3000, len=3D1000 > 5. Both sides don't accept the packet (and don't update SND.UNA) since = the sequence on the packet is less than RCV.NXT (sequence number check = in page 69 of RFC793) and send a pure ACK instead > A->B - pkt.seq=3D2000, pkt.ack=3D13000, len=3D0 (pure ACK) > B->A - pkt.seq=3D12000, pkt.ack=3D3000, len=3D0 (pure ACK) > 6. This will continue forever (until the connection will be terminated = by RST) since every packet that ends before RCV.NXT (even a retransmit = from SND.UNA) will be dropped. >=20 > Did anyone encountered this issue before? Is the anything we missed on = this sequence? > If this is indeed a real deadlock, there might be several solutions to = this which will require a modification in receive processing of RFC793. = But I would like to know if you think this is a real issue before = dealing with solutions. > Thanks, >=20 > Kobby >=20 >=20 > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm From nobody Sun Aug 7 07:51:46 2016 Return-Path: X-Original-To: tcpm@ietf.org Delivered-To: tcpm@ietfa.amsl.com Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id DF56B12D131; Sun, 7 Aug 2016 07:51:40 -0700 (PDT) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit From: internet-drafts@ietf.org To: X-Test-IDTracker: no X-IETF-IDTracker: 6.29.0 Auto-Submitted: auto-generated Precedence: bulk Message-ID: <147058150090.16481.1026605609273658921.idtracker@ietfa.amsl.com> Date: Sun, 07 Aug 2016 07:51:40 -0700 Archived-At: Cc: tcpm@ietf.org Subject: [tcpm] I-D Action: draft-ietf-tcpm-cubic-02.txt X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Aug 2016 14:51:41 -0000 A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the TCP Maintenance and Minor Extensions of the IETF. Title : CUBIC for Fast Long-Distance Networks Authors : Injong Rhee Lisong Xu Sangtae Ha Alexander Zimmermann Lars Eggert Richard Scheffenegger Filename : draft-ietf-tcpm-cubic-02.txt Pages : 15 Date : 2016-08-07 Abstract: CUBIC is an extension to the current TCP standards. The protocol differs from the current TCP standards only in the congestion window adjustment function in the sender side. In particular, it uses a cubic function instead of a linear window increase of the current TCP standards to improve scalability and stability under fast and long distance networks. BIC-TCP, a predecessor of CUBIC, has been a default TCP adopted by Linux since year 2005 and has already been deployed globally and in use for several years by the Internet community at large. CUBIC is using a similar window growth function as BIC-TCP and is designed to be less aggressive and fairer to TCP in bandwidth usage than BIC-TCP while maintaining the strengths of BIC- TCP such as stability, window scalability and RTT fairness. Through extensive testing in various Internet scenarios, we believe that CUBIC is safe for deployment and testing in the global Internet. The intent of this document is to provide the protocol specification of CUBIC for a third party implementation and solicit the community feedback through experimentation on the performance of CUBIC. The IETF datatracker status page for this draft is: https://datatracker.ietf.org/doc/draft-ietf-tcpm-cubic/ There's also a htmlized version available at: https://tools.ietf.org/html/draft-ietf-tcpm-cubic-02 A diff from the previous version is available at: https://www.ietf.org/rfcdiff?url2=draft-ietf-tcpm-cubic-02 Please note that it may take a couple of minutes from the time of submission until the htmlized version and diff are available at tools.ietf.org. Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ From nobody Sun Aug 7 07:58:41 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0AFCE12D0E7; Sun, 7 Aug 2016 07:58:40 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.902 X-Spam-Level: X-Spam-Status: No, score=-1.902 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=uofnelincoln.onmicrosoft.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7KL3mdGP-42K; Sun, 7 Aug 2016 07:58:38 -0700 (PDT) Received: from NAM03-CO1-obe.outbound.protection.outlook.com (mail-co1nam03lp0017.outbound.protection.outlook.com [216.32.181.17]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id D635B12D0BA; Sun, 7 Aug 2016 07:58:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=uofnelincoln.onmicrosoft.com; s=selector1-unl-edu; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=UAVaQpXk2hnRYaefJmiKtTkPEpKnldNTsXCaYcsRwEg=; b=IOz3rFhjEzc1AKbKsXylwfhc4IrTh4MKXUkYKefhtj5ypewUibzeEUaG6BhHi/ut2l/DporszFULJSlq30JTSL9rbSylXQUDV7B1d1tm7mZ2KpctXlqzQ5wgJ0hsmZGt6EeLxJHoiEu5cgUhxKxbC7g4fTObt37v1dWGUzFkNEM= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=xu@unl.edu; Received: from [192.168.0.164] (97.98.140.59) by DM5PR08MB2748.namprd08.prod.outlook.com (10.173.222.20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.549.15; Sun, 7 Aug 2016 14:58:26 +0000 To: From: Lisong Xu Organization: University of Nebraska-Lincoln Message-ID: <85d0d695-d8f1-e404-d0e5-23b13963bfd9@unl.edu> Date: Sun, 7 Aug 2016 09:58:24 -0500 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [97.98.140.59] X-ClientProxiedBy: SN1PR12CA0023.namprd12.prod.outlook.com (10.162.96.161) To DM5PR08MB2748.namprd08.prod.outlook.com (10.173.222.20) X-MS-Office365-Filtering-Correlation-Id: 90b28a06-612b-4ec5-f4e9-08d3bed34a8b X-Microsoft-Exchange-Diagnostics: 1; DM5PR08MB2748; 2:fKynTky9Ed6Uc7vu8CF/VMq/6+HQilSQjNsaaT4A7KtnUMMAw6ZWquEZGmv+K3eeqhsygwBV590SmB4ZITTFRltyWjvRh3455z9gbut4/VS1m5GLjClKV1lszhHHdO81GCzn9WJpmm3Z7I6G/og3kmDB26BipCybjwVK3mXHaZt6DyKM9PkyXHLcoO4bN8KN; 3:Xto/L294De2cogelCuPuSJvWflVlUnaH9ipl2tdQTPJ+bdlq5hNKqh6LV+Qldf3/kA+8Nt+cRnMg8fsXu7AZUoxFsDOHu3HE75ZoDSFzEY32ZrJ/P3aDKK5yb8H9p/4O X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:DM5PR08MB2748; X-Microsoft-Exchange-Diagnostics: 1; DM5PR08MB2748; 25:EnMlKnrGM7FxbUkeRhNui3hyAxQ44dwNW5F98ES1fgCIOwSozZIPvPvSYdPHhaDG3kUtOBKs8xd8VLdwOTOpjOX4W1kawm9yuGTzIrgN5esEsE4+sK3mwN3+hI/PJKgfmiG6pGvcP7sqDFlOm9VQ9kRAENCqFuqfuQbQIc5anQLUXCmqvjZEyex0IALYmn3Se7df/Daq8P4yvDAN7iF7Q5ahPlPrsEnbo6dr0bAnYON0WlwcqaAYVRZOT4AG1lffTG+k/kGuDjT6j2wDfTAU/KdLubIx8i3HgJtOP56Ytk2d1IMO40l0JxmaIcqusjDMgTTFqB6Q+D4iSklcUG+SFYcqWMVvemjZbO4e/dBYxNACtoEYYAk1hx/MhU4m5b4zdTIOaGCZE1jMnolP3Zvp8IXFnYLpjWXTqLU01J5C1ZFu4VDfwcmi3ThMXG/MhwgvXqQ5orfmRo52eawlLfvxIRJvymT5Fb9JiW/8wCUk2J6mKeFkNteF7dtl+EyOkjFq61rWQgQPTOwEWdREBr+e4FYIBlk4SXXFEmp1NHzGDAjjyFarCL5CrkF61oLdqwwqas7zWK3kCccIRSA4bzgXPkzoKaKd2ext8vitAaH65eKUT5p1RTVLgl0PCxceTIpYEH0KGu0dtvw93gt7ckG4LVPFDXwUhNYRospP84EnE81rqlxqS6Egwk3ylHp7kWJCwhE5B8X9uJOme9lTQjTd1CjfVedpAaHwhT5IvyfI/U0voxTsPYBRsnu2TeTia5uILNmS885SOVMKugqkW5BP9H3SaNlK3s2NQQNlBqWA0PsiUgcXRQJPnSjcjDY796QV X-Microsoft-Exchange-Diagnostics: 1; DM5PR08MB2748; 31:RJznuQjnd5+Bf/8Z2kayXy81+LeEUZcoYL/8p8/WUKItTdZNlOEKwk9VVnbSNtfUY16yFnBxALxxsgmFV25uG6iBYHZvE+8t87FIrpgmLM1aGNvmK62VMavkr4HTDm4bRquymrXSIBF319naSEKkNDYlHFSc7MJqWqaXaCt05gwLi6o366KPSPFnOyybkLSy6gvLWo9O7zBUAlWAUeLPkxo7i4Ld6E8sd7W5AKCNyyY=; 20:zqko3WrSAWFq9SOLpkIbSHYhtQQzxARD/Kxc+fTe78Cn3nl45qlTSvkwSa2mYpnlGO4aTpjhID+rj2Ael9ofpR9MRsdEoabxBbDU3YeGe8+wCconFjn4pD8UXTAbwHXAV/SBcsv/4lUBniPYEKSX13s7DRdFVSn7Dff7YhtJ6lK11hy7t/zWIV1F3ycjBQ4aFZ900abaoPNVJ01i3OyhtKGydb2q/oZLW/dI9YoqHxyAxVqkrfrXl3rLHuVGdKEA6ws1WCZahTPFvYKdLaf30o1Y/mCS0JjDD1KKmmsR/Q9jQwJ2tKHN/tVBWt4VS07AqpxtDkWXx8ToUkr4xHpCIdKg2y97aj+ecrjXMIMvYLCvLdlPF+gf7dlo9a+I+Hw1MD3wsHkjwrn9eKbEAFoUxmbMLoSUyXXjkg/2iy2egyPxtSBtwwyr+TPGg3dhCpNLQaWZS0Zh+iFaGs7Q25b26i58XMAthPCn0tvqQt/7QZ4JdtsozUwHEuBF1iL4/cHd X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(601004)(2401047)(8121501046)(5005006)(10201501046)(3002001); SRVR:DM5PR08MB2748; BCL:0; PCL:0; RULEID:; SRVR:DM5PR08MB2748; X-Microsoft-Exchange-Diagnostics: 1; DM5PR08MB2748; 4:yVlHRBGaViYn3n+7wPxJ2IpjD730FQBxraVF+/52p8uG3Vo13YHiFIDGXYyh+WErpVYPayYdMXsbJkBRc8gwZDe5QhylZh7jUPs4z7SEuDJSNTtzyA2K4kwgyWdH3Wsb5QgLL2kA6M1fh1tpcvsDtVG/VrCSoYiJrjLEa/6XMMYqPoOLT6mh8XYKskUD1XCoi1Pyzgtb05HmJfMo1BV0trC8NXNjtQgHxMpyaEtqww2kLAc9NIbgsfjO7qHSu2Imlcp8QjtVpTdzTuaeMD2f+xPqVW8QzsDKpYuhcfmiTDP2DfXCtGGWtV77JHMt/qtc47Y3Cd2ZQsQSCNp9Bd5vg7U4P79y1HPq6MM0bttcPQ2kleFn5d+9hFL5ug0G+KBS X-Forefront-PRVS: 0027ED21E7 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10019020)(4630300001)(6009001)(6049001)(7916002)(199003)(189002)(105586002)(101416001)(2906002)(64126003)(88552002)(4326007)(68736007)(2351001)(81156014)(75432002)(23676002)(42186005)(66066001)(65806001)(65956001)(586003)(229853001)(50986999)(8676002)(117156001)(54356999)(33646002)(47776003)(31686004)(89122001)(106356001)(81166006)(97736004)(19580405001)(19580395003)(31696002)(4001350100001)(305945005)(230700001)(189998001)(83506001)(90282001)(110136002)(7736002)(7846002)(50466002)(86362001)(3846002)(6116002)(77096005)(92566002)(15975445007)(36756003)(65826006); DIR:OUT; SFP:1102; SCL:1; SRVR:DM5PR08MB2748; H:[192.168.0.164]; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; Received-SPF: None (protection.outlook.com: unl.edu does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtETTVQUjA4TUIyNzQ4OzIzOlhZczVkUGFwTDljU3BBcm5xeEkwY25JYUJR?= =?utf-8?B?TDhuWUYyQmFHdE5pNHBCYmRycU00YTVCbTFvRnNBNm9UdGdGOXdKVlk1UGlX?= =?utf-8?B?MlM0dHNRRGsrSWhjNmVSQ2dPWnMreWc5QUZjekRqMjhkdXFmY0Nzam02ek00?= =?utf-8?B?aG1wdWR1NzZwaTk0S0VFVDRENC96eXloUk5CZ0JrUHpnK0M5b1B4NWZFdEhV?= =?utf-8?B?Z09pM0FhbHRoamczZzlZdDU2Yi81Y2E3T09YMUw2dVdsYkhuTGwwMUNCbVFJ?= =?utf-8?B?MXVLOHZaa0g5bDV5eUE0TUVBNjdnejFMWUpINzJTR0Q5S3NIU0x1M1FGYlBZ?= =?utf-8?B?aHhzU01WZTUyMzBUdnpUbjlMU2VWMXNTYjYyYzdlajMwOHQvcWZoa2ZjNXpo?= =?utf-8?B?RE9vcFFTekpSMGt1NzEvNlF1ZWpWcktIcUxMZkZiaHpaVjVzaDhIMFE3Y1dn?= =?utf-8?B?RzI1cXQ0Y3NPMnFQTXVVcHN1cjhrTlJuMUlZUmw4S3lXTkJDQzJTYXVYN1Rj?= =?utf-8?B?MFZQZkc5ZEJRZytKZ1UxaDl1Vlc2SXU0SGpDR1Vjb25wUFpWUDd6K05SYTdJ?= =?utf-8?B?WG5ueFpuNWpCUjIvcHBjS3hESTFyaGhHOW1MaUh0UThackdHVjkyRHZxd3Bq?= =?utf-8?B?M0ZPZjBIU0VSOFBXWTJVampOWitKRzRTaERhV3dUVStoOTkzdkpJWHE2cmF1?= =?utf-8?B?ZytlWnBiUHI1K01qRWFrV2R3YzBvOG1lZU4rQ3ozUHk5TWw5SzdsdUxnT1pW?= =?utf-8?B?OTJjbkZnVklzWXF0UXphVWxjdCtEOG1RVUJEUWpPVk1CdzN2cGw0eVY0SWlE?= =?utf-8?B?QUNqL0hIRzh4RGpEZzR6L1JCWU83UFkzQXVIdlNVYkRzWlNuTWlJNkhwTlZ2?= =?utf-8?B?L0k5eUdoc1lDNXFmaEM1cE5pZWFuNWFpc05ibWx3cWgzN1FkcDlrWUNGblRX?= =?utf-8?B?elVSR29KOFFqb040U0xzVE1Gd1NITVpvNFhtamlnM3VRLzFaNHRXNmZGQ255?= =?utf-8?B?Q29RdWxZOGhBMCtkWmNTYWJrWHpDOE9lclZycjlCQXJ5aVVRSm11ZXMvOWtj?= =?utf-8?B?L2ZSU29ESXZGREdDSk8rN2hHZ3ZSanVuR1RuQ3J0K0pGVVFJTnhtY3VFU0ZL?= =?utf-8?B?QVVkb2hlUXhIblBHbVBneVpQVExzbmhudytQWjFVZjRDeG1PeGh6ci9CSSt6?= =?utf-8?B?RGRYMHd3ZVNUMEZPbFFSTG56MGdhck9zUEFVaUxhSCt4MmxZZmU0b3VmMmFY?= =?utf-8?B?QkxXUW1ZUWFxT1FUYnJBWWNObGtUY25SbWxrZUNLdUF3SHRtaHpLZEFITHJ3?= =?utf-8?B?NUlGejFYVmNoMjhEVnMwZ3ZaNFFSSnA0eXFyeUNBRjNWbW95dlFvditrZHJ6?= =?utf-8?B?cys0UVFndGV2cG1aazNzL1IzbzZaVGRFOUhsdG1tdVJrWFJySS9KTjF4RHVl?= =?utf-8?B?VUFOK2NuV0dVVENvL2VRbG1ZZmdTcUdBZWNkV1hxbk05Z2p0TlNRZ3lPZUxj?= =?utf-8?B?NlJUOEZhQ01oSnYxU25hY0pwZFNVTHVISStKME1wNEoxL0t6K2k3Z0dsMSt2?= =?utf-8?B?NWRFSEtsZ1pQdnExdmVCWWoyYjlPUnVGNExnY0EyaUxEeWtCMWJnWHhBVjRL?= =?utf-8?B?WkJIRGZjTktOTFV5bGJtU3dPR0ZRMVFwRFEwUXRtY0R4ZWUvZ2Z6SndicWJn?= =?utf-8?B?MXJjT3hSNXFqMmtPd25tTElOUmI3dGp6TnR4ZDNUR09HZ2NEWi9sL1dZcVNI?= =?utf-8?B?VFhMNWxKNmVsdjRiZ3VHU0FhMlhkT05CbnFnY1V5WmFwRWRCM0ozaG1JNXlP?= =?utf-8?B?MVNMOTg5ZnljUVI5dUszUFozRjlOL2tsU2FNSHlITldvZlE9PQ==?= X-Microsoft-Exchange-Diagnostics: 1; DM5PR08MB2748; 6:wUmCuOAs6BcF/AKVulDfAm9E0J7EP2qxywXFoVJzRN0RBWMhpHvVjn5d27exqawAB+HrkLHBvCeExf0703yIgAKpU7WduRd0FRTaXXrTNeiRhD6pObjpf9hfcvg2uDUlmD+dECep4vf0c7Hifx8KGaAVp59XfZP9PMrOnTLbg9z1TMcgHyRIPhSjuMnp7SMIqvpcHwLRWdG07o5lM+nSSdI8PMIeO4cM36s4UlRYSgaPuMFWf4LwJipZR2naUaF4GsT4wYXjrpe2vZbAiQjbw4Ffr1Qp6SLBJhBQdxipwH8=; 5:jAXbPFnO5oWXXSj1WkRDDVvH3ngBx03DU5t/HecTATbtEertpYgt6VeizBKdQWg2hPpv6apemPH2VOUqYjsLk8A5mEU6XxPvtlbZ+Cyf0uVc+cuCsHBUNBd8PyOXsRycJF5BNeHqmBTP1u6C7eTyqw==; 24:MTKvdMey3ZZCoURVnaPmikvxyw5uw/nEOnPQCOTx2OnPHH2FP3gVGtMXHV49/rVNesbsuHY90W0nUymCjlnJaGkr43Iv6hc8NcZR8NKup5U=; 7:ev3r4S3kknOuxKiSlkDwCx/DXS1AzepZghI+jdN2Cs9ufDTDbKjw6oAh8o7WT0SZ1TrI6yCvAfOeF8IqpJbjhpC3BHEWsUFsAFU3T59WsZmL4KFK05e17o9fuIF3/ouEJsSnq1LfyAgU+ImLxdh3tsSn1PbKXgkwX4y1KHxGNpalkUW/8XFm4DUgVJUSFuw1v5ollWzJoA8ZQWL0ZwevLdNnVmXLM0drKjrnp3QnyVN0tW6IpEJPLgft6WC7FxHe SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: unl.edu X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Aug 2016 14:58:26.6917 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR08MB2748 Archived-At: Cc: draft-ietf-tcpm-cubic@ietf.org, tcpm-chairs@ietf.org Subject: [tcpm] A new Internet draft for TCP Cubic X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Aug 2016 14:58:40 -0000 Hello, We just submitted a new Internet draft for TCP Cubic. https://tools.ietf.org/html/draft-ietf-tcpm-cubic-02 Compare to the previous version 01, this version has the following changes. Most changes are based on the email discussions with Karen and Neal on draft-ietf-tcpm-cubic@ietf.org in March 2016. Thanks, Karen and Neal! Section 3.1: to more clearly describe the behavior of CUBIC * From "The protocol does not make any change to the fast recovery and retransmit of TCP-NewReno [RFC6582] and TCP-SACK [RFC2018]" to "The protocol does not make any change to the fast recovery and retransmit of TCP, such as TCP-NewReno [RFC6582] and TCP-SACK [RFC2018]" * From "t is the elapsed time from the last window reduction" to "t is the elapsed time from the last window reduction (measured right after fast recovery)" * From "K is the time period that the above function takes to increase the current window size to W_max when there is no further loss event" to "K is the time period that the above function takes to increase the current window size to W_max if there is no further loss event" Section 3.2: to more clearly explain TCP friendly region * From " This is done as follows. We can analyze the window size of a TCP-friendly AIMD in terms of the elapsed time t. Using a simple analysis in [FHP00], we can analyze the average window size of additive increase and multiplicative decrease (AIMD) with an additive factor alpha_aimd and a multiplicative factor beta_aimd with the following function:" to " This is done by estimating the average rate of the Standard TCP using a simple analysis described in . It considers the Standard TCP as a special case of an Additive Increase and Multiplicative Decrease algorithm (AIMD), which has an additive factor alpha_aimd and a multiplicative factor beta_aimd with the following function:" New Subsection 4.8 to reflect the recent changes by Jana Iyengar in 2015 * Add "4.8. Behavior for Application-Limited Flows CUBIC does not raise its congestion window size if the flow is currently limited by the application instead of the congestion window. In cases of idle periods, t in Eq. 1 should not include the idle time; otherwise, W_cubic(t) might be very high after restarting from a long idle time." Thanks Lisong From nobody Sun Aug 7 11:37:24 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 10AB712D549 for ; Sun, 7 Aug 2016 01:37:35 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.142 X-Spam-Level: X-Spam-Status: No, score=-1.142 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_PASS=-0.001, SPF_NEUTRAL=0.779] autolearn=no autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=qlgc.onmicrosoft.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id NuKBrPx7aeQb for ; Sun, 7 Aug 2016 01:37:32 -0700 (PDT) Received: from NAM01-SN1-obe.outbound.protection.outlook.com (mail-sn1nam01on0120.outbound.protection.outlook.com [104.47.32.120]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id AFFC612D544 for ; Sun, 7 Aug 2016 01:37:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=qlgc.onmicrosoft.com; s=selector1-qlogic-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=ooWe6cUO0KV9s59fuReZ0Gd9rvSpzCelNvQSsnATv+s=; b=ZcZb+zVPvOyZqXreUoL0gDDUZnKQjJlrdtzR8lOt9ksmaj4L1Hi4SpjtM3Skp3fdZKHT5W2sFqdKf4niKBDSiOpugA1W1K89rL12fMlUpu9d5qrGQROEVp38TyOqQuOLZ9TIDLdPSVhIhYsDmTlKuYyxWc2/pDWQO/O1Yj7849U= Received: from CY4PR11MB1878.namprd11.prod.outlook.com (10.175.61.140) by CY4PR11MB1878.namprd11.prod.outlook.com (10.175.61.140) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.549.15; Sun, 7 Aug 2016 08:37:28 +0000 Received: from CY4PR11MB1878.namprd11.prod.outlook.com ([10.175.61.140]) by CY4PR11MB1878.namprd11.prod.outlook.com ([10.175.61.140]) with mapi id 15.01.0549.025; Sun, 7 Aug 2016 08:37:28 +0000 From: Kobby Carmona To: David Borman Thread-Topic: [tcpm] Possible deadlock scenario with retransmission on both sides at the same time Thread-Index: AdHth2zqZmjYio0mRrKVRRoD8V9FcgBABbmAAH+o2jA= Date: Sun, 7 Aug 2016 08:37:28 +0000 Message-ID: References: <7070553C-65D1-46EE-95F4-DAE82E1F5A5E@weston.borman.com> In-Reply-To: <7070553C-65D1-46EE-95F4-DAE82E1F5A5E@weston.borman.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=kobby.Carmona@qlogic.com; x-originating-ip: [31.168.140.228] x-ms-office365-filtering-correlation-id: a1a0f996-ff6c-4050-f2f3-08d3be9e1105 x-microsoft-exchange-diagnostics: 1; CY4PR11MB1878; 20:LdcviSr6AyTW2rURqjeTBncT+/Miz0M/45Gq2lWaJ2pJWMIYXy8elbu3ZROy6tZdK9yLlMR8Mrlq8/4otkx/zdmpzlhbh3IS7rk4np6oA9u9K4KUafNIeRYsg7m9yz4biwI6lTIqI/I4gC0c19LBqIrj9md4tt1JwEdIl346eeI= x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:CY4PR11MB1878; x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:; x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(601004)(2401047)(5005006)(8121501046)(3002001)(10201501046); SRVR:CY4PR11MB1878; BCL:0; PCL:0; RULEID:; SRVR:CY4PR11MB1878; x-forefront-prvs: 0027ED21E7 x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(6009001)(7916002)(199003)(189002)(377454003)(24454002)(53754006)(13464003)(66066001)(99286002)(87936001)(586003)(3280700002)(122556002)(2906002)(19580405001)(77096005)(86362001)(2950100001)(74316002)(305945005)(2900100001)(105586002)(4326007)(76576001)(106356001)(19580395003)(9686002)(189998001)(15975445007)(33656002)(110136002)(102836003)(3846002)(10400500002)(50986999)(8936002)(6116002)(8676002)(97736004)(3660700001)(76176999)(81156014)(81166006)(54356999)(68736007)(101416001)(5002640100001)(7696003)(7736002)(11100500001)(92566002)(7846002); DIR:OUT; SFP:1102; SCL:1; SRVR:CY4PR11MB1878; H:CY4PR11MB1878.namprd11.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; received-spf: None (protection.outlook.com: qlogic.com does not designate permitted sender hosts) spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 MIME-Version: 1.0 X-OriginatorOrg: qlogic.com X-MS-Exchange-CrossTenant-originalarrivaltime: 07 Aug 2016 08:37:28.3049 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 0d68a1f9-1490-4d0e-8767-a87dab3ef2ba X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR11MB1878 Archived-At: X-Mailman-Approved-At: Sun, 07 Aug 2016 11:37:22 -0700 Cc: "tcpm@ietf.org" Subject: Re: [tcpm] Possible deadlock scenario with retransmission on both sides at the same time X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Aug 2016 08:37:35 -0000 SGkgRGF2aWQsDQpUaGUgY29kZSBiZWxvdyBpcyB0cnVlIGZvciBmYXN0LXJldHJhbnNtaXQuDQpC dXQgaW4gY2FzZSBvZiByZXRyYW5zbWl0IHRpbWVyIGV4cGlyYXRpb24gKFRDUFRfUkVYTVQpIHNu ZF9ueHQgaXMgc2V0IHRvIHNuZF91bmEuIEFuZCBpbiB0aGlzIGNhc2UgQ1FORCB3aWxsIGJlIHNl dCB0byAxTVNTIHNvIGluIHRoZSBleGFtcGxlIGJlbG93IHRoZSB0cmFuc21pdHRlcnMgY2FuIHNl bmQgb25seSBhIHNpbmdsZSBzZWdtZW50IGZyb20gc2VxdWVuY2UgMjAwMC8xMjAwMC4NCg0KCUtv YmJ5DQoNCi0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQpGcm9tOiBEYXZpZCBCb3JtYW4gW21h aWx0bzpkYWJAd2VzdG9uLmJvcm1hbi5jb21dIA0KU2VudDogVGh1cnNkYXksIEF1Z3VzdCAwNCwg MjAxNiAxMDozNyBQTQ0KVG86IEtvYmJ5IENhcm1vbmEgPGtvYmJ5LkNhcm1vbmFAcWxvZ2ljLmNv bT4NCkNjOiB0Y3BtQGlldGYub3JnDQpTdWJqZWN0OiBSZTogW3RjcG1dIFBvc3NpYmxlIGRlYWRs b2NrIHNjZW5hcmlvIHdpdGggcmV0cmFuc21pc3Npb24gb24gYm90aCBzaWRlcyBhdCB0aGUgc2Ft ZSB0aW1lDQoNCkFmdGVyIHlvdSBwdWxsIGJhY2sgU05ELk5YVCBhbmQgZG8gdGhlIHJldHJhbnNt aXNzaW9uLCB5b3Ugc2hvdWxkIHRoZW4gcmVzdG9yZSBTTkQuTlhUIGJhY2sgdG8gd2hlcmUgaXQg d2FzLCBub3QgbGVhdmUgaXQgYXQgdGhlIGJhY2tlZCBvZmYgdmFsdWU7IHRoZW4gdGhlIEFDS3Mg d291bGRu4oCZdCBiZSBkcm9wcGVkLCBzaW5jZSB0aGV5IHdvdWxkbuKAmXQgaGF2ZSBvbGQgc2Vx IHZhbHVlcy4gIEZvciBleGFtcGxlLCBpbiB0aGUgNC40QlNEIGZhc3QgcmV0cmFuc21pdCBjb2Rl IGl0IGhhZDoNCg0KCQkJdGNwX3NlcSBvbnh0ID0gdHAtPnNuZF9ueHQ7DQoJCQkuLi4NCgkJCXRw LT5zbmRfbnh0ID0gdGgtPnRoX2FjazsNCgkJCS4uLg0KCQkJKHZvaWQpIHRjcF9vdXRwdXQodHAp Ow0KCQkJLi4uDQoJCQlpZiAoU0VRX0dUKG9ueHQsIHRwLT5zbmRfbnh0KSkgIA0KCQkJCXRwLT5z bmRfbnh0ID0gb254dDsNCg0KDQoJCQktRGF2aWQgQm9ybWFuDQoNCj4gT24gQXVnIDQsIDIwMTYs IGF0IDQ6MDggQU0sIEtvYmJ5IENhcm1vbmEgPGtvYmJ5LkNhcm1vbmFAcWxvZ2ljLmNvbT4gd3Jv dGU6DQo+IA0KPiBIaSBhbGwsDQo+IFdoaWxlIHJ1bm5pbmcgYSBiaWRpcmVjdGlvbmFsIHNjZW5h cmlvIHdpdGggcmFuZG9tIGRyb3BzIGluIGEgbmV0d29yayBzaW11bGF0b3Igb2Ygb3VyIChRTG9n aWMncyBOSUMpIFRDUCBzdGFjayB3ZSBmb3VuZCBhIGNhc2Ugd2hlcmUgaXQgc2VlbXMgdGhlcmUg aXMgZGVhZGxvY2sgaW4gdGhlIFRDUCBwcm90b2NvbCAodGhlIGNvbm5lY3Rpb24gd2lsbCBrZWVw IHNlbmRpbmcgcHVyZSBhY2tzIGZyb20gYm90aCBzaWRlcyB1bnRpbCBSVE8gd2lsbCBleHBpcmUg bXVsdGlwbGUgdGltZXMgYW5kIGEgUlNUIHdpbGwgc2VudCB0byBjbG9zZSB0aGUgY29ubmVjdGlv bikuDQo+IFRoZSBzY2VuYXJpbyBpcyBhcyBmb2xsb3dzICh0aGVyZSBpcyBhbiBleGFtcGxlIHdp dGggbnVtYmVycyBmb3IgZWFjaCBzdGFnZSBhc3N1bWluZyB0aGUgTVNTIGFuZCBlYWNoIHBhY2tl dCBpcyAxMDAwQik6DQo+IDEuIEJvdGggc2lkZXMgYXJlIHRyYW5zbWl0dGluZyBkYXRhIGFuZCBh IHNpbmdsZSBwYWNrZXQgaXMgZHJvcHBlZCBvbiBlaXRoZXIgc2lkZSBhbmQgdGhlIG5leHQgdHdv IHBhY2tldHMgYXJlIHJlY2VpdmVkIHByb3Blcmx5DQo+IAlTaWRlIEEgLSBTTkQuTUFYPTMwMDAs IFNORC5OWFQ9MzAwMCwgU05ELlVOQT0xMDAwLCBSQ1YuTlhUPTExMDAwLCBvdXQtb2Ytb3JkZXIg YmxvY2sgMTIwMDAtMTMwMDANCj4gCVNpZGUgQiAtIFNORC5NQVggPTEzMDAwLCBTTkQuTlhUID0x MzAwMCwgU05ELlVOQT0xMTAwMCwgUkNWLk5YVD0xMDAwLCANCj4gb3V0LW9mLW9yZGVyIGJsb2Nr IDIwMDAtMzAwMCAyLiBSVE8gdGltZXIgZXhwaXJlcyBvbiBib3RoIHNpZGVzDQo+IAlTaWRlIEEg LSBTTkQuTUFYPTMwMDAsIFNORC5OWFQ9MTAwMCwgU05ELlVOQT0xMDAwLCBSQ1YuTlhUPTExMDAw LCBvdXQtb2Ytb3JkZXIgYmxvY2sgMTIwMDAtMTMwMDANCj4gCVNpZGUgQiAtIFNORC5NQVggPTEz MDAwLCBTTkQuTlhUPTExMDAwLCBTTkQuVU5BPTExMDAwLCBSQ1YuTlhUPTEwMDAsIA0KPiBvdXQt b2Ytb3JkZXIgYmxvY2sgMjAwMC0zMDAwIDMuIEJvdGggc2lkZXMgdHJhbnNtaXQgYSBzaW5nbGUg cGFja2V0IHRvIHRoZSBwZWVyOg0KPiAJQS0+QiAtIHBrdC5zZXE9MTAwMCwgcGt0LmFjaz0xMTAw MCwgbGVuPTEwMDANCj4gCUItPkEgLSBwa3Quc2VxPTExMDAwLCBwa3QuYWNrPTEwMDAsIGxlbj0x MDAwIDMuIEJvdGggc2lkZXMgcmVjZWl2ZSANCj4gdGhlIHBhY2tldHMgYW5kIHVwZGF0ZSB0aGUg cmVjZWl2ZSBjb250ZXh0Og0KPiAJU2lkZSBBIC0gU05ELk1BWD0zMDAwLCBTTkQuTlhUPTIwMDAs IFNORC5VTkE9MTAwMCwgUkNWLk5YVD0xMzAwMA0KPiAJU2lkZSBCIC0gU05ELk1BWD0xMzAwMCwg U05ELk5YVD0xMjAwMCwgU05ELlVOQT0xMTAwMCwgUkNWLk5YVD0zMDAwIDQuIA0KPiBCb3RoIHNp ZGVzIHNlbmQgYW5vdGhlciBzZWdtZW50Og0KPiAJQS0+QiAtIHBrdC5zZXE9MjAwMCwgcGt0LmFj az0xMzAwMCwgbGVuPTEwMDANCj4gCUItPkEgLSBwa3Quc2VxPTEyMDAwLCBwa3QuYWNrPTMwMDAs IGxlbj0xMDAwIDUuIEJvdGggc2lkZXMgZG9uJ3QgDQo+IGFjY2VwdCB0aGUgcGFja2V0IChhbmQg ZG9uJ3QgdXBkYXRlIFNORC5VTkEpIHNpbmNlIHRoZSBzZXF1ZW5jZSBvbiB0aGUgcGFja2V0IGlz IGxlc3MgdGhhbiBSQ1YuTlhUIChzZXF1ZW5jZSBudW1iZXIgY2hlY2sgaW4gcGFnZSA2OSBvZiBS RkM3OTMpIGFuZCBzZW5kIGEgcHVyZSBBQ0sgaW5zdGVhZA0KPiAJQS0+QiAtIHBrdC5zZXE9MjAw MCwgcGt0LmFjaz0xMzAwMCwgbGVuPTAgKHB1cmUgQUNLKQ0KPiAJQi0+QSAtIHBrdC5zZXE9MTIw MDAsIHBrdC5hY2s9MzAwMCwgbGVuPTAgKHB1cmUgQUNLKSA2LiBUaGlzIHdpbGwgDQo+IGNvbnRp bnVlIGZvcmV2ZXIgKHVudGlsIHRoZSBjb25uZWN0aW9uIHdpbGwgYmUgdGVybWluYXRlZCBieSBS U1QpIHNpbmNlIGV2ZXJ5IHBhY2tldCB0aGF0IGVuZHMgYmVmb3JlIFJDVi5OWFQgKGV2ZW4gYSBy ZXRyYW5zbWl0IGZyb20gU05ELlVOQSkgd2lsbCBiZSBkcm9wcGVkLg0KPiANCj4gRGlkIGFueW9u ZSBlbmNvdW50ZXJlZCB0aGlzIGlzc3VlIGJlZm9yZT8gSXMgdGhlIGFueXRoaW5nIHdlIG1pc3Nl ZCBvbiB0aGlzIHNlcXVlbmNlPw0KPiBJZiB0aGlzIGlzIGluZGVlZCBhIHJlYWwgZGVhZGxvY2ss IHRoZXJlIG1pZ2h0IGJlIHNldmVyYWwgc29sdXRpb25zIHRvIHRoaXMgd2hpY2ggd2lsbCByZXF1 aXJlIGEgbW9kaWZpY2F0aW9uIGluIHJlY2VpdmUgcHJvY2Vzc2luZyBvZiBSRkM3OTMuIEJ1dCBJ IHdvdWxkIGxpa2UgdG8ga25vdyBpZiB5b3UgdGhpbmsgdGhpcyBpcyBhIHJlYWwgaXNzdWUgYmVm b3JlIGRlYWxpbmcgd2l0aCBzb2x1dGlvbnMuDQo+IFRoYW5rcywNCj4gDQo+IAlLb2JieQ0KPiAN Cj4gDQo+IF9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fDQo+ IHRjcG0gbWFpbGluZyBsaXN0DQo+IHRjcG1AaWV0Zi5vcmcNCj4gaHR0cHM6Ly93d3cuaWV0Zi5v cmcvbWFpbG1hbi9saXN0aW5mby90Y3BtDQoNCg== From nobody Sun Aug 7 14:35:11 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 34EA912D69B for ; Sun, 7 Aug 2016 14:35:11 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.147 X-Spam-Level: X-Spam-Status: No, score=-3.147 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RP_MATCHES_RCVD=-1.247] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id sXMoQKhz934M for ; Sun, 7 Aug 2016 14:35:09 -0700 (PDT) Received: from frantic.weston.borman.com (frantic.weston.borman.com [70.57.156.33]) (using TLSv1 with cipher EDH-RSA-DES-CBC3-SHA (112/168 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 19D4512D589 for ; Sun, 7 Aug 2016 14:35:09 -0700 (PDT) Received: from local-54.weston.borman.com (local-54.weston.borman.com [192.168.1.54]) by frantic.weston.borman.com (8.14.7/8.14.7) with ESMTP id u77LZ4X5004649; Sun, 7 Aug 2016 16:35:05 -0500 (CDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) From: David Borman In-Reply-To: Date: Sun, 7 Aug 2016 16:35:04 -0500 Content-Transfer-Encoding: quoted-printable Message-Id: <2D524A8D-A5CA-45A6-B94D-FA1DA0CEE609@weston.borman.com> References: <7070553C-65D1-46EE-95F4-DAE82E1F5A5E@weston.borman.com> To: Kobby Carmona X-Mailer: Apple Mail (2.3124) Archived-At: Cc: "tcpm@ietf.org" Subject: Re: [tcpm] Possible deadlock scenario with retransmission on both sides at the same time X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Aug 2016 21:35:11 -0000 > On Aug 7, 2016, at 3:37 AM, Kobby Carmona = wrote: >=20 > Hi David, > The code below is true for fast-retransmit. > But in case of retransmit timer expiration (TCPT_REXMT) snd_nxt is set = to snd_una. And in this case CQND will be set to 1MSS so in the example = below the transmitters can send only a single segment from sequence = 2000/12000. Your underlying problem is that the ACK-only packets are being sent with = the wrong sequence number, and that is what is causing them to be = dropped. One way to fix that is to put SND.NXT back to its previous = value after do the the retransmit. But you are correct, in the BSD code = it only does that in the fast retransmit code; when the timer based = retransmit code fires, it just pulls back SND.NXT to SND.UNA. However, = in the BSD code that I=E2=80=99m looking at, it keeps track of the = largest sequence number sent in SND.MAX, and in the tcp_output() path = there is this bit of code:=20 /* * If we are doing retransmissions, then snd_nxt will * not reflect the first unsent octet. For ACK only * packets, we do not want the sequence number of the=20 * retransmitted packet, we want the sequence number * of the next unsent octet. So, if there is no data * (and no SYN or FIN), use snd_max instead of snd_nxt * when filling in th_seq. But if we are in persist * state, snd_max might reflect one byte beyond the * right edge of the window, so use snd_nxt in that * case, since we know we aren't doing a retransmission. * (retransmit and persist are mutually exclusive...) */ if (len || (flags & (TH_SYN|TH_FIN)) || = tp->t_timer[TCPT_PERSIST]) th->th_seq =3D htonl(tp->snd_nxt); else th->th_seq =3D htonl(tp->snd_max); That is what your implementation appears to be missing, and what is = causing your ACK storm. So yes, some variant of your problem has been = seen before, and this is how the BSD code fixed it. -David Borman >=20 > Kobby >=20 > -----Original Message----- > From: David Borman [mailto:dab@weston.borman.com]=20 > Sent: Thursday, August 04, 2016 10:37 PM > To: Kobby Carmona > Cc: tcpm@ietf.org > Subject: Re: [tcpm] Possible deadlock scenario with retransmission on = both sides at the same time >=20 > After you pull back SND.NXT and do the retransmission, you should then = restore SND.NXT back to where it was, not leave it at the backed off = value; then the ACKs wouldn=E2=80=99t be dropped, since they wouldn=E2=80=99= t have old seq values. For example, in the 4.4BSD fast retransmit code = it had: >=20 > tcp_seq onxt =3D tp->snd_nxt; > ... > tp->snd_nxt =3D th->th_ack; > ... > (void) tcp_output(tp); > ... > if (SEQ_GT(onxt, tp->snd_nxt)) =20 > tp->snd_nxt =3D onxt; >=20 >=20 > -David Borman >=20 >> On Aug 4, 2016, at 4:08 AM, Kobby Carmona = wrote: >>=20 >> Hi all, >> While running a bidirectional scenario with random drops in a network = simulator of our (QLogic's NIC) TCP stack we found a case where it seems = there is deadlock in the TCP protocol (the connection will keep sending = pure acks from both sides until RTO will expire multiple times and a RST = will sent to close the connection). >> The scenario is as follows (there is an example with numbers for each = stage assuming the MSS and each packet is 1000B): >> 1. Both sides are transmitting data and a single packet is dropped on = either side and the next two packets are received properly >> Side A - SND.MAX=3D3000, SND.NXT=3D3000, SND.UNA=3D1000, = RCV.NXT=3D11000, out-of-order block 12000-13000 >> Side B - SND.MAX =3D13000, SND.NXT =3D13000, SND.UNA=3D11000, = RCV.NXT=3D1000,=20 >> out-of-order block 2000-3000 2. RTO timer expires on both sides >> Side A - SND.MAX=3D3000, SND.NXT=3D1000, SND.UNA=3D1000, = RCV.NXT=3D11000, out-of-order block 12000-13000 >> Side B - SND.MAX =3D13000, SND.NXT=3D11000, SND.UNA=3D11000, = RCV.NXT=3D1000,=20 >> out-of-order block 2000-3000 3. Both sides transmit a single packet = to the peer: >> A->B - pkt.seq=3D1000, pkt.ack=3D11000, len=3D1000 >> B->A - pkt.seq=3D11000, pkt.ack=3D1000, len=3D1000 3. Both sides = receive=20 >> the packets and update the receive context: >> Side A - SND.MAX=3D3000, SND.NXT=3D2000, SND.UNA=3D1000, = RCV.NXT=3D13000 >> Side B - SND.MAX=3D13000, SND.NXT=3D12000, SND.UNA=3D11000, = RCV.NXT=3D3000 4.=20 >> Both sides send another segment: >> A->B - pkt.seq=3D2000, pkt.ack=3D13000, len=3D1000 >> B->A - pkt.seq=3D12000, pkt.ack=3D3000, len=3D1000 5. Both sides = don't=20 >> accept the packet (and don't update SND.UNA) since the sequence on = the packet is less than RCV.NXT (sequence number check in page 69 of = RFC793) and send a pure ACK instead >> A->B - pkt.seq=3D2000, pkt.ack=3D13000, len=3D0 (pure ACK) >> B->A - pkt.seq=3D12000, pkt.ack=3D3000, len=3D0 (pure ACK) 6. = This will=20 >> continue forever (until the connection will be terminated by RST) = since every packet that ends before RCV.NXT (even a retransmit from = SND.UNA) will be dropped. >>=20 >> Did anyone encountered this issue before? Is the anything we missed = on this sequence? >> If this is indeed a real deadlock, there might be several solutions = to this which will require a modification in receive processing of = RFC793. But I would like to know if you think this is a real issue = before dealing with solutions. >> Thanks, >>=20 >> Kobby >>=20 >>=20 >> _______________________________________________ >> tcpm mailing list >> tcpm@ietf.org >> https://www.ietf.org/mailman/listinfo/tcpm >=20 > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm From nobody Thu Aug 11 09:13:15 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0D74412D5DB for ; Wed, 10 Aug 2016 11:36:57 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.869 X-Spam-Level: X-Spam-Status: No, score=-103.869 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-1.247, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, USER_IN_WHITELIST=-100] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id T-7qUG-Fq7lC for ; Wed, 10 Aug 2016 11:36:54 -0700 (PDT) Received: from rfc-editor.org (rfc-editor.org [4.31.198.49]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1470C12D1C8 for ; Wed, 10 Aug 2016 11:36:54 -0700 (PDT) Received: by rfc-editor.org (Postfix, from userid 30) id 05358B80C3A; Wed, 10 Aug 2016 11:36:54 -0700 (PDT) To: ananth@cisco.com, randall@lakerest.net, mdalal@cisco.com, spencerdawkins.ietf@gmail.com, ietf@kuehlewind.net, michael.scharf@nokia.com, nishida@sfc.wide.ad.jp, pasi.sarolahti@iki.fi X-PHP-Originating-Script: 30:errata_mail_lib.php From: RFC Errata System Message-Id: <20160810183654.05358B80C3A@rfc-editor.org> Date: Wed, 10 Aug 2016 11:36:54 -0700 (PDT) Archived-At: X-Mailman-Approved-At: Thu, 11 Aug 2016 09:13:14 -0700 Cc: bortzmeyer+ietf@nic.fr, tcpm@ietf.org, rfc-editor@rfc-editor.org Subject: [tcpm] [Technical Errata Reported] RFC5961 (4772) X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Aug 2016 18:36:57 -0000 The following errata report has been submitted for RFC5961, "Improving TCP's Robustness to Blind In-Window Attacks". -------------------------------------- You may review the report below and at: http://www.rfc-editor.org/errata_search.php?rfc=5961&eid=4772 -------------------------------------- Type: Technical Reported by: Stéphane Bortzmeyer Section: 7 Original Text ------------- [The entire section] Corrected Text -------------- No suggested text because it requires a much more serious analysis. May be adding that the rate-limit counter SHOULD be per-connection, in the spirit of RFC 6528? Notes ----- It appears the section does not specify that the counter for ACK throttling SHOULD be per-connection. In Linux, it is apparently global, which allowed its use as a side channel enabling nasty attacks (CVE-2016-5696 and the paper "Off-Path TCP Exploits: Global Rate Limit Considered Dangerous" ) Instructions: ------------- This erratum is currently posted as "Reported". If necessary, please use "Reply All" to discuss whether it should be verified or rejected. When a decision is reached, the verifying party (IESG) can log in to change the status and edit the report, if necessary. -------------------------------------- RFC5961 (draft-ietf-tcpm-tcpsecure-13) -------------------------------------- Title : Improving TCP's Robustness to Blind In-Window Attacks Publication Date : August 2010 Author(s) : A. Ramaiah, R. Stewart, M. Dalal Category : PROPOSED STANDARD Source : TCP Maintenance and Minor Extensions Area : Transport Stream : IETF Verifying Party : IESG From nobody Thu Aug 11 09:27:56 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1FB3712D81C for ; Thu, 11 Aug 2016 09:27:54 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.678 X-Spam-Level: X-Spam-Status: No, score=-1.678 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, MALFORMED_FREEMAIL=0.001, MISSING_HEADERS=1.021, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=no autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kFqsFro_f46l for ; Thu, 11 Aug 2016 09:27:52 -0700 (PDT) Received: from mail-pf0-x22b.google.com (mail-pf0-x22b.google.com [IPv6:2607:f8b0:400e:c00::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id ADF4712D800 for ; Thu, 11 Aug 2016 09:27:52 -0700 (PDT) Received: by mail-pf0-x22b.google.com with SMTP id h186so160328pfg.3 for ; Thu, 11 Aug 2016 09:27:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:cc :content-transfer-encoding; bh=VoDHKn+lxxgNTnXNpGBY23w+N7ezsHmldfr9PrBhoWE=; b=T8ZYETyYlerVFZUAj9IlTu3pkpRkq9ikp67NG4sXbOgFkYetrq8AlXW7CkppF9oWYR S1F2vZLsZBD7d4jtB7rZbeXzITR0PhI6vZ4eSLtH8EKf1wVY/3iLnNtiQtVY5BgfOYl5 1fcK0dt4CgTpzPsP/zKNjEjJGC/G+ou4xe4B9uhhHVIIBew7CyjnAmRHqvucbwP8jFsK /alG7PGCnhNp82RBZk6TOzEE5HCpFWbgrY1fsunNlLfCNEzSyn+NG2QR8syh1+GuvQ40 Ah15id3tiIWhx8v34UtR3aJJyBMmN1s5DZEByGMwl7jJHKV2aKwEDqqP6nmm3pQsu+5O qH5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:cc:content-transfer-encoding; bh=VoDHKn+lxxgNTnXNpGBY23w+N7ezsHmldfr9PrBhoWE=; b=Z9b4pLnVOZK3xCNS79jh40mV87UAOv9nUgTQovnmIfum/YM7iWOL6wM/4f8YK9qoCZ ZttobPf80Q0iI5AhHLOxBCRn5I487HOTe2SOBmP19CcE6TYt+eEqU64y88kFj2rZB/Rg ZQwtz6KOy66YMtDiydj6Eq6IRrL7fU0Z7+6SozXYSVwVXGGeV4sdgxTRVPSABsNkylBE w7KylDx0i8yzYz71LEG0ZY5CzwxEqR3bjOgNgJabKnqXTfg373DYmITv4brh1x0FG/m4 P6WsjQitpeumJ0RlERWh4EoUm9wTPzjoTsYFEG/4663uRgioZOlW2e3UytrFrWoNnP/V +/Kg== X-Gm-Message-State: AEkoouuoKSOBvq9ZzRBIh7zINfPvhU1vFimwOE9m+GdzW8pY2v9Bzo4GWBvZNwNQlJTBxi1RMxPAxV8jQZ3peg== X-Received: by 10.98.47.132 with SMTP id v126mr18944882pfv.152.1470932872186; Thu, 11 Aug 2016 09:27:52 -0700 (PDT) MIME-Version: 1.0 Received: by 10.66.142.133 with HTTP; Thu, 11 Aug 2016 09:27:51 -0700 (PDT) In-Reply-To: <20160810183654.05358B80C3A@rfc-editor.org> References: <20160810183654.05358B80C3A@rfc-editor.org> From: Loganaden Velvindron Date: Thu, 11 Aug 2016 20:27:51 +0400 Message-ID: Cc: tcpm@ietf.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Archived-At: Subject: Re: [tcpm] [Technical Errata Reported] RFC5961 (4772) X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Aug 2016 16:27:54 -0000 I submitted this draft, yesterday.: https://tools.ietf.org/html/draft-lvelvindron-ack-throttling-02 It's a work-in-progress, but I welcome feedback. On Wed, Aug 10, 2016 at 10:36 PM, RFC Errata System wrote: > The following errata report has been submitted for RFC5961, > "Improving TCP's Robustness to Blind In-Window Attacks". > > -------------------------------------- > You may review the report below and at: > http://www.rfc-editor.org/errata_search.php?rfc=3D5961&eid=3D4772 > > -------------------------------------- > Type: Technical > Reported by: St=C3=A9phane Bortzmeyer > > Section: 7 > > Original Text > ------------- > [The entire section] > > Corrected Text > -------------- > No suggested text because it requires a much more serious analysis. > May be adding that the rate-limit counter SHOULD be per-connection, > in the spirit of RFC 6528? > > Notes > ----- > It appears the section does not specify that the counter for ACK throttli= ng SHOULD be per-connection. In Linux, it is apparently global, which allow= ed its use as a side channel enabling nasty attacks (CVE-2016-5696 and the = paper "Off-Path TCP Exploits: Global Rate Limit Considered Dangerous" ) > > Instructions: > ------------- > This erratum is currently posted as "Reported". If necessary, please > use "Reply All" to discuss whether it should be verified or > rejected. When a decision is reached, the verifying party (IESG) > can log in to change the status and edit the report, if necessary. > > -------------------------------------- > RFC5961 (draft-ietf-tcpm-tcpsecure-13) > -------------------------------------- > Title : Improving TCP's Robustness to Blind In-Window Attac= ks > Publication Date : August 2010 > Author(s) : A. Ramaiah, R. Stewart, M. Dalal > Category : PROPOSED STANDARD > Source : TCP Maintenance and Minor Extensions > Area : Transport > Stream : IETF > Verifying Party : IESG > > > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm > From nobody Thu Aug 11 12:29:00 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8061212D149 for ; Thu, 11 Aug 2016 12:28:58 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.022 X-Spam-Level: X-Spam-Status: No, score=-2.022 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=microsoft.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id wvshY45zKv_Y for ; Thu, 11 Aug 2016 12:28:56 -0700 (PDT) Received: from NAM03-BY2-obe.outbound.protection.outlook.com (mail-by2nam03on0119.outbound.protection.outlook.com [104.47.42.119]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1B9D612B005 for ; Thu, 11 Aug 2016 12:28:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=DxfkdDz2FJ5kwgI8yDBxZkPf1sw5/bkZnH35qAul6ng=; b=jfEGqN5EmK1j7JuuRNBtUz35lwdCW3/2Ov06tNv3A1Fuq2/vTWfy+Jew7NqIWkc+Qfq0gpRCM/jVix/fMPoc3kn7u//wbJgyko5kGiYkHkBF8Ay2fnXeipHyz8noHQj0PobycT+u+/BRMg/dCsQ8nJx6gV38n9JoKacerIzWyx8= Received: from BN1PR03MB008.namprd03.prod.outlook.com (10.255.224.38) by BN1PR03MB006.namprd03.prod.outlook.com (10.255.224.36) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.539.14; Thu, 11 Aug 2016 19:28:53 +0000 Received: from BN1PR03MB008.namprd03.prod.outlook.com ([169.254.5.80]) by BN1PR03MB008.namprd03.prod.outlook.com ([169.254.5.80]) with mapi id 15.01.0523.031; Thu, 11 Aug 2016 19:28:52 +0000 From: Praveen Balasubramanian To: Loganaden Velvindron Thread-Topic: [tcpm] [Technical Errata Reported] RFC5961 (4772) Thread-Index: AQHR8+tIvuFs/wXw4k+7lDUeDoRk/6BD8sWAgAAxdeA= Date: Thu, 11 Aug 2016 19:28:51 +0000 Message-ID: References: <20160810183654.05358B80C3A@rfc-editor.org> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=pravb@microsoft.com; x-originating-ip: [2001:4898:80e8:3::584] x-ms-office365-filtering-correlation-id: b26d8a93-ecd2-4774-97ee-08d3c21dba0a x-microsoft-exchange-diagnostics: 1; BN1PR03MB006; 6:nHghnox5Hob6gJkHbbUwm5x66EdSCrlet/dssdinXURI3160LbvfqQXG8QNkEbbs3t7YHmatvG3a9w+opZCThBbLDTy//dmJVRwnu11OO6NgbmRkLLjP0bZVjAQqMdTkfjON35KRE4AwP9XCRkdhA4OM0pY0kK2LmcuLcmZFBe1h8R5NQZYOpE01VEplGMSXN8AzN0Z1u/LD5BoQyJfPIoztztWjankTi/FiHfR+5hXqFMYXanFKuzDrJMujq5ALRkyrzAO1z2fLjBQBS1km4cYUMfC0mg7Feyms2cE6NIFErK2G5jSfbP867QFsXR2VbfzNiFqM6K2YrHUQWhw6UQ==; 5:kDAU+gwzEa6sjMRZEjEiSJ9xe+nRXnlm7/3lkO9ehCDsLyq6JGjVmLhSux3Lw1O/h5tqRk18go7AqaTxCznUP9RVCyR/OSMfDYBIKet9M4f0jghyOP2ELvTS+huYRZoBLD4sIpB6dbVkXC34lds6fA==; 24:l2eSh2fH7SxWGEdLXTzQubzNXrQrygSRJJq7xTc0lzjgpbchvERDyE61ETmdHpqfjI8MdqTxyQRz/4P+Yybt6qLU1Qeo3fHi9InOPWNnyGM=; 7:V1kBIBd91IzSCrxCteVajV1aDdCxcMETV1wyXRomUtl+HT/VNGT0i2QdJu6bXsRBPkQSiwm8LUazOlk3ckhf6Teiqw21ODIdYW6+eoOxvoBSm2diG47Z6blNoArdevtoQIz5S/wGdePcZq6B+tJH97tePQETuVNvdYBWTdnVHOWhJUgdgJWOIT04Iu8/YOB3sVJThb4Yc7kXD273+ERarxfuCkZO/njPlXpKszbaKA6k8ZwWisSuAD1PRUxWwLKl x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:BN1PR03MB006; x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:; x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(61425038)(6040173)(601004)(2401047)(5005006)(8121501046)(3002001)(10201501046)(6055026)(61426038)(61427038); SRVR:BN1PR03MB006; BCL:0; PCL:0; RULEID:; SRVR:BN1PR03MB006; x-forefront-prvs: 0031A0FFAF x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(6009001)(7916002)(189002)(199003)(13464003)(24454002)(377454003)(189998001)(76576001)(87936001)(15188155005)(97736004)(3280700002)(1411001)(9686002)(19580405001)(106356001)(101416001)(122556002)(16799955002)(81166006)(106116001)(99286002)(105586002)(19580395003)(74316002)(81156014)(8936002)(7736002)(7846002)(92566002)(86612001)(8990500004)(5002640100001)(5005710100001)(4326007)(54356999)(10090500001)(50986999)(6116002)(10400500002)(586003)(76176999)(68736007)(110136002)(2906002)(86362001)(15975445007)(2900100001)(10290500002)(2950100001)(33656002)(7696003)(305945005)(102836003)(3660700001)(8676002)(3826002); DIR:OUT; SFP:1102; SCL:1; SRVR:BN1PR03MB006; H:BN1PR03MB008.namprd03.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; received-spf: None (protection.outlook.com: microsoft.com does not designate permitted sender hosts) spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 MIME-Version: 1.0 X-OriginatorOrg: microsoft.com X-MS-Exchange-CrossTenant-originalarrivaltime: 11 Aug 2016 19:28:51.9108 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN1PR03MB006 Archived-At: Cc: "tcpm@ietf.org" Subject: Re: [tcpm] [Technical Errata Reported] RFC5961 (4772) X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Aug 2016 19:28:58 -0000 VGhlIFdpbmRvd3MgVENQIGltcGxlbWVudGF0aW9uIGRvZXMgcGVyLXNvY2tldCB0aHJvdHRsaW5n IGluc3RlYWQgb2YgZ2xvYmFsIGFuZCB3YXMgZm91bmQgaW4gdGhlIHBhcGVyIHRvIGJlIG5vdCB2 dWxuZXJhYmxlIHRvIHRoZSBleHBsb2l0LiBJIHRoaW5rIHdlIGRvIG5lZWQgYW4gZXJyYXRhIHRv IDU5NjEuDQoNCi0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQpGcm9tOiB0Y3BtIFttYWlsdG86 dGNwbS1ib3VuY2VzQGlldGYub3JnXSBPbiBCZWhhbGYgT2YgTG9nYW5hZGVuIFZlbHZpbmRyb24N ClNlbnQ6IFRodXJzZGF5LCBBdWd1c3QgMTEsIDIwMTYgOToyOCBBTQ0KQ2M6IHRjcG1AaWV0Zi5v cmcNClN1YmplY3Q6IFJlOiBbdGNwbV0gW1RlY2huaWNhbCBFcnJhdGEgUmVwb3J0ZWRdIFJGQzU5 NjEgKDQ3NzIpDQoNCkkgc3VibWl0dGVkIHRoaXMgZHJhZnQsIHllc3RlcmRheS46DQoNCmh0dHBz Oi8vdG9vbHMuaWV0Zi5vcmcvaHRtbC9kcmFmdC1sdmVsdmluZHJvbi1hY2stdGhyb3R0bGluZy0w Mg0KDQpJdCdzIGEgd29yay1pbi1wcm9ncmVzcywgYnV0IEkgd2VsY29tZSBmZWVkYmFjay4NCg0K DQpPbiBXZWQsIEF1ZyAxMCwgMjAxNiBhdCAxMDozNiBQTSwgUkZDIEVycmF0YSBTeXN0ZW0gPHJm Yy1lZGl0b3JAcmZjLWVkaXRvci5vcmc+IHdyb3RlOg0KPiBUaGUgZm9sbG93aW5nIGVycmF0YSBy ZXBvcnQgaGFzIGJlZW4gc3VibWl0dGVkIGZvciBSRkM1OTYxLCAiSW1wcm92aW5nIA0KPiBUQ1An cyBSb2J1c3RuZXNzIHRvIEJsaW5kIEluLVdpbmRvdyBBdHRhY2tzIi4NCj4NCj4gLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0NCj4gWW91IG1heSByZXZpZXcgdGhlIHJlcG9y dCBiZWxvdyBhbmQgYXQ6DQo+IGh0dHA6Ly93d3cucmZjLWVkaXRvci5vcmcvZXJyYXRhX3NlYXJj aC5waHA/cmZjPTU5NjEmZWlkPTQ3NzINCj4NCj4gLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0NCj4gVHlwZTogVGVjaG5pY2FsDQo+IFJlcG9ydGVkIGJ5OiBTdMOpcGhhbmUg Qm9ydHptZXllciA8Ym9ydHptZXllcitpZXRmQG5pYy5mcj4NCj4NCj4gU2VjdGlvbjogNw0KPg0K PiBPcmlnaW5hbCBUZXh0DQo+IC0tLS0tLS0tLS0tLS0NCj4gW1RoZSBlbnRpcmUgc2VjdGlvbl0N Cj4NCj4gQ29ycmVjdGVkIFRleHQNCj4gLS0tLS0tLS0tLS0tLS0NCj4gTm8gc3VnZ2VzdGVkIHRl eHQgYmVjYXVzZSBpdCByZXF1aXJlcyBhIG11Y2ggbW9yZSBzZXJpb3VzIGFuYWx5c2lzLg0KPiBN YXkgYmUgYWRkaW5nIHRoYXQgdGhlIHJhdGUtbGltaXQgY291bnRlciBTSE9VTEQgYmUgcGVyLWNv bm5lY3Rpb24sIGluIA0KPiB0aGUgc3Bpcml0IG9mIFJGQyA2NTI4Pw0KPg0KPiBOb3Rlcw0KPiAt LS0tLQ0KPiBJdCBhcHBlYXJzIHRoZSBzZWN0aW9uIGRvZXMgbm90IHNwZWNpZnkgdGhhdCB0aGUg Y291bnRlciBmb3IgQUNLIA0KPiB0aHJvdHRsaW5nIFNIT1VMRCBiZSBwZXItY29ubmVjdGlvbi4g SW4gTGludXgsIGl0IGlzIGFwcGFyZW50bHkgDQo+IGdsb2JhbCwgd2hpY2ggYWxsb3dlZCBpdHMg dXNlIGFzIGEgc2lkZSBjaGFubmVsIGVuYWJsaW5nIG5hc3R5IGF0dGFja3MgDQo+IChDVkUtMjAx Ni01Njk2IGFuZCB0aGUgcGFwZXIgIk9mZi1QYXRoIFRDUCBFeHBsb2l0czogR2xvYmFsIFJhdGUg TGltaXQgDQo+IENvbnNpZGVyZWQgRGFuZ2Vyb3VzIiANCj4gPGh0dHA6Ly93d3cuY3MudWNyLmVk dS9+emhpeXVucS9wdWIvc2VjMTZfVENQX3B1cmVfb2ZmcGF0aC5wZGY+KQ0KPg0KPiBJbnN0cnVj dGlvbnM6DQo+IC0tLS0tLS0tLS0tLS0NCj4gVGhpcyBlcnJhdHVtIGlzIGN1cnJlbnRseSBwb3N0 ZWQgYXMgIlJlcG9ydGVkIi4gSWYgbmVjZXNzYXJ5LCBwbGVhc2UgDQo+IHVzZSAiUmVwbHkgQWxs IiB0byBkaXNjdXNzIHdoZXRoZXIgaXQgc2hvdWxkIGJlIHZlcmlmaWVkIG9yIHJlamVjdGVkLiAN Cj4gV2hlbiBhIGRlY2lzaW9uIGlzIHJlYWNoZWQsIHRoZSB2ZXJpZnlpbmcgcGFydHkgKElFU0cp IGNhbiBsb2cgaW4gdG8gDQo+IGNoYW5nZSB0aGUgc3RhdHVzIGFuZCBlZGl0IHRoZSByZXBvcnQs IGlmIG5lY2Vzc2FyeS4NCj4NCj4gLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0NCj4gUkZDNTk2MSAoZHJhZnQtaWV0Zi10Y3BtLXRjcHNlY3VyZS0xMykNCj4gLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0NCj4gVGl0bGUgICAgICAgICAgICAgICA6IElt cHJvdmluZyBUQ1AncyBSb2J1c3RuZXNzIHRvIEJsaW5kIEluLVdpbmRvdyBBdHRhY2tzDQo+IFB1 YmxpY2F0aW9uIERhdGUgICAgOiBBdWd1c3QgMjAxMA0KPiBBdXRob3IocykgICAgICAgICAgIDog QS4gUmFtYWlhaCwgUi4gU3Rld2FydCwgTS4gRGFsYWwNCj4gQ2F0ZWdvcnkgICAgICAgICAgICA6 IFBST1BPU0VEIFNUQU5EQVJEDQo+IFNvdXJjZSAgICAgICAgICAgICAgOiBUQ1AgTWFpbnRlbmFu Y2UgYW5kIE1pbm9yIEV4dGVuc2lvbnMNCj4gQXJlYSAgICAgICAgICAgICAgICA6IFRyYW5zcG9y dA0KPiBTdHJlYW0gICAgICAgICAgICAgIDogSUVURg0KPiBWZXJpZnlpbmcgUGFydHkgICAgIDog SUVTRw0KPg0KPg0KPiBfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fXw0KPiB0Y3BtIG1haWxpbmcgbGlzdA0KPiB0Y3BtQGlldGYub3JnDQo+IGh0dHBzOi8vd3d3 LmlldGYub3JnL21haWxtYW4vbGlzdGluZm8vdGNwbQ0KPg0KDQpfX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fXw0KdGNwbSBtYWlsaW5nIGxpc3QNCnRjcG1AaWV0 Zi5vcmcNCmh0dHBzOi8vd3d3LmlldGYub3JnL21haWxtYW4vbGlzdGluZm8vdGNwbQ0K From nobody Thu Aug 11 12:37:23 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 00D0B12D14A for ; Thu, 11 Aug 2016 12:37:22 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -8.167 X-Spam-Level: X-Spam-Status: No, score=-8.167 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-1.247] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id N-RFi_GPBXj3 for ; Thu, 11 Aug 2016 12:37:20 -0700 (PDT) Received: from boreas.isi.edu (boreas.isi.edu [128.9.160.161]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 6AB6D12D0EC for ; Thu, 11 Aug 2016 12:37:20 -0700 (PDT) Received: from [128.9.184.139] ([128.9.184.139]) (authenticated bits=0) by boreas.isi.edu (8.13.8/8.13.8) with ESMTP id u7BJadwa026094 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Thu, 11 Aug 2016 12:36:39 -0700 (PDT) To: Loganaden Velvindron References: <20160810183654.05358B80C3A@rfc-editor.org> From: Joe Touch Message-ID: <7ac89d58-fc3a-9e12-9d22-0602944f8677@isi.edu> Date: Thu, 11 Aug 2016 12:36:37 -0700 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Archived-At: Cc: tcpm@ietf.org Subject: Re: [tcpm] [Technical Errata Reported] RFC5961 (4772) X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Aug 2016 19:37:22 -0000 FWIW, I see nothing in RFC5961 that implies that ACK throttling - or anything in the doc - should be across connections. This new doc can claim to clarify Sec 7 of that RFC, but I see no reason to deprecate it. Joe On 8/11/2016 9:27 AM, Loganaden Velvindron wrote: > I submitted this draft, yesterday.: > > https://tools.ietf.org/html/draft-lvelvindron-ack-throttling-02 > > It's a work-in-progress, but I welcome feedback. > > > On Wed, Aug 10, 2016 at 10:36 PM, RFC Errata System > wrote: >> The following errata report has been submitted for RFC5961, >> "Improving TCP's Robustness to Blind In-Window Attacks". >> >> -------------------------------------- >> You may review the report below and at: >> http://www.rfc-editor.org/errata_search.php?rfc=5961&eid=4772 >> >> -------------------------------------- >> Type: Technical >> Reported by: Stéphane Bortzmeyer >> >> Section: 7 >> >> Original Text >> ------------- >> [The entire section] >> >> Corrected Text >> -------------- >> No suggested text because it requires a much more serious analysis. >> May be adding that the rate-limit counter SHOULD be per-connection, >> in the spirit of RFC 6528? >> >> Notes >> ----- >> It appears the section does not specify that the counter for ACK throttling SHOULD be per-connection. In Linux, it is apparently global, which allowed its use as a side channel enabling nasty attacks (CVE-2016-5696 and the paper "Off-Path TCP Exploits: Global Rate Limit Considered Dangerous" ) >> >> Instructions: >> ------------- >> This erratum is currently posted as "Reported". If necessary, please >> use "Reply All" to discuss whether it should be verified or >> rejected. When a decision is reached, the verifying party (IESG) >> can log in to change the status and edit the report, if necessary. >> >> -------------------------------------- >> RFC5961 (draft-ietf-tcpm-tcpsecure-13) >> -------------------------------------- >> Title : Improving TCP's Robustness to Blind In-Window Attacks >> Publication Date : August 2010 >> Author(s) : A. Ramaiah, R. Stewart, M. Dalal >> Category : PROPOSED STANDARD >> Source : TCP Maintenance and Minor Extensions >> Area : Transport >> Stream : IETF >> Verifying Party : IESG >> >> >> _______________________________________________ >> tcpm mailing list >> tcpm@ietf.org >> https://www.ietf.org/mailman/listinfo/tcpm >> > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm From nobody Thu Aug 11 12:41:18 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6867512D633 for ; Thu, 11 Aug 2016 12:41:17 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -8.167 X-Spam-Level: X-Spam-Status: No, score=-8.167 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-1.247] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id SP5lMVACu5wC for ; Thu, 11 Aug 2016 12:41:13 -0700 (PDT) Received: from boreas.isi.edu (boreas.isi.edu [128.9.160.161]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 3201612D14A for ; Thu, 11 Aug 2016 12:41:13 -0700 (PDT) Received: from [128.9.184.139] ([128.9.184.139]) (authenticated bits=0) by boreas.isi.edu (8.13.8/8.13.8) with ESMTP id u7BJenWh026639 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Thu, 11 Aug 2016 12:40:49 -0700 (PDT) To: Loganaden Velvindron References: <20160810183654.05358B80C3A@rfc-editor.org> <7ac89d58-fc3a-9e12-9d22-0602944f8677@isi.edu> From: Joe Touch Message-ID: Date: Thu, 11 Aug 2016 12:40:47 -0700 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: <7ac89d58-fc3a-9e12-9d22-0602944f8677@isi.edu> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Archived-At: Cc: tcpm@ietf.org Subject: Re: [tcpm] [Technical Errata Reported] RFC5961 (4772) X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Aug 2016 19:41:17 -0000 FWIW, I don't really see the need for this new doc at all. Everything in 5961 is within the context of a single connection. The misreading of this doc doesn't warrant updating, clarifying, etc. The behavior of Linux should be treated simply as the bug that it is and corrected, but there's no reason to pollute the RFC stream or increase the complexity of TCP requirements on the basis of a bug. Joe On 8/11/2016 12:36 PM, Joe Touch wrote: > FWIW, I see nothing in RFC5961 that implies that ACK throttling - or > anything in the doc - should be across connections. > > This new doc can claim to clarify Sec 7 of that RFC, but I see no reason > to deprecate it. > > Joe > > > On 8/11/2016 9:27 AM, Loganaden Velvindron wrote: >> I submitted this draft, yesterday.: >> >> https://tools.ietf.org/html/draft-lvelvindron-ack-throttling-02 >> >> It's a work-in-progress, but I welcome feedback. >> >> >> On Wed, Aug 10, 2016 at 10:36 PM, RFC Errata System >> wrote: >>> The following errata report has been submitted for RFC5961, >>> "Improving TCP's Robustness to Blind In-Window Attacks". >>> >>> -------------------------------------- >>> You may review the report below and at: >>> http://www.rfc-editor.org/errata_search.php?rfc=5961&eid=4772 >>> >>> -------------------------------------- >>> Type: Technical >>> Reported by: Stéphane Bortzmeyer >>> >>> Section: 7 >>> >>> Original Text >>> ------------- >>> [The entire section] >>> >>> Corrected Text >>> -------------- >>> No suggested text because it requires a much more serious analysis. >>> May be adding that the rate-limit counter SHOULD be per-connection, >>> in the spirit of RFC 6528? >>> >>> Notes >>> ----- >>> It appears the section does not specify that the counter for ACK throttling SHOULD be per-connection. In Linux, it is apparently global, which allowed its use as a side channel enabling nasty attacks (CVE-2016-5696 and the paper "Off-Path TCP Exploits: Global Rate Limit Considered Dangerous" ) >>> >>> Instructions: >>> ------------- >>> This erratum is currently posted as "Reported". If necessary, please >>> use "Reply All" to discuss whether it should be verified or >>> rejected. When a decision is reached, the verifying party (IESG) >>> can log in to change the status and edit the report, if necessary. >>> >>> -------------------------------------- >>> RFC5961 (draft-ietf-tcpm-tcpsecure-13) >>> -------------------------------------- >>> Title : Improving TCP's Robustness to Blind In-Window Attacks >>> Publication Date : August 2010 >>> Author(s) : A. Ramaiah, R. Stewart, M. Dalal >>> Category : PROPOSED STANDARD >>> Source : TCP Maintenance and Minor Extensions >>> Area : Transport >>> Stream : IETF >>> Verifying Party : IESG >>> >>> >>> _______________________________________________ >>> tcpm mailing list >>> tcpm@ietf.org >>> https://www.ietf.org/mailman/listinfo/tcpm >>> >> _______________________________________________ >> tcpm mailing list >> tcpm@ietf.org >> https://www.ietf.org/mailman/listinfo/tcpm From nobody Fri Aug 12 04:30:50 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A63D912D9FE for ; Fri, 12 Aug 2016 04:30:46 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.701 X-Spam-Level: X-Spam-Status: No, score=-2.701 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=unavailable autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=tieto.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id APnmmihg6r3q for ; Fri, 12 Aug 2016 04:30:42 -0700 (PDT) Received: from mail-io0-x22e.google.com (mail-io0-x22e.google.com [IPv6:2607:f8b0:4001:c06::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 9448612D9F3 for ; Fri, 12 Aug 2016 04:30:39 -0700 (PDT) Received: by mail-io0-x22e.google.com with SMTP id 38so21975200iol.0 for ; Fri, 12 Aug 2016 04:30:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tieto.com; s=google; h=from:references:in-reply-to:mime-version:thread-index:date :message-id:subject:to:cc; bh=RsGwwstmgyf79WF7Ns8R2aPaWCg0r9YR+T9wCuyy1GU=; b=TGRuFyh0CrGpcSfIZXp1stAlbUQp1jQ3DtTBw7szcl0RrhTxS1vwcgkdkHWwox7Lle RIqMiC+1/zsAwdVggtYmv0gAV6bljYjCiS1OChka2+5wh173WsY4S14ZplsB3LvL8btQ C17lgTVeNw1GLDMdD3mmV+5cSIbJKs+oJ2gbM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:references:in-reply-to:mime-version :thread-index:date:message-id:subject:to:cc; bh=RsGwwstmgyf79WF7Ns8R2aPaWCg0r9YR+T9wCuyy1GU=; b=dr6vHh1mtaER4qDYypJiZM6DSgP+OSrOV9ZcW5UYAsBaLp2UQEBOnQgOaD3DSmJllH LdEjJuPTPCPFGO8a5HXlgNJQRJ7oeT6p0T3M7zG3C+5pKrkNET+VMIHZMo+d7YhcBZ4P BEaWMUxTlhsfGRw1NoqhpWpZMTUx+Fcl/lmnT9/oxqtBUZthiOCCr/TXAr3RnpMwxxRO KkIk1fD9eh6Fc7dl9+qZWxxAQFQHmGbgXlyoy8Gy860l4Au5ODdQz290eZDzaxkF9ClJ QkBYWg0QEA/mNsJu5uIdT/UhHAjmw9vlnY+AyOlWs/qFf2xREr/ERh0qG4PJfbflNEMw 1i9w== X-Gm-Message-State: AEkooutlZ85YNL2X/hdVfgx70aGagug1I9fDAgEaYp7CW15mqEeTY1DzyjqEOpGvOIG2319zRagYns4Vfn3XBhTz9PieLCpzswDbbso7nz61uLJKvkCK40Ulvz3ouBXu2UrbkZ8= X-Received: by 10.107.37.198 with SMTP id l189mr16958232iol.117.1471001439087; Fri, 12 Aug 2016 04:30:39 -0700 (PDT) From: Karen Elisabeth Egede Nielsen References: <20160708211635.32095.77047.idtracker@ietfa.amsl.com> <2f56d1b8-7450-42de-4a24-53df9cf4c045@it.uc3m.es> In-Reply-To: <2f56d1b8-7450-42de-4a24-53df9cf4c045@it.uc3m.es> MIME-Version: 1.0 X-Mailer: Microsoft Outlook 15.0 Thread-Index: AQGd4OoGFyImcsqDlVk0oY8T3VfV3QH845XxoJ1vbUA= Date: Fri, 12 Aug 2016 13:30:38 +0200 Message-ID: To: marcelo bagnulo braun Content-Type: text/plain; charset=UTF-8 X-DomainID: tieto.com Archived-At: Cc: tcpm IETF list , TCP Prague List Subject: Re: [tcpm] Fwd: I-D Action: draft-bagnulo-tcpm-tcp-low-rtt-00.txt X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Aug 2016 11:30:47 -0000 Hi Marcello, I very much appreciate your draft. Even more so as it addresses one of the things I find to be a bit "disturbing" in the l4s drafts - Namely the claim that the finer saw tooth functionality of l4s/DCTCP alone solves the latency problems (while DCTCP indeed also is tuning the RTO timeout aggressively). E.g., the formulation in section 1.1 of https://datatracker.ietf.org/doc/draft-briscoe-tsvwg-ecn-l4s-id/?include_t ext=1 It turns out that a TCP algorithm like DCTCP that solves TCP's scalability problem also solves the latency problem, because the finer sawteeth cause very little queuing delay. A supporting paper [DCttH15] gives the full explanation of why the design solves both the latency and the scaling problems, both in plain English and in more precise mathematical form. The explanation is summarised without the maths in [I-D.briscoe-aqm-dualq-coupled]. could benefit from a reference to your draft or the issue that you addresses here (at least for comprehensiveness). An additional comment is that new approaches to retransmissions - like TCP TLP and TCP RACK (also SCTP TLR which however is not progressing at the moment) might fundamentally alter the picture. I.e., if retransmissions are sent pro-actively in tail loss situations then more conservative RTOs may be kept for situations where it is prudent to wait longer. Don't know if TCP TLP is so widely deployed that it is something that you should relate to even if it may be superseded by RACK. Just a thought. BR, Karen > -----Original Message----- > From: tcpm [mailto:tcpm-bounces@ietf.org] On Behalf Of marcelo bagnulo > braun > Sent: 8. juli 2016 23:19 > To: tcpm IETF list > Subject: [tcpm] Fwd: I-D Action: draft-bagnulo-tcpm-tcp-low-rtt-00.txt > > Hi, > > We just submitted this draft for consideration of the WG. Comments are > appreciated. > > Regards, marcelo > > > > > -------- Mensaje reenviado -------- > Asunto: I-D Action: draft-bagnulo-tcpm-tcp-low-rtt-00.txt > Fecha: Fri, 08 Jul 2016 14:16:35 -0700 > De: internet-drafts@ietf.org > Responder a: internet-drafts@ietf.org > Para: i-d-announce@ietf.org > > > > A New Internet-Draft is available from the on-line Internet-Drafts directories. > > > Title : Recommendations for increasing TCP performance in low RTT > networks. > Authors : Marcelo Bagnulo > Koen De Schepper > Glenn Judd > Filename : draft-bagnulo-tcpm-tcp-low-rtt-00.txt > Pages : 7 > Date : 2016-07-08 > > Abstract: > This documents compiles a set of issues that negatively affect TCP > performance in low RTT networks as well as the recommendations to > overcome them. > > > The IETF datatracker status page for this draft is: > https://datatracker.ietf.org/doc/draft-bagnulo-tcpm-tcp-low-rtt/ > > There's also a htmlized version available at: > https://tools.ietf.org/html/draft-bagnulo-tcpm-tcp-low-rtt-00 > > > Please note that it may take a couple of minutes from the time of submission > until the htmlized version and diff are available at tools.ietf.org. > > Internet-Drafts are also available by anonymous FTP at: > ftp://ftp.ietf.org/internet-drafts/ > > _______________________________________________ > I-D-Announce mailing list > I-D-Announce@ietf.org > https://www.ietf.org/mailman/listinfo/i-d-announce > Internet-Draft directories: http://www.ietf.org/shadow.html or > ftp://ftp.ietf.org/ietf/1shadow-sites.txt > > > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm From nobody Fri Aug 12 08:48:04 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id EFD0612D738 for ; Thu, 11 Aug 2016 09:24:38 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2 X-Spam-Level: X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id wqFTSvNIKeqH for ; Thu, 11 Aug 2016 09:24:37 -0700 (PDT) Received: from mail-pa0-x229.google.com (mail-pa0-x229.google.com [IPv6:2607:f8b0:400e:c03::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 78CC412D5AD for ; Thu, 11 Aug 2016 09:24:37 -0700 (PDT) Received: by mail-pa0-x229.google.com with SMTP id i5so120342pat.2 for ; Thu, 11 Aug 2016 09:24:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=A88E/EEeccZI/jXUZ2erBJPHQsg95JZtZ/n0wKE4R/0=; b=xsC6ga18lLnmyR4HuAhCpozQXR1h+4nBlbWeuok/RL9B5wXEEyWVJlVCl9ceaSeevl AhMaYpUUP9ZQbMYNvaUZFwwnaFlI5vHyrMQdmUMRfjOGyIOK1movaCjRZ+639P8g3+58 k5Pacd+EFduCUsnZGeGTFsW/8BU1nFHKu9dUkGnzqUAH/46K8SGbuTVIbU4eEEqpiI0m PMr2OieM4jpiqksVtLsCzxQELndTBmEJFQ3KrK2c0umQyqLAoqvYqR12WKxKe3vgJwpa FyT6ke1Htc23n1pxjMaCOA30UoiVqdhsCQwtw+WbBfHwfiPXDydMw3IeY9VvQzWGOfZD 6DCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=A88E/EEeccZI/jXUZ2erBJPHQsg95JZtZ/n0wKE4R/0=; b=GX4JGwPHDTQqTAVeb7MKJWJufGwr+kcw7BLKVkt6fiJ5m68a11uZI0TV1kXKUYmWlZ ajLWs0CIwSyVIyyuc7cQSJzijbqGfgG4Sz5Dx1nuy728/LLxhD20ERBqoR5ChCDzqV3C U91EiEY5C87NJ+yVm5WSHoV+ygtoKkF2Q+HQGo9lvB8z4AUQx72hcJ/X6tqFOBrrK9QF FxfXJYS2naovRn9/9xuGi50BIdvJmAUg2c7UuPSS8SU6tEZQTeicBy+95yKwArH+yOZE XpLcKWkOduu7rnbkJP6fqA5OovVp5J81AKRUXYv7BJtb70+2jCLqaDphcxlzpOuXrQ1T T2ZQ== X-Gm-Message-State: AEkoouvv2wljOJmGCe1NpgOpuwiT9DTA3H/Aw22PuKpiuTCSTy2F8PdH57OwB36VAwom1mE8X/Vf0FFukuwMww== X-Received: by 10.66.54.229 with SMTP id m5mr18549361pap.91.1470932676995; Thu, 11 Aug 2016 09:24:36 -0700 (PDT) MIME-Version: 1.0 Received: by 10.66.142.133 with HTTP; Thu, 11 Aug 2016 09:24:36 -0700 (PDT) In-Reply-To: <20160810183654.05358B80C3A@rfc-editor.org> References: <20160810183654.05358B80C3A@rfc-editor.org> From: Loganaden Velvindron Date: Thu, 11 Aug 2016 20:24:36 +0400 Message-ID: To: RFC Errata System Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Archived-At: X-Mailman-Approved-At: Fri, 12 Aug 2016 08:48:04 -0700 Cc: tcpm@ietf.org, bortzmeyer+ietf@nic.fr, ananth@cisco.com, ietf@kuehlewind.net, randall@lakerest.net, mdalal@cisco.com Subject: Re: [tcpm] [Technical Errata Reported] RFC5961 (4772) X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Aug 2016 16:24:39 -0000 Hi folks, I submitted this, independently: https://tools.ietf.org/html/draft-lvelvindron-ack-throttling-02 On Wed, Aug 10, 2016 at 10:36 PM, RFC Errata System wrote: > The following errata report has been submitted for RFC5961, > "Improving TCP's Robustness to Blind In-Window Attacks". > > -------------------------------------- > You may review the report below and at: > http://www.rfc-editor.org/errata_search.php?rfc=3D5961&eid=3D4772 > > -------------------------------------- > Type: Technical > Reported by: St=C3=A9phane Bortzmeyer > > Section: 7 > > Original Text > ------------- > [The entire section] > > Corrected Text > -------------- > No suggested text because it requires a much more serious analysis. > May be adding that the rate-limit counter SHOULD be per-connection, > in the spirit of RFC 6528? > > Notes > ----- > It appears the section does not specify that the counter for ACK throttli= ng SHOULD be per-connection. In Linux, it is apparently global, which allow= ed its use as a side channel enabling nasty attacks (CVE-2016-5696 and the = paper "Off-Path TCP Exploits: Global Rate Limit Considered Dangerous" ) > > Instructions: > ------------- > This erratum is currently posted as "Reported". If necessary, please > use "Reply All" to discuss whether it should be verified or > rejected. When a decision is reached, the verifying party (IESG) > can log in to change the status and edit the report, if necessary. > > -------------------------------------- > RFC5961 (draft-ietf-tcpm-tcpsecure-13) > -------------------------------------- > Title : Improving TCP's Robustness to Blind In-Window Attac= ks > Publication Date : August 2010 > Author(s) : A. Ramaiah, R. Stewart, M. Dalal > Category : PROPOSED STANDARD > Source : TCP Maintenance and Minor Extensions > Area : Transport > Stream : IETF > Verifying Party : IESG > > > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm > From nobody Fri Aug 12 08:48:38 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 005E612D927 for ; Thu, 11 Aug 2016 13:46:26 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.142 X-Spam-Level: X-Spam-Status: No, score=-1.142 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_PASS=-0.001, SPF_NEUTRAL=0.779] autolearn=no autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=qlgc.onmicrosoft.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id je2nKEwqByQQ for ; Thu, 11 Aug 2016 13:46:24 -0700 (PDT) Received: from NAM02-SN1-obe.outbound.protection.outlook.com (mail-sn1nam02on0113.outbound.protection.outlook.com [104.47.36.113]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id E727712D925 for ; Thu, 11 Aug 2016 13:46:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=qlgc.onmicrosoft.com; s=selector1-qlogic-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=e2iGuoshFisrm8CoF467lTwSg+RUwZQiHYw3x8Y2XtM=; b=AxAPndwfvEWeFSJuj2k1GdbbjSykhqA4Mr8/jKRmskZWBCy9iazEj12g7A4j8SNr0hZWEHWtNAvpC8sdk1Y4lxhH1OxlS41kMCBRbHQeOMSvCEwO8J5L5MgSUgnlqH0H4epr7BSbu3I9bnY6IpIZpfwc3rqKqGP7OcOamHIC6cw= Received: from CY4PR11MB1878.namprd11.prod.outlook.com (10.175.61.140) by CY4PR11MB1878.namprd11.prod.outlook.com (10.175.61.140) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.549.15; Thu, 11 Aug 2016 20:46:21 +0000 Received: from CY4PR11MB1878.namprd11.prod.outlook.com ([10.175.61.140]) by CY4PR11MB1878.namprd11.prod.outlook.com ([10.175.61.140]) with mapi id 15.01.0549.025; Thu, 11 Aug 2016 20:46:21 +0000 From: Kobby Carmona To: David Borman Thread-Topic: [tcpm] Possible deadlock scenario with retransmission on both sides at the same time Thread-Index: AdHth2zqZmjYio0mRrKVRRoD8V9FcgBABbmAAH+o2jAAG1n8AADHNNgA Date: Thu, 11 Aug 2016 20:46:20 +0000 Message-ID: References: <7070553C-65D1-46EE-95F4-DAE82E1F5A5E@weston.borman.com> <2D524A8D-A5CA-45A6-B94D-FA1DA0CEE609@weston.borman.com> In-Reply-To: <2D524A8D-A5CA-45A6-B94D-FA1DA0CEE609@weston.borman.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=kobby.Carmona@qlogic.com; x-originating-ip: [46.19.86.168] x-ms-office365-filtering-correlation-id: 7c5f6f7c-f4d1-4d80-1253-08d3c2288d2e x-microsoft-exchange-diagnostics: 1; CY4PR11MB1878; 20:FPas+AHxTN2mgz/VI9L7JeaFtTA3lIBkklQBEHF1ansGeC3dLXv2Zh7z0klosXDGXwYqk2JeKKBuoL14zwXI1iHuvEJe8O6l7q2d2B+H04IjqsVzWn4AOPaKs59QoJ5zPMiEqs0XF/RgLmb6iCOnwyvBIOwtKmQL9kDl6d4Iek8= x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:CY4PR11MB1878; x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:; x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040174)(601004)(2401047)(5005006)(8121501046)(10201501046)(3002001); SRVR:CY4PR11MB1878; BCL:0; PCL:0; RULEID:; SRVR:CY4PR11MB1878; x-forefront-prvs: 0031A0FFAF x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(6009001)(7916002)(377454003)(40764003)(24454002)(13464003)(199003)(189002)(53754006)(81166006)(81156014)(101416001)(10400500002)(33656002)(15975445007)(586003)(7696003)(8676002)(9686002)(122556002)(3280700002)(76576001)(2900100001)(7736002)(68736007)(7846002)(11100500001)(8936002)(3660700001)(2950100001)(92566002)(86362001)(102836003)(87936001)(2906002)(6116002)(19580405001)(3846002)(5002640100001)(4326007)(305945005)(189998001)(54356999)(99286002)(77096005)(110136002)(93886004)(105586002)(76176999)(97736004)(50986999)(19580395003)(74316002)(106356001)(66066001); DIR:OUT; SFP:1102; SCL:1; SRVR:CY4PR11MB1878; H:CY4PR11MB1878.namprd11.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; received-spf: None (protection.outlook.com: qlogic.com does not designate permitted sender hosts) spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 MIME-Version: 1.0 X-OriginatorOrg: qlogic.com X-MS-Exchange-CrossTenant-originalarrivaltime: 11 Aug 2016 20:46:20.7534 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 0d68a1f9-1490-4d0e-8767-a87dab3ef2ba X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR11MB1878 Archived-At: X-Mailman-Approved-At: Fri, 12 Aug 2016 08:48:38 -0700 Cc: "tcpm@ietf.org" Subject: Re: [tcpm] Possible deadlock scenario with retransmission on both sides at the same time X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Aug 2016 20:46:26 -0000 SGkgRGF2aWQsDQpUaGlzIG1ha2VzIGEgbG90IG9mIHNlbnNlLiBXZSB3aWxsIGZpeCBvdXIgY29k ZS4NClRoYW5rcyBmb3IgeW91ciBoZWxwIG9uIHRoaXMsDQoNCkJUVywNCklzIHRoaXMgaXNzdWUg b2YgbWVudGlvbmVkIGluIGFueSBSRkM/IElmIG5vdCBkbyB5b3Ugc2VlIGEgcG9pbnQgaW4gYWRk aW5nIGV4cGxpY2l0IG5vdGUgb24gdGhlIFNFUSBvZiBwdXJlIEFDSyBpbiBjYXNlIG9mIHJldHJh bnNtaXNzaW9uPw0KDQoJS29iYnkNCg0KLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCkZyb206 IERhdmlkIEJvcm1hbiBbbWFpbHRvOmRhYkB3ZXN0b24uYm9ybWFuLmNvbV0gDQpTZW50OiBNb25k YXksIEF1Z3VzdCAwOCwgMjAxNiAxMjozNSBBTQ0KVG86IEtvYmJ5IENhcm1vbmEgPGtvYmJ5LkNh cm1vbmFAcWxvZ2ljLmNvbT4NCkNjOiB0Y3BtQGlldGYub3JnDQpTdWJqZWN0OiBSZTogW3RjcG1d IFBvc3NpYmxlIGRlYWRsb2NrIHNjZW5hcmlvIHdpdGggcmV0cmFuc21pc3Npb24gb24gYm90aCBz aWRlcyBhdCB0aGUgc2FtZSB0aW1lDQoNCj4gT24gQXVnIDcsIDIwMTYsIGF0IDM6MzcgQU0sIEtv YmJ5IENhcm1vbmEgPGtvYmJ5LkNhcm1vbmFAcWxvZ2ljLmNvbT4gd3JvdGU6DQo+IA0KPiBIaSBE YXZpZCwNCj4gVGhlIGNvZGUgYmVsb3cgaXMgdHJ1ZSBmb3IgZmFzdC1yZXRyYW5zbWl0Lg0KPiBC dXQgaW4gY2FzZSBvZiByZXRyYW5zbWl0IHRpbWVyIGV4cGlyYXRpb24gKFRDUFRfUkVYTVQpIHNu ZF9ueHQgaXMgc2V0IHRvIHNuZF91bmEuIEFuZCBpbiB0aGlzIGNhc2UgQ1FORCB3aWxsIGJlIHNl dCB0byAxTVNTIHNvIGluIHRoZSBleGFtcGxlIGJlbG93IHRoZSB0cmFuc21pdHRlcnMgY2FuIHNl bmQgb25seSBhIHNpbmdsZSBzZWdtZW50IGZyb20gc2VxdWVuY2UgMjAwMC8xMjAwMC4NCg0KWW91 ciB1bmRlcmx5aW5nIHByb2JsZW0gaXMgdGhhdCB0aGUgQUNLLW9ubHkgcGFja2V0cyBhcmUgYmVp bmcgc2VudCB3aXRoIHRoZSB3cm9uZyBzZXF1ZW5jZSBudW1iZXIsIGFuZCB0aGF0IGlzIHdoYXQg aXMgY2F1c2luZyB0aGVtIHRvIGJlIGRyb3BwZWQuICBPbmUgd2F5IHRvIGZpeCB0aGF0IGlzIHRv IHB1dCBTTkQuTlhUIGJhY2sgdG8gaXRzIHByZXZpb3VzIHZhbHVlIGFmdGVyIGRvIHRoZSB0aGUg cmV0cmFuc21pdC4gIEJ1dCB5b3UgYXJlIGNvcnJlY3QsIGluIHRoZSBCU0QgY29kZSBpdCBvbmx5 IGRvZXMgdGhhdCBpbiB0aGUgZmFzdCByZXRyYW5zbWl0IGNvZGU7IHdoZW4gdGhlIHRpbWVyIGJh c2VkIHJldHJhbnNtaXQgY29kZSBmaXJlcywgaXQganVzdCBwdWxscyBiYWNrIFNORC5OWFQgdG8g U05ELlVOQS4gIEhvd2V2ZXIsIGluIHRoZSBCU0QgY29kZSB0aGF0IEnigJltIGxvb2tpbmcgYXQs IGl0IGtlZXBzIHRyYWNrIG9mIHRoZSBsYXJnZXN0IHNlcXVlbmNlIG51bWJlciBzZW50IGluIFNO RC5NQVgsIGFuZCBpbiB0aGUgdGNwX291dHB1dCgpIHBhdGggdGhlcmUgaXMgdGhpcyBiaXQgb2Yg Y29kZTogDQoNCiAgICAgICAvKg0KICAgICAgICAqIElmIHdlIGFyZSBkb2luZyByZXRyYW5zbWlz c2lvbnMsIHRoZW4gc25kX254dCB3aWxsDQogICAgICAgICogbm90IHJlZmxlY3QgdGhlIGZpcnN0 IHVuc2VudCBvY3RldC4gIEZvciBBQ0sgb25seQ0KICAgICAgICAqIHBhY2tldHMsIHdlIGRvIG5v dCB3YW50IHRoZSBzZXF1ZW5jZSBudW1iZXIgb2YgdGhlIA0KICAgICAgICAqIHJldHJhbnNtaXR0 ZWQgcGFja2V0LCB3ZSB3YW50IHRoZSBzZXF1ZW5jZSBudW1iZXINCiAgICAgICAgKiBvZiB0aGUg bmV4dCB1bnNlbnQgb2N0ZXQuICBTbywgaWYgdGhlcmUgaXMgbm8gZGF0YQ0KICAgICAgICAqIChh bmQgbm8gU1lOIG9yIEZJTiksIHVzZSBzbmRfbWF4IGluc3RlYWQgb2Ygc25kX254dA0KICAgICAg ICAqIHdoZW4gZmlsbGluZyBpbiB0aF9zZXEuICBCdXQgaWYgd2UgYXJlIGluIHBlcnNpc3QNCiAg ICAgICAgKiBzdGF0ZSwgc25kX21heCBtaWdodCByZWZsZWN0IG9uZSBieXRlIGJleW9uZCB0aGUN CiAgICAgICAgKiByaWdodCBlZGdlIG9mIHRoZSB3aW5kb3csIHNvIHVzZSBzbmRfbnh0IGluIHRo YXQNCiAgICAgICAgKiBjYXNlLCBzaW5jZSB3ZSBrbm93IHdlIGFyZW4ndCBkb2luZyBhIHJldHJh bnNtaXNzaW9uLg0KICAgICAgICAqIChyZXRyYW5zbWl0IGFuZCBwZXJzaXN0IGFyZSBtdXR1YWxs eSBleGNsdXNpdmUuLi4pDQogICAgICAgICovDQogICAgICAgaWYgKGxlbiB8fCAoZmxhZ3MgJiAo VEhfU1lOfFRIX0ZJTikpIHx8IHRwLT50X3RpbWVyW1RDUFRfUEVSU0lTVF0pDQogICAgICAgICAg ICAgICB0aC0+dGhfc2VxID0gaHRvbmwodHAtPnNuZF9ueHQpOw0KICAgICAgIGVsc2UNCiAgICAg ICAgICAgICAgIHRoLT50aF9zZXEgPSBodG9ubCh0cC0+c25kX21heCk7DQoNClRoYXQgaXMgd2hh dCB5b3VyIGltcGxlbWVudGF0aW9uIGFwcGVhcnMgdG8gYmUgbWlzc2luZywgYW5kIHdoYXQgaXMg Y2F1c2luZyB5b3VyIEFDSyBzdG9ybS4gIFNvIHllcywgc29tZSB2YXJpYW50IG9mIHlvdXIgcHJv YmxlbSBoYXMgYmVlbiBzZWVuIGJlZm9yZSwgYW5kIHRoaXMgaXMgaG93IHRoZSBCU0QgY29kZSBm aXhlZCBpdC4NCg0KCQkJLURhdmlkIEJvcm1hbg0KDQo+IA0KPiAJS29iYnkNCj4gDQo+IC0tLS0t T3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+IEZyb206IERhdmlkIEJvcm1hbiBbbWFpbHRvOmRhYkB3 ZXN0b24uYm9ybWFuLmNvbV0NCj4gU2VudDogVGh1cnNkYXksIEF1Z3VzdCAwNCwgMjAxNiAxMDoz NyBQTQ0KPiBUbzogS29iYnkgQ2FybW9uYSA8a29iYnkuQ2FybW9uYUBxbG9naWMuY29tPg0KPiBD YzogdGNwbUBpZXRmLm9yZw0KPiBTdWJqZWN0OiBSZTogW3RjcG1dIFBvc3NpYmxlIGRlYWRsb2Nr IHNjZW5hcmlvIHdpdGggcmV0cmFuc21pc3Npb24gb24gDQo+IGJvdGggc2lkZXMgYXQgdGhlIHNh bWUgdGltZQ0KPiANCj4gQWZ0ZXIgeW91IHB1bGwgYmFjayBTTkQuTlhUIGFuZCBkbyB0aGUgcmV0 cmFuc21pc3Npb24sIHlvdSBzaG91bGQgdGhlbiByZXN0b3JlIFNORC5OWFQgYmFjayB0byB3aGVy ZSBpdCB3YXMsIG5vdCBsZWF2ZSBpdCBhdCB0aGUgYmFja2VkIG9mZiB2YWx1ZTsgdGhlbiB0aGUg QUNLcyB3b3VsZG7igJl0IGJlIGRyb3BwZWQsIHNpbmNlIHRoZXkgd291bGRu4oCZdCBoYXZlIG9s ZCBzZXEgdmFsdWVzLiAgRm9yIGV4YW1wbGUsIGluIHRoZSA0LjRCU0QgZmFzdCByZXRyYW5zbWl0 IGNvZGUgaXQgaGFkOg0KPiANCj4gCQkJdGNwX3NlcSBvbnh0ID0gdHAtPnNuZF9ueHQ7DQo+IAkJ CS4uLg0KPiAJCQl0cC0+c25kX254dCA9IHRoLT50aF9hY2s7DQo+IAkJCS4uLg0KPiAJCQkodm9p ZCkgdGNwX291dHB1dCh0cCk7DQo+IAkJCS4uLg0KPiAJCQlpZiAoU0VRX0dUKG9ueHQsIHRwLT5z bmRfbnh0KSkgIA0KPiAJCQkJdHAtPnNuZF9ueHQgPSBvbnh0Ow0KPiANCj4gDQo+IAkJCS1EYXZp ZCBCb3JtYW4NCj4gDQo+PiBPbiBBdWcgNCwgMjAxNiwgYXQgNDowOCBBTSwgS29iYnkgQ2FybW9u YSA8a29iYnkuQ2FybW9uYUBxbG9naWMuY29tPiB3cm90ZToNCj4+IA0KPj4gSGkgYWxsLA0KPj4g V2hpbGUgcnVubmluZyBhIGJpZGlyZWN0aW9uYWwgc2NlbmFyaW8gd2l0aCByYW5kb20gZHJvcHMg aW4gYSBuZXR3b3JrIHNpbXVsYXRvciBvZiBvdXIgKFFMb2dpYydzIE5JQykgVENQIHN0YWNrIHdl IGZvdW5kIGEgY2FzZSB3aGVyZSBpdCBzZWVtcyB0aGVyZSBpcyBkZWFkbG9jayBpbiB0aGUgVENQ IHByb3RvY29sICh0aGUgY29ubmVjdGlvbiB3aWxsIGtlZXAgc2VuZGluZyBwdXJlIGFja3MgZnJv bSBib3RoIHNpZGVzIHVudGlsIFJUTyB3aWxsIGV4cGlyZSBtdWx0aXBsZSB0aW1lcyBhbmQgYSBS U1Qgd2lsbCBzZW50IHRvIGNsb3NlIHRoZSBjb25uZWN0aW9uKS4NCj4+IFRoZSBzY2VuYXJpbyBp cyBhcyBmb2xsb3dzICh0aGVyZSBpcyBhbiBleGFtcGxlIHdpdGggbnVtYmVycyBmb3IgZWFjaCBz dGFnZSBhc3N1bWluZyB0aGUgTVNTIGFuZCBlYWNoIHBhY2tldCBpcyAxMDAwQik6DQo+PiAxLiBC b3RoIHNpZGVzIGFyZSB0cmFuc21pdHRpbmcgZGF0YSBhbmQgYSBzaW5nbGUgcGFja2V0IGlzIGRy b3BwZWQgb24gZWl0aGVyIHNpZGUgYW5kIHRoZSBuZXh0IHR3byBwYWNrZXRzIGFyZSByZWNlaXZl ZCBwcm9wZXJseQ0KPj4gCVNpZGUgQSAtIFNORC5NQVg9MzAwMCwgU05ELk5YVD0zMDAwLCBTTkQu VU5BPTEwMDAsIFJDVi5OWFQ9MTEwMDAsIG91dC1vZi1vcmRlciBibG9jayAxMjAwMC0xMzAwMA0K Pj4gCVNpZGUgQiAtIFNORC5NQVggPTEzMDAwLCBTTkQuTlhUID0xMzAwMCwgU05ELlVOQT0xMTAw MCwgDQo+PiBSQ1YuTlhUPTEwMDAsIG91dC1vZi1vcmRlciBibG9jayAyMDAwLTMwMDAgMi4gUlRP IHRpbWVyIGV4cGlyZXMgb24gYm90aCBzaWRlcw0KPj4gCVNpZGUgQSAtIFNORC5NQVg9MzAwMCwg U05ELk5YVD0xMDAwLCBTTkQuVU5BPTEwMDAsIFJDVi5OWFQ9MTEwMDAsIG91dC1vZi1vcmRlciBi bG9jayAxMjAwMC0xMzAwMA0KPj4gCVNpZGUgQiAtIFNORC5NQVggPTEzMDAwLCBTTkQuTlhUPTEx MDAwLCBTTkQuVU5BPTExMDAwLCBSQ1YuTlhUPTEwMDAsIA0KPj4gb3V0LW9mLW9yZGVyIGJsb2Nr IDIwMDAtMzAwMCAzLiBCb3RoIHNpZGVzIHRyYW5zbWl0IGEgc2luZ2xlIHBhY2tldCB0byB0aGUg cGVlcjoNCj4+IAlBLT5CIC0gcGt0LnNlcT0xMDAwLCBwa3QuYWNrPTExMDAwLCBsZW49MTAwMA0K Pj4gCUItPkEgLSBwa3Quc2VxPTExMDAwLCBwa3QuYWNrPTEwMDAsIGxlbj0xMDAwIDMuIEJvdGgg c2lkZXMgcmVjZWl2ZSANCj4+IHRoZSBwYWNrZXRzIGFuZCB1cGRhdGUgdGhlIHJlY2VpdmUgY29u dGV4dDoNCj4+IAlTaWRlIEEgLSBTTkQuTUFYPTMwMDAsIFNORC5OWFQ9MjAwMCwgU05ELlVOQT0x MDAwLCBSQ1YuTlhUPTEzMDAwDQo+PiAJU2lkZSBCIC0gU05ELk1BWD0xMzAwMCwgU05ELk5YVD0x MjAwMCwgU05ELlVOQT0xMTAwMCwgUkNWLk5YVD0zMDAwIDQuIA0KPj4gQm90aCBzaWRlcyBzZW5k IGFub3RoZXIgc2VnbWVudDoNCj4+IAlBLT5CIC0gcGt0LnNlcT0yMDAwLCBwa3QuYWNrPTEzMDAw LCBsZW49MTAwMA0KPj4gCUItPkEgLSBwa3Quc2VxPTEyMDAwLCBwa3QuYWNrPTMwMDAsIGxlbj0x MDAwIDUuIEJvdGggc2lkZXMgZG9uJ3QgDQo+PiBhY2NlcHQgdGhlIHBhY2tldCAoYW5kIGRvbid0 IHVwZGF0ZSBTTkQuVU5BKSBzaW5jZSB0aGUgc2VxdWVuY2Ugb24gdGhlIHBhY2tldCBpcyBsZXNz IHRoYW4gUkNWLk5YVCAoc2VxdWVuY2UgbnVtYmVyIGNoZWNrIGluIHBhZ2UgNjkgb2YgUkZDNzkz KSBhbmQgc2VuZCBhIHB1cmUgQUNLIGluc3RlYWQNCj4+IAlBLT5CIC0gcGt0LnNlcT0yMDAwLCBw a3QuYWNrPTEzMDAwLCBsZW49MCAocHVyZSBBQ0spDQo+PiAJQi0+QSAtIHBrdC5zZXE9MTIwMDAs IHBrdC5hY2s9MzAwMCwgbGVuPTAgKHB1cmUgQUNLKSA2LiBUaGlzIHdpbGwgDQo+PiBjb250aW51 ZSBmb3JldmVyICh1bnRpbCB0aGUgY29ubmVjdGlvbiB3aWxsIGJlIHRlcm1pbmF0ZWQgYnkgUlNU KSBzaW5jZSBldmVyeSBwYWNrZXQgdGhhdCBlbmRzIGJlZm9yZSBSQ1YuTlhUIChldmVuIGEgcmV0 cmFuc21pdCBmcm9tIFNORC5VTkEpIHdpbGwgYmUgZHJvcHBlZC4NCj4+IA0KPj4gRGlkIGFueW9u ZSBlbmNvdW50ZXJlZCB0aGlzIGlzc3VlIGJlZm9yZT8gSXMgdGhlIGFueXRoaW5nIHdlIG1pc3Nl ZCBvbiB0aGlzIHNlcXVlbmNlPw0KPj4gSWYgdGhpcyBpcyBpbmRlZWQgYSByZWFsIGRlYWRsb2Nr LCB0aGVyZSBtaWdodCBiZSBzZXZlcmFsIHNvbHV0aW9ucyB0byB0aGlzIHdoaWNoIHdpbGwgcmVx dWlyZSBhIG1vZGlmaWNhdGlvbiBpbiByZWNlaXZlIHByb2Nlc3Npbmcgb2YgUkZDNzkzLiBCdXQg SSB3b3VsZCBsaWtlIHRvIGtub3cgaWYgeW91IHRoaW5rIHRoaXMgaXMgYSByZWFsIGlzc3VlIGJl Zm9yZSBkZWFsaW5nIHdpdGggc29sdXRpb25zLg0KPj4gVGhhbmtzLA0KPj4gDQo+PiAJS29iYnkN Cj4+IA0KPj4gDQo+PiBfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fXw0KPj4gdGNwbSBtYWlsaW5nIGxpc3QNCj4+IHRjcG1AaWV0Zi5vcmcNCj4+IGh0dHBzOi8v d3d3LmlldGYub3JnL21haWxtYW4vbGlzdGluZm8vdGNwbQ0KPiANCj4gX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18NCj4gdGNwbSBtYWlsaW5nIGxpc3QNCj4g dGNwbUBpZXRmLm9yZw0KPiBodHRwczovL3d3dy5pZXRmLm9yZy9tYWlsbWFuL2xpc3RpbmZvL3Rj cG0NCg0K From nobody Fri Aug 12 09:54:52 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A11C712D859 for ; Fri, 12 Aug 2016 09:54:51 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.948 X-Spam-Level: X-Spam-Status: No, score=-3.948 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RP_MATCHES_RCVD=-1.247, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=google.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HWbmcb-rEdL6 for ; Fri, 12 Aug 2016 09:54:49 -0700 (PDT) Received: from mail-it0-x22a.google.com (mail-it0-x22a.google.com [IPv6:2607:f8b0:4001:c0b::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 95B6312D840 for ; Fri, 12 Aug 2016 09:54:49 -0700 (PDT) Received: by mail-it0-x22a.google.com with SMTP id f6so19007686ith.0 for ; Fri, 12 Aug 2016 09:54:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=HwabPsmXNh7L+dPcdwspJVRbSHdSVgyvCle1hgIU+3U=; b=UBePv6L3TRUtp7DMsFMyuleFAxgipyc7hTueeNMQt04tHFUU+ZwhIScIfwjMbXRNcM IJZi9fSubdTscLDDlhZEWEXC5w6wCkKgwZOuOo6DRmmH8wkuHLAD/BpQYBEevCCbF5NX y5eEp9E2BzdNxnrVw/UE89Fq2H/o4R1861gxYos6XpFlrjUNZEAmUoZx+9+YhkxrwjP7 LhP2qKky1ZJzpy7YdG7X3T0VAAK9N+9F02Nnd0xFLsarPQxUItjqMxHdaUWWkz1uv77q ChNz2bt6k6EA21/bMtjjgPLg4yosxMPblUZHwkKOR6pZXXiPcYvz4rITDRqWbrPMAAe2 YMng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=HwabPsmXNh7L+dPcdwspJVRbSHdSVgyvCle1hgIU+3U=; b=Nxr6/GzgL5RgCBeW1C4Xj2vZ+cqqbLlUaf/eSIEWDV3NiFQta92uBbIjJIKHnRI8kK rPLrrJxhE+fBdZjboDKvHayDerOhGY45eHXoSc4QBsO0x5ndMjTt59LQYtbjF5XfPNfQ dEw1jYuVfDxZnXibo2fUL4AxrYI0wcC3gctJlMX05mxXSS/LnITjCNTHaL4Dhq/wmkqA ZMovcSy2+tTc6fmd5UeKUADDcYiKTYn9tZCLSD0ZwYSV2KAQ5sCRj4KAtXTazdtw6aiI 3ymr4TYhFUwEfquDznC7XVnwxB+VHwqoyZ3KRNpAuMdlejVjGpVyEyk4sHg2tBRJ+nXN Yz6A== X-Gm-Message-State: AEkoouuzA0j9V1ahnDchKaSXXUxUeWVagmNphthhIY1fTkm+zO4aM0STxkFn6mv+T7tcVMUCg1uGIew+aV6BHtuP X-Received: by 10.36.73.195 with SMTP id e64mr4296377itd.80.1471020888444; Fri, 12 Aug 2016 09:54:48 -0700 (PDT) MIME-Version: 1.0 Received: by 10.64.244.103 with HTTP; Fri, 12 Aug 2016 09:54:07 -0700 (PDT) In-Reply-To: References: <20160810183654.05358B80C3A@rfc-editor.org> <7ac89d58-fc3a-9e12-9d22-0602944f8677@isi.edu> From: Yuchung Cheng Date: Fri, 12 Aug 2016 09:54:07 -0700 Message-ID: To: Joe Touch Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Archived-At: Cc: "tcpm@ietf.org Extensions" Subject: Re: [tcpm] [Technical Errata Reported] RFC5961 (4772) X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Aug 2016 16:54:51 -0000 On Thu, Aug 11, 2016 at 12:40 PM, Joe Touch wrote: > FWIW, I don't really see the need for this new doc at all. +1 > > Everything in 5961 is within the context of a single connection. The > misreading of this doc doesn't warrant updating, clarifying, etc. I think an errata on recommending per-connection limit and cite the research finding is worthy. > > The behavior of Linux should be treated simply as the bug that it is and > corrected, but there's no reason to pollute the RFC stream or increase > the complexity of TCP requirements on the basis of a bug. > > Joe > > > On 8/11/2016 12:36 PM, Joe Touch wrote: >> FWIW, I see nothing in RFC5961 that implies that ACK throttling - or >> anything in the doc - should be across connections. >> >> This new doc can claim to clarify Sec 7 of that RFC, but I see no reason >> to deprecate it. >> >> Joe >> >> >> On 8/11/2016 9:27 AM, Loganaden Velvindron wrote: >>> I submitted this draft, yesterday.: >>> >>> https://tools.ietf.org/html/draft-lvelvindron-ack-throttling-02 >>> >>> It's a work-in-progress, but I welcome feedback. >>> >>> >>> On Wed, Aug 10, 2016 at 10:36 PM, RFC Errata System >>> wrote: >>>> The following errata report has been submitted for RFC5961, >>>> "Improving TCP's Robustness to Blind In-Window Attacks". >>>> >>>> -------------------------------------- >>>> You may review the report below and at: >>>> http://www.rfc-editor.org/errata_search.php?rfc=3D5961&eid=3D4772 >>>> >>>> -------------------------------------- >>>> Type: Technical >>>> Reported by: St=C3=A9phane Bortzmeyer >>>> >>>> Section: 7 >>>> >>>> Original Text >>>> ------------- >>>> [The entire section] >>>> >>>> Corrected Text >>>> -------------- >>>> No suggested text because it requires a much more serious analysis. >>>> May be adding that the rate-limit counter SHOULD be per-connection, >>>> in the spirit of RFC 6528? >>>> >>>> Notes >>>> ----- >>>> It appears the section does not specify that the counter for ACK throt= tling SHOULD be per-connection. In Linux, it is apparently global, which al= lowed its use as a side channel enabling nasty attacks (CVE-2016-5696 and t= he paper "Off-Path TCP Exploits: Global Rate Limit Considered Dangerous" ) >>>> >>>> Instructions: >>>> ------------- >>>> This erratum is currently posted as "Reported". If necessary, please >>>> use "Reply All" to discuss whether it should be verified or >>>> rejected. When a decision is reached, the verifying party (IESG) >>>> can log in to change the status and edit the report, if necessary. >>>> >>>> -------------------------------------- >>>> RFC5961 (draft-ietf-tcpm-tcpsecure-13) >>>> -------------------------------------- >>>> Title : Improving TCP's Robustness to Blind In-Window At= tacks >>>> Publication Date : August 2010 >>>> Author(s) : A. Ramaiah, R. Stewart, M. Dalal >>>> Category : PROPOSED STANDARD >>>> Source : TCP Maintenance and Minor Extensions >>>> Area : Transport >>>> Stream : IETF >>>> Verifying Party : IESG >>>> >>>> >>>> _______________________________________________ >>>> tcpm mailing list >>>> tcpm@ietf.org >>>> https://www.ietf.org/mailman/listinfo/tcpm >>>> >>> _______________________________________________ >>> tcpm mailing list >>> tcpm@ietf.org >>> https://www.ietf.org/mailman/listinfo/tcpm > > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm From nobody Fri Aug 12 12:17:35 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id EB9D312D13D for ; Fri, 12 Aug 2016 12:17:33 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.948 X-Spam-Level: X-Spam-Status: No, score=-3.948 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RP_MATCHES_RCVD=-1.247, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=google.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gcCYWNPUh2FO for ; Fri, 12 Aug 2016 12:17:31 -0700 (PDT) Received: from mail-it0-x22c.google.com (mail-it0-x22c.google.com [IPv6:2607:f8b0:4001:c0b::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id CCBF512D60E for ; Fri, 12 Aug 2016 12:17:31 -0700 (PDT) Received: by mail-it0-x22c.google.com with SMTP id u186so19346454ita.0 for ; Fri, 12 Aug 2016 12:17:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=atQvp5QLu9eHcACWMpKPdIBlW1Saxwbelb4UhU5bfE8=; b=DGgOLleQgI8VI5s9iJbu3gEndOf3QlmOLUTkcykYtb64XZb2hhinVA9JmkXaHdKdUz BDCo5ak7aOBLwz3cn5QHk0FrIShgKNUyqOtSxdVdlXCrCvaaW7rDZFMm+2AsxPS4HiLp 5sYC33V582aeqAD+V+IiharOBKAVb7KMu4HFXR61nYGm2QZq5hBqC+bBkFVeXy/8bDR1 DGL8bo6E4K9+DR+ZOq2b0feZaSgAYZf3mPGp5wJ+1pgEE8uxyetZ1TL1OukV8SWZXAIF /fbVmCmxJuVNmhS5lHxkUKF7k27YdpNj5yEKP4BrKfL+Fw3TXVn19m05C3sWudc8VSUq nt3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=atQvp5QLu9eHcACWMpKPdIBlW1Saxwbelb4UhU5bfE8=; b=eamYBdeGJ9pQ0/dp09V0QPV3vpA0TCNFp+Pis8e+8KzaoK3EvMNf9iBVo7VjMB+gK5 3lDIsTnnGxSlL4tbJgIW9p1V//1oMnlDrELCcrYD/pTPL/cpbVwhkVtGuuH38bj9rZEv HVmALw/WWqcyPx5VpKwneQJ2vUQF49Vwk9x4sV+9HoF/oWGNltbMDq7T26z2j37i1n4N VFKf0AW/wfg6MwyxJoTssuauLZt3zWaGa2Xf5s7FHponTXtjKtACgzDjkKgeDtRZ5dDT ymtg6kOz5Vgcbv3ksVzMhq07nzPa2SPVsRyCyeiGZRKCo3+rULjkXxsSdNSJjOhzhHHi tqtg== X-Gm-Message-State: AEkoousWLBZKLNoWokGluAFhcquQqCdrZOb1XCdj8d42BQkZTqKdNuZNXvNYU8s/UAkiWApewllKs4kaijqLE1bL X-Received: by 10.36.150.70 with SMTP id z67mr516444itd.80.1471029450970; Fri, 12 Aug 2016 12:17:30 -0700 (PDT) MIME-Version: 1.0 Received: by 10.64.244.103 with HTTP; Fri, 12 Aug 2016 12:16:50 -0700 (PDT) In-Reply-To: References: <20160708211635.32095.77047.idtracker@ietfa.amsl.com> <2f56d1b8-7450-42de-4a24-53df9cf4c045@it.uc3m.es> From: Yuchung Cheng Date: Fri, 12 Aug 2016 12:16:50 -0700 Message-ID: To: Karen Elisabeth Egede Nielsen Content-Type: text/plain; charset=UTF-8 Archived-At: Cc: tcpm IETF list , TCP Prague List Subject: Re: [tcpm] Fwd: I-D Action: draft-bagnulo-tcpm-tcp-low-rtt-00.txt X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Aug 2016 19:17:34 -0000 On Fri, Aug 12, 2016 at 4:30 AM, Karen Elisabeth Egede Nielsen wrote: > > Hi Marcello, > > I very much appreciate your draft. > > Even more so as it addresses one of the things I find to be a bit > "disturbing" in the l4s drafts - Namely the claim that the finer saw > tooth functionality of l4s/DCTCP alone solves the latency problems (while > DCTCP indeed also is tuning the RTO timeout aggressively). > E.g., the formulation in section 1.1 of > https://datatracker.ietf.org/doc/draft-briscoe-tsvwg-ecn-l4s-id/?include_t > ext=1 > > It turns out that a TCP algorithm like DCTCP that solves TCP's > scalability problem also solves the latency problem, because the > finer sawteeth cause very little queuing delay. A supporting paper > [DCttH15] gives the full explanation of why the design solves both > the latency and the scaling problems, both in plain English and in > more precise mathematical form. The explanation is summarised > without the maths in [I-D.briscoe-aqm-dualq-coupled]. > > could benefit from a reference to your draft or the issue that you > addresses here (at least for comprehensiveness). > > An additional comment is that new approaches to retransmissions - like > TCP TLP and TCP RACK (also SCTP TLR which however is not progressing at > the moment) might fundamentally alter the picture. I.e., if > retransmissions are sent pro-actively in tail loss situations then more > conservative RTOs may be kept for situations where it is prudent to wait > longer. Don't know if TCP TLP is so widely deployed that it is something > that you should relate to even if it may be superseded by RACK. Just a > thought. TLP and RACK help reduce timeout cases in DC environment for sure. But still it can not completely avoid timeouts. So I agree we should keep timeouts conservative, but the current RFCs are way too conservative: min RTO of 1 second is 4-5 orders of magnitude of DC RTTs. The issue is, the "right" min-RTO depends on the actual DC and stacks: they all have different RTTs and timer granularity. The draft cites Glenn's paper, but Morgan Stanley's DC could be different than others. Other comments on the draft: 1. min ssthresh or cwnd: with very large incast ((tens of) thousands of senders into one receiver), cwnd of 0.1 pkt can still be too big. the only way is to pace the packets over N*RTT intervals. 2. delayed acks: as mentioned the actual time depends a lot on the DC implementation, which is really "vender/owner"-specific. instead of perhaps an option during setup to inform the sender about the max delay in the ack works better. > > BR, Karen > > > -----Original Message----- > > From: tcpm [mailto:tcpm-bounces@ietf.org] On Behalf Of marcelo bagnulo > > braun > > Sent: 8. juli 2016 23:19 > > To: tcpm IETF list > > Subject: [tcpm] Fwd: I-D Action: draft-bagnulo-tcpm-tcp-low-rtt-00.txt > > > > Hi, > > > > We just submitted this draft for consideration of the WG. Comments are > > appreciated. > > > > Regards, marcelo > > > > > > > > > > -------- Mensaje reenviado -------- > > Asunto: I-D Action: draft-bagnulo-tcpm-tcp-low-rtt-00.txt > > Fecha: Fri, 08 Jul 2016 14:16:35 -0700 > > De: internet-drafts@ietf.org > > Responder a: internet-drafts@ietf.org > > Para: i-d-announce@ietf.org > > > > > > > > A New Internet-Draft is available from the on-line Internet-Drafts > directories. > > > > > > Title : Recommendations for increasing TCP > performance in low RTT > > networks. > > Authors : Marcelo Bagnulo > > Koen De Schepper > > Glenn Judd > > Filename : draft-bagnulo-tcpm-tcp-low-rtt-00.txt > > Pages : 7 > > Date : 2016-07-08 > > > > Abstract: > > This documents compiles a set of issues that negatively affect TCP > > performance in low RTT networks as well as the recommendations to > > overcome them. > > > > > > The IETF datatracker status page for this draft is: > > https://datatracker.ietf.org/doc/draft-bagnulo-tcpm-tcp-low-rtt/ > > > > There's also a htmlized version available at: > > https://tools.ietf.org/html/draft-bagnulo-tcpm-tcp-low-rtt-00 > > > > > > Please note that it may take a couple of minutes from the time of > submission > > until the htmlized version and diff are available at tools.ietf.org. > > > > Internet-Drafts are also available by anonymous FTP at: > > ftp://ftp.ietf.org/internet-drafts/ > > > > _______________________________________________ > > I-D-Announce mailing list > > I-D-Announce@ietf.org > > https://www.ietf.org/mailman/listinfo/i-d-announce > > Internet-Draft directories: http://www.ietf.org/shadow.html or > > ftp://ftp.ietf.org/ietf/1shadow-sites.txt > > > > > > _______________________________________________ > > tcpm mailing list > > tcpm@ietf.org > > https://www.ietf.org/mailman/listinfo/tcpm From nobody Fri Aug 12 12:17:56 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5077612DA79 for ; Fri, 12 Aug 2016 12:17:52 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.948 X-Spam-Level: X-Spam-Status: No, score=-3.948 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RP_MATCHES_RCVD=-1.247, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=google.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Ga1kv5R1jW8L for ; Fri, 12 Aug 2016 12:17:40 -0700 (PDT) Received: from mail-oi0-x22c.google.com (mail-oi0-x22c.google.com [IPv6:2607:f8b0:4003:c06::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 51ECB12DA78 for ; Fri, 12 Aug 2016 12:17:40 -0700 (PDT) Received: by mail-oi0-x22c.google.com with SMTP id c15so46670078oig.0 for ; Fri, 12 Aug 2016 12:17:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=7+Y4rMQW+oKpVNUsCt7Gze9jJkKeI/rho9PoGY4wTYg=; b=Fe4ulnvHNGgU6gJ7qb4ckosIeNXtmpMQSLZIMt4pW9gcQllRdbSZdiuE/tF+bWP1Zs 9sxNW1kKI6+oEx7WllvVdxAg/ukWNd6TSs+Sfxe9AYxwswzDpEMG2VuIMxgBIxKlADAc uV0TNjF+wXmsW6HAiigAVDQ+OfetFztqXf0RJ+5cyg1mUf4fp5uKvlE/4e1yHG1eEcPN I4+XRP3LBN5KsqiSB3bx4fRhZEoUS6lP6NBEd2q4V1ZBpXIo09hH6l/kCWNKXAfmznFf GXByMPg0wASG6frmUgpgl6A3vfxyLOKYaHf+XdwYwsdnMpeh5zRjBIGlZpAI7ccYq4wZ +WuQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=7+Y4rMQW+oKpVNUsCt7Gze9jJkKeI/rho9PoGY4wTYg=; b=UyIvTtws/ePyzcDrt65PPjC2ZGNz5W1txxHVo5QXAm3BBkObzy/+jAyG9NwpDo39uI rbwkYmIOasQVRFEg99IRe9os2mPm9uIo2aahmyT+gH1AsMj8ZMqyE7E2LxoQiURFWnrX kTwzPxwae8fwzLlNAYKgZ1iluHQNTxlsVh0xtanbWJo7gwElOiGKqiUtRMCOugOLdNYh mPbx60Lv5Av8Zkdoj4r0l21AZYI7pnvQcXRQ63hu8LxmUPVH8XU34WR1Qhd9KP4TmWLt jstLwkgsY6+zordjYul4UYtfcpBWKOSjVrZDwBvIq2QH4IxB8sb42HDQyPRLWrdrgtyq qYmQ== X-Gm-Message-State: AEkooutcgPf/jQbk4z0YczuVJzyHtQAmYJIfTk+ehVS1Y4u6DSTsuAoX+AQh1Cn/O3jgKKaa/s/iIhOKKExYfPxC X-Received: by 10.157.1.49 with SMTP id 46mr5417685otu.99.1471029459510; Fri, 12 Aug 2016 12:17:39 -0700 (PDT) MIME-Version: 1.0 Received: by 10.202.92.194 with HTTP; Fri, 12 Aug 2016 12:17:09 -0700 (PDT) In-Reply-To: References: <20160810183654.05358B80C3A@rfc-editor.org> <7ac89d58-fc3a-9e12-9d22-0602944f8677@isi.edu> From: Neal Cardwell Date: Fri, 12 Aug 2016 12:17:09 -0700 Message-ID: To: Yuchung Cheng Content-Type: text/plain; charset=UTF-8 Archived-At: Cc: "tcpm@ietf.org Extensions" , Joe Touch Subject: Re: [tcpm] [Technical Errata Reported] RFC5961 (4772) X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Aug 2016 19:17:52 -0000 On Fri, Aug 12, 2016 at 9:54 AM, Yuchung Cheng wrote: > I think an errata on recommending per-connection limit and cite the > research finding is worthy. I agree that it's worthwhile to have an errata to clarify that a per-connection limit is best. At the very least there seems some risky ambiguity here. In fact, AFAICT RFC 5961, Section 7, page 13 -- https://tools.ietf.org/html/rfc5961#section-7 -- has several passages that seem to implicitly suggest that the rate-limiting mechanism it specifies as a SHOULD is not per-connection but rather global. First: ... an administrator who is more interested in faster cleanup of stale connections (i.e., concerned about excess TCP state) may decide to set a higher value thus allowing more RST's to be processed in any given time period. If this is a single-connection rate limit, then what practical reason would an administrator have to allow "more RST's to be processed in any given time period"? I can't think of a case where a legit live connection would send more than one RST. If this is about a single connection, then allowing one mid-window RST (and responding to that one with a single challenge ACK) should be enough, in most cases, to send a challenge ACK that elicits a proper RST from the remote endpoint. It's not clear to me what the motivation would be for intentionally sending a challenge ACK in response to more than one mid-window RST in a single connection. Thus AFAICT "allowing more RST's to be processed in any given time period" suggests that the rate limit is across connections. Second: The time limit SHOULD be tunable to help timeout brute force attacks faster than a potential legitimate flood of RSTs. I may be missing something, but if this rate-limiting mechanism is only about a single connection, then what is a "legitimate flood of RSTs"? Why would it be important for a single connection to allow a "legitimate flood of RSTs" destinted for itself? A single legitimate live connection should not send out a flood of RSTs; it should send out one. Once there is one legitimate RST that arrives and is processed, the receiver deletes the connection, and any other arriving RSTs don't match any existing connection, so are ignored and dropped, in which case is no challenge ACK to rate-limit. By contrast, if this rate limit applies across connections then "a potential legitimate flood of RSTs" makes sense, since a group of legitimate live connections that all close in a burst may, in certain cases, generate a legitimate flood of RSTs. So tuning the limit upward would make sense. Sorry if I'm missing something obvious. But it seems like there are at least real ambiguities there, on a point significant enough to be worth clearing up. neal From nobody Fri Aug 12 12:40:47 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6B66A12D853 for ; Fri, 12 Aug 2016 12:40:43 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.601 X-Spam-Level: X-Spam-Status: No, score=-102.601 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, USER_IN_WHITELIST=-100] autolearn=unavailable autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=it-uc3m-es.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FyVkSXwIqK4Z for ; Fri, 12 Aug 2016 12:40:38 -0700 (PDT) Received: from mail-wm0-x235.google.com (mail-wm0-x235.google.com [IPv6:2a00:1450:400c:c09::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 68BF212D11A for ; Fri, 12 Aug 2016 12:40:38 -0700 (PDT) Received: by mail-wm0-x235.google.com with SMTP id f65so44634470wmi.0 for ; Fri, 12 Aug 2016 12:40:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=it-uc3m-es.20150623.gappssmtp.com; s=20150623; h=subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding; bh=p4un+84blmidMyNts2jWdQdqsHoWYebNr4FeqtYg8PQ=; b=ovpKTWA0iWdtTbZhaNyRvDwFLtOf9XJPLSnpgsrK3VZEznsPacQK+oTPerlAwJwxSC N3IDZgP2ZEsQNMMO9DcmWNCf0Wc59BAwA6HhSPZaY2PYckFfT0HMZb86o+oQYgjW+I9G GnLrVx0zHO/nE+KsVG9NdvFPy+fJ/7qCYOASHYAgmSZn1VvnYjIs7ZlKdZAmjFUkf1ba 1lNCWlBNWl1A7MHtqKjcn+KEiXL9YMGN3Ss77s6AFJOCDa93O5HahfFMa0SVMzLdgmK2 ZCatwc2tr94xrMdd2yjK5xHXetdKyIS05RfuMlGN4IpkAAdMDpnSROdXp/F2JfZhRfgp zfkQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=p4un+84blmidMyNts2jWdQdqsHoWYebNr4FeqtYg8PQ=; b=S2+Hr6XNSf47DFwRWGcKd3xWBCdNw71TdFwerUZ3GS9xbjSSj80Z0KYTB0z24EdOL4 9Ic3ByDNYnhdTEtZ1AcGwodcw/i1msXTiqErsRggIfz3QWKmUrRLyf6g81N6DfABUrd+ Xo1SWv1gBLHQY2wVRpmCyRchfpFQ8CIjpQqTJvAioLUIBElI/1ZWvso5K3ZIgxE5UIRA f2n7ynioWsdKkDqjTWGOlFDCKK7ISAOB+bBO/IA9qApHDAhXL02NlDvquFvFkv8rnja4 Z7vK3jqPM+m0hbyB8gegf+8IqHvAOYSzjP8laIx+yYkKX1kcLd5Vf2KlhVO7qpDTESHz PhoQ== X-Gm-Message-State: AEkooutODaqzi/WLFMLr1ufWZBtfcM78b7EWw78Rv3GUg9wUcO2ZkboKNapRpc92aZyXIoJT X-Received: by 10.28.50.199 with SMTP id y190mr489145wmy.61.1471030836637; Fri, 12 Aug 2016 12:40:36 -0700 (PDT) Received: from Macintosh-6.local (av2-84-91-226-53.netvisao.pt. [84.91.226.53]) by smtp.gmail.com with ESMTPSA id ko7sm8800640wjc.48.2016.08.12.12.40.34 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 12 Aug 2016 12:40:35 -0700 (PDT) To: Yuchung Cheng , Karen Elisabeth Egede Nielsen References: <20160708211635.32095.77047.idtracker@ietfa.amsl.com> <2f56d1b8-7450-42de-4a24-53df9cf4c045@it.uc3m.es> From: marcelo bagnulo braun Message-ID: <5e5d0f96-9094-689e-8104-40ba48a6692b@it.uc3m.es> Date: Fri, 12 Aug 2016 21:40:33 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Archived-At: Cc: tcpm IETF list , TCP Prague List Subject: Re: [tcpm] Fwd: I-D Action: draft-bagnulo-tcpm-tcp-low-rtt-00.txt X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Aug 2016 19:40:43 -0000 Karen, Yuchung, Thanks for the comments. Replies below. El 12/08/16 a las 21:16, Yuchung Cheng escribió: > On Fri, Aug 12, 2016 at 4:30 AM, Karen Elisabeth Egede Nielsen > wrote: >> Hi Marcello, >> >> I very much appreciate your draft. >> >> Even more so as it addresses one of the things I find to be a bit >> "disturbing" in the l4s drafts - Namely the claim that the finer saw >> tooth functionality of l4s/DCTCP alone solves the latency problems (while >> DCTCP indeed also is tuning the RTO timeout aggressively). >> E.g., the formulation in section 1.1 of >> https://datatracker.ietf.org/doc/draft-briscoe-tsvwg-ecn-l4s-id/?include_t >> ext=1 >> >> It turns out that a TCP algorithm like DCTCP that solves TCP's >> scalability problem also solves the latency problem, because the >> finer sawteeth cause very little queuing delay. A supporting paper >> [DCttH15] gives the full explanation of why the design solves both >> the latency and the scaling problems, both in plain English and in >> more precise mathematical form. The explanation is summarised >> without the maths in [I-D.briscoe-aqm-dualq-coupled]. >> >> could benefit from a reference to your draft or the issue that you >> addresses here (at least for comprehensiveness). >> >> An additional comment is that new approaches to retransmissions - like >> TCP TLP and TCP RACK (also SCTP TLR which however is not progressing at >> the moment) might fundamentally alter the picture. I.e., if >> retransmissions are sent pro-actively in tail loss situations then more >> conservative RTOs may be kept for situations where it is prudent to wait >> longer. Don't know if TCP TLP is so widely deployed that it is something >> that you should relate to even if it may be superseded by RACK. Just a >> thought. > TLP and RACK help reduce timeout cases in DC environment for sure. But > still it can not completely avoid timeouts. So I agree we should keep > timeouts conservative, but the current RFCs are way too conservative: > min RTO of 1 second is 4-5 orders of magnitude of DC RTTs. > > The issue is, the "right" min-RTO depends on the actual DC and stacks: > they all have different RTTs and timer granularity. The draft cites > Glenn's paper, but Morgan Stanley's DC could be different than others. > Right. I guess we should could say that the min RTO should be in the order of the RTT of the considered network when there is zero queueng delay. But if the draft is aiming to be a reccomendation for imeplenters of DC network TCP stacks, then i guess it should provide a guidance that is independent of the DC network itself, since the implementation should work for a variety of DC networks, no? I guess the more general question is whether we need this document i.e. a document describing reccoemndation about how to set TCP for low RTT networks. If people think it is a good idea to have such a document, we then figure how if this is > Other comments on the draft: > > 1. min ssthresh or cwnd: with very large incast ((tens of) thousands > of senders into one receiver), cwnd of 0.1 pkt can still be too big. the > only way is to pace the packets over N*RTT intervals. > Right, If i understood correctly Bob's paper (cited in the draft) this is exactly what they are proposing. I dont think this document is the right place to define the behaviour of CWND less than 1 MSS. I think this should be done in a separated document, which is then referred by the low-rtt draft. Are you aware of other documents (draft or papers others than the one from Bob) describing mechanisms to achieve this? > 2. delayed acks: > as mentioned the actual time depends a lot on the DC implementation, > which is really "vender/owner"-specific. instead of perhaps an option > during setup to inform the sender about the max delay in the ack works > better. > I am not sure if a follow. What is the benefit of allowing dynamic negotiation of the delay ack time? Thanks, marcelo > >> BR, Karen >> >>> -----Original Message----- >>> From: tcpm [mailto:tcpm-bounces@ietf.org] On Behalf Of marcelo bagnulo >>> braun >>> Sent: 8. juli 2016 23:19 >>> To: tcpm IETF list >>> Subject: [tcpm] Fwd: I-D Action: draft-bagnulo-tcpm-tcp-low-rtt-00.txt >>> >>> Hi, >>> >>> We just submitted this draft for consideration of the WG. Comments are >>> appreciated. >>> >>> Regards, marcelo >>> >>> >>> >>> >>> -------- Mensaje reenviado -------- >>> Asunto: I-D Action: draft-bagnulo-tcpm-tcp-low-rtt-00.txt >>> Fecha: Fri, 08 Jul 2016 14:16:35 -0700 >>> De: internet-drafts@ietf.org >>> Responder a: internet-drafts@ietf.org >>> Para: i-d-announce@ietf.org >>> >>> >>> >>> A New Internet-Draft is available from the on-line Internet-Drafts >> directories. >>> >>> Title : Recommendations for increasing TCP >> performance in low RTT >>> networks. >>> Authors : Marcelo Bagnulo >>> Koen De Schepper >>> Glenn Judd >>> Filename : draft-bagnulo-tcpm-tcp-low-rtt-00.txt >>> Pages : 7 >>> Date : 2016-07-08 >>> >>> Abstract: >>> This documents compiles a set of issues that negatively affect TCP >>> performance in low RTT networks as well as the recommendations to >>> overcome them. >>> >>> >>> The IETF datatracker status page for this draft is: >>> https://datatracker.ietf.org/doc/draft-bagnulo-tcpm-tcp-low-rtt/ >>> >>> There's also a htmlized version available at: >>> https://tools.ietf.org/html/draft-bagnulo-tcpm-tcp-low-rtt-00 >>> >>> >>> Please note that it may take a couple of minutes from the time of >> submission >>> until the htmlized version and diff are available at tools.ietf.org. >>> >>> Internet-Drafts are also available by anonymous FTP at: >>> ftp://ftp.ietf.org/internet-drafts/ >>> >>> _______________________________________________ >>> I-D-Announce mailing list >>> I-D-Announce@ietf.org >>> https://www.ietf.org/mailman/listinfo/i-d-announce >>> Internet-Draft directories: http://www.ietf.org/shadow.html or >>> ftp://ftp.ietf.org/ietf/1shadow-sites.txt >>> >>> >>> _______________________________________________ >>> tcpm mailing list >>> tcpm@ietf.org >>> https://www.ietf.org/mailman/listinfo/tcpm > From nobody Fri Aug 12 12:51:44 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8E80E12DA94 for ; Fri, 12 Aug 2016 12:51:43 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.948 X-Spam-Level: X-Spam-Status: No, score=-3.948 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RP_MATCHES_RCVD=-1.247, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=google.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id s8lG2N5GXDKv for ; Fri, 12 Aug 2016 12:51:41 -0700 (PDT) Received: from mail-io0-x22c.google.com (mail-io0-x22c.google.com [IPv6:2607:f8b0:4001:c06::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1C87612DA93 for ; Fri, 12 Aug 2016 12:51:41 -0700 (PDT) Received: by mail-io0-x22c.google.com with SMTP id b62so33972855iod.3 for ; Fri, 12 Aug 2016 12:51:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=sWnygOL5qe+EkDikO78f4hdLR9aOs/np/u3uoJeefpM=; b=c3XQXGc74jL+1nQDhtxrJFd1OtoOcuUOtRrbPl9PIm+YNIfPVQ3sdnDMvsRo0OGQU8 Er0bSWR7tMusAkPNNnlDVd7vU1eoi7xyS+SEksjpdl28gwaq68/hoMxmh0yuL3/hX86i 9egD+5fLV9YEjeZbWEghZgfI/kmEYv9Z1IuNeuY1xqKIWzjMI8Pupb9MFOq/n76YEPgn 6G6pZ9XAxEykRIaqv9HQqzOrJH/GG5hk1u4sRdnZCKkApQ8p+vvQ2O7dgx3AVdcRVVXC SwgYNREWkOYzTMCg7w8bGTvStZ7JHX4AFc8ZwFftVWeIydDjlAFXvqyuuPhaK3tVQeZJ sCoQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=sWnygOL5qe+EkDikO78f4hdLR9aOs/np/u3uoJeefpM=; b=l/M26Zq9keOb2CLVX7H/f8ik6Eg6YpBuAzfzunxlfmi3Drk4eOAMp33c0WN1vbNUhR Zpqxos5j5vQCShDHkNrbgF1+A0XXKzDwRDld6l457ARxWy5vinaeapoRx58Rvk9G46DC L5EFdOFJCpcAhvjTTskcRB2SLCEftpflVzhaMi1gIuGqSe4KfXW4bba0dp4j+uJKXpP5 HJ3pK/h33TtekJ7FQp9XHZjhrNM7GQ6TSz/HmC3HJPUErRjwBt4X+da3JSG2a/szW1Xs /94a5w79z5JEMT0AEs9+fL4bqkzDoUwHXS6hfaTjpdNlp62r8kI6iW72N/IaWRg/CEer mnEA== X-Gm-Message-State: AEkoouu578iyktu9gFTwFjV2Gtih6dCdOB8XWcy6DydSxCtTpwqPgip0kCUrk1i/bilAqndxK31VOaHJ+FeIjlCj X-Received: by 10.107.201.135 with SMTP id z129mr22528293iof.114.1471031500135; Fri, 12 Aug 2016 12:51:40 -0700 (PDT) MIME-Version: 1.0 Received: by 10.64.244.103 with HTTP; Fri, 12 Aug 2016 12:50:59 -0700 (PDT) In-Reply-To: <5e5d0f96-9094-689e-8104-40ba48a6692b@it.uc3m.es> References: <20160708211635.32095.77047.idtracker@ietfa.amsl.com> <2f56d1b8-7450-42de-4a24-53df9cf4c045@it.uc3m.es> <5e5d0f96-9094-689e-8104-40ba48a6692b@it.uc3m.es> From: Yuchung Cheng Date: Fri, 12 Aug 2016 12:50:59 -0700 Message-ID: To: marcelo bagnulo braun Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Archived-At: Cc: tcpm IETF list , TCP Prague List Subject: Re: [tcpm] Fwd: I-D Action: draft-bagnulo-tcpm-tcp-low-rtt-00.txt X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Aug 2016 19:51:43 -0000 On Fri, Aug 12, 2016 at 12:40 PM, marcelo bagnulo braun wrote: > Karen, Yuchung, > Thanks for the comments. > Replies below. > > > El 12/08/16 a las 21:16, Yuchung Cheng escribi=C3=B3: > >> On Fri, Aug 12, 2016 at 4:30 AM, Karen Elisabeth Egede Nielsen >> wrote: >>> >>> Hi Marcello, >>> >>> I very much appreciate your draft. >>> >>> Even more so as it addresses one of the things I find to be a bit >>> "disturbing" in the l4s drafts - Namely the claim that the finer saw >>> tooth functionality of l4s/DCTCP alone solves the latency problems (whi= le >>> DCTCP indeed also is tuning the RTO timeout aggressively). >>> E.g., the formulation in section 1.1 of >>> >>> https://datatracker.ietf.org/doc/draft-briscoe-tsvwg-ecn-l4s-id/?includ= e_t >>> ext=3D1 >>> >>> It turns out that a TCP algorithm like DCTCP that solves TCP's >>> scalability problem also solves the latency problem, because the >>> finer sawteeth cause very little queuing delay. A supporting paper >>> [DCttH15] gives the full explanation of why the design solves both >>> the latency and the scaling problems, both in plain English and in >>> more precise mathematical form. The explanation is summarised >>> without the maths in [I-D.briscoe-aqm-dualq-coupled]. >>> >>> could benefit from a reference to your draft or the issue that you >>> addresses here (at least for comprehensiveness). >>> >>> An additional comment is that new approaches to retransmissions - like >>> TCP TLP and TCP RACK (also SCTP TLR which however is not progressing at >>> the moment) might fundamentally alter the picture. I.e., if >>> retransmissions are sent pro-actively in tail loss situations then more >>> conservative RTOs may be kept for situations where it is prudent to wai= t >>> longer. Don't know if TCP TLP is so widely deployed that it is somethin= g >>> that you should relate to even if it may be superseded by RACK. Just a >>> thought. >> >> TLP and RACK help reduce timeout cases in DC environment for sure. But >> still it can not completely avoid timeouts. So I agree we should keep >> timeouts conservative, but the current RFCs are way too conservative: >> min RTO of 1 second is 4-5 orders of magnitude of DC RTTs. >> >> The issue is, the "right" min-RTO depends on the actual DC and stacks: >> they all have different RTTs and timer granularity. The draft cites >> Glenn's paper, but Morgan Stanley's DC could be different than others. >> > > Right. > > I guess we should could say that the min RTO should be in the order of th= e > RTT of the considered network when there is zero queueng delay. > But if the draft is aiming to be a reccomendation for imeplenters of DC > network TCP stacks, then i guess it should provide a guidance that is > independent of the DC network itself, since the implementation should wor= k > for a variety of DC networks, no? > > > I guess the more general question is whether we need this document i.e. a > document describing reccoemndation about how to set TCP for low RTT > networks. If people think it is a good idea to have such a document, we t= hen > figure how if this is my conundrum is that DC tcp stuff is usually not an inter-op issue for the Internet so this draft is more of a "how to tune your DC tcp stack". but I can be convinced either ways. > >> Other comments on the draft: >> >> 1. min ssthresh or cwnd: with very large incast ((tens of) thousands >> of senders into one receiver), cwnd of 0.1 pkt can still be too big. the >> only way is to pace the packets over N*RTT intervals. >> > Right, If i understood correctly Bob's paper (cited in the draft) this is > exactly what they are proposing. > I dont think this document is the right place to define the behaviour of > CWND less than 1 MSS. I think this should be done in a separated document= , > which is then referred by the low-rtt draft. > > Are you aware of other documents (draft or papers others than the one fro= m > Bob) describing mechanisms to achieve this? > >> 2. delayed acks: >> as mentioned the actual time depends a lot on the DC implementation, >> which is really "vender/owner"-specific. instead of perhaps an option >> during setup to inform the sender about the max delay in the ack works >> better. >> > I am not sure if a follow. > What is the benefit of allowing dynamic negotiation of the delay ack time= ? > Thanks, marcelo the receiver may decide to delay ack up to certain time. it could be based on DC implementation or host limit. It'd be good for sender to learn about that to adapt his RTO. For example, say it's a cloud VM. VM of OS1 may use a max delay for d-ack say 2ms. but VM of OS2 may use 40ms. The lack of such information makes the sender to these VMs use something conservative of known max of all, otherwise he risks spurious RTOs. > > > > >> >>> BR, Karen >>> >>>> -----Original Message----- >>>> From: tcpm [mailto:tcpm-bounces@ietf.org] On Behalf Of marcelo bagnulo >>>> braun >>>> Sent: 8. juli 2016 23:19 >>>> To: tcpm IETF list >>>> Subject: [tcpm] Fwd: I-D Action: draft-bagnulo-tcpm-tcp-low-rtt-00.txt >>>> >>>> Hi, >>>> >>>> We just submitted this draft for consideration of the WG. Comments are >>>> appreciated. >>>> >>>> Regards, marcelo >>>> >>>> >>>> >>>> >>>> -------- Mensaje reenviado -------- >>>> Asunto: I-D Action: draft-bagnulo-tcpm-tcp-low-rtt-00.txt >>>> Fecha: Fri, 08 Jul 2016 14:16:35 -0700 >>>> De: internet-drafts@ietf.org >>>> Responder a: internet-drafts@ietf.org >>>> Para: i-d-announce@ietf.org >>>> >>>> >>>> >>>> A New Internet-Draft is available from the on-line Internet-Drafts >>> >>> directories. >>>> >>>> >>>> Title : Recommendations for increasing TCP >>> >>> performance in low RTT >>>> >>>> networks. >>>> Authors : Marcelo Bagnulo >>>> Koen De Schepper >>>> Glenn Judd >>>> Filename : draft-bagnulo-tcpm-tcp-low-rtt-00.txt >>>> Pages : 7 >>>> Date : 2016-07-08 >>>> >>>> Abstract: >>>> This documents compiles a set of issues that negatively affect TC= P >>>> performance in low RTT networks as well as the recommendations to >>>> overcome them. >>>> >>>> >>>> The IETF datatracker status page for this draft is: >>>> https://datatracker.ietf.org/doc/draft-bagnulo-tcpm-tcp-low-rtt/ >>>> >>>> There's also a htmlized version available at: >>>> https://tools.ietf.org/html/draft-bagnulo-tcpm-tcp-low-rtt-00 >>>> >>>> >>>> Please note that it may take a couple of minutes from the time of >>> >>> submission >>>> >>>> until the htmlized version and diff are available at tools.ietf.org. >>>> >>>> Internet-Drafts are also available by anonymous FTP at: >>>> ftp://ftp.ietf.org/internet-drafts/ >>>> >>>> _______________________________________________ >>>> I-D-Announce mailing list >>>> I-D-Announce@ietf.org >>>> https://www.ietf.org/mailman/listinfo/i-d-announce >>>> Internet-Draft directories: http://www.ietf.org/shadow.html or >>>> ftp://ftp.ietf.org/ietf/1shadow-sites.txt >>>> >>>> >>>> _______________________________________________ >>>> tcpm mailing list >>>> tcpm@ietf.org >>>> https://www.ietf.org/mailman/listinfo/tcpm >> >> > From nobody Fri Aug 12 13:19:52 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D729712D80E for ; Fri, 12 Aug 2016 13:19:50 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -8.167 X-Spam-Level: X-Spam-Status: No, score=-8.167 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-1.247] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id e5OT0cPwk7ol for ; Fri, 12 Aug 2016 13:19:49 -0700 (PDT) Received: from boreas.isi.edu (boreas.isi.edu [128.9.160.161]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id CB64D12DA94 for ; Fri, 12 Aug 2016 13:19:46 -0700 (PDT) Received: from [128.9.184.139] ([128.9.184.139]) (authenticated bits=0) by boreas.isi.edu (8.13.8/8.13.8) with ESMTP id u7CKJHAx007238 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Fri, 12 Aug 2016 13:19:18 -0700 (PDT) To: Neal Cardwell , Yuchung Cheng References: <20160810183654.05358B80C3A@rfc-editor.org> <7ac89d58-fc3a-9e12-9d22-0602944f8677@isi.edu> From: Joe Touch Message-ID: Date: Fri, 12 Aug 2016 13:19:15 -0700 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Archived-At: Cc: "tcpm@ietf.org Extensions" Subject: Re: [tcpm] [Technical Errata Reported] RFC5961 (4772) X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Aug 2016 20:19:51 -0000 On 8/12/2016 12:17 PM, Neal Cardwell wrote: > On Fri, Aug 12, 2016 at 9:54 AM, Yuchung Cheng wrote: >> I think an errata on recommending per-connection limit and cite the >> research finding is worthy. > I agree that it's worthwhile to have an errata to clarify that a > per-connection limit is best. At the very least there seems > some risky ambiguity here. I disagree, as per below... > > In fact, AFAICT RFC 5961, Section 7, page 13 -- > > https://tools.ietf.org/html/rfc5961#section-7 > > -- has several passages that seem to implicitly suggest that > the rate-limiting mechanism it specifies as a SHOULD is not > per-connection but rather global. There's plenty of benefit to setting a single configuration for an entire system vs. setting per-connection limits. That's completely different from whether the ACKs are tallied globally or per connection. > First: > > ... an > administrator who is more interested in faster cleanup of stale > connections (i.e., concerned about excess TCP state) may decide to > set a higher value thus allowing more RST's to be processed in any > given time period. > > If this is a single-connection rate limit, then what practical reason > would an administrator have to allow "more RST's to be processed in > any given time period"? I can't think of a case where a legit live > connection would send more than one RST. A single connection might process many RSTs because some can be out of window (they'd be dropped) or not at the end of the window (dropped under some configurations). I.e., receiving and evaluating RSTs doesn't always result in resetting the connection. > If this is about a single > connection, then allowing one mid-window RST (and responding to that > one with a single challenge ACK) should be enough, It would be IF the other end issued a legitimate RST. If this is a spoofing attack, the data can still keep coming which means that the window might still be moving forward by the time the next attack RST arrives. As a result, this sort of 'catch up' can go on until the connection ends. The point of the text here is to avoid wasting resources in such cases. > in most cases, to > send a challenge ACK that elicits a proper RST from the remote > endpoint. It's not clear to me what the motivation would be for > intentionally sending a challenge ACK in response to more than one > mid-window RST in a single connection. Protecting against that case is the motivation *of the entire document*. > Thus AFAICT "allowing more RST's to be processed in any given time > period" suggests that the rate limit is across connections. There's no benefit to doing this across connections at all, unless you think that there's some reason that legitimate connections shouldn't be reset by valid RSTs (there's no benefit to that). > Second: > > The time limit SHOULD be tunable to help timeout brute force attacks > faster than a potential legitimate flood of RSTs. > > I may be missing something, but if this rate-limiting mechanism is > only about a single connection, then what is a "legitimate flood of > RSTs"? A sequence of RSTs that pick different values, as described in RFC 4953. Or a sequence of RSTs aimed at responses to the ACK-after-RST that is chasing legitimate data, as described above. > Why would it be important for a single connection to allow a > "legitimate flood of RSTs" destinted for itself? It wouldn't - that's the point of this text. You should treat a flood of RSTs for a single connection as an attack and throttle them accordingly. > A single legitimate > live connection should not send out a flood of RSTs; it should send > out one. Once there is one legitimate RST that arrives and is > processed, the receiver deletes the connection, and any other arriving > RSTs don't match any existing connection, so are ignored and > dropped, in which case is no challenge ACK to rate-limit. Again, the point of this document are attacks, not legitimate connections. > By contrast, if this rate limit applies across connections then "a > potential legitimate flood of RSTs" makes sense, since a group of > legitimate live connections that all close in a burst may, in certain > cases, generate a legitimate flood of RSTs. So tuning the limit > upward would make sense. There's no case I can imagine where a set of attacks on one connection should be cause for concern on another necessarily. Yes, there's no real need to set per-connection limits (i.e., a global per-connection limit seems fine), but counting them as a group serves no purpose against any threat, unless you think that "if I'm attacked on one connection, I will be attacked on another" - but that's an incorrect inference IMO. > > Sorry if I'm missing something obvious. But it seems like there are at > least real ambiguities there, on a point significant enough to be worth > clearing up. I think you're missing the point of the document, but that's just my interpretation of the exchange. Joe From nobody Fri Aug 12 13:35:53 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 44E7612DA96; Fri, 12 Aug 2016 13:35:48 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.022 X-Spam-Level: X-Spam-Status: No, score=-2.022 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=microsoft.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id wWzErIqYppEr; Fri, 12 Aug 2016 13:35:46 -0700 (PDT) Received: from NAM03-DM3-obe.outbound.protection.outlook.com (mail-dm3nam03on0108.outbound.protection.outlook.com [104.47.41.108]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id A966212DAB1; Fri, 12 Aug 2016 13:35:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=9NcYhyERbF57EMrvaxLFccBxCiluk5LSqAfi86+kiRE=; b=EI/oWmSp8tvL6NOeb+mm+AN+dzREdaD8Z0WhkZFmRZxB3kFQ1g0qyg0DO2haSeKIXsleFbS9pIgtvYCghQRAUN0OQAzjIHhaiutOJb05zCIP4NTlwda2FJ6ftod8KOpBG1VyTyH1Pnj36drsnO4LlY3W1OoWAK+6U+eA1uIm2Wc= Received: from BLUPR03MB004.namprd03.prod.outlook.com (10.255.208.38) by BLUPR03MB001.namprd03.prod.outlook.com (10.255.208.35) with Microsoft SMTP Server (version=TLS1_0, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P384) id 15.1.544.10; Fri, 12 Aug 2016 20:35:41 +0000 Received: from BLUPR03MB004.namprd03.prod.outlook.com ([169.254.2.30]) by BLUPR03MB004.namprd03.prod.outlook.com ([169.254.2.30]) with mapi id 15.01.0557.021; Fri, 12 Aug 2016 20:35:40 +0000 From: Praveen Balasubramanian To: Yuchung Cheng , Karen Elisabeth Egede Nielsen Thread-Topic: [tcpm] Fwd: I-D Action: draft-bagnulo-tcpm-tcp-low-rtt-00.txt Thread-Index: AQHR9M4ywbpExZthr0er6u5O8CZMX6BFxyYA Date: Fri, 12 Aug 2016 20:35:40 +0000 Message-ID: References: <20160708211635.32095.77047.idtracker@ietfa.amsl.com> <2f56d1b8-7450-42de-4a24-53df9cf4c045@it.uc3m.es> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=pravb@microsoft.com; x-originating-ip: [2001:4898:80e8:4::584] x-ms-office365-filtering-correlation-id: b584871a-10e7-48c9-56d7-08d3c2f039e5 x-ms-office365-filtering-ht: Tenant x-microsoft-exchange-diagnostics: 1; BLUPR03MB001; 6:hoKvihImVq4lnNHg4LkA81A2grR0WNtg1f3/Zf8ITGEAnTHxXFLfSAe8N5B41WmhGzsV8PK/cN1AHe6hLw0VmibEvufWwgNIvSd7LnZJVungnfI9WVzlZmcBxlP6zpUpW1roufWB6g9Wi2EmpiVfQvmvQmQhMx+a9l/+pilSHrjPsbk+d5uFbHom3rBBkTCnBKL5dQmOuu0psOk4j7e4ANry5XVgdB/ruxQKTBrrz1/76pzamEs3m/5JB+KqCEp8EcObBwyJ0QYZ0JHXB8xITiajqughH2jFs9Kky44TyCSt+GDD0m42S3WbxlHVk+K6qvbPi0COTquDzxJ85AXH8A==; 5:+88adDQbdb4zvuYCR4TdNPFS+81sUp5zMHgh0CPPRu+hJch6YQnJbphIrjByKH3kjt1LpTIdOWigvVdKoIOB4BMONqkbNyfHqJLsY/bhhC1/WcbCuSgdy0wRjIeBV6e0imjgX1Yc4CRHO+l/OuP79/anNK75OimU9Vap2ciqkdw=; 24:vJpqa696B16ZXlB6mRX5WUFcYgmi/POMVU4yoopK/YdLDuTz6k2c63kLCYY1MLOkmNHcsARpk/n5npguYeLpNFUxgR04P3H83JcTn2fg0Zk=; 7:+5aoddz1iIrtyuSN9nRBy/+Xm3eU7++NLiwxHVrf1PP+nKWu064efKmHM7ragx9cltmVRmpHpE1xvKR4ynAxnFkUS9PSllM+EgqR7OGlfphHETNg+QIKPlYsBs3yN1mYh2z0ZulRgHLZRpZOjL5gf0/qE9Il9Vu4Oecn36rhAADVJK49JihQCtkgnAJVdQ7Ktwg+9zbBtBH5TGt/la7OBeUXVwP2CAVFjkNX4O95rGbfBLXcP4x6gVmE6s6BfpUr x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:BLUPR03MB001; x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(158342451672863)(120809045254105); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(61425038)(6040176)(601004)(2401047)(8121501046)(5005006)(3002001)(10201501046)(6055026)(61426038)(61427038); SRVR:BLUPR03MB001; BCL:0; PCL:0; RULEID:(304825118); SRVR:BLUPR03MB001; x-forefront-prvs: 003245E729 x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(6009001)(7916002)(189002)(76094002)(377454003)(377424004)(199003)(2473001)(13464003)(24454002)(68736007)(586003)(97736004)(189998001)(2906002)(8990500004)(5002640100001)(87936001)(93886004)(2900100001)(2950100001)(76576001)(105586002)(5001770100001)(9686002)(6116002)(81156014)(7846002)(81166006)(102836003)(4326007)(8936002)(7736002)(7696003)(8676002)(101416001)(106356001)(106116001)(54356999)(76176999)(86362001)(50986999)(19580395003)(10090500001)(92566002)(19580405001)(305945005)(74316002)(5005710100001)(33656002)(15975445007)(10290500002)(122556002)(99286002)(3660700001)(10400500002)(86612001)(3280700002)(230783001)(3826002); DIR:OUT; SFP:1102; SCL:1; SRVR:BLUPR03MB001; H:BLUPR03MB004.namprd03.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; received-spf: None (protection.outlook.com: microsoft.com does not designate permitted sender hosts) spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: microsoft.com X-MS-Exchange-CrossTenant-originalarrivaltime: 12 Aug 2016 20:35:40.7032 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BLUPR03MB001 Archived-At: Cc: tcpm IETF list , TCP Prague List Subject: Re: [tcpm] Fwd: I-D Action: draft-bagnulo-tcpm-tcp-low-rtt-00.txt X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Aug 2016 20:35:48 -0000 >>2. delayed acks: as mentioned the actual time depends a lot on the DC implementation, which = is really "vender/owner"-specific. instead of perhaps an option during setu= p to inform the sender about the max delay in the ack works better. Yeah this is a real problem in IaaS environment today since TCP sender pick= s minrto and receiver picks delayed ACK timeout independently and can cause= spurious RTOs if the minrto is too aggressive. Windows server tries to use= the connection handshake RTT (< 10 msec) to auto pick the DC values but th= is is a heuristic that can fail and doesn't really work in the Linux intero= p cases.=20 -----Original Message----- From: tcpm [mailto:tcpm-bounces@ietf.org] On Behalf Of Yuchung Cheng Sent: Friday, August 12, 2016 12:17 PM To: Karen Elisabeth Egede Nielsen Cc: tcpm IETF list ; TCP Prague List Subject: Re: [tcpm] Fwd: I-D Action: draft-bagnulo-tcpm-tcp-low-rtt-00.txt On Fri, Aug 12, 2016 at 4:30 AM, Karen Elisabeth Egede Nielsen wrote: > > Hi Marcello, > > I very much appreciate your draft. > > Even more so as it addresses one of the things I find to be a bit=20 > "disturbing" in the l4s drafts - Namely the claim that the finer saw=20 > tooth functionality of l4s/DCTCP alone solves the latency problems=20 > (while DCTCP indeed also is tuning the RTO timeout aggressively). > E.g., the formulation in section 1.1 of=20 > https://datatracker.ietf.org/doc/draft-briscoe-tsvwg-ecn-l4s-id/?inclu > de_t > ext=3D1 > > It turns out that a TCP algorithm like DCTCP that solves TCP's > scalability problem also solves the latency problem, because the > finer sawteeth cause very little queuing delay. A supporting paper > [DCttH15] gives the full explanation of why the design solves both > the latency and the scaling problems, both in plain English and in > more precise mathematical form. The explanation is summarised > without the maths in [I-D.briscoe-aqm-dualq-coupled]. > > could benefit from a reference to your draft or the issue that you=20 > addresses here (at least for comprehensiveness). > > An additional comment is that new approaches to retransmissions -=20 > like TCP TLP and TCP RACK (also SCTP TLR which however is not=20 > progressing at the moment) might fundamentally alter the picture.=20 > I.e., if retransmissions are sent pro-actively in tail loss situations=20 > then more conservative RTOs may be kept for situations where it is=20 > prudent to wait longer. Don't know if TCP TLP is so widely deployed=20 > that it is something that you should relate to even if it may be=20 > superseded by RACK. Just a thought. TLP and RACK help reduce timeout cases in DC environment for sure. But stil= l it can not completely avoid timeouts. So I agree we should keep timeouts = conservative, but the current RFCs are way too conservative: min RTO of 1 second is 4-5 orders of magnitude of DC RTTs. The issue is, the "right" min-RTO depends on the actual DC and stacks: they all have different RTTs and timer granularity. The draft cites Glenn's= paper, but Morgan Stanley's DC could be different than others. Other comments on the draft: 1. min ssthresh or cwnd: with very large incast ((tens of) thousands of sen= ders into one receiver), cwnd of 0.1 pkt can still be too big. the only way= is to pace the packets over N*RTT intervals. 2. delayed acks: as mentioned the actual time depends a lot on the DC implementation, which = is really "vender/owner"-specific. instead of perhaps an option during setu= p to inform the sender about the max delay in the ack works better. > > BR, Karen > > > -----Original Message----- > > From: tcpm [mailto:tcpm-bounces@ietf.org] On Behalf Of marcelo=20 > > bagnulo braun > > Sent: 8. juli 2016 23:19 > > To: tcpm IETF list > > Subject: [tcpm] Fwd: I-D Action:=20 > > draft-bagnulo-tcpm-tcp-low-rtt-00.txt > > > > Hi, > > > > We just submitted this draft for consideration of the WG. Comments=20 > > are appreciated. > > > > Regards, marcelo > > > > > > > > > > -------- Mensaje reenviado -------- > > Asunto: I-D Action: draft-bagnulo-tcpm-tcp-low-rtt-00.txt > > Fecha: Fri, 08 Jul 2016 14:16:35 -0700 > > De: internet-drafts@ietf.org > > Responder a: internet-drafts@ietf.org > > Para: i-d-announce@ietf.org > > > > > > > > A New Internet-Draft is available from the on-line Internet-Drafts > directories. > > > > > > Title : Recommendations for increasing TCP > performance in low RTT > > networks. > > Authors : Marcelo Bagnulo > > Koen De Schepper > > Glenn Judd > > Filename : draft-bagnulo-tcpm-tcp-low-rtt-00.txt > > Pages : 7 > > Date : 2016-07-08 > > > > Abstract: > > This documents compiles a set of issues that negatively affect TCP > > performance in low RTT networks as well as the recommendations to > > overcome them. > > > > > > The IETF datatracker status page for this draft is: > > https://datatracker.ietf.org/doc/draft-bagnulo-tcpm-tcp-low-rtt/ > > > > There's also a htmlized version available at: > > https://tools.ietf.org/html/draft-bagnulo-tcpm-tcp-low-rtt-00 > > > > > > Please note that it may take a couple of minutes from the time of > submission > > until the htmlized version and diff are available at tools.ietf.org. > > > > Internet-Drafts are also available by anonymous FTP at: > > ftp://ftp.ietf.org/internet-drafts/ > > > > _______________________________________________ > > I-D-Announce mailing list > > I-D-Announce@ietf.org > > https://www.ietf.org/mailman/listinfo/i-d-announce > > Internet-Draft directories: http://www.ietf.org/shadow.html or=20 > > ftp://ftp.ietf.org/ietf/1shadow-sites.txt > > > > > > _______________________________________________ > > tcpm mailing list > > tcpm@ietf.org > > https://www.ietf.org/mailman/listinfo/tcpm _______________________________________________ tcpm mailing list tcpm@ietf.org https://www.ietf.org/mailman/listinfo/tcpm From nobody Sat Aug 13 11:47:47 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 57A5312B025 for ; Sat, 13 Aug 2016 11:47:46 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -8.167 X-Spam-Level: X-Spam-Status: No, score=-8.167 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-1.247] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rO_F7cHJDSVF for ; Sat, 13 Aug 2016 11:47:45 -0700 (PDT) Received: from boreas.isi.edu (boreas.isi.edu [128.9.160.161]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 4120212B005 for ; Sat, 13 Aug 2016 11:47:45 -0700 (PDT) Received: from [192.168.1.189] (cpe-172-250-251-17.socal.res.rr.com [172.250.251.17]) (authenticated bits=0) by boreas.isi.edu (8.13.8/8.13.8) with ESMTP id u7DIlBKc026690 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Sat, 13 Aug 2016 11:47:12 -0700 (PDT) To: Yoav Nir References: <6E459CA7-767F-43DE-A128-C50E50025B99@gmail.com> <9BD95DAD-5278-4252-B179-7EE5F82CA038@gmail.com> From: Joe Touch Message-ID: <57516f46-bb25-73af-615a-b058bd44270c@isi.edu> Date: Sat, 13 Aug 2016 11:47:08 -0700 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: <9BD95DAD-5278-4252-B179-7EE5F82CA038@gmail.com> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Archived-At: Cc: "tcpm@ietf.org" Subject: Re: [tcpm] [saag] Linux flaw from RFC5961 implementation X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Aug 2016 18:47:46 -0000 Hi, all, Moving this thread back to TCPM where it is ongong... I'll address the note below. Joe On 8/13/2016 10:45 AM, Yoav Nir wrote: >> On 13 Aug 2016, at 6:35 PM, Joe Touch wrote: >> >> Hi, all, >> >> This issue is already been discussed in TCPM. Please review the existing >> thread and continue discussion there. >> >> A quick summary from my viewpoint: >> >> a) there already is a draft >> >> b) IMO, the Linux code is just (and yet another) bug based on an >> incorrect read of RFC 5961, not worthy of even an errata >> >> All of this is in the thread on TCPM. Please discuss it there. > Hi, Joe > > I’ve read the thread, but I disagree with your conclusion there ([1]) for several reasons. > > First, RFCs should be clear enough that reasonably competent implementers will get it right. I think it’s safe to say that Linux kernel developers are reasonably competent as a whole, so if they got it wrong the RFC is not clear enough and requires a clarification in one form or another. That is demonstrably false on many levels. My students just found a two significant bugs in the Linux TCP code just this summer. Then again, RFC5961 exists because Linux was all to happy to take over-eager "security" fixes that could impact normal TCP operation. > The fact that Microsoft got it right is not proof that any competent developer will get it right. And the fact that Linux got it wrong is not proof that others will as well. > Second, section 7 talks about rate-limiting to prevent DoS. It talks about protecting bandwidth and CPU utilization. It states that "An implementation may take an excessive number of invocations of the throttling mechanism as an indication that network conditions are unusual or hostile.” All of this is global to the system, not specific to one connection or another. As far as bandwidth is concerned, there is no difference between sending 5 Challenge Acks on each of 20 connections, or 100 Challenge Acks on a single connection. The entire doc is written in the context of a single connection except where it indicates otherwise (per-machine values for per-connection limits). Anyone who thinks that an attack on one TCP connection implies an attack on another is already incorrect. The document is already written to take that into account. > Third, if a machine has enough CPU and bandwidth to allow X challenge acks per second, then dividing X by the number of concurrent connections sets a per-connection limit is too low for the mechanism to function well. If the CPU and bandwidth are that limited, then TCP won't function well either. > Of course, now that we know about the attack it seems obvious that the limit should be per-connection and that we shouldn’t let one connection starve other connections for challenge-acks. But it’s not so obvious that the RFC did not need to mention it. The RFC doesn't mention not to put your credit card information in the header too - hopefully we don't need to explain everything you shouldn't do. Joe From nobody Sun Aug 14 07:54:35 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4C41412D1D5 for ; Sun, 14 Aug 2016 07:54:33 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.902 X-Spam-Level: X-Spam-Status: No, score=-6.902 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id S_A3aHmZcmKA for ; Sun, 14 Aug 2016 07:54:30 -0700 (PDT) Received: from smtp-fr.alcatel-lucent.com (fr-hpida-esg-02.alcatel-lucent.com [135.245.210.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id BC41512D1A9 for ; Sun, 14 Aug 2016 07:54:30 -0700 (PDT) Received: from fr712umx4.dmz.alcatel-lucent.com (unknown [135.245.210.45]) by Websense Email Security Gateway with ESMTPS id 3053E393D4541; Sun, 14 Aug 2016 14:54:25 +0000 (GMT) Received: from fr712usmtp2.zeu.alcatel-lucent.com (fr712usmtp2.zeu.alcatel-lucent.com [135.239.2.42]) by fr712umx4.dmz.alcatel-lucent.com (GMO-o) with ESMTP id u7EEsRtB010895 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Sun, 14 Aug 2016 14:54:28 GMT Received: from FR711WXCHHUB02.zeu.alcatel-lucent.com (fr711wxchhub02.zeu.alcatel-lucent.com [135.239.2.112]) by fr712usmtp2.zeu.alcatel-lucent.com (GMO) with ESMTP id u7EEsO6B012708 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Sun, 14 Aug 2016 16:54:25 +0200 Received: from FR712WXCHMBA15.zeu.alcatel-lucent.com ([169.254.7.108]) by FR711WXCHHUB02.zeu.alcatel-lucent.com ([135.239.2.112]) with mapi id 14.03.0195.001; Sun, 14 Aug 2016 16:54:24 +0200 From: "Scharf, Michael (Nokia - DE)" To: Joe Touch , Yoav Nir Thread-Topic: [tcpm] [saag] Linux flaw from RFC5961 implementation Thread-Index: AQHR9ZNHhdnleqm4ZEijW5ctFaZqFqBIhd9Q Date: Sun, 14 Aug 2016 14:54:23 +0000 Message-ID: <655C07320163294895BBADA28372AF5D489D23F5@FR712WXCHMBA15.zeu.alcatel-lucent.com> References: <6E459CA7-767F-43DE-A128-C50E50025B99@gmail.com> <9BD95DAD-5278-4252-B179-7EE5F82CA038@gmail.com> <57516f46-bb25-73af-615a-b058bd44270c@isi.edu> In-Reply-To: <57516f46-bb25-73af-615a-b058bd44270c@isi.edu> Accept-Language: de-DE, en-US Content-Language: de-DE X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.239.27.38] Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 MIME-Version: 1.0 Archived-At: Cc: "tcpm@ietf.org" Subject: Re: [tcpm] [saag] Linux flaw from RFC5961 implementation X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Aug 2016 14:54:33 -0000 QWNjb3JkaW5nIHRvIGh0dHBzOi8vd3d3LmlldGYub3JnL2llc2cvc3RhdGVtZW50L2VycmF0YS1w cm9jZXNzaW5nLmh0bWwsIHRoZSBjb25kaXRpb24gZm9yICJ2ZXJpZmllZCIgZXJyYXRhIGlzOg0K DQogICJPbmx5IGVycm9ycyB0aGF0IGNvdWxkIGNhdXNlIGltcGxlbWVudGF0aW9uIG9yIGRlcGxv eW1lbnQgcHJvYmxlbXMgb3Igc2lnbmlmaWNhbnQgY29uZnVzaW9uIHNob3VsZCBiZSBWZXJpZmll ZC4iDQoNCkFzIGFuIGluZGl2aWR1YWwgY29udHJpYnV0b3IgdG8gVENQTSwgaW4gdGhpcyBjYXNl IEkgYmVsaWV2ZSB0aGVyZSBpcyBlbXBpcmljYWwgZXZpZGVuY2UgdGhhdCB0aGUgd29yZGluZyBv ZiBSRkMgNTk2MSBjYW4gY2F1c2UgInNpZ25pZmljYW50IGNvbmZ1c2lvbiIuIFRvIG1lLCB0aGUg Y29uZGl0aW9ucyBmb3IgYSAidmVyaWZpZWQiIGVycmF0YSBjYW4gdGh1cyBiZSBmdWxmaWxsZWQg aW4gdGhpcyBjYXNlLiBCVFcsIHRoZSBvdGhlciB0d28gcG90ZW50aWFsIHN0YXRlcyBmb3IgZXJy YXRhcyBhcmUgImhvbGQgZm9yIGRvY3VtZW50IHVwZGF0ZSIgb3IgInJlamVjdGVkIi4NCg0KQWxz bywgSSBkb24ndCBiZWxpZXZlIHRoYXQgYW4gZXJyYXRhIHdpdGggc29tZSBhZGRpdGlvbmFsIGV4 cGxhbmF0aW9uIG9yIGltcGxlbWVudGF0aW9uIGd1aWRhbmNlIHdvdWxkIGNhdXNlIGhhcm0gdG8g dGhlIFJGQyBzZXJpZXMuDQoNCk1pY2hhZWwNCg0KDQo+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0t LS0tDQo+IEZyb206IHRjcG0gW21haWx0bzp0Y3BtLWJvdW5jZXNAaWV0Zi5vcmddIE9uIEJlaGFs ZiBPZiBKb2UgVG91Y2gNCj4gU2VudDogU2F0dXJkYXksIEF1Z3VzdCAxMywgMjAxNiA4OjQ3IFBN DQo+IFRvOiBZb2F2IE5pcg0KPiBDYzogdGNwbUBpZXRmLm9yZw0KPiBTdWJqZWN0OiBSZTogW3Rj cG1dIFtzYWFnXSBMaW51eCBmbGF3IGZyb20gUkZDNTk2MSBpbXBsZW1lbnRhdGlvbg0KPiANCj4g SGksIGFsbCwNCj4gDQo+IE1vdmluZyB0aGlzIHRocmVhZCBiYWNrIHRvIFRDUE0gd2hlcmUgaXQg aXMgb25nb25nLi4uDQo+IA0KPiBJJ2xsIGFkZHJlc3MgdGhlIG5vdGUgYmVsb3cuDQo+IA0KPiBK b2UNCj4gDQo+IA0KPiBPbiA4LzEzLzIwMTYgMTA6NDUgQU0sIFlvYXYgTmlyIHdyb3RlOg0KPiA+ PiBPbiAxMyBBdWcgMjAxNiwgYXQgNjozNSBQTSwgSm9lIFRvdWNoIDx0b3VjaEBpc2kuZWR1PiB3 cm90ZToNCj4gPj4NCj4gPj4gSGksIGFsbCwNCj4gPj4NCj4gPj4gVGhpcyBpc3N1ZSBpcyBhbHJl YWR5IGJlZW4gZGlzY3Vzc2VkIGluIFRDUE0uIFBsZWFzZSByZXZpZXcgdGhlDQo+IGV4aXN0aW5n DQo+ID4+IHRocmVhZCBhbmQgY29udGludWUgZGlzY3Vzc2lvbiB0aGVyZS4NCj4gPj4NCj4gPj4g QSBxdWljayBzdW1tYXJ5IGZyb20gbXkgdmlld3BvaW50Og0KPiA+Pg0KPiA+PiBhKSB0aGVyZSBh bHJlYWR5IGlzIGEgZHJhZnQNCj4gPj4NCj4gPj4gYikgSU1PLCB0aGUgTGludXggY29kZSBpcyBq dXN0IChhbmQgeWV0IGFub3RoZXIpIGJ1ZyBiYXNlZCBvbiBhbg0KPiA+PiBpbmNvcnJlY3QgcmVh ZCBvZiBSRkMgNTk2MSwgbm90IHdvcnRoeSBvZiBldmVuIGFuIGVycmF0YQ0KPiA+Pg0KPiA+PiBB bGwgb2YgdGhpcyBpcyBpbiB0aGUgdGhyZWFkIG9uIFRDUE0uIFBsZWFzZSBkaXNjdXNzIGl0IHRo ZXJlLg0KPiA+IEhpLCBKb2UNCj4gPg0KPiA+IEnigJl2ZSByZWFkIHRoZSB0aHJlYWQsIGJ1dCBJ IGRpc2FncmVlIHdpdGggeW91ciBjb25jbHVzaW9uIHRoZXJlIChbMV0pDQo+IGZvciBzZXZlcmFs IHJlYXNvbnMuDQo+ID4NCj4gPiBGaXJzdCwgUkZDcyBzaG91bGQgYmUgY2xlYXIgZW5vdWdoIHRo YXQgcmVhc29uYWJseSBjb21wZXRlbnQNCj4gaW1wbGVtZW50ZXJzIHdpbGwgZ2V0IGl0IHJpZ2h0 LiBJIHRoaW5rIGl04oCZcyBzYWZlIHRvIHNheSB0aGF0IExpbnV4DQo+IGtlcm5lbCBkZXZlbG9w ZXJzIGFyZSByZWFzb25hYmx5IGNvbXBldGVudCBhcyBhIHdob2xlLCBzbyBpZiB0aGV5IGdvdA0K PiBpdCB3cm9uZyB0aGUgUkZDIGlzIG5vdCBjbGVhciBlbm91Z2ggYW5kIHJlcXVpcmVzIGEgY2xh cmlmaWNhdGlvbiBpbg0KPiBvbmUgZm9ybSBvciBhbm90aGVyLg0KPiBUaGF0IGlzIGRlbW9uc3Ry YWJseSBmYWxzZSBvbiBtYW55IGxldmVscy4gTXkgc3R1ZGVudHMganVzdCBmb3VuZCBhIHR3bw0K PiBzaWduaWZpY2FudCBidWdzIGluIHRoZSBMaW51eCBUQ1AgY29kZSBqdXN0IHRoaXMgc3VtbWVy Lg0KPiANCj4gVGhlbiBhZ2FpbiwgUkZDNTk2MSBleGlzdHMgYmVjYXVzZSBMaW51eCB3YXMgYWxs IHRvIGhhcHB5IHRvIHRha2UNCj4gb3Zlci1lYWdlciAic2VjdXJpdHkiIGZpeGVzIHRoYXQgY291 bGQgaW1wYWN0IG5vcm1hbCBUQ1Agb3BlcmF0aW9uLg0KPiANCj4gPiBUaGUgZmFjdCB0aGF0IE1p Y3Jvc29mdCBnb3QgaXQgcmlnaHQgaXMgbm90IHByb29mIHRoYXQgYW55IGNvbXBldGVudA0KPiBk ZXZlbG9wZXIgd2lsbCBnZXQgaXQgcmlnaHQuDQo+IEFuZCB0aGUgZmFjdCB0aGF0IExpbnV4IGdv dCBpdCB3cm9uZyBpcyBub3QgcHJvb2YgdGhhdCBvdGhlcnMgd2lsbCBhcw0KPiB3ZWxsLg0KPiAN Cj4gPiBTZWNvbmQsIHNlY3Rpb24gNyB0YWxrcyBhYm91dCByYXRlLWxpbWl0aW5nIHRvIHByZXZl bnQgRG9TLiBJdCB0YWxrcw0KPiBhYm91dCBwcm90ZWN0aW5nIGJhbmR3aWR0aCBhbmQgQ1BVIHV0 aWxpemF0aW9uLiBJdCBzdGF0ZXMgdGhhdCAiQW4NCj4gaW1wbGVtZW50YXRpb24gbWF5IHRha2Ug YW4gZXhjZXNzaXZlIG51bWJlciBvZiBpbnZvY2F0aW9ucyBvZiB0aGUNCj4gdGhyb3R0bGluZyBt ZWNoYW5pc20gYXMgYW4gaW5kaWNhdGlvbiB0aGF0IG5ldHdvcmsgY29uZGl0aW9ucyBhcmUNCj4g dW51c3VhbCBvciBob3N0aWxlLuKAnSBBbGwgb2YgdGhpcyBpcyBnbG9iYWwgdG8gdGhlIHN5c3Rl bSwgbm90IHNwZWNpZmljDQo+IHRvIG9uZSBjb25uZWN0aW9uIG9yIGFub3RoZXIuIEFzIGZhciBh cyBiYW5kd2lkdGggaXMgY29uY2VybmVkLCB0aGVyZQ0KPiBpcyBubyBkaWZmZXJlbmNlIGJldHdl ZW4gc2VuZGluZyA1IENoYWxsZW5nZSBBY2tzIG9uIGVhY2ggb2YgMjANCj4gY29ubmVjdGlvbnMs IG9yIDEwMCBDaGFsbGVuZ2UgQWNrcyBvbiBhIHNpbmdsZSBjb25uZWN0aW9uLg0KPiBUaGUgZW50 aXJlIGRvYyBpcyB3cml0dGVuIGluIHRoZSBjb250ZXh0IG9mIGEgc2luZ2xlIGNvbm5lY3Rpb24g ZXhjZXB0DQo+IHdoZXJlIGl0IGluZGljYXRlcyBvdGhlcndpc2UgKHBlci1tYWNoaW5lIHZhbHVl cyBmb3IgcGVyLWNvbm5lY3Rpb24NCj4gbGltaXRzKS4gQW55b25lIHdobyB0aGlua3MgdGhhdCBh biBhdHRhY2sgb24gb25lIFRDUCBjb25uZWN0aW9uIGltcGxpZXMNCj4gYW4gYXR0YWNrIG9uIGFu b3RoZXIgaXMgYWxyZWFkeSBpbmNvcnJlY3QuIFRoZSBkb2N1bWVudCBpcyBhbHJlYWR5DQo+IHdy aXR0ZW4gdG8gdGFrZSB0aGF0IGludG8gYWNjb3VudC4NCj4gDQo+ID4gVGhpcmQsIGlmIGEgbWFj aGluZSBoYXMgZW5vdWdoIENQVSBhbmQgYmFuZHdpZHRoIHRvIGFsbG93IFggY2hhbGxlbmdlDQo+ IGFja3MgcGVyIHNlY29uZCwgdGhlbiBkaXZpZGluZyBYIGJ5IHRoZSBudW1iZXIgb2YgY29uY3Vy cmVudA0KPiBjb25uZWN0aW9ucyBzZXRzIGEgcGVyLWNvbm5lY3Rpb24gbGltaXQgaXMgdG9vIGxv dyBmb3IgdGhlIG1lY2hhbmlzbSB0bw0KPiBmdW5jdGlvbiB3ZWxsLg0KPiBJZiB0aGUgQ1BVIGFu ZCBiYW5kd2lkdGggYXJlIHRoYXQgbGltaXRlZCwgdGhlbiBUQ1Agd29uJ3QgZnVuY3Rpb24gd2Vs bA0KPiBlaXRoZXIuDQo+IA0KPiA+IE9mIGNvdXJzZSwgbm93IHRoYXQgd2Uga25vdyBhYm91dCB0 aGUgYXR0YWNrIGl0IHNlZW1zIG9idmlvdXMgdGhhdA0KPiB0aGUgbGltaXQgc2hvdWxkIGJlIHBl ci1jb25uZWN0aW9uIGFuZCB0aGF0IHdlIHNob3VsZG7igJl0IGxldCBvbmUNCj4gY29ubmVjdGlv biBzdGFydmUgb3RoZXIgY29ubmVjdGlvbnMgZm9yIGNoYWxsZW5nZS1hY2tzLiBCdXQgaXTigJlz IG5vdCBzbw0KPiBvYnZpb3VzIHRoYXQgdGhlIFJGQyBkaWQgbm90IG5lZWQgdG8gbWVudGlvbiBp dC4NCj4gDQo+IFRoZSBSRkMgZG9lc24ndCBtZW50aW9uIG5vdCB0byBwdXQgeW91ciBjcmVkaXQg Y2FyZCBpbmZvcm1hdGlvbiBpbiB0aGUNCj4gaGVhZGVyIHRvbyAtIGhvcGVmdWxseSB3ZSBkb24n dCBuZWVkIHRvIGV4cGxhaW4gZXZlcnl0aGluZyB5b3UNCj4gc2hvdWxkbid0IGRvLg0KPiANCj4g Sm9lDQo+IA0KPiBfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f Xw0KPiB0Y3BtIG1haWxpbmcgbGlzdA0KPiB0Y3BtQGlldGYub3JnDQo+IGh0dHBzOi8vd3d3Lmll dGYub3JnL21haWxtYW4vbGlzdGluZm8vdGNwbQ0K From nobody Mon Aug 15 02:00:41 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1756512B051 for ; Mon, 15 Aug 2016 02:00:41 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.701 X-Spam-Level: X-Spam-Status: No, score=-2.701 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=tieto.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zBqoftd0RdkV for ; Mon, 15 Aug 2016 02:00:38 -0700 (PDT) Received: from mail-io0-x230.google.com (mail-io0-x230.google.com [IPv6:2607:f8b0:4001:c06::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 0AF6112D0C9 for ; Mon, 15 Aug 2016 02:00:35 -0700 (PDT) Received: by mail-io0-x230.google.com with SMTP id 38so75073143iol.0 for ; Mon, 15 Aug 2016 02:00:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tieto.com; s=google; h=from:references:in-reply-to:mime-version:thread-index:date :message-id:subject:to:cc; bh=bW/cl1L/5rIdSFG7oqVz3C6vbUO5QPH1OdO76PBz8WU=; b=1bXYT2Ru9iH8MVr+DBd14UHeUknksiQeYhy4mRiLl7VkpQm0mDxU0bzUDjv+bYhidv Lln+NtchRFlTLY5LK44XngBV0inpbdcBElAz1mH63bG4+RijavEEUHztmFMwKoFIz0Gq SvCEQOO27GyvTgSZc1fwwJd9uFkpQj4bikhVU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:references:in-reply-to:mime-version :thread-index:date:message-id:subject:to:cc; bh=bW/cl1L/5rIdSFG7oqVz3C6vbUO5QPH1OdO76PBz8WU=; b=e1eox1iWnTGnSYenRBvzJB+ZnIuFWSKAzmtLhwlQ6XXsCBiQ/RuwVPRbVG10MOzbzu 1U7LteQ0XJAvHj6VeoVhm1kAoAxeS5+wUfdT5UyjM9rUpr5kQwKoIPn4Dx/hlzaioBqE p+f3YMb2uKrf8ks49JizHUrE618ubKzEiTseasd8cTIYoh22w6FmXDVQtrOBk8aGLN6T f4NHLsVSOAInWQcgN7OxL9foZNy/dkBPkdLCfSCkFu7GUrA7lJCFvdj5Wu4Jc223ZbEV wwzW6ZKwqyitKniwjFrMsTDqt2IQhFYR/3teEwRjX6EkmUSvUpDY/AohDQVzJRcoY2iU XfYg== X-Gm-Message-State: AEkoout91zLE8j7qh9RMXi0AuiJ2+a/yKZp91wqVZtq7uvVEaKe2QVfUe62pT4BwihTxXTZFLaimI/uJzhXghx27kqUJSAEVQB37nZ08O0eL3WfHFHGCZ8crgpcIBv8GUfkEkFY= X-Received: by 10.107.20.82 with SMTP id 79mr30901765iou.108.1471251634349; Mon, 15 Aug 2016 02:00:34 -0700 (PDT) From: Karen Elisabeth Egede Nielsen References: <20160708211635.32095.77047.idtracker@ietfa.amsl.com> <2f56d1b8-7450-42de-4a24-53df9cf4c045@it.uc3m.es> In-Reply-To: MIME-Version: 1.0 X-Mailer: Microsoft Outlook 15.0 Thread-Index: AQGd4OoGFyImcsqDlVk0oY8T3VfV3QH845XxAem+5TgCNV3HJ6CA+6qg Date: Mon, 15 Aug 2016 11:00:24 +0200 Message-ID: <623f46dedcc661ae74563f6bf08f46a5@mail.gmail.com> To: Yuchung Cheng Content-Type: text/plain; charset=UTF-8 X-DomainID: tieto.com Archived-At: Cc: tcpm IETF list , TCP Prague List Subject: Re: [tcpm] Fwd: I-D Action: draft-bagnulo-tcpm-tcp-low-rtt-00.txt X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Aug 2016 09:00:41 -0000 Hi Yuchung, > > > > An additional comment is that new approaches to retransmissions - like > > TCP TLP and TCP RACK (also SCTP TLR which however is not progressing at > > the moment) might fundamentally alter the picture. I.e., if > > retransmissions are sent pro-actively in tail loss situations then more > > conservative RTOs may be kept for situations where it is prudent to wait > > longer. Don't know if TCP TLP is so widely deployed that it is something > > that you should relate to even if it may be superseded by RACK. Just a > > thought. > TLP and RACK help reduce timeout cases in DC environment for sure. But > still it can not completely avoid timeouts. [Karen Elisabeth Egede Nielsen] Agreed. For SCTP TLR we still see RTO-timeouts when also probes/retransmissions are lost. I assume that it would be something like this also for Rack/TLP - though potentially depending on how many consecutive probes that are allowed. The role of RTO-timeouts are different when RACK/TLP is enabled and it might be counterproductive to have RTO-timeouts be too aggressive as the function indeed with TLP/RACK becomes something much closer to the original intend (read: how I understand the original intend) - namely to introduce a pause for the network to recover when things are really (consistently) bad. If the RTO is lowered down to be comparable in order of magnitude with the RTT (with some delay_ack considerations) we are narrowing the situation where TLP/RACK probing is able to kick in before RTO-retransmissions starts anyway. I understand that RTOs should be adjusted to the network dynamics, but I do think that it is important to understand - for the DC environment - if the need for the low RTOs in DC's is motivated mainly by having RTO-retransmissions fix the tail loss recovery deficiencies of TCP, which RACK/TLP addresses, or more generally for a need for more timely probe and reactivation for/after network recovery. Or perhaps you see the TLP probe timeout and the RTO timeout eventually being driven by the same timer, the reactions (e.g. CC reactions) then possibly depending on state (probe already sent or not). BR, Karen So I agree we should keep > timeouts conservative, but the current RFCs are way too conservative: > min RTO of 1 second is 4-5 orders of magnitude of DC RTTs. > > The issue is, the "right" min-RTO depends on the actual DC and stacks: > they all have different RTTs and timer granularity. The draft cites > Glenn's paper, but Morgan Stanley's DC could be different than others. > > > Other comments on the draft: > > 1. min ssthresh or cwnd: with very large incast ((tens of) thousands > of senders into one receiver), cwnd of 0.1 pkt can still be too big. the > only way is to pace the packets over N*RTT intervals. > > > 2. delayed acks: > as mentioned the actual time depends a lot on the DC implementation, > which is really "vender/owner"-specific. instead of perhaps an option > during setup to inform the sender about the max delay in the ack works > better. > > > > > > > BR, Karen > > > > > -----Original Message----- > > > From: tcpm [mailto:tcpm-bounces@ietf.org] On Behalf Of marcelo > bagnulo > > > braun > > > Sent: 8. juli 2016 23:19 > > > To: tcpm IETF list > > > Subject: [tcpm] Fwd: I-D Action: draft-bagnulo-tcpm-tcp-low-rtt-00.txt > > > > > > Hi, > > > > > > We just submitted this draft for consideration of the WG. Comments are > > > appreciated. > > > > > > Regards, marcelo > > > > > > > > > > > > > > > -------- Mensaje reenviado -------- > > > Asunto: I-D Action: draft-bagnulo-tcpm-tcp-low-rtt-00.txt > > > Fecha: Fri, 08 Jul 2016 14:16:35 -0700 > > > De: internet-drafts@ietf.org > > > Responder a: internet-drafts@ietf.org > > > Para: i-d-announce@ietf.org > > > > > > > > > > > > A New Internet-Draft is available from the on-line Internet-Drafts > > directories. > > > > > > > > > Title : Recommendations for increasing TCP > > performance in low RTT > > > networks. > > > Authors : Marcelo Bagnulo > > > Koen De Schepper > > > Glenn Judd > > > Filename : draft-bagnulo-tcpm-tcp-low-rtt-00.txt > > > Pages : 7 > > > Date : 2016-07-08 > > > > > > Abstract: > > > This documents compiles a set of issues that negatively affect TCP > > > performance in low RTT networks as well as the recommendations to > > > overcome them. > > > > > > > > > The IETF datatracker status page for this draft is: > > > https://datatracker.ietf.org/doc/draft-bagnulo-tcpm-tcp-low-rtt/ > > > > > > There's also a htmlized version available at: > > > https://tools.ietf.org/html/draft-bagnulo-tcpm-tcp-low-rtt-00 > > > > > > > > > Please note that it may take a couple of minutes from the time of > > submission > > > until the htmlized version and diff are available at tools.ietf.org. > > > > > > Internet-Drafts are also available by anonymous FTP at: > > > ftp://ftp.ietf.org/internet-drafts/ > > > > > > _______________________________________________ > > > I-D-Announce mailing list > > > I-D-Announce@ietf.org > > > https://www.ietf.org/mailman/listinfo/i-d-announce > > > Internet-Draft directories: http://www.ietf.org/shadow.html or > > > ftp://ftp.ietf.org/ietf/1shadow-sites.txt > > > > > > > > > _______________________________________________ > > > tcpm mailing list > > > tcpm@ietf.org > > > https://www.ietf.org/mailman/listinfo/tcpm From nobody Mon Aug 15 02:03:06 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2611712B02B for ; Mon, 15 Aug 2016 02:03:05 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.701 X-Spam-Level: X-Spam-Status: No, score=-2.701 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=unavailable autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=tieto.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4kNNJ2muBfg7 for ; Mon, 15 Aug 2016 02:03:03 -0700 (PDT) Received: from mail-io0-x22b.google.com (mail-io0-x22b.google.com [IPv6:2607:f8b0:4001:c06::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 61D51126FDC for ; Mon, 15 Aug 2016 02:03:03 -0700 (PDT) Received: by mail-io0-x22b.google.com with SMTP id q83so75096278iod.1 for ; Mon, 15 Aug 2016 02:03:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tieto.com; s=google; h=from:references:in-reply-to:mime-version:thread-index:date :message-id:subject:to:cc; bh=bW/cl1L/5rIdSFG7oqVz3C6vbUO5QPH1OdO76PBz8WU=; b=XXWHhvmZqCULdFgb81+GNqpMaH6Exgv0QfGkWybkxyQ5QseFYSn8yJDOgWWdJoSOcU b5l7TNtV4jXYyCeBgVI2nDpByLS+MWGrTMYpS5nuE9IxOBOujYOyww1R8NFeR8bHiCXO FTYjEeAdCd5fGwMJYVlOvcfzruYrvHunBUON0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:references:in-reply-to:mime-version :thread-index:date:message-id:subject:to:cc; bh=bW/cl1L/5rIdSFG7oqVz3C6vbUO5QPH1OdO76PBz8WU=; b=dNPWB32gqnziP7qlSPy89oUNHoGuCOCfwnFngcUA0BHkXGaOhmdEsJvpzkCNcCvEsZ TVZ2ps/85slTaNzKceNZNOzZjRYpV4XioAP7jEleKXZqm9kEFpt94ORcry64I5YFZ+c6 gPPcWra1mBirart1tInIo9C+vZ+o1zol7zlUrXReNEO+NhDzFuXg1EZr7wosq1gm1FF0 PJE2GNVKsAH35HBzZyGlI9hyCqF8REjDahgqTLEgJbEOkKQc+gdMA2LvyTfuKOW3cVYy GE8cy0SdI4qkegbYne2MIGOw+qJdEzawB3bQmICkHNGSjZPS+N3p4DFlAdbhpHVO+waF LbOw== X-Gm-Message-State: AEkooutSCQLxc2TY0VMiFE1LCEHuYB7cKNqXySvjTGjK6iPM17WiUXRKQ26sXKTjEdALh8XA6w4NrBdNursqP/ZEhsclHfm/xN5hW05lma4KGIy7WHw4QVZZFNeGBXbJWgwgyjw= X-Received: by 10.107.37.198 with SMTP id l189mr30922932iol.117.1471251782673; Mon, 15 Aug 2016 02:03:02 -0700 (PDT) From: Karen Elisabeth Egede Nielsen References: <20160708211635.32095.77047.idtracker@ietfa.amsl.com> <2f56d1b8-7450-42de-4a24-53df9cf4c045@it.uc3m.es> In-Reply-To: MIME-Version: 1.0 X-Mailer: Microsoft Outlook 15.0 Thread-Index: AQGd4OoGFyImcsqDlVk0oY8T3VfV3QH845XxAem+5TgCNV3HJ6CA+6qg Date: Mon, 15 Aug 2016 11:00:24 +0200 Message-ID: <1ed9d8e85fb779cfae5c187eddb2fd8b@mail.gmail.com> To: Yuchung Cheng Content-Type: text/plain; charset=UTF-8 X-DomainID: tieto.com Archived-At: Cc: tcpm IETF list , TCP Prague List Subject: Re: [tcpm] Fwd: I-D Action: draft-bagnulo-tcpm-tcp-low-rtt-00.txt X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Aug 2016 09:03:05 -0000 Hi Yuchung, > > > > An additional comment is that new approaches to retransmissions - like > > TCP TLP and TCP RACK (also SCTP TLR which however is not progressing at > > the moment) might fundamentally alter the picture. I.e., if > > retransmissions are sent pro-actively in tail loss situations then more > > conservative RTOs may be kept for situations where it is prudent to wait > > longer. Don't know if TCP TLP is so widely deployed that it is something > > that you should relate to even if it may be superseded by RACK. Just a > > thought. > TLP and RACK help reduce timeout cases in DC environment for sure. But > still it can not completely avoid timeouts. [Karen Elisabeth Egede Nielsen] Agreed. For SCTP TLR we still see RTO-timeouts when also probes/retransmissions are lost. I assume that it would be something like this also for Rack/TLP - though potentially depending on how many consecutive probes that are allowed. The role of RTO-timeouts are different when RACK/TLP is enabled and it might be counterproductive to have RTO-timeouts be too aggressive as the function indeed with TLP/RACK becomes something much closer to the original intend (read: how I understand the original intend) - namely to introduce a pause for the network to recover when things are really (consistently) bad. If the RTO is lowered down to be comparable in order of magnitude with the RTT (with some delay_ack considerations) we are narrowing the situation where TLP/RACK probing is able to kick in before RTO-retransmissions starts anyway. I understand that RTOs should be adjusted to the network dynamics, but I do think that it is important to understand - for the DC environment - if the need for the low RTOs in DC's is motivated mainly by having RTO-retransmissions fix the tail loss recovery deficiencies of TCP, which RACK/TLP addresses, or more generally for a need for more timely probe and reactivation for/after network recovery. Or perhaps you see the TLP probe timeout and the RTO timeout eventually being driven by the same timer, the reactions (e.g. CC reactions) then possibly depending on state (probe already sent or not). BR, Karen So I agree we should keep > timeouts conservative, but the current RFCs are way too conservative: > min RTO of 1 second is 4-5 orders of magnitude of DC RTTs. > > The issue is, the "right" min-RTO depends on the actual DC and stacks: > they all have different RTTs and timer granularity. The draft cites > Glenn's paper, but Morgan Stanley's DC could be different than others. > > > Other comments on the draft: > > 1. min ssthresh or cwnd: with very large incast ((tens of) thousands > of senders into one receiver), cwnd of 0.1 pkt can still be too big. the > only way is to pace the packets over N*RTT intervals. > > > 2. delayed acks: > as mentioned the actual time depends a lot on the DC implementation, > which is really "vender/owner"-specific. instead of perhaps an option > during setup to inform the sender about the max delay in the ack works > better. > > > > > > > BR, Karen > > > > > -----Original Message----- > > > From: tcpm [mailto:tcpm-bounces@ietf.org] On Behalf Of marcelo > bagnulo > > > braun > > > Sent: 8. juli 2016 23:19 > > > To: tcpm IETF list > > > Subject: [tcpm] Fwd: I-D Action: draft-bagnulo-tcpm-tcp-low-rtt-00.txt > > > > > > Hi, > > > > > > We just submitted this draft for consideration of the WG. Comments are > > > appreciated. > > > > > > Regards, marcelo > > > > > > > > > > > > > > > -------- Mensaje reenviado -------- > > > Asunto: I-D Action: draft-bagnulo-tcpm-tcp-low-rtt-00.txt > > > Fecha: Fri, 08 Jul 2016 14:16:35 -0700 > > > De: internet-drafts@ietf.org > > > Responder a: internet-drafts@ietf.org > > > Para: i-d-announce@ietf.org > > > > > > > > > > > > A New Internet-Draft is available from the on-line Internet-Drafts > > directories. > > > > > > > > > Title : Recommendations for increasing TCP > > performance in low RTT > > > networks. > > > Authors : Marcelo Bagnulo > > > Koen De Schepper > > > Glenn Judd > > > Filename : draft-bagnulo-tcpm-tcp-low-rtt-00.txt > > > Pages : 7 > > > Date : 2016-07-08 > > > > > > Abstract: > > > This documents compiles a set of issues that negatively affect TCP > > > performance in low RTT networks as well as the recommendations to > > > overcome them. > > > > > > > > > The IETF datatracker status page for this draft is: > > > https://datatracker.ietf.org/doc/draft-bagnulo-tcpm-tcp-low-rtt/ > > > > > > There's also a htmlized version available at: > > > https://tools.ietf.org/html/draft-bagnulo-tcpm-tcp-low-rtt-00 > > > > > > > > > Please note that it may take a couple of minutes from the time of > > submission > > > until the htmlized version and diff are available at tools.ietf.org. > > > > > > Internet-Drafts are also available by anonymous FTP at: > > > ftp://ftp.ietf.org/internet-drafts/ > > > > > > _______________________________________________ > > > I-D-Announce mailing list > > > I-D-Announce@ietf.org > > > https://www.ietf.org/mailman/listinfo/i-d-announce > > > Internet-Draft directories: http://www.ietf.org/shadow.html or > > > ftp://ftp.ietf.org/ietf/1shadow-sites.txt > > > > > > > > > _______________________________________________ > > > tcpm mailing list > > > tcpm@ietf.org > > > https://www.ietf.org/mailman/listinfo/tcpm From nobody Mon Aug 15 09:02:18 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4207012D908 for ; Mon, 15 Aug 2016 09:02:16 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -8.167 X-Spam-Level: X-Spam-Status: No, score=-8.167 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-1.247] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fZ780Bp83Vyv for ; Mon, 15 Aug 2016 09:02:14 -0700 (PDT) Received: from boreas.isi.edu (boreas.isi.edu [128.9.160.161]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 4C35912B05B for ; Mon, 15 Aug 2016 09:02:14 -0700 (PDT) Received: from [192.168.1.189] (cpe-172-250-251-17.socal.res.rr.com [172.250.251.17]) (authenticated bits=0) by boreas.isi.edu (8.13.8/8.13.8) with ESMTP id u7FG1fq9025098 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Mon, 15 Aug 2016 09:01:43 -0700 (PDT) To: "Scharf, Michael (Nokia - DE)" , Yoav Nir References: <6E459CA7-767F-43DE-A128-C50E50025B99@gmail.com> <9BD95DAD-5278-4252-B179-7EE5F82CA038@gmail.com> <57516f46-bb25-73af-615a-b058bd44270c@isi.edu> <655C07320163294895BBADA28372AF5D489D23F5@FR712WXCHMBA15.zeu.alcatel-lucent.com> From: Joe Touch Message-ID: <52a50015-252f-a4e1-bd0c-d61b51db6e40@isi.edu> Date: Mon, 15 Aug 2016 09:01:39 -0700 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: <655C07320163294895BBADA28372AF5D489D23F5@FR712WXCHMBA15.zeu.alcatel-lucent.com> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Archived-At: Cc: "tcpm@ietf.org" Subject: Re: [tcpm] [saag] Linux flaw from RFC5961 implementation X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Aug 2016 16:02:16 -0000 Hi, Michael, The key question is whether there are actually *errors* in the RFC that cause this problem. AFAICT, this is just an incorrect misreading, not based on an error in the text. I'm not sure why we want to address every one-off incorrect misreading this way, but an errata seems like a better way to handle this than a new RFC. FWIW, the errata should be clear that this isn't based on an *error* in the text. Joe On 8/14/2016 7:54 AM, Scharf, Michael (Nokia - DE) wrote: > According to https://www.ietf.org/iesg/statement/errata-processing.html, the condition for "verified" errata is: > > "Only errors that could cause implementation or deployment problems or significant confusion should be Verified." > > As an individual contributor to TCPM, in this case I believe there is empirical evidence that the wording of RFC 5961 can cause "significant confusion". To me, the conditions for a "verified" errata can thus be fulfilled in this case. BTW, the other two potential states for erratas are "hold for document update" or "rejected". > > Also, I don't believe that an errata with some additional explanation or implementation guidance would cause harm to the RFC series. > > Michael > > >> -----Original Message----- >> From: tcpm [mailto:tcpm-bounces@ietf.org] On Behalf Of Joe Touch >> Sent: Saturday, August 13, 2016 8:47 PM >> To: Yoav Nir >> Cc: tcpm@ietf.org >> Subject: Re: [tcpm] [saag] Linux flaw from RFC5961 implementation >> >> Hi, all, >> >> Moving this thread back to TCPM where it is ongong... >> >> I'll address the note below. >> >> Joe >> >> >> On 8/13/2016 10:45 AM, Yoav Nir wrote: >>>> On 13 Aug 2016, at 6:35 PM, Joe Touch wrote: >>>> >>>> Hi, all, >>>> >>>> This issue is already been discussed in TCPM. Please review the >> existing >>>> thread and continue discussion there. >>>> >>>> A quick summary from my viewpoint: >>>> >>>> a) there already is a draft >>>> >>>> b) IMO, the Linux code is just (and yet another) bug based on an >>>> incorrect read of RFC 5961, not worthy of even an errata >>>> >>>> All of this is in the thread on TCPM. Please discuss it there. >>> Hi, Joe >>> >>> I’ve read the thread, but I disagree with your conclusion there ([1]) >> for several reasons. >>> First, RFCs should be clear enough that reasonably competent >> implementers will get it right. I think it’s safe to say that Linux >> kernel developers are reasonably competent as a whole, so if they got >> it wrong the RFC is not clear enough and requires a clarification in >> one form or another. >> That is demonstrably false on many levels. My students just found a two >> significant bugs in the Linux TCP code just this summer. >> >> Then again, RFC5961 exists because Linux was all to happy to take >> over-eager "security" fixes that could impact normal TCP operation. >> >>> The fact that Microsoft got it right is not proof that any competent >> developer will get it right. >> And the fact that Linux got it wrong is not proof that others will as >> well. >> >>> Second, section 7 talks about rate-limiting to prevent DoS. It talks >> about protecting bandwidth and CPU utilization. It states that "An >> implementation may take an excessive number of invocations of the >> throttling mechanism as an indication that network conditions are >> unusual or hostile.” All of this is global to the system, not specific >> to one connection or another. As far as bandwidth is concerned, there >> is no difference between sending 5 Challenge Acks on each of 20 >> connections, or 100 Challenge Acks on a single connection. >> The entire doc is written in the context of a single connection except >> where it indicates otherwise (per-machine values for per-connection >> limits). Anyone who thinks that an attack on one TCP connection implies >> an attack on another is already incorrect. The document is already >> written to take that into account. >> >>> Third, if a machine has enough CPU and bandwidth to allow X challenge >> acks per second, then dividing X by the number of concurrent >> connections sets a per-connection limit is too low for the mechanism to >> function well. >> If the CPU and bandwidth are that limited, then TCP won't function well >> either. >> >>> Of course, now that we know about the attack it seems obvious that >> the limit should be per-connection and that we shouldn’t let one >> connection starve other connections for challenge-acks. But it’s not so >> obvious that the RFC did not need to mention it. >> >> The RFC doesn't mention not to put your credit card information in the >> header too - hopefully we don't need to explain everything you >> shouldn't do. >> >> Joe >> >> _______________________________________________ >> tcpm mailing list >> tcpm@ietf.org >> https://www.ietf.org/mailman/listinfo/tcpm From nobody Mon Aug 15 09:50:55 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2322712D0DC for ; Mon, 15 Aug 2016 09:50:54 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.248 X-Spam-Level: X-Spam-Status: No, score=-3.248 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RP_MATCHES_RCVD=-1.247, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=google.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nlnH3KQUMGqj for ; Mon, 15 Aug 2016 09:50:51 -0700 (PDT) Received: from mail-it0-x229.google.com (mail-it0-x229.google.com [IPv6:2607:f8b0:4001:c0b::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 4B4EE12D89B for ; Mon, 15 Aug 2016 09:49:54 -0700 (PDT) Received: by mail-it0-x229.google.com with SMTP id f6so9485348ith.0 for ; Mon, 15 Aug 2016 09:49:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=jFbq+qmd924+GMTxXpIpWeGG/lrHFOqdk12Yn0hq4YE=; b=Buub3y2ihWU+aG2sE5NvOkVw+fGseV1CNn3UFwMhmQlHSibAO+0aQyhu1/2GMcOLaY rovmKsqnKr2rpINYOgWOg/Mjhjxmnk10EgvhBWPvDIQiDltc9UoO2M/cC5crIf/xsker N2Teua70ckcLJb5PN06Ehmw25Vh+f+4jYpRg+uU1M/FcVw/iT3hIABCsMGqqo4Y4KR3e VMWVSgw8A1/3aT70DK0bimBR4Np06zCcJ7CQbzcWXIB0TH26GVmcRCCJTmPr5uR6WnNM +YOBjh7FR6MdCrPer71xzv6vHengBkRXl67ruxmDOcJBXeFA98RR2H/fmCPe/SA6kANv 0AkA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=jFbq+qmd924+GMTxXpIpWeGG/lrHFOqdk12Yn0hq4YE=; b=d4BPDNTshgc3pNN+8Mrwou42mSRud+Bd/+ceGsOawP10cfIVWyt36Lsug43WwNkYq5 CaD0huhgS0i4g3aDSkNn8dSgpiEgB9bWJRJ/42Ve4pik9j32C4r7aLdBtXHF2s3QFdVb bW7Us/svJQAt3oFC5LSNizVvLFfiNE2s/NN1uPcYkwto4RkEBXIMdsJNoLM9hdb4eVUf ncuhaScpY+ANl5QIiovlTJU+dpEOrau4QBhPFNyGAyJt88nTiAY4tjzcXk/00eWcSuO4 WkqcPbzqmluyeLYNoMYlvm4fADtbBBqEwH+hjJuWN7OlzOy1AaIGr3CnyhKJUStJEKc/ 2cIA== X-Gm-Message-State: AEkoousFEowpZLiZq3HyLMla8gP6a7ti1xmgMKq7ZWGh28GcAcFv5XIVYrh79caTQHi45DjpE3GLBS9WMjrCB4hY X-Received: by 10.36.82.81 with SMTP id d78mr14561037itb.65.1471279793306; Mon, 15 Aug 2016 09:49:53 -0700 (PDT) MIME-Version: 1.0 Received: by 10.64.244.103 with HTTP; Mon, 15 Aug 2016 09:49:12 -0700 (PDT) In-Reply-To: <623f46dedcc661ae74563f6bf08f46a5@mail.gmail.com> References: <20160708211635.32095.77047.idtracker@ietfa.amsl.com> <2f56d1b8-7450-42de-4a24-53df9cf4c045@it.uc3m.es> <623f46dedcc661ae74563f6bf08f46a5@mail.gmail.com> From: Yuchung Cheng Date: Mon, 15 Aug 2016 09:49:12 -0700 Message-ID: To: Karen Elisabeth Egede Nielsen Content-Type: text/plain; charset=UTF-8 Archived-At: Cc: tcpm IETF list , TCP Prague List Subject: Re: [tcpm] Fwd: I-D Action: draft-bagnulo-tcpm-tcp-low-rtt-00.txt X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Aug 2016 16:50:54 -0000 On Mon, Aug 15, 2016 at 2:00 AM, Karen Elisabeth Egede Nielsen wrote: > Hi Yuchung, > >> > >> > An additional comment is that new approaches to retransmissions - like >> > TCP TLP and TCP RACK (also SCTP TLR which however is not progressing at >> > the moment) might fundamentally alter the picture. I.e., if >> > retransmissions are sent pro-actively in tail loss situations then more >> > conservative RTOs may be kept for situations where it is prudent to wait >> > longer. Don't know if TCP TLP is so widely deployed that it is something >> > that you should relate to even if it may be superseded by RACK. Just a >> > thought. >> TLP and RACK help reduce timeout cases in DC environment for sure. But >> still it can not completely avoid timeouts. > > [Karen Elisabeth Egede Nielsen] Agreed. > > For SCTP TLR we still see RTO-timeouts when also probes/retransmissions are > lost. > I assume that it would be something like this also for Rack/TLP - though > potentially depending on how many consecutive probes that are allowed. > The role of RTO-timeouts are different when RACK/TLP is enabled and it might > be counterproductive to have RTO-timeouts be too aggressive as the function > indeed with TLP/RACK becomes something much closer to the original intend > (read: how I understand the original intend) - namely to introduce a pause > for the network to recover when things are really (consistently) bad. > > If the RTO is lowered down to be comparable in order of magnitude with the > RTT (with some delay_ack considerations) we are narrowing the situation > where TLP/RACK probing is able to kick in before RTO-retransmissions starts > anyway. > > I understand that RTOs should be adjusted to the network dynamics, but I do > think that it is important to understand - for the DC environment - if the > need for the low RTOs in DC's is > motivated mainly by having RTO-retransmissions fix the tail loss recovery > deficiencies of TCP, which RACK/TLP addresses, or more generally for a need > for more timely probe and reactivation for/after network recovery. > > Or perhaps you see the TLP probe timeout and the RTO timeout eventually > being driven by the same timer, the reactions (e.g. CC reactions) then > possibly depending on state (probe already sent or not). I agree with your thought here. More generally, I'd like to evolve the state of art recovery with these principles: 1. React with ack events as much as possible 2. Send tiny probes to trigger (1) after 2-3 RTTs 3. Have conservative RTOs that (exp-backoff) expire at an order of RTT to repair the head for reliability. On thing about RTO: RTO current reset cwnd to 1 upon firing no matter what. We should ONLY reset cwnd to 1 if no news (acks/data) for a long duration. This is quite important for performance: today cwnd is reset to 1 even if every packet but the first one has been recently delivered. This unnecessarily conservative because the ack clock isn't lost yet. But the collateral damage to performance is huge on large BDP networks. This results work-arounds to shorten and avoid RTOs. I'd like to address these points in the upcoming RACK/TLP draft. > > BR, Karen > > So I agree we should keep >> timeouts conservative, but the current RFCs are way too conservative: >> min RTO of 1 second is 4-5 orders of magnitude of DC RTTs. >> >> The issue is, the "right" min-RTO depends on the actual DC and stacks: >> they all have different RTTs and timer granularity. The draft cites >> Glenn's paper, but Morgan Stanley's DC could be different than others. >> >> >> Other comments on the draft: >> >> 1. min ssthresh or cwnd: with very large incast ((tens of) thousands >> of senders into one receiver), cwnd of 0.1 pkt can still be too big. the >> only way is to pace the packets over N*RTT intervals. >> >> >> 2. delayed acks: >> as mentioned the actual time depends a lot on the DC implementation, >> which is really "vender/owner"-specific. instead of perhaps an option >> during setup to inform the sender about the max delay in the ack works >> better. >> >> >> >> > >> > BR, Karen >> > >> > > -----Original Message----- >> > > From: tcpm [mailto:tcpm-bounces@ietf.org] On Behalf Of marcelo >> bagnulo >> > > braun >> > > Sent: 8. juli 2016 23:19 >> > > To: tcpm IETF list >> > > Subject: [tcpm] Fwd: I-D Action: draft-bagnulo-tcpm-tcp-low-rtt-00.txt >> > > >> > > Hi, >> > > >> > > We just submitted this draft for consideration of the WG. Comments are >> > > appreciated. >> > > >> > > Regards, marcelo >> > > >> > > >> > > >> > > >> > > -------- Mensaje reenviado -------- >> > > Asunto: I-D Action: draft-bagnulo-tcpm-tcp-low-rtt-00.txt >> > > Fecha: Fri, 08 Jul 2016 14:16:35 -0700 >> > > De: internet-drafts@ietf.org >> > > Responder a: internet-drafts@ietf.org >> > > Para: i-d-announce@ietf.org >> > > >> > > >> > > >> > > A New Internet-Draft is available from the on-line Internet-Drafts >> > directories. >> > > >> > > >> > > Title : Recommendations for increasing TCP >> > performance in low RTT >> > > networks. >> > > Authors : Marcelo Bagnulo >> > > Koen De Schepper >> > > Glenn Judd >> > > Filename : draft-bagnulo-tcpm-tcp-low-rtt-00.txt >> > > Pages : 7 >> > > Date : 2016-07-08 >> > > >> > > Abstract: >> > > This documents compiles a set of issues that negatively affect TCP >> > > performance in low RTT networks as well as the recommendations to >> > > overcome them. >> > > >> > > >> > > The IETF datatracker status page for this draft is: >> > > https://datatracker.ietf.org/doc/draft-bagnulo-tcpm-tcp-low-rtt/ >> > > >> > > There's also a htmlized version available at: >> > > https://tools.ietf.org/html/draft-bagnulo-tcpm-tcp-low-rtt-00 >> > > >> > > >> > > Please note that it may take a couple of minutes from the time of >> > submission >> > > until the htmlized version and diff are available at tools.ietf.org. >> > > >> > > Internet-Drafts are also available by anonymous FTP at: >> > > ftp://ftp.ietf.org/internet-drafts/ >> > > >> > > _______________________________________________ >> > > I-D-Announce mailing list >> > > I-D-Announce@ietf.org >> > > https://www.ietf.org/mailman/listinfo/i-d-announce >> > > Internet-Draft directories: http://www.ietf.org/shadow.html or >> > > ftp://ftp.ietf.org/ietf/1shadow-sites.txt >> > > >> > > >> > > _______________________________________________ >> > > tcpm mailing list >> > > tcpm@ietf.org >> > > https://www.ietf.org/mailman/listinfo/tcpm From nobody Mon Aug 15 10:03:05 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D0E8212D0B6 for ; Mon, 15 Aug 2016 10:03:03 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.902 X-Spam-Level: X-Spam-Status: No, score=-6.902 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id B59rlqioaRAG for ; Mon, 15 Aug 2016 10:03:01 -0700 (PDT) Received: from smtp-fr.alcatel-lucent.com (fr-hpida-esg-02.alcatel-lucent.com [135.245.210.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 5A99312D0B1 for ; Mon, 15 Aug 2016 10:03:01 -0700 (PDT) Received: from fr712umx4.dmz.alcatel-lucent.com (unknown [135.245.210.45]) by Websense Email Security Gateway with ESMTPS id F20B4AD7A1935; Mon, 15 Aug 2016 17:02:55 +0000 (GMT) Received: from fr712usmtp2.zeu.alcatel-lucent.com (fr712usmtp2.zeu.alcatel-lucent.com [135.239.2.42]) by fr712umx4.dmz.alcatel-lucent.com (GMO-o) with ESMTP id u7FH2wpQ000600 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Mon, 15 Aug 2016 17:02:58 GMT Received: from FR711WXCHHUB01.zeu.alcatel-lucent.com (fr711wxchhub01.zeu.alcatel-lucent.com [135.239.2.111]) by fr712usmtp2.zeu.alcatel-lucent.com (GMO) with ESMTP id u7FH2v4g005033 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Mon, 15 Aug 2016 19:02:58 +0200 Received: from FR712WXCHMBA15.zeu.alcatel-lucent.com ([169.254.7.108]) by FR711WXCHHUB01.zeu.alcatel-lucent.com ([135.239.2.111]) with mapi id 14.03.0195.001; Mon, 15 Aug 2016 19:02:57 +0200 From: "Scharf, Michael (Nokia - DE)" To: Joe Touch , Yoav Nir Thread-Topic: [tcpm] [saag] Linux flaw from RFC5961 implementation Thread-Index: AQHR9ZNHhdnleqm4ZEijW5ctFaZqFqBIhd9QgAGKEICAACu5kA== Date: Mon, 15 Aug 2016 17:02:56 +0000 Message-ID: <655C07320163294895BBADA28372AF5D489DB3B9@FR712WXCHMBA15.zeu.alcatel-lucent.com> References: <6E459CA7-767F-43DE-A128-C50E50025B99@gmail.com> <9BD95DAD-5278-4252-B179-7EE5F82CA038@gmail.com> <57516f46-bb25-73af-615a-b058bd44270c@isi.edu> <655C07320163294895BBADA28372AF5D489D23F5@FR712WXCHMBA15.zeu.alcatel-lucent.com> <52a50015-252f-a4e1-bd0c-d61b51db6e40@isi.edu> In-Reply-To: <52a50015-252f-a4e1-bd0c-d61b51db6e40@isi.edu> Accept-Language: de-DE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.239.27.39] Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 MIME-Version: 1.0 Archived-At: Cc: "tcpm@ietf.org" Subject: Re: [tcpm] [saag] Linux flaw from RFC5961 implementation X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Aug 2016 17:03:04 -0000 U3VyZSwgdGhlIHdvcmRpbmcgaW4gUkZDIDU5NjEgbWF5IG5vdCByZWFsbHkgYmUgZXJyb25lb3Vz LCBidXQgUkZDIHdvcmRpbmcgdGhhdCBpcyBub3QgYnVsbGV0IHByb29mIGlzIGFsc28gYW4gaXNz dWUsIGluIHBhcnRpY3VsYXIgaWYgdGhlIHNlY3VyaXR5IGltcGxpY2F0aW9ucyBhcmUgbm90IHJl YWxseSBvYnZpb3VzLiBUQ1BNIGhhZCB0byBkZWFsIHdpdGggYW1iaWd1aXRpZXMgaW4gUkZDIHdv cmRpbmdzIGluIHRoZSBwYXN0IGFscmVhZHkgKGUuZy4sIFJGQyA2NDI5KS4gDQoNCkl0IGlzIHVw IHRvIHRoZSB3b3JraW5nIGdyb3VwIHRvIGRlY2lkZSB3aGV0aGVyIGFuIGVycmF0YSBpcyBzdWZm aWNpZW50IGluIHRoaXMgY2FzZSAoaS5lLiwgaHR0cHM6Ly93d3cucmZjLWVkaXRvci5vcmcvZXJy YXRhX3NlYXJjaC5waHA/cmZjPTU5NjEmZWlkPTQ3NzIpLiBXZSBjYW4gY2VydGFpbmx5IGRpc2N1 c3MgdGhlIGV4YWN0IHdvcmRpbmcuDQoNCk1pY2hhZWwNCg0KDQotLS0tLU9yaWdpbmFsIE1lc3Nh Z2UtLS0tLQ0KRnJvbTogSm9lIFRvdWNoIFttYWlsdG86dG91Y2hAaXNpLmVkdV0gDQpTZW50OiBN b25kYXksIEF1Z3VzdCAxNSwgMjAxNiA2OjAyIFBNDQpUbzogU2NoYXJmLCBNaWNoYWVsIChOb2tp YSAtIERFKTsgWW9hdiBOaXINCkNjOiB0Y3BtQGlldGYub3JnDQpTdWJqZWN0OiBSZTogW3RjcG1d IFtzYWFnXSBMaW51eCBmbGF3IGZyb20gUkZDNTk2MSBpbXBsZW1lbnRhdGlvbg0KDQpIaSwgTWlj aGFlbCwNCg0KVGhlIGtleSBxdWVzdGlvbiBpcyB3aGV0aGVyIHRoZXJlIGFyZSBhY3R1YWxseSAq ZXJyb3JzKiBpbiB0aGUgUkZDIHRoYXQgY2F1c2UgdGhpcyBwcm9ibGVtLg0KDQpBRkFJQ1QsIHRo aXMgaXMganVzdCBhbiBpbmNvcnJlY3QgbWlzcmVhZGluZywgbm90IGJhc2VkIG9uIGFuIGVycm9y IGluIHRoZSB0ZXh0LiBJJ20gbm90IHN1cmUgd2h5IHdlIHdhbnQgdG8gYWRkcmVzcyBldmVyeSBv bmUtb2ZmIGluY29ycmVjdCBtaXNyZWFkaW5nIHRoaXMgd2F5LCBidXQgYW4gZXJyYXRhIHNlZW1z IGxpa2UgYSBiZXR0ZXIgd2F5IHRvIGhhbmRsZSB0aGlzIHRoYW4gYSBuZXcgUkZDLg0KDQpGV0lX LCB0aGUgZXJyYXRhIHNob3VsZCBiZSBjbGVhciB0aGF0IHRoaXMgaXNuJ3QgYmFzZWQgb24gYW4g KmVycm9yKiBpbiB0aGUgdGV4dC4NCg0KSm9lDQoNCg0KT24gOC8xNC8yMDE2IDc6NTQgQU0sIFNj aGFyZiwgTWljaGFlbCAoTm9raWEgLSBERSkgd3JvdGU6DQo+IEFjY29yZGluZyB0byBodHRwczov L3d3dy5pZXRmLm9yZy9pZXNnL3N0YXRlbWVudC9lcnJhdGEtcHJvY2Vzc2luZy5odG1sLCB0aGUg Y29uZGl0aW9uIGZvciAidmVyaWZpZWQiIGVycmF0YSBpczoNCj4NCj4gICAiT25seSBlcnJvcnMg dGhhdCBjb3VsZCBjYXVzZSBpbXBsZW1lbnRhdGlvbiBvciBkZXBsb3ltZW50IHByb2JsZW1zIG9y IHNpZ25pZmljYW50IGNvbmZ1c2lvbiBzaG91bGQgYmUgVmVyaWZpZWQuIg0KPg0KPiBBcyBhbiBp bmRpdmlkdWFsIGNvbnRyaWJ1dG9yIHRvIFRDUE0sIGluIHRoaXMgY2FzZSBJIGJlbGlldmUgdGhl cmUgaXMgZW1waXJpY2FsIGV2aWRlbmNlIHRoYXQgdGhlIHdvcmRpbmcgb2YgUkZDIDU5NjEgY2Fu IGNhdXNlICJzaWduaWZpY2FudCBjb25mdXNpb24iLiBUbyBtZSwgdGhlIGNvbmRpdGlvbnMgZm9y IGEgInZlcmlmaWVkIiBlcnJhdGEgY2FuIHRodXMgYmUgZnVsZmlsbGVkIGluIHRoaXMgY2FzZS4g QlRXLCB0aGUgb3RoZXIgdHdvIHBvdGVudGlhbCBzdGF0ZXMgZm9yIGVycmF0YXMgYXJlICJob2xk IGZvciBkb2N1bWVudCB1cGRhdGUiIG9yICJyZWplY3RlZCIuDQo+DQo+IEFsc28sIEkgZG9uJ3Qg YmVsaWV2ZSB0aGF0IGFuIGVycmF0YSB3aXRoIHNvbWUgYWRkaXRpb25hbCBleHBsYW5hdGlvbiBv ciBpbXBsZW1lbnRhdGlvbiBndWlkYW5jZSB3b3VsZCBjYXVzZSBoYXJtIHRvIHRoZSBSRkMgc2Vy aWVzLg0KPg0KPiBNaWNoYWVsDQo+DQo+DQo+PiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0K Pj4gRnJvbTogdGNwbSBbbWFpbHRvOnRjcG0tYm91bmNlc0BpZXRmLm9yZ10gT24gQmVoYWxmIE9m IEpvZSBUb3VjaA0KPj4gU2VudDogU2F0dXJkYXksIEF1Z3VzdCAxMywgMjAxNiA4OjQ3IFBNDQo+ PiBUbzogWW9hdiBOaXINCj4+IENjOiB0Y3BtQGlldGYub3JnDQo+PiBTdWJqZWN0OiBSZTogW3Rj cG1dIFtzYWFnXSBMaW51eCBmbGF3IGZyb20gUkZDNTk2MSBpbXBsZW1lbnRhdGlvbg0KPj4NCj4+ IEhpLCBhbGwsDQo+Pg0KPj4gTW92aW5nIHRoaXMgdGhyZWFkIGJhY2sgdG8gVENQTSB3aGVyZSBp dCBpcyBvbmdvbmcuLi4NCj4+DQo+PiBJJ2xsIGFkZHJlc3MgdGhlIG5vdGUgYmVsb3cuDQo+Pg0K Pj4gSm9lDQo+Pg0KPj4NCj4+IE9uIDgvMTMvMjAxNiAxMDo0NSBBTSwgWW9hdiBOaXIgd3JvdGU6 DQo+Pj4+IE9uIDEzIEF1ZyAyMDE2LCBhdCA2OjM1IFBNLCBKb2UgVG91Y2ggPHRvdWNoQGlzaS5l ZHU+IHdyb3RlOg0KPj4+Pg0KPj4+PiBIaSwgYWxsLA0KPj4+Pg0KPj4+PiBUaGlzIGlzc3VlIGlz IGFscmVhZHkgYmVlbiBkaXNjdXNzZWQgaW4gVENQTS4gUGxlYXNlIHJldmlldyB0aGUNCj4+IGV4 aXN0aW5nDQo+Pj4+IHRocmVhZCBhbmQgY29udGludWUgZGlzY3Vzc2lvbiB0aGVyZS4NCj4+Pj4N Cj4+Pj4gQSBxdWljayBzdW1tYXJ5IGZyb20gbXkgdmlld3BvaW50Og0KPj4+Pg0KPj4+PiBhKSB0 aGVyZSBhbHJlYWR5IGlzIGEgZHJhZnQNCj4+Pj4NCj4+Pj4gYikgSU1PLCB0aGUgTGludXggY29k ZSBpcyBqdXN0IChhbmQgeWV0IGFub3RoZXIpIGJ1ZyBiYXNlZCBvbiBhbiANCj4+Pj4gaW5jb3Jy ZWN0IHJlYWQgb2YgUkZDIDU5NjEsIG5vdCB3b3J0aHkgb2YgZXZlbiBhbiBlcnJhdGENCj4+Pj4N Cj4+Pj4gQWxsIG9mIHRoaXMgaXMgaW4gdGhlIHRocmVhZCBvbiBUQ1BNLiBQbGVhc2UgZGlzY3Vz cyBpdCB0aGVyZS4NCj4+PiBIaSwgSm9lDQo+Pj4NCj4+PiBJ4oCZdmUgcmVhZCB0aGUgdGhyZWFk LCBidXQgSSBkaXNhZ3JlZSB3aXRoIHlvdXIgY29uY2x1c2lvbiB0aGVyZSANCj4+PiAoWzFdKQ0K Pj4gZm9yIHNldmVyYWwgcmVhc29ucy4NCj4+PiBGaXJzdCwgUkZDcyBzaG91bGQgYmUgY2xlYXIg ZW5vdWdoIHRoYXQgcmVhc29uYWJseSBjb21wZXRlbnQNCj4+IGltcGxlbWVudGVycyB3aWxsIGdl dCBpdCByaWdodC4gSSB0aGluayBpdOKAmXMgc2FmZSB0byBzYXkgdGhhdCBMaW51eCANCj4+IGtl cm5lbCBkZXZlbG9wZXJzIGFyZSByZWFzb25hYmx5IGNvbXBldGVudCBhcyBhIHdob2xlLCBzbyBp ZiB0aGV5IGdvdCANCj4+IGl0IHdyb25nIHRoZSBSRkMgaXMgbm90IGNsZWFyIGVub3VnaCBhbmQg cmVxdWlyZXMgYSBjbGFyaWZpY2F0aW9uIGluIA0KPj4gb25lIGZvcm0gb3IgYW5vdGhlci4NCj4+ IFRoYXQgaXMgZGVtb25zdHJhYmx5IGZhbHNlIG9uIG1hbnkgbGV2ZWxzLiBNeSBzdHVkZW50cyBq dXN0IGZvdW5kIGEgDQo+PiB0d28gc2lnbmlmaWNhbnQgYnVncyBpbiB0aGUgTGludXggVENQIGNv ZGUganVzdCB0aGlzIHN1bW1lci4NCj4+DQo+PiBUaGVuIGFnYWluLCBSRkM1OTYxIGV4aXN0cyBi ZWNhdXNlIExpbnV4IHdhcyBhbGwgdG8gaGFwcHkgdG8gdGFrZSANCj4+IG92ZXItZWFnZXIgInNl Y3VyaXR5IiBmaXhlcyB0aGF0IGNvdWxkIGltcGFjdCBub3JtYWwgVENQIG9wZXJhdGlvbi4NCj4+ DQo+Pj4gVGhlIGZhY3QgdGhhdCBNaWNyb3NvZnQgZ290IGl0IHJpZ2h0IGlzIG5vdCBwcm9vZiB0 aGF0IGFueSBjb21wZXRlbnQNCj4+IGRldmVsb3BlciB3aWxsIGdldCBpdCByaWdodC4NCj4+IEFu ZCB0aGUgZmFjdCB0aGF0IExpbnV4IGdvdCBpdCB3cm9uZyBpcyBub3QgcHJvb2YgdGhhdCBvdGhl cnMgd2lsbCBhcyANCj4+IHdlbGwuDQo+Pg0KPj4+IFNlY29uZCwgc2VjdGlvbiA3IHRhbGtzIGFi b3V0IHJhdGUtbGltaXRpbmcgdG8gcHJldmVudCBEb1MuIEl0IHRhbGtzDQo+PiBhYm91dCBwcm90 ZWN0aW5nIGJhbmR3aWR0aCBhbmQgQ1BVIHV0aWxpemF0aW9uLiBJdCBzdGF0ZXMgdGhhdCAiQW4g DQo+PiBpbXBsZW1lbnRhdGlvbiBtYXkgdGFrZSBhbiBleGNlc3NpdmUgbnVtYmVyIG9mIGludm9j YXRpb25zIG9mIHRoZSANCj4+IHRocm90dGxpbmcgbWVjaGFuaXNtIGFzIGFuIGluZGljYXRpb24g dGhhdCBuZXR3b3JrIGNvbmRpdGlvbnMgYXJlIA0KPj4gdW51c3VhbCBvciBob3N0aWxlLuKAnSBB bGwgb2YgdGhpcyBpcyBnbG9iYWwgdG8gdGhlIHN5c3RlbSwgbm90IA0KPj4gc3BlY2lmaWMgdG8g b25lIGNvbm5lY3Rpb24gb3IgYW5vdGhlci4gQXMgZmFyIGFzIGJhbmR3aWR0aCBpcyANCj4+IGNv bmNlcm5lZCwgdGhlcmUgaXMgbm8gZGlmZmVyZW5jZSBiZXR3ZWVuIHNlbmRpbmcgNSBDaGFsbGVu Z2UgQWNrcyBvbiANCj4+IGVhY2ggb2YgMjAgY29ubmVjdGlvbnMsIG9yIDEwMCBDaGFsbGVuZ2Ug QWNrcyBvbiBhIHNpbmdsZSBjb25uZWN0aW9uLg0KPj4gVGhlIGVudGlyZSBkb2MgaXMgd3JpdHRl biBpbiB0aGUgY29udGV4dCBvZiBhIHNpbmdsZSBjb25uZWN0aW9uIA0KPj4gZXhjZXB0IHdoZXJl IGl0IGluZGljYXRlcyBvdGhlcndpc2UgKHBlci1tYWNoaW5lIHZhbHVlcyBmb3IgDQo+PiBwZXIt Y29ubmVjdGlvbiBsaW1pdHMpLiBBbnlvbmUgd2hvIHRoaW5rcyB0aGF0IGFuIGF0dGFjayBvbiBv bmUgVENQIA0KPj4gY29ubmVjdGlvbiBpbXBsaWVzIGFuIGF0dGFjayBvbiBhbm90aGVyIGlzIGFs cmVhZHkgaW5jb3JyZWN0LiBUaGUgDQo+PiBkb2N1bWVudCBpcyBhbHJlYWR5IHdyaXR0ZW4gdG8g dGFrZSB0aGF0IGludG8gYWNjb3VudC4NCj4+DQo+Pj4gVGhpcmQsIGlmIGEgbWFjaGluZSBoYXMg ZW5vdWdoIENQVSBhbmQgYmFuZHdpZHRoIHRvIGFsbG93IFggDQo+Pj4gY2hhbGxlbmdlDQo+PiBh Y2tzIHBlciBzZWNvbmQsIHRoZW4gZGl2aWRpbmcgWCBieSB0aGUgbnVtYmVyIG9mIGNvbmN1cnJl bnQgDQo+PiBjb25uZWN0aW9ucyBzZXRzIGEgcGVyLWNvbm5lY3Rpb24gbGltaXQgaXMgdG9vIGxv dyBmb3IgdGhlIG1lY2hhbmlzbSANCj4+IHRvIGZ1bmN0aW9uIHdlbGwuDQo+PiBJZiB0aGUgQ1BV IGFuZCBiYW5kd2lkdGggYXJlIHRoYXQgbGltaXRlZCwgdGhlbiBUQ1Agd29uJ3QgZnVuY3Rpb24g DQo+PiB3ZWxsIGVpdGhlci4NCj4+DQo+Pj4gT2YgY291cnNlLCBub3cgdGhhdCB3ZSBrbm93IGFi b3V0IHRoZSBhdHRhY2sgaXQgc2VlbXMgb2J2aW91cyB0aGF0DQo+PiB0aGUgbGltaXQgc2hvdWxk IGJlIHBlci1jb25uZWN0aW9uIGFuZCB0aGF0IHdlIHNob3VsZG7igJl0IGxldCBvbmUgDQo+PiBj b25uZWN0aW9uIHN0YXJ2ZSBvdGhlciBjb25uZWN0aW9ucyBmb3IgY2hhbGxlbmdlLWFja3MuIEJ1 dCBpdOKAmXMgbm90IA0KPj4gc28gb2J2aW91cyB0aGF0IHRoZSBSRkMgZGlkIG5vdCBuZWVkIHRv IG1lbnRpb24gaXQuDQo+Pg0KPj4gVGhlIFJGQyBkb2Vzbid0IG1lbnRpb24gbm90IHRvIHB1dCB5 b3VyIGNyZWRpdCBjYXJkIGluZm9ybWF0aW9uIGluIA0KPj4gdGhlIGhlYWRlciB0b28gLSBob3Bl ZnVsbHkgd2UgZG9uJ3QgbmVlZCB0byBleHBsYWluIGV2ZXJ5dGhpbmcgeW91IA0KPj4gc2hvdWxk bid0IGRvLg0KPj4NCj4+IEpvZQ0KPj4NCj4+IF9fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fDQo+PiB0Y3BtIG1haWxpbmcgbGlzdA0KPj4gdGNwbUBpZXRmLm9y Zw0KPj4gaHR0cHM6Ly93d3cuaWV0Zi5vcmcvbWFpbG1hbi9saXN0aW5mby90Y3BtDQoNCg== From nobody Mon Aug 15 15:39:25 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0D3C912D797 for ; Mon, 15 Aug 2016 15:39:23 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.147 X-Spam-Level: X-Spam-Status: No, score=-3.147 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RP_MATCHES_RCVD=-1.247, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id j8YaPjK9gxC1 for ; Mon, 15 Aug 2016 15:39:19 -0700 (PDT) Received: from mail.sfc.wide.ad.jp (shonan.sfc.wide.ad.jp [IPv6:2001:200:0:8803::53]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id AFC3112D796 for ; Mon, 15 Aug 2016 15:39:19 -0700 (PDT) Received: from mail-ua0-f182.google.com (mail-ua0-f182.google.com [209.85.217.182]) by mail.sfc.wide.ad.jp (Postfix) with ESMTPSA id 7DDC7278283 for ; Tue, 16 Aug 2016 07:39:17 +0900 (JST) Received: by mail-ua0-f182.google.com with SMTP id k90so94883045uak.1 for ; Mon, 15 Aug 2016 15:39:17 -0700 (PDT) X-Gm-Message-State: AEkoous075/pFUFx/oPTf7iVzL0SYCtA/TdAF88+zOhA3Ul7sSx40GxNO2d0JTtaAF5HY2aBx8pTc0XFw/NG3A== X-Received: by 10.159.32.2 with SMTP id 2mr4330428uam.74.1471300756039; Mon, 15 Aug 2016 15:39:16 -0700 (PDT) MIME-Version: 1.0 Received: by 10.159.37.1 with HTTP; Mon, 15 Aug 2016 15:39:15 -0700 (PDT) In-Reply-To: References: <7070553C-65D1-46EE-95F4-DAE82E1F5A5E@weston.borman.com> <2D524A8D-A5CA-45A6-B94D-FA1DA0CEE609@weston.borman.com> From: Yoshifumi Nishida Date: Mon, 15 Aug 2016 15:39:15 -0700 X-Gmail-Original-Message-ID: Message-ID: To: Kobby Carmona Content-Type: multipart/alternative; boundary=94eb2c04c796ebf35b053a23e6fc Archived-At: Cc: David Borman , "tcpm@ietf.org" Subject: Re: [tcpm] Possible deadlock scenario with retransmission on both sides at the same time X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Aug 2016 22:39:23 -0000 --94eb2c04c796ebf35b053a23e6fc Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Hello, I personally think this is an interesting corner case for discussion. It looks a minor one, but I'm not not very sure if we can leave it for each implementation. I also guess a question would be if the BSD's fix is the best way for the issue. Thanks, -- Yoshi On Thu, Aug 11, 2016 at 1:46 PM, Kobby Carmona wrote: > Hi David, > This makes a lot of sense. We will fix our code. > Thanks for your help on this, > > BTW, > Is this issue of mentioned in any RFC? If not do you see a point in addin= g > explicit note on the SEQ of pure ACK in case of retransmission? > > Kobby > > -----Original Message----- > From: David Borman [mailto:dab@weston.borman.com] > Sent: Monday, August 08, 2016 12:35 AM > To: Kobby Carmona > Cc: tcpm@ietf.org > Subject: Re: [tcpm] Possible deadlock scenario with retransmission on bot= h > sides at the same time > > > On Aug 7, 2016, at 3:37 AM, Kobby Carmona > wrote: > > > > Hi David, > > The code below is true for fast-retransmit. > > But in case of retransmit timer expiration (TCPT_REXMT) snd_nxt is set > to snd_una. And in this case CQND will be set to 1MSS so in the example > below the transmitters can send only a single segment from sequence > 2000/12000. > > Your underlying problem is that the ACK-only packets are being sent with > the wrong sequence number, and that is what is causing them to be dropped= . > One way to fix that is to put SND.NXT back to its previous value after do > the the retransmit. But you are correct, in the BSD code it only does th= at > in the fast retransmit code; when the timer based retransmit code fires, = it > just pulls back SND.NXT to SND.UNA. However, in the BSD code that I=E2= =80=99m > looking at, it keeps track of the largest sequence number sent in SND.MAX= , > and in the tcp_output() path there is this bit of code: > > /* > * If we are doing retransmissions, then snd_nxt will > * not reflect the first unsent octet. For ACK only > * packets, we do not want the sequence number of the > * retransmitted packet, we want the sequence number > * of the next unsent octet. So, if there is no data > * (and no SYN or FIN), use snd_max instead of snd_nxt > * when filling in th_seq. But if we are in persist > * state, snd_max might reflect one byte beyond the > * right edge of the window, so use snd_nxt in that > * case, since we know we aren't doing a retransmission. > * (retransmit and persist are mutually exclusive...) > */ > if (len || (flags & (TH_SYN|TH_FIN)) || tp->t_timer[TCPT_PERSIST]) > th->th_seq =3D htonl(tp->snd_nxt); > else > th->th_seq =3D htonl(tp->snd_max); > > That is what your implementation appears to be missing, and what is > causing your ACK storm. So yes, some variant of your problem has been se= en > before, and this is how the BSD code fixed it. > > -David Borman > > > > > Kobby > > > > -----Original Message----- > > From: David Borman [mailto:dab@weston.borman.com] > > Sent: Thursday, August 04, 2016 10:37 PM > > To: Kobby Carmona > > Cc: tcpm@ietf.org > > Subject: Re: [tcpm] Possible deadlock scenario with retransmission on > > both sides at the same time > > > > After you pull back SND.NXT and do the retransmission, you should then > restore SND.NXT back to where it was, not leave it at the backed off valu= e; > then the ACKs wouldn=E2=80=99t be dropped, since they wouldn=E2=80=99t ha= ve old seq > values. For example, in the 4.4BSD fast retransmit code it had: > > > > tcp_seq onxt =3D tp->snd_nxt; > > ... > > tp->snd_nxt =3D th->th_ack; > > ... > > (void) tcp_output(tp); > > ... > > if (SEQ_GT(onxt, tp->snd_nxt)) > > tp->snd_nxt =3D onxt; > > > > > > -David Borman > > > >> On Aug 4, 2016, at 4:08 AM, Kobby Carmona > wrote: > >> > >> Hi all, > >> While running a bidirectional scenario with random drops in a network > simulator of our (QLogic's NIC) TCP stack we found a case where it seems > there is deadlock in the TCP protocol (the connection will keep sending > pure acks from both sides until RTO will expire multiple times and a RST > will sent to close the connection). > >> The scenario is as follows (there is an example with numbers for each > stage assuming the MSS and each packet is 1000B): > >> 1. Both sides are transmitting data and a single packet is dropped on > either side and the next two packets are received properly > >> Side A - SND.MAX=3D3000, SND.NXT=3D3000, SND.UNA=3D1000, RCV.NXT= =3D11000, > out-of-order block 12000-13000 > >> Side B - SND.MAX =3D13000, SND.NXT =3D13000, SND.UNA=3D11000, > >> RCV.NXT=3D1000, out-of-order block 2000-3000 2. RTO timer expires on b= oth > sides > >> Side A - SND.MAX=3D3000, SND.NXT=3D1000, SND.UNA=3D1000, RCV.NXT= =3D11000, > out-of-order block 12000-13000 > >> Side B - SND.MAX =3D13000, SND.NXT=3D11000, SND.UNA=3D11000, > RCV.NXT=3D1000, > >> out-of-order block 2000-3000 3. Both sides transmit a single packet to > the peer: > >> A->B - pkt.seq=3D1000, pkt.ack=3D11000, len=3D1000 > >> B->A - pkt.seq=3D11000, pkt.ack=3D1000, len=3D1000 3. Both sides = receive > >> the packets and update the receive context: > >> Side A - SND.MAX=3D3000, SND.NXT=3D2000, SND.UNA=3D1000, RCV.NXT= =3D13000 > >> Side B - SND.MAX=3D13000, SND.NXT=3D12000, SND.UNA=3D11000, RCV.N= XT=3D3000 > 4. > >> Both sides send another segment: > >> A->B - pkt.seq=3D2000, pkt.ack=3D13000, len=3D1000 > >> B->A - pkt.seq=3D12000, pkt.ack=3D3000, len=3D1000 5. Both sides = don't > >> accept the packet (and don't update SND.UNA) since the sequence on the > packet is less than RCV.NXT (sequence number check in page 69 of RFC793) > and send a pure ACK instead > >> A->B - pkt.seq=3D2000, pkt.ack=3D13000, len=3D0 (pure ACK) > >> B->A - pkt.seq=3D12000, pkt.ack=3D3000, len=3D0 (pure ACK) 6. Thi= s will > >> continue forever (until the connection will be terminated by RST) sinc= e > every packet that ends before RCV.NXT (even a retransmit from SND.UNA) wi= ll > be dropped. > >> > >> Did anyone encountered this issue before? Is the anything we missed on > this sequence? > >> If this is indeed a real deadlock, there might be several solutions to > this which will require a modification in receive processing of RFC793. B= ut > I would like to know if you think this is a real issue before dealing wit= h > solutions. > >> Thanks, > >> > >> Kobby > >> > >> > >> _______________________________________________ > >> tcpm mailing list > >> tcpm@ietf.org > >> https://www.ietf.org/mailman/listinfo/tcpm > > > > _______________________________________________ > > tcpm mailing list > > tcpm@ietf.org > > https://www.ietf.org/mailman/listinfo/tcpm > > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm > --94eb2c04c796ebf35b053a23e6fc Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Hello,
I personally think this is an interesting= corner case for discussion.=C2=A0
It looks a minor one, but I'm no= t not very sure if we can leave it for each implementation.=C2=A0
=
I also guess a question would be if the BSD's fix is the best= way for the issue.

Thanks,
--
Yoshi

On Thu, Aug 11, 2016 = at 1:46 PM, Kobby Carmona <kobby.Carmona@qlogic.com> = wrote:
Hi David,
This makes a lot of sense. We will fix our code.
Thanks for your help on this,

BTW,
Is this issue of mentioned in any RFC? If not do you see a point in adding = explicit note on the SEQ of pure ACK in case of retransmission?

=C2=A0 =C2=A0 =C2=A0 =C2=A0 Kobby

-----Original Message-----
From: David Borman [mailto:dab@weston.borman.com]
Sent: Monday, August 08, 2016 12:35 AM
To: Kobby Carmona <kobby.Carmona@qlogic.com>
Cc: tcpm@ietf.org Subject: Re: [tcpm] Possible deadlock scenario with retransmission on both = sides at the same time

> On Aug 7, 2016, at 3:37 AM, Kobby Carmona <kobby.Carmona@qlogic.com> wrot= e:
>
> Hi David,
> The code below is true for fast-retransmit.
> But in case of retransmit timer expiration (TCPT_REXMT) snd_nxt is set= to snd_una. And in this case CQND will be set to 1MSS so in the example be= low the transmitters can send only a single segment from sequence 2000/1200= 0.

Your underlying problem is that the ACK-only packets are being sent with th= e wrong sequence number, and that is what is causing them to be dropped.=C2= =A0 One way to fix that is to put SND.NXT back to its previous value after = do the the retransmit.=C2=A0 But you are correct, in the BSD code it only d= oes that in the fast retransmit code; when the timer based retransmit code = fires, it just pulls back SND.NXT to SND.UNA.=C2=A0 However, in the BSD cod= e that I=E2=80=99m looking at, it keeps track of the largest sequence numbe= r sent in SND.MAX, and in the tcp_output() path there is this bit of code:<= br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0/*
=C2=A0 =C2=A0 =C2=A0 =C2=A0 * If we are doing retransmissions, then snd_nxt= will
=C2=A0 =C2=A0 =C2=A0 =C2=A0 * not reflect the first unsent octet.=C2=A0 For= ACK only
=C2=A0 =C2=A0 =C2=A0 =C2=A0 * packets, we do not want the sequence number o= f the
=C2=A0 =C2=A0 =C2=A0 =C2=A0 * retransmitted packet, we want the sequence nu= mber
=C2=A0 =C2=A0 =C2=A0 =C2=A0 * of the next unsent octet.=C2=A0 So, if there = is no data
=C2=A0 =C2=A0 =C2=A0 =C2=A0 * (and no SYN or FIN), use snd_max instead of s= nd_nxt
=C2=A0 =C2=A0 =C2=A0 =C2=A0 * when filling in th_seq.=C2=A0 But if we are i= n persist
=C2=A0 =C2=A0 =C2=A0 =C2=A0 * state, snd_max might reflect one byte beyond = the
=C2=A0 =C2=A0 =C2=A0 =C2=A0 * right edge of the window, so use snd_nxt in t= hat
=C2=A0 =C2=A0 =C2=A0 =C2=A0 * case, since we know we aren't doing a ret= ransmission.
=C2=A0 =C2=A0 =C2=A0 =C2=A0 * (retransmit and persist are mutually exclusiv= e...)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 */
=C2=A0 =C2=A0 =C2=A0 =C2=A0if (len || (flags & (TH_SYN|TH_FIN)) || tp-&= gt;t_timer[TCPT_PERSIST])
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0th->th_seq =3D ht= onl(tp->snd_nxt);
=C2=A0 =C2=A0 =C2=A0 =C2=A0else
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0th->th_seq =3D ht= onl(tp->snd_max);

That is what your implementation appears to be missing, and what is causing= your ACK storm.=C2=A0 So yes, some variant of your problem has been seen b= efore, and this is how the BSD code fixed it.

=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 -David Borman

>
>=C2=A0 =C2=A0 =C2=A0 =C2=A0Kobby
>
> -----Original Message-----
> From: David Borman [mailto:dab@weston.borman.com]
> Sent: Thursday, August 04, 2016 10:37 PM
> To: Kobby Carmona <kobby.Carmona@qlogic.com>
> Cc: tcpm@ietf.org
> Subject: Re: [tcpm] Possible deadlock scenario with retransmission on<= br> > both sides at the same time
>
> After you pull back SND.NXT and do the retransmission, you should then= restore SND.NXT back to where it was, not leave it at the backed off value= ; then the ACKs wouldn=E2=80=99t be dropped, since they wouldn=E2=80=99t ha= ve old seq values.=C2=A0 For example, in the 4.4BSD fast retransmit code it= had:
>
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0tcp_seq onxt =3D tp->snd_nxt;
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0...
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0tp->snd_nxt =3D th->th_ack;
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0...
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0(void) tcp_output(tp);
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0...
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0if (SEQ_GT(onxt, tp->snd_nxt))
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0tp->snd_nxt =3D onxt;
>
>
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0-David Borman
>
>> On Aug 4, 2016, at 4:08 AM, Kobby Carmona <kobby.Carmona@qlogic.com> = wrote:
>>
>> Hi all,
>> While running a bidirectional scenario with random drops in a netw= ork simulator of our (QLogic's NIC) TCP stack we found a case where it = seems there is deadlock in the TCP protocol (the connection will keep sendi= ng pure acks from both sides until RTO will expire multiple times and a RST= will sent to close the connection).
>> The scenario is as follows (there is an example with numbers for e= ach stage assuming the MSS and each packet is 1000B):
>> 1. Both sides are transmitting data and a single packet is dropped= on either side and the next two packets are received properly
>>=C2=A0 =C2=A0 =C2=A0 Side A - SND.MAX=3D3000, SND.NXT=3D3000, SND.U= NA=3D1000, RCV.NXT=3D11000, out-of-order block 12000-13000
>>=C2=A0 =C2=A0 =C2=A0 Side B - SND.MAX =3D13000, SND.NXT =3D13000, S= ND.UNA=3D11000,
>> RCV.NXT=3D1000, out-of-order block 2000-3000 2. RTO timer expires = on both sides
>>=C2=A0 =C2=A0 =C2=A0 Side A - SND.MAX=3D3000, SND.NXT=3D1000, SND.U= NA=3D1000, RCV.NXT=3D11000, out-of-order block 12000-13000
>>=C2=A0 =C2=A0 =C2=A0 Side B - SND.MAX =3D13000, SND.NXT=3D11000, SN= D.UNA=3D11000, RCV.NXT=3D1000,
>> out-of-order block 2000-3000 3. Both sides transmit a single packe= t to the peer:
>>=C2=A0 =C2=A0 =C2=A0 A->B - pkt.seq=3D1000, pkt.ack=3D11000, len= =3D1000
>>=C2=A0 =C2=A0 =C2=A0 B->A - pkt.seq=3D11000, pkt.ack=3D1000, len= =3D1000 3. Both sides receive
>> the packets and update the receive context:
>>=C2=A0 =C2=A0 =C2=A0 Side A - SND.MAX=3D3000, SND.NXT=3D2000, SND.U= NA=3D1000, RCV.NXT=3D13000
>>=C2=A0 =C2=A0 =C2=A0 Side B - SND.MAX=3D13000, SND.NXT=3D12000, SND= .UNA=3D11000, RCV.NXT=3D3000 4.
>> Both sides send another segment:
>>=C2=A0 =C2=A0 =C2=A0 A->B - pkt.seq=3D2000, pkt.ack=3D13000, len= =3D1000
>>=C2=A0 =C2=A0 =C2=A0 B->A - pkt.seq=3D12000, pkt.ack=3D3000, len= =3D1000 5. Both sides don't
>> accept the packet (and don't update SND.UNA) since the sequenc= e on the packet is less than RCV.NXT (sequence number check in page 69 of R= FC793) and send a pure ACK instead
>>=C2=A0 =C2=A0 =C2=A0 A->B - pkt.seq=3D2000, pkt.ack=3D13000, len= =3D0 (pure ACK)
>>=C2=A0 =C2=A0 =C2=A0 B->A - pkt.seq=3D12000, pkt.ack=3D3000, len= =3D0 (pure ACK) 6. This will
>> continue forever (until the connection will be terminated by RST) = since every packet that ends before RCV.NXT (even a retransmit from SND.UNA= ) will be dropped.
>>
>> Did anyone encountered this issue before? Is the anything we misse= d on this sequence?
>> If this is indeed a real deadlock, there might be several solution= s to this which will require a modification in receive processing of RFC793= . But I would like to know if you think this is a real issue before dealing= with solutions.
>> Thanks,
>>
>>=C2=A0 =C2=A0 =C2=A0 Kobby
>>
>>
>> _______________________________________________
>> tcpm mailing list
>> tcpm@ietf.org
>> https://www.ietf.org/mailman/listinfo/tcpm
>
> _______________________________________________
> tcpm mailing list
> tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm
_______________________________________________
tcpm mailing list
tcpm@ietf.org
https://www.ietf.org/mailman/listinfo/tcpm

--94eb2c04c796ebf35b053a23e6fc-- From nobody Mon Aug 15 18:19:20 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9C0C712D58A for ; Mon, 15 Aug 2016 18:19:19 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.948 X-Spam-Level: X-Spam-Status: No, score=-3.948 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RP_MATCHES_RCVD=-1.247, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=google.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2H8IcDA0LrKa for ; Mon, 15 Aug 2016 18:19:18 -0700 (PDT) Received: from mail-oi0-x22e.google.com (mail-oi0-x22e.google.com [IPv6:2607:f8b0:4003:c06::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 5418412D586 for ; Mon, 15 Aug 2016 18:19:18 -0700 (PDT) Received: by mail-oi0-x22e.google.com with SMTP id 4so81280068oih.2 for ; Mon, 15 Aug 2016 18:19:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=ZqX4IdRpbXKXt9+KCdEQniYvitWLkFXL+EmWmOqqVoo=; b=k/JgeBbJceXxB8QDkqLItj5Qt2hL8+t9ykUBU0MyiaXAEEMkNtElreOJKTZSWwAK6L vQX9AzQ2XygtMtBBjB1mB3TsKRHW3LaQLXYVqpqiPdS2342rGVpPaTIrCOrsZ4rwIBk4 KJX92up63BdWlOqgZ3bxqeJoOSO8jdCTusGGmom+63NmgY7J6StKr8Bt3GEIucimdLSO n7BKorbpgwIF5FvONnmJ4Ki7lcQtAbZrW2kEgIRUQgELsdCmD54ndEnQ2fY/wB1tTBR9 OGXl5TvmbRbd6VNRNUrXG2tydF8j/8vdfBDKsia8ZQ44iNKf50a0oH8faz0SHOVKNnIy 3a5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=ZqX4IdRpbXKXt9+KCdEQniYvitWLkFXL+EmWmOqqVoo=; b=kz/FX1+3bOgKQL+FlpcesJapCTHJI6bI4siEa86eYMAwlDJqCfCS/IPHLZ3BPba4B9 xC2onEoVFiSbYNuSlKGk2NoYpkaRBaMMUFc2iSnJtaKfoiSWGTfAfIU5Dww7xd6ytlhd jOj4QDaJ0/wHLjXX0VBpZyBshN6whN7i2Ndfz85nnT1fsp/9/TaKDdzP2eBQ6cVb0zql MNuiWD4WbAdQWAK+z7vgheSdNYbw8a0UqQG/Dt8uSHCN4SInBe0t5FsOIgyN0lRhveCu wsdacISFutHtiosKsO2EFIaC8jJ2AG7Bhb8VJSDXiYb2v+OQBiXsOAOIXYa+0890ZxBs R/tw== X-Gm-Message-State: AEkoouvsxDpeHWBNArhsVVLDwRseMsw1SOMDXThZ5VQmYinhOSdY4vCH8huDXgi0jN4Lh0lvQJ+nMlOWuSWReEQB X-Received: by 10.157.34.135 with SMTP id y7mr2301342ota.111.1471310357565; Mon, 15 Aug 2016 18:19:17 -0700 (PDT) MIME-Version: 1.0 Received: by 10.202.92.194 with HTTP; Mon, 15 Aug 2016 18:18:47 -0700 (PDT) In-Reply-To: References: <7070553C-65D1-46EE-95F4-DAE82E1F5A5E@weston.borman.com> <2D524A8D-A5CA-45A6-B94D-FA1DA0CEE609@weston.borman.com> From: Neal Cardwell Date: Mon, 15 Aug 2016 21:18:47 -0400 Message-ID: To: Yoshifumi Nishida Content-Type: text/plain; charset=UTF-8 Archived-At: Cc: Kobby Carmona , David Borman , "tcpm@ietf.org" Subject: Re: [tcpm] Possible deadlock scenario with retransmission on both sides at the same time X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Aug 2016 01:19:19 -0000 On Mon, Aug 15, 2016 at 6:39 PM, Yoshifumi Nishida wrote: > Hello, > I personally think this is an interesting corner case for discussion. > It looks a minor one, but I'm not not very sure if we can leave it for each > implementation. > I also guess a question would be if the BSD's fix is the best way for the > issue. Yes, I agree this is an interesting case for discussion. FWIW, as a point of comparison for discussion, Linux's approach is a little different: in Linux, the sender does not rewind SND.NXT on retransmissions (RTO or Fast Recovery). Then the sender usually uses SND.NXT for the seq field of outgoing pure ACKs. I say "usually" because the Linux code has some code to deal with the case where the receiver has withdrawn the receive window, so that SND.NXT is now beyond the receive window. The code tcp_send_ack() uses to pick a seq for outgoing pure ACKs looks like: /* SND.NXT, if window was not shrunk. * If window has been shrunk, what should we make? It is not clear at all. * Using SND.UNA we will fail to open window, SND.NXT is out of window. :-( * Anything in between SND.UNA...SND.UNA+SND.WND also can be already * invalid. OK, let's make this for now: */ static inline __u32 tcp_acceptable_seq(const struct sock *sk) { const struct tcp_sock *tp = tcp_sk(sk); if (!before(tcp_wnd_end(tp), tp->snd_nxt)) return tp->snd_nxt; else return tcp_wnd_end(tp); } neal From nobody Mon Aug 15 22:09:41 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 076AE12B043 for ; Mon, 15 Aug 2016 22:09:39 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.147 X-Spam-Level: X-Spam-Status: No, score=-3.147 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RP_MATCHES_RCVD=-1.247] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vbMk93gyprRc for ; Mon, 15 Aug 2016 22:09:36 -0700 (PDT) Received: from frantic.weston.borman.com (frantic.weston.borman.com [70.57.156.33]) (using TLSv1 with cipher EDH-RSA-DES-CBC3-SHA (112/168 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id EDD8912B019 for ; Mon, 15 Aug 2016 22:09:35 -0700 (PDT) Received: from [192.168.1.36] (local-36.weston.borman.com [192.168.1.36]) by frantic.weston.borman.com (8.14.7/8.14.7) with ESMTP id u7G4gLkW013295; Mon, 15 Aug 2016 23:42:22 -0500 (CDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) From: David Borman In-Reply-To: Date: Tue, 16 Aug 2016 00:09:36 -0500 Content-Transfer-Encoding: quoted-printable Message-Id: References: <7070553C-65D1-46EE-95F4-DAE82E1F5A5E@weston.borman.com> <2D524A8D-A5CA-45A6-B94D-FA1DA0CEE609@weston.borman.com> To: Yoshifumi Nishida X-Mailer: Apple Mail (2.3124) Archived-At: Cc: Kobby Carmona , "tcpm@ietf.org" Subject: Re: [tcpm] Possible deadlock scenario with retransmission on both sides at the same time X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Aug 2016 05:09:39 -0000 Probably the more correct way of stating it is that ACK-only packets = should be sent with the largest in-window sequence number that has ever = been sent. It is the only way to guarantee that the sequence number of = the ACK will be within the window at the receiving side. That being said, probe packets can be out of window. And if the window = is truly closed, you don=E2=80=99t want to send and ACK-only packet with = that sequence number, as it will also be out of window. Hence, send = ACK-only packets with the largest in window sequence number. In the BSD = code probes are only one byte, and in persist state SND.NXT will be the = largest in-window that has ever been sent. But if we are in persist = state because the ACK was lost or the window just opened and the probe = data is accepted, the ACK will be to the left of the window by the size = of the probe packet. Hence the receiver needs to accept ACK packets one = byte to the left of the window to prevent an ACK war. The more general = case is that you need to accept ACKs of at least N bytes to the left of = the window, where N represents the largest size of probe that you will = ever send. That, along with sending ACKs with the largest in-window = sequence number, will guarantee that you don=E2=80=99t get into an ACK = war, even the two sides have different values for N (only one side = needs to accept the ACK to stop the ACK-war, and that will happen on the = side with the larger value for N). -David Borman > On Aug 15, 2016, at 5:39 PM, Yoshifumi Nishida = wrote: >=20 > Hello, > I personally think this is an interesting corner case for discussion.=20= > It looks a minor one, but I'm not not very sure if we can leave it for = each implementation.=20 > I also guess a question would be if the BSD's fix is the best way for = the issue. >=20 > Thanks, > -- > Yoshi >=20 > On Thu, Aug 11, 2016 at 1:46 PM, Kobby Carmona = wrote: > Hi David, > This makes a lot of sense. We will fix our code. > Thanks for your help on this, >=20 > BTW, > Is this issue of mentioned in any RFC? If not do you see a point in = adding explicit note on the SEQ of pure ACK in case of retransmission? >=20 > Kobby >=20 > -----Original Message----- > From: David Borman [mailto:dab@weston.borman.com] > Sent: Monday, August 08, 2016 12:35 AM > To: Kobby Carmona > Cc: tcpm@ietf.org > Subject: Re: [tcpm] Possible deadlock scenario with retransmission on = both sides at the same time >=20 >> On Aug 7, 2016, at 3:37 AM, Kobby Carmona = wrote: >>=20 >> Hi David, >> The code below is true for fast-retransmit. >> But in case of retransmit timer expiration (TCPT_REXMT) snd_nxt is = set to snd_una. And in this case CQND will be set to 1MSS so in the = example below the transmitters can send only a single segment from = sequence 2000/12000. >=20 > Your underlying problem is that the ACK-only packets are being sent = with the wrong sequence number, and that is what is causing them to be = dropped. One way to fix that is to put SND.NXT back to its previous = value after do the the retransmit. But you are correct, in the BSD code = it only does that in the fast retransmit code; when the timer based = retransmit code fires, it just pulls back SND.NXT to SND.UNA. However, = in the BSD code that I=E2=80=99m looking at, it keeps track of the = largest sequence number sent in SND.MAX, and in the tcp_output() path = there is this bit of code: >=20 > /* > * If we are doing retransmissions, then snd_nxt will > * not reflect the first unsent octet. For ACK only > * packets, we do not want the sequence number of the > * retransmitted packet, we want the sequence number > * of the next unsent octet. So, if there is no data > * (and no SYN or FIN), use snd_max instead of snd_nxt > * when filling in th_seq. But if we are in persist > * state, snd_max might reflect one byte beyond the > * right edge of the window, so use snd_nxt in that > * case, since we know we aren't doing a retransmission. > * (retransmit and persist are mutually exclusive...) > */ > if (len || (flags & (TH_SYN|TH_FIN)) || = tp->t_timer[TCPT_PERSIST]) > th->th_seq =3D htonl(tp->snd_nxt); > else > th->th_seq =3D htonl(tp->snd_max); >=20 > That is what your implementation appears to be missing, and what is = causing your ACK storm. So yes, some variant of your problem has been = seen before, and this is how the BSD code fixed it. >=20 > -David Borman >=20 >>=20 >> Kobby >>=20 >> -----Original Message----- >> From: David Borman [mailto:dab@weston.borman.com] >> Sent: Thursday, August 04, 2016 10:37 PM >> To: Kobby Carmona >> Cc: tcpm@ietf.org >> Subject: Re: [tcpm] Possible deadlock scenario with retransmission on >> both sides at the same time >>=20 >> After you pull back SND.NXT and do the retransmission, you should = then restore SND.NXT back to where it was, not leave it at the backed = off value; then the ACKs wouldn=E2=80=99t be dropped, since they = wouldn=E2=80=99t have old seq values. For example, in the 4.4BSD fast = retransmit code it had: >>=20 >> tcp_seq onxt =3D tp->snd_nxt; >> ... >> tp->snd_nxt =3D th->th_ack; >> ... >> (void) tcp_output(tp); >> ... >> if (SEQ_GT(onxt, tp->snd_nxt)) >> tp->snd_nxt =3D onxt; >>=20 >>=20 >> -David Borman >>=20 >>> On Aug 4, 2016, at 4:08 AM, Kobby Carmona = wrote: >>>=20 >>> Hi all, >>> While running a bidirectional scenario with random drops in a = network simulator of our (QLogic's NIC) TCP stack we found a case where = it seems there is deadlock in the TCP protocol (the connection will keep = sending pure acks from both sides until RTO will expire multiple times = and a RST will sent to close the connection). >>> The scenario is as follows (there is an example with numbers for = each stage assuming the MSS and each packet is 1000B): >>> 1. Both sides are transmitting data and a single packet is dropped = on either side and the next two packets are received properly >>> Side A - SND.MAX=3D3000, SND.NXT=3D3000, SND.UNA=3D1000, = RCV.NXT=3D11000, out-of-order block 12000-13000 >>> Side B - SND.MAX =3D13000, SND.NXT =3D13000, SND.UNA=3D11000, >>> RCV.NXT=3D1000, out-of-order block 2000-3000 2. RTO timer expires on = both sides >>> Side A - SND.MAX=3D3000, SND.NXT=3D1000, SND.UNA=3D1000, = RCV.NXT=3D11000, out-of-order block 12000-13000 >>> Side B - SND.MAX =3D13000, SND.NXT=3D11000, SND.UNA=3D11000, = RCV.NXT=3D1000, >>> out-of-order block 2000-3000 3. Both sides transmit a single packet = to the peer: >>> A->B - pkt.seq=3D1000, pkt.ack=3D11000, len=3D1000 >>> B->A - pkt.seq=3D11000, pkt.ack=3D1000, len=3D1000 3. Both sides = receive >>> the packets and update the receive context: >>> Side A - SND.MAX=3D3000, SND.NXT=3D2000, SND.UNA=3D1000, = RCV.NXT=3D13000 >>> Side B - SND.MAX=3D13000, SND.NXT=3D12000, SND.UNA=3D11000, = RCV.NXT=3D3000 4. >>> Both sides send another segment: >>> A->B - pkt.seq=3D2000, pkt.ack=3D13000, len=3D1000 >>> B->A - pkt.seq=3D12000, pkt.ack=3D3000, len=3D1000 5. Both sides = don't >>> accept the packet (and don't update SND.UNA) since the sequence on = the packet is less than RCV.NXT (sequence number check in page 69 of = RFC793) and send a pure ACK instead >>> A->B - pkt.seq=3D2000, pkt.ack=3D13000, len=3D0 (pure ACK) >>> B->A - pkt.seq=3D12000, pkt.ack=3D3000, len=3D0 (pure ACK) 6. = This will >>> continue forever (until the connection will be terminated by RST) = since every packet that ends before RCV.NXT (even a retransmit from = SND.UNA) will be dropped. >>>=20 >>> Did anyone encountered this issue before? Is the anything we missed = on this sequence? >>> If this is indeed a real deadlock, there might be several solutions = to this which will require a modification in receive processing of = RFC793. But I would like to know if you think this is a real issue = before dealing with solutions. >>> Thanks, >>>=20 >>> Kobby >>>=20 >>>=20 >>> _______________________________________________ >>> tcpm mailing list >>> tcpm@ietf.org >>> https://www.ietf.org/mailman/listinfo/tcpm >>=20 >> _______________________________________________ >> tcpm mailing list >> tcpm@ietf.org >> https://www.ietf.org/mailman/listinfo/tcpm >=20 > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm >=20 > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm From nobody Tue Aug 16 06:08:52 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D75C412D7FE for ; Tue, 16 Aug 2016 06:08:50 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -5.447 X-Spam-Level: X-Spam-Status: No, score=-5.447 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RP_MATCHES_RCVD=-1.247] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tTKhnq_Auget for ; Tue, 16 Aug 2016 06:08:48 -0700 (PDT) Received: from smtp.ee.ethz.ch (smtp.ee.ethz.ch [129.132.2.219]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id A9ED912D1A6 for ; Tue, 16 Aug 2016 06:08:48 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by smtp.ee.ethz.ch (Postfix) with ESMTP id 44B4ED9310; Tue, 16 Aug 2016 15:08:46 +0200 (MEST) X-Virus-Scanned: by amavisd-new on smtp.ee.ethz.ch Received: from smtp.ee.ethz.ch ([127.0.0.1]) by localhost (.ee.ethz.ch [127.0.0.1]) (amavisd-new, port 10024) with LMTP id WV2PtZ6Wa3nD; Tue, 16 Aug 2016 15:08:46 +0200 (MEST) Received: from [82.130.103.143] (nb-10510.ethz.ch [82.130.103.143]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: mirjak) by smtp.ee.ethz.ch (Postfix) with ESMTPSA id C42E4D930D; Tue, 16 Aug 2016 15:08:45 +0200 (MEST) To: Joe Touch , Neal Cardwell , Yuchung Cheng References: <20160810183654.05358B80C3A@rfc-editor.org> <7ac89d58-fc3a-9e12-9d22-0602944f8677@isi.edu> From: =?UTF-8?Q?Mirja_K=c3=bchlewind?= Message-ID: <57B31020.60506@tik.ee.ethz.ch> Date: Tue, 16 Aug 2016 15:07:44 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.8.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Archived-At: Cc: "tcpm@ietf.org Extensions" Subject: Re: [tcpm] [Technical Errata Reported] RFC5961 (4772) X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Aug 2016 13:08:51 -0000 Hi all, let me try to wrap this up: To me this does not seem to be a case for an errata. For sure some clarification seems to be helpful here but that's not what erratas are for. Therefore, I would suggest to simply set this into "Held for Document Update" state such that it is noticed in case we decide to update the document at some point. Does that make sense? Thanks, Mirja On 12.08.2016 22:19, Joe Touch wrote: > > > On 8/12/2016 12:17 PM, Neal Cardwell wrote: >> On Fri, Aug 12, 2016 at 9:54 AM, Yuchung Cheng wrote: >>> I think an errata on recommending per-connection limit and cite the >>> research finding is worthy. >> I agree that it's worthwhile to have an errata to clarify that a >> per-connection limit is best. At the very least there seems >> some risky ambiguity here. > > I disagree, as per below... >> >> In fact, AFAICT RFC 5961, Section 7, page 13 -- >> >> https://tools.ietf.org/html/rfc5961#section-7 >> >> -- has several passages that seem to implicitly suggest that >> the rate-limiting mechanism it specifies as a SHOULD is not >> per-connection but rather global. > > There's plenty of benefit to setting a single configuration for an > entire system vs. setting per-connection limits. That's completely > different from whether the ACKs are tallied globally or per connection. > >> First: >> >> ... an >> administrator who is more interested in faster cleanup of stale >> connections (i.e., concerned about excess TCP state) may decide to >> set a higher value thus allowing more RST's to be processed in any >> given time period. >> >> If this is a single-connection rate limit, then what practical reason >> would an administrator have to allow "more RST's to be processed in >> any given time period"? I can't think of a case where a legit live >> connection would send more than one RST. > A single connection might process many RSTs because some can be out of > window (they'd be dropped) or not at the end of the window (dropped > under some configurations). I.e., receiving and evaluating RSTs doesn't > always result in resetting the connection. > >> If this is about a single >> connection, then allowing one mid-window RST (and responding to that >> one with a single challenge ACK) should be enough, > It would be IF the other end issued a legitimate RST. If this is a > spoofing attack, the data can still keep coming which means that the > window might still be moving forward by the time the next attack RST > arrives. As a result, this sort of 'catch up' can go on until the > connection ends. The point of the text here is to avoid wasting > resources in such cases. >> in most cases, to >> send a challenge ACK that elicits a proper RST from the remote >> endpoint. It's not clear to me what the motivation would be for >> intentionally sending a challenge ACK in response to more than one >> mid-window RST in a single connection. > Protecting against that case is the motivation *of the entire document*. > >> Thus AFAICT "allowing more RST's to be processed in any given time >> period" suggests that the rate limit is across connections. > > There's no benefit to doing this across connections at all, unless you > think that there's some reason that legitimate connections shouldn't be > reset by valid RSTs (there's no benefit to that). > >> Second: >> >> The time limit SHOULD be tunable to help timeout brute force attacks >> faster than a potential legitimate flood of RSTs. >> >> I may be missing something, but if this rate-limiting mechanism is >> only about a single connection, then what is a "legitimate flood of >> RSTs"? > A sequence of RSTs that pick different values, as described in RFC 4953. > > Or a sequence of RSTs aimed at responses to the ACK-after-RST that is > chasing legitimate data, as described above. > >> Why would it be important for a single connection to allow a >> "legitimate flood of RSTs" destinted for itself? > It wouldn't - that's the point of this text. You should treat a flood of > RSTs for a single connection as an attack and throttle them accordingly. > >> A single legitimate >> live connection should not send out a flood of RSTs; it should send >> out one. Once there is one legitimate RST that arrives and is >> processed, the receiver deletes the connection, and any other arriving >> RSTs don't match any existing connection, so are ignored and >> dropped, in which case is no challenge ACK to rate-limit. > Again, the point of this document are attacks, not legitimate connections. >> By contrast, if this rate limit applies across connections then "a >> potential legitimate flood of RSTs" makes sense, since a group of >> legitimate live connections that all close in a burst may, in certain >> cases, generate a legitimate flood of RSTs. So tuning the limit >> upward would make sense. > > There's no case I can imagine where a set of attacks on one connection > should be cause for concern on another necessarily. > > Yes, there's no real need to set per-connection limits (i.e., a global > per-connection limit seems fine), but counting them as a group serves no > purpose against any threat, unless you think that "if I'm attacked on > one connection, I will be attacked on another" - but that's an incorrect > inference IMO. > >> >> Sorry if I'm missing something obvious. But it seems like there are at >> least real ambiguities there, on a point significant enough to be worth >> clearing up. > > I think you're missing the point of the document, but that's just my > interpretation of the exchange. > > Joe > > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm > From nobody Tue Aug 16 10:43:34 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id EE12E12B034 for ; Tue, 16 Aug 2016 10:43:33 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.947 X-Spam-Level: X-Spam-Status: No, score=-3.947 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, RP_MATCHES_RCVD=-1.247, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=google.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JAOZrPwvKqPR for ; Tue, 16 Aug 2016 10:43:31 -0700 (PDT) Received: from mail-it0-x22a.google.com (mail-it0-x22a.google.com [IPv6:2607:f8b0:4001:c0b::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1219E12B032 for ; Tue, 16 Aug 2016 10:43:30 -0700 (PDT) Received: by mail-it0-x22a.google.com with SMTP id x131so21859768ite.0 for ; Tue, 16 Aug 2016 10:43:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=b/FpIHi6sQ3awdTFFFEeNlwMGaPeP1EQAX1Js245RRA=; b=E0ggBxTyC5HzQ9blKLJ0FwAsH/f/Tk2CJPfbpdAKLfYFg3jwsrV24npp1e2nBVEI+Q oeu6k1Dzy20NXBFWt1hSt0H0MRcbAUrJ1BgRyZ2/7DgKp6cEoggSPrSwTnYJN14ReZEa +PgbkjR+iM931CDlz7mqDdZW3fZTgoOBm/olk4sfhQq/IIiyE712zU0i5p3HxnjFw782 LGfugCKjD0x/j/eo15V4JNLOdAVtTNkC0ohv7ZJ2PiUsU9FYyyznuQtdfdcjUbke+mWQ wfM+Yer3evTXwTQzKkeGzMA4VEPR8FWAWO2OaDH8Seh8VehwggNuqiJpYBuyJl4fjWut n86w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=b/FpIHi6sQ3awdTFFFEeNlwMGaPeP1EQAX1Js245RRA=; b=AQMIPgVN17qPR5RxrQ6Zlh8ZDvztj7hf5nh3wV8aAsN2rumpMpf5xQXRaZm1/WvOwj y9uiPhRws2o91QbFJMW6zwPABm33R5pdBvFGST2XsHWgAydrnXeet/0qO8dYMYu0afgQ srSJrOCewwkq6A4j8gZSoPO4+3l+CL5M/uJxuzl6bmtWWeNcgb8/U7My59fiqXGZGHBb qekMX5hJ331IzLQLqlLbeWdIm6rN4aa5U4USq2lp/k4Vl6uhX2JUmMd6ID7S1/Rn8fx3 aUEfLNv1v95FU5ppxve7JobJ9qNWr1rfNPN19zTP9GLklCOFb+OhgP0PhBTLwVHXgKEA S14A== X-Gm-Message-State: AEkoousNT/5v2BmaoobzMatNvTSW/lSBMynZRITYmRYU/AkUAPtl8X8J1nPo35A5/gbNaRWk/cnOT06chEJXJgtl X-Received: by 10.36.41.205 with SMTP id p196mr23715429itp.76.1471369409996; Tue, 16 Aug 2016 10:43:29 -0700 (PDT) MIME-Version: 1.0 Received: by 10.64.244.103 with HTTP; Tue, 16 Aug 2016 10:42:49 -0700 (PDT) In-Reply-To: <57B31020.60506@tik.ee.ethz.ch> References: <20160810183654.05358B80C3A@rfc-editor.org> <7ac89d58-fc3a-9e12-9d22-0602944f8677@isi.edu> <57B31020.60506@tik.ee.ethz.ch> From: Yuchung Cheng Date: Tue, 16 Aug 2016 10:42:49 -0700 Message-ID: To: =?UTF-8?Q?Mirja_K=C3=BChlewind?= Content-Type: multipart/alternative; boundary=001a113f6d5e04a744053a33e374 Archived-At: Cc: "tcpm@ietf.org Extensions" , Joe Touch Subject: Re: [tcpm] [Technical Errata Reported] RFC5961 (4772) X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Aug 2016 17:43:34 -0000 --001a113f6d5e04a744053a33e374 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On Tue, Aug 16, 2016 at 6:07 AM, Mirja K=C3=BChlewind < mirja.kuehlewind@tik.ee.ethz.ch> wrote: > Hi all, > > let me try to wrap this up: > > To me this does not seem to be a case for an errata. For sure some > clarification seems to be helpful here but that's not what erratas are fo= r. > > Therefore, I would suggest to simply set this into "Held for Document > Update" state such that it is noticed in case we decide to update the > document at some point. Does that make sense? > sounds fine with me but do documents really get updated? > > Thanks, > Mirja > > > > > On 12.08.2016 22:19, Joe Touch wrote: > >> >> >> On 8/12/2016 12:17 PM, Neal Cardwell wrote: >> >>> On Fri, Aug 12, 2016 at 9:54 AM, Yuchung Cheng >>> wrote: >>> >>>> I think an errata on recommending per-connection limit and cite the >>>> research finding is worthy. >>>> >>> I agree that it's worthwhile to have an errata to clarify that a >>> per-connection limit is best. At the very least there seems >>> some risky ambiguity here. >>> >> >> I disagree, as per below... >> >>> >>> In fact, AFAICT RFC 5961, Section 7, page 13 -- >>> >>> https://tools.ietf.org/html/rfc5961#section-7 >>> >>> -- has several passages that seem to implicitly suggest that >>> the rate-limiting mechanism it specifies as a SHOULD is not >>> per-connection but rather global. >>> >> >> There's plenty of benefit to setting a single configuration for an >> entire system vs. setting per-connection limits. That's completely >> different from whether the ACKs are tallied globally or per connection. >> >> First: >>> >>> ... an >>> administrator who is more interested in faster cleanup of stale >>> connections (i.e., concerned about excess TCP state) may decide to >>> set a higher value thus allowing more RST's to be processed in any >>> given time period. >>> >>> If this is a single-connection rate limit, then what practical reason >>> would an administrator have to allow "more RST's to be processed in >>> any given time period"? I can't think of a case where a legit live >>> connection would send more than one RST. >>> >> A single connection might process many RSTs because some can be out of >> window (they'd be dropped) or not at the end of the window (dropped >> under some configurations). I.e., receiving and evaluating RSTs doesn't >> always result in resetting the connection. >> >> If this is about a single >>> connection, then allowing one mid-window RST (and responding to that >>> one with a single challenge ACK) should be enough, >>> >> It would be IF the other end issued a legitimate RST. If this is a >> spoofing attack, the data can still keep coming which means that the >> window might still be moving forward by the time the next attack RST >> arrives. As a result, this sort of 'catch up' can go on until the >> connection ends. The point of the text here is to avoid wasting >> resources in such cases. >> >>> in most cases, to >>> send a challenge ACK that elicits a proper RST from the remote >>> endpoint. It's not clear to me what the motivation would be for >>> intentionally sending a challenge ACK in response to more than one >>> mid-window RST in a single connection. >>> >> Protecting against that case is the motivation *of the entire document*. >> >> Thus AFAICT "allowing more RST's to be processed in any given time >>> period" suggests that the rate limit is across connections. >>> >> >> There's no benefit to doing this across connections at all, unless you >> think that there's some reason that legitimate connections shouldn't be >> reset by valid RSTs (there's no benefit to that). >> >> Second: >>> >>> The time limit SHOULD be tunable to help timeout brute force attack= s >>> faster than a potential legitimate flood of RSTs. >>> >>> I may be missing something, but if this rate-limiting mechanism is >>> only about a single connection, then what is a "legitimate flood of >>> RSTs"? >>> >> A sequence of RSTs that pick different values, as described in RFC 4953. >> >> Or a sequence of RSTs aimed at responses to the ACK-after-RST that is >> chasing legitimate data, as described above. >> >> Why would it be important for a single connection to allow a >>> "legitimate flood of RSTs" destinted for itself? >>> >> It wouldn't - that's the point of this text. You should treat a flood of >> RSTs for a single connection as an attack and throttle them accordingly. >> >> A single legitimate >>> live connection should not send out a flood of RSTs; it should send >>> out one. Once there is one legitimate RST that arrives and is >>> processed, the receiver deletes the connection, and any other arriving >>> RSTs don't match any existing connection, so are ignored and >>> dropped, in which case is no challenge ACK to rate-limit. >>> >> Again, the point of this document are attacks, not legitimate connection= s. >> >>> By contrast, if this rate limit applies across connections then "a >>> potential legitimate flood of RSTs" makes sense, since a group of >>> legitimate live connections that all close in a burst may, in certain >>> cases, generate a legitimate flood of RSTs. So tuning the limit >>> upward would make sense. >>> >> >> There's no case I can imagine where a set of attacks on one connection >> should be cause for concern on another necessarily. >> >> Yes, there's no real need to set per-connection limits (i.e., a global >> per-connection limit seems fine), but counting them as a group serves no >> purpose against any threat, unless you think that "if I'm attacked on >> one connection, I will be attacked on another" - but that's an incorrect >> inference IMO. >> >> >>> Sorry if I'm missing something obvious. But it seems like there are at >>> least real ambiguities there, on a point significant enough to be worth >>> clearing up. >>> >> >> I think you're missing the point of the document, but that's just my >> interpretation of the exchange. >> >> Joe >> >> _______________________________________________ >> tcpm mailing list >> tcpm@ietf.org >> https://www.ietf.org/mailman/listinfo/tcpm >> >> --001a113f6d5e04a744053a33e374 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable


On Tue, Aug 16, 2016 at 6:07 AM, Mirja K=C3=BChlewind <m= irja.kuehlewind@tik.ee.ethz.ch> wrote:
Hi all,

let me try to wrap this up:

To me this does not seem to be a case for an errata. For sure some clarific= ation seems to be helpful here but that's not what erratas are for.

Therefore, I would suggest to simply set this into "Held for Document = Update" state such that it is noticed in case we decide to update the = document at some point. Does that make sense?
sounds f= ine with me but do documents really get updated?
=C2=A0

Thanks,
Mirja




On 12.08.2016 22:19, Joe Touch wrote:


On 8/12/2016 12:17 PM, Neal Cardwell wrote:
On Fri, Aug 12, 2016 at 9:54 AM, Yuchung Cheng <ycheng@google.com> wrote:
I think an errata on recommending per-connection limit and cite the
research finding is worthy.
I agree that it's worthwhile to have an errata to clarify that a
per-connection limit is best. At the very least there seems
some risky ambiguity here.

I disagree, as per below...

In fact, AFAICT RFC 5961, Section 7, page 13 --

=C2=A0 =C2=A0https://tools.ietf.org/html/rfc5961#= section-7

-- has several passages that seem to implicitly suggest that
the rate-limiting mechanism it specifies as a SHOULD is not
per-connection but rather global.

There's plenty of benefit to setting a single configuration for an
entire system vs. setting per-connection limits. That's completely
different from whether the ACKs are tallied globally or per connection.

First:

=C2=A0 =C2=A0 ... an
=C2=A0 =C2=A0 administrator who is more interested in faster cleanup of sta= le
=C2=A0 =C2=A0 connections (i.e., concerned about excess TCP state) may deci= de to
=C2=A0 =C2=A0 set a higher value thus allowing more RST's to be process= ed in any
=C2=A0 =C2=A0 given time period.

If this is a single-connection rate limit, then what practical reason
would an administrator have to allow "more RST's to be processed i= n
any given time period"? I can't think of a case where a legit live=
connection would send more than one RST.
A single connection might process many RSTs because some can be out of
window (they'd be dropped) or not at the end of the window (dropped
under some configurations). I.e., receiving and evaluating RSTs doesn't=
always result in resetting the connection.

If this is about a single
connection, then allowing one mid-window RST (and responding to that
one with a single challenge ACK) should be enough,
It would be IF the other end issued a legitimate RST. If this is a
spoofing attack, the data can still keep coming which means that the
window might still be moving forward by the time the next attack RST
arrives. As a result, this sort of 'catch up' can go on until the connection ends. The point of the text here is to avoid wasting
resources in such cases.
in most cases, to
send a challenge ACK that elicits a proper RST from the remote
endpoint. It's not clear to me what the motivation would be for
intentionally sending a challenge ACK in response to more than one
mid-window RST in a single connection.
Protecting against that case is the motivation *of the entire document*.
Thus AFAICT "allowing more RST's to be processed in any given time=
period" suggests that the rate limit is across connections.

There's no benefit to doing this across connections at all, unless you<= br> think that there's some reason that legitimate connections shouldn'= t be
reset by valid RSTs (there's no benefit to that).

Second:

=C2=A0 =C2=A0 The time limit SHOULD be tunable to help timeout brute force = attacks
=C2=A0 =C2=A0 faster than a potential legitimate flood of RSTs.

I may be missing something, but if this rate-limiting mechanism is
only about a single connection, then what is a "legitimate flood of RSTs"?
A sequence of RSTs that pick different values, as described in RFC 4953.
Or a sequence of RSTs aimed at responses to the ACK-after-RST that is
chasing legitimate data, as described above.

=C2=A0 Why would it be important for a single connection to allow a
"legitimate flood of RSTs" destinted for itself?
It wouldn't - that's the point of this text. You should treat a flo= od of
RSTs for a single connection as an attack and throttle them accordingly.
A single legitimate
live connection should not send out a flood of RSTs; it should send
out one. Once there is one legitimate RST that arrives and is
processed, the receiver deletes the connection, and any other arriving
RSTs don't match any existing connection, so are ignored and
dropped, in which case is no challenge ACK to rate-limit.
Again, the point of this document are attacks, not legitimate connections.<= br>
By contrast, if this rate limit applies across connections then "a
potential legitimate flood of RSTs" makes sense, since a group of
legitimate live connections that all close in a burst may, in certain
cases, generate a legitimate flood of RSTs. So tuning the limit
upward would make sense.

There's no case I can imagine where a set of attacks on one connection<= br> should be cause for concern on another necessarily.

Yes, there's no real need to set per-connection limits (i.e., a global<= br> per-connection limit seems fine), but counting them as a group serves no purpose against any threat, unless you think that "if I'm attacked= on
one connection, I will be attacked on another" - but that's an inc= orrect
inference IMO.


Sorry if I'm missing something obvious. But it seems like there are at<= br> least real ambiguities there, on a point significant enough to be worth
clearing up.

I think you're missing the point of the document, but that's just m= y
interpretation of the exchange.

Joe

_______________________________________________
tcpm mailing list
tcpm@ietf.org
https://www.ietf.org/mailman/listinfo/tcpm


--001a113f6d5e04a744053a33e374-- From nobody Tue Aug 16 19:03:36 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 367D812D17A for ; Tue, 16 Aug 2016 19:03:35 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.602 X-Spam-Level: X-Spam-Status: No, score=-2.602 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tCEy5LY_n5qQ for ; Tue, 16 Aug 2016 19:03:33 -0700 (PDT) Received: from mxout-07.mxes.net (mxout-07.mxes.net [216.86.168.182]) by ietfa.amsl.com (Postfix) with ESMTP id 0E58512D179 for ; Tue, 16 Aug 2016 19:03:33 -0700 (PDT) Received: from [192.168.3.100] (unknown [124.189.98.244]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by smtp.mxes.net (Postfix) with ESMTPSA id BB48722E255; Tue, 16 Aug 2016 22:03:01 -0400 (EDT) From: Mark Nottingham Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Date: Wed, 17 Aug 2016 12:02:57 +1000 Message-Id: <0CC24FC1-37E1-4125-9627-05726A9D9406@mnot.net> To: tcpm@ietf.org Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) X-Mailer: Apple Mail (2.3124) Archived-At: Cc: Patrick McManus , Daniel Stenberg Subject: [tcpm] TCP Tuning for HTTP - update X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Aug 2016 02:03:35 -0000 Hi TCPM, Just a quick note; Daniel and Tim have made an update to the TCP Tuning = for HTTP draft: https://tools.ietf.org/html/draft-stenberg-httpbis-tcp We've had a Call for Adoption open for this draft for a while, and will = likely adopt it soon. However, we'd like to get feedback from this = community first -- both about the latest version of the input document, = and to see if there's interest in helping out. You can give feedback on the HTTP WG mailing list = , or by responding = to this e-mail (Please leave the CC line; Patrick and I will try to = summarise the feedback to the WG). Cheers, -- Mark Nottingham https://www.mnot.net/ From nobody Tue Aug 16 21:21:45 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3914912D5AE for ; Tue, 16 Aug 2016 21:21:43 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -8.167 X-Spam-Level: X-Spam-Status: No, score=-8.167 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-1.247] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7_3vioqUXVah for ; Tue, 16 Aug 2016 21:21:41 -0700 (PDT) Received: from boreas.isi.edu (boreas.isi.edu [128.9.160.161]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 26EA312D126 for ; Tue, 16 Aug 2016 21:21:41 -0700 (PDT) Received: from [192.168.1.189] (cpe-172-250-251-17.socal.res.rr.com [172.250.251.17]) (authenticated bits=0) by boreas.isi.edu (8.13.8/8.13.8) with ESMTP id u7H4LAoU025750 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Tue, 16 Aug 2016 21:21:12 -0700 (PDT) To: Mark Nottingham , tcpm@ietf.org, HTTP Working Group References: <0CC24FC1-37E1-4125-9627-05726A9D9406@mnot.net> From: Joe Touch Message-ID: <7fa95741-ac58-3183-1b92-238bd4b4dae6@isi.edu> Date: Tue, 16 Aug 2016 21:21:06 -0700 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: <0CC24FC1-37E1-4125-9627-05726A9D9406@mnot.net> Content-Type: multipart/mixed; boundary="------------B84BA9D3A06B2BC51706E71F" X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Archived-At: Cc: Patrick McManus , Daniel Stenberg Subject: Re: [tcpm] TCP Tuning for HTTP - update X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Aug 2016 04:21:43 -0000 This is a multi-part message in MIME format. --------------B84BA9D3A06B2BC51706E71F Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit Hi, Mark, et al., I posted a review of this document to both to TCPM and HTTP WGs. This update fails to address the issues I raised - notably that many of the issues therein are known *and published*. So first, can we discuss the issue of PLAGIARISM? Not only of two of my works, but of many others that pointed out most of the information summarized in this doc. Second, the step of "adoption" needs to wait until there's something new here that wasn't known 20 years ago and the issue of plagiarism is addressed. Joe On 8/16/2016 7:02 PM, Mark Nottingham wrote: > Hi TCPM, > > Just a quick note; Daniel and Tim have made an update to the TCP Tuning for HTTP draft: > https://tools.ietf.org/html/draft-stenberg-httpbis-tcp > > We've had a Call for Adoption open for this draft for a while, and will likely adopt it soon. However, we'd like to get feedback from this community first -- both about the latest version of the input document, and to see if there's interest in helping out. > > You can give feedback on the HTTP WG mailing list , or by responding to this e-mail (Please leave the CC line; Patrick and I will try to summarise the feedback to the WG). > > Cheers, > > -- > Mark Nottingham https://www.mnot.net/ > > > > > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm --------------B84BA9D3A06B2BC51706E71F Content-Type: message/rfc822; name="Attached Message" Content-Transfer-Encoding: 8bit Content-Disposition: attachment; filename="Attached Message" Return-Path: X-Spam-Checker-Version: SpamAssassin 3.1.0 (2005-09-13) on boreas.isi.edu X-Spam-Level: X-Spam-Status: No, score=-2.4 required=4.0 tests=ALL_TRUSTED,AWL,BAYES_00, DNS_FROM_AHBL_RHSBL,RATWARE_GECKO_BUILD autolearn=no version=3.1.0 Received: from vapor.isi.edu (vapor.isi.edu [128.9.64.64]) by boreas.isi.edu (8.13.8/8.13.8) with ESMTP id u22MYuoj005474 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT) for ; Wed, 2 Mar 2016 14:34:56 -0800 (PST) Received: from [128.9.184.68] ([128.9.184.68]) (authenticated bits=0) by vapor.isi.edu (8.13.8/8.13.8) with ESMTP id u22MYfIv005024 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Wed, 2 Mar 2016 14:34:42 -0800 (PST) Subject: Fwd: Re: [tcpm] FW: Call for Adoption: TCP Tuning for HTTP Cc: touch@isi.edu References: <56D74C23.5010705@isi.edu> From: Joe Touch To: ietf-http-wg@w3.org X-Forwarded-Message-Id: <56D74C23.5010705@isi.edu> Message-ID: <56D76A7E.7090507@isi.edu> Date: Wed, 2 Mar 2016 14:34:38 -0800 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.6.0 MIME-Version: 1.0 In-Reply-To: <56D74C23.5010705@isi.edu> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit X-ISI-4-43-8-MailScanner: Found to be clean, Found to be clean X-MailScanner-From: touch@isi.edu Hi, all, This doc was noted on the TCPM list. See my observations below. Joe -------- Forwarded Message -------- Subject: Re: [tcpm] FW: Call for Adoption: TCP Tuning for HTTP Date: Wed, 2 Mar 2016 12:25:07 -0800 From: Joe Touch To: Scharf, Michael (Nokia - DE) , tcpm@ietf.org Extensions CC: touch@isi.edu On 3/2/2016 1:39 AM, Scharf, Michael (Nokia - DE) wrote: > I assume this could be of interest to the TCPM community. I have doubts: - it reads like a Linux manual page All Linux-specific references and commands would need to be moved to an appendix to be useful as an RFC. - this repeats (sometimes correctly, sometimes in error) existing advice J. Heidemann, K. Obraczka, J. Touch, Modeling the Performance of HTTP Over Several Transport Protocols, IEEE/ACM Transactions on Networking, V5, N5, Oct. 1997, pp.616-630. T. Faber, J. Touch, and W. Yue, The TIME-WAIT state in TCP and Its Effect on Busy Servers, in Proc. IEEE Infocom, 1999, pp. 1573-1583. - it has significant errors TIME-WAIT issues apply to servers, not clients. Nagle has been known to perform poorly for multibyte interactive traffic for a very long time, including not only web traffic but also multi-byte character or keyboard signals. Disabling slow-start after idle is safe only with pacing. Without pacing, the resulting traffic can generate a burst that was never experienced and result in both poor performance for the current connection and potential impact to competing traffic. (those are just a few) Overall, I think a man page might be useful, but this summary isn't useful for the IETF. Joe > -----Original Message----- > From: Scharf, Michael (Nokia - DE) > Sent: Wednesday, March 02, 2016 10:37 AM > To: 'Mark Nottingham'; HTTP WG > Cc: amankin@verisign.com; Daniel Stenberg > Subject: RE: Call for Adoption: TCP Tuning for HTTP > > The document refers to several TCPM RFCs with experimental status, e.g., in Section 3. That may have to be taken into account when heading towards BCP status. > > Michael > (TCPM co-chair) > > > -----Original Message----- > From: Mark Nottingham [mailto:mnot@mnot.net] > Sent: Wednesday, March 02, 2016 6:47 AM > To: HTTP WG > Cc: amankin@verisign.com; Daniel Stenberg > Subject: Call for Adoption: TCP Tuning for HTTP > > [ copying Alison as our Transport Tech Advisor ] > > Daniel has kindly started a document about how HTTP uses TCP, both for /1 and /2: > > > We haven't explicitly discussed this at a meeting, but I have heard interest in this topic from a variety of folks. > > What do people think about adopting this with a target of Best Current Practice? > > Please comment on-list. > > Regards, > > -- > Mark Nottingham https://www.mnot.net/ > > > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm > --------------B84BA9D3A06B2BC51706E71F Content-Type: message/rfc822; name="Attached Message" Content-Transfer-Encoding: 8bit Content-Disposition: attachment; filename="Attached Message" Return-Path: X-Spam-Checker-Version: SpamAssassin 3.1.0 (2005-09-13) on boreas.isi.edu X-Spam-Level: X-Spam-Status: No, score=-2.5 required=4.0 tests=ALL_TRUSTED,BAYES_00, DNS_FROM_AHBL_RHSBL,RATWARE_GECKO_BUILD autolearn=no version=3.1.0 Received: from [128.9.184.68] ([128.9.184.68]) (authenticated bits=0) by boreas.isi.edu (8.13.8/8.13.8) with ESMTP id u22KPAAC016052 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Wed, 2 Mar 2016 12:25:10 -0800 (PST) Subject: Re: [tcpm] FW: Call for Adoption: TCP Tuning for HTTP To: "Scharf, Michael (Nokia - DE)" , "tcpm@ietf.org Extensions" References: <655C07320163294895BBADA28372AF5D486E00B7@FR712WXCHMBA15.zeu.alcatel-lucent.com> Cc: touch@isi.edu From: Joe Touch Message-ID: <56D74C23.5010705@isi.edu> Date: Wed, 2 Mar 2016 12:25:07 -0800 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.6.0 MIME-Version: 1.0 In-Reply-To: <655C07320163294895BBADA28372AF5D486E00B7@FR712WXCHMBA15.zeu.alcatel-lucent.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu On 3/2/2016 1:39 AM, Scharf, Michael (Nokia - DE) wrote: > I assume this could be of interest to the TCPM community. I have doubts: - it reads like a Linux manual page All Linux-specific references and commands would need to be moved to an appendix to be useful as an RFC. - this repeats (sometimes correctly, sometimes in error) existing advice J. Heidemann, K. Obraczka, J. Touch, Modeling the Performance of HTTP Over Several Transport Protocols, IEEE/ACM Transactions on Networking, V5, N5, Oct. 1997, pp.616-630. T. Faber, J. Touch, and W. Yue, The TIME-WAIT state in TCP and Its Effect on Busy Servers, in Proc. IEEE Infocom, 1999, pp. 1573-1583. - it has significant errors TIME-WAIT issues apply to servers, not clients. Nagle has been known to perform poorly for multibyte interactive traffic for a very long time, including not only web traffic but also multi-byte character or keyboard signals. Disabling slow-start after idle is safe only with pacing. Without pacing, the resulting traffic can generate a burst that was never experienced and result in both poor performance for the current connection and potential impact to competing traffic. (those are just a few) Overall, I think a man page might be useful, but this summary isn't useful for the IETF. Joe > -----Original Message----- > From: Scharf, Michael (Nokia - DE) > Sent: Wednesday, March 02, 2016 10:37 AM > To: 'Mark Nottingham'; HTTP WG > Cc: amankin@verisign.com; Daniel Stenberg > Subject: RE: Call for Adoption: TCP Tuning for HTTP > > The document refers to several TCPM RFCs with experimental status, e.g., in Section 3. That may have to be taken into account when heading towards BCP status. > > Michael > (TCPM co-chair) > > > -----Original Message----- > From: Mark Nottingham [mailto:mnot@mnot.net] > Sent: Wednesday, March 02, 2016 6:47 AM > To: HTTP WG > Cc: amankin@verisign.com; Daniel Stenberg > Subject: Call for Adoption: TCP Tuning for HTTP > > [ copying Alison as our Transport Tech Advisor ] > > Daniel has kindly started a document about how HTTP uses TCP, both for /1 and /2: > > > We haven't explicitly discussed this at a meeting, but I have heard interest in this topic from a variety of folks. > > What do people think about adopting this with a target of Best Current Practice? > > Please comment on-list. > > Regards, > > -- > Mark Nottingham https://www.mnot.net/ > > > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm > --------------B84BA9D3A06B2BC51706E71F-- From nobody Tue Aug 16 21:43:29 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 486F012B057 for ; Tue, 16 Aug 2016 21:43:28 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.602 X-Spam-Level: X-Spam-Status: No, score=-2.602 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id NIM3IgQdgjO2 for ; Tue, 16 Aug 2016 21:43:26 -0700 (PDT) Received: from mxout-07.mxes.net (mxout-07.mxes.net [216.86.168.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 01F7612B040 for ; Tue, 16 Aug 2016 21:43:25 -0700 (PDT) Received: from [192.168.3.100] (unknown [124.189.98.244]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by smtp.mxes.net (Postfix) with ESMTPSA id 0756122E1F3; Wed, 17 Aug 2016 00:43:18 -0400 (EDT) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) From: Mark Nottingham In-Reply-To: <7fa95741-ac58-3183-1b92-238bd4b4dae6@isi.edu> Date: Wed, 17 Aug 2016 14:43:15 +1000 Content-Transfer-Encoding: quoted-printable Message-Id: <5CD67877-19E3-4E79-BBF2-3E270343A378@mnot.net> References: <0CC24FC1-37E1-4125-9627-05726A9D9406@mnot.net> <7fa95741-ac58-3183-1b92-238bd4b4dae6@isi.edu> To: Joe Touch X-Mailer: Apple Mail (2.3124) Archived-At: Cc: tcpm@ietf.org, HTTP Working Group , Daniel Stenberg , Patrick McManus Subject: Re: [tcpm] TCP Tuning for HTTP - update X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Aug 2016 04:43:28 -0000 Joe, > On 17 Aug 2016, at 2:21 PM, Joe Touch wrote: >=20 > Hi, Mark, et al., >=20 > I posted a review of this document to both to TCPM and HTTP WGs. >=20 > This update fails to address the issues I raised - notably that many = of > the issues therein are known *and published*. I'm assuming you're talking about = . > So first, can we discuss the issue of PLAGIARISM? Sure, let's discuss it. That's a very serious accusation. Are you saying = that your material has been intentionally used without proper = acknowledgement? Personally, I doubt that. What may have happened is that the text = brushes up against things that you've written about in the past, and you = feel that you're not adequately acknowledged.=20 If that's the case, I'd observe that the IETF isn't an academic = publisher, and acknowledging all prior work in an area is neither = practical, nor required, nor current practice. On the other hand, if it turns out that directing readers toward other = documents (including yours) adds value, a reference might make sense. > Not only of two of my works, but of many others that pointed out most = of > the information summarized in this doc. >=20 > Second, the step of "adoption" needs to wait until there's something = new > here that wasn't known 20 years ago and the issue of plagiarism is > addressed. Other people in the HTTP *and* TCP communities have commented that such = a document would be very useful, whether or not it's something "new that = wasn't known 20 years ago".=20 Cheers, >=20 > Joe >=20 > On 8/16/2016 7:02 PM, Mark Nottingham wrote: >> Hi TCPM, >>=20 >> Just a quick note; Daniel and Tim have made an update to the TCP = Tuning for HTTP draft: >> https://tools.ietf.org/html/draft-stenberg-httpbis-tcp >>=20 >> We've had a Call for Adoption open for this draft for a while, and = will likely adopt it soon. However, we'd like to get feedback from this = community first -- both about the latest version of the input document, = and to see if there's interest in helping out. >>=20 >> You can give feedback on the HTTP WG mailing list = , or by responding = to this e-mail (Please leave the CC line; Patrick and I will try to = summarise the feedback to the WG). >>=20 >> Cheers, >>=20 >> -- >> Mark Nottingham https://www.mnot.net/ >>=20 >>=20 >>=20 >>=20 >> _______________________________________________ >> tcpm mailing list >> tcpm@ietf.org >> https://www.ietf.org/mailman/listinfo/tcpm >=20 > -- Mark Nottingham https://www.mnot.net/ From nobody Tue Aug 16 22:23:40 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 73ECC12D757 for ; Tue, 16 Aug 2016 22:23:39 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -8.167 X-Spam-Level: X-Spam-Status: No, score=-8.167 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-1.247] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id BYr8cgbIzQ3S for ; Tue, 16 Aug 2016 22:23:38 -0700 (PDT) Received: from boreas.isi.edu (boreas.isi.edu [128.9.160.161]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 34E8B126579 for ; Tue, 16 Aug 2016 22:23:38 -0700 (PDT) Received: from [192.168.1.189] (cpe-172-250-251-17.socal.res.rr.com [172.250.251.17]) (authenticated bits=0) by boreas.isi.edu (8.13.8/8.13.8) with ESMTP id u7H5N4Pv005787 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Tue, 16 Aug 2016 22:23:06 -0700 (PDT) To: Mark Nottingham References: <0CC24FC1-37E1-4125-9627-05726A9D9406@mnot.net> <7fa95741-ac58-3183-1b92-238bd4b4dae6@isi.edu> <5CD67877-19E3-4E79-BBF2-3E270343A378@mnot.net> From: Joe Touch Message-ID: <2197232f-10d7-28cb-fcc9-05bd495e3c22@isi.edu> Date: Tue, 16 Aug 2016 22:23:01 -0700 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: <5CD67877-19E3-4E79-BBF2-3E270343A378@mnot.net> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Archived-At: Cc: tcpm@ietf.org, HTTP Working Group , Daniel Stenberg , Patrick McManus Subject: Re: [tcpm] TCP Tuning for HTTP - update X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Aug 2016 05:23:39 -0000 Hi, Mark, On 8/16/2016 9:43 PM, Mark Nottingham wrote: > Joe, > >> On 17 Aug 2016, at 2:21 PM, Joe Touch wrote: >> >> Hi, Mark, et al., >> >> I posted a review of this document to both to TCPM and HTTP WGs. >> >> This update fails to address the issues I raised - notably that many of >> the issues therein are known *and published*. > I'm assuming you're talking about . Yes. > > >> So first, can we discuss the issue of PLAGIARISM? > Sure, let's discuss it. That's a very serious accusation. Are you saying that your material has been intentionally used without proper acknowledgement? Yes. > Personally, I doubt that. What may have happened is that the text brushes up against things that you've written about in the past, and you feel that you're not adequately acknowledged. Plagiarism requires only that the material was published elsewhere before. Intent has no bearing. In addition, I informed the author - and both lists - about this over 5 months ago. You might claim that the first two versions were issued out of ignorance, but you cannot claim that of the update. > If that's the case, I'd observe that the IETF isn't an academic publisher, and acknowledging all prior work in an area is neither practical, nor required, nor current practice. Plagiarism isn't an issue limited to academic environments. Publication of a document on the web is still publication. > On the other hand, if it turns out that directing readers toward other documents (including yours) adds value, a reference might make sense. Those docs explain the issues more correctly and in more detail. That should add enough value. The real question is whether this draft adds value to those - which are *already published*. > >> Not only of two of my works, but of many others that pointed out most of >> the information summarized in this doc. >> >> Second, the step of "adoption" needs to wait until there's something new >> here that wasn't known 20 years ago and the issue of plagiarism is >> addressed. > Other people in the HTTP *and* TCP communities have commented that such a document would be very useful, whether or not it's something "new that wasn't known 20 years ago". We don't need to issue new documents for people who don't read old ones. Joe > > Cheers, > > >> Joe >> >> On 8/16/2016 7:02 PM, Mark Nottingham wrote: >>> Hi TCPM, >>> >>> Just a quick note; Daniel and Tim have made an update to the TCP Tuning for HTTP draft: >>> https://tools.ietf.org/html/draft-stenberg-httpbis-tcp >>> >>> We've had a Call for Adoption open for this draft for a while, and will likely adopt it soon. However, we'd like to get feedback from this community first -- both about the latest version of the input document, and to see if there's interest in helping out. >>> >>> You can give feedback on the HTTP WG mailing list , or by responding to this e-mail (Please leave the CC line; Patrick and I will try to summarise the feedback to the WG). >>> >>> Cheers, >>> >>> -- >>> Mark Nottingham https://www.mnot.net/ >>> >>> >>> >>> >>> _______________________________________________ >>> tcpm mailing list >>> tcpm@ietf.org >>> https://www.ietf.org/mailman/listinfo/tcpm >> > -- > Mark Nottingham https://www.mnot.net/ > > > > From nobody Tue Aug 16 23:43:07 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7192D12B008 for ; Tue, 16 Aug 2016 23:43:05 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.602 X-Spam-Level: X-Spam-Status: No, score=-2.602 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3rHiYiIzC1hn for ; Tue, 16 Aug 2016 23:43:03 -0700 (PDT) Received: from mxout-07.mxes.net (mxout-07.mxes.net [216.86.168.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 6A691127077 for ; Tue, 16 Aug 2016 23:43:03 -0700 (PDT) Received: from [192.168.3.104] (unknown [124.189.98.244]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by smtp.mxes.net (Postfix) with ESMTPSA id 29D9922E257; Wed, 17 Aug 2016 02:42:55 -0400 (EDT) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) From: Mark Nottingham In-Reply-To: <2197232f-10d7-28cb-fcc9-05bd495e3c22@isi.edu> Date: Wed, 17 Aug 2016 16:42:53 +1000 Content-Transfer-Encoding: quoted-printable Message-Id: <87D56B16-B091-4E9C-9B88-D136ED4BC819@mnot.net> References: <0CC24FC1-37E1-4125-9627-05726A9D9406@mnot.net> <7fa95741-ac58-3183-1b92-238bd4b4dae6@isi.edu> <5CD67877-19E3-4E79-BBF2-3E270343A378@mnot.net> <2197232f-10d7-28cb-fcc9-05bd495e3c22@isi.edu> To: Joe Touch X-Mailer: Apple Mail (2.3124) Archived-At: Cc: tcpm@ietf.org, HTTP Working Group , Daniel Stenberg , Patrick McManus Subject: Re: [tcpm] TCP Tuning for HTTP - update X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Aug 2016 06:43:05 -0000 Joe, I've pinged some ADs to weigh in on this. In the meantime - > On 17 Aug 2016, at 3:23 PM, Joe Touch wrote: >=20 > Plagiarism requires only that the material was published elsewhere > before. Intent has no bearing. So to paraphrase, you're accusing Daniel of failure to cite a relevant = source out of ignorance in the first case...=20 > In addition, I informed the author - and both lists - about this over = 5 > months ago. You might claim that the first two versions were issued = out > of ignorance, but you cannot claim that of the update. ... and then continuing to do so after being notified. Correct? Looking over and = , I'm having = trouble identifying passages that are plagiarised; can you show us? In particular, the latter paper proposes TIME-WAIT negotiation using a = TCP option, sending a RST from clients, and a HTTP extension, whereas = Daniel's draft only says this about TIME_WAIT: """ 2.5. Lower the TCP FIN timeout High connection completion rates will consume ephemeral ports quickly. Lower the time during which connections are in FIN-WAIT-2/ TIME_WAIT states so that they can be purged faster and thus maintain a maximal number of available sockets. The primitives for the assignment of these values were described in [RFC0793], however significantly lower values are commonly used. 2.6. Reuse sockets in TIME_WAIT state When running backend servers on a managed, low latency network you might allow the reuse of sockets in TIME_WAIT state for new connections when a protocol complete termination has occurred. There is no RFC that covers this behaviour. """ So, where is the plagiarism? Also, how do you reconcile that with the statement in your original = e-mail that the draft repeats your material "sometimes correctly, = sometimes in error"? Have you considered that what you consider to be an = error in his plagiarism might in fact be a different idea (no matter = whose is ultimately correct)? >> If that's the case, I'd observe that the IETF isn't an academic = publisher, and acknowledging all prior work in an area is neither = practical, nor required, nor current practice. > Plagiarism isn't an issue limited to academic environments. = Publication > of a document on the web is still publication. Sure. It also isn't a legal issue in this form (unless you're asserting = copyright?). Effectively, it's a cultural norm. Again, I will point out = that in the culture of the IETF, we historically have not cited the = complete provenance of every idea, both because it's impractical and = because it doesn't benefit the reader.=20 As far as I know, the IETF does not have a stated position about what = you regard as PLAGIARISM. Hopefully we can get some clarity about that = from the ADs, as well as some definitive evidence of what you're = asserting. >> On the other hand, if it turns out that directing readers toward = other documents (including yours) adds value, a reference might make = sense. >=20 > Those docs explain the issues more correctly and in more detail. That > should add enough value. >=20 > The real question is whether this draft adds value to those - which = are > *already published*. >=20 >=20 >>=20 >>> Not only of two of my works, but of many others that pointed out = most of >>> the information summarized in this doc. >>>=20 >>> Second, the step of "adoption" needs to wait until there's something = new >>> here that wasn't known 20 years ago and the issue of plagiarism is >>> addressed. >> Other people in the HTTP *and* TCP communities have commented that = such a document would be very useful, whether or not it's something "new = that wasn't known 20 years ago".=20 >=20 > We don't need to issue new documents for people who don't read old = ones. -- Mark Nottingham https://www.mnot.net/ From nobody Tue Aug 16 23:45:56 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7B20612B008 for ; Tue, 16 Aug 2016 23:45:54 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.902 X-Spam-Level: X-Spam-Status: No, score=-1.902 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vS-w7lxNI4Or for ; Tue, 16 Aug 2016 23:45:53 -0700 (PDT) Received: from 1wt.eu (wtarreau.pck.nerim.net [62.212.114.60]) by ietfa.amsl.com (Postfix) with ESMTP id C3650127077 for ; Tue, 16 Aug 2016 23:45:51 -0700 (PDT) Received: (from willy@localhost) by pcw.home.local (8.15.2/8.15.2/Submit) id u7H6jjLW016312; Wed, 17 Aug 2016 08:45:45 +0200 Date: Wed, 17 Aug 2016 08:45:45 +0200 From: Willy Tarreau To: Joe Touch Message-ID: <20160817064545.GD16017@1wt.eu> References: <0CC24FC1-37E1-4125-9627-05726A9D9406@mnot.net> <7fa95741-ac58-3183-1b92-238bd4b4dae6@isi.edu> <5CD67877-19E3-4E79-BBF2-3E270343A378@mnot.net> <2197232f-10d7-28cb-fcc9-05bd495e3c22@isi.edu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <2197232f-10d7-28cb-fcc9-05bd495e3c22@isi.edu> User-Agent: Mutt/1.6.0 (2016-04-01) Archived-At: Cc: Mark Nottingham , tcpm@ietf.org, HTTP Working Group , Daniel Stenberg , Patrick McManus Subject: Re: [tcpm] TCP Tuning for HTTP - update X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Aug 2016 06:45:54 -0000 Hi Joe, On Tue, Aug 16, 2016 at 10:23:01PM -0700, Joe Touch wrote: > > Other people in the HTTP *and* TCP communities have commented that such a > > document would be very useful, whether or not it's something "new that > > wasn't known 20 years ago". > > We don't need to issue new documents for people who don't read old ones. I've just checked the two documents you referenced. They seem to be very well detailed but they are *scientific* research. What Daniel created are configuration advises for people who need to configure their servers. Yes as you mentionned they look like man pages precisely because the purpose is to ensure they're easily understood by people who are just seeking some help to improve their configuration. You cannot expect a server admin to read scientific papers explaining some TCP models with some formulas to know what to do on their servers. Also, I don't know if there have been any update, but these documents use SunOS 4.1.3 running on a sparc 20 as a reference. While I used to love working on such systems 20 years ago, they predate the web era and systems have evolved a lot since to deal with high traffic. I remember by then I used to be impressed when seeing a server sustain 50 connections per second. Now when someone tells me he's limited to 100000 connections per second on a single server, I ask how they bound their interrupts to see if there's some headroom to go further. So you need to expect that only researchers and maybe TCP stack developers will find your work useful these days, server admins can hardly use this anymore. However it is very possible that some TCP stacks have taken benefit of your work to reach the level of performance they achieve right now, I don't know. Thus I think that Daniel's work completes quite well what you've done in that it directly addresses people's concerns without requiring the scientific background. Regards, Willy From nobody Wed Aug 17 08:08:26 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 608DB12DF8E for ; Wed, 17 Aug 2016 08:08:25 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -8.167 X-Spam-Level: X-Spam-Status: No, score=-8.167 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-1.247] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rcxnRLroE_xQ for ; Wed, 17 Aug 2016 08:08:23 -0700 (PDT) Received: from boreas.isi.edu (boreas.isi.edu [128.9.160.161]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 5EC5712DC3C for ; Wed, 17 Aug 2016 08:08:22 -0700 (PDT) Received: from [192.168.1.189] (cpe-172-250-251-17.socal.res.rr.com [172.250.251.17]) (authenticated bits=0) by boreas.isi.edu (8.13.8/8.13.8) with ESMTP id u7HF87LG027468 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Wed, 17 Aug 2016 08:08:09 -0700 (PDT) To: Mark Nottingham References: <0CC24FC1-37E1-4125-9627-05726A9D9406@mnot.net> <7fa95741-ac58-3183-1b92-238bd4b4dae6@isi.edu> <5CD67877-19E3-4E79-BBF2-3E270343A378@mnot.net> <2197232f-10d7-28cb-fcc9-05bd495e3c22@isi.edu> <87D56B16-B091-4E9C-9B88-D136ED4BC819@mnot.net> From: Joe Touch Message-ID: Date: Wed, 17 Aug 2016 08:08:05 -0700 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: <87D56B16-B091-4E9C-9B88-D136ED4BC819@mnot.net> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Archived-At: Cc: tcpm@ietf.org, HTTP Working Group , Daniel Stenberg , Patrick McManus Subject: Re: [tcpm] TCP Tuning for HTTP - update X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Aug 2016 15:08:25 -0000 On 8/16/2016 11:42 PM, Mark Nottingham wrote: > Joe, > > I've pinged some ADs to weigh in on this. > > In the meantime - > >> On 17 Aug 2016, at 3:23 PM, Joe Touch wrote: >> >> Plagiarism requires only that the material was published elsewhere >> before. Intent has no bearing. > So to paraphrase, you're accusing Daniel of failure to cite a relevant source out of ignorance in the first case... I'm being generous in assuming ignorance, but yes. > >> In addition, I informed the author - and both lists - about this over 5 >> months ago. You might claim that the first two versions were issued out >> of ignorance, but you cannot claim that of the update. > ... and then continuing to do so after being notified. > > Correct? Yes. > Looking over and , I'm having trouble identifying passages that are plagiarised; can you show us? It's the content - turning off Nagle, slow-start restart after idle, etc. > In particular, the latter paper proposes TIME-WAIT negotiation using a TCP option, sending a RST from clients, and a HTTP extension, It also outlines the whole problem for the first time and explains the choice and impact of lowering TW timeout. > ... > > Also, how do you reconcile that with the statement in your original e-mail that the draft repeats your material "sometimes correctly, sometimes in error"? Have you considered that what you consider to be an error in his plagiarism might in fact be a different idea (no matter whose is ultimately correct)? That might be a reasonable conclusion if there were discussion explaining why the "different idea" was correct in comparison. But that's missing. Saying to lower the timer for TW without discussing the potential impact isn't a new idea. It's an incomplete discussion of an old idea already discussed in detail elsewhere. > >>> If that's the case, I'd observe that the IETF isn't an academic publisher, and acknowledging all prior work in an area is neither practical, nor required, nor current practice. >> Plagiarism isn't an issue limited to academic environments. Publication >> of a document on the web is still publication. > Sure. It also isn't a legal issue in this form (unless you're asserting copyright?). Effectively, it's a cultural norm. Again, I will point out that in the culture of the IETF, we historically have not cited the complete provenance of every idea, both because it's impractical and because it doesn't benefit the reader. Although that's true in the smallest cases, the IETF does have two concepts that support this norm: an author list and a set of references. Can you explain how it helps the reader to not cite two documents that are both squarely in the same area as this doc (interaction between HTTP and TCP and the impact of running many small connections closed at the client as for HTTP)? > As far as I know, the IETF does not have a stated position about what you regard as PLAGIARISM. Hopefully we can get some clarity about that from the ADs, as well as some definitive evidence of what you're asserting. You can if you want, but my primary point here is to have this work corrected - and to stop the myth that "it doesn't matter" whether *reasonable* citations are included. Joe From nobody Wed Aug 17 08:15:27 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 21D0012D815 for ; Wed, 17 Aug 2016 08:15:26 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -8.167 X-Spam-Level: X-Spam-Status: No, score=-8.167 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-1.247] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id PglaA2-rSg4V for ; Wed, 17 Aug 2016 08:15:25 -0700 (PDT) Received: from boreas.isi.edu (boreas.isi.edu [128.9.160.161]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 0694B12DF91 for ; Wed, 17 Aug 2016 08:15:25 -0700 (PDT) Received: from [192.168.1.189] (cpe-172-250-251-17.socal.res.rr.com [172.250.251.17]) (authenticated bits=0) by boreas.isi.edu (8.13.8/8.13.8) with ESMTP id u7HFE9x0029345 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Wed, 17 Aug 2016 08:14:11 -0700 (PDT) To: Willy Tarreau References: <0CC24FC1-37E1-4125-9627-05726A9D9406@mnot.net> <7fa95741-ac58-3183-1b92-238bd4b4dae6@isi.edu> <5CD67877-19E3-4E79-BBF2-3E270343A378@mnot.net> <2197232f-10d7-28cb-fcc9-05bd495e3c22@isi.edu> <20160817064545.GD16017@1wt.eu> From: Joe Touch Message-ID: <7f7b129c-f156-d067-bef8-4a2213f461ac@isi.edu> Date: Wed, 17 Aug 2016 08:14:08 -0700 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: <20160817064545.GD16017@1wt.eu> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Archived-At: Cc: Mark Nottingham , tcpm@ietf.org, HTTP Working Group , Daniel Stenberg , Patrick McManus Subject: Re: [tcpm] TCP Tuning for HTTP - update X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Aug 2016 15:15:26 -0000 On 8/16/2016 11:45 PM, Willy Tarreau wrote: > Hi Joe, > > On Tue, Aug 16, 2016 at 10:23:01PM -0700, Joe Touch wrote: >>> Other people in the HTTP *and* TCP communities have commented that such a >>> document would be very useful, whether or not it's something "new that >>> wasn't known 20 years ago". >> We don't need to issue new documents for people who don't read old ones. > I've just checked the two documents you referenced. They seem to be very > well detailed but they are *scientific* research. What Daniel created are > configuration advises for people who need to configure their servers. Yes > as you mentionned they look like man pages precisely because the purpose > is to ensure they're easily understood by people who are just seeking some > help to improve their configuration. You cannot expect a server admin to > read scientific papers explaining some TCP models with some formulas to > know what to do on their servers. Actually, my docs are the tip of the iceberg, addressing the reasons why some of these configuration recommendations make sense. There are many other sites - and books - that already indicate how to configure systems efficiently. So if your argument is that a man page summary is needed, sure - but again, is a new one needed? And why is this then needed as an RFC? > Also, I don't know if there have been any update, but these documents use > SunOS 4.1.3 running on a sparc 20 as a reference. While I used to love > working on such systems 20 years ago, they predate the web era and systems > have evolved a lot since to deal with high traffic. ... Yes, and discussing those issues would be useful - but not in this document either. > ... > So you need to expect that only researchers and maybe TCP stack developers > will find your work useful these days, server admins can hardly use this > anymore. However it is very possible that some TCP stacks have taken benefit > of your work to reach the level of performance they achieve right now, I > don't know. Thus I think that Daniel's work completes quite well what you've > done in that it directly addresses people's concerns without requiring the > scientific background. Let me see if I get your complete argument: - the appropriate refs are 20 years old - server admins need a doc What exactly do server admins need regarding Nagle (which is configured inside the app already), socket sizing (configured inside the app), etc? I.e., at the most this is a man page (specific to an OS). At the least, this isn't useful at all. Joe From nobody Wed Aug 17 08:29:10 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 285D212DCEF for ; Wed, 17 Aug 2016 08:29:08 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.248 X-Spam-Level: X-Spam-Status: No, score=-3.248 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RP_MATCHES_RCVD=-1.247, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=isode.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id psQUGGoEbNeV for ; Wed, 17 Aug 2016 08:29:07 -0700 (PDT) Received: from waldorf.isode.com (waldorf.isode.com [62.232.206.188]) by ietfa.amsl.com (Postfix) with ESMTP id F2AEC12E02F for ; Wed, 17 Aug 2016 08:29:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1471447745; d=isode.com; s=june2016; i=@isode.com; bh=CWf1klbjaRPeCp7NuHMfSMOeiR0RZrj7+m8IslesOw4=; h=From:Sender:Reply-To:Subject:Date:Message-ID:To:Cc:MIME-Version: In-Reply-To:References:Content-Type:Content-Transfer-Encoding: Content-ID:Content-Description; b=r4r66WkJEqI5rkt3AEspT2hLg2LzqUaAGPrbOWvNKZceeHmCdkov9IfHp/P2R4XgUgC9Lz TeUWpP5Vu8ykh+JcXlEIzqR4fVWoHpXOYGVw8B6jFR0nJrjGe5pKBo9jZ9zt30YvbiIaTK gf07M774BQUGcs9tVtmP8g2boJeP/tk=; Received: from [172.20.1.215] (dhcp-215.isode.net [172.20.1.215]) by waldorf.isode.com (submission channel) via TCP with ESMTPSA id ; Wed, 17 Aug 2016 16:29:05 +0100 To: Joe Touch , Mark Nottingham References: <0CC24FC1-37E1-4125-9627-05726A9D9406@mnot.net> <7fa95741-ac58-3183-1b92-238bd4b4dae6@isi.edu> <5CD67877-19E3-4E79-BBF2-3E270343A378@mnot.net> <2197232f-10d7-28cb-fcc9-05bd495e3c22@isi.edu> <87D56B16-B091-4E9C-9B88-D136ED4BC819@mnot.net> From: Alexey Melnikov Message-ID: <2f276fc8-45d9-50e3-08bf-d712bc94302d@isode.com> Date: Wed, 17 Aug 2016 16:28:45 +0100 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Archived-At: Cc: tcpm@ietf.org, HTTP Working Group , Daniel Stenberg , Patrick McManus Subject: Re: [tcpm] TCP Tuning for HTTP - update X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Aug 2016 15:29:08 -0000 Joe, On 17/08/2016 16:08, Joe Touch wrote: > On 8/16/2016 11:42 PM, Mark Nottingham wrote: >>> On 17 Aug 2016, at 3:23 PM, Joe Touch wrote: >>> [snip] >>>> If that's the case, I'd observe that the IETF isn't an academic publisher, and acknowledging all prior work in an area is neither practical, nor required, nor current practice. >>> Plagiarism isn't an issue limited to academic environments. Publication >>> of a document on the web is still publication. >> Sure. It also isn't a legal issue in this form (unless you're asserting copyright?). Effectively, it's a cultural norm. Again, I will point out that in the culture of the IETF, we historically have not cited the complete provenance of every idea, both because it's impractical and because it doesn't benefit the reader. > Although that's true in the smallest cases, the IETF does have two > concepts that support this norm: an author list and a set of references. > > Can you explain how it helps the reader to not cite two documents that > are both squarely in the same area as this doc (interaction between HTTP > and TCP and the impact of running many small connections closed at the > client as for HTTP)? Instead of starting your discussion with words like "plagiarism", you could have just asked for information to be clarified and a citation/acknowledgement added? With your current introduction you pissed off lots of people. >> As far as I know, the IETF does not have a stated position about what you regard as PLAGIARISM. Hopefully we can get some clarity about that from the ADs, as well as some definitive evidence of what you're asserting. > You can if you want, but my primary point here is to have this work > corrected - and to stop the myth that "it doesn't matter" whether > *reasonable* citations are included. Noted. From nobody Wed Aug 17 08:30:06 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 837FB12E038 for ; Wed, 17 Aug 2016 08:30:04 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.149 X-Spam-Level: X-Spam-Status: No, score=-3.149 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RP_MATCHES_RCVD=-1.247, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id C3VNrjb6v4iJ for ; Wed, 17 Aug 2016 08:30:02 -0700 (PDT) Received: from kuehlewind.net (kuehlewind.net [83.169.45.111]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id E103412DCCB for ; Wed, 17 Aug 2016 08:30:01 -0700 (PDT) Received: (qmail 23951 invoked from network); 17 Aug 2016 17:30:00 +0200 Received: from nb-10510.ethz.ch (HELO ?82.130.103.143?) (82.130.103.143) by kuehlewind.net with ESMTPSA (DHE-RSA-AES128-SHA encrypted, authenticated); 17 Aug 2016 17:30:00 +0200 To: "tcpm@ietf.org Extensions" From: =?UTF-8?Q?Mirja_K=c3=bchlewind?= Message-ID: <57B482B9.5090205@kuehlewind.net> Date: Wed, 17 Aug 2016 17:28:57 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.8.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Archived-At: Cc: "tcpm-chairs@ietf.org" Subject: [tcpm] Personal change in chairing X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Aug 2016 15:30:04 -0000 Hi all, I have to announce a personal change in the tcpm chair position: Pasi Sarolahti is stepping down due to time restrictions. Pasi, thanks a lot for your work and engagement as chair! Your did a great job and we hope to still see you as often as possible as contributor in the working group! Michael Tüxen is starting as a new tcpm chair. Micheal, great to have you on broad! We look forward to work with you! Mirja, as responsible AD From nobody Wed Aug 17 08:55:05 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0956C12D89C for ; Wed, 17 Aug 2016 08:55:03 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.147 X-Spam-Level: X-Spam-Status: No, score=-3.147 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RP_MATCHES_RCVD=-1.247] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id WyCm4Oe1GBIk for ; Wed, 17 Aug 2016 08:55:01 -0700 (PDT) Received: from nitro.isi.edu (nitro.isi.edu [128.9.208.207]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 76FA412D88E for ; Wed, 17 Aug 2016 08:55:01 -0700 (PDT) Received: from [192.168.1.130] (cpe-172-250-251-17.socal.res.rr.com [172.250.251.17]) (authenticated bits=0) by nitro.isi.edu (8.13.8/8.13.8) with ESMTP id u7HFrsKN028857 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Wed, 17 Aug 2016 08:53:57 -0700 (PDT) References: <0CC24FC1-37E1-4125-9627-05726A9D9406@mnot.net> <7fa95741-ac58-3183-1b92-238bd4b4dae6@isi.edu> <5CD67877-19E3-4E79-BBF2-3E270343A378@mnot.net> <2197232f-10d7-28cb-fcc9-05bd495e3c22@isi.edu> <87D56B16-B091-4E9C-9B88-D136ED4BC819@mnot.net> <2f276fc8-45d9-50e3-08bf-d712bc94302d@isode.com> In-Reply-To: <2f276fc8-45d9-50e3-08bf-d712bc94302d@isode.com> Mime-Version: 1.0 (1.0) Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Message-Id: <1D23B817-4326-4C21-ADFC-E1447BEFF031@isi.edu> X-Mailer: iPad Mail (13G35) From: Joe Touch Date: Wed, 17 Aug 2016 08:53:54 -0700 To: Alexey Melnikov X-MailScanner-ID: u7HFrsKN028857 X-ISI-4-69-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Archived-At: Cc: Mark Nottingham , tcpm@ietf.org, HTTP Working Group , Daniel Stenberg , Patrick McManus Subject: Re: [tcpm] TCP Tuning for HTTP - update X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Aug 2016 15:55:03 -0000 > On Aug 17, 2016, at 8:28 AM, Alexey Melnikov w= rote: >=20 > Joe, >=20 >> On 17/08/2016 16:08, Joe Touch wrote: >> ... >> Can you explain how it helps the reader to not cite two documents that >> are both squarely in the same area as this doc (interaction between HTTP >> and TCP and the impact of running many small connections closed at the >> client as for HTTP)? > Instead of starting your discussion with words like "plagiarism", you coul= d have just asked for information to be clarified and a citation/acknowledge= ment added? ... I did that 5 months ago, as noted in the attachments I included yesterday. Joe= From nobody Wed Aug 17 09:27:06 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4481A12DE9F for ; Wed, 17 Aug 2016 09:27:04 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -15.769 X-Spam-Level: X-Spam-Status: No, score=-15.769 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H4=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-1.247, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, USER_IN_DEF_DKIM_WL=-7.5] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=cisco.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id LZIu8zUCRLBj for ; Wed, 17 Aug 2016 09:27:03 -0700 (PDT) Received: from aer-iport-3.cisco.com (aer-iport-3.cisco.com [173.38.203.53]) (using TLSv1.2 with cipher DHE-RSA-SEED-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 7FDB412D9F5 for ; Wed, 17 Aug 2016 09:27:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=4135; q=dns/txt; s=iport; t=1471451222; x=1472660822; h=subject:to:references:cc:from:message-id:date: mime-version:in-reply-to; bh=U1ZU8DGNPrYpw3HMD9Rh1BakyMLPoHUKoD9NKroxRuk=; b=NUZIM4Ny3DW+HYmugLqyHfzCwFQ8dF2r2S35rC8dKFAtdGQkU7Cz5rK4 fpAZelFtxA8QjJg8qHGucrb3FtT8coTWGTA/pjBXmTPn+p2aDOtMReFLD +El2w3IHraRBwJcwCYfA55XgRZ8vwISu9fGL2jJ1QxDip4MLTZF0wDGKZ s=; X-Files: signature.asc : 481 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: =?us-ascii?q?A0D7AQC2j7RX/xbLJq1eDoQMKlK3KIIPg?= =?us-ascii?q?X2GHQKCIRQCAQEBAQEBAV4nhF8BBSNWEAsYKgICVwYBDAYCAQERiBytKZAYAQE?= =?us-ascii?q?BAQEBAQEBAQEBAQEBAQEBARAOiCIIgk2EHIMlgloFmUSDPoFziW2Ba4drhXOGZ?= =?us-ascii?q?4VUg3geNoM/PToyhS+BRQEBAQ?= X-IronPort-AV: E=Sophos;i="5.28,529,1464652800"; d="asc'?scan'208";a="642612116" Received: from aer-iport-nat.cisco.com (HELO aer-core-3.cisco.com) ([173.38.203.22]) by aer-iport-3.cisco.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 17 Aug 2016 16:27:00 +0000 Received: from [10.61.64.2] (ams3-vpn-dhcp2.cisco.com [10.61.64.2]) by aer-core-3.cisco.com (8.14.5/8.14.5) with ESMTP id u7HGQxep011051; Wed, 17 Aug 2016 16:27:00 GMT To: Alexey Melnikov , Joe Touch , Mark Nottingham References: <0CC24FC1-37E1-4125-9627-05726A9D9406@mnot.net> <7fa95741-ac58-3183-1b92-238bd4b4dae6@isi.edu> <5CD67877-19E3-4E79-BBF2-3E270343A378@mnot.net> <2197232f-10d7-28cb-fcc9-05bd495e3c22@isi.edu> <87D56B16-B091-4E9C-9B88-D136ED4BC819@mnot.net> <2f276fc8-45d9-50e3-08bf-d712bc94302d@isode.com> From: Eliot Lear Message-ID: Date: Wed, 17 Aug 2016 18:26:58 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: <2f276fc8-45d9-50e3-08bf-d712bc94302d@isode.com> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="DQ4muqgQV3PxbLNGhTHwp0IfqTdx9Bd8w" Archived-At: Cc: tcpm@ietf.org, HTTP Working Group , Daniel Stenberg , Patrick McManus Subject: Re: [tcpm] TCP Tuning for HTTP - update X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Aug 2016 16:27:04 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --DQ4muqgQV3PxbLNGhTHwp0IfqTdx9Bd8w Content-Type: multipart/mixed; boundary="Odbb6s7gS4EWOfN6KQnQd924hf6TVGrFI" From: Eliot Lear To: Alexey Melnikov , Joe Touch , Mark Nottingham Cc: tcpm@ietf.org, HTTP Working Group , Patrick McManus , Daniel Stenberg Message-ID: Subject: Re: [tcpm] TCP Tuning for HTTP - update References: <0CC24FC1-37E1-4125-9627-05726A9D9406@mnot.net> <7fa95741-ac58-3183-1b92-238bd4b4dae6@isi.edu> <5CD67877-19E3-4E79-BBF2-3E270343A378@mnot.net> <2197232f-10d7-28cb-fcc9-05bd495e3c22@isi.edu> <87D56B16-B091-4E9C-9B88-D136ED4BC819@mnot.net> <2f276fc8-45d9-50e3-08bf-d712bc94302d@isode.com> In-Reply-To: <2f276fc8-45d9-50e3-08bf-d712bc94302d@isode.com> --Odbb6s7gS4EWOfN6KQnQd924hf6TVGrFI Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Perhaps we can agree that the reasonable course of action here is for Joe to (re)-recommend a compact set of citations to the authors, perhaps even in some easily consumable form to them (kramdown-2629 or XML)? Eliot On 8/17/16 5:28 PM, Alexey Melnikov wrote: > Joe, > > On 17/08/2016 16:08, Joe Touch wrote: >> On 8/16/2016 11:42 PM, Mark Nottingham wrote: >>>> On 17 Aug 2016, at 3:23 PM, Joe Touch wrote: >>>> > [snip] >>>>> If that's the case, I'd observe that the IETF isn't an academic >>>>> publisher, and acknowledging all prior work in an area is neither >>>>> practical, nor required, nor current practice. >>>> Plagiarism isn't an issue limited to academic environments. >>>> Publication >>>> of a document on the web is still publication. >>> Sure. It also isn't a legal issue in this form (unless you're >>> asserting copyright?). Effectively, it's a cultural norm. Again, I >>> will point out that in the culture of the IETF, we historically have >>> not cited the complete provenance of every idea, both because it's >>> impractical and because it doesn't benefit the reader. >> Although that's true in the smallest cases, the IETF does have two >> concepts that support this norm: an author list and a set of reference= s. >> >> Can you explain how it helps the reader to not cite two documents that= >> are both squarely in the same area as this doc (interaction between HT= TP >> and TCP and the impact of running many small connections closed at the= >> client as for HTTP)? > Instead of starting your discussion with words like "plagiarism", you > could have just asked for information to be clarified and a > citation/acknowledgement added? With your current introduction you > pissed off lots of people. >>> As far as I know, the IETF does not have a stated position about >>> what you regard as PLAGIARISM. Hopefully we can get some clarity >>> about that from the ADs, as well as some definitive evidence of what >>> you're asserting. >> You can if you want, but my primary point here is to have this work >> corrected - and to stop the myth that "it doesn't matter" whether >> *reasonable* citations are included. > Noted. > > > --Odbb6s7gS4EWOfN6KQnQd924hf6TVGrFI-- --DQ4muqgQV3PxbLNGhTHwp0IfqTdx9Bd8w Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2 iQEcBAEBCAAGBQJXtJBTAAoJEIe2a0bZ0noz7/QIAJ4R4Xl+L7qbbeabCKt/RulR PyMdm1uEdAKYJNiO2yfkCb5lFXA6L2GmHYc2l3SSbkWl3t0bVfwYGD9nj7co3IVf 9MSFk3CwknZo2yrChatb6E38sv6kMity3rrHsJUnxWzE0cyqW/ps1Vtc4PB3IiOo m7iWNPpKWYk18lxpCqpwyUC/IrwPXmBedh39DzBUP7x6v2u3v/9VChUKC6mRGy6B KAhRnFESJ3/fL607hYKNQx2gxM7i4I1zZOt4j0IThHGayvbT21iOGeMubE75SRD/ kHGw7y+Cws3BY1b0nn+POWbppZo5lZRbahhatpAUH8J1oDjM4S+x3t0TuZ+bOAA= =2ovk -----END PGP SIGNATURE----- --DQ4muqgQV3PxbLNGhTHwp0IfqTdx9Bd8w-- From nobody Wed Aug 17 10:00:16 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CB3F712D9E0 for ; Wed, 17 Aug 2016 10:00:14 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.7 X-Spam-Level: X-Spam-Status: No, score=-2.7 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id h5nZplaXQWy1 for ; Wed, 17 Aug 2016 10:00:13 -0700 (PDT) Received: from mail-qt0-x242.google.com (mail-qt0-x242.google.com [IPv6:2607:f8b0:400d:c0d::242]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 3C71112D9D0 for ; Wed, 17 Aug 2016 10:00:13 -0700 (PDT) Received: by mail-qt0-x242.google.com with SMTP id q11so5735637qtb.2 for ; Wed, 17 Aug 2016 10:00:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding; bh=VlUDqdozY179jb4qlqjHyCuh5A4ucchuua6SlhqRImM=; b=DmR1rXnwqZXM/OgDYjvspu5DI8PZ1GTn+5+69V9fMR0ZAEfCLiksK+XeDwOZSIjYIw ZCaS2N3qKxJFXYwPaErktF2LYy2vLf8IeFwiWdcLu1cTV4Ayp+DiA23HECWEbJpye6is LLr/r09ALQjNMIVdiaTUvo3CO8O8u63PHx5S3wy0tU783Rw2Gb9YYl3U1I8UcRSrovlK L5c7VW/u17to1205kNn3l6vZ4mjbhxxuwXpOLUgcqBjvG+iEX/OtiYskLnBTsnSq++QQ mKL0qE6ZswhqDEnQvuTgyv8VVyyjb7w/Ees68kUbDuxbPIUUXsDVC++HZQ5Wh8z6TjbB NHqQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=VlUDqdozY179jb4qlqjHyCuh5A4ucchuua6SlhqRImM=; b=k82QaHi2ff1gvGvHERfn3fkP/94dfpSx5EevjivVyld7gyzrPq9PxdMf3BnGYfr2pF rz+cZ0/6DK8FXUMws3vsnwD04cqWo3z+Ykr6JokZ67+BuYLC8sb2WgNMm/V6qt8hQi3Q 6XV9l/vWIVXNdBW7rbL+pUOVonwFf5GYnDnCQOSNmRYk2GlnyZDcf28FMhc+2P2TMX86 CBOnbvyyAxnXf9cFYDbMgZTzt+Xmukm4HZAGdq46dIwKU/8dyaIKd9vOC4x3+ee4Wmq8 GfB6M0Z5XVDpB6tNOyOOOqde8fSOhmG9EI6wTnxBfp6qzgkxUq5TbYNs0Jtq+KfBNMzl sCLg== X-Gm-Message-State: AEkoouvIClXt+G5ZEUts1LEBZJGgDjRZYvcG2yJOiardrUg56+9yqQAqiAku789dKM31Vw== X-Received: by 10.200.54.185 with SMTP id a54mr46301673qtc.63.1471453212363; Wed, 17 Aug 2016 10:00:12 -0700 (PDT) Received: from twicinski-ltm.internal.salesforce.com ([204.14.236.152]) by smtp.googlemail.com with ESMTPSA id p26sm16957153qkh.28.2016.08.17.10.00.11 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 17 Aug 2016 10:00:11 -0700 (PDT) To: Eliot Lear , Alexey Melnikov , Joe Touch , Mark Nottingham References: <0CC24FC1-37E1-4125-9627-05726A9D9406@mnot.net> <7fa95741-ac58-3183-1b92-238bd4b4dae6@isi.edu> <5CD67877-19E3-4E79-BBF2-3E270343A378@mnot.net> <2197232f-10d7-28cb-fcc9-05bd495e3c22@isi.edu> <87D56B16-B091-4E9C-9B88-D136ED4BC819@mnot.net> <2f276fc8-45d9-50e3-08bf-d712bc94302d@isode.com> From: Tim Wicinski Message-ID: <23743483-d4aa-9800-eda8-84edb4bf9053@gmail.com> Date: Wed, 17 Aug 2016 13:00:12 -0400 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Archived-At: Cc: tcpm@ietf.org, HTTP Working Group , Daniel Stenberg , Patrick McManus Subject: Re: [tcpm] TCP Tuning for HTTP - update X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Aug 2016 17:00:15 -0000 That sounds like a fine idea. I'll be glad to go through those. tim On 8/17/16 12:26 PM, Eliot Lear wrote: > Perhaps we can agree that the reasonable course of action here is for > Joe to (re)-recommend a compact set of citations to the authors, perhaps > even in some easily consumable form to them (kramdown-2629 or XML)? > > Eliot > > > On 8/17/16 5:28 PM, Alexey Melnikov wrote: >> Joe, >> >> On 17/08/2016 16:08, Joe Touch wrote: >>> On 8/16/2016 11:42 PM, Mark Nottingham wrote: >>>>> On 17 Aug 2016, at 3:23 PM, Joe Touch wrote: >>>>> >> [snip] >>>>>> If that's the case, I'd observe that the IETF isn't an academic >>>>>> publisher, and acknowledging all prior work in an area is neither >>>>>> practical, nor required, nor current practice. >>>>> Plagiarism isn't an issue limited to academic environments. >>>>> Publication >>>>> of a document on the web is still publication. >>>> Sure. It also isn't a legal issue in this form (unless you're >>>> asserting copyright?). Effectively, it's a cultural norm. Again, I >>>> will point out that in the culture of the IETF, we historically have >>>> not cited the complete provenance of every idea, both because it's >>>> impractical and because it doesn't benefit the reader. >>> Although that's true in the smallest cases, the IETF does have two >>> concepts that support this norm: an author list and a set of references. >>> >>> Can you explain how it helps the reader to not cite two documents that >>> are both squarely in the same area as this doc (interaction between HTTP >>> and TCP and the impact of running many small connections closed at the >>> client as for HTTP)? >> Instead of starting your discussion with words like "plagiarism", you >> could have just asked for information to be clarified and a >> citation/acknowledgement added? With your current introduction you >> pissed off lots of people. >>>> As far as I know, the IETF does not have a stated position about >>>> what you regard as PLAGIARISM. Hopefully we can get some clarity >>>> about that from the ADs, as well as some definitive evidence of what >>>> you're asserting. >>> You can if you want, but my primary point here is to have this work >>> corrected - and to stop the myth that "it doesn't matter" whether >>> *reasonable* citations are included. >> Noted. >> >> >> > > > > > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm > From nobody Wed Aug 17 10:17:08 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 924E612DBEB for ; Wed, 17 Aug 2016 10:17:05 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -8.167 X-Spam-Level: X-Spam-Status: No, score=-8.167 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-1.247] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id iAku4oUT7DHg for ; Wed, 17 Aug 2016 10:17:03 -0700 (PDT) Received: from boreas.isi.edu (boreas.isi.edu [128.9.160.161]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id C8CCA12DBEE for ; Wed, 17 Aug 2016 10:17:03 -0700 (PDT) Received: from [128.9.184.210] ([128.9.184.210]) (authenticated bits=0) by boreas.isi.edu (8.13.8/8.13.8) with ESMTP id u7HHGFA1018244 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Wed, 17 Aug 2016 10:16:15 -0700 (PDT) To: Tim Wicinski , Eliot Lear , Alexey Melnikov , Mark Nottingham References: <0CC24FC1-37E1-4125-9627-05726A9D9406@mnot.net> <7fa95741-ac58-3183-1b92-238bd4b4dae6@isi.edu> <5CD67877-19E3-4E79-BBF2-3E270343A378@mnot.net> <2197232f-10d7-28cb-fcc9-05bd495e3c22@isi.edu> <87D56B16-B091-4E9C-9B88-D136ED4BC819@mnot.net> <2f276fc8-45d9-50e3-08bf-d712bc94302d@isode.com> <23743483-d4aa-9800-eda8-84edb4bf9053@gmail.com> From: Joe Touch Message-ID: <5ae1ef70-d21d-ae8d-4fc4-c5112dc8495b@isi.edu> Date: Wed, 17 Aug 2016 10:16:13 -0700 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: <23743483-d4aa-9800-eda8-84edb4bf9053@gmail.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Archived-At: Cc: tcpm@ietf.org, HTTP Working Group , Daniel Stenberg , Patrick McManus Subject: Re: [tcpm] TCP Tuning for HTTP - update X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Aug 2016 17:17:07 -0000 All, It sounds like the job of an author or a search engine (or both). I've given you a place to start. Joe On 8/17/2016 10:00 AM, Tim Wicinski wrote: > > That sounds like a fine idea. I'll be glad to go through those. > > tim > > On 8/17/16 12:26 PM, Eliot Lear wrote: >> Perhaps we can agree that the reasonable course of action here is for >> Joe to (re)-recommend a compact set of citations to the authors, perhaps >> even in some easily consumable form to them (kramdown-2629 or XML)? >> >> Eliot >> >> >> On 8/17/16 5:28 PM, Alexey Melnikov wrote: >>> Joe, >>> >>> On 17/08/2016 16:08, Joe Touch wrote: >>>> On 8/16/2016 11:42 PM, Mark Nottingham wrote: >>>>>> On 17 Aug 2016, at 3:23 PM, Joe Touch wrote: >>>>>> >>> [snip] >>>>>>> If that's the case, I'd observe that the IETF isn't an academic >>>>>>> publisher, and acknowledging all prior work in an area is neither >>>>>>> practical, nor required, nor current practice. >>>>>> Plagiarism isn't an issue limited to academic environments. >>>>>> Publication >>>>>> of a document on the web is still publication. >>>>> Sure. It also isn't a legal issue in this form (unless you're >>>>> asserting copyright?). Effectively, it's a cultural norm. Again, I >>>>> will point out that in the culture of the IETF, we historically have >>>>> not cited the complete provenance of every idea, both because it's >>>>> impractical and because it doesn't benefit the reader. >>>> Although that's true in the smallest cases, the IETF does have two >>>> concepts that support this norm: an author list and a set of >>>> references. >>>> >>>> Can you explain how it helps the reader to not cite two documents that >>>> are both squarely in the same area as this doc (interaction between >>>> HTTP >>>> and TCP and the impact of running many small connections closed at the >>>> client as for HTTP)? >>> Instead of starting your discussion with words like "plagiarism", you >>> could have just asked for information to be clarified and a >>> citation/acknowledgement added? With your current introduction you >>> pissed off lots of people. >>>>> As far as I know, the IETF does not have a stated position about >>>>> what you regard as PLAGIARISM. Hopefully we can get some clarity >>>>> about that from the ADs, as well as some definitive evidence of what >>>>> you're asserting. >>>> You can if you want, but my primary point here is to have this work >>>> corrected - and to stop the myth that "it doesn't matter" whether >>>> *reasonable* citations are included. >>> Noted. >>> >>> >>> >> >> >> >> >> _______________________________________________ >> tcpm mailing list >> tcpm@ietf.org >> https://www.ietf.org/mailman/listinfo/tcpm >> From nobody Wed Aug 17 11:08:14 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CD16F12D66F for ; Wed, 17 Aug 2016 11:08:11 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.902 X-Spam-Level: X-Spam-Status: No, score=-1.902 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id wATox0whPTT0 for ; Wed, 17 Aug 2016 11:08:10 -0700 (PDT) Received: from 1wt.eu (wtarreau.pck.nerim.net [62.212.114.60]) by ietfa.amsl.com (Postfix) with ESMTP id 80D0712B01C for ; Wed, 17 Aug 2016 11:08:08 -0700 (PDT) Received: (from willy@localhost) by pcw.home.local (8.15.2/8.15.2/Submit) id u7HI82nu016788; Wed, 17 Aug 2016 20:08:02 +0200 Date: Wed, 17 Aug 2016 20:08:02 +0200 From: Willy Tarreau To: Joe Touch Message-ID: <20160817180802.GA16773@1wt.eu> References: <0CC24FC1-37E1-4125-9627-05726A9D9406@mnot.net> <7fa95741-ac58-3183-1b92-238bd4b4dae6@isi.edu> <5CD67877-19E3-4E79-BBF2-3E270343A378@mnot.net> <2197232f-10d7-28cb-fcc9-05bd495e3c22@isi.edu> <20160817064545.GD16017@1wt.eu> <7f7b129c-f156-d067-bef8-4a2213f461ac@isi.edu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <7f7b129c-f156-d067-bef8-4a2213f461ac@isi.edu> User-Agent: Mutt/1.6.0 (2016-04-01) Archived-At: Cc: Mark Nottingham , tcpm@ietf.org, HTTP Working Group , Daniel Stenberg , Patrick McManus Subject: Re: [tcpm] TCP Tuning for HTTP - update X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Aug 2016 18:08:12 -0000 On Wed, Aug 17, 2016 at 08:14:08AM -0700, Joe Touch wrote: > There are many other sites - and books - that already indicate how to > configure systems efficiently. > > So if your argument is that a man page summary is needed, sure - but > again, is a new one needed? And why is this then needed as an RFC? The difference is that a man page is OS-specific while an RFC gets a unique number and serves as a reference. It can be cited in new RFCs to justify certain choices. It can receive errata. You would probably say that the current document is very linux-centric for now but I understood it as a beginning of something more generic where advices are given based on principles before sysctls and that the resulting sysctls are just examples of applications of the principles in the document. > > Also, I don't know if there have been any update, but these documents use > > SunOS 4.1.3 running on a sparc 20 as a reference. While I used to love > > working on such systems 20 years ago, they predate the web era and systems > > have evolved a lot since to deal with high traffic. ... > Yes, and discussing those issues would be useful - but not in this > document either. Why ? Lots of admins don't understand why the time_wait timeout remains at 240 seconds on Solaris with people saying "if you want to be conservative don't touch it but if you want to be modern simply shrink it to 30 seconds or so". People need to understand why advices have changed over 3 decades. > > So you need to expect that only researchers and maybe TCP stack developers > > will find your work useful these days, server admins can hardly use this > > anymore. However it is very possible that some TCP stacks have taken benefit > > of your work to reach the level of performance they achieve right now, I > > don't know. Thus I think that Daniel's work completes quite well what you've > > done in that it directly addresses people's concerns without requiring the > > scientific background. > Let me see if I get your complete argument: > > - the appropriate refs are 20 years old > - server admins need a doc > > What exactly do server admins need regarding Nagle (which is configured > inside the app already), socket sizing (configured inside the app), etc? Lots of things : - time_wait tuning (which everybody gives different advices on, I've even seen firewall vendors recommend to shrink it to one second because it allowed their product to perform better in benchmarks) - TCP timestamps: what they provide, what are the risks (some people in banking environments refuse to enable them so that they cannot be used as an oracle to help in timing attacks). - window scaling : how much is needed. - socket sizing : contrary to what you write, there's a lot of tuning on the web where people set the default buffer sizes to 16MB without understanding the impacts when dealing with many sockets - SACK : why it's better. DSACK what it adds on top of SACK. - ECN : is it needed ? does it really work ? where does it cause issues ? - SYN cookies : benefits, risks - TCP reuse/recycling : benefits, risks - dealing with buffer bloat : tradeoffs between NIC-based acceleration and pacing - what are orphans and why you should care about them in HTTP close mode - TCP fastopen : how does it work, what type of workload is improved, what are the risks (ie: do not enable socket creation without cookie by default just because you find it reduces your server load) - whether to choose a short or a large SYN backlog depending on your workload (ie: do you prefer to process everything even if the dequeuing is expensive or to drop early in order to recover fast). ... and probably many other that don't immediately come to my mind. None of these ones was a real issue 20 years ago. All of them became issues for many web server admins who just copy-paste random settings from various blogs found on the net who just copy the same stupidities over and over resulting in the same trouble being caused to each of their reader. > I.e., at the most this is a man page (specific to an OS). At the least, > this isn't useful at all. As you can see above, nothing I cited was OS-specific but only workload specific. That's why I think that an informational RFC is much more suited to this than an OS-specific man page. The OS man page may rely on the RFC to propose various tuning profiles for different workloads however. Regards, Willy From nobody Wed Aug 17 11:32:21 2016 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3BE3B12D0F0 for ; Wed, 17 Aug 2016 11:32:20 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -8.166 X-Spam-Level: X-Spam-Status: No, score=-8.166 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-1.247] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id eJQ9r-KA0OHX for ; Wed, 17 Aug 2016 11:32:17 -0700 (PDT) Received: from boreas.isi.edu (boreas.isi.edu [128.9.160.161]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 8736812B015 for ; Wed, 17 Aug 2016 11:32:17 -0700 (PDT) Received: from [128.9.184.210] ([128.9.184.210]) (authenticated bits=0) by boreas.isi.edu (8.13.8/8.13.8) with ESMTP id u7HIVZO5002274 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Wed, 17 Aug 2016 11:31:36 -0700 (PDT) To: Willy Tarreau References: <0CC24FC1-37E1-4125-9627-05726A9D9406@mnot.net> <7fa95741-ac58-3183-1b92-238bd4b4dae6@isi.edu> <5CD67877-19E3-4E79-BBF2-3E270343A378@mnot.net> <2197232f-10d7-28cb-fcc9-05bd495e3c22@isi.edu> <20160817064545.GD16017@1wt.eu> <7f7b129c-f156-d067-bef8-4a2213f461ac@isi.edu> <20160817180802.GA16773@1wt.eu> From: Joe Touch Message-ID: <4ab7c5b0-3722-1346-f481-a8d76de70034@isi.edu> Date: Wed, 17 Aug 2016 11:31:33 -0700 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: <20160817180802.GA16773@1wt.eu> Content-Type: multipart/alternative; boundary="------------99199A629C86782671B44500" X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Archived-At: Cc: Mark Nottingham , tcpm@ietf.org, HTTP Working Group , Daniel Stenberg , Patrick McManus Subject: Re: [tcpm] TCP Tuning for HTTP - update X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Aug 2016 18:32:20 -0000 This is a multi-part message in MIME format. --------------99199A629C86782671B44500 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit On 8/17/2016 11:08 AM, Willy Tarreau wrote: > On Wed, Aug 17, 2016 at 08:14:08AM -0700, Joe Touch wrote: >> There are many other sites - and books - that already indicate how to >> configure systems efficiently. >> >> So if your argument is that a man page summary is needed, sure - but >> again, is a new one needed? And why is this then needed as an RFC? > The difference is that a man page is OS-specific while an RFC gets a > unique number and serves as a reference. That means the OS-specific stuff here needs to be removed... > It can be cited in new RFCs > to justify certain choices. Hmm. Like the refs I gave could be cited in this doc to justify *its* choices? :-) ... >>> Also, I don't know if there have been any update, but these documents use >>> SunOS 4.1.3 running on a sparc 20 as a reference. While I used to love >>> working on such systems 20 years ago, they predate the web era and systems >>> have evolved a lot since to deal with high traffic. ... >> Yes, and discussing those issues would be useful - but not in this >> document either. > Why ? Lots of admins don't understand why the time_wait timeout remains > at 240 seconds on Solaris with people saying "if you want to be conservative > don't touch it but if you want to be modern simply shrink it to 30 seconds > or so". People need to understand why advices have changed over 3 decades. The advice hasn't really changed - the advice was given in the 99 ref, which includes some cases where it can still be appropriate to decrease that timer. Again, though - the 99 ref explains the implications of both decisions - why it's 240 seconds and what happens when it's lowered. > >>> So you need to expect that only researchers and maybe TCP stack developers >>> will find your work useful these days, server admins can hardly use this >>> anymore. However it is very possible that some TCP stacks have taken benefit >>> of your work to reach the level of performance they achieve right now, I >>> don't know. Thus I think that Daniel's work completes quite well what you've >>> done in that it directly addresses people's concerns without requiring the >>> scientific background. >> Let me see if I get your complete argument: >> >> - the appropriate refs are 20 years old >> - server admins need a doc >> >> What exactly do server admins need regarding Nagle (which is configured >> inside the app already), socket sizing (configured inside the app), etc? > Lots of things : > - time_wait tuning (which everybody gives different advices on, I've > even seen firewall vendors recommend to shrink it to one second because > it allowed their product to perform better in benchmarks) Again, the 99 ref gives that detail. > - TCP timestamps: what they provide, what are the risks (some people in > banking environments refuse to enable them so that they cannot be used > as an oracle to help in timing attacks). That's already covered in the security considerations of RFC 7323. How is HTTP different, if at all, from any other app? > - window scaling : how much is needed. Same issue here, same ref - how is HTTP different? > - socket sizing : contrary to what you write, there's a lot of tuning > on the web where people set the default buffer sizes to 16MB without > understanding the impacts when dealing with many sockets There's a whole book that encompasses that and some related issues: http://www.onlamp.com/pub/a/onlamp/2005/11/17/tcp_tuning.html Some advice is also given in Sec 6.3.3 of this: J. Sterbenz, J. Touch, /High Speed Networking: A Systematic Approach to High-Bandwidth Low-Latency Communication/ , Wiley, May 2001. > - SACK : why it's better. DSACK what it adds on top of SACK. That's in the SACK docs, which aren't cited. Again, how is HTTP different from any app? > - ECN : is it needed ? does it really work ? where does it cause issues ? That's in the ECN docs, which aren't cited. Again, how is HTTP different from any app? > - SYN cookies : benefits, risks That's in the RFC 4987, which at least IS cited. Again, how is HTTP different from any app? > - TCP reuse/recycling : benefits, risks Not sure what you mean here. There are a lot of docs on the issues with persistent-HTTP vs per-connection HTTP. > - dealing with buffer bloat : tradeoffs between NIC-based acceleration > and pacing Bufferbloat typically involves large *uplink* transfers and how they interact with other uplink connections. Neither TCP nor HTTP is involved in this really. > - what are orphans and why you should care about them in HTTP close mode Orphaned TCP connections or orphaned HTTP processes? > - TCP fastopen : how does it work, what type of workload is improved, > what are the risks (ie: do not enable socket creation without cookie > by default just because you find it reduces your server load) Another doc that exists. > - whether to choose a short or a large SYN backlog depending on your > workload (ie: do you prefer to process everything even if the dequeuing > is expensive or to drop early in order to recover fast). > > ... and probably many other that don't immediately come to my mind. None > of these ones was a real issue 20 years ago. See above. Many were known around that time, but weren't documented in detail (it took a while for a proper ref to SYN cookies, and the book I wrote with Sterbenz came about because we'd seen wheels being rediscovered for 15 years). > All of them became issues for > many web server admins who just copy-paste random settings from various > blogs found on the net who just copy the same stupidities over and over > resulting in the same trouble being caused to each of their reader. This doc is all over the place. If you want a doc to advise web admins, do so. But most of the items above aren't under admin control; they're buried in app and OS implementations, and most have already evolved do to the right thing. I agree that a summary of a *focused set* of these might be useful *as a review* (note that review discussions usually include lots of refs). The key question is "what is the focus": - HTTP/TCP interactions - server administration advice - ? IMO, RFCs should focus on issues specific to the protocols and their deployment - not general knowledge that exists in courses and textbooks. >> I.e., at the most this is a man page (specific to an OS). At the least, >> this isn't useful at all. > As you can see above, nothing I cited was OS-specific but only workload > specific. That's why I think that an informational RFC is much more suited > to this than an OS-specific man page. The OS man page may rely on the RFC > to propose various tuning profiles for different workloads however. You have a good point that this is general info, but OS issues are not in the scope of the IETF and there are courses and books that already provide good advice on efficiently running web (and other) servers. Joe --------------99199A629C86782671B44500 Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: 8bit



On 8/17/2016 11:08 AM, Willy Tarreau wrote:
On Wed, Aug 17, 2016 at 08:14:08AM -0700, Joe Touch wrote:

There are many other sites - and books - that already indicate how to
configure systems efficiently.

So if your argument is that a man page summary is needed, sure - but
again, is a new one needed? And why is this then needed as an RFC?

The difference is that a man page is OS-specific while an RFC gets a
unique number and serves as a reference.
That means the OS-specific stuff here needs to be removed...

It can be cited in new RFCs
to justify certain choices.
Hmm. Like the refs I gave could be cited in this doc to justify *its* choices? :-)
...

      

        

          
Also, I don't know if there have been any update, but these documents use
SunOS 4.1.3 running on a sparc 20 as a reference. While I used to love
working on such systems 20 years ago, they predate the web era and systems
have evolved a lot since to deal with high traffic. ...


        

        
Yes, and discussing those issues would be useful - but not in this
document either.


      

      
Why ? Lots of admins don't understand why the time_wait timeout remains
at 240 seconds on Solaris with people saying "if you want to be conservative
don't touch it but if you want to be modern simply shrink it to 30 seconds
or so". People need to understand why advices have changed over 3 decades.

    

    

    The advice hasn't really changed - the advice was given in the 99
    ref, which includes some cases where it can still be appropriate to
    decrease that timer.

    

    Again, though - the 99 ref explains the implications of both
    decisions - why it's 240 seconds and what happens when it's lowered.

    

    


      

        

          
So you need to expect that only researchers and maybe TCP stack developers
will find your work useful these days, server admins can hardly use this
anymore. However it is very possible that some TCP stacks have taken benefit
of your work to reach the level of performance they achieve right now, I
don't know. Thus I think that Daniel's work completes quite well what you've
done in that it directly addresses people's concerns without requiring the
scientific background.


        

        
Let me see if I get your complete argument:

    - the appropriate refs are 20 years old
    - server admins need a doc

What exactly do server admins need regarding Nagle (which is configured
inside the app already), socket sizing (configured inside the app), etc?


      

      
Lots of things : 
  - time_wait tuning (which everybody gives different advices on, I've
    even seen firewall vendors recommend to shrink it to one second because
    it allowed their product to perform better in benchmarks)

    

    Again, the 99 ref gives that detail.

    

      
  - TCP timestamps: what they provide, what are the risks (some people in
    banking environments refuse to enable them so that they cannot be used
    as an oracle to help in timing attacks).

    

    That's already covered in the security considerations of RFC 7323.
    How is HTTP different, if at all, from any other app?

    

      
  - window scaling : how much is needed.

    

    Same issue here, same ref - how is HTTP different?

    

    

      
  - socket sizing : contrary to what you write, there's a lot of tuning
    on the web where people set the default buffer sizes to 16MB without
    understanding the impacts when dealing with many sockets

    

    There's a whole book that encompasses that and some related issues:

    http://www.onlamp.com/pub/a/onlamp/2005/11/17/tcp_tuning.html

    

    Some advice is also given in Sec 6.3.3 of this:

    
    J. Sterbenz, J. Touch, High Speed Networking: A
            Systematic Approach to
            High-Bandwidth Low-Latency Communication, Wiley, May 2001.
    
    
    
    
    
    
    
    

    

    

      
  - SACK : why it's better. DSACK what it adds on top of SACK.

    

    That's in the SACK docs, which aren't cited. Again, how is HTTP
    different from any app?

    

      
  - ECN : is it needed ? does it really work ? where does it cause issues ?

    

    That's in the ECN docs, which aren't cited. Again, how is HTTP
    different from any app?
    

      
  - SYN cookies : benefits, risks

    

    That's in the RFC 4987, which at least IS cited. Again, how is HTTP
    different from any app?

    

    

      
  - TCP reuse/recycling : benefits, risks

    

    Not sure what you mean here. There are a lot of docs on the issues
    with persistent-HTTP vs per-connection HTTP.

    

    

      
  - dealing with buffer bloat : tradeoffs between NIC-based acceleration
    and pacing

    

    Bufferbloat typically involves large *uplink* transfers and how they
    interact with other uplink connections. Neither TCP nor HTTP is
    involved in this really.

    

    

      
  - what are orphans and why you should care about them in HTTP close mode

    

    Orphaned TCP connections or orphaned HTTP processes?

    

    

      
  - TCP fastopen : how does it work, what type of workload is improved,
    what are the risks (ie: do not enable socket creation without cookie
    by default just because you find it reduces your server load)

    

    

    Another doc that exists.

    

    

      
  - whether to choose a short or a large SYN backlog depending on your
    workload (ie: do you prefer to process everything even if the dequeuing
    is expensive or to drop early in order to recover fast).

... and probably many other that don't immediately come to my mind. None
of these ones was a real issue 20 years ago.

    

    

    See above. Many were known around that time, but weren't documented
    in detail (it took a while for a proper ref to SYN cookies, and the
    book I wrote with Sterbenz came about because we'd seen wheels being
    rediscovered for 15 years).

    

    

      
 All of them became issues for
many web server admins who just copy-paste random settings from various
blogs found on the net who just copy the same stupidities over and over
resulting in the same trouble being caused to each of their reader.

    

    

    This doc is all over the place.

    

    If you want a doc to advise web admins, do so. But most of the items
    above aren't under admin control; they're buried in app and OS
    implementations, and most have already evolved do to the right
    thing.

    

    I agree that a summary of a *focused set* of these might be useful
    *as a review* (note that review discussions usually include lots of
    refs). The key question is "what is the focus":

    

     - HTTP/TCP interactions

     - server administration advice

     - ?

    

    IMO, RFCs should focus on issues specific to the protocols and their
    deployment - not general knowledge that exists in courses and
    textbooks.

    

    

      

        
I.e., at the most this is a man page (specific to an OS). At the least,
this isn't useful at all.


      

      
As you can see above, nothing I cited was OS-specific but only workload
specific. That's why I think that an informational RFC is much more suited
to this than an OS-specific man page. The OS man page may rely on the RFC
to propose various tuning profiles for different workloads however.

    

    

    You have a good point that this is general info, but OS issues are
    not in the scope of the IETF and there are courses and books that
    already provide good advice on efficiently running web (and other)
    servers.

    

    Joe

    

      

    

    

  


--------------99199A629C86782671B44500--


From nobody Wed Aug 17 11:44:41 2016
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 66F4812D1A6 for ; Wed, 17 Aug 2016 11:44:40 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -6.902
X-Spam-Level: 
X-Spam-Status: No, score=-6.902 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id RyNB7gCvbfqT for ; Wed, 17 Aug 2016 11:44:38 -0700 (PDT)
Received: from smtp-fr.alcatel-lucent.com (fr-hpida-esg-02.alcatel-lucent.com [135.245.210.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 4404212B014 for ; Wed, 17 Aug 2016 11:44:38 -0700 (PDT)
Received: from fr712umx4.dmz.alcatel-lucent.com (unknown [135.245.210.45]) by Websense Email Security Gateway with ESMTPS id AD3F988F1E10E; Wed, 17 Aug 2016 18:44:32 +0000 (GMT)
Received: from fr712usmtp2.zeu.alcatel-lucent.com (fr712usmtp2.zeu.alcatel-lucent.com [135.239.2.42]) by fr712umx4.dmz.alcatel-lucent.com (GMO-o) with ESMTP id u7HIiZHw031411 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Wed, 17 Aug 2016 18:44:35 GMT
Received: from FR711WXCHHUB01.zeu.alcatel-lucent.com (fr711wxchhub01.zeu.alcatel-lucent.com [135.239.2.111]) by fr712usmtp2.zeu.alcatel-lucent.com (GMO) with ESMTP id u7HIiXUm010620 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Wed, 17 Aug 2016 20:44:34 +0200
Received: from FR712WXCHMBA15.zeu.alcatel-lucent.com ([169.254.7.108]) by FR711WXCHHUB01.zeu.alcatel-lucent.com ([135.239.2.111]) with mapi id 14.03.0195.001; Wed, 17 Aug 2016 20:44:33 +0200
From: "Scharf, Michael (Nokia - DE)" 
To: Mark Nottingham , "tcpm@ietf.org" 
Thread-Topic: [tcpm] TCP Tuning for HTTP - update
Thread-Index: AQHR+CudIob/6hMO6kqGYKe7IdcfpKBNdUvA
Date: Wed, 17 Aug 2016 18:44:33 +0000
Message-ID: <655C07320163294895BBADA28372AF5D489E64AF@FR712WXCHMBA15.zeu.alcatel-lucent.com>
References: <0CC24FC1-37E1-4125-9627-05726A9D9406@mnot.net>
In-Reply-To: <0CC24FC1-37E1-4125-9627-05726A9D9406@mnot.net>
Accept-Language: de-DE, en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [135.239.27.40]
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
Archived-At: 
Cc: Patrick McManus , Daniel Stenberg 
Subject: Re: [tcpm] TCP Tuning for HTTP - update
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Wed, 17 Aug 2016 18:44:40 -0000

Speaking as individual contributor...

* I support an *informational* document in httpbis under the constraint tha=
t the document is finally also last called in tcpm. However, I have some do=
ubts that this document has to be a BCP. Apart from the formal aspects that=
 a BCP would have to be carefully reviewed regarding consistency with TCPM =
documents, I also believe that some of the content can easily get outdated =
as TCP and operating systems evolve, so "current" seems relative. See below=
 for one example for how fast TCP and its implementations can evolve.

* There is a related discussion on a document currently discussed in LWIP d=
raft-gomez-core-tcp-constrained-node-networks, which focuses quite on the o=
pposite type of TCP deployment. I believe the IETF should handle both docum=
ents consistently (but this is a topic for our ADs). In case of LWIP, the c=
urrent AD guidance is to have a brief readouts in TCPM from time to time (m=
aybe every second meeting or so). Given that a lot if TCP implementers foll=
ow TCPM, I believe such a readout would be a good opportunity to get feedba=
ck.

* I believe TCP is not only implemented on a single operating system only ;=
-) I'd actually be interested in the roadmap for adding a more comprehensiv=
e coverage of TCP stacks, if stack-specific recommendations should be inclu=
ded e.g. in the appendix. I do not object to OS-specific examples in genera=
l. To me this seems doable in an informational document.

* I suggest to reference RFC 7414 for a more complete overview of TCP featu=
res=20

* There has recently been quite a bit of TCPM discussion on RACK, which is =
still under design but if the document wants to cover TLP it may be reasona=
ble to consider RACK. Some context can be found e.g. in https://www.ietf.or=
g/proceedings/96/slides/slides-96-tcpm-3.pdf and https://www.ietf.org/proce=
edings/96/slides/slides-96-tcpm-4.pdf . Others may know more than me.

* I offer to more review the technical content once the document has mature=
d a bit; right now I still see quite a bit of work to be completed

Michael




-----Original Message-----
From: tcpm [mailto:tcpm-bounces@ietf.org] On Behalf Of Mark Nottingham
Sent: Wednesday, August 17, 2016 4:03 AM
To: tcpm@ietf.org
Cc: Patrick McManus; Daniel Stenberg
Subject: [tcpm] TCP Tuning for HTTP - update

Hi TCPM,

Just a quick note; Daniel and Tim have made an update to the TCP Tuning for=
 HTTP draft:
  https://tools.ietf.org/html/draft-stenberg-httpbis-tcp

We've had a Call for Adoption open for this draft for a while, and will lik=
ely adopt it soon. However, we'd like to get feedback from this community f=
irst -- both about the latest version of the input document, and to see if =
there's interest in helping out.

You can give feedback on the HTTP WG mailing list , or  by responding to this e-mail (Please leave=
 the CC line; Patrick and I will try to summarise the feedback to the WG).

Cheers,

--
Mark Nottingham   https://www.mnot.net/




_______________________________________________
tcpm mailing list
tcpm@ietf.org
https://www.ietf.org/mailman/listinfo/tcpm


From nobody Wed Aug 17 14:13:30 2016
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CE4A512D6AD for ; Wed, 17 Aug 2016 14:13:27 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.902
X-Spam-Level: 
X-Spam-Status: No, score=-1.902 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bG-MvRskG6QP for ; Wed, 17 Aug 2016 14:13:25 -0700 (PDT)
Received: from 1wt.eu (wtarreau.pck.nerim.net [62.212.114.60]) by ietfa.amsl.com (Postfix) with ESMTP id 8110D12D67B for ; Wed, 17 Aug 2016 14:13:24 -0700 (PDT)
Received: (from willy@localhost) by pcw.home.local (8.15.2/8.15.2/Submit) id u7HLDH7O016960; Wed, 17 Aug 2016 23:13:17 +0200
Date: Wed, 17 Aug 2016 23:13:17 +0200
From: Willy Tarreau 
To: Joe Touch 
Message-ID: <20160817211317.GA16929@1wt.eu>
References: <0CC24FC1-37E1-4125-9627-05726A9D9406@mnot.net> <7fa95741-ac58-3183-1b92-238bd4b4dae6@isi.edu> <5CD67877-19E3-4E79-BBF2-3E270343A378@mnot.net> <2197232f-10d7-28cb-fcc9-05bd495e3c22@isi.edu> <20160817064545.GD16017@1wt.eu> <7f7b129c-f156-d067-bef8-4a2213f461ac@isi.edu> <20160817180802.GA16773@1wt.eu> <4ab7c5b0-3722-1346-f481-a8d76de70034@isi.edu>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <4ab7c5b0-3722-1346-f481-a8d76de70034@isi.edu>
User-Agent: Mutt/1.6.0 (2016-04-01)
Archived-At: 
Cc: Mark Nottingham , tcpm@ietf.org, HTTP Working Group , Daniel Stenberg , Patrick McManus 
Subject: Re: [tcpm] TCP Tuning for HTTP - update
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Wed, 17 Aug 2016 21:13:28 -0000

On Wed, Aug 17, 2016 at 11:31:33AM -0700, Joe Touch wrote:
> > It can be cited in new RFCs
> > to justify certain choices. 
> Hmm. Like the refs I gave could be cited in this doc to justify *its*
> choices? :-)

I think it would be nice that this is cited, but to be clear on one
point, I've never heard about your papers before you advertised them
here in this thread, and yet I've been dealing with timewait issues
for 15 years like many people facing moderate to large web sites
nowadays. It just happens that you probably identified these issues
very early at low connection rates, but today anyone dealing with
more than 500-1000 connections per second on a server or worse on
a gateway quickly discovers that he has to make a choice.

> >> Yes, and discussing those issues would be useful - but not in this
> >> document either.
> > Why ? Lots of admins don't understand why the time_wait timeout remains
> > at 240 seconds on Solaris with people saying "if you want to be conservative
> > don't touch it but if you want to be modern simply shrink it to 30 seconds
> > or so". People need to understand why advices have changed over 3 decades.
> 
> The advice hasn't really changed - the advice was given in the 99 ref,
> which includes some cases where it can still be appropriate to decrease
> that timer.

Most people see it the other way around : they see no valid case to *increase*
it beyond a few seconds, because for them the default value should be extremely
low (ie this firewall vendor several years ago trying to insist on one second).
Yes that's really sad but that's reality. And you can tell them to read 6191
they won't care.

> >   - TCP timestamps: what they provide, what are the risks (some people in
> >     banking environments refuse to enable them so that they cannot be used
> >     as an oracle to help in timing attacks).
> That's already covered in the security considerations of RFC 7323. How
> is HTTP different, if at all, from any other app?

HTTP is special in that it is fairly common to have to deal with tens of
thousands of connections per second between one client and one server when
you are on the server side, because you place a number of gateways (also
called reverse-proxies) which combine all of the possible issues you can
think of at a single place. Timestamps are one way to improve fast connection
recycling between the client and the server without having to cheat on
timewaits. But since they consume 12 bytes per packet, it's often advised
to disable them in benchmarks to get the highest throughput...

> >   - window scaling : how much is needed.
> Same issue here, same ref - how is HTTP different?

Same as above.

> >   - socket sizing : contrary to what you write, there's a lot of tuning
> >     on the web where people set the default buffer sizes to 16MB without
> >     understanding the impacts when dealing with many sockets
> There's a whole book that encompasses that and some related issues:
> http://www.onlamp.com/pub/a/onlamp/2005/11/17/tcp_tuning.html

Looks fine, could be added to the list of references.

> Some advice is also given in Sec 6.3.3 of this:
> J. Sterbenz, J. Touch, /High Speed Networking: A Systematic Approach to
> High-Bandwidth Low-Latency Communication/
> ,
> Wiley, May 2001.
> 
> >   - SACK : why it's better. DSACK what it adds on top of SACK.
> That's in the SACK docs, which aren't cited. Again, how is HTTP
> different from any app?
> >   - ECN : is it needed ? does it really work ? where does it cause issues ?
> That's in the ECN docs, which aren't cited. Again, how is HTTP different
> from any app?
> >   - SYN cookies : benefits, risks
> That's in the RFC 4987, which at least IS cited. Again, how is HTTP
> different from any app?

So probably you're starting to see the benefit of having a single doc
to concentrate all this. You provided at least 3 different articles
to read and 2 or 3 different RFCs in addition to the original ones,
of course. A hosting provider whose web sites are down due to a lack
of tuning doesn't start to read many very long articles and even less
the most scientific ones, they need to find quick responses that they
can apply immediately (matter of minutes). So they launch google, they
type "web site dead, time-wait overflow" and they get plenty of
responses on stackoverflow and serverfault, many from people having
done the same in the past and repeating the same mistakes over and over.

A document validated by several people and giving links for further
reading can help improve this situation.

> >   - TCP reuse/recycling : benefits, risks
> Not sure what you mean here. There are a lot of docs on the issues with
> persistent-HTTP vs per-connection HTTP.

Often on a gateway you cannot completely chose. You have a mix of both.

> >   - dealing with buffer bloat : tradeoffs between NIC-based acceleration
> >     and pacing
> Bufferbloat typically involves large *uplink* transfers and how they
> interact with other uplink connections. Neither TCP nor HTTP is involved
> in this really.

Maybe in the end Daniel is not the only one not to read all articles
published on the web :-)

     http://www.cringely.com/2012/03/25/linux-3-3-finally-a-little-good-news-for-bufferbloat/
     https://www.ietf.org/proceedings/86/slides/slides-86-iccrg-0.pdf
     https://lwn.net/Articles/564978/

In short, by sending 64kB segments to your NIC and counting on it to
cut them into small pieces for you and sending the resulting packets
very close together, you increase the risk of losses on many network
equipments which run with not-that-large buffers. It's easy to observe
even on some 10Gbps switches. When you switch mixes 10G and 40G, it's
horrible, really.

> >   - what are orphans and why you should care about them in HTTP close mode
> Orphaned TCP connections or orphaned HTTP processes?

TCP connections. The server sends a response, performs a close() on the
socket, the data remain in the kernel buffers for the time it takes to
deliver these data to the client and to get them ACKed. From this point
the socket is called orphaned on the system because it doesn't belong to
any process anymore. But immediately after this, a process which runs with
a limited connection count can accept a new connection, which will in turn
be fed with large chunks of data and closed. And this runs over and over,
eating a huge amount of socket memory despite a small limit imposed on the
server's connection concurrency. At some point the kernel doesn't have
enough socket buffers anymore (especially after admins have set them to
16MB as instructed on $random tuning guide) and starts to kill some orphaned
connections to get some memory back. The not-so-nice effect that the admin
cannot detect is that the client gets truncated responses. Only a small part
of the tail is missing but in the logs, it's said that everything was sent.
But sent by the process means sent to the system only.

> >   - TCP fastopen : how does it work, what type of workload is improved,
> >     what are the risks (ie: do not enable socket creation without cookie
> >     by default just because you find it reduces your server load)
> 
> Another doc that exists.
> 
> >   - whether to choose a short or a large SYN backlog depending on your
> >     workload (ie: do you prefer to process everything even if the dequeuing
> >     is expensive or to drop early in order to recover fast).
> >
> > ... and probably many other that don't immediately come to my mind. None
> > of these ones was a real issue 20 years ago.
> 
> See above. Many were known around that time, but weren't documented in
> detail (it took a while for a proper ref to SYN cookies, and the book I
> wrote with Sterbenz came about because we'd seen wheels being
> rediscovered for 15 years).

People rediscover wheels because it's hard to find simple and accurate
information on the net. Basically you have the choice :
  - either uneducated blog posts saying "how I saved my web site using 2
    sysctls"
  - or academic papers which are only understandable by scientific people
    having enough time

At least the first ones have the merit of being easy to test, and since
they appear to work they are viral.

> >  All of them became issues for
> > many web server admins who just copy-paste random settings from various
> > blogs found on the net who just copy the same stupidities over and over
> > resulting in the same trouble being caused to each of their reader.
> 
> This doc is all over the place.
> 
> If you want a doc to advise web admins, do so.

That's *exactly* what Daniel started to do when you told him he shouldn't
do it.

> But most of the items
> above aren't under admin control; they're buried in app and OS
> implementations, and most have already evolved do to the right thing.

That's not true anymore. There was a time where Solaris had one of the
most configurable TCP stack, you could do ndd /dev/tcp with a lot of
things. Nowadays all modern operating systems let you fiddle with a ton
of stuff. And even for the missing parts you can very easily find patches
all over the web because opensource has changed this considerably.

> I agree that a summary of a *focused set* of these might be useful *as a
> review* (note that review discussions usually include lots of refs).

The purpose as I understood it was precisely to gather knowledge from
people here operating such systems and adding some elements found from
various sources. I also expressed my interest in sharing such experience
which some will find valid and others not, which is perfect because it
means there's a choice to make depending on a use case.

> The key question is "what is the focus":
> 
>     - HTTP/TCP interactions
>     - server administration advice
>     - ?

Those two go hand-in-hand nowadays. You probably know that the difficulty
for HTTP implementors to find a properly tuned TCP stack is what makes them
consider UDP-based alternatives, so that they can totally control the
behaviour from user-space and provide quick updates for their protocol.
Want an example ?

    https://www.chromium.org/quic

If it was possible to always have a perfectly tuned TCP stack I'm pretty
sure we wouldn't even hear about such initiatives. And this is not something
new, I happened to discuss about this subject with some people at IETF83 in
2012 already. By then I thought "they have no idea what they're trying to
reinvent" and now I'm thinking "well, they have their reasons and they
might be right after all given all the resistance they may be facing on
the TCP side to get some default timers changed".

> IMO, RFCs should focus on issues specific to the protocols and their
> deployment - not general knowledge that exists in courses and textbooks.

Courses and textbooks are totally outdated when they're not simply wrong.
I've first been tought that it was normal to fork a process after performing
an accept(), resulting in my very first proxy working this way. Then I've
been taught that it was mandatory to send an empty ACK after a SYN-ACK to
validate a connection (which is totally wrong otherwise it would not allow
this ACK to be lost). I've been taught that in order to accept an incoming
connection you had to have your socket in listen mode exclusively. This is
false as well, two clients can connect together during their connect()
phase, it even has some security implications that are often overlooked.
Like it or not, the HTTP protocol has brought TCP to an area it was not
initially intended for and I find it fantastic to see that this protocol
still scales so well. But we need to consider modern usages of this protocol
for the web, and not just academic research and e-mail.

> >> I.e., at the most this is a man page (specific to an OS). At the least,
> >> this isn't useful at all.
> > As you can see above, nothing I cited was OS-specific but only workload
> > specific. That's why I think that an informational RFC is much more suited
> > to this than an OS-specific man page. The OS man page may rely on the RFC
> > to propose various tuning profiles for different workloads however.
> 
> You have a good point that this is general info, but OS issues are not
> in the scope of the IETF and there are courses and books that already
> provide good advice on efficiently running web (and other) servers.

Well, you want to run a quick test ? Google "tcp tuning for http server".
Skip the first two responses which are Daniel's document, take the next one :

    https://gist.github.com/kgriffs/4027835

Read a bit... Not bad, could have been much worse. OK found : 5 seconds
time_wait timeout for conntrack. Dig a little bit more... suggests 2M
time_wait sockets. At 5s per socket, that's an expected 400k connections
per second limit. This with buffers that can be as large as 16MB on both
sides (one could ask why to put 16MB buffers on the Rx path for a web
server). So basically the guy probably thinks that there's a need to
fill up to 6.4 TB/s and despite this the max-orphans was not set according
to the time-wait value, so users of his doc will start to cause some data
truncation before they run into connection failures, thus only their visitors
will know there are problems.

And this is the highest ranked doc on google after Daniel's. And overall
it's really not bad, much better than what we easily find everywhere.

The fact that Daniel's doc found before is a good hope that this sad state
of affairs can reach an end. That's why I think we should encourage him to
continue and give him all the reference he needs to have an undiscutably
good reference doc.

Regards,
Willy


From nobody Wed Aug 17 15:23:59 2016
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id BB3C012D19F for ; Wed, 17 Aug 2016 15:23:57 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -8.167
X-Spam-Level: 
X-Spam-Status: No, score=-8.167 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-1.247] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id BD5A26IA3YBx for ; Wed, 17 Aug 2016 15:23:54 -0700 (PDT)
Received: from boreas.isi.edu (boreas.isi.edu [128.9.160.161]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 8784112D18E for ; Wed, 17 Aug 2016 15:23:54 -0700 (PDT)
Received: from [128.9.184.210] ([128.9.184.210]) (authenticated bits=0) by boreas.isi.edu (8.13.8/8.13.8) with ESMTP id u7HMNCoT018451 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Wed, 17 Aug 2016 15:23:12 -0700 (PDT)
To: Willy Tarreau 
References: <0CC24FC1-37E1-4125-9627-05726A9D9406@mnot.net> <7fa95741-ac58-3183-1b92-238bd4b4dae6@isi.edu> <5CD67877-19E3-4E79-BBF2-3E270343A378@mnot.net> <2197232f-10d7-28cb-fcc9-05bd495e3c22@isi.edu> <20160817064545.GD16017@1wt.eu> <7f7b129c-f156-d067-bef8-4a2213f461ac@isi.edu> <20160817180802.GA16773@1wt.eu> <4ab7c5b0-3722-1346-f481-a8d76de70034@isi.edu> <20160817211317.GA16929@1wt.eu>
From: Joe Touch 
Message-ID: 
Date: Wed, 17 Aug 2016 15:23:10 -0700
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0
MIME-Version: 1.0
In-Reply-To: <20160817211317.GA16929@1wt.eu>
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: 7bit
X-ISI-4-43-8-MailScanner: Found to be clean
X-MailScanner-From: touch@isi.edu
Archived-At: 
Cc: Mark Nottingham , tcpm@ietf.org, HTTP Working Group , Daniel Stenberg , Patrick McManus 
Subject: Re: [tcpm] TCP Tuning for HTTP - update
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Wed, 17 Aug 2016 22:23:58 -0000

On 8/17/2016 2:13 PM, Willy Tarreau wrote:
> On Wed, Aug 17, 2016 at 11:31:33AM -0700, Joe Touch wrote:
>>> It can be cited in new RFCs
>>> to justify certain choices. 
>> Hmm. Like the refs I gave could be cited in this doc to justify *its*
>> choices? :-)
> I think it would be nice that this is cited, but to be clear on one
> point, I've never heard about your papers before you advertised them
> here in this thread, 

A search engine on the terms "TCP HTTP interaction" would have popped
them up rather quickly.

> and yet I've been dealing with timewait issues
> for 15 years like many people facing moderate to large web sites
> nowadays. 

"timewait issues" and we're the 5th hit in Google.


> It just happens that you probably identified these issues
> very early at low connection rates, but today anyone dealing with
> more than 500-1000 connections per second on a server or worse on
> a gateway quickly discovers that he has to make a choice.

The issue was very common when the doc was written in 99, when even at
that time there were two issues - running out of the number space and
running out of kernel memory.

The number space issue of running out of ports was the basis of the IETF
port names doc in 2006
(https://tools.ietf.org/html/draft-touch-tcp-portnames-00) that became
the current proposal for a TCP "service number option" in 2013 (which
has been discussed at various IETFs in TCPM since then).

>>>> Yes, and discussing those issues would be useful - but not in this
>>>> document either.
>>> Why ? Lots of admins don't understand why the time_wait timeout remains
>>> at 240 seconds on Solaris with people saying "if you want to be conservative
>>> don't touch it but if you want to be modern simply shrink it to 30 seconds
>>> or so". People need to understand why advices have changed over 3 decades.
>> The advice hasn't really changed - the advice was given in the 99 ref,
>> which includes some cases where it can still be appropriate to decrease
>> that timer.
> Most people see it the other way around : they see no valid case to *increase*
> it beyond a few seconds, because for them the default value should be extremely
> low (ie this firewall vendor several years ago trying to insist on one second).
> Yes that's really sad but that's reality. And you can tell them to read 6191
> they won't care.

Most people's servers don't need to run fast enough to care (note that
nearly everyone runs some sort of web server on nearly every device,
whether for control or configuration). The only issue are high-volume
servers (the kind sysadmins deal with), and those people tend to already
know what the tradeoffs are and accept the risks.

>
>>>   - TCP timestamps: what they provide, what are the risks (some people in
>>>     banking environments refuse to enable them so that they cannot be used
>>>     as an oracle to help in timing attacks).
>> That's already covered in the security considerations of RFC 7323. How
>> is HTTP different, if at all, from any other app?
> HTTP is special in that it is fairly common to have to deal with tens of
> thousands of connections per second between one client and one server when
> you are on the server side, because you place a number of gateways (also
> called reverse-proxies) which combine all of the possible issues you can
> think of at a single place.

There are lots of services that have that many transactions - DNS
servers (even local ones), remote databases, etc.

The point is that HTTP doesn't make the problem different, so this isn't
an HTTP issue. It's a high rate server issue.


>  Timestamps are one way to improve fast connection
> recycling between the client and the server without having to cheat on
> timewaits. But since they consume 12 bytes per packet, it's often advised
> to disable them in benchmarks to get the highest throughput...
>
>>>   - window scaling : how much is needed.
>> Same issue here, same ref - how is HTTP different?
> Same as above.
>
>>>   - socket sizing : contrary to what you write, there's a lot of tuning
>>>     on the web where people set the default buffer sizes to 16MB without
>>>     understanding the impacts when dealing with many sockets
>> There's a whole book that encompasses that and some related issues:
>> http://www.onlamp.com/pub/a/onlamp/2005/11/17/tcp_tuning.html
> Looks fine, could be added to the list of references.
>
>> Some advice is also given in Sec 6.3.3 of this:
>> J. Sterbenz, J. Touch, /High Speed Networking: A Systematic Approach to
>> High-Bandwidth Low-Latency Communication/
>> ,
>> Wiley, May 2001.
>>
>>>   - SACK : why it's better. DSACK what it adds on top of SACK.
>> That's in the SACK docs, which aren't cited. Again, how is HTTP
>> different from any app?
>>>   - ECN : is it needed ? does it really work ? where does it cause issues ?
>> That's in the ECN docs, which aren't cited. Again, how is HTTP different
>> from any app?
>>>   - SYN cookies : benefits, risks
>> That's in the RFC 4987, which at least IS cited. Again, how is HTTP
>> different from any app?
> So probably you're starting to see the benefit of having a single doc
> to concentrate all this.
The same reason it's useful to have this all in one place is the reason
we already do - there are books and courses on this.

>  You provided at least 3 different articles
> to read and 2 or 3 different RFCs in addition to the original ones,
> of course. A hosting provider whose web sites are down due to a lack
> of tuning doesn't start to read many very long articles and even less
> the most scientific ones, they need to find quick responses that they
> can apply immediately (matter of minutes). So they launch google, they
> type "web site dead, time-wait overflow" and they get plenty of
> responses on stackoverflow and serverfault, many from people having
> done the same in the past and repeating the same mistakes over and over.

These people don't read RFCs to fix problems. They take online courses
or read "how to" books - which do already exist in this space.

> A document validated by several people and giving links for further
> reading can help improve this situation.
Those are the books and courses I'm talking about already.

>
>>>   - TCP reuse/recycling : benefits, risks
>> Not sure what you mean here. There are a lot of docs on the issues with
>> persistent-HTTP vs per-connection HTTP.
> Often on a gateway you cannot completely chose. You have a mix of both.

Sure... but again, that's not in this doc either.

>
>>>   - dealing with buffer bloat : tradeoffs between NIC-based acceleration
>>>     and pacing
>> Bufferbloat typically involves large *uplink* transfers and how they
>> interact with other uplink connections. Neither TCP nor HTTP is involved
>> in this really.
> Maybe in the end Daniel is not the only one not to read all articles
> published on the web :-)
>
>      http://www.cringely.com/2012/03/25/linux-3-3-finally-a-little-good-news-for-bufferbloat/
>      https://www.ietf.org/proceedings/86/slides/slides-86-iccrg-0.pdf
>      https://lwn.net/Articles/564978/
>
> In short, by sending 64kB segments to your NIC and counting on it to
> cut them into small pieces for you and sending the resulting packets
> very close together, you increase the risk of losses on many network
> equipments which run with not-that-large buffers.

Bufferbloat describes what happens when the buffers are too large, not
too small.

The problem you're describing is the interaction between burstiness and
tail-drop, which is addressed by ECN.

>  It's easy to observe
> even on some 10Gbps switches. When you switch mixes 10G and 40G, it's
> horrible, really.
RFCs aren't typically used as context in switch design...


>
>>>   - what are orphans and why you should care about them in HTTP close mode
>> Orphaned TCP connections or orphaned HTTP processes?
> TCP connections. The server sends a response, performs a close() on the
> socket, the data remain in the kernel buffers for the time it takes to
> deliver these data to the client and to get them ACKed. From this point
> the socket is called orphaned on the system because it doesn't belong to
> any process anymore. But immediately after this, a process which runs with
> a limited connection count can accept a new connection, which will in turn
> be fed with large chunks of data and closed. And this runs over and over,
> eating a huge amount of socket memory despite a small limit imposed on the
> server's connection concurrency. At some point the kernel doesn't have
> enough socket buffers anymore (especially after admins have set them to
> 16MB as instructed on $random tuning guide) and starts to kill some orphaned
> connections to get some memory back. The not-so-nice effect that the admin
> cannot detect is that the client gets truncated responses. Only a small part
> of the tail is missing but in the logs, it's said that everything was sent.
> But sent by the process means sent to the system only.

That's an implementation issue of the OS (or a bug, depending on whether
you consider TCP a reliable transport or not, IMO).

>
>>>   - TCP fastopen : how does it work, what type of workload is improved,
>>>     what are the risks (ie: do not enable socket creation without cookie
>>>     by default just because you find it reduces your server load)
>> Another doc that exists.
>>
>>>   - whether to choose a short or a large SYN backlog depending on your
>>>     workload (ie: do you prefer to process everything even if the dequeuing
>>>     is expensive or to drop early in order to recover fast).
>>>
>>> ... and probably many other that don't immediately come to my mind. None
>>> of these ones was a real issue 20 years ago.
>> See above. Many were known around that time, but weren't documented in
>> detail (it took a while for a proper ref to SYN cookies, and the book I
>> wrote with Sterbenz came about because we'd seen wheels being
>> rediscovered for 15 years).
> People rediscover wheels because it's hard to find simple and accurate
> information on the net.
Nobody looks to RFCs to solve that problem...

>  Basically you have the choice :
>   - either uneducated blog posts saying "how I saved my web site using 2
>     sysctls"
>   - or academic papers which are only understandable by scientific people
>     having enough time

... that's what net FAQs are for, as well as courses and books.

> At least the first ones have the merit of being easy to test, and since
> they appear to work they are viral.
>
>>>  All of them became issues for
>>> many web server admins who just copy-paste random settings from various
>>> blogs found on the net who just copy the same stupidities over and over
>>> resulting in the same trouble being caused to each of their reader.
>> This doc is all over the place.
>>
>> If you want a doc to advise web admins, do so.
> That's *exactly* what Daniel started to do when you told him he shouldn't
> do it.

I didn't say a doc to advise web admins wasn't useful. I said it wasn't
an RFC.

It's a web FAQ, a book, etc.


>
>> But most of the items
>> above aren't under admin control; they're buried in app and OS
>> implementations, and most have already evolved do to the right thing.
> That's not true anymore. There was a time where Solaris had one of the
> most configurable TCP stack, you could do ndd /dev/tcp with a lot of
> things. Nowadays all modern operating systems let you fiddle with a ton
> of stuff. And even for the missing parts you can very easily find patches
> all over the web because opensource has changed this considerably.
There are no kernel configs to tell apps to open more than one
connection at a time. You don't need a kernel config to tell Firefox to
disable Nagle, use a reasonable socket size, etc - yes, there are OS
defaults, but most web servers and clients build in the correct
overrides already.

>
>> I agree that a summary of a *focused set* of these might be useful *as a
>> review* (note that review discussions usually include lots of refs).
> The purpose as I understood it was precisely to gather knowledge from
> people here operating such systems and adding some elements found from
> various sources. I also expressed my interest in sharing such experience
> which some will find valid and others not, which is perfect because it
> means there's a choice to make depending on a use case.
>
>> The key question is "what is the focus":
>>
>>     - HTTP/TCP interactions
>>     - server administration advice
>>     - ?
> Those two go hand-in-hand nowadays. You probably know that the difficulty
> for HTTP implementors to find a properly tuned TCP stack is what makes them
> consider UDP-based alternatives, 
They want something different for a variety of reasons - the same kind
of airtight logic by which TBL developed HTTP instead of using FTP (he
said that you'd only typically need one file from a location, so why
open 2 connections? now we're stuck trying to mux control and data
rather than having a proper solution that already existed at the time -
it took nearly a decade for HTTP servers to catch up to the performance
of FTP).

> so that they can totally control the
> behaviour from user-space and provide quick updates for their protocol.
> Want an example ?
>
>     https://www.chromium.org/quic

There are many thousands of monkeys typing everywhere - look at the
Linux source if you want even better examples.

> If it was possible to always have a perfectly tuned TCP stack I'm pretty
> sure we wouldn't even hear about such initiatives. And this is not something
> new, I happened to discuss about this subject with some people at IETF83 in
> 2012 already. By then I thought "they have no idea what they're trying to
> reinvent" and now I'm thinking "well, they have their reasons and they
> might be right after all given all the resistance they may be facing on
> the TCP side to get some default timers changed".
>
>> IMO, RFCs should focus on issues specific to the protocols and their
>> deployment - not general knowledge that exists in courses and textbooks.
> Courses and textbooks are totally outdated when they're not simply wrong.
I'm now confused.

You don't want sysadmins to read books or take courses because they're
outdated and thus wrong, but you want to issue an immutable RFC (which
will likely be outdated by the time it's issued)?
> I've first been tought that it was normal to fork a process after performing
> an accept(), resulting in my very first proxy working this way. Then I've
> been taught that it was mandatory to send an empty ACK after a SYN-ACK to
> validate a connection (which is totally wrong otherwise it would not allow
> this ACK to be lost). I've been taught that in order to accept an incoming
> connection you had to have your socket in listen mode exclusively. This is
> false as well, two clients can connect together during their connect()
> phase, it even has some security implications that are often overlooked.
I'm the first to admit there are bad courses, certainly.

> Like it or not, the HTTP protocol has brought TCP to an area it was not
> initially intended for 
HTTP makes mistakes that people blame on TCP (like HOL blocking), and
TCP is based on assumptions that are no longer true (not just for HTTP,
but for many other app protocols, e.g., the issue of burst after idle is
based on the outdated assumption that most transfers are roughly symmetric).

> and I find it fantastic to see that this protocol
> still scales so well. 
> But we need to consider modern usages of this protocol
> for the web, and not just academic research and e-mail.
You might consider that TCPM and TSVWG don't exist for just "academic
research and e-mail". What do you think we've been doing for the past 40
years?

>
>>>> I.e., at the most this is a man page (specific to an OS). At the least,
>>>> this isn't useful at all.
>>> As you can see above, nothing I cited was OS-specific but only workload
>>> specific. That's why I think that an informational RFC is much more suited
>>> to this than an OS-specific man page. The OS man page may rely on the RFC
>>> to propose various tuning profiles for different workloads however.
>> You have a good point that this is general info, but OS issues are not
>> in the scope of the IETF and there are courses and books that already
>> provide good advice on efficiently running web (and other) servers.
> Well, you want to run a quick test ? Google "tcp tuning for http server".
> Skip the first two responses which are Daniel's document, take the next one :
>
>     https://gist.github.com/kgriffs/4027835

Like I said about Linux... ;-)

> Read a bit... Not bad, could have been much worse. OK found : 5 seconds
> time_wait timeout for conntrack. Dig a little bit more... suggests 2M
> time_wait sockets. At 5s per socket, that's an expected 400k connections
> per second limit. This with buffers that can be as large as 16MB on both
> sides (one could ask why to put 16MB buffers on the Rx path for a web
> server). So basically the guy probably thinks that there's a need to
> fill up to 6.4 TB/s and despite this the max-orphans was not set according
> to the time-wait value, so users of his doc will start to cause some data
> truncation before they run into connection failures, thus only their visitors
> will know there are problems.
>
> And this is the highest ranked doc on google after Daniel's. And overall
> it's really not bad, much better than what we easily find everywhere.
>
> The fact that Daniel's doc found before is a good hope that this sad state
> of affairs can reach an end. That's why I think we should encourage him to
> continue and give him all the reference he needs to have an undiscutably
> good reference doc.

You might try some other terms in your searches. Like just "TCP tuning",
or "TCP HTTP interactions".

Joe


From nobody Wed Aug 17 15:51:25 2016
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D3DD612B074 for ; Wed, 17 Aug 2016 15:51:22 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -3.148
X-Spam-Level: 
X-Spam-Status: No, score=-3.148 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RP_MATCHES_RCVD=-1.247, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 19en9uCsOd6D for ; Wed, 17 Aug 2016 15:51:21 -0700 (PDT)
Received: from smtp.qbik.com (smtp.qbik.com [122.56.26.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 0DFF212B016 for ; Wed, 17 Aug 2016 15:51:20 -0700 (PDT)
Received: From [192.168.1.146] (unverified [192.168.1.146]) by SMTP Server [192.168.1.3] (WinGate SMTP Receiver v9.0.0 (Build 5855)) with SMTP id <0000805476@smtp.qbik.com>; Thu, 18 Aug 2016 10:51:18 +1200
From: "Adrien de Croy" 
To: "Joe Touch" , "Willy Tarreau" 
Date: Wed, 17 Aug 2016 22:51:18 +0000
Message-Id: 
In-Reply-To: 
User-Agent: eM_Client/6.0.24928.0
Mime-Version: 1.0
Content-Type: text/plain; format=flowed; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Archived-At: 
Cc: Mark Nottingham , "tcpm@ietf.org" , HTTP Working Group , Daniel Stenberg , Patrick McManus 
Subject: Re: [tcpm] TCP Tuning for HTTP - update
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
Reply-To: Adrien de Croy 
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Wed, 17 Aug 2016 22:51:23 -0000

------ Original Message ------
From: "Joe Touch" 

>They want something different for a variety of reasons - the same kind
>of airtight logic by which TBL developed HTTP instead of using FTP (he
>said that you'd only typically need one file from a location, so why
>open 2 connections? now we're stuck trying to mux control and data
>rather than having a proper solution that already existed at the time -
>it took nearly a decade for HTTP servers to catch up to the performance
>of FTP).
>
Whilst I've been finding this discussion very informative and=20
interesting, I have to raise an objection on this point.

FTP was never going to be suitable for the web, and a very simple RTT=20
analysis shows that.

Apart from initial 3 way TCP handshake and close, which is the same for=20
both, with http you have a request and a response, whereas FTP requires=20
you to wait for the server welcome, log in, negotiate another port, set=20
up a data connection in addition to retrieving the file

So it's at minimum 5 round trips more.

Then try adding all the firewall issues due to transmitting data=20
connection endpoint information over the control connection and it's no=20
surprise FTP is not favoured for downloads.
So FTP was never going to be a "proper solution" for the web without a=20
complete re-architecture.

Adrien


>


From nobody Wed Aug 17 16:02:53 2016
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C6E5B12B074 for ; Wed, 17 Aug 2016 16:02:51 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -8.167
X-Spam-Level: 
X-Spam-Status: No, score=-8.167 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-1.247] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id YBx7QTD_d8vF for ; Wed, 17 Aug 2016 16:02:50 -0700 (PDT)
Received: from boreas.isi.edu (boreas.isi.edu [128.9.160.161]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id D89B212B04A for ; Wed, 17 Aug 2016 16:02:50 -0700 (PDT)
Received: from [128.9.184.210] ([128.9.184.210]) (authenticated bits=0) by boreas.isi.edu (8.13.8/8.13.8) with ESMTP id u7HN1xAQ026629 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Wed, 17 Aug 2016 16:02:00 -0700 (PDT)
To: Adrien de Croy , Willy Tarreau 
References: 
From: Joe Touch 
Message-ID: <05d949e3-07aa-4cf5-8aeb-122b2be24519@isi.edu>
Date: Wed, 17 Aug 2016 16:01:57 -0700
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0
MIME-Version: 1.0
In-Reply-To: 
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-ISI-4-43-8-MailScanner: Found to be clean
X-MailScanner-From: touch@isi.edu
Archived-At: 
Cc: Mark Nottingham , "tcpm@ietf.org" , HTTP Working Group , Daniel Stenberg , Patrick McManus 
Subject: Re: [tcpm] TCP Tuning for HTTP - update
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Wed, 17 Aug 2016 23:02:52 -0000

This is a bit of a side track, but...

On 8/17/2016 3:51 PM, Adrien de Croy wrote:
>
>
> ------ Original Message ------
> From: "Joe Touch" 
>
>> They want something different for a variety of reasons - the same kind
>> of airtight logic by which TBL developed HTTP instead of using FTP (he
>> said that you'd only typically need one file from a location, so why
>> open 2 connections? now we're stuck trying to mux control and data
>> rather than having a proper solution that already existed at the time -
>> it took nearly a decade for HTTP servers to catch up to the performance
>> of FTP).
>>
> Whilst I've been finding this discussion very informative and
> interesting, I have to raise an objection on this point.
>
> FTP was never going to be suitable for the web, and a very simple RTT
> analysis shows that.
>
> Apart from initial 3 way TCP handshake and close, which is the same
> for both, with http you have a request and a response, whereas FTP
> requires you to wait for the server welcome, log in, negotiate another
> port, set up a data connection in addition to retrieving the file

That's only the first time you go somewhere new. You don't need to close
both ports so quickly; the control channel can stay open and you thus
avoid HOL blocking between data and control (and thus the need to
chunk-and-mux within persistent HTTP), which increases other delays for
HTTP.

Neither protocol matches exactly what is really needed for a true
transaction-oriented protocol.

> ...
> Then try adding all the firewall issues due to transmitting data
> connection endpoint information over the control connection and it's
> no surprise FTP is not favoured for downloads.

FTP had a passive mode even back then that avoids this issue. It also
had suspend/resume, compression, and format conversion.

Joe


From nobody Wed Aug 17 16:17:19 2016
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 225FF12D52B for ; Wed, 17 Aug 2016 16:17:18 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -3.148
X-Spam-Level: 
X-Spam-Status: No, score=-3.148 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RP_MATCHES_RCVD=-1.247, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Oxi46Zydgfq7 for ; Wed, 17 Aug 2016 16:17:16 -0700 (PDT)
Received: from smtp.qbik.com (smtp.qbik.com [122.56.26.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 0E34712B05E for ; Wed, 17 Aug 2016 16:17:15 -0700 (PDT)
Received: From [192.168.1.146] (unverified [192.168.1.146]) by SMTP Server [192.168.1.3] (WinGate SMTP Receiver v9.0.0 (Build 5855)) with SMTP id <0000805490@smtp.qbik.com>; Thu, 18 Aug 2016 11:17:13 +1200
From: "Adrien de Croy" 
To: "Joe Touch" , "Willy Tarreau" 
Date: Wed, 17 Aug 2016 23:17:13 +0000
Message-Id: 
In-Reply-To: <05d949e3-07aa-4cf5-8aeb-122b2be24519@isi.edu>
User-Agent: eM_Client/6.0.24928.0
Mime-Version: 1.0
Content-Type: text/plain; format=flowed; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Archived-At: 
Cc: Mark Nottingham , "tcpm@ietf.org" , HTTP Working Group , Daniel Stenberg , Patrick McManus 
Subject: Re: [tcpm] TCP Tuning for HTTP - update
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
Reply-To: Adrien de Croy 
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Wed, 17 Aug 2016 23:17:18 -0000

------ Original Message ------
From: "Joe Touch" 
To: "Adrien de Croy" ; "Willy Tarreau" 
Cc: "Mark Nottingham" ; "tcpm@ietf.org" ;=20
"HTTP Working Group" ; "Patrick McManus"=20
; "Daniel Stenberg" 
Sent: 18/08/2016 11:01:57 AM
Subject: Re: [tcpm] TCP Tuning for HTTP - update

>This is a bit of a side track, but...
>
>On 8/17/2016 3:51 PM, Adrien de Croy wrote:
>>
>>
>>  ------ Original Message ------
>>  From: "Joe Touch" 
>>
>>>  They want something different for a variety of reasons - the same=20
>>>kind
>>>  of airtight logic by which TBL developed HTTP instead of using FTP=20
>>>(he
>>>  said that you'd only typically need one file from a location, so why
>>>  open 2 connections? now we're stuck trying to mux control and data
>>>  rather than having a proper solution that already existed at the=20
>>>time -
>>>  it took nearly a decade for HTTP servers to catch up to the=20
>>>performance
>>>  of FTP).
>>>
>>  Whilst I've been finding this discussion very informative and
>>  interesting, I have to raise an objection on this point.
>>
>>  FTP was never going to be suitable for the web, and a very simple RTT
>>  analysis shows that.
>>
>>  Apart from initial 3 way TCP handshake and close, which is the same
>>  for both, with http you have a request and a response, whereas FTP
>>  requires you to wait for the server welcome, log in, negotiate=20
>>another
>>  port, set up a data connection in addition to retrieving the file
>
>That's only the first time you go somewhere new. You don't need to=20
>close
>both ports so quickly; the control channel can stay open and you thus
>avoid HOL blocking between data and control (and thus the need to
>chunk-and-mux within persistent HTTP), which increases other delays for
>HTTP.
You still need to send another PORT/PASV and wait for the response=20
before making another TCP connection for data, since you can't re-use=20
this one.

So subsequent requests to the same server will be quicker, still at=20
least 3 round-trips more than a subsequent http request on a persistent=20
connection.

>
>Neither protocol matches exactly what is really needed for a true
>transaction-oriented protocol.
We are probably in agreement here.

>
>>  ...
>>  Then try adding all the firewall issues due to transmitting data
>>  connection endpoint information over the control connection and it's
>>  no surprise FTP is not favoured for downloads.
>
>FTP had a passive mode even back then that avoids this issue. It also
>had suspend/resume, compression, and format conversion.

Those all take more round trips to negotiate.  As for format=20
conversion..... bad idea, never should have been the server's job.

We've seen a lot of server-side firewall problems with PASV as well.

Adrien


>
>Joe


From nobody Wed Aug 17 16:22:06 2016
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A77DA12D7C9 for ; Wed, 17 Aug 2016 16:22:04 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -8.146
X-Spam-Level: 
X-Spam-Status: No, score=-8.146 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-5, RP_MATCHES_RCVD=-1.247] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rI5OGdpu33Yh for ; Wed, 17 Aug 2016 16:22:03 -0700 (PDT)
Received: from vapor.isi.edu (vapor.isi.edu [128.9.64.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id CA30712D7A2 for ; Wed, 17 Aug 2016 16:22:03 -0700 (PDT)
Received: from [128.9.184.210] ([128.9.184.210]) (authenticated bits=0) by vapor.isi.edu (8.13.8/8.13.8) with ESMTP id u7HNKuqh021529 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Wed, 17 Aug 2016 16:20:57 -0700 (PDT)
To: Adrien de Croy , Willy Tarreau 
References: 
From: Joe Touch 
Message-ID: <33f35249-7c7f-40bc-c24b-a0f471427e4c@isi.edu>
Date: Wed, 17 Aug 2016 16:20:54 -0700
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0
MIME-Version: 1.0
In-Reply-To: 
Content-Type: multipart/alternative; boundary="------------C37E2CC18571EAC4F9C39822"
X-ISI-4-43-8-MailScanner: Found to be clean
X-MailScanner-From: touch@isi.edu
Archived-At: 
Cc: Mark Nottingham , "tcpm@ietf.org" , HTTP Working Group , Daniel Stenberg , Patrick McManus 
Subject: Re: [tcpm] TCP Tuning for HTTP - update
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Wed, 17 Aug 2016 23:22:04 -0000

This is a multi-part message in MIME format.
--------------C37E2CC18571EAC4F9C39822
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit



On 8/17/2016 4:17 PM, Adrien de Croy wrote:
>> FTP had a passive mode even back then that avoids this issue. It also
>> had suspend/resume, compression, and format conversion.
>
> Those all take more round trips to negotiate.  As for format
> conversion..... bad idea, never should have been the server's job.
Sure, but they can be negotiated once for a connection.

FTP also supports wildcard retrieval (e.g., "get *"), which can help
with anticipation.

My point is that FTP could have been much better place to start towards
a transaction protocol.

Joe

--------------C37E2CC18571EAC4F9C39822
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: 8bit


  
    
  
  
    


    

    

    
On 8/17/2016 4:17 PM, Adrien de Croy
      wrote:

    

    

      
FTP had a passive
        mode even back then that avoids this issue. It also
        

        had suspend/resume, compression, and format conversion.
        

      

      

      Those all take more round trips to negotiate.  As for format
      conversion..... bad idea, never should have been the server's job.
      

    

    Sure, but they can be negotiated once for a connection.

    

    FTP also supports wildcard retrieval (e.g., "get *"), which can help
    with anticipation.

    

    My point is that FTP could have been much better place to start
    towards a transaction protocol.

    

    Joe

  


--------------C37E2CC18571EAC4F9C39822--


From nobody Wed Aug 17 16:35:20 2016
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1881012D80F for ; Wed, 17 Aug 2016 16:35:18 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.347
X-Spam-Level: 
X-Spam-Status: No, score=-2.347 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FORGED_FROMDOMAIN=0.001, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id q347qQcEAelU for ; Wed, 17 Aug 2016 16:35:15 -0700 (PDT)
Received: from mail-io0-x241.google.com (mail-io0-x241.google.com [IPv6:2607:f8b0:4001:c06::241]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 31D8812D80D for ; Wed, 17 Aug 2016 16:35:15 -0700 (PDT)
Received: by mail-io0-x241.google.com with SMTP id y34so964752ioi.3 for ; Wed, 17 Aug 2016 16:35:15 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;  h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=IquHlWw+16SLhck91t990eDgI3BmqBiOvybP6ROKhfA=; b=ERAiDezbgTFjO9lg8IxxPbPCMuhuPdn2qgPok8W6i57Eh1yt4lfaUdsKr3cP82Yz4e L00FSxAqwyrsDWIpz30Aa8RydGhCXSRrVE0iCnWrHrT/0pZvdIbJIljwym33S5RXmhex MFQ3r5fOkVTYifhJddafnOvNCl/mTp8KiJOQCZ872/DOlShKif8O4/C+7NhG/Pl2yKDq 6S7KuVMOSeFxMDBc4eE329/jC+m5L9Jk+AFI86rX9mNGdAs9NY4+5T3c30aF90jObtfZ LeAVk0fKoq9TFu+FeJzq/6JWusOevo4Bea9vK5dQ2zGmp4a9i6ujhkTVZ19Vm6v+dVZj FbcQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=IquHlWw+16SLhck91t990eDgI3BmqBiOvybP6ROKhfA=; b=TM/fYZOEV505PO/nEVfRShHedGaK72PObG3DNPxlxvhL7oIv2poxqzo92LyDwTA/aJ 4OekPf/411s13EToBM23tSPp4FkmKrdqfrZ8ppfsvvKAL8IAD3YFh+WdafKrAMEzXn+Q CCxVo/RCSSEWVbK50wsAFc3m/1UfuOoJJtgnz89adB3oakKu43L7gbxc8sMp7YB6q6gg wIfX2kAQHA5+dJBrQii1bE20Gmp2lIvdHakhz+BFgmpF9s1UK3wAsj10YO7br/TrXKqy hOLV2pxu8aQ6e0MZ5XndBDS3zO47EV/vclaeAc3JkS0WCSgRPPiDvmESot2Dig4s+hjr IxPg==
X-Gm-Message-State: AEkoouuY6gYuP7nNV+kFE6e+FU440hMpiNT/hgl6sm4YTBCbxCqIzTr/ckCWnnyvu9wdd7GBfaTexJ/eir4GXw==
X-Received: by 10.107.128.210 with SMTP id k79mr50243507ioi.3.1471476914297; Wed, 17 Aug 2016 16:35:14 -0700 (PDT)
MIME-Version: 1.0
Sender: phluid61@gmail.com
Received: by 10.107.158.207 with HTTP; Wed, 17 Aug 2016 16:35:13 -0700 (PDT)
In-Reply-To: 
References: <0CC24FC1-37E1-4125-9627-05726A9D9406@mnot.net> <7fa95741-ac58-3183-1b92-238bd4b4dae6@isi.edu> <5CD67877-19E3-4E79-BBF2-3E270343A378@mnot.net> <2197232f-10d7-28cb-fcc9-05bd495e3c22@isi.edu> <20160817064545.GD16017@1wt.eu> <7f7b129c-f156-d067-bef8-4a2213f461ac@isi.edu> <20160817180802.GA16773@1wt.eu> <4ab7c5b0-3722-1346-f481-a8d76de70034@isi.edu> <20160817211317.GA16929@1wt.eu> 
From: Matthew Kerwin 
Date: Thu, 18 Aug 2016 09:35:13 +1000
X-Google-Sender-Auth: aCRU2QWp0nVaOguTPQVQW63KmpE
Message-ID: 
To: Joe Touch 
Content-Type: multipart/alternative; boundary=001a113f8aaec5a037053a4cea10
Archived-At: 
Cc: tcpm@ietf.org, Daniel Stenberg , Mark Nottingham , Patrick McManus , Willy Tarreau , HTTP Working Group 
Subject: Re: [tcpm] TCP Tuning for HTTP - update
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Wed, 17 Aug 2016 23:35:18 -0000

--001a113f8aaec5a037053a4cea10
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Hi folks, I'm stepping in here on just a couple of points. I'll snip the
bits I can't or won't talk to.

On 18 August 2016 at 08:23, Joe Touch  wrote:

>
>
> On 8/17/2016 2:13 PM, Willy Tarreau wrote:
> > On Wed, Aug 17, 2016 at 11:31:33AM -0700, Joe Touch wrote:
> >>> It can be cited in new RFCs
> >>> to justify certain choices.
> >> Hmm. Like the refs I gave could be cited in this doc to justify *its*
> >> choices? :-)
> > I think it would be nice that this is cited, but to be clear on one
> > point, I've never heard about your papers before you advertised them
> > here in this thread,
>
> A search engine on the terms "TCP HTTP interaction" would have popped
> them up rather quickly.
>
> > and yet I've been dealing with timewait issues
> > for 15 years like many people facing moderate to large web sites
> > nowadays.
>
> "
> =E2=80=8B=E2=80=8B
> timewait issues" and we're the 5th hit in Google.
>
> =E2=80=8B
Google, unless it's changed again recently, tailors search results for the
user. My first page of hits for that query are all serverfault.com,
superuser.com, serverframework.com, stackoverflow.com, etc.

Guess where I get most of my advice.
=E2=80=8B


>
> >>>> Yes, and discussing those issues would be useful - but not in this
> >>>> document either.
> >>> Why ? Lots of admins don't understand why the time_wait timeout remai=
ns
> >>> at 240 seconds on Solaris with people saying "if you want to be
> conservative
> >>> don't touch it but if you want to be modern simply shrink it to 30
> seconds
> >>> or so". People need to understand why advices have changed over 3
> decades.
> >> The advice hasn't really changed - the advice was given in the 99 ref,
> >> which includes some cases where it can still be appropriate to decreas=
e
> >> that timer.
> > Most people see it the other way around : they see no valid case to
> *increase*
> > it beyond a few seconds, because for them the default value should be
> extremely
> > low (ie this firewall vendor several years ago trying to insist on one
> second).
> > Yes that's really sad but that's reality. And you can tell them to read
> 6191
> > they won't care.
>
> Most people's servers don't need to run fast enough to care (note that
> nearly everyone runs some sort of web server on nearly every device,
> whether for control or configuration). The only issue are high-volume
> servers (the kind sysadmins deal with), and those people tend to already
> know what the tradeoffs are and accept the risks.
>
>
=E2=80=8BYour sysadmins are not like my sysadmins. But these are generalisa=
tions
and anecdotes.=E2=80=8B



> >
> >>>   - TCP timestamps: what they provide, what are the risks (some peopl=
e
> in
> >>>     banking environments refuse to enable them so that they cannot be
> used
> >>>     as an oracle to help in timing attacks).
> >> That's already covered in the security considerations of RFC 7323. How
> >> is HTTP different, if at all, from any other app?
> > HTTP is special in that it is fairly common to have to deal with tens o=
f
> > thousands of connections per second between one client and one server
> when
> > you are on the server side, because you place a number of gateways (als=
o
> > called reverse-proxies) which combine all of the possible issues you ca=
n
> > think of at a single place.
>
> There are lots of services that have that many transactions - DNS
> servers (even local ones), remote databases, etc.
>
> The point is that HTTP doesn't make the problem different, so this isn't
> an HTTP issue. It's a high rate server issue.
>
>
=E2=80=8BWhat makes HTTP different is that I expect most high-rate applicat=
ions
would exist in a context where the people running the servers and
applications have some amount of specialist experience and knowledge, or at
least an expectation that they're doing something that requires such
knowledge, in high-rate throughput.

HTTP is ubiquitous, and resides all along the traffic scale, from my
website (~no bits per second) to google.com (all the bits); and the slide
-- or sudden jump -- up that scale doesn't always correspond with acquiring
expertise in TCP stack tuning.=E2=80=8B



>
> > So probably you're starting to see the benefit of having a single doc
> > to concentrate all this.
> The same reason it's useful to have this all in one place is the reason
> we already do - there are books and courses on this.
>
>
=E2=80=8BMy users aren't getting my content right at the time my site is bo=
oming.
I've just been slashdotted/hackernewsed/whatever. Do I enroll in a course?
Buy a textbook and swot up (which I haven't done since I finished my IT
degree 15+ years ago)? Or do I hit up serverfault.com and bash the keyboard
until the fires go out?

Summary information is really important. Having it published by the same
people who published the protocol spec adds some serious cred, at least in
my eyes.=E2=80=8B



> >  You provided at least 3 different articles
> > to read and 2 or 3 different RFCs in addition to the original ones,
> > of course. A hosting provider whose web sites are down due to a lack
> > of tuning doesn't start to read many very long articles and even less
> > the most scientific ones, they need to find quick responses that they
> > can apply immediately (matter of minutes). So they launch google, they
> > type "web site dead, time-wait overflow" and they get plenty of
> > responses on stackoverflow and serverfault, many from people having
> > done the same in the past and repeating the same mistakes over and over=
.
>
> These people don't read RFCs to fix problems. They take online courses
> or read "how to" books - which do already exist in this space.
>
>
=E2=80=8BWhich people? I tend to google the error and see if there's some s=
ort of
consensus on stackoverflow. And then, as often as not, I have to advise my
sysadmins of a course of action because they know as much as me, or they
don't want to deal with my situation, or some other reason.=E2=80=8B



> > A document validated by several people and giving links for further
> > reading can help improve this situation.
> Those are the books and courses I'm talking about already.
> =E2=80=8B=E2=80=8B
>
> > People rediscover wheels because it's hard to find simple and accurate
> > information on the net.
> Nobody looks to RFCs to solve that problem...
>
> >  Basically you have the choice :
> >   - either uneducated blog posts saying "how I saved my web site using =
2
> >     sysctls"
> >   - or academic papers which are only understandable by scientific peop=
le
> >     having enough time
>
> ... that's what net FAQs are for, as well as courses and books.
>
> > At least the first ones have the merit of being easy to test, and since
> > they appear to work they are viral.
> >
> >>>  All of them became issues for
> >>> many web server admins who just copy-paste random settings from vario=
us
> >>> blogs found on the net who just copy the same stupidities over and ov=
er
> >>> resulting in the same trouble being caused to each of their reader.
> >> This doc is all over the place.
> >>
> >> If you want a doc to advise web admins, do so.
> > That's *exactly* what Daniel started to do when you told him he shouldn=
't
> > do it.
>
> I didn't say a doc to advise web admins wasn't useful. I said it wasn't
> an RFC.
>
> It's a web FAQ, a book, etc.
>
>
=E2=80=8BHere's the crux of the issue. What do you think an RFC is, that we
(apparently) don't? Why is an informational RFC not allowed to present the
same sort of information as an FAQ? (Isn't that what a BCP is?)

Personally I'd be happy if it was written up exactly like an FAQ, and
published as an informational RFC; because that tells me that this FAQ was
published by the IETF. It's not some random dude's unreliable blog full of
cargo-cult advice and anecdotes; it met the consensus of the organisation
that published the HTTP protocol spec. It's legitimate and reliable. And
one day, when it's out of date, it'll be updated or obsoleted by the new
consensus wisdom of the IETF.

That, as far as I know, doesn't defy the official definition of what an RFC
is*, nor does it devalue any existing (or future) RFCs.

And Google indexes RFCs. If this ends up being a really useful document
(which I imagine it would), with lots of inbound links and references,
people won't need to "look to RFCs to solve [their] problem", they'll do
what they already do -- look to Google, and Google will point them to this
RFC.


* heh=E2=80=8B



>
> > and I find it fantastic to see that this protocol
> > still scales so well.
> > But we need to consider modern usages of this protocol
> > for the web, and not just academic research and e-mail.
> You might consider that TCPM and TSVWG don't exist for just "academic
> research and e-mail". What do you think we've been doing for the past 40
> years?
>
>
=E2=80=8BDunno, I was only born 35-odd years ago. ;) Shall we talk about ge=
neration
gaps? Greybeards vs millennials? (Of which I'm neither, BTW. Not yet, at
least.)

Where we come from, how we find information to solve our problems, the way
we view the IETF and its RFCs are all different. If this argument is just
that this stuff doesn't belong in an RFC, that's a cultural issue and not
one I think we can resolve in this one technical working group.

Cheers
--=20
  Matthew Kerwin
  http://matthew.kerwin.net.au/

--001a113f8aaec5a037053a4cea10
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable


Hi folks, I'm stepping in here on just a couple of=
 points. I'll snip the bits I can't or won't talk to.

On 18 August 2016 at =
08:23, Joe Touch <touch@isi.edu> wrote:




On 8/17/2016 2:13 PM, Willy Tarreau wrote:

> On Wed, Aug 17, 2016 at 11:31:33AM -0700, Joe Touch wrote:

>>> It can be cited in new RFCs

>>> to justify certain choices.

>> Hmm. Like the refs I gave could be cited in this doc to justify *i=
ts*

>> choices? :-)

> I think it would be nice that this is cited, but to be clear on one
> point, I've never heard about your papers before you advertised th=
em

> here in this thread,



A search engine on the terms "TCP HTTP interaction" would =
have popped

them up rather quickly.



> and yet I've been dealing with timewait issues

> for 15 years like many people facing moderate to large web sites

> nowadays.



"
=E2=80=8B=E2=80=8B
timewait issue=
s" and we're the 5th hit in Google.


=E2=
=80=8B
Google, unless it's chan=
ged again recently, tailors search results for the user. My first page of h=
its for that query are all serverfault.c=
om, superuser.com, serverframework.com, stackoverflow.com, etc.

Guess where I get most of =
my advice.
=E2=80=8B
=C2=A0


>>>> Yes, and discussing those issues would be useful - but not=
 in this

>>>> document either.

>>> Why ? Lots of admins don't understand why the time_wait ti=
meout remains

>>> at 240 seconds on Solaris with people saying "if you want=
 to be conservative

>>> don't touch it but if you want to be modern simply shrink =
it to 30 seconds

>>> or so". People need to understand why advices have change=
d over 3 decades.

>> The advice hasn't really changed - the advice was given in the=
 99 ref,

>> which includes some cases where it can still be appropriate to dec=
rease

>> that timer.

> Most people see it the other way around : they see no valid case to *i=
ncrease*

> it beyond a few seconds, because for them the default value should be =
extremely

> low (ie this firewall vendor several years ago trying to insist on one=
 second).

> Yes that's really sad but that's reality. And you can tell the=
m to read 6191

> they won't care.



Most people's servers don't need to run fast enough to care =
(note that

nearly everyone runs some sort of web server on nearly every device,

whether for control or configuration). The only issue are high-volume

servers (the kind sysadmins deal with), and those people tend to already
know what the tradeoffs are and accept the risks.



=E2=
=80=8BYour sysadmins are not like my sysadmins. But these are generalisatio=
ns and anecdotes.=E2=80=8B

=C2=A0

>

>>>=C2=A0 =C2=A0- TCP timestamps: what they provide, what are the =
risks (some people in

>>>=C2=A0 =C2=A0 =C2=A0banking environments refuse to enable them =
so that they cannot be used

>>>=C2=A0 =C2=A0 =C2=A0as an oracle to help in timing attacks).
>> That's already covered in the security considerations of RFC 7=
323. How

>> is HTTP different, if at all, from any other app?

> HTTP is special in that it is fairly common to have to deal with tens =
of

> thousands of connections per second between one client and one server =
when

> you are on the server side, because you place a number of gateways (al=
so

> called reverse-proxies) which combine all of the possible issues you c=
an

> think of at a single place.



There are lots of services that have that many transactions - DNS
servers (even local ones), remote databases, etc.



The point is that HTTP doesn't make the problem different, so this isn&=
#39;t

an HTTP issue. It's a high rate server issue.




=E2=80=8BWhat makes HTTP different is that I expect most high-rate ap=
plications would exist in a context where the people running the servers an=
d applications have some amount of specialist experience and knowledge, or =
at least an expectation that they're doing something that requires such=
 knowledge, in high-rate throughput.

HTTP =
is ubiquitous, and resides all along the traffic scale, from my website (~n=
o bits per second) to google.com (all the=
 bits); and the slide -- or sudden jump -- up that scale doesn't always=
 correspond with acquiring expertise in TCP stack tuning.=E2=80=8B
=C2=A0


> So probably you're starting to see the benefit of having a sin=
gle doc

> to concentrate all this.


The same reason it's useful to have this all in one place i=
s the reason

we already do - there are books and courses on this.



=E2=
=80=8BMy users aren't getting my content right at the time my site is b=
ooming. I've just been slashdotted/hackernewsed/whatever. Do I enroll i=
n a course? Buy a textbook and swot up (which I haven't done since I fi=
nished my IT degree 15+ years ago)? Or do I hit up serverfault.com and bash the keyboard until the fires go out=
?

Summary information is really important.=
 Having it published by the same people who published the protocol spec add=
s some serious cred, at least in my eyes.=E2=80=8B

=C2=
=A0

>=C2=A0 You provided at least 3 different articles

> to read and 2 or 3 different RFCs in addition to the original ones,
> of course. A hosting provider whose web sites are down due to a lack
> of tuning doesn't start to read many very long articles and even l=
ess

> the most scientific ones, they need to find quick responses that they<=
br>
> can apply immediately (matter of minutes). So they launch google, they=


> type "web site dead, time-wait overflow" and they get plenty=
 of

> responses on stackoverflow and serverfault, many from people having
> done the same in the past and repeating the same mistakes over and ove=
r.



These people don't read RFCs to fix problems. They take online c=
ourses

or read "how to" books - which do already exist in this space.


=E2=
=80=8BWhich people? I tend to google the error and see if there's some =
sort of consensus on stackoverflow. And then, as often as not, I have to ad=
vise my sysadmins of a course of action because they know as much as me, or=
 they don't want to deal with my situation, or some other reason.=E2=80=
=8B

=C2=A0

> A document validated by several people and giving links for further
> reading can help improve this situation.

Those are the books and courses I'm talking about already.
=E2=80=8B=E2=80=8B


> People rediscover wheels because it's hard to find simple and accu=
rate

> information on the net.

Nobody looks to RFCs to solve that problem...



>=C2=A0 Basically you have the choice :

>=C2=A0 =C2=A0- either uneducated blog posts saying "how I saved my=
 web site using 2

>=C2=A0 =C2=A0 =C2=A0sysctls"

>=C2=A0 =C2=A0- or academic papers which are only understandable by scie=
ntific people

>=C2=A0 =C2=A0 =C2=A0having enough time



... that's what net FAQs are for, as well as courses and books.<=
br>


> At least the first ones have the merit of being easy to test, and sinc=
e

> they appear to work they are viral.

>

>>>=C2=A0 All of them became issues for

>>> many web server admins who just copy-paste random settings fro=
m various

>>> blogs found on the net who just copy the same stupidities over=
 and over

>>> resulting in the same trouble being caused to each of their re=
ader.

>> This doc is all over the place.

>>

>> If you want a doc to advise web admins, do so.

> That's *exactly* what Daniel started to do when you told him he sh=
ouldn't

> do it.



I didn't say a doc to advise web admins wasn't useful. I sai=
d it wasn't

an RFC.



It's a web FAQ, a book, etc.



=E2=
=80=8BHere's the crux of the issue. What do you think an RFC is, that w=
e (apparently) don't? Why is an informational RFC not allowed to presen=
t the same sort of information as an FAQ? (Isn't that what a BCP is?)

Personally I'd be happy if it was writt=
en up exactly like an FAQ, and published as an informational RFC; because t=
hat tells me that this FAQ was published by the IETF. It's not some ran=
dom dude's unreliable blog full of cargo-cult advice and anecdotes; it =
met the consensus of the organisation that published the HTTP protocol spec=
. It's legitimate and reliable. And one day, when it's out of date,=
 it'll be updated or obsoleted by the new consensus wisdom of the IETF.=


That, as far as I know, doesn't defy =
the official definition of what an RFC is*, nor does it devalue any existin=
g (or future) RFCs.

And Google indexes RFC=
s. If this ends up being a really useful document (which I imagine it would=
), with lots of inbound links and references, people won't need to &quo=
t;look to RFCs to solve [their] problem", they'll do what they alr=
eady do -- look to Google, and Google will point them to this RFC.


* heh=E2=80=8B

<=
div>=C2=A0


> and I find it fantastic to see that this protocol

> still scales so well.

> But we need to consider modern usages of this protocol

> for the web, and not just academic research and e-mail.

You might consider that TCPM and TSVWG don't exist for just &quo=
t;academic

research and e-mail". What do you think we've been doing for the p=
ast 40

years?



=E2=
=80=8BDunno, I was only born 35-odd years ago. ;) Shall we talk about gener=
ation gaps? Greybeards vs millennials? (Of which I'm neither, BTW. Not =
yet, at least.)

Where we come from, =
how we find information to solve our problems, the way we view the IETF and=
 its RFCs are all different. If this argument is just that this stuff doesn=
't belong in an RFC, that's a cultural issue and not one I think we=
 can resolve in this one technical working group.

<=
div class=3D"gmail_default" style=3D"font-family:georgia,serif;color:rgb(7,=
55,99)">Cheers
-- 
=C2=A0 Matthew Kerwin
=C2=A0 http://matthew.ke=
rwin.net.au/



--001a113f8aaec5a037053a4cea10--


From nobody Wed Aug 17 17:22:27 2016
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4BAD212D75D for ; Wed, 17 Aug 2016 17:22:26 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -8.166
X-Spam-Level: 
X-Spam-Status: No, score=-8.166 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-1.247] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id c3bmy0D2Qqdu for ; Wed, 17 Aug 2016 17:22:25 -0700 (PDT)
Received: from boreas.isi.edu (boreas.isi.edu [128.9.160.161]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 175B912D740 for ; Wed, 17 Aug 2016 17:22:25 -0700 (PDT)
Received: from [192.168.1.189] (cpe-172-250-251-17.socal.res.rr.com [172.250.251.17]) (authenticated bits=0) by boreas.isi.edu (8.13.8/8.13.8) with ESMTP id u7I0LNGJ015083 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Wed, 17 Aug 2016 17:21:24 -0700 (PDT)
To: Matthew Kerwin 
References: <0CC24FC1-37E1-4125-9627-05726A9D9406@mnot.net> <7fa95741-ac58-3183-1b92-238bd4b4dae6@isi.edu> <5CD67877-19E3-4E79-BBF2-3E270343A378@mnot.net> <2197232f-10d7-28cb-fcc9-05bd495e3c22@isi.edu> <20160817064545.GD16017@1wt.eu> <7f7b129c-f156-d067-bef8-4a2213f461ac@isi.edu> <20160817180802.GA16773@1wt.eu> <4ab7c5b0-3722-1346-f481-a8d76de70034@isi.edu> <20160817211317.GA16929@1wt.eu>  
From: Joe Touch 
Message-ID: <27b58b64-48cd-39af-78b3-ef583c585fa6@isi.edu>
Date: Wed, 17 Aug 2016 17:21:21 -0700
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0
MIME-Version: 1.0
In-Reply-To: 
Content-Type: multipart/alternative; boundary="------------F10D1AF5F07272054D777B16"
X-ISI-4-43-8-MailScanner: Found to be clean
X-MailScanner-From: touch@isi.edu
Archived-At: 
Cc: tcpm@ietf.org, Daniel Stenberg , Mark Nottingham , Patrick McManus , Willy Tarreau , HTTP Working Group 
Subject: Re: [tcpm] TCP Tuning for HTTP - update
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Thu, 18 Aug 2016 00:22:26 -0000

This is a multi-part message in MIME format.
--------------F10D1AF5F07272054D777B16
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

Popping up to the key point...

On 8/17/2016 4:35 PM, Matthew Kerwin wrote:
> Hi folks, I'm stepping in here on just a couple of points. I'll snip
> the bits I can't or won't talk to.
...
> Where we come from, how we find information to solve our problems, the
> way we view the IETF and its RFCs are all different. If this argument
> is just that this stuff doesn't belong in an RFC, that's a cultural
> issue and not one I think we can resolve in this one technical working
> group.

IMO, RFCs deal with *protocols* and protocol deployment issues.

Here that would mean for HTTP in specific
    - turn Nagle off
    - watch out for ACK compression effects (turn it off in favor of ABC
if you can)
    - deal with slow-start restart (there are a variety of approaches here)

You also want to be generally efficient with TCP, which isn't related to
HTTP but still good advice for all apps:
    - be generally efficient and robust (use SACK, ECN, good window
scaling, etc.)
    - watch out for resource overloads (use SYN cookies/fast open,
time-wait buildup, etc.)

That advice is useful for all TCP for transactions, too.

Here's what it definitely does not mean:
    - setting socket buffer sizes (that's an OS-ism, and may be useful
for app designers to know, but isn''t directly part of TCP or HTTP)
    - running an efficient server for small connections (that's a pure
implementation performance issue, and depends on OS support for things
like threads, process/thread pools, etc.)
 
I.e., OS-specific issues in running servers are not in scope IMO.

Keep in mind who reads RFCs - mostly protocol people. Managers take
training courses to get certification, and that's where they learn their
operational issues.

Joe

--------------F10D1AF5F07272054D777B16
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: 8bit


  
    
  
  
    
Popping up to the key point...

    

    On 8/17/2016 4:35 PM, Matthew Kerwin wrote:

    

      

        
Hi folks, I'm stepping in here on just
          a couple of points. I'll snip the bits I can't or won't talk
          to.

      

    

    ...
    

      

        

          
Where
            we come from, how we find information to solve our problems,
            the way we view the IETF and its RFCs are all different. If
            this argument is just that this stuff doesn't belong in an
            RFC, that's a cultural issue and not one I think we can
            resolve in this one technical working group.

          

        

      

    

    

    IMO, RFCs deal with *protocols* and protocol deployment issues.

    

    Here that would mean for HTTP in specific

        - turn Nagle off

        - watch out for ACK compression effects (turn it off in favor of
    ABC if you can)

        - deal with slow-start restart (there are a variety of
    approaches here)

    

    You also want to be generally efficient with TCP, which isn't
    related to HTTP but still good advice for all apps:

        - be generally efficient and robust (use SACK, ECN, good window
    scaling, etc.)

        - watch out for resource overloads (use SYN cookies/fast open,
    time-wait buildup, etc.)

    

    That advice is useful for all TCP for transactions, too.

    

    Here's what it definitely does not mean:

        - setting socket buffer sizes (that's an OS-ism, and may be
    useful for app designers to know, but isn''t directly part of TCP or
    HTTP)

        - running an efficient server for small connections (that's a
    pure implementation performance issue, and depends on OS support for
    things like threads, process/thread pools, etc.)

      

    I.e., OS-specific issues in running servers are not in scope IMO.

    

    Keep in mind who reads RFCs - mostly protocol people. Managers take
    training courses to get certification, and that's where they learn
    their operational issues.

    

    Joe

  


--------------F10D1AF5F07272054D777B16--


From nobody Wed Aug 17 17:51:59 2016
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 172DD12D598 for ; Wed, 17 Aug 2016 17:51:58 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.602
X-Spam-Level: 
X-Spam-Status: No, score=-2.602 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xbQgKecL-z3H for ; Wed, 17 Aug 2016 17:51:55 -0700 (PDT)
Received: from mxout-07.mxes.net (mxout-07.mxes.net [216.86.168.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id B5246128E19 for ; Wed, 17 Aug 2016 17:51:55 -0700 (PDT)
Received: from [192.168.3.104] (unknown [124.189.98.244]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by smtp.mxes.net (Postfix) with ESMTPSA id 7CB5122E1FA; Wed, 17 Aug 2016 20:51:42 -0400 (EDT)
Content-Type: text/plain; charset=windows-1252
Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\))
From: Mark Nottingham 
In-Reply-To: 
Date: Thu, 18 Aug 2016 10:51:39 +1000
Content-Transfer-Encoding: quoted-printable
Message-Id: 
References: <0CC24FC1-37E1-4125-9627-05726A9D9406@mnot.net> <7fa95741-ac58-3183-1b92-238bd4b4dae6@isi.edu> <5CD67877-19E3-4E79-BBF2-3E270343A378@mnot.net> <2197232f-10d7-28cb-fcc9-05bd495e3c22@isi.edu> <87D56B16-B091-4E9C-9B88-D136ED4BC819@mnot.net> 
To: Joe Touch 
X-Mailer: Apple Mail (2.3124)
Archived-At: 
Cc: tcpm@ietf.org, HTTP Working Group , Daniel Stenberg , Patrick McManus 
Subject: Re: [tcpm] TCP Tuning for HTTP - update
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Thu, 18 Aug 2016 00:51:58 -0000

The AD has spoken, and I think we're rapidly approaching the point of =
being off-topic here. I'll respond to your question below, and of course =
if you change your mind and do want to follow up with a helpful list of =
suggested references, you're most welcome.


> On 18 Aug 2016, at 1:08 AM, Joe Touch  wrote:
>=20
>>>> If that's the case, I'd observe that the IETF isn't an academic =
publisher, and acknowledging all prior work in an area is neither =
practical, nor required, nor current practice.
>>> Plagiarism isn't an issue limited to academic environments. =
Publication
>>> of a document on the web is still publication.
>> Sure. It also isn't a legal issue in this form (unless you're =
asserting copyright?). Effectively, it's a cultural norm. Again, I will =
point out that in the culture of the IETF, we historically have not =
cited the complete provenance of every idea, both because it's =
impractical and because it doesn't benefit the reader.=20
>=20
> Although that's true in the smallest cases, the IETF does have two
> concepts that support this norm: an author list and a set of =
references.
>=20
> Can you explain how it helps the reader to not cite two documents that
> are both squarely in the same area as this doc (interaction between =
HTTP
> and TCP and the impact of running many small connections closed at the
> client as for HTTP)?

Oh, it can be very helpful; I don't think anyone disputes that =
appropriate references are helpful.=20

Whether or not specific documents -- such as yours -- should be =
referenced is a matter for editorial discretion and eventually WG =
consensus. =20

I note that no one else has supported your claims, but others have said =
(e.g., [1]) that your work probably isn't appropriate for the intended =
audience (HTTP implementers and administrators).

Directing Daniel to add citations to mollify you at this point would be =
entirely inappropriate, because it would give an incentive to others who =
want their works cited without full review to make similar claims -- =
whether that be due to a genuine desire to help, a desire to increase =
their h-index, or to bolster an IPR claim against a technology.

Or, of course, Daniel could use his editorial discretion to decide to =
cite your work before the WG adopts it; however, I'd very much =
understand if he didn't want to do so at this point.

Regards,


1. http://www.w3.org/mid/20160817064545.GD16017@1wt.eu

--
Mark Nottingham   https://www.mnot.net/


From nobody Wed Aug 17 18:00:10 2016
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 997AE12D62A for ; Wed, 17 Aug 2016 18:00:09 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -8.167
X-Spam-Level: 
X-Spam-Status: No, score=-8.167 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-1.247] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Bw_qirXQ30SP for ; Wed, 17 Aug 2016 18:00:08 -0700 (PDT)
Received: from boreas.isi.edu (boreas.isi.edu [128.9.160.161]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 3401512D0BC for ; Wed, 17 Aug 2016 18:00:08 -0700 (PDT)
Received: from [192.168.1.189] (cpe-172-250-251-17.socal.res.rr.com [172.250.251.17]) (authenticated bits=0) by boreas.isi.edu (8.13.8/8.13.8) with ESMTP id u7I0xX8w029196 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Wed, 17 Aug 2016 17:59:34 -0700 (PDT)
To: Mark Nottingham 
References: <0CC24FC1-37E1-4125-9627-05726A9D9406@mnot.net> <7fa95741-ac58-3183-1b92-238bd4b4dae6@isi.edu> <5CD67877-19E3-4E79-BBF2-3E270343A378@mnot.net> <2197232f-10d7-28cb-fcc9-05bd495e3c22@isi.edu> <87D56B16-B091-4E9C-9B88-D136ED4BC819@mnot.net>  
From: Joe Touch 
Message-ID: 
Date: Wed, 17 Aug 2016 17:59:30 -0700
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0
MIME-Version: 1.0
In-Reply-To: 
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: 7bit
X-ISI-4-43-8-MailScanner: Found to be clean
X-MailScanner-From: touch@isi.edu
Archived-At: 
Cc: tcpm@ietf.org, HTTP Working Group , Daniel Stenberg , Patrick McManus 
Subject: Re: [tcpm] TCP Tuning for HTTP - update
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Thu, 18 Aug 2016 01:00:09 -0000

Mark,

My point is simple: this work is useless if it fails to cite the
relevant prior work that explains the rationale behind its recommendations.

Mine isn't the only work not being cited here - there's all the work in
TCP tuning from PSC, many web workshops from the 90s, and numerous RFCs
and books. I've pointed out just a few.

I don't care to do the author's homework, but as AD I'm disappointed you
don't seem to care about that either.

Joe


On 8/17/2016 5:51 PM, Mark Nottingham wrote:
> The AD has spoken, and I think we're rapidly approaching the point of being off-topic here. I'll respond to your question below, and of course if you change your mind and do want to follow up with a helpful list of suggested references, you're most welcome.
>
>
>> On 18 Aug 2016, at 1:08 AM, Joe Touch  wrote:
>>
>>>>> If that's the case, I'd observe that the IETF isn't an academic publisher, and acknowledging all prior work in an area is neither practical, nor required, nor current practice.
>>>> Plagiarism isn't an issue limited to academic environments. Publication
>>>> of a document on the web is still publication.
>>> Sure. It also isn't a legal issue in this form (unless you're asserting copyright?). Effectively, it's a cultural norm. Again, I will point out that in the culture of the IETF, we historically have not cited the complete provenance of every idea, both because it's impractical and because it doesn't benefit the reader. 
>> Although that's true in the smallest cases, the IETF does have two
>> concepts that support this norm: an author list and a set of references.
>>
>> Can you explain how it helps the reader to not cite two documents that
>> are both squarely in the same area as this doc (interaction between HTTP
>> and TCP and the impact of running many small connections closed at the
>> client as for HTTP)?
> Oh, it can be very helpful; I don't think anyone disputes that appropriate references are helpful. 
>
> Whether or not specific documents -- such as yours -- should be referenced is a matter for editorial discretion and eventually WG consensus.  
>
> I note that no one else has supported your claims, but others have said (e.g., [1]) that your work probably isn't appropriate for the intended audience (HTTP implementers and administrators).
>
> Directing Daniel to add citations to mollify you at this point would be entirely inappropriate, because it would give an incentive to others who want their works cited without full review to make similar claims -- whether that be due to a genuine desire to help, a desire to increase their h-index, or to bolster an IPR claim against a technology.
>
> Or, of course, Daniel could use his editorial discretion to decide to cite your work before the WG adopts it; however, I'd very much understand if he didn't want to do so at this point.
>
> Regards,
>
>
> 1. http://www.w3.org/mid/20160817064545.GD16017@1wt.eu
>
> --
> Mark Nottingham   https://www.mnot.net/
>


From nobody Wed Aug 17 18:10:05 2016
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D51F312D51D for ; Wed, 17 Aug 2016 18:10:02 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -8.167
X-Spam-Level: 
X-Spam-Status: No, score=-8.167 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-1.247] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bEPzi7JqxObM for ; Wed, 17 Aug 2016 18:10:02 -0700 (PDT)
Received: from boreas.isi.edu (boreas.isi.edu [128.9.160.161]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1C45B12D0FD for ; Wed, 17 Aug 2016 18:10:02 -0700 (PDT)
Received: from [192.168.1.189] (cpe-172-250-251-17.socal.res.rr.com [172.250.251.17]) (authenticated bits=0) by boreas.isi.edu (8.13.8/8.13.8) with ESMTP id u7I19mBA001181 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Wed, 17 Aug 2016 18:09:49 -0700 (PDT)
To: Mark Nottingham 
References: <0CC24FC1-37E1-4125-9627-05726A9D9406@mnot.net> <7fa95741-ac58-3183-1b92-238bd4b4dae6@isi.edu> <5CD67877-19E3-4E79-BBF2-3E270343A378@mnot.net> <2197232f-10d7-28cb-fcc9-05bd495e3c22@isi.edu> <87D56B16-B091-4E9C-9B88-D136ED4BC819@mnot.net>   
From: Joe Touch 
Message-ID: 
Date: Wed, 17 Aug 2016 18:09:45 -0700
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0
MIME-Version: 1.0
In-Reply-To: 
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: 7bit
X-ISI-4-43-8-MailScanner: Found to be clean
X-MailScanner-From: touch@isi.edu
Archived-At: 
Cc: tcpm@ietf.org, HTTP Working Group , Daniel Stenberg , Patrick McManus 
Subject: Re: [tcpm] TCP Tuning for HTTP - update
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Thu, 18 Aug 2016 01:10:04 -0000

On 8/17/2016 5:59 PM, Joe Touch wrote:
> Mark,
>
> ...
> I don't care to do the author's homework, but as AD ...
WG chair.

Sorry for the continued confusion on that.

Joe



From nobody Wed Aug 17 18:16:37 2016
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id BE59C12D51D for ; Wed, 17 Aug 2016 18:16:35 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -8.167
X-Spam-Level: 
X-Spam-Status: No, score=-8.167 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-1.247] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id BJkrxJOktPL3 for ; Wed, 17 Aug 2016 18:16:35 -0700 (PDT)
Received: from boreas.isi.edu (boreas.isi.edu [128.9.160.161]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id F3ABA12D0FD for ; Wed, 17 Aug 2016 18:16:31 -0700 (PDT)
Received: from [192.168.1.189] (cpe-172-250-251-17.socal.res.rr.com [172.250.251.17]) (authenticated bits=0) by boreas.isi.edu (8.13.8/8.13.8) with ESMTP id u7I1FxjU002563 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Wed, 17 Aug 2016 18:16:00 -0700 (PDT)
To: Mark Nottingham 
References: <0CC24FC1-37E1-4125-9627-05726A9D9406@mnot.net> <7fa95741-ac58-3183-1b92-238bd4b4dae6@isi.edu> <5CD67877-19E3-4E79-BBF2-3E270343A378@mnot.net> <2197232f-10d7-28cb-fcc9-05bd495e3c22@isi.edu> <87D56B16-B091-4E9C-9B88-D136ED4BC819@mnot.net>  
From: Joe Touch 
Message-ID: 
Date: Wed, 17 Aug 2016 18:15:56 -0700
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0
MIME-Version: 1.0
In-Reply-To: 
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: 7bit
X-ISI-4-43-8-MailScanner: Found to be clean
X-MailScanner-From: touch@isi.edu
Archived-At: 
Cc: tcpm@ietf.org, HTTP Working Group , Daniel Stenberg , Patrick McManus 
Subject: Re: [tcpm] TCP Tuning for HTTP - update
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Thu, 18 Aug 2016 01:16:36 -0000

One other point...

On 8/17/2016 5:51 PM, Mark Nottingham wrote:
> The AD has spoken,
The only post from the AD so far was to ask why I didn't just point out
the relevant citations before.

Which, as I already replied, I had - many months ago.

Joe


From nobody Wed Aug 17 19:30:41 2016
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 57151128E18 for ; Wed, 17 Aug 2016 19:30:38 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -8.167
X-Spam-Level: 
X-Spam-Status: No, score=-8.167 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-1.247] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ljo3G1ts9uAK for ; Wed, 17 Aug 2016 19:30:35 -0700 (PDT)
Received: from boreas.isi.edu (boreas.isi.edu [128.9.160.161]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id DB6F912D587 for ; Wed, 17 Aug 2016 19:30:35 -0700 (PDT)
Received: from [192.168.1.189] (cpe-172-250-251-17.socal.res.rr.com [172.250.251.17]) (authenticated bits=0) by boreas.isi.edu (8.13.8/8.13.8) with ESMTP id u7I2U0dC014338 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Wed, 17 Aug 2016 19:30:02 -0700 (PDT)
To: Mark Nottingham 
References: <0CC24FC1-37E1-4125-9627-05726A9D9406@mnot.net> <7fa95741-ac58-3183-1b92-238bd4b4dae6@isi.edu> <5CD67877-19E3-4E79-BBF2-3E270343A378@mnot.net> <2197232f-10d7-28cb-fcc9-05bd495e3c22@isi.edu> <87D56B16-B091-4E9C-9B88-D136ED4BC819@mnot.net>  
From: Joe Touch 
Message-ID: <9eb63b81-a9bc-cb65-913a-d9a5af0d2fe2@isi.edu>
Date: Wed, 17 Aug 2016 19:29:58 -0700
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0
MIME-Version: 1.0
In-Reply-To: 
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: 7bit
X-ISI-4-43-8-MailScanner: Found to be clean
X-MailScanner-From: touch@isi.edu
Archived-At: 
Cc: tcpm@ietf.org, HTTP Working Group , Daniel Stenberg , Patrick McManus 
Subject: Re: [tcpm] TCP Tuning for HTTP - update
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Thu, 18 Aug 2016 02:30:38 -0000

On the point of references:


On 8/17/2016 5:51 PM, Mark Nottingham wrote:
> ...
> Whether or not specific documents -- such as yours -- should be referenced is a matter for editorial discretion and eventually WG consensus.  
>
> I note that no one else has supported your claims, but others have said (e.g., [1]) that your work probably isn't appropriate for the intended audience (HTTP implementers and administrators).
Others have claimed that the papers I cited are not a sufficient
replacement for this document, but the only claim made to not cite them
at all is based on "RFCs don't need to cite sources or detail" - which
nearly ever RFC published disproves. Citations serve many purposes, esp.
informational ones - to provide detailed background information and
context among them.

Note that if this doc is limited to *protocol* recommendations (i.e.,
the scope of the IETF), I agree that it remains useful. My primary
objection to the document itself is it includes far too many OS
operational considerations that are out of scope for the IETF.

> Directing Daniel to add citations to mollify you at this point would be entirely inappropriate, because it would give an incentive to others who want their works cited without full review to make similar claims -- whether that be due to a genuine desire to help, a desire to increase their h-index, or to bolster an IPR claim against a technology.
I agree that directing anyone to cite someone for mollification would be
inappropriate; I never asked for that.

There are plenty of ways that citation considerations avoid the issues
you raise:
    - does the document provide informational background?
        are you claiming that the docs I cite do not?
    - is it the original or most complete reference (it can be useful to
cite surveys rather than original literature where the surveys cite the
originals)
        have you found an earlier or more complete reference?

These considerations prevent people from merely claiming that their work
should be gratuitously cited.

I made my claim in the original post back in March - the bulk of the
actual TCP interactions are discussed in more detail with rationale in
the one document, and the same is true for the TIMEWAIT issue for the
second document.

The only argument I've seen put forth is that "RFCs don't need to cite
things", which is false by nearly every RFC published.

Joe


From nobody Wed Aug 17 19:52:08 2016
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6AB9E12D130 for ; Wed, 17 Aug 2016 19:52:06 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.602
X-Spam-Level: 
X-Spam-Status: No, score=-2.602 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=unavailable autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id eDqh52QRVPma for ; Wed, 17 Aug 2016 19:52:05 -0700 (PDT)
Received: from mxout-07.mxes.net (mxout-07.mxes.net [216.86.168.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 330B412B041 for ; Wed, 17 Aug 2016 19:43:29 -0700 (PDT)
Received: from [192.168.3.104] (unknown [124.189.98.244]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by smtp.mxes.net (Postfix) with ESMTPSA id 2290B22E255; Wed, 17 Aug 2016 22:43:23 -0400 (EDT)
Content-Type: text/plain; charset=windows-1252
Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\))
From: Mark Nottingham 
In-Reply-To: <9eb63b81-a9bc-cb65-913a-d9a5af0d2fe2@isi.edu>
Date: Thu, 18 Aug 2016 12:43:21 +1000
Content-Transfer-Encoding: quoted-printable
Message-Id: <781AFF22-3532-44A0-AB14-628FF86BE866@mnot.net>
References: <0CC24FC1-37E1-4125-9627-05726A9D9406@mnot.net> <7fa95741-ac58-3183-1b92-238bd4b4dae6@isi.edu> <5CD67877-19E3-4E79-BBF2-3E270343A378@mnot.net> <2197232f-10d7-28cb-fcc9-05bd495e3c22@isi.edu> <87D56B16-B091-4E9C-9B88-D136ED4BC819@mnot.net>   <9eb63b81-a9bc-cb65-913a-d9a5af0d2fe2@isi.edu>
To: Joe Touch 
X-Mailer: Apple Mail (2.3124)
Archived-At: 
Cc: tcpm@ietf.org, HTTP Working Group , Daniel Stenberg , Patrick McManus 
Subject: Re: [tcpm] TCP Tuning for HTTP - update
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Thu, 18 Aug 2016 02:52:06 -0000

> On 18 Aug 2016, at 12:29 PM, Joe Touch  wrote:

> There are plenty of ways that citation considerations avoid the issues
> you raise:
>    - does the document provide informational background?
>        are you claiming that the docs I cite do not?
>    - is it the original or most complete reference (it can be useful =
to
> cite surveys rather than original literature where the surveys cite =
the
> originals)
>        have you found an earlier or more complete reference?
>=20
> These considerations prevent people from merely claiming that their =
work
> should be gratuitously cited.

Sure, and if the WG adopts the document, we can have those discussions. =
Adopting the document does not mean that we're going to rubber-stamp its =
content; it's just being taken as a starting point.

In the meantime, let's discuss the scope of work being proposed and =
whether it's appropriate, what modifications might be needed, etc., so =
as to inform the decision to adopt.

I'd especially like to hear from other people in the TCP community; =
we've heard from Joe and Michael; do others share their opinions, or =
have another view?


> I made my claim in the original post back in March - the bulk of the
> actual TCP interactions are discussed in more detail with rationale in
> the one document, and the same is true for the TIMEWAIT issue for the
> second document.
>=20
> The only argument I've seen put forth is that "RFCs don't need to cite
> things", which is false by nearly every RFC published.

Now you're misrepresenting what others have said.  Please stop.

--
Mark Nottingham   https://www.mnot.net/


From nobody Wed Aug 17 20:20:08 2016
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8423112D857 for ; Wed, 17 Aug 2016 20:20:07 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -8.166
X-Spam-Level: 
X-Spam-Status: No, score=-8.166 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-1.247] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lVrnZWUYHYVK for ; Wed, 17 Aug 2016 20:20:06 -0700 (PDT)
Received: from boreas.isi.edu (boreas.isi.edu [128.9.160.161]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 6276E12D58A for ; Wed, 17 Aug 2016 20:20:06 -0700 (PDT)
Received: from [192.168.1.189] (cpe-172-250-251-17.socal.res.rr.com [172.250.251.17]) (authenticated bits=0) by boreas.isi.edu (8.13.8/8.13.8) with ESMTP id u7I3JmDm023134 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Wed, 17 Aug 2016 20:19:50 -0700 (PDT)
To: Mark Nottingham 
References: <0CC24FC1-37E1-4125-9627-05726A9D9406@mnot.net> <7fa95741-ac58-3183-1b92-238bd4b4dae6@isi.edu> <5CD67877-19E3-4E79-BBF2-3E270343A378@mnot.net> <2197232f-10d7-28cb-fcc9-05bd495e3c22@isi.edu> <87D56B16-B091-4E9C-9B88-D136ED4BC819@mnot.net>   <9eb63b81-a9bc-cb65-913a-d9a5af0d2fe2@isi.edu> <781AFF22-3532-44A0-AB14-628FF86BE866@mnot.net>
From: Joe Touch 
Message-ID: <53e6ee78-f308-0618-0a83-0da28c29d2fb@isi.edu>
Date: Wed, 17 Aug 2016 20:19:45 -0700
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0
MIME-Version: 1.0
In-Reply-To: <781AFF22-3532-44A0-AB14-628FF86BE866@mnot.net>
Content-Type: multipart/alternative; boundary="------------3DF91FCEB386408A0C7120EA"
X-ISI-4-43-8-MailScanner: Found to be clean
X-MailScanner-From: touch@isi.edu
Archived-At: 
Cc: tcpm@ietf.org, HTTP Working Group , Daniel Stenberg , Patrick McManus 
Subject: Re: [tcpm] TCP Tuning for HTTP - update
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Thu, 18 Aug 2016 03:20:07 -0000

This is a multi-part message in MIME format.
--------------3DF91FCEB386408A0C7120EA
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: 7bit



On 8/17/2016 7:43 PM, Mark Nottingham wrote:

>> > The only argument I've seen put forth is that "RFCs don't need to cite
>> > things", which is false by nearly every RFC published.
> Now you're misrepresenting what others have said.  Please stop.
Here's a list of everyone I can find from the list who has commented on
the issue of citation.

I agree that the quote above misrepresents your position, and I
apologize for that.

I leave the sum of everyone's words - hopefully quoted with enough
context to remain accurate - as context for how to proceed.

Joe

--------

You:

Again, I will point out that in the culture of the IETF, we historically have not cited the complete provenance of every idea, both because it's impractical and because it doesn't benefit the reader.

{and in a later post:)

Whether or not specific documents -- such as yours -- should be referenced is a matter for editorial discretion and eventually WG consensus.  

I note that no one else has supported your claims, but others have said (e.g., [1]) that your work probably isn't appropriate for the intended audience (HTTP implementers and administrators).

{NB: you cite Willy, below)

---

Eliot Lear:

Perhaps we can agree that the reasonable course of action here is for
Joe to (re)-recommend a compact set of citations to the authors, perhaps
even in some easily consumable form to them (kramdown-2629 or XML)?

--

Tim Wicinsk (in ref to Eliot's post):

That sounds like a fine idea.  I'll be glad to go through those.


--
Alexy Melnikov:

Instead of starting your discussion with words like "plagiarism", you 
could have just asked for information to be clarified and a 
citation/acknowledgement added?

--


Willy Tarreau:

I've just checked the two documents you referenced. They seem to be very
well detailed but they are *scientific* research. ...
...

... Thus I think that Daniel's work completes quite well what you've
done in that it directly addresses people's concerns without requiring the
scientific background.

{and later, addressing the references I asked to be cited)

I think it would be nice that this is cited, ...

...
A document validated by several people and giving links for further
reading can help improve this situation.
...
That's why I think we should encourage him to
continue and give him all the reference he needs to have an undisputably
good reference doc.


--


--------------3DF91FCEB386408A0C7120EA
Content-Type: text/html; charset=windows-1252
Content-Transfer-Encoding: 8bit


  
    
  
  
    


    

    

    
On 8/17/2016 7:43 PM, Mark Nottingham
      wrote:

      

    

    

      

        
> The only argument I've seen put forth is that "RFCs don't need to cite
> things", which is false by nearly every RFC published.


      

      
Now you're misrepresenting what others have said.  Please stop.

    

    Here's a list of everyone I can find from the list who has commented
    on the issue of citation. 

    

    I agree that the quote above misrepresents your position, and I
    apologize for that.

    

    I leave the sum of everyone's words - hopefully quoted with enough
    context to remain accurate - as context for how to proceed.

    

    Joe

    

    --------

    

    You:

    
Again, I will point out that in the culture of the IETF, we historically have not cited the complete provenance of every idea, both because it's impractical and because it doesn't benefit the reader.

{and in a later post:)

Whether or not specific documents -- such as yours -- should be referenced is a matter for editorial discretion and eventually WG consensus.  

I note that no one else has supported your claims, but others have said (e.g., [1]) that your work probably isn't appropriate for the intended audience (HTTP implementers and administrators).

{NB: you cite Willy, below)


    ---

    

    Eliot Lear:

    
Perhaps we can agree that the reasonable course of action here is for
Joe to (re)-recommend a compact set of citations to the authors, perhaps
even in some easily consumable form to them (kramdown-2629 or XML)?

--

Tim Wicinsk (in ref to Eliot's post):

That sounds like a fine idea. I'll be glad to go through those.


--
Alexy Melnikov:

Instead of starting your discussion with words like "plagiarism", you 
could have just asked for information to be clarified and a 
citation/acknowledgement added?

--


Willy Tarreau:

I've just checked the two documents you referenced. They seem to be very
well detailed but they are *scientific* research. ...
...

... Thus I think that Daniel's work completes quite well what you've
done in that it directly addresses people's concerns without requiring the
scientific background.

{and later, addressing the references I asked to be cited)

I think it would be nice that this is cited, ...

...
A document validated by several people and giving links for further
reading can help improve this situation.
...
That's why I think we should encourage him to
continue and give him all the reference he needs to have an undisputably
good reference doc.


--



  


--------------3DF91FCEB386408A0C7120EA--


From nobody Wed Aug 17 22:38:50 2016
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3639712B023 for ; Wed, 17 Aug 2016 22:38:49 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.902
X-Spam-Level: 
X-Spam-Status: No, score=-1.902 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id CJI7YPk030uG for ; Wed, 17 Aug 2016 22:38:47 -0700 (PDT)
Received: from 1wt.eu (wtarreau.pck.nerim.net [62.212.114.60]) by ietfa.amsl.com (Postfix) with ESMTP id 600F212B015 for ; Wed, 17 Aug 2016 22:38:45 -0700 (PDT)
Received: (from willy@localhost) by pcw.home.local (8.15.2/8.15.2/Submit) id u7I5cbuM017303; Thu, 18 Aug 2016 07:38:37 +0200
Date: Thu, 18 Aug 2016 07:38:37 +0200
From: Willy Tarreau 
To: Joe Touch 
Message-ID: <20160818053837.GC16773@1wt.eu>
References: <5CD67877-19E3-4E79-BBF2-3E270343A378@mnot.net> <2197232f-10d7-28cb-fcc9-05bd495e3c22@isi.edu> <20160817064545.GD16017@1wt.eu> <7f7b129c-f156-d067-bef8-4a2213f461ac@isi.edu> <20160817180802.GA16773@1wt.eu> <4ab7c5b0-3722-1346-f481-a8d76de70034@isi.edu> <20160817211317.GA16929@1wt.eu>   <27b58b64-48cd-39af-78b3-ef583c585fa6@isi.edu>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <27b58b64-48cd-39af-78b3-ef583c585fa6@isi.edu>
User-Agent: Mutt/1.6.0 (2016-04-01)
Archived-At: 
Cc: tcpm@ietf.org, Matthew Kerwin , Daniel Stenberg , Mark Nottingham , HTTP Working Group , Patrick McManus 
Subject: Re: [tcpm] TCP Tuning for HTTP - update
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Thu, 18 Aug 2016 05:38:49 -0000

On Wed, Aug 17, 2016 at 05:21:21PM -0700, Joe Touch wrote:
> Popping up to the key point...
> 
> On 8/17/2016 4:35 PM, Matthew Kerwin wrote:
> > Hi folks, I'm stepping in here on just a couple of points. I'll snip
> > the bits I can't or won't talk to.
> ...
> > Where we come from, how we find information to solve our problems, the
> > way we view the IETF and its RFCs are all different. If this argument
> > is just that this stuff doesn't belong in an RFC, that's a cultural
> > issue and not one I think we can resolve in this one technical working
> > group.
> 
> IMO, RFCs deal with *protocols* and protocol deployment issues.
> 
> Here that would mean for HTTP in specific
>     - turn Nagle off

All applications have been doing this for almost two decades.

>     - watch out for ACK compression effects (turn it off in favor of ABC
> if you can)

It does not happen that much with HTTP. Many connections on the server side
see only one, sometimes two requests, and most responses are small (about
20kB on average, with favicon fitting in a single segment). Note, I'm talking
about observations on average web sites.

>     - deal with slow-start restart (there are a variety of approaches here)

Due to the above this doesn't even happen most of the time, though with H/2
it will be more common.

> You also want to be generally efficient with TCP, which isn't related to
> HTTP but still good advice for all apps:
>     - be generally efficient and robust (use SACK, ECN, good window
> scaling, etc.)
>     - watch out for resource overloads (use SYN cookies/fast open,
> time-wait buildup, etc.)
> 
> That advice is useful for all TCP for transactions, too.

Absolutely, it's just that HTTP tends to magnify those effects because you
can get a lot of traffic concentrated on one or a few machines, and there's
a huge pressure on the admins as they know that there are tens of thousands
of visitors witnessing their performance issues. Performance is constantly
monitored by customers as well, trying to shrink the last millisecond to rank
better on search engines, which further adds to the pressure.

> Here's what it definitely does not mean:
>     - setting socket buffer sizes (that's an OS-ism, and may be useful
> for app designers to know, but isn''t directly part of TCP or HTTP)
>     - running an efficient server for small connections (that's a pure
> implementation performance issue, and depends on OS support for things
> like threads, process/thread pools, etc.)
>  
> I.e., OS-specific issues in running servers are not in scope IMO.

So what you have cited above are implementation details which are either
basic to any implementer or covered by the application itself. Admins
have already dealt with this in the default tuning and the application
has already done what is supposed to be done. Yet the admin is left with
a web site that is down under load, and they need to understand what
sysctls to change and what their effects can be on the short term (ie:
put the site back online) and the mid term (ie: limit the amount of
problems caused to random visitors). Lots of people for example enable
time-wait recycling because this is done by some people in benchmarks,
it's a single sysctl and they don't realize that they cause a lot of
issues even under moderate load.

> Keep in mind who reads RFCs - mostly protocol people. Managers take
> training courses to get certification, and that's where they learn their
> operational issues.

As Matthew said, admins tend to trust better the people who write the
protocols. Having an RFC authored by various protocol people will
definitely help make admins do the right thing.

Willy


From nobody Thu Aug 18 00:26:23 2016
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id BAB0A12D74F for ; Thu, 18 Aug 2016 00:26:21 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -3.149
X-Spam-Level: 
X-Spam-Status: No, score=-3.149 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RP_MATCHES_RCVD=-1.247, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XyfRBxhtiOUK for ; Thu, 18 Aug 2016 00:26:20 -0700 (PDT)
Received: from kuehlewind.net (kuehlewind.net [83.169.45.111]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 9C0E012D6AE for ; Thu, 18 Aug 2016 00:26:19 -0700 (PDT)
Received: (qmail 20280 invoked from network); 18 Aug 2016 09:26:17 +0200
Received: from 178-83-155-34.dynamic.hispeed.ch (HELO ?192.168.220.145?) (178.83.155.34) by kuehlewind.net with ESMTPSA (DHE-RSA-AES256-SHA encrypted, authenticated);  18 Aug 2016 09:26:17 +0200
Content-Type: text/plain; charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\))
From: "Mirja Kuehlewind (IETF)" 
In-Reply-To: <57B482B9.5090205@kuehlewind.net>
Date: Thu, 18 Aug 2016 09:26:18 +0200
Content-Transfer-Encoding: quoted-printable
Message-Id: 
References: <57B482B9.5090205@kuehlewind.net>
To: =?utf-8?Q?Mirja_K=C3=BChlewind?= 
X-Mailer: Apple Mail (2.3124)
Archived-At: 
Cc: "tcpm@ietf.org Extensions" , "tcpm-chairs@ietf.org" 
Subject: Re: [tcpm] Personal change in chairing
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Thu, 18 Aug 2016 07:26:22 -0000

s/personal change/personnel change/

> Am 17.08.2016 um 17:28 schrieb Mirja K=C3=BChlewind =
:
>=20
> Hi all,
>=20
> I have to announce a personal change in the tcpm chair position:
>=20
> Pasi Sarolahti is stepping down due to time restrictions. Pasi, thanks =
a lot for your work and engagement as chair! Your did a great job and we =
hope to still see you as often as possible as contributor in the working =
group!
>=20
> Michael T=C3=BCxen is starting as a new tcpm chair. Micheal, great to =
have you on broad! We look forward to work with you!
>=20
> Mirja, as responsible AD
>=20
> _______________________________________________
> tcpm mailing list
> tcpm@ietf.org
> https://www.ietf.org/mailman/listinfo/tcpm


From nobody Thu Aug 18 01:05:53 2016
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1A4E212D0BD for ; Thu, 18 Aug 2016 01:05:52 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.701
X-Spam-Level: 
X-Spam-Status: No, score=-2.701 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=tieto.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id E-tq-HS8wvzT for ; Thu, 18 Aug 2016 01:05:49 -0700 (PDT)
Received: from mail-it0-x234.google.com (mail-it0-x234.google.com [IPv6:2607:f8b0:4001:c0b::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 8C6DB12D0A1 for ; Thu, 18 Aug 2016 01:05:47 -0700 (PDT)
Received: by mail-it0-x234.google.com with SMTP id n128so11650607ith.1 for ; Thu, 18 Aug 2016 01:05:47 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tieto.com; s=google; h=from:references:in-reply-to:mime-version:thread-index:date :message-id:subject:to:cc; bh=y6LJiuW44bFkTnJCT9ljMTLoVk22Vqf22OsO5lA1H0s=; b=ufKhuypXfDIsU8z+U70prs+7GkdrnsOlxcZRL0wVYN86SHJ2qVZUKMvn5k2g5q6EWx KvO/+HKxMZz2L6zDrMONy11STbjS+9zD1rLW+LxhoUcCq/oTCj8KmXCNWp2+fG0y3SvK K2RVny1VHGMvFIc2ks8q0qpqPeUSUEL/mKEV4=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:references:in-reply-to:mime-version :thread-index:date:message-id:subject:to:cc; bh=y6LJiuW44bFkTnJCT9ljMTLoVk22Vqf22OsO5lA1H0s=; b=PiOX0fsMiInNaTt46pvfNnsGUnuKlWRMrz0hi4jaGHRmsO1HSBZlVd/V0YiU/F+kpE zFW0ahRo87pfY8s2D2b9KoULVicUsKn8eneCDpRD5Wdsxu3vCyyTL98nAHjLrRbCBpJB HIu+JmYV0LvCzaQKB9SF12tWPQyxcrmZ3MRzTJStikw1cjqd71rwROTAFVIJmotUfijQ VCntb69q2+DjUDDHFkqjVIiWuw63gJpsPaxDJvLWykom7mTEpOEWrpFcAmIdBhcZP0VQ JnFa0EiMn2FOLvuHXR7mFgP/slVIba7mCKT9KN5ISD32ovz2/iGKPCj3fiHyaDMhPRa3 GSXA==
X-Gm-Message-State: AEkoouslxtXN6o2kJftt1MSFKYQpQu0BxiBXWt4q0x5RJ29+5BDEMHHglO9XUQknZp8/jyWwcCJnWm1IDcCZvohU+efUU20VpGkt45uh0yk9DNnqeXawZrUQSGlMnG/KJ4okIOw=
X-Received: by 10.36.210.68 with SMTP id z65mr1760438itf.32.1471507547133; Thu, 18 Aug 2016 01:05:47 -0700 (PDT)
From: Karen Elisabeth Egede Nielsen 
References: <20160708211635.32095.77047.idtracker@ietfa.amsl.com> <2f56d1b8-7450-42de-4a24-53df9cf4c045@it.uc3m.es>   <623f46dedcc661ae74563f6bf08f46a5@mail.gmail.com> 
In-Reply-To: 
MIME-Version: 1.0
X-Mailer: Microsoft Outlook 15.0
Thread-Index: AQGd4OoGFyImcsqDlVk0oY8T3VfV3QH845XxAem+5TgCNV3HJwFbXVb9ANTgh9KgdCWb0A==
Date: Thu, 18 Aug 2016 10:05:45 +0200
Message-ID: <00594c32f56877bfc2fde20132250174@mail.gmail.com>
To: Yuchung Cheng 
Content-Type: text/plain; charset=UTF-8
X-DomainID: tieto.com
Archived-At: 
Cc: tcpm IETF list , TCP Prague List 
Subject: Re: [tcpm] Fwd: I-D Action: draft-bagnulo-tcpm-tcp-low-rtt-00.txt
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Thu, 18 Aug 2016 08:05:52 -0000

HI Yuchung, Marcello

> -----Original Message-----
> From: Yuchung Cheng [mailto:ycheng@google.com]
> Sent: 15. august 2016 18:49
> To: Karen Elisabeth Egede Nielsen 
> Cc: marcelo bagnulo braun ; TCP Prague List
> ; tcpm IETF list 
> Subject: Re: [tcpm] Fwd: I-D Action: draft-bagnulo-tcpm-tcp-low-rtt-00.txt
>
> On Mon, Aug 15, 2016 at 2:00 AM, Karen Elisabeth Egede Nielsen
>  wrote:
> > Hi Yuchung,
> >
> >> >
> >> > An additional comment is  that new approaches to retransmissions -
> >> > like TCP TLP and TCP RACK (also SCTP TLR which however is not
> >> > progressing at the moment) might fundamentally alter the picture.
> >> > I.e., if retransmissions are sent pro-actively in tail loss
> >> > situations then more conservative RTOs may be kept for situations
> >> > where it is prudent to wait longer. Don't know if TCP TLP is so
> >> > widely deployed that it is something that you should relate to even
> >> > if it may be superseded by RACK. Just a thought.
> >> TLP and RACK help reduce timeout cases in DC environment for sure.
> >> But still it can not completely avoid timeouts.
> >
> > [Karen Elisabeth Egede Nielsen] Agreed.
> >
> > For SCTP TLR we still see RTO-timeouts when also
> > probes/retransmissions are lost.
> > I assume that it would be something like this also for Rack/TLP -
> > though potentially depending on how many consecutive probes that are
> allowed.
> > The role of RTO-timeouts are different when RACK/TLP is enabled and it
> > might be counterproductive to have RTO-timeouts be too aggressive as
> > the function indeed with TLP/RACK becomes something much closer to the
> > original intend
> > (read: how I understand the original intend) - namely to introduce a
> > pause for the network to recover when things are really (consistently)
> > bad.
> >
> > If the RTO is lowered down to be comparable in order of magnitude with
> > the RTT (with some delay_ack considerations) we are narrowing the
> > situation where TLP/RACK probing is able to kick in before
> > RTO-retransmissions starts anyway.
> >
> > I understand that RTOs should be adjusted to the network dynamics, but
> > I do think that it is important to understand - for the DC environment
> > - if the need for the low RTOs in DC's is motivated mainly by having
> > RTO-retransmissions fix the tail loss recovery deficiencies of TCP,
> > which RACK/TLP addresses, or more generally for a need for more timely
> > probe and reactivation for/after network recovery.
> >
> > Or perhaps you see the TLP probe timeout and the RTO timeout
> > eventually being driven by the same timer, the reactions (e.g. CC
> > reactions)  then possibly depending on state (probe already sent or
> > not).
> I agree with your thought here.
>
> More generally, I'd like to evolve the state of art recovery with these
> principles:
> 1. React with ack events as much as possible
>
> 2. Send tiny probes to trigger (1) after 2-3 RTTs
>
> 3. Have conservative  RTOs that (exp-backoff) expire at an order of RTT to
> repair the head for reliability.
>
>
> On thing about RTO: RTO current reset cwnd to 1 upon firing no matter
> what.
> We should ONLY reset cwnd to 1 if no news (acks/data) for a long duration.
> This is quite important for performance: today cwnd is reset to 1 even if
> every packet but the first one has been recently delivered. This
> unnecessarily conservative because the ack clock isn't lost yet. But the
> collateral damage to performance is huge on large BDP networks. This
> results
> work-arounds to shorten and avoid RTOs.
>
>
> I'd like to address these points in the upcoming RACK/TLP draft.
>
>
[Karen Elisabeth Egede Nielsen] I agree completely with your view and
approach.

In respect to the draft draft-bagnulo-tcpm-tcp-low-rtt-00.txt and the given
recommendations for low RTO
is that they do not (likely) take 2. Into account - meaning that they are
looking for the low RTO to solve 2 for tail recovery.
The bad thing is that they then also get the CWND reductions (as you also
say).


I would like for that this aspect is taken into consideration/given thoughts
in the draft draft-bagnulo-tcpm-tcp-low-rtt-00.txt and the general 4LS
effort.
If we end with very low recommendations for RTO in order to handle 2. we
risk undermining the RTO-retransmission function as well as the TLP function
for now and forever.

It might be that one for TCPs, that do not support TLP(/RACK), need to
operate with very low RTOs to handle tail loss's "fast enough", but the
recommendation for
"future standard ?" TCP with TLP always enabled might be different. In this
context lowering the RTO to handle 2. should be considered as a "temporary
hack" ?

BR, Karen


From nobody Thu Aug 18 08:08:21 2016
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C6E3C12DD93 for ; Thu, 18 Aug 2016 08:08:19 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -8.166
X-Spam-Level: 
X-Spam-Status: No, score=-8.166 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-1.247] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id f9ntpoOHHNPc for ; Thu, 18 Aug 2016 08:08:18 -0700 (PDT)
Received: from boreas.isi.edu (boreas.isi.edu [128.9.160.161]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id A361212D995 for ; Thu, 18 Aug 2016 08:08:18 -0700 (PDT)
Received: from [192.168.1.189] (cpe-172-250-251-17.socal.res.rr.com [172.250.251.17]) (authenticated bits=0) by boreas.isi.edu (8.13.8/8.13.8) with ESMTP id u7IF7Xia007157 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Thu, 18 Aug 2016 08:07:34 -0700 (PDT)
To: Willy Tarreau 
References: <5CD67877-19E3-4E79-BBF2-3E270343A378@mnot.net> <2197232f-10d7-28cb-fcc9-05bd495e3c22@isi.edu> <20160817064545.GD16017@1wt.eu> <7f7b129c-f156-d067-bef8-4a2213f461ac@isi.edu> <20160817180802.GA16773@1wt.eu> <4ab7c5b0-3722-1346-f481-a8d76de70034@isi.edu> <20160817211317.GA16929@1wt.eu>   <27b58b64-48cd-39af-78b3-ef583c585fa6@isi.edu> <20160818053837.GC16773@1wt.eu>
From: Joe Touch 
Message-ID: <7a36e025-4882-4f8b-7a83-9fdcd990a971@isi.edu>
Date: Thu, 18 Aug 2016 08:07:31 -0700
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0
MIME-Version: 1.0
In-Reply-To: <20160818053837.GC16773@1wt.eu>
Content-Type: multipart/alternative; boundary="------------680C336DB0D7739977BCD7F8"
X-ISI-4-43-8-MailScanner: Found to be clean
X-MailScanner-From: touch@isi.edu
Archived-At: 
Cc: tcpm@ietf.org, Matthew Kerwin , Daniel Stenberg , Mark Nottingham , HTTP Working Group , Patrick McManus 
Subject: Re: [tcpm] TCP Tuning for HTTP - update
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Thu, 18 Aug 2016 15:08:20 -0000

This is a multi-part message in MIME format.
--------------680C336DB0D7739977BCD7F8
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: 7bit



On 8/17/2016 10:38 PM, Willy Tarreau wrote:
>> Keep in mind who reads RFCs - mostly protocol people. Managers take
>> > training courses to get certification, and that's where they learn their
>> > operational issues.
> As Matthew said, admins tend to trust better the people who write the
> protocols. Having an RFC authored by various protocol people will
> definitely help make admins do the right thing.
Admins don't know who protocol people are.

I do think that this doc needs to figure out whom it is speaking to,
what advice they actually need, etc.

If the result is a set of recommendations that involve the word
"sysctl", I remain skeptical it is appropriate as an RFC.

Joe

--------------680C336DB0D7739977BCD7F8
Content-Type: text/html; charset=windows-1252
Content-Transfer-Encoding: 7bit


  
    
  
  
    


    

    

    
On 8/17/2016 10:38 PM, Willy Tarreau
      wrote:

    

    

      

        
Keep in mind who reads RFCs - mostly protocol people. Managers take
> training courses to get certification, and that's where they learn their
> operational issues.


      

      
As Matthew said, admins tend to trust better the people who write the
protocols. Having an RFC authored by various protocol people will
definitely help make admins do the right thing.

    

    Admins don't know who protocol people are.

    

    I do think that this doc needs to figure out whom it is speaking to,
    what advice they actually need, etc.

    

    If the result is a set of recommendations that involve the word
    "sysctl", I remain skeptical it is appropriate as an RFC.

    

    Joe

  


--------------680C336DB0D7739977BCD7F8--


From nobody Thu Aug 18 09:35:55 2016
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 137F212D898 for ; Thu, 18 Aug 2016 09:35:53 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.89
X-Spam-Level: 
X-Spam-Status: No, score=-1.89 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, T_KAM_HTML_FONT_INVALID=0.01] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xTiVdmtXKNGq for ; Thu, 18 Aug 2016 09:35:47 -0700 (PDT)
Received: from server.dnsblock1.com (server.dnsblock1.com [85.13.236.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 7639612D09C for ; Thu, 18 Aug 2016 09:35:45 -0700 (PDT)
Received: from host81-153-214-243.range81-153.btcentralplus.com ([81.153.214.243]:53567 helo=[192.168.1.82]) by server.dnsblock1.com with esmtpsa (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim 4.87) (envelope-from ) id 1baQIB-0002xf-9x; Thu, 18 Aug 2016 17:35:43 +0100
From: Bob Briscoe 
To: Praveen Balasubramanian 
References: <563CF0F7.6070302@bobbriscoe.net> 
Message-ID: <71a48e5c-c992-de4d-12c3-7779ee702bd6@bobbriscoe.net>
Date: Thu, 18 Aug 2016 17:35:42 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0
MIME-Version: 1.0
In-Reply-To: 
Content-Type: multipart/alternative; boundary="------------24759AAECE54BB7370BFF4EE"
X-AntiAbuse: This header was added to track abuse, please include it with any abuse report
X-AntiAbuse: Primary Hostname - server.dnsblock1.com
X-AntiAbuse: Original Domain - ietf.org
X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12]
X-AntiAbuse: Sender Address Domain - bobbriscoe.net
X-Get-Message-Sender-Via: server.dnsblock1.com: authenticated_id: in@bobbriscoe.net
X-Authenticated-Sender: server.dnsblock1.com: in@bobbriscoe.net
Archived-At: 
Cc: tcpm IETF list 
Subject: Re: [tcpm] Review of draft-ietf-tcpm-dctcp-01
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Thu, 18 Aug 2016 16:35:53 -0000

This is a multi-part message in MIME format.
--------------24759AAECE54BB7370BFF4EE
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit

Praveen,


On 17/07/16 21:25, Praveen Balasubramanian wrote:
>
> Thanks Bob. Sorry for the  delay.
>
[BB] And now sorry for my delay. I tried to finish this email a number 
of times, but kept getting interrupted.
>
> Comments inline prefixed by . Most of these edits will be in 
> the next revision.
>
[BB] I've tried very hard to only comment where I really do think there 
is still a problem (even if minor), and suggested specific text to help 
you...

> Thanks
>
> *From:*Bob Briscoe [mailto:ietf@bobbriscoe.net]
> *Sent:* Friday, November 6, 2015 10:27 AM
> *To:* Praveen Balasubramanian ; EGGERT, Lars 
> ; Stephen Bensley ; Dave Thaler 
> ; glenn.judd@morganstanley.com
> *Cc:* tcpm IETF list 
> *Subject:* Review of draft-ietf-tcpm-dctcp-01
>
> Praveen & co-authors,
>
> As promised, here's my review of draft-ietf-tcpm-dctcp-01
>
> *_General Comments
> _*
> In general, in the comments below I have tried to reduce the multiple 
> assumptions about controlled environments down to the one assumption 
> that they are controlled by a single authority. I believe DCTCP can 
> work without the other assumptions, so the following do not apply in 
> all controlled environments:
>   * not always low RTT (e.g. inter-DC scenarios, or global private 
> networks)
>   * not always no risk of traffic attacks (e.g. internally arranged 
> attacks)
> I may have missed some other occurrences of these assumptions, so pls 
> feel free to seek out them all.
>
>  DCTCP can be extended to other environments but this draft 
> does not cover any of that. It is explicitly for deployment in a 
> datacenter where the RTT is low and there is no risk of internal 
> attack. We cannot cover other scenarios that have not been tested or 
> deployed. All the known deployments are in controlled datacenters. The 
> TCP Prague effort should result in a draft that covers additional 
> requirements that will make DCTCP work well on high latency links.
>
>
> Many of the comments argue with your choices of MUST, SHOULD, MAY etc. 
> This might seem nit-picking, given the primary purpose is to document 
> the algo. However, it would be wrong to require things that aren't 
> required or to allow exceptions when exceptions would not be 
> interoperable. An alternative approach would be to just remove all 
> capitalised language.
>
>  I am fine with removing such capitalization except for when 
> removing it will lead to interop problems between different 
> implementations of DCTCP. I am responding to each comment below and 
> adding whether the new revision will fix it not.
>
>
> *_Section-by-section review.
> _*
> *Abstract
> *
> I think it needs to have an applicability statement in the abstract 
> (and introduction) that refers to the deployment and implementation 
> status sections at the end. Something like:
>
> "This is an informational specification of the implementation of DCTCP 
> in Microsoft Windows Server 2012. It is applicable to deployments in 
> controlled environments like data centres but it MUST NOT be deployed 
> over the public Internet without additional measures, as detailed in 
> sections 4-6. "
>
>  This is already addressed in the introduction. I don’t see 
> the value of repeating it multiple times. An implementer will read the 
> introduction. If you still feel strongly, we can add it to the 
> abstract and remove it from the introduction section. This draft will 
> not cover any additional measures that are intended for deployments 
> outside the datacenter – that’s not the intended purpose.
>
[BB] It is very common for the IETF to require even much less minor 
applicability statements to be highlighted in the abstract (e.g. see TFO 
RFC). This is because nowadays implementers / readers often skip 
introductory text assuming they understand the problem, etc. and they 
just scan for the normative parts. Anyone reading an IETF RFCs assumes 
the context is the public Internet, so any RFC that doesn't have that 
context needs to flag that. I know the name says Data Centre, but the 
abstract needs to explain that this is not an IETF RFC that is 
standardizing DCTCP for the public Internet.

If you still don't think so, we should put this to the WG (which will 
slow it down).
>
>
> *1. Intro
> *s/its many servers/
>  /their many servers/
>
>  Fixed in next revision.
>
>
> "...limited queue capacities..."
> I understand that the motivation has been abbreviated, but I think 
> this has lost too much. Perhaps mention memory shared between 
> interfaces, so either bloated buffers or too little, making traffic 
> susceptible to packet losses?
>
>    [RFC3168] describes a mechanism for using Explicit Congestion
>    Notification (ECN) from the switches for early detection of
>    congestion, rather than waiting for packet loss to occur.
>
> This isn't correct. 3168 requires ECN marking to occur no earlier than 
> drop. It is precisely that 3168 does not allow early marking unless 
> there is also early drop that is the problem.
>
>  Fixed in next revision.
>
>
>    It is recommended that DCTCP be deployed...
>
> "recommended" is too weak. I suggest you use something like the 
> applicability text given above, both here and in the abstract.
>
>  Added the word “only” to make this stronger per Michael’s 
> review. Fixed in next revision.
>
[BB] I suggest the applicability para (here in the intro) refers forward 
to section 5 for details (e.g. so you don't have to explain why it 
shouldn't be deployed in the intro).
>
>
>
> *2. Terminology
> *
> Suggest you explain that capitalised words are not quite the same as 
> in most RFCs:
> "Normative language is used to describe how necessary the various 
> aspects of the Microsoft implementation are for interoperability, but 
> even compliant implementations without the measures in sections 4-6 
> would still only be safe to deploy in controlled environments."
>
>  Fair enough. Added text in next revision.
>
>
> *3. DCTCP Algo
> *
> The three bullets say no more than "this is normal stuff". Would it be 
> better to supplement each sentence with a brief phrase saying what is 
> different about each aspect compared to RFC3168?
>
>  The goal here it to describe the changes needed on top of 
> 3168 so I don’t think further explanation is necessary.
>
>
> *3.1 Marking
> *
> This needs to say something about how you mark the IP header from L2. 
> I suggest you refer to the up-and-forward mode of 
> draft-ietf-tsvwg-ecn-encap.
>
>  Why is that relevant to this document? It is not intended as 
> a specification for operation of L2 switches. Won’t fix.
>
[BB] Well, you specify a lot about the marking algo, e.g.
K > (RTT *  C)/7

A simple solution would be: s/switch/L3 switch/

I just thought that a reader would think, "You say switches (i.e. L2 
devices) mark the IP header - how can that work?" unless you explain.

>
> s/sending rate/
>  /link rate/
>
>  Fixed in next revision
>
>
> *3.2
> *
> CURRENT:
>
>     1.  If the CE codepoint is set and DCTCP.CE is false, send an ACK for
>         any previously unacknowledged packets and set DCTCP.CE to true.
>     2.  If the CE codepoint is not set and DCTCP.CE is true, send an ACK
>         for any previously unacknowledged packets and set DCTCP.CE to
>         false.
>
> SUGGESTED:
>
>     1.  If the CE codepoint is set and DCTCP.CE is false, set DCTCP.CE to
>         true and send an ACK for any previously unacknowledged packets.
>     2.  If the CE codepoint is not set and DCTCP.CE is true, set DCTCP.CE
>         to false and send an ACK for any previously unacknowledged packets.
>
>     The figure would have to be changed to be consistent too.
> RATIONALE:
> If the receiver changes DCTCP.CE /after/ sending the ACK like this, it 
> delays each signal until the following ACK is sent. That could add a 
> delay of anything from 1 to m packets. I've never understood why the 
> order of these operations was specified in this way. If there is no 
> reason, then I suggest that it is specified in the opposite order, as 
> I describe above.
>
>
> A note could be added to say the Windows Server 12 implementation does 
> these steps in reverse order, but the order specified is preferred 
> because it reduces signalling delay and has no interoperability issues 
> with the original Windows implementation.
>
>  Won’t fix. This is how the Windows implementation behaves so 
> the ordering is required. Both the paper and the draft are consistent 
> in this.
>
[BB] The paper and the draft are not consistent. In the draft, the 
labels on the top and bottom arrows in the diagram have been switched 
round relative to those in the paper.

However, the text in the draft has not (yet?) been changed to reflect this.
>
> There is no delay being introduced here. This ACK is sent immediately 
> so the order does not matter.
>

[BB] I agree that the ACK itself is not delayed. However, even tho a CE 
has been received, feedback of this event is held back until the next ACK.

This is a minor 'unnecessary delay' bug.
* I believe it was fixed in the FreeBSD implementation after I pointed 
it out.
* I've just checked the Linux code, and it's not been fixed there.
* Obviously I cannot see the Windows source code.

If this bug is still in the Windows implementation, it's up to you if 
you also want the spec to require that the bugs are implemented exactly 
as they are in Windows ;)

>        
>     The handling of the "Congestion Window Reduced" (CWR) bit is also
>     exactly as per [RFC3168 
> ] including [RFC3168-ERRATA3639 
> ].  That is, on
>     receipt of a segment with both the CE and CWR bits set, CWR is
>     processed first and then ECE is processed.
>
> Even tho this is the receiver section, I suggest:
> s/The handling/Receiver handling/
>
>  Isn’t this assumed by a reader familiar with 3168? But this 
> is a minor editorial fix so fixed in the next revision.
>
>
> Whatever, this cannot be correct. A DCTCP receiver surely does nothing 
> on receipt of CWR, whereas an RFC3168 receiver does something. A DCTCP 
> receiver certainly cannot do exactly what RFC3168 says, because that 
> says stop setting ECE on receipt of CWR until the next CE arrives, 
> which would surely break this feedback protocol.
>
>  I don’t see why this would break the feedback protocol. The 
> receiver will start echoing again as soon as it sees the first CE 
> arrives again. If packet has both CWR and CE set then CE takes 
> precedence and the echoing is done.
>
[BB] I'm not saying it breaks the feedback protocol. I'm just saying 
it's confusing for the reader.

Of course, when the receiver is already not doing something, ignoring a 
signal to stop has the same outcome as stopping. However, describing 
this as "handling CWR ... is also exactly as per RFC3168" is misleading 
for the reader/implementer.

I guess changing to "handling CWR ... is as per RFC3168" would be a 
compromise.
>
>
> *3.3 *
>
>     DCTCP.Alpha, which is initialized to 1 and MUST be updated
>     as follows:
>
> s/MUST/SHOULD/
> And add "An alternative congestion avoidance algorithm MAY be used, 
> but it MUST lead to flow rate inversely proportional to 1/M".
>
> Rationale: There are many ways to implement interoperable 1/p cc. This 
> is just one.
>
>  Alternate algorithms are out of scope of this draft. We 
> cannot say without interop testing if a different algorithm will 
> perform well in datacenters or interop well with existing 
> implementations. If there are other ways to implement 1/p they should 
> be covered by other drafts. This draft will only describe the 
> algorithm that has been tested and deployed. That said I am willing to 
> make it a SHOULD so fixed in the next revision.
>
>     DCTCP.BytesSent
>
> This variable actually counts bytes received. it's pretty confusing to 
> call it BytesSent.
>
>  No, this is tracking bytes that have been sent.
>
[BB] I meant it counts bytes now acknowledged as received by the other 
end (which is not the same as bytes sent by this end).
DCTCP.BytesAcked would have been more meaningful. I'm not insisting on 
this - the original comment was just to improve comprehensibility.
>
>
> s/TCP SACK options [RFC2018] are ignored./
>  /The sender MUST ignore TCP SACK options [RFC2018] for this computation./
> RATIONALE:
>   a) I think the idea is to ignore CE marks after a gap because, if 
> the gap becomes considered a loss, the response to loss will override 
> any need to count incoming CE marks for the following round trip. 
> However, if the gap is filled before it is considered a loss {Note 1}, 
> then CE marks that were ignored because they arrived after a gap will 
> need to be "unignored".
>   b) We need to make clear that SACK is not ignored altogether, only 
> for this computation.
>
> {Note 1}: I can think of two cases:
> * Less than three dupacks then the gap is filled
> * The gap is filled by a delayed segment so that an earlier 
> retransmission was spurious
>
>  Ok I will update the text.
>
> the sender MUST update cwnd as follows:
>        cwnd = cwnd * (1 - DCTCP.Alpha / 2)
>
> s/MUST/SHOULD/
> Rationale: Alternative interoperable response functions are possible. 
> For instance an algorithm like Relentless TCP that simply reduces cwnd 
> by half the size of any CE-marked segment. Then there is no Alpha and 
> no g variable.
>
>  Fixed
>
>
> CURRENT:
>
>     Just as specified in [RFC3168 
> ], TCP should not react to congestion
>     indications more than once for every window of data.
>
> SUGGESTED:
>
>     Just as specified in [RFC3168 
> ], DCTCP does not react to congestion
>     indications more than once for every window of data.
>
> I don't know whether you intended this to be an upper-case SHOULD NOT. 
> If so, I would argue against this being normative, because it isn't 
> necessary for interop; therefore I've suggested avoiding he 'should' word.
>
>  Fixed
>
>     The setting of
>     the "Congestion Window Reduced" (CWR) bit is also exactly as per
>     [RFC3168 
> ].
>   
>
> It is wrong to say "exactly". The sender certainly sets CWR when it 
> does a congestion response. But in DCTCP the congestion response 
> occurs every RTT, triggered by its own measurement-period logic, 
> whereas in RFC3168 it is triggered by the arrival of an ECE after the 
> ACK of any previous segment it sent with CWR set.
>
> I think it is worth saying that setting CWR is necessary for safety, 
> otherwise in the case where there has been a configuration management 
> error and the receiver complies with RFC 3168 not DCTCP, it will 
> continue sending ECE for ever and starve the session.
>
> It might help to explicitly confirm that:
> "Other aspects of TCP congestion avoidance and control such as the 
> slow start phase are no different to those in RFC5681."
>
>  Fixed that CWR is needed for safety. Not adding the text 
> around 5681, it is presumed.
>
>
> *3.4 SYN, SYN-ACK, RST
> *
>    [RFC3168] requires that a compliant TCP MUST NOT set ECT on SYN or
>    SYN-ACK packets.
>
> s/MUST NOT/'MUST NOT'/
> Rationale: Best to make it clear this is quoting a normative statement 
> in another RFC, because it is going to be contradicted.
>
[BB] I don't know whether you saw this one - it's not been changed in 
draft-02.

I suggested adding quotes around 'MUST NOT'. This is good practice when 
quoting another RFC. Otherwise, to a careless reader (or non-native 
English speaker), it can look like it is also a requirement of this RFC.
>
>
> CURRENT
>
>     These RFCs, however, are
>     intended for general Internet use, and do not directly apply to a
>     controlled datacenter environment.
> ...
>     For DCTCP connections, the sender
>     SHOULD set ECT for SYN, SYN-ACK and RST packets.
>
> SUGGESTED:
>
>    Also, a SYN is sent before the ECN-capability has been
>    negotiated, so if it is CE-marked, a non-ECN TCP server would not
>    recognise the marking. RFC 3168 does not specify the ECN field on
>    a RST. The security concerns addressed by these RFCs
>    might not apply in controlled environments like data centres, and
>    it might not be necessary to cater for the presence of non-ECN
>    servers.
> ...
>
>     Therefore, the sender MUST set ECT for SYN/ACK and RST packets and a
>     configuration option SHOULD be provided to set ECT for SYNs.
>
> RATIONALE:
>    The draft should avoid assuming that security concerns do not apply 
> in all controlled environments.
> NOTE:
>    This might not reflect the way your implementation is coded, so you 
> might want to make that clear.
>
>  Adding some text around security concerns.
>
[BB] The security sentence you have added in draft-02 doesn't make sense 
where it is. It ought to have been added once sentence earlier (and it 
would be better to start a new para). Specifically:
CURRENT:
                 be dropped with high probability.  For DCTCP 
connections, the sender
                 SHOULD set ECT for SYN, SYN-ACK and RST packets.  The 
security
                 concerns addressed by both these RFCs might not apply 
in controlled
                 environments like datacenters, and it might not be 
necessary to cater
                 to both the presence of non-ECN servers.
SUGGESTED:
                 be dropped with high probability.

The security concerns addressed by both these RFCs might not apply in 
controlled
                 environments like datacenters, and it might not be 
necessary to cater
for the presence of non-ECN servers. For DCTCP connections, the sender
                 SHOULD set ECT for SYN, SYN-ACK and RST packets.

(Note I also fixed a glitch after "cater".)


>
> *4. Implementation Issues
> *
> CURRENT:
>
>     the implementation MUST choose a suitable
>     estimation gain.  [DCTCP10 
> ] provides a theoretical basis
>
> SUGGEST:
>
>     the implementation will need to choose a suitable
>     estimation gain.  [ADCTCP10 
> ] provides a theoretical basis ...
>
> RATIONALE:
>    a) As above, this is only one way to implement the cc, and others 
> could well prove to be interoperable when tested.
>    b) More particularly, it would be better for an implementation to 
> adapt the estimation gain to the RTT, so the spec ought not to imply 
> that one value has to be chosen.
>    c) The later paper [ADCTCP10] provides a corrected way to determine 
> the gain. From memory (I'm offline at the mo) it at least makes 
> throughput inversely proportional to 1/RTT, whereas the approach in 
> [DCTCP10] made it inversely proportional to 1/(RTT^2). I believe the 
> Linux implementation uses the [ADCTCP10] approach.
>
>  Fair enough. Updated.
>
>     The implementation must also decide when to use DCTCP.
> ...
>     likely in the same datacenter network.
>
> The 2nd & 3rd paras are a mix of implementation and deployment issues, 
> but mostly deployment, so they might be better moved to the deployment 
> issues section.
>
>  I think they are mainly implementation issues – the fact 
> that a lack of negotiation mechanism places a burden on the 
> implementation to provide knobs or heuristics to use DCTCP.
>
>
> CURRENT:
>
>     It is RECOMMENDED that an implementation deal with loss episodes in
>     the same way as conventional TCP.
> ...
>     DCTCP implementation MAY also allow configuration of resetting the
>     value of DCTCP.Alpha as part of processing any loss episodes.
>
> SUGGESTED: Move this whole para to 3.3 Sender Behaviour section, and:
>
>     An implementation MUST deal with loss episodes in
>     the same way as conventional TCP.
> ...
>     DCTCP implementation SHOULD also reset the
>     value of DCTCP.Alpha as part of processing any loss episodes.
>     This aspect of the behaviour MAY be configurable.
>
> RATIONALE:
>    a) There would never be a case for not falling back to conventional 
> TCP  (or do you have one?), so "RECOMMENDED" is too weak.
>    b) I have tried to interpret what you might have meant about 
> resetting Alpha. I'm not sure why you think it might need to be 
> configurable though?
>
>  Yes this is a useful suggestion. Making loss handling a must 
> in sender section. Reset of alpha was discussed in tcpm list, and 
> Linux implementation supports it.
>
>     ...it is also RECOMMENDED that an implementation allow
>     configuration of restarting the congestion window (cwnd) of idle
>     DCTCP connections
>
> Where special implementation measures are necessary for low RTT, these 
> should be related to the measured RTT, not hard-coded. There is an 
> assumption in a number of places  that "data centre" implies low RTT. 
> Of course this is true in a single DC, but I believe the the single 
> admin aspect of a DC is what characterises DCTCP, not the low RTT 
> aspect. With the modifications in [ADCTCP], DCTCP generalises to a 
> network with larger RTTs as long as it is still  operated by a single 
> admin.
>
>  this paragraph has now been removed per Michaels’ feedback.
>
>
> CURRENT:
>    [RFC3168] forbids the ECN-marking of pure ACK packets, because of the
>    inability of TCP to mitigate ACK-path congestion and protocol-wise
>    preferential treatment by routers.  However, dropping pure ACKs -
>    rather than ECN marking them - has disadvantages for typical
>    datacenter traffic patterns.  Because of the prevalence of bursty
>    traffic patterns that feature transient congestion, dropping of ACKs
>    causes subsequent retransmissions. It is RECOMMENDED that an
>    implementation provide a configuration knob that will cause ECT to 
> be set
>    on such control packes, which can be used in environments where such
>    concerns do not apply.
> SUGGESTED:
>    [RFC3168] forbids the ECN-marking of pure ACK packets, because of the
>    inability of TCP to mitigate ACK-path congestion and the extra
>    advantage to injection attackers that ECN is perceived to offer.
>    For the latter reason RFC 3168 also forbids ECN-marking of
>    retransmissions, window probes and RSTs.
>
>    However, dropping all these control packets -
>    rather than ECN marking them - has considerable performance
>    disadvantages.  It is RECOMMENDED that an
>    implementation provide a configuration knob that will cause ECT to 
> be set
>    on such control packes, which can be used in environments where such
>    concerns do not apply.
> NOTE:
>    This might not reflect the way your implementation is coded, so you 
> might want to make that clear.
>
>  Fixed
>
[BB] A new comment...

As pointed out in draft-bagulo-tsvwg-generalized-ecn-00, RFC3168 only 
talks about injection attacks with retransmissions (not ACKs etc). But 
you imply injection attacks were the concern for other control packets 
as well.

I'm not insisting you change this. However, unwarranted accusations of 
security vulnerabilities tend to stick, whether or not they are correct. 
So it would be better not to throw mud around onto even more control 
packets than RFC3168 did.

Cheers


Bob

>
>
> *5. Deployment Issues*
>
>     If
>     the traffic in the datacenter is a mix of conventional TCP and DCTCP,
>     it is RECOMMENDED that DCTCP traffic be segregated from conventional
>     TCP traffic.
>
> In a data centre (or anywhere), is there any case where not 
> segregating them would make sense? If not, this needs to be a MUST, 
> not just a RECOMMENDED. I can imagine cases where there is no 
> long-running conventional TCP or no long-running DCTCP, but that's 
> covered by the "if" at the start of the sentence.
>
> You might want to refer to draft-briscoe-aqm-dualq-coupled as another 
> segregation approach here too.
>
>     Since DCTCP relies on congestion marking by the switches, DCTCP can
>     only be deployed in datacenters where the entire network
>     infrastructure supports ECN.
>
> Surely, the "fall-back to conventional TCP on loss" rule means you can 
> delete this constraint.
>
>  Reworded to imply that DCTCP will only work as expected in 
> such cases.
>
>     A
>     variant of DCTCP that can be deployed unilaterally and only requires
>     standard ECN behavior has been described in [ODCTCP 
> ][BSDCAN],
>
>
> Which part of "standard ECN behaviour" do you mean? The host part? The 
> network part? The receiver part? And "can be deployed unilaterally" 
> ought to be in the active voice not the passive, which would help 
> resolve the ambiguity too.
>
> I haven't read [ODCTCP] and [BSDCAN] (I'm offline on a plane at the 
> mo). Are they similar to "Instant ECN" in [Wu12] listed below? If not, 
> that could be referred to as well for a technique that uses the 
> regular ECN host behaviour.
>
>  This edit predates me and  I don’t have the full context, I 
> am leaving this as is.
>
>
> *6. Known Issues
> *
> First para could refer to appendix A of RFC7560, which describes the 
> problem precisely.
>
> A method for improving the fairness of DCTCP has been proposed
>     in [ADCTCP 
> ], but requires additional experimental evaluation.
>
> s/fairness/RTT fairness/
>
> (In our experiments, it's much better than without.)
>
>  Goal is to cite other work that can address potential 
> issues. It is not a recommendation. s/fairness/RTT fairness/ Fixed in 
> next revision.
>
>
> *11. References
> *
> I think the following are cited normatively, not just informatively:
> RFC5681, RFC5562.
>
> Additional informational Reference:
>
> [Wu12] Wu, H., Ju, J., Lu, G., Guo, C., Xiong, Y. & Zhang, Y., "Tuning 
> ECN for Data Center Networks," In: Proceedings of the 8th 
> International Conference on Emerging Networking Experiments and 
> Technologies CoNEXT '12 pp.25-36 ACM (2012)
>
>  Citation to RFCs updated as normative. Not including the new 
> reference since even though it may be relevant – it was not used as a 
> reference in writing the draft.
>
>
> That's it. HTH.
>
>
> Bob
>
> -- 
> ________________________________________________________________
> Bob Briscoehttp://bobbriscoe.net/ 
> 

-- 
________________________________________________________________
Bob Briscoehttp://bobbriscoe.net/

-- 
________________________________________________________________
Bob Briscoehttp://bobbriscoe.net/


--------------24759AAECE54BB7370BFF4EE
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: 8bit


  
    
  
  
    Praveen,

    

    

    
On 17/07/16 21:25, Praveen
      Balasubramanian wrote:

    

    

      

        
Thanks
            Bob. Sorry for the <ridiculous> delay. 

      

    

    [BB] And now sorry for my delay. I tried to finish this email a
    number of times, but kept getting interrupted.

    

      

        
Comments
            inline prefixed by <Praveen>. Most of these edits will
            be in the next revision.

      

    

    [BB] I've tried very hard to only comment where I really do think
    there is still a problem (even if minor), and suggested specific
    text to help you...

    

    

      

        
 

        
Thanks

        
 

        

          

            
From:
                Bob Briscoe [mailto:ietf@bobbriscoe.net]
                

                Sent: Friday, November 6, 2015 10:27 AM

                To: Praveen Balasubramanian <pravb@microsoft.com>;
                EGGERT, Lars <lars@netapp.com>;
                Stephen Bensley <sbens@microsoft.com>;
                Dave Thaler <dthaler@microsoft.com>;
                glenn.judd@morganstanley.com

                Cc: tcpm IETF list <tcpm@ietf.org>

                Subject: Review of draft-ietf-tcpm-dctcp-01

          

        

        
 

        
Praveen & co-authors,

          

          As promised, here's my review of draft-ietf-tcpm-dctcp-01

          

          General Comments

            

          In general, in the comments below I have tried to reduce the
          multiple assumptions about controlled environments down to the
          one assumption that they are controlled by a single authority.
          I believe DCTCP can work without the other assumptions, so the
          following do not apply in all controlled environments:

            * not always low RTT (e.g. inter-DC scenarios, or global
          private networks)

            * not always no risk of traffic attacks (e.g. internally
          arranged attacks)

          I may have missed some other occurrences of these assumptions,
          so pls feel free to seek out them all.

          

          

        
<Praveen>
            DCTCP can be extended to other environments but this draft
            does not cover any of that. It is explicitly for deployment
            in a datacenter where the RTT is low and there is no risk of
            internal attack. We cannot cover other scenarios that have
            not been tested or deployed. All the known deployments are
            in controlled datacenters. The TCP Prague effort should
            result in a draft that covers additional requirements that
            will make DCTCP work well on high latency links. 

        
 

        


          Many of the comments argue with your choices of MUST, SHOULD,
          MAY etc. This might seem nit-picking, given the primary
          purpose is to document the algo. However, it would be wrong to
          require things that aren't required or to allow exceptions
          when exceptions would not be interoperable. An alternative
          approach would be to just remove all capitalised language.

          

          

        
<Praveen>
            I am fine with removing such capitalization except for when
            removing it will lead to interop problems between different
            implementations of DCTCP. I am responding to each comment
            below and adding whether the new revision will fix it not.

        


          Section-by-section review.

            

          Abstract

          

          I think it needs to have an applicability statement in the
          abstract (and introduction) that refers to the deployment and
          implementation status sections at the end. Something like:

          

          "This is an informational specification of the implementation
          of DCTCP in Microsoft Windows Server 2012. It is applicable to
          deployments in controlled environments like data centres but
          it MUST NOT be deployed over the public Internet without
          additional measures, as detailed in sections 4-6. "

          

          

        
<Praveen>
            This is already addressed in the introduction. I don’t see
            the value of repeating it multiple times. An implementer
            will read the introduction. If you still feel strongly, we
            can add it to the abstract and remove it from the
            introduction section. This draft will not cover any
            additional measures that are intended for deployments
            outside the datacenter – that’s not the intended purpose. 

      

    

    [BB] It is very common for the IETF to require even much less minor
    applicability statements to be highlighted in the abstract (e.g. see
    TFO RFC). This is because nowadays implementers / readers often skip
    introductory text assuming they understand the problem, etc. and
    they just scan for the normative parts. Anyone reading an IETF RFCs
    assumes the context is the public Internet, so any RFC that doesn't
    have that context needs to flag that. I know the name says Data
    Centre, but the abstract needs to explain that this is not an IETF
    RFC that is standardizing DCTCP for the public Internet. 

    

    If you still don't think so, we should put this to the WG (which
    will slow it down).

    

      

        

        
 

        


          1. Intro

          s/its many servers/

           /their many servers/

          

          

        
<Praveen>
            Fixed in next revision.

        
 

        


          "...limited queue capacities..." 

          I understand that the motivation has been abbreviated, but I
          think this has lost too much. Perhaps mention memory shared
          between interfaces, so either bloated buffers or too little,
          making traffic susceptible to packet losses?

          

             [RFC3168] describes a
              mechanism for using Explicit Congestion

               Notification (ECN) from the switches for early
              detection of

               congestion, rather than waiting for packet loss to
              occur.

          

          This isn't correct. 3168 requires ECN marking to occur no
          earlier than drop. It is precisely that 3168 does not allow
          early marking unless there is also early drop that is the
          problem.

          

          

        
<Praveen>
            Fixed in next revision.

        
 

        


             It is recommended that
              DCTCP be deployed...

          

          "recommended" is too weak. I suggest you use something like
          the applicability text given above, both here and in the
          abstract.

        
 

        
<Praveen>
            Added the word “only” to make this stronger per Michael’s
            review. Fixed in next revision.

      

    

    [BB] I suggest the applicability para (here in the intro) refers
    forward to section 5 for details (e.g. so you don't have to explain
    why it shouldn't be deployed in the intro).

    

      

        

        
 

        


          

          2. Terminology

          

          Suggest you explain that capitalised words are not quite the
          same as in most RFCs:

          "Normative language is used to describe how necessary the
          various aspects of the Microsoft implementation are for
          interoperability, but even compliant implementations without
          the measures in sections 4-6 would still only be safe to
          deploy in controlled environments."

          

          

        
<Praveen>
            Fair enough. Added text in next revision.

        
 

        


          3. DCTCP Algo

          

          The three bullets say no more than "this is normal stuff".
          Would it be better to supplement each sentence with a brief
          phrase saying what is different about each aspect compared to
          RFC3168?

          

          

        
<Praveen>
            The goal here it to describe the changes needed on top of
            3168 so I don’t think further explanation is necessary. 

        
 

        


          3.1 Marking

          

          This needs to say something about how you mark the IP header
          from L2. I suggest you refer to the up-and-forward mode of
          draft-ietf-tsvwg-ecn-encap.

          

          

        
<Praveen>
            Why is that relevant to this document? It is not intended as
            a specification for operation of L2 switches. Won’t fix.

      

    

    [BB] Well, you specify a lot about the marking algo, e.g. 

    K > (RTT *  C)/7

    

    A simple solution would be: s/switch/L3 switch/ 

    

    I just thought that a reader would think, "You say switches (i.e. L2
    devices) mark the IP header - how can that work?" unless you
    explain.

    

    

      

        

        


          s/sending rate/

           /link rate/

          

          

        
 

        
<Praveen>
            Fixed in next revision

        
 

        
 

        


          3.2 

          

          CURRENT:

        
   1.  If the CE codepoint is set and DCTCP.CE is false, send an ACK for

        
       any previously unacknowledged packets and set DCTCP.CE to true.

        
 

        
   2.  If the CE codepoint is not set and DCTCP.CE is true, send an ACK

        
       for any previously unacknowledged packets and set DCTCP.CE to

        
       false.

        
SUGGESTED:

        
   1.  If the CE codepoint is set and DCTCP.CE is false, set DCTCP.CE to 

        
       true and send an ACK for any previously unacknowledged packets.

        
 

        
   2.  If the CE codepoint is not set and DCTCP.CE is true, set DCTCP.CE

        
       to false and send an ACK for any previously unacknowledged packets.

        
    The figure
          would have to be changed to be consistent too.

          RATIONALE:

          If the receiver changes DCTCP.CE /after/ sending the ACK like
          this, it delays each signal until the following ACK is sent.
          That could add a delay of anything from 1 to m packets. I've
          never understood why the order of these operations was
          specified in this way. If there is no reason, then I suggest
          that it is specified in the opposite order, as I describe
          above.

        


          A note could be added to say the Windows Server 12
          implementation does these steps in reverse order, but the
          order specified is preferred because it reduces signalling
          delay and has no interoperability issues with the original
          Windows implementation. 

        
<Praveen>
            Won’t fix. This is how the Windows implementation behaves so
            the ordering is required. Both the paper and the draft are
            consistent in this. 

      

    

    [BB] The paper and the draft are not consistent. In the draft, the
    labels on the top and bottom arrows in the diagram have been
    switched round relative to those in the paper. 

    

    However, the text in the draft has not (yet?) been changed to
    reflect this. 

    

      

        
There
            is no delay being introduced here. This ACK is sent
            immediately so the order does not matter. 

      

    

    

    [BB] I agree that the ACK itself is not delayed. However, even tho a
    CE has been received, feedback of this event is held back until the
    next ACK. 

    

    This is a minor 'unnecessary delay' bug. 

    * I believe it was fixed in the FreeBSD implementation after I
    pointed it out. 

    * I've just checked the Linux code, and it's not been fixed there. 

    * Obviously I cannot see the Windows source code. 

    

    If this bug is still in the Windows implementation, it's up to you
    if you also want the spec to require that the bugs are implemented
    exactly as they are in Windows ;)

    

    

      

        

        
       

        
   The handling of the "Congestion Window Reduced" (CWR) bit is also

        
   exactly as per [RFC3168] including [RFC3168-ERRATA3639].  That is, on

        
   receipt of a segment with both the CE and CWR bits set, CWR is

        
   processed first and then ECE is processed.

        
Even tho this is the receiver section, I
          suggest:

          s/The handling/Receiver handling/

          

          

        
<Praveen>
            Isn’t this assumed by a reader familiar with 3168? But this
            is a minor editorial fix so fixed in the next revision.

        


          Whatever, this cannot be correct. A DCTCP receiver surely does
          nothing on receipt of CWR, whereas an RFC3168 receiver does
          something. A DCTCP receiver certainly cannot do exactly what
          RFC3168 says, because that says stop setting ECE on receipt of
          CWR until the next CE arrives, which would surely break this
          feedback protocol.

          

          

        
<Praveen>
            I don’t see why this would break the feedback protocol. The
            receiver will start echoing again as soon as it sees the
            first CE arrives again. If packet has both CWR and CE set
            then CE takes precedence and the echoing is done.

      

    

    [BB] I'm not saying it breaks the feedback protocol. I'm just saying
    it's confusing for the reader.

    

    Of course, when the receiver is already not doing something,
    ignoring a signal to stop has the same outcome as stopping. However,
    describing this as "handling CWR ... is also exactly as per RFC3168"
    is misleading for the reader/implementer.

    

    I guess changing to "handling CWR ... is as per RFC3168" would be a
    compromise.

    

      

        

        
 

        


          3.3 

        
   DCTCP.Alpha, which is initialized to 1 and MUST be updated

        
   as follows:

        
s/MUST/SHOULD/

          And add "An alternative congestion avoidance algorithm MAY be
          used, but it MUST lead to flow rate inversely proportional to
          1/M".

          

          Rationale: There are many ways to implement interoperable 1/p
          cc. This is just one. 

        
 

        
<Praveen>
            Alternate algorithms are out of scope of this draft. We
            cannot say without interop testing if a different algorithm
            will perform well in datacenters or interop well with
            existing implementations. If there are other ways to
            implement 1/p they should be covered by other drafts. This
            draft will only describe the algorithm that has been tested
            and deployed. That said I am willing to make it a SHOULD so
            fixed in the next revision.

        
 

        
 

        
   DCTCP.BytesSent

        
This variable
          actually counts bytes received. it's pretty confusing to call
          it BytesSent.

          

          

        
<Praveen>
            No, this is tracking bytes that have been sent. 

      

    

    [BB] I meant it counts bytes now acknowledged as received by the
    other end (which is not the same as bytes sent by this end). 

    DCTCP.BytesAcked would have been more meaningful. I'm not insisting
    on this - the original comment was just to improve
    comprehensibility.

    

      

        

        
 

        


          s/TCP SACK options [RFC2018] are ignored./

           /The sender MUST ignore TCP SACK options [RFC2018] for this
          computation./

          RATIONALE: 

            a) I think the idea is to ignore CE marks after a gap
          because, if the gap becomes considered a loss, the response to
          loss will override any need to count incoming CE marks for the
          following round trip. However, if the gap is filled before it
          is considered a loss {Note 1}, then CE marks that were ignored
          because they arrived after a gap will need to be "unignored".
          

            b) We need to make clear that SACK is not ignored
          altogether, only for this computation.

          

          {Note 1}: I can think of two cases:

          * Less than three dupacks then the gap is filled

          * The gap is filled by a delayed segment so that an earlier
          retransmission was spurious

        
<Praveen>
            Ok I will update the text.

        
 

        
 

        
the sender MUST update cwnd as follows:

        
      cwnd = cwnd * (1 - DCTCP.Alpha / 2)

        
s/MUST/SHOULD/

          Rationale: Alternative interoperable response functions are
          possible. For instance an algorithm like Relentless TCP that
          simply reduces cwnd by half the size of any CE-marked segment.
          Then there is no Alpha and no g variable.

          

          

        
 

        
<Praveen>
            Fixed

        
 

        
 

        


          CURRENT:

        
   Just as specified in [RFC3168], TCP should not react to congestion

        
   indications more than once for every window of data.

        
SUGGESTED:

        
   Just as specified in [RFC3168], DCTCP does not react to congestion

        
   indications more than once for every window of data.

        
I don't know whether you intended this to
          be an upper-case SHOULD NOT. If so, I would argue against this
          being normative, because it isn't necessary for interop;
          therefore I've suggested avoiding he 'should' word.

        
 

        
<Praveen> Fixed

        
 

        
 

        
 

        
   The setting of

        
   the "Congestion Window Reduced" (CWR) bit is also exactly as per

        
   [RFC3168].

        
 

        
 

        
It is wrong to say "exactly". The sender
          certainly sets CWR when it does a congestion response. But in
          DCTCP the congestion response occurs every RTT, triggered by
          its own measurement-period logic, whereas in RFC3168 it is
          triggered by the arrival of an ECE after the ACK of any
          previous segment it sent with CWR set.

          

          I think it is worth saying that setting CWR is necessary for
          safety, otherwise in the case where there has been a
          configuration management error and the receiver complies with
          RFC 3168 not DCTCP, it will continue sending ECE for ever and
          starve the session.

          

          It might help to explicitly confirm that: 

          "Other aspects of TCP congestion avoidance and control such as
          the slow start phase are no different to those in RFC5681."

          

          

        
 

        
<Praveen>
            Fixed that CWR is needed for safety. Not adding the text
            around 5681, it is presumed.

        
 

        


          3.4 SYN, SYN-ACK, RST

          

             [RFC3168] requires that
              a compliant TCP MUST NOT set ECT on SYN or

               SYN-ACK packets.

          

          s/MUST NOT/'MUST NOT'/

          Rationale: Best to make it clear this is quoting a normative
          statement in another RFC, because it is going to be
          contradicted.

        

      

    

    [BB] I don't know whether you saw this one - it's not been changed
    in draft-02.

    

    I suggested adding quotes around 'MUST NOT'. This is good practice
    when quoting another RFC. Otherwise, to a careless reader (or
    non-native English speaker), it can look like it is also a
    requirement of this RFC.

    

      

        
 

          CURRENT

        
   These RFCs, however, are

        
   intended for general Internet use, and do not directly apply to a

        
   controlled datacenter environment.

        
...

        
   For DCTCP connections, the sender

        
   SHOULD set ECT for SYN, SYN-ACK and RST packets.

        
SUGGESTED:

          

             Also, a SYN is sent
              before the ECN-capability has been

               negotiated, so if it is CE-marked, a non-ECN TCP
              server would not

               recognise the marking. RFC 3168 does not specify the
              ECN field on 

               a RST. The security concerns addressed by these RFCs
            

               might not apply in controlled environments like data
              centres, and 

               it might not be necessary to cater for the presence
              of non-ECN 

               servers.

            ...

        
   Therefore, the sender MUST set ECT for SYN/ACK and RST packets and a 

        
   configuration option SHOULD be provided to set ECT for SYNs.

        
RATIONALE:

             The draft should avoid assuming that security concerns do
          not apply in all controlled environments.

          NOTE:

             This might not reflect the way your implementation is
          coded, so you might want to make that clear.

          

          

        
<Praveen>
            Adding some text around security concerns.

      

    

    [BB] The security sentence you have added in draft-02 doesn't make
    sense where it is. It ought to have been added once sentence earlier
    (and it would be better to start a new para). Specifically:

    CURRENT:

                    be dropped with high probability.  For DCTCP
      connections, the sender    

                    SHOULD set ECT for SYN, SYN-ACK and RST
      packets.  The security    

                    concerns addressed by both these RFCs might
      not apply in controlled    

                    environments like datacenters, and it might
      not be necessary to cater    

                    to both the presence of non-ECN servers.

    SUGGESTED:

                    be dropped with high probability.  

        <NEW PARA>

                        The security concerns
        addressed by both these RFCs might not apply in controlled    

                      environments like datacenters, and it
        might not be necessary to cater    

                      for the presence of non-ECN
        servers. For DCTCP connections, the sender    

                    SHOULD set ECT for SYN, SYN-ACK and RST
      packets.  

    

     (Note I also fixed a glitch after "cater".)

    

    

    

      

        

        
 

        
 

        


          4. Implementation Issues

          

          CURRENT:

        
   the implementation MUST choose a suitable

        
   estimation gain.  [DCTCP10] provides a theoretical basis 

        
SUGGEST:

        
   the implementation will need to choose a suitable

        
   estimation gain.  [ADCTCP10] provides a theoretical basis ...

        
RATIONALE:

             a) As above, this is only one way to implement the cc, and
          others could well prove to be interoperable when tested.

             b) More particularly, it would be better for an
          implementation to adapt the estimation gain to the RTT, so the
          spec ought not to imply that one value has to be chosen.

             c) The later paper [ADCTCP10] provides a corrected way to
          determine the gain. From memory (I'm offline at the mo) it at
          least makes throughput inversely proportional to 1/RTT,
          whereas the approach in [DCTCP10] made it inversely
          proportional to 1/(RTT^2). I believe the Linux implementation
          uses the [ADCTCP10] approach.

        
 

        
<Praveen>
          Fair enough. Updated.

        
 

        
   The implementation must also decide when to use DCTCP.

        
...

        
   likely in the same datacenter network.

        
The 2nd & 3rd paras are a mix of
          implementation and deployment issues, but mostly deployment,
          so they might be better moved to the deployment issues
          section.

          

          

        
 

        
<Praveen>
            I think they are mainly implementation issues – the fact
            that a lack of negotiation mechanism places a burden on the
            implementation to provide knobs or heuristics to use DCTCP.

        
 

        
 

        


          CURRENT:

        
   It is RECOMMENDED that an implementation deal with loss episodes in

        
   the same way as conventional TCP.

        
...

        
   DCTCP implementation MAY also allow configuration of resetting the

        
   value of DCTCP.Alpha as part of processing any loss episodes.

        
SUGGESTED: Move this whole para to 3.3
          Sender Behaviour section, and:

        
   An implementation MUST deal with loss episodes in

        
   the same way as conventional TCP.

        
...

        
   DCTCP implementation SHOULD also reset the

        
   value of DCTCP.Alpha as part of processing any loss episodes. 

        
   This aspect of the behaviour MAY be configurable.

        
RATIONALE:

             a) There would never be a case for not falling back to
          conventional TCP  (or do you have one?), so "RECOMMENDED" is
          too weak.

             b) I have tried to interpret what you might have meant
          about resetting Alpha. I'm not sure why you think it might
          need to be configurable though?

        
<Praveen>
          Yes this is a useful suggestion. Making loss handling a must
          in sender section. Reset of alpha was discussed in tcpm list,
          and Linux implementation supports it. 

          

      

    

    

      

        
 

        
   ...it is also RECOMMENDED that an implementation allow

        
   configuration of restarting the congestion window (cwnd) of idle

        
   DCTCP connections

        
Where special implementation measures are
          necessary for low RTT, these should be related to the measured
          RTT, not hard-coded. There is an assumption in a number of
          places  that "data centre" implies low RTT. Of course this is
          true in a single DC, but I believe the the single admin aspect
          of a DC is what characterises DCTCP, not the low RTT aspect.
          With the modifications in [ADCTCP], DCTCP generalises to a
          network with larger RTTs as long as it is still  operated by a
          single admin. 

          

          

        
 

        
<Praveen>
            this paragraph has now been removed per Michaels’ feedback.

        
 

        


          CURRENT:

             [RFC3168] forbids the
              ECN-marking of pure ACK packets, because of the

               inability of TCP to mitigate ACK-path congestion and
              protocol-wise

               preferential treatment by routers.  However, dropping
              pure ACKs -

               rather than ECN marking them - has disadvantages for
              typical

               datacenter traffic patterns.  Because of the
              prevalence of bursty

               traffic patterns that feature transient congestion,
              dropping of ACKs

               causes subsequent retransmissions. It is RECOMMENDED
              that an

               implementation provide a configuration knob that will
              cause ECT to be set

               on such control packes, which can be used in
              environments where such  

               concerns do not apply.

          SUGGESTED:

             [RFC3168] forbids the
              ECN-marking of pure ACK packets, because of the

               inability of TCP to mitigate ACK-path congestion and
              the extra 

               advantage to injection attackers that ECN is
              perceived to offer. 

               For the latter reason RFC 3168 also forbids
              ECN-marking of 

               retransmissions, window probes and RSTs. 

            

               However, dropping all these control packets -

               rather than ECN marking them - has considerable
              performance 

               disadvantages.  It is RECOMMENDED that an

               implementation provide a configuration knob that will
              cause ECT to be set

               on such control packes, which can be used in
              environments where such  

               concerns do not apply.

          NOTE:

             This might not reflect the way your implementation is
          coded, so you might want to make that clear.

          

          

        
 

        
 

        
<Praveen>
            Fixed

      

    

    [BB] A new comment...

    

    As pointed out in draft-bagulo-tsvwg-generalized-ecn-00, RFC3168
    only talks about injection attacks with retransmissions (not ACKs
    etc). But you imply injection attacks were the concern for other
    control packets as well.

    

    I'm not insisting you change this. However, unwarranted accusations
    of security vulnerabilities tend to stick, whether or not they are
    correct. So it would be better not to throw mud around onto even
    more control packets than RFC3168 did.

    

    Cheers

    

    

    Bob

    

    

      

        

        


          

          5. Deployment Issues

        
   If

        
   the traffic in the datacenter is a mix of conventional TCP and DCTCP,

        
   it is RECOMMENDED that DCTCP traffic be segregated from conventional

        
   TCP traffic.

        
In a data
          centre (or anywhere), is there any case where not segregating
          them would make sense? If not, this needs to be a MUST, not
          just a RECOMMENDED. I can imagine cases where there is no
          long-running conventional TCP or no long-running DCTCP, but
          that's covered by the "if" at the start of the sentence.

          

          You might want to refer to draft-briscoe-aqm-dualq-coupled as
          another segregation approach here too.

        
   Since DCTCP relies on congestion marking by the switches, DCTCP can

        
   only be deployed in datacenters where the entire network

        
   infrastructure supports ECN.

        
Surely, the
          "fall-back to conventional TCP on loss" rule means you can
          delete this
constraint.                                                                       

        
 

        
<Praveen>
          Reworded to imply that DCTCP will only work as expected in
          such cases.

        
 

        
   A

        
   variant of DCTCP that can be deployed unilaterally and only requires

        
   standard ECN behavior has been described in [ODCTCP][BSDCAN],

        


          Which part of "standard ECN behaviour" do you mean? The host
          part? The network part? The receiver part? And "can be
          deployed unilaterally" ought to be in the active voice not the
          passive, which would help resolve the ambiguity too.

          

          I haven't read [ODCTCP] and [BSDCAN] (I'm offline on a plane
          at the mo). Are they similar to "Instant ECN" in [Wu12] listed
          below? If not, that could be referred to as well for a
          technique that uses the regular ECN host behaviour.

          

          

        
 

        
<Praveen>
            This edit predates me and  I don’t have the full context, I
            am leaving this as is.

        
 

        
 

        


          6. Known Issues

          

          First para could refer to appendix A of RFC7560, which
          describes the problem precisely.

        
 

        
 

        
 

        
A method for improving the fairness of DCTCP has been proposed

        
   in [ADCTCP], but requires additional experimental evaluation.

        
s/fairness/RTT fairness/

          

          (In our experiments, it's much better than without.)

          

          

        
 

        
<Praveen>
            Goal is to cite other work that can address potential
            issues. It is not a recommendation. s/fairness/RTT fairness/
            Fixed in next revision.

        
 

        


          11. References

          

          I think the following are cited normatively, not just
          informatively:

          RFC5681, RFC5562.

          

          Additional informational Reference:

          

          [Wu12] Wu, H., Ju, J., Lu, G., Guo, C., Xiong, Y. & Zhang,
          Y., "Tuning ECN for Data Center Networks," In: Proceedings of
          the 8th International Conference on Emerging Networking
          Experiments and Technologies CoNEXT '12 pp.25-36 ACM (2012)

          

          

        
 

        
<Praveen>
            Citation to RFCs updated as normative. Not including the new
            reference since even though it may be relevant – it was not
            used as a reference in writing the draft.

        
 

        


          That's it. HTH.

          

          

          Bob

          

          

        
-- 

        
________________________________________________________________

        
Bob Briscoe                               http://bobbriscoe.net/

      

    

    

    
-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/

    
-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/

  


--------------24759AAECE54BB7370BFF4EE--


From nobody Thu Aug 18 09:56:44 2016
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id BFFFF12D1ED for ; Thu, 18 Aug 2016 09:56:23 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -5.447
X-Spam-Level: 
X-Spam-Status: No, score=-5.447 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RP_MATCHES_RCVD=-1.247] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TqZVJ-UA2Ny5 for ; Thu, 18 Aug 2016 09:55:28 -0700 (PDT)
Received: from pegasus.erg.abdn.ac.uk (pegasus.erg.abdn.ac.uk [139.133.204.173]) by ietfa.amsl.com (Postfix) with ESMTP id 1614F12DA15 for ; Thu, 18 Aug 2016 09:54:54 -0700 (PDT)
Received: from dhcp-207-153.erg.abdn.ac.uk (unknown [IPv6:2001:630:241:207:b51c:f872:e0b8:c526]) by pegasus.erg.abdn.ac.uk (Postfix) with ESMTPSA id 264D81B0024B; Thu, 18 Aug 2016 17:54:45 +0100 (BST)
References: <0CC24FC1-37E1-4125-9627-05726A9D9406@mnot.net> <655C07320163294895BBADA28372AF5D489E64AF@FR712WXCHMBA15.zeu.alcatel-lucent.com>
To: "Scharf, Michael (Nokia - DE)" , Mark Nottingham , "tcpm@ietf.org" 
From: Gorry Fairhurst 
Organization: The University of Aberdeen is a charity registered in Scotland,  No SC013683.
Message-ID: <61fa7faa-d2a3-e8f3-9497-1f0f93e7b81e@erg.abdn.ac.uk>
Date: Thu, 18 Aug 2016 17:54:48 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:45.0) Gecko/20100101 Thunderbird/45.0
MIME-Version: 1.0
In-Reply-To: <655C07320163294895BBADA28372AF5D489E64AF@FR712WXCHMBA15.zeu.alcatel-lucent.com>
Content-Type: text/plain; charset=windows-1252; format=flowed
Content-Transfer-Encoding: 7bit
Archived-At: 
Cc: Patrick McManus , Daniel Stenberg 
Subject: Re: [tcpm] TCP Tuning for HTTP - update
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
Reply-To: gorry@erg.abdn.ac.uk
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Thu, 18 Aug 2016 16:56:23 -0000

Speaking as someone who commented on an earlier version:

I think it is good to offer advice on how TCP is used for web traffic. I 
can see there is likely useful information that can document how some 
things are done, and perhaps also recommend some changes. I support 
continuing work on this.

Publishing a document like this in the RFC series needs to be consistent 
with IETF consensus. The ID touches on many things that are assigned to 
TCPM. (The breadth of topics may be a good reason for a document - but 
is also a concern, because many things need more clear description 
before the recommendations become clear).

I think this document should NOT be considered as BCP. It talks about 
system configuration of endpoints, and of the recommendations I see, 
some do support current PSs, some set new precedents against current PS 
RFCs and in some cases the present text appears to endorse EXP RFCs. If 
published as a BCP, that sort of advice is likely to be confusing. Such 
information can also easily be outdated, I suggest if adopted, this 
should be Informational and clearly set the scope of the recommendations.

If this work continues, it needs to be reviewed by TCPM and HTTPbis. 
This would need much more than simply a last-call in both working groups 
- but I'm sure with proper coordination progress could be made so that 
there were no surprises for either group at the time of WGLC!

Gorry


>
> -----Original Message-----
> From: tcpm [mailto:tcpm-bounces@ietf.org] On Behalf Of Mark Nottingham
> Sent: Wednesday, August 17, 2016 4:03 AM
> To: tcpm@ietf.org
> Cc: Patrick McManus; Daniel Stenberg
> Subject: [tcpm] TCP Tuning for HTTP - update
>
> Hi TCPM,
>
> Just a quick note; Daniel and Tim have made an update to the TCP Tuning for HTTP draft:
>   https://tools.ietf.org/html/draft-stenberg-httpbis-tcp
>
> We've had a Call for Adoption open for this draft for a while, and will likely adopt it soon. However, we'd like to get feedback from this community first -- both about the latest version of the input document, and to see if there's interest in helping out.
>
> You can give feedback on the HTTP WG mailing list , or  by responding to this e-mail (Please leave the CC line; Patrick and I will try to summarise the feedback to the WG).
>
> Cheers,
>
> --
> Mark Nottingham   https://www.mnot.net/
>
>
>
>
> _______________________________________________
> tcpm mailing list
> tcpm@ietf.org
> https://www.ietf.org/mailman/listinfo/tcpm
>
> _______________________________________________
> tcpm mailing list
> tcpm@ietf.org
> https://www.ietf.org/mailman/listinfo/tcpm
>


From nobody Thu Aug 18 10:34:54 2016
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 210DF12DA59 for ; Thu, 18 Aug 2016 10:34:48 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -3.247
X-Spam-Level: 
X-Spam-Status: No, score=-3.247 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RP_MATCHES_RCVD=-1.247, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=google.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 01KqztDQ7-Yz for ; Thu, 18 Aug 2016 10:34:43 -0700 (PDT)
Received: from mail-it0-x229.google.com (mail-it0-x229.google.com [IPv6:2607:f8b0:4001:c0b::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 97BDA12D9B2 for ; Thu, 18 Aug 2016 10:34:43 -0700 (PDT)
Received: by mail-it0-x229.google.com with SMTP id e63so4494240ith.1 for ; Thu, 18 Aug 2016 10:34:43 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=+gxCj5K6AaULF+yMhjRz/a7m1kZ6oz8pv7AXHlxXFUc=; b=IFmTgNzoZEd7lDV7X41ZbvzszoykguK4NUB9DabtwXgOdX4X6c9mJeNIbo9KzHV58b KTZN/ZAOsFskcDp5lgdRknc/PZN0AP9Pr34/pvodUZN9me/nHLJ2pQd5jOARo0VBDd4g RkDuA64rDF8zrECb4UlDA1DOQYQR5rCcKLheCKLU2gYqgOBbveqzPkeG3b3NBNlwz+B5 Z5f2dlkEO4aJa0GkZYuwMjWUCsS6eQHcmbxWalUDE935tG4g08tYIiZWJWv8MZIzFUS5 QoJ4npkJNiSx6CllIW2Zdv1gl1h5ykG9iym9VqZ9IKmWHi0alnN6J50leb58oJup7hxY zAsw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=+gxCj5K6AaULF+yMhjRz/a7m1kZ6oz8pv7AXHlxXFUc=; b=lvZ/K1J0Adq2CkFBOSXUJ8cY93HTF/hTmtz5SgcZYNq/9kyYnobJs64eOEiQeMgVAJ IamfZAgM4KQxhlQmJ5hTI6879cZw35ylzu/fQ69G58BcRLWTXbkKm2iaT1PQqGyaxQgt 8edArqCrK4tTdDRfDT1r7BVKEvxOFTIn9DDPFkazGi7LxR4q5J/8EPP5pEqAXFfDU7hy X/GcXdpw+AER1zCko8bTOBh9UhFFjA1YJT7Hm7S7LQPASaT1XvEkMleLPNLU9bfnnkMl 0Q/ib092/3u94u7ZxQ97gGnMagC4CNieOMu3zgvxulWrgAeJQ+1JejpP16+xOudd0+kP dBGQ==
X-Gm-Message-State: AEkoous9yi2hjp86ifCcaJjqDkXrNweQX6I+MBaoQ1Xj3FraI75dmxUFLMnDnR7TkB5K1vuXBGJEHfhNcUz/xp5o
X-Received: by 10.36.142.4 with SMTP id h4mr816316ite.65.1471541682658; Thu, 18 Aug 2016 10:34:42 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.64.244.103 with HTTP; Thu, 18 Aug 2016 10:34:01 -0700 (PDT)
In-Reply-To: <61fa7faa-d2a3-e8f3-9497-1f0f93e7b81e@erg.abdn.ac.uk>
References: <0CC24FC1-37E1-4125-9627-05726A9D9406@mnot.net> <655C07320163294895BBADA28372AF5D489E64AF@FR712WXCHMBA15.zeu.alcatel-lucent.com> <61fa7faa-d2a3-e8f3-9497-1f0f93e7b81e@erg.abdn.ac.uk>
From: Yuchung Cheng 
Date: Thu, 18 Aug 2016 10:34:01 -0700
Message-ID: 
To: Gorry Fairhurst 
Content-Type: multipart/alternative; boundary=94eb2c04a26c44c108053a5bffc6
Archived-At: 
Cc: "tcpm@ietf.org" , Mark Nottingham , Patrick McManus , Daniel Stenberg 
Subject: Re: [tcpm] TCP Tuning for HTTP - update
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Thu, 18 Aug 2016 17:34:48 -0000

--94eb2c04a26c44c108053a5bffc6
Content-Type: text/plain; charset=UTF-8

On Thu, Aug 18, 2016 at 9:54 AM, Gorry Fairhurst 
wrote:

> Speaking as someone who commented on an earlier version:
>
> I think it is good to offer advice on how TCP is used for web traffic. I
> can see there is likely useful information that can document how some
> things are done, and perhaps also recommend some changes. I support
> continuing work on this.
>
> Publishing a document like this in the RFC series needs to be consistent
> with IETF consensus. The ID touches on many things that are assigned to
> TCPM. (The breadth of topics may be a good reason for a document - but is
> also a concern, because many things need more clear description before the
> recommendations become clear).
>
> I think this document should NOT be considered as BCP. It talks about
> system configuration of endpoints, and of the recommendations I see, some
> do support current PSs, some set new precedents against current PS RFCs and
> in some cases the present text appears to endorse EXP RFCs. If published as
> a BCP, that sort of advice is likely to be confusing. Such information can
> also easily be outdated, I suggest if adopted, this should be Informational
> and clearly set the scope of the recommendations.
>
+1 on informational

It'd be great for the document to cite more detailed technical reports, for
readers to learn the pros and cons, and the history. IMO it'd significantly
improve this document. The reason is that TCP and network will continue to
evolve, so knowing the past and details can help readers to judge whether a
specific recommendation is still useful or not, if he is willing to spend
time to explore. For example, he can use some online tools to track the
citation of these reports and see new reports.

For the document, I'd recommend two more items that my personal experience
gives remarkable performance enhancement on reducing and recover from
losses.

1) make sure your stack supports SACK
2) enable pacing.

On disabling Nagle, there are also better APIs like Linux Cork so you don't
have to send small packets even on continuous writes. One common problem I
often see is developers unconditionally disable Nagle, but their HTTP
workload may lead to lots of small packets.

my 2c


> If this work continues, it needs to be reviewed by TCPM and HTTPbis. This
> would need much more than simply a last-call in both working groups - but
> I'm sure with proper coordination progress could be made so that there were
> no surprises for either group at the time of WGLC!
>
> Gorry
>
>
>
>
>> -----Original Message-----
>> From: tcpm [mailto:tcpm-bounces@ietf.org] On Behalf Of Mark Nottingham
>> Sent: Wednesday, August 17, 2016 4:03 AM
>> To: tcpm@ietf.org
>> Cc: Patrick McManus; Daniel Stenberg
>> Subject: [tcpm] TCP Tuning for HTTP - update
>>
>> Hi TCPM,
>>
>> Just a quick note; Daniel and Tim have made an update to the TCP Tuning
>> for HTTP draft:
>>   https://tools.ietf.org/html/draft-stenberg-httpbis-tcp
>>
>> We've had a Call for Adoption open for this draft for a while, and will
>> likely adopt it soon. However, we'd like to get feedback from this
>> community first -- both about the latest version of the input document, and
>> to see if there's interest in helping out.
>>
>> You can give feedback on the HTTP WG mailing list <
>> https://lists.w3.org/Archives/Public/ietf-http-wg/>, or  by responding
>> to this e-mail (Please leave the CC line; Patrick and I will try to
>> summarise the feedback to the WG).
>>
>> Cheers,
>>
>> --
>> Mark Nottingham   https://www.mnot.net/
>>
>>
>>
>>
>> _______________________________________________
>> tcpm mailing list
>> tcpm@ietf.org
>> https://www.ietf.org/mailman/listinfo/tcpm
>>
>> _______________________________________________
>> tcpm mailing list
>> tcpm@ietf.org
>> https://www.ietf.org/mailman/listinfo/tcpm
>>
>>
> _______________________________________________
> tcpm mailing list
> tcpm@ietf.org
> https://www.ietf.org/mailman/listinfo/tcpm
>

--94eb2c04a26c44c108053a5bffc6
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable




On Thu, Aug 18, 2016 at 9:54 AM, Gorry Fairhurst <=
gorry@erg.abdn.ac=
.uk> wrote:
Speaking as som=
eone who commented on an earlier version:



I think it is good to offer advice on how TCP is used for web traffic. I ca=
n see there is likely useful information that can document how some things =
are done, and perhaps also recommend some changes. I support continuing wor=
k on this.



Publishing a document like this in the RFC series needs to be consistent wi=
th IETF consensus. The ID touches on many things that are assigned to TCPM.=
 (The breadth of topics may be a good reason for a document - but is also a=
 concern, because many things need more clear description before the recomm=
endations become clear).



I think this document should NOT be considered as BCP. It talks about syste=
m configuration of endpoints, and of the recommendations I see, some do sup=
port current PSs, some set new precedents against current PS RFCs and in so=
me cases the present text appears to endorse EXP RFCs. If published as a BC=
P, that sort of advice is likely to be confusing. Such information can also=
 easily be outdated, I suggest if adopted, this should be Informational and=
 clearly set the scope of the recommendations.
+1 on i=
nformational=C2=A0

It'd be great for the docum=
ent to cite more detailed technical reports, for readers to learn the pros =
and cons, and the history. IMO it'd significantly improve this document=
. The reason is that TCP and network will continue to evolve, so knowing th=
e past and details can help readers to judge whether a specific recommendat=
ion is still useful or not, if he is willing to spend time to explore. For =
example, he can use some online tools to track the citation of these report=
s and see new reports.

For the document, I'd r=
ecommend two more items that my personal experience gives remarkable perfor=
mance enhancement on reducing and recover from losses.

=

1) make sure your stack supports SACK
2) enable pacing.=C2=
=A0

On disabling Nagle, there are also better APIs=
 like Linux Cork so you don't have to send small packets even on contin=
uous writes. One common problem I often see is developers unconditionally d=
isable Nagle, but their HTTP workload may lead to lots of small packets.=C2=
=A0

my 2c



If this work continues, it needs to be reviewed by TCPM and HTTPbis. This w=
ould need much more than simply a last-call in both working groups - but I&=
#39;m sure with proper coordination progress could be made so that there we=
re no surprises for either group at the time of WGLC!



Gorry










-----Original Message-----

From: tcpm [mailto:tcpm-bounces@ietf.org] On Behalf Of Mark Nottingham

Sent: Wednesday, August 17, 2016 4:03 AM

To: tcpm@ietf.org
Cc: Patrick McManus; Daniel Stenberg

Subject: [tcpm] TCP Tuning for HTTP - update



Hi TCPM,



Just a quick note; Daniel and Tim have made an update to the TCP Tuning for=
 HTTP draft:

=C2=A0 https://tools.ietf.org/html/draft-=
stenberg-httpbis-tcp



We've had a Call for Adoption open for this draft for a while, and will=
 likely adopt it soon. However, we'd like to get feedback from this com=
munity first -- both about the latest version of the input document, and to=
 see if there's interest in helping out.



You can give feedback on the HTTP WG mailing list <https://lists.w3.org/Archives/Public/ietf-http-wg/>, or=C2=
=A0 by responding to this e-mail (Please leave the CC line; Patrick and I w=
ill try to summarise the feedback to the WG).



Cheers,



--

Mark Nottingham=C2=A0 =C2=A0https://www.mnot.net/









_______________________________________________

tcpm mailing list

tcpm@ietf.org

https://www.ietf.org/mailman/listinfo/tcpm



_______________________________________________

tcpm mailing list

tcpm@ietf.org

https://www.ietf.org/mailman/listinfo/tcpm







_______________________________________________

tcpm mailing list

tcpm@ietf.org

https://www.ietf.org/mailman/listinfo/tcpm





--94eb2c04a26c44c108053a5bffc6--


From nobody Thu Aug 18 10:55:52 2016
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6656912DA40 for ; Thu, 18 Aug 2016 10:55:15 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -8.166
X-Spam-Level: 
X-Spam-Status: No, score=-8.166 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-1.247] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id VFfX0Py8cZdd for ; Thu, 18 Aug 2016 10:54:52 -0700 (PDT)
Received: from boreas.isi.edu (boreas.isi.edu [128.9.160.161]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 5FE8312DA61 for ; Thu, 18 Aug 2016 10:51:56 -0700 (PDT)
Received: from [128.9.184.42] ([128.9.184.42]) (authenticated bits=0) by boreas.isi.edu (8.13.8/8.13.8) with ESMTP id u7IHotxD014616 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Thu, 18 Aug 2016 10:50:56 -0700 (PDT)
To: Willy Tarreau 
References: <5CD67877-19E3-4E79-BBF2-3E270343A378@mnot.net> <2197232f-10d7-28cb-fcc9-05bd495e3c22@isi.edu> <20160817064545.GD16017@1wt.eu> <7f7b129c-f156-d067-bef8-4a2213f461ac@isi.edu> <20160817180802.GA16773@1wt.eu> <4ab7c5b0-3722-1346-f481-a8d76de70034@isi.edu> <20160817211317.GA16929@1wt.eu>   <27b58b64-48cd-39af-78b3-ef583c585fa6@isi.edu> <20160818053837.GC16773@1wt.eu>
From: Joe Touch 
Message-ID: <3c93805c-05f5-2b69-235e-e1c68363bba6@isi.edu>
Date: Thu, 18 Aug 2016 10:50:52 -0700
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0
MIME-Version: 1.0
In-Reply-To: <20160818053837.GC16773@1wt.eu>
Content-Type: multipart/alternative; boundary="------------F3F89CB6AD11B845A96AEA1A"
X-ISI-4-43-8-MailScanner: Found to be clean
X-MailScanner-From: touch@isi.edu
Archived-At: 
Cc: tcpm@ietf.org, Matthew Kerwin , Daniel Stenberg , Mark Nottingham , HTTP Working Group , Patrick McManus 
Subject: Re: [tcpm] TCP Tuning for HTTP - update
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Thu, 18 Aug 2016 17:55:15 -0000
X-List-Received-Date: Thu, 18 Aug 2016 17:55:15 -0000

This is a multi-part message in MIME format.
--------------F3F89CB6AD11B845A96AEA1A
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: 7bit

Hi, Willy,

On the issue of ACK compression effects:


On 8/17/2016 10:38 PM, Willy Tarreau wrote:
>>     - watch out for ACK compression effects (turn it off in favor of ABC
>> > if you can)
> It does not happen that much with HTTP. Many connections on the server side
> see only one, sometimes two requests, and most responses are small (about
> 20kB on average, with favicon fitting in a single segment). Note, I'm talking
> about observations on average web sites.
The effect is more pronounced for smaller responses, for any response
using an odd number of packets. The last odd packet will be stalled
because the client needs to timeout before it will send an ACK for a
single segment (it's waiting for the second segment).

It doesn't matter how many requests you have, but the impact can be
complicated on persistent connections.

Joe

--------------F3F89CB6AD11B845A96AEA1A
Content-Type: text/html; charset=windows-1252
Content-Transfer-Encoding: 7bit


  
    
  
  
    
Hi, Willy,

    
On the issue of ACK compression effects:

    

    

    
On 8/17/2016 10:38 PM, Willy Tarreau
      wrote:

    

    

      

        
    - watch out for ACK compression effects (turn it off in favor of ABC
> if you can)


      

      
It does not happen that much with HTTP. Many connections on the server side
see only one, sometimes two requests, and most responses are small (about
20kB on average, with favicon fitting in a single segment). Note, I'm talking
about observations on average web sites.

    

    The effect is more pronounced for smaller responses, for any
    response using an odd number of packets. The last odd packet will be
    stalled because the client needs to timeout before it will send an
    ACK for a single segment (it's waiting for the second segment).

    

    It doesn't matter how many requests you have, but the impact can be
    complicated on persistent connections.

    

    Joe

  


--------------F3F89CB6AD11B845A96AEA1A--


From nobody Thu Aug 18 13:00:03 2016
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id EC22812D0C5 for ; Thu, 18 Aug 2016 13:00:00 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.902
X-Spam-Level: 
X-Spam-Status: No, score=-1.902 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lshfegff8ZQ4 for ; Thu, 18 Aug 2016 12:59:59 -0700 (PDT)
Received: from 1wt.eu (wtarreau.pck.nerim.net [62.212.114.60]) by ietfa.amsl.com (Postfix) with ESMTP id 96CCE12B069 for ; Thu, 18 Aug 2016 12:59:57 -0700 (PDT)
Received: (from willy@localhost) by pcw.home.local (8.15.2/8.15.2/Submit) id u7IJxmUi017969; Thu, 18 Aug 2016 21:59:48 +0200
Date: Thu, 18 Aug 2016 21:59:48 +0200
From: Willy Tarreau 
To: Joe Touch 
Message-ID: <20160818195948.GD17944@1wt.eu>
References: <20160817064545.GD16017@1wt.eu> <7f7b129c-f156-d067-bef8-4a2213f461ac@isi.edu> <20160817180802.GA16773@1wt.eu> <4ab7c5b0-3722-1346-f481-a8d76de70034@isi.edu> <20160817211317.GA16929@1wt.eu>   <27b58b64-48cd-39af-78b3-ef583c585fa6@isi.edu> <20160818053837.GC16773@1wt.eu> <3c93805c-05f5-2b69-235e-e1c68363bba6@isi.edu>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <3c93805c-05f5-2b69-235e-e1c68363bba6@isi.edu>
User-Agent: Mutt/1.6.0 (2016-04-01)
Archived-At: 
Cc: tcpm@ietf.org, Matthew Kerwin , Daniel Stenberg , Mark Nottingham , HTTP Working Group , Patrick McManus 
Subject: Re: [tcpm] TCP Tuning for HTTP - update
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Thu, 18 Aug 2016 20:00:01 -0000

On Thu, Aug 18, 2016 at 10:50:52AM -0700, Joe Touch wrote:
> Hi, Willy,
> 
> On the issue of ACK compression effects:
> 
> 
> On 8/17/2016 10:38 PM, Willy Tarreau wrote:
> >>     - watch out for ACK compression effects (turn it off in favor of ABC
> >> > if you can)
> > It does not happen that much with HTTP. Many connections on the server side
> > see only one, sometimes two requests, and most responses are small (about
> > 20kB on average, with favicon fitting in a single segment). Note, I'm talking
> > about observations on average web sites.
> The effect is more pronounced for smaller responses, for any response
> using an odd number of packets. The last odd packet will be stalled
> because the client needs to timeout before it will send an ACK for a
> single segment (it's waiting for the second segment).

With short keep-alive responses yes but not short-lived connections
since the server sends the FIN immediately afterwards, which hints
the client not to wait for anything else and to ACK immediately.

> It doesn't matter how many requests you have, but the impact can be
> complicated on persistent connections.

Yes I agree. Please note that usually the last segment of a message
carries a push which precisely speeds up delivery to the application
and acking (as most OSes ack on push but apparently not all), so it
is not often emphasized that much. But I've met some 3G networks
where ACK compression was causing excessive retransmits from the
sender, resulting in excess retransmits of ACKs in turn, maintaining
the ACK link congested. It generally gets worse with many parallel
connections than with a single one, and in this regard HTTP/2 has
improved things a lot.

Willy


From nobody Thu Aug 18 14:53:26 2016
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3E82B12D5A5 for ; Thu, 18 Aug 2016 14:53:25 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -3.147
X-Spam-Level: 
X-Spam-Status: No, score=-3.147 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RP_MATCHES_RCVD=-1.247] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2_dVODsDlciw for ; Thu, 18 Aug 2016 14:53:23 -0700 (PDT)
Received: from nitro.isi.edu (nitro.isi.edu [128.9.208.207]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 2BCFB12D0B6 for ; Thu, 18 Aug 2016 14:53:23 -0700 (PDT)
Received: from [128.9.184.42] ([128.9.184.42]) (authenticated bits=0) by nitro.isi.edu (8.13.8/8.13.8) with ESMTP id u7ILqX0N018654 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Thu, 18 Aug 2016 14:52:34 -0700 (PDT)
To: Willy Tarreau 
References: <20160817064545.GD16017@1wt.eu> <7f7b129c-f156-d067-bef8-4a2213f461ac@isi.edu> <20160817180802.GA16773@1wt.eu> <4ab7c5b0-3722-1346-f481-a8d76de70034@isi.edu> <20160817211317.GA16929@1wt.eu>   <27b58b64-48cd-39af-78b3-ef583c585fa6@isi.edu> <20160818053837.GC16773@1wt.eu> <3c93805c-05f5-2b69-235e-e1c68363bba6@isi.edu> <20160818195948.GD17944@1wt.eu>
From: Joe Touch 
Message-ID: 
Date: Thu, 18 Aug 2016 14:52:31 -0700
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0
MIME-Version: 1.0
In-Reply-To: <20160818195948.GD17944@1wt.eu>
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: 7bit
X-MailScanner-ID: u7ILqX0N018654
X-ISI-4-69-MailScanner: Found to be clean
X-MailScanner-From: touch@isi.edu
Archived-At: 
Cc: tcpm@ietf.org, Matthew Kerwin , Daniel Stenberg , Mark Nottingham , HTTP Working Group , Patrick McManus 
Subject: Re: [tcpm] TCP Tuning for HTTP - update
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Thu, 18 Aug 2016 21:53:25 -0000

On 8/18/2016 12:59 PM, Willy Tarreau wrote:
> On Thu, Aug 18, 2016 at 10:50:52AM -0700, Joe Touch wrote:
>> Hi, Willy,
>>
>> On the issue of ACK compression effects:
>>
>>
>> On 8/17/2016 10:38 PM, Willy Tarreau wrote:
>>>>     - watch out for ACK compression effects (turn it off in favor of ABC
>>>>> if you can)
>>> It does not happen that much with HTTP. Many connections on the server side
>>> see only one, sometimes two requests, and most responses are small (about
>>> 20kB on average, with favicon fitting in a single segment). Note, I'm talking
>>> about observations on average web sites.
>> The effect is more pronounced for smaller responses, for any response
>> using an odd number of packets. The last odd packet will be stalled
>> because the client needs to timeout before it will send an ACK for a
>> single segment (it's waiting for the second segment).
> With short keep-alive responses yes but not short-lived connections
> since the server sends the FIN immediately afterwards, which hints
> the client not to wait for anything else and to ACK immediately.
If you have connection-per-transfer, but not if you have persistent
connections (the server wouldn't issue a close()).

>> It doesn't matter how many requests you have, but the impact can be
>> complicated on persistent connections.
> Yes I agree. Please note that usually the last segment of a message
> carries a push which precisely speeds up delivery to the application
> and acking (as most OSes ack on push but apparently not all), so it
> is not often emphasized that much.
PSH is optional.

The PSH bit forces data not to be aggregated by the sending TCP (i.e.,
to gather multiple send() calls) and forces the receiver to do the same,
but I don't see anywhere that PSH forces ACK compression to be
circumvented. If you have a pointer that'd be useful (I'm speaking of an
RFC). I looked for "delayed ACK" (the term used in 4.2.3.2 of RFC 1122)
and could not find any indication of a relation to PSH there, in fact
the delayed ACK discussion has no exceptions at all.

Additionally, that were true, setting the PSH bit constantly could cause
TCPs to open their slow-start windows much faster (2x per RTT, rather
than 1.5x as currently).

> But I've met some 3G networks
> where ACK compression was causing excessive retransmits from the
> sender, resulting in excess retransmits of ACKs in turn, maintaining
> the ACK link congested. It generally gets worse with many parallel
> connections than with a single one, and in this regard HTTP/2 has
> improved things a lot.

Most TCP variant assume that loss=congestion; that's often the wrong
interpretation for wireless.

(exceptions include TCP variants tuned for wireless)

Joe


From nobody Thu Aug 18 15:06:07 2016
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9DE6712D729 for ; Thu, 18 Aug 2016 15:06:06 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.902
X-Spam-Level: 
X-Spam-Status: No, score=-1.902 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id NEBbDgHY2fy0 for ; Thu, 18 Aug 2016 15:06:04 -0700 (PDT)
Received: from 1wt.eu (wtarreau.pck.nerim.net [62.212.114.60]) by ietfa.amsl.com (Postfix) with ESMTP id 5088E12D689 for ; Thu, 18 Aug 2016 15:06:04 -0700 (PDT)
Received: (from willy@localhost) by pcw.home.local (8.15.2/8.15.2/Submit) id u7IM5vYE018100; Fri, 19 Aug 2016 00:05:57 +0200
Date: Fri, 19 Aug 2016 00:05:57 +0200
From: Willy Tarreau 
To: Joe Touch 
Message-ID: <20160818220557.GA18091@1wt.eu>
References: <20160817180802.GA16773@1wt.eu> <4ab7c5b0-3722-1346-f481-a8d76de70034@isi.edu> <20160817211317.GA16929@1wt.eu>   <27b58b64-48cd-39af-78b3-ef583c585fa6@isi.edu> <20160818053837.GC16773@1wt.eu> <3c93805c-05f5-2b69-235e-e1c68363bba6@isi.edu> <20160818195948.GD17944@1wt.eu> 
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: 
User-Agent: Mutt/1.6.0 (2016-04-01)
Archived-At: 
Cc: tcpm@ietf.org, Matthew Kerwin , Daniel Stenberg , Mark Nottingham , HTTP Working Group , Patrick McManus 
Subject: Re: [tcpm] TCP Tuning for HTTP - update
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Thu, 18 Aug 2016 22:06:06 -0000

On Thu, Aug 18, 2016 at 02:52:31PM -0700, Joe Touch wrote:
> The PSH bit forces data not to be aggregated by the sending TCP (i.e.,
> to gather multiple send() calls) and forces the receiver to do the same,
> but I don't see anywhere that PSH forces ACK compression to be
> circumvented.

No that's not what I was saying, just that it was often avoiding the
extra 200ms delay before sending the ACK for the odd segment number
which further amplifies the ACK compression issue.

> If you have a pointer that'd be useful (I'm speaking of an
> RFC). I looked for "delayed ACK" (the term used in 4.2.3.2 of RFC 1122)
> and could not find any indication of a relation to PSH there, in fact
> the delayed ACK discussion has no exceptions at all.

On some systems you can force to have delayed ACKs even on PSH, I do it
on linux (disable quick-ack). That's very useful to merge the response
with the ACK for the request. But these are tricky must not be abused.

> Additionally, that were true, setting the PSH bit constantly could cause
> TCPs to open their slow-start windows much faster (2x per RTT, rather
> than 1.5x as currently).

I wasn't aware.

> > But I've met some 3G networks
> > where ACK compression was causing excessive retransmits from the
> > sender, resulting in excess retransmits of ACKs in turn, maintaining
> > the ACK link congested. It generally gets worse with many parallel
> > connections than with a single one, and in this regard HTTP/2 has
> > improved things a lot.
> 
> Most TCP variant assume that loss=congestion; that's often the wrong
> interpretation for wireless.
> (exceptions include TCP variants tuned for wireless)

Yep definitely. And it's even worse with deep buffers at the intersection
of fast and slow networks like the RAN for some mobile operators because
the stacks here tend to be tuned to assume that a loss is a loss and cause
fast retransmit which further fill the RAN's buffers. Sometimes the uplink
is filled with ACKs and I even found it could be efficient to kill one every
two ACKs in such a case. But I think this asymmetry is progressively going
away in favor of more balanced links.

Willy


From nobody Thu Aug 18 16:03:55 2016
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id EAAF512D79D for ; Thu, 18 Aug 2016 16:03:51 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -3.147
X-Spam-Level: 
X-Spam-Status: No, score=-3.147 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RP_MATCHES_RCVD=-1.247] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Qz4VINYQages for ; Thu, 18 Aug 2016 16:03:50 -0700 (PDT)
Received: from nitro.isi.edu (nitro.isi.edu [128.9.208.207]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id C6D3F12D18E for ; Thu, 18 Aug 2016 16:03:50 -0700 (PDT)
Received: from [128.9.184.42] ([128.9.184.42]) (authenticated bits=0) by nitro.isi.edu (8.13.8/8.13.8) with ESMTP id u7IN3BkG029348 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Thu, 18 Aug 2016 16:03:11 -0700 (PDT)
To: Willy Tarreau 
References: <20160817180802.GA16773@1wt.eu> <4ab7c5b0-3722-1346-f481-a8d76de70034@isi.edu> <20160817211317.GA16929@1wt.eu>   <27b58b64-48cd-39af-78b3-ef583c585fa6@isi.edu> <20160818053837.GC16773@1wt.eu> <3c93805c-05f5-2b69-235e-e1c68363bba6@isi.edu> <20160818195948.GD17944@1wt.eu>  <20160818220557.GA18091@1wt.eu>
From: Joe Touch 
Message-ID: <25c09f11-a07d-1530-7965-522015c196b7@isi.edu>
Date: Thu, 18 Aug 2016 16:03:08 -0700
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0
MIME-Version: 1.0
In-Reply-To: <20160818220557.GA18091@1wt.eu>
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: 7bit
X-MailScanner-ID: u7IN3BkG029348
X-ISI-4-69-MailScanner: Found to be clean
X-MailScanner-From: touch@isi.edu
Archived-At: 
Cc: tcpm@ietf.org, Matthew Kerwin , Daniel Stenberg , Mark Nottingham , HTTP Working Group , Patrick McManus 
Subject: Re: [tcpm] TCP Tuning for HTTP - update
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Thu, 18 Aug 2016 23:03:52 -0000

Hi, Willy,

Yes - this is yet another example of TCP's "twisty maze of passages, all
alike" ;-)

Joe


On 8/18/2016 3:05 PM, Willy Tarreau wrote:
> On Thu, Aug 18, 2016 at 02:52:31PM -0700, Joe Touch wrote:
>> The PSH bit forces data not to be aggregated by the sending TCP (i.e.,
>> to gather multiple send() calls) and forces the receiver to do the same,
>> but I don't see anywhere that PSH forces ACK compression to be
>> circumvented.
> No that's not what I was saying, just that it was often avoiding the
> extra 200ms delay before sending the ACK for the odd segment number
> which further amplifies the ACK compression issue.
>
>> If you have a pointer that'd be useful (I'm speaking of an
>> RFC). I looked for "delayed ACK" (the term used in 4.2.3.2 of RFC 1122)
>> and could not find any indication of a relation to PSH there, in fact
>> the delayed ACK discussion has no exceptions at all.
> On some systems you can force to have delayed ACKs even on PSH, I do it
> on linux (disable quick-ack). That's very useful to merge the response
> with the ACK for the request. But these are tricky must not be abused.
>
>> Additionally, that were true, setting the PSH bit constantly could cause
>> TCPs to open their slow-start windows much faster (2x per RTT, rather
>> than 1.5x as currently).
> I wasn't aware.
>
>>> But I've met some 3G networks
>>> where ACK compression was causing excessive retransmits from the
>>> sender, resulting in excess retransmits of ACKs in turn, maintaining
>>> the ACK link congested. It generally gets worse with many parallel
>>> connections than with a single one, and in this regard HTTP/2 has
>>> improved things a lot.
>> Most TCP variant assume that loss=congestion; that's often the wrong
>> interpretation for wireless.
>> (exceptions include TCP variants tuned for wireless)
> Yep definitely. And it's even worse with deep buffers at the intersection
> of fast and slow networks like the RAN for some mobile operators because
> the stacks here tend to be tuned to assume that a loss is a loss and cause
> fast retransmit which further fill the RAN's buffers. Sometimes the uplink
> is filled with ACKs and I even found it could be efficient to kill one every
> two ACKs in such a case. But I think this asymmetry is progressively going
> away in favor of more balanced links.
>
> Willy


From nobody Thu Aug 18 17:04:49 2016
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5357612D0A8 for ; Thu, 18 Aug 2016 17:04:47 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -3.947
X-Spam-Level: 
X-Spam-Status: No, score=-3.947 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, RP_MATCHES_RCVD=-1.247, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=google.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id T0DXAVTidiOf for ; Thu, 18 Aug 2016 17:04:45 -0700 (PDT)
Received: from mail-it0-x22b.google.com (mail-it0-x22b.google.com [IPv6:2607:f8b0:4001:c0b::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 473B612B02A for ; Thu, 18 Aug 2016 17:04:45 -0700 (PDT)
Received: by mail-it0-x22b.google.com with SMTP id f6so11996456ith.0 for ; Thu, 18 Aug 2016 17:04:45 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=6I1gPX/3MdCWxpe9DcbTVG1eDtF0huZVGxE8mN0dNI4=; b=YvT+O/tzmnekFgqn2eIw/1jHydUdMK2QjyCuC8n88wKs+sf3uvBHAfxeeiFRs8gR7K SHeE0dzqcSj/FqYIHWQ2CVolejYaQOdr2GSjjnuzTx0ohI6K8oz669l5ycOnfpZnkVCy pod7f1aS5ZiFKJESEhsKJAx+Ra5lXs8GeKq8vV+HxqzkOwE+vTZuP9DVPKdgb4DUz2UP nWGHWaNKOmf6Qmrlu/5bfVzqLl8DWVj/kQQZyh0Cn5y/gGatb54ipZtn8xMFhbcXOun8 WetjBx14xftHNIRE9uGlzlBnCeSdu1T9gCn5HnlTNI+xHuTUVmBisgnodqoRq6hgQLXA uXMg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=6I1gPX/3MdCWxpe9DcbTVG1eDtF0huZVGxE8mN0dNI4=; b=j7sd4zF5iKzylLJqLExaqTkqCQl4CbuxWclpH694CARIUBRV27NdcuRa4q8WGalu+C 4w5UJtQbSC6z+OH+J4mH2AICnPFOieNVbJJN9rwHPVW1BHykWFcvGwJbmL5Br7YsOP18 EFBEp7c/EDwc+liGfEozNUL44tM1rPmKiD3YXtBZ0jDnOz71Xd+UovBc3z1a1nqE+Kdj 4nbqbYwrJ/US/5QAe67snpDYSTIT5lVx+bH9ubsKgyux4YvIbMunz6h8r4KIJm5JYdSu rblmUsa21FBB+tt+7ijCuTVAE4CY20eiUUl18GxgLaR5V+pUXBoGMRo9SUm8SB4exUxA 2GqQ==
X-Gm-Message-State: AEkooutSYGrg0jU1PTvqqPr03hU5ozCoWC3Rr9+mguXvhhbiWIlQ5fFfDvxfqdx1QLcxJGIyiI6QQUJl6qaycPMh
X-Received: by 10.36.150.70 with SMTP id z67mr2581936itd.80.1471565083417; Thu, 18 Aug 2016 17:04:43 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.64.244.103 with HTTP; Thu, 18 Aug 2016 17:04:03 -0700 (PDT)
In-Reply-To: <57B482B9.5090205@kuehlewind.net>
References: <57B482B9.5090205@kuehlewind.net>
From: Yuchung Cheng 
Date: Thu, 18 Aug 2016 17:04:03 -0700
Message-ID: 
To: =?UTF-8?Q?Mirja_K=C3=BChlewind?= 
Content-Type: multipart/alternative; boundary=94eb2c07d07410062e053a617222
Archived-At: 
Cc: "tcpm@ietf.org Extensions" , "tcpm-chairs@ietf.org" 
Subject: Re: [tcpm] Personal change in chairing
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Fri, 19 Aug 2016 00:04:47 -0000

--94eb2c07d07410062e053a617222
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

On Wed, Aug 17, 2016 at 8:28 AM, Mirja K=C3=BChlewind 
wrote:

> Hi all,
>
> I have to announce a personal change in the tcpm chair position:
>
> Pasi Sarolahti is stepping down due to time restrictions. Pasi, thanks a
> lot for your work and engagement as chair! Your did a great job and we ho=
pe
> to still see you as often as possible as contributor in the working group=
!
>
Thanks for your service Pasi


>
> Michael T=C3=BCxen is starting as a new tcpm chair. Micheal, great to hav=
e you
> on broad! We look forward to work with you!
>
> Mirja, as responsible AD
>
> _______________________________________________
> tcpm mailing list
> tcpm@ietf.org
> https://www.ietf.org/mailman/listinfo/tcpm
>

--94eb2c07d07410062e053a617222
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable




On Wed, Aug 17, 2016 at 8:28 AM, Mirja K=C3=BChlewind <ietf@kuehlewi=
nd.net> wrote:
Hi all,



I have to announce a personal change in the tcpm chair position:



Pasi Sarolahti is stepping down due to time restrictions. Pasi, thanks a lo=
t for your work and engagement as chair! Your did a great job and we hope t=
o still see you as often as possible as contributor in the working group!
Thanks for your service Pasi
=C2=A0



Michael T=C3=BCxen is starting as a new tcpm chair. Micheal, great to have =
you on broad! We look forward to work with you!



Mirja, as responsible AD



_______________________________________________

tcpm mailing list

tcpm@ietf.org

https://www.ietf.org/mailman/listinfo/tcpm





--94eb2c07d07410062e053a617222--


From nobody Thu Aug 18 20:49:38 2016
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0145012D593 for ; Thu, 18 Aug 2016 20:49:36 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.602
X-Spam-Level: 
X-Spam-Status: No, score=-2.602 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TC4a6R62B3Of for ; Thu, 18 Aug 2016 20:49:33 -0700 (PDT)
Received: from mxout-07.mxes.net (mxout-07.mxes.net [216.86.168.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 667A612B041 for ; Thu, 18 Aug 2016 20:49:33 -0700 (PDT)
Received: from [192.168.3.104] (unknown [124.189.98.244]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by smtp.mxes.net (Postfix) with ESMTPSA id 98DD022E259; Thu, 18 Aug 2016 23:49:24 -0400 (EDT)
Content-Type: text/plain; charset=windows-1252
Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\))
From: Mark Nottingham 
In-Reply-To: <7a36e025-4882-4f8b-7a83-9fdcd990a971@isi.edu>
Date: Fri, 19 Aug 2016 13:49:21 +1000
Content-Transfer-Encoding: quoted-printable
Message-Id: 
References: <5CD67877-19E3-4E79-BBF2-3E270343A378@mnot.net> <2197232f-10d7-28cb-fcc9-05bd495e3c22@isi.edu> <20160817064545.GD16017@1wt.eu> <7f7b129c-f156-d067-bef8-4a2213f461ac@isi.edu> <20160817180802.GA16773@1wt.eu> <4ab7c5b0-3722-1346-f481-a8d76de70034@isi.edu> <20160817211317.GA16929@1wt.eu>   <27b58b64-48cd-39af-78b3-ef583c585fa6@isi.edu> <20160818053837.GC16773@1wt.eu> <7a36e025-4882-4f8b-7a83-9fdcd990a971@isi.edu>
To: Joe Touch 
X-Mailer: Apple Mail (2.3124)
Archived-At: 
Cc: tcpm@ietf.org, Matthew Kerwin , Daniel Stenberg , Patrick McManus , Willy Tarreau , HTTP Working Group 
Subject: Re: [tcpm] TCP Tuning for HTTP - update
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Fri, 19 Aug 2016 03:49:36 -0000

> On 19 Aug 2016, at 1:07 AM, Joe Touch  wrote:
>=20
> I do think that this doc needs to figure out whom it is speaking to, =
what advice they actually need, etc.
>=20
> If the result is a set of recommendations that involve the word =
"sysctl", I remain skeptical it is appropriate as an RFC


I think there's broad agreement on both of these points.

I'm wondering if it makes sense to aim it primarily at HTTP implementers =
rather than administrators, with the notion that it would inform:

- Their implementation decisions
- The configuration choices they offer to administrators / users
- Their documentation (e.g., advice to their administrators when the =
implementation can't change the appropriate parts of the OS)

Would that help?=20

If so, it might make sense to organise it into sections for clients and =
servers (and intermediaries, if there's anything that isn't covered by =
the combination of the first two). Although IIRC Daniel was already =
talking about doing that.

Cheers,

--
Mark Nottingham   https://www.mnot.net/


From nobody Fri Aug 19 00:48:02 2016
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 43D4112DA40 for ; Fri, 19 Aug 2016 00:47:59 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -3.147
X-Spam-Level: 
X-Spam-Status: No, score=-3.147 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RP_MATCHES_RCVD=-1.247] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Vdh8PWjY8Wui for ; Fri, 19 Aug 2016 00:47:55 -0700 (PDT)
Received: from pegasus.erg.abdn.ac.uk (pegasus.erg.abdn.ac.uk [IPv6:2001:630:241:204::f0f0]) by ietfa.amsl.com (Postfix) with ESMTP id 8E34512D8AE for ; Fri, 19 Aug 2016 00:47:55 -0700 (PDT)
Received: from Gs-MacBook-Pro.local (fgrpf.plus.com [212.159.18.54]) by pegasus.erg.abdn.ac.uk (Postfix) with ESMTPA id E1F111B00269; Fri, 19 Aug 2016 08:47:37 +0100 (BST)
Message-ID: <57B6B9A0.10402@erg.abdn.ac.uk>
Date: Fri, 19 Aug 2016 08:47:44 +0100
From: Gorry Fairhurst 
Organization: University of Aberdeen
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:12.0) Gecko/20120428 Thunderbird/12.0.1
MIME-Version: 1.0
To: Yuchung Cheng 
References: <0CC24FC1-37E1-4125-9627-05726A9D9406@mnot.net> <655C07320163294895BBADA28372AF5D489E64AF@FR712WXCHMBA15.zeu.alcatel-lucent.com> <61fa7faa-d2a3-e8f3-9497-1f0f93e7b81e@erg.abdn.ac.uk> 
In-Reply-To: 
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Archived-At: 
Cc: "tcpm@ietf.org" , Mark Nottingham , Patrick McManus , Daniel Stenberg 
Subject: Re: [tcpm] TCP Tuning for HTTP - update
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
Reply-To: gorry@erg.abdn.ac.uk
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Fri, 19 Aug 2016 07:47:59 -0000

On 18/08/2016 18:34, Yuchung Cheng wrote:
>
>
> On Thu, Aug 18, 2016 at 9:54 AM, Gorry Fairhurst  > wrote:
>
>     Speaking as someone who commented on an earlier version:
>
>     I think it is good to offer advice on how TCP is used for web
>     traffic. I can see there is likely useful information that can
>     document how some things are done, and perhaps also recommend some
>     changes. I support continuing work on this.
>
>     Publishing a document like this in the RFC series needs to be
>     consistent with IETF consensus. The ID touches on many things that
>     are assigned to TCPM. (The breadth of topics may be a good reason
>     for a document - but is also a concern, because many things need
>     more clear description before the recommendations become clear).
>
>     I think this document should NOT be considered as BCP. It talks
>     about system configuration of endpoints, and of the
>     recommendations I see, some do support current PSs, some set new
>     precedents against current PS RFCs and in some cases the present
>     text appears to endorse EXP RFCs. If published as a BCP, that sort
>     of advice is likely to be confusing. Such information can also
>     easily be outdated, I suggest if adopted, this should be
>     Informational and clearly set the scope of the recommendations.
>
> +1 on informational
>
> It'd be great for the document to cite more detailed technical 
> reports, for readers to learn the pros and cons, and the history. IMO 
> it'd significantly improve this document. The reason is that TCP and 
> network will continue to evolve, so knowing the past and details can 
> help readers to judge whether a specific recommendation is still 
> useful or not, if he is willing to spend time to explore. For example, 
> he can use some online tools to track the citation of these reports 
> and see new reports.
>
I agree
> For the document, I'd recommend two more items that my personal 
> experience gives remarkable performance enhancement on reducing and 
> recover from losses.
>
> 1) make sure your stack supports SACK
> 2) enable pacing.
>
+1

> On disabling Nagle, there are also better APIs like Linux Cork so you 
> don't have to send small packets even on continuous writes. One common 
> problem I often see is developers unconditionally disable Nagle, but 
> their HTTP workload may lead to lots of small packets.
>
Worth metioning.
> my 2c
>
>
>     If this work continues, it needs to be reviewed by TCPM and
>     HTTPbis. This would need much more than simply a last-call in both
>     working groups - but I'm sure with proper coordination progress
>     could be made so that there were no surprises for either group at
>     the time of WGLC!
>
>     Gorry
>
>
>
>
>         -----Original Message-----
>         From: tcpm [mailto:tcpm-bounces@ietf.org
>         ] On Behalf Of Mark Nottingham
>         Sent: Wednesday, August 17, 2016 4:03 AM
>         To: tcpm@ietf.org 
>         Cc: Patrick McManus; Daniel Stenberg
>         Subject: [tcpm] TCP Tuning for HTTP - update
>
>         Hi TCPM,
>
>         Just a quick note; Daniel and Tim have made an update to the
>         TCP Tuning for HTTP draft:
>         https://tools.ietf.org/html/draft-stenberg-httpbis-tcp
>         
>
>         We've had a Call for Adoption open for this draft for a while,
>         and will likely adopt it soon. However, we'd like to get
>         feedback from this community first -- both about the latest
>         version of the input document, and to see if there's interest
>         in helping out.
>
>         You can give feedback on the HTTP WG mailing list
>                  >, or  by
>         responding to this e-mail (Please leave the CC line; Patrick
>         and I will try to summarise the feedback to the WG).
>
>         Cheers,
>
>         --
>         Mark Nottingham https://www.mnot.net/
>
>
>
>
>         _______________________________________________
>         tcpm mailing list
>         tcpm@ietf.org 
>         https://www.ietf.org/mailman/listinfo/tcpm
>         
>
>         _______________________________________________
>         tcpm mailing list
>         tcpm@ietf.org 
>         https://www.ietf.org/mailman/listinfo/tcpm
>         
>
>
>     _______________________________________________
>     tcpm mailing list
>     tcpm@ietf.org 
>     https://www.ietf.org/mailman/listinfo/tcpm
>     
>
>


From nobody Fri Aug 19 01:13:56 2016
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 900B112D76A for ; Fri, 19 Aug 2016 01:13:54 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -3.148
X-Spam-Level: 
X-Spam-Status: No, score=-3.148 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, RP_MATCHES_RCVD=-1.247, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mBKAAvxjZ-zO for ; Fri, 19 Aug 2016 01:13:51 -0700 (PDT)
Received: from mx1.cnes.fr (mx1.cnes.fr [194.199.174.200]) by ietfa.amsl.com (Postfix) with ESMTP id C889C12D75A for ; Fri, 19 Aug 2016 01:13:50 -0700 (PDT)
X-IronPort-AV: E=Sophos;i="5.28,544,1464652800";  d="scan'208";a="5351853"
X-IPAS-Result: A2EPAwC1vrZX/wEBeApeDgsBAQEBAYMmSQ18B40mqj6BdgcZC4V5AoFiOBQBAQEBAQEBAQEDWyeEXgEBAQECAQEBARpRCwUHBAIBBQMNBAQBAQEKHQcnAQoUCQgCBAENBQgMiBUIDr0WAQEBAQEBAQEBAQEBAQEBAQEBAQEBFwWJdYEDhBwmgyqCLwWOWIpvXYVDhUSEIIEFhFyDJoVfhmiFVYN4HjaDPzs8NIVoRgF+AQEB
From: Kuhn Nicolas 
To: "gorry@erg.abdn.ac.uk" , Yuchung Cheng 
Thread-Topic: [tcpm] TCP Tuning for HTTP - update
Thread-Index: AQHR+CukFeI6FpjPD0CnMDLA8byFbqBNXOuAgAFzqwCAAAr1gIAA7ocAgAAihNA=
Date: Fri, 19 Aug 2016 08:13:35 +0000
Deferred-Delivery: Fri, 19 Aug 2016 08:13:00 +0000
Message-ID: 
References: <0CC24FC1-37E1-4125-9627-05726A9D9406@mnot.net> <655C07320163294895BBADA28372AF5D489E64AF@FR712WXCHMBA15.zeu.alcatel-lucent.com> <61fa7faa-d2a3-e8f3-9497-1f0f93e7b81e@erg.abdn.ac.uk>  <57B6B9A0.10402@erg.abdn.ac.uk>
In-Reply-To: <57B6B9A0.10402@erg.abdn.ac.uk>
Accept-Language: en-US
Content-Language: fr-FR
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-tm-as-product-ver: SMEX-11.0.0.4255-8.000.1202-22522.006
x-tm-as-result: No--62.458500-8.000000-31
x-tm-as-user-approved-sender: No
x-tm-as-user-blocked-sender: No
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
Archived-At: 
Cc: Mark Nottingham , "tcpm@ietf.org" , Patrick McManus , Daniel Stenberg 
Subject: Re: [tcpm] TCP Tuning for HTTP - update
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Fri, 19 Aug 2016 08:13:54 -0000

Hi all,=20

I agree with the recommendation on enabling pacing.
If flows are multiplexed such as in HTTP2, losses at slow start can be very=
 damaging.=20
I have proposed some text inline.=20

Cheers,

Nico

-----Message d'origine-----
De=A0: tcpm [mailto:tcpm-bounces@ietf.org] De la part de Gorry Fairhurst
Envoy=E9=A0: vendredi 19 ao=FBt 2016 09:48
=C0=A0: Yuchung Cheng
Cc=A0: tcpm@ietf.org; Mark Nottingham; Patrick McManus; Daniel Stenberg
Objet=A0: Re: [tcpm] TCP Tuning for HTTP - update

On 18/08/2016 18:34, Yuchung Cheng wrote:
>
>
> On Thu, Aug 18, 2016 at 9:54 AM, Gorry Fairhurst  > wrote:
>
>     Speaking as someone who commented on an earlier version:
>
>     I think it is good to offer advice on how TCP is used for web
>     traffic. I can see there is likely useful information that can
>     document how some things are done, and perhaps also recommend some
>     changes. I support continuing work on this.
>
>     Publishing a document like this in the RFC series needs to be
>     consistent with IETF consensus. The ID touches on many things that
>     are assigned to TCPM. (The breadth of topics may be a good reason
>     for a document - but is also a concern, because many things need
>     more clear description before the recommendations become clear).
>
>     I think this document should NOT be considered as BCP. It talks
>     about system configuration of endpoints, and of the
>     recommendations I see, some do support current PSs, some set new
>     precedents against current PS RFCs and in some cases the present
>     text appears to endorse EXP RFCs. If published as a BCP, that sort
>     of advice is likely to be confusing. Such information can also
>     easily be outdated, I suggest if adopted, this should be
>     Informational and clearly set the scope of the recommendations.
>
> +1 on informational
>
> It'd be great for the document to cite more detailed technical=20
> reports, for readers to learn the pros and cons, and the history. IMO=20
> it'd significantly improve this document. The reason is that TCP and=20
> network will continue to evolve, so knowing the past and details can=20
> help readers to judge whether a specific recommendation is still=20
> useful or not, if he is willing to spend time to explore. For example,=20
> he can use some online tools to track the citation of these reports=20
> and see new reports.
>
I agree
> For the document, I'd recommend two more items that my personal=20
> experience gives remarkable performance enhancement on reducing and=20
> recover from losses.
>
> 1) make sure your stack supports SACK
> 2) enable pacing.
>
+1

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
[NK]
Regarding the point on enabling pacing, I would suggest text changes for th=
e section "3.2.  Initial Window".

From
"
   There has been experimentation with larger initial windows in
   combination with packet pacing, however IW10 has been reported to
   perform fairly well even in both general and high volume use cases.
"

To=20
"
  IW10 has been reported to
  perform fairly well even in both general and high volume use cases.=20
  There has been experimentation with larger initial windows in
  combination with packet pacing. Packet pacing at slow start
  reduces congestion losses at the beginning of the connection.
  Moreover, losses at the beginning of the connection result in retransmiss=
ion=20
  delay and an early slow start exit. In this case, degraded performances w=
hen flows are=20
  multiplexed such as in HTTP2 can be expected.  =20
  Packet pacing at slow start also provides clear benefits for challenged n=
etworks,=20
  such as capacity-limited or long-RTT networks.=20
"
Then, there could be text mentioning that "enabling packet pacing is recomm=
ended".
Or something similar.=20
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D

> On disabling Nagle, there are also better APIs like Linux Cork so you=20
> don't have to send small packets even on continuous writes. One common=20
> problem I often see is developers unconditionally disable Nagle, but=20
> their HTTP workload may lead to lots of small packets.
>
Worth metioning.
> my 2c
>
>
>     If this work continues, it needs to be reviewed by TCPM and
>     HTTPbis. This would need much more than simply a last-call in both
>     working groups - but I'm sure with proper coordination progress
>     could be made so that there were no surprises for either group at
>     the time of WGLC!
>
>     Gorry
>
>
>
>
>         -----Original Message-----
>         From: tcpm [mailto:tcpm-bounces@ietf.org
>         ] On Behalf Of Mark Nottingham
>         Sent: Wednesday, August 17, 2016 4:03 AM
>         To: tcpm@ietf.org 
>         Cc: Patrick McManus; Daniel Stenberg
>         Subject: [tcpm] TCP Tuning for HTTP - update
>
>         Hi TCPM,
>
>         Just a quick note; Daniel and Tim have made an update to the
>         TCP Tuning for HTTP draft:
>         https://tools.ietf.org/html/draft-stenberg-httpbis-tcp
>         
>
>         We've had a Call for Adoption open for this draft for a while,
>         and will likely adopt it soon. However, we'd like to get
>         feedback from this community first -- both about the latest
>         version of the input document, and to see if there's interest
>         in helping out.
>
>         You can give feedback on the HTTP WG mailing list
>                  >, or  by
>         responding to this e-mail (Please leave the CC line; Patrick
>         and I will try to summarise the feedback to the WG).
>
>         Cheers,
>
>         --
>         Mark Nottingham https://www.mnot.net/
>
>
>
>
>         _______________________________________________
>         tcpm mailing list
>         tcpm@ietf.org 
>         https://www.ietf.org/mailman/listinfo/tcpm
>         
>
>         _______________________________________________
>         tcpm mailing list
>         tcpm@ietf.org 
>         https://www.ietf.org/mailman/listinfo/tcpm
>         
>
>
>     _______________________________________________
>     tcpm mailing list
>     tcpm@ietf.org 
>     https://www.ietf.org/mailman/listinfo/tcpm
>     
>
>

_______________________________________________
tcpm mailing list
tcpm@ietf.org
https://www.ietf.org/mailman/listinfo/tcpm


From nobody Fri Aug 19 05:17:57 2016
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6393112D963 for ; Fri, 19 Aug 2016 05:17:56 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -6.92
X-Spam-Level: 
X-Spam-Status: No, score=-6.92 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id q7JWkHeqi760 for ; Fri, 19 Aug 2016 05:17:53 -0700 (PDT)
Received: from thsbbfxrt01p.thalesgroup.com (thsbbfxrt01p.thalesgroup.com [192.54.144.131]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 7822212D950 for ; Fri, 19 Aug 2016 05:17:52 -0700 (PDT)
Received: from thsbbfxrt01p.thalesgroup.com (localhost [127.0.0.1]) by localhost (Postfix) with SMTP id 3sG26Q0gWfz1VR; Fri, 19 Aug 2016 14:17:50 +0200 (CEST)
From: SALLANTIN Renaud 
To: 'Kuhn Nicolas' , "'gorry@erg.abdn.ac.uk'" , 'Yuchung Cheng' 
Date: Fri, 19 Aug 2016 14:17:51 +0200
Thread-Topic: [tcpm] TCP Tuning for HTTP - update
Thread-Index: AQHR+CukFeI6FpjPD0CnMDLA8byFbqBNXOuAgAFzqwCAAAr1gIAA7ocAgAAihNCAAEI34A==
Message-ID: 
References: <0CC24FC1-37E1-4125-9627-05726A9D9406@mnot.net> <655C07320163294895BBADA28372AF5D489E64AF@FR712WXCHMBA15.zeu.alcatel-lucent.com> <61fa7faa-d2a3-e8f3-9497-1f0f93e7b81e@erg.abdn.ac.uk>  <57B6B9A0.10402@erg.abdn.ac.uk> 
In-Reply-To: 
Accept-Language: fr-FR
Content-Language: fr-FR
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
acceptlanguage: fr-FR
x-pmwin-version: 3.1.3.0, Antivirus-Engine: 3.64.3, Antivirus-Data: 5.30
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
Archived-At: 
Cc: 'Mark Nottingham' , "'tcpm@ietf.org'" , 'Patrick McManus' , 'Daniel Stenberg' 
Subject: Re: [tcpm] TCP Tuning for HTTP - update
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Fri, 19 Aug 2016 12:17:56 -0000

Hi all,=20

IMO, such a document could be very useful,=20
But it can also be dangerous.

For instance, the following statement seems to shut the door on other solut=
ions than IW10. Unfortunately, if IW10 improved TCP startup efficiency, it =
remains not efficient in case of large RTT networks (such as satellite syst=
ems for instance).

   There has been experimentation with larger initial windows in
   combination with packet pacing, however IW10 has been reported to
   perform fairly well even in both general and high volume use cases.=20

IMO, this doc must describe the best current practices, but also emphasize =
the remaining issues.

BR,=20
Renaud



-----Message d'origine-----
De=A0: tcpm [mailto:tcpm-bounces@ietf.org] De la part de Kuhn Nicolas
Envoy=E9=A0: vendredi 19 ao=FBt 2016 10:14
=C0=A0: gorry@erg.abdn.ac.uk; Yuchung Cheng
Cc=A0: Mark Nottingham; tcpm@ietf.org; Patrick McManus; Daniel Stenberg
Objet=A0: Re: [tcpm] TCP Tuning for HTTP - update

Hi all,=20

I agree with the recommendation on enabling pacing.
If flows are multiplexed such as in HTTP2, losses at slow start can be very=
 damaging.=20
I have proposed some text inline.=20

Cheers,

Nico

-----Message d'origine-----
De=A0: tcpm [mailto:tcpm-bounces@ietf.org] De la part de Gorry Fairhurst En=
voy=E9=A0: vendredi 19 ao=FBt 2016 09:48 =C0=A0: Yuchung Cheng Cc=A0: tcpm@=
ietf.org; Mark Nottingham; Patrick McManus; Daniel Stenberg Objet=A0: Re: [=
tcpm] TCP Tuning for HTTP - update

On 18/08/2016 18:34, Yuchung Cheng wrote:
>
>
> On Thu, Aug 18, 2016 at 9:54 AM, Gorry Fairhurst  > wrote:
>
>     Speaking as someone who commented on an earlier version:
>
>     I think it is good to offer advice on how TCP is used for web
>     traffic. I can see there is likely useful information that can
>     document how some things are done, and perhaps also recommend some
>     changes. I support continuing work on this.
>
>     Publishing a document like this in the RFC series needs to be
>     consistent with IETF consensus. The ID touches on many things that
>     are assigned to TCPM. (The breadth of topics may be a good reason
>     for a document - but is also a concern, because many things need
>     more clear description before the recommendations become clear).
>
>     I think this document should NOT be considered as BCP. It talks
>     about system configuration of endpoints, and of the
>     recommendations I see, some do support current PSs, some set new
>     precedents against current PS RFCs and in some cases the present
>     text appears to endorse EXP RFCs. If published as a BCP, that sort
>     of advice is likely to be confusing. Such information can also
>     easily be outdated, I suggest if adopted, this should be
>     Informational and clearly set the scope of the recommendations.
>
> +1 on informational
>
> It'd be great for the document to cite more detailed technical=20
> reports, for readers to learn the pros and cons, and the history. IMO=20
> it'd significantly improve this document. The reason is that TCP and=20
> network will continue to evolve, so knowing the past and details can=20
> help readers to judge whether a specific recommendation is still=20
> useful or not, if he is willing to spend time to explore. For example,=20
> he can use some online tools to track the citation of these reports=20
> and see new reports.
>
I agree
> For the document, I'd recommend two more items that my personal=20
> experience gives remarkable performance enhancement on reducing and=20
> recover from losses.
>
> 1) make sure your stack supports SACK
> 2) enable pacing.
>
+1

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
[NK]
Regarding the point on enabling pacing, I would suggest text changes for th=
e section "3.2.  Initial Window".

From
"
   There has been experimentation with larger initial windows in
   combination with packet pacing, however IW10 has been reported to
   perform fairly well even in both general and high volume use cases.
"

To
"
  IW10 has been reported to
  perform fairly well even in both general and high volume use cases.=20
  There has been experimentation with larger initial windows in
  combination with packet pacing. Packet pacing at slow start
  reduces congestion losses at the beginning of the connection.
  Moreover, losses at the beginning of the connection result in retransmiss=
ion
  delay and an early slow start exit. In this case, degraded performances w=
hen flows are=20
  multiplexed such as in HTTP2 can be expected.  =20
  Packet pacing at slow start also provides clear benefits for challenged n=
etworks,
  such as capacity-limited or long-RTT networks.=20
"
Then, there could be text mentioning that "enabling packet pacing is recomm=
ended".
Or something similar.=20
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D

> On disabling Nagle, there are also better APIs like Linux Cork so you=20
> don't have to send small packets even on continuous writes. One common=20
> problem I often see is developers unconditionally disable Nagle, but=20
> their HTTP workload may lead to lots of small packets.
>
Worth metioning.
> my 2c
>
>
>     If this work continues, it needs to be reviewed by TCPM and
>     HTTPbis. This would need much more than simply a last-call in both
>     working groups - but I'm sure with proper coordination progress
>     could be made so that there were no surprises for either group at
>     the time of WGLC!
>
>     Gorry
>
>
>
>
>         -----Original Message-----
>         From: tcpm [mailto:tcpm-bounces@ietf.org
>         ] On Behalf Of Mark Nottingham
>         Sent: Wednesday, August 17, 2016 4:03 AM
>         To: tcpm@ietf.org 
>         Cc: Patrick McManus; Daniel Stenberg
>         Subject: [tcpm] TCP Tuning for HTTP - update
>
>         Hi TCPM,
>
>         Just a quick note; Daniel and Tim have made an update to the
>         TCP Tuning for HTTP draft:
>         https://tools.ietf.org/html/draft-stenberg-httpbis-tcp
>         
>
>         We've had a Call for Adoption open for this draft for a while,
>         and will likely adopt it soon. However, we'd like to get
>         feedback from this community first -- both about the latest
>         version of the input document, and to see if there's interest
>         in helping out.
>
>         You can give feedback on the HTTP WG mailing list
>                  >, or  by
>         responding to this e-mail (Please leave the CC line; Patrick
>         and I will try to summarise the feedback to the WG).
>
>         Cheers,
>
>         --
>         Mark Nottingham https://www.mnot.net/
>
>
>
>
>         _______________________________________________
>         tcpm mailing list
>         tcpm@ietf.org 
>         https://www.ietf.org/mailman/listinfo/tcpm
>         
>
>         _______________________________________________
>         tcpm mailing list
>         tcpm@ietf.org 
>         https://www.ietf.org/mailman/listinfo/tcpm
>         
>
>
>     _______________________________________________
>     tcpm mailing list
>     tcpm@ietf.org 
>     https://www.ietf.org/mailman/listinfo/tcpm
>     
>
>

_______________________________________________
tcpm mailing list
tcpm@ietf.org
https://www.ietf.org/mailman/listinfo/tcpm

_______________________________________________
tcpm mailing list
tcpm@ietf.org
https://www.ietf.org/mailman/listinfo/tcpm


From nobody Fri Aug 19 06:44:57 2016
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id EC10512DA14 for ; Fri, 19 Aug 2016 06:44:55 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -5.447
X-Spam-Level: 
X-Spam-Status: No, score=-5.447 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RP_MATCHES_RCVD=-1.247] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id EDXswWHrHygn for ; Fri, 19 Aug 2016 06:44:53 -0700 (PDT)
Received: from smtp.ee.ethz.ch (smtp.ee.ethz.ch [129.132.2.219]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 11C1D12DA30 for ; Fri, 19 Aug 2016 06:43:41 -0700 (PDT)
Received: from localhost (localhost [127.0.0.1]) by smtp.ee.ethz.ch (Postfix) with ESMTP id AB863D9316; Fri, 19 Aug 2016 15:43:39 +0200 (MEST)
X-Virus-Scanned: by amavisd-new on smtp.ee.ethz.ch
Received: from smtp.ee.ethz.ch ([127.0.0.1]) by localhost (.ee.ethz.ch [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 1ubmbndKea-G; Fri, 19 Aug 2016 15:43:39 +0200 (MEST)
Received: from [192.168.178.33] (p5DEC255C.dip0.t-ipconnect.de [93.236.37.92]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: mirjak) by smtp.ee.ethz.ch (Postfix) with ESMTPSA id 5C508D9303; Fri, 19 Aug 2016 15:43:39 +0200 (MEST)
Content-Type: text/plain; charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\))
From: =?utf-8?Q?Mirja_K=C3=BChlewind?= 
In-Reply-To: 
Date: Fri, 19 Aug 2016 15:43:37 +0200
Content-Transfer-Encoding: quoted-printable
Message-Id: 
References: <20160810183654.05358B80C3A@rfc-editor.org>  <7ac89d58-fc3a-9e12-9d22-0602944f8677@isi.edu>     <57B31020.60506@tik.ee.ethz.ch> 
To: Yuchung Cheng 
X-Mailer: Apple Mail (2.3124)
Archived-At: 
Cc: "tcpm@ietf.org Extensions" , Joe Touch 
Subject: Re: [tcpm] [Technical Errata Reported] RFC5961 (4772)
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Fri, 19 Aug 2016 13:44:56 -0000

> sounds fine with me but do documents really get updated?
That depends on the working group. However the point here is, if the doc =
gets updated, then this is still listed.

Mirja


> Am 16.08.2016 um 19:42 schrieb Yuchung Cheng :
>=20
>=20
>=20
> On Tue, Aug 16, 2016 at 6:07 AM, Mirja K=C3=BChlewind =
 wrote:
> Hi all,
>=20
> let me try to wrap this up:
>=20
> To me this does not seem to be a case for an errata. For sure some =
clarification seems to be helpful here but that's not what erratas are =
for.
>=20
> Therefore, I would suggest to simply set this into "Held for Document =
Update" state such that it is noticed in case we decide to update the =
document at some point. Does that make sense?
> sounds fine with me but do documents really get updated?
> =20
>=20
> Thanks,
> Mirja
>=20
>=20
>=20
>=20
> On 12.08.2016 22:19, Joe Touch wrote:
>=20
>=20
> On 8/12/2016 12:17 PM, Neal Cardwell wrote:
> On Fri, Aug 12, 2016 at 9:54 AM, Yuchung Cheng  =
wrote:
> I think an errata on recommending per-connection limit and cite the
> research finding is worthy.
> I agree that it's worthwhile to have an errata to clarify that a
> per-connection limit is best. At the very least there seems
> some risky ambiguity here.
>=20
> I disagree, as per below...
>=20
> In fact, AFAICT RFC 5961, Section 7, page 13 --
>=20
>    https://tools.ietf.org/html/rfc5961#section-7
>=20
> -- has several passages that seem to implicitly suggest that
> the rate-limiting mechanism it specifies as a SHOULD is not
> per-connection but rather global.
>=20
> There's plenty of benefit to setting a single configuration for an
> entire system vs. setting per-connection limits. That's completely
> different from whether the ACKs are tallied globally or per =
connection.
>=20
> First:
>=20
>     ... an
>     administrator who is more interested in faster cleanup of stale
>     connections (i.e., concerned about excess TCP state) may decide to
>     set a higher value thus allowing more RST's to be processed in any
>     given time period.
>=20
> If this is a single-connection rate limit, then what practical reason
> would an administrator have to allow "more RST's to be processed in
> any given time period"? I can't think of a case where a legit live
> connection would send more than one RST.
> A single connection might process many RSTs because some can be out of
> window (they'd be dropped) or not at the end of the window (dropped
> under some configurations). I.e., receiving and evaluating RSTs =
doesn't
> always result in resetting the connection.
>=20
> If this is about a single
> connection, then allowing one mid-window RST (and responding to that
> one with a single challenge ACK) should be enough,
> It would be IF the other end issued a legitimate RST. If this is a
> spoofing attack, the data can still keep coming which means that the
> window might still be moving forward by the time the next attack RST
> arrives. As a result, this sort of 'catch up' can go on until the
> connection ends. The point of the text here is to avoid wasting
> resources in such cases.
> in most cases, to
> send a challenge ACK that elicits a proper RST from the remote
> endpoint. It's not clear to me what the motivation would be for
> intentionally sending a challenge ACK in response to more than one
> mid-window RST in a single connection.
> Protecting against that case is the motivation *of the entire =
document*.
>=20
> Thus AFAICT "allowing more RST's to be processed in any given time
> period" suggests that the rate limit is across connections.
>=20
> There's no benefit to doing this across connections at all, unless you
> think that there's some reason that legitimate connections shouldn't =
be
> reset by valid RSTs (there's no benefit to that).
>=20
> Second:
>=20
>     The time limit SHOULD be tunable to help timeout brute force =
attacks
>     faster than a potential legitimate flood of RSTs.
>=20
> I may be missing something, but if this rate-limiting mechanism is
> only about a single connection, then what is a "legitimate flood of
> RSTs"?
> A sequence of RSTs that pick different values, as described in RFC =
4953.
>=20
> Or a sequence of RSTs aimed at responses to the ACK-after-RST that is
> chasing legitimate data, as described above.
>=20
>   Why would it be important for a single connection to allow a
> "legitimate flood of RSTs" destinted for itself?
> It wouldn't - that's the point of this text. You should treat a flood =
of
> RSTs for a single connection as an attack and throttle them =
accordingly.
>=20
> A single legitimate
> live connection should not send out a flood of RSTs; it should send
> out one. Once there is one legitimate RST that arrives and is
> processed, the receiver deletes the connection, and any other arriving
> RSTs don't match any existing connection, so are ignored and
> dropped, in which case is no challenge ACK to rate-limit.
> Again, the point of this document are attacks, not legitimate =
connections.
> By contrast, if this rate limit applies across connections then "a
> potential legitimate flood of RSTs" makes sense, since a group of
> legitimate live connections that all close in a burst may, in certain
> cases, generate a legitimate flood of RSTs. So tuning the limit
> upward would make sense.
>=20
> There's no case I can imagine where a set of attacks on one connection
> should be cause for concern on another necessarily.
>=20
> Yes, there's no real need to set per-connection limits (i.e., a global
> per-connection limit seems fine), but counting them as a group serves =
no
> purpose against any threat, unless you think that "if I'm attacked on
> one connection, I will be attacked on another" - but that's an =
incorrect
> inference IMO.
>=20
>=20
> Sorry if I'm missing something obvious. But it seems like there are at
> least real ambiguities there, on a point significant enough to be =
worth
> clearing up.
>=20
> I think you're missing the point of the document, but that's just my
> interpretation of the exchange.
>=20
> Joe
>=20
> _______________________________________________
> tcpm mailing list
> tcpm@ietf.org
> https://www.ietf.org/mailman/listinfo/tcpm
>=20
>=20


From nobody Fri Aug 19 09:52:16 2016
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4062A12D1E6 for ; Fri, 19 Aug 2016 09:52:15 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -15.769
X-Spam-Level: 
X-Spam-Status: No, score=-15.769 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-1.247, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, USER_IN_DEF_DKIM_WL=-7.5] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=cisco.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id OTGvPAZN79Nq for ; Fri, 19 Aug 2016 09:52:13 -0700 (PDT)
Received: from aer-iport-4.cisco.com (aer-iport-4.cisco.com [173.38.203.54]) (using TLSv1.2 with cipher DHE-RSA-SEED-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1F9D912B02C for ; Fri, 19 Aug 2016 09:52:12 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=3622; q=dns/txt; s=iport; t=1471625533; x=1472835133; h=subject:to:references:cc:from:message-id:date: mime-version:in-reply-to; bh=uui4kGVxrtkimUq8mjHqkC9VSC4nK2htAgq1zqxgitc=; b=FfTgivm4naE9RZuWVcA+kFx/mymcBDWadig+xAruYAeXCcXjpQQ0uve8 ckuidRmHCtnQj4ozfstf4G8YPaiseiv5ckawzU+DnizmCqYo7fgH329Fb Devl+rDNHmiEWZBsiITkkdl7HYOwM6RwBVrzJjHeA7Gg5U/fEnSEz9ABm k=;
X-Files: signature.asc : 481
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: =?us-ascii?q?A0ABBQDpN7dX/xbLJq1eDoQMKlK1YYQMh?= =?us-ascii?q?h0CghwRAgEBAQEBAQFeJ4RfAQUjVhALDgoqAgJXBgEMBgIBAYgtqjGQAAEBAQE?= =?us-ascii?q?BAQEBAQEBAQEBAQEBAQEQDogjglWHQYJaAQSZR4M+gXOBYIgNgWuHa4V2hmiFV?= =?us-ascii?q?YN4NCCCDYEyPTo0hy0BAQE?=
X-IronPort-AV: E=Sophos;i="5.28,545,1464652800";  d="asc'?scan'208";a="645025295"
Received: from aer-iport-nat.cisco.com (HELO aer-core-1.cisco.com) ([173.38.203.22]) by aer-iport-4.cisco.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 19 Aug 2016 16:52:11 +0000
Received: from [10.61.211.158] ([10.61.211.158]) by aer-core-1.cisco.com (8.14.5/8.14.5) with ESMTP id u7JGqAVQ027584; Fri, 19 Aug 2016 16:52:10 GMT
To: Mark Nottingham , Joe Touch 
References: <5CD67877-19E3-4E79-BBF2-3E270343A378@mnot.net> <2197232f-10d7-28cb-fcc9-05bd495e3c22@isi.edu> <20160817064545.GD16017@1wt.eu> <7f7b129c-f156-d067-bef8-4a2213f461ac@isi.edu> <20160817180802.GA16773@1wt.eu> <4ab7c5b0-3722-1346-f481-a8d76de70034@isi.edu> <20160817211317.GA16929@1wt.eu>   <27b58b64-48cd-39af-78b3-ef583c585fa6@isi.edu> <20160818053837.GC16773@1wt.eu> <7a36e025-4882-4f8b-7a83-9fdcd990a971@isi.edu> 
From: Eliot Lear 
Message-ID: <8aaa9b2b-9dde-a198-70cd-299a7c04dda2@cisco.com>
Date: Fri, 19 Aug 2016 18:52:10 +0200
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Thunderbird/45.2.0
MIME-Version: 1.0
In-Reply-To: 
Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="JKF0GXWvi4J6UhLqV5PGDHUvfEtjgCpN4"
Archived-At: 
Cc: tcpm@ietf.org, Matthew Kerwin , Daniel Stenberg , Patrick McManus , Willy Tarreau , HTTP Working Group 
Subject: Re: [tcpm] TCP Tuning for HTTP - update
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Fri, 19 Aug 2016 16:52:15 -0000

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--JKF0GXWvi4J6UhLqV5PGDHUvfEtjgCpN4
Content-Type: multipart/mixed; boundary="huc2Duh27xacxWP1kMwLFD030nxee9OPB"
From: Eliot Lear 
To: Mark Nottingham , Joe Touch 
Cc: Willy Tarreau , Matthew Kerwin ,
 tcpm@ietf.org, HTTP Working Group ,
 Patrick McManus , Daniel Stenberg 
Message-ID: <8aaa9b2b-9dde-a198-70cd-299a7c04dda2@cisco.com>
Subject: Re: [tcpm] TCP Tuning for HTTP - update
References: <5CD67877-19E3-4E79-BBF2-3E270343A378@mnot.net>
 <2197232f-10d7-28cb-fcc9-05bd495e3c22@isi.edu>
 <20160817064545.GD16017@1wt.eu>
 <7f7b129c-f156-d067-bef8-4a2213f461ac@isi.edu>
 <20160817180802.GA16773@1wt.eu>
 <4ab7c5b0-3722-1346-f481-a8d76de70034@isi.edu>
 <20160817211317.GA16929@1wt.eu>
 
 
 <27b58b64-48cd-39af-78b3-ef583c585fa6@isi.edu>
 <20160818053837.GC16773@1wt.eu>
 <7a36e025-4882-4f8b-7a83-9fdcd990a971@isi.edu>
 
In-Reply-To: 

--huc2Duh27xacxWP1kMwLFD030nxee9OPB
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

Just to be the stick in the mud for a moment...


On 8/19/16 5:49 AM, Mark Nottingham wrote:
>> On 19 Aug 2016, at 1:07 AM, Joe Touch  wrote:
>>
>> I do think that this doc needs to figure out whom it is speaking to, w=
hat advice they actually need, etc.
>>
>> If the result is a set of recommendations that involve the word "sysct=
l", I remain skeptical it is appropriate as an RFC
>
> I think there's broad agreement on both of these points.
>
> I'm wondering if it makes sense to aim it primarily at HTTP implementer=
s rather than administrators, with the notion that it would inform:
>
> - Their implementation decisions
> - The configuration choices they offer to administrators / users
> - Their documentation (e.g., advice to their administrators when the im=
plementation can't change the appropriate parts of the OS)
>
> Would that help?=20
>
> If so, it might make sense to organise it into sections for clients and=
 servers (and intermediaries, if there's anything that isn't covered by t=
he combination of the first two). Although IIRC Daniel was already talkin=
g about doing that.
>
>

If there are very specific settings for Linux and they are well vetted,
I see no reason not to continue to include them in an appendix, as they
are now.  Cheat sheets are nice, and there is nothing wrong with a cheat
sheet being in an RFC (and I wouoldn't mind a few being RFCs).

Eliot


--huc2Duh27xacxWP1kMwLFD030nxee9OPB--

--JKF0GXWvi4J6UhLqV5PGDHUvfEtjgCpN4
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2

iQEcBAEBCAAGBQJXtzk7AAoJEIe2a0bZ0noz3j8H/AlKSO/YieosQNc79BhgjJSu
hZBWGNCqseKInf/AkKC4yalGAc4bYrqEk/B2lolIFqzWYKEYuV0ZvcZXng5xFE2u
GUIGJSqPfqNBreMHxxbsLyMphxxe7QaDS3m1Q8itr8C9PqiCp9yQ3UFK5b7dJT2N
7uFRsGMpAyfLoCr0JynJM3vnIcOs85XuiARySw1EgJeU5bmKnvCDJSqEXZHIRfKs
VxjTN4TP9yRHrr+PWGl9qWMCqZUyIbK8YIqkpkTMwVkUfGfz2ZGbpm1iAdWKJAoN
3S3nCYP3bMEuizn+LOXxW1Dw061gZsngYeRkRhLsSDY5LOBgnR1IjltIYqZKhA8=
=PcHm
-----END PGP SIGNATURE-----

--JKF0GXWvi4J6UhLqV5PGDHUvfEtjgCpN4--


From nobody Fri Aug 19 10:11:14 2016
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A947E12D623 for ; Fri, 19 Aug 2016 10:11:10 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -3.948
X-Spam-Level: 
X-Spam-Status: No, score=-3.948 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RP_MATCHES_RCVD=-1.247, SPF_PASS=-0.001] autolearn=unavailable autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=google.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TVCuqesGO48i for ; Fri, 19 Aug 2016 10:11:08 -0700 (PDT)
Received: from mail-it0-x22c.google.com (mail-it0-x22c.google.com [IPv6:2607:f8b0:4001:c0b::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 3EDB112D5FC for ; Fri, 19 Aug 2016 10:11:08 -0700 (PDT)
Received: by mail-it0-x22c.google.com with SMTP id x131so33762586ite.0 for ; Fri, 19 Aug 2016 10:11:08 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=HKK//jZo6Ukf7wuP/TqqjvIZfroiM5nrv+pBHcghv6M=; b=QyblFUFDfD+j48CJwp9jbWdfJr99DEYlq4EtJNsEMXI23QfZP6olOWZqcj/wv8RH7v bH9MRLtTQk5xvZNx09IB2eKJqWDamMLETtS3OLfTES+oZ4+fmVg+oK9SStbDzA13jX88 n3vt1lvQ7/UAL3WVXe2F65wCtY8rYQJkgPlugNgROTlqrKc+k83afolB3azR4Y3fIeOF 89Cs9XSE0yJv5bnRx+odLZLufzPM19KtAgj/R/DW/EpVZI14YKdzLHbXHj873gi3gR8S CXwiXdFWB8nJUNKqOf5jTg4bs7R7Cxhh0MtpSf3eQIWkSHROkhrtGEmLKXiSFVmfxFem D0ow==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=HKK//jZo6Ukf7wuP/TqqjvIZfroiM5nrv+pBHcghv6M=; b=h1jv7UFYAsX7JDrRLcVJvUIvo4ARrsxJjzELeNu8PW4i5WpaZDp2WMoNmNC0csGrvB 84AblIIiaJ4hBrYbLwdLyBym+WishCkaOChoxHiMHwZho/i1lbZ9dnHaXunTeUFMpdkx eQBJWFdtJdeA2+CVkBGGlaL9hZzJMKXAfSsAN1amc20u+mSOwY+gO6V2OKC4HBI6eacR I0AyPsj4N7mKsM7sNDZCK5w1S+3oG1ctGGYz6IWEhAzDRLfM05TxoHqKegjphvLhDmGm jEIEP2XYBpDayKwsovrL661IqdRW3vtbSWE9W+NWIOPraS5cW0EZK/QE5J07Nh3tZ7vp D9AQ==
X-Gm-Message-State: AEkooutxkKvqIMbTHiVjFw337aX/7qP7zuyzvUwEdKYIp7/xLo1gK6/vYTT3dr6Fabesg6/E2Gyklb2IyTsBW15X
X-Received: by 10.36.82.81 with SMTP id d78mr7394747itb.65.1471626667205; Fri, 19 Aug 2016 10:11:07 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.64.244.103 with HTTP; Fri, 19 Aug 2016 10:10:26 -0700 (PDT)
In-Reply-To: <00594c32f56877bfc2fde20132250174@mail.gmail.com>
References: <20160708211635.32095.77047.idtracker@ietfa.amsl.com> <2f56d1b8-7450-42de-4a24-53df9cf4c045@it.uc3m.es>   <623f46dedcc661ae74563f6bf08f46a5@mail.gmail.com>  <00594c32f56877bfc2fde20132250174@mail.gmail.com>
From: Yuchung Cheng 
Date: Fri, 19 Aug 2016 10:10:26 -0700
Message-ID: 
To: Karen Elisabeth Egede Nielsen 
Content-Type: text/plain; charset=UTF-8
Archived-At: 
Cc: tcpm IETF list , TCP Prague List 
Subject: Re: [tcpm] Fwd: I-D Action: draft-bagnulo-tcpm-tcp-low-rtt-00.txt
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Fri, 19 Aug 2016 17:11:11 -0000

On Thu, Aug 18, 2016 at 1:05 AM, Karen Elisabeth Egede Nielsen
 wrote:
> [Karen Elisabeth Egede Nielsen] I agree completely with your view and
> approach.
>
> In respect to the draft draft-bagnulo-tcpm-tcp-low-rtt-00.txt and the given
> recommendations for low RTO
> is that they do not (likely) take 2. Into account - meaning that they are
> looking for the low RTO to solve 2 for tail recovery.
> The bad thing is that they then also get the CWND reductions (as you also
> say).
>
>
> I would like for that this aspect is taken into consideration/given thoughts
> in the draft draft-bagnulo-tcpm-tcp-low-rtt-00.txt and the general 4LS
> effort.
> If we end with very low recommendations for RTO in order to handle 2. we
> risk undermining the RTO-retransmission function as well as the TLP function
> for now and forever.
>
> It might be that one for TCPs, that do not support TLP(/RACK), need to
> operate with very low RTOs to handle tail loss's "fast enough", but the
> recommendation for
> "future standard ?" TCP with TLP always enabled might be different. In this
> context lowering the RTO to handle 2. should be considered as a "temporary
> hack" ?

One feature worth discussion in the draft is the penalty of cwnd=1 on
spurious timeouts.

For DC, the RTT is so fairly low that we may consider a false reset of
cwnd to 1 is not a big deal. But consider in world of 100Gbps BDP will
still be hundreds of packets so it is going to be a non-trivial
penalty.

Therefore it is important to discuss the timeouts with the remedy
operations: how well can the connection detect spurious timeouts:
F-RTO or Eifel are the two algorithms by IETF, Linux also supports
DSACK-based undo.

However, both F-RTO and Eifel have their limitations: F-RTO requires
the sender to have new data to transmit after the first DUPACK of
timeout. In the DC world, the requests or responses are often short
enough to prevent that. Eifel relies on timestamps which is ticked at
an interval higher than the DC RTT (e.g., Linux == jiffies == 1ms
often), therefore the timestamps fail to distinguish original and
retransmit if RTO was fired before the next tick.

I plan to cover these aspects in next RACK rev.

>
> BR, Karen


From nobody Fri Aug 19 10:28:26 2016
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B693212D76F for ; Fri, 19 Aug 2016 10:28:24 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -8.147
X-Spam-Level: 
X-Spam-Status: No, score=-8.147 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, RP_MATCHES_RCVD=-1.247] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id IOyhq1yj63My for ; Fri, 19 Aug 2016 10:28:23 -0700 (PDT)
Received: from boreas.isi.edu (boreas.isi.edu [128.9.160.161]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 2E4FC12D12F for ; Fri, 19 Aug 2016 10:28:23 -0700 (PDT)
Received: from [128.9.184.42] ([128.9.184.42]) (authenticated bits=0) by boreas.isi.edu (8.13.8/8.13.8) with ESMTP id u7JHRS15014309 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Fri, 19 Aug 2016 10:27:29 -0700 (PDT)
To: Mark Nottingham 
References: <5CD67877-19E3-4E79-BBF2-3E270343A378@mnot.net> <2197232f-10d7-28cb-fcc9-05bd495e3c22@isi.edu> <20160817064545.GD16017@1wt.eu> <7f7b129c-f156-d067-bef8-4a2213f461ac@isi.edu> <20160817180802.GA16773@1wt.eu> <4ab7c5b0-3722-1346-f481-a8d76de70034@isi.edu> <20160817211317.GA16929@1wt.eu>   <27b58b64-48cd-39af-78b3-ef583c585fa6@isi.edu> <20160818053837.GC16773@1wt.eu> <7a36e025-4882-4f8b-7a83-9fdcd990a971@isi.edu> 
From: Joe Touch 
Message-ID: 
Date: Fri, 19 Aug 2016 10:27:26 -0700
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0
MIME-Version: 1.0
In-Reply-To: 
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: 7bit
X-ISI-4-43-8-MailScanner: Found to be clean
X-MailScanner-From: touch@isi.edu
Archived-At: 
Cc: tcpm@ietf.org, Matthew Kerwin , Daniel Stenberg , Patrick McManus , Willy Tarreau , HTTP Working Group 
Subject: Re: [tcpm] TCP Tuning for HTTP - update
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Fri, 19 Aug 2016 17:28:25 -0000

Mark,


On 8/18/2016 8:49 PM, Mark Nottingham wrote:
>> On 19 Aug 2016, at 1:07 AM, Joe Touch  wrote:
>>
>> I do think that this doc needs to figure out whom it is speaking to, what advice they actually need, etc.
>>
>> If the result is a set of recommendations that involve the word "sysctl", I remain skeptical it is appropriate as an RFC
>
> I think there's broad agreement on both of these points.
>
> I'm wondering if it makes sense to aim it primarily at HTTP implementers rather than administrators, with the notion that it would inform:
>
> - Their implementation decisions
> - The configuration choices they offer to administrators / users
> - Their documentation (e.g., advice to their administrators when the implementation can't change the appropriate parts of the OS)
>
> Would that help? 

I think so. I also think it would be very important to differentiate the
issues that are unique to HTTP (if any) vs. those that are generic to
any transaction system.

> If so, it might make sense to organise it into sections for clients and servers (and intermediaries, if there's anything that isn't covered by the combination of the first two). Although IIRC Daniel was already talking about doing that.

Clients, servers, proxies, and (even though a violation of so many parts
of the Internet) transparent proxies.

Joe


From nobody Fri Aug 19 13:12:42 2016
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 00BD512B05D for ; Fri, 19 Aug 2016 13:12:40 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -3.247
X-Spam-Level: 
X-Spam-Status: No, score=-3.247 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RP_MATCHES_RCVD=-1.247, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=google.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FRnuuU43md3N for ; Fri, 19 Aug 2016 13:12:38 -0700 (PDT)
Received: from mail-yb0-x22b.google.com (mail-yb0-x22b.google.com [IPv6:2607:f8b0:4002:c09::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id C97E512DCB8 for ; Fri, 19 Aug 2016 13:05:43 -0700 (PDT)
Received: by mail-yb0-x22b.google.com with SMTP id e31so19188663ybi.3 for ; Fri, 19 Aug 2016 13:05:43 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=+aPFlWyRgqXxxt2mVJwtmFaY58L/L9U/APF+4KIHfy8=; b=fEhdXArxg7kQKQ9U5gFaVyEc8Oj1oQPR4KGJQAOLr9eyQSp+lzsifexHbxEse4Du3A Nj5gKOgSdzYmg9VPVZS1IS7CmnnvfGWmoN5V336MgOKEtcf+4u7p/KfLigoU4bZrG8Dq 37Zqp2DLK8+5y9/qtaAQnRCeoAHvTaq2aInKm6yxBkUE/XxfwTZ+fUPLDCLZ4OR+sAMr z1hnWReNWZr4vFdAu0EnXM5p0GLI5Hh44wm8B3NXNqNZybNYCQijQVkG4oOLMiCrVFd+ 3IWJTEXs9HQqswVErGSwjBUCkflwP0fX6D5wklM/xDrQyaruNDh7e9Ojce2V/tpKIu2d sVGA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=+aPFlWyRgqXxxt2mVJwtmFaY58L/L9U/APF+4KIHfy8=; b=cD3jeZuy8D9idNklCYEVYmeBD8Ywao0zmczevFhBGXgD3BpRjg/HVImEnU8VtCIufN sLmKh2GVUvwpwtj4ltAhNGnDkpvf2Pv2uIXTKpb/oIlGzS8cM++5tUUzURgkc8HwJ7ak cpp6ILnjfytOGFp6/aiX4izUevkGwbfEwXPgOY89r7CpKJH5FnsK6TtBxzK5Mm/TLFx/ j9oGOMo5sn+60Iw9UwjYqDOn8GvVn7wH0vVIw4i1DSk/8ojtGX6DtxzT1nDCvvNepV8L ApVZh5d1W7M/IsozZL1XiJe4d45j/D+Y4deqwRsjbaZJNOLC0aZfhJ/lFNgPI/HwdMEu 14GA==
X-Gm-Message-State: AEkoouuio/B2n05pftEZYgPYqXYxBWen0dWoWfI15LNhdqxS0J3KvIYojATAXbkcvjp0ME+LLxVSrsbUmVbQdyAb
X-Received: by 10.37.15.9 with SMTP id 9mr7202778ybp.139.1471637142849; Fri, 19 Aug 2016 13:05:42 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.37.197.8 with HTTP; Fri, 19 Aug 2016 13:05:41 -0700 (PDT)
In-Reply-To: <781AFF22-3532-44A0-AB14-628FF86BE866@mnot.net>
References: <0CC24FC1-37E1-4125-9627-05726A9D9406@mnot.net> <7fa95741-ac58-3183-1b92-238bd4b4dae6@isi.edu> <5CD67877-19E3-4E79-BBF2-3E270343A378@mnot.net> <2197232f-10d7-28cb-fcc9-05bd495e3c22@isi.edu> <87D56B16-B091-4E9C-9B88-D136ED4BC819@mnot.net>   <9eb63b81-a9bc-cb65-913a-d9a5af0d2fe2@isi.edu> <781AFF22-3532-44A0-AB14-628FF86BE866@mnot.net>
From: Jana Iyengar 
Date: Fri, 19 Aug 2016 13:05:41 -0700
Message-ID: 
To: Mark Nottingham 
Content-Type: multipart/alternative; boundary=001a113f5b9a23cd0f053a7239b3
Archived-At: 
Cc: Patrick McManus , tcpm@ietf.org, HTTP Working Group , Daniel Stenberg , Joe Touch 
Subject: Re: [tcpm] TCP Tuning for HTTP - update
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Fri, 19 Aug 2016 20:12:40 -0000

--001a113f5b9a23cd0f053a7239b3
Content-Type: text/plain; charset=UTF-8

Mark,

[I have not have read the entire thread in detail, but responding to one
question.]

I'd especially like to hear from other people in the TCP community; we've
> heard from Joe and Michael; do others share their opinions, or have another
> view?
>

Responding to this specific question: I think this document is useful.
There's tremendous value in having a document that discusses protocol
interactions, especially for the most important protocol interaction today
(HTTP/TCP).  Pursuing this document as Informational makes complete sense,
and is well within IETF norms.

Implementation notes/advice is actually quite a useful bit that
traditionally shows up in appendices, but arguably it really doesn't matter
where it shows up. It's valuable. I would remove specific APIs and instead
talk about mechanisms that are common enough (remove setsockopt() and
sysctls, but talk about Nagle, socket buffers, etc.) but there are a number
of commonly deployed mechanisms that it makes sense to talk about them
here. I would suggest that the document discuss important interactions from
a browser's point of view as separate from those at a web server.

I am happy to review and suggest changes to the document, but I think it's
a fine fit.

- jana

--001a113f5b9a23cd0f053a7239b3
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable


Mark,

[I have not have read the entire thread in =
detail, but responding to one question.]


I'd especially like to hear from other people in the TCP community; we&=
#39;ve heard from Joe and Michael; do others share their opinions, or have =
another view?

Responding to this specif=
ic question: I think this document is useful. There's tremendous value =
in having a document that discusses protocol interactions, especially for t=
he most important protocol interaction today (HTTP/TCP).=C2=A0 Pursuing thi=
s document as Informational makes complete sense, and is well within IETF n=
orms.

Implementation notes/advice is actually quit=
e a useful bit that traditionally shows up in appendices, but arguably it r=
eally doesn't matter where it shows up. It's valuable. I would remo=
ve specific APIs and instead talk about mechanisms that are common enough (=
remove setsockopt() and sysctls, but talk about Nagle, socket buffers, etc.=
) but there are a number of commonly deployed mechanisms that it makes sens=
e to talk about them here. I would suggest that the document discuss import=
ant interactions from a browser's point of view as separate from those =
at a web server.

I am happy to review and sugg=
est changes to the document, but I think it's a fine fit.
- jana


--001a113f5b9a23cd0f053a7239b3--


From nobody Sun Aug 21 10:25:12 2016
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1DB3F12D13E for ; Sun, 21 Aug 2016 10:25:11 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.448
X-Spam-Level: 
X-Spam-Status: No, score=-2.448 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RP_MATCHES_RCVD=-0.548, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id iC24ztKN_3Fn for ; Sun, 21 Aug 2016 10:25:09 -0700 (PDT)
Received: from mail.sfc.wide.ad.jp (shonan.sfc.wide.ad.jp [IPv6:2001:200:0:8803::53]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 397E912D08C for ; Sun, 21 Aug 2016 10:25:08 -0700 (PDT)
Received: from mail-ua0-f172.google.com (mail-ua0-f172.google.com [209.85.217.172]) by mail.sfc.wide.ad.jp (Postfix) with ESMTPSA id C84512783D3 for ; Mon, 22 Aug 2016 02:25:06 +0900 (JST)
Received: by mail-ua0-f172.google.com with SMTP id 74so154654049uau.0 for ; Sun, 21 Aug 2016 10:25:06 -0700 (PDT)
X-Gm-Message-State: AEkoouv0V4uFJhWZ3OrP9jPUASxKvW01d5l6y7zoIpIWB05jMN4EoalrmYMXnio9Pqc9epZ3+3jJt+xayt50XA==
X-Received: by 10.176.80.19 with SMTP id b19mr9423371uaa.11.1471800305308; Sun, 21 Aug 2016 10:25:05 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.159.37.1 with HTTP; Sun, 21 Aug 2016 10:25:04 -0700 (PDT)
In-Reply-To: 
References:  <7070553C-65D1-46EE-95F4-DAE82E1F5A5E@weston.borman.com>  <2D524A8D-A5CA-45A6-B94D-FA1DA0CEE609@weston.borman.com>   
From: Yoshifumi Nishida 
Date: Sun, 21 Aug 2016 10:25:04 -0700
X-Gmail-Original-Message-ID: 
Message-ID: 
To: Neal Cardwell 
Content-Type: multipart/alternative; boundary=94eb2c18f87860de54053a9836e4
Archived-At: 
Cc: Kobby Carmona , "tcpm@ietf.org" , David Borman 
Subject: Re: [tcpm] Possible deadlock scenario with retransmission on both sides at the same time
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Sun, 21 Aug 2016 17:25:11 -0000

--94eb2c18f87860de54053a9836e4
Content-Type: text/plain; charset=UTF-8

Hi Neal,

Thanks for the info.
So, it seems to me that the linux code has the issue Kobby pointed out.
Or, am I missing something?
--
Yoshi


On Mon, Aug 15, 2016 at 6:18 PM, Neal Cardwell  wrote:

> On Mon, Aug 15, 2016 at 6:39 PM, Yoshifumi Nishida
>  wrote:
> > Hello,
> > I personally think this is an interesting corner case for discussion.
> > It looks a minor one, but I'm not not very sure if we can leave it for
> each
> > implementation.
> > I also guess a question would be if the BSD's fix is the best way for the
> > issue.
>
> Yes, I agree this is an interesting case for discussion.
>
> FWIW, as a point of comparison for discussion, Linux's approach is a
> little different: in Linux, the sender does not rewind SND.NXT on
> retransmissions (RTO or Fast Recovery). Then the sender usually uses
> SND.NXT for the seq field of outgoing pure ACKs. I say "usually"
> because the Linux code has some code to deal with the case where the
> receiver has withdrawn the receive window, so that SND.NXT is now
> beyond the receive window. The code tcp_send_ack() uses to pick a seq
> for outgoing pure ACKs looks like:
>
> /* SND.NXT, if window was not shrunk.
>  * If window has been shrunk, what should we make? It is not clear at
> all.
>  * Using SND.UNA we will fail to open window, SND.NXT is out of
> window. :-(
>  * Anything in between SND.UNA...SND.UNA+SND.WND also can be already
>  * invalid. OK, let's make this for now:
>  */
> static inline __u32 tcp_acceptable_seq(const struct sock *sk)
> {
>         const struct tcp_sock *tp = tcp_sk(sk);
>
>         if (!before(tcp_wnd_end(tp), tp->snd_nxt))
>                 return tp->snd_nxt;
>         else
>                 return tcp_wnd_end(tp);
> }
>
> neal
>

--94eb2c18f87860de54053a9836e4
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable


Hi Neal,

Thanks for the info.
So, it seems to me that the linux code has the issue Kobby pointed out.
Or, am I missing something?
--
Yoshi
<=
div>

On Mon, A=
ug 15, 2016 at 6:18 PM, Neal Cardwell <ncardwell@google.com> wrote:
On Mon, Aug 15, 2016 at =
6:39 PM, Yoshifumi Nishida

<nishida@sfc=
.wide.ad.jp> wrote:

> Hello,

> I personally think this is an interesting corner case for discussion.<=
br>
> It looks a minor one, but I'm not not very sure if we can leave it=
 for each

> implementation.

> I also guess a question would be if the BSD's fix is the best way =
for the

> issue.



Yes, I agree this is an interesting case for discussion.



FWIW, as a point of comparison for discussion, Linux's approach is a
little different: in Linux, the sender does not rewind SND.NXT on

retransmissions (RTO or Fast Recovery). Then the sender usually uses

SND.NXT for the seq field of outgoing pure ACKs. I say "usually"<=
br>
because the Linux code has some code to deal with the case where the

receiver has withdrawn the receive window, so that SND.NXT is now

beyond the receive window. The code tcp_send_ack() uses to pick a seq

for outgoing pure ACKs looks like:



/* SND.NXT, if window was not shrunk.

=C2=A0* If window has been shrunk, what should we make? It is not clear at<=
br>
all.

=C2=A0* Using SND.UNA we will fail to open window, SND.NXT is out of

window. :-(

=C2=A0* Anything in between SND.UNA...SND.UNA+SND.WND also can be already
=C2=A0* invalid. OK, let's make this for now:

=C2=A0*/

static inline __u32 tcp_acceptable_seq(const struct sock *sk)

{

=C2=A0 =C2=A0 =C2=A0 =C2=A0 const struct tcp_sock *tp =3D tcp_sk(sk);



=C2=A0 =C2=A0 =C2=A0 =C2=A0 if (!before(tcp_wnd_end(tp), tp->snd_nxt))
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 return tp->snd_n=
xt;

=C2=A0 =C2=A0 =C2=A0 =C2=A0 else

=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 return tcp_wnd_end(=
tp);

}



neal





--94eb2c18f87860de54053a9836e4--


From nobody Sun Aug 21 12:45:28 2016
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7053C12D811 for ; Sun, 21 Aug 2016 12:45:27 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -3.249
X-Spam-Level: 
X-Spam-Status: No, score=-3.249 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RP_MATCHES_RCVD=-0.548, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=google.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id M8erlPcln7MG for ; Sun, 21 Aug 2016 12:45:26 -0700 (PDT)
Received: from mail-oi0-x233.google.com (mail-oi0-x233.google.com [IPv6:2607:f8b0:4003:c06::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 10A7012D80E for ; Sun, 21 Aug 2016 12:45:26 -0700 (PDT)
Received: by mail-oi0-x233.google.com with SMTP id c15so126133305oig.0 for ; Sun, 21 Aug 2016 12:45:26 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=7mMdzEvAXatHRonBcTLg6/Z1j4kgE92mV74Ax1sefdI=; b=ZKNdn5+iTwpPHhOeUb8KtJbmPKVW0JJOGlVY7ZUg4Mm6whL7O5gamJxh9bxzRotI1J TyeuXvM9AL0BWi77QLk9G71kF1H6kcJAdmBAcqllksir46SshudAiouJKj/Bbce2pvnF BDmJr0mQm1mSrQwjGkdjAo3/MyXMykn5pniI3C+T0zjxCKJButvdAukRgSU6LKfrvRtm nL/D8nV+dEnTOoFXGBMTR+afvws3/lszjJbBivkkBJCOOLeOXYe7gPvEA/LWo9EIsfZs Aa2y5YvnEFTfjDiLWZ1A/kH2i9PPXcwfD3USlArzZtqsiuaKMQlpi7TZSy8G2ROwqz1a X9Pg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=7mMdzEvAXatHRonBcTLg6/Z1j4kgE92mV74Ax1sefdI=; b=Dq58CT3BCpOiqFz1P3gktjRHBrrnV34jBV8gKzZL5ESsmSvkVHSo7HPd4nP2LKjH8k EQ2YN5yrfgXYg9A9tmye5nCG4IcKuanqN4L99Z7SSdU8mv4GL9PMD4bh8jCW9r+VJvVL F54SpgThaHY80Vy/1aK+N5zUnmm/Z3tRX0KNwD2ODPj61/7BqbxS8ZQ3LAfuDCxpCI9h UBVsXvrEA1+ae38htG5sIeitZpH1PGWYUt9hMMznLrXjzpsQFoJZDk0XwNRirw1tgsZs fZe/ltkGeqIuM9TcYwFTY7KhjqJSYTFN4uNeL8QZmO7k7+QNi/48LFDTzojicXSFBq7B jcEQ==
X-Gm-Message-State: AEkoouutNjcCLuFQ8V6uUmGKt1yR82s5eMwLRhwY7nWkPiIaXDc4ORr3jHeIXb6cGoVriknVyd2dszklldOyl6NQ
X-Received: by 10.202.253.208 with SMTP id b199mr10824224oii.171.1471808725202;  Sun, 21 Aug 2016 12:45:25 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.202.52.130 with HTTP; Sun, 21 Aug 2016 12:44:54 -0700 (PDT)
In-Reply-To: 
References:  <7070553C-65D1-46EE-95F4-DAE82E1F5A5E@weston.borman.com>  <2D524A8D-A5CA-45A6-B94D-FA1DA0CEE609@weston.borman.com>    
From: Neal Cardwell 
Date: Sun, 21 Aug 2016 15:44:54 -0400
Message-ID: 
To: Yoshifumi Nishida 
Content-Type: text/plain; charset=UTF-8
Archived-At: 
Cc: Kobby Carmona , David Borman , "tcpm@ietf.org" 
Subject: Re: [tcpm] Possible deadlock scenario with retransmission on both sides at the same time
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Sun, 21 Aug 2016 19:45:27 -0000

I am pretty sure Linux does not have the issue Kobby pointed out in this thread.

At a high level Linux should be OK because it follows the principle
David Borman laid out in his August 16 email: "ACK-only packets should
be sent with the largest in-window sequence number that has ever been
sent."

Linux obeys that principle by using tp->snd_nxt to store the largest
sequence number that has ever been sent, and having
tcp_acceptable_seq() use tp->snd_nxt but clamp the outgoing sequence
number to make sure it is in-window. To be able to do this, in Linux,
the sender does not rewind tp->snd_nxt on retransmissions.

neal

On Sun, Aug 21, 2016 at 1:25 PM, Yoshifumi Nishida
 wrote:
> Hi Neal,
>
> Thanks for the info.
> So, it seems to me that the linux code has the issue Kobby pointed out.
> Or, am I missing something?
> --
> Yoshi
>
>
> On Mon, Aug 15, 2016 at 6:18 PM, Neal Cardwell  wrote:
>>
>> On Mon, Aug 15, 2016 at 6:39 PM, Yoshifumi Nishida
>>  wrote:
>> > Hello,
>> > I personally think this is an interesting corner case for discussion.
>> > It looks a minor one, but I'm not not very sure if we can leave it for
>> > each
>> > implementation.
>> > I also guess a question would be if the BSD's fix is the best way for
>> > the
>> > issue.
>>
>> Yes, I agree this is an interesting case for discussion.
>>
>> FWIW, as a point of comparison for discussion, Linux's approach is a
>> little different: in Linux, the sender does not rewind SND.NXT on
>> retransmissions (RTO or Fast Recovery). Then the sender usually uses
>> SND.NXT for the seq field of outgoing pure ACKs. I say "usually"
>> because the Linux code has some code to deal with the case where the
>> receiver has withdrawn the receive window, so that SND.NXT is now
>> beyond the receive window. The code tcp_send_ack() uses to pick a seq
>> for outgoing pure ACKs looks like:
>>
>> /* SND.NXT, if window was not shrunk.
>>  * If window has been shrunk, what should we make? It is not clear at
>> all.
>>  * Using SND.UNA we will fail to open window, SND.NXT is out of
>> window. :-(
>>  * Anything in between SND.UNA...SND.UNA+SND.WND also can be already
>>  * invalid. OK, let's make this for now:
>>  */
>> static inline __u32 tcp_acceptable_seq(const struct sock *sk)
>> {
>>         const struct tcp_sock *tp = tcp_sk(sk);
>>
>>         if (!before(tcp_wnd_end(tp), tp->snd_nxt))
>>                 return tp->snd_nxt;
>>         else
>>                 return tcp_wnd_end(tp);
>> }
>>
>> neal
>
>


From nobody Mon Aug 22 01:14:46 2016
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 06AEB12B00D; Mon, 22 Aug 2016 01:14:45 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: 0.879
X-Spam-Level: 
X-Spam-Status: No, score=0.879 tagged_above=-999 required=5 tests=[BAYES_50=0.8, RCVD_IN_DNSWL_LOW=-0.7, SPF_NEUTRAL=0.779] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Ou7yO7j-HzNY; Mon, 22 Aug 2016 01:14:42 -0700 (PDT)
Received: from julia1.inet.fi (mta-out1.inet.fi [62.71.2.232]) by ietfa.amsl.com (Postfix) with ESMTP id 9302512B014; Mon, 22 Aug 2016 01:14:42 -0700 (PDT)
Received: from t40700-la020.lan (80.223.92.46) by julia1.inet.fi (9.0.002.03-2-gbe5d057) (authenticated as saropa-1) id 5782991C00FD3795; Mon, 22 Aug 2016 11:13:35 +0300
Content-Type: text/plain; charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\))
From: Pasi Sarolahti 
In-Reply-To: <57B482B9.5090205@kuehlewind.net>
Date: Mon, 22 Aug 2016 11:14:39 +0300
Content-Transfer-Encoding: quoted-printable
Message-Id: <1411D263-8B35-4335-B1F5-799B7544209C@iki.fi>
References: <57B482B9.5090205@kuehlewind.net>
To: =?utf-8?Q?Mirja_K=C3=BChlewind?= 
X-Mailer: Apple Mail (2.3124)
Archived-At: 
Cc: "tcpm@ietf.org Extensions" , "tcpm-chairs@ietf.org" 
Subject: Re: [tcpm] Personal change in chairing
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Mon, 22 Aug 2016 08:14:45 -0000

Thanks, Mirja, and thanks to everyone for collaboration! I think Michael =
is an excellent choice for a new TCPM co-chair.

Hope to see many of you in future some time!

- Pasi


> On 17 Aug 2016, at 18:28, Mirja K=C3=BChlewind  =
wrote:
>=20
> Hi all,
>=20
> I have to announce a personal change in the tcpm chair position:
>=20
> Pasi Sarolahti is stepping down due to time restrictions. Pasi, thanks =
a lot for your work and engagement as chair! Your did a great job and we =
hope to still see you as often as possible as contributor in the working =
group!
>=20
> Michael T=C3=BCxen is starting as a new tcpm chair. Micheal, great to =
have you on broad! We look forward to work with you!
>=20
> Mirja, as responsible AD


From nobody Mon Aug 22 23:33:14 2016
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 76BC112B035; Mon, 22 Aug 2016 23:33:12 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -3.149
X-Spam-Level: 
X-Spam-Status: No, score=-3.149 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RP_MATCHES_RCVD=-0.548, SPF_HELO_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kgKYHqRuTwW8; Mon, 22 Aug 2016 23:33:10 -0700 (PDT)
Received: from drew.franken.de (drew.ipv6.franken.de [IPv6:2001:638:a02:a001:20e:cff:fe4a:feaa]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 27B7812D520; Mon, 22 Aug 2016 23:33:09 -0700 (PDT)
Received: from [192.168.1.7] (p4FE30C28.dip0.t-ipconnect.de [79.227.12.40]) (Authenticated sender: macmic) by mail-n.franken.de (Postfix) with ESMTPSA id 74182721E281A; Tue, 23 Aug 2016 08:33:06 +0200 (CEST)
Content-Type: text/plain; charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\))
From: Michael Tuexen 
In-Reply-To: <1411D263-8B35-4335-B1F5-799B7544209C@iki.fi>
Date: Tue, 23 Aug 2016 08:33:06 +0200
Content-Transfer-Encoding: quoted-printable
Message-Id: <581CA397-2A6D-4B2C-925F-FFE898DDDFB7@lurchi.franken.de>
References: <57B482B9.5090205@kuehlewind.net> <1411D263-8B35-4335-B1F5-799B7544209C@iki.fi>
To: Pasi Sarolahti 
X-Mailer: Apple Mail (2.3124)
Archived-At: 
Cc: "tcpm@ietf.org Extensions" , =?utf-8?Q?Mirja_K=C3=BChlewind?= , "tcpm-chairs@ietf.org" 
Subject: Re: [tcpm] Personal change in chairing
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Tue, 23 Aug 2016 06:33:12 -0000

> On 22 Aug 2016, at 10:14, Pasi Sarolahti  =
wrote:
>=20
> Thanks, Mirja, and thanks to everyone for collaboration! I think =
Michael is an excellent choice for a new TCPM co-chair.
>=20
> Hope to see many of you in future some time!
Hi Pasi,

thank you very much for serving as WG chair! I hope to see you soon.

Best regards
Michael
>=20
> - Pasi
>=20
>=20
>> On 17 Aug 2016, at 18:28, Mirja K=C3=BChlewind  =
wrote:
>>=20
>> Hi all,
>>=20
>> I have to announce a personal change in the tcpm chair position:
>>=20
>> Pasi Sarolahti is stepping down due to time restrictions. Pasi, =
thanks a lot for your work and engagement as chair! Your did a great job =
and we hope to still see you as often as possible as contributor in the =
working group!
>>=20
>> Michael T=C3=BCxen is starting as a new tcpm chair. Micheal, great to =
have you on broad! We look forward to work with you!
>>=20
>> Mirja, as responsible AD
>=20
> _______________________________________________
> tcpm mailing list
> tcpm@ietf.org
> https://www.ietf.org/mailman/listinfo/tcpm


From nobody Tue Aug 23 00:06:55 2016
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id EADA812B041 for ; Tue, 23 Aug 2016 00:06:53 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.448
X-Spam-Level: 
X-Spam-Status: No, score=-2.448 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RP_MATCHES_RCVD=-0.548, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id M7HEVR4P9K21 for ; Tue, 23 Aug 2016 00:06:51 -0700 (PDT)
Received: from mail.sfc.wide.ad.jp (shonan.sfc.wide.ad.jp [IPv6:2001:200:0:8803::53]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id AA9EB12B038 for ; Tue, 23 Aug 2016 00:06:51 -0700 (PDT)
Received: from mail-ua0-f173.google.com (mail-ua0-f173.google.com [209.85.217.173]) by mail.sfc.wide.ad.jp (Postfix) with ESMTPSA id 296C1278386 for ; Tue, 23 Aug 2016 16:06:50 +0900 (JST)
Received: by mail-ua0-f173.google.com with SMTP id n59so229857128uan.2 for ; Tue, 23 Aug 2016 00:06:50 -0700 (PDT)
X-Gm-Message-State: AEkoouuTj71P3knoRh8l3GvupOfUqFVthJfUGfPnbKdaJ2LTJMtwva1aCNVcDJ+wQp442hy/UsHKnXZ5EHGQQA==
X-Received: by 10.159.36.108 with SMTP id 99mr10792287uaq.79.1471936008614; Tue, 23 Aug 2016 00:06:48 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.159.37.1 with HTTP; Tue, 23 Aug 2016 00:06:48 -0700 (PDT)
In-Reply-To: 
References:  <7070553C-65D1-46EE-95F4-DAE82E1F5A5E@weston.borman.com>  <2D524A8D-A5CA-45A6-B94D-FA1DA0CEE609@weston.borman.com>     
From: Yoshifumi Nishida 
Date: Tue, 23 Aug 2016 00:06:48 -0700
X-Gmail-Original-Message-ID: 
Message-ID: 
To: Neal Cardwell 
Content-Type: multipart/alternative; boundary=001a113d1268ed3477053ab7ced2
Archived-At: 
Cc: Kobby Carmona , "tcpm@ietf.org" , David Borman 
Subject: Re: [tcpm] Possible deadlock scenario with retransmission on both sides at the same time
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Tue, 23 Aug 2016 07:06:54 -0000

--001a113d1268ed3477053ab7ced2
Content-Type: text/plain; charset=UTF-8

Hi Neal,

Oh. I see. Thanks for the explanation.
I'd like to think about it a bit more.
Thanks,
--
Yoshi

On Sun, Aug 21, 2016 at 12:44 PM, Neal Cardwell 
wrote:

> I am pretty sure Linux does not have the issue Kobby pointed out in this
> thread.
>
> At a high level Linux should be OK because it follows the principle
> David Borman laid out in his August 16 email: "ACK-only packets should
> be sent with the largest in-window sequence number that has ever been
> sent."
>
> Linux obeys that principle by using tp->snd_nxt to store the largest
> sequence number that has ever been sent, and having
> tcp_acceptable_seq() use tp->snd_nxt but clamp the outgoing sequence
> number to make sure it is in-window. To be able to do this, in Linux,
> the sender does not rewind tp->snd_nxt on retransmissions.
>
> neal
>
> On Sun, Aug 21, 2016 at 1:25 PM, Yoshifumi Nishida
>  wrote:
> > Hi Neal,
> >
> > Thanks for the info.
> > So, it seems to me that the linux code has the issue Kobby pointed out.
> > Or, am I missing something?
> > --
> > Yoshi
> >
> >
> > On Mon, Aug 15, 2016 at 6:18 PM, Neal Cardwell 
> wrote:
> >>
> >> On Mon, Aug 15, 2016 at 6:39 PM, Yoshifumi Nishida
> >>  wrote:
> >> > Hello,
> >> > I personally think this is an interesting corner case for discussion.
> >> > It looks a minor one, but I'm not not very sure if we can leave it for
> >> > each
> >> > implementation.
> >> > I also guess a question would be if the BSD's fix is the best way for
> >> > the
> >> > issue.
> >>
> >> Yes, I agree this is an interesting case for discussion.
> >>
> >> FWIW, as a point of comparison for discussion, Linux's approach is a
> >> little different: in Linux, the sender does not rewind SND.NXT on
> >> retransmissions (RTO or Fast Recovery). Then the sender usually uses
> >> SND.NXT for the seq field of outgoing pure ACKs. I say "usually"
> >> because the Linux code has some code to deal with the case where the
> >> receiver has withdrawn the receive window, so that SND.NXT is now
> >> beyond the receive window. The code tcp_send_ack() uses to pick a seq
> >> for outgoing pure ACKs looks like:
> >>
> >> /* SND.NXT, if window was not shrunk.
> >>  * If window has been shrunk, what should we make? It is not clear at
> >> all.
> >>  * Using SND.UNA we will fail to open window, SND.NXT is out of
> >> window. :-(
> >>  * Anything in between SND.UNA...SND.UNA+SND.WND also can be already
> >>  * invalid. OK, let's make this for now:
> >>  */
> >> static inline __u32 tcp_acceptable_seq(const struct sock *sk)
> >> {
> >>         const struct tcp_sock *tp = tcp_sk(sk);
> >>
> >>         if (!before(tcp_wnd_end(tp), tp->snd_nxt))
> >>                 return tp->snd_nxt;
> >>         else
> >>                 return tcp_wnd_end(tp);
> >> }
> >>
> >> neal
> >
> >
>

--001a113d1268ed3477053ab7ced2
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable


Hi Neal,

Oh. I see. Thanks for the expl=
anation.
I'd like to think about it a bit more.
Tha=
nks,
--
Yoshi

On Sun, Aug 21, 2016 at 12:44 PM, Neal Cardwell <ncardwell@google.com> wrote:
I am pretty sure Linux does not have the issue Kobby pointed out in thi=
s thread.



At a high level Linux should be OK because it follows the principle

David Borman laid out in his August 16 email: "ACK-only packets should=


be sent with the largest in-window sequence number that has ever been

sent."



Linux obeys that principle by using tp->snd_nxt to store the largest

sequence number that has ever been sent, and having

tcp_acceptable_seq() use tp->snd_nxt but clamp the outgoing sequence

number to make sure it is in-window. To be able to do this, in Linux,

the sender does not rewind tp->snd_nxt on retransmissions.



neal



On Sun, Aug 21, 2016 at 1:25 PM, Yoshifumi Nishida


<n=
ishida@sfc.wide.ad.jp> wrote:

> Hi Neal,

>

> Thanks for the info.

> So, it seems to me that the linux code has the issue Kobby pointed out=
.

> Or, am I missing something?

> --

> Yoshi

>

>

> On Mon, Aug 15, 2016 at 6:18 PM, Neal Cardwell <ncardwell@google.com> wrote:<=
br>
>>

>> On Mon, Aug 15, 2016 at 6:39 PM, Yoshifumi Nishida

>> <ni=
shida@sfc.wide.ad.jp> wrote:

>> > Hello,

>> > I personally think this is an interesting corner case for dis=
cussion.

>> > It looks a minor one, but I'm not not very sure if we can=
 leave it for

>> > each

>> > implementation.

>> > I also guess a question would be if the BSD's fix is the =
best way for

>> > the

>> > issue.

>>

>> Yes, I agree this is an interesting case for discussion.

>>

>> FWIW, as a point of comparison for discussion, Linux's approac=
h is a

>> little different: in Linux, the sender does not rewind SND.NXT on<=
br>
>> retransmissions (RTO or Fast Recovery). Then the sender usually us=
es

>> SND.NXT for the seq field of outgoing pure ACKs. I say "usual=
ly"

>> because the Linux code has some code to deal with the case where t=
he

>> receiver has withdrawn the receive window, so that SND.NXT is now<=
br>
>> beyond the receive window. The code tcp_send_ack() uses to pick a =
seq

>> for outgoing pure ACKs looks like:

>>

>> /* SND.NXT, if window was not shrunk.

>>=C2=A0 * If window has been shrunk, what should we make? It is not =
clear at

>> all.

>>=C2=A0 * Using SND.UNA we will fail to open window, SND.NXT is out =
of

>> window. :-(

>>=C2=A0 * Anything in between SND.UNA...SND.UNA+SND.WND also can be =
already

>>=C2=A0 * invalid. OK, let's make this for now:

>>=C2=A0 */

>> static inline __u32 tcp_acceptable_seq(const struct sock *sk)

>> {

>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0const struct tcp_sock *tp =3D tcp=
_sk(sk);

>>

>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if (!before(tcp_wnd_end(tp), tp-&=
gt;snd_nxt))

>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0retur=
n tp->snd_nxt;

>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0else

>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0retur=
n tcp_wnd_end(tp);

>> }

>>

>> neal

>

>





--001a113d1268ed3477053ab7ced2--


From nobody Wed Aug 24 16:21:09 2016
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id BD0E212D555 for ; Wed, 24 Aug 2016 16:21:08 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.45
X-Spam-Level: 
X-Spam-Status: No, score=-2.45 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RP_MATCHES_RCVD=-0.548, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id LYlaIA0Aec-R for ; Wed, 24 Aug 2016 16:21:07 -0700 (PDT)
Received: from kuehlewind.net (kuehlewind.net [83.169.45.111]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id DC9EF12D144 for ; Wed, 24 Aug 2016 16:21:05 -0700 (PDT)
Received: (qmail 29970 invoked from network); 25 Aug 2016 01:21:01 +0200
Received: from p5dec2dcf.dip0.t-ipconnect.de (HELO ?192.168.178.33?) (93.236.45.207) by kuehlewind.net with ESMTPSA (DHE-RSA-AES256-SHA encrypted, authenticated);  25 Aug 2016 01:21:01 +0200
From: "Mirja Kuehlewind (IETF)" 
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Date: Thu, 25 Aug 2016 01:21:02 +0200
References: <20160820132955.49560B80EE3@rfc-editor.org>
To: "tcpm@ietf.org Extensions" 
Message-Id: <471798EE-293E-44E6-A7A3-EAB2ACF6CAEC@kuehlewind.net>
Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\))
X-Mailer: Apple Mail (2.3124)
Archived-At: 
Subject: [tcpm] Fwd: [Technical Errata Reported] RFC0793 (4785)
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Wed, 24 Aug 2016 23:21:09 -0000

FYI.

> Anfang der weitergeleiteten Nachricht:
> 
> Von: RFC Errata System 
> Betreff: [Technical Errata Reported] RFC0793 (4785)
> Datum: 20. August 2016 um 15:29:55 MESZ
> An: iesg@ietf.org
> Kopie: sanjeev.ranot.81@gmail.com, rfc-editor@rfc-editor.org
> 
> The following errata report has been submitted for RFC0793,
> "Transmission Control Protocol".
> 
> --------------------------------------
> You may review the report below and at:
> http://www.rfc-editor.org/errata_search.php?rfc=793&eid=4785
> 
> --------------------------------------
> Type: Technical
> Reported by: Sanjeev Ranot 
> 
> Section: 3.9  p. 72
> 
> Original Text
> -------------
> 
> 
> If the ACK is a duplicate
> (SEG.ACK < SND.UNA), it can be ignored.
> 
> Corrected Text
> --------------
> If the ACK is a duplicate
> (SEG.ACK =< SND.UNA), it can be ignored except when equality is met, 
> then window should be updated. This can happen when there are 
> segments in flight but the receiver shrinks its RCV.BUF to drop all 
> of them and send an ACK carrying zero window update. Upon its 
> arrival at the sending TCP, condition SND.UNA = SEG.ACK is met and 
> we must update SND.WND <- 0. Then sender starts persist timer for 
> sending zero-window probes [Ref. RFC 1122 Section 4.2.2.17, page 92]
> 
> 
> 
> 
> 
> Notes
> -----
> The condition is corrected as Duplicate ACK in 
> RFC 1122, Section 4.2.2.20 (g) p. 94. Accordingly old text must be 
> modified and new text should also be present to support the corrected 
> condition in RFC 793. 
> 
> This is one case where duplicate acknowledgment cannot be ignored. i.e. 
> when SEG.ACK == SND.UNA and advertised window in the incoming ACK is 0
> in which case sender needs to :
> 1. update window
> 2. start persist timer
> 3. send zero window probe segments. 
> 
> 
> Note:
> Such ACKs should not be called as duplicates as it fails condition (e) 
> of definition of "DUPLICATE ACKNOWLEDGMENT" in Ref 5681 Section 2, pg 4
> 
> Instructions:
> -------------
> This erratum is currently posted as "Reported". If necessary, please
> use "Reply All" to discuss whether it should be verified or
> rejected. When a decision is reached, the verifying party (IESG)
> can log in to change the status and edit the report, if necessary. 
> 
> --------------------------------------
> RFC0793 (no draft string recorded)
> --------------------------------------
> Title               : Transmission Control Protocol
> Publication Date    : September 1981
> Author(s)           : J. Postel
> Category            : INTERNET STANDARD
> Source              : Legacy
> Area                : Legacy
> Stream              : IETF
> Verifying Party     : IESG
> 


From nobody Thu Aug 25 06:46:44 2016
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 23B3512B068 for ; Thu, 25 Aug 2016 06:46:43 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.6
X-Spam-Level: 
X-Spam-Status: No, score=-2.6 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_LOW=-0.7] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=mti-systems-com.20150623.gappssmtp.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id QFChULXoylMn for ; Thu, 25 Aug 2016 06:46:41 -0700 (PDT)
Received: from mail-it0-x22a.google.com (mail-it0-x22a.google.com [IPv6:2607:f8b0:4001:c0b::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 389F812D74D for ; Thu, 25 Aug 2016 06:46:41 -0700 (PDT)
Received: by mail-it0-x22a.google.com with SMTP id f6so32826649ith.0 for ; Thu, 25 Aug 2016 06:46:41 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mti-systems-com.20150623.gappssmtp.com; s=20150623; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to:content-transfer-encoding; bh=gpxEX9d7xI6M/tZgXDgOEOanOUqMgyfPRHAa9PWPhSg=; b=WOOFNL+yJETqDG8q2ffw5OLvYEyaSr/t4I/8DdQlFSkM8Dhy5TbQ/+w5ddjeGSycUy br++kBstnv/zTQLzylp6UltZkX+BTCht47hJMHOKBbLOfY1+qaKQtx+mCRHndqxDQMtM QAHIBy05C8XA6rW26VQGsYn07WEIcXNCOtk0ocO1GYW4F1+iKEFLxd9Yw0KAu+X+2054 VorMNljifHuWKs5XBBHC3w/akpNDgLVQwpnDwEjlYBMeXBhMUIeHkOhJO3uhd2OIPQyC 6/H+rT13tKCer9YfKDTt7xedi1KZtfbDkxUl907CdVaiNODxl03r7bvqsAiijxqlhoGt BNhw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=gpxEX9d7xI6M/tZgXDgOEOanOUqMgyfPRHAa9PWPhSg=; b=MMZYgPsqU9UHP720yL4mHP+aU9ZVl/sKeq+YOa+TvKbWVE/OgRYxtDyYZmV656+lR2 29V68VZtQ4EyCsrhwWOuhMXEQrzlTXTkL1d+ivvErZ/cGODV++0ofNZgRJ6Cw2o+2Uj9 AQWkVzBZpcjE2WdqXvZkStcJjVrr3z9F1ePxrJu82fEHf/A4x/RbMkEybdeX4CAVIkNQ hE/Ku26LgTknLCIvj0YmDD/7i+Q/xgU2APIVAht7RFt8s1d1nBjHZV3BP7RRSDufB+uZ cobd8Svz8Ftng6he/AZvI0+l/BQcxL7vY5PshQ4hdzKv+RkL+L+M0qRQHDxCDDI12n9W YmVA==
X-Gm-Message-State: AEkoout0OOdv4TJ0MzU+bVxwaUdrE/Pp61E9cUz3FyusHhaLnYfRr/+KvSGsG1mcf1KIyw==
X-Received: by 10.36.23.11 with SMTP id 11mr5030118ith.26.1472132800095; Thu, 25 Aug 2016 06:46:40 -0700 (PDT)
Received: from [192.168.1.104] (cpe-76-188-215-129.neo.res.rr.com. [76.188.215.129]) by smtp.gmail.com with ESMTPSA id s185sm5487786ita.21.2016.08.25.06.46.39 for  (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 25 Aug 2016 06:46:39 -0700 (PDT)
To: tcpm@ietf.org
References: <20160820132955.49560B80EE3@rfc-editor.org> <471798EE-293E-44E6-A7A3-EAB2ACF6CAEC@kuehlewind.net>
From: Wesley Eddy 
Message-ID: <5f1e73bd-2594-5146-3731-dd77590cf9fb@mti-systems.com>
Date: Thu, 25 Aug 2016 09:46:36 -0400
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0
MIME-Version: 1.0
In-Reply-To: <471798EE-293E-44E6-A7A3-EAB2ACF6CAEC@kuehlewind.net>
Content-Type: text/plain; charset=windows-1252; format=flowed
Content-Transfer-Encoding: 7bit
Archived-At: 
Subject: Re: [tcpm] Fwd: [Technical Errata Reported] RFC0793 (4785)
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Thu, 25 Aug 2016 13:46:43 -0000

As noted, RFC1122 already has a correction on this.  The fix from 1122 
is in the 793bis draft.


Original 793 text:

           If the ACK is a duplicate
           (SEG.ACK < SND.UNA), it can be ignored.  If the ACK acks
           something not yet sent (SEG.ACK > SND.NXT) then send an ACK,
           drop the segment, and return.

           If SND.UNA < SEG.ACK =< SND.NXT, the send window should be
           updated.  If (SND.WL1 < SEG.SEQ or (SND.WL1 = SEG.SEQ and
           SND.WL2 =< SEG.ACK)), set SND.WND <- SEG.WND, set
           SND.WL1 <- SEG.SEQ, and set SND.WL2 <- SEG.ACK.


Text in 793bis:

           If the ACK is a duplicate (SEG.ACK =<
           SND.UNA), it can be ignored.  If the ACK acks
           something not yet sent (SEG.ACK > SND.NXT) then send
           an ACK, drop the segment, and return.

           If SND.UNA =< SEG.ACK =< SND.NXT, the send window
           should be updated.  If (SND.WL1 < SEG.SEQ or (SND.WL1
           = SEG.SEQ and SND.WL2 =< SEG.ACK)), set SND.WND <-
           SEG.WND, set SND.WL1 <- SEG.SEQ, and set SND.WL2 <-
           SEG.ACK.





On 8/24/2016 7:21 PM, Mirja Kuehlewind (IETF) wrote:
> FYI.
>
>> Anfang der weitergeleiteten Nachricht:
>>
>> Von: RFC Errata System 
>> Betreff: [Technical Errata Reported] RFC0793 (4785)
>> Datum: 20. August 2016 um 15:29:55 MESZ
>> An: iesg@ietf.org
>> Kopie: sanjeev.ranot.81@gmail.com, rfc-editor@rfc-editor.org
>>
>> The following errata report has been submitted for RFC0793,
>> "Transmission Control Protocol".
>>
>> --------------------------------------
>> You may review the report below and at:
>> http://www.rfc-editor.org/errata_search.php?rfc=793&eid=4785
>>
>> --------------------------------------
>> Type: Technical
>> Reported by: Sanjeev Ranot 
>>
>> Section: 3.9  p. 72
>>
>> Original Text
>> -------------
>>
>>
>> If the ACK is a duplicate
>> (SEG.ACK < SND.UNA), it can be ignored.
>>
>> Corrected Text
>> --------------
>> If the ACK is a duplicate
>> (SEG.ACK =< SND.UNA), it can be ignored except when equality is met,
>> then window should be updated. This can happen when there are
>> segments in flight but the receiver shrinks its RCV.BUF to drop all
>> of them and send an ACK carrying zero window update. Upon its
>> arrival at the sending TCP, condition SND.UNA = SEG.ACK is met and
>> we must update SND.WND <- 0. Then sender starts persist timer for
>> sending zero-window probes [Ref. RFC 1122 Section 4.2.2.17, page 92]
>>
>>
>>
>>
>>
>> Notes
>> -----
>> The condition is corrected as Duplicate ACK in
>> RFC 1122, Section 4.2.2.20 (g) p. 94. Accordingly old text must be
>> modified and new text should also be present to support the corrected
>> condition in RFC 793.
>>
>> This is one case where duplicate acknowledgment cannot be ignored. i.e.
>> when SEG.ACK == SND.UNA and advertised window in the incoming ACK is 0
>> in which case sender needs to :
>> 1. update window
>> 2. start persist timer
>> 3. send zero window probe segments.
>>
>>
>> Note:
>> Such ACKs should not be called as duplicates as it fails condition (e)
>> of definition of "DUPLICATE ACKNOWLEDGMENT" in Ref 5681 Section 2, pg 4
>>
>> Instructions:
>> -------------
>> This erratum is currently posted as "Reported". If necessary, please
>> use "Reply All" to discuss whether it should be verified or
>> rejected. When a decision is reached, the verifying party (IESG)
>> can log in to change the status and edit the report, if necessary.
>>
>> --------------------------------------
>> RFC0793 (no draft string recorded)
>> --------------------------------------
>> Title               : Transmission Control Protocol
>> Publication Date    : September 1981
>> Author(s)           : J. Postel
>> Category            : INTERNET STANDARD
>> Source              : Legacy
>> Area                : Legacy
>> Stream              : IETF
>> Verifying Party     : IESG
>>
> _______________________________________________
> tcpm mailing list
> tcpm@ietf.org
> https://www.ietf.org/mailman/listinfo/tcpm


From Cheng.Cui@netapp.com  Wed Aug 24 14:06:30 2016
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 41EC412D66B; Wed, 24 Aug 2016 14:06:30 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -7.469
X-Spam-Level: 
X-Spam-Status: No, score=-7.469 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-0.548, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2FXrabv7sKiy; Wed, 24 Aug 2016 14:06:28 -0700 (PDT)
Received: from mx142.netapp.com (mx142.netapp.com [216.240.21.19]) (using TLSv1.2 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 7D8C212D124; Wed, 24 Aug 2016 14:06:27 -0700 (PDT)
X-IronPort-AV: E=Sophos;i="5.28,572,1464678000"; d="scan'208";a="133202750"
Received: from hioexcmbx07-prd.hq.netapp.com ([10.122.105.40]) by mx142-out.netapp.com with ESMTP; 24 Aug 2016 14:06:12 -0700
Received: from HIOEXCMBX03-PRD.hq.netapp.com (10.122.105.36) by hioexcmbx07-prd.hq.netapp.com (10.122.105.40) with Microsoft SMTP Server (TLS) id 15.0.1210.3; Wed, 24 Aug 2016 14:06:11 -0700
Received: from HIOEXCMBX03-PRD.hq.netapp.com ([::1]) by hioexcmbx03-prd.hq.netapp.com ([fe80::a009:cb7a:e519:7347%21]) with mapi id 15.00.1210.000; Wed, 24 Aug 2016 14:06:10 -0700
From: "Cui, Cheng" 
To: "david.borman@windriver.com" , "dab@weston.borman.com" , "david.borman@quantum.com" 
Thread-Topic: question about if a recent Linux patch is compliant with RFC7323 on window scaling
Thread-Index: AQHR/ktWq33ah5ClWE2yMFWt4EpCvg==
Date: Wed, 24 Aug 2016 21:06:09 +0000
Message-ID: 
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-ms-exchange-messagesentrepresentingtype: 1
x-ms-exchange-transport-fromentityheader: Hosted
x-originating-ip: [10.122.56.79]
Content-Type: text/plain; charset="iso-8859-1"
Content-ID: 
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
Archived-At: 
X-Mailman-Approved-At: Thu, 25 Aug 2016 08:34:57 -0700
Cc: "tcpm@ietf.org" , "tsvwg@ietf.org" 
Subject: [tcpm] question about if a recent Linux patch is compliant with RFC7323 on window scaling
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Wed, 24 Aug 2016 23:26:20 -0000

Hello David,

I hope this email could reach you well, because I found related
discussions about this topic on window scaling and the case of window
shrinking (or retraction or loss of precision). And I try to make this
question simple.

There is a recent Linux patch at receiver side to round-up advertised
window due to precision loss of window scaling. It reaches my attention
because the same problem could also happen between a pair of Linux and
FreeBSD nodes, and I am not aware of any similar patch in FreeBSD yet.

The Linux patch is this:
http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=
=3D6
07bfbf2d55dd1cfe5368b41c2a81a8c9ccf4723

And I quote some description of the Linux patch below:
> If the sender uses up the entire window before it is shrunk, this can
>have
> chaotic effects on the connection. When sending ACKs,
>tcp_acceptable_seq()
> will notice that the window has been shrunk since tcp_wnd_end() is before
> tp->snd_nxt, which makes it choose tcp_wnd_end() as sequence number.
> This will fail the receivers checks in tcp_sequence() however since it
> is before it's tp->rcv_wup, making it respond with a dupack.


If the Linux sender=B9s behavior is right ("ACK-only packets should be sent
with the largest in-window sequence number that has ever been sent." ref:
https://www.ietf.org/mail-archive/web/tcpm/current/msg10512.html), it
actually chooses "tp->snd_una+tp->snd_wnd" (tcp_wnd_end()) instead of
tp->snd_nxt, as it thought tp->snd_nxt is out of window, in case of
precision loss which made the receiver=B9s advertise-window smaller.
However, the RFC7323 explicitly specifies the receiver should right shift
the exact scale factor, without mentioning any further round-up:

https://tools.ietf.org/html/rfc7323#page-9
>    o  The window field (SEG.WND) of every outgoing segment, with the
>       exception of  segments, MUST be right-shifted by
>       Rcv.Wind.Shift bits:
>=20
>                     SEG.WND =3D RCV.WND >> Rcv.Wind.Shift

And also if I am understanding correctly, RFC7323 page 10 (2.4. Addressing
Window Retraction) specifies it is the sender=B9s responsibility to handle
the sequence number out of window:
https://tools.ietf.org/html/rfc7323#page-10
>    4)  On first retransmission, or if the sequence number is out of
>        window by less than 2^Rcv.Wind.Shift, then do normal
>        retransmission(s) without regard to the receiver window as long
>        as the original segment was in window when it was sent.
>=20
>    5)  Subsequent retransmissions MAY only be sent if they are within
>        the window announced by the most recent .

In your discussion of the link below, if I am understanding correctly, you
were also in favor of "Yes. It is ok to have more receive buffer space
available than you advertise in the window, but not ok to advertise a
larger window than you are willing to accept." So I think you were not in
favor of rounding-up the advertise-window.

https://www.ietf.org/mail-archive/web/tsvwg/current/msg06475.html


So my question is: Is the Linux patch (on receiver side to round-up the
advertise-window) RFC compliant? Or is it just the right thing (vendor
specific) and other systems like FreeBSD have to follow up, so that their
talks can have no problem?

Thanks and apologize in advance if I did not do enough research,
--Cheng Cui




From sanjeev.ranot.81@gmail.com  Thu Aug 25 01:29:33 2016
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 219D412D775 for ; Thu, 25 Aug 2016 01:29:33 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.749
X-Spam-Level: 
X-Spam-Status: No, score=-1.749 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=no autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gHh1zeAOqYQw for ; Thu, 25 Aug 2016 01:29:32 -0700 (PDT)
Received: from mail-ua0-x22a.google.com (mail-ua0-x22a.google.com [IPv6:2607:f8b0:400c:c08::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 05AAA12B038 for ; Thu, 25 Aug 2016 01:29:32 -0700 (PDT)
Received: by mail-ua0-x22a.google.com with SMTP id l94so8992564ual.0 for ; Thu, 25 Aug 2016 01:29:31 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;  h=mime-version:from:date:message-id:subject:to; bh=j4++ejeE3FfLVNK+0U+ygQpz1OBPr01H9C9kEu81zGE=; b=JcF87rDESclQC/Mg5JMPEjIGoAj7xoSNdyRC00G/3viCAR+EZWNKLZYlbdPGFIdDgl pc/nwB5UX2BaEAgRVYubGPcvW5Lh7qatLBowWVGG336N9TsB+6/FGzMdqgv0/ci4Kpp5 r0uBpELIFFzvFPTOi9T1IKh29LaXCNisTcH865cij5WrnH58PmegpW8ZAE/o3FT/CC9V fDEcbTj4+AU3uqsmZ2QNKDj2TTgQt5MlLVdRKRTfoNU0ft5h95bWe1v20/ZHFpvxBi1x d9MfZdiER02EowQyUe4sXowQLl3qps9h8t6OlS/CKhfsroblNIoKfLkj/4PeX6migPWG Z/LQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=j4++ejeE3FfLVNK+0U+ygQpz1OBPr01H9C9kEu81zGE=; b=T3SnBeJM5/GK5DOR/+YaXT28PJfOj6y/c3Y6/2XoHtl/TFMUuQ289ydUjYvpiyysji 5dgzKYObw3ISYkOCy0qXAhtaTRyJJynI5t6QgihCwc37CJfrzIE7U7AX9pIHKTLSdZkq iOGQA9w1sc3X4UtC62HQcvN2lSPtPGOt09VDV/4QpN/p2/BKCo3M0X/wOGgk1+Rgxx9d YqTGXb0arwli5j24471y50QEIEW5PWMLdpZ3zNHzYZuFhXDj3Sm1ElnF98zfTpe03KGZ p15ZG+p4HZMb9CFj5vWtLCrwfKfL1Tbqzd2YtJKZi2JusZ7ipJ9JR3tu7aeKn5j4Uw2z kHUQ==
X-Gm-Message-State: AE9vXwMMXLy1qzPOPWMwxdH0nV7noh5YiTBgj7Xpi0bc0V5a5sCbl0MpNk/VKaaw3smklJx4zZn9a0mrnaTsAw==
X-Received: by 10.176.69.228 with SMTP id u91mr4285963uau.144.1472113770994; Thu, 25 Aug 2016 01:29:30 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.176.4.199 with HTTP; Thu, 25 Aug 2016 01:29:30 -0700 (PDT)
From: Sanjeev Ranot 
Date: Thu, 25 Aug 2016 13:59:30 +0530
Message-ID: 
To: tcpm@ietf.org
Content-Type: multipart/alternative; boundary=94eb2c11aff463a473053ae132f7
Archived-At: 
X-Mailman-Approved-At: Thu, 25 Aug 2016 08:35:04 -0700
Subject: [tcpm] (ref. RFC 7323) Increase of shift count from 14 to 15.
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Thu, 25 Aug 2016 08:36:19 -0000

--94eb2c11aff463a473053ae132f7
Content-Type: text/plain; charset=UTF-8

Hi,

please can the team explain why we don't use 15 as a shift count instead
of 14 as mentioned in RFC 7323. Is it because of machines not supporting
unsigned storage or is it more than sufficient guard for denying old or
unwanted segment retransmissions from the previous window.

My understanding:
We know that 2^32 sequence counter (sequence counter is the the sequence
field in TCP header) has valid sequence numbers from 0 to 4294967295. To
determine if a data segment is "old" or "new" it sets the window size to
floor((2^32-1 )/2) +1 since sender's and receiver's window can be out of
phase by at-most 1 window.
1st part has valid range of sequence numbers from = 0 to 2147483647 and
since its contained inside size of of 2^32 therefore we get another valid
range of sequence numbers from 2147483648 to 4294967295. So if the sender
has sent all its packets within the window from 0 to 2147483647 and receiver
has accepted all the packets and sent cumulative acks as well for all of
these then it advances RCV.NXT to start of next maximum available range
of sequence numbers.
Upon receiving all cumulative acks the sender also advance to next set of
available sequence numbers. Otherwise, if considering the scenario when
spurious retransmission occurs or another scenario where all cumulative acks
are lost and retransmission timeout or another scenario when cumulative ack
carrying widow update information is lost, then in such cases the sender
will retransmit segment with sequence number that will fail receiver's
segment acceptability test.

If there is a reason behind this then please cite in response. Also please
tell if such change is valid then do we need to report it as correction or
as a
change for RFC 7323.

Thanks & Regards,
Sanjeev Ranot

--94eb2c11aff463a473053ae132f7
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable


Hi,

please can the team explain why we don't us=
e 15 as a shift count instead
of 14 as mentioned in RFC 7323. Is it beca=
use of machines not supporting
unsigned storage or is it more than suffi=
cient guard for denying old or
unwanted segment retransmissions from the=
 previous window.

My understanding:
We know that 2^32 sequence co=
unter (sequence counter is the the sequence
field in TCP header) has val=
id sequence numbers from 0 to 4294967295. To
determine if a data segment=
 is "old" or "new" it sets the window size to
floor(=
(2^32-1 )/2) +1 since sender's and receiver's window can be out of<=
br>phase by at-most 1 window.
1st part has valid range of sequence numbe=
rs from =3D 0 to 2147483647 and
since its contained inside size of of 2^=
32 therefore we get another valid
range of sequence numbers from 2147483=
648 to 4294967295. So if the sender
has sent all its packets within the =
window from 0 to 2147483647 and receiver
has accepted all the packets an=
d sent cumulative acks as well for all of
these then it advances RCV.NXT=
 to start of next maximum available range
of sequence numbers.
Upon r=
eceiving all cumulative acks the sender also advance to next set of 
ava=
ilable sequence numbers. Otherwise, if considering the scenario when 
sp=
urious retransmission occurs or another scenario where all cumulative acks<=
br>are lost and retransmission timeout or another scenario when cumulative =
ack
carrying widow update information is lost, then in such cases the se=
nder 
will retransmit segment with sequence number that will fail receiv=
er's 
segment acceptability test.

If there is a reason behind=
 this then please cite in response. Also please 
tell if such change is =
valid then do we need to report it as correction or as a 
change for RFC=
 7323.

Thanks & Regards,
Sanjeev Ranot


--94eb2c11aff463a473053ae132f7--


From nobody Thu Aug 25 12:01:38 2016
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1CB2B12D61A for ; Thu, 25 Aug 2016 12:01:36 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.448
X-Spam-Level: 
X-Spam-Status: No, score=-2.448 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RP_MATCHES_RCVD=-0.548, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HQKNF2P-kwZA for ; Thu, 25 Aug 2016 12:01:33 -0700 (PDT)
Received: from mail.sfc.wide.ad.jp (shonan.sfc.wide.ad.jp [203.178.142.130]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id B398712D5F9 for ; Thu, 25 Aug 2016 12:01:08 -0700 (PDT)
Received: from mail-ua0-f170.google.com (mail-ua0-f170.google.com [209.85.217.170]) by mail.sfc.wide.ad.jp (Postfix) with ESMTPSA id 6020F27829D for ; Fri, 26 Aug 2016 04:01:06 +0900 (JST)
Received: by mail-ua0-f170.google.com with SMTP id m60so59934525uam.3 for ; Thu, 25 Aug 2016 12:01:06 -0700 (PDT)
X-Gm-Message-State: AE9vXwPedhgwPmoBiTG21wm3trq3TZB76J7Ybw7xvbVHnVdeJNwx8Cx1fmLmqNNbSHkfRMoNlfwyDsn9nW3ybQ==
X-Received: by 10.176.3.199 with SMTP id 65mr6107400uau.21.1472151664730; Thu, 25 Aug 2016 12:01:04 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.159.37.1 with HTTP; Thu, 25 Aug 2016 12:01:04 -0700 (PDT)
In-Reply-To: 
References: 
From: Yoshifumi Nishida 
Date: Thu, 25 Aug 2016 12:01:04 -0700
X-Gmail-Original-Message-ID: 
Message-ID: 
To: Sanjeev Ranot 
Content-Type: multipart/alternative; boundary=94eb2c0568b00841f3053aea0559
Archived-At: 
Cc: "tcpm@ietf.org" 
Subject: Re: [tcpm] (ref. RFC 7323) Increase of shift count from 14 to 15.
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Thu, 25 Aug 2016 19:01:37 -0000

--94eb2c0568b00841f3053aea0559
Content-Type: text/plain; charset=UTF-8

Hello Sanjeev,

I wrote a draft about it before.
I am still working on it as I need to do some studies a bit more, but I
think it's probably ok to use 15 as a shift count.
A question would be how to migrate it even if using 15 is possible.
Or, we can publish this draft as reference information as we've seen this
question from time to time.

https://tools.ietf.org/html/draft-nishida-tcpm-maxwin-00

Thanks,
--
Yoshi

On Thu, Aug 25, 2016 at 1:29 AM, Sanjeev Ranot 
wrote:

> Hi,
>
> please can the team explain why we don't use 15 as a shift count instead
> of 14 as mentioned in RFC 7323. Is it because of machines not supporting
> unsigned storage or is it more than sufficient guard for denying old or
> unwanted segment retransmissions from the previous window.
>
> My understanding:
> We know that 2^32 sequence counter (sequence counter is the the sequence
> field in TCP header) has valid sequence numbers from 0 to 4294967295. To
> determine if a data segment is "old" or "new" it sets the window size to
> floor((2^32-1 )/2) +1 since sender's and receiver's window can be out of
> phase by at-most 1 window.
> 1st part has valid range of sequence numbers from = 0 to 2147483647 and
> since its contained inside size of of 2^32 therefore we get another valid
> range of sequence numbers from 2147483648 to 4294967295. So if the sender
> has sent all its packets within the window from 0 to 2147483647 and
> receiver
> has accepted all the packets and sent cumulative acks as well for all of
> these then it advances RCV.NXT to start of next maximum available range
> of sequence numbers.
> Upon receiving all cumulative acks the sender also advance to next set of
> available sequence numbers. Otherwise, if considering the scenario when
> spurious retransmission occurs or another scenario where all cumulative
> acks
> are lost and retransmission timeout or another scenario when cumulative ack
> carrying widow update information is lost, then in such cases the sender
> will retransmit segment with sequence number that will fail receiver's
> segment acceptability test.
>
> If there is a reason behind this then please cite in response. Also please
> tell if such change is valid then do we need to report it as correction or
> as a
> change for RFC 7323.
>
> Thanks & Regards,
> Sanjeev Ranot
>
> _______________________________________________
> tcpm mailing list
> tcpm@ietf.org
> https://www.ietf.org/mailman/listinfo/tcpm
>
>

--94eb2c0568b00841f3053aea0559
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable


Hello Sanjeev,

I wrote a draft about it=
 before.=C2=A0
I am still working on it as I need to do some stud=
ies a bit more, but I think it's probably ok to use 15 as a shift count=
.
A question would be how to migrate it even if using 15 is possi=
ble.=C2=A0
Or, we can publish this draft as reference information=
 as we've seen this question from time to time.

http=
s://tools.ietf.org/html/draft-nishida-tcpm-maxwin-00

Thanks,
--
Yoshi

On Thu, Aug 25, 2016 at 1:29 AM, Sanjeev Ranot <sanjeev.ranot.81=
@gmail.com> wrote:
Hi,

please can the team explain why we d=
on't use 15 as a shift count instead
of 14 as mentioned in RFC 7323.=
 Is it because of machines not supporting
unsigned storage or is it more=
 than sufficient guard for denying old or
unwanted segment retransmissio=
ns from the previous window.

My understanding:
We know that 2^32 =
sequence counter (sequence counter is the the sequence
field in TCP head=
er) has valid sequence numbers from 0 to 4294967295. To
determine if a d=
ata segment is "old" or "new" it sets the window size t=
o
floor((2^32-1 )/2) +1 since sender's and receiver's window can=
 be out of
phase by at-most 1 window.
1st part has valid range of seq=
uence numbers from =3D 0 to 2147483647 and
since its contained inside si=
ze of of 2^32 therefore we get another valid
range of sequence numbers f=
rom 2147483648 to 4294967295. So if the sender
has sent all its packets =
within the window from 0 to 2147483647 and receiver
has accepted all the=
 packets and sent cumulative acks as well for all of
these then it advan=
ces RCV.NXT to start of next maximum available range
of sequence numbers=
.
Upon receiving all cumulative acks the sender also advance to next set=
 of 
available sequence numbers. Otherwise, if considering the scenario =
when 
spurious retransmission occurs or another scenario where all cumul=
ative acks
are lost and retransmission timeout or another scenario when =
cumulative ack
carrying widow update information is lost, then in such c=
ases the sender 
will retransmit segment with sequence number that will =
fail receiver's 
segment acceptability test.

If there is a re=
ason behind this then please cite in response. Also please 
tell if such=
 change is valid then do we need to report it as correction or as a 
cha=
nge for RFC 7323.

Thanks & Regards,
Sanjeev Ranot


_______________________________________________

tcpm mailing list

tcpm@ietf.org

https://www.ietf.org/mailman/listinfo/tcpm





--94eb2c0568b00841f3053aea0559--


From nobody Sun Aug 28 08:05:22 2016
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CAFB712D5A0 for ; Sun, 28 Aug 2016 01:55:12 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.739
X-Spam-Level: 
X-Spam-Status: No, score=-1.739 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, SPF_PASS=-0.001, T_FREEMAIL_DOC_PDF=0.01] autolearn=no autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id trBMULKS3CI8 for ; Sun, 28 Aug 2016 01:55:11 -0700 (PDT)
Received: from mail-ua0-x229.google.com (mail-ua0-x229.google.com [IPv6:2607:f8b0:400c:c08::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 62F9C12D1AE for ; Sun, 28 Aug 2016 01:55:11 -0700 (PDT)
Received: by mail-ua0-x229.google.com with SMTP id k90so202647663uak.1 for ; Sun, 28 Aug 2016 01:55:11 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;  h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=bkF3br3yCqjV+2oE/s4eff5BUoaUD2ZmiV0t1XDvCvg=; b=UhnaZi1bfqHrZYYIvYZdJ4uDicQ6fh7CbFR62Q8cR1F9jDC5Dn/7BqZmTzf6RnQCj8 HQRuzaM/zwLdgaKVjj9L5g6CbzrZwSTq4lavvmby5hiX2DqWFGeLZBqU7WFB6Z3CEdep EMkcuJdbxVoHh4iKnoutFpQTt9Yprt2W5IxsOJnfBBOuXmdNziapPBZTz3MaSkez74oZ yTkRjIDhyt2Djy0DDkCU1HPG4z+6NELdFSBr4RtklryPTB9lIrSn8IFpYAXs4Fi/WrmV 7sAgDKqV0pL4VAONQyGj++a9w4BotSBtlpZdvFzKP2Otx99w71xwWG8frHMcXGiXU0nw AkJg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=bkF3br3yCqjV+2oE/s4eff5BUoaUD2ZmiV0t1XDvCvg=; b=kLja5rvOZGzwFmkWQzYHksPxYUTCzSsEgTL+UONLh50RMQQxBTEH+wnCzh2e+xUMl3 A2mKNwebXFENyuj4Yh8gEO5BiB2e9H/1Ne97yvd+50E5/G98Jbku3ih24uuWpMeAesMy 3OZWGz/gC/KYU8+HJJw0bJolpMeD/ft7UfouSEpKxmA8irdaTOd585KyEmwfilc4YD9b 6ZmwEyMcE/LwuMptjwMl5+ZWr0oZv73V3b5Ug+VczC3+scThqp85OMC7TrWcFpFV5yj8 htkUatJA2f34Z6W0f1b9Q+73c3T+/fdtQLnaUhYUfUd5Mk88tzp2x4Iqx6wSt25sO6ov /JaQ==
X-Gm-Message-State: AE9vXwOvCG/dfYPwSDXHHVRqe5RdPe9rA5VlgJZwsO4Vr7WXCNmLpqEPxWZ5zdrbOs7ynkaxlDd+scXR25Tr1A==
X-Received: by 10.31.61.10 with SMTP id k10mr5844367vka.19.1472374510413; Sun, 28 Aug 2016 01:55:10 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.176.4.199 with HTTP; Sun, 28 Aug 2016 01:55:09 -0700 (PDT)
In-Reply-To: 
References:  
From: Sanjeev Ranot 
Date: Sun, 28 Aug 2016 14:25:09 +0530
Message-ID: 
To: Yoshifumi Nishida 
Content-Type: multipart/mixed; boundary=001a114d9492ab9370053b1de7cd
Archived-At: 
X-Mailman-Approved-At: Sun, 28 Aug 2016 08:05:20 -0700
Cc: "tcpm@ietf.org" 
Subject: Re: [tcpm] (ref. RFC 7323) Increase of shift count from 14 to 15.
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Sun, 28 Aug 2016 08:55:13 -0000

--001a114d9492ab9370053b1de7cd
Content-Type: multipart/alternative; boundary=001a114d9492ab936c053b1de7cb

--001a114d9492ab936c053b1de7cb
Content-Type: text/plain; charset=UTF-8

Hi Yoshi,

thanks for replying. I have done some work in these couple of days on
how exchange and interoperability of shift count 15 or greater will work.

Basically my idea is for the new TCP stack (which will support shf.cnt >=
15)
to exchange shf.cnt even in Data Transfer Phase and not only in connection
establishment. I have presented this exchange work in the document enclosed.

Please have a look at the document and let me know for any info or
clarification
required. I hope it helps in your related studies and work.

Please feel free to write and discuss.

Thanks & Regards,
Sanjeev Ranot


On 26 August 2016 at 00:31, Yoshifumi Nishida 
wrote:

> Hello Sanjeev,
>
> I wrote a draft about it before.
> I am still working on it as I need to do some studies a bit more, but I
> think it's probably ok to use 15 as a shift count.
> A question would be how to migrate it even if using 15 is possible.
> Or, we can publish this draft as reference information as we've seen this
> question from time to time.
>
> https://tools.ietf.org/html/draft-nishida-tcpm-maxwin-00
>
> Thanks,
> --
> Yoshi
>
> On Thu, Aug 25, 2016 at 1:29 AM, Sanjeev Ranot  > wrote:
>
>> Hi,
>>
>> please can the team explain why we don't use 15 as a shift count instead
>> of 14 as mentioned in RFC 7323. Is it because of machines not supporting
>> unsigned storage or is it more than sufficient guard for denying old or
>> unwanted segment retransmissions from the previous window.
>>
>> My understanding:
>> We know that 2^32 sequence counter (sequence counter is the the sequence
>> field in TCP header) has valid sequence numbers from 0 to 4294967295. To
>> determine if a data segment is "old" or "new" it sets the window size to
>> floor((2^32-1 )/2) +1 since sender's and receiver's window can be out of
>> phase by at-most 1 window.
>> 1st part has valid range of sequence numbers from = 0 to 2147483647 and
>> since its contained inside size of of 2^32 therefore we get another valid
>> range of sequence numbers from 2147483648 to 4294967295. So if the sender
>> has sent all its packets within the window from 0 to 2147483647 and
>> receiver
>> has accepted all the packets and sent cumulative acks as well for all of
>> these then it advances RCV.NXT to start of next maximum available range
>> of sequence numbers.
>> Upon receiving all cumulative acks the sender also advance to next set of
>> available sequence numbers. Otherwise, if considering the scenario when
>> spurious retransmission occurs or another scenario where all cumulative
>> acks
>> are lost and retransmission timeout or another scenario when cumulative
>> ack
>> carrying widow update information is lost, then in such cases the sender
>> will retransmit segment with sequence number that will fail receiver's
>> segment acceptability test.
>>
>> If there is a reason behind this then please cite in response. Also
>> please
>> tell if such change is valid then do we need to report it as correction
>> or as a
>> change for RFC 7323.
>>
>> Thanks & Regards,
>> Sanjeev Ranot
>>
>> _______________________________________________
>> tcpm mailing list
>> tcpm@ietf.org
>> https://www.ietf.org/mailman/listinfo/tcpm
>>
>>
>

--001a114d9492ab936c053b1de7cb
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable


Hi Yoshi,

thanks for replying.=
 I have done some work in these couple of days on 
how exchange an=
d interoperability of shift count 15 or greater will work. 

Basically my idea is for the new TCP stack (which will=
 support shf.cnt >=3D 15)
to exchange shf.cnt even in Data=
 Transfer Phase and not only in connection
establishment. I h=
ave presented this exchange work in the document enclosed.

Please have a look at the document and let me know for any info or clarif=
ication
required. I hope it helps in your related studies and=
 work. 

Please feel free to write and discuss.

Thanks & Regards,
Sanjeev Ranot


On 26 August 2016 at 00:31, Yoshifumi Nishida <=
nishida@sfc.wid=
e.ad.jp> wrote:
Hello Sanjeev,

I wrote a draft about it before.=C2=
=A0
I am still working on it as I need to do some studies a bit m=
ore, but I think it's probably ok to use 15 as a shift count.
A question would be how to migrate it even if using 15 is possible.=C2=A0<=
/div>
Or, we can publish this draft as reference information as we'=
ve seen this question from time to time.

https://tools.ietf.org/html/draft-nishida-tcpm-maxwin-00

Thanks,
--
Yoshi

On Thu, Aug 25, 2016 at 1:29 AM, Sa=
njeev Ranot <sanjeev.ranot.81@gmail.com> wrote:
=

Hi,

please can the team explain why we do=
n't use 15 as a shift count instead
of 14 as mentioned in RFC 7323. =
Is it because of machines not supporting
unsigned storage or is it more =
than sufficient guard for denying old or
unwanted segment retransmission=
s from the previous window.

My understanding:
We know that 2^32 s=
equence counter (sequence counter is the the sequence
field in TCP heade=
r) has valid sequence numbers from 0 to 4294967295. To
determine if a da=
ta segment is "old" or "new" it sets the window size to=

floor((2^32-1 )/2) +1 since sender's and receiver's window can =
be out of
phase by at-most 1 window.
1st part has valid range of sequ=
ence numbers from =3D 0 to 2147483647 and
since its contained inside siz=
e of of 2^32 therefore we get another valid
range of sequence numbers fr=
om 2147483648 to 4294967295. So if the sender
has sent all its packets w=
ithin the window from 0 to 2147483647 and receiver
has accepted all the =
packets and sent cumulative acks as well for all of
these then it advanc=
es RCV.NXT to start of next maximum available range
of sequence numbers.=

Upon receiving all cumulative acks the sender also advance to next set =
of 
available sequence numbers. Otherwise, if considering the scenario w=
hen 
spurious retransmission occurs or another scenario where all cumula=
tive acks
are lost and retransmission timeout or another scenario when c=
umulative ack
carrying widow update information is lost, then in such ca=
ses the sender 
will retransmit segment with sequence number that will f=
ail receiver's 
segment acceptability test.

If there is a rea=
son behind this then please cite in response. Also please 
tell if such =
change is valid then do we need to report it as correction or as a 
chan=
ge for RFC 7323.

Thanks & Regards,
Sanjeev Ranot


_______________________________________________

tcpm mailing list

tcpm@ietf.org

https://www.ietf.org/mailman/listinfo/tcpm








--001a114d9492ab936c053b1de7cb--

--001a114d9492ab9370053b1de7cd
Content-Type: application/pdf; name="shf.cnt_Interoperability.pdf"
Content-Disposition: attachment; filename="shf.cnt_Interoperability.pdf"
Content-Transfer-Encoding: base64
X-Attachment-Id: f_isedqde50

JVBERi0xLjUKJbXtrvsKMyAwIG9iago8PCAvTGVuZ3RoIDQgMCBSCiAgIC9GaWx0ZXIgL0ZsYXRl
RGVjb2RlCj4+CnN0cmVhbQp4nL1XXW/bRhB8169Y+CUJEDP6cB3HQAskjpO4aNLWVuqHoAVO5FG8
mrxjeEcr6q/v7B4lyw1pxEBj+8WmqLnZ3dnZvc+jMfFvs6RnakxLP3o1H72QRy/o+Tg5Onw+OTqi
o8Np8nw2nR5MaF6NnuX74/0x4e989Onxj/f7ebI/u/WVJ3/OfwaJ/UkyOTwYH05pngH0o9fkcvKF
yQOlrrWBjqn/1e9yfnBkbO6ailwodEPaZsxnfvIb2Fir02CcpcKtqGrTghgUH/djnZ/8kbz6+IYM
ImmbRttQrqlQnoBPoVHW57ppjF1SpoKKWE0/1PZAkALeI0+qrkuTKqGTKsvsfFtpUgHvMJTuQ3rc
EUmezP/+6jPqefid0nwuxFVJku+00Qp1v1X0h635AolblJrpLHWgwiy5+Atls5XJQkG9FX7IhP1i
KhNYKrlKg2uihnyRJymS9RNNDh40Y691jcbQNjXaU7mhxtq0WmfCbYmaBuQwFMoyKFPsTeImCOjY
r33Q1XF/WicJvdeVa9bHUAyQkQiNbmqylWo0rUxZkm/r2jUBjOyVnNnPnoh8zTzRuZPxmN4uak+g
DHBNi5YbSK+lqRa9TSQIEjUwFmva9Hkl9JJ7SGWa0Nk2lnddLMd8PphUak3WBdEm8ppqzzF5E9bD
cXHmC3XN1fBsF6VqlrcbC/YDPzIoG6PFIPoZE8fGqYzZuodlzBK6cHmIweA0jkSVXtpsJ28bCxMi
KP9gVKvCIJbO4Tx5vazgYQjEAt1Tg9yYay5dnmsGa5KBfqU+vgNBfIDlZqJzzmZbgyALPBaZ4Ays
HZGOQoApn1sYqweiYDVlOjWZvhkeUpakK0tdN+6LqdAzGBKG0aBDdaV9fyyfHl8a+NKGQtQzk9LW
tctiQxN+1jWF9IMKJZw2DCBO/5pNntJKi/Q9aobeDZv/W88y3FVSN/smPwxluzet8622IYbY8B4F
5ocxCJam9l4asXJSzm3W+3njG+pamVIcnEfr3UcwZMxVP9ymwvw6FGVgcoj2JpNsuyFaFkcvid24
Rz8ih8JwO9pnKUVB1zLCAaXsmuoG9i79kGI49qPtTn6w6triXkX4LkPhbeNW3Cqc2YsCe80V/4fA
8DSD5lcs2JuRxaAPPLYuC23jhELeYJNR0fmmljJGO7l/bnXDoEOay11ZxnDTQqdXmMeNq6SqUXDw
WVRlOxVlIVsNYPFy6YNKrzYK85HKTrP1DhWZiZX6cjPNbgaj+DYEEq2zW1+ZxWaFHTRbdJMGm0Vp
fMGeUjoEIbXrdkc6zfPE2yx5f8GAF5zMK+uGogMkpwC6biHp+Rdq4HFDpoaBKPmUdtNYdfNN699q
ch5zHLgMRAl+OJzbKamd9wYQQwRmXxPY7bdtgbreFZ/u+nd42WCz55VSQudXed+/V8OeRJFNpLmm
sAntuRKKsrVVlUnJqtA2sr+m3cK1ZPH1c9po/lLHwonPe+zhms5P5+cvT+anr+ny7MPrXy8pa8Uw
GXkQLLrhEjeaFgsHLygrrBmqxAqYrXlK1yUXgFcPe8dedv7mZB+XzZkEKftC5LTnv3aTPVKi5qHK
7y07N9r5xi6pOCNATQ59wVizO2YCvzWZTKeivH9047a+AUsBRbhG5uyjEF1FHHwoX6HoWlOyXrol
qrdiY1oxn9C4rE31jS0xFI+ZAbRu8CR0rpF/WVVEKzR7inDo9qxw0jBsbwOD1O5nBtsU31NjzqGw
zr3RCNjkMqc9hynyZ7BqaObtLmnbi7DdnajRB85iIkQWKd8Chz1S1pJIix2lI+biErjQhSrzOBCr
iuFay5EPAaIMnt8Wy+6WPByS0Jw/YLRrg+Tib26RKFszlDivkZAM5mLiJPGdye5Nk4OEXmYZ+tXz
CZfC+a4e0IFXAHx5j7b9UCeTyX2uFe9EkwtuPYgpFp8teOEgzk5xfF/qdvLY39IQvYf896r5zTeB
b3/4/wz80/no99G/4mZI3AplbmRzdHJlYW0KZW5kb2JqCjQgMCBvYmoKICAgMTQ4MQplbmRvYmoK
MiAwIG9iago8PAogICAvRXh0R1N0YXRlIDw8CiAgICAgIC9hMCA8PCAvQ0EgMSAvY2EgMSA+Pgog
ICA+PgogICAvRm9udCA8PAogICAgICAvZi0wLTAgNSAwIFIKICAgPj4KPj4KZW5kb2JqCjYgMCBv
YmoKPDwgL1R5cGUgL1BhZ2UKICAgL1BhcmVudCAxIDAgUgogICAvTWVkaWFCb3ggWyAwIDAgNjY3
LjU1OTA1NSA5MTMuNjA2MjY1IF0KICAgL0NvbnRlbnRzIDMgMCBSCiAgIC9Hcm91cCA8PAogICAg
ICAvVHlwZSAvR3JvdXAKICAgICAgL1MgL1RyYW5zcGFyZW5jeQogICAgICAvSSB0cnVlCiAgICAg
IC9DUyAvRGV2aWNlUkdCCiAgID4+CiAgIC9SZXNvdXJjZXMgMiAwIFIKPj4KZW5kb2JqCjggMCBv
YmoKPDwgL0xlbmd0aCA5IDAgUgogICAvRmlsdGVyIC9GbGF0ZURlY29kZQo+PgpzdHJlYW0KeJzF
WFtz2kYUfudXnLfiBzaSAAEZ2zNg48Rt6jgBN+14+rCIFWyrC5EWE2b643vOStzESph02soztged
PffzfWf5WrOAfpIZvOEWzNLaYFzr6Y960LFY1+3Y3S50XYd1mo7TsmEc1t74DathAf7v1+r3kRJJ
vBAJn8hAqjW8vRj/gVobNrPdluU6MJ7WnutX5z0XjebBkYvfxz8WldbBaOmmPxqC/RbSuc+8SMFM
KCWjGYhv3pxHMzGFOArWICMY/fYAnCxFU9AGWqxru91WwQyKNfo3P0EqZqGIVEpWG0bRfxymUaZm
iv1jMIXxzSOkint/Qp98ajZZx+50uw4Wj0QexGpPZKDd3sl8n+N/ne24Oae6dHaLtdpW27FzhwHA
bsG1fuWybrvba+WvnvFAw+5aLsroSjV2AoVAgCQdq0Xa4BLsdi7fY47babpH8vcKVjIIgKfpMhSg
5lyRFzLV55wu61m21XI2fnyMIBGekC/UUpcomM6lr3KndtIFI2ouIODJTKQqP+DFS+xN+nz/uWi0
unavnr30kziEGEUSENih2LAkYeqG5/pOLPa11nSdKhHq0MyB6KAV/1PAVHgylXGEcwErCv9UNOly
sYgT7X0i/DgRIBVmJeQyKslaiM4fxB3r4YuTSkt1X37T44oFYdQVR3oLo0l+LFNxwv1Xtvq2jb97
Rk34VAJaX+YigihWwKkqWL8FT3goEFpT4JRgrM4yAZ+/xAmYwBB7TIRYFZ7IYM1gtPTmwMEXXC3x
uMejDPUm2BlZ9SgsMS3RhSmPxArlvThcyEDsYU0Kc66bn7z04ikpJF1aKX0uFTNqPScdGsOdYwwn
d5ZBGlOQe4BO9nM8L8fx5zrH8dh0DP0f8jVp3lOESm654jBOOLYyavVFYozlcc5TgZgBK4ED9n8Q
ghlXDzFf04LjslbPcZ1KWrBZ0+q13P+OFjrMcdqOa6QFh+HU2laBFrJ3Wxg/jxd2bEDhOjZzO+1O
94gMbpeCsImQ0+eeinH6AhlKxRXi42sIAc5hhCivhIbilMB70++FR8f4XbTgk0OTJSIu9mmqZzY7
mvHDRaPnNtv1DIuhiKYlOEMUgUCb40gK1zrmzPHJ2pwlmbHsRcO1nPoh61RTQJpPvh/wGQx/fbyy
jUSwoa4C0ZwowtMi3q/e+xjpefBDih99XSJVm0PB/kCOYbBZHNDDU8WmbAnM1Ojhlo3e37Gb/iNc
wfjz09Bkom4Q011bpv6eaCOND218Hn7Cw/n6cxTEzvld111tdqUqQ9pGH3O0qfgVdRjfbNBHhiZr
MJSwsuSDQ+1tQ8HrhShbVQmqf7755Ui4qLAg0wY4qRJF2d2HdyhuGXtyk61iPKe6hWLLddO8IKKi
AWNyKe3FWhjpyuDu0VyXmq3oBx9CWv9CgX/wSoW7yguXAZ8EwowdAVe40hAMRQQhRMCEA1ONuuYj
MvISQWSLEpg7hCltEqlXliHUCl/iphAuI+lxhbSOTbgFTGN+NpiGg52iFXbGqtI47yGGPCljBnO9
mSjaTHArgWwDWUk135b/YI/xyFIcRcIj7oLhaNwffLgfvf95+DCGBR0uMfOvBVRxEyzbnr7I6Gkx
xRpSCfvUAgMjzhwKDkiwX42ZOAY0FIOnO7i+3h+iTP3hzvSs14fDI4PCkcbuzKsW/q3Dms8zl18R
W5aE6tgoGprmLw+3m0kmb2nVQF1wJ0UwNQPK4Phk33iybIXCRF1eHuTGgLVapn8o02i6zHbcpvXK
9CGCiBccg76mMAQeHIVoGq9KaKjA9GQ8kz+BcMss8brx8h0GEjmb59Bj7JP8zEY8EH5+6a/sE7xW
671Fo1B2j0ohXiFOMsFKopJKt8QeAxAj8DSVs6iaY/uVFdKaMWaptu0AL3izJGBnZswdb78LGOjb
psYp3Jz1tbNikQz1RTXUmy1uqyu5qckxqs8lisp8I89QXW9sGIv5RP4dyFQssEVood3bDFl1fsoJ
oMfcZtM9vjpcn/cQbOJvo999PXDeDryJJ/HWTay3DkOhEunB+Bsk1GbiRd/kozIuJKqdIPuBHhOt
ZfP9Deb7Gux3k3wU8i4bjmufan8D+ZOmcwplbmRzdHJlYW0KZW5kb2JqCjkgMCBvYmoKICAgMTU2
NwplbmRvYmoKNyAwIG9iago8PAogICAvRXh0R1N0YXRlIDw8CiAgICAgIC9hMCA8PCAvQ0EgMSAv
Y2EgMSA+PgogICA+PgogICAvRm9udCA8PAogICAgICAvZi0wLTAgNSAwIFIKICAgPj4KPj4KZW5k
b2JqCjEwIDAgb2JqCjw8IC9UeXBlIC9QYWdlCiAgIC9QYXJlbnQgMSAwIFIKICAgL01lZGlhQm94
IFsgMCAwIDY2Ny41NTkwNTUgOTEzLjYwNjI2NSBdCiAgIC9Db250ZW50cyA4IDAgUgogICAvR3Jv
dXAgPDwKICAgICAgL1R5cGUgL0dyb3VwCiAgICAgIC9TIC9UcmFuc3BhcmVuY3kKICAgICAgL0kg
dHJ1ZQogICAgICAvQ1MgL0RldmljZVJHQgogICA+PgogICAvUmVzb3VyY2VzIDcgMCBSCj4+CmVu
ZG9iagoxMiAwIG9iago8PCAvTGVuZ3RoIDEzIDAgUgogICAvRmlsdGVyIC9GbGF0ZURlY29kZQo+
PgpzdHJlYW0KeJy9WFlT20gQfvev6LeYAk10WcipQJXtmKR2U4mDxWarNvswlkb2bHQ4mjEOW/7x
2yPJmMAIfMCKAoTV8/X19SF+tExQX8UUXlMTpqLVD1rd8qMuWJ1T0jk1HcsG37PJqWPbrgVB2nod
G6ZhAt7Hrb/aQUHjmIcQF3kKPeO8D0ueJCAW83leSEjzgkHwEwoqGVABR38Hv6FWwyKW55qeDUGE
IDENecIlykQwuYE+yBnNoG+c92A54+EMuIAwz4QsKM9YdGQ47QYkPN0j8JXBjF4rhWKRIibCSRiM
rkDMGf6JaFkugWox2glPueTZFNAqmRfkKPjnoZ63u13KYvyp1Bld4jmO51v31IJWj7HbpfQ8KdPS
Rg5ggEGfMuAZiFlMwkyq23dUUsC4ZyJmBYxmVDDIM6BK0zXlCZ2ozN1AHoO4EZKlkDJM+g00qPm/
HGqIZzAYQQ8KRhP+LxMVMbiEEOmmJ1TJo5LFl4M/SP/qQgWlIqX+QKgihJDpQiBunqaLjIeK/Xpx
OUM28izOQeZI/IVQzMP4S5UA0nAoUIeuabJgVbVNGAim0qUX51KU1o8/XJDL4Rc4w+LWS44oGj1h
s2ueF7CkdYDkjAmmP0ALRYfkBtjPsKRPBNGiUD40RCfPMhZKjhQajoM+CEmnrMnNHrpGw5DN5Zpl
kgn0JeHf2Rtt7VoEAzFNWSY12W/bBOG+Z/kyYVGjkEMwplmUL2ExjzBvOhmXIIXKoki5EMoZLdmq
TshomjAhMEYYoChh618NucpCZKcoy7C2QyBViV4aG908Qe4idxZ4RtAUuy1S/Os4n8sGDXV+ol8K
e14WdkOOkxzll1zOsGXGCZ3C8M+RIlyDUcGMqW4d8TLN/C6JGjSs+00V8VtON1RMFXephkVJvYIl
nE7wRksJNKF0teYF5IU2VyiW5ZnB0jnSDFlyK6834lv70+egrtbNGb3sHdUwX2DBoOi3I62xOh41
dLKPTFZDkUeYP5Fj5lXrEfrCeOHp0tDsB4pUNg7kMU/nSPlj7KUF+7HAKq4WhqodYz2UjxR+wcQc
nWKb531489KzZPsJ0obj+syxemp5xO/4XVyMzOoplIccj1i255h2/fEvZ4yOTbp+x3adB0RZ1QFZ
wduD/HlwndfQfVg1+PqSbmFiLdt00b+Sm66Pj33XdDbot9eqhHI75LTT7fqaCNVQ2IHOrBJto7ZC
Wz0V4xpBOxD3BRt/eleCDXojBAsur4bPAvcstvUGv5Px8D1+K7Aoz16pfadgB1uI1pGLj+8R1Dw4
ExusA7OKOBtnbdN8FK5DnK7jeU8ydiO45fGSyYZn2tZddq/q/rdaL9hnVgcLs+Ry2fxWUNt+piyH
86pW7xu/qZNtSm7LDrFDM1n3FH0recZ6b1eP77UdpeBzhnEMGb9W8xc3i9upUqs0Nme2RKzeX0/q
LonLjTgyXN/y2/sC3q9hXHIPBMTVUL1ZCIh4HNcGvxJVsL29Yde7lxrCR4bvOubBLm+64ImitWe5
e0NibgsW4+vXiXqnKhdDDKNt7e/vlGWs+m9EvY5U+wZG0TefSk7VaMoYqZGoRDUaJrgMRpFiZrWG
4yZfLofbaHgkT/USqfKkXvbK9mLvDXdnKpx1sNnsRs+d+ml7U+mOTzqOe2rbt61ANT9Tt3M82uGa
/Xow1g8Lk2aw7xAm2yNu1/bsO4D3rwejXtnr1PDDoPWl9R9n4bD4CmVuZHN0cmVhbQplbmRvYmoK
MTMgMCBvYmoKICAgMTI4NAplbmRvYmoKMTEgMCBvYmoKPDwKICAgL0V4dEdTdGF0ZSA8PAogICAg
ICAvYTAgPDwgL0NBIDEgL2NhIDEgPj4KICAgPj4KICAgL0ZvbnQgPDwKICAgICAgL2YtMC0wIDUg
MCBSCiAgID4+Cj4+CmVuZG9iagoxNCAwIG9iago8PCAvVHlwZSAvUGFnZQogICAvUGFyZW50IDEg
MCBSCiAgIC9NZWRpYUJveCBbIDAgMCA2NjcuNTU5MDU1IDkxMy42MDYyNjUgXQogICAvQ29udGVu
dHMgMTIgMCBSCiAgIC9Hcm91cCA8PAogICAgICAvVHlwZSAvR3JvdXAKICAgICAgL1MgL1RyYW5z
cGFyZW5jeQogICAgICAvSSB0cnVlCiAgICAgIC9DUyAvRGV2aWNlUkdCCiAgID4+CiAgIC9SZXNv
dXJjZXMgMTEgMCBSCj4+CmVuZG9iagoxNiAwIG9iago8PCAvTGVuZ3RoIDE3IDAgUgogICAvRmls
dGVyIC9GbGF0ZURlY29kZQo+PgpzdHJlYW0KeJy9WG1z2jgQ/u5fsd9qJmNVkmXH7sTMBEp7d+1l
mkCbm0nvg48Y4l4wKXba3gw/viu/YIMx+IUemQyJLT16tKvdZ1dfFQryZzWHly6FeagMJoodP7LB
5sQQ3DZtsExOznXOBYPJQnk506hGAf+eKSqse5MvCjeJsLnJOU6c3Ct3KpQ+l8N3ZDx6i7/X4IBB
KetpwmK2uu79PflD0XIIjRFmCmpyCZUuoJuEcVOnhQXGV6/JzeiavHn/FgERzTS5yNDy8TXRboaf
Cmi0Kxoi5Zu9XwYvIpi6K6+nMU7NDaxFDF2c725ZAsQDhVxAjhQWvrYE1ZMVtqwbU9CEQc4N27b0
EtQasXTKWDbpAsYxuWtH+gBfS57oHEcXpvx39NcHh+J3+DAjU5yqBpHDDHxw9s2DlRc+4YqKZGdI
RLQT5WrCI3EjEdQSh3Z0odX6yKXrjdx8kJjkQH+tOX+fgR+F4G4MsgxCL8Y1OLEtg4sMN8HKn5ax
fOIR8Gdb58UPoSVa6EXgBveZRyWSH4A7/bclnjudek+R/88j7tMN5l4M3hKrsDvkFHrzhRdEbcH2
EevAS9qq78BnteiGs7bk0pAoYJH3oyuNfe51oIlnJA5ItJnjxMlq/NsbmbBa4kH04AU755g0x0oz
YMUcg+i2bprl9eNAg+FD7LvLF2G2t1cJg828zgwq58g0J1UDU0Ob2QUHSBSjMUhy9lDBJMrwUnKZ
3HwcHfLAEWtmYBtKjU0Zc9qW6YJytTkbOxpNW9u6C0a8rW1B5rL6aBc4hdMKpwhEibe6xbzqwGrw
PPvTW0C/X1yl3ZFoLHibEQegMtVHaTYZV9dw6wcfn+7dyHM2+8gLiKyuMHG4oibFReoCBys9Bv12
R7RRSdCkiuin9UONGiZmnpdvWd3X02yq8x3zHqog72KzWkKnaiI35Vryahl5mK3dCCbDD1I+w8h/
fMSlsBqnRwrUu8xrohL+OfSDefG8OUygLhynXln8fpf0pkluR6BoickIlsHjf+0RE63CqsELZTkj
MeVXe45Z/RGD4reb6WA7RDXRz0RMmEksw7KxV6oRq2oegxWBcHTezjFUkwA8wOSgkJZtVdK6Tq3R
Hs3TsMs8dpKP4p2G3b4mtSu7ra4S3WS2BSu1qJ3AqjtUi1qt7beljgXf7DS+Lcx4e/UaEpnZqA40
ISsosYXORPGK4uJiS2kl2nmGJnSiG8wwS2jHArYqvH7sq1x+HA5y+ZbWeHjXtFWuoYsVzfTQDT3g
AwJjf/GEDVjSPXx99sIIZqvlIpapy7hV1L5JF+lq1iRjh7XyvnjTCFuhzdABvKpY6X/aUoWVz9I5
Z/syKSSpflcGtuYcLPsSI62hzmVIgxqmn0IPYP9dyK/d1imvWVIoqWUtS/UUoSRfXcCadWy14U7C
7XSd2w7DouZ09UTxwrYTVqN+rm6f1KgEyzrv+L6HFU/3Os2J67ymNtKbVRYnxLwRksyxEVL2kc/j
pE7I1cwQza9Vs7ZoNFGulZ+8iVX0CmVuZHN0cmVhbQplbmRvYmoKMTcgMCBvYmoKICAgMTEzNApl
bmRvYmoKMTUgMCBvYmoKPDwKICAgL0V4dEdTdGF0ZSA8PAogICAgICAvYTAgPDwgL0NBIDEgL2Nh
IDEgPj4KICAgPj4KICAgL0ZvbnQgPDwKICAgICAgL2YtMC0wIDUgMCBSCiAgID4+Cj4+CmVuZG9i
agoxOCAwIG9iago8PCAvVHlwZSAvUGFnZQogICAvUGFyZW50IDEgMCBSCiAgIC9NZWRpYUJveCBb
IDAgMCA2NjcuNTU5MDU1IDkxMy42MDYyNjUgXQogICAvQ29udGVudHMgMTYgMCBSCiAgIC9Hcm91
cCA8PAogICAgICAvVHlwZSAvR3JvdXAKICAgICAgL1MgL1RyYW5zcGFyZW5jeQogICAgICAvSSB0
cnVlCiAgICAgIC9DUyAvRGV2aWNlUkdCCiAgID4+CiAgIC9SZXNvdXJjZXMgMTUgMCBSCj4+CmVu
ZG9iagoyMCAwIG9iago8PCAvTGVuZ3RoIDIxIDAgUgogICAvRmlsdGVyIC9GbGF0ZURlY29kZQo+
PgpzdHJlYW0KeJzNWG1Po1gU/s6vOJ+0jQvDu3RjTbSrs7szmYxadzYx+wHh0rJTLx2gdibhx+85
F0pphVpBM1vT2hTuc889b8/D+SapQH/xBN65KkwS6XwsDcRPAzhWFcc+1hwHHFtXjg1dNzUYP0jv
AlmVVcDvgXTXA+jLmq6aPcj6/4z/lExH0WzHVA3EGPsSXl+/sv74X0k2LeXYGgwcA2QN7zVVW8/v
FJcNW9F021D1fP1dLwwgTCFMwAX5kUHMknnEE1YA0pbyes1eiOmUcbj4+/NQBZf7kEwDxeMpLMPZ
rCXiPYOEpZBGcHaYrAD7sqlrdq8d4qcoZbBk4LmcRyl40cN9yBl5QLi7LWzpvWWYTuFI+PPbgiVo
rOaoTkvUJOQeI4/CENPigbk8WePamBvtYL8w8CN+mArfno0+KDcX7/F91ZcxE9US1FEswzzWt0Df
PDN7IoPogmYrjuUMsDryCzmYpSsDx9LNPdP8evSXcvP7pXJ9cUVetPoyHqltmG8+/SbARmcUkvH1
7QXFQdc7whW2mZ1sqwQSwfIAe27MtqL6cgvROuXy43sEVdFAR2+bzRSJV8NCnLrDwla91edwb52b
lqUYA8O2jX1y7I7MQVM1o0zykxthydXQUlUtI5MwDkPDtLVM5HFWNK0hRdfoZdVWm/UlsnK92YYJ
pWHN5Xci7/Wijfe7s3wVpqlvWPvfhfu3zv59++y6YuiOplv5GrqqbgPV/Jj3+V/hS8hv576LLR95
bsI4i/G7D26QshiSheexJAkWM3DJRZOYsQeGhBUFJXdFHO4j7OhJ6LNEsFqY5K7RFVN1zJrmmBBE
EkE6dZG5phRuj4WPuCFyDvBoCe4ML3tTl08Y7avAH0HF0glLqctHsc9itFUQSmM00DY6CW3pLxgR
JWfLlfW/gGBk/EBimrsebfaV0JeiXnAj8ONoPsdd7g89d5Gwxn1W/mBBwDxECPEdLdGf6KHZDyWv
tlqXNMTnhfnYIoVxUf1pRi66Qx8pMMrJ3y/4umDwgyp9QxDT1tEDjEef4ZwyIM97xaG8f+IobwVZ
xpMIwugdbO4RxAXiWe65Brif5qSGSjsq1hzVsTNWfB2bb6zZ1Vhzf+D/fbraC456WkCfQ31Te9tj
vVa/zKlHQBGvaP0dzLGLvATCE03UBaxGE70G3KvY1qiJulpY1TFdI7HG6hjVDU2kox7ZBbeHvMiX
1wuk5uUik0kVa9XszoqmmkEpiCwsTJHLIJQRFLYPyXI4rddG6zrZp+T2bYG1zWTX6tNshwho0qzI
8gWJEP8vXY4cmkZCT7Z9kovdENmEyHhF0Kn7NeSTbRn8MlTBTqRaeDr7gfUy8xYzoZs6oT66M1Qo
qK0o589vL5Fo54sUQt4Jlc5eaWeF5AnTTk89JNRy17obYoD0X0vEYs4QptUJQTLvZCZNcILq6cEP
g4CEJ29p5vl6yiIOW8HeHDg0zgZEMO1V81jftzI5gVOotHihfYkzqGrbTzQO4CkL5anQaaoTpuVD
A43JSnHXKWqr0GP+Y171Zcc0npu5PP+8Wt+OK+u2ItErhkovHrNU+AbyED8zcLH/xwOXbsOgTXFB
z/+dbatKgdcbtXTE2pAVNcd80xnLCVQHLJCrhNWIpRQQ2Xo2bPSGmo0/bJYZaX8aG1srqxsf33/a
cOU5h12MpSvpP9sAke4KZW5kc3RyZWFtCmVuZG9iagoyMSAwIG9iagogICAxMjU5CmVuZG9iagox
OSAwIG9iago8PAogICAvRXh0R1N0YXRlIDw8CiAgICAgIC9hMCA8PCAvQ0EgMSAvY2EgMSA+Pgog
ICA+PgogICAvRm9udCA8PAogICAgICAvZi0wLTAgNSAwIFIKICAgPj4KPj4KZW5kb2JqCjIyIDAg
b2JqCjw8IC9UeXBlIC9QYWdlCiAgIC9QYXJlbnQgMSAwIFIKICAgL01lZGlhQm94IFsgMCAwIDY2
Ny41NTkwNTUgOTEzLjYwNjI2NSBdCiAgIC9Db250ZW50cyAyMCAwIFIKICAgL0dyb3VwIDw8CiAg
ICAgIC9UeXBlIC9Hcm91cAogICAgICAvUyAvVHJhbnNwYXJlbmN5CiAgICAgIC9JIHRydWUKICAg
ICAgL0NTIC9EZXZpY2VSR0IKICAgPj4KICAgL1Jlc291cmNlcyAxOSAwIFIKPj4KZW5kb2JqCjI0
IDAgb2JqCjw8IC9MZW5ndGggMjUgMCBSCiAgIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlCj4+CnN0cmVh
bQp4nO1YbW+bMBD+zq+4TxtowjMGXJiaSE2abtpL1bXZOqnaB5Y6KVMLDdB2k/LjdzYJkLc2kLXb
h1I1Ihg/fu58fu5yY42C/EtG8DqgMEq1Tl/z1SMffEZch/ncB48zsmMz5ljQv9JeD01qUsD7oXam
wwTCIaQXQzKIMmjDcfcrOXl3QI57n43v/feay4jvucyxEbN/rukTo/9TM8unpkUs7lDO5KiEM0yL
UUcHCLMUXt0KSMQYXuBnek3qI0pAHK47B3rfjqAFtNHcigsQw3JrgyivnhzuK5TunmTSP/7Sa2R9
gTNlw5ux2et+ICe9t/gvURyfWc3ZIBNy8PGtZNPYvyVG/T1SBkmA0iBGaQODcs+8TIvwb20f/gov
OY3OESzp3Aw/iStot6urPM0hUFwO40y8gTsBqUBCg/jqRxiJczgNoy/X50Em8FzmJzS9jg3T1qNU
3MfOJbZvc768Ep55j3JJU80u3tuU56SggbdTN7Usjl8Kqq2ZXyXP/IC3qAxIjAEM7hZ3uDX9iiHR
khEObUMzOWpRTosuL2zWueTCm77bnhjaijUbqpkctjmxGLcpy+ec6VJeg5VbqRSYT602y4mPREVX
WyEHLE481/MxzWxh14L6csPcYay0xiOu7eCTe2IQ95wxfCAnlO9P4VfIcv7+Ru5aYrugzu4C23pg
8yLtSk3blltVZ80dj3kNsRY1eyusOene2sx5Aa9siIqIpiQlzunhPiJK/PKYGSaWUXRD0MXgA7x2
d+eSAUJYzgPRvY5jZ23q2sbyznICM0ysJDc02qHEd2zLKY3G9FdlKj1I/RmYYxPbtVy+fKBhVklO
4L60smFSWtqLgzAKLi9/y+x4EaByZrHKkrgh1HooFFXSqnOppFW9hnJ1yJJwNBKJXLvfPYI9Q6uG
w+rksbRfKL/q/OjyXMp5TyHDzVTyWXSfRfdZdP+B6CpVsx7Q3Fyh8rKwbgmvXKo0breowSsluarQ
bc+qFO6z3ZXSKFqF/dPCufZvnd3HKuXNKSOTEYd6zl/0GXQvgmgkUoijXP4hzWS03wZJGPy4FGmz
n+hFOf7c+Hi0xkdz//4/jY/OWglvhleKd2devJvBzS6U7s5TN090+LVqzq/7Kk4sOHF0qcmQt2CO
kjBOwuw3xEO4TuKBSNMwGsHdhYhkAZqIgQixBh2oLsy0S/Om4sN22SCZ3Y7XdBhWsFhDrY5i1hXN
YtIqkmd6N0gFsH0C3VlPquhDRWnZyxjfiDSDYSKXjq+USHYgwCJdgvb62mftD8swMhkKZW5kc3Ry
ZWFtCmVuZG9iagoyNSAwIG9iagogICA5NjAKZW5kb2JqCjIzIDAgb2JqCjw8CiAgIC9FeHRHU3Rh
dGUgPDwKICAgICAgL2EwIDw8IC9DQSAxIC9jYSAxID4+CiAgID4+CiAgIC9Gb250IDw8CiAgICAg
IC9mLTAtMCA1IDAgUgogICA+Pgo+PgplbmRvYmoKMjYgMCBvYmoKPDwgL1R5cGUgL1BhZ2UKICAg
L1BhcmVudCAxIDAgUgogICAvTWVkaWFCb3ggWyAwIDAgNjY3LjU1OTA1NSA5MTMuNjA2MjY1IF0K
ICAgL0NvbnRlbnRzIDI0IDAgUgogICAvR3JvdXAgPDwKICAgICAgL1R5cGUgL0dyb3VwCiAgICAg
IC9TIC9UcmFuc3BhcmVuY3kKICAgICAgL0kgdHJ1ZQogICAgICAvQ1MgL0RldmljZVJHQgogICA+
PgogICAvUmVzb3VyY2VzIDIzIDAgUgo+PgplbmRvYmoKMjggMCBvYmoKPDwgL0xlbmd0aCAyOSAw
IFIKICAgL0ZpbHRlciAvRmxhdGVEZWNvZGUKPj4Kc3RyZWFtCnic7Vhbb9pIFH7nV5ynFJR6OjO+
YKoaKaSkl92tcnG2K1V9cGCceJfY1DaJVuLH75mxwTYQgu10H1ZrhECG+eZcv/ONf3QoyFd8C288
CrdJZ+R2BurWABgzCGd23wTb4qSvc24wcO87b3yNahTwu9/51oVJdH8ThGIKX4Pwej71UtHTLKp3
j+D4QUAsknkUJgK8cJrf+LEQSQp+HN2De3oOJ73v7ueOZhCbWbYBGiPMMqjFwZ0ivFbv6ml63SW4
qCMtoNWtu9Bz/9y+eZyvOZa/MovYpj3AsNDsV1CLdIswbumU57crazSTk4FtckPfdBWWWTzw893L
ujrMoUewfMLXn+kW9DTGqYH+qUwbNpGJpnqBvr6WCsowSd8cDOwdEcqhxn+cO0yhFdtmaMvnYpwj
XJ7+Tq4+npHL8QU4wMxWYFdf3iuw05NzBHMvr8cvAvcitp2c/kKuxh/wLcGmUfgqhYkXi9YWonXk
7NcPCEpbZ6LAaplVxCmc5ZTuhTOJPtAt69mKLf544HJVyUiCnJWre5nT3xKSO59MwtRhJjamqmWQ
3byE3HZHWg7DrFc3jS/65JCWO5QCd5LJvtXDnEo4Maht8E1CWe5gjG/dT37OQ94sieDRC9ME0gg3
t5jRzYKtFYsOgoy9AKdLgEB5VCH1/grCWxkCm9oNUdV0isVEhOnsb+yX2WQxw8k2bYf64M0WAiIf
ZM2Prs/gCOaLFIKwFar0vURnryG9EyFGpKfhwKYNQSdeCFlovcrYlmO8IWKmE9AweAzSu7U6aGVm
4EPgl72HaeD7GAE0vxni6FVRSdLZErZs6aJMbWLqRp/zreZTybRW5FH8b2VyAkMoUTwB2RU4M2TX
ljeoZ/cRbE+hrBQUF1iNiwtuRShiLP4ES2Et9lplbZV6rH+sq55mGzp9Jq7dQh/UouPSuo1MSBGh
Bs4OxXPovIEsxZv+bYgLq6fhrk0zsENcyNHCW8KtxUUb26riwpTztq1tZSmg9W3elBU3ZUUrrIqs
2OFm05rdV2A4E3WF8A5lwYWSBXJnyFQCRt7RDYutBcRaVchR7jALb1TbTGp/zeDMXFm9c3JXxMMh
x5AnxcPe66lzSPOAyTmw4u1hhbXbCEpQyiIfgCqOyZzUR8wLq+4ayCiKNlq7dcKpC6Ki+hJHmzLO
mhIbWVPlG2PAGxwXVtaUyaFpfMtnoEYO1Tuw7ItMSbY47ctf4cVfUf84EI8W/m/iHobD8i7/ThMo
W75EqXgLjyhFYwGJQIU/xw+0bfvRkzpHHWVjuXiMQIuDWU8zbNZfUeCeJw0Fe96gRE+CcCLKKXcc
KsVVLPwoznYNGoREbTQVk2Aq5EEo88oLI4lckd1RTm5yI0nxyOMr9VG7cFmfNMuDu/Y3SWWsH7w4
8G5mUhjG4m0zY0oirC3D9f9nuJ/JcPUxdjBcM4f+8wxXUNwTvPa6oKOM4XJyqP946zmNNXY7F51/
AOFdd2QKZW5kc3RyZWFtCmVuZG9iagoyOSAwIG9iagogICAxMTA4CmVuZG9iagoyNyAwIG9iago8
PAogICAvRXh0R1N0YXRlIDw8CiAgICAgIC9hMCA8PCAvQ0EgMSAvY2EgMSA+PgogICA+PgogICAv
Rm9udCA8PAogICAgICAvZi0wLTAgNSAwIFIKICAgPj4KPj4KZW5kb2JqCjMwIDAgb2JqCjw8IC9U
eXBlIC9QYWdlCiAgIC9QYXJlbnQgMSAwIFIKICAgL01lZGlhQm94IFsgMCAwIDY2Ny41NTkwNTUg
OTEzLjYwNjI2NSBdCiAgIC9Db250ZW50cyAyOCAwIFIKICAgL0dyb3VwIDw8CiAgICAgIC9UeXBl
IC9Hcm91cAogICAgICAvUyAvVHJhbnNwYXJlbmN5CiAgICAgIC9JIHRydWUKICAgICAgL0NTIC9E
ZXZpY2VSR0IKICAgPj4KICAgL1Jlc291cmNlcyAyNyAwIFIKPj4KZW5kb2JqCjMyIDAgb2JqCjw8
IC9MZW5ndGggMzMgMCBSCiAgIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlCj4+CnN0cmVhbQp4nO1YS3Pa
MBC++1fsqcXTWpFkW7Y7wAwkJJ0+mCQ4pTOZHtJgUjoTEzBtL/7xXfmBzaOtI0NPNgf8kD59u9J+
K+1CoyB/ywc4uaPwEGl9X/OSVx44lLjCYa4LruDEMTm3GPiP2snUoAYFvJ9qty3QDeZS0YJY/+K/
02ybmJ4phIkQ/kRrxbr/XTNsTjzX5pYJBiNMWFRw+RV7xxC/+hnAMoie4AX+LeLo25Tch6sOc2IY
z8Kbp8ndKuj0XkawHOuG2QonEA8+X3ZYDKPBBemdvu8ISzBIH0eDq47lcQZdXTMEtVopLbo7sPGc
Sw5ctW031rU9Y0oPoS8KV1RwUNaHC4I2Cc7TPsg+vWarCO4KJ8FrKPkyvV1k82IUGIdiZQrCuDDp
mlVvugqWcLmc3wdRNAsfYBxOUmZvchZFnwqILTnNVH5ggri26+H6q0Hv+vQTGb09J9eDK+gAE7rh
cC5aOTOX2KblbPtnY4HjguIcX8gORfsMfjQ8S+BPe5cI71/fDLL2KqYXcBlbe4vt88AwSEgWHQhm
U8pqc0Ne5PzDBcJRpOZyVxFLzkqBxephIc4BzZSak6kRwpUmJFkRqiQlznh4hogSvwhe3UB9pRVB
txefVIN2G0qMUbEEs/6xuo8a/Wstgib6m+ivEv01sZroV4r+HY79DctLgVPL8n6yg8PdWweW/R/T
j8EjGm1bvKLRFiWeZTKrMLrbhTJT6UHq5WBF80rkxgGEQTCB1RyiADkWE1NRvI4poQtoJLS2hDoH
lFB53jighNbc9DQS2kjoUSTUJKbNbLEb0DKiOZ6t8cy9P5S35TTfCladIni2pN4m8pKwGs7lgfNX
kA59P3/8OguDsqTnZ2N5qEdOSdlCQb5V+bXXhYpS3SIpY5iufExEG/+Tase6ALKe+xiKCgk+rF2r
SfRc3/bUO9rHqncYh6t3JAWhv5QRlOZKOjRRRoU0DVs5z1EzaU9qU7JkK6cJRQdvJDNZLVNnU848
qv4tZ0IlgzbTlppBCVD/jwlLDa9IVf3NVKUGl1+YqEpM/5+A7YZmrj7zMGqis3J0DnztSvsNVcpT
7AplbmRzdHJlYW0KZW5kb2JqCjMzIDAgb2JqCiAgIDgwNgplbmRvYmoKMzEgMCBvYmoKPDwKICAg
L0V4dEdTdGF0ZSA8PAogICAgICAvYTAgPDwgL0NBIDEgL2NhIDEgPj4KICAgPj4KICAgL0ZvbnQg
PDwKICAgICAgL2YtMC0wIDUgMCBSCiAgID4+Cj4+CmVuZG9iagozNCAwIG9iago8PCAvVHlwZSAv
UGFnZQogICAvUGFyZW50IDEgMCBSCiAgIC9NZWRpYUJveCBbIDAgMCA2NjcuNTU5MDU1IDkxMy42
MDYyNjUgXQogICAvQ29udGVudHMgMzIgMCBSCiAgIC9Hcm91cCA8PAogICAgICAvVHlwZSAvR3Jv
dXAKICAgICAgL1MgL1RyYW5zcGFyZW5jeQogICAgICAvSSB0cnVlCiAgICAgIC9DUyAvRGV2aWNl
UkdCCiAgID4+CiAgIC9SZXNvdXJjZXMgMzEgMCBSCj4+CmVuZG9iagozNiAwIG9iago8PCAvTGVu
Z3RoIDM3IDAgUgogICAvRmlsdGVyIC9GbGF0ZURlY29kZQo+PgpzdHJlYW0KeJytVttym0gQfecr
+m1RrRnDgEbgilQlK8ptN44vJN4qZR+wGNnsyqAAvqSi/Ht6EBYXjyWBgkpVEtNz5nT36e75pugg
PvE1HHo6XCfKsas42SsHHEq6FnWYAzajpGdSahng3iqHM03XdMDfM2WiwhKGo7/Ixfgtfs+gD5ZD
jc6/7gelS4ljd6llIprrK+qy4/6naMVbzSAGs3RGs1XEuTh5Tc7HZ+TN328RRxfmjTHOR1/2wsgc
EgD7OpQBHf+RQHIzI9MwRSDh38W7N4JfSzyBcHnyGrEE8mUQfl74XsqhJdzT8+pVmWlLsGHFV5GH
/XwVePFl6CNYfHw3+8hvW7rZ0UxmmCr6ORiUWbbk9cAh5NyHNIKEIz2vyMMmxC4xHZOx54gdzbB1
Jnhmu9d2u/LpaD2b2uqyoNFfh24J439O+7pQDsoZ67TfQzHnf1Hd/R4GR7WxrgcdpVU4tCaPOGxX
28FyxUh/VuNt+oJYNhkxKDN1utozUYezlMdwGkdTniRBeF0qqDwbWrFnB0Q1C7ZYMBixu7aDcd2D
XqmEsAYMhommlKlPzGzSNS18s0FQGtMpVVdlU9jn8E/daDQ8RXj3/PM4t2/jegGXs+3V2DYDq04U
ZjFjb27loZBXTCus+oDZC6syZ7q6vp+bwxenjVBEW5LFzBlWZ46G9wF9R9C6+PKpU+7GoktbW9St
wqOs+h83tVjUP64+ayO7vpyofdnTkfSmifrFmwc+TKPbqyD00iAKEziSWzYAHd9jl4rC+XeYRfN5
9CA6VeWIh2A+hysOiwjb2JVosnMOHrb/+4zOww0Ps1ElNoo+h6tRnENw/+iFPiuNRb4djuDPew4x
/3bHk1ROu26aLJArl9mWTItRKjt9TbmwO9hOpNhVJtJo47PjVhik8b7stANpRCgxqW3Q7gZ9vPjI
eYyi0F8pJApRBcH0Bhbe9H+eJuDFqJDplC9SvMmIP01k2pBGMIOvtRZsZPKstdKvnWw/JZZuW/XC
/yEVxMqFcldC3EMU95bcqj8FXI9Q2qWsthTFWTdxCDNNZhs7eKP/Zm/4I95LQm9e9iI7YuqFcM1D
HgtPA1HooXYT3SW84q+0nDOHNSmbifp+twwZvy1DsE7Rhq6wkfIuaejvQ7hMr8Z5i6zklOUT5+ST
Oz6S8jAIXIrGXUkB5uAAUvHaHZ0KOYRRulKEEMMquhBIREGkqpioND+lEknpKVlcxCn4c2scGgzS
hs3kU+zz+BAv7UEUB+l3iGawKC7w3nqqQdBs9DakUch5UFXKoFK0gzwZIm64+FS7/vZ6zSI4dpUz
5ReVZFYBCmVuZHN0cmVhbQplbmRvYmoKMzcgMCBvYmoKICAgOTg3CmVuZG9iagozNSAwIG9iago8
PAogICAvRXh0R1N0YXRlIDw8CiAgICAgIC9hMCA8PCAvQ0EgMSAvY2EgMSA+PgogICA+PgogICAv
Rm9udCA8PAogICAgICAvZi0wLTAgNSAwIFIKICAgPj4KPj4KZW5kb2JqCjM4IDAgb2JqCjw8IC9U
eXBlIC9QYWdlCiAgIC9QYXJlbnQgMSAwIFIKICAgL01lZGlhQm94IFsgMCAwIDY2Ny41NTkwNTUg
OTEzLjYwNjI2NSBdCiAgIC9Db250ZW50cyAzNiAwIFIKICAgL0dyb3VwIDw8CiAgICAgIC9UeXBl
IC9Hcm91cAogICAgICAvUyAvVHJhbnNwYXJlbmN5CiAgICAgIC9JIHRydWUKICAgICAgL0NTIC9E
ZXZpY2VSR0IKICAgPj4KICAgL1Jlc291cmNlcyAzNSAwIFIKPj4KZW5kb2JqCjM5IDAgb2JqCjw8
IC9MZW5ndGggNDAgMCBSCiAgIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlCiAgIC9MZW5ndGgxIDE1NjQw
Cj4+CnN0cmVhbQp4nOV7eXhURdZ3VZ279ZLe0kl3tk46TSeEELIRIGFJE9nCJoIDYUeJiCCCC8oy
DGEYZEkgIpgmyEhkGMQMo4gYI2hACCiCgyhxQRlkxAWN6OsAInSK99TtDqLP+L5/fH98z/N9aeru
t+qcU2f5nVMXQgkhRlJBgKRMnXXHnE8v/2UHIVnNhLBxUx9+KOV8UDUTkpOC5znT5tw9azybeYaQ
vBH41o67750/7cvHD8jiGN9xT7/rjnL409pv8H4Ir3Wbjhe0RjaKkPyueN5h+qyH5i0ZUZCD52WE
0MC9s6feQZTT9xLSdQOe95t1x7w58mUpnZCCT/D5lDkP3DWn6I39Kp5jf2oCYWQ6r5Gmy1uRWpUk
v0ok6sQHFepsoJq8lEkku/lkay6xnWw92ZoTbffa/V67d7pEQg9CQuhzXqNarvzwgJKB72BfZXSK
NB+26H0lBMxSncLqiCZTiWi2k72a8270443BXmK8ZZL32hnRmHkHK6nHPihZd/0zqbfcQpJIRsBl
BQdSZAi6a210hcfqsecyD8nx2EKtIUHUubyLrXZXIdKVb3fGujw0P69b9z60QJwqqoX6UtPSu9B1
R83WlPmjx8zzWs1vFQ4oqZ9xT/0t/YtYb9gQMs+aGNezV6+ecRPuhcuh6Z+/XtyzqKhnn0NEp6UF
aVGkhSSdzA0EoszMYnIlezQDU40uT7KnJMnjNpo8yVIMqaT7JWdlzH53lV2q8jfZazsmGU3JCSq5
NUGxlKqKM7V/R9vFZqT6nN1RiH+C+IutNn7pgu3SBYerEK/mDBlVptos3yJDqr4dm0pjuiAPSowz
Npl6aIxTEQwV6GwWdM1mXWhB1275ebHw9q11oxYtnPDy4JWrW98d1TDj7ldvX/DoJa3/5sdPvTVu
m1S4u0uX20YNGeyzxG9atG2vz9dUUDB1bEUusySvW/z081597hYir2XIawxJJP0DaSQWqLHSsFqJ
baBKVRR9Pa4quimqNglYos0Qq5DSRIdtUJLtIrLULFgKM3TOdgF/Fy84Cu2FOTTGi5SLKYnBqUgh
dhvJz3OoOkuqVBb6bN9zZftn3XNwPL/GT9GU79//sVFau2LZDhubNE55+c0ehS9nZtJCGk3NNMD/
eWjLszs3IZ0rkc5BSKeJuIgvEK1UOUiVuclR6zY4rP3AEdPHrRMVEfCFHKrLT5dYWnqew25jvlRm
tznY9KrHHqta/dhjq8//ePnr85cvw5lT77d8/HHL+6c28Xf5v/hZ/h7NokLwXYQuHCRESlecJJp0
DcTLNmrWGhRaRZ60KAeMLFolBlmJspqcQs9DqOpCItnn8kJIQh6KB4WBXCdRofg+u7fAK2ZOSj8x
dRhdwhc38haa+eyLirP2trunVoey4UT18Mbnwjooxk3Qx+0ccBssQADnw95kftJImUIGRxkU0wAn
spwneM7u1apbBI6Hw3RNE8qDB/n2PqgzsVLCKzPuX1LZ2Ji7/cG/Pcsa2gazhuCal//Wtlxxtm2d
MvWs0INdOF4/HM9AokiXQJzWQMzNxifJAYU1SDDURBV5KBliVC3I6LnWUC8xaGveuZAuat2oC3BL
82N8lNAP6KzQfXQWP0uTGxulKaHs6mooYbecF3wtxXE0+SnkKzfgtsiaFRqInR7QGoyayYB+R7E5
LL+UZuvFXs2teXadOSFHZ2xPIdW0AiFPO5tGt/BJk+YeP3u8XohTfoofqG6r+/2d67YdZVOqaR8h
S9QdP+pOHMkOxJsro/bbSGXc/tgqsFUZmqA2PtphJsqAeLTUvHYF4heFheb42zXYmxfrumGVQrcl
/+A/j+Qh/jH1U+m2jcNL1038+569O6Zs6FuI6tuDOvHXo1Pn1/sWffrO8bM9+5AIHQnSpF/Naa2R
HjCL+RyME6vPaag1T9eiyJyG3ZxQZTzwFej+7WDjzDl/XPXKK7nP3r9jO90uJlVMKZt/bcv2O8rP
3qRDyPevdejAf9ahc/9Rh2L+Fx2SpmzXVYgwihPMPseoE44DUWwL2SFtUWSaFA4EOleoqDk0X1dO
Hz0fwj8pnZ/ghfy4TjOdgTQvRZrt5Fa0ctUExA5BS5PhgGpUFKI5bCebw9K52HrymF34m4DfRmzU
Zk8hKTTFnkOwe1uOPUAC9BZbwD6CjKAjbCPsjok0PH0/8+SSlvZcULqj4ZVXuuxb2u/hbjC/S6cP
j7W9J035+JHFqR3CMpyF9MxGelLJXwLpbofVIKkkKUFRY8xVKdCUcCDOphK7VRumDLcPsw5PdA+L
7++zXRyy03z7kJ3228eXvULir+/vMTbUS8Qu9JhC0CjpXkKjUadFGAsMzJFy5BwlR83Rcgw5xhxT
cWyxq9hdHFccX5xQnFicVOypgAqpQq5QKtQKrcJQYawwVcdWu6rd1XHV8dUJ1YnVSdUeH51Idcbi
6E0T+OuZZFtT7xm8fPb2ggEjem4vHFxa+Mwz3qnFQ++CC4P6n+Bn2h5hS759cOHnbYvZku/niL00
ZUqv4gHtscKBPsKGsSIHvSFxxldZnFVaraWJPgkuNF420O4wReJDnj7dujcOG2+78go3jITQdveM
9MG4xsYuteVHz3/91l0buXXFsmWVlcuWrYAT7JafWlePGkPR5Kmddh/DTe+f+uRky8enxNxk49xs
QcygksSARWFBskKiReAhRTKqmx4MslG5w7ADvW+29CDPOsoz5Zb6qy1yZnh+K5GnXPl79A1dAvFx
L5JdsUGIetG8yxY01AjXkGcmuUpOPEKPdt8gGLr8n1xDmC90DbnDtozHOHKQFtOk8VuGDa4ffai5
+VDZs6UFGRm0hs6h99HajIzjvQP8Hf42/wd/J9Bbj8UIP6S5yI+GFpsZcNOgjQQNKxw2o4aOUc6L
6mEnHkO+U9ASEqaq84e4CkUbFqTfmyfFOKVMtGBa/XXVnzet5kPp7quU8evXvn5Lzm47vn7ZsnXb
Pvv49L/atof5xz/phDSFmKk90N/EgBkNRsCtySjJkkGWJcWlaqrskmRNU5kLmBkfdRFziVFmoIFC
Dpg0s8lo0BRZAkZNKsk+plPWC9220G4Bb+RvVZsWaeJYPoLHkSYfGZu6y2qkZGKgymSQTMkkmSax
eEiU4uREQ4Ix3pRsTiNpNIOlyRlKhurTOpsLWTe5UClU87Ve5v5aqWGAabB5tFZmGGMaa57G7oFp
0j3yDMNdpofZAnme9nvDA6YsqzGOJUMydppsTDZmsky5D+sj9zIOMo41zmD3yNONC9limC8tlhbI
vzcuNron0on2/AQq/lGfifoaDtb9ecvBBn5154u7dmJQe4SVtDXBymt1bFrbxjAOrhayxPmLRvuY
Eehh0FQwKn6QQPZLEpQoEokBKSZocAajVpgk2R5rUMAIdpJoAWNcnmTPdRo95pwkfX6bxQTbC/WN
Lk2caEeh+IUBoxBjO1z0UFe0D7pQMe92XQWUGyrBrgy/b+TRj3YNWzj19aO07jJVnm575/TaNcHH
2OvO2dv4dDr/6YltK+WWf767fi8b23bhj48uWYl6UYi8bEBeFDI0kEH9qBjErxAQO0pQNRQ/6kYJ
keluUCSgRGbZco6KpOuT35p3E5k/z/zYVL+QJs1n0+nkE3zIJT7kRL2ceRVNUvh+PX7QXIwftgay
hdEkItlO6khDDxkiXNBcPVAA2Yb03a/bip2kkN4BXxwJGg1BxwoaNL6QbDfFJUdrTCYWT6ycl5hr
IB5HjjdsOCf1KKdTqdty4c/2AzcZUixKURXoW8qg9P7vdtY8tf27bx5fumQ9H0z3fH5l6dJ1z/DL
/Cc+gB1pO71w9eMr2TTeZ86i+8u3HX5p5VPO2ON1R4+jHJEnqQzpNJAOGM+Cku6lXsDkKFdFT9XV
GJ7scGBELNWeItl952F22wR2V9uTR9Ff8UH1bd11e63D/oxKMqK0tEC0gYCxwXwIGmScBKKYhhos
wvEdC3OHPkL43V/gJbZ4/NgPzv/1Jf4RPUNrfv+HTScPwE+PYb/T+WhpCfoBB6kL9I9yWaKEKzAx
g9mFc253OYjD5rLaiNjZbVGWKKvLYokqsUVHEYuhSj1otxyIttusUUaVqA7NUWrpH41BoFl3UuF8
rfAXqvsLb6DnPniKjqCHg9KJgXjJ9rCNaXbNEWeOi0qzpFl72EvtpY6xUcaJZCJVVIZZXiYtEN6W
5htoPozrMnBqapdVE9bPum3CQH7rQTqUDjxIp8/dx68UDR++cVKzNDy0Hu4N2+ky9Pk+PZfIIPcF
OiHN8Slmt1UlDW61yuGtTNmXVNVB5BZR1C3FWYyKuV+KpMT06SRYEoHNrgu4+dzFkJg0Ed/syJ1I
5gJJOck5KTnenNQ6UkfrWJ2xzrQlts5V566Lq4u3TLwJQRZ0bwdzmBT0pAXhBKkgDE08lDX1fvov
C+7d8Bx95ZWeL1b87di1f/9IH103af/4aXvLKg/1Tkth+ffPuWvOe3syhrYt2VY++fUtew8kPTq/
W9fG9PSRI/PWhXldyAdJjgivgwLpcWZPQroBHJXRsYYqT2yVFZBTj/VgxwOd0uOIYh6oOBzeQZ3C
SauI4LZz55rDTNpaLoiMT9epMKjQ+Yh2sp/T097spgDvKsB5Wb5k+ZrqFYuXN174cui2UXfW3fLE
is4bZjV//XXzvTXZjazw+AcfHD/+0Sl+OsRDiQkNXTpv2aktmjSBFlGVarRo9Li1epxiNqhnTWhH
QBwvUzQhIrGwc0BSdcTKmp6TW9paWGaY7ybU59uRb4FV8gOJN7BKk6WWHoQDSYhTBuqIZYBAK3l5
4dTx3A244r+BmMK5Y9rNPNNHEDMiXPnyq7em1dLvly97dNWqR5ctX912RDFWjxrDD/Ov+Xf8yBh6
6f1TH7ec/ORUmKbXRa6N+MkUxk9uswGqrIaqmFprU8KTccThGOg2K0r8TRTdjJ9uHv8mutJjkFCo
WRGmYUXbZ52fnPbWV+ePltd2aWxk2REC2LyRZfwIEvUNPzRm1GokAm3+A/QlD6JMZeIOmMPwyQO5
pKuCTkkMfq41JzpfeKMPjrIToXK55WpLvXivjKyW5kuPYGzwBhyKjDFBlp6jf1cV1lEi6RgEerUK
s7/YKmo0BYjcCqhe8oGR8HoDn7SNT3qZ9sZ+LuP4A3B8I/oyJwjfKK9QEfJoPsUDxEdNOMHNYTHk
6UDH77XLBX6dJGqhRfw5OuIt2i30Zr00d2jjYJ06IedpArNjvwy9ZEkgmUZBFEauqBICJjUoU1hh
oGYj8WiSYjV3sNhCoZM6HBcci6N2PyVF0ImBeiEflcGnp/DsDM+gH3i/fuONI23L5aTQN/B2KH8r
30TL94mxV+LYIyNxqWMgRkEAR4JWhVgNKuSZc1WPLceh17HshZEghAmRP6Wb3Zbm9dmjdUSZgVB3
N7n+1pHrZEo5HU378Vd5PT+x7RodRIdcu/aInM3X8wr+R/6EHjfJaBxzPh6aybJAZw0dtyoQHGXU
wBgtMaoMiAZkt0k2aAjRZEykpGyjQnKiwoSE4Vpr3g24puMz6cjPEojgsw6yZjS4qBtcmtuAiAzS
tAxDN1oI3bQeBotVxZ8REDdFe1FkKCyM9HbfaJwpRiVq3csv1/JLe9BINXYFQ3566FuIvvoR0t+K
9O/S7eJQoEhzGTRZkVWXosgGxJsa4k9gsktBvMkkEY1KNDyEA+RJo6Ih6FQMVFH6kf5mVDicxgg3
51y/jDb4T/q2/Qw50lQRaEYqTNNimVt2awWsq9xNG8CmsbnsYRlNU9PiMKdxy3FKvOrWOkK63FHt
CT2l7nKh2l0bDKXqWGWsOhPuke5R7lHnwzx5vjJfTcQAhUYjopKPqnZfaxP97FM+kGI+cOfKJsUZ
WkhP8HFtA1jJEl6M3kzEoqGYf+h1LWoMxMNfDY6gxxx013hqO8R4EhQvSUi1erzJHYRVnkRv0F7j
OnkhJ/B+Nsmm2SwbsqVsOVvJVrO1bEO2MdtUTIppMSuGYqlYLlaK1WKt2FBsLDbdSm6lt7Jbjbea
JpPJdDKbbJxs2kw2081sM2yWNsublc3qZm2zYbNxs+l58jx9nj0Pz0vPy88rz6vPa88bnjc+b9pH
9tF9bB/sk/bJ+5R96j5tn2GfcZ9pwG8RExkKJkuT5cnKZHWyNtkgBv6tjjqgElG9xhkufkZHQopw
eTcX8+hnw7r1GHFrYY8hy1ZXVq5eU1m55rtLl7777uJFdqH7iBHdewwfyjZhknWEv8XfoTm0G+1O
c+r4PL4EjWceXUH/QBfTFTqumobzMALnIYn0Cnjj4kksxAcTf5ELehLiPGbSLTpPKdAr0c0CtkUS
Qn5SxEZ74f+QFWKw90ojbjv6MD/Hm2mAJk3aMeK2+smHDu07fMtdGR/RjQuHDKNBOlskhwXdjg8r
5ccxNTzCW5K9dO02ncYNaCc+pDGR3BbIIIl+NBN3XDy4EvxoKyU2+zNRQWeNRIKM2IyMGj2uVBt0
wHQCfej+/eHag3CkF4+1W4aOu+RvXXkIzEUJrxsRZXQ90ISBiEglEGrRuey50Ny91F1QPmBdxfg3
59z9xh0fU9PY8h4t9fX1h2iXPguCty5aU3LLsdy8869N2f9Q3y8EvUeQXgX9kon4A9FGEWA0KajI
tDt4VNLdYEb3nheudQmy8m6AX1FKPAK9+Ru0MHSQFvI3EAFfW1xfLy0J56zCxw7S+8W4oQXZCxJZ
YVQQT8u5BuohOeYw0M/Ti1p4oEcxUX7ArW/lUZZx9GjbBwir2zax8quZ7I22wnC/5djvLMTACrkr
kIG2L0vMRZksdsAUomDeS5QS9KYHZBH1qCyVkv6qHuf0et1v5Dy70pnwnhaExlQDTZJk5mdM+EkB
XamvfB/tTwfs4w98i2nl7bDjWp3gEfWxO851RzIQ5zo6GGOsNGyLCirJlSnbEoO+GqU25tmM2GgC
zjhPms0DqclOQ3IGznVzJMNB3Wy9Ua3Qsxw9mbl5KSEtDKF/PeGgrdvEv7l09/t3Tzt857ZduzZs
3Fi5ae2jY5umz3+t9BSVV0Jy+htP/OObtA5HCrrWrP5j7bYFsx5c2LHjnpSUj19auPXnuspgpD+d
/CnQW19D8Xsiiyj+5GRPib6CQmNedD7jDtrpi+QZKeivsdd29IQXUNISCi25TrUwNacjTmbzrxZQ
wkzdCFfhxZP2dHiXxoRrF5kIxdCnMipyhl8tqfxSDtlUrKl0EGsqFye8Mql626wnZry7n18J3XNy
5oNvT6+tn7v23ndepVFnRjfJW97u2WvZ/VOn+9y5777U8mlW1gel/VYunPNwSlx2U92b/5UueI/g
GgOZFUhUMfoxRVVKRBjejaENibJLuaqd5BjbgUA4/hbehDskEXVdVDCS5mJuNYOlqT3UMayczVDn
sgWq0a1gAFYG0FJlDL2bTlc0EXe8osiGG18lJfTKweuEawfllmv50ttXM6W3r+UjbUORtmWo4yby
bGAQRlVVwZgqqWInS0gquBAzmFz4pNFlMFKxMxlVTTW4NE1FLCFRSUP9Z5EjVqopIvKiM4xE3nN5
rt/M835lF6qOKjwYv9Vp9B71YTpfVXDOVMUYYyySuhrHSaONGtqJgfkwmhpETJUm8dm0sYU38lda
aCOffYxm0HRpSttXbY30dV7MBjE3n0mfuKm2peOis4F+aLIGo0EyGsBoQu/ITCaj4helLX+4tOUP
l7b8N5W2dv9WaSv026WtcElLDVe4kEe3QczhJFF6SjaKyJhpFEG5h3EcGyebXKY01gn8Uie5o5Zp
8JvSzAWsCAqkIrmnVmToaupmvp3cTsexMhgjjZbHK+PVkdoYQ5lpovlubbp5PjwszTc8bPLh3LfX
qkDUqg5V17+3o/rQmX1H39qH898PUkOfSE1XM+GRkKjtDILt0kq9tuMOGEHCdEoFKqu2k8d02H4M
HZkubx9lr9M3d/G1/PFd9E30l+/RSj6XZYlazTp+lv1As/XcwQR/IVsU7CiJKGKl9xj2EhI1mxjw
RdPLoUMbp9Ls43wJXRyel2r0DVn6Wm/vQKqkJsUHVXulbY0zGCViRFStWu8BD8Ec2JhKbMkeEcCa
I7jnXMSf8WZE6Xq8EF6LxDjJLwxa+LPj7GJbc+aYzl9RG//sx0cODZ+w545nXtr7zG0bB4iKylqb
lV/4upV/n5Lydl7uzi11u/x+gd/5SCkZcxtTOG+8kaM1WWsTDsYdSNIztIGYq/X5ucp9c954c1Wb
/jIZtvvo0OXLlq1cuWzZchafVXvXkfNfvYVp2iuvsEyRprV8fKpt1aixiFHsNJYWjRlV/dNlXV6i
Ljxcj3cZgdhIDUkLSi8YqWzQy0j54YjX3HyjqPVzIQnbIXi47Xa2pu0Bti/0iOB9QH3bZ5FcyY99
79BzJSPZEOgbqe4xvyyp/kiVz+AHo4aN+o1gJH6GuZQRcyk1SHdiKoVJBToPtA9ikLONJtsnrRE/
EGptT6QithH+d9ORDsRR9UTFx0ispBp/ElATk8AsGVRFe5SuZMKxUfCKQAle8LHpLbSev3CJHjlx
X9ule0/IvjYJnruaSZfxhYKfuahbE9rxdCDaELRCMKbGWusmeeZuSp6jwH3jK4LwOnE7lkR0GX3T
MaxcvrF2+fLajcvf+7Gt7fKPobYf2XlaSuP5F/xldD9f0DhayhfwCsSQK+hyWsEr9PWrcnoGZrGl
iOztDaSOoXFJtpPN+hcQOWIZmJYzA3a1dGskTnTX7bAgECeLGiu6AlkqkXF+QQZMmlJJsqipNodj
BGaKNxyrcKA4x2ij3krY0fb1e0xrK5BbRl9dIlY2GB0J9bBYr5eoJClgkelKBEkakZgiSWJ15BO9
dIJRRy+eiB8sfu5EpISil1EYWYyyHKDLsgPpGUiNVkgw9m+2qErzGlswVQkm1qTW+qMVoMmpBo85
LS7Zbwuda9VLB7Z2O710QXwR0G4QGH5Bt1GHXTeNgq6O/BSHQMupaez2JevWLXl0xfKzxWuG7T+U
XX/fh9/++59U+oGf5t+Urmc1e7Zu3fPC35/b2bZyjz+demlc+UxqvPhf1MBX8dl8OX8wWdjKUZSp
TboTY29ZIBHE2oXmYiJnbCAHoEHGPFiipUTpbxQlrkghr/XXgt3VQxNKGdMNemj9YbA2Bsq0u2CG
NhcWaMaJQuwCpVLvZ5DVtogtDe1lS9uekO7cHvqkph78us2225VGxga66NPK/GEsiYFGIxoGGqIJ
LLlbwXQdQ69KpF6yh/RSDfralV2vJ9r/x2J6GD/i1Pk/oPfSmR9xrwC1s9i60JG2O9lTN7DyEd13
BANezR/Orf2YLuhJNNkdyaHpz8DZtr+9IPCLECeAlSSkMlSVY2WXlianad3YAPY7NkYrM5SzBfJ8
baHBwlBtVZkgU+KhdLmzkqmmaUVQCgOVgep4GCOXKWXqGG0mLIB5qutX2TKC80e+bdvLRl/kcRsF
M9PZhranQqtZ69a2Rr0O6OBJrElJitQBX7qpDhg6FqkD7lCSrqDcw7HmF7UqlD4lMjyn/B1RGeko
iVoVSre9VmVgolQlzQ89ByPFB0rwOt/PD75Mt2yjW0RfDpRlky7Lw4EhBj/iCA0RhEns2uWKecGv
5tpETGKuTSUmlLiEEjcoJkQdTCZSTyWb9DQKqZ8LnbP/Sur/O3zqrqcVPaJkyeSW02lHyJDTTGWm
eWy+/IjJhPbLrJpVc7J4zYv+PFPzmwpwtsZqZaa72IPM1p6AREL96I/oIDr8I96XnvmIP87nnqIX
Uf5vs/y2ktB51O9F4NL9K/9B9692PXqnEI/Nbk0IQoyeIts9drfNbCVuBzpbd3uiLBCuXVdlW7Pu
F8LLWHKq4rspr1dd6s3+l1Wr8drVH4XfXVn5ZtDW7o0TGK2gj4YdLp/PT/HaQU3l8OMvPTP64Xr+
A8xVHCSKJAesyibypCVKJeBAyGMUayR5YditwxSFxTgdLl+acEfdYe6ypUuX1QXXrw8qjnO85xef
86IvLtCDn56hza3Y7wTsN729X1X0q1KTRtySw4T9hiIuJQx/Yh0xTqb6ujkKujI6Qe+zDntXHK28
15mzvFfrV/TwZ+fo4bCe3o96ug51Kz3gJE8baJ38tApmFi+RRC3eqJjFBxT6dzeRg8jXML7wVzCv
0tf4Ldt5P/qqtI73qxcH9fTVyFqHqC+50Yf3DvjU5LigMTloM/5NopVkjRSMrbHV+lM9JC0qVVUS
abRw4q2t6MdvZFqfh1c4cmiknKHXnpExXwoL1zlu/tQKzgjvfThHeO+Lp3noB9Qs5+D1/PSS9evR
ua+Qd6Pz5mf4l+Uz+Y///oFfpnPpWrqArk5uu7fdweu4ZA8fDTbFiXK2kx6BJJlQY5VGG8DWbHkS
DmgHHSaNGZhVUizEoX9b0twc+RxN/ygpDMZkXThOJZznihIuK+MH6aUX/8q/HjeusdEJn1dfqzr4
3qZFyl/Ph/FQIcrrEMqrE+lOXgzciYrpsLrMUWaLKyrKnNWZdckUuVFefteCbnJud8zQSEl3s0FB
+GzNjXKTTEhVc4OOVK8/6Klx1KpRpCNVFfCaNal7ptfdMcHqlaK1jg5vghZlS+jiiOmBgAnhre1w
eBnJoX8UgvYRWfc9fO7yhff/gbfCHwv+x5T3plU0UawWFZxsxN5ORXX50kWcJQVdu3Vv/2rw58+Y
VAuNLEX1od2Zj9alpwHdOnhIfcvwUXtmfMi/WFWZl/PG3/suHdBzUf875hR1mzD+zSc7Teic4LNB
ZduJ3JrpKQPU9Kf/Xfb5rAH9aNSxlV+XD1o1sOY5t+vldP/4YT2XHC/9y4TgG26X1RWlr9kuRdkO
1/GEj2STWwJ+t5kE05WgJyvoqPHUpj+b4zZ36OSJ6eCxGjwxCangsXqTcxD+t+r4X5/TSFlDnAm0
fZPq+ds5vLFw60vtgFeib0LkbPrydU8sW7HuCf7WkrU/vHPih7VLajZzfu4cv755WMX8BRWLF86v
YIeCq1bVBqtWbrjdu3vxrhMndi3e7fW+ufmtc58dqTtC75z3hz/MW1ARqUWd5rvgIuppBzItUBBn
jTEpfkO8LSbJJKd4gZgaDKSBHja8EdMQ/ZLfbDDKHWLjSJJRjmZOkhLX22iV/T9/7BaO+YWR7ypD
YT5x4tuVQtzIkcVabm/aXrzBo249I18Z6nwniQwELgZ+enZmVZ8+q2c8+1NgQNXvxt83e9zvqpoe
W3/6u+BD1Q/UfH96XfWYNVf+vCYuYc2mK9WjIzXQEh2PgsjpmL4cRJgHJP3r3chKm4Dh85tCFzCv
/GnWL/ifEeie7E+INVu1BFNMnFWSU4DENSSQBt/hhDesDfaX/Ilx8TFW8VVwfIzPIZH45JjeNmKS
dAnkiZza8dsyCC89huWQ44owjkp9QxKxSe0r3bokMunpfqtHj5t93/jRq/v1+Wn7zNWBwOqZ23/q
01S2+sqmNQlxa/58ZXVZ9brT39c8UP1Q8LvT60kEMYBeOTAjsBiOew+x4RULWUyu01H0DjoP8f7j
7DD7JCUtJSelKGWHN/X6dfGNNamjI+kUvL8ocj8a7xfeuP/bfxTH+IRupJvoU/iri/wO4+9N+ibe
F1+F9448O4QkkwFkIB4NIgWkJ3r14SQDZZ+J9uQn3Ugp6UWGkj5ILyG3YM8+/a1CEkeiSRrpQeLR
r2WRznitL0kl6SSXFBEn6U9iSQ6iG4Y5Uz7pRwI3aOuqb5OIF23WTiTMJTyIrAxkMOlIElAyUYir
UtBTWlFGRpRZFxJD8jBzLibDSAm59X/k+/+3v0bMTUSrp2txP02/spwtFjgt8mskB/E+059rpEfp
SroHj7eREG6Xkh+oEd6g3fGoCd8tk7x4tZps0t+shi/JXHiVvEeOkI/x6EtaCPgufY946RnsbeXP
o0ATnh3E7UJogjKaTGeRrVR8dLwQx5xNFjPcs5HY89vSCbz6NlmOv3VkK5mNx4KypUj/abIbs9eL
ZAM7T8bh8R5yCOnhqHf6GLSFXMae6llvNg2fO4S9bSQb6VLSQh6UMJLjk2dlkWNuxXc34Sh3kk1y
i7xByAP3LRgjRBk5SWlUnKoPuRBy20Zfpblok+/h+wvJ7TAB7oeP6TLJJz0C50k1IzCFzCDH5Rb0
RtWqj1Qr0+h8aYr+Wyj4Y49IU2g9OY993glX8NyLlG3SOSZkNxspD5eHI8/T8NomfVsd3io28jZc
RbmvZZwOkgZAMd5ZKA0lGwimBWhFs3E7Gwpw9Nlkobw6/CP1+MuSV0MN9q9Lg+az3mQTm4bAq5pd
RmnOhn5oO9NIknyBLKO7xaK8uoiIFXk0ypfVcNmRdE6x7WT+0vKdgdvKUt4c683q/KvTFJuaspOM
2Bk1P6Xx+vURZVKCPHannLgT/NpOye87+1s3z2Z1HjKiLKWRuvr3i3Tbf0o/vDiqDA/FGV7G6/37
6ffEqDtlP/4rnbIzZer0lFW2Vb6iVba7irJ0j8nG1v5r3re7Jlt7XSLJ4fTrxJ9+jGrfX1vVRuUm
TSy1iJt6dVZs1Vk8SdRpr6267pSbItd//huBMzIdWxm2ddhasC0UmoltJbaDkbYL29KbzrHR89hm
4PGsyDvZ2CqxHYq8X42tMPwc2YZN7Osi4y0Lv8NsuG/C9jq2DyJ0XBZaEhl/NLbWyPPi2gZsRyL3
yiP7ykgbGhl3EI65LjL+ygg9fmxz8Xp5+Fk6EveLsR2N3MPnmCMyviPybD22CZFrYvw9YX50OZyO
0HM6Iseh2B5DgSfiRKH82c5wg47Y6kTNBNs8bK0YcB7Cdo4Q5XZsG3CG8BkNI4HWTIgB3zVgqDTi
3oT3zSnY8Ny8n5Co6dgwS7dgX5YQIdavCLHhGLZPCbFjnxjyiaMXtj9h+4SQ6DJsx1DXqwiJWUGI
C99z7SbEXY7tVULi0B7ikLb4KYQkTMB2hZBEpCsJ+/YEsP2ekGQ3th0YfpGelA8J8WKfXuzfi2OK
T7hT8ZoP3/chLR2Qtg649yMvaT2w4TPpuE9HuaTj9Y44Vgb21wlpzRxESGccv/Mg8f+xdK0cwcwY
Z2dixGMY+WqFFstjqB3js/QKq6Axu9ZPlPsm0hhSQwC3Ffr/qeJ4HK1vHRhgAR8XxzZ9a0WnCNSi
H0ft+mag3NdPo8giPDNjMAdqwlAK6OxEfwb9KQ0dLFBVP1b0Z2T9WNKvg36F6VdoYCwHzgHT9hCH
axyu5sFPe+HKIvjxcpX8I4cf90uXL42VL1fB5Qrp0sU0+dJYuBSQLqbBv3/Ilv99FX7Ihv/i8D2H
7/LgghO+rYFWJLGVQ2vj9ROB69I3A+Hr8+Xy1zVwvhy+4vDlFwnylxy+SIDPOZybCZ9x+NdeOPtp
nHz2KnwaB2dq4J8cTnP45OMY+RMOH8fAqRr46MMY+SMOH642yR/GwAeL4P0iaMGTliI4yeG9d43y
exzeNcIJDu9wOL7KLh9PhH/EwtscjtXA0Uq/fJTDWxyOLII3ObzB4TCHQxuj5GYOBzkc4PA6h/3Y
334n7DND02t75SYOr706UX5tL7xWIb261y+/OhFeDUh7/bCHwys10FjdV36ZQwPuGq7CS9jXbg4v
lsOucnjBAjsd8DyH53igDf7OYQeHvzmgnsOz2y3ys3mw3QLPbLPLz3SEbXb469Ys+a+LYGsW/IXD
Fg5Pc6jbHCfXlcPmp2zy5jh4ygZ/NsImDk/iIE9y2BgFtRu6yLUcNnSBII4frIGaJ/bKNRyeQN16
Yi88USGtf8wvr58I6wOY58PjHNbi+dq98JgfqlEY1X1hDXK7xgmrTVCFF6rKoRKFVumHVXZYyWEF
h+UcHl1mlx/lsMwOf+KwlMMf7SXyH0fBEg4V82DxHxbJizn8YREs8sDvOSy0wAIOj3B4mMPch8zy
XCvMbaQkcEp6yAwP7ZcedMCDAekBDvdzmMNh9n2j5Nk1cN+sjvJ9o2BWR7iXw8w8mMHhnjyYfhXu
3gvTONzFoZzD1Ds98lQOdxKbfKcH7uAwhcNkDpPGmeRJFphYDhPehPF4Mt4J40yAGl3mhDEcRnP4
XUKc/Ls8uJ3DKA4jOdy2CEZwuNUJwzkMo1nyMA5D98KQjjC41C0P7g6ltzjkUjcM6u+WB3EYiGcD
y2EAng3YC/3d0A8v9OsOt5TY5VsccEsjCwQMUklfq1xih5JGRvCsb8Ai97VC30a6H88CxWY5YIFA
I63As2KzQS42Q3EjDQTKpT4ceiMJva9CLw49O0IRh0IUcGE59MiNl3sMge4cumU55W4cCoZA15x4
uesQyMddPoc8fDCPQy7ezo2HnHjIxqNsN3QxxMpd9kJW52g5ywlZjUwM29lmlztHQ2dBbo2U2ckv
Z3LohE928kMGK5IzOHTkkM4hzQr+2BLZ3x86WMHHIdVqlVM5eFOyZO8iSMmC5CHgwZE9HJI4JKJs
EzkgvpcT4iCeQxwHNwcX9uAaALExWXJsCcQ4bXJMFjhtEI3PRTvBge87ONiRc3sJ2HAEmx1sYdlZ
LWbZagVrWHaWKKNsMYMlLLsolF2UEaJQdrslswHMQre6SyYORuTEyMEQC5oNVA4Kdq1wkJ0AyBxc
RXCUJbMioEgAzQJiA9pIy5etppn/7/yR/9sE/B/+JZH/BjiTOhgKZW5kc3RyZWFtCmVuZG9iago0
MCAwIG9iagogICAxMDkxNAplbmRvYmoKNDEgMCBvYmoKPDwgL0xlbmd0aCA0MiAwIFIKICAgL0Zp
bHRlciAvRmxhdGVEZWNvZGUKPj4Kc3RyZWFtCnicXVTLbtswELzrK3hMD4Ht5ZKMAcFAkV586AN1
+wGyRDkCakmQ5YP/vpydIAV6SDgeLWdnVhQ3r8cvx3FY3ebHMrWnvLp+GLsl36b70mZ3zpdhrHbi
uqFd33/Z//bazNWmbD49bmu+Hsd+qurabX6Wh7d1ebinz910zp8q59zm+9LlZRgv7un364nU6T7P
f/I1j6vbVoeD63Jf5L4287fmmt3GNj8fu/J8WB/PZdu/il+POTux3ztaaqcu3+amzUszXnJVb7cH
V/f9ocpj99+ztOeWc9++NUtV+66UbrdlqeoQDJelqpM3XJaqjuQjeNkaLkvhe/I9cCSOwC/EL8B7
4j00lZoKnvoR+on6Cfoxk88F+4beGtQIawQ19BzhOVIzQjMQB2ClvkI/0HOA55Sok8BTM0Az0H+A
f6EHgQdljaJGWaOWsWXfFprMmJAx7sjvgNkropcQC3Cin2QzpH40fWoqND09eJsDNT00PTN6y0is
hjkTtZmcqXlGPXt59FJ6UHjw7OutL/0r/CvfneLdJeZNyOs5T2/zJA7ASp8Kn4l7E/YG9go2Z9YE
1AgzimXke/d23pgxIaOSV/BKXo3neVOcN2FeQV7PXt7mzFyCXJ65vJ09nqWEsxTIB8vLWSlmlegt
mTdiDxyYK1gu+gnwI+QFvFBToCnUFJs/Z+jtzNBDhAchLzZb6nibOfcm7BXmFeRNzFsWfNTvXy8+
b9xDH/dGe1+WcmXYZWV3BW6JYcwf99k8zdhlf38By8os7gplbmRzdHJlYW0KZW5kb2JqCjQyIDAg
b2JqCiAgIDU3NgplbmRvYmoKNDMgMCBvYmoKPDwgL1R5cGUgL0ZvbnREZXNjcmlwdG9yCiAgIC9G
b250TmFtZSAvWEJRV1JZK0RlamFWdVNhbnNNb25vCiAgIC9Gb250RmFtaWx5IChEZWphVnUgU2Fu
cyBNb25vKQogICAvRmxhZ3MgMzIKICAgL0ZvbnRCQm94IFsgLTU1NyAtMzc0IDcxNyAxMDI3IF0K
ICAgL0l0YWxpY0FuZ2xlIDAKICAgL0FzY2VudCA5MjgKICAgL0Rlc2NlbnQgLTIzNQogICAvQ2Fw
SGVpZ2h0IDEwMjcKICAgL1N0ZW1WIDgwCiAgIC9TdGVtSCA4MAogICAvRm9udEZpbGUyIDM5IDAg
Ugo+PgplbmRvYmoKNSAwIG9iago8PCAvVHlwZSAvRm9udAogICAvU3VidHlwZSAvVHJ1ZVR5cGUK
ICAgL0Jhc2VGb250IC9YQlFXUlkrRGVqYVZ1U2Fuc01vbm8KICAgL0ZpcnN0Q2hhciAzMgogICAv
TGFzdENoYXIgMTI1CiAgIC9Gb250RGVzY3JpcHRvciA0MyAwIFIKICAgL0VuY29kaW5nIC9XaW5B
bnNpRW5jb2RpbmcKICAgL1dpZHRocyBbIDYwMiAwIDYwMiAwIDAgMCA2MDIgNjAyIDYwMiA2MDIg
MCA2MDIgNjAyIDYwMiA2MDIgNjAyIDYwMiA2MDIgNjAyIDYwMiA2MDIgNjAyIDYwMiA2MDIgNjAy
IDYwMiA2MDIgMCA2MDIgNjAyIDYwMiAwIDAgNjAyIDYwMiA2MDIgNjAyIDYwMiA2MDIgNjAyIDYw
MiA2MDIgMCA2MDIgNjAyIDYwMiA2MDIgNjAyIDYwMiA2MDIgNjAyIDYwMiA2MDIgNjAyIDYwMiA2
MDIgNjAyIDYwMiAwIDAgMCAwIDYwMiAwIDAgNjAyIDYwMiA2MDIgNjAyIDYwMiA2MDIgNjAyIDYw
MiA2MDIgNjAyIDYwMiA2MDIgNjAyIDYwMiA2MDIgNjAyIDYwMiA2MDIgNjAyIDYwMiA2MDIgNjAy
IDYwMiA2MDIgNjAyIDYwMiA2MDIgNjAyIDYwMiBdCiAgICAvVG9Vbmljb2RlIDQxIDAgUgo+Pgpl
bmRvYmoKMSAwIG9iago8PCAvVHlwZSAvUGFnZXMKICAgL0tpZHMgWyA2IDAgUiAxMCAwIFIgMTQg
MCBSIDE4IDAgUiAyMiAwIFIgMjYgMCBSIDMwIDAgUiAzNCAwIFIgMzggMCBSIF0KICAgL0NvdW50
IDkKPj4KZW5kb2JqCjQ0IDAgb2JqCjw8IC9DcmVhdG9yIChjYWlybyAxLjE0LjYgKGh0dHA6Ly9j
YWlyb2dyYXBoaWNzLm9yZykpCiAgIC9Qcm9kdWNlciAoY2Fpcm8gMS4xNC42IChodHRwOi8vY2Fp
cm9ncmFwaGljcy5vcmcpKQo+PgplbmRvYmoKNDUgMCBvYmoKPDwgL1R5cGUgL0NhdGFsb2cKICAg
L1BhZ2VzIDEgMCBSCj4+CmVuZG9iagp4cmVmCjAgNDYKMDAwMDAwMDAwMCA2NTUzNSBmIAowMDAw
MDI3MTM4IDAwMDAwIG4gCjAwMDAwMDE1OTYgMDAwMDAgbiAKMDAwMDAwMDAxNSAwMDAwMCBuIAow
MDAwMDAxNTczIDAwMDAwIG4gCjAwMDAwMjY1NzQgMDAwMDAgbiAKMDAwMDAwMTcwNSAwMDAwMCBu
IAowMDAwMDAzNjAwIDAwMDAwIG4gCjAwMDAwMDE5MzMgMDAwMDAgbiAKMDAwMDAwMzU3NyAwMDAw
MCBuIAowMDAwMDAzNzA5IDAwMDAwIG4gCjAwMDAwMDUzMjUgMDAwMDAgbiAKMDAwMDAwMzkzOCAw
MDAwMCBuIAowMDAwMDA1MzAxIDAwMDAwIG4gCjAwMDAwMDU0MzUgMDAwMDAgbiAKMDAwMDAwNjkw
MyAwMDAwMCBuIAowMDAwMDA1NjY2IDAwMDAwIG4gCjAwMDAwMDY4NzkgMDAwMDAgbiAKMDAwMDAw
NzAxMyAwMDAwMCBuIAowMDAwMDA4NjA2IDAwMDAwIG4gCjAwMDAwMDcyNDQgMDAwMDAgbiAKMDAw
MDAwODU4MiAwMDAwMCBuIAowMDAwMDA4NzE2IDAwMDAwIG4gCjAwMDAwMTAwMDkgMDAwMDAgbiAK
MDAwMDAwODk0NyAwMDAwMCBuIAowMDAwMDA5OTg2IDAwMDAwIG4gCjAwMDAwMTAxMTkgMDAwMDAg
biAKMDAwMDAxMTU2MSAwMDAwMCBuIAowMDAwMDEwMzUwIDAwMDAwIG4gCjAwMDAwMTE1MzcgMDAw
MDAgbiAKMDAwMDAxMTY3MSAwMDAwMCBuIAowMDAwMDEyODEwIDAwMDAwIG4gCjAwMDAwMTE5MDIg
MDAwMDAgbiAKMDAwMDAxMjc4NyAwMDAwMCBuIAowMDAwMDEyOTIwIDAwMDAwIG4gCjAwMDAwMTQy
NDAgMDAwMDAgbiAKMDAwMDAxMzE1MSAwMDAwMCBuIAowMDAwMDE0MjE3IDAwMDAwIG4gCjAwMDAw
MTQzNTAgMDAwMDAgbiAKMDAwMDAxNDU4MSAwMDAwMCBuIAowMDAwMDI1NTkyIDAwMDAwIG4gCjAw
MDAwMjU2MTcgMDAwMDAgbiAKMDAwMDAyNjI3MiAwMDAwMCBuIAowMDAwMDI2Mjk1IDAwMDAwIG4g
CjAwMDAwMjcyNTkgMDAwMDAgbiAKMDAwMDAyNzM4NyAwMDAwMCBuIAp0cmFpbGVyCjw8IC9TaXpl
IDQ2CiAgIC9Sb290IDQ1IDAgUgogICAvSW5mbyA0NCAwIFIKPj4Kc3RhcnR4cmVmCjI3NDQwCiUl
RU9GCg==
--001a114d9492ab9370053b1de7cd--


From nobody Mon Aug 29 00:32:22 2016
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 666CA127071 for ; Mon, 29 Aug 2016 00:32:20 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.449
X-Spam-Level: 
X-Spam-Status: No, score=-2.449 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RP_MATCHES_RCVD=-0.548, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id WxK7RUX2lnZs for ; Mon, 29 Aug 2016 00:32:18 -0700 (PDT)
Received: from mail.sfc.wide.ad.jp (shonan.sfc.wide.ad.jp [IPv6:2001:200:0:8803::53]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id A978F127058 for ; Mon, 29 Aug 2016 00:32:17 -0700 (PDT)
Received: from mail-ua0-f176.google.com (mail-ua0-f176.google.com [209.85.217.176]) by mail.sfc.wide.ad.jp (Postfix) with ESMTPSA id E7526278382 for ; Mon, 29 Aug 2016 16:32:14 +0900 (JST)
Received: by mail-ua0-f176.google.com with SMTP id l94so168859107ual.0 for ; Mon, 29 Aug 2016 00:32:14 -0700 (PDT)
X-Gm-Message-State: AE9vXwPWhsBMpQBxZra7M5y60tVl2AjeWXdWPDF08oVhKP+P2AESbRQgSEOhXDGD0p6qo0RtWsqaLJtzwArSWg==
X-Received: by 10.31.188.144 with SMTP id m138mr8889520vkf.84.1472455933408; Mon, 29 Aug 2016 00:32:13 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.176.1.46 with HTTP; Mon, 29 Aug 2016 00:32:12 -0700 (PDT)
In-Reply-To: 
References:   
From: Yoshifumi Nishida 
Date: Mon, 29 Aug 2016 00:32:12 -0700
X-Gmail-Original-Message-ID: 
Message-ID: 
To: Sanjeev Ranot 
Content-Type: text/plain; charset=UTF-8
Archived-At: 
Cc: "tcpm@ietf.org" 
Subject: Re: [tcpm] (ref. RFC 7323) Increase of shift count from 14 to 15.
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Mon, 29 Aug 2016 07:32:20 -0000

Hi Sanjeev,

On Sun, Aug 28, 2016 at 1:55 AM, Sanjeev Ranot
 wrote:
> Hi Yoshi,
>
> thanks for replying. I have done some work in these couple of days on
> how exchange and interoperability of shift count 15 or greater will work.
>
> Basically my idea is for the new TCP stack (which will support shf.cnt >=
> 15)
> to exchange shf.cnt even in Data Transfer Phase and not only in connection
> establishment. I have presented this exchange work in the document enclosed.
>
> Please have a look at the document and let me know for any info or
> clarification

Thanks for the doc.
I personally think you might want to clarify why we want to change
shift count in data transfer phase a bit more.
This is because even if we use shift count=15, I think we can specify
rather small window size by using a small number for the timestamp
value. So, if the stack supports shift count=15, I think it can just
use it from the beginning of the data transfer in most cases.

If your focus is to have a feature to update shift count during data
transfer phase in general, I think it can be a separate proposal from
increasing max shift count. Because it looks a more broad topic to me.

Thanks,
--
Yoshi


From nobody Tue Aug 30 00:02:43 2016
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C62DF12D114 for ; Tue, 30 Aug 2016 00:02:41 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -4.221
X-Spam-Level: 
X-Spam-Status: No, score=-4.221 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=ericsson.onmicrosoft.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HzCE-rM62h7o for ; Tue, 30 Aug 2016 00:02:38 -0700 (PDT)
Received: from sessmg22.ericsson.net (sessmg22.ericsson.net [193.180.251.58]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1FD0812D0AC for ; Tue, 30 Aug 2016 00:02:37 -0700 (PDT)
X-AuditID: c1b4fb3a-c91fe700000009bd-f5-57c52f8bae2a
Received: from ESESSHC024.ericsson.se (Unknown_Domain [153.88.183.90]) by  (Symantec Mail Security) with SMTP id 22.44.02493.B8F25C75; Tue, 30 Aug 2016 09:02:36 +0200 (CEST)
Received: from EUR02-AM5-obe.outbound.protection.outlook.com (153.88.183.145) by oa.msg.ericsson.com (153.88.183.90) with Microsoft SMTP Server (TLS) id 14.3.301.0; Tue, 30 Aug 2016 09:02:35 +0200
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.onmicrosoft.com; s=selector1-ericsson-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=G2Rk5KQA8YSqQ5PYoqgtXYq9K8kwSt5mw/hk4fuUcx0=; b=UYpfsJnhk0vBylmNj/LM9x4Fl1iFY5i+MGCY36wn4IJxdcw8m4GBV8dXNbdWHC+s6qSH3w8uBxgCf7xQT8AoY/IAJ3139c61veuxSUskITnR9iz2dFBZs9Ksfhd1YtfcorXhDqucElZwnC00199mUWw2CX6GbLuS/Cu7KYMR2T4=
Received: from DB4PR07MB348.eurprd07.prod.outlook.com (10.141.234.148) by DB4PR07MB345.eurprd07.prod.outlook.com (10.141.234.142) with Microsoft SMTP Server (version=TLS1_0, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P384) id 15.1.557.21; Tue, 30 Aug 2016 07:02:33 +0000
Received: from DB4PR07MB348.eurprd07.prod.outlook.com ([10.141.234.148]) by DB4PR07MB348.eurprd07.prod.outlook.com ([10.141.234.148]) with mapi id 15.01.0587.013; Tue, 30 Aug 2016 07:02:33 +0000
From: Ingemar Johansson S 
To: Veaceslav ROMAN , Neal Cardwell 
Thread-Topic: [tcpm] [e2e] TCP HyStart patch deployment
Thread-Index: AdCXp+zNDAZ7zydkTra0wALqZYyJLQH+JqqAA0qsYsAOCTWagAADKaaAABZGAgAAAuJsAABGVozQ///TRACAAB/3AIAAKj8AgAEIjoCAABVIgP//hZhA/9ZLHXD/q+ph8P9XvOWw/q9KtpD9XnrA8Pq87vsQ9Xnw1wDq7PsqANXZxFgAq6D0qADXJ4+XAK5OzT0A2sVIJmA=
Date: Tue, 30 Aug 2016 07:02:33 +0000
Message-ID: 
References: <81564C0D7D4D2A4B9A86C8C7404A13DA32145DE4@ESESSMB205.ericsson.se> <1FFD9A11-ADF2-4BA6-A9D3-D8997E1A13E5@gmail.com> <81564C0D7D4D2A4B9A86C8C7404A13DA34B2E666@ESESSMB205.ericsson.se> <7DBBB686E19D2049ADAACD210B474BB10166E64B65@XCHSRV01.main.orange.md>  <7DBBB686E19D2049ADAACD210B474BB10166E65133@XCHSRV01.main.orange.md>  <7DBBB686E19D2049ADAACD210B474BB10166E676EF@XCHSRV01.main.orange.md>  <1441314842.8932.222.camel@edumazet-glaptop2.roam.corp.google.com>   <1441385297.17208.6.camel@edumazet-glaptop2.roam.corp.google.com> <7DBBB686E19D2049ADAACD210B474BB10166E856CA@XCHSRV01.main.orange.md> <81564C0D7D4D2A4B9A86C8C7404A13DA34BD84BC@ESESSMB205.ericsson.se> <7DBBB686E19D2049ADAACD210B474BB10166E859FB@XCHSRV01.main.orange.md> <81564C0D7D4D2A4B9A86C8C7404A13DA34BD86EC@ESESSMB205.ericsson.se> <7DBBB686E19D2049ADAACD210B474BB10166E85C4E@XCHSRV01.main.orange.md> <81564C0D7D4D2A4B9A86C8C7404A13DA34BD880B@ESESSMB205.ericsson.se>  <7DBBB686E19D2049ADAACD210B474BB10166E8BDBA@XCHSRV01.main.orange.md>  <7DBBB686E19D2049ADAACD210B474BB10167E64F3E@XCHSRV01.main.orange.md> <81564C0D7D4D2A4B9A86C8C7404A13DA34EB08A1@ESESSMB205.ericsson.se> <7DBBB686E19D2049ADAACD210B474BB10167E8DC6E@XCHSRV01.main.orange.md>
In-Reply-To: <7DBBB686E19D2049ADAACD210B474BB10167E8DC6E@XCHSRV01.main.orange.md>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
authentication-results: spf=none (sender IP is ) smtp.mailfrom=ingemar.s.johansson@ericsson.com; 
x-originating-ip: [192.176.1.92]
x-ms-office365-filtering-correlation-id: ab4b4f09-e32d-4705-0140-08d3d0a39de3
x-microsoft-exchange-diagnostics: 1; DB4PR07MB345; 6:pzXi6rOmFKYtU7JBSiGMg35xcFnb3J8rm+22Cj2xFAtsM8fXSWt55QyMG9057iTG7B205MeTjtnEfqH31jMFN6oeGcXSV6MivHQWxM4+TjNy18KffzbZ8aCiF/tqEBVVg41RWLwx9GcVD+3QXQPu4iIV3wun1hCV2L9Wo/GJ+r8sjZ66RLIQV3yRyKDP9KJ/INg3OdgOlls2bgvzILt+2zsmcBS3xa7iCtoCS8BPvlm5Aw4aQlz8QAiHHOAhRW/FibjlQ5M9IBEqXVMsj2NGGKqm3SD9ip4Z8QHYQC1yLXI=; 5:MGMXg3atgoki9PMiEkN/olb+2GDvFsoNNj6M0tEKlaEWYuxDxo//CI7M9M/1Ufqp/CfQmk+SnJ8Qkve1Cory++TnwRUyVxvh6sIWYZkQRDnlfXPNJK5NHLaEE5DQEmER4vyKtkXcg8EG/jP5Vy3C5Q==; 24:U0dWjoZwz3jUoic+iaqY4eG4O7GELZf93RKbRUhMlPY4pfVfWDO+lhDt1sZt9KyTTymgcsBXy5Yn+VvsRwKcDOVmXnedzgLiw281Ooxta8g=; 7:B38LZ1ebwZfwBDO41Yum20ClFYG18B2/xuElgyy6N4stUdip0UPlWfnN0x8B7YMgWm790U9oRHB5bQ1DC1WcC3TNA6ZGpJRcMU0kOY4pr5Z98M5ZCsP7peUTyzFXrFuz7cDVXfRXyOHbFUrmFfOQagY1e/54ZdYdiRjBxv9ONS28Vf5KI5OWyX3zvU89rgdtIkIyIUUPDBJQW7ILSgCiBtE33t1BOiFp2/8fpF+QzqbdWaWzySSw/ODo4Oh/1O06
x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:DB4PR07MB345;
x-microsoft-antispam-prvs: 
x-exchange-antispam-report-test: UriScan:(37575265505322)(211936372134217);
x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040176)(601004)(2401047)(8121501046)(5005006)(3002001)(10201501046);  SRVR:DB4PR07MB345; BCL:0; PCL:0; RULEID:; SRVR:DB4PR07MB345; 
x-forefront-prvs: 0050CEFE70
x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(6009001)(6029001)(7916002)(199003)(13464003)(377454003)(24454002)(189002)(97736004)(74316002)(105586002)(77096005)(2900100001)(2950100001)(11100500001)(93886004)(122556002)(106356001)(2906002)(7846002)(10400500002)(101416001)(5002640100001)(4326007)(305945005)(7696003)(76176999)(7736002)(50986999)(3280700002)(66066001)(5660300001)(54356999)(76576001)(345774005)(3660700001)(92566002)(19580395003)(33656002)(5001770100001)(86362001)(19580405001)(6116002)(586003)(68736007)(81166006)(9686002)(102836003)(8676002)(107886002)(8936002)(81156014)(87936001)(4001430100002)(189998001)(3846002); DIR:OUT; SFP:1101; SCL:1; SRVR:DB4PR07MB345; H:DB4PR07MB348.eurprd07.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; 
received-spf: None (protection.outlook.com: ericsson.com does not designate permitted sender hosts)
spamdiagnosticoutput: 1:99
spamdiagnosticmetadata: NSPM
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-originalarrivaltime: 30 Aug 2016 07:02:33.3916 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 92e84ceb-fbfd-47ab-be52-080c6b87953f
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB4PR07MB345
X-OriginatorOrg: ericsson.com
X-Brightmail-Tracker: H4sIAAAAAAAAA01Se0hTURjn7N7dXYeD083Hl0boeku6lMgRJkYJEwkzqIkoNvSmli92VVQQ NNR0ZmQm6BAVW4X2UIeZpmkuXWpZ+SB1aKRNw8JXgmIPze0a9N/vdb7fx8ehCaZB6ETHJiSz 6gRVnJQSk+Uhz0Ldb8h6lEenHjrLZ4zTInnbu1JS3rE4QMnzJ15sobJiJO8eMSJ5c1+VQL64 2Uj40YpW7aRIUa1PUeh06wKF7v4MUuS/biDOCUPFPlFsXGwqq5b5XhLHbA5qqKTewLS7nzOz 0IcADaJpwMcgZ8hPg8Q0g+sRrPQsIJ70IpjLfiu0EBIXEfB9+KOId0oE8Ke9kuCJEUFjdolA g2xoCvtArWENWbAdVkJp7pj1BYGbBDDQsEFZjJ34OJiuLRB8yBtuv8mzTrLDJgSD2m+kZSsS 74fRYaElI8GhMGR+KuDbBiTQOdZqbbPBwWDI6bOGEHaAtf5HVp3AjmAyV1kxYAy69vcEj+1h 7svGdj4SfowXCXndBX4PTVM8PgtPaiooSxngeQrKikcRb/jDSkfvdigWsrqXKP58gVAws5eX mxGslop4vBv6Ruu25+RQoO8ZtxoMZuHB41zEX8IJJkcK0C3kpv1vb+3WWAIfhvrnMl52hTuF UyKt9RY7oK/cTFYjsg7ZcyzHxUd7eXmw6thIjktM8Ehgk/Vo6x91Nf060YK6vp4yIEwjqa3E Ja1byQhVqVx6vAEBTUjtJFVHepSMJEqVnsGqEyPUKXEsZ0DONCl1lATNuSoZHK1KZq+ybBKr /ucKaBunLBRypYs7r3GNOPhp3qN9wqjRV7pnMGGXRf5BhVVsy64L+5rBPOtep186YLT1CTfM RgZ71XeSSZmnx4ZqtGEOeal7mNxheyQ2zQWcacsz/ewv1nffM0an1L7suh52UXbo5LLvatL4 8qvOilRP2+ywdfOgTNpovJnLeFWUeAeGB0pJLkbl6UaoOdVfdSsgsUMDAAA=
Archived-At: 
Cc: "tcpm@ietf.org" , Piers O'Hanlon , Sangtae Ha , Ingemar Johansson S , Eric Dumazet , "end2end-interest@postel.org" 
Subject: Re: [tcpm] [e2e] TCP HyStart patch deployment
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Tue, 30 Aug 2016 07:02:42 -0000

SGkNCg0KQW4gb2xkIHRocmVhZCByZXN1bWVkIGZyb20gb3VyIHNpZGUgYXMgd2Ugc2VlIGlzc3Vl
cyB3aXRoIHRoZSBIeVN0YXJ0IGFsZ29yaXRobSBpbiBDdWJpYyB3aGVuIHdlIGV2YWx1YXRlIHNv
bWUgbmV3IExURSBhbmQgNUcgYWNjZXNzIGNvbmNlcHRzLiANCk9uZSBwcm9ibGVtIGlzIHRoZSBw
YWNrZXQgdHJhaW4gYWxnb3JpdGhtIDogSSB1bmRlcnN0YW5kIHRoYXQgaXQgZG9lcyBub3Qgd29y
ayB3ZWxsIHdpdGggcGFja2V0IHBhY2luZyBhbmQgaXMgdGhlcmVmb3JlIGRpc2FibGVkIGluIFFV
SUMuIElzIHRoZSBwYWNrZXQgdHJhaW4gYWxnb3JpdGhtIHVzZWQgaW4gZ2VuZXJhbCwgSSBndWVz
cyBub3QgaW4gdGhlIGNhc2VzIHdoZW4gcGFja2V0IHBhY2luZyBpcyBlbmFibGVkIG9yID8NCg0K
VGhlIG90aGVyIHByb2JsZW0gaXMgdGhlIGRlbGF5IGFsZ29yaXRobSwgbXkgaW1wcmVzc2lvbiBi
YXNlZCBvbiBsaW1pdGVkIHNpbXVsYXRpb24gZXhwZXJpbWVudHMgaXMgdGhhdCB0aGUgSFlTVEFS
VF9ERUxBWV9NSU4gbmVlZHMgdG8gYmUgc2V0IHRvIH4xNm1zIHRvIGF2b2lkIHRoYXQgSHlTdGFy
dCBleGl0cyBwcmVtYXR1cmVseS4gVGhlIHF1ZXN0aW9uIGlzIGlmIHRoaXMgY2F1c2VzIHByb2Js
ZW0gd2l0aCBvdGhlciBhY2Nlc3MgdHlwZXMgKERTTCA/KQ0KDQpDb21tZW50cyB3ZWxjb21lIA0K
L0luZ2VtYXINCg0KDQo+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+IEZyb206IFZlYWNl
c2xhdiBST01BTiBbbWFpbHRvOlZlYWNlc2xhdi5Sb21hbkBvcmFuZ2UubWRdDQo+IFNlbnQ6IGRl
biAzIG5vdmVtYmVyIDIwMTUgMjA6MDYNCj4gVG86IEluZ2VtYXIgSm9oYW5zc29uIFMgPGluZ2Vt
YXIucy5qb2hhbnNzb25AZXJpY3Nzb24uY29tPjsgTmVhbA0KPiBDYXJkd2VsbCA8bmNhcmR3ZWxs
QGdvb2dsZS5jb20+DQo+IENjOiBFcmljIER1bWF6ZXQgPGVkdW1hemV0QGdvb2dsZS5jb20+OyBF
cmljIER1bWF6ZXQNCj4gPGVyaWMuZHVtYXpldEBnbWFpbC5jb20+OyB0Y3BtQGlldGYub3JnOyBQ
aWVycyBPJ0hhbmxvbg0KPiA8cC5vaGFubG9uQGdtYWlsLmNvbT47IFNhbmd0YWUgSGEgPHNhbmd0
YWUuaGFAZ21haWwuY29tPjsgZW5kMmVuZC0NCj4gaW50ZXJlc3RAcG9zdGVsLm9yZw0KPiBTdWJq
ZWN0OiBSRTogW3RjcG1dIFtlMmVdIFRDUCBIeVN0YXJ0IHBhdGNoIGRlcGxveW1lbnQNCj4gDQo+
IFlvdSBhcmUgcmlnaHQsIGl0IHdhc24ndCBtb2RpZmllZCBhbmQgc3RheWVkIGF0IDE2LiBUaGFu
ayB5b3UgZm9yIHN1Z2dlc3Rpb24uDQo+IEN1cnJlbnRseSB3ZSBhcmUgdHJ5aW5nIHRvIG1ha2Ug
YW4gZXhwZXJpbWVudGFsIHZlcnNpb24gd2VyZSB0aGlzIHdvdWxkIGJlDQo+IHBvc3NpYmxlIHRv
IG1ha2UgdGhlc2UgcGFyYW1ldGVycyBhbmQgZmV3IG90aGVyIHRoaW5ncyBleHRlcm5hbGx5IHNl
dHRhYmxlLg0KPiANCj4gQnV0LCBob25lc3RseSwgdGhlIGZ1cnRoZXIgd2UgZGlnIGluIHRoZSBj
b2RlIGFuZCB0cmFjZXMgdGhlIG1vcmUgdGhlDQo+IGNvbmZ1c2lvbiBhbmQgbW9yZSBxdWVzdGlv
bnMgYXJpc2VzLg0KPiAJSHlzdGFydCByZWx5IG9uIGNvcnJlY3QgaWRlbnRpZmljYXRpb24gb2Yg
cm91bmQgYm9yZGVycy4gRm9yIHRoaXMNCj4gaHlzdGFydF9yZXNldCBzZXRzIHRoZSBib3JkZXIg
YXQgc25kX254dC4NCj4gQnV0LCBpZiBzZXZlcmFsIEFDSyBhcnJpdmUgYXQgaGlnaCBzcGVlZCB3
aGVuIHRoZSBjdXJyZW50IHJvdW5kIGJvcmRlciBpcw0KPiBjcm9zc2VkIGFuZCB0aGUgc2VuZGVy
IGhhcyBub3QgdGltZSB0byB0cmFuc21pdCB0d28gcGFja2V0cyBpbW1lZGlhdGVseQ0KPiBhZnRl
ciBBQ0ssIHRoZW4gdGhlIHNuZF9ueHQgZG9lcyBub3QgcHJvZ3Jlc3MgYW5kIGh5c3RhcnRfcmVz
ZXQgc2V0cyB0aGUNCj4gcm91bmQgb2YgdGhlIGJvcmRlciB0byBhIHByZXR0eSByYW5kb20gdmFs
dWUgYW5kIHRoZW4gaW4gdGhlIG5leHQgcm91bmQgdGhlDQo+IGh5c3RhcnQgbWVhc3VyZSBwaWVj
ZXMgb2YgdHdvIHJvdW5kcy4NCj4gCUFsc28sIHRoZSBjb2RlIGlzIGNhbGxpbmcgaHlzdGFydF91
cGRhdGUgb24gZWFjaCBBQ0sgYmVmb3JlIHRoZQ0KPiBoeXN0YXJ0X3Jlc2V0LCBhcyBhIHJlc3Vs
dCBhdCB0aGUgYm9yZGVyIG9mIHJvdW5kcyB0aGUgaGVhZCBvZiBsaW5lIChIT0wpIEFDSw0KPiBk
ZWxheSBvZiByb3VuZCBOKzEgaXMgYWNjb3VudGVkIGluIHRoZSByb3VuZCBOIGFuZCwgYmVjYXVz
ZSwgdGhlIEhPTA0KPiB1c3VhbGx5IGhhcyBsZXNzIGRlbGF5LCBpdCBpcyBmcmVxdWVudGx5IGJl
Y29tZSBkZWxheV9taW4gYnV0IG5vdCBpbiByb3VuZA0KPiBOKzEgYW5kLCBhcyBhIHJlc3VsdCBl
YXJseSBleGl0IGluIHJvdW5kIE4rMSBhZnRlciBjb3VudGluZyBkZWxheXMgb2YgcGFja2V0cyAy
DQo+IHRocm91Z2ggOS4NCj4gCVRoaXMgc2hvdWxkIGFsc28gYWZmZWN0IHRyYWluIGRldGVjdCBh
cywgd2hlbiBjcm9zc2luZyB0aGUgcm91bmQNCj4gYm9yZGVyLCBpbnN0ZWFkIG9mIHN1YnRyYWN0
aW5nIHRoZSB0aW1lIG9mIHRoZSBmaXJzdCBhbmQgdGhlIGxhc3QgQUNLIG9mIHRoZQ0KPiB0cmFp
biBOIHRoZXJlIHdpbGwgYmUgc3VidHJhY3RlZCB0aW1lcyBvZiBmaXJzdCBBQ0sgb2YgdHJhaW4g
TiBhbmQgZmlyc3QgQUNLIG9mDQo+IHRoZSB0cmFpbiBOKzEgd2hpY2ggd2lsbCBiZSBhcHByb3hp
bWF0ZWx5IGVxdWFsIHRvIFJUVCBhbmQgZGVmaW5pdGVseSBiaWdnZXINCj4gdGhhbiBSVFQvMiBh
cyBwZXIgZXhwZWN0ZWQgYWxnb3JpdGhtLiBUaGlzIHNoYWxsIHR5cGljYWxseSBsZWFkIHRvIGFs
d2F5cyBleGl0IGluDQo+IHRyYWluIDIuIFRoaXMgd2lsbCBhZmZlY3QgaGlnaCBzcGVlZCBuZXR3
b3JrcyB3aXRoIFJUVCBsZXNzIHRoYW4NCj4gaHlzdGFydF9hY2tfZGVsdGEgKDJtcykuDQo+ICAg
ICAgICAgICAgICBUaGVuIGNhbGwgb2YgaHlzdGFydF9yZXNldCBpcyBzdWJqZWN0IG9mIGNoZWNr
IG9mIHNldmVyYWwgY29uZGl0aW9ucyBsaWtlDQo+IG1heV9yaXNlX2N3bmQgb3IgaXNfY3duZF9s
aW1pdGVkLCB3aGlsZSBpdCBkb2Vzbid0IGxvb2sgdGhhdA0KPiBoeXN0YXJ0X3VwZGF0ZSBpcywg
c28gdGhhdCBpdCBtYXkgYmVjb21lIHRvdGFsbHkgb3V0IG9mIHN5bmMgd2l0aCByb3VuZA0KPiBi
b3JkZXIuDQo+IA0KPiAJV2UgYXJlIHN0aWxsIGRvaW5nIGVmZm9ydHMgdG8gdHJ5IHRvIGZpbmQg
cGFyYW1ldGVycyB3aGljaCB3b3VsZCBtYWtlDQo+IGh5c3RhcnQgYmV0dGVyLCBidXQgYW5hbHlz
aXMgb2YgbWFueSB0cmFjZXMgbWFrZSBtdWNoIGltcHJlc3Npb24gb2YNCj4gcmFuZG9tbmVzcyBv
ZiBleGl0IGFuZCBub3Qgc3VyZSB3aGV0aGVyIHBhcmFtZXRlcnMgbWF5IGhlbHAuDQo+IA0KPiBW
ZWFjZXNsYXYgUm9tYW4NCj4gDQo+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+IEZyb206
IEluZ2VtYXIgSm9oYW5zc29uIFMgW21haWx0bzppbmdlbWFyLnMuam9oYW5zc29uQGVyaWNzc29u
LmNvbV0NCj4gU2VudDogVHVlc2RheSwgMDMgTm92ZW1iZXIgMjAxNSAxNToxOQ0KPiBUbzogVmVh
Y2VzbGF2IFJPTUFOOyBOZWFsIENhcmR3ZWxsDQo+IENjOiBFcmljIER1bWF6ZXQ7IEVyaWMgRHVt
YXpldDsgdGNwbUBpZXRmLm9yZzsgUGllcnMgTydIYW5sb247IFNhbmd0YWUgSGE7DQo+IGVuZDJl
bmQtaW50ZXJlc3RAcG9zdGVsLm9yZzsgSW5nZW1hciBKb2hhbnNzb24gUw0KPiBTdWJqZWN0OiBS
RTogW3RjcG1dIFtlMmVdIFRDUCBIeVN0YXJ0IHBhdGNoIGRlcGxveW1lbnQNCj4gDQo+IEhpDQo+
IA0KPiBUaGFua3MgZm9yIGRvaW5nIHRoZSBleHBlcmltZW50cy4NCj4gSXQgbWF5IGJlIHBvc3Np
YmxlIHRoYXQgSHlTdGFydCBtYXkgbmVlZCBtb3JlIGV4dGVuc2l2ZSBtb2RpZmljYXRpb25zLg0K
PiBOZWVkIHRvIGFzayB0aGUgZHVtYiBxdWVzdGlvbiBmaXJzdCB0aG91Z2guIElzIEhZU1RBUlRf
REVMQVlfTUFYDQo+IG1vZGlmaWVkIGluIHlvdXIgZXhwZXJpbWVudCBvciBpcyBpdCBzdGlsbCAx
Nm1zID8NCj4gSSBndWVzcyBIWVNUQVJUX0RFTEFZX01BWCBtYXkgbmVlZCB0byBiZSBzZXQgdG8N
Cj4gICAgIEhZU1RBUlRfREVMQVlfTUFYID0gTUFYKDE2bXMsIEhZU1RBUlRfREVMQVlfTUlOKjIp
IE9yDQo+IHNvbWV0aGluZyBzaW1pbGFyLCBvbmUgY2FuIGFsd2F5cyBhcmd1ZSBhcm91bmQgdGhl
IGZhY3RvciAyIGFib3ZlLg0KPiANCj4gL0luZ2VtYXINCj4gDQo+ID4gLS0tLS1PcmlnaW5hbCBN
ZXNzYWdlLS0tLS0NCj4gPiBGcm9tOiBWZWFjZXNsYXYgUk9NQU4gW21haWx0bzpWZWFjZXNsYXYu
Um9tYW5Ab3JhbmdlLm1kXQ0KPiA+IFNlbnQ6IGRlbiAxNyBva3RvYmVyIDIwMTUgMjI6NDgNCj4g
PiBUbzogTmVhbCBDYXJkd2VsbA0KPiA+IENjOiBFcmljIER1bWF6ZXQ7IEluZ2VtYXIgSm9oYW5z
c29uIFM7IEVyaWMgRHVtYXpldDsgdGNwbUBpZXRmLm9yZzsNCj4gPiBQaWVycyBPJ0hhbmxvbjsg
U2FuZ3RhZSBIYTsgZW5kMmVuZC1pbnRlcmVzdEBwb3N0ZWwub3JnDQo+ID4gU3ViamVjdDogUkU6
IFt0Y3BtXSBbZTJlXSBUQ1AgSHlTdGFydCBwYXRjaCBkZXBsb3ltZW50DQo+ID4NCj4gPiBIaSwN
Cj4gPiBSZWNvbXBpbGVkIGtlcm5lbCB0byBIWVNUQVJUX0RFTEFZX01JTiAgMjAgbXM6DQo+ID4N
Cj4gPiAgIERvd25sb2FkIDEwTUIsIEhpZ2ggQmFuZHdpZHRocywgSFlTVEFSVF9ERUxBWV9NSU4g
IDIwIG1zOg0KPiA+ICAgICAgICAgICAgSHlzdGFydCBhdmVyYWdlIGRvd25sb2FkIHRpbWU6IDEu
MTggcw0KPiA+ICAgICAgICAgICAgTm9IeXN0YXJ0IGF2ZXJhZ2UgZG93bmxvYWQgdGltZTogMS4w
MCAgcw0KPiA+DQo+ID4gTm8gc2lnbmlmaWNhbnQgaW1wcm92ZW1lbnQgYWdhaW5zdCB0aGUgY2Fz
ZSB3aXRoIEhZU1RBUlRfREVMQVlfTUlOICAxMA0KPiA+IG1zLg0KPiA+DQo+ID4gU2lsbCBIeXN0
YXJ0IGVhcmx5IGV4aXQgaXMgdmlzaWJsZS4NCj4gPg0KPiA+IE90aGVyIHRlc3RzIHdpdGggSFlT
VEFSVF9ERUxBWV9NSU4gIDIwIG1zIGFuZCBmaWxlcyBvZiAzTUIuDQo+ID4NCj4gPiAgIERvd25s
b2FkIDNNQiwgSGlnaCBCYW5kd2lkdGhzLCBIWVNUQVJUX0RFTEFZX01JTiAgMjAgbXM6DQo+ID4g
ICAgICAgICAgICBIeXN0YXJ0IGF2ZXJhZ2UgZG93bmxvYWQgdGltZTogMC40NyBzDQo+ID4gICAg
ICAgICAgICBOb0h5c3RhcnQgYXZlcmFnZSBkb3dubG9hZCB0aW1lOiAwLjM5ICBzDQo+ID4NCj4g
PiBjb21wYXJlIHdpdGggNCBtcw0KPiA+DQo+ID4gICBEb3dubG9hZCAzTUIsIEhpZ2ggQmFuZHdp
ZHRocywgSFlTVEFSVF9ERUxBWV9NSU4gIDQgbXM6DQo+ID4gICAgICAgICAgICBIeXN0YXJ0IGF2
ZXJhZ2UgZG93bmxvYWQgdGltZTogMC42NSBzDQo+ID4gICAgICAgICAgICBOb0h5c3RhcnQgYXZl
cmFnZSBkb3dubG9hZCB0aW1lOiAwLjM3ICBzDQo+ID4NCj4gPiBGZXcgbW9yZSBjb21tZW50OiBI
aWdoIEJhbmR3aWR0aCBpcyB0eXBpY2FsbHkgYWJvdmUgMzAtNDAgTWJwcy4NCj4gPiBGb3IgTG93
IEJhbmR3aWR0aCwgd2hlbiByYWRpbyBkbyBub3QgcGVybWl0IGFib3ZlIDQwIE1icHMgSHlzdGFy
dA0KPiA+IGVhcmx5IGV4aXQgZG9lcyBub3QgaGF2ZSBhIHNpZ25pZmljYW50IGltcGFjdC4gSSBk
byBleHBsYWluIGl0IHdpdGgNCj4gPiB0aGUgZmFjdCB0aGF0IHRoZSBtaW5pbXVtIGV4aXQgQ1dO
RCBmb3IgSHlzc3RhcnQgaXMgMjkgd2hpY2gsIGZvciBSVFQNCj4gPiBvZiB+MTVtcyAobWluaW11
bSBwb3NzaWJsZSBpbiBMVEUgYW5kIHZpc2libGUgaW4gbWFqb3JpdHkgb2YgdHJhY2VzKQ0KPiA+
IGxlYWRzIHRvIH4yNSBNYnBzLCBzbyBIeXN0YXJ0IGV4aXQsIGF0IGxlYXN0LCBhdCAyNSBNYnBz
IGFuZCB0aGVuIGl0DQo+ID4gdGFrZXMgbm90IHRvbyBsb25nIHRvIGN1YmljIHRvIHJlYWNoIDMw
LTQwIE1icHMuDQo+ID4NCj4gPiBWZWFjZXNsYXYgUm9tYW4NCj4gPg0KPiA+IC0tLS0tT3JpZ2lu
YWwgTWVzc2FnZS0tLS0tDQo+ID4gRnJvbTogTmVhbCBDYXJkd2VsbCBbbWFpbHRvOm5jYXJkd2Vs
bEBnb29nbGUuY29tXQ0KPiA+IFNlbnQ6IFR1ZXNkYXksIDA2IE9jdG9iZXIgMjAxNSAwNDowNg0K
PiA+IFRvOiBWZWFjZXNsYXYgUk9NQU4NCj4gPiBDYzogRXJpYyBEdW1hemV0OyBJbmdlbWFyIEpv
aGFuc3NvbiBTOyBFcmljIER1bWF6ZXQ7IHRjcG1AaWV0Zi5vcmc7DQo+ID4gUGllcnMgTydIYW5s
b247IFNhbmd0YWUgSGE7IGVuZDJlbmQtaW50ZXJlc3RAcG9zdGVsLm9yZw0KPiA+IFN1YmplY3Q6
IFJlOiBbdGNwbV0gW2UyZV0gVENQIEh5U3RhcnQgcGF0Y2ggZGVwbG95bWVudA0KPiA+DQo+ID4g
T24gTW9uLCBPY3QgNSwgMjAxNSBhdCA2OjA3IFBNLCBWZWFjZXNsYXYgUk9NQU4NCj4gPiA8VmVh
Y2VzbGF2LlJvbWFuQG9yYW5nZS5tZD4gd3JvdGU6DQo+ID4gPiBXZSd2ZSBtYW5hZ2VkIHRvIHJl
Y29tcGlsZSB0aGUga2VybmVsIDQuMSB3aXRoIHRjcF9jdWJpYw0KPiA+IEhZU1RBUlRfREVMQVlf
TUlOICAoMTBVPDwzKSAsIDEwbXMuDQo+ID4gPiBXZWxsLCB3ZSBoYWQgdGltZSBvbmx5IGZvciBv
bmUgdGVzdCBjeWNsZSwgYnV0IHJlc3VsdHMgYXJlIHZlcnkgcHJvbWlzaW5nOg0KPiA+ID4NCj4g
PiA+IERvd25sb2FkIDEwTUIsIEhpZ2ggQmFuZHdpZHRocywgSFlTVEFSVF9ERUxBWV9NSU4gIDEw
IG1zOg0KPiA+ID4gICAgICAgICAgICBIeXN0YXJ0IGF2ZXJhZ2UgZG93bmxvYWQgdGltZTogMS4x
NiBzDQo+ID4gPiAgICAgICAgICAgIE5vSHlzdGFydCBhdmVyYWdlIGRvd25sb2FkIHRpbWU6IDAu
OTMgIHMNCj4gPiAuLi4NCj4gPiA+ICBEb3dubG9hZCAxME1CLCBIaWdoIEJhbmR3aWR0aHMsIEhZ
U1RBUlRfREVMQVlfTUlOICA0IG1zOg0KPiA+ID4gICAgICAgICAgICBIeXN0YXJ0IGF2ZXJhZ2Ug
ZG93bmxvYWQgdGltZTogMS40MiBzDQo+ID4gPiAgICAgICAgICAgIE5vSHlzdGFydCBhdmVyYWdl
IGRvd25sb2FkIHRpbWU6IDAuODkgIHMNCj4gPg0KPiA+IFRoYW5rcyEgVGhhdCBpcyB2ZXJ5IHVz
ZWZ1bCBhbmQgaW50ZXJlc3RpbmcgZGF0YS4gV291bGQgeW91IGJlIGFibGUgdG8NCj4gPiB0cnkg
YSBmZXcgb3RoZXIgdmFsdWVzLCBsaWtlIDE1bXMgYW5kIDIwbXM/DQo+ID4NCj4gPiBuZWFsDQo=


From nobody Tue Aug 30 07:01:01 2016
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 78F9012D5F8 for ; Tue, 30 Aug 2016 07:01:00 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -3.249
X-Spam-Level: 
X-Spam-Status: No, score=-3.249 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RP_MATCHES_RCVD=-0.548, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=google.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gGRhA8W2jEkD for ; Tue, 30 Aug 2016 07:00:56 -0700 (PDT)
Received: from mail-oi0-x230.google.com (mail-oi0-x230.google.com [IPv6:2607:f8b0:4003:c06::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id CEC7412D5F4 for ; Tue, 30 Aug 2016 07:00:55 -0700 (PDT)
Received: by mail-oi0-x230.google.com with SMTP id j203so26811831oih.2 for ; Tue, 30 Aug 2016 07:00:55 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=y/MbrxEu51fMrOK69dtGGpDeF6Rz+xFaS/Yxog97DBU=; b=oUUxsoViGvL0wuNExH37eTojNZnSMrV6EkYivTlvuUFvwla2vyDGXX5rRuaVhsPkwP 9VXuTXep5uOvl1R9W8cRzPNrrnftETeE35nqDOsI4s6NMHiud6gQKdqb2vCT0kZtHGOX P8xcHE/DE1k4I1Bhb8FAfZZfuFnPfktzAGP3q+LhFXRg4OMCjhBAqP6/g7lrs2XfMBoP 5mfqhWd8YzW4c+aJMa881vZdidP3ITb3pT7VfI8sWdPF3CVgvmqrp7066XTqdrtzJT/S XVN28hgPlEtSl4g5F5x3UEVREC6+BDwjdXNDdde553RqViSLuo3zmS6UX43miylGjJ6p qUJw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=y/MbrxEu51fMrOK69dtGGpDeF6Rz+xFaS/Yxog97DBU=; b=T5leG1EEWWDEcOqZUVpriYkAaCYJh6yR9ltMsAgp6jcTRSSZuq1mlVcPWA12FCalMH cZE+pUoyrm+I6RehUiuqSq6l3UIvyYXWQVwzci5OxfrI9PE9FqDsiQZ/GsTgNdm3rICd LsMm6UJHRVxAiEV+RCRligFqvavGJ/COEvNremFALxBKRv1Y/IQmioJgCIxxJ/L16Y7d B1UT1e4HY6FHbUkMIJo+1ZFlhEon+6/bFgADdwjEjFYYtfuGxQWBtwGuNOORv8BB9aBc zVtmF7vqlM4dQuK3qNduCfKEj/Cv8Wnfqvnhgi9ku1UAzpVBYqSPQw80ATxXbvMHRaVg SGRQ==
X-Gm-Message-State: AE9vXwPbB99FllX0IrU/eSR8l3QmquctRHgVmliyAwMPWQEmj/H+MEDGs7igpLDS3MrRp71L6itZV0aFTEKoSEYa
X-Received: by 10.157.20.73 with SMTP id h67mr4238338oth.60.1472565654990; Tue, 30 Aug 2016 07:00:54 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.202.63.11 with HTTP; Tue, 30 Aug 2016 07:00:24 -0700 (PDT)
In-Reply-To: 
References: <81564C0D7D4D2A4B9A86C8C7404A13DA32145DE4@ESESSMB205.ericsson.se> <1FFD9A11-ADF2-4BA6-A9D3-D8997E1A13E5@gmail.com> <81564C0D7D4D2A4B9A86C8C7404A13DA34B2E666@ESESSMB205.ericsson.se> <7DBBB686E19D2049ADAACD210B474BB10166E64B65@XCHSRV01.main.orange.md>  <7DBBB686E19D2049ADAACD210B474BB10166E65133@XCHSRV01.main.orange.md>  <7DBBB686E19D2049ADAACD210B474BB10166E676EF@XCHSRV01.main.orange.md>  <1441314842.8932.222.camel@edumazet-glaptop2.roam.corp.google.com>   <1441385297.17208.6.camel@edumazet-glaptop2.roam.corp.google.com> <7DBBB686E19D2049ADAACD210B474BB10166E856CA@XCHSRV01.main.orange.md> <81564C0D7D4D2A4B9A86C8C7404A13DA34BD84BC@ESESSMB205.ericsson.se> <7DBBB686E19D2049ADAACD210B474BB10166E859FB@XCHSRV01.main.orange.md> <81564C0D7D4D2A4B9A86C8C7404A13DA34BD86EC@ESESSMB205.ericsson.se> <7DBBB686E19D2049ADAACD210B474BB10166E85C4E@XCHSRV01.main.orange.md> <81564C0D7D4D2A4B9A86C8C7404A13DA34BD880B@ESESSMB205.ericsson.se>  <7DBBB686E19D2049ADAACD210B474BB10166E8BDBA@XCHSRV01.main.orange.md>  <7DBBB686E19D2049ADAACD210B474BB10167E64F3E@XCHSRV01.main.orange.md> <81564C0D7D4D2A4B9A86C8C7404A13DA34EB08A1@ESESSMB205.ericsson.se> <7DBBB686E19D2049ADAACD210B474BB10167E8DC6E@XCHSRV01.main.orange.md> 
From: Neal Cardwell 
Date: Tue, 30 Aug 2016 10:00:24 -0400
Message-ID: 
To: Ingemar Johansson S 
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
Archived-At: 
Cc: "tcpm@ietf.org" , Piers O'Hanlon , Sangtae Ha , Eric Dumazet , "end2end-interest@postel.org" 
Subject: Re: [tcpm] [e2e] TCP HyStart patch deployment
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Tue, 30 Aug 2016 14:01:00 -0000

Hi,

As one data point, at Google the google.com and YouTube TCP stacks use
Eric Dumazet's fq pacing qdisc:
  https://www.ietf.org/proceedings/88/slides/slides-88-tcpm-9.pdf
  https://lwn.net/Articles/564978/
Eric discovered that the Hystart Ack Train mechanism does not work
well with pacing, so for TCP at Google we have long ago disabled the
Hystart Ack Train mechanism, and just use the delay mechanism.

It is also our experience that the Hystart delay mechanism often
triggers spuriously for cellular and wifi links, and is difficult to
tune in a way that works well generally. It may be useful for someone
in the community to do a broad investigation to see if there are
HYSTART_DELAY_MIN,  HYSTART_DELAY_MAX, and threshold ratio values that
work better than the current values across a variety of important
last-mile technologies (cellular, cable, DSL, wifi,...).

neal

On Tue, Aug 30, 2016 at 3:02 AM, Ingemar Johansson S
 wrote:
> Hi
>
> An old thread resumed from our side as we see issues with the HyStart alg=
orithm in Cubic when we evaluate some new LTE and 5G access concepts.
> One problem is the packet train algorithm : I understand that it does not=
 work well with packet pacing and is therefore disabled in QUIC. Is the pac=
ket train algorithm used in general, I guess not in the cases when packet p=
acing is enabled or ?
>
> The other problem is the delay algorithm, my impression based on limited =
simulation experiments is that the HYSTART_DELAY_MIN needs to be set to ~16=
ms to avoid that HyStart exits prematurely. The question is if this causes =
problem with other access types (DSL ?)
>
> Comments welcome
> /Ingemar
>
>
>> -----Original Message-----
>> From: Veaceslav ROMAN [mailto:Veaceslav.Roman@orange.md]
>> Sent: den 3 november 2015 20:06
>> To: Ingemar Johansson S ; Neal
>> Cardwell 
>> Cc: Eric Dumazet ; Eric Dumazet
>> ; tcpm@ietf.org; Piers O'Hanlon
>> ; Sangtae Ha ; end2end-
>> interest@postel.org
>> Subject: RE: [tcpm] [e2e] TCP HyStart patch deployment
>>
>> You are right, it wasn't modified and stayed at 16. Thank you for sugges=
tion.
>> Currently we are trying to make an experimental version were this would =
be
>> possible to make these parameters and few other things externally settab=
le.
>>
>> But, honestly, the further we dig in the code and traces the more the
>> confusion and more questions arises.
>>       Hystart rely on correct identification of round borders. For this
>> hystart_reset sets the border at snd_nxt.
>> But, if several ACK arrive at high speed when the current round border i=
s
>> crossed and the sender has not time to transmit two packets immediately
>> after ACK, then the snd_nxt does not progress and hystart_reset sets the
>> round of the border to a pretty random value and then in the next round =
the
>> hystart measure pieces of two rounds.
>>       Also, the code is calling hystart_update on each ACK before the
>> hystart_reset, as a result at the border of rounds the head of line (HOL=
) ACK
>> delay of round N+1 is accounted in the round N and, because, the HOL
>> usually has less delay, it is frequently become delay_min but not in rou=
nd
>> N+1 and, as a result early exit in round N+1 after counting delays of pa=
ckets 2
>> through 9.
>>       This should also affect train detect as, when crossing the round
>> border, instead of subtracting the time of the first and the last ACK of=
 the
>> train N there will be subtracted times of first ACK of train N and first=
 ACK of
>> the train N+1 which will be approximately equal to RTT and definitely bi=
gger
>> than RTT/2 as per expected algorithm. This shall typically lead to alway=
s exit in
>> train 2. This will affect high speed networks with RTT less than
>> hystart_ack_delta (2ms).
>>              Then call of hystart_reset is subject of check of several c=
onditions like
>> may_rise_cwnd or is_cwnd_limited, while it doesn't look that
>> hystart_update is, so that it may become totally out of sync with round
>> border.
>>
>>       We are still doing efforts to try to find parameters which would m=
ake
>> hystart better, but analysis of many traces make much impression of
>> randomness of exit and not sure whether parameters may help.
>>
>> Veaceslav Roman
>>
>> -----Original Message-----
>> From: Ingemar Johansson S [mailto:ingemar.s.johansson@ericsson.com]
>> Sent: Tuesday, 03 November 2015 15:19
>> To: Veaceslav ROMAN; Neal Cardwell
>> Cc: Eric Dumazet; Eric Dumazet; tcpm@ietf.org; Piers O'Hanlon; Sangtae H=
a;
>> end2end-interest@postel.org; Ingemar Johansson S
>> Subject: RE: [tcpm] [e2e] TCP HyStart patch deployment
>>
>> Hi
>>
>> Thanks for doing the experiments.
>> It may be possible that HyStart may need more extensive modifications.
>> Need to ask the dumb question first though. Is HYSTART_DELAY_MAX
>> modified in your experiment or is it still 16ms ?
>> I guess HYSTART_DELAY_MAX may need to be set to
>>     HYSTART_DELAY_MAX =3D MAX(16ms, HYSTART_DELAY_MIN*2) Or
>> something similar, one can always argue around the factor 2 above.
>>
>> /Ingemar
>>
>> > -----Original Message-----
>> > From: Veaceslav ROMAN [mailto:Veaceslav.Roman@orange.md]
>> > Sent: den 17 oktober 2015 22:48
>> > To: Neal Cardwell
>> > Cc: Eric Dumazet; Ingemar Johansson S; Eric Dumazet; tcpm@ietf.org;
>> > Piers O'Hanlon; Sangtae Ha; end2end-interest@postel.org
>> > Subject: RE: [tcpm] [e2e] TCP HyStart patch deployment
>> >
>> > Hi,
>> > Recompiled kernel to HYSTART_DELAY_MIN  20 ms:
>> >
>> >   Download 10MB, High Bandwidths, HYSTART_DELAY_MIN  20 ms:
>> >            Hystart average download time: 1.18 s
>> >            NoHystart average download time: 1.00  s
>> >
>> > No significant improvement against the case with HYSTART_DELAY_MIN  10
>> > ms.
>> >
>> > Sill Hystart early exit is visible.
>> >
>> > Other tests with HYSTART_DELAY_MIN  20 ms and files of 3MB.
>> >
>> >   Download 3MB, High Bandwidths, HYSTART_DELAY_MIN  20 ms:
>> >            Hystart average download time: 0.47 s
>> >            NoHystart average download time: 0.39  s
>> >
>> > compare with 4 ms
>> >
>> >   Download 3MB, High Bandwidths, HYSTART_DELAY_MIN  4 ms:
>> >            Hystart average download time: 0.65 s
>> >            NoHystart average download time: 0.37  s
>> >
>> > Few more comment: High Bandwidth is typically above 30-40 Mbps.
>> > For Low Bandwidth, when radio do not permit above 40 Mbps Hystart
>> > early exit does not have a significant impact. I do explain it with
>> > the fact that the minimum exit CWND for Hysstart is 29 which, for RTT
>> > of ~15ms (minimum possible in LTE and visible in majority of traces)
>> > leads to ~25 Mbps, so Hystart exit, at least, at 25 Mbps and then it
>> > takes not too long to cubic to reach 30-40 Mbps.
>> >
>> > Veaceslav Roman
>> >
>> > -----Original Message-----
>> > From: Neal Cardwell [mailto:ncardwell@google.com]
>> > Sent: Tuesday, 06 October 2015 04:06
>> > To: Veaceslav ROMAN
>> > Cc: Eric Dumazet; Ingemar Johansson S; Eric Dumazet; tcpm@ietf.org;
>> > Piers O'Hanlon; Sangtae Ha; end2end-interest@postel.org
>> > Subject: Re: [tcpm] [e2e] TCP HyStart patch deployment
>> >
>> > On Mon, Oct 5, 2015 at 6:07 PM, Veaceslav ROMAN
>> >  wrote:
>> > > We've managed to recompile the kernel 4.1 with tcp_cubic
>> > HYSTART_DELAY_MIN  (10U<<3) , 10ms.
>> > > Well, we had time only for one test cycle, but results are very prom=
ising:
>> > >
>> > > Download 10MB, High Bandwidths, HYSTART_DELAY_MIN  10 ms:
>> > >            Hystart average download time: 1.16 s
>> > >            NoHystart average download time: 0.93  s
>> > ...
>> > >  Download 10MB, High Bandwidths, HYSTART_DELAY_MIN  4 ms:
>> > >            Hystart average download time: 1.42 s
>> > >            NoHystart average download time: 0.89  s
>> >
>> > Thanks! That is very useful and interesting data. Would you be able to
>> > try a few other values, like 15ms and 20ms?
>> >
>> > neal


From nobody Tue Aug 30 08:33:43 2016
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 406DF12D8EE for ; Tue, 30 Aug 2016 07:22:44 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -7.469
X-Spam-Level: 
X-Spam-Status: No, score=-7.469 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-0.548, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1_OmPmJS4yt3 for ; Tue, 30 Aug 2016 07:22:40 -0700 (PDT)
Received: from mx141.netapp.com (mx141.netapp.com [216.240.21.12]) (using TLSv1.2 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id E1F5812D8B3 for ; Tue, 30 Aug 2016 07:17:48 -0700 (PDT)
X-IronPort-AV: E=Sophos;i="5.30,256,1470726000"; d="scan'208";a="143422408"
Received: from hioexcmbx06-prd.hq.netapp.com ([10.122.105.39]) by mx141-out.netapp.com with ESMTP; 30 Aug 2016 07:16:47 -0700
Received: from HIOEXCMBX03-PRD.hq.netapp.com (10.122.105.36) by hioexcmbx06-prd.hq.netapp.com (10.122.105.39) with Microsoft SMTP Server (TLS) id 15.0.1210.3; Tue, 30 Aug 2016 07:16:47 -0700
Received: from HIOEXCMBX03-PRD.hq.netapp.com ([::1]) by hioexcmbx03-prd.hq.netapp.com ([fe80::a009:cb7a:e519:7347%21]) with mapi id 15.00.1210.000; Tue, 30 Aug 2016 07:16:47 -0700
From: "Cui, Cheng" 
To: "tcpm@ietf.org" 
Thread-Topic: question about if a recent Linux patch is compliant with RFC7323 on window scaling
Thread-Index: AQHR/ktWq33ah5ClWE2yMFWt4EpCvqBhx/sA
Date: Tue, 30 Aug 2016 14:16:47 +0000
Message-ID: 
References: 
In-Reply-To: 
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-ms-exchange-messagesentrepresentingtype: 1
x-ms-exchange-transport-fromentityheader: Hosted
x-originating-ip: [10.122.56.79]
Content-Type: text/plain; charset="iso-8859-1"
Content-ID: <6AF8547DD8EBC541BC5CC2B605347475@hq.netapp.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
Archived-At: 
X-Mailman-Approved-At: Tue, 30 Aug 2016 08:33:42 -0700
Subject: Re: [tcpm] question about if a recent Linux patch is compliant with RFC7323 on window scaling
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Tue, 30 Aug 2016 14:22:44 -0000

Six days have passed and no response yet. Can anyone in this board make a c=
omment?

Thanks,
--Cheng Cui
NetApp Scale Out Networking


On 8/24/16, 5:06 PM, "Cui, Cheng"  wrote:

    Hello David,
   =20
    I hope this email could reach you well, because I found related
    discussions about this topic on window scaling and the case of window
    shrinking (or retraction or loss of precision). And I try to make this
    question simple.
   =20
    There is a recent Linux patch at receiver side to round-up advertised
    window due to precision loss of window scaling. It reaches my attention
    because the same problem could also happen between a pair of Linux and
    FreeBSD nodes, and I am not aware of any similar patch in FreeBSD yet.
   =20
    The Linux patch is this:
    http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?=
id=3D6
    07bfbf2d55dd1cfe5368b41c2a81a8c9ccf4723
   =20
    And I quote some description of the Linux patch below:
    > If the sender uses up the entire window before it is shrunk, this can
    >have
    > chaotic effects on the connection. When sending ACKs,
    >tcp_acceptable_seq()
    > will notice that the window has been shrunk since tcp_wnd_end() is be=
fore
    > tp->snd_nxt, which makes it choose tcp_wnd_end() as sequence number.
    > This will fail the receivers checks in tcp_sequence() however since i=
t
    > is before it's tp->rcv_wup, making it respond with a dupack.
   =20
   =20
    If the Linux sender?s behavior is right ("ACK-only packets should be se=
nt
    with the largest in-window sequence number that has ever been sent." re=
f:
    https://www.ietf.org/mail-archive/web/tcpm/current/msg10512.html), it
    actually chooses "tp->snd_una+tp->snd_wnd" (tcp_wnd_end()) instead of
    tp->snd_nxt, as it thought tp->snd_nxt is out of window, in case of
    precision loss which made the receiver?s advertise-window smaller.
    However, the RFC7323 explicitly specifies the receiver should right shi=
ft
    the exact scale factor, without mentioning any further round-up:
   =20
    https://tools.ietf.org/html/rfc7323#page-9
    >    o  The window field (SEG.WND) of every outgoing segment, with the
    >       exception of  segments, MUST be right-shifted by
    >       Rcv.Wind.Shift bits:
    >=20
    >                     SEG.WND =3D RCV.WND >> Rcv.Wind.Shift
   =20
    And also if I am understanding correctly, RFC7323 page 10 (2.4. Address=
ing
    Window Retraction) specifies it is the sender?s responsibility to handl=
e
    the sequence number out of window:
    https://tools.ietf.org/html/rfc7323#page-10
    >    4)  On first retransmission, or if the sequence number is out of
    >        window by less than 2^Rcv.Wind.Shift, then do normal
    >        retransmission(s) without regard to the receiver window as lon=
g
    >        as the original segment was in window when it was sent.
    >=20
    >    5)  Subsequent retransmissions MAY only be sent if they are within
    >        the window announced by the most recent .
   =20
    In your discussion of the link below, if I am understanding correctly, =
you
    were also in favor of "Yes. It is ok to have more receive buffer space
    available than you advertise in the window, but not ok to advertise a
    larger window than you are willing to accept." So I think you were not =
in
    favor of rounding-up the advertise-window.
   =20
    https://www.ietf.org/mail-archive/web/tsvwg/current/msg06475.html
   =20
   =20
    So my question is: Is the Linux patch (on receiver side to round-up the
    advertise-window) RFC compliant? Or is it just the right thing (vendor
    specific) and other systems like FreeBSD have to follow up, so that the=
ir
    talks can have no problem?
   =20
    Thanks and apologize in advance if I did not do enough research,
    --Cheng Cui
   =20
   =20
   =20
   =20
   =20
   =20



From nobody Tue Aug 30 13:35:35 2016
Return-Path: 
X-Original-To: tcpm@ietf.org
Delivered-To: tcpm@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 96B5C12D81F for ; Tue, 30 Aug 2016 13:35:33 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
To: 
X-Test-IDTracker: no
X-IETF-IDTracker: 6.31.0
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <147258933361.23660.2102455637324840350.idtracker@ietfa.amsl.com>
Date: Tue, 30 Aug 2016 13:35:33 -0700
From: IETF Secretariat 
Archived-At: 
Subject: [tcpm] Milestones changed for tcpm WG
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.17
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Tue, 30 Aug 2016 20:35:33 -0000

Changed milestone "Submit document on a time-based fast loss detection
algorithm for TCP to the IESG for publication as an Experimental RFC",
set state to active from review, accepting new milestone.

URL: https://datatracker.ietf.org/wg/tcpm/charter/


From nobody Wed Aug 31 02:23:08 2016
Return-Path: 
X-Original-To: tcpm@ietf.org
Delivered-To: tcpm@ietfa.amsl.com
Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id DE60B12D8C0; Wed, 31 Aug 2016 02:23:07 -0700 (PDT)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
From: "\"IETF Meeting Session Request Tool\"" 
To: 
X-Test-IDTracker: no
X-IETF-IDTracker: 6.31.0
Auto-Submitted: auto-generated
Precedence: bulk
Message-ID: <147263538787.5250.4872177586457189410.idtracker@ietfa.amsl.com>
Date: Wed, 31 Aug 2016 02:23:07 -0700
Archived-At: 
Cc: tcpm@ietf.org, ietf@kuehlewind.net, tcpm-chairs@ietf.org
Subject: [tcpm] tcpm - New Meeting Session Request for IETF 97
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.17
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Wed, 31 Aug 2016 09:23:08 -0000

A new meeting session request has just been submitted by Michael Scharf, a Chair of the tcpm working group.


---------------------------------------------------------
Working Group Name: TCP Maintenance and Minor Extensions
Area Name: Transport Area
Session Requester: Michael Scharf

Number of Sessions: 1
Length of Session(s):  2.5 Hours
Number of Attendees: 80
Conflicts to Avoid: 
 First Priority: tsvwg tsvarea taps mptcp tcpinc httpbis iccrg 
 Second Priority: rmcat rtcweb aqm dclcrg maprg
 Third Priority: ippm teas


Special Requests:
  Further conflicts: quic (if approved), plus (if approved), other BoFs in TSV
---------------------------------------------------------


From nobody Wed Aug 31 02:41:27 2016
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 31C1C12DA7C; Wed, 31 Aug 2016 02:41:26 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -6.921
X-Spam-Level: 
X-Spam-Status: No, score=-6.921 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7GUEXiMppU73; Wed, 31 Aug 2016 02:41:24 -0700 (PDT)
Received: from smtp-fr.alcatel-lucent.com (fr-hpida-esg-02.alcatel-lucent.com [135.245.210.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 49C0312D8D2; Wed, 31 Aug 2016 02:41:21 -0700 (PDT)
Received: from fr712umx4.dmz.alcatel-lucent.com (unknown [135.245.210.45]) by Websense Email Security Gateway with ESMTPS id A2E9F7276196B; Wed, 31 Aug 2016 09:41:16 +0000 (GMT)
Received: from fr712usmtp2.zeu.alcatel-lucent.com (fr712usmtp2.zeu.alcatel-lucent.com [135.239.2.42]) by fr712umx4.dmz.alcatel-lucent.com (GMO-o) with ESMTP id u7V9fIaR000872 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Wed, 31 Aug 2016 09:41:18 GMT
Received: from FR712WXCHHUB03.zeu.alcatel-lucent.com (fr712wxchhub03.zeu.alcatel-lucent.com [135.239.2.74]) by fr712usmtp2.zeu.alcatel-lucent.com (GMO) with ESMTP id u7V9evfw020763 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Wed, 31 Aug 2016 11:41:16 +0200
Received: from FR712WXCHMBA15.zeu.alcatel-lucent.com ([169.254.7.108]) by FR712WXCHHUB03.zeu.alcatel-lucent.com ([135.239.2.74]) with mapi id 14.03.0195.001; Wed, 31 Aug 2016 11:41:09 +0200
From: "Scharf, Michael (Nokia - DE)" 
To: "tcpm@ietf.org" 
Thread-Topic: [tcpm] WG acceptance of draft-cheng-tcpm-rack-01
Thread-Index: AdHhiAFlT51H/JVWTKmYQaN+WFmZZwh4i7dQ
Date: Wed, 31 Aug 2016 09:41:08 +0000
Message-ID: <655C07320163294895BBADA28372AF5D48A0C84B@FR712WXCHMBA15.zeu.alcatel-lucent.com>
References: <655C07320163294895BBADA28372AF5D4892769C@FR712WXCHMBA15.zeu.alcatel-lucent.com>
In-Reply-To: <655C07320163294895BBADA28372AF5D4892769C@FR712WXCHMBA15.zeu.alcatel-lucent.com>
Accept-Language: de-DE, en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [135.239.27.39]
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
Archived-At: 
Cc: "draft-cheng-tcpm-rack@ietf.org" 
Subject: Re: [tcpm] WG acceptance of draft-cheng-tcpm-rack-01
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Wed, 31 Aug 2016 09:41:26 -0000

Hi all,

I am not aware of further comments as a response to my email below. The con=
sensus is therefore to adopt draft-cheng-tcpm-rack-01 as new TCPM WG item.=
=20

@authors: Please resubmit as draft-ietf-tcpm-...

I'd suggest that the authors make a proposal whether to include TLP. We can=
 come back to this question in one of the upcoming TCPM meetings, preferabl=
y based on a text proposal.=20

Thanks

Michael=20


-----Original Message-----
From: tcpm [mailto:tcpm-bounces@ietf.org] On Behalf Of Scharf, Michael (Nok=
ia - DE)
Sent: Tuesday, July 19, 2016 8:37 AM
To: tcpm@ietf.org
Subject: [tcpm] WG acceptance of draft-cheng-tcpm-rack-01

Hi all,

Yesterday we had uniform consensus in the room to adopt draft-cheng-tcpm-ra=
ck-01 as a new TCPM working group item. As a remark, the support was the st=
rongest I have recently seen in TCPM.=20

Therefore I'd like to confirm on the list that it is working group consensu=
s to add a new milestone to the TCPM charter

  Submit document on a time-based fast loss detection algorithm for TCP (RA=
CK) to the IESG for publication as an Experimental RFC

and to use draft-cheng-tcpm-rack-01 as a starting point.

Two additional notes:

- It has been mentioned that further experimentation is needed e.g. regardi=
ng the parameters and therefore the suggested status is Experimental

- It has been proposed to also document TLP in this document as it is timer=
-based mechanism as well

If you have any feedback or suggestions, please let us know in the next two=
 weeks.

Thanks

Michael

_______________________________________________
tcpm mailing list
tcpm@ietf.org
https://www.ietf.org/mailman/listinfo/tcpm


From nobody Wed Aug 31 08:01:44 2016
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6D7C912D0D1 for ; Tue, 30 Aug 2016 21:02:08 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.749
X-Spam-Level: 
X-Spam-Status: No, score=-1.749 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=no autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id F6OjFRlJP3ak for ; Tue, 30 Aug 2016 21:02:07 -0700 (PDT)
Received: from mail-ua0-x235.google.com (mail-ua0-x235.google.com [IPv6:2607:f8b0:400c:c08::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id D9FA812D0A4 for ; Tue, 30 Aug 2016 21:02:06 -0700 (PDT)
Received: by mail-ua0-x235.google.com with SMTP id l94so68074191ual.0 for ; Tue, 30 Aug 2016 21:02:06 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;  h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=CVjDCUUUIh2Cy1dbYnAOrTVUPM/9oNgiiP7f4bOhSrk=; b=wUBhDo76SphWblIeNEj7UCA73CHI7MTZaNkU53SJoRUgx438JnyYWsmqtMwI5IiYbh 09W91EW09SsHxj+QiPeSOyYt9WF+SgErgt/ekcPQE5qp98azC14YtV3/411Fm7WsylrE son5WJnsCG4ENJQmgKR7uQ2VN+laypxjrpYGfasJYyHYetSLNVo5+Uk1+Dez1oaAD5mN KeQEDsueU/AHfWBXENVZyguA8ciJdvZVvNxpHR2MP9N/c7HO7xGkxAeRU4hZSnTZamoN 7tny31mYZ55A8n/AHgm/hArDWI9n3rzFZv/mhRLaxxqtbyqOJjPzQ9n5ZFKsl1K63AY9 Shrg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=CVjDCUUUIh2Cy1dbYnAOrTVUPM/9oNgiiP7f4bOhSrk=; b=k2NIDtpqlCt4H6NDd3gjED8yf0SX8sXKMJXkUq/Z+Ee/bvevaZmhntqnl7kCXIpY3h gSWyfffc++PfV3WyXxMX1jPV2twUJtfDcOHDq8u/ZuGeV3XE5rGEFQL6u/AXivyYbl5g vq/9GnQzmD4Jj+7ypNKXNk4lh9DEidySb8fYRJyk+HksDHR8HBzVHlP3SLEGrfmgw7qG ovspT/RJzO1YIZcPAG/Ab9idp56WEFWgnHNvaw47qcRcUC3vIMkJxfiBYj17PxYxsscO ltOYBf+O/1yE0coPjYIGwkLelW3Mpi6mvYHZmL34+rNEqsFAgERhHWaj2njGVx7x9Pk9 Y8gw==
X-Gm-Message-State: AE9vXwNZwcfn2pGrB8sZKZ7wSH5Zx2GEdyT3bGXabQNDRUHNBDg05akwJaKtdZhWUy83vsOkhX3tgVUVVeQZDw==
X-Received: by 10.159.40.165 with SMTP id d34mr4410396uad.110.1472616125900; Tue, 30 Aug 2016 21:02:05 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.176.4.199 with HTTP; Tue, 30 Aug 2016 21:02:05 -0700 (PDT)
In-Reply-To: 
References:    
From: Sanjeev Ranot 
Date: Wed, 31 Aug 2016 09:32:05 +0530
Message-ID: 
To: Yoshifumi Nishida 
Content-Type: multipart/alternative; boundary=94eb2c12474e132df3053b56294f
Archived-At: 
X-Mailman-Approved-At: Wed, 31 Aug 2016 08:01:42 -0700
Cc: "tcpm@ietf.org" 
Subject: Re: [tcpm] (ref. RFC 7323) Increase of shift count from 14 to 15.
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Wed, 31 Aug 2016 04:02:08 -0000

--94eb2c12474e132df3053b56294f
Content-Type: text/plain; charset=UTF-8

Hi Yoshi,

Yes, using shf.cnt of 15 in connection establishment phase is sufficient
enough, but  while thinking from migration and interoperability this
suggestion of changing shf.cnt during Data Transfer Phase came to mind.

Present behavior is that if window limit is reached then we cannot
transfer no matter how big cwnd is. This limits the rate of growth of
congestion window if not growth completely. Now with high speed interfaces
it will not always be the case that we increase size of MTU (as for jumbo
frames) to get the bandwidth with support of bundling (etherChannels).

Generally when shf.cnt is chosen it defines the size of SND.WND
within range from 0 to max SND.WND. SND.WND proves to be a
constraint in situation discussed below.

Note: With PAWS we need to make sure that it is sufficiently large
enough to provide ample monotonically non-decreasing numbers so that
they can be differentiated or make its use in such a way so that
its always sufficient to distinguish at-least between 2 successive
windows. RFC 7323 defines such a way by putting same TSval for all
segments belonging to the same window.

Before citing situation for the need of dynamic exchange of shf.cnt we
need to discuss why sequence number of greater than 32 bit is not needed
because of PAWS and PAWS timestamp Interface transfer speed and window
size are related

With PAWS, each wrap of timestamp must be >= 2*MSL. This MSL is based
on transfer rate. Assuming 1 timestamp tick is 1 ms, hence if we
transfer 2^30 bytes of data in each window i.e. with shf.cnt = 14 then
maximum transfer rate which we need to get this big a window is of
1 Tera bytes (TbE) with PAWS in place

1000 ms (1s) = 1000 Gb (1TbE)
1ms  = 1 Gb ~ 2^30

Therefore with PAWS timestamp tick of 1ms allows 1 Gb of window size that
we can transmit safely without data corruption due to duplicate data or
ACK segments. This minimum timestamp click needed for PAWS safety is
provided with a base consideration of sequence number of 32 bits in size.
It takes 49.6 days for a 32 bit sequence number to wrap again when
timestamp tick is 1ms.  If in future we were to increase sequence numbers
to 64 bits, then this timestamp tick can be reduced or we MAY never need
PAWS again thereafter if 64 bit sequence number is used and we can never
increase bandwidth speed no matter how much we bundle cables.

The situation discussed assume that we are never going to increase the
sequence number size and we are good with PAWS to provide us safety from
duplicate data and ACK segment or of old packets from previous incarnations
of the same TCP connection.

Another example with shf.cnt = 15 :
With shf.cnt = 15
1000ms  = 2000 Gb (2TbE)
1ms        = 2000000000 bits = 2 Gb ~ 2^31
Therefore with a speed of 2Tbps and 1 ms timestamp tick for PAWS we can
get a window size of 2 Gb.

*Migration Notes:*
This means that when we introduce a shf.cnt of 15, then we MAY either let
PAWS timestamp tick to remain same as used for 1 Gb of window size with
shf.cnt of 14 if its sufficient enough or we may think to readjust if we
want to gain more speed by lowering the previous PAWS timestamp.

Similary with a speed of 4Tbps and 1 ms timestamp tick for PAWS we can
get a window size of 4 Gb.

*Situation:*
TCP connection is going on for days using same shf.cnt which was
exchanged on day 1 when this TCP connection was established. Suppose
due to incredible high transfer rate of the inter-continent link
cwnd == SND.WND was already reached at end of day 1 and now the
transfer rate is limited by SND.WND (decided by shf.cnt) when only
very small fraction of link speed is utilized. Now suppose a very
huge surplus of memory is made available by hot-swappable memory
card inserted on both ends while TCP connection is running its
Data Transfer Phase. Shf.cnt was already decided in beginning of
this TCP connection and it was decided on the basis of amount on
total memory available at that time to be allocated to its RCV.BUF.
It required restarting this ongoing connection to have a new shf.cnt
that will now consider this new additional memory surplus (i.e.
increase RCV.BUF) and start all over again from start or further
from the last index received is area of improvement.

*My proposal is that (this is separate from the on going work on shf.cnt =
15) :*
If TCP stack have support of exchanging shf.cnt in data transfer phase
when large amount of memory is available dynamically then we can achieve
significant high transfer rates without ever having to restart the
connection.

This feature comes in heading "Dynamically adjustable TCP window" and
shf.cnt_interoperability.pdf mentions one of the possible mechanisms
for implementing this introducing 7 new state variables and 1 TCP flag.

Also, just to reiterate regarding migration purposes please look into
PAWS and discuss with stake holders if they are ok with existing
timestamp for new shf.cnt of 15.


Thanks & Regards,
Sanjeev Ranot



On 29 August 2016 at 13:02, Yoshifumi Nishida 
wrote:

> Hi Sanjeev,
>
> On Sun, Aug 28, 2016 at 1:55 AM, Sanjeev Ranot
>  wrote:
> > Hi Yoshi,
> >
> > thanks for replying. I have done some work in these couple of days on
> > how exchange and interoperability of shift count 15 or greater will work.
> >
> > Basically my idea is for the new TCP stack (which will support shf.cnt >=
> > 15)
> > to exchange shf.cnt even in Data Transfer Phase and not only in
> connection
> > establishment. I have presented this exchange work in the document
> enclosed.
> >
> > Please have a look at the document and let me know for any info or
> > clarification
>
> Thanks for the doc.
> I personally think you might want to clarify why we want to change
> shift count in data transfer phase a bit more.
> This is because even if we use shift count=15, I think we can specify
> rather small window size by using a small number for the timestamp
> value. So, if the stack supports shift count=15, I think it can just
> use it from the beginning of the data transfer in most cases.
>
> If your focus is to have a feature to update shift count during data
> transfer phase in general, I think it can be a separate proposal from
> increasing max shift count. Because it looks a more broad topic to me.
>
> Thanks,
> --
> Yoshi
>

--94eb2c12474e132df3053b56294f
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable


Hi Yoshi,

Yes, using shf.cnt of 15 in connecti=
on establishment phase is sufficient 
enough, but=C2=A0 while thinking f=
rom migration and interoperability this 
suggestion of changing shf.cnt =
during Data Transfer Phase came to mind.

Present behavior is that if=
 window limit is reached then we cannot 
transfer no matter how big cwnd=
 is. This limits the rate of growth of 
congestion window if not growth =
completely. Now with high speed interfaces 
it will not always be the ca=
se that we increase size of MTU (as for jumbo 
frames) to get the bandwi=
dth with support of bundling (etherChannels).

Generally when shf.cnt=
 is chosen it defines the size of SND.WND
within range from 0 to max SND=
.WND. SND.WND proves to be a 
constraint in situation discussed below.
Note: With PAWS we need to make sure that it is sufficiently large enough to provide ample monotonically non-decreasing numbers so that 
=
they can be differentiated or make its use in such a way so that 
its al=
ways sufficient to distinguish at-least between 2 successive 
windows. R=
FC 7323 defines such a way by putting same TSval for all 
segments belon=
ging to the same window.

Before citing situation for the need of dyn=
amic exchange of shf.cnt we 
need to discuss why sequence number of grea=
ter than 32 bit is not needed 
because of PAWS and PAWS timestamp Interf=
ace transfer speed and window 
size are related

With PAWS, each w=
rap of timestamp must be >=3D 2*MSL. This MSL is based 
on transfer r=
ate. Assuming 1 timestamp tick is 1 ms, hence if we 
transfer 2^30 bytes=
 of data in each window i.e. with shf.cnt =3D 14 then 
maximum transfer =
rate which we need to get this big a window is of 
1 Tera bytes (TbE) wi=
th PAWS in place

1000 ms (1s) =3D 1000 Gb (1TbE)
1ms=C2=A0 =3D 1 =
Gb ~ 2^30 

Therefore with PAWS timestamp tick of 1ms allows 1 Gb of =
window size that 
we can transmit safely without data corruption due to =
duplicate data or
ACK segments. This minimum timestamp click needed for =
PAWS safety is 
provided with a base consideration of sequence number of=
 32 bits in size. 
It takes 49.6 days for a 32 bit sequence number to wr=
ap again when 
timestamp tick is 1ms.=C2=A0 If in future we were to incr=
ease sequence numbers 
to 64 bits, then this timestamp tick can be reduc=
ed or we MAY never need 
PAWS again thereafter if 64 bit sequence number=
 is used and we can never 
increase bandwidth speed no matter how much w=
e bundle cables. 

The situation discussed assume that we are never g=
oing to increase the 
sequence number size and we are good with PAWS to =
provide us safety from 
duplicate data and ACK segment or of old packets=
 from previous incarnations 
of the same TCP connection.

Another =
example with shf.cnt =3D 15 :
With shf.cnt =3D 15 
1000ms=C2=A0 =3D 2=
000 Gb (2TbE)
1ms=C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 =3D 2000000000 bi=
ts =3D 2 Gb ~ 2^31
Therefore with a speed of 2Tbps and 1 ms timestamp ti=
ck for PAWS we can 
get a window size of 2 Gb. 

Migration Note=
s:
This means that when we introduce a shf.cnt of 15, then we MAY ei=
ther let 
PAWS timestamp tick to remain same as used for 1 Gb of window =
size with 
shf.cnt of 14 if its sufficient enough or we may think to rea=
djust if we 
want to gain more speed by lowering the previous PAWS times=
tamp.

Similary with a speed of 4Tbps and 1 ms timestamp tick for PAW=
S we can 
get a window size of 4 Gb. 

Situation: 
TCP c=
onnection is going on for days using same shf.cnt which was 
exchanged o=
n day 1 when this TCP connection was established. Suppose 
due to incred=
ible high transfer rate of the inter-continent link 
cwnd =3D=3D SND.WND=
 was already reached at end of day 1 and now the 
transfer rate is limit=
ed by SND.WND (decided by shf.cnt) when only 
very small fraction of lin=
k speed is utilized. Now suppose a very 
huge surplus of memory is made =
available by hot-swappable memory 
card inserted on both ends while TCP =
connection is running its 
Data Transfer Phase. Shf.cnt was already deci=
ded in beginning of 
this TCP connection and it was decided on the basis=
 of amount on 
total memory available at that time to be allocated to it=
s RCV.BUF. 
It required restarting this ongoing connection to have a new=
 shf.cnt 
that will now consider this new additional memory surplus (i.e=
. 
increase RCV.BUF) and start all over again from start or further 
=
from the last index received is area of improvement. 

My proposal=
 is that (this is separate from the on going work on shf.cnt =3D 15) :<=
br>If TCP stack have support of exchanging shf.cnt in data transfer phase <=
br>when large amount of memory is available dynamically then we can achieve=
 
significant high transfer rates without ever having to restart the con=
nection.

This feature comes in heading "Dynamically adjustable =
TCP window" and 
shf.cnt_interoperability.pdf mentions one of the p=
ossible mechanisms 
for implementing this introducing 7 new state variab=
les and 1 TCP flag.

Also, just to reiterate regarding mig=
ration purposes please look into 
PAWS and discuss with stake holders if=
 they are ok with existing
timestamp for new shf.cnt of 15. <=
br>


Thanks & Regards,
Sanjeev Ranot
=




On 29 August 2016 at 13:02, Yoshifumi Nishida <nishida@=
sfc.wide.ad.jp> wrote:
Hi S=
anjeev,



On Sun, Aug 28, 2016 at 1:55 AM, Sanjeev Ranot

<sanjeev.ranot.81@gmail.co=
m> wrote:

> Hi Yoshi,

>

> thanks for replying. I have done some work in these couple of days on<=
br>
> how exchange and interoperability of shift count 15 or greater will wo=
rk.

>

> Basically my idea is for the new TCP stack (which will support shf.cnt=
 >=3D

> 15)

> to exchange shf.cnt even in Data Transfer Phase and not only in connec=
tion

> establishment. I have presented this exchange work in the document enc=
losed.

>

> Please have a look at the document and let me know for any info or

> clarification



Thanks for the doc.

I personally think you might want to clarify why we want to change

shift count in data transfer phase a bit more.

This is because even if we use shift count=3D15, I think we can specify

rather small window size by using a small number for the timestamp

value. So, if the stack supports shift count=3D15, I think it can just

use it from the beginning of the data transfer in most cases.



If your focus is to have a feature to update shift count during data

transfer phase in general, I think it can be a separate proposal from

increasing max shift count. Because it looks a more broad topic to me.



Thanks,

--

Yoshi





--94eb2c12474e132df3053b56294f--


From nobody Wed Aug 31 08:43:00 2016
Return-Path: 
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2E89512B02A for ; Wed, 31 Aug 2016 08:42:59 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -3.249
X-Spam-Level: 
X-Spam-Status: No, score=-3.249 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RP_MATCHES_RCVD=-0.548, SPF_PASS=-0.001] autolearn=unavailable autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=google.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mNWXz0sxz6Ra for ; Wed, 31 Aug 2016 08:42:55 -0700 (PDT)
Received: from mail-it0-x236.google.com (mail-it0-x236.google.com [IPv6:2607:f8b0:4001:c0b::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id AF56312D195 for ; Wed, 31 Aug 2016 08:25:33 -0700 (PDT)
Received: by mail-it0-x236.google.com with SMTP id e124so49985293ith.0 for ; Wed, 31 Aug 2016 08:25:33 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=Cmf8D4Vra7CF77RIVOrWPgJLWZ7BAtq4tQrMMUCMXpQ=; b=aSP0YKdIVgffKUp6DsE7UGVuLsIr9issj5JUKmLD0yiYdfUywlP7ES9iL5S6Ghmrtr Kd/FzgSqdOXQRnyJfg3Qh+DZBlU94jVKNhjMt51X0fycIBklyYgvMsbXB0lfXgZZCU/3 HP4EODufVUTXsAzRJ0TOlp+V87mWKCvjqt0+oXS0v1BsMxA9GzdR70i1hLLVRBHvHPhl 6+DM/Vr5KouDlw4UshkIxFub85TlRMCjAapQYHy33TYzbce1wTavVTQl9BKa9QLXbdY3 /vOlspiDXYPrkZakhw7t9S9GpoFuLrK1mkeTe7rCULydT4l5KLR9hLJKyAb/mVcexl18 /lmA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=Cmf8D4Vra7CF77RIVOrWPgJLWZ7BAtq4tQrMMUCMXpQ=; b=m0IAEoP/VkThIDG23t7iVfKw/J9DltvR4jXoSKfVbh/nut6uPFvejQ49QvdOHbOVlL PuzkDNk4AT9lGXrZHtVDaP1roH08sfB8ftw8un3r2j6NR66+SkiLBOc7Z2MTI46RYxGk HjLQQ9fGVN9yn2h3qczfXY3gorQCx0+AByAUVSua5rnbbB3LfKzNfSAwaP98214skvtK xG6Oelfy/BgChGdtg5Z0fdz/DXITB0ia8d3oBOesXwFjW42W/2grHVNsRFwn14sv8NYY Nm2hQywBw/PwK5HkefOF3F+950DJnjaDMtoOPKfpdAGGALMEnu6f/BEsQU/IR5Q2Cbfk ongw==
X-Gm-Message-State: AE9vXwMmv/Fut61aXEKlhRjJACAP4ArdQAn2KzvICra/RHGzmsbFVenGZKzhqz2CYJd3Gcl2HUA/5muiIQtcA8cz
X-Received: by 10.36.147.68 with SMTP id y65mr15044160itd.76.1472657132794; Wed, 31 Aug 2016 08:25:32 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.64.167.140 with HTTP; Wed, 31 Aug 2016 08:24:52 -0700 (PDT)
In-Reply-To: <655C07320163294895BBADA28372AF5D48A0C84B@FR712WXCHMBA15.zeu.alcatel-lucent.com>
References: <655C07320163294895BBADA28372AF5D4892769C@FR712WXCHMBA15.zeu.alcatel-lucent.com> <655C07320163294895BBADA28372AF5D48A0C84B@FR712WXCHMBA15.zeu.alcatel-lucent.com>
From: Yuchung Cheng 
Date: Wed, 31 Aug 2016 08:24:52 -0700
Message-ID: 
To: "Scharf, Michael (Nokia - DE)" 
Content-Type: text/plain; charset=UTF-8
Archived-At: 
Cc: "tcpm@ietf.org" , Matthew Mathis , "draft-cheng-tcpm-rack@ietf.org" 
Subject: Re: [tcpm] WG acceptance of draft-cheng-tcpm-rack-01
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Wed, 31 Aug 2016 15:42:59 -0000

On Wed, Aug 31, 2016 at 2:41 AM, Scharf, Michael (Nokia - DE)
 wrote:
>
> Hi all,
>
> I am not aware of further comments as a response to my email below. The consensus is therefore to adopt draft-cheng-tcpm-rack-01 as new TCPM WG item.
>
> @authors: Please resubmit as draft-ietf-tcpm-...
>
> I'd suggest that the authors make a proposal whether to include TLP. We can come back to this question in one of the upcoming TCPM meetings, preferably based on a text proposal.
Thanks Michael. I've discussed with the authors of TLP Nandita and
Matt. Our consensus was to first fuse TLP (as an optimization) and
RACK together in one draft to get WG feedback. We'd resubmit a draft
before the next meeting deadline.

Personally I am continue to work on RACK implementation. One problem
is the current algorithm requires scanning the scoreboard for every
ACK. This can take too much cycles on fast connection with lots of
losses and sacks. Another test I am running is to use RACK + TLP alone
without any other recovery heuristics in Linux, and make sure we don't
see regressions in boundary cases. I think both are important to make
sure for the "working code" principle for the new proposal .


>
> Thanks
>
> Michael
>
>
> -----Original Message-----
> From: tcpm [mailto:tcpm-bounces@ietf.org] On Behalf Of Scharf, Michael (Nokia - DE)
> Sent: Tuesday, July 19, 2016 8:37 AM
> To: tcpm@ietf.org
> Subject: [tcpm] WG acceptance of draft-cheng-tcpm-rack-01
>
> Hi all,
>
> Yesterday we had uniform consensus in the room to adopt draft-cheng-tcpm-rack-01 as a new TCPM working group item. As a remark, the support was the strongest I have recently seen in TCPM.
>
> Therefore I'd like to confirm on the list that it is working group consensus to add a new milestone to the TCPM charter
>
>   Submit document on a time-based fast loss detection algorithm for TCP (RACK) to the IESG for publication as an Experimental RFC
>
> and to use draft-cheng-tcpm-rack-01 as a starting point.
>
> Two additional notes:
>
> - It has been mentioned that further experimentation is needed e.g. regarding the parameters and therefore the suggested status is Experimental
>
> - It has been proposed to also document TLP in this document as it is timer-based mechanism as well
>
> If you have any feedback or suggestions, please let us know in the next two weeks.
>
> Thanks
>
> Michael
>
> _______________________________________________
> tcpm mailing list
> tcpm@ietf.org
> https://www.ietf.org/mailman/listinfo/tcpm