From per.hurtig@gmail.com Sun Jul 1 23:45:49 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9B5B511E8136 for ; Sun, 1 Jul 2012 23:45:49 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.977 X-Spam-Level: X-Spam-Status: No, score=-2.977 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GEIThR3qnfpe for ; Sun, 1 Jul 2012 23:45:48 -0700 (PDT) Received: from mail-gh0-f172.google.com (mail-gh0-f172.google.com [209.85.160.172]) by ietfa.amsl.com (Postfix) with ESMTP id 29A5811E8179 for ; Sun, 1 Jul 2012 23:45:47 -0700 (PDT) Received: by ghbg16 with SMTP id g16so4286255ghb.31 for ; Sun, 01 Jul 2012 23:45:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:content-type; bh=yIhiuEP0QZmWB7eZPfzcHcTJOXpnxEHPiGOW9Vdm90g=; b=fWC35e4hkZYt12CtuzPT0nkjC+rj42D1SFJeiCrUGTjKYwsr7l8a4gPt6J8M4IC8wM i/wvNc65F5cJJeAEk/xPvVNim/VtJQeJGLjBnpmfTd+kKRwvCqQ86hH5vMolmxLbQu29 Dzn1RDM4Fu/nr/VVFzsuBrevi0pU0cwWh6Irxv4M1BUbYrmkdmjQcaB1fLqRgPkGPhs5 gHyWnUy1I36iirzF5yHNW4F0ILmwqz7O/PyCz1ZKfzPk7quleV9TxoosJRIStheeqRkU iuq2k1IKt21KPJ9drB6beNUrBMYQd5LhqGa9Hf6/XwTOUdryc+CWdCjn0AUrfFg3Mu1B A5Cw== MIME-Version: 1.0 Received: by 10.50.36.227 with SMTP id t3mr6561625igj.13.1341211551042; Sun, 01 Jul 2012 23:45:51 -0700 (PDT) Sender: per.hurtig@gmail.com Received: by 10.231.189.91 with HTTP; Sun, 1 Jul 2012 23:45:51 -0700 (PDT) In-Reply-To: <20120628104113.8709.30553.idtracker@ietfa.amsl.com> References: <20120628104113.8709.30553.idtracker@ietfa.amsl.com> Date: Mon, 2 Jul 2012 08:45:51 +0200 X-Google-Sender-Auth: WUfF3p78Dg0cACtIXdfcFb3G5ok Message-ID: From: Per Hurtig To: tcpm Content-Type: text/plain; charset=ISO-8859-1 Subject: [tcpm] New Version Notification for draft-hurtig-tcpm-rtorestart-02.txt X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Jul 2012 06:45:49 -0000 Hi all, we've now updated the modified RTO restart draft to not require implementations to save the transmission times of every segment. There are also some changes to the discussion section and various small text changes. Per A new version of I-D, draft-hurtig-tcpm-rtorestart-02.txt has been successfully submitted by Per Hurtig and posted to the IETF repository. Filename: draft-hurtig-tcpm-rtorestart Revision: 02 Title: TCP and SCTP RTO Restart Creation date: 2012-06-28 WG ID: Individual Submission Number of pages: 9 URL: http://www.ietf.org/internet-drafts/draft-hurtig-tcpm-rtorestart-02.txt Status: http://datatracker.ietf.org/doc/draft-hurtig-tcpm-rtorestart Htmlized: http://tools.ietf.org/html/draft-hurtig-tcpm-rtorestart-02 Diff: http://tools.ietf.org/rfcdiff?url2=draft-hurtig-tcpm-rtorestart-02 Abstract: This document describes a modified algorithm for managing the TCP and SCTP retransmission timers that provides faster loss recovery when a connection's amount of outstanding data is small. The modification allows the transport to restart its retransmission timer more aggressively in situations where fast retransmit cannot be used. This enables faster loss detection and recovery for connections that are short-lived or application-limited. The IETF Secretariat From ycheng@google.com Mon Jul 2 12:21:02 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 69E7611E80A3 for ; Mon, 2 Jul 2012 12:21:02 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.977 X-Spam-Level: X-Spam-Status: No, score=-102.977 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id IPs64aEk+pw9 for ; Mon, 2 Jul 2012 12:21:01 -0700 (PDT) Received: from mail-ob0-f172.google.com (mail-ob0-f172.google.com [209.85.214.172]) by ietfa.amsl.com (Postfix) with ESMTP id 33F9011E8088 for ; Mon, 2 Jul 2012 12:21:01 -0700 (PDT) Received: by obbwc20 with SMTP id wc20so9976923obb.31 for ; Mon, 02 Jul 2012 12:21:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type:content-transfer-encoding:x-system-of-record; bh=kA8FN3L/PSxiMjFz14YIMvmJ0vsMZBy+9ADUJecR6sE=; b=L2xtI8YHn8s6BNcQk2XuhDShN5zTGxBCM41IQlQN7SLar7f/OmquE6P90i5PEfBsNH XKNcUeEkqcANAZtaY6T1nq1zeJWuA20e4NgeDRhYMviLG+CvpnfydarVOBlD+dUprZTK 6Awg2K6wGqeXhC/zAdm+j96Yj+e4HjT9NyVgEORrEjaXxSDsmZ4heOZEyY29b0qxJOBv H1C7reDTbfcEb7FDqG8Mnqm/97vJrIWXOZXRqqL7+iy2W08HOw0fccco+8K39+Sbz7nN RzBvFsQ88jB1NDWaAgvdHnOm5i0+a7gPDjaQ9lKNHnfMzVHeA+TrMATukXzhhGLfyJ/Q QRCA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type:content-transfer-encoding:x-system-of-record :x-gm-message-state; bh=kA8FN3L/PSxiMjFz14YIMvmJ0vsMZBy+9ADUJecR6sE=; b=GBBAfUX6SaTOIla4ryX/iiT6nzqDAImHcT/qEQdO7YlrUaRPODpqlTSUyeYMOineu8 b/Uv7TvyWuLrQkqqhI5looRFL6RiB5VQfFqsn3Fw7nEp2BGsU2TFg8DBfO5iA/Z+PZSC nGbX+k7RXHur2GY1SFQHSIwnMQ0Thx/lOqNL5byemVXb0xC+oLC64uN/Mv8qMggY+XOh MqnSQflAARl1f7ch4muZbo1eUgOHv86V/y4DmljfvSwgERS9CN6z2yt2g1fCRbq2Uvv0 +mN8zCKLCpdQlUmKT3vYIFDXS5qM3aXu//l+JppLBhBPDdaOLurBwniBthOT19F44FMq cI7Q== Received: by 10.182.169.98 with SMTP id ad2mr9639677obc.23.1341256866825; Mon, 02 Jul 2012 12:21:06 -0700 (PDT) Received: by 10.182.169.98 with SMTP id ad2mr9639657obc.23.1341256866618; Mon, 02 Jul 2012 12:21:06 -0700 (PDT) MIME-Version: 1.0 Received: by 10.60.140.164 with HTTP; Mon, 2 Jul 2012 12:20:46 -0700 (PDT) In-Reply-To: References: <20120628104113.8709.30553.idtracker@ietfa.amsl.com> From: Yuchung Cheng Date: Mon, 2 Jul 2012 12:20:46 -0700 Message-ID: To: Per Hurtig Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-System-Of-Record: true X-Gm-Message-State: ALoCoQk5RvHEm/vyv1fPs81kqgThmNc6yzezf9vFxjBCAcXdynF23G36d1acQhldUkrLNrPwKStkrkU4JXxFqqsxNjzeMZS5zU6D/ug9qqXeqbKvGm11uX/3AiDZRUpqKK31u+kezXAFK4v7SRai+oG2FYC2ZblsjUOjSpAT8cZsuVI6dVXly5E= Cc: tcpm Subject: Re: [tcpm] New Version Notification for draft-hurtig-tcpm-rtorestart-02.txt X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Jul 2012 19:21:02 -0000 On Sun, Jul 1, 2012 at 11:45 PM, Per Hurtig wrote: > Hi all, > > we've now updated the modified RTO restart draft to not require > implementations to save the transmission times of every segment. There > are also some changes to the discussion section and various small text > changes. > > > Per > > A new version of I-D, draft-hurtig-tcpm-rtorestart-02.txt > has been successfully submitted by Per Hurtig and posted to the > IETF repository. > > Filename: =A0 =A0 =A0 =A0draft-hurtig-tcpm-rtorestart > Revision: =A0 =A0 =A0 =A002 > Title: =A0 =A0 =A0 =A0 =A0 TCP and SCTP RTO Restart > Creation date: =A0 2012-06-28 > WG ID: =A0 =A0 =A0 =A0 =A0 Individual Submission > Number of pages: 9 > URL: > http://www.ietf.org/internet-drafts/draft-hurtig-tcpm-rtorestart-02.txt > Status: =A0 =A0 =A0 =A0 =A0http://datatracker.ietf.org/doc/draft-hurtig-t= cpm-rtorestart > Htmlized: =A0 =A0 =A0 =A0http://tools.ietf.org/html/draft-hurtig-tcpm-rto= restart-02 > Diff: > http://tools.ietf.org/rfcdiff?url2=3Ddraft-hurtig-tcpm-rtorestart-02 > > Abstract: > =A0 =A0This document describes a modified algorithm for managing the TCP = and > =A0 =A0SCTP retransmission timers that provides faster loss recovery when= a > =A0 =A0connection's amount of outstanding data is small. =A0The modificat= ion > =A0 =A0allows the transport to restart its retransmission timer more > =A0 =A0aggressively in situations where fast retransmit cannot be used. > =A0 =A0This enables faster loss detection and recovery for connections th= at > =A0 =A0are short-lived or application-limited. I asked similar questions for every revision but didn't see the response in the list or the newest draft: I am concerned about this following hypothetical scenario: t0.0s: bursts packets 1-20, mount RTO to 2second t1.8s: gets ack of packets 2,4,6,8,10,12,14,16, 18 every 200ms. Ack of 18 arrives at 1.8s This draft will mount an RTO of 200ms for packet 19 on ack18, which is likely to be spurious. The problem is that T_earliest could be starting from large send a while back even when current FlightSize is small. Moreover, I don't think this is rare so I suspect the performance benefit will be offset by the spurious RTO. The (un-cited) references [EL04] thus argues current standard offers a safe margin. Some data or response to this question is appreciated. Thanks. Yuchung > > > > > The IETF Secretariat > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm From nishida@sfc.wide.ad.jp Wed Jul 4 02:28:23 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 02CA621F8744 for ; Wed, 4 Jul 2012 02:28:23 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -101.442 X-Spam-Level: X-Spam-Status: No, score=-101.442 tagged_above=-999 required=5 tests=[AWL=0.535, BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Ks2y0-rOOxZs for ; Wed, 4 Jul 2012 02:28:22 -0700 (PDT) Received: from mail.sfc.wide.ad.jp (mail.sfc.wide.ad.jp [IPv6:2001:200:0:8803:203:178:142:146]) by ietfa.amsl.com (Postfix) with ESMTP id 0B01C21F86EA for ; Wed, 4 Jul 2012 02:28:19 -0700 (PDT) Received: from mail-lb0-f172.google.com (mail-lb0-f172.google.com [209.85.217.172]) by mail.sfc.wide.ad.jp (Postfix) with ESMTPSA id 8F53D2780EC for ; Wed, 4 Jul 2012 18:28:28 +0900 (JST) Received: by lbbgo11 with SMTP id go11so11122402lbb.31 for ; Wed, 04 Jul 2012 02:28:26 -0700 (PDT) MIME-Version: 1.0 Received: by 10.152.147.33 with SMTP id th1mr21315059lab.9.1341394105992; Wed, 04 Jul 2012 02:28:25 -0700 (PDT) Received: by 10.112.90.176 with HTTP; Wed, 4 Jul 2012 02:28:25 -0700 (PDT) Date: Wed, 4 Jul 2012 02:28:25 -0700 Message-ID: From: Yoshifumi Nishida To: "tcpm@ietf.org\"" Content-Type: text/plain; charset=ISO-8859-1 Subject: [tcpm] WG Last Call: draft-ietf-tcpm-initcwnd X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Jul 2012 09:28:23 -0000 Hi Folks, We decide to start a WG last call on draft-ietf-tcpm-initcwnd. Since there has been lots of discussions on this draft, we'll run for longer than usual - until July 30th. This means we can make some time at Vancouver if critical points are raised during the last call. Please provide your comments before the deadline. Thanks, -- tcpm co-chairs -------------------------------- A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the TCP Maintenance and Minor Extensions Working Group of the IETF. Title : Increasing TCP's Initial Window Author(s) : Jerry Chu Nandita Dukkipati Yuchung Cheng Matt Mathis Filename : draft-ietf-tcpm-initcwnd-04.txt Pages : 24 Date : 2012-06-28 Abstract: This document proposes an experiment to increase the permitted TCP initial window (IW) from between 2 and 4 segments, as specified in RFC 3390, to 10 segments, with a fallback to the existing recommendation when performance issues are detected. It discusses the motivation behind the increase, the advantages and disadvantages of the higher initial window, and presents results from several large scale experiments showing that the higher initial window improves the overall performance of many web services without resulting in a congestion collapse. The document closes with a discussion of usage and deployment for further experimental purpose recommended by the IETF TCP Maintenance and Minor Extensions (TCPM) working group. From michawe@ifi.uio.no Fri Jul 6 02:25:09 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 93D7021F8700 for ; Fri, 6 Jul 2012 02:25:09 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.599 X-Spam-Level: X-Spam-Status: No, score=-102.599 tagged_above=-999 required=5 tests=[AWL=0.000, BAYES_00=-2.599, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Sath1AVW55Zo for ; Fri, 6 Jul 2012 02:25:08 -0700 (PDT) Received: from mail-out1.uio.no (mail-out1.uio.no [IPv6:2001:700:100:10::57]) by ietfa.amsl.com (Postfix) with ESMTP id 78CB821F86BD for ; Fri, 6 Jul 2012 02:25:07 -0700 (PDT) Received: from mail-mx3.uio.no ([129.240.10.44]) by mail-out1.uio.no with esmtp (Exim 4.75) (envelope-from ) id 1Sn4mw-0007n2-UN; Fri, 06 Jul 2012 11:25:22 +0200 Received: from boomerang.ifi.uio.no ([129.240.68.135]) by mail-mx3.uio.no with esmtp (Exim 4.76) (envelope-from ) id 1Sn4mw-00071z-EY; Fri, 06 Jul 2012 11:25:22 +0200 Message-ID: <4FF6AF01.7000601@ifi.uio.no> Date: Fri, 06 Jul 2012 11:25:21 +0200 From: Michael Welzl User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:13.0) Gecko/20120614 Thunderbird/13.0.1 MIME-Version: 1.0 To: tcpm@ietf.org, "Per Hurtig (work)" , Andreas Petlund References: <20120628104113.8709.30553.idtracker@ietfa.amsl.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-UiO-Ratelimit-Test: rcpts/h 10 msgs/h 6 sum rcpts/h 14 sum msgs/h 8 total rcpts 21336 max rcpts/h 58 ratelimit 0 X-UiO-Spam-info: not spam, SpamAssassin (score=-5.0, required=5.0, autolearn=disabled, T_RP_MATCHES_RCVD=-0.01, UIO_MAIL_IS_INTERNAL=-5, uiobl=NO, uiouri=NO) X-UiO-Scanned: D2B3AF0EDFD296808FC3A7438ED3763F793DF068 X-UiO-SPAM-Test: remote_host: 129.240.68.135 spam_score: -49 maxlevel 80 minaction 2 bait 0 mail/h: 6 total 8913 max/h 20 blacklist 0 greylist 0 ratelimit 0 Subject: Re: [tcpm] New Version Notification for draft-hurtig-tcpm-rtorestart-02.txt X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Jul 2012 09:25:09 -0000 Hi Yuchung, First of all, my apologies for answering so slowly, and for forgetting to answer your previous similar comment. What happened then is that each of us authors assumed that the other would take care of it, and in the end, I had forgotten about it. See my answer in line below: On 7/2/12 9:20 PM, Yuchung Cheng wrote: > On Sun, Jul 1, 2012 at 11:45 PM, Per Hurtig wrote: >> Hi all, >> >> we've now updated the modified RTO restart draft to not require >> implementations to save the transmission times of every segment. There >> are also some changes to the discussion section and various small text >> changes. >> >> >> Per >> >> A new version of I-D, draft-hurtig-tcpm-rtorestart-02.txt >> has been successfully submitted by Per Hurtig and posted to the >> IETF repository. >> >> Filename: draft-hurtig-tcpm-rtorestart >> Revision: 02 >> Title: TCP and SCTP RTO Restart >> Creation date: 2012-06-28 >> WG ID: Individual Submission >> Number of pages: 9 >> URL: >> http://www.ietf.org/internet-drafts/draft-hurtig-tcpm-rtorestart-02.txt >> Status: http://datatracker.ietf.org/doc/draft-hurtig-tcpm-rtorestart >> Htmlized: http://tools.ietf.org/html/draft-hurtig-tcpm-rtorestart-02 >> Diff: >> http://tools.ietf.org/rfcdiff?url2=draft-hurtig-tcpm-rtorestart-02 >> >> Abstract: >> This document describes a modified algorithm for managing the TCP and >> SCTP retransmission timers that provides faster loss recovery when a >> connection's amount of outstanding data is small. The modification >> allows the transport to restart its retransmission timer more >> aggressively in situations where fast retransmit cannot be used. >> This enables faster loss detection and recovery for connections that >> are short-lived or application-limited. > I asked similar questions for every revision but didn't see the response in > the list or the newest draft: > > I am concerned about this following hypothetical scenario: > > t0.0s: bursts packets 1-20, mount RTO to 2second > t1.8s: gets ack of packets 2,4,6,8,10,12,14,16, 18 every 200ms. > Ack of 18 arrives at 1.8s > > This draft will mount an RTO of 200ms for packet 19 on ack18, which > is likely to be spurious. The problem is that T_earliest could be > starting from large send a while back even when current FlightSize > is small. Moreover, I don't think this is rare so I suspect the > performance benefit will be offset by the spurious RTO. ( I wonder what made you pick 20, not 10 packets for your example? ;-) ) Seriously: you raise a very valid concern here, a concern which we haven't yet investigated (hence: no data). However, it seems to me that the situation you describe can really only happen when the sender transmits a sudden burst of data without waiting for new incoming ACKs. So this could be fixed by disabling the mechanism in Slow Start, right? What do you think? (or, maybe better, define usage in Slow Start as a more experimental, "aggressive" variant of our scheme, and only have it enabled in Congestion Avoidance by default) > The (un-cited) references [EL04] thus argues current standard > offers a safe margin. Some data or response to this question is > appreciated. Thanks. Thanks a lot for spotting that! Of course it wasn't the intention to have EL04 uncited. Cheers, Michael From nishida@sfc.wide.ad.jp Fri Jul 6 12:05:30 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1F3A611E809F for ; Fri, 6 Jul 2012 12:05:30 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -101.561 X-Spam-Level: X-Spam-Status: No, score=-101.561 tagged_above=-999 required=5 tests=[AWL=0.416, BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tEfSVUUhznoZ for ; Fri, 6 Jul 2012 12:05:29 -0700 (PDT) Received: from mail.sfc.wide.ad.jp (mail.sfc.wide.ad.jp [IPv6:2001:200:0:8803:203:178:142:146]) by ietfa.amsl.com (Postfix) with ESMTP id 2EE7C11E8079 for ; Fri, 6 Jul 2012 12:05:28 -0700 (PDT) Received: from mail-lb0-f172.google.com (mail-lb0-f172.google.com [209.85.217.172]) by mail.sfc.wide.ad.jp (Postfix) with ESMTPSA id 10F0D2780C7 for ; Sat, 7 Jul 2012 04:05:42 +0900 (JST) Received: by lbbgo11 with SMTP id go11so14889301lbb.31 for ; Fri, 06 Jul 2012 12:05:40 -0700 (PDT) MIME-Version: 1.0 Received: by 10.112.49.100 with SMTP id t4mr14142323lbn.10.1341601540477; Fri, 06 Jul 2012 12:05:40 -0700 (PDT) Received: by 10.112.90.176 with HTTP; Fri, 6 Jul 2012 12:05:40 -0700 (PDT) In-Reply-To: <4FF6AF01.7000601@ifi.uio.no> References: <20120628104113.8709.30553.idtracker@ietfa.amsl.com> <4FF6AF01.7000601@ifi.uio.no> Date: Fri, 6 Jul 2012 12:05:40 -0700 Message-ID: From: Yoshifumi Nishida To: Michael Welzl Content-Type: text/plain; charset=ISO-8859-1 Cc: tcpm@ietf.org, Andreas Petlund Subject: Re: [tcpm] New Version Notification for draft-hurtig-tcpm-rtorestart-02.txt X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Jul 2012 19:05:30 -0000 Hello, On Fri, Jul 6, 2012 at 2:25 AM, Michael Welzl wrote: > Hi Yuchung, > > First of all, my apologies for answering so slowly, and for forgetting to > answer your previous similar comment. > What happened then is that each of us authors assumed that the other would > take care of it, and in the end, I had forgotten about it. > > See my answer in line below: > > > > On 7/2/12 9:20 PM, Yuchung Cheng wrote: >> >> On Sun, Jul 1, 2012 at 11:45 PM, Per Hurtig wrote: >>> >>> Hi all, >>> >>> we've now updated the modified RTO restart draft to not require >>> implementations to save the transmission times of every segment. There >>> are also some changes to the discussion section and various small text >>> changes. >>> >>> >>> Per >>> >>> A new version of I-D, draft-hurtig-tcpm-rtorestart-02.txt >>> has been successfully submitted by Per Hurtig and posted to the >>> IETF repository. >>> >>> Filename: draft-hurtig-tcpm-rtorestart >>> Revision: 02 >>> Title: TCP and SCTP RTO Restart >>> Creation date: 2012-06-28 >>> WG ID: Individual Submission >>> Number of pages: 9 >>> URL: >>> http://www.ietf.org/internet-drafts/draft-hurtig-tcpm-rtorestart-02.txt >>> Status: >>> http://datatracker.ietf.org/doc/draft-hurtig-tcpm-rtorestart >>> Htmlized: >>> http://tools.ietf.org/html/draft-hurtig-tcpm-rtorestart-02 >>> Diff: >>> http://tools.ietf.org/rfcdiff?url2=draft-hurtig-tcpm-rtorestart-02 >>> >>> Abstract: >>> This document describes a modified algorithm for managing the TCP and >>> SCTP retransmission timers that provides faster loss recovery when a >>> connection's amount of outstanding data is small. The modification >>> allows the transport to restart its retransmission timer more >>> aggressively in situations where fast retransmit cannot be used. >>> This enables faster loss detection and recovery for connections that >>> are short-lived or application-limited. >> >> I asked similar questions for every revision but didn't see the response >> in >> the list or the newest draft: >> >> I am concerned about this following hypothetical scenario: >> >> t0.0s: bursts packets 1-20, mount RTO to 2second >> t1.8s: gets ack of packets 2,4,6,8,10,12,14,16, 18 every 200ms. >> Ack of 18 arrives at 1.8s >> >> This draft will mount an RTO of 200ms for packet 19 on ack18, which >> is likely to be spurious. The problem is that T_earliest could be >> starting from large send a while back even when current FlightSize >> is small. Moreover, I don't think this is rare so I suspect the >> performance benefit will be offset by the spurious RTO. > > ( I wonder what made you pick 20, not 10 packets for your example? ;-) ) > > Seriously: you raise a very valid concern here, a concern which we haven't > yet investigated (hence: no data). > However, it seems to me that the situation you describe can really only > happen when the sender transmits a sudden burst of data without waiting for > new incoming ACKs. So this could be fixed by disabling the mechanism in Slow > Start, right? What do you think? I think there are some situations where TCP sends burst of packets. ACK lost/compression, recovery from packet loss, small pause of application.. Is it ok to exclude these situations? Thanks, -- Yoshifumi From mattmathis@google.com Sat Jul 7 15:02:53 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7A5FC21F8547 for ; Sat, 7 Jul 2012 15:02:53 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.977 X-Spam-Level: X-Spam-Status: No, score=-102.977 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id X+c6+HZFo2Oq for ; Sat, 7 Jul 2012 15:02:52 -0700 (PDT) Received: from mail-wi0-f178.google.com (mail-wi0-f178.google.com [209.85.212.178]) by ietfa.amsl.com (Postfix) with ESMTP id 65C6D21F854D for ; Sat, 7 Jul 2012 15:02:52 -0700 (PDT) Received: by wibhr14 with SMTP id hr14so1464400wib.13 for ; Sat, 07 Jul 2012 15:03:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type :x-system-of-record; bh=CcVQ8pIFwpmz/p4LmCfpsP2fQgCIvd1wkazLrIzS83g=; b=ms1vGi5A4lvGSGma7t1/Zd7A0Q+jwGF6PSrl1Wffxa8b5iEOAA4uJGKtDF/uBaooX8 du/F9OEXSmeBxm5F5CSENkQMdw15o+xhLFsyXPgREVlyfmvBzlm9lTK+S74tdmMp5/DG +Q6papvyg8n47cZ9ASh+gD2fvfYHhNRXP/kUtpC3s2ecQmHCgEaiTfVMUe/jRGkpLgwl DUvHDAH/sOBYYkHMo2Y6vAf1lqqdd9hhjz80Kz5DFxG8crdrkYY3WpVV1E9IE6gbR1bb pvOx4X3jAJpwsEPyHrfGHDwmJeoOW+NlPFN/oDOhhiAksS3LvH4N4uJcdLcoWIBgdsfQ AFUw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type :x-system-of-record:x-gm-message-state; bh=CcVQ8pIFwpmz/p4LmCfpsP2fQgCIvd1wkazLrIzS83g=; b=A5o6R9vwiju/EC1grgdc+4Dr2U8Adiw78Yn72jgpkmu7luq+kd5w0jyxA129+EoNCw vDMZKqgDStn/cYbMwO/PoMKtidhMWt3+D5Myb1Nd6vAPb5vC/B5PnM2YZ5VVBmLmYuT4 8iR1p76E0WPqJrDkyFgbfcZJVr/6htYvWVqmAsXhDC7Fe7UMXDmCMCHGjQZAHLs4bH93 WzAW6e/Jy9I8kHMe5B3rW2AxKqLcN46dCvzzRH1CjMUO+XqcTpPk1EZGgEHUIqltsyzH eqQUcOwLnyvnq6LK70bWgcILV8yvFoeU14S5j3NXem3o/jteuZFajvTjcN7pjBLKH5EO k9UA== Received: by 10.180.14.193 with SMTP id r1mr17691492wic.19.1341698591160; Sat, 07 Jul 2012 15:03:11 -0700 (PDT) MIME-Version: 1.0 Received: by 10.180.14.193 with SMTP id r1mr17691477wic.19.1341698591012; Sat, 07 Jul 2012 15:03:11 -0700 (PDT) Received: by 10.216.17.78 with HTTP; Sat, 7 Jul 2012 15:03:10 -0700 (PDT) Date: Sat, 7 Jul 2012 15:03:10 -0700 Message-ID: From: Matt Mathis To: TCP Maintenance and Minor Extensions WG , iccrg@cs.ucl.ac.uk Content-Type: text/plain; charset=ISO-8859-1 X-System-Of-Record: true X-Gm-Message-State: ALoCoQkTml6qa+h2L0yEPogPcT/jTRm5jEGgSXVTy9dKcOZ9ogKG2fE7JHe3YxCAfeeilLSNMoRU+tM7JHTPnCb0kq8tBGjIkjGWeZdxksYs3RwXEmlrZlMQ8pwTfcb5ea3xYP7qdZqUpiXPYAG9pcnzuP9d3hXbPKlM4PDx/nf6rbJ6nt2+WMs= Subject: [tcpm] TCP Laminar patch posted X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Jul 2012 22:02:53 -0000 I posted a "version 0.1" Laminar patch at https://developers.google.com/speed/protocols/tcp-laminar . It is still very much a work in progress, but all of the core functionality is present and seems to be working. There are still a number of pieces missing: non-Reno, non-CUBIC congestion control, and cwnd validation. People who work on Linux might be interested in trying it, or at least looking at the code. Thanks, --MM-- The best way to predict the future is to create it. - Alan Kay From per.hurtig@gmail.com Sun Jul 8 05:24:08 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0BEA421F861E for ; Sun, 8 Jul 2012 05:24:08 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.977 X-Spam-Level: X-Spam-Status: No, score=-2.977 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id SYH4vLGowwg3 for ; Sun, 8 Jul 2012 05:24:07 -0700 (PDT) Received: from mail-ob0-f172.google.com (mail-ob0-f172.google.com [209.85.214.172]) by ietfa.amsl.com (Postfix) with ESMTP id 36B7521F85F4 for ; Sun, 8 Jul 2012 05:24:07 -0700 (PDT) Received: by obbwc20 with SMTP id wc20so20326126obb.31 for ; Sun, 08 Jul 2012 05:24:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=8+opniDwB/jPozLzTID/0YiYk7JbyaU5IvEy/liMUJU=; b=mdSgNt9fKq4EnOAeyZARGxVxhWH9/HGQXpSlfjgLoYcye1xkoaN1bUjofKTk7wNdIW AbwtnAS73FqwrdDKiynspAB69i3FyrXsb92V3AeO5wTMcwM6zaEe1rlDIXGsRwVEjyUw VzNwGVA1egh63+XLymE+omYiJo6tyJj6qN/yR6U0FDdM+o2NZurqyz0RFN9xOkTrcDE4 ixanpdaVQlh42FJF1vOwe5vhTiMTee6edm6t2UsAGM8ExyDHwq3ZmLts00tgxgJOuyE/ dEPxQhkHU+e7hpyzfLoWJnRGoEkW8l7BkEszdWSynBnse8BSVgIQRln9AYLi2neEkHFP E6ww== MIME-Version: 1.0 Received: by 10.50.184.129 with SMTP id eu1mr6330503igc.18.1341750268535; Sun, 08 Jul 2012 05:24:28 -0700 (PDT) Sender: per.hurtig@gmail.com Received: by 10.231.189.91 with HTTP; Sun, 8 Jul 2012 05:24:28 -0700 (PDT) In-Reply-To: References: <20120628104113.8709.30553.idtracker@ietfa.amsl.com> <4FF6AF01.7000601@ifi.uio.no> Date: Sun, 8 Jul 2012 14:24:28 +0200 X-Google-Sender-Auth: 9rz8e7zjgqTX8mUGk3xnxPaeLUs Message-ID: From: Per Hurtig To: Yoshifumi Nishida Content-Type: text/plain; charset=ISO-8859-1 Cc: tcpm@ietf.org, Michael Welzl , Andreas Petlund Subject: Re: [tcpm] New Version Notification for draft-hurtig-tcpm-rtorestart-02.txt X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 08 Jul 2012 12:24:08 -0000 Hi, For a "careful" variant of the algorithm I would agree that it's ok to exclude these scenarios. However, it would still be interesting to have an "aggressive" version of the algorithm that is enabled in burst situations as well. Per On Fri, Jul 6, 2012 at 9:05 PM, Yoshifumi Nishida wrote: > Hello, > > On Fri, Jul 6, 2012 at 2:25 AM, Michael Welzl wrote: >> Hi Yuchung, >> >> First of all, my apologies for answering so slowly, and for forgetting to >> answer your previous similar comment. >> What happened then is that each of us authors assumed that the other would >> take care of it, and in the end, I had forgotten about it. >> >> See my answer in line below: >> >> >> >> On 7/2/12 9:20 PM, Yuchung Cheng wrote: >>> >>> On Sun, Jul 1, 2012 at 11:45 PM, Per Hurtig wrote: >>>> >>>> Hi all, >>>> >>>> we've now updated the modified RTO restart draft to not require >>>> implementations to save the transmission times of every segment. There >>>> are also some changes to the discussion section and various small text >>>> changes. >>>> >>>> >>>> Per >>>> >>>> A new version of I-D, draft-hurtig-tcpm-rtorestart-02.txt >>>> has been successfully submitted by Per Hurtig and posted to the >>>> IETF repository. >>>> >>>> Filename: draft-hurtig-tcpm-rtorestart >>>> Revision: 02 >>>> Title: TCP and SCTP RTO Restart >>>> Creation date: 2012-06-28 >>>> WG ID: Individual Submission >>>> Number of pages: 9 >>>> URL: >>>> http://www.ietf.org/internet-drafts/draft-hurtig-tcpm-rtorestart-02.txt >>>> Status: >>>> http://datatracker.ietf.org/doc/draft-hurtig-tcpm-rtorestart >>>> Htmlized: >>>> http://tools.ietf.org/html/draft-hurtig-tcpm-rtorestart-02 >>>> Diff: >>>> http://tools.ietf.org/rfcdiff?url2=draft-hurtig-tcpm-rtorestart-02 >>>> >>>> Abstract: >>>> This document describes a modified algorithm for managing the TCP and >>>> SCTP retransmission timers that provides faster loss recovery when a >>>> connection's amount of outstanding data is small. The modification >>>> allows the transport to restart its retransmission timer more >>>> aggressively in situations where fast retransmit cannot be used. >>>> This enables faster loss detection and recovery for connections that >>>> are short-lived or application-limited. >>> >>> I asked similar questions for every revision but didn't see the response >>> in >>> the list or the newest draft: >>> >>> I am concerned about this following hypothetical scenario: >>> >>> t0.0s: bursts packets 1-20, mount RTO to 2second >>> t1.8s: gets ack of packets 2,4,6,8,10,12,14,16, 18 every 200ms. >>> Ack of 18 arrives at 1.8s >>> >>> This draft will mount an RTO of 200ms for packet 19 on ack18, which >>> is likely to be spurious. The problem is that T_earliest could be >>> starting from large send a while back even when current FlightSize >>> is small. Moreover, I don't think this is rare so I suspect the >>> performance benefit will be offset by the spurious RTO. >> >> ( I wonder what made you pick 20, not 10 packets for your example? ;-) ) >> >> Seriously: you raise a very valid concern here, a concern which we haven't >> yet investigated (hence: no data). >> However, it seems to me that the situation you describe can really only >> happen when the sender transmits a sudden burst of data without waiting for >> new incoming ACKs. So this could be fixed by disabling the mechanism in Slow >> Start, right? What do you think? > > I think there are some situations where TCP sends burst of packets. > ACK lost/compression, > recovery from packet loss, small pause of application.. Is it ok to > exclude these situations? > > Thanks, > -- > Yoshifumi > From tsabatini@hotmail.com Sun Jul 8 20:54:46 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D256A21F873E for ; Sun, 8 Jul 2012 20:54:46 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.002 X-Spam-Level: X-Spam-Status: No, score=0.002 tagged_above=-999 required=5 tests=[BAYES_50=0.001, HTML_MESSAGE=0.001] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KqIQrmIbVVcv for ; Sun, 8 Jul 2012 20:54:46 -0700 (PDT) Received: from bay0-omc2-s10.bay0.hotmail.com (bay0-omc2-s10.bay0.hotmail.com [65.54.190.85]) by ietfa.amsl.com (Postfix) with ESMTP id 39A6F21F8669 for ; Sun, 8 Jul 2012 20:54:46 -0700 (PDT) Received: from BAY158-W52 ([65.54.190.125]) by bay0-omc2-s10.bay0.hotmail.com with Microsoft SMTPSVC(6.0.3790.4675); Sun, 8 Jul 2012 20:55:09 -0700 Message-ID: Content-Type: multipart/alternative; boundary="_843d852d-c6c4-4172-8d30-a5024183b33b_" X-Originating-IP: [74.64.96.39] From: Anthony Sabatini To: Date: Sun, 8 Jul 2012 23:55:09 -0400 Importance: Normal MIME-Version: 1.0 X-OriginalArrivalTime: 09 Jul 2012 03:55:09.0654 (UTC) FILETIME=[A27A5760:01CD5D86] X-Mailman-Approved-At: Mon, 09 Jul 2012 08:05:55 -0700 Subject: [tcpm] TCP Error recovery and efficiency X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Jul 2012 03:56:46 -0000 --_843d852d-c6c4-4172-8d30-a5024183b33b_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable All - While data communications was constantly improving=2C due mostly to the wid= e adoption of fiber optics=2C the fact that TCP had an inferior recovery me= chanism was moot as it came into play less and less often. Sadly those day= s are now behind us and we face a world increasingly hostile to data transm= ission. The sources of packet loss have grown legion=2C "traffic shapers"= =2C congested links to the subscibers=2C overloaded wifi bands=2C content m= onitor that can not keep up=2C rate caps on mobile phone data usage=2C the = list goes on and on. Complicating this is the increase in data rates and f= ile sizes which allow many more packets and their acknowledgements to be in= flight between two endpoints at once. We as a community must move along th= e proliferation of SACK and if we are going to do that we should fix the un= derlying issues in the protocol first. All current TCP recovery implementations are based on guessing the state at= the remote node. No recovery mechanism currently in place gives sufficien= t weight to in-flight or delayed messages. This problem can be solved easi= ly by transmitting a token related to the transmission state in each out go= ing message which the destination node will return on its next transmission= . By having this token I know all of the messages the far end should have = seen and if I do not have an acknowledgement for any of them=2C I know that= those need to be retransmitted. To start this discussion I have created an I-D "Highly Efficient Selective = Acknowledgement (SACK) for TCP" - draft-sabatini-tcp-sack-00 which I hope w= ill be accepted by the working group. Note that even though no TCP implementation exists=2C the HDLC version I cr= eated for the financial community has 20 years of working history all over = the world and the capability for being 100% efficient right down to one err= or per message no matter what the data rate or the delay (including multipl= e satellite hops). Tony=20 = --_843d852d-c6c4-4172-8d30-a5024183b33b_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
All -

While data communications was constantly improving=2C due most= ly to the wide adoption of fiber optics=2C the fact that TCP had an inferio= r recovery mechanism was moot as it came into play less and less often.&nbs= p=3B Sadly those days are now behind us and we face a world increasingly ho= stile to data transmission. =3B The sources of packet loss have grown l= egion=2C "traffic shapers"=2C congested links to the subscibers=2C overload= ed wifi bands=2C content monitor that can not keep up=2C rate caps on mobil= e phone data usage=2C the list goes on and on. =3B Complicating this is= the increase in data rates and file sizes which allow many more packets an= d their acknowledgements to be inflight between two endpoints at once. = =3B We as a community must move along the proliferation of SACK and if we a= re going to do that we should fix the underlying issues in the protocol fir= st.

All current TCP recovery implementations are based on guessing t= he state at the remote node. =3B No recovery mechanism currently in pla= ce gives sufficient weight to in-flight or delayed messages. =3B This p= roblem can be solved easily by transmitting a token related to the transmis= sion state in each out going message which the destination node will return= on its next transmission. =3B By having this token I know all of the m= essages the far end should have seen and if I do not have an acknowledgemen= t for any of them=2C I know that those need to be retransmitted.

To = start this discussion I have created an I-D "Highly Efficient Selective Ack= nowledgement (SACK) for TCP" - draft-sabatini-tcp-sack-00 which I hope will= be accepted by the working group.

Note that even though no TCP impl= ementation exists=2C the HDLC version I created for the financial community= has 20 years of working history all over the world and the capability for = being 100% efficient right down to one error per message no matter what the= data rate or the delay (including multiple satellite hops).

Tony
= --_843d852d-c6c4-4172-8d30-a5024183b33b_-- From ycheng@google.com Mon Jul 9 12:19:33 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7BA8B11E8144 for ; Mon, 9 Jul 2012 12:19:33 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.677 X-Spam-Level: X-Spam-Status: No, score=-102.677 tagged_above=-999 required=5 tests=[AWL=-0.300, BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, J_CHICKENPOX_33=0.6, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id SfsZ6nmEUHwn for ; Mon, 9 Jul 2012 12:19:32 -0700 (PDT) Received: from mail-ob0-f172.google.com (mail-ob0-f172.google.com [209.85.214.172]) by ietfa.amsl.com (Postfix) with ESMTP id B879111E80F3 for ; Mon, 9 Jul 2012 12:19:32 -0700 (PDT) Received: by obbwc20 with SMTP id wc20so1012232obb.31 for ; Mon, 09 Jul 2012 12:19:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type:x-system-of-record; bh=GjiaZfjtWipzgBnGMkDApJD3rNqgdQABW+UAQS60mpk=; b=pcndb4FrJknsDWDkcjLyGfRPpd1Xpqf4D4zlk4eI0FLKxf2TpYScF5oDF1iL6qVkpM oEQAc1fN+KKxpbtyJEKpRBoQmVZWDit4cUz7twfEmP10RIuk4wvhDRHBV4lBW1xJsl8U niabdNR/9arGLoTTAf2+lQZm/xJKC4LgSLtwUQzlmhOoBHNh7vbGgMmqfH0gKvRVTZk3 tUdVIHNqxqSwqyCNSZpvvnpnQ0euTMnWR75rFpowiU7fhsjsgf7sAV4S/+UkkhAqBDnd 2eArrSGNLH4DJS2n1Hs4e2av3HChsFXB7+PDImfvBhfEpcBk/p/blEf2t1xajuceWXsh QXaQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type:x-system-of-record:x-gm-message-state; bh=GjiaZfjtWipzgBnGMkDApJD3rNqgdQABW+UAQS60mpk=; b=PKtKg8T2ZkbUuxcKuzzlGF0KE3ZbNco1udPKFc5gS8HtKaIlcUMW66eIZuBjcR8fE6 xMRPxJPWGX6CTYGfC9p+GFmGQZRKilyt6gVoZjVmnMUfNtmTsaEE8bObnQBLxDAe5GEr 8x/4kERfnBARBc6Q9p4zgKKu4jbbABjy2AWEOxKl9UDvg1tC3tLrDuv3oZyBMPnixKy2 cdoFGIpdttgvg6xaz/N4gqo13/XM1BNVUekmQwI2Cina8HdWcDF2ftzVwLS5AwIPoanF OqlRx5e/tTrA4pk7P/yWw8tvRxO5QZ8xd3nbBnK5zI0otIwaSvw4wxcFb3PmARirfEyI 6cTg== Received: by 10.182.117.71 with SMTP id kc7mr37196587obb.62.1341861598094; Mon, 09 Jul 2012 12:19:58 -0700 (PDT) Received: by 10.182.117.71 with SMTP id kc7mr37196576obb.62.1341861597938; Mon, 09 Jul 2012 12:19:57 -0700 (PDT) MIME-Version: 1.0 Received: by 10.60.14.168 with HTTP; Mon, 9 Jul 2012 12:19:37 -0700 (PDT) In-Reply-To: References: <20120628104113.8709.30553.idtracker@ietfa.amsl.com> <4FF6AF01.7000601@ifi.uio.no> From: Yuchung Cheng Date: Mon, 9 Jul 2012 12:19:37 -0700 Message-ID: To: Yoshifumi Nishida Content-Type: text/plain; charset=ISO-8859-1 X-System-Of-Record: true X-Gm-Message-State: ALoCoQl/vsWkSHk2+xB/4n0cpu49q6Tnip8OrqTo90FK0Mw4YA4hdCCOUHQvCO6McjRStFmfzVYO/IPKJd65pBtGko2796E26h2HXeivTrD2VKdBlzWXOishq+aB0zKo7M4gtCIYDRw5fmHqLB+73xK5n03yH/tjBgceWSkxsl4erV84QjltbAI= Cc: tcpm@ietf.org, Michael Welzl , Andreas Petlund Subject: Re: [tcpm] New Version Notification for draft-hurtig-tcpm-rtorestart-02.txt X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Jul 2012 19:19:33 -0000 On Fri, Jul 6, 2012 at 12:05 PM, Yoshifumi Nishida wrote: > Hello, > > On Fri, Jul 6, 2012 at 2:25 AM, Michael Welzl wrote: >> Hi Yuchung, >> >> First of all, my apologies for answering so slowly, and for forgetting to >> answer your previous similar comment. >> What happened then is that each of us authors assumed that the other would >> take care of it, and in the end, I had forgotten about it. >> >> See my answer in line below: >> >> >> >> On 7/2/12 9:20 PM, Yuchung Cheng wrote: >>> >>> On Sun, Jul 1, 2012 at 11:45 PM, Per Hurtig wrote: >>>> >>>> Hi all, >>>> >>>> we've now updated the modified RTO restart draft to not require >>>> implementations to save the transmission times of every segment. There >>>> are also some changes to the discussion section and various small text >>>> changes. >>>> >>>> >>>> Per >>>> >>>> A new version of I-D, draft-hurtig-tcpm-rtorestart-02.txt >>>> has been successfully submitted by Per Hurtig and posted to the >>>> IETF repository. >>>> >>>> Filename: draft-hurtig-tcpm-rtorestart >>>> Revision: 02 >>>> Title: TCP and SCTP RTO Restart >>>> Creation date: 2012-06-28 >>>> WG ID: Individual Submission >>>> Number of pages: 9 >>>> URL: >>>> http://www.ietf.org/internet-drafts/draft-hurtig-tcpm-rtorestart-02.txt >>>> Status: >>>> http://datatracker.ietf.org/doc/draft-hurtig-tcpm-rtorestart >>>> Htmlized: >>>> http://tools.ietf.org/html/draft-hurtig-tcpm-rtorestart-02 >>>> Diff: >>>> http://tools.ietf.org/rfcdiff?url2=draft-hurtig-tcpm-rtorestart-02 >>>> >>>> Abstract: >>>> This document describes a modified algorithm for managing the TCP and >>>> SCTP retransmission timers that provides faster loss recovery when a >>>> connection's amount of outstanding data is small. The modification >>>> allows the transport to restart its retransmission timer more >>>> aggressively in situations where fast retransmit cannot be used. >>>> This enables faster loss detection and recovery for connections that >>>> are short-lived or application-limited. >>> >>> I asked similar questions for every revision but didn't see the response >>> in >>> the list or the newest draft: >>> >>> I am concerned about this following hypothetical scenario: >>> >>> t0.0s: bursts packets 1-20, mount RTO to 2second >>> t1.8s: gets ack of packets 2,4,6,8,10,12,14,16, 18 every 200ms. >>> Ack of 18 arrives at 1.8s >>> >>> This draft will mount an RTO of 200ms for packet 19 on ack18, which >>> is likely to be spurious. The problem is that T_earliest could be >>> starting from large send a while back even when current FlightSize >>> is small. Moreover, I don't think this is rare so I suspect the >>> performance benefit will be offset by the spurious RTO. >> >> ( I wonder what made you pick 20, not 10 packets for your example? ;-) ) >> >> Seriously: you raise a very valid concern here, a concern which we haven't >> yet investigated (hence: no data). >> However, it seems to me that the situation you describe can really only >> happen when the sender transmits a sudden burst of data without waiting for >> new incoming ACKs. So this could be fixed by disabling the mechanism in Slow >> Start, right? What do you think? > > I think there are some situations where TCP sends burst of packets. > ACK lost/compression, > recovery from packet loss, small pause of application.. Is it ok to in particular TSO and application pauses (or user-level pacing) make bursts common so I am concerned not to address that. slow start after idle can not prevent pauses within RTO (it happens on YouTube serving already). > exclude these situations? It's probably not OK but not difficult to deal with that either, e.g., sender can simply avoid offsetting the RTO for such large send in the proposal. But in principle isn't RTO meant for SND.UNA. This draft essentially suggests RTO for every packet. It'd be interesting to see some real data benchmarks. Thanks. > > Thanks, > -- > Yoshifumi > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm From mjfox@us.ibm.com Mon Jul 9 14:46:31 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 48DF511E8102 for ; Mon, 9 Jul 2012 14:46:31 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.598 X-Spam-Level: X-Spam-Status: No, score=-10.598 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xL6Eo6xnCN50 for ; Mon, 9 Jul 2012 14:46:30 -0700 (PDT) Received: from e36.co.us.ibm.com (e36.co.us.ibm.com [32.97.110.154]) by ietfa.amsl.com (Postfix) with ESMTP id 9965111E80F2 for ; Mon, 9 Jul 2012 14:46:28 -0700 (PDT) Received: from /spool/local by e36.co.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Mon, 9 Jul 2012 15:46:54 -0600 Received: from d03dlp02.boulder.ibm.com (9.17.202.178) by e36.co.us.ibm.com (192.168.1.136) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Mon, 9 Jul 2012 15:46:52 -0600 Received: from d03relay03.boulder.ibm.com (d03relay03.boulder.ibm.com [9.17.195.228]) by d03dlp02.boulder.ibm.com (Postfix) with ESMTP id 934D03E40054 for ; Mon, 9 Jul 2012 21:46:21 +0000 (WET) Received: from d03av01.boulder.ibm.com (d03av01.boulder.ibm.com [9.17.195.167]) by d03relay03.boulder.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id q69LkLhk287712 for ; Mon, 9 Jul 2012 15:46:21 -0600 Received: from d03av01.boulder.ibm.com (loopback [127.0.0.1]) by d03av01.boulder.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id q69LkLFf001805 for ; Mon, 9 Jul 2012 15:46:21 -0600 Received: from d03nm118.boulder.ibm.com (d03nm118.boulder.ibm.com [9.63.40.219]) by d03av01.boulder.ibm.com (8.14.4/8.13.1/NCO v10.0 AVin) with ESMTP id q69LkLir001802 for ; Mon, 9 Jul 2012 15:46:21 -0600 To: tcpm@ietf.org MIME-Version: 1.0 X-KeepSent: 63CD4C49:784B3B30-85257A36:0077294A; type=4; name=$KeepSent X-Mailer: Lotus Notes Release 8.5.3 September 15, 2011 From: Mike Fox X-MIMETrack: S/MIME Sign by Notes Client on Mike Fox/Raleigh/IBM(Release 8.5.3|September 15, 2011) at 07/09/2012 05:46:17 PM, Serialize by Notes Client on Mike Fox/Raleigh/IBM(Release 8.5.3|September 15, 2011) at 07/09/2012 05:46:17 PM, Serialize complete at 07/09/2012 05:46:17 PM, S/MIME Sign failed at 07/09/2012 05:46:17 PM: The cryptographic key was not found, Serialize by Router on D03NM118/03/M/IBM(Release 8.5.3 ZX853HP5|January 12, 2012) at 07/09/2012 15:46:21, Serialize complete at 07/09/2012 15:46:21 Message-ID: Date: Mon, 9 Jul 2012 17:46:19 -0400 Content-Type: multipart/alternative; boundary="=_alternative 0077983D85257A36_=" X-Content-Scanned: Fidelis XPS MAILER x-cbid: 12070921-7606-0000-0000-000001DD902B Subject: [tcpm] Fw: New Version Notification for draft-fox-tcpm-shared-memory-rdma-00.txt X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Jul 2012 21:46:31 -0000 This is a multipart message in MIME format. --=_alternative 0077983D85257A36_= Content-Type: text/plain; charset="US-ASCII" The submitted internet draft describes a new protocol that exploits RDMA over CEE (RoCE) transparently for TCP sockets based applications. This protocol (Shared Memory Communications over RDMA or SMC-R for short) achieves this transparency by using normal TCP connection establishment flows but allows each end point to dynamically negotiate the use of SMC-R via TCP experimental options as proposed in draft-ietf-tcpm-experimental-options-01.txt. When both end points agree to use the SMC-R protocol, all data flows across the two TCP endpoints take place using out of band RDMA Write operations to pre-defined memory regions allocated by the two peers. The goal of the SMC-R protocol is to achieve this transparently for TCP socket applications which means supporting the same socket API semantics and simulating the behavior of the TCP protocol as observed by the application layer. As such, SMC-R includes provisions for handling TCP specific features such as urgent data. The hybrid nature of the SMC-R protocol allows it to maintain many of the qualities of service associated with TCP/IP communications while providing the performance benefits of leveraging RDMA technology for moving data on Ethernet networks capable of RDMA. Any review and comments on this draft would be very much appreciated. Thank you, Mike Fox > A new version of I-D, draft-fox-tcpm-shared-memory-rdma-00.txt > has been successfully submitted by Mike Fox and posted to the > IETF repository. > > Filename: draft-fox-tcpm-shared-memory-rdma > Revision: 00 > Title: Shared Memory Communications over RDMA > Creation date: 2012-07-09 > WG ID: Individual Submission > Number of pages: 133 > URL: http://www.ietf.org/internet-drafts/draft-fox-tcpm-shared-memory-rdma-00.txt > Status: http://datatracker.ietf.org/doc/draft-fox-tcpm-shared-memory-rdma > Htmlized: http://tools.ietf.org/html/draft-fox-tcpm-shared-memory-rdma-00 > > > Abstract: > This document describes the Shared Memory Communications over RDMA > (SMC-R) protocol. This protocol provides RDMA communications to TCP > endpoints in a manner that is transparent to socket applications. It > further provides for dynamic discovery of partner RDMA capabilities > and dynamic setup of RDMA connections, transparent high availability > and load balancing when redundant RDMA network paths are available, > and it maintains many of the traditional TCP/IP qualities of service > such as filtering that enterprise users demand, as well as TCP socket > semantics such as urgent data. > > > > > The IETF Secretariat --=_alternative 0077983D85257A36_= Content-Type: text/html; charset="US-ASCII"
The submitted internet draft describes a new protocol that exploits RDMA over CEE (RoCE) transparently for TCP sockets based applications. This protocol (Shared Memory Communications over RDMA or SMC-R for short) achieves this transparency by using normal TCP connection establishment flows but allows each end point to dynamically negotiate the use of SMC-R via TCP experimental options as proposed in draft-ietf-tcpm-experimental-options-01.txt. When both end points agree to use the SMC-R protocol, all data flows across the two TCP endpoints take place using out of band RDMA Write operations to pre-defined memory regions allocated by the two peers. The goal of the SMC-R protocol is to achieve this transparently for TCP socket applications which means supporting the same socket API semantics and simulating the behavior of the TCP protocol as observed by the application layer. As such, SMC-R includes provisions for handling TCP specific features such as urgent data. The hybrid nature of the SMC-R protocol allows it to maintain many of the qualities of service associated with TCP/IP communications while providing the performance benefits of leveraging RDMA technology for moving data on Ethernet networks capable of RDMA.

Any review and comments on this draft would be very much appreciated.  

Thank you,
Mike Fox




> A new version of I-D, draft-fox-tcpm-shared-memory-rdma-00.txt
> has been successfully submitted by Mike Fox and posted to the
> IETF repository.
>
> Filename:                  draft-fox-tcpm-shared-memory-rdma
> Revision:                  00
> Title:                                   Shared Memory Communications over RDMA
> Creation date:                  2012-07-09
> WG ID:                                   Individual Submission
> Number of pages: 133
> URL:             http://www.ietf.org/internet-drafts/draft-fox-tcpm-shared-memory-rdma-00.txt
> Status:          http://datatracker.ietf.org/doc/draft-fox-tcpm-shared-memory-rdma
> Htmlized:        http://tools.ietf.org/html/draft-fox-tcpm-shared-memory-rdma-00
>
>
> Abstract:
>   This document describes the Shared Memory Communications over RDMA
>   (SMC-R) protocol.  This protocol provides RDMA communications to TCP
>   endpoints in a manner that is transparent to socket applications.  It
>   further provides for dynamic discovery of partner RDMA capabilities
>   and dynamic setup of RDMA connections, transparent high availability
>   and load balancing when redundant RDMA network paths are available,
>   and it maintains many of the traditional TCP/IP qualities of service
>   such as filtering that enterprise users demand, as well as TCP socket
>   semantics such as urgent data.
>
>                                                                                  
>
>
> The IETF Secretariat

 --=_alternative 0077983D85257A36_=-- From mallman@icir.org Tue Jul 10 08:13:27 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 824FB21F8555 for ; Tue, 10 Jul 2012 08:13:27 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.599 X-Spam-Level: X-Spam-Status: No, score=-106.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id yEjCqzAAtePq for ; Tue, 10 Jul 2012 08:13:27 -0700 (PDT) Received: from fruitcake.ICSI.Berkeley.EDU (fruitcake.ICSI.Berkeley.EDU [192.150.186.11]) by ietfa.amsl.com (Postfix) with ESMTP id 0699421F8530 for ; Tue, 10 Jul 2012 08:13:27 -0700 (PDT) Received: from lawyers.icir.org (envoy.icir.org [192.150.187.30]) by fruitcake.ICSI.Berkeley.EDU (8.12.11.20060614/8.12.11) with ESMTP id q6AFDqb6026453; Tue, 10 Jul 2012 08:13:53 -0700 (PDT) Received: from lawyers.icir.org (localhost [127.0.0.1]) by lawyers.icir.org (Postfix) with ESMTP id A743D22F931F; Tue, 10 Jul 2012 11:13:52 -0400 (EDT) To: Yuchung Cheng From: Mark Allman In-Reply-To: Organization: International Computer Science Institute (ICSI) Song-of-the-Day: Dead Flowers X-URL-0: http://www.icir.org/mallman-files/Document28019.jpg X-URL-1: http://www.icir.org/mallman-files/Document12543.html X-URL-2: http://www.icir.org/mallman-files/Document69981.pdf MIME-Version: 1.0 Content-Type: multipart/signed; boundary="--------ma18096-1"; micalg=pgp-sha1; protocol="application/pgp-signature" Date: Tue, 10 Jul 2012 11:13:52 -0400 Sender: mallman@icir.org Message-Id: <20120710151352.A743D22F931F@lawyers.icir.org> Cc: tcpm Subject: Re: [tcpm] New Version Notification for draft-hurtig-tcpm-rtorestart-02.txt X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list Reply-To: mallman@icir.org List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Jul 2012 15:13:27 -0000 ----------ma18096-1 Content-Type: text/plain Content-Disposition: inline > I am concerned about this following hypothetical scenario: > > t0.0s: bursts packets 1-20, mount RTO to 2second > t1.8s: gets ack of packets 2,4,6,8,10,12,14,16, 18 every 200ms. > Ack of 18 arrives at 1.8s > > This draft will mount an RTO of 200ms for packet 19 on ack18, which > is likely to be spurious. The problem is that T_earliest could be > starting from large send a while back even when current FlightSize > is small. I hear you. But, I don't buy your framing. You keep the RTO at a constant throughout the process. But, with a variance like you describe above it certainly seems plausible that the RTO will balloon. And, the way the document is framed when 18 arrives we restart the timer with RTO-T_earliest. But, if RTO is > 2sec then the timeout will be > 200ms. So, I don't think it is as easy as you have portrayed it. allman ----------ma18096-1 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (Darwin) iEYEARECAAYFAk/8RrAACgkQWyrrWs4yIs5nYgCfWpIw7rqTordotsf2oOI6Pqk+ N8gAn3AvqHHW2oqYkjXd7dafNxZvDKdv =5WnL -----END PGP SIGNATURE----- ----------ma18096-1-- From rs@netapp.com Wed Jul 11 04:39:16 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 398CF21F85F4 for ; Wed, 11 Jul 2012 04:39:16 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.598 X-Spam-Level: X-Spam-Status: No, score=-10.598 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4A0BQ2wIMOMz for ; Wed, 11 Jul 2012 04:39:12 -0700 (PDT) Received: from mx2.netapp.com (mx2.netapp.com [216.240.18.37]) by ietfa.amsl.com (Postfix) with ESMTP id 7EB8621F85E0 for ; Wed, 11 Jul 2012 04:39:12 -0700 (PDT) X-IronPort-AV: E=Sophos;i="4.77,567,1336374000"; d="scan'208,217";a="661720444" Received: from smtp2.corp.netapp.com ([10.57.159.114]) by mx2-out.netapp.com with ESMTP; 11 Jul 2012 04:39:42 -0700 Received: from vmwexceht01-prd.hq.netapp.com (vmwexceht01-prd.hq.netapp.com [10.106.76.239]) by smtp2.corp.netapp.com (8.13.1/8.13.1/NTAP-1.6) with ESMTP id q6BBdfAL000630; Wed, 11 Jul 2012 04:39:42 -0700 (PDT) Received: from SACEXCMBX02-PRD.hq.netapp.com ([169.254.1.66]) by vmwexceht01-prd.hq.netapp.com ([10.106.76.239]) with mapi id 14.02.0298.004; Wed, 11 Jul 2012 04:39:38 -0700 From: "Scheffenegger, Richard" To: Anthony Sabatini , "tcpm@ietf.org" , "draft-sack@tsabatini.com" Thread-Topic: [tcpm] TCP Error recovery and efficiency Thread-Index: AQHNXeR5wlqJnmRWo0WikYEea35jtJcj7dlQ Date: Wed, 11 Jul 2012 11:39:38 +0000 Message-ID: <012C3117EDDB3C4781FD802A8C27DD4F080D40D3@SACEXCMBX02-PRD.hq.netapp.com> References: In-Reply-To: Accept-Language: de-AT, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.106.53.53] Content-Type: multipart/alternative; boundary="_000_012C3117EDDB3C4781FD802A8C27DD4F080D40D3SACEXCMBX02PRDh_" MIME-Version: 1.0 Subject: Re: [tcpm] TCP Error recovery and efficiency X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Jul 2012 11:39:16 -0000 --_000_012C3117EDDB3C4781FD802A8C27DD4F080D40D3SACEXCMBX02PRDh_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi Anthony, first off, a quick observation: Modifying existing options, particularly th= eir length or encoding, as you propose in that draft, will probably not wor= k; there are many devices, which may react undefined (ignore the extended S= ACK permitted, accept it in the old way,...) which wouldn't be compatible. Also, certain middleboxes do modify the contents of regular SACK blocks (re= writing TCP sequence numbers), and they would definitely interact very badl= y when a well-defined tcp option suddenly has different semantics. Second, I try to understand your language around the token; I'm not sure I = follow the reasoning around artificially delaying ACK / SACK updates on the= receiver. A worked out example as appendix would do good (showing which fields contai= n what when, and the exchange of these tokens between sender and receiver). Third, the draft appears to propose sending multiple ACKs in response to a = (receiver delay timeout), with varying SACK option contents between these A= CKs (all having the same TCP header fields)? The purpose of these ACKs bein= g to clock out the missing segments from the sender, if I read this correct= ly? However, no precaution is there to prevent a burst or train of ACKs (wh= ich would cause a transmit spike on the sender side), correct? I'm not quite clear on the sender behavior; the text seems to imply that th= e SACK scoreboard should be deleted for each received ACK/SACK, and reconst= ructed anew? The new "token" fields serving as secondary ACK counter to only resend thos= e bytes that were not retransmitted by the time the receiver got that token= ? Reconstructing the order in which packets arrive at the destination is, IMH= O, a valuable goal to optimize the error recovery of TCP SACK. So, to summarize, there are two main ideas in this draft: a) Use of a deterministically reflected token, that serves to reconstr= uct the exact order of packets received at the receiver (including retransm= issions). b) Sending of unsolicidated ACKs based on a short timeout (1/4 RTT) in= order to pull lost/delayed data from the sender without reverting to a sen= der-side RTO c) Modification of a well-known TCP option with new data fields Please confirm that I have properly summarized your ideas in that draft. I'm very much in favor of idea a). However, I still believe that the proper= approach would be to synergistically use Timestamp together with SACK to a= chieve the same (which requires "only" a semantic/behavior change of the TS= field in the presence of the SACK option, but doesn't alter the encoding/i= nterpretation per se). With that modification to the Timestamp semantics, t= he timestamps can serve a "tokens" delivering the information you seek to b= etter reconstruct the receiver state at the sender. Please have a look at h= ttp://tools.ietf.org/id/draft-scheffenegger-tcpm-timestamp-negotiation-03.t= xt. I'm not so sure about idea b) since that would appear to violate the conser= vation of packets rule - even under optimal conditions (no packet loss, onl= y very variable packet delay/reordering), a higher rate of ACKs would be se= nt back, potentially completely unnecessary. With c) - this will break this proposal. Please use experimental options i= n the draft (and newly assigned option numbers if adopted) so to avoid comp= atibility issues with existing deployed hardware in the internet. Will you be present in Vancouver? Best regards, Richard Scheffenegger From: tcpm-bounces@ietf.org [mailto:tcpm-bounces@ietf.org] On Behalf Of Ant= hony Sabatini Sent: Montag, 09. Juli 2012 05:55 To: tcpm@ietf.org Subject: [tcpm] TCP Error recovery and efficiency All - While data communications was constantly improving, due mostly to the wide = adoption of fiber optics, the fact that TCP had an inferior recovery mechan= ism was moot as it came into play less and less often. Sadly those days ar= e now behind us and we face a world increasingly hostile to data transmissi= on. The sources of packet loss have grown legion, "traffic shapers", conge= sted links to the subscibers, overloaded wifi bands, content monitor that c= an not keep up, rate caps on mobile phone data usage, the list goes on and = on. Complicating this is the increase in data rates and file sizes which a= llow many more packets and their acknowledgements to be inflight between tw= o endpoints at once. We as a community must move along the proliferation o= f SACK and if we are going to do that we should fix the underlying issues i= n the protocol first. All current TCP recovery implementations are based on guessing the state at= the remote node. No recovery mechanism currently in place gives sufficien= t weight to in-flight or delayed messages. This problem can be solved easi= ly by transmitting a token related to the transmission state in each out go= ing message which the destination node will return on its next transmission= . By having this token I know all of the messages the far end should have = seen and if I do not have an acknowledgement for any of them, I know that t= hose need to be retransmitted. To start this discussion I have created an I-D "Highly Efficient Selective = Acknowledgement (SACK) for TCP" - draft-sabatini-tcp-sack-00 which I hope w= ill be accepted by the working group. Note that even though no TCP implementation exists, the HDLC version I crea= ted for the financial community has 20 years of working history all over th= e world and the capability for being 100% efficient right down to one error= per message no matter what the data rate or the delay (including multiple = satellite hops). Tony --_000_012C3117EDDB3C4781FD802A8C27DD4F080D40D3SACEXCMBX02PRDh_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

 <= /p>

Hi Anthony= ,

 = ;

first off,= a quick observation: Modifying existing options, particularly their length= or encoding, as you propose in that draft, will probably not work; there are many devices, which may react undefined (ignore the ex= tended SACK permitted, accept it in the old way,…) which wouldn’= ;t be compatible.

 = ;

Also, cert= ain middleboxes do modify the contents of regular SACK blocks (rewriting TC= P sequence numbers), and they would definitely interact very badly when a well-defined tcp option suddenly has different semantics.

 = ;

 = ;

Second, I = try to understand your language around the token; I’m not sure I foll= ow the reasoning around artificially delaying ACK / SACK updates on the receiver.

 = ;

A worked o= ut example as appendix would do good (showing which fields contain what whe= n, and the exchange of these tokens between sender and receiver).

 = ;

Third, the= draft appears to propose sending multiple ACKs in response to a (receiver = delay timeout), with varying SACK option contents between these ACKs (all having the same TCP header fields)? The purpose of these A= CKs being to clock out the missing segments from the sender, if I read this= correctly? However, no precaution is there to prevent a burst or train of = ACKs (which would cause a transmit spike on the sender side), correct?

 = ;

I’m = not quite clear on the sender behavior; the text seems to imply that the SA= CK scoreboard should be deleted for each received ACK/SACK, and reconstructed anew?

 = ;

The new &#= 8220;token” fields serving as secondary ACK counter to only resend th= ose bytes that were not retransmitted by the time the receiver got that token?

 = ;

 = ;

 = ;

 = ;

Reconstruc= ting the order in which packets arrive at the destination is, IMHO, a valua= ble goal to optimize the error recovery of TCP SACK.

 = ;

 = ;

So, to sum= marize, there are two main ideas in this draft:

 = ;

a)      Us= e of a deterministically reflected token, that serves to reconstruct the ex= act order of packets received at the receiver (including retransmissions).

 

b)      Se= nding of unsolicidated ACKs based on a short timeout (1/4 RTT) in order to = pull lost/delayed data from the sender without reverting to a sender-side RTO

 = ;

c)       Mo= dification of a well-known TCP option with new data fields

 

 = ;

Please con= firm that I have properly summarized your ideas in that draft.

 = ;

I’m = very much in favor of idea a). However, I still believe that the proper app= roach would be to synergistically use Timestamp together with SACK to achieve the same (which requires “only” a semantic/behavior= change of the TS field in the presence of the SACK option, but doesn’= ;t alter the encoding/interpretation per se). With that modification to the= Timestamp semantics, the timestamps can serve a “tokens” delivering the information you seek to better reconstruct the receiver sta= te at the sender. Please have a look at http://tools.ietf.org/id/draft-sche= ffenegger-tcpm-timestamp-negotiation-03.txt.

 = ;

I’m = not so sure about idea b) since that would appear to violate the conservati= on of packets rule – even under optimal conditions (no packet loss, only very variable packet delay/reordering), a higher rate of ACKs would b= e sent back, potentially completely unnecessary.

 = ;

With c)&nb= sp; - this will break this proposal. Please use experimental options in the= draft (and newly assigned option numbers if adopted) so to avoid compatibility issues with existing deployed hardware in the internet.=

 = ;

Will you b= e present in Vancouver?

 = ;

Best regar= ds,

 = ;

Richard Scheffenegger


 <= /p>

From: tcpm-bounces@ietf.org [mailto:tcpm-bounces@ietf.org] On Behalf Of Anthony Sabatini
Sent: Montag, 09. Juli 2012 05:55
To: tcpm@ietf.org
Subject: [tcpm] TCP Error recovery and efficiency<= /p>

 

All -

While data communications was constantly improving, due mostly to the wide = adoption of fiber optics, the fact that TCP had an inferior recovery mechan= ism was moot as it came into play less and less often.  Sadly those da= ys are now behind us and we face a world increasingly hostile to data transmission.  The sources of packet los= s have grown legion, "traffic shapers", congested links to the su= bscibers, overloaded wifi bands, content monitor that can not keep up, rate= caps on mobile phone data usage, the list goes on and on.  Complicating this is the increase in data rates and file = sizes which allow many more packets and their acknowledgements to be inflig= ht between two endpoints at once.  We as a community must move along t= he proliferation of SACK and if we are going to do that we should fix the underlying issues in the protocol first.

All current TCP recovery implementations are based on guessing the state at= the remote node.  No recovery mechanism currently in place gives suff= icient weight to in-flight or delayed messages.  This problem can be s= olved easily by transmitting a token related to the transmission state in each out going message which the destination = node will return on its next transmission.  By having this token I kno= w all of the messages the far end should have seen and if I do not have an = acknowledgement for any of them, I know that those need to be retransmitted.

To start this discussion I have created an I-D "Highly Efficient Selec= tive Acknowledgement (SACK) for TCP" - draft-sabatini-tcp-sack-00 whic= h I hope will be accepted by the working group.

Note that even though no TCP implementation exists, the HDLC version I crea= ted for the financial community has 20 years of working history all over th= e world and the capability for being 100% efficient right down to one error= per message no matter what the data rate or the delay (including multiple satellite hops).

Tony

--_000_012C3117EDDB3C4781FD802A8C27DD4F080D40D3SACEXCMBX02PRDh_-- From internet-drafts@ietf.org Wed Jul 11 06:50:36 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3D56011E80C7; Wed, 11 Jul 2012 06:50:36 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.599 X-Spam-Level: X-Spam-Status: No, score=-102.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 90pWsJk8RHxA; Wed, 11 Jul 2012 06:50:35 -0700 (PDT) Received: from ietfa.amsl.com (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id BF1C911E8086; Wed, 11 Jul 2012 06:50:35 -0700 (PDT) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: internet-drafts@ietf.org To: i-d-announce@ietf.org X-Test-IDTracker: no X-IETF-IDTracker: 4.30p3 Message-ID: <20120711135035.11985.1052.idtracker@ietfa.amsl.com> Date: Wed, 11 Jul 2012 06:50:35 -0700 Cc: tcpm@ietf.org Subject: [tcpm] I-D Action: draft-ietf-tcpm-1323bis-03.txt X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Jul 2012 13:50:36 -0000 A New Internet-Draft is available from the on-line Internet-Drafts director= ies. This draft is a work item of the TCP Maintenance and Minor Extensions Work= ing Group of the IETF. Title : TCP Extensions for High Performance Author(s) : David Borman Bob Braden Van Jacobson Richard Scheffenegger Filename : draft-ietf-tcpm-1323bis-03.txt Pages : 46 Date : 2012-07-11 Abstract: This memo presents a set of TCP extensions to improve performance over large bandwidth*delay product paths and to provide reliable operation over very high-speed paths. It defines TCP options for scaled windows and timestamps, which are designed to provide compatible interworking with TCP's that do not implement the extensions. The timestamps are used for two distinct mechanisms: RTTM (Round Trip Time Measurement) and PAWS (Protection Against Wrapped Sequences). Selective acknowledgments are not included in this memo. This memo updates and obsoletes RFC 1323. The IETF datatracker status page for this draft is: https://datatracker.ietf.org/doc/draft-ietf-tcpm-1323bis There's also a htmlized version available at: http://tools.ietf.org/html/draft-ietf-tcpm-1323bis-03 A diff from previous version is available at: http://tools.ietf.org/rfcdiff?url2=3Ddraft-ietf-tcpm-1323bis-03 Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ From rs@netapp.com Wed Jul 11 06:56:32 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8781B21F85D2 for ; Wed, 11 Jul 2012 06:56:32 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.599 X-Spam-Level: X-Spam-Status: No, score=-10.599 tagged_above=-999 required=5 tests=[AWL=0.000, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jfRuXSN7nwSC for ; Wed, 11 Jul 2012 06:56:32 -0700 (PDT) Received: from mx2.netapp.com (mx2.netapp.com [216.240.18.37]) by ietfa.amsl.com (Postfix) with ESMTP id 07E1221F8569 for ; Wed, 11 Jul 2012 06:56:32 -0700 (PDT) X-IronPort-AV: E=Sophos;i="4.77,567,1336374000"; d="scan'208";a="661778130" Received: from smtp2.corp.netapp.com ([10.57.159.114]) by mx2-out.netapp.com with ESMTP; 11 Jul 2012 06:56:47 -0700 Received: from vmwexceht03-prd.hq.netapp.com (vmwexceht03-prd.hq.netapp.com [10.106.76.241]) by smtp2.corp.netapp.com (8.13.1/8.13.1/NTAP-1.6) with ESMTP id q6BDulqB011670 for ; Wed, 11 Jul 2012 06:56:47 -0700 (PDT) Received: from SACEXCMBX02-PRD.hq.netapp.com ([169.254.1.66]) by vmwexceht03-prd.hq.netapp.com ([10.106.76.241]) with mapi id 14.02.0298.004; Wed, 11 Jul 2012 06:56:47 -0700 From: "Scheffenegger, Richard" To: "tcpm (tcpm@ietf.org)" Thread-Topic: New Version Notification for draft-ietf-tcpm-1323bis-03.txt Thread-Index: AQHNX2xKTLxjsqlHr0GOstYXWTN9K5ckGcfg Date: Wed, 11 Jul 2012 13:56:46 +0000 Message-ID: <012C3117EDDB3C4781FD802A8C27DD4F080D4374@SACEXCMBX02-PRD.hq.netapp.com> References: <20120711135035.11985.13199.idtracker@ietfa.amsl.com> In-Reply-To: <20120711135035.11985.13199.idtracker@ietfa.amsl.com> Accept-Language: de-AT, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.106.53.53] Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 MIME-Version: 1.0 Subject: [tcpm] New Version Notification for draft-ietf-tcpm-1323bis-03.txt X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Jul 2012 13:56:32 -0000 DQpUaGFua3MgdG8gQWxleCBaaW1tZXJtYW5uLCBhbmQgQWxmcmVkIEhvZW5lcywgdG8gcG9pbnQg b3V0IGEgY29uc2lkZXJhYmxlIG51bWJlciBvZiBOaXRzIGFuZCBhIG51bWJlciBjb250ZW50IGVy cm9ycy9sYXBzZXMuDQoNClBsZWFzZSByZXZpZXcgdGhpcyB2ZXJzaW9uIG9mIHRoZSBkcmFmdCAo Y2hhbmdlcyBmcm9tIHByZXZpb3VzIHZlcnNpb25zIGNhbiBiZSBmb3VuZCBpbiBBcHBlbmRpeCBC KSwgaWYgdGhpcyBkb2N1bWVudCBjYW4gbW92ZSB0byB0aGUgbmV4dCBwaGFzZSBzb29uLg0KDQpC ZXN0IHJlZ2FyZHMsDQoNCg0KUmljaGFyZCBTY2hlZmZlbmVnZ2VyDQoNCg0KLS0tLS1PcmlnaW5h bCBNZXNzYWdlLS0tLS0NCkZyb206IGludGVybmV0LWRyYWZ0c0BpZXRmLm9yZyBbbWFpbHRvOmlu dGVybmV0LWRyYWZ0c0BpZXRmLm9yZ10gDQpTZW50OiBNaXR0d29jaCwgMTEuIEp1bGkgMjAxMiAx NTo1MQ0KVG86IFNjaGVmZmVuZWdnZXIsIFJpY2hhcmQNCkNjOiBicmFkZW5AaXNpLmVkdTsgZGF2 aWQuYm9ybWFuQHF1YW50dW0uY29tOyB2YW5AcGFja2V0ZGVzaWduLmNvbQ0KU3ViamVjdDogTmV3 IFZlcnNpb24gTm90aWZpY2F0aW9uIGZvciBkcmFmdC1pZXRmLXRjcG0tMTMyM2Jpcy0wMy50eHQN Cg0KDQpBIG5ldyB2ZXJzaW9uIG9mIEktRCwgZHJhZnQtaWV0Zi10Y3BtLTEzMjNiaXMtMDMudHh0 IGhhcyBiZWVuIHN1Y2Nlc3NmdWxseSBzdWJtaXR0ZWQgYnkgUmljaGFyZCBTY2hlZmZlbmVnZ2Vy IGFuZCBwb3N0ZWQgdG8gdGhlIElFVEYgcmVwb3NpdG9yeS4NCg0KRmlsZW5hbWU6CSBkcmFmdC1p ZXRmLXRjcG0tMTMyM2Jpcw0KUmV2aXNpb246CSAwMw0KVGl0bGU6CQkgVENQIEV4dGVuc2lvbnMg Zm9yIEhpZ2ggUGVyZm9ybWFuY2UNCkNyZWF0aW9uIGRhdGU6CSAyMDEyLTA3LTExDQpXRyBJRDoJ CSB0Y3BtDQpOdW1iZXIgb2YgcGFnZXM6IDQ2DQpVUkw6ICAgICAgICAgICAgIGh0dHA6Ly93d3cu aWV0Zi5vcmcvaW50ZXJuZXQtZHJhZnRzL2RyYWZ0LWlldGYtdGNwbS0xMzIzYmlzLTAzLnR4dA0K U3RhdHVzOiAgICAgICAgICBodHRwOi8vZGF0YXRyYWNrZXIuaWV0Zi5vcmcvZG9jL2RyYWZ0LWll dGYtdGNwbS0xMzIzYmlzDQpIdG1saXplZDogICAgICAgIGh0dHA6Ly90b29scy5pZXRmLm9yZy9o dG1sL2RyYWZ0LWlldGYtdGNwbS0xMzIzYmlzLTAzDQpEaWZmOiAgICAgICAgICAgIGh0dHA6Ly90 b29scy5pZXRmLm9yZy9yZmNkaWZmP3VybDI9ZHJhZnQtaWV0Zi10Y3BtLTEzMjNiaXMtMDMNCg0K QWJzdHJhY3Q6DQogICBUaGlzIG1lbW8gcHJlc2VudHMgYSBzZXQgb2YgVENQIGV4dGVuc2lvbnMg dG8gaW1wcm92ZSBwZXJmb3JtYW5jZQ0KICAgb3ZlciBsYXJnZSBiYW5kd2lkdGgqZGVsYXkgcHJv ZHVjdCBwYXRocyBhbmQgdG8gcHJvdmlkZSByZWxpYWJsZQ0KICAgb3BlcmF0aW9uIG92ZXIgdmVy eSBoaWdoLXNwZWVkIHBhdGhzLiAgSXQgZGVmaW5lcyBUQ1Agb3B0aW9ucyBmb3INCiAgIHNjYWxl ZCB3aW5kb3dzIGFuZCB0aW1lc3RhbXBzLCB3aGljaCBhcmUgZGVzaWduZWQgdG8gcHJvdmlkZQ0K ICAgY29tcGF0aWJsZSBpbnRlcndvcmtpbmcgd2l0aCBUQ1AncyB0aGF0IGRvIG5vdCBpbXBsZW1l bnQgdGhlDQogICBleHRlbnNpb25zLiAgVGhlIHRpbWVzdGFtcHMgYXJlIHVzZWQgZm9yIHR3byBk aXN0aW5jdCBtZWNoYW5pc21zOg0KICAgUlRUTSAoUm91bmQgVHJpcCBUaW1lIE1lYXN1cmVtZW50 KSBhbmQgUEFXUyAoUHJvdGVjdGlvbiBBZ2FpbnN0DQogICBXcmFwcGVkIFNlcXVlbmNlcykuICBT ZWxlY3RpdmUgYWNrbm93bGVkZ21lbnRzIGFyZSBub3QgaW5jbHVkZWQgaW4NCiAgIHRoaXMgbWVt by4NCg0KICAgVGhpcyBtZW1vIHVwZGF0ZXMgYW5kIG9ic29sZXRlcyBSRkMgMTMyMy4NCg0KICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgIA0KDQoNClRoZSBJRVRGIFNlY3JldGFyaWF0DQo= From pasi.sarolahti@iki.fi Thu Jul 12 02:47:49 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id F19DC21F876C for ; Thu, 12 Jul 2012 02:47:49 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -101.669 X-Spam-Level: X-Spam-Status: No, score=-101.669 tagged_above=-999 required=5 tests=[AWL=0.930, BAYES_00=-2.599, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ohoguCjLvnl3 for ; Thu, 12 Jul 2012 02:47:49 -0700 (PDT) Received: from kirsi1.inet.fi (mta-out.inet.fi [195.156.147.13]) by ietfa.amsl.com (Postfix) with ESMTP id A95D721F847F for ; Thu, 12 Jul 2012 02:47:48 -0700 (PDT) Received: from [192.168.1.65] (80.223.92.46) by kirsi1.inet.fi (8.5.140.03) (authenticated as saropa-1) id 4FD08A2301FEC0F9 for tcpm@ietf.org; Thu, 12 Jul 2012 12:48:20 +0300 From: Pasi Sarolahti Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Date: Thu, 12 Jul 2012 12:49:23 +0300 Message-Id: To: tcpm@ietf.org Mime-Version: 1.0 (Apple Message framework v1278) X-Mailer: Apple Mail (2.1278) Subject: [tcpm] Draft TCPM agenda for IETF-84 X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Jul 2012 09:47:50 -0000 Hi, Here is the current draft agenda for the TCPM meeting in IETF-84. There = is still a bit of room left, so if you want to give a talk, let us = chairs know about the title, speaker and time request ASAP. Thanks! - Pasi TCPM meeting, IETF-84, Vancouver, BC, Canada Monday, July 30, 09:00 - 11:30 -------------------------------------------- * WG Status Chairs Time: 5 min WG items -------- * Shared Use of Experimental TCP Options draft-ietf-tcpm-experimental-options Speaker: chairs proxying Joe Touch Time: 5 minutes * A TCP Authentication Option NAT Extension draft-touch-tcp-ao-nat Speaker: chairs proxying Joe Touch Time: 5 minutes * RFC 1323bis draft-ietf-tcpm-1323bis Speaker: Richard Scheffenegger Time: 15 mins Other items ----------- * TCP and SCTP RTO Restart draft-hurtig-tcpm-rtorestart Speaker: Michael Welzl Time: 10 minutes * HOST_ID TCP Options: Implementation & Preliminary Test Results draft-abdo-hostid-tcpopt-implementation Speaker: Jaqueline Queiroz Time: 10 minutes * TCP loss probe (TLP): a mechanism to reduce retransmission timeouts draft TBA later Speaker: Nandita Dukkipati Time: 10 minutes * Forward Error Correction (FEC) for TCP No draft yet, just to discuss the initial idea Speaker: Nandita Dukkipati Time: 10 minutes * Evaluating TCP Laminar draft-mathis-tcpm-tcp-laminar-00 Speaker: Matt Mathis Time: 15 minutes * Impact of IW10 on Interactive Real-Time Communication Ilpo J=E4rvinen Time: 15 minutes * Highly Efficient Selective Acknowledgement (SACK) for TCP draft-sabatini-tcp-sack-00 Anthony Sabatini Time: 10 minutes From rs@netapp.com Thu Jul 12 06:29:58 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1CFE021F8845 for ; Thu, 12 Jul 2012 06:29:58 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -9.9 X-Spam-Level: X-Spam-Status: No, score=-9.9 tagged_above=-999 required=5 tests=[AWL=-0.698, BAYES_00=-2.599, HTML_MESSAGE=0.001, MIME_QP_LONG_LINE=1.396, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id EeFG86JxhRdx for ; Thu, 12 Jul 2012 06:29:51 -0700 (PDT) Received: from mx2.netapp.com (mx2.netapp.com [216.240.18.37]) by ietfa.amsl.com (Postfix) with ESMTP id E691021F8629 for ; Thu, 12 Jul 2012 06:29:50 -0700 (PDT) X-IronPort-AV: E=Sophos;i="4.77,573,1336374000"; d="scan'208,217";a="662076475" Received: from smtp2.corp.netapp.com ([10.57.159.114]) by mx2-out.netapp.com with ESMTP; 12 Jul 2012 06:30:24 -0700 Received: from vmwexceht03-prd.hq.netapp.com (vmwexceht03-prd.hq.netapp.com [10.106.76.241]) by smtp2.corp.netapp.com (8.13.1/8.13.1/NTAP-1.6) with ESMTP id q6CDHgE2014079; Thu, 12 Jul 2012 06:29:20 -0700 (PDT) Received: from SACEXCMBX02-PRD.hq.netapp.com ([169.254.1.34]) by vmwexceht03-prd.hq.netapp.com ([10.106.76.241]) with mapi id 14.02.0298.004; Thu, 12 Jul 2012 06:18:24 -0700 From: "Scheffenegger, Richard" To: Anthony Sabatini , tcpm , "draft-sack@tsabatini.com" Thread-Topic: [tcpm] TCP Error recovery and efficiency (updated reply) Thread-Index: AQHNX4T2CV3SVsfrukKVPogI7DItr5clnWww Date: Thu, 12 Jul 2012 13:18:23 +0000 Message-ID: <012C3117EDDB3C4781FD802A8C27DD4F0BF1A342@SACEXCMBX02-PRD.hq.netapp.com> References: , <012C3117EDDB3C4781FD802A8C27DD4F080D40D3@SACEXCMBX02-PRD.hq.netapp.com>, In-Reply-To: Accept-Language: de-AT, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.106.53.53] Content-Type: multipart/alternative; boundary="_000_012C3117EDDB3C4781FD802A8C27DD4F0BF1A342SACEXCMBX02PRDh_" MIME-Version: 1.0 Subject: Re: [tcpm] TCP Error recovery and efficiency (updated reply) X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Jul 2012 13:29:58 -0000 --_000_012C3117EDDB3C4781FD802A8C27DD4F0BF1A342SACEXCMBX02PRDh_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hello Anthony, As a summary of our offline discussion, I believe I understand your loss re= covery mechanism. Also, the really important signal that is missing from cu= rrent TCP, to allow the implementation of this mechanism is the so-called "= token". I still believe that modifying the SACK signaling option is the wrong place= to introduce this token. SACK, as defined and implemented today, should keep the scoreboard in the s= ender in a very accurate state, with high reliability. And I think it perfo= rms this task very well. >From the description so far, it's apparent that the tokens described are ba= sically packet counters, that are monotonically increasing for each newly s= ent packet - regardless of original packet, or retransmission. With only 16= bits available for this counter, under extreme circumstances a BDP of ~100= MB could result in issues; However, the bounds for that token are not expl= icitly defined. I believe that it really only needs to increase once per RT= T, or once per loss recovery epoch (resending any one segment anew starts o= ne such epoch) - which can be considerably slower than a mere packet counte= r. If the above observations hold, that your token has much more resemblance w= ith the TCP timestamps. The major difference being, that timestamps are not= immediately reflected when an ACK is sent - particularly not during loss = / reorder events. OTOH, using both SACK and TS synergistically, could retain the wire encodin= g and signaling, and existing mechanisms using these options, while allowin= g new features, such as a more efficient loss recovery scheme as proposed. I'm looking forward to a lively discussion around these topics in Vancouver= , if no one wants to share his views on list. Richard Scheffenegger From: Anthony Sabatini [mailto:tsabatini@hotmail.com] Sent: Mittwoch, 11. Juli 2012 18:47 To: Scheffenegger, Richard; tcpm; draft-sack@tsabatini.com Subject: RE: [tcpm] TCP Error recovery and efficiency (updated reply) Sorry - previous message sent in error. Richard - First off thank you for taking the time to read the draft. With respect to the option numbers, as I wrote to the tcpm chairs - I can n= ot presume in a draft what option numbers will be assigned in a standard bu= t I needed a place holder for illustration. I therefore used the ones in S= ACK to show how they relate and to guarantee in the final document they wil= l be corrected to the assigned appropriate values. With respect to sending multiple ACKs and what is contained in those ACKs -= in IPv4 the TCP options fields are very limited and I have had to steal fo= ur more of them. Fortunately SACK uses 8 byte pairs so I have not decrease= d the number of pairs sent, but in order to render a complete state I may n= eed to send a multiple number of ACKs. Like SACK, I have to lay out at lea= st a framework of what pairs get transmitted in what order with how many re= peats. Like SACK the first pairs have to be any blocks which have changed = sizes since my last acknowledgement. The next most important block is the = oldest since it defines the oldest, and thus the most important, missing se= gment. The rest of the blocks are then transmitted, oldest first. Repeati= ng this sequence of ACKs is an homage to any of the various TCP fast recove= ry mechanisms, they are right in that you are as likely to lose the ACK as = you are to lose the message. Two things I am sure of, 1) someone is going = to do a masters thesis on what is the optimal order to send the blocks in a= nd how many repeats are best and 2) someone is going to suggest compressing= the segment limits in the TCP options (and we might want to make that chan= ge in this standard). Note that the 1/4 RTT time was a restatement of a SA= CK philosophy to which I personally find of minimal value. An immediate re= peat of the ACK set insures against transient packet loss, the 1/4 RTT only= adds robustness where the link drops a burst of segments such as happens i= n an overloaded content inspector. Note that unless you have a SACK situat= ion, normal TCP acknowledgement rules apply, even if you have a couple of S= ACK segments, normal TCP acknowledgement rules apply. In both cases the on= ly thing that changed was there are options in the ACK. It is only when w= e are fairly far down into the weeds that either the original SACK or my ve= rsion changes anything and mostly it is to generate multiple ACKS so we can= get the state out. Thus I change nothing and violate no rules on a clean = link or even a slightly dirty one. The violation that occurs is in the ori= ginal, approved, SACK RFC. With respect to timestamps - having written many network drivers with many= operating systems, I have tried to avoid them on a driver level if possibl= e as calling one device driver from within another can lead to unintended c= onsequences. For example many clocks lock a call when they are processing = a time increment so that yo do not get a ripple effect of half an updated t= imestamp. This lead to an all block at the network level for that period o= f time which is not good. It also means that you have to timestamp every o= utgoing message in the transmit queue. By using as your token the current = index into the transmitted segment queue you can immediately work forward i= n that queue to see what segments have been retransmitted but not yet recei= ved, regardless of timestamp. With respect to the scoreboard - the acknowledgement in the TCP header form= s a floor for the scoreboard, all entries with segment finishes prior to th= at are released when this value is updated (others are retained as not yet = complete). With respect to selective acknowledgements, a SACK is permanent= , it creates a begin/end pair in the scoreboard, as subsequent SACKs come i= n entries may be inserted, added to the end, or merged with one or more of = the existing entries. The holes between these entries represent the segmen= ts (and between the floor and the first) that must be retransmitted. Likew= ise in the reciever the TCP acknowledgement rules, the receiver must remove= all SACK blocks on its side which are now below the updated floor. With respect to the delay returning a received token - we as a group take t= oo little time understanding why a protocol works. TCP is tolerant of out = of sequence packet arrivals because it saves segments beyond the one it exp= ects and will fill in when they arrive out of order. Since the proposed ch= ange will tightly monitor missing segments, if the far end updated its retu= rned token immediately then any out of order packets would be considered pe= rmanently lost. By delaying the update of the returned token I allow time = for the out of order packet to arrive and be processed. Yes this hurts eff= iciency and yes the delay should be set to zero in a point to point environ= ment but having this feature means that the modified protocol has all of th= e robustness of the original. A seconday important feature of the delay is= that it insures that all of the ACKs that comprise the receiver state have= been received and processed. With respect to Vancouver - I will go if anyone cares about this - since yo= u do, I could be easily convinced. I have asked the tcpm chairs for 15 min= utes to explain myself but have not heard from them as of yet. Note for the record - I am probably not the best person to rewrite this dra= ft as I am too close to the issue, if anyone wants to co-author it and rew= rite it please let me know, we will then, at least, have one other person w= ho understands what I am trying to express. Tony Anthony Sabatini 200 West 20th Street Apt. 1216 New York, NY 10011 Phone: (212) 867-7179 Mobile: (917) 224-8388 ________________________________ From: rs@netapp.com To: tsabatini@hotmail.com; tcpm@ietf.org; draft-sack@tsabatini.com Subject: RE: [tcpm] TCP Error recovery and efficiency Date: Wed, 11 Jul 2012 11:39:38 +0000 Hi Anthony, first off, a quick observation: Modifying existing options, particularly th= eir length or encoding, as you propose in that draft, will probably not wor= k; there are many devices, which may react undefined (ignore the extended S= ACK permitted, accept it in the old way,...) which wouldn't be compatible. Also, certain middleboxes do modify the contents of regular SACK blocks (re= writing TCP sequence numbers), and they would definitely interact very badl= y when a well-defined tcp option suddenly has different semantics. Second, I try to understand your language around the token; I'm not sure I = follow the reasoning around artificially delaying ACK / SACK updates on the= receiver. A worked out example as appendix would do good (showing which fields contai= n what when, and the exchange of these tokens between sender and receiver). Third, the draft appears to propose sending multiple ACKs in response to a = (receiver delay timeout), with varying SACK option contents between these A= CKs (all having the same TCP header fields)? The purpose of these ACKs bein= g to clock out the missing segments from the sender, if I read this correct= ly? However, no precaution is there to prevent a burst or train of ACKs (wh= ich would cause a transmit spike on the sender side), correct? I'm not quite clear on the sender behavior; the text seems to imply that th= e SACK scoreboard should be deleted for each received ACK/SACK, and reconst= ructed anew? The new "token" fields serving as secondary ACK counter to only resend thos= e bytes that were not retransmitted by the time the receiver got that token= ? Reconstructing the order in which packets arrive at the destination is, IMH= O, a valuable goal to optimize the error recovery of TCP SACK. So, to summarize, there are two main ideas in this draft: a) Use of a deterministically reflected token, that serves to reconstr= uct the exact order of packets received at the receiver (including retransm= issions). b) Sending of unsolicidated ACKs based on a short timeout (1/4 RTT) in= order to pull lost/delayed data from the sender without reverting to a sen= der-side RTO c) Modification of a well-known TCP option with new data fields Please confirm that I have properly summarized your ideas in that draft. I'm very much in favor of idea a). However, I still believe that the proper= approach would be to synergistically use Timestamp together with SACK to a= chieve the same (which requires "only" a semantic/behavior change of the TS= field in the presence of the SACK option, but doesn't alter the encoding/i= nterpretation per se). With that modification to the Timestamp semantics, t= he timestamps can serve a "tokens" delivering the information you seek to b= etter reconstruct the receiver state at the sender. Please have a look at h= ttp://tools.ietf.org/id/draft-scheffenegger-tcpm-timestamp-negotiation-03.t= xt. I'm not so sure about idea b) since that would appear to violate the conser= vation of packets rule - even under optimal conditions (no packet loss, onl= y very variable packet delay/reordering), a higher rate of ACKs would be se= nt back, potentially completely unnecessary. With c) - this will break this proposal. Please use experimental options i= n the draft (and newly assigned option numbers if adopted) so to avoid comp= atibility issues with existing deployed hardware in the internet. Will you be present in Vancouver? Best regards, Richard Scheffenegger From: tcpm-bounces@ietf.org [mailto:tcpm-boun= ces@ietf.org] On Behalf Of Anthony S= abatini Sent: Montag, 09. Juli 2012 05:55 To: tcpm@ietf.org Subject: [tcpm] TCP Error recovery and efficiency All - While data communications was constantly improving, due mostly to the wide = adoption of fiber optics, the fact that TCP had an inferior recovery mechan= ism was moot as it came into play less and less often. Sadly those days ar= e now behind us and we face a world increasingly hostile to data transmissi= on. The sources of packet loss have grown legion, "traffic shapers", conge= sted links to the subscibers, overloaded wifi bands, content monitor that c= an not keep up, rate caps on mobile phone data usage, the list goes on and = on. Complicating this is the increase in data rates and file sizes which a= llow many more packets and their acknowledgements to be inflight between tw= o endpoints at once. We as a community must move along the proliferation o= f SACK and if we are going to do that we should fix the underlying issues i= n the protocol first. All current TCP recovery implementations are based on guessing the state at= the remote node. No recovery mechanism currently in place gives sufficien= t weight to in-flight or delayed messages. This problem can be solved easi= ly by transmitting a token related to the transmission state in each out go= ing message which the destination node will return on its next transmission= . By having this token I know all of the messages the far end should have = seen and if I do not have an acknowledgement for any of them, I know that t= hose need to be retransmitted. To start this discussion I have created an I-D "Highly Efficient Selective = Acknowledgement (SACK) for TCP" - draft-sabatini-tcp-sack-00 which I hope w= ill be accepted by the working group. Note that even though no TCP implementation exists, the HDLC version I crea= ted for the financial community has 20 years of working history all over th= e world and the capability for being 100% efficient right down to one error= per message no matter what the data rate or the delay (including multiple = satellite hops). Tony --_000_012C3117EDDB3C4781FD802A8C27DD4F0BF1A342SACEXCMBX02PRDh_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Hello Anthony,=

 <= /p>

As a summa= ry of our offline discussion, I believe I understand your loss recovery mec= hanism. Also, the really important signal that is missing from current TCP, to allow the implementation of this mechanism is the so-= called “token”.

 = ;

I still be= lieve that modifying the SACK signaling option is the wrong place to introd= uce this token.

 = ;

SACK, as d= efined and implemented today, should keep the scoreboard in the sender in a= very accurate state, with high reliability. And I think it performs this task very well.

 = ;

 = ;

From the d= escription so far, it’s apparent that the tokens described are basica= lly packet counters, that are monotonically increasing for each newly sent packet – regardless of original packet, or retransmission= . With only 16 bits available for this counter, under extreme circumstances= a BDP of ~100 MB could result in issues; However, the bounds for that toke= n are not explicitly defined. I believe that it really only needs to increase once per RTT, or once per loss recov= ery epoch (resending any one segment anew starts one such epoch) – wh= ich can be considerably slower than a mere packet counter.

 = ;

If the abo= ve observations hold, that your token has much more resemblance with the TC= P timestamps. The major difference being, that timestamps are not immediately reflected when an ACK is sent -  particularly not= during loss / reorder events.

 = ;

OTOH, usin= g both SACK and TS synergistically, could retain the wire encoding and sign= aling, and existing mechanisms using these options, while allowing new features, such as a more efficient loss recovery scheme as pr= oposed.

 = ;

 = ;

I’m = looking forward to a lively discussion around these topics in Vancouver, if= no one wants to share his views on list.

 = ;

 = ;

Richard Scheffenegger

 <= /p>

From: Anthony Sabatini [mailto:tsabatini@hotmail.com]
Sent: Mittwoch, 11. Juli 2012 18:47
To: Scheffenegger, Richard; tcpm; draft-sack@tsabatini.com
Subject: RE: [tcpm] TCP Error recovery and efficiency (updated reply= )

 

Sorry - p= revious message sent in error.

Richard -=

First off thank you for taking the time to read the draft.

With respect to the option numbers, as I wrote to the tcpm chairs - I can n= ot presume in a draft what option numbers will be assigned in a standard bu= t I needed a place holder for illustration.  I therefore used the ones= in SACK to show how they relate and to guarantee in the final document they will be corrected to the assigned = appropriate values.

With respect to sending multiple ACKs and what is contained in those ACKs -= in IPv4 the TCP options fields are very limited and I have had to steal fo= ur more of them.  Fortunately SACK uses 8 byte pairs so I have not dec= reased the number of pairs sent, but in order to render a complete state I may need to send a multiple number o= f ACKs.  Like SACK, I have to lay out at least a framework of what pai= rs get transmitted in what order with how many repeats.  Like SACK the= first pairs have to be any blocks which have changed sizes since my last acknowledgement.  The next most important= block is the oldest since it defines the oldest, and thus the most importa= nt, missing segment.  The rest of the blocks are then transmitted, old= est first.  Repeating this sequence of ACKs is an homage to any of the various TCP fast recovery mechanisms, they are = right in that you are as likely to lose the ACK as you are to lose the mess= age.  Two things I am sure of, 1) someone is going to do a masters the= sis on what is the optimal order to send the blocks in and how many repeats are best and 2) someone is going to sug= gest compressing the segment limits in the TCP options (and we might want t= o make that change in this standard).  Note that the 1/4 RTT time was = a restatement of a SACK philosophy to which I personally find of minimal value.  An immediate repeat of the= ACK set insures against transient packet loss, the 1/4 RTT only adds robus= tness where the link drops a burst of segments such as happens in an overlo= aded content inspector.  Note that unless you have a SACK situation, normal TCP acknowledgement rules apply, even if= you have a couple of SACK segments, normal TCP acknowledgement rules apply= .  In both cases the only thing that changed was there are options in = the ACK.  It is  only when we are fairly far down into the weeds that either the original SACK or my version change= s anything and mostly it is to generate multiple ACKS so we can get the sta= te out.  Thus I change nothing and violate no rules on a clean link or= even a slightly dirty one.  The violation that occurs is in the original, approved, SACK RFC.

 With respect to timestamps - having written many network drivers with= many operating systems, I have tried to avoid them on a driver level if po= ssible as calling one device driver from within another can lead to uninten= ded consequences.  For example many clocks lock a call when they are processing a time increment so that yo do not ge= t a ripple effect of half an updated timestamp.  This lead to an all b= lock at the network level for that period of time which is not good.  = It also means that you have to timestamp every outgoing message in the transmit queue.  By using as your token the c= urrent index into the transmitted segment queue you can immediately work fo= rward in that queue to see what segments have been retransmitted but not ye= t received, regardless of timestamp.

With respect to the scoreboard - the acknowledgement in the TCP header form= s a floor for the scoreboard, all entries with segment finishes prior to th= at are released when this value is updated (others are retained as not yet = complete).  With respect to selective acknowledgements, a SACK is permanent, it creates a begin/end pair in the = scoreboard, as subsequent SACKs come in entries may be inserted, added to t= he end, or merged with one or more of the existing entries.  The holes= between these entries represent the segments (and between the floor and the first) that must be retransmitted.=   Likewise in the reciever the TCP acknowledgement rules, the receiver= must remove all SACK blocks on its side which are now below the updated fl= oor.

With respect to the delay returning a received token - we as a group take t= oo little time understanding why a protocol works.  TCP is tolerant of= out of sequence packet arrivals because it saves segments beyond the one i= t expects and will fill in when they arrive out of order.  Since the proposed change will tightly monitor = missing segments, if the far end updated its returned token immediately the= n any out of order packets would be considered permanently lost.  By d= elaying the update of the returned token I allow time for the out of order packet to arrive and be processed.  Y= es this hurts efficiency and yes the delay should be set to zero in a point= to point environment but having this feature means that the modified proto= col has all of the robustness of the original.  A seconday important feature of the delay is that it insur= es that all of the ACKs that comprise the receiver state have been received= and processed.

With respect to Vancouver - I will go if anyone cares about this - since yo= u do, I could be easily convinced.  I have asked the tcpm chairs for 1= 5 minutes to explain myself but have not heard from them as of yet.

Note for the record - I am probably not the best person to rewrite this dra= ft as I am too close to the issue,  if anyone wants to co-author it an= d rewrite it please let me know, we will then, at least, have one other per= son who understands what I am trying to express.

Tony

Anthony Sabatini
200 West 20th Street
Apt. 1216
New York, NY 10011

Phone: (212) 867-7179
Mobile: (917) 224-8388

 

From: rs@netapp.com
To: tsabatini@hotmail.com; tcpm@ietf.org; draft-sack@t= sabatini.com
Subject: RE: [tcpm] TCP Error recovery and efficiency
Date: Wed, 11 Jul 2012 11:39:38 +0000

 

Hi Anthony= ,

 

first off,= a quick observation: Modifying existing options, particularly their length= or encoding, as you propose in that draft, will probably not work; there are many devices, which may react undefined (ignore the ex= tended SACK permitted, accept it in the old way,…) which wouldn’= ;t be compatible.

 

Also, cert= ain middleboxes do modify the contents of regular SACK blocks (rewriting TC= P sequence numbers), and they would definitely interact very badly when a well-defined tcp option suddenly has different semantics.

 

 

Second, I = try to understand your language around the token; I’m not sure I foll= ow the reasoning around artificially delaying ACK / SACK updates on the receiver.

 

A worked o= ut example as appendix would do good (showing which fields contain what whe= n, and the exchange of these tokens between sender and receiver).

 

Third, the= draft appears to propose sending multiple ACKs in response to a (receiver = delay timeout), with varying SACK option contents between these ACKs (all having the same TCP header fields)? The purpose of these A= CKs being to clock out the missing segments from the sender, if I read this= correctly? However, no precaution is there to prevent a burst or train of = ACKs (which would cause a transmit spike on the sender side), correct?

 

I’m = not quite clear on the sender behavior; the text seems to imply that the SA= CK scoreboard should be deleted for each received ACK/SACK, and reconstructed anew?

 

The new &#= 8220;token” fields serving as secondary ACK counter to only resend th= ose bytes that were not retransmitted by the time the receiver got that token?

 

 

 

 

Reconstruc= ting the order in which packets arrive at the destination is, IMHO, a valua= ble goal to optimize the error recovery of TCP SACK.

 

 

So, to sum= marize, there are two main ideas in this draft:

 

a)      Use of a deterministically= reflected token, that serves to reconstruct the exact order of packets rec= eived at the receiver (including retransmissions).

 

b)      Sending of unsolicidated A= CKs based on a short timeout (1/4 RTT) in order to pull lost/delayed data f= rom the sender without reverting to a sender-side RTO

 

c)       Modification of a well-kno= wn TCP option with new data fields

 

 

Please con= firm that I have properly summarized your ideas in that draft.

 

I’m = very much in favor of idea a). However, I still believe that the proper app= roach would be to synergistically use Timestamp together with SACK to achieve the same (which requires “only” a semantic/behavior= change of the TS field in the presence of the SACK option, but doesn’= ;t alter the encoding/interpretation per se). With that modification to the= Timestamp semantics, the timestamps can serve a “tokens” delivering the information you seek to better reconstruct the receiver sta= te at the sender. Please have a look at http://tools.ietf.org/id/draft-scheffenegger-tcpm-timestamp-negotiation-03.= txt.

 

I’m = not so sure about idea b) since that would appear to violate the conservati= on of packets rule – even under optimal conditions (no packet loss, only very variable packet delay/reordering), a higher rate of ACKs would b= e sent back, potentially completely unnecessary.

 

With c)&nb= sp; - this will break this proposal. Please use experimental options in the= draft (and newly assigned option numbers if adopted) so to avoid compatibility issues with existing deployed hardware in the internet.

 

Will you b= e present in Vancouver?

 

Best regar= ds,

 

Richard Scheffenegger

 

From: tcpm-bounces@ietf.org [mailto:tcpm-bounces@ietf.org] On Behalf Of Anthony Sabatini
Sent: Montag, 09. Juli 2012 05:55
To: tcpm@ietf.org
Subject: [tcpm] TCP Error recovery and efficiency

 

All -

While data communications was constantly improving, due mostly to the wide = adoption of fiber optics, the fact that TCP had an inferior recovery mechan= ism was moot as it came into play less and less often.  Sadly those da= ys are now behind us and we face a world increasingly hostile to data transmission.  The sources of packet los= s have grown legion, "traffic shapers", congested links to the su= bscibers, overloaded wifi bands, content monitor that can not keep up, rate= caps on mobile phone data usage, the list goes on and on.  Complicating this is the increase in data rates and file = sizes which allow many more packets and their acknowledgements to be inflig= ht between two endpoints at once.  We as a community must move along t= he proliferation of SACK and if we are going to do that we should fix the underlying issues in the protocol first.

All current TCP recovery implementations are based on guessing the state at= the remote node.  No recovery mechanism currently in place gives suff= icient weight to in-flight or delayed messages.  This problem can be s= olved easily by transmitting a token related to the transmission state in each out going message which the destination = node will return on its next transmission.  By having this token I kno= w all of the messages the far end should have seen and if I do not have an = acknowledgement for any of them, I know that those need to be retransmitted.

To start this discussion I have created an I-D "Highly Efficient Selec= tive Acknowledgement (SACK) for TCP" - draft-sabatini-tcp-sack-00 whic= h I hope will be accepted by the working group.

Note that even though no TCP implementation exists, the HDLC version I crea= ted for the financial community has 20 years of working history all over th= e world and the capability for being 100% efficient right down to one error= per message no matter what the data rate or the delay (including multiple satellite hops).

Tony

--_000_012C3117EDDB3C4781FD802A8C27DD4F0BF1A342SACEXCMBX02PRDh_-- From ycheng@google.com Thu Jul 12 08:05:53 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C778C21F8777 for ; Thu, 12 Jul 2012 08:05:53 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.827 X-Spam-Level: X-Spam-Status: No, score=-102.827 tagged_above=-999 required=5 tests=[AWL=0.150, BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TgS1n+iGSYGU for ; Thu, 12 Jul 2012 08:05:53 -0700 (PDT) Received: from mail-yw0-f44.google.com (mail-yw0-f44.google.com [209.85.213.44]) by ietfa.amsl.com (Postfix) with ESMTP id 1898821F8770 for ; Thu, 12 Jul 2012 08:05:53 -0700 (PDT) Received: by yhq56 with SMTP id 56so2900705yhq.31 for ; Thu, 12 Jul 2012 08:06:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type:x-system-of-record; bh=Uat3VUbp3vo3GrCmyTIoNS7i8SDo6sDXNItxbbrJEbY=; b=TNm19q5dYJ4RHXfY2rFCfiC16Ulqn/IlW1AGYYCvwkp3xM/BiLymQTrP+ZfofYGqXK aJDfepXg93fLyLGhoMQnjPNx+34LXbQIDQEt/5Vl5vTYP7WHknV6mTsYScfynrJTD1k4 lgQYzlmL22u1iYQoDz/B20rk6lRatJ79pIauEGi1o97BYi2vyX7a7JvwiaeOTrAuKJq+ PKBfQo/b+bb3Q1Bgl01GZwGQo/fJg63eDGjA5vRnEYg37H7/5MujQ1lWaxdvtawx1945 WThfRevbwvxpRGvR9Qcsi7Qt1hq6VlLs/H8hQHcy7bSat3SDCDd1Bfy57SZ/hqrrFSxQ 23Tg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type:x-system-of-record:x-gm-message-state; bh=Uat3VUbp3vo3GrCmyTIoNS7i8SDo6sDXNItxbbrJEbY=; b=FQ6uWzC7brXmAi9x4dmm6FS/czBdMvlE8tOvRgB32URj+LZCRpXij0u/z2xOT8JIAC /SDyO2TjciXhoeTMG1CdD8HZ96g29JU89vbW45SYhD69TcaVTj6vCNKvYPCZwPprHwvJ DZCvT2AUobmN/ZAVZZgSOmH0NSDFLjnpI093Jk83iuxsKlWpZapoZ1GyUBQVSUu7yXDa OBmRARsvAPJScyop6FatawxnwSVAxtlWNQz3QlGw1bUoWvUsu2dMer5Zn88hP0ZSGTYn TYZEhVa8yQqMkwK2K2BJVrLaZkDTLVVMtAN39NwXPnrohnnsDYC9ua/ONAVk8jvnr/Qd CVJw== Received: by 10.60.18.134 with SMTP id w6mr2662266oed.56.1342105586434; Thu, 12 Jul 2012 08:06:26 -0700 (PDT) Received: by 10.60.18.134 with SMTP id w6mr2662245oed.56.1342105586224; Thu, 12 Jul 2012 08:06:26 -0700 (PDT) MIME-Version: 1.0 Received: by 10.60.14.168 with HTTP; Thu, 12 Jul 2012 08:06:06 -0700 (PDT) In-Reply-To: References: From: Yuchung Cheng Date: Thu, 12 Jul 2012 08:06:06 -0700 Message-ID: To: Anthony Sabatini Content-Type: text/plain; charset=ISO-8859-1 X-System-Of-Record: true X-Gm-Message-State: ALoCoQmXpr7LI+BoW/QIZwQ7rY2uPqFLT3KpAmzRX7XFDMSvNHw2E5n7ULFxHFe5qDXvyi+2OH/sPmywKTdoqo0RxpJOjr2PMq3dQtcZOs/x2U354qgoXH/myXwhqZNp2/A5KCAdCu9hVVnerg16zf7kBJ62RiINCQPHjP3Z1MNRp8FMJMy+tds= Cc: tcpm@ietf.org Subject: Re: [tcpm] TCP Error recovery and efficiency X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Jul 2012 15:05:53 -0000 Hi Anthony, On Sun, Jul 8, 2012 at 8:55 PM, Anthony Sabatini wrote: > All - > > While data communications was constantly improving, due mostly to the wide > adoption of fiber optics, the fact that TCP had an inferior recovery > mechanism was moot as it came into play less and less often. Sadly those > days are now behind us and we face a world increasingly hostile to data > transmission. The sources of packet loss have grown legion, "traffic > shapers", congested links to the subscibers, overloaded wifi bands, content > monitor that can not keep up, rate caps on mobile phone data usage, the list > goes on and on. Complicating this is the increase in data rates and file > sizes which allow many more packets and their acknowledgements to be > inflight between two endpoints at once. We as a community must move along > the proliferation of SACK and if we are going to do that we should fix the > underlying issues in the protocol first. > > All current TCP recovery implementations are based on guessing the state at > the remote node. No recovery mechanism currently in place gives sufficient > weight to in-flight or delayed messages. This problem can be solved easily > by transmitting a token related to the transmission state in each out going > message which the destination node will return on its next transmission. By What exactly do sender and receiver put in the sender/receiver state field? Maybe we can have an example? say senders send 1,2,3,4 packets, receiver receives only first 2nd then 4th packets. The state you described resembles sack scoreboard in RFC 3517, which is implemented in all major operating systems. Can you compare your approach to that RFC? > having this token I know all of the messages the far end should have seen > and if I do not have an acknowledgement for any of them, I know that those > need to be retransmitted. > > To start this discussion I have created an I-D "Highly Efficient Selective > Acknowledgement (SACK) for TCP" - draft-sabatini-tcp-sack-00 which I hope > will be accepted by the working group. > > Note that even though no TCP implementation exists, the HDLC version I > created for the financial community has 20 years of working history all over > the world and the capability for being 100% efficient right down to one > error per message no matter what the data rate or the delay (including > multiple satellite hops). > > Tony > > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm > From wes@mti-systems.com Thu Jul 12 08:44:18 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 88A7E11E8098 for ; Thu, 12 Jul 2012 08:44:18 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.338 X-Spam-Level: X-Spam-Status: No, score=-2.338 tagged_above=-999 required=5 tests=[AWL=0.261, BAYES_00=-2.599] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id B5pOyZj9vTck for ; Thu, 12 Jul 2012 08:44:18 -0700 (PDT) Received: from omr16.networksolutionsemail.com (omr16.networksolutionsemail.com [205.178.146.66]) by ietfa.amsl.com (Postfix) with ESMTP id C019F11E808E for ; Thu, 12 Jul 2012 08:44:17 -0700 (PDT) Received: from cm-omr10 (mail.networksolutionsemail.com [205.178.146.50]) by omr16.networksolutionsemail.com (8.13.8/8.13.8) with ESMTP id q6CFinaV023766 for ; Thu, 12 Jul 2012 11:44:50 -0400 Authentication-Results: cm-omr10 smtp.user=wes@mti-systems.com; auth=pass (PLAIN) X-Authenticated-UID: wes@mti-systems.com Received: from [63.226.32.150] ([63.226.32.150:31237] helo=[172.27.250.202]) by cm-omr10 (envelope-from ) (ecelerity 2.2.2.41 r(31179/31189)) with ESMTPA id DF/08-04850-1F0FEFF4; Thu, 12 Jul 2012 11:44:49 -0400 Message-ID: <4FFEF0F0.4010805@mti-systems.com> Date: Thu, 12 Jul 2012 11:44:48 -0400 From: Wesley Eddy Organization: MTI Systems User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:12.0) Gecko/20120428 Thunderbird/12.0.1 MIME-Version: 1.0 To: Pasi Sarolahti References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: tcpm@ietf.org Subject: Re: [tcpm] Draft TCPM agenda for IETF-84 X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Jul 2012 15:44:18 -0000 On 7/12/2012 5:49 AM, Pasi Sarolahti wrote: > > * HOST_ID TCP Options: Implementation & Preliminary Test Results > draft-abdo-hostid-tcpopt-implementation > Speaker: Jaqueline Queiroz > Time: 10 minutes Agenda bashing ... I don't think this is useful to discuss in our precious face-to-face time. It is not of value to TCP. I'm happy to be wrong if there are others who think it needs face time, but I'd rather just use the mailing list to say that this is stupid and leave the meeting time to talk about more interesting things (e.g. Laminar). There are a couple of other topics on the agenda that I don't think have seen any mailing list traffic. -- Wes Eddy MTI Systems From tsabatini@hotmail.com Wed Jul 11 08:34:39 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3B31721F8685 for ; Wed, 11 Jul 2012 08:34:39 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.948 X-Spam-Level: X-Spam-Status: No, score=-1.948 tagged_above=-999 required=5 tests=[AWL=0.650, BAYES_00=-2.599, HTML_MESSAGE=0.001] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id raGtnT+5Il2B for ; Wed, 11 Jul 2012 08:34:37 -0700 (PDT) Received: from bay0-omc4-s20.bay0.hotmail.com (bay0-omc4-s20.bay0.hotmail.com [65.54.190.222]) by ietfa.amsl.com (Postfix) with ESMTP id 98BEC21F8688 for ; Wed, 11 Jul 2012 08:34:37 -0700 (PDT) Received: from BAY158-W28 ([65.54.190.201]) by bay0-omc4-s20.bay0.hotmail.com with Microsoft SMTPSVC(6.0.3790.4675); Wed, 11 Jul 2012 08:35:07 -0700 Message-ID: Content-Type: multipart/alternative; boundary="_3f1272e0-cc2b-48ad-b6f0-2397069a8a48_" X-Originating-IP: [74.64.96.39] From: Anthony Sabatini To: , tcpm , Date: Wed, 11 Jul 2012 11:35:06 -0400 Importance: Normal In-Reply-To: <012C3117EDDB3C4781FD802A8C27DD4F080D40D3@SACEXCMBX02-PRD.hq.netapp.com> References: , <012C3117EDDB3C4781FD802A8C27DD4F080D40D3@SACEXCMBX02-PRD.hq.netapp.com> MIME-Version: 1.0 X-OriginalArrivalTime: 11 Jul 2012 15:35:07.0652 (UTC) FILETIME=[C00FC440:01CD5F7A] X-Mailman-Approved-At: Thu, 12 Jul 2012 08:52:00 -0700 Subject: Re: [tcpm] TCP Error recovery and efficiency X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Jul 2012 15:34:39 -0000 --_3f1272e0-cc2b-48ad-b6f0-2397069a8a48_ Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable Richard - First off thank you for taking the time to read the draft. With respect to the option numbers=2C as I wrote to the tcpm chairs - I can= not presume in a draft what option numbers will be assigned in a standard = but I needed a place holder for illustration. I therefore used the ones in= SACK to show how they relate and to guarantee in the final document they w= ill be corrected to the assigned appropriate values. With respect to sending multiple acks and what is contained in those acks -= in IPv4 the TCP options fields are very limited and I have had to steal fo= ur more of them. Fortunately SACK uses 8 byte pairs so I have not decrease= d the number of pairs sent=2C but in order to render a complete state I may= need to send a multiple number of ACKs. Like SACK=2C I have to lay out at= least a framework of what pairs get transmitted in what order with how man= y repeats. Like SACK the first pairs have to be any blocks which have chan= ged sizes since my last acknowledgement. The next most important block is = the oldest since it defines the oldest=2C and thus the most important=2C mi= ssing segment. The rest o the blocks are then transmitted=2C oldest first.= Repeating this sequence of ACKs is an homage to any of the various TCP fa= st recovery mechanisms=2C they are right in that you are as likely to lose = the ACK as you are to lose the message. With respect to the delay returning a received token - we as a group take t= oo little time understanding why a protocol works. TCP is tolerant of out = of sequence packet arrivals because it saves segments beyond the one it exp= ects and will fill in when they arrive out of order. Since the proposed ch= ange will tightly monitor missing segments=2C if the far end updated its re= turned token immediately then any out of order packets would be considered = permanently lost. By delaying the update of the returned token I allow tim= e for the out of order packet to arrive and be processed. Yes this hurts e= fficiency and yes the delay should be set to zero in a point to point envir= onment but having this feature means that the modified protocol has all of = the robustness of the original. A seconday important feature of the delay = is that it insures that all of the ACKs that comprise the receiver state ha= ve been received and processed. Tony Anthony Sabatini 200 West 20th Street Apt. 1216 New York=2C NY 10011 Phone: (212) 867-7179 Mobile: (917) 224-8388 =20 From: rs@netapp.com To: tsabatini@hotmail.com=3B tcpm@ietf.org=3B draft-sack@tsabatini.com Subject: RE: [tcpm] TCP Error recovery and efficiency Date: Wed=2C 11 Jul 2012 11:39:38 +0000 =20 Hi Anthony=2C =20 first off=2C a quick observation: Modifying existing options=2C particularl= y their length or encoding=2C as you propose in that draft=2C will probably not work=3B there are many devices=2C which may react undefined (ignore th= e extended SACK permitted=2C accept it in the old way=2C=85) which wouldn= =92t be compatible. =20 Also=2C certain middleboxes do modify the contents of regular SACK blocks (= rewriting TCP sequence numbers)=2C and they would definitely interact very badly when a well-defined tcp option suddenly has different semantics. =20 =20 Second=2C I try to understand your language around the token=3B I=92m not s= ure I follow the reasoning around artificially delaying ACK / SACK updates on the receiver. =20 A worked out example as appendix would do good (showing which fields contai= n what when=2C and the exchange of these tokens between sender and receiver= ). =20 Third=2C the draft appears to propose sending multiple ACKs in response to = a (receiver delay timeout)=2C with varying SACK option contents between these ACKs (all having the same TCP header fields)? The purpose of these A= CKs being to clock out the missing segments from the sender=2C if I read th= is correctly? However=2C no precaution is there to prevent a burst or train= of ACKs (which would cause a transmit spike on the sender side)=2C correct? =20 I=92m not quite clear on the sender behavior=3B the text seems to imply tha= t the SACK scoreboard should be deleted for each received ACK/SACK=2C and reconstructed anew? =20 The new =93token=94 fields serving as secondary ACK counter to only resend = those bytes that were not retransmitted by the time the receiver got that token? =20 =20 =20 =20 Reconstructing the order in which packets arrive at the destination is=2C I= MHO=2C a valuable goal to optimize the error recovery of TCP SACK. =20 =20 So=2C to summarize=2C there are two main ideas in this draft: =20 a) =20 Use of a deterministically reflected token=2C that serves to reconstruct th= e exact order of packets received at the receiver (including retransmissions).=20 =20 b) =20 Sending of unsolicidated ACKs based on a short timeout (1/4 RTT) in order t= o pull lost/delayed data from the sender without reverting to a sender-side RTO =20 c) =20 Modification of a well-known TCP option with new data fields =20 =20 Please confirm that I have properly summarized your ideas in that draft. =20 I=92m very much in favor of idea a). However=2C I still believe that the pr= oper approach would be to synergistically use Timestamp together with SACK to achieve the same (which requires =93only=94 a semantic/behavior change = of the TS field in the presence of the SACK option=2C but doesn=92t alter t= he encoding/interpretation per se). With that modification to the Timestamp= semantics=2C the timestamps can serve a =93tokens=94 delivering the information you seek to better reconstruct the receiver sta= te at the sender. Please have a look at http://tools.ietf.org/id/draft-sche= ffenegger-tcpm-timestamp-negotiation-03.txt. =20 I=92m not so sure about idea b) since that would appear to violate the cons= ervation of packets rule =96 even under optimal conditions (no packet loss= =2C only very variable packet delay/reordering)=2C a higher rate of ACKs would= be sent back=2C potentially completely unnecessary. =20 With c) - this will break this proposal. Please use experimental options i= n the draft (and newly assigned option numbers if adopted) so to avoid compatibility issues with existing deployed hardware in the internet. =20 Will you be present in Vancouver? =20 Best regards=2C =20 Richard Scheffenegger =20 From: tcpm-bounces@ietf.org [mailto:tcpm-bounces@ietf.org] On Behalf Of Anthony Sabatini Sent: Montag=2C 09. Juli 2012 05:55 To: tcpm@ietf.org Subject: [tcpm] TCP Error recovery and efficiency =20 All - While data communications was constantly improving=2C due mostly to the wid= e adoption of fiber optics=2C the fact that TCP had an inferior recovery me= chanism was moot as it came into play less and less often. Sadly those day= s are now behind us and we face a world increasingly hostile to data transmission. The sources of packet loss hav= e grown legion=2C "traffic shapers"=2C congested links to the subscibers=2C= overloaded wifi bands=2C content monitor that can not keep up=2C rate caps= on mobile phone data usage=2C the list goes on and on. Complicating this is the increase in data rates and file sizes= which allow many more packets and their acknowledgements to be inflight be= tween two endpoints at once. We as a community must move along the prolife= ration of SACK and if we are going to do that we should fix the underlying issues in the protocol first. All current TCP recovery implementations are based on guessing the state at= the remote node. No recovery mechanism currently in place gives sufficien= t weight to in-flight or delayed messages. This problem can be solved easi= ly by transmitting a token related to the transmission state in each out going message which the destination = node will return on its next transmission. By having this token I know all= of the messages the far end should have seen and if I do not have an ackno= wledgement for any of them=2C I know that those need to be retransmitted. To start this discussion I have created an I-D "Highly Efficient Selective = Acknowledgement (SACK) for TCP" - draft-sabatini-tcp-sack-00 which I hope w= ill be accepted by the working group. Note that even though no TCP implementation exists=2C the HDLC version I cr= eated for the financial community has 20 years of working history all over = the world and the capability for being 100% efficient right down to one err= or per message no matter what the data rate or the delay (including multiple satellite hops). Tony=20 = --_3f1272e0-cc2b-48ad-b6f0-2397069a8a48_ Content-Type: text/html; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable
Richard -

First off thank you for taking the time to read the draft.=

With respect to the option numbers=2C as I wrote to the tcpm chairs= - I can not presume in a draft what option numbers will be assigned in a s= tandard but I needed a place holder for illustration. =3B I therefore u= sed the ones in SACK to show how they relate and to guarantee in the final = document they will be corrected to the assigned appropriate values.

= With respect to sending multiple acks and what is contained in those acks -= in IPv4 the TCP options fields are very limited and I have had to steal fo= ur more of them. =3B Fortunately SACK uses 8 byte pairs so I have not d= ecreased the number of pairs sent=2C but in order to render a complete stat= e I may need to send a multiple number of ACKs. =3B Like SACK=2C I have= to lay out at least a framework of what pairs get transmitted in what orde= r with how many repeats. =3B Like SACK the first pairs have to be any b= locks which have changed sizes since my last acknowledgement. =3B The n= ext most important block is the oldest since it defines the oldest=2C and t= hus the most important=2C missing segment. =3B The rest o the blocks ar= e then transmitted=2C oldest first. =3B Repeating this sequence of ACKs= is an homage to any of the various TCP fast recovery mechanisms=2C they ar= e right in that you are as likely to lose the ACK as you are to lose the me= ssage.



With respect to the delay returning a received token = - we as a group take too little time understanding why a protocol works.&nb= sp=3B TCP is tolerant of out of sequence packet arrivals because it saves s= egments beyond the one it expects and will fill in when they arrive out of = order. =3B Since the proposed change will tightly monitor missing segme= nts=2C if the far end updated its returned token immediately then any out o= f order packets would be considered permanently lost. =3B By delaying t= he update of the returned token I allow time for the out of order packet to= arrive and be processed. =3B Yes this hurts efficiency and yes the del= ay should be set to zero in a point to point environment but having this fe= ature means that the modified protocol has all of the robustness of the ori= ginal. =3B A seconday important feature of the delay is that it insures= that all of the ACKs that comprise the receiver state have been received a= nd processed.

Tony

Anthony Sabatini
200 =3BWest 20th S= treet
Apt. 1216
New York=2C NY 10011

Phone: (212) 867-7179
= Mobile: (917) 224-8388

 =3B


From: rs@netapp.com
To: tsabat= ini@hotmail.com=3B tcpm@ietf.org=3B draft-sack@tsabatini.com
Subject: RE= : [tcpm] TCP Error recovery and efficiency
Date: Wed=2C 11 Jul 2012 11:3= 9:38 +0000

 =3B=

Hi Anthony=2C

 =3B

first off=2C a quick observation: Modifying existing options=2C part= icularly their length or encoding=2C as you propose in that draft=2C will p= robably not work=3B there are many devices=2C which may react undefined (ignore th= e extended SACK permitted=2C accept it in the old way=2C=85) which wouldn= =92t be compatible.

 =3B

Also=2C certain middleboxes do modify the contents of regular SACK b= locks (rewriting TCP sequence numbers)=2C and they would definitely interac= t very badly when a well-defined tcp option suddenly has different semantics.

 =3B

 =3B

Second=2C I try to understand your language around the token=3B I=92= m not sure I follow the reasoning around artificially delaying ACK / SACK u= pdates on the receiver.

 =3B

A worked out example as appendix would do good (showing which fields= contain what when=2C and the exchange of these tokens between sender and r= eceiver).

 =3B

Third=2C the draft appears to propose sending multiple ACKs in respo= nse to a (receiver delay timeout)=2C with varying SACK option contents betw= een these ACKs (all having the same TCP header fields)? The purpose of these A= CKs being to clock out the missing segments from the sender=2C if I read th= is correctly? However=2C no precaution is there to prevent a burst or train= of ACKs (which would cause a transmit spike on the sender side)=2C correct?

 =3B

I=92m not quite clear on the sender behavior=3B the text seems to im= ply that the SACK scoreboard should be deleted for each received ACK/SACK= =2C and reconstructed anew?

 =3B

The new =93token=94 fields serving as secondary ACK counter to only = resend those bytes that were not retransmitted by the time the receiver got= that token?

 =3B

 =3B

 =3B

 =3B

Reconstructing the order in which packets arrive at the destination = is=2C IMHO=2C a valuable goal to optimize the error recovery of TCP SACK.

 =3B

 =3B

So=2C to summarize=2C there are two main ideas in this draft:=

 =3B

a) =3B =3B =3B&nb= sp=3B =3B Use of a deterministically reflected token=2C that serves to reconstruct = the exact order of packets received at the receiver (including retransmissions).

 =3B

b) =3B =3B =3B&nb= sp=3B =3B Sending of unsolicidated ACKs based on a short timeout (1/4 RTT) in order= to pull lost/delayed data from the sender without reverting to a sender-side RTO

 =3B

c) =3B =3B =3B&nb= sp=3B =3B =3B Modification of a well-known TCP option with new data fields

 =3B

 =3B

Please confirm that I have properly summarized your ideas in that dr= aft.

 =3B

I=92m very much in favor of idea a). However=2C I still believe that= the proper approach would be to synergistically use Timestamp together wit= h SACK to achieve the same (which requires =93only=94 a semantic/behavior change = of the TS field in the presence of the SACK option=2C but doesn=92t alter t= he encoding/interpretation per se). With that modification to the Timestamp= semantics=2C the timestamps can serve a =93tokens=94 delivering the information you seek to better reconstruct the receiver sta= te at the sender. Please have a look at http://tools.ietf.org/id/draft-sche= ffenegger-tcpm-timestamp-negotiation-03.txt.

 =3B

I=92m not so sure about idea b) since that would appear to violate t= he conservation of packets rule =96 even under optimal conditions (no packe= t loss=2C only very variable packet delay/reordering)=2C a higher rate of ACKs would= be sent back=2C potentially completely unnecessary.

 =3B

With c) =3B - this will break this proposal. Please use experime= ntal options in the draft (and newly assigned option numbers if adopted) so= to avoid compatibility issues with existing deployed hardware in the internet.

 =3B

Will you be present in Vancouver?

 =3B

Best regards=2C

 =3B

Richard = Scheffenegger


 =3B=

From: tcpm-bounces@ietf.org [ma= ilto:tcpm-bounces@ietf.org] On Behalf Of Anthony Sabatini
Sent: Montag=2C 09. Juli 2012 05:55
To: tcpm@ietf.org
Subject: [tcpm] TCP Error recovery and efficiency

 =3B

All -

While data communications was constantly improving=2C due mostly to the wid= e adoption of fiber optics=2C the fact that TCP had an inferior recovery me= chanism was moot as it came into play less and less often. =3B Sadly th= ose days are now behind us and we face a world increasingly hostile to data transmission. =3B The sources of packet l= oss have grown legion=2C "traffic shapers"=2C congested links to the subsci= bers=2C overloaded wifi bands=2C content monitor that can not keep up=2C ra= te caps on mobile phone data usage=2C the list goes on and on. =3B Complicating this is the increase in data rates and fil= e sizes which allow many more packets and their acknowledgements to be infl= ight between two endpoints at once. =3B We as a community must move alo= ng the proliferation of SACK and if we are going to do that we should fix the underlying issues in the protocol first.

All current TCP recovery implementations are based on guessing the state at= the remote node. =3B No recovery mechanism currently in place gives su= fficient weight to in-flight or delayed messages. =3B This problem can = be solved easily by transmitting a token related to the transmission state in each out going message which the destination = node will return on its next transmission. =3B By having this token I k= now all of the messages the far end should have seen and if I do not have a= n acknowledgement for any of them=2C I know that those need to be retransmitted.

To start this discussion I have created an I-D "Highly Efficient Selective = Acknowledgement (SACK) for TCP" - draft-sabatini-tcp-sack-00 which I hope w= ill be accepted by the working group.

Note that even though no TCP implementation exists=2C the HDLC version I cr= eated for the financial community has 20 years of working history all over = the world and the capability for being 100% efficient right down to one err= or per message no matter what the data rate or the delay (including multiple satellite hops).

Tony

= --_3f1272e0-cc2b-48ad-b6f0-2397069a8a48_-- From tsabatini@hotmail.com Wed Jul 11 09:46:59 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 11F5011E80C6 for ; Wed, 11 Jul 2012 09:46:59 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.165 X-Spam-Level: X-Spam-Status: No, score=-2.165 tagged_above=-999 required=5 tests=[AWL=0.433, BAYES_00=-2.599, HTML_MESSAGE=0.001] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id EAqinLBZbaq4 for ; Wed, 11 Jul 2012 09:46:56 -0700 (PDT) Received: from bay0-omc4-s5.bay0.hotmail.com (bay0-omc4-s5.bay0.hotmail.com [65.54.190.207]) by ietfa.amsl.com (Postfix) with ESMTP id 9D07F21F85EF for ; Wed, 11 Jul 2012 09:46:56 -0700 (PDT) Received: from BAY158-W23 ([65.54.190.199]) by bay0-omc4-s5.bay0.hotmail.com with Microsoft SMTPSVC(6.0.3790.4675); Wed, 11 Jul 2012 09:47:26 -0700 Message-ID: Content-Type: multipart/alternative; boundary="_0e5784cc-26d0-4612-b41d-cf85ad3b537f_" X-Originating-IP: [74.64.96.39] From: Anthony Sabatini To: , tcpm , Date: Wed, 11 Jul 2012 12:47:26 -0400 Importance: Normal In-Reply-To: References: , <012C3117EDDB3C4781FD802A8C27DD4F080D40D3@SACEXCMBX02-PRD.hq.netapp.com>, MIME-Version: 1.0 X-OriginalArrivalTime: 11 Jul 2012 16:47:26.0989 (UTC) FILETIME=[DA820BD0:01CD5F84] X-Mailman-Approved-At: Thu, 12 Jul 2012 08:52:11 -0700 Subject: Re: [tcpm] TCP Error recovery and efficiency (updated reply) X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Jul 2012 16:46:59 -0000 --_0e5784cc-26d0-4612-b41d-cf85ad3b537f_ Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable Sorry - previous message sent in error. Richard - First off thank you for taking the time to read the draft. With respect to the option numbers=2C as I wrote to the tcpm chairs - I can= not presume in a draft what option numbers will be assigned in a standard = but I needed a place holder for illustration. I therefore used the ones in= SACK to show how they relate and to guarantee in the final document they w= ill be corrected to the assigned appropriate values. With respect to sending multiple ACKs and what is contained in those ACKs -= in IPv4 the TCP options fields are very limited and I have had to steal fo= ur more of them. Fortunately SACK uses 8 byte pairs so I have not decrease= d the number of pairs sent=2C but in order to render a complete state I may= need to send a multiple number of ACKs. Like SACK=2C I have to lay out at= least a framework of what pairs get transmitted in what order with how man= y repeats. Like SACK the first pairs have to be any blocks which have chan= ged sizes since my last acknowledgement. The next most important block is = the oldest since it defines the oldest=2C and thus the most important=2C mi= ssing segment. The rest of the blocks are then transmitted=2C oldest first= . Repeating this sequence of ACKs is an homage to any of the various TCP f= ast recovery mechanisms=2C they are right in that you are as likely to lose= the ACK as you are to lose the message. Two things I am sure of=2C 1) som= eone is going to do a masters thesis on what is the optimal order to send t= he blocks in and how many repeats are best and 2) someone is going to sugge= st compressing the segment limits in the TCP options (and we might want to = make that change in this standard). Note that the 1/4 RTT time was a resta= tement of a SACK philosophy to which I personally find of minimal value. A= n immediate repeat of the ACK set insures against transient packet loss=2C = the 1/4 RTT only adds robustness where the link drops a burst of segments s= uch as happens in an overloaded content inspector. Note that unless you ha= ve a SACK situation=2C normal TCP acknowledgement rules apply=2C even if yo= u have a couple of SACK segments=2C normal TCP acknowledgement rules apply.= In both cases the only thing that changed was there are options in the AC= K. It is only when we are fairly far down into the weeds that either the = original SACK or my version changes anything and mostly it is to generate m= ultiple ACKS so we can get the state out. Thus I change nothing and violat= e no rules on a clean link or even a slightly dirty one. The violation tha= t occurs is in the original=2C approved=2C SACK RFC. With respect to timestamps - having written many network drivers with many= operating systems=2C I have tried to avoid them on a driver level if possi= ble as calling one device driver from within another can lead to unintended= consequences. For example many clocks lock a call when they are processin= g a time increment so that yo do not get a ripple effect of half an updated= timestamp. This lead to an all block at the network level for that period= of time which is not good. It also means that you have to timestamp every= outgoing message in the transmit queue. By using as your token the curren= t index into the transmitted segment queue you can immediately work forward= in that queue to see what segments have been retransmitted but not yet rec= eived=2C regardless of timestamp. With respect to the scoreboard - the acknowledgement in the TCP header form= s a floor for the scoreboard=2C all entries with segment finishes prior to = that are released when this value is updated (others are retained as not ye= t complete). With respect to selective acknowledgements=2C a SACK is perma= nent=2C it creates a begin/end pair in the scoreboard=2C as subsequent SACK= s come in entries may be inserted=2C added to the end=2C or merged with one= or more of the existing entries. The holes between these entries represen= t the segments (and between the floor and the first) that must be retransmi= tted. Likewise in the reciever the TCP acknowledgement rules=2C the receiv= er must remove all SACK blocks on its side which are now below the updated = floor. With respect to the delay returning a received token - we as a group take t= oo little time understanding why a protocol works. TCP is tolerant of out = of sequence packet arrivals because it saves segments beyond the one it exp= ects and will fill in when they arrive out of order. Since the proposed ch= ange will tightly monitor missing segments=2C if the far end updated its re= turned token immediately then any out of order packets would be considered = permanently lost. By delaying the update of the returned token I allow tim= e for the out of order packet to arrive and be processed. Yes this hurts e= fficiency and yes the delay should be set to zero in a point to point envir= onment but having this feature means that the modified protocol has all of = the robustness of the original. A seconday important feature of the delay = is that it insures that all of the ACKs that comprise the receiver state ha= ve been received and processed. With respect to Vancouver - I will go if anyone cares about this - since yo= u do=2C I could be easily convinced. I have asked the tcpm chairs for 15 m= inutes to explain myself but have not heard from them as of yet. Note for the record - I am probably not the best person to rewrite this dra= ft as I am too close to the issue=2C if anyone wants to co-author it and r= ewrite it please let me know=2C we will then=2C at least=2C have one other = person who understands what I am trying to express. Tony Anthony Sabatini 200 West 20th Street Apt. 1216 New York=2C NY 10011 Phone: (212) 867-7179 Mobile: (917) 224-8388 =20 From: rs@netapp.com To: tsabatini@hotmail.com=3B tcpm@ietf.org=3B draft-sack@tsabatini.com Subject: RE: [tcpm] TCP Error recovery and efficiency Date: Wed=2C 11 Jul 2012 11:39:38 +0000 =20 Hi Anthony=2C =20 first off=2C a quick observation: Modifying existing options=2C particularl= y their length or encoding=2C as you propose in that draft=2C will probably not work=3B there are many devices=2C which may react undefined (ignore th= e extended SACK permitted=2C accept it in the old way=2C=85) which wouldn= =92t be compatible. =20 Also=2C certain middleboxes do modify the contents of regular SACK blocks (= rewriting TCP sequence numbers)=2C and they would definitely interact very badly when a well-defined tcp option suddenly has different semantics. =20 =20 Second=2C I try to understand your language around the token=3B I=92m not s= ure I follow the reasoning around artificially delaying ACK / SACK updates on the receiver. =20 A worked out example as appendix would do good (showing which fields contai= n what when=2C and the exchange of these tokens between sender and receiver= ). =20 Third=2C the draft appears to propose sending multiple ACKs in response to = a (receiver delay timeout)=2C with varying SACK option contents between these ACKs (all having the same TCP header fields)? The purpose of these A= CKs being to clock out the missing segments from the sender=2C if I read th= is correctly? However=2C no precaution is there to prevent a burst or train= of ACKs (which would cause a transmit spike on the sender side)=2C correct? =20 I=92m not quite clear on the sender behavior=3B the text seems to imply tha= t the SACK scoreboard should be deleted for each received ACK/SACK=2C and reconstructed anew? =20 The new =93token=94 fields serving as secondary ACK counter to only resend = those bytes that were not retransmitted by the time the receiver got that token? =20 =20 =20 =20 Reconstructing the order in which packets arrive at the destination is=2C I= MHO=2C a valuable goal to optimize the error recovery of TCP SACK. =20 =20 So=2C to summarize=2C there are two main ideas in this draft: =20 a) =20 Use of a deterministically reflected token=2C that serves to reconstruct th= e exact order of packets received at the receiver (including retransmissions).=20 =20 b) =20 Sending of unsolicidated ACKs based on a short timeout (1/4 RTT) in order t= o pull lost/delayed data from the sender without reverting to a sender-side RTO =20 c) =20 Modification of a well-known TCP option with new data fields =20 =20 Please confirm that I have properly summarized your ideas in that draft. =20 I=92m very much in favor of idea a). However=2C I still believe that the pr= oper approach would be to synergistically use Timestamp together with SACK to achieve the same (which requires =93only=94 a semantic/behavior change = of the TS field in the presence of the SACK option=2C but doesn=92t alter t= he encoding/interpretation per se). With that modification to the Timestamp= semantics=2C the timestamps can serve a =93tokens=94 delivering the information you seek to better reconstruct the receiver sta= te at the sender. Please have a look at http://tools.ietf.org/id/draft-sche= ffenegger-tcpm-timestamp-negotiation-03.txt. =20 I=92m not so sure about idea b) since that would appear to violate the cons= ervation of packets rule =96 even under optimal conditions (no packet loss= =2C only very variable packet delay/reordering)=2C a higher rate of ACKs would= be sent back=2C potentially completely unnecessary. =20 With c) - this will break this proposal. Please use experimental options i= n the draft (and newly assigned option numbers if adopted) so to avoid compatibility issues with existing deployed hardware in the internet. =20 Will you be present in Vancouver? =20 Best regards=2C =20 Richard Scheffenegger =20 From: tcpm-bounces@ietf.org [mailto:tcpm-bounces@ietf.org] On Behalf Of Anthony Sabatini Sent: Montag=2C 09. Juli 2012 05:55 To: tcpm@ietf.org Subject: [tcpm] TCP Error recovery and efficiency =20 All - While data communications was constantly improving=2C due mostly to the wid= e adoption of fiber optics=2C the fact that TCP had an inferior recovery me= chanism was moot as it came into play less and less often. Sadly those day= s are now behind us and we face a world increasingly hostile to data transmission. The sources of packet loss hav= e grown legion=2C "traffic shapers"=2C congested links to the subscibers=2C= overloaded wifi bands=2C content monitor that can not keep up=2C rate caps= on mobile phone data usage=2C the list goes on and on. Complicating this is the increase in data rates and file sizes= which allow many more packets and their acknowledgements to be inflight be= tween two endpoints at once. We as a community must move along the prolife= ration of SACK and if we are going to do that we should fix the underlying issues in the protocol first. All current TCP recovery implementations are based on guessing the state at= the remote node. No recovery mechanism currently in place gives sufficien= t weight to in-flight or delayed messages. This problem can be solved easi= ly by transmitting a token related to the transmission state in each out going message which the destination = node will return on its next transmission. By having this token I know all= of the messages the far end should have seen and if I do not have an ackno= wledgement for any of them=2C I know that those need to be retransmitted. To start this discussion I have created an I-D "Highly Efficient Selective = Acknowledgement (SACK) for TCP" - draft-sabatini-tcp-sack-00 which I hope w= ill be accepted by the working group. Note that even though no TCP implementation exists=2C the HDLC version I cr= eated for the financial community has 20 years of working history all over = the world and the capability for being 100% efficient right down to one err= or per message no matter what the data rate or the delay (including multiple satellite hops). Tony=20 = --_0e5784cc-26d0-4612-b41d-cf85ad3b537f_ Content-Type: text/html; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable
Sorry - previous message sent in error.

Richard -

First off thank you for taking the time to read the draft.=

With respect to the option numbers=2C as I wrote to the tcpm chairs= - I can not presume in a draft what option numbers will be assigned in a s= tandard but I needed a place holder for illustration. =3B I therefore u= sed the ones in SACK to show how they relate and to guarantee in the final = document they will be corrected to the assigned appropriate values.

= With respect to sending multiple ACKs and what is contained in those ACKs -= in IPv4 the TCP options fields are very limited and I have had to steal fo= ur more of them. =3B Fortunately SACK uses 8 byte pairs so I have not d= ecreased the number of pairs sent=2C but in order to render a complete stat= e I may need to send a multiple number of ACKs. =3B Like SACK=2C I have= to lay out at least a framework of what pairs get transmitted in what orde= r with how many repeats. =3B Like SACK the first pairs have to be any b= locks which have changed sizes since my last acknowledgement. =3B The n= ext most important block is the oldest since it defines the oldest=2C and t= hus the most important=2C missing segment. =3B The rest of the blocks a= re then transmitted=2C oldest first. =3B Repeating this sequence of ACK= s is an homage to any of the various TCP fast recovery mechanisms=2C they a= re right in that you are as likely to lose the ACK as you are to lose the m= essage. =3B Two things I am sure of=2C 1) someone is going to do a mast= ers thesis on what is the optimal order to send the blocks in and how many = repeats are best and 2) someone is going to suggest compressing the segment= limits in the TCP options (and we might want to make that change in this s= tandard). =3B Note that the 1/4 RTT time was a restatement of a SACK ph= ilosophy to which I personally find of minimal value. =3B An immediate = repeat of the ACK set insures against transient packet loss=2C the 1/4 RTT = only adds robustness where the link drops a burst of segments such as happe= ns in an overloaded content inspector. =3B Note that unless you have a = SACK situation=2C normal TCP acknowledgement rules apply=2C even if you hav= e a couple of SACK segments=2C normal TCP acknowledgement rules apply. = =3B In both cases the only thing that changed was there are options in the = ACK. =3B It is =3B only when we are fairly far down into the weeds = that either the original SACK or my version changes anything and mostly it = is to generate multiple ACKS so we can get the state out. =3B Thus I ch= ange nothing and violate no rules on a clean link or even a slightly dirty = one. =3B The violation that occurs is in the original=2C approved=2C SA= CK RFC.

 =3BWith respect to timestamps - having written many net= work drivers with many operating systems=2C I have tried to avoid them on a= driver level if possible as calling one device driver from within another = can lead to unintended consequences. =3B For example many clocks lock a= call when they are processing a time increment so that yo do not get a rip= ple effect of half an updated timestamp. =3B This lead to an all block = at the network level for that period of time which is not good. =3B It = also means that you have to timestamp every outgoing message in the transmi= t queue. =3B By using as your token the current index into the transmit= ted segment queue you can immediately work forward in that queue to see wha= t segments have been retransmitted but not yet received=2C regardless of ti= mestamp.

With respect to the scoreboard - the acknowledgement in the= TCP header forms a floor for the scoreboard=2C all entries with segment fi= nishes prior to that are released when this value is updated (others are re= tained as not yet complete). =3B With respect to selective acknowledgem= ents=2C a SACK is permanent=2C it creates a begin/end pair in the scoreboar= d=2C as subsequent SACKs come in entries may be inserted=2C added to the en= d=2C or merged with one or more of the existing entries. =3B The holes = between these entries represent the segments (and between the floor and the= first) that must be retransmitted. =3B Likewise in the reciever the TC= P acknowledgement rules=2C the receiver must remove all SACK blocks on its = side which are now below the updated floor.

With respect to the dela= y returning a received token - we as a group take too little time understan= ding why a protocol works. =3B TCP is tolerant of out of sequence packe= t arrivals because it saves segments beyond the one it expects and will fil= l in when they arrive out of order. =3B Since the proposed change will = tightly monitor missing segments=2C if the far end updated its returned tok= en immediately then any out of order packets would be considered permanentl= y lost. =3B By delaying the update of the returned token I allow time f= or the out of order packet to arrive and be processed. =3B Yes this hur= ts efficiency and yes the delay should be set to zero in a point to point e= nvironment but having this feature means that the modified protocol has all= of the robustness of the original. =3B A seconday important feature of= the delay is that it insures that all of the ACKs that comprise the receiv= er state have been received and processed.

With respect to Vancouver= - I will go if anyone cares about this - since you do=2C I could be easily= convinced. =3B I have asked the tcpm chairs for 15 minutes to explain = myself but have not heard from them as of yet.

Note for the record -= I am probably not the best person to rewrite this draft as I am too close = to the issue=2C =3B if anyone wants to co-author it and rewrite it plea= se let me know=2C we will then=2C at least=2C have one other person who und= erstands what I am trying to express.

Tony

Anthony Sabatini200 =3BWest 20th Street
Apt. 1216
New York=2C NY 10011

P= hone: (212) 867-7179
Mobile: (917) 224-8388

 =3B


<= div>
Fro= m: rs@netapp.com
To: tsabatini@hotmail.com=3B tcpm@ietf.org=3B draft-sac= k@tsabatini.com
Subject: RE: [tcpm] TCP Error recovery and efficiencyDate: Wed=2C 11 Jul 2012 11:39:38 +0000

 =3B=

Hi Anthony=2C

 =3B

first off=2C a quick observation: Modifying existing options=2C part= icularly their length or encoding=2C as you propose in that draft=2C will p= robably not work=3B there are many devices=2C which may react undefined (ignore th= e extended SACK permitted=2C accept it in the old way=2C=85) which wouldn= =92t be compatible.

 =3B

Also=2C certain middleboxes do modify the contents of regular SACK b= locks (rewriting TCP sequence numbers)=2C and they would definitely interac= t very badly when a well-defined tcp option suddenly has different semantics.

 =3B

 =3B

Second=2C I try to understand your language around the token=3B I=92= m not sure I follow the reasoning around artificially delaying ACK / SACK u= pdates on the receiver.

 =3B

A worked out example as appendix would do good (showing which fields= contain what when=2C and the exchange of these tokens between sender and r= eceiver).

 =3B

Third=2C the draft appears to propose sending multiple ACKs in respo= nse to a (receiver delay timeout)=2C with varying SACK option contents betw= een these ACKs (all having the same TCP header fields)? The purpose of these A= CKs being to clock out the missing segments from the sender=2C if I read th= is correctly? However=2C no precaution is there to prevent a burst or train= of ACKs (which would cause a transmit spike on the sender side)=2C correct?

 =3B

I=92m not quite clear on the sender behavior=3B the text seems to im= ply that the SACK scoreboard should be deleted for each received ACK/SACK= =2C and reconstructed anew?

 =3B

The new =93token=94 fields serving as secondary ACK counter to only = resend those bytes that were not retransmitted by the time the receiver got= that token?

 =3B

 =3B

 =3B

 =3B

Reconstructing the order in which packets arrive at the destination = is=2C IMHO=2C a valuable goal to optimize the error recovery of TCP SACK.

 =3B

 =3B

So=2C to summarize=2C there are two main ideas in this draft:=

 =3B

a) =3B =3B =3B&nb= sp=3B =3B Use of a deterministically reflected token=2C that serves to reconstruct = the exact order of packets received at the receiver (including retransmissions).

 =3B

b) =3B =3B =3B&nb= sp=3B =3B Sending of unsolicidated ACKs based on a short timeout (1/4 RTT) in order= to pull lost/delayed data from the sender without reverting to a sender-side RTO

 =3B

c) =3B =3B =3B&nb= sp=3B =3B =3B Modification of a well-known TCP option with new data fields

 =3B

 =3B

Please confirm that I have properly summarized your ideas in that dr= aft.

 =3B

I=92m very much in favor of idea a). However=2C I still believe that= the proper approach would be to synergistically use Timestamp together wit= h SACK to achieve the same (which requires =93only=94 a semantic/behavior change = of the TS field in the presence of the SACK option=2C but doesn=92t alter t= he encoding/interpretation per se). With that modification to the Timestamp= semantics=2C the timestamps can serve a =93tokens=94 delivering the information you seek to better reconstruct the receiver sta= te at the sender. Please have a look at http://tools.ietf.org/id/draft-sche= ffenegger-tcpm-timestamp-negotiation-03.txt.

 =3B

I=92m not so sure about idea b) since that would appear to violate t= he conservation of packets rule =96 even under optimal conditions (no packe= t loss=2C only very variable packet delay/reordering)=2C a higher rate of ACKs would= be sent back=2C potentially completely unnecessary.

 =3B

With c) =3B - this will break this proposal. Please use experime= ntal options in the draft (and newly assigned option numbers if adopted) so= to avoid compatibility issues with existing deployed hardware in the internet.

 =3B

Will you be present in Vancouver?

 =3B

Best regards=2C

 =3B

Richard = Scheffenegger


 =3B=

From: tcpm-bounces@ietf.org [ma= ilto:tcpm-bounces@ietf.org] On Behalf Of Anthony Sabatini
Sent: Montag=2C 09. Juli 2012 05:55
To: tcpm@ietf.org
Subject: [tcpm] TCP Error recovery and efficiency

 =3B

All -

While data communications was constantly improving=2C due mostly to the wid= e adoption of fiber optics=2C the fact that TCP had an inferior recovery me= chanism was moot as it came into play less and less often. =3B Sadly th= ose days are now behind us and we face a world increasingly hostile to data transmission. =3B The sources of packet l= oss have grown legion=2C "traffic shapers"=2C congested links to the subsci= bers=2C overloaded wifi bands=2C content monitor that can not keep up=2C ra= te caps on mobile phone data usage=2C the list goes on and on. =3B Complicating this is the increase in data rates and fil= e sizes which allow many more packets and their acknowledgements to be infl= ight between two endpoints at once. =3B We as a community must move alo= ng the proliferation of SACK and if we are going to do that we should fix the underlying issues in the protocol first.

All current TCP recovery implementations are based on guessing the state at= the remote node. =3B No recovery mechanism currently in place gives su= fficient weight to in-flight or delayed messages. =3B This problem can = be solved easily by transmitting a token related to the transmission state in each out going message which the destination = node will return on its next transmission. =3B By having this token I k= now all of the messages the far end should have seen and if I do not have a= n acknowledgement for any of them=2C I know that those need to be retransmitted.

To start this discussion I have created an I-D "Highly Efficient Selective = Acknowledgement (SACK) for TCP" - draft-sabatini-tcp-sack-00 which I hope w= ill be accepted by the working group.

Note that even though no TCP implementation exists=2C the HDLC version I cr= eated for the financial community has 20 years of working history all over = the world and the capability for being 100% efficient right down to one err= or per message no matter what the data rate or the delay (including multiple satellite hops).

Tony

= --_0e5784cc-26d0-4612-b41d-cf85ad3b537f_-- From nanditad@google.com Thu Jul 12 11:26:41 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 83EF511E80E8 for ; Thu, 12 Jul 2012 11:26:41 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.976 X-Spam-Level: X-Spam-Status: No, score=-102.976 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tuCuVzmNQ0mr for ; Thu, 12 Jul 2012 11:26:40 -0700 (PDT) Received: from mail-qc0-f172.google.com (mail-qc0-f172.google.com [209.85.216.172]) by ietfa.amsl.com (Postfix) with ESMTP id 9801611E80D1 for ; Thu, 12 Jul 2012 11:26:40 -0700 (PDT) Received: by qcac10 with SMTP id c10so1874193qca.31 for ; Thu, 12 Jul 2012 11:27:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:date:message-id:subject:from:to:cc:content-type :x-system-of-record; bh=1Qwvc2eB1AD4+rlpcnDX2IhQt04AO2NLO4DEx6TIUOY=; b=IpKwqK2a2/9Rm+LYVNKlvEoS9oX2Z1scqO3a8OPWDmdIPRjjBkZsi1K7q0dQORMroy ke+kAQbnyPIN7js38vcrGVXSQw78/ij+yk4w88U9HoCWL7GPU7OJN04Pg9dvVFlu604s B+xs7iF6BMF/FMFmE8ZvGPFYX9T59StfacSFZx4Q3w+U5UbAgaokzILyEWCQ//i9Wdr8 RscIau87qDKgQR2i1O7a/taBCytd0VtjdQru/QbNr9gmuSXVZ0k9Hl+y72/FgjmgOE0j 8QIrRX/GzU+f2oUAPGf+H0WtG+rZLj6IYXWJtcH/cLmYNm3/j1kxqdjwoC5lSg8kOj9C 0RgQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:date:message-id:subject:from:to:cc:content-type :x-system-of-record:x-gm-message-state; bh=1Qwvc2eB1AD4+rlpcnDX2IhQt04AO2NLO4DEx6TIUOY=; b=lSFT7wpUH2CNatQhOarvV2CBKN2l6R6S8jKIdZIsgqIBDU7aOEcVM06NpzFc7Re6k7 Wj5f1n0mTvXksa5yYv+nUcFzP6j30wnnYHVp/qk1yQg9pKEmnDmMUZ8WZ2GUS6q3H2/J 4mRhwucVLjuaK9oMG8dbSoZVSVDmO4Ag8z2xZ+y6lETzE/UD9wls60/mF+0udUKgkQN2 m6PdrFji/UxUq6sBnlESORF656wy5fJ5Bn+z5KyUjbSbdpdGSQ9kEOSIX+USbVptLfc5 pZwCkzmSoy8kxxwee/9anHC034dzUGHGv3TSX+PWwddQAwsxcSAWXHZAFkz7vwc61PGp f5rw== Received: by 10.60.169.134 with SMTP id ae6mr55874109oec.55.1342117633959; Thu, 12 Jul 2012 11:27:13 -0700 (PDT) MIME-Version: 1.0 Received: by 10.60.169.134 with SMTP id ae6mr55874075oec.55.1342117633663; Thu, 12 Jul 2012 11:27:13 -0700 (PDT) Received: by 10.182.39.131 with HTTP; Thu, 12 Jul 2012 11:27:11 -0700 (PDT) Date: Thu, 12 Jul 2012 11:27:11 -0700 Message-ID: From: Nandita Dukkipati To: tcpm@ietf.org Content-Type: multipart/alternative; boundary=bcaec550b36ccd21e504c4a61c3f X-System-Of-Record: true X-Gm-Message-State: ALoCoQldvGoKAi2kMSs8TbOQUNiNKuIt74Zad4Hr/hVlCWWcSCLywElo5SQWXvfUzeb+qzZy2/Eccwr2eSt88jBNcQHOn4rU2JjBxT+BA24YTnrmts9+bz1uY/ROcjl9BSa0fymoCqyHm3uVtKSmjJ8pH+dMYrQbigs0f1cv54s2/JtGBx58nSs= Cc: Matt Mathis Subject: [tcpm] TCP Loss Probe (TLP): An Algorithm for Fast Recovery of Tail Losses X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Jul 2012 18:26:41 -0000 --bcaec550b36ccd21e504c4a61c3f Content-Type: text/plain; charset=ISO-8859-1 Hi! We have a new draft on an algorithm called TLP (TCP Loss Probe). We will have a presentation on TLP at upcoming Vancouver meeting. Below is the abstract and the Internet Draft. TCP Loss Probe (TLP): An Algorithm for Fast Recovery of Tail Losses http://tools.ietf.org/html/draft-dukkipati-tcpm-tcp-loss-probe-00 Abstract Retransmission timeouts are detrimental to application latency, especially for short transfers such as Web transactions where timeouts can often take longer than all of the rest of a transaction. The primary cause of retransmission timeouts are lost segments at the tail of transactions. This document describes an experimental algorithm for TCP to quickly recover lost segments at the end of transactions or when an entire window of data or acknowledgments are lost. TCP Loss Probe (TLP) is a sender-only algorithm that allows the transport to recover tail losses through fast recovery as opposed to lengthy retransmission timeouts. If a connection is not receiving any acknowledgments for a certain period of time, TLP transmits the last unacknowledged segment (loss probe). In the event of a tail loss in the original transmissions, the acknowledgment from the loss probe triggers SACK/FACK based fast recovery. TLP effectively avoids long timeouts and thereby improves TCP performance. Feedback and ideas are welcome! Nandita --bcaec550b36ccd21e504c4a61c3f Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
Hi!

We have a new draft on a= n algorithm called TLP (TCP Loss Probe). We will have a presentation on TLP= at upcoming Vancouver =A0meeting.=A0Below is the abstract and the Internet= Draft.

=
TCP Loss Probe (= TLP): An Algorithm for Fast Recovery of Tail Losses

Abstra= ct
Retransm= ission timeouts are detrimental to application latency, especially for shor= t transfers such as Web transactions where timeouts can often take longer t= han all of the rest of a transaction. =A0The primary cause of retransmissio= n timeouts are lost segments at the tail of transactions. =A0This document = describes an experimental algorithm for TCP to quickly recover lost segment= s at the end of transactions or when an entire window of data or acknowledg= ments are lost. =A0TCP Loss Probe (TLP) is a sender-only algorithm that all= ows the transport to recover tail losses through fast recovery as opposed t= o lengthy retransmission timeouts. =A0If a connection is not receiving any = acknowledgments for a certain period of time, TLP transmits the last unackn= owledged segment (loss probe). =A0In the event of a tail loss in the origin= al transmissions, the acknowledgment from the loss probe triggers SACK/FACK= based fast recovery. =A0TLP effectively avoids long timeouts and thereby i= mproves TCP performance.

Feedback and ideas are= welcome!
N= andita
--bcaec550b36ccd21e504c4a61c3f-- From mattmathis@google.com Fri Jul 13 15:03:44 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1DE8811E80BB for ; Fri, 13 Jul 2012 15:03:44 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.577 X-Spam-Level: X-Spam-Status: No, score=-102.577 tagged_above=-999 required=5 tests=[AWL=0.400, BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FK3lyKpHyZYz for ; Fri, 13 Jul 2012 15:03:43 -0700 (PDT) Received: from mail-we0-f172.google.com (mail-we0-f172.google.com [74.125.82.172]) by ietfa.amsl.com (Postfix) with ESMTP id B888811E8097 for ; Fri, 13 Jul 2012 15:03:42 -0700 (PDT) Received: by weyu54 with SMTP id u54so2669498wey.31 for ; Fri, 13 Jul 2012 15:04:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:x-system-of-record; bh=vWgxexsdANNzMxCQ/u7s3WHqDRtErZjRReyVeFv60E0=; b=ZVL6sW9j5J5tIfTcjjkd3wPqd3XgmvEJFkJ+kiRlCEvNs/iAgKftjMeREE+7Tne8Mm WB0PpnNV8EgBEL2a8Ro+S1/87JyVSZygZWBPfmHvCrgsgPJQlSyZWKZ49jOuf25po0jO 0ZkcLRJtOx+FWW2E9GNlgTpPw0JonL2YDKjL1OPAu8qUXfcJIImYOKlRpwImb9Os5vQs N8PoBPaIPM3duhHh9nDQDXCbKD/CHU8QYVoA+2IecsQYELhH+5ESV5Mz+MSIhEXlPHI5 XbW9yMZIiz4llpYPdT7ylp6oKH7rb3YRanNnQZwLPb5cNPtKKDyZJ+tbSRMF8Vsl1KVq sJnQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:x-system-of-record:x-gm-message-state; bh=vWgxexsdANNzMxCQ/u7s3WHqDRtErZjRReyVeFv60E0=; b=T2ycANrdFZ7979826ceLRwwUjHFByUduOQybt7JWSnOxfdS56NAaKA/XuzeDbBuJX0 oInTUrenkeARF3qSNnkkU3PWRP3weYCia7dlMI6J7mBFshMjqb6HL81U8BYyPwxkAc1N l6kOSKTVqy0YZF4gUMyQ9mOsuBXfVtMNOITEy9yOfY/9BgzvoWLMZzzmfXRxn5l/YSBQ H+tfYGO3eG4L96NzwOlhWQk+DvlgF86c1OfPL3eFwPl3XXQNf+Bx5dadVEwRBflJlVib ATvYmHRuKQplqYNDchd2Nw5cZ54YWWplzwwon+Vf4uEDtZ1vexabPEZ/HheNaYyAozSZ ITCg== Received: by 10.180.88.229 with SMTP id bj5mr797635wib.19.1342217059332; Fri, 13 Jul 2012 15:04:19 -0700 (PDT) MIME-Version: 1.0 Received: by 10.180.88.229 with SMTP id bj5mr797620wib.19.1342217059193; Fri, 13 Jul 2012 15:04:19 -0700 (PDT) Received: by 10.216.17.78 with HTTP; Fri, 13 Jul 2012 15:04:19 -0700 (PDT) In-Reply-To: References: Date: Fri, 13 Jul 2012 15:04:19 -0700 Message-ID: From: Matt Mathis To: Yoshifumi Nishida Content-Type: text/plain; charset=ISO-8859-1 X-System-Of-Record: true X-Gm-Message-State: ALoCoQmLxi67117Egu4yK7gycVvyYK8WHLkkmwBmP8QiDk0z9GhU3LauZCA71kn87tX8bXCwXxRI7PWnEuDLVadcMq/b127eyDSQp/z7Plj3SA3FLwLlC0HGtzxWTQciBN4/IuguUfCdPhs2/5E6WIsbpjKt/XZOMoq5tRjYvVtgd2/qRCeK7MU= Cc: "tcpm@ietf.org\"" Subject: Re: [tcpm] Review of draft-ietf-tcpm-proportional-rate-reduction-01 X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Jul 2012 22:03:44 -0000 Sorry I didn't get back on this sooner: I incorporated your changes as suggested, except as detailed below. The new draft will be posted "Real Soon Now." I believe it will be ready to advance. Formal WGLC next? On Tue, Mar 20, 2012 at 10:14 PM, Yoshifumi Nishida wrote: > Hi, I reviewed draft-ietf-tcpm-proportional-rate-reduction-01. > I think this is valid and useful idea. I would like to support publishing it > to do more broad experiments. > Below are my comments on the draft. > Please check and update if you agree with them. > > > Technical Comments > > 1: This is just out of my curiosity and I know there's no major > difference, > but is there any reasons to use Limited Transmit in the examples in > Section 3.1? > I just personally thought these examples might be a bit simpler if you > just use RFC3517 logic only. > (You don't have to count packets transmitted by limited transmit for > pipe calculation) > But, this is a really minor point. Please keep it if you prefer. Perhaps the examples would have been simpler, but PRR is "philosophically aligned" with limited transmit, and it would have been sort of odd not to use it. > 2: I have a question in the pseudo code in Section 3. > > if (pipe > ssthresh) { > > Is this supposed to be like this? > > if (pipe >= ssthresh) { > > Otherwise, if pipe equals to ssthresh, sndcnt will be zero. I think you are correct. The psuedo code matches our actual code and causes us to forfeit an opportunity to send one segment slightly earlier than we do. However, we have chosen not to make this change in the document because it would invalidate our measurement experiment. To preserve the integrity of our experimental process we would either need to remove the results or re-do them. > 3: If the assumption above is correct, I think the example in Page 6: > > PRR > ack# X 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 > pipe: 19 19 18 18 18 17 17 16 16 15 15 14 14 13 13 12 12 11 10 > sent: N N R N N N N N N N N > RB: s s > > will be something like this. Is this correct? Yep, but we didn't change it. > PRR > pipe: 19 19 18 18 17 17 16 16 15 15 14 14 13 13 12 12 11 11 10 > sent: N N R N N N N N N N N > RB: s > > > BTW, is there any reason that there is no cwnd data for PRR in the > figures? cwnd is not used (and made the text more prominent.) > 4: In the following case, > > PRR-CRB > ack# X X X X X X X X X X X X X X X 15 16 17 18 19 > pipe: 19 19 4 4 4 > sent: N N R R R > RB: f f f > > I think PRR_CRB works like this way and this is understandable. > > ack #: 17 18 19 > prr_delivered: 1 2 3 > prr_out: 0 1 2 > limit: 1 1 1 > sndcnt: 1 1 1 > > However, in the following case, I think the parameters change like this > way according to the logic in Section 3. > This result doesn't match the explanation in the draft. Am I > misunderstanding something? limit = DeliverdData+SMSS which is 2 for ACKs 18 and 19. > PRR-SSRB > ack# X X X X X X X X X X X X X X X 15 16 17 18 19 > pipe: 19 19 4 5 6 > sent: N N 2R 2R 2R > RB: d d d > > ack #: 17 18 19 > prr_delivered: 1 2 3 > prr_out: 0 2 4 > limit: 2 1 1 > sndcnt: 2 1 1 > > > 5: Is 'DeliveredData' necessary in the following equation? > I couldn't come up with good examples. But, I might miss something.. > > limit = MAX(prr_delivered - prr_out, DeliveredData) + MSS See above. Think of prr_delivered - prr_out as the bank: they reflect the cumulative totals since the beginning of recovery. By themselves they won't let you grow pipe. The DeliveredData term, on the other hand, is unconstrained by the past and lets you grow inflight by one MSS on every ACK. > Editorial Comments: [snip - thanks] (Except RFC 3042 is only used informatively.) > Regards, > -- > Yoshifumi Nishida Thanks, --MM-- The best way to predict the future is to create it. - Alan Kay From tsabatini@hotmail.com Fri Jul 13 09:11:18 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8D62E11E8087 for ; Fri, 13 Jul 2012 09:11:18 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.273 X-Spam-Level: X-Spam-Status: No, score=-2.273 tagged_above=-999 required=5 tests=[AWL=0.325, BAYES_00=-2.599, HTML_MESSAGE=0.001] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id s-PNnwbDrHRR for ; Fri, 13 Jul 2012 09:11:16 -0700 (PDT) Received: from bay0-omc2-s16.bay0.hotmail.com (bay0-omc2-s16.bay0.hotmail.com [65.54.190.91]) by ietfa.amsl.com (Postfix) with ESMTP id 1BDF211E80A3 for ; Fri, 13 Jul 2012 09:11:16 -0700 (PDT) Received: from BAY158-W63 ([65.54.190.125]) by bay0-omc2-s16.bay0.hotmail.com with Microsoft SMTPSVC(6.0.3790.4675); Fri, 13 Jul 2012 09:11:52 -0700 Message-ID: Content-Type: multipart/alternative; boundary="_24152f6a-f2c0-4723-a9d3-82da211a9b0a_" X-Originating-IP: [74.64.96.39] From: Anthony Sabatini To: Date: Fri, 13 Jul 2012 12:11:52 -0400 Importance: Normal In-Reply-To: References: , MIME-Version: 1.0 X-OriginalArrivalTime: 13 Jul 2012 16:11:52.0613 (UTC) FILETIME=[37259D50:01CD6112] X-Mailman-Approved-At: Sat, 14 Jul 2012 10:15:58 -0700 Cc: tcpm Subject: Re: [tcpm] TCP Error recovery and efficiency X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Jul 2012 16:11:18 -0000 --_24152f6a-f2c0-4723-a9d3-82da211a9b0a_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Yuchung - > What exactly do sender and receiver put in the sender/receiver state fiel= d? > Maybe we can have an example? say senders send 1=2C2=2C3=2C4 packets=2C > receiver receives only first 2nd then 4th packets. > > The state you described resembles sack scoreboard in RFC 3517=2C which > is implemented > in all major operating systems. Can you compare your approach to that RFC= ? The sender puts into the send state token the actual ordinal representing t= his message's sequence in the transmission stream regardless of whether the= message is new or a retransmission and then uses that ordinal to save the = information about the segment that was sent. When the transmit side receives a sack block it inserts it into the list of= acknowledged blocks=2C merging them or adding to the end of the list as ne= eded as with the current RFC. It also removes or modifies entries in the r= etransmit queue as needed. Each time the transmitter receives a change to i= ts token that was returned to it=2C it regenerates the retransmission queue= based on what segments are missing from the sack receive list and the age= s it forward (starting at the returned transmit token plus one) by removing= all entries which correspond to segments that have been sent since then. = This new list is the used as the retransmission queue=2C all entries in the= retransmission queue must be transmitted before new segments are transmitt= ed. Thus (p =3D packet=2C t =3D token) - Transmit - Network - Recvieve P1T0 --> P2T1 --> P3T2 --> P4T3 --> (Waits for work) --X (first packet lost) =20 P2T1 --> <-- SACK T1P2 --X (third packet lost) P4T3 --> <-- SACK T3P2P4 (Waits for work) <-- SACK T1P2 =20 P1T4 --> <-- SACK T3P2P4 P3T5 --> (It knows P1 is "inflight") P1T4 --> <-- ACK 2 SACK T4P4 P3T5 --> <-- ACK 4 SACK T5 <-- ACK 2 SACK T4P2=20 (Moves new floor to 2) (It knows P3 "inflight") <-- ACK 4 SACK T5 (Moves new floor to 4) Tony Anthony Sabatini 200 West 20th Street Apt. 1216 New York=2C NY 10011 Phone: (212) 867-7179 Mobile: (917) 224-8388 =20 > From: ycheng@google.com > Date: Thu=2C 12 Jul 2012 08:06:06 -0700 > Subject: Re: [tcpm] TCP Error recovery and efficiency > To: tsabatini@hotmail.com > CC: tcpm@ietf.org >=20 > Hi Anthony=2C >=20 > On Sun=2C Jul 8=2C 2012 at 8:55 PM=2C Anthony Sabatini wrote: > > All - > > > > While data communications was constantly improving=2C due mostly to the= wide > > adoption of fiber optics=2C the fact that TCP had an inferior recovery > > mechanism was moot as it came into play less and less often. Sadly tho= se > > days are now behind us and we face a world increasingly hostile to data > > transmission. The sources of packet loss have grown legion=2C "traffic > > shapers"=2C congested links to the subscibers=2C overloaded wifi bands= =2C content > > monitor that can not keep up=2C rate caps on mobile phone data usage=2C= the list > > goes on and on. Complicating this is the increase in data rates and fi= le > > sizes which allow many more packets and their acknowledgements to be > > inflight between two endpoints at once. We as a community must move al= ong > > the proliferation of SACK and if we are going to do that we should fix = the > > underlying issues in the protocol first. > > > > All current TCP recovery implementations are based on guessing the stat= e at > > the remote node. No recovery mechanism currently in place gives suffic= ient > > weight to in-flight or delayed messages. This problem can be solved ea= sily > > by transmitting a token related to the transmission state in each out g= oing > > message which the destination node will return on its next transmission= . By > What exactly do sender and receiver put in the sender/receiver state fiel= d? > Maybe we can have an example? say senders send 1=2C2=2C3=2C4 packets=2C > receiver receives only first 2nd then 4th packets. >=20 > The state you described resembles sack scoreboard in RFC 3517=2C which > is implemented > in all major operating systems. Can you compare your approach to that RFC= ? >=20 >=20 >=20 > > having this token I know all of the messages the far end should have se= en > > and if I do not have an acknowledgement for any of them=2C I know that = those > > need to be retransmitted. > > > > To start this discussion I have created an I-D "Highly Efficient Select= ive > > Acknowledgement (SACK) for TCP" - draft-sabatini-tcp-sack-00 which I ho= pe > > will be accepted by the working group. > > > > Note that even though no TCP implementation exists=2C the HDLC version = I > > created for the financial community has 20 years of working history all= over > > the world and the capability for being 100% efficient right down to one > > error per message no matter what the data rate or the delay (including > > multiple satellite hops). > > > > Tony > > > > _______________________________________________ > > tcpm mailing list > > tcpm@ietf.org > > https://www.ietf.org/mailman/listinfo/tcpm > > = --_24152f6a-f2c0-4723-a9d3-82da211a9b0a_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
Yuchung -

>=3B What exactly do sender and r= eceiver put in the sender/receiver state field?
>=3B Ma= ybe we can have an example? say senders send 1=2C2=2C3=2C4 packets=2C
>=3B receiver receives only first 2nd then 4th packets.
>=3B
>=3B The state you described resembles sack score= board in RFC 3517=2C which
>=3B is implemented
>=3B in all major operating = systems. Can you compare your approach to that RFC?

The sender puts into the s= end state token the actual ordinal representing this message's sequence in = the transmission stream regardless of whether the message is new or a retra= nsmission and then uses that ordinal to save the information about the segm= ent that was sent.
<= font style=3D"" face=3D"Courier New">
When the transmit side receives a sack block it inserts it into = the list of acknowledged blocks=2C merging them or adding to the end of the= list as needed as with the current RFC. =3B It also removes or modifie= s entries in the retransmit queue as needed. Each time the transmitter rece= ives a change to its token that was returned to it=2C it regenerates the re= transmission queue based on what segments are missing from the sack receive= list =3B and the ages it = forward (starting at the returned transmit token plus one) by removing all = entries which correspond to segments that have been sent since then. = =3B This new list is the used as the retransmission queue=2C all entries in= the retransmission queue must be transmitted before new segments are trans= mitted.

Thus (p =3D packet=2C t =3D token) -
Transmit - Network&= nbsp=3B =3B -  =3B =3B Recvieve
P1T0 =3B&= nbsp=3B =3B =3B =3B =3B -->=3B
P2T1&nbs= p=3B =3B =3B =3B =3B =3B -->=3B
P3T= 2 =3B =3B =3B =3B =3B =3B -->=3B

(Waits for work)
 =3B =3B =3B =3B&= nbsp=3B =3B =3B =3B =3B =3B --X =3B =3B =3B=  =3B =3B =3B =3B =3B =3B =3B =3B =3B&nb= sp=3B (first packet lost)  =3B
 =3B =3B = =3B =3B =3B =3B =3B =3B =3B =3B P2T1 =3B =3B =3B =3B =3B&= nbsp=3B =3B =3B =3B =3B =3B =3B -->=3B
 =3B =3B =3B =3B =3B =3B =3B =3B&nb= sp=3B =3B <=3B--  =3B  =3B  =3B  =3B  =3B =3B=  =3B =3B SACK T1P2
=  =3B =3B =3B = =3B =3B =3B =3B =3B =3B =3B --X =3B =3B&nbs= p=3B =3B =3B =3B =3B =3B =3B =3B =3B = =3B =3B (third packet lost)
 =3B  =3B  = =3B  =3B  =3B =3B P4T3  =3B =3B =3B =3B =3B=  =3B =3B =3B =3B =3B =3B -->=3B
 =3B  =3B  =3B  =3B  =3B =3B <=3B-- =3B = =3B =3B =3B =3B =3B =3B =3B =3B =3B =3B=  =3B =3B SACK T3P2P4 =3B  =3B  =3B &= nbsp=3B  =3B  =3B =3B =3B =3B =3B =3B =3B&n= bsp=3B =3B =3B =3B =3B =3B =3B =3B =3B = =3B (Waits for work)
<=3B-- =3B =3B =3B&nbs= p=3B =3B =3B =3B SACK T1P2 =3B
<= font style=3D"" face=3D"Courier New">P1T4 =3B =3B =3B =3B&n= bsp=3B =3B -->=3B
<=3B-- =3B =3B =3B&= nbsp=3B =3B =3B =3B SACK T3P2P4
P3T5 =3B&= nbsp=3B =3B =3B =3B =3B -->=3B
(It know= s P1 is "inflight")
=  =3B =3B =3B =3B = =3B =3B =3B =3B =3B =3B P1T4 =3B =3B =3B&nb= sp=3B =3B =3B =3B =3B =3B =3B =3B =3B -->= =3B
 =3B =3B =3B =3B =3B =3B = =3B =3B =3B =3B <=3B-- =3B =3B =3B =3B = =3B =3B =3B =3B =3B =3B =3B =3B =3B ACK 2 S= ACK T4P4
 =3B =3B =3B =3B =3B =3B=  =3B =3B =3B =3B P3T5 =3B =3B =3B =3B = =3B =3B =3B =3B =3B =3B =3B =3B -->=3B
&nb= sp=3B =3B =3B =3B =3B =3B =3B =3B =3B = =3B <=3B-- =3B =3B =3B =3B =3B =3B =3B = =3B =3B =3B =3B =3B =3B ACK 4 SACK T5
<=3B-- = =3B =3B =3B =3B =3B =3B =3B ACK 2 SACK T4P2
(Mo= ves new floor to 2)
(It knows P3 "inflight")
<=3B-- =3B =3B=  =3B =3B =3B =3B =3B ACK 4 SACK T5
(Moves new floor = to 4)

Tony

Anthony Sabatini
200 =3BWest 20th Street
Apt. 1216
New York=2C NY 10011

Phone: (212) 867-7179
Mobil= e: (917) 224-8388

 =3B
<= font style=3D"" face=3D"Courier New">

>=3B Fro= m: ycheng@google.com
>=3B Date: Thu=2C 12 Jul 2012 08:0= 6:06 -0700
>=3B Subject: Re: [tcpm] TCP Error recovery = and efficiency
>=3B To: tsabatini@hotmail.com
>=3B CC: tcpm@ietf.org
>=3B
>=3B Hi Anthony=2C
>=3B
>=3B = On Sun=2C Jul 8=2C 2012 at 8:55 PM=2C Anthony Sabatini <=3Btsabatini@hotm= ail.com>=3B wrote:
>=3B >=3B All -

>=3B >=3B While data communications w= as constantly improving=2C due mostly to the wide
>=3B = >=3B adoption of fiber optics=2C the fact that TCP had an inferior recove= ry
>=3B >=3B mechanism was moot as it came into play = less and less often. Sadly those
>=3B >=3B days are = now behind us and we face a world increasingly hostile to data
>=3B >=3B transmission. The sources of packet loss have grown le= gion=2C "traffic
>=3B >=3B shapers"=2C congested link= s to the subscibers=2C overloaded wifi bands=2C content
>=3B >=3B monitor that can not keep up=2C rate caps on mobile phone da= ta usage=2C the list
>=3B >=3B goes on and on. Compl= icating this is the increase in data rates and file
>= =3B >=3B sizes which allow many more packets and their acknowledgements t= o be
>=3B >=3B inflight between two endpoints at once= . We as a community must move along
>=3B >=3B the pr= oliferation of SACK and if we are going to do that we should fix the=
>=3B >=3B underlying issues in the protocol first.
>=3B >=3B
>=3B >=3B All current TCP rec= overy implementations are based on guessing the state at
>=3B >=3B the remote node. No recovery mechanism currently in place g= ives sufficient
>=3B >=3B weight to in-flight or dela= yed messages. This problem can be solved easily
>=3B &= gt=3B by transmitting a token related to the transmission state in each out= going
>=3B >=3B message which the destination node= will return on its next transmission. By
>=3B What ex= actly do sender and receiver put in the sender/receiver state field?=
>=3B Maybe we can have an example? say senders send 1=2C2=2C3= =2C4 packets=2C
>=3B receiver receives only first 2nd t= hen 4th packets.
>=3B
>=3B The sta= te you described resembles sack scoreboard in RFC 3517=2C which
>=3B is implemented>=3B in all major operatin= g systems. Can you compare your approach to that RFC?
>= =3B
>=3B =
>=3B
>=3B >=3B having this token I know all of the messages the far end sho= uld have seen
>=3B >=3B and if I do not have an ackno= wledgement for any of them=2C I know that those
>=3B &g= t=3B need to be retransmitted.=
>=3B >=3B
>=3B >=3B To start this discussion I have created an I-D "Highly = Efficient Selective
= >=3B >=3B Acknowledgement (SACK) = for TCP" - draft-sabatini-tcp-sack-00 which I hope
>=3B= >=3B will be accepted by the working group.
>=3B &g= t=3B
>=3B >=3B Note that even though no TCP implement= ation exists=2C the HDLC version I
>=3B >=3B created = for the financial community has 20 years of working history all over=
>=3B >=3B the world and the capability for being 100% effic= ient right down to one
>=3B >=3B error per message no= matter what the data rate or the delay (including
>=3B= >=3B multiple satellite hops).
>=3B >=3B
>=3B >=3B Tony>=3B >=3B
>=3B >=3B _______________________________________________
>=3B >=3B tcpm mailing list
>=3B >=3B tcp= m@ietf.org
>=3B >=3B https://www.ietf.org/mailman/lis= tinfo/tcpm
>=3B >=3B
= --_24152f6a-f2c0-4723-a9d3-82da211a9b0a_-- From rs@netapp.com Mon Jul 16 01:53:13 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 78FD721F8516 for ; Mon, 16 Jul 2012 01:53:13 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -9.614 X-Spam-Level: X-Spam-Status: No, score=-9.614 tagged_above=-999 required=5 tests=[AWL=0.985, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id WlA1lcvEl271 for ; Mon, 16 Jul 2012 01:53:12 -0700 (PDT) Received: from mx2.netapp.com (mx2.netapp.com [216.240.18.37]) by ietfa.amsl.com (Postfix) with ESMTP id A028E21F84EC for ; Mon, 16 Jul 2012 01:53:12 -0700 (PDT) X-IronPort-AV: E=Sophos;i="4.77,592,1336374000"; d="scan'208";a="666050983" Received: from smtp1.corp.netapp.com ([10.57.156.124]) by mx2-out.netapp.com with ESMTP; 16 Jul 2012 01:53:25 -0700 Received: from vmwexceht05-prd.hq.netapp.com (vmwexceht05-prd.hq.netapp.com [10.106.77.35]) by smtp1.corp.netapp.com (8.13.1/8.13.1/NTAP-1.6) with ESMTP id q6G8rP6A023033 for ; Mon, 16 Jul 2012 01:53:25 -0700 (PDT) Received: from SACEXCMBX02-PRD.hq.netapp.com ([169.254.1.34]) by vmwexceht05-prd.hq.netapp.com ([10.106.77.35]) with mapi id 14.02.0298.004; Mon, 16 Jul 2012 01:53:25 -0700 From: "Scheffenegger, Richard" To: "tcpm (tcpm@ietf.org)" Thread-Topic: New Version Notification for draft-scheffenegger-tcpm-timestamp-negotiation-04.txt Thread-Index: Ac1jMEoFBSSlToDnQNWbGMwYRyuCmQ== Date: Mon, 16 Jul 2012 08:53:23 +0000 Message-ID: <012C3117EDDB3C4781FD802A8C27DD4F0D4A88D2@SACEXCMBX02-PRD.hq.netapp.com> Accept-Language: de-AT, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.104.60.116] Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 MIME-Version: 1.0 Subject: [tcpm] FW: New Version Notification for draft-scheffenegger-tcpm-timestamp-negotiation-04.txt X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Jul 2012 08:53:13 -0000 LS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCkZyb206IGludGVybmV0LWRyYWZ0c0BpZXRmLm9y ZyBbbWFpbHRvOmludGVybmV0LWRyYWZ0c0BpZXRmLm9yZ10gDQpTZW50OiBNb250YWcsIDE2LiBK dWxpIDIwMTIgMTA6NDMNClRvOiBTY2hlZmZlbmVnZ2VyLCBSaWNoYXJkDQpDYzogbWlyamEua3Vl aGxld2luZEBpa3IudW5pLXN0dXR0Z2FydC5kZQ0KU3ViamVjdDogTmV3IFZlcnNpb24gTm90aWZp Y2F0aW9uIGZvciBkcmFmdC1zY2hlZmZlbmVnZ2VyLXRjcG0tdGltZXN0YW1wLW5lZ290aWF0aW9u LTA0LnR4dA0KDQoNCkEgbmV3IHZlcnNpb24gb2YgSS1ELCBkcmFmdC1zY2hlZmZlbmVnZ2VyLXRj cG0tdGltZXN0YW1wLW5lZ290aWF0aW9uLTA0LnR4dA0KaGFzIGJlZW4gc3VjY2Vzc2Z1bGx5IHN1 Ym1pdHRlZCBieSBSaWNoYXJkIFNjaGVmZmVuZWdnZXIgYW5kIHBvc3RlZCB0byB0aGUNCklFVEYg cmVwb3NpdG9yeS4NCg0KRmlsZW5hbWU6CSBkcmFmdC1zY2hlZmZlbmVnZ2VyLXRjcG0tdGltZXN0 YW1wLW5lZ290aWF0aW9uDQpSZXZpc2lvbjoJIDA0DQpUaXRsZToJCSBBZGRpdGlvbmFsIG5lZ290 aWF0aW9uIGluIHRoZSBUQ1AgVGltZXN0YW1wIE9wdGlvbiBmaWVsZCBkdXJpbmcgdGhlIFRDUCBo YW5kc2hha2UNCkNyZWF0aW9uIGRhdGU6CSAyMDEyLTA3LTE2DQpXRyBJRDoJCSBJbmRpdmlkdWFs IFN1Ym1pc3Npb24NCk51bWJlciBvZiBwYWdlczogMzQNClVSTDogICAgICAgICAgICAgaHR0cDov L3d3dy5pZXRmLm9yZy9pbnRlcm5ldC1kcmFmdHMvZHJhZnQtc2NoZWZmZW5lZ2dlci10Y3BtLXRp bWVzdGFtcC1uZWdvdGlhdGlvbi0wNC50eHQNClN0YXR1czogICAgICAgICAgaHR0cDovL2RhdGF0 cmFja2VyLmlldGYub3JnL2RvYy9kcmFmdC1zY2hlZmZlbmVnZ2VyLXRjcG0tdGltZXN0YW1wLW5l Z290aWF0aW9uDQpIdG1saXplZDogICAgICAgIGh0dHA6Ly90b29scy5pZXRmLm9yZy9odG1sL2Ry YWZ0LXNjaGVmZmVuZWdnZXItdGNwbS10aW1lc3RhbXAtbmVnb3RpYXRpb24tMDQNCkRpZmY6ICAg ICAgICAgICAgaHR0cDovL3Rvb2xzLmlldGYub3JnL3JmY2RpZmY/dXJsMj1kcmFmdC1zY2hlZmZl bmVnZ2VyLXRjcG0tdGltZXN0YW1wLW5lZ290aWF0aW9uLTA0DQoNCkFic3RyYWN0Og0KICAgQSBu dW1iZXIgb2YgVENQIGVuaGFuY2VtZW50cyBpbiBzbyBkaXZlcnNlIGZpZWxkcyBhcyBjb25nZXN0 aW9uDQogICBjb250cm9sLCBsb3NzIHJlY292ZXJ5IG9yIHNpZGUtYmFuZCBzaWduYWxpbmcgY291 bGQgYmUgaW1wcm92ZWQgYnkNCiAgIGFsbG93aW5nIGJvdGggZW5kcyBvZiBhIFRDUCBzZXNzaW9u IHRvIGludGVycHJldCB0aGUgdmFsdWUgY2FycmllZCBpbg0KICAgdGhlIFRpbWVzdGFtcCBvcHRp b24uICBGdXJ0aGVyIGVuaGFuY2VtZW50cyBhcmUgZW5hYmxlZCBieSBjaGFuZ2luZw0KICAgdGhl IHJlY2VpdmVyIHNpZGUgcHJvY2Vzc2luZyBvZiB0aW1lc3RhbXBzIGluIHRoZSBwcmVzZW5jZSBv Zg0KICAgU2VsZWN0aXZlIEFja25vd2xlZGdlbWVudHMuDQoNCiAgIFRoaXMgZG9jdW1lbnRzIHVw ZGF0ZXMgUkZDMTMyMyBhbmQgc3BlY2lmaWVzIGEgYmFja3dhcmRzIGNvbXBhdGlibGUNCiAgIHdh eSBvZiBuZWdvdGlhdGluZyBmb3IgVGltZXN0YW1wIGNhcGFiaWxpdGllcywgYW5kIGxpc3RzIGEg bnVtYmVyIG9mDQogICBiZW5lZml0cyBhbmQgZHJhd2JhY2tzIG9mIHRoaXMgYXBwcm9hY2guDQoN CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICANCg0KDQpUaGUgSUVURiBTZWNyZXRhcmlhdA0K From mirja.kuehlewind@ikr.uni-stuttgart.de Mon Jul 16 05:54:04 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4865421F844C for ; Mon, 16 Jul 2012 05:54:04 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 1.551 X-Spam-Level: * X-Spam-Status: No, score=1.551 tagged_above=-999 required=5 tests=[AWL=3.800, BAYES_00=-2.599, HELO_EQ_DE=0.35] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Mmb+oq4EOeDS for ; Mon, 16 Jul 2012 05:54:03 -0700 (PDT) Received: from mailsrv.ikr.uni-stuttgart.de (mailsrv.ikr.uni-stuttgart.de [129.69.170.2]) by ietfa.amsl.com (Postfix) with ESMTP id E8DFA21F8763 for ; Mon, 16 Jul 2012 05:54:02 -0700 (PDT) Received: from netsrv1.ikr.uni-stuttgart.de (netsrv1-c [10.11.12.12]) by mailsrv.ikr.uni-stuttgart.de (Postfix) with ESMTP id F177E633B1; Mon, 16 Jul 2012 14:54:45 +0200 (CEST) Received: from vpn-2-cl177 (vpn-2-cl177 [10.41.21.177]) by netsrv1.ikr.uni-stuttgart.de (Postfix) with ESMTP id E0A5A59A8A; Mon, 16 Jul 2012 14:54:45 +0200 (CEST) From: Mirja Kuehlewind Organization: University of Stuttgart (Germany), IKR To: tcpm@ietf.org Date: Mon, 16 Jul 2012 14:54:44 +0200 User-Agent: KMail/1.9.10 (enterprise35 0.20101217.1207316) MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Message-Id: <201207161454.45011.mirja.kuehlewind@ikr.uni-stuttgart.de> Subject: [tcpm] New Version Notification for draft-kuehlewind-tcpm-accurate-ecn-01.txt and draft-kuehlewind-tcpm-accurate-ecn-option-01.txt X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Jul 2012 12:54:04 -0000 Hi everybody, we updated the two drafts describing more accurate ECN feedback: =2D draft-kuehlewind-tcpm-accurate-ecn-option-01.txt=20 specifies an additional TCP option =2D draft-kuehlewind-tcpm-accurate-ecn-01.txt specifies the re-use of the ECN/NS TCP header bits Mirja =2D--------- Forwarded Message ---------- Subject: New Version Notification for=20 draft-kuehlewind-tcpm-accurate-ecn-option-01.txt Date: Friday 13 July 2012, 16:25:43 =46rom: internet-drafts@ietf.org To: mirja.kuehlewind@ikr.uni-stuttgart.de CC: rs@netapp.com A new version of I-D, draft-kuehlewind-tcpm-accurate-ecn-option-01.txt has been successfully submitted by Mirja Kuehlewind and posted to the IETF repository. =46ilename: draft-kuehlewind-tcpm-accurate-ecn-option Revision: 01 Title: Accurate ECN Feedback Option in TCP Creation date: 2012-07-13 WG ID: Individual Submission Number of pages: 7 URL: =20 http://www.ietf.org/internet-drafts/draft-kuehlewind-tcpm-accurate-ecn-opti= on-01.txt Status: =20 http://datatracker.ietf.org/doc/draft-kuehlewind-tcpm-accurate-ecn-option Htmlized: =20 http://tools.ietf.org/html/draft-kuehlewind-tcpm-accurate-ecn-option-01 Diff: =20 http://tools.ietf.org/rfcdiff?url2=3Ddraft-kuehlewind-tcpm-accurate-ecn-opt= ion-01 Abstract: This document specifies an TCP option to get accurate Explicit Congestion Notification (ECN) feedback from the receiver. ECN is an IP/TCP mechanism where network nodes can mark IP packets instead of dropping them to indicate congestion to the end-points. An ECN- capable receiver will feedback this information to the sender. ECN is specified for TCP in such a way that only one feedback signal can be transmitted per Round-Trip Time (RTT). Recently new TCP mechanisms like ConEx or DCTCP need more accurate feedback information in the case where more than one marking is received in one RTT. This TCP extension can be used in addition to the classic ECN as well as with a more accurate ECN scheme recently proposed which reuses the ECN bit in the TCP header. = =20 The IETF Secretariat =2D------------------------------------------------------ =2D--------- Forwarded Message ---------- Subject: New Version Notification for=20 draft-kuehlewind-tcpm-accurate-ecn-01.txt Date: Monday 16 July 2012, 14:50:02 =46rom: internet-drafts@ietf.org To: mirja.kuehlewind@ikr.uni-stuttgart.de CC: rs@netapp.com A new version of I-D, draft-kuehlewind-tcpm-accurate-ecn-01.txt has been successfully submitted by Mirja Kuehlewind and posted to the IETF repository. =46ilename: draft-kuehlewind-tcpm-accurate-ecn Revision: 01 Title: More Accurate ECN Feedback in TCP Creation date: 2012-07-16 WG ID: Individual Submission Number of pages: 19 URL: =20 http://www.ietf.org/internet-drafts/draft-kuehlewind-tcpm-accurate-ecn-01.t= xt Status: =20 http://datatracker.ietf.org/doc/draft-kuehlewind-tcpm-accurate-ecn Htmlized: =20 http://tools.ietf.org/html/draft-kuehlewind-tcpm-accurate-ecn-01 Diff: =20 http://tools.ietf.org/rfcdiff?url2=3Ddraft-kuehlewind-tcpm-accurate-ecn-01 Abstract: Explicit Congestion Notification (ECN) is an IP/TCP mechanism where network nodes can mark IP packets instead of dropping them to indicate congestion to the end-points. An ECN-capable receiver will feedback this information to the sender. ECN is specified for TCP in such a way that only one feedback signal can be transmitted per Round-Trip Time (RTT). Recently, new TCP mechanisms like ConEx or DCTCP need more accurate ECN feedback information in the case where more than one marking is received in one RTT. This documents specifies a different scheme for the ECN feedback in the TCP header to provide more than one feedback signal per RTT. = =20 The IETF Secretariat =2D------------------------------------------------------ =2D-=20 =2D------------------------------------------------------------------ Dipl.-Ing. Mirja K=FChlewind Institute of Communication Networks and Computer Engineering (IKR) University of Stuttgart, Germany Pfaffenwaldring 47, D-70569 Stuttgart tel: +49(0)711/685-67973 email: mirja.kuehlewind@ikr.uni-stuttgart.de web: www.ikr.uni-stuttgart.de =2D------------------------------------------------------------------ From cuttyroyce@yahoo.com Mon Jul 16 07:35:04 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 523AD21F85DD for ; Mon, 16 Jul 2012 07:35:04 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.602 X-Spam-Level: X-Spam-Status: No, score=0.602 tagged_above=-999 required=5 tests=[BAYES_50=0.001, HTML_MESSAGE=0.001, J_CHICKENPOX_72=0.6] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id URHxCMtAUhVc for ; Mon, 16 Jul 2012 07:35:03 -0700 (PDT) Received: from nm28-vm3.bullet.mail.ne1.yahoo.com (nm28-vm3.bullet.mail.ne1.yahoo.com [98.138.91.158]) by ietfa.amsl.com (Postfix) with SMTP id A0B2221F85A5 for ; Mon, 16 Jul 2012 07:35:03 -0700 (PDT) Received: from [98.138.90.54] by nm28.bullet.mail.ne1.yahoo.com with NNFMP; 16 Jul 2012 14:35:45 -0000 Received: from [98.138.88.232] by tm7.bullet.mail.ne1.yahoo.com with NNFMP; 16 Jul 2012 14:35:45 -0000 Received: from [127.0.0.1] by omp1032.mail.ne1.yahoo.com with NNFMP; 16 Jul 2012 14:35:45 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 770386.61735.bm@omp1032.mail.ne1.yahoo.com Received: (qmail 18280 invoked by uid 60001); 16 Jul 2012 14:35:45 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1342449345; bh=6tF9tq2yRfGLEhJFZivmnLMdXc+IsuURaBI8c4zhDdw=; h=X-YMail-OSG:Received:X-Mailer:Message-ID:Date:From:Reply-To:Subject:To:MIME-Version:Content-Type; b=W8OGvMJAPp3qM64t2TzVL7YKPWmOUJJy9vuc88sIemt0ocPPZ6YqUI4t/5lhKtqaFXqM9CfTg/7dXOEA8P868ltSUE1b+tWyVpPUwoqQVljN6cfsFbZfhu/rRKNMk/L0jiWxkfkExisYkM7imYfEvDiYFIosTTFJvc5JPDBBKrU= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Mailer:Message-ID:Date:From:Reply-To:Subject:To:MIME-Version:Content-Type; b=j5aSjIGvxsQ4rcGBNtspeMafoFsVC37T7yYfOizJnBYhrBuUSj8gc7u2OXopeT8DBlmNFFAK6WQibrg6XlryCb3bb1q0VexMeNlhN3cWHItWMq9URYA9T2YPYdrkOhkczVapsqHFEVRLK00fWYvDSQyhQlpz2R4hXoG+Q0yAhfQ=; X-YMail-OSG: vm3qEcwVM1kLqfuebSH_iCj_4shXGOGscVcBSKxbGqHchSh _JyfoYUD94AOIFtcpSoYQigytgP.bNs02sTxCKIluyBvG07FHprRCkqyk0pS mje0lxeHbybB7Sf4qKIw0LSFxASwF2h6d03SsymFMvxh1qkxeXRwJ._lEK.c SVAlensmVdy6Pj1awDJvCcmzlQg4kKb62fZaEIzUpdaAL58eqcJGVsyV4QcG wRyOogCnAJSNjSSn_2gxY_nDsraJGhSXo9M.IxxYjf.gj5Ao9OVycrWDE66f JGjfNaUKqtstB62.SX9sMruvc8vH59DuHTDDtlXv.IoyuQXnqHUqb0jYmuB8 wdLcZRyvOC5NwoDZ8Ps32jlB0i5mR6uyMBQI75SlCvIRgCS0M8b89K77L1oE 77TM- Received: from [113.23.132.198] by web125003.mail.ne1.yahoo.com via HTTP; Mon, 16 Jul 2012 07:35:45 PDT X-Mailer: YahooMailWebService/0.8.120.356233 Message-ID: <1342449345.2366.YahooMailNeo@web125003.mail.ne1.yahoo.com> Date: Mon, 16 Jul 2012 07:35:45 -0700 (PDT) From: Jamal Cynthia To: "tcpm@ietf.org" MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="-723168630-1380551272-1342449345=:2366" Subject: [tcpm] Shared memory X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list Reply-To: Jamal Cynthia List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Jul 2012 14:35:04 -0000 ---723168630-1380551272-1342449345=:2366 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable In computing, shared memory is memory that may be simultaneously accessed b= y multiple programs with an intent to provide communication among them or a= void redundant copies. Shared memory is an efficient means of passing data = between programs.=A0=0ADepending on context, programs may run on a single p= rocessor or on multiple separate processors.=A0=0AUsing memory for communic= ation inside a single program, for example among its multiple threads, is g= enerally not referred to as shared memory.=0AWhen using shared memory commu= nication between multiple processors the WRAPPER level shields user applica= tions from certain counter-intuitive system behaviors. In particular, one i= ssue the WRAPPER layer must deal with is a systems memory model. In general= the order of reads and writes expressed by the textual order of an applica= tion code may not be the ordering of instructions executed by the processor= performing the application.=A0=0AThe processor performing the application = instructions will always operate so that, for the application instructions = the processor is executing, any reordering is not apparent. However,in gene= ral machines are often designed so that reordering of instructions is not h= idden from other second processors.=A0=0AThis means that, in general, even = on a shared memory system two processors can observe inconsistent memory va= lues.=A0 ---723168630-1380551272-1342449345=:2366 Content-Type: text/html; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable
In computing, shared memory is= memory that may be simultaneously accessed by multiple programs with an in= tent to provide communication among them or avoid redundant copies. Shared = memory is an efficient means of passing data between programs. =
Depending= on context, programs may run on a single processor or on multiple separate= processors. 
Using memory for communication inside a single program, for= example among its multiple threads, is generally not referred to as shared= memory.
When using shared memory communication between multiple processors th= e WRAPPER level shields user applications from certain counter-intuitive sys= tem behaviors. In particular, one issue the WRAPPER layer must deal with is= a systems memory model. In general the order of reads and writes expressed= by the textual order of an application code may not be the ordering of ins= tructions executed by the processor performing the application. 
The proc= essor performing the application instructions will always operate so that, = for the application instructions the processor is executing, any reordering= is not apparent. However,in general machines are often designed so that re= ordering of instructions is not hidden from other second processors. <= /font>
Thi= s means that, in general, even on a shared memory system two processors can= observe inconsistent memory values. 
---723168630-1380551272-1342449345=:2366-- From cuttyroyce@yahoo.com Mon Jul 16 08:02:17 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 536D321F878E for ; Mon, 16 Jul 2012 08:02:17 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.098 X-Spam-Level: X-Spam-Status: No, score=-0.098 tagged_above=-999 required=5 tests=[AWL=0.700, BAYES_00=-2.599, HTML_MESSAGE=0.001, J_CHICKENPOX_53=0.6, J_CHICKENPOX_74=0.6, J_CHICKENPOX_83=0.6] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id AvPlQD8ljSEg for ; Mon, 16 Jul 2012 08:02:16 -0700 (PDT) Received: from nm25-vm0.bullet.mail.ne1.yahoo.com (nm25-vm0.bullet.mail.ne1.yahoo.com [98.138.91.73]) by ietfa.amsl.com (Postfix) with SMTP id BF73F21F876D for ; Mon, 16 Jul 2012 08:02:16 -0700 (PDT) Received: from [98.138.90.55] by nm25.bullet.mail.ne1.yahoo.com with NNFMP; 16 Jul 2012 15:02:58 -0000 Received: from [98.138.89.234] by tm8.bullet.mail.ne1.yahoo.com with NNFMP; 16 Jul 2012 15:02:58 -0000 Received: from [127.0.0.1] by omp1049.mail.ne1.yahoo.com with NNFMP; 16 Jul 2012 15:02:58 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 875099.66161.bm@omp1049.mail.ne1.yahoo.com Received: (qmail 69272 invoked by uid 60001); 16 Jul 2012 15:02:58 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1342450978; bh=WM0iCdCH5ye4AZuikkA4SsHF0L//lDOK166zKh81nTo=; h=X-YMail-OSG:Received:X-Mailer:Message-ID:Date:From:Reply-To:Subject:To:MIME-Version:Content-Type; b=pAn/wKIr958AnlgtxBsOm0k87j4r+8BRDv98fCd0D57SAq+jPsWH/R4gMiqVTgoSaBgDHNp9gf8WLr3R+AtvhYubbuQp534bB9U0JbVLZUnCNXbqdMMoxDpbWdBBm4Jgw2cpt9ZKWn2PCsdhRTKPEUSfyTkJBYFC45A8dJGGFkA= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Mailer:Message-ID:Date:From:Reply-To:Subject:To:MIME-Version:Content-Type; b=rKN1CWuiP7ao9WgX4K6QbqS6XH/G50pUxiEmRpNVbrVDXp08xftS61vSesUj5i1XTNXb7r+qI7ktEgAuX+5ZIKC9euxj3zx0fjZrtc8L+ay0RReaCfXL8sM2DBFqbdl4l08kL9C0w9RsPtEXgJtL9mWNgq5EI8XpIsH2RmW4cuI=; X-YMail-OSG: h1PopHwVM1krb3PJhVM1qYyIU3ooSlurSkq9JnT.NHeGN0Z 7s2NeUPpo Received: from [113.23.132.198] by web125005.mail.ne1.yahoo.com via HTTP; Mon, 16 Jul 2012 08:02:58 PDT X-Mailer: YahooMailWebService/0.8.120.356233 Message-ID: <1342450978.58887.YahooMailNeo@web125005.mail.ne1.yahoo.com> Date: Mon, 16 Jul 2012 08:02:58 -0700 (PDT) From: Jamal Cynthia To: "tcpm@ietf.org" MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="1988236031-955511987-1342450978=:58887" Subject: [tcpm] TCP error recovery and efficiency X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list Reply-To: Jamal Cynthia List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Jul 2012 15:02:17 -0000 --1988236031-955511987-1342450978=:58887 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable TCP error recovery is needed because packet losses can occur at network lay= er E.g buffer overflos and also some link layers may not be reliable.TCP us= es a timeout mechanism for packet retransmission which are timeout calculat= ion and fast retransmission.=0ATCP error-control does have some energy cons= erving =A0capabilities, in response to segment drops it reduces its window = size and =A0therefore conserves transmission =A0effort.The errorrecovery me= chanism however is not always efficient,especially when the error pattern c= hanges since packet loss is invariably interpreted by the protocol as resul= ting from congestion.For example,when relatively infrequent random or short= burst errors occur,the sender backs off and then applies a graduated incre= ase to its reduced window size. --1988236031-955511987-1342450978=:58887 Content-Type: text/html; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable
TCP error recovery is needed b= ecause packet losses can occur at network layer E.g buffer overflos and als= o some link layers may not be reliable.TCP uses a timeout mechanism for pac= ket retransmission which are timeout calculation and fast retransmission.
TCP = error-control does have some energy conserving  capabilities, in respo= nse to segment drops it reduces its window size and  therefore conserv= es transmission  effort.The errorrecovery mechanism however is not alw= ays efficient,especially when the error pattern changes since packet loss i= s invariably interpreted by the protocol as resulting from congestion.For e= xample,when relatively infrequent random or short burst errors occur,the sender backs off and then applies a graduated increase to its reduced wind= ow size.
--1988236031-955511987-1342450978=:58887-- From cuttyroyce@yahoo.com Mon Jul 16 08:41:09 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5440221F86A6 for ; Mon, 16 Jul 2012 08:41:09 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.898 X-Spam-Level: X-Spam-Status: No, score=0.898 tagged_above=-999 required=5 tests=[AWL=-0.996, BAYES_50=0.001, HTML_MESSAGE=0.001, J_CHICKENPOX_83=0.6, MISSING_HEADERS=1.292] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id wQBxIpx5nIzD for ; Mon, 16 Jul 2012 08:41:08 -0700 (PDT) Received: from nm31-vm2.bullet.mail.ne1.yahoo.com (nm31-vm2.bullet.mail.ne1.yahoo.com [98.138.229.42]) by ietfa.amsl.com (Postfix) with SMTP id AECF521F86A4 for ; Mon, 16 Jul 2012 08:41:08 -0700 (PDT) Received: from [98.138.90.56] by nm31.bullet.mail.ne1.yahoo.com with NNFMP; 16 Jul 2012 15:41:49 -0000 Received: from [98.138.87.8] by tm9.bullet.mail.ne1.yahoo.com with NNFMP; 16 Jul 2012 15:41:49 -0000 Received: from [127.0.0.1] by omp1008.mail.ne1.yahoo.com with NNFMP; 16 Jul 2012 15:41:49 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 202569.28245.bm@omp1008.mail.ne1.yahoo.com Received: (qmail 82771 invoked by uid 60001); 16 Jul 2012 15:41:49 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1342453309; bh=rmp+HYuH4mLRBmMBryq9wjMb9jARsOqubsbnlxzQGc4=; h=X-YMail-OSG:Received:X-Mailer:Message-ID:Date:From:Reply-To:Subject:Cc:MIME-Version:Content-Type; b=AVqPH/2/8PQYa7wHXtpiZoaBY166UsD/pSZFSse104dKttiR/AldhdGu2H9ppk7l3V++NVggndwJ2SjQwXbiUxSDVg533mYiwXypOFrwbjq9+zNgQyY+CSeI7X6Bk73DRWGGaFyGnGSKedaLWyEIFOnkS3ag8WhxNY6rf+0wuc0= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Mailer:Message-ID:Date:From:Reply-To:Subject:Cc:MIME-Version:Content-Type; b=AmSa7MFkrJCfvlD79yxoiv9+WXoVVPS0I/KZWtT0u0aieWwbCwuF15YdsHUqYjXv3oy3H2WdLe1twiZZy3eCr+2Q/p/jkTcfHir0ZkRXDpRTCdhcLjEETO+XNe91jfV/qzLDRPhwb5Db2NvEt2ZcmdxAZr5fdrw+IX/IuJEc19w=; X-YMail-OSG: yKEhO6UVM1kJ.tSKCWeXsTgsmR1nocCUA0iS.icETrCB2qF y.x6rU13u0udQvpRb7e1pMkWkchsJ7q1e9.NZXLuakk52z7mwHncnA53PySd qh49lX48bgUgZBIKvCyhesWJM4QIcfszo0RJu1z7QXSYM4V6pG4hv4NML4rE mbURn31mWDJ9gC5y.2syaLMojSv3JSGM.7F7D7u4HL7nsKNHVlmyq.oPMUNL gBo_ARY.tBC4dknUMMEAsVdJuY1j_voj_SH3jDEGDJExXjZ9q7jIz8nYnxES AyY7B3p6adWGPwG_tmGiXMzKdKtdxW69pdDhCWfIcT6WJYBYqlXCni69gN2B _mnDgIIol.GcPxZ2vPug83fZqb_QdnrjJam_5rLiY__lyX_.Xw6NN2udQ1sE lgRk- Received: from [113.23.132.198] by web125003.mail.ne1.yahoo.com via HTTP; Mon, 16 Jul 2012 08:41:49 PDT X-Mailer: YahooMailWebService/0.8.120.356233 Message-ID: <1342453309.82036.YahooMailNeo@web125003.mail.ne1.yahoo.com> Date: Mon, 16 Jul 2012 08:41:49 -0700 (PDT) From: Jamal Cynthia Cc: "tcpm@ietf.org" MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="-723168630-998602467-1342453309=:82036" Subject: Re: [tcpm] tcpm timestamp X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list Reply-To: Jamal Cynthia List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Jul 2012 15:41:09 -0000 ---723168630-998602467-1342453309=:82036 Content-Type: text/plain; charset=us-ascii TCP timestamps are used to provide protection against wrapped sequence numbers. It is possible to calculate system uptime (and boot time) by analyzing TCP timestamps.These calculated uptimes (and boot times) can help in detecting hidden network-enabled operating systems,linking spoofed IP and MAC addresses together,linking IP addresses with Ad-Hoc wireless APs, etc.The TCP time-stamp option provides better TCP round-trip time measurements. Because the time stamps are always sent and echoed in both directions and the time-stamp value in the header is always changing, TCP header compression will not compress the outgoing packet. To allow TCP header compression over a serial link, the TCP time-stamp option is disabled.TCP timestamps help TCP determine in which order packets were sent. TCP timestamps are not normally aligned to the system clock and start at some random value. Many operating systems will increment the timestamp for every elapsed milisecond; however RFC only states that the tics should be proportional. ---723168630-998602467-1342453309=:82036 Content-Type: text/html; charset=us-ascii
TCP timestamps are used to provide protection against wrapped sequence numbers. It is possible to calculate system uptime (and boot time) by analyzing TCP timestamps.These calculated uptimes (and boot times) can help in detecting hidden network-enabled operating systems,linking spoofed IP and MAC addresses together,linking IP addresses with Ad-Hoc wireless APs, etc.The TCP time-stamp option provides better TCP round-trip time measurements. Because the time stamps are always sent and echoed in both directions and the time-stamp value in the header is always changing, TCP header compression will not compress the outgoing packet. To allow TCP header compression over a serial link, the TCP time-stamp option is disabled.TCP timestamps help TCP determine in which order packets were sent. TCP timestamps are not normally aligned to the system clock and start at some random value. Many operating systems will increment the timestamp for every elapsed milisecond; however RFC only states that the tics should be proportional.
---723168630-998602467-1342453309=:82036-- From lars@netapp.com Mon Jul 16 09:10:55 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 55DA711E80DC for ; Mon, 16 Jul 2012 09:10:55 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.448 X-Spam-Level: X-Spam-Status: No, score=-10.448 tagged_above=-999 required=5 tests=[AWL=0.151, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Q76IGZb-nmvd for ; Mon, 16 Jul 2012 09:10:54 -0700 (PDT) Received: from mx2.netapp.com (mx2.netapp.com [216.240.18.37]) by ietfa.amsl.com (Postfix) with ESMTP id D2F6911E80A1 for ; Mon, 16 Jul 2012 09:10:54 -0700 (PDT) X-IronPort-AV: E=Sophos;i="4.77,594,1336374000"; d="p7s'?scan'208";a="666139692" Received: from smtp2.corp.netapp.com ([10.57.159.114]) by mx2-out.netapp.com with ESMTP; 16 Jul 2012 09:11:39 -0700 Received: from vmwexceht05-prd.hq.netapp.com (vmwexceht05-prd.hq.netapp.com [10.106.77.35]) by smtp2.corp.netapp.com (8.13.1/8.13.1/NTAP-1.6) with ESMTP id q6GGBdfc012670 for ; Mon, 16 Jul 2012 09:11:39 -0700 (PDT) Received: from SACEXCMBX01-PRD.hq.netapp.com ([169.254.2.13]) by vmwexceht05-prd.hq.netapp.com ([10.106.77.35]) with mapi id 14.02.0298.004; Mon, 16 Jul 2012 09:11:39 -0700 From: "Eggert, Lars" To: "tcpm@ietf.org" Thread-Topic: bot? cuttyroyce@yahoo.com Thread-Index: AQHNY22sjOqvC5k3eU6vMNwbqwnLbQ== Date: Mon, 16 Jul 2012 16:11:35 +0000 Message-ID: <00B78AB4-72CB-40B6-BE25-6197A3A7EEF9@netapp.com> References: <1342449345.2366.YahooMailNeo@web125003.mail.ne1.yahoo.com> In-Reply-To: <1342449345.2366.YahooMailNeo@web125003.mail.ne1.yahoo.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: x-originating-ip: [10.104.60.115] Content-Type: multipart/signed; boundary="Apple-Mail=_AE6EA100-A584-49AE-AC58-7FB19C256E13"; protocol="application/pkcs7-signature"; micalg=sha1 MIME-Version: 1.0 Subject: [tcpm] bot? cuttyroyce@yahoo.com X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Jul 2012 16:10:55 -0000 --Apple-Mail=_AE6EA100-A584-49AE-AC58-7FB19C256E13 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=iso-8859-1 Can we block this address from the list? I suspect it's a bot that spews = Wikipedia paragraphs to the list. (Google for some of the sentences from = the recent emails.) Lars= --Apple-Mail=_AE6EA100-A584-49AE-AC58-7FB19C256E13 Content-Disposition: attachment; filename="smime.p7s" Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIMQDCCBUow ggQyoAMCAQICEFcfSRTG0jNknqb9LV9GuFkwDQYJKoZIhvcNAQEFBQAwgd0xCzAJBgNVBAYTAlVT MRcwFQYDVQQKEw5WZXJpU2lnbiwgSW5jLjEfMB0GA1UECxMWVmVyaVNpZ24gVHJ1c3QgTmV0d29y azE7MDkGA1UECxMyVGVybXMgb2YgdXNlIGF0IGh0dHBzOi8vd3d3LnZlcmlzaWduLmNvbS9ycGEg KGMpMDkxHjAcBgNVBAsTFVBlcnNvbmEgTm90IFZhbGlkYXRlZDE3MDUGA1UEAxMuVmVyaVNpZ24g Q2xhc3MgMSBJbmRpdmlkdWFsIFN1YnNjcmliZXIgQ0EgLSBHMzAeFw0xMTEyMTAwMDAwMDBaFw0x MjEyMDkyMzU5NTlaMIIBDTEXMBUGA1UEChMOVmVyaVNpZ24sIEluYy4xHzAdBgNVBAsTFlZlcmlT aWduIFRydXN0IE5ldHdvcmsxRjBEBgNVBAsTPXd3dy52ZXJpc2lnbi5jb20vcmVwb3NpdG9yeS9S UEEgSW5jb3JwLiBieSBSZWYuLExJQUIuTFREKGMpOTgxHjAcBgNVBAsTFVBlcnNvbmEgTm90IFZh bGlkYXRlZDEzMDEGA1UECxMqRGlnaXRhbCBJRCBDbGFzcyAxIC0gTmV0c2NhcGUgRnVsbCBTZXJ2 aWNlMRQwEgYDVQQDFAtMYXJzIEVnZ2VydDEeMBwGCSqGSIb3DQEJARYPbGFyc0BuZXRhcHAuY29t MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAokrhJTcXt6J/VEpZOicLoguBlYTjXP9v Ze4HuuhXnURUS8YouAfgaqA0zYbt5yd6fh4PBMdAaEWr5yJyHuFykXlrCumjUWSpLuqTS2A+pt4q cZaAQk9iLDN/UVd3SpkUuvWbxXlqzG7/BSqa3VNObBzCmyh+V7aXxri+30CT//DSsNRC4VFy6sn6 dMgSaFenXLwe/FBwY0qTMfICT1PrrX6Sw1S8OfH9rykLlZXbmfkFExxQngp1DJH9xMHeODHGbCv/ ty5gdxMOrLe+vENxFEcy1YQWBZd1kNL4UObugF8A/jE/s+Oa3H1VFH8ghqZTdqGDysVxmtKHuNFx 6jIBSQIDAQABo4HSMIHPMAkGA1UdEwQCMAAwRAYDVR0gBD0wOzA5BgtghkgBhvhFAQcXATAqMCgG CCsGAQUFBwIBFhxodHRwczovL3d3dy52ZXJpc2lnbi5jb20vcnBhMAsGA1UdDwQEAwIFoDAdBgNV HSUEFjAUBggrBgEFBQcDBAYIKwYBBQUHAwIwUAYDVR0fBEkwRzBFoEOgQYY/aHR0cDovL2luZGMx ZGlnaXRhbGlkLWczLWNybC52ZXJpc2lnbi5jb20vSW5kQzFEaWdpdGFsSUQtRzMuY3JsMA0GCSqG SIb3DQEBBQUAA4IBAQBA7q6tR92qpd7xo7VBsrOfGCWzoxIVfTc7t0RhB/Oz/+c3lnhYnNScIuKN JmyZvznmVxqB9BJ72+NkvmdB/hnILSBTRawL2tyLo9PkBtN0nRt4gS6wjpWnD8G83hlJLE7r25jk 7HkRev61dTIXsANFpJKF02C4XSoDfEzNV6MpuEvHvcgHCqMrlwWwfKc7+NoDnE8PBuRzwSXvlD5L mswCY2iiOsd7ImNO4OzTCxETvKTDu92+FTIbRJJpYjVNv1UF7e3w9Kq65BkZJErUH19beUeQl0Wh 2BJQE6/15rQyCnP0iJ/Nmx2/kI6M0PWunEsI6FMs0MbosreaWGHlQmomMIIG7jCCBdagAwIBAgIQ cRVmBUrkkSFN6bxE+azT3DANBgkqhkiG9w0BAQUFADCByjELMAkGA1UEBhMCVVMxFzAVBgNVBAoT DlZlcmlTaWduLCBJbmMuMR8wHQYDVQQLExZWZXJpU2lnbiBUcnVzdCBOZXR3b3JrMTowOAYDVQQL EzEoYykgMTk5OSBWZXJpU2lnbiwgSW5jLiAtIEZvciBhdXRob3JpemVkIHVzZSBvbmx5MUUwQwYD VQQDEzxWZXJpU2lnbiBDbGFzcyAxIFB1YmxpYyBQcmltYXJ5IENlcnRpZmljYXRpb24gQXV0aG9y aXR5IC0gRzMwHhcNMDkwNTAxMDAwMDAwWhcNMTkwNDMwMjM1OTU5WjCB3TELMAkGA1UEBhMCVVMx FzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMR8wHQYDVQQLExZWZXJpU2lnbiBUcnVzdCBOZXR3b3Jr MTswOQYDVQQLEzJUZXJtcyBvZiB1c2UgYXQgaHR0cHM6Ly93d3cudmVyaXNpZ24uY29tL3JwYSAo YykwOTEeMBwGA1UECxMVUGVyc29uYSBOb3QgVmFsaWRhdGVkMTcwNQYDVQQDEy5WZXJpU2lnbiBD bGFzcyAxIEluZGl2aWR1YWwgU3Vic2NyaWJlciBDQSAtIEczMIIBIjANBgkqhkiG9w0BAQEFAAOC AQ8AMIIBCgKCAQEA7cRH3yooHXwGa7vXITLJbBOP6bGNQU4099oL42r6ZYggCxET6ZvgSU6Lb9UB 0F8NR5GKWkx0Pj/GkQm7TDSejW6hglFi92l2WJYHr54UGAdPWr2f0jGyVBlzRmoZQhHsEnMhjfXc MM3l2VYKMcU2bSkUl70t2olHGYjYSwQ967Y8Zx50ABMN0Ibak2f4MwOuGjxraXj2wCyO4YM/d/mZ //6fUlrCtIcK2GypR8FUKWVDPkrAlh/Brfd3r2yxBF6+wbaULZeQLSfSux7pg2qE9sSyriMGZSal J1grByK0b6ZiSBp38tVQJ5op05b7KPW6JHZi44xZ6/tu1ULEvkHH9QIDAQABo4ICuTCCArUwNAYI KwYBBQUHAQEEKDAmMCQGCCsGAQUFBzABhhhodHRwOi8vb2NzcC52ZXJpc2lnbi5jb20wEgYDVR0T AQH/BAgwBgEB/wIBADBwBgNVHSAEaTBnMGUGC2CGSAGG+EUBBxcBMFYwKAYIKwYBBQUHAgEWHGh0 dHBzOi8vd3d3LnZlcmlzaWduLmNvbS9jcHMwKgYIKwYBBQUHAgIwHhocaHR0cHM6Ly93d3cudmVy aXNpZ24uY29tL3JwYTA0BgNVHR8ELTArMCmgJ6AlhiNodHRwOi8vY3JsLnZlcmlzaWduLmNvbS9w Y2ExLWczLmNybDAOBgNVHQ8BAf8EBAMCAQYwbgYIKwYBBQUHAQwEYjBgoV6gXDBaMFgwVhYJaW1h Z2UvZ2lmMCEwHzAHBgUrDgMCGgQUS2u5KJYGDLvQUjibKaxLB4shBRgwJhYkaHR0cDovL2xvZ28u dmVyaXNpZ24uY29tL3ZzbG9nbzEuZ2lmMC4GA1UdEQQnMCWkIzAhMR8wHQYDVQQDExZQcml2YXRl TGFiZWw0LTIwNDgtMTE4MB0GA1UdDgQWBBR5R2EIQf04BKJL57XM9UP2SSsR+DCB8QYDVR0jBIHp MIHmoYHQpIHNMIHKMQswCQYDVQQGEwJVUzEXMBUGA1UEChMOVmVyaVNpZ24sIEluYy4xHzAdBgNV BAsTFlZlcmlTaWduIFRydXN0IE5ldHdvcmsxOjA4BgNVBAsTMShjKSAxOTk5IFZlcmlTaWduLCBJ bmMuIC0gRm9yIGF1dGhvcml6ZWQgdXNlIG9ubHkxRTBDBgNVBAMTPFZlcmlTaWduIENsYXNzIDEg UHVibGljIFByaW1hcnkgQ2VydGlmaWNhdGlvbiBBdXRob3JpdHkgLSBHM4IRAItbdVaEVIULAM+v OEjOsaQwDQYJKoZIhvcNAQEFBQADggEBADlNz0GZgbWpBbVSOOk5hIls5DSoWufYbAlMJBq6WaSH O3Mh8ZOBz79oY1pn/jWFK6HDXaNKwjoZ3TDWzE3v8dKBl8pUWkO/N4t6jhmND0OojPKvYLMVirOV nDzgnrMnmKQ1chfl/Cpdh9OKDcLRRSr4wPSsKpM61a4ScAjr+zvid+zoK2Q1ds262uDRyxTWcVib vtU+fbbZ6CTFJGZMXZEfdrMXPn8NxiGJL7M3uKH/XLJtSd5lUkL7DojS7Uodv0vj+Mxy+kgOZY5J yNb4mZg7t5Q+MXEGh/psWVMu198r7V9jAKwV7QO4VRaMxmgD5yKocwuxvKDaUljdCg5/wYIxggSL MIIEhwIBATCB8jCB3TELMAkGA1UEBhMCVVMxFzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMR8wHQYD VQQLExZWZXJpU2lnbiBUcnVzdCBOZXR3b3JrMTswOQYDVQQLEzJUZXJtcyBvZiB1c2UgYXQgaHR0 cHM6Ly93d3cudmVyaXNpZ24uY29tL3JwYSAoYykwOTEeMBwGA1UECxMVUGVyc29uYSBOb3QgVmFs aWRhdGVkMTcwNQYDVQQDEy5WZXJpU2lnbiBDbGFzcyAxIEluZGl2aWR1YWwgU3Vic2NyaWJlciBD QSAtIEczAhBXH0kUxtIzZJ6m/S1fRrhZMAkGBSsOAwIaBQCgggJtMBgGCSqGSIb3DQEJAzELBgkq hkiG9w0BBwEwHAYJKoZIhvcNAQkFMQ8XDTEyMDcxNjE2MTEzNVowIwYJKoZIhvcNAQkEMRYEFEki zL49ZUMDEZlmIVo4yhHQRlEsMIIBAwYJKwYBBAGCNxAEMYH1MIHyMIHdMQswCQYDVQQGEwJVUzEX MBUGA1UEChMOVmVyaVNpZ24sIEluYy4xHzAdBgNVBAsTFlZlcmlTaWduIFRydXN0IE5ldHdvcmsx OzA5BgNVBAsTMlRlcm1zIG9mIHVzZSBhdCBodHRwczovL3d3dy52ZXJpc2lnbi5jb20vcnBhIChj KTA5MR4wHAYDVQQLExVQZXJzb25hIE5vdCBWYWxpZGF0ZWQxNzA1BgNVBAMTLlZlcmlTaWduIENs YXNzIDEgSW5kaXZpZHVhbCBTdWJzY3JpYmVyIENBIC0gRzMCEFcfSRTG0jNknqb9LV9GuFkwggEF BgsqhkiG9w0BCRACCzGB9aCB8jCB3TELMAkGA1UEBhMCVVMxFzAVBgNVBAoTDlZlcmlTaWduLCBJ bmMuMR8wHQYDVQQLExZWZXJpU2lnbiBUcnVzdCBOZXR3b3JrMTswOQYDVQQLEzJUZXJtcyBvZiB1 c2UgYXQgaHR0cHM6Ly93d3cudmVyaXNpZ24uY29tL3JwYSAoYykwOTEeMBwGA1UECxMVUGVyc29u YSBOb3QgVmFsaWRhdGVkMTcwNQYDVQQDEy5WZXJpU2lnbiBDbGFzcyAxIEluZGl2aWR1YWwgU3Vi c2NyaWJlciBDQSAtIEczAhBXH0kUxtIzZJ6m/S1fRrhZMA0GCSqGSIb3DQEBAQUABIIBADx8z22h xi/cD9BvOkUE0+UCcce6vvrvcpjXvFyNWWctYBCyTLn9hjNQY5kaSxAcNF2VJ6/73J3tw6Iay8Tb iAROmURuhDbMtcs7iNZZCmc/g2yxAcBaJzv77Iliv6CN+x3aN7FHV6aWivDFI3mGa7y4AI6vVNGA o7Wre6IAA0YLi870u1qwLb1mYRR0MQRoCDLvJ0jr71g2bj9ErmLEsXMPoDFZmEvZhZVHz1tG4YCE WTgdZZ+PaY+jwc8dT/sr9F3RVr5uB/ElBniGO9/7HKi/DMS9VtBI6XVuW5wiXqZEurtB6Rj/UYzc BKmxaAwk9wZw/Ta7kfwu/0Exs6DbQucAAAAAAAA= --Apple-Mail=_AE6EA100-A584-49AE-AC58-7FB19C256E13-- From tsabatini@hotmail.com Sun Jul 15 12:58:59 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DD7E221F8472 for ; Sun, 15 Jul 2012 12:58:59 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.338 X-Spam-Level: X-Spam-Status: No, score=-2.338 tagged_above=-999 required=5 tests=[AWL=0.260, BAYES_00=-2.599, HTML_MESSAGE=0.001] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Guc4eA6MFVK9 for ; Sun, 15 Jul 2012 12:58:58 -0700 (PDT) Received: from bay0-omc2-s19.bay0.hotmail.com (bay0-omc2-s19.bay0.hotmail.com [65.54.190.94]) by ietfa.amsl.com (Postfix) with ESMTP id 3DBA421F8470 for ; Sun, 15 Jul 2012 12:58:58 -0700 (PDT) Received: from BAY158-W2 ([65.54.190.124]) by bay0-omc2-s19.bay0.hotmail.com with Microsoft SMTPSVC(6.0.3790.4675); Sun, 15 Jul 2012 12:59:40 -0700 Message-ID: Content-Type: multipart/alternative; boundary="_e275bef3-7fe1-440e-8be9-b1b14b8def39_" X-Originating-IP: [74.64.96.39] From: Anthony Sabatini To: Google Yuchung Cheng , Richard Scheffenegger Date: Sun, 15 Jul 2012 15:59:40 -0400 Importance: Normal In-Reply-To: References: , MIME-Version: 1.0 X-OriginalArrivalTime: 15 Jul 2012 19:59:40.0963 (UTC) FILETIME=[5EF1CF30:01CD62C4] X-Mailman-Approved-At: Mon, 16 Jul 2012 09:49:38 -0700 Cc: tcpm Subject: Re: [tcpm] TCP Error recovery and efficiency X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Jul 2012 19:59:00 -0000 --_e275bef3-7fe1-440e-8be9-b1b14b8def39_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable All - First off a thank you to Richard Scheffenegger for his comments but most im= portantly when he asked whether the number of ACKs being sent was excessive= (since we are trying for efficiency unnecessary ACKs diminish that efficie= ncy). When I first wrote this RFC six years ago for my thesis=2C my concer= n was with my extensions not in reanalyzing the base protocol. This turns = out to be a mistake because=2C by moving the retransmission process farther= away from using timers and changing it to purely a state basis=2C a number= of statements/features of the base protocol no longer work the same. As a statement of policy SACK should send no more ACKs then a current moder= n version of TCP does. For the most part SACK only adds from 2 to 34 bytes= in the options area of existing messages. Even with this change we are on= ly talking 6 to 38 bytes. It is only in extreme situations where there are= more than four gaps in the acknowledged data that SACK requires additional= ACKs. As to when these ACKs are sent again normal TCP practice applies fo= r the most part (as long as all SACK blocks occupy a single ACK). The recovery basis in Enhanced SACK (E-SACK) except for the "Link Broken" d= isaster timer is entirely state driven on the transmit side. Since timers = by their very nature=2C must be set to the worst case value=2C this automat= ed retransmission is much quicker and more accurate. The combination of TC= P ACKs=2C SACK block changes and Returned Transmit Token changes form the t= riggers that guide the recovery and transmission processes. In order to in= sure the proper flow of those signals the receiver is running three classes= of timer - Returned Transmit Token delay timer=2C Second ACK timer and the= "I did not receive what I needed" keep alive timer. More about those in t= he commentary. The principle that all state changing information should be sent twice (sec= ond ACK) from TCP is a good one and should be preserved in this protocol si= nce the loss of an ACK is as disastrous as the loss of a message=2C we just= need to be cognizant that there are now three state change elements=2C any= of which triggers the ACK pair=2C specifically the TCP ACK (acknowledging = a change in the TCP transmit floor)=2C a change to a SACK block=2C and fina= lly a change to one of the tokens (note I Include both Transmit and Returni= ng here). The question then becomes how do we implement this. The underly= ing reality here is that on a link with even moderate activity a second int= entional ACK is not needed since the ACK triggered by the next arriving mes= sage would replace it. The problem we encounter is that ACK messages are s= ignificantly shorter than full data messages so we would always send a seco= nd ACK if we send it immediately after the first since nothing has changed.= The thought is then to delay the second ACK until the link goes idle (1.3= times the length of time to receive a full packet on the fast side to 2.6 = x the length of time to receive a full packet on the slow side). Note that= this delay has a direct effect on the Returned Transmit Token delay timer = as that timer can not be less than this delay plus the amount of time to tr= ansmit all of the ACKs to hold the complete list of SACK blocks in order to= insure that state information is up to date before the token changes=2C th= us possibly degrading link efficiency. One note of clarification=2C if a state change triggers another pair of ACK= s=2C this takes precedence over completing the remainder of the ACK group i= f multiple ACKs are required to transmit the SACK blocks. The reason we trigger an ACK on either token changing is to get around the = dreaded "last character hang" of very early TCP implementations. If the tr= ansmitter sends an ACK with the updated Transmit Token from the last outgoi= ng segment when the link goes idle it insures the receiver has the updated = value of that Transmit Token which will lead to the retransmission of that = last missing segment. We delay the return of the received Transmit Token for two reasons=2C first= it allows packets that got out of order to be received and put back in ord= er before being declared missing and secondly=2C because a group of SACK bl= ocks might span multiple ACKs=2C to make sure that state information is up = to date before we declare a segment as missing. This leads the to a discussion of very bad burst mode errors. If the recei= ver has not received the segments it requires and has not sent out an ACK f= or RTT*1.3 then the receiver needs to send another ACK pair. Note this app= lies to the transmitter also in that if its Returned Transmit Token does no= t equal Last Transmit Token after the same RTT*1.3 it is obligated to send = an ACK pair to the receiver to prompt it to restart. Obviously RTT*1.3 is = a long time on transcontinental or satellite links which is why 1/4 RTT for= an ACK pair from the receiver as a keep alive is under consideration=2C th= e maximum length of time to cover the burst error without degrading efficie= ncy too badly. In filling the outgoing ACKs with SACK blocks at the receiver=2C the list i= s maintained in the order oldest (smallest segment start) to the youngest. = The list is queried three times when building the ACK group=2C first for a= ll blocks that just changed and thus have the most critical information (AC= K NEEDED =3D 2)=2C then a scan for those blocks which already had their fir= st ACK (ACK NEEDED =3D 1) and then for the blocks which have already been t= ransmitted twice (ACK NEEDED =3D 0). The ACK NEEDED count is then decremen= ted.=20 To make E-SACK work three lists must be maintained=2C one at the receiver a= nd two at the transmitter. The one in the receiver is identical in concept= to the one in the original RFC with the addition of ACK NEEDED and the pro= viso that it be in oldest to youngest order. At the transmitter an identic= al list is maintained built from the recovered SACK blocks. When a changed= Returned Transmit Token is received=2C the recovered SACK block list is us= ed to construct a retransmit queue of all unacknowledged segments. Then st= arting at the transmit queue entry plus one derived from the Returned Trans= mit Token=2C and proceeding forward until all transmitted segment extents h= ave been processed=2C entries are removed or modified in the new retransmit= queue to compensate for segments sent after the time of the Returned Trans= mit Token. The adjusted new transmit queue replaces the existing queue and= the transmission process is started again starting at the first entry on t= he queue=2C not proceeding with new transmission until the queue is complet= ed. Please note that incoming SACK blocks must be checked against the retr= ansmit queue in order to remove segments that have been subsequently correc= tly received. Noto Bene - This protocol is an update to RFC 2018. It is not compatible w= ith nor do I believe in the philosophy of RFC2883/RFC3708 as the whole purp= ose of this change is to eliminate duplicate segments and increase efficien= cy and reliability where the changes outlined in those RFCs degrade efficie= ncy (especially by generating extra SACK blocks) without substantially impr= oving anything. Also as adopted in the RFC there is a major fallacy=2C it = assumes that there can be only one SACK block with a status change where in= reality=2C due to processing considerations=2C there can be two or even mo= re blocks with changes before an ACK can be sent. Tony Anthony Sabatini 200 West 20th Street Apt. 1216 New York=2C NY 10011 Phone: (212) 867-7179 Mobile: (917) 224-8388 = --_e275bef3-7fe1-440e-8be9-b1b14b8def39_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
All -

First off a thank you to Richard Scheffenegger for his comment= s but most importantly when he asked whether the number of ACKs being sent = was excessive (since we are trying for efficiency unnecessary ACKs diminish= that efficiency). =3B When I first wrote this RFC six years ago for my= thesis=2C my concern was with my extensions not in reanalyzing the base pr= otocol. =3B This turns out to be a mistake because=2C by moving the ret= ransmission process farther away from using timers and changing it to purel= y a state basis=2C a number of statements/features of the base protocol no = longer work the same.

As a statement of policy SACK should send no m= ore ACKs then a current modern version of TCP does. =3B For the most pa= rt SACK only adds from 2 to 34 bytes in the options area of existing messag= es. =3B Even with this change we are only talking 6 to 38 bytes. = =3B It is only in extreme situations where there are more than four gaps in= the acknowledged data that SACK requires additional ACKs. =3B As to wh= en these ACKs are sent again normal TCP practice applies for the most part = (as long as all SACK blocks occupy a single ACK).

The recovery basis= in Enhanced SACK (E-SACK) except for the "Link Broken" disaster timer is e= ntirely state driven on the transmit side. =3B Since timers by their ve= ry nature=2C must be set to the worst case value=2C this automated retransm= ission is much quicker and more accurate. =3B The combination of TCP AC= Ks=2C SACK block changes and Returned Transmit Token changes form the trigg= ers that guide the recovery and transmission processes. =3B In order to= insure the proper flow of those signals the receiver is running three clas= ses of timer - Returned Transmit Token delay timer=2C Second ACK timer and = the "I did not receive what I needed" keep alive timer. =3B More about = those in the commentary.

The principle that all state changing infor= mation should be sent twice (second ACK) from TCP is a good one and should = be preserved in this protocol since the loss of an ACK is as disastrous as = the loss of a message=2C we just need to be cognizant that there are now th= ree state change elements=2C any of which triggers the ACK pair=2C specific= ally the TCP ACK (acknowledging a change in the TCP transmit floor)=2C a ch= ange to a SACK block=2C and finally a change to one of the tokens (note I I= nclude both Transmit and Returning here). =3B The question then becomes= how do we implement this. =3B The underlying reality here is that on a= link with even moderate activity a second intentional ACK is not needed si= nce the ACK triggered by the next arriving message would replace it. = =3B The problem we encounter is that ACK messages are significantly shorter= than full data messages so we would always send a second ACK if we send it= immediately after the first since nothing has changed. =3B The thought= is then to delay the second ACK until the link goes idle (1.3 times the le= ngth of time to receive a full packet on the fast side to 2.6 x the length = of time to receive a full packet on the slow side). =3B Note that this = delay has a direct effect on the Returned Transmit Token delay timer as tha= t timer can not be less than this delay plus the amount of time to transmit= all of the ACKs to hold the complete list of SACK blocks in order to insur= e that state information is up to date before the token changes=2C thus pos= sibly degrading link efficiency.

One note of clarification=2C if a s= tate change triggers another pair of ACKs=2C this takes precedence over com= pleting the remainder of the ACK group if multiple ACKs are required to tra= nsmit the SACK blocks.

The reason we trigger an ACK on either token = changing is to get around the dreaded "last character hang" of very early T= CP implementations. =3B If the transmitter sends an ACK with the update= d Transmit Token from the last outgoing segment when the link goes idle it = insures the receiver has the updated value of that Transmit Token which wil= l lead to the retransmission of that last missing segment.

We delay = the return of the received Transmit Token for two reasons=2C first it allow= s packets that got out of order to be received and put back in order before= being declared missing and secondly=2C because a group of SACK blocks migh= t span multiple ACKs=2C to make sure that state information is up to date b= efore we declare a segment as missing.

This leads the to a discussio= n of very bad burst mode errors. =3B If the receiver has not received t= he segments it requires and has not sent out an ACK for RTT*1.3 then the re= ceiver needs to send another ACK pair. =3B Note this applies to the tra= nsmitter also in that if its Returned Transmit Token does not equal Last Tr= ansmit Token after the same RTT*1.3 it is obligated to send an ACK pair to = the receiver to prompt it to restart. =3B Obviously RTT*1.3 is a long t= ime on transcontinental or satellite links which is why 1/4 RTT for an ACK = pair from the receiver as a keep alive is under consideration=2C the maximu= m length of time to cover the burst error without degrading efficiency too = badly.

In filling the outgoing ACKs with SACK blocks at the receiver= =2C the list is maintained in the order oldest (smallest segment start) to = the youngest. =3B The list is queried three times when building the ACK= group=2C first for all blocks that just changed and thus have the most cri= tical information (ACK NEEDED =3D 2)=2C then a scan for those blocks which = already had their first ACK (ACK NEEDED =3D 1) and then for the blocks whic= h have already been transmitted twice (ACK NEEDED =3D 0). =3B The ACK N= EEDED count is then decremented.

To make E-SACK work three lists mu= st be maintained=2C one at the receiver and two at the transmitter. =3B= The one in the receiver is identical in concept to the one in the original= RFC with the addition of ACK NEEDED and the proviso that it be in oldest t= o youngest order. =3B At the transmitter an identical list is maintaine= d built from the recovered SACK blocks. =3B When a changed Returned Tra= nsmit Token is received=2C the recovered SACK block list is used to constru= ct a retransmit queue of all unacknowledged segments. =3B Then starting= at the transmit queue entry plus one derived from the Returned Transmit To= ken=2C and proceeding forward until all transmitted segment extents have be= en processed=2C entries are removed or modified in the new retransmit queue= to compensate for segments sent after the time of the Returned Transmit To= ken. =3B The adjusted new transmit queue replaces the existing queue an= d the transmission process is started again starting at the first entry on = the queue=2C not proceeding with new transmission until the queue is comple= ted. =3B Please note that incoming SACK blocks must be checked against = the retransmit queue in order to remove segments that have been subsequentl= y correctly received.

Noto Bene - This protocol is an update to RFC = 2018. =3B It is not compatible with nor do I believe in the philosophy = of RFC2883/RFC3708 as the whole purpose of this change is to eliminate dupl= icate segments and increase efficiency and reliability where the changes ou= tlined in those RFCs degrade efficiency (especially by generating extra SAC= K blocks) without substantially improving anything. =3B Also as adopted= in the RFC there is a major fallacy=2C it assumes that there can be only o= ne SACK block with a status change where in reality=2C due to processing co= nsiderations=2C there can be two or even more blocks with changes before an= ACK can be sent.

Tony

Anthony Sabatini
200 =3BWest 20= th Street
Apt. 1216
New York=2C NY 10011

Phone: (212) 867-7179=
Mobile: (917) 224-8388

= --_e275bef3-7fe1-440e-8be9-b1b14b8def39_-- From michael.scharf@alcatel-lucent.com Mon Jul 16 10:00:00 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 62C1D11E8163 for ; Mon, 16 Jul 2012 10:00:00 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.027 X-Spam-Level: X-Spam-Status: No, score=-10.027 tagged_above=-999 required=5 tests=[AWL=0.222, BAYES_00=-2.599, HELO_EQ_FR=0.35, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xAAJ46RUPTWU for ; Mon, 16 Jul 2012 09:59:59 -0700 (PDT) Received: from smail5.alcatel.fr (smail5.alcatel.fr [64.208.49.27]) by ietfa.amsl.com (Postfix) with ESMTP id 484CE11E815E for ; Mon, 16 Jul 2012 09:59:59 -0700 (PDT) Received: from FRMRSSXCHHUB03.dc-m.alcatel-lucent.com (FRMRSSXCHHUB03.dc-m.alcatel-lucent.com [135.120.45.63]) by smail5.alcatel.fr (8.14.3/8.14.3/ICT) with ESMTP id q6GH0gsw030945 (version=TLSv1/SSLv3 cipher=RC4-MD5 bits=128 verify=NOT); Mon, 16 Jul 2012 19:00:42 +0200 Received: from FRMRSSXCHMBSE3.dc-m.alcatel-lucent.com ([135.120.45.55]) by FRMRSSXCHHUB03.dc-m.alcatel-lucent.com ([135.120.45.63]) with mapi; Mon, 16 Jul 2012 19:00:42 +0200 From: "Scharf, Michael (Michael)" To: "Eggert, Lars" Date: Mon, 16 Jul 2012 19:00:40 +0200 Thread-Topic: bot? cuttyroyce@yahoo.com Thread-Index: AQHNY22sjOqvC5k3eU6vMNwbqwnLbZcsITgg Message-ID: <2A886F9088894347A3BE0CC5B7A85F3E891A9BBADB@FRMRSSXCHMBSE3.dc-m.alcatel-lucent.com> References: <1342449345.2366.YahooMailNeo@web125003.mail.ne1.yahoo.com> <00B78AB4-72CB-40B6-BE25-6197A3A7EEF9@netapp.com> In-Reply-To: <00B78AB4-72CB-40B6-BE25-6197A3A7EEF9@netapp.com> Accept-Language: de-DE, en-US Content-Language: de-DE X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: de-DE, en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.69 on 155.132.188.13 Cc: "tcpm@ietf.org" Subject: Re: [tcpm] bot? cuttyroyce@yahoo.com X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Jul 2012 17:00:00 -0000 I removed the address from the list. Michael > -----Original Message----- > From: tcpm-bounces@ietf.org [mailto:tcpm-bounces@ietf.org] On=20 > Behalf Of Eggert, Lars > Sent: Monday, July 16, 2012 6:12 PM > To: tcpm@ietf.org > Subject: [tcpm] bot? cuttyroyce@yahoo.com >=20 > Can we block this address from the list? I suspect it's a bot=20 > that spews Wikipedia paragraphs to the list. (Google for some=20 > of the sentences from the recent emails.) >=20 > Lars= From touch@isi.edu Mon Jul 16 10:05:16 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C529F11E8192 for ; Mon, 16 Jul 2012 10:05:16 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.599 X-Spam-Level: X-Spam-Status: No, score=-102.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1G9NNIHdI9Ib for ; Mon, 16 Jul 2012 10:05:15 -0700 (PDT) Received: from darkstar.isi.edu (darkstar.isi.edu [128.9.128.127]) by ietfa.amsl.com (Postfix) with ESMTP id CE07411E8181 for ; Mon, 16 Jul 2012 10:05:13 -0700 (PDT) Received: from [75.211.171.201] (201.sub-75-211-171.myvzw.com [75.211.171.201]) (authenticated bits=0) by darkstar.isi.edu (8.13.8/8.13.8) with ESMTP id q6GH4Utf004874 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Mon, 16 Jul 2012 10:04:54 -0700 (PDT) Message-ID: <5004499F.6030607@isi.edu> Date: Mon, 16 Jul 2012 10:04:31 -0700 From: Joe Touch User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:13.0) Gecko/20120614 Thunderbird/13.0.1 MIME-Version: 1.0 To: Anthony Sabatini References: , <012C3117EDDB3C4781FD802A8C27DD4F080D40D3@SACEXCMBX02-PRD.hq.netapp.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Cc: tcpm , draft-sack@tsabatini.com Subject: Re: [tcpm] TCP Error recovery and efficiency X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Jul 2012 17:05:17 -0000 On 7/11/2012 8:35 AM, Anthony Sabatini wrote: > Richard - > > First off thank you for taking the time to read the draft. > > With respect to the option numbers, as I wrote to the tcpm chairs - I > can not presume in a draft what option numbers will be assigned in a > standard but I needed a place holder for illustration. I therefore used > the ones in SACK to show how they relate and to guarantee in the final > document they will be corrected to the assigned appropriate values. You should be using either "TBD" or a number that won't fit (256). You should NEVER use an existing assignment unless you intend to be compatible with that assignment. You should NEVER use a currently unassigned value. Specific values will be determined by IANA in the late stages of publication, not before. Joe From tsabatini@hotmail.com Mon Jul 16 10:36:53 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E797211E8252 for ; Mon, 16 Jul 2012 10:36:53 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.381 X-Spam-Level: X-Spam-Status: No, score=-2.381 tagged_above=-999 required=5 tests=[AWL=0.217, BAYES_00=-2.599, HTML_MESSAGE=0.001] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Kz5d0wvma84t for ; Mon, 16 Jul 2012 10:36:53 -0700 (PDT) Received: from bay0-omc2-s26.bay0.hotmail.com (bay0-omc2-s26.bay0.hotmail.com [65.54.190.101]) by ietfa.amsl.com (Postfix) with ESMTP id 3171811E824E for ; Mon, 16 Jul 2012 10:36:53 -0700 (PDT) Received: from BAY158-W28 ([65.54.190.124]) by bay0-omc2-s26.bay0.hotmail.com with Microsoft SMTPSVC(6.0.3790.4675); Mon, 16 Jul 2012 10:37:38 -0700 Message-ID: Content-Type: multipart/alternative; boundary="_ac06eeac-43dc-46b8-8c94-1153e09bc565_" X-Originating-IP: [74.64.96.39] From: Anthony Sabatini To: Date: Mon, 16 Jul 2012 13:37:37 -0400 Importance: Normal In-Reply-To: <5004499F.6030607@isi.edu> References: , <012C3117EDDB3C4781FD802A8C27DD4F080D40D3@SACEXCMBX02-PRD.hq.netapp.com> , <5004499F.6030607@isi.edu> MIME-Version: 1.0 X-OriginalArrivalTime: 16 Jul 2012 17:37:38.0591 (UTC) FILETIME=[B1A0CAF0:01CD6379] Cc: tcpm , draft-sack@tsabatini.com Subject: Re: [tcpm] TCP Error recovery and efficiency X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Jul 2012 17:36:54 -0000 --_ac06eeac-43dc-46b8-8c94-1153e09bc565_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Joe - As I mentioned this will be addressed in the next release of the draft=2C t= his draft was there to get a discussion going and to work out the final imp= lementation details=2C I used the same numbers in the discussion document t= o correspond them to the existing RFC2018 for comparison=2C that as previou= sly stated was probably a mistake. What are your comments on the mechanics= of the protocol? Tony Anthony Sabatini 200 West 20th Street Apt. 1216 New York=2C NY 10011 Phone: (212) 867-7179 Mobile: (917) 224-8388 =20 > Date: Mon=2C 16 Jul 2012 10:04:31 -0700 > From: touch@isi.edu > To: tsabatini@hotmail.com > CC: rs@netapp.com=3B tcpm@ietf.org=3B draft-sack@tsabatini.com > Subject: Re: [tcpm] TCP Error recovery and efficiency >=20 >=20 >=20 > On 7/11/2012 8:35 AM=2C Anthony Sabatini wrote: > > Richard - > > > > First off thank you for taking the time to read the draft. > > > > With respect to the option numbers=2C as I wrote to the tcpm chairs - I > > can not presume in a draft what option numbers will be assigned in a > > standard but I needed a place holder for illustration. I therefore use= d > > the ones in SACK to show how they relate and to guarantee in the final > > document they will be corrected to the assigned appropriate values. >=20 > You should be using either "TBD" or a number that won't fit (256). You=20 > should NEVER use an existing assignment unless you intend to be=20 > compatible with that assignment. >=20 > You should NEVER use a currently unassigned value. >=20 > Specific values will be determined by IANA in the late stages of=20 > publication=2C not before. >=20 > Joe = --_ac06eeac-43dc-46b8-8c94-1153e09bc565_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
Joe -

As I mentioned this will be addressed in the next release of t= he draft=2C this draft was there to get a discussion going and to work out = the final implementation details=2C I used the same numbers in the discussi= on document to correspond them to the existing RFC2018 for comparison=2C th= at as previously stated was probably a mistake. =3B What are your comme= nts on the mechanics of the protocol?

Tony

Anthony Sabatini200 =3BWest 20th Street
Apt. 1216
New York=2C NY 10011

P= hone: (212) 867-7179
Mobile: (917) 224-8388

 =3B


<= div>
>=3B Date: Mon=2C 16 Jul 2012 1= 0:04:31 -0700
>=3B From: touch@isi.edu
>=3B To: tsabatini@hotmail= .com
>=3B CC: rs@netapp.com=3B tcpm@ietf.org=3B draft-sack@tsabatini.c= om
>=3B Subject: Re: [tcpm] TCP Error recovery and efficiency
>= =3B
>=3B
>=3B
>=3B On 7/11/2012 8:35 AM=2C Anthony Sabati= ni wrote:
>=3B >=3B Richard -
>=3B >=3B
>=3B >=3B Firs= t off thank you for taking the time to read the draft.
>=3B >=3B
= >=3B >=3B With respect to the option numbers=2C as I wrote to the tcpm = chairs - I
>=3B >=3B can not presume in a draft what option numbers = will be assigned in a
>=3B >=3B standard but I needed a place holder= for illustration. I therefore used
>=3B >=3B the ones in SACK to s= how how they relate and to guarantee in the final
>=3B >=3B document= they will be corrected to the assigned appropriate values.
>=3B
&= gt=3B You should be using either "TBD" or a number that won't fit (256). Yo= u
>=3B should NEVER use an existing assignment unless you intend to b= e
>=3B compatible with that assignment.
>=3B
>=3B You shou= ld NEVER use a currently unassigned value.
>=3B
>=3B Specific va= lues will be determined by IANA in the late stages of
>=3B publicatio= n=2C not before.
>=3B
>=3B Joe
= --_ac06eeac-43dc-46b8-8c94-1153e09bc565_-- From touch@isi.edu Mon Jul 16 10:45:37 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A690111E826D for ; Mon, 16 Jul 2012 10:45:37 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.599 X-Spam-Level: X-Spam-Status: No, score=-102.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id IvyI8jEY0rTy for ; Mon, 16 Jul 2012 10:45:37 -0700 (PDT) Received: from darkstar.isi.edu (darkstar.isi.edu [128.9.128.127]) by ietfa.amsl.com (Postfix) with ESMTP id 158DF11E8273 for ; Mon, 16 Jul 2012 10:45:37 -0700 (PDT) Received: from [75.211.171.201] (201.sub-75-211-171.myvzw.com [75.211.171.201]) (authenticated bits=0) by darkstar.isi.edu (8.13.8/8.13.8) with ESMTP id q6GHk4Qm016917 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Mon, 16 Jul 2012 10:46:08 -0700 (PDT) Message-ID: <5004535D.8050003@isi.edu> Date: Mon, 16 Jul 2012 10:46:05 -0700 From: Joe Touch User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:13.0) Gecko/20120614 Thunderbird/13.0.1 MIME-Version: 1.0 To: Anthony Sabatini References: , <012C3117EDDB3C4781FD802A8C27DD4F080D40D3@SACEXCMBX02-PRD.hq.netapp.com> , <5004499F.6030607@isi.edu> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Cc: tcpm , draft-sack@tsabatini.com Subject: Re: [tcpm] TCP Error recovery and efficiency X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Jul 2012 17:45:37 -0000 I haven't looked at the protocol; there are others more familiar with SACK thank I. Joe On 7/16/2012 10:37 AM, Anthony Sabatini wrote: > Joe - > > As I mentioned this will be addressed in the next release of the draft, > this draft was there to get a discussion going and to work out the final > implementation details, I used the same numbers in the discussion > document to correspond them to the existing RFC2018 for comparison, that > as previously stated was probably a mistake. What are your comments on > the mechanics of the protocol? > > Tony > > Anthony Sabatini > 200 West 20th Street > Apt. 1216 > New York, NY 10011 > > Phone: (212) 867-7179 > Mobile: (917) 224-8388 > > > > > > Date: Mon, 16 Jul 2012 10:04:31 -0700 > > From: touch@isi.edu > > To: tsabatini@hotmail.com > > CC: rs@netapp.com; tcpm@ietf.org; draft-sack@tsabatini.com > > Subject: Re: [tcpm] TCP Error recovery and efficiency > > > > > > > > On 7/11/2012 8:35 AM, Anthony Sabatini wrote: > > > Richard - > > > > > > First off thank you for taking the time to read the draft. > > > > > > With respect to the option numbers, as I wrote to the tcpm chairs - I > > > can not presume in a draft what option numbers will be assigned in a > > > standard but I needed a place holder for illustration. I therefore used > > > the ones in SACK to show how they relate and to guarantee in the final > > > document they will be corrected to the assigned appropriate values. > > > > You should be using either "TBD" or a number that won't fit (256). You > > should NEVER use an existing assignment unless you intend to be > > compatible with that assignment. > > > > You should NEVER use a currently unassigned value. > > > > Specific values will be determined by IANA in the late stages of > > publication, not before. > > > > Joe From internet-drafts@ietf.org Mon Jul 16 12:01:34 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 43EF311E82CF; Mon, 16 Jul 2012 12:01:34 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.475 X-Spam-Level: X-Spam-Status: No, score=-102.475 tagged_above=-999 required=5 tests=[AWL=0.124, BAYES_00=-2.599, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jkuxjpYRyxY7; Mon, 16 Jul 2012 12:01:33 -0700 (PDT) Received: from ietfa.amsl.com (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6974E21F8864; Mon, 16 Jul 2012 12:01:28 -0700 (PDT) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: internet-drafts@ietf.org To: i-d-announce@ietf.org X-Test-IDTracker: no X-IETF-IDTracker: 4.30p3 Message-ID: <20120716190128.23960.13322.idtracker@ietfa.amsl.com> Date: Mon, 16 Jul 2012 12:01:28 -0700 Cc: tcpm@ietf.org Subject: [tcpm] I-D Action: draft-ietf-tcpm-proportional-rate-reduction-02.txt X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Jul 2012 19:01:34 -0000 A New Internet-Draft is available from the on-line Internet-Drafts director= ies. This draft is a work item of the TCP Maintenance and Minor Extensions Work= ing Group of the IETF. Title : Proportional Rate Reduction for TCP Author(s) : Matt Mathis Nandita Dukkipati Yuchung Cheng Filename : draft-ietf-tcpm-proportional-rate-reduction-02.txt Pages : 16 Date : 2012-07-16 Abstract: This document describes an experimental algorithm, Proportional Rate Reduction (PPR) to improve the accuracy of the amount of data sent by TCP during loss recovery. Standard Congestion Control requires that TCP and other protocols reduce their congestion window in response to losses. This window reduction naturally occurs in the same round trip as the data retransmissions to repair the losses, and is implemented by choosing not to transmit any data in response to some ACKs arriving from the receiver. Two widely deployed algorithms are used to implement this window reduction: Fast Recovery and Rate Halving. Both algorithms are needlessly fragile under a number of conditions, particularly when there is a burst of losses such that the number of ACKs returning to the sender is small. Proportional Rate Reduction minimizes these excess window reductions such that at the end of recovery the actual window size will be as close as possible to ssthresh, the window size determined by the congestion control algorithm. It is patterned after Rate Halving, but using the fraction that is appropriate for target window chosen by the congestion control algorithm. The IETF datatracker status page for this draft is: https://datatracker.ietf.org/doc/draft-ietf-tcpm-proportional-rate-reduction There's also a htmlized version available at: http://tools.ietf.org/html/draft-ietf-tcpm-proportional-rate-reduction-02 A diff from previous version is available at: http://tools.ietf.org/rfcdiff?url2=3Ddraft-ietf-tcpm-proportional-rate-redu= ction-02 Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ From internet-drafts@ietf.org Mon Jul 16 12:28:26 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1CA5E21F891F; Mon, 16 Jul 2012 12:28:26 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.491 X-Spam-Level: X-Spam-Status: No, score=-102.491 tagged_above=-999 required=5 tests=[AWL=0.108, BAYES_00=-2.599, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TUQx55DQdHnM; Mon, 16 Jul 2012 12:28:25 -0700 (PDT) Received: from ietfa.amsl.com (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2E1F121F8918; Mon, 16 Jul 2012 12:28:25 -0700 (PDT) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: internet-drafts@ietf.org To: i-d-announce@ietf.org X-Test-IDTracker: no X-IETF-IDTracker: 4.30p3 Message-ID: <20120716192825.21229.2707.idtracker@ietfa.amsl.com> Date: Mon, 16 Jul 2012 12:28:25 -0700 Cc: tcpm@ietf.org Subject: [tcpm] I-D Action: draft-ietf-tcpm-fastopen-01.txt X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Jul 2012 19:28:26 -0000 A New Internet-Draft is available from the on-line Internet-Drafts director= ies. This draft is a work item of the TCP Maintenance and Minor Extensions Work= ing Group of the IETF. Title : TCP Fast Open Author(s) : Yuchung Cheng Jerry Chu Sivasankar Radhakrishnan Arvind Jain Filename : draft-ietf-tcpm-fastopen-01.txt Pages : 22 Date : 2012-07-16 Abstract: TCP Fast Open (TFO) allows data to be carried in the SYN and SYN-ACK packets and consumed by the receiving end during the initial connection handshake, thus providing a saving of up to one full round trip time (RTT) compared to standard TCP requiring a three-way handshake (3WHS) to complete before data can be exchanged. Terminology The IETF datatracker status page for this draft is: https://datatracker.ietf.org/doc/draft-ietf-tcpm-fastopen There's also a htmlized version available at: http://tools.ietf.org/html/draft-ietf-tcpm-fastopen-01 A diff from previous version is available at: http://tools.ietf.org/rfcdiff?url2=3Ddraft-ietf-tcpm-fastopen-01 Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ From ycheng@google.com Mon Jul 16 15:55:41 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 595C511E80EB for ; Mon, 16 Jul 2012 15:55:41 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.877 X-Spam-Level: X-Spam-Status: No, score=-102.877 tagged_above=-999 required=5 tests=[AWL=0.100, BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rhVfH5j287Yn for ; Mon, 16 Jul 2012 15:55:40 -0700 (PDT) Received: from mail-ob0-f172.google.com (mail-ob0-f172.google.com [209.85.214.172]) by ietfa.amsl.com (Postfix) with ESMTP id B49CD21F87A2 for ; Mon, 16 Jul 2012 15:55:39 -0700 (PDT) Received: by obbwc20 with SMTP id wc20so11361737obb.31 for ; Mon, 16 Jul 2012 15:56:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:from:date:message-id:subject:to:cc:content-type :x-system-of-record; bh=nVpCTsc1cHouA8oMB8mowLXhs8xhZf2pYEUQijIXGwg=; b=hs2fEvy/espmLYKb9kVZznzW2opaDAAaH319rdDt0P5VGElpUYX9f2XvOZMzWkI/jU e7YjkXrwaFefqsPsa+L4zzZKjnw5lEulF85CvOa7WkAh4jjBgro4Z69iKrszyhNOeok1 0g7VZIv/ku+ULGNrsyNXcnUOUogkp/amMsUh0udLX0OXSSpvNwbDHHN5KypFdr8OLADg I+EcI8X2XLSSGtQSm17UDNXzD+2/5J6SHb86FKZkYpzMzzxlb7TmLyhEqkTxoWwP7POG G7/3YMFJC6IuxLPfEVEt+/BQB4K0B8gSN0Um8hvXNEDnElEVEx83HobHMT78EWKO9w+e FqVQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:from:date:message-id:subject:to:cc:content-type :x-system-of-record:x-gm-message-state; bh=nVpCTsc1cHouA8oMB8mowLXhs8xhZf2pYEUQijIXGwg=; b=RRK/yE6HNTPgKOBTPvX1ou/7yDKOCgOXoIWwiDbJP22ST1aTxHHgLkh05ekR0cVVG/ 4iXCHc2SpSqc5g6Bfc+45uCdvGc4kI/GEx3dJ2uf+8rE+/iBY5p9AY1yYlwkITscOp+A fDN6PWj3Aav/YoPA/58eqFOtJQrT1QcJ4kttpifKiAUY6QeDTkDxNvnS61R5Qb2J67Jz QmAZT3gmZ7YIPToA75r5DvZcnT4Z1/L9uZeTZcq4vM5ot7XoB6kmyunnHueXbyy2LNP8 m9OPODEUxGl7rfNf9X3GFuyPebnyN94Dl3HrffgpO07e18ssV8o00hGdlwV4jawqtzJZ JBsw== Received: by 10.182.231.6 with SMTP id tc6mr145696obc.63.1342479385375; Mon, 16 Jul 2012 15:56:25 -0700 (PDT) Received: by 10.182.231.6 with SMTP id tc6mr145671obc.63.1342479385096; Mon, 16 Jul 2012 15:56:25 -0700 (PDT) MIME-Version: 1.0 Received: by 10.60.14.168 with HTTP; Mon, 16 Jul 2012 15:56:04 -0700 (PDT) From: Yuchung Cheng Date: Mon, 16 Jul 2012 15:56:04 -0700 Message-ID: To: "tcpm@ietf.org Extensions" Content-Type: text/plain; charset=ISO-8859-1 X-System-Of-Record: true X-Gm-Message-State: ALoCoQnGrzciyYpHQhUIRwh7D3owZOQCmJFYK0XEQouHYq7cwKVVP+MV+g7mJJAhBstnJpURy7exGADzjSXrT3LNSD+M7ewyEVCb5QJZbLwV8VHEmmZmzLY8tnUXLpP1PIyGrZFvj+N7pjvIhR/1M6MvBEMUjI8clG9p+K9L+IQ302YOlImP9k0= Cc: "Arjuna Sathiaseelan \(work\)" , "iccrg@cs.ucl.ac.uk" Subject: [tcpm] Review of draft-fairhurst-tcpm-newcwv-03 X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Jul 2012 22:55:41 -0000 Summary: First of all the problem this draft is trying to solve is important: AFAIK servers and data-centers disable slow-start after idle because it simply hurts latency too badly. Modern TCP-smart applications all use persistent connections to amortize handshake and slow-start overhead and the RFC2861 standard nullifies that to certain extent. While the applications don't want to create big burst to cause congestion, they have no better choices (low latency vs occasional burst losses: choose one). This draft proposes a middle road. Thanks for starting this! Details: Section 1 I found the term "variable-rate" application little awkward: for example some constant-rate (CBR) app may still go through slow-start, overshoots, and experiences non-validated phases. But it's nit. Section 4.1 Since the draft already requires SACK, I recommend to use RFC3517 pipe to better measure the amount of outstanding data in the network. Section 4.2 a) Justification of 2/3 and 1/3 is much appreciated. b) During the validated phase when FS >= 2/3cwnd, are ACKs used to compute new cwnd. E.g., in slow-start with cwnd=30 and FS=20, will the next 10 ACKs increase the cwnd to 40? also is this phase check applied to both transmitting new data and receiving acks? Section 4.3.1 " An application that remains in the non-validated phase for a period greater than five minutes is required to adjust its congestion control state. At the end of the non-validated phase, the sender MUST update cwnd: cwnd = max(FlightSize*2, IW)." Some justification to set cwnd to double the FlightSize? it seems with 2/3 rule in the non-validated phase, cwnd can grow to as big as 4/3 of original value at the end of the non-validated phase. btw, it flows better if 4.3.1 and 4.3.2 are swapped. Section 4.3.2 "A sender that detects a packet-drop or receives an ECN marked packet MUST calculate a safe cwnd, based on the volume of acknowledged data: cwnd = FlightSize - R. Where, R is the volume of data that was reported as unacknowledged by the SACK information. This follows the method proposed for Jump Start [[Liu07]." so is R 0 or 1 of of an ECN marked packet? Some justification to your approach compared to RFC5861? i.e. why (FlightSize - R)/2 vs FlightSize/2? After thougths: Since this RFC attempts to replace RFC2861, maybe it can also address this performance handicap in the introduction of RFC2861? "We propose that the TCP sender should not increase the congestion window when the TCP sender has been application-limited (and therefore has not fully used the current congestion window). We have explored these algorithms both with simulations and with experiments from an implementation in FreeBSD." This happens frequently today because many application pauses are within RTOs. For example, the acks of smaller HTTP responses after some big one will now not used to compute cwnd. They are used for RTO estimation only. I think we need a general solution to cover when cwnd > pipe. So far we have laminar and your draft. But a unified solution would be great but we are making progress :) Thanks. From touch@isi.edu Mon Jul 16 16:10:32 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B7C1A21F88D2 for ; Mon, 16 Jul 2012 16:10:32 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.599 X-Spam-Level: X-Spam-Status: No, score=-102.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Vtxhm1eRqljk for ; Mon, 16 Jul 2012 16:10:27 -0700 (PDT) Received: from vapor.isi.edu (vapor.isi.edu [128.9.64.64]) by ietfa.amsl.com (Postfix) with ESMTP id 0B05821F88E1 for ; Mon, 16 Jul 2012 16:10:27 -0700 (PDT) Received: from [75.248.108.153] (153.sub-75-248-108.myvzw.com [75.248.108.153]) (authenticated bits=0) by vapor.isi.edu (8.13.8/8.13.8) with ESMTP id q6GNAGIg027843 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Mon, 16 Jul 2012 16:10:27 -0700 (PDT) Message-ID: <50049F5A.7000709@isi.edu> Date: Mon, 16 Jul 2012 16:10:18 -0700 From: Joe Touch User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:13.0) Gecko/20120614 Thunderbird/13.0.1 MIME-Version: 1.0 To: Yuchung Cheng References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Cc: "iccrg@cs.ucl.ac.uk" , "tcpm@ietf.org Extensions" , "Arjuna Sathiaseelan \(work\)" Subject: Re: [tcpm] Review of draft-fairhurst-tcpm-newcwv-03 X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Jul 2012 23:10:32 -0000 On 7/16/2012 3:56 PM, Yuchung Cheng wrote: > Summary: > > First of all the problem this draft is trying to solve is important: > AFAIK servers and data-centers disable slow-start after idle because > it simply hurts latency too badly. If they're mostly doing HTTP, it shouldn't matter at all. HTTP's transaction pattern defeats slow-start after idle anyway. Joe From nishida@sfc.wide.ad.jp Wed Jul 18 00:35:11 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id EAF1221F861E for ; Wed, 18 Jul 2012 00:35:10 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -101.636 X-Spam-Level: X-Spam-Status: No, score=-101.636 tagged_above=-999 required=5 tests=[AWL=0.340, BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, HTML_MESSAGE=0.001, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 0z1suNQ72LfI for ; Wed, 18 Jul 2012 00:35:10 -0700 (PDT) Received: from mail.sfc.wide.ad.jp (mail.sfc.wide.ad.jp [IPv6:2001:200:0:8803:203:178:142:146]) by ietfa.amsl.com (Postfix) with ESMTP id 9C22B21F8623 for ; Wed, 18 Jul 2012 00:35:09 -0700 (PDT) Received: from mail-lb0-f172.google.com (mail-lb0-f172.google.com [209.85.217.172]) by mail.sfc.wide.ad.jp (Postfix) with ESMTPSA id 21B282780AA for ; Wed, 18 Jul 2012 16:35:55 +0900 (JST) Received: by lbbgo11 with SMTP id go11so1909297lbb.31 for ; Wed, 18 Jul 2012 00:35:53 -0700 (PDT) MIME-Version: 1.0 Received: by 10.152.104.47 with SMTP id gb15mr2093689lab.45.1342596953679; Wed, 18 Jul 2012 00:35:53 -0700 (PDT) Received: by 10.112.90.176 with HTTP; Wed, 18 Jul 2012 00:35:53 -0700 (PDT) In-Reply-To: References: Date: Wed, 18 Jul 2012 00:35:53 -0700 Message-ID: From: Yoshifumi Nishida To: Matt Mathis Content-Type: multipart/alternative; boundary=f46d0421824d8026aa04c515b650 Cc: "tcpm@ietf.org\"" Subject: Re: [tcpm] Review of draft-ietf-tcpm-proportional-rate-reduction-01 X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jul 2012 07:35:11 -0000 --f46d0421824d8026aa04c515b650 Content-Type: text/plain; charset=ISO-8859-1 Hi Matt, Thanks for the detailed clarification. I agree with the most of your comments, but just one thing.... >> if (pipe > ssthresh) { >> >> Is this supposed to be like this? >> >> if (pipe >= ssthresh) { >> >> Otherwise, if pipe equals to ssthresh, sndcnt will be zero. > I think you are correct. The psuedo code matches our actual code and > causes us to forfeit an opportunity to send one segment slightly > earlier than we do. I see the situation. But, you don't need to explore (or encourage exploring) the latter case (pipe >= ssthresh) for future experiments? The latter case seems to be a bit more optimal with regard to spacing packets although it might be minor. Thanks, -- Yoshifumi Nishida On Fri, Jul 13, 2012 at 3:04 PM, Matt Mathis wrote: > Sorry I didn't get back on this sooner: I incorporated your changes > as suggested, except as detailed below. The new draft will be > posted "Real Soon Now." > > I believe it will be ready to advance. Formal WGLC next? > > On Tue, Mar 20, 2012 at 10:14 PM, Yoshifumi Nishida > wrote: > > Hi, I reviewed draft-ietf-tcpm-proportional-rate-reduction-01. > > I think this is valid and useful idea. I would like to support > publishing it > > to do more broad experiments. > > Below are my comments on the draft. > > Please check and update if you agree with them. > > > > > > Technical Comments > > > > 1: This is just out of my curiosity and I know there's no major > > difference, > > but is there any reasons to use Limited Transmit in the examples in > > Section 3.1? > > I just personally thought these examples might be a bit simpler if > you > > just use RFC3517 logic only. > > (You don't have to count packets transmitted by limited transmit for > > pipe calculation) > > But, this is a really minor point. Please keep it if you prefer. > > Perhaps the examples would have been simpler, but PRR is > "philosophically aligned" with limited transmit, and it would have > been sort of odd not to use it. > > > 2: I have a question in the pseudo code in Section 3. > > > > if (pipe > ssthresh) { > > > > Is this supposed to be like this? > > > > if (pipe >= ssthresh) { > > > > Otherwise, if pipe equals to ssthresh, sndcnt will be zero. > > I think you are correct. The psuedo code matches our actual code and > causes us to forfeit an opportunity to send one segment slightly > earlier than we do. However, we have chosen not to make this change > in the document because it would invalidate our measurement > experiment. To preserve the integrity of our experimental process > we would either need to remove the results or re-do them. > > > 3: If the assumption above is correct, I think the example in Page 6: > > > > PRR > > ack# X 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 > > pipe: 19 19 18 18 18 17 17 16 16 15 15 14 14 13 13 12 12 11 10 > > sent: N N R N N N N N N N N > > RB: s s > > > > will be something like this. Is this correct? > > Yep, but we didn't change it. > > > PRR > > pipe: 19 19 18 18 17 17 16 16 15 15 14 14 13 13 12 12 11 11 10 > > sent: N N R N N N N N N N N > > RB: s > > > > > > BTW, is there any reason that there is no cwnd data for PRR in the > > figures? > > cwnd is not used (and made the text more prominent.) > > > 4: In the following case, > > > > PRR-CRB > > ack# X X X X X X X X X X X X X X X 15 16 17 18 19 > > pipe: 19 19 4 4 4 > > sent: N N R R R > > RB: f f f > > > > I think PRR_CRB works like this way and this is understandable. > > > > ack #: 17 18 19 > > prr_delivered: 1 2 3 > > prr_out: 0 1 2 > > limit: 1 1 1 > > sndcnt: 1 1 1 > > > > However, in the following case, I think the parameters change like > this > > way according to the logic in Section 3. > > This result doesn't match the explanation in the draft. Am I > > misunderstanding something? > > limit = DeliverdData+SMSS > which is 2 for ACKs 18 and 19. > > > PRR-SSRB > > ack# X X X X X X X X X X X X X X X 15 16 17 18 19 > > pipe: 19 19 4 5 6 > > sent: N N 2R 2R 2R > > RB: d d d > > > > ack #: 17 18 19 > > prr_delivered: 1 2 3 > > prr_out: 0 2 4 > > limit: 2 1 1 > > sndcnt: 2 1 1 > > > > > > 5: Is 'DeliveredData' necessary in the following equation? > > I couldn't come up with good examples. But, I might miss something.. > > > > limit = MAX(prr_delivered - prr_out, DeliveredData) + MSS > > See above. Think of prr_delivered - prr_out as the bank: they > reflect the cumulative totals since the beginning of recovery. By > themselves they won't let you grow pipe. The DeliveredData term, on > the other hand, is unconstrained by the past and lets you grow > inflight by one MSS on every ACK. > > > > Editorial Comments: > > [snip - thanks] > > (Except RFC 3042 is only used informatively.) > > > Regards, > > -- > > Yoshifumi Nishida > > Thanks, > --MM-- > The best way to predict the future is to create it. - Alan Kay > --f46d0421824d8026aa04c515b650 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Hi Matt,

Thanks for the detailed clarification.
I agree with the= most of your comments, but just one thing....

>> =A0 =A0 =A0 = =A0 if (pipe > ssthresh) =A0{
>>
>> =A0 =A0 =A0Is this supposed to be like this?
>>
>> =A0 =A0 =A0 =A0 if (pipe >=3D ssthresh) =A0{
>>
>> =A0 =A0 =A0Otherwise, if pipe equals to ssthresh, sndcnt will be z= ero.

> I think you are correct. =A0 The psuedo code matches our actual code a= nd
> causes us to forfeit an opportunity to send one segment slightly
> earlier than we do.=A0

I see the situation. But, you don't= need to explore (or encourage exploring) the latter case (pipe >=3D sst= hresh) for future experiments? The latter case seems to be a bit more optimal with regard to spacing packets= although it might be minor.

Thanks,
--
Yoshifumi Nishida


On Fri, Jul 13, 2012 at 3:04 PM, Matt Mathis <mattmathis@google.com<= /a>> wrote:
Sorry I didn't get back on this sooner: = =A0I incorporated your changes
as suggested, except as detailed below. =A0 =A0 The new draft will be
posted "Real Soon Now."

I believe it will be ready to advance. =A0 Formal WGLC next?

On Tue, Mar 20, 2012 at 10:14 PM, Yoshifumi Nishida
<nishida@sfc.wide.ad.jp>= ; wrote:
> Hi, I reviewed draft-ietf-tcpm-proportional-rate-reduction-01.
> I think this is valid and useful idea. I would like to support publish= ing it
> to do more broad experiments.
> Below are my comments on the draft.
> Please check and update if you agree with them.
>
>
> Technical Comments
>
> =A0 1: This is just out of my curiosity and I know there's no majo= r
> difference,
> =A0 =A0 =A0but is there any reasons to use Limited Transmit in the exa= mples in
> Section 3.1?
> =A0 =A0 =A0I just personally thought these examples might be a bit sim= pler if you
> just use RFC3517 logic only.
> =A0 =A0 =A0(You don't have to count packets transmitted by limited= transmit for
> pipe calculation)
> =A0 =A0 =A0But, this is a really minor point. Please keep it if you pr= efer.

Perhaps the examples would have been simpler, but PRR is
"philosophically aligned" with limited transmit, and it would hav= e
been sort of odd not to use it.

> =A0 2: I have a question in the pseudo code in Section 3.
>
> =A0 =A0 =A0 =A0 if (pipe > ssthresh) =A0{
>
> =A0 =A0 =A0Is this supposed to be like this?
>
> =A0 =A0 =A0 =A0 if (pipe >=3D ssthresh) =A0{
>
> =A0 =A0 =A0Otherwise, if pipe equals to ssthresh, sndcnt will be zero.=

I think you are correct. =A0 The psuedo code matches our actual code = and
causes us to forfeit an opportunity to send one segment slightly
earlier than we do. =A0 However, we have chosen not to make this change
in the document because it would invalidate our measurement
experiment. =A0 =A0To preserve the integrity of our experimental process we would either need to remove the results or re-do them.

> =A0 3: If the assumption above is correct, I think the example in Page= 6:
>
> =A0PRR
> =A0 =A0ack# =A0 X =A01 =A02 =A03 =A04 =A05 =A06 =A07 =A08 =A09 10 11 1= 2 13 14 15 16 17 18 19
> =A0 =A0pipe: =A0 =A019 19 18 18 18 17 17 16 16 15 15 14 14 13 13 12 12= 11 10
> =A0 =A0sent: =A0 =A0 N =A0N =A0R =A0 =A0 N =A0 =A0 N =A0 =A0 N =A0 =A0= N =A0 =A0 N =A0 =A0 N =A0 =A0 =A0 =A0N =A0N
> =A0 =A0RB: =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0= =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0s =A0s
>
> =A0 =A0 =A0 will be something like this. Is this correct?

Yep, but we didn't change it.

> =A0PRR
> =A0 =A0pipe: =A0 =A019 19 18 18 17 17 16 16 15 15 14 14 13 13 12 12 11= 11 10
> =A0 =A0sent: =A0 =A0 N =A0N =A0R =A0 =A0 N =A0 =A0 N =A0 =A0 N =A0 =A0= N =A0 =A0 N =A0 =A0 N =A0 =A0 N =A0 =A0 N
> =A0 =A0RB: =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0= =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 s
>
>
> =A0 =A0BTW, is there any reason that there is no cwnd data for PRR in = the
> figures?

cwnd is not used (and made the text more prominent.)

> =A0 4: In the following case,
>
> PRR-CRB
> =A0 ack# =A0 X =A0X =A0X =A0X =A0X =A0X =A0X =A0X =A0X =A0X =A0X =A0X = =A0X =A0X =A0X 15 16 17 18 19
> =A0 pipe: =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A019 19 =A04 =A04 =A04
> =A0 sent: =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 N =A0N =A0R =A0R =A0R
> =A0 RB: =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 f =A0f =A0f
>
> =A0 =A0 =A0I think PRR_CRB works like this way and this is understanda= ble.
>
> =A0 =A0 =A0 =A0 =A0 ack #: 17 18 19
> =A0 prr_delivered: =A01 =A02 =A03
> =A0 prr_out: =A0 =A0 =A0 =A00 =A01 =A02
> =A0 limit: =A0 =A0 =A0 =A0 =A01 =A01 =A01
> =A0 sndcnt: =A0 =A0 =A0 =A0 1 =A01 =A01
>
> =A0 =A0 =A0However, in the following case, I think the parameters chan= ge like this
> way according to the logic in Section 3.
> =A0 =A0 =A0This result doesn't match the explanation in the draft.= Am I
> misunderstanding something?

limit =3D DeliverdData+SMSS
which is 2 for ACKs 18 and 19.

> PRR-SSRB
> =A0 ack# =A0 X =A0X =A0X =A0X =A0X =A0X =A0X =A0X =A0X =A0X =A0X =A0X = =A0X =A0X =A0X 15 16 17 18 19
> =A0 pipe: =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A019 19 =A04 =A05 =A06
> =A0 sent: =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 N =A0N 2R 2R 2R
> =A0 RB: =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 d =A0d =A0d
>
> =A0 =A0 =A0 =A0 =A0 ack #: 17 18 19
> =A0 prr_delivered: =A01 =A02 =A03
> =A0 prr_out: =A0 =A0 =A0 =A00 =A02 =A04
> =A0 limit: =A0 =A0 =A0 =A0 =A02 =A01 =A01
> =A0 sndcnt: =A0 =A0 =A0 =A0 2 =A01 =A01
>
>
> =A0 5: Is 'DeliveredData' necessary in the following equation?=
> =A0 =A0 =A0I couldn't come up with good examples. But, I might mis= s something..
>
> =A0 =A0 =A0limit =3D MAX(prr_delivered - prr_out, DeliveredData) + MSS=

See above. =A0 Think of prr_delivered - prr_out as the bank: they
reflect the cumulative totals since the beginning of recovery. =A0By
themselves they won't let you grow pipe. =A0 The DeliveredData term, on=
the other hand, is unconstrained by the past and lets you grow
inflight by one MSS on every ACK.


> Editorial Comments:

[snip - thanks]

(Except RFC 3042 is only used informatively.)

> Regards,
> --
> Yoshifumi Nishida

Thanks,
--MM--
The best way to predict the future is to create it. =A0- Alan Kay

--f46d0421824d8026aa04c515b650-- From mirja.kuehlewind@ikr.uni-stuttgart.de Wed Jul 18 10:54:35 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id F13D621F87CB for ; Wed, 18 Jul 2012 10:54:34 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.199 X-Spam-Level: X-Spam-Status: No, score=-0.199 tagged_above=-999 required=5 tests=[AWL=1.750, BAYES_00=-2.599, HELO_EQ_DE=0.35, MIME_8BIT_HEADER=0.3] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1XEaoT1cOk-G for ; Wed, 18 Jul 2012 10:54:33 -0700 (PDT) Received: from mailsrv.ikr.uni-stuttgart.de (mailsrv.ikr.uni-stuttgart.de [129.69.170.2]) by ietfa.amsl.com (Postfix) with ESMTP id 3DB6621F8739 for ; Wed, 18 Jul 2012 10:54:25 -0700 (PDT) Received: from netsrv1.ikr.uni-stuttgart.de (netsrv1-c [10.11.12.12]) by mailsrv.ikr.uni-stuttgart.de (Postfix) with ESMTP id 4AF7A633B1; Wed, 18 Jul 2012 19:55:15 +0200 (CEST) Received: from vpn-2-cl177 (vpn-2-cl177 [10.41.21.177]) by netsrv1.ikr.uni-stuttgart.de (Postfix) with ESMTP id 2BC8E59A8A; Wed, 18 Jul 2012 19:55:15 +0200 (CEST) From: Mirja =?iso-8859-1?q?K=FChlewind?= To: iccrg@cs.ucl.ac.uk, "tcpm@ietf.org Extensions" , "gorry@erg.abdn.ac.uk" Date: Wed, 18 Jul 2012 19:55:14 +0200 User-Agent: KMail/1.9.10 (enterprise35 0.20101217.1207316) References: In-Reply-To: X-KMail-QuotePrefix: > MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <201207181955.14233.mkuehle@ikr.uni-stuttgart.de> Subject: [tcpm] Review draft-fairhurst-tcpm-newcwv-03 X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jul 2012 17:54:35 -0000 Hi Gorry, I believe this draft addresses a very important problem which exists and need to be solved. But I'm not too sure about the solution that is proposed. But I also don't know what the right/perfect solution would be. My biggest concern is regarding potential bursts you might introduce. Maybe I'm too conservative but I believe whatever you gone chance in TCP, one thing to avoid is the chance to cause large bursts of packets. Recently, it was actually recognized that the block sending behavior of youTube (eventough it is using normal standard-conform TCP) leads to bursts and large loss probabilities. Actually that is a problem that exists in today's TCP and needs to be addressed too. So I guess what I propose is to limit the burst size. This is not completely though through yet but you could allow some slow start like behavior in CA if the flow has been application limited. So maybe your mechanism could be extended to allow only bursts of 2 (or 3) packets. That would mean that you would be able to increase your sending rate form 1*flightsize to 2*(or 3*)flightsize in one RTT if the flow was application limited before. Don't you think that might be already sufficient? In case of idle times this would (unfortunately) lead back to normal slow start behavior but the IW10 draft also specifies a restart window of 10. This might help the problem already a lot...? Btw. in Linux (and potentially in some RFC?) there is already a maximum burst size of 3 implemented but this is only used to limited the congestion window to flightsize + maxburstsize. I will send some more detailed comments on the draft tomorrow (don't have my notes with me at the moment). But one more point are the values you have chosen for the several thresholds. I know there is a whole section reasoning why 5 minutes have been chosen but that didn't really convince me. One more though on my proposal above (which is still not though through): If you would track the max cwnd since the last congestion notification event and use this as a threshold upto which the kind of slow-start-behavior in CA (as explained above) is allow, I don't think that there is any time limit needed at all... I guess this whole idea leads to something similar as proposed by TCP Laminar. As Yuchung mentioned those two proposals should be regarded together. Some more detailed comments tomorrow... Mirja From mirja.kuehlewind@ikr.uni-stuttgart.de Thu Jul 19 03:00:48 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2879321F86A0 for ; Thu, 19 Jul 2012 03:00:48 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.632 X-Spam-Level: X-Spam-Status: No, score=-0.632 tagged_above=-999 required=5 tests=[AWL=1.017, BAYES_00=-2.599, HELO_EQ_DE=0.35, J_CHICKENPOX_38=0.6] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id EUJsXmwzkBra for ; Thu, 19 Jul 2012 03:00:47 -0700 (PDT) Received: from mailsrv.ikr.uni-stuttgart.de (mailsrv.ikr.uni-stuttgart.de [129.69.170.2]) by ietfa.amsl.com (Postfix) with ESMTP id C366221F87A4 for ; Thu, 19 Jul 2012 03:00:46 -0700 (PDT) Received: from netsrv1.ikr.uni-stuttgart.de (netsrv1-c [10.11.12.12]) by mailsrv.ikr.uni-stuttgart.de (Postfix) with ESMTP id F344C633B1; Thu, 19 Jul 2012 12:01:37 +0200 (CEST) Received: from vpn-2-cl177 (vpn-2-cl177 [10.41.21.177]) by netsrv1.ikr.uni-stuttgart.de (Postfix) with ESMTP id E03CA59A8A; Thu, 19 Jul 2012 12:01:37 +0200 (CEST) From: Mirja Kuehlewind Organization: University of Stuttgart (Germany), IKR To: tcpm@ietf.org Date: Thu, 19 Jul 2012 12:01:37 +0200 User-Agent: KMail/1.9.10 (enterprise35 0.20101217.1207316) References: <201207181955.14233.mkuehle@ikr.uni-stuttgart.de> In-Reply-To: <201207181955.14233.mkuehle@ikr.uni-stuttgart.de> X-KMail-QuotePrefix: > MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Message-Id: <201207191201.37368.mirja.kuehlewind@ikr.uni-stuttgart.de> Cc: iccrg@cs.ucl.ac.uk Subject: Re: [tcpm] Review draft-fairhurst-tcpm-newcwv-03 X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jul 2012 10:00:48 -0000 Hi Gorry, here as promised the more detailed comments: =2D section 4: Please add a reference to the later chapter that the value o= f 5=20 minutes will be further discussed. =2D section 4.2: Please explain where the 2/3*cwnd comes from. I would have= =20 though validated phase is when flightsize >=3D cwnd or maybe >=3D cwnd -=20 maxburstsize (as in today's implementation the cwnd will be limited to=20 flightsize+maxburstsize anyway) =2D section 4.3: Should the case that the 5 minutes are over here be mentio= ned=20 as well...? =2D section 4.3: in 2. bulletpoint the reference is wrong; should be to sec= tion=20 4.3.2. =2D section 4.3: I would merge the last paragraph into there first bulletpo= int=20 as I was asking me exactly that question when reading the first bulletpoint =2D section 4.3.1: "by setting a burst size limit, using a pacing algorithm= , or=20 some other method." -> I think there should be one (or more) mechanisms=20 specified here as part of the algorithm =2D section 4.3.1: "cwnd =3D max(FlightSize*2, IW)" -> Why flightsize*2? Th= is can=20 still be a very large not validated window. I would go for=20 flightsize+maxburstsize (at least have static offset here).=20 =2D section 4.3.1: "cwnd =3D max(FlightSize*2, IW)" -> If you keep this, it= should=20 be "cwnd =3D min(max(FlightSize*2, IW),cwnd)" =2D section 4.3.1: "ssthresh =3D max(ssthresh, 3*cwnd/4)" -> Is this the ne= w or=20 the old cwnd? Why are you taking a different value than usually? =2D section 4.3.2: Why is the reaction to congestion different than what sh= ould=20 be done when leaving the nonvalidated phase? =2D section 4.3.2: "cwnd =3D ((FlightSize - R)/2)" -> I wouldn't wanne spec= ify the=20 concrete congestion response in this document. There might be congestion=20 control algorithms like DCTCP that so something different than halving (at= =20 least for ECN). The document should just say something like "perform the=20 normal congestion response specified by the used congestion control algorit= hm=20 after the adaption of the congestion window". You might even think about=20 having two window variables here. Will the congestion control only controls= =20 the cwnd, your second window, which than determines the allowed packets in= =20 flight, might be larger. Moreover, you have to make sure that the congestio= n=20 control will not raise the cwnd anyfurther while being application-limited! =2D section 4.4: I would make this an own section 5 =2D section 4.4: Are there references available on what the "idle intervals= of=20 common applications" and "period for which the capacity of an Internet path= =20 may commonly be regarded as stable" are? =2D section 4.4: 3. paragraph: Maybe be even more strict and say something= =20 like "If rapid changes in the path characteristics are detected, the new=20 method SHOULD not be used anymore". Maybe even require to monitor the RTT's= =20 and give a threshold when to disable this method. =2D section 4.4: Also refer the respective TCP mechanisms that use a 5 minu= tes=20 timer. Mirja On Wednesday 18 July 2012 19:55:14 Mirja K=FChlewind wrote: > Hi Gorry, > > I believe this draft addresses a very important problem which exists and > need to be solved. But I'm not too sure about the solution that is > proposed. But I also don't know what the right/perfect solution would be. > > My biggest concern is regarding potential bursts you might introduce. May= be > I'm too conservative but I believe whatever you gone chance in TCP, one > thing to avoid is the chance to cause large bursts of packets. Recently, = it > was actually recognized that the block sending behavior of youTube > (eventough it is using normal standard-conform TCP) leads to bursts and > large loss probabilities. Actually that is a problem that exists in today= 's > TCP and needs to be addressed too. > > So I guess what I propose is to limit the burst size. This is not > completely though through yet but you could allow some slow start like > behavior in CA if the flow has been application limited. So maybe your > mechanism could be extended to allow only bursts of 2 (or 3) packets. That > would mean that you would be able to increase your sending rate form > 1*flightsize to 2*(or 3*)flightsize in one RTT if the flow was application > limited before. Don't you think that might be already sufficient? > In case of idle times this would (unfortunately) lead back to normal slow > start behavior but the IW10 draft also specifies a restart window of 10. > This might help the problem already a lot...? > Btw. in Linux (and potentially in some RFC?) there is already a maximum > burst size of 3 implemented but this is only used to limited the congesti= on > window to flightsize + maxburstsize. > > I will send some more detailed comments on the draft tomorrow (don't have > my notes with me at the moment). But one more point are the values you ha= ve > chosen for the several thresholds. I know there is a whole section > reasoning why 5 minutes have been chosen but that didn't really convince > me. One more though on my proposal above (which is still not though > through): If you would track the max cwnd since the last congestion > notification event and use this as a threshold upto which the kind of > slow-start-behavior in CA (as explained above) is allow, I don't think th= at > there is any time limit needed at all... > > I guess this whole idea leads to something similar as proposed by TCP > Laminar. As Yuchung mentioned those two proposals should be regarded > together. > > Some more detailed comments tomorrow... > > Mirja > > > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm =2D-=20 =2D------------------------------------------------------------------ Dipl.-Ing. Mirja K=FChlewind Institute of Communication Networks and Computer Engineering (IKR) University of Stuttgart, Germany Pfaffenwaldring 47, D-70569 Stuttgart tel: +49(0)711/685-67973 email: mirja.kuehlewind@ikr.uni-stuttgart.de web: www.ikr.uni-stuttgart.de =2D------------------------------------------------------------------ From touch@isi.edu Thu Jul 19 08:05:07 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D6A2C21F86F3 for ; Thu, 19 Jul 2012 08:05:07 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.055 X-Spam-Level: X-Spam-Status: No, score=-103.055 tagged_above=-999 required=5 tests=[AWL=-0.456, BAYES_00=-2.599, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gahOt7-csV1q for ; Thu, 19 Jul 2012 08:05:07 -0700 (PDT) Received: from darkstar.isi.edu (darkstar.isi.edu [128.9.128.127]) by ietfa.amsl.com (Postfix) with ESMTP id 5FE1F21F86F1 for ; Thu, 19 Jul 2012 08:05:07 -0700 (PDT) Received: from [192.168.1.92] (pool-71-105-89-105.lsanca.dsl-w.verizon.net [71.105.89.105]) (authenticated bits=0) by darkstar.isi.edu (8.13.8/8.13.8) with ESMTP id q6JF556N020125 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Thu, 19 Jul 2012 08:05:08 -0700 (PDT) Message-ID: <50082223.9010701@isi.edu> Date: Thu, 19 Jul 2012 08:05:07 -0700 From: Joe Touch User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:13.0) Gecko/20120614 Thunderbird/13.0.1 MIME-Version: 1.0 To: Yuchung Cheng References: <50049F5A.7000709@isi.edu> In-Reply-To: <50049F5A.7000709@isi.edu> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Cc: "iccrg@cs.ucl.ac.uk" , "tcpm@ietf.org Extensions" , "Arjuna Sathiaseelan \(work\)" Subject: Re: [tcpm] Review of draft-fairhurst-tcpm-newcwv-03 X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jul 2012 15:05:08 -0000 On 7/16/2012 4:10 PM, Joe Touch wrote: > > > On 7/16/2012 3:56 PM, Yuchung Cheng wrote: >> Summary: >> >> First of all the problem this draft is trying to solve is important: >> AFAIK servers and data-centers disable slow-start after idle because >> it simply hurts latency too badly. > > If they're mostly doing HTTP, it shouldn't matter at all. HTTP's > transaction pattern defeats slow-start after idle anyway. > > Joe PS - we did try to deal with this issue a decade ago, but the effort was not take up by the WG: http://www.isi.edu/touch/pubs/draft-hughes-restart-00.txt Joe From ycheng@google.com Thu Jul 19 10:25:51 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DCCB421F87EA for ; Thu, 19 Jul 2012 10:25:50 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.902 X-Spam-Level: X-Spam-Status: No, score=-102.902 tagged_above=-999 required=5 tests=[AWL=0.075, BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ZzFpWSWtMJst for ; Thu, 19 Jul 2012 10:25:50 -0700 (PDT) Received: from mail-ob0-f172.google.com (mail-ob0-f172.google.com [209.85.214.172]) by ietfa.amsl.com (Postfix) with ESMTP id EA0B321F87E7 for ; Thu, 19 Jul 2012 10:25:49 -0700 (PDT) Received: by obbwc20 with SMTP id wc20so4658465obb.31 for ; Thu, 19 Jul 2012 10:26:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type:x-system-of-record; bh=XBubKJV0wCM5efpIsm2IBLZuasPMctKqZyln8d+8MCU=; b=i24HVxGUDZ5ETAb67h/rxAczbIS+THBdjwVfR+/3wKLR3dqmJ00L57G0gXKISGDCYe mHcpVRlK1ZXqaUa8pWvgjxg1gEQn43o/619nP4cu8AfX9jW8WzbPPUUYjk6HVvQQC4fm ZBQVr/34fWfnJqHtX/R+hkg0MzimUEwBJhODKSvhlwKJoxnIDOPpkPVGUVluH2UkfKG3 JWOCJ+BFub1GJ0FDcQ5qdSnLDF1+vK+gAzAcsWfyX+MwbYs8jZuN2w/oGdmssnrKGG1L CJic8YlL59Gaci4IbhOE96dyEQy+42355D1LZfsqkYUkDQQQi6zeP3VLfYkEhXcyI4Pq Dwuw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type:x-system-of-record:x-gm-message-state; bh=XBubKJV0wCM5efpIsm2IBLZuasPMctKqZyln8d+8MCU=; b=Lo3YYdtO4DMb5hMLNf6I5VaAmiOZ3sFzwyRzk/nOZA03XO8AIfPDIRHwgmmyK+o8+u JMajGRFv3GRpB6lPw+BNlGF0Nqw+YUjHnJTSOZ1BmjzUasYe/3Uviv4mScV0QUl7XSNc d+df6G3q3/7SlfZeu+WNzBRg37CEdiOr9+rJD/h++sEncvQziadxGn3cVxcxeOj5Pnx4 glqMN2dDVBGmlY95JtPmhavT8l37oSsi/nkTlbElq5X0NukTtsNJbrOfpy2fzpmDS3DX ZxjfIob+e/U6EaSTD6C48py446d1j9V4OfQ3/7XmkR3FyMUn6hble7MKKkiRu2OVfzdl YtBw== Received: by 10.182.16.3 with SMTP id b3mr3777319obd.72.1342718803334; Thu, 19 Jul 2012 10:26:43 -0700 (PDT) Received: by 10.182.16.3 with SMTP id b3mr3777298obd.72.1342718803089; Thu, 19 Jul 2012 10:26:43 -0700 (PDT) MIME-Version: 1.0 Received: by 10.60.14.168 with HTTP; Thu, 19 Jul 2012 10:26:21 -0700 (PDT) In-Reply-To: <50082223.9010701@isi.edu> References: <50049F5A.7000709@isi.edu> <50082223.9010701@isi.edu> From: Yuchung Cheng Date: Thu, 19 Jul 2012 10:26:21 -0700 Message-ID: To: Joe Touch Content-Type: text/plain; charset=ISO-8859-1 X-System-Of-Record: true X-Gm-Message-State: ALoCoQmE9JwAub6LYUvQTjgmhjK0M7vtBXUPvUScGhuFO9rRE1Q/X9IteBJ73iZ+FqZqOh35Rn99QWQTiand5YCdlO2WWpRsD9f/RfRZh0LPs34XZiIUfBE7apPYaQBWS/SaO4qeSQU7Dr7tQ4AdnfUR5bqtve8x6E7OZl3E/eqRLEtV2vL0CV4= Cc: "iccrg@cs.ucl.ac.uk" , "tcpm@ietf.org Extensions" , "Arjuna Sathiaseelan \(work\)" Subject: Re: [tcpm] Review of draft-fairhurst-tcpm-newcwv-03 X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jul 2012 17:25:51 -0000 On Thu, Jul 19, 2012 at 8:05 AM, Joe Touch wrote: > > > On 7/16/2012 4:10 PM, Joe Touch wrote: >> >> >> >> On 7/16/2012 3:56 PM, Yuchung Cheng wrote: >>> >>> Summary: >>> >>> First of all the problem this draft is trying to solve is important: >>> AFAIK servers and data-centers disable slow-start after idle because >>> it simply hurts latency too badly. >> >> >> If they're mostly doing HTTP, it shouldn't matter at all. HTTP's >> transaction pattern defeats slow-start after idle anyway. Why not if the interval between two HTTP responses are large, say two minutes? Your draft below suggested receiving HTTP requests have some effects but I didn't see that in RFC 2861 (and I couldn't find the mailing list discussion cited either). Could you provide an example? Thanks. >> >> Joe > > > PS - we did try to deal with this issue a decade ago, but the effort was not > take up by the WG: > > http://www.isi.edu/touch/pubs/draft-hughes-restart-00.txt > > Joe > From touch@isi.edu Thu Jul 19 10:37:41 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id EFCEE21F8800 for ; Thu, 19 Jul 2012 10:37:40 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.998 X-Spam-Level: X-Spam-Status: No, score=-102.998 tagged_above=-999 required=5 tests=[AWL=-0.399, BAYES_00=-2.599, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lvRPUQbU38gu for ; Thu, 19 Jul 2012 10:37:40 -0700 (PDT) Received: from vapor.isi.edu (vapor.isi.edu [128.9.64.64]) by ietfa.amsl.com (Postfix) with ESMTP id 4E65421F86AF for ; Thu, 19 Jul 2012 10:37:40 -0700 (PDT) Received: from [192.168.1.92] (pool-71-105-89-105.lsanca.dsl-w.verizon.net [71.105.89.105]) (authenticated bits=0) by vapor.isi.edu (8.13.8/8.13.8) with ESMTP id q6JHbXUW002547 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Thu, 19 Jul 2012 10:37:45 -0700 (PDT) Message-ID: <500845E0.8050601@isi.edu> Date: Thu, 19 Jul 2012 10:37:36 -0700 From: Joe Touch User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:13.0) Gecko/20120614 Thunderbird/13.0.1 MIME-Version: 1.0 To: Yuchung Cheng References: <50049F5A.7000709@isi.edu> <50082223.9010701@isi.edu> In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Cc: "iccrg@cs.ucl.ac.uk" , "tcpm@ietf.org Extensions" , "Arjuna Sathiaseelan \(work\)" Subject: Re: [tcpm] Review of draft-fairhurst-tcpm-newcwv-03 X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jul 2012 17:37:41 -0000 On 7/19/2012 10:26 AM, Yuchung Cheng wrote: > On Thu, Jul 19, 2012 at 8:05 AM, Joe Touch wrote: >> >> >> On 7/16/2012 4:10 PM, Joe Touch wrote: >>> >>> >>> >>> On 7/16/2012 3:56 PM, Yuchung Cheng wrote: >>>> >>>> Summary: >>>> >>>> First of all the problem this draft is trying to solve is important: >>>> AFAIK servers and data-centers disable slow-start after idle because >>>> it simply hurts latency too badly. >>> >>> >>> If they're mostly doing HTTP, it shouldn't matter at all. HTTP's >>> transaction pattern defeats slow-start after idle anyway. > > Why not if the interval between two HTTP responses are large, say two minutes? > Your draft below suggested receiving HTTP requests have some effects but > I didn't see that in RFC 2861 (and I couldn't find the mailing list discussion > cited either). Let's say the connection goes idle for 2 minutes. TCP doesn't do anything when a connection goes idle; it only does things when something is sent (in general). The most likely reason the server would have something new to send is that the client sends a new request. The client request is short enough to fit in 1-2 segments, so shouldn't be an issue (and Nagle should be off, since it's interactive with multibyte exchanges). The server receives the request, and then sends a response. The server has a response that's typically larger - thus the burst. The server decides "how long has it been since I *received* anything? Since that time is very short, there's no slow-start restart, and the burst goes out. The only time the server would timeout and do slowstart restart would be if it decides to send something a while after anything has been received - that might occur for auto-refresh pages, but not much else. The issue is explained in detail here: J. Heidemann, K. Obraczka, J. Touch, “Modeling the Performance of HTTP Over Several Transport Protocols,” IEEE/ACM Transactions on Networking, V5, N5, Oct. 1997, pp.616-630. Joe From mattmathis@google.com Thu Jul 19 13:01:54 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9449F21F8799 for ; Thu, 19 Jul 2012 13:01:54 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.776 X-Spam-Level: X-Spam-Status: No, score=-102.776 tagged_above=-999 required=5 tests=[AWL=0.200, BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id p8rgJWedD-8h for ; Thu, 19 Jul 2012 13:01:51 -0700 (PDT) Received: from mail-wg0-f44.google.com (mail-wg0-f44.google.com [74.125.82.44]) by ietfa.amsl.com (Postfix) with ESMTP id E9A1721F8595 for ; Thu, 19 Jul 2012 13:01:50 -0700 (PDT) Received: by wgbdr13 with SMTP id dr13so2036858wgb.13 for ; Thu, 19 Jul 2012 13:02:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:x-system-of-record; bh=hy1FuPzG9FJx2AgqHYX3lVi6j29DKUWIV0UKyz2/TLo=; b=EY6XXYGlLVQ/+tUxkryDsuTPYySywyxE5MuyCR237t+DZGODaKTnsTn8HNTAXRNwQK prG2fHpPDPkD21HpM2Ku3AIXzA10C5g2Z6569W2mwPmAAV2J2nZcBw/RBTfLv5vF5Fb5 KUoq419A0V8zaEg7h1f/pNp4qvG8KICWPoAtvBYLJyC1Ryg2/dWLL9BLg7ZSRhib58zL Be1J61NnT0GpYWoekeMNc85ceXZ3rGhtZ1U1eVUDiqg2jm9OwjF+LFPdQUeaCuSYY520 s+XXULHSSo12eSpDJQa3z6Kn0/fBYDexuBa2Xd9hfbUK706WoBlC9fLdvliiYWQxR5kT tlTg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:x-system-of-record:x-gm-message-state; bh=hy1FuPzG9FJx2AgqHYX3lVi6j29DKUWIV0UKyz2/TLo=; b=NJku0xSTlcXu/mR39tAGG40mEDOnv/8VXqwcCFL4OCvID248vkduNr1lxjYhwJ9Dpr 8PdBmZTfHD5wvgSRmxl9++mLHr3m3rT7hXXQrHqzLWlXl2R70RQpVCkq+tB2bB686jUv +0dRTRVM/Srl0420ZOFwUNVpZhDzmWI7/ytYMvnPHSCwLFlgvaOKWWIt37LgteswRZkU Q8VVhfORXJCG9kL3KbL6FUoSU5G+PVmzLXz98vVCwfPezQeME0ZUQz+4ROH6b9e1TMsV izm6Kg6Hs7RMs+Y+ZN/YdTqZcm4ZbjOB15CvkpDpghEUSgFreZsp8ZvZOH5YtQLwANqv WFxg== Received: by 10.180.98.138 with SMTP id ei10mr7602512wib.1.1342728164146; Thu, 19 Jul 2012 13:02:44 -0700 (PDT) MIME-Version: 1.0 Received: by 10.180.98.138 with SMTP id ei10mr7602488wib.1.1342728163904; Thu, 19 Jul 2012 13:02:43 -0700 (PDT) Received: by 10.216.205.168 with HTTP; Thu, 19 Jul 2012 13:02:43 -0700 (PDT) In-Reply-To: References: Date: Thu, 19 Jul 2012 13:02:43 -0700 Message-ID: From: Matt Mathis To: Anthony Sabatini Content-Type: multipart/alternative; boundary=f46d044267fe3d4b3004c53443f9 X-System-Of-Record: true X-Gm-Message-State: ALoCoQnyXYH+p7o5mUL7jmbKq4dKcH8D1JHWAMgCoxV79vuMfldwBBZQLtg3MzcTmPMKob6xl3I1gYQJ5kwmYXbb6gVHr2J6m0YyAFe8H5MOPo+9MIKAB76m8rjQw7b/oSaiVjjiSBLpBsfSjGnBS//Ho3BLIghzgdeSytG36c0lEoXggncspWU= Cc: tcpm Subject: Re: [tcpm] TCP Error recovery and efficiency X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jul 2012 20:01:54 -0000 --f46d044267fe3d4b3004c53443f9 Content-Type: text/plain; charset=ISO-8859-1 I don't understand what problem you are trying to solve: As written 2018 provides the sender with perfect information about the state of the receivers reassembly queue (e.g. which data is still missing) as long as most ACKs are delivered, mostly in order. The minimum cases which causes the sender to send duplicate data are timeouts (but see the errata against 2018), or fairly complicated patterns of sustained losses on both sides of the link. In fact, if there is no loss or reordering on the return path, a single SACK block is sufficient to give the sender perfect information. With classic TCP timestamps (not for PAWS) it is even possible to disambiguate between holes that were filled by ooo data or retransmissions. Due to the TS echo rules for PAWS, it actually does less well. I am not the expert here, but Richard Scheffenegger has explored this problem extensively. But the ambiguity is whether congestion control undo is permitted, not which data is still needed at the receiver. If you look at 2018, (and 2883) the first SACK block is described as MUST be the chronologically newest. The rest of the option is described with SHOULDs, because we knew that SACK could theoretically be improved by dithering the rest of the SACK option across all past SACK blocks. Such a scheme would make SACK work well with unreasonable levels of loss on the ACK path. (e.g. the 2nd SACK block could alternate between the two most recent prior 1st blocks, the 3rd across all earlier blocks). But now, would that really a feature for TCP to work well on the forward path, while the (low bandwidth) return path is being mauled by SACK? Thanks, --MM-- The best way to predict the future is to create it. - Alan Kay On Sun, Jul 15, 2012 at 12:59 PM, Anthony Sabatini wrote: > All - > > First off a thank you to Richard Scheffenegger for his comments but most > importantly when he asked whether the number of ACKs being sent was > excessive (since we are trying for efficiency unnecessary ACKs diminish > that efficiency). When I first wrote this RFC six years ago for my thesis, > my concern was with my extensions not in reanalyzing the base protocol. > This turns out to be a mistake because, by moving the retransmission > process farther away from using timers and changing it to purely a state > basis, a number of statements/features of the base protocol no longer work > the same. > > As a statement of policy SACK should send no more ACKs then a current > modern version of TCP does. For the most part SACK only adds from 2 to 34 > bytes in the options area of existing messages. Even with this change we > are only talking 6 to 38 bytes. It is only in extreme situations where > there are more than four gaps in the acknowledged data that SACK requires > additional ACKs. As to when these ACKs are sent again normal TCP practice > applies for the most part (as long as all SACK blocks occupy a single ACK). > > The recovery basis in Enhanced SACK (E-SACK) except for the "Link Broken" > disaster timer is entirely state driven on the transmit side. Since timers > by their very nature, must be set to the worst case value, this automated > retransmission is much quicker and more accurate. The combination of TCP > ACKs, SACK block changes and Returned Transmit Token changes form the > triggers that guide the recovery and transmission processes. In order to > insure the proper flow of those signals the receiver is running three > classes of timer - Returned Transmit Token delay timer, Second ACK timer > and the "I did not receive what I needed" keep alive timer. More about > those in the commentary. > > The principle that all state changing information should be sent twice > (second ACK) from TCP is a good one and should be preserved in this > protocol since the loss of an ACK is as disastrous as the loss of a > message, we just need to be cognizant that there are now three state change > elements, any of which triggers the ACK pair, specifically the TCP ACK > (acknowledging a change in the TCP transmit floor), a change to a SACK > block, and finally a change to one of the tokens (note I Include both > Transmit and Returning here). The question then becomes how do we > implement this. The underlying reality here is that on a link with even > moderate activity a second intentional ACK is not needed since the ACK > triggered by the next arriving message would replace it. The problem we > encounter is that ACK messages are significantly shorter than full data > messages so we would always send a second ACK if we send it immediately > after the first since nothing has changed. The thought is then to delay > the second ACK until the link goes idle (1.3 times the length of time to > receive a full packet on the fast side to 2.6 x the length of time to > receive a full packet on the slow side). Note that this delay has a direct > effect on the Returned Transmit Token delay timer as that timer can not be > less than this delay plus the amount of time to transmit all of the ACKs to > hold the complete list of SACK blocks in order to insure that state > information is up to date before the token changes, thus possibly degrading > link efficiency. > > One note of clarification, if a state change triggers another pair of > ACKs, this takes precedence over completing the remainder of the ACK group > if multiple ACKs are required to transmit the SACK blocks. > > The reason we trigger an ACK on either token changing is to get around the > dreaded "last character hang" of very early TCP implementations. If the > transmitter sends an ACK with the updated Transmit Token from the last > outgoing segment when the link goes idle it insures the receiver has the > updated value of that Transmit Token which will lead to the retransmission > of that last missing segment. > > We delay the return of the received Transmit Token for two reasons, first > it allows packets that got out of order to be received and put back in > order before being declared missing and secondly, because a group of SACK > blocks might span multiple ACKs, to make sure that state information is up > to date before we declare a segment as missing. > > This leads the to a discussion of very bad burst mode errors. If the > receiver has not received the segments it requires and has not sent out an > ACK for RTT*1.3 then the receiver needs to send another ACK pair. Note > this applies to the transmitter also in that if its Returned Transmit Token > does not equal Last Transmit Token after the same RTT*1.3 it is obligated > to send an ACK pair to the receiver to prompt it to restart. Obviously > RTT*1.3 is a long time on transcontinental or satellite links which is why > 1/4 RTT for an ACK pair from the receiver as a keep alive is under > consideration, the maximum length of time to cover the burst error without > degrading efficiency too badly. > > In filling the outgoing ACKs with SACK blocks at the receiver, the list is > maintained in the order oldest (smallest segment start) to the youngest. > The list is queried three times when building the ACK group, first for all > blocks that just changed and thus have the most critical information (ACK > NEEDED = 2), then a scan for those blocks which already had their first ACK > (ACK NEEDED = 1) and then for the blocks which have already been > transmitted twice (ACK NEEDED = 0). The ACK NEEDED count is then > decremented. > > To make E-SACK work three lists must be maintained, one at the receiver > and two at the transmitter. The one in the receiver is identical in > concept to the one in the original RFC with the addition of ACK NEEDED and > the proviso that it be in oldest to youngest order. At the transmitter an > identical list is maintained built from the recovered SACK blocks. When a > changed Returned Transmit Token is received, the recovered SACK block list > is used to construct a retransmit queue of all unacknowledged segments. > Then starting at the transmit queue entry plus one derived from the > Returned Transmit Token, and proceeding forward until all transmitted > segment extents have been processed, entries are removed or modified in the > new retransmit queue to compensate for segments sent after the time of the > Returned Transmit Token. The adjusted new transmit queue replaces the > existing queue and the transmission process is started again starting at > the first entry on the queue, not proceeding with new transmission until > the queue is completed. Please note that incoming SACK blocks must be > checked against the retransmit queue in order to remove segments that have > been subsequently correctly received. > > Noto Bene - This protocol is an update to RFC 2018. It is not compatible > with nor do I believe in the philosophy of RFC2883/RFC3708 as the whole > purpose of this change is to eliminate duplicate segments and increase > efficiency and reliability where the changes outlined in those RFCs degrade > efficiency (especially by generating extra SACK blocks) without > substantially improving anything. Also as adopted in the RFC there is a > major fallacy, it assumes that there can be only one SACK block with a > status change where in reality, due to processing considerations, there can > be two or even more blocks with changes before an ACK can be sent. > > > Tony > > Anthony Sabatini > 200 West 20th Street > Apt. 1216 > New York, NY 10011 > > Phone: (212) 867-7179 > Mobile: (917) 224-8388 > > > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm > > --f46d044267fe3d4b3004c53443f9 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable I don't understand what problem you are trying to solve: As written 201= 8 provides the sender with perfect information about the state of the recei= vers reassembly queue (e.g. which data is still missing) as long as most AC= Ks are delivered, mostly in order. =A0 The minimum cases which causes the s= ender to send=A0duplicate=A0data are timeouts (but see the errata against 2= 018), or fairly complicated patterns of sustained=A0losses=A0on both sides = of the link.

In fact, if there is no loss or=A0reordering=A0on the return= path, a single SACK block is=A0sufficient=A0to give the sender perfect inf= ormation.

With classic TCP timestamps (not for PAWS)= it is even possible to disambiguate between holes that were filled by ooo = data or retransmissions. =A0Due to the TS echo rules for PAWS, it actually = does less well. =A0I am not the expert here, but=A0Richard Scheffenegger ha= s=A0explored=A0this problem extensively. =A0 But the=A0ambiguity=A0is=A0whe= ther congestion control undo is permitted, not which data is still needed a= t the receiver.

If you look at 2018, (and 2883) the first SACK block is= described as MUST be the chronologically newest. =A0The rest of the option= is described with SHOULDs, because we knew that SACK could theoretically b= e improved by dithering the rest of the SACK option across all past SACK bl= ocks. =A0 Such a scheme would make SACK work well with unreasonable levels = of loss on the ACK path. =A0(e.g. the 2nd SACK block could alternate betwee= n the two most recent prior 1st=A0blocks, the 3rd=A0across=A0all earlier bl= ocks).

But now, would that really a feature for TCP to work we= ll on the forward path, while the (low bandwidth) return path is being maul= ed by SACK?

Thanks,
--MM--
The best way= to predict the future is to create it. =A0- Alan Kay



On Sun, Jul 15, 2012 at 12:59 PM, Anthon= y Sabatini <tsabatini@hotmail.com> wrote:
All -

First off a thank you to Richard Scheffenegger for his comment= s but most importantly when he asked whether the number of ACKs being sent = was excessive (since we are trying for efficiency unnecessary ACKs diminish= that efficiency).=A0 When I first wrote this RFC six years ago for my thes= is, my concern was with my extensions not in reanalyzing the base protocol.= =A0 This turns out to be a mistake because, by moving the retransmission pr= ocess farther away from using timers and changing it to purely a state basi= s, a number of statements/features of the base protocol no longer work the = same.

As a statement of policy SACK should send no more ACKs then a current m= odern version of TCP does.=A0 For the most part SACK only adds from 2 to 34= bytes in the options area of existing messages.=A0 Even with this change w= e are only talking 6 to 38 bytes.=A0 It is only in extreme situations where= there are more than four gaps in the acknowledged data that SACK requires = additional ACKs.=A0 As to when these ACKs are sent again normal TCP practic= e applies for the most part (as long as all SACK blocks occupy a single ACK= ).

The recovery basis in Enhanced SACK (E-SACK) except for the "Link = Broken" disaster timer is entirely state driven on the transmit side.= =A0 Since timers by their very nature, must be set to the worst case value,= this automated retransmission is much quicker and more accurate.=A0 The co= mbination of TCP ACKs, SACK block changes and Returned Transmit Token chang= es form the triggers that guide the recovery and transmission processes.=A0= In order to insure the proper flow of those signals the receiver is runnin= g three classes of timer - Returned Transmit Token delay timer, Second ACK = timer and the "I did not receive what I needed" keep alive timer.= =A0 More about those in the commentary.

The principle that all state changing information should be sent twice = (second ACK) from TCP is a good one and should be preserved in this protoco= l since the loss of an ACK is as disastrous as the loss of a message, we ju= st need to be cognizant that there are now three state change elements, any= of which triggers the ACK pair, specifically the TCP ACK (acknowledging a = change in the TCP transmit floor), a change to a SACK block, and finally a = change to one of the tokens (note I Include both Transmit and Returning her= e).=A0 The question then becomes how do we implement this.=A0 The underlyin= g reality here is that on a link with even moderate activity a second inten= tional ACK is not needed since the ACK triggered by the next arriving messa= ge would replace it.=A0 The problem we encounter is that ACK messages are s= ignificantly shorter than full data messages so we would always send a seco= nd ACK if we send it immediately after the first since nothing has changed.= =A0 The thought is then to delay the second ACK until the link goes idle (1= .3 times the length of time to receive a full packet on the fast side to 2.= 6 x the length of time to receive a full packet on the slow side).=A0 Note = that this delay has a direct effect on the Returned Transmit Token delay ti= mer as that timer can not be less than this delay plus the amount of time t= o transmit all of the ACKs to hold the complete list of SACK blocks in orde= r to insure that state information is up to date before the token changes, = thus possibly degrading link efficiency.

One note of clarification, if a state change triggers another pair of A= CKs, this takes precedence over completing the remainder of the ACK group i= f multiple ACKs are required to transmit the SACK blocks.

The reason= we trigger an ACK on either token changing is to get around the dreaded &q= uot;last character hang" of very early TCP implementations.=A0 If the = transmitter sends an ACK with the updated Transmit Token from the last outg= oing segment when the link goes idle it insures the receiver has the update= d value of that Transmit Token which will lead to the retransmission of tha= t last missing segment.

We delay the return of the received Transmit Token for two reasons, fir= st it allows packets that got out of order to be received and put back in o= rder before being declared missing and secondly, because a group of SACK bl= ocks might span multiple ACKs, to make sure that state information is up to= date before we declare a segment as missing.

This leads the to a discussion of very bad burst mode errors.=A0 If the= receiver has not received the segments it requires and has not sent out an= ACK for RTT*1.3 then the receiver needs to send another ACK pair.=A0 Note = this applies to the transmitter also in that if its Returned Transmit Token= does not equal Last Transmit Token after the same RTT*1.3 it is obligated = to send an ACK pair to the receiver to prompt it to restart.=A0 Obviously R= TT*1.3 is a long time on transcontinental or satellite links which is why 1= /4 RTT for an ACK pair from the receiver as a keep alive is under considera= tion, the maximum length of time to cover the burst error without degrading= efficiency too badly.

In filling the outgoing ACKs with SACK blocks at the receiver, the list= is maintained in the order oldest (smallest segment start) to the youngest= .=A0 The list is queried three times when building the ACK group, first for= all blocks that just changed and thus have the most critical information (= ACK NEEDED =3D 2), then a scan for those blocks which already had their fir= st ACK (ACK NEEDED =3D 1) and then for the blocks which have already been t= ransmitted twice (ACK NEEDED =3D 0).=A0 The ACK NEEDED count is then decrem= ented.

To make E-SACK work three lists must be maintained, one at the receiver= and two at the transmitter.=A0 The one in the receiver is identical in con= cept to the one in the original RFC with the addition of ACK NEEDED and the= proviso that it be in oldest to youngest order.=A0 At the transmitter an i= dentical list is maintained built from the recovered SACK blocks.=A0 When a= changed Returned Transmit Token is received, the recovered SACK block list= is used to construct a retransmit queue of all unacknowledged segments.=A0= Then starting at the transmit queue entry plus one derived from the Return= ed Transmit Token, and proceeding forward until all transmitted segment ext= ents have been processed, entries are removed or modified in the new retran= smit queue to compensate for segments sent after the time of the Returned T= ransmit Token.=A0 The adjusted new transmit queue replaces the existing que= ue and the transmission process is started again starting at the first entr= y on the queue, not proceeding with new transmission until the queue is com= pleted.=A0 Please note that incoming SACK blocks must be checked against th= e retransmit queue in order to remove segments that have been subsequently = correctly received.

Noto Bene - This protocol is an update to RFC 2018.=A0 It is not compat= ible with nor do I believe in the philosophy of RFC2883/RFC3708 as the whol= e purpose of this change is to eliminate duplicate segments and increase ef= ficiency and reliability where the changes outlined in those RFCs degrade e= fficiency (especially by generating extra SACK blocks) without substantiall= y improving anything.=A0 Also as adopted in the RFC there is a major fallac= y, it assumes that there can be only one SACK block with a status change wh= ere in reality, due to processing considerations, there can be two or even = more blocks with changes before an ACK can be sent.


Tony

Anthony Sabatini
200=A0West 20th Street
Apt. 1216=
New York, NY 10011

Phone: (212) 867-7179
Mobile: (9= 17) 224-8388


_______________________________________________
tcpm mailing list
tcpm@ietf.org
ht= tps://www.ietf.org/mailman/listinfo/tcpm


--f46d044267fe3d4b3004c53443f9-- From mattmathis@google.com Thu Jul 19 15:25:58 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 61E7621F86CA for ; Thu, 19 Jul 2012 15:25:58 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.844 X-Spam-Level: X-Spam-Status: No, score=-102.844 tagged_above=-999 required=5 tests=[AWL=0.133, BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5aDPW3+C-0LU for ; Thu, 19 Jul 2012 15:25:57 -0700 (PDT) Received: from mail-we0-f172.google.com (mail-we0-f172.google.com [74.125.82.172]) by ietfa.amsl.com (Postfix) with ESMTP id 7D85121F85F2 for ; Thu, 19 Jul 2012 15:25:57 -0700 (PDT) Received: by weyu54 with SMTP id u54so2432698wey.31 for ; Thu, 19 Jul 2012 15:26:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type :x-system-of-record; bh=EByfBep85PikMruP9vjkmXLYfHFIscTqkFdnZI3S9E4=; b=WbtTlV/mJLTFl8ZfATW0JmF1eys1CCdIugGRToPHTxLveMesQGDyQIg2HIbqGnND4S qZdDW66L1rL1ELqjWDjFVjmweY2JS6uM9CYFW5wqtY4ZbWmGTe6dZOapYdBDH94W/eoB AlWC7JpyQezRRhXmvz0qFS52kp16ykxJXZxe8J9JsQrYwUD94iSU3NtNttUe5DmLiFPb yBKZwPq+wE7wN9XC8ID4SirmlU9EIdq7w/B6qomiHkONyJGJbzMvPK98SRs+/ZKf655H TzhdDs9MKCf4oelPtEe82Es7hFyzXpcVxonanlaTTW4+SsQrIMkIV7iKdKpX9DAgXlgm DecA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type :x-system-of-record:x-gm-message-state; bh=EByfBep85PikMruP9vjkmXLYfHFIscTqkFdnZI3S9E4=; b=WM/LYFtfXHyiR9ylIt3Er+qHgR7zBcDM2CvPTti1t+gb9BA+ANW+gElJHfcHDnM8kj 7wyIaEYrbtDyktCHssJHps6/Q9tKDANH7BpOooYAPKE7HSZT2nTPJoAM2A53pFQFi0cS BHUUqmlq+wgSq2uZ9XZGDkf4al+i+D9x+Ih8tJ/9qWb8/ix+1NfZfoWFOycNudsU64WO 7FAAYP3r0ISfeVsFsNBYmdtmujtM204EALMX7pkXVSjOfE9Dd9yzkKpXPFDT7YPbuE3b dyVdIYSweDfmFXT5gBmtidFsUlFZG4M8qROzlRYgu7TuEAu7qy/g5PkaA4Q2J2EULWXv p9pg== Received: by 10.216.143.105 with SMTP id k83mr1938270wej.99.1342736810858; Thu, 19 Jul 2012 15:26:50 -0700 (PDT) MIME-Version: 1.0 Received: by 10.216.143.105 with SMTP id k83mr1938266wej.99.1342736810629; Thu, 19 Jul 2012 15:26:50 -0700 (PDT) Received: by 10.216.205.168 with HTTP; Thu, 19 Jul 2012 15:26:50 -0700 (PDT) Date: Thu, 19 Jul 2012 15:26:50 -0700 Message-ID: From: Matt Mathis To: TCP Maintenance and Minor Extensions WG Content-Type: text/plain; charset=ISO-8859-1 X-System-Of-Record: true X-Gm-Message-State: ALoCoQn4jnj0/Zo+o+g3bbcBnr1V1Yd7KElifWiugxYhpitHp+6Y5fYjbCcGPuckIl2IGoEjrFVR998/J/GEAfj71h8U3N6qqX03WgsLJbC9Kn9sjjctW9eJ3kFEUp57ZZVhFe7lOIRjLhcJliS1tc8cAkWxFbdpPGQT0B9xqXyllkaCKxbutYg= Subject: [tcpm] Running code: Laminar TCP X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jul 2012 22:25:58 -0000 BTW, if you want to play with Laminar TCP, try the experimental patch at https://developers.google.com/speed/protocols/tcp-laminar It replaces nearly all of congestion control code in Linux 3.5.... YMMV Thanks, --MM-- The best way to predict the future is to create it. - Alan Kay ---------- Forwarded message ---------- From: Date: Mon, Jul 16, 2012 at 2:52 PM Subject: New Version Notification for draft-mathis-tcpm-tcp-laminar-01.txt To: mattmathis@google.com A new version of I-D, draft-mathis-tcpm-tcp-laminar-01.txt has been successfully submitted by Matt Mathis and posted to the IETF repository. Filename: draft-mathis-tcpm-tcp-laminar Revision: 01 Title: Laminar TCP and the case for refactoring TCP congestion control Creation date: 2012-07-15 WG ID: Individual Submission Number of pages: 15 URL: http://www.ietf.org/internet-drafts/draft-mathis-tcpm-tcp-laminar-01.txt Status: http://datatracker.ietf.org/doc/draft-mathis-tcpm-tcp-laminar Htmlized: http://tools.ietf.org/html/draft-mathis-tcpm-tcp-laminar-01 Diff: http://tools.ietf.org/rfcdiff?url2=draft-mathis-tcpm-tcp-laminar-01 Abstract: The primary state variables used by all TCP congestion control algorithms, cwnd and ssthresh, are heavily overloaded, carrying different semantics in different states. This leads to excess implementation complexity and poorly defined behaviors under some combinations of events, such as application stalls during loss recovery. We propose a new framework for TCP congestion control, and to recast current standard algorithms to use new state variables. This new framework will not generally change the behavior of any of the primary congestion control algorithms when they are invoked in isolation. It will permit new algorithms with better behaviors in many corner cases, such as when two distinct primary algorithms are invoked concurrently. It will also foster the creation of new algorithms to address some events that are poorly treated by today's standards. For the vast majority of traditional algorithms the transformation to the new state variables is completely straightforward. However, the resulting implementation is likely to technically be in violation of existing TCP standards, even if it is fully compliant with their principles and intent. The IETF Secretariat From ycheng@google.com Fri Jul 20 07:44:17 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 10C4521F85A4 for ; Fri, 20 Jul 2012 07:44:17 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.917 X-Spam-Level: X-Spam-Status: No, score=-102.917 tagged_above=-999 required=5 tests=[AWL=0.060, BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id BFduhvAa8k+w for ; Fri, 20 Jul 2012 07:44:16 -0700 (PDT) Received: from mail-yx0-f172.google.com (mail-yx0-f172.google.com [209.85.213.172]) by ietfa.amsl.com (Postfix) with ESMTP id EEABB21F8552 for ; Fri, 20 Jul 2012 07:44:15 -0700 (PDT) Received: by yenq13 with SMTP id q13so4437116yen.31 for ; Fri, 20 Jul 2012 07:45:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type:content-transfer-encoding:x-system-of-record; bh=jgcihMxSNhd3CorkLreiiULbddSJz/XqOL3GHSMATUE=; b=FDjPfs8Dft7e3/EEtjxV+A3jTdz5UqCzYdaHxloq47pbC8m1RbFLkgns067Ou76Xi6 BkknzYkch6DvzGJ20uJzWfRN9SkRw4fliTkwQuNZNgWsJcIaZJ7vLey5+iXCPEUc+PRZ HQlEYzbz9bJFXTr3HHInxayV3OxntS+Cn31lKYlodThAga5xzZavN4INNRnGNJ4yWDbR xtveZHY+JrLFlJWArIjT5589UWihZrBXPj7UHOludvls0jyYtuLpu/FqzQ9RtpBBg3MO IW6/FkbtskJS/lQ/XXqmNiCHzUcFvISa4SlmACMQKBtXuD6owtHTTp1tzWz5sB4bp1M/ ULEQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type:content-transfer-encoding:x-system-of-record :x-gm-message-state; bh=jgcihMxSNhd3CorkLreiiULbddSJz/XqOL3GHSMATUE=; b=c9zGonTuxV9KcvM9idQYyl8zjxRwLguPxoATsxgeW4qfl1zPYTeUNr/8fcR/wa+MEI gv2CNciPLrtPEvYdcgXisVlc64swvvwIEu6ip2YTquHxS8TVdMokkFFHuig9I/sA+H51 2K8Jnlyiy1YhfSVvRi4XNVpZ3S+UQ5jydKGffYZatf4zIOFyH6y927LrijAvYZ9GqV1q To+stD/t0jRIbokv85oWmTEa2R7SgbzoIsK9DfVRrgwZ1XCMmQGcK2iChd+xgcM+akBf S+/IN3vdHbzf4n8movQmvmg01qEazUX/qfQA7Q8hr+NZaBuLiLG14CsCOKoPYpPXynwb 4bwA== Received: by 10.60.18.134 with SMTP id w6mr7298776oed.56.1342795273076; Fri, 20 Jul 2012 07:41:13 -0700 (PDT) Received: by 10.60.18.134 with SMTP id w6mr7284506oed.56.1342795087172; Fri, 20 Jul 2012 07:38:07 -0700 (PDT) MIME-Version: 1.0 Received: by 10.60.14.168 with HTTP; Fri, 20 Jul 2012 07:37:47 -0700 (PDT) In-Reply-To: <500845E0.8050601@isi.edu> References: <50049F5A.7000709@isi.edu> <50082223.9010701@isi.edu> <500845E0.8050601@isi.edu> From: Yuchung Cheng Date: Fri, 20 Jul 2012 07:37:47 -0700 Message-ID: To: Joe Touch Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable X-System-Of-Record: true X-Gm-Message-State: ALoCoQmErYirD5DLd7ViE2sSTDekGwIZKY2/7HZpSmiXalBzmOh3lrfenIkDW+AQijjLLj5udk6ReNHomAICgQg9zrit67IY1uA1cAtay5l+7rQzbhz9eUFLVH7TCQDhgep9TYf12Rh3+nIqL35OTjFrudChvW/SSvVRlP5hSd2ajeCKk3lTypo= Cc: "iccrg@cs.ucl.ac.uk" , "tcpm@ietf.org Extensions" , "Arjuna Sathiaseelan \(work\)" Subject: Re: [tcpm] Review of draft-fairhurst-tcpm-newcwv-03 X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jul 2012 14:44:17 -0000 On Thu, Jul 19, 2012 at 10:37 AM, Joe Touch wrote: > > > On 7/19/2012 10:26 AM, Yuchung Cheng wrote: >> >> On Thu, Jul 19, 2012 at 8:05 AM, Joe Touch wrote: >>> >>> >>> >>> On 7/16/2012 4:10 PM, Joe Touch wrote: >>>> >>>> >>>> >>>> >>>> On 7/16/2012 3:56 PM, Yuchung Cheng wrote: >>>>> >>>>> >>>>> Summary: >>>>> >>>>> First of all the problem this draft is trying to solve is important: >>>>> AFAIK servers and data-centers disable slow-start after idle because >>>>> it simply hurts latency too badly. >>>> >>>> >>>> >>>> If they're mostly doing HTTP, it shouldn't matter at all. HTTP's >>>> transaction pattern defeats slow-start after idle anyway. > >> >> >> Why not if the interval between two HTTP responses are large, say two >> minutes? >> Your draft below suggested receiving HTTP requests have some effects but >> I didn't see that in RFC 2861 (and I couldn't find the mailing list >> discussion >> cited either). > > > Let's say the connection goes idle for 2 minutes. TCP doesn't do anything > when a connection goes idle; it only does things when something is sent (= in > general). > > The most likely reason the server would have something new to send is tha= t > the client sends a new request. The client request is short enough to fit= in > 1-2 segments, so shouldn't be an issue (and Nagle should be off, since it= 's > interactive with multibyte exchanges). > > The server receives the request, and then sends a response. The server ha= s a > response that's typically larger - thus the burst. The server decides "ho= w > long has it been since I *received* anything? Since that time is very sho= rt, but RFC 2861 does not use the most recently received timing? not in their main algorithm (section 3.2) at least. > there's no slow-start restart, and the burst goes out. > > The only time the server would timeout and do slowstart restart would be = if > it decides to send something a while after anything has been received - t= hat > might occur for auto-refresh pages, but not much else. > > The issue is explained in detail here: J. Heidemann, K. Obraczka, J. Touc= h, > =93Modeling the Performance of HTTP Over Several Transport Protocols,=94 > IEEE/ACM Transactions on Networking, V5, N5, Oct. 1997, pp.616-630. But why does receiving something recently justify the sending cwnd is safe = to not cause burst-induced losses? Sorry but I can't find the explanation in the paper. > > Joe From touch@isi.edu Fri Jul 20 08:25:16 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9509A21F8526 for ; Fri, 20 Jul 2012 08:25:16 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -104.954 X-Spam-Level: X-Spam-Status: No, score=-104.954 tagged_above=-999 required=5 tests=[AWL=1.645, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 9abp+gaMfmgP for ; Fri, 20 Jul 2012 08:25:15 -0700 (PDT) Received: from boreas.isi.edu (boreas.isi.edu [128.9.160.161]) by ietfa.amsl.com (Postfix) with ESMTP id B37AD21F849C for ; Fri, 20 Jul 2012 08:25:15 -0700 (PDT) Received: from [192.168.1.92] (pool-71-105-89-105.lsanca.dsl-w.verizon.net [71.105.89.105]) (authenticated bits=0) by boreas.isi.edu (8.13.8/8.13.8) with ESMTP id q6KFPW7Q013886 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Fri, 20 Jul 2012 08:25:41 -0700 (PDT) Message-ID: <5009786B.80707@isi.edu> Date: Fri, 20 Jul 2012 08:25:31 -0700 From: Joe Touch User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:13.0) Gecko/20120614 Thunderbird/13.0.1 MIME-Version: 1.0 To: Yuchung Cheng References: <50049F5A.7000709@isi.edu> <50082223.9010701@isi.edu> <500845E0.8050601@isi.edu> In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Cc: "iccrg@cs.ucl.ac.uk" , "tcpm@ietf.org Extensions" , "Arjuna Sathiaseelan \(work\)" Subject: Re: [tcpm] Review of draft-fairhurst-tcpm-newcwv-03 X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jul 2012 15:25:16 -0000 On 7/20/2012 7:37 AM, Yuchung Cheng wrote: ... >> The server receives the request, and then sends a response. The server has a >> response that's typically larger - thus the burst. The server decides "how >> long has it been since I *received* anything? Since that time is very short, > > but RFC 2861 does not use the most recently received timing? not in their > main algorithm (section 3.2) at least. I'm referring to the algorithm as per Standard TCP [RFC5681] which, resets CWND to the restart window after the app goes idle (as described in this doc). However, RFC 2681 failed to note that the restart algorithm was widely deployed at the time, and was based on erroneous logic in a footnote of an unpublished extension of Van Jacobson's 1988 congestion control paper (ax explained in detail in draft-hughes-restart-00.txt). >> there's no slow-start restart, and the burst goes out. >> >> The only time the server would timeout and do slowstart restart would be if >> it decides to send something a while after anything has been received - that >> might occur for auto-refresh pages, but not much else. >> >> The issue is explained in detail here: J. Heidemann, K. Obraczka, J. Touch, >> “Modeling the Performance of HTTP Over Several Transport Protocols,” >> IEEE/ACM Transactions on Networking, V5, N5, Oct. 1997, pp.616-630. > > But why does receiving something recently justify the sending cwnd is safe to > not cause burst-induced losses? Sorry but I can't find the explanation in > the paper. It doesn't justify it - it was an incorrect conclusion. The goal is "restart if you haven't SENT in a while", but Van's footnote said that most implementations at that time had a 'time since last received' timer, but not a 'time since last sent', and that because TCP is symmetric you can use the receive timer instead of the send one. That logic was false, and persistent HTTP turned out to have exactly that pattern. However, the mechanism proposed in this doc still allows most HTTP exchanges to burst an entire window - because it would be very likely that users will click on URLs within a received page within the 5 minute timeout. I.e., I don't think section 4 of this doc is appropriate (it still amounts to a send timer), and still favor the "burst-or-lose" style approach as recommended in draft-hughes-restart-00.txt. Joe From tsabatini@hotmail.com Fri Jul 20 11:15:27 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 36A1D11E8097 for ; Fri, 20 Jul 2012 11:15:27 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.598 X-Spam-Level: X-Spam-Status: No, score=-2.598 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id NSvvWkbXel0a for ; Fri, 20 Jul 2012 11:15:25 -0700 (PDT) Received: from bay0-omc2-s18.bay0.hotmail.com (bay0-omc2-s18.bay0.hotmail.com [65.54.190.93]) by ietfa.amsl.com (Postfix) with ESMTP id 58C3711E8093 for ; Fri, 20 Jul 2012 11:15:24 -0700 (PDT) Received: from BAY158-W13 ([65.54.190.124]) by bay0-omc2-s18.bay0.hotmail.com with Microsoft SMTPSVC(6.0.3790.4675); Fri, 20 Jul 2012 11:16:20 -0700 Message-ID: Content-Type: multipart/alternative; boundary="_9cdc9bb6-e57a-42fe-b338-791177fe51c6_" X-Originating-IP: [69.112.169.100] From: Anthony Sabatini To: Date: Fri, 20 Jul 2012 14:16:20 -0400 Importance: Normal In-Reply-To: References: , , , MIME-Version: 1.0 X-OriginalArrivalTime: 20 Jul 2012 18:16:20.0314 (UTC) FILETIME=[C3229BA0:01CD66A3] Cc: tcpm Subject: Re: [tcpm] TCP Error recovery and efficiency X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jul 2012 18:15:27 -0000 --_9cdc9bb6-e57a-42fe-b338-791177fe51c6_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Matt - =20 First off I would like to thank you for your excellent work on RFC2018=2C t= he purpose of this update is not to diminish your work but rather extend an= d expand on it. Without your lead I would never have had the impetus to ex= tend my work from HDLC into the TCP domain. Please understand that the fir= st draft was a trial draft to toss around and get essential comments. Also= please understand that it was originally written as part of a thesis prese= ntation on adapting token=2C state driven recovery to TCP=2C not as an RFC = per se. =20 1) A problem with RFC2018 is that it provides no real information with resp= ect to the receivers reassembly queue at the time the transmitter receives = the acknowledgement. The protocol presupposes conditions often not present= in the real world=2C that the information transfer is near instantaneous a= nd that a rather low number of messages can be in-flight i.e. 7 SACK blocks= total combined in both directions (i.e. messages for which SACK blocks hav= e not yet been received). The RFC2018 DOES NOT send information about mess= ages that were lost (nor can it with the implied requirement of packet frag= mentation or out of order delivery)=2C it can only say what it has seen. =20 2) On an active link SACK goes into the weeds whenever noise bursts affecti= ng receipt of SACK blocks exceed approximately 100=2C000 bit times=2C a 10 = ms noise burst at 10 mbps or 1 ms at 100 mbps causing a rollover of the act= ive three SACK block window. At that point SACK goes into timer based reco= very. The problem with time based recovery is that timers must always be se= t to the maximum possible delay=2C destroying efficiency. The purpose of t= his change is to take recovery from timer based to state based so that it i= s maximally responsive. =20 3) SACK presupposes a relatively stable link with a relatively low random p= acket drop. Neither are true anymore and it is why you need only to look a= t RFC2883 to realize that SACK sends a lot of duplicate segments destroying= efficiency. I am simply trying to reduce that number. =20 4) SACK has an implicit "last character problem"=3B normally you would get = multiple confirmation SACK blocks as additional messages arrive equal to th= e number of SACK blocks. This is not true if the link goes idle where the = next to last only has two acknowledgements and the last only has one. If t= he last transmit segment is dropped and the link goes idle or the last ACK = packet from the receiver is lost you are now into a condition where it take= s a major link timeout to recover it. The changes in the revised draft red= uce the pssibility of that occurrence. =20 5) I admit the current draft does not properly express my views. In the cu= rrent RFC2018 we normally have three SACK blocks=2C in this version normall= y four. What I was basically saying is that UNUSED SACK blocks should be f= illed with older=2C still not completed SACK blocks rather than transmit th= e same set of SACK blocks five=2C six or more times. This will be clarifie= d in the next revision. =20 6) Note that the argument has moved to whether we need to allow more that 1= 6=2C000 messages to be in process during any particular second in the face = of 10 Gbps links=2C proposal is on the table to expand that to 4=2C000=2C00= 0. Also there is a proposal on SACK block compression in process (RFC1072 = was not wrong in concept only in implementation) so that we can include mor= e segments. =20 I do appreciate your comments and feedback. =20 Tony=20 Anthony Sabatini 200 West 20th Street Apt. 1216 New York=2C NY 10011 Phone: (212) 867-7179 Mobile: (917) 224-8388 =20 =20 Date: Thu=2C 19 Jul 2012 13:02:43 -0700 Subject: Re: [tcpm] TCP Error recovery and efficiency From: mattmathis@google.com To: tsabatini@hotmail.com CC: tcpm@ietf.org I don't understand what problem you are trying to solve: As written 2018 pr= ovides the sender with perfect information about the state of the receivers= reassembly queue (e.g. which data is still missing) as long as most ACKs a= re delivered=2C mostly in order. The minimum cases which causes the sende= r to send duplicate data are timeouts (but see the errata against 2018)=2C = or fairly complicated patterns of sustained losses on both sides of the lin= k. In fact=2C if there is no loss or reordering on the return path=2C a single= SACK block is sufficient to give the sender perfect information. With classic TCP timestamps (not for PAWS) it is even possible to disambigu= ate between holes that were filled by ooo data or retransmissions. Due to = the TS echo rules for PAWS=2C it actually does less well. I am not the exp= ert here=2C but Richard Scheffenegger has explored this problem extensively= . But the ambiguity is whether congestion control undo is permitted=2C no= t which data is still needed at the receiver. If you look at 2018=2C (and 2883) the first SACK block is described as MUST= be the chronologically newest. The rest of the option is described with S= HOULDs=2C because we knew that SACK could theoretically be improved by dith= ering the rest of the SACK option across all past SACK blocks. Such a sch= eme would make SACK work well with unreasonable levels of loss on the ACK p= ath. (e.g. the 2nd SACK block could alternate between the two most recent = prior 1st blocks=2C the 3rd across all earlier blocks). But now=2C would that really a feature for TCP to work well on the forward = path=2C while the (low bandwidth) return path is being mauled by SACK? Thanks=2C --MM-- The best way to predict the future is to create it. - Alan Kay On Sun=2C Jul 15=2C 2012 at 12:59 PM=2C Anthony Sabatini wrote: All - First off a thank you to Richard Scheffenegger for his comments but most im= portantly when he asked whether the number of ACKs being sent was excessive= (since we are trying for efficiency unnecessary ACKs diminish that efficie= ncy). When I first wrote this RFC six years ago for my thesis=2C my concer= n was with my extensions not in reanalyzing the base protocol. This turns = out to be a mistake because=2C by moving the retransmission process farther= away from using timers and changing it to purely a state basis=2C a number= of statements/features of the base protocol no longer work the same. As a statement of policy SACK should send no more ACKs then a current moder= n version of TCP does. For the most part SACK only adds from 2 to 34 bytes= in the options area of existing messages. Even with this change we are on= ly talking 6 to 38 bytes. It is only in extreme situations where there are= more than four gaps in the acknowledged data that SACK requires additional= ACKs. As to when these ACKs are sent again normal TCP practice applies fo= r the most part (as long as all SACK blocks occupy a single ACK). The recovery basis in Enhanced SACK (E-SACK) except for the "Link Broken" d= isaster timer is entirely state driven on the transmit side. Since timers = by their very nature=2C must be set to the worst case value=2C this automat= ed retransmission is much quicker and more accurate. The combination of TC= P ACKs=2C SACK block changes and Returned Transmit Token changes form the t= riggers that guide the recovery and transmission processes. In order to in= sure the proper flow of those signals the receiver is running three classes= of timer - Returned Transmit Token delay timer=2C Second ACK timer and the= "I did not receive what I needed" keep alive timer. More about those in t= he commentary. The principle that all state changing information should be sent twice (sec= ond ACK) from TCP is a good one and should be preserved in this protocol si= nce the loss of an ACK is as disastrous as the loss of a message=2C we just= need to be cognizant that there are now three state change elements=2C any= of which triggers the ACK pair=2C specifically the TCP ACK (acknowledging = a change in the TCP transmit floor)=2C a change to a SACK block=2C and fina= lly a change to one of the tokens (note I Include both Transmit and Returni= ng here). The question then becomes how do we implement this. The underly= ing reality here is that on a link with even moderate activity a second int= entional ACK is not needed since the ACK triggered by the next arriving mes= sage would replace it. The problem we encounter is that ACK messages are s= ignificantly shorter than full data messages so we would always send a seco= nd ACK if we send it immediately after the first since nothing has changed.= The thought is then to delay the second ACK until the link goes idle (1.3= times the length of time to receive a full packet on the fast side to 2.6 = x the length of time to receive a full packet on the slow side). Note that= this delay has a direct effect on the Returned Transmit Token delay timer = as that timer can not be less than this delay plus the amount of time to tr= ansmit all of the ACKs to hold the complete list of SACK blocks in order to= insure that state information is up to date before the token changes=2C th= us possibly degrading link efficiency. One note of clarification=2C if a state change triggers another pair of ACK= s=2C this takes precedence over completing the remainder of the ACK group i= f multiple ACKs are required to transmit the SACK blocks. The reason we trigger an ACK on either token changing is to get around the = dreaded "last character hang" of very early TCP implementations. If the tr= ansmitter sends an ACK with the updated Transmit Token from the last outgoi= ng segment when the link goes idle it insures the receiver has the updated = value of that Transmit Token which will lead to the retransmission of that = last missing segment. We delay the return of the received Transmit Token for two reasons=2C first= it allows packets that got out of order to be received and put back in ord= er before being declared missing and secondly=2C because a group of SACK bl= ocks might span multiple ACKs=2C to make sure that state information is up = to date before we declare a segment as missing. This leads the to a discussion of very bad burst mode errors. If the recei= ver has not received the segments it requires and has not sent out an ACK f= or RTT*1.3 then the receiver needs to send another ACK pair. Note this app= lies to the transmitter also in that if its Returned Transmit Token does no= t equal Last Transmit Token after the same RTT*1.3 it is obligated to send = an ACK pair to the receiver to prompt it to restart. Obviously RTT*1.3 is = a long time on transcontinental or satellite links which is why 1/4 RTT for= an ACK pair from the receiver as a keep alive is under consideration=2C th= e maximum length of time to cover the burst error without degrading efficie= ncy too badly. In filling the outgoing ACKs with SACK blocks at the receiver=2C the list i= s maintained in the order oldest (smallest segment start) to the youngest. = The list is queried three times when building the ACK group=2C first for a= ll blocks that just changed and thus have the most critical information (AC= K NEEDED =3D 2)=2C then a scan for those blocks which already had their fir= st ACK (ACK NEEDED =3D 1) and then for the blocks which have already been t= ransmitted twice (ACK NEEDED =3D 0). The ACK NEEDED count is then decremen= ted.=20 To make E-SACK work three lists must be maintained=2C one at the receiver a= nd two at the transmitter. The one in the receiver is identical in concept= to the one in the original RFC with the addition of ACK NEEDED and the pro= viso that it be in oldest to youngest order. At the transmitter an identic= al list is maintained built from the recovered SACK blocks. When a changed= Returned Transmit Token is received=2C the recovered SACK block list is us= ed to construct a retransmit queue of all unacknowledged segments. Then st= arting at the transmit queue entry plus one derived from the Returned Trans= mit Token=2C and proceeding forward until all transmitted segment extents h= ave been processed=2C entries are removed or modified in the new retransmit= queue to compensate for segments sent after the time of the Returned Trans= mit Token. The adjusted new transmit queue replaces the existing queue and= the transmission process is started again starting at the first entry on t= he queue=2C not proceeding with new transmission until the queue is complet= ed. Please note that incoming SACK blocks must be checked against the retr= ansmit queue in order to remove segments that have been subsequently correc= tly received. Noto Bene - This protocol is an update to RFC 2018. It is not compatible w= ith nor do I believe in the philosophy of RFC2883/RFC3708 as the whole purp= ose of this change is to eliminate duplicate segments and increase efficien= cy and reliability where the changes outlined in those RFCs degrade efficie= ncy (especially by generating extra SACK blocks) without substantially impr= oving anything. Also as adopted in the RFC there is a major fallacy=2C it = assumes that there can be only one SACK block with a status change where in= reality=2C due to processing considerations=2C there can be two or even mo= re blocks with changes before an ACK can be sent. Tony Anthony Sabatini 200 West 20th Street Apt. 1216 New York=2C NY 10011 Phone: (212) 867-7179 Mobile: (917) 224-8388 _______________________________________________ tcpm mailing list tcpm@ietf.org https://www.ietf.org/mailman/listinfo/tcpm = --_9cdc9bb6-e57a-42fe-b338-791177fe51c6_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
Matt -
 =3B
First off I would like to thank you for your excellent work on RFC2018=2C t= he purpose of this update is not to diminish your work but rather extend an= d expand on it. =3B Without your lead I would never have had the impetu= s to extend my work from HDLC into the TCP domain. =3B Please understan= d that the first draft was a trial draft to toss around and get essential c= omments. =3B Also please understand that it was originally written as p= art of a thesis presentation on adapting token=2C state driven recovery to = TCP=2C not as an RFC per se.
 =3B
1) =3BA problem with =3BRFC2018 is that it provides =3Bno real = information with respect to the receivers reassembly queue at the time the = transmitter receives the acknowledgement. =3B =3BThe =3Bprotoco= l presupposes conditions often not present in the real world=2C that the in= formation transfer is near instantaneous and that a rather low number of me= ssages can be in-flight i.e. 7 SACK blocks total combined in both direction= s (i.e. messages for which SACK blocks have not yet been received). =3B= The RFC2018 DOES NOT send information about messages that were lost (nor c= an it with the implied requirement of packet fragmentation or out of order = delivery)=2C it can only say what it has seen.
 =3B
2) On an active link SACK goes into the weeds whenever noise bursts affecti= ng receipt of SACK blocks exceed approximately 100=2C000 bit times=2C a 10 = ms noise burst at 10 mbps or 1 ms at 100 mbps causing a rollover of the act= ive three SACK block window. =3B At that point SACK goes into timer bas= ed recovery. The problem with time based recovery is that timers must alway= s be set to the maximum possible delay=2C destroying efficiency. =3B Th= e purpose of this change is to take recovery from timer based to state base= d so that it is maximally responsive.
 =3B
3) SACK presupposes a relatively stable link with a relatively low random p= acket drop. =3B Neither are true anymore and it is why you =3Bneed = only to look at RFC2883 to realize that SACK sends a lot of duplicate = =3Bsegments destroying efficiency. =3B I am simply trying to =3Bred= uce that number.
 =3B
4) SACK has an implicit "last character problem"=3B normally you would get = multiple confirmation SACK blocks as additional messages arrive equal to th= e number of SACK blocks. =3B This is not true if the link goes idle whe= re the next to last only has two acknowledgements and the last only has one= . =3B If the last transmit segment is dropped and the link goes idle or= the last ACK packet from the receiver is lost you are now into a condition= where it takes a major link timeout to recover it. =3B The changes in = the revised draft reduce the pssibility of that occurrence.
 =3B
5) I admit the current draft does not properly express my views. =3B In= the current =3BRFC2018 we normally have three SACK blocks=2C in this v= ersion normally four. =3B What I was basically saying is that UNUSED&nb= sp=3BSACK blocks should be filled with older=2C still not completed SACK bl= ocks rather than transmit the same set of SACK blocks five=2C six or more t= imes. =3B This will be clarified in the next =3Brevision.
 =3B
6) Note that the argument has moved to whether we need to allow more that&n= bsp=3B16=2C000 messages to be in process during any particular second in th= e face of 10 Gbps links=2C proposal is on the table to expand that to 4=2C0= 00=2C000. =3B Also there is a proposal on =3BSACK block =3Bcomp= ression in process (RFC1072 was not wrong in concept only in implementation= ) so that we can include more segments.
 =3B
I do appreciate your comments and feedback.
 =3B
Tony =3B

Anthony Sabatini
200 =3BWest 20th Street
Apt.= 1216
New York=2C NY 10011

Phone: (212) 867-7179
Mobile: (917)= 224-8388

 =3B

 =3B
Date: Thu=2C 19 Jul 2012 13:02:43 -0700
Subject: Re: [tcpm] TCP Error re= covery and efficiency
From: mattmathis@google.com
To: tsabatini@hotma= il.com
CC: tcpm@ietf.org

I don't understand what problem you are = trying to solve: As written 2018 provides the sender with perfect informati= on about the state of the receivers reassembly queue (e.g. which data is st= ill missing) as long as most ACKs are delivered=2C mostly in order.  = =3B The minimum cases which causes the sender to send =3Bduplicate = =3Bdata are timeouts (but see the errata against 2018)=2C or fairly complic= ated patterns of sustained =3Blosses =3Bon both sides of the link.

In fact=2C if there is no loss or =3Breordering =3Bon the retu= rn path=2C a single SACK block is =3Bsufficient =3Bto give the send= er perfect information.

With classic TCP timestamps (not for PAWS) it is even possible to disa= mbiguate between holes that were filled by ooo data or retransmissions. &nb= sp=3BDue to the TS echo rules for PAWS=2C it actually does less well.  = =3BI am not the expert here=2C but =3BRichard Scheffenegger has =3B= explored =3Bthis problem extensively.  =3B But the =3Bambiguity=  =3Bis =3Bwhether congestion control undo is permitted=2C not which= data is still needed at the receiver.

If you look at 2018=2C (and 2883) the first SACK block is described as= MUST be the chronologically newest.  =3BThe rest of the option is desc= ribed with SHOULDs=2C because we knew that SACK could theoretically be impr= oved by dithering the rest of the SACK option across all past SACK blocks. =  =3B Such a scheme would make SACK work well with unreasonable levels o= f loss on the ACK path.  =3B(e.g. the 2nd SACK block could alternate be= tween the two most recent prior 1st =3Bblocks=2C the 3rd =3Bacross&= nbsp=3Ball earlier blocks).

But now=2C would that really a feature for TCP to work well on the for= ward path=2C while the (low bandwidth) return path is being mauled by SACK?=

Thanks=2C
--MM--
The best way to predict the future is to create it.  =3B= - Alan Kay



On Sun=2C Jul 15=2C 2012 at 12:59 PM=2C Anthony= Sabatini <=3Bts= abatini@hotmail.com>=3B wrote:
All -

First off a thank you to Richard Scheffenegger = for his comments but most importantly when he asked whether the number of A= CKs being sent was excessive (since we are trying for efficiency unnecessar= y ACKs diminish that efficiency). =3B When I first wrote this RFC six y= ears ago for my thesis=2C my concern was with my extensions not in reanalyz= ing the base protocol. =3B This turns out to be a mistake because=2C by= moving the retransmission process farther away from using timers and chang= ing it to purely a state basis=2C a number of statements/features of the ba= se protocol no longer work the same.

As a statement of policy SACK s= hould send no more ACKs then a current modern version of TCP does. =3B = For the most part SACK only adds from 2 to 34 bytes in the options area of = existing messages. =3B Even with this change we are only talking 6 to 3= 8 bytes. =3B It is only in extreme situations where there are more than= four gaps in the acknowledged data that SACK requires additional ACKs.&nbs= p=3B As to when these ACKs are sent again normal TCP practice applies for t= he most part (as long as all SACK blocks occupy a single ACK).

The r= ecovery basis in Enhanced SACK (E-SACK) except for the "Link Broken" disast= er timer is entirely state driven on the transmit side. =3B Since timer= s by their very nature=2C must be set to the worst case value=2C this autom= ated retransmission is much quicker and more accurate. =3B The combinat= ion of TCP ACKs=2C SACK block changes and Returned Transmit Token changes f= orm the triggers that guide the recovery and transmission processes. = =3B In order to insure the proper flow of those signals the receiver is run= ning three classes of timer - Returned Transmit Token delay timer=2C Second= ACK timer and the "I did not receive what I needed" keep alive timer. = =3B More about those in the commentary.

The principle that all state= changing information should be sent twice (second ACK) from TCP is a good = one and should be preserved in this protocol since the loss of an ACK is as= disastrous as the loss of a message=2C we just need to be cognizant that t= here are now three state change elements=2C any of which triggers the ACK p= air=2C specifically the TCP ACK (acknowledging a change in the TCP transmit= floor)=2C a change to a SACK block=2C and finally a change to one of the t= okens (note I Include both Transmit and Returning here). =3B The questi= on then becomes how do we implement this. =3B The underlying reality he= re is that on a link with even moderate activity a second intentional ACK i= s not needed since the ACK triggered by the next arriving message would rep= lace it. =3B The problem we encounter is that ACK messages are signific= antly shorter than full data messages so we would always send a second ACK = if we send it immediately after the first since nothing has changed. = =3B The thought is then to delay the second ACK until the link goes idle (1= .3 times the length of time to receive a full packet on the fast side to 2.= 6 x the length of time to receive a full packet on the slow side). =3B = Note that this delay has a direct effect on the Returned Transmit Token del= ay timer as that timer can not be less than this delay plus the amount of t= ime to transmit all of the ACKs to hold the complete list of SACK blocks in= order to insure that state information is up to date before the token chan= ges=2C thus possibly degrading link efficiency.

One note of clarific= ation=2C if a state change triggers another pair of ACKs=2C this takes prec= edence over completing the remainder of the ACK group if multiple ACKs are = required to transmit the SACK blocks.

The reason we trigger an ACK o= n either token changing is to get around the dreaded "last character hang" = of very early TCP implementations. =3B If the transmitter sends an ACK = with the updated Transmit Token from the last outgoing segment when the lin= k goes idle it insures the receiver has the updated value of that Transmit = Token which will lead to the retransmission of that last missing segment.
We delay the return of the received Transmit Token for two reasons=2C= first it allows packets that got out of order to be received and put back = in order before being declared missing and secondly=2C because a group of S= ACK blocks might span multiple ACKs=2C to make sure that state information = is up to date before we declare a segment as missing.

This leads the= to a discussion of very bad burst mode errors. =3B If the receiver has= not received the segments it requires and has not sent out an ACK for RTT*= 1.3 then the receiver needs to send another ACK pair. =3B Note this app= lies to the transmitter also in that if its Returned Transmit Token does no= t equal Last Transmit Token after the same RTT*1.3 it is obligated to send = an ACK pair to the receiver to prompt it to restart. =3B Obviously RTT*= 1.3 is a long time on transcontinental or satellite links which is why 1/4 = RTT for an ACK pair from the receiver as a keep alive is under consideratio= n=2C the maximum length of time to cover the burst error without degrading = efficiency too badly.

In filling the outgoing ACKs with SACK blocks = at the receiver=2C the list is maintained in the order oldest (smallest seg= ment start) to the youngest. =3B The list is queried three times when b= uilding the ACK group=2C first for all blocks that just changed and thus ha= ve the most critical information (ACK NEEDED =3D 2)=2C then a scan for thos= e blocks which already had their first ACK (ACK NEEDED =3D 1) and then for = the blocks which have already been transmitted twice (ACK NEEDED =3D 0).&nb= sp=3B The ACK NEEDED count is then decremented.

To make E-SACK work= three lists must be maintained=2C one at the receiver and two at the trans= mitter. =3B The one in the receiver is identical in concept to the one = in the original RFC with the addition of ACK NEEDED and the proviso that it= be in oldest to youngest order. =3B At the transmitter an identical li= st is maintained built from the recovered SACK blocks. =3B When a chang= ed Returned Transmit Token is received=2C the recovered SACK block list is = used to construct a retransmit queue of all unacknowledged segments. = =3B Then starting at the transmit queue entry plus one derived from the Ret= urned Transmit Token=2C and proceeding forward until all transmitted segmen= t extents have been processed=2C entries are removed or modified in the new= retransmit queue to compensate for segments sent after the time of the Ret= urned Transmit Token. =3B The adjusted new transmit queue replaces the = existing queue and the transmission process is started again starting at th= e first entry on the queue=2C not proceeding with new transmission until th= e queue is completed. =3B Please note that incoming SACK blocks must be= checked against the retransmit queue in order to remove segments that have= been subsequently correctly received.

Noto Bene - This protocol is = an update to RFC 2018. =3B It is not compatible with nor do I believe i= n the philosophy of RFC2883/RFC3708 as the whole purpose of this change is = to eliminate duplicate segments and increase efficiency and reliability whe= re the changes outlined in those RFCs degrade efficiency (especially by gen= erating extra SACK blocks) without substantially improving anything. = =3B Also as adopted in the RFC there is a major fallacy=2C it assumes that = there can be only one SACK block with a status change where in reality=2C d= ue to processing considerations=2C there can be two or even more blocks wit= h changes before an ACK can be sent.


Tony

Anthony Sabatini
200 =3BWest = 20th Street
Apt. 1216
New York=2C NY 10011

Phone: (212) 867-7179
Mobile: (917) 224-8388<= BR>

______________________________________________= _
tcpm mailing list
tcpm@ietf.org
https://www.ietf.org/mailman/listinfo/tcpm

<= BR>
= --_9cdc9bb6-e57a-42fe-b338-791177fe51c6_-- From rs@netapp.com Fri Jul 20 14:19:48 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A8B3D11E809C for ; Fri, 20 Jul 2012 14:19:48 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -9.561 X-Spam-Level: X-Spam-Status: No, score=-9.561 tagged_above=-999 required=5 tests=[AWL=0.439, BAYES_00=-2.599, J_CHICKENPOX_42=0.6, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id J3likxxL4xeV for ; Fri, 20 Jul 2012 14:19:47 -0700 (PDT) Received: from mx2.netapp.com (mx2.netapp.com [216.240.18.37]) by ietfa.amsl.com (Postfix) with ESMTP id 8304911E809B for ; Fri, 20 Jul 2012 14:19:47 -0700 (PDT) X-IronPort-AV: E=Sophos;i="4.77,626,1336374000"; d="scan'208";a="667450105" Received: from smtp2.corp.netapp.com ([10.57.159.114]) by mx2-out.netapp.com with ESMTP; 20 Jul 2012 14:20:29 -0700 Received: from vmwexceht03-prd.hq.netapp.com (vmwexceht03-prd.hq.netapp.com [10.106.76.241]) by smtp2.corp.netapp.com (8.13.1/8.13.1/NTAP-1.6) with ESMTP id q6KLKT97020469; Fri, 20 Jul 2012 14:20:29 -0700 (PDT) Received: from SACEXCMBX02-PRD.hq.netapp.com ([169.254.1.34]) by vmwexceht03-prd.hq.netapp.com ([10.106.76.241]) with mapi id 14.02.0298.004; Fri, 20 Jul 2012 14:20:28 -0700 From: "Scheffenegger, Richard" To: Anthony Sabatini , "mattmathis@google.com" Thread-Topic: [tcpm] TCP Error recovery and efficiency Thread-Index: AQHNXeR5wlqJnmRWo0WikYEea35jtJcmOUgAgAUJBACABkotgIABdJwA//+yzVA= Date: Fri, 20 Jul 2012 21:20:28 +0000 Message-ID: <012C3117EDDB3C4781FD802A8C27DD4F0D4C4E58@SACEXCMBX02-PRD.hq.netapp.com> References: , , , In-Reply-To: Accept-Language: de-AT, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.104.60.115] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: tcpm Subject: Re: [tcpm] TCP Error recovery and efficiency X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jul 2012 21:19:48 -0000 Hi Tony, I think you need to clarify more the issues you see in the current signalin= g of SACK; 2018 SACK gives the most complete view of the receiver state at = the time when the ACK carrying the SACK was sent... I don't quite follow th= e argument around larger window links... Ad 1) Your point here seems to indicate that the receiver is supposed to extrapol= ate the (then future) state of the receiver at the time, when the sender (i= n your notion, transmitter) is receiving an ACK.. But to make that extrapol= ation, the receiver has to run a model of the path, which is much more comp= lex in a packet switched network than on p2p links (over "simple" media).=20 Also, in the real world of TCP/IP, one cannot know if a packet was lost or = delayed; this is also a major differentiating factor to P2P links (HDLC, SD= LC etc., where you seem to be coming from). Therefore, the receiver cannot = deduce that Packet 1 was lost, once it has received Packet 2 (with these pa= cket numbers encoded in the tokens). It can merely state that Packet 1 was = either delayed (any may be received still), or lost - less information here= than in a P2P link! In accordance with the principles of TCP, though, the sender could make the= deduction that if the 1st retransmission of packet 1 was not received by t= he time packet 3 - sent out after the retransmission of 1 - was confirmed o= n the receiving side, that a 2nd retransmission is in order.=20 And this really is a undefined space so far (linux does some tricks that ar= e not in the IETF specs, but in the spirit). Your token scheme (making each packet uniquely identifiable) would also go = a long way to enable this. Ad 2) Burst loss of the ACKs: True; but burst loss of ACKs of that severity has o= ther implications in TCP, e.g. the ACK clock will fail (or SACK without PRR= -RB/Laminar TCP will send out burst of new packets, very possibly making th= ings worse). As Matt mentioned in his reply, a receiver is free to dither the returned S= ACK blocks from the available SACK blocks. This could address this continge= ncy to some point. But I figure it's not trivial to figure out an optimal d= ithering strategy - e.g. blocks could also be repeated with varying probabi= lity to indicate their "importance". As indicated earlier, you place a very= high value at the oldest block, to facilitate some forward progress on the= receiver side as soon as possible (and to reset the RTO timer?) Thus the o= ldest block could be included in a certain fraction of ACKs, overriding the= current LIFO ordering scheme... Ad 3) I don't buy this one; you proposed to send ACKs multiple times over and ove= r - clearly much less efficient than establishing reliability by sending th= e same SACK block at least 3 times (thus covering at least 2 consecutive AC= K losses). Ad 4) See the rescue retransmission in RFC3517bis and draft-dukkipati-tcpm-tcp-lo= ss-probe; but having a fast re-ACK timer (~ inter packet interval) on the r= eceiver appears to be quicker and possibly more efficient (often, tx power = < rx power on mobile devices; smaller packets don't use up too much bandwid= th as simply resending the last packet after a ~RTO timer). Ad 5) > 5) I admit the current draft does not properly express > my views. In the current RFC2018 we normally have > three SACK blocks, in this version normally four.=20 > What I was basically saying is that UNUSED SACK blocks > should be filled with older, still not completed SACK > blocks rather than transmit the same set of SACK blocks > five, six or more times. This will be clarified in the > next revision. SACK usually has 4 too - it's just often deployed/enabled together with Tim= estamps that take away option space. A combined SACK+TS (SACK+token) option= could yield 4 SACK block plus the two Tokens / Timestamps. (2 + 2x(4) + nx= (4+4)). In 2018, there are no unused SACK blocks - whenever the receiver has discon= tinuous data, it makes it into a SACK block until all available SACK blocks= are used up... usually from most recently received to least recently recei= ved... And as Matt mentioned, a receiver is free to cycle periodically thro= ugh all the discontinuous blocks except for the first SACK block (it's just= not implemented today;=20 The resiliency achieved by the redundant sending of SACK blocks is viewed t= o be sufficient. If you have data to the contrary, please share. Richard Scheffenegger From: tcpm-bounces@ietf.org [mailto:tcpm-bounces@ietf.org] On Behalf Of Ant= hony Sabatini Sent: Freitag, 20. Juli 2012 20:16 To: mattmathis@google.com Cc: tcpm Subject: Re: [tcpm] TCP Error recovery and efficiency Matt - =20 First off I would like to thank you for your excellent work on RFC2018, the= purpose of this update is not to diminish your work but rather extend and = expand on it. Without your lead I would never have had the impetus to exte= nd my work from HDLC into the TCP domain. Please understand that the first= draft was a trial draft to toss around and get essential comments. Also p= lease understand that it was originally written as part of a thesis present= ation on adapting token, state driven recovery to TCP, not as an RFC per se= . =20 1) A problem with RFC2018 is that it provides no real information with resp= ect to the receivers reassembly queue at the time the transmitter receives = the acknowledgement. The protocol presupposes conditions often not present= in the real world, that the information transfer is near instantaneous and= that a rather low number of messages can be in-flight i.e. 7 SACK blocks t= otal combined in both directions (i.e. messages for which SACK blocks have = not yet been received). The RFC2018 DOES NOT send information about messag= es that were lost (nor can it with the implied requirement of packet fragme= ntation or out of order delivery), it can only say what it has seen. =20 2) On an active link SACK goes into the weeds whenever noise bursts affecti= ng receipt of SACK blocks exceed approximately 100,000 bit times, a 10 ms n= oise burst at 10 mbps or 1 ms at 100 mbps causing a rollover of the active = three SACK block window. At that point SACK goes into timer based recovery= . The problem with time based recovery is that timers must always be set to= the maximum possible delay, destroying efficiency. The purpose of this ch= ange is to take recovery from timer based to state based so that it is maxi= mally responsive. =20 3) SACK presupposes a relatively stable link with a relatively low random p= acket drop. Neither are true anymore and it is why you need only to look a= t RFC2883 to realize that SACK sends a lot of duplicate segments destroying= efficiency. I am simply trying to reduce that number. =20 4) SACK has an implicit "last character problem"; normally you would get mu= ltiple confirmation SACK blocks as additional messages arrive equal to the = number of SACK blocks. This is not true if the link goes idle where the ne= xt to last only has two acknowledgements and the last only has one. If the= last transmit segment is dropped and the link goes idle or the last ACK pa= cket from the receiver is lost you are now into a condition where it takes = a major link timeout to recover it. The changes in the revised draft reduc= e the pssibility of that occurrence. =20 5) I admit the current draft does not properly express my views. In the cu= rrent RFC2018 we normally have three SACK blocks, in this version normally = four. What I was basically saying is that UNUSED SACK blocks should be fil= led with older, still not completed SACK blocks rather than transmit the sa= me set of SACK blocks five, six or more times. This will be clarified in t= he next revision. =20 6) Note that the argument has moved to whether we need to allow more that 1= 6,000 messages to be in process during any particular second in the face of= 10 Gbps links, proposal is on the table to expand that to 4,000,000. Also= there is a proposal on SACK block compression in process (RFC1072 was not = wrong in concept only in implementation) so that we can include more segmen= ts. =20 I do appreciate your comments and feedback. From ycheng@google.com Fri Jul 20 14:42:29 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 183F911E8086 for ; Fri, 20 Jul 2012 14:42:29 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.927 X-Spam-Level: X-Spam-Status: No, score=-102.927 tagged_above=-999 required=5 tests=[AWL=0.050, BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id F3UKH61DraZq for ; Fri, 20 Jul 2012 14:42:28 -0700 (PDT) Received: from mail-ob0-f172.google.com (mail-ob0-f172.google.com [209.85.214.172]) by ietfa.amsl.com (Postfix) with ESMTP id CCA8021F84F3 for ; Fri, 20 Jul 2012 14:42:25 -0700 (PDT) Received: by obbwc20 with SMTP id wc20so6322102obb.31 for ; Fri, 20 Jul 2012 14:43:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type:content-transfer-encoding:x-system-of-record; bh=lWGJsxXarwomNqa0quVuxxTAhX8OQFCOlygfeVH6L3o=; b=LxWJ34V+K4yQt7EGFDav/foyvdz9f+PPvnuiJaug8GMwYDlLcrbP1JPzoxYIspLabL Y8kfVbeNzP2w7tqVTHRIBkMOGYTbEVA9Vx+K3oyTP92RW0GTC0vpF1Z028jbQPN8Q5no QuGXtqc6w7/+Wh8kBVSthHu7/+gdD7rYz9AQAavKmlMDz4iHulhAGrwJerX+yRMx720q tAks9gfspNCffYFbL4Z7bYjZzMvNcQiNatXN36eL05j7FwjcCBwSzzt6QPhdqk6argCu wxi7acC208p6N2Ca+blpeqN1GHP+7W06uzElfWUVu3Zo7bsfLWsUOLKruVk/YvwPyXbE xbdg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type:content-transfer-encoding:x-system-of-record :x-gm-message-state; bh=lWGJsxXarwomNqa0quVuxxTAhX8OQFCOlygfeVH6L3o=; b=BMkfJP4bdTLgtnd1wPSH7fnOOwuv0terZghmoHMy7GZYUfelEIRgCkM7QfvNeLjBLR jQpJHQoBKdBxrLpWWC7chCtVkrLsLXXayW8jK+1bZSpPoiifUfwBdwsWbiq8mB0yTVAF ikDIVCGSidTDosIGAdZIUz07NghGfm6cnz++jkKcV6w0lIbFPFM3CFEWb1wPIWIVwJnm xIR50AI7u4sS+0AmuM6hwSePn9ZW9d2Zg7WDfhHkYNa30vKX+5Av6/6ifC713flyJtFV ptS52qzKQL4SRaMwpiX7wEALY3zdnEzdW4yItZVWmjOV+GSbjwHTdkMqkACRy4jaV/zJ xCOw== Received: by 10.182.1.72 with SMTP id 8mr9021533obk.61.1342820602517; Fri, 20 Jul 2012 14:43:22 -0700 (PDT) Received: by 10.182.1.72 with SMTP id 8mr9021518obk.61.1342820602258; Fri, 20 Jul 2012 14:43:22 -0700 (PDT) MIME-Version: 1.0 Received: by 10.60.14.168 with HTTP; Fri, 20 Jul 2012 14:43:02 -0700 (PDT) In-Reply-To: <5009786B.80707@isi.edu> References: <50049F5A.7000709@isi.edu> <50082223.9010701@isi.edu> <500845E0.8050601@isi.edu> <5009786B.80707@isi.edu> From: Yuchung Cheng Date: Fri, 20 Jul 2012 14:43:02 -0700 Message-ID: To: Joe Touch Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable X-System-Of-Record: true X-Gm-Message-State: ALoCoQmlR+RzocTgZPifrd5q0a8mGpBp6RUli3Z7iMyZV2YDL4XFo9ecOq52hhE7+KiXRrGS/i6IjpZht27Gcj33u22Ux/I6ZoCj0JiynXykggdhcTRXuWR1PoHMwNuLwT3lnjl6EvZIzJz8qgkACSIUoeBflCGF6wEoBMLTbOZlj95vaPjK6KU= Cc: "iccrg@cs.ucl.ac.uk" , "tcpm@ietf.org Extensions" , "Arjuna Sathiaseelan \(work\)" Subject: Re: [tcpm] Review of draft-fairhurst-tcpm-newcwv-03 X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jul 2012 21:42:29 -0000 On Fri, Jul 20, 2012 at 8:25 AM, Joe Touch wrote: > > > On 7/20/2012 7:37 AM, Yuchung Cheng wrote: > ... > >>> The server receives the request, and then sends a response. The server >>> has a >>> response that's typically larger - thus the burst. The server decides >>> "how >>> long has it been since I *received* anything? Since that time is very >>> short, >> >> >> but RFC 2861 does not use the most recently received timing? not in thei= r >> main algorithm (section 3.2) at least. > > > I'm referring to the algorithm as per Standard TCP [RFC5681] which, rese= ts > CWND to the restart window after the app goes idle (as described in this > doc). However, RFC 2681 failed to note that the restart algorithm was > widely deployed at the time, and was based on erroneous logic in a footno= te > of an unpublished extension of Van Jacobson's 1988 congestion control pap= er > (ax explained in detail in draft-hughes-restart-00.txt). > > >>> there's no slow-start restart, and the burst goes out. >>> >>> The only time the server would timeout and do slowstart restart would b= e >>> if >>> it decides to send something a while after anything has been received - >>> that >>> might occur for auto-refresh pages, but not much else. >>> >>> The issue is explained in detail here: J. Heidemann, K. Obraczka, J. >>> Touch, >>> =93Modeling the Performance of HTTP Over Several Transport Protocols,= =94 >>> IEEE/ACM Transactions on Networking, V5, N5, Oct. 1997, pp.616-630. >> >> >> But why does receiving something recently justify the sending cwnd is sa= fe >> to >> not cause burst-induced losses? Sorry but I can't find the explanation >> in >> the paper. > > > It doesn't justify it - it was an incorrect conclusion. The goal is "rest= art > if you haven't SENT in a while", but Van's footnote said that most > implementations at that time had a 'time since last received' timer, but = not > a 'time since last sent', and that because TCP is symmetric you can use t= he > receive timer instead of the send one. That logic was false, and persiste= nt > HTTP turned out to have exactly that pattern. > > However, the mechanism proposed in this doc still allows most HTTP exchan= ges > to burst an entire window - because it would be very likely that users wi= ll > click on URLs within a received page within the 5 minute timeout. > > I.e., I don't think section 4 of this doc is appropriate (it still amount= s > to a send timer), and still favor the "burst-or-lose" style approach as > recommended in draft-hughes-restart-00.txt. I enjoy reading this draft. Very nice summary of all the mechanisms propose= d! I also like burst-or-lose due to its simplicity (and is interested in rate-pacing as well). In burst-or-lose, do the acks of the N packets increment the cwnd if FlightSize < cwnd? I'd recommend newcwv draft to cite and discuss hugest-restart draft. > > Joe > > > From bob.briscoe@bt.com Sun Jul 22 20:02:58 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2A47721F8679 for ; Sun, 22 Jul 2012 20:02:58 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.901 X-Spam-Level: X-Spam-Status: No, score=-2.901 tagged_above=-999 required=5 tests=[AWL=-0.698, BAYES_00=-2.599, MIME_QP_LONG_LINE=1.396, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id w4XiT8v+rngc for ; Sun, 22 Jul 2012 20:02:53 -0700 (PDT) Received: from hubrelay-rd.bt.com (hubrelay-rd.bt.com [62.239.224.99]) by ietfa.amsl.com (Postfix) with ESMTP id 07FFA21F8668 for ; Sun, 22 Jul 2012 20:02:46 -0700 (PDT) Received: from EVMHR01-UKBR.domain1.systemhost.net (193.113.108.40) by EVMHR68-UKRD.bt.com (10.187.101.23) with Microsoft SMTP Server (TLS) id 8.3.264.0; Mon, 23 Jul 2012 04:02:44 +0100 Received: from rdw02134app71.domain1.systemhost.net (193.113.234.138) by EVMHR01-UKBR.domain1.systemhost.net (193.113.108.40) with Microsoft SMTP Server (TLS) id 8.3.264.0; Mon, 23 Jul 2012 04:02:44 +0100 Received: from cbibipnt08.iuser.iroot.adidom.com (147.149.100.81) by rdw02134app71.domain1.systemhost.net (10.36.6.87) with Microsoft SMTP Server id 14.2.309.2; Mon, 23 Jul 2012 04:02:38 +0100 Received: From bagheera.jungle.bt.co.uk ([132.146.168.158]) by cbibipnt08.iuser.iroot.adidom.com (WebShield SMTP v4.5 MR1a P0803.399); id 1343012554131; Mon, 23 Jul 2012 04:02:34 +0100 Received: from MUT.jungle.bt.co.uk ([10.73.24.18]) by bagheera.jungle.bt.co.uk (8.13.5/8.12.8) with ESMTP id q6N32Whj000504; Mon, 23 Jul 2012 04:02:32 +0100 Message-ID: <201207230302.q6N32Whj000504@bagheera.jungle.bt.co.uk> X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Mon, 23 Jul 2012 04:01:24 +0100 To: Mirja Kuehlewind , "Richard Scheffenegger" From: Bob Briscoe In-Reply-To: <201207161454.45011.mirja.kuehlewind@ikr.uni-stuttgart.de> References: <201207161454.45011.mirja.kuehlewind@ikr.uni-stuttgart.de> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1"; format=flowed Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.56 on 132.146.168.158 Cc: tcpm@ietf.org Subject: [tcpm] Review of draft-kuehlewind-tcpm-accurate-ecn-01.txt X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jul 2012 03:02:58 -0000 Mirja, Richard, I have the following top level comments about this draft. (I'll send another posting with all the nits later.) I believe the problem this draft addresses ought=20 to be adopted as TCPM WG business. I agree it should be Experimental. I am particularly concerned to ensure that DCTCP=20 uses a standardised feedback scheme, even if the=20 DCTCP algorithm as a whole isn't brought to the=20 IETF. Otherwise, with DCTCP already in Win8=20 Server beta, we will start to see a new=20 proprietary variant of the TCP wire protocol,=20 where the DCTCP receiver won't interwork properly=20 with senders that do not use the same proprietary=20 DCTCP capability negotiation and the same=20 proprietary extension to TCP feedback (that=20 wouldn't pass standardisation, because it isn't robust to ACK losses). However, the draft needs some work to make it=20 clear what the spec is asking for. The mechanism=20 is fairly clear, but not the roadmap in relation to other RFCs. 1/ Needs to be clearer what an implementer must do. At the end of "1.1 Use Cases" it says it is best=20 to have one generic feedback mechanism, whether=20 it is used or not. I agree that feedback about=20 every congestion signal (rather than just one per=20 RTT) is certainly more generic, because a source=20 can pick out one per RTT if it needs to (as in=20 one of the example use-cases you give). However, this section could confuse implementers:=20 it needs to say that, in order to be compliant=20 with this spec, a receiver will still need to=20 implement classic ECN feedback as well. In the capability negotiation section, it says a=20 receiver might be negotiating with any of four=20 different types of ECN sender (altho it doesn't=20 use the 'classic' terminology introduced at the=20 start for any of them), and the result could be=20 one of four modes. So it needs to says which of=20 these modes a compliant receiver must implement. It also needs to explain what mode means: is the=20 receiver in this mode? or the connection? or the half-connection? 2/ Relation to RFC3168? You're certainly correct that this doesn't=20 obsolete RFC3168. But what does it do to RFC3168?=20 At the end of "1.4. Design Choices" it says that=20 this proposed scheme will only be used instead of=20 classic ECN feedback [RFC3168], not in parallel.=20 I think that implies that this draft is intended=20 to update RFC3168 (say this in the header block=20 and in the abstract & introduction). It is best=20 to avoid a design fork (this draft is an=20 evolution of 3168, not an alternative, but it can=20 interwork with an RFC3168 TCP sender if necessary). You also ought to make it clear in the=20 introduction that this scheme manages to give=20 much more feedback information without using any=20 new bits on the wire relative to the classic scheme. 3a/ Single Repeat Unnecessary? You propose that the receiver must repeat each=20 count of CE that it feeds back exactly once, even=20 if its internal count has already increased again=20 by the next ACK. I don't understand why you need=20 this, and it introduces considerable complexity. I think you are assuming that this will improve=20 resilience against lost pure ACKs. But re-stating=20 a counter in every packet is already resilient to=20 loss. For instance if the receiver's internal=20 count of CE marks is 3 then 4, when it feeds back=20 3 then 4, if the ACK with 3 is lost, the 4=20 recovers the lost information without having to=20 repeat the 3. IOW, the ack count is cumulative. This criticism also applies to the repetition of=20 the other (nonce) counter in Appendix B. Where we do have a problem is in detecting wrap=20 (every 5 increments in your proposal). However,=20 repetition doesn't help with that. For instance,=20 if the receiver feeds back 3 twice, the second 3=20 could represent 5 increments on from the first=20 (or zero increments, or any multiple of 5),=20 because a string of pure ACKS may have been lost=20 between the two ACKs received. Similarly, 4 could=20 represent an increment of 5i+1, where i is an integer. 3b/ Better solution?: I proposed (in the re-ECN protcol, which was very=20 similar to this proposal - thanks for the=20 acknowledgement of that BTW) a way for the sender=20 to calculate whether wrap could have happened,=20 and to calculate the worst-case count of CE marks=20 that the receiver might have fed back, given=20 worst-case losses of pure ACKs (assuming regular sized data packets). The sender calculates: n =3D how many new packets the latest ACK acknowledges. x =3D (AcE_latest - AcE_previous) % b, where AcE is the congestion ack field and b is its base (5 in your= proposal). Then the sender can calculate the worst case CE=20 marks signalled by the receiver as: D =3D n - (n-x)%b For the sender to get a good calculation of the=20 number of new packets acknowledged, SACK would be recommended. In all the following examples, the base of AcE field is b =3D 5 #1 example: Sender sees ACK of 1 new packet, and AcE rises=20 from 2 to 3. This can only mean feedback of 1 CE mark. In maths: n =3D 1 AcE_latest =3D 3 AcE_previous =3D 2 Then x =3D (3-2)%5 =3D 1 D =3D 1 - (1-1)%5 =3D 1 #2 example: Sender sees ACK of 7 new packets (some ACKs must=20 have got lost), and AcE rises from 2 to 3. This=20 could mean feedback of 1 or 6 CE marks. In maths: n =3D 7 AcE_latest =3D 3 AcE_previous =3D 2 Then x =3D (3-2)%5 =3D 1 D =3D 7 - (7-1)%5 =3D 6 # The formula=20 automatically takes the worst-case #3 example: Sender sees delayed ACK of 2 new packets, and AcE=20 wraps from 4 to 1 (base 5). This can only mean feedback of 2 CE mark. In= maths: n =3D 2 AcE_latest =3D 1 AcE_previous =3D 4 Then x =3D (1-4)%5 =3D 2 D =3D 2 - (2-2)%5 =3D 2 4/ ECT in the IP header of the SYN-ACK You need to say whether this is optional or=20 mandatory for compliance with the accurate-ecn spec. 5/ ECT in the IP header of the SYN You say this is optional. I think a better way to=20 say this is that the accurate-ecn spec does not=20 preclude turning on ECN support in a SYN, but=20 RFC3168 advises against it. If some future scheme=20 uses ECN on the SYN, accurate-ecn will be able to=20 support it, but it can also support RFC3168 not using ECN on the SYN. 6/ ECN Nonce support needs to be mandatory (only for receiver feedback). I suggest the receiver MUST feedback a count of=20 ECT(1), but make it clear that this doesn't mean=20 the sender has to use the ECN nonce. In a=20 connection where the sender doesn't use the=20 nonce, the receiver's count will always be zero. Reasoning: The nonce is a security mechanism for=20 the sender to detect misbehaving receivers. If=20 it's optional for receivers to do the feedback,=20 then lots of receivers (good and bad) won't=20 support it. Then any misbehaving receivers will=20 just have to say they don't support it, and they=20 can avoid their misbehaviour being detected. 7/ Advanced Compatibility Mode I'm dubious about this part. It seems a nice=20 idea, but it is quite unsafe, in that a lot of=20 congestion indications can get lost without detection. 8/ I have an alternative proposal that you might like (see a following mail) 9/ And, as already promised, nits will follow=20 too. It needs a fair amount of word-smithing. HTH Bob At 13:54 16/07/2012, Mirja Kuehlewind wrote: >Hi everybody, > >we updated the two drafts describing more accurate ECN feedback: > >- draft-kuehlewind-tcpm-accurate-ecn-option-01.txt >specifies an additional TCP option > >- draft-kuehlewind-tcpm-accurate-ecn-01.txt >specifies the re-use of the ECN/NS TCP header bits > >Mirja > > >---------- Forwarded Message ---------- > >Subject: New Version Notification for >draft-kuehlewind-tcpm-accurate-ecn-option-01.txt >Date: Friday 13 July 2012, 16:25:43 >From: internet-drafts@ietf.org >To: mirja.kuehlewind@ikr.uni-stuttgart.de >CC: rs@netapp.com > > >A new version of I-D, draft-kuehlewind-tcpm-accurate-ecn-option-01.txt >has been successfully submitted by Mirja Kuehlewind and posted to the >IETF repository. > >Filename: draft-kuehlewind-tcpm-accurate-ecn-option >Revision: 01 >Title: Accurate ECN Feedback Option in TCP >Creation date: 2012-07-13 >WG ID: Individual Submission >Number of pages: 7 >URL: >http://www.ietf.org/internet-drafts/draft-kuehlewind-tcpm-accurate-ecn-opti= on-01.txt >Status: >http://datatracker.ietf.org/doc/draft-kuehlewind-tcpm-accurate-ecn-option >Htmlized: >http://tools.ietf.org/html/draft-kuehlewind-tcpm-accurate-ecn-option-01 >Diff: >http://tools.ietf.org/rfcdiff?url2=3Ddraft-kuehlewind-tcpm-accurate-ecn-opt= ion-01 > >Abstract: > This document specifies an TCP option to get accurate Explicit > Congestion Notification (ECN) feedback from the receiver. ECN is an > IP/TCP mechanism where network nodes can mark IP packets instead of > dropping them to indicate congestion to the end-points. An ECN- > capable receiver will feedback this information to the sender. ECN > is specified for TCP in such a way that only one feedback signal can > be transmitted per Round-Trip Time (RTT). Recently new TCP > mechanisms like ConEx or DCTCP need more accurate feedback > information in the case where more than one marking is received in > one RTT. This TCP extension can be used in addition to the classic > ECN as well as with a more accurate ECN scheme recently proposed > which reuses the ECN bit in the TCP header. > >=20 > > > >The IETF Secretariat > >------------------------------------------------------- > >---------- Forwarded Message ---------- > >Subject: New Version Notification for >draft-kuehlewind-tcpm-accurate-ecn-01.txt >Date: Monday 16 July 2012, 14:50:02 >From: internet-drafts@ietf.org >To: mirja.kuehlewind@ikr.uni-stuttgart.de >CC: rs@netapp.com > > >A new version of I-D, draft-kuehlewind-tcpm-accurate-ecn-01.txt >has been successfully submitted by Mirja Kuehlewind and posted to the >IETF repository. > >Filename: draft-kuehlewind-tcpm-accurate-ecn >Revision: 01 >Title: More Accurate ECN Feedback in TCP >Creation date: 2012-07-16 >WG ID: Individual Submission >Number of pages: 19 >URL: >http://www.ietf.org/internet-drafts/draft-kuehlewind-tcpm-accurate-ecn-01.t= xt >Status: >http://datatracker.ietf.org/doc/draft-kuehlewind-tcpm-accurate-ecn >Htmlized: >http://tools.ietf.org/html/draft-kuehlewind-tcpm-accurate-ecn-01 >Diff: >http://tools.ietf.org/rfcdiff?url2=3Ddraft-kuehlewind-tcpm-accurate-ecn-01 > >Abstract: > Explicit Congestion Notification (ECN) is an IP/TCP mechanism where > network nodes can mark IP packets instead of dropping them to > indicate congestion to the end-points. An ECN-capable receiver will > feedback this information to the sender. ECN is specified for TCP in > such a way that only one feedback signal can be transmitted per > Round-Trip Time (RTT). Recently, new TCP mechanisms like ConEx or > DCTCP need more accurate ECN feedback information in the case where > more than one marking is received in one RTT. This documents > specifies a different scheme for the ECN feedback in the TCP header > to provide more than one feedback signal per RTT. > >=20 > > > >The IETF Secretariat > >------------------------------------------------------- > >-- >------------------------------------------------------------------- >Dipl.-Ing. Mirja K=FChlewind >Institute of Communication Networks and Computer Engineering (IKR) >University of Stuttgart, Germany >Pfaffenwaldring 47, D-70569 Stuttgart > >tel: +49(0)711/685-67973 >email: mirja.kuehlewind@ikr.uni-stuttgart.de >web: www.ikr.uni-stuttgart.de >------------------------------------------------------------------- >_______________________________________________ >tcpm mailing list >tcpm@ietf.org >https://www.ietf.org/mailman/listinfo/tcpm ________________________________________________________________ Bob Briscoe, BT Innovate & Design=20 From michael.scharf@alcatel-lucent.com Mon Jul 23 10:23:42 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4480521F85F6 for ; Mon, 23 Jul 2012 10:23:42 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.049 X-Spam-Level: X-Spam-Status: No, score=-10.049 tagged_above=-999 required=5 tests=[AWL=0.200, BAYES_00=-2.599, HELO_EQ_FR=0.35, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id PclJTUyhvJ8Z for ; Mon, 23 Jul 2012 10:23:39 -0700 (PDT) Received: from smail3.alcatel.fr (smail3.alcatel.fr [64.208.49.56]) by ietfa.amsl.com (Postfix) with ESMTP id C1F8521F85EF for ; Mon, 23 Jul 2012 10:23:38 -0700 (PDT) Received: from FRMRSSXCHHUB01.dc-m.alcatel-lucent.com (FRMRSSXCHHUB01.dc-m.alcatel-lucent.com [135.120.45.61]) by smail3.alcatel.fr (8.14.3/8.14.3/ICT) with ESMTP id q6NHNava026147 (version=TLSv1/SSLv3 cipher=RC4-MD5 bits=128 verify=NOT); Mon, 23 Jul 2012 19:23:37 +0200 Received: from FRMRSSXCHMBSE3.dc-m.alcatel-lucent.com ([135.120.45.55]) by FRMRSSXCHHUB01.dc-m.alcatel-lucent.com ([135.120.45.61]) with mapi; Mon, 23 Jul 2012 19:23:36 +0200 From: "Scharf, Michael (Michael)" To: "Scheffenegger, Richard" Date: Mon, 23 Jul 2012 19:23:35 +0200 Thread-Topic: Questions on draft-scheffenegger-tcpm-timestamp-negotiation-04 Thread-Index: Ac1jMEoFBSSlToDnQNWbGMwYRyuCmQFpVItQ Message-ID: <2A886F9088894347A3BE0CC5B7A85F3E891A9BBC24@FRMRSSXCHMBSE3.dc-m.alcatel-lucent.com> References: <012C3117EDDB3C4781FD802A8C27DD4F0D4A88D2@SACEXCMBX02-PRD.hq.netapp.com> In-Reply-To: <012C3117EDDB3C4781FD802A8C27DD4F0D4A88D2@SACEXCMBX02-PRD.hq.netapp.com> Accept-Language: de-DE, en-US Content-Language: de-DE X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: de-DE, en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.69 on 155.132.188.83 Cc: "tcpm@ietf.org" Subject: [tcpm] Questions on draft-scheffenegger-tcpm-timestamp-negotiation-04 X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jul 2012 17:23:42 -0000 Hi all, In order to trigger some discussion on that draft, here are some questions = on draft-scheffenegger-tcpm-timestamp-negotiation-04 (as naive and memoryle= ss individual): * Very, very globally: We apparently have 4 spare bytes in the SYN option i= f TS are used. This draft suggests a really intelligent way of using those = bits. However, if we touch TCP timestamp semantics, wouldn't it be a better= , though more disruptive approach to replace the TS option by a new option = with less bytes at least in the initial in the SYN? 4 bytes of 40 is 10%...= I also recall some interest in late enablement of timestamps earlier this = year; this draft seems to suggest quite the opposite... * Is there really a need to announce the clock speed in the SYN? We need th= e SYN for atomic negotiation of features. I could imagine that for the cloc= k speed a post-SYN option could be used as well. Thoughts? * I also wonder whether there is some data on how much better explicit sign= aling of the clock speed is than learning from heuristics (sorry if I shoul= d have missed that data). * Some thought on security considerations: Can a host mess up a delay measu= rement by announcing a wrong clock frequency? What would be the implication= s? * The masking of the timestamps seems to be motivated by a heuristic that i= s deployed in some stacks, but AFAIK not described in an RFC. Maybe it woul= d make sense first to have some draft that specifies that behavior? * The draft is a "one-in-a-box" proposal for one-way delay measurement and = improved SACK processing. I wonder whether it could make sense to allow sep= arate enablement of those features. For instance, would it make have some b= enefit if a host just announces its clock frequency but leaves the TS echoi= ng semantics unchanged? * Given that the draft changes the TS echoing semantics, it has to update t= he RTTM to get some additional offset for the delayed ACKs. Currently, the = draft suggests: "R =3D RTT * ( 1 + 1/(cwnd/smss) )". Is my understanding co= rrect that this formula assumes that every second segment is acked? Accordi= ng to RFC 1122, acking every second segment is only a SHOULD. The MUST just= mandates that the additional delay is less than 0.5 seconds. Thus, is it p= ossible that "R =3D RTT + 0.5s" is needed in some scenarios? Or is there ev= en a need to signal the delACK strategy to the remote end, so that it can a= dapt its RTTM/RTP calculation? This could be challenging... Sorry if I should have missed something. Thanks Michael =20 > -----Original Message----- > From: tcpm-bounces@ietf.org [mailto:tcpm-bounces@ietf.org] On=20 > Behalf Of Scheffenegger, Richard > Sent: Monday, July 16, 2012 10:53 AM > To: tcpm (tcpm@ietf.org) > Subject: [tcpm] FW: New Version Notification for=20 > draft-scheffenegger-tcpm-timestamp-negotiation-04.txt >=20 > -----Original Message----- > From: internet-drafts@ietf.org [mailto:internet-drafts@ietf.org] > Sent: Montag, 16. Juli 2012 10:43 > To: Scheffenegger, Richard > Cc: mirja.kuehlewind@ikr.uni-stuttgart.de > Subject: New Version Notification for=20 > draft-scheffenegger-tcpm-timestamp-negotiation-04.txt >=20 >=20 > A new version of I-D,=20 > draft-scheffenegger-tcpm-timestamp-negotiation-04.txt > has been successfully submitted by Richard Scheffenegger and=20 > posted to the > IETF repository. >=20 > Filename: draft-scheffenegger-tcpm-timestamp-negotiation > Revision: 04 > Title: Additional negotiation in the TCP=20 > Timestamp Option field during the TCP handshake > Creation date: 2012-07-16 > WG ID: Individual Submission > Number of pages: 34 > URL: =20 > http://www.ietf.org/internet-drafts/draft-scheffenegger-tcpm-t > imestamp-negotiation-04.txt > Status: =20 > http://datatracker.ietf.org/doc/draft-scheffenegger-tcpm-times > tamp-negotiation > Htmlized: =20 > http://tools.ietf.org/html/draft-scheffenegger-tcpm-timestamp- > negotiation-04 > Diff: =20 > http://tools.ietf.org/rfcdiff?url2=3Ddraft-scheffenegger-tcpm-ti > mestamp-negotiation-04 >=20 > Abstract: > A number of TCP enhancements in so diverse fields as congestion > control, loss recovery or side-band signaling could be improved by > allowing both ends of a TCP session to interpret the value=20 > carried in > the Timestamp option. Further enhancements are enabled by changing > the receiver side processing of timestamps in the presence of > Selective Acknowledgements. >=20 > This documents updates RFC1323 and specifies a backwards compatible > way of negotiating for Timestamp capabilities, and lists a=20 > number of > benefits and drawbacks of this approach. >=20 > =20 > =20 >=20 >=20 > The IETF Secretariat > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm > = From michael.scharf@alcatel-lucent.com Mon Jul 23 12:10:42 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 81D3311E80CF for ; Mon, 23 Jul 2012 12:10:42 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.067 X-Spam-Level: X-Spam-Status: No, score=-10.067 tagged_above=-999 required=5 tests=[AWL=0.182, BAYES_00=-2.599, HELO_EQ_FR=0.35, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tOk4OnyVUZGg for ; Mon, 23 Jul 2012 12:10:42 -0700 (PDT) Received: from smail6.alcatel.fr (smail6.alcatel.fr [64.208.49.42]) by ietfa.amsl.com (Postfix) with ESMTP id B18E611E80AE for ; Mon, 23 Jul 2012 12:10:41 -0700 (PDT) Received: from FRMRSSXCHHUB03.dc-m.alcatel-lucent.com (FRMRSSXCHHUB03.dc-m.alcatel-lucent.com [135.120.45.63]) by smail6.alcatel.fr (8.14.3/8.14.3/ICT) with ESMTP id q6NJAduM023690 (version=TLSv1/SSLv3 cipher=RC4-MD5 bits=128 verify=NOT); Mon, 23 Jul 2012 21:10:39 +0200 Received: from FRMRSSXCHMBSE3.dc-m.alcatel-lucent.com ([135.120.45.55]) by FRMRSSXCHHUB03.dc-m.alcatel-lucent.com ([135.120.45.63]) with mapi; Mon, 23 Jul 2012 21:10:39 +0200 From: "Scharf, Michael (Michael)" To: "tcpm@ietf.org" Date: Mon, 23 Jul 2012 21:10:39 +0200 Thread-Topic: IETF 84 meeting: Slides please Thread-Index: Ac1pBtjEHEMXIkXlRQi+6Pt8XcBlCQ== Message-ID: <2A886F9088894347A3BE0CC5B7A85F3E891A9BBC27@FRMRSSXCHMBSE3.dc-m.alcatel-lucent.com> Accept-Language: de-DE, en-US Content-Language: de-DE X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: de-DE, en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.69 on 155.132.188.84 Cc: "tcpm-chairs@tools.ietf.org" Subject: [tcpm] IETF 84 meeting: Slides please X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jul 2012 19:10:42 -0000 Hi all, The current TCPM agenda is available at: https://datatracker.ietf.org/meeti= ng/84/agenda/tcpm/ The agenda is pretty full, i. e., there is a high risk that the last items = will have to be moved to the mailing list. Could the speakers please send the slides to the chairs? Since we are scheduled on Monday morning, the deadline for submitting the s= lides to the chairs is Sunday noon. PDF format would be preferred. Also, please put numbers on the slides for r= emote participants. Thanks Michael (TCPM co-chair) From rs@netapp.com Wed Jul 25 08:15:14 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D16BE21F85B4 for ; Wed, 25 Jul 2012 08:15:14 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -9.909 X-Spam-Level: X-Spam-Status: No, score=-9.909 tagged_above=-999 required=5 tests=[AWL=0.690, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hX1o-1Qb4+AM for ; Wed, 25 Jul 2012 08:15:12 -0700 (PDT) Received: from mx2.netapp.com (mx2.netapp.com [216.240.18.37]) by ietfa.amsl.com (Postfix) with ESMTP id 79B8F21F8582 for ; Wed, 25 Jul 2012 08:15:12 -0700 (PDT) X-IronPort-AV: E=Sophos;i="4.77,653,1336374000"; d="scan'208";a="668496342" Received: from smtp1.corp.netapp.com ([10.57.156.124]) by mx2-out.netapp.com with ESMTP; 25 Jul 2012 08:15:11 -0700 Received: from vmwexceht01-prd.hq.netapp.com (vmwexceht01-prd.hq.netapp.com [10.106.76.239]) by smtp1.corp.netapp.com (8.13.1/8.13.1/NTAP-1.6) with ESMTP id q6PFFAwI023224; Wed, 25 Jul 2012 08:15:11 -0700 (PDT) Received: from SACEXCMBX02-PRD.hq.netapp.com ([169.254.1.34]) by vmwexceht01-prd.hq.netapp.com ([10.106.76.239]) with mapi id 14.02.0298.004; Wed, 25 Jul 2012 08:15:10 -0700 From: "Scheffenegger, Richard" To: Bob Briscoe , Mirja Kuehlewind Thread-Topic: [tcpm] Review of draft-kuehlewind-tcpm-accurate-ecn-01.txt Thread-Index: AQHNaH+uJyuGoLRTMk2MyxwT10S8iZc6Fvvw Date: Wed, 25 Jul 2012 15:15:08 +0000 Message-ID: <012C3117EDDB3C4781FD802A8C27DD4F0D4C98D0@SACEXCMBX02-PRD.hq.netapp.com> References: <201207161454.45011.mirja.kuehlewind@ikr.uni-stuttgart.de> <201207230302.q6N32Whj000504@bagheera.jungle.bt.co.uk> In-Reply-To: <201207230302.q6N32Whj000504@bagheera.jungle.bt.co.uk> Accept-Language: de-AT, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.104.60.115] Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "tcpm@ietf.org" Subject: Re: [tcpm] Review of draft-kuehlewind-tcpm-accurate-ecn-01.txt X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Jul 2012 15:15:15 -0000 Hi Bob, thank you for your valuable feedback! > I am particularly concerned to ensure that DCTCP uses a standardised > feedback scheme, even if the DCTCP algorithm as a whole isn't brought to > the IETF. Otherwise, with DCTCP already in Win8 Server beta, we will star= t > to see a new proprietary variant of the TCP wire protocol, where the DCTC= P > receiver won't interwork properly with senders that do not use the same > proprietary DCTCP capability negotiation and the same proprietary > extension to TCP feedback (that wouldn't pass standardisation, because it > isn't robust to ACK losses). Agreed. Since DCTCP isn't really disclosed, particular the handling of CWR = on the sender side, it's interaction with legacy RFC3168 receivers and send= ers can lead to "interesting" effects... Ad 2) Yes, the draft should say that it updates (but does not obsolete!) RF= C3168. Ad 3a/b) The latest draft has a pseudo-code example, which uses the heurist= ic you describe. Also, I need to go back to my simulation - the repetition = of each signaled value was intended to primarily address counter wrap when = multiple ACKs are lost. We did some simulation runs with burst losses, and = mean burst length=3D2, where the repetition scheme fared slightly better in= not overestimating the true CE marks, IIRC. However, some complexity remains, as a new ACK must be sent whenever the co= unter increases by (b-1). With b=3D3 (ECT1 counter signal), delayed ACK m m= ust not exceed 2 (when a ECT(1) is set on a number of consecutive Packets..= .). This is the main driver for having a counter (the last Base(b)-digit to= be signaled) and a Gauge. Without repeating each signaled codepoint twice,= there is more complexity on the sender side to detect counter wraps, as yo= u mention (average segment size, or knowing the exact sequence number bound= aries/number of packets between two sequence numbers, etc). Having a single= bit (per counter) to facilitate the repetition seemed rather small in comp= arison. I'll try to run my simulations with and without the repetition of the same = codepoint, and post the results to the WG. Ad 4/5) A reference to RFC5562 may be in order... Ad 6) Very reasonable argument; the idea was to allow some codepoints to re= main reserved for other future uses. But your point is absolutely valid, an= d a security mechanism must be mandatory. Best regards, Richard Scheffenegger > -----Original Message----- > From: tcpm-bounces@ietf.org [mailto:tcpm-bounces@ietf.org] On Behalf Of > Bob Briscoe > Sent: Montag, 23. Juli 2012 05:01 > To: Mirja Kuehlewind; Scheffenegger, Richard > Cc: tcpm@ietf.org > Subject: [tcpm] Review of draft-kuehlewind-tcpm-accurate-ecn-01.txt >=20 > Mirja, Richard, >=20 > I have the following top level comments about this draft. > (I'll send another posting with all the nits later.) >=20 > I believe the problem this draft addresses ought to be adopted as TCPM WG > business. > I agree it should be Experimental. >=20 > I am particularly concerned to ensure that DCTCP uses a standardised > feedback scheme, even if the DCTCP algorithm as a whole isn't brought to > the IETF. Otherwise, with DCTCP already in Win8 Server beta, we will star= t > to see a new proprietary variant of the TCP wire protocol, where the DCTC= P > receiver won't interwork properly with senders that do not use the same > proprietary DCTCP capability negotiation and the same proprietary > extension to TCP feedback (that wouldn't pass standardisation, because it > isn't robust to ACK losses). >=20 >=20 > However, the draft needs some work to make it clear what the spec is > asking for. The mechanism is fairly clear, but not the roadmap in relatio= n > to other RFCs. >=20 > 1/ Needs to be clearer what an implementer must do. >=20 > At the end of "1.1 Use Cases" it says it is best to have one generic > feedback mechanism, whether it is used or not. I agree that feedback abou= t > every congestion signal (rather than just one per > RTT) is certainly more generic, because a source can pick out one per RTT > if it needs to (as in one of the example use-cases you give). >=20 > However, this section could confuse implementers: > it needs to say that, in order to be compliant with this spec, a receiver > will still need to implement classic ECN feedback as well. >=20 > In the capability negotiation section, it says a receiver might be > negotiating with any of four different types of ECN sender (altho it > doesn't use the 'classic' terminology introduced at the start for any of > them), and the result could be one of four modes. So it needs to says > which of these modes a compliant receiver must implement. >=20 > It also needs to explain what mode means: is the receiver in this mode? o= r > the connection? or the half-connection? >=20 > 2/ Relation to RFC3168? >=20 > You're certainly correct that this doesn't obsolete RFC3168. But what doe= s > it do to RFC3168? > At the end of "1.4. Design Choices" it says that this proposed scheme wil= l > only be used instead of classic ECN feedback [RFC3168], not in parallel. > I think that implies that this draft is intended to update RFC3168 (say > this in the header block and in the abstract & introduction). It is best > to avoid a design fork (this draft is an evolution of 3168, not an > alternative, but it can interwork with an RFC3168 TCP sender if > necessary). >=20 > You also ought to make it clear in the > introduction that this scheme manages to give much more feedback > information without using any new bits on the wire relative to the classi= c > scheme. >=20 > 3a/ Single Repeat Unnecessary? >=20 > You propose that the receiver must repeat each count of CE that it feeds > back exactly once, even if its internal count has already increased again > by the next ACK. I don't understand why you need this, and it introduces > considerable complexity. >=20 > I think you are assuming that this will improve resilience against lost > pure ACKs. But re-stating a counter in every packet is already resilient > to loss. For instance if the receiver's internal count of CE marks is 3 > then 4, when it feeds back > 3 then 4, if the ACK with 3 is lost, the 4 recovers the lost information > without having to repeat the 3. IOW, the ack count is cumulative. >=20 > This criticism also applies to the repetition of the other (nonce) counte= r > in Appendix B. >=20 > Where we do have a problem is in detecting wrap (every 5 increments in > your proposal). However, repetition doesn't help with that. For instance, > if the receiver feeds back 3 twice, the second 3 could represent 5 > increments on from the first (or zero increments, or any multiple of 5), > because a string of pure ACKS may have been lost between the two ACKs > received. Similarly, 4 could represent an increment of 5i+1, where i is a= n > integer. >=20 > 3b/ Better solution?: > I proposed (in the re-ECN protcol, which was very similar to this proposa= l > - thanks for the acknowledgement of that BTW) a way for the sender to > calculate whether wrap could have happened, and to calculate the worst- > case count of CE marks that the receiver might have fed back, given worst= - > case losses of pure ACKs (assuming regular sized data packets). >=20 > The sender calculates: > n =3D how many new packets the latest ACK acknowledges. > x =3D (AcE_latest - AcE_previous) % b, where AcE is the congesti= on > ack field and b is its base (5 in your proposal). > Then the sender can calculate the worst case CE marks signalled by the > receiver as: > D =3D n - (n-x)%b >=20 > For the sender to get a good calculation of the number of new packets > acknowledged, SACK would be recommended. >=20 > In all the following examples, the base of AcE field is > b =3D 5 >=20 > #1 example: > Sender sees ACK of 1 new packet, and AcE rises from 2 to 3. This can only > mean feedback of 1 CE mark. In maths: > n =3D 1 > AcE_latest =3D 3 > AcE_previous =3D 2 > Then > x =3D (3-2)%5 =3D 1 > D =3D 1 - (1-1)%5 =3D 1 >=20 > #2 example: > Sender sees ACK of 7 new packets (some ACKs must have got lost), and AcE > rises from 2 to 3. This could mean feedback of 1 or 6 CE marks. In maths: > n =3D 7 > AcE_latest =3D 3 > AcE_previous =3D 2 > Then > x =3D (3-2)%5 =3D 1 > D =3D 7 - (7-1)%5 =3D 6 # The formula > automatically takes the worst-case >=20 > #3 example: > Sender sees delayed ACK of 2 new packets, and AcE wraps from 4 to 1 (base > 5). This can only mean feedback of 2 CE mark. In maths: > n =3D 2 > AcE_latest =3D 1 > AcE_previous =3D 4 > Then > x =3D (1-4)%5 =3D 2 > D =3D 2 - (2-2)%5 =3D 2 >=20 > 4/ ECT in the IP header of the SYN-ACK > You need to say whether this is optional or mandatory for compliance with > the accurate-ecn spec. >=20 > 5/ ECT in the IP header of the SYN > You say this is optional. I think a better way to say this is that the > accurate-ecn spec does not preclude turning on ECN support in a SYN, but > RFC3168 advises against it. If some future scheme uses ECN on the SYN, > accurate-ecn will be able to support it, but it can also support RFC3168 > not using ECN on the SYN. >=20 > 6/ ECN Nonce support needs to be mandatory (only for receiver feedback). > I suggest the receiver MUST feedback a count of ECT(1), but make it clear > that this doesn't mean the sender has to use the ECN nonce. In a > connection where the sender doesn't use the nonce, the receiver's count > will always be zero. >=20 > Reasoning: The nonce is a security mechanism for the sender to detect > misbehaving receivers. If it's optional for receivers to do the feedback, > then lots of receivers (good and bad) won't support it. Then any > misbehaving receivers will just have to say they don't support it, and > they can avoid their misbehaviour being detected. >=20 > 7/ Advanced Compatibility Mode > I'm dubious about this part. It seems a nice idea, but it is quite unsafe= , > in that a lot of congestion indications can get lost without detection. >=20 > 8/ I have an alternative proposal that you might like (see a following > mail) >=20 > 9/ And, as already promised, nits will follow too. It needs a fair amount > of word-smithing. >=20 > HTH >=20 >=20 > Bob >=20 >=20 >=20 > At 13:54 16/07/2012, Mirja Kuehlewind wrote: > >Hi everybody, > > > >we updated the two drafts describing more accurate ECN feedback: > > > >- draft-kuehlewind-tcpm-accurate-ecn-option-01.txt > >specifies an additional TCP option > > > >- draft-kuehlewind-tcpm-accurate-ecn-01.txt > >specifies the re-use of the ECN/NS TCP header bits > > > >Mirja > > > > > >---------- Forwarded Message ---------- > > > >Subject: New Version Notification for > >draft-kuehlewind-tcpm-accurate-ecn-option-01.txt > >Date: Friday 13 July 2012, 16:25:43 > >From: internet-drafts@ietf.org > >To: mirja.kuehlewind@ikr.uni-stuttgart.de > >CC: rs@netapp.com > > > > > >A new version of I-D, draft-kuehlewind-tcpm-accurate-ecn-option-01.txt > >has been successfully submitted by Mirja Kuehlewind and posted to the > >IETF repository. > > > >Filename: draft-kuehlewind-tcpm-accurate-ecn-option > >Revision: 01 > >Title: Accurate ECN Feedback Option in TCP > >Creation date: 2012-07-13 > >WG ID: Individual Submission > >Number of pages: 7 > >URL: > >http://www.ietf.org/internet-drafts/draft-kuehlewind-tcpm-accurate-ecn- > >option-01.txt > >Status: > >http://datatracker.ietf.org/doc/draft-kuehlewind-tcpm-accurate-ecn-opti > >on > >Htmlized: > >http://tools.ietf.org/html/draft-kuehlewind-tcpm-accurate-ecn-option-01 > >Diff: > >http://tools.ietf.org/rfcdiff?url2=3Ddraft-kuehlewind-tcpm-accurate-ecn-= o > >ption-01 > > > >Abstract: > > This document specifies an TCP option to get accurate Explicit > > Congestion Notification (ECN) feedback from the receiver. ECN is an > > IP/TCP mechanism where network nodes can mark IP packets instead of > > dropping them to indicate congestion to the end-points. An ECN- > > capable receiver will feedback this information to the sender. ECN > > is specified for TCP in such a way that only one feedback signal can > > be transmitted per Round-Trip Time (RTT). Recently new TCP > > mechanisms like ConEx or DCTCP need more accurate feedback > > information in the case where more than one marking is received in > > one RTT. This TCP extension can be used in addition to the classic > > ECN as well as with a more accurate ECN scheme recently proposed > > which reuses the ECN bit in the TCP header. > > > > > > > > > > > >The IETF Secretariat > > > >------------------------------------------------------- > > > >---------- Forwarded Message ---------- > > > >Subject: New Version Notification for > >draft-kuehlewind-tcpm-accurate-ecn-01.txt > >Date: Monday 16 July 2012, 14:50:02 > >From: internet-drafts@ietf.org > >To: mirja.kuehlewind@ikr.uni-stuttgart.de > >CC: rs@netapp.com > > > > > >A new version of I-D, draft-kuehlewind-tcpm-accurate-ecn-01.txt > >has been successfully submitted by Mirja Kuehlewind and posted to the > >IETF repository. > > > >Filename: draft-kuehlewind-tcpm-accurate-ecn > >Revision: 01 > >Title: More Accurate ECN Feedback in TCP > >Creation date: 2012-07-16 > >WG ID: Individual Submission > >Number of pages: 19 > >URL: > >http://www.ietf.org/internet-drafts/draft-kuehlewind-tcpm-accurate-ecn- > >01.txt > >Status: > >http://datatracker.ietf.org/doc/draft-kuehlewind-tcpm-accurate-ecn > >Htmlized: > >http://tools.ietf.org/html/draft-kuehlewind-tcpm-accurate-ecn-01 > >Diff: > >http://tools.ietf.org/rfcdiff?url2=3Ddraft-kuehlewind-tcpm-accurate-ecn-= 0 > >1 > > > >Abstract: > > Explicit Congestion Notification (ECN) is an IP/TCP mechanism where > > network nodes can mark IP packets instead of dropping them to > > indicate congestion to the end-points. An ECN-capable receiver will > > feedback this information to the sender. ECN is specified for TCP i= n > > such a way that only one feedback signal can be transmitted per > > Round-Trip Time (RTT). Recently, new TCP mechanisms like ConEx or > > DCTCP need more accurate ECN feedback information in the case where > > more than one marking is received in one RTT. This documents > > specifies a different scheme for the ECN feedback in the TCP header > > to provide more than one feedback signal per RTT. > > > > > > > > > > > >The IETF Secretariat > > > >------------------------------------------------------- > > > >-- > >------------------------------------------------------------------- > >Dipl.-Ing. Mirja K=FChlewind > >Institute of Communication Networks and Computer Engineering (IKR) > >University of Stuttgart, Germany Pfaffenwaldring 47, D-70569 Stuttgart > > > >tel: +49(0)711/685-67973 > >email: mirja.kuehlewind@ikr.uni-stuttgart.de > >web: www.ikr.uni-stuttgart.de > >------------------------------------------------------------------- > >_______________________________________________ > >tcpm mailing list > >tcpm@ietf.org > >https://www.ietf.org/mailman/listinfo/tcpm >=20 > ________________________________________________________________ > Bob Briscoe, BT Innovate & Design >=20 > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm From mirja.kuehlewind@ikr.uni-stuttgart.de Wed Jul 25 09:11:47 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6A3F921F86DC for ; Wed, 25 Jul 2012 09:11:47 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.037 X-Spam-Level: X-Spam-Status: No, score=-1.037 tagged_above=-999 required=5 tests=[AWL=0.912, BAYES_00=-2.599, HELO_EQ_DE=0.35, MIME_8BIT_HEADER=0.3] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7LhdLsxavqD8 for ; Wed, 25 Jul 2012 09:11:43 -0700 (PDT) Received: from mailsrv.ikr.uni-stuttgart.de (mailsrv.ikr.uni-stuttgart.de [129.69.170.2]) by ietfa.amsl.com (Postfix) with ESMTP id 291DC21F86DB for ; Wed, 25 Jul 2012 09:11:43 -0700 (PDT) Received: from netsrv1.ikr.uni-stuttgart.de (netsrv1-c [10.11.12.12]) by mailsrv.ikr.uni-stuttgart.de (Postfix) with ESMTP id 7378D633B9; Wed, 25 Jul 2012 18:11:41 +0200 (CEST) Received: from vpn-2-cl177 (vpn-2-cl177 [10.41.21.177]) by netsrv1.ikr.uni-stuttgart.de (Postfix) with ESMTP id 6437F59A8A; Wed, 25 Jul 2012 18:11:41 +0200 (CEST) From: Mirja =?iso-8859-1?q?K=FChlewind?= Date: Wed, 25 Jul 2012 18:11:40 +0200 User-Agent: KMail/1.9.10 (enterprise35 0.20101217.1207316) MIME-Version: 1.0 Content-Disposition: inline To: tcpm@ietf.org, Jerry Chu Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201207251811.40438.mkuehle@ikr.uni-stuttgart.de> Subject: [tcpm] Review of draft-ietf-tcpm-initcwnd-04 X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Jul 2012 16:11:47 -0000 Hi Jerry, I'd say the draft now covers all the points discussed in tcpm and as such has all the content needed. As you actually only propose a small change to TCP, I'd prefer to have a quite short document describing mainly only the changes itself and move most of the reasoning to the appendix. Hence I'd like to propose quite a little restructuring before moving the document forward as it is hard to easily catch the main points at the moment. More concrete, I'd move at least section 4, 10 and 11, maybe also 5, 6 and 7 (or at least partly) to the appendix. The first two paragraph of section 7 might actually belong in the security consideration section. And section 15 (conclusion) is not needed at all. More detailed comments below. Mirja ============= Section 1 Introduction: ---------------------------- - s/roughly 15KB/maxium 14600B/ - 2. paragraph: Don't like this two sentences at all. I hope the reason to increase IW is not just the fact there the queues are large enough to store 10 packets. You basically saying "Let's send more packets at once to blow the queues!". I'd say the reason is that flow sizes have increased and therefore IW10 would improve latency a lot. Only because queues are likely to be large than 10 packets today, it is possible at all to send more than 10 packet at once. Btw. even if the queue would be small, pacing could help to still allow an IW of 10. Moreover, the fact that there might be several concurrent flows, does not mean that the buffer has to be larger (only if those flow start at the same time with IW10). And even if there is a large buffer, if there is a long-lived TCP connection, this flow will fill up the queue and there might be by chance still no space in the queue for 10 additional packets. - 3. paragraph: Don't agree. You might not know that there is a low speed link somewhere on the path. There are cases where the endpoint can protect its path by setting the receiver window but that's not the general case. - 5. paragraph: "We recommend that all TCP implementations have a settable TCP IW parameter..." -> I'd like to see this not only in the introduction but also in the main section (2.) that is describing the changes and maybe even as a SHOULD Section 2 TCP Modifications ------------- - Maybe have a more specific title like "Setting the IW" and some subsections - equation 1: say somewhere that numbers are in KByte - 4. paragraph ("Note that...") does not belong in this section - maybe new subsection for paragraph 5 "Resetting the IW after SYN loss"; what are "multiple SYN or SYN/ACK retransmissions" -> maybe say a number - last paragraph: maybe s/implementations SHOULD fall back/implementations MUST fall back/ ? Section 3 Implementation Issues ------------- - Maybe move the 3 paragraph into section 2 such that the setting of the receive window is a MUST or SHOULD of the changes to TCP - 4. paragraph: Please give some explanation (or remove) Section 4 Background ------------ - Move most parts in the appendix - Move 2. paragraph into Introduction - Maybe improve structure with subsections e.g. paragraph 5 and 6 as "Recommendation for HTTP Traffic" or something - 7. paragraph: "pacing will not be necessary" -> I can't remember having seen any results on this. Please explain further or remove! Section 5 Advantages of a Larger Initial Window ----------- - section 5.1 Reducing Latency You should also mention that for larger transmission with several 100s of RTT a save of 4 RTT does not really help. Thus transmissions which will finish within a small number of RTT will benefit the most. - section 5.2 You make some quite general statements here and then u only refer to google data. Maybe you can find some other references as well. Section 6 ---------- - paragraph 3 and 4 belong in an evaluation section/appendix (maybe own subsection on "Influence of simultaneous opened connections") Section 7 -------- - Move 1. paragraph to security consideration - Move rest in appendix or even parts in the introduction Section 9 ---------- - move to main section which is introducing the TCP changes as own subsection as a MUST is in here - can you explain better why this is need or what would happen otherwise? Section 10 ---------- - Move to appendix, have more subsections and better titles for the subsections e.g. "Improvements in Latency", "Impact on the Retransmission Rate" and "Multiple TCP Connections" - The numbers for the latency improvement are only on (google) web search. Of course there are quite large improvements possible but the Internet contains also other traffic. Not sure if your number are really that significant...? Section 11 --------- - move to appendix - It's nice that you list all the studies. Could you please quickly summarize their results/findings, too? Because I guess that the intersting part to know. Section 12 -------- - Its good to have this section but it's quite unspecific. E.g. you say "An increased initial window MUST NOT be turned on by default on systems without such monitoring capabilities." But what exactly should be monitored and how frequently and so on. Also "any significant deterioration" doesn't say anything to me. Is it possible to have a concrete approach/algorithm here what to monitor when and what to do in which cases? And than have this even as part of the main section with TCP changes as this section has also some SHOULD and MUST and might be overread otherwise. Section 14 --------- - "highly unlikely to lead to a persistent state of network congestion" -> Even a "highly unlikely" is concerning me in this case! Also there is no reasoning for this. But I guess you can even say, that this change cannot cause persistent network congestion as congestion control is still used.... Section 15 Conclusion -------- - Is not needed here at all, as no new information Appendix --------- - "One notable exception is YouTube because we don't think the large initial window will have much material impact, either positive or negative, on bulk data services." -> Why is this document than not recommending to NOT use IW10 for bulk data services? - "[CW10] contains some result from a testbed study on how short flows with a larger initial window might affect the throughput performance of other co-existing, long lived, bulk data transfers." -> Please also describe what the results are! - paragraph on "Why 10 segment?" -> This is something I would prefer to have in the body of the document not in the appendix - "Some simulation studies [JNDK10, JNDK10-1] have been conducted to study the effect of a larger initial window on wireless links from 2G to 4G networks (EGDE/HSPA/LTE). The overall result seems mixed in both raw performance and the fairness index." -> Should it be recommend to not use IW10 in wireless scenarios (if known)? From hkchu@google.com Wed Jul 25 10:58:51 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3260C21F86DB for ; Wed, 25 Jul 2012 10:58:51 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.827 X-Spam-Level: X-Spam-Status: No, score=-102.827 tagged_above=-999 required=5 tests=[AWL=-0.150, BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, MIME_8BIT_HEADER=0.3, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6VX3U6nz-wyp for ; Wed, 25 Jul 2012 10:58:50 -0700 (PDT) Received: from mail-lpp01m010-f44.google.com (mail-lpp01m010-f44.google.com [209.85.215.44]) by ietfa.amsl.com (Postfix) with ESMTP id 845CE21F86B0 for ; Wed, 25 Jul 2012 10:58:49 -0700 (PDT) Received: by lagv3 with SMTP id v3so788383lag.31 for ; Wed, 25 Jul 2012 10:58:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding:x-system-of-record; bh=LQZP7q/g2vXMLTpYlP6V+ulyyw/5fhT2Fd0gGo8aHxU=; b=K1vNZmiBfusC21QrJa89b77PTfdzPgkurV2T18h6QELHwhgpNxguKtEJdcSMYFnT+8 WZFD6GgBjYLI5LftkA76dPclgQj69XMRRk2Aw9vQXGy3bUMRgWxaus38D/y/KEXFpeUG PR39kfplpDVXCCfFWe4jKpd7IjOy/27ojOCZXvazKXBmtBb+EIbmV8YXdydEU9nzD/Iy Z09vBBNmX0cKMhg4+VITaVnQaGrgF0pgYMqqwlBum27QfL9gtWaUdfyweEMYtg5bVAJm 8TabLrTG9Rvj8WON8QrG0q9OAcXBB8z+ZhzPKzq5C3sqo77IxTs2CB2KtnC97DinNKca A0pA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding:x-system-of-record :x-gm-message-state; bh=LQZP7q/g2vXMLTpYlP6V+ulyyw/5fhT2Fd0gGo8aHxU=; b=Gj14rv5Oa56b/PvCOe49Si04AIWwkFPnHi//ovPeR/NCpxXOZfEeo1K/9qhaI/IqoO 2rxbKMQjzuKrg29Y+gt6it61j/ZzlUPa/HHBZSoigVvJycKOb5ffqVB62wE2XW+tbNLJ G+sqWSahaTb8FQZr9hL3WvwwFZWsWdVl+g0aSeg6O51yFdoUJ6vRuT7S6LgOU0BZ5laN qNvr9ieduR97UF/RnSrXQIhnWpTMEmjv145zgkROyvYqgDzpLMmq1Lc+Nk9Bm+PU3+K9 k4DrC60aJkql/rBLkB8jsS/N+yZVe6+nW9C1jTxityFoujWk2MwBeNxBJlMpeWLzdiJB uhgQ== Received: by 10.112.17.227 with SMTP id r3mr12014275lbd.41.1343239128378; Wed, 25 Jul 2012 10:58:48 -0700 (PDT) MIME-Version: 1.0 Received: by 10.112.17.227 with SMTP id r3mr12014263lbd.41.1343239128101; Wed, 25 Jul 2012 10:58:48 -0700 (PDT) Received: by 10.152.6.1 with HTTP; Wed, 25 Jul 2012 10:58:48 -0700 (PDT) In-Reply-To: <201207251811.40438.mkuehle@ikr.uni-stuttgart.de> References: <201207251811.40438.mkuehle@ikr.uni-stuttgart.de> Date: Wed, 25 Jul 2012 10:58:48 -0700 Message-ID: From: Jerry Chu To: =?ISO-8859-1?Q?Mirja_K=FChlewind?= Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-System-Of-Record: true X-Gm-Message-State: ALoCoQmgibZy0dMcVj7d/oOxCWi/6ER7e4i0iVl8zfap1ZRRxQCLM9Bv58+fg41Rj+xgn2x8oTF+p40Thurap6ghfIKKKO+tzgfA5M4gszplAjvCgosnusxZkJqpRZJGlywW+c8ni8l7lDQa9OnQWXpifT1pkY3STg1Evs1NtJSVDydHi3dy/K0= Cc: tcpm@ietf.org Subject: Re: [tcpm] Review of draft-ietf-tcpm-initcwnd-04 X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Jul 2012 17:58:51 -0000 On Wed, Jul 25, 2012 at 9:11 AM, Mirja K=FChlewind wrote: > Hi Jerry, > > I'd say the draft now covers all the points discussed in tcpm and as such= has > all the content needed. As you actually only propose a small change to TC= P, > I'd prefer to have a quite short document describing mainly only the chan= ges > itself and move most of the reasoning to the appendix. A document describing only the change can be done, or at least summed up in one line. But this is not how it is or should be done. We are proposing a s= mall change to a critical TCP parameter because the impact can be profound. (Tha= t's why we've been wrestling with it for more than 2 years now!) So the focus is really on the ramification and justification of the change rather than the change itself (which can be summed up in one line). For this reason we've been following = the structure of RFC3390. Moreover, the TOC provides a very clear index of the contents so a savvy reader can quickly pick the sections of interesting. E.g., if you only want to focus on the change you'll know to read section 1 and 2 only. That's it. >Hence I'd like to > propose quite a little restructuring before moving the document forward a= s it > is hard to easily catch the main points at the moment. More concrete, I'd= move > at least section 4, 10 and 11, maybe also 5, 6 and 7 (or at least partly)= to > the appendix. The first two paragraph of section 7 might actually belong = in Yes it's doable but I think the current flow is fine if you realize the main purpose of the document is to justify the change, not the change itself. Also TOC will make it much easier to navigate through. > the security consideration section. And section 15 (conclusion) is not ne= eded > at all. More detailed comments below. > > Mirja > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > Section 1 Introduction: > ---------------------------- > - s/roughly 15KB/maxium 14600B/ Ok. > > - 2. paragraph: Don't like this two sentences at all. I hope the reason t= o > increase IW is not just the fact there the queues are large enough to sto= re 10 > packets. You basically saying "Let's send more packets at once to blow th= e > queues!". I'd say the reason is that flow sizes have increased and theref= ore > IW10 would improve latency a lot. Only because queues are likely to be la= rge > than 10 packets today, it is possible at all to send more than 10 packet = at > once. Btw. even if the queue would be small, pacing could help to still a= llow > an IW of 10. Moreover, the fact that there might be several concurrent fl= ows, > does not mean that the buffer has to be larger (only if those flow start = at > the same time with IW10). And even if there is a large buffer, if there i= s a > long-lived TCP connection, this flow will fill up the queue and there mig= ht > be by chance still no space in the queue for 10 additional packets. Guess you read the paragraph differently from what the authors had in mind. What we have in mind was "Ten segments are likely to fit into queue space" because access link drains much faster these days (due to b/w growth)... Will try to make it more clear. Will continue later, Jerry > > - 3. paragraph: Don't agree. You might not know that there is a low speed= link > somewhere on the path. There are cases where the endpoint can protect its= path > by setting the receiver window but that's not the general case. > > - 5. paragraph: "We recommend that all TCP implementations have a settabl= e TCP > IW parameter..." > -> I'd like to see this not only in the introduction but also in the ma= in > section (2.) that is describing the changes and maybe even as a SHOULD > > > Section 2 TCP Modifications > ------------- > - Maybe have a more specific title like "Setting the IW" and some subsect= ions > > - equation 1: say somewhere that numbers are in KByte > > - 4. paragraph ("Note that...") does not belong in this section > > - maybe new subsection for paragraph 5 "Resetting the IW after SYN loss"; > what are "multiple SYN or SYN/ACK retransmissions" -> maybe say a numbe= r > > - last paragraph: maybe s/implementations SHOULD fall back/implementation= s > MUST fall back/ ? > > > Section 3 Implementation Issues > ------------- > - Maybe move the 3 paragraph into section 2 such that the setting of the > receive window is a MUST or SHOULD of the changes to TCP > > - 4. paragraph: Please give some explanation (or remove) > > > Section 4 Background > ------------ > - Move most parts in the appendix > > - Move 2. paragraph into Introduction > > - Maybe improve structure with subsections e.g. paragraph 5 and 6 > as "Recommendation for HTTP Traffic" or something > > - 7. paragraph: "pacing will not be necessary" > -> I can't remember having seen any results on this. Please explain fur= ther > or remove! > > Section 5 Advantages of a Larger Initial Window > ----------- > - section 5.1 Reducing Latency > You should also mention that for larger transmission with several 100s= of > RTT a save of 4 RTT does not really help. Thus transmissions which will > finish within a small number of RTT will benefit the most. > > - section 5.2 > You make some quite general statements here and then u only refer to go= ogle > data. Maybe you can find some other references as well. > > Section 6 > ---------- > - paragraph 3 and 4 belong in an evaluation section/appendix (maybe own > subsection on "Influence of simultaneous opened connections") > > Section 7 > -------- > - Move 1. paragraph to security consideration > > - Move rest in appendix or even parts in the introduction > > Section 9 > ---------- > - move to main section which is introducing the TCP changes as own subse= ction > as a MUST is in here > > - can you explain better why this is need or what would happen otherwise? > > Section 10 > ---------- > - Move to appendix, have more subsections and better titles for the > subsections e.g. "Improvements in Latency", "Impact on the Retransmission > Rate" and "Multiple TCP Connections" > > - The numbers for the latency improvement are only on (google) web search= . Of > course there are quite large improvements possible but the Internet conta= ins > also other traffic. Not sure if your number are really that significant..= .? > > Section 11 > --------- > - move to appendix > > - It's nice that you list all the studies. Could you please quickly summa= rize > their results/findings, too? Because I guess that the intersting part to > know. > > Section 12 > -------- > - Its good to have this section but it's quite unspecific. E.g. you say "= An > increased initial window MUST NOT be turned on by default on systems with= out > such monitoring capabilities." But what exactly should be monitored and h= ow > frequently and so on. Also "any significant deterioration" doesn't say > anything to me. Is it possible to have a concrete approach/algorithm here > what to monitor when and what to do in which cases? And than have this ev= en > as part of the main section with TCP changes as this section has also som= e > SHOULD and MUST and might be overread otherwise. > > Section 14 > --------- > - "highly unlikely to lead to a persistent state of network congestion" -= > > Even a "highly unlikely" is concerning me in this case! Also there is no > reasoning for this. But I guess you can even say, that this change cannot > cause persistent network congestion as congestion control is still used..= .. > > Section 15 Conclusion > -------- > - Is not needed here at all, as no new information > > Appendix > --------- > - "One notable exception is YouTube because we don't think > the large initial window will have much material impact, either > positive or negative, on bulk data services." > -> Why is this document than not recommending to NOT use IW10 for bulk = data > services? > > - "[CW10] contains some result from a testbed study on how short flows > with a larger initial window might affect the throughput > performance of other co-existing, long lived, bulk data transfers." > -> Please also describe what the results are! > > - paragraph on "Why 10 segment?" > -> This is something I would prefer to have in the body of the document= not > in the appendix > > - "Some simulation studies [JNDK10, JNDK10-1] have been conducted to > study the effect of a larger initial window on wireless links from > 2G to 4G networks (EGDE/HSPA/LTE). The overall result seems mixed > in both raw performance and the fairness index." > -> Should it be recommend to not use IW10 in wireless scenarios (if kno= wn)? From mirja.kuehlewind@ikr.uni-stuttgart.de Wed Jul 25 11:12:56 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4187921F8741 for ; Wed, 25 Jul 2012 11:12:56 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.369 X-Spam-Level: X-Spam-Status: No, score=-1.369 tagged_above=-999 required=5 tests=[AWL=0.880, BAYES_00=-2.599, HELO_EQ_DE=0.35] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id PiR6+qGv43pd for ; Wed, 25 Jul 2012 11:12:55 -0700 (PDT) Received: from mailsrv.ikr.uni-stuttgart.de (mailsrv.ikr.uni-stuttgart.de [129.69.170.2]) by ietfa.amsl.com (Postfix) with ESMTP id 2DEFE21F8734 for ; Wed, 25 Jul 2012 11:12:54 -0700 (PDT) Received: from netsrv1.ikr.uni-stuttgart.de (netsrv1-c [10.11.12.12]) by mailsrv.ikr.uni-stuttgart.de (Postfix) with ESMTP id 03686633B1; Wed, 25 Jul 2012 20:12:53 +0200 (CEST) Received: from vpn-2-cl177 (vpn-2-cl177 [10.41.21.177]) by netsrv1.ikr.uni-stuttgart.de (Postfix) with ESMTP id E8A7659A8A; Wed, 25 Jul 2012 20:12:52 +0200 (CEST) From: Mirja Kuehlewind Organization: University of Stuttgart (Germany), IKR To: Jerry Chu Date: Wed, 25 Jul 2012 20:12:51 +0200 User-Agent: KMail/1.9.10 (enterprise35 0.20101217.1207316) References: <201207251811.40438.mkuehle@ikr.uni-stuttgart.de> In-Reply-To: X-KMail-QuotePrefix: > MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Message-Id: <201207252012.51426.mirja.kuehlewind@ikr.uni-stuttgart.de> Cc: tcpm@ietf.org Subject: Re: [tcpm] Review of draft-ietf-tcpm-initcwnd-04 X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Jul 2012 18:12:56 -0000 Hi Jerry, just one quick comment: > Moreover, the TOC provides a very clear index of the contents so a > savvy reader can > quickly pick the sections of interesting. E.g., if you only want to > focus on the change > you'll know to read section 1 and 2 only. That's it. And that's not really true (anymore). There are several SHOULD and MUST in = the=20 other sections as well. Thus you have to read all the doc to correctly=20 implement the changes. I know that the intent was to have everything in=20 section 1 and 2 that is of interest for implementers but due to the many=20 changes that have been done in previous versions, it's not the case anymore= =2E=20 That why I said, all the content is there, would be nice to=20 clean-up/restructure a little and then it's done. It's good to have a similar structure than RFC3390 (didn't realize that you= =20 tried to have exactly the same sections here) but if the content would fit= =20 better with a different structure, I'd prefer to have a good readable doc=20 over having a doc that looks similar to RFC3390. Mirja > > >Hence I'd like to > > propose quite a little restructuring before moving the document forward > > as it is hard to easily catch the main points at the moment. More > > concrete, I'd move at least section 4, 10 and 11, maybe also 5, 6 and 7 > > (or at least partly) to the appendix. The first two paragraph of section > > 7 might actually belong in > > Yes it's doable but I think the current flow is fine if you realize > the main purpose of > the document is to justify the change, not the change itself. Also TOC > will make it > much easier to navigate through. > > > the security consideration section. And section 15 (conclusion) is not > > needed at all. More detailed comments below. > > > > Mirja > > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > Section 1 Introduction: > > ---------------------------- > > - s/roughly 15KB/maxium 14600B/ > > Ok. > > > - 2. paragraph: Don't like this two sentences at all. I hope the reason > > to increase IW is not just the fact there the queues are large enough to > > store 10 packets. You basically saying "Let's send more packets at once > > to blow the queues!". I'd say the reason is that flow sizes have > > increased and therefore IW10 would improve latency a lot. Only because > > queues are likely to be large than 10 packets today, it is possible at > > all to send more than 10 packet at once. Btw. even if the queue would be > > small, pacing could help to still allow an IW of 10. Moreover, the fact > > that there might be several concurrent flows, does not mean that the > > buffer has to be larger (only if those flow start at the same time with > > IW10). And even if there is a large buffer, if there is a long-lived TCP > > connection, this flow will fill up the queue and there might be by chan= ce > > still no space in the queue for 10 additional packets. > > Guess you read the paragraph differently from what the authors had in min= d. > What we have in mind was "Ten segments are likely to fit into queue spac= e" > because access link drains much faster these days (due to b/w growth)... > > Will try to make it more clear. > > Will continue later, > > Jerry > > > - 3. paragraph: Don't agree. You might not know that there is a low spe= ed > > link somewhere on the path. There are cases where the endpoint can > > protect its path by setting the receiver window but that's not the > > general case. > > > > - 5. paragraph: "We recommend that all TCP implementations have a > > settable TCP IW parameter..." > > -> I'd like to see this not only in the introduction but also in the > > main section (2.) that is describing the changes and maybe even as a > > SHOULD > > > > > > Section 2 TCP Modifications > > ------------- > > - Maybe have a more specific title like "Setting the IW" and some > > subsections > > > > - equation 1: say somewhere that numbers are in KByte > > > > - 4. paragraph ("Note that...") does not belong in this section > > > > - maybe new subsection for paragraph 5 "Resetting the IW after SYN loss= "; > > what are "multiple SYN or SYN/ACK retransmissions" -> maybe say a > > number > > > > - last paragraph: maybe s/implementations SHOULD fall > > back/implementations MUST fall back/ ? > > > > > > Section 3 Implementation Issues > > ------------- > > - Maybe move the 3 paragraph into section 2 such that the setting of the > > receive window is a MUST or SHOULD of the changes to TCP > > > > - 4. paragraph: Please give some explanation (or remove) > > > > > > Section 4 Background > > ------------ > > - Move most parts in the appendix > > > > - Move 2. paragraph into Introduction > > > > - Maybe improve structure with subsections e.g. paragraph 5 and 6 > > as "Recommendation for HTTP Traffic" or something > > > > - 7. paragraph: "pacing will not be necessary" > > -> I can't remember having seen any results on this. Please explain > > further or remove! > > > > Section 5 Advantages of a Larger Initial Window > > ----------- > > - section 5.1 Reducing Latency > > You should also mention that for larger transmission with several 10= 0s > > of RTT a save of 4 RTT does not really help. Thus transmissions which > > will finish within a small number of RTT will benefit the most. > > > > - section 5.2 > > You make some quite general statements here and then u only refer to > > google data. Maybe you can find some other references as well. > > > > Section 6 > > ---------- > > - paragraph 3 and 4 belong in an evaluation section/appendix (maybe own > > subsection on "Influence of simultaneous opened connections") > > > > Section 7 > > -------- > > - Move 1. paragraph to security consideration > > > > - Move rest in appendix or even parts in the introduction > > > > Section 9 > > ---------- > > - move to main section which is introducing the TCP changes as own > > subsection as a MUST is in here > > > > - can you explain better why this is need or what would happen otherwis= e? > > > > Section 10 > > ---------- > > - Move to appendix, have more subsections and better titles for the > > subsections e.g. "Improvements in Latency", "Impact on the Retransmissi= on > > Rate" and "Multiple TCP Connections" > > > > - The numbers for the latency improvement are only on (google) web > > search. Of course there are quite large improvements possible but the > > Internet contains also other traffic. Not sure if your number are really > > that significant...? > > > > Section 11 > > --------- > > - move to appendix > > > > - It's nice that you list all the studies. Could you please quickly > > summarize their results/findings, too? Because I guess that the > > intersting part to know. > > > > Section 12 > > -------- > > - Its good to have this section but it's quite unspecific. E.g. you say > > "An increased initial window MUST NOT be turned on by default on systems > > without such monitoring capabilities." But what exactly should be > > monitored and how frequently and so on. Also "any significant > > deterioration" doesn't say anything to me. Is it possible to have a > > concrete approach/algorithm here what to monitor when and what to do in > > which cases? And than have this even as part of the main section with T= CP > > changes as this section has also some SHOULD and MUST and might be > > overread otherwise. > > > > Section 14 > > --------- > > - "highly unlikely to lead to a persistent state of network congestion" > > -> Even a "highly unlikely" is concerning me in this case! Also there is > > no reasoning for this. But I guess you can even say, that this change > > cannot cause persistent network congestion as congestion control is sti= ll > > used.... > > > > Section 15 Conclusion > > -------- > > - Is not needed here at all, as no new information > > > > Appendix > > --------- > > - "One notable exception is YouTube because we don't think > > the large initial window will have much material impact, either > > positive or negative, on bulk data services." > > -> Why is this document than not recommending to NOT use IW10 for bulk > > data services? > > > > - "[CW10] contains some result from a testbed study on how short flows > > with a larger initial window might affect the throughput > > performance of other co-existing, long lived, bulk data transfers." > > -> Please also describe what the results are! > > > > - paragraph on "Why 10 segment?" > > -> This is something I would prefer to have in the body of the docume= nt > > not in the appendix > > > > - "Some simulation studies [JNDK10, JNDK10-1] have been conducted to > > study the effect of a larger initial window on wireless links from > > 2G to 4G networks (EGDE/HSPA/LTE). The overall result seems mixed > > in both raw performance and the fairness index." > > -> Should it be recommend to not use IW10 in wireless scenarios (if > > known)? =2D-=20 =2D------------------------------------------------------------------ Dipl.-Ing. Mirja K=FChlewind Institute of Communication Networks and Computer Engineering (IKR) University of Stuttgart, Germany Pfaffenwaldring 47, D-70569 Stuttgart tel: +49(0)711/685-67973 email: mirja.kuehlewind@ikr.uni-stuttgart.de web: www.ikr.uni-stuttgart.de =2D------------------------------------------------------------------ From pasi.sarolahti@iki.fi Wed Jul 25 11:35:10 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7A55821F866B for ; Wed, 25 Jul 2012 11:35:10 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.134 X-Spam-Level: X-Spam-Status: No, score=-102.134 tagged_above=-999 required=5 tests=[AWL=0.465, BAYES_00=-2.599, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id eR34Vssh30FG for ; Wed, 25 Jul 2012 11:35:09 -0700 (PDT) Received: from kirsi2.inet.fi (mta-out.inet.fi [195.156.147.13]) by ietfa.amsl.com (Postfix) with ESMTP id 423BE21F8602 for ; Wed, 25 Jul 2012 11:35:08 -0700 (PDT) Received: from [192.168.1.65] (80.223.92.46) by kirsi2.inet.fi (8.5.140.03) (authenticated as saropa-1) id 4FD1DCFC0047E185 for tcpm@ietf.org; Wed, 25 Jul 2012 21:35:07 +0300 From: Pasi Sarolahti Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Date: Wed, 25 Jul 2012 21:35:06 +0300 Message-Id: <2886F8D8-617C-48BD-914A-D6D52861C5DE@iki.fi> To: tcpm@ietf.org Mime-Version: 1.0 (Apple Message framework v1278) X-Mailer: Apple Mail (2.1278) Subject: [tcpm] TCPM WG Status X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Jul 2012 18:35:10 -0000 Hi, Below is a status summary of various TCPM WG drafts, and individual = drafts that are related to the TCPM work. Please let us know, if there = are errors. - Pasi Recent RFCs ----------- The NewReno Modification to TCP's Fast Recovery Algorithm (draft-ietf-tcpm-rfc3782-bis) * Milestone Target: Proposed Standard in April 2011 * Status: RFC 6582 (April 2012) 1948bis (draft-ietf-tcpm-rfc1948bis) * Milestone Target: Proposed Standard in September 2011 * Status: RFC 6528 (Feb 2012) WG Items Nearing RFC Publication -------------------------------- Conservative SACK-based Recovery Algorithm / RFC 3517bis (draft-ietf-tcpm-3517bis) * Milestone Target: Proposed Standard in October 2010 * Status: RFC Editor's queue * Action: RFC Editor processing, then publish RFC MSS Option (draft-ietf-tcpm-tcpmss) * Milestone Target: Proposed Standard in July 2009 * Status: RFC Editor's queue / AUTH48 * Action: Authors to check the document, then publish WG Items in WGLC or being revised after WGLC -------------------------------------------- Increasing the Initial Window (draft-ietf-tcpm-initcwnd) * Milestone Target: Experimental in September 2011 * Status: In WGLC that ends on July 30 (after the TCPM meeting) * Action: WG to review and send comments, or acknowledge that the draft = is ok Active WG Items --------------- 1323bis (draft-ietf-tcpm-1323bis) * Milestone Target: Proposed Standard in July 2009 * Status: Revised after being expired, Richard Scheffenegger has joined = editing team * Action: Should get feedback from WG Proportional Rate Reduction for TCP (draft-ietf-tcpm-proportional-rate-reduction) * Milestone Target: Experimental in May 2012 * Status: Probably ready for WGLC * Action: Chairs plan to start WGLC after IETF-84 meeting Shared Use of Experimental TCP Options (draft-ietf-tcpm-experimental-options) * Milestone Target: PS in Sept. 2012 * Status: Revised after IETF-83, needs WG review * Action: WG review, then perhaps ready for WGLC? TCP Fast Open (draft-ietf-tcpm-fastopen) * Milestone Target: Experimental in Sept. 2012 * Status: Revised after IETF-83 * Action: Needs WG review Discontinued WG Items --------------------- TCP Security (draft-ietf-tcpm-tcp-security) * Milestone Target: BCP in August 2010 * Status: Will be discontinued due to lack of consensus and progress * Action: Chairs to ask milestone removal Active Individual Internet-Drafts --------------------------------- Laminar TCP and the case for refactoring TCP congestion control (draft-mathis-tcpm-tcp-laminar) * Status: Recently updated * Action: Presentation at IETF-84 Updating TCP to support Variable-Rate Traffic (draft-fairhurst-tcpm-newcwv) * Status: ver -03 updated in June 2012, some discussion ongoing * Action: Continue discussion on mailing list, no presentation at = IETF-84 A TCP Authentication Option NAT Extension (draft-touch-tcp-ao-nat) * Status: ver 03 updated on May 30, individual RFC submission planned * Action: Presentation at IETF-84 Automating the Initial Window in TCP (draft-touch-tcpm-automatic-iw) * Status: Recently updated * Action: continue discussion on mailing list (no presentation at = IETF-84) Accurate ECN Feedback Option in TCP (draft-kuehlewind-tcpm-accurate-ecn) (draft-kuehlewind-tcpm-accurate-ecn-option) * Status: Recently updated * Action: Presentation at IETF-84 Additional negotiation in the TCP Timestamp Option field during the TCP = handshake (draft-scheffenegger-tcpm-timestamp-negotiation) * Status: Recently updated * Action: Presentation at IETF-84 TCP and SCTP RTO Restart (draft-hurtig-tcpm-rtorestart) * Status: Recently updated * Action: Presentation at IETF-84 Processing of IP Security/Compartment and Precedence Information by TCP (draft-gont-tcpm-tcp-seccomp-prec) * Status: Not updated since IETF-83 Processing of TCP segments with Mirrored End-points (draft-gont-tcpm-tcp-mirrored-endpoints) * Status: Not updated since IETF-83 TCP Loss Probe (TLP): An Algorithm for Fast Recovery of Tail Losses (draft-dukkipati-tcpm-tcp-loss-probe) * Status: New draft * Action: Presentation at IETF-84 Highly Efficient Selective Acknowledgement (SACK) for TCP (draft-sabatini-tcp-sack) * Status: New draft * Action: Presentation at IETF-84 From hkchu@google.com Wed Jul 25 14:58:52 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A8BBE21F86A0 for ; Wed, 25 Jul 2012 14:58:52 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.902 X-Spam-Level: X-Spam-Status: No, score=-102.902 tagged_above=-999 required=5 tests=[AWL=0.075, BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id p7wZlBxcIs39 for ; Wed, 25 Jul 2012 14:58:51 -0700 (PDT) Received: from mail-lb0-f172.google.com (mail-lb0-f172.google.com [209.85.217.172]) by ietfa.amsl.com (Postfix) with ESMTP id CEEE221F8692 for ; Wed, 25 Jul 2012 14:58:50 -0700 (PDT) Received: by lbbgo11 with SMTP id go11so1044296lbb.31 for ; Wed, 25 Jul 2012 14:58:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding:x-system-of-record; bh=Qh/2h/0BFJDPm0jfDDwe2a1peTf6Z9bAtMy+akqw3RY=; b=oS6qOuO4E9guoe0TJKXb3ree1BqLcJ+SLimwTJ9B1O91WvbR8q1KKl4KOb5D/yXGdG UIEm4h/6i48ItFjxHDQj4rlnRAzgU/v4Y3f7NyU40lDDQJ/OTGu30SXbcjGio9aPSJg3 /49jgREZuMEh7zHg9kw7KMKPt1U1+9mVA7afzYFDSo49AwkxGjegt9zgnX5ViNxbIp+R 1hQ1O4bAKWcVL7Lkibe/1eNNetV0VCfddLhrqDui+1ydT4NxD/8opzj99O9VUpAVrmbH voamvGdSeO6JoegxrgSBSwWJTpVXZpuF8hmBhLsUtS4mQ7utqFxnMSO4cIfUrcBRTqCM DHfA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding:x-system-of-record :x-gm-message-state; bh=Qh/2h/0BFJDPm0jfDDwe2a1peTf6Z9bAtMy+akqw3RY=; b=Q3e9uRV8QPkJ3i2MPYfHvDnKS5g300kXm5MijxHYGpX/av5erCVNm7BgP01pkpuno2 7OAMoARGjhy0VmgpTu1Wp6Ajwn16VfGY6eL21aF6Y780zM+0Q1tzIhFCu1ssGWzwGaT0 FghFGjnuqv8+ewvrtfBm45EGlCRqvy9wdEjoMuYwKQryY482NwiSW6bfIxxWrx273mKa 4UtcMlDWRt9GrjPiU2IMqqpN8Yptr/QxHEYUrqMggdHk2yEu1BX86FSpWDIg5Koe91fC FdDR6se0OlbA1DpRS6lHlsjL4A+rqQmO0JNqwO1VY2D9pDfG4j+Lu8BC0OdNDjUwi3lI QvOA== Received: by 10.112.25.4 with SMTP id y4mr12536327lbf.61.1343253529480; Wed, 25 Jul 2012 14:58:49 -0700 (PDT) MIME-Version: 1.0 Received: by 10.112.25.4 with SMTP id y4mr12536311lbf.61.1343253529017; Wed, 25 Jul 2012 14:58:49 -0700 (PDT) Received: by 10.152.6.1 with HTTP; Wed, 25 Jul 2012 14:58:48 -0700 (PDT) In-Reply-To: <201207252012.51426.mirja.kuehlewind@ikr.uni-stuttgart.de> References: <201207251811.40438.mkuehle@ikr.uni-stuttgart.de> <201207252012.51426.mirja.kuehlewind@ikr.uni-stuttgart.de> Date: Wed, 25 Jul 2012 14:58:48 -0700 Message-ID: From: Jerry Chu To: Mirja Kuehlewind Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-System-Of-Record: true X-Gm-Message-State: ALoCoQlfN6U/f5/dP+utRakBCV4Ul6PLZprYIoJr3ftXc7THnA1/OznSAwP7e4SOQeVtfp7We61ug2yPR5YBOrn51CRnn+l42IoKZih2tl5XxLCH7aU4knWL41fdazDZXUH58i8MkYspweJmqBLKVNj3G4F98ri0dUhMDuNIBOmY/bAMqrGCbww= Cc: tcpm@ietf.org Subject: Re: [tcpm] Review of draft-ietf-tcpm-initcwnd-04 X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Jul 2012 21:58:52 -0000 On Wed, Jul 25, 2012 at 11:12 AM, Mirja Kuehlewind wrote: > Hi Jerry, > > just one quick comment: > >> Moreover, the TOC provides a very clear index of the contents so a >> savvy reader can >> quickly pick the sections of interesting. E.g., if you only want to >> focus on the change >> you'll know to read section 1 and 2 only. That's it. > And that's not really true (anymore). There are several SHOULD and MUST i= n the > other sections as well. Thus you have to read all the doc to correctly > implement the changes. I know that the intent was to have everything in > section 1 and 2 that is of interest for implementers but due to the many > changes that have been done in previous versions, it's not the case anymo= re. Actually the only other specification related languages is section 9, and it's not essential as it simply reiterates what is described in rfc6298. Yes other SHOULD and MUST can be found in the newly added section 12 "Usage and Deployment Recommendations", but that section is mainly about what one must do in any future large scale experiment, closely monitoring a number of key metrics... and not really about protocol specification. So to summarize, this draft is not just about protocol specification, it also addresses many other questions like why, and how to go about doing more experiments safely. > That why I said, all the content is there, would be nice to > clean-up/restructure a little and then it's done. > > It's good to have a similar structure than RFC3390 (didn't realize that y= ou > tried to have exactly the same sections here) but if the content would fi= t It was spelled out up front in 1. Introduction "This document proposes to raise the upper bound on TCP's initial window (IW) to 10 segments or roughly 15KB. It is patterned after and borrows heavily from RFC 3390 [RFC3390] and earlier work in this area." > better with a different structure, I'd prefer to have a good readable doc > over having a doc that looks similar to RFC3390. Let's stick to the current format but I will respond to your one other criticism on section 12 first. The section is quite specific about the metrics to monito= r: "A key metric to monitor is the rate of packet losses, ECN marking, or segment retransmissions during the initial burst." But as others have pointed out it can be quite difficult to try to get more specific than that, i.e., how much performance deteriorates before one must back off. As such all we can do for now is to = have enough warning signs for now but leave some details for future investigatio= n. Jerry > > Mirja > > >> >> >Hence I'd like to >> > propose quite a little restructuring before moving the document forwar= d >> > as it is hard to easily catch the main points at the moment. More >> > concrete, I'd move at least section 4, 10 and 11, maybe also 5, 6 and = 7 >> > (or at least partly) to the appendix. The first two paragraph of secti= on >> > 7 might actually belong in >> >> Yes it's doable but I think the current flow is fine if you realize >> the main purpose of >> the document is to justify the change, not the change itself. Also TOC >> will make it >> much easier to navigate through. >> >> > the security consideration section. And section 15 (conclusion) is not >> > needed at all. More detailed comments below. >> > >> > Mirja >> > >> > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >> > Section 1 Introduction: >> > ---------------------------- >> > - s/roughly 15KB/maxium 14600B/ >> >> Ok. >> >> > - 2. paragraph: Don't like this two sentences at all. I hope the reaso= n >> > to increase IW is not just the fact there the queues are large enough = to >> > store 10 packets. You basically saying "Let's send more packets at onc= e >> > to blow the queues!". I'd say the reason is that flow sizes have >> > increased and therefore IW10 would improve latency a lot. Only because >> > queues are likely to be large than 10 packets today, it is possible at >> > all to send more than 10 packet at once. Btw. even if the queue would = be >> > small, pacing could help to still allow an IW of 10. Moreover, the fac= t >> > that there might be several concurrent flows, does not mean that the >> > buffer has to be larger (only if those flow start at the same time wit= h >> > IW10). And even if there is a large buffer, if there is a long-lived T= CP >> > connection, this flow will fill up the queue and there might be by cha= nce >> > still no space in the queue for 10 additional packets. >> >> Guess you read the paragraph differently from what the authors had in mi= nd. >> What we have in mind was "Ten segments are likely to fit into queue spa= ce" >> because access link drains much faster these days (due to b/w growth)... >> >> Will try to make it more clear. >> >> Will continue later, >> >> Jerry >> >> > - 3. paragraph: Don't agree. You might not know that there is a low sp= eed >> > link somewhere on the path. There are cases where the endpoint can >> > protect its path by setting the receiver window but that's not the >> > general case. >> > >> > - 5. paragraph: "We recommend that all TCP implementations have a >> > settable TCP IW parameter..." >> > -> I'd like to see this not only in the introduction but also in the >> > main section (2.) that is describing the changes and maybe even as a >> > SHOULD >> > >> > >> > Section 2 TCP Modifications >> > ------------- >> > - Maybe have a more specific title like "Setting the IW" and some >> > subsections >> > >> > - equation 1: say somewhere that numbers are in KByte >> > >> > - 4. paragraph ("Note that...") does not belong in this section >> > >> > - maybe new subsection for paragraph 5 "Resetting the IW after SYN los= s"; >> > what are "multiple SYN or SYN/ACK retransmissions" -> maybe say a >> > number >> > >> > - last paragraph: maybe s/implementations SHOULD fall >> > back/implementations MUST fall back/ ? >> > >> > >> > Section 3 Implementation Issues >> > ------------- >> > - Maybe move the 3 paragraph into section 2 such that the setting of t= he >> > receive window is a MUST or SHOULD of the changes to TCP >> > >> > - 4. paragraph: Please give some explanation (or remove) >> > >> > >> > Section 4 Background >> > ------------ >> > - Move most parts in the appendix >> > >> > - Move 2. paragraph into Introduction >> > >> > - Maybe improve structure with subsections e.g. paragraph 5 and 6 >> > as "Recommendation for HTTP Traffic" or something >> > >> > - 7. paragraph: "pacing will not be necessary" >> > -> I can't remember having seen any results on this. Please explain >> > further or remove! >> > >> > Section 5 Advantages of a Larger Initial Window >> > ----------- >> > - section 5.1 Reducing Latency >> > You should also mention that for larger transmission with several 1= 00s >> > of RTT a save of 4 RTT does not really help. Thus transmissions which >> > will finish within a small number of RTT will benefit the most. >> > >> > - section 5.2 >> > You make some quite general statements here and then u only refer to >> > google data. Maybe you can find some other references as well. >> > >> > Section 6 >> > ---------- >> > - paragraph 3 and 4 belong in an evaluation section/appendix (maybe ow= n >> > subsection on "Influence of simultaneous opened connections") >> > >> > Section 7 >> > -------- >> > - Move 1. paragraph to security consideration >> > >> > - Move rest in appendix or even parts in the introduction >> > >> > Section 9 >> > ---------- >> > - move to main section which is introducing the TCP changes as own >> > subsection as a MUST is in here >> > >> > - can you explain better why this is need or what would happen otherwi= se? >> > >> > Section 10 >> > ---------- >> > - Move to appendix, have more subsections and better titles for the >> > subsections e.g. "Improvements in Latency", "Impact on the Retransmiss= ion >> > Rate" and "Multiple TCP Connections" >> > >> > - The numbers for the latency improvement are only on (google) web >> > search. Of course there are quite large improvements possible but the >> > Internet contains also other traffic. Not sure if your number are real= ly >> > that significant...? >> > >> > Section 11 >> > --------- >> > - move to appendix >> > >> > - It's nice that you list all the studies. Could you please quickly >> > summarize their results/findings, too? Because I guess that the >> > intersting part to know. >> > >> > Section 12 >> > -------- >> > - Its good to have this section but it's quite unspecific. E.g. you sa= y >> > "An increased initial window MUST NOT be turned on by default on syste= ms >> > without such monitoring capabilities." But what exactly should be >> > monitored and how frequently and so on. Also "any significant >> > deterioration" doesn't say anything to me. Is it possible to have a >> > concrete approach/algorithm here what to monitor when and what to do i= n >> > which cases? And than have this even as part of the main section with = TCP >> > changes as this section has also some SHOULD and MUST and might be >> > overread otherwise. >> > >> > Section 14 >> > --------- >> > - "highly unlikely to lead to a persistent state of network congestion= " >> > -> Even a "highly unlikely" is concerning me in this case! Also there = is >> > no reasoning for this. But I guess you can even say, that this change >> > cannot cause persistent network congestion as congestion control is st= ill >> > used.... >> > >> > Section 15 Conclusion >> > -------- >> > - Is not needed here at all, as no new information >> > >> > Appendix >> > --------- >> > - "One notable exception is YouTube because we don't think >> > the large initial window will have much material impact, either >> > positive or negative, on bulk data services." >> > -> Why is this document than not recommending to NOT use IW10 for bu= lk >> > data services? >> > >> > - "[CW10] contains some result from a testbed study on how short flows >> > with a larger initial window might affect the throughput >> > performance of other co-existing, long lived, bulk data transfers= ." >> > -> Please also describe what the results are! >> > >> > - paragraph on "Why 10 segment?" >> > -> This is something I would prefer to have in the body of the docum= ent >> > not in the appendix >> > >> > - "Some simulation studies [JNDK10, JNDK10-1] have been conducted to >> > study the effect of a larger initial window on wireless links fro= m >> > 2G to 4G networks (EGDE/HSPA/LTE). The overall result seems mixed >> > in both raw performance and the fairness index." >> > -> Should it be recommend to not use IW10 in wireless scenarios (if >> > known)? > > > > -- > ------------------------------------------------------------------- > Dipl.-Ing. Mirja K=FChlewind > Institute of Communication Networks and Computer Engineering (IKR) > University of Stuttgart, Germany > Pfaffenwaldring 47, D-70569 Stuttgart > > tel: +49(0)711/685-67973 > email: mirja.kuehlewind@ikr.uni-stuttgart.de > web: www.ikr.uni-stuttgart.de > ------------------------------------------------------------------- From jaqueline.queiroz@orange.com Thu Jul 26 00:23:51 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id EC06E21F866B for ; Thu, 26 Jul 2012 00:23:51 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.488 X-Spam-Level: X-Spam-Status: No, score=-1.488 tagged_above=-999 required=5 tests=[AWL=1.111, BAYES_00=-2.599, UNPARSEABLE_RELAY=0.001] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tdH7HFN9J1ro for ; Thu, 26 Jul 2012 00:23:51 -0700 (PDT) Received: from relais-inet.francetelecom.com (relais-ias92.francetelecom.com [193.251.215.92]) by ietfa.amsl.com (Postfix) with ESMTP id 799C421F85D4 for ; Thu, 26 Jul 2012 00:23:50 -0700 (PDT) Received: from omfedm07.si.francetelecom.fr (unknown [xx.xx.xx.3]) by omfedm11.si.francetelecom.fr (ESMTP service) with ESMTP id DD6593B43E2; Thu, 26 Jul 2012 09:23:49 +0200 (CEST) Received: from Exchangemail-eme1.itn.ftgroup (unknown [10.114.1.186]) by omfedm07.si.francetelecom.fr (ESMTP service) with ESMTP id B15844C070; Thu, 26 Jul 2012 09:23:49 +0200 (CEST) Received: from PEXCVZYM12.corporate.adroot.infra.ftgroup ([fe80::81f:1640:4749:5d13]) by PEXCVZYH01.corporate.adroot.infra.ftgroup ([::1]) with mapi id 14.02.0298.004; Thu, 26 Jul 2012 09:23:47 +0200 From: To: "tcpm@ietf.org" Thread-Topic: draft-abdo-hostid-tcpopt-implementation Thread-Index: Ac1q/5bmla1TfDHBRUqnuXltYe9O+Q== Date: Thu, 26 Jul 2012 07:23:47 +0000 Message-ID: <7b205346-aa08-4038-b8ca-348cdd0def59@PEXCVZYH01.corporate.adroot.infra.ftgroup> Accept-Language: fr-FR, en-US Content-Language: fr-FR X-MS-Has-Attach: yes X-MS-TNEF-Correlator: x-originating-ip: [10.197.38.6] Content-Type: multipart/signed; protocol="application/x-pkcs7-signature"; micalg=SHA1; boundary="----=_NextPart_000_000C_01CD6B10.5A827400" MIME-Version: 1.0 X-PMX-Version: 5.6.1.2065439, Antispam-Engine: 2.7.2.376379, Antispam-Data: 2012.6.19.115414 Cc: BOUCADAIR Mohamed OLNC/NAD/TIP , Dan Wing Subject: [tcpm] draft-abdo-hostid-tcpopt-implementation X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Jul 2012 07:23:52 -0000 ------=_NextPart_000_000C_01CD6B10.5A827400 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Hello, As suggested by the chairs, I'm bringing this subject to the mailing list prior to the meeting. So let me first explain the context and the rationale behind this work. All started within intarea WG with RFC6269 that states the issues when addressing sharing mechanisms are used. With IPv4 exhaustion, and sharing addresses mechanisms being deployed, we're more and more exposed to those issues. In the continuity of this work, I-D.intarea-nat-reveal-analysis describes the possible solutions to mitigate some encountered issues with its pros and cons. The idea is to have a recommendation although for the moment, no consensus yet. HOST_ID as a TCP option is one of the studied solutions, I-D.abdo-hostid-tcpopt-implementation was presented in behave WG during IETF82, but we bring this subject to this WG so TCPM WG is informed about this work and to get a feedback. Tks, Jaqueline ------=_NextPart_000_000C_01CD6B10.5A827400 Content-Type: application/x-pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIZEjCCBggw ggPwoAMCAQICAwC21DANBgkqhkiG9w0BAQUFADBMMRwwGgYDVQQDExNPcmFuZ2UgTGFicyBVc2Vy IENBMQ4wDAYDVQQLEwVBc3BpYzEPMA0GA1UEChMGT3JhbmdlMQswCQYDVQQGEwJGUjAeFw0xMjA3 MDkxMzA2MDBaFw0xNzA3MDgxMzA2MDBaMIGRMQswCQYDVQQGEwJGUjEPMA0GA1UEChMGT3Jhbmdl MQ4wDAYDVQQLEwVBU1BJQzEaMBgGA1UEAxMRSmFxdWVsaW5lIFFVRUlST1oxGDAWBgoJkiaJk/Is ZAEBEwhJSkNVNzQ3NjErMCkGCSqGSIb3DQEJARYcamFxdWVsaW5lLnF1ZWlyb3pAb3JhbmdlLmNv bTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAJBFhevgZhhTOU/ASorq+UvnrJN3Z74u OGn/SbRHPdtJYR79LWX+q3NZ/wkpJZYzGeP0NGNvpCL6KHvyk7ylJFBhgL2SIjawgG35pc2jAnBR 98wo713TcKMHFHtQS7sMQ/jOfrTDSjck1S3wZBUwK8C+6V0eWx0uU+ZSJP1dvZSmH0c2wBTR5bQo VS5ZpDC/8rqpFzo8459kt5cekxAm311bbCdGzRSo5Dtks/ZsCAgiuB/T3JTaU03f8itkWlYGiEDF chE8WTdBwlNBKcKNZfqWX2gnCvOnedb4TWpgiVw+ky79A+wJtfgCLqqBFgWh84HGwIFKzn7lQbrR TcWWEHkCAwEAAaOCAaswggGnMA8GA1UdDwEB/wQFAwMHMAAwEwYDVR0lBAwwCgYIKwYBBQUHAwQw HQYDVR0OBBYEFCj0neD/YSfpGkm3zTCFbE+kWyJbMFUGA1UdIwROMEyAFGOOJTfswgeGFNSOgaMf YRkBfyswoS+BLUNOPU9yYW5nZSBMYWJzIFVzZXIgQ0EsT1U9QXNwaWMsTz1PcmFuZ2UsQz1GUoID AJUxMIGmBgNVHR8EgZ4wgZswMqAwoC6GLGh0dHA6Ly9wLXBraS5yZC5mcmFuY2V0ZWxlY29tLmZy L3VzZXJjcmwuY3JsMDKgMKAuhixodHRwOi8vbC1wa2kucmQuZnJhbmNldGVsZWNvbS5mci91c2Vy Y3JsLmNybDAxoC+gLYYraHR0cDovL3BraS5yZC5mcmFuY2V0ZWxlY29tLmNvbS91c2VyY3JsLmNy bDARBglghkgBhvhCAQEEBAMCBSAwTQYDVR0RBEYwRIEcamFxdWVsaW5lLnF1ZWlyb3pAb3Jhbmdl LmNvbYEkamFxdWVsaW5lLnF1ZWlyb3pAb3JhbmdlLWZ0Z3JvdXAuY29tMA0GCSqGSIb3DQEBBQUA A4ICAQAjKpPMIF4SmiG8VpPgkIXykS2yxl3ZtQHbwxj/lN5hzt9MaWT+qPzQx/OXellyR1rSRWse dPjjeZWq6Bpjrc+yNtFMxgpzjdHYqDGz5vc7o79P1luUIe4a7AeCkaGPdnOn6WbqB23OZ9z9Gsgv JOjfO9o4EuhuD0aiTKtefatOxtGskEjlTIy5TIZNnDdQUe7Pb/6O8uOfbev71cR5pGiJV55TTrOZ X0tGUn4w4rijsuwgukuMroy207OutrA6OHsUxs2Rw51C7dq/I1STBjU4/v9LeQxoLR+ndkVmYYHU f2iR/Puwk++Se49ArnPuIzhnKeCRMwZPvA1b17NAZxvkJc8tGgRbaRSgGgHoJITyffMjClWlhDBe RCIoMAW5my/kkVY3fE1EUUCYhiPE1uLPi69mdjPoKKXjowRyHjfL931nFrn4xUFQ8iYg5nuJofUh XLZxrxatHYc2GJ1820WVBrqmtMtDH/aH1W9obLnPOt7ey3OdVM92UsOnn4LIoj9Mkc824sUJ4gv1 Hy3vVTa8vjDTSHHAbToUEe/I+7NyqXMxAzNKByaT8ZeKfFDQ1QKuac7SnY9zrVdXSiKwmpuiR00S bH97/6QUzcwwCHOiZ06Ku6tIwJ8F9B8YW2frNNRxycTU2l0q5kaBqiS4wf6MaSwqhvOLjsvVCUO9 B6/sCDCCBjwwggQkoAMCAQICAwCVMDANBgkqhkiG9w0BAQUFADBMMQswCQYDVQQGEwJGUjEcMBoG A1UEAxMTT3JhbmdlIExhYnMgUm9vdCBDQTEPMA0GA1UEChMGT3JhbmdlMQ4wDAYDVQQLEwVBc3Bp YzAeFw0xMTA3MDgxMzE2MjFaFw0yNjA3MDQxMzE2MjFaMEwxCzAJBgNVBAYTAkZSMRwwGgYDVQQD ExNPcmFuZ2UgTGFicyBSb290IENBMQ8wDQYDVQQKEwZPcmFuZ2UxDjAMBgNVBAsTBUFzcGljMIIC IjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAxDxdlZmLRPMYS8Ii414DHVq39nfO5CSRO9cg Vlu+nuXLGv1/KByqfxTFpXLOYILhTNmlUuJ4CVzRTp7/D+GpNxCZ4tclUS7DjE0CKvphvsxxQdx6 8pYP1EUV4wUDF09Te+VMQlHGBTrWxV2q3E6F3Ksj1yXhYsR5QmPThNi5kYDH1qi5amAU1bgWoHh+ 7y/fHpzAGOTI5Om/nIg0+agWJONnDfKNWnaT4v5Pyv1ytj1aWCz0VYv1M/DpPH/Sl8rcI3AZ+zZR OrcnNO09BWPsyU+gOeqtZe6nAt6h2mj8vimrnwNkJBUMlMc96do+PJy6WctJaWxwg+Tu2BtTBJa7 k89syJVPXdFb9yH1tnyqYDCjG3zKzCKyfnpOIA/gZrjzRNJDxJRScfEpyX4BenF6zuCgVONofKwz MtL2oyNjXhUD/ivUbKe+NMOfTkCrvCMkogxbM2mzOKHTbntG3cYsprzhMF+aNKu7HPBPuXbmZdC/ AV9Wj/UpehN35K/rLalsEcnrsclbVJ0S+I1ZsCqjJoIQVgUxiLcQs5Hv0LxyDxkDtlaR21jxJtkh XnZm+4YOsjPz7AXseLgKLT0o0HkPQiSV9Da5EIWyislYDmKYdgpSajAh3M2Mo64arjyBiuiz1V64 YwIcw+aLzeN3QcopBoFtqr3pkkY6jbtBtBmiInsCAwEAAaOCASUwggEhMBIGA1UdEwEB/wQIMAYB Af8CAQEwDwYDVR0PAQH/BAUDAwcGADAdBgNVHQ4EFgQUVlvqwX4luyGOma/ubx1FjCuYQ7swHwYD VR0jBBgwFoAUVlvqwX4luyGOma/ubx1FjCuYQ7swgaYGA1UdHwSBnjCBmzAyoDCgLoYsaHR0cDov L3AtcGtpLnJkLmZyYW5jZXRlbGVjb20uZnIvcm9vdGNybC5jcmwwMqAwoC6GLGh0dHA6Ly9sLXBr aS5yZC5mcmFuY2V0ZWxlY29tLmZyL3Jvb3RjcmwuY3JsMDGgL6AthitodHRwOi8vcGtpLnJkLmZy YW5jZXRlbGVjb20uY29tL3Jvb3RjcmwuY3JsMBEGCWCGSAGG+EIBAQQEAwIBFjANBgkqhkiG9w0B AQUFAAOCAgEAMnuCSthwtwJsiI5oXsGsCp1qQDFNWprQKFNE2GwkMIi/bxxEx+4mzkbG6+3rnyLg oUPW1QTpTEVOk4DUL9oOxEkolka5LSmSVKXLZTvfJLy03AJno3waGkCAkhYFnBjVToWSoivQM45E +BNg+cWBOvpICq377zpinWvsUEBvelyjLAFxZa1m+Nth3cUTUdyjO4CpOM/osjTBQ5YjxQ7g7d4g fo/+rJyWS8kC1qGbRIX4khd45Fh8ysZTldK6ScHm76rGZNxMbeHDY5EvZ87Xrn95j4AU9pPO4uhd xp18066gutf1O9eYi/SvtB4C7mTYD/AmIDnPzcyNSOGxxyISnUSkqaGAj74bhdx3f5ceIJ1FsO0M onNTge+QACEZkftvjAhY0bH7ro3stzKD9Kts+cbIfADDqzmtlAmIGhvUN9mhEgBezWdIiIi2RtG0 R9MtLDk33dOSj3XaOW6w5bYxB05uQXwssAYwvJ0Fu3pCUVwNMswutwA46jQLduXH269WfN5e/iNT f8KBgU0CCH6Xhvqs78bCRJ7JRzbA3sSPNTQHz+au99I2E193EnByws44VvLiIXkOyLKoin1eb0uX g/hL5bEUONIgw8D48fIKXHLYFSrTHIC27KQpf5Td5OxfPdSmT1jSPSsgGRblHzpQpI58U2zxxdCQ +1/yq6kDCeUwggZMMIIENKADAgECAgMAttMwDQYJKoZIhvcNAQEFBQAwTDEcMBoGA1UEAxMTT3Jh bmdlIExhYnMgVXNlciBDQTEOMAwGA1UECxMFQXNwaWMxDzANBgNVBAoTBk9yYW5nZTELMAkGA1UE BhMCRlIwHhcNMTIwNzA5MTMwNjAwWhcNMTcwNzA4MTMwNjAwWjCBkTELMAkGA1UEBhMCRlIxDzAN BgNVBAoTBk9yYW5nZTEOMAwGA1UECxMFQVNQSUMxGjAYBgNVBAMTEUphcXVlbGluZSBRVUVJUk9a MRgwFgYKCZImiZPyLGQBARMISUpDVTc0NzYxKzApBgkqhkiG9w0BCQEWHGphcXVlbGluZS5xdWVp cm96QG9yYW5nZS5jb20wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCUYEZ3awDZD9jm uts4NIkV1rBaqmxVde6HBKxNuZaVqGXGoS2j3O55CgzDsQ/mJBJi2Ko2GqRWMSxlsnrKAx+5PfjO ipPhXZqvlNDdiRoO+bfbHJm37m+axidQHUxyjMOTHMHBCTvnhC8WUheZqXLBZiijUV5NQ5z0KbvB Lby0QKOy+/izibbz+lXfqLE9S8siq8ADJIufyR7YiTY7ZAnvXGMWTLiqMkHA40/CKckZ0edZMPs7 CRslOEDvK4FoFLlDLTU/JH5t7chugRmakJMvHESCIM2GLCuRiwucUTL2uPiLUR1fRUSvaOqrrmo2 mbxvRaxZtY+CSk2zOTswOaOfAgMBAAGjggHvMIIB6zAPBgNVHQ8BAf8EBQMDB8AAMCkGA1UdJQQi MCAGCCsGAQUFBwMCBggrBgEFBQcDBAYKKwYBBAGCNxQCAjAdBgNVHQ4EFgQUa9O8IDII3gve2SNg lsRljURT5XcwVQYDVR0jBE4wTIAUY44lN+zCB4YU1I6Box9hGQF/KzChL4EtQ049T3JhbmdlIExh YnMgVXNlciBDQSxPVT1Bc3BpYyxPPU9yYW5nZSxDPUZSggMAlTEwgaYGA1UdHwSBnjCBmzAyoDCg LoYsaHR0cDovL3AtcGtpLnJkLmZyYW5jZXRlbGVjb20uZnIvdXNlcmNybC5jcmwwMqAwoC6GLGh0 dHA6Ly9sLXBraS5yZC5mcmFuY2V0ZWxlY29tLmZyL3VzZXJjcmwuY3JsMDGgL6AthitodHRwOi8v cGtpLnJkLmZyYW5jZXRlbGVjb20uY29tL3VzZXJjcmwuY3JsMBEGCWCGSAGG+EIBAQQEAwIFoDB7 BgNVHREEdDByoCwGCisGAQQBgjcUAgOgHgwcaWpjdTc0NzZAcmQuZnJhbmNldGVsZWNvbS5mcoEc amFxdWVsaW5lLnF1ZWlyb3pAb3JhbmdlLmNvbYEkamFxdWVsaW5lLnF1ZWlyb3pAb3JhbmdlLWZ0 Z3JvdXAuY29tMA0GCSqGSIb3DQEBBQUAA4ICAQBLERFPOfvbU57cfUYN575wbBxhwHEyUN4Tw8OV tdFQ6ZwHcerQ74HB6SykKcz2kLr8eI3Ww7GJXLkYcyyb/j+pvXOeWpoRV6PzXpjYF/p7D6FL6m8r AacP2/2DfXO9mxbKoy1jXkuu8L1uUU+hUnvDjXlAFUp1P7iBSaWlumDGTeaeSKslVkLx0Le2uvyl 7JKab3mNo0VJBf4zvECMTn7Zqd2UVLLBBaM/ykxGQvN/hQ0nHq3nq0Z6xvrZigqEhntfb+49LpsV H+ZTSmr1QnHHEbKDEdBsolFK5Fx+isu6Tv804XfYPDV7YDU63E7wNmSBLgDnm8nQqAjrB/NkNnka jEGlmdj2dp8fEDePeyxkv8KBSPPXimw734+M9DCJfWx9xc0FrnLfVKGNsRzu3P0UhR0qbdPXMxPO IzpeGMjXcR2TDerVdy6y/LM8eVBcMCJeSSJtBWORlnSOSjklC8RViU1MtXtd0JzQz0A3IpXdnutg JriXLEkVZ1+RhHZGG1m5VuvUXmMs/B1uPaWC8DLgbdaexy8koi1O9A5jC8olCb8Bikff2OFmBVAF pqHZ1UtFa1AthVSP8peCg13uEfvOcGaznoBImReon53hZHen/V/ik9XuupM+GG9AGgaNrjzOfxRJ TVb+P/U4JuV7IzRtSpZ7kdNE1L3Cadz8b1ixADCCBnIwggRaoAMCAQICAwCVMTANBgkqhkiG9w0B AQUFADBMMQswCQYDVQQGEwJGUjEcMBoGA1UEAxMTT3JhbmdlIExhYnMgUm9vdCBDQTEPMA0GA1UE ChMGT3JhbmdlMQ4wDAYDVQQLEwVBc3BpYzAeFw0xMTA3MDgxMzMwMDBaFw0yNjA2MjkxMzMwMDBa MEwxHDAaBgNVBAMTE09yYW5nZSBMYWJzIFVzZXIgQ0ExDjAMBgNVBAsTBUFzcGljMQ8wDQYDVQQK EwZPcmFuZ2UxCzAJBgNVBAYTAkZSMIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAjPMm HtccQ+z0QWkL1O2HgtJfqklqw0vvUAF8zkku6Aw6aDzRq7uI9OFuZEIeAgHGIVpH0yOkKAoExazi /q6B8brLuAs0oNiZq6gtPm5C+UnV1/sEuJSWC4MuiActRAMXpOEXuI3VnTDz+s8d6N/Bn45cevbS vM6+ygLH0o2Nn2zVEHyN4zvRdriQv1VDUfqAo7uVjxtbzTvr4NLAIJXUeHdAvop/Lqa1Bd4hUWyB zse8HUTV6zD84baxO7nJb7pBBeFUyhkNRmGyMg/wZJ7LeD2BLSIZ9zY/2T7yMhc0eU+kXDePOxdN 6m83P4W8TXBLFQPM7ZNz7K1YWD/XciF6ZRTExDoUsZ73GYajPVjPgoaVQQcc7HkzfTE+YTLagIjL 4BRbgq/nBUwexPoYEChQ132QFfIBkHjJz3yVzXFGawPGeG+wYGkE+JTsrg1BlXPm6iUCtdVCIovK T2YElcV3Cm5DCI//Ccx/G1+Zu0uCjPzXvzsSAarlhOlP/D9CohA0AIY2xkPNHWZqdeQJ5/DQxzH8 9D5jx9bsfPJONLQvzNSGQbjFYVyb3gf1pBL/BtzoXOJjXetUWt3Hu0nPcMF2r8KgTs7ZcuyPbUiF PmFiWmb16ry+Ls1+w8Q1IGLNZNb695VHEBFQA0icw8rA4zDmozPIK6E1Fi3xyZixByXIbxUCAwEA AaOCAVswggFXMBIGA1UdEwEB/wQIMAYBAf8CAQAwDwYDVR0PAQH/BAUDAwcGADAdBgNVHQ4EFgQU Y44lN+zCB4YU1I6Box9hGQF/KzAwVQYDVR0jBE4wTIAUVlvqwX4luyGOma/ubx1FjCuYQ7uhL4Et Qz1GUixDTj1PcmFuZ2UgTGFicyBSb290IENBLE89T3JhbmdlLE9VPUFzcGljggMAlTAwgaYGA1Ud HwSBnjCBmzAyoDCgLoYsaHR0cDovL3AtcGtpLnJkLmZyYW5jZXRlbGVjb20uZnIvcm9vdGNybC5j cmwwMqAwoC6GLGh0dHA6Ly9sLXBraS5yZC5mcmFuY2V0ZWxlY29tLmZyL3Jvb3RjcmwuY3JsMDGg L6AthitodHRwOi8vcGtpLnJkLmZyYW5jZXRlbGVjb20uY29tL3Jvb3RjcmwuY3JsMBEGCWCGSAGG +EIBAQQEAwIABzANBgkqhkiG9w0BAQUFAAOCAgEASDhiLXUUbhFNYUcKznQeXbsOF73KHfjvpt4G 16yWnq9lhRnBpCwFoe1VsBdyHNQhTNaaC15rdgCJWIj3sTmfGw1/gtBZ8UmT2czrpYTsY5HQ4GVy bq2Ra3N9tnjgOZFUQGtoezSjFbmjAI2MXMFYrHF+s2sKPyssEjuiuylTNyvFB4c0lOF9/zTwZcJ1 rPepcE9a0cFq70JhZYOVmIHMrPKaMnbaLkkuZWex2AKGkt8pXL2I95JMJ1qqB9kOCE+cwCnCJn/G 7CVnLYgZfqF+tptYUom1+QhYj/xZOmr2xBFM3CQDFMUuj15+MzdEHkUC/CplnMloC8EsYZlB5t6o FgTgAvsjdTAzMMGGqusjg5ImbA5Er7p/XxDVuEooWyi6ogaYQuUiRTOr+3xd+173Thb0C7eMRozi hFpcddGhhvD6m819Nuv1SIf22Qr5M4jVQrcBS7rixUuqb4E8wJfFaoCAItHcMK2/ZibXqGe/GXGz R8bZrY4/joXVdPr7vEn+EGQN15zbwS/PKazE2E86ghqFNaSLPA44jNbRH91RRTIszHEoPfc/tyJQ mgGav/7BN4fXrx0rNr+xyzx/eLOvGqCTzJq7m0M5lM3BqAAhhWrLrviNkMbzg6pWzVvw5ThKDZJV yU0COrW7SNpuyoKscNbclhI3TaLsgpYyH38FoSExggMOMIIDCgIBATBTMEwxHDAaBgNVBAMTE09y YW5nZSBMYWJzIFVzZXIgQ0ExDjAMBgNVBAsTBUFzcGljMQ8wDQYDVQQKEwZPcmFuZ2UxCzAJBgNV BAYTAkZSAgMAttMwCQYFKw4DAhoFAKCCAZAwGAYJKoZIhvcNAQkDMQsGCSqGSIb3DQEHATAcBgkq hkiG9w0BCQUxDxcNMTIwNzI2MDcyMzQ0WjAjBgkqhkiG9w0BCQQxFgQUX/WKlGHlHy39CIwrv98G jEo2ajYwYgYJKwYBBAGCNxAEMVUwUzBMMRwwGgYDVQQDExNPcmFuZ2UgTGFicyBVc2VyIENBMQ4w DAYDVQQLEwVBc3BpYzEPMA0GA1UEChMGT3JhbmdlMQswCQYDVQQGEwJGUgIDALbUMGQGCyqGSIb3 DQEJEAILMVWgUzBMMRwwGgYDVQQDExNPcmFuZ2UgTGFicyBVc2VyIENBMQ4wDAYDVQQLEwVBc3Bp YzEPMA0GA1UEChMGT3JhbmdlMQswCQYDVQQGEwJGUgIDALbUMGcGCSqGSIb3DQEJDzFaMFgwCgYI KoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0GCCqGSIb3DQMCAgFAMAcGBSsOAwIHMA0GCCqGSIb3 DQMCAgEoMAcGBSsOAwIaMAoGCCqGSIb3DQIFMA0GCSqGSIb3DQEBAQUABIIBAHF+gGpQVwRJGweJ rJsiiqXIPG95EjI0LFN8318BX73iK4yX1Mu9I8R4qsOVhpDir3Ou7xGd72QOLeTqNGDHpr7xn4cV Rfq2UGHSe3AkwNTODP97e22qiISAMTxnUZVaz0vE+oZ5ICUCUwU6OX5KuD4d0+MQLcCE2YtJDNoe ZCGoYdVmFEXSVbVzrS+i6nq3uX3tTKuQ/iPQl5sF3aPvWGIwYnV5l0NHF5DDPN3EPkt1sbTAJzzI Wfy/nUW2vIFV+Gow6lehgnC2S0qEYPpy77xIu6I4u3kVCvVhlkL4Xkwz/BGcyocxSyjs01658gRl j+PdLk1tBofepliBOpUQCRkAAAAAAAA= ------=_NextPart_000_000C_01CD6B10.5A827400-- From trammell@tik.ee.ethz.ch Thu Jul 26 02:14:07 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DB7AB21F8735 for ; Thu, 26 Jul 2012 02:14:07 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.599 X-Spam-Level: X-Spam-Status: No, score=-6.599 tagged_above=-999 required=5 tests=[AWL=0.000, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Vcic1om5Yo41 for ; Thu, 26 Jul 2012 02:14:06 -0700 (PDT) Received: from smtp.ee.ethz.ch (smtp.ee.ethz.ch [129.132.2.219]) by ietfa.amsl.com (Postfix) with ESMTP id 856AE21F8715 for ; Thu, 26 Jul 2012 02:14:05 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by smtp.ee.ethz.ch (Postfix) with ESMTP id 7BE26D930B for ; Thu, 26 Jul 2012 11:14:01 +0200 (MEST) X-Virus-Scanned: by amavisd-new on smtp.ee.ethz.ch Received: from smtp.ee.ethz.ch ([127.0.0.1]) by localhost (.ee.ethz.ch [127.0.0.1]) (amavisd-new, port 10024) with LMTP id rQ-FJB3dDkg9 for ; Thu, 26 Jul 2012 11:14:01 +0200 (MEST) Received: from [10.0.27.100] (cust-integra-121-161.antanet.ch [80.75.121.161]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: briant) by smtp.ee.ethz.ch (Postfix) with ESMTPSA id F16C6D9307 for ; Thu, 26 Jul 2012 11:14:00 +0200 (MEST) From: Brian Trammell Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Date: Thu, 26 Jul 2012 11:14:00 +0200 References: <21461F16-D33B-4483-B717-3D27614E8934@tik.ee.ethz.ch> To: tcpm@ietf.org Message-Id: Mime-Version: 1.0 (Apple Message framework v1278) X-Mailer: Apple Mail (2.1278) Subject: Re: [tcpm] Questions on draft-scheffenegger-tcpm-timestamp-negotiation-04 X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Jul 2012 09:14:08 -0000 greetings, Michael, all, I've commented on the draft off-list. To continue this discussion = though, I wanted to respond on further thought to a few of these = points.. (oh, and by way of introduction: Hi! I've lurked in TCPM for a = little while; my primary interest in this draft specifically and on the = work here in general is in the passive/hybrid measurability of TCP = parameters for performance profiling, troubleshooting, and = connectivity/bandwidth/latency issue cause analysis.) On Jul 23, 2012, at 7:23 PM, Scharf, Michael (Michael) wrote: > * Is there really a need to announce the clock speed in the SYN? We = need the SYN for atomic negotiation of features. I could imagine that = for the clock speed a post-SYN option could be used as well. Thoughts? >=20 > * I also wonder whether there is some data on how much better explicit = signaling of the clock speed is than learning from heuristics (sorry if = I should have missed that data). >=20 > * Some thought on security considerations: Can a host mess up a delay = measurement by announcing a wrong clock frequency? What would be the = implications? Indeed, I think all of these questions touch on the same issue: if (1) = both implementations use a constant-rate TCP timestamp clock (which, the = draft seems to assume anyway, many do), (2) both implementations know = the rate of their timestamp clock and (3) the error between the desired = rate and actual rate is low for some value of "low", then a small number = of measurements and not-very-complicated mathematics serves to allow = each peer to infer the TS clock rate of each of the other peers it's = talking to. I'm also not aware of published experiments on how well this rate = inference would work, but suspect if it exists it's lurking in the = references of the pile of papers on my desk I haven't read yet. I've = done somewhat related work in recovering precision in truncated = timestamp data from NetFlow exports, and that works reasonably well, so = i wouldn't expect too many surprises (heh, famous last words). =46rom a security standpoint, a malicious remote implementation exposing = an incorrect TS clock rate would seem to be equivalent to one which = erroneously negotiates nonsensical TS clock rate values as in section = 5.3 in the current draft -- it can disadvantage the flow from the = attacker (kind of a silly attack) or, depending on how the = implementation stores rate information, all flows from the attacker's = host (a less silly attack -- although, come to think of it, caching = clock rates by remote address is a bad idea in a world full of NATs). An = implementation that wished to defend against this would have to do rate = inference for verification anyway. All of this falls apart when the rate is explicitly irregular or = dynamic, of course. So rate inference would need to have heuristics for = detecting when a constant rate wasn't, adding complexity. To _expose_ = dynamic TS clock rates, a separate post-SYN option would seem to be = necessary. The other question this raises: while an implementation could certainly = use rate inference to enable some of the use cases outlined in Appendix = B of the current draft, an implementation that did lost retransmission = detection as in B.4, or otherwise corrected RTTM based on inferred = remote clock rate wouldn't be RFC1323-compliant, would it?=20 > * The draft is a "one-in-a-box" proposal for one-way delay measurement = and improved SACK processing. I wonder whether it could make sense to = allow separate enablement of those features. For instance, would it make = have some benefit if a host just announces its clock frequency but = leaves the TS echoing semantics unchanged? I personally believe splitting the draft into "improved SACK processing = (+ extended negotiation mechanism for future use)" and "timestamp rate = exposure/inference" makes sense -- if rate inference works, clock rate = exposure alone without SACK improvements would seem to be a minimal win, = while SACK improvements on their own are more clearly useful. And while = we're at it, if that's specified in such a way that conserves bits in = the TS SYN option for potential future use, so much the better. Best regards, Brian=20 >> -----Original Message----- >> From: tcpm-bounces@ietf.org [mailto:tcpm-bounces@ietf.org] On=20 >> Behalf Of Scheffenegger, Richard >> Sent: Monday, July 16, 2012 10:53 AM >> To: tcpm (tcpm@ietf.org) >> Subject: [tcpm] FW: New Version Notification for=20 >> draft-scheffenegger-tcpm-timestamp-negotiation-04.txt >>=20 >> -----Original Message----- >> From: internet-drafts@ietf.org [mailto:internet-drafts@ietf.org] >> Sent: Montag, 16. Juli 2012 10:43 >> To: Scheffenegger, Richard >> Cc: mirja.kuehlewind@ikr.uni-stuttgart.de >> Subject: New Version Notification for=20 >> draft-scheffenegger-tcpm-timestamp-negotiation-04.txt >>=20 >>=20 >> A new version of I-D,=20 >> draft-scheffenegger-tcpm-timestamp-negotiation-04.txt >> has been successfully submitted by Richard Scheffenegger and=20 >> posted to the >> IETF repository. >>=20 >> Filename: draft-scheffenegger-tcpm-timestamp-negotiation >> Revision: 04 >> Title: Additional negotiation in the TCP=20 >> Timestamp Option field during the TCP handshake >> Creation date: 2012-07-16 >> WG ID: Individual Submission >> Number of pages: 34 >> URL: =20 >> http://www.ietf.org/internet-drafts/draft-scheffenegger-tcpm-t >> imestamp-negotiation-04.txt >> Status: =20 >> http://datatracker.ietf.org/doc/draft-scheffenegger-tcpm-times >> tamp-negotiation >> Htmlized: =20 >> http://tools.ietf.org/html/draft-scheffenegger-tcpm-timestamp- >> negotiation-04 >> Diff: =20 >> http://tools.ietf.org/rfcdiff?url2=3Ddraft-scheffenegger-tcpm-ti >> mestamp-negotiation-04 >>=20 >> Abstract: >> A number of TCP enhancements in so diverse fields as congestion >> control, loss recovery or side-band signaling could be improved by >> allowing both ends of a TCP session to interpret the value=20 >> carried in >> the Timestamp option. Further enhancements are enabled by changing >> the receiver side processing of timestamps in the presence of >> Selective Acknowledgements. >>=20 >> This documents updates RFC1323 and specifies a backwards compatible >> way of negotiating for Timestamp capabilities, and lists a=20 >> number of >> benefits and drawbacks of this approach. >>=20 >>=20 >>=20 >>=20 >>=20 >> The IETF Secretariat >> _______________________________________________ >> tcpm mailing list >> tcpm@ietf.org >> https://www.ietf.org/mailman/listinfo/tcpm >>=20 > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm From rs@netapp.com Thu Jul 26 07:39:09 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7ED6F21F877A for ; Thu, 26 Jul 2012 07:39:09 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -9.978 X-Spam-Level: X-Spam-Status: No, score=-9.978 tagged_above=-999 required=5 tests=[AWL=0.621, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ejk6NM75SesO for ; Thu, 26 Jul 2012 07:39:07 -0700 (PDT) Received: from mx2.netapp.com (mx2.netapp.com [216.240.18.37]) by ietfa.amsl.com (Postfix) with ESMTP id 9071021F85D1 for ; Thu, 26 Jul 2012 07:39:06 -0700 (PDT) X-IronPort-AV: E=Sophos;i="4.77,659,1336374000"; d="scan'208";a="668789738" Received: from smtp1.corp.netapp.com ([10.57.156.124]) by mx2-out.netapp.com with ESMTP; 26 Jul 2012 07:39:05 -0700 Received: from vmwexceht03-prd.hq.netapp.com (vmwexceht03-prd.hq.netapp.com [10.106.76.241]) by smtp1.corp.netapp.com (8.13.1/8.13.1/NTAP-1.6) with ESMTP id q6QEd4Ax012325; Thu, 26 Jul 2012 07:39:04 -0700 (PDT) Received: from SACEXCMBX02-PRD.hq.netapp.com ([169.254.1.34]) by vmwexceht03-prd.hq.netapp.com ([10.106.76.241]) with mapi id 14.02.0298.004; Thu, 26 Jul 2012 07:39:03 -0700 From: "Scheffenegger, Richard" To: "Scharf, Michael (Michael)" Thread-Topic: [tcpm] Questions on draft-scheffenegger-tcpm-timestamp-negotiation-04 Thread-Index: AQHNaPfwfJ3v/ubPIkau/VNdKyA0i5c7kU8w Date: Thu, 26 Jul 2012 14:39:02 +0000 Message-ID: <012C3117EDDB3C4781FD802A8C27DD4F0D4CAACA@SACEXCMBX02-PRD.hq.netapp.com> References: <012C3117EDDB3C4781FD802A8C27DD4F0D4A88D2@SACEXCMBX02-PRD.hq.netapp.com> <2A886F9088894347A3BE0CC5B7A85F3E891A9BBC24@FRMRSSXCHMBSE3.dc-m.alcatel-lucent.com> In-Reply-To: <2A886F9088894347A3BE0CC5B7A85F3E891A9BBC24@FRMRSSXCHMBSE3.dc-m.alcatel-lucent.com> Accept-Language: de-AT, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.104.60.115] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "tcpm@ietf.org" Subject: Re: [tcpm] Questions on draft-scheffenegger-tcpm-timestamp-negotiation-04 X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Jul 2012 14:39:09 -0000 Hi Michael, > I also recall some interest in late enablement of timestamps > earlier this year; this draft seems to suggest quite the opposite... >=20 > * Is there really a need to announce the clock speed in the SYN? We need > the SYN for atomic negotiation of features. I could imagine that for the > clock speed a post-SYN option could be used as well. Thoughts? In this draft, the 1323 TS option is overloaded with additional meaning dur= ing the initial handshake (actually only announcing local settings, when th= e SYN bit is set). Reducing the TS negotiation to a bare minimum was not th= e aim of this draft, rather some unused bytes are redefined/overloaded to f= acilitate a (legacy compatible) negotiation. This overload scheme requires a means to differentiate between the normal u= se of timestamps, and the "announcement" / negotiation phase. In the draft,= this is implicitly done by modifying the signaling only when the SYN bit i= s set. So you are correct, the draft as it is now, rules out late negotiati= on of timestamps. For late negotiation, some kind of reliable signaling would be required, to= carry out that function. If the WG concludes that smaller TS negotiation option (akin to SACK permit= ted option) would be preferable, that could be addressed in a new version o= f the draft. > * I also wonder whether there is some data on how much better explicit > signaling of the clock speed is than learning from heuristics (sorry if I > should have missed that data). Well, the heuristics I have investigated typically assume either a single t= imestamp clock interval (good enough for closed experiments), or select one= of a limited set of predefined timestamp clock intervals. An implicit assu= mption in these heuristics is, that the network conditions don't change too= much during the training phase. However, such an assumption may be skewed = (e.g. slow-path during flow setup (SYN/SYNACK), fast-path during regular da= ta exchange; path changes during training phase etc). Further, new timestam= p clock intervals, or unusual ones used by less common stacks, won't be inc= luded in such a predefined, limited set... However, papers I have read on the (receiver side) use of timestamps for ti= ming purposes usually focus on their new mechanisms, rather than how to obt= ain a good input signal.=20 > * Some thought on security considerations: Can a host mess up a delay > measurement by announcing a wrong clock frequency? What would be the > implications? This is a very good observation - however it can only be answered in the co= ntext of a specific mechanism using timestamps as input. Mirja may have mor= e information as to the effects of a constant skew in one-way delay varianc= e for TCP chirp... > * The masking of the timestamps seems to be motivated by a heuristic that > is deployed in some stacks, but AFAIK not described in an RFC. Maybe it > would make sense first to have some draft that specifies that behavior? Actually, the motivation has to do with the use of timestamps in more core = mechanisms like congestion control. The draft has the relevant reference - = early versions of CUBIC used timestamps directly to determine the congestio= n epoch (to determine the inflection point). An attach was shown where a ma= licious receiver can speed up the inflection point of Cubic by manipulating= the timestamp values. The solution in Linux / Cubic at the time was to NOT= use timestamps directly, but rather make the timestamp option a token into= a lookup table, which the sender can resolve. http://citeseerx.ist.psu.edu/viewdoc/download?doi=3D10.1.1.153.3152&rep=3Dr= ep1&type=3Dpdf, page 11: "[Linux] 2.6.23 The use of received timestamp option value from RTT calcu= lation is removed for preventing possible malicious receiver attacks that r= eports wrong timestamps to reduce RTTs for more throughput." > * Given that the draft changes the TS echoing semantics, it has to update > the RTTM to get some additional offset for the delayed ACKs. Currently, > the draft suggests: "R =3D RTT * ( 1 + 1/(cwnd/smss) )". Is my understand= ing > correct that this formula assumes that every second segment is acked? Correct. > According to RFC 1122, acking every second segment is only a SHOULD. The > MUST just mandates that the additional delay is less than 0.5 seconds. > Thus, is it possible that "R =3D RTT + 0.5s" is needed in some scenarios?= Or > is there even a need to signal the delACK strategy to the remote end, so > that it can adapt its RTTM/RTP calculation? This could be challenging... The formula was a initial attempt to keep the RFC1323 RTTM calculation (for= RTO purpose only) mostly intact, when timestamps are immediately reflected= ... We do have other drafts in current discussion to update the way how to = deal with RTOs (when to restart timers). More sophisticated RTTM methods co= uld also be deployed (making use of the network-only RTT measurement). The = largest deviation is expected for small windows (<=3D 4 packets in flight),= so a different approach could be a constant offset when the window is smal= l... But this draft really is around the modified signaling (and arriving a= t a different rtt input signal that more closely tracks network conditions)= . Best regards, Richard Scheffenegger >=20 > * Very, very globally: We apparently have 4 spare bytes in the SYN option > if TS are used. This draft suggests a really intelligent way of using > those bits. However, if we touch TCP timestamp semantics, wouldn't it be = a > better, though more disruptive approach to replace the TS option by a new > option with less bytes at least in the initial in the SYN? 4 bytes of 40 > is 10%...=20 > I also recall some interest in late enablement of timestamps > earlier this year; this draft seems to suggest quite the opposite... >=20 > * Is there really a need to announce the clock speed in the SYN? We need > the SYN for atomic negotiation of features. I could imagine that for the > clock speed a post-SYN option could be used as well. Thoughts? >=20 > * I also wonder whether there is some data on how much better explicit > signaling of the clock speed is than learning from heuristics (sorry if I > should have missed that data). >=20 > * Some thought on security considerations: Can a host mess up a delay > measurement by announcing a wrong clock frequency? What would be the > implications? >=20 > * The masking of the timestamps seems to be motivated by a heuristic that > is deployed in some stacks, but AFAIK not described in an RFC. Maybe it > would make sense first to have some draft that specifies that behavior? >=20 > * The draft is a "one-in-a-box" proposal for one-way delay measurement an= d > improved SACK processing. I wonder whether it could make sense to allow > separate enablement of those features. For instance, would it make have > some benefit if a host just announces its clock frequency but leaves the > TS echoing semantics unchanged? >=20 > * Given that the draft changes the TS echoing semantics, it has to update > the RTTM to get some additional offset for the delayed ACKs. Currently, > the draft suggests: "R =3D RTT * ( 1 + 1/(cwnd/smss) )". Is my understand= ing > correct that this formula assumes that every second segment is acked? > According to RFC 1122, acking every second segment is only a SHOULD. The > MUST just mandates that the additional delay is less than 0.5 seconds. > Thus, is it possible that "R =3D RTT + 0.5s" is needed in some scenarios?= Or > is there even a need to signal the delACK strategy to the remote end, so > that it can adapt its RTTM/RTP calculation? This could be challenging... >=20 > Sorry if I should have missed something. >=20 > Thanks >=20 > Michael >=20 >=20 > > -----Original Message----- > > From: tcpm-bounces@ietf.org [mailto:tcpm-bounces@ietf.org] On Behalf > > Of Scheffenegger, Richard > > Sent: Monday, July 16, 2012 10:53 AM > > To: tcpm (tcpm@ietf.org) > > Subject: [tcpm] FW: New Version Notification for > > draft-scheffenegger-tcpm-timestamp-negotiation-04.txt > > > > -----Original Message----- > > From: internet-drafts@ietf.org [mailto:internet-drafts@ietf.org] > > Sent: Montag, 16. Juli 2012 10:43 > > To: Scheffenegger, Richard > > Cc: mirja.kuehlewind@ikr.uni-stuttgart.de > > Subject: New Version Notification for > > draft-scheffenegger-tcpm-timestamp-negotiation-04.txt > > > > > > A new version of I-D, > > draft-scheffenegger-tcpm-timestamp-negotiation-04.txt > > has been successfully submitted by Richard Scheffenegger and posted to > > the IETF repository. > > > > Filename: draft-scheffenegger-tcpm-timestamp-negotiation > > Revision: 04 > > Title: Additional negotiation in the TCP > > Timestamp Option field during the TCP handshake > > Creation date: 2012-07-16 > > WG ID: Individual Submission > > Number of pages: 34 > > URL: > > http://www.ietf.org/internet-drafts/draft-scheffenegger-tcpm-t > > imestamp-negotiation-04.txt > > Status: > > http://datatracker.ietf.org/doc/draft-scheffenegger-tcpm-times > > tamp-negotiation > > Htmlized: > > http://tools.ietf.org/html/draft-scheffenegger-tcpm-timestamp- > > negotiation-04 > > Diff: > > http://tools.ietf.org/rfcdiff?url2=3Ddraft-scheffenegger-tcpm-ti > > mestamp-negotiation-04 > > > > Abstract: > > A number of TCP enhancements in so diverse fields as congestion > > control, loss recovery or side-band signaling could be improved by > > allowing both ends of a TCP session to interpret the value carried > > in > > the Timestamp option. Further enhancements are enabled by changing > > the receiver side processing of timestamps in the presence of > > Selective Acknowledgements. > > > > This documents updates RFC1323 and specifies a backwards compatible > > way of negotiating for Timestamp capabilities, and lists a number > > of > > benefits and drawbacks of this approach. > > > > > > > > > > > > The IETF Secretariat > > _______________________________________________ > > tcpm mailing list > > tcpm@ietf.org > > https://www.ietf.org/mailman/listinfo/tcpm > > > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm From rs@netapp.com Thu Jul 26 08:26:07 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 579E021F85DA for ; Thu, 26 Jul 2012 08:26:07 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.035 X-Spam-Level: X-Spam-Status: No, score=-10.035 tagged_above=-999 required=5 tests=[AWL=0.564, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4hldQAP40H6p for ; Thu, 26 Jul 2012 08:26:06 -0700 (PDT) Received: from mx2.netapp.com (mx2.netapp.com [216.240.18.37]) by ietfa.amsl.com (Postfix) with ESMTP id 0CAD321F85BB for ; Thu, 26 Jul 2012 08:26:05 -0700 (PDT) X-IronPort-AV: E=Sophos;i="4.77,659,1336374000"; d="scan'208";a="668808162" Received: from smtp1.corp.netapp.com ([10.57.156.124]) by mx2-out.netapp.com with ESMTP; 26 Jul 2012 08:25:53 -0700 Received: from vmwexceht01-prd.hq.netapp.com (vmwexceht01-prd.hq.netapp.com [10.106.76.239]) by smtp1.corp.netapp.com (8.13.1/8.13.1/NTAP-1.6) with ESMTP id q6QFPrC2026567; Thu, 26 Jul 2012 08:25:53 -0700 (PDT) Received: from SACEXCMBX02-PRD.hq.netapp.com ([169.254.1.34]) by vmwexceht01-prd.hq.netapp.com ([10.106.76.239]) with mapi id 14.02.0298.004; Thu, 26 Jul 2012 08:25:53 -0700 From: "Scheffenegger, Richard" To: Brian Trammell , "tcpm@ietf.org" Thread-Topic: [tcpm] Questions on draft-scheffenegger-tcpm-timestamp-negotiation-04 Thread-Index: AQHNaw7+fJ3v/ubPIkau/VNdKyA0i5c7pxgA Date: Thu, 26 Jul 2012 15:25:52 +0000 Message-ID: <012C3117EDDB3C4781FD802A8C27DD4F0D4CAC0C@SACEXCMBX02-PRD.hq.netapp.com> References: <21461F16-D33B-4483-B717-3D27614E8934@tik.ee.ethz.ch> In-Reply-To: Accept-Language: de-AT, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.104.60.115] Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [tcpm] Questions on draft-scheffenegger-tcpm-timestamp-negotiation-04 X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Jul 2012 15:26:07 -0000 Hi Brian, Michael, > >From a security standpoint, a malicious remote implementation exposing a= n > incorrect TS clock rate would seem to be equivalent to one which > erroneously negotiates nonsensical TS clock rate values as in section 5.3 > in the current draft -- it can disadvantage the flow from the attacker > (kind of a silly attack) or, depending on how the implementation stores > rate information, all flows from the attacker's host (a less silly attack > -- although, come to think of it, caching clock rates by remote address i= s > a bad idea in a world full of NATs).=20 Well, if a sender wishes to have one-way delay variation as input signal, a= nd the receiver tells the sender that its timestamp clock interval runs at = 1ms, when in fact it runs at 100ms (or a variable rate), the receiver may b= e able to tweak the senders congestion control mechanism into sending more = data than it would otherwise...=20 I think this is a bit different; a nonsensical TS clock rate is clearly an = error (and would have the sender revert to RFC1323 mode, not using received= TSval for timing purposes - at least it shouldn't use it for that purpose)= . The proposed MASK field would allow a sender to protect the integrity of th= e TSvals - but that (modifying the TSval when inserting it into TSecr) is a= different attack than what Michael brings up... > An implementation that wished to > defend against this would have to do rate inference for verification > anyway. This appears to be the only prudent protection; a slight deviation (e.g. 10= ms signaled, but a few samples indicate a "true" timestamp clock of 11ms) n= eeds to be addressed by mechanisms making use of this input signal... Note that the problems of rate inference (basically assuming that the netwo= rk conditions don't change during the measurement interval) still have to b= e addressed... > All of this falls apart when the rate is explicitly irregular or dynamic, > of course. So rate inference would need to have heuristics for detecting > when a constant rate wasn't, adding complexity. To _expose_ dynamic TS > clock rates, a separate post-SYN option would seem to be necessary. RFC1323 allows the timestamp to serve as simple packet counter (TSval still= monotonic increasing, roughly related to wall time - but with variable int= ervals). A rate inference algorithm will probably measure RTT during the in= itial phase of the session as timestamp clock interval... The other issue with rate inference is that a good probing requires exactly= timed sending of packets, and multiple measurements - thus only for longer= running sessions a good value can be expected. An explicit (trusted) signa= l would alleviate this delay... =20 > I personally believe splitting the draft into "improved SACK processing (= + > extended negotiation mechanism for future use)" and "timestamp rate > exposure/inference" makes sense -- if rate inference works, clock rate > exposure alone without SACK improvements would seem to be a minimal win, > while SACK improvements on their own are more clearly useful. And while > we're at it, if that's specified in such a way that conserves bits in the > TS SYN option for potential future use, so much the better. Sounds like a case for a new TS permitted option, plus an optional, reliabl= e, late handshake mechanism? Best regards, Richard Scheffenegger NetApp rs@netapp.com +43 1 3676811 3146 Office (2143 3146 - internal) +43 676 654 3146 Mobile www.netapp.com=A0=A0 EURO PLAZA Geb=E4ude G, Stiege 7, 3.OG Am Euro Platz 2 A-1120 Wien > -----Original Message----- > From: tcpm-bounces@ietf.org [mailto:tcpm-bounces@ietf.org] On Behalf Of > Brian Trammell > Sent: Donnerstag, 26. Juli 2012 11:14 > To: tcpm@ietf.org > Subject: Re: [tcpm] Questions on draft-scheffenegger-tcpm-timestamp- > negotiation-04 >=20 > greetings, Michael, all, >=20 > I've commented on the draft off-list. To continue this discussion though, > I wanted to respond on further thought to a few of these points.. (oh, an= d > by way of introduction: Hi! I've lurked in TCPM for a little while; my > primary interest in this draft specifically and on the work here in > general is in the passive/hybrid measurability of TCP parameters for > performance profiling, troubleshooting, and connectivity/bandwidth/latenc= y > issue cause analysis.) >=20 > On Jul 23, 2012, at 7:23 PM, Scharf, Michael (Michael) wrote: >=20 > > * Is there really a need to announce the clock speed in the SYN? We nee= d > the SYN for atomic negotiation of features. I could imagine that for the > clock speed a post-SYN option could be used as well. Thoughts? > > > > * I also wonder whether there is some data on how much better explicit > signaling of the clock speed is than learning from heuristics (sorry if I > should have missed that data). > > > > * Some thought on security considerations: Can a host mess up a delay > measurement by announcing a wrong clock frequency? What would be the > implications? >=20 > Indeed, I think all of these questions touch on the same issue: if (1) > both implementations use a constant-rate TCP timestamp clock (which, the > draft seems to assume anyway, many do), (2) both implementations know the > rate of their timestamp clock and (3) the error between the desired rate > and actual rate is low for some value of "low", then a small number of > measurements and not-very-complicated mathematics serves to allow each > peer to infer the TS clock rate of each of the other peers it's talking > to. >=20 > I'm also not aware of published experiments on how well this rate > inference would work, but suspect if it exists it's lurking in the > references of the pile of papers on my desk I haven't read yet. I've done > somewhat related work in recovering precision in truncated timestamp data > from NetFlow exports, and that works reasonably well, so i wouldn't expec= t > too many surprises (heh, famous last words). >=20 > >From a security standpoint, a malicious remote implementation exposing a= n > incorrect TS clock rate would seem to be equivalent to one which > erroneously negotiates nonsensical TS clock rate values as in section 5.3 > in the current draft -- it can disadvantage the flow from the attacker > (kind of a silly attack) or, depending on how the implementation stores > rate information, all flows from the attacker's host (a less silly attack > -- although, come to think of it, caching clock rates by remote address i= s > a bad idea in a world full of NATs). An implementation that wished to > defend against this would have to do rate inference for verification > anyway. >=20 > All of this falls apart when the rate is explicitly irregular or dynamic, > of course. So rate inference would need to have heuristics for detecting > when a constant rate wasn't, adding complexity. To _expose_ dynamic TS > clock rates, a separate post-SYN option would seem to be necessary. >=20 > The other question this raises: while an implementation could certainly > use rate inference to enable some of the use cases outlined in Appendix B > of the current draft, an implementation that did lost retransmission > detection as in B.4, or otherwise corrected RTTM based on inferred remote > clock rate wouldn't be RFC1323-compliant, would it? >=20 > > * The draft is a "one-in-a-box" proposal for one-way delay measurement > and improved SACK processing. I wonder whether it could make sense to > allow separate enablement of those features. For instance, would it make > have some benefit if a host just announces its clock frequency but leaves > the TS echoing semantics unchanged? >=20 > I personally believe splitting the draft into "improved SACK processing (= + > extended negotiation mechanism for future use)" and "timestamp rate > exposure/inference" makes sense -- if rate inference works, clock rate > exposure alone without SACK improvements would seem to be a minimal win, > while SACK improvements on their own are more clearly useful. And while > we're at it, if that's specified in such a way that conserves bits in the > TS SYN option for potential future use, so much the better. >=20 > Best regards, >=20 > Brian >=20 >=20 > >> -----Original Message----- > >> From: tcpm-bounces@ietf.org [mailto:tcpm-bounces@ietf.org] On Behalf > >> Of Scheffenegger, Richard > >> Sent: Monday, July 16, 2012 10:53 AM > >> To: tcpm (tcpm@ietf.org) > >> Subject: [tcpm] FW: New Version Notification for > >> draft-scheffenegger-tcpm-timestamp-negotiation-04.txt > >> > >> -----Original Message----- > >> From: internet-drafts@ietf.org [mailto:internet-drafts@ietf.org] > >> Sent: Montag, 16. Juli 2012 10:43 > >> To: Scheffenegger, Richard > >> Cc: mirja.kuehlewind@ikr.uni-stuttgart.de > >> Subject: New Version Notification for > >> draft-scheffenegger-tcpm-timestamp-negotiation-04.txt > >> > >> > >> A new version of I-D, > >> draft-scheffenegger-tcpm-timestamp-negotiation-04.txt > >> has been successfully submitted by Richard Scheffenegger and posted > >> to the IETF repository. > >> > >> Filename: draft-scheffenegger-tcpm-timestamp-negotiation > >> Revision: 04 > >> Title: Additional negotiation in the TCP > >> Timestamp Option field during the TCP handshake > >> Creation date: 2012-07-16 > >> WG ID: Individual Submission > >> Number of pages: 34 > >> URL: > >> http://www.ietf.org/internet-drafts/draft-scheffenegger-tcpm-t > >> imestamp-negotiation-04.txt > >> Status: > >> http://datatracker.ietf.org/doc/draft-scheffenegger-tcpm-times > >> tamp-negotiation > >> Htmlized: > >> http://tools.ietf.org/html/draft-scheffenegger-tcpm-timestamp- > >> negotiation-04 > >> Diff: > >> http://tools.ietf.org/rfcdiff?url2=3Ddraft-scheffenegger-tcpm-ti > >> mestamp-negotiation-04 > >> > >> Abstract: > >> A number of TCP enhancements in so diverse fields as congestion > >> control, loss recovery or side-band signaling could be improved by > >> allowing both ends of a TCP session to interpret the value carried in > >> the Timestamp option. Further enhancements are enabled by changing > >> the receiver side processing of timestamps in the presence of > >> Selective Acknowledgements. > >> > >> This documents updates RFC1323 and specifies a backwards compatible > >> way of negotiating for Timestamp capabilities, and lists a number of > >> benefits and drawbacks of this approach. > >> > >> > >> > >> > >> > >> The IETF Secretariat > >> _______________________________________________ > >> tcpm mailing list > >> tcpm@ietf.org > >> https://www.ietf.org/mailman/listinfo/tcpm > >> > > _______________________________________________ > > tcpm mailing list > > tcpm@ietf.org > > https://www.ietf.org/mailman/listinfo/tcpm >=20 >=20 > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm From wwwrun@rfc-editor.org Thu Jul 26 13:28:23 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A220311E80C2; Thu, 26 Jul 2012 13:28:23 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.268 X-Spam-Level: X-Spam-Status: No, score=-102.268 tagged_above=-999 required=5 tests=[AWL=0.332, BAYES_00=-2.599, NO_RELAYS=-0.001, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mxkGZhcZPeg5; Thu, 26 Jul 2012 13:28:23 -0700 (PDT) Received: from rfc-editor.org (rfc-editor.org [IPv6:2001:1890:123a::1:2f]) by ietfa.amsl.com (Postfix) with ESMTP id 2615811E8080; Thu, 26 Jul 2012 13:28:23 -0700 (PDT) Received: by rfc-editor.org (Postfix, from userid 30) id AF02672E009; Thu, 26 Jul 2012 13:28:06 -0700 (PDT) To: ietf-announce@ietf.org, rfc-dist@rfc-editor.org From: rfc-editor@rfc-editor.org Message-Id: <20120726202806.AF02672E009@rfc-editor.org> Date: Thu, 26 Jul 2012 13:28:06 -0700 (PDT) Cc: tcpm@ietf.org, rfc-editor@rfc-editor.org Subject: [tcpm] RFC 6691 on TCP Options and Maximum Segment Size (MSS) X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Jul 2012 20:28:23 -0000 A new Request for Comments is now available in online RFC libraries. RFC 6691 Title: TCP Options and Maximum Segment Size (MSS) Author: D. Borman Status: Informational Stream: IETF Date: July 2012 Mailbox: david.borman@quantum.com Pages: 9 Characters: 16707 Updates: RFC0879, RFC2385 I-D Tag: draft-ietf-tcpm-tcpmss-05.txt URL: http://www.rfc-editor.org/rfc/rfc6691.txt This memo discusses what value to use with the TCP Maximum Segment Size (MSS) option, and updates RFC 879 and RFC 2385. This document is not an Internet Standards Track specification; it is published for informational purposes. This document is a product of the TCP Maintenance and Minor Extensions Working Group of the IETF. INFORMATIONAL: This memo provides information for the Internet community. It does not specify an Internet standard of any kind. Distribution of this memo is unlimited. This announcement is sent to the IETF-Announce and rfc-dist lists. To subscribe or unsubscribe, see http://www.ietf.org/mailman/listinfo/ietf-announce http://mailman.rfc-editor.org/mailman/listinfo/rfc-dist For searching the RFC series, see http://www.rfc-editor.org/rfcsearch.html. For downloading RFCs, see http://www.rfc-editor.org/rfc.html. Requests for special distribution should be addressed to either the author of the RFC in question, or to rfc-editor@rfc-editor.org. Unless specifically noted otherwise on the RFC itself, all RFCs are for unlimited distribution. The RFC Editor Team Association Management Solutions, LLC From bob.briscoe@bt.com Thu Jul 26 22:19:54 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0B11F21F85C2 for ; Thu, 26 Jul 2012 22:19:54 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.598 X-Spam-Level: X-Spam-Status: No, score=-3.598 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id yRF+InBkxHnw for ; Thu, 26 Jul 2012 22:19:52 -0700 (PDT) Received: from smtpe1.intersmtp.com (smtp62.intersmtp.com [62.239.224.235]) by ietfa.amsl.com (Postfix) with ESMTP id 3C30E21F85C0 for ; Thu, 26 Jul 2012 22:19:52 -0700 (PDT) Received: from EVMHT63-UKRD.domain1.systemhost.net (10.36.3.100) by RDW083A006ED62.smtp-e2.hygiene.service (10.187.98.11) with Microsoft SMTP Server (TLS) id 8.3.264.0; Fri, 27 Jul 2012 06:19:51 +0100 Received: from EVMHT04-UKBR.domain1.systemhost.net (193.113.108.57) by EVMHT63-UKRD.domain1.systemhost.net (10.36.3.100) with Microsoft SMTP Server (TLS) id 8.3.264.0; Fri, 27 Jul 2012 06:19:51 +0100 Received: from EMV04-UKBR.domain1.systemhost.net ([169.254.1.122]) by EVMHT04-UKBR.domain1.systemhost.net ([193.113.108.57]) with mapi; Fri, 27 Jul 2012 06:19:50 +0100 From: To: Date: Fri, 27 Jul 2012 06:19:49 +0100 Thread-Topic: Some ideas building on draft-ietf-tcpm-fastopen-01 Thread-Index: AQHNa7dyBV6au7zMHEKTqw5AZStHJQ== Message-ID: <69ACA3A4BA39B0499C1917FB0718BB254BE747BB86@EMV04-UKBR.domain1.systemhost.net> Accept-Language: en-US, en-GB Content-Language: en-GB X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: multipart/alternative; boundary="_000_69ACA3A4BA39B0499C1917FB0718BB254BE747BB86EMV04UKBRdoma_" MIME-Version: 1.0 Cc: tcpm@ietf.org Subject: [tcpm] Some ideas building on draft-ietf-tcpm-fastopen-01 X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Jul 2012 05:19:54 -0000 --_000_69ACA3A4BA39B0499C1917FB0718BB254BE747BB86EMV04UKBRdoma_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Yuchung, Jerry, Sivasankar, Arvind, I liked this enough to want to work on improving it. I have a couple of pro= posals that occurred to me, which I've finally made time to write up. (I'll send editorial nits from reviewing the draft later) =3D=3DProposal #1/ =3D=3D Fast Open Cookie Request MUST (SHOULD?) be on a FIN I believe it would be better for the reasons below if: * the client put a fast open cookie request in its FIN, not its SYN (whethe= r or not the server has sent a FIN first). * Then the server can send the fastopen cookie on the FIN-ACK. * If the client times out waiting for a FIN-ACK, it just re-sends as usual. * If the server's FIN-ACK never arrives (perhaps because a middlebox has ea= ten it), the client just doesn't have a cookie for the next session and use= s a 3WHS as normal. Note, only the Request would have to be on a FIN. The Fast Open Cookie itse= lf would obviously still be on the SYN of subsequent connections. Then, whe= n a subsequent connection finishes, it can refresh the cookie on its FIN, r= eady for another connection later. This solves a couple of potential problems with a fastopen request-response= on the initial SYN exchange: a) A FIN-cookie sidesteps the risk of unpredictable delay of the very first= SYN due to some middleboxes dropping new TCP options (any timeouts due to = non-delivery don't hold up the session, which has already finished) b) Transports GETting typical Web pages could (ab)use the fastopen cookie o= n the SYN exchange by opening multiple TCP connections during the initial c= onnection without each one suffering the delay of a 3WHS. This creates a pe= rverse incentive to use multiple parallel connections, because you can get = the same latency benefit as HTTP pipelining [Yoshifumi posed this problem s= ome time ago]. If clients can only request a fastopen cookie on a FIN, the= y don't have this perverse incentive to not bother with pipelining. I have no evidence, but it seems intuitive that if the TCP option got throu= gh on the first FIN, it might be more likely to get through on the next SYN= , altho I admit options on SYNs are looked at more closely by middleboxes. =3D=3DProposal #2/ =3D=3D Fastopen cookies (optionally) authenticate both the client's network addres= s and the its knowledge of the TCP timestamp option on the segment carrying= the server's cookie. I would like to propose that if the server TCP-timestamps the segment carry= ing its cookie, the client MUST use a timestamp option on the segment that = starts the subsequent fast open, and in its timestamp echo field it must ec= ho the timestamp sent with the earlier cookie. Then: i) the server can use the timestamp as part of the material for generating = and validating the cookie and ii) the server can set an expiry_duration, and check whether the echoed tim= estamp is older than time(now) - expiry_duration. Reasoning: The draft currently says: " 3. The cookie expires after a certain amount of time. The reason is detailed in the "Security Consideration" section. This can be done by either periodically changing the server key used to generate cookies or including a timestamp in the cookie. " Let's take each in turn: * Periodically changing the server key: Let's imagine the key changes every= 30 min. Then all cookies issued in the last 30 mins stop being useful, eve= n those just issued a second ago, or a minute ago, or 15 minutes ago... * Including a timestamp in the cookie. This is only useful if the timestamp= is in the clear within the cookie, otherwise the server cannot use it to k= ey the cookie. But a timestamp in the clear takes valuable bits away from t= he cookie, weakening its strength. This is unnecessary if there's already s= pace for a timestamp option on the segment (which is very common). Security Considerations mentions only two ways a client can collect valid c= ookies from other hosts: i) compromising other hosts, or ii) (legally) taking over the lease of an IP address from a DHCP server. But there is a much more problematic third way: iii) sitting behind the same NAT as a neighbour (esp. a carrier-grade NAT s= erving thousands of clients), where the NAT uses a common IP address for ma= ny clients. As it stands, the cookie is only dependent on the client's network address.= Binding a TCP timestamp option into the cookie as well adds a lot more ent= ropy. With only the IP address for entropy, a client behind a NAT (or an attacker= in control of a compromised host behind a NAT) can get its own (valid) coo= kie and use it in a brute-force spoof of numerous neighbouring clients - to= either SYN-flood the server or launch a reflection attack on its neighbour= s by fast-opening loads of new TCP connections requesting large data items = on behalf of its neighbours, bypassing the safety check of the 3WHS. In contrast, if a cookie depends on both the client's network address and s= erver's timestamp, a cookie requested by a client behind a NAT will only be= the same as a neighbours cookie served by the same server within the same = clock tick. This doesn't stop attackers getting valid cookies from compromised hosts, b= ut it does effectively prevent these more trivial attacks on neighbours. =3D=3DProposal #3/ =3D=3D Special cookie values (A minor point compared to the two above) 4.1.2 Server Cookie handling " Also note the server may want to use special cookie values, e.g., "0", for specific scenarios. For example, the server wants to notify the client the support of TFO, but chooses not to return a valid cookie for security or performance reasons upon receiving a TFO request. " I suggest the server just returns the cookie option with length 2 and no co= okie data, to show it supports the option, but chooses not to give a cookie= . Then the server doesn't have to run an extra 'if' test every time it gene= rates a cookie, just to check the cookie doesn't collide with a special val= ue. This would require the value of the cookie option request kind to be differ= ent from the cookie option kind. Otherwise a response meaning "cookie suppo= rted but empty" would be confused with a cookie request for the half-connec= tion in the opposite direction. =3D=3D Outstanding concern =3D=3D Similar to a concern of Joe Touch's and Richard Scheffeneger's: dividing up= TCP implementations into ones with idempotent semantics and ones without. = This confuses the simple applicability of TCP. We need to work on that - bu= t manyana. Bob [PS. Apologies if you already got this once - I've been 'experimenting' wit= h my email] --_000_69ACA3A4BA39B0499C1917FB0718BB254BE747BB86EMV04UKBRdoma_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
Yuchung= , Jerry, Sivasankar, Arvind,

I liked this enough to want to work on improving it. I have a couple of pro= posals that occurred to me, which I've finally made time to write up.

(I'll send editorial nits from reviewing the draft later)

=3D=3DProposal #1/ =3D=3D
Fast Open Cookie Request MUST (SHOULD?) be on a FIN

I believe it would be better for the reasons below if:
* the client put a fast open cookie request in its FIN, not its SYN (whethe= r or not the server has sent a FIN first).
* Then the server can send the fastopen cookie on the FIN-ACK.
* If the client times out waiting for a FIN-ACK, it just re-sends as usual.=
* If the server's FIN-ACK never arrives (perhaps because a middlebox has ea= ten it), the client just doesn't have a cookie for the next session and use= s a 3WHS as normal.

Note, only the Request would have to be on a FIN. The Fast Open Cookie itse= lf would obviously still be on the SYN of subsequent connections. Then, whe= n a subsequent connection finishes, it can refresh the cookie on its FIN, r= eady for another connection later.

This solves a couple of potential problems with a fastopen request-response= on the initial SYN exchange:
a) A FIN-cookie sidesteps the risk of unpredictable delay of the very first= SYN due to some middleboxes dropping new TCP options (any timeouts due to = non-delivery don't hold up the session, which has already finished)
b) Transports GETting typical Web pages could (ab)use the fastopen cookie o= n the SYN exchange by opening multiple TCP connections during the initial c= onnection without each one suffering the delay of a 3WHS. This creates a pe= rverse incentive to use multiple parallel connections, because you can get the same latency benefit as HTTP= pipelining [Yoshifumi posed this problem some time ago].  If clients = can only request a fastopen cookie on a FIN, they don't have this perverse = incentive to not bother with pipelining.

I have no evidence, but it seems intuitive that if the TCP option got throu= gh on the first FIN, it might be more likely to get through on the next SYN= , altho I admit options on SYNs are looked at more closely by middleboxes.<= br>

=3D=3DProposal #2/ =3D=3D
Fastopen cookies (optionally) authenticate both the client's network addres= s and the its knowledge of the TCP timestamp option on the segment carrying= the server's cookie.

I would like to propose that if the server TCP-timestamps the segment carry= ing its cookie, the client MUST use a timestamp option on the segment that = starts the subsequent fast open, and in its timestamp echo field it must ec= ho the timestamp sent with the earlier cookie. Then:
i) the server can use the timestamp as part of the material for generating = and validating the cookie and
ii) the server can set an expiry_duration, and check whether the echoed tim= estamp is older than time(now) - expiry_duration.

Reasoning:
The draft currently says:
"   3. The cookie expires after a certain amount of time. Th= e reason is
       detailed in the "Security Conside= ration" section. This can be
       done by either periodically changing t= he server key used to
       generate cookies or including a timest= amp in the cookie.
"

Let's take each in turn:
* Periodically changing the server key: Let's imagine the key changes every= 30 min. Then all cookies issued in the last 30 mins stop being useful, eve= n those just issued a second ago, or a minute ago, or 15 minutes ago...
* Including a timestamp in the cookie. This is only useful if the timestamp= is in the clear within the cookie, otherwise the server cannot use it to k= ey the cookie. But a timestamp in the clear takes valuable bits away from t= he cookie, weakening its strength. This is unnecessary if there's already space for a timestamp option on the= segment (which is very common).

Security Considerations mentions only two ways a client can collect valid c= ookies from other hosts:
i) compromising other hosts, or
ii) (legally) taking over the lease of an IP address from a DHCP server. But there is a much more problematic third way:
iii) sitting behind the same NAT as a neighbour (esp. a carrier-grade NAT s= erving thousands of clients), where the NAT uses a common IP address for ma= ny clients.

As it stands, the cookie is only dependent on the client's network address.= Binding a TCP timestamp option into the cookie as well adds a lot more ent= ropy.

With only the IP address for entropy, a client behind a NAT (or an attacker= in control of a compromised host behind a NAT) can get its own (valid) coo= kie and use it in a brute-force spoof of numerous neighbouring clients - to= either SYN-flood the server or launch a reflection attack on its neighbours by fast-opening loads of new = TCP connections requesting large data items on behalf of its neighbours, by= passing the safety check of the 3WHS.

In contrast, if a cookie depends on both the client's network address and s= erver's timestamp, a cookie requested by a client behind a NAT will only be= the same as a neighbours cookie served by the same server within the same = clock tick.

This doesn't stop attackers getting valid cookies from compromised hosts, b= ut it does effectively prevent these more trivial attacks on neighbours.

=3D=3DProposal #3/ =3D=3D
Special cookie values

(A minor point compared to the two above)

4.1.2 Server Cookie handling

"  Also note the server may want to use special cookie values, e.= g.,
   "0", for specific scenarios. For example, the server= wants to notify
   the client the support of TFO, but chooses not to return a val= id
   cookie for security or performance reasons upon receiving a TF= O
   request.
"

I suggest the server just returns the cookie option with length 2 and no co= okie data, to show it supports the option, but chooses not to give a cookie= . Then the server doesn't have to run an extra 'if' test every time it gene= rates a cookie, just to check the cookie doesn't collide with a special value.

This would require the value of the cookie option request kind to be differ= ent from the cookie option kind. Otherwise a response meaning "cookie = supported but empty" would be confused with a cookie request for the h= alf-connection in the opposite direction.


=3D=3D Outstanding concern =3D=3D

Similar to a concern of Joe Touch's and Richard Scheffeneger's: dividing up= TCP implementations into ones with idempotent semantics and ones without. = This confuses the simple applicability of TCP. We need to work on that - bu= t manyana.



Bob

 

[PS. Apologies if you already got this once - I've been 'experimenting' wi= th my email]


--_000_69ACA3A4BA39B0499C1917FB0718BB254BE747BB86EMV04UKBRdoma_-- From olivier.bonaventure@uclouvain.be Fri Jul 27 03:17:10 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 657E821F863C for ; Fri, 27 Jul 2012 03:17:10 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -5.999 X-Spam-Level: X-Spam-Status: No, score=-5.999 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, J_CHICKENPOX_33=0.6, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HkumjBLNQ1NC for ; Fri, 27 Jul 2012 03:17:09 -0700 (PDT) Received: from smtp5.sgsi.ucl.ac.be (smtp.sgsi.ucl.ac.be [130.104.5.67]) by ietfa.amsl.com (Postfix) with ESMTP id AEBA921F85FF for ; Fri, 27 Jul 2012 03:17:09 -0700 (PDT) Received: from mbpobo.dhcp.info.ucl.ac.be (haproxy2.sipr.ucl.ac.be [130.104.5.120]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: obonaventure@smtp5.sgsi.ucl.ac.be) by smtp5.sgsi.ucl.ac.be (Postfix) with ESMTPSA id 29A6411EB7D; Fri, 27 Jul 2012 12:17:04 +0200 (CEST) X-DKIM: Sendmail DKIM Filter v2.8.3 smtp5.sgsi.ucl.ac.be 29A6411EB7D DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=uclouvain.be; s=selucl; t=1343384224; bh=zLXxdx9dtoTkSZjKn2vLi2r0RUVOxKWX2ALDnHoiA8s=; h=Message-ID:Date:From:Reply-To:MIME-Version:To:Subject: Content-Type:Content-Transfer-Encoding; b=UfSTXrfHzDG2zUfJDflZYD4dMa5Vhz2d9OaZ3CbfnFv5aPHoqYhxp8lt9Mk9THF0y fOpn/RzsdoubLmt/aM34j+ivXYSXlhxfXqYTRn2MrWaUqDAhU4bOr0B67fvtVmbCTM HnwSD1G3ev21D55Zx7xouMQpy6UN3vK3zcphHTXw= Message-ID: <50126AA0.4050506@uclouvain.be> Date: Fri, 27 Jul 2012 12:17:04 +0200 From: Olivier Bonaventure User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:14.0) Gecko/20120713 Thunderbird/14.0 MIME-Version: 1.0 To: rs@netapp.com, tcpm@ietf.org X-Enigmail-Version: 1.4.3 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Virus-Scanned: clamav-milter 0.97.3-exp at smtp-5.sipr-dc.ucl.ac.be X-Virus-Status: Clean X-Sgsi-Spamcheck: SASL authenticated, X-SGSI-MailScanner-ID: 29A6411EB7D.A1E70 X-SGSI-MailScanner: Found to be clean X-SGSI-From: olivier.bonaventure@uclouvain.be X-SGSI-Spam-Status: No Subject: [tcpm] RFC1323 revision X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list Reply-To: Olivier.Bonaventure@uclouvain.be List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Jul 2012 10:17:10 -0000 Richard, I won't be at IETF next week and I saw that you were responsible for a new version of RFC1323. Since the first publication of RFC1323, the Internet has changed and a revision of this RFC should at least detail one problem that occurs in practice but was not anticipated by the designers of TCP or the authors of RFC1323. RFC1323 assumes that if the client receives the WScale option in the SYN+ACK, then the window scale can be safely used during the entire connection. This is unfortunately not true since a stateful firewall can (and some still do - http://www.cisco.com/application/pdf/paws/71602/pix-sa-tcp-scaling-ts.pdf ) accept the WSCALE option in the SYN and SYN+ACK segments while not understanding them and they discard the packets that appear to be outside the window. I think that the RFC should at least document this problem and encourage firewall developpers to either : - remove WSCALE option in the SYN segment if they don't support RFC1323 - support RFC1323 We would expect that all firewalls support RFC1323, almost 20 years after the initial publication, but this is probably still not the case everywhere... Note that the presence of middleboxes causes other types of failure scenarios that can affect RFC1323 implementations. For example, consider a firewall that removes the WSCALE option from the SYN+ACK segment but not from the SYN segment (e.g. because the path is assymetrical and the packets go through different firewalls). In this case, the server will be using WSCALE while the client will not be using WSCALE. This is another failure situation with middleboxes that should at least be mentionned. I'll be on holidays during the next two weeks, but if needed, I'd be ready to contribute text on these issues. Best regards, Olivier Bonaventure -- INL, ICTEAM, UCLouvain, Belgium, http://inl.info.ucl.ac.be From tsabatini@hotmail.com Fri Jul 27 10:50:13 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 341A911E80A1 for ; Fri, 27 Jul 2012 10:50:13 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.412 X-Spam-Level: X-Spam-Status: No, score=-2.412 tagged_above=-999 required=5 tests=[AWL=0.186, BAYES_00=-2.599, HTML_MESSAGE=0.001] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id EuCAoX0ZmkGe for ; Fri, 27 Jul 2012 10:50:11 -0700 (PDT) Received: from bay0-omc1-s4.bay0.hotmail.com (bay0-omc1-s4.bay0.hotmail.com [65.54.190.15]) by ietfa.amsl.com (Postfix) with ESMTP id 1A70E21F864E for ; Fri, 27 Jul 2012 10:50:11 -0700 (PDT) Received: from BAY158-W21 ([65.54.190.61]) by bay0-omc1-s4.bay0.hotmail.com with Microsoft SMTPSVC(6.0.3790.4675); Fri, 27 Jul 2012 10:50:05 -0700 Message-ID: Content-Type: multipart/mixed; boundary="_8d3e7d6a-7987-4a4f-bfeb-b6841a0add5b_" X-Originating-IP: [74.64.96.39] From: Anthony Sabatini To: tcpm Michael Scharf , tcpm Date: Fri, 27 Jul 2012 13:50:05 -0400 Importance: Normal In-Reply-To: <2A886F9088894347A3BE0CC5B7A85F3E891A9BBC27@FRMRSSXCHMBSE3.dc-m.alcatel-lucent.com> References: <2A886F9088894347A3BE0CC5B7A85F3E891A9BBC27@FRMRSSXCHMBSE3.dc-m.alcatel-lucent.com> MIME-Version: 1.0 X-OriginalArrivalTime: 27 Jul 2012 17:50:05.0932 (UTC) FILETIME=[419F3AC0:01CD6C20] Cc: tcpm-chairs@tools.ietf.org Subject: Re: [tcpm] IETF 84 meeting: Slides please X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Jul 2012 17:50:13 -0000 --_8d3e7d6a-7987-4a4f-bfeb-b6841a0add5b_ Content-Type: multipart/alternative; boundary="_767ade1d-accd-47e4-b865-162f693e4838_" --_767ade1d-accd-47e4-b865-162f693e4838_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Michael - Slides for my presentation "Improving and Extending SACK". Thank you for all of your help. Tony Anthony Sabatini 200 West 20th Street Apt. 1216 New York=2C NY 10011 Phone: (212) 867-7179 Mobile: (917) 224-8388 =20 > From: michael.scharf@alcatel-lucent.com > To: tcpm@ietf.org > Date: Mon=2C 23 Jul 2012 21:10:39 +0200 > CC: tcpm-chairs@tools.ietf.org > Subject: [tcpm] IETF 84 meeting: Slides please >=20 > Hi all=2C >=20 > The current TCPM agenda is available at: https://datatracker.ietf.org/mee= ting/84/agenda/tcpm/ >=20 > The agenda is pretty full=2C i. e.=2C there is a high risk that the last = items will have to be moved to the mailing list. >=20 > Could the speakers please send the slides to the chairs? >=20 > Since we are scheduled on Monday morning=2C the deadline for submitting t= he slides to the chairs is Sunday noon. >=20 > PDF format would be preferred. Also=2C please put numbers on the slides f= or remote participants. >=20 > Thanks >=20 > Michael (TCPM co-chair) > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm = --_767ade1d-accd-47e4-b865-162f693e4838_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
Michael -

Slides for my presentation "Improving and Extending SACK".=

Thank you for all of your help.

Tony

Anthony Sabatini=
200 =3BWest 20th Street
Apt. 1216
New York=2C NY 10011
Phone: (212) 867-7179
Mobile: (917) 224-8388

 =3B

>=3B From: michael.scharf@alc= atel-lucent.com
>=3B To: tcpm@ietf.org
>=3B Date: Mon=2C 23 Jul 2= 012 21:10:39 +0200
>=3B CC: tcpm-chairs@tools.ietf.org
>=3B Subje= ct: [tcpm] IETF 84 meeting: Slides please
>=3B
>=3B Hi all=2C>=3B
>=3B The current TCPM agenda is available at: https://datatra= cker.ietf.org/meeting/84/agenda/tcpm/
>=3B
>=3B The agenda is pr= etty full=2C i. e.=2C there is a high risk that the last items will have to= be moved to the mailing list.
>=3B
>=3B Could the speakers plea= se send the slides to the chairs?
>=3B
>=3B Since we are schedul= ed on Monday morning=2C the deadline for submitting the slides to the chair= s is Sunday noon.
>=3B
>=3B PDF format would be preferred. Also= =2C please put numbers on the slides for remote participants.
>=3B >=3B Thanks
>=3B
>=3B Michael (TCPM co-chair)
>=3B _____= __________________________________________
>=3B tcpm mailing list
&= gt=3B tcpm@ietf.org
>=3B https://www.ietf.org/mailman/listinfo/tcpm
= --_767ade1d-accd-47e4-b865-162f693e4838_-- --_8d3e7d6a-7987-4a4f-bfeb-b6841a0add5b_ Content-Type: application/pdf Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="Improving and Enhancing SACK.pdf" JVBERi0xLjMKJcfsj6IKNSAwIG9iago8PC9MZW5ndGggNiAwIFIvRmlsdGVyIC9GbGF0ZURlY29k ZT4+CnN0cmVhbQp4nJVUXW9cNRAVpZDtbdWkLZTPgoEXu9Iaezwej1+REBJvrVbigfLUL4TYSgn/ X+LM3W2uLyUoZJPV5N4Z+8w5Z+bcpZjJJfu8DZ7vp++fNvf6r+l8Kik5+9vPkdjXnxPVuo4P4e/T L4/dmyk7+1y8vqzlaqUJXxcvp1ePpyfTfJ17+tMxQO75pLHYz/xgjJ/v3Q87AFLHHBO73StUZbfF b3LUJBZxOWtUt9tPv/r3Qo7Uci/+RkiRDSX598MWcede2N8MWSKl1vwHgWNW6t1/aO+l1qrNn6BM kop2vxnKbs0pKpWyf/YmUBROXf1te1wlZZJV+p1jTKKWjn+SFs2d/bMLXFCZtKwK7g7xvxwqJOxP rTIpFT2i7KXyf6Ecrj0LLeYqVf290GMuVMjftwStosh+EGrusZb82+7n6ccdRDpftN2vJGdN6/j/ yn8dsQu9I3ZpNYojrWbUWex7EFsLzzQpwIt0/xGeCXwBRjOOkCQQGn02lgziNgFttoaSj2eWWRQG Ma4olo7wLGzhoJTrQYbSRbn4h5eH3QaRIL4NN53YS2qw1OYy7SQQdchCxudW5uHaskAPt3sB6J+E XIADFhrR3YSsBZ7EATfsqFYh06cHO2jjNiZ8dnnrPzoBPykxm7zoqTX2n4eCsla7/2KWPOfeyty0 wAq1EjI6MlJeXbbcsIYABhPXtqaCokpeZw5YbCxTFh0YOp3plc45+y8DVYrMi/twCOQQ2OKdwPbG 1W+vuUzMQqO5MPyu9Xg01qOwLTFjxMV/tYToBwI2JvJfm0tyJcycC1ueZ1FBmDWfmuYCUbAjijQa y4Zzv1nCb5fwUahJ4GkxImAu227JHIOk3R/TFmRFYtcM6IvjuuuYWlt3hqhlVVt3bxFBLAUMmm0G QFWoYt3BuFgv5tIF8GapugWXtVQycgG+HNAf5qF3pPCYfGcJkZyPyVhzwAM4rY7Jd5fwiuNOwZtW rKrrgaPjfWfYaoyxFiwFdJyt5fuhYJKNnAfzjhQDY67H6HXCKvwOnFCJyrYVAbeo6bWBa2cvwdbb GpPFI/CBSbj5kLuWmw6oTEM1g3Hkg1xnc0vUCvOq/4cjLZgu3JnH26+gYrVk5rF5Mv0NpQh6omVu ZHN0cmVhbQplbmRvYmoKNiAwIG9iago4NDIKZW5kb2JqCjEzIDAgb2JqCjw8L0xlbmd0aCAxNCAw IFIvRmlsdGVyIC9GbGF0ZURlY29kZT4+CnN0cmVhbQp4nOVX3W8cRQwXCR/JEqWhQERBheWj7W7g hrHny/OKhJB4ozqJh7ZPhRYhrlLC/y/x8+zt7ezlLgSQeGnTptasx2P/bP88c9laQ9xa/RmF56vm 28epfflnc9k4a1v9typS1F9/NBzCXB7E35qfL9pXDbX6c/Vys9cH3Wrx6+rX5sVF81NTjmsf/7AW oHvZiHH6pyzU8vNV+90SDknrvbG+Xb7ALmoX+Iv/KRkXW4rOcLtcNU+6j/uFNd655Lg7LXL22fnu o35BxqZsqXv6qg/YFVLqDiuNN3uKhi1W31JdjjF259X3e7010WYXZsYOsOoVC1a70LbihNh3X/UL ZngMsT7lXZVDtMSxu4O9wQo76R70jg37GGDbG5dskO4DVSWRDNXKdp65Mvn6sEghcYboMhki92z5 I6Cj0ML6gN1CfLtIgrPa5S9N90a//L2Au1HgBNDx6Qm+IUiK0h31YoKLMcMlYiPRRYVA45DkE5Cz xpGwjd2jngUxM28pEJkQ/Hz1SO0jZAIQWEXQjueGEbUY8SEBgK6sE9sQund0JyekCg6ZyPh+Ohiz A1TRkAsBKekVlOQijp42PYIk1ueMNeaM1PAOoADEwkdg/joAdVFWRbSerEk+EoryfKM62bm3kbbO X7BxGV3wtS5zYJeqo2qrQzSe3czCTSV7+0x8U1oGPUeETksGPZL2JaPQAFm4AlgTyI/gUmlPl6N4 h7b1JmVy6hzAszY63UQGiBGPMaFVeczQGtQ1Ege6mILjW4IqqKA9oB5uNt2Eqa7uxfT7JYj3Enzt QcZRGb2IKfiRxStRFQqhg8//nprrZE3cDC2UZJvyQM1D3ooXgx8srUgofkAE9/F4ei2ryu09kRsc IRunIRG9kmlpuKzVKFW5gB4c1mjGE2Oqhg6MWSusStBZIV9LoUrQWEtO4jqBgfx2u69Pv56qAaQw jt965nqxc/mfzt/LKcurWe7F5Vr8P8pgnePVPPUCJ2r5tS6Dfwk8IM+8Bfwe3ryrrBZT5i1WuXmc zb4P0+U6/Q33qR1MOZtNa+v16dWwnFSn7ccD1NEORJmNiBW/a87NBsHk5jj6dpP/jNFHyp9GV+3o ruBxn8Nwoa1BXsuH2wfEHLp2s3jYB8YtNdN/moef6IFClJPr3u6zScQh77uSTKO9jq4G5eTamMFl 82RT09VEnWztyt0Dva3kmOO84OeBaX/i2kzOWa6yuTUhx9TsOvBhqSsXaXfiZ4o632kzcyUnv+sK hvOncHdfhSZb16xqYazhDJnlmtHtS9lxD6qC//oocIYSniwaSYRVqYr5pu4AaVWH424YTU4SNve8 lOOcf0aX8QzJCVfRvGEiRAgjEdRzTdCX3P6vt3ze4fU5468Qwoa8nnT3+wUgiB6xfTqJiA3lkjyu 0Z9pxIR7p0cXLXx5HwlypJmxSch1H/ZsQDKJ622V3c8n8YtJvN8HTAxkQYEA3vreLN2FL2i8BSaA PqaSOjo+BnJwOO+geJTwgtMX4OgRwBa4wSUFcChEDuhN4I5HoSZzcvho2nWMVsC1H+2rzrvB+6GY coaKr5VPJlE7ZK389Kq8hUhSqJVPJ3GPuTvATQKFWzrH6/POwDc+4WnavYd2B5cg5LuoY7YKzvvl TRzVGS1QjNrMKMsv9dWsc1qwC+5iaCblXaVCvX/jTRyMVbl2vEISLTDoztPNg1eaQ9EC88YP6Tor IXFy3s/iP69hQXfgTKpP3wPF7EG5HuB/AR6vTd5lbmRzdHJlYW0KZW5kb2JqCjE0IDAgb2JqCjEz MDgKZW5kb2JqCjIwIDAgb2JqCjw8L0xlbmd0aCAyMSAwIFIvRmlsdGVyIC9GbGF0ZURlY29kZT4+ CnN0cmVhbQp4nO1YW48URRSOssrSbFhchCABbW/YjZmiqk5dX02MifFFMokPrE8oGOOS7Pr/E79T Pd1VNds9O0j0CZbLobrq1Ll951LnrRRKt5J/RuLFWfP0mW9f/d2cNyRly3/OEuX4r78abW1ND+Qf zS9P2teNavnn4tV01lg+KvHXxe/NyyfNz026rn32w4bA3vMmCOJfaaGkX5y1360hUGiNEdK065c4 pdoVfuNf5QW5VmkrrG7XZ83z7kG/ksIQedLdrURHE8l09/uVEtJHqbrT173FMet9916vhPYqUveY v2vrdSzJdpED1mWgoLTpvupXWjvhTcTtKghHMXQnvCNYF7TqvuWTSkaXTmrhjMSO04teCmt0oO6A NyujJK486JUTWkK4D5IczrnqO9+tRZDBad5LwoBBIdtxDwmCdoTzRqigY+w+7JUUGlYpN4IpzBYl +NztjYVY5H9d/whbK9uS3hh7ZZwg3a68Efj/bw1Mtv4z+WPaoj38hE9sfBWFItx4AsOS81Gz8rjb avKD4ZWUxnTX8V1qDy0fg0J0KL/RjKLU3U0+RNEFQ+Vy3jucD5q695nyYHmNCeOk6z6ZmN9nPnCC NwN7B+ms1cVemNAK2MiyuSWc6BSF7uNewcbO1KvHvQdIlKZqNd+GG6C1tUqPDMhVWwtVTrskmtLS WgQqi2PAONvlOJnARaPU5juLu2wjDkHYgT1IBjq365+a9ZPnLIlw2idNlbAIuMoqg/ks1auH04VH E7VRBDsUcMGrVpnafaP2pWj5gtIS7cT2EPFqyQMbk6Slp8pD32BDkCbG8q77vRNRqxD5M4GpAqEN YO1idfoAcCPgAXdWN7E5tST2CFDiNcVK/isVzLE0UD6UMXF9NhaPKodGEYBnsxw0hW/ZpZlnNu58 RBOgH4VHgozbWBiROC6n1MMWQt5TC7AoTFhwmA+sEiyjGGRVtQcpCPkLCVghBfGycYGx8ZoNYSOW 52Nm8t92QmCbHE1g2kb/BjjT6QyrhZ070oQhGRccdlWaeNgjHQpjFPvz+zUK4nmjdTCtc1xpE+mt GatrQfKGVGhRZ68umWUazzWTJHGl91Gkijnk8yTFIIfXbQg2yQHSRrLj7SXNW/aXJOwQREknptq9 b/nYTh/OxezWHApVap5gNsbr5PboUAPLfFunwksXZLZbUTuKN6YSE65OJXkxK1VksjHjzReG+ewy iGzCVqIb9b83G/Y5akuR2ymlZTm3wJITcQnsKVHsk0mK/JYjcewsz6qG0wRZ02/afJ5nKJ3VAAtU kv8D1kYgnW3hK4aKfoe1d1j7b7H272KKPAaNMqYOOKbgE/aEFzKSL0tkKXPlnSCCsbDbfN3Mrejj xArhMTLVrrLfm0TP1F8ifNA/+mDDKKAPNMurMH/ZFFPdymdxH4Ky0sq67c/kA7gFcWU8VY3I2M1u CzzT9V9qvFxUocTG4bKcOyaGIl7mxrS8ltnfwFjJLQ2ayBVPmEabYdgdxZ53/qa/yl3hpeyQLysV z25ddPAyUqLSewFlDMsM4Zw0cqSPQanmmtKq09uaqzE1r4zGVH7lVH07vQNIZStl5ofc0nYzSXpx GhiV/YiTIBntKk5fY9VIhzFjR0RdEigB6aiIESQKWKpWY2n+3AU/MTyHBEWwDTP1pCB5FIQqUBa0 O+m1w4XuaT5Rgn/HK0QIqIL7TwpzxWNnkFye6rIlbiIOZVSGUEAk3xdJcthfPYrtWxfSEF+Me0UE 7D/NbYfRZppL70mzw9zbz63DOTJxj2gZXTOXr8ogGM5EQ8te3vLUnq9Hbz0WQgRo4FCLLxH8eLr8 dc8XVXSEVUW31k696vPuEbtTOeC++zSTsEayhdbdZ+lZE+0i94grA/zEpBCbTnrG2l1+7AQUdXms 4Pt5Jr/I5KPeoqcAXNgQ8CY/8aZU2CnOkitkJE6bngXl/MiPuNGS4rcmlsgrtFDXskQHw6OsDmx4 CGSdtvwkKtDiK86OWeDDfOoG0OElIQRZeBqkH6pQjNhiys1HmeR432w+vWB5II635eZbmVxgdwy7 BavsnsLpzX23gRnjteM0Do0Vq3zSE2oHG+dOest2LAyHP7reiEGn+5JfrwngDzgFcSmwvw75iZxj CcG8skIyXQpeWPKQMZC+V+4uMBM4wDbvx6mcQSUNFJtK/3ulWZA4cacqb18wxXXcBLWjLjrafwB1 ybPCZW5kc3RyZWFtCmVuZG9iagoyMSAwIG9iagoxNjQ4CmVuZG9iagoyNSAwIG9iago8PC9MZW5n dGggMjYgMCBSL0ZpbHRlciAvRmxhdGVEZWNvZGU+PgpzdHJlYW0KeJztWFuPFEUUDggytEQQBQWD tppAN3GKOnWvVxNjYnyRTOLD4hMKxjgki/8/8TvV3VNVM927w+2BRBaWk65Tde7X01YKUq3knwl4 um0ePfbt83+b00ZL2fK/bYIc//qnUdbW8AD+1fz2sH3RUMs/L5/v7hrLVyV+vfyzefaw+bVJ5NrH P40AcE+bIDT/SR9K+Om2/WEDhkJrjJCm3TzDLWrX+Iv/yQvtWvJa4GTbnHQ3+7UUwbqgqGsZNtFE bbpLDJMhqSJgAn7Q3Rf9moT0UVK36r1wPhrbfZYwQ4jKdU9e9BYErPfdDSCQdTZ0n/RRkJYxVsQ+ ZThK48nztbUSAS/bTvVS+EhOhe4KQCejtqa7CNCwCtX4NUilQZqATFJ3HzE7ZMBCyU7xwGVmXTnn wHmW8n6vIwki/fvmZ+iMbKvVqLR1NEK1a29YU3803YV+83dS6w5DeagbRyfd9z0F4ZRRrCEpIKA3 vrvOGlAEPi/3uvsQWvCkbKxwoGaoVnmi7lb6bBwkY4WQEjaShQxQufLQ6H1ANo5qhr50BCZUAMre FXgXGfJWqz1KUmgKSlbXbzOucdFDg8DV0QWjEyU4ICxTvnBtwK0fKK+tdgg1qoP5rT3g1UazyCtF 3Deh4KR8KN9qEyEfEiZcSDvKZMort3cclYSAoZTDW4YdQBt8bze/NJuHJ5UOtCAPp65etoIkWORv oOu0YxeDNxpHOpQaetL1TkQfEAklxiij1RPnpYLY/gJ8kyq8aMlCJnphbZxxYXjo2iihzvfgC6we cqGwYSbR7r494EwgDTGAeDWI6ex31xN/Lho4c/5aqnvVB2G1JztK6lxMmooiBBk46RBc3O7F0QeF 6Q4skQntfQqu1vdqzntLhCxSof0bsJ0KkUzKYlqZwFkMp8h9inMYHnURqbF4NLM0yMtSPugV0okH VPvALiazS3Mqk9G62diuJbIK2TYSW/7HDerEaaO0Ca1zXIAS6K2Zik4BMkKqPyg/51eS0qVyKdFI vXg/IkmikAzOlZgY2ECVCcEmNgAaZ+JEvIQZ5XhGwjIfJN3AyEn3ecr0Ek6W00gZOleHcwdDz2ef bLUxts10HB2sUuW2KQvMZ8yzSPmgdU1qsHWd8Way7HyQFLwsygIC1ofCm+afurM73ytJRaqsLZ1M Opoar+36nBIezf4WbG2tR74YjF1KcJAb5u1fZrii+meJpg5tWzVuJsgaftUm7jTH3raKSGd8Cb77 4Jwib1sHJJy7gv8Pzvc+OHfm3dYByeYt4fcmOF8zHCx3LXvxsNAJaa7uGFcO+rnJgncWivHEeOFX hZOIYZ4KpHlagGCYPu4Ck1tO9BR3eb4hrYH5KGMW949qRLysaL6h/idXz9GxP4sEXQbXUpcy03fN NOslP0UDNPbWC4F7qzdWCaf9GzW/55p8PhTPm+/S5JzGu4AJdXa8o/M70Hl3W5h2oF4MvvFdWj6n qrOG5+OV/2XaCRBFr4tBY1+qo5r1vfHKChWjnUqCpapQ1OPw9MI7qhRnz3LtjOb3qkfJ3xB2eSjK 1wuZ5mNxn5LaTV/K+beYrl/bGXj5EzV6J9qfdhe8vVqBHOnYTo9mRGRXfu14rxQXsk690pgm1gOL JX94tS5mYL4oEPnRvVowLTMOcoKPzhfXZp0vG39VeMxyYtPSnrm4mrt57WAkxRGM5FClDwDeZy6f HrnkJFWXemvtrs6fdPc4/5IzEPWrDEIpqVdSqvuaLUUWszJvP03aFgbol0WSnvV+q+cC47wqrxXv fpPBbzN4r7doeuGyrAjYl7euycc7w+6/NtAe4sEzo9P+JVoUITgMc+QpBMT4jqNLacOoVdppgCFU RwvLwMxOEod7ZniVb13lXarUMB4zrwfuh+CIESimRL6WQfaaEfnJS+YH7HhbIn+cwYXnrkNvwZI9 kjk10rsBbzMeTSn6HEhMLPJNpCMlWTlpaYwa6tOdNWkRlQ3dd9CJ0iIYXjaDXR3YXiveRbMv8U7Y CslwyXihyRVXi3RembvoqQM72LgLPgGXLJLyGimklP92qRbkItCkkvqCKq6AEsSOquh1/wM86Z7b ZW5kc3RyZWFtCmVuZG9iagoyNiAwIG9iagoxNTU3CmVuZG9iagozMCAwIG9iago8PC9MZW5ndGgg MzEgMCBSL0ZpbHRlciAvRmxhdGVEZWNvZGU+PgpzdHJlYW0KeJztWNtuHEUQFXFC7MFKQhIigggM QZgZi226qu+vSAgJ8UK0Eg92ngIJQmwkh/+XODUzu11j7zp2hHgBX0vbt1OnTlVfzlpriFsr32vj xar59llqX/3VnDXO2lZ+V4MV5c+fDYcwt0fz9+aX4/Z1Q618v3m1GeuDDLX48+a35uVx83MzLNc+ +2Ey0PesycbJ1/CBtl+s2u+WAJRb74317fIlRlG7wA/+UzIutiUMLavmpLvXL8hYSpm6tl9Y44sv zncH/SKbHG0hmMn4nHLqPh76pmKpO+qLiZ7ZdQ97siaRdZvmmLpDNdXp6z4bW2xI3Xs9GU5UXHej R7tQwt3e1Jdj7m72FA3blLpbvTeUuZTufVmAvXN6fTV+/DRbLLUPqDEVH4AKk1LOhSOwogOHxEWw BBAQsEDFfWtoj4GTtGOczS5T8eAmGQox5O5DuEvOAs196ZBDzEzdgz5QMcHR8+WPYJxC63iifEHB ZN8uUgb0dvlrA9+Xfwxh2fThhHCh6QTTk8kODAgAMi4E4u4RLOujjfDQGUpALaCjiTYXsRgBctIK RACUfAJ/1jjQZmcznXaDr+x990FPEAvn3H00DPMxg0LVt857SzokH8nlgRc2rqDv14LVelBxWwBy ApePt1gyJII0TAAiEdVcyA9EOvZZiMSSCBaDRglgnK1Sna8zakB3xXaxeKLuK/Tw1vui/TiChRyi tGOCh9LOKaqOa8QhsPJtf4NkgpezSzv82C0HxHrhI5T1Vi180kcvCpRgYjr4lRE18bbE7J3CU5Hf kM9ScBp3bZ2s6FTrREWy8HXu4ZoBTVaNuQpR5XUbWWNQoDk9pHY83BJfLWSN5KqaET9DQR6dl+5a E3UpTejRoHgHmW40oSbVQrszgE55V0AOBpmjvMxcOdT0MkMEyYtCXDYI/U/N8vjkGglysTpcmiC5 BD/FInjSE2iwqrxsSYqLKaXduytFkpnmrFdSZu6rKjMudF2NO18u0zilixo/h+fTngMb74cs/X6J 3fWsYZ9iG6Ns24OZgl9v1cqUDsOujU377fuvTv+6ATvEn7BAMSz771gJBhQjjujbnMOAAya2JLde XdvS5epI8iVAyMYRyY5NaG/DGyKF9AqeZ9HXdq1Ce9i8U6HsaqDHsMQcsJ8X7LscyvmEX6+pV7qg zAuCCYXzuxfFKwhGFYN11ZQjztaaqEW8a1Ocuu5O+ZLCLOVVzahKWR8jV7PTpc92bl/3pHlWpb6a JUBKQZv/Qi6shb6a6x/Hv5n9fy7813Ph3WKOU1aYxfx6R07cjkos53bKYMgiHorGrcxMsyLapNYa N8oAErQWLj+d7CYOp9R/6GAdHesJ6rlBDxsh4zh5/vCwpqTdrLsP7oKLsVz5KjHxEZFF8+PEghwo iXIpYY+bJebUoOa3oR1Hr6dyJyXIGCcDpJoNNndPNp+FjRXHmyHWxn3wG0lLci7guL4gUVKGVZJk +rYjBuCDvAhdXjDkVr+79YpXfeK5ukMIm7p6AncWqB0Roew+qyYoGQhh7j4XoVBgJ3FaeESvgL7H A+02ZXK4LbLBpTCxHqbm/aKaT6v5pA/IL4hFiIB25e1huOaAV9T6BUQm1+MkQOXuI+8DBRcoeR8Q RAk3eHkfWCO6KbdX6DFLmAEoRJYaCrlFS1JXK+D9OuoAMkjWocwKeDeiH8tTKejidefDakoZnjqf vhE8gINypDrfqeaO6e6CtxwoXBEcT+vdQzb7xDEir+Exicv3sQGwFXIeDC8mUcBMSVAYovwSnLAz 2ct7BeC6LPHalxcP0ZK8iQRjxdbAFZOo82PfebhV4mQRmDd+DNe9wSXsxcPDQvX/kaYFOxTWJL36 DipuYyW4XVhV978B/Dqw9GVuZHN0cmVhbQplbmRvYmoKMzEgMCBvYmoKMTQwNQplbmRvYmoKMzUg MCBvYmoKPDwvTGVuZ3RoIDM2IDAgUi9GaWx0ZXIgL0ZsYXRlRGVjb2RlPj4Kc3RyZWFtCnic7Vjf jxRFEA5wwDEQDvAkaoKOGszMxW27uqt/vZoYE+OLZBMfDp9QMMYjufP/T/y6Z3e6ep1lgQfjgxwc lZnu6qrvq189l71WZHqdf7bCi4vum2ehf/VXd9lZrfv876JIPv/6szPOtfIk/t79fNa/7qjPP1ev 5r3s8laNX1e/dS/Pup+6clz/7PuNgLWXXVQ2/ykPpPziov92DYNiz6w09+uX2EX9Cn/xPwVlfU8m KMari+58eH41auXYRDscjyutOHGyPNwX8t0sO6/J+GbNSd6po7Fx+GhckdIhaRruiRXPX48OB7oQ hgdjUOS8i8PDMSmyOqXhUV4anSfnhg+ynDQHCnnbyqiokwnDlxCNUwT7YCok7f1wayStDFsrDz7G CT4kdsNp1kUxJhicdWmlo42UeAiTK4l5OCqLmLRJw2144nWyjvepdjjZJvpl/QOwJddbswF3xRHL +1Uo/61/7YZr4/qPQsC8xgQQg1fnQAE6jSaXtWdLYuAA9TiIvfYAbyvBblLWOTLDJ2PSysKMm3lP YE9A/OlogQqxmTXBj0mRI95AaJM2cA4PTQAHX0FCYM0Il9eFXZt8ZCuOr9JGfYjWtuZtFZyOBLK8 9Y19YoF0VZ52UmSQQYR4W3D8en4YLKdGRfWiSlLtndErhLNvMV5l5NmYEOUJFZvTrMsE34JkjIcF nGm3UYHMH7v12Tlih0g5V8C3igIivCroZ1eOEFaWotFeUFxygrRmbqz+eLakSjdHBP0Ad1IIfg99 9eGhiJJECssEGgeZfIqlrL014oCNkVk3GY1M3ii0iMfmeaX4LjIV2RZSBa2Np+nZ/V3TfaK4w8ou GxUCGV3yeaX8FgpRIOPa8NqBxqO2et7C7O3GqpxjEqRKeqWvzZakYtSRGxiIlYvMG0usay0VzCyz vMy42CYDbDck2Nhil0cxds68H1dyXTnKaeK9KfTdGs3ssjNAtI/RoUsW0UaurVHIeUlpk+iSb9Pw 5npbG55FMY89aVSEqd0tp9xyBC1Xxg/LCvYR7otcfrwbrLkeb8TkvVkm6I2FBC9TJiou8ZhkPZgK JVKuLWger2MSaS29uyd7WmGmcLOdTy6asYWjbuV3HWEuK6UXLdPAppH/Z/3fZ/0QynLgETA7j5rW hzTB/J8bfCR+tT4vN7mK5VTwMIdWSiq8J3mKNYTSeah37wTMXP6Xg67GUT8X59sLWitM7TCwLeLS pRsjq5DIbmMrsdQpHV7y5D2HnzdGb2P2bvTiHtG0mBwEOKNa1MyxoNB6anvSgU62EHgpLA/KS63u wBj9TiOht0EG/94E3blqIJ1WDE7iwXvG1yNFHM1mz5ByIPEEnfMwVP2/MW3BtLJF3bNdypZK8fL8 LklbKMEumShWL+XDTpZt7T+UpnI2XJrcphpAKeytogv3oCbsFmJpisQQ4x5OZNDtTLhyw/7xeGF8 3fapCohUdTwTuZtRGLWlpmqkRE5cmRCvqMl2biowB5h4dJF/CPkzxv63b/ltg0zbipxzcx86H56M K6Sgx01l+LSKgKOAaszwWe5G5IxluLbicveP4D/DqEMki8AyCqwHI7cJvZ9X8YsqPhkdhg7gmIEA NPljS8nIIeZkXQG1/JUgZENzml4DgslZnHe9WBQo5ro9W3RU0tiaEjIwyHnj8hcK5b2mnGnV4OO6 6w6IC9riZpONt5P1UyakhCUsF9+r4nTdKYufX2V7YE5wcvH9Ku5RdwLcoiP3lsaZzXkPcAHigLAb Ho7wmLLLj1BHjc7glG9DSPJQ9qzIqmRcnD4NWRU5f1OCuTZmvo7zJ6ccS/kzkFM6y9JwgSTif1rb 0i2SL+YAY8XzEAOXTLDbu/vG/8cSFrQ/nEny9D1Q3MZJcDsZMYv9DazZ2illbmRzdHJlYW0KZW5k b2JqCjM2IDAgb2JqCjE0MTcKZW5kb2JqCjQwIDAgb2JqCjw8L0xlbmd0aCA0MSAwIFIvRmlsdGVy IC9GbGF0ZURlY29kZT4+CnN0cmVhbQp4nO1XUW8cNRAWLZB0idrQQkQfCguIajfSGntsj8evSAgJ 8UJ1Eg8pT4UWIa5S0v8v9Rvf3tmb3NEgHiGXXEbesf3NN9/Ys5e9NY56q5+t8WLdffss9a/edJed t7bXv3WxWL/+6ijGpb0x/+h+Oe9fd67Xz9Wr3dwQdarF19Xv3cvz7ueubNc/+2E24HvZifH6UwZa +8W6/24FQNKHYGzoVy8xy/UTfvHfJeO5p8QmSL9adxfDw3GyRiILueGDcXKGmHm4M1oTFDINd9Uh 5JB9GD4ak2EXiIdPdNSJZNhHcGabfQzDWePcDD+uZjParvHpGCIZ9unX1Y9A72LvaYY/uZwMST8l MYSR37rhvXH1Z4lx50QJsePRBZ45Yx0LYoCRotcQYAW2PLw/umgo54hY/fDhmE1yFPPwmUIBCSmk 7byUtqOZmYbjUUz0zBmwnWFQODxVw1qXhuevYfoYHYEizPGZJXjdzZkYAy3Wh3M0ztoQ5qUkkrpm YAyyWOB4B7zuVYN591Yb0EG4gsbuExsHsKSDZIQ9a+atSYGdl+JBWNcug3ms+1ICLdVqXB8UV87B uaEvEJMIYGHQCW2oP4hwQSsWJWKkIKgWvBgk+KdudX4xnI4MIWQXho+ROU9BMvQL5jllGh6pei1L m42jHdQbaoiuCaTNdPU8CHRmTcsBUk4N+M2GAhQtpW0efXbGOb9H59DwFBC4vFPjz680QslIbF36 /l5h1Odthv693mveEw5CR/7ArrMrijzjdNiOXquHm+m6V44iyzk3a7WU7pfTSfENQNOsuj8TTQxV BxVAdfwGTEgGKzMB1E7dkdOyf0MVbdGd7A1nKfrvVzj1LzsKuAREIq6TYkaSsL1DWltdyn2C6+Q2 N8NOUfVmQJHqleAsKmxzMeynvQ1py2/VRb+bc3ePNa+DVLn5YEiU3ZKlbESshH287tEsanRSoQQC ujYtszqiKt0blzimA6d9c0LtFzqSHyIGz8Z95bUpKiwboBMAseyv6+Pvj9hrJ8fyqJqBcnbS3BCL M6RIpYhl21msFw1HELu0/2nzcVk1tl5KLxEv7P9l+J+XIdywHCPTNwxtXw8/vWVPi5Z7oRdw26e8 1cqTcQLLjOiHz6uJKMvBSjR8oVlykdCd9uMUSiMqOP41PJvEefSh2oVyonZas+6X1fyqmk/GCMWK Z6UCNGqTXa5q4MAtPuHK1dY1KdBtj5qjx353CqKELlgb7S0ikCyAQaWLAKDIFCEcpIjR2iJHFfBx nXUPGkrWQ1sK3m/Qb2SSM1xC63xSTa2h2RmNBfAAToqt8/1qHljuAXiT6OItwdG83ymKIeDsYHR0 iNhpyA9RKmSVnEflbYIVjErVeZMpyvA1OCGPehDMmrQR0nwdo6suWtL3imis2i3whsljfQsozxfp bi5gUYEFEzbpOi0hUfIhLOI/a2lBVWBP1+5+gIoj7ISwMzWF8xZPp7XoZW5kc3RyZWFtCmVuZG9i ago0MSAwIG9iagoxMTAyCmVuZG9iago0NSAwIG9iago8PC9MZW5ndGggNDYgMCBSL0ZpbHRlciAv RmxhdGVEZWNvZGU+PgpzdHJlYW0KeJztWFuLHEUUJm5MdjshdxcVoq2SpDsyZdWpu74JIogvhgUf Nj5FExEnsPH/g9/p6e46PVOT3RURH5JNNoeuy7l951ZnrVaGWs0/E/Fi3Xz1LLav/mrOGqt1y//W AxX4158Neb+kN+Tvzc9P29eNafnnzav5rPN8VOPXm9+al0+bn5qBXfvs+5HA3rMmKct/hg+SfrFu vz2BQKl1TmnXnrzEKdOu8Bf/m6hsaCkYRVhaN6fdx/1KK2dttNTdGujssnXd+/0Km0IIXSu+HjEd UvBksDuqELPz3QP+alLKFLrnr3sPHj7G7sse7HKQd13vcVxn6113WD3+uLfZKGPsLyc/QAvjW0uj GiuTBnIV0yD9r013pT/5Y9B13kQRNsDSaff8TW+UTTkFsDJYDjpA5om6wUxtDslZsX57+BqyM6b7 kGkfUnSxu7nZ4Q3rtyIc1NTd6YOilIi6u31WllzK3T1mCrWou89K65Dkiet8DUXY5jEo+NhEuXwN 10Rj/ZL31X5llaMIkT4YPruQsBnnwMp7Q9hhvKKcPVtaq+iCsak7nrVqZwpsSSUbDIwOASiGmT9p K+QrJuH1oCCVJ2G+svNocK4OOYvlj+blUWSXo+WriILS0bF3bVJw2Y/NydNT2D0inAzZhQZFiKPB 1IDVwjILbUAFG/Z7zSsNC9FiAxgEltPC9NDCk7VStbpdi1TvMRUDpF5empS3IeRi47ehGohdOUiR LoJostDSpBEp5PN5nAVKbs5yX0UUWkTTwo8j+PRs4hwCbeFgxGkBr7SLYFVuXcBsgtF2GBid5fUy NBdxMl3/pCcIaE3Y3lrD9sT03Fs3mPXZVcPgESing6XFpQW39XyyYzQSwXhQoaRBhZse9NY65FLD APruBDXhrCEXQxsCF5uBjN5NBUaQvGGoNSg151cNicxSNiyC1oBBVsRFYwPSQYqNHMG1KflBDpAe lWLiLmnecnFJ0lsEMTpsJDlFjSFSZBCi0gEyjT/qrUfoOY58q0xEaVoAJKnkPMgn+Ja0QxpD2Gew dmm+06Yw5bmYx7D3djuXiOAbk9ENwAPVLoqALNgaXe2JthGzL062ysWhQA2SazcIY0h7L2E96pBC 3BPNNS6FKnCVyorjE19NFGuBf0vyFOm/4GdqidaLTsklvaQv2zWdlQBYL8IiRi/J/yBCJvivl1ER 85J+FyHvIqQWIf8MCcFBFomE0gAUIeBTi4+Ga2rCN6Kt3rPSKci2bO79pFcO5r5rNC3CLF4KcpNr ipUL+9yTU+jUJkFjsrbu/nrXJjZcuJ/dsIqaw0DYpNoTHVTxUOlL9axqQHSWlrnefiAKM3qyf6+X /N9NR8Us0nF7vTxh5LB6bGe+WcRxXSekNQwB3tHC3/XGcSvWgWaNhCiGrl0TxpSk62U2KpKVDefm BzlAbYu+Eezrygi4GG8qZ3ZzN+UxdfuY5EwjfDGpX4tXEVCHFSk2VcVnI77ta6t3xpudOXMajVdZ paSTW5p5O9652ZcAWFj6vKF9ynbFYCP+nNV5j3GFqSru3TtDCKezswtHeaIgZ4vjFCi1yUMqctw7 FbNJdjJmjGbPBFm7qpbApCO50Lu4b1av1I8ti1ymfky+k3Fck69YspYb9jYnOWzVFk8eDMt0xuNp xrTY7hL8jLd/9YJve4aWxd57Pzefp91DfjEyAeNq90khYZYBRcjIn3JIGk/W8eueG57kEgzExtIx GUYAKbgokjwm7v2skJ8X8mHv0W5gBmdDwF782DgUnO4brkUruI6f7iILylXoCqyZMchbOIIligYp 8qBIhCBKEIMSmxsC+UC+u8a+CNqwM4rAh+XUEQp01NbQ0OXZjfQbKOWMLU5uvllIRvq4GeUR8kCc 6OXmW4Xcc91t2C154y8oHI387qBEukghoFhCY8Mq3wOySbNx7g8vsIGFYWyjY8vkU/cFbMIPQmjD 77K4duxgGYrAEr+reqWZloILSx7y+9awvnC3yDaJAeaU27jrzqASRYv0JfU/lmbBYACeRnLfY4rr 4AS1M4lm92+qEm35ZW5kc3RyZWFtCmVuZG9iago0NiAwIG9iagoxNTMwCmVuZG9iago1MCAwIG9i ago8PC9MZW5ndGggNTEgMCBSL0ZpbHRlciAvRmxhdGVEZWNvZGU+PgpzdHJlYW0KeJztWltPXDcQ VlII5IC4pYQGCjikTY6h6/h+kdqXSlWlqi+JVupD6VPapKq6SND/L9Vzdvec8WLvbhpSVrQhoNHs 2P7m5vHYe0k4E5Jw+BkTbwbVy9eOvPuruqwU5wR+Bw1l4c+flTQmpYfk79VPp+SiEgR+rt61Y7WB oTz+ufqtentavaqa5cjr70dElL2sPFPwr2Fg+s2AfNuPgDwRhgVL+m/jKEF68T8nzjJJlBbMatIf VD/X+1QrZoNx9S4VzHAf6k9oj7MgjbCmXqE9xUzgRtabNDAlnFP1Gu1JJkIQ9ZPu4/MLKhXj1rr6 /Apm0JobY+qvqQjMOGHjBIK54IOqN9AK3WSriPuQamaUc6L+jOrgmDHhl/4PUSsjSbR61KknrYxr kJ40zJP+r1GXe5QzHpw1VtX3ER0VCixwqXS9hNjLiH7QiawkIi17Nc/G0g8RHc3Ryqwh/nrHBjON 6Y1GxHvnElhYBPMLYKI48I0UOkRzzwKwhdjbuRlNvUPB7EKzGC4/Vv3T1MiPEuWz82KDf9qJ7M5Q Uscw2qE9y5sP6sdFC3b8vY6PVMcQ1/JWW87ExGh9qWWUijGIl887+Qni73bs9bzEOo35DR4H84JZ 99GHB4i+143/vBC7RVXmZQ+V7UR2Ov8f0hzAozzAg4UBeJwHeLQwAEke4PGtAnyABJ7mU3sib3Ge Z6Gc4KTEG0dWeg/RxRxu2ds4h1xg1sci1+xUUAwK2j4r2WnqdjDHjnEjbppSlk7y+wuy5FLJNzdW rrJTfpE4LRsRydBt4PsQQmqAKfvhUSeHq8/BQrtvVEHGIjuI3WXcl7kgBeGe1wYqz/OO+QK7q0bG pXkgBVOlhp2w+l1LmiTw7uNtLLu5nC5w0kzJD5LPj+OF9tRC5cdxKT86er8TIdgZInY9UuPKg+Kv WEXzXl7Km++kECzFKjqr5q6VIuu7fmw2LyupoV+10MU2pJNt54pIEGia2NjDzm5H28ata0YVj60n iZVbQi8and3/Y4igxQCrjTB4LcYLI3II54MxeMtG/XC6A06apgUmVPT5CJkIMozxJDTI3AA4YTzz fgjvOYL3AkVRjYOfZhM/rQfZwE7Px1nlx7cXg+RSQ3ue0u97wXHZhdRgItAcJj9yzJku5pJAc5i8 jZgzKObSOHMpfQdj7p8oIXzr47l6a5KpUP9O44XvVtJylwzSXg3r3Y344ni2L9IDxpTTR6HXPluY VvartC/KroMr4la+fqKyWqjTpThbxpU/e9ZDEj2k5m4eFT4P4IMkOlaw/Lm3PW3A7L15W+lUr2xA kawud+us+Z75VrpsKh9ZS1PO3LFKZ9PrqZp13tn/zptUvlQPbsh5Z9hLvWsJN7sOTGl2soY9ykOa s5xkpyyk/9T7Bbwtv0y6mnZ8ekte2C2z2yKW3stLpztndptP7h0nJmneZ5KObTO/Ds79a637rdem 47xIWnla9ocUnpxypl0oa52bj33ycWP/7L8a5KTkxubkzplRgdt4VL9GwDt2+dN5H7cnjvzGGHTk P4SHaWE1t7D1jcnzC2oZd1pKKIWCCTOKD80sD8rDe3bEwZ0XENSSWWWdxMPQvE878qQjD6nhlnll wRBSWPiOgANUwwdqwYJRojmTxeWd8B5ep8fLL1Hh45rSQ6zF1Y2VJoaTYLHtFRYe41t0q90ouJJ0 XIkoG5GqIVR4KOEshCiisfB6R0ZhMRKG7BQswnEGC290ZGG6zWgkb4SZE5wcrbdFA9NOWhsDJ2os QOUdqpjkYJxHzdcHLICBK1ah4FsBvn4WbSIV89rDG4JgyoNzVqlhTeDA87JhHGgMHFlylYqhbOpb OUQFDvMCl+GtRiXplNaJ/o+xWQLjcU2BVy+YYiWuFNUOElZSisO14Sg06m/QPcTfTyB62mVuZHN0 cmVhbQplbmRvYmoKNTEgMCBvYmoKMTQyOQplbmRvYmoKNCAwIG9iago8PC9UeXBlL1BhZ2UvTWVk aWFCb3ggWzAgMCA2MTIgNzkyXQovUm90YXRlIDkwL1BhcmVudCAzIDAgUgovUmVzb3VyY2VzPDwv UHJvY1NldFsvUERGIC9UZXh0XQovRXh0R1N0YXRlIDEwIDAgUgovRm9udCAxMSAwIFIKPj4KL0Nv bnRlbnRzIDUgMCBSCj4+CmVuZG9iagoxMiAwIG9iago8PC9UeXBlL1BhZ2UvTWVkaWFCb3ggWzAg MCA2MTIgNzkyXQovUm90YXRlIDkwL1BhcmVudCAzIDAgUgovUmVzb3VyY2VzPDwvUHJvY1NldFsv UERGIC9UZXh0XQovRXh0R1N0YXRlIDE3IDAgUgovRm9udCAxOCAwIFIKPj4KL0NvbnRlbnRzIDEz IDAgUgo+PgplbmRvYmoKMTkgMCBvYmoKPDwvVHlwZS9QYWdlL01lZGlhQm94IFswIDAgNjEyIDc5 Ml0KL1JvdGF0ZSA5MC9QYXJlbnQgMyAwIFIKL1Jlc291cmNlczw8L1Byb2NTZXRbL1BERiAvVGV4 dF0KL0V4dEdTdGF0ZSAyMiAwIFIKL0ZvbnQgMjMgMCBSCj4+Ci9Db250ZW50cyAyMCAwIFIKPj4K ZW5kb2JqCjI0IDAgb2JqCjw8L1R5cGUvUGFnZS9NZWRpYUJveCBbMCAwIDYxMiA3OTJdCi9Sb3Rh dGUgOTAvUGFyZW50IDMgMCBSCi9SZXNvdXJjZXM8PC9Qcm9jU2V0Wy9QREYgL1RleHRdCi9FeHRH U3RhdGUgMjcgMCBSCi9Gb250IDI4IDAgUgo+PgovQ29udGVudHMgMjUgMCBSCj4+CmVuZG9iagoy OSAwIG9iago8PC9UeXBlL1BhZ2UvTWVkaWFCb3ggWzAgMCA2MTIgNzkyXQovUm90YXRlIDkwL1Bh cmVudCAzIDAgUgovUmVzb3VyY2VzPDwvUHJvY1NldFsvUERGIC9UZXh0XQovRXh0R1N0YXRlIDMy IDAgUgovRm9udCAzMyAwIFIKPj4KL0NvbnRlbnRzIDMwIDAgUgo+PgplbmRvYmoKMzQgMCBvYmoK PDwvVHlwZS9QYWdlL01lZGlhQm94IFswIDAgNjEyIDc5Ml0KL1JvdGF0ZSA5MC9QYXJlbnQgMyAw IFIKL1Jlc291cmNlczw8L1Byb2NTZXRbL1BERiAvVGV4dF0KL0V4dEdTdGF0ZSAzNyAwIFIKL0Zv bnQgMzggMCBSCj4+Ci9Db250ZW50cyAzNSAwIFIKPj4KZW5kb2JqCjM5IDAgb2JqCjw8L1R5cGUv UGFnZS9NZWRpYUJveCBbMCAwIDYxMiA3OTJdCi9Sb3RhdGUgOTAvUGFyZW50IDMgMCBSCi9SZXNv dXJjZXM8PC9Qcm9jU2V0Wy9QREYgL1RleHRdCi9FeHRHU3RhdGUgNDIgMCBSCi9Gb250IDQzIDAg Ugo+PgovQ29udGVudHMgNDAgMCBSCj4+CmVuZG9iago0NCAwIG9iago8PC9UeXBlL1BhZ2UvTWVk aWFCb3ggWzAgMCA2MTIgNzkyXQovUm90YXRlIDkwL1BhcmVudCAzIDAgUgovUmVzb3VyY2VzPDwv UHJvY1NldFsvUERGIC9UZXh0XQovRXh0R1N0YXRlIDQ3IDAgUgovRm9udCA0OCAwIFIKPj4KL0Nv bnRlbnRzIDQ1IDAgUgo+PgplbmRvYmoKNDkgMCBvYmoKPDwvVHlwZS9QYWdlL01lZGlhQm94IFsw IDAgNjEyIDc5Ml0KL1JvdGF0ZSA5MC9QYXJlbnQgMyAwIFIKL1Jlc291cmNlczw8L1Byb2NTZXRb L1BERiAvVGV4dF0KL0V4dEdTdGF0ZSA1NCAwIFIKL0ZvbnQgNTUgMCBSCj4+Ci9Db250ZW50cyA1 MCAwIFIKPj4KZW5kb2JqCjMgMCBvYmoKPDwgL1R5cGUgL1BhZ2VzIC9LaWRzIFsKNCAwIFIKMTIg MCBSCjE5IDAgUgoyNCAwIFIKMjkgMCBSCjM0IDAgUgozOSAwIFIKNDQgMCBSCjQ5IDAgUgpdIC9D b3VudCA5Ci9Sb3RhdGUgOTA+PgplbmRvYmoKMSAwIG9iago8PC9UeXBlIC9DYXRhbG9nIC9QYWdl cyAzIDAgUgo+PgplbmRvYmoKNyAwIG9iago8PC9UeXBlL0V4dEdTdGF0ZQovT1BNIDE+PmVuZG9i agoxMCAwIG9iago8PC9SNwo3IDAgUj4+CmVuZG9iagoxMSAwIG9iago8PC9SOAo4IDAgUj4+CmVu ZG9iagoxNyAwIG9iago8PC9SNwo3IDAgUj4+CmVuZG9iagoxOCAwIG9iago8PC9SMTUKMTUgMCBS L1I4CjggMCBSPj4KZW5kb2JqCjIyIDAgb2JqCjw8L1I3CjcgMCBSPj4KZW5kb2JqCjIzIDAgb2Jq Cjw8L1IxNQoxNSAwIFIvUjgKOCAwIFI+PgplbmRvYmoKMjcgMCBvYmoKPDwvUjcKNyAwIFI+Pgpl bmRvYmoKMjggMCBvYmoKPDwvUjE1CjE1IDAgUi9SOAo4IDAgUj4+CmVuZG9iagozMiAwIG9iago8 PC9SNwo3IDAgUj4+CmVuZG9iagozMyAwIG9iago8PC9SMTUKMTUgMCBSL1I4CjggMCBSPj4KZW5k b2JqCjM3IDAgb2JqCjw8L1I3CjcgMCBSPj4KZW5kb2JqCjM4IDAgb2JqCjw8L1IxNQoxNSAwIFIv UjgKOCAwIFI+PgplbmRvYmoKNDIgMCBvYmoKPDwvUjcKNyAwIFI+PgplbmRvYmoKNDMgMCBvYmoK PDwvUjE1CjE1IDAgUi9SOAo4IDAgUj4+CmVuZG9iago0NyAwIG9iago8PC9SNwo3IDAgUj4+CmVu ZG9iago0OCAwIG9iago8PC9SMTUKMTUgMCBSL1I4CjggMCBSPj4KZW5kb2JqCjU0IDAgb2JqCjw8 L1I3CjcgMCBSPj4KZW5kb2JqCjU1IDAgb2JqCjw8L1I1Mgo1MiAwIFIvUjgKOCAwIFI+PgplbmRv YmoKMTUgMCBvYmoKPDwvQmFzZUZvbnQvS1BTSEJPK0FyaWFsL0ZvbnREZXNjcmlwdG9yIDE2IDAg Ui9UeXBlL0ZvbnQKL0ZpcnN0Q2hhciAxL0xhc3RDaGFyIDEvV2lkdGhzWyAzNTBdCi9FbmNvZGlu ZyA1OSAwIFIvU3VidHlwZS9UcnVlVHlwZT4+CmVuZG9iago1OSAwIG9iago8PC9UeXBlL0VuY29k aW5nL0Jhc2VFbmNvZGluZy9XaW5BbnNpRW5jb2RpbmcvRGlmZmVyZW5jZXNbCjEvYnVsbGV0XT4+ CmVuZG9iago1MiAwIG9iago8PC9CYXNlRm9udC9WS0pOR1QrQ291cmllck5ldy9Gb250RGVzY3Jp cHRvciA1MyAwIFIvVHlwZS9Gb250Ci9GaXJzdENoYXIgMS9MYXN0Q2hhciA0Ny9XaWR0aHNbIDYw MCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAw CjYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAg NjAwIDYwMAo2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2 MDAgNjAwIDYwMCA2MDBdCi9FbmNvZGluZyA2MCAwIFIvU3VidHlwZS9UcnVlVHlwZT4+CmVuZG9i ago2MCAwIG9iago8PC9UeXBlL0VuY29kaW5nL0Jhc2VFbmNvZGluZy9XaW5BbnNpRW5jb2Rpbmcv RGlmZmVyZW5jZXNbCjEvVC9oL3Uvcy9zcGFjZS9wYXJlbmxlZnQvcC9lcXVhbC9hL2Mvay9lL3Qv Y29tbWEvby9uCi9wYXJlbnJpZ2h0L2h5cGhlbi9yL20vaS9OL3cvUi92L1Avb25lL3plcm8vZ3Jl YXRlci90d28vdGhyZWUvZm91cgovVy9mL1gvbC9sZXNzL1MvQS9DL0svZC9maXZlL0kvcXVvdGVk YmwvZy9NXT4+CmVuZG9iago2MSAwIG9iago8PC9GaWx0ZXIvRmxhdGVEZWNvZGUvTGVuZ3RoIDUx Mj4+c3RyZWFtCnicXdQ9jtswEAXgXqfQDcyZkSgbMNhsmi0SBEkuIEvUQsXKgtZb5PZ58yZOkeIZ eNYP+ZGmTy+vX1639dGevh/36Wd9tMu6zUf9uH8eU21v9W3dGtF2XqfH38bP6X3cm9PL13H/9Xuv LW6oS/Rv43s9/cjCbySeme5z/djHqR7j9laba0rluiylqdv836X+HE/cluetUiKpuxRULZGUZ69W ImlIXrsSSYN67Usk5cVrLpE0ZK9DiaTMN59LJOXq9VIiKQ9exxJJyoFuJZKyeJ1KJOXO61wiqeu9 1hJJ+ex1KZGUDVWwFp6UelZYJbz+ZoFV6O14FVaht7t5hVXC6+MKrELv4CKBVejNvhoCq9DbOV9g lfD6JAVWCe/kFVaht+ebYRV6O19JgVXoVV8rgVXoNQ4Eq9BrvnQCq9DbjahYPwaT9HEVViXQfGEV OCVQfX8VOCaJ8xU4jQ11vgKnsaG8CpzGhjpQgVMCB5+GAqcE9qzAKYEdBwJOCew5DeCUwN5FmAuD WfniAH1VitQ3FBNnMK4DDSLjDprPCkvCYCB/Ft8xqL50BpxRZL7OBo1RpHwVNBZbxqvQGEXmQIPG KBp4MzRG0eC/HIPGKDJOEhqjaGCFxmLLLjyHzwPnR9LP9vMot9PncdTtwT8AHnA/2OtW//1H7Pfd n2qR5g8v0QwICmVuZHN0cmVhbQplbmRvYmoKOCAwIG9iago8PC9CYXNlRm9udC9BT1FKWUgrQ2Fs aWJyaS9Gb250RGVzY3JpcHRvciA5IDAgUi9Ub1VuaWNvZGUgNjEgMCBSL1R5cGUvRm9udAovRmly c3RDaGFyIDEvTGFzdENoYXIgNjEvV2lkdGhzWyAyNTIgNzk5IDUyNSAzNDkgNTI3IDQ1MiAyMjkg NTI1IDQ3MSAyMjYgNDc5IDUyNSA0ODggNTI1IDQyMwo0NTkgNTc5IDUzMyA1MjAgMzM1IDQ1MyA1 MjUgNDU5IDQ5OCAyMjkgNDg3IDY2MiAyNTIgNTA3IDUwNyAzMTkKNTI1IDUwNyA1MDcgMzA2IDQ5 OCA0NTUgMzkxIDMwNSA3MTUgODkwIDg1NSA1MTcgNTQzIDI1MCAzMDMgMzAzCjIzOSA1MDcgNDIw IDY0NiA1MDcgNTA3IDQ5OCA2MTUgNTA3IDM5NSA1MjUgNTA3IDQzMyA1MDddCi9FbmNvZGluZyA2 MiAwIFIvU3VidHlwZS9UcnVlVHlwZT4+CmVuZG9iago2MiAwIG9iago8PC9UeXBlL0VuY29kaW5n L0Jhc2VFbmNvZGluZy9XaW5BbnNpRW5jb2RpbmcvRGlmZmVyZW5jZXNbCjEvZzQ3L2czNzMvZzM5 My9nMzk2L2czODEvZzQ0OC9nMzQ5L2czNzQvZzMzNi9nMy9nMjU4L2cyODIvZzI4L2czNDYvZzI3 Mi9nOTQKL2c0L2cxOC9nNjAvZzQxMC9nNDU1L2cyNzEvZzM4L2cyODYvZzM2Ny9nMTAwL2c3NS9n ODU2L2cxMDA2L2cxMDExL2c1OC9nNDM3Ci9nMTAwNC9nMTAwNS9nODgyL2c4ODQvZzM2NC9nNDAw L2cyOTYvZzQ0OS9nMTE2L2c2OC9nODcvZzkwL2c4NTMvZzg5NC9nODk1L2czNjEKL2cxMDA3L2c2 Mi9nNjkvZzEwMDgvZzEwMDkvZzkxMC9nMjQvZzEwMTAvZzQ2MC9nMzk1L2cxMDEyL2c0NTQvZzEw MTNdPj4KZW5kb2JqCjE2IDAgb2JqCjw8L1R5cGUvRm9udERlc2NyaXB0b3IvRm9udE5hbWUvS1BT SEJPK0FyaWFsL0ZvbnRCQm94WzUzIDAgNjI1IDYyNV0vRmxhZ3MgNjU1NDAKL0FzY2VudCA2MjUK L0NhcEhlaWdodCA2MjUKL0Rlc2NlbnQgMAovSXRhbGljQW5nbGUgMAovU3RlbVYgOTMKL01pc3Np bmdXaWR0aCA3NTAKL0ZvbnRGaWxlMiA1NiAwIFI+PgplbmRvYmoKNTYgMCBvYmoKPDwvRmlsdGVy L0ZsYXRlRGVjb2RlCi9MZW5ndGgxIDE0NTgwL0xlbmd0aCA0NjU0Pj5zdHJlYW0KeJztW3t4VNW1 X3vvM4+8yCRCnkPmTCYZJZMYCGACwWTymICNSICAMxRkQogkKBIN4AthqCI4PrBepYoPfEtrlZNJ pBOgBUWtogjXWm2tBXz0qr1Fot+n1lfO/e0zQyT3evv51/3nzln5rbX22mu/1llnn9nJhBgRpVGI BLXMmlteQcZ1ThHY/PYVbd2x8qQjRGxa+5pV6kP2t/4ThneIzK0Xdy9b8erVgW1ElvEoX7bs0qsv jvnnPgq2prOjbenRy1sHiar2yk47YcicmFWIAR9Euahzxaqr4uOtJeLqpSvb22LlPDvQv6Ltqu7M /rQQ/J+HUb2sbUVH3D8VLLt7Zc+qWLnqPlnffUVHd/GbM/fB/wOilN+ZdlMukGd6gnIVN+UQ6R8C H0k51KV/JOul5H9H62gcRDvoKdZFT9E+eo4NotVOGqB+eomyqZHuo7V0J20iMy2A5SaaAzLBfifL 1fupnB5CHB+iQ/C9kNbRbspiOfrHtJ42ij+g1UZEupDqqIVW0q3sfH01LaRjyvVUSefTZdTNQrpf v02/Q3+UHqMB8ZL+HaVQHrWDDumfmP6kv0NlaHEX3UPH2B1Jz5AXo4TgeT9dQdvEIoXpy/SvMQMn XYk5KDSTDrH93IPeO+hDlsPWigb08oiu6c/Dy06LqJO20W42mU3nTtNCfaZ+iLIwxlXo9R6K0C5Q lH5Lb7NU06D+qD5IuVRK52E9/fQa2y+GvtswVIuImRClcTQFNSvpd/R7OsJc7Fm+0pRqqjB5Tdfo b9BomkDzMNsn0PI/2Jd8HWi9eFFp0utpFOLycxlteoHeZXmsnM1i8/k4vpI/IK4gK0acAFpKXYj3 3ej9KPOwXTyVHxaPKE8q35jHDh3XR+GOuOleup+eZWlYqcp62M/Ym+x93sAX83v5e+JO5ZfK65Y2 rPoiWkG30pP0JctkVWw2+ynrZGvZJvZzdg87xI6wj3gdb+WX8JOiU1wufqvUg+YqPcr1phtNN5s/ GvIPPT/070Nf6hX6jTQb+bABs7+LHsDKBugw/Rl0jN5jJpbCRoFU5mTz2LWgdexW9jDbwX7J+jHK EfYe+5h9xj5n33ACmXk+d/JCkItfwa/kd/L7+GHQEf4P/pXIFoXCIyaLaSIgVmJWm8TtoGfEu0qe cljREecK01bTdtMO05Om50yD5lTLz6xkffXbR74r+e7oEA1tHto6FBnq19+lMbiHeYiCg6Zh9m2g 5bjfW5FxO+kPLBWxy2MlrIadj8gsZsvZ5ewqRPIGto09Zsz9abYXUXqLncSc07jdmPPZfDKv57NA F/EOfjm/nd/B+/mb/GthESkiXYwRJWK6WCQ6xCpxtdgqNPGq+Kt4T3whvgXpSrLiUAoVt+JRpiuL ldXKA8qHyoemhaZXTH8zJ5tXmG80R82fWs6x1FhaLLMtiyxbLLssb1iDyM4D9Az9hk672HGxQfjE M3Qbn6jk8tf4a8jnxbRUzOTIVL6DbebXsX5eZLrKXM2r2QU0qLgR6xf5dv4FrxYzWTObS8v5hFhv 5tHKryCmKQfohLIXa3sNPV9lTmXr+ElzKkUY8SkY8wUxXvGIV+htcYxZlIfoL0oyy2Yn+BOiBVnw W6XG5CenuI+eFpez6+gZ7iNK/sZ6C/L4AvYr7AutrIL9U+gk+AXIokrxPl1Pl/A/0Qk8x5vpF2yp soxuo4lsLX1Ij+OpGGe6zFxiHsNe5l1KmJ/B+okrv8TqprAiJkyj6Qa2SGwzn+R/ptV0WEmmo+LX mP1h/rSYqQya5rBOPAHX0Y10ub6Brjb5ldfZMhJsPhUrx7G7rRUVihNyPXaVhdjTduHp3o19oE7M hCUHmXM+8mIedohtoLuxTyjIoC484xdiF3uN+s2tPErLTKMYdh0i5ZWhObRAf5zu0ZfRZfodVIb9 YJO+Fj3uoL/RFtrBNg5dS91UgCfnKDvf1MQPm5r0Mh7mf+Zz+daR9xfRLmY59HfQ0yjUmPZQWHmL 5lKtfov+R2T3Wdhh76El9BP6AKv8BCPMEPtp4tAFvFdvEt1Y7zGarT+hO1gydeqX0izaS49ZTNRm 8eAea+x1rPda6uBz9FWiY6gLcdiCKHgRrdXYf27yNsxrrfPW1pw7rXrqlKrKyZMmVkwYX352Wamn ZNxZZ7qLi1yFTtVRMNaen5ebk501ZvQZmRm29FFpqSnJSVaL2aQIzqjU52oKqpo7qClu14wZZbLs aoOh7TRDUFNhahrpo6lBw00d6emF58X/zdMb8/QOezKbOo2mlZWqPpeqHWp0qVG2YLYf+q2NroCq nTD0mYZ+u6GnQXc60UD15XQ2qhoLqj6taU1n2BdsRHe9KckNroaO5LJS6k1OgZoCTct2dfey7Bpm KDzbN7WXkzUNk9LyXI0+LdfVKGegiWJf21KtZbbf15jvdAbKSjXW0O5aopGrXkv3GC7UYAyjmRs0 izGM2iVXQzervaX7w7dEbbQk6Eld6lrattCvibaAHCPDg3EbtexrPsj5vojOMxv8m06vzRdhX06X Kovh8CZVe3C2//Rap+SBAPpAW17cFAw3YehbEMTmuSpG4xsDfo1txJCqXIlcVWx9HS6ftASXq1qS q97VGV4exK3JC2s052pnJC/PO6AfpzyfGm71u5xabb4r0NZo7x1N4TlX9+V61dyRNWWlvbaMWGB7 R6XHldS005WO4TpDM9yl1jxnOLJMzsh1HhJCU9tVzMTvwpqqJOuoonB7FdxwBRhaaUtxR7q0pIZg 2DZV2mV7zVRsc6nhzwkZ4Drxj5GWtrjFXGz7nKQq82Q41VB/Stc8Hq2kRKaIpQH3FHOsMcqTy0rX RLnL1W1TIRA+akFs2wJTyxF+p1Pe4JujXlqCghaa7Y+VVVqSHyFvuSeg8aCs2X+qZsw8WRM6VTPc POhCJveT/Lg7RrO6h3/SbVln+DqnaizrX1R3xOqb57qaZy/wq75wMB7b5tYRpVh91XBdXNPOaPCL fB7XeL4wapGUC4edZcGfqinF+DEbSb00arEiKw0LU5s0W3BGjAeSnc4f2SiqD8pWhvi+WXya2lTP yHL1iPKI6aWGBSaMV2Vz64JwOHlEHVItNuB5cYGMp1a/U23QaB6ezGL8RPX9VRKBfM2LkDVIB+Rf zBQvjnDMj+sBXDI7y0qbsNGFw00utSkcDLdF9dASl2pzhQf4c/y5cLcveCpxovrum/O1plsCiFUn m4qHglN9r4ttnt3rZZvnLvAP2HBW2Nzqj3DGG4L1gd4i1PkHVCKvYeXSKo2yoMoCNTMsMsKthn/+ gJcoZNQqhsEot0cZGTbrKRuj9iiP2WynbBw2JWbzGjZ5yT2modV/evYYj2SgTL7ucLqqGbqAGmz0 9c6vr7EZlhHXZmlJC9Bn+ER3P1kwhg1v7fl48/5a1/HZnA9Qqzirz53jOLJXjKPjABfjIp6xjgFx phgbqXZ4o8LVlzmmIr2uTKjordzgKvhKYCewD1BosSiA3Qa+HggBO4F9wBHATAQua1VgJbAdOC5r xFhhj6gOW92ZIhdtczHHdJFNJwEdEOQALwdmAYuBLcB2wGz4SctKYD2wDxg0arwiO3LHRMw9O3Kz IfqWX1phFNtixYWLjGLfhYGYnDk7JhvPi7lNjblNmBQzn10fk2eWxmRmcUVIyuS0iv11WSILi8zC xLvBGX+e0hnDR6EHxRjSAC7McYtXZPYVuSu27xMKMcEFw9HFoe8XLJKWUVGXzHV+kjLJwT/hJ2I1 /ETfqIyK7XU/4e/RTmAfIPh7oHf5u7SeH5cxB68FtgP7gMPAScDMj4OOgY7yo5TO/0rlQC2wGNgO 7ANOAhb+V3Abf0dmi8GlXgtw/g64jf8Fy/oLeDp/G9rb/G1M7Q+RyikVA4biKY8rjuK4kp0fVzKz KqL89chX45BRbtxpZNQeUUg1NFEURoonOKIiJzKtyxHl7/epHseDdeP5G6QBHDN5AyO/QSrQAgSB bsAM7U1ob1IIuB14ENAAZBm4DVD5QeBV4E0aD3iBFsDKj0QwTJQfjrjrHXVZOAT8HgdyBz/EXzLk q/xFQ77CXzDky5AFkAf5i5ECB9WloJ7QxgZpgyxHvYk/21eU6dDrMvg+xM4BXg7UArOAxcAWwMz3 8cLIUkcmOtlDB60Ezwh9bMjH6WEreZc7vO4GJKAqmXvqudDAtqvb3dzr3noPipK5b7sDmmTuG26B Jpn7mg3QJHNfugaaZO6ly6FJ5l6wGJpk7lmt0MCi/IHfFJ3pqJx1CVPr0vmViNKViNKViNKVpOCM CaKvFDm3eyMlJYjYNq9nXIkjtJuF9rLQHBZ6mIU6WGgdC21goWksdBELeVjIzkIFLORloT2sCqEI MW//iOIUbw4LHWShp1ioh4XcLFTMQkUspLJKb5Q7I+dNNITPEH118qGDPLcGu086dyKiTuS8E3vC PvDDgG6UvHBSC2POuQVSFvaV1MbKZ0+tWFk3gx9AwwO4DQfoGKDgBh1AGh1AJwfQQTp4LbAY2A+c BHTADO9CTHyLwdPBy4FaYDGwHjgJmI3pnAQ4rYxPcacxsfL4pGfJEj8Akod4J3d6x9rsNo9ththi Z+kFbFaBXsArKSsLW3ZmhjUjytJ2fZn2zy/TKKkuid/Gt9BY3Ijb43JL5Kuxjii7O+Le46gbw35B BQqyjk0hNyuGrKIeozyZ7FYpJ5GdPwlZEbHPR7P0iLvUsZuNkq12Ob6yf+D42B7lUD+y73G8pUYV FnH8EZYndznesN/keLk8aoVlrzvKIHarhuuAvcrx1EHDdQMqtkUc66TY5bjOPt1xid2o6IhVXNSD kjfdMce9wDED/TXalzi8Pehzl6PWfpFjWsxrsmyzyzEeU/DE1BJMdpzdGNRVYHQ4rzLKOr2llq0W v2UWTvwVllKL0+KwjLXkW0ZbM6026yhrqjXZarWarYqVW8k6Oqof93rkW3G02Xg5mhXJFUO3ccl5 7DXKmZXjWKidIZp589x61qztb6fmJar2xVxXlCXjE4/JVc+0zGZqbq3XqjzNUYs+R6v0NGuWlp/6 exm7LQCrxjfjTd/qjzJdmjbmy7PFADGWsfHWfCnP2nhrIEA5WWtqc2ozazKmNDX+AAvGuef7K2eE Plbb2jzXr/1qbECrkIo+NtCs/Zs8fAywz9igr3GAfSpFwD8gathnvjnSLmoaA4HmKJtv+JHKPoUf MuZTw8+KF7P0I9VaEPPbFvMrRnv4FUkBv6QkKjb8ipOSDD+FSb/eniJfY29RkeGTrVKP4dOTrZ7u c7AYPsXFhk9WiA4aPgezQtJHqzFc7Ha4FNgNF5ZHdsPFzvIMl/nfu5THXW4adrnJGEmw733sMZ+0 46d80o7Dx/Njr456j4f1VQfaF8qDW9Dl6wCC2s1rOnO00BJV7W0PxE907uCS9k4p2zq0gKujUWt3 Naq91Qt/oHqhrK52NfbSQl+rv3eht6MxUu2t9rnaGgN901smVY4Y66bhsSa1/EBnLbKzSXKs6ZU/ UF0pq6fLsSrlWJVyrOne6cZYZOR4i7/XSvUBnBMM2cdTkpGvwXxnoD7L1l1jJG+1M2dd/m58WtlB KTg2peIIngbIqrK6sjpZhWdKVo2Sp/N4Vc66amf+brYjXmWDOcNVT55Vq3tWU46vqzH204MLplWr ZcBj3NPzv12o8+Gg3Sh/r9+slcxt1mrxibjXYoE1KJekTT1lS0nx4XwQM54N41RpFGLYUdqmSVtS Utzxf97/1XHZIJ+CEN/Tx7wFbBX1BIRW0NzKsRW0xo9Bu/FZSr4eegJYYA/zsJ5TfcSn7fFQrExy zaewanVci8diVVzGWqJJz6mQDF8yWJ7hiK1Ch3LnMn7JbsIHbHy+J2eGM6MYDLscfauK/d96TfQN qcp+uc2tYEd4p3iFUsgxQHhWvKOSzK+qNB674OrUC5/I8di+WHSCyk9MGH/GpHMmVmSNGW12FbpX 3NXZddddXZ138de67ryzC3rsbDEmQadRdYISlKAEJShBCUpQghKUoAQlKEEJSlCCEpSgBCUoQQlK UIL+z4mIx78hN5qE8X3xPMAMZRP9/74UKjS4IuMzyHT9FJdfl6BYfBhlyn9GgWaWf26su6Kr7VLj yxPsdln+kZd1ZHGQBvURhvg3GM3y2xoJJPAD4IdoRez5/ReX8Wfx3G292s7di9OnfW7NjyXew++f WSLly79+ec/XO79bZiOr/H/BJIrn3X8Bq40kYwplbmRzdHJlYW0KZW5kb2JqCjUzIDAgb2JqCjw8 L1R5cGUvRm9udERlc2NyaXB0b3IvRm9udE5hbWUvVktKTkdUK0NvdXJpZXJOZXcvRm9udEJCb3hb MCAtMTg4IDU5MyA2MzNdL0ZsYWdzIDYKL0FzY2VudCA2MzMKL0NhcEhlaWdodCA1ODQKL0Rlc2Nl bnQgLTE4OAovSXRhbGljQW5nbGUgMAovU3RlbVYgODgKL01pc3NpbmdXaWR0aCA2MDAKL1hIZWln aHQgNDM3Ci9Gb250RmlsZTIgNTcgMCBSPj4KZW5kb2JqCjU3IDAgb2JqCjw8L0ZpbHRlci9GbGF0 ZURlY29kZQovTGVuZ3RoMSAzMzM3Ni9MZW5ndGggMTc5MTk+PnN0cmVhbQp4nO29eXgURf44XFXd c89kemZ6jp6z585kJvfkgISkQw4QhATlSIBIABFQECJ4u4InCiooXoiu7HqsikoIoAOo4C6uuuqi Kx6r7sK6eOyuUfSHrt+FzLyf6plcePz2+7x/vM/zPkxN3dXd1Z/63FWTIIwQMqDViEFtrWcXlyH5 s/p7SKbPXzp3eba+SkII3zv/kpXiE3kLeGj4CCHVLectX7h0u7kSrlEvQkjx/sIll5+XHS9UI3Tm rxctmHvu258qz0fo2nxorFwEDdqnSS1CeXA/FFq0dOVl2fHXtEGSWrJs/txsfQHcX5i3dO5ly01n qIph/LnQKF44d+mC3Pzeh8SzfNmKldn6tWrav/yiBcuf27LeB+OvR0i7nf0MIfYO5ILcy8xDXoQy h3Px4/TV0Af96f5MhrwHV0/NxexnKoS75HQqnpTN0bnoEFqKbkf3QFs5/iN6HEnICO2HEIMRbke1 aCO6FL2DpmW+hlY/egh9hRJoFFqUSSMTWoXS+BfoIUwQgauq0dtoAdpAapk4+y+EUQEuYbbia1Eh 3GUquhvZ0UG4Y0FGC/UdxAMwI9D+GjNHnciUZL7B+9lXM/PQr3EteZd9Gr2O+nCARenrMusymzP3 ozx0nPH0/y5TmlkKV01DXehidBXMYDX6JXoDd5AxZF/mZphTO8xhFXoWvYbjLGK7kBmdBaOvR/ei 3egFdBC9jz7BGBtxPl6N38aHFKj/QPpA5ozMvMwy1Iwmoza0Gno9OIwbyExmJvMU817/39NHMl64 91R0CboMXYnWow1oK3oP/Rl9iBmiJVPJNOYp5EJj0Ew0D6C5Eeb0OHoVHcZqnMSjsYRvxE+SS1im /wDgJIusAMHxMvRvR5sBpo+gbegAehO9Bff8GmDKYAHH8TQ8G/8C34Bvw3fiR/CT+Gn8L6Ig7zMM cw37e/Zf6Xcz2sx9mcfhuS7kRiKKwcpUozNhPd9A/4T3K8AJXI//ROIkwWBW359Ol2fGZVZlXsq8 h4IoCmPHoCZ450loBsz6cnQd2ot+D9e+gf6IPkX/BigxWIvNAAsRB/FZ+Gx8McziKfwV7ic2WL9q soT0kkNMnHmDncE+3b8zbU33pr9KZzJbMz2Z32Vel9e3Ep7TCCvQiZajFfKK7YLnvISOon+gb+EZ SuyDuY7HE+F974X7H8YnAZ3U5GryJMkwY5gNzKuswN6bnpxemr43vSOTzEwC3GKQAgkoCWE0YNM0 1AH3vhag+RB6AlZmB2DPu+hL7MBeXILPwNNxO+7Ci/AyvBx34yvxVQDVx/FOvBe/iz/EXxKWKIkV 4BQn88m1ZCPZSQ6Qd8lRBjFnM+1MN3Mls5HZybzJfM5ybIItYSexXezl7BUKpGCUNvXrJ+0nl/bP 67+v/3fponRT+oL0uvSL6XfTH2d0mX2ZT5ASlcAcO9BCmOMv4P1vRLehBwE/noA5/g19hv4Fa/4N wILBGuyEGfvkdWuEeU+Cmc/AHfg8CIvw+QD/1Xgr7sXP4f34Rfwqfg3/CX+EvyIYZl8EoQaoYBo5 D97hPrKV9JA/Q/iW/A8TYRJMGVPO1DFd8DZrmJvgfe5hPmI+YQlrZUvZs9lV7MsKRnGu4m7FZsUB xSuKfyo55awcjxjiIPBhXicvsnXMErQFtRGG+Sf5E6nFvyAn8G+IB78IT/MwbUwbaSQ1iOC9gOVL Ea/arPQr/YRHnKqL3oNsIoXMDDbC6NFKoDdEZpIbSRd6FD+HTpDxgGmXMG+QLWQOs5m9g63D76FV 8ExEDPg71IAacB2s3duoG1aokNnG/pHeUaFmTiqWEkNmDfuZgjB/Aj44BhPmD3gm7sNtxAbQqiG3 oSDUOdwH+RlAgX8GzN+NZ6Bq9ghzC5lAPoS2JWgjfhHecS9aQvbiX8O6VAM9XoTb8P1MKboadwM0 RqHzyZ0oQJaTAODzNPR/8LXYCpR7AtYmRM5DLGMg89Eh0gGr/iY2kyJ8NeDpUrQOr0UJ3I/3o9fJ 7agSL2BeOCn05xN8sg9vZ8aj7fgE+yr7KmHhTi8CNEuAe0iAIQ8Bj5gGlOlnIoA11UhBEoD/ncAB z0Qm8i2+iixBi/G9zD/wI6QBtaIFzArSgu9Of8s2MOUAsT3ATRqVo9RIUavwsElY8c9QHWDjQoSU i9jDimtpmXmbOZ7pyPjTcxR56Y/QFQCd8cDd1gEtjUcfYBs+B09hM2Qim8lMR1vJNvajjB3rsR+9 lQEKS+/CtTiUEXF3RoenAIafo3y8fxO7jr2BvZi9CmTTCeCaN6I70H3otyBNHga5FQU4ngnQnA28 ZzHIiBJUhirg7erQWOBKZ0BfG5oO/LQLuOR56ELUDZz3AfQk2g4SaiLA4xy47jx0PrSvAAl1Jboa 6H8NugV4wN3oUfQWeYI8yPjJTeQlcglZjD5AHzAvMxKejg6xN7Or0NkohKZgCzy5ClbJB9fdknkb nhZDLuD+SaBSwPvMvzLvZh7rPwj3exTmfodyLPqXshHlo1b8HevECqlhqlRfN6a2ZvSo6qqKZHlZ aUlxUWEiXhDLj0bCoWDAL/q8HrfLKTjsNitvMZs4Y55Br9Nq1CqlgmUIRonmYEuX2BPp6mEjwfHj C2k9OBca5g5r6OoRoall5JgesUseJo4cKcHI804ZKWVHSoMjMSfWotrChNgcFHveaAqKKTxzSjuU b20Kdog9fXJ5klzeIJcNUPb74QKx2bGoSezBXWJzT8sli9Y2dzXB7bbrtI3BxgXawgTartVBUQel Hntw+XZsr8NygdibR28nSG2ASfU4g03NPUKwic6ghwk3zz23p21Ke3OTy+/vKEz04Mb5wXk9KDi2 xxiXh6BG+TE9ysYelfwYcTF9G7RO3J7Yv/aWFIfmdcX15wbPnTu7vYeZ20GfYYrDc5t67FccdQxV 4ebmxvY1w3tdzNpmx2KRVteuXSP2bJnSPrzXT9OODrgHXEvCLV1rW+DRtwAQJ54twtPIDR3tPfgG eKRI34S+Vfb9FgSbaUvX+WKPJjg2uGjt+V2wNM61Peisy/29Tqe0O3MEOZvFtVPbg/6eelewY26T ezuP1p51+Q5BEoWRPYWJ7ZwpC9jtecZcQW8YXlgw2CeX5OG0NPGsQchiOqPgGYAQPeJ8EWbSHoR3 qqbJgmq0dn41DINPB4ares6FFVnco2nsWsuNpu30+h5FmAuKa79FgAHBvi9GtszNtSjD3LeIFime DKIa9A+Ue+LxnoICiiKqRlhTmGOdXK8oTFySIouDyzkRMgAfagPYzu0YXQzg9/vpAq9LSWgeVHpW T2nP1kU0z9WLpOJ4Rw/poj37B3qs02jP6oGewcu7goDJOxE1Eqw96sjg18jZLM2LRvdg2890L8j2 Tzw7OHHKzHaxeW1XDrYTp46oZfurB/typR5LYzvjIrkScTFyLyDl7MHBtNKu72HD8FXKSH1uSqUG rJRbsNjSw3WNz6YdWr//v7wolTlGr5Kzocty0+wZHR9ZrxlRHzE9/VoGJsxGyMSpM9eu1Y7oawEO tHZtS1BsWdu1dm4qs3peUOSCa3eDAhJZu7y5a2BFU5k961w9Lbd0wEsswqMBWwkauz2Ib5qyXcI3 nT2zfTcHps9NU9t7QbVp7BrbsT0Efe27RYQkuZUMttKaSGtoIgZM7wXNkXa5doM1tlruZeUGuT4/ hZHcph5ow2h+imTbOLkNPoV07an8Ai3ijYwq8yf2hIwNwz+Ytuh7cAAk1XWgixLEoWKQSoj5WyYD Gj7ZA+JjP7O/d1q5lIJstJztyAuVraa5ziDnvZry+oZiZj9aDnEbxIMQWTQH0lW5Fgb5IK2HSFvX y/1bmL2oB+J+iG9CpC17oGUPtOyBlj3QUs+kEGaeZZ7pDfng0Tt3CKGyrxqczA6UgUiY25l1YM75 mHNy+Zxcvh7yAsg35PJbmXW9NT5jgwbqGH0FaQYigXe7v3dca9luuVBVKxc2D7Rs3gEtvgaBuR9m dT/M6n6Y1f0wq68gxXDXzdC+Gdo3Q/tmuX0zwvKt/LHcrXKF+3uNtlwLFBq0TAczHTQFH+jl2XwG M723zLevoYuZBrfeJqdbmKmQrpfTOXLaKqer5N5VcnmZXF4ml+vlcn2uTNPiYalPTo00Zc5izgYd wcdMYSbIeRvTjMKQt0Kd5pOZM+R8EjNOzs+EdgfkE2GcGfIJTItcPwPqTZCPhzrNxzEtvU2+kobl UJ8DfWBPM7S9CebQBHNqAiDRlvUQt0A8LLfMgXQVxIMQGXkkZpogNEJoYBrgCgnuIUGPhBhGglAP oY6pg54xMHYMpBJTK79jLYyqhSfVAqxq4c61sDygv0JUMbWQikwFKoEoQWyD2AVRAfdJwHUJmBfo pGBlFIJe5QO96xbEQy7mch9ZBxqfj/GSdb1en9SgITvBetiJuiAuh7ia7OxVmI0NPIyjY4shtkKc A3EVxAchboOoRvXZHklH6kk900paGRawO7ajtrZMzssrs7nbk831zjJjw0VMDMAUQw9CZGDKMZhy DF51oOaDSAB1omgfxIMQD0OkAI8CMKIAjCi8YBSuj8qjlPK4ryBmIDKARFG4/8gxCvlqH8TiYXeh rfnQkg+1fLgmH8bmQ+thSLF8Be1vg7ge4r5cX0BG5oCMnAG4VwBmWwxpvVwyQupjAr1EY0wBfPFo Y0MVwL0VInSSWwGatwLcbqUYQigRF0NPfW7EeojbICqY3RBiEKIQ8iEEIPghiBBgBRkvrN4GCOsh 3AbhVgi3QFgHq8Fvi++LkzkVyypWVayveLBiW8W+CtVeMhdCF+mStMhmA55pNqmdDRyYN7ORAf9H Tp+S04vkVJJTu+ScbTg62/DKbMOm2Ya7ZhvaZxsmzza0zDYUzzak8DzJHjd8GDdsiBumxw2VcUNF 3FAeN8TihgYTGMozkAG9IKdj5bRMTgNy6sEzeg1I8xyehfxqwHgc3em/xveJP8XiXt91/pQasmuz tVnZrIY2PuMr8S/0JbItkWwW8j/Pwh3QNPwkUuG4lFC9qpqjklSjVEWqQlW+KqoKqnwqXm1Wc+o8 tV6tVavVSjWrJmqk5lOZI1KcShBeydFMydKUlcscoSmRBQzYz2qCJqAeCzORTDx7LJ7Ys38+mjhP 7Pnu7GAKa0EuK4JjcY95Ipo4dayjpyo+MaXKnNVTHZ/Yo2mb1b4d49s6oNZDbgKxN7U9hTO06QYX VYF3I4wTN9zqyuUdHfSa9u0svvXWDmS7pN5Rb64zjWpp+pGkK5fGhz6O+PAKzMTTc/fEs9t7nvB0 9JTRQsbTMREgRzXm3aSaVDY37SZVNOto361dTaqbz6Lt2tVNHUPjkAjtTbuRn2byOCTScUg8ZZyX VNFxYZplx3nlcd4R47aP8Tc3bff7B8aMkceMGTlm4cgxC+UxC3NjmOwY/7AxqiPIL4/xq478YIz3 vxgT/tExw6C5YGz8Zz54N5qA393eeAU1N7qCzQsgdvWsu2SRo2f1PFHcjRrxuzlLJNI1b/4ims9d kMLvBhc09TQGm8TtE674YX/PFbR7QrBpO7qieWr79iukBU29E6QJzcG5TR07xs0teGrE424eeNz2 grk/crO59GYF9FnjnvqR7qdo9zj6rKfos56izxonjZOfJWM9oKUaje0A/VbOdxCdFhC4y+XvGGvj ltfJ2Fzjd1zt2sMi/BjSgbqvB9PRAJF2FTYUNtAuoDLalUetylyX4+oav2sPfizXxUGzKTgWOZoX N8F3xYpc4b/8rlixYuU5K85ZQXP5u2LlxRDpMqEVaMVKBG/QoJflmw+4MeXN6yDeIvNoZsWKjpVI XtMVFyN6t5U0Gbr5YOliuDNeMRwJ0IpTPxQz4igb4XYrLsYwig68OIc2KzB0wm0QnWTuLtQxR31C 7EIFqLFIhVq2K1UprN9JMFKwtMAgrVIBhWcYhjg1Ktr2DEaCuvVKR3wyd7x2Un/tZO672klcPygS tf21NJaWlJv8prDf5F/IopMis/+kpEAnkMjuB/6mJb9hXmT/BHLdhLq25ylS5EZJi7UaDbA/7Xua PeRhpCMvSHrRtM900HTY9JVJYdqDbYiQF3ao8XsoRR7eVaJeBnz1ObIJ9KuvcRtyxLnvOo/3cf3f dfYd74N51HK1MLfSEuxnlMpgIBIdKsCzWpSiIIhKvFAuOpyigv1T2hnx+SL402yOMOlMn2S2sZ8h F2qTojF9AUcU9jyL1mxTKhWc3Wax1lkUkzQay5a8EAJDgCDB/Yc9WIEcWLiBQqZzUv/xWq6POwrT qa81mUeNwjSBOXXipNlcVVleZrPyKiWx8mbqKYLZRSMkQjprn4jq88yC6sJzzrlQJZjz9OHHJPzN CkzwWUGdw6TV/yGdeviRdOpVvdYk6AJ4Qhr4eWH6JFmVm21MQzROgQhOls5YY1babZxCCbPVamHS MF8jiDGCnJ6H9+BJA/P9js73KExYnu6I2fKEqCjsKpJVleaKJIlCCWZvt5ltZNWPzvbrFelM+qmA XoDZvorHP/wIHv8HmK1DF0g/Q2cbTx8ib+EipEHlkuO36E/oCDoGxPoMi/8PeRH9yQiClKiew/ci LVqKPdnlPdp/FBX3yRPy49x08H5sSr/nighBBhf1v18WFLR6Kkb3EBVrIasAo52SHu2Hd1UQgZ2/ laLsUe5TVDyJ3sjqr2AtJ39DVl12GYDjk8zHzOcKusNQjJ/aYSba4J7MN4jJHO8tVMcaNFDOzxxH 0cy/kQ2iNfPvZ9x5mjx1HtmT+R5xmW96PXmF9IqCzDdSMKZw5/nyAualaq/bjIpwVGEIBPP8Y8yJ MQqzQmFwjgE8fv2Z0tCYPKHkV3uwEpYhIaPNpD7uO1iDesCcPlgAWATTqOxKNF4uzSRFXMQh2AWb YBV4QaF0uzwur8vnYpXRSH4kFimIsEqdXqvX6NV6lV6hZCIBU0hCosUp4bgyLKFCtljCQaNfwi4B kog+IaEiAonMImSmUACf+DWoOvfB1cM/wM8kq8lrEep5r8leb6KJzes11wdSmROSBIUo7zZB4uIg EYyQ2PPqgzSJ8jYDlCBheBjHeM26+kItJDZa8vCCn97kC8kOBSNv99GrfPVEy5nq7DTBPyIL6bQ7 sJWT8TMagW9FBSeTlt0GX1USWqKRYIBYrTzU7bbyMnMF8/k1C+6bcF2Rp9loh9LEa4u8TZxtamOB kD9q3K1bGuOO/FHjb9lCPnwz/fUvr6qp8N8xZvqKNzFHy4E7aqevuvSNMUEhmD6yf/elfxwTEELY v59i3VFgoZ+z3wMNbu81q12pzPeS0aREao1LcrWZ21ysxriHPI70eLOk4fR6I/eCRk1oiwJazFih IPgFdc5trDK7+D3kPWQiC59FCo1aLxB+L7kGeKad/BGU64UmE16IOMw9T5YjN/oV/mMWg4BQ+mq5 /j5OZoP1fX1AxvZRiOsfYx5V7MDct8cPjKiUlqBOeZVN/jKZCfkDMnX7TYoBkq8iG7DodTq9/Uto isX0l7zGKGjVAvv9idl2i9nhMFvsbMl0pWAyGtSUj28FSLwHtBTH4nYlaZza/qxLF1ewPEIpPGuX Vs+PCShQfV99P51caQlopLbMX6WEK5Qcb7wi78bojfk3xh7NfzS2V7+zQGMwa20V+uoCNhYs8Mb5 qDc/qOd1FFMM/zT32f5j7rex+eoBSH70bA6QiufxUWAwOmwAhjNrp0aj1TtT+H92ys/eCwaBDoTR rJ3qv5nGhBsMZBkqRHZo9cJ4HVmKEvj2AarkvjtOiRISgGlfXz3A9yjXh3NgRFkwAnW6fSGzwxYW I1a/Q0KWoEnCdh8vYXMIkhx1XXNNFt7wQd24O95R5ZeZqc0K3ChUVUcqkoCuSpUyx3Ers2JCqVQh VT+5wQHQPnkIo2+6p/qevvLCJwSlRs+Z7It3z33g48isS9Lv75nqp4t08VWffrlsUWv+kkev7nSo tHau5OFzPlg7eu6KlemPfkVx9XeZj1kAFIKF37GkGqMUcK3ysrIK0+jQGaEJ4cbqi5Bylf/G6rvY jRV3Vz9S8Wj1bsse+2uW1/g37B9a/mL/wvIfe6bYRK/bxQdg4UwpWEE3FGJqoy6eb2KKYSIOpAi6 keAV8yMJAZZ+hyiaEyl8647ImPI8yHeZxyiDYypT2CBprWMYt3sU4xxdvAeWwE2ueVYnjCpXKA1f 7MGrswsBbBFTFnn06GTuU4D9JI5Ke7oa/Ueh2gd8krJLGeXha8oyTXeyIhS28KwinAxK2KKwSjhU EZEwz5olhOR1uQY+kFV3dlej6m5sy4rjyKCgKy+rhHWJZFek3C7X5FUaoJHsIjGWlVd8m1ryeZHR znH85qfueGnuM51epyCM795431Uz7khwJp3JMePy+x58fR7Zmtw1757PZpdwZs5hXPHs8okbzqa0 hNfOOmdDbZLX2Ln8MdP2XT/1bpBN71J6AunuQX70lmQAuSYSr1/h8bltANZPn/F4XrAZreYU7pLM eXkvWEW/fyFhQG4zxO8TAfDPMgyr8HsNXij3ojwQPiCvPG5KBjZkhDablUmR6yQjVuQt9Hh8yOjF QArePeRC5MezJB3QEBYCLGvVg7T6EyxHaHA5uieB2tVdCzpXfy1oO1xtHy18CaoEaGFUEeuvNY1S rCmK/4I7ANQChPPtodqB3Fha0o39FbjclAVhMDhQyDGicpMpiBmm/2389rYWn9Ppa5HT9Cs0fSCR noHnzGWiJ1+nsEt/O8CN8BxyuN8PeH6A4jlALoH+KgV0Lo07oIkJox2KwtiZsTmxC2P3xl4VPnT8 y6EWKBLbKBJboOASg2qeE0M2nxP7PH70PAAqTPkIQOOopPGMYVktioQtKfx3SWMfo3WO4VRYtYfc gGJkyS4YuTAcSuG/PMsJhWFWO4DCQzCbdBzUZK6vvzOLvlQ9LO4Drk1RWFa8zAPY63C4FRq3AuS3 QwOJS+mRsKC2D2EukHA83tmNTQOsguqRp2BuMJBlJwNMHq+acNOYX75zbOelF06WIg7OZLmnd+P+ R1dfd51oAKVuAmUh7B3pBT7fX3e98n1FuMpvMwvmW1/9zW1PNXMOGymkfAigOw81sKPYSSCVrpO8 25VYrdGEEOaRRouoRh9CJiiDoq+ehVP4iR1IO8vUoMFPIC1+DunJ3SDdtiI1fq5XuRunyFZJWwz3 FMyo9RpHCgcBWEIxdsiq9NG+Pvgi4bijT+AgWaMuiucBPkHukAtUL7TgnE6NcQ4O85hbbDqTEDj5 HaMJCCadjYzD/wGtVLCk29JtFijogboW4LXsYkaPDCC5q3pVIZjL95LLGjLqBKeHbTVj+BrNxeZ6 M2MW3DnDpxNQHuweUCX7v6PK5IDczDKCETW29KQsOZk7aTqsTJ6lcKQx/bbdYrHTCBC4E6B6gQzV G6XAPuVb+m/0zAhoDofyLJMMWYBwgw4gS6FKoZuDLNgxW0ED3l2sxEoAbNuVAFj/cMD2H/0vAIsH 9W2cwyv2gpMrZcgy2pP/zkE2rZQhi3twzwBkTWBZ7mPvQAWgAj8jxStNgNeuukRV4TjzGc4zEy2F oBbZ5jjnJNoKvy8wxlFBQaIIE1Ko5cDKk2yG9YYHDeSwARtiJoOBM3m0JnMwRrvyIpHygkgkVuAJ FiQ0jNykVJYTpZIhHg0pFCxyk8023WyzWcwewWwKuGnTeB/yrfZt8DFv+rAv5vL5QIMOuJzOREGB 1+XkXS6n2WTykkLgnYWhYFAL8MbeuLHIV0SKijRCYSLitETAuHLuwe2gI9RJfEHEJRk19ciEjaCI H3Edc7GuFE48U0IipsKIeQ+uQ6bM/h0mbT2IyP0SB2ONJoxMrWDZZkwsLF9iR3HzEliYrPrWDeRM VY1ssV/W4yhHpSyiUzawgaeuUcjsdA0s0ppfAFdVD+p0X3d2F4OON6zhf1WVr1YB36Yxa2wxpzBk nBODfnxKB8MEGebK/ve7f0UxPP17mjbgFd/LWuNv8H0NcvPLlHFv2fi57294TfqNAYbN/JMi/4nf DjLwNWR+/wNUU1mVOcyeyT6FqtFh6bJCHhejetSKGIXNaptuX8Cfa1tctJxfYVvu2GnXVrkrSybY JlTOss+qON++qOIG96ZibXmpUXQFMGLUeTZ7VZkY9BpBgpp1wZ1xc7hKt471huNVDEvimryIussf iThHuyLGUl9pcWl9KVsqjFpzeXZxZILpA1N5Un8/NdFkG42TmbZdNtSyVjPoHqNk7o0m9ujOntgT mjKzfTdyZ77oBfrdk/kCeTJf7ALrxO2w5TTBDtzZjWC1BzTBnPIRlfk2DdCEgJMP+gwobytiKiqS Zmhh3s/yDpOdKKavvHPudCkyNurG3M4lW9tMVrMtftYbi2edM/6cm8tu+GzNm6yvhi7DP3xOh2tq Q0fcVzh5Tkv7xufS/zpnjtVmshfP7gy6xm+9fcbWq7C8/fcJ6B7/ABr2ofekwmK2SBHUiwaRF63F 7mJvnaJcX8KXWOvd9d7Jika9xEvWie5WT6vXSncmHpZ0+krZ1PFoBJ9cd1cit9uHPIKayHVFpayv e9QOM61HrZUmq9Vs8jh8EcEcERyERNTGiEajpoRoagWLRxBvOZxlwdT5BNRBl4DSSN9PWjk/Z/8A iv/A9hnBvoPk9lPsnyM0JzI3Z+tO2kbisMzEAW4dwPs2AdzGknOlTkeho9LZECqvKK+qHOef1bDQ v6ThUv9VDWultQ2bpM0N2xr2NrxWbjGiyvLm8hlJ1hiIV7YkGyqmlx6o/520v0HtCrhKFwcWl96Z 3Fb4eOXngf8p/J9KbdlYhEoH4BwfAec85MbuchFALXqEghKZnYqFGwpJSSEuLNxQWlhYUuopKEXZ VcgDw0lRPmIhdNZ4biFiQVpvixgjvkhJhInEA1QT9sQC/oakVMnWjw2UIjPy+gO83x9A/tIAK+KS SEEwUhCLCaWBgAgrCUvpINVVkbr6erWai0jAVlPkip1+v0NTlsLtz4pjx5aisZGyPfgxFCBXSHap rbSrdHkpg0ql0rZS5kjpsVJS2lC1F7iuiOpxpWRq8osUI8AGPkaRonHCHjx10IsCwhlwo7YWZJqz 3wHVbic1FSgPdQp9srblqHdSBZWjLLWP4lBtNuToGfRV6usQkOStrUeSuwoSoQwSeyEkfH591nHb sUbxiwOIXuAYzkWLfxrnOk/hv53dP+gefqUqDxhx1ojejQKZIzuEUNKfyhzphRym0EERWGYeP9Cb gyOQOGga4NsDijamHhL7QI1Eonj6Zlm13kIRu7tbRvYu2rKZNM2m+b9pUyBdtvCm+lDXYtry6+t3 rsF/SK/9IQn0nyCKgXJ6fsHVKxuOyebN4oMFlDocQB37gTqqSVQa/bHnUy9pQROq96M30dv4ffdb nu/Qd/g7jzaMop6oN1I9zj3D/Zh3t/cQOoQPef6JP/cY2r1YL2Oq5UEjNhp9RmKMWYxGs8Wj94Vp O4cCbQESiEUCgXDE4yuukNG6rLwSzNxKT7FOIdfV5axarWA9Opc1ezMHNjp8DuKI8Q6Hlfe4ivKz FBJvi5N4LBqP50c9RanMOsntwUh0ezxeTHhMU281Ql6Pl4cmwG6PpPOGIz6f1+v2RDCtT3C7XdVV hLFGXKSoOFoZKS7W6fSsJaJXR6LV1R6v11NV6Y1K6CD2RedEl0W3RfdFFVEpGktGJXOFMbo++mb0 SPQYtKXI3ySrx4fnYLIeH8QEY9btZglhPSlyuWSziAzLs95Wy0HLYctXFtYijPrtMkdOgAGOOwWu z2EaVZz9dnZDtTMe73Zwnzpla422csBa+7M0QbN6Sk5yJUspQCZUCQHtQ1YVFaCNxB0/icenIPnP U8fIywDBL+ruRN04iH9oGw6gNMY/aT4GyS+70s9zm2Xl4w80HVdB0z/iOjzqjxSJk1mL8jVQAH2b zdR0PJWn9yfIIZoPb6dYXARYfA1gcQIvk2xqgjVuwU1eJliHlS4XtrlYnUlGsrwYmOUms0cXjmeR KR/nxxL5+fGEJ6xl5SGqckalYhmP1snLdXscaIe3eJwhL60H/OUev9/r8YRcBJux1+3iAZuwC1ni kXDYGwmFgJFf8YyLj7icBDTdKyQt1mm1WO1xe8H2SkguhBJSuMKYaE3MSSxLrE8cTigTziLCeM0u OtxinmNZZllvOWZhjRZsEQpHXzCo9XSDpKUqKN0liNdO4j7Nit3anNiVLf0+2WzNcU0jVgOHxLzJ DQnnqscyp6SK6v9aRJ/KD0tLZHbnD/4kwyvHp7I+lizpvyvL216TXQgyb/uILAGkEHy4UkYK1n5y zCna6GfMS0MSnaAdsNqzYLWDqBAdkxysk3WpvMhncZl9YVeFq9m1O64tMEdTmS8l7mLn9U4SVReo Nzrv8pFTZfSP6z6lgyI3IWs+QeQNm42h+hAJhRygAMXCRhDqzuJCEHucUPTdkFqa24UD4kZ076a4 E8mrEJJ05vqQpDFCojPW00XI7dD998oR9VNWV+PuU5WkU+SL7LAM+8sqZXeDH4xAWWVKu4bJks+f +nBc2cS20dPT/4P1nQ9NfOLa9Dv4SHrlSKi/fvOUa8PVTsvUsy+rm/9LCvd2gPsLAPdCVIV/tRv5 MwekyaK/Ls7bHXWzKs4rvbiUUcVHl04onelsL10prkxcVnFrxSMFT5QejLzje1s8HHmn8KuICZTI 0mZfi/+yxA2+tYnbfb/2bU28Ir7q/zRu8O7NfI80yPijazRSLaoZWiOfWBD3KwOFiaCvCFXmdJxC 5C0uomAvohAvKlKD+hQpKKD6q28PuQIVki2SAcGLeLnysBtFcCSFO3etcq93A9XifIkeXGkLbAm8 GTgWYANUahhNEoeLuWMc4YTqCUuGTBGq4nR2H+082kk1X+DNdN9IpkrZVgTlYEC1kf2i/5eNgFMX vhqsF3POeun16cU9meMA+eM74/oKmy+V+a43KZamMv8c8GWDDQNMupO6uX9eEVHZRugc4fJBlJn2 Q+3j5APv3XD/zNW3SrS2/P6ty9LffnLhjimPX55+jWjTE0Yizsu/mPlgRd3938hqhv2FiqltS6qn 3gtcejdYkDxYkE3oQ6lgTPmZrtbyzvJLbTfa1jhvdt0yatNY7RliSwOhKPF4w2Nj37F/av/WrnLR l7Q4KqlLsCMuxcbUOB1GBY9wVV5ZSZApSlIr0qQTIrW1SVO4EezIonXRZNjfCKZkVO2Xjcmq8Bzv Mi/xOlv4sFQaCUakhmWxVbH1sQdj22KKmND8wB7sG1JaqZuL68vuxWYtzAETs98ku7epmWnPOQhl kxOMTLp5A+Kx81TrkTqorbyX0F2vnDss21WRHQitWedgNDLoL2Q2Zvmd2Y4VD12/7uGiM7vO29ow o+PT3314HQVrtmfvL3/5bEtzyb1vzZ799lM9bJ2brs67XmpQ3rh+btlZ5T6T2xNde86G124uoV2f U1tz9j2/XDJ2odfqDI4ff8P1L1DpuR7oulbmp7dJBUaNvoIDsgu4fZXBoMdN1IoKDGRnEWyVZrNH CJpBPBMgIyGFlz3DcSYv3AKKksi5i91d7oNu1uiud7e657iXAzVtcx92q93/CFO1h3pSjue80/Uy +8NcH6Le55wTemQNkP8HKOw38TkH6kCBbPhAtgllnfiD9GMUPMxTFHwjTcL0XyhW40vTN8t5EN47 DO/9MLx3Pvk6q9hLDquD2OwKJYtZdb6TV0ZEPdGEiDWWJVo65Vr5EIG8Wyadvcy5zLXMvcxzk+1G +37Ffv5zm6aL6zJ1mbss7EGCORtnl2ySnXUQl90r+Dze/Ji9klTaSu0tpMXWYO/As2zt9pvsj9lf Ja/YPrDzBtlfZuLawJSq4DnOwnsMvNUfpa3ekBhaHiIoxIXaQvtDb4YUoQ35oVA03+PPR3qlPERj 1Pg0xKjZpzms+UqT0Sg0GxQajVLh0StY0UmH8J45HuypEDwep+ARBQeCFxZT6f9ISSvLiLyCZb1W ngfszQcl2iGA8i0QTBjsddihbCcMwYzXaoMRNhKxp8glktcRQRiDNs2w6mjE76RfUbREDMqIQU/w CziBENBWJxIAUzqlsoMC9glYkAoqBClZmRRWF0MhGEoKUiSaFCKSMd+XPyd/Vf76/AfzD+Z/la/O 30suB/FjB+3JboPLbFIxRLjUJjkrjLavbMQGtutOIkUqQFZc3qsQrc/D43jEwKNZXChZfTzez2M+ wikwUrQq1isOKljF89AbQ82ytXpu1uXXBwj6pcAdBe073t8tH6j7VOD6u52OPtnK7O48Cr0O7ks0 qHDlfB7UYO2XNXE19QcqQBUfLAw5COF+6FTz9Odchj9sKJaFwsSeCIiEAhAJz5LVxGl32pw55j+x xzno6iKZL3qJ2p7KHNtu4waEAzVROzs7/EHqJTxFP7NYyi2WU9qY967/8h/XX+WT5UA1pZ0Dy/5+ zT+WviQ3VNIGH1N/8kW2bkBFOxlgik++xfx1mCe9DShtNVBaIymX7jJX4NH+6mBFo6RvtbUWNVRP 0c+xdRZNqZ6rv9B2YdHc6l8Wbaj+TSBlTvlTyVTjK+ZX/K8kX2l8H32R/Kq+r/Hf6Gv8NReg2x5l 2NxoMjcGuUCQ8yfLy7A/mWw0m81ef5L3+5NlQc7MeXEZj3EZAQnPRYwRrSVijvgjYsQ5NtIYSUYq IjWlkbKImCKXSW7QELRqp7qGFJCvkjgZaWysr66uDwaLiqKNVCkw1zcouAjGCr1e4fHobTYPps0m o6JYUQ94NUehUDibyyJBaN0VPc8DT6L92laguWUexiM07cURrAZ8s2YFjTDpuKOPO069/lTaCJOO OswD1qBAl5x2ym0CVAYbO3MiSRZTfcMTioBU5fRy1EnCUScJR50kXMDsrefyDDZI9Hx97ggHNQUo ZjbSk5zmzKEdcI05BTlcJudwJeSf74CL5TpcT/PekbcwwgeQsrtTvk955mvJkmevNxnB6jAprDTh rPUY7iNZoMnvgM5GmnAOI5+9P+RlkD8LuSkPtEk87FhJvAOf6uA+paESGk71qZzaQDbh9VlN+Bua 3ph+JP34jXL9uNfJ8eV4bfomGcc/oRh9Dm7CY8+hpU9pm0hm9PcPesRfSI/NlvNsSmCMnw16Vzrx Q4M4T9AG0HamM6tRPqrE86QpT6ge9j1RxERUYV8Nu9JyqfMS12r+Bucd/F3Oraot/MPOp4t3qZ7L 287vdO72vpZ3vNSqxQIuwMx9pjud5MqitUWbi57I21r0Uuk7pZ+UqvNBJ31acoaL/eFwwB/IN3ss 9lilH1XGMFOu1yQqU/iINBPflI+05X5Gp/GjBJdYnmASsRq9Pp+/n/N7VLTDgETRL8G6Gv242F/v b/XP8T/o3+bf5z/sV/ud1fb1JX4l7V+mfFC5T3lYySqFqoK9Q9ovjk/q/3RydpMk6wfuq+/ro469 4k4o1dcel9Ve+9B26qnnXib2CDmWtQ+pwABIZo6hCohC5vgOs7pInfPNA+PKOfJ5GLoXeWGIJbOf 9oDS1emvyB3UoCrVsK1WenBG1scGdAkmIvdZeap2VTHtz755zxNH3ht9U+vq1fO2ixrOrs2bf3/b g73L6fK/VHP9Gc8unHzpRUv3zr/8vk3LrnjGyN3UfN4orcNs0hqdBQ/M7z8k67m/NnGtNWeduWjG HKpRbYW1nwhrX4CO7fJrwQCxghIrJaDwsvWj8J+jR3xH/P8K/zOqClmjtiZxUnhSdJrYGZ4ZPd94 vrA4fLOgt9GDICssfIdluvWC8HnR75wKpVPgrM4YFzOHnWu5zdzdjrucj1gfgbHBiNlkFHiXvLci uO1ZjRjdZPLHVLodrNL9a7s/qMurUXds8eENvv0+4nMmeH+E7pdtiWDqUN4QYSJC/MBtw8waWFmZ K3VPOp49bwPhaG6LZWh7Jav8dndS21Q2OwbUX+Vw9dc2fNckGEAVSQRa7ktAT2Ai0B0T5bY79/72 3SfmvXaWlTPZFzz0ymvpE1j32ouMwU3X4QWf0+4at/qf9zx0aHwbbzfFx16AmZdfw3pql14N0N5K /yYHwPtvz5xRsKiAUKPx6aw7vVi2GwNqr4M2ca5iu8vlsAe8WlsgX9OpTeH5O/L9AG88XxIDft6L 9DpeRX8EYPdpxNX0r1Vg7EyE/atBJUvhW3bEC1YP7BF25+BDjb5aeSMKNIKj8D1Ocf2nHSmlJRN7 bDmc35GnNqspEk/s0Q9K7gKQ3CIfpScaI5nPdgTVIWGQCgaV4qCM8bmzSfYB686SHII4S7JIfMff Lnrr8svfWvHR3XJ9+ft33f3++3ff9T772YmlFHt/88rlRy697PAVr+AP5INNr2z56KMtD/7lLwDb 1QDbYsBkAYnoTWmx1rbJSsrIWHIWmU9+T35v+YPwgfkD4SPX3x2f+P5jMwjuAneSVHsnuM70zXbN 9C1zLfFd7brFtcm9yfuswnixbY/7AHPA/Kr7Va9S/ZLJKYqgOpo8fruK9Zt0+qnOmi0ILwcKSuFP JHtArME1W3i8jN/HH+QP8ywv+AueHIaik/rk7dm+ozLXAQDLpzlknjIAzF4brwQNaKeL93lJKvPF IDPB8PXbbKds52UxE6lkvFWxhScfs33y+Dl/bLDkcQ6u5Ntr3k8fxsZX/oi1M4R3Nm485MQPPPRy XblRMJm4shnY9eqzWJn+P9ese/rJWykfeA/0npmAmUn0mhSW9G2K1Yrr9NeUbtH36nfGfxs/FNfa 1WBwvcJxAU2yCJXi0hRhn0EoUARmVwpLkhMD5obyAyjcGfN7EDKLQlGhQ6lRawOAi5K2EiWw6Dwo o+ZdkqHYKlmXW9+0slah4uLd+HWU8zUfpxvWtdynstlVS90R/UdlfjzyGFDn0Hkg+cxLXkHcBQua 8KG4K+bD9FD7Ndf8cJ9uyOOXO+wycPqFIqJ1YPOuGMu+hP5lNH3tGZo+8+Rtl64ptzp4teWeRRde im+WT2QY+scNyFqym+LjqvPvt6ltZrOdsS9pXiUfeQHM/EX6avZqwMwoKsdeqbSZX86Tj/xvh7/w Hw2f8B8PKS+ILS2cXzy//ArDVbHu8ltiq8sfiN1evjW2pXyPN4+oKTeYJzMIjUKh1gQI8sZLHSJn F2Et87wbS/2iNu5HGyMqUAiVWInzPSIWtVpOs0XTo2GMmlbNHM02zUGwsZwVRf7VwQ3BLcGeILsv eDB4JHgsyAaFZMHcEcgqc4vaSVwfLAawi776o5SlZjdOR3KMzpxkHMDivciVOY6cmeO9BWrQlL7v 9apRCmoJdQnNYvpy2lhoKx5yBeUQncpGXDHoledVeSQ4dH6uqrKCchFSkTSXlw1nHcw1soLfHXIs nz3pS1r8esKlUduad546ceKpd9a8duutf/jDrbe+Rl65T+YYu6eOTZyTD5qPA595RkHDyd0Y79qF UXrina+/sfHON94AWpgGtLAUaKEaXyQVbnKeEAmLrfhc5cXKDfhOsgU/THrwDqJ9RPmoaqdil+r3 qvdVh50qp9pkl/m2kffxhJ/t4Hm7I2CKFdNGXWJ2SSJRXBKIcdosvzdgw2yNwaDVBLishqQLz85p SNVltB6sKC6tqCgrDVRjMeb2s7H8fFjuasSqOK1aIwqHHRjkxEOSbjTyi6X7Sg6WkJIU/teOUePm Dp4MoUxGpqgcy5d9faafZPg/60Yf2TWwv4gz+3e4QklM9xdNztz+IhAk53QpVMqwSyH4sFPlzpIk PXs25CPcjZSZ47tEvY/32nJnGigOZE+lDWlBg6Sb1ZRUP+UsxGe1bZw17+bZ5/gEwZf+SlaOr7t4 dkPxkjmy9iyjxhyZstnP+k/MGNe8vrX/34P0y8y6olC8tP+LwdPJdTL5EvQ8YINNYUIMcqNVUkFA KBMk4SxhvrBSuF5QWQxcO88HDEq9pl2hCOhtbuEuqzXgZl4iKXznM26lQa9FeC+eA9cTUHTzWBaM /lYw8QXPlFVDhxK4fnmVauu/6zvlvOMQnwPysAYrLD84e5ADANlw1So8gb53v0M2eSd8S/enFKY/ /zk95eQ3wzgV6DKU5++DN9sIeF5BUrtRDBbSZqiPpSDn9XIutZp19Qstj1rIgSQu4AvCRbGCZH7F qFB9eEysPnk+f35Qd54FBy2VFhLnW2N/Dv85+UX4i+SJ8ImkenR4dPL80PkVW/mtQWWoIhhEWUam G+Ribor2O5EP+3z0oXqu3icfdwLd0zc76PMFggF3EBWWy/RSUtKSLCkpTwYKkxUmnXyjvGJtXp5O GzDRfdinJUt2E9axSd6FDbh4SyJC28fFYrPDsVgkHEiEQ+FQSKxI8hUVySBvMVtEFOQRCiJLRYhX BHGgxu221riUkZpEeU1hYSJBdDVmE1LXYKLlqZmiWRbEwfvCoWkVe/AWFIYWw/Lk6iQRkyXJriST pPToqbKA9AP+u1yzWkM4jagpgQLlxEqNULkXP4BWZ904Q4cO6E+PQDc4DjZ0p3x4K+enkY+p5HbK 7KPWsEVZ+9cC9q23tt5C7Vx3VTYHe9iSs4dp3jt03ADT8wZr8nI7ro6fO9j1A5L/6bFAzD8YPuLg gSVzZIczlKS/Y91hNCUtlEFALv/8QvYLDemQQdAheXWY/hQmmfkuywro1kA3jArBqLbcqO93hIWk OHj6iZregxt3Q5J98NSCf9DC/oFR/txLQ1sGL+EFcZlcDJR05qZT+MG58rbCMdpak74HX5JeO8QR Tv4HJygByWcwv0x3DBrbK4Ci9gJF8UBRDtQpJedZV1ivs4L41bdTrQn0pHaqI5kd1rtMpoADgWqE sGjiuFZuH8dwgjCcH8g/xfhpPvCTPOD2kRzgG8oBBpTnYWwN5mqlJy1BK2khBVKtscpYnTfKONpY axxjlIyNxmaNOaKv1O909SbYKK7EZJp7nmqee6VqpVtRqSpzN6ua3dNUihJ11RiZPg+PxqNb6kaP HlMXqLIaaZNXNOM285vmI+ZjZhaZObNkZswteWazMS9gDftkUYkCXIAEWryBgM8bCFeWZBvLuXJS 3lJcXl5SHKhskWjjgsONuLGlvrFRqg8UFiu9kaLCfI9biVUFVVINalEW+BmnX6NhVFWVleGwVWvI E+02yVdRYlttI7aTEY9XjEZoPbI6QiIn61CxWF9HnQWobl/dwTqmThhX8JQj5yaQ+fHxznjtYJbd JasdPDg2sKdiHoV+6hzCz9V+eIBngGqUlAvL4vRUsZqTq2J+zCFo9axCF46xUR9WKAWt3YfzFQU+ 7NA7qZwFQcvVZn+hgDo7QeC6cpTWoEXazJeIhajKfADP+gDE99sD2hfulpmLis7AWadMZXM6k17I sz+b6rRYZa+FbMcNieigKfvjhpH1YbL6VCL9/IIlDfP81StGz6ocN45i6ubJ5UXnNbTIxdbSwsSY Rrn5Y3mfXS4y86ataG5paa45c2b/LorN5B5pavOC/rfl8u2NMzyxc7OVIXUcsHwJYPkMwPJqvEaq ekf5jpocUB5Qk4fUvcpeNdOtWq0i81Xnqs91MZtdjyjJlb4deCdh3L7zfQRhlhAv0GtWq7P6rMTa IlitDiFgPlWry4qkPJSH81pyUimr1XEozIXJKaqdoaIlq9qV1VQr8R58BIlg01s8flYFWp7ZbNJq tKLzsIAFKlA4WcHbULIFFDyBandDSkNOt8siZ/9xEBb/1RmZ/1eaHe9yK9QqtVJNlG4FIJxL7clq dwWyduca3AHm4dK/bnfxWfTqBtUOYifoMJU50+sH2DESi36g4M1ov62jq7V6lowPf5OPZVy79Owr uofrdzlcWdXRFPOuO6P/qyH9ruPKxhv6vz4FQUC/uz1zmK0FDNEhOx4vVZttrI2325hX8au6d8iH ir+o3tEpL1AtNpEFZAG7WL1Ye75hiWmB5Ty72upnjH4No9Oo9H5E6cUo1Mt5nl3OJYO1ogdhDpWg LlD/UmSN5DD7lRIMU0owZplyn/Kg8ojymFKhTOGPdziABQ1o7iDc+vo7u6nSPPDbzhHHg/ciG5hZ fOb4To7P4+17Mh+DxP14h8Fr8g5ZVJ1UjFKylnQ2eqiGp4mJOvgsRm+9jodErYVERRMT/X2cBzQ+ Fa8zQyckNt5kr+NpYuGpwzuVOSCZoaDVgrKmpglhjL5anPV+D/t0YB4NeSiG+3lq032/PZD+EpsP /BZbpv1ty5a/0Yi37U8fw6Z99Me5x1785V8PP3D/kcOwNoVgOVPqDaNSXCjVl2qNo6IQKwqn4Gmk 03AuhjVRXmBYia8suKhI9zvlfu2fVX/WfBD9c+mnyk+0aoFJMFeqbmE2MU8ySptbJlmh2CMIbk/A lpVSOvMrI0RSQ6A4J42wIVZsrLG6awBT84r9Om3MjzeyKuSrCSsjfqMaq53lCZQneo2e7K4J6xHK zhnmlpRVu4EzNn21sgn9Yxb0zx+rGO4gyteX0EMVhfKhCoOI6aqXZv6yPRocXHN5xalz05ozo+hC RH+SpIYtDS6Y+OTFV/1pRbr/+b/d8rpMUsuGtCTmgbfv3XTo0KZ7DjHzNs2avfLgRbvSmWfTyuwe P+gVNbJCtPj2g29uuP3Ng9lfdLIzmUuBM1gl/qo8nNC0as83X26+2Xy38gGLyp01en2v5DR9l3UP eRoUY0nS5BT4eJiOaM2fHMrPD4cCcV0eL/9RRIXKgC2Iz+O0oXANiiu19RwwTtDbqfru0hpVx1RE 5SxEvBgyBtuCWXfHsaAyKCT6bxtinpO5TzuBdU7KKtl9dMc8dyI/K91H/bdHX36WYcLymXLLt8vC 59nM7gFpm1uh4YcvVD/lriLk4YeaJ14jWLR5lmBSqNq8D6+U1byl9Djaa/KhNGbeoTunLXBawCYN Otu3ppPy0phNdvJcThYezBxm0kBNTfhr6Sa+3t1AzGeiDrS46UnxyapfVb9ueXXsXy3v2t6t+3Ds vyxHk5+PPWk5nvx+rFlnUdoUdZqxPovVZq1zjV0XuCu516ibYZlZvbj6/Jorqq+uubn65ppH+F5e e1vNLh+Zoo7HgpFSaUxt0ukw5qms+lEoWVYSZIsqjXl6RosYk1AzZozf5G/UpnDFTkYswkUpfLfk jlT6/ahGNW2Uv9VLD8IwXmdL6dRgTczqlyhHtQHvlDqWxXBMaG5UMcqI1q87J+fllrcacdaLiON9 XP/RwXMxdI07hw7FjBp2LCa7SzDKnP3tXHXVWLPoDlvC9jqrD9W4RvlwlQiJeSxUbfUOH7I76saM 9tSC3HPW1Fb7Kn2IbzDJahcVwtkktykoy8OB1d9Zwye17ucynyF75gvUBCZPHV8FLHdHwFbrHvKC Ucsmtz1ZDfxYAypqDQ9JNeXODs4KNUiaKDtu4oEBN/FgLrvpfQAydNCzVAjxNBnGjkES/NgPRay8 3TbssA+vHHbYh/Zlj/tEI6FI7rDPVVnLh+6CVJ+15tbJNS0lN25rmjvnjy+/vEptNcjHfQR7cNOy h7dMOSv98k1nHtr4NBP3AKZu8DptQm20elS8ojbfbbQ4gleNv+A3CwJ8ntP7FKCvtchXUn9F0+Ti YjG5qHbJKmqh3AGSuYaelUWvSqETLmxwOV3kYe0u7W+1b2uPahWX5N2Yd1feo3m/172rU9rV9NeU TyMWXyRZ1SyrUgcwx2usJiNnMvMKQR9L4Yckk7cmFFLVYIyUer+g429iU/hxiU8k1Box4v89cnNu 0b3cvc+tAGnxyY5CahTQvyQhO6WPy857+exqf192W4li0CkMm3qjnS6tTufU+JDWpfehrDea+rbg O0DhJv5Uh36kYqR32mYF1fAVmRFXX9w97fdVvIFzGMR/d298Wj6avJkuBjOPEnf/W2fMKxcN9Nf0 /klrLybFtFH+FRWF4yyAYwczD0WBE+u17C4bybdhp9qokTmwvlit12vUAWN220nnmpzbdor6ab0Q hXCoRQyF/GIgim1GXvTXoKjW7qjxeb1GtaaGMyp5P6MTRYTsNqqvamKcSVQfVGEVdUfmn+qOrJX/ YojMb3O/pchy3lH/lYI6wG4lLZYosxVHOB3NFlapCFtYkw+ZlXwW8lkytOTI8HlkBfKzgeA0Zz7O 7a/IG7LRYeCX16ZqqDqwH3vjk69cKZ2d9RosmvzGVnkZvpJVzivvb2y/mHjlxbj1rPOfyxazXje6 BjX07/HDGgTxHKl0K95qftLCiFpRJ9KfRuWJRhG0/BpcbR5tOY8sNC3mFwe3waAnLGbJh+lvG5+W rAZk4AzFBsYwWf5pY0BrMmeFKLytDw9zmdFfNz5Nf904W/51Y0BDcNZFVu/I+sgmD7rITARj0Wzi webggwiJFp63WHiLGSNtzhnm4mq0TI1WowzW8Cl8vqSzkJpiU71pm4kx7cHnIwvWSAbJjEvMy8xb wNxnzc/jbYAzYezPnVY63tn96fHO3N+XGDz8XV9bXDzwc4Af+03i/+VHhz9+2oh6jfzBHziAyk9t IT23pX8zXXaRyL9tW4eTYVx0i9xQS/3H0xjDwA9t+8dlbYUBv0l1JsPeCSuZzySkR/JtUfuNzBO2 R+wpstu2065GhCOrbOtt22wv2A7b0jb1FtJDDhJGzaqtDtZhzScxNt8atVez1dbx7HjrDHYG325t F9rzz8MXsIusC+0LhYX5V7KXWe+13W1/lGxlH7Nuse8ie9mUtcf+rPBs/qu2l+0f2g7Z/2E7ao/r bC5bnMRtcfsaYU3+k7a9tt8rfs9/ZPscf27/npywfW830YN6gA0cVyyf0gvkTuk9LSWWhzAKiSEp xByjpS2hN0PM8tDqEKHH9kgotEk+sxfIndl7WorN0azSrKcbSj5Nq4b5SoO3ycf3oAFrNJvk43uB 3PE9wEqPp1g+uxcQBcdd8tm9zASpbODsnjh4dk8cdnZPHHZ2T8yd3dsH9rEdrwRsOkLdVviIFGTR VIyZqaw2WuN31oiWGoOyRu8XRYNBr1zmwI7fCZhuOkbQRkEqqRCk/HhSkMJRSDxeSAQnJEZTUqiR uvJx/l78G/nQ3jrJbptGpNJRSULHETqOSJwpSVL4N5JBIXZZsfV3PLuRr1FQt0hJBc12VI9KytV4 tgqPkXO4g5zD9XION6O5ZLbZkwrJWrFKsV5B6Bk/ongef4xiwyjmu87OQdndR8/xddJjfvDplw/5 dQ4c8osf/5R2Ikd97YC7WNZmj9dyR2nhf33M7wcO4c7O7u4ftv2wMXfWb8A63ZWvFtQsN6TQ4Iv8 KoaJMqecdRp+jG+gjblp0e7UoqdjlBg/o8kFd+04N7X+fOrJ/JQqvfmYuPuP4mEUeh7h+78g9w1R Kf1UDYZ1w8LHeJQcFuPraCAPMzHmP+xbiguVnhFhtfIVVSEN6lZ1q8Y4LGzVbNWN1o3WP2C403Bn Xltem3E5gD5Fg3mp5bNTA/8J/4n1ftsE2z/s9ziWCZcIfxf+7rrF9bFnmvc8X1I0Q/gwUB54L7gp dHVkyulwOpwOp8PpcDqcDqfD6XA6nA6nw+lwOpwOp8PpcDqcDqfD6XA6nA6nw/93AWX/E0z2Px7y iJH/3aETopLuvUxsn87ozhabWiaPn2E8a4qem9Z2ZunMio5y3uK32uyjz6hp9VWGIkXjHAnFhBL0 /7cPi66WU5bC51hxJgMppin9F16I7k1NRO1oOkBNh85GImpCLWgyGo9mICM6C01BesShaagNnYlK 0UxUgTpQOcDYgvzIimzIjkajM1ANakU+VIlCKIKK0DjkQAmkQBNQifwUM6wN/QdhSngCalx28UWL F1wkTl5wKZJ7Ed4AY//bj3pk9Rg6lhnRkPuvl+wb8n+LXig3aYcC6cSFcoijPXL4BB1FW9Hv0Lvo wIgwDy1Ad8rBBGHVsPAJhA4IDnjTIrQDQjvajdbLIQxwOjVsgLAV1mA1eg/9AiD5PIR9aC9Abwm6 HRVC2IoOojvQLIBidRZ3f+ZD30/hjm3v2bZnjrH2W7UmC5CHwlPm0vylX06JZlT/2cCeUNN/j6QZ gMf/A47Xo3kKZW5kc3RyZWFtCmVuZG9iago5IDAgb2JqCjw8L1R5cGUvRm9udERlc2NyaXB0b3Iv Rm9udE5hbWUvQU9RSllIK0NhbGlicmkvRm9udEJCb3hbLTIwIC0xNzggODU5IDY5Ml0vRmxhZ3Mg NAovQXNjZW50IDY5MgovQ2FwSGVpZ2h0IDY5MgovRGVzY2VudCAtMTc4Ci9JdGFsaWNBbmdsZSAw Ci9TdGVtViAxMjgKL01pc3NpbmdXaWR0aCA1MDYKL0ZvbnRGaWxlMiA1OCAwIFI+PgplbmRvYmoK NTggMCBvYmoKPDwvRmlsdGVyL0ZsYXRlRGVjb2RlCi9MZW5ndGgxIDQyNjI4L0xlbmd0aCAxNzU1 Nj4+c3RyZWFtCnic7b0HfFzFtT8+c+/2erc3bdNqdyWtpFXvlla9WbZlWbZkW7bkbuNesHGng4HQ i+kQMMWAJdnGsoFgEgcCiYEQWhqBl0IJJkAIzUj6n7lzR5KN4ZH3fp/f+73P3xp/93xnbtk7Z+ac OTN3FhBGCOnRTsSjKZPb47lI/LtpCnxMn7+idzXN3wjAufPPXR/ou+3YK5D5A0LypEWrF6/4/PNW HfAvEVK7Fy8/bxE9P28JQjVTlizsXfC38PAEhPZMhMLCJVCgf9Rci5DRCvmUJSvWb5K+70Eou2D5 qvm9NH/uIEK++hW9m1YHa/ib4VgxFAZW9q5YKJ2fDx+u1avWraf5PRnk+Oq1C1efs48bhvM74fYC Qso7ERq+Ho3/m4KWoXVQ353oEnQVuh49jX6P5qELge1Gd6M96CHUh55Bz6M30P/Bv+Hz5CuQjj+E FMiC0MjXIyeG9wAG5YZxJddDziILjJWMCCMfnVb20fD1I8LwoMKMNOK1eu4VKP0nHhr5mqsk+ZFC kucuBW4Ur/hEeefwvuEHTtNBG5qJZqHZqBv1oF6o/wK0BC0FzZyDlqMVaKWYWwnHFsPnIsjNhbPm w1mEj521Cq0GrEXr0QZ0LqTVwNdJOXJsjZjfgDZC2oTOQ5vRFrQVbZM+N4olW+HIZjG/CbAd7YCW OR9dIDImacmF6CJ0MbTapegydPn35i4fZbvQFehKaOcfoau/k191Su4aSNei66A/3ADd/iZ0C/SL 29Dtp5XeLJbfiu5Ed0GfIcduhJK7REaOPomeRQfRY2gfelzU5XzQGtUI08siUYerQQdboYYXjnti qr+No9raDnUnddsl1XQTlF8w7opzJT2SMy+EM+ldaDuQu2w7TRPXQB0oH6sRzd0o1n+sdLxWvq+U 6eP2cZq5TcwRdnrpd/Gb0B1ggffAJ9EqYfcCp+wukY8vv3P03LvF/I/Rfeh+aIsHRMYkLdkD/AH0 INj2w2gvegTSGB/PqHwMPSq2XB/qRwNoPzoALfk4OoQGxfLvO3am8v1S+cBoyWF0BD0BPeQn6Ch4 mp9CYiVPQdnTUukxsYzmf4p+BnlyFs09i54DD/UC+iX6FXoJ/RxyL4qfv4Dcy+gV9Bv0BtYD+zV6 Hz6H0MvyvyADqgI/fQT0fDuag+YkGhbMndM9e9bMrs6Oae1T26ZMntQ6saW5qbGhvq62proqUVkx obystKS4qLAgnpWZkRoJp4SS/U6rSTDqtRq1SqmQy3gOo4y6UH1PoC/S0yeLhBobM0k+1AsFveMK evoCUFR/6jl9gR7xtMCpZybgzEWnnZmgZyZGz8RCoByVZ2YE6kKBvuO1ocAgntnWCfyq2lBXoO+E yFtFLouIGT1kgkG4IlDnXFIb6MM9gbq++nOX7KrrqYX79Ws1NaGahZrMDNSv0QLVAutLDa3ux6kV WCRcal1pP4dUevK1fXy4rndB35S2zrpaTzDYJZahGvFefYqaPqV4r8BS8szoikB/xtFdVw4KaF5P TLcgtKB3dmcf3wsX7eLrdu26tM8U60sL1falbf6LE6q8sC8jVFvXFwvBzVqmjn4B7pOHhVBg178Q PHzoxIenlvRKJYqw8C9EKKniqJrgOOMIng2eEOoXDJJnuWIwgeZBpm9nWyfNB9A8zwBKxGNdfVwP OXKUHbF1kCM72ZHRy3tCQdJUdT3Sv3OXOPt2zgtkZoD2xX9h+AfHA318pGfe/CVE9i7cFaqtpXqb 1tmXqAWS6JXqWtefHYfze3ugEkuJGto6++Kh1X3WUDU9AQoCpA2WtneKl0iX9Vlr+lDPfOmqvnhd LXmuQN2unlr6gOReobbOwyhv5O3+/IBnfx7KR13kOfrsNdAokbpdnQsW9fl7PAugfy4KdHqCfYku UF9XqHNhF2mlkNCX9jZ8XVD8RvEqqNtpZ7OTSc2VYVWgk/PwXaS1oCBQDx+h6nI4IEBziVnSotXl gU7sQew0+BbpDMJOuQ9k+HBNIznEk0trGj3BriD9+55H8kjPJA/3qcbdS4CC0Wei3/Odj0bPJg+U FqhbWDvuAU+5qVx6QOluZ35OjuhC+mK4QkWas5Ed4sNguVDGwW3EItKKzkAfmhLoDC0MdYWgDyWm dJK6EV2L7dvSHmppm9kptrbUS6adkqPHi2muDwXhMMtwNdAH62Me1qxivkHMj2YbTzvcxA4HdqlC Le27yM1D0g1RACwIKq2INPVeUWzOB9OsB+8Wqu8NBYRA/a7ewZGd83b1JxK7Vtf1LCkl9wg1LdgV au8s94jPOrVzm2cz+SozasEt06ozM8D3VPeH8GVt/Ql8WfvMzsMQ4AYum9Y5wGGupqe6qz8FjnUe DiCUEEs5UkoKSSZAMuROUyGjEs/3HE4gtFM8KhMLxPz8QYzEMhUrw2j+IEfLBFbGQZmMliXEMvIH jeRcAioGd1sXWECaZ2vXkl09XcS4kB2aEv7hPhyqQH1cqKIfcwpdnya0sLpPG6om5ZWkvJKWK0i5 EjoGtmNQDvFJu3pC4KegQ3UiD6ZdkSe3DAyOjEzrDB73nOgKQlebDZjZ2aeOge+Xh5vhvAaCHihu 6Ns5v5c8B+roJNcqw03zu6DbshvCKU19ariDWroDnFEvXkO6I1w0H9oGGlC8fidk+nZ29XXFyJd2 Lu0Su7PQhxpDpdDs9J7yCPmieNcucyhXtE0wBU34UiLU8GyovZOWeCALX9ZFlaTUwZPPD8Gh+T0B 0LYMzW+Hrk59qcZDSxaCS5RFForQeKSDiFSLD2v1mj51FtwQ/hGuzSImKQ8ru7row4u5S6UT4LuF Pi08UWScKqULQDtwqIk8C/y7FB6VnPoMuU3bIJoa2gSehTy0eCclHO7Th5t6wfnT67VQEipmF6uI j9BK9zhGS5Wk5jrQOx+eNjjyQOi84Li/zIwQGRxIx0Sew9CxUdeu0wv6ZsUyM1Snl+rF4l27VPoz X0D1pdKPSlIYqINRA06EObECDSN8THP3ya+/vlv9ISkZ/8f/kpQYk3EACRCbKcEeBBRHMDs1XTMy Aq2DB9R8YJC76IDaiZuBXMjIBYycz8hORnYwsp2RbYxsZWQLI5sZOY+RTYxsZORcRjYwsp6RdYys YWQ1I6sYWcnICkaWM3IOI8sYWcrIEkYWM7KIkYWMLGBkPiPzGOllpIeRuYzMYaSbkdmMzGJkJiNd jHQyMoOR6Yx0MDKNkXZGpjLSxsgURiYzMomRVkYmMtLCSDMjTYw0MtLASD0jdYzUMlLDSDUjVYwk GKlkpIKRCYyUM1LGSCkjJYwUM1LESCEjBYzkM5LHSC4jOYxkMxJnJIuRTEYyGIkxks5IGiOpjEQZ iTASZiSFkRAjyYwEGQkw4mfEx4iXkSRGPIy4GXEx4mTEwYidERsjVkYsjJgZMTEiMGJkxMCInhEd I1pGNIyoGVExomREwYicERkjPCMcI5gRJBE8wsgwI0OMfMPISUa+ZuQrRr5k5AtGPmfkX4x8xsg/ GfmUkU8Y+ZiRfzDyESMnGPmQkb8z8gEj7zPyHiPvMvI3Rv7KyF8Y+TMj/8HIO4y8zcifGHmLkT8y 8gdGfs/I7xj5LSNvMvIGI68z8hojrzLyG0ZeYeTXjLzMyEuMvMjIcUZ+xcgvGXmBkecZ+QUjzzHy LCM/Z+QYIz9j5KeMPMPIUUaeZuQnjDzFyJOMPMHIEUYOMzLIyCFGHmfkICMHGNnPyAAj/Yz0MbKP kccYeZSRRxjZy8jDjDzEyIOMPMDIHkbuZ+Q+Rn7MyL2M3MPI3YzcxcidjNzByO2M3MbIrYzsZuQW Rm5m5CZGbmTkBkauZ+Q6Rq5l5BpGrmbkR4xcxciVjFzByC5GLmfkMkYuZeQSRi5mhIU9mIU9mIU9 mIU9mIU9mIU9mIU9mIU9mIU9mIU9mIU9mIU9mIU9mIU9mIU9mIU9mIU9mIU9eC0jLP7BLP7BLP7B LP7BLP7BLP7BLP7BLP7BLP7BLP7BLP7BLP7BLP7BLP7BLP7BLP7BLP7BLP7BLP7BLP7BLP7BLP7B LP7BLP7BLP7BLP7BLP7BLP7BLP7BLP7BLP7BLP7BLOzBLOzBLOzBLNrBLNrBLNrBLNrBLNrBLNrB LNrBLNrBLNrBNfsJgah5wFfhh5h5wGcDcQHNnT/gKwWxk+Z2ULF9wKcDsY3mtlKxhYrNVJw34K0C sWnAWwNiIxXnUrGBHltPc+uoWEsL1wx4q0GspmIVFSvpKSuoWE7FOQNJdSCWUbGUiiVULKZi0UBS LYiFNLeAivlUzKOil4oeKuZSMYde101zs6mYRcVMKrqo6KRiBhXTqeigYhoV7VRMpaKNiilUTKZi EhWtVEykooWK5gFPE4gmKhoHPM0gGqioH/C0gKgb8EwEUUtFDRXV9FgVvS5BRSW9roKKCVSU0zPL qCill5dQUUxFERWFVBTQm+VTkUfvkktFDhXZ9GZxKrLodZlUZFARoyKdijQqUqmI0ltHqAjTe6ZQ EaIimd46SEWAXuenwkeFl4okKjxUuAfck0C4qHAOuCeDcFBhp4U2Kqy00EKFmQoTPSZQYaSFBir0 VOjoMS0VGirU9JiKCiUVigHXFBDyAVcbCBkVPC3kaA5TgUSBR6gYFk/BQzT3DRUnqfiaHvuK5r6k 4gsqPqfiXwPOaSA+G3C2g/gnzX1KxSdUfEyP/YPmPqLiBBUf0mN/p+IDWvg+Fe9R8S4Vf6On/JXm /kJzf6a5/6DiHSrepsf+RMVbtPCPVPyBit9T8Tt6ym9p7k0q3hhwzADx+oBjOojXqHiVFv6Gileo +DUVL9NTXqLiRVp4nIpfUfFLKl6gpzxPxS9o4XNUPEvFz6k4RsXP6Jk/pblnqDhKxdP02E+oeIoW PknFE1QcoeIwFYP0zEM09zgVB6k4QMX+AXsliIEB+ywQ/VT0UbGPiseoeJSKR6jYS8XDA3bw1/gh epcHqXiAHttDxf1U3EfFj6m4l4p7qLibirvoze6kd7mDitvpsduouJWK3VTcQi+4meZuouJGKm6g x66nd7mOimvpsWuouJqKH1FxFRVX0jOvoLldVFxOxWVUXErFJQO2XhAXD9jmgbiIigsHbItAXEDF +QO2DhA7B2zgjPGOAVshiO1UbKOXb6XXbaFi84BtAYjz6OWbqNhIxblUbKBiPRXr6K3X0svXULF6 wDYfxCp6s5X0zBVULKfiHCqWUbGUXreEisX0yRbRyxdSsYCeOZ+KeVT0UtFDxVwq5tBKd9Mnm03F LFrpmfTWXfSLOqmYQR93Ov2iDnqXaVS0UzGVirYBawLElAEr+YbJA1bSvScNWC8E0TpgzQQxkZ7S QkXzgBXiAtxEc41UNNDC+gHrdhB1A9ZLQdQOWHeAqBmw7gRRPWCuB1FFRYKKSioqBswwvuMJNFc+ YOoCUUZF6YCJdI0SKooHTA0gigZMnSAKB0wzQRTQY/lU5A2YMkDk0jNzBkykYtkDJmKbcSqy6OWZ 9BsyqIjRm6VTkUZvlkpFlIoIFeEBE9FSChUhes9kes8gvVmA3sVPhY9e56UiiQoPFW4qXANCNwjn gDAHhGNAmAvCToWNCisVFirM9AITvUCghUYqDFToqdDRM7X0TA0tVFOhokJJhYKeKadnymghTwVH BaYCJUaM8/wEw8b5/iHjAv83wE8CvgZ8BWVfQtkXgM8B/wJ8BuX/BHwKxz6B/MeAfwA+ApyA8g8B f4djH0D+fcB7gHcBfzMs9v/VsMT/F8CfAf8BeAfK3gb5J8BbgD9C/g8gfw/4HeC3gDf15/jf0Of4 Xwf5mn65/1V9xP8bwCvAf62P+V8GvAR4EY4fh7Jf6Vf4fwn8BeDPA/+Ffpn/Of1S/7P6Jf6f6xf7 j8G1P4P7/RTwDCAxchQ+nwb8BPCUbo3/Sd1a/xO6df4juvX+w4BBwCEofxxwEI4dgGP7oWwA0A/o A+zTnud/TLvZ/6h2q/8R7Tb/Xu12/8OAhwAPAh4A7AHcr8303wfyx4B74Zp7QN6tPcd/F/A7gd8B uB34bXCvW+Feu+Fet0DZzYCbADcCbgBcD7gOrrsW7neNZpL/as1k/480i/1Xae73X6l5wH8xH/Zf xBf7L8TF/gs6dnacv3dnx46ObR3b927r0G7D2m2ebS3btmzbu+332xJmhWZrx+aOLXs3d5zXsbFj 096NHUe4S9Ai7uJEece5ezd0yDZYN6zfwH+2Ae/dgGs34OwNmEMbhA2BDbxufcfajnV713agtVPW 7lzbt1ZW1rf27bUcWos1gyNH96/1+OpBJrau1Qv1azpWdazeu6pj5aIVHcvgAZcWL+5Ysndxx6Li BR0L9y7omF88r6O3uKdjbnF3x5y93R2zi2d2zNo7s6OruLNjBpw/vXhaR8feaR3txW0dU/e2dUwu ntQxCcpbi1s6Ju5t6Wgubuxo2tvY0VBc31EHlUdJQlIgiRfIA0xKgidBHlyd7Ul43vZ87JEhT5/n qIc3G91+N5dmdOGayS68yrXDdbWLNzpfcnIJZ1pGvdHxkuNPjn84ZJaEIy2rHtkFe8DO20jd7K3T 6kVZWUtlToFY11Z7KFJvtGGjzW/j6vw2jExvmz428banhZcEzmjERuOIkUsY4XSjwW/gyMeIgU8Y corqjXq/niMfI3rentBDCbljVDdlWr1R69dyHZXayVouoa2sqU9oM7PrEY8DGCMsgOBV5CmwzV8P dr3fjuUYxvP+ae2xWMugCk1t6VNNmdWHL+sLt5PPRNvMPsVlfahj5qzOfox/1NWPuZppfVby1lfM X3zVVaja29Lnbe/su9vb1dK3E0iCkBEgyNtvR9VdsTnrNqyLxdbPgY8569bHxH+QwxtILkYKyb91 6yFP0gYxj2Lf+0dPAzF3HfytZ4Xrv/+q/9f/8P/0A/zv/+tHZKNC1Qh3EVrAXQi4AHA+YCdgB2A7 YBtgK2ALYDPgPMAmwEbAuYANgPWAdYA1gNWAVYCVgBWA5YBzAMsASwFLAIsBiwALAQsA8wHzAL2A HsBcwBxAN2A2YBZgJqAL0AmYAZgO6ABMA7QDpgLaAFMAkwGTAK2AiYAWQDOgCdAIaADUA+oAtYAa QDWgCpAAVAIqABMA5YAyQCmgBFAMKAIUAgoA+YA8QC4gB5ANiAOyAJmADEAMkA5IA6QCooAIIAxI AYQAyYAgIADwA3wALyAJ4AG4AS6AE+AA2AE2gBVgAZgBJoAAMAIMAD1AB9ACNAA1QAVQAhQAOUBW NQKfPIADYABCCzCU4WHAEOAbwEnA14CvAF8CvgB8DvgX4DPAPwGfAj4BfAz4B+AjwAnAh4C/Az4A vA94D/Au4G+AvwL+Avgz4D8A7wDeBvwJ8Bbgj4A/AH4P+B3gt4A3AW8AXge8BngV8BvAK4BfA14G vAR4EXAc8CvALwEvAJ4H/ALwHOBZwM8BxwA/A/wU8AzgKOBpwE8ATwGeBDwBOAI4DBgEHAI8DjgI OADYDxgA9AP6APsAjwEeBTwC2At4GPAQ4EHAA4A9gPsB9wF+DLgXcA/gbsBdgDsBdwBuB9wGuBWw G3AL4GbATYAbATcArgdcB7gWcA3gasCPAFcBrgRcAdgFuBxwGeBSwCWAi9GCqp0Y7B+D/WOwfwz2 j8H+Mdg/BvvHYP8Y7B+D/WOwfwz2j8H+Mdg/BvvHYP8Y7B+D/eO1APABGHwABh+AwQdg8AEYfAAG H4DBB2DwARh8AAYfgMEHYPABGHwABh+AwQdg8AEYfAAGH4DBB2DwARh8AAYfgMEHYPABGHwABh+A wQdg8AEYfAAGH4DBB2Cwfwz2j8H+Mdg+BtvHYPsYbB+D7WOwfQy2j8H2Mdg+Btv/n/bD/8v/uv6n H+B/+Z9z7hwkR2h4Hf+K3IB4pEQlqBVNQrOeRHro0nZUig8etNXWqjKVP4HuyqEAdHgVwrgmYZRx +kNud2XoUIHiKt7UBJP3A5XKq8CVVw69NfRifOitE+aS+Akc/+M7b70jfPKiqSSe986r7+RkY1PQ JMJq4JRKqyKUnMUVRCOFeXm5FVxBfiSUbODEsvzCogo+L9fH8VZWUsGRPOZf+WYmP3lIwW0PVU7P k/vcRqteIeeSnObM8rDQPitcnuVV8koFL1cpU4uqk1uW1yX/Tmny2uxes0pl9tptXpNy6Pdyw9ef yg0na2TLT97AK8pmV6bwt2hUnEyhGPQ5XellwabpRosg01oEk12lNJt0qbWzhy6xJZF7JNls9F5D raCW0MjXsu1yK0pGEXTHYZQy8t4BnYAnhgYlEhkc+fiAFoiWEZhTfZxwExYWyKde/NSJn4lUHCaH M7S4NSUUCX+m0+qcyd6QRo/tMh3SCTpuX+jp0EshPqQL6czeqeYOeQeqrKw0l5TE493dJkeJCagp TziRa8oDjce6aXNDtB622xWiyqN8kDfwoeRIpLAIUz07lCE+KNugwkLY7w9b1LJVQ39bxmssoSRv 2IhVeECmd0V9gXS3QbYF/wn/dILdY5DxSp0alw0/r9arZXKDxy4b0BpUPK8yaq8a2kJ2rPWOfCzT yX3Qs+btT0JlMdDJfgG3gvx4v1GUH+7Xi/Kj/TpRvrcfKh77CcQ2BuTEcRREEZwxYGmXPYHTUQHK xln96unQzV49QYDj74iVE14/lpMdthoU47qKwiZ1HdKpbFYfR/oYqapMx8lV1sTcLU3bf3l1a/tN v95RvGxmvUcl52UqrcqQO3nN5OlXLSgqmH/NrNZ1bflGpUbBHxKcZoM1LeqZdt8nd9zzzb7ZtkC6 x2Bxm61JFnU0Hq275JmtW57aURWJRxQmH/SKRxCSXQ12ZUZ+tDHhrQxiixNqbhGg2hYr1Nlihgpb nFBbyxMc+YWbm+rGLelGlHpRfk5045Z0434C4i016EY3YGjzDOJIv3waqjxROaqLV6nIye4mVhYK JkcKTPmFeUGouTIftBEyEUXIrp5+/8d7hj9ypKU5cPjB9+5oO5i/6uFL9vVvfXhtCXfrgyfvn+qP yi6I+mf8+L3dSw9e1PyNqWLnM6RNoWb8VqhZBjq33x2VWjQqPXVUeuqo9NRR6amjg5wpoVZbApYA PLx7EKsS+p0RfDSCX47gSEThIgtj+rYoiH4FrQ94kO41a6FacbFrC7RauWI7n1otsaGDptMov1Wm 0auGric15Bap9Cq5HD6GFXhABd1VpgY+icMqvUbWYPaYVbS2KrPHavaYVMPL1EKSxewWlMM5KpNH rPfI1/w0qHcUze5XWqR6W6R6W6R6W6R6W6R6W6DeB/Ve5PMqoWr7LRaXYhCn7k9ucxGjlbxk/Jip ZLR2+FuVYR6QVZefBhVTDoP2lPDwIk+orAG3M9mqgqrWi6XHLElQi0al4LFZPCb10F+VeqVcDh+y x0gtvVKNcCd4LhuacqjSMdmxz8EjqV5IqheS6oWkeiGpXugI9ELNyNFDNtyqEaaKLgjHY6NdLzz2 0NTobLhTZQ26yDOqbUGHK2hVuVU68kg6lex3jJGnUo58hP8CT5WKOg8j9G88jhcex4RbvYbQVPUT OBdZwEiy+uWStwA1jz4epk+nYIOKOPqMPelfkmpXTU0qykrWKuUcDz5B5Qpl+ZOzAwKtgkWN61t3 zsxRG006nclltsOIYjQbTVltVfydpD4yqI/UY76EmuSheQlTjgDPmU3sPU5YUCNpWiNVTSNVTSNV TSNVTUMsR2eLTg1qBM9UYczbVzKDh04Dn1TjkUgUm76le5Pk5G1WhRJju53/UmlN9oQy7MrhFNYq Kmuyk7QKfkEhOIJud8Ci1JuH2/GLJmUSMR6FoOEuHTpvtBuNttnQM1ylWqeUyaFA73YMjQzd6rZI fqIFau9GjYeRjVbWJlXWJlXWJlXWJlXWRvYmIrVxqm0QxyRHgOPHWbuNs3xWM9EgWsCa1UPHHGmj lXiZDEktVo9FDXb9GHvUk/eoTUlSz1fEwJbL0SMJoadidQWnz852xOOaLKfTPfgDHTFpGF9Kjk6n IZ5dQzy7RiAtqCHNR1paQ7olGjmacJE+mlLYpnU69HFnTpbCn9rm72BDd6UZBu08qOirUg+FkVsY ZaaSCfG8PDKWj7OqECbjN4zkOHSKfxCHcpxH2lvUjyKmsvpdjqBFxQ3n8Vqb12rzWbXccAMGj+Fy QiNneJYEslOcarxRji/Ruv0R1wqjx6IbM87FJ29QapS8DIZBCJZ2j5bvSU/RuVM938zg9/jSXVq1 xWujmoVoyIQmoIv3R41Gq6RMURolqRflx0SZVkmZVlGZPk1WVi5RZq7TSD7gxFxBRxickktOEZCv eKomyxiVuYgPJT1EVB9R3rd0F8+TugzVFNhGyG63nUFfPt6RFxnXq2Tb9Ta3vsgdDYVsw0sCVUkc x6ksfqfTb1ZluKd6o36vCZd6C3NznBiGEIvfZQ+YVQ1WiA613two93bJtrLGm5q/+eeotTycmqxx pPmHfpE/v6c7PnnvZO4nEDvBKASOgvxuZeSE7D15EFxWFG1NuK1EB1bSoawkVLCSUMHqpGrKS6gD KFv8bbxPUq5P6qk+KZzySeGUT1Ku7wkIpzTIhdMGjO0hYlnEKY4PGbpP84yj4bYYMYyLn2TvNV// 1g3XvXZFbfMNb91w9atX1R2Mzrpl9epb5qZFZt68ds2tc1K5m+74pn/ujD2f3737631zp9//z4dW PnXFpGlXPrF47dErWqdd/SSJjsAzPgf2l4TS0Kb+FIVUEYVUEYVkcgrJ5BRSRRSkCzhMXqIeL1GP V9Dp8URvAI55ySYbZAoPYs1+hUIH1dTut7Xpxg2ztIMIp460odOHV9m4IIl/LrHx0U3Xqy1BF/Eq 6W5sS29dumJi2sGyGd0Zd902aXF9Cn997+0ry4ezRu0CmlrpqJx93ozJy/INQ1+lNsxHtMYyLdS4 ENWiaxM+IctUpIKnLiK1KBJrUURqVURauQha+VBaArJplSaiCmAmSTUmSTUmSTUmSTUmsvkmKUuA yOrx1QmcSDgmgAYOBtsckpMR46kTJaODBvM1MJJIViJODLL4b6nE7vDxJBRRgplY7HacH4lGIiyM 1CqsKT530KqVbbRlVkwrW8eUBWGlJafK3bJuUjRUPbskkJ+Zal1vUA0P1U5xVeZd+2Dt/Go/OBkY LNVg4jn5MypDQ78dVSIEKXJeXzx9VU3V4smlVkOsfFLO8J9TvPzFE5c6lIrhicGyKeBtGkZO8PPB bprQu4dRFUy4jDCFqpJUVCWprkryNVWSqqoGuYxELDdhseKJuQmIGFJyU3J1Hie51kMcuEcQyAdc 4iHN4TnC5RAvvt8jBhxH97skaaXycaMJT0S6rCdwFBUhDY4ktKZAES5KaHV4oom8+dEQVmQqMtnL IQo8WOWRp7XbB3GaZIfQBCdMJMaNxbqFEwLpqqRlRtuHHDjNQGWnhC75o6HM6ZMeBT+/ZuM93VWr ZpQ5tBCWqAx5U9Y0F3fXpOROXbpyydS8sqXXTovNaC23KGQcr9AqtfHa7tLCKfnu3PZlK5e15+Fz Zv1ofq49kOwM+2EGrUxODfmKpuQVTSrLyauYtmZy247pmUaX36I1OS1mmAslhbze7Opw4aTy3LwJ 7WugjYxg629Az09GCw85E6Bep4lo7QAJ436w4ZOB1DRy9CDp+QozCaG9km3nQtj5iaicn8eEY7HR AHq0DweZOxNDhTfEwP8GFvUAkyYG/EXitECMm0/eOdoR56lMSRYLne6TyGECRA5vQ1RTjhbvj5Tj 3MGRLxM1pHnD8CAqQlLjGKb1pCSMk52EpCVjZ4CQzBycmY0zU3BmCBdNTZ8aytby46fwMI5VwjAP f2TaLqXw6EjPMwbz9sJxI/04BjN8pfxCmZCU5vPHkgyy4U+4r3mDOy0QzEgy8sMPK7ApEvCnWJQc DmFs5dXWsC8paFXzOI3DXl5hCXl9IQHLIwYTGZ1MBv7X38QZl+11wPSfVxm0J4/JSrVGEugatSef lZVpgMsNbgfxdLPBJiv5FyDqTaC+RMBY7a+OV/NatSNfBy2aTwwsn5hVvkAMLn8Qf5GASVXUiLAO EW+ISiV7LZVihVKpExApGnjpIKdKWE2On6N8IZ8rO5qPUT7Oz8+qSh/EnoTx5WScnCzzfpDVPOEP ulYZirPZ5AmTOKec080GvmOxOd0l0swyF9zgHIiwyAoJxAIFirG1hLwCaRSUSmRikKCkJmbPyy0s 4iuFJI/bbyi7tq1hXVtmxfoHl26150wqmdDblKNTwUCv9FRPX5Tfe9m0yH1X1S6o9ndNqVo1wanT wUilm1lZH65fVDVxdXO4Pn9Kgccb8qoEl9HldYe8loyO7dOOOTIr0+rbq2tBu7tBu6/J16B0EmEd hEmXJlgoGUuhZDyFkr5IXtRX4SD+MuGxxUgYEQuQ9Rai/xgZb2KCuAzDaRJqZNMUFgRl8uxBLH88 0uypFyaWAO2XtxIfRQYQR8lolDWms24WMkRt35530M7JggilyW4Xh9XX8uZf0x1rqq+PwnzbBmGT QmkJOF0QQ6W2NDamzrtiRupjtvzpiUBFoi5au7WmorPIhd/d8MRF9aZIadpKME6ZDIxTXqyiky3V 0F/TikPCpAv7NtRdsGCCOb06d3h3+4zy+VvAYmeCxgL886gAXd6fJPptOqF8W5pIvneABOdnWMj4 6NQFjJEP6MIGp03o4wZscL3rT2j0jf6UQcwdsDTzf88hXk2tb8zJGMSKfnUrmXfGTogfOC5FWMdG lzBOW6pSUKetGL9QxQc4udJV3tIZ771pYUHVmt1dsbbaAqdawZn1xmh5R+nGHcFEd3nJ9MqYjoTo 95pcJr0r7DUntuzfcPHTm8sEd7LTYHGao/5gavDQYzMu7IylxEIqi5fYaQ/o5Xb5ChRBJeiKhL+y DGs9JcQ6S8j8pYSMgSWkd5SQzlLyBP4KIRSnWotLyopLyopLFhuXlBUnHUpjCdZrS6IemSGdbGJy NoOpy/YbWuUTidsWu1PlaWtWYn8aneSMN0EIQkZ7FR+JjA9Ji/jblaYkK1mabdg9a/6VM1Jz5107 d/KFCaXVT/qUek/NttpK6EHQo6qCExL1URfrQBtbp7de2D9v/RMXNdTVcFoWrQ/VQd+ZtzVRe8FC 6Es1OURb3aCt3eDVYigfPZZIjxdWFq4q5C3EmiwBsgBkCWaQiCGDaCuDqDFD9G/QF746WBu7L8aR Rc+DxNryZVLnk0l9TMxrRUkdnIzoLxjMeG6n7BoZd1SGX5ZhmSwp/odIs/ODHsNqA2dQf5AkdrDu 8Stl1Cj/GKOdDYqlZUBFKDiuW9lO7XycLVooKlTJ7466hgZ89avbEgua4jqlVsFzvFJbOH1NYtUD a0vL19w9f9mNPZl7+PM2TphdkQyTomiwZdP0LJvbpjS4zHqLUad1OS0Vmwc3rz98fl3tuts6LRfc kDVxYREZOcMjX3OXyDfByLlgwC4QAxQNzyN5LQ/zVh7JnXmkzuQh27Wz08ODIy8nzAKEWmHNicIG d+REdmNgotAoxra5ZC4TO5b3CbWxvGNjMxmxq9ik1Y/xsS24eebdRT3IuEtkcpVCafOlecL5AcPz Kq1abjY+rwLXBBNl1Q5BIK5mR6hxRXOoOkWn4uVGi8MgV2vVzry20nlKk9uSEvjm7yot8UlaFW8L pFjcJmX3nEunp+mNOosHwZytYPh6/nL+F6gCTUJz0csJmzmzgVhZgwqq3BAQLHhiQ14lRBVEBZWS fYF8+3FyqFI5GWhCbzTjiZM9MmM2n6dUkt4jiPo6mtADycxTejzKvEwZ0XEinyi5k3xFZ0CAyzrT wwktyLAxW8kXN/9O1/6ezdZTzL9f3pgeqP5tcfOs3wYmSwuwleKIeeJ16vpjeceJch0Qj5KI1ASF wvEY/IuxD6J10DFMsuliVFQB/szukOYPrM8VwfCaXyh+UsuGKQbOj4wOpxWcBaYYUQMv5fjLLcbz Q0m53TsnFc33mB1VhX+vWT01K/+cPWtW7J6XIQRzAjnx3LA/JX/2+RPTGvxYMJmGhxd2ZzfEHQtn 5TTGHe1z294PpDnVF53bsrDCw68P+VNmxCdtas/w2s1ZvlAWp+GCE7rKKlZ35IQTXfnBiuI8l2ti xoSeSLi7unXztEy1Kjj8yezFgeKm1K5F/qLGoTmllZzKlZmWaquq8WZXkP69G+Lcu2FkzkXnHajM x+ljS8BSxx63NiytFcOw7PBpibvVEo+hJb5DK7oNLTmmQQmyxulLd8G0TnEoszml3jVRdJ/idG50 RZMOxiWnLruKo4nyDGuBNHS08XerzHTMdWY1ZVdsrYWsuCDEhuKGa5pmbpkYdLH+zBlb59SmdHYM XcFKxo+/LU0TFl3eSzzlxSNf4zZ5HNlQEF15qDI0ObQqxNulWM4u6UDMW0Qpdl671NPtktLsT3Br UBKyfdcyoaRSG6jpcY0/AVeSjcsHXEKTqJ/XT8QkbyiNLGdek7aQYZd0RuiFuOJ0BVgyykpjBKMq 4C9iq7s4uzQ9rQQANR55bfh6vABqnIKy0SX7J+eSN3hisADyU/LcYebYyas9UoEw+Q1XTIek88Yt Z9N6ja5rg+9LaFwulJtF6pgFddyf6m+ywkjaLxetFGpqystj8SytLdRVfso00X7qAvcp1W7zJRY0 BDKdahnmlWqlIuQIxn0G5vSIDtJjZWXpxgVbpsVUGr3JrCdvReTWzMYmfu+31UHtYCvYQT66MaGr LMRpOTgnYcatEB69LFYuRxr+ckjtdaIUh7+cJ7gozBJ1kg50kpnoJOXoJJ3oiGm47ZmZiKiEmog9 WStPbUqqNzHzgFkzjkOwBdG9OCbkvs16wWg3+EEL5VtVlmS3J+Q0KoYvOr1/4GkqsyvZ6Uq2qfXG 4SN4pV4rLmjwSr0afzqs/7aZfPMKPlejV/MwqKp1TmH4yHDYZJN8B64AndlQQnwTs0p8E3PmVx1j fQR/eUAj1Is1ljrAmd+8fKtnu779aNJTyF+GGGcK+iDhMZO3FOL7yYg4m42KU9nVU3H9t99x0XWW ce/CPhj1bz6fnaxI+nLpqri4Pi4ujYtuTgP9+9AUsgYzpeLbrwzpbb/1avEJ/CU4WQErBlqaIfhW JPRVzRX1mcVNmRNd49p//AJnibTaZSphL4GItxQ3sH6fy/wuH2qTpt9SZ5G/TF2pRWXNqM0qWVdH rMcRtCjtGTVZJetHPavCnOSwewXlxKubirtqs4XMtpaGlBnnNvnHfGyo5DQf++0S/iIITHherVVt 7Jjsjlel5tSmW8D5TmRjELRgLrohYaQtSD6k4ej0VvqON5ZksujTkvifjkokdqCDlDg+wfFD0sBE hqWEJrM53ZXSxFRPooaxd23CKdr+AcOT7T8bnkaVeHPrfzI8naIoUFAPGZ3IbPAt0BBZaX8wkVSZ hlPNOM2EI3oc0eGICkeUOF1cDTnD6vrbZ1xdJ8G6L67BmnHL9oFTl+2PcBqygnjIiFpXQzO5yC83 jM0hmDlK02syQ5RUFh9djO9mf//Zqjz/Vum6R9euun9lYcm6R9aBLHrMU7FsctPS2qCnctnkxmW1 AfzXlYcvaanefmAtyGaQW5sumFeSP/eC1uYLekvy51xA1haGb+BfA92QtYWdZG0hWHiGt5LU+4y9 niRBjI0uK4gLDOI6Kl1hOOO6QpMw+TvXFc60rHCGPvLdywrXzUmtrUqkjOssVpvHrEyb2NqWOW8X WVbIE5cV6qO1m2squorc+P1zn7ywQUjODw1XMF8oex/6DM9D7zkvvSLNNvGifRvqzl9QbkmryRm+ tb2zfMFWcf4M2rpd0tYlCQ+oy6+NEYOJaXRsiUV0cjEyd05HebTb5EndKU/yknlSN8uTFJonzp1t 4SbthJhfJmSRubO7uZjMnYVWMuafee58is4KTHSlkPUXR8F3z53VxMz8VmVac2NTlKgod/61c1Pr 6xrSyUYna5JJ+a358/ABpil8PK0kZGRzaFO4LG0FU93wv+gkmi7IwCRa9E7cA+LK4PwDqwtwxCh1 KqNUdSPrXEap1xlJ5zKjhIWM8zBIINLLkBv6XDihjjVHjLZAk414HdHdiwN+bDQWHj8BPJOjETuR gnuAU6hVKoc3xebKLigNne5mwlWlJV59MMWrk/GYn2f3mdRqtcqaNbFoqO/bjubCwtqokVdpNGqD uGekbeQE9yLUuAm9mNDFWypbJrfsaNnXIh/3iuJz6dWE2CmqyPKU5bRXF+IrC/yHhJ++pxDfUJAu Jr2mIFNk4nM8R/Dn4stmDQmLdAkxVIJsBO5Xqdun43RZfyzS/N00xdRjWm3i6euI35N3Ec3296gx jr6IkF5DdEPvGv8aYiyW/ndfQ3Av5s25YFL2jLpsu0ZGXjPEKqcXp9fmeqKJKR1tiWja1C1TUxpL 02xKHqIjjUKdXNgUT0+k2VITUzvaE1FsqFsO7e1wWVP8Fog/PQGPOVQYjuSn+pNjFdPLC3qbMnRm m6Az2gWTS1DaXXZLKDspWpAaSE4vn0baIjjyD26F7FFUimYfSEOmUKak80ypLTKltsiUDDJT6pWZ pBPqHPrME6FGr/6EozGHRN9K6raPk26XJ61eHT9Gl/ZkZ15gOHUZws6WY7gVKiGQluWoX5Dwbjea ybuIbSxQe5esHZuN7xY1OFKSrCq5Wi6b5U0WDGpFuGXdJM5AVxheZ6+SX6drEMOa7rlqjVpucJJ6 30DW+fgnISa4LuGHSEAbJT0oSnpQlLyCjIpOKiqIIRf+6nFqaX5JK35JKyC/FG2TEKIWPzNWv9RH /WSuorZkNkW1clcTBGbyscW+8dtVRrvUGRf7xkJx0UUVFo0t+92uNHttDq9J0XqTOPQrrXSO4og3 ZldsqVNa/WC5ZvVoRLCxY1L54svnccnMOoc+mzy3JtzZwW1gJUQ/yRAzbQH9ZKA/H0ahERjNSKDr F9/lhP3YR4kP26V62iRpHQt/RWkefRM78nGiiLzGhajChKMCTpXj5FQomJCMU5JxkNDKIE4J4oBY GsApARw14nODOEgWudQmW2MwAFYLufcSauiKQbLCSHKkJYLk/jqyhSi1Kah1N2mpAyTbr8gfinWL kUOM/sMkfqB6J2+TYuLO29HNI+OGCIujyCJtud2COZ4bPi7Tu1N9vlSXQTb8okxOtjk4vCGLWjYs 409yGkvQ4/CZlPxdMrVGp/zmIbLzU6YyaPgZOrOahzkhBx/qIbdOx/1NrVPxnEpLtF0Ac4yLQNt1 6K3DqAHc0wSoWjFZ/EorxkVEhrNwJIgjARzx44gPR7w4moRTZTiNx6VluKwUl2XicvLfWrbhVkFa PiAyoYHuKgTgDoJRKiYyoSMDCSk2VjWJ5xFlVgqThVXCDkEmJMz2RiGvKdxUek0GziDHMojXFCz2 xsUZGzO4Oih1TFQTJb9GNNl9rLLyOGiS6jtO/SESo7TReI0qWjGqZz6qHPfu7gwqH0flF8nkw1/w ekeqz5/u0vFPcdw+Xu9O8/mjkBv+Si6D2YUjKdms4n/Lcc9xajN0e79Zxb3B4dc5tSXodnpJsyit xrFG4a5Sq4fWjTWR0apUa6GFYKY65FaroYX04HjJZi4ny3EqDWmvNLCOFmivOLrkMMoBxZjI+j7x G1nEY5RlYSf0x8fJ+zwndki+wc6K7FhNems6mbeSa8oRLg7hQi3WBsj0grSKVpuTndYU0pq8TabR KURJpcmM6fI1IoolnZf231jYbmWbmMf2MI+9EbVY2GtQzNeoLFG/L2TTyt58Q6a1JSd5wyasxs7h L1TYEg14Q1aN7PjLMo3J7/GGzZx6+KsMg0Unh9m5Ei8cvg0EL9dZDPgQfsBg0ct4hUY53I8nK8hu KK3VODyHeA+IAreCflLQ1MPIA3UtIJbvwWke7BQnz04cMRQauKgau8mQXOrGrmKiOBf2N7k0liZN i2wyapEmreTtb4waLTHeIE+rWmQh+/oi+aNvfS3iqo7dquTyNilyct0BE6fYqhb44adVQorPl2xV yzHmv1SYkgNJKSbF8EHBJNdZDbhEZtbws21Og5xXGfVDWdzrFq0cxgkz1KQLgto3+EMohsoOIwFq Yie7UiLi3pQ4HM9X16o5ddgEk5b9rkZjVJy8wIOTxfdciBWOd5N9iKNb9MSVXhyUxkBxO6u4iQQT yr2hUBlUQ6/bPKQ/4quGdwgWsoePk2lNOiUpG96A96j0akW9xWNSJgWTDXa7S+CWBcNmyCsMdlPA 4HS4haGblIIYaaUPv4XXobeRB2kGtI4kJLx6nG5rUSqp5RVZRn9CsE5hcJgul+stLovJocGyi7XO FLcrxaG92p+flel6UalRicaALTs9AUGhEALkG24e+QKvhG/QIkc/2bhw9HGyQUHNQ5c9Dr7gGfJ1 49bVVsYryrMIVjTEs+oA5B51+ACXxU1ARmQ4gJTaEzJENiZJ6+ZBeq2onyyzaXiOGf7wvaAFOf4q 6vNHIj6FyY3wyOf4hIzjtsNdTANwl8M4CX3XjWScxfJNpcVstvDPqI1qOVcYCYUi4ZCa7mm+ePgB /E/5FSiEkhM2nhg0T0JJXmx03ubXXowq4xDg0K0nCohdzI7RHxFk8WIbU7eF/zG3e+4sOTZ4XWa3 RccXTi1O8pdMzcNqIcnuSBI4+bznh7tef2N45i91Jq2cU6jki3795h/XrPnDb19ZLFMowLgE8kSb 4YnehScKorzDyExHWrMUqRF5kDyZWdx2oxXnAvQJY7mju2OUzCsUmgvyuWhEcrZ2M343qbitkNdZ 3Ga3V4/ls+fMmSPjhCSHLcmk4hZv4Fxr/vjmrxfJVQpODt3wBfzAG6/jB55XCxp4OoXs+PBkeL47 udv5GfLLwB86EwZfqj8adyiNgkKjDWlRPA7zMngIcD4KRTRqsZPuUARzCbJhqqgoSp6p0OHgI6SD KPmiQjsoU6nkmwycw+HVvZbEB7KyAnzSqzqfw4ENn3xiwA6HT/cqK39N53U4OMMn/AOKUDTVrL5t +GujAO2luE1tTo2GFOcsU4aiUbP6ViwX4G/45K1QHgkpl8Fs9RyIAp+SB1A+akS7D6Nm8OQOI9fa 04xjGyrxokpcU4nzK3FKJa4c5GoSVl1Skm5zAV5WgFsKcGkBjhXgAjjw+GqEA6AGMsDS/djvHYLb oGwdhsnG1zD34Fp1pSPZ2fLIIEYDlq7aQWzrl89lO/rJa7ruV8GXd78jjpRmsv1CZLnE9Y2bVshO n0YoT5v1s7WPp/KX71nTtnX2hLBgzpq8cc/K8MREhkEp47BSq9ZGClvzui/pSOPdVa3Tc5Ze0xV5 zFE4szrcXFfpDlbOqUzMqfDiH3fcdV5TavPyXffNaX/4zisWl6uNZq3eaDGY3YLKYDJM3PnQbKPP aSxZeHlP6dzqFL3Dbz7/saWZ2W0LyTu9qaDbI/IgItu/GvAFh1EhCY1NZOMFENJhCwalkgJWks9K 8lmJuDBgGlsgaBK3lkETNeFsdk42C7rHl4iL59mDnCvhsqaKlpsqhvQSJ5sjUwc5Z8LtM4Z8PrKz 1ip++Kw+TbF4TjEJO21eCMTEC6VCcmHxEa4GJpWv7ieNPNboo3vfpL0UR6WV6qPiK91qMuZryD2q s+Gm1eyhq9lDV0sPXU26mklDxkVNwQR55pCrq25otLOUjP6g4FUawJ6yIQ6EMG7NiPSe0f8gwfj1 siLqpqQX32QTI58/utPCUVhogRx711jIHylfs+ecBXeuLE1tWVlXPjsRzJm/e9G8q7szyEaLhlUt 0Te9xe0Fy1d5SmaUL1yenly3uLZy7gT/xRftvBBPnHbhzKz0qZtaJyya3pLsr2ubXVi7sTMv3ray Mm/OtKZAqLljLjc3vTbbNa8jWlNe4s/fPnRvVkvVhKC/oropo3fZOWCnjdCXnhN3AsfQBwnXaYuT YbY4mUli1DDpHZl43LIjWWu3kpmdlTSe1UnYExz5r4cH6KQ2IHWugLT6FJCmdyDfI+MZzETIfxs5 odaQTcYJxIu/VVOTPR6ayRoOifMTcaM77RBHRYtHGqTJzPCQ/5yNsZ3swGUbjEkUJ7YWCd7A0Mev CYtN9j0rnLJxC1Uy/rn4ir7zNz+wKJa9vG/nFpB9Bk+svDW7Y9kEu69qYWNxx4RUp5rbdePn/b0z Hvri7hu+EOUjvbee21HkmnLlk8uv/eXO0pSaOWsvBvf1GJjtXXIHykJ/TaSk+HCKF6ck4ZAHp7hx iguToM2B00Tdm0mkmi2+WyfqzsaIqBalSasEaZJC06T5cpqk0DQpFE4jW5YNPie5yKkln1qTZEcg RbsySXY0rvyotLUXVA9X3G3CJot5EFfuD01NEwaxkv4yIrdy6Li4RkP+jpNtD2wfJDWGsflItxSV sY2QEEEp6DykKCy9wzCJM7+7FBq9cmi2UqdVKNR6FTZ8TXY48AqtGqfLdGanGSbYig9UBrW8lqzC KAW3xew2qfk3b9TI9D6HySnoFE/zMhmWKbWKk1erIVwBba8Fbd8OfboC3ZDQpxXimA+necncLjHI hqEEtpNebBc9jz0gziG4zMfzwpBQiaTrkiPcDqSlytGSmZyWvJkwFZcEAiXQ+bIez7MrstqFkkGc yjREV7Ti1JmAAzk++lMyUUfinO0U5ZBp2Gnb/RSjvkMpbiG9XQ6R1FCBwWZU8hqj7uSMpSXmpIIp +eJmP5g0yDi5ylnWdU7ZnKu6s+wNl6w6zuWpjFp5M9kfqxR8disM73qsmX3dpnmxWGtpcnJqssrs sxntgsGWEnIWzN5cV7Hl6n1rX1ebxShtMfiE60B/nVh+GM0ElSURlc3EOSpQSg4x/BxRbzlEbzmD XEFCM6k9MmmS04JbE2QNIQKnRMjUNgGlkQRv8KgEtmooXukJiBttaJf1gOYPitM1cXccsW+D1DUN Um83kIazQDMYyshL2LKEOEkow2LXlbowHQHKTGUme+Eg1iY0Te0Z/wwE5E1k67N2dOtz/ESJMLr7 GVx3nPp7ydeLm03IiztzyZifH/v5VuG4tUf6Mw+64sZKztSINhgBrqtY//A5VWs6S40qBW/Qqwva V9VWL6hNjrWf17oF2kqp0BrUa6qXNkXd+W0Fpb0TczVkNghRq6W0Y1Vi5mWzMgMVM8tqVk3JxGu7 rl5UZPP6DQar15aSFAgHkis6cos6E8lgHjaLy6hMTnQVpTYV+kOpIbnRYzc6TAYLtHPWtA0NE5a2 lWg5ZcEU4vuzYa79G7kVpYNfOpkoJQshmTiagVOiOCWCw0k44sEh0UGFnTjswBE7jthwxIojAoYm TpHjFBmOebDorczUW2XanUDsAUHaa0H3WLx9iOzBSMrKEgZHvkl44QyBmB8JJuGDrKOQQUQgEwKB /N40imTUV8lgAGBb1hIasmdNlh2PerLEBpbFgoKgCU7VdIgrUmB1eSdyc6WZfExaJSU/0DkuyjEL PO0Pn7pRa9Q08ZivsuMQDvK/sZqvY79jGvpAJ+hhXqFR4lfkFl+GL5jjE64z2Ybv4YZn4Qfw6mBk +GO2NIgFheBzWnwuh543q8hWLphlffNsiHt/qJRY3EKwuJvkBvBYzyT00SIcLRRfDfKix3qcOqwi ySsViT/rJj/gIJvUU0H1qVCaSuwi1TA5d1Xujlw+98w/WTnC5Ym/cJTG0oPifgbLIHlRSPYLWZxg OBkJXUbpZwGyp1me0eY8xXS6TxDTicew8LpkMce6X6XGQ5VLtHvGHzvSECh0yo9oYeIobQ7ib6rf 2b+8fPm0QqNC/AWkUpPesLSxZnVbVrRt6/QJnZEkp9/LTVAZNXKredgbaspetWdVCb57yb2rSk0u p0FncptNHpPK5XUHahc3V8yt9OvcYc4YDKjBCaakDt8o5wp6d42MsHkJp+BfEP8fJ/PBBvaB5v3o jcPIBL5LYwriiSZBkH7qcupPYN6Txskvxb64XlxwFQbZVYJAlwbFqwTpKvGwlqzpbhCI4Sik5dwg a9kgHhfYvikGtDZpRB63/+g96WeMbx+Ea2xy0yDO3O9u047+JEEcksVWiEnrr2wZdmwFVly6Gr9C wu/j5WrFcJbc6EhxJ0dMnAJ/MHS9xSLXGNTcpwabViE7ZvZ6XIaTL+qMal6ht+hlzakpFhhXFOYk 0KY0EwFt/gqRd5QkvwdGjmxUjZ5KWNKycLocp4lrqekRHNHgWuIqAqTatTCc6NlI4t2cg0tymnKW 5vCxHJxDfjCjRgZDAK1GHJ0G0OnAAdJjy8i4AZeWkXjFTC7fUIYLy+rLFpXxKWW4bJCLJQzxMA4n Pg0ElIWfpbdDL1b1K6ePmxSK00Fxe3S3NCPMHd+HxV4sO/2FU9H4DcLSD7dGY8ZCfo81u23LQ6tj bVUZVlCWVqVNnTA1r/eKzgyu4Iae5dd3RXOX3be2bdvsRNS0L7m6p7JqdlmSq3hmdcuV3JFpj9x1 xZIyrWA2+912t0FuNBtbtu+Z7c8uW3Rl+/Tbzq1Pa12x6576nfuWZ8cnLygom1cbJmE2uocmHP+e 9MhY4lxSuvEMafj0xDedMT3GPybLhLRlLMlT5T8bS4qy70gHSFLOpUmVMy7d8v1JnXY2fWd687uS Zo1m6NtJewNNuoIzpMH/s0l//7eTIU1M9545GdPE9DOSYNQcS++aNo1PZsV3pDvMd1gKLftosm44 Q/rkv5Js558p2Ysdwmi62ykfTQvOprPpbBqXbhTTof9C+h2kkW8nV6Prlu9Nn/3PJPe97ns9v/jv paQt3lzvK75n/VcEEoHGwLuBfwQfPJvOprPpbDqbzqaz6Ww6m86ms+lsOpvOprPpbDqbzqaz6Ww6 m86mfyeR18iIQ1iUVsSL8hakQBeJOyQmoDjegC/HV+Jt+CjuxufiKYjHHHajJDwTC2gOkiEzKse3 4uewEUWwj/w/UVAL38l/xP+T/E8d+A/5E/xafj1ejq/BqfgZtB7VohloJt/Bb+d34KX8P1AFquM/ 5j/hf4Sc/Kf4V/gK/jP8LP8vuPOt8AwySPB0H08YGYFPTD4hLyP/fbD/sSdD4jOYQW8cMAVSIlTT u3zpvLVL6RGEryH/t4sf+Kc6Nfsx+njklALaNkhmGAN+CeQ9KPRfQi3qPSM+RI+MB//eD8Sj/zk4 2b+Bq5CSQJYKzwHgu88MxRvoEXn6d2Aimv/vQJYM38XQeSrg+RvOiL8h438HSh+acBZn8e+Cfw3N /nchy0e7+Xng234A4Nye8eBPou4fAm4NCv+/DP4YKvghILpiwK+ji/8tfDDyOgP/Y9C5BDh2RigW jH3XGVEM7fHvYOzaHu4FtHs8+CBq+yHgHkPB/1uA57zhh4K/AyXLB1HB6eA3ojT+LpT8LaShrrM4 i7M4i7M4izMBj6B0Bk6Obv6/gnWojoD/ZuSLHwJuBboYsPm/Cv4bdOf/nwDz43MgTf1vpkb0GFqL FqNstBDuNl8sQ9L6yHf/kbm6/M7B/r59R+Yay/+FXHRy/8Tft5LfSKCfJ9+y+eTXQ1eoP1Q+Dlm1 uH4Af/8fp7JEqgplbmRzdHJlYW0KZW5kb2JqCjIgMCBvYmoKPDwvUHJvZHVjZXIoTml0cm8gUERG IFByaW1vUERGKQovQ3JlYXRpb25EYXRlKEQ6MjAxMjA3MjcxMzQ0NTIrMDQnMDAnKQovTW9kRGF0 ZShEOjIwMTIwNzI3MTM0NDUyKzA0JzAwJykKL1RpdGxlKE1pY3Jvc29mdCBQb3dlclBvaW50IC0g RXh0ZW5kaW5nIGFuZCBFbmhhbmNpbmcgU0FDSy5wcHQgW0NvbXBhdGliaWxpdHkgTW9kZV0pCi9D cmVhdG9yKFByaW1vUERGIGh0dHA6Ly93d3cucHJpbW9wZGYuY29tKQovQXV0aG9yKFRvbnkpPj5l bmRvYmoKeHJlZgowIDYzCjAwMDAwMDAwMDAgNjU1MzUgZiAKMDAwMDAxNDY3NiAwMDAwMCBuIAow MDAwMDU4NTY3IDAwMDAwIG4gCjAwMDAwMTQ1NTEgMDAwMDAgbiAKMDAwMDAxMzA4NiAwMDAwMCBu IAowMDAwMDAwMDE1IDAwMDAwIG4gCjAwMDAwMDA5MjcgMDAwMDAgbiAKMDAwMDAxNDcyNCAwMDAw MCBuIAowMDAwMDE2Nzg1IDAwMDAwIG4gCjAwMDAwNDA3MjQgMDAwMDAgbiAKMDAwMDAxNDc2NSAw MDAwMCBuIAowMDAwMDE0Nzk1IDAwMDAwIG4gCjAwMDAwMTMyNDcgMDAwMDAgbiAKMDAwMDAwMDk0 NiAwMDAwMCBuIAowMDAwMDAyMzI2IDAwMDAwIG4gCjAwMDAwMTUzOTMgMDAwMDAgbiAKMDAwMDAx NzU2OSAwMDAwMCBuIAowMDAwMDE0ODI1IDAwMDAwIG4gCjAwMDAwMTQ4NTUgMDAwMDAgbiAKMDAw MDAxMzQxMCAwMDAwMCBuIAowMDAwMDAyMzQ3IDAwMDAwIG4gCjAwMDAwMDQwNjcgMDAwMDAgbiAK MDAwMDAxNDg5NiAwMDAwMCBuIAowMDAwMDE0OTI2IDAwMDAwIG4gCjAwMDAwMTM1NzMgMDAwMDAg biAKMDAwMDAwNDA4OCAwMDAwMCBuIAowMDAwMDA1NzE3IDAwMDAwIG4gCjAwMDAwMTQ5NjcgMDAw MDAgbiAKMDAwMDAxNDk5NyAwMDAwMCBuIAowMDAwMDEzNzM2IDAwMDAwIG4gCjAwMDAwMDU3Mzgg MDAwMDAgbiAKMDAwMDAwNzIxNSAwMDAwMCBuIAowMDAwMDE1MDM4IDAwMDAwIG4gCjAwMDAwMTUw NjggMDAwMDAgbiAKMDAwMDAxMzg5OSAwMDAwMCBuIAowMDAwMDA3MjM2IDAwMDAwIG4gCjAwMDAw MDg3MjUgMDAwMDAgbiAKMDAwMDAxNTEwOSAwMDAwMCBuIAowMDAwMDE1MTM5IDAwMDAwIG4gCjAw MDAwMTQwNjIgMDAwMDAgbiAKMDAwMDAwODc0NiAwMDAwMCBuIAowMDAwMDA5OTIwIDAwMDAwIG4g CjAwMDAwMTUxODAgMDAwMDAgbiAKMDAwMDAxNTIxMCAwMDAwMCBuIAowMDAwMDE0MjI1IDAwMDAw IG4gCjAwMDAwMDk5NDEgMDAwMDAgbiAKMDAwMDAxMTU0MyAwMDAwMCBuIAowMDAwMDE1MjUxIDAw MDAwIG4gCjAwMDAwMTUyODEgMDAwMDAgbiAKMDAwMDAxNDM4OCAwMDAwMCBuIAowMDAwMDExNTY0 IDAwMDAwIG4gCjAwMDAwMTMwNjUgMDAwMDAgbiAKMDAwMDAxNTYyNiAwMDAwMCBuIAowMDAwMDIy NTA0IDAwMDAwIG4gCjAwMDAwMTUzMjIgMDAwMDAgbiAKMDAwMDAxNTM1MiAwMDAwMCBuIAowMDAw MDE3NzY2IDAwMDAwIG4gCjAwMDAwMjI3MjAgMDAwMDAgbiAKMDAwMDA0MDkyNiAwMDAwMCBuIAow MDAwMDE1NTM5IDAwMDAwIG4gCjAwMDAwMTU5NjIgMDAwMDAgbiAKMDAwMDAxNjIwNSAwMDAwMCBu IAowMDAwMDE3MTg5IDAwMDAwIG4gCnRyYWlsZXIKPDwgL1NpemUgNjMgL1Jvb3QgMSAwIFIgL0lu Zm8gMiAwIFIKL0lEIFs8QUQ2MjExQTlCMzk1N0U2NDM2OTg5QjZBOTg4QzY4RDk+PEFENjIxMUE5 QjM5NTdFNjQzNjk4OUI2QTk4OEM2OEQ5Pl0KPj4Kc3RhcnR4cmVmCjU4ODMwCiUlRU9GCg== --_8d3e7d6a-7987-4a4f-bfeb-b6841a0add5b_-- From touch@isi.edu Fri Jul 27 15:44:30 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id BA13A11E80C5 for ; Fri, 27 Jul 2012 15:44:30 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -105.497 X-Spam-Level: X-Spam-Status: No, score=-105.497 tagged_above=-999 required=5 tests=[AWL=1.102, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id RhBWRpesd-jG for ; Fri, 27 Jul 2012 15:44:29 -0700 (PDT) Received: from boreas.isi.edu (boreas.isi.edu [128.9.160.161]) by ietfa.amsl.com (Postfix) with ESMTP id 1BEFC11E8098 for ; Fri, 27 Jul 2012 15:44:27 -0700 (PDT) Received: from [128.9.160.166] (abc.isi.edu [128.9.160.166]) (authenticated bits=0) by boreas.isi.edu (8.13.8/8.13.8) with ESMTP id q6RMhZ3D023603 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Fri, 27 Jul 2012 15:43:35 -0700 (PDT) Message-ID: <50131997.7000305@isi.edu> Date: Fri, 27 Jul 2012 15:43:35 -0700 From: Joe Touch User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:13.0) Gecko/20120614 Thunderbird/13.0.1 MIME-Version: 1.0 To: Yuchung Cheng References: <50049F5A.7000709@isi.edu> <50082223.9010701@isi.edu> <500845E0.8050601@isi.edu> <5009786B.80707@isi.edu> In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Cc: "iccrg@cs.ucl.ac.uk" , "tcpm@ietf.org Extensions" , "Arjuna Sathiaseelan \(work\)" Subject: Re: [tcpm] Review of draft-fairhurst-tcpm-newcwv-03 X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Jul 2012 22:44:30 -0000 Hi, On 7/20/2012 2:43 PM, Yuchung Cheng wrote: > On Fri, Jul 20, 2012 at 8:25 AM, Joe Touch wrote: >> >> >> On 7/20/2012 7:37 AM, Yuchung Cheng wrote: >> ... >> >>>> The server receives the request, and then sends a response. The server >>>> has a >>>> response that's typically larger - thus the burst. The server decides >>>> "how >>>> long has it been since I *received* anything? Since that time is very >>>> short, >>> >>> >>> but RFC 2861 does not use the most recently received timing? not in their >>> main algorithm (section 3.2) at least. >> >> >> I'm referring to the algorithm as per Standard TCP [RFC5681] which, resets >> CWND to the restart window after the app goes idle (as described in this >> doc). However, RFC 2681 failed to note that the restart algorithm was >> widely deployed at the time, and was based on erroneous logic in a footnote >> of an unpublished extension of Van Jacobson's 1988 congestion control paper >> (ax explained in detail in draft-hughes-restart-00.txt). >> >> >>>> there's no slow-start restart, and the burst goes out. >>>> >>>> The only time the server would timeout and do slowstart restart would be >>>> if >>>> it decides to send something a while after anything has been received - >>>> that >>>> might occur for auto-refresh pages, but not much else. >>>> >>>> The issue is explained in detail here: J. Heidemann, K. Obraczka, J. >>>> Touch, >>>> “Modeling the Performance of HTTP Over Several Transport Protocols,” >>>> IEEE/ACM Transactions on Networking, V5, N5, Oct. 1997, pp.616-630. >>> >>> >>> But why does receiving something recently justify the sending cwnd is safe >>> to >>> not cause burst-induced losses? Sorry but I can't find the explanation >>> in >>> the paper. >> >> >> It doesn't justify it - it was an incorrect conclusion. The goal is "restart >> if you haven't SENT in a while", but Van's footnote said that most >> implementations at that time had a 'time since last received' timer, but not >> a 'time since last sent', and that because TCP is symmetric you can use the >> receive timer instead of the send one. That logic was false, and persistent >> HTTP turned out to have exactly that pattern. >> >> However, the mechanism proposed in this doc still allows most HTTP exchanges >> to burst an entire window - because it would be very likely that users will >> click on URLs within a received page within the 5 minute timeout. >> >> I.e., I don't think section 4 of this doc is appropriate (it still amounts >> to a send timer), and still favor the "burst-or-lose" style approach as >> recommended in draft-hughes-restart-00.txt. > > I enjoy reading this draft. Very nice summary of all the mechanisms proposed! > I also like burst-or-lose due to its simplicity (and is interested in > rate-pacing as well). > > In burst-or-lose, do the acks of the N packets increment the cwnd if > FlightSize < cwnd? Not sure - it's been a long time since I cared about this ;-) The doc should explain what you need to figure it out... Joe > > I'd recommend newcwv draft to cite and discuss hugest-restart draft. >> >> Joe >> >> >> From rs@netapp.com Sun Jul 29 16:08:26 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5800421F855B for ; Sun, 29 Jul 2012 16:08:26 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.082 X-Spam-Level: X-Spam-Status: No, score=-10.082 tagged_above=-999 required=5 tests=[AWL=0.517, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id oapwvt1+nfVP for ; Sun, 29 Jul 2012 16:08:25 -0700 (PDT) Received: from mx2.netapp.com (mx2.netapp.com [216.240.18.37]) by ietfa.amsl.com (Postfix) with ESMTP id 6869E21F8518 for ; Sun, 29 Jul 2012 16:08:25 -0700 (PDT) X-IronPort-AV: E=Sophos;i="4.77,675,1336374000"; d="png'150?scan'150,208,150";a="669539139" Received: from smtp1.corp.netapp.com ([10.57.156.124]) by mx2-out.netapp.com with ESMTP; 29 Jul 2012 16:08:10 -0700 Received: from vmwexceht04-prd.hq.netapp.com (vmwexceht04-prd.hq.netapp.com [10.106.77.34]) by smtp1.corp.netapp.com (8.13.1/8.13.1/NTAP-1.6) with ESMTP id q6TN89q5006833; Sun, 29 Jul 2012 16:08:09 -0700 (PDT) Received: from SACEXCMBX02-PRD.hq.netapp.com ([169.254.1.34]) by vmwexceht04-prd.hq.netapp.com ([10.106.77.34]) with mapi id 14.02.0298.004; Sun, 29 Jul 2012 16:08:09 -0700 From: "Scheffenegger, Richard" To: Bob Briscoe , Mirja Kuehlewind Thread-Topic: [tcpm] Review of draft-kuehlewind-tcpm-accurate-ecn-01.txt Thread-Index: AQHNaH+uJyuGoLRTMk2MyxwT10S8iZdA35aA Date: Sun, 29 Jul 2012 23:08:07 +0000 Message-ID: <012C3117EDDB3C4781FD802A8C27DD4F0D4D8465@SACEXCMBX02-PRD.hq.netapp.com> References: <201207161454.45011.mirja.kuehlewind@ikr.uni-stuttgart.de> <201207230302.q6N32Whj000504@bagheera.jungle.bt.co.uk> In-Reply-To: <201207230302.q6N32Whj000504@bagheera.jungle.bt.co.uk> Accept-Language: de-AT, en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: x-originating-ip: [10.104.60.116] Content-Type: multipart/mixed; boundary="_005_012C3117EDDB3C4781FD802A8C27DD4F0D4D8465SACEXCMBX02PRDh_" MIME-Version: 1.0 Cc: "tcpm@ietf.org" Subject: Re: [tcpm] Review of draft-kuehlewind-tcpm-accurate-ecn-01.txt X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jul 2012 23:08:26 -0000 --_005_012C3117EDDB3C4781FD802A8C27DD4F0D4D8465SACEXCMBX02PRDh_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi Bob, Regarding this observation: ######### 3a/ Single Repeat Unnecessary? You propose that the receiver must repeat each count of CE that it feeds ba= ck exactly once, even if its internal count has already increased again by = the next ACK. I don't understand why you need this, and it introduces consi= derable complexity. I think you are assuming that this will improve resilience against lost pur= e ACKs. But re-stating a counter in every packet is already resilient to lo= ss. For instance if the receiver's internal count of CE marks is 3 then 4, = when it feeds back 3 then 4, if the ACK with 3 is lost, the 4 recovers the lost information wi= thout having to repeat the 3. IOW, the ack count is cumulative. This criticism also applies to the repetition of the other (nonce) counter = in Appendix B. Where we do have a problem is in detecting wrap (every 5 increments in your= proposal). However, repetition doesn't help with that. For instance, if th= e receiver feeds back 3 twice, the second 3 could represent 5 increments on= from the first (or zero increments, or any multiple of 5), because a strin= g of pure ACKS may have been lost between the two ACKs received. Similarly,= 4 could represent an increment of 5i+1, where i is an integer. 3b/ Better solution?: I proposed (in the re-ECN protcol, which was very similar to this proposal = - thanks for the acknowledgement of that BTW) a way for the sender to calcu= late whether wrap could have happened, and to calculate the worst-case coun= t of CE marks that the receiver might have fed back, given worst-case losse= s of pure ACKs (assuming regular sized data packets). ######### After having run the simulations more, I agree with critism 3a - the added = value of the repetition only shows under specific circumstances. In the maj= ority of cases, the difference between a direct encoding of the current cou= nter value, vs. repeating the value two times is negligible... Nevertheless, with delayed ACKs there should still be a skew rate limit, to= have at least a minimum number of ACKs in between two counter wraps (e.g. = whenever the counter has changed by <=3D 1/2 of the available codepoints, a= n ACK should be triggered... That would equate to sending an ACK for every = other data packet, when every data packet is marked; for lower CE marking r= ates, the delAck rate can be significantly larger than 2...). However, the conservative approach you describe in 3b has a major drawback,= by assuming that whenever enough ACKs got lost that the counters could hav= e wraped, that the sender needs to assume that they indeed have wraped, you= end up with the sender overestimating the congestion of the network... You mention that when there is congestion on the ACK path, the sensible thi= ng to do would be to throttle the sending rate. However, when the delACK ra= te is larger than 6, this would need to be interpreted by the sender as 5 c= onsecutive CE-marks (out of 6 data packets). Two consecutive lost ACKs woul= d already be enough to cause that effect... With receive side coalescing, h= owever, I don't think it is too uncommon for the receiver to skip such a sm= all number of ACKs... I don't yet know how to properly address this. Attached are 4 graphs, showing these effects; All are for burst loss simula= tions (with the burst size increasing in the X-axis for the marked data pac= kets, Y-axis in lost ACKs). The colors run from black =3D 100% too few mark= ed noted on the receiver side, to white (1% underestimation), yellow (0.1% = underestimation), green (perfect tracking of the CE marks), cyan (1% overes= timation), blue (10% overestimate), magenta (100% overestimate or more). Note that the yellow color in the simple graphs comes from a 1% background,= random loss rate in both forward and return path, independent of the burst= loss size;=20 The cons(ervative) does address that case (the line along the x-axis remain= s green), but overestimates the CE marks extremely in most other instances.= .. Note that in these simulations, the (data) CE mark / ACK loss rate are mode= led independent of any reaction of the sender... One mechanism to limit the overestimating of the conservative algorithm (as= laid out in 3b) might be to track the recent marking rate, and limit the s= kew during ack loss to some value related to the recently seen marking rate= ? (Obviously this wouldn't work well with step-function marking, though). Guidance from you or the group on these topics would be highly appreciated. But I think 3a (and the complexity due to that) can be removed from the dra= ft... Best regards, Richard Scheffenegger --_005_012C3117EDDB3C4781FD802A8C27DD4F0D4D8465SACEXCMBX02PRDh_ Content-Type: image/png; name="CP-burst-twice-simple.png" Content-Description: CP-burst-twice-simple.png Content-Disposition: attachment; filename="CP-burst-twice-simple.png"; size=4085; creation-date="Sun, 29 Jul 2012 22:38:54 GMT"; modification-date="Sun, 29 Jul 2012 22:38:54 GMT" Content-Transfer-Encoding: base64 iVBORw0KGgoAAAANSUhEUgAAAZAAAAGQCAMAAAC3Ycb+AAADAFBMVEX///8AAABAQED/AAAAwAAA gP/AAP8A7u7AQADIyABBaeH/wCAAgEDAgP8wYICLAABAgAD/gP9//9SlKir//wBA4NAAAAAaGhoz MzNNTU1mZmZ/f3+ZmZmzs7PAwMDMzMzl5eX////wMjKQ7pCt2ObwVfDg///u3YL/tsGv7u7/1wAA /wAAZAAA/38iiyIui1cAAP8AAIsZGXAAAIAAAM2HzusA////AP8AztH/FJP/f1DwgID/RQD6gHLp lnrw5oy9t2u4hgv19dyggCD/pQDugu6UANPdoN2QUEBVay+AFACAFBSAQBSAQICAYMCAYP+AgAD/ gED/oED/oGD/oHD/wMD//4D//8DNt57w//Cgts3B/8HNwLB8/0Cg/yC+vr4AAAACAgIEBAQFBQUH BwcJCQkLCwsNDQ0ODg4QEBASEhIUFBQVFRUXFxcZGRkbGxsdHR0eHh4gICAiIiIkJCQmJiYnJycp KSkrKystLS0vLy8wMDAyMjI0NDQ2NjY3Nzc5OTk7Ozs9PT0/Pz9AQEBCQkJERERGRkZISEhJSUlL S0tNTU1PT09RUVFSUlJUVFRWVlZYWFhZWVlbW1tdXV1fX19hYWFiYmJkZGRmZmZoaGhqampra2tt bW1vb29xcXFycnJ0dHR2dnZ4eHh6enp7e3t9fX1/f3+IiIiampqrq6u9vb3Pz8/h4eHy8vL//5YA /6AA5v8Awv8An/8Ae/8AV/8ANP8AEP8CAP8GAP8JAP8NAP8QAP8UAP8XAP8bAP8eAP8iAP8mAP8p AP8tAP8wAP80AP83AP87AP8/AP9CAP9GAP9JAP9NAP9QAP9UAP9XAP9bAP9fAP9iAP9mAP9pAP9t AP9wAP90AP94AP97AP9/AP+CAP+GAP+JAP+NAP+RAP+UAP+YAP+bAP+fAP+iAP+mAP+pAP+tAP+x AP+0AP+4AP+7AP+/AP/CAP/GAP/KAP/NAP/RAP/UAP/YAP/bAP/fAP/iAP/mAP/qAP/tAP/xAP/0 AP/4AP/7AP//AP9idicVAAAMsElEQVR4nO3dB48kRxmA4a6fCggsgjBCBgQILKIA2T7OBg6fueMc z8aAbQSIKEAEkYQRQYAIRoAIIlgEAQIE7GxXz3xdoau6u6a6dr73lbx7t5PW/exMdXfVzXYdERER ERERERERERERERERERERERERERERERERERERERERERERERERERER0S6z4Cpm1/mn/kIjLgnd3P9i xqMqLbS1/OsEv9BTDCjDl0IbH5Ds7M/3+c+6/InvN3Q3PBH2fz7cav/RAelG97a7eHxv+0fBJNSw +b2PRmx2c7imvZWRlxsHxLk3M743496GZP1oMPqJ78SLjNjydtxwrjneuPuh5RxhuOfhYycfC49w cvP7IMOGPFzaOde0G3p0QfjejHPPiAQLbjD5nNm/4IdBTBpk/5J1uGfDGBLLH9SHYbi/2G5Ad1CP 7famBvUueCsiIiKiE06eJQlcGv1L7O78+1/2fWltdEgdvrjAI1B2o0Nz/1J58GCGUynd6ODkcNWu Gx13DDfogtencNMg4/OLh//867uXH84syv8o2f4Y2z0cFxcbQWD86ww329/AeLcIXJ/CyRNQsYsD IBPXDYKU+4ZPPXkqNnxxaPMO1x8/laZfsqplvI8XKjHnFL642w8a7vTf6HaTk481B/XhO0v8qJ1k k/+3G20K4zyRK34bRmmB/3mHZDOQJ/ses920PWx7yPag7X7bDZu53nfN9j7bfbartntt77VdOc+8 x/Zu27uc7rHdbbs8zrzTdsnpLqc7x1mQ/4kAAQQQF+S/Ih9kk0EdkBDIMKZssNurG+Q/okb2bQeP xSDXV4FcWQlyGZCTAvm3CBBAQnkgj9oesaVA3m8rBOLC3ONUCMaC/EsECCChdIP8UwQIIKF0g/xD BAggoXSD/F0ECCChioEMB4gzQa5EYJaCpGDGIH8TAQJIKN0gfxUBAkgo3SB/EQHSMojxlowBUgPk z6LRFG63X28JSDsgmywojk5QDSDDRNUA8oAtBuIeIF51cmBiIDGYu50WHiCKhXLPnuUtlDOb/Xt4 3SDPiniGNADyJxEgzYO0MairAvmjqNXdXlUgfxC1emAIyLbpBvm9qDWQD9jcA8MYyHCAmAJJHCC6 ILEDxMIwgLQJ8jsRIICE0g3yWxEggITSDfIbESCAhNIN8msRIKPmHiCunLACpE2QX4kAASSUbpBf igABJJRukF+IAGkaRL61BiDVQH4uGk/h2o8bzqm7ILE3EHBBcieqIjAxkNIHiA5MAuTwpjPbrTpR CbLrmbPchXKmgYVyKkGeEbW2LkslyM9EW4D0z8zR8xOQOEidQd15EN0gPxVttFDODCMWIFMg9XJ3 Hux71ioCke/b+xPRNiDG+dzMM2TuAeJKGAvyYxEgykEO52bGL1mqQX4k2g6ka3FQ1wcSSTfID0WA ABJKN8gPRIAAEmo2yNJ/uOOCROarrjjNhcmcuLIg3xcBAkgo3SDfEwECSCjdIN8VAQJIKN0g3xEB 0jbI5m+tEQMZFszFQNwFc0PXnCIw9zpVgrEg3xa5Kxe70XR38LdPAlINxLhTq2b0CZAjgex6+ixv oZw31w1IFZCnRdMg1dIN8i2Ru5S01ZWLKkE6uXjx/G+H5deAHBHkm6LEbq+ptJIRkAiIu532o0pt kNgbmaVACh8gLoVxQVwYmwX5hmhyY/e7wTUGFECyQPbrdQA5KsjXRa2ey1IF8jURIBcKZLsjdVUg XxUBcrFANjvbqwrkK6KLNoYME1VLQSIwVxMwLkguTAJoQ5DEc003yJdF1UBGn/yLAQGkHZAviRhD AAmlG+SLooogU/vPgNQHmVxhpBvkC6LxqpPAOzkUmw9ZBjJMVLkg7gHi/U65B4iR48TYhNXcA8UE UALEDNvfjL5WasZwilY3yOdF7iZyFzmYYk+RyUND3SC7zjH87eMtAyoHMplukM+JnKWknQuy9Zy6 bpD9uFueYOmRugqQz4rcQb0LDeIFfABZAmL89+0t9XxZerZXBchnRMmNXeddfFeD5B4gRmDuc1oL k3nAuASkzjpf3SCfFrU+hQvINukG+ZQoYwypESC5IJWeQbpBPikCpAGQT4h4ybpgIAX3srJOvz9u c0FSC+bcA8QbTjMPECvBWJCPi6rOGMYPMgHJBin2DJk8ka8b5GOi9KBebsIQkEIgpadwAycxdYN8 VFQRRMrs7xqQmSC76xfyGD1DnBP7ukE+ItpiL6unGIMMvz9EDYj8/SGzQI60l+WCzH2GLD1AjME4 PjGYUkA2C/KUyF/k4G7GUgEyHyS0h1tuOqR/zTqMHqsG9ZMCeVKUmj8vPcb0+7veklXdIP3weYbh v4GZD1LwXFbO6neVIE+IjrHCJLLN9x8AAaRtkA+JkoP6RiDDvMhakEyYawthlgLZUiDHnY2aNYao AvmgqLVVJ4BsLGHTDfK4qDIIY0gRkAJquYutVYI8JspZBlTiWWRGnwBZDFJ79btKkEdF6a1dZZSJ gpQ6QIzBXB8XgykNZJsNUn/1u0qQm6LWd3sB8TZVlRct3SCPiHL2smr+O3WVIA+LAAEk+DCqQR4S Za06KbHFzeEjIGtAijW5d1AcZOGBYgymNJDNgjwgqrcua3IFJCC5IOVWv08ORbpB5Dc2nsL131Gu 1GLrpWd7VYMEFrAdZfU7IA7IDZG/MM474V5qaa97v4AMILvOMTLeUa5US+dDVIDIHW9h1B0w5GtW oeMQQOaAyK02Bil1pJ4LMiy+cEGGbjqVgpmev1p84JiAsiDyHt1BvfMG9TIgiUXCgARBAu8oV/v0 u0oQeQ/1ZgzzjkMAKbGtszzyjtRVgvhfim6nYs+QxM40IHkgXbkxhGdIHMTfE57cjqVEGEPaApl+ GBdk7gFiDCY2ceUA3ZgJkwuUgLIg/pzV1KaqsSwLkHyQOukG8VeXTm2qKmSA5ILUng9RCSLfBQUQ QABZA8JeVgUQ/zfpbd5skMIwiePFKNRcMAduCUije1knBeL/zvuJLdXoGKIF5GjrsgCZALksGm9r O2G44bosQEabJbAMqNG9rJMC2XXprMAbmB1tXRYgEyCXRHJdVgCkloxukLtE/qoTCVLt2ZIEKQ3j ALnHi6WgEmAJkL2AGf8VkCOD3ClK7PZWf+MAQNLb6gib//zde6W0bpA7RFucy/IOdgDZFuT8mxgf 4OgGeYdoIxBv93r47QhqQORvR9gexDiflT9D3i7aBMQ/7tyDDG9fmzpAXAoTAXowUi7UQrgGQIbf juC+ZKkGeZtoExAz7PcevgzIRiCRdIO8VQQIIKF0g7xFBAggoXSDvFl0UUCWHijGgCJQiePHaDG4 BCAgbYK8SQRIAyBvFAHSNEjgnRwAOT7IG0TOuqz+Y30RQEIggZN+gNQA2XX7Wd5COeP9nnNAaoDc LnIWyoXe4m9TkLkwMaAY1DwvrxhgJqTd+q8XuS9ZgDQH0tagrgLkdaLxXpb/vr2AbAmyXbpBXisC BJBQukFeIwIEkFB7kA/bYjC5QDGoBFjEKwm3ENSCvFoECCChdIO8SgQIIKF0g7xSBAggoXSDvEIE CCChPBAXZi5QCirTLeGXLOFrQW4TAQJIKN0gLxcBAkgoQIIgDcwYqgR5mai1OXVAPJBtV52oBNl1 61neG/00sFDOBUkB5YLlAmaW6ZzKgtwqam2hnEqQl4paf8kCpLFBXQXIS0St7/bqBtku3SAvFgEC SCjdIC8StQbyRKQUVG4p0EJl/lgA0ibIC0WAABJKN8gtIkAACaUb5AUiQAAJpRvk+aKLAnKaNQxi VoKYdRtm1a2X396CPE8EyPpbnwCIER/7P6oGea5oExA5WwzIFIjxPh7Hw5sq1g3yHJEzY9h1oZ/f GiD294fU3iQlbr3o9vL3h8RADhvpuEsd4iDqCvzPBzbVBiA0bqA5PoiRd49HqkogR99tOJnqDOqU mXzh4ueXiIiIiIhOvJWHPkb8AtElN171a1P62y999DZbeXJg3ZkFI7+HxfdxOhhdmdNnplv6I9rf btWjm27xozdZAZAVm3TbR2+y9ZvEOJ/n337LR2+u1SDG+8PsOwDEtnbGanj9X/aiIUaPxY8+TFqc EsiK3V6xw7tgWO036PJ/DS6nK05oUCciIiIiIiIioo3KmN91/tmSCf6ZypRzHtlE/3IaZ3Cbyj+R PLxFcX/u3Z6BN+MbmM4sPwNNU+1PjotZLLNnsihyu/cgpusWn5SnqQLPkB6gX73Tf8W4NxhNPFHJ YiBybAk8QzpAjpQ/qJvDc+TA5Y4hvGQdLW+3d5hEl2OJXChxGOzZ7W0mHBoLECIiIiIiIiIiIiIi IiIiIiIiIiIiIiIiIiKihvo/JZeriU8SjqwAAAAASUVORK5CYII= --_005_012C3117EDDB3C4781FD802A8C27DD4F0D4D8465SACEXCMBX02PRDh_ Content-Type: image/png; name="CP-burst-single-cons.png" Content-Description: CP-burst-single-cons.png Content-Disposition: attachment; filename="CP-burst-single-cons.png"; size=4362; creation-date="Sun, 29 Jul 2012 22:36:33 GMT"; modification-date="Sun, 29 Jul 2012 22:38:54 GMT" Content-Transfer-Encoding: base64 iVBORw0KGgoAAAANSUhEUgAAAZAAAAGQCAMAAAC3Ycb+AAADAFBMVEX///8AAABAQED/AAAAwAAA gP/AAP8A7u7AQADIyABBaeH/wCAAgEDAgP8wYICLAABAgAD/gP9//9SlKir//wBA4NAAAAAaGhoz MzNNTU1mZmZ/f3+ZmZmzs7PAwMDMzMzl5eX////wMjKQ7pCt2ObwVfDg///u3YL/tsGv7u7/1wAA /wAAZAAA/38iiyIui1cAAP8AAIsZGXAAAIAAAM2HzusA////AP8AztH/FJP/f1DwgID/RQD6gHLp lnrw5oy9t2u4hgv19dyggCD/pQDugu6UANPdoN2QUEBVay+AFACAFBSAQBSAQICAYMCAYP+AgAD/ gED/oED/oGD/oHD/wMD//4D//8DNt57w//Cgts3B/8HNwLB8/0Cg/yC+vr4AAAACAgIEBAQFBQUH BwcJCQkLCwsNDQ0ODg4QEBASEhIUFBQVFRUXFxcZGRkbGxsdHR0eHh4gICAiIiIkJCQmJiYnJycp KSkrKystLS0vLy8wMDAyMjI0NDQ2NjY3Nzc5OTk7Ozs9PT0/Pz9AQEBCQkJERERGRkZISEhJSUlL S0tNTU1PT09RUVFSUlJUVFRWVlZYWFhZWVlbW1tdXV1fX19hYWFiYmJkZGRmZmZoaGhqampra2tt bW1vb29xcXFycnJ0dHR2dnZ4eHh6enp7e3t9fX1/f3+IiIiampqrq6u9vb3Pz8/h4eHy8vL//5YA /6AA5v8Awv8An/8Ae/8AV/8ANP8AEP8CAP8GAP8JAP8NAP8QAP8UAP8XAP8bAP8eAP8iAP8mAP8p AP8tAP8wAP80AP83AP87AP8/AP9CAP9GAP9JAP9NAP9QAP9UAP9XAP9bAP9fAP9iAP9mAP9pAP9t AP9wAP90AP94AP97AP9/AP+CAP+GAP+JAP+NAP+RAP+UAP+YAP+bAP+fAP+iAP+mAP+pAP+tAP+x AP+0AP+4AP+7AP+/AP/CAP/GAP/KAP/NAP/RAP/UAP/YAP/bAP/fAP/iAP/mAP/qAP/tAP/xAP/0 AP/4AP/7AP//AP9idicVAAANxUlEQVR4nO2dia8lRRWHu/5UNWpcIsa4RI0S16hRiRrjEjVqXKNG wGEbxgFkQJYRcBYBAUEQARcQEVlc596u0/2r01W9VndXd/2+ZO57824v79V3u2s5p6uKghBCCCGE EEIIIYQQQgghhBBCCCGEEEIIIYQQQgghhBBCCCGEEEIIIYQQQgghhBBCCCGEEEIIOWBGbGIOHL+U bxp4x7d784c9zpopvtJqbuP9QalCpMiPfIVPIb2xn+/jZx0/8WVBF3IhVN/Xe1WvSkjhHO3wtnu0 6ix04kOKv/FqoNhNvaXdy+D7RglRRzPu0YzehyBlbeB84gu4yUDJ23pDbekWblW1HCXIkeW1wHPR hx8s/qYQKcj63UJtaQvaecN/NKOOTCNevAWG10x1w/cLMd1CqltWfWTDOiREs1KXarh82xagrtRD zd6uSr3w7kUIIYQQsg9wQCS8UXOvSGcPHD9fnN7zoH2inZ4gTi/c87apux5OD8TU7/q2N2p7+An2 doxzfOgF5Uu7EIP/qj66/qe3r/X5BinrveoeulH7ZkzVndY9b/lPtZVbYEZviUfDLU29pXMsHCTG 43dVZ3sHx5rCGzRvKf5PskcIXAhV6cONyzkJB7Ww8MLv6isE3zl+b9zt/a8NIRCbml+IabymCt45 Au86Q4BGV9XO3q3bo0zfUKWpjxAbUytv+fxtEc+tagN/HtZ9CzcbTKZ4/nilZDUh/5uG7P+q5XnL M5bHLY9YHrRcPGLOWe61nLXcZbnD8kvLbZYzJeZWyy8st1huVtzkYoU4fwiFUAiFKCH/BTw9pDUq dQrxCZE6ZYVmb95C/g0k0vib6qNinJCLE4WcoZBdCfkXQCEU4iO6kL9ZnrZoIQ9YVhJyusQKeRWg EArxkbeQlwEKoRAfeQt5CaAQCvGRt5B/AhRCIT6iCXnF8pxFhPzeooVcKPm1ZR0h/wAohEJ85C3k BYBCKMRH3kL+DlBIykJMI4OMQpYQ8jzghHALmyS5vBEKCQtZON8kshAZgRAhT1lEyMOWgJB7LCEh t1viCjlw+GUbiXJmtefh8xbyHMArJAEhzwIUkryQbVfqmxTyV2Bvzd5NCvkLsLeOIYXEIW8hfwb2 KkTqSC3kIcv9FivkPosIudtCIVkKeQagEArxkbeQpwEKoRAfeQt5CqAQCvGRt5A/AnsTIvlmMjQk f+djFi3kfElIyJ0WCslKyB8ACqEQH3kLeQKgEArxkbeQxwEKSVoITq1BIYsJeQxwQ7j2dcMx9Rct EoATIY9aRMhvLErIryxaiCTIaSEhEVrITYp+QupJZ7abdbJJIQcOv5xOlDM7SJTbpJBHgb3lZW1S yO+ANYSUV6ZzfVJIWMgylbo6Sd5CHgFWSpQzUmNRSJuQ5dCNBztnbUZCcN7eh4F1hBj1db4r5EmL CPmt5aI7Y8BQIV0dwmFXyEMAhWQupB6bcW9ZWQt5EFhPSLFIpU4h48hbyAMAhVCIj7yF3A9QCIX4 mCxE5rKXWY7kcQtJ5pChIjWBsggJTRighfQNTA0TchGgEArxkbeQCwCFUIiPvIWcByiEQnzkLeQc QCFpC9ns1Br/schckn+ySMpsSMjMEwZoEX4h9wE6c7Fwwt3e1ScpZDEhRodWjfOFQmYScuBww2wk yjVi3RSyiJB7gXYhi5G3kHsAnUq61czFXQopMHnx+L86/ZpCZhRyFuho9pqFMhkpJCBEl1NVq6Qu RFYulRm7QwtKqonLLighEpgaKmRgh/C0K+RuoLWwy2bwEhUKhfQSUuXrUMisQu4C9jKWtWkhdwIU sikh2+mpb1rIHQCFbEvIZkZ7Ny3kdmAvdYisDy/rouiFXOR5GDVhwPnICXI3K9IV0nGt5S3kNmAx Ic6X5tsUQiHpCDkDsA6hEB95C7kVWFBIW/uZQpYX0pphlLcQ7MG4WSeemRyixUPmFSIL2+uFXPTE ZWrCAC1EdwiHJsiFhJz20yHESPkb52exIoZtavMWgnvoItJJDibaJdLaNcxbyIGjjGb5NNKA4glp JW8hWMuoVNJCC9lKTH2fQqp6N76CuXvqmxbyc0BX6oWvEo/gh0LGCDHNeXtjXS9zj/ZuWsgpoLOw l5nFd7KQly0ygXLPCQPOBR7UCQWmQiK0kI4O4WnnchgmZJk837yF3AjsJYSbiZDFyFvISaBHHbIE FNJXyEJXUN5CbgAoJAEh1wO8ZW1MSMRW1qzD77KQi0xc1jVhgFrRPlZgSovoJ+RaYNGIYbiTSSG9 hUS7QloH8vMWcg3QXanHCxhSSCQhsUO4nkHMvIWcABYUgmaqQ1PIQCGH7SP5cK4QNbCft5CrgTVa WaUKV8jU9UM2JwTXDxkkZKZWlhYy9QqRictkwoDQgzoqMCWRKRHSFZjq2yEMXBEaK+QqoJnkoIsx FhQyXIivhRsvHFLes+raI2qlvmkhVwJd8fPYdUzZ3m2krOYt5MBRRnMCs6aQiGNZc2a/b1rIT4E5 MkwCZV69UAiFpC3kJ0BnpZ64EJknS55P18+FhPKx1ES9EgeJ1f/oEuLk/ISFzBuNmqUO2YWQHwNb zzqhkHnIW8iPgIWFsA6JIiSCtbmSrXch5IdAnzSgGFeRcb5QyGghqWe/70LID4Du0l6klhktRJ5P 18+FhPKxVBxEAiFd64SEOoQjRZwaKyT97PddCPk+sPVmb35C4uc4eM+StZDvAX1aWSk/p74LId8F KIRCvKfJWsh3gF5ZJzFK3NSvFDJFSDRaWwejhbxkkedCJB+rZ2BKHkzXz4MM7RAOFKGEfBtYLi+r NQOSQvoKiZf93loV5S3kW4Abwm3OKBcr2Xqu0d5dC/EksM2S/U4hSsg3gWZiXGPAPVZqrz4uhYiQ A0cZPWaUi8Vc8ZBdCPkGAI6KWgbesyL1QyhkiBAsNVdIrJ76XEJCCXI6MCXPpeuFQgIdwrEBqZ4i TjkzzZivA7pSLxqVehwhHUnCFOIV4plRLvXh910I+RqwXMRwnn5IZkIi0n6l5S3kq8BSV0hHY5pC +gkp4tUhvELCQr4CLBegYh2SmJD204wVIuuFSGBKP5cu6xUGOoTSI+wKSPXtCHYI0FghXwaWihh2 nYVC+gpZhryFfAnocYXMbqOgkP5CUo+H7ELIFwEKoRAKmSKErawFhHwB2GorS9YLkcDUExa9XmGo Q6giUjogNVVEQMCNJ13GCEm0lbUrIZ8HtlqH5CJktrwsCmkR8jnALWsbMNxAXlYeQqrSj591QiEt Qg5ccQnPBGaz5WVRSIuQKwDMy/IIWcpM3kI+CzSzTlDIYlfLYCEvWmSiAFmvUHcI1QQBZ+XJHJUR 11dEXwEnO3CW0wkKqQwY978UMrOQzwAdzd5kJw7IRYivrGYo/uPsvWg6byGfBtYYy2p0dihkXSHH X8Lt4OQt5FPASkIazeuhqyNsXgiujrC+EKO+Zn6FfBJYRUiz39lbiExYJhMFyAM60iGUxDiZIEAC UqGZAUaK6CvghnYSECKrI+hbVtZCPgGsIsRIu7f+MYWsJCRA3kI+DlAIhfjIW8jHAAqhEB95C/ko sDUhsl6hdAhlojKZIEACUjohTkQEIlFTBYRK/Pp2KCRNIR8BKCQBIR8GKCRpIZ6ZHChkfiEfAlRe Vvm6vBEK8QnxDPpRyBJCDlx+iUainGmsc04hSwi5HFCJcr4p/pIS8qxFFmyRgJROiBMRekYyeRJH 9QD7Chha8Ne1Y0v/g4C+ZVFIckLSrtR3KeQDgNvKas7bSyFrClmPvIW8H6AQCvGRt5D3ARRCIT46 hbxgkYCUiJAJknUgSs9EdovqASoDXQU/sKCvu7YfVsh7AQqhEB95C3kPQCEU4iNvIe8GKIRCfOQt 5F0AhVCIj6CQVyxPWkSEJMLJRAA6ANUhQBvQBT+0gK/pyQkXK+SdAIVQiI+8hbwDoBAK8UEhXiEJ RgyzEPJ2IPWYOoUklnWShZADl12iMdFPgoly0hGUB3GkAygCpOMnASfJdNMF31HiXQV8YiA/64cV chmQeqJcFkLeBqR+y6KQxCr1LIS8FUi92Zu3kPXIW8hbAAqhEB95C3kzkKoQWRhSJh6TDp909KTA JaKkC7pvyXaU2NWRuMoPhaQp5E0AhVCIj7yFvBGgEArxkbeQNwAUQiE+8hbyeiAVIVICV2ZFwkLM RCFmWsFM2nv8/lbI6wAKmb73DoQYeC2/zVrIa4FVhGC0mELahJjG6zw+GqHivIW8BlARw6LwfX6X EHIkIyG4fkhISF1I86Y6hIVkh+eP9xTVCkKIi6iZX4jBw9NHFwsJmb3ZsBuWqdRJT/DGxc8vIYQQ QgghZOdM7PoYWEB0zM6Tlk0p9x979jSZODgwbWTB4O8w+hj7kVHEGT4zxdiPaLnfpLObYvTZkySC kAlFuu7Zk2R6kRj1dfj+a549OSYLMY1vBh+AQixTI1Zy/x9304DaY/TZJWixJyETmr3Q4B1RrZYF Ov5pcAxX7KhSJ4QQQgghhBBCCCEr0SO+qx5bMt7vSRz6jCOb4H/2MYKbFM2BZJmiuBx7tyPwxt3B FGb8CDRpoxochyiWqTRZKVjupRBTFKMH5UkbniukFFBm75Q/MXoHJ/BEYhISgnWL5wopKGQmmpW6 qa+RWpeuQ3jLmo1Gs1eC6FiXYKJEXdmz2ZsM9JAYFEIIIYQQQgghhBBCCCGEEEIIIYQQQgghhBBC CCGEEEIIIQnxfzW3raal8BcAAAAAAElFTkSuQmCC --_005_012C3117EDDB3C4781FD802A8C27DD4F0D4D8465SACEXCMBX02PRDh_ Content-Type: image/png; name="CP-burst-single-simple.png" Content-Description: CP-burst-single-simple.png Content-Disposition: attachment; filename="CP-burst-single-simple.png"; size=4071; creation-date="Sun, 29 Jul 2012 22:38:54 GMT"; modification-date="Sun, 29 Jul 2012 22:38:54 GMT" Content-Transfer-Encoding: base64 iVBORw0KGgoAAAANSUhEUgAAAZAAAAGQCAMAAAC3Ycb+AAADAFBMVEX///8AAABAQED/AAAAwAAA gP/AAP8A7u7AQADIyABBaeH/wCAAgEDAgP8wYICLAABAgAD/gP9//9SlKir//wBA4NAAAAAaGhoz MzNNTU1mZmZ/f3+ZmZmzs7PAwMDMzMzl5eX////wMjKQ7pCt2ObwVfDg///u3YL/tsGv7u7/1wAA /wAAZAAA/38iiyIui1cAAP8AAIsZGXAAAIAAAM2HzusA////AP8AztH/FJP/f1DwgID/RQD6gHLp lnrw5oy9t2u4hgv19dyggCD/pQDugu6UANPdoN2QUEBVay+AFACAFBSAQBSAQICAYMCAYP+AgAD/ gED/oED/oGD/oHD/wMD//4D//8DNt57w//Cgts3B/8HNwLB8/0Cg/yC+vr4AAAACAgIEBAQFBQUH BwcJCQkLCwsNDQ0ODg4QEBASEhIUFBQVFRUXFxcZGRkbGxsdHR0eHh4gICAiIiIkJCQmJiYnJycp KSkrKystLS0vLy8wMDAyMjI0NDQ2NjY3Nzc5OTk7Ozs9PT0/Pz9AQEBCQkJERERGRkZISEhJSUlL S0tNTU1PT09RUVFSUlJUVFRWVlZYWFhZWVlbW1tdXV1fX19hYWFiYmJkZGRmZmZoaGhqampra2tt bW1vb29xcXFycnJ0dHR2dnZ4eHh6enp7e3t9fX1/f3+IiIiampqrq6u9vb3Pz8/h4eHy8vL//5YA /6AA5v8Awv8An/8Ae/8AV/8ANP8AEP8CAP8GAP8JAP8NAP8QAP8UAP8XAP8bAP8eAP8iAP8mAP8p AP8tAP8wAP80AP83AP87AP8/AP9CAP9GAP9JAP9NAP9QAP9UAP9XAP9bAP9fAP9iAP9mAP9pAP9t AP9wAP90AP94AP97AP9/AP+CAP+GAP+JAP+NAP+RAP+UAP+YAP+bAP+fAP+iAP+mAP+pAP+tAP+x AP+0AP+4AP+7AP+/AP/CAP/GAP/KAP/NAP/RAP/UAP/YAP/bAP/fAP/iAP/mAP/qAP/tAP/xAP/0 AP/4AP/7AP//AP9idicVAAAMoklEQVR4nO3dia821xzA8Tl/KoJYoiKWIDTWIGi60VYXVVpV2hLE GsQSW1QsQSwVxBJLYwmC4D53zjzPb84y58zMmTPneX7fb9J73/c+2+187jwzZ+a8c7uOiIiIiIiI iIiIiIiIiIiIiIiIiIiIiIiIiIiIiIiIiIiIiIiIiIiIiIiI6JBZcBdz6PpTf6MRt4Qe7n8x41WV Flpa/n2CX+gpBpThS6GFD0h29uf7+mdd/sT3C7obVoTjn0+POn50QLrRsx1uHj/b8VUwCTUsfu+j EYvdnO5pH2Xk7cYBcZ7NjJ/NuI8hWb81GP3Ed+JNRix5u91w7jleuMdNyzXC8MzDx06+Fh7h5OL3 QYYFebq1c+5pF/TohvCzGeeZEQkWXGBynTm+4YdBTBrk+JZ1embDNiSWv1EfNsP9zXYBuhv12G5v aqPeBR9FREREdKnJQyTxO/mPmn7SmfenY6Px9KzHzH8dymg0Lg/cbE6DkdGYxJxujd3fjgEn709u 0yBG/mfkumSCjxrfMgwL4/cnr+MA2x2LD3853mv85mbcewbub9L3Jzd59Cl+B7FYxVtW+E3udKsH UuqbvuTkcdjYre4aIm+5/rMJ3z/0llUn4308n8QJp9ito4OC48OHpw125P7+GcMKDd/W9E/bZRT4 SZ83gqmQGa2lVb8Ho7TA/7xDshvI430P2x6yPWh7r+0B23ts77aZ+/rutd1ju9v2LttdtjtHmTts 77S9w+l2p9vGmVudbnG6OZwF+Z8IEEAAcUH+K/JBdtmoAxICGbYpO+z26gb5j6iRfdvBYzHIfatA 7lwJchsgFwXybxEggITyQN5v2wrEgXFBUjApkEwYC/IvESCAhNIN8k8RIICE0g3yDxEggITSDfJ3 ESCAhDqCfMCWC3K/LRckMkC8wykGM3eAmICxIH8TAQJIKN0gfxUBAkgo3SB/EQHSMojx5osBUgPk z6LRKdz+vz2myAESB9llNnES5H22GEhsgHi3UwTEPVHlwsQGiCthxES5p67yJsqZ3f49vG6Qp0Ss IQ2A/EkESPMgbW7ULxrkj6Jz2e29aJA/iM5lYAhI3XSD/F506SCZA8QUSGqAmAsDyFmA/E4ECCCh dIP8VgQIIKF0g/xGBAggoXSD/Fp0biDDiaqNQOYOEFcOFAFpE+RXIkAACaUb5JciQAAJpRvkFyJA mgaRl9YApBrIz0XjU7j2Y8Pn1N0Jc/c7DTAuSAzmrnExmNgAMRckApMAOV10pt1ZJxcJcujJq9yJ cuYMJspdJMiTonObl3WRID8T7QHSr5mj9ROQOEidjbrzIrpBfiraaaKcGbZYgEyB1MvdebDXrFUE Iq/b+xPRPiDG+RxfQ4YLmcVACg0QUyBLYRJAFuTHIkCUg5yOzYzfslSD/Ei0H0iXtVEHZJ90g/xQ BAggoXSD/EAECCChkiDDAHEtSATGGR+uhskEsiDfFwECSCjdIN8TAQJIKN0g3xUBAkgo3SDfEQHS Nkhzl9YoBRIbIE6fr9oaxoJ8W+TOXOxGp7uDv30SkGogxj21akafANkI5NATV3kT5bxz3YBUAXlC NA1SLd0g3xK5U0lbm7moGqSTkxev/3aafg3IhiDfFCV2e02lmYyAREDc5XTcqtQCecQWAxlOVD3g dL/TTBjnfNVmMA6QBfmGaHJh97vBNTYogGSBHOfrALIpyNdFrR3LUgnyNREgZwVSf6SuEuSrIkDO C6T60V6VIF8RtbYNGQaGsV90v3bCnAsSPl+VDbMSaEeQxLqmG+TLomogo0/+zYAA0g7Il0RsQwAJ pRvki6KKIFP7z4DUB5mcYaQb5Aui8ayTwJUcip0PWQYyDBDngsyEcc5XrR4oxmAcoASIGZa/GX2t 1BnDKVrdIJ8XuYvIneRgiq0ik0ND3SCHrjH85eNNAyoHMplukM+JnKmknQtS+5w6IK7HRvOylo7U VYB8VuRu1LvQRryADyBLQIx/3d5S68vSo70qQD4jSi7sOlfx9U5QuSClBoguTGSAmAuzEmgJSJ15 vrpBPi1q9RQuIPumG+RTooxtSI0AyQWptAbpBvmkCJAGQD4h4i3rzEAK7mVlHX4vDZIJc89MmBiQ C5OAsiAfF1U9YxgfZAKSDVJsDZk8kK8b5GOi9Ea93AlDQAqBlD6FGziIqRvko6KKIFLm+NSAzAQ5 3L+Qx2gNcQ7s6wb5iGiPvayeYgwy/P4QNSDy94fMAtloL8sFKbWGpAaIEZjIODEbyB8qTgON15AP i/xJDu5iLBUg80FCe7jlTof071mnrccmG/WzBHlclDp/Xnob0+/velNWdYP0m88rDP8CZj5IwWNZ ObPfVYI8JtpihklkmR8/AAJI2yAfEiU36juBuNfNGnJBUuORTJh7I6Vg5o5THKgUyLZno2ZtQ1SB fFDU6qwTQPZNN8gjosogbEOKgBRQy51srRLkYVHONKASa5EZfQJkMches99Vgcj/wfTSrrKViYLM HSDOPT8SHifWApoNst/sd1UgD4la3+0FxFtUVd60dIM8KMrZy6r579RVgsi9E0AACb6MahD5jWfN OimxxM3pIyBrQIo1uXdwBBkOeuaCLD1hFRsorgRKQTlgFkSOYOvNy5qcAQlILki52e+TmyLdIPIb Gp/C9a8oV2qyde7RXkBGC9+bwLbJ7HdAHBD5yv7EOO+Ae6mpve7zAjKAHLrGyLiiXKlyz4eoBJHP KIy6E4Z8zyo0DgFkDohcamOQUiP1pSBzB4i5MBGgiI8HVAjKgsh7uBv1ztuolwFJTBIGJAgSuKJc 7cPvKkH8L0WXU8EzhsvGIYBs1fSaphvEP94YXU7F1pDEzjQgeSBduW0Ia0gcRP5DhXonqNiGNAYy /TIuSO6JqqUwDpA7XiwNFIGyIP61BKYWVY1pWYDkg9RJN4h/kbmpRVWFDJBckL3Oh6gCuV0ECCCA rAFhL6sCyK2ic93LWgqTAEqMG5NQKTAHbglIo3tZFwVyi+hctyFaQDablwXIBMjNovGyticMG5iX Bcho6ZefdQLIBMihm64KXMBss3lZgEyA3CSS87ICILVkdIO8XeTPOpEg1daWJEguzIOREkARHw+o FJgtAXIUMOO/ArIxyNtEid3e6hcOACS9rDZY/NdX75XSukHeKtrjWJY32AFkX5Drb2I8wNEN8hbR TiDe7vXw2xHUgMjfjrA/iHE+K19D3izaBcQfdx5BhsvXroWZCxR2SoKthGsAZPjtCO5blmqQN4l2 ATHDfu/py4DsBBJJN8gbRYAAEko3yBtEgAASSjfI60XnApKCyQVKQEWcolBz4ZwAaRPkdSJAGgB5 rQiQpkECV3IAZHuQ14iceVn9x/oigIRAAgf9AKkBcujGq7yJcsb7PeeA1AC5UeRMlAtd4m8XkLUw MaBMsIhXNmAmrF36rxa5b1mANAfSxkZdFcirROO9LP+6vYDsCbJfukFeKQIEkFC6QV4hAgSQUEeQ R20xmBhQDGot2LRbtAEsAWxBXi4CBJBQukFeJgIEkFC6QV4qAgSQULpBXiICBJBQSZAYTC5UCmym Y6pMZwvyYhEggITSDfIiESCAhAIkCNLAGUOVIC8UtXZOHRAPZN9ZJypBDt1wlXehnwYmyj2aKAY1 F26m49IS3hbkBlFrE+VUgrxA1OpbFiBdWxt1VSDPF7W62wvIvukGeZ4IEEBC6QZ5rqg1kMcipaCW lgu8TYC0CfIcESCAhNIN8mwRIICE0g3yLBEggITSDfJMUWsgLkwK6rxrGMQVGb7lXBCzbsGsevTy x1uQZ4gAWf/oCwAx4mP/R9UgTxftAiLPFgMyBWK8j9t4eKeKdYM8TeScMey60M9vDRD7+0PUgMjf HxIDOS2kbac6xEHUFfifDyyqHUBo3ECzPYiRT49Hqkogm+82XEx1NuqUmXzj4ueXiIiIiIguvJVD HyN+geiSB6/6tSn945e+eputPDiw7siCkd/D4ue4HIyuzOEz0y39Ee0ft+rVTbf41ZusAMiKRbrv qzfZ+kVinM/zH7/nqzfXahDj/WH2EwBiW3vGanj/X/amIbYei199OGlxSSArdnvFDu+CzWq/QJf/ a3B5uuKCNupERERERERERLRTGed3nX+2ZIJ/pjLlHEc20b9cxhHcpvIPJA+XKO6Pvdsj8Gb8ANOZ 5UegaarjwXFxFsscmSyKXO49iOm6xQflaarAGtID9LN3+q8Y9wGjE09UshiI3LYE1pAOkI3yN+rm tI6cuNxtCG9Zm+Xt9g4n0eW2RE6UOG3s2e1tJhwaCxAiIiIiIiIiIiIiIiIiIiIiIiIiIiIiIiIi oob6PwrAxuMXaABRAAAAAElFTkSuQmCC --_005_012C3117EDDB3C4781FD802A8C27DD4F0D4D8465SACEXCMBX02PRDh_ Content-Type: image/png; name="CP-burst-twice-cons.png" Content-Description: CP-burst-twice-cons.png Content-Disposition: attachment; filename="CP-burst-twice-cons.png"; size=4331; creation-date="Sun, 29 Jul 2012 22:36:33 GMT"; modification-date="Sun, 29 Jul 2012 22:38:54 GMT" Content-Transfer-Encoding: base64 iVBORw0KGgoAAAANSUhEUgAAAZAAAAGQCAMAAAC3Ycb+AAADAFBMVEX///8AAABAQED/AAAAwAAA gP/AAP8A7u7AQADIyABBaeH/wCAAgEDAgP8wYICLAABAgAD/gP9//9SlKir//wBA4NAAAAAaGhoz MzNNTU1mZmZ/f3+ZmZmzs7PAwMDMzMzl5eX////wMjKQ7pCt2ObwVfDg///u3YL/tsGv7u7/1wAA /wAAZAAA/38iiyIui1cAAP8AAIsZGXAAAIAAAM2HzusA////AP8AztH/FJP/f1DwgID/RQD6gHLp lnrw5oy9t2u4hgv19dyggCD/pQDugu6UANPdoN2QUEBVay+AFACAFBSAQBSAQICAYMCAYP+AgAD/ gED/oED/oGD/oHD/wMD//4D//8DNt57w//Cgts3B/8HNwLB8/0Cg/yC+vr4AAAACAgIEBAQFBQUH BwcJCQkLCwsNDQ0ODg4QEBASEhIUFBQVFRUXFxcZGRkbGxsdHR0eHh4gICAiIiIkJCQmJiYnJycp KSkrKystLS0vLy8wMDAyMjI0NDQ2NjY3Nzc5OTk7Ozs9PT0/Pz9AQEBCQkJERERGRkZISEhJSUlL S0tNTU1PT09RUVFSUlJUVFRWVlZYWFhZWVlbW1tdXV1fX19hYWFiYmJkZGRmZmZoaGhqampra2tt bW1vb29xcXFycnJ0dHR2dnZ4eHh6enp7e3t9fX1/f3+IiIiampqrq6u9vb3Pz8/h4eHy8vL//5YA /6AA5v8Awv8An/8Ae/8AV/8ANP8AEP8CAP8GAP8JAP8NAP8QAP8UAP8XAP8bAP8eAP8iAP8mAP8p AP8tAP8wAP80AP83AP87AP8/AP9CAP9GAP9JAP9NAP9QAP9UAP9XAP9bAP9fAP9iAP9mAP9pAP9t AP9wAP90AP94AP97AP9/AP+CAP+GAP+JAP+NAP+RAP+UAP+YAP+bAP+fAP+iAP+mAP+pAP+tAP+x AP+0AP+4AP+7AP+/AP/CAP/GAP/KAP/NAP/RAP/UAP/YAP/bAP/fAP/iAP/mAP/qAP/tAP/xAP/0 AP/4AP/7AP//AP9idicVAAANpklEQVR4nO2dic8kRRmHu/5UNWo8IsZ4RI0Sz6hRiRrjETVqPKNG RHblXF32EFhgF5ZdWeVQQGRFXEABOaLuTNfb86urz6ru6q7fk+zM901Pd+9Xz9TU8b5dXVWEEEII IYQQQgghhBBCCCGEEEIIIYQQQgghhBBCCCGEEEIIIYQQQgghhBBCCCGEEEIIIYQQQsgONeItasf+ qd6oYItvd/fFHmctFF9pue/xvlCrECnykq/wKaQ3+vO9/6zjJ74u6EoqQvPzYa/m0RJSGUfbbTaP 1pyFTnxI8TuPCopdHd6p91K4XVlCrKMp82jK3ocgdWtgfOIr+JKBktfthvVOs3CbpmUvQY4sjxWe iz78YPG7QqQgD1sr6526oI0N/qMp68g04sVbYFhnmi98vxDVLaT5yjocWbENCeE26tIM15t1AdqN eqjb29WoV969CCGEEEI2A86JeLYGf5l83gTH3ALGANq/Od2JiYsxEHe34lBBycRJZQxFDm+tmqGF O4NoHOdQJ5WcWfmOWSLtQszZxMM/9/3GEF/Bz84r+t0K92utpWXRjKjtwTdsVlBcyn0PHqeCopUK YsxZNpXDFRJsyIoCp5tCmz1CWo4Dn3Vlvb+ZuTKFcEYLOBRhaLNyhOAsu3sc/5SuflJN3cFprNRC lPOYMfj94d1cNaVrN9XGfk6o0X6leb9nnhK6CwlQB+ttH74NsI6/Dav3zD0HVSieP95SspiQ/01D 9n9N84Lm75onNY9p/qi5uEed15zT3Ke5R3OX5k7Nac2pGnVSc0Jzh+a45nea32qO1Wghxh9CIRRC IZaQ/wKeQdISjTqF+IRIm7JAt7dsIa8DmfT/pvpoGCfk4kQhpyhkU0JeAyiEQnxEF/K8RoQ8oekp 5F6NCLlbk0bIqwCFUIiPsoX8B6AQCvFRtpCXAQqhEB9lC3kJoBAK8RFNiPQeQ0Ie1QwcGNpCfq8J CTlu0S7k3wCFUIiPsoX8C6AQCvFRtpAXAArJWYhyksgoZA4hzwNGCLdqMjApJB8hM+ebRBbyiuY5 zWXN4xoRckmjhTygsQeGZzRpA1Q7rlzFSZRTi10PX7aQKwBrSAZC/glQSPZC1t2or1LIs8DWur2r FPIPYGsDQwqJQ9lCngG2LuRpjQh5RCNCLtTMLURDIXkKuQxQCIX4KFvI0wCFUIiPsoX8DaAQCvFR tpCngK0JkYxMmc0eKOSsJiREAlMUsmkhTwIUQiE+yhbyBEAhFOKjbCGPAxSStRBcWoNCZhPyZ8AM 4erHFcfURYjkDMgAWIQ8rHlIk7WQw6Iz6806WaWQHbvLuuxEObWBRLlVCnkM2Fpe1iqFPAosIaSu mUb9pJCwkHkadeskZQt5BFgoUU5Ji0UhbULmw+486DVrCxKC6/Y+DCwjRFnP8WqIXH8vQiTM8BeN LeTBmvs1IkQu1Okr5ITFcYv2GvIngEIKF3KYmzG/sooWcglYTkiVpFGnkDiULeQhgEIoxEfZQv4A UAiF+Igm5EWNXLInQiTcIL1L+fstIX0XDBAhoQFh18DQvGDnAkAhFOKjbCEPAhRCIT7KFnIeoBAK 8VG2kAcACslbyOqX1hAhsjDCXzWS1GELGbhw2WmLvoGp9oHhOcDOXKyMcLf37pMUMpsQZYdWlfFE IYmE7NhN2DiJck6sm0JmEXIWaBcyG2ULuQ+wU0nXnrm4KSEVJi/ufzukX1NIQiH3AB3dXjVTJiOF BITY5dS0KrkLkXv7ypr29g0lAwsoXwwIkcBUSMjQBLn2geEZoLWw627wHA0KhfQS0uTrUEhSIXcD W5nLWrWQuwAKWZWQ9YzUVy3kToBC1iVkNbO9qxaCI5uttCGva+wb24du5HLBvFIn9YU6+QnpqGtl CzkFzCbEeHI3UwiF5CPkJMA2hEJ8lC3kBDCjkLb+M4XML6Q1w6hsIccBM+vEs5JDtHhIWiFyY/uu BZStC3UkQ86+UCdVgtwwIUrKXxmvxYoYtqktWwjWIbuI7CQHFa2KtA4NyxayYy/DLR8nDSiekFbK FoJbrFTSyhaylpj6NoU07W58BalH6qsWcjtgN+qVrxGP4IdCxghR7rq9sepL6tneVQu5Degs7HlW 8Z0sRG5sby9cZi8YYF2oI0L6XqgzdUAYQcg8eb5lC7kV2EoItxAhs1G2kFuAHm3IHFBIXyEz1aCy hdwMUEgGQm4C+JW1MiERe1lJp9/tBZS7Fgw4b2bIhQJTiRLkjplCjgKzRgzDg0wK6S0kWg1pncgv W8gRoLtRjxcwpJBIQmKHcD2TmGULuRGYUQiaaQ5NIQOF7N4fyYdRQ6yJ/bKF3AAs0cuqVZhCpt4/ ZHVC8P4hg4Qk6mXZQqbWkK4FAwIJcpIh1zdBrktEz8CUVUN+BbhJDnYxxoJChgvx9XDjhUPq76xD 6xG1UV+1kOuBrvh57Dam7u86KatlC9mxl+EuYOYKiTiXlTL7fdVCfgmkyDAJlHnzQCEUkreQXwCd jXrmQkLXp/fMx7q/Iw6SaEB4zMiKCwtJG41K0oZsQsjPgbVnnVBIGsoW8jNgZiFsQ6IIiWAtVbL1 JoT8FOiTBhSjFinjiUJGC8k9+30TQn4CdJf2LK3MaCFyfbp9XUgoH8uKg5yLNCDsK2KqkPyz3zch 5MfA2ru95QmJn+PgPUvRQn4E9Oll5Xyd+iaE/BCgEArxnqZoIT8AemWdxChxdXikkClCotHaOxgt 5BWN5GP1DUxZF4QMHRB2iegYEJqrBKjvA/PlZbVmQFJIXyHxst9bm6KyhXwPMEO47opysZKtU832 blqIJ4EtSfY7hVhCvgu4iXHOhHus1F77uBQiQnbsZfRYUS4WqeIhmxDyHQAcVQcZ+J0VaRxCIUOE YKmZQmKN1FMJeUnzjEbW6Q0FpqwB4b2BAeHQgFRIxO0BjLWY1LcBu1GvnEY9jpCOJGEK8QrxrCiX +/T7JoR8C5gvYphmHFKYkIi017SyhXwTmKuGdHSmKaSfkCpeG8IaEhbyDWC+ABXbkMyEtJ9mrBC5 X8hljQSmZEAo9yu0EuOazLiJAanQANAWcpsfLeTrwFwRw66zUEhfIfNQtpCvAT1qSHIbFYX0F5J7 PGQTQr4KUAiFUMgUIexlzSDkK8Bae1lyv5BnNXLf9L4DQisiFUtEYADYcKvJGCGZ9rI2JeTLwFrb kFKEJMvLopAWIV8CzLLWAcMV5GWVIaQp/fhZJxTSImTHdVfxLGCWLC+LQlqEXAdgXpZHyFxmyhby RcDNOkEhs9WWwUJkwTJZKMAeEEpinAwIrYDUGSsjzhZhCwiJGCigwbi/UVBIY0CZv1JIYiFfADq6 vdkuHFCKEF9ZJSj+/eq9aLpsIZ8HlpjLcgY7FLKskP1/whzglC3kc8BCQpzu9dC7I6xeCN4dYXkh ynouvIZ8FlhEiDvuHCzkisYeEMoCAfaFOQERpztEdA0AuwTc0k4GQuTuCPZXVtFCPgMsIkRJv/fw MoUsJCRA2UI+DVAIhfgoW8inAAqhEB9lC/kksDYhcr9CSYyT+xRKQEoWKNM3ajnbIeJkIAAVS8DN 7VBInkI+AVBIBkI+DlBI1kI8KzlQSHohHwOsvKz6cX4jFOIT4pn0o5A5hOy49ipOopxy7nNOIXMI uRawEuV8S/xlJUQWKpMbtlzS2CLsFclkJYDACDAUeJpa8De1o0v/o4D9lUUh2QnJu1HfpJCPAGYv y123l0KWFLIcZQv5MEAhFOKjbCEfAiiEQnx0CpHEOAlISUKciJBAlIgIDADvsEd+HQO+oQX+m54c rdFCPghQCIX4KFvIBwAKoRAfZQt5P0AhFOKjbCHvAyiEQnwEhbyqeUojA0G5EEcWIpMBoC0gNOIL jPCGFvTRDo60o4W8F6AQCvFRtpD3ABRCIT4oxCskw4hhEULeDeQeU6eQzLJOihCy45qrOAv9ZJgo JwNBWRBZFgAQARJwsgd8XSO8QEkPLNAjN05DC7kGyD1Rrggh7wJy/8qikMwa9SKEvBPIvdtbtpDl KFvIOwAKoRAfZQt5O5CrEAlAycBPBnwy0JMCtws6NHLrKMFf9+SGuFBInkLeBlAIhfgoW8hbAQqh EB9lC3kLQCEU4qNsIW8GchEiJXN9UWQsRE0UoqYVzKS9x++vhbwJoJDpe29AiILH+seihbwRWEQI RosppE2Ich7T+HBCxWULeQNgRQyryvf5nUPInoKE4P1DQkIOhZQ21SEspDg8f7ynqBYQQkxETXoh Cg9PH13MJCR5t2EzzNOok57gFxc/v4QQQgghhJCNM3Hoo+AGomN2nnTblHr/sWfPk4mTA9NmFhT+ H0YfYzsyqjjTZ6oa+xGt95t0dlWNPnuWRBAyoUiXPXuWTC8SZT0P33/Js2fHZCHK+WHwAShEMzVi Jd//4740oPUYfXYJWmxJyIRuL3R4RzSrdYGOvxocwxUbatQJIYQQQgghhBBCyEL0iO9aly0p788k Dn3mkVXwl23M4GaFO5EsSxTXc+96Bl6ZO6hKjZ+BJm00k+MQxVKNJi0Fy70Woqpq9KQ8acNTQ2oB dfZO/YqydzACTyQmISHYtnhqSEUhiXAbdXWoIwdddhvCr6xkON1eCaJjW4KJEofGnt3ebKCHzKAQ QgghhBBCCCGEEEIIIYQQQgghhBBCCCGEEEIIIYQQQgjJiP8Dd6aUXfmn6foAAAAASUVORK5CYII= --_005_012C3117EDDB3C4781FD802A8C27DD4F0D4D8465SACEXCMBX02PRDh_-- From lars@netapp.com Mon Jul 30 11:28:07 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A801F21F86D4 for ; Mon, 30 Jul 2012 11:28:07 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.452 X-Spam-Level: X-Spam-Status: No, score=-10.452 tagged_above=-999 required=5 tests=[AWL=0.147, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id g8yKrZl0qC8E for ; Mon, 30 Jul 2012 11:28:06 -0700 (PDT) Received: from mx2.netapp.com (mx2.netapp.com [216.240.18.37]) by ietfa.amsl.com (Postfix) with ESMTP id B127821F86D3 for ; Mon, 30 Jul 2012 11:28:06 -0700 (PDT) X-IronPort-AV: E=Sophos;i="4.77,681,1336374000"; d="p7s'?scan'208";a="669747057" Received: from smtp2.corp.netapp.com ([10.57.159.114]) by mx2-out.netapp.com with ESMTP; 30 Jul 2012 11:28:06 -0700 Received: from vmwexceht01-prd.hq.netapp.com (vmwexceht01-prd.hq.netapp.com [10.106.76.239]) by smtp2.corp.netapp.com (8.13.1/8.13.1/NTAP-1.6) with ESMTP id q6UIS618028462 for ; Mon, 30 Jul 2012 11:28:06 -0700 (PDT) Received: from SACEXCMBX01-PRD.hq.netapp.com ([169.254.2.13]) by vmwexceht01-prd.hq.netapp.com ([10.106.76.239]) with mapi id 14.02.0298.004; Mon, 30 Jul 2012 11:28:05 -0700 From: "Eggert, Lars" To: "tcpm@ietf.org" Thread-Topic: proposal for the next WG session Thread-Index: AQHNboEP1Py85f0+vkWsN7lVIbOrZg== Date: Mon, 30 Jul 2012 18:28:05 +0000 Message-ID: <4EEB89AE-6450-48DC-9737-973D698665D2@netapp.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: x-originating-ip: [10.104.60.116] Content-Type: multipart/signed; boundary="Apple-Mail=_5862B851-1DA7-4125-B04E-FD11EA92AA6D"; protocol="application/pkcs7-signature"; micalg=sha1 MIME-Version: 1.0 Subject: [tcpm] proposal for the next WG session X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2012 18:28:07 -0000 --Apple-Mail=_5862B851-1DA7-4125-B04E-FD11EA92AA6D Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Hi, I don't like that we have no time to discuss very many of the = presentations today. For the next sessions, can we limit the amount of = presentations and set aside more time for discussion? My proposal would be that no contribution (ID or slide set) gets = presentation time without having seen some minimum discussion on the = list first, to be judged by the chairs. Lars= --Apple-Mail=_5862B851-1DA7-4125-B04E-FD11EA92AA6D Content-Disposition: attachment; filename="smime.p7s" Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIMQDCCBUow ggQyoAMCAQICEFcfSRTG0jNknqb9LV9GuFkwDQYJKoZIhvcNAQEFBQAwgd0xCzAJBgNVBAYTAlVT MRcwFQYDVQQKEw5WZXJpU2lnbiwgSW5jLjEfMB0GA1UECxMWVmVyaVNpZ24gVHJ1c3QgTmV0d29y azE7MDkGA1UECxMyVGVybXMgb2YgdXNlIGF0IGh0dHBzOi8vd3d3LnZlcmlzaWduLmNvbS9ycGEg KGMpMDkxHjAcBgNVBAsTFVBlcnNvbmEgTm90IFZhbGlkYXRlZDE3MDUGA1UEAxMuVmVyaVNpZ24g Q2xhc3MgMSBJbmRpdmlkdWFsIFN1YnNjcmliZXIgQ0EgLSBHMzAeFw0xMTEyMTAwMDAwMDBaFw0x MjEyMDkyMzU5NTlaMIIBDTEXMBUGA1UEChMOVmVyaVNpZ24sIEluYy4xHzAdBgNVBAsTFlZlcmlT aWduIFRydXN0IE5ldHdvcmsxRjBEBgNVBAsTPXd3dy52ZXJpc2lnbi5jb20vcmVwb3NpdG9yeS9S UEEgSW5jb3JwLiBieSBSZWYuLExJQUIuTFREKGMpOTgxHjAcBgNVBAsTFVBlcnNvbmEgTm90IFZh bGlkYXRlZDEzMDEGA1UECxMqRGlnaXRhbCBJRCBDbGFzcyAxIC0gTmV0c2NhcGUgRnVsbCBTZXJ2 aWNlMRQwEgYDVQQDFAtMYXJzIEVnZ2VydDEeMBwGCSqGSIb3DQEJARYPbGFyc0BuZXRhcHAuY29t MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAokrhJTcXt6J/VEpZOicLoguBlYTjXP9v Ze4HuuhXnURUS8YouAfgaqA0zYbt5yd6fh4PBMdAaEWr5yJyHuFykXlrCumjUWSpLuqTS2A+pt4q cZaAQk9iLDN/UVd3SpkUuvWbxXlqzG7/BSqa3VNObBzCmyh+V7aXxri+30CT//DSsNRC4VFy6sn6 dMgSaFenXLwe/FBwY0qTMfICT1PrrX6Sw1S8OfH9rykLlZXbmfkFExxQngp1DJH9xMHeODHGbCv/ ty5gdxMOrLe+vENxFEcy1YQWBZd1kNL4UObugF8A/jE/s+Oa3H1VFH8ghqZTdqGDysVxmtKHuNFx 6jIBSQIDAQABo4HSMIHPMAkGA1UdEwQCMAAwRAYDVR0gBD0wOzA5BgtghkgBhvhFAQcXATAqMCgG CCsGAQUFBwIBFhxodHRwczovL3d3dy52ZXJpc2lnbi5jb20vcnBhMAsGA1UdDwQEAwIFoDAdBgNV HSUEFjAUBggrBgEFBQcDBAYIKwYBBQUHAwIwUAYDVR0fBEkwRzBFoEOgQYY/aHR0cDovL2luZGMx ZGlnaXRhbGlkLWczLWNybC52ZXJpc2lnbi5jb20vSW5kQzFEaWdpdGFsSUQtRzMuY3JsMA0GCSqG SIb3DQEBBQUAA4IBAQBA7q6tR92qpd7xo7VBsrOfGCWzoxIVfTc7t0RhB/Oz/+c3lnhYnNScIuKN JmyZvznmVxqB9BJ72+NkvmdB/hnILSBTRawL2tyLo9PkBtN0nRt4gS6wjpWnD8G83hlJLE7r25jk 7HkRev61dTIXsANFpJKF02C4XSoDfEzNV6MpuEvHvcgHCqMrlwWwfKc7+NoDnE8PBuRzwSXvlD5L mswCY2iiOsd7ImNO4OzTCxETvKTDu92+FTIbRJJpYjVNv1UF7e3w9Kq65BkZJErUH19beUeQl0Wh 2BJQE6/15rQyCnP0iJ/Nmx2/kI6M0PWunEsI6FMs0MbosreaWGHlQmomMIIG7jCCBdagAwIBAgIQ cRVmBUrkkSFN6bxE+azT3DANBgkqhkiG9w0BAQUFADCByjELMAkGA1UEBhMCVVMxFzAVBgNVBAoT DlZlcmlTaWduLCBJbmMuMR8wHQYDVQQLExZWZXJpU2lnbiBUcnVzdCBOZXR3b3JrMTowOAYDVQQL EzEoYykgMTk5OSBWZXJpU2lnbiwgSW5jLiAtIEZvciBhdXRob3JpemVkIHVzZSBvbmx5MUUwQwYD VQQDEzxWZXJpU2lnbiBDbGFzcyAxIFB1YmxpYyBQcmltYXJ5IENlcnRpZmljYXRpb24gQXV0aG9y aXR5IC0gRzMwHhcNMDkwNTAxMDAwMDAwWhcNMTkwNDMwMjM1OTU5WjCB3TELMAkGA1UEBhMCVVMx FzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMR8wHQYDVQQLExZWZXJpU2lnbiBUcnVzdCBOZXR3b3Jr MTswOQYDVQQLEzJUZXJtcyBvZiB1c2UgYXQgaHR0cHM6Ly93d3cudmVyaXNpZ24uY29tL3JwYSAo YykwOTEeMBwGA1UECxMVUGVyc29uYSBOb3QgVmFsaWRhdGVkMTcwNQYDVQQDEy5WZXJpU2lnbiBD bGFzcyAxIEluZGl2aWR1YWwgU3Vic2NyaWJlciBDQSAtIEczMIIBIjANBgkqhkiG9w0BAQEFAAOC AQ8AMIIBCgKCAQEA7cRH3yooHXwGa7vXITLJbBOP6bGNQU4099oL42r6ZYggCxET6ZvgSU6Lb9UB 0F8NR5GKWkx0Pj/GkQm7TDSejW6hglFi92l2WJYHr54UGAdPWr2f0jGyVBlzRmoZQhHsEnMhjfXc MM3l2VYKMcU2bSkUl70t2olHGYjYSwQ967Y8Zx50ABMN0Ibak2f4MwOuGjxraXj2wCyO4YM/d/mZ //6fUlrCtIcK2GypR8FUKWVDPkrAlh/Brfd3r2yxBF6+wbaULZeQLSfSux7pg2qE9sSyriMGZSal J1grByK0b6ZiSBp38tVQJ5op05b7KPW6JHZi44xZ6/tu1ULEvkHH9QIDAQABo4ICuTCCArUwNAYI KwYBBQUHAQEEKDAmMCQGCCsGAQUFBzABhhhodHRwOi8vb2NzcC52ZXJpc2lnbi5jb20wEgYDVR0T AQH/BAgwBgEB/wIBADBwBgNVHSAEaTBnMGUGC2CGSAGG+EUBBxcBMFYwKAYIKwYBBQUHAgEWHGh0 dHBzOi8vd3d3LnZlcmlzaWduLmNvbS9jcHMwKgYIKwYBBQUHAgIwHhocaHR0cHM6Ly93d3cudmVy aXNpZ24uY29tL3JwYTA0BgNVHR8ELTArMCmgJ6AlhiNodHRwOi8vY3JsLnZlcmlzaWduLmNvbS9w Y2ExLWczLmNybDAOBgNVHQ8BAf8EBAMCAQYwbgYIKwYBBQUHAQwEYjBgoV6gXDBaMFgwVhYJaW1h Z2UvZ2lmMCEwHzAHBgUrDgMCGgQUS2u5KJYGDLvQUjibKaxLB4shBRgwJhYkaHR0cDovL2xvZ28u dmVyaXNpZ24uY29tL3ZzbG9nbzEuZ2lmMC4GA1UdEQQnMCWkIzAhMR8wHQYDVQQDExZQcml2YXRl TGFiZWw0LTIwNDgtMTE4MB0GA1UdDgQWBBR5R2EIQf04BKJL57XM9UP2SSsR+DCB8QYDVR0jBIHp MIHmoYHQpIHNMIHKMQswCQYDVQQGEwJVUzEXMBUGA1UEChMOVmVyaVNpZ24sIEluYy4xHzAdBgNV BAsTFlZlcmlTaWduIFRydXN0IE5ldHdvcmsxOjA4BgNVBAsTMShjKSAxOTk5IFZlcmlTaWduLCBJ bmMuIC0gRm9yIGF1dGhvcml6ZWQgdXNlIG9ubHkxRTBDBgNVBAMTPFZlcmlTaWduIENsYXNzIDEg UHVibGljIFByaW1hcnkgQ2VydGlmaWNhdGlvbiBBdXRob3JpdHkgLSBHM4IRAItbdVaEVIULAM+v OEjOsaQwDQYJKoZIhvcNAQEFBQADggEBADlNz0GZgbWpBbVSOOk5hIls5DSoWufYbAlMJBq6WaSH O3Mh8ZOBz79oY1pn/jWFK6HDXaNKwjoZ3TDWzE3v8dKBl8pUWkO/N4t6jhmND0OojPKvYLMVirOV nDzgnrMnmKQ1chfl/Cpdh9OKDcLRRSr4wPSsKpM61a4ScAjr+zvid+zoK2Q1ds262uDRyxTWcVib vtU+fbbZ6CTFJGZMXZEfdrMXPn8NxiGJL7M3uKH/XLJtSd5lUkL7DojS7Uodv0vj+Mxy+kgOZY5J yNb4mZg7t5Q+MXEGh/psWVMu198r7V9jAKwV7QO4VRaMxmgD5yKocwuxvKDaUljdCg5/wYIxggSL MIIEhwIBATCB8jCB3TELMAkGA1UEBhMCVVMxFzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMR8wHQYD VQQLExZWZXJpU2lnbiBUcnVzdCBOZXR3b3JrMTswOQYDVQQLEzJUZXJtcyBvZiB1c2UgYXQgaHR0 cHM6Ly93d3cudmVyaXNpZ24uY29tL3JwYSAoYykwOTEeMBwGA1UECxMVUGVyc29uYSBOb3QgVmFs aWRhdGVkMTcwNQYDVQQDEy5WZXJpU2lnbiBDbGFzcyAxIEluZGl2aWR1YWwgU3Vic2NyaWJlciBD QSAtIEczAhBXH0kUxtIzZJ6m/S1fRrhZMAkGBSsOAwIaBQCgggJtMBgGCSqGSIb3DQEJAzELBgkq hkiG9w0BBwEwHAYJKoZIhvcNAQkFMQ8XDTEyMDczMDE4MjgwNFowIwYJKoZIhvcNAQkEMRYEFGQk BGpojq5CoPjWytBih2XSqVM4MIIBAwYJKwYBBAGCNxAEMYH1MIHyMIHdMQswCQYDVQQGEwJVUzEX MBUGA1UEChMOVmVyaVNpZ24sIEluYy4xHzAdBgNVBAsTFlZlcmlTaWduIFRydXN0IE5ldHdvcmsx OzA5BgNVBAsTMlRlcm1zIG9mIHVzZSBhdCBodHRwczovL3d3dy52ZXJpc2lnbi5jb20vcnBhIChj KTA5MR4wHAYDVQQLExVQZXJzb25hIE5vdCBWYWxpZGF0ZWQxNzA1BgNVBAMTLlZlcmlTaWduIENs YXNzIDEgSW5kaXZpZHVhbCBTdWJzY3JpYmVyIENBIC0gRzMCEFcfSRTG0jNknqb9LV9GuFkwggEF BgsqhkiG9w0BCRACCzGB9aCB8jCB3TELMAkGA1UEBhMCVVMxFzAVBgNVBAoTDlZlcmlTaWduLCBJ bmMuMR8wHQYDVQQLExZWZXJpU2lnbiBUcnVzdCBOZXR3b3JrMTswOQYDVQQLEzJUZXJtcyBvZiB1 c2UgYXQgaHR0cHM6Ly93d3cudmVyaXNpZ24uY29tL3JwYSAoYykwOTEeMBwGA1UECxMVUGVyc29u YSBOb3QgVmFsaWRhdGVkMTcwNQYDVQQDEy5WZXJpU2lnbiBDbGFzcyAxIEluZGl2aWR1YWwgU3Vi c2NyaWJlciBDQSAtIEczAhBXH0kUxtIzZJ6m/S1fRrhZMA0GCSqGSIb3DQEBAQUABIIBADtUz53N Bi8u45wyIn65C8L879pB49WJ0vR2awxjRRwDr9WF9SOTKbaYUMJjmJ5SBxk2oBlgFjb/Lnq9NKXV at/EskYC0+1swFQl9vjHT0TqEIDA+VTu7donyCrrpkfUrH3wQFrxM5RJStCxVibdWrRHZpckrTiy 8MHyRQPeNZtJvUDo/rYWBpBMWZCUsmWo745A4jPwLIDaDL3nq7EE5bc5xuS87bnBiGc1JvCTQRzF c7dbl/MXT4TRMuLm5/3qIE9lIHV5cEPaTCCa11lHaKrsP/eldnp2TWYLvaevU7M1B0iW0FQFlk38 YDcCjA8dcvQoT20WNToh8W5/BMemlx0AAAAAAAA= --Apple-Mail=_5862B851-1DA7-4125-B04E-FD11EA92AA6D-- From cabo@tzi.org Mon Jul 30 11:34:41 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B465B11E808E for ; Mon, 30 Jul 2012 11:34:41 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.249 X-Spam-Level: X-Spam-Status: No, score=-106.249 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vnsKq-r6-dNd for ; Mon, 30 Jul 2012 11:34:41 -0700 (PDT) Received: from informatik.uni-bremen.de (mailhost.informatik.uni-bremen.de [IPv6:2001:638:708:30c9::12]) by ietfa.amsl.com (Postfix) with ESMTP id 9B4D611E8087 for ; Mon, 30 Jul 2012 11:34:40 -0700 (PDT) X-Virus-Scanned: amavisd-new at informatik.uni-bremen.de Received: from smtp-fb3.informatik.uni-bremen.de (smtp-fb3.informatik.uni-bremen.de [134.102.224.120]) by informatik.uni-bremen.de (8.14.3/8.14.3) with ESMTP id q6UIYX8R010924; Mon, 30 Jul 2012 20:34:33 +0200 (CEST) Received: from dhcp-9069.meeting.ietf.org (dhcp-9069.meeting.ietf.org [130.129.8.105]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by smtp-fb3.informatik.uni-bremen.de (Postfix) with ESMTPSA id 579EA232; Mon, 30 Jul 2012 20:34:32 +0200 (CEST) Mime-Version: 1.0 (Mac OS X Mail 6.0 \(1485\)) Content-Type: text/plain; charset=iso-8859-1 From: Carsten Bormann In-Reply-To: <4EEB89AE-6450-48DC-9737-973D698665D2@netapp.com> Date: Mon, 30 Jul 2012 11:34:29 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: <47336CE2-1CD2-48CE-99A2-84726F7B3400@tzi.org> References: <4EEB89AE-6450-48DC-9737-973D698665D2@netapp.com> To: "Eggert, Lars" X-Mailer: Apple Mail (2.1485) Cc: "tcpm@ietf.org" Subject: Re: [tcpm] proposal for the next WG session X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2012 18:34:41 -0000 > I don't like that we have no time to discuss very many of the = presentations today. For the next sessions, can we limit the amount of = presentations and set aside more time for discussion? >=20 > My proposal would be that no contribution (ID or slide set) gets = presentation time without having seen some minimum discussion on the = list first, to be judged by the chairs. Yes, please. There is no point in having a WG meeting without progress. = Progress is generated by discussion. E.g., if we had had 3 more minutes, we could have communicated the = resounding "NO" on HOST_ID in a more decisive way. Now this will drag out for months in a couple of mailing lists. =20 Avoiding this is exactly the reason to have f2f meetings in the first = place! Gr=FC=DFe, Carsten From michael.scharf@alcatel-lucent.com Mon Jul 30 13:04:51 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id BC77721F8513 for ; Mon, 30 Jul 2012 13:04:51 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -8.082 X-Spam-Level: X-Spam-Status: No, score=-8.082 tagged_above=-999 required=5 tests=[AWL=-1.833, BAYES_00=-2.599, HELO_EQ_FR=0.35, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Q7NfZxHiplax for ; Mon, 30 Jul 2012 13:04:51 -0700 (PDT) Received: from smail6.alcatel.fr (smail6.alcatel.fr [62.23.212.42]) by ietfa.amsl.com (Postfix) with ESMTP id CEDC421F850B for ; Mon, 30 Jul 2012 13:04:50 -0700 (PDT) Received: from FRMRSSXCHHUB01.dc-m.alcatel-lucent.com (FRMRSSXCHHUB01.dc-m.alcatel-lucent.com [135.120.45.61]) by smail6.alcatel.fr (8.14.3/8.14.3/ICT) with ESMTP id q6UK4laq022441 (version=TLSv1/SSLv3 cipher=RC4-MD5 bits=128 verify=NOT); Mon, 30 Jul 2012 22:04:48 +0200 Received: from FRMRSSXCHMBSE3.dc-m.alcatel-lucent.com ([135.120.45.55]) by FRMRSSXCHHUB01.dc-m.alcatel-lucent.com ([135.120.45.61]) with mapi; Mon, 30 Jul 2012 22:04:47 +0200 From: "Scharf, Michael (Michael)" To: Carsten Bormann , "Eggert, Lars" Date: Mon, 30 Jul 2012 22:04:45 +0200 Thread-Topic: [tcpm] proposal for the next WG session Thread-Index: Ac1ugir3v9RzQrM8QRGd1tKr9w9ioQAC2E1Q Message-ID: <2A886F9088894347A3BE0CC5B7A85F3E891A9BBCFD@FRMRSSXCHMBSE3.dc-m.alcatel-lucent.com> References: <4EEB89AE-6450-48DC-9737-973D698665D2@netapp.com> <47336CE2-1CD2-48CE-99A2-84726F7B3400@tzi.org> In-Reply-To: <47336CE2-1CD2-48CE-99A2-84726F7B3400@tzi.org> Accept-Language: de-DE, en-US Content-Language: de-DE X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: de-DE, en-US Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.69 on 155.132.188.84 Cc: "tcpm@ietf.org" Subject: Re: [tcpm] proposal for the next WG session X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2012 20:04:51 -0000 TCPM has been open in the past for short heads-up presentations regarding w= ork that is clearly outside the scope of TCPM (another recent example was t= cpcrypt), to make people aware of such work, if they don't read all drafts = that have "TCP" in the title... This practice can be changed, of course.=20 Michael > -----Original Message----- > From: tcpm-bounces@ietf.org [mailto:tcpm-bounces@ietf.org] On=20 > Behalf Of Carsten Bormann > Sent: Monday, July 30, 2012 8:34 PM > To: Eggert, Lars > Cc: tcpm@ietf.org > Subject: Re: [tcpm] proposal for the next WG session >=20 > > I don't like that we have no time to discuss very many of=20 > the presentations today. For the next sessions, can we limit=20 > the amount of presentations and set aside more time for discussion? > >=20 > > My proposal would be that no contribution (ID or slide set)=20 > gets presentation time without having seen some minimum=20 > discussion on the list first, to be judged by the chairs. >=20 > Yes, please. There is no point in having a WG meeting=20 > without progress. Progress is generated by discussion. >=20 > E.g., if we had had 3 more minutes, we could have=20 > communicated the resounding "NO" on HOST_ID in a more decisive way. > Now this will drag out for months in a couple of mailing lists. =20 > Avoiding this is exactly the reason to have f2f meetings in=20 > the first place! >=20 > Gr=FC=DFe, Carsten >=20 > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm > = From rs@netapp.com Mon Jul 30 13:32:44 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7692411E8179 for ; Mon, 30 Jul 2012 13:32:44 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.121 X-Spam-Level: X-Spam-Status: No, score=-10.121 tagged_above=-999 required=5 tests=[AWL=0.477, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id oJiU43Up3sfx for ; Mon, 30 Jul 2012 13:32:43 -0700 (PDT) Received: from mx2.netapp.com (mx2.netapp.com [216.240.18.37]) by ietfa.amsl.com (Postfix) with ESMTP id 7A4E911E816F for ; Mon, 30 Jul 2012 13:32:43 -0700 (PDT) X-IronPort-AV: E=Sophos;i="4.77,681,1336374000"; d="scan'208,217";a="669782934" Received: from smtp1.corp.netapp.com ([10.57.156.124]) by mx2-out.netapp.com with ESMTP; 30 Jul 2012 13:32:43 -0700 Received: from vmwexceht03-prd.hq.netapp.com (vmwexceht03-prd.hq.netapp.com [10.106.76.241]) by smtp1.corp.netapp.com (8.13.1/8.13.1/NTAP-1.6) with ESMTP id q6UKWhCQ010868 for ; Mon, 30 Jul 2012 13:32:43 -0700 (PDT) Received: from SACEXCMBX02-PRD.hq.netapp.com ([169.254.1.34]) by vmwexceht03-prd.hq.netapp.com ([10.106.76.241]) with mapi id 14.02.0298.004; Mon, 30 Jul 2012 13:32:43 -0700 From: "Scheffenegger, Richard" To: "tcpm (tcpm@ietf.org)" Thread-Topic: Accurate ECN side-meeting Thread-Index: Ac1ukMIzB6G1AClqT3iwdJIUpoFdAg== Date: Mon, 30 Jul 2012 20:32:41 +0000 Message-ID: <012C3117EDDB3C4781FD802A8C27DD4F0D4DA610@SACEXCMBX02-PRD.hq.netapp.com> Accept-Language: de-AT, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.104.60.116] Content-Type: multipart/alternative; boundary="_000_012C3117EDDB3C4781FD802A8C27DD4F0D4DA610SACEXCMBX02PRDh_" MIME-Version: 1.0 Subject: [tcpm] Accurate ECN side-meeting X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2012 20:32:44 -0000 --_000_012C3117EDDB3C4781FD802A8C27DD4F0D4DA610SACEXCMBX02PRDh_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi, Since there was virtually no time for discussion today in the TCPM meeting,= who would be interested in (and have time for) a side meeting to discuss f= urther on the Accurate ECN topic? For me, it became apparent that the problems and requirements statement are= not too well understood so far (and I'm not excluding myself). Currently, we have feedback with perfect Reliability (getting one signal pe= r RTT, with a simple handshake mechanism). Ack loss (or delayed ACKs with more than than 2 data packets) presents anot= her problem,... I think Matt has a very valid point that the solution space needs more disc= ussions, ideally from more participants than only between the authors! Richard Scheffenegger --_000_012C3117EDDB3C4781FD802A8C27DD4F0D4DA610SACEXCMBX02PRDh_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable
Hi,
 
Since there was virtually no time for discussion today in the TCPM mee= ting, who would be interested in (and have time for) a side meeting to disc= uss further on the Accurate ECN topic?
 
 
For me, it became apparent that the problems and requirements statemen= t are not too well understood so far (and I’m not excluding myself).<= /div>
 
Currently, we have feedback with perfect Reliability (getting one sign= al per RTT, with a simple handshake mechanism).
 
Ack loss (or delayed ACKs with more than than 2 data packets) presents= another problem,…
 
 
 
I think Matt has a very valid point that the solution space needs more= discussions, ideally from more participants than only between the authors!=
 
 
 
Richard Scheffenegger
 
 --_000_012C3117EDDB3C4781FD802A8C27DD4F0D4DA610SACEXCMBX02PRDh_-- From bob.briscoe@bt.com Mon Jul 30 13:49:31 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 008B711E818E for ; Mon, 30 Jul 2012 13:49:31 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.998 X-Spam-Level: X-Spam-Status: No, score=-2.998 tagged_above=-999 required=5 tests=[AWL=-0.600, BAYES_00=-2.599, HTML_MESSAGE=0.001, J_CHICKENPOX_34=0.6, J_CHICKENPOX_64=0.6, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5Ji67+jxM8Pj for ; Mon, 30 Jul 2012 13:49:30 -0700 (PDT) Received: from smtpe1.intersmtp.com (smtp64.intersmtp.com [62.239.224.237]) by ietfa.amsl.com (Postfix) with ESMTP id 9F3C811E8155 for ; Mon, 30 Jul 2012 13:49:29 -0700 (PDT) Received: from EVHUB71-UKRD.domain1.systemhost.net (10.36.3.154) by RDW083A008ED64.smtp-e4.hygiene.service (10.187.98.13) with Microsoft SMTP Server (TLS) id 8.3.264.0; Mon, 30 Jul 2012 21:49:28 +0100 Received: from EVMHT03-UKBR.domain1.systemhost.net (193.113.108.56) by EVHUB71-UKRD.domain1.systemhost.net (10.36.3.154) with Microsoft SMTP Server (TLS) id 14.2.298.4; Mon, 30 Jul 2012 21:49:27 +0100 Received: from EMV04-UKBR.domain1.systemhost.net ([169.254.1.122]) by EVMHT03-UKBR.domain1.systemhost.net ([193.113.108.56]) with mapi; Mon, 30 Jul 2012 21:49:27 +0100 From: To: Date: Mon, 30 Jul 2012 21:49:26 +0100 Thread-Topic: Some ideas building on draft-ietf-tcpm-fastopen-01 Thread-Index: AQHNbpTOd9k1XdzVuk606ETC/srmOg== Message-ID: <69ACA3A4BA39B0499C1917FB0718BB254BE747BBA2@EMV04-UKBR.domain1.systemhost.net> Accept-Language: en-US, en-GB Content-Language: en-GB X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-GB Content-Type: multipart/alternative; boundary="_000_69ACA3A4BA39B0499C1917FB0718BB254BE747BBA2EMV04UKBRdoma_" MIME-Version: 1.0 Cc: tcpm@ietf.org, draft-ietf-tcpm-fastopen@tools.ietf.org Subject: Re: [tcpm] Some ideas building on draft-ietf-tcpm-fastopen-01 X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2012 20:49:31 -0000 --_000_69ACA3A4BA39B0499C1917FB0718BB254BE747BBA2EMV04UKBRdoma_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Jerry, Re-instating tcpm list on distr. Inline... At 19:41 30/07/2012, Jerry Chu wrote: Hi Bob, Thanks for your interest and sorry for the delay (Yuchung has been traveling and I've been busy preparing TFO's server side implementation for publication). On Thu, Jul 26, 2012 at 9:35 PM, Bob Briscoe wrote: > Yuchung, Jerry, Sivasankar, Arvind, > > I liked this enough to want to work on improving it. I have a couple of > proposals that occurred to me, which I've finally made time to write up. > > (I'll send editorial nits from reviewing the draft later) Sure. > > =3D=3DProposal #1/ =3D=3D > Fast Open Cookie Request MUST (SHOULD?) be on a FIN > > I believe it would be better for the reasons below if: > * the client put a fast open cookie request in its FIN, not its SYN (whet= her > or not the server has sent a FIN first). > * Then the server can send the fastopen cookie on the FIN-ACK. > * If the client times out waiting for a FIN-ACK, it just re-sends as usua= l. > * If the server's FIN-ACK never arrives (perhaps because a middlebox has > eaten it), the client just doesn't have a cookie for the next session and > uses a 3WHS as normal. Very interesting idea. > > Note, only the Request would have to be on a FIN. The Fast Open Cookie > itself would obviously still be on the SYN of subsequent connections. The= n, > when a subsequent connection finishes, it can refresh the cookie on its F= IN, > ready for another connection later. > > This solves a couple of potential problems with a fastopen request-respon= se > on the initial SYN exchange: > a) A FIN-cookie sidesteps the risk of unpredictable delay of the very fir= st > SYN due to some middleboxes dropping new TCP options (any timeouts due to > non-delivery don't hold up the session, which has already finished) This is a good point but from our experience unknown TCP options don't seem to pose a real problem wrt middleboxes, SYN pkts w/ data do. This is only half the reason for doing it (ie the two reasons together stre= gthen the case). Also if there is middlebox on the route that will drop SYN pkts with unrecognized TCP options, requesting TFO cookie in FIN pkt only postpones the delay for those servers that support TFO, i.e., to the subsequent connections with TF= O cookie+data. My point is that sending the risky packet after you've sent all the data do= esn't delay anything the user cares about - you just retransmit the FIN to = close the connection - no big deal for anyone. But I agree if there is still non-negligible % of middleboxes dropping The refs say there might be problems (today) with 6% of paths. In future it= might be more. Might be less. That's not negligible. SYN w/ unrecognized TCP option then requesting TFO cookie through FIN can avoid the delay for those cases where the server doesn't support TFO. Getting really bad delay variance on a proposal to cut delay isn't good. > b) Transports GETting typical Web pages could (ab)use the fastopen cookie= on > the SYN exchange by opening multiple TCP connections during the initial > connection without each one suffering the delay of a 3WHS. This creates a > perverse incentive to use multiple parallel connections, because you can = get > the same latency benefit as HTTP pipelining [Yoshifumi posed this problem > some time ago]. If clients can only request a fastopen cookie on a FIN, > they don't have this perverse incentive to not bother with pipelining. Interesting point. A motivated abuser may rush to FIN to get a cookie first= . Client can't send a FIN without closing the connection. That's the dilemma = I'm proposing it has to face. I'm not talking about an attack. I'm talking about lazy Web developers. Thi= s removes the benefit that fast-open gives to those who shard connections. = If they want to avoid 3WHS for the /current/ set of objects they are trying= to get, they have to use pipelining or SPDY. Cookie on FIN deliberately defers the benefit of no 3WHS to subsequent conn= ections. > > I have no evidence, but it seems intuitive that if the TCP option got > through on the first FIN, it might be more likely to get through on the n= ext > SYN, altho I admit options on SYNs are looked at more closely by > middleboxes. > I won't worry about the difference here because the real test for the middl= ebox is SYN+data I think. Again do you have data showing middlebox still having trouble with unrecognized TCP options? 6% of paths - From memory this was Michio's work in the Trilogy project tha= t you cite in the draft. Will comment on your following proposals later. Understood Bob Jerry > --_000_69ACA3A4BA39B0499C1917FB0718BB254BE747BBA2EMV04UKBRdoma_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
Jerry,<= br>
Re-instating tcpm list on distr. Inline...

At 19:41 30/07/2012, Jerry Chu wrote:
Hi Bob,

Thanks for your interest and sorry for the delay (Yuchung has been
traveling and I've
been busy preparing TFO's server side implementation for publication).

On Thu, Jul 26, 2012 at 9:35 PM, Bob Briscoe <bob.briscoe@bt.com> wro= te:
> Yuchung, Jerry, Sivasankar, Arvind,
>
> I liked this enough to want to work on improving it. I have a couple o= f
> proposals that occurred to me, which I've finally made time to write u= p.
>
> (I'll send editorial nits from reviewing the draft later)

Sure.

>
> =3D=3DProposal #1/ =3D=3D
> Fast Open Cookie Request MUST (SHOULD?) be on a FIN
>
> I believe it would be better for the reasons below if:
> * the client put a fast open cookie request in its FIN, not its SYN (w= hether
> or not the server has sent a FIN first).
> * Then the server can send the fastopen cookie on the FIN-ACK.
> * If the client times out waiting for a FIN-ACK, it just re-sends as u= sual.
> * If the server's FIN-ACK never arrives (perhaps because a middlebox h= as
> eaten it), the client just doesn't have a cookie for the next session = and
> uses a 3WHS as normal.

Very interesting idea.

>
> Note, only the Request would have to be on a FIN. The Fast Open Cookie=
> itself would obviously still be on the SYN of subsequent connections. = Then,
> when a subsequent connection finishes, it can refresh the cookie on it= s FIN,
> ready for another connection later.
>
> This solves a couple of potential problems with a fastopen request-res= ponse
> on the initial SYN exchange:
> a) A FIN-cookie sidesteps the risk of unpredictable delay of the very = first
> SYN due to some middleboxes dropping new TCP options (any timeouts due= to
> non-delivery don't hold up the session, which has already finished)
This is a good point but from our experience unknown TCP options don't seem=
to pose a real problem wrt middleboxes, SYN pkts w/ data do.

This is only half the reason for doing it (ie the two reasons together stre= gthen the case).


Also if there is middleb= ox on the route that will drop SYN pkts with
unrecognized
TCP options, requesting TFO cookie in FIN pkt only postpones the delay for<= br> those servers that support TFO, i.e., to the subsequent connections with TF= O
cookie+data.

My point is that sending the risky packet after you've sent all the data do= esn't delay anything the user cares about - you just retransmit the FIN to = close the connection - no big deal for anyone.

But I agree if there is = still non-negligible % of
middleboxes dropping

The refs say there might be problems (today) with 6% of paths. In future it= might be more. Might be less. That's not negligible.

SYN w/ unrecognized TCP = option then requesting TFO cookie through FIN can
avoid the delay for those cases where the server doesn't support TFO.
Getting really bad delay variance on a proposal to cut delay isn't good.
> b) Transports GETti= ng typical Web pages could (ab)use the fastopen cookie on
> the SYN exchange by opening multiple TCP connections during the initia= l
> connection without each one suffering the delay of a 3WHS. This create= s a
> perverse incentive to use multiple parallel connections, because you c= an get
> the same latency benefit as HTTP pipelining [Yoshifumi posed this prob= lem
> some time ago].  If clients can only request a fastopen cookie on= a FIN,
> they don't have this perverse incentive to not bother with pipelining.=

Interesting point. A motivated abuser may rush to FIN to get a cookie first= .

Client can't send a FIN without closing the connection. That's the dilemma = I'm proposing it has to face.

I'm not talking about an attack. I'm talking about lazy Web developers. Thi= s removes the benefit that fast-open gives to those who shard connections. = If they want to avoid 3WHS for the /current/ set of objects they are trying= to get, they have to use pipelining or SPDY.

Cookie on FIN deliberately defers the benefit of no 3WHS to subsequent conn= ections.


>
> I have no evidence, but it seems intuitive that if the TCP option got<= br> > through on the first FIN, it might be more likely to get through on th= e next
> SYN, altho I admit options on SYNs are looked at more closely by
> middleboxes.
>

I won't worry about the difference here because the real test for the middl= ebox
is SYN+data I think. Again do you have data showing middlebox still
having trouble
with unrecognized TCP options?

6% of paths - From memory this was Michio's work in the Trilogy project tha= t you cite in the draft.


Will comment on your fol= lowing proposals later.

Understood



Bob


Jerry

>
--_000_69ACA3A4BA39B0499C1917FB0718BB254BE747BBA2EMV04UKBRdoma_-- From lars@netapp.com Mon Jul 30 14:12:12 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D337821F8559 for ; Mon, 30 Jul 2012 14:12:11 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.455 X-Spam-Level: X-Spam-Status: No, score=-10.455 tagged_above=-999 required=5 tests=[AWL=0.144, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KCeRRfLharQc for ; Mon, 30 Jul 2012 14:12:11 -0700 (PDT) Received: from mx2.netapp.com (mx2.netapp.com [216.240.18.37]) by ietfa.amsl.com (Postfix) with ESMTP id D17E521F850B for ; Mon, 30 Jul 2012 14:12:08 -0700 (PDT) X-IronPort-AV: E=Sophos;i="4.77,681,1336374000"; d="p7s'?scan'208";a="669794026" Received: from smtp2.corp.netapp.com ([10.57.159.114]) by mx2-out.netapp.com with ESMTP; 30 Jul 2012 14:12:08 -0700 Received: from vmwexceht04-prd.hq.netapp.com (vmwexceht04-prd.hq.netapp.com [10.106.77.34]) by smtp2.corp.netapp.com (8.13.1/8.13.1/NTAP-1.6) with ESMTP id q6ULC8Xt019322; Mon, 30 Jul 2012 14:12:08 -0700 (PDT) Received: from SACEXCMBX01-PRD.hq.netapp.com ([169.254.2.13]) by vmwexceht04-prd.hq.netapp.com ([10.106.77.34]) with mapi id 14.02.0298.004; Mon, 30 Jul 2012 14:12:07 -0700 From: "Eggert, Lars" To: "Scharf, Michael (Michael)" Thread-Topic: [tcpm] proposal for the next WG session Thread-Index: AQHNboEP1Py85f0+vkWsN7lVIbOrZpdCnD6AgAAZOYCAABLRAA== Date: Mon, 30 Jul 2012 21:12:07 +0000 Message-ID: <9FDF3D2D-2AE5-46DE-BA6E-2A98CACC7521@netapp.com> References: <4EEB89AE-6450-48DC-9737-973D698665D2@netapp.com> <47336CE2-1CD2-48CE-99A2-84726F7B3400@tzi.org> <2A886F9088894347A3BE0CC5B7A85F3E891A9BBCFD@FRMRSSXCHMBSE3.dc-m.alcatel-lucent.com> In-Reply-To: <2A886F9088894347A3BE0CC5B7A85F3E891A9BBCFD@FRMRSSXCHMBSE3.dc-m.alcatel-lucent.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: x-originating-ip: [10.104.60.116] Content-Type: multipart/signed; boundary="Apple-Mail=_D54FFF0E-3581-4C73-9A72-5B9C59430768"; protocol="application/pkcs7-signature"; micalg=sha1 MIME-Version: 1.0 Cc: "tcpm@ietf.org" Subject: Re: [tcpm] proposal for the next WG session X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2012 21:12:13 -0000 --Apple-Mail=_D54FFF0E-3581-4C73-9A72-5B9C59430768 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=iso-8859-1 On Jul 30, 2012, at 13:04, "Scharf, Michael (Michael)" = wrote: > TCPM has been open in the past for short heads-up presentations = regarding work that is clearly outside the scope of TCPM (another recent = example was tcpcrypt), to make people aware of such work, if they don't = read all drafts that have "TCP" in the title... >=20 > This practice can be changed, of course.=20 I don't want to change that. IFF we have the time. Heads-ups should go to the list, and be session-wise be scheduled in the = "if time permits" category. We had to cut so many important discussions short today that I'm grumpy. Lars= --Apple-Mail=_D54FFF0E-3581-4C73-9A72-5B9C59430768 Content-Disposition: attachment; filename="smime.p7s" Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIMQDCCBUow ggQyoAMCAQICEFcfSRTG0jNknqb9LV9GuFkwDQYJKoZIhvcNAQEFBQAwgd0xCzAJBgNVBAYTAlVT MRcwFQYDVQQKEw5WZXJpU2lnbiwgSW5jLjEfMB0GA1UECxMWVmVyaVNpZ24gVHJ1c3QgTmV0d29y azE7MDkGA1UECxMyVGVybXMgb2YgdXNlIGF0IGh0dHBzOi8vd3d3LnZlcmlzaWduLmNvbS9ycGEg KGMpMDkxHjAcBgNVBAsTFVBlcnNvbmEgTm90IFZhbGlkYXRlZDE3MDUGA1UEAxMuVmVyaVNpZ24g Q2xhc3MgMSBJbmRpdmlkdWFsIFN1YnNjcmliZXIgQ0EgLSBHMzAeFw0xMTEyMTAwMDAwMDBaFw0x MjEyMDkyMzU5NTlaMIIBDTEXMBUGA1UEChMOVmVyaVNpZ24sIEluYy4xHzAdBgNVBAsTFlZlcmlT aWduIFRydXN0IE5ldHdvcmsxRjBEBgNVBAsTPXd3dy52ZXJpc2lnbi5jb20vcmVwb3NpdG9yeS9S UEEgSW5jb3JwLiBieSBSZWYuLExJQUIuTFREKGMpOTgxHjAcBgNVBAsTFVBlcnNvbmEgTm90IFZh bGlkYXRlZDEzMDEGA1UECxMqRGlnaXRhbCBJRCBDbGFzcyAxIC0gTmV0c2NhcGUgRnVsbCBTZXJ2 aWNlMRQwEgYDVQQDFAtMYXJzIEVnZ2VydDEeMBwGCSqGSIb3DQEJARYPbGFyc0BuZXRhcHAuY29t MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAokrhJTcXt6J/VEpZOicLoguBlYTjXP9v Ze4HuuhXnURUS8YouAfgaqA0zYbt5yd6fh4PBMdAaEWr5yJyHuFykXlrCumjUWSpLuqTS2A+pt4q cZaAQk9iLDN/UVd3SpkUuvWbxXlqzG7/BSqa3VNObBzCmyh+V7aXxri+30CT//DSsNRC4VFy6sn6 dMgSaFenXLwe/FBwY0qTMfICT1PrrX6Sw1S8OfH9rykLlZXbmfkFExxQngp1DJH9xMHeODHGbCv/ ty5gdxMOrLe+vENxFEcy1YQWBZd1kNL4UObugF8A/jE/s+Oa3H1VFH8ghqZTdqGDysVxmtKHuNFx 6jIBSQIDAQABo4HSMIHPMAkGA1UdEwQCMAAwRAYDVR0gBD0wOzA5BgtghkgBhvhFAQcXATAqMCgG CCsGAQUFBwIBFhxodHRwczovL3d3dy52ZXJpc2lnbi5jb20vcnBhMAsGA1UdDwQEAwIFoDAdBgNV HSUEFjAUBggrBgEFBQcDBAYIKwYBBQUHAwIwUAYDVR0fBEkwRzBFoEOgQYY/aHR0cDovL2luZGMx ZGlnaXRhbGlkLWczLWNybC52ZXJpc2lnbi5jb20vSW5kQzFEaWdpdGFsSUQtRzMuY3JsMA0GCSqG SIb3DQEBBQUAA4IBAQBA7q6tR92qpd7xo7VBsrOfGCWzoxIVfTc7t0RhB/Oz/+c3lnhYnNScIuKN JmyZvznmVxqB9BJ72+NkvmdB/hnILSBTRawL2tyLo9PkBtN0nRt4gS6wjpWnD8G83hlJLE7r25jk 7HkRev61dTIXsANFpJKF02C4XSoDfEzNV6MpuEvHvcgHCqMrlwWwfKc7+NoDnE8PBuRzwSXvlD5L mswCY2iiOsd7ImNO4OzTCxETvKTDu92+FTIbRJJpYjVNv1UF7e3w9Kq65BkZJErUH19beUeQl0Wh 2BJQE6/15rQyCnP0iJ/Nmx2/kI6M0PWunEsI6FMs0MbosreaWGHlQmomMIIG7jCCBdagAwIBAgIQ cRVmBUrkkSFN6bxE+azT3DANBgkqhkiG9w0BAQUFADCByjELMAkGA1UEBhMCVVMxFzAVBgNVBAoT DlZlcmlTaWduLCBJbmMuMR8wHQYDVQQLExZWZXJpU2lnbiBUcnVzdCBOZXR3b3JrMTowOAYDVQQL EzEoYykgMTk5OSBWZXJpU2lnbiwgSW5jLiAtIEZvciBhdXRob3JpemVkIHVzZSBvbmx5MUUwQwYD VQQDEzxWZXJpU2lnbiBDbGFzcyAxIFB1YmxpYyBQcmltYXJ5IENlcnRpZmljYXRpb24gQXV0aG9y aXR5IC0gRzMwHhcNMDkwNTAxMDAwMDAwWhcNMTkwNDMwMjM1OTU5WjCB3TELMAkGA1UEBhMCVVMx FzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMR8wHQYDVQQLExZWZXJpU2lnbiBUcnVzdCBOZXR3b3Jr MTswOQYDVQQLEzJUZXJtcyBvZiB1c2UgYXQgaHR0cHM6Ly93d3cudmVyaXNpZ24uY29tL3JwYSAo YykwOTEeMBwGA1UECxMVUGVyc29uYSBOb3QgVmFsaWRhdGVkMTcwNQYDVQQDEy5WZXJpU2lnbiBD bGFzcyAxIEluZGl2aWR1YWwgU3Vic2NyaWJlciBDQSAtIEczMIIBIjANBgkqhkiG9w0BAQEFAAOC AQ8AMIIBCgKCAQEA7cRH3yooHXwGa7vXITLJbBOP6bGNQU4099oL42r6ZYggCxET6ZvgSU6Lb9UB 0F8NR5GKWkx0Pj/GkQm7TDSejW6hglFi92l2WJYHr54UGAdPWr2f0jGyVBlzRmoZQhHsEnMhjfXc MM3l2VYKMcU2bSkUl70t2olHGYjYSwQ967Y8Zx50ABMN0Ibak2f4MwOuGjxraXj2wCyO4YM/d/mZ //6fUlrCtIcK2GypR8FUKWVDPkrAlh/Brfd3r2yxBF6+wbaULZeQLSfSux7pg2qE9sSyriMGZSal J1grByK0b6ZiSBp38tVQJ5op05b7KPW6JHZi44xZ6/tu1ULEvkHH9QIDAQABo4ICuTCCArUwNAYI KwYBBQUHAQEEKDAmMCQGCCsGAQUFBzABhhhodHRwOi8vb2NzcC52ZXJpc2lnbi5jb20wEgYDVR0T AQH/BAgwBgEB/wIBADBwBgNVHSAEaTBnMGUGC2CGSAGG+EUBBxcBMFYwKAYIKwYBBQUHAgEWHGh0 dHBzOi8vd3d3LnZlcmlzaWduLmNvbS9jcHMwKgYIKwYBBQUHAgIwHhocaHR0cHM6Ly93d3cudmVy aXNpZ24uY29tL3JwYTA0BgNVHR8ELTArMCmgJ6AlhiNodHRwOi8vY3JsLnZlcmlzaWduLmNvbS9w Y2ExLWczLmNybDAOBgNVHQ8BAf8EBAMCAQYwbgYIKwYBBQUHAQwEYjBgoV6gXDBaMFgwVhYJaW1h Z2UvZ2lmMCEwHzAHBgUrDgMCGgQUS2u5KJYGDLvQUjibKaxLB4shBRgwJhYkaHR0cDovL2xvZ28u dmVyaXNpZ24uY29tL3ZzbG9nbzEuZ2lmMC4GA1UdEQQnMCWkIzAhMR8wHQYDVQQDExZQcml2YXRl TGFiZWw0LTIwNDgtMTE4MB0GA1UdDgQWBBR5R2EIQf04BKJL57XM9UP2SSsR+DCB8QYDVR0jBIHp MIHmoYHQpIHNMIHKMQswCQYDVQQGEwJVUzEXMBUGA1UEChMOVmVyaVNpZ24sIEluYy4xHzAdBgNV BAsTFlZlcmlTaWduIFRydXN0IE5ldHdvcmsxOjA4BgNVBAsTMShjKSAxOTk5IFZlcmlTaWduLCBJ bmMuIC0gRm9yIGF1dGhvcml6ZWQgdXNlIG9ubHkxRTBDBgNVBAMTPFZlcmlTaWduIENsYXNzIDEg UHVibGljIFByaW1hcnkgQ2VydGlmaWNhdGlvbiBBdXRob3JpdHkgLSBHM4IRAItbdVaEVIULAM+v OEjOsaQwDQYJKoZIhvcNAQEFBQADggEBADlNz0GZgbWpBbVSOOk5hIls5DSoWufYbAlMJBq6WaSH O3Mh8ZOBz79oY1pn/jWFK6HDXaNKwjoZ3TDWzE3v8dKBl8pUWkO/N4t6jhmND0OojPKvYLMVirOV nDzgnrMnmKQ1chfl/Cpdh9OKDcLRRSr4wPSsKpM61a4ScAjr+zvid+zoK2Q1ds262uDRyxTWcVib vtU+fbbZ6CTFJGZMXZEfdrMXPn8NxiGJL7M3uKH/XLJtSd5lUkL7DojS7Uodv0vj+Mxy+kgOZY5J yNb4mZg7t5Q+MXEGh/psWVMu198r7V9jAKwV7QO4VRaMxmgD5yKocwuxvKDaUljdCg5/wYIxggSL MIIEhwIBATCB8jCB3TELMAkGA1UEBhMCVVMxFzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMR8wHQYD VQQLExZWZXJpU2lnbiBUcnVzdCBOZXR3b3JrMTswOQYDVQQLEzJUZXJtcyBvZiB1c2UgYXQgaHR0 cHM6Ly93d3cudmVyaXNpZ24uY29tL3JwYSAoYykwOTEeMBwGA1UECxMVUGVyc29uYSBOb3QgVmFs aWRhdGVkMTcwNQYDVQQDEy5WZXJpU2lnbiBDbGFzcyAxIEluZGl2aWR1YWwgU3Vic2NyaWJlciBD QSAtIEczAhBXH0kUxtIzZJ6m/S1fRrhZMAkGBSsOAwIaBQCgggJtMBgGCSqGSIb3DQEJAzELBgkq hkiG9w0BBwEwHAYJKoZIhvcNAQkFMQ8XDTEyMDczMDIxMTIwN1owIwYJKoZIhvcNAQkEMRYEFFbb RSHGICba4TR/uxHqiQL2UoaZMIIBAwYJKwYBBAGCNxAEMYH1MIHyMIHdMQswCQYDVQQGEwJVUzEX MBUGA1UEChMOVmVyaVNpZ24sIEluYy4xHzAdBgNVBAsTFlZlcmlTaWduIFRydXN0IE5ldHdvcmsx OzA5BgNVBAsTMlRlcm1zIG9mIHVzZSBhdCBodHRwczovL3d3dy52ZXJpc2lnbi5jb20vcnBhIChj KTA5MR4wHAYDVQQLExVQZXJzb25hIE5vdCBWYWxpZGF0ZWQxNzA1BgNVBAMTLlZlcmlTaWduIENs YXNzIDEgSW5kaXZpZHVhbCBTdWJzY3JpYmVyIENBIC0gRzMCEFcfSRTG0jNknqb9LV9GuFkwggEF BgsqhkiG9w0BCRACCzGB9aCB8jCB3TELMAkGA1UEBhMCVVMxFzAVBgNVBAoTDlZlcmlTaWduLCBJ bmMuMR8wHQYDVQQLExZWZXJpU2lnbiBUcnVzdCBOZXR3b3JrMTswOQYDVQQLEzJUZXJtcyBvZiB1 c2UgYXQgaHR0cHM6Ly93d3cudmVyaXNpZ24uY29tL3JwYSAoYykwOTEeMBwGA1UECxMVUGVyc29u YSBOb3QgVmFsaWRhdGVkMTcwNQYDVQQDEy5WZXJpU2lnbiBDbGFzcyAxIEluZGl2aWR1YWwgU3Vi c2NyaWJlciBDQSAtIEczAhBXH0kUxtIzZJ6m/S1fRrhZMA0GCSqGSIb3DQEBAQUABIIBAFb2P6oC HdWB0cKHMFweCkXK7DuFkuOr8EqbY1zzDHGSXySFtkaTk5dFJcZfzTN9IVoDpF74+rljU2e0N8xL yIaO4LSVVth4TQJW9OYL1Fz6rckVyzNrREUBJkLy8OQKsAQjnikiVlv8DCWGTeEm2s6wFO9ToTMD RPJN06S0hjRZrswUg12yUOTAl+yGfM1XO00tn5h9paaExwbe9v0nQj2ZQUC3bDPwaXPMtMO7y6t2 RygfhMB9p86uRAEvhaFS4Yksf4WhEtHSEdXlf8sUZAnm3YgZKhBlFApv7b2n/NfKQ/JXSrynrssW 02PLOi0TTDNI0lGWPRkKLtEh7PsosTMAAAAAAAA= --Apple-Mail=_D54FFF0E-3581-4C73-9A72-5B9C59430768-- From muraris@microsoft.com Mon Jul 30 15:59:17 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 368DE21F861F for ; Mon, 30 Jul 2012 15:59:17 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -100.466 X-Spam-Level: X-Spam-Status: No, score=-100.466 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1, UNRESOLVED_TEMPLATE=3.132, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8NEvfeL0tdDV for ; Mon, 30 Jul 2012 15:59:16 -0700 (PDT) Received: from co1outboundpool.messaging.microsoft.com (co1ehsobe004.messaging.microsoft.com [216.32.180.187]) by ietfa.amsl.com (Postfix) with ESMTP id CA86421F861B for ; Mon, 30 Jul 2012 15:59:15 -0700 (PDT) Received: from mail219-co1-R.bigfish.com (10.243.78.229) by CO1EHSOBE010.bigfish.com (10.243.66.73) with Microsoft SMTP Server id 14.1.225.23; Mon, 30 Jul 2012 22:59:11 +0000 Received: from mail219-co1 (localhost [127.0.0.1]) by mail219-co1-R.bigfish.com (Postfix) with ESMTP id 2817A4C02EE for ; Mon, 30 Jul 2012 22:59:11 +0000 (UTC) X-Forefront-Antispam-Report: CIP:131.107.125.8; KIP:(null); UIP:(null); IPV:NLI; H:TK5EX14MLTC102.redmond.corp.microsoft.com; RD:none; EFVD:NLI X-SpamScore: -26 X-BigFish: VS-26(zz9371Ic85fhc3f2Mzz1202hzz1033IL8275bh8275dhz2fh2a8h683h839hd25hf0ah107ah) Received-SPF: pass (mail219-co1: domain of microsoft.com designates 131.107.125.8 as permitted sender) client-ip=131.107.125.8; envelope-from=muraris@microsoft.com; helo=TK5EX14MLTC102.redmond.corp.microsoft.com ; icrosoft.com ; X-Forefront-Antispam-Report-Untrusted: CIP:157.56.244.37; KIP:(null); UIP:(null); (null); H:CH1PRD0310HT004.namprd03.prod.outlook.com; R:internal; EFV:INT Received: from mail219-co1 (localhost.localdomain [127.0.0.1]) by mail219-co1 (MessageSwitch) id 1343689148974020_11236; Mon, 30 Jul 2012 22:59:08 +0000 (UTC) Received: from CO1EHSMHS002.bigfish.com (unknown [10.243.78.226]) by mail219-co1.bigfish.com (Postfix) with ESMTP id E194484004C for ; Mon, 30 Jul 2012 22:59:08 +0000 (UTC) Received: from TK5EX14MLTC102.redmond.corp.microsoft.com (131.107.125.8) by CO1EHSMHS002.bigfish.com (10.243.66.12) with Microsoft SMTP Server (TLS) id 14.1.225.23; Mon, 30 Jul 2012 22:59:06 +0000 Received: from tx2outboundpool.messaging.microsoft.com (157.54.51.112) by mail.microsoft.com (157.54.79.180) with Microsoft SMTP Server (TLS) id 14.2.298.5; Mon, 30 Jul 2012 22:58:56 +0000 Received: from mail72-tx2-R.bigfish.com (10.9.14.241) by TX2EHSOBE013.bigfish.com (10.9.40.33) with Microsoft SMTP Server id 14.1.225.23; Mon, 30 Jul 2012 22:58:56 +0000 Received: from mail72-tx2 (localhost [127.0.0.1]) by mail72-tx2-R.bigfish.com (Postfix) with ESMTP id 191F43E0280 for ; Mon, 30 Jul 2012 22:58:56 +0000 (UTC) Received: from mail72-tx2 (localhost.localdomain [127.0.0.1]) by mail72-tx2 (MessageSwitch) id 1343689133981659_16207; Mon, 30 Jul 2012 22:58:53 +0000 (UTC) Received: from TX2EHSMHS020.bigfish.com (unknown [10.9.14.243]) by mail72-tx2.bigfish.com (Postfix) with ESMTP id ECE2C10004B; Mon, 30 Jul 2012 22:58:53 +0000 (UTC) Received: from CH1PRD0310HT004.namprd03.prod.outlook.com (157.56.244.37) by TX2EHSMHS020.bigfish.com (10.9.99.120) with Microsoft SMTP Server (TLS) id 14.1.225.23; Mon, 30 Jul 2012 22:58:53 +0000 Received: from CH1PRD0310MB369.namprd03.prod.outlook.com ([169.254.9.117]) by CH1PRD0310HT004.namprd03.prod.outlook.com ([10.255.137.39]) with mapi id 14.16.0175.005; Mon, 30 Jul 2012 22:58:52 +0000 From: Murari Sridharan To: "Scheffenegger, Richard" , "tcpm (tcpm@ietf.org)" Thread-Topic: Accurate ECN side-meeting Thread-Index: Ac1ukMIzB6G1AClqT3iwdJIUpoFdAgAFdZdQ Date: Mon, 30 Jul 2012 22:58:51 +0000 Message-ID: References: <012C3117EDDB3C4781FD802A8C27DD4F0D4DA610@SACEXCMBX02-PRD.hq.netapp.com> In-Reply-To: <012C3117EDDB3C4781FD802A8C27DD4F0D4DA610@SACEXCMBX02-PRD.hq.netapp.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [131.107.192.92] Content-Type: multipart/alternative; boundary="_000_A6B45266685BCD49876ABB5AF8EE5BF814676FB5CH1PRD0310MB369_" MIME-Version: 1.0 X-OrganizationHeadersPreserved: CH1PRD0310HT004.namprd03.prod.outlook.com X-FOPE-CONNECTOR: Id%0$Dn%*$RO%0$TLS%0$FQDN%$TlsDn% X-FOPE-CONNECTOR: Id%59$Dn%NETAPP.COM$RO%2$TLS%6$FQDN%131.107.125.5$TlsDn% X-FOPE-CONNECTOR: Id%59$Dn%IETF.ORG$RO%2$TLS%6$FQDN%131.107.125.5$TlsDn% X-CrossPremisesHeadersPromoted: TK5EX14MLTC102.redmond.corp.microsoft.com X-CrossPremisesHeadersFiltered: TK5EX14MLTC102.redmond.corp.microsoft.com X-OriginatorOrg: microsoft.com Cc: Yu-Shun Wang Subject: Re: [tcpm] Accurate ECN side-meeting X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2012 22:59:17 -0000 --_000_A6B45266685BCD49876ABB5AF8EE5BF814676FB5CH1PRD0310MB369_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable I think this is an important topic that deserves more discussion. I unfortu= nately couldn't make it to Vancouver. Appreciate if you can forward notes f= rom your side meeting to the list. Personally I am interested as this appli= es to DCTCP and if we have to take it from datacenter to Internet then this= becomes critical to understand and solve. Thanks Murari From: tcpm-bounces@ietf.org [mailto:tcpm-bounces@ietf.org] On Behalf Of Sch= effenegger, Richard Sent: Monday, July 30, 2012 1:33 PM To: tcpm (tcpm@ietf.org) Subject: [tcpm] Accurate ECN side-meeting Hi, Since there was virtually no time for discussion today in the TCPM meeting,= who would be interested in (and have time for) a side meeting to discuss f= urther on the Accurate ECN topic? For me, it became apparent that the problems and requirements statement are= not too well understood so far (and I'm not excluding myself). Currently, we have feedback with perfect Reliability (getting one signal pe= r RTT, with a simple handshake mechanism). Ack loss (or delayed ACKs with more than than 2 data packets) presents anot= her problem,... I think Matt has a very valid point that the solution space needs more disc= ussions, ideally from more participants than only between the authors! Richard Scheffenegger --_000_A6B45266685BCD49876ABB5AF8EE5BF814676FB5CH1PRD0310MB369_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

I think this is an import= ant topic that deserves more discussion. I unfortunately couldn’t mak= e it to Vancouver. Appreciate if you can forward notes from your side meeting to the list. Personally I am interested as this applies to DC= TCP and if we have to take it from datacenter to Internet then this becomes= critical to understand and solve.

 <= /p>

Thanks<= /p>

Murari

 <= /p>

From: tcpm-b= ounces@ietf.org [mailto:tcpm-bounces@ietf.org] On Behalf Of Scheffenegger, Richard
Sent: Monday, July 30, 2012 1:33 PM
To: tcpm (tcpm@ietf.org)
Subject: [tcpm] Accurate ECN side-meeting

 

Hi,

 

Since there was virtually no time for d= iscussion today in the TCPM meeting, who would be interested in (and have t= ime for) a side meeting to discuss further on the Accurate ECN topic?

 

 

For me, it became apparent that the pro= blems and requirements statement are not too well understood so far (and I&= #8217;m not excluding myself).

 

Currently, we have feedback with perfec= t Reliability (getting one signal per RTT, with a simple handshake mechanis= m).

 

Ack loss (or delayed ACKs with more tha= n than 2 data packets) presents another problem,…

 

 

 

I think Matt has a very valid point tha= t the solution space needs more discussions, ideally from more participants= than only between the authors!

 

 

 

Richard Scheffenegger=

 

--_000_A6B45266685BCD49876ABB5AF8EE5BF814676FB5CH1PRD0310MB369_-- From ycheng@google.com Mon Jul 30 18:08:25 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 12B5D11E80F2 for ; Mon, 30 Jul 2012 18:08:25 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.94 X-Spam-Level: X-Spam-Status: No, score=-102.94 tagged_above=-999 required=5 tests=[AWL=0.037, BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id dytO+dwrKswu for ; Mon, 30 Jul 2012 18:08:24 -0700 (PDT) Received: from mail-ob0-f172.google.com (mail-ob0-f172.google.com [209.85.214.172]) by ietfa.amsl.com (Postfix) with ESMTP id 3195B21F859F for ; Mon, 30 Jul 2012 18:08:23 -0700 (PDT) Received: by obbwc20 with SMTP id wc20so11159546obb.31 for ; Mon, 30 Jul 2012 18:08:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type:content-transfer-encoding:x-system-of-record; bh=bRDFIKZ/0X0LisX7jEDCXuVSIe+GIaNcP8Q1gLtqJYw=; b=iiiwPuFvDYgBx9Cx+8ybD9JAxlxbLqOC8CXgRGBr4haXyZ60suRDOkM0+qXuz+uUoR dXLIqvxEg3jOL5gtuYZlKNiFMQoOay3YBDDXvqqIvGk30lJCqheee7qDh4Nk2L1YE8KF G+N3R+zoHCyHqXY6OPK+xsnRzVR4Xmegktz5Qj2+pgsH8+wR902jJ7nXGHQ+xuy2fVse jjq9gXF27+BqLqV/2zeaRI+tQaUMUbx2f3Db47G1zJ9bpJ3pLv55CPTHeY+PFkXBLdS5 G7snXlF3+8tmrSdlT5F9MQm4ZX02pd2z26Vr6K+OFF3EHCwTFUitQtabWBynYEtf89Qp FFNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type:content-transfer-encoding:x-system-of-record :x-gm-message-state; bh=bRDFIKZ/0X0LisX7jEDCXuVSIe+GIaNcP8Q1gLtqJYw=; b=pE1xktuyW4EE+mU0YWJEn4GfKLT9j78CuF+D2VydtrzifLpxoq+DLE3z7SgDYP/f7n 1O00N3kDaKzQDMNGtx5MR9y757jcR6fR7wtIDmDD47z6fXPc3OgEE7Hy/GACKP/ZhjMp 0o6NkdnbRCUwHOJveQSisFUxpbEXWqavvyl+83Mo+C3p8R+If31AC4q9syN/dwS8diOB 34nMJn5gz77iL6ZhBy2aXIo8nUlR4SfWenbbar1lqec19IKvqHHvD4VhS9H79sl2MXEJ Iph3KpI5JV7HowWTVBch9CSesDLXleQchFDbtgQFAKocw06GVHxpB1mPDq/2Tt39SKen hdQg== Received: by 10.182.144.68 with SMTP id sk4mr20710765obb.0.1343696903527; Mon, 30 Jul 2012 18:08:23 -0700 (PDT) Received: by 10.182.144.68 with SMTP id sk4mr20710753obb.0.1343696903356; Mon, 30 Jul 2012 18:08:23 -0700 (PDT) MIME-Version: 1.0 Received: by 10.60.29.33 with HTTP; Mon, 30 Jul 2012 18:08:03 -0700 (PDT) In-Reply-To: References: <012C3117EDDB3C4781FD802A8C27DD4F0D4DA610@SACEXCMBX02-PRD.hq.netapp.com> From: Yuchung Cheng Date: Mon, 30 Jul 2012 18:08:03 -0700 Message-ID: To: Murari Sridharan Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable X-System-Of-Record: true X-Gm-Message-State: ALoCoQk7Cq1E2rPVxdEySP0ILWZhbXU/yfA/VLlXfOylppM5pBUBsq69EWCzAiTDRDDNTCUiYmjIM7nmSfEWZ+zfFD67J7kToycE19ViLV1OSfYbbwpuVOWgWNwY/Hjmg8MTgCi7TXqjg7kdl9I400jNoDJD2TT99RgWjjYvK7mAkluR4JbUgyE= Cc: "tcpm \(tcpm@ietf.org\)" , Yu-Shun Wang Subject: Re: [tcpm] Accurate ECN side-meeting X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2012 01:08:25 -0000 On Mon, Jul 30, 2012 at 3:58 PM, Murari Sridharan w= rote: > I think this is an important topic that deserves more discussion. I > unfortunately couldn=92t make it to Vancouver. Appreciate if you can forw= ard > notes from your side meeting to the list. Personally I am interested as t= his > applies to DCTCP and if we have to take it from datacenter to Internet th= en > this becomes critical to understand and solve. My observation is that the interests to improve ECN is high. Bob likes DCTCP, Matt likes better ECN and encourage Mirja's works. Some dumb questions: 1. is tcpm the right place for that? if not is there a better one? 2. any real data on the "I"internet to justify the current ECN RFC is not good enough and need improvement. NOTE: I want to see data, not theories. 3. will a better ECN motivate rapid deployment. > > > > Thanks > > Murari > > > > From: tcpm-bounces@ietf.org [mailto:tcpm-bounces@ietf.org] On Behalf Of > Scheffenegger, Richard > Sent: Monday, July 30, 2012 1:33 PM > To: tcpm (tcpm@ietf.org) > Subject: [tcpm] Accurate ECN side-meeting > > > > Hi, > > > > Since there was virtually no time for discussion today in the TCPM meetin= g, > who would be interested in (and have time for) a side meeting to discuss > further on the Accurate ECN topic? > > > > > > For me, it became apparent that the problems and requirements statement a= re > not too well understood so far (and I=92m not excluding myself). > > > > Currently, we have feedback with perfect Reliability (getting one signal = per > RTT, with a simple handshake mechanism). > > > > Ack loss (or delayed ACKs with more than than 2 data packets) presents > another problem,=85 > > > > > > > > I think Matt has a very valid point that the solution space needs more > discussions, ideally from more participants than only between the authors= ! > > > > > > > > Richard Scheffenegger > > > > > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm > From wes@mti-systems.com Mon Jul 30 18:37:55 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7A00411E80E4 for ; Mon, 30 Jul 2012 18:37:55 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6uiNyFIgJfyt for ; Mon, 30 Jul 2012 18:37:54 -0700 (PDT) Received: from omr9.networksolutionsemail.com (omr9.networksolutionsemail.com [205.178.146.59]) by ietfa.amsl.com (Postfix) with ESMTP id 6821311E80F5 for ; Mon, 30 Jul 2012 18:37:54 -0700 (PDT) Received: from cm-omr11 (mail.networksolutionsemail.com [205.178.146.50]) by omr9.networksolutionsemail.com (8.13.8/8.13.8) with ESMTP id q6V1brpr016000 for ; Mon, 30 Jul 2012 21:37:53 -0400 Authentication-Results: cm-omr11 smtp.user=wes@mti-systems.com; auth=pass (PLAIN) X-Authenticated-UID: wes@mti-systems.com Received: from [95.211.92.234] ([95.211.92.234:30949] helo=[10.8.0.42]) by cm-omr11 (envelope-from ) (ecelerity 2.2.2.41 r(31179/31189)) with ESMTPA id 42/40-25600-EE637105; Mon, 30 Jul 2012 21:37:53 -0400 Message-ID: <501736E8.6060103@mti-systems.com> Date: Mon, 30 Jul 2012 21:37:44 -0400 From: Wesley Eddy Organization: MTI Systems User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:12.0) Gecko/20120428 Thunderbird/12.0.1 MIME-Version: 1.0 To: Yuchung Cheng References: <012C3117EDDB3C4781FD802A8C27DD4F0D4DA610@SACEXCMBX02-PRD.hq.netapp.com> In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit Cc: Murari Sridharan , "tcpm \(tcpm@ietf.org\)" , Yu-Shun Wang Subject: Re: [tcpm] Accurate ECN side-meeting X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2012 01:37:55 -0000 On 7/30/2012 9:08 PM, Yuchung Cheng wrote: > > Some dumb questions: > 1. is tcpm the right place for that? if not is there a better one? I think it's the right place, with responsible AD hat on :) -- Wes Eddy MTI Systems From rs@netapp.com Mon Jul 30 22:25:34 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4B78D21F8549 for ; Mon, 30 Jul 2012 22:25:34 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.051 X-Spam-Level: X-Spam-Status: No, score=-10.051 tagged_above=-999 required=5 tests=[AWL=0.548, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id eXORw2EbFEDQ for ; Mon, 30 Jul 2012 22:25:33 -0700 (PDT) Received: from mx2.netapp.com (mx2.netapp.com [216.240.18.37]) by ietfa.amsl.com (Postfix) with ESMTP id 15FB821F851B for ; Mon, 30 Jul 2012 22:25:32 -0700 (PDT) X-IronPort-AV: E=Sophos;i="4.77,681,1336374000"; d="scan'208";a="670150538" Received: from smtp2.corp.netapp.com ([10.57.159.114]) by mx2-out.netapp.com with ESMTP; 30 Jul 2012 22:25:32 -0700 Received: from vmwexceht04-prd.hq.netapp.com (vmwexceht04-prd.hq.netapp.com [10.106.77.34]) by smtp2.corp.netapp.com (8.13.1/8.13.1/NTAP-1.6) with ESMTP id q6V5PWkU028244; Mon, 30 Jul 2012 22:25:32 -0700 (PDT) Received: from SACEXCMBX02-PRD.hq.netapp.com ([169.254.1.34]) by vmwexceht04-prd.hq.netapp.com ([10.106.77.34]) with mapi id 14.02.0298.004; Mon, 30 Jul 2012 22:25:30 -0700 From: "Scheffenegger, Richard" To: Yuchung Cheng , Murari Sridharan Thread-Topic: [tcpm] Accurate ECN side-meeting Thread-Index: Ac1ukMIzB6G1AClqT3iwdJIUpoFdAgAFdZdQABNAwYAADO/lkA== Date: Tue, 31 Jul 2012 05:25:30 +0000 Message-ID: <012C3117EDDB3C4781FD802A8C27DD4F0D4DB349@SACEXCMBX02-PRD.hq.netapp.com> References: <012C3117EDDB3C4781FD802A8C27DD4F0D4DA610@SACEXCMBX02-PRD.hq.netapp.com> In-Reply-To: Accept-Language: de-AT, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.104.60.116] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: Yu-Shun Wang , "tcpm \(tcpm@ietf.org\)" Subject: Re: [tcpm] Accurate ECN side-meeting X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2012 05:25:34 -0000 > 2. any real data on the "I"internet to justify the current ECN RFC is not > good enough and need improvement. NOTE: I want to see data, not theories. Well, since the Internet at large doesn't run ECN, what kind of data would = there be? DCTCP which reacts to the extent of congestion in a different way than stan= dard TCP, while not penalizing it with long standing queues, appears to be = quite a solid data point that ECN can be made to work... But current 3168 E= CN is to restrictive for that application... > 3. will a better ECN motivate rapid deployment. ECN on itself probably not; so far, the inventives have not been high enoug= h. However, as Bob pointed out (well, I'm paraphrasing), with the convergen= ce of CoDel and Bufferbloat and DCTCP, it seems there is currently a window= of opportunity to address the open signaling topics that would work best i= n the internet and with these new protocols, not? After all, a new ECN sign= aling scheme (actually a TCP ECN feedback scheme) needs deployment with new= stacks, that run new mechanisms making use of that additional information.= .. Best regards, Richard Scheffenegger From muraris@microsoft.com Tue Jul 31 12:00:10 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 52D3A21F8959 for ; Tue, 31 Jul 2012 12:00:10 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -100.457 X-Spam-Level: X-Spam-Status: No, score=-100.457 tagged_above=-999 required=5 tests=[AWL=0.011, BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1, UNRESOLVED_TEMPLATE=3.132, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lIjxTrN2YJ7B for ; Tue, 31 Jul 2012 12:00:09 -0700 (PDT) Received: from am1outboundpool.messaging.microsoft.com (am1ehsobe006.messaging.microsoft.com [213.199.154.209]) by ietfa.amsl.com (Postfix) with ESMTP id E044E21F894A for ; Tue, 31 Jul 2012 12:00:08 -0700 (PDT) Received: from mail67-am1-R.bigfish.com (10.3.201.228) by AM1EHSOBE010.bigfish.com (10.3.204.30) with Microsoft SMTP Server id 14.1.225.23; Tue, 31 Jul 2012 19:00:07 +0000 Received: from mail67-am1 (localhost [127.0.0.1]) by mail67-am1-R.bigfish.com (Postfix) with ESMTP id 3B3BD4A0356 for ; Tue, 31 Jul 2012 19:00:07 +0000 (UTC) X-Forefront-Antispam-Report: CIP:131.107.125.8; KIP:(null); UIP:(null); IPV:NLI; H:TK5EX14MLTC102.redmond.corp.microsoft.com; RD:none; EFVD:NLI X-SpamScore: -22 X-BigFish: VS-22(zz9371I1432Izz1202hzz1033IL8275dhz2fh2a8h683h839h944hd25hf0ah107ah) Received-SPF: pass (mail67-am1: domain of microsoft.com designates 131.107.125.8 as permitted sender) client-ip=131.107.125.8; envelope-from=muraris@microsoft.com; helo=TK5EX14MLTC102.redmond.corp.microsoft.com ; icrosoft.com ; X-Forefront-Antispam-Report-Untrusted: CIP:157.56.244.37; KIP:(null); UIP:(null); (null); H:CH1PRD0310HT005.namprd03.prod.outlook.com; R:internal; EFV:INT Received: from mail67-am1 (localhost.localdomain [127.0.0.1]) by mail67-am1 (MessageSwitch) id 1343761205420232_31541; Tue, 31 Jul 2012 19:00:05 +0000 (UTC) Received: from AM1EHSMHS001.bigfish.com (unknown [10.3.201.237]) by mail67-am1.bigfish.com (Postfix) with ESMTP id 5A6DF60059 for ; Tue, 31 Jul 2012 19:00:05 +0000 (UTC) Received: from TK5EX14MLTC102.redmond.corp.microsoft.com (131.107.125.8) by AM1EHSMHS001.bigfish.com (10.3.207.101) with Microsoft SMTP Server (TLS) id 14.1.225.23; Tue, 31 Jul 2012 19:00:04 +0000 Received: from db3outboundpool.messaging.microsoft.com (157.54.51.114) by mail.microsoft.com (157.54.79.180) with Microsoft SMTP Server (TLS) id 14.2.298.5; Tue, 31 Jul 2012 19:00:00 +0000 Received: from mail14-db3-R.bigfish.com (10.3.81.227) by DB3EHSOBE001.bigfish.com (10.3.84.21) with Microsoft SMTP Server id 14.1.225.23; Tue, 31 Jul 2012 18:59:37 +0000 Received: from mail14-db3 (localhost [127.0.0.1]) by mail14-db3-R.bigfish.com (Postfix) with ESMTP id 2DD1DA05E6 for ; Tue, 31 Jul 2012 18:59:37 +0000 (UTC) Received: from mail14-db3 (localhost.localdomain [127.0.0.1]) by mail14-db3 (MessageSwitch) id 1343761175477152_1468; Tue, 31 Jul 2012 18:59:35 +0000 (UTC) Received: from DB3EHSMHS013.bigfish.com (unknown [10.3.81.228]) by mail14-db3.bigfish.com (Postfix) with ESMTP id 71B483C0067; Tue, 31 Jul 2012 18:59:35 +0000 (UTC) Received: from CH1PRD0310HT005.namprd03.prod.outlook.com (157.56.244.37) by DB3EHSMHS013.bigfish.com (10.3.87.113) with Microsoft SMTP Server (TLS) id 14.1.225.23; Tue, 31 Jul 2012 18:59:35 +0000 Received: from CH1PRD0310MB369.namprd03.prod.outlook.com ([169.254.9.117]) by CH1PRD0310HT005.namprd03.prod.outlook.com ([10.255.137.40]) with mapi id 14.16.0175.005; Tue, 31 Jul 2012 18:59:30 +0000 From: Murari Sridharan To: Yuchung Cheng Thread-Topic: [tcpm] Accurate ECN side-meeting Thread-Index: Ac1ukMIzB6G1AClqT3iwdJIUpoFdAgAFdZdQAASVqYAAJQP+oA== Date: Tue, 31 Jul 2012 18:59:29 +0000 Message-ID: References: <012C3117EDDB3C4781FD802A8C27DD4F0D4DA610@SACEXCMBX02-PRD.hq.netapp.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [131.107.192.92] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OrganizationHeadersPreserved: CH1PRD0310HT005.namprd03.prod.outlook.com X-FOPE-CONNECTOR: Id%0$Dn%*$RO%0$TLS%0$FQDN%$TlsDn% X-FOPE-CONNECTOR: Id%59$Dn%GOOGLE.COM$RO%2$TLS%6$FQDN%131.107.125.5$TlsDn% X-FOPE-CONNECTOR: Id%59$Dn%NETAPP.COM$RO%2$TLS%6$FQDN%131.107.125.5$TlsDn% X-FOPE-CONNECTOR: Id%59$Dn%IETF.ORG$RO%2$TLS%6$FQDN%131.107.125.5$TlsDn% X-CrossPremisesHeadersPromoted: TK5EX14MLTC102.redmond.corp.microsoft.com X-CrossPremisesHeadersFiltered: TK5EX14MLTC102.redmond.corp.microsoft.com X-OriginatorOrg: microsoft.com Cc: "tcpm \(tcpm@ietf.org\)" , Yu-Shun Wang Subject: Re: [tcpm] Accurate ECN side-meeting X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2012 19:00:10 -0000 Some dumb questions: 1. is tcpm the right place for that? if not is there a better one? [MS] I believe so.=20 2. any real data on the "I"internet to justify the current ECN RFC is not g= ood enough and need improvement. NOTE: I want to see data, not theories. [MS] ECN isn't widely deployed to start with. Secondly the problem here is = quite obvious which is the way the ECN marks are echoed by the receiver and= used by the source. Unless you fix both you cannot hope to get full link u= tilization. You could split this into two problems one is what Richard and = Mirja have taken up which is how do you reflect back accurate ECN feedback = and DCTCP provides one example of how sources can use those marks to averag= e and pick a rate to transmit.=20 3. will a better ECN motivate rapid deployment. [MS] Personally my reasons for lack of ECN deployment are as follows, in no= specific order, a) used to be chicken and egg problem between OS vendors a= nd network operators for the longest time. Then OS vendors did implement fu= lly ECN capability b) then we started finding crappy implementations of ECN= , WS in home gateways that made ECN not be default c) RED is hard to config= ure and get right. d) Standard TCP's use of ECN the point above about 3168 = and way sources were reacting to it casues a loss of link utilization.=20 To really solve the problem of simultaneously catering to throughput sensit= ive and latency sensitive flows we need accurate ECN feedback. I also think= we have a real opportunity to move away from complex AQMs to something muc= h simpler and let the sources (E2E design) handle the RTT smoothing. No mat= ter how you slice and dice it AQMs of today including CoDel are forced to p= ick parameters (even if they are hardcoded) that don't translate across net= works and have to be arbitrarily chosen.=20 Thanks > > > > Thanks > > Murari > > > > From: tcpm-bounces@ietf.org [mailto:tcpm-bounces@ietf.org] On Behalf=20 > Of Scheffenegger, Richard > Sent: Monday, July 30, 2012 1:33 PM > To: tcpm (tcpm@ietf.org) > Subject: [tcpm] Accurate ECN side-meeting > > > > Hi, > > > > Since there was virtually no time for discussion today in the TCPM=20 > meeting, who would be interested in (and have time for) a side meeting=20 > to discuss further on the Accurate ECN topic? > > > > > > For me, it became apparent that the problems and requirements=20 > statement are not too well understood so far (and I'm not excluding mysel= f). > > > > Currently, we have feedback with perfect Reliability (getting one=20 > signal per RTT, with a simple handshake mechanism). > > > > Ack loss (or delayed ACKs with more than than 2 data packets) presents=20 > another problem,... > > > > > > > > I think Matt has a very valid point that the solution space needs more=20 > discussions, ideally from more participants than only between the authors= ! > > > > > > > > Richard Scheffenegger > > > > > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm > From muraris@microsoft.com Tue Jul 31 12:10:47 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D9A1211E8111 for ; Tue, 31 Jul 2012 12:10:47 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -100.458 X-Spam-Level: X-Spam-Status: No, score=-100.458 tagged_above=-999 required=5 tests=[AWL=0.009, BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1, UNRESOLVED_TEMPLATE=3.132, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id yl9B0Ex589LK for ; Tue, 31 Jul 2012 12:10:46 -0700 (PDT) Received: from am1outboundpool.messaging.microsoft.com (am1ehsobe001.messaging.microsoft.com [213.199.154.204]) by ietfa.amsl.com (Postfix) with ESMTP id 2594C11E810E for ; Tue, 31 Jul 2012 12:10:46 -0700 (PDT) Received: from mail50-am1-R.bigfish.com (10.3.201.248) by AM1EHSOBE001.bigfish.com (10.3.204.21) with Microsoft SMTP Server id 14.1.225.23; Tue, 31 Jul 2012 19:10:45 +0000 Received: from mail50-am1 (localhost [127.0.0.1]) by mail50-am1-R.bigfish.com (Postfix) with ESMTP id F40E04E0485 for ; Tue, 31 Jul 2012 19:10:44 +0000 (UTC) X-Forefront-Antispam-Report: CIP:131.107.125.8; KIP:(null); UIP:(null); IPV:NLI; H:TK5EX14MLTC101.redmond.corp.microsoft.com; RD:none; EFVD:NLI X-SpamScore: -27 X-BigFish: VS-27(zz9371I542M1432Izz1202hzz1033IL8275dhz2fh2a8h683h839h944hd25hf0ah107ah) Received-SPF: pass (mail50-am1: domain of microsoft.com designates 131.107.125.8 as permitted sender) client-ip=131.107.125.8; envelope-from=muraris@microsoft.com; helo=TK5EX14MLTC101.redmond.corp.microsoft.com ; icrosoft.com ; X-Forefront-Antispam-Report-Untrusted: CIP:157.56.244.37; KIP:(null); UIP:(null); (null); H:CH1PRD0310HT002.namprd03.prod.outlook.com; R:internal; EFV:INT Received: from mail50-am1 (localhost.localdomain [127.0.0.1]) by mail50-am1 (MessageSwitch) id 1343761843194747_31646; Tue, 31 Jul 2012 19:10:43 +0000 (UTC) Received: from AM1EHSMHS003.bigfish.com (unknown [10.3.201.241]) by mail50-am1.bigfish.com (Postfix) with ESMTP id 240B5360063 for ; Tue, 31 Jul 2012 19:10:43 +0000 (UTC) Received: from TK5EX14MLTC101.redmond.corp.microsoft.com (131.107.125.8) by AM1EHSMHS003.bigfish.com (10.3.207.103) with Microsoft SMTP Server (TLS) id 14.1.225.23; Tue, 31 Jul 2012 19:10:42 +0000 Received: from tx2outboundpool.messaging.microsoft.com (157.54.51.80) by mail.microsoft.com (157.54.79.178) with Microsoft SMTP Server (TLS) id 14.2.298.5; Tue, 31 Jul 2012 19:10:33 +0000 Received: from mail144-tx2-R.bigfish.com (10.9.14.235) by TX2EHSOBE013.bigfish.com (10.9.40.33) with Microsoft SMTP Server id 14.1.225.23; Tue, 31 Jul 2012 19:10:33 +0000 Received: from mail144-tx2 (localhost [127.0.0.1]) by mail144-tx2-R.bigfish.com (Postfix) with ESMTP id 28DE8E00F5 for ; Tue, 31 Jul 2012 19:10:33 +0000 (UTC) Received: from mail144-tx2 (localhost.localdomain [127.0.0.1]) by mail144-tx2 (MessageSwitch) id 1343761830475336_1662; Tue, 31 Jul 2012 19:10:30 +0000 (UTC) Received: from TX2EHSMHS033.bigfish.com (unknown [10.9.14.253]) by mail144-tx2.bigfish.com (Postfix) with ESMTP id 64C1B2400A4; Tue, 31 Jul 2012 19:10:30 +0000 (UTC) Received: from CH1PRD0310HT002.namprd03.prod.outlook.com (157.56.244.37) by TX2EHSMHS033.bigfish.com (10.9.99.133) with Microsoft SMTP Server (TLS) id 14.1.225.23; Tue, 31 Jul 2012 19:10:30 +0000 Received: from CH1PRD0310MB369.namprd03.prod.outlook.com ([169.254.9.117]) by CH1PRD0310HT002.namprd03.prod.outlook.com ([10.255.137.37]) with mapi id 14.16.0175.005; Tue, 31 Jul 2012 19:10:28 +0000 From: Murari Sridharan To: Yuchung Cheng Thread-Topic: [tcpm] Accurate ECN side-meeting Thread-Index: Ac1ukMIzB6G1AClqT3iwdJIUpoFdAgAFdZdQAASVqYAAJQP+oAAAoDTA Date: Tue, 31 Jul 2012 19:10:28 +0000 Message-ID: References: <012C3117EDDB3C4781FD802A8C27DD4F0D4DA610@SACEXCMBX02-PRD.hq.netapp.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [131.107.192.92] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OrganizationHeadersPreserved: CH1PRD0310HT002.namprd03.prod.outlook.com X-FOPE-CONNECTOR: Id%0$Dn%*$RO%0$TLS%0$FQDN%$TlsDn% X-FOPE-CONNECTOR: Id%59$Dn%GOOGLE.COM$RO%2$TLS%6$FQDN%131.107.125.5$TlsDn% X-FOPE-CONNECTOR: Id%59$Dn%NETAPP.COM$RO%2$TLS%6$FQDN%131.107.125.5$TlsDn% X-FOPE-CONNECTOR: Id%59$Dn%IETF.ORG$RO%2$TLS%6$FQDN%131.107.125.5$TlsDn% X-CrossPremisesHeadersPromoted: TK5EX14MLTC101.redmond.corp.microsoft.com X-CrossPremisesHeadersFiltered: TK5EX14MLTC101.redmond.corp.microsoft.com X-OriginatorOrg: microsoft.com Cc: "tcpm \(tcpm@ietf.org\)" , Yu-Shun Wang Subject: Re: [tcpm] Accurate ECN side-meeting X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2012 19:10:48 -0000 In fact I'd go as far as to say "how to get TCP to consume significantly le= ss buffers to achieve the same throughput" is one of the most important pro= blems for TCPM to solve. Without this folks are going to do brute force wor= karounds to try and lower the latency for everybody else on the network. Th= e discussion around congestion control RTCWeb at the recent workshop is a g= reat example. If we don't fix TCP on the Internet everything else (protocol= s, hardware) will have to become more complex (and less resilient) just to = work around this problem.=20 IMHO the starting point for all of this is to get ECN enabled on the Intern= et. It requires the kind of effort and drive that IETF is putting behind IP= v6.=20 Thanks -----Original Message----- From: Murari Sridharan=20 Sent: Tuesday, July 31, 2012 12:00 PM To: 'Yuchung Cheng' Cc: Scheffenegger, Richard; tcpm (tcpm@ietf.org); Yu-Shun Wang Subject: RE: [tcpm] Accurate ECN side-meeting Some dumb questions: 1. is tcpm the right place for that? if not is there a better one? [MS] I believe so.=20 2. any real data on the "I"internet to justify the current ECN RFC is not g= ood enough and need improvement. NOTE: I want to see data, not theories. [MS] ECN isn't widely deployed to start with. Secondly the problem here is = quite obvious which is the way the ECN marks are echoed by the receiver and= used by the source. Unless you fix both you cannot hope to get full link u= tilization. You could split this into two problems one is what Richard and = Mirja have taken up which is how do you reflect back accurate ECN feedback = and DCTCP provides one example of how sources can use those marks to averag= e and pick a rate to transmit.=20 3. will a better ECN motivate rapid deployment. [MS] Personally my reasons for lack of ECN deployment are as follows, in no= specific order, a) used to be chicken and egg problem between OS vendors a= nd network operators for the longest time. Then OS vendors did implement fu= lly ECN capability b) then we started finding crappy implementations of ECN= , WS in home gateways that made ECN not be default c) RED is hard to config= ure and get right. d) Standard TCP's use of ECN the point above about 3168 = and way sources were reacting to it casues a loss of link utilization.=20 To really solve the problem of simultaneously catering to throughput sensit= ive and latency sensitive flows we need accurate ECN feedback. I also think= we have a real opportunity to move away from complex AQMs to something muc= h simpler and let the sources (E2E design) handle the RTT smoothing. No mat= ter how you slice and dice it AQMs of today including CoDel are forced to p= ick parameters (even if they are hardcoded) that don't translate across net= works and have to be arbitrarily chosen.=20 Thanks > > > > Thanks > > Murari > > > > From: tcpm-bounces@ietf.org [mailto:tcpm-bounces@ietf.org] On Behalf=20 > Of Scheffenegger, Richard > Sent: Monday, July 30, 2012 1:33 PM > To: tcpm (tcpm@ietf.org) > Subject: [tcpm] Accurate ECN side-meeting > > > > Hi, > > > > Since there was virtually no time for discussion today in the TCPM=20 > meeting, who would be interested in (and have time for) a side meeting=20 > to discuss further on the Accurate ECN topic? > > > > > > For me, it became apparent that the problems and requirements=20 > statement are not too well understood so far (and I'm not excluding mysel= f). > > > > Currently, we have feedback with perfect Reliability (getting one=20 > signal per RTT, with a simple handshake mechanism). > > > > Ack loss (or delayed ACKs with more than than 2 data packets) presents=20 > another problem,... > > > > > > > > I think Matt has a very valid point that the solution space needs more=20 > discussions, ideally from more participants than only between the authors= ! > > > > > > > > Richard Scheffenegger > > > > > _______________________________________________ > tcpm mailing list > tcpm@ietf.org > https://www.ietf.org/mailman/listinfo/tcpm > From hkchu@google.com Tue Jul 31 22:32:35 2012 Return-Path: X-Original-To: tcpm@ietfa.amsl.com Delivered-To: tcpm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 723CC21F8644 for ; Tue, 31 Jul 2012 22:32:35 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.927 X-Spam-Level: X-Spam-Status: No, score=-102.927 tagged_above=-999 required=5 tests=[AWL=0.050, BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gCTVPL1U2rPO for ; Tue, 31 Jul 2012 22:32:33 -0700 (PDT) Received: from mail-lb0-f172.google.com (mail-lb0-f172.google.com [209.85.217.172]) by ietfa.amsl.com (Postfix) with ESMTP id 043F421F8608 for ; Tue, 31 Jul 2012 22:32:32 -0700 (PDT) Received: by lbbgo11 with SMTP id go11so142222lbb.31 for ; Tue, 31 Jul 2012 22:32:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding:x-system-of-record; bh=oSkbxu/70VhlyslvBGDgNeHbSQ+TcBbPZJz1RicyOgA=; b=DbFmGZSFkHo+lnAsn8VB/M+HJDPsDKzo3Wu3zyQSsylvGAeZwB9QPGNykqAPBc8PRJ /q1kp7Q52Krq1X9WUk+sXfEovolBA2rcTf/r2m+piN6poGVFhGPIbEpTl66OUsb6Jclz fRC/aLmI3heUgWgs7/8iEZVTBsbWM3qNJCapv/tGULpqDs3x6GURlqMBqpO8rSX49Uob NAatA3sBd0pkRaFQA9QWOdMkl+rsO9bnyj+0OinIZmDgTjTe5LpiX0fbhmYcwYjVY8DA VhqgTFyJPCAK83SnBrEZ/X93sS9YYvkPHvuy2Q6BFaHWylJmlk6oXB2iVPB3bPiYbdZL CrIw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding:x-system-of-record :x-gm-message-state; bh=oSkbxu/70VhlyslvBGDgNeHbSQ+TcBbPZJz1RicyOgA=; b=QghQbK2G9TPjV70JZVKvmWG+uvGA97DrUSUiakYa5KG8WxvLQ9Iuo4vsFIvYyhmYb8 zPGTDL8bCoDZbd6bN5k5asKz4a3S1BM5ALfcwBtD47W/WIa0kLci06NqB8jTTNSaPJqh UomeH832Rc3cRHNaL+0wAmmFDdJL65JKfOt98zxtyac8CmaQOrD5K4aeTyeNCtzVSIXd zZEGI1SYK+rjeuLocG5c7PpwaJREp6C0EcpXG9i+AidT2ttVolj1CfsEL0yVzrS8pgq7 dBSVvs983T+Ku9Oz7pTjwNTyVhKBIEDtAG71/NqcXGZJILLgcPARnRxwsmhaoVxuExGw PRKQ== Received: by 10.152.148.1 with SMTP id to1mr16886311lab.34.1343799151862; Tue, 31 Jul 2012 22:32:31 -0700 (PDT) MIME-Version: 1.0 Received: by 10.152.148.1 with SMTP id to1mr16886306lab.34.1343799151713; Tue, 31 Jul 2012 22:32:31 -0700 (PDT) Received: by 10.152.6.1 with HTTP; Tue, 31 Jul 2012 22:32:31 -0700 (PDT) In-Reply-To: References: <201207251811.40438.mkuehle@ikr.uni-stuttgart.de> <201207252012.51426.mirja.kuehlewind@ikr.uni-stuttgart.de> Date: Tue, 31 Jul 2012 22:32:31 -0700 Message-ID: From: Jerry Chu To: Mirja Kuehlewind Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-System-Of-Record: true X-Gm-Message-State: ALoCoQn8O1cewoLn+ZNITQCwfgdJMWVBXgT4mXAXq6AzQv1OnGrYxUIhwtQkm+/CZSHHl40MSIvElAC2kf0NKrRz7k4AywTzWXZ5GU9nJcJdao3Wxtfa5vF8m9dPuntRb77X6GWhwJ+jU21bi181GoQlXikF7MAigfhqQ34UpW8Xh02pw3u1u4c= Cc: tcpm@ietf.org Subject: Re: [tcpm] Review of draft-ietf-tcpm-initcwnd-04 X-BeenThere: tcpm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: TCP Maintenance and Minor Extensions Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Aug 2012 05:32:35 -0000 On Wed, Jul 25, 2012 at 2:58 PM, Jerry Chu wrote: > On Wed, Jul 25, 2012 at 11:12 AM, Mirja Kuehlewind > wrote: >> Hi Jerry, >> >> just one quick comment: >> >>> Moreover, the TOC provides a very clear index of the contents so a >>> savvy reader can >>> quickly pick the sections of interesting. E.g., if you only want to >>> focus on the change >>> you'll know to read section 1 and 2 only. That's it. >> And that's not really true (anymore). There are several SHOULD and MUST = in the >> other sections as well. Thus you have to read all the doc to correctly >> implement the changes. I know that the intent was to have everything in >> section 1 and 2 that is of interest for implementers but due to the many >> changes that have been done in previous versions, it's not the case anym= ore. > > Actually the only other specification related languages is section 9, > and it's not > essential as it simply reiterates what is described in rfc6298. > > Yes other SHOULD and MUST can be found in the newly added section 12 > "Usage and Deployment Recommendations", but that section is mainly about > what one must do in any future large scale experiment, closely monitoring= a > number of key metrics... and not really about protocol specification. > > So to summarize, this draft is not just about protocol specification, > it also addresses > many other questions like why, and how to go about doing more > experiments safely. > >> That why I said, all the content is there, would be nice to >> clean-up/restructure a little and then it's done. >> >> It's good to have a similar structure than RFC3390 (didn't realize that = you >> tried to have exactly the same sections here) but if the content would f= it > > It was spelled out up front in 1. Introduction > > "This document proposes to raise the upper bound on TCP's initial > window (IW) to 10 segments or roughly 15KB. It is patterned after and > borrows heavily from RFC 3390 [RFC3390] and earlier work in this > area." > >> better with a different structure, I'd prefer to have a good readable do= c >> over having a doc that looks similar to RFC3390. > > Let's stick to the current format but I will respond to your one other > criticism on > section 12 first. The section is quite specific about the metrics to moni= tor: "A > key metric to monitor is the rate of packet losses, ECN marking, or > segment retransmissions during the initial burst." But as others have > pointed out it can be > quite difficult to try to get more specific than that, i.e., how much > performance > deteriorates before one must back off. As such all we can do for now is t= o have > enough warning signs for now but leave some details for future investigat= ion. > > Jerry > >> >> Mirja >> >> >>> >>> >Hence I'd like to >>> > propose quite a little restructuring before moving the document forwa= rd >>> > as it is hard to easily catch the main points at the moment. More >>> > concrete, I'd move at least section 4, 10 and 11, maybe also 5, 6 and= 7 >>> > (or at least partly) to the appendix. The first two paragraph of sect= ion >>> > 7 might actually belong in >>> >>> Yes it's doable but I think the current flow is fine if you realize >>> the main purpose of >>> the document is to justify the change, not the change itself. Also TOC >>> will make it >>> much easier to navigate through. >>> >>> > the security consideration section. And section 15 (conclusion) is no= t >>> > needed at all. More detailed comments below. >>> > >>> > Mirja >>> > >>> > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >>> > Section 1 Introduction: >>> > ---------------------------- >>> > - s/roughly 15KB/maxium 14600B/ >>> >>> Ok. >>> >>> > - 2. paragraph: Don't like this two sentences at all. I hope the reas= on >>> > to increase IW is not just the fact there the queues are large enough= to >>> > store 10 packets. You basically saying "Let's send more packets at on= ce >>> > to blow the queues!". I'd say the reason is that flow sizes have >>> > increased and therefore IW10 would improve latency a lot. Only becaus= e >>> > queues are likely to be large than 10 packets today, it is possible a= t >>> > all to send more than 10 packet at once. Btw. even if the queue would= be >>> > small, pacing could help to still allow an IW of 10. Moreover, the fa= ct >>> > that there might be several concurrent flows, does not mean that the >>> > buffer has to be larger (only if those flow start at the same time wi= th >>> > IW10). And even if there is a large buffer, if there is a long-lived = TCP >>> > connection, this flow will fill up the queue and there might be by ch= ance >>> > still no space in the queue for 10 additional packets. >>> >>> Guess you read the paragraph differently from what the authors had in m= ind. >>> What we have in mind was "Ten segments are likely to fit into queue sp= ace" >>> because access link drains much faster these days (due to b/w growth)..= . >>> >>> Will try to make it more clear. >>> >>> Will continue later, >>> >>> Jerry >>> >>> > - 3. paragraph: Don't agree. You might not know that there is a low s= peed >>> > link somewhere on the path. There are cases where the endpoint can >>> > protect its path by setting the receiver window but that's not the >>> > general case. This is exactly the point - it's not the general case but may be the vast majority. I.e., 95% of severely b/w limited hosts may be just 1 hop away or even dire= ctly attached to a slow link (e.g., dailup modem) hence can be mitigated much ea= sily. No one suggested a solution here to solve the general problem of detecting bottleneck b/w. That's the job TCP's CC algorithm. >>> > >>> > - 5. paragraph: "We recommend that all TCP implementations have a >>> > settable TCP IW parameter..." >>> > -> I'd like to see this not only in the introduction but also in th= e >>> > main section (2.) that is describing the changes and maybe even as a >>> > SHOULD >>> > >>> > >>> > Section 2 TCP Modifications >>> > ------------- >>> > - Maybe have a more specific title like "Setting the IW" and some >>> > subsections Plagiarized from RFC3390 :) >>> > >>> > - equation 1: say somewhere that numbers are in KByte Not really. >>> > >>> > - 4. paragraph ("Note that...") does not belong in this section Why not? (It's more or less a disclaim because no one has done any study on non-Ethernet MTU. >>> > >>> > - maybe new subsection for paragraph 5 "Resetting the IW after SYN lo= ss"; >>> > what are "multiple SYN or SYN/ACK retransmissions" -> maybe say a >>> > number Ok. Changed to "more than one". >>> > >>> > - last paragraph: maybe s/implementations SHOULD fall >>> > back/implementations MUST fall back/ ? >>> > >>> > >>> > Section 3 Implementation Issues >>> > ------------- >>> > - Maybe move the 3 paragraph into section 2 such that the setting of = the >>> > receive window is a MUST or SHOULD of the changes to TCP Non-essential. Remember even IW10 is just an upperbound. >>> > >>> > - 4. paragraph: Please give some explanation (or remove) This was brought up and discussed on the list long ago. >>> > >>> > >>> > Section 4 Background >>> > ------------ >>> > - Move most parts in the appendix >>> > >>> > - Move 2. paragraph into Introduction >>> > >>> > - Maybe improve structure with subsections e.g. paragraph 5 and 6 >>> > as "Recommendation for HTTP Traffic" or something See my other response. I prefer not to change the structure at this time. >>> > >>> > - 7. paragraph: "pacing will not be necessary" >>> > -> I can't remember having seen any results on this. Please explain >>> > further or remove! The statement was really "We suspect" pacing will not be necessary because our test results do not show any severe congestion from IW10... >>> > >>> > Section 5 Advantages of a Larger Initial Window >>> > ----------- >>> > - section 5.1 Reducing Latency >>> > You should also mention that for larger transmission with several = 100s >>> > of RTT a save of 4 RTT does not really help. Thus transmissions which >>> > will finish within a small number of RTT will benefit the most. Isn't it obvious already? >>> > >>> > - section 5.2 >>> > You make some quite general statements here and then u only refer t= o >>> > google data. Maybe you can find some other references as well. It's not just google data (see all the refs.) >>> > >>> > Section 6 >>> > ---------- >>> > - paragraph 3 and 4 belong in an evaluation section/appendix (maybe o= wn >>> > subsection on "Influence of simultaneous opened connections") >>> > >>> > Section 7 >>> > -------- >>> > - Move 1. paragraph to security consideration >>> > >>> > - Move rest in appendix or even parts in the introduction >>> > >>> > Section 9 >>> > ---------- >>> > - move to main section which is introducing the TCP changes as own >>> > subsection as a MUST is in here Non-essential because it simply restates what is said in rfc6298. It was ad= ded to address some concern (or confusion) that was brought up in the past that the initial burst of 10 pkts won't bid well for RTO hence causing spurious rexmits. >>> > >>> > - can you explain better why this is need or what would happen otherw= ise? >>> > >>> > Section 10 >>> > ---------- >>> > - Move to appendix, have more subsections and better titles for the >>> > subsections e.g. "Improvements in Latency", "Impact on the Retransmis= sion >>> > Rate" and "Multiple TCP Connections" >>> > >>> > - The numbers for the latency improvement are only on (google) web >>> > search. Of course there are quite large improvements possible but the >>> > Internet contains also other traffic. Not sure if your number are rea= lly >>> > that significant...? We just stated what we have. >>> > >>> > Section 11 >>> > --------- >>> > - move to appendix >>> > >>> > - It's nice that you list all the studies. Could you please quickly >>> > summarize their results/findings, too? Because I guess that the >>> > intersting part to know. If you are interested please go read them yourself. >>> > >>> > Section 12 >>> > -------- >>> > - Its good to have this section but it's quite unspecific. E.g. you s= ay >>> > "An increased initial window MUST NOT be turned on by default on syst= ems >>> > without such monitoring capabilities." But what exactly should be >>> > monitored and how frequently and so on. Also "any significant >>> > deterioration" doesn't say anything to me. Is it possible to have a >>> > concrete approach/algorithm here what to monitor when and what to do = in >>> > which cases? And than have this even as part of the main section with= TCP >>> > changes as this section has also some SHOULD and MUST and might be >>> > overread otherwise. >>> > >>> > Section 14 >>> > --------- >>> > - "highly unlikely to lead to a persistent state of network congestio= n" >>> > -> Even a "highly unlikely" is concerning me in this case! Also there= is >>> > no reasoning for this. But I guess you can even say, that this change >>> > cannot cause persistent network congestion as congestion control is s= till >>> > used.... See 1st paragraph of section 7. >>> > >>> > Section 15 Conclusion >>> > -------- >>> > - Is not needed here at all, as no new information Most rfcs have a conclusion section for those who have time only for the 1st and last sections. >>> > >>> > Appendix >>> > --------- >>> > - "One notable exception is YouTube because we don't think >>> > the large initial window will have much material impact, either >>> > positive or negative, on bulk data services." >>> > -> Why is this document than not recommending to NOT use IW10 for b= ulk >>> > data services? Why should it recommend NOT? How would the TCP stack know apriori bulk or not? So it will require input from the apps... Is it really worth the complexity? >>> > >>> > - "[CW10] contains some result from a testbed study on how short flow= s >>> > with a larger initial window might affect the throughput >>> > performance of other co-existing, long lived, bulk data transfer= s." >>> > -> Please also describe what the results are! Read the slides please. >>> > >>> > - paragraph on "Why 10 segment?" >>> > -> This is something I would prefer to have in the body of the docu= ment >>> > not in the appendix >>> > >>> > - "Some simulation studies [JNDK10, JNDK10-1] have been conducted to >>> > study the effect of a larger initial window on wireless links fr= om >>> > 2G to 4G networks (EGDE/HSPA/LTE). The overall result seems mixe= d >>> > in both raw performance and the fairness index." >>> > -> Should it be recommend to not use IW10 in wireless scenarios (if >>> > known)? Studies are still on-going. I don't remember there has been a conclusive re= sult. Jerry >> >> >> >> -- >> ------------------------------------------------------------------- >> Dipl.-Ing. Mirja K=FChlewind >> Institute of Communication Networks and Computer Engineering (IKR) >> University of Stuttgart, Germany >> Pfaffenwaldring 47, D-70569 Stuttgart >> >> tel: +49(0)711/685-67973 >> email: mirja.kuehlewind@ikr.uni-stuttgart.de >> web: www.ikr.uni-stuttgart.de >> -------------------------------------------------------------------